2022-06-29T15:33:01.450 INFO:root:teuthology version: 1.1.1.dev379+gf58709e3 2022-06-29T15:33:01.452 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --description fs/multiclient/{begin clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests} --archive /home/teuthworker/archive/yuriw-2022-06-29_13:31:31-fs-wip-yuri5-testing-2022-06-27-0923-octopus-distro-default-smithi/6905516 --name yuriw-2022-06-29_13:31:31-fs-wip-yuri5-testing-2022-06-27-0923-octopus-distro-default-smithi --owner scheduled_yuriw@teuthology -- /home/teuthworker/archive/yuriw-2022-06-29_13:31:31-fs-wip-yuri5-testing-2022-06-27-0923-octopus-distro-default-smithi/6905516/orig.config.yaml 2022-06-29T15:33:01.505 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2022-06-29T15:33:01.575 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2022-06-29_13:31:31-fs-wip-yuri5-testing-2022-06-27-0923-octopus-distro-default-smithi/6905516 branch: wip-yuri5-testing-2022-06-27-0923-octopus description: fs/multiclient/{begin clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests} email: null first_in_suite: false job_id: '6905516' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2022-06-29_13:31:31-fs-wip-yuri5-testing-2022-06-27-0923-octopus-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 30 - machine: disk: 200 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: wip-yuri5-testing-2022-06-27-0923-octopus ceph: cephfs_ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 15m rados osd op timeout: 15m mds: debug mds: 20 debug ms: 1 mds bal frag: true mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 15m rados osd op timeout: 15m mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - evicting unresponsive client - POOL_APP_NOT_ENABLED - has not responded to cap revoke by MDS for over - MDS_CLIENT_LATE_RELEASE - responding to mclientcaps sha1: f25ffac72e5a9e63860ff7deb6767a11f0559a16 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: flavor: default sha1: f25ffac72e5a9e63860ff7deb6767a11f0559a16 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri5-testing-2022-06-27-0923-octopus sha1: f25ffac72e5a9e63860ff7deb6767a11f0559a16 owner: scheduled_yuriw@teuthology priority: 79 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - - mon.b - mon.c - mgr.x - mds.b - osd.4 - osd.5 - osd.6 - osd.7 - - client.0 - - client.1 sha1: f25ffac72e5a9e63860ff7deb6767a11f0559a16 sleep_before_teardown: 0 suite: fs suite_branch: wip-yuri5-testing-2022-06-27-0923-octopus suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_f25ffac72e5a9e63860ff7deb6767a11f0559a16/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: f25ffac72e5a9e63860ff7deb6767a11f0559a16 targets: smithi047.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4e6XGqkwGvpDrF5tk+4DCUrEpdP3CkMgMrSSS9ef5+Qelxe2P8ud2lIOIDMsxRgrE5O5QC/wbcUiwn8+qxPo3wnqMRFoTukU4ntoB9u0OukaxiytT46IxAjcQxYqYivrRCl4hHvmed4EBBaJIkW/raEv6tYaYTy2edkofooheORk6Hm9cmMDLCh2ajAbHIY/81h9pV7Hu43cLS/MiWPZLhDSW5oCu8cPXA4gAt6Yr3f4ri90mHCrOpOjq1/GrmoOY8f88WVs+PbPBu+qLLYzysc/6ywX8VIXBoo8ZkrGXmSIjwW5Zu1JZW1Bj3Ppk/bNbZRWL8LAWcEOqqlRQZJ/i6uZRgXX3Q5V2XPDRBxReKRKIW3C44Q2MdQWjkEe96bCJefVfxu9+0FkSH/E81B6EelO8+s3Ge1FRtFkMMPxwxjSWbnXbcPSz75RMs3+0Um98kiDO3+tkkBkkL5QiosHaFYHgDY2diV5ao84ECNE5pD8mWIHiMyYFKN23PiSNvt8= smithi064.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDdkS0ro2f1U277A5L7yO3qFO5OBFhLAUb8ADG3fUuoJHbGGpBlumoJOhZsDW/AK8e18jCbJrJIkBVZnMiLklYmoOXQ4XDbmPllwAVojFEXDAwzpVfAmIf2pVnEAJfTZmMXTpP8tDSGRY4X4ZM9LEOrvdBmsRP+ACrBfii9LEXpFix2NStpaSWKqcMYkCbVpxOIg5BqrtDlMf1fe9j/N9tWl22uDnuzpYKS6pwdwbxWBWWT6Qh0s5Dsd1swGRaRqgZkiEFowJAKMDVR1DHHc7VCr7b+19YPsGljki+x4Uxi4eey62Yj1OtPn2x2mY9bwkWxMZLi+lzEajyWOpj5QP7Adw/egmEtXD/uAPGL2atq05qLogwaoxlz112YVoyJ/UjQulnsKnCnrcX8wtzJq2x40MprbfyORKysYXyFhqgXQKUlnYM3lU5H9g99Af8Oyzq9vl6uKTpjWSkr79J/m3CKmI3Kg8n2CcgNFKZI91nFCfP25OfVbruX5e+txVlZnA8= smithi099.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZ3lGbsmV5Dubh02wgTX/9Gf/jBDlc9lpQ/t9F8VdBmXsK915xExtQc4o8oAdSd0gN+vcaX4xveET1oQLs0zN4dW64kOMM6qiEGOzy4aisfPmQgHh2w7YJJ5i6RqMR8j7So5CHCDN2NvDLNilTxkpx1qTW6BkgtiNwH3O4UwNPxaqBKTm8VNjHzgPiMQdkBmR0sMLS9aEfi9G9wdeH5oFr7QOPtdu84fUm6idqpTF20r1Q878w78Z8iYz73GltKHx+lcB6mzoG08ZqhYJpeBs/mvSJzdLUTxoCp3w1aFuev6en61UthxLBkt5Vz8ZyO3g8ImP8jiYtGuTNzkgTPNguxzIrg6tW/PHJElnYEvFD+dM1xr5xFDHsofqH7KRILl4gJm5oEIViPpj/ocEmRakJ1hFRNF4myz83SxnsX1Bqe8BIgYgLABQnvL9zwoAkoIir3V/jnqXwiJZS1qbuY2gfDt9W8JAZd5Tj68MfHjoVrr0iv/DMAIL31ozQqU3iwDc= smithi164.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCjyDdZDCWeXC2ZkZ0RXUp0FFOWbaZFutsSv/nsLHjcLzc6p2L6UAsSnJBpzL6L490K/NRMpS82MM67GIcQIFGWf9lxvQWMNpjzpRws8/C2d41stWRXtlGdYkd5ai/gqbN7Cs+YOn8A9QEW+gWvr6oxL+c3AwqTucFCjX/wjLS5Do+qFRx6r+h8I8adX7gb7CJfl0I+pkEozjrDkUavpONv+PJ9vX5oBGqMVK97/I3TbVimPLDj7Gingn9dQRdZdFZtKqjlk44Nlp2PScVZbDv/IHW9xigYq4nlxUVu47pbSkB24Ob+oj4bgAvX8FCZshWyPqf8N/4vrQiTwYW/r24PcS20iEWL4IjnLSvqR5eEAMfYzh3QIrFge2bWYOwhZnSWCFhzCaFyZsDiFMG0kXMlMC9mxVDH4eHc61iOxrul7i8wbv7f4tVBrs2GsYEnEekg+aqhCT6B8dOrlQmPpk8G0tL8Pgs3OsFZ51M+Ia3DGUs2XWjVz5PYlz3QrjNE/DU= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell rpm: - python3-cephfs extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: null - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_misc teuthology_branch: main teuthology_sha1: f58709e3fda190343157063e5edb0d7a36a41afc timestamp: 2022-06-29_13:31:31 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.1713 2022-06-29T15:33:01.576 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_f25ffac72e5a9e63860ff7deb6767a11f0559a16/qa; will attempt to use it 2022-06-29T15:33:01.576 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_f25ffac72e5a9e63860ff7deb6767a11f0559a16/qa/tasks 2022-06-29T15:33:01.577 INFO:teuthology.run_tasks:Running task internal.check_packages... 2022-06-29T15:33:01.579 INFO:teuthology.task.internal:Checking packages... 2022-06-29T15:33:01.600 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'f25ffac72e5a9e63860ff7deb6767a11f0559a16' 2022-06-29T15:33:01.601 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2022-06-29T15:33:01.601 INFO:teuthology.packaging:ref: None 2022-06-29T15:33:01.601 INFO:teuthology.packaging:tag: None 2022-06-29T15:33:01.602 INFO:teuthology.packaging:branch: wip-yuri5-testing-2022-06-27-0923-octopus 2022-06-29T15:33:01.602 INFO:teuthology.packaging:sha1: f25ffac72e5a9e63860ff7deb6767a11f0559a16 2022-06-29T15:33:01.603 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=wip-yuri5-testing-2022-06-27-0923-octopus 2022-06-29T15:33:01.923 INFO:teuthology.task.internal:Found packages for ceph version 15.2.16-304-gf25ffac7-1focal 2022-06-29T15:33:01.924 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2022-06-29T15:33:01.931 INFO:teuthology.task.internal:no buildpackages task found 2022-06-29T15:33:01.931 INFO:teuthology.run_tasks:Running task internal.save_config... 2022-06-29T15:33:01.972 INFO:teuthology.task.internal:Saving configuration 2022-06-29T15:33:01.989 INFO:teuthology.run_tasks:Running task internal.check_lock... 2022-06-29T15:33:01.996 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-06-29T15:33:02.020 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi047.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2022-06-29_13:31:31-fs-wip-yuri5-testing-2022-06-27-0923-octopus-distro-default-smithi/6905516', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2022-06-29 15:27:32.035598', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIEQtFb1tcGeQs4oysc28CrB/pYBFfSvW1FxxB/iDT53C'} 2022-06-29T15:33:02.042 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi064.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2022-06-29_13:31:31-fs-wip-yuri5-testing-2022-06-27-0923-octopus-distro-default-smithi/6905516', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2022-06-29 15:27:32.034554', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHN79gCkZc4kPI3N41nJfIBR+OGEAD2HLMZ/fifj6cfP'} 2022-06-29T15:33:02.066 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi099.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2022-06-29_13:31:31-fs-wip-yuri5-testing-2022-06-27-0923-octopus-distro-default-smithi/6905516', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2022-06-29 15:27:32.036726', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAILzapFVZtJfvuaEJIjxd2lYDutTGMsu3u8yIUHnUbJZo'} 2022-06-29T15:33:02.090 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi164.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2022-06-29_13:31:31-fs-wip-yuri5-testing-2022-06-27-0923-octopus-distro-default-smithi/6905516', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2022-06-29 15:27:32.033208', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAINdu1TMx+gTVdorRMQy/Ed353BlGMkD1dFYa33C5s2NX'} 2022-06-29T15:33:02.091 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2022-06-29T15:33:02.098 INFO:teuthology.task.internal:roles: ubuntu@smithi047.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3'] 2022-06-29T15:33:02.098 INFO:teuthology.task.internal:roles: ubuntu@smithi064.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2022-06-29T15:33:02.098 INFO:teuthology.task.internal:roles: ubuntu@smithi099.front.sepia.ceph.com - ['client.0'] 2022-06-29T15:33:02.099 INFO:teuthology.task.internal:roles: ubuntu@smithi164.front.sepia.ceph.com - ['client.1'] 2022-06-29T15:33:02.099 INFO:teuthology.run_tasks:Running task console_log... 2022-06-29T15:33:02.267 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f368dcd68c8>, signals=[15]) 2022-06-29T15:33:02.269 INFO:teuthology.run_tasks:Running task internal.connect... 2022-06-29T15:33:02.278 INFO:teuthology.task.internal:Opening connections... 2022-06-29T15:33:02.279 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi047.front.sepia.ceph.com 2022-06-29T15:33:02.281 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-06-29T15:33:02.384 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi064.front.sepia.ceph.com 2022-06-29T15:33:02.386 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-06-29T15:33:02.487 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi099.front.sepia.ceph.com 2022-06-29T15:33:02.488 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-06-29T15:33:02.582 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi164.front.sepia.ceph.com 2022-06-29T15:33:02.583 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-06-29T15:33:02.679 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2022-06-29T15:33:02.686 DEBUG:teuthology.orchestra.run.smithi047:> uname -m 2022-06-29T15:33:02.693 INFO:teuthology.orchestra.run.smithi047.stdout:x86_64 2022-06-29T15:33:02.694 DEBUG:teuthology.orchestra.run.smithi047:> cat /etc/os-release 2022-06-29T15:33:02.700 INFO:teuthology.orchestra.run.smithi047.stdout:NAME="Ubuntu" 2022-06-29T15:33:02.700 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2022-06-29T15:33:02.701 INFO:teuthology.orchestra.run.smithi047.stdout:ID=ubuntu 2022-06-29T15:33:02.701 INFO:teuthology.orchestra.run.smithi047.stdout:ID_LIKE=debian 2022-06-29T15:33:02.701 INFO:teuthology.orchestra.run.smithi047.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2022-06-29T15:33:02.701 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION_ID="20.04" 2022-06-29T15:33:02.702 INFO:teuthology.orchestra.run.smithi047.stdout:HOME_URL="https://www.ubuntu.com/" 2022-06-29T15:33:02.702 INFO:teuthology.orchestra.run.smithi047.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-06-29T15:33:02.702 INFO:teuthology.orchestra.run.smithi047.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-06-29T15:33:02.702 INFO:teuthology.orchestra.run.smithi047.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-06-29T15:33:02.703 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION_CODENAME=focal 2022-06-29T15:33:02.703 INFO:teuthology.orchestra.run.smithi047.stdout:UBUNTU_CODENAME=focal 2022-06-29T15:33:02.704 INFO:teuthology.lock.ops:Updating smithi047.front.sepia.ceph.com on lock server 2022-06-29T15:33:02.724 DEBUG:teuthology.orchestra.run.smithi064:> uname -m 2022-06-29T15:33:02.731 INFO:teuthology.orchestra.run.smithi064.stdout:x86_64 2022-06-29T15:33:02.731 DEBUG:teuthology.orchestra.run.smithi064:> cat /etc/os-release 2022-06-29T15:33:02.777 INFO:teuthology.orchestra.run.smithi064.stdout:NAME="Ubuntu" 2022-06-29T15:33:02.778 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2022-06-29T15:33:02.778 INFO:teuthology.orchestra.run.smithi064.stdout:ID=ubuntu 2022-06-29T15:33:02.778 INFO:teuthology.orchestra.run.smithi064.stdout:ID_LIKE=debian 2022-06-29T15:33:02.778 INFO:teuthology.orchestra.run.smithi064.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2022-06-29T15:33:02.778 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION_ID="20.04" 2022-06-29T15:33:02.779 INFO:teuthology.orchestra.run.smithi064.stdout:HOME_URL="https://www.ubuntu.com/" 2022-06-29T15:33:02.779 INFO:teuthology.orchestra.run.smithi064.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-06-29T15:33:02.779 INFO:teuthology.orchestra.run.smithi064.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-06-29T15:33:02.779 INFO:teuthology.orchestra.run.smithi064.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-06-29T15:33:02.780 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION_CODENAME=focal 2022-06-29T15:33:02.780 INFO:teuthology.orchestra.run.smithi064.stdout:UBUNTU_CODENAME=focal 2022-06-29T15:33:02.780 INFO:teuthology.lock.ops:Updating smithi064.front.sepia.ceph.com on lock server 2022-06-29T15:33:02.797 DEBUG:teuthology.orchestra.run.smithi099:> uname -m 2022-06-29T15:33:02.805 INFO:teuthology.orchestra.run.smithi099.stdout:x86_64 2022-06-29T15:33:02.805 DEBUG:teuthology.orchestra.run.smithi099:> cat /etc/os-release 2022-06-29T15:33:02.854 INFO:teuthology.orchestra.run.smithi099.stdout:NAME="Ubuntu" 2022-06-29T15:33:02.855 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2022-06-29T15:33:02.855 INFO:teuthology.orchestra.run.smithi099.stdout:ID=ubuntu 2022-06-29T15:33:02.855 INFO:teuthology.orchestra.run.smithi099.stdout:ID_LIKE=debian 2022-06-29T15:33:02.855 INFO:teuthology.orchestra.run.smithi099.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2022-06-29T15:33:02.856 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION_ID="20.04" 2022-06-29T15:33:02.856 INFO:teuthology.orchestra.run.smithi099.stdout:HOME_URL="https://www.ubuntu.com/" 2022-06-29T15:33:02.856 INFO:teuthology.orchestra.run.smithi099.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-06-29T15:33:02.856 INFO:teuthology.orchestra.run.smithi099.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-06-29T15:33:02.856 INFO:teuthology.orchestra.run.smithi099.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-06-29T15:33:02.857 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION_CODENAME=focal 2022-06-29T15:33:02.857 INFO:teuthology.orchestra.run.smithi099.stdout:UBUNTU_CODENAME=focal 2022-06-29T15:33:02.857 INFO:teuthology.lock.ops:Updating smithi099.front.sepia.ceph.com on lock server 2022-06-29T15:33:02.876 DEBUG:teuthology.orchestra.run.smithi164:> uname -m 2022-06-29T15:33:02.884 INFO:teuthology.orchestra.run.smithi164.stdout:x86_64 2022-06-29T15:33:02.884 DEBUG:teuthology.orchestra.run.smithi164:> cat /etc/os-release 2022-06-29T15:33:02.931 INFO:teuthology.orchestra.run.smithi164.stdout:NAME="Ubuntu" 2022-06-29T15:33:02.931 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2022-06-29T15:33:02.932 INFO:teuthology.orchestra.run.smithi164.stdout:ID=ubuntu 2022-06-29T15:33:02.932 INFO:teuthology.orchestra.run.smithi164.stdout:ID_LIKE=debian 2022-06-29T15:33:02.932 INFO:teuthology.orchestra.run.smithi164.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2022-06-29T15:33:02.932 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_ID="20.04" 2022-06-29T15:33:02.933 INFO:teuthology.orchestra.run.smithi164.stdout:HOME_URL="https://www.ubuntu.com/" 2022-06-29T15:33:02.933 INFO:teuthology.orchestra.run.smithi164.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-06-29T15:33:02.933 INFO:teuthology.orchestra.run.smithi164.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-06-29T15:33:02.933 INFO:teuthology.orchestra.run.smithi164.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-06-29T15:33:02.934 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_CODENAME=focal 2022-06-29T15:33:02.934 INFO:teuthology.orchestra.run.smithi164.stdout:UBUNTU_CODENAME=focal 2022-06-29T15:33:02.935 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server 2022-06-29T15:33:02.952 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2022-06-29T15:33:02.963 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2022-06-29T15:33:02.989 INFO:teuthology.task.internal:Checking for old test directory... 2022-06-29T15:33:02.990 DEBUG:teuthology.orchestra.run.smithi047:> test '!' -e /home/ubuntu/cephtest 2022-06-29T15:33:02.992 DEBUG:teuthology.orchestra.run.smithi064:> test '!' -e /home/ubuntu/cephtest 2022-06-29T15:33:02.995 DEBUG:teuthology.orchestra.run.smithi099:> test '!' -e /home/ubuntu/cephtest 2022-06-29T15:33:02.998 DEBUG:teuthology.orchestra.run.smithi164:> test '!' -e /home/ubuntu/cephtest 2022-06-29T15:33:03.004 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2022-06-29T15:33:03.013 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2022-06-29T15:33:03.014 DEBUG:teuthology.orchestra.run.smithi047:> test -z $(ls -A /var/lib/ceph) 2022-06-29T15:33:03.016 DEBUG:teuthology.orchestra.run.smithi064:> test -z $(ls -A /var/lib/ceph) 2022-06-29T15:33:03.045 DEBUG:teuthology.orchestra.run.smithi099:> test -z $(ls -A /var/lib/ceph) 2022-06-29T15:33:03.048 DEBUG:teuthology.orchestra.run.smithi164:> test -z $(ls -A /var/lib/ceph) 2022-06-29T15:33:03.066 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2022-06-29T15:33:03.174 INFO:teuthology.run_tasks:Running task kernel... 2022-06-29T15:33:03.189 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2022-06-29T15:33:03.189 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}, 'client.0': {'kdb': True, 'sha1': 'distro'}, 'client.1': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2022-06-29T15:33:03.190 DEBUG:teuthology.orchestra.run.smithi047:> test -f /run/.containerenv -o -f /.dockerenv 2022-06-29T15:33:03.192 DEBUG:teuthology.orchestra.run.smithi064:> test -f /run/.containerenv -o -f /.dockerenv 2022-06-29T15:33:03.192 DEBUG:teuthology.orchestra.run.smithi099:> test -f /run/.containerenv -o -f /.dockerenv 2022-06-29T15:33:03.193 DEBUG:teuthology.orchestra.run.smithi164:> test -f /run/.containerenv -o -f /.dockerenv 2022-06-29T15:33:03.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T15:33:03.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T15:33:03.200 DEBUG:teuthology.orchestra.run.smithi064:> uname -r 2022-06-29T15:33:03.201 DEBUG:teuthology.orchestra.run.smithi099:> uname -r 2022-06-29T15:33:03.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T15:33:03.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T15:33:03.204 DEBUG:teuthology.orchestra.run.smithi164:> uname -r 2022-06-29T15:33:03.204 DEBUG:teuthology.orchestra.run.smithi047:> uname -r 2022-06-29T15:33:03.208 INFO:teuthology.orchestra.run.smithi064.stdout:5.4.0-120-generic 2022-06-29T15:33:03.208 INFO:teuthology.task.kernel:Running kernel on smithi064: 5.4.0-120-generic 2022-06-29T15:33:03.209 DEBUG:teuthology.orchestra.run.smithi064:> sudo apt-get clean 2022-06-29T15:33:03.210 INFO:teuthology.orchestra.run.smithi164.stdout:5.4.0-120-generic 2022-06-29T15:33:03.210 INFO:teuthology.task.kernel:Running kernel on smithi164: 5.4.0-120-generic 2022-06-29T15:33:03.211 DEBUG:teuthology.orchestra.run.smithi164:> sudo apt-get clean 2022-06-29T15:33:03.212 INFO:teuthology.orchestra.run.smithi047.stdout:5.4.0-120-generic 2022-06-29T15:33:03.212 INFO:teuthology.task.kernel:Running kernel on smithi047: 5.4.0-120-generic 2022-06-29T15:33:03.213 DEBUG:teuthology.orchestra.run.smithi047:> sudo apt-get clean 2022-06-29T15:33:03.250 INFO:teuthology.orchestra.run.smithi099.stdout:5.4.0-120-generic 2022-06-29T15:33:03.250 INFO:teuthology.task.kernel:Running kernel on smithi099: 5.4.0-120-generic 2022-06-29T15:33:03.251 DEBUG:teuthology.orchestra.run.smithi099:> sudo apt-get clean 2022-06-29T15:33:03.339 DEBUG:teuthology.orchestra.run.smithi064:> sudo apt-get update 2022-06-29T15:33:03.356 DEBUG:teuthology.orchestra.run.smithi099:> sudo apt-get update 2022-06-29T15:33:03.361 DEBUG:teuthology.orchestra.run.smithi047:> sudo apt-get update 2022-06-29T15:33:03.374 DEBUG:teuthology.orchestra.run.smithi164:> sudo apt-get update 2022-06-29T15:33:03.566 INFO:teuthology.orchestra.run.smithi047.stdout:Get:1 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2022-06-29T15:33:03.577 INFO:teuthology.orchestra.run.smithi099.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2022-06-29T15:33:03.620 INFO:teuthology.orchestra.run.smithi164.stdout:Get:1 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2022-06-29T15:33:03.625 INFO:teuthology.orchestra.run.smithi099.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2022-06-29T15:33:03.626 INFO:teuthology.orchestra.run.smithi099.stdout:Get:3 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2022-06-29T15:33:03.633 INFO:teuthology.orchestra.run.smithi047.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2022-06-29T15:33:03.651 INFO:teuthology.orchestra.run.smithi064.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2022-06-29T15:33:03.652 INFO:teuthology.orchestra.run.smithi064.stdout:Get:2 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2022-06-29T15:33:03.675 INFO:teuthology.orchestra.run.smithi164.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2022-06-29T15:33:03.697 INFO:teuthology.orchestra.run.smithi064.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2022-06-29T15:33:03.713 INFO:teuthology.orchestra.run.smithi047.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2022-06-29T15:33:03.735 INFO:teuthology.orchestra.run.smithi164.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2022-06-29T15:33:03.844 INFO:teuthology.orchestra.run.smithi099.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2022-06-29T15:33:03.846 INFO:teuthology.orchestra.run.smithi047.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [1,583 kB] 2022-06-29T15:33:03.966 INFO:teuthology.orchestra.run.smithi047.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [459 kB] 2022-06-29T15:33:03.984 INFO:teuthology.orchestra.run.smithi047.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [268 kB] 2022-06-29T15:33:03.985 INFO:teuthology.orchestra.run.smithi047.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [10.6 kB] 2022-06-29T15:33:03.985 INFO:teuthology.orchestra.run.smithi047.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [25.3 kB] 2022-06-29T15:33:03.985 INFO:teuthology.orchestra.run.smithi047.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [1,042 kB] 2022-06-29T15:33:03.986 INFO:teuthology.orchestra.run.smithi047.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [148 kB] 2022-06-29T15:33:03.987 INFO:teuthology.orchestra.run.smithi047.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [572 B] 2022-06-29T15:33:03.987 INFO:teuthology.orchestra.run.smithi047.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [708 kB] 2022-06-29T15:33:03.991 INFO:teuthology.orchestra.run.smithi047.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [554 kB] 2022-06-29T15:33:03.992 INFO:teuthology.orchestra.run.smithi099.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [685 kB] 2022-06-29T15:33:03.995 INFO:teuthology.orchestra.run.smithi047.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [14.6 kB] 2022-06-29T15:33:04.043 INFO:teuthology.orchestra.run.smithi064.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2022-06-29T15:33:04.048 INFO:teuthology.orchestra.run.smithi047.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2022-06-29T15:33:04.083 INFO:teuthology.orchestra.run.smithi164.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2022-06-29T15:33:04.094 INFO:teuthology.orchestra.run.smithi164.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [459 kB] 2022-06-29T15:33:04.122 INFO:teuthology.orchestra.run.smithi064.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [459 kB] 2022-06-29T15:33:04.187 INFO:teuthology.orchestra.run.smithi047.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [685 kB] 2022-06-29T15:33:04.196 INFO:teuthology.orchestra.run.smithi099.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [1,935 kB] 2022-06-29T15:33:04.256 INFO:teuthology.orchestra.run.smithi164.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [1,935 kB] 2022-06-29T15:33:04.269 INFO:teuthology.orchestra.run.smithi064.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [1,935 kB] 2022-06-29T15:33:04.288 INFO:teuthology.orchestra.run.smithi099.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [350 kB] 2022-06-29T15:33:04.296 INFO:teuthology.orchestra.run.smithi099.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [459 kB] 2022-06-29T15:33:04.307 INFO:teuthology.orchestra.run.smithi099.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [15.6 kB] 2022-06-29T15:33:04.307 INFO:teuthology.orchestra.run.smithi099.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [26.6 kB] 2022-06-29T15:33:04.314 INFO:teuthology.orchestra.run.smithi164.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [1,583 kB] 2022-06-29T15:33:04.370 INFO:teuthology.orchestra.run.smithi099.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [1,119 kB] 2022-06-29T15:33:04.394 INFO:teuthology.orchestra.run.smithi047.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [1,935 kB] 2022-06-29T15:33:04.395 INFO:teuthology.orchestra.run.smithi099.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [159 kB] 2022-06-29T15:33:04.397 INFO:teuthology.orchestra.run.smithi099.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [592 B] 2022-06-29T15:33:04.398 INFO:teuthology.orchestra.run.smithi099.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [677 kB] 2022-06-29T15:33:04.409 INFO:teuthology.orchestra.run.smithi099.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [924 kB] 2022-06-29T15:33:04.420 INFO:teuthology.orchestra.run.smithi064.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [1,583 kB] 2022-06-29T15:33:04.426 INFO:teuthology.orchestra.run.smithi099.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [208 kB] 2022-06-29T15:33:04.443 INFO:teuthology.orchestra.run.smithi099.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [20.9 kB] 2022-06-29T15:33:04.547 INFO:teuthology.orchestra.run.smithi047.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [350 kB] 2022-06-29T15:33:04.562 INFO:teuthology.orchestra.run.smithi047.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [15.6 kB] 2022-06-29T15:33:04.563 INFO:teuthology.orchestra.run.smithi047.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [26.6 kB] 2022-06-29T15:33:04.564 INFO:teuthology.orchestra.run.smithi047.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [1,119 kB] 2022-06-29T15:33:04.573 INFO:teuthology.orchestra.run.smithi099.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [1,583 kB] 2022-06-29T15:33:04.593 INFO:teuthology.orchestra.run.smithi064.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [268 kB] 2022-06-29T15:33:04.598 INFO:teuthology.orchestra.run.smithi047.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [159 kB] 2022-06-29T15:33:04.602 INFO:teuthology.orchestra.run.smithi047.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [592 B] 2022-06-29T15:33:04.602 INFO:teuthology.orchestra.run.smithi047.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [677 kB] 2022-06-29T15:33:04.603 INFO:teuthology.orchestra.run.smithi164.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [268 kB] 2022-06-29T15:33:04.606 INFO:teuthology.orchestra.run.smithi064.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [10.6 kB] 2022-06-29T15:33:04.606 INFO:teuthology.orchestra.run.smithi064.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [25.3 kB] 2022-06-29T15:33:04.607 INFO:teuthology.orchestra.run.smithi064.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [1,042 kB] 2022-06-29T15:33:04.611 INFO:teuthology.orchestra.run.smithi164.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [10.6 kB] 2022-06-29T15:33:04.611 INFO:teuthology.orchestra.run.smithi164.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [25.3 kB] 2022-06-29T15:33:04.611 INFO:teuthology.orchestra.run.smithi164.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [1,042 kB] 2022-06-29T15:33:04.619 INFO:teuthology.orchestra.run.smithi047.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [924 kB] 2022-06-29T15:33:04.643 INFO:teuthology.orchestra.run.smithi047.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [208 kB] 2022-06-29T15:33:04.644 INFO:teuthology.orchestra.run.smithi164.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [148 kB] 2022-06-29T15:33:04.646 INFO:teuthology.orchestra.run.smithi164.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [572 B] 2022-06-29T15:33:04.646 INFO:teuthology.orchestra.run.smithi164.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [554 kB] 2022-06-29T15:33:04.647 INFO:teuthology.orchestra.run.smithi047.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [20.9 kB] 2022-06-29T15:33:04.652 INFO:teuthology.orchestra.run.smithi164.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [708 kB] 2022-06-29T15:33:04.658 INFO:teuthology.orchestra.run.smithi064.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [148 kB] 2022-06-29T15:33:04.665 INFO:teuthology.orchestra.run.smithi064.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [572 B] 2022-06-29T15:33:04.665 INFO:teuthology.orchestra.run.smithi064.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [554 kB] 2022-06-29T15:33:04.667 INFO:teuthology.orchestra.run.smithi164.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [685 kB] 2022-06-29T15:33:04.680 INFO:teuthology.orchestra.run.smithi064.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [685 kB] 2022-06-29T15:33:04.682 INFO:teuthology.orchestra.run.smithi064.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [708 kB] 2022-06-29T15:33:04.686 INFO:teuthology.orchestra.run.smithi164.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [350 kB] 2022-06-29T15:33:04.691 INFO:teuthology.orchestra.run.smithi164.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [14.6 kB] 2022-06-29T15:33:04.695 INFO:teuthology.orchestra.run.smithi164.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [15.6 kB] 2022-06-29T15:33:04.696 INFO:teuthology.orchestra.run.smithi164.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [26.6 kB] 2022-06-29T15:33:04.697 INFO:teuthology.orchestra.run.smithi164.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [1,119 kB] 2022-06-29T15:33:04.698 INFO:teuthology.orchestra.run.smithi064.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [350 kB] 2022-06-29T15:33:04.700 INFO:teuthology.orchestra.run.smithi064.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [14.6 kB] 2022-06-29T15:33:04.708 INFO:teuthology.orchestra.run.smithi064.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [15.6 kB] 2022-06-29T15:33:04.708 INFO:teuthology.orchestra.run.smithi064.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [1,119 kB] 2022-06-29T15:33:04.745 INFO:teuthology.orchestra.run.smithi099.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [268 kB] 2022-06-29T15:33:04.751 INFO:teuthology.orchestra.run.smithi164.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [159 kB] 2022-06-29T15:33:04.756 INFO:teuthology.orchestra.run.smithi164.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [592 B] 2022-06-29T15:33:04.756 INFO:teuthology.orchestra.run.smithi164.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [924 kB] 2022-06-29T15:33:04.758 INFO:teuthology.orchestra.run.smithi099.stdout:Get:20 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [10.6 kB] 2022-06-29T15:33:04.759 INFO:teuthology.orchestra.run.smithi099.stdout:Get:21 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [25.3 kB] 2022-06-29T15:33:04.760 INFO:teuthology.orchestra.run.smithi099.stdout:Get:22 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [1,042 kB] 2022-06-29T15:33:04.764 INFO:teuthology.orchestra.run.smithi064.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [26.6 kB] 2022-06-29T15:33:04.765 INFO:teuthology.orchestra.run.smithi064.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [159 kB] 2022-06-29T15:33:04.769 INFO:teuthology.orchestra.run.smithi064.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [592 B] 2022-06-29T15:33:04.769 INFO:teuthology.orchestra.run.smithi064.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [677 kB] 2022-06-29T15:33:04.782 INFO:teuthology.orchestra.run.smithi164.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [677 kB] 2022-06-29T15:33:04.787 INFO:teuthology.orchestra.run.smithi064.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [924 kB] 2022-06-29T15:33:04.800 INFO:teuthology.orchestra.run.smithi164.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [208 kB] 2022-06-29T15:33:04.804 INFO:teuthology.orchestra.run.smithi164.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [20.9 kB] 2022-06-29T15:33:04.811 INFO:teuthology.orchestra.run.smithi099.stdout:Get:23 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [148 kB] 2022-06-29T15:33:04.813 INFO:teuthology.orchestra.run.smithi064.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [208 kB] 2022-06-29T15:33:04.817 INFO:teuthology.orchestra.run.smithi099.stdout:Get:24 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [572 B] 2022-06-29T15:33:04.818 INFO:teuthology.orchestra.run.smithi064.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [20.9 kB] 2022-06-29T15:33:04.818 INFO:teuthology.orchestra.run.smithi099.stdout:Get:25 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [708 kB] 2022-06-29T15:33:04.839 INFO:teuthology.orchestra.run.smithi099.stdout:Get:26 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [554 kB] 2022-06-29T15:33:04.852 INFO:teuthology.orchestra.run.smithi099.stdout:Get:27 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [14.6 kB] 2022-06-29T15:33:08.403 INFO:teuthology.orchestra.run.smithi047.stdout:Fetched 11.3 MB in 2s (6,225 kB/s) 2022-06-29T15:33:08.474 INFO:teuthology.orchestra.run.smithi099.stdout:Fetched 11.3 MB in 2s (5,522 kB/s) 2022-06-29T15:33:08.801 INFO:teuthology.orchestra.run.smithi164.stdout:Fetched 11.3 MB in 2s (4,887 kB/s) 2022-06-29T15:33:09.111 INFO:teuthology.orchestra.run.smithi064.stdout:Fetched 11.3 MB in 2s (4,684 kB/s) 2022-06-29T15:33:09.703 INFO:teuthology.orchestra.run.smithi047.stdout:Reading package lists... 2022-06-29T15:33:09.729 DEBUG:teuthology.orchestra.run.smithi047:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2022-06-29T15:33:09.789 INFO:teuthology.orchestra.run.smithi099.stdout:Reading package lists... 2022-06-29T15:33:09.797 INFO:teuthology.orchestra.run.smithi047.stdout:Reading package lists... 2022-06-29T15:33:09.813 DEBUG:teuthology.orchestra.run.smithi099:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2022-06-29T15:33:09.883 INFO:teuthology.orchestra.run.smithi099.stdout:Reading package lists... 2022-06-29T15:33:09.978 INFO:teuthology.orchestra.run.smithi047.stdout:Building dependency tree... 2022-06-29T15:33:09.979 INFO:teuthology.orchestra.run.smithi047.stdout:Reading state information... 2022-06-29T15:33:10.050 INFO:teuthology.orchestra.run.smithi099.stdout:Building dependency tree... 2022-06-29T15:33:10.051 INFO:teuthology.orchestra.run.smithi099.stdout:Reading state information... 2022-06-29T15:33:10.125 INFO:teuthology.orchestra.run.smithi164.stdout:Reading package lists... 2022-06-29T15:33:10.150 DEBUG:teuthology.orchestra.run.smithi164:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2022-06-29T15:33:10.151 INFO:teuthology.orchestra.run.smithi047.stdout:The following additional packages will be installed: 2022-06-29T15:33:10.152 INFO:teuthology.orchestra.run.smithi047.stdout: linux-generic linux-headers-5.4.0-121 linux-headers-5.4.0-121-generic 2022-06-29T15:33:10.152 INFO:teuthology.orchestra.run.smithi047.stdout: linux-headers-generic linux-image-5.4.0-121-generic 2022-06-29T15:33:10.152 INFO:teuthology.orchestra.run.smithi047.stdout: linux-modules-5.4.0-121-generic linux-modules-extra-5.4.0-121-generic 2022-06-29T15:33:10.152 INFO:teuthology.orchestra.run.smithi047.stdout:Suggested packages: 2022-06-29T15:33:10.152 INFO:teuthology.orchestra.run.smithi047.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2022-06-29T15:33:10.153 INFO:teuthology.orchestra.run.smithi047.stdout:Recommended packages: 2022-06-29T15:33:10.153 INFO:teuthology.orchestra.run.smithi047.stdout: thermald 2022-06-29T15:33:10.192 INFO:teuthology.orchestra.run.smithi099.stdout:The following additional packages will be installed: 2022-06-29T15:33:10.193 INFO:teuthology.orchestra.run.smithi099.stdout: linux-generic linux-headers-5.4.0-121 linux-headers-5.4.0-121-generic 2022-06-29T15:33:10.193 INFO:teuthology.orchestra.run.smithi099.stdout: linux-headers-generic linux-image-5.4.0-121-generic 2022-06-29T15:33:10.193 INFO:teuthology.orchestra.run.smithi099.stdout: linux-modules-5.4.0-121-generic linux-modules-extra-5.4.0-121-generic 2022-06-29T15:33:10.194 INFO:teuthology.orchestra.run.smithi099.stdout:Suggested packages: 2022-06-29T15:33:10.194 INFO:teuthology.orchestra.run.smithi099.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2022-06-29T15:33:10.194 INFO:teuthology.orchestra.run.smithi099.stdout:Recommended packages: 2022-06-29T15:33:10.194 INFO:teuthology.orchestra.run.smithi099.stdout: thermald 2022-06-29T15:33:10.204 INFO:teuthology.orchestra.run.smithi047.stdout:The following NEW packages will be installed: 2022-06-29T15:33:10.205 INFO:teuthology.orchestra.run.smithi047.stdout: linux-headers-5.4.0-121 linux-headers-5.4.0-121-generic 2022-06-29T15:33:10.205 INFO:teuthology.orchestra.run.smithi047.stdout: linux-image-5.4.0-121-generic linux-modules-5.4.0-121-generic 2022-06-29T15:33:10.205 INFO:teuthology.orchestra.run.smithi047.stdout: linux-modules-extra-5.4.0-121-generic 2022-06-29T15:33:10.206 INFO:teuthology.orchestra.run.smithi047.stdout:The following packages will be upgraded: 2022-06-29T15:33:10.207 INFO:teuthology.orchestra.run.smithi047.stdout: linux-generic linux-headers-generic linux-image-generic 2022-06-29T15:33:10.218 INFO:teuthology.orchestra.run.smithi164.stdout:Reading package lists... 2022-06-29T15:33:10.250 INFO:teuthology.orchestra.run.smithi099.stdout:The following NEW packages will be installed: 2022-06-29T15:33:10.250 INFO:teuthology.orchestra.run.smithi099.stdout: linux-headers-5.4.0-121 linux-headers-5.4.0-121-generic 2022-06-29T15:33:10.251 INFO:teuthology.orchestra.run.smithi099.stdout: linux-image-5.4.0-121-generic linux-modules-5.4.0-121-generic 2022-06-29T15:33:10.251 INFO:teuthology.orchestra.run.smithi099.stdout: linux-modules-extra-5.4.0-121-generic 2022-06-29T15:33:10.252 INFO:teuthology.orchestra.run.smithi099.stdout:The following packages will be upgraded: 2022-06-29T15:33:10.252 INFO:teuthology.orchestra.run.smithi099.stdout: linux-generic linux-headers-generic linux-image-generic 2022-06-29T15:33:10.296 INFO:teuthology.orchestra.run.smithi047.stdout:3 upgraded, 5 newly installed, 0 to remove and 17 not upgraded. 2022-06-29T15:33:10.296 INFO:teuthology.orchestra.run.smithi047.stdout:Need to get 77.1 MB of archives. 2022-06-29T15:33:10.297 INFO:teuthology.orchestra.run.smithi047.stdout:After this operation, 380 MB of additional disk space will be used. 2022-06-29T15:33:10.297 INFO:teuthology.orchestra.run.smithi047.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-121-generic amd64 5.4.0-121.137 [15.0 MB] 2022-06-29T15:33:10.378 INFO:teuthology.orchestra.run.smithi164.stdout:Building dependency tree... 2022-06-29T15:33:10.379 INFO:teuthology.orchestra.run.smithi164.stdout:Reading state information... 2022-06-29T15:33:10.415 INFO:teuthology.orchestra.run.smithi064.stdout:Reading package lists... 2022-06-29T15:33:10.430 INFO:teuthology.orchestra.run.smithi099.stdout:3 upgraded, 5 newly installed, 0 to remove and 17 not upgraded. 2022-06-29T15:33:10.431 INFO:teuthology.orchestra.run.smithi099.stdout:Need to get 77.1 MB of archives. 2022-06-29T15:33:10.431 INFO:teuthology.orchestra.run.smithi099.stdout:After this operation, 380 MB of additional disk space will be used. 2022-06-29T15:33:10.431 INFO:teuthology.orchestra.run.smithi099.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-121-generic amd64 5.4.0-121.137 [15.0 MB] 2022-06-29T15:33:10.440 DEBUG:teuthology.orchestra.run.smithi064:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2022-06-29T15:33:10.509 INFO:teuthology.orchestra.run.smithi064.stdout:Reading package lists... 2022-06-29T15:33:10.522 INFO:teuthology.orchestra.run.smithi164.stdout:The following additional packages will be installed: 2022-06-29T15:33:10.523 INFO:teuthology.orchestra.run.smithi164.stdout: linux-generic linux-headers-5.4.0-121 linux-headers-5.4.0-121-generic 2022-06-29T15:33:10.523 INFO:teuthology.orchestra.run.smithi164.stdout: linux-headers-generic linux-image-5.4.0-121-generic 2022-06-29T15:33:10.523 INFO:teuthology.orchestra.run.smithi164.stdout: linux-modules-5.4.0-121-generic linux-modules-extra-5.4.0-121-generic 2022-06-29T15:33:10.524 INFO:teuthology.orchestra.run.smithi164.stdout:Suggested packages: 2022-06-29T15:33:10.524 INFO:teuthology.orchestra.run.smithi164.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2022-06-29T15:33:10.524 INFO:teuthology.orchestra.run.smithi164.stdout:Recommended packages: 2022-06-29T15:33:10.524 INFO:teuthology.orchestra.run.smithi164.stdout: thermald 2022-06-29T15:33:10.584 INFO:teuthology.orchestra.run.smithi164.stdout:The following NEW packages will be installed: 2022-06-29T15:33:10.585 INFO:teuthology.orchestra.run.smithi164.stdout: linux-headers-5.4.0-121 linux-headers-5.4.0-121-generic 2022-06-29T15:33:10.585 INFO:teuthology.orchestra.run.smithi164.stdout: linux-image-5.4.0-121-generic linux-modules-5.4.0-121-generic 2022-06-29T15:33:10.585 INFO:teuthology.orchestra.run.smithi164.stdout: linux-modules-extra-5.4.0-121-generic 2022-06-29T15:33:10.586 INFO:teuthology.orchestra.run.smithi164.stdout:The following packages will be upgraded: 2022-06-29T15:33:10.586 INFO:teuthology.orchestra.run.smithi164.stdout: linux-generic linux-headers-generic linux-image-generic 2022-06-29T15:33:10.688 INFO:teuthology.orchestra.run.smithi064.stdout:Building dependency tree... 2022-06-29T15:33:10.689 INFO:teuthology.orchestra.run.smithi064.stdout:Reading state information... 2022-06-29T15:33:10.719 INFO:teuthology.orchestra.run.smithi164.stdout:3 upgraded, 5 newly installed, 0 to remove and 17 not upgraded. 2022-06-29T15:33:10.720 INFO:teuthology.orchestra.run.smithi164.stdout:Need to get 77.1 MB of archives. 2022-06-29T15:33:10.720 INFO:teuthology.orchestra.run.smithi164.stdout:After this operation, 380 MB of additional disk space will be used. 2022-06-29T15:33:10.720 INFO:teuthology.orchestra.run.smithi164.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-121-generic amd64 5.4.0-121.137 [15.0 MB] 2022-06-29T15:33:10.854 INFO:teuthology.orchestra.run.smithi064.stdout:The following additional packages will be installed: 2022-06-29T15:33:10.855 INFO:teuthology.orchestra.run.smithi064.stdout: linux-generic linux-headers-5.4.0-121 linux-headers-5.4.0-121-generic 2022-06-29T15:33:10.855 INFO:teuthology.orchestra.run.smithi064.stdout: linux-headers-generic linux-image-5.4.0-121-generic 2022-06-29T15:33:10.855 INFO:teuthology.orchestra.run.smithi064.stdout: linux-modules-5.4.0-121-generic linux-modules-extra-5.4.0-121-generic 2022-06-29T15:33:10.856 INFO:teuthology.orchestra.run.smithi064.stdout:Suggested packages: 2022-06-29T15:33:10.856 INFO:teuthology.orchestra.run.smithi064.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2022-06-29T15:33:10.856 INFO:teuthology.orchestra.run.smithi064.stdout:Recommended packages: 2022-06-29T15:33:10.856 INFO:teuthology.orchestra.run.smithi064.stdout: thermald 2022-06-29T15:33:10.910 INFO:teuthology.orchestra.run.smithi064.stdout:The following NEW packages will be installed: 2022-06-29T15:33:10.910 INFO:teuthology.orchestra.run.smithi064.stdout: linux-headers-5.4.0-121 linux-headers-5.4.0-121-generic 2022-06-29T15:33:10.910 INFO:teuthology.orchestra.run.smithi064.stdout: linux-image-5.4.0-121-generic linux-modules-5.4.0-121-generic 2022-06-29T15:33:10.911 INFO:teuthology.orchestra.run.smithi064.stdout: linux-modules-extra-5.4.0-121-generic 2022-06-29T15:33:10.911 INFO:teuthology.orchestra.run.smithi064.stdout:The following packages will be upgraded: 2022-06-29T15:33:10.912 INFO:teuthology.orchestra.run.smithi064.stdout: linux-generic linux-headers-generic linux-image-generic 2022-06-29T15:33:10.994 INFO:teuthology.orchestra.run.smithi064.stdout:3 upgraded, 5 newly installed, 0 to remove and 17 not upgraded. 2022-06-29T15:33:10.994 INFO:teuthology.orchestra.run.smithi064.stdout:Need to get 77.1 MB of archives. 2022-06-29T15:33:10.995 INFO:teuthology.orchestra.run.smithi064.stdout:After this operation, 380 MB of additional disk space will be used. 2022-06-29T15:33:10.995 INFO:teuthology.orchestra.run.smithi064.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-121-generic amd64 5.4.0-121.137 [15.0 MB] 2022-06-29T15:33:11.025 INFO:teuthology.orchestra.run.smithi047.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-121-generic amd64 5.4.0-121.137 [10.5 MB] 2022-06-29T15:33:11.350 INFO:teuthology.orchestra.run.smithi047.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-121-generic amd64 5.4.0-121.137 [39.2 MB] 2022-06-29T15:33:11.581 INFO:teuthology.orchestra.run.smithi099.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-121-generic amd64 5.4.0-121.137 [10.5 MB] 2022-06-29T15:33:11.856 INFO:teuthology.orchestra.run.smithi164.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-121-generic amd64 5.4.0-121.137 [10.5 MB] 2022-06-29T15:33:11.903 INFO:teuthology.orchestra.run.smithi099.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-121-generic amd64 5.4.0-121.137 [39.2 MB] 2022-06-29T15:33:12.348 INFO:teuthology.orchestra.run.smithi064.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-121-generic amd64 5.4.0-121.137 [10.5 MB] 2022-06-29T15:33:12.391 INFO:teuthology.orchestra.run.smithi164.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-121-generic amd64 5.4.0-121.137 [39.2 MB] 2022-06-29T15:33:12.798 INFO:teuthology.orchestra.run.smithi064.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-121-generic amd64 5.4.0-121.137 [39.2 MB] 2022-06-29T15:33:13.193 INFO:teuthology.orchestra.run.smithi099.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.121.122 [1,896 B] 2022-06-29T15:33:13.194 INFO:teuthology.orchestra.run.smithi099.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.121.122 [2,560 B] 2022-06-29T15:33:13.194 INFO:teuthology.orchestra.run.smithi099.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-121 all 5.4.0-121.137 [11.0 MB] 2022-06-29T15:33:13.262 INFO:teuthology.orchestra.run.smithi047.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.121.122 [1,896 B] 2022-06-29T15:33:13.263 INFO:teuthology.orchestra.run.smithi047.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.121.122 [2,560 B] 2022-06-29T15:33:13.263 INFO:teuthology.orchestra.run.smithi047.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-121 all 5.4.0-121.137 [11.0 MB] 2022-06-29T15:33:13.598 INFO:teuthology.orchestra.run.smithi099.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-121-generic amd64 5.4.0-121.137 [1,380 kB] 2022-06-29T15:33:13.643 INFO:teuthology.orchestra.run.smithi099.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.121.122 [2,424 B] 2022-06-29T15:33:13.645 INFO:teuthology.orchestra.run.smithi047.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-121-generic amd64 5.4.0-121.137 [1,380 kB] 2022-06-29T15:33:13.677 INFO:teuthology.orchestra.run.smithi047.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.121.122 [2,424 B] 2022-06-29T15:33:14.077 INFO:teuthology.orchestra.run.smithi099.stdout:Fetched 77.1 MB in 3s (22.8 MB/s) 2022-06-29T15:33:14.094 INFO:teuthology.orchestra.run.smithi047.stdout:Fetched 77.1 MB in 3s (22.2 MB/s) 2022-06-29T15:33:14.231 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package linux-modules-5.4.0-121-generic. 2022-06-29T15:33:14.253 INFO:teuthology.orchestra.run.smithi064.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.121.122 [1,896 B] 2022-06-29T15:33:14.254 INFO:teuthology.orchestra.run.smithi064.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.121.122 [2,560 B] 2022-06-29T15:33:14.254 INFO:teuthology.orchestra.run.smithi064.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-121 all 5.4.0-121.137 [11.0 MB] 2022-06-29T15:33:14.260 INFO:teuthology.orchestra.run.smithi164.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.121.122 [1,896 B] 2022-06-29T15:33:14.260 INFO:teuthology.orchestra.run.smithi164.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.121.122 [2,560 B] 2022-06-29T15:33:14.260 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package linux-modules-5.4.0-121-generic. 2022-06-29T15:33:14.261 INFO:teuthology.orchestra.run.smithi164.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-121 all 5.4.0-121.137 [11.0 MB] 2022-06-29T15:33:14.512 INFO:teuthology.orchestra.run.smithi064.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-121-generic amd64 5.4.0-121.137 [1,380 kB] 2022-06-29T15:33:14.526 INFO:teuthology.orchestra.run.smithi164.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-121-generic amd64 5.4.0-121.137 [1,380 kB] 2022-06-29T15:33:14.587 INFO:teuthology.orchestra.run.smithi164.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.121.122 [2,424 B] 2022-06-29T15:33:14.596 INFO:teuthology.orchestra.run.smithi064.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.121.122 [2,424 B] 2022-06-29T15:33:15.025 INFO:teuthology.orchestra.run.smithi164.stdout:Fetched 77.1 MB in 4s (19.3 MB/s) 2022-06-29T15:33:15.045 INFO:teuthology.orchestra.run.smithi064.stdout:Fetched 77.1 MB in 4s (21.0 MB/s) 2022-06-29T15:33:15.200 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package linux-modules-5.4.0-121-generic. 2022-06-29T15:33:15.226 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package linux-modules-5.4.0-121-generic. 2022-06-29T15:33:15.787 INFO:teuthology.orchestra.run.smithi099.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102341 files and directories currently installed.) 2022-06-29T15:33:15.792 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../0-linux-modules-5.4.0-121-generic_5.4.0-121.137_amd64.deb ... 2022-06-29T15:33:15.795 INFO:teuthology.orchestra.run.smithi047.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102341 files and directories currently installed.) 2022-06-29T15:33:15.800 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../0-linux-modules-5.4.0-121-generic_5.4.0-121.137_amd64.deb ... 2022-06-29T15:33:15.931 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking linux-modules-5.4.0-121-generic (5.4.0-121.137) ... 2022-06-29T15:33:15.958 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking linux-modules-5.4.0-121-generic (5.4.0-121.137) ... 2022-06-29T15:33:16.769 INFO:teuthology.orchestra.run.smithi164.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102341 files and directories currently installed.) 2022-06-29T15:33:16.774 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../0-linux-modules-5.4.0-121-generic_5.4.0-121.137_amd64.deb ... 2022-06-29T15:33:16.775 INFO:teuthology.orchestra.run.smithi064.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102341 files and directories currently installed.) 2022-06-29T15:33:16.779 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../0-linux-modules-5.4.0-121-generic_5.4.0-121.137_amd64.deb ... 2022-06-29T15:33:16.924 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking linux-modules-5.4.0-121-generic (5.4.0-121.137) ... 2022-06-29T15:33:16.949 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking linux-modules-5.4.0-121-generic (5.4.0-121.137) ... 2022-06-29T15:33:17.812 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package linux-image-5.4.0-121-generic. 2022-06-29T15:33:17.827 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../1-linux-image-5.4.0-121-generic_5.4.0-121.137_amd64.deb ... 2022-06-29T15:33:17.883 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package linux-image-5.4.0-121-generic. 2022-06-29T15:33:17.897 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../1-linux-image-5.4.0-121-generic_5.4.0-121.137_amd64.deb ... 2022-06-29T15:33:17.948 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking linux-image-5.4.0-121-generic (5.4.0-121.137) ... 2022-06-29T15:33:17.981 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking linux-image-5.4.0-121-generic (5.4.0-121.137) ... 2022-06-29T15:33:18.911 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package linux-image-5.4.0-121-generic. 2022-06-29T15:33:18.925 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../1-linux-image-5.4.0-121-generic_5.4.0-121.137_amd64.deb ... 2022-06-29T15:33:18.953 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package linux-image-5.4.0-121-generic. 2022-06-29T15:33:18.970 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../1-linux-image-5.4.0-121-generic_5.4.0-121.137_amd64.deb ... 2022-06-29T15:33:19.023 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking linux-image-5.4.0-121-generic (5.4.0-121.137) ... 2022-06-29T15:33:19.045 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-121-generic. 2022-06-29T15:33:19.058 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-121-generic_5.4.0-121.137_amd64.deb ... 2022-06-29T15:33:19.070 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking linux-image-5.4.0-121-generic (5.4.0-121.137) ... 2022-06-29T15:33:19.091 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-121-generic. 2022-06-29T15:33:19.095 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking linux-modules-extra-5.4.0-121-generic (5.4.0-121.137) ... 2022-06-29T15:33:19.105 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-121-generic_5.4.0-121.137_amd64.deb ... 2022-06-29T15:33:19.149 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking linux-modules-extra-5.4.0-121-generic (5.4.0-121.137) ... 2022-06-29T15:33:20.119 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-121-generic. 2022-06-29T15:33:20.133 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-121-generic_5.4.0-121.137_amd64.deb ... 2022-06-29T15:33:20.177 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking linux-modules-extra-5.4.0-121-generic (5.4.0-121.137) ... 2022-06-29T15:33:20.186 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-121-generic. 2022-06-29T15:33:20.200 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-121-generic_5.4.0-121.137_amd64.deb ... 2022-06-29T15:33:20.244 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking linux-modules-extra-5.4.0-121-generic (5.4.0-121.137) ... 2022-06-29T15:33:23.770 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../3-linux-generic_5.4.0.121.122_amd64.deb ... 2022-06-29T15:33:23.913 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking linux-generic (5.4.0.121.122) over (5.4.0.120.121) ... 2022-06-29T15:33:24.073 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../3-linux-generic_5.4.0.121.122_amd64.deb ... 2022-06-29T15:33:24.197 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking linux-generic (5.4.0.121.122) over (5.4.0.120.121) ... 2022-06-29T15:33:24.248 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.121.122_amd64.deb ... 2022-06-29T15:33:24.383 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking linux-image-generic (5.4.0.121.122) over (5.4.0.120.121) ... 2022-06-29T15:33:24.573 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.121.122_amd64.deb ... 2022-06-29T15:33:24.700 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking linux-image-generic (5.4.0.121.122) over (5.4.0.120.121) ... 2022-06-29T15:33:24.736 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package linux-headers-5.4.0-121. 2022-06-29T15:33:24.751 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../5-linux-headers-5.4.0-121_5.4.0-121.137_all.deb ... 2022-06-29T15:33:24.795 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking linux-headers-5.4.0-121 (5.4.0-121.137) ... 2022-06-29T15:33:24.961 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../3-linux-generic_5.4.0.121.122_amd64.deb ... 2022-06-29T15:33:25.037 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package linux-headers-5.4.0-121. 2022-06-29T15:33:25.051 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../5-linux-headers-5.4.0-121_5.4.0-121.137_all.deb ... 2022-06-29T15:33:25.086 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking linux-headers-5.4.0-121 (5.4.0-121.137) ... 2022-06-29T15:33:25.102 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking linux-generic (5.4.0.121.122) over (5.4.0.120.121) ... 2022-06-29T15:33:25.118 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../3-linux-generic_5.4.0.121.122_amd64.deb ... 2022-06-29T15:33:25.245 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking linux-generic (5.4.0.121.122) over (5.4.0.120.121) ... 2022-06-29T15:33:25.445 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.121.122_amd64.deb ... 2022-06-29T15:33:25.581 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking linux-image-generic (5.4.0.121.122) over (5.4.0.120.121) ... 2022-06-29T15:33:25.611 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.121.122_amd64.deb ... 2022-06-29T15:33:25.740 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking linux-image-generic (5.4.0.121.122) over (5.4.0.120.121) ... 2022-06-29T15:33:25.900 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package linux-headers-5.4.0-121. 2022-06-29T15:33:25.916 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../5-linux-headers-5.4.0-121_5.4.0-121.137_all.deb ... 2022-06-29T15:33:25.958 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking linux-headers-5.4.0-121 (5.4.0-121.137) ... 2022-06-29T15:33:26.051 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package linux-headers-5.4.0-121. 2022-06-29T15:33:26.064 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../5-linux-headers-5.4.0-121_5.4.0-121.137_all.deb ... 2022-06-29T15:33:26.118 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking linux-headers-5.4.0-121 (5.4.0-121.137) ... 2022-06-29T15:33:30.226 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package linux-headers-5.4.0-121-generic. 2022-06-29T15:33:30.249 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../6-linux-headers-5.4.0-121-generic_5.4.0-121.137_amd64.deb ... 2022-06-29T15:33:30.301 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking linux-headers-5.4.0-121-generic (5.4.0-121.137) ... 2022-06-29T15:33:30.584 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package linux-headers-5.4.0-121-generic. 2022-06-29T15:33:30.607 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../6-linux-headers-5.4.0-121-generic_5.4.0-121.137_amd64.deb ... 2022-06-29T15:33:30.651 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking linux-headers-5.4.0-121-generic (5.4.0-121.137) ... 2022-06-29T15:33:31.657 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package linux-headers-5.4.0-121-generic. 2022-06-29T15:33:31.678 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../6-linux-headers-5.4.0-121-generic_5.4.0-121.137_amd64.deb ... 2022-06-29T15:33:31.731 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking linux-headers-5.4.0-121-generic (5.4.0-121.137) ... 2022-06-29T15:33:31.739 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package linux-headers-5.4.0-121-generic. 2022-06-29T15:33:31.762 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../6-linux-headers-5.4.0-121-generic_5.4.0-121.137_amd64.deb ... 2022-06-29T15:33:31.806 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking linux-headers-5.4.0-121-generic (5.4.0-121.137) ... 2022-06-29T15:33:32.894 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.121.122_amd64.deb ... 2022-06-29T15:33:33.059 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking linux-headers-generic (5.4.0.121.122) over (5.4.0.120.121) ... 2022-06-29T15:33:33.368 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.121.122_amd64.deb ... 2022-06-29T15:33:33.465 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up linux-modules-5.4.0-121-generic (5.4.0-121.137) ... 2022-06-29T15:33:33.507 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking linux-headers-generic (5.4.0.121.122) over (5.4.0.120.121) ... 2022-06-29T15:33:33.834 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up linux-modules-5.4.0-121-generic (5.4.0-121.137) ... 2022-06-29T15:33:34.569 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.121.122_amd64.deb ... 2022-06-29T15:33:34.716 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking linux-headers-generic (5.4.0.121.122) over (5.4.0.120.121) ... 2022-06-29T15:33:34.729 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up linux-headers-5.4.0-121 (5.4.0-121.137) ... 2022-06-29T15:33:34.855 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up linux-image-5.4.0-121-generic (5.4.0-121.137) ... 2022-06-29T15:33:34.954 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.121.122_amd64.deb ... 2022-06-29T15:33:35.038 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up linux-headers-5.4.0-121 (5.4.0-121.137) ... 2022-06-29T15:33:35.190 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up linux-image-5.4.0-121-generic (5.4.0-121.137) ... 2022-06-29T15:33:35.818 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking linux-headers-generic (5.4.0.121.122) over (5.4.0.120.121) ... 2022-06-29T15:33:35.960 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up linux-modules-5.4.0-121-generic (5.4.0-121.137) ... 2022-06-29T15:33:36.025 INFO:teuthology.orchestra.run.smithi047.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-121-generic 2022-06-29T15:33:36.025 INFO:teuthology.orchestra.run.smithi047.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-121-generic 2022-06-29T15:33:36.190 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up linux-headers-5.4.0-121-generic (5.4.0-121.137) ... 2022-06-29T15:33:36.316 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up linux-headers-generic (5.4.0.121.122) ... 2022-06-29T15:33:36.327 INFO:teuthology.orchestra.run.smithi099.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-121-generic 2022-06-29T15:33:36.327 INFO:teuthology.orchestra.run.smithi099.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-121-generic 2022-06-29T15:33:36.383 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up linux-modules-5.4.0-121-generic (5.4.0-121.137) ... 2022-06-29T15:33:36.442 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up linux-modules-extra-5.4.0-121-generic (5.4.0-121.137) ... 2022-06-29T15:33:36.483 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up linux-headers-5.4.0-121-generic (5.4.0-121.137) ... 2022-06-29T15:33:36.609 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up linux-headers-generic (5.4.0.121.122) ... 2022-06-29T15:33:36.753 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up linux-modules-extra-5.4.0-121-generic (5.4.0-121.137) ... 2022-06-29T15:33:37.531 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up linux-headers-5.4.0-121 (5.4.0-121.137) ... 2022-06-29T15:33:37.620 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up linux-headers-5.4.0-121 (5.4.0-121.137) ... 2022-06-29T15:33:37.656 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up linux-image-5.4.0-121-generic (5.4.0-121.137) ... 2022-06-29T15:33:37.686 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up linux-image-generic (5.4.0.121.122) ... 2022-06-29T15:33:37.779 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up linux-image-5.4.0-121-generic (5.4.0-121.137) ... 2022-06-29T15:33:37.794 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up linux-generic (5.4.0.121.122) ... 2022-06-29T15:33:37.921 INFO:teuthology.orchestra.run.smithi047.stdout:Processing triggers for linux-image-5.4.0-121-generic (5.4.0-121.137) ... 2022-06-29T15:33:37.972 INFO:teuthology.orchestra.run.smithi047.stdout:/etc/kernel/postinst.d/initramfs-tools: 2022-06-29T15:33:37.973 INFO:teuthology.orchestra.run.smithi047.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-121-generic 2022-06-29T15:33:38.187 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up linux-image-generic (5.4.0.121.122) ... 2022-06-29T15:33:38.313 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up linux-generic (5.4.0.121.122) ... 2022-06-29T15:33:38.450 INFO:teuthology.orchestra.run.smithi099.stdout:Processing triggers for linux-image-5.4.0-121-generic (5.4.0-121.137) ... 2022-06-29T15:33:38.500 INFO:teuthology.orchestra.run.smithi099.stdout:/etc/kernel/postinst.d/initramfs-tools: 2022-06-29T15:33:38.501 INFO:teuthology.orchestra.run.smithi099.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-121-generic 2022-06-29T15:33:38.837 INFO:teuthology.orchestra.run.smithi064.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-121-generic 2022-06-29T15:33:38.837 INFO:teuthology.orchestra.run.smithi064.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-121-generic 2022-06-29T15:33:38.933 INFO:teuthology.orchestra.run.smithi164.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-121-generic 2022-06-29T15:33:38.934 INFO:teuthology.orchestra.run.smithi164.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-121-generic 2022-06-29T15:33:38.975 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up linux-headers-5.4.0-121-generic (5.4.0-121.137) ... 2022-06-29T15:33:39.101 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up linux-headers-generic (5.4.0.121.122) ... 2022-06-29T15:33:39.114 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up linux-headers-5.4.0-121-generic (5.4.0-121.137) ... 2022-06-29T15:33:39.241 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up linux-headers-generic (5.4.0.121.122) ... 2022-06-29T15:33:39.244 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up linux-modules-extra-5.4.0-121-generic (5.4.0-121.137) ... 2022-06-29T15:33:39.383 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up linux-modules-extra-5.4.0-121-generic (5.4.0-121.137) ... 2022-06-29T15:33:40.479 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up linux-image-generic (5.4.0.121.122) ... 2022-06-29T15:33:40.568 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up linux-image-generic (5.4.0.121.122) ... 2022-06-29T15:33:40.605 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up linux-generic (5.4.0.121.122) ... 2022-06-29T15:33:40.711 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up linux-generic (5.4.0.121.122) ... 2022-06-29T15:33:40.731 INFO:teuthology.orchestra.run.smithi064.stdout:Processing triggers for linux-image-5.4.0-121-generic (5.4.0-121.137) ... 2022-06-29T15:33:40.783 INFO:teuthology.orchestra.run.smithi064.stdout:/etc/kernel/postinst.d/initramfs-tools: 2022-06-29T15:33:40.783 INFO:teuthology.orchestra.run.smithi064.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-121-generic 2022-06-29T15:33:40.837 INFO:teuthology.orchestra.run.smithi164.stdout:Processing triggers for linux-image-5.4.0-121-generic (5.4.0-121.137) ... 2022-06-29T15:33:40.889 INFO:teuthology.orchestra.run.smithi164.stdout:/etc/kernel/postinst.d/initramfs-tools: 2022-06-29T15:33:40.889 INFO:teuthology.orchestra.run.smithi164.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-121-generic 2022-06-29T15:34:00.567 INFO:teuthology.orchestra.run.smithi099.stdout:/etc/kernel/postinst.d/zz-update-grub: 2022-06-29T15:34:00.568 INFO:teuthology.orchestra.run.smithi099.stdout:Sourcing file `/etc/default/grub' 2022-06-29T15:34:00.571 INFO:teuthology.orchestra.run.smithi099.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2022-06-29T15:34:00.591 INFO:teuthology.orchestra.run.smithi099.stdout:Generating grub configuration file ... 2022-06-29T15:34:00.633 INFO:teuthology.orchestra.run.smithi047.stdout:/etc/kernel/postinst.d/zz-update-grub: 2022-06-29T15:34:00.633 INFO:teuthology.orchestra.run.smithi047.stdout:Sourcing file `/etc/default/grub' 2022-06-29T15:34:00.643 INFO:teuthology.orchestra.run.smithi047.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2022-06-29T15:34:00.672 INFO:teuthology.orchestra.run.smithi047.stdout:Generating grub configuration file ... 2022-06-29T15:34:00.960 INFO:teuthology.orchestra.run.smithi099.stdout:Found linux image: /boot/vmlinuz-5.4.0-121-generic 2022-06-29T15:34:00.983 INFO:teuthology.orchestra.run.smithi099.stdout:Found initrd image: /boot/initrd.img-5.4.0-121-generic 2022-06-29T15:34:00.990 INFO:teuthology.orchestra.run.smithi047.stdout:Found linux image: /boot/vmlinuz-5.4.0-121-generic 2022-06-29T15:34:01.016 INFO:teuthology.orchestra.run.smithi047.stdout:Found initrd image: /boot/initrd.img-5.4.0-121-generic 2022-06-29T15:34:01.388 INFO:teuthology.orchestra.run.smithi047.stdout:Found linux image: /boot/vmlinuz-5.4.0-120-generic 2022-06-29T15:34:01.393 INFO:teuthology.orchestra.run.smithi099.stdout:Found linux image: /boot/vmlinuz-5.4.0-120-generic 2022-06-29T15:34:01.399 INFO:teuthology.orchestra.run.smithi047.stdout:Found initrd image: /boot/initrd.img-5.4.0-120-generic 2022-06-29T15:34:01.405 INFO:teuthology.orchestra.run.smithi099.stdout:Found initrd image: /boot/initrd.img-5.4.0-120-generic 2022-06-29T15:34:03.034 INFO:teuthology.orchestra.run.smithi164.stdout:/etc/kernel/postinst.d/zz-update-grub: 2022-06-29T15:34:03.034 INFO:teuthology.orchestra.run.smithi164.stdout:Sourcing file `/etc/default/grub' 2022-06-29T15:34:03.044 INFO:teuthology.orchestra.run.smithi164.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2022-06-29T15:34:03.101 INFO:teuthology.orchestra.run.smithi164.stdout:Generating grub configuration file ... 2022-06-29T15:34:03.125 INFO:teuthology.orchestra.run.smithi047.stdout:done 2022-06-29T15:34:03.284 INFO:teuthology.orchestra.run.smithi064.stdout:/etc/kernel/postinst.d/zz-update-grub: 2022-06-29T15:34:03.284 INFO:teuthology.orchestra.run.smithi064.stdout:Sourcing file `/etc/default/grub' 2022-06-29T15:34:03.293 INFO:teuthology.orchestra.run.smithi064.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2022-06-29T15:34:03.312 INFO:teuthology.orchestra.run.smithi064.stdout:Generating grub configuration file ... 2022-06-29T15:34:03.472 INFO:teuthology.orchestra.run.smithi164.stdout:Found linux image: /boot/vmlinuz-5.4.0-121-generic 2022-06-29T15:34:03.489 DEBUG:teuthology.orchestra.run.smithi047:> dpkg -s linux-image-generic 2022-06-29T15:34:03.496 INFO:teuthology.orchestra.run.smithi164.stdout:Found initrd image: /boot/initrd.img-5.4.0-121-generic 2022-06-29T15:34:03.523 INFO:teuthology.orchestra.run.smithi047.stdout:Package: linux-image-generic 2022-06-29T15:34:03.523 INFO:teuthology.orchestra.run.smithi047.stdout:Status: install ok installed 2022-06-29T15:34:03.523 INFO:teuthology.orchestra.run.smithi047.stdout:Priority: optional 2022-06-29T15:34:03.523 INFO:teuthology.orchestra.run.smithi047.stdout:Section: kernel 2022-06-29T15:34:03.524 INFO:teuthology.orchestra.run.smithi047.stdout:Installed-Size: 19 2022-06-29T15:34:03.524 INFO:teuthology.orchestra.run.smithi047.stdout:Maintainer: Ubuntu Kernel Team 2022-06-29T15:34:03.524 INFO:teuthology.orchestra.run.smithi047.stdout:Architecture: amd64 2022-06-29T15:34:03.525 INFO:teuthology.orchestra.run.smithi047.stdout:Source: linux-meta 2022-06-29T15:34:03.525 INFO:teuthology.orchestra.run.smithi047.stdout:Version: 5.4.0.121.122 2022-06-29T15:34:03.525 INFO:teuthology.orchestra.run.smithi047.stdout:Provides: virtualbox-guest-modules (= 6.1.32-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.14) 2022-06-29T15:34:03.525 INFO:teuthology.orchestra.run.smithi047.stdout:Depends: linux-image-5.4.0-121-generic, linux-modules-extra-5.4.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2022-06-29T15:34:03.525 INFO:teuthology.orchestra.run.smithi047.stdout:Recommends: thermald 2022-06-29T15:34:03.526 INFO:teuthology.orchestra.run.smithi047.stdout:Description: Generic Linux kernel image 2022-06-29T15:34:03.526 INFO:teuthology.orchestra.run.smithi047.stdout: This package will always depend on the latest generic kernel image 2022-06-29T15:34:03.526 INFO:teuthology.orchestra.run.smithi047.stdout: available. 2022-06-29T15:34:03.527 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-120-generic Expected: 5.4.0-121-generic 2022-06-29T15:34:03.527 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2022-06-29T15:34:03.527 DEBUG:teuthology.task.kernel:src is distro, skipping download 2022-06-29T15:34:03.528 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2022-06-29T15:34:03.528 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi047.front.sepia.ceph.com, path=None, version=distro) 2022-06-29T15:34:03.528 DEBUG:teuthology.orchestra.run.smithi047:> sudo apt-get clean 2022-06-29T15:34:03.606 INFO:teuthology.orchestra.run.smithi064.stdout:Found linux image: /boot/vmlinuz-5.4.0-121-generic 2022-06-29T15:34:03.626 DEBUG:teuthology.orchestra.run.smithi047:> sudo apt-get update 2022-06-29T15:34:03.631 INFO:teuthology.orchestra.run.smithi064.stdout:Found initrd image: /boot/initrd.img-5.4.0-121-generic 2022-06-29T15:34:03.832 INFO:teuthology.orchestra.run.smithi047.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2022-06-29T15:34:03.833 INFO:teuthology.orchestra.run.smithi099.stdout:done 2022-06-29T15:34:03.872 INFO:teuthology.orchestra.run.smithi047.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2022-06-29T15:34:03.875 INFO:teuthology.orchestra.run.smithi047.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2022-06-29T15:34:03.913 INFO:teuthology.orchestra.run.smithi047.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2022-06-29T15:34:03.993 INFO:teuthology.orchestra.run.smithi164.stdout:Found linux image: /boot/vmlinuz-5.4.0-120-generic 2022-06-29T15:34:04.004 INFO:teuthology.orchestra.run.smithi164.stdout:Found initrd image: /boot/initrd.img-5.4.0-120-generic 2022-06-29T15:34:04.017 INFO:teuthology.orchestra.run.smithi064.stdout:Found linux image: /boot/vmlinuz-5.4.0-120-generic 2022-06-29T15:34:04.029 INFO:teuthology.orchestra.run.smithi064.stdout:Found initrd image: /boot/initrd.img-5.4.0-120-generic 2022-06-29T15:34:04.215 DEBUG:teuthology.orchestra.run.smithi099:> dpkg -s linux-image-generic 2022-06-29T15:34:04.241 INFO:teuthology.orchestra.run.smithi099.stdout:Package: linux-image-generic 2022-06-29T15:34:04.242 INFO:teuthology.orchestra.run.smithi099.stdout:Status: install ok installed 2022-06-29T15:34:04.242 INFO:teuthology.orchestra.run.smithi099.stdout:Priority: optional 2022-06-29T15:34:04.242 INFO:teuthology.orchestra.run.smithi099.stdout:Section: kernel 2022-06-29T15:34:04.242 INFO:teuthology.orchestra.run.smithi099.stdout:Installed-Size: 19 2022-06-29T15:34:04.243 INFO:teuthology.orchestra.run.smithi099.stdout:Maintainer: Ubuntu Kernel Team 2022-06-29T15:34:04.243 INFO:teuthology.orchestra.run.smithi099.stdout:Architecture: amd64 2022-06-29T15:34:04.243 INFO:teuthology.orchestra.run.smithi099.stdout:Source: linux-meta 2022-06-29T15:34:04.243 INFO:teuthology.orchestra.run.smithi099.stdout:Version: 5.4.0.121.122 2022-06-29T15:34:04.244 INFO:teuthology.orchestra.run.smithi099.stdout:Provides: virtualbox-guest-modules (= 6.1.32-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.14) 2022-06-29T15:34:04.244 INFO:teuthology.orchestra.run.smithi099.stdout:Depends: linux-image-5.4.0-121-generic, linux-modules-extra-5.4.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2022-06-29T15:34:04.244 INFO:teuthology.orchestra.run.smithi099.stdout:Recommends: thermald 2022-06-29T15:34:04.244 INFO:teuthology.orchestra.run.smithi099.stdout:Description: Generic Linux kernel image 2022-06-29T15:34:04.244 INFO:teuthology.orchestra.run.smithi099.stdout: This package will always depend on the latest generic kernel image 2022-06-29T15:34:04.245 INFO:teuthology.orchestra.run.smithi099.stdout: available. 2022-06-29T15:34:04.245 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-120-generic Expected: 5.4.0-121-generic 2022-06-29T15:34:04.245 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2022-06-29T15:34:04.246 DEBUG:teuthology.task.kernel:src is distro, skipping download 2022-06-29T15:34:04.246 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2022-06-29T15:34:04.246 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi099.front.sepia.ceph.com, path=None, version=distro) 2022-06-29T15:34:04.246 DEBUG:teuthology.orchestra.run.smithi099:> sudo apt-get clean 2022-06-29T15:34:04.322 DEBUG:teuthology.orchestra.run.smithi099:> sudo apt-get update 2022-06-29T15:34:04.527 INFO:teuthology.orchestra.run.smithi099.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2022-06-29T15:34:04.528 INFO:teuthology.orchestra.run.smithi099.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2022-06-29T15:34:04.572 INFO:teuthology.orchestra.run.smithi099.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2022-06-29T15:34:04.612 INFO:teuthology.orchestra.run.smithi099.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2022-06-29T15:34:05.472 INFO:teuthology.orchestra.run.smithi047.stdout:Reading package lists... 2022-06-29T15:34:05.497 DEBUG:teuthology.orchestra.run.smithi047:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2022-06-29T15:34:05.564 INFO:teuthology.orchestra.run.smithi047.stdout:Reading package lists... 2022-06-29T15:34:05.740 INFO:teuthology.orchestra.run.smithi047.stdout:Building dependency tree... 2022-06-29T15:34:05.741 INFO:teuthology.orchestra.run.smithi047.stdout:Reading state information... 2022-06-29T15:34:05.761 INFO:teuthology.orchestra.run.smithi164.stdout:done 2022-06-29T15:34:05.940 INFO:teuthology.orchestra.run.smithi047.stdout:linux-image-generic is already the newest version (5.4.0.121.122). 2022-06-29T15:34:05.941 INFO:teuthology.orchestra.run.smithi047.stdout:0 upgraded, 0 newly installed, 0 to remove and 17 not upgraded. 2022-06-29T15:34:05.942 DEBUG:teuthology.orchestra.run.smithi047:> dpkg -s linux-image-generic 2022-06-29T15:34:05.966 INFO:teuthology.orchestra.run.smithi047.stdout:Package: linux-image-generic 2022-06-29T15:34:05.966 INFO:teuthology.orchestra.run.smithi047.stdout:Status: install ok installed 2022-06-29T15:34:05.967 INFO:teuthology.orchestra.run.smithi047.stdout:Priority: optional 2022-06-29T15:34:05.967 INFO:teuthology.orchestra.run.smithi047.stdout:Section: kernel 2022-06-29T15:34:05.968 INFO:teuthology.orchestra.run.smithi047.stdout:Installed-Size: 19 2022-06-29T15:34:05.968 INFO:teuthology.orchestra.run.smithi047.stdout:Maintainer: Ubuntu Kernel Team 2022-06-29T15:34:05.968 INFO:teuthology.orchestra.run.smithi047.stdout:Architecture: amd64 2022-06-29T15:34:05.969 INFO:teuthology.orchestra.run.smithi047.stdout:Source: linux-meta 2022-06-29T15:34:05.969 INFO:teuthology.orchestra.run.smithi047.stdout:Version: 5.4.0.121.122 2022-06-29T15:34:05.969 INFO:teuthology.orchestra.run.smithi047.stdout:Provides: virtualbox-guest-modules (= 6.1.32-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.14) 2022-06-29T15:34:05.970 INFO:teuthology.orchestra.run.smithi047.stdout:Depends: linux-image-5.4.0-121-generic, linux-modules-extra-5.4.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2022-06-29T15:34:05.970 INFO:teuthology.orchestra.run.smithi047.stdout:Recommends: thermald 2022-06-29T15:34:05.970 INFO:teuthology.orchestra.run.smithi047.stdout:Description: Generic Linux kernel image 2022-06-29T15:34:05.971 INFO:teuthology.orchestra.run.smithi047.stdout: This package will always depend on the latest generic kernel image 2022-06-29T15:34:05.971 INFO:teuthology.orchestra.run.smithi047.stdout: available. 2022-06-29T15:34:05.972 DEBUG:teuthology.orchestra.run.smithi047:> mktemp 2022-06-29T15:34:06.017 INFO:teuthology.orchestra.run.smithi047.stdout:/tmp/tmp.7cLsRvZ6qo 2022-06-29T15:34:06.018 DEBUG:teuthology.orchestra.run.smithi047:> sudo cp /boot/grub/grub.cfg /tmp/tmp.7cLsRvZ6qo 2022-06-29T15:34:06.033 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0666 /tmp/tmp.7cLsRvZ6qo 2022-06-29T15:34:06.099 INFO:teuthology.orchestra.run.smithi099.stdout:Reading package lists... 2022-06-29T15:34:06.110 DEBUG:teuthology.orchestra.remote:smithi047:/tmp/tmp.7cLsRvZ6qo is 9KB 2022-06-29T15:34:06.121 DEBUG:teuthology.orchestra.run.smithi099:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2022-06-29T15:34:06.126 DEBUG:teuthology.orchestra.run.smithi047:> rm -fr /tmp/tmp.7cLsRvZ6qo 2022-06-29T15:34:06.133 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2022-06-29T15:34:06.141 DEBUG:teuthology.orchestra.run.smithi164:> dpkg -s linux-image-generic 2022-06-29T15:34:06.173 INFO:teuthology.orchestra.run.smithi164.stdout:Package: linux-image-generic 2022-06-29T15:34:06.173 INFO:teuthology.orchestra.run.smithi164.stdout:Status: install ok installed 2022-06-29T15:34:06.174 INFO:teuthology.orchestra.run.smithi164.stdout:Priority: optional 2022-06-29T15:34:06.174 INFO:teuthology.orchestra.run.smithi164.stdout:Section: kernel 2022-06-29T15:34:06.174 INFO:teuthology.orchestra.run.smithi164.stdout:Installed-Size: 19 2022-06-29T15:34:06.174 INFO:teuthology.orchestra.run.smithi164.stdout:Maintainer: Ubuntu Kernel Team 2022-06-29T15:34:06.175 INFO:teuthology.orchestra.run.smithi164.stdout:Architecture: amd64 2022-06-29T15:34:06.175 INFO:teuthology.orchestra.run.smithi164.stdout:Source: linux-meta 2022-06-29T15:34:06.175 INFO:teuthology.orchestra.run.smithi164.stdout:Version: 5.4.0.121.122 2022-06-29T15:34:06.175 INFO:teuthology.orchestra.run.smithi164.stdout:Provides: virtualbox-guest-modules (= 6.1.32-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.14) 2022-06-29T15:34:06.176 INFO:teuthology.orchestra.run.smithi164.stdout:Depends: linux-image-5.4.0-121-generic, linux-modules-extra-5.4.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2022-06-29T15:34:06.176 INFO:teuthology.orchestra.run.smithi164.stdout:Recommends: thermald 2022-06-29T15:34:06.176 INFO:teuthology.orchestra.run.smithi164.stdout:Description: Generic Linux kernel image 2022-06-29T15:34:06.176 INFO:teuthology.orchestra.run.smithi164.stdout: This package will always depend on the latest generic kernel image 2022-06-29T15:34:06.176 INFO:teuthology.orchestra.run.smithi164.stdout: available. 2022-06-29T15:34:06.177 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-120-generic Expected: 5.4.0-121-generic 2022-06-29T15:34:06.177 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2022-06-29T15:34:06.177 DEBUG:teuthology.task.kernel:src is distro, skipping download 2022-06-29T15:34:06.177 INFO:teuthology.task.kernel:Installing distro kernel on client.1... 2022-06-29T15:34:06.178 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi164.front.sepia.ceph.com, path=None, version=distro) 2022-06-29T15:34:06.178 DEBUG:teuthology.orchestra.run.smithi164:> sudo apt-get clean 2022-06-29T15:34:06.190 INFO:teuthology.orchestra.run.smithi099.stdout:Reading package lists... 2022-06-29T15:34:06.192 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2022-06-29T15:34:06.192 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/grub.d/01_ceph_kernel 2022-06-29T15:34:06.193 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2022-06-29T15:34:06.229 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-121-generic 2022-06-29T15:34:06.229 DEBUG:teuthology.orchestra.run.smithi047:> sudo update-grub 2022-06-29T15:34:06.233 DEBUG:teuthology.orchestra.run.smithi164:> sudo apt-get update 2022-06-29T15:34:06.367 INFO:teuthology.orchestra.run.smithi099.stdout:Building dependency tree... 2022-06-29T15:34:06.367 INFO:teuthology.orchestra.run.smithi099.stdout:Reading state information... 2022-06-29T15:34:06.493 INFO:teuthology.orchestra.run.smithi164.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2022-06-29T15:34:06.493 INFO:teuthology.orchestra.run.smithi164.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2022-06-29T15:34:06.557 INFO:teuthology.orchestra.run.smithi099.stdout:linux-image-generic is already the newest version (5.4.0.121.122). 2022-06-29T15:34:06.557 INFO:teuthology.orchestra.run.smithi099.stdout:0 upgraded, 0 newly installed, 0 to remove and 17 not upgraded. 2022-06-29T15:34:06.559 DEBUG:teuthology.orchestra.run.smithi099:> dpkg -s linux-image-generic 2022-06-29T15:34:06.562 INFO:teuthology.orchestra.run.smithi164.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2022-06-29T15:34:06.577 INFO:teuthology.orchestra.run.smithi099.stdout:Package: linux-image-generic 2022-06-29T15:34:06.578 INFO:teuthology.orchestra.run.smithi099.stdout:Status: install ok installed 2022-06-29T15:34:06.578 INFO:teuthology.orchestra.run.smithi099.stdout:Priority: optional 2022-06-29T15:34:06.578 INFO:teuthology.orchestra.run.smithi099.stdout:Section: kernel 2022-06-29T15:34:06.578 INFO:teuthology.orchestra.run.smithi099.stdout:Installed-Size: 19 2022-06-29T15:34:06.579 INFO:teuthology.orchestra.run.smithi099.stdout:Maintainer: Ubuntu Kernel Team 2022-06-29T15:34:06.579 INFO:teuthology.orchestra.run.smithi099.stdout:Architecture: amd64 2022-06-29T15:34:06.579 INFO:teuthology.orchestra.run.smithi099.stdout:Source: linux-meta 2022-06-29T15:34:06.579 INFO:teuthology.orchestra.run.smithi099.stdout:Version: 5.4.0.121.122 2022-06-29T15:34:06.579 INFO:teuthology.orchestra.run.smithi099.stdout:Provides: virtualbox-guest-modules (= 6.1.32-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.14) 2022-06-29T15:34:06.580 INFO:teuthology.orchestra.run.smithi099.stdout:Depends: linux-image-5.4.0-121-generic, linux-modules-extra-5.4.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2022-06-29T15:34:06.580 INFO:teuthology.orchestra.run.smithi099.stdout:Recommends: thermald 2022-06-29T15:34:06.580 INFO:teuthology.orchestra.run.smithi099.stdout:Description: Generic Linux kernel image 2022-06-29T15:34:06.580 INFO:teuthology.orchestra.run.smithi099.stdout: This package will always depend on the latest generic kernel image 2022-06-29T15:34:06.581 INFO:teuthology.orchestra.run.smithi099.stdout: available. 2022-06-29T15:34:06.581 DEBUG:teuthology.orchestra.run.smithi099:> mktemp 2022-06-29T15:34:06.585 INFO:teuthology.orchestra.run.smithi099.stdout:/tmp/tmp.d3L75s9EU2 2022-06-29T15:34:06.586 DEBUG:teuthology.orchestra.run.smithi099:> sudo cp /boot/grub/grub.cfg /tmp/tmp.d3L75s9EU2 2022-06-29T15:34:06.587 INFO:teuthology.orchestra.run.smithi064.stdout:done 2022-06-29T15:34:06.643 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 0666 /tmp/tmp.d3L75s9EU2 2022-06-29T15:34:06.651 INFO:teuthology.orchestra.run.smithi164.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2022-06-29T15:34:06.683 DEBUG:teuthology.orchestra.remote:smithi099:/tmp/tmp.d3L75s9EU2 is 9KB 2022-06-29T15:34:06.698 DEBUG:teuthology.orchestra.run.smithi099:> rm -fr /tmp/tmp.d3L75s9EU2 2022-06-29T15:34:06.704 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2022-06-29T15:34:06.761 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2022-06-29T15:34:06.762 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/grub.d/01_ceph_kernel 2022-06-29T15:34:06.762 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2022-06-29T15:34:06.789 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-121-generic 2022-06-29T15:34:06.789 DEBUG:teuthology.orchestra.run.smithi099:> sudo update-grub 2022-06-29T15:34:06.991 DEBUG:teuthology.orchestra.run.smithi064:> dpkg -s linux-image-generic 2022-06-29T15:34:07.023 INFO:teuthology.orchestra.run.smithi064.stdout:Package: linux-image-generic 2022-06-29T15:34:07.023 INFO:teuthology.orchestra.run.smithi064.stdout:Status: install ok installed 2022-06-29T15:34:07.024 INFO:teuthology.orchestra.run.smithi064.stdout:Priority: optional 2022-06-29T15:34:07.024 INFO:teuthology.orchestra.run.smithi064.stdout:Section: kernel 2022-06-29T15:34:07.024 INFO:teuthology.orchestra.run.smithi064.stdout:Installed-Size: 19 2022-06-29T15:34:07.024 INFO:teuthology.orchestra.run.smithi064.stdout:Maintainer: Ubuntu Kernel Team 2022-06-29T15:34:07.025 INFO:teuthology.orchestra.run.smithi064.stdout:Architecture: amd64 2022-06-29T15:34:07.025 INFO:teuthology.orchestra.run.smithi064.stdout:Source: linux-meta 2022-06-29T15:34:07.025 INFO:teuthology.orchestra.run.smithi064.stdout:Version: 5.4.0.121.122 2022-06-29T15:34:07.025 INFO:teuthology.orchestra.run.smithi064.stdout:Provides: virtualbox-guest-modules (= 6.1.32-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.14) 2022-06-29T15:34:07.025 INFO:teuthology.orchestra.run.smithi064.stdout:Depends: linux-image-5.4.0-121-generic, linux-modules-extra-5.4.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2022-06-29T15:34:07.026 INFO:teuthology.orchestra.run.smithi064.stdout:Recommends: thermald 2022-06-29T15:34:07.026 INFO:teuthology.orchestra.run.smithi064.stdout:Description: Generic Linux kernel image 2022-06-29T15:34:07.026 INFO:teuthology.orchestra.run.smithi064.stdout: This package will always depend on the latest generic kernel image 2022-06-29T15:34:07.026 INFO:teuthology.orchestra.run.smithi064.stdout: available. 2022-06-29T15:34:07.027 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-120-generic Expected: 5.4.0-121-generic 2022-06-29T15:34:07.027 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2022-06-29T15:34:07.027 DEBUG:teuthology.task.kernel:src is distro, skipping download 2022-06-29T15:34:07.028 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2022-06-29T15:34:07.028 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi064.front.sepia.ceph.com, path=None, version=distro) 2022-06-29T15:34:07.028 DEBUG:teuthology.orchestra.run.smithi064:> sudo apt-get clean 2022-06-29T15:34:07.148 DEBUG:teuthology.orchestra.run.smithi064:> sudo apt-get update 2022-06-29T15:34:07.331 INFO:teuthology.orchestra.run.smithi064.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2022-06-29T15:34:07.353 INFO:teuthology.orchestra.run.smithi064.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2022-06-29T15:34:07.388 INFO:teuthology.orchestra.run.smithi064.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2022-06-29T15:34:07.407 INFO:teuthology.orchestra.run.smithi064.stdout:Hit:4 http://security.ubuntu.com/ubuntu focal-security InRelease 2022-06-29T15:34:07.519 INFO:teuthology.orchestra.run.smithi047.stderr:Sourcing file `/etc/default/grub' 2022-06-29T15:34:07.523 INFO:teuthology.orchestra.run.smithi047.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2022-06-29T15:34:07.533 INFO:teuthology.orchestra.run.smithi047.stderr:Generating grub configuration file ... 2022-06-29T15:34:07.779 INFO:teuthology.orchestra.run.smithi047.stderr:Found linux image: /boot/vmlinuz-5.4.0-121-generic 2022-06-29T15:34:07.791 INFO:teuthology.orchestra.run.smithi047.stderr:Found initrd image: /boot/initrd.img-5.4.0-121-generic 2022-06-29T15:34:08.081 INFO:teuthology.orchestra.run.smithi164.stdout:Reading package lists... 2022-06-29T15:34:08.103 DEBUG:teuthology.orchestra.run.smithi164:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2022-06-29T15:34:08.171 INFO:teuthology.orchestra.run.smithi164.stdout:Reading package lists... 2022-06-29T15:34:08.179 INFO:teuthology.orchestra.run.smithi047.stderr:Found linux image: /boot/vmlinuz-5.4.0-120-generic 2022-06-29T15:34:08.191 INFO:teuthology.orchestra.run.smithi047.stderr:Found initrd image: /boot/initrd.img-5.4.0-120-generic 2022-06-29T15:34:08.205 INFO:teuthology.orchestra.run.smithi099.stderr:Sourcing file `/etc/default/grub' 2022-06-29T15:34:08.206 INFO:teuthology.orchestra.run.smithi099.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2022-06-29T15:34:08.218 INFO:teuthology.orchestra.run.smithi099.stderr:Generating grub configuration file ... 2022-06-29T15:34:08.347 INFO:teuthology.orchestra.run.smithi164.stdout:Building dependency tree... 2022-06-29T15:34:08.348 INFO:teuthology.orchestra.run.smithi164.stdout:Reading state information... 2022-06-29T15:34:08.496 INFO:teuthology.orchestra.run.smithi099.stderr:Found linux image: /boot/vmlinuz-5.4.0-121-generic 2022-06-29T15:34:08.507 INFO:teuthology.orchestra.run.smithi099.stderr:Found initrd image: /boot/initrd.img-5.4.0-121-generic 2022-06-29T15:34:08.530 INFO:teuthology.orchestra.run.smithi164.stdout:linux-image-generic is already the newest version (5.4.0.121.122). 2022-06-29T15:34:08.531 INFO:teuthology.orchestra.run.smithi164.stdout:0 upgraded, 0 newly installed, 0 to remove and 17 not upgraded. 2022-06-29T15:34:08.532 DEBUG:teuthology.orchestra.run.smithi164:> dpkg -s linux-image-generic 2022-06-29T15:34:08.555 INFO:teuthology.orchestra.run.smithi164.stdout:Package: linux-image-generic 2022-06-29T15:34:08.555 INFO:teuthology.orchestra.run.smithi164.stdout:Status: install ok installed 2022-06-29T15:34:08.555 INFO:teuthology.orchestra.run.smithi164.stdout:Priority: optional 2022-06-29T15:34:08.556 INFO:teuthology.orchestra.run.smithi164.stdout:Section: kernel 2022-06-29T15:34:08.556 INFO:teuthology.orchestra.run.smithi164.stdout:Installed-Size: 19 2022-06-29T15:34:08.556 INFO:teuthology.orchestra.run.smithi164.stdout:Maintainer: Ubuntu Kernel Team 2022-06-29T15:34:08.556 INFO:teuthology.orchestra.run.smithi164.stdout:Architecture: amd64 2022-06-29T15:34:08.556 INFO:teuthology.orchestra.run.smithi164.stdout:Source: linux-meta 2022-06-29T15:34:08.557 INFO:teuthology.orchestra.run.smithi164.stdout:Version: 5.4.0.121.122 2022-06-29T15:34:08.557 INFO:teuthology.orchestra.run.smithi164.stdout:Provides: virtualbox-guest-modules (= 6.1.32-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.14) 2022-06-29T15:34:08.557 INFO:teuthology.orchestra.run.smithi164.stdout:Depends: linux-image-5.4.0-121-generic, linux-modules-extra-5.4.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2022-06-29T15:34:08.557 INFO:teuthology.orchestra.run.smithi164.stdout:Recommends: thermald 2022-06-29T15:34:08.558 INFO:teuthology.orchestra.run.smithi164.stdout:Description: Generic Linux kernel image 2022-06-29T15:34:08.558 INFO:teuthology.orchestra.run.smithi164.stdout: This package will always depend on the latest generic kernel image 2022-06-29T15:34:08.558 INFO:teuthology.orchestra.run.smithi164.stdout: available. 2022-06-29T15:34:08.559 DEBUG:teuthology.orchestra.run.smithi164:> mktemp 2022-06-29T15:34:08.564 INFO:teuthology.orchestra.run.smithi164.stdout:/tmp/tmp.e6X9GxUgGA 2022-06-29T15:34:08.565 DEBUG:teuthology.orchestra.run.smithi164:> sudo cp /boot/grub/grub.cfg /tmp/tmp.e6X9GxUgGA 2022-06-29T15:34:08.624 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 0666 /tmp/tmp.e6X9GxUgGA 2022-06-29T15:34:08.653 DEBUG:teuthology.orchestra.remote:smithi164:/tmp/tmp.e6X9GxUgGA is 9KB 2022-06-29T15:34:08.667 DEBUG:teuthology.orchestra.run.smithi164:> rm -fr /tmp/tmp.e6X9GxUgGA 2022-06-29T15:34:08.673 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2022-06-29T15:34:08.731 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2022-06-29T15:34:08.731 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/grub.d/01_ceph_kernel 2022-06-29T15:34:08.732 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2022-06-29T15:34:08.780 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-121-generic 2022-06-29T15:34:08.781 DEBUG:teuthology.orchestra.run.smithi164:> sudo update-grub 2022-06-29T15:34:08.881 INFO:teuthology.orchestra.run.smithi099.stderr:Found linux image: /boot/vmlinuz-5.4.0-120-generic 2022-06-29T15:34:08.893 INFO:teuthology.orchestra.run.smithi099.stderr:Found initrd image: /boot/initrd.img-5.4.0-120-generic 2022-06-29T15:34:08.964 INFO:teuthology.orchestra.run.smithi047.stderr:done 2022-06-29T15:34:08.965 INFO:teuthology.orchestra.run.smithi064.stdout:Reading package lists... 2022-06-29T15:34:08.967 DEBUG:teuthology.orchestra.run.smithi047:> sudo shutdown -r now 2022-06-29T15:34:08.987 DEBUG:teuthology.orchestra.run.smithi064:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2022-06-29T15:34:09.060 INFO:teuthology.orchestra.run.smithi064.stdout:Reading package lists... 2022-06-29T15:34:09.239 INFO:teuthology.orchestra.run.smithi064.stdout:Building dependency tree... 2022-06-29T15:34:09.240 INFO:teuthology.orchestra.run.smithi064.stdout:Reading state information... 2022-06-29T15:34:09.428 INFO:teuthology.orchestra.run.smithi064.stdout:linux-image-generic is already the newest version (5.4.0.121.122). 2022-06-29T15:34:09.429 INFO:teuthology.orchestra.run.smithi064.stdout:0 upgraded, 0 newly installed, 0 to remove and 17 not upgraded. 2022-06-29T15:34:09.430 DEBUG:teuthology.orchestra.run.smithi064:> dpkg -s linux-image-generic 2022-06-29T15:34:09.451 INFO:teuthology.orchestra.run.smithi064.stdout:Package: linux-image-generic 2022-06-29T15:34:09.451 INFO:teuthology.orchestra.run.smithi064.stdout:Status: install ok installed 2022-06-29T15:34:09.451 INFO:teuthology.orchestra.run.smithi064.stdout:Priority: optional 2022-06-29T15:34:09.451 INFO:teuthology.orchestra.run.smithi064.stdout:Section: kernel 2022-06-29T15:34:09.452 INFO:teuthology.orchestra.run.smithi064.stdout:Installed-Size: 19 2022-06-29T15:34:09.452 INFO:teuthology.orchestra.run.smithi064.stdout:Maintainer: Ubuntu Kernel Team 2022-06-29T15:34:09.452 INFO:teuthology.orchestra.run.smithi064.stdout:Architecture: amd64 2022-06-29T15:34:09.452 INFO:teuthology.orchestra.run.smithi064.stdout:Source: linux-meta 2022-06-29T15:34:09.452 INFO:teuthology.orchestra.run.smithi064.stdout:Version: 5.4.0.121.122 2022-06-29T15:34:09.453 INFO:teuthology.orchestra.run.smithi064.stdout:Provides: virtualbox-guest-modules (= 6.1.32-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.14) 2022-06-29T15:34:09.453 INFO:teuthology.orchestra.run.smithi064.stdout:Depends: linux-image-5.4.0-121-generic, linux-modules-extra-5.4.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2022-06-29T15:34:09.453 INFO:teuthology.orchestra.run.smithi064.stdout:Recommends: thermald 2022-06-29T15:34:09.453 INFO:teuthology.orchestra.run.smithi064.stdout:Description: Generic Linux kernel image 2022-06-29T15:34:09.454 INFO:teuthology.orchestra.run.smithi064.stdout: This package will always depend on the latest generic kernel image 2022-06-29T15:34:09.454 INFO:teuthology.orchestra.run.smithi064.stdout: available. 2022-06-29T15:34:09.454 DEBUG:teuthology.orchestra.run.smithi064:> mktemp 2022-06-29T15:34:09.459 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/tmp.HZInbQjCvu 2022-06-29T15:34:09.460 DEBUG:teuthology.orchestra.run.smithi064:> sudo cp /boot/grub/grub.cfg /tmp/tmp.HZInbQjCvu 2022-06-29T15:34:09.523 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0666 /tmp/tmp.HZInbQjCvu 2022-06-29T15:34:09.568 DEBUG:teuthology.orchestra.remote:smithi064:/tmp/tmp.HZInbQjCvu is 9KB 2022-06-29T15:34:09.582 DEBUG:teuthology.orchestra.run.smithi064:> rm -fr /tmp/tmp.HZInbQjCvu 2022-06-29T15:34:09.589 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2022-06-29T15:34:09.645 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2022-06-29T15:34:09.646 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/grub.d/01_ceph_kernel 2022-06-29T15:34:09.646 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2022-06-29T15:34:09.676 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-121-generic 2022-06-29T15:34:09.676 DEBUG:teuthology.orchestra.run.smithi064:> sudo update-grub 2022-06-29T15:34:10.175 INFO:teuthology.orchestra.run.smithi164.stderr:Sourcing file `/etc/default/grub' 2022-06-29T15:34:10.176 INFO:teuthology.orchestra.run.smithi164.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2022-06-29T15:34:10.184 INFO:teuthology.orchestra.run.smithi164.stderr:Generating grub configuration file ... 2022-06-29T15:34:10.274 INFO:teuthology.orchestra.run.smithi099.stderr:done 2022-06-29T15:34:10.276 DEBUG:teuthology.orchestra.run.smithi099:> sudo shutdown -r now 2022-06-29T15:34:10.507 INFO:teuthology.orchestra.run.smithi164.stderr:Found linux image: /boot/vmlinuz-5.4.0-121-generic 2022-06-29T15:34:10.519 INFO:teuthology.orchestra.run.smithi164.stderr:Found initrd image: /boot/initrd.img-5.4.0-121-generic 2022-06-29T15:34:10.972 INFO:teuthology.orchestra.run.smithi164.stderr:Found linux image: /boot/vmlinuz-5.4.0-120-generic 2022-06-29T15:34:10.984 INFO:teuthology.orchestra.run.smithi164.stderr:Found initrd image: /boot/initrd.img-5.4.0-120-generic 2022-06-29T15:34:11.073 INFO:teuthology.orchestra.run.smithi064.stderr:Sourcing file `/etc/default/grub' 2022-06-29T15:34:11.075 INFO:teuthology.orchestra.run.smithi064.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2022-06-29T15:34:11.082 INFO:teuthology.orchestra.run.smithi064.stderr:Generating grub configuration file ... 2022-06-29T15:34:11.320 INFO:teuthology.orchestra.run.smithi064.stderr:Found linux image: /boot/vmlinuz-5.4.0-121-generic 2022-06-29T15:34:11.332 INFO:teuthology.orchestra.run.smithi064.stderr:Found initrd image: /boot/initrd.img-5.4.0-121-generic 2022-06-29T15:34:11.678 INFO:teuthology.orchestra.run.smithi064.stderr:Found linux image: /boot/vmlinuz-5.4.0-120-generic 2022-06-29T15:34:11.689 INFO:teuthology.orchestra.run.smithi064.stderr:Found initrd image: /boot/initrd.img-5.4.0-120-generic 2022-06-29T15:34:11.720 INFO:teuthology.orchestra.run.smithi164.stderr:done 2022-06-29T15:34:11.722 DEBUG:teuthology.orchestra.run.smithi164:> sudo shutdown -r now 2022-06-29T15:34:13.297 INFO:teuthology.orchestra.run.smithi064.stderr:done 2022-06-29T15:34:13.299 DEBUG:teuthology.orchestra.run.smithi064:> sudo shutdown -r now 2022-06-29T15:34:38.996 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2022-06-29T15:34:38.996 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2022-06-29T15:34:38.997 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-06-29T15:34:40.281 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2022-06-29T15:34:40.282 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2022-06-29T15:34:40.283 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-06-29T15:34:41.727 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2022-06-29T15:34:41.728 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2022-06-29T15:34:41.728 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-06-29T15:34:43.304 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2022-06-29T15:34:43.305 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2022-06-29T15:34:43.305 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-06-29T15:34:57.530 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.47 2022-06-29T15:34:58.810 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.99 2022-06-29T15:35:00.090 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.164 2022-06-29T15:35:01.626 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.64 2022-06-29T15:35:27.547 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2022-06-29T15:35:27.548 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-06-29T15:35:28.813 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2022-06-29T15:35:28.814 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-06-29T15:35:30.092 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2022-06-29T15:35:30.093 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-06-29T15:35:31.629 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2022-06-29T15:35:31.630 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-06-29T15:35:43.049 DEBUG:teuthology.orchestra.run.smithi047:> true 2022-06-29T15:35:43.642 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi047.front.sepia.ceph.com' 2022-06-29T15:35:43.642 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.4.0-121-generic"... 2022-06-29T15:35:43.643 DEBUG:teuthology.orchestra.run.smithi047:> uname -r 2022-06-29T15:35:43.688 INFO:teuthology.orchestra.run.smithi047.stdout:5.4.0-121-generic 2022-06-29T15:35:43.689 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-121-generic vs 5.4.0-121-generic 2022-06-29T15:35:43.689 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2022-06-29T15:35:43.690 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2022-06-29T15:35:44.690 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2022-06-29T15:35:44.691 DEBUG:teuthology.orchestra.run.smithi047:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2022-06-29T15:35:44.770 INFO:teuthology.orchestra.run.smithi047.stdout:ttyS1 2022-06-29T15:35:44.797 DEBUG:teuthology.parallel:result is None 2022-06-29T15:35:47.198 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.99 2022-06-29T15:35:48.474 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.164 2022-06-29T15:35:50.010 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.64 2022-06-29T15:36:17.200 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2022-06-29T15:36:17.202 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-06-29T15:36:17.434 DEBUG:teuthology.orchestra.run.smithi099:> true 2022-06-29T15:36:17.991 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi099.front.sepia.ceph.com' 2022-06-29T15:36:17.991 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.4.0-121-generic"... 2022-06-29T15:36:17.992 DEBUG:teuthology.orchestra.run.smithi099:> uname -r 2022-06-29T15:36:17.998 INFO:teuthology.orchestra.run.smithi099.stdout:5.4.0-121-generic 2022-06-29T15:36:17.999 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-121-generic vs 5.4.0-121-generic 2022-06-29T15:36:17.999 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2022-06-29T15:36:17.999 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2022-06-29T15:36:18.475 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2022-06-29T15:36:18.485 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-06-29T15:36:18.793 DEBUG:teuthology.orchestra.run.smithi164:> true 2022-06-29T15:36:19.000 INFO:teuthology.task.kernel:Enabling kdb on client.0... 2022-06-29T15:36:19.001 DEBUG:teuthology.orchestra.run.smithi099:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2022-06-29T15:36:19.064 INFO:teuthology.orchestra.run.smithi099.stdout:ttyS1 2022-06-29T15:36:19.091 DEBUG:teuthology.parallel:result is None 2022-06-29T15:36:19.323 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi164.front.sepia.ceph.com' 2022-06-29T15:36:19.323 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "5.4.0-121-generic"... 2022-06-29T15:36:19.323 DEBUG:teuthology.orchestra.run.smithi164:> uname -r 2022-06-29T15:36:19.373 INFO:teuthology.orchestra.run.smithi164.stdout:5.4.0-121-generic 2022-06-29T15:36:19.373 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-121-generic vs 5.4.0-121-generic 2022-06-29T15:36:19.373 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2022-06-29T15:36:19.374 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2022-06-29T15:36:20.011 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2022-06-29T15:36:20.012 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-06-29T15:36:20.343 DEBUG:teuthology.orchestra.run.smithi064:> true 2022-06-29T15:36:20.374 INFO:teuthology.task.kernel:Enabling kdb on client.1... 2022-06-29T15:36:20.375 DEBUG:teuthology.orchestra.run.smithi164:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2022-06-29T15:36:20.463 INFO:teuthology.orchestra.run.smithi164.stdout:ttyS1 2022-06-29T15:36:20.485 DEBUG:teuthology.parallel:result is None 2022-06-29T15:36:20.902 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi064.front.sepia.ceph.com' 2022-06-29T15:36:20.903 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.4.0-121-generic"... 2022-06-29T15:36:20.903 DEBUG:teuthology.orchestra.run.smithi064:> uname -r 2022-06-29T15:36:20.910 INFO:teuthology.orchestra.run.smithi064.stdout:5.4.0-121-generic 2022-06-29T15:36:20.910 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-121-generic vs 5.4.0-121-generic 2022-06-29T15:36:20.910 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2022-06-29T15:36:20.911 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2022-06-29T15:36:21.911 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2022-06-29T15:36:21.912 DEBUG:teuthology.orchestra.run.smithi064:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2022-06-29T15:36:21.996 INFO:teuthology.orchestra.run.smithi064.stdout:ttyS1 2022-06-29T15:36:22.031 DEBUG:teuthology.parallel:result is None 2022-06-29T15:36:22.032 INFO:teuthology.run_tasks:Running task internal.base... 2022-06-29T15:36:22.044 INFO:teuthology.task.internal:Creating test directory... 2022-06-29T15:36:22.046 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2022-06-29T15:36:22.050 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2022-06-29T15:36:22.053 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2022-06-29T15:36:22.056 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2022-06-29T15:36:22.065 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2022-06-29T15:36:22.081 INFO:teuthology.run_tasks:Running task internal.archive... 2022-06-29T15:36:22.131 INFO:teuthology.task.internal:Creating archive directory... 2022-06-29T15:36:22.132 DEBUG:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2022-06-29T15:36:22.135 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2022-06-29T15:36:22.137 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2022-06-29T15:36:22.140 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2022-06-29T15:36:22.173 INFO:teuthology.run_tasks:Running task internal.coredump... 2022-06-29T15:36:22.183 INFO:teuthology.task.internal:Enabling coredump saving... 2022-06-29T15:36:22.184 DEBUG:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2022-06-29T15:36:22.186 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2022-06-29T15:36:22.202 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2022-06-29T15:36:22.205 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2022-06-29T15:36:22.210 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-06-29T15:36:22.222 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-06-29T15:36:22.224 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-06-29T15:36:22.226 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-06-29T15:36:22.227 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-06-29T15:36:22.232 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-06-29T15:36:22.235 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-06-29T15:36:22.237 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-06-29T15:36:22.239 INFO:teuthology.run_tasks:Running task internal.sudo... 2022-06-29T15:36:22.250 INFO:teuthology.task.internal:Configuring sudo... 2022-06-29T15:36:22.251 DEBUG:teuthology.orchestra.run.smithi047:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2022-06-29T15:36:22.271 DEBUG:teuthology.orchestra.run.smithi064:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2022-06-29T15:36:22.274 DEBUG:teuthology.orchestra.run.smithi099:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2022-06-29T15:36:22.278 DEBUG:teuthology.orchestra.run.smithi164:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2022-06-29T15:36:22.294 INFO:teuthology.run_tasks:Running task internal.syslog... 2022-06-29T15:36:22.304 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2022-06-29T15:36:22.304 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2022-06-29T15:36:22.307 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2022-06-29T15:36:22.310 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2022-06-29T15:36:22.312 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2022-06-29T15:36:22.319 DEBUG:teuthology.orchestra.run.smithi047:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2022-06-29T15:36:22.327 DEBUG:teuthology.orchestra.run.smithi047:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2022-06-29T15:36:22.376 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2022-06-29T15:36:22.377 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2022-06-29T15:36:22.400 DEBUG:teuthology.orchestra.run.smithi064:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2022-06-29T15:36:22.408 DEBUG:teuthology.orchestra.run.smithi064:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2022-06-29T15:36:22.415 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2022-06-29T15:36:22.416 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2022-06-29T15:36:22.441 DEBUG:teuthology.orchestra.run.smithi099:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2022-06-29T15:36:22.448 DEBUG:teuthology.orchestra.run.smithi099:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2022-06-29T15:36:22.456 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2022-06-29T15:36:22.456 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2022-06-29T15:36:22.484 DEBUG:teuthology.orchestra.run.smithi164:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2022-06-29T15:36:22.493 DEBUG:teuthology.orchestra.run.smithi164:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2022-06-29T15:36:22.501 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2022-06-29T15:36:22.501 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2022-06-29T15:36:22.542 DEBUG:teuthology.orchestra.run.smithi047:> sudo service rsyslog restart 2022-06-29T15:36:22.545 DEBUG:teuthology.orchestra.run.smithi064:> sudo service rsyslog restart 2022-06-29T15:36:22.547 DEBUG:teuthology.orchestra.run.smithi099:> sudo service rsyslog restart 2022-06-29T15:36:22.549 DEBUG:teuthology.orchestra.run.smithi164:> sudo service rsyslog restart 2022-06-29T15:36:22.632 INFO:teuthology.run_tasks:Running task internal.timer... 2022-06-29T15:36:22.641 INFO:teuthology.task.internal:Starting timer... 2022-06-29T15:36:22.641 INFO:teuthology.run_tasks:Running task pcp... 2022-06-29T15:36:22.686 INFO:teuthology.run_tasks:Running task selinux... 2022-06-29T15:36:22.735 DEBUG:teuthology.task.selinux:Excluding smithi047: OS 'ubuntu' does not support SELinux 2022-06-29T15:36:22.736 DEBUG:teuthology.task.selinux:Excluding smithi064: OS 'ubuntu' does not support SELinux 2022-06-29T15:36:22.736 DEBUG:teuthology.task.selinux:Excluding smithi099: OS 'ubuntu' does not support SELinux 2022-06-29T15:36:22.736 DEBUG:teuthology.task.selinux:Excluding smithi164: OS 'ubuntu' does not support SELinux 2022-06-29T15:36:22.737 DEBUG:teuthology.task.selinux:Getting current SELinux state 2022-06-29T15:36:22.737 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2022-06-29T15:36:22.737 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2022-06-29T15:36:22.738 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2022-06-29T15:36:22.785 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2022-06-29T15:36:22.786 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_main to origin/main 2022-06-29T15:36:22.806 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2022-06-29T15:36:22.808 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi047.front.sepia.ceph.com,smithi064.front.sepia.ceph.com,smithi099.front.sepia.ceph.com,smithi164.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_main/cephlab.yml 2022-06-29T15:39:18.062 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi047.front.sepia.ceph.com'), Remote(name='ubuntu@smithi064.front.sepia.ceph.com'), Remote(name='ubuntu@smithi099.front.sepia.ceph.com'), Remote(name='ubuntu@smithi164.front.sepia.ceph.com')] 2022-06-29T15:39:18.065 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2022-06-29T15:39:18.067 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-06-29T15:39:18.169 DEBUG:teuthology.orchestra.run.smithi047:> true 2022-06-29T15:39:18.289 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2022-06-29T15:39:18.290 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-06-29T15:39:18.391 DEBUG:teuthology.orchestra.run.smithi064:> true 2022-06-29T15:39:18.516 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2022-06-29T15:39:18.518 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-06-29T15:39:18.618 DEBUG:teuthology.orchestra.run.smithi099:> true 2022-06-29T15:39:18.740 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2022-06-29T15:39:18.741 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-06-29T15:39:18.839 DEBUG:teuthology.orchestra.run.smithi164:> true 2022-06-29T15:39:18.960 INFO:teuthology.run_tasks:Running task clock... 2022-06-29T15:39:18.973 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2022-06-29T15:39:18.974 INFO:teuthology.orchestra.run:Running command with timeout 360 2022-06-29T15:39:18.976 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-06-29T15:39:18.979 INFO:teuthology.orchestra.run:Running command with timeout 360 2022-06-29T15:39:18.980 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-06-29T15:39:18.982 INFO:teuthology.orchestra.run:Running command with timeout 360 2022-06-29T15:39:18.983 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-06-29T15:39:18.985 INFO:teuthology.orchestra.run:Running command with timeout 360 2022-06-29T15:39:18.986 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-06-29T15:39:19.009 INFO:teuthology.orchestra.run.smithi047.stdout:29 Jun 15:39:19 ntpd[7945]: ntpd 4.2.8p12@1.3728-o (1): Starting 2022-06-29T15:39:19.010 INFO:teuthology.orchestra.run.smithi047.stdout:29 Jun 15:39:19 ntpd[7945]: Command line: ntpd -gq 2022-06-29T15:39:19.010 INFO:teuthology.orchestra.run.smithi047.stdout:29 Jun 15:39:19 ntpd[7945]: proto: precision = 0.235 usec (-22) 2022-06-29T15:39:19.011 INFO:teuthology.orchestra.run.smithi047.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2022-06-29T15:39:19.012 INFO:teuthology.orchestra.run.smithi047.stdout:29 Jun 15:39:19 ntpd[7945]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2022-06-29T15:39:19.012 INFO:teuthology.orchestra.run.smithi047.stderr:restrict ::: KOD does nothing without LIMITED. 2022-06-29T15:39:19.014 INFO:teuthology.orchestra.run.smithi047.stdout:29 Jun 15:39:19 ntpd[7945]: restrict ::: KOD does nothing without LIMITED. 2022-06-29T15:39:19.014 INFO:teuthology.orchestra.run.smithi047.stdout:29 Jun 15:39:19 ntpd[7945]: Listen and drop on 0 v6wildcard [::]:123 2022-06-29T15:39:19.014 INFO:teuthology.orchestra.run.smithi047.stdout:29 Jun 15:39:19 ntpd[7945]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2022-06-29T15:39:19.015 INFO:teuthology.orchestra.run.smithi047.stdout:29 Jun 15:39:19 ntpd[7945]: Listen normally on 2 lo 127.0.0.1:123 2022-06-29T15:39:19.015 INFO:teuthology.orchestra.run.smithi047.stdout:29 Jun 15:39:19 ntpd[7945]: Listen normally on 3 ens1f0 172.21.15.47:123 2022-06-29T15:39:19.015 INFO:teuthology.orchestra.run.smithi047.stdout:29 Jun 15:39:19 ntpd[7945]: Listen normally on 4 lo [::1]:123 2022-06-29T15:39:19.015 INFO:teuthology.orchestra.run.smithi047.stdout:29 Jun 15:39:19 ntpd[7945]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:148e%3]:123 2022-06-29T15:39:19.015 INFO:teuthology.orchestra.run.smithi047.stdout:29 Jun 15:39:19 ntpd[7945]: Listening on routing socket on fd #22 for interface updates 2022-06-29T15:39:19.016 INFO:teuthology.orchestra.run.smithi064.stdout:29 Jun 15:39:19 ntpd[8044]: ntpd 4.2.8p12@1.3728-o (1): Starting 2022-06-29T15:39:19.017 INFO:teuthology.orchestra.run.smithi064.stdout:29 Jun 15:39:19 ntpd[8044]: Command line: ntpd -gq 2022-06-29T15:39:19.017 INFO:teuthology.orchestra.run.smithi064.stdout:29 Jun 15:39:19 ntpd[8044]: proto: precision = 0.089 usec (-23) 2022-06-29T15:39:19.018 INFO:teuthology.orchestra.run.smithi064.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2022-06-29T15:39:19.019 INFO:teuthology.orchestra.run.smithi064.stdout:29 Jun 15:39:19 ntpd[8044]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2022-06-29T15:39:19.019 INFO:teuthology.orchestra.run.smithi064.stderr:restrict ::: KOD does nothing without LIMITED. 2022-06-29T15:39:19.020 INFO:teuthology.orchestra.run.smithi064.stdout:29 Jun 15:39:19 ntpd[8044]: restrict ::: KOD does nothing without LIMITED. 2022-06-29T15:39:19.021 INFO:teuthology.orchestra.run.smithi064.stdout:29 Jun 15:39:19 ntpd[8044]: Listen and drop on 0 v6wildcard [::]:123 2022-06-29T15:39:19.021 INFO:teuthology.orchestra.run.smithi064.stdout:29 Jun 15:39:19 ntpd[8044]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2022-06-29T15:39:19.021 INFO:teuthology.orchestra.run.smithi064.stdout:29 Jun 15:39:19 ntpd[8044]: Listen normally on 2 lo 127.0.0.1:123 2022-06-29T15:39:19.022 INFO:teuthology.orchestra.run.smithi064.stdout:29 Jun 15:39:19 ntpd[8044]: Listen normally on 3 enp3s0f1 172.21.15.64:123 2022-06-29T15:39:19.022 INFO:teuthology.orchestra.run.smithi064.stdout:29 Jun 15:39:19 ntpd[8044]: Listen normally on 4 lo [::1]:123 2022-06-29T15:39:19.022 INFO:teuthology.orchestra.run.smithi064.stdout:29 Jun 15:39:19 ntpd[8044]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:feb6:f8df%5]:123 2022-06-29T15:39:19.024 INFO:teuthology.orchestra.run.smithi064.stdout:29 Jun 15:39:19 ntpd[8044]: Listening on routing socket on fd #22 for interface updates 2022-06-29T15:39:19.024 INFO:teuthology.orchestra.run.smithi164.stdout:29 Jun 15:39:19 ntpd[8041]: ntpd 4.2.8p12@1.3728-o (1): Starting 2022-06-29T15:39:19.025 INFO:teuthology.orchestra.run.smithi164.stdout:29 Jun 15:39:19 ntpd[8041]: Command line: ntpd -gq 2022-06-29T15:39:19.025 INFO:teuthology.orchestra.run.smithi164.stdout:29 Jun 15:39:19 ntpd[8041]: proto: precision = 0.076 usec (-24) 2022-06-29T15:39:19.025 INFO:teuthology.orchestra.run.smithi099.stdout:29 Jun 15:39:19 ntpd[8053]: ntpd 4.2.8p12@1.3728-o (1): Starting 2022-06-29T15:39:19.026 INFO:teuthology.orchestra.run.smithi099.stdout:29 Jun 15:39:19 ntpd[8053]: Command line: ntpd -gq 2022-06-29T15:39:19.026 INFO:teuthology.orchestra.run.smithi099.stdout:29 Jun 15:39:19 ntpd[8053]: proto: precision = 0.076 usec (-24) 2022-06-29T15:39:19.026 INFO:teuthology.orchestra.run.smithi164.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2022-06-29T15:39:19.027 INFO:teuthology.orchestra.run.smithi099.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2022-06-29T15:39:19.027 INFO:teuthology.orchestra.run.smithi164.stdout:29 Jun 15:39:19 ntpd[8041]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2022-06-29T15:39:19.028 INFO:teuthology.orchestra.run.smithi164.stdout:29 Jun 15:39:19 ntpd[8041]: restrict ::: KOD does nothing without LIMITED. 2022-06-29T15:39:19.028 INFO:teuthology.orchestra.run.smithi099.stdout:29 Jun 15:39:19 ntpd[8053]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2022-06-29T15:39:19.029 INFO:teuthology.orchestra.run.smithi164.stderr:restrict ::: KOD does nothing without LIMITED. 2022-06-29T15:39:19.029 INFO:teuthology.orchestra.run.smithi099.stderr:restrict ::: KOD does nothing without LIMITED. 2022-06-29T15:39:19.031 INFO:teuthology.orchestra.run.smithi164.stdout:29 Jun 15:39:19 ntpd[8041]: Listen and drop on 0 v6wildcard [::]:123 2022-06-29T15:39:19.031 INFO:teuthology.orchestra.run.smithi164.stdout:29 Jun 15:39:19 ntpd[8041]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2022-06-29T15:39:19.032 INFO:teuthology.orchestra.run.smithi164.stdout:29 Jun 15:39:19 ntpd[8041]: Listen normally on 2 lo 127.0.0.1:123 2022-06-29T15:39:19.032 INFO:teuthology.orchestra.run.smithi164.stdout:29 Jun 15:39:19 ntpd[8041]: Listen normally on 3 enp3s0f1 172.21.15.164:123 2022-06-29T15:39:19.032 INFO:teuthology.orchestra.run.smithi164.stdout:29 Jun 15:39:19 ntpd[8041]: Listen normally on 4 lo [::1]:123 2022-06-29T15:39:19.032 INFO:teuthology.orchestra.run.smithi164.stdout:29 Jun 15:39:19 ntpd[8041]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:73f3%5]:123 2022-06-29T15:39:19.033 INFO:teuthology.orchestra.run.smithi164.stdout:29 Jun 15:39:19 ntpd[8041]: Listening on routing socket on fd #22 for interface updates 2022-06-29T15:39:19.033 INFO:teuthology.orchestra.run.smithi099.stdout:29 Jun 15:39:19 ntpd[8053]: restrict ::: KOD does nothing without LIMITED. 2022-06-29T15:39:19.034 INFO:teuthology.orchestra.run.smithi099.stdout:29 Jun 15:39:19 ntpd[8053]: Listen and drop on 0 v6wildcard [::]:123 2022-06-29T15:39:19.034 INFO:teuthology.orchestra.run.smithi099.stdout:29 Jun 15:39:19 ntpd[8053]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2022-06-29T15:39:19.034 INFO:teuthology.orchestra.run.smithi099.stdout:29 Jun 15:39:19 ntpd[8053]: Listen normally on 2 lo 127.0.0.1:123 2022-06-29T15:39:19.034 INFO:teuthology.orchestra.run.smithi099.stdout:29 Jun 15:39:19 ntpd[8053]: Listen normally on 3 enp3s0f1 172.21.15.99:123 2022-06-29T15:39:19.034 INFO:teuthology.orchestra.run.smithi099.stdout:29 Jun 15:39:19 ntpd[8053]: Listen normally on 4 lo [::1]:123 2022-06-29T15:39:19.035 INFO:teuthology.orchestra.run.smithi099.stdout:29 Jun 15:39:19 ntpd[8053]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d803%5]:123 2022-06-29T15:39:19.035 INFO:teuthology.orchestra.run.smithi099.stdout:29 Jun 15:39:19 ntpd[8053]: Listening on routing socket on fd #22 for interface updates 2022-06-29T15:39:20.018 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:20 ntpd[8044]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:20.019 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:20 ntpd[8044]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:20.019 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:20 ntpd[8044]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:20.020 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:20 ntpd[8044]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:20.020 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:20 ntpd[8044]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-29T15:39:20.020 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:20 ntpd[8044]: can't open /var/log/ntpstats/peerstats.20220629: Permission denied 2022-06-29T15:39:20.023 INFO:teuthology.orchestra.run.smithi164.stderr:29 Jun 15:39:20 ntpd[8041]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:20.024 INFO:teuthology.orchestra.run.smithi164.stderr:29 Jun 15:39:20 ntpd[8041]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:20.024 INFO:teuthology.orchestra.run.smithi164.stderr:29 Jun 15:39:20 ntpd[8041]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:20.024 INFO:teuthology.orchestra.run.smithi164.stderr:29 Jun 15:39:20 ntpd[8041]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:20.025 INFO:teuthology.orchestra.run.smithi164.stderr:29 Jun 15:39:20 ntpd[8041]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-29T15:39:20.025 INFO:teuthology.orchestra.run.smithi164.stderr:29 Jun 15:39:20 ntpd[8041]: can't open /var/log/ntpstats/peerstats.20220629: Permission denied 2022-06-29T15:39:20.026 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:20 ntpd[8053]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:20.026 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:20 ntpd[8053]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:20.026 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:20 ntpd[8053]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:20.027 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:20 ntpd[8053]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:20.027 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:20 ntpd[8053]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:20.027 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:20 ntpd[8053]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:20.028 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:20 ntpd[8053]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-29T15:39:20.028 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:20 ntpd[8053]: can't open /var/log/ntpstats/peerstats.20220629: Permission denied 2022-06-29T15:39:21.011 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:21 ntpd[7945]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:21.011 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:21 ntpd[7945]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:21.011 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:21 ntpd[7945]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:21.012 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:21 ntpd[7945]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:21.012 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:21 ntpd[7945]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:21.012 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:21 ntpd[7945]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:21.012 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:21 ntpd[7945]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-29T15:39:21.013 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:21 ntpd[7945]: can't open /var/log/ntpstats/peerstats.20220629: Permission denied 2022-06-29T15:39:21.013 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:21 ntpd[7945]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:21.013 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:21 ntpd[7945]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:21.013 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:21 ntpd[7945]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-29T15:39:21.013 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:21 ntpd[7945]: can't open /var/log/ntpstats/peerstats.20220629: Permission denied 2022-06-29T15:39:21.018 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:21 ntpd[8044]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:21.018 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:21 ntpd[8044]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:21.018 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:21 ntpd[8044]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:21.018 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:21 ntpd[8044]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:21.019 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:21 ntpd[8044]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:21.019 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:21 ntpd[8044]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:21.019 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:21 ntpd[8044]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-29T15:39:21.019 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:21 ntpd[8044]: can't open /var/log/ntpstats/peerstats.20220629: Permission denied 2022-06-29T15:39:21.020 INFO:teuthology.orchestra.run.smithi164.stderr:29 Jun 15:39:21 ntpd[8041]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:21.020 INFO:teuthology.orchestra.run.smithi164.stderr:29 Jun 15:39:21 ntpd[8041]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:21.021 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:21 ntpd[8053]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:21.021 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:21 ntpd[8053]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:21.022 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:21 ntpd[8053]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:21.022 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:21 ntpd[8053]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:21.022 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:21 ntpd[8053]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-29T15:39:21.022 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:21 ntpd[8053]: can't open /var/log/ntpstats/peerstats.20220629: Permission denied 2022-06-29T15:39:22.019 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:22 ntpd[8044]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:22.020 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:22 ntpd[8044]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:22.020 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:22 ntpd[8044]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:22.020 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:22 ntpd[8044]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:22.021 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:22 ntpd[8044]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:22.021 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:22 ntpd[8044]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:22.022 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:22 ntpd[8044]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-29T15:39:22.022 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:22 ntpd[8044]: can't open /var/log/ntpstats/peerstats.20220629: Permission denied 2022-06-29T15:39:22.022 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:22 ntpd[8044]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:22.023 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:22 ntpd[8044]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:22.023 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:22 ntpd[8044]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-29T15:39:22.023 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:22 ntpd[8044]: can't open /var/log/ntpstats/peerstats.20220629: Permission denied 2022-06-29T15:39:22.026 INFO:teuthology.orchestra.run.smithi164.stderr:29 Jun 15:39:22 ntpd[8041]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:22.026 INFO:teuthology.orchestra.run.smithi164.stderr:29 Jun 15:39:22 ntpd[8041]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:22.026 INFO:teuthology.orchestra.run.smithi164.stderr:29 Jun 15:39:22 ntpd[8041]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:22.027 INFO:teuthology.orchestra.run.smithi164.stderr:29 Jun 15:39:22 ntpd[8041]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:22.027 INFO:teuthology.orchestra.run.smithi164.stderr:29 Jun 15:39:22 ntpd[8041]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-29T15:39:22.027 INFO:teuthology.orchestra.run.smithi164.stderr:29 Jun 15:39:22 ntpd[8041]: can't open /var/log/ntpstats/peerstats.20220629: Permission denied 2022-06-29T15:39:22.028 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:22 ntpd[8053]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:22.028 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:22 ntpd[8053]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:22.028 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:22 ntpd[8053]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:22.029 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:22 ntpd[8053]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:22.029 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:22 ntpd[8053]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-29T15:39:22.029 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:22 ntpd[8053]: can't open /var/log/ntpstats/peerstats.20220629: Permission denied 2022-06-29T15:39:23.011 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:23 ntpd[7945]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:23.012 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:23 ntpd[7945]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:23.012 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:23 ntpd[7945]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:23.012 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:23 ntpd[7945]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:23.012 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:23 ntpd[7945]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:23.012 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:23 ntpd[7945]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:23.013 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:23 ntpd[7945]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:23.013 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:23 ntpd[7945]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:23.013 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:23 ntpd[7945]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-29T15:39:23.013 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:23 ntpd[7945]: can't open /var/log/ntpstats/peerstats.20220629: Permission denied 2022-06-29T15:39:23.014 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:23 ntpd[7945]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:23.014 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:23 ntpd[7945]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:23.014 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:23 ntpd[7945]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-29T15:39:23.014 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:23 ntpd[7945]: can't open /var/log/ntpstats/peerstats.20220629: Permission denied 2022-06-29T15:39:23.014 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:23 ntpd[7945]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:23.015 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:23 ntpd[7945]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:23.015 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:23 ntpd[7945]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-29T15:39:23.015 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:23 ntpd[7945]: can't open /var/log/ntpstats/peerstats.20220629: Permission denied 2022-06-29T15:39:23.017 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:23 ntpd[8044]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:23.018 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:23 ntpd[8044]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:23.018 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:23 ntpd[8044]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:23.018 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:23 ntpd[8044]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:23.018 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:23 ntpd[8044]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-29T15:39:23.019 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:23 ntpd[8044]: can't open /var/log/ntpstats/peerstats.20220629: Permission denied 2022-06-29T15:39:23.020 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:23 ntpd[8053]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:23.020 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:23 ntpd[8053]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:23.020 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:23 ntpd[8053]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:23.021 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:23 ntpd[8053]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:23.021 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:23 ntpd[8053]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-29T15:39:23.021 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:23 ntpd[8053]: can't open /var/log/ntpstats/peerstats.20220629: Permission denied 2022-06-29T15:39:23.022 INFO:teuthology.orchestra.run.smithi164.stderr:29 Jun 15:39:23 ntpd[8041]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:23.022 INFO:teuthology.orchestra.run.smithi164.stderr:29 Jun 15:39:23 ntpd[8041]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:23.023 INFO:teuthology.orchestra.run.smithi164.stderr:29 Jun 15:39:23 ntpd[8041]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:23.023 INFO:teuthology.orchestra.run.smithi164.stderr:29 Jun 15:39:23 ntpd[8041]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:23.023 INFO:teuthology.orchestra.run.smithi164.stderr:29 Jun 15:39:23 ntpd[8041]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-29T15:39:23.023 INFO:teuthology.orchestra.run.smithi164.stderr:29 Jun 15:39:23 ntpd[8041]: can't open /var/log/ntpstats/peerstats.20220629: Permission denied 2022-06-29T15:39:24.010 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:24 ntpd[7945]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:24.010 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:24 ntpd[7945]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:24.018 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:24 ntpd[8044]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:24.018 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:24 ntpd[8044]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:24.019 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:24 ntpd[8044]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:24.019 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:24 ntpd[8044]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:24.019 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:24 ntpd[8044]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:24.019 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:24 ntpd[8044]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:24.020 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:24 ntpd[8044]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-29T15:39:24.020 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:24 ntpd[8044]: can't open /var/log/ntpstats/peerstats.20220629: Permission denied 2022-06-29T15:39:24.020 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:24 ntpd[8044]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:24.020 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:24 ntpd[8044]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:24.021 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:24 ntpd[8044]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-29T15:39:24.021 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:24 ntpd[8044]: can't open /var/log/ntpstats/peerstats.20220629: Permission denied 2022-06-29T15:39:24.023 INFO:teuthology.orchestra.run.smithi164.stderr:29 Jun 15:39:24 ntpd[8041]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:24.024 INFO:teuthology.orchestra.run.smithi164.stderr:29 Jun 15:39:24 ntpd[8041]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:24.024 INFO:teuthology.orchestra.run.smithi164.stderr:29 Jun 15:39:24 ntpd[8041]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:24.024 INFO:teuthology.orchestra.run.smithi164.stderr:29 Jun 15:39:24 ntpd[8041]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:24.024 INFO:teuthology.orchestra.run.smithi164.stderr:29 Jun 15:39:24 ntpd[8041]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:24.025 INFO:teuthology.orchestra.run.smithi164.stderr:29 Jun 15:39:24 ntpd[8041]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:24.025 INFO:teuthology.orchestra.run.smithi164.stderr:29 Jun 15:39:24 ntpd[8041]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-29T15:39:24.025 INFO:teuthology.orchestra.run.smithi164.stderr:29 Jun 15:39:24 ntpd[8041]: can't open /var/log/ntpstats/peerstats.20220629: Permission denied 2022-06-29T15:39:24.025 INFO:teuthology.orchestra.run.smithi164.stderr:29 Jun 15:39:24 ntpd[8041]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:24.025 INFO:teuthology.orchestra.run.smithi164.stderr:29 Jun 15:39:24 ntpd[8041]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:24.026 INFO:teuthology.orchestra.run.smithi164.stderr:29 Jun 15:39:24 ntpd[8041]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-29T15:39:24.026 INFO:teuthology.orchestra.run.smithi164.stderr:29 Jun 15:39:24 ntpd[8041]: can't open /var/log/ntpstats/peerstats.20220629: Permission denied 2022-06-29T15:39:24.026 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:24 ntpd[8053]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:24.027 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:24 ntpd[8053]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:24.027 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:24 ntpd[8053]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:24.027 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:24 ntpd[8053]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:24.027 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:24 ntpd[8053]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:24.028 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:24 ntpd[8053]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:24.028 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:24 ntpd[8053]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-29T15:39:24.028 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:24 ntpd[8053]: can't open /var/log/ntpstats/peerstats.20220629: Permission denied 2022-06-29T15:39:24.028 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:24 ntpd[8053]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:24.029 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:24 ntpd[8053]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:24.029 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:24 ntpd[8053]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-29T15:39:24.029 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:24 ntpd[8053]: can't open /var/log/ntpstats/peerstats.20220629: Permission denied 2022-06-29T15:39:25.013 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:25 ntpd[7945]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:25.014 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:25 ntpd[7945]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:25.014 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:25 ntpd[7945]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:25.015 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:25 ntpd[7945]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:25.015 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:25 ntpd[7945]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:25.016 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:25 ntpd[7945]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:25.016 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:25 ntpd[7945]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:25.017 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:25 ntpd[7945]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:25.017 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:25 ntpd[7945]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-29T15:39:25.018 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:25 ntpd[7945]: can't open /var/log/ntpstats/peerstats.20220629: Permission denied 2022-06-29T15:39:25.018 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:25 ntpd[7945]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:25.019 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:25 ntpd[7945]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:25.019 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:25 ntpd[7945]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-29T15:39:25.019 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:25 ntpd[7945]: can't open /var/log/ntpstats/peerstats.20220629: Permission denied 2022-06-29T15:39:25.020 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:25 ntpd[7945]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:25.020 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:25 ntpd[7945]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:25.021 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:25 ntpd[7945]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-29T15:39:25.021 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:25 ntpd[7945]: can't open /var/log/ntpstats/peerstats.20220629: Permission denied 2022-06-29T15:39:25.026 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:25 ntpd[8044]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:25.026 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:25 ntpd[8044]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:25.026 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:25 ntpd[8044]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:25.027 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:25 ntpd[8044]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:25.027 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:25 ntpd[8044]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-29T15:39:25.027 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:25 ntpd[8044]: can't open /var/log/ntpstats/peerstats.20220629: Permission denied 2022-06-29T15:39:25.028 INFO:teuthology.orchestra.run.smithi164.stderr:29 Jun 15:39:25 ntpd[8041]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:25.028 INFO:teuthology.orchestra.run.smithi164.stderr:29 Jun 15:39:25 ntpd[8041]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:25.029 INFO:teuthology.orchestra.run.smithi164.stderr:29 Jun 15:39:25 ntpd[8041]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:25.029 INFO:teuthology.orchestra.run.smithi164.stderr:29 Jun 15:39:25 ntpd[8041]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:25.029 INFO:teuthology.orchestra.run.smithi164.stderr:29 Jun 15:39:25 ntpd[8041]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-29T15:39:25.030 INFO:teuthology.orchestra.run.smithi164.stderr:29 Jun 15:39:25 ntpd[8041]: can't open /var/log/ntpstats/peerstats.20220629: Permission denied 2022-06-29T15:39:25.030 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:25 ntpd[8053]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:25.031 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:25 ntpd[8053]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:25.031 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:25 ntpd[8053]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:25.032 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:25 ntpd[8053]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:25.032 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:25 ntpd[8053]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-29T15:39:25.032 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:25 ntpd[8053]: can't open /var/log/ntpstats/peerstats.20220629: Permission denied 2022-06-29T15:39:26.018 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:26 ntpd[8044]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:26.019 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:26 ntpd[8044]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:26.019 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:26 ntpd[8044]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:26.019 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:26 ntpd[8044]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:26.019 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:26 ntpd[8044]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:26.019 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:26 ntpd[8044]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:26.020 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:26 ntpd[8044]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-29T15:39:26.020 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:26 ntpd[8044]: can't open /var/log/ntpstats/peerstats.20220629: Permission denied 2022-06-29T15:39:26.020 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:26 ntpd[8044]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:26.021 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:26 ntpd[8044]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:26.021 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:26 ntpd[8044]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-29T15:39:26.021 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:26 ntpd[8044]: can't open /var/log/ntpstats/peerstats.20220629: Permission denied 2022-06-29T15:39:26.022 INFO:teuthology.orchestra.run.smithi064.stdout:29 Jun 15:39:26 ntpd[8044]: ntpd: time slew +0.001739 s 2022-06-29T15:39:26.022 INFO:teuthology.orchestra.run.smithi064.stdout:ntpd: time slew +0.001739s 2022-06-29T15:39:26.023 INFO:teuthology.orchestra.run.smithi064.stderr:29 Jun 15:39:26 ntpd[8044]: can't open /var/log/ntpstats/loopstats.20220629: Permission denied 2022-06-29T15:39:26.025 INFO:teuthology.orchestra.run.smithi164.stderr:29 Jun 15:39:26 ntpd[8041]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:26.026 INFO:teuthology.orchestra.run.smithi164.stderr:29 Jun 15:39:26 ntpd[8041]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:26.026 INFO:teuthology.orchestra.run.smithi164.stderr:29 Jun 15:39:26 ntpd[8041]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:26.026 INFO:teuthology.orchestra.run.smithi164.stderr:29 Jun 15:39:26 ntpd[8041]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:26.026 INFO:teuthology.orchestra.run.smithi164.stderr:29 Jun 15:39:26 ntpd[8041]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:26.027 INFO:teuthology.orchestra.run.smithi164.stderr:29 Jun 15:39:26 ntpd[8041]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:26.027 INFO:teuthology.orchestra.run.smithi164.stderr:29 Jun 15:39:26 ntpd[8041]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-29T15:39:26.027 INFO:teuthology.orchestra.run.smithi164.stderr:29 Jun 15:39:26 ntpd[8041]: can't open /var/log/ntpstats/peerstats.20220629: Permission denied 2022-06-29T15:39:26.027 INFO:teuthology.orchestra.run.smithi164.stderr:29 Jun 15:39:26 ntpd[8041]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:26.027 INFO:teuthology.orchestra.run.smithi164.stderr:29 Jun 15:39:26 ntpd[8041]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:26.028 INFO:teuthology.orchestra.run.smithi164.stderr:29 Jun 15:39:26 ntpd[8041]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-29T15:39:26.028 INFO:teuthology.orchestra.run.smithi164.stderr:29 Jun 15:39:26 ntpd[8041]: can't open /var/log/ntpstats/peerstats.20220629: Permission denied 2022-06-29T15:39:26.028 INFO:teuthology.orchestra.run.smithi164.stdout:29 Jun 15:39:26 ntpd[8041]: ntpd: time slew +0.001766 s 2022-06-29T15:39:26.029 INFO:teuthology.orchestra.run.smithi164.stdout:ntpd: time slew +0.001766s 2022-06-29T15:39:26.029 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:26 ntpd[8053]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:26.030 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:26 ntpd[8053]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:26.030 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:26 ntpd[8053]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:26.030 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:26 ntpd[8053]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:26.030 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:26 ntpd[8053]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:26.031 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:26 ntpd[8053]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:26.031 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:26 ntpd[8053]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-29T15:39:26.031 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:26 ntpd[8053]: can't open /var/log/ntpstats/peerstats.20220629: Permission denied 2022-06-29T15:39:26.031 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:26 ntpd[8053]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:26.032 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:26 ntpd[8053]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:26.032 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:26 ntpd[8053]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-29T15:39:26.032 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:26 ntpd[8053]: can't open /var/log/ntpstats/peerstats.20220629: Permission denied 2022-06-29T15:39:26.033 INFO:teuthology.orchestra.run.smithi164.stderr:29 Jun 15:39:26 ntpd[8041]: can't open /var/log/ntpstats/loopstats.20220629: Permission denied 2022-06-29T15:39:26.033 INFO:teuthology.orchestra.run.smithi099.stdout:29 Jun 15:39:26 ntpd[8053]: ntpd: time slew -0.000328 s 2022-06-29T15:39:26.033 INFO:teuthology.orchestra.run.smithi099.stdout:ntpd: time slew -0.000328s 2022-06-29T15:39:26.034 INFO:teuthology.orchestra.run.smithi099.stderr:29 Jun 15:39:26 ntpd[8053]: can't open /var/log/ntpstats/loopstats.20220629: Permission denied 2022-06-29T15:39:26.078 INFO:teuthology.orchestra.run.smithi164.stdout: remote refid st t when poll reach delay offset jitter 2022-06-29T15:39:26.079 INFO:teuthology.orchestra.run.smithi164.stdout:============================================================================== 2022-06-29T15:39:26.079 INFO:teuthology.orchestra.run.smithi164.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-06-29T15:39:26.079 INFO:teuthology.orchestra.run.smithi164.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-06-29T15:39:26.080 INFO:teuthology.orchestra.run.smithi164.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-06-29T15:39:26.080 INFO:teuthology.orchestra.run.smithi164.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-06-29T15:39:26.084 INFO:teuthology.orchestra.run.smithi099.stdout: remote refid st t when poll reach delay offset jitter 2022-06-29T15:39:26.084 INFO:teuthology.orchestra.run.smithi099.stdout:============================================================================== 2022-06-29T15:39:26.085 INFO:teuthology.orchestra.run.smithi099.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-06-29T15:39:26.085 INFO:teuthology.orchestra.run.smithi099.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-06-29T15:39:26.085 INFO:teuthology.orchestra.run.smithi099.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-06-29T15:39:26.086 INFO:teuthology.orchestra.run.smithi099.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-06-29T15:39:26.087 INFO:teuthology.orchestra.run.smithi064.stdout: remote refid st t when poll reach delay offset jitter 2022-06-29T15:39:26.087 INFO:teuthology.orchestra.run.smithi064.stdout:============================================================================== 2022-06-29T15:39:26.087 INFO:teuthology.orchestra.run.smithi064.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-06-29T15:39:26.088 INFO:teuthology.orchestra.run.smithi064.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-06-29T15:39:26.088 INFO:teuthology.orchestra.run.smithi064.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-06-29T15:39:26.089 INFO:teuthology.orchestra.run.smithi064.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-06-29T15:39:27.011 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:27 ntpd[7945]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:27.011 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:27 ntpd[7945]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:27.012 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:27 ntpd[7945]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:27.012 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:27 ntpd[7945]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:27.013 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:27 ntpd[7945]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:27.013 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:27 ntpd[7945]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:27.013 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:27 ntpd[7945]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-06-29T15:39:27.014 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:27 ntpd[7945]: can't open /var/log/ntpstats/rawstats.20220629: Permission denied 2022-06-29T15:39:27.014 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:27 ntpd[7945]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-06-29T15:39:27.015 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:27 ntpd[7945]: can't open /var/log/ntpstats/peerstats.20220629: Permission denied 2022-06-29T15:39:27.015 INFO:teuthology.orchestra.run.smithi047.stdout:29 Jun 15:39:27 ntpd[7945]: ntpd: time slew -0.002637 s 2022-06-29T15:39:27.016 INFO:teuthology.orchestra.run.smithi047.stdout:ntpd: time slew -0.002637s 2022-06-29T15:39:27.016 INFO:teuthology.orchestra.run.smithi047.stderr:29 Jun 15:39:27 ntpd[7945]: can't open /var/log/ntpstats/loopstats.20220629: Permission denied 2022-06-29T15:39:27.069 INFO:teuthology.orchestra.run.smithi047.stdout: remote refid st t when poll reach delay offset jitter 2022-06-29T15:39:27.069 INFO:teuthology.orchestra.run.smithi047.stdout:============================================================================== 2022-06-29T15:39:27.069 INFO:teuthology.orchestra.run.smithi047.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-06-29T15:39:27.070 INFO:teuthology.orchestra.run.smithi047.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-06-29T15:39:27.070 INFO:teuthology.orchestra.run.smithi047.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-06-29T15:39:27.070 INFO:teuthology.orchestra.run.smithi047.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-06-29T15:39:27.072 INFO:teuthology.run_tasks:Running task install... 2022-06-29T15:39:27.081 DEBUG:teuthology.task.install:project ceph 2022-06-29T15:39:27.081 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'f25ffac72e5a9e63860ff7deb6767a11f0559a16'}} 2022-06-29T15:39:27.082 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': 'f25ffac72e5a9e63860ff7deb6767a11f0559a16'} 2022-06-29T15:39:27.082 INFO:teuthology.task.install:Using flavor: default 2022-06-29T15:39:27.090 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2022-06-29T15:39:27.090 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']} 2022-06-29T15:39:27.091 DEBUG:teuthology.orchestra.run.smithi047:> sudo lsb_release -is 2022-06-29T15:39:27.142 INFO:teuthology.orchestra.run.smithi047.stdout:Ubuntu 2022-06-29T15:39:27.147 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-06-29T15:39:27.148 DEBUG:teuthology.orchestra.run.smithi064:> sudo lsb_release -is 2022-06-29T15:39:27.149 DEBUG:teuthology.orchestra.run.smithi047:> sudo apt-key list | grep Ceph 2022-06-29T15:39:27.201 INFO:teuthology.orchestra.run.smithi064.stdout:Ubuntu 2022-06-29T15:39:27.206 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-06-29T15:39:27.207 DEBUG:teuthology.orchestra.run.smithi099:> sudo lsb_release -is 2022-06-29T15:39:27.207 DEBUG:teuthology.orchestra.run.smithi064:> sudo apt-key list | grep Ceph 2022-06-29T15:39:27.241 INFO:teuthology.orchestra.run.smithi047.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2022-06-29T15:39:27.259 INFO:teuthology.orchestra.run.smithi099.stdout:Ubuntu 2022-06-29T15:39:27.265 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-06-29T15:39:27.267 DEBUG:teuthology.orchestra.run.smithi164:> sudo lsb_release -is 2022-06-29T15:39:27.268 DEBUG:teuthology.orchestra.run.smithi099:> sudo apt-key list | grep Ceph 2022-06-29T15:39:27.304 INFO:teuthology.orchestra.run.smithi064.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2022-06-29T15:39:27.305 INFO:teuthology.orchestra.run.smithi047.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2022-06-29T15:39:27.306 INFO:teuthology.orchestra.run.smithi047.stdout:uid [ unknown] Ceph.com (release key) 2022-06-29T15:39:27.306 DEBUG:teuthology.orchestra.run.smithi047:> sudo lsb_release -is 2022-06-29T15:39:27.319 INFO:teuthology.orchestra.run.smithi164.stdout:Ubuntu 2022-06-29T15:39:27.325 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-06-29T15:39:27.325 DEBUG:teuthology.orchestra.run.smithi164:> sudo apt-key list | grep Ceph 2022-06-29T15:39:27.354 INFO:teuthology.orchestra.run.smithi047.stdout:Ubuntu 2022-06-29T15:39:27.359 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-06-29T15:39:27.360 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2022-06-29T15:39:27.360 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2022-06-29T15:39:27.361 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=f25ffac72e5a9e63860ff7deb6767a11f0559a16 2022-06-29T15:39:27.367 INFO:teuthology.orchestra.run.smithi099.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2022-06-29T15:39:27.368 INFO:teuthology.orchestra.run.smithi064.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2022-06-29T15:39:27.368 INFO:teuthology.orchestra.run.smithi064.stdout:uid [ unknown] Ceph.com (release key) 2022-06-29T15:39:27.368 DEBUG:teuthology.orchestra.run.smithi064:> sudo lsb_release -is 2022-06-29T15:39:27.426 INFO:teuthology.orchestra.run.smithi164.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2022-06-29T15:39:27.428 INFO:teuthology.orchestra.run.smithi099.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2022-06-29T15:39:27.429 INFO:teuthology.orchestra.run.smithi099.stdout:uid [ unknown] Ceph.com (release key) 2022-06-29T15:39:27.429 INFO:teuthology.orchestra.run.smithi064.stdout:Ubuntu 2022-06-29T15:39:27.430 DEBUG:teuthology.orchestra.run.smithi099:> sudo lsb_release -is 2022-06-29T15:39:27.431 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-06-29T15:39:27.431 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2022-06-29T15:39:27.431 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2022-06-29T15:39:27.432 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=f25ffac72e5a9e63860ff7deb6767a11f0559a16 2022-06-29T15:39:27.482 INFO:teuthology.orchestra.run.smithi099.stdout:Ubuntu 2022-06-29T15:39:27.484 INFO:teuthology.orchestra.run.smithi164.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2022-06-29T15:39:27.484 INFO:teuthology.orchestra.run.smithi164.stdout:uid [ unknown] Ceph.com (release key) 2022-06-29T15:39:27.496 DEBUG:teuthology.orchestra.run.smithi164:> sudo lsb_release -is 2022-06-29T15:39:27.498 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-06-29T15:39:27.498 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2022-06-29T15:39:27.499 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2022-06-29T15:39:27.499 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=f25ffac72e5a9e63860ff7deb6767a11f0559a16 2022-06-29T15:39:27.551 INFO:teuthology.orchestra.run.smithi164.stdout:Ubuntu 2022-06-29T15:39:27.566 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-06-29T15:39:27.566 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2022-06-29T15:39:27.567 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2022-06-29T15:39:27.567 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=f25ffac72e5a9e63860ff7deb6767a11f0559a16 2022-06-29T15:39:27.669 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default/ 2022-06-29T15:39:27.670 INFO:teuthology.task.install.deb:Package version is 15.2.16-304-gf25ffac7-1focal 2022-06-29T15:39:27.733 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default/ 2022-06-29T15:39:27.734 INFO:teuthology.task.install.deb:Package version is 15.2.16-304-gf25ffac7-1focal 2022-06-29T15:39:27.823 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default/ 2022-06-29T15:39:27.823 INFO:teuthology.task.install.deb:Package version is 15.2.16-304-gf25ffac7-1focal 2022-06-29T15:39:27.893 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default/ 2022-06-29T15:39:27.894 INFO:teuthology.task.install.deb:Package version is 15.2.16-304-gf25ffac7-1focal 2022-06-29T15:39:27.918 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2022-06-29T15:39:27.918 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2022-06-29T15:39:27.941 DEBUG:teuthology.orchestra.run.smithi047:> sudo apt-get update 2022-06-29T15:39:27.953 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2022-06-29T15:39:27.954 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2022-06-29T15:39:27.971 DEBUG:teuthology.orchestra.run.smithi064:> sudo apt-get update 2022-06-29T15:39:28.013 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2022-06-29T15:39:28.013 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2022-06-29T15:39:28.031 DEBUG:teuthology.orchestra.run.smithi099:> sudo apt-get update 2022-06-29T15:39:28.067 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2022-06-29T15:39:28.067 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2022-06-29T15:39:28.084 DEBUG:teuthology.orchestra.run.smithi164:> sudo apt-get update 2022-06-29T15:39:28.112 INFO:teuthology.orchestra.run.smithi064.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2022-06-29T15:39:28.159 INFO:teuthology.orchestra.run.smithi047.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2022-06-29T15:39:28.182 INFO:teuthology.orchestra.run.smithi047.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2022-06-29T15:39:28.214 INFO:teuthology.orchestra.run.smithi047.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2022-06-29T15:39:28.226 INFO:teuthology.orchestra.run.smithi064.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2022-06-29T15:39:28.241 INFO:teuthology.orchestra.run.smithi047.stdout:Hit:4 http://security.ubuntu.com/ubuntu focal-security InRelease 2022-06-29T15:39:28.270 INFO:teuthology.orchestra.run.smithi047.stdout:Ign:5 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal InRelease 2022-06-29T15:39:28.270 INFO:teuthology.orchestra.run.smithi064.stdout:Ign:3 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal InRelease 2022-06-29T15:39:28.298 INFO:teuthology.orchestra.run.smithi099.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2022-06-29T15:39:28.298 INFO:teuthology.orchestra.run.smithi099.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2022-06-29T15:39:28.306 INFO:teuthology.orchestra.run.smithi064.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2022-06-29T15:39:28.308 INFO:teuthology.orchestra.run.smithi099.stdout:Ign:3 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal InRelease 2022-06-29T15:39:28.309 INFO:teuthology.orchestra.run.smithi064.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal Release [7,680 B] 2022-06-29T15:39:28.318 INFO:teuthology.orchestra.run.smithi047.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal Release [7,680 B] 2022-06-29T15:39:28.346 INFO:teuthology.orchestra.run.smithi164.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2022-06-29T15:39:28.346 INFO:teuthology.orchestra.run.smithi164.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2022-06-29T15:39:28.347 INFO:teuthology.orchestra.run.smithi164.stdout:Ign:3 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal InRelease 2022-06-29T15:39:28.347 INFO:teuthology.orchestra.run.smithi099.stdout:Get:4 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal Release [7,680 B] 2022-06-29T15:39:28.348 INFO:teuthology.orchestra.run.smithi064.stdout:Ign:6 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal Release.gpg 2022-06-29T15:39:28.357 INFO:teuthology.orchestra.run.smithi047.stdout:Ign:7 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal Release.gpg 2022-06-29T15:39:28.365 INFO:teuthology.orchestra.run.smithi099.stdout:Hit:5 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2022-06-29T15:39:28.383 INFO:teuthology.orchestra.run.smithi164.stdout:Get:4 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal Release [7,680 B] 2022-06-29T15:39:28.384 INFO:teuthology.orchestra.run.smithi099.stdout:Ign:6 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal Release.gpg 2022-06-29T15:39:28.385 INFO:teuthology.orchestra.run.smithi064.stdout:Hit:7 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2022-06-29T15:39:28.390 INFO:teuthology.orchestra.run.smithi064.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main i386 Packages [3,062 B] 2022-06-29T15:39:28.393 INFO:teuthology.orchestra.run.smithi047.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main i386 Packages [3,062 B] 2022-06-29T15:39:28.414 INFO:teuthology.orchestra.run.smithi164.stdout:Hit:5 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2022-06-29T15:39:28.423 INFO:teuthology.orchestra.run.smithi099.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 Packages [15.8 kB] 2022-06-29T15:39:28.424 INFO:teuthology.orchestra.run.smithi164.stdout:Ign:6 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal Release.gpg 2022-06-29T15:39:28.447 INFO:teuthology.orchestra.run.smithi099.stdout:Hit:8 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2022-06-29T15:39:28.456 INFO:teuthology.orchestra.run.smithi047.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 Packages [15.8 kB] 2022-06-29T15:39:28.461 INFO:teuthology.orchestra.run.smithi164.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main i386 Packages [3,062 B] 2022-06-29T15:39:28.461 INFO:teuthology.orchestra.run.smithi064.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 Packages [15.8 kB] 2022-06-29T15:39:28.477 INFO:teuthology.orchestra.run.smithi099.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main i386 Packages [3,062 B] 2022-06-29T15:39:28.495 INFO:teuthology.orchestra.run.smithi164.stdout:Hit:8 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2022-06-29T15:39:28.520 INFO:teuthology.orchestra.run.smithi164.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 Packages [15.8 kB] 2022-06-29T15:39:28.796 INFO:teuthology.orchestra.run.smithi064.stdout:Fetched 26.5 kB in 1s (40.0 kB/s) 2022-06-29T15:39:28.843 INFO:teuthology.orchestra.run.smithi047.stdout:Fetched 26.5 kB in 1s (38.1 kB/s) 2022-06-29T15:39:28.954 INFO:teuthology.orchestra.run.smithi099.stdout:Fetched 26.5 kB in 1s (34.7 kB/s) 2022-06-29T15:39:29.006 INFO:teuthology.orchestra.run.smithi164.stdout:Fetched 26.5 kB in 1s (34.6 kB/s) 2022-06-29T15:39:29.704 INFO:teuthology.orchestra.run.smithi064.stdout:Reading package lists... 2022-06-29T15:39:29.727 DEBUG:teuthology.orchestra.run.smithi064:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=15.2.16-304-gf25ffac7-1focal cephadm=15.2.16-304-gf25ffac7-1focal ceph-mds=15.2.16-304-gf25ffac7-1focal ceph-mgr=15.2.16-304-gf25ffac7-1focal ceph-common=15.2.16-304-gf25ffac7-1focal ceph-fuse=15.2.16-304-gf25ffac7-1focal ceph-test=15.2.16-304-gf25ffac7-1focal radosgw=15.2.16-304-gf25ffac7-1focal python3-rados=15.2.16-304-gf25ffac7-1focal python3-rgw=15.2.16-304-gf25ffac7-1focal python3-cephfs=15.2.16-304-gf25ffac7-1focal python3-rbd=15.2.16-304-gf25ffac7-1focal libcephfs2=15.2.16-304-gf25ffac7-1focal libcephfs-dev=15.2.16-304-gf25ffac7-1focal librados2=15.2.16-304-gf25ffac7-1focal librbd1=15.2.16-304-gf25ffac7-1focal rbd-fuse=15.2.16-304-gf25ffac7-1focal python3-cephfs=15.2.16-304-gf25ffac7-1focal cephfs-shell=15.2.16-304-gf25ffac7-1focal 2022-06-29T15:39:29.757 INFO:teuthology.orchestra.run.smithi047.stdout:Reading package lists... 2022-06-29T15:39:29.781 DEBUG:teuthology.orchestra.run.smithi047:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=15.2.16-304-gf25ffac7-1focal cephadm=15.2.16-304-gf25ffac7-1focal ceph-mds=15.2.16-304-gf25ffac7-1focal ceph-mgr=15.2.16-304-gf25ffac7-1focal ceph-common=15.2.16-304-gf25ffac7-1focal ceph-fuse=15.2.16-304-gf25ffac7-1focal ceph-test=15.2.16-304-gf25ffac7-1focal radosgw=15.2.16-304-gf25ffac7-1focal python3-rados=15.2.16-304-gf25ffac7-1focal python3-rgw=15.2.16-304-gf25ffac7-1focal python3-cephfs=15.2.16-304-gf25ffac7-1focal python3-rbd=15.2.16-304-gf25ffac7-1focal libcephfs2=15.2.16-304-gf25ffac7-1focal libcephfs-dev=15.2.16-304-gf25ffac7-1focal librados2=15.2.16-304-gf25ffac7-1focal librbd1=15.2.16-304-gf25ffac7-1focal rbd-fuse=15.2.16-304-gf25ffac7-1focal python3-cephfs=15.2.16-304-gf25ffac7-1focal cephfs-shell=15.2.16-304-gf25ffac7-1focal 2022-06-29T15:39:29.799 INFO:teuthology.orchestra.run.smithi064.stdout:Reading package lists... 2022-06-29T15:39:29.830 INFO:teuthology.orchestra.run.smithi099.stdout:Reading package lists... 2022-06-29T15:39:29.849 INFO:teuthology.orchestra.run.smithi047.stdout:Reading package lists... 2022-06-29T15:39:29.853 DEBUG:teuthology.orchestra.run.smithi099:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=15.2.16-304-gf25ffac7-1focal cephadm=15.2.16-304-gf25ffac7-1focal ceph-mds=15.2.16-304-gf25ffac7-1focal ceph-mgr=15.2.16-304-gf25ffac7-1focal ceph-common=15.2.16-304-gf25ffac7-1focal ceph-fuse=15.2.16-304-gf25ffac7-1focal ceph-test=15.2.16-304-gf25ffac7-1focal radosgw=15.2.16-304-gf25ffac7-1focal python3-rados=15.2.16-304-gf25ffac7-1focal python3-rgw=15.2.16-304-gf25ffac7-1focal python3-cephfs=15.2.16-304-gf25ffac7-1focal python3-rbd=15.2.16-304-gf25ffac7-1focal libcephfs2=15.2.16-304-gf25ffac7-1focal libcephfs-dev=15.2.16-304-gf25ffac7-1focal librados2=15.2.16-304-gf25ffac7-1focal librbd1=15.2.16-304-gf25ffac7-1focal rbd-fuse=15.2.16-304-gf25ffac7-1focal python3-cephfs=15.2.16-304-gf25ffac7-1focal cephfs-shell=15.2.16-304-gf25ffac7-1focal 2022-06-29T15:39:29.876 INFO:teuthology.orchestra.run.smithi164.stdout:Reading package lists... 2022-06-29T15:39:29.900 DEBUG:teuthology.orchestra.run.smithi164:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=15.2.16-304-gf25ffac7-1focal cephadm=15.2.16-304-gf25ffac7-1focal ceph-mds=15.2.16-304-gf25ffac7-1focal ceph-mgr=15.2.16-304-gf25ffac7-1focal ceph-common=15.2.16-304-gf25ffac7-1focal ceph-fuse=15.2.16-304-gf25ffac7-1focal ceph-test=15.2.16-304-gf25ffac7-1focal radosgw=15.2.16-304-gf25ffac7-1focal python3-rados=15.2.16-304-gf25ffac7-1focal python3-rgw=15.2.16-304-gf25ffac7-1focal python3-cephfs=15.2.16-304-gf25ffac7-1focal python3-rbd=15.2.16-304-gf25ffac7-1focal libcephfs2=15.2.16-304-gf25ffac7-1focal libcephfs-dev=15.2.16-304-gf25ffac7-1focal librados2=15.2.16-304-gf25ffac7-1focal librbd1=15.2.16-304-gf25ffac7-1focal rbd-fuse=15.2.16-304-gf25ffac7-1focal python3-cephfs=15.2.16-304-gf25ffac7-1focal cephfs-shell=15.2.16-304-gf25ffac7-1focal 2022-06-29T15:39:29.921 INFO:teuthology.orchestra.run.smithi099.stdout:Reading package lists... 2022-06-29T15:39:29.961 INFO:teuthology.orchestra.run.smithi064.stdout:Building dependency tree... 2022-06-29T15:39:29.961 INFO:teuthology.orchestra.run.smithi064.stdout:Reading state information... 2022-06-29T15:39:29.971 INFO:teuthology.orchestra.run.smithi164.stdout:Reading package lists... 2022-06-29T15:39:30.031 INFO:teuthology.orchestra.run.smithi047.stdout:Building dependency tree... 2022-06-29T15:39:30.032 INFO:teuthology.orchestra.run.smithi047.stdout:Reading state information... 2022-06-29T15:39:30.084 INFO:teuthology.orchestra.run.smithi099.stdout:Building dependency tree... 2022-06-29T15:39:30.085 INFO:teuthology.orchestra.run.smithi099.stdout:Reading state information... 2022-06-29T15:39:30.110 INFO:teuthology.orchestra.run.smithi064.stdout:The following packages were automatically installed and are no longer required: 2022-06-29T15:39:30.112 INFO:teuthology.orchestra.run.smithi064.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2022-06-29T15:39:30.112 INFO:teuthology.orchestra.run.smithi064.stdout:Use 'sudo apt autoremove' to remove them. 2022-06-29T15:39:30.113 INFO:teuthology.orchestra.run.smithi064.stdout:The following additional packages will be installed: 2022-06-29T15:39:30.113 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2022-06-29T15:39:30.113 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2022-06-29T15:39:30.114 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mon ceph-osd jq libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 2022-06-29T15:39:30.114 INFO:teuthology.orchestra.run.smithi064.stdout: libleveldb1d liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 2022-06-29T15:39:30.114 INFO:teuthology.orchestra.run.smithi064.stdout: librdkafka1 librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl 2022-06-29T15:39:30.115 INFO:teuthology.orchestra.run.smithi064.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-blinker python3-bs4 2022-06-29T15:39:30.115 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common 2022-06-29T15:39:30.115 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2022-06-29T15:39:30.115 INFO:teuthology.orchestra.run.smithi064.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2022-06-29T15:39:30.116 INFO:teuthology.orchestra.run.smithi064.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2022-06-29T15:39:30.116 INFO:teuthology.orchestra.run.smithi064.stdout: python3-mako python3-markupsafe python3-oauthlib python3-openssl 2022-06-29T15:39:30.116 INFO:teuthology.orchestra.run.smithi064.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2022-06-29T15:39:30.116 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2022-06-29T15:39:30.117 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2022-06-29T15:39:30.117 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2022-06-29T15:39:30.117 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2022-06-29T15:39:30.117 INFO:teuthology.orchestra.run.smithi064.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2022-06-29T15:39:30.118 INFO:teuthology.orchestra.run.smithi064.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2022-06-29T15:39:30.118 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug socat 2022-06-29T15:39:30.118 INFO:teuthology.orchestra.run.smithi064.stdout: xmlstarlet 2022-06-29T15:39:30.119 INFO:teuthology.orchestra.run.smithi064.stdout:Suggested packages: 2022-06-29T15:39:30.119 INFO:teuthology.orchestra.run.smithi064.stdout: python3-influxdb python-attr-doc python-blinker-doc python-cryptography-doc 2022-06-29T15:39:30.119 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2022-06-29T15:39:30.120 INFO:teuthology.orchestra.run.smithi064.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2022-06-29T15:39:30.120 INFO:teuthology.orchestra.run.smithi064.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2022-06-29T15:39:30.120 INFO:teuthology.orchestra.run.smithi064.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2022-06-29T15:39:30.120 INFO:teuthology.orchestra.run.smithi064.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2022-06-29T15:39:30.120 INFO:teuthology.orchestra.run.smithi064.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2022-06-29T15:39:30.121 INFO:teuthology.orchestra.run.smithi064.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2022-06-29T15:39:30.121 INFO:teuthology.orchestra.run.smithi064.stdout:Recommended packages: 2022-06-29T15:39:30.121 INFO:teuthology.orchestra.run.smithi064.stdout: btrfs-tools 2022-06-29T15:39:30.156 INFO:teuthology.orchestra.run.smithi164.stdout:Building dependency tree... 2022-06-29T15:39:30.156 INFO:teuthology.orchestra.run.smithi164.stdout:Reading state information... 2022-06-29T15:39:30.188 INFO:teuthology.orchestra.run.smithi047.stdout:The following packages were automatically installed and are no longer required: 2022-06-29T15:39:30.189 INFO:teuthology.orchestra.run.smithi047.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2022-06-29T15:39:30.189 INFO:teuthology.orchestra.run.smithi047.stdout:Use 'sudo apt autoremove' to remove them. 2022-06-29T15:39:30.189 INFO:teuthology.orchestra.run.smithi047.stdout:The following additional packages will be installed: 2022-06-29T15:39:30.190 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2022-06-29T15:39:30.190 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2022-06-29T15:39:30.190 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mon ceph-osd jq libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 2022-06-29T15:39:30.191 INFO:teuthology.orchestra.run.smithi047.stdout: libleveldb1d liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 2022-06-29T15:39:30.192 INFO:teuthology.orchestra.run.smithi047.stdout: librdkafka1 librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl 2022-06-29T15:39:30.192 INFO:teuthology.orchestra.run.smithi047.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-blinker python3-bs4 2022-06-29T15:39:30.192 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common 2022-06-29T15:39:30.193 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2022-06-29T15:39:30.193 INFO:teuthology.orchestra.run.smithi047.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2022-06-29T15:39:30.193 INFO:teuthology.orchestra.run.smithi047.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2022-06-29T15:39:30.193 INFO:teuthology.orchestra.run.smithi047.stdout: python3-mako python3-markupsafe python3-oauthlib python3-openssl 2022-06-29T15:39:30.193 INFO:teuthology.orchestra.run.smithi047.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2022-06-29T15:39:30.194 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2022-06-29T15:39:30.194 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2022-06-29T15:39:30.194 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2022-06-29T15:39:30.194 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2022-06-29T15:39:30.195 INFO:teuthology.orchestra.run.smithi047.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2022-06-29T15:39:30.195 INFO:teuthology.orchestra.run.smithi047.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2022-06-29T15:39:30.195 INFO:teuthology.orchestra.run.smithi047.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug socat 2022-06-29T15:39:30.195 INFO:teuthology.orchestra.run.smithi047.stdout: xmlstarlet 2022-06-29T15:39:30.196 INFO:teuthology.orchestra.run.smithi047.stdout:Suggested packages: 2022-06-29T15:39:30.196 INFO:teuthology.orchestra.run.smithi047.stdout: python3-influxdb python-attr-doc python-blinker-doc python-cryptography-doc 2022-06-29T15:39:30.196 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2022-06-29T15:39:30.197 INFO:teuthology.orchestra.run.smithi047.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2022-06-29T15:39:30.197 INFO:teuthology.orchestra.run.smithi047.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2022-06-29T15:39:30.197 INFO:teuthology.orchestra.run.smithi047.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2022-06-29T15:39:30.197 INFO:teuthology.orchestra.run.smithi047.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2022-06-29T15:39:30.197 INFO:teuthology.orchestra.run.smithi047.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2022-06-29T15:39:30.198 INFO:teuthology.orchestra.run.smithi047.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2022-06-29T15:39:30.198 INFO:teuthology.orchestra.run.smithi047.stdout:Recommended packages: 2022-06-29T15:39:30.198 INFO:teuthology.orchestra.run.smithi047.stdout: btrfs-tools 2022-06-29T15:39:30.223 INFO:teuthology.orchestra.run.smithi064.stdout:The following NEW packages will be installed: 2022-06-29T15:39:30.223 INFO:teuthology.orchestra.run.smithi064.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2022-06-29T15:39:30.224 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2022-06-29T15:39:30.224 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2022-06-29T15:39:30.224 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mon ceph-osd ceph-test cephadm cephfs-shell jq libcephfs-dev libcephfs2 2022-06-29T15:39:30.224 INFO:teuthology.orchestra.run.smithi064.stdout: libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 libleveldb1d 2022-06-29T15:39:30.225 INFO:teuthology.orchestra.run.smithi064.stdout: liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 librdkafka1 2022-06-29T15:39:30.226 INFO:teuthology.orchestra.run.smithi064.stdout: librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl python3-atomicwrites 2022-06-29T15:39:30.226 INFO:teuthology.orchestra.run.smithi064.stdout: python3-attr python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2022-06-29T15:39:30.226 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cherrypy3 2022-06-29T15:39:30.226 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cmd2 python3-colorama python3-cryptography python3-decorator 2022-06-29T15:39:30.227 INFO:teuthology.orchestra.run.smithi064.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2022-06-29T15:39:30.227 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2022-06-29T15:39:30.227 INFO:teuthology.orchestra.run.smithi064.stdout: python3-markupsafe python3-oauthlib python3-openssl python3-packaging 2022-06-29T15:39:30.227 INFO:teuthology.orchestra.run.smithi064.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2022-06-29T15:39:30.228 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2022-06-29T15:39:30.228 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2022-06-29T15:39:30.228 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests-oauthlib 2022-06-29T15:39:30.228 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rgw python3-routes python3-rsa python3-scipy python3-simplegeneric 2022-06-29T15:39:30.229 INFO:teuthology.orchestra.run.smithi064.stdout: python3-simplejson python3-singledispatch python3-sklearn 2022-06-29T15:39:30.229 INFO:teuthology.orchestra.run.smithi064.stdout: python3-sklearn-lib python3-soupsieve python3-tempita python3-waitress 2022-06-29T15:39:30.229 INFO:teuthology.orchestra.run.smithi064.stdout: python3-wcwidth python3-webencodings python3-webob python3-websocket 2022-06-29T15:39:30.229 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webtest python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2022-06-29T15:39:30.229 INFO:teuthology.orchestra.run.smithi064.stdout:The following packages will be upgraded: 2022-06-29T15:39:30.230 INFO:teuthology.orchestra.run.smithi064.stdout: librados2 librbd1 2022-06-29T15:39:30.231 INFO:teuthology.orchestra.run.smithi099.stdout:The following packages were automatically installed and are no longer required: 2022-06-29T15:39:30.231 INFO:teuthology.orchestra.run.smithi099.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2022-06-29T15:39:30.231 INFO:teuthology.orchestra.run.smithi099.stdout:Use 'sudo apt autoremove' to remove them. 2022-06-29T15:39:30.232 INFO:teuthology.orchestra.run.smithi099.stdout:The following additional packages will be installed: 2022-06-29T15:39:30.232 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2022-06-29T15:39:30.232 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2022-06-29T15:39:30.232 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mon ceph-osd jq libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 2022-06-29T15:39:30.233 INFO:teuthology.orchestra.run.smithi099.stdout: libleveldb1d liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 2022-06-29T15:39:30.233 INFO:teuthology.orchestra.run.smithi099.stdout: librdkafka1 librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl 2022-06-29T15:39:30.233 INFO:teuthology.orchestra.run.smithi099.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-blinker python3-bs4 2022-06-29T15:39:30.233 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common 2022-06-29T15:39:30.234 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2022-06-29T15:39:30.234 INFO:teuthology.orchestra.run.smithi099.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2022-06-29T15:39:30.234 INFO:teuthology.orchestra.run.smithi099.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2022-06-29T15:39:30.234 INFO:teuthology.orchestra.run.smithi099.stdout: python3-mako python3-markupsafe python3-oauthlib python3-openssl 2022-06-29T15:39:30.234 INFO:teuthology.orchestra.run.smithi099.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2022-06-29T15:39:30.235 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2022-06-29T15:39:30.235 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2022-06-29T15:39:30.235 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2022-06-29T15:39:30.236 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2022-06-29T15:39:30.236 INFO:teuthology.orchestra.run.smithi099.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2022-06-29T15:39:30.236 INFO:teuthology.orchestra.run.smithi099.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2022-06-29T15:39:30.236 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug socat 2022-06-29T15:39:30.237 INFO:teuthology.orchestra.run.smithi099.stdout: xmlstarlet 2022-06-29T15:39:30.237 INFO:teuthology.orchestra.run.smithi099.stdout:Suggested packages: 2022-06-29T15:39:30.238 INFO:teuthology.orchestra.run.smithi099.stdout: python3-influxdb python-attr-doc python-blinker-doc python-cryptography-doc 2022-06-29T15:39:30.238 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2022-06-29T15:39:30.238 INFO:teuthology.orchestra.run.smithi099.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2022-06-29T15:39:30.238 INFO:teuthology.orchestra.run.smithi099.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2022-06-29T15:39:30.239 INFO:teuthology.orchestra.run.smithi099.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2022-06-29T15:39:30.239 INFO:teuthology.orchestra.run.smithi099.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2022-06-29T15:39:30.239 INFO:teuthology.orchestra.run.smithi099.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2022-06-29T15:39:30.240 INFO:teuthology.orchestra.run.smithi099.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2022-06-29T15:39:30.240 INFO:teuthology.orchestra.run.smithi099.stdout:Recommended packages: 2022-06-29T15:39:30.240 INFO:teuthology.orchestra.run.smithi099.stdout: btrfs-tools 2022-06-29T15:39:30.286 INFO:teuthology.orchestra.run.smithi064.stdout:2 upgraded, 101 newly installed, 0 to remove and 7 not upgraded. 2022-06-29T15:39:30.287 INFO:teuthology.orchestra.run.smithi064.stdout:Need to get 153 MB of archives. 2022-06-29T15:39:30.287 INFO:teuthology.orchestra.run.smithi064.stdout:After this operation, 699 MB of additional disk space will be used. 2022-06-29T15:39:30.287 INFO:teuthology.orchestra.run.smithi064.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2022-06-29T15:39:30.296 INFO:teuthology.orchestra.run.smithi047.stdout:The following NEW packages will be installed: 2022-06-29T15:39:30.296 INFO:teuthology.orchestra.run.smithi047.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2022-06-29T15:39:30.296 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2022-06-29T15:39:30.297 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2022-06-29T15:39:30.298 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mon ceph-osd ceph-test cephadm cephfs-shell jq libcephfs-dev libcephfs2 2022-06-29T15:39:30.299 INFO:teuthology.orchestra.run.smithi047.stdout: libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 libleveldb1d 2022-06-29T15:39:30.299 INFO:teuthology.orchestra.run.smithi047.stdout: liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 librdkafka1 2022-06-29T15:39:30.299 INFO:teuthology.orchestra.run.smithi047.stdout: librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl python3-atomicwrites 2022-06-29T15:39:30.299 INFO:teuthology.orchestra.run.smithi047.stdout: python3-attr python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2022-06-29T15:39:30.300 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cherrypy3 2022-06-29T15:39:30.300 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cmd2 python3-colorama python3-cryptography python3-decorator 2022-06-29T15:39:30.300 INFO:teuthology.orchestra.run.smithi047.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2022-06-29T15:39:30.300 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2022-06-29T15:39:30.301 INFO:teuthology.orchestra.run.smithi047.stdout: python3-markupsafe python3-oauthlib python3-openssl python3-packaging 2022-06-29T15:39:30.301 INFO:teuthology.orchestra.run.smithi047.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2022-06-29T15:39:30.301 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2022-06-29T15:39:30.301 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2022-06-29T15:39:30.301 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests-oauthlib 2022-06-29T15:39:30.302 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rgw python3-routes python3-rsa python3-scipy python3-simplegeneric 2022-06-29T15:39:30.302 INFO:teuthology.orchestra.run.smithi047.stdout: python3-simplejson python3-singledispatch python3-sklearn 2022-06-29T15:39:30.302 INFO:teuthology.orchestra.run.smithi047.stdout: python3-sklearn-lib python3-soupsieve python3-tempita python3-waitress 2022-06-29T15:39:30.303 INFO:teuthology.orchestra.run.smithi047.stdout: python3-wcwidth python3-webencodings python3-webob python3-websocket 2022-06-29T15:39:30.303 INFO:teuthology.orchestra.run.smithi047.stdout: python3-webtest python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2022-06-29T15:39:30.303 INFO:teuthology.orchestra.run.smithi047.stdout:The following packages will be upgraded: 2022-06-29T15:39:30.303 INFO:teuthology.orchestra.run.smithi047.stdout: librados2 librbd1 2022-06-29T15:39:30.305 INFO:teuthology.orchestra.run.smithi064.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2022-06-29T15:39:30.308 INFO:teuthology.orchestra.run.smithi064.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2022-06-29T15:39:30.311 INFO:teuthology.orchestra.run.smithi164.stdout:The following packages were automatically installed and are no longer required: 2022-06-29T15:39:30.312 INFO:teuthology.orchestra.run.smithi164.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2022-06-29T15:39:30.313 INFO:teuthology.orchestra.run.smithi164.stdout:Use 'sudo apt autoremove' to remove them. 2022-06-29T15:39:30.313 INFO:teuthology.orchestra.run.smithi164.stdout:The following additional packages will be installed: 2022-06-29T15:39:30.313 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2022-06-29T15:39:30.313 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2022-06-29T15:39:30.315 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mon ceph-osd jq libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 2022-06-29T15:39:30.315 INFO:teuthology.orchestra.run.smithi164.stdout: libleveldb1d liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 2022-06-29T15:39:30.316 INFO:teuthology.orchestra.run.smithi164.stdout: librdkafka1 librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl 2022-06-29T15:39:30.316 INFO:teuthology.orchestra.run.smithi164.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-blinker python3-bs4 2022-06-29T15:39:30.316 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common 2022-06-29T15:39:30.316 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2022-06-29T15:39:30.316 INFO:teuthology.orchestra.run.smithi164.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2022-06-29T15:39:30.317 INFO:teuthology.orchestra.run.smithi164.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2022-06-29T15:39:30.317 INFO:teuthology.orchestra.run.smithi164.stdout: python3-mako python3-markupsafe python3-oauthlib python3-openssl 2022-06-29T15:39:30.317 INFO:teuthology.orchestra.run.smithi164.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2022-06-29T15:39:30.317 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2022-06-29T15:39:30.318 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2022-06-29T15:39:30.318 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2022-06-29T15:39:30.318 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2022-06-29T15:39:30.318 INFO:teuthology.orchestra.run.smithi164.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2022-06-29T15:39:30.319 INFO:teuthology.orchestra.run.smithi164.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2022-06-29T15:39:30.319 INFO:teuthology.orchestra.run.smithi164.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug socat 2022-06-29T15:39:30.319 INFO:teuthology.orchestra.run.smithi164.stdout: xmlstarlet 2022-06-29T15:39:30.319 INFO:teuthology.orchestra.run.smithi164.stdout:Suggested packages: 2022-06-29T15:39:30.319 INFO:teuthology.orchestra.run.smithi164.stdout: python3-influxdb python-attr-doc python-blinker-doc python-cryptography-doc 2022-06-29T15:39:30.320 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2022-06-29T15:39:30.320 INFO:teuthology.orchestra.run.smithi164.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2022-06-29T15:39:30.320 INFO:teuthology.orchestra.run.smithi164.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2022-06-29T15:39:30.320 INFO:teuthology.orchestra.run.smithi164.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2022-06-29T15:39:30.320 INFO:teuthology.orchestra.run.smithi164.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2022-06-29T15:39:30.321 INFO:teuthology.orchestra.run.smithi164.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2022-06-29T15:39:30.321 INFO:teuthology.orchestra.run.smithi164.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2022-06-29T15:39:30.321 INFO:teuthology.orchestra.run.smithi164.stdout:Recommended packages: 2022-06-29T15:39:30.321 INFO:teuthology.orchestra.run.smithi164.stdout: btrfs-tools 2022-06-29T15:39:30.332 INFO:teuthology.orchestra.run.smithi064.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2022-06-29T15:39:30.335 INFO:teuthology.orchestra.run.smithi099.stdout:The following NEW packages will be installed: 2022-06-29T15:39:30.335 INFO:teuthology.orchestra.run.smithi099.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2022-06-29T15:39:30.336 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2022-06-29T15:39:30.336 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2022-06-29T15:39:30.337 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mon ceph-osd ceph-test cephadm cephfs-shell jq libcephfs-dev libcephfs2 2022-06-29T15:39:30.337 INFO:teuthology.orchestra.run.smithi099.stdout: libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 libleveldb1d 2022-06-29T15:39:30.338 INFO:teuthology.orchestra.run.smithi099.stdout: liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 librdkafka1 2022-06-29T15:39:30.338 INFO:teuthology.orchestra.run.smithi099.stdout: librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl python3-atomicwrites 2022-06-29T15:39:30.339 INFO:teuthology.orchestra.run.smithi099.stdout: python3-attr python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2022-06-29T15:39:30.339 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cherrypy3 2022-06-29T15:39:30.339 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cmd2 python3-colorama python3-cryptography python3-decorator 2022-06-29T15:39:30.340 INFO:teuthology.orchestra.run.smithi099.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2022-06-29T15:39:30.340 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2022-06-29T15:39:30.340 INFO:teuthology.orchestra.run.smithi099.stdout: python3-markupsafe python3-oauthlib python3-openssl python3-packaging 2022-06-29T15:39:30.340 INFO:teuthology.orchestra.run.smithi099.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2022-06-29T15:39:30.340 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2022-06-29T15:39:30.341 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2022-06-29T15:39:30.341 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests-oauthlib 2022-06-29T15:39:30.341 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rgw python3-routes python3-rsa python3-scipy python3-simplegeneric 2022-06-29T15:39:30.341 INFO:teuthology.orchestra.run.smithi099.stdout: python3-simplejson python3-singledispatch python3-sklearn 2022-06-29T15:39:30.342 INFO:teuthology.orchestra.run.smithi099.stdout: python3-sklearn-lib python3-soupsieve python3-tempita python3-waitress 2022-06-29T15:39:30.342 INFO:teuthology.orchestra.run.smithi099.stdout: python3-wcwidth python3-webencodings python3-webob python3-websocket 2022-06-29T15:39:30.342 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webtest python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2022-06-29T15:39:30.342 INFO:teuthology.orchestra.run.smithi099.stdout:The following packages will be upgraded: 2022-06-29T15:39:30.343 INFO:teuthology.orchestra.run.smithi099.stdout: librados2 librbd1 2022-06-29T15:39:30.362 INFO:teuthology.orchestra.run.smithi064.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2022-06-29T15:39:30.363 INFO:teuthology.orchestra.run.smithi064.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2022-06-29T15:39:30.367 INFO:teuthology.orchestra.run.smithi064.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2022-06-29T15:39:30.389 INFO:teuthology.orchestra.run.smithi064.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 libleveldb1d amd64 1.22-3ubuntu2 [139 kB] 2022-06-29T15:39:30.393 INFO:teuthology.orchestra.run.smithi064.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2022-06-29T15:39:30.394 INFO:teuthology.orchestra.run.smithi064.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2022-06-29T15:39:30.398 INFO:teuthology.orchestra.run.smithi064.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2022-06-29T15:39:30.399 INFO:teuthology.orchestra.run.smithi064.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2022-06-29T15:39:30.400 INFO:teuthology.orchestra.run.smithi064.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2022-06-29T15:39:30.404 INFO:teuthology.orchestra.run.smithi064.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jwt all 1.7.1-2ubuntu2 [17.4 kB] 2022-06-29T15:39:30.404 INFO:teuthology.orchestra.run.smithi064.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2022-06-29T15:39:30.405 INFO:teuthology.orchestra.run.smithi064.stdout:Get:16 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 15.2.16-304-gf25ffac7-1focal [1,699 kB] 2022-06-29T15:39:30.405 INFO:teuthology.orchestra.run.smithi064.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2022-06-29T15:39:30.407 INFO:teuthology.orchestra.run.smithi099.stdout:2 upgraded, 101 newly installed, 0 to remove and 7 not upgraded. 2022-06-29T15:39:30.407 INFO:teuthology.orchestra.run.smithi099.stdout:Need to get 153 MB of archives. 2022-06-29T15:39:30.407 INFO:teuthology.orchestra.run.smithi099.stdout:After this operation, 699 MB of additional disk space will be used. 2022-06-29T15:39:30.407 INFO:teuthology.orchestra.run.smithi099.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2022-06-29T15:39:30.413 INFO:teuthology.orchestra.run.smithi064.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2 [59.1 kB] 2022-06-29T15:39:30.414 INFO:teuthology.orchestra.run.smithi064.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2022-06-29T15:39:30.414 INFO:teuthology.orchestra.run.smithi064.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2022-06-29T15:39:30.419 INFO:teuthology.orchestra.run.smithi164.stdout:The following NEW packages will be installed: 2022-06-29T15:39:30.419 INFO:teuthology.orchestra.run.smithi164.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2022-06-29T15:39:30.420 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2022-06-29T15:39:30.420 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2022-06-29T15:39:30.421 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mon ceph-osd ceph-test cephadm cephfs-shell jq libcephfs-dev libcephfs2 2022-06-29T15:39:30.422 INFO:teuthology.orchestra.run.smithi164.stdout: libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 libleveldb1d 2022-06-29T15:39:30.422 INFO:teuthology.orchestra.run.smithi164.stdout: liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 librdkafka1 2022-06-29T15:39:30.422 INFO:teuthology.orchestra.run.smithi164.stdout: librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl python3-atomicwrites 2022-06-29T15:39:30.423 INFO:teuthology.orchestra.run.smithi164.stdout: python3-attr python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2022-06-29T15:39:30.423 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cherrypy3 2022-06-29T15:39:30.423 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cmd2 python3-colorama python3-cryptography python3-decorator 2022-06-29T15:39:30.423 INFO:teuthology.orchestra.run.smithi164.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2022-06-29T15:39:30.424 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2022-06-29T15:39:30.424 INFO:teuthology.orchestra.run.smithi164.stdout: python3-markupsafe python3-oauthlib python3-openssl python3-packaging 2022-06-29T15:39:30.424 INFO:teuthology.orchestra.run.smithi164.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2022-06-29T15:39:30.424 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2022-06-29T15:39:30.425 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2022-06-29T15:39:30.425 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests-oauthlib 2022-06-29T15:39:30.425 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rgw python3-routes python3-rsa python3-scipy python3-simplegeneric 2022-06-29T15:39:30.425 INFO:teuthology.orchestra.run.smithi164.stdout: python3-simplejson python3-singledispatch python3-sklearn 2022-06-29T15:39:30.425 INFO:teuthology.orchestra.run.smithi164.stdout: python3-sklearn-lib python3-soupsieve python3-tempita python3-waitress 2022-06-29T15:39:30.426 INFO:teuthology.orchestra.run.smithi164.stdout: python3-wcwidth python3-webencodings python3-webob python3-websocket 2022-06-29T15:39:30.426 INFO:teuthology.orchestra.run.smithi164.stdout: python3-webtest python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2022-06-29T15:39:30.426 INFO:teuthology.orchestra.run.smithi164.stdout:The following packages will be upgraded: 2022-06-29T15:39:30.426 INFO:teuthology.orchestra.run.smithi164.stdout: librados2 librbd1 2022-06-29T15:39:30.430 INFO:teuthology.orchestra.run.smithi064.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2022-06-29T15:39:30.431 INFO:teuthology.orchestra.run.smithi064.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2022-06-29T15:39:30.432 INFO:teuthology.orchestra.run.smithi064.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2022-06-29T15:39:30.433 INFO:teuthology.orchestra.run.smithi099.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2022-06-29T15:39:30.434 INFO:teuthology.orchestra.run.smithi064.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2022-06-29T15:39:30.435 INFO:teuthology.orchestra.run.smithi064.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2022-06-29T15:39:30.437 INFO:teuthology.orchestra.run.smithi064.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2022-06-29T15:39:30.440 INFO:teuthology.orchestra.run.smithi099.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2022-06-29T15:39:30.440 INFO:teuthology.orchestra.run.smithi064.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2022-06-29T15:39:30.447 INFO:teuthology.orchestra.run.smithi064.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2022-06-29T15:39:30.447 INFO:teuthology.orchestra.run.smithi064.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2022-06-29T15:39:30.448 INFO:teuthology.orchestra.run.smithi064.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2022-06-29T15:39:30.463 INFO:teuthology.orchestra.run.smithi064.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-werkzeug all 0.16.1+dfsg1-2 [183 kB] 2022-06-29T15:39:30.464 INFO:teuthology.orchestra.run.smithi047.stdout:2 upgraded, 101 newly installed, 0 to remove and 7 not upgraded. 2022-06-29T15:39:30.465 INFO:teuthology.orchestra.run.smithi047.stdout:Need to get 153 MB of archives. 2022-06-29T15:39:30.465 INFO:teuthology.orchestra.run.smithi047.stdout:After this operation, 699 MB of additional disk space will be used. 2022-06-29T15:39:30.465 INFO:teuthology.orchestra.run.smithi047.stdout:Get:1 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 15.2.16-304-gf25ffac7-1focal [1,699 kB] 2022-06-29T15:39:30.466 INFO:teuthology.orchestra.run.smithi064.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2022-06-29T15:39:30.468 INFO:teuthology.orchestra.run.smithi064.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2022-06-29T15:39:30.469 INFO:teuthology.orchestra.run.smithi064.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2022-06-29T15:39:30.471 INFO:teuthology.orchestra.run.smithi064.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2022-06-29T15:39:30.473 INFO:teuthology.orchestra.run.smithi064.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2022-06-29T15:39:30.475 INFO:teuthology.orchestra.run.smithi099.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2022-06-29T15:39:30.475 INFO:teuthology.orchestra.run.smithi064.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2022-06-29T15:39:30.480 INFO:teuthology.orchestra.run.smithi064.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2022-06-29T15:39:30.481 INFO:teuthology.orchestra.run.smithi064.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2022-06-29T15:39:30.487 INFO:teuthology.orchestra.run.smithi047.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2022-06-29T15:39:30.520 INFO:teuthology.orchestra.run.smithi099.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 15.2.16-304-gf25ffac7-1focal [1,699 kB] 2022-06-29T15:39:30.524 INFO:teuthology.orchestra.run.smithi099.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2022-06-29T15:39:30.528 INFO:teuthology.orchestra.run.smithi099.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2022-06-29T15:39:30.534 INFO:teuthology.orchestra.run.smithi099.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2022-06-29T15:39:30.554 INFO:teuthology.orchestra.run.smithi064.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2022-06-29T15:39:30.569 INFO:teuthology.orchestra.run.smithi047.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2022-06-29T15:39:30.570 INFO:teuthology.orchestra.run.smithi064.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2022-06-29T15:39:30.570 INFO:teuthology.orchestra.run.smithi064.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2022-06-29T15:39:30.576 INFO:teuthology.orchestra.run.smithi099.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 libleveldb1d amd64 1.22-3ubuntu2 [139 kB] 2022-06-29T15:39:30.582 INFO:teuthology.orchestra.run.smithi064.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2022-06-29T15:39:30.582 INFO:teuthology.orchestra.run.smithi064.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2022-06-29T15:39:30.583 INFO:teuthology.orchestra.run.smithi099.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2022-06-29T15:39:30.586 INFO:teuthology.orchestra.run.smithi099.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2022-06-29T15:39:30.599 INFO:teuthology.orchestra.run.smithi099.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2022-06-29T15:39:30.601 INFO:teuthology.orchestra.run.smithi099.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2022-06-29T15:39:30.603 INFO:teuthology.orchestra.run.smithi099.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2022-06-29T15:39:30.606 INFO:teuthology.orchestra.run.smithi047.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2022-06-29T15:39:30.611 INFO:teuthology.orchestra.run.smithi164.stdout:2 upgraded, 101 newly installed, 0 to remove and 7 not upgraded. 2022-06-29T15:39:30.611 INFO:teuthology.orchestra.run.smithi164.stdout:Need to get 153 MB of archives. 2022-06-29T15:39:30.611 INFO:teuthology.orchestra.run.smithi164.stdout:After this operation, 699 MB of additional disk space will be used. 2022-06-29T15:39:30.611 INFO:teuthology.orchestra.run.smithi164.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2022-06-29T15:39:30.612 INFO:teuthology.orchestra.run.smithi099.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jwt all 1.7.1-2ubuntu2 [17.4 kB] 2022-06-29T15:39:30.612 INFO:teuthology.orchestra.run.smithi099.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2022-06-29T15:39:30.613 INFO:teuthology.orchestra.run.smithi099.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2022-06-29T15:39:30.626 INFO:teuthology.orchestra.run.smithi099.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2 [59.1 kB] 2022-06-29T15:39:30.628 INFO:teuthology.orchestra.run.smithi099.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2022-06-29T15:39:30.628 INFO:teuthology.orchestra.run.smithi099.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2022-06-29T15:39:30.633 INFO:teuthology.orchestra.run.smithi164.stdout:Get:2 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 15.2.16-304-gf25ffac7-1focal [1,699 kB] 2022-06-29T15:39:30.642 INFO:teuthology.orchestra.run.smithi064.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2022-06-29T15:39:30.642 INFO:teuthology.orchestra.run.smithi064.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2022-06-29T15:39:30.643 INFO:teuthology.orchestra.run.smithi064.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2022-06-29T15:39:30.644 INFO:teuthology.orchestra.run.smithi064.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2022-06-29T15:39:30.645 INFO:teuthology.orchestra.run.smithi064.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2022-06-29T15:39:30.646 INFO:teuthology.orchestra.run.smithi064.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2022-06-29T15:39:30.647 INFO:teuthology.orchestra.run.smithi064.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2022-06-29T15:39:30.657 INFO:teuthology.orchestra.run.smithi099.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2022-06-29T15:39:30.658 INFO:teuthology.orchestra.run.smithi064.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2022-06-29T15:39:30.659 INFO:teuthology.orchestra.run.smithi099.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2022-06-29T15:39:30.661 INFO:teuthology.orchestra.run.smithi099.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2022-06-29T15:39:30.662 INFO:teuthology.orchestra.run.smithi064.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2022-06-29T15:39:30.663 INFO:teuthology.orchestra.run.smithi099.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2022-06-29T15:39:30.664 INFO:teuthology.orchestra.run.smithi064.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2022-06-29T15:39:30.665 INFO:teuthology.orchestra.run.smithi099.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2022-06-29T15:39:30.666 INFO:teuthology.orchestra.run.smithi099.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2022-06-29T15:39:30.673 INFO:teuthology.orchestra.run.smithi099.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2022-06-29T15:39:30.675 INFO:teuthology.orchestra.run.smithi064.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2022-06-29T15:39:30.676 INFO:teuthology.orchestra.run.smithi064.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2022-06-29T15:39:30.678 INFO:teuthology.orchestra.run.smithi064.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/main amd64 libxslt1.1 amd64 1.1.34-4 [152 kB] 2022-06-29T15:39:30.680 INFO:teuthology.orchestra.run.smithi064.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2022-06-29T15:39:30.683 INFO:teuthology.orchestra.run.smithi064.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2022-06-29T15:39:30.684 INFO:teuthology.orchestra.run.smithi064.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2022-06-29T15:39:30.688 INFO:teuthology.orchestra.run.smithi064.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2022-06-29T15:39:30.691 INFO:teuthology.orchestra.run.smithi099.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2022-06-29T15:39:30.692 INFO:teuthology.orchestra.run.smithi099.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2022-06-29T15:39:30.694 INFO:teuthology.orchestra.run.smithi164.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2022-06-29T15:39:30.694 INFO:teuthology.orchestra.run.smithi099.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2022-06-29T15:39:30.695 INFO:teuthology.orchestra.run.smithi064.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2022-06-29T15:39:30.696 INFO:teuthology.orchestra.run.smithi064.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2022-06-29T15:39:30.696 INFO:teuthology.orchestra.run.smithi064.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2022-06-29T15:39:30.709 INFO:teuthology.orchestra.run.smithi064.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2022-06-29T15:39:30.723 INFO:teuthology.orchestra.run.smithi099.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-werkzeug all 0.16.1+dfsg1-2 [183 kB] 2022-06-29T15:39:30.726 INFO:teuthology.orchestra.run.smithi064.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2022-06-29T15:39:30.727 INFO:teuthology.orchestra.run.smithi099.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2022-06-29T15:39:30.728 INFO:teuthology.orchestra.run.smithi064.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2022-06-29T15:39:30.728 INFO:teuthology.orchestra.run.smithi064.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2022-06-29T15:39:30.729 INFO:teuthology.orchestra.run.smithi064.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2022-06-29T15:39:30.730 INFO:teuthology.orchestra.run.smithi164.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2022-06-29T15:39:30.730 INFO:teuthology.orchestra.run.smithi099.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2022-06-29T15:39:30.731 INFO:teuthology.orchestra.run.smithi099.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2022-06-29T15:39:30.731 INFO:teuthology.orchestra.run.smithi064.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2022-06-29T15:39:30.732 INFO:teuthology.orchestra.run.smithi064.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2022-06-29T15:39:30.732 INFO:teuthology.orchestra.run.smithi064.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2022-06-29T15:39:30.732 INFO:teuthology.orchestra.run.smithi064.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2022-06-29T15:39:30.733 INFO:teuthology.orchestra.run.smithi099.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2022-06-29T15:39:30.734 INFO:teuthology.orchestra.run.smithi099.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2022-06-29T15:39:30.736 INFO:teuthology.orchestra.run.smithi099.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2022-06-29T15:39:30.740 INFO:teuthology.orchestra.run.smithi047.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2022-06-29T15:39:30.758 INFO:teuthology.orchestra.run.smithi099.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2022-06-29T15:39:30.758 INFO:teuthology.orchestra.run.smithi099.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2022-06-29T15:39:30.862 INFO:teuthology.orchestra.run.smithi164.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2022-06-29T15:39:30.870 INFO:teuthology.orchestra.run.smithi047.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2022-06-29T15:39:30.880 INFO:teuthology.orchestra.run.smithi047.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2022-06-29T15:39:30.895 INFO:teuthology.orchestra.run.smithi047.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2022-06-29T15:39:30.993 INFO:teuthology.orchestra.run.smithi164.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2022-06-29T15:39:30.998 INFO:teuthology.orchestra.run.smithi047.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 libleveldb1d amd64 1.22-3ubuntu2 [139 kB] 2022-06-29T15:39:31.001 INFO:teuthology.orchestra.run.smithi164.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2022-06-29T15:39:31.017 INFO:teuthology.orchestra.run.smithi164.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2022-06-29T15:39:31.019 INFO:teuthology.orchestra.run.smithi047.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2022-06-29T15:39:31.025 INFO:teuthology.orchestra.run.smithi047.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2022-06-29T15:39:31.029 INFO:teuthology.orchestra.run.smithi099.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2022-06-29T15:39:31.050 INFO:teuthology.orchestra.run.smithi047.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2022-06-29T15:39:31.054 INFO:teuthology.orchestra.run.smithi047.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2022-06-29T15:39:31.058 INFO:teuthology.orchestra.run.smithi047.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2022-06-29T15:39:31.073 INFO:teuthology.orchestra.run.smithi047.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jwt all 1.7.1-2ubuntu2 [17.4 kB] 2022-06-29T15:39:31.074 INFO:teuthology.orchestra.run.smithi099.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2022-06-29T15:39:31.074 INFO:teuthology.orchestra.run.smithi047.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2022-06-29T15:39:31.075 INFO:teuthology.orchestra.run.smithi047.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2022-06-29T15:39:31.076 INFO:teuthology.orchestra.run.smithi099.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2022-06-29T15:39:31.111 INFO:teuthology.orchestra.run.smithi099.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2022-06-29T15:39:31.111 INFO:teuthology.orchestra.run.smithi099.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2022-06-29T15:39:31.112 INFO:teuthology.orchestra.run.smithi099.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2022-06-29T15:39:31.113 INFO:teuthology.orchestra.run.smithi099.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2022-06-29T15:39:31.113 INFO:teuthology.orchestra.run.smithi047.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2 [59.1 kB] 2022-06-29T15:39:31.114 INFO:teuthology.orchestra.run.smithi099.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2022-06-29T15:39:31.115 INFO:teuthology.orchestra.run.smithi099.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2022-06-29T15:39:31.115 INFO:teuthology.orchestra.run.smithi099.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2022-06-29T15:39:31.116 INFO:teuthology.orchestra.run.smithi047.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2022-06-29T15:39:31.117 INFO:teuthology.orchestra.run.smithi047.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2022-06-29T15:39:31.120 INFO:teuthology.orchestra.run.smithi164.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 libleveldb1d amd64 1.22-3ubuntu2 [139 kB] 2022-06-29T15:39:31.121 INFO:teuthology.orchestra.run.smithi099.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2022-06-29T15:39:31.121 INFO:teuthology.orchestra.run.smithi099.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2022-06-29T15:39:31.140 INFO:teuthology.orchestra.run.smithi164.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2022-06-29T15:39:31.146 INFO:teuthology.orchestra.run.smithi164.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2022-06-29T15:39:31.152 INFO:teuthology.orchestra.run.smithi099.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2022-06-29T15:39:31.158 INFO:teuthology.orchestra.run.smithi099.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2022-06-29T15:39:31.161 INFO:teuthology.orchestra.run.smithi099.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2022-06-29T15:39:31.163 INFO:teuthology.orchestra.run.smithi099.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2022-06-29T15:39:31.163 INFO:teuthology.orchestra.run.smithi099.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2022-06-29T15:39:31.166 INFO:teuthology.orchestra.run.smithi099.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/main amd64 libxslt1.1 amd64 1.1.34-4 [152 kB] 2022-06-29T15:39:31.167 INFO:teuthology.orchestra.run.smithi099.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2022-06-29T15:39:31.169 INFO:teuthology.orchestra.run.smithi099.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2022-06-29T15:39:31.172 INFO:teuthology.orchestra.run.smithi164.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2022-06-29T15:39:31.175 INFO:teuthology.orchestra.run.smithi164.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2022-06-29T15:39:31.179 INFO:teuthology.orchestra.run.smithi164.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2022-06-29T15:39:31.190 INFO:teuthology.orchestra.run.smithi099.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2022-06-29T15:39:31.193 INFO:teuthology.orchestra.run.smithi047.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2022-06-29T15:39:31.194 INFO:teuthology.orchestra.run.smithi164.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jwt all 1.7.1-2ubuntu2 [17.4 kB] 2022-06-29T15:39:31.195 INFO:teuthology.orchestra.run.smithi164.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2022-06-29T15:39:31.195 INFO:teuthology.orchestra.run.smithi099.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2022-06-29T15:39:31.196 INFO:teuthology.orchestra.run.smithi047.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2022-06-29T15:39:31.197 INFO:teuthology.orchestra.run.smithi164.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2022-06-29T15:39:31.197 INFO:teuthology.orchestra.run.smithi047.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2022-06-29T15:39:31.201 INFO:teuthology.orchestra.run.smithi047.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2022-06-29T15:39:31.204 INFO:teuthology.orchestra.run.smithi047.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2022-06-29T15:39:31.205 INFO:teuthology.orchestra.run.smithi047.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2022-06-29T15:39:31.215 INFO:teuthology.orchestra.run.smithi064.stdout:Get:74 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 15.2.16-304-gf25ffac7-1focal [3,257 kB] 2022-06-29T15:39:31.223 INFO:teuthology.orchestra.run.smithi099.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2022-06-29T15:39:31.223 INFO:teuthology.orchestra.run.smithi099.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2022-06-29T15:39:31.224 INFO:teuthology.orchestra.run.smithi047.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2022-06-29T15:39:31.225 INFO:teuthology.orchestra.run.smithi099.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2022-06-29T15:39:31.226 INFO:teuthology.orchestra.run.smithi099.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2022-06-29T15:39:31.235 INFO:teuthology.orchestra.run.smithi164.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2 [59.1 kB] 2022-06-29T15:39:31.238 INFO:teuthology.orchestra.run.smithi164.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2022-06-29T15:39:31.239 INFO:teuthology.orchestra.run.smithi164.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2022-06-29T15:39:31.244 INFO:teuthology.orchestra.run.smithi099.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2022-06-29T15:39:31.245 INFO:teuthology.orchestra.run.smithi099.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2022-06-29T15:39:31.246 INFO:teuthology.orchestra.run.smithi099.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2022-06-29T15:39:31.246 INFO:teuthology.orchestra.run.smithi099.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2022-06-29T15:39:31.266 INFO:teuthology.orchestra.run.smithi099.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2022-06-29T15:39:31.266 INFO:teuthology.orchestra.run.smithi099.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2022-06-29T15:39:31.273 INFO:teuthology.orchestra.run.smithi047.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2022-06-29T15:39:31.274 INFO:teuthology.orchestra.run.smithi047.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2022-06-29T15:39:31.276 INFO:teuthology.orchestra.run.smithi047.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2022-06-29T15:39:31.296 INFO:teuthology.orchestra.run.smithi099.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2022-06-29T15:39:31.299 INFO:teuthology.orchestra.run.smithi099.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2022-06-29T15:39:31.300 INFO:teuthology.orchestra.run.smithi047.stdout:Get:31 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 15.2.16-304-gf25ffac7-1focal [3,257 kB] 2022-06-29T15:39:31.315 INFO:teuthology.orchestra.run.smithi164.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2022-06-29T15:39:31.318 INFO:teuthology.orchestra.run.smithi164.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2022-06-29T15:39:31.319 INFO:teuthology.orchestra.run.smithi164.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2022-06-29T15:39:31.322 INFO:teuthology.orchestra.run.smithi164.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2022-06-29T15:39:31.326 INFO:teuthology.orchestra.run.smithi164.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2022-06-29T15:39:31.327 INFO:teuthology.orchestra.run.smithi164.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2022-06-29T15:39:31.345 INFO:teuthology.orchestra.run.smithi164.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2022-06-29T15:39:31.354 INFO:teuthology.orchestra.run.smithi047.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-werkzeug all 0.16.1+dfsg1-2 [183 kB] 2022-06-29T15:39:31.358 INFO:teuthology.orchestra.run.smithi099.stdout:Get:74 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 15.2.16-304-gf25ffac7-1focal [3,257 kB] 2022-06-29T15:39:31.360 INFO:teuthology.orchestra.run.smithi047.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2022-06-29T15:39:31.363 INFO:teuthology.orchestra.run.smithi047.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2022-06-29T15:39:31.364 INFO:teuthology.orchestra.run.smithi047.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2022-06-29T15:39:31.367 INFO:teuthology.orchestra.run.smithi047.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2022-06-29T15:39:31.370 INFO:teuthology.orchestra.run.smithi047.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2022-06-29T15:39:31.373 INFO:teuthology.orchestra.run.smithi047.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2022-06-29T15:39:31.395 INFO:teuthology.orchestra.run.smithi164.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2022-06-29T15:39:31.396 INFO:teuthology.orchestra.run.smithi164.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2022-06-29T15:39:31.397 INFO:teuthology.orchestra.run.smithi164.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2022-06-29T15:39:31.434 INFO:teuthology.orchestra.run.smithi047.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2022-06-29T15:39:31.434 INFO:teuthology.orchestra.run.smithi047.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2022-06-29T15:39:31.476 INFO:teuthology.orchestra.run.smithi164.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-werkzeug all 0.16.1+dfsg1-2 [183 kB] 2022-06-29T15:39:31.481 INFO:teuthology.orchestra.run.smithi164.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2022-06-29T15:39:31.485 INFO:teuthology.orchestra.run.smithi164.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2022-06-29T15:39:31.486 INFO:teuthology.orchestra.run.smithi164.stdout:Get:34 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 15.2.16-304-gf25ffac7-1focal [3,257 kB] 2022-06-29T15:39:31.486 INFO:teuthology.orchestra.run.smithi164.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2022-06-29T15:39:31.489 INFO:teuthology.orchestra.run.smithi164.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2022-06-29T15:39:31.492 INFO:teuthology.orchestra.run.smithi164.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2022-06-29T15:39:31.495 INFO:teuthology.orchestra.run.smithi164.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2022-06-29T15:39:31.555 INFO:teuthology.orchestra.run.smithi164.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2022-06-29T15:39:31.556 INFO:teuthology.orchestra.run.smithi164.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2022-06-29T15:39:31.783 INFO:teuthology.orchestra.run.smithi064.stdout:Get:75 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 15.2.16-304-gf25ffac7-1focal [463 kB] 2022-06-29T15:39:31.813 INFO:teuthology.orchestra.run.smithi047.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2022-06-29T15:39:31.826 INFO:teuthology.orchestra.run.smithi064.stdout:Get:76 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 15.2.16-304-gf25ffac7-1focal [366 kB] 2022-06-29T15:39:31.836 INFO:teuthology.orchestra.run.smithi047.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2022-06-29T15:39:31.838 INFO:teuthology.orchestra.run.smithi047.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2022-06-29T15:39:31.868 INFO:teuthology.orchestra.run.smithi064.stdout:Get:77 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 15.2.16-304-gf25ffac7-1focal [20.9 kB] 2022-06-29T15:39:31.868 INFO:teuthology.orchestra.run.smithi064.stdout:Get:78 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 15.2.16-304-gf25ffac7-1focal [156 kB] 2022-06-29T15:39:31.870 INFO:teuthology.orchestra.run.smithi047.stdout:Get:44 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 15.2.16-304-gf25ffac7-1focal [463 kB] 2022-06-29T15:39:31.887 INFO:teuthology.orchestra.run.smithi164.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2022-06-29T15:39:31.902 INFO:teuthology.orchestra.run.smithi064.stdout:Get:79 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 15.2.16-304-gf25ffac7-1focal [29.4 kB] 2022-06-29T15:39:31.904 INFO:teuthology.orchestra.run.smithi064.stdout:Get:80 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 15.2.16-304-gf25ffac7-1focal [364 kB] 2022-06-29T15:39:31.905 INFO:teuthology.orchestra.run.smithi047.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2022-06-29T15:39:31.905 INFO:teuthology.orchestra.run.smithi047.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2022-06-29T15:39:31.906 INFO:teuthology.orchestra.run.smithi047.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2022-06-29T15:39:31.906 INFO:teuthology.orchestra.run.smithi047.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2022-06-29T15:39:31.907 INFO:teuthology.orchestra.run.smithi047.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2022-06-29T15:39:31.915 INFO:teuthology.orchestra.run.smithi047.stdout:Get:50 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 15.2.16-304-gf25ffac7-1focal [366 kB] 2022-06-29T15:39:31.920 INFO:teuthology.orchestra.run.smithi047.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2022-06-29T15:39:31.920 INFO:teuthology.orchestra.run.smithi047.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2022-06-29T15:39:31.929 INFO:teuthology.orchestra.run.smithi099.stdout:Get:75 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 15.2.16-304-gf25ffac7-1focal [463 kB] 2022-06-29T15:39:31.942 INFO:teuthology.orchestra.run.smithi064.stdout:Get:81 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 15.2.16-304-gf25ffac7-1focal [3,145 kB] 2022-06-29T15:39:31.944 INFO:teuthology.orchestra.run.smithi164.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2022-06-29T15:39:31.945 INFO:teuthology.orchestra.run.smithi164.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2022-06-29T15:39:31.955 INFO:teuthology.orchestra.run.smithi047.stdout:Get:53 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 15.2.16-304-gf25ffac7-1focal [20.9 kB] 2022-06-29T15:39:31.956 INFO:teuthology.orchestra.run.smithi047.stdout:Get:54 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 15.2.16-304-gf25ffac7-1focal [156 kB] 2022-06-29T15:39:31.973 INFO:teuthology.orchestra.run.smithi099.stdout:Get:76 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 15.2.16-304-gf25ffac7-1focal [366 kB] 2022-06-29T15:39:31.975 INFO:teuthology.orchestra.run.smithi047.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2022-06-29T15:39:31.975 INFO:teuthology.orchestra.run.smithi047.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2022-06-29T15:39:31.983 INFO:teuthology.orchestra.run.smithi164.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2022-06-29T15:39:31.983 INFO:teuthology.orchestra.run.smithi164.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2022-06-29T15:39:31.988 INFO:teuthology.orchestra.run.smithi047.stdout:Get:57 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 15.2.16-304-gf25ffac7-1focal [29.4 kB] 2022-06-29T15:39:31.990 INFO:teuthology.orchestra.run.smithi047.stdout:Get:58 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 15.2.16-304-gf25ffac7-1focal [364 kB] 2022-06-29T15:39:32.007 INFO:teuthology.orchestra.run.smithi164.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2022-06-29T15:39:32.008 INFO:teuthology.orchestra.run.smithi164.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2022-06-29T15:39:32.008 INFO:teuthology.orchestra.run.smithi164.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2022-06-29T15:39:32.011 INFO:teuthology.orchestra.run.smithi164.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2022-06-29T15:39:32.011 INFO:teuthology.orchestra.run.smithi164.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2022-06-29T15:39:32.024 INFO:teuthology.orchestra.run.smithi047.stdout:Get:59 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 15.2.16-304-gf25ffac7-1focal [3,145 kB] 2022-06-29T15:39:32.049 INFO:teuthology.orchestra.run.smithi164.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2022-06-29T15:39:32.050 INFO:teuthology.orchestra.run.smithi164.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2022-06-29T15:39:32.055 INFO:teuthology.orchestra.run.smithi047.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2022-06-29T15:39:32.060 INFO:teuthology.orchestra.run.smithi047.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2022-06-29T15:39:32.063 INFO:teuthology.orchestra.run.smithi047.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2022-06-29T15:39:32.067 INFO:teuthology.orchestra.run.smithi047.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2022-06-29T15:39:32.068 INFO:teuthology.orchestra.run.smithi047.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2022-06-29T15:39:32.071 INFO:teuthology.orchestra.run.smithi047.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/main amd64 libxslt1.1 amd64 1.1.34-4 [152 kB] 2022-06-29T15:39:32.072 INFO:teuthology.orchestra.run.smithi047.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2022-06-29T15:39:32.074 INFO:teuthology.orchestra.run.smithi047.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2022-06-29T15:39:32.129 INFO:teuthology.orchestra.run.smithi164.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2022-06-29T15:39:32.133 INFO:teuthology.orchestra.run.smithi164.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2022-06-29T15:39:32.135 INFO:teuthology.orchestra.run.smithi047.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2022-06-29T15:39:32.135 INFO:teuthology.orchestra.run.smithi164.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2022-06-29T15:39:32.138 INFO:teuthology.orchestra.run.smithi164.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2022-06-29T15:39:32.139 INFO:teuthology.orchestra.run.smithi164.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2022-06-29T15:39:32.141 INFO:teuthology.orchestra.run.smithi047.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2022-06-29T15:39:32.143 INFO:teuthology.orchestra.run.smithi164.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/main amd64 libxslt1.1 amd64 1.1.34-4 [152 kB] 2022-06-29T15:39:32.144 INFO:teuthology.orchestra.run.smithi164.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2022-06-29T15:39:32.146 INFO:teuthology.orchestra.run.smithi164.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2022-06-29T15:39:32.209 INFO:teuthology.orchestra.run.smithi164.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2022-06-29T15:39:32.214 INFO:teuthology.orchestra.run.smithi164.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2022-06-29T15:39:32.215 INFO:teuthology.orchestra.run.smithi047.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2022-06-29T15:39:32.216 INFO:teuthology.orchestra.run.smithi047.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2022-06-29T15:39:32.217 INFO:teuthology.orchestra.run.smithi047.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2022-06-29T15:39:32.219 INFO:teuthology.orchestra.run.smithi047.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2022-06-29T15:39:32.237 INFO:teuthology.orchestra.run.smithi047.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2022-06-29T15:39:32.237 INFO:teuthology.orchestra.run.smithi047.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2022-06-29T15:39:32.238 INFO:teuthology.orchestra.run.smithi047.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2022-06-29T15:39:32.238 INFO:teuthology.orchestra.run.smithi047.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2022-06-29T15:39:32.289 INFO:teuthology.orchestra.run.smithi164.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2022-06-29T15:39:32.298 INFO:teuthology.orchestra.run.smithi047.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2022-06-29T15:39:32.299 INFO:teuthology.orchestra.run.smithi047.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2022-06-29T15:39:32.299 INFO:teuthology.orchestra.run.smithi164.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2022-06-29T15:39:32.299 INFO:teuthology.orchestra.run.smithi164.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2022-06-29T15:39:32.300 INFO:teuthology.orchestra.run.smithi164.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2022-06-29T15:39:32.309 INFO:teuthology.orchestra.run.smithi164.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2022-06-29T15:39:32.310 INFO:teuthology.orchestra.run.smithi164.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2022-06-29T15:39:32.310 INFO:teuthology.orchestra.run.smithi164.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2022-06-29T15:39:32.311 INFO:teuthology.orchestra.run.smithi164.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2022-06-29T15:39:32.364 INFO:teuthology.orchestra.run.smithi099.stdout:Get:77 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 15.2.16-304-gf25ffac7-1focal [20.9 kB] 2022-06-29T15:39:32.364 INFO:teuthology.orchestra.run.smithi099.stdout:Get:78 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 15.2.16-304-gf25ffac7-1focal [156 kB] 2022-06-29T15:39:32.369 INFO:teuthology.orchestra.run.smithi164.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2022-06-29T15:39:32.370 INFO:teuthology.orchestra.run.smithi164.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2022-06-29T15:39:32.375 INFO:teuthology.orchestra.run.smithi047.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2022-06-29T15:39:32.379 INFO:teuthology.orchestra.run.smithi047.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2022-06-29T15:39:32.411 INFO:teuthology.orchestra.run.smithi099.stdout:Get:79 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 15.2.16-304-gf25ffac7-1focal [29.4 kB] 2022-06-29T15:39:32.412 INFO:teuthology.orchestra.run.smithi099.stdout:Get:80 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 15.2.16-304-gf25ffac7-1focal [364 kB] 2022-06-29T15:39:32.447 INFO:teuthology.orchestra.run.smithi164.stdout:Get:73 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 15.2.16-304-gf25ffac7-1focal [463 kB] 2022-06-29T15:39:32.448 INFO:teuthology.orchestra.run.smithi164.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2022-06-29T15:39:32.452 INFO:teuthology.orchestra.run.smithi164.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2022-06-29T15:39:32.508 INFO:teuthology.orchestra.run.smithi099.stdout:Get:81 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 15.2.16-304-gf25ffac7-1focal [3,145 kB] 2022-06-29T15:39:32.519 INFO:teuthology.orchestra.run.smithi064.stdout:Get:82 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 15.2.16-304-gf25ffac7-1focal [116 kB] 2022-06-29T15:39:32.519 INFO:teuthology.orchestra.run.smithi047.stdout:Get:82 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 15.2.16-304-gf25ffac7-1focal [116 kB] 2022-06-29T15:39:32.524 INFO:teuthology.orchestra.run.smithi064.stdout:Get:83 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 15.2.16-304-gf25ffac7-1focal [361 kB] 2022-06-29T15:39:32.525 INFO:teuthology.orchestra.run.smithi047.stdout:Get:83 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 15.2.16-304-gf25ffac7-1focal [361 kB] 2022-06-29T15:39:32.549 INFO:teuthology.orchestra.run.smithi164.stdout:Get:76 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 15.2.16-304-gf25ffac7-1focal [366 kB] 2022-06-29T15:39:32.566 INFO:teuthology.orchestra.run.smithi064.stdout:Get:84 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 15.2.16-304-gf25ffac7-1focal [20.1 MB] 2022-06-29T15:39:32.567 INFO:teuthology.orchestra.run.smithi047.stdout:Get:84 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 15.2.16-304-gf25ffac7-1focal [20.1 MB] 2022-06-29T15:39:32.609 INFO:teuthology.orchestra.run.smithi164.stdout:Get:77 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 15.2.16-304-gf25ffac7-1focal [20.9 kB] 2022-06-29T15:39:32.610 INFO:teuthology.orchestra.run.smithi164.stdout:Get:78 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 15.2.16-304-gf25ffac7-1focal [156 kB] 2022-06-29T15:39:32.661 INFO:teuthology.orchestra.run.smithi164.stdout:Get:79 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 15.2.16-304-gf25ffac7-1focal [29.4 kB] 2022-06-29T15:39:32.662 INFO:teuthology.orchestra.run.smithi164.stdout:Get:80 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 15.2.16-304-gf25ffac7-1focal [364 kB] 2022-06-29T15:39:32.763 INFO:teuthology.orchestra.run.smithi164.stdout:Get:81 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 15.2.16-304-gf25ffac7-1focal [3,145 kB] 2022-06-29T15:39:33.114 INFO:teuthology.orchestra.run.smithi099.stdout:Get:82 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 15.2.16-304-gf25ffac7-1focal [116 kB] 2022-06-29T15:39:33.183 INFO:teuthology.orchestra.run.smithi099.stdout:Get:83 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 15.2.16-304-gf25ffac7-1focal [361 kB] 2022-06-29T15:39:33.196 INFO:teuthology.orchestra.run.smithi099.stdout:Get:84 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 15.2.16-304-gf25ffac7-1focal [20.1 MB] 2022-06-29T15:39:33.420 INFO:teuthology.orchestra.run.smithi164.stdout:Get:82 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 15.2.16-304-gf25ffac7-1focal [116 kB] 2022-06-29T15:39:33.422 INFO:teuthology.orchestra.run.smithi164.stdout:Get:83 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 15.2.16-304-gf25ffac7-1focal [361 kB] 2022-06-29T15:39:33.511 INFO:teuthology.orchestra.run.smithi164.stdout:Get:84 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 15.2.16-304-gf25ffac7-1focal [20.1 MB] 2022-06-29T15:39:34.634 INFO:teuthology.orchestra.run.smithi047.stdout:Get:85 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 15.2.16-304-gf25ffac7-1focal [5,344 kB] 2022-06-29T15:39:34.693 INFO:teuthology.orchestra.run.smithi064.stdout:Get:85 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 15.2.16-304-gf25ffac7-1focal [5,344 kB] 2022-06-29T15:39:35.143 INFO:teuthology.orchestra.run.smithi064.stdout:Get:86 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 15.2.16-304-gf25ffac7-1focal [163 kB] 2022-06-29T15:39:35.147 INFO:teuthology.orchestra.run.smithi064.stdout:Get:87 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 15.2.16-304-gf25ffac7-1focal [1,402 kB] 2022-06-29T15:39:35.260 INFO:teuthology.orchestra.run.smithi047.stdout:Get:86 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 15.2.16-304-gf25ffac7-1focal [163 kB] 2022-06-29T15:39:35.281 INFO:teuthology.orchestra.run.smithi047.stdout:Get:87 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 15.2.16-304-gf25ffac7-1focal [1,402 kB] 2022-06-29T15:39:35.297 INFO:teuthology.orchestra.run.smithi064.stdout:Get:88 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 15.2.16-304-gf25ffac7-1focal [6,164 kB] 2022-06-29T15:39:35.441 INFO:teuthology.orchestra.run.smithi047.stdout:Get:88 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 15.2.16-304-gf25ffac7-1focal [6,164 kB] 2022-06-29T15:39:35.857 INFO:teuthology.orchestra.run.smithi064.stdout:Get:89 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 15.2.16-304-gf25ffac7-1focal [24.0 MB] 2022-06-29T15:39:36.184 INFO:teuthology.orchestra.run.smithi164.stdout:Get:85 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 15.2.16-304-gf25ffac7-1focal [5,344 kB] 2022-06-29T15:39:36.240 INFO:teuthology.orchestra.run.smithi047.stdout:Get:89 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 15.2.16-304-gf25ffac7-1focal [24.0 MB] 2022-06-29T15:39:36.791 INFO:teuthology.orchestra.run.smithi164.stdout:Get:86 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 15.2.16-304-gf25ffac7-1focal [163 kB] 2022-06-29T15:39:36.811 INFO:teuthology.orchestra.run.smithi164.stdout:Get:87 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 15.2.16-304-gf25ffac7-1focal [1,402 kB] 2022-06-29T15:39:36.974 INFO:teuthology.orchestra.run.smithi164.stdout:Get:88 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 15.2.16-304-gf25ffac7-1focal [6,164 kB] 2022-06-29T15:39:37.716 INFO:teuthology.orchestra.run.smithi099.stdout:Get:85 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 15.2.16-304-gf25ffac7-1focal [5,344 kB] 2022-06-29T15:39:37.777 INFO:teuthology.orchestra.run.smithi164.stdout:Get:89 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 15.2.16-304-gf25ffac7-1focal [24.0 MB] 2022-06-29T15:39:38.224 INFO:teuthology.orchestra.run.smithi064.stdout:Get:90 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 15.2.16-304-gf25ffac7-1focal [4,032 B] 2022-06-29T15:39:38.279 INFO:teuthology.orchestra.run.smithi064.stdout:Get:91 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 15.2.16-304-gf25ffac7-1focal [538 kB] 2022-06-29T15:39:38.290 INFO:teuthology.orchestra.run.smithi064.stdout:Get:92 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 15.2.16-304-gf25ffac7-1focal [2,007 kB] 2022-06-29T15:39:38.407 INFO:teuthology.orchestra.run.smithi064.stdout:Get:93 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 15.2.16-304-gf25ffac7-1focal [53.2 kB] 2022-06-29T15:39:38.408 INFO:teuthology.orchestra.run.smithi064.stdout:Get:94 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 15.2.16-304-gf25ffac7-1focal [59.1 kB] 2022-06-29T15:39:38.408 INFO:teuthology.orchestra.run.smithi064.stdout:Get:95 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 15.2.16-304-gf25ffac7-1focal [3,218 kB] 2022-06-29T15:39:38.725 INFO:teuthology.orchestra.run.smithi099.stdout:Get:86 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 15.2.16-304-gf25ffac7-1focal [163 kB] 2022-06-29T15:39:38.728 INFO:teuthology.orchestra.run.smithi064.stdout:Get:96 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-cloud all 15.2.16-304-gf25ffac7-1focal [34.4 kB] 2022-06-29T15:39:38.728 INFO:teuthology.orchestra.run.smithi064.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 15.2.16-304-gf25ffac7-1focal [7,866 kB] 2022-06-29T15:39:38.763 INFO:teuthology.orchestra.run.smithi099.stdout:Get:87 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 15.2.16-304-gf25ffac7-1focal [1,402 kB] 2022-06-29T15:39:39.066 INFO:teuthology.orchestra.run.smithi099.stdout:Get:88 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 15.2.16-304-gf25ffac7-1focal [6,164 kB] 2022-06-29T15:39:39.513 INFO:teuthology.orchestra.run.smithi064.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 15.2.16-304-gf25ffac7-1focal [4,144 B] 2022-06-29T15:39:39.514 INFO:teuthology.orchestra.run.smithi064.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 15.2.16-304-gf25ffac7-1focal [40.2 MB] 2022-06-29T15:39:39.604 INFO:teuthology.orchestra.run.smithi047.stdout:Get:90 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 15.2.16-304-gf25ffac7-1focal [4,032 B] 2022-06-29T15:39:39.663 INFO:teuthology.orchestra.run.smithi047.stdout:Get:91 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 15.2.16-304-gf25ffac7-1focal [538 kB] 2022-06-29T15:39:39.674 INFO:teuthology.orchestra.run.smithi047.stdout:Get:92 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 15.2.16-304-gf25ffac7-1focal [2,007 kB] 2022-06-29T15:39:39.912 INFO:teuthology.orchestra.run.smithi047.stdout:Get:93 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 15.2.16-304-gf25ffac7-1focal [53.2 kB] 2022-06-29T15:39:39.913 INFO:teuthology.orchestra.run.smithi047.stdout:Get:94 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 15.2.16-304-gf25ffac7-1focal [59.1 kB] 2022-06-29T15:39:39.914 INFO:teuthology.orchestra.run.smithi047.stdout:Get:95 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 15.2.16-304-gf25ffac7-1focal [3,218 kB] 2022-06-29T15:39:40.361 INFO:teuthology.orchestra.run.smithi047.stdout:Get:96 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-cloud all 15.2.16-304-gf25ffac7-1focal [34.4 kB] 2022-06-29T15:39:40.362 INFO:teuthology.orchestra.run.smithi047.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 15.2.16-304-gf25ffac7-1focal [7,866 kB] 2022-06-29T15:39:40.384 INFO:teuthology.orchestra.run.smithi099.stdout:Get:89 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 15.2.16-304-gf25ffac7-1focal [24.0 MB] 2022-06-29T15:39:40.800 INFO:teuthology.orchestra.run.smithi164.stdout:Get:90 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 15.2.16-304-gf25ffac7-1focal [4,032 B] 2022-06-29T15:39:40.800 INFO:teuthology.orchestra.run.smithi164.stdout:Get:91 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 15.2.16-304-gf25ffac7-1focal [538 kB] 2022-06-29T15:39:40.811 INFO:teuthology.orchestra.run.smithi164.stdout:Get:92 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 15.2.16-304-gf25ffac7-1focal [2,007 kB] 2022-06-29T15:39:41.056 INFO:teuthology.orchestra.run.smithi164.stdout:Get:93 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 15.2.16-304-gf25ffac7-1focal [53.2 kB] 2022-06-29T15:39:41.058 INFO:teuthology.orchestra.run.smithi164.stdout:Get:94 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 15.2.16-304-gf25ffac7-1focal [59.1 kB] 2022-06-29T15:39:41.060 INFO:teuthology.orchestra.run.smithi164.stdout:Get:95 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 15.2.16-304-gf25ffac7-1focal [3,218 kB] 2022-06-29T15:39:41.428 INFO:teuthology.orchestra.run.smithi164.stdout:Get:96 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-cloud all 15.2.16-304-gf25ffac7-1focal [34.4 kB] 2022-06-29T15:39:41.429 INFO:teuthology.orchestra.run.smithi164.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 15.2.16-304-gf25ffac7-1focal [7,866 kB] 2022-06-29T15:39:41.640 INFO:teuthology.orchestra.run.smithi047.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 15.2.16-304-gf25ffac7-1focal [4,144 B] 2022-06-29T15:39:41.641 INFO:teuthology.orchestra.run.smithi047.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 15.2.16-304-gf25ffac7-1focal [40.2 MB] 2022-06-29T15:39:42.382 INFO:teuthology.orchestra.run.smithi164.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 15.2.16-304-gf25ffac7-1focal [4,144 B] 2022-06-29T15:39:42.382 INFO:teuthology.orchestra.run.smithi164.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 15.2.16-304-gf25ffac7-1focal [40.2 MB] 2022-06-29T15:39:43.414 INFO:teuthology.orchestra.run.smithi064.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 15.2.16-304-gf25ffac7-1focal [15.9 kB] 2022-06-29T15:39:43.415 INFO:teuthology.orchestra.run.smithi064.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 15.2.16-304-gf25ffac7-1focal [18.1 kB] 2022-06-29T15:39:43.416 INFO:teuthology.orchestra.run.smithi064.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 15.2.16-304-gf25ffac7-1focal [9,668 kB] 2022-06-29T15:39:44.140 INFO:teuthology.orchestra.run.smithi064.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 15.2.16-304-gf25ffac7-1focal [60.9 kB] 2022-06-29T15:39:44.746 INFO:teuthology.orchestra.run.smithi064.stdout:Fetched 153 MB in 14s (11.0 MB/s) 2022-06-29T15:39:44.768 INFO:teuthology.orchestra.run.smithi099.stdout:Get:90 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 15.2.16-304-gf25ffac7-1focal [4,032 B] 2022-06-29T15:39:44.769 INFO:teuthology.orchestra.run.smithi099.stdout:Get:91 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 15.2.16-304-gf25ffac7-1focal [538 kB] 2022-06-29T15:39:44.781 INFO:teuthology.orchestra.run.smithi099.stdout:Get:92 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 15.2.16-304-gf25ffac7-1focal [2,007 kB] 2022-06-29T15:39:44.876 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-pyperclip. 2022-06-29T15:39:44.933 INFO:teuthology.orchestra.run.smithi064.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 138771 files and directories currently installed.) 2022-06-29T15:39:44.937 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2022-06-29T15:39:44.990 INFO:teuthology.orchestra.run.smithi099.stdout:Get:93 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 15.2.16-304-gf25ffac7-1focal [53.2 kB] 2022-06-29T15:39:45.015 INFO:teuthology.orchestra.run.smithi099.stdout:Get:94 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 15.2.16-304-gf25ffac7-1focal [59.1 kB] 2022-06-29T15:39:45.016 INFO:teuthology.orchestra.run.smithi099.stdout:Get:95 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 15.2.16-304-gf25ffac7-1focal [3,218 kB] 2022-06-29T15:39:45.124 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2022-06-29T15:39:45.356 INFO:teuthology.orchestra.run.smithi099.stdout:Get:96 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-cloud all 15.2.16-304-gf25ffac7-1focal [34.4 kB] 2022-06-29T15:39:45.357 INFO:teuthology.orchestra.run.smithi099.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 15.2.16-304-gf25ffac7-1focal [7,866 kB] 2022-06-29T15:39:45.444 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-wcwidth. 2022-06-29T15:39:45.461 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2022-06-29T15:39:45.502 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2022-06-29T15:39:45.872 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-cmd2. 2022-06-29T15:39:45.887 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2022-06-29T15:39:45.930 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2022-06-29T15:39:46.287 INFO:teuthology.orchestra.run.smithi099.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 15.2.16-304-gf25ffac7-1focal [4,144 B] 2022-06-29T15:39:46.288 INFO:teuthology.orchestra.run.smithi099.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 15.2.16-304-gf25ffac7-1focal [40.2 MB] 2022-06-29T15:39:46.299 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2022-06-29T15:39:46.319 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2022-06-29T15:39:46.383 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2022-06-29T15:39:46.697 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../004-librbd1_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:39:46.837 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking librbd1 (15.2.16-304-gf25ffac7-1focal) over (15.2.16-0ubuntu0.20.04.1) ... 2022-06-29T15:39:46.947 INFO:teuthology.orchestra.run.smithi047.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 15.2.16-304-gf25ffac7-1focal [15.9 kB] 2022-06-29T15:39:46.948 INFO:teuthology.orchestra.run.smithi047.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 15.2.16-304-gf25ffac7-1focal [18.1 kB] 2022-06-29T15:39:46.949 INFO:teuthology.orchestra.run.smithi047.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 15.2.16-304-gf25ffac7-1focal [9,668 kB] 2022-06-29T15:39:47.145 INFO:teuthology.orchestra.run.smithi164.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 15.2.16-304-gf25ffac7-1focal [15.9 kB] 2022-06-29T15:39:47.146 INFO:teuthology.orchestra.run.smithi164.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 15.2.16-304-gf25ffac7-1focal [18.1 kB] 2022-06-29T15:39:47.147 INFO:teuthology.orchestra.run.smithi164.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 15.2.16-304-gf25ffac7-1focal [9,668 kB] 2022-06-29T15:39:47.425 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../005-librados2_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:39:47.574 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking librados2 (15.2.16-304-gf25ffac7-1focal) over (15.2.16-0ubuntu0.20.04.1) ... 2022-06-29T15:39:48.211 INFO:teuthology.orchestra.run.smithi164.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 15.2.16-304-gf25ffac7-1focal [60.9 kB] 2022-06-29T15:39:48.245 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libcephfs2. 2022-06-29T15:39:48.264 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../006-libcephfs2_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:39:48.295 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libcephfs2 (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:39:48.485 INFO:teuthology.orchestra.run.smithi047.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 15.2.16-304-gf25ffac7-1focal [60.9 kB] 2022-06-29T15:39:48.622 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-rados. 2022-06-29T15:39:48.640 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../007-python3-rados_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:39:48.688 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-rados (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:39:48.797 INFO:teuthology.orchestra.run.smithi164.stdout:Fetched 153 MB in 18s (8,620 kB/s) 2022-06-29T15:39:48.936 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-pyperclip. 2022-06-29T15:39:48.992 INFO:teuthology.orchestra.run.smithi164.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 138771 files and directories currently installed.) 2022-06-29T15:39:48.996 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2022-06-29T15:39:49.043 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2022-06-29T15:39:49.058 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-ceph-argparse. 2022-06-29T15:39:49.074 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../008-python3-ceph-argparse_15.2.16-304-gf25ffac7-1focal_all.deb ... 2022-06-29T15:39:49.095 INFO:teuthology.orchestra.run.smithi047.stdout:Fetched 153 MB in 18s (8,432 kB/s) 2022-06-29T15:39:49.141 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-ceph-argparse (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:39:49.221 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-pyperclip. 2022-06-29T15:39:49.278 INFO:teuthology.orchestra.run.smithi047.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 138771 files and directories currently installed.) 2022-06-29T15:39:49.282 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2022-06-29T15:39:49.320 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2022-06-29T15:39:49.320 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-wcwidth. 2022-06-29T15:39:49.337 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2022-06-29T15:39:49.370 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2022-06-29T15:39:49.444 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-cephfs. 2022-06-29T15:39:49.461 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../009-python3-cephfs_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:39:49.496 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-cephfs (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:39:49.589 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-wcwidth. 2022-06-29T15:39:49.598 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-cmd2. 2022-06-29T15:39:49.605 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2022-06-29T15:39:49.613 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2022-06-29T15:39:49.639 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2022-06-29T15:39:49.648 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2022-06-29T15:39:49.907 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-ceph-common. 2022-06-29T15:39:49.925 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../010-python3-ceph-common_15.2.16-304-gf25ffac7-1focal_all.deb ... 2022-06-29T15:39:49.942 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-cmd2. 2022-06-29T15:39:49.959 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2022-06-29T15:39:49.965 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-ceph-common (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:39:50.018 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2022-06-29T15:39:50.033 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2022-06-29T15:39:50.034 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2022-06-29T15:39:50.118 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2022-06-29T15:39:50.326 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-prettytable. 2022-06-29T15:39:50.344 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../011-python3-prettytable_0.7.2-5_all.deb ... 2022-06-29T15:39:50.378 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2022-06-29T15:39:50.384 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2022-06-29T15:39:50.395 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2022-06-29T15:39:50.478 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../004-librbd1_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:39:50.503 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2022-06-29T15:39:50.613 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking librbd1 (15.2.16-304-gf25ffac7-1focal) over (15.2.16-0ubuntu0.20.04.1) ... 2022-06-29T15:39:50.671 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-rbd. 2022-06-29T15:39:50.688 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../012-python3-rbd_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:39:50.720 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-rbd (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:39:50.818 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../004-librbd1_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:39:50.932 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking librbd1 (15.2.16-304-gf25ffac7-1focal) over (15.2.16-0ubuntu0.20.04.1) ... 2022-06-29T15:39:51.065 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package librabbitmq4:amd64. 2022-06-29T15:39:51.081 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../013-librabbitmq4_0.10.0-1_amd64.deb ... 2022-06-29T15:39:51.115 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2022-06-29T15:39:51.127 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../005-librados2_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:39:51.234 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking librados2 (15.2.16-304-gf25ffac7-1focal) over (15.2.16-0ubuntu0.20.04.1) ... 2022-06-29T15:39:51.501 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package librdkafka1:amd64. 2022-06-29T15:39:51.519 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../005-librados2_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:39:51.520 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../014-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2022-06-29T15:39:51.559 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2022-06-29T15:39:51.644 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking librados2 (15.2.16-304-gf25ffac7-1focal) over (15.2.16-0ubuntu0.20.04.1) ... 2022-06-29T15:39:51.904 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libcephfs2. 2022-06-29T15:39:51.905 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package librgw2. 2022-06-29T15:39:51.919 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../015-librgw2_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:39:51.922 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../006-libcephfs2_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:39:51.962 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libcephfs2 (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:39:51.979 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking librgw2 (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:39:52.022 INFO:teuthology.orchestra.run.smithi099.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 15.2.16-304-gf25ffac7-1focal [15.9 kB] 2022-06-29T15:39:52.023 INFO:teuthology.orchestra.run.smithi099.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 15.2.16-304-gf25ffac7-1focal [18.1 kB] 2022-06-29T15:39:52.023 INFO:teuthology.orchestra.run.smithi099.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 15.2.16-304-gf25ffac7-1focal [9,668 kB] 2022-06-29T15:39:52.248 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-rados. 2022-06-29T15:39:52.267 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../007-python3-rados_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:39:52.306 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-rados (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:39:52.323 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libcephfs2. 2022-06-29T15:39:52.339 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../006-libcephfs2_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:39:52.382 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libcephfs2 (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:39:52.565 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-rgw. 2022-06-29T15:39:52.581 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../016-python3-rgw_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:39:52.584 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-ceph-argparse. 2022-06-29T15:39:52.600 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../008-python3-ceph-argparse_15.2.16-304-gf25ffac7-1focal_all.deb ... 2022-06-29T15:39:52.623 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-rgw (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:39:52.642 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-ceph-argparse (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:39:52.709 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-rados. 2022-06-29T15:39:52.729 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../007-python3-rados_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:39:52.767 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-rados (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:39:52.962 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-cephfs. 2022-06-29T15:39:52.970 INFO:teuthology.orchestra.run.smithi099.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2022-06-27-0923-octopus/f25ffac72e5a9e63860ff7deb6767a11f0559a16/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 15.2.16-304-gf25ffac7-1focal [60.9 kB] 2022-06-29T15:39:52.977 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../009-python3-cephfs_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:39:53.001 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libleveldb1d:amd64. 2022-06-29T15:39:53.017 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../017-libleveldb1d_1.22-3ubuntu2_amd64.deb ... 2022-06-29T15:39:53.020 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-cephfs (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:39:53.051 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libleveldb1d:amd64 (1.22-3ubuntu2) ... 2022-06-29T15:39:53.070 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-ceph-argparse. 2022-06-29T15:39:53.085 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../008-python3-ceph-argparse_15.2.16-304-gf25ffac7-1focal_all.deb ... 2022-06-29T15:39:53.144 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-ceph-argparse (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:39:53.348 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-ceph-common. 2022-06-29T15:39:53.363 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../010-python3-ceph-common_15.2.16-304-gf25ffac7-1focal_all.deb ... 2022-06-29T15:39:53.406 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-ceph-common (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:39:53.423 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-cephfs. 2022-06-29T15:39:53.441 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../009-python3-cephfs_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:39:53.445 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package liboath0. 2022-06-29T15:39:53.461 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../018-liboath0_2.6.1-1.3_amd64.deb ... 2022-06-29T15:39:53.481 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-cephfs (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:39:53.503 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2022-06-29T15:39:53.633 INFO:teuthology.orchestra.run.smithi099.stdout:Fetched 153 MB in 23s (6,754 kB/s) 2022-06-29T15:39:53.692 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-prettytable. 2022-06-29T15:39:53.708 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../011-python3-prettytable_0.7.2-5_all.deb ... 2022-06-29T15:39:53.742 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2022-06-29T15:39:53.745 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-pyperclip. 2022-06-29T15:39:53.799 INFO:teuthology.orchestra.run.smithi099.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 138771 files and directories currently installed.) 2022-06-29T15:39:53.803 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2022-06-29T15:39:53.834 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-ceph-common. 2022-06-29T15:39:53.835 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2022-06-29T15:39:53.850 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../010-python3-ceph-common_15.2.16-304-gf25ffac7-1focal_all.deb ... 2022-06-29T15:39:53.865 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libradosstriper1. 2022-06-29T15:39:53.884 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../019-libradosstriper1_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:39:53.892 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-ceph-common (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:39:53.923 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libradosstriper1 (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:39:53.961 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-rbd. 2022-06-29T15:39:53.978 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../012-python3-rbd_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:39:54.011 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-rbd (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:39:54.105 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-wcwidth. 2022-06-29T15:39:54.122 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2022-06-29T15:39:54.163 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2022-06-29T15:39:54.228 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-prettytable. 2022-06-29T15:39:54.244 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../011-python3-prettytable_0.7.2-5_all.deb ... 2022-06-29T15:39:54.286 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2022-06-29T15:39:54.343 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-common. 2022-06-29T15:39:54.358 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../020-ceph-common_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:39:54.374 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package librabbitmq4:amd64. 2022-06-29T15:39:54.391 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../013-librabbitmq4_0.10.0-1_amd64.deb ... 2022-06-29T15:39:54.392 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-common (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:39:54.432 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2022-06-29T15:39:54.483 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-cmd2. 2022-06-29T15:39:54.502 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2022-06-29T15:39:54.539 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-rbd. 2022-06-29T15:39:54.540 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2022-06-29T15:39:54.558 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../012-python3-rbd_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:39:54.597 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-rbd (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:39:54.735 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package librdkafka1:amd64. 2022-06-29T15:39:54.752 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../014-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2022-06-29T15:39:54.794 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2022-06-29T15:39:54.894 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2022-06-29T15:39:54.913 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2022-06-29T15:39:54.983 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package librabbitmq4:amd64. 2022-06-29T15:39:54.999 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../013-librabbitmq4_0.10.0-1_amd64.deb ... 2022-06-29T15:39:55.007 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2022-06-29T15:39:55.042 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2022-06-29T15:39:55.105 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package librgw2. 2022-06-29T15:39:55.121 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../015-librgw2_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:39:55.168 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking librgw2 (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:39:55.315 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../004-librbd1_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:39:55.403 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package librdkafka1:amd64. 2022-06-29T15:39:55.414 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking librbd1 (15.2.16-304-gf25ffac7-1focal) over (15.2.16-0ubuntu0.20.04.1) ... 2022-06-29T15:39:55.421 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../014-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2022-06-29T15:39:55.461 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2022-06-29T15:39:55.730 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-rgw. 2022-06-29T15:39:55.747 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../016-python3-rgw_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:39:55.789 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package librgw2. 2022-06-29T15:39:55.790 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-rgw (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:39:55.807 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../015-librgw2_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:39:55.864 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking librgw2 (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:39:55.954 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../005-librados2_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:39:56.076 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking librados2 (15.2.16-304-gf25ffac7-1focal) over (15.2.16-0ubuntu0.20.04.1) ... 2022-06-29T15:39:56.110 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libleveldb1d:amd64. 2022-06-29T15:39:56.125 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../017-libleveldb1d_1.22-3ubuntu2_amd64.deb ... 2022-06-29T15:39:56.160 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libleveldb1d:amd64 (1.22-3ubuntu2) ... 2022-06-29T15:39:56.383 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-rgw. 2022-06-29T15:39:56.402 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../016-python3-rgw_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:39:56.433 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-rgw (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:39:56.437 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package liboath0. 2022-06-29T15:39:56.455 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../018-liboath0_2.6.1-1.3_amd64.deb ... 2022-06-29T15:39:56.487 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2022-06-29T15:39:56.615 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-base. 2022-06-29T15:39:56.631 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../021-ceph-base_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:39:56.684 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-base (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:39:56.713 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libcephfs2. 2022-06-29T15:39:56.731 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../006-libcephfs2_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:39:56.771 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libcephfs2 (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:39:56.785 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libleveldb1d:amd64. 2022-06-29T15:39:56.803 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../017-libleveldb1d_1.22-3ubuntu2_amd64.deb ... 2022-06-29T15:39:56.824 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libradosstriper1. 2022-06-29T15:39:56.840 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../019-libradosstriper1_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:39:56.844 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libleveldb1d:amd64 (1.22-3ubuntu2) ... 2022-06-29T15:39:56.882 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libradosstriper1 (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:39:57.015 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-rados. 2022-06-29T15:39:57.031 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../007-python3-rados_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:39:57.090 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-rados (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:39:57.222 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package liboath0. 2022-06-29T15:39:57.239 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../018-liboath0_2.6.1-1.3_amd64.deb ... 2022-06-29T15:39:57.280 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2022-06-29T15:39:57.327 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-common. 2022-06-29T15:39:57.344 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../020-ceph-common_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:39:57.385 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-common (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:39:57.426 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-ceph-argparse. 2022-06-29T15:39:57.445 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../008-python3-ceph-argparse_15.2.16-304-gf25ffac7-1focal_all.deb ... 2022-06-29T15:39:57.484 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-ceph-argparse (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:39:57.533 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libradosstriper1. 2022-06-29T15:39:57.550 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../019-libradosstriper1_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:39:57.583 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libradosstriper1 (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:39:57.603 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-cryptography. 2022-06-29T15:39:57.620 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../022-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2022-06-29T15:39:57.653 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2022-06-29T15:39:57.787 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-cephfs. 2022-06-29T15:39:57.806 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../009-python3-cephfs_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:39:57.845 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-cephfs (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:39:58.003 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package ceph-common. 2022-06-29T15:39:58.021 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../020-ceph-common_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:39:58.086 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking ceph-common (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:39:58.123 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-openssl. 2022-06-29T15:39:58.139 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../023-python3-openssl_19.0.0-1build1_all.deb ... 2022-06-29T15:39:58.148 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-ceph-common. 2022-06-29T15:39:58.165 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../010-python3-ceph-common_15.2.16-304-gf25ffac7-1focal_all.deb ... 2022-06-29T15:39:58.215 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-ceph-common (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:39:58.216 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2022-06-29T15:39:58.459 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2022-06-29T15:39:58.477 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../024-ceph-mgr-modules-core_15.2.16-304-gf25ffac7-1focal_all.deb ... 2022-06-29T15:39:58.509 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-mgr-modules-core (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:39:58.525 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-prettytable. 2022-06-29T15:39:58.541 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../011-python3-prettytable_0.7.2-5_all.deb ... 2022-06-29T15:39:58.584 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2022-06-29T15:39:58.828 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-rbd. 2022-06-29T15:39:58.844 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../012-python3-rbd_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:39:58.878 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-rbd (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:39:58.922 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-bcrypt. 2022-06-29T15:39:58.940 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../025-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2022-06-29T15:39:58.980 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2022-06-29T15:39:59.238 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package librabbitmq4:amd64. 2022-06-29T15:39:59.257 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../013-librabbitmq4_0.10.0-1_amd64.deb ... 2022-06-29T15:39:59.297 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2022-06-29T15:39:59.317 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-cherrypy3. 2022-06-29T15:39:59.334 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../026-python3-cherrypy3_8.9.1-6_all.deb ... 2022-06-29T15:39:59.400 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2022-06-29T15:39:59.491 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-base. 2022-06-29T15:39:59.506 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../021-ceph-base_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:39:59.551 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-base (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:39:59.635 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package librdkafka1:amd64. 2022-06-29T15:39:59.651 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../014-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2022-06-29T15:39:59.693 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2022-06-29T15:39:59.769 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-jwt. 2022-06-29T15:39:59.786 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../027-python3-jwt_1.7.1-2ubuntu2_all.deb ... 2022-06-29T15:39:59.861 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2) ... 2022-06-29T15:40:00.004 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package librgw2. 2022-06-29T15:40:00.024 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../015-librgw2_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:40:00.054 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking librgw2 (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:00.225 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package ceph-base. 2022-06-29T15:40:00.242 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../021-ceph-base_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:40:00.284 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking ceph-base (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:00.387 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-cryptography. 2022-06-29T15:40:00.403 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../022-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2022-06-29T15:40:00.437 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2022-06-29T15:40:00.607 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-rgw. 2022-06-29T15:40:00.622 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../016-python3-rgw_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:40:00.665 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-rgw (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:00.823 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-logutils. 2022-06-29T15:40:00.839 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../028-python3-logutils_0.3.3-7_all.deb ... 2022-06-29T15:40:00.890 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-logutils (0.3.3-7) ... 2022-06-29T15:40:01.009 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libleveldb1d:amd64. 2022-06-29T15:40:01.027 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../017-libleveldb1d_1.22-3ubuntu2_amd64.deb ... 2022-06-29T15:40:01.092 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libleveldb1d:amd64 (1.22-3ubuntu2) ... 2022-06-29T15:40:01.179 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-cryptography. 2022-06-29T15:40:01.197 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../022-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2022-06-29T15:40:01.229 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2022-06-29T15:40:01.261 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-markupsafe. 2022-06-29T15:40:01.277 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../029-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2022-06-29T15:40:01.319 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2022-06-29T15:40:01.412 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package liboath0. 2022-06-29T15:40:01.427 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../018-liboath0_2.6.1-1.3_amd64.deb ... 2022-06-29T15:40:01.461 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2022-06-29T15:40:01.651 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-mako. 2022-06-29T15:40:01.668 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../030-python3-mako_1.1.0+ds1-1ubuntu2_all.deb ... 2022-06-29T15:40:01.689 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libradosstriper1. 2022-06-29T15:40:01.704 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../019-libradosstriper1_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:40:01.735 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2) ... 2022-06-29T15:40:01.762 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libradosstriper1 (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:01.791 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-openssl. 2022-06-29T15:40:01.810 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../023-python3-openssl_19.0.0-1build1_all.deb ... 2022-06-29T15:40:01.849 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2022-06-29T15:40:01.900 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-openssl. 2022-06-29T15:40:01.915 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../023-python3-openssl_19.0.0-1build1_all.deb ... 2022-06-29T15:40:01.949 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2022-06-29T15:40:02.088 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-simplegeneric. 2022-06-29T15:40:02.102 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2022-06-29T15:40:02.108 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../031-python3-simplegeneric_0.8.1-3_all.deb ... 2022-06-29T15:40:02.119 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../024-ceph-mgr-modules-core_15.2.16-304-gf25ffac7-1focal_all.deb ... 2022-06-29T15:40:02.147 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2022-06-29T15:40:02.185 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking ceph-mgr-modules-core (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:02.186 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2022-06-29T15:40:02.201 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../024-ceph-mgr-modules-core_15.2.16-304-gf25ffac7-1focal_all.deb ... 2022-06-29T15:40:02.218 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-common. 2022-06-29T15:40:02.237 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../020-ceph-common_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:40:02.243 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-mgr-modules-core (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:02.275 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-common (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:02.500 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-singledispatch. 2022-06-29T15:40:02.516 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../032-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2022-06-29T15:40:02.558 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2022-06-29T15:40:02.581 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-bcrypt. 2022-06-29T15:40:02.601 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../025-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2022-06-29T15:40:02.623 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-bcrypt. 2022-06-29T15:40:02.640 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../025-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2022-06-29T15:40:02.664 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2022-06-29T15:40:02.681 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2022-06-29T15:40:02.927 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-webob. 2022-06-29T15:40:02.947 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../033-python3-webob_1%3a1.8.5-2_all.deb ... 2022-06-29T15:40:02.967 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-cherrypy3. 2022-06-29T15:40:02.968 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-cherrypy3. 2022-06-29T15:40:02.983 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../026-python3-cherrypy3_8.9.1-6_all.deb ... 2022-06-29T15:40:02.985 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2022-06-29T15:40:02.986 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../026-python3-cherrypy3_8.9.1-6_all.deb ... 2022-06-29T15:40:03.026 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2022-06-29T15:40:03.026 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2022-06-29T15:40:03.362 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-jwt. 2022-06-29T15:40:03.380 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../027-python3-jwt_1.7.1-2ubuntu2_all.deb ... 2022-06-29T15:40:03.381 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-soupsieve. 2022-06-29T15:40:03.397 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../034-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2022-06-29T15:40:03.429 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-jwt. 2022-06-29T15:40:03.438 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2022-06-29T15:40:03.442 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2) ... 2022-06-29T15:40:03.449 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../027-python3-jwt_1.7.1-2ubuntu2_all.deb ... 2022-06-29T15:40:03.520 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2) ... 2022-06-29T15:40:03.758 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-bs4. 2022-06-29T15:40:03.759 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-logutils. 2022-06-29T15:40:03.774 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../035-python3-bs4_4.8.2-1_all.deb ... 2022-06-29T15:40:03.777 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../028-python3-logutils_0.3.3-7_all.deb ... 2022-06-29T15:40:03.817 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-logutils (0.3.3-7) ... 2022-06-29T15:40:03.842 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-logutils. 2022-06-29T15:40:03.849 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2022-06-29T15:40:03.859 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../028-python3-logutils_0.3.3-7_all.deb ... 2022-06-29T15:40:03.898 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-logutils (0.3.3-7) ... 2022-06-29T15:40:04.118 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-markupsafe. 2022-06-29T15:40:04.138 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../029-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2022-06-29T15:40:04.177 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2022-06-29T15:40:04.218 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-markupsafe. 2022-06-29T15:40:04.234 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../029-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2022-06-29T15:40:04.235 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-waitress. 2022-06-29T15:40:04.251 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../036-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2022-06-29T15:40:04.276 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2022-06-29T15:40:04.280 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-base. 2022-06-29T15:40:04.297 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../021-ceph-base_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:40:04.330 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2022-06-29T15:40:04.416 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-base (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:04.496 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-mako. 2022-06-29T15:40:04.515 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../030-python3-mako_1.1.0+ds1-1ubuntu2_all.deb ... 2022-06-29T15:40:04.554 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2) ... 2022-06-29T15:40:04.699 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-tempita. 2022-06-29T15:40:04.715 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../037-python3-tempita_0.5.2-6_all.deb ... 2022-06-29T15:40:04.781 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-tempita (0.5.2-6) ... 2022-06-29T15:40:04.901 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-simplegeneric. 2022-06-29T15:40:04.917 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../031-python3-simplegeneric_0.8.1-3_all.deb ... 2022-06-29T15:40:04.960 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2022-06-29T15:40:05.093 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-paste. 2022-06-29T15:40:05.112 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../038-python3-paste_3.4.0+dfsg1-1_all.deb ... 2022-06-29T15:40:05.151 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2022-06-29T15:40:05.263 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-singledispatch. 2022-06-29T15:40:05.278 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../032-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2022-06-29T15:40:05.280 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-mako. 2022-06-29T15:40:05.296 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../030-python3-mako_1.1.0+ds1-1ubuntu2_all.deb ... 2022-06-29T15:40:05.318 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2022-06-29T15:40:05.338 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2) ... 2022-06-29T15:40:05.521 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2022-06-29T15:40:05.538 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../039-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2022-06-29T15:40:05.579 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2022-06-29T15:40:05.654 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-webob. 2022-06-29T15:40:05.674 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../033-python3-webob_1%3a1.8.5-2_all.deb ... 2022-06-29T15:40:05.693 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-simplegeneric. 2022-06-29T15:40:05.711 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../031-python3-simplegeneric_0.8.1-3_all.deb ... 2022-06-29T15:40:05.712 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2022-06-29T15:40:05.751 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2022-06-29T15:40:05.827 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-cryptography. 2022-06-29T15:40:05.843 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../022-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2022-06-29T15:40:05.910 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2022-06-29T15:40:06.023 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-soupsieve. 2022-06-29T15:40:06.040 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../034-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2022-06-29T15:40:06.082 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2022-06-29T15:40:06.084 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-singledispatch. 2022-06-29T15:40:06.100 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../032-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2022-06-29T15:40:06.142 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2022-06-29T15:40:06.238 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-openssl. 2022-06-29T15:40:06.257 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../023-python3-openssl_19.0.0-1build1_all.deb ... 2022-06-29T15:40:06.274 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-pastedeploy. 2022-06-29T15:40:06.290 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../040-python3-pastedeploy_2.1.0-1_all.deb ... 2022-06-29T15:40:06.296 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2022-06-29T15:40:06.349 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2022-06-29T15:40:06.351 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-bs4. 2022-06-29T15:40:06.370 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../035-python3-bs4_4.8.2-1_all.deb ... 2022-06-29T15:40:06.409 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2022-06-29T15:40:06.470 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-webob. 2022-06-29T15:40:06.487 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../033-python3-webob_1%3a1.8.5-2_all.deb ... 2022-06-29T15:40:06.528 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2022-06-29T15:40:06.582 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2022-06-29T15:40:06.597 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../024-ceph-mgr-modules-core_15.2.16-304-gf25ffac7-1focal_all.deb ... 2022-06-29T15:40:06.649 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-mgr-modules-core (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:06.677 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-webtest. 2022-06-29T15:40:06.693 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../041-python3-webtest_2.0.32-2_all.deb ... 2022-06-29T15:40:06.735 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-webtest (2.0.32-2) ... 2022-06-29T15:40:06.746 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-waitress. 2022-06-29T15:40:06.761 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../036-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2022-06-29T15:40:06.807 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2022-06-29T15:40:06.890 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-soupsieve. 2022-06-29T15:40:06.907 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../034-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2022-06-29T15:40:06.948 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2022-06-29T15:40:07.011 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-bcrypt. 2022-06-29T15:40:07.027 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../025-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2022-06-29T15:40:07.069 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2022-06-29T15:40:07.115 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-tempita. 2022-06-29T15:40:07.131 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../037-python3-tempita_0.5.2-6_all.deb ... 2022-06-29T15:40:07.138 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-pecan. 2022-06-29T15:40:07.154 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../042-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2022-06-29T15:40:07.174 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-tempita (0.5.2-6) ... 2022-06-29T15:40:07.196 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2022-06-29T15:40:07.301 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-bs4. 2022-06-29T15:40:07.317 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../035-python3-bs4_4.8.2-1_all.deb ... 2022-06-29T15:40:07.359 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2022-06-29T15:40:07.397 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-cherrypy3. 2022-06-29T15:40:07.413 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../026-python3-cherrypy3_8.9.1-6_all.deb ... 2022-06-29T15:40:07.455 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2022-06-29T15:40:07.510 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-paste. 2022-06-29T15:40:07.525 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../038-python3-paste_3.4.0+dfsg1-1_all.deb ... 2022-06-29T15:40:07.568 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2022-06-29T15:40:07.626 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-werkzeug. 2022-06-29T15:40:07.644 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../043-python3-werkzeug_0.16.1+dfsg1-2_all.deb ... 2022-06-29T15:40:07.684 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2) ... 2022-06-29T15:40:07.712 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-waitress. 2022-06-29T15:40:07.730 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../036-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2022-06-29T15:40:07.773 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2022-06-29T15:40:07.800 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-jwt. 2022-06-29T15:40:07.816 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../027-python3-jwt_1.7.1-2ubuntu2_all.deb ... 2022-06-29T15:40:07.858 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2) ... 2022-06-29T15:40:07.890 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2022-06-29T15:40:07.910 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../039-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2022-06-29T15:40:07.949 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2022-06-29T15:40:08.113 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-mgr. 2022-06-29T15:40:08.122 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-tempita. 2022-06-29T15:40:08.130 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../044-ceph-mgr_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:40:08.141 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../037-python3-tempita_0.5.2-6_all.deb ... 2022-06-29T15:40:08.171 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-mgr (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:08.181 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-tempita (0.5.2-6) ... 2022-06-29T15:40:08.196 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-logutils. 2022-06-29T15:40:08.211 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../028-python3-logutils_0.3.3-7_all.deb ... 2022-06-29T15:40:08.254 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-logutils (0.3.3-7) ... 2022-06-29T15:40:08.285 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-pastedeploy. 2022-06-29T15:40:08.301 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../040-python3-pastedeploy_2.1.0-1_all.deb ... 2022-06-29T15:40:08.343 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2022-06-29T15:40:08.559 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-paste. 2022-06-29T15:40:08.576 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../038-python3-paste_3.4.0+dfsg1-1_all.deb ... 2022-06-29T15:40:08.605 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-markupsafe. 2022-06-29T15:40:08.617 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2022-06-29T15:40:08.620 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../029-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2022-06-29T15:40:08.646 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-webtest. 2022-06-29T15:40:08.662 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../041-python3-webtest_2.0.32-2_all.deb ... 2022-06-29T15:40:08.663 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2022-06-29T15:40:08.683 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-mon. 2022-06-29T15:40:08.698 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../045-ceph-mon_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:40:08.704 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-webtest (2.0.32-2) ... 2022-06-29T15:40:08.765 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-mon (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:08.947 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2022-06-29T15:40:08.966 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../039-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2022-06-29T15:40:08.967 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-mako. 2022-06-29T15:40:08.983 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../030-python3-mako_1.1.0+ds1-1ubuntu2_all.deb ... 2022-06-29T15:40:08.999 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-pecan. 2022-06-29T15:40:09.018 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../042-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2022-06-29T15:40:09.022 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2022-06-29T15:40:09.024 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2) ... 2022-06-29T15:40:09.057 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2022-06-29T15:40:09.325 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-pastedeploy. 2022-06-29T15:40:09.341 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../040-python3-pastedeploy_2.1.0-1_all.deb ... 2022-06-29T15:40:09.362 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-simplegeneric. 2022-06-29T15:40:09.381 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../031-python3-simplegeneric_0.8.1-3_all.deb ... 2022-06-29T15:40:09.383 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2022-06-29T15:40:09.420 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2022-06-29T15:40:09.437 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-werkzeug. 2022-06-29T15:40:09.453 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../043-python3-werkzeug_0.16.1+dfsg1-2_all.deb ... 2022-06-29T15:40:09.495 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2) ... 2022-06-29T15:40:09.712 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-webtest. 2022-06-29T15:40:09.719 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-singledispatch. 2022-06-29T15:40:09.727 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../041-python3-webtest_2.0.32-2_all.deb ... 2022-06-29T15:40:09.735 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../032-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2022-06-29T15:40:09.770 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-webtest (2.0.32-2) ... 2022-06-29T15:40:09.778 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2022-06-29T15:40:09.819 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-osd. 2022-06-29T15:40:09.835 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../046-ceph-osd_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:40:09.840 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-mgr. 2022-06-29T15:40:09.859 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../044-ceph-mgr_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:40:09.899 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-mgr (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:09.911 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-osd (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:10.098 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-pecan. 2022-06-29T15:40:10.114 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../042-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2022-06-29T15:40:10.156 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2022-06-29T15:40:10.360 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-mon. 2022-06-29T15:40:10.363 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-webob. 2022-06-29T15:40:10.377 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../045-ceph-mon_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:40:10.382 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../033-python3-webob_1%3a1.8.5-2_all.deb ... 2022-06-29T15:40:10.418 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-mon (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:10.421 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2022-06-29T15:40:10.774 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-soupsieve. 2022-06-29T15:40:10.793 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../034-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2022-06-29T15:40:10.832 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2022-06-29T15:40:10.853 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-werkzeug. 2022-06-29T15:40:10.871 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../043-python3-werkzeug_0.16.1+dfsg1-2_all.deb ... 2022-06-29T15:40:10.911 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2) ... 2022-06-29T15:40:11.169 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-bs4. 2022-06-29T15:40:11.185 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../035-python3-bs4_4.8.2-1_all.deb ... 2022-06-29T15:40:11.227 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2022-06-29T15:40:11.301 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package ceph-mgr. 2022-06-29T15:40:11.315 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../044-ceph-mgr_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:40:11.322 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-osd. 2022-06-29T15:40:11.342 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../046-ceph-osd_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:40:11.356 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking ceph-mgr (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:11.380 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-osd (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:11.597 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-waitress. 2022-06-29T15:40:11.612 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../036-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2022-06-29T15:40:11.658 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2022-06-29T15:40:11.842 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package ceph-mon. 2022-06-29T15:40:11.861 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../045-ceph-mon_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:40:11.909 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking ceph-mon (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:12.024 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-tempita. 2022-06-29T15:40:12.040 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../037-python3-tempita_0.5.2-6_all.deb ... 2022-06-29T15:40:12.082 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-tempita (0.5.2-6) ... 2022-06-29T15:40:12.350 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph. 2022-06-29T15:40:12.367 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../047-ceph_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:40:12.409 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-paste. 2022-06-29T15:40:12.425 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:12.426 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../038-python3-paste_3.4.0+dfsg1-1_all.deb ... 2022-06-29T15:40:12.468 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2022-06-29T15:40:12.794 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-fuse. 2022-06-29T15:40:12.795 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2022-06-29T15:40:12.810 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../048-ceph-fuse_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:40:12.814 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../039-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2022-06-29T15:40:12.852 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-fuse (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:12.862 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2022-06-29T15:40:12.998 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package ceph-osd. 2022-06-29T15:40:13.015 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../046-ceph-osd_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:40:13.081 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking ceph-osd (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:13.173 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-pastedeploy. 2022-06-29T15:40:13.193 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../040-python3-pastedeploy_2.1.0-1_all.deb ... 2022-06-29T15:40:13.231 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2022-06-29T15:40:13.323 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-mds. 2022-06-29T15:40:13.338 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../049-ceph-mds_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:40:13.380 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-mds (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:13.542 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-webtest. 2022-06-29T15:40:13.561 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../041-python3-webtest_2.0.32-2_all.deb ... 2022-06-29T15:40:13.601 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-webtest (2.0.32-2) ... 2022-06-29T15:40:13.753 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph. 2022-06-29T15:40:13.769 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../047-ceph_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:40:13.802 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:13.952 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-pecan. 2022-06-29T15:40:13.970 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../042-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2022-06-29T15:40:13.975 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package cephadm. 2022-06-29T15:40:13.991 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../050-cephadm_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:40:14.012 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2022-06-29T15:40:14.034 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking cephadm (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:14.072 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-fuse. 2022-06-29T15:40:14.088 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../048-ceph-fuse_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:40:14.121 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-fuse (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:14.350 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-werkzeug. 2022-06-29T15:40:14.367 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../043-python3-werkzeug_0.16.1+dfsg1-2_all.deb ... 2022-06-29T15:40:14.403 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-jinja2. 2022-06-29T15:40:14.408 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2) ... 2022-06-29T15:40:14.423 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../051-python3-jinja2_2.10.1-2_all.deb ... 2022-06-29T15:40:14.461 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2022-06-29T15:40:14.500 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-mds. 2022-06-29T15:40:14.516 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../049-ceph-mds_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:40:14.558 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-mds (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:14.811 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-mgr. 2022-06-29T15:40:14.823 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2022-06-29T15:40:14.827 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../044-ceph-mgr_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:40:14.840 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../052-ceph-mgr-cephadm_15.2.16-304-gf25ffac7-1focal_all.deb ... 2022-06-29T15:40:14.869 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-mgr (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:14.906 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-mgr-cephadm (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:15.086 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package cephadm. 2022-06-29T15:40:15.102 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../050-cephadm_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:40:15.144 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking cephadm (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:15.268 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-repoze.lru. 2022-06-29T15:40:15.285 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../053-python3-repoze.lru_0.7-2_all.deb ... 2022-06-29T15:40:15.351 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2022-06-29T15:40:15.364 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-mon. 2022-06-29T15:40:15.382 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../045-ceph-mon_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:40:15.422 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-mon (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:15.428 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package ceph. 2022-06-29T15:40:15.439 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-jinja2. 2022-06-29T15:40:15.445 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../047-ceph_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:40:15.458 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../051-python3-jinja2_2.10.1-2_all.deb ... 2022-06-29T15:40:15.494 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking ceph (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:15.497 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2022-06-29T15:40:15.612 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libjs-underscore. 2022-06-29T15:40:15.632 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../054-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2022-06-29T15:40:15.662 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2022-06-29T15:40:15.817 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2022-06-29T15:40:15.832 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../052-ceph-mgr-cephadm_15.2.16-304-gf25ffac7-1focal_all.deb ... 2022-06-29T15:40:15.847 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package ceph-fuse. 2022-06-29T15:40:15.863 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../048-ceph-fuse_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:40:15.875 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-mgr-cephadm (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:15.905 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking ceph-fuse (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:15.923 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libjs-sphinxdoc. 2022-06-29T15:40:15.941 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../055-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2022-06-29T15:40:15.973 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2022-06-29T15:40:16.237 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-repoze.lru. 2022-06-29T15:40:16.253 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../053-python3-repoze.lru_0.7-2_all.deb ... 2022-06-29T15:40:16.295 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2022-06-29T15:40:16.300 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package ceph-mds. 2022-06-29T15:40:16.318 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../049-ceph-mds_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:40:16.358 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking ceph-mds (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:16.426 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-routes. 2022-06-29T15:40:16.442 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../056-python3-routes_2.4.1-2_all.deb ... 2022-06-29T15:40:16.443 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-osd. 2022-06-29T15:40:16.461 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../046-ceph-osd_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:40:16.484 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-routes (2.4.1-2) ... 2022-06-29T15:40:16.500 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-osd (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:16.573 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libjs-underscore. 2022-06-29T15:40:16.589 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../054-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2022-06-29T15:40:16.631 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2022-06-29T15:40:16.871 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2022-06-29T15:40:16.886 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package cephadm. 2022-06-29T15:40:16.889 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../057-ceph-mgr-dashboard_15.2.16-304-gf25ffac7-1focal_all.deb ... 2022-06-29T15:40:16.903 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../050-cephadm_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:40:16.930 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-mgr-dashboard (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:16.942 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libjs-sphinxdoc. 2022-06-29T15:40:16.952 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking cephadm (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:16.962 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../055-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2022-06-29T15:40:17.000 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2022-06-29T15:40:17.306 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-jinja2. 2022-06-29T15:40:17.320 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-routes. 2022-06-29T15:40:17.322 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../051-python3-jinja2_2.10.1-2_all.deb ... 2022-06-29T15:40:17.336 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../056-python3-routes_2.4.1-2_all.deb ... 2022-06-29T15:40:17.364 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2022-06-29T15:40:17.378 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-routes (2.4.1-2) ... 2022-06-29T15:40:17.715 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2022-06-29T15:40:17.717 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2022-06-29T15:40:17.732 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../057-ceph-mgr-dashboard_15.2.16-304-gf25ffac7-1focal_all.deb ... 2022-06-29T15:40:17.733 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../052-ceph-mgr-cephadm_15.2.16-304-gf25ffac7-1focal_all.deb ... 2022-06-29T15:40:17.773 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-mgr-dashboard (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:17.775 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking ceph-mgr-cephadm (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:17.991 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-mgr-diskprediction-cloud. 2022-06-29T15:40:18.012 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../058-ceph-mgr-diskprediction-cloud_15.2.16-304-gf25ffac7-1focal_all.deb ... 2022-06-29T15:40:18.047 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-mgr-diskprediction-cloud (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:18.120 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-repoze.lru. 2022-06-29T15:40:18.138 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../053-python3-repoze.lru_0.7-2_all.deb ... 2022-06-29T15:40:18.178 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2022-06-29T15:40:18.325 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-decorator. 2022-06-29T15:40:18.344 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../059-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2022-06-29T15:40:18.383 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2022-06-29T15:40:18.449 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libjs-underscore. 2022-06-29T15:40:18.469 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../054-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2022-06-29T15:40:18.514 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2022-06-29T15:40:18.691 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-mgr-diskprediction-cloud. 2022-06-29T15:40:18.711 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../058-ceph-mgr-diskprediction-cloud_15.2.16-304-gf25ffac7-1focal_all.deb ... 2022-06-29T15:40:18.749 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-mgr-diskprediction-cloud (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:18.770 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2022-06-29T15:40:18.775 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libjs-sphinxdoc. 2022-06-29T15:40:18.786 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../060-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2022-06-29T15:40:18.793 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../055-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2022-06-29T15:40:18.828 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2022-06-29T15:40:18.842 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2022-06-29T15:40:19.006 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph. 2022-06-29T15:40:19.023 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../047-ceph_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:40:19.064 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:19.094 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-decorator. 2022-06-29T15:40:19.110 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../059-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2022-06-29T15:40:19.139 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-scipy. 2022-06-29T15:40:19.152 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2022-06-29T15:40:19.157 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../061-python3-scipy_1.3.3-3build1_amd64.deb ... 2022-06-29T15:40:19.197 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2022-06-29T15:40:19.211 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-routes. 2022-06-29T15:40:19.228 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../056-python3-routes_2.4.1-2_all.deb ... 2022-06-29T15:40:19.269 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-routes (2.4.1-2) ... 2022-06-29T15:40:19.383 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-fuse. 2022-06-29T15:40:19.399 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../048-ceph-fuse_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:40:19.455 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2022-06-29T15:40:19.466 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-fuse (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:19.472 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../060-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2022-06-29T15:40:19.513 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2022-06-29T15:40:19.632 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2022-06-29T15:40:19.647 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../057-ceph-mgr-dashboard_15.2.16-304-gf25ffac7-1focal_all.deb ... 2022-06-29T15:40:19.690 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking ceph-mgr-dashboard (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:19.849 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-scipy. 2022-06-29T15:40:19.861 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-mds. 2022-06-29T15:40:19.865 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../061-python3-scipy_1.3.3-3build1_amd64.deb ... 2022-06-29T15:40:19.879 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../049-ceph-mds_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:40:19.907 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2022-06-29T15:40:19.928 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-mds (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:20.414 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package cephadm. 2022-06-29T15:40:20.434 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../050-cephadm_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:40:20.480 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking cephadm (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:20.707 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-sklearn-lib. 2022-06-29T15:40:20.725 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../062-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2022-06-29T15:40:20.730 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package ceph-mgr-diskprediction-cloud. 2022-06-29T15:40:20.748 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../058-ceph-mgr-diskprediction-cloud_15.2.16-304-gf25ffac7-1focal_all.deb ... 2022-06-29T15:40:20.765 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2022-06-29T15:40:20.789 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking ceph-mgr-diskprediction-cloud (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:20.817 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-jinja2. 2022-06-29T15:40:20.834 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../051-python3-jinja2_2.10.1-2_all.deb ... 2022-06-29T15:40:20.875 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2022-06-29T15:40:21.186 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2022-06-29T15:40:21.200 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-decorator. 2022-06-29T15:40:21.203 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../052-ceph-mgr-cephadm_15.2.16-304-gf25ffac7-1focal_all.deb ... 2022-06-29T15:40:21.216 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../059-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2022-06-29T15:40:21.244 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-mgr-cephadm (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:21.258 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2022-06-29T15:40:21.387 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-sklearn-lib. 2022-06-29T15:40:21.402 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-joblib. 2022-06-29T15:40:21.403 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../062-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2022-06-29T15:40:21.421 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../063-python3-joblib_0.14.0-3_all.deb ... 2022-06-29T15:40:21.443 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2022-06-29T15:40:21.460 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-joblib (0.14.0-3) ... 2022-06-29T15:40:21.594 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2022-06-29T15:40:21.614 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../060-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2022-06-29T15:40:21.614 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-repoze.lru. 2022-06-29T15:40:21.631 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../053-python3-repoze.lru_0.7-2_all.deb ... 2022-06-29T15:40:21.661 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2022-06-29T15:40:21.672 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2022-06-29T15:40:21.864 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-sklearn. 2022-06-29T15:40:21.884 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../064-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2022-06-29T15:40:21.908 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libjs-underscore. 2022-06-29T15:40:21.922 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2022-06-29T15:40:21.924 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../054-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2022-06-29T15:40:21.966 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2022-06-29T15:40:21.989 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-scipy. 2022-06-29T15:40:22.005 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../061-python3-scipy_1.3.3-3build1_amd64.deb ... 2022-06-29T15:40:22.022 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-joblib. 2022-06-29T15:40:22.040 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../063-python3-joblib_0.14.0-3_all.deb ... 2022-06-29T15:40:22.072 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2022-06-29T15:40:22.088 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-joblib (0.14.0-3) ... 2022-06-29T15:40:22.244 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libjs-sphinxdoc. 2022-06-29T15:40:22.260 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../055-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2022-06-29T15:40:22.302 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2022-06-29T15:40:22.492 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-sklearn. 2022-06-29T15:40:22.508 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../064-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2022-06-29T15:40:22.550 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2022-06-29T15:40:22.639 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2022-06-29T15:40:22.655 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-routes. 2022-06-29T15:40:22.655 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../065-ceph-mgr-diskprediction-local_15.2.16-304-gf25ffac7-1focal_all.deb ... 2022-06-29T15:40:22.672 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../056-python3-routes_2.4.1-2_all.deb ... 2022-06-29T15:40:22.715 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-mgr-diskprediction-local (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:22.721 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-routes (2.4.1-2) ... 2022-06-29T15:40:23.066 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2022-06-29T15:40:23.085 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../057-ceph-mgr-dashboard_15.2.16-304-gf25ffac7-1focal_all.deb ... 2022-06-29T15:40:23.124 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-mgr-dashboard (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:23.258 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2022-06-29T15:40:23.279 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../065-ceph-mgr-diskprediction-local_15.2.16-304-gf25ffac7-1focal_all.deb ... 2022-06-29T15:40:23.317 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-mgr-diskprediction-local (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:23.556 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-sklearn-lib. 2022-06-29T15:40:23.574 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../062-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2022-06-29T15:40:23.616 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2022-06-29T15:40:23.883 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-cachetools. 2022-06-29T15:40:23.903 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../066-python3-cachetools_4.0.0-1_all.deb ... 2022-06-29T15:40:23.975 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2022-06-29T15:40:24.089 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-mgr-diskprediction-cloud. 2022-06-29T15:40:24.107 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../058-ceph-mgr-diskprediction-cloud_15.2.16-304-gf25ffac7-1focal_all.deb ... 2022-06-29T15:40:24.164 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-mgr-diskprediction-cloud (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:24.176 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-joblib. 2022-06-29T15:40:24.195 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../063-python3-joblib_0.14.0-3_all.deb ... 2022-06-29T15:40:24.234 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-joblib (0.14.0-3) ... 2022-06-29T15:40:24.319 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-pyasn1. 2022-06-29T15:40:24.336 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../067-python3-pyasn1_0.4.2-3build1_all.deb ... 2022-06-29T15:40:24.377 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2022-06-29T15:40:24.428 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-cachetools. 2022-06-29T15:40:24.446 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../066-python3-cachetools_4.0.0-1_all.deb ... 2022-06-29T15:40:24.475 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-decorator. 2022-06-29T15:40:24.486 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2022-06-29T15:40:24.492 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../059-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2022-06-29T15:40:24.533 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2022-06-29T15:40:24.630 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-sklearn. 2022-06-29T15:40:24.649 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../064-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2022-06-29T15:40:24.688 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2022-06-29T15:40:24.773 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-pyasn1-modules. 2022-06-29T15:40:24.790 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../068-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2022-06-29T15:40:24.798 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-pyasn1. 2022-06-29T15:40:24.818 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../067-python3-pyasn1_0.4.2-3build1_all.deb ... 2022-06-29T15:40:24.822 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2022-06-29T15:40:24.848 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2022-06-29T15:40:24.861 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2022-06-29T15:40:24.879 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../060-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2022-06-29T15:40:24.920 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2022-06-29T15:40:25.159 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-rsa. 2022-06-29T15:40:25.177 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../069-python3-rsa_4.0-3_all.deb ... 2022-06-29T15:40:25.193 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-pyasn1-modules. 2022-06-29T15:40:25.209 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../068-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2022-06-29T15:40:25.216 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-rsa (4.0-3) ... 2022-06-29T15:40:25.251 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2022-06-29T15:40:25.264 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-scipy. 2022-06-29T15:40:25.280 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../061-python3-scipy_1.3.3-3build1_amd64.deb ... 2022-06-29T15:40:25.322 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2022-06-29T15:40:25.429 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2022-06-29T15:40:25.449 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../065-ceph-mgr-diskprediction-local_15.2.16-304-gf25ffac7-1focal_all.deb ... 2022-06-29T15:40:25.470 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-google-auth. 2022-06-29T15:40:25.487 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking ceph-mgr-diskprediction-local (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:25.489 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../070-python3-google-auth_1.5.1-2_all.deb ... 2022-06-29T15:40:25.520 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2022-06-29T15:40:25.579 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-rsa. 2022-06-29T15:40:25.596 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../069-python3-rsa_4.0-3_all.deb ... 2022-06-29T15:40:25.637 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-rsa (4.0-3) ... 2022-06-29T15:40:25.882 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-blinker. 2022-06-29T15:40:25.901 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../071-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2022-06-29T15:40:25.940 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2022-06-29T15:40:25.949 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-google-auth. 2022-06-29T15:40:25.965 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../070-python3-google-auth_1.5.1-2_all.deb ... 2022-06-29T15:40:26.007 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2022-06-29T15:40:26.276 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-oauthlib. 2022-06-29T15:40:26.293 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../072-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2022-06-29T15:40:26.327 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-blinker. 2022-06-29T15:40:26.347 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../071-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2022-06-29T15:40:26.359 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2022-06-29T15:40:26.385 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2022-06-29T15:40:26.624 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-cachetools. 2022-06-29T15:40:26.642 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../066-python3-cachetools_4.0.0-1_all.deb ... 2022-06-29T15:40:26.682 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2022-06-29T15:40:26.730 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-oauthlib. 2022-06-29T15:40:26.746 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../072-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2022-06-29T15:40:26.755 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-requests-oauthlib. 2022-06-29T15:40:26.773 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../073-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2022-06-29T15:40:26.788 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2022-06-29T15:40:26.821 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2022-06-29T15:40:26.915 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-sklearn-lib. 2022-06-29T15:40:26.934 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../062-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2022-06-29T15:40:26.973 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2022-06-29T15:40:27.035 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-pyasn1. 2022-06-29T15:40:27.051 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../067-python3-pyasn1_0.4.2-3build1_all.deb ... 2022-06-29T15:40:27.101 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2022-06-29T15:40:27.125 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-requests-oauthlib. 2022-06-29T15:40:27.142 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../073-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2022-06-29T15:40:27.166 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-websocket. 2022-06-29T15:40:27.182 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../074-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2022-06-29T15:40:27.183 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2022-06-29T15:40:27.224 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2022-06-29T15:40:27.471 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-pyasn1-modules. 2022-06-29T15:40:27.490 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../068-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2022-06-29T15:40:27.520 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-websocket. 2022-06-29T15:40:27.521 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-joblib. 2022-06-29T15:40:27.529 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2022-06-29T15:40:27.537 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../074-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2022-06-29T15:40:27.541 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../063-python3-joblib_0.14.0-3_all.deb ... 2022-06-29T15:40:27.577 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-kubernetes. 2022-06-29T15:40:27.578 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2022-06-29T15:40:27.579 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-joblib (0.14.0-3) ... 2022-06-29T15:40:27.594 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../075-python3-kubernetes_7.0.0~a1-2_all.deb ... 2022-06-29T15:40:27.639 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2022-06-29T15:40:27.858 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-rsa. 2022-06-29T15:40:27.878 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../069-python3-rsa_4.0-3_all.deb ... 2022-06-29T15:40:27.890 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-kubernetes. 2022-06-29T15:40:27.910 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../075-python3-kubernetes_7.0.0~a1-2_all.deb ... 2022-06-29T15:40:27.924 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-rsa (4.0-3) ... 2022-06-29T15:40:27.925 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-sklearn. 2022-06-29T15:40:27.943 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../064-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2022-06-29T15:40:27.948 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2022-06-29T15:40:27.982 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2022-06-29T15:40:28.235 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-google-auth. 2022-06-29T15:40:28.252 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../070-python3-google-auth_1.5.1-2_all.deb ... 2022-06-29T15:40:28.274 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2022-06-29T15:40:28.290 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../076-ceph-mgr-k8sevents_15.2.16-304-gf25ffac7-1focal_all.deb ... 2022-06-29T15:40:28.318 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2022-06-29T15:40:28.332 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-mgr-k8sevents (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:28.486 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2022-06-29T15:40:28.507 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../076-ceph-mgr-k8sevents_15.2.16-304-gf25ffac7-1focal_all.deb ... 2022-06-29T15:40:28.544 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-mgr-k8sevents (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:28.647 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-blinker. 2022-06-29T15:40:28.663 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../071-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2022-06-29T15:40:28.691 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2022-06-29T15:40:28.701 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libonig5:amd64. 2022-06-29T15:40:28.705 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2022-06-29T15:40:28.710 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../065-ceph-mgr-diskprediction-local_15.2.16-304-gf25ffac7-1focal_all.deb ... 2022-06-29T15:40:28.719 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../077-libonig5_6.9.4-1_amd64.deb ... 2022-06-29T15:40:28.749 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-mgr-diskprediction-local (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:28.759 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2022-06-29T15:40:28.889 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libonig5:amd64. 2022-06-29T15:40:28.905 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../077-libonig5_6.9.4-1_amd64.deb ... 2022-06-29T15:40:28.947 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2022-06-29T15:40:29.033 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-oauthlib. 2022-06-29T15:40:29.051 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../072-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2022-06-29T15:40:29.091 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2022-06-29T15:40:29.188 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libjq1:amd64. 2022-06-29T15:40:29.206 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../078-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2022-06-29T15:40:29.246 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2022-06-29T15:40:29.300 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libjq1:amd64. 2022-06-29T15:40:29.317 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../078-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2022-06-29T15:40:29.358 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2022-06-29T15:40:29.540 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package jq. 2022-06-29T15:40:29.558 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../079-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2022-06-29T15:40:29.598 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2022-06-29T15:40:29.628 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-requests-oauthlib. 2022-06-29T15:40:29.644 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package jq. 2022-06-29T15:40:29.646 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../073-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2022-06-29T15:40:29.662 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../079-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2022-06-29T15:40:29.702 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2022-06-29T15:40:29.703 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2022-06-29T15:40:29.877 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-cachetools. 2022-06-29T15:40:29.898 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../066-python3-cachetools_4.0.0-1_all.deb ... 2022-06-29T15:40:29.918 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package socat. 2022-06-29T15:40:29.935 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../080-socat_1.7.3.3-2_amd64.deb ... 2022-06-29T15:40:29.935 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2022-06-29T15:40:29.972 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package socat. 2022-06-29T15:40:29.976 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking socat (1.7.3.3-2) ... 2022-06-29T15:40:29.988 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../080-socat_1.7.3.3-2_amd64.deb ... 2022-06-29T15:40:30.023 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-websocket. 2022-06-29T15:40:30.030 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking socat (1.7.3.3-2) ... 2022-06-29T15:40:30.039 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../074-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2022-06-29T15:40:30.114 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2022-06-29T15:40:30.263 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-pyasn1. 2022-06-29T15:40:30.280 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../067-python3-pyasn1_0.4.2-3build1_all.deb ... 2022-06-29T15:40:30.321 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2022-06-29T15:40:30.467 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-kubernetes. 2022-06-29T15:40:30.483 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../075-python3-kubernetes_7.0.0~a1-2_all.deb ... 2022-06-29T15:40:30.528 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libxslt1.1:amd64. 2022-06-29T15:40:30.532 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libxslt1.1:amd64. 2022-06-29T15:40:30.545 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../081-libxslt1.1_1.1.34-4_amd64.deb ... 2022-06-29T15:40:30.549 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2022-06-29T15:40:30.550 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../081-libxslt1.1_1.1.34-4_amd64.deb ... 2022-06-29T15:40:30.586 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libxslt1.1:amd64 (1.1.34-4) ... 2022-06-29T15:40:30.615 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libxslt1.1:amd64 (1.1.34-4) ... 2022-06-29T15:40:30.642 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-pyasn1-modules. 2022-06-29T15:40:30.662 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../068-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2022-06-29T15:40:30.713 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2022-06-29T15:40:30.873 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package xmlstarlet. 2022-06-29T15:40:30.891 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../082-xmlstarlet_1.6.1-2build1_amd64.deb ... 2022-06-29T15:40:30.918 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package xmlstarlet. 2022-06-29T15:40:30.935 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../082-xmlstarlet_1.6.1-2build1_amd64.deb ... 2022-06-29T15:40:30.939 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2022-06-29T15:40:30.976 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2022-06-29T15:40:31.053 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-rsa. 2022-06-29T15:40:31.071 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../069-python3-rsa_4.0-3_all.deb ... 2022-06-29T15:40:31.111 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-rsa (4.0-3) ... 2022-06-29T15:40:31.139 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2022-06-29T15:40:31.156 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../076-ceph-mgr-k8sevents_15.2.16-304-gf25ffac7-1focal_all.deb ... 2022-06-29T15:40:31.197 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking ceph-mgr-k8sevents (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:31.277 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-test. 2022-06-29T15:40:31.293 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../083-ceph-test_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:40:31.351 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-test (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:31.381 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-test. 2022-06-29T15:40:31.400 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../083-ceph-test_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:40:31.431 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-google-auth. 2022-06-29T15:40:31.439 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-test (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:31.447 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../070-python3-google-auth_1.5.1-2_all.deb ... 2022-06-29T15:40:31.514 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2022-06-29T15:40:31.541 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libonig5:amd64. 2022-06-29T15:40:31.558 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../077-libonig5_6.9.4-1_amd64.deb ... 2022-06-29T15:40:31.599 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2022-06-29T15:40:31.859 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-blinker. 2022-06-29T15:40:31.876 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../071-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2022-06-29T15:40:31.918 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2022-06-29T15:40:31.977 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libjq1:amd64. 2022-06-29T15:40:31.995 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../078-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2022-06-29T15:40:32.035 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2022-06-29T15:40:32.221 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-oauthlib. 2022-06-29T15:40:32.237 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../072-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2022-06-29T15:40:32.287 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2022-06-29T15:40:32.380 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package jq. 2022-06-29T15:40:32.398 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../079-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2022-06-29T15:40:32.438 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2022-06-29T15:40:32.575 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-requests-oauthlib. 2022-06-29T15:40:32.594 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../073-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2022-06-29T15:40:32.626 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2022-06-29T15:40:32.691 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package socat. 2022-06-29T15:40:32.708 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../080-socat_1.7.3.3-2_amd64.deb ... 2022-06-29T15:40:32.749 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking socat (1.7.3.3-2) ... 2022-06-29T15:40:32.895 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-websocket. 2022-06-29T15:40:32.914 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../074-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2022-06-29T15:40:32.953 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2022-06-29T15:40:33.246 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libxslt1.1:amd64. 2022-06-29T15:40:33.263 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../081-libxslt1.1_1.1.34-4_amd64.deb ... 2022-06-29T15:40:33.273 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-kubernetes. 2022-06-29T15:40:33.289 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../075-python3-kubernetes_7.0.0~a1-2_all.deb ... 2022-06-29T15:40:33.304 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libxslt1.1:amd64 (1.1.34-4) ... 2022-06-29T15:40:33.331 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2022-06-29T15:40:33.607 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package xmlstarlet. 2022-06-29T15:40:33.626 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../082-xmlstarlet_1.6.1-2build1_amd64.deb ... 2022-06-29T15:40:33.689 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2022-06-29T15:40:33.895 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2022-06-29T15:40:33.912 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../076-ceph-mgr-k8sevents_15.2.16-304-gf25ffac7-1focal_all.deb ... 2022-06-29T15:40:33.953 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-mgr-k8sevents (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:34.011 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package ceph-test. 2022-06-29T15:40:34.028 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../083-ceph-test_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:40:34.069 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking ceph-test (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:34.298 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libonig5:amd64. 2022-06-29T15:40:34.315 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../077-libonig5_6.9.4-1_amd64.deb ... 2022-06-29T15:40:34.356 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2022-06-29T15:40:34.714 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-colorama. 2022-06-29T15:40:34.717 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libjq1:amd64. 2022-06-29T15:40:34.732 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../084-python3-colorama_0.4.3-1build1_all.deb ... 2022-06-29T15:40:34.734 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../078-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2022-06-29T15:40:34.773 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2022-06-29T15:40:34.775 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2022-06-29T15:40:35.000 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package cephfs-shell. 2022-06-29T15:40:35.019 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../085-cephfs-shell_15.2.16-304-gf25ffac7-1focal_all.deb ... 2022-06-29T15:40:35.051 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-colorama. 2022-06-29T15:40:35.058 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking cephfs-shell (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:35.069 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../084-python3-colorama_0.4.3-1build1_all.deb ... 2022-06-29T15:40:35.070 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package jq. 2022-06-29T15:40:35.089 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../079-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2022-06-29T15:40:35.110 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2022-06-29T15:40:35.137 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2022-06-29T15:40:35.286 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libcephfs-dev. 2022-06-29T15:40:35.304 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../086-libcephfs-dev_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:40:35.336 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libcephfs-dev (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:35.389 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package socat. 2022-06-29T15:40:35.407 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../080-socat_1.7.3.3-2_amd64.deb ... 2022-06-29T15:40:35.438 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package cephfs-shell. 2022-06-29T15:40:35.447 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking socat (1.7.3.3-2) ... 2022-06-29T15:40:35.456 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../085-cephfs-shell_15.2.16-304-gf25ffac7-1focal_all.deb ... 2022-06-29T15:40:35.496 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking cephfs-shell (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:35.714 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package nvme-cli. 2022-06-29T15:40:35.732 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../087-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2022-06-29T15:40:35.740 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libcephfs-dev. 2022-06-29T15:40:35.758 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../086-libcephfs-dev_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:40:35.790 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libcephfs-dev (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:35.790 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2022-06-29T15:40:35.895 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libxslt1.1:amd64. 2022-06-29T15:40:35.912 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../081-libxslt1.1_1.1.34-4_amd64.deb ... 2022-06-29T15:40:35.953 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libxslt1.1:amd64 (1.1.34-4) ... 2022-06-29T15:40:36.160 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package nvme-cli. 2022-06-29T15:40:36.170 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-atomicwrites. 2022-06-29T15:40:36.177 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../087-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2022-06-29T15:40:36.188 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../088-python3-atomicwrites_1.1.5-2build1_all.deb ... 2022-06-29T15:40:36.223 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package xmlstarlet. 2022-06-29T15:40:36.228 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2022-06-29T15:40:36.236 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2022-06-29T15:40:36.240 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../082-xmlstarlet_1.6.1-2build1_amd64.deb ... 2022-06-29T15:40:36.306 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2022-06-29T15:40:36.481 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-attr. 2022-06-29T15:40:36.498 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../089-python3-attr_19.3.0-2_all.deb ... 2022-06-29T15:40:36.539 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-attr (19.3.0-2) ... 2022-06-29T15:40:36.611 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-test. 2022-06-29T15:40:36.628 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../083-ceph-test_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:40:36.669 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-test (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:36.859 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-webencodings. 2022-06-29T15:40:36.878 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../090-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2022-06-29T15:40:36.917 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2022-06-29T15:40:37.115 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-atomicwrites. 2022-06-29T15:40:37.135 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../088-python3-atomicwrites_1.1.5-2build1_all.deb ... 2022-06-29T15:40:37.215 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2022-06-29T15:40:37.237 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-html5lib. 2022-06-29T15:40:37.253 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../091-python3-html5lib_1.0.1-2_all.deb ... 2022-06-29T15:40:37.295 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2022-06-29T15:40:37.485 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-attr. 2022-06-29T15:40:37.502 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../089-python3-attr_19.3.0-2_all.deb ... 2022-06-29T15:40:37.539 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-attr (19.3.0-2) ... 2022-06-29T15:40:37.549 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-colorama. 2022-06-29T15:40:37.569 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../084-python3-colorama_0.4.3-1build1_all.deb ... 2022-06-29T15:40:37.607 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2022-06-29T15:40:37.648 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-lxml:amd64. 2022-06-29T15:40:37.665 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../092-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2022-06-29T15:40:37.706 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2022-06-29T15:40:37.838 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-webencodings. 2022-06-29T15:40:37.854 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../090-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2022-06-29T15:40:37.893 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package cephfs-shell. 2022-06-29T15:40:37.906 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2022-06-29T15:40:37.913 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../085-cephfs-shell_15.2.16-304-gf25ffac7-1focal_all.deb ... 2022-06-29T15:40:37.951 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking cephfs-shell (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:38.187 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libcephfs-dev. 2022-06-29T15:40:38.204 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../086-libcephfs-dev_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:40:38.218 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-packaging. 2022-06-29T15:40:38.236 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../093-python3-packaging_20.3-1_all.deb ... 2022-06-29T15:40:38.240 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-html5lib. 2022-06-29T15:40:38.258 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../091-python3-html5lib_1.0.1-2_all.deb ... 2022-06-29T15:40:38.270 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libcephfs-dev (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:38.276 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-packaging (20.3-1) ... 2022-06-29T15:40:38.299 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2022-06-29T15:40:38.571 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-pastescript. 2022-06-29T15:40:38.591 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../094-python3-pastescript_2.0.2-4_all.deb ... 2022-06-29T15:40:38.629 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2022-06-29T15:40:38.677 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-lxml:amd64. 2022-06-29T15:40:38.690 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package nvme-cli. 2022-06-29T15:40:38.696 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../092-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2022-06-29T15:40:38.707 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../087-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2022-06-29T15:40:38.735 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2022-06-29T15:40:38.773 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2022-06-29T15:40:38.950 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-pluggy. 2022-06-29T15:40:38.966 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../095-python3-pluggy_0.13.0-2_all.deb ... 2022-06-29T15:40:39.025 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2022-06-29T15:40:39.171 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-atomicwrites. 2022-06-29T15:40:39.190 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../088-python3-atomicwrites_1.1.5-2build1_all.deb ... 2022-06-29T15:40:39.206 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-packaging. 2022-06-29T15:40:39.224 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../093-python3-packaging_20.3-1_all.deb ... 2022-06-29T15:40:39.228 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2022-06-29T15:40:39.255 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-packaging (20.3-1) ... 2022-06-29T15:40:39.336 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-psutil. 2022-06-29T15:40:39.355 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../096-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2022-06-29T15:40:39.394 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2022-06-29T15:40:39.540 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-attr. 2022-06-29T15:40:39.558 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../089-python3-attr_19.3.0-2_all.deb ... 2022-06-29T15:40:39.600 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-pastescript. 2022-06-29T15:40:39.606 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-attr (19.3.0-2) ... 2022-06-29T15:40:39.617 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../094-python3-pastescript_2.0.2-4_all.deb ... 2022-06-29T15:40:39.658 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2022-06-29T15:40:39.732 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-py. 2022-06-29T15:40:39.748 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../097-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2022-06-29T15:40:39.789 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2022-06-29T15:40:39.943 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-webencodings. 2022-06-29T15:40:39.962 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../090-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2022-06-29T15:40:40.011 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-pluggy. 2022-06-29T15:40:40.029 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../095-python3-pluggy_0.13.0-2_all.deb ... 2022-06-29T15:40:40.034 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2022-06-29T15:40:40.073 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-colorama. 2022-06-29T15:40:40.091 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../084-python3-colorama_0.4.3-1build1_all.deb ... 2022-06-29T15:40:40.101 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-pyinotify. 2022-06-29T15:40:40.103 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2022-06-29T15:40:40.120 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../098-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2022-06-29T15:40:40.131 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2022-06-29T15:40:40.151 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2022-06-29T15:40:40.370 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-html5lib. 2022-06-29T15:40:40.387 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../091-python3-html5lib_1.0.1-2_all.deb ... 2022-06-29T15:40:40.418 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package cephfs-shell. 2022-06-29T15:40:40.435 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../085-cephfs-shell_15.2.16-304-gf25ffac7-1focal_all.deb ... 2022-06-29T15:40:40.437 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2022-06-29T15:40:40.438 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-pytest. 2022-06-29T15:40:40.439 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-psutil. 2022-06-29T15:40:40.454 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../099-python3-pytest_4.6.9-1_all.deb ... 2022-06-29T15:40:40.457 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../096-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2022-06-29T15:40:40.476 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking cephfs-shell (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:40.495 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-pytest (4.6.9-1) ... 2022-06-29T15:40:40.497 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2022-06-29T15:40:40.687 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libcephfs-dev. 2022-06-29T15:40:40.704 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../086-libcephfs-dev_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:40:40.737 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libcephfs-dev (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:40.815 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-lxml:amd64. 2022-06-29T15:40:40.831 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../092-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2022-06-29T15:40:40.833 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-simplejson. 2022-06-29T15:40:40.834 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-py. 2022-06-29T15:40:40.849 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../100-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2022-06-29T15:40:40.852 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../097-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2022-06-29T15:40:40.873 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2022-06-29T15:40:40.884 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2022-06-29T15:40:40.916 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2022-06-29T15:40:41.255 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-pyinotify. 2022-06-29T15:40:41.275 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../098-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2022-06-29T15:40:41.313 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2022-06-29T15:40:41.344 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package radosgw. 2022-06-29T15:40:41.361 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../101-radosgw_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:40:41.411 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking radosgw (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:41.573 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package nvme-cli. 2022-06-29T15:40:41.591 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../087-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2022-06-29T15:40:41.649 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2022-06-29T15:40:41.657 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-pytest. 2022-06-29T15:40:41.674 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../099-python3-pytest_4.6.9-1_all.deb ... 2022-06-29T15:40:41.740 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-pytest (4.6.9-1) ... 2022-06-29T15:40:41.816 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-packaging. 2022-06-29T15:40:41.835 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../093-python3-packaging_20.3-1_all.deb ... 2022-06-29T15:40:41.876 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-packaging (20.3-1) ... 2022-06-29T15:40:42.029 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-atomicwrites. 2022-06-29T15:40:42.049 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../088-python3-atomicwrites_1.1.5-2build1_all.deb ... 2022-06-29T15:40:42.087 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2022-06-29T15:40:42.119 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-simplejson. 2022-06-29T15:40:42.137 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../100-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2022-06-29T15:40:42.196 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-pastescript. 2022-06-29T15:40:42.202 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2022-06-29T15:40:42.212 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../094-python3-pastescript_2.0.2-4_all.deb ... 2022-06-29T15:40:42.254 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2022-06-29T15:40:42.332 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-attr. 2022-06-29T15:40:42.348 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../089-python3-attr_19.3.0-2_all.deb ... 2022-06-29T15:40:42.415 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-attr (19.3.0-2) ... 2022-06-29T15:40:42.456 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package rbd-fuse. 2022-06-29T15:40:42.473 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../102-rbd-fuse_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:40:42.514 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking rbd-fuse (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:42.681 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package radosgw. 2022-06-29T15:40:42.699 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../101-radosgw_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:40:42.718 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-webencodings. 2022-06-29T15:40:42.733 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-pluggy. 2022-06-29T15:40:42.737 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../090-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2022-06-29T15:40:42.739 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking radosgw (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:42.751 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../095-python3-pluggy_0.13.0-2_all.deb ... 2022-06-29T15:40:42.776 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2022-06-29T15:40:42.806 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-attr (19.3.0-2) ... 2022-06-29T15:40:42.816 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2022-06-29T15:40:43.096 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-html5lib. 2022-06-29T15:40:43.114 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../091-python3-html5lib_1.0.1-2_all.deb ... 2022-06-29T15:40:43.127 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-psutil. 2022-06-29T15:40:43.136 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libleveldb1d:amd64 (1.22-3ubuntu2) ... 2022-06-29T15:40:43.144 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../096-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2022-06-29T15:40:43.154 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2022-06-29T15:40:43.185 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2022-06-29T15:40:43.279 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2022-06-29T15:40:43.404 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2022-06-29T15:40:43.499 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-lxml:amd64. 2022-06-29T15:40:43.516 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../092-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2022-06-29T15:40:43.548 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2022-06-29T15:40:43.589 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-py. 2022-06-29T15:40:43.605 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../097-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2022-06-29T15:40:43.606 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up cephadm (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:43.655 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2022-06-29T15:40:43.859 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package rbd-fuse. 2022-06-29T15:40:43.875 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../102-rbd-fuse_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:40:43.917 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking rbd-fuse (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:44.009 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-pyinotify. 2022-06-29T15:40:44.027 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../098-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2022-06-29T15:40:44.066 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2022-06-29T15:40:44.219 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-packaging. 2022-06-29T15:40:44.235 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../093-python3-packaging_20.3-1_all.deb ... 2022-06-29T15:40:44.253 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-attr (19.3.0-2) ... 2022-06-29T15:40:44.277 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-packaging (20.3-1) ... 2022-06-29T15:40:44.301 INFO:teuthology.orchestra.run.smithi164.stdout:Adding system user cephadm....done 2022-06-29T15:40:44.411 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-pytest. 2022-06-29T15:40:44.428 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../099-python3-pytest_4.6.9-1_all.deb ... 2022-06-29T15:40:44.462 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2022-06-29T15:40:44.469 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-pytest (4.6.9-1) ... 2022-06-29T15:40:44.563 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-pastescript. 2022-06-29T15:40:44.582 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../094-python3-pastescript_2.0.2-4_all.deb ... 2022-06-29T15:40:44.613 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2022-06-29T15:40:44.730 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libleveldb1d:amd64 (1.22-3ubuntu2) ... 2022-06-29T15:40:44.814 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-repoze.lru (0.7-2) ... 2022-06-29T15:40:44.831 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-simplejson. 2022-06-29T15:40:44.848 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../100-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2022-06-29T15:40:44.890 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2022-06-29T15:40:44.915 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2022-06-29T15:40:44.933 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-pluggy. 2022-06-29T15:40:44.954 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../095-python3-pluggy_0.13.0-2_all.deb ... 2022-06-29T15:40:45.000 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2022-06-29T15:40:45.074 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2022-06-29T15:40:45.093 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up liboath0 (2.6.1-1.3) ... 2022-06-29T15:40:45.216 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2022-06-29T15:40:45.301 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up cephadm (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:45.336 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-psutil. 2022-06-29T15:40:45.343 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package radosgw. 2022-06-29T15:40:45.357 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../096-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2022-06-29T15:40:45.359 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../101-radosgw_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:40:45.401 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking radosgw (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:45.403 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2022-06-29T15:40:45.543 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-joblib (0.14.0-3) ... 2022-06-29T15:40:45.723 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-py. 2022-06-29T15:40:45.739 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../097-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2022-06-29T15:40:45.814 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2022-06-29T15:40:45.995 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-jwt (1.7.1-2ubuntu2) ... 2022-06-29T15:40:46.054 INFO:teuthology.orchestra.run.smithi064.stdout:Adding system user cephadm....done 2022-06-29T15:40:46.143 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-pyinotify. 2022-06-29T15:40:46.159 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../098-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2022-06-29T15:40:46.201 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2022-06-29T15:40:46.249 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2022-06-29T15:40:46.288 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-cachetools (4.0.0-1) ... 2022-06-29T15:40:46.496 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package rbd-fuse. 2022-06-29T15:40:46.512 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../102-rbd-fuse_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:40:46.554 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking rbd-fuse (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:46.562 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-pytest. 2022-06-29T15:40:46.579 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../099-python3-pytest_4.6.9-1_all.deb ... 2022-06-29T15:40:46.589 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2022-06-29T15:40:46.609 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-repoze.lru (0.7-2) ... 2022-06-29T15:40:46.620 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-pytest (4.6.9-1) ... 2022-06-29T15:40:46.860 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-attr (19.3.0-2) ... 2022-06-29T15:40:46.899 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2022-06-29T15:40:46.943 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up liboath0 (2.6.1-1.3) ... 2022-06-29T15:40:46.957 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-simplejson. 2022-06-29T15:40:46.978 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../100-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2022-06-29T15:40:47.015 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2022-06-29T15:40:47.069 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2022-06-29T15:40:47.200 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2022-06-29T15:40:47.348 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libleveldb1d:amd64 (1.22-3ubuntu2) ... 2022-06-29T15:40:47.455 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-joblib (0.14.0-3) ... 2022-06-29T15:40:47.461 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package radosgw. 2022-06-29T15:40:47.476 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2022-06-29T15:40:47.481 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../101-radosgw_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:40:47.501 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-ceph-argparse (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:47.519 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking radosgw (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:47.629 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2022-06-29T15:40:47.845 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up cephadm (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:47.881 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-jwt (1.7.1-2ubuntu2) ... 2022-06-29T15:40:47.902 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2022-06-29T15:40:48.045 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2022-06-29T15:40:48.191 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-cachetools (4.0.0-1) ... 2022-06-29T15:40:48.337 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2022-06-29T15:40:48.492 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2022-06-29T15:40:48.523 INFO:teuthology.orchestra.run.smithi047.stdout:Adding system user cephadm....done 2022-06-29T15:40:48.630 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2022-06-29T15:40:48.681 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package rbd-fuse. 2022-06-29T15:40:48.697 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../102-rbd-fuse_15.2.16-304-gf25ffac7-1focal_amd64.deb ... 2022-06-29T15:40:48.710 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2022-06-29T15:40:48.739 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking rbd-fuse (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:48.827 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2022-06-29T15:40:48.923 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2022-06-29T15:40:49.051 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-attr (19.3.0-2) ... 2022-06-29T15:40:49.069 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-repoze.lru (0.7-2) ... 2022-06-29T15:40:49.128 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2022-06-29T15:40:49.308 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2022-06-29T15:40:49.360 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libleveldb1d:amd64 (1.22-3ubuntu2) ... 2022-06-29T15:40:49.395 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up liboath0 (2.6.1-1.3) ... 2022-06-29T15:40:49.429 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-ceph-argparse (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:49.495 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2022-06-29T15:40:49.521 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2022-06-29T15:40:49.592 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2022-06-29T15:40:49.621 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2022-06-29T15:40:49.755 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2022-06-29T15:40:49.823 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up cephadm (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:49.889 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2022-06-29T15:40:49.907 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-joblib (0.14.0-3) ... 2022-06-29T15:40:49.927 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2022-06-29T15:40:50.216 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2022-06-29T15:40:50.228 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2022-06-29T15:40:50.342 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-jwt (1.7.1-2ubuntu2) ... 2022-06-29T15:40:50.508 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2022-06-29T15:40:50.550 INFO:teuthology.orchestra.run.smithi099.stdout:Adding system user cephadm....done 2022-06-29T15:40:50.571 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-tempita (0.5.2-6) ... 2022-06-29T15:40:50.635 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-cachetools (4.0.0-1) ... 2022-06-29T15:40:50.712 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2022-06-29T15:40:50.802 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2022-06-29T15:40:50.881 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-packaging (20.3-1) ... 2022-06-29T15:40:50.961 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2022-06-29T15:40:51.063 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-repoze.lru (0.7-2) ... 2022-06-29T15:40:51.182 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2022-06-29T15:40:51.211 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2022-06-29T15:40:51.253 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2022-06-29T15:40:51.308 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2022-06-29T15:40:51.356 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up liboath0 (2.6.1-1.3) ... 2022-06-29T15:40:51.512 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2022-06-29T15:40:51.538 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2022-06-29T15:40:51.576 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2022-06-29T15:40:51.615 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2022-06-29T15:40:51.914 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2022-06-29T15:40:51.942 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-joblib (0.14.0-3) ... 2022-06-29T15:40:51.960 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-ceph-argparse (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:51.977 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2022-06-29T15:40:52.181 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2022-06-29T15:40:52.295 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2) ... 2022-06-29T15:40:52.298 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2022-06-29T15:40:52.377 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-jwt (1.7.1-2ubuntu2) ... 2022-06-29T15:40:52.441 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2022-06-29T15:40:52.607 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-tempita (0.5.2-6) ... 2022-06-29T15:40:52.664 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2022-06-29T15:40:52.687 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-cachetools (4.0.0-1) ... 2022-06-29T15:40:52.742 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2022-06-29T15:40:52.900 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-packaging (20.3-1) ... 2022-06-29T15:40:52.980 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2022-06-29T15:40:53.027 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2022-06-29T15:40:53.032 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up socat (1.7.3.3-2) ... 2022-06-29T15:40:53.158 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-ceph-common (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:53.265 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2022-06-29T15:40:53.268 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2022-06-29T15:40:53.328 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2022-06-29T15:40:53.394 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2022-06-29T15:40:53.443 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-html5lib (1.0.1-2) ... 2022-06-29T15:40:53.549 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2022-06-29T15:40:53.704 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2022-06-29T15:40:53.754 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2022-06-29T15:40:53.795 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2022-06-29T15:40:53.850 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-ceph-argparse (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:53.921 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-pluggy (0.13.0-2) ... 2022-06-29T15:40:53.989 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2022-06-29T15:40:54.132 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2022-06-29T15:40:54.168 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2022-06-29T15:40:54.206 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libxslt1.1:amd64 (1.1.34-4) ... 2022-06-29T15:40:54.302 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2022-06-29T15:40:54.351 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2022-06-29T15:40:54.374 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2022-06-29T15:40:54.473 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2) ... 2022-06-29T15:40:54.595 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2022-06-29T15:40:54.675 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2022-06-29T15:40:54.699 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2022-06-29T15:40:54.841 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2022-06-29T15:40:54.881 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2022-06-29T15:40:54.951 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-logutils (0.3.3-7) ... 2022-06-29T15:40:55.026 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-tempita (0.5.2-6) ... 2022-06-29T15:40:55.189 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2022-06-29T15:40:55.235 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up socat (1.7.3.3-2) ... 2022-06-29T15:40:55.235 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2022-06-29T15:40:55.327 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-packaging (20.3-1) ... 2022-06-29T15:40:55.382 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-ceph-common (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:55.386 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2022-06-29T15:40:55.590 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2022-06-29T15:40:55.661 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2022-06-29T15:40:55.662 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-prettytable (0.7.2-5) ... 2022-06-29T15:40:55.704 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-html5lib (1.0.1-2) ... 2022-06-29T15:40:55.787 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2022-06-29T15:40:55.867 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2022-06-29T15:40:55.964 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2022-06-29T15:40:56.047 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2022-06-29T15:40:56.131 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2022-06-29T15:40:56.181 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-pluggy (0.13.0-2) ... 2022-06-29T15:40:56.218 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2022-06-29T15:40:56.282 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2022-06-29T15:40:56.466 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libxslt1.1:amd64 (1.1.34-4) ... 2022-06-29T15:40:56.495 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2022-06-29T15:40:56.557 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2022-06-29T15:40:56.622 INFO:teuthology.orchestra.run.smithi164.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2022-06-29T15:40:56.625 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2022-06-29T15:40:56.854 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-tempita (0.5.2-6) ... 2022-06-29T15:40:56.874 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2) ... 2022-06-29T15:40:56.884 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2022-06-29T15:40:56.960 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2022-06-29T15:40:57.002 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2) ... 2022-06-29T15:40:57.130 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-packaging (20.3-1) ... 2022-06-29T15:40:57.234 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2022-06-29T15:40:57.303 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-logutils (0.3.3-7) ... 2022-06-29T15:40:57.320 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2022-06-29T15:40:57.465 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2022-06-29T15:40:57.578 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up socat (1.7.3.3-2) ... 2022-06-29T15:40:57.591 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2022-06-29T15:40:57.612 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2022-06-29T15:40:57.671 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2022-06-29T15:40:57.729 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-ceph-common (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:57.738 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2022-06-29T15:40:57.823 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2022-06-29T15:40:57.875 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2022-06-29T15:40:57.949 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-webob (1:1.8.5-2) ... 2022-06-29T15:40:58.014 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-prettytable (0.7.2-5) ... 2022-06-29T15:40:58.038 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-html5lib (1.0.1-2) ... 2022-06-29T15:40:58.258 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2022-06-29T15:40:58.290 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2022-06-29T15:40:58.310 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2022-06-29T15:40:58.408 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2022-06-29T15:40:58.509 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-pluggy (0.13.0-2) ... 2022-06-29T15:40:58.543 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up librados2 (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:58.600 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2022-06-29T15:40:58.628 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2) ... 2022-06-29T15:40:58.660 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2022-06-29T15:40:58.802 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libxslt1.1:amd64 (1.1.34-4) ... 2022-06-29T15:40:58.903 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2022-06-29T15:40:58.905 INFO:teuthology.orchestra.run.smithi064.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2022-06-29T15:40:58.988 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2022-06-29T15:40:59.202 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2022-06-29T15:40:59.279 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2022-06-29T15:40:59.337 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2) ... 2022-06-29T15:40:59.373 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up socat (1.7.3.3-2) ... 2022-06-29T15:40:59.499 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-ceph-common (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:40:59.572 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-logutils (0.3.3-7) ... 2022-06-29T15:40:59.671 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2022-06-29T15:40:59.817 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-html5lib (1.0.1-2) ... 2022-06-29T15:40:59.865 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2022-06-29T15:41:00.006 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2022-06-29T15:41:00.024 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2022-06-29T15:41:00.157 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2022-06-29T15:41:00.158 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up librgw2 (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:00.161 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2022-06-29T15:41:00.268 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2022-06-29T15:41:00.283 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-webob (1:1.8.5-2) ... 2022-06-29T15:41:00.286 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-pluggy (0.13.0-2) ... 2022-06-29T15:41:00.308 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-prettytable (0.7.2-5) ... 2022-06-29T15:41:00.385 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-pytest (4.6.9-1) ... 2022-06-29T15:41:00.599 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libxslt1.1:amd64 (1.1.34-4) ... 2022-06-29T15:41:00.618 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2022-06-29T15:41:00.651 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2022-06-29T15:41:00.731 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2022-06-29T15:41:00.762 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2022-06-29T15:41:00.886 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2022-06-29T15:41:00.952 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up librados2 (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:01.057 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2022-06-29T15:41:01.090 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2022-06-29T15:41:01.103 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2022-06-29T15:41:01.158 INFO:teuthology.orchestra.run.smithi047.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2022-06-29T15:41:01.350 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-logutils (0.3.3-7) ... 2022-06-29T15:41:01.391 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-bs4 (4.8.2-1) ... 2022-06-29T15:41:01.413 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2022-06-29T15:41:01.539 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2) ... 2022-06-29T15:41:01.634 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2022-06-29T15:41:01.743 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libcephfs2 (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:01.786 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2022-06-29T15:41:01.869 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2022-06-29T15:41:01.874 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2022-06-29T15:41:02.079 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-prettytable (0.7.2-5) ... 2022-06-29T15:41:02.187 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libradosstriper1 (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:02.233 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2022-06-29T15:41:02.313 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2022-06-29T15:41:02.334 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2022-06-29T15:41:02.380 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2022-06-29T15:41:02.435 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-webob (1:1.8.5-2) ... 2022-06-29T15:41:02.601 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up librgw2 (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:02.639 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2022-06-29T15:41:02.681 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2022-06-29T15:41:02.745 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2022-06-29T15:41:02.760 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2022-06-29T15:41:02.920 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-pytest (4.6.9-1) ... 2022-06-29T15:41:03.003 INFO:teuthology.orchestra.run.smithi099.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2022-06-29T15:41:03.062 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up librados2 (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:03.188 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2022-06-29T15:41:03.242 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2022-06-29T15:41:03.305 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2022-06-29T15:41:03.393 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2) ... 2022-06-29T15:41:03.622 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2022-06-29T15:41:03.653 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up librbd1 (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:03.719 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2022-06-29T15:41:03.780 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2022-06-29T15:41:03.890 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-bs4 (4.8.2-1) ... 2022-06-29T15:41:03.905 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2022-06-29T15:41:04.054 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2022-06-29T15:41:04.056 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2022-06-29T15:41:04.155 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2022-06-29T15:41:04.216 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libcephfs2 (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:04.256 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-webob (1:1.8.5-2) ... 2022-06-29T15:41:04.350 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2022-06-29T15:41:04.376 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2022-06-29T15:41:04.475 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-rsa (4.0-3) ... 2022-06-29T15:41:04.566 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2022-06-29T15:41:04.669 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up librgw2 (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:04.711 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libradosstriper1 (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:04.760 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-webtest (2.0.32-2) ... 2022-06-29T15:41:04.795 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2022-06-29T15:41:04.871 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2022-06-29T15:41:04.875 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up librados2 (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:04.946 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-pytest (4.6.9-1) ... 2022-06-29T15:41:05.001 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2022-06-29T15:41:05.053 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-mgr-modules-core (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:05.187 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-fuse (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:05.205 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2022-06-29T15:41:05.314 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2022-06-29T15:41:05.409 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2022-06-29T15:41:05.409 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2022-06-29T15:41:05.615 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2022-06-29T15:41:05.914 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libcephfs-dev (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:05.950 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-bs4 (4.8.2-1) ... 2022-06-29T15:41:06.040 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2022-06-29T15:41:06.276 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libcephfs2 (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:06.311 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up librbd1 (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:06.333 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-pastescript (2.0.2-4) ... 2022-06-29T15:41:06.377 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2022-06-29T15:41:06.432 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up librgw2 (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:06.464 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2022-06-29T15:41:06.558 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2022-06-29T15:41:06.590 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2022-06-29T15:41:06.643 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2022-06-29T15:41:06.692 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-pytest (4.6.9-1) ... 2022-06-29T15:41:06.695 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libradosstriper1 (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:06.741 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2022-06-29T15:41:06.821 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2022-06-29T15:41:06.995 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-rados (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:07.044 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2022-06-29T15:41:07.067 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2022-06-29T15:41:07.096 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-rbd (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:07.182 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2022-06-29T15:41:07.213 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up rbd-fuse (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:07.226 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-rsa (4.0-3) ... 2022-06-29T15:41:07.339 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-routes (2.4.1-2) ... 2022-06-29T15:41:07.354 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2022-06-29T15:41:07.520 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-webtest (2.0.32-2) ... 2022-06-29T15:41:07.647 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-bs4 (4.8.2-1) ... 2022-06-29T15:41:07.726 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-google-auth (1.5.1-2) ... 2022-06-29T15:41:08.006 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libcephfs2 (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:08.052 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-rgw (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:08.132 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2022-06-29T15:41:08.141 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-mgr-modules-core (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:08.153 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-cephfs (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:08.253 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up librbd1 (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:08.272 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-fuse (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:08.362 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2022-06-29T15:41:08.459 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libradosstriper1 (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:08.513 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2022-06-29T15:41:08.523 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2022-06-29T15:41:08.524 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2022-06-29T15:41:08.560 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2022-06-29T15:41:08.639 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2022-06-29T15:41:08.796 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up cephfs-shell (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:08.862 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2022-06-29T15:41:08.896 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2022-06-29T15:41:08.965 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2022-06-29T15:41:09.023 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libcephfs-dev (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:09.099 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-rsa (4.0-3) ... 2022-06-29T15:41:09.174 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2022-06-29T15:41:09.409 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-webtest (2.0.32-2) ... 2022-06-29T15:41:09.492 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-pastescript (2.0.2-4) ... 2022-06-29T15:41:09.694 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up ceph-mgr-modules-core (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:09.824 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up ceph-fuse (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:09.825 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-common (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:09.835 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2022-06-29T15:41:09.892 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up librbd1 (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:10.014 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2022-06-29T15:41:10.030 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2022-06-29T15:41:10.030 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2022-06-29T15:41:10.119 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2022-06-29T15:41:10.204 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-rados (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:10.229 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2022-06-29T15:41:10.352 INFO:teuthology.orchestra.run.smithi164.stdout:Adding group ceph....done 2022-06-29T15:41:10.363 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-rbd (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:10.487 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libcephfs-dev (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:10.515 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up rbd-fuse (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:10.538 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2022-06-29T15:41:10.613 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2022-06-29T15:41:10.639 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-rsa (4.0-3) ... 2022-06-29T15:41:10.640 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-routes (2.4.1-2) ... 2022-06-29T15:41:10.771 INFO:teuthology.orchestra.run.smithi164.stdout:Adding system user ceph....done 2022-06-29T15:41:10.923 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-pastescript (2.0.2-4) ... 2022-06-29T15:41:10.924 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-webtest (2.0.32-2) ... 2022-06-29T15:41:10.955 INFO:teuthology.orchestra.run.smithi164.stdout:Setting system user ceph properties....done 2022-06-29T15:41:10.978 INFO:teuthology.orchestra.run.smithi164.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2022-06-29T15:41:11.051 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-google-auth (1.5.1-2) ... 2022-06-29T15:41:11.128 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2022-06-29T15:41:11.209 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-mgr-modules-core (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:11.224 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2022-06-29T15:41:11.345 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-rgw (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:11.368 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-fuse (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:11.467 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2022-06-29T15:41:11.504 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-cephfs (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:11.571 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2022-06-29T15:41:11.571 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2022-06-29T15:41:11.585 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-rados (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:11.711 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-rbd (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:11.805 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up cephfs-shell (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:11.844 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up rbd-fuse (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:11.956 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2022-06-29T15:41:11.962 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-routes (2.4.1-2) ... 2022-06-29T15:41:12.219 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-test (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:12.353 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up radosgw (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:12.386 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libcephfs-dev (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:12.512 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2022-06-29T15:41:12.628 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2022-06-29T15:41:12.628 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2022-06-29T15:41:12.790 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-google-auth (1.5.1-2) ... 2022-06-29T15:41:12.805 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-pastescript (2.0.2-4) ... 2022-06-29T15:41:12.917 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-common (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:13.098 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2022-06-29T15:41:13.125 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-rgw (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:13.280 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-base (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:13.284 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-cephfs (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:13.420 INFO:teuthology.orchestra.run.smithi064.stdout:Adding group ceph....done 2022-06-29T15:41:13.433 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-rados (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:13.562 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-rbd (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:13.577 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up cephfs-shell (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:13.685 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2022-06-29T15:41:13.687 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up rbd-fuse (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:13.737 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2022-06-29T15:41:13.797 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-routes (2.4.1-2) ... 2022-06-29T15:41:13.847 INFO:teuthology.orchestra.run.smithi064.stdout:Adding system user ceph....done 2022-06-29T15:41:14.065 INFO:teuthology.orchestra.run.smithi064.stdout:Setting system user ceph properties....done 2022-06-29T15:41:14.085 INFO:teuthology.orchestra.run.smithi064.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2022-06-29T15:41:14.141 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-mds (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:14.150 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-google-auth (1.5.1-2) ... 2022-06-29T15:41:14.236 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2022-06-29T15:41:14.358 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2022-06-29T15:41:14.358 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2022-06-29T15:41:14.451 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-rgw (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:14.577 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-cephfs (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:14.587 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2022-06-29T15:41:14.665 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up ceph-common (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:14.817 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-mgr (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:14.879 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up cephfs-shell (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:15.004 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2022-06-29T15:41:15.050 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2022-06-29T15:41:15.051 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2022-06-29T15:41:15.111 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-test (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:15.245 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up radosgw (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:15.276 INFO:teuthology.orchestra.run.smithi047.stdout:Adding group ceph....done 2022-06-29T15:41:15.520 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2022-06-29T15:41:15.521 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2022-06-29T15:41:15.544 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-mgr-diskprediction-cloud (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:15.686 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-osd (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:15.694 INFO:teuthology.orchestra.run.smithi047.stdout:Adding system user ceph....done 2022-06-29T15:41:15.887 INFO:teuthology.orchestra.run.smithi047.stdout:Setting system user ceph properties....done 2022-06-29T15:41:15.913 INFO:teuthology.orchestra.run.smithi047.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2022-06-29T15:41:15.924 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-common (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:16.069 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2022-06-29T15:41:16.175 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2022-06-29T15:41:16.175 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2022-06-29T15:41:16.239 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-base (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:16.399 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2022-06-29T15:41:16.494 INFO:teuthology.orchestra.run.smithi099.stdout:Adding group ceph....done 2022-06-29T15:41:16.630 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-mgr-k8sevents (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:16.764 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-mgr-diskprediction-local (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:16.888 INFO:teuthology.orchestra.run.smithi099.stdout:Adding system user ceph....done 2022-06-29T15:41:16.891 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up ceph-test (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:16.915 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-mon (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:17.030 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2022-06-29T15:41:17.034 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up radosgw (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:17.072 INFO:teuthology.orchestra.run.smithi099.stdout:Setting system user ceph properties....done 2022-06-29T15:41:17.093 INFO:teuthology.orchestra.run.smithi099.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2022-06-29T15:41:17.144 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2022-06-29T15:41:17.144 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2022-06-29T15:41:17.242 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2022-06-29T15:41:17.307 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2022-06-29T15:41:17.307 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2022-06-29T15:41:17.533 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-mds (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:17.578 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2022-06-29T15:41:17.617 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-mgr-cephadm (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:17.759 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:17.780 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2022-06-29T15:41:17.780 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2022-06-29T15:41:17.888 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-mgr-dashboard (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:17.978 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up ceph-base (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:18.029 INFO:teuthology.orchestra.run.smithi164.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2022-06-29T15:41:18.051 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-test (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:18.203 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up radosgw (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:18.252 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-mgr (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:18.330 INFO:teuthology.orchestra.run.smithi164.stdout:Processing triggers for man-db (2.9.1-1) ... 2022-06-29T15:41:18.391 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2022-06-29T15:41:18.480 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2022-06-29T15:41:18.480 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2022-06-29T15:41:18.485 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2022-06-29T15:41:18.485 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2022-06-29T15:41:18.838 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up ceph-mds (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:18.987 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-mgr-diskprediction-cloud (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:19.079 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2022-06-29T15:41:19.079 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2022-06-29T15:41:19.155 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-osd (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:19.205 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-base (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:19.573 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up ceph-mgr (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:19.622 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2022-06-29T15:41:19.663 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2022-06-29T15:41:19.663 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2022-06-29T15:41:19.827 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2022-06-29T15:41:19.827 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2022-06-29T15:41:20.132 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-mds (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:20.140 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-mgr-k8sevents (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:20.299 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-mgr-diskprediction-local (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:20.350 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up ceph-mgr-diskprediction-cloud (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:20.362 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2022-06-29T15:41:20.362 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2022-06-29T15:41:20.442 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-mon (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:20.468 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up ceph-osd (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:20.694 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2022-06-29T15:41:20.695 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2022-06-29T15:41:20.843 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-mgr (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:20.987 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2022-06-29T15:41:20.987 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2022-06-29T15:41:21.066 INFO:teuthology.orchestra.run.smithi164.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2022-06-29T15:41:21.067 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2022-06-29T15:41:21.067 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2022-06-29T15:41:21.168 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-mgr-cephadm (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:21.344 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:21.436 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up ceph-mgr-k8sevents (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:21.495 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-mgr-dashboard (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:21.521 INFO:teuthology.orchestra.run.smithi164.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2022-06-29T15:41:21.524 DEBUG:teuthology.orchestra.run.smithi164:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2022-06-29T15:41:21.544 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-mgr-diskprediction-cloud (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:21.562 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up ceph-mgr-diskprediction-local (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:21.596 INFO:teuthology.orchestra.run.smithi164.stdout:Reading package lists... 2022-06-29T15:41:21.639 INFO:teuthology.orchestra.run.smithi064.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2022-06-29T15:41:21.704 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-osd (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:21.730 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up ceph-mon (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:21.775 INFO:teuthology.orchestra.run.smithi164.stdout:Building dependency tree... 2022-06-29T15:41:21.775 INFO:teuthology.orchestra.run.smithi164.stdout:Reading state information... 2022-06-29T15:41:21.932 INFO:teuthology.orchestra.run.smithi164.stdout:libtool-bin is already the newest version (2.4.6-14). 2022-06-29T15:41:21.933 INFO:teuthology.orchestra.run.smithi164.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2022-06-29T15:41:21.933 INFO:teuthology.orchestra.run.smithi164.stdout:uuid-dev is already the newest version (2.34-0.1ubuntu9.3). 2022-06-29T15:41:21.933 INFO:teuthology.orchestra.run.smithi164.stdout:The following packages were automatically installed and are no longer required: 2022-06-29T15:41:21.934 INFO:teuthology.orchestra.run.smithi164.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2022-06-29T15:41:21.934 INFO:teuthology.orchestra.run.smithi164.stdout:Use 'sudo apt autoremove' to remove them. 2022-06-29T15:41:21.935 INFO:teuthology.orchestra.run.smithi164.stdout:The following additional packages will be installed: 2022-06-29T15:41:21.935 INFO:teuthology.orchestra.run.smithi164.stdout: libfl-dev libfl2 liblzo2-2 libssl1.1 2022-06-29T15:41:21.935 INFO:teuthology.orchestra.run.smithi164.stdout:Suggested packages: 2022-06-29T15:41:21.936 INFO:teuthology.orchestra.run.smithi164.stdout: bison-doc flex-doc indent-doc libssl-doc 2022-06-29T15:41:21.964 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2022-06-29T15:41:21.964 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2022-06-29T15:41:21.965 INFO:teuthology.orchestra.run.smithi064.stdout:Processing triggers for man-db (2.9.1-1) ... 2022-06-29T15:41:21.969 INFO:teuthology.orchestra.run.smithi164.stdout:The following NEW packages will be installed: 2022-06-29T15:41:21.969 INFO:teuthology.orchestra.run.smithi164.stdout: bison dump flex indent libaio-dev libelf-dev libfl-dev libfl2 liblzo2-2 2022-06-29T15:41:21.970 INFO:teuthology.orchestra.run.smithi164.stdout: libssl-dev 2022-06-29T15:41:21.971 INFO:teuthology.orchestra.run.smithi164.stdout:The following packages will be upgraded: 2022-06-29T15:41:21.972 INFO:teuthology.orchestra.run.smithi164.stdout: libssl1.1 2022-06-29T15:41:22.150 INFO:teuthology.orchestra.run.smithi164.stdout:1 upgraded, 10 newly installed, 0 to remove and 6 not upgraded. 2022-06-29T15:41:22.151 INFO:teuthology.orchestra.run.smithi164.stdout:Need to get 4,234 kB of archives. 2022-06-29T15:41:22.151 INFO:teuthology.orchestra.run.smithi164.stdout:After this operation, 12.4 MB of additional disk space will be used. 2022-06-29T15:41:22.151 INFO:teuthology.orchestra.run.smithi164.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 flex amd64 2.6.4-6.2 [317 kB] 2022-06-29T15:41:22.211 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2022-06-29T15:41:22.211 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2022-06-29T15:41:22.398 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up ceph-mgr-cephadm (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:22.541 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up ceph (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:22.630 INFO:teuthology.orchestra.run.smithi164.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.15 [1,321 kB] 2022-06-29T15:41:22.667 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up ceph-mgr-dashboard (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:22.689 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-mgr-k8sevents (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:22.786 INFO:teuthology.orchestra.run.smithi047.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2022-06-29T15:41:22.833 INFO:teuthology.orchestra.run.smithi164.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 bison amd64 2:3.5.1+dfsg-1 [657 kB] 2022-06-29T15:41:22.849 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-mgr-diskprediction-local (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:22.882 INFO:teuthology.orchestra.run.smithi164.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2022-06-29T15:41:22.884 INFO:teuthology.orchestra.run.smithi164.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2022-06-29T15:41:22.890 INFO:teuthology.orchestra.run.smithi164.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 indent amd64 2.2.12-1 [86.9 kB] 2022-06-29T15:41:22.895 INFO:teuthology.orchestra.run.smithi164.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2022-06-29T15:41:22.896 INFO:teuthology.orchestra.run.smithi164.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 libelf-dev amd64 0.176-1.1build1 [57.0 kB] 2022-06-29T15:41:22.898 INFO:teuthology.orchestra.run.smithi164.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl2 amd64 2.6.4-6.2 [11.5 kB] 2022-06-29T15:41:22.899 INFO:teuthology.orchestra.run.smithi164.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl-dev amd64 2.6.4-6.2 [6,316 B] 2022-06-29T15:41:22.899 INFO:teuthology.orchestra.run.smithi164.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.15 [1,584 kB] 2022-06-29T15:41:22.991 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-mon (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:23.128 INFO:teuthology.orchestra.run.smithi047.stdout:Processing triggers for man-db (2.9.1-1) ... 2022-06-29T15:41:23.218 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2022-06-29T15:41:23.219 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2022-06-29T15:41:23.263 INFO:teuthology.orchestra.run.smithi164.stdout:Preconfiguring packages ... 2022-06-29T15:41:23.401 INFO:teuthology.orchestra.run.smithi164.stdout:Fetched 4,234 kB in 1s (4,333 kB/s) 2022-06-29T15:41:23.522 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package flex. 2022-06-29T15:41:23.580 INFO:teuthology.orchestra.run.smithi164.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 146195 files and directories currently installed.) 2022-06-29T15:41:23.584 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../00-flex_2.6.4-6.2_amd64.deb ... 2022-06-29T15:41:23.628 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking flex (2.6.4-6.2) ... 2022-06-29T15:41:23.693 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-mgr-cephadm (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:23.836 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:23.962 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-mgr-dashboard (15.2.16-304-gf25ffac7-1focal) ... 2022-06-29T15:41:24.114 INFO:teuthology.orchestra.run.smithi099.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2022-06-29T15:41:24.448 INFO:teuthology.orchestra.run.smithi099.stdout:Processing triggers for man-db (2.9.1-1) ... 2022-06-29T15:41:24.576 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../01-libssl1.1_1.1.1f-1ubuntu2.15_amd64.deb ... 2022-06-29T15:41:24.769 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.15) over (1.1.1f-1ubuntu2.13) ... 2022-06-29T15:41:24.810 INFO:teuthology.orchestra.run.smithi064.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2022-06-29T15:41:25.235 INFO:teuthology.orchestra.run.smithi064.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2022-06-29T15:41:25.237 DEBUG:teuthology.orchestra.run.smithi064:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2022-06-29T15:41:25.311 INFO:teuthology.orchestra.run.smithi064.stdout:Reading package lists... 2022-06-29T15:41:25.314 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package bison. 2022-06-29T15:41:25.331 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../02-bison_2%3a3.5.1+dfsg-1_amd64.deb ... 2022-06-29T15:41:25.376 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking bison (2:3.5.1+dfsg-1) ... 2022-06-29T15:41:25.493 INFO:teuthology.orchestra.run.smithi064.stdout:Building dependency tree... 2022-06-29T15:41:25.493 INFO:teuthology.orchestra.run.smithi064.stdout:Reading state information... 2022-06-29T15:41:25.657 INFO:teuthology.orchestra.run.smithi064.stdout:libtool-bin is already the newest version (2.4.6-14). 2022-06-29T15:41:25.658 INFO:teuthology.orchestra.run.smithi064.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2022-06-29T15:41:25.658 INFO:teuthology.orchestra.run.smithi064.stdout:uuid-dev is already the newest version (2.34-0.1ubuntu9.3). 2022-06-29T15:41:25.658 INFO:teuthology.orchestra.run.smithi064.stdout:The following packages were automatically installed and are no longer required: 2022-06-29T15:41:25.659 INFO:teuthology.orchestra.run.smithi064.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2022-06-29T15:41:25.659 INFO:teuthology.orchestra.run.smithi064.stdout:Use 'sudo apt autoremove' to remove them. 2022-06-29T15:41:25.660 INFO:teuthology.orchestra.run.smithi064.stdout:The following additional packages will be installed: 2022-06-29T15:41:25.660 INFO:teuthology.orchestra.run.smithi064.stdout: libfl-dev libfl2 liblzo2-2 libssl1.1 2022-06-29T15:41:25.660 INFO:teuthology.orchestra.run.smithi064.stdout:Suggested packages: 2022-06-29T15:41:25.661 INFO:teuthology.orchestra.run.smithi064.stdout: bison-doc flex-doc indent-doc libssl-doc 2022-06-29T15:41:25.696 INFO:teuthology.orchestra.run.smithi064.stdout:The following NEW packages will be installed: 2022-06-29T15:41:25.697 INFO:teuthology.orchestra.run.smithi064.stdout: bison dump flex indent libaio-dev libelf-dev libfl-dev libfl2 liblzo2-2 2022-06-29T15:41:25.697 INFO:teuthology.orchestra.run.smithi064.stdout: libssl-dev 2022-06-29T15:41:25.699 INFO:teuthology.orchestra.run.smithi064.stdout:The following packages will be upgraded: 2022-06-29T15:41:25.700 INFO:teuthology.orchestra.run.smithi064.stdout: libssl1.1 2022-06-29T15:41:25.802 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package liblzo2-2:amd64. 2022-06-29T15:41:25.819 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2022-06-29T15:41:25.860 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2022-06-29T15:41:25.881 INFO:teuthology.orchestra.run.smithi064.stdout:1 upgraded, 10 newly installed, 0 to remove and 6 not upgraded. 2022-06-29T15:41:25.882 INFO:teuthology.orchestra.run.smithi064.stdout:Need to get 4,234 kB of archives. 2022-06-29T15:41:25.882 INFO:teuthology.orchestra.run.smithi064.stdout:After this operation, 12.4 MB of additional disk space will be used. 2022-06-29T15:41:25.882 INFO:teuthology.orchestra.run.smithi064.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 flex amd64 2.6.4-6.2 [317 kB] 2022-06-29T15:41:26.223 INFO:teuthology.orchestra.run.smithi047.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2022-06-29T15:41:26.229 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package dump. 2022-06-29T15:41:26.247 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2022-06-29T15:41:26.288 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking dump (0.4b46-6) ... 2022-06-29T15:41:26.360 INFO:teuthology.orchestra.run.smithi064.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.15 [1,321 kB] 2022-06-29T15:41:26.564 INFO:teuthology.orchestra.run.smithi064.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 bison amd64 2:3.5.1+dfsg-1 [657 kB] 2022-06-29T15:41:26.591 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package indent. 2022-06-29T15:41:26.609 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../05-indent_2.2.12-1_amd64.deb ... 2022-06-29T15:41:26.612 INFO:teuthology.orchestra.run.smithi064.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2022-06-29T15:41:26.615 INFO:teuthology.orchestra.run.smithi064.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2022-06-29T15:41:26.621 INFO:teuthology.orchestra.run.smithi064.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 indent amd64 2.2.12-1 [86.9 kB] 2022-06-29T15:41:26.625 INFO:teuthology.orchestra.run.smithi064.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2022-06-29T15:41:26.626 INFO:teuthology.orchestra.run.smithi064.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 libelf-dev amd64 0.176-1.1build1 [57.0 kB] 2022-06-29T15:41:26.629 INFO:teuthology.orchestra.run.smithi064.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl2 amd64 2.6.4-6.2 [11.5 kB] 2022-06-29T15:41:26.630 INFO:teuthology.orchestra.run.smithi064.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl-dev amd64 2.6.4-6.2 [6,316 B] 2022-06-29T15:41:26.630 INFO:teuthology.orchestra.run.smithi064.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.15 [1,584 kB] 2022-06-29T15:41:26.640 INFO:teuthology.orchestra.run.smithi047.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2022-06-29T15:41:26.643 DEBUG:teuthology.orchestra.run.smithi047:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2022-06-29T15:41:26.649 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking indent (2.2.12-1) ... 2022-06-29T15:41:26.711 INFO:teuthology.orchestra.run.smithi047.stdout:Reading package lists... 2022-06-29T15:41:26.893 INFO:teuthology.orchestra.run.smithi047.stdout:Building dependency tree... 2022-06-29T15:41:26.894 INFO:teuthology.orchestra.run.smithi047.stdout:Reading state information... 2022-06-29T15:41:26.919 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libaio-dev:amd64. 2022-06-29T15:41:26.936 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2022-06-29T15:41:26.977 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2022-06-29T15:41:26.995 INFO:teuthology.orchestra.run.smithi064.stdout:Preconfiguring packages ... 2022-06-29T15:41:27.057 INFO:teuthology.orchestra.run.smithi047.stdout:libtool-bin is already the newest version (2.4.6-14). 2022-06-29T15:41:27.058 INFO:teuthology.orchestra.run.smithi047.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2022-06-29T15:41:27.058 INFO:teuthology.orchestra.run.smithi047.stdout:uuid-dev is already the newest version (2.34-0.1ubuntu9.3). 2022-06-29T15:41:27.058 INFO:teuthology.orchestra.run.smithi047.stdout:The following packages were automatically installed and are no longer required: 2022-06-29T15:41:27.059 INFO:teuthology.orchestra.run.smithi047.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2022-06-29T15:41:27.059 INFO:teuthology.orchestra.run.smithi047.stdout:Use 'sudo apt autoremove' to remove them. 2022-06-29T15:41:27.060 INFO:teuthology.orchestra.run.smithi047.stdout:The following additional packages will be installed: 2022-06-29T15:41:27.060 INFO:teuthology.orchestra.run.smithi047.stdout: libfl-dev libfl2 liblzo2-2 libssl1.1 2022-06-29T15:41:27.060 INFO:teuthology.orchestra.run.smithi047.stdout:Suggested packages: 2022-06-29T15:41:27.061 INFO:teuthology.orchestra.run.smithi047.stdout: bison-doc flex-doc indent-doc libssl-doc 2022-06-29T15:41:27.094 INFO:teuthology.orchestra.run.smithi047.stdout:The following NEW packages will be installed: 2022-06-29T15:41:27.095 INFO:teuthology.orchestra.run.smithi047.stdout: bison dump flex indent libaio-dev libelf-dev libfl-dev libfl2 liblzo2-2 2022-06-29T15:41:27.095 INFO:teuthology.orchestra.run.smithi047.stdout: libssl-dev 2022-06-29T15:41:27.097 INFO:teuthology.orchestra.run.smithi047.stdout:The following packages will be upgraded: 2022-06-29T15:41:27.097 INFO:teuthology.orchestra.run.smithi047.stdout: libssl1.1 2022-06-29T15:41:27.126 INFO:teuthology.orchestra.run.smithi064.stdout:Fetched 4,234 kB in 1s (4,326 kB/s) 2022-06-29T15:41:27.264 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libelf-dev:amd64. 2022-06-29T15:41:27.265 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package flex. 2022-06-29T15:41:27.267 INFO:teuthology.orchestra.run.smithi099.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2022-06-29T15:41:27.275 INFO:teuthology.orchestra.run.smithi047.stdout:1 upgraded, 10 newly installed, 0 to remove and 6 not upgraded. 2022-06-29T15:41:27.275 INFO:teuthology.orchestra.run.smithi047.stdout:Need to get 4,234 kB of archives. 2022-06-29T15:41:27.276 INFO:teuthology.orchestra.run.smithi047.stdout:After this operation, 12.4 MB of additional disk space will be used. 2022-06-29T15:41:27.276 INFO:teuthology.orchestra.run.smithi047.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 flex amd64 2.6.4-6.2 [317 kB] 2022-06-29T15:41:27.281 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../07-libelf-dev_0.176-1.1build1_amd64.deb ... 2022-06-29T15:41:27.322 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libelf-dev:amd64 (0.176-1.1build1) ... 2022-06-29T15:41:27.322 INFO:teuthology.orchestra.run.smithi064.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 146195 files and directories currently installed.) 2022-06-29T15:41:27.327 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../00-flex_2.6.4-6.2_amd64.deb ... 2022-06-29T15:41:27.389 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking flex (2.6.4-6.2) ... 2022-06-29T15:41:27.667 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libfl2:amd64. 2022-06-29T15:41:27.686 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../08-libfl2_2.6.4-6.2_amd64.deb ... 2022-06-29T15:41:27.733 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libfl2:amd64 (2.6.4-6.2) ... 2022-06-29T15:41:27.745 INFO:teuthology.orchestra.run.smithi099.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2022-06-29T15:41:27.746 INFO:teuthology.orchestra.run.smithi047.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.15 [1,321 kB] 2022-06-29T15:41:27.747 DEBUG:teuthology.orchestra.run.smithi099:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2022-06-29T15:41:27.819 INFO:teuthology.orchestra.run.smithi099.stdout:Reading package lists... 2022-06-29T15:41:27.950 INFO:teuthology.orchestra.run.smithi047.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 bison amd64 2:3.5.1+dfsg-1 [657 kB] 2022-06-29T15:41:27.994 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libfl-dev:amd64. 2022-06-29T15:41:27.995 INFO:teuthology.orchestra.run.smithi099.stdout:Building dependency tree... 2022-06-29T15:41:27.995 INFO:teuthology.orchestra.run.smithi099.stdout:Reading state information... 2022-06-29T15:41:27.997 INFO:teuthology.orchestra.run.smithi047.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2022-06-29T15:41:28.000 INFO:teuthology.orchestra.run.smithi047.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2022-06-29T15:41:28.006 INFO:teuthology.orchestra.run.smithi047.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 indent amd64 2.2.12-1 [86.9 kB] 2022-06-29T15:41:28.010 INFO:teuthology.orchestra.run.smithi047.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2022-06-29T15:41:28.011 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../09-libfl-dev_2.6.4-6.2_amd64.deb ... 2022-06-29T15:41:28.011 INFO:teuthology.orchestra.run.smithi047.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 libelf-dev amd64 0.176-1.1build1 [57.0 kB] 2022-06-29T15:41:28.014 INFO:teuthology.orchestra.run.smithi047.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl2 amd64 2.6.4-6.2 [11.5 kB] 2022-06-29T15:41:28.015 INFO:teuthology.orchestra.run.smithi047.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl-dev amd64 2.6.4-6.2 [6,316 B] 2022-06-29T15:41:28.015 INFO:teuthology.orchestra.run.smithi047.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.15 [1,584 kB] 2022-06-29T15:41:28.028 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../01-libssl1.1_1.1.1f-1ubuntu2.15_amd64.deb ... 2022-06-29T15:41:28.052 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libfl-dev:amd64 (2.6.4-6.2) ... 2022-06-29T15:41:28.151 INFO:teuthology.orchestra.run.smithi099.stdout:libtool-bin is already the newest version (2.4.6-14). 2022-06-29T15:41:28.152 INFO:teuthology.orchestra.run.smithi099.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2022-06-29T15:41:28.152 INFO:teuthology.orchestra.run.smithi099.stdout:uuid-dev is already the newest version (2.34-0.1ubuntu9.3). 2022-06-29T15:41:28.152 INFO:teuthology.orchestra.run.smithi099.stdout:The following packages were automatically installed and are no longer required: 2022-06-29T15:41:28.153 INFO:teuthology.orchestra.run.smithi099.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2022-06-29T15:41:28.153 INFO:teuthology.orchestra.run.smithi099.stdout:Use 'sudo apt autoremove' to remove them. 2022-06-29T15:41:28.154 INFO:teuthology.orchestra.run.smithi099.stdout:The following additional packages will be installed: 2022-06-29T15:41:28.154 INFO:teuthology.orchestra.run.smithi099.stdout: libfl-dev libfl2 liblzo2-2 libssl1.1 2022-06-29T15:41:28.154 INFO:teuthology.orchestra.run.smithi099.stdout:Suggested packages: 2022-06-29T15:41:28.155 INFO:teuthology.orchestra.run.smithi099.stdout: bison-doc flex-doc indent-doc libssl-doc 2022-06-29T15:41:28.187 INFO:teuthology.orchestra.run.smithi099.stdout:The following NEW packages will be installed: 2022-06-29T15:41:28.187 INFO:teuthology.orchestra.run.smithi099.stdout: bison dump flex indent libaio-dev libelf-dev libfl-dev libfl2 liblzo2-2 2022-06-29T15:41:28.188 INFO:teuthology.orchestra.run.smithi099.stdout: libssl-dev 2022-06-29T15:41:28.189 INFO:teuthology.orchestra.run.smithi099.stdout:The following packages will be upgraded: 2022-06-29T15:41:28.190 INFO:teuthology.orchestra.run.smithi099.stdout: libssl1.1 2022-06-29T15:41:28.243 INFO:teuthology.orchestra.run.smithi099.stdout:1 upgraded, 10 newly installed, 0 to remove and 6 not upgraded. 2022-06-29T15:41:28.243 INFO:teuthology.orchestra.run.smithi099.stdout:Need to get 4,234 kB of archives. 2022-06-29T15:41:28.243 INFO:teuthology.orchestra.run.smithi099.stdout:After this operation, 12.4 MB of additional disk space will be used. 2022-06-29T15:41:28.244 INFO:teuthology.orchestra.run.smithi099.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 flex amd64 2.6.4-6.2 [317 kB] 2022-06-29T15:41:28.253 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.15) over (1.1.1f-1ubuntu2.13) ... 2022-06-29T15:41:28.322 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libssl-dev:amd64. 2022-06-29T15:41:28.339 INFO:teuthology.orchestra.run.smithi099.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.15 [1,321 kB] 2022-06-29T15:41:28.342 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../10-libssl-dev_1.1.1f-1ubuntu2.15_amd64.deb ... 2022-06-29T15:41:28.380 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.15) ... 2022-06-29T15:41:28.381 INFO:teuthology.orchestra.run.smithi099.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 bison amd64 2:3.5.1+dfsg-1 [657 kB] 2022-06-29T15:41:28.382 INFO:teuthology.orchestra.run.smithi047.stdout:Preconfiguring packages ... 2022-06-29T15:41:28.391 INFO:teuthology.orchestra.run.smithi099.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2022-06-29T15:41:28.392 INFO:teuthology.orchestra.run.smithi099.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2022-06-29T15:41:28.393 INFO:teuthology.orchestra.run.smithi099.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 indent amd64 2.2.12-1 [86.9 kB] 2022-06-29T15:41:28.394 INFO:teuthology.orchestra.run.smithi099.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2022-06-29T15:41:28.394 INFO:teuthology.orchestra.run.smithi099.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 libelf-dev amd64 0.176-1.1build1 [57.0 kB] 2022-06-29T15:41:28.395 INFO:teuthology.orchestra.run.smithi099.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl2 amd64 2.6.4-6.2 [11.5 kB] 2022-06-29T15:41:28.395 INFO:teuthology.orchestra.run.smithi099.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl-dev amd64 2.6.4-6.2 [6,316 B] 2022-06-29T15:41:28.395 INFO:teuthology.orchestra.run.smithi099.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.15 [1,584 kB] 2022-06-29T15:41:28.600 INFO:teuthology.orchestra.run.smithi047.stdout:Fetched 4,234 kB in 1s (4,382 kB/s) 2022-06-29T15:41:28.713 INFO:teuthology.orchestra.run.smithi099.stdout:Preconfiguring packages ... 2022-06-29T15:41:28.746 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package flex. 2022-06-29T15:41:28.804 INFO:teuthology.orchestra.run.smithi047.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 146195 files and directories currently installed.) 2022-06-29T15:41:28.808 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../00-flex_2.6.4-6.2_amd64.deb ... 2022-06-29T15:41:28.844 INFO:teuthology.orchestra.run.smithi099.stdout:Fetched 4,234 kB in 0s (19.9 MB/s) 2022-06-29T15:41:28.849 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package bison. 2022-06-29T15:41:28.866 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../02-bison_2%3a3.5.1+dfsg-1_amd64.deb ... 2022-06-29T15:41:28.867 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up flex (2.6.4-6.2) ... 2022-06-29T15:41:28.877 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking flex (2.6.4-6.2) ... 2022-06-29T15:41:28.912 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking bison (2:3.5.1+dfsg-1) ... 2022-06-29T15:41:28.991 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package flex. 2022-06-29T15:41:28.993 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.15) ... 2022-06-29T15:41:29.048 INFO:teuthology.orchestra.run.smithi099.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 146195 files and directories currently installed.) 2022-06-29T15:41:29.052 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../00-flex_2.6.4-6.2_amd64.deb ... 2022-06-29T15:41:29.089 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking flex (2.6.4-6.2) ... 2022-06-29T15:41:29.370 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2022-06-29T15:41:29.386 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package liblzo2-2:amd64. 2022-06-29T15:41:29.404 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2022-06-29T15:41:29.445 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2022-06-29T15:41:29.496 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2022-06-29T15:41:29.526 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../01-libssl1.1_1.1.1f-1ubuntu2.15_amd64.deb ... 2022-06-29T15:41:29.639 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.15) ... 2022-06-29T15:41:29.700 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.15) over (1.1.1f-1ubuntu2.13) ... 2022-06-29T15:41:29.711 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../01-libssl1.1_1.1.1f-1ubuntu2.15_amd64.deb ... 2022-06-29T15:41:29.782 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libelf-dev:amd64 (0.176-1.1build1) ... 2022-06-29T15:41:29.840 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package dump. 2022-06-29T15:41:29.857 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2022-06-29T15:41:29.879 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.15) over (1.1.1f-1ubuntu2.13) ... 2022-06-29T15:41:29.898 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking dump (0.4b46-6) ... 2022-06-29T15:41:29.908 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libfl2:amd64 (2.6.4-6.2) ... 2022-06-29T15:41:30.059 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up dump (0.4b46-6) ... 2022-06-29T15:41:30.155 INFO:teuthology.orchestra.run.smithi164.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2022-06-29T15:41:30.212 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package bison. 2022-06-29T15:41:30.219 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up bison (2:3.5.1+dfsg-1) ... 2022-06-29T15:41:30.232 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../02-bison_2%3a3.5.1+dfsg-1_amd64.deb ... 2022-06-29T15:41:30.251 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package indent. 2022-06-29T15:41:30.271 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../05-indent_2.2.12-1_amd64.deb ... 2022-06-29T15:41:30.275 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking bison (2:3.5.1+dfsg-1) ... 2022-06-29T15:41:30.309 INFO:teuthology.orchestra.run.smithi164.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2022-06-29T15:41:30.309 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking indent (2.2.12-1) ... 2022-06-29T15:41:30.379 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up indent (2.2.12-1) ... 2022-06-29T15:41:30.416 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package bison. 2022-06-29T15:41:30.434 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../02-bison_2%3a3.5.1+dfsg-1_amd64.deb ... 2022-06-29T15:41:30.479 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking bison (2:3.5.1+dfsg-1) ... 2022-06-29T15:41:30.505 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libfl-dev:amd64 (2.6.4-6.2) ... 2022-06-29T15:41:30.612 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libaio-dev:amd64. 2022-06-29T15:41:30.629 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2022-06-29T15:41:30.640 INFO:teuthology.orchestra.run.smithi164.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2022-06-29T15:41:30.679 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2022-06-29T15:41:30.750 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package liblzo2-2:amd64. 2022-06-29T15:41:30.766 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2022-06-29T15:41:30.800 INFO:teuthology.orchestra.run.smithi164.stdout:Processing triggers for man-db (2.9.1-1) ... 2022-06-29T15:41:30.808 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2022-06-29T15:41:30.946 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package liblzo2-2:amd64. 2022-06-29T15:41:30.962 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2022-06-29T15:41:30.999 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libelf-dev:amd64. 2022-06-29T15:41:31.004 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2022-06-29T15:41:31.016 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../07-libelf-dev_0.176-1.1build1_amd64.deb ... 2022-06-29T15:41:31.057 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libelf-dev:amd64 (0.176-1.1build1) ... 2022-06-29T15:41:31.169 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package dump. 2022-06-29T15:41:31.186 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2022-06-29T15:41:31.227 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking dump (0.4b46-6) ... 2022-06-29T15:41:31.349 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package dump. 2022-06-29T15:41:31.366 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2022-06-29T15:41:31.407 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking dump (0.4b46-6) ... 2022-06-29T15:41:31.469 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libfl2:amd64. 2022-06-29T15:41:31.486 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../08-libfl2_2.6.4-6.2_amd64.deb ... 2022-06-29T15:41:31.506 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package indent. 2022-06-29T15:41:31.522 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../05-indent_2.2.12-1_amd64.deb ... 2022-06-29T15:41:31.526 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libfl2:amd64 (2.6.4-6.2) ... 2022-06-29T15:41:31.581 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking indent (2.2.12-1) ... 2022-06-29T15:41:31.745 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package indent. 2022-06-29T15:41:31.761 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../05-indent_2.2.12-1_amd64.deb ... 2022-06-29T15:41:31.803 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking indent (2.2.12-1) ... 2022-06-29T15:41:31.834 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libaio-dev:amd64. 2022-06-29T15:41:31.852 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2022-06-29T15:41:31.854 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libfl-dev:amd64. 2022-06-29T15:41:31.871 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../09-libfl-dev_2.6.4-6.2_amd64.deb ... 2022-06-29T15:41:31.892 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2022-06-29T15:41:31.921 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libfl-dev:amd64 (2.6.4-6.2) ... 2022-06-29T15:41:32.056 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libaio-dev:amd64. 2022-06-29T15:41:32.072 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2022-06-29T15:41:32.105 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2022-06-29T15:41:32.203 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libelf-dev:amd64. 2022-06-29T15:41:32.216 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libssl-dev:amd64. 2022-06-29T15:41:32.221 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../07-libelf-dev_0.176-1.1build1_amd64.deb ... 2022-06-29T15:41:32.233 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../10-libssl-dev_1.1.1f-1ubuntu2.15_amd64.deb ... 2022-06-29T15:41:32.261 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libelf-dev:amd64 (0.176-1.1build1) ... 2022-06-29T15:41:32.274 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.15) ... 2022-06-29T15:41:32.409 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libelf-dev:amd64. 2022-06-29T15:41:32.429 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../07-libelf-dev_0.176-1.1build1_amd64.deb ... 2022-06-29T15:41:32.467 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libelf-dev:amd64 (0.176-1.1build1) ... 2022-06-29T15:41:32.623 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libfl2:amd64. 2022-06-29T15:41:32.639 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../08-libfl2_2.6.4-6.2_amd64.deb ... 2022-06-29T15:41:32.681 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libfl2:amd64 (2.6.4-6.2) ... 2022-06-29T15:41:32.802 INFO:teuthology.orchestra.run.smithi164.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2022-06-29T15:41:32.819 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up flex (2.6.4-6.2) ... 2022-06-29T15:41:32.920 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.15) ... 2022-06-29T15:41:32.934 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libfl-dev:amd64. 2022-06-29T15:41:32.953 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../09-libfl-dev_2.6.4-6.2_amd64.deb ... 2022-06-29T15:41:32.963 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libfl2:amd64. 2022-06-29T15:41:32.980 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../08-libfl2_2.6.4-6.2_amd64.deb ... 2022-06-29T15:41:32.983 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libfl-dev:amd64 (2.6.4-6.2) ... 2022-06-29T15:41:33.022 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libfl2:amd64 (2.6.4-6.2) ... 2022-06-29T15:41:33.203 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libssl-dev:amd64. 2022-06-29T15:41:33.219 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../10-libssl-dev_1.1.1f-1ubuntu2.15_amd64.deb ... 2022-06-29T15:41:33.241 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libfl-dev:amd64. 2022-06-29T15:41:33.258 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../09-libfl-dev_2.6.4-6.2_amd64.deb ... 2022-06-29T15:41:33.261 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.15) ... 2022-06-29T15:41:33.289 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2022-06-29T15:41:33.290 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libfl-dev:amd64 (2.6.4-6.2) ... 2022-06-29T15:41:33.414 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2022-06-29T15:41:33.519 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libssl-dev:amd64. 2022-06-29T15:41:33.527 INFO:teuthology.orchestra.run.smithi164.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2022-06-29T15:41:33.530 DEBUG:teuthology.parallel:result is None 2022-06-29T15:41:33.535 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../10-libssl-dev_1.1.1f-1ubuntu2.15_amd64.deb ... 2022-06-29T15:41:33.602 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.15) ... 2022-06-29T15:41:33.817 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.15) ... 2022-06-29T15:41:33.913 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up flex (2.6.4-6.2) ... 2022-06-29T15:41:33.943 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libelf-dev:amd64 (0.176-1.1build1) ... 2022-06-29T15:41:34.034 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.15) ... 2022-06-29T15:41:34.053 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up flex (2.6.4-6.2) ... 2022-06-29T15:41:34.110 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libfl2:amd64 (2.6.4-6.2) ... 2022-06-29T15:41:34.182 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.15) ... 2022-06-29T15:41:34.261 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up dump (0.4b46-6) ... 2022-06-29T15:41:34.352 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2022-06-29T15:41:34.357 INFO:teuthology.orchestra.run.smithi064.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2022-06-29T15:41:34.430 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up bison (2:3.5.1+dfsg-1) ... 2022-06-29T15:41:34.478 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2022-06-29T15:41:34.517 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2022-06-29T15:41:34.544 INFO:teuthology.orchestra.run.smithi064.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2022-06-29T15:41:34.604 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.15) ... 2022-06-29T15:41:34.606 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up indent (2.2.12-1) ... 2022-06-29T15:41:34.643 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2022-06-29T15:41:34.754 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libelf-dev:amd64 (0.176-1.1build1) ... 2022-06-29T15:41:34.769 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.15) ... 2022-06-29T15:41:34.782 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libfl-dev:amd64 (2.6.4-6.2) ... 2022-06-29T15:41:34.880 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libfl2:amd64 (2.6.4-6.2) ... 2022-06-29T15:41:34.909 INFO:teuthology.orchestra.run.smithi064.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2022-06-29T15:41:34.912 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libelf-dev:amd64 (0.176-1.1build1) ... 2022-06-29T15:41:35.006 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up dump (0.4b46-6) ... 2022-06-29T15:41:35.055 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libfl2:amd64 (2.6.4-6.2) ... 2022-06-29T15:41:35.085 INFO:teuthology.orchestra.run.smithi064.stdout:Processing triggers for man-db (2.9.1-1) ... 2022-06-29T15:41:35.152 INFO:teuthology.orchestra.run.smithi047.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2022-06-29T15:41:35.189 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up dump (0.4b46-6) ... 2022-06-29T15:41:35.217 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up bison (2:3.5.1+dfsg-1) ... 2022-06-29T15:41:35.285 INFO:teuthology.orchestra.run.smithi099.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2022-06-29T15:41:35.306 INFO:teuthology.orchestra.run.smithi047.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2022-06-29T15:41:35.349 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up bison (2:3.5.1+dfsg-1) ... 2022-06-29T15:41:35.376 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up indent (2.2.12-1) ... 2022-06-29T15:41:35.456 INFO:teuthology.orchestra.run.smithi099.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2022-06-29T15:41:35.502 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libfl-dev:amd64 (2.6.4-6.2) ... 2022-06-29T15:41:35.526 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up indent (2.2.12-1) ... 2022-06-29T15:41:35.663 INFO:teuthology.orchestra.run.smithi047.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2022-06-29T15:41:35.686 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libfl-dev:amd64 (2.6.4-6.2) ... 2022-06-29T15:41:35.797 INFO:teuthology.orchestra.run.smithi047.stdout:Processing triggers for man-db (2.9.1-1) ... 2022-06-29T15:41:35.813 INFO:teuthology.orchestra.run.smithi099.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2022-06-29T15:41:35.972 INFO:teuthology.orchestra.run.smithi099.stdout:Processing triggers for man-db (2.9.1-1) ... 2022-06-29T15:41:37.121 INFO:teuthology.orchestra.run.smithi064.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2022-06-29T15:41:37.809 INFO:teuthology.orchestra.run.smithi047.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2022-06-29T15:41:37.879 INFO:teuthology.orchestra.run.smithi064.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2022-06-29T15:41:37.882 DEBUG:teuthology.parallel:result is None 2022-06-29T15:41:38.001 INFO:teuthology.orchestra.run.smithi099.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2022-06-29T15:41:38.564 INFO:teuthology.orchestra.run.smithi047.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2022-06-29T15:41:38.566 DEBUG:teuthology.parallel:result is None 2022-06-29T15:41:38.744 INFO:teuthology.orchestra.run.smithi099.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2022-06-29T15:41:38.747 DEBUG:teuthology.parallel:result is None 2022-06-29T15:41:38.748 DEBUG:teuthology.orchestra.run.smithi047:> sudo lsb_release -is 2022-06-29T15:41:38.803 INFO:teuthology.orchestra.run.smithi047.stdout:Ubuntu 2022-06-29T15:41:38.808 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-06-29T15:41:38.810 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=f25ffac72e5a9e63860ff7deb6767a11f0559a16 2022-06-29T15:41:38.966 DEBUG:teuthology.orchestra.run.smithi047:> dpkg-query -W -f '${Version}' ceph 2022-06-29T15:41:38.990 INFO:teuthology.orchestra.run.smithi047.stdout:15.2.16-304-gf25ffac7-1focal 2022-06-29T15:41:38.991 INFO:teuthology.packaging:The installed version of ceph is 15.2.16-304-gf25ffac7-1focal 2022-06-29T15:41:38.992 INFO:teuthology.task.install:The correct ceph version 15.2.16-304-gf25ffac7-1focal is installed. 2022-06-29T15:41:38.992 DEBUG:teuthology.orchestra.run.smithi064:> sudo lsb_release -is 2022-06-29T15:41:39.046 INFO:teuthology.orchestra.run.smithi064.stdout:Ubuntu 2022-06-29T15:41:39.051 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-06-29T15:41:39.051 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=f25ffac72e5a9e63860ff7deb6767a11f0559a16 2022-06-29T15:41:39.204 DEBUG:teuthology.orchestra.run.smithi064:> dpkg-query -W -f '${Version}' ceph 2022-06-29T15:41:39.229 INFO:teuthology.orchestra.run.smithi064.stdout:15.2.16-304-gf25ffac7-1focal 2022-06-29T15:41:39.230 INFO:teuthology.packaging:The installed version of ceph is 15.2.16-304-gf25ffac7-1focal 2022-06-29T15:41:39.231 INFO:teuthology.task.install:The correct ceph version 15.2.16-304-gf25ffac7-1focal is installed. 2022-06-29T15:41:39.231 DEBUG:teuthology.orchestra.run.smithi099:> sudo lsb_release -is 2022-06-29T15:41:39.285 INFO:teuthology.orchestra.run.smithi099.stdout:Ubuntu 2022-06-29T15:41:39.290 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-06-29T15:41:39.291 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=f25ffac72e5a9e63860ff7deb6767a11f0559a16 2022-06-29T15:41:39.471 DEBUG:teuthology.orchestra.run.smithi099:> dpkg-query -W -f '${Version}' ceph 2022-06-29T15:41:39.495 INFO:teuthology.orchestra.run.smithi099.stdout:15.2.16-304-gf25ffac7-1focal 2022-06-29T15:41:39.495 INFO:teuthology.packaging:The installed version of ceph is 15.2.16-304-gf25ffac7-1focal 2022-06-29T15:41:39.496 INFO:teuthology.task.install:The correct ceph version 15.2.16-304-gf25ffac7-1focal is installed. 2022-06-29T15:41:39.496 DEBUG:teuthology.orchestra.run.smithi164:> sudo lsb_release -is 2022-06-29T15:41:39.548 INFO:teuthology.orchestra.run.smithi164.stdout:Ubuntu 2022-06-29T15:41:39.552 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-06-29T15:41:39.553 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=f25ffac72e5a9e63860ff7deb6767a11f0559a16 2022-06-29T15:41:39.706 DEBUG:teuthology.orchestra.run.smithi164:> dpkg-query -W -f '${Version}' ceph 2022-06-29T15:41:39.728 INFO:teuthology.orchestra.run.smithi164.stdout:15.2.16-304-gf25ffac7-1focal 2022-06-29T15:41:39.729 INFO:teuthology.packaging:The installed version of ceph is 15.2.16-304-gf25ffac7-1focal 2022-06-29T15:41:39.729 INFO:teuthology.task.install:The correct ceph version 15.2.16-304-gf25ffac7-1focal is installed. 2022-06-29T15:41:39.730 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2022-06-29T15:41:39.731 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2022-06-29T15:41:39.731 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2022-06-29T15:41:39.748 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2022-06-29T15:41:39.749 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2022-06-29T15:41:39.767 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2022-06-29T15:41:39.768 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2022-06-29T15:41:39.785 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2022-06-29T15:41:39.785 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2022-06-29T15:41:39.802 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2022-06-29T15:41:39.803 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2022-06-29T15:41:39.804 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/daemon-helper 2022-06-29T15:41:39.821 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/daemon-helper 2022-06-29T15:41:39.832 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2022-06-29T15:41:39.833 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/daemon-helper 2022-06-29T15:41:39.849 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/daemon-helper 2022-06-29T15:41:39.865 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2022-06-29T15:41:39.866 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/daemon-helper 2022-06-29T15:41:39.882 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/daemon-helper 2022-06-29T15:41:39.892 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2022-06-29T15:41:39.892 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/daemon-helper 2022-06-29T15:41:39.907 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/daemon-helper 2022-06-29T15:41:39.922 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2022-06-29T15:41:39.923 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2022-06-29T15:41:39.923 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/adjust-ulimits 2022-06-29T15:41:39.940 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2022-06-29T15:41:39.951 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2022-06-29T15:41:39.951 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/adjust-ulimits 2022-06-29T15:41:39.967 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2022-06-29T15:41:40.022 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2022-06-29T15:41:40.023 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/adjust-ulimits 2022-06-29T15:41:40.039 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2022-06-29T15:41:40.054 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2022-06-29T15:41:40.054 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/adjust-ulimits 2022-06-29T15:41:40.070 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2022-06-29T15:41:40.082 INFO:teuthology.run_tasks:Running task ceph... 2022-06-29T15:41:40.141 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2022-06-29T15:41:40.142 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 777 /var/log/ceph 2022-06-29T15:41:40.145 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 777 /var/log/ceph 2022-06-29T15:41:40.148 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 777 /var/log/ceph 2022-06-29T15:41:40.150 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 777 /var/log/ceph 2022-06-29T15:41:40.166 INFO:tasks.ceph:Disabling ceph logrotate... 2022-06-29T15:41:40.166 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f -- /etc/logrotate.d/ceph 2022-06-29T15:41:40.204 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /etc/logrotate.d/ceph 2022-06-29T15:41:40.206 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /etc/logrotate.d/ceph 2022-06-29T15:41:40.209 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /etc/logrotate.d/ceph 2022-06-29T15:41:40.225 INFO:tasks.ceph:Creating extra log directories... 2022-06-29T15:41:40.225 DEBUG:teuthology.orchestra.run.smithi047:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2022-06-29T15:41:40.228 DEBUG:teuthology.orchestra.run.smithi064:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2022-06-29T15:41:40.230 DEBUG:teuthology.orchestra.run.smithi099:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2022-06-29T15:41:40.268 DEBUG:teuthology.orchestra.run.smithi164:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2022-06-29T15:41:40.284 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2022-06-29T15:41:40.285 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2022-06-29T15:41:40.285 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2022-06-29T15:41:40.286 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2022-06-29T15:41:40.286 DEBUG:teuthology.orchestra.run.smithi047:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2022-06-29T15:41:40.293 DEBUG:teuthology.orchestra.run.smithi047:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2022-06-29T15:41:40.376 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2022-06-29T15:41:40.376 DEBUG:teuthology.orchestra.run.smithi064:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2022-06-29T15:41:40.384 DEBUG:teuthology.orchestra.run.smithi064:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2022-06-29T15:41:40.464 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2022-06-29T15:41:40.464 DEBUG:teuthology.orchestra.run.smithi099:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2022-06-29T15:41:40.471 DEBUG:teuthology.orchestra.run.smithi099:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2022-06-29T15:41:40.553 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2022-06-29T15:41:40.553 DEBUG:teuthology.orchestra.run.smithi164:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2022-06-29T15:41:40.560 DEBUG:teuthology.orchestra.run.smithi164:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2022-06-29T15:41:40.641 INFO:tasks.ceph:Creating ceph cluster ceph... 2022-06-29T15:41:40.642 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal frag': True, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'evicting unresponsive client', 'POOL_APP_NOT_ENABLED', 'has not responded to cap revoke by MDS for over', 'MDS_CLIENT_LATE_RELEASE', 'responding to mclientcaps'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2022-06-29T15:41:40.642 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2022-06-29_13:31:31-fs-wip-yuri5-testing-2022-06-27-0923-octopus-distro-default-smithi/6905516', 'branch': 'wip-yuri5-testing-2022-06-27-0923-octopus', 'description': 'fs/multiclient/{begin clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests}', 'email': None, 'first_in_suite': False, 'job_id': '6905516', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2022-06-29_13:31:31-fs-wip-yuri5-testing-2022-06-27-0923-octopus-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 30}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '20.04', 'overrides': {'admin_socket': {'branch': 'wip-yuri5-testing-2022-06-27-0923-octopus'}, 'ceph': {'cephfs_ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal frag': True, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'evicting unresponsive client', 'POOL_APP_NOT_ENABLED', 'has not responded to cap revoke by MDS for over', 'MDS_CLIENT_LATE_RELEASE', 'responding to mclientcaps'], 'sha1': 'f25ffac72e5a9e63860ff7deb6767a11f0559a16'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'f25ffac72e5a9e63860ff7deb6767a11f0559a16'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri5-testing-2022-06-27-0923-octopus', 'sha1': 'f25ffac72e5a9e63860ff7deb6767a11f0559a16'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 79, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7'], ['client.0'], ['client.1']], 'sha1': 'f25ffac72e5a9e63860ff7deb6767a11f0559a16', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'wip-yuri5-testing-2022-06-27-0923-octopus', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_f25ffac72e5a9e63860ff7deb6767a11f0559a16/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'f25ffac72e5a9e63860ff7deb6767a11f0559a16', 'targets': {'smithi047.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4e6XGqkwGvpDrF5tk+4DCUrEpdP3CkMgMrSSS9ef5+Qelxe2P8ud2lIOIDMsxRgrE5O5QC/wbcUiwn8+qxPo3wnqMRFoTukU4ntoB9u0OukaxiytT46IxAjcQxYqYivrRCl4hHvmed4EBBaJIkW/raEv6tYaYTy2edkofooheORk6Hm9cmMDLCh2ajAbHIY/81h9pV7Hu43cLS/MiWPZLhDSW5oCu8cPXA4gAt6Yr3f4ri90mHCrOpOjq1/GrmoOY8f88WVs+PbPBu+qLLYzysc/6ywX8VIXBoo8ZkrGXmSIjwW5Zu1JZW1Bj3Ppk/bNbZRWL8LAWcEOqqlRQZJ/i6uZRgXX3Q5V2XPDRBxReKRKIW3C44Q2MdQWjkEe96bCJefVfxu9+0FkSH/E81B6EelO8+s3Ge1FRtFkMMPxwxjSWbnXbcPSz75RMs3+0Um98kiDO3+tkkBkkL5QiosHaFYHgDY2diV5ao84ECNE5pD8mWIHiMyYFKN23PiSNvt8=', 'smithi064.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDdkS0ro2f1U277A5L7yO3qFO5OBFhLAUb8ADG3fUuoJHbGGpBlumoJOhZsDW/AK8e18jCbJrJIkBVZnMiLklYmoOXQ4XDbmPllwAVojFEXDAwzpVfAmIf2pVnEAJfTZmMXTpP8tDSGRY4X4ZM9LEOrvdBmsRP+ACrBfii9LEXpFix2NStpaSWKqcMYkCbVpxOIg5BqrtDlMf1fe9j/N9tWl22uDnuzpYKS6pwdwbxWBWWT6Qh0s5Dsd1swGRaRqgZkiEFowJAKMDVR1DHHc7VCr7b+19YPsGljki+x4Uxi4eey62Yj1OtPn2x2mY9bwkWxMZLi+lzEajyWOpj5QP7Adw/egmEtXD/uAPGL2atq05qLogwaoxlz112YVoyJ/UjQulnsKnCnrcX8wtzJq2x40MprbfyORKysYXyFhqgXQKUlnYM3lU5H9g99Af8Oyzq9vl6uKTpjWSkr79J/m3CKmI3Kg8n2CcgNFKZI91nFCfP25OfVbruX5e+txVlZnA8=', 'smithi099.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZ3lGbsmV5Dubh02wgTX/9Gf/jBDlc9lpQ/t9F8VdBmXsK915xExtQc4o8oAdSd0gN+vcaX4xveET1oQLs0zN4dW64kOMM6qiEGOzy4aisfPmQgHh2w7YJJ5i6RqMR8j7So5CHCDN2NvDLNilTxkpx1qTW6BkgtiNwH3O4UwNPxaqBKTm8VNjHzgPiMQdkBmR0sMLS9aEfi9G9wdeH5oFr7QOPtdu84fUm6idqpTF20r1Q878w78Z8iYz73GltKHx+lcB6mzoG08ZqhYJpeBs/mvSJzdLUTxoCp3w1aFuev6en61UthxLBkt5Vz8ZyO3g8ImP8jiYtGuTNzkgTPNguxzIrg6tW/PHJElnYEvFD+dM1xr5xFDHsofqH7KRILl4gJm5oEIViPpj/ocEmRakJ1hFRNF4myz83SxnsX1Bqe8BIgYgLABQnvL9zwoAkoIir3V/jnqXwiJZS1qbuY2gfDt9W8JAZd5Tj68MfHjoVrr0iv/DMAIL31ozQqU3iwDc=', 'smithi164.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCjyDdZDCWeXC2ZkZ0RXUp0FFOWbaZFutsSv/nsLHjcLzc6p2L6UAsSnJBpzL6L490K/NRMpS82MM67GIcQIFGWf9lxvQWMNpjzpRws8/C2d41stWRXtlGdYkd5ai/gqbN7Cs+YOn8A9QEW+gWvr6oxL+c3AwqTucFCjX/wjLS5Do+qFRx6r+h8I8adX7gb7CJfl0I+pkEozjrDkUavpONv+PJ9vX5oBGqMVK97/I3TbVimPLDj7Gingn9dQRdZdFZtKqjlk44Nlp2PScVZbDv/IHW9xigYq4nlxUVu47pbSkB24Ob+oj4bgAvX8FCZshWyPqf8N/4vrQiTwYW/r24PcS20iEWL4IjnLSvqR5eEAMfYzh3QIrFge2bWYOwhZnSWCFhzCaFyZsDiFMG0kXMlMC9mxVDH4eHc61iOxrul7i8wbv7f4tVBrs2GsYEnEekg+aqhCT6B8dOrlQmPpk8G0tL8Pgs3OsFZ51M+Ia3DGUs2XWjVz5PYlz3QrjNE/DU='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': 'f25ffac72e5a9e63860ff7deb6767a11f0559a16'}}, {'ceph': None}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_misc']}}], 'teuthology_branch': 'main', 'teuthology_sha1': 'f58709e3fda190343157063e5edb0d7a36a41afc', 'timestamp': '2022-06-29_13:31:31', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.1713'} 2022-06-29T15:41:40.643 DEBUG:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2022-06-29T15:41:40.646 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2022-06-29T15:41:40.649 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2022-06-29T15:41:40.652 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2022-06-29T15:41:40.658 DEBUG:teuthology.orchestra.run.smithi047:> sudo install -d -m0777 -- /var/run/ceph 2022-06-29T15:41:40.660 DEBUG:teuthology.orchestra.run.smithi064:> sudo install -d -m0777 -- /var/run/ceph 2022-06-29T15:41:40.663 DEBUG:teuthology.orchestra.run.smithi099:> sudo install -d -m0777 -- /var/run/ceph 2022-06-29T15:41:40.665 DEBUG:teuthology.orchestra.run.smithi164:> sudo install -d -m0777 -- /var/run/ceph 2022-06-29T15:41:40.681 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2022-06-29T15:41:40.681 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/scratch_devs of=/dev/stdout 2022-06-29T15:41:40.724 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2022-06-29T15:41:40.725 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2022-06-29T15:41:40.732 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2022-06-29T15:41:40.732 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-06-29T15:41:40.732 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 6h/6d Inode: 364 Links: 1 2022-06-29T15:41:40.733 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-06-29T15:41:40.733 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2022-06-29 15:38:34.939997922 +0000 2022-06-29T15:41:40.733 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2022-06-29 15:38:34.619997925 +0000 2022-06-29T15:41:40.733 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2022-06-29 15:38:34.619997925 +0000 2022-06-29T15:41:40.734 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: - 2022-06-29T15:41:40.743 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2022-06-29T15:41:40.743 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2022-06-29T15:41:40.743 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.00031389 s, 1.6 MB/s 2022-06-29T15:41:40.748 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2022-06-29T15:41:40.755 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2022-06-29T15:41:40.755 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-06-29T15:41:40.755 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 6h/6d Inode: 368 Links: 1 2022-06-29T15:41:40.756 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-06-29T15:41:40.756 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2022-06-29 15:38:35.463997916 +0000 2022-06-29T15:41:40.756 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2022-06-29 15:38:35.143997919 +0000 2022-06-29T15:41:40.756 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2022-06-29 15:38:35.143997919 +0000 2022-06-29T15:41:40.756 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: - 2022-06-29T15:41:40.765 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2022-06-29T15:41:40.766 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2022-06-29T15:41:40.766 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000290893 s, 1.8 MB/s 2022-06-29T15:41:40.770 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2022-06-29T15:41:40.776 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2022-06-29T15:41:40.777 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-06-29T15:41:40.777 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 6h/6d Inode: 376 Links: 1 2022-06-29T15:41:40.777 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-06-29T15:41:40.777 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2022-06-29 15:38:35.999997910 +0000 2022-06-29T15:41:40.778 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2022-06-29 15:38:35.679997913 +0000 2022-06-29T15:41:40.778 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2022-06-29 15:38:35.679997913 +0000 2022-06-29T15:41:40.778 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: - 2022-06-29T15:41:40.785 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2022-06-29T15:41:40.786 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2022-06-29T15:41:40.786 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.00030128 s, 1.7 MB/s 2022-06-29T15:41:40.790 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2022-06-29T15:41:40.797 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2022-06-29T15:41:40.798 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-06-29T15:41:40.798 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 6h/6d Inode: 379 Links: 1 2022-06-29T15:41:40.798 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-06-29T15:41:40.799 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2022-06-29 15:38:36.207997908 +0000 2022-06-29T15:41:40.799 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2022-06-29 15:38:36.203997908 +0000 2022-06-29T15:41:40.799 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2022-06-29 15:38:36.203997908 +0000 2022-06-29T15:41:40.799 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: - 2022-06-29T15:41:40.806 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2022-06-29T15:41:40.806 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2022-06-29T15:41:40.807 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000265543 s, 1.9 MB/s 2022-06-29T15:41:40.811 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-06-29T15:41:40.812 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2022-06-29T15:41:40.812 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/scratch_devs of=/dev/stdout 2022-06-29T15:41:40.819 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2022-06-29T15:41:40.820 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2022-06-29T15:41:40.827 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2022-06-29T15:41:40.828 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-06-29T15:41:40.828 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 6h/6d Inode: 364 Links: 1 2022-06-29T15:41:40.828 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-06-29T15:41:40.828 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2022-06-29 15:38:38.764005776 +0000 2022-06-29T15:41:40.829 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2022-06-29 15:38:38.760005776 +0000 2022-06-29T15:41:40.829 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2022-06-29 15:38:38.760005776 +0000 2022-06-29T15:41:40.829 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: - 2022-06-29T15:41:40.838 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2022-06-29T15:41:40.839 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2022-06-29T15:41:40.839 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.00041649 s, 1.2 MB/s 2022-06-29T15:41:40.843 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2022-06-29T15:41:40.849 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2022-06-29T15:41:40.850 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-06-29T15:41:40.850 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 6h/6d Inode: 369 Links: 1 2022-06-29T15:41:40.850 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-06-29T15:41:40.850 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2022-06-29 15:38:39.644005804 +0000 2022-06-29T15:41:40.851 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2022-06-29 15:38:39.324005794 +0000 2022-06-29T15:41:40.851 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2022-06-29 15:38:39.324005794 +0000 2022-06-29T15:41:40.851 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: - 2022-06-29T15:41:40.859 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2022-06-29T15:41:40.859 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2022-06-29T15:41:40.860 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000273804 s, 1.9 MB/s 2022-06-29T15:41:40.864 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2022-06-29T15:41:40.911 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2022-06-29T15:41:40.911 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-06-29T15:41:40.912 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 6h/6d Inode: 377 Links: 1 2022-06-29T15:41:40.912 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-06-29T15:41:40.913 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2022-06-29 15:38:40.180005822 +0000 2022-06-29T15:41:40.913 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2022-06-29 15:38:39.856005811 +0000 2022-06-29T15:41:40.914 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2022-06-29 15:38:39.856005811 +0000 2022-06-29T15:41:40.914 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: - 2022-06-29T15:41:40.921 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2022-06-29T15:41:40.922 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2022-06-29T15:41:40.922 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000397861 s, 1.3 MB/s 2022-06-29T15:41:40.926 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2022-06-29T15:41:40.932 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2022-06-29T15:41:40.933 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-06-29T15:41:40.933 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 6h/6d Inode: 381 Links: 1 2022-06-29T15:41:40.933 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-06-29T15:41:40.934 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2022-06-29 15:38:40.740005840 +0000 2022-06-29T15:41:40.934 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2022-06-29 15:38:40.404005829 +0000 2022-06-29T15:41:40.934 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2022-06-29 15:38:40.404005829 +0000 2022-06-29T15:41:40.935 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: - 2022-06-29T15:41:40.943 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2022-06-29T15:41:40.943 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2022-06-29T15:41:40.943 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000334875 s, 1.5 MB/s 2022-06-29T15:41:40.947 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-06-29T15:41:40.948 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi047.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi064.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2022-06-29T15:41:40.948 INFO:tasks.ceph:Generating config... 2022-06-29T15:41:40.950 INFO:tasks.ceph:[client] client mount timeout = 600 2022-06-29T15:41:40.950 INFO:tasks.ceph:[client] debug client = 20 2022-06-29T15:41:40.951 INFO:tasks.ceph:[client] debug ms = 1 2022-06-29T15:41:40.951 INFO:tasks.ceph:[client] rados mon op timeout = 15m 2022-06-29T15:41:40.952 INFO:tasks.ceph:[client] rados osd op timeout = 15m 2022-06-29T15:41:40.952 INFO:tasks.ceph:[mds] debug mds = 20 2022-06-29T15:41:40.952 INFO:tasks.ceph:[mds] debug ms = 1 2022-06-29T15:41:40.953 INFO:tasks.ceph:[mds] mds bal frag = True 2022-06-29T15:41:40.953 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2022-06-29T15:41:40.954 INFO:tasks.ceph:[mds] mds bal merge size = 5 2022-06-29T15:41:40.954 INFO:tasks.ceph:[mds] mds bal split bits = 3 2022-06-29T15:41:40.955 INFO:tasks.ceph:[mds] mds bal split size = 100 2022-06-29T15:41:40.956 INFO:tasks.ceph:[mds] mds debug frag = True 2022-06-29T15:41:40.956 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2022-06-29T15:41:40.957 INFO:tasks.ceph:[mds] mds op complaint time = 180 2022-06-29T15:41:40.957 INFO:tasks.ceph:[mds] mds verify scatter = True 2022-06-29T15:41:40.958 INFO:tasks.ceph:[mds] osd op complaint time = 180 2022-06-29T15:41:40.958 INFO:tasks.ceph:[mds] rados mon op timeout = 15m 2022-06-29T15:41:40.959 INFO:tasks.ceph:[mds] rados osd op timeout = 15m 2022-06-29T15:41:40.960 INFO:tasks.ceph:[mgr] debug mgr = 20 2022-06-29T15:41:40.960 INFO:tasks.ceph:[mgr] debug ms = 1 2022-06-29T15:41:40.961 INFO:tasks.ceph:[mon] debug mon = 20 2022-06-29T15:41:40.961 INFO:tasks.ceph:[mon] debug ms = 1 2022-06-29T15:41:40.962 INFO:tasks.ceph:[mon] debug paxos = 20 2022-06-29T15:41:40.962 INFO:tasks.ceph:[mon] mon op complaint time = 120 2022-06-29T15:41:40.963 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2022-06-29T15:41:40.963 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2022-06-29T15:41:40.964 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2022-06-29T15:41:40.965 INFO:tasks.ceph:[osd] debug bluefs = 20 2022-06-29T15:41:40.965 INFO:tasks.ceph:[osd] debug bluestore = 20 2022-06-29T15:41:40.966 INFO:tasks.ceph:[osd] debug ms = 1 2022-06-29T15:41:40.967 INFO:tasks.ceph:[osd] debug osd = 20 2022-06-29T15:41:40.967 INFO:tasks.ceph:[osd] debug rocksdb = 10 2022-06-29T15:41:40.968 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2022-06-29T15:41:40.968 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2022-06-29T15:41:40.969 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2022-06-29T15:41:40.970 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2022-06-29T15:41:40.970 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2022-06-29T15:41:40.971 INFO:tasks.ceph:[osd] osd op complaint time = 180 2022-06-29T15:41:40.972 INFO:tasks.ceph:Setting up mon.a... 2022-06-29T15:41:40.973 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2022-06-29T15:41:41.016 INFO:teuthology.orchestra.run.smithi047.stdout:creating /etc/ceph/ceph.keyring 2022-06-29T15:41:41.019 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2022-06-29T15:41:41.051 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-06-29T15:41:41.069 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.47'), ('mon.b', '172.21.15.64'), ('mon.c', '[v2:172.21.15.64:3301,v1:172.21.15.64:6790]')] 2022-06-29T15:41:41.070 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool --create --clobber --enable-all-features --add a 172.21.15.47 --add b 172.21.15.64 --addv c '[v2:172.21.15.64:3301,v1:172.21.15.64:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2022-06-29T15:41:41.103 INFO:teuthology.orchestra.run.smithi047.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2022-06-29T15:41:41.104 INFO:teuthology.orchestra.run.smithi047.stdout:monmaptool: generated fsid 9a68894a-53ce-400e-838b-14eab91b834c 2022-06-29T15:41:41.104 INFO:teuthology.orchestra.run.smithi047.stdout:epoch 0 2022-06-29T15:41:41.104 INFO:teuthology.orchestra.run.smithi047.stdout:fsid 9a68894a-53ce-400e-838b-14eab91b834c 2022-06-29T15:41:41.104 INFO:teuthology.orchestra.run.smithi047.stdout:last_changed 2022-06-29T15:41:41.101515+0000 2022-06-29T15:41:41.105 INFO:teuthology.orchestra.run.smithi047.stdout:created 2022-06-29T15:41:41.101515+0000 2022-06-29T15:41:41.105 INFO:teuthology.orchestra.run.smithi047.stdout:min_mon_release 0 (unknown) 2022-06-29T15:41:41.105 INFO:teuthology.orchestra.run.smithi047.stdout:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2022-06-29T15:41:41.105 INFO:teuthology.orchestra.run.smithi047.stdout:1: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.b 2022-06-29T15:41:41.106 INFO:teuthology.orchestra.run.smithi047.stdout:2: [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] mon.c 2022-06-29T15:41:41.106 INFO:teuthology.orchestra.run.smithi047.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2022-06-29T15:41:41.107 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 9a68894a-53ce-400e-838b-14eab91b834c... 2022-06-29T15:41:41.109 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-06-29T15:41:41.152 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-06-29T15:41:41.156 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-06-29T15:41:41.158 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-06-29T15:41:41.183 INFO:teuthology.orchestra.run.smithi047.stdout:[global] 2022-06-29T15:41:41.183 INFO:teuthology.orchestra.run.smithi047.stdout: chdir = "" 2022-06-29T15:41:41.183 INFO:teuthology.orchestra.run.smithi047.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-06-29T15:41:41.184 INFO:teuthology.orchestra.run.smithi047.stdout: auth supported = cephx 2022-06-29T15:41:41.184 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:41:41.184 INFO:teuthology.orchestra.run.smithi047.stdout: filestore xattr use omap = true 2022-06-29T15:41:41.185 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:41:41.185 INFO:teuthology.orchestra.run.smithi047.stdout: mon clock drift allowed = 1.000 2022-06-29T15:41:41.185 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:41:41.185 INFO:teuthology.orchestra.run.smithi047.stdout: osd crush chooseleaf type = 0 2022-06-29T15:41:41.186 INFO:teuthology.orchestra.run.smithi047.stdout: auth debug = true 2022-06-29T15:41:41.186 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:41:41.186 INFO:teuthology.orchestra.run.smithi047.stdout: ms die on old message = true 2022-06-29T15:41:41.186 INFO:teuthology.orchestra.run.smithi047.stdout: ms die on bug = true 2022-06-29T15:41:41.187 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:41:41.187 INFO:teuthology.orchestra.run.smithi047.stdout: mon max pg per osd = 10000 # >= luminous 2022-06-29T15:41:41.187 INFO:teuthology.orchestra.run.smithi047.stdout: mon pg warn max object skew = 0 2022-06-29T15:41:41.187 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:41:41.188 INFO:teuthology.orchestra.run.smithi047.stdout: # disable pg_autoscaler by default for new pools 2022-06-29T15:41:41.188 INFO:teuthology.orchestra.run.smithi047.stdout: osd_pool_default_pg_autoscale_mode = off 2022-06-29T15:41:41.188 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:41:41.188 INFO:teuthology.orchestra.run.smithi047.stdout: osd pool default size = 2 2022-06-29T15:41:41.189 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:41:41.189 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd allow primary affinity = true 2022-06-29T15:41:41.189 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd allow pg remap = true 2022-06-29T15:41:41.189 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on legacy crush tunables = false 2022-06-29T15:41:41.189 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on crush straw calc version zero = false 2022-06-29T15:41:41.190 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on no sortbitwise = false 2022-06-29T15:41:41.190 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on osd down out interval zero = false 2022-06-29T15:41:41.190 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on too few osds = false 2022-06-29T15:41:41.191 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-06-29T15:41:41.191 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_pool_no_redundancy = false 2022-06-29T15:41:41.191 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:41:41.191 INFO:teuthology.orchestra.run.smithi047.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-06-29T15:41:41.192 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:41:41.192 INFO:teuthology.orchestra.run.smithi047.stdout: osd default data pool replay window = 5 2022-06-29T15:41:41.192 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:41:41.192 INFO:teuthology.orchestra.run.smithi047.stdout: mon allow pool delete = true 2022-06-29T15:41:41.193 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:41:41.194 INFO:teuthology.orchestra.run.smithi047.stdout: mon cluster log file level = debug 2022-06-29T15:41:41.194 INFO:teuthology.orchestra.run.smithi047.stdout: debug asserts on shutdown = true 2022-06-29T15:41:41.194 INFO:teuthology.orchestra.run.smithi047.stdout: mon health detail to clog = false 2022-06-29T15:41:41.194 INFO:teuthology.orchestra.run.smithi047.stdout: mon host = "172.21.15.47,172.21.15.64,[v2:172.21.15.64:3301,v1:172.21.15.64:6790]" 2022-06-29T15:41:41.195 INFO:teuthology.orchestra.run.smithi047.stdout: fsid = 9a68894a-53ce-400e-838b-14eab91b834c 2022-06-29T15:41:41.195 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:41:41.195 INFO:teuthology.orchestra.run.smithi047.stdout:[osd] 2022-06-29T15:41:41.195 INFO:teuthology.orchestra.run.smithi047.stdout: osd journal size = 100 2022-06-29T15:41:41.196 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:41:41.196 INFO:teuthology.orchestra.run.smithi047.stdout: osd scrub load threshold = 5.0 2022-06-29T15:41:41.196 INFO:teuthology.orchestra.run.smithi047.stdout: osd scrub max interval = 600 2022-06-29T15:41:41.196 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:41:41.196 INFO:teuthology.orchestra.run.smithi047.stdout: osd recover clone overlap = true 2022-06-29T15:41:41.197 INFO:teuthology.orchestra.run.smithi047.stdout: osd recovery max chunk = 1048576 2022-06-29T15:41:41.197 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:41:41.197 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug shutdown = true 2022-06-29T15:41:41.197 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug op order = true 2022-06-29T15:41:41.197 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug verify stray on activate = true 2022-06-29T15:41:41.198 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:41:41.198 INFO:teuthology.orchestra.run.smithi047.stdout: osd open classes on start = true 2022-06-29T15:41:41.198 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug pg log writeout = true 2022-06-29T15:41:41.198 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:41:41.199 INFO:teuthology.orchestra.run.smithi047.stdout: osd deep scrub update digest min age = 30 2022-06-29T15:41:41.199 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:41:41.199 INFO:teuthology.orchestra.run.smithi047.stdout: osd map max advance = 10 2022-06-29T15:41:41.200 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:41:41.200 INFO:teuthology.orchestra.run.smithi047.stdout: journal zero on create = true 2022-06-29T15:41:41.200 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:41:41.200 INFO:teuthology.orchestra.run.smithi047.stdout: filestore ondisk finisher threads = 3 2022-06-29T15:41:41.201 INFO:teuthology.orchestra.run.smithi047.stdout: filestore apply finisher threads = 3 2022-06-29T15:41:41.201 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:41:41.201 INFO:teuthology.orchestra.run.smithi047.stdout: bdev debug aio = true 2022-06-29T15:41:41.201 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug misdirected ops = true 2022-06-29T15:41:41.201 INFO:teuthology.orchestra.run.smithi047.stdout: bluestore block size = 96636764160 2022-06-29T15:41:41.202 INFO:teuthology.orchestra.run.smithi047.stdout: bluestore compression mode = aggressive 2022-06-29T15:41:41.202 INFO:teuthology.orchestra.run.smithi047.stdout: bluestore fsck on mount = True 2022-06-29T15:41:41.202 INFO:teuthology.orchestra.run.smithi047.stdout: debug bluefs = 20 2022-06-29T15:41:41.202 INFO:teuthology.orchestra.run.smithi047.stdout: debug bluestore = 20 2022-06-29T15:41:41.203 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2022-06-29T15:41:41.203 INFO:teuthology.orchestra.run.smithi047.stdout: debug osd = 20 2022-06-29T15:41:41.203 INFO:teuthology.orchestra.run.smithi047.stdout: debug rocksdb = 10 2022-06-29T15:41:41.203 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd backfillfull_ratio = 0.85 2022-06-29T15:41:41.203 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd full ratio = 0.9 2022-06-29T15:41:41.204 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd nearfull ratio = 0.8 2022-06-29T15:41:41.204 INFO:teuthology.orchestra.run.smithi047.stdout: osd failsafe full ratio = 0.95 2022-06-29T15:41:41.204 INFO:teuthology.orchestra.run.smithi047.stdout: osd objectstore = bluestore 2022-06-29T15:41:41.204 INFO:teuthology.orchestra.run.smithi047.stdout: osd op complaint time = 180 2022-06-29T15:41:41.205 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:41:41.205 INFO:teuthology.orchestra.run.smithi047.stdout:[mgr] 2022-06-29T15:41:41.205 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2022-06-29T15:41:41.205 INFO:teuthology.orchestra.run.smithi047.stdout: debug mgr = 20 2022-06-29T15:41:41.206 INFO:teuthology.orchestra.run.smithi047.stdout: debug mon = 20 2022-06-29T15:41:41.206 INFO:teuthology.orchestra.run.smithi047.stdout: debug auth = 20 2022-06-29T15:41:41.206 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min pgs per osd = 4 2022-06-29T15:41:41.206 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min bytes per osd = 10 2022-06-29T15:41:41.206 INFO:teuthology.orchestra.run.smithi047.stdout: mgr/telemetry/nag = false 2022-06-29T15:41:41.207 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:41:41.207 INFO:teuthology.orchestra.run.smithi047.stdout:[mon] 2022-06-29T15:41:41.207 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2022-06-29T15:41:41.207 INFO:teuthology.orchestra.run.smithi047.stdout: debug mon = 20 2022-06-29T15:41:41.207 INFO:teuthology.orchestra.run.smithi047.stdout: debug paxos = 20 2022-06-29T15:41:41.208 INFO:teuthology.orchestra.run.smithi047.stdout: debug auth = 20 2022-06-29T15:41:41.208 INFO:teuthology.orchestra.run.smithi047.stdout: mon data avail warn = 5 2022-06-29T15:41:41.208 INFO:teuthology.orchestra.run.smithi047.stdout: mon mgr mkfs grace = 240 2022-06-29T15:41:41.208 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min pgs per osd = 4 2022-06-29T15:41:41.209 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd reporter subtree level = osd 2022-06-29T15:41:41.209 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd prime pg temp = true 2022-06-29T15:41:41.209 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min bytes per osd = 10 2022-06-29T15:41:41.209 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:41:41.209 INFO:teuthology.orchestra.run.smithi047.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-06-29T15:41:41.210 INFO:teuthology.orchestra.run.smithi047.stdout: auth mon ticket ttl = 660 # 11m 2022-06-29T15:41:41.210 INFO:teuthology.orchestra.run.smithi047.stdout: auth service ticket ttl = 240 # 4m 2022-06-29T15:41:41.210 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:41:41.211 INFO:teuthology.orchestra.run.smithi047.stdout: # don't complain about insecure global_id in the test suite 2022-06-29T15:41:41.211 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-06-29T15:41:41.211 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-06-29T15:41:41.211 INFO:teuthology.orchestra.run.smithi047.stdout: mon op complaint time = 120 2022-06-29T15:41:41.211 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:41:41.212 INFO:teuthology.orchestra.run.smithi047.stdout:[client] 2022-06-29T15:41:41.212 INFO:teuthology.orchestra.run.smithi047.stdout: rgw cache enabled = true 2022-06-29T15:41:41.212 INFO:teuthology.orchestra.run.smithi047.stdout: rgw enable ops log = true 2022-06-29T15:41:41.212 INFO:teuthology.orchestra.run.smithi047.stdout: rgw enable usage log = true 2022-06-29T15:41:41.213 INFO:teuthology.orchestra.run.smithi047.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-06-29T15:41:41.213 INFO:teuthology.orchestra.run.smithi047.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-06-29T15:41:41.213 INFO:teuthology.orchestra.run.smithi047.stdout: client mount timeout = 600 2022-06-29T15:41:41.213 INFO:teuthology.orchestra.run.smithi047.stdout: debug client = 20 2022-06-29T15:41:41.213 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2022-06-29T15:41:41.214 INFO:teuthology.orchestra.run.smithi047.stdout: rados mon op timeout = 15m 2022-06-29T15:41:41.214 INFO:teuthology.orchestra.run.smithi047.stdout: rados osd op timeout = 15m 2022-06-29T15:41:41.214 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.a] 2022-06-29T15:41:41.215 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.b] 2022-06-29T15:41:41.215 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.c] 2022-06-29T15:41:41.215 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.a] 2022-06-29T15:41:41.215 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.c] 2022-06-29T15:41:41.215 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.b] 2022-06-29T15:41:41.216 INFO:teuthology.orchestra.run.smithi047.stdout:[mds] 2022-06-29T15:41:41.216 INFO:teuthology.orchestra.run.smithi047.stdout: debug mds = 20 2022-06-29T15:41:41.216 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2022-06-29T15:41:41.216 INFO:teuthology.orchestra.run.smithi047.stdout: mds bal frag = True 2022-06-29T15:41:41.217 INFO:teuthology.orchestra.run.smithi047.stdout: mds bal fragment size max = 10000 2022-06-29T15:41:41.217 INFO:teuthology.orchestra.run.smithi047.stdout: mds bal merge size = 5 2022-06-29T15:41:41.217 INFO:teuthology.orchestra.run.smithi047.stdout: mds bal split bits = 3 2022-06-29T15:41:41.217 INFO:teuthology.orchestra.run.smithi047.stdout: mds bal split size = 100 2022-06-29T15:41:41.217 INFO:teuthology.orchestra.run.smithi047.stdout: mds debug frag = True 2022-06-29T15:41:41.218 INFO:teuthology.orchestra.run.smithi047.stdout: mds debug scatterstat = True 2022-06-29T15:41:41.218 INFO:teuthology.orchestra.run.smithi047.stdout: mds op complaint time = 180 2022-06-29T15:41:41.218 INFO:teuthology.orchestra.run.smithi047.stdout: mds verify scatter = True 2022-06-29T15:41:41.218 INFO:teuthology.orchestra.run.smithi047.stdout: osd op complaint time = 180 2022-06-29T15:41:41.218 INFO:teuthology.orchestra.run.smithi047.stdout: rados mon op timeout = 15m 2022-06-29T15:41:41.219 INFO:teuthology.orchestra.run.smithi047.stdout: rados osd op timeout = 15m 2022-06-29T15:41:41.222 INFO:teuthology.orchestra.run.smithi099.stdout:[global] 2022-06-29T15:41:41.222 INFO:teuthology.orchestra.run.smithi099.stdout: chdir = "" 2022-06-29T15:41:41.223 INFO:teuthology.orchestra.run.smithi099.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-06-29T15:41:41.223 INFO:teuthology.orchestra.run.smithi099.stdout: auth supported = cephx 2022-06-29T15:41:41.223 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:41:41.223 INFO:teuthology.orchestra.run.smithi099.stdout: filestore xattr use omap = true 2022-06-29T15:41:41.224 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:41:41.224 INFO:teuthology.orchestra.run.smithi099.stdout: mon clock drift allowed = 1.000 2022-06-29T15:41:41.224 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:41:41.224 INFO:teuthology.orchestra.run.smithi099.stdout: osd crush chooseleaf type = 0 2022-06-29T15:41:41.224 INFO:teuthology.orchestra.run.smithi099.stdout: auth debug = true 2022-06-29T15:41:41.225 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:41:41.225 INFO:teuthology.orchestra.run.smithi099.stdout: ms die on old message = true 2022-06-29T15:41:41.225 INFO:teuthology.orchestra.run.smithi099.stdout: ms die on bug = true 2022-06-29T15:41:41.225 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:41:41.226 INFO:teuthology.orchestra.run.smithi099.stdout: mon max pg per osd = 10000 # >= luminous 2022-06-29T15:41:41.226 INFO:teuthology.orchestra.run.smithi099.stdout: mon pg warn max object skew = 0 2022-06-29T15:41:41.226 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:41:41.226 INFO:teuthology.orchestra.run.smithi099.stdout: # disable pg_autoscaler by default for new pools 2022-06-29T15:41:41.227 INFO:teuthology.orchestra.run.smithi099.stdout: osd_pool_default_pg_autoscale_mode = off 2022-06-29T15:41:41.227 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:41:41.227 INFO:teuthology.orchestra.run.smithi099.stdout: osd pool default size = 2 2022-06-29T15:41:41.227 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:41:41.227 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd allow primary affinity = true 2022-06-29T15:41:41.228 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd allow pg remap = true 2022-06-29T15:41:41.228 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on legacy crush tunables = false 2022-06-29T15:41:41.228 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on crush straw calc version zero = false 2022-06-29T15:41:41.228 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on no sortbitwise = false 2022-06-29T15:41:41.228 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on osd down out interval zero = false 2022-06-29T15:41:41.229 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on too few osds = false 2022-06-29T15:41:41.229 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-06-29T15:41:41.229 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_pool_no_redundancy = false 2022-06-29T15:41:41.229 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:41:41.230 INFO:teuthology.orchestra.run.smithi099.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-06-29T15:41:41.230 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:41:41.230 INFO:teuthology.orchestra.run.smithi099.stdout: osd default data pool replay window = 5 2022-06-29T15:41:41.230 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:41:41.231 INFO:teuthology.orchestra.run.smithi099.stdout: mon allow pool delete = true 2022-06-29T15:41:41.231 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:41:41.231 INFO:teuthology.orchestra.run.smithi099.stdout: mon cluster log file level = debug 2022-06-29T15:41:41.231 INFO:teuthology.orchestra.run.smithi099.stdout: debug asserts on shutdown = true 2022-06-29T15:41:41.232 INFO:teuthology.orchestra.run.smithi099.stdout: mon health detail to clog = false 2022-06-29T15:41:41.232 INFO:teuthology.orchestra.run.smithi099.stdout: mon host = "172.21.15.47,172.21.15.64,[v2:172.21.15.64:3301,v1:172.21.15.64:6790]" 2022-06-29T15:41:41.232 INFO:teuthology.orchestra.run.smithi099.stdout: fsid = 9a68894a-53ce-400e-838b-14eab91b834c 2022-06-29T15:41:41.232 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:41:41.232 INFO:teuthology.orchestra.run.smithi099.stdout:[osd] 2022-06-29T15:41:41.233 INFO:teuthology.orchestra.run.smithi099.stdout: osd journal size = 100 2022-06-29T15:41:41.233 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:41:41.233 INFO:teuthology.orchestra.run.smithi099.stdout: osd scrub load threshold = 5.0 2022-06-29T15:41:41.233 INFO:teuthology.orchestra.run.smithi099.stdout: osd scrub max interval = 600 2022-06-29T15:41:41.234 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:41:41.234 INFO:teuthology.orchestra.run.smithi099.stdout: osd recover clone overlap = true 2022-06-29T15:41:41.234 INFO:teuthology.orchestra.run.smithi099.stdout: osd recovery max chunk = 1048576 2022-06-29T15:41:41.234 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:41:41.234 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug shutdown = true 2022-06-29T15:41:41.235 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug op order = true 2022-06-29T15:41:41.235 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug verify stray on activate = true 2022-06-29T15:41:41.235 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:41:41.235 INFO:teuthology.orchestra.run.smithi099.stdout: osd open classes on start = true 2022-06-29T15:41:41.236 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug pg log writeout = true 2022-06-29T15:41:41.236 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:41:41.236 INFO:teuthology.orchestra.run.smithi099.stdout: osd deep scrub update digest min age = 30 2022-06-29T15:41:41.236 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:41:41.236 INFO:teuthology.orchestra.run.smithi099.stdout: osd map max advance = 10 2022-06-29T15:41:41.237 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:41:41.237 INFO:teuthology.orchestra.run.smithi099.stdout: journal zero on create = true 2022-06-29T15:41:41.237 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:41:41.237 INFO:teuthology.orchestra.run.smithi099.stdout: filestore ondisk finisher threads = 3 2022-06-29T15:41:41.237 INFO:teuthology.orchestra.run.smithi099.stdout: filestore apply finisher threads = 3 2022-06-29T15:41:41.238 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:41:41.238 INFO:teuthology.orchestra.run.smithi099.stdout: bdev debug aio = true 2022-06-29T15:41:41.238 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug misdirected ops = true 2022-06-29T15:41:41.238 INFO:teuthology.orchestra.run.smithi099.stdout: bluestore block size = 96636764160 2022-06-29T15:41:41.239 INFO:teuthology.orchestra.run.smithi099.stdout: bluestore compression mode = aggressive 2022-06-29T15:41:41.239 INFO:teuthology.orchestra.run.smithi099.stdout: bluestore fsck on mount = True 2022-06-29T15:41:41.239 INFO:teuthology.orchestra.run.smithi099.stdout: debug bluefs = 20 2022-06-29T15:41:41.239 INFO:teuthology.orchestra.run.smithi099.stdout: debug bluestore = 20 2022-06-29T15:41:41.239 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2022-06-29T15:41:41.240 INFO:teuthology.orchestra.run.smithi099.stdout: debug osd = 20 2022-06-29T15:41:41.240 INFO:teuthology.orchestra.run.smithi099.stdout: debug rocksdb = 10 2022-06-29T15:41:41.240 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd backfillfull_ratio = 0.85 2022-06-29T15:41:41.240 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd full ratio = 0.9 2022-06-29T15:41:41.240 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd nearfull ratio = 0.8 2022-06-29T15:41:41.241 INFO:teuthology.orchestra.run.smithi099.stdout: osd failsafe full ratio = 0.95 2022-06-29T15:41:41.241 INFO:teuthology.orchestra.run.smithi099.stdout: osd objectstore = bluestore 2022-06-29T15:41:41.241 INFO:teuthology.orchestra.run.smithi099.stdout: osd op complaint time = 180 2022-06-29T15:41:41.242 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:41:41.242 INFO:teuthology.orchestra.run.smithi099.stdout:[mgr] 2022-06-29T15:41:41.242 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2022-06-29T15:41:41.242 INFO:teuthology.orchestra.run.smithi099.stdout: debug mgr = 20 2022-06-29T15:41:41.243 INFO:teuthology.orchestra.run.smithi099.stdout: debug mon = 20 2022-06-29T15:41:41.243 INFO:teuthology.orchestra.run.smithi099.stdout: debug auth = 20 2022-06-29T15:41:41.243 INFO:teuthology.orchestra.run.smithi099.stdout: mon reweight min pgs per osd = 4 2022-06-29T15:41:41.243 INFO:teuthology.orchestra.run.smithi099.stdout: mon reweight min bytes per osd = 10 2022-06-29T15:41:41.243 INFO:teuthology.orchestra.run.smithi099.stdout: mgr/telemetry/nag = false 2022-06-29T15:41:41.244 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:41:41.244 INFO:teuthology.orchestra.run.smithi099.stdout:[mon] 2022-06-29T15:41:41.244 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2022-06-29T15:41:41.244 INFO:teuthology.orchestra.run.smithi099.stdout: debug mon = 20 2022-06-29T15:41:41.245 INFO:teuthology.orchestra.run.smithi099.stdout: debug paxos = 20 2022-06-29T15:41:41.245 INFO:teuthology.orchestra.run.smithi099.stdout: debug auth = 20 2022-06-29T15:41:41.245 INFO:teuthology.orchestra.run.smithi099.stdout: mon data avail warn = 5 2022-06-29T15:41:41.245 INFO:teuthology.orchestra.run.smithi099.stdout: mon mgr mkfs grace = 240 2022-06-29T15:41:41.245 INFO:teuthology.orchestra.run.smithi099.stdout: mon reweight min pgs per osd = 4 2022-06-29T15:41:41.246 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd reporter subtree level = osd 2022-06-29T15:41:41.246 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd prime pg temp = true 2022-06-29T15:41:41.246 INFO:teuthology.orchestra.run.smithi099.stdout: mon reweight min bytes per osd = 10 2022-06-29T15:41:41.246 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:41:41.246 INFO:teuthology.orchestra.run.smithi099.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-06-29T15:41:41.247 INFO:teuthology.orchestra.run.smithi099.stdout: auth mon ticket ttl = 660 # 11m 2022-06-29T15:41:41.247 INFO:teuthology.orchestra.run.smithi099.stdout: auth service ticket ttl = 240 # 4m 2022-06-29T15:41:41.247 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:41:41.247 INFO:teuthology.orchestra.run.smithi099.stdout: # don't complain about insecure global_id in the test suite 2022-06-29T15:41:41.248 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-06-29T15:41:41.248 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-06-29T15:41:41.248 INFO:teuthology.orchestra.run.smithi099.stdout: mon op complaint time = 120 2022-06-29T15:41:41.248 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:41:41.248 INFO:teuthology.orchestra.run.smithi099.stdout:[client] 2022-06-29T15:41:41.249 INFO:teuthology.orchestra.run.smithi099.stdout: rgw cache enabled = true 2022-06-29T15:41:41.249 INFO:teuthology.orchestra.run.smithi099.stdout: rgw enable ops log = true 2022-06-29T15:41:41.249 INFO:teuthology.orchestra.run.smithi099.stdout: rgw enable usage log = true 2022-06-29T15:41:41.249 INFO:teuthology.orchestra.run.smithi099.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-06-29T15:41:41.250 INFO:teuthology.orchestra.run.smithi099.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-06-29T15:41:41.250 INFO:teuthology.orchestra.run.smithi099.stdout: client mount timeout = 600 2022-06-29T15:41:41.250 INFO:teuthology.orchestra.run.smithi099.stdout: debug client = 20 2022-06-29T15:41:41.250 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2022-06-29T15:41:41.250 INFO:teuthology.orchestra.run.smithi099.stdout: rados mon op timeout = 15m 2022-06-29T15:41:41.251 INFO:teuthology.orchestra.run.smithi099.stdout: rados osd op timeout = 15m 2022-06-29T15:41:41.251 INFO:teuthology.orchestra.run.smithi099.stdout:[mon.a] 2022-06-29T15:41:41.252 INFO:teuthology.orchestra.run.smithi099.stdout:[mon.b] 2022-06-29T15:41:41.252 INFO:teuthology.orchestra.run.smithi099.stdout:[mon.c] 2022-06-29T15:41:41.252 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.a] 2022-06-29T15:41:41.252 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.c] 2022-06-29T15:41:41.252 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.b] 2022-06-29T15:41:41.253 INFO:teuthology.orchestra.run.smithi099.stdout:[mds] 2022-06-29T15:41:41.253 INFO:teuthology.orchestra.run.smithi099.stdout: debug mds = 20 2022-06-29T15:41:41.253 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2022-06-29T15:41:41.253 INFO:teuthology.orchestra.run.smithi099.stdout: mds bal frag = True 2022-06-29T15:41:41.253 INFO:teuthology.orchestra.run.smithi099.stdout: mds bal fragment size max = 10000 2022-06-29T15:41:41.254 INFO:teuthology.orchestra.run.smithi099.stdout: mds bal merge size = 5 2022-06-29T15:41:41.254 INFO:teuthology.orchestra.run.smithi099.stdout: mds bal split bits = 3 2022-06-29T15:41:41.254 INFO:teuthology.orchestra.run.smithi099.stdout: mds bal split size = 100 2022-06-29T15:41:41.254 INFO:teuthology.orchestra.run.smithi099.stdout: mds debug frag = True 2022-06-29T15:41:41.255 INFO:teuthology.orchestra.run.smithi099.stdout: mds debug scatterstat = True 2022-06-29T15:41:41.255 INFO:teuthology.orchestra.run.smithi099.stdout: mds op complaint time = 180 2022-06-29T15:41:41.255 INFO:teuthology.orchestra.run.smithi099.stdout: mds verify scatter = True 2022-06-29T15:41:41.255 INFO:teuthology.orchestra.run.smithi099.stdout: osd op complaint time = 180 2022-06-29T15:41:41.255 INFO:teuthology.orchestra.run.smithi099.stdout: rados mon op timeout = 15m 2022-06-29T15:41:41.256 INFO:teuthology.orchestra.run.smithi099.stdout: rados osd op timeout = 15m 2022-06-29T15:41:41.256 INFO:teuthology.orchestra.run.smithi164.stdout:[global] 2022-06-29T15:41:41.256 INFO:teuthology.orchestra.run.smithi164.stdout: chdir = "" 2022-06-29T15:41:41.256 INFO:teuthology.orchestra.run.smithi164.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-06-29T15:41:41.257 INFO:teuthology.orchestra.run.smithi164.stdout: auth supported = cephx 2022-06-29T15:41:41.257 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:41:41.257 INFO:teuthology.orchestra.run.smithi164.stdout: filestore xattr use omap = true 2022-06-29T15:41:41.257 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:41:41.257 INFO:teuthology.orchestra.run.smithi164.stdout: mon clock drift allowed = 1.000 2022-06-29T15:41:41.258 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:41:41.258 INFO:teuthology.orchestra.run.smithi164.stdout: osd crush chooseleaf type = 0 2022-06-29T15:41:41.258 INFO:teuthology.orchestra.run.smithi164.stdout: auth debug = true 2022-06-29T15:41:41.258 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:41:41.259 INFO:teuthology.orchestra.run.smithi164.stdout: ms die on old message = true 2022-06-29T15:41:41.259 INFO:teuthology.orchestra.run.smithi164.stdout: ms die on bug = true 2022-06-29T15:41:41.259 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:41:41.259 INFO:teuthology.orchestra.run.smithi164.stdout: mon max pg per osd = 10000 # >= luminous 2022-06-29T15:41:41.260 INFO:teuthology.orchestra.run.smithi164.stdout: mon pg warn max object skew = 0 2022-06-29T15:41:41.260 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:41:41.260 INFO:teuthology.orchestra.run.smithi164.stdout: # disable pg_autoscaler by default for new pools 2022-06-29T15:41:41.260 INFO:teuthology.orchestra.run.smithi164.stdout: osd_pool_default_pg_autoscale_mode = off 2022-06-29T15:41:41.260 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:41:41.261 INFO:teuthology.orchestra.run.smithi164.stdout: osd pool default size = 2 2022-06-29T15:41:41.261 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:41:41.261 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd allow primary affinity = true 2022-06-29T15:41:41.261 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd allow pg remap = true 2022-06-29T15:41:41.261 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on legacy crush tunables = false 2022-06-29T15:41:41.262 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on crush straw calc version zero = false 2022-06-29T15:41:41.262 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on no sortbitwise = false 2022-06-29T15:41:41.262 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on osd down out interval zero = false 2022-06-29T15:41:41.262 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on too few osds = false 2022-06-29T15:41:41.263 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-06-29T15:41:41.263 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_pool_no_redundancy = false 2022-06-29T15:41:41.263 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:41:41.263 INFO:teuthology.orchestra.run.smithi164.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-06-29T15:41:41.263 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:41:41.264 INFO:teuthology.orchestra.run.smithi164.stdout: osd default data pool replay window = 5 2022-06-29T15:41:41.264 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:41:41.264 INFO:teuthology.orchestra.run.smithi164.stdout: mon allow pool delete = true 2022-06-29T15:41:41.264 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:41:41.265 INFO:teuthology.orchestra.run.smithi164.stdout: mon cluster log file level = debug 2022-06-29T15:41:41.265 INFO:teuthology.orchestra.run.smithi164.stdout: debug asserts on shutdown = true 2022-06-29T15:41:41.265 INFO:teuthology.orchestra.run.smithi164.stdout: mon health detail to clog = false 2022-06-29T15:41:41.265 INFO:teuthology.orchestra.run.smithi164.stdout: mon host = "172.21.15.47,172.21.15.64,[v2:172.21.15.64:3301,v1:172.21.15.64:6790]" 2022-06-29T15:41:41.265 INFO:teuthology.orchestra.run.smithi164.stdout: fsid = 9a68894a-53ce-400e-838b-14eab91b834c 2022-06-29T15:41:41.266 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:41:41.266 INFO:teuthology.orchestra.run.smithi164.stdout:[osd] 2022-06-29T15:41:41.266 INFO:teuthology.orchestra.run.smithi164.stdout: osd journal size = 100 2022-06-29T15:41:41.266 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:41:41.267 INFO:teuthology.orchestra.run.smithi164.stdout: osd scrub load threshold = 5.0 2022-06-29T15:41:41.267 INFO:teuthology.orchestra.run.smithi164.stdout: osd scrub max interval = 600 2022-06-29T15:41:41.267 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:41:41.267 INFO:teuthology.orchestra.run.smithi164.stdout: osd recover clone overlap = true 2022-06-29T15:41:41.267 INFO:teuthology.orchestra.run.smithi164.stdout: osd recovery max chunk = 1048576 2022-06-29T15:41:41.268 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:41:41.268 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug shutdown = true 2022-06-29T15:41:41.268 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug op order = true 2022-06-29T15:41:41.268 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug verify stray on activate = true 2022-06-29T15:41:41.268 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:41:41.269 INFO:teuthology.orchestra.run.smithi164.stdout: osd open classes on start = true 2022-06-29T15:41:41.269 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug pg log writeout = true 2022-06-29T15:41:41.269 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:41:41.269 INFO:teuthology.orchestra.run.smithi164.stdout: osd deep scrub update digest min age = 30 2022-06-29T15:41:41.270 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:41:41.270 INFO:teuthology.orchestra.run.smithi164.stdout: osd map max advance = 10 2022-06-29T15:41:41.270 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:41:41.270 INFO:teuthology.orchestra.run.smithi164.stdout: journal zero on create = true 2022-06-29T15:41:41.270 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:41:41.271 INFO:teuthology.orchestra.run.smithi164.stdout: filestore ondisk finisher threads = 3 2022-06-29T15:41:41.271 INFO:teuthology.orchestra.run.smithi164.stdout: filestore apply finisher threads = 3 2022-06-29T15:41:41.271 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:41:41.271 INFO:teuthology.orchestra.run.smithi164.stdout: bdev debug aio = true 2022-06-29T15:41:41.271 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug misdirected ops = true 2022-06-29T15:41:41.272 INFO:teuthology.orchestra.run.smithi164.stdout: bluestore block size = 96636764160 2022-06-29T15:41:41.272 INFO:teuthology.orchestra.run.smithi164.stdout: bluestore compression mode = aggressive 2022-06-29T15:41:41.272 INFO:teuthology.orchestra.run.smithi164.stdout: bluestore fsck on mount = True 2022-06-29T15:41:41.272 INFO:teuthology.orchestra.run.smithi164.stdout: debug bluefs = 20 2022-06-29T15:41:41.273 INFO:teuthology.orchestra.run.smithi164.stdout: debug bluestore = 20 2022-06-29T15:41:41.273 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2022-06-29T15:41:41.273 INFO:teuthology.orchestra.run.smithi164.stdout: debug osd = 20 2022-06-29T15:41:41.273 INFO:teuthology.orchestra.run.smithi164.stdout: debug rocksdb = 10 2022-06-29T15:41:41.273 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd backfillfull_ratio = 0.85 2022-06-29T15:41:41.274 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd full ratio = 0.9 2022-06-29T15:41:41.274 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd nearfull ratio = 0.8 2022-06-29T15:41:41.274 INFO:teuthology.orchestra.run.smithi164.stdout: osd failsafe full ratio = 0.95 2022-06-29T15:41:41.274 INFO:teuthology.orchestra.run.smithi164.stdout: osd objectstore = bluestore 2022-06-29T15:41:41.275 INFO:teuthology.orchestra.run.smithi164.stdout: osd op complaint time = 180 2022-06-29T15:41:41.275 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:41:41.275 INFO:teuthology.orchestra.run.smithi164.stdout:[mgr] 2022-06-29T15:41:41.275 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2022-06-29T15:41:41.276 INFO:teuthology.orchestra.run.smithi164.stdout: debug mgr = 20 2022-06-29T15:41:41.276 INFO:teuthology.orchestra.run.smithi164.stdout: debug mon = 20 2022-06-29T15:41:41.276 INFO:teuthology.orchestra.run.smithi164.stdout: debug auth = 20 2022-06-29T15:41:41.276 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min pgs per osd = 4 2022-06-29T15:41:41.277 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min bytes per osd = 10 2022-06-29T15:41:41.277 INFO:teuthology.orchestra.run.smithi164.stdout: mgr/telemetry/nag = false 2022-06-29T15:41:41.277 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:41:41.277 INFO:teuthology.orchestra.run.smithi164.stdout:[mon] 2022-06-29T15:41:41.278 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2022-06-29T15:41:41.278 INFO:teuthology.orchestra.run.smithi164.stdout: debug mon = 20 2022-06-29T15:41:41.278 INFO:teuthology.orchestra.run.smithi164.stdout: debug paxos = 20 2022-06-29T15:41:41.278 INFO:teuthology.orchestra.run.smithi164.stdout: debug auth = 20 2022-06-29T15:41:41.278 INFO:teuthology.orchestra.run.smithi164.stdout: mon data avail warn = 5 2022-06-29T15:41:41.279 INFO:teuthology.orchestra.run.smithi164.stdout: mon mgr mkfs grace = 240 2022-06-29T15:41:41.279 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min pgs per osd = 4 2022-06-29T15:41:41.279 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd reporter subtree level = osd 2022-06-29T15:41:41.279 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd prime pg temp = true 2022-06-29T15:41:41.279 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min bytes per osd = 10 2022-06-29T15:41:41.280 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:41:41.280 INFO:teuthology.orchestra.run.smithi164.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-06-29T15:41:41.280 INFO:teuthology.orchestra.run.smithi164.stdout: auth mon ticket ttl = 660 # 11m 2022-06-29T15:41:41.280 INFO:teuthology.orchestra.run.smithi164.stdout: auth service ticket ttl = 240 # 4m 2022-06-29T15:41:41.281 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:41:41.281 INFO:teuthology.orchestra.run.smithi164.stdout: # don't complain about insecure global_id in the test suite 2022-06-29T15:41:41.281 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-06-29T15:41:41.281 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-06-29T15:41:41.281 INFO:teuthology.orchestra.run.smithi164.stdout: mon op complaint time = 120 2022-06-29T15:41:41.282 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:41:41.282 INFO:teuthology.orchestra.run.smithi164.stdout:[client] 2022-06-29T15:41:41.282 INFO:teuthology.orchestra.run.smithi164.stdout: rgw cache enabled = true 2022-06-29T15:41:41.282 INFO:teuthology.orchestra.run.smithi164.stdout: rgw enable ops log = true 2022-06-29T15:41:41.282 INFO:teuthology.orchestra.run.smithi164.stdout: rgw enable usage log = true 2022-06-29T15:41:41.283 INFO:teuthology.orchestra.run.smithi164.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-06-29T15:41:41.283 INFO:teuthology.orchestra.run.smithi164.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-06-29T15:41:41.283 INFO:teuthology.orchestra.run.smithi164.stdout: client mount timeout = 600 2022-06-29T15:41:41.283 INFO:teuthology.orchestra.run.smithi164.stdout: debug client = 20 2022-06-29T15:41:41.284 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2022-06-29T15:41:41.284 INFO:teuthology.orchestra.run.smithi164.stdout: rados mon op timeout = 15m 2022-06-29T15:41:41.284 INFO:teuthology.orchestra.run.smithi164.stdout: rados osd op timeout = 15m 2022-06-29T15:41:41.284 INFO:teuthology.orchestra.run.smithi164.stdout:[mon.a] 2022-06-29T15:41:41.284 INFO:teuthology.orchestra.run.smithi164.stdout:[mon.b] 2022-06-29T15:41:41.285 INFO:teuthology.orchestra.run.smithi164.stdout:[mon.c] 2022-06-29T15:41:41.285 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.a] 2022-06-29T15:41:41.285 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.c] 2022-06-29T15:41:41.285 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.b] 2022-06-29T15:41:41.285 INFO:teuthology.orchestra.run.smithi164.stdout:[mds] 2022-06-29T15:41:41.286 INFO:teuthology.orchestra.run.smithi164.stdout: debug mds = 20 2022-06-29T15:41:41.286 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2022-06-29T15:41:41.286 INFO:teuthology.orchestra.run.smithi164.stdout: mds bal frag = True 2022-06-29T15:41:41.286 INFO:teuthology.orchestra.run.smithi164.stdout: mds bal fragment size max = 10000 2022-06-29T15:41:41.287 INFO:teuthology.orchestra.run.smithi164.stdout: mds bal merge size = 5 2022-06-29T15:41:41.287 INFO:teuthology.orchestra.run.smithi164.stdout: mds bal split bits = 3 2022-06-29T15:41:41.287 INFO:teuthology.orchestra.run.smithi164.stdout: mds bal split size = 100 2022-06-29T15:41:41.287 INFO:teuthology.orchestra.run.smithi164.stdout: mds debug frag = True 2022-06-29T15:41:41.287 INFO:teuthology.orchestra.run.smithi164.stdout: mds debug scatterstat = True 2022-06-29T15:41:41.288 INFO:teuthology.orchestra.run.smithi164.stdout: mds op complaint time = 180 2022-06-29T15:41:41.288 INFO:teuthology.orchestra.run.smithi164.stdout: mds verify scatter = True 2022-06-29T15:41:41.288 INFO:teuthology.orchestra.run.smithi164.stdout: osd op complaint time = 180 2022-06-29T15:41:41.288 INFO:teuthology.orchestra.run.smithi164.stdout: rados mon op timeout = 15m 2022-06-29T15:41:41.288 INFO:teuthology.orchestra.run.smithi164.stdout: rados osd op timeout = 15m 2022-06-29T15:41:41.289 INFO:teuthology.orchestra.run.smithi064.stdout:[global] 2022-06-29T15:41:41.289 INFO:teuthology.orchestra.run.smithi064.stdout: chdir = "" 2022-06-29T15:41:41.289 INFO:teuthology.orchestra.run.smithi064.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-06-29T15:41:41.289 INFO:teuthology.orchestra.run.smithi064.stdout: auth supported = cephx 2022-06-29T15:41:41.290 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:41:41.290 INFO:teuthology.orchestra.run.smithi064.stdout: filestore xattr use omap = true 2022-06-29T15:41:41.290 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:41:41.290 INFO:teuthology.orchestra.run.smithi064.stdout: mon clock drift allowed = 1.000 2022-06-29T15:41:41.290 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:41:41.291 INFO:teuthology.orchestra.run.smithi064.stdout: osd crush chooseleaf type = 0 2022-06-29T15:41:41.291 INFO:teuthology.orchestra.run.smithi064.stdout: auth debug = true 2022-06-29T15:41:41.291 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:41:41.291 INFO:teuthology.orchestra.run.smithi064.stdout: ms die on old message = true 2022-06-29T15:41:41.292 INFO:teuthology.orchestra.run.smithi064.stdout: ms die on bug = true 2022-06-29T15:41:41.292 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:41:41.292 INFO:teuthology.orchestra.run.smithi064.stdout: mon max pg per osd = 10000 # >= luminous 2022-06-29T15:41:41.292 INFO:teuthology.orchestra.run.smithi064.stdout: mon pg warn max object skew = 0 2022-06-29T15:41:41.292 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:41:41.293 INFO:teuthology.orchestra.run.smithi064.stdout: # disable pg_autoscaler by default for new pools 2022-06-29T15:41:41.293 INFO:teuthology.orchestra.run.smithi064.stdout: osd_pool_default_pg_autoscale_mode = off 2022-06-29T15:41:41.293 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:41:41.293 INFO:teuthology.orchestra.run.smithi064.stdout: osd pool default size = 2 2022-06-29T15:41:41.294 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:41:41.294 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd allow primary affinity = true 2022-06-29T15:41:41.294 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd allow pg remap = true 2022-06-29T15:41:41.294 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on legacy crush tunables = false 2022-06-29T15:41:41.294 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on crush straw calc version zero = false 2022-06-29T15:41:41.295 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on no sortbitwise = false 2022-06-29T15:41:41.295 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on osd down out interval zero = false 2022-06-29T15:41:41.295 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on too few osds = false 2022-06-29T15:41:41.295 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-06-29T15:41:41.295 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_pool_no_redundancy = false 2022-06-29T15:41:41.296 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:41:41.296 INFO:teuthology.orchestra.run.smithi064.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-06-29T15:41:41.296 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:41:41.296 INFO:teuthology.orchestra.run.smithi064.stdout: osd default data pool replay window = 5 2022-06-29T15:41:41.297 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:41:41.297 INFO:teuthology.orchestra.run.smithi064.stdout: mon allow pool delete = true 2022-06-29T15:41:41.297 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:41:41.297 INFO:teuthology.orchestra.run.smithi064.stdout: mon cluster log file level = debug 2022-06-29T15:41:41.297 INFO:teuthology.orchestra.run.smithi064.stdout: debug asserts on shutdown = true 2022-06-29T15:41:41.298 INFO:teuthology.orchestra.run.smithi064.stdout: mon health detail to clog = false 2022-06-29T15:41:41.298 INFO:teuthology.orchestra.run.smithi064.stdout: mon host = "172.21.15.47,172.21.15.64,[v2:172.21.15.64:3301,v1:172.21.15.64:6790]" 2022-06-29T15:41:41.298 INFO:teuthology.orchestra.run.smithi064.stdout: fsid = 9a68894a-53ce-400e-838b-14eab91b834c 2022-06-29T15:41:41.298 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:41:41.298 INFO:teuthology.orchestra.run.smithi064.stdout:[osd] 2022-06-29T15:41:41.299 INFO:teuthology.orchestra.run.smithi064.stdout: osd journal size = 100 2022-06-29T15:41:41.299 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:41:41.299 INFO:teuthology.orchestra.run.smithi064.stdout: osd scrub load threshold = 5.0 2022-06-29T15:41:41.299 INFO:teuthology.orchestra.run.smithi064.stdout: osd scrub max interval = 600 2022-06-29T15:41:41.300 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:41:41.300 INFO:teuthology.orchestra.run.smithi064.stdout: osd recover clone overlap = true 2022-06-29T15:41:41.300 INFO:teuthology.orchestra.run.smithi064.stdout: osd recovery max chunk = 1048576 2022-06-29T15:41:41.300 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:41:41.300 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug shutdown = true 2022-06-29T15:41:41.301 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug op order = true 2022-06-29T15:41:41.301 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug verify stray on activate = true 2022-06-29T15:41:41.301 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:41:41.301 INFO:teuthology.orchestra.run.smithi064.stdout: osd open classes on start = true 2022-06-29T15:41:41.302 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug pg log writeout = true 2022-06-29T15:41:41.302 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:41:41.302 INFO:teuthology.orchestra.run.smithi064.stdout: osd deep scrub update digest min age = 30 2022-06-29T15:41:41.302 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:41:41.302 INFO:teuthology.orchestra.run.smithi064.stdout: osd map max advance = 10 2022-06-29T15:41:41.303 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:41:41.303 INFO:teuthology.orchestra.run.smithi064.stdout: journal zero on create = true 2022-06-29T15:41:41.303 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:41:41.303 INFO:teuthology.orchestra.run.smithi064.stdout: filestore ondisk finisher threads = 3 2022-06-29T15:41:41.303 INFO:teuthology.orchestra.run.smithi064.stdout: filestore apply finisher threads = 3 2022-06-29T15:41:41.304 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:41:41.304 INFO:teuthology.orchestra.run.smithi064.stdout: bdev debug aio = true 2022-06-29T15:41:41.304 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug misdirected ops = true 2022-06-29T15:41:41.304 INFO:teuthology.orchestra.run.smithi064.stdout: bluestore block size = 96636764160 2022-06-29T15:41:41.305 INFO:teuthology.orchestra.run.smithi064.stdout: bluestore compression mode = aggressive 2022-06-29T15:41:41.305 INFO:teuthology.orchestra.run.smithi064.stdout: bluestore fsck on mount = True 2022-06-29T15:41:41.305 INFO:teuthology.orchestra.run.smithi064.stdout: debug bluefs = 20 2022-06-29T15:41:41.305 INFO:teuthology.orchestra.run.smithi064.stdout: debug bluestore = 20 2022-06-29T15:41:41.305 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2022-06-29T15:41:41.306 INFO:teuthology.orchestra.run.smithi064.stdout: debug osd = 20 2022-06-29T15:41:41.306 INFO:teuthology.orchestra.run.smithi064.stdout: debug rocksdb = 10 2022-06-29T15:41:41.306 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd backfillfull_ratio = 0.85 2022-06-29T15:41:41.306 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd full ratio = 0.9 2022-06-29T15:41:41.306 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd nearfull ratio = 0.8 2022-06-29T15:41:41.307 INFO:teuthology.orchestra.run.smithi064.stdout: osd failsafe full ratio = 0.95 2022-06-29T15:41:41.307 INFO:teuthology.orchestra.run.smithi064.stdout: osd objectstore = bluestore 2022-06-29T15:41:41.307 INFO:teuthology.orchestra.run.smithi064.stdout: osd op complaint time = 180 2022-06-29T15:41:41.307 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:41:41.308 INFO:teuthology.orchestra.run.smithi064.stdout:[mgr] 2022-06-29T15:41:41.308 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2022-06-29T15:41:41.308 INFO:teuthology.orchestra.run.smithi064.stdout: debug mgr = 20 2022-06-29T15:41:41.308 INFO:teuthology.orchestra.run.smithi064.stdout: debug mon = 20 2022-06-29T15:41:41.308 INFO:teuthology.orchestra.run.smithi064.stdout: debug auth = 20 2022-06-29T15:41:41.309 INFO:teuthology.orchestra.run.smithi064.stdout: mon reweight min pgs per osd = 4 2022-06-29T15:41:41.309 INFO:teuthology.orchestra.run.smithi064.stdout: mon reweight min bytes per osd = 10 2022-06-29T15:41:41.309 INFO:teuthology.orchestra.run.smithi064.stdout: mgr/telemetry/nag = false 2022-06-29T15:41:41.309 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:41:41.309 INFO:teuthology.orchestra.run.smithi064.stdout:[mon] 2022-06-29T15:41:41.310 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2022-06-29T15:41:41.310 INFO:teuthology.orchestra.run.smithi064.stdout: debug mon = 20 2022-06-29T15:41:41.310 INFO:teuthology.orchestra.run.smithi064.stdout: debug paxos = 20 2022-06-29T15:41:41.310 INFO:teuthology.orchestra.run.smithi064.stdout: debug auth = 20 2022-06-29T15:41:41.311 INFO:teuthology.orchestra.run.smithi064.stdout: mon data avail warn = 5 2022-06-29T15:41:41.311 INFO:teuthology.orchestra.run.smithi064.stdout: mon mgr mkfs grace = 240 2022-06-29T15:41:41.311 INFO:teuthology.orchestra.run.smithi064.stdout: mon reweight min pgs per osd = 4 2022-06-29T15:41:41.311 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd reporter subtree level = osd 2022-06-29T15:41:41.311 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd prime pg temp = true 2022-06-29T15:41:41.312 INFO:teuthology.orchestra.run.smithi064.stdout: mon reweight min bytes per osd = 10 2022-06-29T15:41:41.312 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:41:41.312 INFO:teuthology.orchestra.run.smithi064.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-06-29T15:41:41.312 INFO:teuthology.orchestra.run.smithi064.stdout: auth mon ticket ttl = 660 # 11m 2022-06-29T15:41:41.312 INFO:teuthology.orchestra.run.smithi064.stdout: auth service ticket ttl = 240 # 4m 2022-06-29T15:41:41.313 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:41:41.313 INFO:teuthology.orchestra.run.smithi064.stdout: # don't complain about insecure global_id in the test suite 2022-06-29T15:41:41.313 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-06-29T15:41:41.313 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-06-29T15:41:41.313 INFO:teuthology.orchestra.run.smithi064.stdout: mon op complaint time = 120 2022-06-29T15:41:41.314 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:41:41.314 INFO:teuthology.orchestra.run.smithi064.stdout:[client] 2022-06-29T15:41:41.314 INFO:teuthology.orchestra.run.smithi064.stdout: rgw cache enabled = true 2022-06-29T15:41:41.314 INFO:teuthology.orchestra.run.smithi064.stdout: rgw enable ops log = true 2022-06-29T15:41:41.315 INFO:teuthology.orchestra.run.smithi064.stdout: rgw enable usage log = true 2022-06-29T15:41:41.315 INFO:teuthology.orchestra.run.smithi064.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-06-29T15:41:41.315 INFO:teuthology.orchestra.run.smithi064.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-06-29T15:41:41.315 INFO:teuthology.orchestra.run.smithi064.stdout: client mount timeout = 600 2022-06-29T15:41:41.315 INFO:teuthology.orchestra.run.smithi064.stdout: debug client = 20 2022-06-29T15:41:41.316 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2022-06-29T15:41:41.316 INFO:teuthology.orchestra.run.smithi064.stdout: rados mon op timeout = 15m 2022-06-29T15:41:41.316 INFO:teuthology.orchestra.run.smithi064.stdout: rados osd op timeout = 15m 2022-06-29T15:41:41.316 INFO:teuthology.orchestra.run.smithi064.stdout:[mon.a] 2022-06-29T15:41:41.316 INFO:teuthology.orchestra.run.smithi064.stdout:[mon.b] 2022-06-29T15:41:41.317 INFO:teuthology.orchestra.run.smithi064.stdout:[mon.c] 2022-06-29T15:41:41.317 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.a] 2022-06-29T15:41:41.317 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.c] 2022-06-29T15:41:41.317 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.b] 2022-06-29T15:41:41.318 INFO:teuthology.orchestra.run.smithi064.stdout:[mds] 2022-06-29T15:41:41.318 INFO:teuthology.orchestra.run.smithi064.stdout: debug mds = 20 2022-06-29T15:41:41.318 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2022-06-29T15:41:41.318 INFO:teuthology.orchestra.run.smithi064.stdout: mds bal frag = True 2022-06-29T15:41:41.318 INFO:teuthology.orchestra.run.smithi064.stdout: mds bal fragment size max = 10000 2022-06-29T15:41:41.319 INFO:teuthology.orchestra.run.smithi064.stdout: mds bal merge size = 5 2022-06-29T15:41:41.319 INFO:teuthology.orchestra.run.smithi064.stdout: mds bal split bits = 3 2022-06-29T15:41:41.319 INFO:teuthology.orchestra.run.smithi064.stdout: mds bal split size = 100 2022-06-29T15:41:41.319 INFO:teuthology.orchestra.run.smithi064.stdout: mds debug frag = True 2022-06-29T15:41:41.319 INFO:teuthology.orchestra.run.smithi064.stdout: mds debug scatterstat = True 2022-06-29T15:41:41.320 INFO:teuthology.orchestra.run.smithi064.stdout: mds op complaint time = 180 2022-06-29T15:41:41.320 INFO:teuthology.orchestra.run.smithi064.stdout: mds verify scatter = True 2022-06-29T15:41:41.320 INFO:teuthology.orchestra.run.smithi064.stdout: osd op complaint time = 180 2022-06-29T15:41:41.320 INFO:teuthology.orchestra.run.smithi064.stdout: rados mon op timeout = 15m 2022-06-29T15:41:41.321 INFO:teuthology.orchestra.run.smithi064.stdout: rados osd op timeout = 15m 2022-06-29T15:41:41.321 INFO:tasks.ceph:Creating admin key on mon.a... 2022-06-29T15:41:41.321 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2022-06-29T15:41:41.370 INFO:tasks.ceph:Copying monmap to all nodes... 2022-06-29T15:41:41.379 DEBUG:teuthology.orchestra.remote:smithi047:/etc/ceph/ceph.keyring is 206B 2022-06-29T15:41:41.405 DEBUG:teuthology.orchestra.remote:smithi047:/home/ubuntu/cephtest/ceph.monmap is 379B 2022-06-29T15:41:41.418 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi047.front.sepia.ceph.com 2022-06-29T15:41:41.419 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2022-06-29T15:41:41.419 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/ceph/ceph.keyring 2022-06-29T15:41:41.420 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-06-29T15:41:41.446 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2022-06-29T15:41:41.446 DEBUG:teuthology.orchestra.run.smithi047:> dd of=/home/ubuntu/cephtest/ceph.monmap 2022-06-29T15:41:41.496 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi064.front.sepia.ceph.com 2022-06-29T15:41:41.497 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2022-06-29T15:41:41.497 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/ceph/ceph.keyring 2022-06-29T15:41:41.497 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-06-29T15:41:41.524 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2022-06-29T15:41:41.525 DEBUG:teuthology.orchestra.run.smithi064:> dd of=/home/ubuntu/cephtest/ceph.monmap 2022-06-29T15:41:41.531 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi099.front.sepia.ceph.com 2022-06-29T15:41:41.531 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2022-06-29T15:41:41.532 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/ceph/ceph.keyring 2022-06-29T15:41:41.532 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-06-29T15:41:41.559 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2022-06-29T15:41:41.559 DEBUG:teuthology.orchestra.run.smithi099:> dd of=/home/ubuntu/cephtest/ceph.monmap 2022-06-29T15:41:41.609 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi164.front.sepia.ceph.com 2022-06-29T15:41:41.610 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2022-06-29T15:41:41.611 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/ceph/ceph.keyring 2022-06-29T15:41:41.612 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-06-29T15:41:41.640 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2022-06-29T15:41:41.640 DEBUG:teuthology.orchestra.run.smithi164:> dd of=/home/ubuntu/cephtest/ceph.monmap 2022-06-29T15:41:41.694 INFO:tasks.ceph:Setting up mon nodes... 2022-06-29T15:41:41.694 INFO:tasks.ceph:Setting up mgr nodes... 2022-06-29T15:41:41.695 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2022-06-29T15:41:41.740 INFO:teuthology.orchestra.run.smithi047.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2022-06-29T15:41:41.743 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2022-06-29T15:41:41.814 INFO:teuthology.orchestra.run.smithi064.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2022-06-29T15:41:41.817 INFO:tasks.ceph:Setting up mds nodes... 2022-06-29T15:41:41.818 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2022-06-29T15:41:41.862 INFO:teuthology.orchestra.run.smithi047.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2022-06-29T15:41:41.864 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2022-06-29T15:41:41.922 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2022-06-29T15:41:41.963 INFO:teuthology.orchestra.run.smithi047.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2022-06-29T15:41:41.965 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2022-06-29T15:41:41.981 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2022-06-29T15:41:42.025 INFO:teuthology.orchestra.run.smithi064.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2022-06-29T15:41:42.028 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2022-06-29T15:41:42.084 INFO:tasks.ceph_client:Setting up client nodes... 2022-06-29T15:41:42.084 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2022-06-29T15:41:42.123 INFO:teuthology.orchestra.run.smithi099.stdout:creating /etc/ceph/ceph.client.0.keyring 2022-06-29T15:41:42.137 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2022-06-29T15:41:42.192 INFO:teuthology.orchestra.run.smithi164.stdout:creating /etc/ceph/ceph.client.1.keyring 2022-06-29T15:41:42.206 INFO:tasks.ceph:Running mkfs on osd nodes... 2022-06-29T15:41:42.207 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi047.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi064.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2022-06-29T15:41:42.207 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2022-06-29T15:41:42.224 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-06-29T15:41:42.224 INFO:tasks.ceph:role: osd.0 2022-06-29T15:41:42.224 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi047.front.sepia.ceph.com 2022-06-29T15:41:42.225 DEBUG:teuthology.orchestra.run.smithi047:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2022-06-29T15:41:42.268 INFO:teuthology.orchestra.run.smithi047.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2022-06-29T15:41:42.269 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 attr=2, projid32bit=1 2022-06-29T15:41:42.269 INFO:teuthology.orchestra.run.smithi047.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-06-29T15:41:42.269 INFO:teuthology.orchestra.run.smithi047.stdout: = reflink=1 2022-06-29T15:41:42.269 INFO:teuthology.orchestra.run.smithi047.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-06-29T15:41:42.270 INFO:teuthology.orchestra.run.smithi047.stdout: = sunit=0 swidth=0 blks 2022-06-29T15:41:42.270 INFO:teuthology.orchestra.run.smithi047.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-06-29T15:41:42.270 INFO:teuthology.orchestra.run.smithi047.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-06-29T15:41:42.270 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-06-29T15:41:42.271 INFO:teuthology.orchestra.run.smithi047.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-06-29T15:41:42.272 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi047.front.sepia.ceph.com -o noatime 2022-06-29T15:41:42.272 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2022-06-29T15:41:42.292 DEBUG:teuthology.orchestra.run.smithi047:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2022-06-29T15:41:42.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T15:41:42.301 INFO:teuthology.orchestra.run.smithi047.stderr:sudo: /sbin/restorecon: command not found 2022-06-29T15:41:42.302 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2022-06-29T15:41:42.358 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-06-29T15:41:42.359 INFO:tasks.ceph:role: osd.1 2022-06-29T15:41:42.359 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi047.front.sepia.ceph.com 2022-06-29T15:41:42.359 DEBUG:teuthology.orchestra.run.smithi047:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2022-06-29T15:41:42.401 INFO:teuthology.orchestra.run.smithi047.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2022-06-29T15:41:42.401 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 attr=2, projid32bit=1 2022-06-29T15:41:42.402 INFO:teuthology.orchestra.run.smithi047.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-06-29T15:41:42.402 INFO:teuthology.orchestra.run.smithi047.stdout: = reflink=1 2022-06-29T15:41:42.402 INFO:teuthology.orchestra.run.smithi047.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-06-29T15:41:42.402 INFO:teuthology.orchestra.run.smithi047.stdout: = sunit=0 swidth=0 blks 2022-06-29T15:41:42.403 INFO:teuthology.orchestra.run.smithi047.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-06-29T15:41:42.403 INFO:teuthology.orchestra.run.smithi047.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-06-29T15:41:42.403 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-06-29T15:41:42.403 INFO:teuthology.orchestra.run.smithi047.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-06-29T15:41:42.404 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi047.front.sepia.ceph.com -o noatime 2022-06-29T15:41:42.405 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2022-06-29T15:41:42.425 DEBUG:teuthology.orchestra.run.smithi047:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2022-06-29T15:41:42.478 INFO:teuthology.orchestra.run.smithi047.stderr:sudo: /sbin/restorecon: command not found 2022-06-29T15:41:42.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T15:41:42.480 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2022-06-29T15:41:42.494 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-06-29T15:41:42.494 INFO:tasks.ceph:role: osd.2 2022-06-29T15:41:42.495 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi047.front.sepia.ceph.com 2022-06-29T15:41:42.495 DEBUG:teuthology.orchestra.run.smithi047:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2022-06-29T15:41:42.537 INFO:teuthology.orchestra.run.smithi047.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2022-06-29T15:41:42.537 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 attr=2, projid32bit=1 2022-06-29T15:41:42.538 INFO:teuthology.orchestra.run.smithi047.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-06-29T15:41:42.538 INFO:teuthology.orchestra.run.smithi047.stdout: = reflink=1 2022-06-29T15:41:42.538 INFO:teuthology.orchestra.run.smithi047.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-06-29T15:41:42.539 INFO:teuthology.orchestra.run.smithi047.stdout: = sunit=0 swidth=0 blks 2022-06-29T15:41:42.539 INFO:teuthology.orchestra.run.smithi047.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-06-29T15:41:42.539 INFO:teuthology.orchestra.run.smithi047.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-06-29T15:41:42.540 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-06-29T15:41:42.540 INFO:teuthology.orchestra.run.smithi047.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-06-29T15:41:42.542 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi047.front.sepia.ceph.com -o noatime 2022-06-29T15:41:42.542 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2022-06-29T15:41:42.608 DEBUG:teuthology.orchestra.run.smithi047:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2022-06-29T15:41:42.617 INFO:teuthology.orchestra.run.smithi047.stderr:sudo: /sbin/restorecon: command not found 2022-06-29T15:41:42.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T15:41:42.619 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2022-06-29T15:41:42.629 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-06-29T15:41:42.630 INFO:tasks.ceph:role: osd.3 2022-06-29T15:41:42.631 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi047.front.sepia.ceph.com 2022-06-29T15:41:42.631 DEBUG:teuthology.orchestra.run.smithi047:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2022-06-29T15:41:42.718 INFO:teuthology.orchestra.run.smithi047.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2022-06-29T15:41:42.718 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 attr=2, projid32bit=1 2022-06-29T15:41:42.718 INFO:teuthology.orchestra.run.smithi047.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-06-29T15:41:42.719 INFO:teuthology.orchestra.run.smithi047.stdout: = reflink=1 2022-06-29T15:41:42.719 INFO:teuthology.orchestra.run.smithi047.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-06-29T15:41:42.719 INFO:teuthology.orchestra.run.smithi047.stdout: = sunit=0 swidth=0 blks 2022-06-29T15:41:42.719 INFO:teuthology.orchestra.run.smithi047.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-06-29T15:41:42.719 INFO:teuthology.orchestra.run.smithi047.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-06-29T15:41:42.720 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-06-29T15:41:42.720 INFO:teuthology.orchestra.run.smithi047.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-06-29T15:41:42.721 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi047.front.sepia.ceph.com -o noatime 2022-06-29T15:41:42.722 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2022-06-29T15:41:42.740 DEBUG:teuthology.orchestra.run.smithi047:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2022-06-29T15:41:42.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T15:41:42.751 INFO:teuthology.orchestra.run.smithi047.stderr:sudo: /sbin/restorecon: command not found 2022-06-29T15:41:42.752 DEBUG:teuthology.orchestra.run.smithi047:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-06-29T15:41:42.861 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:41:42.856+0000 7f6cf1cf9d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2022-06-29T15:41:42.861 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:41:42.856+0000 7f6cf1cf9d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2022-06-29T15:41:42.861 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:41:42.856+0000 7f6cf1cf9d80 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2022-06-29T15:41:42.862 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:41:42.856+0000 7f6cf1cf9d80 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2022-06-29T15:41:42.862 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:41:42.856+0000 7f6cf1cf9d80 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2022-06-29T15:41:42.863 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:41:42.856+0000 7f6cf1cf9d80 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2022-06-29T15:41:42.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:41:42.876+0000 7f6cf1cf9d80 -1 freelist read_size_meta_from_db missing size meta in DB 2022-06-29T15:41:45.333 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2022-06-29T15:41:45.351 DEBUG:teuthology.orchestra.run.smithi047:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-06-29T15:41:45.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:41:45.384+0000 7f585d6ecd80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2022-06-29T15:41:45.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:41:45.384+0000 7f585d6ecd80 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2022-06-29T15:41:45.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:41:45.384+0000 7f585d6ecd80 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2022-06-29T15:41:45.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:41:45.384+0000 7f585d6ecd80 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2022-06-29T15:41:45.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:41:45.384+0000 7f585d6ecd80 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2022-06-29T15:41:45.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:41:45.384+0000 7f585d6ecd80 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2022-06-29T15:41:45.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:41:45.404+0000 7f585d6ecd80 -1 freelist read_size_meta_from_db missing size meta in DB 2022-06-29T15:41:47.836 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2022-06-29T15:41:47.892 DEBUG:teuthology.orchestra.run.smithi047:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-06-29T15:41:47.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:41:47.924+0000 7f15e0e0bd80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2022-06-29T15:41:47.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:41:47.924+0000 7f15e0e0bd80 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2022-06-29T15:41:47.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:41:47.924+0000 7f15e0e0bd80 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2022-06-29T15:41:47.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:41:47.924+0000 7f15e0e0bd80 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2022-06-29T15:41:47.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:41:47.924+0000 7f15e0e0bd80 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2022-06-29T15:41:47.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:41:47.924+0000 7f15e0e0bd80 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2022-06-29T15:41:47.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:41:47.940+0000 7f15e0e0bd80 -1 freelist read_size_meta_from_db missing size meta in DB 2022-06-29T15:41:50.392 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2022-06-29T15:41:50.409 DEBUG:teuthology.orchestra.run.smithi047:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-06-29T15:41:50.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:41:50.440+0000 7f950202ed80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2022-06-29T15:41:50.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:41:50.440+0000 7f950202ed80 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2022-06-29T15:41:50.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:41:50.440+0000 7f950202ed80 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2022-06-29T15:41:50.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:41:50.440+0000 7f950202ed80 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2022-06-29T15:41:50.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:41:50.440+0000 7f950202ed80 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2022-06-29T15:41:50.446 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:41:50.440+0000 7f950202ed80 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2022-06-29T15:41:50.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:41:50.460+0000 7f950202ed80 -1 freelist read_size_meta_from_db missing size meta in DB 2022-06-29T15:41:52.907 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2022-06-29T15:41:52.925 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2022-06-29T15:41:52.943 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-06-29T15:41:52.943 INFO:tasks.ceph:role: osd.4 2022-06-29T15:41:52.944 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi064.front.sepia.ceph.com 2022-06-29T15:41:52.944 DEBUG:teuthology.orchestra.run.smithi064:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2022-06-29T15:41:53.497 INFO:teuthology.orchestra.run.smithi064.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2022-06-29T15:41:53.497 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 attr=2, projid32bit=1 2022-06-29T15:41:53.497 INFO:teuthology.orchestra.run.smithi064.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-06-29T15:41:53.498 INFO:teuthology.orchestra.run.smithi064.stdout: = reflink=1 2022-06-29T15:41:53.498 INFO:teuthology.orchestra.run.smithi064.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-06-29T15:41:53.498 INFO:teuthology.orchestra.run.smithi064.stdout: = sunit=0 swidth=0 blks 2022-06-29T15:41:53.498 INFO:teuthology.orchestra.run.smithi064.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-06-29T15:41:53.499 INFO:teuthology.orchestra.run.smithi064.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-06-29T15:41:53.499 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-06-29T15:41:53.499 INFO:teuthology.orchestra.run.smithi064.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-06-29T15:41:53.500 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi064.front.sepia.ceph.com -o noatime 2022-06-29T15:41:53.501 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2022-06-29T15:41:53.598 DEBUG:teuthology.orchestra.run.smithi064:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2022-06-29T15:41:53.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T15:41:53.611 INFO:teuthology.orchestra.run.smithi064.stderr:sudo: /sbin/restorecon: command not found 2022-06-29T15:41:53.611 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2022-06-29T15:41:53.669 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-06-29T15:41:53.669 INFO:tasks.ceph:role: osd.5 2022-06-29T15:41:53.669 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi064.front.sepia.ceph.com 2022-06-29T15:41:53.670 DEBUG:teuthology.orchestra.run.smithi064:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2022-06-29T15:41:54.192 INFO:teuthology.orchestra.run.smithi064.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2022-06-29T15:41:54.192 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 attr=2, projid32bit=1 2022-06-29T15:41:54.193 INFO:teuthology.orchestra.run.smithi064.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-06-29T15:41:54.193 INFO:teuthology.orchestra.run.smithi064.stdout: = reflink=1 2022-06-29T15:41:54.193 INFO:teuthology.orchestra.run.smithi064.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-06-29T15:41:54.194 INFO:teuthology.orchestra.run.smithi064.stdout: = sunit=0 swidth=0 blks 2022-06-29T15:41:54.194 INFO:teuthology.orchestra.run.smithi064.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-06-29T15:41:54.195 INFO:teuthology.orchestra.run.smithi064.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-06-29T15:41:54.195 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-06-29T15:41:54.195 INFO:teuthology.orchestra.run.smithi064.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-06-29T15:41:54.196 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi064.front.sepia.ceph.com -o noatime 2022-06-29T15:41:54.197 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2022-06-29T15:41:54.234 DEBUG:teuthology.orchestra.run.smithi064:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2022-06-29T15:41:54.288 INFO:teuthology.orchestra.run.smithi064.stderr:sudo: /sbin/restorecon: command not found 2022-06-29T15:41:54.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T15:41:54.290 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2022-06-29T15:41:54.307 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-06-29T15:41:54.308 INFO:tasks.ceph:role: osd.6 2022-06-29T15:41:54.308 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi064.front.sepia.ceph.com 2022-06-29T15:41:54.308 DEBUG:teuthology.orchestra.run.smithi064:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2022-06-29T15:41:54.779 INFO:teuthology.orchestra.run.smithi064.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2022-06-29T15:41:54.780 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 attr=2, projid32bit=1 2022-06-29T15:41:54.780 INFO:teuthology.orchestra.run.smithi064.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-06-29T15:41:54.780 INFO:teuthology.orchestra.run.smithi064.stdout: = reflink=1 2022-06-29T15:41:54.781 INFO:teuthology.orchestra.run.smithi064.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-06-29T15:41:54.781 INFO:teuthology.orchestra.run.smithi064.stdout: = sunit=0 swidth=0 blks 2022-06-29T15:41:54.781 INFO:teuthology.orchestra.run.smithi064.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-06-29T15:41:54.781 INFO:teuthology.orchestra.run.smithi064.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-06-29T15:41:54.782 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-06-29T15:41:54.782 INFO:teuthology.orchestra.run.smithi064.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-06-29T15:41:54.783 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi064.front.sepia.ceph.com -o noatime 2022-06-29T15:41:54.783 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2022-06-29T15:41:54.843 DEBUG:teuthology.orchestra.run.smithi064:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2022-06-29T15:41:54.856 INFO:teuthology.orchestra.run.smithi064.stderr:sudo: /sbin/restorecon: command not found 2022-06-29T15:41:54.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T15:41:54.858 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2022-06-29T15:41:54.873 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-06-29T15:41:54.874 INFO:tasks.ceph:role: osd.7 2022-06-29T15:41:54.874 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi064.front.sepia.ceph.com 2022-06-29T15:41:54.875 DEBUG:teuthology.orchestra.run.smithi064:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2022-06-29T15:41:55.375 INFO:teuthology.orchestra.run.smithi064.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2022-06-29T15:41:55.376 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 attr=2, projid32bit=1 2022-06-29T15:41:55.377 INFO:teuthology.orchestra.run.smithi064.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-06-29T15:41:55.377 INFO:teuthology.orchestra.run.smithi064.stdout: = reflink=1 2022-06-29T15:41:55.378 INFO:teuthology.orchestra.run.smithi064.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-06-29T15:41:55.378 INFO:teuthology.orchestra.run.smithi064.stdout: = sunit=0 swidth=0 blks 2022-06-29T15:41:55.379 INFO:teuthology.orchestra.run.smithi064.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-06-29T15:41:55.379 INFO:teuthology.orchestra.run.smithi064.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-06-29T15:41:55.379 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-06-29T15:41:55.380 INFO:teuthology.orchestra.run.smithi064.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-06-29T15:41:55.382 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi064.front.sepia.ceph.com -o noatime 2022-06-29T15:41:55.382 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2022-06-29T15:41:55.459 DEBUG:teuthology.orchestra.run.smithi064:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2022-06-29T15:41:55.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T15:41:55.471 INFO:teuthology.orchestra.run.smithi064.stderr:sudo: /sbin/restorecon: command not found 2022-06-29T15:41:55.471 DEBUG:teuthology.orchestra.run.smithi064:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-06-29T15:41:55.580 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:41:55.574+0000 7f853277fd80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2022-06-29T15:41:55.581 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:41:55.578+0000 7f853277fd80 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2022-06-29T15:41:55.582 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:41:55.578+0000 7f853277fd80 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2022-06-29T15:41:55.582 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:41:55.578+0000 7f853277fd80 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2022-06-29T15:41:55.582 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:41:55.578+0000 7f853277fd80 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2022-06-29T15:41:55.590 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:41:55.586+0000 7f853277fd80 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2022-06-29T15:41:55.627 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:41:55.622+0000 7f853277fd80 -1 freelist read_size_meta_from_db missing size meta in DB 2022-06-29T15:41:58.051 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2022-06-29T15:41:58.065 DEBUG:teuthology.orchestra.run.smithi064:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-06-29T15:41:58.104 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:41:58.098+0000 7f6eeb69ad80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2022-06-29T15:41:58.104 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:41:58.098+0000 7f6eeb69ad80 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2022-06-29T15:41:58.104 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:41:58.098+0000 7f6eeb69ad80 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2022-06-29T15:41:58.105 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:41:58.098+0000 7f6eeb69ad80 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2022-06-29T15:41:58.105 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:41:58.098+0000 7f6eeb69ad80 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2022-06-29T15:41:58.105 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:41:58.102+0000 7f6eeb69ad80 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2022-06-29T15:41:58.124 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:41:58.118+0000 7f6eeb69ad80 -1 freelist read_size_meta_from_db missing size meta in DB 2022-06-29T15:42:00.563 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2022-06-29T15:42:00.620 DEBUG:teuthology.orchestra.run.smithi064:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-06-29T15:42:00.655 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:00.650+0000 7fa831c3cd80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2022-06-29T15:42:00.657 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:00.650+0000 7fa831c3cd80 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2022-06-29T15:42:00.657 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:00.650+0000 7fa831c3cd80 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2022-06-29T15:42:00.657 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:00.650+0000 7fa831c3cd80 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2022-06-29T15:42:00.658 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:00.650+0000 7fa831c3cd80 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2022-06-29T15:42:00.658 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:00.654+0000 7fa831c3cd80 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2022-06-29T15:42:00.676 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:00.674+0000 7fa831c3cd80 -1 freelist read_size_meta_from_db missing size meta in DB 2022-06-29T15:42:03.114 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2022-06-29T15:42:03.137 DEBUG:teuthology.orchestra.run.smithi064:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-06-29T15:42:03.219 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:03.214+0000 7f8ad7b79d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2022-06-29T15:42:03.220 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:03.214+0000 7f8ad7b79d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2022-06-29T15:42:03.220 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:03.214+0000 7f8ad7b79d80 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2022-06-29T15:42:03.221 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:03.214+0000 7f8ad7b79d80 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2022-06-29T15:42:03.221 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:03.214+0000 7f8ad7b79d80 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2022-06-29T15:42:03.221 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:03.218+0000 7f8ad7b79d80 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2022-06-29T15:42:03.239 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:03.234+0000 7f8ad7b79d80 -1 freelist read_size_meta_from_db missing size meta in DB 2022-06-29T15:42:05.662 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2022-06-29T15:42:05.678 INFO:tasks.ceph:Reading keys from all nodes... 2022-06-29T15:42:05.679 DEBUG:teuthology.orchestra.run.smithi047:> mktemp 2022-06-29T15:42:05.687 INFO:teuthology.orchestra.run.smithi047.stdout:/tmp/tmp.gQulRyc0Fr 2022-06-29T15:42:05.688 DEBUG:teuthology.orchestra.run.smithi047:> sudo cp /var/lib/ceph/mgr/ceph-y/keyring /tmp/tmp.gQulRyc0Fr 2022-06-29T15:42:05.704 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0666 /tmp/tmp.gQulRyc0Fr 2022-06-29T15:42:05.724 DEBUG:teuthology.orchestra.remote:smithi047:/tmp/tmp.gQulRyc0Fr is 56B 2022-06-29T15:42:05.739 DEBUG:teuthology.orchestra.run.smithi047:> rm -fr /tmp/tmp.gQulRyc0Fr 2022-06-29T15:42:05.746 DEBUG:teuthology.orchestra.run.smithi047:> mktemp 2022-06-29T15:42:05.796 INFO:teuthology.orchestra.run.smithi047.stdout:/tmp/tmp.rkQWkeV2sy 2022-06-29T15:42:05.796 DEBUG:teuthology.orchestra.run.smithi047:> sudo cp /var/lib/ceph/mds/ceph-a/keyring /tmp/tmp.rkQWkeV2sy 2022-06-29T15:42:05.812 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0666 /tmp/tmp.rkQWkeV2sy 2022-06-29T15:42:05.876 DEBUG:teuthology.orchestra.remote:smithi047:/tmp/tmp.rkQWkeV2sy is 56B 2022-06-29T15:42:05.894 DEBUG:teuthology.orchestra.run.smithi047:> rm -fr /tmp/tmp.rkQWkeV2sy 2022-06-29T15:42:05.902 DEBUG:teuthology.orchestra.run.smithi047:> mktemp 2022-06-29T15:42:05.951 INFO:teuthology.orchestra.run.smithi047.stdout:/tmp/tmp.u202VWtsTq 2022-06-29T15:42:05.952 DEBUG:teuthology.orchestra.run.smithi047:> sudo cp /var/lib/ceph/mds/ceph-c/keyring /tmp/tmp.u202VWtsTq 2022-06-29T15:42:05.968 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0666 /tmp/tmp.u202VWtsTq 2022-06-29T15:42:06.033 DEBUG:teuthology.orchestra.remote:smithi047:/tmp/tmp.u202VWtsTq is 56B 2022-06-29T15:42:06.048 DEBUG:teuthology.orchestra.run.smithi047:> rm -fr /tmp/tmp.u202VWtsTq 2022-06-29T15:42:06.054 DEBUG:teuthology.orchestra.run.smithi047:> mktemp 2022-06-29T15:42:06.107 INFO:teuthology.orchestra.run.smithi047.stdout:/tmp/tmp.jBSoXU0npM 2022-06-29T15:42:06.108 DEBUG:teuthology.orchestra.run.smithi047:> sudo cp /var/lib/ceph/osd/ceph-0/keyring /tmp/tmp.jBSoXU0npM 2022-06-29T15:42:06.124 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0666 /tmp/tmp.jBSoXU0npM 2022-06-29T15:42:06.196 DEBUG:teuthology.orchestra.remote:smithi047:/tmp/tmp.jBSoXU0npM is 56B 2022-06-29T15:42:06.211 DEBUG:teuthology.orchestra.run.smithi047:> rm -fr /tmp/tmp.jBSoXU0npM 2022-06-29T15:42:06.217 DEBUG:teuthology.orchestra.run.smithi047:> mktemp 2022-06-29T15:42:06.224 INFO:teuthology.orchestra.run.smithi047.stdout:/tmp/tmp.DtYQXJUjir 2022-06-29T15:42:06.225 DEBUG:teuthology.orchestra.run.smithi047:> sudo cp /var/lib/ceph/osd/ceph-1/keyring /tmp/tmp.DtYQXJUjir 2022-06-29T15:42:06.282 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0666 /tmp/tmp.DtYQXJUjir 2022-06-29T15:42:06.308 DEBUG:teuthology.orchestra.remote:smithi047:/tmp/tmp.DtYQXJUjir is 56B 2022-06-29T15:42:06.321 DEBUG:teuthology.orchestra.run.smithi047:> rm -fr /tmp/tmp.DtYQXJUjir 2022-06-29T15:42:06.329 DEBUG:teuthology.orchestra.run.smithi047:> mktemp 2022-06-29T15:42:06.375 INFO:teuthology.orchestra.run.smithi047.stdout:/tmp/tmp.1RRFUrfgkG 2022-06-29T15:42:06.376 DEBUG:teuthology.orchestra.run.smithi047:> sudo cp /var/lib/ceph/osd/ceph-2/keyring /tmp/tmp.1RRFUrfgkG 2022-06-29T15:42:06.393 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0666 /tmp/tmp.1RRFUrfgkG 2022-06-29T15:42:06.457 DEBUG:teuthology.orchestra.remote:smithi047:/tmp/tmp.1RRFUrfgkG is 56B 2022-06-29T15:42:06.472 DEBUG:teuthology.orchestra.run.smithi047:> rm -fr /tmp/tmp.1RRFUrfgkG 2022-06-29T15:42:06.479 DEBUG:teuthology.orchestra.run.smithi047:> mktemp 2022-06-29T15:42:06.527 INFO:teuthology.orchestra.run.smithi047.stdout:/tmp/tmp.31rxTyTzNQ 2022-06-29T15:42:06.528 DEBUG:teuthology.orchestra.run.smithi047:> sudo cp /var/lib/ceph/osd/ceph-3/keyring /tmp/tmp.31rxTyTzNQ 2022-06-29T15:42:06.544 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0666 /tmp/tmp.31rxTyTzNQ 2022-06-29T15:42:06.609 DEBUG:teuthology.orchestra.remote:smithi047:/tmp/tmp.31rxTyTzNQ is 56B 2022-06-29T15:42:06.624 DEBUG:teuthology.orchestra.run.smithi047:> rm -fr /tmp/tmp.31rxTyTzNQ 2022-06-29T15:42:06.631 DEBUG:teuthology.orchestra.run.smithi064:> mktemp 2022-06-29T15:42:06.638 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/tmp.1lhM4dyVVK 2022-06-29T15:42:06.638 DEBUG:teuthology.orchestra.run.smithi064:> sudo cp /var/lib/ceph/mgr/ceph-x/keyring /tmp/tmp.1lhM4dyVVK 2022-06-29T15:42:06.696 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0666 /tmp/tmp.1lhM4dyVVK 2022-06-29T15:42:06.759 DEBUG:teuthology.orchestra.remote:smithi064:/tmp/tmp.1lhM4dyVVK is 56B 2022-06-29T15:42:06.775 DEBUG:teuthology.orchestra.run.smithi064:> rm -fr /tmp/tmp.1lhM4dyVVK 2022-06-29T15:42:06.782 DEBUG:teuthology.orchestra.run.smithi064:> mktemp 2022-06-29T15:42:06.830 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/tmp.fqXsKTiUaH 2022-06-29T15:42:06.830 DEBUG:teuthology.orchestra.run.smithi064:> sudo cp /var/lib/ceph/mds/ceph-b/keyring /tmp/tmp.fqXsKTiUaH 2022-06-29T15:42:06.846 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0666 /tmp/tmp.fqXsKTiUaH 2022-06-29T15:42:06.870 DEBUG:teuthology.orchestra.remote:smithi064:/tmp/tmp.fqXsKTiUaH is 56B 2022-06-29T15:42:06.882 DEBUG:teuthology.orchestra.run.smithi064:> rm -fr /tmp/tmp.fqXsKTiUaH 2022-06-29T15:42:06.889 DEBUG:teuthology.orchestra.run.smithi064:> mktemp 2022-06-29T15:42:06.938 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/tmp.X3d6C0rVxI 2022-06-29T15:42:06.938 DEBUG:teuthology.orchestra.run.smithi064:> sudo cp /var/lib/ceph/osd/ceph-4/keyring /tmp/tmp.X3d6C0rVxI 2022-06-29T15:42:06.956 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0666 /tmp/tmp.X3d6C0rVxI 2022-06-29T15:42:06.977 DEBUG:teuthology.orchestra.remote:smithi064:/tmp/tmp.X3d6C0rVxI is 56B 2022-06-29T15:42:07.030 DEBUG:teuthology.orchestra.run.smithi064:> rm -fr /tmp/tmp.X3d6C0rVxI 2022-06-29T15:42:07.037 DEBUG:teuthology.orchestra.run.smithi064:> mktemp 2022-06-29T15:42:07.044 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/tmp.GhkR9kiuom 2022-06-29T15:42:07.045 DEBUG:teuthology.orchestra.run.smithi064:> sudo cp /var/lib/ceph/osd/ceph-5/keyring /tmp/tmp.GhkR9kiuom 2022-06-29T15:42:07.104 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0666 /tmp/tmp.GhkR9kiuom 2022-06-29T15:42:07.171 DEBUG:teuthology.orchestra.remote:smithi064:/tmp/tmp.GhkR9kiuom is 56B 2022-06-29T15:42:07.187 DEBUG:teuthology.orchestra.run.smithi064:> rm -fr /tmp/tmp.GhkR9kiuom 2022-06-29T15:42:07.194 DEBUG:teuthology.orchestra.run.smithi064:> mktemp 2022-06-29T15:42:07.246 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/tmp.XD9L177qTE 2022-06-29T15:42:07.246 DEBUG:teuthology.orchestra.run.smithi064:> sudo cp /var/lib/ceph/osd/ceph-6/keyring /tmp/tmp.XD9L177qTE 2022-06-29T15:42:07.261 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0666 /tmp/tmp.XD9L177qTE 2022-06-29T15:42:07.281 DEBUG:teuthology.orchestra.remote:smithi064:/tmp/tmp.XD9L177qTE is 56B 2022-06-29T15:42:07.296 DEBUG:teuthology.orchestra.run.smithi064:> rm -fr /tmp/tmp.XD9L177qTE 2022-06-29T15:42:07.303 DEBUG:teuthology.orchestra.run.smithi064:> mktemp 2022-06-29T15:42:07.354 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/tmp.rsX6JDa2Z9 2022-06-29T15:42:07.355 DEBUG:teuthology.orchestra.run.smithi064:> sudo cp /var/lib/ceph/osd/ceph-7/keyring /tmp/tmp.rsX6JDa2Z9 2022-06-29T15:42:07.372 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0666 /tmp/tmp.rsX6JDa2Z9 2022-06-29T15:42:07.435 DEBUG:teuthology.orchestra.remote:smithi064:/tmp/tmp.rsX6JDa2Z9 is 56B 2022-06-29T15:42:07.451 DEBUG:teuthology.orchestra.run.smithi064:> rm -fr /tmp/tmp.rsX6JDa2Z9 2022-06-29T15:42:07.476 DEBUG:teuthology.orchestra.remote:smithi099:/etc/ceph/ceph.client.0.keyring is 59B 2022-06-29T15:42:07.505 DEBUG:teuthology.orchestra.remote:smithi164:/etc/ceph/ceph.client.1.keyring is 59B 2022-06-29T15:42:07.521 INFO:tasks.ceph:Adding keys to all mons... 2022-06-29T15:42:07.521 DEBUG:teuthology.orchestra.run.smithi047:> sudo tee -a /etc/ceph/ceph.keyring 2022-06-29T15:42:07.524 DEBUG:teuthology.orchestra.run.smithi064:> sudo tee -a /etc/ceph/ceph.keyring 2022-06-29T15:42:07.537 INFO:teuthology.orchestra.run.smithi047.stdout:[mgr.y] 2022-06-29T15:42:07.537 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQC1crxisSMFLBAAv03r4OoPjOP9nI9G/LK0iw== 2022-06-29T15:42:07.538 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.a] 2022-06-29T15:42:07.538 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQC1crxiQJFFMxAA9Tv7FJD9u/BKvbkuKld6Kg== 2022-06-29T15:42:07.538 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.c] 2022-06-29T15:42:07.538 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQC1crxi/4ZMORAANpLbgTm3ehqi80USc3swGg== 2022-06-29T15:42:07.538 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.0] 2022-06-29T15:42:07.539 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQC2crxiY0YsMxAAZivb+CxL/w7vZwQv37Y4oQ== 2022-06-29T15:42:07.539 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.1] 2022-06-29T15:42:07.539 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQC5crxitLkKFxAA8+++LUUKLiO9SAuAfYgT5g== 2022-06-29T15:42:07.539 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.2] 2022-06-29T15:42:07.540 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQC7crxi1oM9NxAA2FfLdNt32YPwlyNdGQpaQA== 2022-06-29T15:42:07.540 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.3] 2022-06-29T15:42:07.540 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQC+crxiTsBgGhAA+gLBBDaskPbmRNWzgKwcHA== 2022-06-29T15:42:07.540 INFO:teuthology.orchestra.run.smithi047.stdout:[mgr.x] 2022-06-29T15:42:07.540 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQC1crxidmuIMBAAcpNlYVu5+icCHg+yIjcKXw== 2022-06-29T15:42:07.541 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.b] 2022-06-29T15:42:07.541 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQC2crxibt6FARAAEf2bwXKFhXKtqr1nN0rdXQ== 2022-06-29T15:42:07.541 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.4] 2022-06-29T15:42:07.541 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQDDcrxivJebIhAAfwWJa7tK4hamkk1/1CSSfw== 2022-06-29T15:42:07.542 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.5] 2022-06-29T15:42:07.542 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQDGcrxijuozBhAAt+Dx3wH3WKp3DrbCbhmh8Q== 2022-06-29T15:42:07.542 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.6] 2022-06-29T15:42:07.542 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQDIcrxi91kXJxAA9JX7n98P394bD41Q9LUdvQ== 2022-06-29T15:42:07.542 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.7] 2022-06-29T15:42:07.543 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQDLcrxiHj0ZDRAASy6fAix8TmEZbAGtuFCfFg== 2022-06-29T15:42:07.543 INFO:teuthology.orchestra.run.smithi047.stdout:[client.0] 2022-06-29T15:42:07.543 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQC2crxiVDlfBxAAezkUfg4e2g0Rfakaa/Ek1Q== 2022-06-29T15:42:07.543 INFO:teuthology.orchestra.run.smithi047.stdout:[client.1] 2022-06-29T15:42:07.544 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQC2crxisXJ1CxAAhl07oF7AW66NdzfvvZTJmA== 2022-06-29T15:42:07.545 INFO:teuthology.orchestra.run.smithi064.stdout:[mgr.y] 2022-06-29T15:42:07.546 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQC1crxisSMFLBAAv03r4OoPjOP9nI9G/LK0iw== 2022-06-29T15:42:07.546 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.a] 2022-06-29T15:42:07.546 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQC1crxiQJFFMxAA9Tv7FJD9u/BKvbkuKld6Kg== 2022-06-29T15:42:07.546 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.c] 2022-06-29T15:42:07.547 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQC1crxi/4ZMORAANpLbgTm3ehqi80USc3swGg== 2022-06-29T15:42:07.547 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.0] 2022-06-29T15:42:07.547 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQC2crxiY0YsMxAAZivb+CxL/w7vZwQv37Y4oQ== 2022-06-29T15:42:07.547 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.1] 2022-06-29T15:42:07.548 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQC5crxitLkKFxAA8+++LUUKLiO9SAuAfYgT5g== 2022-06-29T15:42:07.548 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.2] 2022-06-29T15:42:07.548 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQC7crxi1oM9NxAA2FfLdNt32YPwlyNdGQpaQA== 2022-06-29T15:42:07.548 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.3] 2022-06-29T15:42:07.548 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQC+crxiTsBgGhAA+gLBBDaskPbmRNWzgKwcHA== 2022-06-29T15:42:07.549 INFO:teuthology.orchestra.run.smithi064.stdout:[mgr.x] 2022-06-29T15:42:07.549 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQC1crxidmuIMBAAcpNlYVu5+icCHg+yIjcKXw== 2022-06-29T15:42:07.549 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.b] 2022-06-29T15:42:07.549 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQC2crxibt6FARAAEf2bwXKFhXKtqr1nN0rdXQ== 2022-06-29T15:42:07.550 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.4] 2022-06-29T15:42:07.550 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQDDcrxivJebIhAAfwWJa7tK4hamkk1/1CSSfw== 2022-06-29T15:42:07.550 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.5] 2022-06-29T15:42:07.550 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQDGcrxijuozBhAAt+Dx3wH3WKp3DrbCbhmh8Q== 2022-06-29T15:42:07.550 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.6] 2022-06-29T15:42:07.551 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQDIcrxi91kXJxAA9JX7n98P394bD41Q9LUdvQ== 2022-06-29T15:42:07.551 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.7] 2022-06-29T15:42:07.551 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQDLcrxiHj0ZDRAASy6fAix8TmEZbAGtuFCfFg== 2022-06-29T15:42:07.551 INFO:teuthology.orchestra.run.smithi064.stdout:[client.0] 2022-06-29T15:42:07.552 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQC2crxiVDlfBxAAezkUfg4e2g0Rfakaa/Ek1Q== 2022-06-29T15:42:07.552 INFO:teuthology.orchestra.run.smithi064.stdout:[client.1] 2022-06-29T15:42:07.552 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQC2crxisXJ1CxAAhl07oF7AW66NdzfvvZTJmA== 2022-06-29T15:42:07.553 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-06-29T15:42:07.555 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-06-29T15:42:07.595 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-06-29T15:42:07.639 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-06-29T15:42:07.688 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-06-29T15:42:07.691 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-06-29T15:42:07.735 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-06-29T15:42:07.738 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-06-29T15:42:07.778 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-06-29T15:42:07.823 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-06-29T15:42:07.870 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-06-29T15:42:07.873 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-06-29T15:42:07.918 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-06-29T15:42:07.960 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-06-29T15:42:08.004 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-06-29T15:42:08.007 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-06-29T15:42:08.089 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-06-29T15:42:08.092 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-06-29T15:42:08.138 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-06-29T15:42:08.141 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-06-29T15:42:08.183 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-06-29T15:42:08.224 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-06-29T15:42:08.269 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-06-29T15:42:08.273 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-06-29T15:42:08.317 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-06-29T15:42:08.355 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-06-29T15:42:08.400 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-06-29T15:42:08.403 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-06-29T15:42:08.446 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-06-29T15:42:08.449 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-06-29T15:42:08.495 INFO:tasks.ceph:Running mkfs on mon nodes... 2022-06-29T15:42:08.496 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2022-06-29T15:42:08.508 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2022-06-29T15:42:08.607 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2022-06-29T15:42:08.620 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2022-06-29T15:42:08.635 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2022-06-29T15:42:08.707 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2022-06-29T15:42:08.724 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2022-06-29T15:42:08.783 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2022-06-29T15:42:08.840 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2022-06-29T15:42:08.855 DEBUG:teuthology.orchestra.run.smithi047:> rm -- /home/ubuntu/cephtest/ceph.monmap 2022-06-29T15:42:08.858 DEBUG:teuthology.orchestra.run.smithi064:> rm -- /home/ubuntu/cephtest/ceph.monmap 2022-06-29T15:42:08.902 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2022-06-29T15:42:08.902 INFO:tasks.ceph.mon.a:Restarting daemon 2022-06-29T15:42:08.903 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2022-06-29T15:42:08.907 INFO:tasks.ceph.mon.a:Started 2022-06-29T15:42:08.907 INFO:tasks.ceph.mon.b:Restarting daemon 2022-06-29T15:42:08.908 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2022-06-29T15:42:08.910 INFO:tasks.ceph.mon.b:Started 2022-06-29T15:42:08.911 INFO:tasks.ceph.mon.c:Restarting daemon 2022-06-29T15:42:08.911 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2022-06-29T15:42:08.914 INFO:tasks.ceph.mon.c:Started 2022-06-29T15:42:08.914 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2022-06-29T15:42:08.915 INFO:tasks.ceph.mgr.y:Restarting daemon 2022-06-29T15:42:08.915 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2022-06-29T15:42:08.918 INFO:tasks.ceph.mgr.y:Started 2022-06-29T15:42:08.919 INFO:tasks.ceph.mgr.x:Restarting daemon 2022-06-29T15:42:08.920 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2022-06-29T15:42:08.923 INFO:tasks.ceph.mgr.x:Started 2022-06-29T15:42:08.924 INFO:tasks.ceph:Setting crush tunables to default 2022-06-29T15:42:08.924 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --cluster ceph osd crush tunables default 2022-06-29T15:42:09.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:09.016+0000 7f807b724700 1 Processor -- start 2022-06-29T15:42:09.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:09.016+0000 7f807b724700 1 -- start start 2022-06-29T15:42:09.022 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:09.016+0000 7f807b724700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f80740608a0 0x7f80741403b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:09.022 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:09.016+0000 7f807b724700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f80740b5e20 0x7f80741408a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:09.022 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:09.016+0000 7f807b724700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8074060c80 con 0x7f80740a94f0 2022-06-29T15:42:09.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:09.016+0000 7f807b724700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8074143120 con 0x7f80740b5e20 2022-06-29T15:42:09.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:09.016+0000 7f807b724700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80740a8d80 con 0x7f80740608a0 2022-06-29T15:42:09.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:09.016+0000 7f8078cbf700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f80740b5e20 0x7f80741408a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:09.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:09.016+0000 7f8079cc1700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f80740a94f0 0x7f8074143510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:58838/0 (socket says 172.21.15.47:58838) 2022-06-29T15:42:09.024 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:09.016+0000 7f8079cc1700 1 -- 172.21.15.47:0/336997049 learned_addr learned my addr 172.21.15.47:0/336997049 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:09.024 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:09.020+0000 7f8078cbf700 1 -- 172.21.15.47:0/336997049 >> v2:172.21.15.47:3300/0 conn(0x7f80740b5e20 msgr2=0x7f80741408a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2022-06-29T15:42:09.024 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:09.020+0000 7f80794c0700 1 --2- 172.21.15.47:0/336997049 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f80740608a0 0x7f80741403b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:09.025 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:09.020+0000 7f8078cbf700 1 -- 172.21.15.47:0/336997049 >> v2:172.21.15.47:3300/0 conn(0x7f80740b5e20 msgr2=0x7f80741408a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-06-29T15:42:09.025 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:09.020+0000 7f8078cbf700 1 --2- 172.21.15.47:0/336997049 >> v2:172.21.15.47:3300/0 conn(0x7f80740b5e20 0x7f80741408a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-29T15:42:09.025 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:09.020+0000 7f8078cbf700 1 --2- 172.21.15.47:0/336997049 >> v2:172.21.15.47:3300/0 conn(0x7f80740b5e20 0x7f80741408a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2022-06-29T15:42:09.025 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:09.020+0000 7f80794c0700 1 -- 172.21.15.47:0/336997049 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f80740608a0 msgr2=0x7f80741403b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-06-29T15:42:09.026 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:09.020+0000 7f80794c0700 1 -- 172.21.15.47:0/336997049 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f80740608a0 msgr2=0x7f80741403b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-06-29T15:42:09.026 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:09.020+0000 7f80794c0700 1 --2- 172.21.15.47:0/336997049 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f80740608a0 0x7f80741403b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-29T15:42:09.026 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:09.020+0000 7f80794c0700 1 --2- 172.21.15.47:0/336997049 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f80740608a0 0x7f80741403b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2022-06-29T15:42:09.226 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:09.224+0000 7f8078cbf700 1 --2- 172.21.15.47:0/336997049 >> v2:172.21.15.47:3300/0 conn(0x7f80740b5e20 0x7f80741408a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:09.227 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:09.224+0000 7f80794c0700 1 --2- 172.21.15.47:0/336997049 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f80740608a0 0x7f80741403b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:09.227 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:09.224+0000 7f8078cbf700 1 -- 172.21.15.47:0/336997049 >> v2:172.21.15.47:3300/0 conn(0x7f80740b5e20 msgr2=0x7f80741408a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-06-29T15:42:09.228 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:09.224+0000 7f8078cbf700 1 -- 172.21.15.47:0/336997049 >> v2:172.21.15.47:3300/0 conn(0x7f80740b5e20 msgr2=0x7f80741408a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-06-29T15:42:09.228 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:09.224+0000 7f8078cbf700 1 --2- 172.21.15.47:0/336997049 >> v2:172.21.15.47:3300/0 conn(0x7f80740b5e20 0x7f80741408a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-29T15:42:09.228 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:09.224+0000 7f8078cbf700 1 --2- 172.21.15.47:0/336997049 >> v2:172.21.15.47:3300/0 conn(0x7f80740b5e20 0x7f80741408a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2022-06-29T15:42:09.229 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:09.224+0000 7f80794c0700 1 -- 172.21.15.47:0/336997049 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f80740608a0 msgr2=0x7f80741403b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2022-06-29T15:42:09.229 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:09.224+0000 7f80794c0700 1 -- 172.21.15.47:0/336997049 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f80740608a0 msgr2=0x7f80741403b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-06-29T15:42:09.229 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:09.224+0000 7f80794c0700 1 --2- 172.21.15.47:0/336997049 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f80740608a0 0x7f80741403b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-29T15:42:09.229 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:09.224+0000 7f80794c0700 1 --2- 172.21.15.47:0/336997049 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f80740608a0 0x7f80741403b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2022-06-29T15:42:09.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:09.628+0000 7f8078cbf700 1 --2- 172.21.15.47:0/336997049 >> v2:172.21.15.47:3300/0 conn(0x7f80740b5e20 0x7f80741408a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:09.631 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:09.628+0000 7f80794c0700 1 --2- 172.21.15.47:0/336997049 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f80740608a0 0x7f80741403b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:09.631 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:09.628+0000 7f8078cbf700 1 -- 172.21.15.47:0/336997049 >> v2:172.21.15.47:3300/0 conn(0x7f80740b5e20 msgr2=0x7f80741408a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2022-06-29T15:42:09.631 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:09.628+0000 7f8078cbf700 1 -- 172.21.15.47:0/336997049 >> v2:172.21.15.47:3300/0 conn(0x7f80740b5e20 msgr2=0x7f80741408a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-06-29T15:42:09.632 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:09.628+0000 7f8078cbf700 1 --2- 172.21.15.47:0/336997049 >> v2:172.21.15.47:3300/0 conn(0x7f80740b5e20 0x7f80741408a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-29T15:42:09.632 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:09.628+0000 7f8078cbf700 1 --2- 172.21.15.47:0/336997049 >> v2:172.21.15.47:3300/0 conn(0x7f80740b5e20 0x7f80741408a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2022-06-29T15:42:09.632 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:09.628+0000 7f80794c0700 1 -- 172.21.15.47:0/336997049 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f80740608a0 msgr2=0x7f80741403b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-06-29T15:42:09.632 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:09.628+0000 7f80794c0700 1 -- 172.21.15.47:0/336997049 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f80740608a0 msgr2=0x7f80741403b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-06-29T15:42:09.633 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:09.628+0000 7f80794c0700 1 --2- 172.21.15.47:0/336997049 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f80740608a0 0x7f80741403b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-29T15:42:09.633 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:09.628+0000 7f80794c0700 1 --2- 172.21.15.47:0/336997049 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f80740608a0 0x7f80741403b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2022-06-29T15:42:10.434 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:10.432+0000 7f8078cbf700 1 --2- 172.21.15.47:0/336997049 >> v2:172.21.15.47:3300/0 conn(0x7f80740b5e20 0x7f80741408a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:10.435 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:10.432+0000 7f80794c0700 1 --2- 172.21.15.47:0/336997049 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f80740608a0 0x7f80741403b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:10.435 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:10.432+0000 7f8078cbf700 1 -- 172.21.15.47:0/336997049 >> v2:172.21.15.47:3300/0 conn(0x7f80740b5e20 msgr2=0x7f80741408a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2022-06-29T15:42:10.435 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:10.432+0000 7f8078cbf700 1 -- 172.21.15.47:0/336997049 >> v2:172.21.15.47:3300/0 conn(0x7f80740b5e20 msgr2=0x7f80741408a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-06-29T15:42:10.436 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:10.432+0000 7f8078cbf700 1 --2- 172.21.15.47:0/336997049 >> v2:172.21.15.47:3300/0 conn(0x7f80740b5e20 0x7f80741408a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-29T15:42:10.436 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:10.432+0000 7f8078cbf700 1 --2- 172.21.15.47:0/336997049 >> v2:172.21.15.47:3300/0 conn(0x7f80740b5e20 0x7f80741408a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2022-06-29T15:42:10.436 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:10.432+0000 7f80794c0700 1 -- 172.21.15.47:0/336997049 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f80740608a0 msgr2=0x7f80741403b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-06-29T15:42:10.437 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:10.432+0000 7f80794c0700 1 -- 172.21.15.47:0/336997049 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f80740608a0 msgr2=0x7f80741403b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-06-29T15:42:10.437 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:10.432+0000 7f80794c0700 1 --2- 172.21.15.47:0/336997049 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f80740608a0 0x7f80741403b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-29T15:42:10.437 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:10.432+0000 7f80794c0700 1 --2- 172.21.15.47:0/336997049 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f80740608a0 0x7f80741403b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2022-06-29T15:42:10.645 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:42:10.649 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:42:10.652 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:42:10.655 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:42:12.022 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:12.016+0000 7f8072ffd700 1 -- 172.21.15.47:0/336997049 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f80740608a0 msgr2=0x7f80741403b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-06-29T15:42:12.022 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:12.016+0000 7f8072ffd700 1 --2- 172.21.15.47:0/336997049 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f80740608a0 0x7f80741403b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:12.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:12.016+0000 7f8072ffd700 1 -- 172.21.15.47:0/336997049 >> v2:172.21.15.47:3300/0 conn(0x7f80740b5e20 msgr2=0x7f80741408a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-06-29T15:42:12.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:12.016+0000 7f8072ffd700 1 --2- 172.21.15.47:0/336997049 >> v2:172.21.15.47:3300/0 conn(0x7f80740b5e20 0x7f80741408a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:12.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:12.016+0000 7f8072ffd700 1 -- 172.21.15.47:0/336997049 >> v1:172.21.15.47:6789/0 conn(0x7f80740a94f0 legacy=0x7f8074143510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:12.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:12.016+0000 7f8072ffd700 1 --2- 172.21.15.47:0/336997049 >> v2:172.21.15.64:3300/0 conn(0x7f80540033b0 0x7f8054005830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:12.024 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:12.020+0000 7f8072ffd700 1 -- 172.21.15.47:0/336997049 --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80740a9020 con 0x7f8054005e30 2022-06-29T15:42:12.024 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:12.020+0000 7f8072ffd700 1 -- 172.21.15.47:0/336997049 --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80740a8d80 con 0x7f80540033b0 2022-06-29T15:42:12.024 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:12.020+0000 7f80794c0700 1 --2- 172.21.15.47:0/336997049 >> v2:172.21.15.64:3300/0 conn(0x7f80540033b0 0x7f8054005830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:12.024 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:12.020+0000 7f80794c0700 1 -- 172.21.15.47:0/336997049 >> v2:172.21.15.64:3300/0 conn(0x7f80540033b0 msgr2=0x7f8054005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-06-29T15:42:12.025 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:12.020+0000 7f80794c0700 1 -- 172.21.15.47:0/336997049 >> v2:172.21.15.64:3300/0 conn(0x7f80540033b0 msgr2=0x7f8054005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-06-29T15:42:12.025 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:12.020+0000 7f80794c0700 1 --2- 172.21.15.47:0/336997049 >> v2:172.21.15.64:3300/0 conn(0x7f80540033b0 0x7f8054005830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-29T15:42:12.025 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:12.020+0000 7f80794c0700 1 --2- 172.21.15.47:0/336997049 >> v2:172.21.15.64:3300/0 conn(0x7f80540033b0 0x7f8054005830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2022-06-29T15:42:12.226 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:12.224+0000 7f80794c0700 1 --2- 172.21.15.47:0/336997049 >> v2:172.21.15.64:3300/0 conn(0x7f80540033b0 0x7f8054005830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:12.226 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:12.224+0000 7f80794c0700 1 -- 172.21.15.47:0/336997049 >> v2:172.21.15.64:3300/0 conn(0x7f80540033b0 msgr2=0x7f8054005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-06-29T15:42:12.227 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:12.224+0000 7f80794c0700 1 -- 172.21.15.47:0/336997049 >> v2:172.21.15.64:3300/0 conn(0x7f80540033b0 msgr2=0x7f8054005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-06-29T15:42:12.227 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:12.224+0000 7f80794c0700 1 --2- 172.21.15.47:0/336997049 >> v2:172.21.15.64:3300/0 conn(0x7f80540033b0 0x7f8054005830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-29T15:42:12.227 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:12.224+0000 7f80794c0700 1 --2- 172.21.15.47:0/336997049 >> v2:172.21.15.64:3300/0 conn(0x7f80540033b0 0x7f8054005830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2022-06-29T15:42:12.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:12.628+0000 7f80794c0700 1 --2- 172.21.15.47:0/336997049 >> v2:172.21.15.64:3300/0 conn(0x7f80540033b0 0x7f8054005830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:12.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:12.628+0000 7f80794c0700 1 -- 172.21.15.47:0/336997049 >> v2:172.21.15.64:3300/0 conn(0x7f80540033b0 msgr2=0x7f8054005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-06-29T15:42:12.631 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:12.628+0000 7f80794c0700 1 -- 172.21.15.47:0/336997049 >> v2:172.21.15.64:3300/0 conn(0x7f80540033b0 msgr2=0x7f8054005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-06-29T15:42:12.631 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:12.628+0000 7f80794c0700 1 --2- 172.21.15.47:0/336997049 >> v2:172.21.15.64:3300/0 conn(0x7f80540033b0 0x7f8054005830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-29T15:42:12.631 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:12.628+0000 7f80794c0700 1 --2- 172.21.15.47:0/336997049 >> v2:172.21.15.64:3300/0 conn(0x7f80540033b0 0x7f8054005830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2022-06-29T15:42:13.434 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:13.432+0000 7f80794c0700 1 --2- 172.21.15.47:0/336997049 >> v2:172.21.15.64:3300/0 conn(0x7f80540033b0 0x7f8054005830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:13.435 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:13.432+0000 7f80794c0700 1 -- 172.21.15.47:0/336997049 >> v2:172.21.15.64:3300/0 conn(0x7f80540033b0 msgr2=0x7f8054005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-06-29T15:42:13.435 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:13.432+0000 7f80794c0700 1 -- 172.21.15.47:0/336997049 >> v2:172.21.15.64:3300/0 conn(0x7f80540033b0 msgr2=0x7f8054005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-06-29T15:42:13.435 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:13.432+0000 7f80794c0700 1 --2- 172.21.15.47:0/336997049 >> v2:172.21.15.64:3300/0 conn(0x7f80540033b0 0x7f8054005830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-29T15:42:13.435 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:13.432+0000 7f80794c0700 1 --2- 172.21.15.47:0/336997049 >> v2:172.21.15.64:3300/0 conn(0x7f80540033b0 0x7f8054005830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2022-06-29T15:42:14.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.068+0000 7f8073fff700 1 -- 172.21.15.47:0/336997049 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 539627987 0 0) 0x7f80740a9020 con 0x7f8054005e30 2022-06-29T15:42:14.074 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.072+0000 7f8073fff700 1 -- 172.21.15.47:0/336997049 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8060003260 con 0x7f8054005e30 2022-06-29T15:42:14.080 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.076+0000 7f8073fff700 1 -- 172.21.15.47:0/336997049 <== mon.1 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1274724881 0 0) 0x7f8060003260 con 0x7f8054005e30 2022-06-29T15:42:14.080 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.076+0000 7f8073fff700 1 -- 172.21.15.47:0/336997049 >> v2:172.21.15.64:3300/0 conn(0x7f80540033b0 msgr2=0x7f8054005830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-06-29T15:42:14.081 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.076+0000 7f8073fff700 1 --2- 172.21.15.47:0/336997049 >> v2:172.21.15.64:3300/0 conn(0x7f80540033b0 0x7f8054005830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:14.081 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.076+0000 7f8073fff700 1 -- 172.21.15.47:0/336997049 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f805400ac10 con 0x7f8054005e30 2022-06-29T15:42:14.081 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.076+0000 7f8073fff700 1 -- 172.21.15.47:0/336997049 <== mon.1 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7f806400c4e0 con 0x7f8054005e30 2022-06-29T15:42:14.081 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.076+0000 7f8073fff700 1 -- 172.21.15.47:0/336997049 >> v1:172.21.15.64:6789/0 conn(0x7f8054005e30 legacy=0x7f80540082d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:14.082 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.076+0000 7f8079cc1700 1 -- 172.21.15.47:0/336997049 reap_dead start 2022-06-29T15:42:14.082 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.076+0000 7f8073fff700 1 --2- 172.21.15.47:0/336997049 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8060003750 0x7f8060003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:14.082 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.076+0000 7f8073fff700 1 --2- 172.21.15.47:0/336997049 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8060004500 0x7f8060004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:14.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.076+0000 7f8073fff700 1 --2- 172.21.15.47:0/336997049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8060004ec0 0x7f806000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:14.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.076+0000 7f8073fff700 1 -- 172.21.15.47:0/336997049 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80740a8d80 con 0x7f8060003750 2022-06-29T15:42:14.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.076+0000 7f8073fff700 1 -- 172.21.15.47:0/336997049 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806000bd80 con 0x7f8060004ec0 2022-06-29T15:42:14.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.076+0000 7f8073fff700 1 -- 172.21.15.47:0/336997049 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f806000bf40 con 0x7f8060004500 2022-06-29T15:42:14.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.076+0000 7f80794c0700 1 --2- 172.21.15.47:0/336997049 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8060003750 0x7f8060003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:14.084 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.076+0000 7f8079cc1700 1 --2- 172.21.15.47:0/336997049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8060004ec0 0x7f806000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:14.084 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.076+0000 7f8078cbf700 1 --2- 172.21.15.47:0/336997049 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8060004500 0x7f8060004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:14.084 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.076+0000 7f8078cbf700 1 -- 172.21.15.47:0/336997049 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8060004500 msgr2=0x7f8060004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2022-06-29T15:42:14.084 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.076+0000 7f8078cbf700 1 -- 172.21.15.47:0/336997049 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8060004500 msgr2=0x7f8060004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-06-29T15:42:14.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.076+0000 7f80794c0700 1 -- 172.21.15.47:0/336997049 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8060004500 msgr2=0x7f8060004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:14.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.076+0000 7f8078cbf700 1 --2- 172.21.15.47:0/336997049 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8060004500 0x7f8060004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-29T15:42:14.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.076+0000 7f8078cbf700 1 --2- 172.21.15.47:0/336997049 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8060004500 0x7f8060004980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2022-06-29T15:42:14.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.076+0000 7f80794c0700 1 --2- 172.21.15.47:0/336997049 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8060004500 0x7f8060004980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:14.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.076+0000 7f80794c0700 1 -- 172.21.15.47:0/336997049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8060004ec0 msgr2=0x7f806000b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:14.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.076+0000 7f80794c0700 1 --2- 172.21.15.47:0/336997049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8060004ec0 0x7f806000b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:14.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.076+0000 7f80794c0700 1 -- 172.21.15.47:0/336997049 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8060003260 con 0x7f8060003750 2022-06-29T15:42:14.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.080+0000 7f80794c0700 1 --2- 172.21.15.47:0/336997049 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8060003750 0x7f8060003b50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f806c00a000 tx=0x7f806c008040).ready entity=mon.0 client_cookie=fbf76a8df10f4ba3 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:14.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.080+0000 7f8073fff700 1 -- 172.21.15.47:0/336997049 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f806c01e070 con 0x7f8060003750 2022-06-29T15:42:14.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.080+0000 7f8073fff700 1 -- 172.21.15.47:0/336997049 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f806c019330 con 0x7f8060003750 2022-06-29T15:42:14.089 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.080+0000 7f8073fff700 1 -- 172.21.15.47:0/336997049 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f806c017aa0 con 0x7f8060003750 2022-06-29T15:42:14.089 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.084+0000 7f807b724700 1 -- 172.21.15.47:0/336997049 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8060003750 msgr2=0x7f8060003b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:14.089 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.084+0000 7f807b724700 1 --2- 172.21.15.47:0/336997049 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8060003750 0x7f8060003b50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f806c00a000 tx=0x7f806c008040).stop 2022-06-29T15:42:14.089 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.084+0000 7f807b724700 1 -- 172.21.15.47:0/336997049 shutdown_connections 2022-06-29T15:42:14.090 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.084+0000 7f807b724700 1 --2- 172.21.15.47:0/336997049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8060004ec0 0x7f806000b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:14.090 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.084+0000 7f807b724700 1 --2- 172.21.15.47:0/336997049 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8060004500 0x7f8060004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:14.090 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.084+0000 7f807b724700 1 --2- 172.21.15.47:0/336997049 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8060003750 0x7f8060003b50 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:14.090 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.084+0000 7f807b724700 1 -- 172.21.15.47:0/336997049 >> 172.21.15.47:0/336997049 conn(0x7f80740626f0 msgr2=0x7f8074062ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:14.091 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.084+0000 7f807b724700 1 -- 172.21.15.47:0/336997049 shutdown_connections 2022-06-29T15:42:14.091 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.084+0000 7f807b724700 1 -- 172.21.15.47:0/336997049 wait complete. 2022-06-29T15:42:14.091 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.084+0000 7f807b724700 1 Processor -- start 2022-06-29T15:42:14.094 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.084+0000 7f807b724700 1 -- start start 2022-06-29T15:42:14.094 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.084+0000 7f807b724700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8060003750 0x7f807413ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:14.095 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.084+0000 7f807b724700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8060004500 0x7f80740b3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:14.095 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.084+0000 7f807b724700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8060004ec0 0x7f80740aab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:14.095 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.084+0000 7f807b724700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807414a4e0 con 0x7f8060004ec0 2022-06-29T15:42:14.095 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.084+0000 7f807b724700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807413dfc0 con 0x7f8060003750 2022-06-29T15:42:14.096 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.084+0000 7f807b724700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80740b5480 con 0x7f8060004500 2022-06-29T15:42:14.096 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.084+0000 7f8079cc1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8060004ec0 0x7f80740aab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:14.096 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.084+0000 7f80794c0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8060003750 0x7f807413ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:14.096 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.084+0000 7f8079cc1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8060004ec0 0x7f80740aab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45120/0 (socket says 172.21.15.47:45120) 2022-06-29T15:42:14.097 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.084+0000 7f8079cc1700 1 -- 172.21.15.47:0/2121654795 learned_addr learned my addr 172.21.15.47:0/2121654795 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:14.097 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.084+0000 7f8078cbf700 1 --2- 172.21.15.47:0/2121654795 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8060004500 0x7f80740b3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:14.097 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.088+0000 7f8078cbf700 1 -- 172.21.15.47:0/2121654795 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8060004500 msgr2=0x7f80740b3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2022-06-29T15:42:14.097 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.088+0000 7f8078cbf700 1 -- 172.21.15.47:0/2121654795 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8060004500 msgr2=0x7f80740b3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2022-06-29T15:42:14.098 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.088+0000 7f8078cbf700 1 --2- 172.21.15.47:0/2121654795 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8060004500 0x7f80740b3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-29T15:42:14.098 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.088+0000 7f8078cbf700 1 --2- 172.21.15.47:0/2121654795 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8060004500 0x7f80740b3430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2022-06-29T15:42:14.098 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.088+0000 7f8079cc1700 1 -- 172.21.15.47:0/2121654795 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8060004500 msgr2=0x7f80740b3430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2022-06-29T15:42:14.098 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.088+0000 7f8079cc1700 1 --2- 172.21.15.47:0/2121654795 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8060004500 0x7f80740b3430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:14.098 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.088+0000 7f8079cc1700 1 -- 172.21.15.47:0/2121654795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8060003750 msgr2=0x7f807413ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:14.099 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.088+0000 7f8079cc1700 1 --2- 172.21.15.47:0/2121654795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8060003750 0x7f807413ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:14.099 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.088+0000 7f8079cc1700 1 -- 172.21.15.47:0/2121654795 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80740b5720 con 0x7f8060004ec0 2022-06-29T15:42:14.099 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.088+0000 7f8079cc1700 1 --2- 172.21.15.47:0/2121654795 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8060004ec0 0x7f80740aab80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f80740b29c0 tx=0x7f8068009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:14.099 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.088+0000 7f80727fc700 1 -- 172.21.15.47:0/2121654795 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8068007030 con 0x7f8060004ec0 2022-06-29T15:42:14.100 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.088+0000 7f80727fc700 1 -- 172.21.15.47:0/2121654795 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8068007190 con 0x7f8060004ec0 2022-06-29T15:42:14.100 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.088+0000 7f807b724700 1 -- 172.21.15.47:0/2121654795 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80740b59c0 con 0x7f8060004ec0 2022-06-29T15:42:14.100 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.088+0000 7f80727fc700 1 -- 172.21.15.47:0/2121654795 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f80680055f0 con 0x7f8060004ec0 2022-06-29T15:42:14.100 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.088+0000 7f807b724700 1 -- 172.21.15.47:0/2121654795 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80740b3e40 con 0x7f8060004ec0 2022-06-29T15:42:14.101 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.088+0000 7f80727fc700 1 -- 172.21.15.47:0/2121654795 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f806801a030 con 0x7f8060004ec0 2022-06-29T15:42:14.101 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.088+0000 7f80727fc700 1 -- 172.21.15.47:0/2121654795 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7f806801d7b0 con 0x7f8060004ec0 2022-06-29T15:42:14.101 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.088+0000 7f807b724700 1 -- 172.21.15.47:0/2121654795 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80740a9200 con 0x7f8060004ec0 2022-06-29T15:42:14.102 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.092+0000 7f80727fc700 1 -- 172.21.15.47:0/2121654795 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f80740a9200 con 0x7f8060004ec0 2022-06-29T15:42:14.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.156+0000 7f807b724700 1 -- 172.21.15.47:0/2121654795 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f807413e150 con 0x7f8060004ec0 2022-06-29T15:42:14.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.160+0000 7f80727fc700 1 -- 172.21.15.47:0/2121654795 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f807413e150 con 0x7f8060004ec0 2022-06-29T15:42:14.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.164+0000 7f807b724700 1 -- 172.21.15.47:0/2121654795 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8060004ec0 msgr2=0x7f80740aab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:14.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.164+0000 7f807b724700 1 --2- 172.21.15.47:0/2121654795 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8060004ec0 0x7f80740aab80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f80740b29c0 tx=0x7f8068009040).stop 2022-06-29T15:42:14.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.164+0000 7f807b724700 1 -- 172.21.15.47:0/2121654795 shutdown_connections 2022-06-29T15:42:14.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.164+0000 7f807b724700 1 --2- 172.21.15.47:0/2121654795 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8060004ec0 0x7f80740aab80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:14.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.164+0000 7f807b724700 1 --2- 172.21.15.47:0/2121654795 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8060004500 0x7f80740b3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:14.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.164+0000 7f807b724700 1 --2- 172.21.15.47:0/2121654795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8060003750 0x7f807413ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:14.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.164+0000 7f807b724700 1 -- 172.21.15.47:0/2121654795 >> 172.21.15.47:0/2121654795 conn(0x7f80740626f0 msgr2=0x7f8074140fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:14.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.164+0000 7f807b724700 1 -- 172.21.15.47:0/2121654795 shutdown_connections 2022-06-29T15:42:14.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:14.164+0000 7f807b724700 1 -- 172.21.15.47:0/2121654795 wait complete. 2022-06-29T15:42:14.168 INFO:teuthology.orchestra.run.smithi047.stderr:adjusted tunables profile to default 2022-06-29T15:42:14.178 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2022-06-29T15:42:14.178 DEBUG:teuthology.orchestra.run.smithi047:> mktemp 2022-06-29T15:42:14.183 INFO:teuthology.orchestra.run.smithi047.stdout:/tmp/tmp.1nLixLIzGv 2022-06-29T15:42:14.184 DEBUG:teuthology.orchestra.run.smithi047:> sudo cp /var/lib/ceph/osd/ceph-0/fsid /tmp/tmp.1nLixLIzGv 2022-06-29T15:42:14.201 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0666 /tmp/tmp.1nLixLIzGv 2022-06-29T15:42:14.264 DEBUG:teuthology.orchestra.remote:smithi047:/tmp/tmp.1nLixLIzGv is 37B 2022-06-29T15:42:14.313 DEBUG:teuthology.orchestra.run.smithi047:> rm -fr /tmp/tmp.1nLixLIzGv 2022-06-29T15:42:14.321 DEBUG:teuthology.orchestra.run.smithi047:> mktemp 2022-06-29T15:42:14.327 INFO:teuthology.orchestra.run.smithi047.stdout:/tmp/tmp.hS9l9bVWte 2022-06-29T15:42:14.328 DEBUG:teuthology.orchestra.run.smithi047:> sudo cp /var/lib/ceph/osd/ceph-1/fsid /tmp/tmp.hS9l9bVWte 2022-06-29T15:42:14.384 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0666 /tmp/tmp.hS9l9bVWte 2022-06-29T15:42:14.402 DEBUG:teuthology.orchestra.remote:smithi047:/tmp/tmp.hS9l9bVWte is 37B 2022-06-29T15:42:14.454 DEBUG:teuthology.orchestra.run.smithi047:> rm -fr /tmp/tmp.hS9l9bVWte 2022-06-29T15:42:14.461 DEBUG:teuthology.orchestra.run.smithi047:> mktemp 2022-06-29T15:42:14.512 INFO:teuthology.orchestra.run.smithi047.stdout:/tmp/tmp.j0H4qHJbPO 2022-06-29T15:42:14.513 DEBUG:teuthology.orchestra.run.smithi047:> sudo cp /var/lib/ceph/osd/ceph-2/fsid /tmp/tmp.j0H4qHJbPO 2022-06-29T15:42:14.527 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0666 /tmp/tmp.j0H4qHJbPO 2022-06-29T15:42:14.593 DEBUG:teuthology.orchestra.remote:smithi047:/tmp/tmp.j0H4qHJbPO is 37B 2022-06-29T15:42:14.609 DEBUG:teuthology.orchestra.run.smithi047:> rm -fr /tmp/tmp.j0H4qHJbPO 2022-06-29T15:42:14.615 DEBUG:teuthology.orchestra.run.smithi047:> mktemp 2022-06-29T15:42:14.663 INFO:teuthology.orchestra.run.smithi047.stdout:/tmp/tmp.bcBQ7lZ52z 2022-06-29T15:42:14.664 DEBUG:teuthology.orchestra.run.smithi047:> sudo cp /var/lib/ceph/osd/ceph-3/fsid /tmp/tmp.bcBQ7lZ52z 2022-06-29T15:42:14.678 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0666 /tmp/tmp.bcBQ7lZ52z 2022-06-29T15:42:14.705 DEBUG:teuthology.orchestra.remote:smithi047:/tmp/tmp.bcBQ7lZ52z is 37B 2022-06-29T15:42:14.715 DEBUG:teuthology.orchestra.run.smithi047:> rm -fr /tmp/tmp.bcBQ7lZ52z 2022-06-29T15:42:14.721 DEBUG:teuthology.orchestra.run.smithi064:> mktemp 2022-06-29T15:42:14.728 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/tmp.0IHvmy8SrI 2022-06-29T15:42:14.729 DEBUG:teuthology.orchestra.run.smithi064:> sudo cp /var/lib/ceph/osd/ceph-4/fsid /tmp/tmp.0IHvmy8SrI 2022-06-29T15:42:14.745 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0666 /tmp/tmp.0IHvmy8SrI 2022-06-29T15:42:14.814 DEBUG:teuthology.orchestra.remote:smithi064:/tmp/tmp.0IHvmy8SrI is 37B 2022-06-29T15:42:14.828 DEBUG:teuthology.orchestra.run.smithi064:> rm -fr /tmp/tmp.0IHvmy8SrI 2022-06-29T15:42:14.835 DEBUG:teuthology.orchestra.run.smithi064:> mktemp 2022-06-29T15:42:14.881 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/tmp.SoKhnid70n 2022-06-29T15:42:14.882 DEBUG:teuthology.orchestra.run.smithi064:> sudo cp /var/lib/ceph/osd/ceph-5/fsid /tmp/tmp.SoKhnid70n 2022-06-29T15:42:14.895 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0666 /tmp/tmp.SoKhnid70n 2022-06-29T15:42:14.958 DEBUG:teuthology.orchestra.remote:smithi064:/tmp/tmp.SoKhnid70n is 37B 2022-06-29T15:42:14.972 DEBUG:teuthology.orchestra.run.smithi064:> rm -fr /tmp/tmp.SoKhnid70n 2022-06-29T15:42:14.979 DEBUG:teuthology.orchestra.run.smithi064:> mktemp 2022-06-29T15:42:15.026 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/tmp.Pq61AdwQap 2022-06-29T15:42:15.026 DEBUG:teuthology.orchestra.run.smithi064:> sudo cp /var/lib/ceph/osd/ceph-6/fsid /tmp/tmp.Pq61AdwQap 2022-06-29T15:42:15.043 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0666 /tmp/tmp.Pq61AdwQap 2022-06-29T15:42:15.064 DEBUG:teuthology.orchestra.remote:smithi064:/tmp/tmp.Pq61AdwQap is 37B 2022-06-29T15:42:15.078 DEBUG:teuthology.orchestra.run.smithi064:> rm -fr /tmp/tmp.Pq61AdwQap 2022-06-29T15:42:15.084 DEBUG:teuthology.orchestra.run.smithi064:> mktemp 2022-06-29T15:42:15.133 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/tmp.I3Nwqdipx0 2022-06-29T15:42:15.134 DEBUG:teuthology.orchestra.run.smithi064:> sudo cp /var/lib/ceph/osd/ceph-7/fsid /tmp/tmp.I3Nwqdipx0 2022-06-29T15:42:15.149 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0666 /tmp/tmp.I3Nwqdipx0 2022-06-29T15:42:15.214 DEBUG:teuthology.orchestra.remote:smithi064:/tmp/tmp.I3Nwqdipx0 is 37B 2022-06-29T15:42:15.229 DEBUG:teuthology.orchestra.run.smithi064:> rm -fr /tmp/tmp.I3Nwqdipx0 2022-06-29T15:42:15.234 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --cluster ceph osd new e23eb023-4706-4201-bf72-4bf19c912c5d 0 2022-06-29T15:42:15.384 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.382+0000 7f4c99cf5700 1 Processor -- start 2022-06-29T15:42:15.384 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.382+0000 7f4c99cf5700 1 -- start start 2022-06-29T15:42:15.387 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.382+0000 7f4c99cf5700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4c940acfd0 0x7f4c940ad3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:15.387 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.382+0000 7f4c99cf5700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4c94060ea0 0x7f4c94060090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:15.387 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.382+0000 7f4c99cf5700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c9405f470 con 0x7f4c94067790 2022-06-29T15:42:15.388 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.382+0000 7f4c99cf5700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c940aff20 con 0x7f4c94060ea0 2022-06-29T15:42:15.388 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.382+0000 7f4c99cf5700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c940afc80 con 0x7f4c940acfd0 2022-06-29T15:42:15.388 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.382+0000 7f4c98cf3700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4c940acfd0 0x7f4c940ad3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:15.388 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.382+0000 7f4c98cf3700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4c940acfd0 0x7f4c940ad3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.64:51672/0 (socket says 172.21.15.64:51672) 2022-06-29T15:42:15.389 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.382+0000 7f4c98cf3700 1 -- 172.21.15.64:0/1575330566 learned_addr learned my addr 172.21.15.64:0/1575330566 (peer_addr_for_me v2:172.21.15.64:0/0) 2022-06-29T15:42:15.389 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.382+0000 7f4c8bfff700 1 --2- 172.21.15.64:0/1575330566 >> v2:172.21.15.64:3300/0 conn(0x7f4c94060ea0 0x7f4c94060090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:15.389 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.382+0000 7f4c98cf3700 1 -- 172.21.15.64:0/1575330566 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4c940acfd0 msgr2=0x7f4c940ad3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-06-29T15:42:15.389 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.382+0000 7f4c98cf3700 1 -- 172.21.15.64:0/1575330566 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4c940acfd0 msgr2=0x7f4c940ad3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-06-29T15:42:15.389 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.382+0000 7f4c98cf3700 1 --2- 172.21.15.64:0/1575330566 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4c940acfd0 0x7f4c940ad3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-29T15:42:15.390 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.382+0000 7f4c98cf3700 1 --2- 172.21.15.64:0/1575330566 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4c940acfd0 0x7f4c940ad3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2022-06-29T15:42:15.390 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.382+0000 7f4c8b7fe700 1 -- 172.21.15.64:0/1575330566 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3132012729 0 0) 0x7f4c9405f470 con 0x7f4c94067790 2022-06-29T15:42:15.390 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.382+0000 7f4c8b7fe700 1 -- 172.21.15.64:0/1575330566 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c80003260 con 0x7f4c94067790 2022-06-29T15:42:15.390 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.382+0000 7f4c8bfff700 1 -- 172.21.15.64:0/1575330566 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4c940acfd0 msgr2=0x7f4c940ad3b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-06-29T15:42:15.391 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.382+0000 7f4c8bfff700 1 --2- 172.21.15.64:0/1575330566 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4c940acfd0 0x7f4c940ad3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:15.391 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.382+0000 7f4c8bfff700 1 -- 172.21.15.64:0/1575330566 >> v1:172.21.15.64:6789/0 conn(0x7f4c94067790 legacy=0x7f4c940b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:15.391 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.382+0000 7f4c8bfff700 1 -- 172.21.15.64:0/1575330566 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c9405f6d0 con 0x7f4c94060ea0 2022-06-29T15:42:15.391 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.382+0000 7f4c8bfff700 1 --2- 172.21.15.64:0/1575330566 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c94060ea0 0x7f4c94060090 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f4c84002f70 tx=0x7f4c84006d80).ready entity=mon.1 client_cookie=2e1e40fe5001b7ac server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:15.391 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.382+0000 7f4c8b7fe700 1 -- 172.21.15.64:0/1575330566 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c8400bac0 con 0x7f4c94060ea0 2022-06-29T15:42:15.392 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.382+0000 7f4c8b7fe700 1 -- 172.21.15.64:0/1575330566 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8400bc20 con 0x7f4c94060ea0 2022-06-29T15:42:15.392 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.382+0000 7f4c8b7fe700 1 -- 172.21.15.64:0/1575330566 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c840153f0 con 0x7f4c94060ea0 2022-06-29T15:42:15.396 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.386+0000 7f4c99cf5700 1 -- 172.21.15.64:0/1575330566 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c94060ea0 msgr2=0x7f4c94060090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:15.396 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.386+0000 7f4c99cf5700 1 --2- 172.21.15.64:0/1575330566 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c94060ea0 0x7f4c94060090 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f4c84002f70 tx=0x7f4c84006d80).stop 2022-06-29T15:42:15.396 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.386+0000 7f4c99cf5700 1 -- 172.21.15.64:0/1575330566 shutdown_connections 2022-06-29T15:42:15.397 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.386+0000 7f4c99cf5700 1 --2- 172.21.15.64:0/1575330566 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c94060ea0 0x7f4c94060090 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:15.397 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.386+0000 7f4c99cf5700 1 --2- 172.21.15.64:0/1575330566 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4c940acfd0 0x7f4c940ad3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:15.397 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.386+0000 7f4c99cf5700 1 -- 172.21.15.64:0/1575330566 >> 172.21.15.64:0/1575330566 conn(0x7f4c940b5b90 msgr2=0x7f4c940b5f70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:15.397 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.386+0000 7f4c99cf5700 1 -- 172.21.15.64:0/1575330566 shutdown_connections 2022-06-29T15:42:15.398 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.386+0000 7f4c99cf5700 1 -- 172.21.15.64:0/1575330566 wait complete. 2022-06-29T15:42:15.398 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.386+0000 7f4c99cf5700 1 Processor -- start 2022-06-29T15:42:15.398 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.386+0000 7f4c99cf5700 1 -- start start 2022-06-29T15:42:15.399 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.386+0000 7f4c99cf5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c94060ea0 0x7f4c940a9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:15.399 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.386+0000 7f4c99cf5700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4c94067790 0x7f4c9409ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:15.399 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.386+0000 7f4c99cf5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c940acfd0 0x7f4c940abae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:15.399 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.386+0000 7f4c99cf5700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c940aad60 con 0x7f4c940acfd0 2022-06-29T15:42:15.400 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.386+0000 7f4c99cf5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9409b280 con 0x7f4c94060ea0 2022-06-29T15:42:15.400 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.386+0000 7f4c99cf5700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9409a1f0 con 0x7f4c94067790 2022-06-29T15:42:15.400 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.386+0000 7f4c8bfff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4c94067790 0x7f4c9409ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:15.400 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.386+0000 7f4c8bfff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4c94067790 0x7f4c9409ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.64:51680/0 (socket says 172.21.15.64:51680) 2022-06-29T15:42:15.400 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.386+0000 7f4c8bfff700 1 -- 172.21.15.64:0/1995867561 learned_addr learned my addr 172.21.15.64:0/1995867561 (peer_addr_for_me v2:172.21.15.64:0/0) 2022-06-29T15:42:15.401 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.386+0000 7f4c98cf3700 1 --2- 172.21.15.64:0/1995867561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c94060ea0 0x7f4c940a9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:15.401 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.386+0000 7f4c8bfff700 1 -- 172.21.15.64:0/1995867561 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4c94067790 msgr2=0x7f4c9409ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2022-06-29T15:42:15.401 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.386+0000 7f4c8bfff700 1 -- 172.21.15.64:0/1995867561 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4c94067790 msgr2=0x7f4c9409ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2022-06-29T15:42:15.401 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.386+0000 7f4c8bfff700 1 --2- 172.21.15.64:0/1995867561 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4c94067790 0x7f4c9409ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-29T15:42:15.402 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.386+0000 7f4c8bfff700 1 --2- 172.21.15.64:0/1995867561 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4c94067790 0x7f4c9409ad90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2022-06-29T15:42:15.402 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.386+0000 7f4c994f4700 1 --2- 172.21.15.64:0/1995867561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c940acfd0 0x7f4c940abae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:15.402 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.386+0000 7f4c98cf3700 1 -- 172.21.15.64:0/1995867561 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4c94067790 msgr2=0x7f4c9409ad90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2022-06-29T15:42:15.402 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.386+0000 7f4c98cf3700 1 --2- 172.21.15.64:0/1995867561 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4c94067790 0x7f4c9409ad90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:15.403 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.386+0000 7f4c98cf3700 1 -- 172.21.15.64:0/1995867561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c940acfd0 msgr2=0x7f4c940abae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:15.403 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.386+0000 7f4c98cf3700 1 --2- 172.21.15.64:0/1995867561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c940acfd0 0x7f4c940abae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:15.403 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.386+0000 7f4c98cf3700 1 -- 172.21.15.64:0/1995867561 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c8400d410 con 0x7f4c94060ea0 2022-06-29T15:42:15.403 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.386+0000 7f4c98cf3700 1 --2- 172.21.15.64:0/1995867561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c94060ea0 0x7f4c940a9ce0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f4c90000e90 tx=0x7f4c90008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:15.403 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.386+0000 7f4c89ffb700 1 -- 172.21.15.64:0/1995867561 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c90011780 con 0x7f4c94060ea0 2022-06-29T15:42:15.404 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.386+0000 7f4c89ffb700 1 -- 172.21.15.64:0/1995867561 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c900118e0 con 0x7f4c94060ea0 2022-06-29T15:42:15.404 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.386+0000 7f4c99cf5700 1 -- 172.21.15.64:0/1995867561 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c9409a6a0 con 0x7f4c94060ea0 2022-06-29T15:42:15.404 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.386+0000 7f4c89ffb700 1 -- 172.21.15.64:0/1995867561 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c90011c40 con 0x7f4c94060ea0 2022-06-29T15:42:15.404 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.386+0000 7f4c99cf5700 1 -- 172.21.15.64:0/1995867561 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c940ab640 con 0x7f4c94060ea0 2022-06-29T15:42:15.405 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.386+0000 7f4c89ffb700 1 -- 172.21.15.64:0/1995867561 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f4c9001c460 con 0x7f4c94060ea0 2022-06-29T15:42:15.405 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.386+0000 7f4c89ffb700 1 -- 172.21.15.64:0/1995867561 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 754+0+0 (secure 0 0 0) 0x7f4c90012420 con 0x7f4c94060ea0 2022-06-29T15:42:15.405 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.390+0000 7f4c99cf5700 1 -- 172.21.15.64:0/1995867561 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c940573d0 con 0x7f4c94060ea0 2022-06-29T15:42:15.405 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.390+0000 7f4c89ffb700 1 -- 172.21.15.64:0/1995867561 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f4c90012630 con 0x7f4c94060ea0 2022-06-29T15:42:15.472 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.470+0000 7f4c99cf5700 1 -- 172.21.15.64:0/1995867561 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e23eb023-4706-4201-bf72-4bf19c912c5d", "id": 0} v 0) v1 -- 0x7f4c94067d20 con 0x7f4c94060ea0 2022-06-29T15:42:15.481 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.478+0000 7f4c89ffb700 1 -- 172.21.15.64:0/1995867561 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e23eb023-4706-4201-bf72-4bf19c912c5d", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4c900127d0 con 0x7f4c94060ea0 2022-06-29T15:42:15.482 INFO:teuthology.orchestra.run.smithi064.stdout:0 2022-06-29T15:42:15.484 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.478+0000 7f4c99cf5700 1 -- 172.21.15.64:0/1995867561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c94060ea0 msgr2=0x7f4c940a9ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:15.484 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.478+0000 7f4c99cf5700 1 --2- 172.21.15.64:0/1995867561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c94060ea0 0x7f4c940a9ce0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f4c90000e90 tx=0x7f4c90008040).stop 2022-06-29T15:42:15.484 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.482+0000 7f4c99cf5700 1 -- 172.21.15.64:0/1995867561 shutdown_connections 2022-06-29T15:42:15.485 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.482+0000 7f4c99cf5700 1 --2- 172.21.15.64:0/1995867561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c940acfd0 0x7f4c940abae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:15.485 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.482+0000 7f4c99cf5700 1 --2- 172.21.15.64:0/1995867561 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4c94067790 0x7f4c9409ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:15.485 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.482+0000 7f4c99cf5700 1 --2- 172.21.15.64:0/1995867561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c94060ea0 0x7f4c940a9ce0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:15.485 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.482+0000 7f4c99cf5700 1 -- 172.21.15.64:0/1995867561 >> 172.21.15.64:0/1995867561 conn(0x7f4c940b5b90 msgr2=0x7f4c940b5f70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:15.486 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.482+0000 7f4c99cf5700 1 -- 172.21.15.64:0/1995867561 shutdown_connections 2022-06-29T15:42:15.486 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.482+0000 7f4c99cf5700 1 -- 172.21.15.64:0/1995867561 wait complete. 2022-06-29T15:42:15.497 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --cluster ceph osd new 228bb08d-2d64-43d9-9560-bfaeff154452 1 2022-06-29T15:42:15.596 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.590+0000 7fe0c879d700 1 Processor -- start 2022-06-29T15:42:15.596 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.590+0000 7fe0c879d700 1 -- start start 2022-06-29T15:42:15.600 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.594+0000 7fe0c879d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe0c00b4450 0x7fe0c0146cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:15.600 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.594+0000 7fe0c879d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe0c00ab0d0 0x7fe0c01471e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:15.600 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.594+0000 7fe0c879d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe0c0060820 0x7fe0c0060c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:15.601 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.594+0000 7fe0c879d700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0c0064660 con 0x7fe0c00b4450 2022-06-29T15:42:15.601 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.594+0000 7fe0c879d700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0c0061700 con 0x7fe0c00ab0d0 2022-06-29T15:42:15.601 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.594+0000 7fe0c879d700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c0060460 con 0x7fe0c0060820 2022-06-29T15:42:15.601 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.594+0000 7fe0c5d38700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe0c00ab0d0 0x7fe0c01471e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:15.602 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.594+0000 7fe0c6d3a700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe0c0060820 0x7fe0c0060c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:15.602 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.594+0000 7fe0c5d38700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe0c00ab0d0 0x7fe0c01471e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42136/0 (socket says 172.21.15.64:42136) 2022-06-29T15:42:15.602 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.594+0000 7fe0c5d38700 1 -- 172.21.15.64:0/3944997058 learned_addr learned my addr 172.21.15.64:0/3944997058 (peer_addr_for_me v2:172.21.15.64:0/0) 2022-06-29T15:42:15.603 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.594+0000 7fe0c6d3a700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe0c0060820 0x7fe0c0060c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.64:51688/0 (socket says 172.21.15.64:51688) 2022-06-29T15:42:15.603 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.594+0000 7fe0c6539700 1 --2- 172.21.15.64:0/3944997058 >> v2:172.21.15.47:3300/0 conn(0x7fe0c00b4450 0x7fe0c0146cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:15.603 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.594+0000 7fe0c6d3a700 1 -- 172.21.15.64:0/3944997058 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe0c0060820 msgr2=0x7fe0c0060c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2022-06-29T15:42:15.603 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.594+0000 7fe0c6d3a700 1 -- 172.21.15.64:0/3944997058 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe0c0060820 msgr2=0x7fe0c0060c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-06-29T15:42:15.604 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.594+0000 7fe0c6d3a700 1 --2- 172.21.15.64:0/3944997058 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe0c0060820 0x7fe0c0060c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-29T15:42:15.604 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.594+0000 7fe0c6d3a700 1 --2- 172.21.15.64:0/3944997058 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe0c0060820 0x7fe0c0060c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2022-06-29T15:42:15.604 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.594+0000 7fe0c5d38700 1 -- 172.21.15.64:0/3944997058 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe0c0060820 msgr2=0x7fe0c0060c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-06-29T15:42:15.604 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.594+0000 7fe0c5d38700 1 --2- 172.21.15.64:0/3944997058 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe0c0060820 0x7fe0c0060c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:15.605 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.594+0000 7fe0c5d38700 1 -- 172.21.15.64:0/3944997058 >> v2:172.21.15.47:3300/0 conn(0x7fe0c00b4450 msgr2=0x7fe0c0146cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:15.605 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.594+0000 7fe0c5d38700 1 --2- 172.21.15.64:0/3944997058 >> v2:172.21.15.47:3300/0 conn(0x7fe0c00b4450 0x7fe0c0146cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:15.605 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.594+0000 7fe0c5d38700 1 -- 172.21.15.64:0/3944997058 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c00b4830 con 0x7fe0c00ab0d0 2022-06-29T15:42:15.605 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.594+0000 7fe0c5d38700 1 --2- 172.21.15.64:0/3944997058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0c00ab0d0 0x7fe0c01471e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7fe0b0002f70 tx=0x7fe0b0006eb0).ready entity=mon.1 client_cookie=a92f0189f562ca9f server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:15.606 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.594+0000 7fe0c5537700 1 -- 172.21.15.64:0/3944997058 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe0b000d570 con 0x7fe0c00ab0d0 2022-06-29T15:42:15.606 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.594+0000 7fe0c5537700 1 -- 172.21.15.64:0/3944997058 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b0000e90 con 0x7fe0c00ab0d0 2022-06-29T15:42:15.606 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.594+0000 7fe0c5537700 1 -- 172.21.15.64:0/3944997058 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe0b000d900 con 0x7fe0c00ab0d0 2022-06-29T15:42:15.606 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.594+0000 7fe0c879d700 1 -- 172.21.15.64:0/3944997058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0c00ab0d0 msgr2=0x7fe0c01471e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:15.607 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.594+0000 7fe0c879d700 1 --2- 172.21.15.64:0/3944997058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0c00ab0d0 0x7fe0c01471e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7fe0b0002f70 tx=0x7fe0b0006eb0).stop 2022-06-29T15:42:15.607 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.594+0000 7fe0c879d700 1 -- 172.21.15.64:0/3944997058 shutdown_connections 2022-06-29T15:42:15.607 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.594+0000 7fe0c879d700 1 --2- 172.21.15.64:0/3944997058 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe0c0060820 0x7fe0c0060c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:15.608 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.594+0000 7fe0c879d700 1 --2- 172.21.15.64:0/3944997058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0c00ab0d0 0x7fe0c01471e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:15.608 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.594+0000 7fe0c879d700 1 --2- 172.21.15.64:0/3944997058 >> v2:172.21.15.47:3300/0 conn(0x7fe0c00b4450 0x7fe0c0146cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:15.608 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.594+0000 7fe0c879d700 1 -- 172.21.15.64:0/3944997058 >> 172.21.15.64:0/3944997058 conn(0x7fe0c0063290 msgr2=0x7fe0c005c510 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:15.608 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.594+0000 7fe0c879d700 1 -- 172.21.15.64:0/3944997058 shutdown_connections 2022-06-29T15:42:15.609 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.594+0000 7fe0c879d700 1 -- 172.21.15.64:0/3944997058 wait complete. 2022-06-29T15:42:15.613 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.598+0000 7fe0c879d700 1 Processor -- start 2022-06-29T15:42:15.613 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.598+0000 7fe0c879d700 1 -- start start 2022-06-29T15:42:15.613 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.598+0000 7fe0c879d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0c0060820 0x7fe0c014a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:15.613 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.598+0000 7fe0c879d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe0c00ab0d0 0x7fe0c0155620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:15.614 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.598+0000 7fe0c879d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0c00b4450 0x7fe0c0152960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:15.614 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.598+0000 7fe0c879d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c0152070 con 0x7fe0c0060820 2022-06-29T15:42:15.614 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.598+0000 7fe0c879d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c00674f0 con 0x7fe0c00b4450 2022-06-29T15:42:15.614 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.598+0000 7fe0c879d700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c0152260 con 0x7fe0c00ab0d0 2022-06-29T15:42:15.615 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.598+0000 7fe0c5d38700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe0c00ab0d0 0x7fe0c0155620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:15.615 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.598+0000 7fe0c6d3a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0c00b4450 0x7fe0c0152960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:15.615 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.598+0000 7fe0c6d3a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0c00b4450 0x7fe0c0152960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42142/0 (socket says 172.21.15.64:42142) 2022-06-29T15:42:15.615 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.598+0000 7fe0c5d38700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe0c00ab0d0 0x7fe0c0155620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.64:51692/0 (socket says 172.21.15.64:51692) 2022-06-29T15:42:15.616 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.598+0000 7fe0c6d3a700 1 -- 172.21.15.64:0/3869190487 learned_addr learned my addr 172.21.15.64:0/3869190487 (peer_addr_for_me v2:172.21.15.64:0/0) 2022-06-29T15:42:15.616 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.598+0000 7fe0c6539700 1 --2- 172.21.15.64:0/3869190487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0c0060820 0x7fe0c014a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:15.616 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.598+0000 7fe0c5d38700 1 -- 172.21.15.64:0/3869190487 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe0c00ab0d0 msgr2=0x7fe0c0155620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2022-06-29T15:42:15.616 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.598+0000 7fe0c5d38700 1 -- 172.21.15.64:0/3869190487 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe0c00ab0d0 msgr2=0x7fe0c0155620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2022-06-29T15:42:15.617 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.598+0000 7fe0c5d38700 1 --2- 172.21.15.64:0/3869190487 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe0c00ab0d0 0x7fe0c0155620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-29T15:42:15.617 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.598+0000 7fe0c5d38700 1 --2- 172.21.15.64:0/3869190487 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe0c00ab0d0 0x7fe0c0155620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2022-06-29T15:42:15.617 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.598+0000 7fe0c6d3a700 1 -- 172.21.15.64:0/3869190487 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe0c00ab0d0 msgr2=0x7fe0c0155620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2022-06-29T15:42:15.617 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.598+0000 7fe0c6d3a700 1 --2- 172.21.15.64:0/3869190487 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe0c00ab0d0 0x7fe0c0155620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:15.618 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.598+0000 7fe0c6d3a700 1 -- 172.21.15.64:0/3869190487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0c0060820 msgr2=0x7fe0c014a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:15.618 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.598+0000 7fe0c6d3a700 1 --2- 172.21.15.64:0/3869190487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0c0060820 0x7fe0c014a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:15.618 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.598+0000 7fe0c6d3a700 1 -- 172.21.15.64:0/3869190487 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b000b1e0 con 0x7fe0c00b4450 2022-06-29T15:42:15.618 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.598+0000 7fe0c6d3a700 1 --2- 172.21.15.64:0/3869190487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0c00b4450 0x7fe0c0152960 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7fe0b8001210 tx=0x7fe0b8008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:15.619 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.598+0000 7fe0af7fe700 1 -- 172.21.15.64:0/3869190487 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe0b800f710 con 0x7fe0c00b4450 2022-06-29T15:42:15.619 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.598+0000 7fe0af7fe700 1 -- 172.21.15.64:0/3869190487 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b800f870 con 0x7fe0c00b4450 2022-06-29T15:42:15.619 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.598+0000 7fe0c879d700 1 -- 172.21.15.64:0/3869190487 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0c0155b10 con 0x7fe0c00b4450 2022-06-29T15:42:15.619 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.598+0000 7fe0af7fe700 1 -- 172.21.15.64:0/3869190487 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe0b800fbd0 con 0x7fe0c00b4450 2022-06-29T15:42:15.619 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.598+0000 7fe0c879d700 1 -- 172.21.15.64:0/3869190487 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0c01524c0 con 0x7fe0c00b4450 2022-06-29T15:42:15.620 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.598+0000 7fe0af7fe700 1 -- 172.21.15.64:0/3869190487 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7fe0b801a030 con 0x7fe0c00b4450 2022-06-29T15:42:15.620 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.598+0000 7fe0af7fe700 1 -- 172.21.15.64:0/3869190487 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 869+0+0 (secure 0 0 0) 0x7fe0b8005e00 con 0x7fe0c00b4450 2022-06-29T15:42:15.620 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.602+0000 7fe0c879d700 1 -- 172.21.15.64:0/3869190487 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0c00573d0 con 0x7fe0c00b4450 2022-06-29T15:42:15.620 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.602+0000 7fe0af7fe700 1 -- 172.21.15.64:0/3869190487 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7fe0b8018070 con 0x7fe0c00b4450 2022-06-29T15:42:15.675 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.670+0000 7fe0c879d700 1 -- 172.21.15.64:0/3869190487 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd new", "uuid": "228bb08d-2d64-43d9-9560-bfaeff154452", "id": 1} v 0) v1 -- 0x7fe0c0152680 con 0x7fe0c00b4450 2022-06-29T15:42:15.682 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.678+0000 7fe0af7fe700 1 -- 172.21.15.64:0/3869190487 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "228bb08d-2d64-43d9-9560-bfaeff154452", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe0b8013550 con 0x7fe0c00b4450 2022-06-29T15:42:15.683 INFO:teuthology.orchestra.run.smithi064.stdout:1 2022-06-29T15:42:15.684 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.678+0000 7fe0c879d700 1 -- 172.21.15.64:0/3869190487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0c00b4450 msgr2=0x7fe0c0152960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:15.684 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.678+0000 7fe0c879d700 1 --2- 172.21.15.64:0/3869190487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0c00b4450 0x7fe0c0152960 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7fe0b8001210 tx=0x7fe0b8008040).stop 2022-06-29T15:42:15.684 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.678+0000 7fe0c879d700 1 -- 172.21.15.64:0/3869190487 shutdown_connections 2022-06-29T15:42:15.685 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.678+0000 7fe0c879d700 1 --2- 172.21.15.64:0/3869190487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0c00b4450 0x7fe0c0152960 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:15.685 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.678+0000 7fe0c879d700 1 --2- 172.21.15.64:0/3869190487 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe0c00ab0d0 0x7fe0c0155620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:15.685 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.678+0000 7fe0c879d700 1 --2- 172.21.15.64:0/3869190487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0c0060820 0x7fe0c014a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:15.685 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.678+0000 7fe0c879d700 1 -- 172.21.15.64:0/3869190487 >> 172.21.15.64:0/3869190487 conn(0x7fe0c0063290 msgr2=0x7fe0c0146c10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:15.686 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.682+0000 7fe0c879d700 1 -- 172.21.15.64:0/3869190487 shutdown_connections 2022-06-29T15:42:15.686 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.682+0000 7fe0c879d700 1 -- 172.21.15.64:0/3869190487 wait complete. 2022-06-29T15:42:15.695 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --cluster ceph osd new 797a5452-1510-4640-a111-c9c2fcf707eb 2 2022-06-29T15:42:15.788 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.786+0000 7fa9fb233700 1 Processor -- start 2022-06-29T15:42:15.789 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.786+0000 7fa9fb233700 1 -- start start 2022-06-29T15:42:15.791 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.786+0000 7fa9fb233700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa9f405fd40 0x7fa9f4061a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:15.791 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.786+0000 7fa9fb233700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9f414a980 con 0x7fa9f40aead0 2022-06-29T15:42:15.791 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.786+0000 7fa9fb233700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9f414ac20 con 0x7fa9f405f960 2022-06-29T15:42:15.791 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.786+0000 7fa9fb233700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9f414b8a0 con 0x7fa9f405fd40 2022-06-29T15:42:15.792 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.786+0000 7fa9fa231700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa9f405f960 0x7fa9f4061350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48816/0 (socket says 172.21.15.64:48816) 2022-06-29T15:42:15.792 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.786+0000 7fa9fa231700 1 -- 172.21.15.64:0/2547922862 learned_addr learned my addr 172.21.15.64:0/2547922862 (peer_addr_for_me v1:172.21.15.64:0/0) 2022-06-29T15:42:15.792 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.786+0000 7fa9faa32700 1 --2- 172.21.15.64:0/2547922862 >> v2:172.21.15.47:3300/0 conn(0x7fa9f405fd40 0x7fa9f4061a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:15.792 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.786+0000 7fa9f922f700 1 -- 172.21.15.64:0/2547922862 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3018738281 0 0) 0x7fa9f414ac20 con 0x7fa9f405f960 2022-06-29T15:42:15.793 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.786+0000 7fa9f922f700 1 -- 172.21.15.64:0/2547922862 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9dc003260 con 0x7fa9f405f960 2022-06-29T15:42:15.793 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.786+0000 7fa9faa32700 1 -- 172.21.15.64:0/2547922862 >> v1:172.21.15.64:6789/0 conn(0x7fa9f405f960 legacy=0x7fa9f4061350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:15.793 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.786+0000 7fa9faa32700 1 -- 172.21.15.64:0/2547922862 >> v1:172.21.15.47:6789/0 conn(0x7fa9f40aead0 legacy=0x7fa9f4060480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:15.793 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.786+0000 7fa9faa32700 1 -- 172.21.15.64:0/2547922862 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f414aec0 con 0x7fa9f405fd40 2022-06-29T15:42:15.793 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.786+0000 7fa9faa32700 1 --2- 172.21.15.64:0/2547922862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9f405fd40 0x7fa9f4061a60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7fa9f0002f70 tx=0x7fa9f0006df0).ready entity=mon.0 client_cookie=9ad2cb981f204df0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:15.794 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.786+0000 7fa9f922f700 1 -- 172.21.15.64:0/2547922862 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9f000d610 con 0x7fa9f405fd40 2022-06-29T15:42:15.794 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.786+0000 7fa9f922f700 1 -- 172.21.15.64:0/2547922862 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9f0000e90 con 0x7fa9f405fd40 2022-06-29T15:42:15.794 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.786+0000 7fa9f922f700 1 -- 172.21.15.64:0/2547922862 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9f000d960 con 0x7fa9f405fd40 2022-06-29T15:42:15.798 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.786+0000 7fa9fb233700 1 -- 172.21.15.64:0/2547922862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9f405fd40 msgr2=0x7fa9f4061a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:15.798 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.786+0000 7fa9fb233700 1 --2- 172.21.15.64:0/2547922862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9f405fd40 0x7fa9f4061a60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7fa9f0002f70 tx=0x7fa9f0006df0).stop 2022-06-29T15:42:15.798 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.786+0000 7fa9fb233700 1 -- 172.21.15.64:0/2547922862 shutdown_connections 2022-06-29T15:42:15.799 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.786+0000 7fa9fb233700 1 --2- 172.21.15.64:0/2547922862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9f405fd40 0x7fa9f4061a60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:15.799 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.786+0000 7fa9fb233700 1 -- 172.21.15.64:0/2547922862 >> 172.21.15.64:0/2547922862 conn(0x7fa9f40b6430 msgr2=0x7fa9f40b6810 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:15.799 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.786+0000 7fa9fb233700 1 -- 172.21.15.64:0/2547922862 shutdown_connections 2022-06-29T15:42:15.801 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.786+0000 7fa9fb233700 1 -- 172.21.15.64:0/2547922862 wait complete. 2022-06-29T15:42:15.801 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.790+0000 7fa9fb233700 1 Processor -- start 2022-06-29T15:42:15.801 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.790+0000 7fa9fb233700 1 -- start start 2022-06-29T15:42:15.801 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.790+0000 7fa9fb233700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9f405f960 0x7fa9f414cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:15.802 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.790+0000 7fa9fb233700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9f405fd40 0x7fa9f40aa490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:15.802 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.790+0000 7fa9fb233700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa9f40aead0 0x7fa9f40ace70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:15.802 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.790+0000 7fa9fb233700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f4156d90 con 0x7fa9f405fd40 2022-06-29T15:42:15.802 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.790+0000 7fa9fb233700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f4156fc0 con 0x7fa9f405f960 2022-06-29T15:42:15.803 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.790+0000 7fa9fb233700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f4157260 con 0x7fa9f40aead0 2022-06-29T15:42:15.803 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.790+0000 7fa9fa231700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9f405f960 0x7fa9f414cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:15.803 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.790+0000 7fa9fa231700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9f405f960 0x7fa9f414cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42152/0 (socket says 172.21.15.64:42152) 2022-06-29T15:42:15.803 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.790+0000 7fa9fa231700 1 -- 172.21.15.64:0/3346618199 learned_addr learned my addr 172.21.15.64:0/3346618199 (peer_addr_for_me v2:172.21.15.64:0/0) 2022-06-29T15:42:15.804 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.790+0000 7fa9faa32700 1 --2- 172.21.15.64:0/3346618199 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa9f40aead0 0x7fa9f40ace70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:15.804 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.790+0000 7fa9f9a30700 1 --2- 172.21.15.64:0/3346618199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9f405fd40 0x7fa9f40aa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:15.804 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.790+0000 7fa9faa32700 1 -- 172.21.15.64:0/3346618199 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa9f40aead0 msgr2=0x7fa9f40ace70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2022-06-29T15:42:15.805 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.790+0000 7fa9faa32700 1 -- 172.21.15.64:0/3346618199 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa9f40aead0 msgr2=0x7fa9f40ace70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2022-06-29T15:42:15.805 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.790+0000 7fa9faa32700 1 --2- 172.21.15.64:0/3346618199 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa9f40aead0 0x7fa9f40ace70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-29T15:42:15.805 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.790+0000 7fa9faa32700 1 --2- 172.21.15.64:0/3346618199 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa9f40aead0 0x7fa9f40ace70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2022-06-29T15:42:15.806 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.790+0000 7fa9fa231700 1 -- 172.21.15.64:0/3346618199 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa9f40aead0 msgr2=0x7fa9f40ace70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2022-06-29T15:42:15.806 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.790+0000 7fa9fa231700 1 --2- 172.21.15.64:0/3346618199 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa9f40aead0 0x7fa9f40ace70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:15.806 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.790+0000 7fa9fa231700 1 -- 172.21.15.64:0/3346618199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9f405fd40 msgr2=0x7fa9f40aa490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:15.807 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.790+0000 7fa9fa231700 1 --2- 172.21.15.64:0/3346618199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9f405fd40 0x7fa9f40aa490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:15.807 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.790+0000 7fa9fa231700 1 -- 172.21.15.64:0/3346618199 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f000b180 con 0x7fa9f405f960 2022-06-29T15:42:15.808 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.790+0000 7fa9f9a30700 1 --2- 172.21.15.64:0/3346618199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9f405fd40 0x7fa9f40aa490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:42:15.809 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.790+0000 7fa9fa231700 1 --2- 172.21.15.64:0/3346618199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9f405f960 0x7fa9f414cd10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7fa9ec001490 tx=0x7fa9ec007e50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:15.809 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.790+0000 7fa9e37fe700 1 -- 172.21.15.64:0/3346618199 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9ec005ba0 con 0x7fa9f405f960 2022-06-29T15:42:15.810 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.790+0000 7fa9e37fe700 1 -- 172.21.15.64:0/3346618199 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9ec005d00 con 0x7fa9f405f960 2022-06-29T15:42:15.810 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.790+0000 7fa9fb233700 1 -- 172.21.15.64:0/3346618199 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9f40ac9d0 con 0x7fa9f405f960 2022-06-29T15:42:15.811 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.790+0000 7fa9e37fe700 1 -- 172.21.15.64:0/3346618199 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9ec007300 con 0x7fa9f405f960 2022-06-29T15:42:15.811 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.790+0000 7fa9fb233700 1 -- 172.21.15.64:0/3346618199 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9f4147a00 con 0x7fa9f405f960 2022-06-29T15:42:15.811 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.790+0000 7fa9e37fe700 1 -- 172.21.15.64:0/3346618199 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7fa9ec010030 con 0x7fa9f405f960 2022-06-29T15:42:15.812 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.790+0000 7fa9e37fe700 1 -- 172.21.15.64:0/3346618199 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 984+0+0 (secure 0 0 0) 0x7fa9ec007630 con 0x7fa9f405f960 2022-06-29T15:42:15.813 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.794+0000 7fa9fb233700 1 -- 172.21.15.64:0/3346618199 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9f40573d0 con 0x7fa9f405f960 2022-06-29T15:42:15.814 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.794+0000 7fa9e37fe700 1 -- 172.21.15.64:0/3346618199 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7fa9ec013410 con 0x7fa9f405f960 2022-06-29T15:42:15.864 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.862+0000 7fa9fb233700 1 -- 172.21.15.64:0/3346618199 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd new", "uuid": "797a5452-1510-4640-a111-c9c2fcf707eb", "id": 2} v 0) v1 -- 0x7fa9f406a0f0 con 0x7fa9f405f960 2022-06-29T15:42:15.873 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.870+0000 7fa9e37fe700 1 -- 172.21.15.64:0/3346618199 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "797a5452-1510-4640-a111-c9c2fcf707eb", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa9ec013610 con 0x7fa9f405f960 2022-06-29T15:42:15.874 INFO:teuthology.orchestra.run.smithi064.stdout:2 2022-06-29T15:42:15.876 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.870+0000 7fa9fb233700 1 -- 172.21.15.64:0/3346618199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9f405f960 msgr2=0x7fa9f414cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:15.876 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.870+0000 7fa9fb233700 1 --2- 172.21.15.64:0/3346618199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9f405f960 0x7fa9f414cd10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7fa9ec001490 tx=0x7fa9ec007e50).stop 2022-06-29T15:42:15.876 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.870+0000 7fa9fb233700 1 -- 172.21.15.64:0/3346618199 shutdown_connections 2022-06-29T15:42:15.876 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.870+0000 7fa9fb233700 1 --2- 172.21.15.64:0/3346618199 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa9f40aead0 0x7fa9f40ace70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:15.877 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.870+0000 7fa9fb233700 1 --2- 172.21.15.64:0/3346618199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9f405fd40 0x7fa9f40aa490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:15.877 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.870+0000 7fa9fb233700 1 --2- 172.21.15.64:0/3346618199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9f405f960 0x7fa9f414cd10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:15.877 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.870+0000 7fa9fb233700 1 -- 172.21.15.64:0/3346618199 >> 172.21.15.64:0/3346618199 conn(0x7fa9f40b6430 msgr2=0x7fa9f40b0180 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:15.877 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.870+0000 7fa9fb233700 1 -- 172.21.15.64:0/3346618199 shutdown_connections 2022-06-29T15:42:15.877 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.870+0000 7fa9fb233700 1 -- 172.21.15.64:0/3346618199 wait complete. 2022-06-29T15:42:15.891 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --cluster ceph osd new 55b24fd7-bb71-4a01-b3fa-a09caa772a62 3 2022-06-29T15:42:15.990 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.986+0000 7fe13eaca700 1 Processor -- start 2022-06-29T15:42:15.991 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.986+0000 7fe13eaca700 1 -- start start 2022-06-29T15:42:15.994 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.986+0000 7fe13eaca700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe1380aec70 0x7fe13805fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:15.994 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.986+0000 7fe13eaca700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe138146fb0 0x7fe1380b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:15.994 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.986+0000 7fe13eaca700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1380afa70 con 0x7fe1380b0450 2022-06-29T15:42:15.994 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.986+0000 7fe13eaca700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1380afd10 con 0x7fe1380aec70 2022-06-29T15:42:15.995 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.986+0000 7fe13eaca700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1380affb0 con 0x7fe138146fb0 2022-06-29T15:42:15.995 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.986+0000 7fe13dac8700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe1380aec70 0x7fe13805fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:15.995 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.986+0000 7fe13d2c7700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe138146fb0 0x7fe1380b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:15.995 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.986+0000 7fe13dac8700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe1380aec70 0x7fe13805fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42158/0 (socket says 172.21.15.64:42158) 2022-06-29T15:42:15.995 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.986+0000 7fe13dac8700 1 -- 172.21.15.64:0/270893673 learned_addr learned my addr 172.21.15.64:0/270893673 (peer_addr_for_me v2:172.21.15.64:0/0) 2022-06-29T15:42:15.996 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.986+0000 7fe13d2c7700 1 --2- 172.21.15.64:0/270893673 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe138146fb0 0x7fe1380b0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.64:51710/0 (socket says 172.21.15.64:51710) 2022-06-29T15:42:15.996 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.986+0000 7fe13d2c7700 1 -- 172.21.15.64:0/270893673 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe138146fb0 msgr2=0x7fe1380b0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2022-06-29T15:42:15.996 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.986+0000 7fe13d2c7700 1 -- 172.21.15.64:0/270893673 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe138146fb0 msgr2=0x7fe1380b0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-06-29T15:42:15.996 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.986+0000 7fe13d2c7700 1 --2- 172.21.15.64:0/270893673 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe138146fb0 0x7fe1380b0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-29T15:42:15.997 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.986+0000 7fe13d2c7700 1 --2- 172.21.15.64:0/270893673 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe138146fb0 0x7fe1380b0dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2022-06-29T15:42:15.997 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.990+0000 7fe13dac8700 1 -- 172.21.15.64:0/270893673 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe138146fb0 msgr2=0x7fe1380b0dd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-06-29T15:42:15.997 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.990+0000 7fe13dac8700 1 --2- 172.21.15.64:0/270893673 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe138146fb0 0x7fe1380b0dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:15.997 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.990+0000 7fe13dac8700 1 -- 172.21.15.64:0/270893673 >> v1:172.21.15.47:6789/0 conn(0x7fe1380b0450 legacy=0x7fe1381402a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:15.997 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.990+0000 7fe13dac8700 1 -- 172.21.15.64:0/270893673 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1380af050 con 0x7fe1380aec70 2022-06-29T15:42:15.998 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.990+0000 7fe13dac8700 1 --2- 172.21.15.64:0/270893673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1380aec70 0x7fe13805fd00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7fe134002f70 tx=0x7fe134006eb0).ready entity=mon.1 client_cookie=faf8c14fd6d7a4b2 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:15.998 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.990+0000 7fe13cac6700 1 -- 172.21.15.64:0/270893673 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe13400d5c0 con 0x7fe1380aec70 2022-06-29T15:42:15.998 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.990+0000 7fe13cac6700 1 -- 172.21.15.64:0/270893673 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe134000e90 con 0x7fe1380aec70 2022-06-29T15:42:15.998 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.990+0000 7fe13cac6700 1 -- 172.21.15.64:0/270893673 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe13400d950 con 0x7fe1380aec70 2022-06-29T15:42:15.998 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.990+0000 7fe13eaca700 1 -- 172.21.15.64:0/270893673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1380aec70 msgr2=0x7fe13805fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:15.999 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.990+0000 7fe13eaca700 1 --2- 172.21.15.64:0/270893673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1380aec70 0x7fe13805fd00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7fe134002f70 tx=0x7fe134006eb0).stop 2022-06-29T15:42:15.999 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.990+0000 7fe13eaca700 1 -- 172.21.15.64:0/270893673 shutdown_connections 2022-06-29T15:42:15.999 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.990+0000 7fe13eaca700 1 --2- 172.21.15.64:0/270893673 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe138146fb0 0x7fe1380b0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:15.999 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.990+0000 7fe13eaca700 1 --2- 172.21.15.64:0/270893673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1380aec70 0x7fe13805fd00 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:16.000 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.990+0000 7fe13eaca700 1 -- 172.21.15.64:0/270893673 >> 172.21.15.64:0/270893673 conn(0x7fe1380b6430 msgr2=0x7fe1380b6810 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:16.000 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.990+0000 7fe13eaca700 1 -- 172.21.15.64:0/270893673 shutdown_connections 2022-06-29T15:42:16.000 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.990+0000 7fe13eaca700 1 -- 172.21.15.64:0/270893673 wait complete. 2022-06-29T15:42:16.003 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.990+0000 7fe13eaca700 1 Processor -- start 2022-06-29T15:42:16.003 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.990+0000 7fe13eaca700 1 -- start start 2022-06-29T15:42:16.004 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.994+0000 7fe13eaca700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe1380aec70 0x7fe138065f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:16.004 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.994+0000 7fe13eaca700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1380b0450 0x7fe138155610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:16.004 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.994+0000 7fe13eaca700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe138146fb0 0x7fe138155b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:16.004 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.994+0000 7fe13eaca700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe138142a90 con 0x7fe1380b0450 2022-06-29T15:42:16.004 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.994+0000 7fe13eaca700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe138142d30 con 0x7fe138146fb0 2022-06-29T15:42:16.005 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.994+0000 7fe13eaca700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe138142fd0 con 0x7fe1380aec70 2022-06-29T15:42:16.005 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.994+0000 7fe13dac8700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe1380aec70 0x7fe138065f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:16.006 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.994+0000 7fe13dac8700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe1380aec70 0x7fe138065f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.64:51714/0 (socket says 172.21.15.64:51714) 2022-06-29T15:42:16.006 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.994+0000 7fe13dac8700 1 -- 172.21.15.64:0/2721534111 learned_addr learned my addr 172.21.15.64:0/2721534111 (peer_addr_for_me v2:172.21.15.64:0/0) 2022-06-29T15:42:16.006 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.994+0000 7fe13e2c9700 1 --2- 172.21.15.64:0/2721534111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe138146fb0 0x7fe138155b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:16.006 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.994+0000 7fe13dac8700 1 -- 172.21.15.64:0/2721534111 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe1380aec70 msgr2=0x7fe138065f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2022-06-29T15:42:16.007 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.994+0000 7fe13dac8700 1 -- 172.21.15.64:0/2721534111 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe1380aec70 msgr2=0x7fe138065f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2022-06-29T15:42:16.007 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.994+0000 7fe13dac8700 1 --2- 172.21.15.64:0/2721534111 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe1380aec70 0x7fe138065f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-29T15:42:16.007 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.994+0000 7fe13d2c7700 1 --2- 172.21.15.64:0/2721534111 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1380b0450 0x7fe138155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:16.007 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.994+0000 7fe13dac8700 1 --2- 172.21.15.64:0/2721534111 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe1380aec70 0x7fe138065f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2022-06-29T15:42:16.007 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.994+0000 7fe13e2c9700 1 -- 172.21.15.64:0/2721534111 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe1380aec70 msgr2=0x7fe138065f90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2022-06-29T15:42:16.008 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.994+0000 7fe13e2c9700 1 --2- 172.21.15.64:0/2721534111 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe1380aec70 0x7fe138065f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:16.008 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.994+0000 7fe13e2c9700 1 -- 172.21.15.64:0/2721534111 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1380b0450 msgr2=0x7fe138155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:16.008 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.994+0000 7fe13e2c9700 1 --2- 172.21.15.64:0/2721534111 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1380b0450 0x7fe138155610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:16.009 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.994+0000 7fe13e2c9700 1 -- 172.21.15.64:0/2721534111 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe13400b1e0 con 0x7fe138146fb0 2022-06-29T15:42:16.009 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.994+0000 7fe13d2c7700 1 --2- 172.21.15.64:0/2721534111 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1380b0450 0x7fe138155610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:42:16.009 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.994+0000 7fe13e2c9700 1 --2- 172.21.15.64:0/2721534111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe138146fb0 0x7fe138155b00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7fe1380b0330 tx=0x7fe130007eb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:16.009 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.994+0000 7fe126ffd700 1 -- 172.21.15.64:0/2721534111 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe13000e4a0 con 0x7fe138146fb0 2022-06-29T15:42:16.010 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.994+0000 7fe126ffd700 1 -- 172.21.15.64:0/2721534111 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe13000e600 con 0x7fe138146fb0 2022-06-29T15:42:16.010 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.994+0000 7fe13eaca700 1 -- 172.21.15.64:0/2721534111 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe138143270 con 0x7fe138146fb0 2022-06-29T15:42:16.010 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.994+0000 7fe126ffd700 1 -- 172.21.15.64:0/2721534111 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe13000e960 con 0x7fe138146fb0 2022-06-29T15:42:16.010 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.994+0000 7fe13eaca700 1 -- 172.21.15.64:0/2721534111 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe138064820 con 0x7fe138146fb0 2022-06-29T15:42:16.010 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.994+0000 7fe126ffd700 1 -- 172.21.15.64:0/2721534111 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7fe130010070 con 0x7fe138146fb0 2022-06-29T15:42:16.011 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.994+0000 7fe126ffd700 1 -- 172.21.15.64:0/2721534111 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1099+0+0 (secure 0 0 0) 0x7fe13001a820 con 0x7fe138146fb0 2022-06-29T15:42:16.011 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.994+0000 7fe13eaca700 1 -- 172.21.15.64:0/2721534111 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe138143400 con 0x7fe138146fb0 2022-06-29T15:42:16.011 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:15.998+0000 7fe126ffd700 1 -- 172.21.15.64:0/2721534111 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7fe138064820 con 0x7fe138146fb0 2022-06-29T15:42:16.073 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.070+0000 7fe13eaca700 1 -- 172.21.15.64:0/2721534111 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd new", "uuid": "55b24fd7-bb71-4a01-b3fa-a09caa772a62", "id": 3} v 0) v1 -- 0x7fe1380573d0 con 0x7fe138146fb0 2022-06-29T15:42:16.081 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.078+0000 7fe126ffd700 1 -- 172.21.15.64:0/2721534111 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "55b24fd7-bb71-4a01-b3fa-a09caa772a62", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe130017070 con 0x7fe138146fb0 2022-06-29T15:42:16.081 INFO:teuthology.orchestra.run.smithi064.stdout:3 2022-06-29T15:42:16.083 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.078+0000 7fe13eaca700 1 -- 172.21.15.64:0/2721534111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe138146fb0 msgr2=0x7fe138155b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:16.083 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.078+0000 7fe13eaca700 1 --2- 172.21.15.64:0/2721534111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe138146fb0 0x7fe138155b00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7fe1380b0330 tx=0x7fe130007eb0).stop 2022-06-29T15:42:16.083 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.078+0000 7fe13eaca700 1 -- 172.21.15.64:0/2721534111 shutdown_connections 2022-06-29T15:42:16.083 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.078+0000 7fe13eaca700 1 --2- 172.21.15.64:0/2721534111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe138146fb0 0x7fe138155b00 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:16.084 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.078+0000 7fe13eaca700 1 --2- 172.21.15.64:0/2721534111 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1380b0450 0x7fe138155610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:16.084 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.078+0000 7fe13eaca700 1 --2- 172.21.15.64:0/2721534111 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe1380aec70 0x7fe138065f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:16.084 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.078+0000 7fe13eaca700 1 -- 172.21.15.64:0/2721534111 >> 172.21.15.64:0/2721534111 conn(0x7fe1380b6430 msgr2=0x7fe1380b5630 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:16.084 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.078+0000 7fe13eaca700 1 -- 172.21.15.64:0/2721534111 shutdown_connections 2022-06-29T15:42:16.084 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.078+0000 7fe13eaca700 1 -- 172.21.15.64:0/2721534111 wait complete. 2022-06-29T15:42:16.096 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --cluster ceph osd new a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3 4 2022-06-29T15:42:16.196 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.194+0000 7f2eff59e700 1 Processor -- start 2022-06-29T15:42:16.196 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.194+0000 7f2eff59e700 1 -- start start 2022-06-29T15:42:16.200 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.194+0000 7f2eff59e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2f000b6100 0x7f2f001431a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:16.200 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.194+0000 7f2eff59e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2f000b64e0 0x7f2f000b1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:16.201 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.194+0000 7f2eff59e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f000b20c0 con 0x7f2f00146eb0 2022-06-29T15:42:16.201 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.194+0000 7f2eff59e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f000b2350 con 0x7f2f000b64e0 2022-06-29T15:42:16.201 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.194+0000 7f2eff59e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f000b25f0 con 0x7f2f000b6100 2022-06-29T15:42:16.202 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.194+0000 7f2efe59c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2f000b6100 0x7f2f001431a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:16.202 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.194+0000 7f2efe59c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2f000b6100 0x7f2f001431a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.64:51720/0 (socket says 172.21.15.64:51720) 2022-06-29T15:42:16.202 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.194+0000 7f2efe59c700 1 -- 172.21.15.64:0/1779320834 learned_addr learned my addr 172.21.15.64:0/1779320834 (peer_addr_for_me v2:172.21.15.64:0/0) 2022-06-29T15:42:16.202 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.194+0000 7f2efe59c700 1 -- 172.21.15.64:0/1779320834 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2f000b6100 msgr2=0x7f2f001431a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-06-29T15:42:16.203 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.194+0000 7f2efe59c700 1 -- 172.21.15.64:0/1779320834 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2f000b6100 msgr2=0x7f2f001431a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-06-29T15:42:16.203 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.194+0000 7f2efe59c700 1 --2- 172.21.15.64:0/1779320834 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2f000b6100 0x7f2f001431a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-29T15:42:16.203 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.194+0000 7f2ef7fff700 1 --2- 172.21.15.64:0/1779320834 >> v2:172.21.15.47:3300/0 conn(0x7f2f000b64e0 0x7f2f000b1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:16.203 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.194+0000 7f2efe59c700 1 --2- 172.21.15.64:0/1779320834 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2f000b6100 0x7f2f001431a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2022-06-29T15:42:16.204 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.194+0000 7f2efdd9b700 1 -- 172.21.15.64:0/1779320834 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1005969357 0 0) 0x7f2f000b20c0 con 0x7f2f00146eb0 2022-06-29T15:42:16.204 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.194+0000 7f2efdd9b700 1 -- 172.21.15.64:0/1779320834 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ee0003260 con 0x7f2f00146eb0 2022-06-29T15:42:16.204 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.194+0000 7f2ef7fff700 1 -- 172.21.15.64:0/1779320834 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2f000b6100 msgr2=0x7f2f001431a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-06-29T15:42:16.204 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.194+0000 7f2efdd9b700 1 -- 172.21.15.64:0/1779320834 <== mon.1 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 943298799 0 0) 0x7f2ee0003260 con 0x7f2f00146eb0 2022-06-29T15:42:16.205 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.194+0000 7f2ef7fff700 1 --2- 172.21.15.64:0/1779320834 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2f000b6100 0x7f2f001431a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:16.205 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.194+0000 7f2ef7fff700 1 -- 172.21.15.64:0/1779320834 >> v1:172.21.15.64:6789/0 conn(0x7f2f00146eb0 legacy=0x7f2f00066fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:16.205 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.194+0000 7f2ef7fff700 1 -- 172.21.15.64:0/1779320834 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f00146770 con 0x7f2f000b64e0 2022-06-29T15:42:16.205 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.194+0000 7f2ef7fff700 1 --2- 172.21.15.64:0/1779320834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f000b64e0 0x7f2f000b1bd0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f2ef8001b80 tx=0x7f2ee8006df0).ready entity=mon.0 client_cookie=1f1c4f8bfae40408 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:16.206 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.194+0000 7f2efdd9b700 1 -- 172.21.15.64:0/1779320834 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2ef80019e0 con 0x7f2f000b64e0 2022-06-29T15:42:16.206 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.194+0000 7f2efdd9b700 1 -- 172.21.15.64:0/1779320834 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ee8000e90 con 0x7f2f000b64e0 2022-06-29T15:42:16.206 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.194+0000 7f2efdd9b700 1 -- 172.21.15.64:0/1779320834 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2ee800d4a0 con 0x7f2f000b64e0 2022-06-29T15:42:16.206 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.194+0000 7f2eff59e700 1 -- 172.21.15.64:0/1779320834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f000b64e0 msgr2=0x7f2f000b1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:16.207 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.194+0000 7f2eff59e700 1 --2- 172.21.15.64:0/1779320834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f000b64e0 0x7f2f000b1bd0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f2ef8001b80 tx=0x7f2ee8006df0).stop 2022-06-29T15:42:16.207 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.194+0000 7f2eff59e700 1 -- 172.21.15.64:0/1779320834 shutdown_connections 2022-06-29T15:42:16.207 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.194+0000 7f2eff59e700 1 --2- 172.21.15.64:0/1779320834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f000b64e0 0x7f2f000b1bd0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:16.207 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.194+0000 7f2eff59e700 1 --2- 172.21.15.64:0/1779320834 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2f000b6100 0x7f2f001431a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:16.207 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.194+0000 7f2eff59e700 1 -- 172.21.15.64:0/1779320834 >> 172.21.15.64:0/1779320834 conn(0x7f2f00060e00 msgr2=0x7f2f000611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:16.208 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.194+0000 7f2eff59e700 1 -- 172.21.15.64:0/1779320834 shutdown_connections 2022-06-29T15:42:16.208 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.194+0000 7f2eff59e700 1 -- 172.21.15.64:0/1779320834 wait complete. 2022-06-29T15:42:16.211 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.198+0000 7f2eff59e700 1 Processor -- start 2022-06-29T15:42:16.211 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.198+0000 7f2eff59e700 1 -- start start 2022-06-29T15:42:16.212 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.198+0000 7f2eff59e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2f000b6100 0x7f2f000afeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:16.212 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.198+0000 7f2eff59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f000b64e0 0x7f2f00141660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:16.212 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.198+0000 7f2eff59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f00146eb0 0x7f2f00141b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:16.212 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.198+0000 7f2eff59e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f00142410 con 0x7f2f00146eb0 2022-06-29T15:42:16.213 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.198+0000 7f2eff59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f001411c0 con 0x7f2f000b64e0 2022-06-29T15:42:16.213 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.198+0000 7f2eff59e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f000b03a0 con 0x7f2f000b6100 2022-06-29T15:42:16.213 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.198+0000 7f2ef7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f000b64e0 0x7f2f00141660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:16.213 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.198+0000 7f2efe59c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2f000b6100 0x7f2f000afeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:16.214 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.198+0000 7f2ef7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f000b64e0 0x7f2f00141660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42178/0 (socket says 172.21.15.64:42178) 2022-06-29T15:42:16.214 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.198+0000 7f2ef7fff700 1 -- 172.21.15.64:0/1040307067 learned_addr learned my addr 172.21.15.64:0/1040307067 (peer_addr_for_me v2:172.21.15.64:0/0) 2022-06-29T15:42:16.214 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.198+0000 7f2efe59c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2f000b6100 0x7f2f000afeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.64:51726/0 (socket says 172.21.15.64:51726) 2022-06-29T15:42:16.214 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.198+0000 7f2efed9d700 1 --2- 172.21.15.64:0/1040307067 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f00146eb0 0x7f2f00141b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:16.215 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.198+0000 7f2efe59c700 1 -- 172.21.15.64:0/1040307067 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2f000b6100 msgr2=0x7f2f000afeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2022-06-29T15:42:16.215 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.198+0000 7f2efe59c700 1 -- 172.21.15.64:0/1040307067 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2f000b6100 msgr2=0x7f2f000afeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2022-06-29T15:42:16.215 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.198+0000 7f2efe59c700 1 --2- 172.21.15.64:0/1040307067 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2f000b6100 0x7f2f000afeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-29T15:42:16.216 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.198+0000 7f2efe59c700 1 --2- 172.21.15.64:0/1040307067 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2f000b6100 0x7f2f000afeb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2022-06-29T15:42:16.216 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.198+0000 7f2ef7fff700 1 -- 172.21.15.64:0/1040307067 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2f000b6100 msgr2=0x7f2f000afeb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2022-06-29T15:42:16.216 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.198+0000 7f2ef7fff700 1 --2- 172.21.15.64:0/1040307067 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2f000b6100 0x7f2f000afeb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:16.216 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.198+0000 7f2ef7fff700 1 -- 172.21.15.64:0/1040307067 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f00146eb0 msgr2=0x7f2f00141b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:16.217 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.198+0000 7f2ef7fff700 1 --2- 172.21.15.64:0/1040307067 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f00146eb0 0x7f2f00141b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:16.217 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.202+0000 7f2ef7fff700 1 -- 172.21.15.64:0/1040307067 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee800b180 con 0x7f2f000b64e0 2022-06-29T15:42:16.217 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.202+0000 7f2ef7fff700 1 --2- 172.21.15.64:0/1040307067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f000b64e0 0x7f2f00141660 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f2ef8001b80 tx=0x7f2ee8014000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:16.217 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.202+0000 7f2ef77fe700 1 -- 172.21.15.64:0/1040307067 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2ee8016070 con 0x7f2f000b64e0 2022-06-29T15:42:16.218 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.202+0000 7f2ef77fe700 1 -- 172.21.15.64:0/1040307067 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ee8013ca0 con 0x7f2f000b64e0 2022-06-29T15:42:16.218 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.202+0000 7f2eff59e700 1 -- 172.21.15.64:0/1040307067 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f00142db0 con 0x7f2f000b64e0 2022-06-29T15:42:16.218 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.202+0000 7f2ef77fe700 1 -- 172.21.15.64:0/1040307067 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2ee801e450 con 0x7f2f000b64e0 2022-06-29T15:42:16.218 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.202+0000 7f2eff59e700 1 -- 172.21.15.64:0/1040307067 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f0013da10 con 0x7f2f000b64e0 2022-06-29T15:42:16.219 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.202+0000 7f2ef77fe700 1 -- 172.21.15.64:0/1040307067 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f2ee800f030 con 0x7f2f000b64e0 2022-06-29T15:42:16.219 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.202+0000 7f2ef77fe700 1 -- 172.21.15.64:0/1040307067 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1214+0+0 (secure 0 0 0) 0x7f2ee801e780 con 0x7f2f000b64e0 2022-06-29T15:42:16.219 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.202+0000 7f2eff59e700 1 -- 172.21.15.64:0/1040307067 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f001435c0 con 0x7f2f000b64e0 2022-06-29T15:42:16.219 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.206+0000 7f2ef77fe700 1 -- 172.21.15.64:0/1040307067 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f2ee803d870 con 0x7f2f000b64e0 2022-06-29T15:42:16.280 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.278+0000 7f2eff59e700 1 -- 172.21.15.64:0/1040307067 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3", "id": 4} v 0) v1 -- 0x7f2f00141350 con 0x7f2f000b64e0 2022-06-29T15:42:16.287 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.282+0000 7f2ef77fe700 1 -- 172.21.15.64:0/1040307067 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2ee802d070 con 0x7f2f000b64e0 2022-06-29T15:42:16.287 INFO:teuthology.orchestra.run.smithi064.stdout:4 2022-06-29T15:42:16.289 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.286+0000 7f2eff59e700 1 -- 172.21.15.64:0/1040307067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f000b64e0 msgr2=0x7f2f00141660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:16.289 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.286+0000 7f2eff59e700 1 --2- 172.21.15.64:0/1040307067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f000b64e0 0x7f2f00141660 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f2ef8001b80 tx=0x7f2ee8014000).stop 2022-06-29T15:42:16.289 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.286+0000 7f2eff59e700 1 -- 172.21.15.64:0/1040307067 shutdown_connections 2022-06-29T15:42:16.290 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.286+0000 7f2eff59e700 1 --2- 172.21.15.64:0/1040307067 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f00146eb0 0x7f2f00141b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:16.290 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.286+0000 7f2eff59e700 1 --2- 172.21.15.64:0/1040307067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f000b64e0 0x7f2f00141660 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:16.290 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.286+0000 7f2eff59e700 1 --2- 172.21.15.64:0/1040307067 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2f000b6100 0x7f2f000afeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:16.290 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.286+0000 7f2eff59e700 1 -- 172.21.15.64:0/1040307067 >> 172.21.15.64:0/1040307067 conn(0x7f2f00060e00 msgr2=0x7f2f0005c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:16.290 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.286+0000 7f2eff59e700 1 -- 172.21.15.64:0/1040307067 shutdown_connections 2022-06-29T15:42:16.291 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.286+0000 7f2eff59e700 1 -- 172.21.15.64:0/1040307067 wait complete. 2022-06-29T15:42:16.303 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --cluster ceph osd new f77beebe-6cca-4244-b2d4-677825ebf919 5 2022-06-29T15:42:16.400 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.398+0000 7f6b833bf700 1 Processor -- start 2022-06-29T15:42:16.401 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.398+0000 7f6b833bf700 1 -- start start 2022-06-29T15:42:16.405 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.398+0000 7f6b833bf700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6b7c0b32a0 0x7f6b7c141550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:16.405 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.398+0000 7f6b833bf700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b7c145610 con 0x7f6b7c0b4450 2022-06-29T15:42:16.406 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.398+0000 7f6b833bf700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b7c0b4830 con 0x7f6b7c0ab6c0 2022-06-29T15:42:16.406 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.398+0000 7f6b833bf700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7c0b1e10 con 0x7f6b7c0b32a0 2022-06-29T15:42:16.406 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.398+0000 7f6b8095a700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6b7c0b32a0 0x7f6b7c141550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:16.406 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.398+0000 7f6b8095a700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6b7c0b32a0 0x7f6b7c141550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.64:51734/0 (socket says 172.21.15.64:51734) 2022-06-29T15:42:16.407 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.398+0000 7f6b8095a700 1 -- 172.21.15.64:0/685179725 learned_addr learned my addr 172.21.15.64:0/685179725 (peer_addr_for_me v2:172.21.15.64:0/0) 2022-06-29T15:42:16.407 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.398+0000 7f6b8095a700 1 -- 172.21.15.64:0/685179725 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6b7c0b32a0 msgr2=0x7f6b7c141550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2022-06-29T15:42:16.407 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.398+0000 7f6b8095a700 1 -- 172.21.15.64:0/685179725 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6b7c0b32a0 msgr2=0x7f6b7c141550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-06-29T15:42:16.407 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.398+0000 7f6b8095a700 1 --2- 172.21.15.64:0/685179725 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6b7c0b32a0 0x7f6b7c141550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-29T15:42:16.408 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.398+0000 7f6b8095a700 1 --2- 172.21.15.64:0/685179725 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6b7c0b32a0 0x7f6b7c141550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2022-06-29T15:42:16.408 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.398+0000 7f6b7bfff700 1 -- 172.21.15.64:0/685179725 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3036729987 0 0) 0x7f6b7c0b4830 con 0x7f6b7c0ab6c0 2022-06-29T15:42:16.408 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.398+0000 7f6b7bfff700 1 -- 172.21.15.64:0/685179725 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b68003260 con 0x7f6b7c0ab6c0 2022-06-29T15:42:16.408 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.398+0000 7f6b7bfff700 1 -- 172.21.15.64:0/685179725 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2800526326 0 0) 0x7f6b7c145610 con 0x7f6b7c0b4450 2022-06-29T15:42:16.408 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.398+0000 7f6b7bfff700 1 -- 172.21.15.64:0/685179725 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b7c0b4830 con 0x7f6b7c0b4450 2022-06-29T15:42:16.409 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.398+0000 7f6b7bfff700 1 -- 172.21.15.64:0/685179725 <== mon.1 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1563501825 0 0) 0x7f6b68003260 con 0x7f6b7c0ab6c0 2022-06-29T15:42:16.409 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.398+0000 7f6b7bfff700 1 -- 172.21.15.64:0/685179725 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6b7c0b32a0 msgr2=0x7f6b7c141550 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-06-29T15:42:16.409 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.398+0000 7f6b7bfff700 1 --2- 172.21.15.64:0/685179725 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6b7c0b32a0 0x7f6b7c141550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:16.409 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.398+0000 7f6b7bfff700 1 -- 172.21.15.64:0/685179725 >> v1:172.21.15.47:6789/0 conn(0x7f6b7c0b4450 legacy=0x7f6b7c0b1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:16.410 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.398+0000 7f6b7bfff700 1 -- 172.21.15.64:0/685179725 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b7c145370 con 0x7f6b7c0ab6c0 2022-06-29T15:42:16.410 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.398+0000 7f6b7bfff700 1 -- 172.21.15.64:0/685179725 <== mon.1 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7f6b6c001660 con 0x7f6b7c0ab6c0 2022-06-29T15:42:16.410 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.398+0000 7f6b7bfff700 1 -- 172.21.15.64:0/685179725 >> v1:172.21.15.64:6789/0 conn(0x7f6b7c0ab6c0 legacy=0x7f6b7c141a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:16.411 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.398+0000 7f6b7bfff700 1 --2- 172.21.15.64:0/685179725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b680038e0 0x7f6b68003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:16.412 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.398+0000 7f6b7bfff700 1 --2- 172.21.15.64:0/685179725 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6b7c0b32a0 0x7f6b68004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:16.412 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.398+0000 7f6b7bfff700 1 --2- 172.21.15.64:0/685179725 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b68004a40 0x7f6b68004ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:16.413 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.398+0000 7f6b7bfff700 1 -- 172.21.15.64:0/685179725 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7c0b1e10 con 0x7f6b680038e0 2022-06-29T15:42:16.413 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.398+0000 7f6b7bfff700 1 -- 172.21.15.64:0/685179725 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6800b8e0 con 0x7f6b68004a40 2022-06-29T15:42:16.414 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.398+0000 7f6b7bfff700 1 -- 172.21.15.64:0/685179725 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6800ba70 con 0x7f6b7c0b32a0 2022-06-29T15:42:16.414 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.398+0000 7f6b8195c700 1 --2- 172.21.15.64:0/685179725 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b68004a40 0x7f6b68004ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:16.415 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.398+0000 7f6b8095a700 1 --2- 172.21.15.64:0/685179725 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6b7c0b32a0 0x7f6b68004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:16.415 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.398+0000 7f6b8115b700 1 --2- 172.21.15.64:0/685179725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b680038e0 0x7f6b68003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:16.415 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.398+0000 7f6b8195c700 1 -- 172.21.15.64:0/685179725 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6b7c0b32a0 msgr2=0x7f6b68004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:16.416 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.398+0000 7f6b8195c700 1 --2- 172.21.15.64:0/685179725 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6b7c0b32a0 0x7f6b68004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:16.416 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.398+0000 7f6b8195c700 1 -- 172.21.15.64:0/685179725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b680038e0 msgr2=0x7f6b68003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:16.417 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.398+0000 7f6b8195c700 1 --2- 172.21.15.64:0/685179725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b680038e0 0x7f6b68003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:16.417 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.398+0000 7f6b8195c700 1 -- 172.21.15.64:0/685179725 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b68003260 con 0x7f6b68004a40 2022-06-29T15:42:16.417 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.398+0000 7f6b8095a700 1 --2- 172.21.15.64:0/685179725 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6b7c0b32a0 0x7f6b68004500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:42:16.418 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.402+0000 7f6b8195c700 1 --2- 172.21.15.64:0/685179725 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b68004a40 0x7f6b68004ec0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f6b6c001430 tx=0x7f6b7c145500).ready entity=mon.1 client_cookie=c8655725c8e9919 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:16.418 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.402+0000 7f6b7bfff700 1 -- 172.21.15.64:0/685179725 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b6c007e90 con 0x7f6b68004a40 2022-06-29T15:42:16.418 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.402+0000 7f6b7bfff700 1 -- 172.21.15.64:0/685179725 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b6c007030 con 0x7f6b68004a40 2022-06-29T15:42:16.419 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.402+0000 7f6b7bfff700 1 -- 172.21.15.64:0/685179725 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b6c007390 con 0x7f6b68004a40 2022-06-29T15:42:16.419 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.402+0000 7f6b833bf700 1 -- 172.21.15.64:0/685179725 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b68004a40 msgr2=0x7f6b68004ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:16.419 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.402+0000 7f6b833bf700 1 --2- 172.21.15.64:0/685179725 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b68004a40 0x7f6b68004ec0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f6b6c001430 tx=0x7f6b7c145500).stop 2022-06-29T15:42:16.419 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.402+0000 7f6b8195c700 1 -- 172.21.15.64:0/685179725 reap_dead start 2022-06-29T15:42:16.419 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.402+0000 7f6b833bf700 1 -- 172.21.15.64:0/685179725 shutdown_connections 2022-06-29T15:42:16.420 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.402+0000 7f6b833bf700 1 -- 172.21.15.64:0/685179725 >> 172.21.15.64:0/685179725 conn(0x7f6b7c063290 msgr2=0x7f6b7c05c510 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:16.420 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.402+0000 7f6b833bf700 1 -- 172.21.15.64:0/685179725 shutdown_connections 2022-06-29T15:42:16.420 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.402+0000 7f6b833bf700 1 -- 172.21.15.64:0/685179725 wait complete. 2022-06-29T15:42:16.424 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.402+0000 7f6b833bf700 1 Processor -- start 2022-06-29T15:42:16.424 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.402+0000 7f6b833bf700 1 -- start start 2022-06-29T15:42:16.425 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.402+0000 7f6b833bf700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6b68004a40 0x7f6b7c150b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:16.425 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.402+0000 7f6b833bf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b7c0651e0 0x7f6b7c066850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:16.425 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.402+0000 7f6b833bf700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b7c145d20 0x7f6b7c146be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:16.425 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.402+0000 7f6b833bf700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7c13cc30 con 0x7f6b7c145d20 2022-06-29T15:42:16.426 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.402+0000 7f6b833bf700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7c1465d0 con 0x7f6b7c0651e0 2022-06-29T15:42:16.426 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.402+0000 7f6b833bf700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7c13e930 con 0x7f6b68004a40 2022-06-29T15:42:16.426 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.402+0000 7f6b8095a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b7c0651e0 0x7f6b7c066850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:16.426 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.402+0000 7f6b8095a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b7c0651e0 0x7f6b7c066850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42194/0 (socket says 172.21.15.64:42194) 2022-06-29T15:42:16.427 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.402+0000 7f6b8095a700 1 -- 172.21.15.64:0/2821956819 learned_addr learned my addr 172.21.15.64:0/2821956819 (peer_addr_for_me v2:172.21.15.64:0/0) 2022-06-29T15:42:16.427 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.402+0000 7f6b8115b700 1 --2- 172.21.15.64:0/2821956819 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6b68004a40 0x7f6b7c150b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:16.427 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.402+0000 7f6b8195c700 1 --2- 172.21.15.64:0/2821956819 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b7c145d20 0x7f6b7c146be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:16.428 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.406+0000 7f6b8115b700 1 -- 172.21.15.64:0/2821956819 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6b68004a40 msgr2=0x7f6b7c150b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2022-06-29T15:42:16.428 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.406+0000 7f6b8095a700 1 -- 172.21.15.64:0/2821956819 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6b68004a40 msgr2=0x7f6b7c150b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:16.429 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.406+0000 7f6b8115b700 1 -- 172.21.15.64:0/2821956819 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6b68004a40 msgr2=0x7f6b7c150b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2022-06-29T15:42:16.429 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.406+0000 7f6b8115b700 1 --2- 172.21.15.64:0/2821956819 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6b68004a40 0x7f6b7c150b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-29T15:42:16.429 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.406+0000 7f6b8115b700 1 --2- 172.21.15.64:0/2821956819 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6b68004a40 0x7f6b7c150b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2022-06-29T15:42:16.430 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.406+0000 7f6b8095a700 1 --2- 172.21.15.64:0/2821956819 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6b68004a40 0x7f6b7c150b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:16.430 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.406+0000 7f6b8095a700 1 -- 172.21.15.64:0/2821956819 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b7c145d20 msgr2=0x7f6b7c146be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:16.431 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.406+0000 7f6b8095a700 1 --2- 172.21.15.64:0/2821956819 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b7c145d20 0x7f6b7c146be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:16.431 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.406+0000 7f6b8095a700 1 -- 172.21.15.64:0/2821956819 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b7c146800 con 0x7f6b7c0651e0 2022-06-29T15:42:16.432 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.406+0000 7f6b8195c700 1 --2- 172.21.15.64:0/2821956819 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b7c145d20 0x7f6b7c146be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:42:16.432 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.406+0000 7f6b8095a700 1 --2- 172.21.15.64:0/2821956819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b7c0651e0 0x7f6b7c066850 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f6b7000c000 tx=0x7f6b70008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:16.432 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.406+0000 7f6b7a7fc700 1 -- 172.21.15.64:0/2821956819 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b7000eac0 con 0x7f6b7c0651e0 2022-06-29T15:42:16.433 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.406+0000 7f6b7a7fc700 1 -- 172.21.15.64:0/2821956819 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b7000ec20 con 0x7f6b7c0651e0 2022-06-29T15:42:16.433 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.406+0000 7f6b7a7fc700 1 -- 172.21.15.64:0/2821956819 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b70014470 con 0x7f6b7c0651e0 2022-06-29T15:42:16.433 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.406+0000 7f6b833bf700 1 -- 172.21.15.64:0/2821956819 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b7c1503c0 con 0x7f6b7c0651e0 2022-06-29T15:42:16.434 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.406+0000 7f6b833bf700 1 -- 172.21.15.64:0/2821956819 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b7c1500f0 con 0x7f6b7c0651e0 2022-06-29T15:42:16.434 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.406+0000 7f6b7a7fc700 1 -- 172.21.15.64:0/2821956819 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f6b700147a0 con 0x7f6b7c0651e0 2022-06-29T15:42:16.434 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.406+0000 7f6b7a7fc700 1 -- 172.21.15.64:0/2821956819 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1329+0+0 (secure 0 0 0) 0x7f6b7001ea10 con 0x7f6b7c0651e0 2022-06-29T15:42:16.435 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.406+0000 7f6b833bf700 1 -- 172.21.15.64:0/2821956819 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b7c0573d0 con 0x7f6b7c0651e0 2022-06-29T15:42:16.435 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.410+0000 7f6b7a7fc700 1 -- 172.21.15.64:0/2821956819 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f6b7001ecf0 con 0x7f6b7c0651e0 2022-06-29T15:42:16.481 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.478+0000 7f6b833bf700 1 -- 172.21.15.64:0/2821956819 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f77beebe-6cca-4244-b2d4-677825ebf919", "id": 5} v 0) v1 -- 0x7f6b7c06a0f0 con 0x7f6b7c0651e0 2022-06-29T15:42:16.488 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.486+0000 7f6b7a7fc700 1 -- 172.21.15.64:0/2821956819 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f77beebe-6cca-4244-b2d4-677825ebf919", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6b7001e460 con 0x7f6b7c0651e0 2022-06-29T15:42:16.488 INFO:teuthology.orchestra.run.smithi064.stdout:5 2022-06-29T15:42:16.490 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.486+0000 7f6b833bf700 1 -- 172.21.15.64:0/2821956819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b7c0651e0 msgr2=0x7f6b7c066850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:16.490 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.486+0000 7f6b833bf700 1 --2- 172.21.15.64:0/2821956819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b7c0651e0 0x7f6b7c066850 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f6b7000c000 tx=0x7f6b70008040).stop 2022-06-29T15:42:16.490 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.486+0000 7f6b833bf700 1 -- 172.21.15.64:0/2821956819 shutdown_connections 2022-06-29T15:42:16.491 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.486+0000 7f6b833bf700 1 --2- 172.21.15.64:0/2821956819 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b7c145d20 0x7f6b7c146be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:16.491 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.486+0000 7f6b833bf700 1 --2- 172.21.15.64:0/2821956819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b7c0651e0 0x7f6b7c066850 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:16.491 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.486+0000 7f6b833bf700 1 --2- 172.21.15.64:0/2821956819 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6b68004a40 0x7f6b7c150b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:16.491 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.486+0000 7f6b833bf700 1 -- 172.21.15.64:0/2821956819 >> 172.21.15.64:0/2821956819 conn(0x7f6b7c063290 msgr2=0x7f6b7c0abba0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:16.492 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.486+0000 7f6b833bf700 1 -- 172.21.15.64:0/2821956819 shutdown_connections 2022-06-29T15:42:16.492 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.486+0000 7f6b833bf700 1 -- 172.21.15.64:0/2821956819 wait complete. 2022-06-29T15:42:16.503 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --cluster ceph osd new af0ca9b3-c02c-4910-834b-bf0b8dc6713c 6 2022-06-29T15:42:16.606 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.602+0000 7faf0035c700 1 Processor -- start 2022-06-29T15:42:16.607 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.602+0000 7faf0035c700 1 -- start start 2022-06-29T15:42:16.608 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.606+0000 7faf0035c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7faef80b1e90 0x7faef8143180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:16.609 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.606+0000 7faf0035c700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faef8061a00 con 0x7faef80b5a50 2022-06-29T15:42:16.609 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.606+0000 7faf0035c700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faef8061ca0 con 0x7faef8146e90 2022-06-29T15:42:16.609 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.606+0000 7faf0035c700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faef8146750 con 0x7faef80b1e90 2022-06-29T15:42:16.609 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.606+0000 7faefe0f8700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7faef80b1e90 0x7faef8143180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:16.610 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.606+0000 7faefe8f9700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7faef8146e90 0x7faef81401f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48874/0 (socket says 172.21.15.64:48874) 2022-06-29T15:42:16.610 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.606+0000 7faefe8f9700 1 -- 172.21.15.64:0/559241585 learned_addr learned my addr 172.21.15.64:0/559241585 (peer_addr_for_me v1:172.21.15.64:0/0) 2022-06-29T15:42:16.611 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.606+0000 7faefe0f8700 1 -- 172.21.15.64:0/559241585 >> v1:172.21.15.64:6789/0 conn(0x7faef8146e90 legacy=0x7faef81401f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:16.611 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.606+0000 7faefe0f8700 1 -- 172.21.15.64:0/559241585 >> v1:172.21.15.47:6789/0 conn(0x7faef80b5a50 legacy=0x7faef80b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:16.611 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.606+0000 7faefe0f8700 1 -- 172.21.15.64:0/559241585 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef81469f0 con 0x7faef80b1e90 2022-06-29T15:42:16.612 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.606+0000 7faefe0f8700 1 --2- 172.21.15.64:0/559241585 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faef80b1e90 0x7faef8143180 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7faef4002f70 tx=0x7faef4006df0).ready entity=mon.1 client_cookie=54a31a10ed7d9954 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:16.619 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.606+0000 7faefd0f6700 1 -- 172.21.15.64:0/559241585 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faef400d610 con 0x7faef80b1e90 2022-06-29T15:42:16.620 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.606+0000 7faefd0f6700 1 -- 172.21.15.64:0/559241585 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faef4000e90 con 0x7faef80b1e90 2022-06-29T15:42:16.620 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.606+0000 7faefd0f6700 1 -- 172.21.15.64:0/559241585 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faef400d960 con 0x7faef80b1e90 2022-06-29T15:42:16.620 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.606+0000 7faf0035c700 1 -- 172.21.15.64:0/559241585 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faef80b1e90 msgr2=0x7faef8143180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:16.621 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.606+0000 7faf0035c700 1 --2- 172.21.15.64:0/559241585 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faef80b1e90 0x7faef8143180 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7faef4002f70 tx=0x7faef4006df0).stop 2022-06-29T15:42:16.621 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.606+0000 7faf0035c700 1 -- 172.21.15.64:0/559241585 shutdown_connections 2022-06-29T15:42:16.622 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.606+0000 7faf0035c700 1 --2- 172.21.15.64:0/559241585 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faef80b1e90 0x7faef8143180 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:16.622 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.606+0000 7faf0035c700 1 -- 172.21.15.64:0/559241585 >> 172.21.15.64:0/559241585 conn(0x7faef80b98f0 msgr2=0x7faef80b9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:16.622 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.606+0000 7faf0035c700 1 -- 172.21.15.64:0/559241585 shutdown_connections 2022-06-29T15:42:16.623 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.606+0000 7faf0035c700 1 -- 172.21.15.64:0/559241585 wait complete. 2022-06-29T15:42:16.623 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.610+0000 7faf0035c700 1 Processor -- start 2022-06-29T15:42:16.624 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.610+0000 7faf0035c700 1 -- start start 2022-06-29T15:42:16.624 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.610+0000 7faf0035c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7faef80b1e90 0x7faef8142e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:16.625 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.610+0000 7faf0035c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faef80b5a50 0x7faef81526e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:16.625 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.610+0000 7faf0035c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faef8146e90 0x7faef812ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:16.626 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.610+0000 7faf0035c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef8155c30 con 0x7faef8146e90 2022-06-29T15:42:16.626 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.610+0000 7faf0035c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef8131610 con 0x7faef80b5a50 2022-06-29T15:42:16.627 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.610+0000 7faf0035c700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faef812d940 con 0x7faef80b1e90 2022-06-29T15:42:16.627 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.610+0000 7faefd8f7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faef80b5a50 0x7faef81526e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:16.627 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.610+0000 7faefe0f8700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7faef80b1e90 0x7faef8142e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:16.628 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.610+0000 7faefd8f7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faef80b5a50 0x7faef81526e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42208/0 (socket says 172.21.15.64:42208) 2022-06-29T15:42:16.628 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.610+0000 7faefd8f7700 1 -- 172.21.15.64:0/517026991 learned_addr learned my addr 172.21.15.64:0/517026991 (peer_addr_for_me v2:172.21.15.64:0/0) 2022-06-29T15:42:16.628 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.610+0000 7faefe8f9700 1 --2- 172.21.15.64:0/517026991 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faef8146e90 0x7faef812ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:16.629 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.610+0000 7faefe0f8700 1 -- 172.21.15.64:0/517026991 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7faef80b1e90 msgr2=0x7faef8142e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2022-06-29T15:42:16.629 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.610+0000 7faefe0f8700 1 -- 172.21.15.64:0/517026991 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7faef80b1e90 msgr2=0x7faef8142e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2022-06-29T15:42:16.629 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.610+0000 7faefe0f8700 1 --2- 172.21.15.64:0/517026991 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7faef80b1e90 0x7faef8142e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-29T15:42:16.630 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.610+0000 7faefe0f8700 1 --2- 172.21.15.64:0/517026991 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7faef80b1e90 0x7faef8142e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2022-06-29T15:42:16.630 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.610+0000 7faefd8f7700 1 -- 172.21.15.64:0/517026991 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7faef80b1e90 msgr2=0x7faef8142e80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2022-06-29T15:42:16.630 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.610+0000 7faefd8f7700 1 --2- 172.21.15.64:0/517026991 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7faef80b1e90 0x7faef8142e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:16.631 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.610+0000 7faefd8f7700 1 -- 172.21.15.64:0/517026991 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faef8146e90 msgr2=0x7faef812ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:16.631 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.610+0000 7faefd8f7700 1 --2- 172.21.15.64:0/517026991 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faef8146e90 0x7faef812ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:16.632 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.610+0000 7faefd8f7700 1 -- 172.21.15.64:0/517026991 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef400b180 con 0x7faef80b5a50 2022-06-29T15:42:16.632 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.610+0000 7faefd8f7700 1 --2- 172.21.15.64:0/517026991 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faef80b5a50 0x7faef81526e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7faef81306b0 tx=0x7faee8007c50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:16.632 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.610+0000 7faee77fe700 1 -- 172.21.15.64:0/517026991 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faee8012400 con 0x7faef80b5a50 2022-06-29T15:42:16.633 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.610+0000 7faee77fe700 1 -- 172.21.15.64:0/517026991 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faee8012560 con 0x7faef80b5a50 2022-06-29T15:42:16.633 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.610+0000 7faf0035c700 1 -- 172.21.15.64:0/517026991 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faef812dbe0 con 0x7faef80b5a50 2022-06-29T15:42:16.634 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.610+0000 7faee77fe700 1 -- 172.21.15.64:0/517026991 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faee80128c0 con 0x7faef80b5a50 2022-06-29T15:42:16.634 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.610+0000 7faf0035c700 1 -- 172.21.15.64:0/517026991 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faef8155e60 con 0x7faef80b5a50 2022-06-29T15:42:16.635 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.610+0000 7faee77fe700 1 -- 172.21.15.64:0/517026991 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7faee8019030 con 0x7faef80b5a50 2022-06-29T15:42:16.635 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.610+0000 7faee77fe700 1 -- 172.21.15.64:0/517026991 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1444+0+0 (secure 0 0 0) 0x7faee801b9d0 con 0x7faef80b5a50 2022-06-29T15:42:16.636 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.614+0000 7faf0035c700 1 -- 172.21.15.64:0/517026991 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faed4003410 con 0x7faef80b5a50 2022-06-29T15:42:16.636 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.614+0000 7faee77fe700 1 -- 172.21.15.64:0/517026991 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7faee8017070 con 0x7faef80b5a50 2022-06-29T15:42:16.687 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.682+0000 7faf0035c700 1 -- 172.21.15.64:0/517026991 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd new", "uuid": "af0ca9b3-c02c-4910-834b-bf0b8dc6713c", "id": 6} v 0) v1 -- 0x7faed4003990 con 0x7faef80b5a50 2022-06-29T15:42:16.693 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.690+0000 7faee77fe700 1 -- 172.21.15.64:0/517026991 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "af0ca9b3-c02c-4910-834b-bf0b8dc6713c", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7faee801bc60 con 0x7faef80b5a50 2022-06-29T15:42:16.694 INFO:teuthology.orchestra.run.smithi064.stdout:6 2022-06-29T15:42:16.695 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.690+0000 7faf0035c700 1 -- 172.21.15.64:0/517026991 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faef80b5a50 msgr2=0x7faef81526e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:16.696 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.690+0000 7faf0035c700 1 --2- 172.21.15.64:0/517026991 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faef80b5a50 0x7faef81526e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7faef81306b0 tx=0x7faee8007c50).stop 2022-06-29T15:42:16.696 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.690+0000 7faf0035c700 1 -- 172.21.15.64:0/517026991 shutdown_connections 2022-06-29T15:42:16.696 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.690+0000 7faf0035c700 1 --2- 172.21.15.64:0/517026991 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faef8146e90 0x7faef812ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:16.696 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.690+0000 7faf0035c700 1 --2- 172.21.15.64:0/517026991 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faef80b5a50 0x7faef81526e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:16.696 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.690+0000 7faf0035c700 1 --2- 172.21.15.64:0/517026991 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7faef80b1e90 0x7faef8142e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:16.697 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.690+0000 7faf0035c700 1 -- 172.21.15.64:0/517026991 >> 172.21.15.64:0/517026991 conn(0x7faef80b98f0 msgr2=0x7faef80b2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:16.697 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.690+0000 7faf0035c700 1 -- 172.21.15.64:0/517026991 shutdown_connections 2022-06-29T15:42:16.697 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.690+0000 7faf0035c700 1 -- 172.21.15.64:0/517026991 wait complete. 2022-06-29T15:42:16.707 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --cluster ceph osd new f19c3ca6-df88-4bde-a150-b51a53f092e1 7 2022-06-29T15:42:16.804 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.798+0000 7f3a7e35c700 1 Processor -- start 2022-06-29T15:42:16.804 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.802+0000 7f3a7e35c700 1 -- start start 2022-06-29T15:42:16.807 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.802+0000 7f3a7e35c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3a780afd40 0x7f3a780b0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:16.808 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.802+0000 7f3a7e35c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3a78061f30 0x7f3a780ae2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:16.808 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.802+0000 7f3a7e35c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3a7813fcc0 0x7f3a7813d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:16.808 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.802+0000 7f3a7e35c700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a78060d80 con 0x7f3a780afd40 2022-06-29T15:42:16.808 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.802+0000 7f3a7e35c700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a780ad910 con 0x7f3a7813fcc0 2022-06-29T15:42:16.809 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.802+0000 7f3a7e35c700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a780adbb0 con 0x7f3a78061f30 2022-06-29T15:42:16.809 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.802+0000 7f3a7db5b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3a7813fcc0 0x7f3a7813d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:16.809 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.802+0000 7f3a7cb59700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3a78061f30 0x7f3a780ae2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:16.810 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.802+0000 7f3a7db5b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3a7813fcc0 0x7f3a7813d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42214/0 (socket says 172.21.15.64:42214) 2022-06-29T15:42:16.810 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.802+0000 7f3a7db5b700 1 -- 172.21.15.64:0/2039175385 learned_addr learned my addr 172.21.15.64:0/2039175385 (peer_addr_for_me v2:172.21.15.64:0/0) 2022-06-29T15:42:16.810 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.802+0000 7f3a7d35a700 1 --2- 172.21.15.64:0/2039175385 >> v2:172.21.15.47:3300/0 conn(0x7f3a780afd40 0x7f3a780b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:16.810 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.802+0000 7f3a7cb59700 1 -- 172.21.15.64:0/2039175385 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3a78061f30 msgr2=0x7f3a780ae2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2022-06-29T15:42:16.811 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.802+0000 7f3a7cb59700 1 -- 172.21.15.64:0/2039175385 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3a78061f30 msgr2=0x7f3a780ae2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-06-29T15:42:16.811 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.802+0000 7f3a7cb59700 1 --2- 172.21.15.64:0/2039175385 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3a78061f30 0x7f3a780ae2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-29T15:42:16.811 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.802+0000 7f3a7cb59700 1 --2- 172.21.15.64:0/2039175385 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3a78061f30 0x7f3a780ae2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2022-06-29T15:42:16.811 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.802+0000 7f3a7db5b700 1 -- 172.21.15.64:0/2039175385 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3a78061f30 msgr2=0x7f3a780ae2f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-06-29T15:42:16.812 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.802+0000 7f3a7db5b700 1 --2- 172.21.15.64:0/2039175385 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3a78061f30 0x7f3a780ae2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:16.812 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.802+0000 7f3a7db5b700 1 -- 172.21.15.64:0/2039175385 >> v2:172.21.15.47:3300/0 conn(0x7f3a780afd40 msgr2=0x7f3a780b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:16.812 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.802+0000 7f3a7db5b700 1 --2- 172.21.15.64:0/2039175385 >> v2:172.21.15.47:3300/0 conn(0x7f3a780afd40 0x7f3a780b0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:16.812 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.802+0000 7f3a7db5b700 1 -- 172.21.15.64:0/2039175385 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a780ade50 con 0x7f3a7813fcc0 2022-06-29T15:42:16.813 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.802+0000 7f3a7d35a700 1 --2- 172.21.15.64:0/2039175385 >> v2:172.21.15.47:3300/0 conn(0x7f3a780afd40 0x7f3a780b0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:42:16.813 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.802+0000 7f3a7db5b700 1 --2- 172.21.15.64:0/2039175385 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a7813fcc0 0x7f3a7813d270 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f3a700032f0 tx=0x7f3a70009290).ready entity=mon.1 client_cookie=c5a6d74202645a4f server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:16.813 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.802+0000 7f3a67fff700 1 -- 172.21.15.64:0/2039175385 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a7000b870 con 0x7f3a7813fcc0 2022-06-29T15:42:16.813 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.802+0000 7f3a67fff700 1 -- 172.21.15.64:0/2039175385 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a7000b9d0 con 0x7f3a7813fcc0 2022-06-29T15:42:16.814 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.802+0000 7f3a67fff700 1 -- 172.21.15.64:0/2039175385 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a7000bb40 con 0x7f3a7813fcc0 2022-06-29T15:42:16.814 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.802+0000 7f3a7e35c700 1 -- 172.21.15.64:0/2039175385 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a7813fcc0 msgr2=0x7f3a7813d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:16.814 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.802+0000 7f3a7e35c700 1 --2- 172.21.15.64:0/2039175385 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a7813fcc0 0x7f3a7813d270 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f3a700032f0 tx=0x7f3a70009290).stop 2022-06-29T15:42:16.814 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.802+0000 7f3a7e35c700 1 -- 172.21.15.64:0/2039175385 shutdown_connections 2022-06-29T15:42:16.815 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.802+0000 7f3a7e35c700 1 --2- 172.21.15.64:0/2039175385 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a7813fcc0 0x7f3a7813d270 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:16.815 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.802+0000 7f3a7e35c700 1 --2- 172.21.15.64:0/2039175385 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3a78061f30 0x7f3a780ae2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:16.815 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.802+0000 7f3a7e35c700 1 --2- 172.21.15.64:0/2039175385 >> v2:172.21.15.47:3300/0 conn(0x7f3a780afd40 0x7f3a780b0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:16.815 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.802+0000 7f3a7e35c700 1 -- 172.21.15.64:0/2039175385 >> 172.21.15.64:0/2039175385 conn(0x7f3a780b6430 msgr2=0x7f3a780b6810 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:16.816 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.802+0000 7f3a7e35c700 1 -- 172.21.15.64:0/2039175385 shutdown_connections 2022-06-29T15:42:16.816 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.802+0000 7f3a7e35c700 1 -- 172.21.15.64:0/2039175385 wait complete. 2022-06-29T15:42:16.816 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.806+0000 7f3a7e35c700 1 Processor -- start 2022-06-29T15:42:16.816 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.806+0000 7f3a7e35c700 1 -- start start 2022-06-29T15:42:16.819 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.806+0000 7f3a7e35c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3a78061f30 0x7f3a78133410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:16.819 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.806+0000 7f3a7e35c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a780afd40 0x7f3a780ab2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:16.820 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.806+0000 7f3a7e35c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a7813fcc0 0x7f3a7812e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:16.820 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.806+0000 7f3a7e35c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a780b5c40 con 0x7f3a7813fcc0 2022-06-29T15:42:16.820 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.806+0000 7f3a7e35c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a780ac010 con 0x7f3a780afd40 2022-06-29T15:42:16.820 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.806+0000 7f3a7e35c700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a780aa240 con 0x7f3a78061f30 2022-06-29T15:42:16.821 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.806+0000 7f3a7d35a700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3a78061f30 0x7f3a78133410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:16.821 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.806+0000 7f3a7cb59700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a780afd40 0x7f3a780ab2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:16.821 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.806+0000 7f3a7d35a700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3a78061f30 0x7f3a78133410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.64:51768/0 (socket says 172.21.15.64:51768) 2022-06-29T15:42:16.821 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.806+0000 7f3a7cb59700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a780afd40 0x7f3a780ab2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42220/0 (socket says 172.21.15.64:42220) 2022-06-29T15:42:16.822 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.806+0000 7f3a7d35a700 1 -- 172.21.15.64:0/2161668321 learned_addr learned my addr 172.21.15.64:0/2161668321 (peer_addr_for_me v2:172.21.15.64:0/0) 2022-06-29T15:42:16.822 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.806+0000 7f3a7db5b700 1 --2- 172.21.15.64:0/2161668321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a7813fcc0 0x7f3a7812e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:16.822 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.806+0000 7f3a7d35a700 1 -- 172.21.15.64:0/2161668321 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3a78061f30 msgr2=0x7f3a78133410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2022-06-29T15:42:16.822 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.806+0000 7f3a7d35a700 1 -- 172.21.15.64:0/2161668321 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3a78061f30 msgr2=0x7f3a78133410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2022-06-29T15:42:16.826 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.806+0000 7f3a7d35a700 1 --2- 172.21.15.64:0/2161668321 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3a78061f30 0x7f3a78133410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-29T15:42:16.826 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.806+0000 7f3a7d35a700 1 --2- 172.21.15.64:0/2161668321 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3a78061f30 0x7f3a78133410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2022-06-29T15:42:16.826 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.806+0000 7f3a7cb59700 1 -- 172.21.15.64:0/2161668321 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3a78061f30 msgr2=0x7f3a78133410 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2022-06-29T15:42:16.827 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.806+0000 7f3a7cb59700 1 --2- 172.21.15.64:0/2161668321 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3a78061f30 0x7f3a78133410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:16.827 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.806+0000 7f3a7cb59700 1 -- 172.21.15.64:0/2161668321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a7813fcc0 msgr2=0x7f3a7812e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:16.828 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.806+0000 7f3a7cb59700 1 --2- 172.21.15.64:0/2161668321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a7813fcc0 0x7f3a7812e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:16.828 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.806+0000 7f3a7cb59700 1 -- 172.21.15.64:0/2161668321 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a70009530 con 0x7f3a780afd40 2022-06-29T15:42:16.828 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.806+0000 7f3a7cb59700 1 --2- 172.21.15.64:0/2161668321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a780afd40 0x7f3a780ab2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f3a68000e90 tx=0x7f3a68008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:16.829 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.806+0000 7f3a667fc700 1 -- 172.21.15.64:0/2161668321 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a68016070 con 0x7f3a780afd40 2022-06-29T15:42:16.829 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.806+0000 7f3a667fc700 1 -- 172.21.15.64:0/2161668321 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a68011780 con 0x7f3a780afd40 2022-06-29T15:42:16.829 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.806+0000 7f3a7e35c700 1 -- 172.21.15.64:0/2161668321 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a780b5e70 con 0x7f3a780afd40 2022-06-29T15:42:16.829 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.806+0000 7f3a667fc700 1 -- 172.21.15.64:0/2161668321 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a6800dea0 con 0x7f3a780afd40 2022-06-29T15:42:16.830 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.806+0000 7f3a7e35c700 1 -- 172.21.15.64:0/2161668321 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a780abd70 con 0x7f3a780afd40 2022-06-29T15:42:16.830 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.806+0000 7f3a667fc700 1 -- 172.21.15.64:0/2161668321 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f3a68011cb0 con 0x7f3a780afd40 2022-06-29T15:42:16.830 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.806+0000 7f3a667fc700 1 -- 172.21.15.64:0/2161668321 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1559+0+0 (secure 0 0 0) 0x7f3a68005d80 con 0x7f3a780afd40 2022-06-29T15:42:16.830 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.806+0000 7f3a7e35c700 1 -- 172.21.15.64:0/2161668321 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a7806b7a0 con 0x7f3a780afd40 2022-06-29T15:42:16.831 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.810+0000 7f3a667fc700 1 -- 172.21.15.64:0/2161668321 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f3a68006430 con 0x7f3a780afd40 2022-06-29T15:42:16.882 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.878+0000 7f3a7e35c700 1 -- 172.21.15.64:0/2161668321 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f19c3ca6-df88-4bde-a150-b51a53f092e1", "id": 7} v 0) v1 -- 0x7f3a780573d0 con 0x7f3a780afd40 2022-06-29T15:42:16.890 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.886+0000 7f3a667fc700 1 -- 172.21.15.64:0/2161668321 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f19c3ca6-df88-4bde-a150-b51a53f092e1", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3a680290b0 con 0x7f3a780afd40 2022-06-29T15:42:16.890 INFO:teuthology.orchestra.run.smithi064.stdout:7 2022-06-29T15:42:16.892 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.886+0000 7f3a7e35c700 1 -- 172.21.15.64:0/2161668321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a780afd40 msgr2=0x7f3a780ab2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:16.893 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.886+0000 7f3a7e35c700 1 --2- 172.21.15.64:0/2161668321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a780afd40 0x7f3a780ab2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f3a68000e90 tx=0x7f3a68008040).stop 2022-06-29T15:42:16.893 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.886+0000 7f3a7e35c700 1 -- 172.21.15.64:0/2161668321 shutdown_connections 2022-06-29T15:42:16.893 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.886+0000 7f3a7e35c700 1 --2- 172.21.15.64:0/2161668321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a7813fcc0 0x7f3a7812e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:16.894 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.886+0000 7f3a7e35c700 1 --2- 172.21.15.64:0/2161668321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a780afd40 0x7f3a780ab2c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:16.894 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.886+0000 7f3a7e35c700 1 --2- 172.21.15.64:0/2161668321 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3a78061f30 0x7f3a78133410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:16.894 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.886+0000 7f3a7e35c700 1 -- 172.21.15.64:0/2161668321 >> 172.21.15.64:0/2161668321 conn(0x7f3a780b6430 msgr2=0x7f3a781400d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:16.894 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.886+0000 7f3a7e35c700 1 -- 172.21.15.64:0/2161668321 shutdown_connections 2022-06-29T15:42:16.895 INFO:teuthology.orchestra.run.smithi064.stderr:2022-06-29T15:42:16.886+0000 7f3a7e35c700 1 -- 172.21.15.64:0/2161668321 wait complete. 2022-06-29T15:42:16.902 INFO:tasks.ceph.osd.0:Restarting daemon 2022-06-29T15:42:16.902 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2022-06-29T15:42:16.905 INFO:tasks.ceph.osd.0:Started 2022-06-29T15:42:16.905 INFO:tasks.ceph.osd.1:Restarting daemon 2022-06-29T15:42:16.906 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2022-06-29T15:42:16.908 INFO:tasks.ceph.osd.1:Started 2022-06-29T15:42:16.909 INFO:tasks.ceph.osd.2:Restarting daemon 2022-06-29T15:42:16.909 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2022-06-29T15:42:16.912 INFO:tasks.ceph.osd.2:Started 2022-06-29T15:42:16.913 INFO:tasks.ceph.osd.3:Restarting daemon 2022-06-29T15:42:16.913 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2022-06-29T15:42:16.915 INFO:tasks.ceph.osd.3:Started 2022-06-29T15:42:16.916 INFO:tasks.ceph.osd.4:Restarting daemon 2022-06-29T15:42:16.916 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2022-06-29T15:42:16.918 INFO:tasks.ceph.osd.4:Started 2022-06-29T15:42:16.919 INFO:tasks.ceph.osd.5:Restarting daemon 2022-06-29T15:42:16.919 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2022-06-29T15:42:16.921 INFO:tasks.ceph.osd.5:Started 2022-06-29T15:42:16.922 INFO:tasks.ceph.osd.6:Restarting daemon 2022-06-29T15:42:16.922 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2022-06-29T15:42:16.924 INFO:tasks.ceph.osd.6:Started 2022-06-29T15:42:16.925 INFO:tasks.ceph.osd.7:Restarting daemon 2022-06-29T15:42:16.925 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2022-06-29T15:42:16.928 INFO:tasks.ceph.osd.7:Started 2022-06-29T15:42:16.928 INFO:tasks.ceph:Waiting for OSDs to come up 2022-06-29T15:42:17.263 INFO:tasks.ceph.osd.3.smithi047.stderr:2022-06-29T15:42:17.260+0000 7fe0108b1d80 -1 Falling back to public interface 2022-06-29T15:42:17.283 INFO:tasks.ceph.osd.2.smithi047.stderr:2022-06-29T15:42:17.280+0000 7f7c938efd80 -1 Falling back to public interface 2022-06-29T15:42:17.290 INFO:tasks.ceph.osd.5.smithi064.stderr:2022-06-29T15:42:17.286+0000 7fb8b61a1d80 -1 Falling back to public interface 2022-06-29T15:42:17.293 INFO:tasks.ceph.osd.7.smithi064.stderr:2022-06-29T15:42:17.290+0000 7feebf533d80 -1 Falling back to public interface 2022-06-29T15:42:17.313 INFO:tasks.ceph.osd.4.smithi064.stderr:2022-06-29T15:42:17.310+0000 7f04c46fed80 -1 Falling back to public interface 2022-06-29T15:42:17.526 INFO:tasks.ceph.osd.6.smithi064.stderr:2022-06-29T15:42:17.522+0000 7f1af3a9dd80 -1 Falling back to public interface 2022-06-29T15:42:17.543 INFO:tasks.ceph.osd.0.smithi047.stderr:2022-06-29T15:42:17.540+0000 7fe25d607d80 -1 Falling back to public interface 2022-06-29T15:42:17.567 INFO:tasks.ceph.osd.1.smithi047.stderr:2022-06-29T15:42:17.564+0000 7f29c1c38d80 -1 Falling back to public interface 2022-06-29T15:42:17.732 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2022-06-29T15:42:17.826 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.820+0000 7f4d3759e700 1 Processor -- start 2022-06-29T15:42:17.827 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.820+0000 7f4d3759e700 1 -- start start 2022-06-29T15:42:17.827 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.820+0000 7f4d3759e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4d38143340 0x7f4d380af070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:17.828 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.820+0000 7f4d3759e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d380604a0 con 0x7f4d38060650 2022-06-29T15:42:17.828 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.820+0000 7f4d3759e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d380ae9a0 con 0x7f4d380a9d90 2022-06-29T15:42:17.829 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.820+0000 7f4d3759e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d3813d990 con 0x7f4d38143340 2022-06-29T15:42:17.829 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.820+0000 7f4d3659c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4d38143340 0x7f4d380af070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:17.829 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.820+0000 7f4d2ffff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4d38060650 0x7f4d380b6390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:58970/0 (socket says 172.21.15.47:58970) 2022-06-29T15:42:17.830 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.820+0000 7f4d2ffff700 1 -- 172.21.15.47:0/93465980 learned_addr learned my addr 172.21.15.47:0/93465980 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:17.830 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.820+0000 7f4d3659c700 1 -- 172.21.15.47:0/93465980 >> v1:172.21.15.64:6789/0 conn(0x7f4d380a9d90 legacy=0x7f4d3813fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:17.830 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.820+0000 7f4d3659c700 1 -- 172.21.15.47:0/93465980 >> v1:172.21.15.47:6789/0 conn(0x7f4d38060650 legacy=0x7f4d380b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:17.830 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.820+0000 7f4d3659c700 1 -- 172.21.15.47:0/93465980 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d380b6aa0 con 0x7f4d38143340 2022-06-29T15:42:17.831 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.820+0000 7f4d35d9b700 1 -- 172.21.15.47:0/93465980 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4027492429 0 0) 0x7f4d380604a0 con 0x7f4d38060650 2022-06-29T15:42:17.831 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.820+0000 7f4d3659c700 1 --2- 172.21.15.47:0/93465980 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d38143340 0x7f4d380af070 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f4d20002f70 tx=0x7f4d20006df0).ready entity=mon.0 client_cookie=156a632fa18251d9 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:17.831 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.820+0000 7f4d35d9b700 1 -- 172.21.15.47:0/93465980 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d2000d610 con 0x7f4d38143340 2022-06-29T15:42:17.831 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.820+0000 7f4d35d9b700 1 -- 172.21.15.47:0/93465980 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d20000e90 con 0x7f4d38143340 2022-06-29T15:42:17.832 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.820+0000 7f4d35d9b700 1 -- 172.21.15.47:0/93465980 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d2000d960 con 0x7f4d38143340 2022-06-29T15:42:17.832 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.820+0000 7f4d3759e700 1 -- 172.21.15.47:0/93465980 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d38143340 msgr2=0x7f4d380af070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:17.832 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.820+0000 7f4d3759e700 1 --2- 172.21.15.47:0/93465980 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d38143340 0x7f4d380af070 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f4d20002f70 tx=0x7f4d20006df0).stop 2022-06-29T15:42:17.832 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.820+0000 7f4d3759e700 1 -- 172.21.15.47:0/93465980 shutdown_connections 2022-06-29T15:42:17.832 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.820+0000 7f4d3759e700 1 --2- 172.21.15.47:0/93465980 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d38143340 0x7f4d380af070 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:17.833 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.820+0000 7f4d3759e700 1 -- 172.21.15.47:0/93465980 >> 172.21.15.47:0/93465980 conn(0x7f4d38063630 msgr2=0x7f4d38063a10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:17.833 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.820+0000 7f4d3759e700 1 -- 172.21.15.47:0/93465980 shutdown_connections 2022-06-29T15:42:17.833 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.820+0000 7f4d3759e700 1 -- 172.21.15.47:0/93465980 wait complete. 2022-06-29T15:42:17.833 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.824+0000 7f4d3759e700 1 Processor -- start 2022-06-29T15:42:17.834 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.824+0000 7f4d3759e700 1 -- start start 2022-06-29T15:42:17.837 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.824+0000 7f4d3759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d38060650 0x7f4d38146830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:17.838 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.824+0000 7f4d3759e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d380a9d90 0x7f4d38155b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:17.838 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.824+0000 7f4d3759e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4d38143340 0x7f4d38156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:17.838 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.824+0000 7f4d3759e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d38067170 con 0x7f4d380a9d90 2022-06-29T15:42:17.838 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.824+0000 7f4d3759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d380b6800 con 0x7f4d38060650 2022-06-29T15:42:17.839 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.824+0000 7f4d3759e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d380642c0 con 0x7f4d38143340 2022-06-29T15:42:17.839 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.824+0000 7f4d2ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d380a9d90 0x7f4d38155b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:17.839 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.824+0000 7f4d2ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d380a9d90 0x7f4d38155b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45188/0 (socket says 172.21.15.47:45188) 2022-06-29T15:42:17.839 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.824+0000 7f4d2ffff700 1 -- 172.21.15.47:0/910219156 learned_addr learned my addr 172.21.15.47:0/910219156 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:17.839 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.824+0000 7f4d36d9d700 1 --2- 172.21.15.47:0/910219156 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4d38143340 0x7f4d38156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:17.840 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.824+0000 7f4d3659c700 1 --2- 172.21.15.47:0/910219156 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d38060650 0x7f4d38146830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:17.840 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.824+0000 7f4d36d9d700 1 -- 172.21.15.47:0/910219156 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4d38143340 msgr2=0x7f4d38156060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2022-06-29T15:42:17.840 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.824+0000 7f4d36d9d700 1 -- 172.21.15.47:0/910219156 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4d38143340 msgr2=0x7f4d38156060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2022-06-29T15:42:17.840 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.824+0000 7f4d36d9d700 1 --2- 172.21.15.47:0/910219156 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4d38143340 0x7f4d38156060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-06-29T15:42:17.841 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.824+0000 7f4d2ffff700 1 -- 172.21.15.47:0/910219156 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4d38143340 msgr2=0x7f4d38156060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:17.841 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.824+0000 7f4d36d9d700 1 --2- 172.21.15.47:0/910219156 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4d38143340 0x7f4d38156060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2022-06-29T15:42:17.841 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.824+0000 7f4d2ffff700 1 --2- 172.21.15.47:0/910219156 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4d38143340 0x7f4d38156060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:17.841 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.824+0000 7f4d2ffff700 1 -- 172.21.15.47:0/910219156 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d38060650 msgr2=0x7f4d38146830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:17.842 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.824+0000 7f4d2ffff700 1 --2- 172.21.15.47:0/910219156 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d38060650 0x7f4d38146830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:17.842 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.824+0000 7f4d2ffff700 1 -- 172.21.15.47:0/910219156 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d2000b180 con 0x7f4d380a9d90 2022-06-29T15:42:17.842 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.824+0000 7f4d2ffff700 1 --2- 172.21.15.47:0/910219156 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d380a9d90 0x7f4d38155b70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f4d28000bc0 tx=0x7f4d28007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:17.842 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.824+0000 7f4d2f7fe700 1 -- 172.21.15.47:0/910219156 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d2800ce20 con 0x7f4d380a9d90 2022-06-29T15:42:17.843 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.824+0000 7f4d2f7fe700 1 -- 172.21.15.47:0/910219156 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d280072e0 con 0x7f4d380a9d90 2022-06-29T15:42:17.843 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.824+0000 7f4d2f7fe700 1 -- 172.21.15.47:0/910219156 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d28005800 con 0x7f4d380a9d90 2022-06-29T15:42:17.843 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.824+0000 7f4d3759e700 1 -- 172.21.15.47:0/910219156 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d38064560 con 0x7f4d380a9d90 2022-06-29T15:42:17.843 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.824+0000 7f4d3759e700 1 -- 172.21.15.47:0/910219156 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d38066890 con 0x7f4d380a9d90 2022-06-29T15:42:17.844 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.828+0000 7f4d2f7fe700 1 -- 172.21.15.47:0/910219156 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 2) v1 ==== 32894+0+0 (secure 0 0 0) 0x7f4d28010070 con 0x7f4d380a9d90 2022-06-29T15:42:17.844 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.828+0000 7f4d2f7fe700 1 -- 172.21.15.47:0/910219156 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1674+0+0 (secure 0 0 0) 0x7f4d28039bf0 con 0x7f4d380a9d90 2022-06-29T15:42:17.844 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.828+0000 7f4d3759e700 1 -- 172.21.15.47:0/910219156 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d380573d0 con 0x7f4d380a9d90 2022-06-29T15:42:17.844 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.832+0000 7f4d2f7fe700 1 -- 172.21.15.47:0/910219156 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f4d28005960 con 0x7f4d380a9d90 2022-06-29T15:42:17.890 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.888+0000 7f4d3759e700 1 -- 172.21.15.47:0/910219156 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4d3805fea0 con 0x7f4d380a9d90 2022-06-29T15:42:17.891 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.888+0000 7f4d2f7fe700 1 -- 172.21.15.47:0/910219156 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+6849 (secure 0 0 0) 0x7f4d28010300 con 0x7f4d380a9d90 2022-06-29T15:42:17.891 INFO:teuthology.misc.health.smithi047.stdout: 2022-06-29T15:42:17.891 INFO:teuthology.misc.health.smithi047.stdout:{"epoch":10,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:42:16.883261+0000","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:42:17.893 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.888+0000 7f4d3759e700 1 -- 172.21.15.47:0/910219156 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d380a9d90 msgr2=0x7f4d38155b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:17.893 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.888+0000 7f4d3759e700 1 --2- 172.21.15.47:0/910219156 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d380a9d90 0x7f4d38155b70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f4d28000bc0 tx=0x7f4d28007a40).stop 2022-06-29T15:42:17.893 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.888+0000 7f4d3759e700 1 -- 172.21.15.47:0/910219156 shutdown_connections 2022-06-29T15:42:17.893 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.888+0000 7f4d3759e700 1 --2- 172.21.15.47:0/910219156 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4d38143340 0x7f4d38156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:17.894 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.888+0000 7f4d3759e700 1 --2- 172.21.15.47:0/910219156 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d380a9d90 0x7f4d38155b70 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:17.894 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.888+0000 7f4d3759e700 1 --2- 172.21.15.47:0/910219156 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d38060650 0x7f4d38146830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:17.894 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.888+0000 7f4d3759e700 1 -- 172.21.15.47:0/910219156 >> 172.21.15.47:0/910219156 conn(0x7f4d38063630 msgr2=0x7f4d380b00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:17.895 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.888+0000 7f4d3759e700 1 -- 172.21.15.47:0/910219156 shutdown_connections 2022-06-29T15:42:17.895 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:17.888+0000 7f4d3759e700 1 -- 172.21.15.47:0/910219156 wait complete. 2022-06-29T15:42:17.908 DEBUG:teuthology.misc:0 of 8 OSDs are up 2022-06-29T15:42:18.447 INFO:tasks.ceph.osd.3.smithi047.stderr:2022-06-29T15:42:18.444+0000 7fe0108b1d80 -1 osd.3 0 log_to_monitors {default=true} 2022-06-29T15:42:18.467 INFO:tasks.ceph.osd.7.smithi064.stderr:2022-06-29T15:42:18.462+0000 7feebf533d80 -1 osd.7 0 log_to_monitors {default=true} 2022-06-29T15:42:18.470 INFO:tasks.ceph.osd.5.smithi064.stderr:2022-06-29T15:42:18.466+0000 7fb8b61a1d80 -1 osd.5 0 log_to_monitors {default=true} 2022-06-29T15:42:18.476 INFO:tasks.ceph.osd.2.smithi047.stderr:2022-06-29T15:42:18.472+0000 7f7c938efd80 -1 osd.2 0 log_to_monitors {default=true} 2022-06-29T15:42:18.488 INFO:tasks.ceph.osd.4.smithi064.stderr:2022-06-29T15:42:18.486+0000 7f04c46fed80 -1 osd.4 0 log_to_monitors {default=true} 2022-06-29T15:42:18.700 INFO:tasks.ceph.osd.6.smithi064.stderr:2022-06-29T15:42:18.698+0000 7f1af3a9dd80 -1 osd.6 0 log_to_monitors {default=true} 2022-06-29T15:42:18.746 INFO:tasks.ceph.osd.1.smithi047.stderr:2022-06-29T15:42:18.744+0000 7f29c1c38d80 -1 osd.1 0 log_to_monitors {default=true} 2022-06-29T15:42:18.762 INFO:tasks.ceph.osd.0.smithi047.stderr:2022-06-29T15:42:18.760+0000 7fe25d607d80 -1 osd.0 0 log_to_monitors {default=true} 2022-06-29T15:42:19.945 INFO:tasks.ceph.osd.5.smithi064.stderr:2022-06-29T15:42:19.942+0000 7fb8a8247700 -1 osd.5 0 waiting for initial osdmap 2022-06-29T15:42:19.945 INFO:tasks.ceph.osd.6.smithi064.stderr:2022-06-29T15:42:19.942+0000 7f1ae5b43700 -1 osd.6 0 waiting for initial osdmap 2022-06-29T15:42:19.946 INFO:tasks.ceph.osd.4.smithi064.stderr:2022-06-29T15:42:19.942+0000 7f04b67a4700 -1 osd.4 0 waiting for initial osdmap 2022-06-29T15:42:19.947 INFO:tasks.ceph.osd.3.smithi047.stderr:2022-06-29T15:42:19.944+0000 7fe002957700 -1 osd.3 0 waiting for initial osdmap 2022-06-29T15:42:19.947 INFO:tasks.ceph.osd.2.smithi047.stderr:2022-06-29T15:42:19.944+0000 7f7c85995700 -1 osd.2 0 waiting for initial osdmap 2022-06-29T15:42:19.948 INFO:tasks.ceph.osd.7.smithi064.stderr:2022-06-29T15:42:19.942+0000 7feeb15d9700 -1 osd.7 0 waiting for initial osdmap 2022-06-29T15:42:19.949 INFO:tasks.ceph.osd.0.smithi047.stderr:2022-06-29T15:42:19.944+0000 7fe24f6ad700 -1 osd.0 0 waiting for initial osdmap 2022-06-29T15:42:19.949 INFO:tasks.ceph.osd.1.smithi047.stderr:2022-06-29T15:42:19.948+0000 7f29b3cde700 -1 osd.1 0 waiting for initial osdmap 2022-06-29T15:42:19.956 INFO:tasks.ceph.osd.0.smithi047.stderr:2022-06-29T15:42:19.952+0000 7fe256b2b700 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-06-29T15:42:19.956 INFO:tasks.ceph.osd.2.smithi047.stderr:2022-06-29T15:42:19.952+0000 7f7c8ce13700 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-06-29T15:42:19.957 INFO:tasks.ceph.osd.1.smithi047.stderr:2022-06-29T15:42:19.952+0000 7f29bb15c700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-06-29T15:42:19.957 INFO:tasks.ceph.osd.3.smithi047.stderr:2022-06-29T15:42:19.952+0000 7fe009dd5700 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-06-29T15:42:19.958 INFO:tasks.ceph.osd.6.smithi064.stderr:2022-06-29T15:42:19.954+0000 7f1aecfc1700 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-06-29T15:42:19.961 INFO:tasks.ceph.osd.4.smithi064.stderr:2022-06-29T15:42:19.958+0000 7f04bdc22700 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-06-29T15:42:19.961 INFO:tasks.ceph.osd.7.smithi064.stderr:2022-06-29T15:42:19.958+0000 7feeb8a57700 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-06-29T15:42:19.962 INFO:tasks.ceph.osd.5.smithi064.stderr:2022-06-29T15:42:19.958+0000 7fb8af6c5700 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-06-29T15:42:24.715 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2022-06-29T15:42:24.819 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.816+0000 7f511e6b7700 1 Processor -- start 2022-06-29T15:42:24.819 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.816+0000 7f511e6b7700 1 -- start start 2022-06-29T15:42:24.821 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.816+0000 7f511e6b7700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f51180af270 0x7f511813fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:24.821 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.816+0000 7f511e6b7700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f51180b0f00 0x7f511813c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:24.822 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.816+0000 7f511e6b7700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51180ad6e0 con 0x7f51180b0a80 2022-06-29T15:42:24.822 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.820+0000 7f511e6b7700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51180adc20 con 0x7f51180b0f00 2022-06-29T15:42:24.822 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.820+0000 7f511e6b7700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51180b15a0 con 0x7f51180af270 2022-06-29T15:42:24.822 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.820+0000 7f511deb6700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f51180b0f00 0x7f511813c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:24.823 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.820+0000 7f511ceb4700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f51180b0a80 0x7f51180af650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59216/0 (socket says 172.21.15.47:59216) 2022-06-29T15:42:24.823 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.820+0000 7f511ceb4700 1 -- 172.21.15.47:0/146239967 learned_addr learned my addr 172.21.15.47:0/146239967 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:24.824 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.820+0000 7f511d6b5700 1 --2- 172.21.15.47:0/146239967 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f51180af270 0x7f511813fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:24.826 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.820+0000 7f5107fff700 1 -- 172.21.15.47:0/146239967 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3851258814 0 0) 0x7f51180ad6e0 con 0x7f51180b0a80 2022-06-29T15:42:24.826 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.820+0000 7f511deb6700 1 -- 172.21.15.47:0/146239967 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f51180af270 msgr2=0x7f511813fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:24.826 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.820+0000 7f511deb6700 1 --2- 172.21.15.47:0/146239967 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f51180af270 0x7f511813fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:24.826 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.820+0000 7f511deb6700 1 -- 172.21.15.47:0/146239967 >> v1:172.21.15.47:6789/0 conn(0x7f51180b0a80 legacy=0x7f51180af650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:24.827 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.820+0000 7f511deb6700 1 -- 172.21.15.47:0/146239967 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51180ad980 con 0x7f51180b0f00 2022-06-29T15:42:24.827 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.820+0000 7f511deb6700 1 --2- 172.21.15.47:0/146239967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51180b0f00 0x7f511813c9d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f51100032f0 tx=0x7f51100091d0).ready entity=mon.0 client_cookie=63e8a63a6e51697d server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:24.827 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.820+0000 7f5107fff700 1 -- 172.21.15.47:0/146239967 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f511000b6c0 con 0x7f51180b0f00 2022-06-29T15:42:24.828 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.820+0000 7f5107fff700 1 -- 172.21.15.47:0/146239967 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f511000b820 con 0x7f51180b0f00 2022-06-29T15:42:24.828 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.820+0000 7f5107fff700 1 -- 172.21.15.47:0/146239967 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f511000b990 con 0x7f51180b0f00 2022-06-29T15:42:24.828 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.820+0000 7f511e6b7700 1 -- 172.21.15.47:0/146239967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51180b0f00 msgr2=0x7f511813c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:24.829 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.820+0000 7f511e6b7700 1 --2- 172.21.15.47:0/146239967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51180b0f00 0x7f511813c9d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f51100032f0 tx=0x7f51100091d0).stop 2022-06-29T15:42:24.829 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.820+0000 7f511e6b7700 1 -- 172.21.15.47:0/146239967 shutdown_connections 2022-06-29T15:42:24.829 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.820+0000 7f511e6b7700 1 --2- 172.21.15.47:0/146239967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51180b0f00 0x7f511813c9d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:24.830 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.820+0000 7f511e6b7700 1 --2- 172.21.15.47:0/146239967 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f51180af270 0x7f511813fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:24.830 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.820+0000 7f511e6b7700 1 -- 172.21.15.47:0/146239967 >> 172.21.15.47:0/146239967 conn(0x7f5118062de0 msgr2=0x7f51180631f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:24.830 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.820+0000 7f511e6b7700 1 -- 172.21.15.47:0/146239967 shutdown_connections 2022-06-29T15:42:24.831 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.820+0000 7f511e6b7700 1 -- 172.21.15.47:0/146239967 wait complete. 2022-06-29T15:42:24.831 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.824+0000 7f511e6b7700 1 Processor -- start 2022-06-29T15:42:24.831 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.824+0000 7f511e6b7700 1 -- start start 2022-06-29T15:42:24.836 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.824+0000 7f511e6b7700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f51180af270 0x7f51180b27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:24.836 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.824+0000 7f511e6b7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51180b0a80 0x7f5118067180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:24.836 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.824+0000 7f511e6b7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51180b0f00 0x7f51180ac060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:24.837 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.824+0000 7f511e6b7700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51180b3430 con 0x7f51180b0a80 2022-06-29T15:42:24.837 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.824+0000 7f511e6b7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5118064130 con 0x7f51180b0f00 2022-06-29T15:42:24.837 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.824+0000 7f511e6b7700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51180b54a0 con 0x7f51180af270 2022-06-29T15:42:24.837 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.824+0000 7f511ceb4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51180b0a80 0x7f5118067180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:24.838 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.824+0000 7f511ceb4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51180b0a80 0x7f5118067180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45436/0 (socket says 172.21.15.47:45436) 2022-06-29T15:42:24.838 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.824+0000 7f511d6b5700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f51180af270 0x7f51180b27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:24.838 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.824+0000 7f511ceb4700 1 -- 172.21.15.47:0/3777628764 learned_addr learned my addr 172.21.15.47:0/3777628764 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:24.838 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.824+0000 7f511deb6700 1 --2- 172.21.15.47:0/3777628764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51180b0f00 0x7f51180ac060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:24.839 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.824+0000 7f511ceb4700 1 -- 172.21.15.47:0/3777628764 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f51180af270 msgr2=0x7f51180b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:24.839 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.824+0000 7f511ceb4700 1 --2- 172.21.15.47:0/3777628764 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f51180af270 0x7f51180b27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:24.839 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.824+0000 7f511ceb4700 1 -- 172.21.15.47:0/3777628764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51180b0f00 msgr2=0x7f51180ac060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:24.839 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.824+0000 7f511ceb4700 1 --2- 172.21.15.47:0/3777628764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51180b0f00 0x7f51180ac060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:24.839 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.824+0000 7f511ceb4700 1 -- 172.21.15.47:0/3777628764 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5110009470 con 0x7f51180b0a80 2022-06-29T15:42:24.840 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.824+0000 7f511ceb4700 1 --2- 172.21.15.47:0/3777628764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51180b0a80 0x7f5118067180 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f5108000ed0 tx=0x7f5108007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:24.840 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.824+0000 7f51067fc700 1 -- 172.21.15.47:0/3777628764 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f51080070d0 con 0x7f51180b0a80 2022-06-29T15:42:24.841 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.824+0000 7f51067fc700 1 -- 172.21.15.47:0/3777628764 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5108007230 con 0x7f51180b0a80 2022-06-29T15:42:24.841 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.824+0000 7f511e6b7700 1 -- 172.21.15.47:0/3777628764 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51180b2cc0 con 0x7f51180b0a80 2022-06-29T15:42:24.841 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.824+0000 7f51067fc700 1 -- 172.21.15.47:0/3777628764 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5108005610 con 0x7f51180b0a80 2022-06-29T15:42:24.842 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.824+0000 7f511e6b7700 1 -- 172.21.15.47:0/3777628764 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5118065a50 con 0x7f51180b0a80 2022-06-29T15:42:24.842 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.828+0000 7f511e6b7700 1 -- 172.21.15.47:0/3777628764 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51180b52e0 con 0x7f51180b0a80 2022-06-29T15:42:24.843 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.828+0000 7f51067fc700 1 -- 172.21.15.47:0/3777628764 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f510801a060 con 0x7f51180b0a80 2022-06-29T15:42:24.843 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.828+0000 7f51067fc700 1 --2- 172.21.15.47:0/3777628764 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f50f8053950 0x7f50f8055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:24.844 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.828+0000 7f51067fc700 1 -- 172.21.15.47:0/3777628764 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4900+0+0 (secure 0 0 0) 0x7f510806ae30 con 0x7f51180b0a80 2022-06-29T15:42:24.844 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.828+0000 7f511d6b5700 1 --2- 172.21.15.47:0/3777628764 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f50f8053950 0x7f50f8055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:24.844 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.828+0000 7f511d6b5700 1 --2- 172.21.15.47:0/3777628764 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f50f8053950 0x7f50f8055e10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f511400b000 tx=0x7f5114006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:24.846 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.832+0000 7f51067fc700 1 -- 172.21.15.47:0/3777628764 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f51080421e0 con 0x7f51180b0a80 2022-06-29T15:42:24.969 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.968+0000 7f511e6b7700 1 -- 172.21.15.47:0/3777628764 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f51180b1790 con 0x7f51180b0a80 2022-06-29T15:42:24.974 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.972+0000 7f51067fc700 1 -- 172.21.15.47:0/3777628764 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+11805 (secure 0 0 0) 0x7f5108018070 con 0x7f51180b0a80 2022-06-29T15:42:24.974 INFO:teuthology.misc.health.smithi047.stdout: 2022-06-29T15:42:24.974 INFO:teuthology.misc.health.smithi047.stdout:{"epoch":16,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:42:22.952054+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:42:24.977 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.972+0000 7f511e6b7700 1 -- 172.21.15.47:0/3777628764 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f50f8053950 msgr2=0x7f50f8055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:24.977 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.972+0000 7f511e6b7700 1 --2- 172.21.15.47:0/3777628764 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f50f8053950 0x7f50f8055e10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f511400b000 tx=0x7f5114006df0).stop 2022-06-29T15:42:24.978 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.972+0000 7f511e6b7700 1 -- 172.21.15.47:0/3777628764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51180b0a80 msgr2=0x7f5118067180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:24.978 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.972+0000 7f511e6b7700 1 --2- 172.21.15.47:0/3777628764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51180b0a80 0x7f5118067180 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f5108000ed0 tx=0x7f5108007ac0).stop 2022-06-29T15:42:24.979 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.972+0000 7f511e6b7700 1 -- 172.21.15.47:0/3777628764 shutdown_connections 2022-06-29T15:42:24.979 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.972+0000 7f511e6b7700 1 --2- 172.21.15.47:0/3777628764 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f50f8053950 0x7f50f8055e10 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:24.979 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.972+0000 7f511e6b7700 1 --2- 172.21.15.47:0/3777628764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51180b0f00 0x7f51180ac060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:24.980 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.972+0000 7f511e6b7700 1 --2- 172.21.15.47:0/3777628764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51180b0a80 0x7f5118067180 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:24.980 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.972+0000 7f511e6b7700 1 --2- 172.21.15.47:0/3777628764 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f51180af270 0x7f51180b27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:24.980 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.972+0000 7f511e6b7700 1 -- 172.21.15.47:0/3777628764 >> 172.21.15.47:0/3777628764 conn(0x7f5118062de0 msgr2=0x7f511813ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:24.981 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.972+0000 7f511e6b7700 1 -- 172.21.15.47:0/3777628764 shutdown_connections 2022-06-29T15:42:24.981 INFO:teuthology.misc.health.smithi047.stderr:2022-06-29T15:42:24.972+0000 7f511e6b7700 1 -- 172.21.15.47:0/3777628764 wait complete. 2022-06-29T15:42:24.986 DEBUG:teuthology.misc:8 of 8 OSDs are up 2022-06-29T15:42:24.987 INFO:tasks.ceph:Creating RBD pool 2022-06-29T15:42:24.987 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --cluster ceph osd pool create rbd 8 2022-06-29T15:42:25.093 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.088+0000 7fd88d86b700 1 Processor -- start 2022-06-29T15:42:25.093 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.088+0000 7fd88d86b700 1 -- start start 2022-06-29T15:42:25.099 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.092+0000 7fd88d86b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd888060ee0 0x7fd8880612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:25.100 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.092+0000 7fd88d86b700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd888067610 con 0x7fd8880b4300 2022-06-29T15:42:25.100 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.092+0000 7fd88d86b700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8881467c0 con 0x7fd8881471a0 2022-06-29T15:42:25.100 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.092+0000 7fd88d86b700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd888146a60 con 0x7fd888060ee0 2022-06-29T15:42:25.100 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.092+0000 7fd8867fc700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd8880b4300 0x7fd8881430e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59228/0 (socket says 172.21.15.47:59228) 2022-06-29T15:42:25.101 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.092+0000 7fd8867fc700 1 -- 172.21.15.47:0/612168618 learned_addr learned my addr 172.21.15.47:0/612168618 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:25.101 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.092+0000 7fd886ffd700 1 --2- 172.21.15.47:0/612168618 >> v2:172.21.15.64:3300/0 conn(0x7fd888060ee0 0x7fd8880612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:25.101 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.092+0000 7fd885ffb700 1 -- 172.21.15.47:0/612168618 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 628468279 0 0) 0x7fd888067610 con 0x7fd8880b4300 2022-06-29T15:42:25.102 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.092+0000 7fd885ffb700 1 -- 172.21.15.47:0/612168618 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd868003260 con 0x7fd8880b4300 2022-06-29T15:42:25.102 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.092+0000 7fd885ffb700 1 -- 172.21.15.47:0/612168618 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2967830255 0 0) 0x7fd868003260 con 0x7fd8880b4300 2022-06-29T15:42:25.102 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.092+0000 7fd885ffb700 1 -- 172.21.15.47:0/612168618 >> v2:172.21.15.64:3300/0 conn(0x7fd888060ee0 msgr2=0x7fd8880612c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:25.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.092+0000 7fd885ffb700 1 --2- 172.21.15.47:0/612168618 >> v2:172.21.15.64:3300/0 conn(0x7fd888060ee0 0x7fd8880612c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:25.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.092+0000 7fd885ffb700 1 -- 172.21.15.47:0/612168618 >> v1:172.21.15.64:6789/0 conn(0x7fd8881471a0 legacy=0x7fd88813cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:25.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.092+0000 7fd885ffb700 1 -- 172.21.15.47:0/612168618 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd888146d00 con 0x7fd8880b4300 2022-06-29T15:42:25.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.092+0000 7fd885ffb700 1 -- 172.21.15.47:0/612168618 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7fd878001660 con 0x7fd8880b4300 2022-06-29T15:42:25.104 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.092+0000 7fd885ffb700 1 -- 172.21.15.47:0/612168618 >> v1:172.21.15.47:6789/0 conn(0x7fd8880b4300 legacy=0x7fd8881430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:25.104 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.092+0000 7fd885ffb700 1 --2- 172.21.15.47:0/612168618 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd868003780 0x7fd868003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:25.104 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.092+0000 7fd885ffb700 1 --2- 172.21.15.47:0/612168618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd868004500 0x7fd868004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:25.105 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.092+0000 7fd885ffb700 1 --2- 172.21.15.47:0/612168618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd868004e70 0x7fd86800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:25.105 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.092+0000 7fd885ffb700 1 -- 172.21.15.47:0/612168618 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd888146a60 con 0x7fd868004e70 2022-06-29T15:42:25.106 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.092+0000 7fd885ffb700 1 -- 172.21.15.47:0/612168618 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd86800bd20 con 0x7fd868004500 2022-06-29T15:42:25.106 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.092+0000 7fd885ffb700 1 -- 172.21.15.47:0/612168618 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd86800bee0 con 0x7fd868003780 2022-06-29T15:42:25.106 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.092+0000 7fd8877fe700 1 --2- 172.21.15.47:0/612168618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd868004e70 0x7fd86800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:25.107 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.092+0000 7fd886ffd700 1 --2- 172.21.15.47:0/612168618 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd868003780 0x7fd868003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:25.107 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.092+0000 7fd8867fc700 1 --2- 172.21.15.47:0/612168618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd868004500 0x7fd868004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:25.108 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.092+0000 7fd8877fe700 1 -- 172.21.15.47:0/612168618 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd868003780 msgr2=0x7fd868003ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:25.108 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.092+0000 7fd8877fe700 1 --2- 172.21.15.47:0/612168618 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd868003780 0x7fd868003ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:25.108 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.092+0000 7fd8877fe700 1 -- 172.21.15.47:0/612168618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd868004500 msgr2=0x7fd868004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:25.109 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.092+0000 7fd8877fe700 1 --2- 172.21.15.47:0/612168618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd868004500 0x7fd868004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:25.109 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.092+0000 7fd8877fe700 1 -- 172.21.15.47:0/612168618 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd868003260 con 0x7fd868004e70 2022-06-29T15:42:25.110 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.092+0000 7fd8867fc700 1 --2- 172.21.15.47:0/612168618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd868004500 0x7fd868004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:42:25.110 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.092+0000 7fd8877fe700 1 -- 172.21.15.47:0/612168618 reap_dead start 2022-06-29T15:42:25.110 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.092+0000 7fd8877fe700 1 --2- 172.21.15.47:0/612168618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd868004e70 0x7fd86800b750 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fd8880b9870 tx=0x7fd880007d80).ready entity=mon.0 client_cookie=a91aace1082fd6b0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:25.111 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.092+0000 7fd885ffb700 1 -- 172.21.15.47:0/612168618 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd880001620 con 0x7fd868004e70 2022-06-29T15:42:25.111 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.092+0000 7fd885ffb700 1 -- 172.21.15.47:0/612168618 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd880006890 con 0x7fd868004e70 2022-06-29T15:42:25.112 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.092+0000 7fd885ffb700 1 -- 172.21.15.47:0/612168618 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd880006bf0 con 0x7fd868004e70 2022-06-29T15:42:25.112 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.096+0000 7fd88d86b700 1 -- 172.21.15.47:0/612168618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd868004e70 msgr2=0x7fd86800b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:25.113 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.096+0000 7fd88d86b700 1 --2- 172.21.15.47:0/612168618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd868004e70 0x7fd86800b750 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fd8880b9870 tx=0x7fd880007d80).stop 2022-06-29T15:42:25.113 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.096+0000 7fd88d86b700 1 -- 172.21.15.47:0/612168618 shutdown_connections 2022-06-29T15:42:25.113 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.096+0000 7fd88d86b700 1 --2- 172.21.15.47:0/612168618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd868004e70 0x7fd86800b750 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:25.114 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.096+0000 7fd88d86b700 1 -- 172.21.15.47:0/612168618 >> 172.21.15.47:0/612168618 conn(0x7fd88805c510 msgr2=0x7fd88805f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:25.114 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.096+0000 7fd88d86b700 1 -- 172.21.15.47:0/612168618 shutdown_connections 2022-06-29T15:42:25.115 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.096+0000 7fd88d86b700 1 -- 172.21.15.47:0/612168618 wait complete. 2022-06-29T15:42:25.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.096+0000 7fd88d86b700 1 Processor -- start 2022-06-29T15:42:25.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.096+0000 7fd88d86b700 1 -- start start 2022-06-29T15:42:25.121 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.100+0000 7fd88d86b700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd868004e70 0x7fd8880b36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:25.121 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.100+0000 7fd88d86b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8880b30b0 0x7fd888142070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:25.121 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.100+0000 7fd88d86b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd88813e600 0x7fd8881147c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:25.121 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.100+0000 7fd88d86b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8881567b0 con 0x7fd8880b30b0 2022-06-29T15:42:25.121 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.100+0000 7fd88d86b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd888142d40 con 0x7fd88813e600 2022-06-29T15:42:25.122 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.100+0000 7fd88d86b700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8880b1e30 con 0x7fd868004e70 2022-06-29T15:42:25.122 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.100+0000 7fd8867fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8880b30b0 0x7fd888142070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:25.122 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.100+0000 7fd8867fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8880b30b0 0x7fd888142070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45456/0 (socket says 172.21.15.47:45456) 2022-06-29T15:42:25.122 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.100+0000 7fd8867fc700 1 -- 172.21.15.47:0/709458973 learned_addr learned my addr 172.21.15.47:0/709458973 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:25.123 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.100+0000 7fd8877fe700 1 --2- 172.21.15.47:0/709458973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd88813e600 0x7fd8881147c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:25.123 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.100+0000 7fd886ffd700 1 --2- 172.21.15.47:0/709458973 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd868004e70 0x7fd8880b36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:25.123 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.100+0000 7fd8867fc700 1 -- 172.21.15.47:0/709458973 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd868004e70 msgr2=0x7fd8880b36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:25.123 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.100+0000 7fd8867fc700 1 --2- 172.21.15.47:0/709458973 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd868004e70 0x7fd8880b36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:25.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.100+0000 7fd8867fc700 1 -- 172.21.15.47:0/709458973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd88813e600 msgr2=0x7fd8881147c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:25.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.100+0000 7fd8867fc700 1 --2- 172.21.15.47:0/709458973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd88813e600 0x7fd8881147c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:25.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.100+0000 7fd8867fc700 1 -- 172.21.15.47:0/709458973 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8880b20d0 con 0x7fd8880b30b0 2022-06-29T15:42:25.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.100+0000 7fd8867fc700 1 --2- 172.21.15.47:0/709458973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8880b30b0 0x7fd888142070 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fd87800b000 tx=0x7fd888146e90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:25.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.100+0000 7fd87ffff700 1 -- 172.21.15.47:0/709458973 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd878015630 con 0x7fd8880b30b0 2022-06-29T15:42:25.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.100+0000 7fd87ffff700 1 -- 172.21.15.47:0/709458973 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd878010040 con 0x7fd8880b30b0 2022-06-29T15:42:25.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.100+0000 7fd88d86b700 1 -- 172.21.15.47:0/709458973 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8880b2370 con 0x7fd8880b30b0 2022-06-29T15:42:25.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.100+0000 7fd87ffff700 1 -- 172.21.15.47:0/709458973 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd878015980 con 0x7fd8880b30b0 2022-06-29T15:42:25.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.100+0000 7fd88d86b700 1 -- 172.21.15.47:0/709458973 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd888142560 con 0x7fd8880b30b0 2022-06-29T15:42:25.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.104+0000 7fd88d86b700 1 -- 172.21.15.47:0/709458973 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8880573d0 con 0x7fd8880b30b0 2022-06-29T15:42:25.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.104+0000 7fd87ffff700 1 -- 172.21.15.47:0/709458973 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd878015ae0 con 0x7fd8880b30b0 2022-06-29T15:42:25.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.104+0000 7fd87ffff700 1 --2- 172.21.15.47:0/709458973 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd86c053a70 0x7fd86c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:25.127 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.104+0000 7fd87ffff700 1 -- 172.21.15.47:0/709458973 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4900+0+0 (secure 0 0 0) 0x7fd87806b420 con 0x7fd8880b30b0 2022-06-29T15:42:25.127 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.104+0000 7fd886ffd700 1 --2- 172.21.15.47:0/709458973 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd86c053a70 0x7fd86c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:25.127 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.104+0000 7fd886ffd700 1 --2- 172.21.15.47:0/709458973 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd86c053a70 0x7fd86c055f30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fd87000b000 tx=0x7fd870006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:25.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.108+0000 7fd87ffff700 1 -- 172.21.15.47:0/709458973 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd8780426c0 con 0x7fd8880b30b0 2022-06-29T15:42:25.240 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.236+0000 7fd88d86b700 1 -- 172.21.15.47:0/709458973 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7fd8880b5e50 con 0x7fd8880b30b0 2022-06-29T15:42:25.981 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.976+0000 7fd87ffff700 1 -- 172.21.15.47:0/709458973 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v17) v1 ==== 109+0+0 (secure 0 0 0) 0x7fd878045ce0 con 0x7fd8880b30b0 2022-06-29T15:42:25.985 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.984+0000 7fd88d86b700 1 -- 172.21.15.47:0/709458973 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd86c053a70 msgr2=0x7fd86c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:25.985 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.984+0000 7fd88d86b700 1 --2- 172.21.15.47:0/709458973 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd86c053a70 0x7fd86c055f30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fd87000b000 tx=0x7fd870006df0).stop 2022-06-29T15:42:25.986 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.984+0000 7fd88d86b700 1 -- 172.21.15.47:0/709458973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8880b30b0 msgr2=0x7fd888142070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:25.986 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.984+0000 7fd88d86b700 1 --2- 172.21.15.47:0/709458973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8880b30b0 0x7fd888142070 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fd87800b000 tx=0x7fd888146e90).stop 2022-06-29T15:42:25.987 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.984+0000 7fd88d86b700 1 -- 172.21.15.47:0/709458973 shutdown_connections 2022-06-29T15:42:25.987 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.984+0000 7fd88d86b700 1 --2- 172.21.15.47:0/709458973 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd86c053a70 0x7fd86c055f30 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:25.988 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.984+0000 7fd88d86b700 1 --2- 172.21.15.47:0/709458973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd88813e600 0x7fd8881147c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:25.988 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.984+0000 7fd88d86b700 1 --2- 172.21.15.47:0/709458973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8880b30b0 0x7fd888142070 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:25.988 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.984+0000 7fd88d86b700 1 --2- 172.21.15.47:0/709458973 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd868004e70 0x7fd8880b36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:25.988 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.984+0000 7fd88d86b700 1 -- 172.21.15.47:0/709458973 >> 172.21.15.47:0/709458973 conn(0x7fd88805c510 msgr2=0x7fd8880b87c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:25.988 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.984+0000 7fd88d86b700 1 -- 172.21.15.47:0/709458973 shutdown_connections 2022-06-29T15:42:25.989 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:25.984+0000 7fd88d86b700 1 -- 172.21.15.47:0/709458973 wait complete. 2022-06-29T15:42:25.989 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'rbd' created 2022-06-29T15:42:25.999 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2022-06-29T15:42:26.106 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.104+0000 7f1b448e7700 1 Processor -- start 2022-06-29T15:42:26.107 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.104+0000 7f1b448e7700 1 -- start start 2022-06-29T15:42:26.113 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.104+0000 7f1b448e7700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1b400619b0 0x7f1b400b1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:26.113 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.104+0000 7f1b448e7700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1b400b1e20 0x7f1b40146790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:26.114 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.104+0000 7f1b448e7700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b40061d90 con 0x7f1b4005f0b0 2022-06-29T15:42:26.114 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.104+0000 7f1b448e7700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b4005f6d0 con 0x7f1b400619b0 2022-06-29T15:42:26.115 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.104+0000 7f1b448e7700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b400b3bf0 con 0x7f1b400b1e20 2022-06-29T15:42:26.115 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.104+0000 7f1b3ed9d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1b4005f0b0 0x7f1b400b01b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59252/0 (socket says 172.21.15.47:59252) 2022-06-29T15:42:26.116 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.104+0000 7f1b3ed9d700 1 -- 172.21.15.47:0/2145333669 learned_addr learned my addr 172.21.15.47:0/2145333669 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:26.116 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.104+0000 7f1b3e59c700 1 --2- 172.21.15.47:0/2145333669 >> v2:172.21.15.64:3300/0 conn(0x7f1b400619b0 0x7f1b400b1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:26.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.104+0000 7f1b3dd9b700 1 --2- 172.21.15.47:0/2145333669 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1b400b1e20 0x7f1b40146790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:26.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.104+0000 7f1b3d59a700 1 -- 172.21.15.47:0/2145333669 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 942743254 0 0) 0x7f1b40061d90 con 0x7f1b4005f0b0 2022-06-29T15:42:26.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.104+0000 7f1b3d59a700 1 -- 172.21.15.47:0/2145333669 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b20003260 con 0x7f1b4005f0b0 2022-06-29T15:42:26.118 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.104+0000 7f1b3dd9b700 1 -- 172.21.15.47:0/2145333669 >> v2:172.21.15.64:3300/0 conn(0x7f1b400619b0 msgr2=0x7f1b400b1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:26.118 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.104+0000 7f1b3dd9b700 1 --2- 172.21.15.47:0/2145333669 >> v2:172.21.15.64:3300/0 conn(0x7f1b400619b0 0x7f1b400b1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:26.119 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.104+0000 7f1b3dd9b700 1 -- 172.21.15.47:0/2145333669 >> v1:172.21.15.47:6789/0 conn(0x7f1b4005f0b0 legacy=0x7f1b400b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:26.119 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.104+0000 7f1b3dd9b700 1 -- 172.21.15.47:0/2145333669 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b400aea80 con 0x7f1b400b1e20 2022-06-29T15:42:26.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.108+0000 7f1b3dd9b700 1 --2- 172.21.15.47:0/2145333669 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1b400b1e20 0x7f1b40146790 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f1b28002f70 tx=0x7f1b28006df0).ready entity=mon.2 client_cookie=6787c55be73ed309 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:26.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.108+0000 7f1b3d59a700 1 -- 172.21.15.47:0/2145333669 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b2800d4a0 con 0x7f1b400b1e20 2022-06-29T15:42:26.121 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.108+0000 7f1b3d59a700 1 -- 172.21.15.47:0/2145333669 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b28000e90 con 0x7f1b400b1e20 2022-06-29T15:42:26.121 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.108+0000 7f1b3d59a700 1 -- 172.21.15.47:0/2145333669 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b2800d7f0 con 0x7f1b400b1e20 2022-06-29T15:42:26.122 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.108+0000 7f1b448e7700 1 -- 172.21.15.47:0/2145333669 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1b400b1e20 msgr2=0x7f1b40146790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:26.122 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.108+0000 7f1b448e7700 1 --2- 172.21.15.47:0/2145333669 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1b400b1e20 0x7f1b40146790 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f1b28002f70 tx=0x7f1b28006df0).stop 2022-06-29T15:42:26.122 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.108+0000 7f1b448e7700 1 -- 172.21.15.47:0/2145333669 shutdown_connections 2022-06-29T15:42:26.123 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.108+0000 7f1b448e7700 1 --2- 172.21.15.47:0/2145333669 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1b400b1e20 0x7f1b40146790 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:26.123 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.108+0000 7f1b448e7700 1 --2- 172.21.15.47:0/2145333669 >> v2:172.21.15.64:3300/0 conn(0x7f1b400619b0 0x7f1b400b1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:26.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.108+0000 7f1b448e7700 1 -- 172.21.15.47:0/2145333669 >> 172.21.15.47:0/2145333669 conn(0x7f1b40063800 msgr2=0x7f1b40063be0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:26.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.108+0000 7f1b448e7700 1 -- 172.21.15.47:0/2145333669 shutdown_connections 2022-06-29T15:42:26.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.108+0000 7f1b448e7700 1 -- 172.21.15.47:0/2145333669 wait complete. 2022-06-29T15:42:26.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.108+0000 7f1b448e7700 1 Processor -- start 2022-06-29T15:42:26.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.108+0000 7f1b448e7700 1 -- start start 2022-06-29T15:42:26.131 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.112+0000 7f1b448e7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b4005f0b0 0x7f1b40145700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:26.132 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.112+0000 7f1b448e7700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1b400619b0 0x7f1b40152740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:26.132 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.112+0000 7f1b448e7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b400b1e20 0x7f1b40152c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:26.132 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.112+0000 7f1b448e7700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b40066a60 con 0x7f1b4005f0b0 2022-06-29T15:42:26.133 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.112+0000 7f1b448e7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b40066d00 con 0x7f1b400b1e20 2022-06-29T15:42:26.133 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.112+0000 7f1b448e7700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b40066fa0 con 0x7f1b400619b0 2022-06-29T15:42:26.134 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.112+0000 7f1b3e59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b4005f0b0 0x7f1b40145700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:26.134 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.112+0000 7f1b3e59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b4005f0b0 0x7f1b40145700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45468/0 (socket says 172.21.15.47:45468) 2022-06-29T15:42:26.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.112+0000 7f1b3e59c700 1 -- 172.21.15.47:0/4013516429 learned_addr learned my addr 172.21.15.47:0/4013516429 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:26.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.112+0000 7f1b3ed9d700 1 --2- 172.21.15.47:0/4013516429 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b400b1e20 0x7f1b40152c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:26.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.112+0000 7f1b3dd9b700 1 --2- 172.21.15.47:0/4013516429 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1b400619b0 0x7f1b40152740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:26.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.112+0000 7f1b3e59c700 1 -- 172.21.15.47:0/4013516429 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1b400619b0 msgr2=0x7f1b40152740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:26.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.112+0000 7f1b3e59c700 1 --2- 172.21.15.47:0/4013516429 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1b400619b0 0x7f1b40152740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:26.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.112+0000 7f1b3e59c700 1 -- 172.21.15.47:0/4013516429 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b400b1e20 msgr2=0x7f1b40152c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:26.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.112+0000 7f1b3e59c700 1 --2- 172.21.15.47:0/4013516429 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b400b1e20 0x7f1b40152c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:26.138 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.112+0000 7f1b3e59c700 1 -- 172.21.15.47:0/4013516429 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b2800b180 con 0x7f1b4005f0b0 2022-06-29T15:42:26.138 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.112+0000 7f1b3ed9d700 1 --2- 172.21.15.47:0/4013516429 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b400b1e20 0x7f1b40152c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:42:26.139 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.112+0000 7f1b3e59c700 1 --2- 172.21.15.47:0/4013516429 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b4005f0b0 0x7f1b40145700 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f1b3000c000 tx=0x7f1b30006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:26.139 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.112+0000 7f1b377fe700 1 -- 172.21.15.47:0/4013516429 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b30011750 con 0x7f1b4005f0b0 2022-06-29T15:42:26.140 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.112+0000 7f1b377fe700 1 -- 172.21.15.47:0/4013516429 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b300118b0 con 0x7f1b4005f0b0 2022-06-29T15:42:26.140 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.112+0000 7f1b448e7700 1 -- 172.21.15.47:0/4013516429 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b40067240 con 0x7f1b4005f0b0 2022-06-29T15:42:26.140 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.112+0000 7f1b377fe700 1 -- 172.21.15.47:0/4013516429 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b30011c10 con 0x7f1b4005f0b0 2022-06-29T15:42:26.141 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.112+0000 7f1b448e7700 1 -- 172.21.15.47:0/4013516429 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b400ad7c0 con 0x7f1b4005f0b0 2022-06-29T15:42:26.141 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.116+0000 7f1b448e7700 1 -- 172.21.15.47:0/4013516429 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b400573d0 con 0x7f1b4005f0b0 2022-06-29T15:42:26.142 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.116+0000 7f1b377fe700 1 -- 172.21.15.47:0/4013516429 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1b30011d70 con 0x7f1b4005f0b0 2022-06-29T15:42:26.142 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.116+0000 7f1b377fe700 1 --2- 172.21.15.47:0/4013516429 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1b24053b30 0x7f1b24055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:26.143 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.116+0000 7f1b377fe700 1 -- 172.21.15.47:0/4013516429 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5242+0+0 (secure 0 0 0) 0x7f1b3006bf70 con 0x7f1b4005f0b0 2022-06-29T15:42:26.143 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.116+0000 7f1b3dd9b700 1 --2- 172.21.15.47:0/4013516429 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1b24053b30 0x7f1b24055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:26.144 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.116+0000 7f1b3dd9b700 1 --2- 172.21.15.47:0/4013516429 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1b24053b30 0x7f1b24055ff0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f1b2801a000 tx=0x7f1b28014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:26.144 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.124+0000 7f1b377fe700 1 -- 172.21.15.47:0/4013516429 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1b30043170 con 0x7f1b4005f0b0 2022-06-29T15:42:26.254 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.252+0000 7f1b448e7700 1 -- 172.21.15.47:0/4013516429 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f1b400b2920 con 0x7f1b4005f0b0 2022-06-29T15:42:26.993 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.988+0000 7f1b377fe700 1 -- 172.21.15.47:0/4013516429 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v18) v1 ==== 173+0+0 (secure 0 0 0) 0x7f1b30046790 con 0x7f1b4005f0b0 2022-06-29T15:42:26.997 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.992+0000 7f1b448e7700 1 -- 172.21.15.47:0/4013516429 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1b24053b30 msgr2=0x7f1b24055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:26.997 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.992+0000 7f1b448e7700 1 --2- 172.21.15.47:0/4013516429 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1b24053b30 0x7f1b24055ff0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f1b2801a000 tx=0x7f1b28014000).stop 2022-06-29T15:42:26.998 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.992+0000 7f1b448e7700 1 -- 172.21.15.47:0/4013516429 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b4005f0b0 msgr2=0x7f1b40145700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:26.998 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.992+0000 7f1b448e7700 1 --2- 172.21.15.47:0/4013516429 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b4005f0b0 0x7f1b40145700 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f1b3000c000 tx=0x7f1b30006df0).stop 2022-06-29T15:42:27.000 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.996+0000 7f1b448e7700 1 -- 172.21.15.47:0/4013516429 shutdown_connections 2022-06-29T15:42:27.000 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.996+0000 7f1b448e7700 1 --2- 172.21.15.47:0/4013516429 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1b24053b30 0x7f1b24055ff0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:27.001 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.996+0000 7f1b448e7700 1 --2- 172.21.15.47:0/4013516429 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b400b1e20 0x7f1b40152c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:27.001 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.996+0000 7f1b448e7700 1 --2- 172.21.15.47:0/4013516429 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1b400619b0 0x7f1b40152740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:27.002 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.996+0000 7f1b448e7700 1 --2- 172.21.15.47:0/4013516429 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b4005f0b0 0x7f1b40145700 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:27.002 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.996+0000 7f1b448e7700 1 -- 172.21.15.47:0/4013516429 >> 172.21.15.47:0/4013516429 conn(0x7f1b40063800 msgr2=0x7f1b400b4540 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:27.002 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.996+0000 7f1b448e7700 1 -- 172.21.15.47:0/4013516429 shutdown_connections 2022-06-29T15:42:27.003 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:26.996+0000 7f1b448e7700 1 -- 172.21.15.47:0/4013516429 wait complete. 2022-06-29T15:42:27.004 INFO:teuthology.orchestra.run.smithi047.stderr:enabled application 'rbd' on pool 'rbd' 2022-06-29T15:42:27.012 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2022-06-29T15:42:27.012 INFO:tasks.ceph.mds.a:Restarting daemon 2022-06-29T15:42:27.013 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-06-29T15:42:27.015 INFO:tasks.ceph.mds.a:Started 2022-06-29T15:42:27.016 INFO:tasks.ceph.mds.c:Restarting daemon 2022-06-29T15:42:27.016 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2022-06-29T15:42:27.018 INFO:tasks.ceph.mds.c:Started 2022-06-29T15:42:27.019 INFO:tasks.ceph.mds.b:Restarting daemon 2022-06-29T15:42:27.019 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-06-29T15:42:27.022 INFO:tasks.ceph.mds.b:Started 2022-06-29T15:42:27.022 INFO:tasks.ceph:Setting up CephFS filesystem... 2022-06-29T15:42:27.023 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:42:27.089 INFO:tasks.ceph.mds.c.smithi047.stdout:starting mds.c at 2022-06-29T15:42:27.097 INFO:tasks.ceph.mds.b.smithi064.stdout:starting mds.b at 2022-06-29T15:42:27.098 INFO:tasks.ceph.mds.a.smithi047.stdout:starting mds.a at 2022-06-29T15:42:27.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.124+0000 7f3c410cb700 1 Processor -- start 2022-06-29T15:42:27.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.124+0000 7f3c410cb700 1 -- start start 2022-06-29T15:42:27.131 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.128+0000 7f3c410cb700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3c3c0b2740 0x7f3c3c0aa9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:27.132 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.128+0000 7f3c410cb700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3c3c1496b0 0x7f3c3c14baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:27.132 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.128+0000 7f3c410cb700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c3c0ae3e0 con 0x7f3c3c0af990 2022-06-29T15:42:27.132 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.128+0000 7f3c410cb700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c3c0af110 con 0x7f3c3c0b2740 2022-06-29T15:42:27.132 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.128+0000 7f3c410cb700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c3c0a9e80 con 0x7f3c3c1496b0 2022-06-29T15:42:27.132 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.128+0000 7f3c337fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f3c3c0af990 0x7f3c3c0ac180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59290/0 (socket says 172.21.15.47:59290) 2022-06-29T15:42:27.133 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.128+0000 7f3c337fe700 1 -- 172.21.15.47:0/721961593 learned_addr learned my addr 172.21.15.47:0/721961593 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:27.133 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.128+0000 7f3c408ca700 1 --2- 172.21.15.47:0/721961593 >> v2:172.21.15.64:3300/0 conn(0x7f3c3c1496b0 0x7f3c3c14baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:27.133 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.128+0000 7f3c3bfff700 1 --2- 172.21.15.47:0/721961593 >> v2:172.21.15.47:3300/0 conn(0x7f3c3c0b2740 0x7f3c3c0aa9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:27.133 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.128+0000 7f3c3b7fe700 1 -- 172.21.15.47:0/721961593 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4112702372 0 0) 0x7f3c3c0ae3e0 con 0x7f3c3c0af990 2022-06-29T15:42:27.134 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.128+0000 7f3c3b7fe700 1 -- 172.21.15.47:0/721961593 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c1c003260 con 0x7f3c3c0af990 2022-06-29T15:42:27.134 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.128+0000 7f3c3bfff700 1 -- 172.21.15.47:0/721961593 >> v2:172.21.15.64:3300/0 conn(0x7f3c3c1496b0 msgr2=0x7f3c3c14baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:27.134 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.128+0000 7f3c3bfff700 1 --2- 172.21.15.47:0/721961593 >> v2:172.21.15.64:3300/0 conn(0x7f3c3c1496b0 0x7f3c3c14baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:27.134 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.128+0000 7f3c3bfff700 1 -- 172.21.15.47:0/721961593 >> v1:172.21.15.47:6789/0 conn(0x7f3c3c0af990 legacy=0x7f3c3c0ac180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:27.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.128+0000 7f3c3bfff700 1 -- 172.21.15.47:0/721961593 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c3c0b1750 con 0x7f3c3c0b2740 2022-06-29T15:42:27.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.128+0000 7f3c408ca700 1 --2- 172.21.15.47:0/721961593 >> v2:172.21.15.64:3300/0 conn(0x7f3c3c1496b0 0x7f3c3c14baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:42:27.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.128+0000 7f3c3bfff700 1 --2- 172.21.15.47:0/721961593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c3c0b2740 0x7f3c3c0aa9b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f3c2c002f70 tx=0x7f3c2c006df0).ready entity=mon.0 client_cookie=53f996ad7e41b363 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:27.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.128+0000 7f3c3b7fe700 1 -- 172.21.15.47:0/721961593 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c2c00d4f0 con 0x7f3c3c0b2740 2022-06-29T15:42:27.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.128+0000 7f3c3b7fe700 1 -- 172.21.15.47:0/721961593 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c2c000e90 con 0x7f3c3c0b2740 2022-06-29T15:42:27.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.128+0000 7f3c3b7fe700 1 -- 172.21.15.47:0/721961593 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c2c00d840 con 0x7f3c3c0b2740 2022-06-29T15:42:27.140 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.128+0000 7f3c410cb700 1 -- 172.21.15.47:0/721961593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c3c0b2740 msgr2=0x7f3c3c0aa9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:27.140 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.128+0000 7f3c410cb700 1 --2- 172.21.15.47:0/721961593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c3c0b2740 0x7f3c3c0aa9b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f3c2c002f70 tx=0x7f3c2c006df0).stop 2022-06-29T15:42:27.141 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.128+0000 7f3c410cb700 1 -- 172.21.15.47:0/721961593 shutdown_connections 2022-06-29T15:42:27.141 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.128+0000 7f3c410cb700 1 --2- 172.21.15.47:0/721961593 >> v2:172.21.15.64:3300/0 conn(0x7f3c3c1496b0 0x7f3c3c14baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:27.141 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.128+0000 7f3c410cb700 1 --2- 172.21.15.47:0/721961593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c3c0b2740 0x7f3c3c0aa9b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:27.141 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.128+0000 7f3c410cb700 1 -- 172.21.15.47:0/721961593 >> 172.21.15.47:0/721961593 conn(0x7f3c3c062df0 msgr2=0x7f3c3c0631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:27.141 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.128+0000 7f3c410cb700 1 -- 172.21.15.47:0/721961593 shutdown_connections 2022-06-29T15:42:27.142 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.128+0000 7f3c410cb700 1 -- 172.21.15.47:0/721961593 wait complete. 2022-06-29T15:42:27.142 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.132+0000 7f3c410cb700 1 Processor -- start 2022-06-29T15:42:27.142 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.132+0000 7f3c410cb700 1 -- start start 2022-06-29T15:42:27.142 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.132+0000 7f3c410cb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c3c0af990 0x7f3c3c1447a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:27.143 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.132+0000 7f3c410cb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c3c0b2740 0x7f3c3c153b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:27.143 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.132+0000 7f3c410cb700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3c3c1496b0 0x7f3c3c154070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:27.143 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.132+0000 7f3c410cb700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3c1534f0 con 0x7f3c3c0af990 2022-06-29T15:42:27.143 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.132+0000 7f3c410cb700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3c067560 con 0x7f3c3c0b2740 2022-06-29T15:42:27.144 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.132+0000 7f3c410cb700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3c067800 con 0x7f3c3c1496b0 2022-06-29T15:42:27.144 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.132+0000 7f3c3bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c3c0af990 0x7f3c3c1447a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:27.144 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.132+0000 7f3c3bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c3c0af990 0x7f3c3c1447a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45508/0 (socket says 172.21.15.47:45508) 2022-06-29T15:42:27.144 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.132+0000 7f3c3bfff700 1 -- 172.21.15.47:0/3296712155 learned_addr learned my addr 172.21.15.47:0/3296712155 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:27.145 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.132+0000 7f3c337fe700 1 --2- 172.21.15.47:0/3296712155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c3c0b2740 0x7f3c3c153b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:27.145 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.132+0000 7f3c408ca700 1 --2- 172.21.15.47:0/3296712155 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3c3c1496b0 0x7f3c3c154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:27.145 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.132+0000 7f3c3bfff700 1 -- 172.21.15.47:0/3296712155 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3c3c1496b0 msgr2=0x7f3c3c154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:27.145 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.132+0000 7f3c3bfff700 1 --2- 172.21.15.47:0/3296712155 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3c3c1496b0 0x7f3c3c154070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:27.146 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.132+0000 7f3c3bfff700 1 -- 172.21.15.47:0/3296712155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c3c0b2740 msgr2=0x7f3c3c153b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:27.146 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.132+0000 7f3c3bfff700 1 --2- 172.21.15.47:0/3296712155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c3c0b2740 0x7f3c3c153b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:27.146 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.132+0000 7f3c3bfff700 1 -- 172.21.15.47:0/3296712155 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c34009470 con 0x7f3c3c0af990 2022-06-29T15:42:27.146 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.132+0000 7f3c3bfff700 1 --2- 172.21.15.47:0/3296712155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c3c0af990 0x7f3c3c1447a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f3c2c002f70 tx=0x7f3c2c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:27.147 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.132+0000 7f3c39ffb700 1 -- 172.21.15.47:0/3296712155 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c2c016070 con 0x7f3c3c0af990 2022-06-29T15:42:27.147 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.132+0000 7f3c39ffb700 1 -- 172.21.15.47:0/3296712155 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c2c0139c0 con 0x7f3c3c0af990 2022-06-29T15:42:27.147 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.132+0000 7f3c410cb700 1 -- 172.21.15.47:0/3296712155 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c2c00b180 con 0x7f3c3c0af990 2022-06-29T15:42:27.147 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.132+0000 7f3c39ffb700 1 -- 172.21.15.47:0/3296712155 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c2c013d20 con 0x7f3c3c0af990 2022-06-29T15:42:27.148 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.132+0000 7f3c410cb700 1 -- 172.21.15.47:0/3296712155 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c3c148fb0 con 0x7f3c3c0af990 2022-06-29T15:42:27.148 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.136+0000 7f3c410cb700 1 -- 172.21.15.47:0/3296712155 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c3c0573d0 con 0x7f3c3c0af990 2022-06-29T15:42:27.148 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.136+0000 7f3c39ffb700 1 -- 172.21.15.47:0/3296712155 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3c2c025030 con 0x7f3c3c0af990 2022-06-29T15:42:27.148 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.136+0000 7f3c39ffb700 1 --2- 172.21.15.47:0/3296712155 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3c200539e0 0x7f3c20055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:27.148 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.136+0000 7f3c39ffb700 1 -- 172.21.15.47:0/3296712155 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f3c2c06cb10 con 0x7f3c3c0af990 2022-06-29T15:42:27.149 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.136+0000 7f3c337fe700 1 --2- 172.21.15.47:0/3296712155 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3c200539e0 0x7f3c20055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:27.149 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.136+0000 7f3c337fe700 1 --2- 172.21.15.47:0/3296712155 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3c200539e0 0x7f3c20055ea0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f3c2400b000 tx=0x7f3c24009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:27.149 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.144+0000 7f3c39ffb700 1 -- 172.21.15.47:0/3296712155 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3c2c043d60 con 0x7f3c3c0af990 2022-06-29T15:42:27.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.272+0000 7f3c410cb700 1 -- 172.21.15.47:0/3296712155 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3c3c06a0f0 con 0x7f3c3c0af990 2022-06-29T15:42:27.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.272+0000 7f3c39ffb700 1 -- 172.21.15.47:0/3296712155 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+13191 (secure 0 0 0) 0x7f3c2c023070 con 0x7f3c3c0af990 2022-06-29T15:42:27.276 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:42:27.276 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":18,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:42:26.977420+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:42:27.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.276+0000 7f3c410cb700 1 -- 172.21.15.47:0/3296712155 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3c200539e0 msgr2=0x7f3c20055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:27.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.276+0000 7f3c410cb700 1 --2- 172.21.15.47:0/3296712155 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3c200539e0 0x7f3c20055ea0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f3c2400b000 tx=0x7f3c24009040).stop 2022-06-29T15:42:27.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.276+0000 7f3c410cb700 1 -- 172.21.15.47:0/3296712155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c3c0af990 msgr2=0x7f3c3c1447a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:27.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.276+0000 7f3c410cb700 1 --2- 172.21.15.47:0/3296712155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c3c0af990 0x7f3c3c1447a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f3c2c002f70 tx=0x7f3c2c014000).stop 2022-06-29T15:42:27.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.276+0000 7f3c410cb700 1 -- 172.21.15.47:0/3296712155 shutdown_connections 2022-06-29T15:42:27.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.276+0000 7f3c410cb700 1 --2- 172.21.15.47:0/3296712155 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3c3c1496b0 0x7f3c3c154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:27.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.276+0000 7f3c410cb700 1 --2- 172.21.15.47:0/3296712155 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3c200539e0 0x7f3c20055ea0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:27.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.276+0000 7f3c410cb700 1 --2- 172.21.15.47:0/3296712155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c3c0b2740 0x7f3c3c153b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:27.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.276+0000 7f3c410cb700 1 --2- 172.21.15.47:0/3296712155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c3c0af990 0x7f3c3c1447a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:27.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.276+0000 7f3c410cb700 1 -- 172.21.15.47:0/3296712155 >> 172.21.15.47:0/3296712155 conn(0x7f3c3c062df0 msgr2=0x7f3c3c09b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:27.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.276+0000 7f3c410cb700 1 -- 172.21.15.47:0/3296712155 shutdown_connections 2022-06-29T15:42:27.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.276+0000 7f3c410cb700 1 -- 172.21.15.47:0/3296712155 wait complete. 2022-06-29T15:42:27.289 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-29T15:42:16.971431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-29T15:42:25.290884+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-06-29T15:42:27.290 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-29T15:42:27.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.396+0000 7f6af2b3d700 1 Processor -- start 2022-06-29T15:42:27.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.396+0000 7f6af2b3d700 1 -- start start 2022-06-29T15:42:27.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.400+0000 7f6af2b3d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6aec1472f0 0x7f6aec13d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:27.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.400+0000 7f6af2b3d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6aec0671a0 0x7f6aec1445d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:27.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.400+0000 7f6af2b3d700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aec1434f0 con 0x7f6aec066dc0 2022-06-29T15:42:27.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.400+0000 7f6af2b3d700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6aec143790 con 0x7f6aec1472f0 2022-06-29T15:42:27.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.400+0000 7f6af2b3d700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec143a30 con 0x7f6aec0671a0 2022-06-29T15:42:27.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.400+0000 7f6af08d9700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f6aec066dc0 0x7f6aec13cab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59302/0 (socket says 172.21.15.47:59302) 2022-06-29T15:42:27.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.400+0000 7f6af08d9700 1 -- 172.21.15.47:0/234832179 learned_addr learned my addr 172.21.15.47:0/234832179 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:27.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.400+0000 7f6aebfff700 1 --2- 172.21.15.47:0/234832179 >> v2:172.21.15.64:3300/0 conn(0x7f6aec1472f0 0x7f6aec13d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:27.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.400+0000 7f6af10da700 1 --2- 172.21.15.47:0/234832179 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6aec0671a0 0x7f6aec1445d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:27.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.400+0000 7f6aeb7fe700 1 -- 172.21.15.47:0/234832179 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2239608926 0 0) 0x7f6aec1434f0 con 0x7f6aec066dc0 2022-06-29T15:42:27.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.400+0000 7f6aeb7fe700 1 -- 172.21.15.47:0/234832179 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ad8003260 con 0x7f6aec066dc0 2022-06-29T15:42:27.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.400+0000 7f6aeb7fe700 1 -- 172.21.15.47:0/234832179 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1259778546 0 0) 0x7f6ad8003260 con 0x7f6aec066dc0 2022-06-29T15:42:27.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.400+0000 7f6aeb7fe700 1 -- 172.21.15.47:0/234832179 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6aec0671a0 msgr2=0x7f6aec1445d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:27.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.400+0000 7f6aeb7fe700 1 --2- 172.21.15.47:0/234832179 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6aec0671a0 0x7f6aec1445d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:27.409 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.400+0000 7f6aeb7fe700 1 -- 172.21.15.47:0/234832179 >> v2:172.21.15.64:3300/0 conn(0x7f6aec1472f0 msgr2=0x7f6aec13d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:27.409 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.400+0000 7f6aeb7fe700 1 --2- 172.21.15.47:0/234832179 >> v2:172.21.15.64:3300/0 conn(0x7f6aec1472f0 0x7f6aec13d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:27.409 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.400+0000 7f6aeb7fe700 1 -- 172.21.15.47:0/234832179 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aec143cd0 con 0x7f6aec066dc0 2022-06-29T15:42:27.410 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.400+0000 7f6aeb7fe700 1 -- 172.21.15.47:0/234832179 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7f6ae0001680 con 0x7f6aec066dc0 2022-06-29T15:42:27.410 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.400+0000 7f6aeb7fe700 1 -- 172.21.15.47:0/234832179 >> v1:172.21.15.47:6789/0 conn(0x7f6aec066dc0 legacy=0x7f6aec13cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:27.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.400+0000 7f6aeb7fe700 1 --2- 172.21.15.47:0/234832179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ad8003750 0x7f6ad8003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:27.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.400+0000 7f6aeb7fe700 1 --2- 172.21.15.47:0/234832179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ad8004500 0x7f6ad8004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:27.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.400+0000 7f6aeb7fe700 1 --2- 172.21.15.47:0/234832179 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6aec0671a0 0x7f6ad8004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:27.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.400+0000 7f6aeb7fe700 1 -- 172.21.15.47:0/234832179 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec143790 con 0x7f6ad8004500 2022-06-29T15:42:27.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.400+0000 7f6aeb7fe700 1 -- 172.21.15.47:0/234832179 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec143a30 con 0x7f6ad8003750 2022-06-29T15:42:27.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.400+0000 7f6aeb7fe700 1 -- 172.21.15.47:0/234832179 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad800b750 con 0x7f6aec0671a0 2022-06-29T15:42:27.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.400+0000 7f6aebfff700 1 --2- 172.21.15.47:0/234832179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ad8004500 0x7f6ad8004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:27.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.400+0000 7f6af08d9700 1 --2- 172.21.15.47:0/234832179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ad8003750 0x7f6ad8003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:27.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.400+0000 7f6af10da700 1 --2- 172.21.15.47:0/234832179 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6aec0671a0 0x7f6ad8004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:27.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.400+0000 7f6aebfff700 1 -- 172.21.15.47:0/234832179 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6aec0671a0 msgr2=0x7f6ad8004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:27.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.400+0000 7f6aebfff700 1 --2- 172.21.15.47:0/234832179 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6aec0671a0 0x7f6ad8004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:27.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.400+0000 7f6aebfff700 1 -- 172.21.15.47:0/234832179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ad8003750 msgr2=0x7f6ad8003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:27.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.400+0000 7f6aebfff700 1 --2- 172.21.15.47:0/234832179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ad8003750 0x7f6ad8003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:27.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.400+0000 7f6aebfff700 1 -- 172.21.15.47:0/234832179 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad8003260 con 0x7f6ad8004500 2022-06-29T15:42:27.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.400+0000 7f6af08d9700 1 --2- 172.21.15.47:0/234832179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ad8003750 0x7f6ad8003b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:42:27.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.400+0000 7f6aebfff700 1 --2- 172.21.15.47:0/234832179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ad8004500 0x7f6ad8004980 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f6adc00c000 tx=0x7f6adc006df0).ready entity=mon.0 client_cookie=a0b25dcf3992bcdc server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:27.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.400+0000 7f6aeb7fe700 1 -- 172.21.15.47:0/234832179 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6adc00e830 con 0x7f6ad8004500 2022-06-29T15:42:27.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.400+0000 7f6aeb7fe700 1 -- 172.21.15.47:0/234832179 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6adc00e990 con 0x7f6ad8004500 2022-06-29T15:42:27.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.400+0000 7f6aeb7fe700 1 -- 172.21.15.47:0/234832179 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6adc00ecf0 con 0x7f6ad8004500 2022-06-29T15:42:27.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.404+0000 7f6af2b3d700 1 -- 172.21.15.47:0/234832179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ad8004500 msgr2=0x7f6ad8004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:27.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.404+0000 7f6af2b3d700 1 --2- 172.21.15.47:0/234832179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ad8004500 0x7f6ad8004980 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f6adc00c000 tx=0x7f6adc006df0).stop 2022-06-29T15:42:27.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.404+0000 7f6af10da700 1 -- 172.21.15.47:0/234832179 reap_dead start 2022-06-29T15:42:27.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.404+0000 7f6af2b3d700 1 -- 172.21.15.47:0/234832179 shutdown_connections 2022-06-29T15:42:27.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.404+0000 7f6af2b3d700 1 -- 172.21.15.47:0/234832179 >> 172.21.15.47:0/234832179 conn(0x7f6aec05c510 msgr2=0x7f6aec05c920 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:27.424 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.404+0000 7f6af2b3d700 1 -- 172.21.15.47:0/234832179 shutdown_connections 2022-06-29T15:42:27.424 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.404+0000 7f6af2b3d700 1 -- 172.21.15.47:0/234832179 wait complete. 2022-06-29T15:42:27.424 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.404+0000 7f6af2b3d700 1 Processor -- start 2022-06-29T15:42:27.425 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.404+0000 7f6af2b3d700 1 -- start start 2022-06-29T15:42:27.425 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.408+0000 7f6af2b3d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ad8004500 0x7f6aec0b4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:27.426 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.408+0000 7f6af2b3d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6aec142e00 0x7f6aec1418a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:27.426 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.408+0000 7f6af2b3d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6aec0b4910 0x7f6aec0b2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:27.426 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.408+0000 7f6af2b3d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec152c30 con 0x7f6aec142e00 2022-06-29T15:42:27.427 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.408+0000 7f6af2b3d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec152e90 con 0x7f6ad8004500 2022-06-29T15:42:27.427 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.408+0000 7f6af2b3d700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec0b2190 con 0x7f6aec0b4910 2022-06-29T15:42:27.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.408+0000 7f6aebfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6aec142e00 0x7f6aec1418a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:27.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.408+0000 7f6af08d9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ad8004500 0x7f6aec0b4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:27.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.408+0000 7f6aebfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6aec142e00 0x7f6aec1418a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45530/0 (socket says 172.21.15.47:45530) 2022-06-29T15:42:27.429 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.408+0000 7f6aebfff700 1 -- 172.21.15.47:0/1067364037 learned_addr learned my addr 172.21.15.47:0/1067364037 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:27.429 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.408+0000 7f6af10da700 1 --2- 172.21.15.47:0/1067364037 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6aec0b4910 0x7f6aec0b2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:27.429 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.408+0000 7f6aebfff700 1 -- 172.21.15.47:0/1067364037 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6aec0b4910 msgr2=0x7f6aec0b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:27.430 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.408+0000 7f6aebfff700 1 --2- 172.21.15.47:0/1067364037 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6aec0b4910 0x7f6aec0b2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:27.430 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.408+0000 7f6aebfff700 1 -- 172.21.15.47:0/1067364037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ad8004500 msgr2=0x7f6aec0b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:27.430 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.408+0000 7f6aebfff700 1 --2- 172.21.15.47:0/1067364037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ad8004500 0x7f6aec0b4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:27.431 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.408+0000 7f6aebfff700 1 -- 172.21.15.47:0/1067364037 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aec141d90 con 0x7f6aec142e00 2022-06-29T15:42:27.431 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.408+0000 7f6af08d9700 1 --2- 172.21.15.47:0/1067364037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ad8004500 0x7f6aec0b4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:42:27.431 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.408+0000 7f6aebfff700 1 --2- 172.21.15.47:0/1067364037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6aec142e00 0x7f6aec1418a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f6adc00c000 tx=0x7f6adc019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:27.431 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.408+0000 7f6ae9ffb700 1 -- 172.21.15.47:0/1067364037 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6adc01b820 con 0x7f6aec142e00 2022-06-29T15:42:27.432 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.408+0000 7f6ae9ffb700 1 -- 172.21.15.47:0/1067364037 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6adc000e90 con 0x7f6aec142e00 2022-06-29T15:42:27.432 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.408+0000 7f6af2b3d700 1 -- 172.21.15.47:0/1067364037 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aec0b2b20 con 0x7f6aec142e00 2022-06-29T15:42:27.432 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.408+0000 7f6ae9ffb700 1 -- 172.21.15.47:0/1067364037 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6adc01bb70 con 0x7f6aec142e00 2022-06-29T15:42:27.432 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.408+0000 7f6af2b3d700 1 -- 172.21.15.47:0/1067364037 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aec142210 con 0x7f6aec142e00 2022-06-29T15:42:27.433 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.408+0000 7f6af2b3d700 1 -- 172.21.15.47:0/1067364037 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aec0573d0 con 0x7f6aec142e00 2022-06-29T15:42:27.433 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.412+0000 7f6ae9ffb700 1 -- 172.21.15.47:0/1067364037 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6adc01bcd0 con 0x7f6aec142e00 2022-06-29T15:42:27.433 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.412+0000 7f6ae9ffb700 1 --2- 172.21.15.47:0/1067364037 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6acc053b00 0x7f6acc055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:27.433 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.412+0000 7f6af08d9700 1 --2- 172.21.15.47:0/1067364037 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6acc053b00 0x7f6acc055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:27.433 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.412+0000 7f6ae9ffb700 1 -- 172.21.15.47:0/1067364037 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f6adc06d190 con 0x7f6aec142e00 2022-06-29T15:42:27.434 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.412+0000 7f6af08d9700 1 --2- 172.21.15.47:0/1067364037 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6acc053b00 0x7f6acc055fc0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f6ae0005000 tx=0x7f6aec0669a0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:27.434 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.420+0000 7f6ae9ffb700 1 -- 172.21.15.47:0/1067364037 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6adc047d00 con 0x7f6aec142e00 2022-06-29T15:42:27.549 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.544+0000 7f6af2b3d700 1 -- 172.21.15.47:0/1067364037 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f6aec153000 con 0x7f6aec142e00 2022-06-29T15:42:27.550 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.548+0000 7f6ae9ffb700 1 -- 172.21.15.47:0/1067364037 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v18) v1 ==== 110+0+10 (secure 0 0 0) 0x7f6adc0478f0 con 0x7f6aec142e00 2022-06-29T15:42:27.551 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2022-06-29T15:42:27.553 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.548+0000 7f6af2b3d700 1 -- 172.21.15.47:0/1067364037 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6acc053b00 msgr2=0x7f6acc055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:27.553 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.548+0000 7f6af2b3d700 1 --2- 172.21.15.47:0/1067364037 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6acc053b00 0x7f6acc055fc0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f6ae0005000 tx=0x7f6aec0669a0).stop 2022-06-29T15:42:27.554 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.548+0000 7f6af2b3d700 1 -- 172.21.15.47:0/1067364037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6aec142e00 msgr2=0x7f6aec1418a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:27.554 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.548+0000 7f6af2b3d700 1 --2- 172.21.15.47:0/1067364037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6aec142e00 0x7f6aec1418a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f6adc00c000 tx=0x7f6adc019000).stop 2022-06-29T15:42:27.555 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.548+0000 7f6af2b3d700 1 -- 172.21.15.47:0/1067364037 shutdown_connections 2022-06-29T15:42:27.555 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.548+0000 7f6af2b3d700 1 --2- 172.21.15.47:0/1067364037 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6aec0b4910 0x7f6aec0b2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:27.556 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.548+0000 7f6af2b3d700 1 --2- 172.21.15.47:0/1067364037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6aec142e00 0x7f6aec1418a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:27.556 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.548+0000 7f6af2b3d700 1 --2- 172.21.15.47:0/1067364037 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6acc053b00 0x7f6acc055fc0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:27.557 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.548+0000 7f6af2b3d700 1 --2- 172.21.15.47:0/1067364037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ad8004500 0x7f6aec0b4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:27.557 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.548+0000 7f6af2b3d700 1 -- 172.21.15.47:0/1067364037 >> 172.21.15.47:0/1067364037 conn(0x7f6aec05c510 msgr2=0x7f6aec13ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:27.557 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.548+0000 7f6af2b3d700 1 -- 172.21.15.47:0/1067364037 shutdown_connections 2022-06-29T15:42:27.558 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.548+0000 7f6af2b3d700 1 -- 172.21.15.47:0/1067364037 wait complete. 2022-06-29T15:42:27.567 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-29T15:42:27.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.668+0000 7f0141e0f700 1 Processor -- start 2022-06-29T15:42:27.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.668+0000 7f0141e0f700 1 -- start start 2022-06-29T15:42:27.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.672+0000 7f0141e0f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f013c14b950 0x7f013c095220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:27.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.672+0000 7f0141e0f700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f013c0b3050 con 0x7f013c0b2c70 2022-06-29T15:42:27.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.672+0000 7f0141e0f700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f013c148960 con 0x7f013c1484f0 2022-06-29T15:42:27.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.672+0000 7f0141e0f700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f013c095930 con 0x7f013c14b950 2022-06-29T15:42:27.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.672+0000 7f013b7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f013c0b2c70 0x7f013c147de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59322/0 (socket says 172.21.15.47:59322) 2022-06-29T15:42:27.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.672+0000 7f013b7fe700 1 -- 172.21.15.47:0/1669000211 learned_addr learned my addr 172.21.15.47:0/1669000211 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:27.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.672+0000 7f013affd700 1 --2- 172.21.15.47:0/1669000211 >> v2:172.21.15.64:3300/0 conn(0x7f013c14b950 0x7f013c095220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:27.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.672+0000 7f013a7fc700 1 -- 172.21.15.47:0/1669000211 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3101801428 0 0) 0x7f013c0b3050 con 0x7f013c0b2c70 2022-06-29T15:42:27.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.672+0000 7f013a7fc700 1 -- 172.21.15.47:0/1669000211 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0128003260 con 0x7f013c0b2c70 2022-06-29T15:42:27.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.672+0000 7f013a7fc700 1 -- 172.21.15.47:0/1669000211 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2004876287 0 0) 0x7f0128003260 con 0x7f013c0b2c70 2022-06-29T15:42:27.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.672+0000 7f013a7fc700 1 -- 172.21.15.47:0/1669000211 >> v2:172.21.15.64:3300/0 conn(0x7f013c14b950 msgr2=0x7f013c095220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:27.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.672+0000 7f013a7fc700 1 --2- 172.21.15.47:0/1669000211 >> v2:172.21.15.64:3300/0 conn(0x7f013c14b950 0x7f013c095220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:27.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.672+0000 7f013a7fc700 1 -- 172.21.15.47:0/1669000211 >> v1:172.21.15.64:6789/0 conn(0x7f013c1484f0 legacy=0x7f013c066fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:27.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.672+0000 7f013a7fc700 1 -- 172.21.15.47:0/1669000211 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f013c095bd0 con 0x7f013c0b2c70 2022-06-29T15:42:27.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.672+0000 7f013a7fc700 1 -- 172.21.15.47:0/1669000211 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7f012c001680 con 0x7f013c0b2c70 2022-06-29T15:42:27.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.672+0000 7f013a7fc700 1 -- 172.21.15.47:0/1669000211 >> v1:172.21.15.47:6789/0 conn(0x7f013c0b2c70 legacy=0x7f013c147de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:27.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.672+0000 7f013a7fc700 1 --2- 172.21.15.47:0/1669000211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0128003750 0x7f0128003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:27.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.672+0000 7f013a7fc700 1 --2- 172.21.15.47:0/1669000211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0128004500 0x7f0128004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:27.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.672+0000 7f013a7fc700 1 --2- 172.21.15.47:0/1669000211 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0128004e70 0x7f012800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:27.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.672+0000 7f013a7fc700 1 -- 172.21.15.47:0/1669000211 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f013c095930 con 0x7f0128003750 2022-06-29T15:42:27.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.672+0000 7f013a7fc700 1 -- 172.21.15.47:0/1669000211 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012800bd20 con 0x7f0128004500 2022-06-29T15:42:27.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.672+0000 7f013a7fc700 1 -- 172.21.15.47:0/1669000211 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f012800bee0 con 0x7f0128004e70 2022-06-29T15:42:27.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.672+0000 7f013b7fe700 1 --2- 172.21.15.47:0/1669000211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0128003750 0x7f0128003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:27.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.672+0000 7f013affd700 1 --2- 172.21.15.47:0/1669000211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0128004500 0x7f0128004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:27.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.672+0000 7f013bfff700 1 --2- 172.21.15.47:0/1669000211 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0128004e70 0x7f012800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:27.748 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.672+0000 7f013b7fe700 1 -- 172.21.15.47:0/1669000211 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0128004e70 msgr2=0x7f012800b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:27.748 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.672+0000 7f013b7fe700 1 --2- 172.21.15.47:0/1669000211 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0128004e70 0x7f012800b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:27.748 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.672+0000 7f013b7fe700 1 -- 172.21.15.47:0/1669000211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0128004500 msgr2=0x7f0128004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:27.748 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.672+0000 7f013b7fe700 1 --2- 172.21.15.47:0/1669000211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0128004500 0x7f0128004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:27.748 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.672+0000 7f013b7fe700 1 -- 172.21.15.47:0/1669000211 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0128003260 con 0x7f0128003750 2022-06-29T15:42:27.749 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.672+0000 7f013bfff700 1 --2- 172.21.15.47:0/1669000211 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0128004e70 0x7f012800b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:42:27.749 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.672+0000 7f013bfff700 1 -- 172.21.15.47:0/1669000211 reap_dead start 2022-06-29T15:42:27.749 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.672+0000 7f013b7fe700 1 --2- 172.21.15.47:0/1669000211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0128003750 0x7f0128003b70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f012c001390 tx=0x7f013c0b3560).ready entity=mon.0 client_cookie=d3b3006ccaf95026 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:27.749 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.672+0000 7f013a7fc700 1 -- 172.21.15.47:0/1669000211 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f012c007170 con 0x7f0128003750 2022-06-29T15:42:27.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.672+0000 7f013a7fc700 1 -- 172.21.15.47:0/1669000211 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f012c0072d0 con 0x7f0128003750 2022-06-29T15:42:27.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.672+0000 7f013a7fc700 1 -- 172.21.15.47:0/1669000211 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f012c00c9c0 con 0x7f0128003750 2022-06-29T15:42:27.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.672+0000 7f0141e0f700 1 -- 172.21.15.47:0/1669000211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0128003750 msgr2=0x7f0128003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:27.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.672+0000 7f0141e0f700 1 --2- 172.21.15.47:0/1669000211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0128003750 0x7f0128003b70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f012c001390 tx=0x7f013c0b3560).stop 2022-06-29T15:42:27.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.672+0000 7f0141e0f700 1 -- 172.21.15.47:0/1669000211 shutdown_connections 2022-06-29T15:42:27.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.672+0000 7f0141e0f700 1 --2- 172.21.15.47:0/1669000211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0128003750 0x7f0128003b70 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:27.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.672+0000 7f0141e0f700 1 -- 172.21.15.47:0/1669000211 >> 172.21.15.47:0/1669000211 conn(0x7f013c05f0b0 msgr2=0x7f013c05f490 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:27.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.672+0000 7f0141e0f700 1 -- 172.21.15.47:0/1669000211 shutdown_connections 2022-06-29T15:42:27.752 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.672+0000 7f0141e0f700 1 -- 172.21.15.47:0/1669000211 wait complete. 2022-06-29T15:42:27.752 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.676+0000 7f0141e0f700 1 Processor -- start 2022-06-29T15:42:27.752 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.676+0000 7f0141e0f700 1 -- start start 2022-06-29T15:42:27.752 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.676+0000 7f0141e0f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0128003750 0x7f013c0ac330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:27.753 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.676+0000 7f0141e0f700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f013c0aaad0 0x7f013c09b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:27.753 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.676+0000 7f0141e0f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f013c09b680 0x7f013c156b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:27.753 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.676+0000 7f0141e0f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f013c0ab020 con 0x7f0128003750 2022-06-29T15:42:27.753 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.676+0000 7f0141e0f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f013c0ab950 con 0x7f013c09b680 2022-06-29T15:42:27.754 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.676+0000 7f0141e0f700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f013c0ac9e0 con 0x7f013c0aaad0 2022-06-29T15:42:27.754 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.676+0000 7f013b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0128003750 0x7f013c0ac330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:27.754 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.676+0000 7f013b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0128003750 0x7f013c0ac330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45548/0 (socket says 172.21.15.47:45548) 2022-06-29T15:42:27.754 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.676+0000 7f013b7fe700 1 -- 172.21.15.47:0/758843952 learned_addr learned my addr 172.21.15.47:0/758843952 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:27.755 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.676+0000 7f013affd700 1 --2- 172.21.15.47:0/758843952 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f013c0aaad0 0x7f013c09b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:27.755 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.676+0000 7f013bfff700 1 --2- 172.21.15.47:0/758843952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f013c09b680 0x7f013c156b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:27.755 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.676+0000 7f013b7fe700 1 -- 172.21.15.47:0/758843952 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f013c0aaad0 msgr2=0x7f013c09b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:27.755 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.676+0000 7f013b7fe700 1 --2- 172.21.15.47:0/758843952 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f013c0aaad0 0x7f013c09b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:27.756 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.676+0000 7f013b7fe700 1 -- 172.21.15.47:0/758843952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f013c09b680 msgr2=0x7f013c156b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:27.756 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.676+0000 7f013b7fe700 1 --2- 172.21.15.47:0/758843952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f013c09b680 0x7f013c156b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:27.756 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.676+0000 7f013b7fe700 1 -- 172.21.15.47:0/758843952 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f013c0abbb0 con 0x7f0128003750 2022-06-29T15:42:27.756 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.676+0000 7f013b7fe700 1 --2- 172.21.15.47:0/758843952 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0128003750 0x7f013c0ac330 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f012c001390 tx=0x7f012c013000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:27.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.676+0000 7f0138ff9700 1 -- 172.21.15.47:0/758843952 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f012c007440 con 0x7f0128003750 2022-06-29T15:42:27.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.676+0000 7f0138ff9700 1 -- 172.21.15.47:0/758843952 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f012c000e90 con 0x7f0128003750 2022-06-29T15:42:27.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.676+0000 7f0141e0f700 1 -- 172.21.15.47:0/758843952 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f013c0aa630 con 0x7f0128003750 2022-06-29T15:42:27.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.676+0000 7f0138ff9700 1 -- 172.21.15.47:0/758843952 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f012c004920 con 0x7f0128003750 2022-06-29T15:42:27.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.676+0000 7f0141e0f700 1 -- 172.21.15.47:0/758843952 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f013c0affc0 con 0x7f0128003750 2022-06-29T15:42:27.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.680+0000 7f0141e0f700 1 -- 172.21.15.47:0/758843952 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f013c09ba90 con 0x7f0128003750 2022-06-29T15:42:27.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.680+0000 7f0138ff9700 1 -- 172.21.15.47:0/758843952 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f012c027070 con 0x7f0128003750 2022-06-29T15:42:27.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.680+0000 7f0138ff9700 1 --2- 172.21.15.47:0/758843952 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f011c053b30 0x7f011c055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:27.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.680+0000 7f0138ff9700 1 -- 172.21.15.47:0/758843952 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f012c06c050 con 0x7f0128003750 2022-06-29T15:42:27.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.680+0000 7f013affd700 1 --2- 172.21.15.47:0/758843952 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f011c053b30 0x7f011c055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:27.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.680+0000 7f013affd700 1 --2- 172.21.15.47:0/758843952 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f011c053b30 0x7f011c055ff0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f013000b000 tx=0x7f0130006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:27.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.684+0000 7f0138ff9700 1 -- 172.21.15.47:0/758843952 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f013c09ba90 con 0x7f0128003750 2022-06-29T15:42:27.815 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.812+0000 7f0141e0f700 1 -- 172.21.15.47:0/758843952 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f013c09ba90 con 0x7f0128003750 2022-06-29T15:42:27.816 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.812+0000 7f0138ff9700 1 -- 172.21.15.47:0/758843952 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v18) v1 ==== 92+0+10 (secure 0 0 0) 0x7f013c09ba90 con 0x7f0128003750 2022-06-29T15:42:27.817 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:42:27.818 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.816+0000 7f0141e0f700 1 -- 172.21.15.47:0/758843952 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f011c053b30 msgr2=0x7f011c055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:27.818 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.816+0000 7f0141e0f700 1 --2- 172.21.15.47:0/758843952 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f011c053b30 0x7f011c055ff0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f013000b000 tx=0x7f0130006df0).stop 2022-06-29T15:42:27.819 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.816+0000 7f0141e0f700 1 -- 172.21.15.47:0/758843952 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0128003750 msgr2=0x7f013c0ac330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:27.819 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.816+0000 7f0141e0f700 1 --2- 172.21.15.47:0/758843952 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0128003750 0x7f013c0ac330 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f012c001390 tx=0x7f012c013000).stop 2022-06-29T15:42:27.819 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.816+0000 7f0141e0f700 1 -- 172.21.15.47:0/758843952 shutdown_connections 2022-06-29T15:42:27.819 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.816+0000 7f0141e0f700 1 --2- 172.21.15.47:0/758843952 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f011c053b30 0x7f011c055ff0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:27.820 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.816+0000 7f0141e0f700 1 --2- 172.21.15.47:0/758843952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f013c09b680 0x7f013c156b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:27.820 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.816+0000 7f0141e0f700 1 --2- 172.21.15.47:0/758843952 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f013c0aaad0 0x7f013c09b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:27.820 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.816+0000 7f0141e0f700 1 --2- 172.21.15.47:0/758843952 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0128003750 0x7f013c0ac330 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:27.820 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.816+0000 7f0141e0f700 1 -- 172.21.15.47:0/758843952 >> 172.21.15.47:0/758843952 conn(0x7f013c05f0b0 msgr2=0x7f013c14ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:27.821 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.816+0000 7f0141e0f700 1 -- 172.21.15.47:0/758843952 shutdown_connections 2022-06-29T15:42:27.821 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.816+0000 7f0141e0f700 1 -- 172.21.15.47:0/758843952 wait complete. 2022-06-29T15:42:27.828 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-06-29T15:42:27.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.924+0000 7fe2e91e3700 1 Processor -- start 2022-06-29T15:42:27.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.928+0000 7fe2e91e3700 1 -- start start 2022-06-29T15:42:27.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.928+0000 7fe2e91e3700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe2e40b1470 0x7fe2e40af590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:27.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.928+0000 7fe2e91e3700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe2e40b04e0 0x7fe2e40ac9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:27.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.928+0000 7fe2e91e3700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe2e40aceb0 0x7fe2e4143090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:27.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.928+0000 7fe2e91e3700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2e40b27c0 con 0x7fe2e40b04e0 2022-06-29T15:42:27.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.928+0000 7fe2e91e3700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2e40b1850 con 0x7fe2e40b1470 2022-06-29T15:42:27.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.928+0000 7fe2e91e3700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e4142d50 con 0x7fe2e40aceb0 2022-06-29T15:42:27.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.928+0000 7fe2db7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe2e40b04e0 0x7fe2e40ac9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:27.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.928+0000 7fe2db7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe2e40b04e0 0x7fe2e40ac9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45556/0 (socket says 172.21.15.47:45556) 2022-06-29T15:42:27.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.928+0000 7fe2db7fe700 1 -- 172.21.15.47:0/300135165 learned_addr learned my addr 172.21.15.47:0/300135165 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:27.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.928+0000 7fe2e89e2700 1 --2- 172.21.15.47:0/300135165 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe2e40aceb0 0x7fe2e4143090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:27.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.928+0000 7fe2e3fff700 1 --2- 172.21.15.47:0/300135165 >> v2:172.21.15.64:3300/0 conn(0x7fe2e40b1470 0x7fe2e40af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:27.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.928+0000 7fe2db7fe700 1 -- 172.21.15.47:0/300135165 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe2e40aceb0 msgr2=0x7fe2e4143090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:27.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.928+0000 7fe2db7fe700 1 --2- 172.21.15.47:0/300135165 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe2e40aceb0 0x7fe2e4143090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:27.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.928+0000 7fe2db7fe700 1 -- 172.21.15.47:0/300135165 >> v2:172.21.15.64:3300/0 conn(0x7fe2e40b1470 msgr2=0x7fe2e40af590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:27.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.928+0000 7fe2db7fe700 1 --2- 172.21.15.47:0/300135165 >> v2:172.21.15.64:3300/0 conn(0x7fe2e40b1470 0x7fe2e40af590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:27.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.928+0000 7fe2db7fe700 1 -- 172.21.15.47:0/300135165 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e40ae9a0 con 0x7fe2e40b04e0 2022-06-29T15:42:27.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.928+0000 7fe2db7fe700 1 --2- 172.21.15.47:0/300135165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2e40b04e0 0x7fe2e40ac9c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fe2cc002f70 tx=0x7fe2cc006eb0).ready entity=mon.0 client_cookie=cbbf56fbced538ae server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:27.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.928+0000 7fe2e37fe700 1 -- 172.21.15.47:0/300135165 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe2cc00d570 con 0x7fe2e40b04e0 2022-06-29T15:42:27.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.928+0000 7fe2e37fe700 1 -- 172.21.15.47:0/300135165 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2cc000e90 con 0x7fe2e40b04e0 2022-06-29T15:42:27.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.928+0000 7fe2e37fe700 1 -- 172.21.15.47:0/300135165 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe2cc00d900 con 0x7fe2e40b04e0 2022-06-29T15:42:27.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.928+0000 7fe2e91e3700 1 -- 172.21.15.47:0/300135165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2e40b04e0 msgr2=0x7fe2e40ac9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:27.938 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.928+0000 7fe2e91e3700 1 --2- 172.21.15.47:0/300135165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2e40b04e0 0x7fe2e40ac9c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fe2cc002f70 tx=0x7fe2cc006eb0).stop 2022-06-29T15:42:27.938 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.928+0000 7fe2e91e3700 1 -- 172.21.15.47:0/300135165 shutdown_connections 2022-06-29T15:42:27.938 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.928+0000 7fe2e91e3700 1 --2- 172.21.15.47:0/300135165 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe2e40aceb0 0x7fe2e4143090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:27.938 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.928+0000 7fe2e91e3700 1 --2- 172.21.15.47:0/300135165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2e40b04e0 0x7fe2e40ac9c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:27.939 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.928+0000 7fe2e91e3700 1 --2- 172.21.15.47:0/300135165 >> v2:172.21.15.64:3300/0 conn(0x7fe2e40b1470 0x7fe2e40af590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:27.939 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.928+0000 7fe2e91e3700 1 -- 172.21.15.47:0/300135165 >> 172.21.15.47:0/300135165 conn(0x7fe2e4062560 msgr2=0x7fe2e4062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:27.939 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.928+0000 7fe2e91e3700 1 -- 172.21.15.47:0/300135165 shutdown_connections 2022-06-29T15:42:27.939 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.928+0000 7fe2e91e3700 1 -- 172.21.15.47:0/300135165 wait complete. 2022-06-29T15:42:27.942 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.932+0000 7fe2e91e3700 1 Processor -- start 2022-06-29T15:42:27.942 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.932+0000 7fe2e91e3700 1 -- start start 2022-06-29T15:42:27.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.932+0000 7fe2e91e3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2e40aceb0 0x7fe2e413c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:27.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.932+0000 7fe2e91e3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2e40b04e0 0x7fe2e413bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:27.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.932+0000 7fe2e91e3700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe2e40b1470 0x7fe2e413c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:27.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.932+0000 7fe2e91e3700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e413ea40 con 0x7fe2e40aceb0 2022-06-29T15:42:27.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.932+0000 7fe2e91e3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e413ec70 con 0x7fe2e40b04e0 2022-06-29T15:42:27.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.932+0000 7fe2e91e3700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e413a540 con 0x7fe2e40b1470 2022-06-29T15:42:27.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.932+0000 7fe2e3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2e40aceb0 0x7fe2e413c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:27.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.932+0000 7fe2e3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2e40aceb0 0x7fe2e413c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45562/0 (socket says 172.21.15.47:45562) 2022-06-29T15:42:27.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.932+0000 7fe2e3fff700 1 -- 172.21.15.47:0/2687439559 learned_addr learned my addr 172.21.15.47:0/2687439559 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:27.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.932+0000 7fe2db7fe700 1 --2- 172.21.15.47:0/2687439559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2e40b04e0 0x7fe2e413bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:27.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.932+0000 7fe2e89e2700 1 --2- 172.21.15.47:0/2687439559 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe2e40b1470 0x7fe2e413c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:27.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.932+0000 7fe2e3fff700 1 -- 172.21.15.47:0/2687439559 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe2e40b1470 msgr2=0x7fe2e413c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:27.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.932+0000 7fe2e3fff700 1 --2- 172.21.15.47:0/2687439559 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe2e40b1470 0x7fe2e413c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:27.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.932+0000 7fe2e3fff700 1 -- 172.21.15.47:0/2687439559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2e40b04e0 msgr2=0x7fe2e413bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:27.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.932+0000 7fe2e3fff700 1 --2- 172.21.15.47:0/2687439559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2e40b04e0 0x7fe2e413bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:27.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.932+0000 7fe2e3fff700 1 -- 172.21.15.47:0/2687439559 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2cc00b1e0 con 0x7fe2e40aceb0 2022-06-29T15:42:27.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.936+0000 7fe2e3fff700 1 --2- 172.21.15.47:0/2687439559 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2e40aceb0 0x7fe2e413c930 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fe2d400a000 tx=0x7fe2d4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:27.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.936+0000 7fe2e1ffb700 1 -- 172.21.15.47:0/2687439559 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe2d4011780 con 0x7fe2e40aceb0 2022-06-29T15:42:27.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.936+0000 7fe2e1ffb700 1 -- 172.21.15.47:0/2687439559 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2d40118e0 con 0x7fe2e40aceb0 2022-06-29T15:42:27.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.936+0000 7fe2e91e3700 1 -- 172.21.15.47:0/2687439559 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2e413f7f0 con 0x7fe2e40aceb0 2022-06-29T15:42:27.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.936+0000 7fe2e1ffb700 1 -- 172.21.15.47:0/2687439559 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe2d4011c40 con 0x7fe2e40aceb0 2022-06-29T15:42:27.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.936+0000 7fe2e91e3700 1 -- 172.21.15.47:0/2687439559 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2e413f0c0 con 0x7fe2e40aceb0 2022-06-29T15:42:27.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.936+0000 7fe2e1ffb700 1 -- 172.21.15.47:0/2687439559 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe2d401c460 con 0x7fe2e40aceb0 2022-06-29T15:42:27.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.936+0000 7fe2e91e3700 1 -- 172.21.15.47:0/2687439559 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2e413b8f0 con 0x7fe2e40aceb0 2022-06-29T15:42:27.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.936+0000 7fe2e1ffb700 1 --2- 172.21.15.47:0/2687439559 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe2c8053980 0x7fe2c8055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:27.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.936+0000 7fe2e1ffb700 1 -- 172.21.15.47:0/2687439559 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fe2d406c100 con 0x7fe2e40aceb0 2022-06-29T15:42:27.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.936+0000 7fe2db7fe700 1 --2- 172.21.15.47:0/2687439559 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe2c8053980 0x7fe2c8055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:27.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.940+0000 7fe2db7fe700 1 --2- 172.21.15.47:0/2687439559 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe2c8053980 0x7fe2c8055e40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fe2cc01a000 tx=0x7fe2cc015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:27.950 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:27.944+0000 7fe2e1ffb700 1 -- 172.21.15.47:0/2687439559 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe2d4043250 con 0x7fe2e40aceb0 2022-06-29T15:42:28.080 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.076+0000 7fe2e91e3700 1 -- 172.21.15.47:0/2687439559 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe2e40573d0 con 0x7fe2e40aceb0 2022-06-29T15:42:28.081 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.080+0000 7fe2e1ffb700 1 -- 172.21.15.47:0/2687439559 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) v1 ==== 84+0+143 (secure 0 0 0) 0x7fe2d4046870 con 0x7fe2e40aceb0 2022-06-29T15:42:28.082 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:42:28.082 INFO:teuthology.orchestra.run.smithi047.stdout:[ 2022-06-29T15:42:28.082 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:42:28.082 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 1, 2022-06-29T15:42:28.083 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": "device_health_metrics" 2022-06-29T15:42:28.083 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:42:28.083 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:42:28.083 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 2, 2022-06-29T15:42:28.084 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": "rbd" 2022-06-29T15:42:28.084 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:42:28.084 INFO:teuthology.orchestra.run.smithi047.stdout:] 2022-06-29T15:42:28.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.080+0000 7fe2e91e3700 1 -- 172.21.15.47:0/2687439559 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe2c8053980 msgr2=0x7fe2c8055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:28.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.080+0000 7fe2e91e3700 1 --2- 172.21.15.47:0/2687439559 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe2c8053980 0x7fe2c8055e40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fe2cc01a000 tx=0x7fe2cc015000).stop 2022-06-29T15:42:28.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.080+0000 7fe2e91e3700 1 -- 172.21.15.47:0/2687439559 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2e40aceb0 msgr2=0x7fe2e413c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:28.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.080+0000 7fe2e91e3700 1 --2- 172.21.15.47:0/2687439559 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2e40aceb0 0x7fe2e413c930 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fe2d400a000 tx=0x7fe2d4008040).stop 2022-06-29T15:42:28.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.080+0000 7fe2e91e3700 1 -- 172.21.15.47:0/2687439559 shutdown_connections 2022-06-29T15:42:28.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.080+0000 7fe2e91e3700 1 --2- 172.21.15.47:0/2687439559 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe2e40b1470 0x7fe2e413c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:28.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.080+0000 7fe2e91e3700 1 --2- 172.21.15.47:0/2687439559 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe2c8053980 0x7fe2c8055e40 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:28.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.080+0000 7fe2e91e3700 1 --2- 172.21.15.47:0/2687439559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2e40b04e0 0x7fe2e413bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:28.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.080+0000 7fe2e91e3700 1 --2- 172.21.15.47:0/2687439559 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2e40aceb0 0x7fe2e413c930 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:28.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.080+0000 7fe2e91e3700 1 -- 172.21.15.47:0/2687439559 >> 172.21.15.47:0/2687439559 conn(0x7fe2e4062560 msgr2=0x7fe2e4142340 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:28.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.080+0000 7fe2e91e3700 1 -- 172.21.15.47:0/2687439559 shutdown_connections 2022-06-29T15:42:28.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.080+0000 7fe2e91e3700 1 -- 172.21.15.47:0/2687439559 wait complete. 2022-06-29T15:42:28.094 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-06-29T15:42:28.094 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-06-29T15:42:28.204 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.200+0000 7f5e16a62700 1 Processor -- start 2022-06-29T15:42:28.204 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.200+0000 7f5e16a62700 1 -- start start 2022-06-29T15:42:28.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.204+0000 7f5e16a62700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5e1005f4a0 0x7f5e100b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:28.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.204+0000 7f5e16a62700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5e1009a1d0 0x7f5e1009a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:28.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.204+0000 7f5e16a62700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e1009ab20 con 0x7f5e10142a30 2022-06-29T15:42:28.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.204+0000 7f5e16a62700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e1005f880 con 0x7f5e1005f4a0 2022-06-29T15:42:28.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.204+0000 7f5e16a62700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1009ae90 con 0x7f5e1009a1d0 2022-06-29T15:42:28.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.204+0000 7f5e16261700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f5e10142a30 0x7f5e100b0550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59362/0 (socket says 172.21.15.47:59362) 2022-06-29T15:42:28.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.204+0000 7f5e16261700 1 -- 172.21.15.47:0/3563239694 learned_addr learned my addr 172.21.15.47:0/3563239694 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:28.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.204+0000 7f5e1525f700 1 --2- 172.21.15.47:0/3563239694 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5e1009a1d0 0x7f5e1009a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:28.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.204+0000 7f5e15a60700 1 --2- 172.21.15.47:0/3563239694 >> v2:172.21.15.64:3300/0 conn(0x7f5e1005f4a0 0x7f5e100b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:28.209 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.204+0000 7f5e14a5e700 1 -- 172.21.15.47:0/3563239694 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1157545794 0 0) 0x7f5e08001490 con 0x7f5e10142a30 2022-06-29T15:42:28.209 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.204+0000 7f5e14a5e700 1 -- 172.21.15.47:0/3563239694 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5df8003260 con 0x7f5e10142a30 2022-06-29T15:42:28.209 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.204+0000 7f5e1525f700 1 -- 172.21.15.47:0/3563239694 >> v2:172.21.15.64:3300/0 conn(0x7f5e1005f4a0 msgr2=0x7f5e100b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:28.209 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.204+0000 7f5e1525f700 1 --2- 172.21.15.47:0/3563239694 >> v2:172.21.15.64:3300/0 conn(0x7f5e1005f4a0 0x7f5e100b2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:28.210 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.204+0000 7f5e1525f700 1 -- 172.21.15.47:0/3563239694 >> v1:172.21.15.47:6789/0 conn(0x7f5e10142a30 legacy=0x7f5e100b0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:28.210 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.204+0000 7f5e1525f700 1 -- 172.21.15.47:0/3563239694 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e1009b130 con 0x7f5e1009a1d0 2022-06-29T15:42:28.210 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.204+0000 7f5e1525f700 1 --2- 172.21.15.47:0/3563239694 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5e1009a1d0 0x7f5e1009a630 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f5e00002f70 tx=0x7f5e00006e80).ready entity=mon.2 client_cookie=37e43e3b73d32c54 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:28.214 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.204+0000 7f5e14a5e700 1 -- 172.21.15.47:0/3563239694 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e0000f730 con 0x7f5e1009a1d0 2022-06-29T15:42:28.215 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.204+0000 7f5e14a5e700 1 -- 172.21.15.47:0/3563239694 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e00000e90 con 0x7f5e1009a1d0 2022-06-29T15:42:28.215 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.204+0000 7f5e14a5e700 1 -- 172.21.15.47:0/3563239694 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e0000fa80 con 0x7f5e1009a1d0 2022-06-29T15:42:28.215 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.204+0000 7f5e16a62700 1 -- 172.21.15.47:0/3563239694 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5e1009a1d0 msgr2=0x7f5e1009a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:28.215 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.204+0000 7f5e16a62700 1 --2- 172.21.15.47:0/3563239694 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5e1009a1d0 0x7f5e1009a630 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f5e00002f70 tx=0x7f5e00006e80).stop 2022-06-29T15:42:28.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.204+0000 7f5e16a62700 1 -- 172.21.15.47:0/3563239694 shutdown_connections 2022-06-29T15:42:28.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.204+0000 7f5e16a62700 1 --2- 172.21.15.47:0/3563239694 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5e1009a1d0 0x7f5e1009a630 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:28.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.204+0000 7f5e16a62700 1 --2- 172.21.15.47:0/3563239694 >> v2:172.21.15.64:3300/0 conn(0x7f5e1005f4a0 0x7f5e100b2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:28.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.204+0000 7f5e16a62700 1 -- 172.21.15.47:0/3563239694 >> 172.21.15.47:0/3563239694 conn(0x7f5e10062df0 msgr2=0x7f5e100631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:28.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.204+0000 7f5e16a62700 1 -- 172.21.15.47:0/3563239694 shutdown_connections 2022-06-29T15:42:28.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.204+0000 7f5e16a62700 1 -- 172.21.15.47:0/3563239694 wait complete. 2022-06-29T15:42:28.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.208+0000 7f5e16a62700 1 Processor -- start 2022-06-29T15:42:28.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.208+0000 7f5e16a62700 1 -- start start 2022-06-29T15:42:28.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.208+0000 7f5e16a62700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e1005f4a0 0x7f5e100af790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:28.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.208+0000 7f5e16a62700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e1009a1d0 0x7f5e10065dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:28.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.208+0000 7f5e16a62700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5e10142a30 0x7f5e10063ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:28.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.208+0000 7f5e16a62700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e100678d0 con 0x7f5e1005f4a0 2022-06-29T15:42:28.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.208+0000 7f5e16a62700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1014e9f0 con 0x7f5e1009a1d0 2022-06-29T15:42:28.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.208+0000 7f5e16a62700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1014ec90 con 0x7f5e10142a30 2022-06-29T15:42:28.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.208+0000 7f5e15a60700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e1005f4a0 0x7f5e100af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:28.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.208+0000 7f5e15a60700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e1005f4a0 0x7f5e100af790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45578/0 (socket says 172.21.15.47:45578) 2022-06-29T15:42:28.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.208+0000 7f5e15a60700 1 -- 172.21.15.47:0/3718030926 learned_addr learned my addr 172.21.15.47:0/3718030926 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:28.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.212+0000 7f5e1525f700 1 --2- 172.21.15.47:0/3718030926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e1009a1d0 0x7f5e10065dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:28.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.212+0000 7f5e16261700 1 --2- 172.21.15.47:0/3718030926 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5e10142a30 0x7f5e10063ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:28.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.212+0000 7f5e15a60700 1 -- 172.21.15.47:0/3718030926 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5e10142a30 msgr2=0x7f5e10063ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:28.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.212+0000 7f5e15a60700 1 --2- 172.21.15.47:0/3718030926 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5e10142a30 0x7f5e10063ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:28.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.212+0000 7f5e15a60700 1 -- 172.21.15.47:0/3718030926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e1009a1d0 msgr2=0x7f5e10065dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:28.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.212+0000 7f5e15a60700 1 --2- 172.21.15.47:0/3718030926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e1009a1d0 0x7f5e10065dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:28.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.212+0000 7f5e15a60700 1 -- 172.21.15.47:0/3718030926 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e0000c310 con 0x7f5e1005f4a0 2022-06-29T15:42:28.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.212+0000 7f5e15a60700 1 --2- 172.21.15.47:0/3718030926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e1005f4a0 0x7f5e100af790 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f5e0c00c000 tx=0x7f5e0c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:28.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.212+0000 7f5dfeffd700 1 -- 172.21.15.47:0/3718030926 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e0c010780 con 0x7f5e1005f4a0 2022-06-29T15:42:28.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.212+0000 7f5dfeffd700 1 -- 172.21.15.47:0/3718030926 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e0c0108e0 con 0x7f5e1005f4a0 2022-06-29T15:42:28.222 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.212+0000 7f5e16a62700 1 -- 172.21.15.47:0/3718030926 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e10063fc0 con 0x7f5e1005f4a0 2022-06-29T15:42:28.222 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.212+0000 7f5dfeffd700 1 -- 172.21.15.47:0/3718030926 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e0c010c40 con 0x7f5e1005f4a0 2022-06-29T15:42:28.222 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.212+0000 7f5e16a62700 1 -- 172.21.15.47:0/3718030926 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e100644a0 con 0x7f5e1005f4a0 2022-06-29T15:42:28.223 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.212+0000 7f5e16a62700 1 -- 172.21.15.47:0/3718030926 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e1014ee50 con 0x7f5e1005f4a0 2022-06-29T15:42:28.224 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.212+0000 7f5dfeffd700 1 -- 172.21.15.47:0/3718030926 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5e0c01c460 con 0x7f5e1005f4a0 2022-06-29T15:42:28.224 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.216+0000 7f5dfeffd700 1 --2- 172.21.15.47:0/3718030926 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5df0053b00 0x7f5df0055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:28.224 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.216+0000 7f5dfeffd700 1 -- 172.21.15.47:0/3718030926 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f5e0c06bfa0 con 0x7f5e1005f4a0 2022-06-29T15:42:28.224 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.216+0000 7f5e1525f700 1 --2- 172.21.15.47:0/3718030926 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5df0053b00 0x7f5df0055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:28.225 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.216+0000 7f5e1525f700 1 --2- 172.21.15.47:0/3718030926 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5df0053b00 0x7f5df0055fc0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f5e0001d000 tx=0x7f5e00017000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:28.226 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.224+0000 7f5dfeffd700 1 -- 172.21.15.47:0/3718030926 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5e0c043190 con 0x7f5e1005f4a0 2022-06-29T15:42:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:28.348+0000 7f5e16a62700 1 -- 172.21.15.47:0/3718030926 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f5e100573d0 con 0x7f5e1005f4a0 2022-06-29T15:42:29.115 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.112+0000 7f5dfeffd700 1 -- 172.21.15.47:0/3718030926 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v20) v1 ==== 133+0+0 (secure 0 0 0) 0x7f5e0c0467b0 con 0x7f5e1005f4a0 2022-06-29T15:42:29.118 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.116+0000 7f5e16a62700 1 -- 172.21.15.47:0/3718030926 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5df0053b00 msgr2=0x7f5df0055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:29.118 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.116+0000 7f5e16a62700 1 --2- 172.21.15.47:0/3718030926 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5df0053b00 0x7f5df0055fc0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f5e0001d000 tx=0x7f5e00017000).stop 2022-06-29T15:42:29.119 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.116+0000 7f5e16a62700 1 -- 172.21.15.47:0/3718030926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e1005f4a0 msgr2=0x7f5e100af790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:29.119 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.116+0000 7f5e16a62700 1 --2- 172.21.15.47:0/3718030926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e1005f4a0 0x7f5e100af790 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f5e0c00c000 tx=0x7f5e0c006df0).stop 2022-06-29T15:42:29.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.116+0000 7f5e16a62700 1 -- 172.21.15.47:0/3718030926 shutdown_connections 2022-06-29T15:42:29.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.116+0000 7f5e16a62700 1 --2- 172.21.15.47:0/3718030926 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5e10142a30 0x7f5e10063ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:29.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.116+0000 7f5e16a62700 1 --2- 172.21.15.47:0/3718030926 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5df0053b00 0x7f5df0055fc0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:29.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.116+0000 7f5e16a62700 1 --2- 172.21.15.47:0/3718030926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e1009a1d0 0x7f5e10065dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:29.121 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.116+0000 7f5e16a62700 1 --2- 172.21.15.47:0/3718030926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e1005f4a0 0x7f5e100af790 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:29.121 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.116+0000 7f5e16a62700 1 -- 172.21.15.47:0/3718030926 >> 172.21.15.47:0/3718030926 conn(0x7f5e10062df0 msgr2=0x7f5e10060050 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:29.121 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.116+0000 7f5e16a62700 1 -- 172.21.15.47:0/3718030926 shutdown_connections 2022-06-29T15:42:29.121 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.116+0000 7f5e16a62700 1 -- 172.21.15.47:0/3718030926 wait complete. 2022-06-29T15:42:29.122 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' created 2022-06-29T15:42:29.135 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2022-06-29T15:42:29.135 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data m=2 k=2 crush-failure-domain=osd 2022-06-29T15:42:29.245 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.240+0000 7f1d1ab7f700 1 Processor -- start 2022-06-29T15:42:29.246 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.240+0000 7f1d1ab7f700 1 -- start start 2022-06-29T15:42:29.252 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.244+0000 7f1d1ab7f700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1d140b2bd0 0x7f1d140b2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:29.252 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.244+0000 7f1d1ab7f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1d1409a060 0x7f1d140600c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:29.253 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.244+0000 7f1d1ab7f700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d14140ae0 con 0x7f1d140b1e80 2022-06-29T15:42:29.253 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.244+0000 7f1d1ab7f700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d14146900 con 0x7f1d1409a060 2022-06-29T15:42:29.254 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.244+0000 7f1d1ab7f700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d14146ba0 con 0x7f1d140b2bd0 2022-06-29T15:42:29.254 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.244+0000 7f1d1a37e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1d1409a060 0x7f1d140600c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:29.255 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.244+0000 7f1d1a37e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1d1409a060 0x7f1d140600c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45588/0 (socket says 172.21.15.47:45588) 2022-06-29T15:42:29.255 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.244+0000 7f1d1a37e700 1 -- 172.21.15.47:0/189404480 learned_addr learned my addr 172.21.15.47:0/189404480 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:29.256 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.244+0000 7f1d19b7d700 1 --2- 172.21.15.47:0/189404480 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1d140b2bd0 0x7f1d140b2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:29.256 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.244+0000 7f1d18b7b700 1 -- 172.21.15.47:0/189404480 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1316491412 0 0) 0x7f1d14140ae0 con 0x7f1d140b1e80 2022-06-29T15:42:29.257 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.244+0000 7f1d18b7b700 1 -- 172.21.15.47:0/189404480 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cfc003260 con 0x7f1d140b1e80 2022-06-29T15:42:29.257 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.244+0000 7f1d1a37e700 1 -- 172.21.15.47:0/189404480 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1d140b2bd0 msgr2=0x7f1d140b2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:29.257 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.244+0000 7f1d1a37e700 1 --2- 172.21.15.47:0/189404480 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1d140b2bd0 0x7f1d140b2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:29.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.244+0000 7f1d1a37e700 1 -- 172.21.15.47:0/189404480 >> v1:172.21.15.64:6789/0 conn(0x7f1d140b1e80 legacy=0x7f1d14099950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:29.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.244+0000 7f1d1a37e700 1 -- 172.21.15.47:0/189404480 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d14146e40 con 0x7f1d1409a060 2022-06-29T15:42:29.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.244+0000 7f1d19b7d700 1 --2- 172.21.15.47:0/189404480 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1d140b2bd0 0x7f1d140b2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:42:29.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.244+0000 7f1d1a37e700 1 --2- 172.21.15.47:0/189404480 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d1409a060 0x7f1d140600c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f1d0c0032f0 tx=0x7f1d0c009290).ready entity=mon.0 client_cookie=fae9feb0c4fc05ad server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:29.260 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.244+0000 7f1d18b7b700 1 -- 172.21.15.47:0/189404480 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d0c00b6c0 con 0x7f1d1409a060 2022-06-29T15:42:29.260 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.244+0000 7f1d18b7b700 1 -- 172.21.15.47:0/189404480 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d0c00b820 con 0x7f1d1409a060 2022-06-29T15:42:29.260 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.244+0000 7f1d18b7b700 1 -- 172.21.15.47:0/189404480 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d0c00b990 con 0x7f1d1409a060 2022-06-29T15:42:29.261 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.244+0000 7f1d1ab7f700 1 -- 172.21.15.47:0/189404480 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d1409a060 msgr2=0x7f1d140600c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:29.261 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.244+0000 7f1d1ab7f700 1 --2- 172.21.15.47:0/189404480 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d1409a060 0x7f1d140600c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f1d0c0032f0 tx=0x7f1d0c009290).stop 2022-06-29T15:42:29.262 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.248+0000 7f1d1ab7f700 1 -- 172.21.15.47:0/189404480 shutdown_connections 2022-06-29T15:42:29.262 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.248+0000 7f1d1ab7f700 1 --2- 172.21.15.47:0/189404480 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d1409a060 0x7f1d140600c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:29.263 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.248+0000 7f1d1ab7f700 1 --2- 172.21.15.47:0/189404480 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1d140b2bd0 0x7f1d140b2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:29.263 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.248+0000 7f1d1ab7f700 1 -- 172.21.15.47:0/189404480 >> 172.21.15.47:0/189404480 conn(0x7f1d14062df0 msgr2=0x7f1d140631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:29.264 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.248+0000 7f1d1ab7f700 1 -- 172.21.15.47:0/189404480 shutdown_connections 2022-06-29T15:42:29.264 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.248+0000 7f1d1ab7f700 1 -- 172.21.15.47:0/189404480 wait complete. 2022-06-29T15:42:29.264 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.248+0000 7f1d1ab7f700 1 Processor -- start 2022-06-29T15:42:29.265 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.248+0000 7f1d1ab7f700 1 -- start start 2022-06-29T15:42:29.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.252+0000 7f1d1ab7f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d1409a060 0x7f1d1413e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:29.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.252+0000 7f1d1ab7f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d140b1e80 0x7f1d140b4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:29.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.252+0000 7f1d1ab7f700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1d140b2bd0 0x7f1d14155f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:29.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.252+0000 7f1d1ab7f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d140b4fe0 con 0x7f1d1409a060 2022-06-29T15:42:29.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.252+0000 7f1d1ab7f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d140b51c0 con 0x7f1d140b1e80 2022-06-29T15:42:29.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.252+0000 7f1d1ab7f700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d140b5460 con 0x7f1d140b2bd0 2022-06-29T15:42:29.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.252+0000 7f1d19b7d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d1409a060 0x7f1d1413e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:29.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.252+0000 7f1d19b7d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d1409a060 0x7f1d1413e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45592/0 (socket says 172.21.15.47:45592) 2022-06-29T15:42:29.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.252+0000 7f1d19b7d700 1 -- 172.21.15.47:0/1196380786 learned_addr learned my addr 172.21.15.47:0/1196380786 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:29.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.252+0000 7f1d1937c700 1 --2- 172.21.15.47:0/1196380786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d140b1e80 0x7f1d140b4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:29.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.252+0000 7f1d1a37e700 1 --2- 172.21.15.47:0/1196380786 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1d140b2bd0 0x7f1d14155f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:29.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.252+0000 7f1d19b7d700 1 -- 172.21.15.47:0/1196380786 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1d140b2bd0 msgr2=0x7f1d14155f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:29.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.252+0000 7f1d19b7d700 1 --2- 172.21.15.47:0/1196380786 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1d140b2bd0 0x7f1d14155f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:29.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.252+0000 7f1d19b7d700 1 -- 172.21.15.47:0/1196380786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d140b1e80 msgr2=0x7f1d140b4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:29.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.252+0000 7f1d19b7d700 1 --2- 172.21.15.47:0/1196380786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d140b1e80 0x7f1d140b4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:29.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.252+0000 7f1d19b7d700 1 -- 172.21.15.47:0/1196380786 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d0400b180 con 0x7f1d1409a060 2022-06-29T15:42:29.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.252+0000 7f1d19b7d700 1 --2- 172.21.15.47:0/1196380786 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d1409a060 0x7f1d1413e3a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f1d0400c000 tx=0x7f1d04006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:29.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.252+0000 7f1d02ffd700 1 -- 172.21.15.47:0/1196380786 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d04010590 con 0x7f1d1409a060 2022-06-29T15:42:29.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.252+0000 7f1d02ffd700 1 -- 172.21.15.47:0/1196380786 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d040106f0 con 0x7f1d1409a060 2022-06-29T15:42:29.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.252+0000 7f1d1ab7f700 1 -- 172.21.15.47:0/1196380786 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d0c009530 con 0x7f1d1409a060 2022-06-29T15:42:29.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.252+0000 7f1d02ffd700 1 -- 172.21.15.47:0/1196380786 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d04010a50 con 0x7f1d1409a060 2022-06-29T15:42:29.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.252+0000 7f1d1ab7f700 1 -- 172.21.15.47:0/1196380786 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d141522f0 con 0x7f1d1409a060 2022-06-29T15:42:29.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.252+0000 7f1d1ab7f700 1 -- 172.21.15.47:0/1196380786 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d141524f0 con 0x7f1d1409a060 2022-06-29T15:42:29.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.256+0000 7f1d02ffd700 1 -- 172.21.15.47:0/1196380786 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1d04010bb0 con 0x7f1d1409a060 2022-06-29T15:42:29.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.256+0000 7f1d02ffd700 1 --2- 172.21.15.47:0/1196380786 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1cf4053b00 0x7f1cf4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:29.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.256+0000 7f1d02ffd700 1 -- 172.21.15.47:0/1196380786 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f1d0406c2e0 con 0x7f1d1409a060 2022-06-29T15:42:29.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.256+0000 7f1d1937c700 1 --2- 172.21.15.47:0/1196380786 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1cf4053b00 0x7f1cf4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:29.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.256+0000 7f1d1937c700 1 --2- 172.21.15.47:0/1196380786 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1cf4053b00 0x7f1cf4055fc0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f1d1000d000 tx=0x7f1d10007a20).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:29.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.264+0000 7f1d02ffd700 1 -- 172.21.15.47:0/1196380786 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1d141524f0 con 0x7f1d1409a060 2022-06-29T15:42:29.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:29.396+0000 7f1d1ab7f700 1 -- 172.21.15.47:0/1196380786 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f1d141524f0 con 0x7f1d1409a060 2022-06-29T15:42:30.122 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.116+0000 7f1d02ffd700 1 -- 172.21.15.47:0/1196380786 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v21) v1 ==== 154+0+0 (secure 0 0 0) 0x7f1d141524f0 con 0x7f1d1409a060 2022-06-29T15:42:30.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.124+0000 7f1d1ab7f700 1 -- 172.21.15.47:0/1196380786 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1cf4053b00 msgr2=0x7f1cf4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:30.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.124+0000 7f1d1ab7f700 1 --2- 172.21.15.47:0/1196380786 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1cf4053b00 0x7f1cf4055fc0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f1d1000d000 tx=0x7f1d10007a20).stop 2022-06-29T15:42:30.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.124+0000 7f1d1ab7f700 1 -- 172.21.15.47:0/1196380786 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d1409a060 msgr2=0x7f1d1413e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:30.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.124+0000 7f1d1ab7f700 1 --2- 172.21.15.47:0/1196380786 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d1409a060 0x7f1d1413e3a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f1d0400c000 tx=0x7f1d04006df0).stop 2022-06-29T15:42:30.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.124+0000 7f1d1ab7f700 1 -- 172.21.15.47:0/1196380786 shutdown_connections 2022-06-29T15:42:30.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.124+0000 7f1d1ab7f700 1 --2- 172.21.15.47:0/1196380786 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1d140b2bd0 0x7f1d14155f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:30.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.124+0000 7f1d1ab7f700 1 --2- 172.21.15.47:0/1196380786 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1cf4053b00 0x7f1cf4055fc0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:30.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.124+0000 7f1d1ab7f700 1 --2- 172.21.15.47:0/1196380786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d140b1e80 0x7f1d140b4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:30.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.124+0000 7f1d1ab7f700 1 --2- 172.21.15.47:0/1196380786 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d1409a060 0x7f1d1413e3a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:30.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.124+0000 7f1d1ab7f700 1 -- 172.21.15.47:0/1196380786 >> 172.21.15.47:0/1196380786 conn(0x7f1d14062df0 msgr2=0x7f1d141449d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:30.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.124+0000 7f1d1ab7f700 1 -- 172.21.15.47:0/1196380786 shutdown_connections 2022-06-29T15:42:30.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.124+0000 7f1d1ab7f700 1 -- 172.21.15.47:0/1196380786 wait complete. 2022-06-29T15:42:30.141 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 erasure cephfs_data 2022-06-29T15:42:30.257 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.252+0000 7fcb21029700 1 Processor -- start 2022-06-29T15:42:30.257 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.256+0000 7fcb21029700 1 -- start start 2022-06-29T15:42:30.261 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.256+0000 7fcb21029700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcb1c09a060 0x7fcb1c0b2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:30.261 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.256+0000 7fcb21029700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcb1c0b1e80 0x7fcb1c0600c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:30.261 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.256+0000 7fcb21029700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb1c140ae0 con 0x7fcb1c0b2bd0 2022-06-29T15:42:30.262 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.256+0000 7fcb21029700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb1c146900 con 0x7fcb1c0b1e80 2022-06-29T15:42:30.262 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.256+0000 7fcb21029700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1c146ba0 con 0x7fcb1c09a060 2022-06-29T15:42:30.262 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.256+0000 7fcb20828700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcb1c0b1e80 0x7fcb1c0600c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:30.262 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.256+0000 7fcb13fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcb1c09a060 0x7fcb1c0b2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:30.263 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.256+0000 7fcb20828700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcb1c0b1e80 0x7fcb1c0600c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:51306/0 (socket says 172.21.15.47:51306) 2022-06-29T15:42:30.263 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.256+0000 7fcb13fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcb1c09a060 0x7fcb1c0b2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:42704/0 (socket says 172.21.15.47:42704) 2022-06-29T15:42:30.263 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.256+0000 7fcb20828700 1 -- 172.21.15.47:0/2996033684 learned_addr learned my addr 172.21.15.47:0/2996033684 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:30.263 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.256+0000 7fcb1bfff700 1 --1- 172.21.15.47:0/2996033684 >> v1:172.21.15.64:6789/0 conn(0x7fcb1c0b2bd0 0x7fcb1c099950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:38718/0 (socket says 172.21.15.47:38718) 2022-06-29T15:42:30.264 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.256+0000 7fcb13fff700 1 -- 172.21.15.47:0/2996033684 >> v2:172.21.15.64:3300/0 conn(0x7fcb1c0b1e80 msgr2=0x7fcb1c0600c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:30.264 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.256+0000 7fcb13fff700 1 --2- 172.21.15.47:0/2996033684 >> v2:172.21.15.64:3300/0 conn(0x7fcb1c0b1e80 0x7fcb1c0600c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:30.264 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.256+0000 7fcb13fff700 1 -- 172.21.15.47:0/2996033684 >> v1:172.21.15.64:6789/0 conn(0x7fcb1c0b2bd0 legacy=0x7fcb1c099950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:30.264 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.256+0000 7fcb13fff700 1 -- 172.21.15.47:0/2996033684 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb1c146e40 con 0x7fcb1c09a060 2022-06-29T15:42:30.264 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.256+0000 7fcb20828700 1 --2- 172.21.15.47:0/2996033684 >> v2:172.21.15.64:3300/0 conn(0x7fcb1c0b1e80 0x7fcb1c0600c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:42:30.265 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.256+0000 7fcb13fff700 1 --2- 172.21.15.47:0/2996033684 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcb1c09a060 0x7fcb1c0b2fb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fcb04002f70 tx=0x7fcb04006df0).ready entity=mon.2 client_cookie=ef93e6b6efd41dc8 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:30.265 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.256+0000 7fcb1b7fe700 1 -- 172.21.15.47:0/2996033684 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb0400d570 con 0x7fcb1c09a060 2022-06-29T15:42:30.265 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.256+0000 7fcb1b7fe700 1 -- 172.21.15.47:0/2996033684 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb04000e90 con 0x7fcb1c09a060 2022-06-29T15:42:30.265 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.256+0000 7fcb1b7fe700 1 -- 172.21.15.47:0/2996033684 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb0400d8c0 con 0x7fcb1c09a060 2022-06-29T15:42:30.266 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.256+0000 7fcb21029700 1 -- 172.21.15.47:0/2996033684 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcb1c09a060 msgr2=0x7fcb1c0b2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:30.266 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.256+0000 7fcb21029700 1 --2- 172.21.15.47:0/2996033684 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcb1c09a060 0x7fcb1c0b2fb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fcb04002f70 tx=0x7fcb04006df0).stop 2022-06-29T15:42:30.266 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.260+0000 7fcb21029700 1 -- 172.21.15.47:0/2996033684 shutdown_connections 2022-06-29T15:42:30.266 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.260+0000 7fcb21029700 1 --2- 172.21.15.47:0/2996033684 >> v2:172.21.15.64:3300/0 conn(0x7fcb1c0b1e80 0x7fcb1c0600c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:30.267 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.260+0000 7fcb21029700 1 --2- 172.21.15.47:0/2996033684 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcb1c09a060 0x7fcb1c0b2fb0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:30.267 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.260+0000 7fcb21029700 1 -- 172.21.15.47:0/2996033684 >> 172.21.15.47:0/2996033684 conn(0x7fcb1c062df0 msgr2=0x7fcb1c0631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:30.267 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.260+0000 7fcb21029700 1 -- 172.21.15.47:0/2996033684 shutdown_connections 2022-06-29T15:42:30.267 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.260+0000 7fcb21029700 1 -- 172.21.15.47:0/2996033684 wait complete. 2022-06-29T15:42:30.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.260+0000 7fcb21029700 1 Processor -- start 2022-06-29T15:42:30.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.260+0000 7fcb21029700 1 -- start start 2022-06-29T15:42:30.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.260+0000 7fcb21029700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb1c09a060 0x7fcb1c149e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:30.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.260+0000 7fcb21029700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb1c0b1e80 0x7fcb1c140ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:30.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.260+0000 7fcb21029700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcb1c0b2bd0 0x7fcb1c141190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:30.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.260+0000 7fcb21029700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1c0aee40 con 0x7fcb1c09a060 2022-06-29T15:42:30.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.260+0000 7fcb21029700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1c143d80 con 0x7fcb1c0b1e80 2022-06-29T15:42:30.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.260+0000 7fcb21029700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1c0af070 con 0x7fcb1c0b2bd0 2022-06-29T15:42:30.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.260+0000 7fcb1bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb1c09a060 0x7fcb1c149e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:30.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.260+0000 7fcb1bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb1c09a060 0x7fcb1c149e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45604/0 (socket says 172.21.15.47:45604) 2022-06-29T15:42:30.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.260+0000 7fcb1bfff700 1 -- 172.21.15.47:0/35019482 learned_addr learned my addr 172.21.15.47:0/35019482 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:30.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.260+0000 7fcb20828700 1 --2- 172.21.15.47:0/35019482 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcb1c0b2bd0 0x7fcb1c141190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:30.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.260+0000 7fcb13fff700 1 --2- 172.21.15.47:0/35019482 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb1c0b1e80 0x7fcb1c140ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:30.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.264+0000 7fcb1bfff700 1 -- 172.21.15.47:0/35019482 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcb1c0b2bd0 msgr2=0x7fcb1c141190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:30.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.264+0000 7fcb1bfff700 1 --2- 172.21.15.47:0/35019482 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcb1c0b2bd0 0x7fcb1c141190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:30.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.264+0000 7fcb1bfff700 1 -- 172.21.15.47:0/35019482 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb1c0b1e80 msgr2=0x7fcb1c140ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:30.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.264+0000 7fcb1bfff700 1 --2- 172.21.15.47:0/35019482 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb1c0b1e80 0x7fcb1c140ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:30.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.264+0000 7fcb1bfff700 1 -- 172.21.15.47:0/35019482 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb14009530 con 0x7fcb1c09a060 2022-06-29T15:42:30.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.264+0000 7fcb1bfff700 1 --2- 172.21.15.47:0/35019482 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb1c09a060 0x7fcb1c149e10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7fcb1c0b5b00 tx=0x7fcb0c007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:30.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.264+0000 7fcb19ffb700 1 -- 172.21.15.47:0/35019482 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb0c007170 con 0x7fcb1c09a060 2022-06-29T15:42:30.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.264+0000 7fcb19ffb700 1 -- 172.21.15.47:0/35019482 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb0c0072d0 con 0x7fcb1c09a060 2022-06-29T15:42:30.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.264+0000 7fcb21029700 1 -- 172.21.15.47:0/35019482 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb0400b180 con 0x7fcb1c09a060 2022-06-29T15:42:30.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.264+0000 7fcb19ffb700 1 -- 172.21.15.47:0/35019482 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb0c004710 con 0x7fcb1c09a060 2022-06-29T15:42:30.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.264+0000 7fcb21029700 1 -- 172.21.15.47:0/35019482 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb1c1438d0 con 0x7fcb1c09a060 2022-06-29T15:42:30.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.264+0000 7fcb21029700 1 -- 172.21.15.47:0/35019482 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb1c0573d0 con 0x7fcb1c09a060 2022-06-29T15:42:30.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.264+0000 7fcb19ffb700 1 -- 172.21.15.47:0/35019482 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fcb0c004870 con 0x7fcb1c09a060 2022-06-29T15:42:30.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.268+0000 7fcb19ffb700 1 --2- 172.21.15.47:0/35019482 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcb00053a70 0x7fcb00055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:30.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.268+0000 7fcb13fff700 1 --2- 172.21.15.47:0/35019482 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcb00053a70 0x7fcb00055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:30.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.268+0000 7fcb19ffb700 1 -- 172.21.15.47:0/35019482 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5863+0+0 (secure 0 0 0) 0x7fcb0c06b710 con 0x7fcb1c09a060 2022-06-29T15:42:30.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.268+0000 7fcb13fff700 1 --2- 172.21.15.47:0/35019482 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcb00053a70 0x7fcb00055f30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fcb0401a000 tx=0x7fcb04014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:30.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.272+0000 7fcb19ffb700 1 -- 172.21.15.47:0/35019482 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcb0c042510 con 0x7fcb1c09a060 2022-06-29T15:42:30.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:30.400+0000 7fcb21029700 1 -- 172.21.15.47:0/35019482 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8, "pool_type": "erasure", "erasure_code_profile": "cephfs_data"} v 0) v1 -- 0x7fcb1c067b30 con 0x7fcb1c09a060 2022-06-29T15:42:32.140 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.136+0000 7fcb19ffb700 1 -- 172.21.15.47:0/35019482 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8, "pool_type": "erasure", "erasure_code_profile": "cephfs_data"}]=0 pool 'cephfs_data' created v23) v1 ==== 188+0+0 (secure 0 0 0) 0x7fcb0c045b30 con 0x7fcb1c09a060 2022-06-29T15:42:32.144 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.140+0000 7fcb21029700 1 -- 172.21.15.47:0/35019482 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcb00053a70 msgr2=0x7fcb00055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:32.144 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.140+0000 7fcb21029700 1 --2- 172.21.15.47:0/35019482 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcb00053a70 0x7fcb00055f30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fcb0401a000 tx=0x7fcb04014000).stop 2022-06-29T15:42:32.145 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.140+0000 7fcb21029700 1 -- 172.21.15.47:0/35019482 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb1c09a060 msgr2=0x7fcb1c149e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:32.145 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.140+0000 7fcb21029700 1 --2- 172.21.15.47:0/35019482 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb1c09a060 0x7fcb1c149e10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7fcb1c0b5b00 tx=0x7fcb0c007c10).stop 2022-06-29T15:42:32.145 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.140+0000 7fcb21029700 1 -- 172.21.15.47:0/35019482 shutdown_connections 2022-06-29T15:42:32.145 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.140+0000 7fcb21029700 1 --2- 172.21.15.47:0/35019482 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcb1c0b2bd0 0x7fcb1c141190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:32.146 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.140+0000 7fcb21029700 1 --2- 172.21.15.47:0/35019482 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcb00053a70 0x7fcb00055f30 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:32.146 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.140+0000 7fcb21029700 1 --2- 172.21.15.47:0/35019482 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb1c0b1e80 0x7fcb1c140ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:32.147 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.140+0000 7fcb21029700 1 --2- 172.21.15.47:0/35019482 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb1c09a060 0x7fcb1c149e10 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:32.147 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.140+0000 7fcb21029700 1 -- 172.21.15.47:0/35019482 >> 172.21.15.47:0/35019482 conn(0x7fcb1c062df0 msgr2=0x7fcb1c05f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:32.147 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.140+0000 7fcb21029700 1 -- 172.21.15.47:0/35019482 shutdown_connections 2022-06-29T15:42:32.147 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.140+0000 7fcb21029700 1 -- 172.21.15.47:0/35019482 wait complete. 2022-06-29T15:42:32.148 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' created 2022-06-29T15:42:32.162 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data allow_ec_overwrites true 2022-06-29T15:42:32.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.272+0000 7f89f80ce700 1 Processor -- start 2022-06-29T15:42:32.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.272+0000 7f89f80ce700 1 -- start start 2022-06-29T15:42:32.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.272+0000 7f89f80ce700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f89f0061670 0x7f89f0061a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:32.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.272+0000 7f89f80ce700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f89f00b0fe0 0x7f89f00b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:32.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.272+0000 7f89f80ce700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89f00af0a0 con 0x7f89f013d480 2022-06-29T15:42:32.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.272+0000 7f89f80ce700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89f013fed0 con 0x7f89f00b0fe0 2022-06-29T15:42:32.282 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.272+0000 7f89f80ce700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89f00af340 con 0x7f89f0061670 2022-06-29T15:42:32.282 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.272+0000 7f89f5669700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f89f00b0fe0 0x7f89f00b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:32.282 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.272+0000 7f89f666b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f89f013d480 0x7f89f01436c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59402/0 (socket says 172.21.15.47:59402) 2022-06-29T15:42:32.282 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.272+0000 7f89f666b700 1 -- 172.21.15.47:0/753557243 learned_addr learned my addr 172.21.15.47:0/753557243 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:32.283 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.272+0000 7f89f5e6a700 1 --2- 172.21.15.47:0/753557243 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f89f0061670 0x7f89f0061a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:32.283 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.272+0000 7f89f5669700 1 -- 172.21.15.47:0/753557243 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f89f0061670 msgr2=0x7f89f0061a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:32.283 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.272+0000 7f89f5669700 1 --2- 172.21.15.47:0/753557243 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f89f0061670 0x7f89f0061a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:32.283 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.272+0000 7f89f5669700 1 -- 172.21.15.47:0/753557243 >> v1:172.21.15.47:6789/0 conn(0x7f89f013d480 legacy=0x7f89f01436c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:32.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.272+0000 7f89f5669700 1 -- 172.21.15.47:0/753557243 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f00af5e0 con 0x7f89f00b0fe0 2022-06-29T15:42:32.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.276+0000 7f89f5669700 1 --2- 172.21.15.47:0/753557243 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89f00b0fe0 0x7f89f00b0700 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f89e0002f70 tx=0x7f89e0006df0).ready entity=mon.0 client_cookie=f915bedaa0f70df1 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:32.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.276+0000 7f89f4e68700 1 -- 172.21.15.47:0/753557243 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89e0014070 con 0x7f89f00b0fe0 2022-06-29T15:42:32.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.276+0000 7f89f4e68700 1 -- 172.21.15.47:0/753557243 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e0000e90 con 0x7f89f00b0fe0 2022-06-29T15:42:32.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.276+0000 7f89f4e68700 1 -- 172.21.15.47:0/753557243 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89e000fb60 con 0x7f89f00b0fe0 2022-06-29T15:42:32.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.276+0000 7f89f80ce700 1 -- 172.21.15.47:0/753557243 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89f00b0fe0 msgr2=0x7f89f00b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:32.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.276+0000 7f89f80ce700 1 --2- 172.21.15.47:0/753557243 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89f00b0fe0 0x7f89f00b0700 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f89e0002f70 tx=0x7f89e0006df0).stop 2022-06-29T15:42:32.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.276+0000 7f89f80ce700 1 -- 172.21.15.47:0/753557243 shutdown_connections 2022-06-29T15:42:32.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.276+0000 7f89f80ce700 1 --2- 172.21.15.47:0/753557243 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89f00b0fe0 0x7f89f00b0700 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:32.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.276+0000 7f89f80ce700 1 --2- 172.21.15.47:0/753557243 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f89f0061670 0x7f89f0061a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:32.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.276+0000 7f89f80ce700 1 -- 172.21.15.47:0/753557243 >> 172.21.15.47:0/753557243 conn(0x7f89f0063800 msgr2=0x7f89f0063be0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:32.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.276+0000 7f89f80ce700 1 -- 172.21.15.47:0/753557243 shutdown_connections 2022-06-29T15:42:32.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.276+0000 7f89f80ce700 1 -- 172.21.15.47:0/753557243 wait complete. 2022-06-29T15:42:32.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.280+0000 7f89f80ce700 1 Processor -- start 2022-06-29T15:42:32.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.280+0000 7f89f80ce700 1 -- start start 2022-06-29T15:42:32.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.280+0000 7f89f80ce700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89f0061670 0x7f89f013efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:32.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.280+0000 7f89f80ce700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f89f00b0fe0 0x7f89f014ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:32.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.280+0000 7f89f80ce700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89f013d480 0x7f89f0151640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:32.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.280+0000 7f89f80ce700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f014e0f0 con 0x7f89f0061670 2022-06-29T15:42:32.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.280+0000 7f89f80ce700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f00b5480 con 0x7f89f013d480 2022-06-29T15:42:32.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.280+0000 7f89f80ce700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89f014e2e0 con 0x7f89f00b0fe0 2022-06-29T15:42:32.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.280+0000 7f89f5e6a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89f0061670 0x7f89f013efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:32.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.280+0000 7f89f5e6a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89f0061670 0x7f89f013efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45618/0 (socket says 172.21.15.47:45618) 2022-06-29T15:42:32.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.280+0000 7f89f5e6a700 1 -- 172.21.15.47:0/3937116478 learned_addr learned my addr 172.21.15.47:0/3937116478 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:32.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.280+0000 7f89f666b700 1 --2- 172.21.15.47:0/3937116478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89f013d480 0x7f89f0151640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:32.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.280+0000 7f89f5669700 1 --2- 172.21.15.47:0/3937116478 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f89f00b0fe0 0x7f89f014ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:32.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.280+0000 7f89f5e6a700 1 -- 172.21.15.47:0/3937116478 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f89f00b0fe0 msgr2=0x7f89f014ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:32.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.280+0000 7f89f5e6a700 1 --2- 172.21.15.47:0/3937116478 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f89f00b0fe0 0x7f89f014ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:32.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.280+0000 7f89f5e6a700 1 -- 172.21.15.47:0/3937116478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89f013d480 msgr2=0x7f89f0151640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:32.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.280+0000 7f89f5e6a700 1 --2- 172.21.15.47:0/3937116478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89f013d480 0x7f89f0151640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:32.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.280+0000 7f89f5e6a700 1 -- 172.21.15.47:0/3937116478 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e000b180 con 0x7f89f0061670 2022-06-29T15:42:32.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.280+0000 7f89f5e6a700 1 --2- 172.21.15.47:0/3937116478 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89f0061670 0x7f89f013efe0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f89ec00c000 tx=0x7f89ec006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:32.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.280+0000 7f89deffd700 1 -- 172.21.15.47:0/3937116478 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89ec010780 con 0x7f89f0061670 2022-06-29T15:42:32.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.280+0000 7f89deffd700 1 -- 172.21.15.47:0/3937116478 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89ec0108e0 con 0x7f89f0061670 2022-06-29T15:42:32.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.280+0000 7f89f80ce700 1 -- 172.21.15.47:0/3937116478 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89f014e580 con 0x7f89f0061670 2022-06-29T15:42:32.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.280+0000 7f89deffd700 1 -- 172.21.15.47:0/3937116478 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89ec010c40 con 0x7f89f0061670 2022-06-29T15:42:32.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.280+0000 7f89f80ce700 1 -- 172.21.15.47:0/3937116478 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89f014ef10 con 0x7f89f0061670 2022-06-29T15:42:32.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.284+0000 7f89f80ce700 1 -- 172.21.15.47:0/3937116478 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89f00573d0 con 0x7f89f0061670 2022-06-29T15:42:32.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.284+0000 7f89deffd700 1 -- 172.21.15.47:0/3937116478 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f89ec01c460 con 0x7f89f0061670 2022-06-29T15:42:32.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.284+0000 7f89deffd700 1 --2- 172.21.15.47:0/3937116478 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f89d0053a10 0x7f89d0055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:32.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.284+0000 7f89f5669700 1 --2- 172.21.15.47:0/3937116478 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f89d0053a10 0x7f89d0055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:32.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.284+0000 7f89deffd700 1 -- 172.21.15.47:0/3937116478 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6315+0+0 (secure 0 0 0) 0x7f89ec06c490 con 0x7f89f0061670 2022-06-29T15:42:32.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.284+0000 7f89f5669700 1 --2- 172.21.15.47:0/3937116478 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f89d0053a10 0x7f89d0055ed0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f89e001d000 tx=0x7f89e0016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:32.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.288+0000 7f89deffd700 1 -- 172.21.15.47:0/3937116478 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f89ec043150 con 0x7f89f0061670 2022-06-29T15:42:32.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:32.416+0000 7f89f80ce700 1 -- 172.21.15.47:0/3937116478 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f89f014e710 con 0x7f89f0061670 2022-06-29T15:42:33.150 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.148+0000 7f89deffd700 1 -- 172.21.15.47:0/3937116478 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v24) v1 ==== 166+0+0 (secure 0 0 0) 0x7f89ec046770 con 0x7f89f0061670 2022-06-29T15:42:33.158 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.156+0000 7f89f80ce700 1 -- 172.21.15.47:0/3937116478 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f89d0053a10 msgr2=0x7f89d0055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:33.159 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.156+0000 7f89f80ce700 1 --2- 172.21.15.47:0/3937116478 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f89d0053a10 0x7f89d0055ed0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f89e001d000 tx=0x7f89e0016000).stop 2022-06-29T15:42:33.159 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.156+0000 7f89f80ce700 1 -- 172.21.15.47:0/3937116478 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89f0061670 msgr2=0x7f89f013efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:33.159 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.156+0000 7f89f80ce700 1 --2- 172.21.15.47:0/3937116478 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89f0061670 0x7f89f013efe0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f89ec00c000 tx=0x7f89ec006df0).stop 2022-06-29T15:42:33.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.156+0000 7f89f80ce700 1 -- 172.21.15.47:0/3937116478 shutdown_connections 2022-06-29T15:42:33.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.156+0000 7f89f80ce700 1 --2- 172.21.15.47:0/3937116478 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f89d0053a10 0x7f89d0055ed0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:33.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.156+0000 7f89f80ce700 1 --2- 172.21.15.47:0/3937116478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89f013d480 0x7f89f0151640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:33.161 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.156+0000 7f89f80ce700 1 --2- 172.21.15.47:0/3937116478 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f89f00b0fe0 0x7f89f014ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:33.161 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.156+0000 7f89f80ce700 1 --2- 172.21.15.47:0/3937116478 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89f0061670 0x7f89f013efe0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:33.161 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.156+0000 7f89f80ce700 1 -- 172.21.15.47:0/3937116478 >> 172.21.15.47:0/3937116478 conn(0x7f89f0063800 msgr2=0x7f89f0143da0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:33.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.156+0000 7f89f80ce700 1 -- 172.21.15.47:0/3937116478 shutdown_connections 2022-06-29T15:42:33.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.156+0000 7f89f80ce700 1 -- 172.21.15.47:0/3937116478 wait complete. 2022-06-29T15:42:33.163 INFO:teuthology.orchestra.run.smithi047.stderr:set pool 4 allow_ec_overwrites to true 2022-06-29T15:42:33.173 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-06-29T15:42:33.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.284+0000 7f9b7b59e700 1 Processor -- start 2022-06-29T15:42:33.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.284+0000 7f9b7b59e700 1 -- start start 2022-06-29T15:42:33.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.288+0000 7f9b7b59e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9b7c098d10 0x7f9b7c0acff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:33.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.288+0000 7f9b7b59e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b7c099760 con 0x7f9b7c0b2d70 2022-06-29T15:42:33.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.288+0000 7f9b7b59e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b7c09b5a0 con 0x7f9b7c0615c0 2022-06-29T15:42:33.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.288+0000 7f9b7b59e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b7c099220 con 0x7f9b7c098d10 2022-06-29T15:42:33.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.288+0000 7f9b7a59c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9b7c0b2d70 0x7f9b7c09ae90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59412/0 (socket says 172.21.15.47:59412) 2022-06-29T15:42:33.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.288+0000 7f9b7a59c700 1 -- 172.21.15.47:0/4244589348 learned_addr learned my addr 172.21.15.47:0/4244589348 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:33.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.288+0000 7f9b73fff700 1 --2- 172.21.15.47:0/4244589348 >> v2:172.21.15.47:3300/0 conn(0x7f9b7c098d10 0x7f9b7c0acff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:33.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.288+0000 7f9b73fff700 1 -- 172.21.15.47:0/4244589348 >> v1:172.21.15.64:6789/0 conn(0x7f9b7c0615c0 legacy=0x7f9b7c0af5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:33.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.288+0000 7f9b73fff700 1 -- 172.21.15.47:0/4244589348 >> v1:172.21.15.47:6789/0 conn(0x7f9b7c0b2d70 legacy=0x7f9b7c09ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:33.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.288+0000 7f9b79d9b700 1 -- 172.21.15.47:0/4244589348 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2271012648 0 0) 0x7f9b7c099760 con 0x7f9b7c0b2d70 2022-06-29T15:42:33.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.288+0000 7f9b73fff700 1 -- 172.21.15.47:0/4244589348 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b7c0994c0 con 0x7f9b7c098d10 2022-06-29T15:42:33.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.288+0000 7f9b73fff700 1 --2- 172.21.15.47:0/4244589348 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b7c098d10 0x7f9b7c0acff0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f9b64002f70 tx=0x7f9b64006df0).ready entity=mon.0 client_cookie=d1ec019e730ffb22 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:33.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.288+0000 7f9b79d9b700 1 -- 172.21.15.47:0/4244589348 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b6400d610 con 0x7f9b7c098d10 2022-06-29T15:42:33.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.288+0000 7f9b79d9b700 1 -- 172.21.15.47:0/4244589348 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b64000e90 con 0x7f9b7c098d10 2022-06-29T15:42:33.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.288+0000 7f9b79d9b700 1 -- 172.21.15.47:0/4244589348 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b6400d960 con 0x7f9b7c098d10 2022-06-29T15:42:33.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.288+0000 7f9b7b59e700 1 -- 172.21.15.47:0/4244589348 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b7c098d10 msgr2=0x7f9b7c0acff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:33.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.288+0000 7f9b7b59e700 1 --2- 172.21.15.47:0/4244589348 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b7c098d10 0x7f9b7c0acff0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f9b64002f70 tx=0x7f9b64006df0).stop 2022-06-29T15:42:33.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.288+0000 7f9b7b59e700 1 -- 172.21.15.47:0/4244589348 shutdown_connections 2022-06-29T15:42:33.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.288+0000 7f9b7b59e700 1 --2- 172.21.15.47:0/4244589348 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b7c098d10 0x7f9b7c0acff0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:33.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.288+0000 7f9b7b59e700 1 -- 172.21.15.47:0/4244589348 >> 172.21.15.47:0/4244589348 conn(0x7f9b7c062df0 msgr2=0x7f9b7c0631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:33.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.288+0000 7f9b7b59e700 1 -- 172.21.15.47:0/4244589348 shutdown_connections 2022-06-29T15:42:33.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.288+0000 7f9b7b59e700 1 -- 172.21.15.47:0/4244589348 wait complete. 2022-06-29T15:42:33.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.292+0000 7f9b7b59e700 1 Processor -- start 2022-06-29T15:42:33.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.292+0000 7f9b7b59e700 1 -- start start 2022-06-29T15:42:33.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.292+0000 7f9b7b59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b7c0615c0 0x7f9b7c141270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:33.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.292+0000 7f9b7b59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b7c098d10 0x7f9b7c152980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:33.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.292+0000 7f9b7b59e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9b7c0b2d70 0x7f9b7c152e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:33.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.292+0000 7f9b7b59e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7c069790 con 0x7f9b7c098d10 2022-06-29T15:42:33.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.292+0000 7f9b7b59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7c0afe80 con 0x7f9b7c0615c0 2022-06-29T15:42:33.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.292+0000 7f9b7b59e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7c0aeb50 con 0x7f9b7c0b2d70 2022-06-29T15:42:33.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.292+0000 7f9b73fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b7c098d10 0x7f9b7c152980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:33.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.292+0000 7f9b73fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b7c098d10 0x7f9b7c152980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45634/0 (socket says 172.21.15.47:45634) 2022-06-29T15:42:33.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.292+0000 7f9b73fff700 1 -- 172.21.15.47:0/3847132915 learned_addr learned my addr 172.21.15.47:0/3847132915 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:33.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.292+0000 7f9b7a59c700 1 --2- 172.21.15.47:0/3847132915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b7c0615c0 0x7f9b7c141270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:33.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.292+0000 7f9b7ad9d700 1 --2- 172.21.15.47:0/3847132915 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9b7c0b2d70 0x7f9b7c152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:33.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.292+0000 7f9b73fff700 1 -- 172.21.15.47:0/3847132915 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9b7c0b2d70 msgr2=0x7f9b7c152e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:33.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.292+0000 7f9b73fff700 1 --2- 172.21.15.47:0/3847132915 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9b7c0b2d70 0x7f9b7c152e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:33.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.292+0000 7f9b73fff700 1 -- 172.21.15.47:0/3847132915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b7c0615c0 msgr2=0x7f9b7c141270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:33.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.292+0000 7f9b73fff700 1 --2- 172.21.15.47:0/3847132915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b7c0615c0 0x7f9b7c141270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:33.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.292+0000 7f9b73fff700 1 -- 172.21.15.47:0/3847132915 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b6400b180 con 0x7f9b7c098d10 2022-06-29T15:42:33.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.292+0000 7f9b7ad9d700 1 --2- 172.21.15.47:0/3847132915 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9b7c0b2d70 0x7f9b7c152e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:42:33.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.292+0000 7f9b73fff700 1 --2- 172.21.15.47:0/3847132915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b7c098d10 0x7f9b7c152980 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f9b64002f70 tx=0x7f9b64015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:33.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.296+0000 7f9b737fe700 1 -- 172.21.15.47:0/3847132915 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b6400de70 con 0x7f9b7c098d10 2022-06-29T15:42:33.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.296+0000 7f9b737fe700 1 -- 172.21.15.47:0/3847132915 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b64011bb0 con 0x7f9b7c098d10 2022-06-29T15:42:33.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.296+0000 7f9b7b59e700 1 -- 172.21.15.47:0/3847132915 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b7c0af5b0 con 0x7f9b7c098d10 2022-06-29T15:42:33.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.296+0000 7f9b737fe700 1 -- 172.21.15.47:0/3847132915 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b6401e460 con 0x7f9b7c098d10 2022-06-29T15:42:33.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.296+0000 7f9b7b59e700 1 -- 172.21.15.47:0/3847132915 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b7c0ae600 con 0x7f9b7c098d10 2022-06-29T15:42:33.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.296+0000 7f9b7b59e700 1 -- 172.21.15.47:0/3847132915 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b7c0573d0 con 0x7f9b7c098d10 2022-06-29T15:42:33.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.296+0000 7f9b737fe700 1 -- 172.21.15.47:0/3847132915 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9b64025030 con 0x7f9b7c098d10 2022-06-29T15:42:33.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.296+0000 7f9b737fe700 1 --2- 172.21.15.47:0/3847132915 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9b600539e0 0x7f9b60055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:33.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.296+0000 7f9b737fe700 1 -- 172.21.15.47:0/3847132915 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6315+0+0 (secure 0 0 0) 0x7f9b6406d340 con 0x7f9b7c098d10 2022-06-29T15:42:33.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.296+0000 7f9b7a59c700 1 --2- 172.21.15.47:0/3847132915 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9b600539e0 0x7f9b60055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:33.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.296+0000 7f9b7a59c700 1 --2- 172.21.15.47:0/3847132915 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9b600539e0 0x7f9b60055ea0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f9b6c00d000 tx=0x7f9b6c007c00).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:33.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.304+0000 7f9b737fe700 1 -- 172.21.15.47:0/3847132915 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9b640441b0 con 0x7f9b7c098d10 2022-06-29T15:42:33.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.468+0000 7f9b7b59e700 1 -- 172.21.15.47:0/3847132915 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f9b7c067b30 con 0x7f9b7c098d10 2022-06-29T15:42:33.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.484+0000 7f9b737fe700 1 -- 172.21.15.47:0/3847132915 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 3 and data pool 4 v5) v1 ==== 187+0+0 (secure 0 0 0) 0x7f9b640230d0 con 0x7f9b7c098d10 2022-06-29T15:42:33.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.492+0000 7f9b7b59e700 1 -- 172.21.15.47:0/3847132915 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9b600539e0 msgr2=0x7f9b60055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:33.497 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.492+0000 7f9b7b59e700 1 --2- 172.21.15.47:0/3847132915 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9b600539e0 0x7f9b60055ea0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f9b6c00d000 tx=0x7f9b6c007c00).stop 2022-06-29T15:42:33.497 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.492+0000 7f9b7b59e700 1 -- 172.21.15.47:0/3847132915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b7c098d10 msgr2=0x7f9b7c152980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:33.497 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.492+0000 7f9b7b59e700 1 --2- 172.21.15.47:0/3847132915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b7c098d10 0x7f9b7c152980 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f9b64002f70 tx=0x7f9b64015000).stop 2022-06-29T15:42:33.498 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.492+0000 7f9b7b59e700 1 -- 172.21.15.47:0/3847132915 shutdown_connections 2022-06-29T15:42:33.498 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.492+0000 7f9b7b59e700 1 --2- 172.21.15.47:0/3847132915 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9b7c0b2d70 0x7f9b7c152e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:33.498 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.492+0000 7f9b7b59e700 1 --2- 172.21.15.47:0/3847132915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b7c098d10 0x7f9b7c152980 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:33.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.492+0000 7f9b7b59e700 1 --2- 172.21.15.47:0/3847132915 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9b600539e0 0x7f9b60055ea0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:33.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.492+0000 7f9b7b59e700 1 --2- 172.21.15.47:0/3847132915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b7c0615c0 0x7f9b7c141270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:33.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.492+0000 7f9b7b59e700 1 -- 172.21.15.47:0/3847132915 >> 172.21.15.47:0/3847132915 conn(0x7f9b7c062df0 msgr2=0x7f9b7c1437e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:33.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.492+0000 7f9b7b59e700 1 -- 172.21.15.47:0/3847132915 shutdown_connections 2022-06-29T15:42:33.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.492+0000 7f9b7b59e700 1 -- 172.21.15.47:0/3847132915 wait complete. 2022-06-29T15:42:33.507 INFO:teuthology.orchestra.run.smithi047.stderr:new fs with metadata pool 3 and data pool 4 2022-06-29T15:42:33.509 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:42:33.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.600+0000 7fd8b4d70700 1 Processor -- start 2022-06-29T15:42:33.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.600+0000 7fd8b4d70700 1 -- start start 2022-06-29T15:42:33.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.600+0000 7fd8b4d70700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd8b00b0cf0 0x7fd8b0146d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:33.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.600+0000 7fd8b4d70700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd8b00b06d0 0x7fd8b013ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:33.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.600+0000 7fd8b4d70700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8b00b4520 con 0x7fd8b00611b0 2022-06-29T15:42:33.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.600+0000 7fd8b4d70700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8b00b4b10 con 0x7fd8b00b0cf0 2022-06-29T15:42:33.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.600+0000 7fd8b4d70700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b00621d0 con 0x7fd8b00b06d0 2022-06-29T15:42:33.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.600+0000 7fd8aed9d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd8b00b06d0 0x7fd8b013ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:33.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.600+0000 7fd8add9b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd8b00b0cf0 0x7fd8b0146d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:33.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.600+0000 7fd8aed9d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd8b00b06d0 0x7fd8b013ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:42756/0 (socket says 172.21.15.47:42756) 2022-06-29T15:42:33.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.600+0000 7fd8aed9d700 1 -- 172.21.15.47:0/1617566336 learned_addr learned my addr 172.21.15.47:0/1617566336 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:33.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.600+0000 7fd8add9b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd8b00b0cf0 0x7fd8b0146d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:51354/0 (socket says 172.21.15.47:51354) 2022-06-29T15:42:33.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.600+0000 7fd8ae59c700 1 --1- 172.21.15.47:0/1617566336 >> v1:172.21.15.64:6789/0 conn(0x7fd8b00611b0 0x7fd8b0146640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:38768/0 (socket says 172.21.15.47:38768) 2022-06-29T15:42:33.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.604+0000 7fd8aed9d700 1 -- 172.21.15.47:0/1617566336 >> v2:172.21.15.64:3300/0 conn(0x7fd8b00b0cf0 msgr2=0x7fd8b0146d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:33.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.604+0000 7fd8aed9d700 1 --2- 172.21.15.47:0/1617566336 >> v2:172.21.15.64:3300/0 conn(0x7fd8b00b0cf0 0x7fd8b0146d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:33.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.604+0000 7fd8aed9d700 1 -- 172.21.15.47:0/1617566336 >> v1:172.21.15.64:6789/0 conn(0x7fd8b00611b0 legacy=0x7fd8b0146640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:33.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.604+0000 7fd8aed9d700 1 -- 172.21.15.47:0/1617566336 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b0061590 con 0x7fd8b00b06d0 2022-06-29T15:42:33.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.604+0000 7fd8add9b700 1 --2- 172.21.15.47:0/1617566336 >> v2:172.21.15.64:3300/0 conn(0x7fd8b00b0cf0 0x7fd8b0146d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:42:33.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.604+0000 7fd8aed9d700 1 --2- 172.21.15.47:0/1617566336 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd8b00b06d0 0x7fd8b013ff00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7fd8a80032f0 tx=0x7fd8a8009290).ready entity=mon.2 client_cookie=10cbad61149913eb server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:33.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.604+0000 7fd8ad59a700 1 -- 172.21.15.47:0/1617566336 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd8a800b8a0 con 0x7fd8b00b06d0 2022-06-29T15:42:33.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.604+0000 7fd8ad59a700 1 -- 172.21.15.47:0/1617566336 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a800ba00 con 0x7fd8b00b06d0 2022-06-29T15:42:33.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.604+0000 7fd8ad59a700 1 -- 172.21.15.47:0/1617566336 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd8a800bb70 con 0x7fd8b00b06d0 2022-06-29T15:42:33.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.604+0000 7fd8b4d70700 1 -- 172.21.15.47:0/1617566336 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd8b00b06d0 msgr2=0x7fd8b013ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:33.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.604+0000 7fd8b4d70700 1 --2- 172.21.15.47:0/1617566336 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd8b00b06d0 0x7fd8b013ff00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7fd8a80032f0 tx=0x7fd8a8009290).stop 2022-06-29T15:42:33.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.604+0000 7fd8b4d70700 1 -- 172.21.15.47:0/1617566336 shutdown_connections 2022-06-29T15:42:33.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.604+0000 7fd8b4d70700 1 --2- 172.21.15.47:0/1617566336 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd8b00b06d0 0x7fd8b013ff00 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:33.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.604+0000 7fd8b4d70700 1 --2- 172.21.15.47:0/1617566336 >> v2:172.21.15.64:3300/0 conn(0x7fd8b00b0cf0 0x7fd8b0146d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:33.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.604+0000 7fd8b4d70700 1 -- 172.21.15.47:0/1617566336 >> 172.21.15.47:0/1617566336 conn(0x7fd8b00b98b0 msgr2=0x7fd8b00b9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:33.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.604+0000 7fd8b4d70700 1 -- 172.21.15.47:0/1617566336 shutdown_connections 2022-06-29T15:42:33.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.604+0000 7fd8b4d70700 1 -- 172.21.15.47:0/1617566336 wait complete. 2022-06-29T15:42:33.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.604+0000 7fd8b4d70700 1 Processor -- start 2022-06-29T15:42:33.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.604+0000 7fd8b4d70700 1 -- start start 2022-06-29T15:42:33.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.608+0000 7fd8b4d70700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8b00611b0 0x7fd8b013daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:33.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.608+0000 7fd8b4d70700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd8b00b06d0 0x7fd8b00ae920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:33.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.608+0000 7fd8b4d70700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd8b00b0cf0 0x7fd8b00aee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:33.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.608+0000 7fd8b4d70700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b00b8d80 con 0x7fd8b00611b0 2022-06-29T15:42:33.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.608+0000 7fd8b4d70700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b00b9060 con 0x7fd8b00b06d0 2022-06-29T15:42:33.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.608+0000 7fd8b4d70700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b00b9300 con 0x7fd8b00b0cf0 2022-06-29T15:42:33.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.608+0000 7fd8ae59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8b00611b0 0x7fd8b013daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:33.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.608+0000 7fd8aed9d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd8b00b0cf0 0x7fd8b00aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:33.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.608+0000 7fd8add9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd8b00b06d0 0x7fd8b00ae920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:33.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.608+0000 7fd8ae59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8b00611b0 0x7fd8b013daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45656/0 (socket says 172.21.15.47:45656) 2022-06-29T15:42:33.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.608+0000 7fd8ae59c700 1 -- 172.21.15.47:0/1995154381 learned_addr learned my addr 172.21.15.47:0/1995154381 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:33.622 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.608+0000 7fd8aed9d700 1 -- 172.21.15.47:0/1995154381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd8b00b06d0 msgr2=0x7fd8b00ae920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:33.622 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.608+0000 7fd8aed9d700 1 --2- 172.21.15.47:0/1995154381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd8b00b06d0 0x7fd8b00ae920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:33.622 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.608+0000 7fd8aed9d700 1 -- 172.21.15.47:0/1995154381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8b00611b0 msgr2=0x7fd8b013daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:33.622 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.608+0000 7fd8aed9d700 1 --2- 172.21.15.47:0/1995154381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8b00611b0 0x7fd8b013daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:33.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.608+0000 7fd8aed9d700 1 -- 172.21.15.47:0/1995154381 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a000b180 con 0x7fd8b00b0cf0 2022-06-29T15:42:33.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.608+0000 7fd8add9b700 1 --2- 172.21.15.47:0/1995154381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd8b00b06d0 0x7fd8b00ae920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:42:33.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.608+0000 7fd8ae59c700 1 --2- 172.21.15.47:0/1995154381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8b00611b0 0x7fd8b013daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:42:33.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.608+0000 7fd8aed9d700 1 --2- 172.21.15.47:0/1995154381 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd8b00b0cf0 0x7fd8b00aee10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fd8a80032f0 tx=0x7fd8a8016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:33.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.608+0000 7fd89f7fe700 1 -- 172.21.15.47:0/1995154381 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd8a800b8a0 con 0x7fd8b00b0cf0 2022-06-29T15:42:33.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.608+0000 7fd89f7fe700 1 -- 172.21.15.47:0/1995154381 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a8021630 con 0x7fd8b00b0cf0 2022-06-29T15:42:33.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.608+0000 7fd8b4d70700 1 -- 172.21.15.47:0/1995154381 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8a8009530 con 0x7fd8b00b0cf0 2022-06-29T15:42:33.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.608+0000 7fd89f7fe700 1 -- 172.21.15.47:0/1995154381 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd8a8021990 con 0x7fd8b00b0cf0 2022-06-29T15:42:33.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.608+0000 7fd8b4d70700 1 -- 172.21.15.47:0/1995154381 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b00ae480 con 0x7fd8b00b0cf0 2022-06-29T15:42:33.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.612+0000 7fd8b4d70700 1 -- 172.21.15.47:0/1995154381 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8b00573d0 con 0x7fd8b00b0cf0 2022-06-29T15:42:33.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.612+0000 7fd89f7fe700 1 -- 172.21.15.47:0/1995154381 <== mon.2 v2:172.21.15.64:3301/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd8a8027020 con 0x7fd8b00b0cf0 2022-06-29T15:42:33.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.612+0000 7fd89f7fe700 1 --2- 172.21.15.47:0/1995154381 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd88c053a70 0x7fd88c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:33.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.612+0000 7fd89f7fe700 1 -- 172.21.15.47:0/1995154381 <== mon.2 v2:172.21.15.64:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fd8a806d470 con 0x7fd8b00b0cf0 2022-06-29T15:42:33.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.612+0000 7fd8ae59c700 1 --2- 172.21.15.47:0/1995154381 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd88c053a70 0x7fd88c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:33.626 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.612+0000 7fd8ae59c700 1 --2- 172.21.15.47:0/1995154381 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd88c053a70 0x7fd88c055f30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fd8a400d000 tx=0x7fd8a4007c40).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:33.627 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.620+0000 7fd89f7fe700 1 -- 172.21.15.47:0/1995154381 <== mon.2 v2:172.21.15.64:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd8a8044160 con 0x7fd8b00b0cf0 2022-06-29T15:42:33.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.748+0000 7fd8b4d70700 1 -- 172.21.15.47:0/1995154381 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd8b0061750 con 0x7fd8b00b0cf0 2022-06-29T15:42:33.752 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.748+0000 7fd89f7fe700 1 -- 172.21.15.47:0/1995154381 <== mon.2 v2:172.21.15.64:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16312 (secure 0 0 0) 0x7fd8a8015020 con 0x7fd8b00b0cf0 2022-06-29T15:42:33.752 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:42:33.753 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":25,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:42:33.471155+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:42:28.400446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-06-29T15:42:31.167705+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:42:33.755 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.752+0000 7fd8b4d70700 1 -- 172.21.15.47:0/1995154381 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd88c053a70 msgr2=0x7fd88c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:33.755 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.752+0000 7fd8b4d70700 1 --2- 172.21.15.47:0/1995154381 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd88c053a70 0x7fd88c055f30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fd8a400d000 tx=0x7fd8a4007c40).stop 2022-06-29T15:42:33.755 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.752+0000 7fd8b4d70700 1 -- 172.21.15.47:0/1995154381 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd8b00b0cf0 msgr2=0x7fd8b00aee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:33.756 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.752+0000 7fd8b4d70700 1 --2- 172.21.15.47:0/1995154381 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd8b00b0cf0 0x7fd8b00aee10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fd8a80032f0 tx=0x7fd8a8016000).stop 2022-06-29T15:42:33.756 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.752+0000 7fd8b4d70700 1 -- 172.21.15.47:0/1995154381 shutdown_connections 2022-06-29T15:42:33.756 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.752+0000 7fd8b4d70700 1 --2- 172.21.15.47:0/1995154381 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd8b00b0cf0 0x7fd8b00aee10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:33.756 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.752+0000 7fd8b4d70700 1 --2- 172.21.15.47:0/1995154381 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd88c053a70 0x7fd88c055f30 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:33.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.752+0000 7fd8b4d70700 1 --2- 172.21.15.47:0/1995154381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd8b00b06d0 0x7fd8b00ae920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:33.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.752+0000 7fd8b4d70700 1 --2- 172.21.15.47:0/1995154381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8b00611b0 0x7fd8b013daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:33.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.752+0000 7fd8b4d70700 1 -- 172.21.15.47:0/1995154381 >> 172.21.15.47:0/1995154381 conn(0x7fd8b00b98b0 msgr2=0x7fd8b0148130 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:33.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.752+0000 7fd8b4d70700 1 -- 172.21.15.47:0/1995154381 shutdown_connections 2022-06-29T15:42:33.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.752+0000 7fd8b4d70700 1 -- 172.21.15.47:0/1995154381 wait complete. 2022-06-29T15:42:33.765 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:42:33.873 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.868+0000 7f2d2edfc700 1 Processor -- start 2022-06-29T15:42:33.873 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.868+0000 7f2d2edfc700 1 -- start start 2022-06-29T15:42:33.877 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.872+0000 7f2d2edfc700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2d280b04e0 0x7f2d280af590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:33.877 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.872+0000 7f2d2edfc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2d280ad0d0 0x7f2d28143090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:33.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.872+0000 7f2d2edfc700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d280ad540 con 0x7f2d280b1470 2022-06-29T15:42:33.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.872+0000 7f2d2edfc700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d280b27c0 con 0x7f2d280ad0d0 2022-06-29T15:42:33.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.872+0000 7f2d2edfc700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d280b1850 con 0x7f2d280b04e0 2022-06-29T15:42:33.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.872+0000 7f2d2e5fb700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2d280ad0d0 0x7f2d28143090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:33.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.872+0000 7f2d2ddfa700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f2d280b1470 0x7f2d280ac9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59452/0 (socket says 172.21.15.47:59452) 2022-06-29T15:42:33.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.872+0000 7f2d2ddfa700 1 -- 172.21.15.47:0/1054039431 learned_addr learned my addr 172.21.15.47:0/1054039431 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:33.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.872+0000 7f2d2d5f9700 1 --2- 172.21.15.47:0/1054039431 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2d280b04e0 0x7f2d280af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:33.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.872+0000 7f2d2e5fb700 1 -- 172.21.15.47:0/1054039431 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2d280b04e0 msgr2=0x7f2d280af590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:33.880 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.872+0000 7f2d2e5fb700 1 --2- 172.21.15.47:0/1054039431 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2d280b04e0 0x7f2d280af590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:33.880 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.872+0000 7f2d2e5fb700 1 -- 172.21.15.47:0/1054039431 >> v1:172.21.15.47:6789/0 conn(0x7f2d280b1470 legacy=0x7f2d280ac9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:33.880 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.872+0000 7f2d2e5fb700 1 -- 172.21.15.47:0/1054039431 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d28142d50 con 0x7f2d280ad0d0 2022-06-29T15:42:33.880 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.872+0000 7f2d2cdf8700 1 -- 172.21.15.47:0/1054039431 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 636130185 0 0) 0x7f2d280ad540 con 0x7f2d280b1470 2022-06-29T15:42:33.880 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.872+0000 7f2d2e5fb700 1 --2- 172.21.15.47:0/1054039431 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d280ad0d0 0x7f2d28143090 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f2d200032f0 tx=0x7f2d200091d0).ready entity=mon.0 client_cookie=313d43101507486e server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:33.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.872+0000 7f2d2cdf8700 1 -- 172.21.15.47:0/1054039431 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d2000b8b0 con 0x7f2d280ad0d0 2022-06-29T15:42:33.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.872+0000 7f2d2cdf8700 1 -- 172.21.15.47:0/1054039431 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d2000ba10 con 0x7f2d280ad0d0 2022-06-29T15:42:33.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.872+0000 7f2d2cdf8700 1 -- 172.21.15.47:0/1054039431 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d2000bd70 con 0x7f2d280ad0d0 2022-06-29T15:42:33.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.872+0000 7f2d2edfc700 1 -- 172.21.15.47:0/1054039431 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d280ad0d0 msgr2=0x7f2d28143090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:33.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.872+0000 7f2d2edfc700 1 --2- 172.21.15.47:0/1054039431 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d280ad0d0 0x7f2d28143090 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f2d200032f0 tx=0x7f2d200091d0).stop 2022-06-29T15:42:33.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.872+0000 7f2d2edfc700 1 -- 172.21.15.47:0/1054039431 shutdown_connections 2022-06-29T15:42:33.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.872+0000 7f2d2edfc700 1 --2- 172.21.15.47:0/1054039431 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d280ad0d0 0x7f2d28143090 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:33.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.872+0000 7f2d2edfc700 1 --2- 172.21.15.47:0/1054039431 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2d280b04e0 0x7f2d280af590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:33.883 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.872+0000 7f2d2edfc700 1 -- 172.21.15.47:0/1054039431 >> 172.21.15.47:0/1054039431 conn(0x7f2d28062560 msgr2=0x7f2d28062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:33.883 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.872+0000 7f2d2edfc700 1 -- 172.21.15.47:0/1054039431 shutdown_connections 2022-06-29T15:42:33.883 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.872+0000 7f2d2edfc700 1 -- 172.21.15.47:0/1054039431 wait complete. 2022-06-29T15:42:33.883 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.876+0000 7f2d2edfc700 1 Processor -- start 2022-06-29T15:42:33.884 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.876+0000 7f2d2edfc700 1 -- start start 2022-06-29T15:42:33.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.876+0000 7f2d2edfc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d280ad0d0 0x7f2d280a3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:33.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.876+0000 7f2d2edfc700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2d280b04e0 0x7f2d28066a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:33.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.876+0000 7f2d2edfc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d280b1470 0x7f2d280ab870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:33.888 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.876+0000 7f2d2edfc700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d280a4bc0 con 0x7f2d280b1470 2022-06-29T15:42:33.888 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.876+0000 7f2d2edfc700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d280a4e60 con 0x7f2d280ad0d0 2022-06-29T15:42:33.888 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.876+0000 7f2d2edfc700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d280a5100 con 0x7f2d280b04e0 2022-06-29T15:42:33.888 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.876+0000 7f2d2e5fb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d280b1470 0x7f2d280ab870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:33.889 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.876+0000 7f2d2e5fb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d280b1470 0x7f2d280ab870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45674/0 (socket says 172.21.15.47:45674) 2022-06-29T15:42:33.889 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.876+0000 7f2d2e5fb700 1 -- 172.21.15.47:0/3340122944 learned_addr learned my addr 172.21.15.47:0/3340122944 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:33.889 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.876+0000 7f2d2ddfa700 1 --2- 172.21.15.47:0/3340122944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d280ad0d0 0x7f2d280a3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:33.889 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.876+0000 7f2d2d5f9700 1 --2- 172.21.15.47:0/3340122944 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2d280b04e0 0x7f2d28066a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:33.889 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.876+0000 7f2d2e5fb700 1 -- 172.21.15.47:0/3340122944 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2d280b04e0 msgr2=0x7f2d28066a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:33.890 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.876+0000 7f2d2e5fb700 1 --2- 172.21.15.47:0/3340122944 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2d280b04e0 0x7f2d28066a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:33.890 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.876+0000 7f2d2e5fb700 1 -- 172.21.15.47:0/3340122944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d280ad0d0 msgr2=0x7f2d280a3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:33.890 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.876+0000 7f2d2e5fb700 1 --2- 172.21.15.47:0/3340122944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d280ad0d0 0x7f2d280a3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:33.891 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.876+0000 7f2d2e5fb700 1 -- 172.21.15.47:0/3340122944 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d20009470 con 0x7f2d280b1470 2022-06-29T15:42:33.891 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.876+0000 7f2d2d5f9700 1 --2- 172.21.15.47:0/3340122944 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2d280b04e0 0x7f2d28066a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:42:33.891 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.876+0000 7f2d2e5fb700 1 --2- 172.21.15.47:0/3340122944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d280b1470 0x7f2d280ab870 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f2d200032f0 tx=0x7f2d20017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:33.891 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.876+0000 7f2d16ffd700 1 -- 172.21.15.47:0/3340122944 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d20010e10 con 0x7f2d280b1470 2022-06-29T15:42:33.892 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.876+0000 7f2d16ffd700 1 -- 172.21.15.47:0/3340122944 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d2001a440 con 0x7f2d280b1470 2022-06-29T15:42:33.892 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.876+0000 7f2d2edfc700 1 -- 172.21.15.47:0/3340122944 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d280a4840 con 0x7f2d280b1470 2022-06-29T15:42:33.892 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.876+0000 7f2d16ffd700 1 -- 172.21.15.47:0/3340122944 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d2001a7a0 con 0x7f2d280b1470 2022-06-29T15:42:33.892 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.876+0000 7f2d2edfc700 1 -- 172.21.15.47:0/3340122944 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d281557d0 con 0x7f2d280b1470 2022-06-29T15:42:33.893 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.880+0000 7f2d2edfc700 1 -- 172.21.15.47:0/3340122944 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d28155ae0 con 0x7f2d280b1470 2022-06-29T15:42:33.893 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.880+0000 7f2d16ffd700 1 -- 172.21.15.47:0/3340122944 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2d2002a070 con 0x7f2d280b1470 2022-06-29T15:42:33.893 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.880+0000 7f2d16ffd700 1 --2- 172.21.15.47:0/3340122944 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2d080539e0 0x7f2d08055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:33.893 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.880+0000 7f2d2ddfa700 1 --2- 172.21.15.47:0/3340122944 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2d080539e0 0x7f2d08055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:33.893 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.880+0000 7f2d16ffd700 1 -- 172.21.15.47:0/3340122944 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f2d2006ddc0 con 0x7f2d280b1470 2022-06-29T15:42:33.894 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.880+0000 7f2d2ddfa700 1 --2- 172.21.15.47:0/3340122944 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2d080539e0 0x7f2d08055ea0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f2d2400d000 tx=0x7f2d24007bc0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:33.894 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:33.884+0000 7f2d16ffd700 1 -- 172.21.15.47:0/3340122944 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2d20044ab0 con 0x7f2d280b1470 2022-06-29T15:42:34.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.016+0000 7f2d2edfc700 1 -- 172.21.15.47:0/3340122944 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2d280a52c0 con 0x7f2d280b1470 2022-06-29T15:42:34.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.016+0000 7f2d16ffd700 1 -- 172.21.15.47:0/3340122944 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16312 (secure 0 0 0) 0x7f2d200480d0 con 0x7f2d280b1470 2022-06-29T15:42:34.019 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:42:34.019 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":25,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:42:33.471155+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:42:28.400446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-06-29T15:42:31.167705+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:42:34.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.016+0000 7f2d2edfc700 1 -- 172.21.15.47:0/3340122944 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2d080539e0 msgr2=0x7f2d08055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:34.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.016+0000 7f2d2edfc700 1 --2- 172.21.15.47:0/3340122944 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2d080539e0 0x7f2d08055ea0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f2d2400d000 tx=0x7f2d24007bc0).stop 2022-06-29T15:42:34.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.016+0000 7f2d2edfc700 1 -- 172.21.15.47:0/3340122944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d280b1470 msgr2=0x7f2d280ab870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:34.022 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.016+0000 7f2d2edfc700 1 --2- 172.21.15.47:0/3340122944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d280b1470 0x7f2d280ab870 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f2d200032f0 tx=0x7f2d20017000).stop 2022-06-29T15:42:34.022 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.016+0000 7f2d2edfc700 1 -- 172.21.15.47:0/3340122944 shutdown_connections 2022-06-29T15:42:34.022 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.016+0000 7f2d2edfc700 1 --2- 172.21.15.47:0/3340122944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d280b1470 0x7f2d280ab870 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:34.022 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.016+0000 7f2d2edfc700 1 --2- 172.21.15.47:0/3340122944 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2d280b04e0 0x7f2d28066a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:34.022 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.016+0000 7f2d2edfc700 1 --2- 172.21.15.47:0/3340122944 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2d080539e0 0x7f2d08055ea0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:34.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.016+0000 7f2d2edfc700 1 --2- 172.21.15.47:0/3340122944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d280ad0d0 0x7f2d280a3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:34.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.020+0000 7f2d2edfc700 1 -- 172.21.15.47:0/3340122944 >> 172.21.15.47:0/3340122944 conn(0x7f2d28062560 msgr2=0x7f2d2813f020 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:34.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.020+0000 7f2d2edfc700 1 -- 172.21.15.47:0/3340122944 shutdown_connections 2022-06-29T15:42:34.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.020+0000 7f2d2edfc700 1 -- 172.21.15.47:0/3340122944 wait complete. 2022-06-29T15:42:34.033 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-06-29T15:42:34.141 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.136+0000 7fd0198be700 1 Processor -- start 2022-06-29T15:42:34.141 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.136+0000 7fd0198be700 1 -- start start 2022-06-29T15:42:34.145 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.140+0000 7fd0198be700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd014064870 0x7fd014066f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:34.145 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.140+0000 7fd0198be700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd0140b2580 0x7fd014140fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:34.145 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.140+0000 7fd0198be700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd014066620 con 0x7fd0140b2a50 2022-06-29T15:42:34.146 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.140+0000 7fd0198be700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd014145340 con 0x7fd014064870 2022-06-29T15:42:34.146 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.140+0000 7fd0198be700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd014066850 con 0x7fd0140b2580 2022-06-29T15:42:34.146 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.140+0000 7fd0190bd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd0140b2a50 0x7fd014144620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59466/0 (socket says 172.21.15.47:59466) 2022-06-29T15:42:34.146 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.140+0000 7fd0190bd700 1 -- 172.21.15.47:0/1461287917 learned_addr learned my addr 172.21.15.47:0/1461287917 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:34.146 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.140+0000 7fd013fff700 1 --2- 172.21.15.47:0/1461287917 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd0140b2580 0x7fd014140fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:34.147 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.140+0000 7fd0188bc700 1 --2- 172.21.15.47:0/1461287917 >> v2:172.21.15.64:3300/0 conn(0x7fd014064870 0x7fd014066f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:34.147 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.140+0000 7fd0137fe700 1 -- 172.21.15.47:0/1461287917 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2129226299 0 0) 0x7fd014066620 con 0x7fd0140b2a50 2022-06-29T15:42:34.147 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.140+0000 7fd0137fe700 1 -- 172.21.15.47:0/1461287917 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd000003260 con 0x7fd0140b2a50 2022-06-29T15:42:34.148 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.140+0000 7fd013fff700 1 -- 172.21.15.47:0/1461287917 >> v2:172.21.15.64:3300/0 conn(0x7fd014064870 msgr2=0x7fd014066f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:34.148 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.140+0000 7fd013fff700 1 --2- 172.21.15.47:0/1461287917 >> v2:172.21.15.64:3300/0 conn(0x7fd014064870 0x7fd014066f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:34.148 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.140+0000 7fd013fff700 1 -- 172.21.15.47:0/1461287917 >> v1:172.21.15.47:6789/0 conn(0x7fd0140b2a50 legacy=0x7fd014144620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:34.148 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.140+0000 7fd013fff700 1 -- 172.21.15.47:0/1461287917 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd014066af0 con 0x7fd0140b2580 2022-06-29T15:42:34.149 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.140+0000 7fd013fff700 1 --2- 172.21.15.47:0/1461287917 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd0140b2580 0x7fd014140fe0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7fd004002f70 tx=0x7fd004006df0).ready entity=mon.2 client_cookie=37be2d4aee273528 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:34.149 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.140+0000 7fd0137fe700 1 -- 172.21.15.47:0/1461287917 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd00400d4a0 con 0x7fd0140b2580 2022-06-29T15:42:34.149 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.140+0000 7fd0137fe700 1 -- 172.21.15.47:0/1461287917 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd004000e90 con 0x7fd0140b2580 2022-06-29T15:42:34.149 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.140+0000 7fd0137fe700 1 -- 172.21.15.47:0/1461287917 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd00400d7f0 con 0x7fd0140b2580 2022-06-29T15:42:34.149 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.140+0000 7fd0198be700 1 -- 172.21.15.47:0/1461287917 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd0140b2580 msgr2=0x7fd014140fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:34.150 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.140+0000 7fd0198be700 1 --2- 172.21.15.47:0/1461287917 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd0140b2580 0x7fd014140fe0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7fd004002f70 tx=0x7fd004006df0).stop 2022-06-29T15:42:34.150 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.140+0000 7fd0198be700 1 -- 172.21.15.47:0/1461287917 shutdown_connections 2022-06-29T15:42:34.151 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.140+0000 7fd0198be700 1 --2- 172.21.15.47:0/1461287917 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd0140b2580 0x7fd014140fe0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:34.152 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.140+0000 7fd0198be700 1 --2- 172.21.15.47:0/1461287917 >> v2:172.21.15.64:3300/0 conn(0x7fd014064870 0x7fd014066f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:34.152 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.140+0000 7fd0198be700 1 -- 172.21.15.47:0/1461287917 >> 172.21.15.47:0/1461287917 conn(0x7fd01405c510 msgr2=0x7fd01405c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:34.152 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.140+0000 7fd0198be700 1 -- 172.21.15.47:0/1461287917 shutdown_connections 2022-06-29T15:42:34.153 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.140+0000 7fd0198be700 1 -- 172.21.15.47:0/1461287917 wait complete. 2022-06-29T15:42:34.156 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.144+0000 7fd0198be700 1 Processor -- start 2022-06-29T15:42:34.156 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.144+0000 7fd0198be700 1 -- start start 2022-06-29T15:42:34.157 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.144+0000 7fd0198be700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd014064870 0x7fd0141338a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:34.157 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.144+0000 7fd0198be700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0140b2a50 0x7fd014135be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:34.157 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.144+0000 7fd0198be700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd0141388c0 0x7fd0141360d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:34.157 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.144+0000 7fd0198be700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd014135280 con 0x7fd014064870 2022-06-29T15:42:34.158 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.144+0000 7fd0198be700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd014145100 con 0x7fd0140b2a50 2022-06-29T15:42:34.158 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.144+0000 7fd0198be700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd014142e80 con 0x7fd0141388c0 2022-06-29T15:42:34.158 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.144+0000 7fd0188bc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd014064870 0x7fd0141338a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:34.159 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.144+0000 7fd0188bc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd014064870 0x7fd0141338a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45684/0 (socket says 172.21.15.47:45684) 2022-06-29T15:42:34.159 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.144+0000 7fd013fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0140b2a50 0x7fd014135be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:34.159 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.144+0000 7fd0190bd700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd0141388c0 0x7fd0141360d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:34.159 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.144+0000 7fd0188bc700 1 -- 172.21.15.47:0/3936945983 learned_addr learned my addr 172.21.15.47:0/3936945983 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:34.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.144+0000 7fd0190bd700 1 -- 172.21.15.47:0/3936945983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0140b2a50 msgr2=0x7fd014135be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:34.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.144+0000 7fd0190bd700 1 --2- 172.21.15.47:0/3936945983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0140b2a50 0x7fd014135be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:34.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.144+0000 7fd0190bd700 1 -- 172.21.15.47:0/3936945983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd014064870 msgr2=0x7fd0141338a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:34.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.144+0000 7fd0190bd700 1 --2- 172.21.15.47:0/3936945983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd014064870 0x7fd0141338a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:34.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.144+0000 7fd0190bd700 1 -- 172.21.15.47:0/3936945983 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd00400b180 con 0x7fd0141388c0 2022-06-29T15:42:34.161 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.144+0000 7fd013fff700 1 --2- 172.21.15.47:0/3936945983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0140b2a50 0x7fd014135be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:42:34.161 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.144+0000 7fd0188bc700 1 --2- 172.21.15.47:0/3936945983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd014064870 0x7fd0141338a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:42:34.161 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.144+0000 7fd0190bd700 1 --2- 172.21.15.47:0/3936945983 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd0141388c0 0x7fd0141360d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7fd00c001490 tx=0x7fd00c007d40).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:34.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.148+0000 7fd011ffb700 1 -- 172.21.15.47:0/3936945983 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd00c00ce70 con 0x7fd0141388c0 2022-06-29T15:42:34.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.148+0000 7fd0198be700 1 -- 172.21.15.47:0/3936945983 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0141354b0 con 0x7fd0141388c0 2022-06-29T15:42:34.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.148+0000 7fd0198be700 1 -- 172.21.15.47:0/3936945983 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd014135950 con 0x7fd0141388c0 2022-06-29T15:42:34.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.148+0000 7fd011ffb700 1 -- 172.21.15.47:0/3936945983 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd00c005ba0 con 0x7fd0141388c0 2022-06-29T15:42:34.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.148+0000 7fd011ffb700 1 -- 172.21.15.47:0/3936945983 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd00c0070d0 con 0x7fd0141388c0 2022-06-29T15:42:34.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.152+0000 7fd0198be700 1 -- 172.21.15.47:0/3936945983 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfec003410 con 0x7fd0141388c0 2022-06-29T15:42:34.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.152+0000 7fd011ffb700 1 -- 172.21.15.47:0/3936945983 <== mon.2 v2:172.21.15.64:3301/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd00c010070 con 0x7fd0141388c0 2022-06-29T15:42:34.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.152+0000 7fd011ffb700 1 --2- 172.21.15.47:0/3936945983 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcff4053a70 0x7fcff4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:34.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.152+0000 7fd011ffb700 1 -- 172.21.15.47:0/3936945983 <== mon.2 v2:172.21.15.64:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fd00c06ba40 con 0x7fd0141388c0 2022-06-29T15:42:34.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.152+0000 7fd0188bc700 1 --2- 172.21.15.47:0/3936945983 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcff4053a70 0x7fcff4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:34.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.152+0000 7fd0188bc700 1 --2- 172.21.15.47:0/3936945983 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcff4053a70 0x7fcff4055f30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fd00800b000 tx=0x7fd008006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:34.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.160+0000 7fd011ffb700 1 -- 172.21.15.47:0/3936945983 <== mon.2 v2:172.21.15.64:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd00c042670 con 0x7fd0141388c0 2022-06-29T15:42:34.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:34.324+0000 7fd0198be700 1 -- 172.21.15.47:0/3936945983 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fcfec003c40 con 0x7fd0141388c0 2022-06-29T15:42:35.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.156+0000 7fd011ffb700 1 -- 172.21.15.47:0/3936945983 <== mon.2 v2:172.21.15.64:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd00c045c90 con 0x7fd0141388c0 2022-06-29T15:42:35.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.160+0000 7fd0198be700 1 -- 172.21.15.47:0/3936945983 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcff4053a70 msgr2=0x7fcff4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:35.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.160+0000 7fd0198be700 1 --2- 172.21.15.47:0/3936945983 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcff4053a70 0x7fcff4055f30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fd00800b000 tx=0x7fd008006df0).stop 2022-06-29T15:42:35.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.160+0000 7fd0198be700 1 -- 172.21.15.47:0/3936945983 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd0141388c0 msgr2=0x7fd0141360d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:35.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.160+0000 7fd0198be700 1 --2- 172.21.15.47:0/3936945983 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd0141388c0 0x7fd0141360d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7fd00c001490 tx=0x7fd00c007d40).stop 2022-06-29T15:42:35.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.160+0000 7fd0198be700 1 -- 172.21.15.47:0/3936945983 shutdown_connections 2022-06-29T15:42:35.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.160+0000 7fd0198be700 1 --2- 172.21.15.47:0/3936945983 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd0141388c0 0x7fd0141360d0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:35.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.160+0000 7fd0198be700 1 --2- 172.21.15.47:0/3936945983 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcff4053a70 0x7fcff4055f30 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:35.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.160+0000 7fd0198be700 1 --2- 172.21.15.47:0/3936945983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0140b2a50 0x7fd014135be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:35.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.160+0000 7fd0198be700 1 --2- 172.21.15.47:0/3936945983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd014064870 0x7fd0141338a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:35.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.160+0000 7fd0198be700 1 -- 172.21.15.47:0/3936945983 >> 172.21.15.47:0/3936945983 conn(0x7fd01405c510 msgr2=0x7fd014147b50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:35.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.160+0000 7fd0198be700 1 -- 172.21.15.47:0/3936945983 shutdown_connections 2022-06-29T15:42:35.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.160+0000 7fd0198be700 1 -- 172.21.15.47:0/3936945983 wait complete. 2022-06-29T15:42:35.181 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:42:35.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.284+0000 7fd0c288d700 1 Processor -- start 2022-06-29T15:42:35.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.288+0000 7fd0c288d700 1 -- start start 2022-06-29T15:42:35.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.288+0000 7fd0c288d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd0bc0b2a90 0x7fd0bc061050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:35.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.288+0000 7fd0c288d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd0bc0601a0 0x7fd0bc140520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:35.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.288+0000 7fd0c288d700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0bc0b2170 con 0x7fd0bc061600 2022-06-29T15:42:35.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.288+0000 7fd0c288d700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0bc0b2e70 con 0x7fd0bc0b2a90 2022-06-29T15:42:35.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.288+0000 7fd0c288d700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0bc0606b0 con 0x7fd0bc0601a0 2022-06-29T15:42:35.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.288+0000 7fd0bb7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd0bc061600 0x7fd0bc146b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59480/0 (socket says 172.21.15.47:59480) 2022-06-29T15:42:35.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.288+0000 7fd0bb7fe700 1 -- 172.21.15.47:0/2008219869 learned_addr learned my addr 172.21.15.47:0/2008219869 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:35.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.288+0000 7fd0c0e2a700 1 --2- 172.21.15.47:0/2008219869 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd0bc0601a0 0x7fd0bc140520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:35.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.288+0000 7fd0bbfff700 1 --2- 172.21.15.47:0/2008219869 >> v2:172.21.15.64:3300/0 conn(0x7fd0bc0b2a90 0x7fd0bc061050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:35.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.288+0000 7fd0baffd700 1 -- 172.21.15.47:0/2008219869 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3488651910 0 0) 0x7fd0bc0b2170 con 0x7fd0bc061600 2022-06-29T15:42:35.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.288+0000 7fd0baffd700 1 -- 172.21.15.47:0/2008219869 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0a0003260 con 0x7fd0bc061600 2022-06-29T15:42:35.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.288+0000 7fd0c0e2a700 1 -- 172.21.15.47:0/2008219869 >> v2:172.21.15.64:3300/0 conn(0x7fd0bc0b2a90 msgr2=0x7fd0bc061050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:35.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.288+0000 7fd0c0e2a700 1 --2- 172.21.15.47:0/2008219869 >> v2:172.21.15.64:3300/0 conn(0x7fd0bc0b2a90 0x7fd0bc061050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:35.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.288+0000 7fd0c0e2a700 1 -- 172.21.15.47:0/2008219869 >> v1:172.21.15.47:6789/0 conn(0x7fd0bc061600 legacy=0x7fd0bc146b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:35.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.288+0000 7fd0c0e2a700 1 -- 172.21.15.47:0/2008219869 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0bc0a9770 con 0x7fd0bc0601a0 2022-06-29T15:42:35.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.288+0000 7fd0c0e2a700 1 --2- 172.21.15.47:0/2008219869 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd0bc0601a0 0x7fd0bc140520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7fd0b4002f70 tx=0x7fd0b4006df0).ready entity=mon.2 client_cookie=35e6c2c4aa5fd689 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:35.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.288+0000 7fd0baffd700 1 -- 172.21.15.47:0/2008219869 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0b400d4a0 con 0x7fd0bc0601a0 2022-06-29T15:42:35.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.288+0000 7fd0baffd700 1 -- 172.21.15.47:0/2008219869 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0b4000e90 con 0x7fd0bc0601a0 2022-06-29T15:42:35.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.288+0000 7fd0baffd700 1 -- 172.21.15.47:0/2008219869 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0b400d7f0 con 0x7fd0bc0601a0 2022-06-29T15:42:35.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.288+0000 7fd0c288d700 1 -- 172.21.15.47:0/2008219869 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd0bc0601a0 msgr2=0x7fd0bc140520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:35.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.288+0000 7fd0c288d700 1 --2- 172.21.15.47:0/2008219869 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd0bc0601a0 0x7fd0bc140520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7fd0b4002f70 tx=0x7fd0b4006df0).stop 2022-06-29T15:42:35.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.288+0000 7fd0c288d700 1 -- 172.21.15.47:0/2008219869 shutdown_connections 2022-06-29T15:42:35.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.288+0000 7fd0c288d700 1 --2- 172.21.15.47:0/2008219869 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd0bc0601a0 0x7fd0bc140520 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:35.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.288+0000 7fd0c288d700 1 --2- 172.21.15.47:0/2008219869 >> v2:172.21.15.64:3300/0 conn(0x7fd0bc0b2a90 0x7fd0bc061050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:35.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.288+0000 7fd0c288d700 1 -- 172.21.15.47:0/2008219869 >> 172.21.15.47:0/2008219869 conn(0x7fd0bc05f0b0 msgr2=0x7fd0bc05f490 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:35.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.288+0000 7fd0c288d700 1 -- 172.21.15.47:0/2008219869 shutdown_connections 2022-06-29T15:42:35.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.288+0000 7fd0c288d700 1 -- 172.21.15.47:0/2008219869 wait complete. 2022-06-29T15:42:35.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.292+0000 7fd0c288d700 1 Processor -- start 2022-06-29T15:42:35.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.292+0000 7fd0c288d700 1 -- start start 2022-06-29T15:42:35.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.292+0000 7fd0c288d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0bc0601a0 0x7fd0bc065480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:35.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.292+0000 7fd0c288d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd0bc061600 0x7fd0bc067150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:35.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.292+0000 7fd0c288d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0bc0b2a90 0x7fd0bc067640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:35.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.292+0000 7fd0c288d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0bc1496c0 con 0x7fd0bc0b2a90 2022-06-29T15:42:35.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.292+0000 7fd0c288d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0bc148ac0 con 0x7fd0bc0601a0 2022-06-29T15:42:35.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.292+0000 7fd0c288d700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0bc149220 con 0x7fd0bc061600 2022-06-29T15:42:35.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.292+0000 7fd0c0e2a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0bc0b2a90 0x7fd0bc067640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:35.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.292+0000 7fd0bb7fe700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd0bc061600 0x7fd0bc067150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:35.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.292+0000 7fd0c0e2a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0bc0b2a90 0x7fd0bc067640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45702/0 (socket says 172.21.15.47:45702) 2022-06-29T15:42:35.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.292+0000 7fd0c0e2a700 1 -- 172.21.15.47:0/3383871399 learned_addr learned my addr 172.21.15.47:0/3383871399 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:35.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.292+0000 7fd0bbfff700 1 --2- 172.21.15.47:0/3383871399 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0bc0601a0 0x7fd0bc065480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:35.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.292+0000 7fd0c0e2a700 1 -- 172.21.15.47:0/3383871399 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd0bc061600 msgr2=0x7fd0bc067150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:35.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.292+0000 7fd0c0e2a700 1 --2- 172.21.15.47:0/3383871399 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd0bc061600 0x7fd0bc067150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:35.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.292+0000 7fd0c0e2a700 1 -- 172.21.15.47:0/3383871399 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0bc0601a0 msgr2=0x7fd0bc065480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:35.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.292+0000 7fd0c0e2a700 1 --2- 172.21.15.47:0/3383871399 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0bc0601a0 0x7fd0bc065480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:35.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.292+0000 7fd0c0e2a700 1 -- 172.21.15.47:0/3383871399 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b400b180 con 0x7fd0bc0b2a90 2022-06-29T15:42:35.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.292+0000 7fd0bbfff700 1 --2- 172.21.15.47:0/3383871399 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0bc0601a0 0x7fd0bc065480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:42:35.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.292+0000 7fd0c0e2a700 1 --2- 172.21.15.47:0/3383871399 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0bc0b2a90 0x7fd0bc067640 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7fd0b4002f70 tx=0x7fd0b4014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:35.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.292+0000 7fd0b97fa700 1 -- 172.21.15.47:0/3383871399 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0b40139c0 con 0x7fd0bc0b2a90 2022-06-29T15:42:35.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.292+0000 7fd0b97fa700 1 -- 172.21.15.47:0/3383871399 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0b4013b20 con 0x7fd0bc0b2a90 2022-06-29T15:42:35.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.292+0000 7fd0c288d700 1 -- 172.21.15.47:0/3383871399 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0bc067b30 con 0x7fd0bc0b2a90 2022-06-29T15:42:35.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.292+0000 7fd0b97fa700 1 -- 172.21.15.47:0/3383871399 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0b4013e80 con 0x7fd0bc0b2a90 2022-06-29T15:42:35.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.292+0000 7fd0c288d700 1 -- 172.21.15.47:0/3383871399 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0bc148580 con 0x7fd0bc0b2a90 2022-06-29T15:42:35.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.296+0000 7fd0c288d700 1 -- 172.21.15.47:0/3383871399 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0bc14a9b0 con 0x7fd0bc0b2a90 2022-06-29T15:42:35.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.296+0000 7fd0b97fa700 1 -- 172.21.15.47:0/3383871399 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd0b4025030 con 0x7fd0bc0b2a90 2022-06-29T15:42:35.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.296+0000 7fd0b97fa700 1 --2- 172.21.15.47:0/3383871399 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd094053950 0x7fd094055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:35.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.296+0000 7fd0bbfff700 1 --2- 172.21.15.47:0/3383871399 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd094053950 0x7fd094055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:35.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.296+0000 7fd0b97fa700 1 -- 172.21.15.47:0/3383871399 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fd0b406cf00 con 0x7fd0bc0b2a90 2022-06-29T15:42:35.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.296+0000 7fd0bbfff700 1 --2- 172.21.15.47:0/3383871399 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd094053950 0x7fd094055e10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fd0ac00b000 tx=0x7fd0ac008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:35.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.304+0000 7fd0b97fa700 1 -- 172.21.15.47:0/3383871399 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd0b4043bf0 con 0x7fd0bc0b2a90 2022-06-29T15:42:35.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.464+0000 7fd0c288d700 1 -- 172.21.15.47:0/3383871399 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd0bc1493e0 con 0x7fd0bc0b2a90 2022-06-29T15:42:35.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.468+0000 7fd0b97fa700 1 -- 172.21.15.47:0/3383871399 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+2483 (secure 0 0 0) 0x7fd0b40471d0 con 0x7fd0bc0b2a90 2022-06-29T15:42:35.470 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:42:35.470 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/438289599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":438289599},{"type":"v1","addr":"172.21.15.47:6835","nonce":438289599}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.64:6833/1472525731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1472525731},{"type":"v1","addr":"172.21.15.64:6833","nonce":1472525731}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":4}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:42:33.471491+0000","modified":"2022-06-29T15:42:35.146757+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":54128},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54128":{"gid":54128,"name":"a","rank":0,"incarnation":5,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6837/354581778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":354581778},{"type":"v1","addr":"172.21.15.47:6837","nonce":354581778}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2022-06-29T15:42:35.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.468+0000 7fd0c288d700 1 -- 172.21.15.47:0/3383871399 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd094053950 msgr2=0x7fd094055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:35.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.468+0000 7fd0c288d700 1 --2- 172.21.15.47:0/3383871399 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd094053950 0x7fd094055e10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fd0ac00b000 tx=0x7fd0ac008040).stop 2022-06-29T15:42:35.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.468+0000 7fd0c288d700 1 -- 172.21.15.47:0/3383871399 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0bc0b2a90 msgr2=0x7fd0bc067640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:35.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.468+0000 7fd0c288d700 1 --2- 172.21.15.47:0/3383871399 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0bc0b2a90 0x7fd0bc067640 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7fd0b4002f70 tx=0x7fd0b4014000).stop 2022-06-29T15:42:35.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.468+0000 7fd0c288d700 1 -- 172.21.15.47:0/3383871399 shutdown_connections 2022-06-29T15:42:35.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.468+0000 7fd0c288d700 1 --2- 172.21.15.47:0/3383871399 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0bc0b2a90 0x7fd0bc067640 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:35.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.468+0000 7fd0c288d700 1 --2- 172.21.15.47:0/3383871399 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd0bc061600 0x7fd0bc067150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:35.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.468+0000 7fd0c288d700 1 --2- 172.21.15.47:0/3383871399 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd094053950 0x7fd094055e10 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:35.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.468+0000 7fd0c288d700 1 --2- 172.21.15.47:0/3383871399 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0bc0601a0 0x7fd0bc065480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:35.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.468+0000 7fd0c288d700 1 -- 172.21.15.47:0/3383871399 >> 172.21.15.47:0/3383871399 conn(0x7fd0bc05f0b0 msgr2=0x7fd0bc097810 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:35.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.468+0000 7fd0c288d700 1 -- 172.21.15.47:0/3383871399 shutdown_connections 2022-06-29T15:42:35.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.468+0000 7fd0c288d700 1 -- 172.21.15.47:0/3383871399 wait complete. 2022-06-29T15:42:35.475 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 7 2022-06-29T15:42:35.483 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:42:35.581 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.580+0000 7f678bba5700 1 Processor -- start 2022-06-29T15:42:35.582 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.580+0000 7f678bba5700 1 -- start start 2022-06-29T15:42:35.583 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.580+0000 7f678bba5700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f67840b0f30 0x7f67840606e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:35.583 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.580+0000 7f678bba5700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6784140d50 con 0x7f67841408e0 2022-06-29T15:42:35.584 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.580+0000 7f678bba5700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6784061d90 con 0x7f67840619b0 2022-06-29T15:42:35.584 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.580+0000 7f678bba5700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67840b2190 con 0x7f67840b0f30 2022-06-29T15:42:35.584 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.580+0000 7f6789140700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f67840b0f30 0x7f67840606e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:35.584 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.580+0000 7f678a142700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f67841408e0 0x7f6784139a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59496/0 (socket says 172.21.15.47:59496) 2022-06-29T15:42:35.584 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.580+0000 7f678a142700 1 -- 172.21.15.47:0/3259750568 learned_addr learned my addr 172.21.15.47:0/3259750568 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:35.585 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.580+0000 7f6789140700 1 -- 172.21.15.47:0/3259750568 >> v1:172.21.15.64:6789/0 conn(0x7f67840619b0 legacy=0x7f67840b1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:35.585 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.580+0000 7f6789140700 1 -- 172.21.15.47:0/3259750568 >> v1:172.21.15.47:6789/0 conn(0x7f67841408e0 legacy=0x7f6784139a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:35.585 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.580+0000 7f6789140700 1 -- 172.21.15.47:0/3259750568 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6784060d80 con 0x7f67840b0f30 2022-06-29T15:42:35.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.580+0000 7f6789140700 1 --2- 172.21.15.47:0/3259750568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67840b0f30 0x7f67840606e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f6774002f70 tx=0x7f6774006df0).ready entity=mon.0 client_cookie=94464d41d5dd7eb7 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:35.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.580+0000 7f678893f700 1 -- 172.21.15.47:0/3259750568 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f677400d610 con 0x7f67840b0f30 2022-06-29T15:42:35.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.580+0000 7f678893f700 1 -- 172.21.15.47:0/3259750568 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6774000e90 con 0x7f67840b0f30 2022-06-29T15:42:35.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.580+0000 7f678893f700 1 -- 172.21.15.47:0/3259750568 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f677400d960 con 0x7f67840b0f30 2022-06-29T15:42:35.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.584+0000 7f678bba5700 1 -- 172.21.15.47:0/3259750568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67840b0f30 msgr2=0x7f67840606e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:35.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.584+0000 7f678bba5700 1 --2- 172.21.15.47:0/3259750568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67840b0f30 0x7f67840606e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f6774002f70 tx=0x7f6774006df0).stop 2022-06-29T15:42:35.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.584+0000 7f678bba5700 1 -- 172.21.15.47:0/3259750568 shutdown_connections 2022-06-29T15:42:35.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.584+0000 7f678bba5700 1 --2- 172.21.15.47:0/3259750568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67840b0f30 0x7f67840606e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:35.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.584+0000 7f678bba5700 1 -- 172.21.15.47:0/3259750568 >> 172.21.15.47:0/3259750568 conn(0x7f678405c510 msgr2=0x7f6784063800 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:35.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.584+0000 7f678bba5700 1 -- 172.21.15.47:0/3259750568 shutdown_connections 2022-06-29T15:42:35.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.584+0000 7f678bba5700 1 -- 172.21.15.47:0/3259750568 wait complete. 2022-06-29T15:42:35.594 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.584+0000 7f678bba5700 1 Processor -- start 2022-06-29T15:42:35.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.584+0000 7f678bba5700 1 -- start start 2022-06-29T15:42:35.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.588+0000 7f678bba5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67840619b0 0x7f67840af710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:35.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.588+0000 7f678bba5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67840b0f30 0x7f6784150640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:35.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.588+0000 7f678bba5700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f67841408e0 0x7f6784150b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:35.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.588+0000 7f678bba5700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678413a5a0 con 0x7f67840619b0 2022-06-29T15:42:35.597 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.588+0000 7f678bba5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6784154500 con 0x7f67840b0f30 2022-06-29T15:42:35.597 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.588+0000 7f678bba5700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67841547a0 con 0x7f67841408e0 2022-06-29T15:42:35.597 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.588+0000 7f6789941700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67840619b0 0x7f67840af710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:35.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.588+0000 7f6789941700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67840619b0 0x7f67840af710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45712/0 (socket says 172.21.15.47:45712) 2022-06-29T15:42:35.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.588+0000 7f6789941700 1 -- 172.21.15.47:0/1927203425 learned_addr learned my addr 172.21.15.47:0/1927203425 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:35.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.588+0000 7f6789140700 1 --2- 172.21.15.47:0/1927203425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67840b0f30 0x7f6784150640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:35.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.588+0000 7f678a142700 1 --2- 172.21.15.47:0/1927203425 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f67841408e0 0x7f6784150b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:35.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.588+0000 7f6789941700 1 -- 172.21.15.47:0/1927203425 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f67841408e0 msgr2=0x7f6784150b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:35.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.588+0000 7f6789941700 1 --2- 172.21.15.47:0/1927203425 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f67841408e0 0x7f6784150b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:35.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.588+0000 7f6789941700 1 -- 172.21.15.47:0/1927203425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67840b0f30 msgr2=0x7f6784150640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:35.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.588+0000 7f6789941700 1 --2- 172.21.15.47:0/1927203425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67840b0f30 0x7f6784150640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:35.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.588+0000 7f6789941700 1 -- 172.21.15.47:0/1927203425 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f677400b180 con 0x7f67840619b0 2022-06-29T15:42:35.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.588+0000 7f678a142700 1 --2- 172.21.15.47:0/1927203425 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f67841408e0 0x7f6784150b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:42:35.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.588+0000 7f6789140700 1 --2- 172.21.15.47:0/1927203425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67840b0f30 0x7f6784150640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:42:35.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.588+0000 7f6789941700 1 --2- 172.21.15.47:0/1927203425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67840619b0 0x7f67840af710 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f6784064120 tx=0x7f6780007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:35.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.588+0000 7f6772ffd700 1 -- 172.21.15.47:0/1927203425 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f678000ce30 con 0x7f67840619b0 2022-06-29T15:42:35.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.588+0000 7f6772ffd700 1 -- 172.21.15.47:0/1927203425 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f678000f470 con 0x7f67840619b0 2022-06-29T15:42:35.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.588+0000 7f678bba5700 1 -- 172.21.15.47:0/1927203425 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f678405f300 con 0x7f67840619b0 2022-06-29T15:42:35.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.588+0000 7f6772ffd700 1 -- 172.21.15.47:0/1927203425 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f678000f7d0 con 0x7f67840619b0 2022-06-29T15:42:35.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.588+0000 7f678bba5700 1 -- 172.21.15.47:0/1927203425 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f678405ff80 con 0x7f67840619b0 2022-06-29T15:42:35.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.588+0000 7f678bba5700 1 -- 172.21.15.47:0/1927203425 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67840573d0 con 0x7f67840619b0 2022-06-29T15:42:35.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.592+0000 7f6772ffd700 1 -- 172.21.15.47:0/1927203425 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6780019030 con 0x7f67840619b0 2022-06-29T15:42:35.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.592+0000 7f6772ffd700 1 --2- 172.21.15.47:0/1927203425 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6764053aa0 0x7f6764055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:35.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.592+0000 7f6772ffd700 1 -- 172.21.15.47:0/1927203425 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f678006b590 con 0x7f67840619b0 2022-06-29T15:42:35.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.592+0000 7f6789140700 1 --2- 172.21.15.47:0/1927203425 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6764053aa0 0x7f6764055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:35.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.592+0000 7f6789140700 1 --2- 172.21.15.47:0/1927203425 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6764053aa0 0x7f6764055f60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f677401a000 tx=0x7f6774015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:35.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.600+0000 7f6772ffd700 1 -- 172.21.15.47:0/1927203425 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6780042240 con 0x7f67840619b0 2022-06-29T15:42:35.732 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.728+0000 7f678bba5700 1 -- 172.21.15.47:0/1927203425 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f678413a7a0 con 0x7f67840619b0 2022-06-29T15:42:35.734 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.732+0000 7f6772ffd700 1 -- 172.21.15.47:0/1927203425 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f6780045860 con 0x7f67840619b0 2022-06-29T15:42:35.734 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:42:35.735 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":26,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:42:34.474438+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:42:28.400446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-06-29T15:42:31.167705+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:42:35.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.732+0000 7f678bba5700 1 -- 172.21.15.47:0/1927203425 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6764053aa0 msgr2=0x7f6764055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:35.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.732+0000 7f678bba5700 1 --2- 172.21.15.47:0/1927203425 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6764053aa0 0x7f6764055f60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f677401a000 tx=0x7f6774015000).stop 2022-06-29T15:42:35.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.732+0000 7f678bba5700 1 -- 172.21.15.47:0/1927203425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67840619b0 msgr2=0x7f67840af710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:35.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.732+0000 7f678bba5700 1 --2- 172.21.15.47:0/1927203425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67840619b0 0x7f67840af710 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f6784064120 tx=0x7f6780007c50).stop 2022-06-29T15:42:35.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.732+0000 7f678bba5700 1 -- 172.21.15.47:0/1927203425 shutdown_connections 2022-06-29T15:42:35.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.732+0000 7f678bba5700 1 --2- 172.21.15.47:0/1927203425 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f67841408e0 0x7f6784150b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:35.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.732+0000 7f678bba5700 1 --2- 172.21.15.47:0/1927203425 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6764053aa0 0x7f6764055f60 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:35.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.732+0000 7f678bba5700 1 --2- 172.21.15.47:0/1927203425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67840b0f30 0x7f6784150640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:35.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.732+0000 7f678bba5700 1 --2- 172.21.15.47:0/1927203425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67840619b0 0x7f67840af710 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:35.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.732+0000 7f678bba5700 1 -- 172.21.15.47:0/1927203425 >> 172.21.15.47:0/1927203425 conn(0x7f678405c510 msgr2=0x7f67840b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:35.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.732+0000 7f678bba5700 1 -- 172.21.15.47:0/1927203425 shutdown_connections 2022-06-29T15:42:35.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.732+0000 7f678bba5700 1 -- 172.21.15.47:0/1927203425 wait complete. 2022-06-29T15:42:35.748 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:42:35.749 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2022-06-29T15:42:35.853 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.852+0000 7fbe962bb700 1 Processor -- start 2022-06-29T15:42:35.854 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.852+0000 7fbe962bb700 1 -- start start 2022-06-29T15:42:35.857 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.852+0000 7fbe962bb700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbe90143bd0 0x7fbe9005c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:35.858 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.852+0000 7fbe962bb700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbe900b2c40 0x7fbe900b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:35.858 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.852+0000 7fbe962bb700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe90142f90 con 0x7fbe9005c510 2022-06-29T15:42:35.858 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.852+0000 7fbe962bb700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe901431f0 con 0x7fbe900b2c40 2022-06-29T15:42:35.858 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.852+0000 7fbe962bb700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe90143490 con 0x7fbe90143bd0 2022-06-29T15:42:35.859 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.852+0000 7fbe952b9700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fbe9005c510 0x7fbe900b4100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59506/0 (socket says 172.21.15.47:59506) 2022-06-29T15:42:35.859 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.852+0000 7fbe952b9700 1 -- 172.21.15.47:0/3720523039 learned_addr learned my addr 172.21.15.47:0/3720523039 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:35.859 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.852+0000 7fbe95aba700 1 --2- 172.21.15.47:0/3720523039 >> v2:172.21.15.64:3300/0 conn(0x7fbe900b2c40 0x7fbe900b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:35.859 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.852+0000 7fbe94ab8700 1 --2- 172.21.15.47:0/3720523039 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbe90143bd0 0x7fbe9005c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:35.860 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.852+0000 7fbe87fff700 1 -- 172.21.15.47:0/3720523039 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1790751008 0 0) 0x7fbe90142f90 con 0x7fbe9005c510 2022-06-29T15:42:35.860 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.852+0000 7fbe87fff700 1 -- 172.21.15.47:0/3720523039 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe74003260 con 0x7fbe9005c510 2022-06-29T15:42:35.860 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.852+0000 7fbe87fff700 1 -- 172.21.15.47:0/3720523039 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3532710714 0 0) 0x7fbe74003260 con 0x7fbe9005c510 2022-06-29T15:42:35.860 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.852+0000 7fbe87fff700 1 -- 172.21.15.47:0/3720523039 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbe90143bd0 msgr2=0x7fbe9005c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:35.861 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.852+0000 7fbe87fff700 1 --2- 172.21.15.47:0/3720523039 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbe90143bd0 0x7fbe9005c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:35.861 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.852+0000 7fbe87fff700 1 -- 172.21.15.47:0/3720523039 >> v2:172.21.15.64:3300/0 conn(0x7fbe900b2c40 msgr2=0x7fbe900b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:35.861 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.852+0000 7fbe87fff700 1 --2- 172.21.15.47:0/3720523039 >> v2:172.21.15.64:3300/0 conn(0x7fbe900b2c40 0x7fbe900b30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:35.861 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.852+0000 7fbe87fff700 1 -- 172.21.15.47:0/3720523039 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe90143730 con 0x7fbe9005c510 2022-06-29T15:42:35.862 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.852+0000 7fbe87fff700 1 -- 172.21.15.47:0/3720523039 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7fbe8c001680 con 0x7fbe9005c510 2022-06-29T15:42:35.862 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.852+0000 7fbe87fff700 1 -- 172.21.15.47:0/3720523039 >> v1:172.21.15.47:6789/0 conn(0x7fbe9005c510 legacy=0x7fbe900b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:35.862 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.852+0000 7fbe87fff700 1 --2- 172.21.15.47:0/3720523039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe74003750 0x7fbe74003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:35.862 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.852+0000 7fbe87fff700 1 --2- 172.21.15.47:0/3720523039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe74004500 0x7fbe74004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:35.863 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.852+0000 7fbe87fff700 1 --2- 172.21.15.47:0/3720523039 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbe90143bd0 0x7fbe74004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:35.863 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.852+0000 7fbe87fff700 1 -- 172.21.15.47:0/3720523039 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe901431f0 con 0x7fbe74004500 2022-06-29T15:42:35.863 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.852+0000 7fbe87fff700 1 -- 172.21.15.47:0/3720523039 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe90143490 con 0x7fbe74003750 2022-06-29T15:42:35.864 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.852+0000 7fbe87fff700 1 -- 172.21.15.47:0/3720523039 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7400b750 con 0x7fbe90143bd0 2022-06-29T15:42:35.864 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.852+0000 7fbe94ab8700 1 --2- 172.21.15.47:0/3720523039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe74004500 0x7fbe74004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:35.864 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.852+0000 7fbe952b9700 1 --2- 172.21.15.47:0/3720523039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe74003750 0x7fbe74003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:35.864 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.852+0000 7fbe95aba700 1 --2- 172.21.15.47:0/3720523039 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbe90143bd0 0x7fbe74004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:35.865 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.852+0000 7fbe94ab8700 1 -- 172.21.15.47:0/3720523039 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbe90143bd0 msgr2=0x7fbe74004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:35.865 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.852+0000 7fbe94ab8700 1 --2- 172.21.15.47:0/3720523039 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbe90143bd0 0x7fbe74004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:35.865 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.852+0000 7fbe94ab8700 1 -- 172.21.15.47:0/3720523039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe74003750 msgr2=0x7fbe74003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:35.865 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.852+0000 7fbe94ab8700 1 --2- 172.21.15.47:0/3720523039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe74003750 0x7fbe74003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:35.866 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.852+0000 7fbe94ab8700 1 -- 172.21.15.47:0/3720523039 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe74003260 con 0x7fbe74004500 2022-06-29T15:42:35.866 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.852+0000 7fbe94ab8700 1 --2- 172.21.15.47:0/3720523039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe74004500 0x7fbe74004980 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7fbe8000c000 tx=0x7fbe80006df0).ready entity=mon.0 client_cookie=b1b1a686c593fb7c server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:35.866 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.852+0000 7fbe87fff700 1 -- 172.21.15.47:0/3720523039 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe8000e830 con 0x7fbe74004500 2022-06-29T15:42:35.866 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.852+0000 7fbe87fff700 1 -- 172.21.15.47:0/3720523039 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe8000e990 con 0x7fbe74004500 2022-06-29T15:42:35.867 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.852+0000 7fbe87fff700 1 -- 172.21.15.47:0/3720523039 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe8000ecf0 con 0x7fbe74004500 2022-06-29T15:42:35.867 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.856+0000 7fbe962bb700 1 -- 172.21.15.47:0/3720523039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe74004500 msgr2=0x7fbe74004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:35.867 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.856+0000 7fbe962bb700 1 --2- 172.21.15.47:0/3720523039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe74004500 0x7fbe74004980 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7fbe8000c000 tx=0x7fbe80006df0).stop 2022-06-29T15:42:35.867 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.856+0000 7fbe95aba700 1 -- 172.21.15.47:0/3720523039 reap_dead start 2022-06-29T15:42:35.867 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.856+0000 7fbe962bb700 1 -- 172.21.15.47:0/3720523039 shutdown_connections 2022-06-29T15:42:35.868 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.856+0000 7fbe962bb700 1 -- 172.21.15.47:0/3720523039 >> 172.21.15.47:0/3720523039 conn(0x7fbe90060e00 msgr2=0x7fbe900611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:35.868 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.856+0000 7fbe962bb700 1 -- 172.21.15.47:0/3720523039 shutdown_connections 2022-06-29T15:42:35.868 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.856+0000 7fbe962bb700 1 -- 172.21.15.47:0/3720523039 wait complete. 2022-06-29T15:42:35.872 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.856+0000 7fbe962bb700 1 Processor -- start 2022-06-29T15:42:35.872 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.856+0000 7fbe962bb700 1 -- start start 2022-06-29T15:42:35.873 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.856+0000 7fbe962bb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe74004500 0x7fbe9013de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:35.873 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.856+0000 7fbe962bb700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbe90146cb0 0x7fbe901401b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:35.873 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.856+0000 7fbe962bb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe900ad670 0x7fbe9013b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:35.873 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.856+0000 7fbe962bb700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe90146820 con 0x7fbe900ad670 2022-06-29T15:42:35.874 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.856+0000 7fbe962bb700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe901471a0 con 0x7fbe74004500 2022-06-29T15:42:35.874 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.856+0000 7fbe962bb700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe900b60c0 con 0x7fbe90146cb0 2022-06-29T15:42:35.874 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.856+0000 7fbe95aba700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe900ad670 0x7fbe9013b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:35.874 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.856+0000 7fbe95aba700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe900ad670 0x7fbe9013b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45736/0 (socket says 172.21.15.47:45736) 2022-06-29T15:42:35.875 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.856+0000 7fbe95aba700 1 -- 172.21.15.47:0/2756422703 learned_addr learned my addr 172.21.15.47:0/2756422703 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:35.875 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.856+0000 7fbe94ab8700 1 --2- 172.21.15.47:0/2756422703 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbe90146cb0 0x7fbe901401b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:35.875 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.856+0000 7fbe952b9700 1 --2- 172.21.15.47:0/2756422703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe74004500 0x7fbe9013de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:35.875 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.856+0000 7fbe95aba700 1 -- 172.21.15.47:0/2756422703 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbe90146cb0 msgr2=0x7fbe901401b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:35.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.856+0000 7fbe95aba700 1 --2- 172.21.15.47:0/2756422703 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbe90146cb0 0x7fbe901401b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:35.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.856+0000 7fbe95aba700 1 -- 172.21.15.47:0/2756422703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe74004500 msgr2=0x7fbe9013de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:35.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.856+0000 7fbe95aba700 1 --2- 172.21.15.47:0/2756422703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe74004500 0x7fbe9013de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:35.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.856+0000 7fbe95aba700 1 -- 172.21.15.47:0/2756422703 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe901406a0 con 0x7fbe900ad670 2022-06-29T15:42:35.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.856+0000 7fbe95aba700 1 --2- 172.21.15.47:0/2756422703 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe900ad670 0x7fbe9013b3f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7fbe8800b000 tx=0x7fbe88008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:35.877 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.856+0000 7fbe867fc700 1 -- 172.21.15.47:0/2756422703 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe880145f0 con 0x7fbe900ad670 2022-06-29T15:42:35.877 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.856+0000 7fbe867fc700 1 -- 172.21.15.47:0/2756422703 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe88014750 con 0x7fbe900ad670 2022-06-29T15:42:35.877 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.856+0000 7fbe867fc700 1 -- 172.21.15.47:0/2756422703 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe88014ab0 con 0x7fbe900ad670 2022-06-29T15:42:35.877 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.856+0000 7fbe962bb700 1 -- 172.21.15.47:0/2756422703 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe9013c7b0 con 0x7fbe900ad670 2022-06-29T15:42:35.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.856+0000 7fbe962bb700 1 -- 172.21.15.47:0/2756422703 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe9013b8e0 con 0x7fbe900ad670 2022-06-29T15:42:35.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.860+0000 7fbe962bb700 1 -- 172.21.15.47:0/2756422703 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe90140890 con 0x7fbe900ad670 2022-06-29T15:42:35.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.860+0000 7fbe867fc700 1 -- 172.21.15.47:0/2756422703 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fbe88012040 con 0x7fbe900ad670 2022-06-29T15:42:35.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.860+0000 7fbe867fc700 1 --2- 172.21.15.47:0/2756422703 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbe78053b00 0x7fbe78055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:35.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.860+0000 7fbe867fc700 1 -- 172.21.15.47:0/2756422703 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fbe8806c6d0 con 0x7fbe900ad670 2022-06-29T15:42:35.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.860+0000 7fbe952b9700 1 --2- 172.21.15.47:0/2756422703 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbe78053b00 0x7fbe78055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:35.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.860+0000 7fbe952b9700 1 --2- 172.21.15.47:0/2756422703 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbe78053b00 0x7fbe78055fc0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fbe8c005000 tx=0x7fbe901438c0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:35.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.868+0000 7fbe867fc700 1 -- 172.21.15.47:0/2756422703 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbe90140890 con 0x7fbe900ad670 2022-06-29T15:42:35.997 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.996+0000 7fbe962bb700 1 -- 172.21.15.47:0/2756422703 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe90140890 con 0x7fbe900ad670 2022-06-29T15:42:35.998 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.996+0000 7fbe867fc700 1 -- 172.21.15.47:0/2756422703 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+16299 (secure 0 0 0) 0x7fbe90140890 con 0x7fbe900ad670 2022-06-29T15:42:35.998 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:42:35.999 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":26,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:42:34.474438+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:42:28.400446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-06-29T15:42:31.167705+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:42:36.001 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.996+0000 7fbe962bb700 1 -- 172.21.15.47:0/2756422703 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbe78053b00 msgr2=0x7fbe78055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:36.001 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.996+0000 7fbe962bb700 1 --2- 172.21.15.47:0/2756422703 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbe78053b00 0x7fbe78055fc0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fbe8c005000 tx=0x7fbe901438c0).stop 2022-06-29T15:42:36.001 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.996+0000 7fbe962bb700 1 -- 172.21.15.47:0/2756422703 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe900ad670 msgr2=0x7fbe9013b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:36.001 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.996+0000 7fbe962bb700 1 --2- 172.21.15.47:0/2756422703 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe900ad670 0x7fbe9013b3f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7fbe8800b000 tx=0x7fbe88008040).stop 2022-06-29T15:42:36.002 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.996+0000 7fbe962bb700 1 -- 172.21.15.47:0/2756422703 shutdown_connections 2022-06-29T15:42:36.002 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.996+0000 7fbe962bb700 1 --2- 172.21.15.47:0/2756422703 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe900ad670 0x7fbe9013b3f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:36.002 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.996+0000 7fbe962bb700 1 --2- 172.21.15.47:0/2756422703 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbe90146cb0 0x7fbe901401b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:36.002 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.996+0000 7fbe962bb700 1 --2- 172.21.15.47:0/2756422703 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbe78053b00 0x7fbe78055fc0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:36.003 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.996+0000 7fbe962bb700 1 --2- 172.21.15.47:0/2756422703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe74004500 0x7fbe9013de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:36.003 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.996+0000 7fbe962bb700 1 -- 172.21.15.47:0/2756422703 >> 172.21.15.47:0/2756422703 conn(0x7fbe90060e00 msgr2=0x7fbe90142940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:36.003 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.996+0000 7fbe962bb700 1 -- 172.21.15.47:0/2756422703 shutdown_connections 2022-06-29T15:42:36.003 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:35.996+0000 7fbe962bb700 1 -- 172.21.15.47:0/2756422703 wait complete. 2022-06-29T15:42:36.012 INFO:tasks.ceph.ceph_manager.ceph:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-29T15:42:16.971431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-29T15:42:25.290884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-29T15:42:28.400446+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2022-06-29T15:42:31.167705+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-29T15:42:36.013 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-29T15:42:36.122 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.120+0000 7f18cd88d700 1 Processor -- start 2022-06-29T15:42:36.122 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.120+0000 7f18cd88d700 1 -- start start 2022-06-29T15:42:36.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.120+0000 7f18cd88d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f18c8141580 0x7f18c81480d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:36.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.120+0000 7f18cd88d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f18c80afef0 0x7f18c80b5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:36.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.120+0000 7f18cd88d700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18c80b0720 con 0x7f18c8060560 2022-06-29T15:42:36.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.120+0000 7f18cd88d700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18c81451f0 con 0x7f18c8141580 2022-06-29T15:42:36.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.120+0000 7f18cd88d700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18c81476f0 con 0x7f18c80afef0 2022-06-29T15:42:36.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.120+0000 7f18c6ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f18c8141580 0x7f18c81480d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:36.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.120+0000 7f18c77fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f18c8060560 0x7f18c814ab40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59530/0 (socket says 172.21.15.47:59530) 2022-06-29T15:42:36.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.120+0000 7f18c77fe700 1 -- 172.21.15.47:0/182673167 learned_addr learned my addr 172.21.15.47:0/182673167 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:36.127 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.120+0000 7f18c67fc700 1 --2- 172.21.15.47:0/182673167 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f18c80afef0 0x7f18c80b5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:36.127 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.120+0000 7f18c5ffb700 1 -- 172.21.15.47:0/182673167 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3669560836 0 0) 0x7f18c80b0720 con 0x7f18c8060560 2022-06-29T15:42:36.127 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.120+0000 7f18c5ffb700 1 -- 172.21.15.47:0/182673167 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18ac003260 con 0x7f18c8060560 2022-06-29T15:42:36.127 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.120+0000 7f18c6ffd700 1 -- 172.21.15.47:0/182673167 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f18c80afef0 msgr2=0x7f18c80b5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:36.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.120+0000 7f18c6ffd700 1 --2- 172.21.15.47:0/182673167 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f18c80afef0 0x7f18c80b5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:36.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.120+0000 7f18c6ffd700 1 -- 172.21.15.47:0/182673167 >> v1:172.21.15.47:6789/0 conn(0x7f18c8060560 legacy=0x7f18c814ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:36.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.120+0000 7f18c6ffd700 1 -- 172.21.15.47:0/182673167 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c8147990 con 0x7f18c8141580 2022-06-29T15:42:36.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.120+0000 7f18c67fc700 1 --2- 172.21.15.47:0/182673167 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f18c80afef0 0x7f18c80b5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:42:36.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.120+0000 7f18c6ffd700 1 --2- 172.21.15.47:0/182673167 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18c8141580 0x7f18c81480d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f18b80032f0 tx=0x7f18b80091d0).ready entity=mon.0 client_cookie=6f3ee7f558e367d2 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:36.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.120+0000 7f18c5ffb700 1 -- 172.21.15.47:0/182673167 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18b800b600 con 0x7f18c8141580 2022-06-29T15:42:36.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.120+0000 7f18c5ffb700 1 -- 172.21.15.47:0/182673167 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18b800b760 con 0x7f18c8141580 2022-06-29T15:42:36.130 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.120+0000 7f18c5ffb700 1 -- 172.21.15.47:0/182673167 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18b800b8d0 con 0x7f18c8141580 2022-06-29T15:42:36.134 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.124+0000 7f18cd88d700 1 -- 172.21.15.47:0/182673167 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18c8141580 msgr2=0x7f18c81480d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:36.134 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.124+0000 7f18cd88d700 1 --2- 172.21.15.47:0/182673167 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18c8141580 0x7f18c81480d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f18b80032f0 tx=0x7f18b80091d0).stop 2022-06-29T15:42:36.134 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.124+0000 7f18cd88d700 1 -- 172.21.15.47:0/182673167 shutdown_connections 2022-06-29T15:42:36.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.124+0000 7f18cd88d700 1 --2- 172.21.15.47:0/182673167 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f18c80afef0 0x7f18c80b5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:36.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.124+0000 7f18cd88d700 1 --2- 172.21.15.47:0/182673167 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18c8141580 0x7f18c81480d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:36.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.124+0000 7f18cd88d700 1 -- 172.21.15.47:0/182673167 >> 172.21.15.47:0/182673167 conn(0x7f18c80626f0 msgr2=0x7f18c8062ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:36.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.124+0000 7f18cd88d700 1 -- 172.21.15.47:0/182673167 shutdown_connections 2022-06-29T15:42:36.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.124+0000 7f18cd88d700 1 -- 172.21.15.47:0/182673167 wait complete. 2022-06-29T15:42:36.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.124+0000 7f18cd88d700 1 Processor -- start 2022-06-29T15:42:36.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.124+0000 7f18cd88d700 1 -- start start 2022-06-29T15:42:36.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.124+0000 7f18cd88d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18c8060560 0x7f18c80ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:36.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.124+0000 7f18cd88d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f18c80afef0 0x7f18c814b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:36.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.124+0000 7f18cd88d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18c8141580 0x7f18c80ade10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:36.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.124+0000 7f18cd88d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c80ad9c0 con 0x7f18c8060560 2022-06-29T15:42:36.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.124+0000 7f18cd88d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c80ad7a0 con 0x7f18c8141580 2022-06-29T15:42:36.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.124+0000 7f18cd88d700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18c80ac810 con 0x7f18c80afef0 2022-06-29T15:42:36.138 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.128+0000 7f18c6ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18c8060560 0x7f18c80ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:36.138 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.128+0000 7f18c6ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18c8060560 0x7f18c80ad2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45746/0 (socket says 172.21.15.47:45746) 2022-06-29T15:42:36.138 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.128+0000 7f18c6ffd700 1 -- 172.21.15.47:0/3069603474 learned_addr learned my addr 172.21.15.47:0/3069603474 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:36.138 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.128+0000 7f18c67fc700 1 --2- 172.21.15.47:0/3069603474 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f18c80afef0 0x7f18c814b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:36.139 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.128+0000 7f18c77fe700 1 --2- 172.21.15.47:0/3069603474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18c8141580 0x7f18c80ade10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:36.139 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.128+0000 7f18c6ffd700 1 -- 172.21.15.47:0/3069603474 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f18c80afef0 msgr2=0x7f18c814b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:36.139 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.128+0000 7f18c6ffd700 1 --2- 172.21.15.47:0/3069603474 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f18c80afef0 0x7f18c814b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:36.139 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.128+0000 7f18c6ffd700 1 -- 172.21.15.47:0/3069603474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18c8141580 msgr2=0x7f18c80ade10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:36.139 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.128+0000 7f18c6ffd700 1 --2- 172.21.15.47:0/3069603474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18c8141580 0x7f18c80ade10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:36.140 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.128+0000 7f18c6ffd700 1 -- 172.21.15.47:0/3069603474 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18bc00b180 con 0x7f18c8060560 2022-06-29T15:42:36.140 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.128+0000 7f18c6ffd700 1 --2- 172.21.15.47:0/3069603474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18c8060560 0x7f18c80ad2b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f18b80032f0 tx=0x7f18b8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:36.140 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.128+0000 7f18b7fff700 1 -- 172.21.15.47:0/3069603474 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18b800b8d0 con 0x7f18c8060560 2022-06-29T15:42:36.140 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.128+0000 7f18cd88d700 1 -- 172.21.15.47:0/3069603474 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18b8009470 con 0x7f18c8060560 2022-06-29T15:42:36.141 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.128+0000 7f18cd88d700 1 -- 172.21.15.47:0/3069603474 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18c8146510 con 0x7f18c8060560 2022-06-29T15:42:36.141 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.128+0000 7f18b7fff700 1 -- 172.21.15.47:0/3069603474 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18b8021630 con 0x7f18c8060560 2022-06-29T15:42:36.142 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.128+0000 7f18b7fff700 1 -- 172.21.15.47:0/3069603474 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18b800b600 con 0x7f18c8060560 2022-06-29T15:42:36.142 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.128+0000 7f18cd88d700 1 -- 172.21.15.47:0/3069603474 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18a8003410 con 0x7f18c8060560 2022-06-29T15:42:36.142 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.128+0000 7f18b7fff700 1 -- 172.21.15.47:0/3069603474 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f18b8028070 con 0x7f18c8060560 2022-06-29T15:42:36.142 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.132+0000 7f18b7fff700 1 --2- 172.21.15.47:0/3069603474 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f18a4053b00 0x7f18a4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:36.142 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.132+0000 7f18b7fff700 1 -- 172.21.15.47:0/3069603474 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f18b806d450 con 0x7f18c8060560 2022-06-29T15:42:36.143 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.132+0000 7f18c67fc700 1 --2- 172.21.15.47:0/3069603474 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f18a4053b00 0x7f18a4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:36.143 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.132+0000 7f18c67fc700 1 --2- 172.21.15.47:0/3069603474 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f18a4053b00 0x7f18a4055fc0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f18bc00b000 tx=0x7f18bc006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:36.144 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.136+0000 7f18b7fff700 1 -- 172.21.15.47:0/3069603474 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f18b8044170 con 0x7f18c8060560 2022-06-29T15:42:36.267 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.264+0000 7f18cd88d700 1 -- 172.21.15.47:0/3069603474 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f18a8003c40 con 0x7f18c8060560 2022-06-29T15:42:36.268 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.264+0000 7f18b7fff700 1 -- 172.21.15.47:0/3069603474 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v26) v1 ==== 110+0+10 (secure 0 0 0) 0x7f18b80150b0 con 0x7f18c8060560 2022-06-29T15:42:36.268 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2022-06-29T15:42:36.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.268+0000 7f18cd88d700 1 -- 172.21.15.47:0/3069603474 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f18a4053b00 msgr2=0x7f18a4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:36.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.268+0000 7f18cd88d700 1 --2- 172.21.15.47:0/3069603474 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f18a4053b00 0x7f18a4055fc0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f18bc00b000 tx=0x7f18bc006df0).stop 2022-06-29T15:42:36.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.268+0000 7f18cd88d700 1 -- 172.21.15.47:0/3069603474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18c8060560 msgr2=0x7f18c80ad2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:36.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.268+0000 7f18cd88d700 1 --2- 172.21.15.47:0/3069603474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18c8060560 0x7f18c80ad2b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f18b80032f0 tx=0x7f18b8016000).stop 2022-06-29T15:42:36.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.268+0000 7f18cd88d700 1 -- 172.21.15.47:0/3069603474 shutdown_connections 2022-06-29T15:42:36.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.268+0000 7f18cd88d700 1 --2- 172.21.15.47:0/3069603474 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f18a4053b00 0x7f18a4055fc0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:36.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.268+0000 7f18cd88d700 1 --2- 172.21.15.47:0/3069603474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18c8141580 0x7f18c80ade10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:36.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.268+0000 7f18cd88d700 1 --2- 172.21.15.47:0/3069603474 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f18c80afef0 0x7f18c814b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:36.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.268+0000 7f18cd88d700 1 --2- 172.21.15.47:0/3069603474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18c8060560 0x7f18c80ad2b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:36.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.268+0000 7f18cd88d700 1 -- 172.21.15.47:0/3069603474 >> 172.21.15.47:0/3069603474 conn(0x7f18c80626f0 msgr2=0x7f18c8140870 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:36.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.268+0000 7f18cd88d700 1 -- 172.21.15.47:0/3069603474 shutdown_connections 2022-06-29T15:42:36.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.268+0000 7f18cd88d700 1 -- 172.21.15.47:0/3069603474 wait complete. 2022-06-29T15:42:36.287 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-29T15:42:36.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.388+0000 7f7ee67c6700 1 Processor -- start 2022-06-29T15:42:36.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.388+0000 7f7ee67c6700 1 -- start start 2022-06-29T15:42:36.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.392+0000 7f7ee67c6700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7ee00a9770 0x7f7ee00b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:36.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.392+0000 7f7ee67c6700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ee00b1ac0 con 0x7f7ee0095e30 2022-06-29T15:42:36.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.392+0000 7f7ee67c6700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ee0095c70 con 0x7f7ee00b2930 2022-06-29T15:42:36.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.392+0000 7f7ee67c6700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee005cc20 con 0x7f7ee00a9770 2022-06-29T15:42:36.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.392+0000 7f7ee4d63700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7ee0095e30 0x7f7ee005c510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59544/0 (socket says 172.21.15.47:59544) 2022-06-29T15:42:36.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.392+0000 7f7ee4d63700 1 -- 172.21.15.47:0/3834755941 learned_addr learned my addr 172.21.15.47:0/3834755941 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:36.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.392+0000 7f7edf7fe700 1 --2- 172.21.15.47:0/3834755941 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7ee00a9770 0x7f7ee00b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:36.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.392+0000 7f7edeffd700 1 -- 172.21.15.47:0/3834755941 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2120921146 0 0) 0x7f7ee00b1ac0 con 0x7f7ee0095e30 2022-06-29T15:42:36.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.392+0000 7f7edeffd700 1 -- 172.21.15.47:0/3834755941 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ec0003260 con 0x7f7ee0095e30 2022-06-29T15:42:36.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.392+0000 7f7edeffd700 1 -- 172.21.15.47:0/3834755941 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3244981519 0 0) 0x7f7ed0001490 con 0x7f7ee00b2930 2022-06-29T15:42:36.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.392+0000 7f7edeffd700 1 -- 172.21.15.47:0/3834755941 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ee00b1ac0 con 0x7f7ee00b2930 2022-06-29T15:42:36.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.392+0000 7f7edeffd700 1 -- 172.21.15.47:0/3834755941 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3440265229 0 0) 0x7f7ec0003260 con 0x7f7ee0095e30 2022-06-29T15:42:36.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.392+0000 7f7edeffd700 1 -- 172.21.15.47:0/3834755941 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7ee00a9770 msgr2=0x7f7ee00b2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:36.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.392+0000 7f7edeffd700 1 --2- 172.21.15.47:0/3834755941 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7ee00a9770 0x7f7ee00b2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:36.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.392+0000 7f7edeffd700 1 -- 172.21.15.47:0/3834755941 >> v1:172.21.15.64:6789/0 conn(0x7f7ee00b2930 legacy=0x7f7ee0097a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:36.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.392+0000 7f7edeffd700 1 -- 172.21.15.47:0/3834755941 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee00965b0 con 0x7f7ee0095e30 2022-06-29T15:42:36.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.392+0000 7f7edeffd700 1 -- 172.21.15.47:0/3834755941 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7f7ed8001660 con 0x7f7ee0095e30 2022-06-29T15:42:36.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.392+0000 7f7edeffd700 1 -- 172.21.15.47:0/3834755941 >> v1:172.21.15.47:6789/0 conn(0x7f7ee0095e30 legacy=0x7f7ee005c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:36.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.392+0000 7f7edeffd700 1 --2- 172.21.15.47:0/3834755941 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ec0003910 0x7f7ec0003d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:36.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.392+0000 7f7edeffd700 1 --2- 172.21.15.47:0/3834755941 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7ee00a9770 0x7f7ec0004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:36.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.392+0000 7f7edeffd700 1 --2- 172.21.15.47:0/3834755941 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ec00049f0 0x7f7ec0004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:36.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.392+0000 7f7edeffd700 1 -- 172.21.15.47:0/3834755941 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee005cc20 con 0x7f7ec00049f0 2022-06-29T15:42:36.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.392+0000 7f7edeffd700 1 -- 172.21.15.47:0/3834755941 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec000b780 con 0x7f7ec0003910 2022-06-29T15:42:36.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.392+0000 7f7edeffd700 1 -- 172.21.15.47:0/3834755941 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec000b940 con 0x7f7ee00a9770 2022-06-29T15:42:36.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.392+0000 7f7edffff700 1 --2- 172.21.15.47:0/3834755941 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ec0003910 0x7f7ec0003d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:36.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.392+0000 7f7ee4d63700 1 --2- 172.21.15.47:0/3834755941 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ec00049f0 0x7f7ec0004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:36.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.392+0000 7f7edf7fe700 1 --2- 172.21.15.47:0/3834755941 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7ee00a9770 0x7f7ec0004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:36.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.396+0000 7f7edf7fe700 1 -- 172.21.15.47:0/3834755941 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ec0003910 msgr2=0x7f7ec0003d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:36.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.396+0000 7f7edf7fe700 1 --2- 172.21.15.47:0/3834755941 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ec0003910 0x7f7ec0003d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:36.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.396+0000 7f7edf7fe700 1 -- 172.21.15.47:0/3834755941 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ec00049f0 msgr2=0x7f7ec0004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:36.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.396+0000 7f7edf7fe700 1 --2- 172.21.15.47:0/3834755941 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ec00049f0 0x7f7ec0004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:36.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.396+0000 7f7edf7fe700 1 -- 172.21.15.47:0/3834755941 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec0003260 con 0x7f7ee00a9770 2022-06-29T15:42:36.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.396+0000 7f7edf7fe700 1 --2- 172.21.15.47:0/3834755941 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7ee00a9770 0x7f7ec0004500 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f7ec800c000 tx=0x7f7ec8006df0).ready entity=mon.2 client_cookie=dde19fcb9801090e server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:36.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.396+0000 7f7edeffd700 1 -- 172.21.15.47:0/3834755941 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ec800e800 con 0x7f7ee00a9770 2022-06-29T15:42:36.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.396+0000 7f7edeffd700 1 -- 172.21.15.47:0/3834755941 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ec800e960 con 0x7f7ee00a9770 2022-06-29T15:42:36.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.396+0000 7f7edeffd700 1 -- 172.21.15.47:0/3834755941 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ec800ecc0 con 0x7f7ee00a9770 2022-06-29T15:42:36.409 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.396+0000 7f7ee67c6700 1 -- 172.21.15.47:0/3834755941 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7ee00a9770 msgr2=0x7f7ec0004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:36.410 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.396+0000 7f7ee67c6700 1 --2- 172.21.15.47:0/3834755941 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7ee00a9770 0x7f7ec0004500 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f7ec800c000 tx=0x7f7ec8006df0).stop 2022-06-29T15:42:36.410 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.396+0000 7f7ee4d63700 1 -- 172.21.15.47:0/3834755941 reap_dead start 2022-06-29T15:42:36.410 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.396+0000 7f7ee67c6700 1 -- 172.21.15.47:0/3834755941 shutdown_connections 2022-06-29T15:42:36.410 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.396+0000 7f7ee67c6700 1 -- 172.21.15.47:0/3834755941 >> 172.21.15.47:0/3834755941 conn(0x7f7ee005f0b0 msgr2=0x7f7ee005f490 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:36.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.396+0000 7f7ee67c6700 1 -- 172.21.15.47:0/3834755941 shutdown_connections 2022-06-29T15:42:36.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.396+0000 7f7ee67c6700 1 -- 172.21.15.47:0/3834755941 wait complete. 2022-06-29T15:42:36.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.396+0000 7f7ee67c6700 1 Processor -- start 2022-06-29T15:42:36.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.396+0000 7f7ee67c6700 1 -- start start 2022-06-29T15:42:36.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.400+0000 7f7ee67c6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ee00a9770 0x7f7ee0134bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:36.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.400+0000 7f7ee67c6700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7ee00ac5f0 0x7f7ee013f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:36.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.400+0000 7f7ee67c6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ee013fa90 0x7f7ee013ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:36.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.400+0000 7f7ee67c6700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee00a9d10 con 0x7f7ee013fa90 2022-06-29T15:42:36.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.400+0000 7f7ee67c6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee00a9fb0 con 0x7f7ee00a9770 2022-06-29T15:42:36.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.400+0000 7f7ee67c6700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee00aa250 con 0x7f7ee00ac5f0 2022-06-29T15:42:36.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.400+0000 7f7ee4d63700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ee013fa90 0x7f7ee013ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:36.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.400+0000 7f7ee4d63700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ee013fa90 0x7f7ee013ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45770/0 (socket says 172.21.15.47:45770) 2022-06-29T15:42:36.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.400+0000 7f7ee4d63700 1 -- 172.21.15.47:0/154204071 learned_addr learned my addr 172.21.15.47:0/154204071 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:36.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.400+0000 7f7edf7fe700 1 --2- 172.21.15.47:0/154204071 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7ee00ac5f0 0x7f7ee013f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:36.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.400+0000 7f7edffff700 1 --2- 172.21.15.47:0/154204071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ee00a9770 0x7f7ee0134bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:36.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.400+0000 7f7ee4d63700 1 -- 172.21.15.47:0/154204071 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7ee00ac5f0 msgr2=0x7f7ee013f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:36.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.400+0000 7f7ee4d63700 1 --2- 172.21.15.47:0/154204071 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7ee00ac5f0 0x7f7ee013f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:36.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.400+0000 7f7ee4d63700 1 -- 172.21.15.47:0/154204071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ee00a9770 msgr2=0x7f7ee0134bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:36.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.400+0000 7f7ee4d63700 1 --2- 172.21.15.47:0/154204071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ee00a9770 0x7f7ee0134bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:36.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.400+0000 7f7ee4d63700 1 -- 172.21.15.47:0/154204071 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee00aa4f0 con 0x7f7ee013fa90 2022-06-29T15:42:36.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.400+0000 7f7ee4d63700 1 --2- 172.21.15.47:0/154204071 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ee013fa90 0x7f7ee013ff10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f7ed8000b60 tx=0x7f7ee00a9c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:36.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.400+0000 7f7edd7fa700 1 -- 172.21.15.47:0/154204071 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ed8012540 con 0x7f7ee013fa90 2022-06-29T15:42:36.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.400+0000 7f7edd7fa700 1 -- 172.21.15.47:0/154204071 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ed80126a0 con 0x7f7ee013fa90 2022-06-29T15:42:36.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.400+0000 7f7ee67c6700 1 -- 172.21.15.47:0/154204071 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ee00aa790 con 0x7f7ee013fa90 2022-06-29T15:42:36.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.400+0000 7f7edd7fa700 1 -- 172.21.15.47:0/154204071 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ed8012a00 con 0x7f7ee013fa90 2022-06-29T15:42:36.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.400+0000 7f7ee67c6700 1 -- 172.21.15.47:0/154204071 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ee00a91f0 con 0x7f7ee013fa90 2022-06-29T15:42:36.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.400+0000 7f7ee67c6700 1 -- 172.21.15.47:0/154204071 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ee00573d0 con 0x7f7ee013fa90 2022-06-29T15:42:36.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.404+0000 7f7edd7fa700 1 -- 172.21.15.47:0/154204071 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7ed801b030 con 0x7f7ee013fa90 2022-06-29T15:42:36.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.404+0000 7f7edd7fa700 1 --2- 172.21.15.47:0/154204071 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7ec4053b00 0x7f7ec4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:36.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.404+0000 7f7edd7fa700 1 -- 172.21.15.47:0/154204071 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f7ed806b800 con 0x7f7ee013fa90 2022-06-29T15:42:36.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.404+0000 7f7edffff700 1 --2- 172.21.15.47:0/154204071 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7ec4053b00 0x7f7ec4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:36.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.404+0000 7f7edffff700 1 --2- 172.21.15.47:0/154204071 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7ec4053b00 0x7f7ec4055fc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f7ed0003000 tx=0x7f7ed0007d80).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:36.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.408+0000 7f7edd7fa700 1 -- 172.21.15.47:0/154204071 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7ed80425c0 con 0x7f7ee013fa90 2022-06-29T15:42:36.530 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.528+0000 7f7ee67c6700 1 -- 172.21.15.47:0/154204071 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7ee0044710 con 0x7f7ee013fa90 2022-06-29T15:42:36.531 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.528+0000 7f7edd7fa700 1 -- 172.21.15.47:0/154204071 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7ed8019030 con 0x7f7ee013fa90 2022-06-29T15:42:36.532 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:42:36.533 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.532+0000 7f7ee67c6700 1 -- 172.21.15.47:0/154204071 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7ec4053b00 msgr2=0x7f7ec4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:36.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.532+0000 7f7ee67c6700 1 --2- 172.21.15.47:0/154204071 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7ec4053b00 0x7f7ec4055fc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f7ed0003000 tx=0x7f7ed0007d80).stop 2022-06-29T15:42:36.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.532+0000 7f7ee67c6700 1 -- 172.21.15.47:0/154204071 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ee013fa90 msgr2=0x7f7ee013ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:36.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.532+0000 7f7ee67c6700 1 --2- 172.21.15.47:0/154204071 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ee013fa90 0x7f7ee013ff10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f7ed8000b60 tx=0x7f7ee00a9c40).stop 2022-06-29T15:42:36.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.532+0000 7f7ee67c6700 1 -- 172.21.15.47:0/154204071 shutdown_connections 2022-06-29T15:42:36.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.532+0000 7f7ee67c6700 1 --2- 172.21.15.47:0/154204071 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ee013fa90 0x7f7ee013ff10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:36.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.532+0000 7f7ee67c6700 1 --2- 172.21.15.47:0/154204071 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7ee00ac5f0 0x7f7ee013f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:36.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.532+0000 7f7ee67c6700 1 --2- 172.21.15.47:0/154204071 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7ec4053b00 0x7f7ec4055fc0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:36.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.532+0000 7f7ee67c6700 1 --2- 172.21.15.47:0/154204071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ee00a9770 0x7f7ee0134bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:36.536 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.532+0000 7f7ee67c6700 1 -- 172.21.15.47:0/154204071 >> 172.21.15.47:0/154204071 conn(0x7f7ee005f0b0 msgr2=0x7f7ee0098310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:36.536 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.532+0000 7f7ee67c6700 1 -- 172.21.15.47:0/154204071 shutdown_connections 2022-06-29T15:42:36.536 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.532+0000 7f7ee67c6700 1 -- 172.21.15.47:0/154204071 wait complete. 2022-06-29T15:42:36.545 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-29T15:42:36.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.652+0000 7f3f174f2700 1 Processor -- start 2022-06-29T15:42:36.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.652+0000 7f3f174f2700 1 -- start start 2022-06-29T15:42:36.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.652+0000 7f3f174f2700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3f100b58d0 0x7f3f100b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:36.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.652+0000 7f3f174f2700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3f1014b4c0 0x7f3f100aa130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:36.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.652+0000 7f3f174f2700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f10060790 con 0x7f3f100b1610 2022-06-29T15:42:36.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.652+0000 7f3f174f2700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f100a99e0 con 0x7f3f100b58d0 2022-06-29T15:42:36.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.652+0000 7f3f174f2700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f1014ad80 con 0x7f3f1014b4c0 2022-06-29T15:42:36.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.652+0000 7f3f164f0700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3f100b58d0 0x7f3f100b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:36.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.652+0000 7f3f164f0700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3f100b58d0 0x7f3f100b5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45774/0 (socket says 172.21.15.47:45774) 2022-06-29T15:42:36.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.652+0000 7f3f16cf1700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3f1014b4c0 0x7f3f100aa130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:36.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.652+0000 7f3f15cef700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f3f100b1610 0x7f3f101475f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:38892/0 (socket says 172.21.15.47:38892) 2022-06-29T15:42:36.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.652+0000 7f3f164f0700 1 -- 172.21.15.47:0/1384781215 learned_addr learned my addr 172.21.15.47:0/1384781215 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:36.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.652+0000 7f3f154ee700 1 -- 172.21.15.47:0/1384781215 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 977046911 0 0) 0x7f3f10060790 con 0x7f3f100b1610 2022-06-29T15:42:36.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.652+0000 7f3f154ee700 1 -- 172.21.15.47:0/1384781215 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3efc003260 con 0x7f3f100b1610 2022-06-29T15:42:36.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.652+0000 7f3f16cf1700 1 -- 172.21.15.47:0/1384781215 >> v2:172.21.15.47:3300/0 conn(0x7f3f100b58d0 msgr2=0x7f3f100b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:36.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.652+0000 7f3f16cf1700 1 --2- 172.21.15.47:0/1384781215 >> v2:172.21.15.47:3300/0 conn(0x7f3f100b58d0 0x7f3f100b5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:36.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.652+0000 7f3f16cf1700 1 -- 172.21.15.47:0/1384781215 >> v1:172.21.15.64:6789/0 conn(0x7f3f100b1610 legacy=0x7f3f101475f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:36.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.652+0000 7f3f16cf1700 1 -- 172.21.15.47:0/1384781215 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f1014b020 con 0x7f3f1014b4c0 2022-06-29T15:42:36.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.652+0000 7f3f164f0700 1 --2- 172.21.15.47:0/1384781215 >> v2:172.21.15.47:3300/0 conn(0x7f3f100b58d0 0x7f3f100b5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:42:36.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.656+0000 7f3f16cf1700 1 --2- 172.21.15.47:0/1384781215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f1014b4c0 0x7f3f100aa130 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f3f08002f70 tx=0x7f3f08006df0).ready entity=mon.1 client_cookie=45ef8263c6f5e998 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:36.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.656+0000 7f3f154ee700 1 -- 172.21.15.47:0/1384781215 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f0800d4f0 con 0x7f3f1014b4c0 2022-06-29T15:42:36.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.656+0000 7f3f154ee700 1 -- 172.21.15.47:0/1384781215 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f08000e90 con 0x7f3f1014b4c0 2022-06-29T15:42:36.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.656+0000 7f3f154ee700 1 -- 172.21.15.47:0/1384781215 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f0800d840 con 0x7f3f1014b4c0 2022-06-29T15:42:36.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.656+0000 7f3f174f2700 1 -- 172.21.15.47:0/1384781215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f1014b4c0 msgr2=0x7f3f100aa130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:36.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.656+0000 7f3f174f2700 1 --2- 172.21.15.47:0/1384781215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f1014b4c0 0x7f3f100aa130 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f3f08002f70 tx=0x7f3f08006df0).stop 2022-06-29T15:42:36.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.656+0000 7f3f174f2700 1 -- 172.21.15.47:0/1384781215 shutdown_connections 2022-06-29T15:42:36.666 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.656+0000 7f3f174f2700 1 --2- 172.21.15.47:0/1384781215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f1014b4c0 0x7f3f100aa130 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:36.666 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.656+0000 7f3f174f2700 1 --2- 172.21.15.47:0/1384781215 >> v2:172.21.15.47:3300/0 conn(0x7f3f100b58d0 0x7f3f100b5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:36.666 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.656+0000 7f3f174f2700 1 -- 172.21.15.47:0/1384781215 >> 172.21.15.47:0/1384781215 conn(0x7f3f10062560 msgr2=0x7f3f10062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:36.666 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.656+0000 7f3f174f2700 1 -- 172.21.15.47:0/1384781215 shutdown_connections 2022-06-29T15:42:36.667 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.656+0000 7f3f174f2700 1 -- 172.21.15.47:0/1384781215 wait complete. 2022-06-29T15:42:36.667 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.660+0000 7f3f174f2700 1 Processor -- start 2022-06-29T15:42:36.667 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.660+0000 7f3f174f2700 1 -- start start 2022-06-29T15:42:36.667 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.660+0000 7f3f174f2700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3f100b1610 0x7f3f1014cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:36.667 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.660+0000 7f3f174f2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f100b58d0 0x7f3f100ad080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:36.668 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.660+0000 7f3f174f2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f1014b4c0 0x7f3f1015a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:36.668 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.660+0000 7f3f174f2700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f100af280 con 0x7f3f1014b4c0 2022-06-29T15:42:36.668 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.660+0000 7f3f174f2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f100accc0 con 0x7f3f100b58d0 2022-06-29T15:42:36.668 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.660+0000 7f3f174f2700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1014d450 con 0x7f3f100b1610 2022-06-29T15:42:36.669 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.660+0000 7f3f16cf1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f1014b4c0 0x7f3f1015a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:36.669 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.660+0000 7f3f16cf1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f1014b4c0 0x7f3f1015a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45784/0 (socket says 172.21.15.47:45784) 2022-06-29T15:42:36.669 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.660+0000 7f3f16cf1700 1 -- 172.21.15.47:0/1280056262 learned_addr learned my addr 172.21.15.47:0/1280056262 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:36.669 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.660+0000 7f3f15cef700 1 --2- 172.21.15.47:0/1280056262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f100b58d0 0x7f3f100ad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:36.669 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.660+0000 7f3f164f0700 1 --2- 172.21.15.47:0/1280056262 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3f100b1610 0x7f3f1014cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:36.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.660+0000 7f3f16cf1700 1 -- 172.21.15.47:0/1280056262 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3f100b1610 msgr2=0x7f3f1014cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:36.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.660+0000 7f3f16cf1700 1 --2- 172.21.15.47:0/1280056262 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3f100b1610 0x7f3f1014cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:36.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.660+0000 7f3f16cf1700 1 -- 172.21.15.47:0/1280056262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f100b58d0 msgr2=0x7f3f100ad080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:36.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.660+0000 7f3f16cf1700 1 --2- 172.21.15.47:0/1280056262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f100b58d0 0x7f3f100ad080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:36.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.660+0000 7f3f16cf1700 1 -- 172.21.15.47:0/1280056262 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f0400b1e0 con 0x7f3f1014b4c0 2022-06-29T15:42:36.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.660+0000 7f3f15cef700 1 --2- 172.21.15.47:0/1280056262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f100b58d0 0x7f3f100ad080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:42:36.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.660+0000 7f3f16cf1700 1 --2- 172.21.15.47:0/1280056262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f1014b4c0 0x7f3f1015a070 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f3f08002f70 tx=0x7f3f08014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:36.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.660+0000 7f3f0f7fe700 1 -- 172.21.15.47:0/1280056262 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f08016070 con 0x7f3f1014b4c0 2022-06-29T15:42:36.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.660+0000 7f3f0f7fe700 1 -- 172.21.15.47:0/1280056262 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f08013850 con 0x7f3f1014b4c0 2022-06-29T15:42:36.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.660+0000 7f3f174f2700 1 -- 172.21.15.47:0/1280056262 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f0800b180 con 0x7f3f1014b4c0 2022-06-29T15:42:36.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.660+0000 7f3f0f7fe700 1 -- 172.21.15.47:0/1280056262 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f08013bb0 con 0x7f3f1014b4c0 2022-06-29T15:42:36.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.664+0000 7f3f174f2700 1 -- 172.21.15.47:0/1280056262 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f1014c820 con 0x7f3f1014b4c0 2022-06-29T15:42:36.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.664+0000 7f3f0f7fe700 1 -- 172.21.15.47:0/1280056262 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3f08025030 con 0x7f3f1014b4c0 2022-06-29T15:42:36.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.664+0000 7f3f174f2700 1 -- 172.21.15.47:0/1280056262 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f100573d0 con 0x7f3f1014b4c0 2022-06-29T15:42:36.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.664+0000 7f3f0f7fe700 1 --2- 172.21.15.47:0/1280056262 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3ef0053950 0x7f3ef0055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:36.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.664+0000 7f3f0f7fe700 1 -- 172.21.15.47:0/1280056262 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f3f0806d070 con 0x7f3f1014b4c0 2022-06-29T15:42:36.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.664+0000 7f3f164f0700 1 --2- 172.21.15.47:0/1280056262 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3ef0053950 0x7f3ef0055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:36.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.664+0000 7f3f164f0700 1 --2- 172.21.15.47:0/1280056262 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3ef0053950 0x7f3ef0055e10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f3f0400b000 tx=0x7f3f04006eb0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:36.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.668+0000 7f3f0f7fe700 1 -- 172.21.15.47:0/1280056262 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3f08043d20 con 0x7f3f1014b4c0 2022-06-29T15:42:36.795 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.792+0000 7f3f174f2700 1 -- 172.21.15.47:0/1280056262 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3f10069e20 con 0x7f3f1014b4c0 2022-06-29T15:42:36.796 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.792+0000 7f3f0f7fe700 1 -- 172.21.15.47:0/1280056262 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+10 (secure 0 0 0) 0x7f3f08023020 con 0x7f3f1014b4c0 2022-06-29T15:42:36.796 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:42:36.798 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.796+0000 7f3f174f2700 1 -- 172.21.15.47:0/1280056262 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3ef0053950 msgr2=0x7f3ef0055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:36.798 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.796+0000 7f3f174f2700 1 --2- 172.21.15.47:0/1280056262 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3ef0053950 0x7f3ef0055e10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f3f0400b000 tx=0x7f3f04006eb0).stop 2022-06-29T15:42:36.798 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.796+0000 7f3f174f2700 1 -- 172.21.15.47:0/1280056262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f1014b4c0 msgr2=0x7f3f1015a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:36.798 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.796+0000 7f3f174f2700 1 --2- 172.21.15.47:0/1280056262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f1014b4c0 0x7f3f1015a070 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f3f08002f70 tx=0x7f3f08014000).stop 2022-06-29T15:42:36.799 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.796+0000 7f3f174f2700 1 -- 172.21.15.47:0/1280056262 shutdown_connections 2022-06-29T15:42:36.799 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.796+0000 7f3f174f2700 1 --2- 172.21.15.47:0/1280056262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f1014b4c0 0x7f3f1015a070 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:36.799 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.796+0000 7f3f174f2700 1 --2- 172.21.15.47:0/1280056262 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3ef0053950 0x7f3ef0055e10 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:36.799 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.796+0000 7f3f174f2700 1 --2- 172.21.15.47:0/1280056262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f100b58d0 0x7f3f100ad080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:36.800 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.796+0000 7f3f174f2700 1 --2- 172.21.15.47:0/1280056262 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3f100b1610 0x7f3f1014cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:36.800 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.796+0000 7f3f174f2700 1 -- 172.21.15.47:0/1280056262 >> 172.21.15.47:0/1280056262 conn(0x7f3f10062560 msgr2=0x7f3f100b0700 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:36.800 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.796+0000 7f3f174f2700 1 -- 172.21.15.47:0/1280056262 shutdown_connections 2022-06-29T15:42:36.800 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.796+0000 7f3f174f2700 1 -- 172.21.15.47:0/1280056262 wait complete. 2022-06-29T15:42:36.813 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-29T15:42:36.925 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.924+0000 7fea76364700 1 Processor -- start 2022-06-29T15:42:36.926 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.924+0000 7fea76364700 1 -- start start 2022-06-29T15:42:36.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.924+0000 7fea76364700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fea70144e60 0x7fea70148040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:36.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.924+0000 7fea76364700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea70148530 con 0x7fea70060a70 2022-06-29T15:42:36.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.924+0000 7fea76364700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea70060e50 con 0x7fea700a72b0 2022-06-29T15:42:36.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.924+0000 7fea76364700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea700a8120 con 0x7fea70144e60 2022-06-29T15:42:36.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.924+0000 7fea6ffff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fea70060a70 0x7fea70144750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59574/0 (socket says 172.21.15.47:59574) 2022-06-29T15:42:36.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.924+0000 7fea6ffff700 1 -- 172.21.15.47:0/1969876510 learned_addr learned my addr 172.21.15.47:0/1969876510 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:36.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.924+0000 7fea74901700 1 --2- 172.21.15.47:0/1969876510 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fea70144e60 0x7fea70148040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:36.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.924+0000 7fea6effd700 1 -- 172.21.15.47:0/1969876510 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 153895401 0 0) 0x7fea70148530 con 0x7fea70060a70 2022-06-29T15:42:36.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.924+0000 7fea6effd700 1 -- 172.21.15.47:0/1969876510 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea50003260 con 0x7fea70060a70 2022-06-29T15:42:36.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.924+0000 7fea6effd700 1 -- 172.21.15.47:0/1969876510 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2159444258 0 0) 0x7fea70060e50 con 0x7fea700a72b0 2022-06-29T15:42:36.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.924+0000 7fea6effd700 1 -- 172.21.15.47:0/1969876510 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea70148530 con 0x7fea700a72b0 2022-06-29T15:42:36.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.924+0000 7fea6effd700 1 -- 172.21.15.47:0/1969876510 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 151859943 0 0) 0x7fea50003260 con 0x7fea70060a70 2022-06-29T15:42:36.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.924+0000 7fea74901700 1 -- 172.21.15.47:0/1969876510 >> v1:172.21.15.64:6789/0 conn(0x7fea700a72b0 legacy=0x7fea70147930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:36.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.924+0000 7fea74901700 1 -- 172.21.15.47:0/1969876510 >> v1:172.21.15.47:6789/0 conn(0x7fea70060a70 legacy=0x7fea70144750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:36.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.924+0000 7fea74901700 1 -- 172.21.15.47:0/1969876510 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea700a8ac0 con 0x7fea70144e60 2022-06-29T15:42:36.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.924+0000 7fea74901700 1 --2- 172.21.15.47:0/1969876510 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fea70144e60 0x7fea70148040 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7fea58001110 tx=0x7fea60006df0).ready entity=mon.2 client_cookie=8da3fde85c57bb8a server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:36.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.924+0000 7fea6effd700 1 -- 172.21.15.47:0/1969876510 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea58001660 con 0x7fea70144e60 2022-06-29T15:42:36.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.924+0000 7fea6effd700 1 -- 172.21.15.47:0/1969876510 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea60000e90 con 0x7fea70144e60 2022-06-29T15:42:36.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.928+0000 7fea6effd700 1 -- 172.21.15.47:0/1969876510 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea60012460 con 0x7fea70144e60 2022-06-29T15:42:36.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.928+0000 7fea76364700 1 -- 172.21.15.47:0/1969876510 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fea70144e60 msgr2=0x7fea70148040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:36.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.928+0000 7fea76364700 1 --2- 172.21.15.47:0/1969876510 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fea70144e60 0x7fea70148040 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7fea58001110 tx=0x7fea60006df0).stop 2022-06-29T15:42:36.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.928+0000 7fea76364700 1 -- 172.21.15.47:0/1969876510 shutdown_connections 2022-06-29T15:42:36.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.928+0000 7fea76364700 1 --2- 172.21.15.47:0/1969876510 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fea70144e60 0x7fea70148040 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:36.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.928+0000 7fea76364700 1 -- 172.21.15.47:0/1969876510 >> 172.21.15.47:0/1969876510 conn(0x7fea700626f0 msgr2=0x7fea70062ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:36.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.928+0000 7fea76364700 1 -- 172.21.15.47:0/1969876510 shutdown_connections 2022-06-29T15:42:36.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.928+0000 7fea76364700 1 -- 172.21.15.47:0/1969876510 wait complete. 2022-06-29T15:42:36.938 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.928+0000 7fea76364700 1 Processor -- start 2022-06-29T15:42:36.938 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.928+0000 7fea76364700 1 -- start start 2022-06-29T15:42:36.938 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.928+0000 7fea76364700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea70060a70 0x7fea70146450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:36.938 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.928+0000 7fea76364700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fea700a72b0 0x7fea70159c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:36.938 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.928+0000 7fea76364700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea70144e60 0x7fea7015a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:36.939 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.928+0000 7fea76364700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea700aa480 con 0x7fea70144e60 2022-06-29T15:42:36.939 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.928+0000 7fea76364700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea70156830 con 0x7fea70060a70 2022-06-29T15:42:36.939 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.928+0000 7fea76364700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea70156a60 con 0x7fea700a72b0 2022-06-29T15:42:36.939 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.932+0000 7fea6ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea70060a70 0x7fea70146450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:36.940 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.932+0000 7fea74901700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea70144e60 0x7fea7015a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:36.940 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.932+0000 7fea6ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea70060a70 0x7fea70146450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:51498/0 (socket says 172.21.15.47:51498) 2022-06-29T15:42:36.940 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.932+0000 7fea6ffff700 1 -- 172.21.15.47:0/3626880323 learned_addr learned my addr 172.21.15.47:0/3626880323 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:36.940 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.932+0000 7fea6f7fe700 1 --2- 172.21.15.47:0/3626880323 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fea700a72b0 0x7fea70159c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:36.941 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.932+0000 7fea74901700 1 -- 172.21.15.47:0/3626880323 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fea700a72b0 msgr2=0x7fea70159c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:36.941 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.932+0000 7fea74901700 1 --2- 172.21.15.47:0/3626880323 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fea700a72b0 0x7fea70159c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:36.941 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.932+0000 7fea74901700 1 -- 172.21.15.47:0/3626880323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea70060a70 msgr2=0x7fea70146450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:36.941 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.932+0000 7fea74901700 1 --2- 172.21.15.47:0/3626880323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea70060a70 0x7fea70146450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:36.942 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.932+0000 7fea74901700 1 -- 172.21.15.47:0/3626880323 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea6000b180 con 0x7fea70144e60 2022-06-29T15:42:36.942 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.932+0000 7fea6ffff700 1 --2- 172.21.15.47:0/3626880323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea70060a70 0x7fea70146450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:42:36.942 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.932+0000 7fea74901700 1 --2- 172.21.15.47:0/3626880323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea70144e60 0x7fea7015a120 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fea58001110 tx=0x7fea60014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:36.942 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.932+0000 7fea6d7fa700 1 -- 172.21.15.47:0/3626880323 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea6000d8a0 con 0x7fea70144e60 2022-06-29T15:42:36.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.932+0000 7fea6d7fa700 1 -- 172.21.15.47:0/3626880323 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea6000da00 con 0x7fea70144e60 2022-06-29T15:42:36.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.932+0000 7fea76364700 1 -- 172.21.15.47:0/3626880323 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea70156d00 con 0x7fea70144e60 2022-06-29T15:42:36.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.932+0000 7fea6d7fa700 1 -- 172.21.15.47:0/3626880323 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea6000dd60 con 0x7fea70144e60 2022-06-29T15:42:36.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.932+0000 7fea76364700 1 -- 172.21.15.47:0/3626880323 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea70157440 con 0x7fea70144e60 2022-06-29T15:42:36.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.932+0000 7fea76364700 1 -- 172.21.15.47:0/3626880323 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea700573d0 con 0x7fea70144e60 2022-06-29T15:42:36.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.932+0000 7fea6d7fa700 1 -- 172.21.15.47:0/3626880323 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fea60025030 con 0x7fea70144e60 2022-06-29T15:42:36.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.936+0000 7fea6d7fa700 1 --2- 172.21.15.47:0/3626880323 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fea54053a70 0x7fea54055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:36.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.936+0000 7fea6ffff700 1 --2- 172.21.15.47:0/3626880323 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fea54053a70 0x7fea54055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:36.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.936+0000 7fea6d7fa700 1 -- 172.21.15.47:0/3626880323 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fea6006cad0 con 0x7fea70144e60 2022-06-29T15:42:36.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.936+0000 7fea6ffff700 1 --2- 172.21.15.47:0/3626880323 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fea54053a70 0x7fea54055f30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fea5800d000 tx=0x7fea58000bc0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:36.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:36.944+0000 7fea6d7fa700 1 -- 172.21.15.47:0/3626880323 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fea60043890 con 0x7fea70144e60 2022-06-29T15:42:37.068 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.064+0000 7fea76364700 1 -- 172.21.15.47:0/3626880323 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fea700675e0 con 0x7fea70144e60 2022-06-29T15:42:37.069 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.064+0000 7fea6d7fa700 1 -- 172.21.15.47:0/3626880323 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+10 (secure 0 0 0) 0x7fea60023020 con 0x7fea70144e60 2022-06-29T15:42:37.069 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:42:37.071 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.068+0000 7fea76364700 1 -- 172.21.15.47:0/3626880323 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fea54053a70 msgr2=0x7fea54055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:37.071 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.068+0000 7fea76364700 1 --2- 172.21.15.47:0/3626880323 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fea54053a70 0x7fea54055f30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fea5800d000 tx=0x7fea58000bc0).stop 2022-06-29T15:42:37.072 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.068+0000 7fea76364700 1 -- 172.21.15.47:0/3626880323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea70144e60 msgr2=0x7fea7015a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:37.072 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.068+0000 7fea76364700 1 --2- 172.21.15.47:0/3626880323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea70144e60 0x7fea7015a120 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fea58001110 tx=0x7fea60014000).stop 2022-06-29T15:42:37.072 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.068+0000 7fea76364700 1 -- 172.21.15.47:0/3626880323 shutdown_connections 2022-06-29T15:42:37.072 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.068+0000 7fea76364700 1 --2- 172.21.15.47:0/3626880323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea70144e60 0x7fea7015a120 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:37.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.068+0000 7fea76364700 1 --2- 172.21.15.47:0/3626880323 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fea700a72b0 0x7fea70159c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:37.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.068+0000 7fea76364700 1 --2- 172.21.15.47:0/3626880323 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fea54053a70 0x7fea54055f30 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:37.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.068+0000 7fea76364700 1 --2- 172.21.15.47:0/3626880323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea70060a70 0x7fea70146450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:37.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.068+0000 7fea76364700 1 -- 172.21.15.47:0/3626880323 >> 172.21.15.47:0/3626880323 conn(0x7fea700626f0 msgr2=0x7fea701453e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:37.074 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.068+0000 7fea76364700 1 -- 172.21.15.47:0/3626880323 shutdown_connections 2022-06-29T15:42:37.074 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.068+0000 7fea76364700 1 -- 172.21.15.47:0/3626880323 wait complete. 2022-06-29T15:42:37.081 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2022-06-29T15:42:37.082 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2022-06-29T15:42:37.082 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2022-06-29T15:42:37.183 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.180+0000 7f7de6e19700 1 Processor -- start 2022-06-29T15:42:37.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.180+0000 7f7de6e19700 1 -- start start 2022-06-29T15:42:37.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.180+0000 7f7de6e19700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7de0143bd0 0x7f7de005c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:37.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.180+0000 7f7de6e19700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7de0142f90 con 0x7f7de005c510 2022-06-29T15:42:37.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.184+0000 7f7de6e19700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7de01431f0 con 0x7f7de00b2c40 2022-06-29T15:42:37.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.184+0000 7f7de6e19700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7de0143490 con 0x7f7de0143bd0 2022-06-29T15:42:37.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.184+0000 7f7de5e17700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7de005c510 0x7f7de00b4100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59588/0 (socket says 172.21.15.47:59588) 2022-06-29T15:42:37.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.184+0000 7f7de5e17700 1 -- 172.21.15.47:0/1933960315 learned_addr learned my addr 172.21.15.47:0/1933960315 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:37.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.184+0000 7f7de5616700 1 --2- 172.21.15.47:0/1933960315 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7de0143bd0 0x7f7de005c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:37.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.184+0000 7f7de4e15700 1 -- 172.21.15.47:0/1933960315 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3136055670 0 0) 0x7f7de0142f90 con 0x7f7de005c510 2022-06-29T15:42:37.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.184+0000 7f7de4e15700 1 -- 172.21.15.47:0/1933960315 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7dcc003260 con 0x7f7de005c510 2022-06-29T15:42:37.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.184+0000 7f7de4e15700 1 -- 172.21.15.47:0/1933960315 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1351474511 0 0) 0x7f7de01431f0 con 0x7f7de00b2c40 2022-06-29T15:42:37.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.184+0000 7f7de4e15700 1 -- 172.21.15.47:0/1933960315 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7de0142f90 con 0x7f7de00b2c40 2022-06-29T15:42:37.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.184+0000 7f7de5616700 1 -- 172.21.15.47:0/1933960315 >> v1:172.21.15.64:6789/0 conn(0x7f7de00b2c40 legacy=0x7f7de013cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:37.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.184+0000 7f7de5616700 1 -- 172.21.15.47:0/1933960315 >> v1:172.21.15.47:6789/0 conn(0x7f7de005c510 legacy=0x7f7de00b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:37.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.184+0000 7f7de5616700 1 -- 172.21.15.47:0/1933960315 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7de0143730 con 0x7f7de0143bd0 2022-06-29T15:42:37.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.184+0000 7f7de5616700 1 --2- 172.21.15.47:0/1933960315 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7de0143bd0 0x7f7de005c8f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f7dd4002f70 tx=0x7f7dd4006df0).ready entity=mon.2 client_cookie=85a6f474115c5a2b server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:37.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.184+0000 7f7de4e15700 1 -- 172.21.15.47:0/1933960315 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7dd400d5c0 con 0x7f7de0143bd0 2022-06-29T15:42:37.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.184+0000 7f7de4e15700 1 -- 172.21.15.47:0/1933960315 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dd4000e90 con 0x7f7de0143bd0 2022-06-29T15:42:37.191 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.184+0000 7f7de4e15700 1 -- 172.21.15.47:0/1933960315 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7dd400d910 con 0x7f7de0143bd0 2022-06-29T15:42:37.194 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.184+0000 7f7de6e19700 1 -- 172.21.15.47:0/1933960315 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7de0143bd0 msgr2=0x7f7de005c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:37.194 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.184+0000 7f7de6e19700 1 --2- 172.21.15.47:0/1933960315 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7de0143bd0 0x7f7de005c8f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f7dd4002f70 tx=0x7f7dd4006df0).stop 2022-06-29T15:42:37.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.184+0000 7f7de6e19700 1 -- 172.21.15.47:0/1933960315 shutdown_connections 2022-06-29T15:42:37.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.184+0000 7f7de6e19700 1 --2- 172.21.15.47:0/1933960315 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7de0143bd0 0x7f7de005c8f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:37.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.184+0000 7f7de6e19700 1 -- 172.21.15.47:0/1933960315 >> 172.21.15.47:0/1933960315 conn(0x7f7de0060e00 msgr2=0x7f7de00611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:37.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.184+0000 7f7de6e19700 1 -- 172.21.15.47:0/1933960315 shutdown_connections 2022-06-29T15:42:37.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.184+0000 7f7de6e19700 1 -- 172.21.15.47:0/1933960315 wait complete. 2022-06-29T15:42:37.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.184+0000 7f7de6e19700 1 Processor -- start 2022-06-29T15:42:37.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.184+0000 7f7de6e19700 1 -- start start 2022-06-29T15:42:37.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.188+0000 7f7de6e19700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7de005c510 0x7f7de00ba250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:37.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.188+0000 7f7de6e19700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7de00b2c40 0x7f7de00ae220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:37.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.188+0000 7f7de6e19700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7de0143bd0 0x7f7de00ae710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:37.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.188+0000 7f7de6e19700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de0153200 con 0x7f7de005c510 2022-06-29T15:42:37.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.188+0000 7f7de6e19700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de013f810 con 0x7f7de0143bd0 2022-06-29T15:42:37.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.188+0000 7f7de6e19700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7de00afef0 con 0x7f7de00b2c40 2022-06-29T15:42:37.198 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.188+0000 7f7de5e17700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7de005c510 0x7f7de00ba250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:37.198 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.188+0000 7f7de5e17700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7de005c510 0x7f7de00ba250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45808/0 (socket says 172.21.15.47:45808) 2022-06-29T15:42:37.198 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.188+0000 7f7de5e17700 1 -- 172.21.15.47:0/3471030240 learned_addr learned my addr 172.21.15.47:0/3471030240 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:37.198 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.188+0000 7f7de5616700 1 --2- 172.21.15.47:0/3471030240 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7de00b2c40 0x7f7de00ae220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:37.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.188+0000 7f7de6618700 1 --2- 172.21.15.47:0/3471030240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7de0143bd0 0x7f7de00ae710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:37.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.188+0000 7f7de5e17700 1 -- 172.21.15.47:0/3471030240 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7de00b2c40 msgr2=0x7f7de00ae220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:37.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.188+0000 7f7de5e17700 1 --2- 172.21.15.47:0/3471030240 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7de00b2c40 0x7f7de00ae220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:37.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.188+0000 7f7de5e17700 1 -- 172.21.15.47:0/3471030240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7de0143bd0 msgr2=0x7f7de00ae710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:37.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.188+0000 7f7de5e17700 1 --2- 172.21.15.47:0/3471030240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7de0143bd0 0x7f7de00ae710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:37.200 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.188+0000 7f7de5e17700 1 -- 172.21.15.47:0/3471030240 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd400b180 con 0x7f7de005c510 2022-06-29T15:42:37.200 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.188+0000 7f7de5e17700 1 --2- 172.21.15.47:0/3471030240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7de005c510 0x7f7de00ba250 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f7dd0001110 tx=0x7f7dd0007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:37.200 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.188+0000 7f7ddeffd700 1 -- 172.21.15.47:0/3471030240 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7dd00070d0 con 0x7f7de005c510 2022-06-29T15:42:37.200 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.188+0000 7f7ddeffd700 1 -- 172.21.15.47:0/3471030240 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dd0007230 con 0x7f7de005c510 2022-06-29T15:42:37.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.188+0000 7f7de6e19700 1 -- 172.21.15.47:0/3471030240 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7de00b04e0 con 0x7f7de005c510 2022-06-29T15:42:37.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.188+0000 7f7ddeffd700 1 -- 172.21.15.47:0/3471030240 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7dd0005610 con 0x7f7de005c510 2022-06-29T15:42:37.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.188+0000 7f7de6e19700 1 -- 172.21.15.47:0/3471030240 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7de013f270 con 0x7f7de005c510 2022-06-29T15:42:37.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.188+0000 7f7de6e19700 1 -- 172.21.15.47:0/3471030240 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7de00573d0 con 0x7f7de005c510 2022-06-29T15:42:37.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.192+0000 7f7ddeffd700 1 -- 172.21.15.47:0/3471030240 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7dd001a060 con 0x7f7de005c510 2022-06-29T15:42:37.202 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.192+0000 7f7ddeffd700 1 --2- 172.21.15.47:0/3471030240 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7dc0053b00 0x7f7dc0055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:37.202 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.192+0000 7f7ddeffd700 1 -- 172.21.15.47:0/3471030240 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f7dd006b350 con 0x7f7de005c510 2022-06-29T15:42:37.202 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.192+0000 7f7de5616700 1 --2- 172.21.15.47:0/3471030240 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7dc0053b00 0x7f7dc0055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:37.203 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.192+0000 7f7de5616700 1 --2- 172.21.15.47:0/3471030240 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7dc0053b00 0x7f7dc0055fc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f7dd401a000 tx=0x7f7dd4014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:37.203 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.196+0000 7f7ddeffd700 1 -- 172.21.15.47:0/3471030240 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7dd0042110 con 0x7f7de005c510 2022-06-29T15:42:37.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.352+0000 7f7de6e19700 1 -- 172.21.15.47:0/3471030240 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f7de0069a60 con 0x7f7de005c510 2022-06-29T15:42:37.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.356+0000 7f7ddeffd700 1 -- 172.21.15.47:0/3471030240 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+131776 (secure 0 0 0) 0x7f7dd0018020 con 0x7f7de005c510 2022-06-29T15:42:37.361 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:42:37.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.360+0000 7f7de6e19700 1 -- 172.21.15.47:0/3471030240 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7dc0053b00 msgr2=0x7f7dc0055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:37.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.360+0000 7f7de6e19700 1 --2- 172.21.15.47:0/3471030240 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7dc0053b00 0x7f7dc0055fc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f7dd401a000 tx=0x7f7dd4014000).stop 2022-06-29T15:42:37.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.360+0000 7f7de6e19700 1 -- 172.21.15.47:0/3471030240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7de005c510 msgr2=0x7f7de00ba250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:37.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.360+0000 7f7de6e19700 1 --2- 172.21.15.47:0/3471030240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7de005c510 0x7f7de00ba250 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f7dd0001110 tx=0x7f7dd0007ad0).stop 2022-06-29T15:42:37.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.360+0000 7f7de6e19700 1 -- 172.21.15.47:0/3471030240 shutdown_connections 2022-06-29T15:42:37.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.360+0000 7f7de6e19700 1 --2- 172.21.15.47:0/3471030240 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7dc0053b00 0x7f7dc0055fc0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:37.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.360+0000 7f7de6e19700 1 --2- 172.21.15.47:0/3471030240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7de0143bd0 0x7f7de00ae710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:37.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.360+0000 7f7de6e19700 1 --2- 172.21.15.47:0/3471030240 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7de00b2c40 0x7f7de00ae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:37.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.360+0000 7f7de6e19700 1 --2- 172.21.15.47:0/3471030240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7de005c510 0x7f7de00ba250 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:37.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.360+0000 7f7de6e19700 1 -- 172.21.15.47:0/3471030240 >> 172.21.15.47:0/3471030240 conn(0x7f7de0060e00 msgr2=0x7f7de00b65d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:37.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.360+0000 7f7de6e19700 1 -- 172.21.15.47:0/3471030240 shutdown_connections 2022-06-29T15:42:37.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.360+0000 7f7de6e19700 1 -- 172.21.15.47:0/3471030240 wait complete. 2022-06-29T15:42:37.376 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":5,"active_gid":4102,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":12928},{"type":"v1","addr":"172.21.15.47:6801","nonce":12928}]},"active_addr":"172.21.15.47:6801/12928","active_change":"2022-06-29T15:42:16.903347+0000","active_mgr_features":4540138292840890367,"available":true,"standbys":[{"gid":4105,"name":"x","mgr_features":4540138292840890367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:6.7.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_cloud","can_run":true,"error_string":"","module_options":{"diskprediction_cert_context":{"name":"diskprediction_cert_context","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_default_authority":{"name":"diskprediction_default_authority","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_password":{"name":"diskprediction_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_port":{"name":"diskprediction_port","type":"str","level":"advanced","flags":0,"default_value":"31400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_retrieve_prediction_interval":{"name":"diskprediction_retrieve_prediction_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_server":{"name":"diskprediction_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_ssl_target_name_override":{"name":"diskprediction_ssl_target_name_override","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_metrics_interval":{"name":"diskprediction_upload_metrics_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_smart_interval":{"name":"diskprediction_upload_smart_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_user":{"name":"diskprediction_user","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:6.7.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_cloud","can_run":true,"error_string":"","module_options":{"diskprediction_cert_context":{"name":"diskprediction_cert_context","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_default_authority":{"name":"diskprediction_default_authority","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_password":{"name":"diskprediction_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_port":{"name":"diskprediction_port","type":"str","level":"advanced","flags":0,"default_value":"31400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_retrieve_prediction_interval":{"name":"diskprediction_retrieve_prediction_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_server":{"name":"diskprediction_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_ssl_target_name_override":{"name":"diskprediction_ssl_target_name_override","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_metrics_interval":{"name":"diskprediction_upload_metrics_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_smart_interval":{"name":"diskprediction_upload_smart_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_user":{"name":"diskprediction_user","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":3468260629}]},{"addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":1124240093}]},{"addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":2670367604}]}]}} 2022-06-29T15:42:37.381 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2022-06-29T15:42:37.381 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2022-06-29T15:42:37.381 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:42:37.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.488+0000 7fbec19fb700 1 Processor -- start 2022-06-29T15:42:37.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.488+0000 7fbec19fb700 1 -- start start 2022-06-29T15:42:37.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.488+0000 7fbec19fb700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbebc1430f0 0x7fbebc1434d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:37.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.488+0000 7fbec19fb700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbebc095dc0 con 0x7fbebc095950 2022-06-29T15:42:37.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.488+0000 7fbec19fb700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbebc060e50 con 0x7fbebc0a93c0 2022-06-29T15:42:37.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.488+0000 7fbec19fb700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbebc13d590 con 0x7fbebc1430f0 2022-06-29T15:42:37.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.488+0000 7fbebaffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbebc1430f0 0x7fbebc1434d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:37.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.488+0000 7fbebb7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fbebc095950 0x7fbebc097690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59604/0 (socket says 172.21.15.47:59604) 2022-06-29T15:42:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.488+0000 7fbebb7fe700 1 -- 172.21.15.47:0/2332120119 learned_addr learned my addr 172.21.15.47:0/2332120119 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.488+0000 7fbebaffd700 1 -- 172.21.15.47:0/2332120119 >> v1:172.21.15.64:6789/0 conn(0x7fbebc0a93c0 legacy=0x7fbebc095240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:37.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.488+0000 7fbebaffd700 1 -- 172.21.15.47:0/2332120119 >> v1:172.21.15.47:6789/0 conn(0x7fbebc095950 legacy=0x7fbebc097690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:37.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.488+0000 7fbebaffd700 1 -- 172.21.15.47:0/2332120119 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbebc0610f0 con 0x7fbebc1430f0 2022-06-29T15:42:37.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.488+0000 7fbeb9ffb700 1 -- 172.21.15.47:0/2332120119 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1372450918 0 0) 0x7fbebc095dc0 con 0x7fbebc095950 2022-06-29T15:42:37.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.488+0000 7fbebaffd700 1 --2- 172.21.15.47:0/2332120119 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbebc1430f0 0x7fbebc1434d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7fbeac002f70 tx=0x7fbeac006df0).ready entity=mon.0 client_cookie=aa2d3e62d8036d5e server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:37.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.488+0000 7fbeb9ffb700 1 -- 172.21.15.47:0/2332120119 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbeac00d610 con 0x7fbebc1430f0 2022-06-29T15:42:37.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.488+0000 7fbeb9ffb700 1 -- 172.21.15.47:0/2332120119 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeac000e90 con 0x7fbebc1430f0 2022-06-29T15:42:37.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.488+0000 7fbeb9ffb700 1 -- 172.21.15.47:0/2332120119 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbeac00d960 con 0x7fbebc1430f0 2022-06-29T15:42:37.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.488+0000 7fbec19fb700 1 -- 172.21.15.47:0/2332120119 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbebc1430f0 msgr2=0x7fbebc1434d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:37.497 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.488+0000 7fbec19fb700 1 --2- 172.21.15.47:0/2332120119 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbebc1430f0 0x7fbebc1434d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7fbeac002f70 tx=0x7fbeac006df0).stop 2022-06-29T15:42:37.497 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.488+0000 7fbec19fb700 1 -- 172.21.15.47:0/2332120119 shutdown_connections 2022-06-29T15:42:37.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.488+0000 7fbec19fb700 1 --2- 172.21.15.47:0/2332120119 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbebc1430f0 0x7fbebc1434d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:37.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.488+0000 7fbec19fb700 1 -- 172.21.15.47:0/2332120119 >> 172.21.15.47:0/2332120119 conn(0x7fbebc05f0b0 msgr2=0x7fbebc05f490 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:37.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.488+0000 7fbec19fb700 1 -- 172.21.15.47:0/2332120119 shutdown_connections 2022-06-29T15:42:37.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.488+0000 7fbec19fb700 1 -- 172.21.15.47:0/2332120119 wait complete. 2022-06-29T15:42:37.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.492+0000 7fbec19fb700 1 Processor -- start 2022-06-29T15:42:37.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.492+0000 7fbec19fb700 1 -- start start 2022-06-29T15:42:37.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.492+0000 7fbec19fb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbebc095950 0x7fbebc09b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:37.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.492+0000 7fbec19fb700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbebc0a93c0 0x7fbebc154020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:37.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.492+0000 7fbec19fb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbebc1430f0 0x7fbebc154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:37.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.492+0000 7fbec19fb700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc1403b0 con 0x7fbebc095950 2022-06-29T15:42:37.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.492+0000 7fbec19fb700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc13dfc0 con 0x7fbebc1430f0 2022-06-29T15:42:37.513 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.492+0000 7fbec19fb700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc13fa20 con 0x7fbebc0a93c0 2022-06-29T15:42:37.513 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.492+0000 7fbebaffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbebc095950 0x7fbebc09b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:37.513 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.492+0000 7fbebaffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbebc095950 0x7fbebc09b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45820/0 (socket says 172.21.15.47:45820) 2022-06-29T15:42:37.513 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.492+0000 7fbebaffd700 1 -- 172.21.15.47:0/1294622730 learned_addr learned my addr 172.21.15.47:0/1294622730 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:37.514 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.492+0000 7fbebb7fe700 1 --2- 172.21.15.47:0/1294622730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbebc1430f0 0x7fbebc154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:37.514 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.492+0000 7fbeba7fc700 1 --2- 172.21.15.47:0/1294622730 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbebc0a93c0 0x7fbebc154020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:37.514 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.492+0000 7fbebaffd700 1 -- 172.21.15.47:0/1294622730 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbebc0a93c0 msgr2=0x7fbebc154020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:37.514 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.492+0000 7fbebaffd700 1 --2- 172.21.15.47:0/1294622730 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbebc0a93c0 0x7fbebc154020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:37.515 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.492+0000 7fbebaffd700 1 -- 172.21.15.47:0/1294622730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbebc1430f0 msgr2=0x7fbebc154510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:37.515 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.492+0000 7fbebaffd700 1 --2- 172.21.15.47:0/1294622730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbebc1430f0 0x7fbebc154510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:37.515 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.492+0000 7fbebaffd700 1 -- 172.21.15.47:0/1294622730 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeac00b180 con 0x7fbebc095950 2022-06-29T15:42:37.515 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.492+0000 7fbebaffd700 1 --2- 172.21.15.47:0/1294622730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbebc095950 0x7fbebc09b2d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7fbeac002f70 tx=0x7fbeac015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:37.515 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.492+0000 7fbeabfff700 1 -- 172.21.15.47:0/1294622730 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbeac00d290 con 0x7fbebc095950 2022-06-29T15:42:37.516 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.492+0000 7fbeabfff700 1 -- 172.21.15.47:0/1294622730 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeac011ce0 con 0x7fbebc095950 2022-06-29T15:42:37.516 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.492+0000 7fbec19fb700 1 -- 172.21.15.47:0/1294622730 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbebc153bc0 con 0x7fbebc095950 2022-06-29T15:42:37.516 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.492+0000 7fbeabfff700 1 -- 172.21.15.47:0/1294622730 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbeac01e650 con 0x7fbebc095950 2022-06-29T15:42:37.516 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.492+0000 7fbec19fb700 1 -- 172.21.15.47:0/1294622730 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbebc13b4e0 con 0x7fbebc095950 2022-06-29T15:42:37.517 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.496+0000 7fbec19fb700 1 -- 172.21.15.47:0/1294622730 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbebc0573d0 con 0x7fbebc095950 2022-06-29T15:42:37.517 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.496+0000 7fbeabfff700 1 -- 172.21.15.47:0/1294622730 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fbeac025030 con 0x7fbebc095950 2022-06-29T15:42:37.517 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.496+0000 7fbeabfff700 1 --2- 172.21.15.47:0/1294622730 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbe98053b00 0x7fbe98055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:37.517 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.496+0000 7fbeabfff700 1 -- 172.21.15.47:0/1294622730 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fbeac06d430 con 0x7fbebc095950 2022-06-29T15:42:37.517 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.496+0000 7fbeba7fc700 1 --2- 172.21.15.47:0/1294622730 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbe98053b00 0x7fbe98055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:37.518 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.496+0000 7fbeba7fc700 1 --2- 172.21.15.47:0/1294622730 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbe98053b00 0x7fbe98055fc0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fbeb000b000 tx=0x7fbeb0009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:37.518 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.504+0000 7fbeabfff700 1 -- 172.21.15.47:0/1294622730 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbeac0441f0 con 0x7fbebc095950 2022-06-29T15:42:37.635 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.632+0000 7fbec19fb700 1 -- 172.21.15.47:0/1294622730 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbebc09bdb0 con 0x7fbebc095950 2022-06-29T15:42:37.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.632+0000 7fbeabfff700 1 -- 172.21.15.47:0/1294622730 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+16299 (secure 0 0 0) 0x7fbeac023020 con 0x7fbebc095950 2022-06-29T15:42:37.637 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:42:37.637 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":26,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:42:34.474438+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:42:28.400446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-06-29T15:42:31.167705+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:42:37.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.636+0000 7fbec19fb700 1 -- 172.21.15.47:0/1294622730 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbe98053b00 msgr2=0x7fbe98055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:37.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.636+0000 7fbec19fb700 1 --2- 172.21.15.47:0/1294622730 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbe98053b00 0x7fbe98055fc0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fbeb000b000 tx=0x7fbeb0009040).stop 2022-06-29T15:42:37.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.636+0000 7fbec19fb700 1 -- 172.21.15.47:0/1294622730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbebc095950 msgr2=0x7fbebc09b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:37.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.636+0000 7fbec19fb700 1 --2- 172.21.15.47:0/1294622730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbebc095950 0x7fbebc09b2d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7fbeac002f70 tx=0x7fbeac015000).stop 2022-06-29T15:42:37.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.636+0000 7fbec19fb700 1 -- 172.21.15.47:0/1294622730 shutdown_connections 2022-06-29T15:42:37.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.636+0000 7fbec19fb700 1 --2- 172.21.15.47:0/1294622730 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbe98053b00 0x7fbe98055fc0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:37.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.636+0000 7fbec19fb700 1 --2- 172.21.15.47:0/1294622730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbebc1430f0 0x7fbebc154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:37.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.636+0000 7fbec19fb700 1 --2- 172.21.15.47:0/1294622730 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbebc0a93c0 0x7fbebc154020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:37.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.636+0000 7fbec19fb700 1 --2- 172.21.15.47:0/1294622730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbebc095950 0x7fbebc09b2d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:37.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.636+0000 7fbec19fb700 1 -- 172.21.15.47:0/1294622730 >> 172.21.15.47:0/1294622730 conn(0x7fbebc05f0b0 msgr2=0x7fbebc0b5470 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:37.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.636+0000 7fbec19fb700 1 -- 172.21.15.47:0/1294622730 shutdown_connections 2022-06-29T15:42:37.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.636+0000 7fbec19fb700 1 -- 172.21.15.47:0/1294622730 wait complete. 2022-06-29T15:42:37.651 INFO:tasks.ceph.ceph_manager.ceph:all up! 2022-06-29T15:42:37.652 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:42:37.756 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.752+0000 7fc7f094a700 1 Processor -- start 2022-06-29T15:42:37.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.752+0000 7fc7f094a700 1 -- start start 2022-06-29T15:42:37.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.756+0000 7fc7f094a700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc7ec0b2120 0x7fc7ec0b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:37.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.756+0000 7fc7f094a700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7ec0b0740 con 0x7fc7ec0b1260 2022-06-29T15:42:37.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.756+0000 7fc7f094a700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7ec0b0970 con 0x7fc7ec05f0b0 2022-06-29T15:42:37.761 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.756+0000 7fc7f094a700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7ec147460 con 0x7fc7ec0b2120 2022-06-29T15:42:37.761 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.756+0000 7fc7ead9d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc7ec0b1260 0x7fc7ec05f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59618/0 (socket says 172.21.15.47:59618) 2022-06-29T15:42:37.761 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.756+0000 7fc7ead9d700 1 -- 172.21.15.47:0/1977378424 learned_addr learned my addr 172.21.15.47:0/1977378424 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:37.761 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.756+0000 7fc7ea59c700 1 --2- 172.21.15.47:0/1977378424 >> v2:172.21.15.64:3300/0 conn(0x7fc7ec0b2120 0x7fc7ec0b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:37.761 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.756+0000 7fc7e959a700 1 -- 172.21.15.47:0/1977378424 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1981468962 0 0) 0x7fc7ec0b0740 con 0x7fc7ec0b1260 2022-06-29T15:42:37.762 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.756+0000 7fc7e959a700 1 -- 172.21.15.47:0/1977378424 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7d0003260 con 0x7fc7ec0b1260 2022-06-29T15:42:37.762 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.756+0000 7fc7ea59c700 1 -- 172.21.15.47:0/1977378424 >> v1:172.21.15.64:6789/0 conn(0x7fc7ec05f0b0 legacy=0x7fc7ec147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:37.762 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.756+0000 7fc7ea59c700 1 -- 172.21.15.47:0/1977378424 >> v1:172.21.15.47:6789/0 conn(0x7fc7ec0b1260 legacy=0x7fc7ec05f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:37.762 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.756+0000 7fc7ea59c700 1 -- 172.21.15.47:0/1977378424 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7ec0b0c10 con 0x7fc7ec0b2120 2022-06-29T15:42:37.763 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.756+0000 7fc7ea59c700 1 --2- 172.21.15.47:0/1977378424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7ec0b2120 0x7fc7ec0b2500 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fc7ec0affe0 tx=0x7fc7e0006df0).ready entity=mon.1 client_cookie=1504436c41bc5a13 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:37.763 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.756+0000 7fc7e959a700 1 -- 172.21.15.47:0/1977378424 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7e000d610 con 0x7fc7ec0b2120 2022-06-29T15:42:37.763 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.756+0000 7fc7e959a700 1 -- 172.21.15.47:0/1977378424 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7e0000e90 con 0x7fc7ec0b2120 2022-06-29T15:42:37.763 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.756+0000 7fc7e959a700 1 -- 172.21.15.47:0/1977378424 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7e000d960 con 0x7fc7ec0b2120 2022-06-29T15:42:37.764 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.756+0000 7fc7f094a700 1 -- 172.21.15.47:0/1977378424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7ec0b2120 msgr2=0x7fc7ec0b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:37.764 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.756+0000 7fc7f094a700 1 --2- 172.21.15.47:0/1977378424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7ec0b2120 0x7fc7ec0b2500 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fc7ec0affe0 tx=0x7fc7e0006df0).stop 2022-06-29T15:42:37.764 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.756+0000 7fc7f094a700 1 -- 172.21.15.47:0/1977378424 shutdown_connections 2022-06-29T15:42:37.765 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.756+0000 7fc7f094a700 1 --2- 172.21.15.47:0/1977378424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7ec0b2120 0x7fc7ec0b2500 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:37.765 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.756+0000 7fc7f094a700 1 -- 172.21.15.47:0/1977378424 >> 172.21.15.47:0/1977378424 conn(0x7fc7ec0626f0 msgr2=0x7fc7ec062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:37.765 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.756+0000 7fc7f094a700 1 -- 172.21.15.47:0/1977378424 shutdown_connections 2022-06-29T15:42:37.765 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.756+0000 7fc7f094a700 1 -- 172.21.15.47:0/1977378424 wait complete. 2022-06-29T15:42:37.768 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.760+0000 7fc7f094a700 1 Processor -- start 2022-06-29T15:42:37.768 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.760+0000 7fc7f094a700 1 -- start start 2022-06-29T15:42:37.769 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.760+0000 7fc7f094a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7ec0b1260 0x7fc7ec1573f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:37.769 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.760+0000 7fc7f094a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7ec0b2120 0x7fc7ec149470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:37.769 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.760+0000 7fc7f094a700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc7ec0ad800 0x7fc7ec0adcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:37.769 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.760+0000 7fc7f094a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ec1578e0 con 0x7fc7ec0b1260 2022-06-29T15:42:37.770 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.760+0000 7fc7f094a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ec1430c0 con 0x7fc7ec0b2120 2022-06-29T15:42:37.770 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.760+0000 7fc7f094a700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ec143360 con 0x7fc7ec0ad800 2022-06-29T15:42:37.770 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.760+0000 7fc7ea59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7ec0b1260 0x7fc7ec1573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:37.770 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.760+0000 7fc7ea59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7ec0b1260 0x7fc7ec1573f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45834/0 (socket says 172.21.15.47:45834) 2022-06-29T15:42:37.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.760+0000 7fc7ea59c700 1 -- 172.21.15.47:0/1887186661 learned_addr learned my addr 172.21.15.47:0/1887186661 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:37.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.760+0000 7fc7ead9d700 1 --2- 172.21.15.47:0/1887186661 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc7ec0ad800 0x7fc7ec0adcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:37.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.760+0000 7fc7e9d9b700 1 --2- 172.21.15.47:0/1887186661 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7ec0b2120 0x7fc7ec149470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:37.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.760+0000 7fc7ea59c700 1 -- 172.21.15.47:0/1887186661 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc7ec0ad800 msgr2=0x7fc7ec0adcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:37.772 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.760+0000 7fc7ea59c700 1 --2- 172.21.15.47:0/1887186661 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc7ec0ad800 0x7fc7ec0adcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:37.772 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.760+0000 7fc7ea59c700 1 -- 172.21.15.47:0/1887186661 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7ec0b2120 msgr2=0x7fc7ec149470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:37.772 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.760+0000 7fc7ea59c700 1 --2- 172.21.15.47:0/1887186661 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7ec0b2120 0x7fc7ec149470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:37.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.760+0000 7fc7ea59c700 1 -- 172.21.15.47:0/1887186661 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7e000b180 con 0x7fc7ec0b1260 2022-06-29T15:42:37.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.760+0000 7fc7ead9d700 1 --2- 172.21.15.47:0/1887186661 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc7ec0ad800 0x7fc7ec0adcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:42:37.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.760+0000 7fc7ea59c700 1 --2- 172.21.15.47:0/1887186661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7ec0b1260 0x7fc7ec1573f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7fc7ec0affe0 tx=0x7fc7e0014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:37.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.760+0000 7fc7db7fe700 1 -- 172.21.15.47:0/1887186661 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7e0016070 con 0x7fc7ec0b1260 2022-06-29T15:42:37.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.760+0000 7fc7db7fe700 1 -- 172.21.15.47:0/1887186661 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7e000de90 con 0x7fc7ec0b1260 2022-06-29T15:42:37.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.760+0000 7fc7f094a700 1 -- 172.21.15.47:0/1887186661 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7ec143600 con 0x7fc7ec0b1260 2022-06-29T15:42:37.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.760+0000 7fc7db7fe700 1 -- 172.21.15.47:0/1887186661 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7e0013e50 con 0x7fc7ec0b1260 2022-06-29T15:42:37.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.760+0000 7fc7f094a700 1 -- 172.21.15.47:0/1887186661 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7ec0ae1a0 con 0x7fc7ec0b1260 2022-06-29T15:42:37.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.760+0000 7fc7f094a700 1 -- 172.21.15.47:0/1887186661 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7ec143aa0 con 0x7fc7ec0b1260 2022-06-29T15:42:37.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.764+0000 7fc7db7fe700 1 -- 172.21.15.47:0/1887186661 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc7e000f030 con 0x7fc7ec0b1260 2022-06-29T15:42:37.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.764+0000 7fc7db7fe700 1 --2- 172.21.15.47:0/1887186661 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc7c80539e0 0x7fc7c8055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:37.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.764+0000 7fc7db7fe700 1 -- 172.21.15.47:0/1887186661 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fc7e006ce10 con 0x7fc7ec0b1260 2022-06-29T15:42:37.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.764+0000 7fc7e9d9b700 1 --2- 172.21.15.47:0/1887186661 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc7c80539e0 0x7fc7c8055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:37.776 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.764+0000 7fc7e9d9b700 1 --2- 172.21.15.47:0/1887186661 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc7c80539e0 0x7fc7c8055ea0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fc7dc00d000 tx=0x7fc7dc000f40).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:37.776 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.768+0000 7fc7db7fe700 1 -- 172.21.15.47:0/1887186661 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc7ec143aa0 con 0x7fc7ec0b1260 2022-06-29T15:42:37.896 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.892+0000 7fc7f094a700 1 -- 172.21.15.47:0/1887186661 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc7ec143aa0 con 0x7fc7ec0b1260 2022-06-29T15:42:37.897 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.896+0000 7fc7db7fe700 1 -- 172.21.15.47:0/1887186661 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+16299 (secure 0 0 0) 0x7fc7ec143aa0 con 0x7fc7ec0b1260 2022-06-29T15:42:37.898 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:42:37.898 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":26,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:42:34.474438+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:42:28.400446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-06-29T15:42:31.167705+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:42:37.900 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.896+0000 7fc7f094a700 1 -- 172.21.15.47:0/1887186661 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc7c80539e0 msgr2=0x7fc7c8055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:37.900 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.896+0000 7fc7f094a700 1 --2- 172.21.15.47:0/1887186661 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc7c80539e0 0x7fc7c8055ea0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fc7dc00d000 tx=0x7fc7dc000f40).stop 2022-06-29T15:42:37.900 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.896+0000 7fc7f094a700 1 -- 172.21.15.47:0/1887186661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7ec0b1260 msgr2=0x7fc7ec1573f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:37.901 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.896+0000 7fc7f094a700 1 --2- 172.21.15.47:0/1887186661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7ec0b1260 0x7fc7ec1573f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7fc7ec0affe0 tx=0x7fc7e0014000).stop 2022-06-29T15:42:37.901 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.896+0000 7fc7f094a700 1 -- 172.21.15.47:0/1887186661 shutdown_connections 2022-06-29T15:42:37.901 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.896+0000 7fc7f094a700 1 --2- 172.21.15.47:0/1887186661 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc7ec0ad800 0x7fc7ec0adcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:37.901 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.896+0000 7fc7f094a700 1 --2- 172.21.15.47:0/1887186661 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc7c80539e0 0x7fc7c8055ea0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:37.902 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.896+0000 7fc7f094a700 1 --2- 172.21.15.47:0/1887186661 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7ec0b2120 0x7fc7ec149470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:37.902 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.896+0000 7fc7f094a700 1 --2- 172.21.15.47:0/1887186661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7ec0b1260 0x7fc7ec1573f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:37.902 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.896+0000 7fc7f094a700 1 -- 172.21.15.47:0/1887186661 >> 172.21.15.47:0/1887186661 conn(0x7fc7ec0626f0 msgr2=0x7fc7ec145300 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:37.903 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.896+0000 7fc7f094a700 1 -- 172.21.15.47:0/1887186661 shutdown_connections 2022-06-29T15:42:37.903 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:37.896+0000 7fc7f094a700 1 -- 172.21.15.47:0/1887186661 wait complete. 2022-06-29T15:42:37.911 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2022-06-29T15:42:38.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.016+0000 7fd37229a700 1 Processor -- start 2022-06-29T15:42:38.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.016+0000 7fd37229a700 1 -- start start 2022-06-29T15:42:38.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.016+0000 7fd37229a700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd36c0b5c60 0x7fd36c144980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:38.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.016+0000 7fd37229a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd36c0a72f0 0x7fd36c0b2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:38.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.016+0000 7fd37229a700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd36c0a9120 con 0x7fd36c060eb0 2022-06-29T15:42:38.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.016+0000 7fd37229a700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd36c0b0b80 con 0x7fd36c0a72f0 2022-06-29T15:42:38.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.016+0000 7fd37229a700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd36c0a89e0 con 0x7fd36c0b5c60 2022-06-29T15:42:38.022 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.016+0000 7fd370a97700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd36c060eb0 0x7fd36c0b1e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59628/0 (socket says 172.21.15.47:59628) 2022-06-29T15:42:38.022 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.016+0000 7fd371a99700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd36c0a72f0 0x7fd36c0b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:38.022 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.016+0000 7fd370a97700 1 -- 172.21.15.47:0/666436119 learned_addr learned my addr 172.21.15.47:0/666436119 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:38.022 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.016+0000 7fd371298700 1 --2- 172.21.15.47:0/666436119 >> v2:172.21.15.64:3300/0 conn(0x7fd36c0b5c60 0x7fd36c144980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:38.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.016+0000 7fd371a99700 1 -- 172.21.15.47:0/666436119 >> v2:172.21.15.64:3300/0 conn(0x7fd36c0b5c60 msgr2=0x7fd36c144980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:38.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.016+0000 7fd371a99700 1 --2- 172.21.15.47:0/666436119 >> v2:172.21.15.64:3300/0 conn(0x7fd36c0b5c60 0x7fd36c144980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:38.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.016+0000 7fd371a99700 1 -- 172.21.15.47:0/666436119 >> v1:172.21.15.47:6789/0 conn(0x7fd36c060eb0 legacy=0x7fd36c0b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:38.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.016+0000 7fd36bfff700 1 -- 172.21.15.47:0/666436119 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1361318466 0 0) 0x7fd36c0a9120 con 0x7fd36c060eb0 2022-06-29T15:42:38.024 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.016+0000 7fd371a99700 1 -- 172.21.15.47:0/666436119 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd36c0a8c80 con 0x7fd36c0a72f0 2022-06-29T15:42:38.024 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.016+0000 7fd371a99700 1 --2- 172.21.15.47:0/666436119 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd36c0a72f0 0x7fd36c0b2540 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fd35c002f70 tx=0x7fd35c006df0).ready entity=mon.0 client_cookie=f9cc3e993916773d server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:38.024 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.016+0000 7fd36bfff700 1 -- 172.21.15.47:0/666436119 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd35c00d570 con 0x7fd36c0a72f0 2022-06-29T15:42:38.025 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.016+0000 7fd36bfff700 1 -- 172.21.15.47:0/666436119 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd35c000e90 con 0x7fd36c0a72f0 2022-06-29T15:42:38.025 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.016+0000 7fd36bfff700 1 -- 172.21.15.47:0/666436119 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd35c00d8c0 con 0x7fd36c0a72f0 2022-06-29T15:42:38.025 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.016+0000 7fd37229a700 1 -- 172.21.15.47:0/666436119 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd36c0a72f0 msgr2=0x7fd36c0b2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:38.025 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.016+0000 7fd37229a700 1 --2- 172.21.15.47:0/666436119 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd36c0a72f0 0x7fd36c0b2540 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fd35c002f70 tx=0x7fd35c006df0).stop 2022-06-29T15:42:38.026 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.016+0000 7fd37229a700 1 -- 172.21.15.47:0/666436119 shutdown_connections 2022-06-29T15:42:38.026 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.016+0000 7fd37229a700 1 --2- 172.21.15.47:0/666436119 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd36c0a72f0 0x7fd36c0b2540 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:38.026 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.016+0000 7fd37229a700 1 --2- 172.21.15.47:0/666436119 >> v2:172.21.15.64:3300/0 conn(0x7fd36c0b5c60 0x7fd36c144980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:38.026 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.016+0000 7fd37229a700 1 -- 172.21.15.47:0/666436119 >> 172.21.15.47:0/666436119 conn(0x7fd36c062560 msgr2=0x7fd36c062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:38.027 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.016+0000 7fd37229a700 1 -- 172.21.15.47:0/666436119 shutdown_connections 2022-06-29T15:42:38.027 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.016+0000 7fd37229a700 1 -- 172.21.15.47:0/666436119 wait complete. 2022-06-29T15:42:38.031 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.020+0000 7fd37229a700 1 Processor -- start 2022-06-29T15:42:38.032 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.020+0000 7fd37229a700 1 -- start start 2022-06-29T15:42:38.032 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.020+0000 7fd37229a700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd36c060eb0 0x7fd36c148980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:38.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.020+0000 7fd37229a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd36c0a72f0 0x7fd36c0b35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:38.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.020+0000 7fd37229a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd36c0b5c60 0x7fd36c156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:38.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.020+0000 7fd37229a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd36c149890 con 0x7fd36c0a72f0 2022-06-29T15:42:38.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.020+0000 7fd37229a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd36c0b44b0 con 0x7fd36c0b5c60 2022-06-29T15:42:38.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.020+0000 7fd37229a700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd36c0b4750 con 0x7fd36c060eb0 2022-06-29T15:42:38.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.020+0000 7fd370a97700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd36c0a72f0 0x7fd36c0b35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:38.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.020+0000 7fd370a97700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd36c0a72f0 0x7fd36c0b35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45848/0 (socket says 172.21.15.47:45848) 2022-06-29T15:42:38.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.020+0000 7fd370a97700 1 -- 172.21.15.47:0/1805879118 learned_addr learned my addr 172.21.15.47:0/1805879118 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:38.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.020+0000 7fd371298700 1 --2- 172.21.15.47:0/1805879118 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd36c060eb0 0x7fd36c148980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:38.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.020+0000 7fd371a99700 1 --2- 172.21.15.47:0/1805879118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd36c0b5c60 0x7fd36c156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:38.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.020+0000 7fd370a97700 1 -- 172.21.15.47:0/1805879118 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd36c060eb0 msgr2=0x7fd36c148980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:38.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.020+0000 7fd370a97700 1 --2- 172.21.15.47:0/1805879118 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd36c060eb0 0x7fd36c148980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:38.038 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.020+0000 7fd370a97700 1 -- 172.21.15.47:0/1805879118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd36c0b5c60 msgr2=0x7fd36c156dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:38.038 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.020+0000 7fd370a97700 1 --2- 172.21.15.47:0/1805879118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd36c0b5c60 0x7fd36c156dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:38.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.020+0000 7fd370a97700 1 -- 172.21.15.47:0/1805879118 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd35c00b180 con 0x7fd36c0a72f0 2022-06-29T15:42:38.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.020+0000 7fd370a97700 1 --2- 172.21.15.47:0/1805879118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd36c0a72f0 0x7fd36c0b35e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7fd360000ed0 tx=0x7fd360007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:38.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.020+0000 7fd36a7fc700 1 -- 172.21.15.47:0/1805879118 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3600070d0 con 0x7fd36c0a72f0 2022-06-29T15:42:38.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.020+0000 7fd36a7fc700 1 -- 172.21.15.47:0/1805879118 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd360007230 con 0x7fd36c0a72f0 2022-06-29T15:42:38.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.024+0000 7fd36a7fc700 1 -- 172.21.15.47:0/1805879118 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd360005610 con 0x7fd36c0a72f0 2022-06-29T15:42:38.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.024+0000 7fd37229a700 1 -- 172.21.15.47:0/1805879118 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd36c156920 con 0x7fd36c0a72f0 2022-06-29T15:42:38.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.024+0000 7fd37229a700 1 -- 172.21.15.47:0/1805879118 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd36c0b41a0 con 0x7fd36c0a72f0 2022-06-29T15:42:38.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.024+0000 7fd37229a700 1 -- 172.21.15.47:0/1805879118 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd36c156ab0 con 0x7fd36c0a72f0 2022-06-29T15:42:38.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.024+0000 7fd36a7fc700 1 -- 172.21.15.47:0/1805879118 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd360010070 con 0x7fd36c0a72f0 2022-06-29T15:42:38.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.024+0000 7fd36a7fc700 1 --2- 172.21.15.47:0/1805879118 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd34c0539e0 0x7fd34c055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:38.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.024+0000 7fd36a7fc700 1 -- 172.21.15.47:0/1805879118 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fd36006b8b0 con 0x7fd36c0a72f0 2022-06-29T15:42:38.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.024+0000 7fd371298700 1 --2- 172.21.15.47:0/1805879118 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd34c0539e0 0x7fd34c055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:38.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.024+0000 7fd36a7fc700 1 --2- 172.21.15.47:0/1805879118 >> [v2:172.21.15.47:6818/13096,v1:172.21.15.47:6819/13096] conn(0x7fd34c05b360 0x7fd34c05d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:38.045 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.024+0000 7fd36a7fc700 1 -- 172.21.15.47:0/1805879118 --> [v2:172.21.15.47:6818/13096,v1:172.21.15.47:6819/13096] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd34c05de80 con 0x7fd34c05b360 2022-06-29T15:42:38.045 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.024+0000 7fd36a7fc700 1 -- 172.21.15.47:0/1805879118 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd360070240 con 0x7fd36c0a72f0 2022-06-29T15:42:38.045 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.024+0000 7fd371a99700 1 --2- 172.21.15.47:0/1805879118 >> [v2:172.21.15.47:6818/13096,v1:172.21.15.47:6819/13096] conn(0x7fd34c05b360 0x7fd34c05d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:38.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.024+0000 7fd371298700 1 --2- 172.21.15.47:0/1805879118 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd34c0539e0 0x7fd34c055ea0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fd36400b000 tx=0x7fd3640091a0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:38.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.024+0000 7fd371a99700 1 --2- 172.21.15.47:0/1805879118 >> [v2:172.21.15.47:6818/13096,v1:172.21.15.47:6819/13096] conn(0x7fd34c05b360 0x7fd34c05d800 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:38.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.028+0000 7fd36a7fc700 1 -- 172.21.15.47:0/1805879118 <== osd.0 v2:172.21.15.47:6818/13096 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7fd34c05de80 con 0x7fd34c05b360 2022-06-29T15:42:38.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.044+0000 7fd37229a700 1 -- 172.21.15.47:0/1805879118 --> [v2:172.21.15.47:6818/13096,v1:172.21.15.47:6819/13096] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd36c0573d0 con 0x7fd34c05b360 2022-06-29T15:42:38.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.044+0000 7fd36a7fc700 1 -- 172.21.15.47:0/1805879118 <== osd.0 v2:172.21.15.47:6818/13096 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd36c0573d0 con 0x7fd34c05b360 2022-06-29T15:42:38.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.044+0000 7fd37229a700 1 -- 172.21.15.47:0/1805879118 >> [v2:172.21.15.47:6818/13096,v1:172.21.15.47:6819/13096] conn(0x7fd34c05b360 msgr2=0x7fd34c05d800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:38.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.044+0000 7fd37229a700 1 --2- 172.21.15.47:0/1805879118 >> [v2:172.21.15.47:6818/13096,v1:172.21.15.47:6819/13096] conn(0x7fd34c05b360 0x7fd34c05d800 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:38.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.044+0000 7fd37229a700 1 -- 172.21.15.47:0/1805879118 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd34c0539e0 msgr2=0x7fd34c055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:38.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.044+0000 7fd37229a700 1 --2- 172.21.15.47:0/1805879118 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd34c0539e0 0x7fd34c055ea0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fd36400b000 tx=0x7fd3640091a0).stop 2022-06-29T15:42:38.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.044+0000 7fd37229a700 1 -- 172.21.15.47:0/1805879118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd36c0a72f0 msgr2=0x7fd36c0b35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:38.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.044+0000 7fd37229a700 1 --2- 172.21.15.47:0/1805879118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd36c0a72f0 0x7fd36c0b35e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7fd360000ed0 tx=0x7fd360007ac0).stop 2022-06-29T15:42:38.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.044+0000 7fd371a99700 1 -- 172.21.15.47:0/1805879118 reap_dead start 2022-06-29T15:42:38.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.044+0000 7fd37229a700 1 -- 172.21.15.47:0/1805879118 shutdown_connections 2022-06-29T15:42:38.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.044+0000 7fd37229a700 1 -- 172.21.15.47:0/1805879118 >> 172.21.15.47:0/1805879118 conn(0x7fd36c062560 msgr2=0x7fd36c1405e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:38.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.044+0000 7fd37229a700 1 -- 172.21.15.47:0/1805879118 shutdown_connections 2022-06-29T15:42:38.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.044+0000 7fd37229a700 1 -- 172.21.15.47:0/1805879118 wait complete. 2022-06-29T15:42:38.058 INFO:teuthology.orchestra.run.smithi047.stdout:60129542149 2022-06-29T15:42:38.059 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2022-06-29T15:42:38.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.160+0000 7fbe017f5700 1 Processor -- start 2022-06-29T15:42:38.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.160+0000 7fbe017f5700 1 -- start start 2022-06-29T15:42:38.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.164+0000 7fbe017f5700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbdfc0b0cf0 0x7fbdfc146d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:38.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.164+0000 7fbe017f5700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbdfc0b06d0 0x7fbdfc13ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:38.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.164+0000 7fbe017f5700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdfc0b4520 con 0x7fbdfc0611b0 2022-06-29T15:42:38.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.164+0000 7fbe017f5700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdfc0b4b10 con 0x7fbdfc0b06d0 2022-06-29T15:42:38.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.164+0000 7fbe017f5700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdfc0621d0 con 0x7fbdfc0b0cf0 2022-06-29T15:42:38.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.164+0000 7fbdfb7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbdfc0b06d0 0x7fbdfc13ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:38.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.164+0000 7fbdfaffd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fbdfc0611b0 0x7fbdfc146640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59646/0 (socket says 172.21.15.47:59646) 2022-06-29T15:42:38.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.164+0000 7fbdfaffd700 1 -- 172.21.15.47:0/462061132 learned_addr learned my addr 172.21.15.47:0/462061132 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:38.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.164+0000 7fbdfa7fc700 1 --2- 172.21.15.47:0/462061132 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbdfc0b0cf0 0x7fbdfc146d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:38.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.164+0000 7fbdfb7fe700 1 -- 172.21.15.47:0/462061132 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbdfc0b0cf0 msgr2=0x7fbdfc146d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:38.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.164+0000 7fbdfb7fe700 1 --2- 172.21.15.47:0/462061132 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbdfc0b0cf0 0x7fbdfc146d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:38.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.164+0000 7fbdfb7fe700 1 -- 172.21.15.47:0/462061132 >> v1:172.21.15.47:6789/0 conn(0x7fbdfc0611b0 legacy=0x7fbdfc146640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:38.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.164+0000 7fbdfb7fe700 1 -- 172.21.15.47:0/462061132 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdfc061590 con 0x7fbdfc0b06d0 2022-06-29T15:42:38.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.164+0000 7fbdf9ffb700 1 -- 172.21.15.47:0/462061132 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3064902319 0 0) 0x7fbdfc0b4520 con 0x7fbdfc0611b0 2022-06-29T15:42:38.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.164+0000 7fbdfb7fe700 1 --2- 172.21.15.47:0/462061132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbdfc0b06d0 0x7fbdfc13ff00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7fbdf40032f0 tx=0x7fbdf40091d0).ready entity=mon.0 client_cookie=ab62d7961ee6d059 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:38.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.164+0000 7fbdf9ffb700 1 -- 172.21.15.47:0/462061132 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbdf400b6c0 con 0x7fbdfc0b06d0 2022-06-29T15:42:38.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.164+0000 7fbdf9ffb700 1 -- 172.21.15.47:0/462061132 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdf400b820 con 0x7fbdfc0b06d0 2022-06-29T15:42:38.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.164+0000 7fbdf9ffb700 1 -- 172.21.15.47:0/462061132 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbdf400b990 con 0x7fbdfc0b06d0 2022-06-29T15:42:38.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.164+0000 7fbe017f5700 1 -- 172.21.15.47:0/462061132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbdfc0b06d0 msgr2=0x7fbdfc13ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:38.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.164+0000 7fbe017f5700 1 --2- 172.21.15.47:0/462061132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbdfc0b06d0 0x7fbdfc13ff00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7fbdf40032f0 tx=0x7fbdf40091d0).stop 2022-06-29T15:42:38.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.164+0000 7fbe017f5700 1 -- 172.21.15.47:0/462061132 shutdown_connections 2022-06-29T15:42:38.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.164+0000 7fbe017f5700 1 --2- 172.21.15.47:0/462061132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbdfc0b06d0 0x7fbdfc13ff00 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:38.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.164+0000 7fbe017f5700 1 --2- 172.21.15.47:0/462061132 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbdfc0b0cf0 0x7fbdfc146d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:38.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.164+0000 7fbe017f5700 1 -- 172.21.15.47:0/462061132 >> 172.21.15.47:0/462061132 conn(0x7fbdfc0b98b0 msgr2=0x7fbdfc0b9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:38.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.164+0000 7fbe017f5700 1 -- 172.21.15.47:0/462061132 shutdown_connections 2022-06-29T15:42:38.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.164+0000 7fbe017f5700 1 -- 172.21.15.47:0/462061132 wait complete. 2022-06-29T15:42:38.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.168+0000 7fbe017f5700 1 Processor -- start 2022-06-29T15:42:38.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.168+0000 7fbe017f5700 1 -- start start 2022-06-29T15:42:38.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.168+0000 7fbe017f5700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbdfc0611b0 0x7fbdfc0b6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:38.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.168+0000 7fbe017f5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdfc0b06d0 0x7fbdfc13c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:38.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.168+0000 7fbe017f5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbdfc0b0cf0 0x7fbdfc13cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:38.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.168+0000 7fbe017f5700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdfc138d30 con 0x7fbdfc0b0cf0 2022-06-29T15:42:38.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.168+0000 7fbe017f5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdfc13b890 con 0x7fbdfc0b06d0 2022-06-29T15:42:38.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.168+0000 7fbe017f5700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdfc138380 con 0x7fbdfc0611b0 2022-06-29T15:42:38.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.168+0000 7fbdfb7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbdfc0b0cf0 0x7fbdfc13cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:38.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.168+0000 7fbdfb7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbdfc0b0cf0 0x7fbdfc13cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45868/0 (socket says 172.21.15.47:45868) 2022-06-29T15:42:38.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.168+0000 7fbdfb7fe700 1 -- 172.21.15.47:0/4214669791 learned_addr learned my addr 172.21.15.47:0/4214669791 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:38.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.168+0000 7fbdfa7fc700 1 --2- 172.21.15.47:0/4214669791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdfc0b06d0 0x7fbdfc13c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:38.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.168+0000 7fbdfaffd700 1 --2- 172.21.15.47:0/4214669791 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbdfc0611b0 0x7fbdfc0b6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:38.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.168+0000 7fbdfb7fe700 1 -- 172.21.15.47:0/4214669791 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbdfc0611b0 msgr2=0x7fbdfc0b6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:38.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.168+0000 7fbdfb7fe700 1 --2- 172.21.15.47:0/4214669791 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbdfc0611b0 0x7fbdfc0b6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:38.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.168+0000 7fbdfb7fe700 1 -- 172.21.15.47:0/4214669791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdfc0b06d0 msgr2=0x7fbdfc13c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:38.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.168+0000 7fbdfb7fe700 1 --2- 172.21.15.47:0/4214669791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdfc0b06d0 0x7fbdfc13c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:38.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.168+0000 7fbdfb7fe700 1 -- 172.21.15.47:0/4214669791 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf4009470 con 0x7fbdfc0b0cf0 2022-06-29T15:42:38.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.168+0000 7fbdfb7fe700 1 --2- 172.21.15.47:0/4214669791 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbdfc0b0cf0 0x7fbdfc13cdf0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fbdf40032f0 tx=0x7fbdf4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:38.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.168+0000 7fbdf3fff700 1 -- 172.21.15.47:0/4214669791 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbdf400b990 con 0x7fbdfc0b0cf0 2022-06-29T15:42:38.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.168+0000 7fbdf3fff700 1 -- 172.21.15.47:0/4214669791 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdf4021630 con 0x7fbdfc0b0cf0 2022-06-29T15:42:38.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.168+0000 7fbe017f5700 1 -- 172.21.15.47:0/4214669791 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdfc138a90 con 0x7fbdfc0b0cf0 2022-06-29T15:42:38.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.168+0000 7fbdf3fff700 1 -- 172.21.15.47:0/4214669791 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbdf400b6c0 con 0x7fbdfc0b0cf0 2022-06-29T15:42:38.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.168+0000 7fbe017f5700 1 -- 172.21.15.47:0/4214669791 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdfc0b5030 con 0x7fbdfc0b0cf0 2022-06-29T15:42:38.182 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.172+0000 7fbe017f5700 1 -- 172.21.15.47:0/4214669791 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbdfc13b710 con 0x7fbdfc0b0cf0 2022-06-29T15:42:38.182 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.172+0000 7fbdf3fff700 1 -- 172.21.15.47:0/4214669791 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fbdf4027020 con 0x7fbdfc0b0cf0 2022-06-29T15:42:38.182 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.172+0000 7fbdf3fff700 1 --2- 172.21.15.47:0/4214669791 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbde0053aa0 0x7fbde0055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:38.182 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.172+0000 7fbdfaffd700 1 --2- 172.21.15.47:0/4214669791 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbde0053aa0 0x7fbde0055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:38.182 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.172+0000 7fbdf3fff700 1 -- 172.21.15.47:0/4214669791 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fbdf406d370 con 0x7fbdfc0b0cf0 2022-06-29T15:42:38.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.172+0000 7fbdf3fff700 1 --2- 172.21.15.47:0/4214669791 >> [v2:172.21.15.47:6826/13100,v1:172.21.15.47:6827/13100] conn(0x7fbde005b3a0 0x7fbde005d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:38.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.172+0000 7fbdf3fff700 1 -- 172.21.15.47:0/4214669791 --> [v2:172.21.15.47:6826/13100,v1:172.21.15.47:6827/13100] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbde005de20 con 0x7fbde005b3a0 2022-06-29T15:42:38.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.172+0000 7fbdfaffd700 1 --2- 172.21.15.47:0/4214669791 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbde0053aa0 0x7fbde0055f60 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fbde400d000 tx=0x7fbde4007a00).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:38.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.172+0000 7fbdf3fff700 1 -- 172.21.15.47:0/4214669791 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbdf4044130 con 0x7fbdfc0b0cf0 2022-06-29T15:42:38.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.172+0000 7fbdfa7fc700 1 --2- 172.21.15.47:0/4214669791 >> [v2:172.21.15.47:6826/13100,v1:172.21.15.47:6827/13100] conn(0x7fbde005b3a0 0x7fbde005d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:38.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.172+0000 7fbdfa7fc700 1 --2- 172.21.15.47:0/4214669791 >> [v2:172.21.15.47:6826/13100,v1:172.21.15.47:6827/13100] conn(0x7fbde005b3a0 0x7fbde005d840 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:38.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.172+0000 7fbdf3fff700 1 -- 172.21.15.47:0/4214669791 <== osd.1 v2:172.21.15.47:6826/13100 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7fbde005de20 con 0x7fbde005b3a0 2022-06-29T15:42:38.194 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.192+0000 7fbe017f5700 1 -- 172.21.15.47:0/4214669791 --> [v2:172.21.15.47:6826/13100,v1:172.21.15.47:6827/13100] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbdfc13bcf0 con 0x7fbde005b3a0 2022-06-29T15:42:38.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.192+0000 7fbdf3fff700 1 -- 172.21.15.47:0/4214669791 <== osd.1 v2:172.21.15.47:6826/13100 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fbdec000e90 con 0x7fbde005b3a0 2022-06-29T15:42:38.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.192+0000 7fbe017f5700 1 -- 172.21.15.47:0/4214669791 >> [v2:172.21.15.47:6826/13100,v1:172.21.15.47:6827/13100] conn(0x7fbde005b3a0 msgr2=0x7fbde005d840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:38.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.192+0000 7fbe017f5700 1 --2- 172.21.15.47:0/4214669791 >> [v2:172.21.15.47:6826/13100,v1:172.21.15.47:6827/13100] conn(0x7fbde005b3a0 0x7fbde005d840 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:38.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.192+0000 7fbe017f5700 1 -- 172.21.15.47:0/4214669791 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbde0053aa0 msgr2=0x7fbde0055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:38.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.192+0000 7fbe017f5700 1 --2- 172.21.15.47:0/4214669791 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbde0053aa0 0x7fbde0055f60 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fbde400d000 tx=0x7fbde4007a00).stop 2022-06-29T15:42:38.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.192+0000 7fbe017f5700 1 -- 172.21.15.47:0/4214669791 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbdfc0b0cf0 msgr2=0x7fbdfc13cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:38.198 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.192+0000 7fbe017f5700 1 --2- 172.21.15.47:0/4214669791 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbdfc0b0cf0 0x7fbdfc13cdf0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fbdf40032f0 tx=0x7fbdf4016000).stop 2022-06-29T15:42:38.198 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.192+0000 7fbdfb7fe700 1 -- 172.21.15.47:0/4214669791 reap_dead start 2022-06-29T15:42:38.198 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.192+0000 7fbe017f5700 1 -- 172.21.15.47:0/4214669791 shutdown_connections 2022-06-29T15:42:38.198 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.192+0000 7fbe017f5700 1 -- 172.21.15.47:0/4214669791 >> 172.21.15.47:0/4214669791 conn(0x7fbdfc0b98b0 msgr2=0x7fbdfc143220 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:38.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.192+0000 7fbe017f5700 1 -- 172.21.15.47:0/4214669791 shutdown_connections 2022-06-29T15:42:38.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.192+0000 7fbe017f5700 1 -- 172.21.15.47:0/4214669791 wait complete. 2022-06-29T15:42:38.207 INFO:teuthology.orchestra.run.smithi047.stdout:60129542149 2022-06-29T15:42:38.208 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2022-06-29T15:42:38.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.312+0000 7fac0c5fd700 1 Processor -- start 2022-06-29T15:42:38.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.312+0000 7fac0c5fd700 1 -- start start 2022-06-29T15:42:38.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.316+0000 7fac0c5fd700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fac040b0b00 0x7fac04097af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:38.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.316+0000 7fac0c5fd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fac04097fe0 0x7fac04098430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:38.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.316+0000 7fac0c5fd700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac040b19f0 con 0x7fac040615f0 2022-06-29T15:42:38.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.316+0000 7fac0c5fd700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac040619d0 con 0x7fac04097fe0 2022-06-29T15:42:38.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.316+0000 7fac0c5fd700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac040b1080 con 0x7fac040b0b00 2022-06-29T15:42:38.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.316+0000 7fac0ab9a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fac04097fe0 0x7fac04098430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:38.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.316+0000 7fac0a399700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fac040615f0 0x7fac041441c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59660/0 (socket says 172.21.15.47:59660) 2022-06-29T15:42:38.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.316+0000 7fac0a399700 1 -- 172.21.15.47:0/703923516 learned_addr learned my addr 172.21.15.47:0/703923516 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:38.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.316+0000 7fac09b98700 1 --2- 172.21.15.47:0/703923516 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fac040b0b00 0x7fac04097af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:38.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.316+0000 7fac09397700 1 -- 172.21.15.47:0/703923516 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3510070219 0 0) 0x7fac040b19f0 con 0x7fac040615f0 2022-06-29T15:42:38.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.316+0000 7fac09397700 1 -- 172.21.15.47:0/703923516 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabec003260 con 0x7fac040615f0 2022-06-29T15:42:38.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.316+0000 7fac0ab9a700 1 -- 172.21.15.47:0/703923516 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fac040b0b00 msgr2=0x7fac04097af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:38.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.316+0000 7fac0ab9a700 1 --2- 172.21.15.47:0/703923516 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fac040b0b00 0x7fac04097af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:38.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.316+0000 7fac0ab9a700 1 -- 172.21.15.47:0/703923516 >> v1:172.21.15.47:6789/0 conn(0x7fac040615f0 legacy=0x7fac041441c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:38.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.316+0000 7fac0ab9a700 1 -- 172.21.15.47:0/703923516 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac040b1c20 con 0x7fac04097fe0 2022-06-29T15:42:38.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.316+0000 7fac09b98700 1 --2- 172.21.15.47:0/703923516 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fac040b0b00 0x7fac04097af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:42:38.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.316+0000 7fac0ab9a700 1 --2- 172.21.15.47:0/703923516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac04097fe0 0x7fac04098430 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7fabfc0032f0 tx=0x7fabfc0091d0).ready entity=mon.0 client_cookie=e76c2ec14136064e server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:38.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.316+0000 7fac09397700 1 -- 172.21.15.47:0/703923516 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fabfc00b600 con 0x7fac04097fe0 2022-06-29T15:42:38.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.316+0000 7fac09397700 1 -- 172.21.15.47:0/703923516 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabfc00b760 con 0x7fac04097fe0 2022-06-29T15:42:38.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.316+0000 7fac09397700 1 -- 172.21.15.47:0/703923516 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fabfc00b8d0 con 0x7fac04097fe0 2022-06-29T15:42:38.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.320+0000 7fac0c5fd700 1 -- 172.21.15.47:0/703923516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac04097fe0 msgr2=0x7fac04098430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:38.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.320+0000 7fac0c5fd700 1 --2- 172.21.15.47:0/703923516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac04097fe0 0x7fac04098430 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7fabfc0032f0 tx=0x7fabfc0091d0).stop 2022-06-29T15:42:38.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.320+0000 7fac0c5fd700 1 -- 172.21.15.47:0/703923516 shutdown_connections 2022-06-29T15:42:38.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.320+0000 7fac0c5fd700 1 --2- 172.21.15.47:0/703923516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac04097fe0 0x7fac04098430 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:38.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.320+0000 7fac0c5fd700 1 --2- 172.21.15.47:0/703923516 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fac040b0b00 0x7fac04097af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:38.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.320+0000 7fac0c5fd700 1 -- 172.21.15.47:0/703923516 >> 172.21.15.47:0/703923516 conn(0x7fac0405f0b0 msgr2=0x7fac0405f490 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:38.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.320+0000 7fac0c5fd700 1 -- 172.21.15.47:0/703923516 shutdown_connections 2022-06-29T15:42:38.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.320+0000 7fac0c5fd700 1 -- 172.21.15.47:0/703923516 wait complete. 2022-06-29T15:42:38.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.320+0000 7fac0c5fd700 1 Processor -- start 2022-06-29T15:42:38.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.320+0000 7fac0c5fd700 1 -- start start 2022-06-29T15:42:38.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.324+0000 7fac0c5fd700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fac040615f0 0x7fac040b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:38.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.324+0000 7fac0c5fd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac04097fe0 0x7fac0413b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:38.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.324+0000 7fac0c5fd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac040b0b00 0x7fac04152430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:38.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.324+0000 7fac0c5fd700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac04140e30 con 0x7fac04097fe0 2022-06-29T15:42:38.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.324+0000 7fac0c5fd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac04155d00 con 0x7fac040b0b00 2022-06-29T15:42:38.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.324+0000 7fac0c5fd700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac04155fa0 con 0x7fac040615f0 2022-06-29T15:42:38.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.324+0000 7fac09b98700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac04097fe0 0x7fac0413b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:38.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.324+0000 7fac09b98700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac04097fe0 0x7fac0413b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45882/0 (socket says 172.21.15.47:45882) 2022-06-29T15:42:38.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.324+0000 7fac09b98700 1 -- 172.21.15.47:0/2759455157 learned_addr learned my addr 172.21.15.47:0/2759455157 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:38.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.324+0000 7fac0a399700 1 --2- 172.21.15.47:0/2759455157 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fac040615f0 0x7fac040b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:38.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.324+0000 7fac0ab9a700 1 --2- 172.21.15.47:0/2759455157 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac040b0b00 0x7fac04152430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:38.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.324+0000 7fac09b98700 1 -- 172.21.15.47:0/2759455157 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fac040615f0 msgr2=0x7fac040b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:38.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.324+0000 7fac09b98700 1 --2- 172.21.15.47:0/2759455157 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fac040615f0 0x7fac040b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:38.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.324+0000 7fac09b98700 1 -- 172.21.15.47:0/2759455157 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac040b0b00 msgr2=0x7fac04152430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:38.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.324+0000 7fac09b98700 1 --2- 172.21.15.47:0/2759455157 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac040b0b00 0x7fac04152430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:38.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.324+0000 7fac09b98700 1 -- 172.21.15.47:0/2759455157 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf400b180 con 0x7fac04097fe0 2022-06-29T15:42:38.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.324+0000 7fac0ab9a700 1 --2- 172.21.15.47:0/2759455157 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac040b0b00 0x7fac04152430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:42:38.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.324+0000 7fac09b98700 1 --2- 172.21.15.47:0/2759455157 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac04097fe0 0x7fac0413b8f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fabf400c000 tx=0x7fabf4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:38.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.324+0000 7fabf37fe700 1 -- 172.21.15.47:0/2759455157 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fabf400c7e0 con 0x7fac04097fe0 2022-06-29T15:42:38.335 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.324+0000 7fabf37fe700 1 -- 172.21.15.47:0/2759455157 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabf400c940 con 0x7fac04097fe0 2022-06-29T15:42:38.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.324+0000 7fac0c5fd700 1 -- 172.21.15.47:0/2759455157 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabfc009470 con 0x7fac04097fe0 2022-06-29T15:42:38.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.324+0000 7fabf37fe700 1 -- 172.21.15.47:0/2759455157 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fabf400cca0 con 0x7fac04097fe0 2022-06-29T15:42:38.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.324+0000 7fac0c5fd700 1 -- 172.21.15.47:0/2759455157 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac041564e0 con 0x7fac04097fe0 2022-06-29T15:42:38.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.328+0000 7fac0c5fd700 1 -- 172.21.15.47:0/2759455157 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fac041566a0 con 0x7fac04097fe0 2022-06-29T15:42:38.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.328+0000 7fabf37fe700 1 -- 172.21.15.47:0/2759455157 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fabf401c460 con 0x7fac04097fe0 2022-06-29T15:42:38.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.328+0000 7fabf37fe700 1 --2- 172.21.15.47:0/2759455157 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fabe4053b30 0x7fabe4055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:38.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.328+0000 7fabf37fe700 1 -- 172.21.15.47:0/2759455157 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fabf406c5b0 con 0x7fac04097fe0 2022-06-29T15:42:38.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.328+0000 7fac0a399700 1 --2- 172.21.15.47:0/2759455157 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fabe4053b30 0x7fabe4055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:38.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.328+0000 7fabf37fe700 1 --2- 172.21.15.47:0/2759455157 >> [v2:172.21.15.47:6810/13098,v1:172.21.15.47:6811/13098] conn(0x7fabe405b4b0 0x7fabe405d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:38.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.328+0000 7fabf37fe700 1 -- 172.21.15.47:0/2759455157 --> [v2:172.21.15.47:6810/13098,v1:172.21.15.47:6811/13098] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fabe405df30 con 0x7fabe405b4b0 2022-06-29T15:42:38.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.328+0000 7fabf37fe700 1 -- 172.21.15.47:0/2759455157 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fabf406ec40 con 0x7fac04097fe0 2022-06-29T15:42:38.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.328+0000 7fac0ab9a700 1 --2- 172.21.15.47:0/2759455157 >> [v2:172.21.15.47:6810/13098,v1:172.21.15.47:6811/13098] conn(0x7fabe405b4b0 0x7fabe405d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:38.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.328+0000 7fac0a399700 1 --2- 172.21.15.47:0/2759455157 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fabe4053b30 0x7fabe4055ff0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fac0000d000 tx=0x7fac00009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:38.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.328+0000 7fac0ab9a700 1 --2- 172.21.15.47:0/2759455157 >> [v2:172.21.15.47:6810/13098,v1:172.21.15.47:6811/13098] conn(0x7fabe405b4b0 0x7fabe405d950 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:38.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.328+0000 7fabf37fe700 1 -- 172.21.15.47:0/2759455157 <== osd.2 v2:172.21.15.47:6810/13098 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7fabe405df30 con 0x7fabe405b4b0 2022-06-29T15:42:38.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.348+0000 7fac0c5fd700 1 -- 172.21.15.47:0/2759455157 --> [v2:172.21.15.47:6810/13098,v1:172.21.15.47:6811/13098] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fac04094990 con 0x7fabe405b4b0 2022-06-29T15:42:38.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.348+0000 7fabf37fe700 1 -- 172.21.15.47:0/2759455157 <== osd.2 v2:172.21.15.47:6810/13098 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fac04094990 con 0x7fabe405b4b0 2022-06-29T15:42:38.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.348+0000 7fac0c5fd700 1 -- 172.21.15.47:0/2759455157 >> [v2:172.21.15.47:6810/13098,v1:172.21.15.47:6811/13098] conn(0x7fabe405b4b0 msgr2=0x7fabe405d950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:38.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.348+0000 7fac0c5fd700 1 --2- 172.21.15.47:0/2759455157 >> [v2:172.21.15.47:6810/13098,v1:172.21.15.47:6811/13098] conn(0x7fabe405b4b0 0x7fabe405d950 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:38.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.348+0000 7fac0c5fd700 1 -- 172.21.15.47:0/2759455157 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fabe4053b30 msgr2=0x7fabe4055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:38.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.348+0000 7fac0c5fd700 1 --2- 172.21.15.47:0/2759455157 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fabe4053b30 0x7fabe4055ff0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fac0000d000 tx=0x7fac00009040).stop 2022-06-29T15:42:38.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.348+0000 7fac0c5fd700 1 -- 172.21.15.47:0/2759455157 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac04097fe0 msgr2=0x7fac0413b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:38.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.348+0000 7fac0c5fd700 1 --2- 172.21.15.47:0/2759455157 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac04097fe0 0x7fac0413b8f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fabf400c000 tx=0x7fabf4006df0).stop 2022-06-29T15:42:38.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.348+0000 7fac0ab9a700 1 -- 172.21.15.47:0/2759455157 reap_dead start 2022-06-29T15:42:38.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.348+0000 7fac0c5fd700 1 -- 172.21.15.47:0/2759455157 shutdown_connections 2022-06-29T15:42:38.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.348+0000 7fac0c5fd700 1 -- 172.21.15.47:0/2759455157 >> 172.21.15.47:0/2759455157 conn(0x7fac0405f0b0 msgr2=0x7fac04140040 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:38.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.348+0000 7fac0c5fd700 1 -- 172.21.15.47:0/2759455157 shutdown_connections 2022-06-29T15:42:38.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.348+0000 7fac0c5fd700 1 -- 172.21.15.47:0/2759455157 wait complete. 2022-06-29T15:42:38.369 INFO:teuthology.orchestra.run.smithi047.stdout:60129542149 2022-06-29T15:42:38.370 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2022-06-29T15:42:38.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.480+0000 7fb35013c700 1 Processor -- start 2022-06-29T15:42:38.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.480+0000 7fb35013c700 1 -- start start 2022-06-29T15:42:38.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.480+0000 7fb35013c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb34813cfc0 0x7fb3480b1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:38.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.480+0000 7fb35013c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb348061670 0x7fb348143040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:38.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.480+0000 7fb35013c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb348061af0 0x7fb348143530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:38.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.480+0000 7fb35013c700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3480ae370 con 0x7fb348061af0 2022-06-29T15:42:38.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.480+0000 7fb35013c700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb34805f150 con 0x7fb348061670 2022-06-29T15:42:38.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.480+0000 7fb35013c700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb34805f3f0 con 0x7fb34813cfc0 2022-06-29T15:42:38.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.480+0000 7fb34e6d9700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb348061af0 0x7fb348143530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:38.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.480+0000 7fb34e6d9700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb348061af0 0x7fb348143530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45894/0 (socket says 172.21.15.47:45894) 2022-06-29T15:42:38.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.480+0000 7fb34e6d9700 1 -- 172.21.15.47:0/2291274725 learned_addr learned my addr 172.21.15.47:0/2291274725 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:38.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.480+0000 7fb34ded8700 1 --2- 172.21.15.47:0/2291274725 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb34813cfc0 0x7fb3480b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:38.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.480+0000 7fb34d6d7700 1 --2- 172.21.15.47:0/2291274725 >> v2:172.21.15.64:3300/0 conn(0x7fb348061670 0x7fb348143040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:38.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.480+0000 7fb34e6d9700 1 -- 172.21.15.47:0/2291274725 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb34813cfc0 msgr2=0x7fb3480b1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:38.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.480+0000 7fb34e6d9700 1 --2- 172.21.15.47:0/2291274725 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb34813cfc0 0x7fb3480b1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:38.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.480+0000 7fb34e6d9700 1 -- 172.21.15.47:0/2291274725 >> v2:172.21.15.64:3300/0 conn(0x7fb348061670 msgr2=0x7fb348143040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:38.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.480+0000 7fb34e6d9700 1 --2- 172.21.15.47:0/2291274725 >> v2:172.21.15.64:3300/0 conn(0x7fb348061670 0x7fb348143040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:38.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.480+0000 7fb34e6d9700 1 -- 172.21.15.47:0/2291274725 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb34805f690 con 0x7fb348061af0 2022-06-29T15:42:38.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.480+0000 7fb34d6d7700 1 --2- 172.21.15.47:0/2291274725 >> v2:172.21.15.64:3300/0 conn(0x7fb348061670 0x7fb348143040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:42:38.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.480+0000 7fb34e6d9700 1 --2- 172.21.15.47:0/2291274725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb348061af0 0x7fb348143530 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fb338002f70 tx=0x7fb338006eb0).ready entity=mon.0 client_cookie=110b026f8f6e15dd server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:38.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.484+0000 7fb34ced6700 1 -- 172.21.15.47:0/2291274725 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb33800d570 con 0x7fb348061af0 2022-06-29T15:42:38.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.484+0000 7fb34ced6700 1 -- 172.21.15.47:0/2291274725 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb338000e90 con 0x7fb348061af0 2022-06-29T15:42:38.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.484+0000 7fb34ced6700 1 -- 172.21.15.47:0/2291274725 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb33800d900 con 0x7fb348061af0 2022-06-29T15:42:38.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.484+0000 7fb35013c700 1 -- 172.21.15.47:0/2291274725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb348061af0 msgr2=0x7fb348143530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:38.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.484+0000 7fb35013c700 1 --2- 172.21.15.47:0/2291274725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb348061af0 0x7fb348143530 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fb338002f70 tx=0x7fb338006eb0).stop 2022-06-29T15:42:38.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.484+0000 7fb35013c700 1 -- 172.21.15.47:0/2291274725 shutdown_connections 2022-06-29T15:42:38.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.484+0000 7fb35013c700 1 --2- 172.21.15.47:0/2291274725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb348061af0 0x7fb348143530 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:38.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.484+0000 7fb35013c700 1 --2- 172.21.15.47:0/2291274725 >> v2:172.21.15.64:3300/0 conn(0x7fb348061670 0x7fb348143040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:38.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.484+0000 7fb35013c700 1 --2- 172.21.15.47:0/2291274725 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb34813cfc0 0x7fb3480b1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:38.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.484+0000 7fb35013c700 1 -- 172.21.15.47:0/2291274725 >> 172.21.15.47:0/2291274725 conn(0x7fb34805c510 msgr2=0x7fb348063800 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:38.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.484+0000 7fb35013c700 1 -- 172.21.15.47:0/2291274725 shutdown_connections 2022-06-29T15:42:38.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.484+0000 7fb35013c700 1 -- 172.21.15.47:0/2291274725 wait complete. 2022-06-29T15:42:38.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.484+0000 7fb35013c700 1 Processor -- start 2022-06-29T15:42:38.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.484+0000 7fb35013c700 1 -- start start 2022-06-29T15:42:38.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.488+0000 7fb35013c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb348061670 0x7fb348143140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:38.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.488+0000 7fb35013c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb348061af0 0x7fb34813c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:38.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.488+0000 7fb35013c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb34813cfc0 0x7fb34813c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:38.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.488+0000 7fb35013c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3481429e0 con 0x7fb348061af0 2022-06-29T15:42:38.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.488+0000 7fb35013c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb348147ce0 con 0x7fb34813cfc0 2022-06-29T15:42:38.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.488+0000 7fb35013c700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb348147f80 con 0x7fb348061670 2022-06-29T15:42:38.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.488+0000 7fb34d6d7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb348061af0 0x7fb34813c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:38.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.488+0000 7fb34d6d7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb348061af0 0x7fb34813c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45898/0 (socket says 172.21.15.47:45898) 2022-06-29T15:42:38.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.488+0000 7fb34d6d7700 1 -- 172.21.15.47:0/1045569682 learned_addr learned my addr 172.21.15.47:0/1045569682 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:38.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.488+0000 7fb34ded8700 1 --2- 172.21.15.47:0/1045569682 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb348061670 0x7fb348143140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:38.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.488+0000 7fb34e6d9700 1 --2- 172.21.15.47:0/1045569682 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb34813cfc0 0x7fb34813c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:38.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.488+0000 7fb34d6d7700 1 -- 172.21.15.47:0/1045569682 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb348061670 msgr2=0x7fb348143140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:38.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.488+0000 7fb34d6d7700 1 --2- 172.21.15.47:0/1045569682 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb348061670 0x7fb348143140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:38.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.488+0000 7fb34d6d7700 1 -- 172.21.15.47:0/1045569682 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb34813cfc0 msgr2=0x7fb34813c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:38.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.488+0000 7fb34d6d7700 1 --2- 172.21.15.47:0/1045569682 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb34813cfc0 0x7fb34813c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:38.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.488+0000 7fb34d6d7700 1 -- 172.21.15.47:0/1045569682 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb33800b1e0 con 0x7fb348061af0 2022-06-29T15:42:38.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.488+0000 7fb34d6d7700 1 --2- 172.21.15.47:0/1045569682 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb348061af0 0x7fb34813c1e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fb33c006940 tx=0x7fb33c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:38.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.488+0000 7fb346ffd700 1 -- 172.21.15.47:0/1045569682 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb33c011780 con 0x7fb348061af0 2022-06-29T15:42:38.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.488+0000 7fb346ffd700 1 -- 172.21.15.47:0/1045569682 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb33c0118e0 con 0x7fb348061af0 2022-06-29T15:42:38.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.488+0000 7fb35013c700 1 -- 172.21.15.47:0/1045569682 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb34813cbc0 con 0x7fb348061af0 2022-06-29T15:42:38.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.488+0000 7fb346ffd700 1 -- 172.21.15.47:0/1045569682 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb33c011c40 con 0x7fb348061af0 2022-06-29T15:42:38.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.488+0000 7fb35013c700 1 -- 172.21.15.47:0/1045569682 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb348064e90 con 0x7fb348061af0 2022-06-29T15:42:38.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.488+0000 7fb35013c700 1 -- 172.21.15.47:0/1045569682 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb348142be0 con 0x7fb348061af0 2022-06-29T15:42:38.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.488+0000 7fb346ffd700 1 -- 172.21.15.47:0/1045569682 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb33c012420 con 0x7fb348061af0 2022-06-29T15:42:38.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.492+0000 7fb346ffd700 1 --2- 172.21.15.47:0/1045569682 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb328053aa0 0x7fb328055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:38.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.492+0000 7fb346ffd700 1 -- 172.21.15.47:0/1045569682 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fb33c06cd10 con 0x7fb348061af0 2022-06-29T15:42:38.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.492+0000 7fb34ded8700 1 --2- 172.21.15.47:0/1045569682 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb328053aa0 0x7fb328055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:38.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.492+0000 7fb346ffd700 1 --2- 172.21.15.47:0/1045569682 >> [v2:172.21.15.47:6802/13097,v1:172.21.15.47:6803/13097] conn(0x7fb32805b420 0x7fb32805d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:38.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.492+0000 7fb346ffd700 1 -- 172.21.15.47:0/1045569682 --> [v2:172.21.15.47:6802/13097,v1:172.21.15.47:6803/13097] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb32805dea0 con 0x7fb32805b420 2022-06-29T15:42:38.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.492+0000 7fb346ffd700 1 -- 172.21.15.47:0/1045569682 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb33c06d0f0 con 0x7fb348061af0 2022-06-29T15:42:38.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.492+0000 7fb34e6d9700 1 --2- 172.21.15.47:0/1045569682 >> [v2:172.21.15.47:6802/13097,v1:172.21.15.47:6803/13097] conn(0x7fb32805b420 0x7fb32805d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:38.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.492+0000 7fb34ded8700 1 --2- 172.21.15.47:0/1045569682 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb328053aa0 0x7fb328055f60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fb34000b000 tx=0x7fb340008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:38.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.492+0000 7fb34e6d9700 1 --2- 172.21.15.47:0/1045569682 >> [v2:172.21.15.47:6802/13097,v1:172.21.15.47:6803/13097] conn(0x7fb32805b420 0x7fb32805d8c0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:38.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.492+0000 7fb346ffd700 1 -- 172.21.15.47:0/1045569682 <== osd.3 v2:172.21.15.47:6802/13097 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7fb32805dea0 con 0x7fb32805b420 2022-06-29T15:42:38.516 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.512+0000 7fb35013c700 1 -- 172.21.15.47:0/1045569682 --> [v2:172.21.15.47:6802/13097,v1:172.21.15.47:6803/13097] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb348147b60 con 0x7fb32805b420 2022-06-29T15:42:38.517 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.516+0000 7fb346ffd700 1 -- 172.21.15.47:0/1045569682 <== osd.3 v2:172.21.15.47:6802/13097 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb33800d570 con 0x7fb32805b420 2022-06-29T15:42:38.518 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.516+0000 7fb35013c700 1 -- 172.21.15.47:0/1045569682 >> [v2:172.21.15.47:6802/13097,v1:172.21.15.47:6803/13097] conn(0x7fb32805b420 msgr2=0x7fb32805d8c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:38.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.516+0000 7fb35013c700 1 --2- 172.21.15.47:0/1045569682 >> [v2:172.21.15.47:6802/13097,v1:172.21.15.47:6803/13097] conn(0x7fb32805b420 0x7fb32805d8c0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:38.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.516+0000 7fb35013c700 1 -- 172.21.15.47:0/1045569682 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb328053aa0 msgr2=0x7fb328055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:38.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.516+0000 7fb35013c700 1 --2- 172.21.15.47:0/1045569682 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb328053aa0 0x7fb328055f60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fb34000b000 tx=0x7fb340008040).stop 2022-06-29T15:42:38.520 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.516+0000 7fb35013c700 1 -- 172.21.15.47:0/1045569682 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb348061af0 msgr2=0x7fb34813c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:38.520 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.516+0000 7fb35013c700 1 --2- 172.21.15.47:0/1045569682 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb348061af0 0x7fb34813c1e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fb33c006940 tx=0x7fb33c008040).stop 2022-06-29T15:42:38.520 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.516+0000 7fb34e6d9700 1 -- 172.21.15.47:0/1045569682 reap_dead start 2022-06-29T15:42:38.520 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.516+0000 7fb35013c700 1 -- 172.21.15.47:0/1045569682 shutdown_connections 2022-06-29T15:42:38.520 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.516+0000 7fb35013c700 1 -- 172.21.15.47:0/1045569682 >> 172.21.15.47:0/1045569682 conn(0x7fb34805c510 msgr2=0x7fb3480afb30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:38.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.516+0000 7fb35013c700 1 -- 172.21.15.47:0/1045569682 shutdown_connections 2022-06-29T15:42:38.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.516+0000 7fb35013c700 1 -- 172.21.15.47:0/1045569682 wait complete. 2022-06-29T15:42:38.530 INFO:teuthology.orchestra.run.smithi047.stdout:60129542149 2022-06-29T15:42:38.531 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2022-06-29T15:42:38.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.640+0000 7ff03b62e700 1 Processor -- start 2022-06-29T15:42:38.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.640+0000 7ff03b62e700 1 -- start start 2022-06-29T15:42:38.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.640+0000 7ff03b62e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff0340af190 0x7ff0340abb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:38.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.640+0000 7ff03b62e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0340af710 con 0x7ff0340aca80 2022-06-29T15:42:38.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.640+0000 7ff03b62e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff034060d40 con 0x7ff03405f760 2022-06-29T15:42:38.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.640+0000 7ff03b62e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0340aeec0 con 0x7ff0340af190 2022-06-29T15:42:38.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.640+0000 7ff0393ca700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff0340aca80 0x7ff03413f5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59694/0 (socket says 172.21.15.47:59694) 2022-06-29T15:42:38.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.640+0000 7ff0393ca700 1 -- 172.21.15.47:0/3681731069 learned_addr learned my addr 172.21.15.47:0/3681731069 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:38.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.640+0000 7ff038bc9700 1 --2- 172.21.15.47:0/3681731069 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff0340af190 0x7ff0340abb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:38.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.644+0000 7ff023fff700 1 -- 172.21.15.47:0/3681731069 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 877582138 0 0) 0x7ff0340af710 con 0x7ff0340aca80 2022-06-29T15:42:38.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.644+0000 7ff023fff700 1 -- 172.21.15.47:0/3681731069 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff01c003260 con 0x7ff0340aca80 2022-06-29T15:42:38.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.644+0000 7ff023fff700 1 -- 172.21.15.47:0/3681731069 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1787522248 0 0) 0x7ff034060d40 con 0x7ff03405f760 2022-06-29T15:42:38.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.644+0000 7ff023fff700 1 -- 172.21.15.47:0/3681731069 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0340af710 con 0x7ff03405f760 2022-06-29T15:42:38.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.644+0000 7ff038bc9700 1 -- 172.21.15.47:0/3681731069 >> v1:172.21.15.64:6789/0 conn(0x7ff03405f760 legacy=0x7ff0340abff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:38.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.644+0000 7ff038bc9700 1 -- 172.21.15.47:0/3681731069 >> v1:172.21.15.47:6789/0 conn(0x7ff0340aca80 legacy=0x7ff03413f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:38.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.644+0000 7ff038bc9700 1 -- 172.21.15.47:0/3681731069 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff03413fce0 con 0x7ff0340af190 2022-06-29T15:42:38.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.644+0000 7ff038bc9700 1 --2- 172.21.15.47:0/3681731069 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff0340af190 0x7ff0340abb00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7ff024002f70 tx=0x7ff024006df0).ready entity=mon.2 client_cookie=a337ae5cac542f4 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:38.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.644+0000 7ff023fff700 1 -- 172.21.15.47:0/3681731069 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff02400d5c0 con 0x7ff0340af190 2022-06-29T15:42:38.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.644+0000 7ff023fff700 1 -- 172.21.15.47:0/3681731069 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff024000e90 con 0x7ff0340af190 2022-06-29T15:42:38.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.644+0000 7ff023fff700 1 -- 172.21.15.47:0/3681731069 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff02400d910 con 0x7ff0340af190 2022-06-29T15:42:38.654 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.644+0000 7ff03b62e700 1 -- 172.21.15.47:0/3681731069 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff0340af190 msgr2=0x7ff0340abb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:38.654 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.644+0000 7ff03b62e700 1 --2- 172.21.15.47:0/3681731069 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff0340af190 0x7ff0340abb00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7ff024002f70 tx=0x7ff024006df0).stop 2022-06-29T15:42:38.654 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.644+0000 7ff03b62e700 1 -- 172.21.15.47:0/3681731069 shutdown_connections 2022-06-29T15:42:38.654 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.644+0000 7ff03b62e700 1 --2- 172.21.15.47:0/3681731069 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff0340af190 0x7ff0340abb00 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:38.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.644+0000 7ff03b62e700 1 -- 172.21.15.47:0/3681731069 >> 172.21.15.47:0/3681731069 conn(0x7ff0340626f0 msgr2=0x7ff034062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:38.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.644+0000 7ff03b62e700 1 -- 172.21.15.47:0/3681731069 shutdown_connections 2022-06-29T15:42:38.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.644+0000 7ff03b62e700 1 -- 172.21.15.47:0/3681731069 wait complete. 2022-06-29T15:42:38.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.648+0000 7ff03b62e700 1 Processor -- start 2022-06-29T15:42:38.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.648+0000 7ff03b62e700 1 -- start start 2022-06-29T15:42:38.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.648+0000 7ff03b62e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff03405f760 0x7ff034142d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:38.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.648+0000 7ff03b62e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0340aca80 0x7ff034142050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:38.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.648+0000 7ff03b62e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0340af190 0x7ff034142540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:38.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.648+0000 7ff03b62e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff034130900 con 0x7ff0340aca80 2022-06-29T15:42:38.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.648+0000 7ff03b62e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff034130b30 con 0x7ff0340af190 2022-06-29T15:42:38.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.648+0000 7ff03b62e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff034131120 con 0x7ff03405f760 2022-06-29T15:42:38.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.648+0000 7ff038bc9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0340aca80 0x7ff034142050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:38.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.648+0000 7ff038bc9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0340aca80 0x7ff034142050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45914/0 (socket says 172.21.15.47:45914) 2022-06-29T15:42:38.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.648+0000 7ff038bc9700 1 -- 172.21.15.47:0/2921422036 learned_addr learned my addr 172.21.15.47:0/2921422036 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:38.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.648+0000 7ff0393ca700 1 --2- 172.21.15.47:0/2921422036 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff03405f760 0x7ff034142d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:38.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.648+0000 7ff039bcb700 1 --2- 172.21.15.47:0/2921422036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0340af190 0x7ff034142540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:38.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.648+0000 7ff038bc9700 1 -- 172.21.15.47:0/2921422036 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff03405f760 msgr2=0x7ff034142d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:38.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.648+0000 7ff038bc9700 1 --2- 172.21.15.47:0/2921422036 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff03405f760 0x7ff034142d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:38.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.648+0000 7ff038bc9700 1 -- 172.21.15.47:0/2921422036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0340af190 msgr2=0x7ff034142540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:38.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.648+0000 7ff038bc9700 1 --2- 172.21.15.47:0/2921422036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0340af190 0x7ff034142540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:38.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.648+0000 7ff038bc9700 1 -- 172.21.15.47:0/2921422036 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff02400b180 con 0x7ff0340aca80 2022-06-29T15:42:38.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.648+0000 7ff038bc9700 1 --2- 172.21.15.47:0/2921422036 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0340aca80 0x7ff034142050 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7ff024002f70 tx=0x7ff024014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:38.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.648+0000 7ff0227fc700 1 -- 172.21.15.47:0/2921422036 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff024013e90 con 0x7ff0340aca80 2022-06-29T15:42:38.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.648+0000 7ff0227fc700 1 -- 172.21.15.47:0/2921422036 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff02401e400 con 0x7ff0340aca80 2022-06-29T15:42:38.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.648+0000 7ff03b62e700 1 -- 172.21.15.47:0/2921422036 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff034130e80 con 0x7ff0340aca80 2022-06-29T15:42:38.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.648+0000 7ff0227fc700 1 -- 172.21.15.47:0/2921422036 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff02401e760 con 0x7ff0340aca80 2022-06-29T15:42:38.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.648+0000 7ff03b62e700 1 -- 172.21.15.47:0/2921422036 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0340abd10 con 0x7ff0340aca80 2022-06-29T15:42:38.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.652+0000 7ff03b62e700 1 -- 172.21.15.47:0/2921422036 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff034141d90 con 0x7ff0340aca80 2022-06-29T15:42:38.664 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.652+0000 7ff0227fc700 1 -- 172.21.15.47:0/2921422036 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff024025030 con 0x7ff0340aca80 2022-06-29T15:42:38.664 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.652+0000 7ff0227fc700 1 --2- 172.21.15.47:0/2921422036 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff014053a70 0x7ff014055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:38.664 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.652+0000 7ff0227fc700 1 -- 172.21.15.47:0/2921422036 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7ff02406cd70 con 0x7ff0340aca80 2022-06-29T15:42:38.664 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.652+0000 7ff0393ca700 1 --2- 172.21.15.47:0/2921422036 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff014053a70 0x7ff014055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:38.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.652+0000 7ff0227fc700 1 --2- 172.21.15.47:0/2921422036 >> [v2:172.21.15.64:6816/13396,v1:172.21.15.64:6817/13396] conn(0x7ff01405b280 0x7ff01405d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:38.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.652+0000 7ff0227fc700 1 -- 172.21.15.47:0/2921422036 --> [v2:172.21.15.64:6816/13396,v1:172.21.15.64:6817/13396] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff01405dd00 con 0x7ff01405b280 2022-06-29T15:42:38.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.652+0000 7ff0227fc700 1 -- 172.21.15.47:0/2921422036 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff02406f400 con 0x7ff0340aca80 2022-06-29T15:42:38.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.652+0000 7ff039bcb700 1 --2- 172.21.15.47:0/2921422036 >> [v2:172.21.15.64:6816/13396,v1:172.21.15.64:6817/13396] conn(0x7ff01405b280 0x7ff01405d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:38.666 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.652+0000 7ff0393ca700 1 --2- 172.21.15.47:0/2921422036 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff014053a70 0x7ff014055f30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7ff03000d000 tx=0x7ff030009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:38.666 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.652+0000 7ff039bcb700 1 --2- 172.21.15.47:0/2921422036 >> [v2:172.21.15.64:6816/13396,v1:172.21.15.64:6817/13396] conn(0x7ff01405b280 0x7ff01405d720 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:38.666 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.656+0000 7ff0227fc700 1 -- 172.21.15.47:0/2921422036 <== osd.4 v2:172.21.15.64:6816/13396 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7ff01405dd00 con 0x7ff01405b280 2022-06-29T15:42:38.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.672+0000 7ff03b62e700 1 -- 172.21.15.47:0/2921422036 --> [v2:172.21.15.64:6816/13396,v1:172.21.15.64:6817/13396] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff0340573d0 con 0x7ff01405b280 2022-06-29T15:42:38.676 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.672+0000 7ff0227fc700 1 -- 172.21.15.47:0/2921422036 <== osd.4 v2:172.21.15.64:6816/13396 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff0340573d0 con 0x7ff01405b280 2022-06-29T15:42:38.678 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.676+0000 7ff03b62e700 1 -- 172.21.15.47:0/2921422036 >> [v2:172.21.15.64:6816/13396,v1:172.21.15.64:6817/13396] conn(0x7ff01405b280 msgr2=0x7ff01405d720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:38.678 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.676+0000 7ff03b62e700 1 --2- 172.21.15.47:0/2921422036 >> [v2:172.21.15.64:6816/13396,v1:172.21.15.64:6817/13396] conn(0x7ff01405b280 0x7ff01405d720 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:38.678 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.676+0000 7ff03b62e700 1 -- 172.21.15.47:0/2921422036 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff014053a70 msgr2=0x7ff014055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:38.679 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.676+0000 7ff03b62e700 1 --2- 172.21.15.47:0/2921422036 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff014053a70 0x7ff014055f30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7ff03000d000 tx=0x7ff030009040).stop 2022-06-29T15:42:38.679 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.676+0000 7ff03b62e700 1 -- 172.21.15.47:0/2921422036 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0340aca80 msgr2=0x7ff034142050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:38.679 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.676+0000 7ff03b62e700 1 --2- 172.21.15.47:0/2921422036 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0340aca80 0x7ff034142050 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7ff024002f70 tx=0x7ff024014000).stop 2022-06-29T15:42:38.679 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.676+0000 7ff039bcb700 1 -- 172.21.15.47:0/2921422036 reap_dead start 2022-06-29T15:42:38.680 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.676+0000 7ff03b62e700 1 -- 172.21.15.47:0/2921422036 shutdown_connections 2022-06-29T15:42:38.680 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.676+0000 7ff03b62e700 1 -- 172.21.15.47:0/2921422036 >> 172.21.15.47:0/2921422036 conn(0x7ff0340626f0 msgr2=0x7ff0340ae420 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:38.680 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.676+0000 7ff03b62e700 1 -- 172.21.15.47:0/2921422036 shutdown_connections 2022-06-29T15:42:38.680 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.676+0000 7ff03b62e700 1 -- 172.21.15.47:0/2921422036 wait complete. 2022-06-29T15:42:38.689 INFO:teuthology.orchestra.run.smithi047.stdout:60129542149 2022-06-29T15:42:38.690 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2022-06-29T15:42:38.804 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.800+0000 7fbda17c0700 1 Processor -- start 2022-06-29T15:42:38.804 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.800+0000 7fbda17c0700 1 -- start start 2022-06-29T15:42:38.809 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.804+0000 7fbda17c0700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbd9c0b2120 0x7fbd9c0b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:38.810 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.804+0000 7fbda17c0700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbd9c05f0b0 0x7fbd9c14b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:38.810 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.804+0000 7fbda17c0700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbd9c0b1260 0x7fbd9c147af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:38.810 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.804+0000 7fbda17c0700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd9c147460 con 0x7fbd9c05f0b0 2022-06-29T15:42:38.810 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.804+0000 7fbda17c0700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd9c0b0740 con 0x7fbd9c0b2120 2022-06-29T15:42:38.811 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.804+0000 7fbda17c0700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c0b0970 con 0x7fbd9c0b1260 2022-06-29T15:42:38.811 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.804+0000 7fbd9a7fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbd9c05f0b0 0x7fbd9c14b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:38.811 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.804+0000 7fbd9a7fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbd9c05f0b0 0x7fbd9c14b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45922/0 (socket says 172.21.15.47:45922) 2022-06-29T15:42:38.811 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.804+0000 7fbd9a7fc700 1 -- 172.21.15.47:0/2983627857 learned_addr learned my addr 172.21.15.47:0/2983627857 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:38.812 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.804+0000 7fbd9affd700 1 --2- 172.21.15.47:0/2983627857 >> v2:172.21.15.64:3300/0 conn(0x7fbd9c0b2120 0x7fbd9c0b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:38.812 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.804+0000 7fbd9b7fe700 1 --2- 172.21.15.47:0/2983627857 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbd9c0b1260 0x7fbd9c147af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:38.812 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.804+0000 7fbd9a7fc700 1 -- 172.21.15.47:0/2983627857 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbd9c0b1260 msgr2=0x7fbd9c147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:38.812 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.804+0000 7fbd9a7fc700 1 --2- 172.21.15.47:0/2983627857 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbd9c0b1260 0x7fbd9c147af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:38.813 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.804+0000 7fbd9a7fc700 1 -- 172.21.15.47:0/2983627857 >> v2:172.21.15.64:3300/0 conn(0x7fbd9c0b2120 msgr2=0x7fbd9c0b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:38.813 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.804+0000 7fbd9a7fc700 1 --2- 172.21.15.47:0/2983627857 >> v2:172.21.15.64:3300/0 conn(0x7fbd9c0b2120 0x7fbd9c0b2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:38.813 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.804+0000 7fbd9a7fc700 1 -- 172.21.15.47:0/2983627857 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd9c0b0c10 con 0x7fbd9c05f0b0 2022-06-29T15:42:38.813 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.804+0000 7fbd9b7fe700 1 --2- 172.21.15.47:0/2983627857 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbd9c0b1260 0x7fbd9c147af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:42:38.813 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.804+0000 7fbd9a7fc700 1 --2- 172.21.15.47:0/2983627857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd9c05f0b0 0x7fbd9c14b520 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fbd8c002f70 tx=0x7fbd8c006eb0).ready entity=mon.0 client_cookie=1fbbe3ed51f2cd5b server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:38.814 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.804+0000 7fbd99ffb700 1 -- 172.21.15.47:0/2983627857 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd8c00d570 con 0x7fbd9c05f0b0 2022-06-29T15:42:38.814 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.804+0000 7fbd99ffb700 1 -- 172.21.15.47:0/2983627857 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd8c000e90 con 0x7fbd9c05f0b0 2022-06-29T15:42:38.814 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.804+0000 7fbd99ffb700 1 -- 172.21.15.47:0/2983627857 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd8c00d900 con 0x7fbd9c05f0b0 2022-06-29T15:42:38.814 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.804+0000 7fbda17c0700 1 -- 172.21.15.47:0/2983627857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd9c05f0b0 msgr2=0x7fbd9c14b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:38.815 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.804+0000 7fbda17c0700 1 --2- 172.21.15.47:0/2983627857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd9c05f0b0 0x7fbd9c14b520 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fbd8c002f70 tx=0x7fbd8c006eb0).stop 2022-06-29T15:42:38.815 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.804+0000 7fbda17c0700 1 -- 172.21.15.47:0/2983627857 shutdown_connections 2022-06-29T15:42:38.815 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.804+0000 7fbda17c0700 1 --2- 172.21.15.47:0/2983627857 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbd9c0b1260 0x7fbd9c147af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:38.815 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.804+0000 7fbda17c0700 1 --2- 172.21.15.47:0/2983627857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd9c05f0b0 0x7fbd9c14b520 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:38.816 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.804+0000 7fbda17c0700 1 --2- 172.21.15.47:0/2983627857 >> v2:172.21.15.64:3300/0 conn(0x7fbd9c0b2120 0x7fbd9c0b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:38.816 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.804+0000 7fbda17c0700 1 -- 172.21.15.47:0/2983627857 >> 172.21.15.47:0/2983627857 conn(0x7fbd9c0626f0 msgr2=0x7fbd9c062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:38.816 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.804+0000 7fbda17c0700 1 -- 172.21.15.47:0/2983627857 shutdown_connections 2022-06-29T15:42:38.816 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.804+0000 7fbda17c0700 1 -- 172.21.15.47:0/2983627857 wait complete. 2022-06-29T15:42:38.816 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.808+0000 7fbda17c0700 1 Processor -- start 2022-06-29T15:42:38.817 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.808+0000 7fbda17c0700 1 -- start start 2022-06-29T15:42:38.820 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.808+0000 7fbda17c0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd9c0b1260 0x7fbd9c1450e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:38.820 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.808+0000 7fbda17c0700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbd9c0b2120 0x7fbd9c14a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:38.821 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.808+0000 7fbda17c0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd9c0ae940 0x7fbd9c159f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:38.821 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.808+0000 7fbda17c0700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c144580 con 0x7fbd9c0ae940 2022-06-29T15:42:38.821 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.808+0000 7fbda17c0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c144820 con 0x7fbd9c0b1260 2022-06-29T15:42:38.821 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.808+0000 7fbda17c0700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c144ac0 con 0x7fbd9c0b2120 2022-06-29T15:42:38.822 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.808+0000 7fbd9b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd9c0ae940 0x7fbd9c159f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:38.822 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.808+0000 7fbd9b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd9c0ae940 0x7fbd9c159f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45932/0 (socket says 172.21.15.47:45932) 2022-06-29T15:42:38.822 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.808+0000 7fbd9affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd9c0b1260 0x7fbd9c1450e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:38.822 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.808+0000 7fbd9b7fe700 1 -- 172.21.15.47:0/281959577 learned_addr learned my addr 172.21.15.47:0/281959577 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:38.822 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.808+0000 7fbd9a7fc700 1 --2- 172.21.15.47:0/281959577 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbd9c0b2120 0x7fbd9c14a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:38.823 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.808+0000 7fbd9b7fe700 1 -- 172.21.15.47:0/281959577 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbd9c0b2120 msgr2=0x7fbd9c14a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:38.823 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.808+0000 7fbd9b7fe700 1 --2- 172.21.15.47:0/281959577 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbd9c0b2120 0x7fbd9c14a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:38.823 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.808+0000 7fbd9b7fe700 1 -- 172.21.15.47:0/281959577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd9c0b1260 msgr2=0x7fbd9c1450e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:38.823 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.808+0000 7fbd9b7fe700 1 --2- 172.21.15.47:0/281959577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd9c0b1260 0x7fbd9c1450e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:38.824 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.808+0000 7fbd9b7fe700 1 -- 172.21.15.47:0/281959577 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd8c00b1e0 con 0x7fbd9c0ae940 2022-06-29T15:42:38.824 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.808+0000 7fbd9affd700 1 --2- 172.21.15.47:0/281959577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd9c0b1260 0x7fbd9c1450e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:42:38.824 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.808+0000 7fbd9b7fe700 1 --2- 172.21.15.47:0/281959577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd9c0ae940 0x7fbd9c159f00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fbd9400a000 tx=0x7fbd94008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:38.824 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.808+0000 7fbd8bfff700 1 -- 172.21.15.47:0/281959577 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd94013620 con 0x7fbd9c0ae940 2022-06-29T15:42:38.824 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.808+0000 7fbd8bfff700 1 -- 172.21.15.47:0/281959577 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd94013780 con 0x7fbd9c0ae940 2022-06-29T15:42:38.825 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.808+0000 7fbd8bfff700 1 -- 172.21.15.47:0/281959577 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd94013ae0 con 0x7fbd9c0ae940 2022-06-29T15:42:38.825 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.812+0000 7fbda17c0700 1 -- 172.21.15.47:0/281959577 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd9c144d60 con 0x7fbd9c0ae940 2022-06-29T15:42:38.825 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.812+0000 7fbda17c0700 1 -- 172.21.15.47:0/281959577 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd9c15a610 con 0x7fbd9c0ae940 2022-06-29T15:42:38.825 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.812+0000 7fbda17c0700 1 -- 172.21.15.47:0/281959577 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbd9c15aba0 con 0x7fbd9c0ae940 2022-06-29T15:42:38.826 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.812+0000 7fbd8bfff700 1 -- 172.21.15.47:0/281959577 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fbd94013c40 con 0x7fbd9c0ae940 2022-06-29T15:42:38.826 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.812+0000 7fbd8bfff700 1 --2- 172.21.15.47:0/281959577 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbd780539e0 0x7fbd78055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:38.826 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.812+0000 7fbd9affd700 1 --2- 172.21.15.47:0/281959577 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbd780539e0 0x7fbd78055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:38.826 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.812+0000 7fbd8bfff700 1 -- 172.21.15.47:0/281959577 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fbd9406be50 con 0x7fbd9c0ae940 2022-06-29T15:42:38.827 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.812+0000 7fbd8bfff700 1 --2- 172.21.15.47:0/281959577 >> [v2:172.21.15.64:6800/13393,v1:172.21.15.64:6801/13393] conn(0x7fbd7805b360 0x7fbd7805d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:38.827 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.812+0000 7fbd8bfff700 1 -- 172.21.15.47:0/281959577 --> [v2:172.21.15.64:6800/13393,v1:172.21.15.64:6801/13393] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbd7805de80 con 0x7fbd7805b360 2022-06-29T15:42:38.827 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.812+0000 7fbd8bfff700 1 -- 172.21.15.47:0/281959577 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd9406e4e0 con 0x7fbd9c0ae940 2022-06-29T15:42:38.827 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.812+0000 7fbd9affd700 1 --2- 172.21.15.47:0/281959577 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbd780539e0 0x7fbd78055ea0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fbd9000c000 tx=0x7fbd90008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:38.827 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.812+0000 7fbd9a7fc700 1 --2- 172.21.15.47:0/281959577 >> [v2:172.21.15.64:6800/13393,v1:172.21.15.64:6801/13393] conn(0x7fbd7805b360 0x7fbd7805d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:38.828 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.816+0000 7fbd9a7fc700 1 --2- 172.21.15.47:0/281959577 >> [v2:172.21.15.64:6800/13393,v1:172.21.15.64:6801/13393] conn(0x7fbd7805b360 0x7fbd7805d800 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:38.828 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.816+0000 7fbd8bfff700 1 -- 172.21.15.47:0/281959577 <== osd.5 v2:172.21.15.64:6800/13393 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7fbd7805de80 con 0x7fbd7805b360 2022-06-29T15:42:38.836 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.832+0000 7fbda17c0700 1 -- 172.21.15.47:0/281959577 --> [v2:172.21.15.64:6800/13393,v1:172.21.15.64:6801/13393] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbd9c144ef0 con 0x7fbd7805b360 2022-06-29T15:42:38.836 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.832+0000 7fbd8bfff700 1 -- 172.21.15.47:0/281959577 <== osd.5 v2:172.21.15.64:6800/13393 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fbd9c144ef0 con 0x7fbd7805b360 2022-06-29T15:42:38.838 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.836+0000 7fbda17c0700 1 -- 172.21.15.47:0/281959577 >> [v2:172.21.15.64:6800/13393,v1:172.21.15.64:6801/13393] conn(0x7fbd7805b360 msgr2=0x7fbd7805d800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:38.838 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.836+0000 7fbda17c0700 1 --2- 172.21.15.47:0/281959577 >> [v2:172.21.15.64:6800/13393,v1:172.21.15.64:6801/13393] conn(0x7fbd7805b360 0x7fbd7805d800 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:38.839 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.836+0000 7fbda17c0700 1 -- 172.21.15.47:0/281959577 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbd780539e0 msgr2=0x7fbd78055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:38.839 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.836+0000 7fbda17c0700 1 --2- 172.21.15.47:0/281959577 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbd780539e0 0x7fbd78055ea0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fbd9000c000 tx=0x7fbd90008040).stop 2022-06-29T15:42:38.839 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.836+0000 7fbda17c0700 1 -- 172.21.15.47:0/281959577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd9c0ae940 msgr2=0x7fbd9c159f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:38.839 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.836+0000 7fbda17c0700 1 --2- 172.21.15.47:0/281959577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd9c0ae940 0x7fbd9c159f00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fbd9400a000 tx=0x7fbd94008040).stop 2022-06-29T15:42:38.840 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.836+0000 7fbd9b7fe700 1 -- 172.21.15.47:0/281959577 reap_dead start 2022-06-29T15:42:38.840 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.836+0000 7fbda17c0700 1 -- 172.21.15.47:0/281959577 shutdown_connections 2022-06-29T15:42:38.840 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.836+0000 7fbda17c0700 1 -- 172.21.15.47:0/281959577 >> 172.21.15.47:0/281959577 conn(0x7fbd9c0626f0 msgr2=0x7fbd9c0ab650 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:38.840 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.836+0000 7fbda17c0700 1 -- 172.21.15.47:0/281959577 shutdown_connections 2022-06-29T15:42:38.841 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.836+0000 7fbda17c0700 1 -- 172.21.15.47:0/281959577 wait complete. 2022-06-29T15:42:38.856 INFO:teuthology.orchestra.run.smithi047.stdout:60129542150 2022-06-29T15:42:38.857 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2022-06-29T15:42:38.969 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.968+0000 7f21cf15c700 1 Processor -- start 2022-06-29T15:42:38.970 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.968+0000 7f21cf15c700 1 -- start start 2022-06-29T15:42:38.971 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.968+0000 7f21cf15c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f21c8143bd0 0x7f21c805c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:38.972 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.968+0000 7f21cf15c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f21c80b2c40 0x7f21c80b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:38.972 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.968+0000 7f21cf15c700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21c8142f90 con 0x7f21c805c510 2022-06-29T15:42:38.972 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.968+0000 7f21cf15c700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21c81431f0 con 0x7f21c80b2c40 2022-06-29T15:42:38.972 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.968+0000 7f21cf15c700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21c8143490 con 0x7f21c8143bd0 2022-06-29T15:42:38.973 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.968+0000 7f21ce95b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f21c80b2c40 0x7f21c80b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:38.973 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.968+0000 7f21ce95b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f21c80b2c40 0x7f21c80b30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45942/0 (socket says 172.21.15.47:45942) 2022-06-29T15:42:38.973 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.968+0000 7f21ce95b700 1 -- 172.21.15.47:0/419169824 learned_addr learned my addr 172.21.15.47:0/419169824 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:38.973 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.968+0000 7f21cd959700 1 --2- 172.21.15.47:0/419169824 >> v2:172.21.15.64:3300/0 conn(0x7f21c8143bd0 0x7f21c805c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:38.973 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.968+0000 7f21ce95b700 1 -- 172.21.15.47:0/419169824 >> v2:172.21.15.64:3300/0 conn(0x7f21c8143bd0 msgr2=0x7f21c805c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:38.974 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.968+0000 7f21ce95b700 1 --2- 172.21.15.47:0/419169824 >> v2:172.21.15.64:3300/0 conn(0x7f21c8143bd0 0x7f21c805c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:38.974 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.968+0000 7f21ce95b700 1 -- 172.21.15.47:0/419169824 >> v1:172.21.15.64:6789/0 conn(0x7f21c805c510 legacy=0x7f21c80b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:38.974 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.968+0000 7f21ce95b700 1 -- 172.21.15.47:0/419169824 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c8143730 con 0x7f21c80b2c40 2022-06-29T15:42:38.974 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.968+0000 7f21ce95b700 1 --2- 172.21.15.47:0/419169824 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21c80b2c40 0x7f21c80b30d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f21c00032f0 tx=0x7f21c0009290).ready entity=mon.0 client_cookie=62b907f538905954 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:38.975 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.968+0000 7f21cd158700 1 -- 172.21.15.47:0/419169824 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f21c000b870 con 0x7f21c80b2c40 2022-06-29T15:42:38.975 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.968+0000 7f21cd158700 1 -- 172.21.15.47:0/419169824 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c000b9d0 con 0x7f21c80b2c40 2022-06-29T15:42:38.975 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.968+0000 7f21cd158700 1 -- 172.21.15.47:0/419169824 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f21c000bb40 con 0x7f21c80b2c40 2022-06-29T15:42:38.980 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.968+0000 7f21cf15c700 1 -- 172.21.15.47:0/419169824 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21c80b2c40 msgr2=0x7f21c80b30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:38.980 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.968+0000 7f21cf15c700 1 --2- 172.21.15.47:0/419169824 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21c80b2c40 0x7f21c80b30d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f21c00032f0 tx=0x7f21c0009290).stop 2022-06-29T15:42:38.981 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.968+0000 7f21cf15c700 1 -- 172.21.15.47:0/419169824 shutdown_connections 2022-06-29T15:42:38.981 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.968+0000 7f21cf15c700 1 --2- 172.21.15.47:0/419169824 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21c80b2c40 0x7f21c80b30d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:38.981 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.972+0000 7f21cf15c700 1 --2- 172.21.15.47:0/419169824 >> v2:172.21.15.64:3300/0 conn(0x7f21c8143bd0 0x7f21c805c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:38.981 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.972+0000 7f21cf15c700 1 -- 172.21.15.47:0/419169824 >> 172.21.15.47:0/419169824 conn(0x7f21c8060e00 msgr2=0x7f21c80611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:38.982 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.972+0000 7f21cf15c700 1 -- 172.21.15.47:0/419169824 shutdown_connections 2022-06-29T15:42:38.982 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.972+0000 7f21cf15c700 1 -- 172.21.15.47:0/419169824 wait complete. 2022-06-29T15:42:38.982 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.972+0000 7f21cf15c700 1 Processor -- start 2022-06-29T15:42:38.982 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.972+0000 7f21cf15c700 1 -- start start 2022-06-29T15:42:38.983 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.972+0000 7f21cf15c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f21c805c510 0x7f21c80b5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:38.983 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.972+0000 7f21cf15c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21c80b2c40 0x7f21c8152830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:38.983 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.972+0000 7f21cf15c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21c8143bd0 0x7f21c8152d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:38.983 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.972+0000 7f21cf15c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c81521a0 con 0x7f21c80b2c40 2022-06-29T15:42:38.983 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.972+0000 7f21cf15c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c80afcb0 con 0x7f21c8143bd0 2022-06-29T15:42:38.984 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.972+0000 7f21cf15c700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21c8152390 con 0x7f21c805c510 2022-06-29T15:42:38.984 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.972+0000 7f21cd959700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21c80b2c40 0x7f21c8152830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:38.984 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.972+0000 7f21cd959700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21c80b2c40 0x7f21c8152830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45946/0 (socket says 172.21.15.47:45946) 2022-06-29T15:42:38.984 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.972+0000 7f21cd959700 1 -- 172.21.15.47:0/2351679681 learned_addr learned my addr 172.21.15.47:0/2351679681 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:38.984 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.972+0000 7f21ce15a700 1 --2- 172.21.15.47:0/2351679681 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f21c805c510 0x7f21c80b5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:38.985 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.972+0000 7f21ce95b700 1 --2- 172.21.15.47:0/2351679681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21c8143bd0 0x7f21c8152d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:38.985 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.972+0000 7f21cd959700 1 -- 172.21.15.47:0/2351679681 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f21c805c510 msgr2=0x7f21c80b5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:38.985 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.972+0000 7f21cd959700 1 --2- 172.21.15.47:0/2351679681 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f21c805c510 0x7f21c80b5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:38.985 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.972+0000 7f21cd959700 1 -- 172.21.15.47:0/2351679681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21c8143bd0 msgr2=0x7f21c8152d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:38.986 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.972+0000 7f21cd959700 1 --2- 172.21.15.47:0/2351679681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21c8143bd0 0x7f21c8152d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:38.986 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.972+0000 7f21cd959700 1 -- 172.21.15.47:0/2351679681 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c0009530 con 0x7f21c80b2c40 2022-06-29T15:42:38.986 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.972+0000 7f21ce15a700 1 --2- 172.21.15.47:0/2351679681 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f21c805c510 0x7f21c80b5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:42:38.986 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.972+0000 7f21cd959700 1 --2- 172.21.15.47:0/2351679681 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21c80b2c40 0x7f21c8152830 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f21c400c000 tx=0x7f21c4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:38.987 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.972+0000 7f21b77fe700 1 -- 172.21.15.47:0/2351679681 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f21c4010780 con 0x7f21c80b2c40 2022-06-29T15:42:38.987 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.972+0000 7f21b77fe700 1 -- 172.21.15.47:0/2351679681 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c40108e0 con 0x7f21c80b2c40 2022-06-29T15:42:38.987 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.972+0000 7f21cf15c700 1 -- 172.21.15.47:0/2351679681 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21c80b6760 con 0x7f21c80b2c40 2022-06-29T15:42:38.987 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.972+0000 7f21b77fe700 1 -- 172.21.15.47:0/2351679681 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f21c4010c40 con 0x7f21c80b2c40 2022-06-29T15:42:38.987 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.976+0000 7f21cf15c700 1 -- 172.21.15.47:0/2351679681 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21c80af3f0 con 0x7f21c80b2c40 2022-06-29T15:42:38.988 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.976+0000 7f21cf15c700 1 -- 172.21.15.47:0/2351679681 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f21c80afb30 con 0x7f21c80b2c40 2022-06-29T15:42:38.988 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.976+0000 7f21b77fe700 1 -- 172.21.15.47:0/2351679681 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f21c401c460 con 0x7f21c80b2c40 2022-06-29T15:42:38.988 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.976+0000 7f21b77fe700 1 --2- 172.21.15.47:0/2351679681 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f21a8053980 0x7f21a8055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:38.988 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.976+0000 7f21b77fe700 1 -- 172.21.15.47:0/2351679681 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f21c406c490 con 0x7f21c80b2c40 2022-06-29T15:42:38.988 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.976+0000 7f21ce15a700 1 --2- 172.21.15.47:0/2351679681 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f21a8053980 0x7f21a8055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:38.989 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.976+0000 7f21b77fe700 1 --2- 172.21.15.47:0/2351679681 >> [v2:172.21.15.64:6824/13395,v1:172.21.15.64:6825/13395] conn(0x7f21a805b3a0 0x7f21a805d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:38.989 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.976+0000 7f21b77fe700 1 -- 172.21.15.47:0/2351679681 --> [v2:172.21.15.64:6824/13395,v1:172.21.15.64:6825/13395] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f21a805dec0 con 0x7f21a805b3a0 2022-06-29T15:42:38.989 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.976+0000 7f21b77fe700 1 -- 172.21.15.47:0/2351679681 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f21c406eb20 con 0x7f21c80b2c40 2022-06-29T15:42:38.989 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.976+0000 7f21ce15a700 1 --2- 172.21.15.47:0/2351679681 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f21a8053980 0x7f21a8055e40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f21b800b000 tx=0x7f21b8009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:38.990 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.976+0000 7f21ce95b700 1 --2- 172.21.15.47:0/2351679681 >> [v2:172.21.15.64:6824/13395,v1:172.21.15.64:6825/13395] conn(0x7f21a805b3a0 0x7f21a805d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:38.990 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.976+0000 7f21ce95b700 1 --2- 172.21.15.47:0/2351679681 >> [v2:172.21.15.64:6824/13395,v1:172.21.15.64:6825/13395] conn(0x7f21a805b3a0 0x7f21a805d840 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:38.990 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:38.980+0000 7f21b77fe700 1 -- 172.21.15.47:0/2351679681 <== osd.6 v2:172.21.15.64:6824/13395 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f21a805dec0 con 0x7f21a805b3a0 2022-06-29T15:42:39.001 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.000+0000 7f21cf15c700 1 -- 172.21.15.47:0/2351679681 --> [v2:172.21.15.64:6824/13395,v1:172.21.15.64:6825/13395] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f21c8152550 con 0x7f21a805b3a0 2022-06-29T15:42:39.002 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.000+0000 7f21b77fe700 1 -- 172.21.15.47:0/2351679681 <== osd.6 v2:172.21.15.64:6824/13395 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f21c8152550 con 0x7f21a805b3a0 2022-06-29T15:42:39.003 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.000+0000 7f21cf15c700 1 -- 172.21.15.47:0/2351679681 >> [v2:172.21.15.64:6824/13395,v1:172.21.15.64:6825/13395] conn(0x7f21a805b3a0 msgr2=0x7f21a805d840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:39.003 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.000+0000 7f21cf15c700 1 --2- 172.21.15.47:0/2351679681 >> [v2:172.21.15.64:6824/13395,v1:172.21.15.64:6825/13395] conn(0x7f21a805b3a0 0x7f21a805d840 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:39.003 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.000+0000 7f21cf15c700 1 -- 172.21.15.47:0/2351679681 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f21a8053980 msgr2=0x7f21a8055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:39.004 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.000+0000 7f21cf15c700 1 --2- 172.21.15.47:0/2351679681 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f21a8053980 0x7f21a8055e40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f21b800b000 tx=0x7f21b8009040).stop 2022-06-29T15:42:39.004 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.000+0000 7f21cf15c700 1 -- 172.21.15.47:0/2351679681 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21c80b2c40 msgr2=0x7f21c8152830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:39.004 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.000+0000 7f21cf15c700 1 --2- 172.21.15.47:0/2351679681 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21c80b2c40 0x7f21c8152830 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f21c400c000 tx=0x7f21c4006df0).stop 2022-06-29T15:42:39.004 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.000+0000 7f21ce95b700 1 -- 172.21.15.47:0/2351679681 reap_dead start 2022-06-29T15:42:39.005 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.000+0000 7f21cf15c700 1 -- 172.21.15.47:0/2351679681 shutdown_connections 2022-06-29T15:42:39.005 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.000+0000 7f21cf15c700 1 -- 172.21.15.47:0/2351679681 >> 172.21.15.47:0/2351679681 conn(0x7f21c8060e00 msgr2=0x7f21c80b7f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:39.005 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.000+0000 7f21cf15c700 1 -- 172.21.15.47:0/2351679681 shutdown_connections 2022-06-29T15:42:39.005 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.000+0000 7f21cf15c700 1 -- 172.21.15.47:0/2351679681 wait complete. 2022-06-29T15:42:39.015 INFO:teuthology.orchestra.run.smithi047.stdout:60129542150 2022-06-29T15:42:39.016 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2022-06-29T15:42:39.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.128+0000 7f20a32b1700 1 Processor -- start 2022-06-29T15:42:39.130 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.128+0000 7f20a32b1700 1 -- start start 2022-06-29T15:42:39.132 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.128+0000 7f20a32b1700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f209c0b7860 0x7f209c0b3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:39.132 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.128+0000 7f20a32b1700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f209c0b2d00 0x7f209c14a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:39.132 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.128+0000 7f20a32b1700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f209c0aac20 con 0x7f209c14a4f0 2022-06-29T15:42:39.133 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.128+0000 7f20a32b1700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f209c0ab2f0 con 0x7f209c0b2d00 2022-06-29T15:42:39.133 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.128+0000 7f20a32b1700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f209c0af9f0 con 0x7f209c0b7860 2022-06-29T15:42:39.133 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.128+0000 7f20a1aae700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f209c0b2d00 0x7f209c14a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:39.133 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.128+0000 7f20a1aae700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f209c0b2d00 0x7f209c14a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45956/0 (socket says 172.21.15.47:45956) 2022-06-29T15:42:39.133 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.128+0000 7f20a1aae700 1 -- 172.21.15.47:0/1977820770 learned_addr learned my addr 172.21.15.47:0/1977820770 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:39.134 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.128+0000 7f20a22af700 1 --2- 172.21.15.47:0/1977820770 >> v2:172.21.15.64:3300/0 conn(0x7f209c0b7860 0x7f209c0b3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:39.134 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.128+0000 7f20a1aae700 1 -- 172.21.15.47:0/1977820770 >> v2:172.21.15.64:3300/0 conn(0x7f209c0b7860 msgr2=0x7f209c0b3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:39.134 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.128+0000 7f20a1aae700 1 --2- 172.21.15.47:0/1977820770 >> v2:172.21.15.64:3300/0 conn(0x7f209c0b7860 0x7f209c0b3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:39.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.128+0000 7f20a1aae700 1 -- 172.21.15.47:0/1977820770 >> v1:172.21.15.64:6789/0 conn(0x7f209c14a4f0 legacy=0x7f209c14ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:39.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.128+0000 7f20a1aae700 1 -- 172.21.15.47:0/1977820770 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f209c0b7c40 con 0x7f209c0b2d00 2022-06-29T15:42:39.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.128+0000 7f20a1aae700 1 --2- 172.21.15.47:0/1977820770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f209c0b2d00 0x7f209c14a000 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f209c0afb30 tx=0x7f208c006eb0).ready entity=mon.0 client_cookie=c69e8b180b04d929 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:39.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.128+0000 7f20a12ad700 1 -- 172.21.15.47:0/1977820770 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f208c00d610 con 0x7f209c0b2d00 2022-06-29T15:42:39.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.128+0000 7f20a12ad700 1 -- 172.21.15.47:0/1977820770 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f208c000e90 con 0x7f209c0b2d00 2022-06-29T15:42:39.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.128+0000 7f20a12ad700 1 -- 172.21.15.47:0/1977820770 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f208c00d960 con 0x7f209c0b2d00 2022-06-29T15:42:39.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.128+0000 7f20a32b1700 1 -- 172.21.15.47:0/1977820770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f209c0b2d00 msgr2=0x7f209c14a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:39.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.128+0000 7f20a32b1700 1 --2- 172.21.15.47:0/1977820770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f209c0b2d00 0x7f209c14a000 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f209c0afb30 tx=0x7f208c006eb0).stop 2022-06-29T15:42:39.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.128+0000 7f20a32b1700 1 -- 172.21.15.47:0/1977820770 shutdown_connections 2022-06-29T15:42:39.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.128+0000 7f20a32b1700 1 --2- 172.21.15.47:0/1977820770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f209c0b2d00 0x7f209c14a000 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:39.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.128+0000 7f20a32b1700 1 --2- 172.21.15.47:0/1977820770 >> v2:172.21.15.64:3300/0 conn(0x7f209c0b7860 0x7f209c0b3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:39.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.128+0000 7f20a32b1700 1 -- 172.21.15.47:0/1977820770 >> 172.21.15.47:0/1977820770 conn(0x7f209c05f480 msgr2=0x7f209c0647d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:39.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.128+0000 7f20a32b1700 1 -- 172.21.15.47:0/1977820770 shutdown_connections 2022-06-29T15:42:39.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.128+0000 7f20a32b1700 1 -- 172.21.15.47:0/1977820770 wait complete. 2022-06-29T15:42:39.141 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.132+0000 7f20a32b1700 1 Processor -- start 2022-06-29T15:42:39.142 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.132+0000 7f20a32b1700 1 -- start start 2022-06-29T15:42:39.142 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.132+0000 7f20a32b1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f209c0b2d00 0x7f209c0b6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:39.142 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.132+0000 7f20a32b1700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f209c0b7860 0x7f209c157310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:39.143 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.132+0000 7f20a32b1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f209c14a4f0 0x7f209c0601a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:39.143 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.132+0000 7f20a32b1700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209c062740 con 0x7f209c14a4f0 2022-06-29T15:42:39.144 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.132+0000 7f20a32b1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209c061de0 con 0x7f209c0b2d00 2022-06-29T15:42:39.144 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.132+0000 7f20a32b1700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f209c062930 con 0x7f209c0b7860 2022-06-29T15:42:39.144 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.132+0000 7f20a2ab0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f209c14a4f0 0x7f209c0601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:39.145 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.132+0000 7f20a2ab0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f209c14a4f0 0x7f209c0601a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45964/0 (socket says 172.21.15.47:45964) 2022-06-29T15:42:39.145 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.132+0000 7f20a2ab0700 1 -- 172.21.15.47:0/3009992457 learned_addr learned my addr 172.21.15.47:0/3009992457 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:39.146 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.132+0000 7f20a22af700 1 --2- 172.21.15.47:0/3009992457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f209c0b2d00 0x7f209c0b6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:39.146 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.132+0000 7f20a1aae700 1 --2- 172.21.15.47:0/3009992457 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f209c0b7860 0x7f209c157310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:39.146 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.132+0000 7f20a2ab0700 1 -- 172.21.15.47:0/3009992457 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f209c0b7860 msgr2=0x7f209c157310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:39.147 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.132+0000 7f20a2ab0700 1 --2- 172.21.15.47:0/3009992457 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f209c0b7860 0x7f209c157310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:39.147 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.132+0000 7f20a2ab0700 1 -- 172.21.15.47:0/3009992457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f209c0b2d00 msgr2=0x7f209c0b6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:39.148 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.132+0000 7f20a2ab0700 1 --2- 172.21.15.47:0/3009992457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f209c0b2d00 0x7f209c0b6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:39.148 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.132+0000 7f20a2ab0700 1 -- 172.21.15.47:0/3009992457 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f208c00b1e0 con 0x7f209c14a4f0 2022-06-29T15:42:39.148 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.132+0000 7f20a1aae700 1 --2- 172.21.15.47:0/3009992457 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f209c0b7860 0x7f209c157310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:42:39.149 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.132+0000 7f20a2ab0700 1 --2- 172.21.15.47:0/3009992457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f209c14a4f0 0x7f209c0601a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f209c0b26f0 tx=0x7f20940013a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:39.149 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.132+0000 7f208b7fe700 1 -- 172.21.15.47:0/3009992457 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2094006aa0 con 0x7f209c14a4f0 2022-06-29T15:42:39.149 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.132+0000 7f208b7fe700 1 -- 172.21.15.47:0/3009992457 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2094006c00 con 0x7f209c14a4f0 2022-06-29T15:42:39.149 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.132+0000 7f20a32b1700 1 -- 172.21.15.47:0/3009992457 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f209c156930 con 0x7f209c14a4f0 2022-06-29T15:42:39.149 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.132+0000 7f208b7fe700 1 -- 172.21.15.47:0/3009992457 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2094007030 con 0x7f209c14a4f0 2022-06-29T15:42:39.150 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.132+0000 7f20a32b1700 1 -- 172.21.15.47:0/3009992457 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f209c156e70 con 0x7f209c14a4f0 2022-06-29T15:42:39.150 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.132+0000 7f20a32b1700 1 -- 172.21.15.47:0/3009992457 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f209c157070 con 0x7f209c14a4f0 2022-06-29T15:42:39.151 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.136+0000 7f208b7fe700 1 -- 172.21.15.47:0/3009992457 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2094010030 con 0x7f209c14a4f0 2022-06-29T15:42:39.151 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.136+0000 7f208b7fe700 1 --2- 172.21.15.47:0/3009992457 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f207c053a10 0x7f207c055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:39.152 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.136+0000 7f208b7fe700 1 -- 172.21.15.47:0/3009992457 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f209406bed0 con 0x7f209c14a4f0 2022-06-29T15:42:39.152 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.136+0000 7f20a22af700 1 --2- 172.21.15.47:0/3009992457 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f207c053a10 0x7f207c055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:39.152 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.136+0000 7f208b7fe700 1 --2- 172.21.15.47:0/3009992457 >> [v2:172.21.15.64:6808/13394,v1:172.21.15.64:6809/13394] conn(0x7f207c05b310 0x7f207c05d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:39.153 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.136+0000 7f208b7fe700 1 -- 172.21.15.47:0/3009992457 --> [v2:172.21.15.64:6808/13394,v1:172.21.15.64:6809/13394] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f207c05de30 con 0x7f207c05b310 2022-06-29T15:42:39.153 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.136+0000 7f208b7fe700 1 -- 172.21.15.47:0/3009992457 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2094042c90 con 0x7f209c14a4f0 2022-06-29T15:42:39.154 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.136+0000 7f20a1aae700 1 --2- 172.21.15.47:0/3009992457 >> [v2:172.21.15.64:6808/13394,v1:172.21.15.64:6809/13394] conn(0x7f207c05b310 0x7f207c05d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:39.154 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.136+0000 7f20a22af700 1 --2- 172.21.15.47:0/3009992457 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f207c053a10 0x7f207c055ed0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f209800c000 tx=0x7f2098008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:39.154 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.136+0000 7f20a1aae700 1 --2- 172.21.15.47:0/3009992457 >> [v2:172.21.15.64:6808/13394,v1:172.21.15.64:6809/13394] conn(0x7f207c05b310 0x7f207c05d7b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:39.155 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.136+0000 7f208b7fe700 1 -- 172.21.15.47:0/3009992457 <== osd.7 v2:172.21.15.64:6808/13394 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f207c05de30 con 0x7f207c05b310 2022-06-29T15:42:39.159 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.156+0000 7f20a32b1700 1 -- 172.21.15.47:0/3009992457 --> [v2:172.21.15.64:6808/13394,v1:172.21.15.64:6809/13394] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f209c0573d0 con 0x7f207c05b310 2022-06-29T15:42:39.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.156+0000 7f208b7fe700 1 -- 172.21.15.47:0/3009992457 <== osd.7 v2:172.21.15.64:6808/13394 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f209c0573d0 con 0x7f207c05b310 2022-06-29T15:42:39.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.160+0000 7f20a32b1700 1 -- 172.21.15.47:0/3009992457 >> [v2:172.21.15.64:6808/13394,v1:172.21.15.64:6809/13394] conn(0x7f207c05b310 msgr2=0x7f207c05d7b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:39.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.160+0000 7f20a32b1700 1 --2- 172.21.15.47:0/3009992457 >> [v2:172.21.15.64:6808/13394,v1:172.21.15.64:6809/13394] conn(0x7f207c05b310 0x7f207c05d7b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:39.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.160+0000 7f20a32b1700 1 -- 172.21.15.47:0/3009992457 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f207c053a10 msgr2=0x7f207c055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:39.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.160+0000 7f20a32b1700 1 --2- 172.21.15.47:0/3009992457 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f207c053a10 0x7f207c055ed0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f209800c000 tx=0x7f2098008040).stop 2022-06-29T15:42:39.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.160+0000 7f20a32b1700 1 -- 172.21.15.47:0/3009992457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f209c14a4f0 msgr2=0x7f209c0601a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:39.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.160+0000 7f20a32b1700 1 --2- 172.21.15.47:0/3009992457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f209c14a4f0 0x7f209c0601a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f209c0b26f0 tx=0x7f20940013a0).stop 2022-06-29T15:42:39.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.160+0000 7f20a2ab0700 1 -- 172.21.15.47:0/3009992457 reap_dead start 2022-06-29T15:42:39.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.160+0000 7f20a32b1700 1 -- 172.21.15.47:0/3009992457 shutdown_connections 2022-06-29T15:42:39.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.160+0000 7f20a32b1700 1 -- 172.21.15.47:0/3009992457 >> 172.21.15.47:0/3009992457 conn(0x7f209c05f480 msgr2=0x7f209c1491d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:39.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.160+0000 7f20a32b1700 1 -- 172.21.15.47:0/3009992457 shutdown_connections 2022-06-29T15:42:39.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.160+0000 7f20a32b1700 1 -- 172.21.15.47:0/3009992457 wait complete. 2022-06-29T15:42:39.176 INFO:teuthology.orchestra.run.smithi047.stdout:60129542149 2022-06-29T15:42:39.177 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2022-06-29T15:42:39.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.288+0000 7fc2cf0b7700 1 Processor -- start 2022-06-29T15:42:39.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.288+0000 7fc2cf0b7700 1 -- start start 2022-06-29T15:42:39.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.288+0000 7fc2cf0b7700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc2c80b5c60 0x7fc2c8144980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:39.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.288+0000 7fc2cf0b7700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc2c8060eb0 0x7fc2c8144e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:39.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.288+0000 7fc2cf0b7700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2c80a9120 con 0x7fc2c80a72f0 2022-06-29T15:42:39.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.288+0000 7fc2cf0b7700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2c8145360 con 0x7fc2c8060eb0 2022-06-29T15:42:39.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.288+0000 7fc2cf0b7700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c80b0b80 con 0x7fc2c80b5c60 2022-06-29T15:42:39.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.288+0000 7fc2cd8b4700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc2c8060eb0 0x7fc2c8144e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:39.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.288+0000 7fc2cd8b4700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc2c8060eb0 0x7fc2c8144e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45970/0 (socket says 172.21.15.47:45970) 2022-06-29T15:42:39.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.288+0000 7fc2cd8b4700 1 -- 172.21.15.47:0/310764460 learned_addr learned my addr 172.21.15.47:0/310764460 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:39.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.288+0000 7fc2ce0b5700 1 --2- 172.21.15.47:0/310764460 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc2c80b5c60 0x7fc2c8144980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:39.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.288+0000 7fc2cd8b4700 1 -- 172.21.15.47:0/310764460 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc2c80b5c60 msgr2=0x7fc2c8144980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:39.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.288+0000 7fc2cd8b4700 1 --2- 172.21.15.47:0/310764460 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc2c80b5c60 0x7fc2c8144980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:39.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.288+0000 7fc2cd8b4700 1 -- 172.21.15.47:0/310764460 >> v1:172.21.15.64:6789/0 conn(0x7fc2c80a72f0 legacy=0x7fc2c80b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:39.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.288+0000 7fc2cd8b4700 1 -- 172.21.15.47:0/310764460 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c80a89e0 con 0x7fc2c8060eb0 2022-06-29T15:42:39.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.288+0000 7fc2cd8b4700 1 --2- 172.21.15.47:0/310764460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2c8060eb0 0x7fc2c8144e70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fc2c4002f70 tx=0x7fc2c4006eb0).ready entity=mon.0 client_cookie=902996fb744ab69e server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:39.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.288+0000 7fc2cd0b3700 1 -- 172.21.15.47:0/310764460 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc2c400d570 con 0x7fc2c8060eb0 2022-06-29T15:42:39.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.288+0000 7fc2cd0b3700 1 -- 172.21.15.47:0/310764460 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2c4000e90 con 0x7fc2c8060eb0 2022-06-29T15:42:39.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.288+0000 7fc2cd0b3700 1 -- 172.21.15.47:0/310764460 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc2c400d900 con 0x7fc2c8060eb0 2022-06-29T15:42:39.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.288+0000 7fc2cf0b7700 1 -- 172.21.15.47:0/310764460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2c8060eb0 msgr2=0x7fc2c8144e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:39.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.288+0000 7fc2cf0b7700 1 --2- 172.21.15.47:0/310764460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2c8060eb0 0x7fc2c8144e70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fc2c4002f70 tx=0x7fc2c4006eb0).stop 2022-06-29T15:42:39.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.288+0000 7fc2cf0b7700 1 -- 172.21.15.47:0/310764460 shutdown_connections 2022-06-29T15:42:39.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.288+0000 7fc2cf0b7700 1 --2- 172.21.15.47:0/310764460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2c8060eb0 0x7fc2c8144e70 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:39.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.288+0000 7fc2cf0b7700 1 --2- 172.21.15.47:0/310764460 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc2c80b5c60 0x7fc2c8144980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:39.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.288+0000 7fc2cf0b7700 1 -- 172.21.15.47:0/310764460 >> 172.21.15.47:0/310764460 conn(0x7fc2c8062560 msgr2=0x7fc2c8062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:39.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.288+0000 7fc2cf0b7700 1 -- 172.21.15.47:0/310764460 shutdown_connections 2022-06-29T15:42:39.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.288+0000 7fc2cf0b7700 1 -- 172.21.15.47:0/310764460 wait complete. 2022-06-29T15:42:39.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.292+0000 7fc2cf0b7700 1 Processor -- start 2022-06-29T15:42:39.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.292+0000 7fc2cf0b7700 1 -- start start 2022-06-29T15:42:39.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.292+0000 7fc2cf0b7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2c8060eb0 0x7fc2c8143740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:39.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.292+0000 7fc2cf0b7700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc2c80a72f0 0x7fc2c80ad7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:39.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.292+0000 7fc2cf0b7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2c80b5c60 0x7fc2c80adc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:39.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.292+0000 7fc2cf0b7700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c8157680 con 0x7fc2c8060eb0 2022-06-29T15:42:39.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.292+0000 7fc2cf0b7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c8156760 con 0x7fc2c80b5c60 2022-06-29T15:42:39.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.292+0000 7fc2cf0b7700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c8156a00 con 0x7fc2c80a72f0 2022-06-29T15:42:39.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.292+0000 7fc2ce0b5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2c8060eb0 0x7fc2c8143740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:39.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.292+0000 7fc2ce0b5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2c8060eb0 0x7fc2c8143740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45976/0 (socket says 172.21.15.47:45976) 2022-06-29T15:42:39.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.292+0000 7fc2ce0b5700 1 -- 172.21.15.47:0/2988080878 learned_addr learned my addr 172.21.15.47:0/2988080878 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:39.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.292+0000 7fc2cd8b4700 1 --2- 172.21.15.47:0/2988080878 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc2c80a72f0 0x7fc2c80ad7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:39.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.292+0000 7fc2ce8b6700 1 --2- 172.21.15.47:0/2988080878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2c80b5c60 0x7fc2c80adc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:39.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.292+0000 7fc2ce0b5700 1 -- 172.21.15.47:0/2988080878 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc2c80a72f0 msgr2=0x7fc2c80ad7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:39.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.292+0000 7fc2ce0b5700 1 --2- 172.21.15.47:0/2988080878 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc2c80a72f0 0x7fc2c80ad7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:39.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.292+0000 7fc2ce0b5700 1 -- 172.21.15.47:0/2988080878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2c80b5c60 msgr2=0x7fc2c80adc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:39.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.292+0000 7fc2ce0b5700 1 --2- 172.21.15.47:0/2988080878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2c80b5c60 0x7fc2c80adc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:39.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.292+0000 7fc2ce0b5700 1 -- 172.21.15.47:0/2988080878 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c400b1e0 con 0x7fc2c8060eb0 2022-06-29T15:42:39.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.292+0000 7fc2ce8b6700 1 --2- 172.21.15.47:0/2988080878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2c80b5c60 0x7fc2c80adc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:42:39.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.292+0000 7fc2ce0b5700 1 --2- 172.21.15.47:0/2988080878 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2c8060eb0 0x7fc2c8143740 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fc2bc00c000 tx=0x7fc2bc006d90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:39.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.292+0000 7fc2bb7fe700 1 -- 172.21.15.47:0/2988080878 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc2bc0104c0 con 0x7fc2c8060eb0 2022-06-29T15:42:39.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.292+0000 7fc2bb7fe700 1 -- 172.21.15.47:0/2988080878 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2bc010620 con 0x7fc2c8060eb0 2022-06-29T15:42:39.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.292+0000 7fc2cf0b7700 1 -- 172.21.15.47:0/2988080878 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2c8156ca0 con 0x7fc2c8060eb0 2022-06-29T15:42:39.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.292+0000 7fc2bb7fe700 1 -- 172.21.15.47:0/2988080878 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc2bc010980 con 0x7fc2c8060eb0 2022-06-29T15:42:39.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.292+0000 7fc2cf0b7700 1 -- 172.21.15.47:0/2988080878 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2c81571e0 con 0x7fc2c8060eb0 2022-06-29T15:42:39.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.296+0000 7fc2cf0b7700 1 -- 172.21.15.47:0/2988080878 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2c80573d0 con 0x7fc2c8060eb0 2022-06-29T15:42:39.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.296+0000 7fc2bb7fe700 1 -- 172.21.15.47:0/2988080878 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc2bc01a070 con 0x7fc2c8060eb0 2022-06-29T15:42:39.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.296+0000 7fc2bb7fe700 1 --2- 172.21.15.47:0/2988080878 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc2a80539e0 0x7fc2a8055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:39.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.296+0000 7fc2bb7fe700 1 -- 172.21.15.47:0/2988080878 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fc2bc06be20 con 0x7fc2c8060eb0 2022-06-29T15:42:39.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.296+0000 7fc2cd8b4700 1 --2- 172.21.15.47:0/2988080878 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc2a80539e0 0x7fc2a8055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:39.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.296+0000 7fc2cd8b4700 1 --2- 172.21.15.47:0/2988080878 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc2a80539e0 0x7fc2a8055ea0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fc2c401c000 tx=0x7fc2c4015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:39.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.304+0000 7fc2bb7fe700 1 -- 172.21.15.47:0/2988080878 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc2bc042be0 con 0x7fc2c8060eb0 2022-06-29T15:42:39.433 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.432+0000 7fc2cf0b7700 1 -- 172.21.15.47:0/2988080878 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc2c80af620 con 0x7fc2c8060eb0 2022-06-29T15:42:39.434 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.432+0000 7fc2bb7fe700 1 -- 172.21.15.47:0/2988080878 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc2bc046200 con 0x7fc2c8060eb0 2022-06-29T15:42:39.435 INFO:teuthology.orchestra.run.smithi047.stdout:60129542149 2022-06-29T15:42:39.436 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.432+0000 7fc2cf0b7700 1 -- 172.21.15.47:0/2988080878 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc2a80539e0 msgr2=0x7fc2a8055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:39.437 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.432+0000 7fc2cf0b7700 1 --2- 172.21.15.47:0/2988080878 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc2a80539e0 0x7fc2a8055ea0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fc2c401c000 tx=0x7fc2c4015000).stop 2022-06-29T15:42:39.437 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.432+0000 7fc2cf0b7700 1 -- 172.21.15.47:0/2988080878 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2c8060eb0 msgr2=0x7fc2c8143740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:39.437 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.432+0000 7fc2cf0b7700 1 --2- 172.21.15.47:0/2988080878 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2c8060eb0 0x7fc2c8143740 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fc2bc00c000 tx=0x7fc2bc006d90).stop 2022-06-29T15:42:39.437 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.432+0000 7fc2cf0b7700 1 -- 172.21.15.47:0/2988080878 shutdown_connections 2022-06-29T15:42:39.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.432+0000 7fc2cf0b7700 1 --2- 172.21.15.47:0/2988080878 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc2a80539e0 0x7fc2a8055ea0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:39.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.432+0000 7fc2cf0b7700 1 --2- 172.21.15.47:0/2988080878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2c80b5c60 0x7fc2c80adc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:39.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.432+0000 7fc2cf0b7700 1 --2- 172.21.15.47:0/2988080878 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc2c80a72f0 0x7fc2c80ad7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:39.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.432+0000 7fc2cf0b7700 1 --2- 172.21.15.47:0/2988080878 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2c8060eb0 0x7fc2c8143740 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:39.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.432+0000 7fc2cf0b7700 1 -- 172.21.15.47:0/2988080878 >> 172.21.15.47:0/2988080878 conn(0x7fc2c8062560 msgr2=0x7fc2c81424e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:39.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.432+0000 7fc2cf0b7700 1 -- 172.21.15.47:0/2988080878 shutdown_connections 2022-06-29T15:42:39.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.432+0000 7fc2cf0b7700 1 -- 172.21.15.47:0/2988080878 wait complete. 2022-06-29T15:42:39.447 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.0 2022-06-29T15:42:39.448 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2022-06-29T15:42:39.556 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.552+0000 7f1ec9936700 1 Processor -- start 2022-06-29T15:42:39.557 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.552+0000 7f1ec9936700 1 -- start start 2022-06-29T15:42:39.570 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.556+0000 7f1ec9936700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1ec405c510 0x7f1ec4143bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:39.571 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.556+0000 7f1ec9936700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1ec40b2c40 0x7f1ec40b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:39.571 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.556+0000 7f1ec9936700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ec4142f90 con 0x7f1ec405c8f0 2022-06-29T15:42:39.571 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.556+0000 7f1ec9936700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ec41431f0 con 0x7f1ec40b2c40 2022-06-29T15:42:39.571 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.556+0000 7f1ec9936700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ec4143490 con 0x7f1ec405c510 2022-06-29T15:42:39.571 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.556+0000 7f1ec9135700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1ec40b2c40 0x7f1ec40b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:39.572 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.556+0000 7f1ec3fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1ec405c8f0 0x7f1ec40b4100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59770/0 (socket says 172.21.15.47:59770) 2022-06-29T15:42:39.572 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.556+0000 7f1ec3fff700 1 -- 172.21.15.47:0/425706609 learned_addr learned my addr 172.21.15.47:0/425706609 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:39.572 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.556+0000 7f1ec8934700 1 --2- 172.21.15.47:0/425706609 >> v2:172.21.15.64:3300/0 conn(0x7f1ec405c510 0x7f1ec4143bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:39.572 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.556+0000 7f1ec37fe700 1 -- 172.21.15.47:0/425706609 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2727209137 0 0) 0x7f1ec4142f90 con 0x7f1ec405c8f0 2022-06-29T15:42:39.573 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.556+0000 7f1ec37fe700 1 -- 172.21.15.47:0/425706609 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ea4003260 con 0x7f1ec405c8f0 2022-06-29T15:42:39.573 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.556+0000 7f1ec37fe700 1 -- 172.21.15.47:0/425706609 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2501572530 0 0) 0x7f1ea4003260 con 0x7f1ec405c8f0 2022-06-29T15:42:39.573 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.556+0000 7f1ec9135700 1 -- 172.21.15.47:0/425706609 >> v2:172.21.15.64:3300/0 conn(0x7f1ec405c510 msgr2=0x7f1ec4143bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:39.573 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.556+0000 7f1ec9135700 1 --2- 172.21.15.47:0/425706609 >> v2:172.21.15.64:3300/0 conn(0x7f1ec405c510 0x7f1ec4143bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:39.574 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.556+0000 7f1ec9135700 1 -- 172.21.15.47:0/425706609 >> v1:172.21.15.47:6789/0 conn(0x7f1ec405c8f0 legacy=0x7f1ec40b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:39.574 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.556+0000 7f1ec9135700 1 -- 172.21.15.47:0/425706609 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec4143730 con 0x7f1ec40b2c40 2022-06-29T15:42:39.574 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.556+0000 7f1ec9135700 1 --2- 172.21.15.47:0/425706609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ec40b2c40 0x7f1ec40b30d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f1eb4001800 tx=0x7f1ebc0091d0).ready entity=mon.0 client_cookie=ca1e5897f4204db server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:39.574 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.556+0000 7f1ec37fe700 1 -- 172.21.15.47:0/425706609 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1eb4001660 con 0x7f1ec40b2c40 2022-06-29T15:42:39.575 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.556+0000 7f1ec37fe700 1 -- 172.21.15.47:0/425706609 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ebc00f510 con 0x7f1ec40b2c40 2022-06-29T15:42:39.575 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.556+0000 7f1ec37fe700 1 -- 172.21.15.47:0/425706609 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ebc00f680 con 0x7f1ec40b2c40 2022-06-29T15:42:39.575 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.556+0000 7f1ec9936700 1 -- 172.21.15.47:0/425706609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ec40b2c40 msgr2=0x7f1ec40b30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:39.575 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.556+0000 7f1ec9936700 1 --2- 172.21.15.47:0/425706609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ec40b2c40 0x7f1ec40b30d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f1eb4001800 tx=0x7f1ebc0091d0).stop 2022-06-29T15:42:39.576 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.556+0000 7f1ec9936700 1 -- 172.21.15.47:0/425706609 shutdown_connections 2022-06-29T15:42:39.576 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.556+0000 7f1ec9936700 1 --2- 172.21.15.47:0/425706609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ec40b2c40 0x7f1ec40b30d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:39.576 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.556+0000 7f1ec9936700 1 --2- 172.21.15.47:0/425706609 >> v2:172.21.15.64:3300/0 conn(0x7f1ec405c510 0x7f1ec4143bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:39.576 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.556+0000 7f1ec9936700 1 -- 172.21.15.47:0/425706609 >> 172.21.15.47:0/425706609 conn(0x7f1ec4060e00 msgr2=0x7f1ec40611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:39.576 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.556+0000 7f1ec9936700 1 -- 172.21.15.47:0/425706609 shutdown_connections 2022-06-29T15:42:39.577 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.556+0000 7f1ec9936700 1 -- 172.21.15.47:0/425706609 wait complete. 2022-06-29T15:42:39.577 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.560+0000 7f1ec9936700 1 Processor -- start 2022-06-29T15:42:39.577 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.560+0000 7f1ec9936700 1 -- start start 2022-06-29T15:42:39.577 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.560+0000 7f1ec9936700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1ec405c510 0x7f1ec40b6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:39.577 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.560+0000 7f1ec9936700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ec405c8f0 0x7f1ec4152030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:39.578 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.560+0000 7f1ec9936700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ec40b2c40 0x7f1ec4155e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:39.578 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.560+0000 7f1ec9936700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec413f8d0 con 0x7f1ec405c8f0 2022-06-29T15:42:39.578 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.560+0000 7f1ec9936700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec4146ef0 con 0x7f1ec40b2c40 2022-06-29T15:42:39.578 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.560+0000 7f1ec9936700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec4140700 con 0x7f1ec405c510 2022-06-29T15:42:39.579 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.560+0000 7f1ec3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ec405c8f0 0x7f1ec4152030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:39.579 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.560+0000 7f1ec3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ec405c8f0 0x7f1ec4152030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45992/0 (socket says 172.21.15.47:45992) 2022-06-29T15:42:39.579 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.560+0000 7f1ec3fff700 1 -- 172.21.15.47:0/1499791231 learned_addr learned my addr 172.21.15.47:0/1499791231 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:39.579 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.560+0000 7f1ec9135700 1 --2- 172.21.15.47:0/1499791231 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ec40b2c40 0x7f1ec4155e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:39.580 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.560+0000 7f1ec8934700 1 --2- 172.21.15.47:0/1499791231 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1ec405c510 0x7f1ec40b6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:39.580 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.560+0000 7f1ec3fff700 1 -- 172.21.15.47:0/1499791231 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1ec405c510 msgr2=0x7f1ec40b6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:39.580 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.560+0000 7f1ec3fff700 1 --2- 172.21.15.47:0/1499791231 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1ec405c510 0x7f1ec40b6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:39.580 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.560+0000 7f1ec3fff700 1 -- 172.21.15.47:0/1499791231 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ec40b2c40 msgr2=0x7f1ec4155e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:39.581 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.560+0000 7f1ec3fff700 1 --2- 172.21.15.47:0/1499791231 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ec40b2c40 0x7f1ec4155e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:39.581 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.560+0000 7f1ec3fff700 1 -- 172.21.15.47:0/1499791231 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ebc009470 con 0x7f1ec405c8f0 2022-06-29T15:42:39.581 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.560+0000 7f1ec8934700 1 --2- 172.21.15.47:0/1499791231 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1ec405c510 0x7f1ec40b6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:42:39.581 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.560+0000 7f1ec3fff700 1 --2- 172.21.15.47:0/1499791231 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ec405c8f0 0x7f1ec4152030 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f1ec4147080 tx=0x7f1eb4001b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:39.582 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.560+0000 7f1ec1ffb700 1 -- 172.21.15.47:0/1499791231 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1eb4007170 con 0x7f1ec405c8f0 2022-06-29T15:42:39.582 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.564+0000 7f1ec1ffb700 1 -- 172.21.15.47:0/1499791231 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb40072d0 con 0x7f1ec405c8f0 2022-06-29T15:42:39.582 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.564+0000 7f1ec9936700 1 -- 172.21.15.47:0/1499791231 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ec4155700 con 0x7f1ec405c8f0 2022-06-29T15:42:39.582 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.564+0000 7f1ec1ffb700 1 -- 172.21.15.47:0/1499791231 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1eb4004920 con 0x7f1ec405c8f0 2022-06-29T15:42:39.582 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.564+0000 7f1ec9936700 1 -- 172.21.15.47:0/1499791231 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ec4156330 con 0x7f1ec405c8f0 2022-06-29T15:42:39.583 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.564+0000 7f1ec1ffb700 1 -- 172.21.15.47:0/1499791231 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1eb4004a80 con 0x7f1ec405c8f0 2022-06-29T15:42:39.583 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.564+0000 7f1ec9936700 1 -- 172.21.15.47:0/1499791231 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ec41408c0 con 0x7f1ec405c8f0 2022-06-29T15:42:39.583 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.564+0000 7f1ec1ffb700 1 --2- 172.21.15.47:0/1499791231 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1ea8053a70 0x7f1ea8055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:39.583 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.564+0000 7f1ec1ffb700 1 -- 172.21.15.47:0/1499791231 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f1eb406bd00 con 0x7f1ec405c8f0 2022-06-29T15:42:39.584 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.564+0000 7f1ec8934700 1 --2- 172.21.15.47:0/1499791231 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1ea8053a70 0x7f1ea8055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:39.584 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.564+0000 7f1ec8934700 1 --2- 172.21.15.47:0/1499791231 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1ea8053a70 0x7f1ea8055f30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f1eac00b000 tx=0x7f1eac008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:39.585 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.568+0000 7f1ec1ffb700 1 -- 172.21.15.47:0/1499791231 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1eb406f030 con 0x7f1ec405c8f0 2022-06-29T15:42:39.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.696+0000 7f1ec9936700 1 -- 172.21.15.47:0/1499791231 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1ec40573d0 con 0x7f1ec405c8f0 2022-06-29T15:42:39.699 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.696+0000 7f1ec1ffb700 1 -- 172.21.15.47:0/1499791231 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1eb4042ac0 con 0x7f1ec405c8f0 2022-06-29T15:42:39.699 INFO:teuthology.orchestra.run.smithi047.stdout:60129542149 2022-06-29T15:42:39.701 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.696+0000 7f1ec9936700 1 -- 172.21.15.47:0/1499791231 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1ea8053a70 msgr2=0x7f1ea8055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:39.702 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.696+0000 7f1ec9936700 1 --2- 172.21.15.47:0/1499791231 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1ea8053a70 0x7f1ea8055f30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f1eac00b000 tx=0x7f1eac008040).stop 2022-06-29T15:42:39.702 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.696+0000 7f1ec9936700 1 -- 172.21.15.47:0/1499791231 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ec405c8f0 msgr2=0x7f1ec4152030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:39.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.696+0000 7f1ec9936700 1 --2- 172.21.15.47:0/1499791231 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ec405c8f0 0x7f1ec4152030 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f1ec4147080 tx=0x7f1eb4001b30).stop 2022-06-29T15:42:39.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.696+0000 7f1ec9936700 1 -- 172.21.15.47:0/1499791231 shutdown_connections 2022-06-29T15:42:39.704 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.696+0000 7f1ec9936700 1 --2- 172.21.15.47:0/1499791231 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1ea8053a70 0x7f1ea8055f30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:39.704 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.696+0000 7f1ec9936700 1 --2- 172.21.15.47:0/1499791231 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ec40b2c40 0x7f1ec4155e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:39.704 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.696+0000 7f1ec9936700 1 --2- 172.21.15.47:0/1499791231 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ec405c8f0 0x7f1ec4152030 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:39.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.696+0000 7f1ec9936700 1 --2- 172.21.15.47:0/1499791231 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1ec405c510 0x7f1ec40b6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:39.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.696+0000 7f1ec9936700 1 -- 172.21.15.47:0/1499791231 >> 172.21.15.47:0/1499791231 conn(0x7f1ec4060e00 msgr2=0x7f1ec4141bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:39.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.700+0000 7f1ec9936700 1 -- 172.21.15.47:0/1499791231 shutdown_connections 2022-06-29T15:42:39.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.700+0000 7f1ec9936700 1 -- 172.21.15.47:0/1499791231 wait complete. 2022-06-29T15:42:39.711 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.1 2022-06-29T15:42:39.711 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2022-06-29T15:42:39.819 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.816+0000 7ff09f59e700 1 Processor -- start 2022-06-29T15:42:39.820 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.816+0000 7ff09f59e700 1 -- start start 2022-06-29T15:42:39.824 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.816+0000 7ff09f59e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff0a005c510 0x7ff0a0143bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:39.824 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.816+0000 7ff09f59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff0a005c8f0 0x7ff0a00b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:39.824 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.816+0000 7ff09f59e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0a0142f90 con 0x7ff0a00b2c40 2022-06-29T15:42:39.825 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.816+0000 7ff09f59e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0a01431f0 con 0x7ff0a005c8f0 2022-06-29T15:42:39.825 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.816+0000 7ff09f59e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a0143490 con 0x7ff0a005c510 2022-06-29T15:42:39.825 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.816+0000 7ff09ed9d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff0a00b2c40 0x7ff0a013cbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59788/0 (socket says 172.21.15.47:59788) 2022-06-29T15:42:39.826 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.816+0000 7ff09ed9d700 1 -- 172.21.15.47:0/1904464996 learned_addr learned my addr 172.21.15.47:0/1904464996 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:39.826 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.816+0000 7ff09dd9b700 1 --2- 172.21.15.47:0/1904464996 >> v2:172.21.15.64:3300/0 conn(0x7ff0a005c8f0 0x7ff0a00b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:39.826 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.816+0000 7ff09e59c700 1 --2- 172.21.15.47:0/1904464996 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff0a005c510 0x7ff0a0143bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:39.827 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.816+0000 7ff09d59a700 1 -- 172.21.15.47:0/1904464996 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 866133041 0 0) 0x7ff0a0142f90 con 0x7ff0a00b2c40 2022-06-29T15:42:39.827 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.820+0000 7ff09d59a700 1 -- 172.21.15.47:0/1904464996 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff080003260 con 0x7ff0a00b2c40 2022-06-29T15:42:39.828 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.820+0000 7ff09d59a700 1 -- 172.21.15.47:0/1904464996 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4007825278 0 0) 0x7ff080003260 con 0x7ff0a00b2c40 2022-06-29T15:42:39.828 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.820+0000 7ff09d59a700 1 -- 172.21.15.47:0/1904464996 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff0a005c510 msgr2=0x7ff0a0143bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:39.828 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.820+0000 7ff09d59a700 1 --2- 172.21.15.47:0/1904464996 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff0a005c510 0x7ff0a0143bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:39.828 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.820+0000 7ff09d59a700 1 -- 172.21.15.47:0/1904464996 >> v2:172.21.15.64:3300/0 conn(0x7ff0a005c8f0 msgr2=0x7ff0a00b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:39.829 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.820+0000 7ff09d59a700 1 --2- 172.21.15.47:0/1904464996 >> v2:172.21.15.64:3300/0 conn(0x7ff0a005c8f0 0x7ff0a00b4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:39.829 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.820+0000 7ff09d59a700 1 -- 172.21.15.47:0/1904464996 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a0143730 con 0x7ff0a00b2c40 2022-06-29T15:42:39.829 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.820+0000 7ff09d59a700 1 -- 172.21.15.47:0/1904464996 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7ff098001a00 con 0x7ff0a00b2c40 2022-06-29T15:42:39.829 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.820+0000 7ff09d59a700 1 -- 172.21.15.47:0/1904464996 >> v1:172.21.15.47:6789/0 conn(0x7ff0a00b2c40 legacy=0x7ff0a013cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:39.830 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.820+0000 7ff09d59a700 1 --2- 172.21.15.47:0/1904464996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff080003750 0x7ff080003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:39.830 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.820+0000 7ff09d59a700 1 --2- 172.21.15.47:0/1904464996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff080004500 0x7ff080004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:39.830 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.820+0000 7ff09d59a700 1 --2- 172.21.15.47:0/1904464996 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff0a005c510 0x7ff080004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:39.830 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.820+0000 7ff09d59a700 1 -- 172.21.15.47:0/1904464996 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a01431f0 con 0x7ff080004500 2022-06-29T15:42:39.831 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.820+0000 7ff09d59a700 1 -- 172.21.15.47:0/1904464996 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a0143490 con 0x7ff080003750 2022-06-29T15:42:39.831 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.820+0000 7ff09d59a700 1 -- 172.21.15.47:0/1904464996 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff08000b750 con 0x7ff0a005c510 2022-06-29T15:42:39.831 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.820+0000 7ff09dd9b700 1 --2- 172.21.15.47:0/1904464996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff080004500 0x7ff080004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:39.831 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.820+0000 7ff09e59c700 1 --2- 172.21.15.47:0/1904464996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff080003750 0x7ff080003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:39.831 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.820+0000 7ff09ed9d700 1 --2- 172.21.15.47:0/1904464996 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff0a005c510 0x7ff080004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:39.832 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.820+0000 7ff09dd9b700 1 -- 172.21.15.47:0/1904464996 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff0a005c510 msgr2=0x7ff080004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:39.832 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.820+0000 7ff09dd9b700 1 --2- 172.21.15.47:0/1904464996 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff0a005c510 0x7ff080004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:39.832 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.820+0000 7ff09dd9b700 1 -- 172.21.15.47:0/1904464996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff080003750 msgr2=0x7ff080003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:39.832 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.820+0000 7ff09dd9b700 1 --2- 172.21.15.47:0/1904464996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff080003750 0x7ff080003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:39.833 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.820+0000 7ff09dd9b700 1 -- 172.21.15.47:0/1904464996 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff080003260 con 0x7ff080004500 2022-06-29T15:42:39.833 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.820+0000 7ff09e59c700 1 --2- 172.21.15.47:0/1904464996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff080003750 0x7ff080003b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:42:39.833 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.820+0000 7ff09dd9b700 1 --2- 172.21.15.47:0/1904464996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff080004500 0x7ff080004980 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7ff08800c000 tx=0x7ff088006df0).ready entity=mon.0 client_cookie=e5bd1a842d3ef3f2 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:39.833 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.820+0000 7ff09d59a700 1 -- 172.21.15.47:0/1904464996 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff08800e830 con 0x7ff080004500 2022-06-29T15:42:39.833 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.820+0000 7ff09d59a700 1 -- 172.21.15.47:0/1904464996 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff08800e990 con 0x7ff080004500 2022-06-29T15:42:39.834 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.820+0000 7ff09d59a700 1 -- 172.21.15.47:0/1904464996 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff08800ecf0 con 0x7ff080004500 2022-06-29T15:42:39.834 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.820+0000 7ff09f59e700 1 -- 172.21.15.47:0/1904464996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff080004500 msgr2=0x7ff080004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:39.834 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.820+0000 7ff09f59e700 1 --2- 172.21.15.47:0/1904464996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff080004500 0x7ff080004980 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7ff08800c000 tx=0x7ff088006df0).stop 2022-06-29T15:42:39.835 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.820+0000 7ff09ed9d700 1 -- 172.21.15.47:0/1904464996 reap_dead start 2022-06-29T15:42:39.835 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.820+0000 7ff09f59e700 1 -- 172.21.15.47:0/1904464996 shutdown_connections 2022-06-29T15:42:39.835 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.820+0000 7ff09f59e700 1 -- 172.21.15.47:0/1904464996 >> 172.21.15.47:0/1904464996 conn(0x7ff0a0060e00 msgr2=0x7ff0a00611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:39.835 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.820+0000 7ff09f59e700 1 -- 172.21.15.47:0/1904464996 shutdown_connections 2022-06-29T15:42:39.835 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.820+0000 7ff09f59e700 1 -- 172.21.15.47:0/1904464996 wait complete. 2022-06-29T15:42:39.839 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.824+0000 7ff09f59e700 1 Processor -- start 2022-06-29T15:42:39.839 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.824+0000 7ff09f59e700 1 -- start start 2022-06-29T15:42:39.839 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.824+0000 7ff09f59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff080004500 0x7ff0a00b8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:39.840 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.824+0000 7ff09f59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0a0144620 0x7ff0a0144aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:39.840 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.824+0000 7ff09f59e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff0a013eed0 0x7ff0a013f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:39.840 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.824+0000 7ff09f59e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a0145230 con 0x7ff0a0144620 2022-06-29T15:42:39.840 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.824+0000 7ff09f59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a00af490 con 0x7ff080004500 2022-06-29T15:42:39.841 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.824+0000 7ff09f59e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a0144f90 con 0x7ff0a013eed0 2022-06-29T15:42:39.841 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.824+0000 7ff09dd9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0a0144620 0x7ff0a0144aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:39.841 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.824+0000 7ff09dd9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0a0144620 0x7ff0a0144aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46010/0 (socket says 172.21.15.47:46010) 2022-06-29T15:42:39.841 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.824+0000 7ff09dd9b700 1 -- 172.21.15.47:0/3484303840 learned_addr learned my addr 172.21.15.47:0/3484303840 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:39.842 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.824+0000 7ff09ed9d700 1 --2- 172.21.15.47:0/3484303840 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff0a013eed0 0x7ff0a013f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:39.842 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.824+0000 7ff09e59c700 1 --2- 172.21.15.47:0/3484303840 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff080004500 0x7ff0a00b8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:39.842 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.824+0000 7ff09dd9b700 1 -- 172.21.15.47:0/3484303840 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff0a013eed0 msgr2=0x7ff0a013f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:39.842 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.824+0000 7ff09dd9b700 1 --2- 172.21.15.47:0/3484303840 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff0a013eed0 0x7ff0a013f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:39.843 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.824+0000 7ff09dd9b700 1 -- 172.21.15.47:0/3484303840 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff080004500 msgr2=0x7ff0a00b8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:39.843 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.824+0000 7ff09dd9b700 1 --2- 172.21.15.47:0/3484303840 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff080004500 0x7ff0a00b8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:39.843 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.824+0000 7ff09dd9b700 1 -- 172.21.15.47:0/3484303840 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a00ad670 con 0x7ff0a0144620 2022-06-29T15:42:39.843 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.824+0000 7ff09dd9b700 1 --2- 172.21.15.47:0/3484303840 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0a0144620 0x7ff0a0144aa0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7ff08800c000 tx=0x7ff088019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:39.843 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.824+0000 7ff0977fe700 1 -- 172.21.15.47:0/3484303840 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff08801b820 con 0x7ff0a0144620 2022-06-29T15:42:39.844 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.824+0000 7ff0977fe700 1 -- 172.21.15.47:0/3484303840 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff088000e90 con 0x7ff0a0144620 2022-06-29T15:42:39.844 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.824+0000 7ff09f59e700 1 -- 172.21.15.47:0/3484303840 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0a01454d0 con 0x7ff0a0144620 2022-06-29T15:42:39.844 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.824+0000 7ff0977fe700 1 -- 172.21.15.47:0/3484303840 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff08801bb70 con 0x7ff0a0144620 2022-06-29T15:42:39.844 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.824+0000 7ff09f59e700 1 -- 172.21.15.47:0/3484303840 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0a00b93d0 con 0x7ff0a0144620 2022-06-29T15:42:39.845 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.828+0000 7ff09f59e700 1 -- 172.21.15.47:0/3484303840 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0a00afb30 con 0x7ff0a0144620 2022-06-29T15:42:39.845 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.828+0000 7ff0977fe700 1 -- 172.21.15.47:0/3484303840 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff08801bcd0 con 0x7ff0a0144620 2022-06-29T15:42:39.845 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.828+0000 7ff0977fe700 1 --2- 172.21.15.47:0/3484303840 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff084053aa0 0x7ff084055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:39.845 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.828+0000 7ff0977fe700 1 -- 172.21.15.47:0/3484303840 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7ff08806d470 con 0x7ff0a0144620 2022-06-29T15:42:39.846 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.828+0000 7ff09e59c700 1 --2- 172.21.15.47:0/3484303840 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff084053aa0 0x7ff084055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:39.846 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.828+0000 7ff09e59c700 1 --2- 172.21.15.47:0/3484303840 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff084053aa0 0x7ff084055f60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7ff09000b000 tx=0x7ff090008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:39.846 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.832+0000 7ff0977fe700 1 -- 172.21.15.47:0/3484303840 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff088044190 con 0x7ff0a0144620 2022-06-29T15:42:39.962 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.960+0000 7ff09f59e700 1 -- 172.21.15.47:0/3484303840 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff0a00af650 con 0x7ff0a0144620 2022-06-29T15:42:39.963 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.960+0000 7ff0977fe700 1 -- 172.21.15.47:0/3484303840 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff0880477b0 con 0x7ff0a0144620 2022-06-29T15:42:39.964 INFO:teuthology.orchestra.run.smithi047.stdout:60129542149 2022-06-29T15:42:39.965 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.960+0000 7ff09f59e700 1 -- 172.21.15.47:0/3484303840 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff084053aa0 msgr2=0x7ff084055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:39.965 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.960+0000 7ff09f59e700 1 --2- 172.21.15.47:0/3484303840 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff084053aa0 0x7ff084055f60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7ff09000b000 tx=0x7ff090008040).stop 2022-06-29T15:42:39.965 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.960+0000 7ff09f59e700 1 -- 172.21.15.47:0/3484303840 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0a0144620 msgr2=0x7ff0a0144aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:39.966 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.960+0000 7ff09f59e700 1 --2- 172.21.15.47:0/3484303840 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0a0144620 0x7ff0a0144aa0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7ff08800c000 tx=0x7ff088019000).stop 2022-06-29T15:42:39.967 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.960+0000 7ff09f59e700 1 -- 172.21.15.47:0/3484303840 shutdown_connections 2022-06-29T15:42:39.967 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.960+0000 7ff09f59e700 1 --2- 172.21.15.47:0/3484303840 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff0a013eed0 0x7ff0a013f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:39.967 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.960+0000 7ff09f59e700 1 --2- 172.21.15.47:0/3484303840 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0a0144620 0x7ff0a0144aa0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:39.967 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.960+0000 7ff09f59e700 1 --2- 172.21.15.47:0/3484303840 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff084053aa0 0x7ff084055f60 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:39.968 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.960+0000 7ff09f59e700 1 --2- 172.21.15.47:0/3484303840 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff080004500 0x7ff0a00b8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:39.968 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.964+0000 7ff09f59e700 1 -- 172.21.15.47:0/3484303840 >> 172.21.15.47:0/3484303840 conn(0x7ff0a0060e00 msgr2=0x7ff0a0066a80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:39.968 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.964+0000 7ff09f59e700 1 -- 172.21.15.47:0/3484303840 shutdown_connections 2022-06-29T15:42:39.968 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:39.964+0000 7ff09f59e700 1 -- 172.21.15.47:0/3484303840 wait complete. 2022-06-29T15:42:39.978 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.2 2022-06-29T15:42:39.978 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2022-06-29T15:42:40.090 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.088+0000 7fdfa359e700 1 Processor -- start 2022-06-29T15:42:40.090 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.088+0000 7fdfa359e700 1 -- start start 2022-06-29T15:42:40.092 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.088+0000 7fdfa359e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdfa40b19a0 0x7fdfa405fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:40.093 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.088+0000 7fdfa359e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdfa40acff0 0x7fdfa40ae840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:40.093 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.088+0000 7fdfa359e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfa40a97e0 con 0x7fdfa41437d0 2022-06-29T15:42:40.093 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.088+0000 7fdfa359e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfa40b1d80 con 0x7fdfa40acff0 2022-06-29T15:42:40.093 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.088+0000 7fdfa359e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa405f430 con 0x7fdfa40b19a0 2022-06-29T15:42:40.094 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.088+0000 7fdfa1d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdfa40acff0 0x7fdfa40ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:40.094 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.088+0000 7fdfa1d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdfa40acff0 0x7fdfa40ae840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46020/0 (socket says 172.21.15.47:46020) 2022-06-29T15:42:40.094 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.088+0000 7fdfa1d9b700 1 -- 172.21.15.47:0/4003460394 learned_addr learned my addr 172.21.15.47:0/4003460394 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:40.094 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.088+0000 7fdfa259c700 1 --2- 172.21.15.47:0/4003460394 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdfa40b19a0 0x7fdfa405fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:40.095 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.088+0000 7fdfa1d9b700 1 -- 172.21.15.47:0/4003460394 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdfa40b19a0 msgr2=0x7fdfa405fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:40.095 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.088+0000 7fdfa1d9b700 1 --2- 172.21.15.47:0/4003460394 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdfa40b19a0 0x7fdfa405fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:40.095 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.088+0000 7fdfa1d9b700 1 -- 172.21.15.47:0/4003460394 >> v1:172.21.15.64:6789/0 conn(0x7fdfa41437d0 legacy=0x7fdfa4143c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:40.096 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.088+0000 7fdfa159a700 1 -- 172.21.15.47:0/4003460394 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1194413271 0 0) 0x7fdfa40a97e0 con 0x7fdfa41437d0 2022-06-29T15:42:40.096 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.088+0000 7fdfa1d9b700 1 -- 172.21.15.47:0/4003460394 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa40a92a0 con 0x7fdfa40acff0 2022-06-29T15:42:40.096 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.092+0000 7fdfa1d9b700 1 --2- 172.21.15.47:0/4003460394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfa40acff0 0x7fdfa40ae840 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7fdf98002f70 tx=0x7fdf98006eb0).ready entity=mon.0 client_cookie=78bfc6ae8a1c7e0b server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:40.101 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.092+0000 7fdfa159a700 1 -- 172.21.15.47:0/4003460394 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf9800d570 con 0x7fdfa40acff0 2022-06-29T15:42:40.101 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.092+0000 7fdfa159a700 1 -- 172.21.15.47:0/4003460394 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf98000e90 con 0x7fdfa40acff0 2022-06-29T15:42:40.101 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.092+0000 7fdfa159a700 1 -- 172.21.15.47:0/4003460394 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf9800d8c0 con 0x7fdfa40acff0 2022-06-29T15:42:40.102 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.092+0000 7fdfa359e700 1 -- 172.21.15.47:0/4003460394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfa40acff0 msgr2=0x7fdfa40ae840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:40.102 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.092+0000 7fdfa359e700 1 --2- 172.21.15.47:0/4003460394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfa40acff0 0x7fdfa40ae840 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7fdf98002f70 tx=0x7fdf98006eb0).stop 2022-06-29T15:42:40.102 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.092+0000 7fdfa359e700 1 -- 172.21.15.47:0/4003460394 shutdown_connections 2022-06-29T15:42:40.102 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.092+0000 7fdfa359e700 1 --2- 172.21.15.47:0/4003460394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfa40acff0 0x7fdfa40ae840 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:40.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.092+0000 7fdfa359e700 1 --2- 172.21.15.47:0/4003460394 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdfa40b19a0 0x7fdfa405fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:40.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.092+0000 7fdfa359e700 1 -- 172.21.15.47:0/4003460394 >> 172.21.15.47:0/4003460394 conn(0x7fdfa4062df0 msgr2=0x7fdfa40631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:40.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.092+0000 7fdfa359e700 1 -- 172.21.15.47:0/4003460394 shutdown_connections 2022-06-29T15:42:40.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.092+0000 7fdfa359e700 1 -- 172.21.15.47:0/4003460394 wait complete. 2022-06-29T15:42:40.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.096+0000 7fdfa359e700 1 Processor -- start 2022-06-29T15:42:40.104 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.096+0000 7fdfa359e700 1 -- start start 2022-06-29T15:42:40.104 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.096+0000 7fdfa359e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdfa40acff0 0x7fdfa409b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:40.104 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.096+0000 7fdfa359e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfa40b19a0 0x7fdfa4142730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:40.104 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.096+0000 7fdfa359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfa41437d0 0x7fdfa4142c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:40.104 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.096+0000 7fdfa359e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa4150d40 con 0x7fdfa40b19a0 2022-06-29T15:42:40.105 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.096+0000 7fdfa359e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa4150aa0 con 0x7fdfa41437d0 2022-06-29T15:42:40.105 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.096+0000 7fdfa359e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa4063b90 con 0x7fdfa40acff0 2022-06-29T15:42:40.105 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.096+0000 7fdfa1d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfa40b19a0 0x7fdfa4142730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:40.105 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.096+0000 7fdfa1d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfa40b19a0 0x7fdfa4142730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46026/0 (socket says 172.21.15.47:46026) 2022-06-29T15:42:40.106 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.096+0000 7fdfa1d9b700 1 -- 172.21.15.47:0/3370142383 learned_addr learned my addr 172.21.15.47:0/3370142383 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:40.106 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.096+0000 7fdfa259c700 1 --2- 172.21.15.47:0/3370142383 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdfa40acff0 0x7fdfa409b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:40.106 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.096+0000 7fdfa2d9d700 1 --2- 172.21.15.47:0/3370142383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfa41437d0 0x7fdfa4142c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:40.106 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.096+0000 7fdfa1d9b700 1 -- 172.21.15.47:0/3370142383 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdfa40acff0 msgr2=0x7fdfa409b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:40.107 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.096+0000 7fdfa1d9b700 1 --2- 172.21.15.47:0/3370142383 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdfa40acff0 0x7fdfa409b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:40.107 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.096+0000 7fdfa1d9b700 1 -- 172.21.15.47:0/3370142383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfa41437d0 msgr2=0x7fdfa4142c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:40.107 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.096+0000 7fdfa1d9b700 1 --2- 172.21.15.47:0/3370142383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfa41437d0 0x7fdfa4142c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:40.107 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.096+0000 7fdfa1d9b700 1 -- 172.21.15.47:0/3370142383 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf9800b1e0 con 0x7fdfa40b19a0 2022-06-29T15:42:40.107 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.096+0000 7fdfa1d9b700 1 --2- 172.21.15.47:0/3370142383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfa40b19a0 0x7fdfa4142730 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7fdf98002f70 tx=0x7fdf98015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:40.108 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.096+0000 7fdf937fe700 1 -- 172.21.15.47:0/3370142383 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf980117e0 con 0x7fdfa40b19a0 2022-06-29T15:42:40.108 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.096+0000 7fdf937fe700 1 -- 172.21.15.47:0/3370142383 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf98011940 con 0x7fdfa40b19a0 2022-06-29T15:42:40.108 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.096+0000 7fdfa359e700 1 -- 172.21.15.47:0/3370142383 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfa41507c0 con 0x7fdfa40b19a0 2022-06-29T15:42:40.109 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.096+0000 7fdf937fe700 1 -- 172.21.15.47:0/3370142383 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf98011ca0 con 0x7fdfa40b19a0 2022-06-29T15:42:40.109 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.096+0000 7fdfa359e700 1 -- 172.21.15.47:0/3370142383 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfa4065ef0 con 0x7fdfa40b19a0 2022-06-29T15:42:40.109 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.096+0000 7fdf937fe700 1 -- 172.21.15.47:0/3370142383 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fdf98025030 con 0x7fdfa40b19a0 2022-06-29T15:42:40.109 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.096+0000 7fdfa359e700 1 -- 172.21.15.47:0/3370142383 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfa40573d0 con 0x7fdfa40b19a0 2022-06-29T15:42:40.110 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.100+0000 7fdf937fe700 1 --2- 172.21.15.47:0/3370142383 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdf80053980 0x7fdf80055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:40.110 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.100+0000 7fdf937fe700 1 -- 172.21.15.47:0/3370142383 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fdf9806d280 con 0x7fdfa40b19a0 2022-06-29T15:42:40.110 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.100+0000 7fdfa259c700 1 --2- 172.21.15.47:0/3370142383 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdf80053980 0x7fdf80055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:40.111 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.100+0000 7fdfa259c700 1 --2- 172.21.15.47:0/3370142383 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdf80053980 0x7fdf80055e40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fdf9400b000 tx=0x7fdf94006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:40.111 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.104+0000 7fdf937fe700 1 -- 172.21.15.47:0/3370142383 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdf98043fa0 con 0x7fdfa40b19a0 2022-06-29T15:42:40.237 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.236+0000 7fdfa359e700 1 -- 172.21.15.47:0/3370142383 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fdfa4063f10 con 0x7fdfa40b19a0 2022-06-29T15:42:40.238 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.236+0000 7fdf937fe700 1 -- 172.21.15.47:0/3370142383 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdf98023090 con 0x7fdfa40b19a0 2022-06-29T15:42:40.238 INFO:teuthology.orchestra.run.smithi047.stdout:60129542149 2022-06-29T15:42:40.240 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.236+0000 7fdfa359e700 1 -- 172.21.15.47:0/3370142383 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdf80053980 msgr2=0x7fdf80055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:40.240 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.236+0000 7fdfa359e700 1 --2- 172.21.15.47:0/3370142383 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdf80053980 0x7fdf80055e40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fdf9400b000 tx=0x7fdf94006df0).stop 2022-06-29T15:42:40.241 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.236+0000 7fdfa359e700 1 -- 172.21.15.47:0/3370142383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfa40b19a0 msgr2=0x7fdfa4142730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:40.241 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.236+0000 7fdfa359e700 1 --2- 172.21.15.47:0/3370142383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfa40b19a0 0x7fdfa4142730 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7fdf98002f70 tx=0x7fdf98015000).stop 2022-06-29T15:42:40.241 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.236+0000 7fdfa359e700 1 -- 172.21.15.47:0/3370142383 shutdown_connections 2022-06-29T15:42:40.241 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.236+0000 7fdfa359e700 1 --2- 172.21.15.47:0/3370142383 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdf80053980 0x7fdf80055e40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:40.242 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.236+0000 7fdfa359e700 1 --2- 172.21.15.47:0/3370142383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfa41437d0 0x7fdfa4142c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:40.242 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.236+0000 7fdfa359e700 1 --2- 172.21.15.47:0/3370142383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfa40b19a0 0x7fdfa4142730 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:40.242 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.236+0000 7fdfa359e700 1 --2- 172.21.15.47:0/3370142383 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdfa40acff0 0x7fdfa409b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:40.243 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.236+0000 7fdfa359e700 1 -- 172.21.15.47:0/3370142383 >> 172.21.15.47:0/3370142383 conn(0x7fdfa4062df0 msgr2=0x7fdfa40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:40.243 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.236+0000 7fdfa359e700 1 -- 172.21.15.47:0/3370142383 shutdown_connections 2022-06-29T15:42:40.243 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.236+0000 7fdfa359e700 1 -- 172.21.15.47:0/3370142383 wait complete. 2022-06-29T15:42:40.251 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.3 2022-06-29T15:42:40.252 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2022-06-29T15:42:40.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.356+0000 7f6649852700 1 Processor -- start 2022-06-29T15:42:40.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.356+0000 7f6649852700 1 -- start start 2022-06-29T15:42:40.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.356+0000 7f6649852700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f664405f0b0 0x7f66440b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:40.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.356+0000 7f6649852700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f66440b1260 0x7f664414b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:40.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.356+0000 7f6649852700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66440b0740 con 0x7f66440b2120 2022-06-29T15:42:40.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.356+0000 7f6649852700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6644147460 con 0x7f66440b1260 2022-06-29T15:42:40.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.356+0000 7f6649852700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66440b0970 con 0x7f664405f0b0 2022-06-29T15:42:40.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.356+0000 7f66437fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f66440b1260 0x7f664414b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:40.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.360+0000 7f6642ffd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f66440b2120 0x7f6644147af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59818/0 (socket says 172.21.15.47:59818) 2022-06-29T15:42:40.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.360+0000 7f66437fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f66440b1260 0x7f664414b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46036/0 (socket says 172.21.15.47:46036) 2022-06-29T15:42:40.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.360+0000 7f6642ffd700 1 -- 172.21.15.47:0/501552999 learned_addr learned my addr 172.21.15.47:0/501552999 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:40.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.360+0000 7f66427fc700 1 --2- 172.21.15.47:0/501552999 >> v2:172.21.15.64:3300/0 conn(0x7f664405f0b0 0x7f66440b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:40.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.360+0000 7f66437fe700 1 -- 172.21.15.47:0/501552999 >> v2:172.21.15.64:3300/0 conn(0x7f664405f0b0 msgr2=0x7f66440b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:40.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.360+0000 7f66437fe700 1 --2- 172.21.15.47:0/501552999 >> v2:172.21.15.64:3300/0 conn(0x7f664405f0b0 0x7f66440b2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:40.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.360+0000 7f66437fe700 1 -- 172.21.15.47:0/501552999 >> v1:172.21.15.47:6789/0 conn(0x7f66440b2120 legacy=0x7f6644147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:40.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.360+0000 7f66437fe700 1 -- 172.21.15.47:0/501552999 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66440b0c10 con 0x7f66440b1260 2022-06-29T15:42:40.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.360+0000 7f66437fe700 1 --2- 172.21.15.47:0/501552999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66440b1260 0x7f664414b520 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f6644060d70 tx=0x7f663c0091d0).ready entity=mon.0 client_cookie=14db564bc4902d04 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:40.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.360+0000 7f6641ffb700 1 -- 172.21.15.47:0/501552999 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f663c00b640 con 0x7f66440b1260 2022-06-29T15:42:40.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.360+0000 7f6641ffb700 1 -- 172.21.15.47:0/501552999 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f663c00b7a0 con 0x7f66440b1260 2022-06-29T15:42:40.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.360+0000 7f6641ffb700 1 -- 172.21.15.47:0/501552999 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f663c00b910 con 0x7f66440b1260 2022-06-29T15:42:40.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.360+0000 7f6649852700 1 -- 172.21.15.47:0/501552999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66440b1260 msgr2=0x7f664414b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:40.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.360+0000 7f6649852700 1 --2- 172.21.15.47:0/501552999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66440b1260 0x7f664414b520 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f6644060d70 tx=0x7f663c0091d0).stop 2022-06-29T15:42:40.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.360+0000 7f6649852700 1 -- 172.21.15.47:0/501552999 shutdown_connections 2022-06-29T15:42:40.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.360+0000 7f6649852700 1 --2- 172.21.15.47:0/501552999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66440b1260 0x7f664414b520 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:40.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.360+0000 7f6649852700 1 --2- 172.21.15.47:0/501552999 >> v2:172.21.15.64:3300/0 conn(0x7f664405f0b0 0x7f66440b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:40.370 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.360+0000 7f6649852700 1 -- 172.21.15.47:0/501552999 >> 172.21.15.47:0/501552999 conn(0x7f66440626f0 msgr2=0x7f6644062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:40.370 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.360+0000 7f6649852700 1 -- 172.21.15.47:0/501552999 shutdown_connections 2022-06-29T15:42:40.370 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.360+0000 7f6649852700 1 -- 172.21.15.47:0/501552999 wait complete. 2022-06-29T15:42:40.370 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.360+0000 7f6649852700 1 Processor -- start 2022-06-29T15:42:40.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.360+0000 7f6649852700 1 -- start start 2022-06-29T15:42:40.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.364+0000 7f6649852700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f66440b1260 0x7f66440aeec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:40.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.364+0000 7f6649852700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66440b2120 0x7f6644140890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:40.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.364+0000 7f6649852700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f664409ffd0 0x7f6644140d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:40.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.364+0000 7f6649852700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66440a0660 con 0x7f664409ffd0 2022-06-29T15:42:40.372 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.364+0000 7f6649852700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66440a0df0 con 0x7f66440b2120 2022-06-29T15:42:40.372 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.364+0000 7f6649852700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66440a1090 con 0x7f66440b1260 2022-06-29T15:42:40.372 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.364+0000 7f66437fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f664409ffd0 0x7f6644140d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:40.372 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.364+0000 7f66437fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f664409ffd0 0x7f6644140d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46042/0 (socket says 172.21.15.47:46042) 2022-06-29T15:42:40.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.364+0000 7f66437fe700 1 -- 172.21.15.47:0/4071778321 learned_addr learned my addr 172.21.15.47:0/4071778321 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:40.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.364+0000 7f6642ffd700 1 --2- 172.21.15.47:0/4071778321 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f66440b1260 0x7f66440aeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:40.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.364+0000 7f66427fc700 1 --2- 172.21.15.47:0/4071778321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66440b2120 0x7f6644140890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:40.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.364+0000 7f66437fe700 1 -- 172.21.15.47:0/4071778321 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f66440b1260 msgr2=0x7f66440aeec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:40.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.364+0000 7f66437fe700 1 --2- 172.21.15.47:0/4071778321 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f66440b1260 0x7f66440aeec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:40.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.364+0000 7f66437fe700 1 -- 172.21.15.47:0/4071778321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66440b2120 msgr2=0x7f6644140890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:40.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.364+0000 7f66437fe700 1 --2- 172.21.15.47:0/4071778321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66440b2120 0x7f6644140890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:40.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.364+0000 7f66437fe700 1 -- 172.21.15.47:0/4071778321 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f663c009470 con 0x7f664409ffd0 2022-06-29T15:42:40.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.364+0000 7f66437fe700 1 --2- 172.21.15.47:0/4071778321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f664409ffd0 0x7f6644140d80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f6644060d70 tx=0x7f663c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:40.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.364+0000 7f6633fff700 1 -- 172.21.15.47:0/4071778321 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f663c011c10 con 0x7f664409ffd0 2022-06-29T15:42:40.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.364+0000 7f6633fff700 1 -- 172.21.15.47:0/4071778321 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f663c011d70 con 0x7f664409ffd0 2022-06-29T15:42:40.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.364+0000 7f6649852700 1 -- 172.21.15.47:0/4071778321 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6644064bc0 con 0x7f664409ffd0 2022-06-29T15:42:40.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.364+0000 7f6633fff700 1 -- 172.21.15.47:0/4071778321 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f663c01f5e0 con 0x7f664409ffd0 2022-06-29T15:42:40.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.364+0000 7f6649852700 1 -- 172.21.15.47:0/4071778321 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f664409fd70 con 0x7f664409ffd0 2022-06-29T15:42:40.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.368+0000 7f6649852700 1 -- 172.21.15.47:0/4071778321 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66440a1250 con 0x7f664409ffd0 2022-06-29T15:42:40.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.368+0000 7f6633fff700 1 -- 172.21.15.47:0/4071778321 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f663c026030 con 0x7f664409ffd0 2022-06-29T15:42:40.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.368+0000 7f6633fff700 1 --2- 172.21.15.47:0/4071778321 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6620053aa0 0x7f6620055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:40.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.368+0000 7f6642ffd700 1 --2- 172.21.15.47:0/4071778321 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6620053aa0 0x7f6620055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:40.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.368+0000 7f6633fff700 1 -- 172.21.15.47:0/4071778321 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f663c06cad0 con 0x7f664409ffd0 2022-06-29T15:42:40.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.368+0000 7f6642ffd700 1 --2- 172.21.15.47:0/4071778321 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6620053aa0 0x7f6620055f60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f663800d000 tx=0x7f6638000bc0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:40.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.376+0000 7f6633fff700 1 -- 172.21.15.47:0/4071778321 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f663c043890 con 0x7f664409ffd0 2022-06-29T15:42:40.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.504+0000 7f6649852700 1 -- 172.21.15.47:0/4071778321 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f66440573d0 con 0x7f664409ffd0 2022-06-29T15:42:40.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.504+0000 7f6633fff700 1 -- 172.21.15.47:0/4071778321 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f663c015020 con 0x7f664409ffd0 2022-06-29T15:42:40.508 INFO:teuthology.orchestra.run.smithi047.stdout:60129542149 2022-06-29T15:42:40.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.508+0000 7f6649852700 1 -- 172.21.15.47:0/4071778321 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6620053aa0 msgr2=0x7f6620055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:40.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.508+0000 7f6649852700 1 --2- 172.21.15.47:0/4071778321 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6620053aa0 0x7f6620055f60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f663800d000 tx=0x7f6638000bc0).stop 2022-06-29T15:42:40.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.508+0000 7f6649852700 1 -- 172.21.15.47:0/4071778321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f664409ffd0 msgr2=0x7f6644140d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:40.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.508+0000 7f6649852700 1 --2- 172.21.15.47:0/4071778321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f664409ffd0 0x7f6644140d80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f6644060d70 tx=0x7f663c016000).stop 2022-06-29T15:42:40.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.508+0000 7f6649852700 1 -- 172.21.15.47:0/4071778321 shutdown_connections 2022-06-29T15:42:40.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.508+0000 7f6649852700 1 --2- 172.21.15.47:0/4071778321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f664409ffd0 0x7f6644140d80 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:40.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.508+0000 7f6649852700 1 --2- 172.21.15.47:0/4071778321 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6620053aa0 0x7f6620055f60 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:40.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.508+0000 7f6649852700 1 --2- 172.21.15.47:0/4071778321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66440b2120 0x7f6644140890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:40.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.508+0000 7f6649852700 1 --2- 172.21.15.47:0/4071778321 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f66440b1260 0x7f66440aeec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:40.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.508+0000 7f6649852700 1 -- 172.21.15.47:0/4071778321 >> 172.21.15.47:0/4071778321 conn(0x7f66440626f0 msgr2=0x7f66440b5480 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:40.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.508+0000 7f6649852700 1 -- 172.21.15.47:0/4071778321 shutdown_connections 2022-06-29T15:42:40.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.508+0000 7f6649852700 1 -- 172.21.15.47:0/4071778321 wait complete. 2022-06-29T15:42:40.520 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.4 2022-06-29T15:42:40.521 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2022-06-29T15:42:40.627 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.624+0000 7f83f96be700 1 Processor -- start 2022-06-29T15:42:40.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.624+0000 7f83f96be700 1 -- start start 2022-06-29T15:42:40.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.624+0000 7f83f96be700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f83f40611b0 0x7f83f4146640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:40.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.624+0000 7f83f96be700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f83f40b0cf0 0x7f83f4146b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:40.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.624+0000 7f83f96be700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83f40b4520 con 0x7f83f40b06d0 2022-06-29T15:42:40.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.624+0000 7f83f96be700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83f40b4b10 con 0x7f83f40611b0 2022-06-29T15:42:40.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.624+0000 7f83f96be700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f40621d0 con 0x7f83f40b0cf0 2022-06-29T15:42:40.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.624+0000 7f83f37fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f83f40b06d0 0x7f83f40667e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59836/0 (socket says 172.21.15.47:59836) 2022-06-29T15:42:40.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.624+0000 7f83f37fe700 1 -- 172.21.15.47:0/2073464130 learned_addr learned my addr 172.21.15.47:0/2073464130 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:40.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.624+0000 7f83f27fc700 1 --2- 172.21.15.47:0/2073464130 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f83f40b0cf0 0x7f83f4146b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:40.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.624+0000 7f83f2ffd700 1 --2- 172.21.15.47:0/2073464130 >> v2:172.21.15.64:3300/0 conn(0x7f83f40611b0 0x7f83f4146640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:40.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.624+0000 7f83f1ffb700 1 -- 172.21.15.47:0/2073464130 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 608726077 0 0) 0x7f83f40b4520 con 0x7f83f40b06d0 2022-06-29T15:42:40.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.628+0000 7f83f1ffb700 1 -- 172.21.15.47:0/2073464130 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83d4003260 con 0x7f83f40b06d0 2022-06-29T15:42:40.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.628+0000 7f83f27fc700 1 -- 172.21.15.47:0/2073464130 >> v2:172.21.15.64:3300/0 conn(0x7f83f40611b0 msgr2=0x7f83f4146640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:40.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.628+0000 7f83f27fc700 1 --2- 172.21.15.47:0/2073464130 >> v2:172.21.15.64:3300/0 conn(0x7f83f40611b0 0x7f83f4146640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:40.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.628+0000 7f83f27fc700 1 -- 172.21.15.47:0/2073464130 >> v1:172.21.15.47:6789/0 conn(0x7f83f40b06d0 legacy=0x7f83f40667e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:40.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.628+0000 7f83f27fc700 1 -- 172.21.15.47:0/2073464130 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f4061590 con 0x7f83f40b0cf0 2022-06-29T15:42:40.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.628+0000 7f83f27fc700 1 --2- 172.21.15.47:0/2073464130 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f83f40b0cf0 0x7f83f4146b30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f83dc0032f0 tx=0x7f83dc0091d0).ready entity=mon.2 client_cookie=9a346a52e5919eb6 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:40.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.628+0000 7f83f1ffb700 1 -- 172.21.15.47:0/2073464130 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83dc00b600 con 0x7f83f40b0cf0 2022-06-29T15:42:40.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.628+0000 7f83f1ffb700 1 -- 172.21.15.47:0/2073464130 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83dc00b760 con 0x7f83f40b0cf0 2022-06-29T15:42:40.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.628+0000 7f83f1ffb700 1 -- 172.21.15.47:0/2073464130 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83dc00b8d0 con 0x7f83f40b0cf0 2022-06-29T15:42:40.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.628+0000 7f83f96be700 1 -- 172.21.15.47:0/2073464130 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f83f40b0cf0 msgr2=0x7f83f4146b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:40.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.628+0000 7f83f96be700 1 --2- 172.21.15.47:0/2073464130 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f83f40b0cf0 0x7f83f4146b30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f83dc0032f0 tx=0x7f83dc0091d0).stop 2022-06-29T15:42:40.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.628+0000 7f83f96be700 1 -- 172.21.15.47:0/2073464130 shutdown_connections 2022-06-29T15:42:40.651 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.628+0000 7f83f96be700 1 --2- 172.21.15.47:0/2073464130 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f83f40b0cf0 0x7f83f4146b30 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:40.651 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.628+0000 7f83f96be700 1 --2- 172.21.15.47:0/2073464130 >> v2:172.21.15.64:3300/0 conn(0x7f83f40611b0 0x7f83f4146640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:40.652 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.628+0000 7f83f96be700 1 -- 172.21.15.47:0/2073464130 >> 172.21.15.47:0/2073464130 conn(0x7f83f40b98b0 msgr2=0x7f83f40b9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:40.652 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.628+0000 7f83f96be700 1 -- 172.21.15.47:0/2073464130 shutdown_connections 2022-06-29T15:42:40.653 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.628+0000 7f83f96be700 1 -- 172.21.15.47:0/2073464130 wait complete. 2022-06-29T15:42:40.653 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.628+0000 7f83f96be700 1 Processor -- start 2022-06-29T15:42:40.653 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.628+0000 7f83f96be700 1 -- start start 2022-06-29T15:42:40.654 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.632+0000 7f83f96be700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83f40611b0 0x7f83f4152fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:40.654 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.632+0000 7f83f96be700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f83f40b06d0 0x7f83f4138760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:40.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.632+0000 7f83f96be700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83f40b0cf0 0x7f83f413eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:40.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.632+0000 7f83f96be700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f41523f0 con 0x7f83f40611b0 2022-06-29T15:42:40.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.632+0000 7f83f96be700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f41525e0 con 0x7f83f40b0cf0 2022-06-29T15:42:40.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.632+0000 7f83f96be700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f4152880 con 0x7f83f40b06d0 2022-06-29T15:42:40.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.632+0000 7f83f2ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83f40611b0 0x7f83f4152fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:40.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.632+0000 7f83f2ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83f40611b0 0x7f83f4152fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46052/0 (socket says 172.21.15.47:46052) 2022-06-29T15:42:40.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.632+0000 7f83f2ffd700 1 -- 172.21.15.47:0/1468784856 learned_addr learned my addr 172.21.15.47:0/1468784856 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:40.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.632+0000 7f83f37fe700 1 --2- 172.21.15.47:0/1468784856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83f40b0cf0 0x7f83f413eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:40.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.632+0000 7f83f27fc700 1 --2- 172.21.15.47:0/1468784856 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f83f40b06d0 0x7f83f4138760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:40.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.632+0000 7f83f2ffd700 1 -- 172.21.15.47:0/1468784856 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f83f40b06d0 msgr2=0x7f83f4138760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:40.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.632+0000 7f83f2ffd700 1 --2- 172.21.15.47:0/1468784856 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f83f40b06d0 0x7f83f4138760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:40.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.632+0000 7f83f2ffd700 1 -- 172.21.15.47:0/1468784856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83f40b0cf0 msgr2=0x7f83f413eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:40.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.632+0000 7f83f2ffd700 1 --2- 172.21.15.47:0/1468784856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83f40b0cf0 0x7f83f413eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:40.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.632+0000 7f83f2ffd700 1 -- 172.21.15.47:0/1468784856 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83dc009470 con 0x7f83f40611b0 2022-06-29T15:42:40.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.632+0000 7f83f2ffd700 1 --2- 172.21.15.47:0/1468784856 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83f40611b0 0x7f83f4152fc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f83e400c000 tx=0x7f83e4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:40.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.632+0000 7f83ebfff700 1 -- 172.21.15.47:0/1468784856 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83e4010780 con 0x7f83f40611b0 2022-06-29T15:42:40.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.632+0000 7f83ebfff700 1 -- 172.21.15.47:0/1468784856 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e40108e0 con 0x7f83f40611b0 2022-06-29T15:42:40.663 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.632+0000 7f83f96be700 1 -- 172.21.15.47:0/1468784856 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83f4152b20 con 0x7f83f40611b0 2022-06-29T15:42:40.663 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.632+0000 7f83ebfff700 1 -- 172.21.15.47:0/1468784856 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83e4010c40 con 0x7f83f40611b0 2022-06-29T15:42:40.664 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.632+0000 7f83f96be700 1 -- 172.21.15.47:0/1468784856 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83f413f8a0 con 0x7f83f40611b0 2022-06-29T15:42:40.664 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.636+0000 7f83f96be700 1 -- 172.21.15.47:0/1468784856 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83f40573d0 con 0x7f83f40611b0 2022-06-29T15:42:40.664 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.636+0000 7f83ebfff700 1 -- 172.21.15.47:0/1468784856 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f83e401c460 con 0x7f83f40611b0 2022-06-29T15:42:40.664 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.636+0000 7f83ebfff700 1 --2- 172.21.15.47:0/1468784856 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f83d8053a70 0x7f83d8055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:40.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.636+0000 7f83ebfff700 1 -- 172.21.15.47:0/1468784856 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f83e406c410 con 0x7f83f40611b0 2022-06-29T15:42:40.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.636+0000 7f83f27fc700 1 --2- 172.21.15.47:0/1468784856 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f83d8053a70 0x7f83d8055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:40.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.636+0000 7f83f27fc700 1 --2- 172.21.15.47:0/1468784856 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f83d8053a70 0x7f83d8055f30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f83dc01c000 tx=0x7f83dc016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:40.666 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.640+0000 7f83ebfff700 1 -- 172.21.15.47:0/1468784856 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f83e4043170 con 0x7f83f40611b0 2022-06-29T15:42:40.772 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.768+0000 7f83f96be700 1 -- 172.21.15.47:0/1468784856 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f83f4061750 con 0x7f83f40611b0 2022-06-29T15:42:40.772 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.768+0000 7f83ebfff700 1 -- 172.21.15.47:0/1468784856 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f83e4046790 con 0x7f83f40611b0 2022-06-29T15:42:40.773 INFO:teuthology.orchestra.run.smithi047.stdout:60129542150 2022-06-29T15:42:40.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.772+0000 7f83f96be700 1 -- 172.21.15.47:0/1468784856 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f83d8053a70 msgr2=0x7f83d8055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:40.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.772+0000 7f83f96be700 1 --2- 172.21.15.47:0/1468784856 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f83d8053a70 0x7f83d8055f30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f83dc01c000 tx=0x7f83dc016000).stop 2022-06-29T15:42:40.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.772+0000 7f83f96be700 1 -- 172.21.15.47:0/1468784856 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83f40611b0 msgr2=0x7f83f4152fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:40.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.772+0000 7f83f96be700 1 --2- 172.21.15.47:0/1468784856 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83f40611b0 0x7f83f4152fc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f83e400c000 tx=0x7f83e4006df0).stop 2022-06-29T15:42:40.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.772+0000 7f83f96be700 1 -- 172.21.15.47:0/1468784856 shutdown_connections 2022-06-29T15:42:40.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.772+0000 7f83f96be700 1 --2- 172.21.15.47:0/1468784856 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f83d8053a70 0x7f83d8055f30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:40.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.772+0000 7f83f96be700 1 --2- 172.21.15.47:0/1468784856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83f40b0cf0 0x7f83f413eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:40.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.772+0000 7f83f96be700 1 --2- 172.21.15.47:0/1468784856 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f83f40b06d0 0x7f83f4138760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:40.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.772+0000 7f83f96be700 1 --2- 172.21.15.47:0/1468784856 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83f40611b0 0x7f83f4152fc0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:40.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.772+0000 7f83f96be700 1 -- 172.21.15.47:0/1468784856 >> 172.21.15.47:0/1468784856 conn(0x7f83f40b98b0 msgr2=0x7f83f41430f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:40.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.772+0000 7f83f96be700 1 -- 172.21.15.47:0/1468784856 shutdown_connections 2022-06-29T15:42:40.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.776+0000 7f83f96be700 1 -- 172.21.15.47:0/1468784856 wait complete. 2022-06-29T15:42:40.791 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.5 2022-06-29T15:42:40.792 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2022-06-29T15:42:40.814 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:42:40.838 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:42:40.841 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:42:40.843 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:42:40.897 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.896+0000 7f8d3a6ca700 1 Processor -- start 2022-06-29T15:42:40.898 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.896+0000 7f8d3a6ca700 1 -- start start 2022-06-29T15:42:40.900 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.896+0000 7f8d3a6ca700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8d341430f0 0x7f8d341434d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:40.900 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.896+0000 7f8d3a6ca700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d34095dc0 con 0x7f8d340a93c0 2022-06-29T15:42:40.901 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.896+0000 7f8d3a6ca700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d34060e50 con 0x7f8d34095950 2022-06-29T15:42:40.901 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.896+0000 7f8d3a6ca700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d3413d590 con 0x7f8d341430f0 2022-06-29T15:42:40.901 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.896+0000 7f8d337fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f8d340a93c0 0x7f8d34095240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59848/0 (socket says 172.21.15.47:59848) 2022-06-29T15:42:40.901 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.896+0000 7f8d337fe700 1 -- 172.21.15.47:0/1041049067 learned_addr learned my addr 172.21.15.47:0/1041049067 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:40.902 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.896+0000 7f8d33fff700 1 --2- 172.21.15.47:0/1041049067 >> v2:172.21.15.64:3300/0 conn(0x7f8d341430f0 0x7f8d341434d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:40.902 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.896+0000 7f8d32ffd700 1 -- 172.21.15.47:0/1041049067 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 296810703 0 0) 0x7f8d34095dc0 con 0x7f8d340a93c0 2022-06-29T15:42:40.902 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.896+0000 7f8d32ffd700 1 -- 172.21.15.47:0/1041049067 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d14003260 con 0x7f8d340a93c0 2022-06-29T15:42:40.902 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.896+0000 7f8d33fff700 1 -- 172.21.15.47:0/1041049067 >> v1:172.21.15.64:6789/0 conn(0x7f8d34095950 legacy=0x7f8d34097690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:40.903 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.896+0000 7f8d32ffd700 1 -- 172.21.15.47:0/1041049067 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1250653044 0 0) 0x7f8d14003260 con 0x7f8d340a93c0 2022-06-29T15:42:40.903 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.896+0000 7f8d33fff700 1 -- 172.21.15.47:0/1041049067 >> v1:172.21.15.47:6789/0 conn(0x7f8d340a93c0 legacy=0x7f8d34095240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:40.903 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.896+0000 7f8d33fff700 1 -- 172.21.15.47:0/1041049067 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d340610f0 con 0x7f8d341430f0 2022-06-29T15:42:40.903 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.896+0000 7f8d33fff700 1 --2- 172.21.15.47:0/1041049067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d341430f0 0x7f8d341434d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f8d1c001800 tx=0x7f8d240091d0).ready entity=mon.1 client_cookie=26db085950969fe1 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:40.904 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.896+0000 7f8d32ffd700 1 -- 172.21.15.47:0/1041049067 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d1c001660 con 0x7f8d341430f0 2022-06-29T15:42:40.904 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.896+0000 7f8d32ffd700 1 -- 172.21.15.47:0/1041049067 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d2400b580 con 0x7f8d341430f0 2022-06-29T15:42:40.904 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.896+0000 7f8d32ffd700 1 -- 172.21.15.47:0/1041049067 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d2400b6f0 con 0x7f8d341430f0 2022-06-29T15:42:40.909 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.900+0000 7f8d3a6ca700 1 -- 172.21.15.47:0/1041049067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d341430f0 msgr2=0x7f8d341434d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:40.909 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.900+0000 7f8d3a6ca700 1 --2- 172.21.15.47:0/1041049067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d341430f0 0x7f8d341434d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f8d1c001800 tx=0x7f8d240091d0).stop 2022-06-29T15:42:40.909 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.900+0000 7f8d3a6ca700 1 -- 172.21.15.47:0/1041049067 shutdown_connections 2022-06-29T15:42:40.910 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.900+0000 7f8d3a6ca700 1 --2- 172.21.15.47:0/1041049067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d341430f0 0x7f8d341434d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:40.910 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.900+0000 7f8d3a6ca700 1 -- 172.21.15.47:0/1041049067 >> 172.21.15.47:0/1041049067 conn(0x7f8d3405f0b0 msgr2=0x7f8d3405f490 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:40.910 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.900+0000 7f8d3a6ca700 1 -- 172.21.15.47:0/1041049067 shutdown_connections 2022-06-29T15:42:40.910 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.900+0000 7f8d3a6ca700 1 -- 172.21.15.47:0/1041049067 wait complete. 2022-06-29T15:42:40.911 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.900+0000 7f8d3a6ca700 1 Processor -- start 2022-06-29T15:42:40.911 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.900+0000 7f8d3a6ca700 1 -- start start 2022-06-29T15:42:40.911 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.900+0000 7f8d3a6ca700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8d34095950 0x7f8d341405e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:40.911 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.900+0000 7f8d3a6ca700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d340a93c0 0x7f8d3409b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:40.912 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.900+0000 7f8d3a6ca700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d341430f0 0x7f8d34065f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:40.912 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.900+0000 7f8d3a6ca700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d34064000 con 0x7f8d340a93c0 2022-06-29T15:42:40.912 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.900+0000 7f8d3a6ca700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3413eb80 con 0x7f8d341430f0 2022-06-29T15:42:40.912 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.900+0000 7f8d3a6ca700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d34066cb0 con 0x7f8d34095950 2022-06-29T15:42:40.913 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.900+0000 7f8d337fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d340a93c0 0x7f8d3409b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:40.913 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.900+0000 7f8d337fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d340a93c0 0x7f8d3409b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46068/0 (socket says 172.21.15.47:46068) 2022-06-29T15:42:40.913 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.900+0000 7f8d337fe700 1 -- 172.21.15.47:0/386897296 learned_addr learned my addr 172.21.15.47:0/386897296 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:40.913 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.900+0000 7f8d38c67700 1 --2- 172.21.15.47:0/386897296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d341430f0 0x7f8d34065f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:40.914 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.900+0000 7f8d33fff700 1 --2- 172.21.15.47:0/386897296 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8d34095950 0x7f8d341405e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:40.914 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.900+0000 7f8d337fe700 1 -- 172.21.15.47:0/386897296 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8d34095950 msgr2=0x7f8d341405e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:40.914 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.900+0000 7f8d337fe700 1 --2- 172.21.15.47:0/386897296 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8d34095950 0x7f8d341405e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:40.914 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.900+0000 7f8d337fe700 1 -- 172.21.15.47:0/386897296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d341430f0 msgr2=0x7f8d34065f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:40.915 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.900+0000 7f8d337fe700 1 --2- 172.21.15.47:0/386897296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d341430f0 0x7f8d34065f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:40.915 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.900+0000 7f8d337fe700 1 -- 172.21.15.47:0/386897296 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d24009470 con 0x7f8d340a93c0 2022-06-29T15:42:40.915 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.900+0000 7f8d38c67700 1 --2- 172.21.15.47:0/386897296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d341430f0 0x7f8d34065f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:42:40.915 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.900+0000 7f8d337fe700 1 --2- 172.21.15.47:0/386897296 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d340a93c0 0x7f8d3409b1d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f8d1c0018c0 tx=0x7f8d1c001a80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:40.916 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.900+0000 7f8d317fa700 1 -- 172.21.15.47:0/386897296 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d1c007170 con 0x7f8d340a93c0 2022-06-29T15:42:40.916 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.900+0000 7f8d317fa700 1 -- 172.21.15.47:0/386897296 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1c0072d0 con 0x7f8d340a93c0 2022-06-29T15:42:40.916 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.900+0000 7f8d3a6ca700 1 -- 172.21.15.47:0/386897296 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d34066f50 con 0x7f8d340a93c0 2022-06-29T15:42:40.916 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.900+0000 7f8d317fa700 1 -- 172.21.15.47:0/386897296 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d1c004920 con 0x7f8d340a93c0 2022-06-29T15:42:40.917 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.900+0000 7f8d3a6ca700 1 -- 172.21.15.47:0/386897296 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d34063bd0 con 0x7f8d340a93c0 2022-06-29T15:42:40.917 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.904+0000 7f8d3a6ca700 1 -- 172.21.15.47:0/386897296 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d340573d0 con 0x7f8d340a93c0 2022-06-29T15:42:40.917 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.904+0000 7f8d317fa700 1 -- 172.21.15.47:0/386897296 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8d1c01b070 con 0x7f8d340a93c0 2022-06-29T15:42:40.917 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.904+0000 7f8d317fa700 1 --2- 172.21.15.47:0/386897296 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8d18053950 0x7f8d18055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:40.918 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.904+0000 7f8d317fa700 1 -- 172.21.15.47:0/386897296 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f8d1c06ba00 con 0x7f8d340a93c0 2022-06-29T15:42:40.918 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.904+0000 7f8d33fff700 1 --2- 172.21.15.47:0/386897296 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8d18053950 0x7f8d18055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:40.918 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.904+0000 7f8d33fff700 1 --2- 172.21.15.47:0/386897296 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8d18053950 0x7f8d18055e10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f8d2401b000 tx=0x7f8d24016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:40.919 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:40.912+0000 7f8d317fa700 1 -- 172.21.15.47:0/386897296 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8d1c042720 con 0x7f8d340a93c0 2022-06-29T15:42:41.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:41.044+0000 7f8d3a6ca700 1 -- 172.21.15.47:0/386897296 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f8d3409bdb0 con 0x7f8d340a93c0 2022-06-29T15:42:41.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:41.044+0000 7f8d317fa700 1 -- 172.21.15.47:0/386897296 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8d1c045d40 con 0x7f8d340a93c0 2022-06-29T15:42:41.047 INFO:teuthology.orchestra.run.smithi047.stdout:60129542149 2022-06-29T15:42:41.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:41.044+0000 7f8d3a6ca700 1 -- 172.21.15.47:0/386897296 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8d18053950 msgr2=0x7f8d18055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:41.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:41.048+0000 7f8d3a6ca700 1 --2- 172.21.15.47:0/386897296 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8d18053950 0x7f8d18055e10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f8d2401b000 tx=0x7f8d24016000).stop 2022-06-29T15:42:41.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:41.048+0000 7f8d3a6ca700 1 -- 172.21.15.47:0/386897296 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d340a93c0 msgr2=0x7f8d3409b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:41.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:41.048+0000 7f8d3a6ca700 1 --2- 172.21.15.47:0/386897296 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d340a93c0 0x7f8d3409b1d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f8d1c0018c0 tx=0x7f8d1c001a80).stop 2022-06-29T15:42:41.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:41.048+0000 7f8d3a6ca700 1 -- 172.21.15.47:0/386897296 shutdown_connections 2022-06-29T15:42:41.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:41.048+0000 7f8d3a6ca700 1 --2- 172.21.15.47:0/386897296 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8d18053950 0x7f8d18055e10 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:41.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:41.048+0000 7f8d3a6ca700 1 --2- 172.21.15.47:0/386897296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d341430f0 0x7f8d34065f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:41.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:41.048+0000 7f8d3a6ca700 1 --2- 172.21.15.47:0/386897296 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d340a93c0 0x7f8d3409b1d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:41.052 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:41.048+0000 7f8d3a6ca700 1 --2- 172.21.15.47:0/386897296 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8d34095950 0x7f8d341405e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:41.052 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:41.048+0000 7f8d3a6ca700 1 -- 172.21.15.47:0/386897296 >> 172.21.15.47:0/386897296 conn(0x7f8d3405f0b0 msgr2=0x7f8d340943d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:41.052 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:41.048+0000 7f8d3a6ca700 1 -- 172.21.15.47:0/386897296 shutdown_connections 2022-06-29T15:42:41.052 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:41.048+0000 7f8d3a6ca700 1 -- 172.21.15.47:0/386897296 wait complete. 2022-06-29T15:42:41.061 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.6 2022-06-29T15:42:42.062 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2022-06-29T15:42:42.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.172+0000 7f45a687b700 1 Processor -- start 2022-06-29T15:42:42.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.172+0000 7f45a687b700 1 -- start start 2022-06-29T15:42:42.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.176+0000 7f45a687b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f45a005f0b0 0x7f45a00b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:42.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.176+0000 7f45a687b700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45a00b0740 con 0x7f45a00b1260 2022-06-29T15:42:42.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.176+0000 7f45a687b700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45a00b0970 con 0x7f45a00b2120 2022-06-29T15:42:42.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.176+0000 7f45a687b700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45a0147460 con 0x7f45a005f0b0 2022-06-29T15:42:42.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.176+0000 7f45a4e18700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f45a00b1260 0x7f45a005f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59864/0 (socket says 172.21.15.47:59864) 2022-06-29T15:42:42.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.176+0000 7f45a4e18700 1 -- 172.21.15.47:0/570597519 learned_addr learned my addr 172.21.15.47:0/570597519 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:42.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.176+0000 7f459f7fe700 1 --2- 172.21.15.47:0/570597519 >> v2:172.21.15.64:3300/0 conn(0x7f45a005f0b0 0x7f45a00b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:42.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.176+0000 7f459effd700 1 -- 172.21.15.47:0/570597519 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 220304623 0 0) 0x7f45a00b0740 con 0x7f45a00b1260 2022-06-29T15:42:42.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.176+0000 7f459effd700 1 -- 172.21.15.47:0/570597519 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f458c003260 con 0x7f45a00b1260 2022-06-29T15:42:42.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.176+0000 7f459effd700 1 -- 172.21.15.47:0/570597519 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4092253472 0 0) 0x7f45a00b0970 con 0x7f45a00b2120 2022-06-29T15:42:42.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.176+0000 7f459effd700 1 -- 172.21.15.47:0/570597519 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45a00b0740 con 0x7f45a00b2120 2022-06-29T15:42:42.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.176+0000 7f459f7fe700 1 -- 172.21.15.47:0/570597519 >> v1:172.21.15.64:6789/0 conn(0x7f45a00b2120 legacy=0x7f45a0147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:42.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.176+0000 7f459f7fe700 1 -- 172.21.15.47:0/570597519 >> v1:172.21.15.47:6789/0 conn(0x7f45a00b1260 legacy=0x7f45a005f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:42.182 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.176+0000 7f459f7fe700 1 -- 172.21.15.47:0/570597519 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a00b0c10 con 0x7f45a005f0b0 2022-06-29T15:42:42.182 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.176+0000 7f459f7fe700 1 --2- 172.21.15.47:0/570597519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45a005f0b0 0x7f45a00b2500 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f45a00affe0 tx=0x7f4590006df0).ready entity=mon.1 client_cookie=a8f7b5a71991b1c8 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:42.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.176+0000 7f459effd700 1 -- 172.21.15.47:0/570597519 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f459000d540 con 0x7f45a005f0b0 2022-06-29T15:42:42.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.176+0000 7f459effd700 1 -- 172.21.15.47:0/570597519 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4590000e90 con 0x7f45a005f0b0 2022-06-29T15:42:42.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.176+0000 7f459effd700 1 -- 172.21.15.47:0/570597519 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f459000d890 con 0x7f45a005f0b0 2022-06-29T15:42:42.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.176+0000 7f45a687b700 1 -- 172.21.15.47:0/570597519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45a005f0b0 msgr2=0x7f45a00b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:42.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.176+0000 7f45a687b700 1 --2- 172.21.15.47:0/570597519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45a005f0b0 0x7f45a00b2500 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f45a00affe0 tx=0x7f4590006df0).stop 2022-06-29T15:42:42.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.176+0000 7f45a687b700 1 -- 172.21.15.47:0/570597519 shutdown_connections 2022-06-29T15:42:42.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.176+0000 7f45a687b700 1 --2- 172.21.15.47:0/570597519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45a005f0b0 0x7f45a00b2500 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:42.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.176+0000 7f45a687b700 1 -- 172.21.15.47:0/570597519 >> 172.21.15.47:0/570597519 conn(0x7f45a00626f0 msgr2=0x7f45a0062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:42.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.176+0000 7f45a687b700 1 -- 172.21.15.47:0/570597519 shutdown_connections 2022-06-29T15:42:42.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.176+0000 7f45a687b700 1 -- 172.21.15.47:0/570597519 wait complete. 2022-06-29T15:42:42.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.180+0000 7f45a687b700 1 Processor -- start 2022-06-29T15:42:42.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.180+0000 7f45a687b700 1 -- start start 2022-06-29T15:42:42.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.180+0000 7f45a687b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45a00b1260 0x7f45a00b4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:42.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.180+0000 7f45a687b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45a00b2120 0x7f45a015a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:42.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.180+0000 7f45a687b700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f45a01435a0 0x7f45a013e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:42.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.180+0000 7f45a687b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a01431f0 con 0x7f45a00b1260 2022-06-29T15:42:42.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.180+0000 7f45a687b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a015ae30 con 0x7f45a00b2120 2022-06-29T15:42:42.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.180+0000 7f45a687b700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45a0142570 con 0x7f45a01435a0 2022-06-29T15:42:42.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.180+0000 7f459ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45a00b1260 0x7f45a00b4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:42.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.180+0000 7f459ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45a00b1260 0x7f45a00b4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46080/0 (socket says 172.21.15.47:46080) 2022-06-29T15:42:42.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.180+0000 7f459ffff700 1 -- 172.21.15.47:0/1404560963 learned_addr learned my addr 172.21.15.47:0/1404560963 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:42.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.180+0000 7f45a4e18700 1 --2- 172.21.15.47:0/1404560963 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f45a01435a0 0x7f45a013e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:42.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.180+0000 7f459f7fe700 1 --2- 172.21.15.47:0/1404560963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45a00b2120 0x7f45a015a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:42.191 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.180+0000 7f459ffff700 1 -- 172.21.15.47:0/1404560963 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f45a01435a0 msgr2=0x7f45a013e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:42.191 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.180+0000 7f459ffff700 1 --2- 172.21.15.47:0/1404560963 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f45a01435a0 0x7f45a013e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:42.191 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.180+0000 7f459ffff700 1 -- 172.21.15.47:0/1404560963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45a00b2120 msgr2=0x7f45a015a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:42.192 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.180+0000 7f459ffff700 1 --2- 172.21.15.47:0/1404560963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45a00b2120 0x7f45a015a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:42.192 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.180+0000 7f459ffff700 1 -- 172.21.15.47:0/1404560963 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f459000b180 con 0x7f45a00b1260 2022-06-29T15:42:42.192 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.180+0000 7f459ffff700 1 --2- 172.21.15.47:0/1404560963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45a00b1260 0x7f45a00b4950 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f45a015af70 tx=0x7f4594007ae0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:42.192 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.180+0000 7f459d7fa700 1 -- 172.21.15.47:0/1404560963 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4594012400 con 0x7f45a00b1260 2022-06-29T15:42:42.193 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.180+0000 7f459d7fa700 1 -- 172.21.15.47:0/1404560963 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4594012560 con 0x7f45a00b1260 2022-06-29T15:42:42.193 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.180+0000 7f45a687b700 1 -- 172.21.15.47:0/1404560963 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45a0142810 con 0x7f45a00b1260 2022-06-29T15:42:42.193 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.180+0000 7f459d7fa700 1 -- 172.21.15.47:0/1404560963 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45940128c0 con 0x7f45a00b1260 2022-06-29T15:42:42.193 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.180+0000 7f45a687b700 1 -- 172.21.15.47:0/1404560963 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45a0142d50 con 0x7f45a00b1260 2022-06-29T15:42:42.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.184+0000 7f459d7fa700 1 -- 172.21.15.47:0/1404560963 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4594019030 con 0x7f45a00b1260 2022-06-29T15:42:42.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.184+0000 7f45a687b700 1 -- 172.21.15.47:0/1404560963 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45a0147650 con 0x7f45a00b1260 2022-06-29T15:42:42.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.184+0000 7f459d7fa700 1 --2- 172.21.15.47:0/1404560963 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4580053aa0 0x7f4580055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:42.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.184+0000 7f459d7fa700 1 -- 172.21.15.47:0/1404560963 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f459406b2e0 con 0x7f45a00b1260 2022-06-29T15:42:42.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.184+0000 7f459f7fe700 1 --2- 172.21.15.47:0/1404560963 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4580053aa0 0x7f4580055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:42.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.184+0000 7f459f7fe700 1 --2- 172.21.15.47:0/1404560963 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4580053aa0 0x7f4580055f60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f459001a000 tx=0x7f4590014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:42.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.188+0000 7f459d7fa700 1 -- 172.21.15.47:0/1404560963 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f45940420a0 con 0x7f45a00b1260 2022-06-29T15:42:42.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.316+0000 7f45a687b700 1 -- 172.21.15.47:0/1404560963 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f45a00573d0 con 0x7f45a00b1260 2022-06-29T15:42:42.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.316+0000 7f459d7fa700 1 -- 172.21.15.47:0/1404560963 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f45940456c0 con 0x7f45a00b1260 2022-06-29T15:42:42.320 INFO:teuthology.orchestra.run.smithi047.stdout:60129542150 2022-06-29T15:42:42.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.320+0000 7f45a687b700 1 -- 172.21.15.47:0/1404560963 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4580053aa0 msgr2=0x7f4580055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:42.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.320+0000 7f45a687b700 1 --2- 172.21.15.47:0/1404560963 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4580053aa0 0x7f4580055f60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f459001a000 tx=0x7f4590014000).stop 2022-06-29T15:42:42.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.320+0000 7f45a687b700 1 -- 172.21.15.47:0/1404560963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45a00b1260 msgr2=0x7f45a00b4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:42.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.320+0000 7f45a687b700 1 --2- 172.21.15.47:0/1404560963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45a00b1260 0x7f45a00b4950 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f45a015af70 tx=0x7f4594007ae0).stop 2022-06-29T15:42:42.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.320+0000 7f45a687b700 1 -- 172.21.15.47:0/1404560963 shutdown_connections 2022-06-29T15:42:42.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.320+0000 7f45a687b700 1 --2- 172.21.15.47:0/1404560963 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f45a01435a0 0x7f45a013e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:42.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.320+0000 7f45a687b700 1 --2- 172.21.15.47:0/1404560963 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4580053aa0 0x7f4580055f60 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:42.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.320+0000 7f45a687b700 1 --2- 172.21.15.47:0/1404560963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45a00b2120 0x7f45a015a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:42.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.320+0000 7f45a687b700 1 --2- 172.21.15.47:0/1404560963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45a00b1260 0x7f45a00b4950 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:42.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.320+0000 7f45a687b700 1 -- 172.21.15.47:0/1404560963 >> 172.21.15.47:0/1404560963 conn(0x7f45a00626f0 msgr2=0x7f45a01407a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:42.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.320+0000 7f45a687b700 1 -- 172.21.15.47:0/1404560963 shutdown_connections 2022-06-29T15:42:42.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.320+0000 7f45a687b700 1 -- 172.21.15.47:0/1404560963 wait complete. 2022-06-29T15:42:42.333 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.6 2022-06-29T15:42:42.333 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2022-06-29T15:42:42.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.436+0000 7f2e8d893700 1 Processor -- start 2022-06-29T15:42:42.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.436+0000 7f2e8d893700 1 -- start start 2022-06-29T15:42:42.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.436+0000 7f2e8d893700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2e8805c8f0 0x7f2e8813cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:42.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.436+0000 7f2e8d893700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e88142f90 con 0x7f2e8805c510 2022-06-29T15:42:42.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.436+0000 7f2e8d893700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e881431f0 con 0x7f2e88143bd0 2022-06-29T15:42:42.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.436+0000 7f2e8d893700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e88143490 con 0x7f2e8805c8f0 2022-06-29T15:42:42.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.436+0000 7f2e8c891700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f2e8805c510 0x7f2e880b4100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59874/0 (socket says 172.21.15.47:59874) 2022-06-29T15:42:42.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.436+0000 7f2e8c891700 1 -- 172.21.15.47:0/1276821242 learned_addr learned my addr 172.21.15.47:0/1276821242 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:42.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.436+0000 7f2e8d092700 1 --2- 172.21.15.47:0/1276821242 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2e8805c8f0 0x7f2e8813cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:42.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.436+0000 7f2e7f7fe700 1 -- 172.21.15.47:0/1276821242 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3231468325 0 0) 0x7f2e88142f90 con 0x7f2e8805c510 2022-06-29T15:42:42.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.436+0000 7f2e7f7fe700 1 -- 172.21.15.47:0/1276821242 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e74003260 con 0x7f2e8805c510 2022-06-29T15:42:42.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.436+0000 7f2e7f7fe700 1 -- 172.21.15.47:0/1276821242 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2195211158 0 0) 0x7f2e881431f0 con 0x7f2e88143bd0 2022-06-29T15:42:42.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.436+0000 7f2e7f7fe700 1 -- 172.21.15.47:0/1276821242 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e88142f90 con 0x7f2e88143bd0 2022-06-29T15:42:42.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.436+0000 7f2e8d092700 1 -- 172.21.15.47:0/1276821242 >> v1:172.21.15.64:6789/0 conn(0x7f2e88143bd0 legacy=0x7f2e880b2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:42.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.436+0000 7f2e8d092700 1 -- 172.21.15.47:0/1276821242 >> v1:172.21.15.47:6789/0 conn(0x7f2e8805c510 legacy=0x7f2e880b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:42.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.436+0000 7f2e8d092700 1 -- 172.21.15.47:0/1276821242 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e88143730 con 0x7f2e8805c8f0 2022-06-29T15:42:42.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.436+0000 7f2e8d092700 1 --2- 172.21.15.47:0/1276821242 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2e8805c8f0 0x7f2e8813cbf0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f2e84001390 tx=0x7f2e800091d0).ready entity=mon.2 client_cookie=b8f330ff6861b342 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:42.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.440+0000 7f2e7f7fe700 1 -- 172.21.15.47:0/1276821242 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e8000f510 con 0x7f2e8805c8f0 2022-06-29T15:42:42.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.440+0000 7f2e7f7fe700 1 -- 172.21.15.47:0/1276821242 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e8000f670 con 0x7f2e8805c8f0 2022-06-29T15:42:42.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.440+0000 7f2e7f7fe700 1 -- 172.21.15.47:0/1276821242 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e8000f7e0 con 0x7f2e8805c8f0 2022-06-29T15:42:42.449 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.440+0000 7f2e8d893700 1 -- 172.21.15.47:0/1276821242 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2e8805c8f0 msgr2=0x7f2e8813cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:42.450 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.440+0000 7f2e8d893700 1 --2- 172.21.15.47:0/1276821242 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2e8805c8f0 0x7f2e8813cbf0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f2e84001390 tx=0x7f2e800091d0).stop 2022-06-29T15:42:42.450 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.440+0000 7f2e8d893700 1 -- 172.21.15.47:0/1276821242 shutdown_connections 2022-06-29T15:42:42.450 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.440+0000 7f2e8d893700 1 --2- 172.21.15.47:0/1276821242 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2e8805c8f0 0x7f2e8813cbf0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:42.450 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.440+0000 7f2e8d893700 1 -- 172.21.15.47:0/1276821242 >> 172.21.15.47:0/1276821242 conn(0x7f2e88060e00 msgr2=0x7f2e880611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:42.451 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.440+0000 7f2e8d893700 1 -- 172.21.15.47:0/1276821242 shutdown_connections 2022-06-29T15:42:42.451 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.440+0000 7f2e8d893700 1 -- 172.21.15.47:0/1276821242 wait complete. 2022-06-29T15:42:42.451 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.440+0000 7f2e8d893700 1 Processor -- start 2022-06-29T15:42:42.451 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.440+0000 7f2e8d893700 1 -- start start 2022-06-29T15:42:42.451 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.444+0000 7f2e8d893700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2e8805c510 0x7f2e881446d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:42.452 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.444+0000 7f2e8d893700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e8805c8f0 0x7f2e88145120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:42.452 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.444+0000 7f2e8d893700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e88143bd0 0x7f2e88145610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:42.452 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.444+0000 7f2e8d893700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e88146470 con 0x7f2e88143bd0 2022-06-29T15:42:42.452 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.444+0000 7f2e8d893700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e880ad670 con 0x7f2e8805c8f0 2022-06-29T15:42:42.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.444+0000 7f2e8d893700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e88144bc0 con 0x7f2e8805c510 2022-06-29T15:42:42.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.444+0000 7f2e8d092700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e88143bd0 0x7f2e88145610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:42.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.444+0000 7f2e8d092700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e88143bd0 0x7f2e88145610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46098/0 (socket says 172.21.15.47:46098) 2022-06-29T15:42:42.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.444+0000 7f2e8d092700 1 -- 172.21.15.47:0/887427348 learned_addr learned my addr 172.21.15.47:0/887427348 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:42.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.444+0000 7f2e7ffff700 1 --2- 172.21.15.47:0/887427348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e8805c8f0 0x7f2e88145120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:42.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.444+0000 7f2e8c891700 1 --2- 172.21.15.47:0/887427348 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2e8805c510 0x7f2e881446d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:42.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.444+0000 7f2e8d092700 1 -- 172.21.15.47:0/887427348 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2e8805c510 msgr2=0x7f2e881446d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:42.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.444+0000 7f2e8d092700 1 --2- 172.21.15.47:0/887427348 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2e8805c510 0x7f2e881446d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:42.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.444+0000 7f2e8d092700 1 -- 172.21.15.47:0/887427348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e8805c8f0 msgr2=0x7f2e88145120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:42.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.444+0000 7f2e8d092700 1 --2- 172.21.15.47:0/887427348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e8805c8f0 0x7f2e88145120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:42.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.444+0000 7f2e8d092700 1 -- 172.21.15.47:0/887427348 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e80009470 con 0x7f2e88143bd0 2022-06-29T15:42:42.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.444+0000 7f2e8d092700 1 --2- 172.21.15.47:0/887427348 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e88143bd0 0x7f2e88145610 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f2e88146260 tx=0x7f2e80016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:42.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.444+0000 7f2e7dffb700 1 -- 172.21.15.47:0/887427348 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e80011ea0 con 0x7f2e88143bd0 2022-06-29T15:42:42.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.444+0000 7f2e7dffb700 1 -- 172.21.15.47:0/887427348 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e800032d0 con 0x7f2e88143bd0 2022-06-29T15:42:42.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.444+0000 7f2e8d893700 1 -- 172.21.15.47:0/887427348 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e88141d50 con 0x7f2e88143bd0 2022-06-29T15:42:42.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.444+0000 7f2e7dffb700 1 -- 172.21.15.47:0/887427348 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e80021630 con 0x7f2e88143bd0 2022-06-29T15:42:42.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.444+0000 7f2e8d893700 1 -- 172.21.15.47:0/887427348 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e88152fa0 con 0x7f2e88143bd0 2022-06-29T15:42:42.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.444+0000 7f2e8d893700 1 -- 172.21.15.47:0/887427348 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e880573d0 con 0x7f2e88143bd0 2022-06-29T15:42:42.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.448+0000 7f2e7dffb700 1 -- 172.21.15.47:0/887427348 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2e80027020 con 0x7f2e88143bd0 2022-06-29T15:42:42.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.448+0000 7f2e7dffb700 1 --2- 172.21.15.47:0/887427348 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2e68053b00 0x7f2e68055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:42.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.448+0000 7f2e7dffb700 1 -- 172.21.15.47:0/887427348 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f2e8006d460 con 0x7f2e88143bd0 2022-06-29T15:42:42.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.448+0000 7f2e8c891700 1 --2- 172.21.15.47:0/887427348 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2e68053b00 0x7f2e68055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:42.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.448+0000 7f2e8c891700 1 --2- 172.21.15.47:0/887427348 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2e68053b00 0x7f2e68055fc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f2e8400b000 tx=0x7f2e84007160).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:42.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.452+0000 7f2e7dffb700 1 -- 172.21.15.47:0/887427348 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2e80044220 con 0x7f2e88143bd0 2022-06-29T15:42:42.582 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.580+0000 7f2e8d893700 1 -- 172.21.15.47:0/887427348 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f2e88061b00 con 0x7f2e88143bd0 2022-06-29T15:42:42.583 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.580+0000 7f2e7dffb700 1 -- 172.21.15.47:0/887427348 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2e80015090 con 0x7f2e88143bd0 2022-06-29T15:42:42.583 INFO:teuthology.orchestra.run.smithi047.stdout:60129542149 2022-06-29T15:42:42.586 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.584+0000 7f2e8d893700 1 -- 172.21.15.47:0/887427348 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2e68053b00 msgr2=0x7f2e68055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:42.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.584+0000 7f2e8d893700 1 --2- 172.21.15.47:0/887427348 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2e68053b00 0x7f2e68055fc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f2e8400b000 tx=0x7f2e84007160).stop 2022-06-29T15:42:42.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.584+0000 7f2e8d893700 1 -- 172.21.15.47:0/887427348 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e88143bd0 msgr2=0x7f2e88145610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:42.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.584+0000 7f2e8d893700 1 --2- 172.21.15.47:0/887427348 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e88143bd0 0x7f2e88145610 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f2e88146260 tx=0x7f2e80016000).stop 2022-06-29T15:42:42.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.584+0000 7f2e8d893700 1 -- 172.21.15.47:0/887427348 shutdown_connections 2022-06-29T15:42:42.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.584+0000 7f2e8d893700 1 --2- 172.21.15.47:0/887427348 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e88143bd0 0x7f2e88145610 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:42.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.584+0000 7f2e8d893700 1 --2- 172.21.15.47:0/887427348 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2e68053b00 0x7f2e68055fc0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:42.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.584+0000 7f2e8d893700 1 --2- 172.21.15.47:0/887427348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e8805c8f0 0x7f2e88145120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:42.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.584+0000 7f2e8d893700 1 --2- 172.21.15.47:0/887427348 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2e8805c510 0x7f2e881446d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:42.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.584+0000 7f2e8d893700 1 -- 172.21.15.47:0/887427348 >> 172.21.15.47:0/887427348 conn(0x7f2e88060e00 msgr2=0x7f2e880b8110 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:42.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.584+0000 7f2e8d893700 1 -- 172.21.15.47:0/887427348 shutdown_connections 2022-06-29T15:42:42.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.584+0000 7f2e8d893700 1 -- 172.21.15.47:0/887427348 wait complete. 2022-06-29T15:42:42.602 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.7 2022-06-29T15:42:42.602 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2022-06-29T15:42:42.603 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2022-06-29T15:42:42.709 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.704+0000 7f13741a2700 1 Processor -- start 2022-06-29T15:42:42.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.704+0000 7f13741a2700 1 -- start start 2022-06-29T15:42:42.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.704+0000 7f13741a2700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f136c0619b0 0x7f136c0afc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:42.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.704+0000 7f13741a2700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f136c0b1dd0 con 0x7f136c05fac0 2022-06-29T15:42:42.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.704+0000 7f13741a2700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f136c061d90 con 0x7f136c0b18c0 2022-06-29T15:42:42.712 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.704+0000 7f13741a2700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f136c1419c0 con 0x7f136c0619b0 2022-06-29T15:42:42.712 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.704+0000 7f1371f3e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f136c0619b0 0x7f136c0afc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:42.713 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.704+0000 7f1371f3e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f136c0619b0 0x7f136c0afc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46102/0 (socket says 172.21.15.47:46102) 2022-06-29T15:42:42.713 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.704+0000 7f1371f3e700 1 -- 172.21.15.47:0/1589998224 learned_addr learned my addr 172.21.15.47:0/1589998224 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:42.714 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.704+0000 7f1371f3e700 1 -- 172.21.15.47:0/1589998224 >> v1:172.21.15.64:6789/0 conn(0x7f136c0b18c0 legacy=0x7f136c0b0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:42.714 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.704+0000 7f1371f3e700 1 -- 172.21.15.47:0/1589998224 >> v1:172.21.15.47:6789/0 conn(0x7f136c05fac0 legacy=0x7f136c147760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:42.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.704+0000 7f1371f3e700 1 -- 172.21.15.47:0/1589998224 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f136c0b2000 con 0x7f136c0619b0 2022-06-29T15:42:42.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.704+0000 7f1371f3e700 1 --2- 172.21.15.47:0/1589998224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f136c0619b0 0x7f136c0afc90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f136c0b0e20 tx=0x7f1360006eb0).ready entity=mon.0 client_cookie=a83a7a618d89fe6f server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:42.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.704+0000 7f1370f3c700 1 -- 172.21.15.47:0/1589998224 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f136000d610 con 0x7f136c0619b0 2022-06-29T15:42:42.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.704+0000 7f1370f3c700 1 -- 172.21.15.47:0/1589998224 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1360000e90 con 0x7f136c0619b0 2022-06-29T15:42:42.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.704+0000 7f1370f3c700 1 -- 172.21.15.47:0/1589998224 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f136000d960 con 0x7f136c0619b0 2022-06-29T15:42:42.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.704+0000 7f13741a2700 1 -- 172.21.15.47:0/1589998224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f136c0619b0 msgr2=0x7f136c0afc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:42.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.704+0000 7f13741a2700 1 --2- 172.21.15.47:0/1589998224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f136c0619b0 0x7f136c0afc90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f136c0b0e20 tx=0x7f1360006eb0).stop 2022-06-29T15:42:42.718 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.704+0000 7f13741a2700 1 -- 172.21.15.47:0/1589998224 shutdown_connections 2022-06-29T15:42:42.718 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.704+0000 7f13741a2700 1 --2- 172.21.15.47:0/1589998224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f136c0619b0 0x7f136c0afc90 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:42.719 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.704+0000 7f13741a2700 1 -- 172.21.15.47:0/1589998224 >> 172.21.15.47:0/1589998224 conn(0x7f136c05c510 msgr2=0x7f136c063800 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:42.719 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.704+0000 7f13741a2700 1 -- 172.21.15.47:0/1589998224 shutdown_connections 2022-06-29T15:42:42.719 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.704+0000 7f13741a2700 1 -- 172.21.15.47:0/1589998224 wait complete. 2022-06-29T15:42:42.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.708+0000 7f13741a2700 1 Processor -- start 2022-06-29T15:42:42.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.708+0000 7f13741a2700 1 -- start start 2022-06-29T15:42:42.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.708+0000 7f13741a2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f136c05fac0 0x7f136c14a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:42.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.708+0000 7f13741a2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f136c0619b0 0x7f136c13fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:42.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.708+0000 7f13741a2700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f136c0b18c0 0x7f136c13fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:42.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.708+0000 7f13741a2700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136c0654e0 con 0x7f136c0619b0 2022-06-29T15:42:42.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.708+0000 7f13741a2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136c1432e0 con 0x7f136c05fac0 2022-06-29T15:42:42.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.708+0000 7f13741a2700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f136c1404e0 con 0x7f136c0b18c0 2022-06-29T15:42:42.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.708+0000 7f137173d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f136c0619b0 0x7f136c13fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:42.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.708+0000 7f137173d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f136c0619b0 0x7f136c13fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46108/0 (socket says 172.21.15.47:46108) 2022-06-29T15:42:42.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.708+0000 7f137173d700 1 -- 172.21.15.47:0/3343683594 learned_addr learned my addr 172.21.15.47:0/3343683594 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:42.731 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.708+0000 7f1371f3e700 1 --2- 172.21.15.47:0/3343683594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f136c05fac0 0x7f136c14a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:42.731 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.708+0000 7f137273f700 1 --2- 172.21.15.47:0/3343683594 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f136c0b18c0 0x7f136c13fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:42.732 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.708+0000 7f137173d700 1 -- 172.21.15.47:0/3343683594 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f136c0b18c0 msgr2=0x7f136c13fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:42.732 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.708+0000 7f137173d700 1 --2- 172.21.15.47:0/3343683594 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f136c0b18c0 0x7f136c13fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:42.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.708+0000 7f137173d700 1 -- 172.21.15.47:0/3343683594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f136c05fac0 msgr2=0x7f136c14a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:42.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.708+0000 7f137173d700 1 --2- 172.21.15.47:0/3343683594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f136c05fac0 0x7f136c14a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:42.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.708+0000 7f137173d700 1 -- 172.21.15.47:0/3343683594 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f136000b1e0 con 0x7f136c0619b0 2022-06-29T15:42:42.734 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.708+0000 7f137173d700 1 --2- 172.21.15.47:0/3343683594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f136c0619b0 0x7f136c13fb00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f136c140670 tx=0x7f13640091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:42.734 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.708+0000 7f135effd700 1 -- 172.21.15.47:0/3343683594 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f136400e470 con 0x7f136c0619b0 2022-06-29T15:42:42.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.708+0000 7f135effd700 1 -- 172.21.15.47:0/3343683594 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f136400e5d0 con 0x7f136c0619b0 2022-06-29T15:42:42.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.708+0000 7f13741a2700 1 -- 172.21.15.47:0/3343683594 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f136c065710 con 0x7f136c0619b0 2022-06-29T15:42:42.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.708+0000 7f135effd700 1 -- 172.21.15.47:0/3343683594 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f136400e930 con 0x7f136c0619b0 2022-06-29T15:42:42.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.712+0000 7f13741a2700 1 -- 172.21.15.47:0/3343683594 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f136c066490 con 0x7f136c0619b0 2022-06-29T15:42:42.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.712+0000 7f135effd700 1 -- 172.21.15.47:0/3343683594 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1364018030 con 0x7f136c0619b0 2022-06-29T15:42:42.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.712+0000 7f13741a2700 1 -- 172.21.15.47:0/3343683594 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f136c0672d0 con 0x7f136c0619b0 2022-06-29T15:42:42.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.712+0000 7f135effd700 1 --2- 172.21.15.47:0/3343683594 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1354053a70 0x7f1354055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:42.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.712+0000 7f135effd700 1 -- 172.21.15.47:0/3343683594 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f136406b940 con 0x7f136c0619b0 2022-06-29T15:42:42.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.712+0000 7f1371f3e700 1 --2- 172.21.15.47:0/3343683594 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1354053a70 0x7f1354055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:42.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.712+0000 7f1371f3e700 1 --2- 172.21.15.47:0/3343683594 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1354053a70 0x7f1354055f30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f136001c000 tx=0x7f1360015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:42.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.716+0000 7f135effd700 1 -- 172.21.15.47:0/3343683594 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1364042700 con 0x7f136c0619b0 2022-06-29T15:42:42.843 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.840+0000 7f13741a2700 1 -- 172.21.15.47:0/3343683594 --> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f136c0573d0 con 0x7f1354053a70 2022-06-29T15:42:42.851 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.848+0000 7f135effd700 1 -- 172.21.15.47:0/3343683594 <== mgr.4102 v2:172.21.15.47:6800/12928 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+82257 (secure 0 0 0) 0x7f136c0573d0 con 0x7f1354053a70 2022-06-29T15:42:42.853 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:42:42.856 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.852+0000 7f13741a2700 1 -- 172.21.15.47:0/3343683594 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1354053a70 msgr2=0x7f1354055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:42.856 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.852+0000 7f13741a2700 1 --2- 172.21.15.47:0/3343683594 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1354053a70 0x7f1354055f30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f136001c000 tx=0x7f1360015000).stop 2022-06-29T15:42:42.857 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.852+0000 7f13741a2700 1 -- 172.21.15.47:0/3343683594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f136c0619b0 msgr2=0x7f136c13fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:42.857 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.852+0000 7f13741a2700 1 --2- 172.21.15.47:0/3343683594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f136c0619b0 0x7f136c13fb00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f136c140670 tx=0x7f13640091a0).stop 2022-06-29T15:42:42.858 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.852+0000 7f13741a2700 1 -- 172.21.15.47:0/3343683594 shutdown_connections 2022-06-29T15:42:42.858 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.852+0000 7f13741a2700 1 --2- 172.21.15.47:0/3343683594 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f136c0b18c0 0x7f136c13fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:42.859 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.852+0000 7f13741a2700 1 --2- 172.21.15.47:0/3343683594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f136c0619b0 0x7f136c13fb00 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:42.859 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.852+0000 7f13741a2700 1 --2- 172.21.15.47:0/3343683594 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1354053a70 0x7f1354055f30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:42.860 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.852+0000 7f13741a2700 1 --2- 172.21.15.47:0/3343683594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f136c05fac0 0x7f136c14a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:42.860 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.852+0000 7f13741a2700 1 -- 172.21.15.47:0/3343683594 >> 172.21.15.47:0/3343683594 conn(0x7f136c05c510 msgr2=0x7f136c0b4170 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:42.861 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.852+0000 7f13741a2700 1 -- 172.21.15.47:0/3343683594 shutdown_connections 2022-06-29T15:42:42.861 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.852+0000 7f13741a2700 1 -- 172.21.15.47:0/3343683594 wait complete. 2022-06-29T15:42:42.862 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2022-06-29T15:42:42.867 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2022-06-29T15:42:40.919394+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2304,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8390880,"kb_used_data":2208,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746583840,"statfs":{"total":773094113280,"available":764501852160,"internally_reserved":8589934592,"allocated":2260992,"data_stored":884800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.774875"},"pg_stats":[{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-29T15:42:33.506940+0000","last_change":"2022-06-29T15:42:33.505292+0000","last_active":"2022-06-29T15:42:33.506940+0000","last_peered":"2022-06-29T15:42:33.506940+0000","last_clean":"2022-06-29T15:42:33.506940+0000","last_became_active":"2022-06-29T15:42:33.504664+0000","last_became_peered":"2022-06-29T15:42:33.504664+0000","last_unstale":"2022-06-29T15:42:33.506940+0000","last_undegraded":"2022-06-29T15:42:33.506940+0000","last_fullsized":"2022-06-29T15:42:33.506940+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:32.123794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:32.123794+0000","last_clean_scrub_stamp":"2022-06-29T15:42:32.123794+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-29T15:42:33.502968+0000","last_change":"2022-06-29T15:42:22.794564+0000","last_active":"2022-06-29T15:42:33.502968+0000","last_peered":"2022-06-29T15:42:33.502968+0000","last_clean":"2022-06-29T15:42:33.502968+0000","last_became_active":"2022-06-29T15:42:22.793734+0000","last_became_peered":"2022-06-29T15:42:22.793734+0000","last_unstale":"2022-06-29T15:42:33.502968+0000","last_undegraded":"2022-06-29T15:42:33.502968+0000","last_fullsized":"2022-06-29T15:42:33.502968+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:17.908903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:17.908903+0000","last_clean_scrub_stamp":"2022-06-29T15:42:17.908903+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-29T15:42:33.491511+0000","last_change":"2022-06-29T15:42:27.002337+0000","last_active":"2022-06-29T15:42:33.491511+0000","last_peered":"2022-06-29T15:42:33.491511+0000","last_clean":"2022-06-29T15:42:33.491511+0000","last_became_active":"2022-06-29T15:42:27.001981+0000","last_became_peered":"2022-06-29T15:42:27.001981+0000","last_unstale":"2022-06-29T15:42:33.491511+0000","last_undegraded":"2022-06-29T15:42:33.491511+0000","last_fullsized":"2022-06-29T15:42:33.491511+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_clean_scrub_stamp":"2022-06-29T15:42:25.967805+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"25'1","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-29T15:42:33.492263+0000","last_change":"2022-06-29T15:42:30.129160+0000","last_active":"2022-06-29T15:42:33.492263+0000","last_peered":"2022-06-29T15:42:33.492263+0000","last_clean":"2022-06-29T15:42:33.492263+0000","last_became_active":"2022-06-29T15:42:30.128708+0000","last_became_peered":"2022-06-29T15:42:30.128708+0000","last_unstale":"2022-06-29T15:42:33.492263+0000","last_undegraded":"2022-06-29T15:42:33.492263+0000","last_fullsized":"2022-06-29T15:42:33.492263+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:29.103452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:29.103452+0000","last_clean_scrub_stamp":"2022-06-29T15:42:29.103452+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-29T15:42:33.504402+0000","last_change":"2022-06-29T15:42:33.503473+0000","last_active":"2022-06-29T15:42:33.504402+0000","last_peered":"2022-06-29T15:42:33.504402+0000","last_clean":"2022-06-29T15:42:33.504402+0000","last_became_active":"2022-06-29T15:42:33.502880+0000","last_became_peered":"2022-06-29T15:42:33.502880+0000","last_unstale":"2022-06-29T15:42:33.504402+0000","last_undegraded":"2022-06-29T15:42:33.504402+0000","last_fullsized":"2022-06-29T15:42:33.504402+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:32.123794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:32.123794+0000","last_clean_scrub_stamp":"2022-06-29T15:42:32.123794+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-29T15:42:33.502872+0000","last_change":"2022-06-29T15:42:27.000351+0000","last_active":"2022-06-29T15:42:33.502872+0000","last_peered":"2022-06-29T15:42:33.502872+0000","last_clean":"2022-06-29T15:42:33.502872+0000","last_became_active":"2022-06-29T15:42:26.999820+0000","last_became_peered":"2022-06-29T15:42:26.999820+0000","last_unstale":"2022-06-29T15:42:33.502872+0000","last_undegraded":"2022-06-29T15:42:33.502872+0000","last_fullsized":"2022-06-29T15:42:33.502872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_clean_scrub_stamp":"2022-06-29T15:42:25.967805+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-29T15:42:33.482558+0000","last_change":"2022-06-29T15:42:30.128532+0000","last_active":"2022-06-29T15:42:33.482558+0000","last_peered":"2022-06-29T15:42:33.482558+0000","last_clean":"2022-06-29T15:42:33.482558+0000","last_became_active":"2022-06-29T15:42:30.128123+0000","last_became_peered":"2022-06-29T15:42:30.128123+0000","last_unstale":"2022-06-29T15:42:33.482558+0000","last_undegraded":"2022-06-29T15:42:33.482558+0000","last_fullsized":"2022-06-29T15:42:33.482558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:29.103452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:29.103452+0000","last_clean_scrub_stamp":"2022-06-29T15:42:29.103452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-29T15:42:33.483765+0000","last_change":"2022-06-29T15:42:33.158808+0000","last_active":"2022-06-29T15:42:33.483765+0000","last_peered":"2022-06-29T15:42:33.483765+0000","last_clean":"2022-06-29T15:42:33.483765+0000","last_became_active":"2022-06-29T15:42:33.158318+0000","last_became_peered":"2022-06-29T15:42:33.158318+0000","last_unstale":"2022-06-29T15:42:33.483765+0000","last_undegraded":"2022-06-29T15:42:33.483765+0000","last_fullsized":"2022-06-29T15:42:33.483765+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:32.123794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:32.123794+0000","last_clean_scrub_stamp":"2022-06-29T15:42:32.123794+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-29T15:42:33.479127+0000","last_change":"2022-06-29T15:42:27.554899+0000","last_active":"2022-06-29T15:42:33.479127+0000","last_peered":"2022-06-29T15:42:33.479127+0000","last_clean":"2022-06-29T15:42:33.479127+0000","last_became_active":"2022-06-29T15:42:27.554328+0000","last_became_peered":"2022-06-29T15:42:27.554328+0000","last_unstale":"2022-06-29T15:42:33.479127+0000","last_undegraded":"2022-06-29T15:42:33.479127+0000","last_fullsized":"2022-06-29T15:42:33.479127+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_clean_scrub_stamp":"2022-06-29T15:42:25.967805+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"25'8","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-29T15:42:33.497662+0000","last_change":"2022-06-29T15:42:30.509276+0000","last_active":"2022-06-29T15:42:33.497662+0000","last_peered":"2022-06-29T15:42:33.497662+0000","last_clean":"2022-06-29T15:42:33.497662+0000","last_became_active":"2022-06-29T15:42:30.508404+0000","last_became_peered":"2022-06-29T15:42:30.508404+0000","last_unstale":"2022-06-29T15:42:33.497662+0000","last_undegraded":"2022-06-29T15:42:33.497662+0000","last_fullsized":"2022-06-29T15:42:33.497662+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:29.103452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:29.103452+0000","last_clean_scrub_stamp":"2022-06-29T15:42:29.103452+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-29T15:42:33.488520+0000","last_change":"2022-06-29T15:42:33.484659+0000","last_active":"2022-06-29T15:42:33.488520+0000","last_peered":"2022-06-29T15:42:33.488520+0000","last_clean":"2022-06-29T15:42:33.488520+0000","last_became_active":"2022-06-29T15:42:33.483803+0000","last_became_peered":"2022-06-29T15:42:33.483803+0000","last_unstale":"2022-06-29T15:42:33.488520+0000","last_undegraded":"2022-06-29T15:42:33.488520+0000","last_fullsized":"2022-06-29T15:42:33.488520+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:32.123794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:32.123794+0000","last_clean_scrub_stamp":"2022-06-29T15:42:32.123794+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-29T15:42:33.491553+0000","last_change":"2022-06-29T15:42:27.001536+0000","last_active":"2022-06-29T15:42:33.491553+0000","last_peered":"2022-06-29T15:42:33.491553+0000","last_clean":"2022-06-29T15:42:33.491553+0000","last_became_active":"2022-06-29T15:42:27.001089+0000","last_became_peered":"2022-06-29T15:42:27.001089+0000","last_unstale":"2022-06-29T15:42:33.491553+0000","last_undegraded":"2022-06-29T15:42:33.491553+0000","last_fullsized":"2022-06-29T15:42:33.491553+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_clean_scrub_stamp":"2022-06-29T15:42:25.967805+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"25'5","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-29T15:42:33.498703+0000","last_change":"2022-06-29T15:42:30.127732+0000","last_active":"2022-06-29T15:42:33.498703+0000","last_peered":"2022-06-29T15:42:33.498703+0000","last_clean":"2022-06-29T15:42:33.498703+0000","last_became_active":"2022-06-29T15:42:30.126907+0000","last_became_peered":"2022-06-29T15:42:30.126907+0000","last_unstale":"2022-06-29T15:42:33.498703+0000","last_undegraded":"2022-06-29T15:42:33.498703+0000","last_fullsized":"2022-06-29T15:42:33.498703+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:29.103452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:29.103452+0000","last_clean_scrub_stamp":"2022-06-29T15:42:29.103452+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-29T15:42:33.483560+0000","last_change":"2022-06-29T15:42:33.159995+0000","last_active":"2022-06-29T15:42:33.483560+0000","last_peered":"2022-06-29T15:42:33.483560+0000","last_clean":"2022-06-29T15:42:33.483560+0000","last_became_active":"2022-06-29T15:42:33.159461+0000","last_became_peered":"2022-06-29T15:42:33.159461+0000","last_unstale":"2022-06-29T15:42:33.483560+0000","last_undegraded":"2022-06-29T15:42:33.483560+0000","last_fullsized":"2022-06-29T15:42:33.483560+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:32.123794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:32.123794+0000","last_clean_scrub_stamp":"2022-06-29T15:42:32.123794+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"25'6","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-29T15:42:33.501329+0000","last_change":"2022-06-29T15:42:30.130523+0000","last_active":"2022-06-29T15:42:33.501329+0000","last_peered":"2022-06-29T15:42:33.501329+0000","last_clean":"2022-06-29T15:42:33.501329+0000","last_became_active":"2022-06-29T15:42:30.130117+0000","last_became_peered":"2022-06-29T15:42:30.130117+0000","last_unstale":"2022-06-29T15:42:33.501329+0000","last_undegraded":"2022-06-29T15:42:33.501329+0000","last_fullsized":"2022-06-29T15:42:33.501329+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:29.103452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:29.103452+0000","last_clean_scrub_stamp":"2022-06-29T15:42:29.103452+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-29T15:42:33.484454+0000","last_change":"2022-06-29T15:42:27.721658+0000","last_active":"2022-06-29T15:42:33.484454+0000","last_peered":"2022-06-29T15:42:33.484454+0000","last_clean":"2022-06-29T15:42:33.484454+0000","last_became_active":"2022-06-29T15:42:27.720770+0000","last_became_peered":"2022-06-29T15:42:27.720770+0000","last_unstale":"2022-06-29T15:42:33.484454+0000","last_undegraded":"2022-06-29T15:42:33.484454+0000","last_fullsized":"2022-06-29T15:42:33.484454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_clean_scrub_stamp":"2022-06-29T15:42:25.967805+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-29T15:42:33.504744+0000","last_change":"2022-06-29T15:42:33.503014+0000","last_active":"2022-06-29T15:42:33.504744+0000","last_peered":"2022-06-29T15:42:33.504744+0000","last_clean":"2022-06-29T15:42:33.504744+0000","last_became_active":"2022-06-29T15:42:33.502453+0000","last_became_peered":"2022-06-29T15:42:33.502453+0000","last_unstale":"2022-06-29T15:42:33.504744+0000","last_undegraded":"2022-06-29T15:42:33.504744+0000","last_fullsized":"2022-06-29T15:42:33.504744+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:32.123794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:32.123794+0000","last_clean_scrub_stamp":"2022-06-29T15:42:32.123794+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"25'8","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-29T15:42:33.501858+0000","last_change":"2022-06-29T15:42:30.509950+0000","last_active":"2022-06-29T15:42:33.501858+0000","last_peered":"2022-06-29T15:42:33.501858+0000","last_clean":"2022-06-29T15:42:33.501858+0000","last_became_active":"2022-06-29T15:42:30.509152+0000","last_became_peered":"2022-06-29T15:42:30.509152+0000","last_unstale":"2022-06-29T15:42:33.501858+0000","last_undegraded":"2022-06-29T15:42:33.501858+0000","last_fullsized":"2022-06-29T15:42:33.501858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:29.103452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:29.103452+0000","last_clean_scrub_stamp":"2022-06-29T15:42:29.103452+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1535,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-29T15:42:33.502709+0000","last_change":"2022-06-29T15:42:27.723852+0000","last_active":"2022-06-29T15:42:33.502709+0000","last_peered":"2022-06-29T15:42:33.502709+0000","last_clean":"2022-06-29T15:42:33.502709+0000","last_became_active":"2022-06-29T15:42:27.722958+0000","last_became_peered":"2022-06-29T15:42:27.722958+0000","last_unstale":"2022-06-29T15:42:33.502709+0000","last_undegraded":"2022-06-29T15:42:33.502709+0000","last_fullsized":"2022-06-29T15:42:33.502709+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_clean_scrub_stamp":"2022-06-29T15:42:25.967805+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-29T15:42:33.504284+0000","last_change":"2022-06-29T15:42:33.503392+0000","last_active":"2022-06-29T15:42:33.504284+0000","last_peered":"2022-06-29T15:42:33.504284+0000","last_clean":"2022-06-29T15:42:33.504284+0000","last_became_active":"2022-06-29T15:42:33.502807+0000","last_became_peered":"2022-06-29T15:42:33.502807+0000","last_unstale":"2022-06-29T15:42:33.504284+0000","last_undegraded":"2022-06-29T15:42:33.504284+0000","last_fullsized":"2022-06-29T15:42:33.504284+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:32.123794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:32.123794+0000","last_clean_scrub_stamp":"2022-06-29T15:42:32.123794+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"25'7","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-29T15:42:33.499523+0000","last_change":"2022-06-29T15:42:30.709758+0000","last_active":"2022-06-29T15:42:33.499523+0000","last_peered":"2022-06-29T15:42:33.499523+0000","last_clean":"2022-06-29T15:42:33.499523+0000","last_became_active":"2022-06-29T15:42:30.708949+0000","last_became_peered":"2022-06-29T15:42:30.708949+0000","last_unstale":"2022-06-29T15:42:33.499523+0000","last_undegraded":"2022-06-29T15:42:33.499523+0000","last_fullsized":"2022-06-29T15:42:33.499523+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:29.103452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:29.103452+0000","last_clean_scrub_stamp":"2022-06-29T15:42:29.103452+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":575,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-29T15:42:33.483851+0000","last_change":"2022-06-29T15:42:26.998189+0000","last_active":"2022-06-29T15:42:33.483851+0000","last_peered":"2022-06-29T15:42:33.483851+0000","last_clean":"2022-06-29T15:42:33.483851+0000","last_became_active":"2022-06-29T15:42:26.997677+0000","last_became_peered":"2022-06-29T15:42:26.997677+0000","last_unstale":"2022-06-29T15:42:33.483851+0000","last_undegraded":"2022-06-29T15:42:33.483851+0000","last_fullsized":"2022-06-29T15:42:33.483851+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_clean_scrub_stamp":"2022-06-29T15:42:25.967805+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-29T15:42:33.506729+0000","last_change":"2022-06-29T15:42:33.505382+0000","last_active":"2022-06-29T15:42:33.506729+0000","last_peered":"2022-06-29T15:42:33.506729+0000","last_clean":"2022-06-29T15:42:33.506729+0000","last_became_active":"2022-06-29T15:42:33.504714+0000","last_became_peered":"2022-06-29T15:42:33.504714+0000","last_unstale":"2022-06-29T15:42:33.506729+0000","last_undegraded":"2022-06-29T15:42:33.506729+0000","last_fullsized":"2022-06-29T15:42:33.506729+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:32.123794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:32.123794+0000","last_clean_scrub_stamp":"2022-06-29T15:42:32.123794+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"26'4","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2022-06-29T15:42:34.798365+0000","last_change":"2022-06-29T15:42:30.125770+0000","last_active":"2022-06-29T15:42:34.798365+0000","last_peered":"2022-06-29T15:42:34.798365+0000","last_clean":"2022-06-29T15:42:34.798365+0000","last_became_active":"2022-06-29T15:42:30.125364+0000","last_became_peered":"2022-06-29T15:42:30.125364+0000","last_unstale":"2022-06-29T15:42:34.798365+0000","last_undegraded":"2022-06-29T15:42:34.798365+0000","last_fullsized":"2022-06-29T15:42:34.798365+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:29.103452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:29.103452+0000","last_clean_scrub_stamp":"2022-06-29T15:42:29.103452+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-29T15:42:33.482363+0000","last_change":"2022-06-29T15:42:26.999647+0000","last_active":"2022-06-29T15:42:33.482363+0000","last_peered":"2022-06-29T15:42:33.482363+0000","last_clean":"2022-06-29T15:42:33.482363+0000","last_became_active":"2022-06-29T15:42:26.999119+0000","last_became_peered":"2022-06-29T15:42:26.999119+0000","last_unstale":"2022-06-29T15:42:33.482363+0000","last_undegraded":"2022-06-29T15:42:33.482363+0000","last_fullsized":"2022-06-29T15:42:33.482363+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_clean_scrub_stamp":"2022-06-29T15:42:25.967805+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":32,"acting":32,"num_store_stats":8},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":16,"acting":16,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048860,"kb_used_data":276,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322980,"statfs":{"total":96636764160,"available":95562731520,"internally_reserved":1073741824,"allocated":282624,"data_stored":110184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93600000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.052}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2769999999999999}]}]},{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048860,"kb_used_data":276,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322980,"statfs":{"total":96636764160,"available":95562731520,"internally_reserved":1073741824,"allocated":282624,"data_stored":110599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.167}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.012}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.198}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3180000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.446}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.506}]}]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048852,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322988,"statfs":{"total":96636764160,"available":95562739712,"internally_reserved":1073741824,"allocated":274432,"data_stored":110024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.137}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]}]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048856,"kb_used_data":272,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322984,"statfs":{"total":96636764160,"available":95562735616,"internally_reserved":1073741824,"allocated":278528,"data_stored":110058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0029999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92100000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.127}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6819999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.153}]}]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048868,"kb_used_data":284,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322972,"statfs":{"total":96636764160,"available":95562723328,"internally_reserved":1073741824,"allocated":290816,"data_stored":110759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0549999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.05}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]}]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048864,"kb_used_data":280,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322976,"statfs":{"total":96636764160,"available":95562727424,"internally_reserved":1073741824,"allocated":286720,"data_stored":111559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96899999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.052}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.125}]}]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048864,"kb_used_data":280,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322976,"statfs":{"total":96636764160,"available":95562727424,"internally_reserved":1073741824,"allocated":286720,"data_stored":111559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.087}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.194}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2749999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048856,"kb_used_data":272,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322984,"statfs":{"total":96636764160,"available":95562735616,"internally_reserved":1073741824,"allocated":278528,"data_stored":110058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0389999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.218}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]}]}]}} 2022-06-29T15:42:42.869 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2022-06-29T15:42:42.976 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.972+0000 7f3fd62f6700 1 Processor -- start 2022-06-29T15:42:42.977 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.972+0000 7f3fd62f6700 1 -- start start 2022-06-29T15:42:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.976+0000 7f3fd62f6700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3fd00b03b0 0x7f3fd0143120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.976+0000 7f3fd62f6700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3fd00b1e50 0x7f3fd0140220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.976+0000 7f3fd62f6700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fd00b26b0 con 0x7f3fd00ac8e0 2022-06-29T15:42:42.981 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.976+0000 7f3fd62f6700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fd00b0790 con 0x7f3fd00b03b0 2022-06-29T15:42:42.981 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.976+0000 7f3fd62f6700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd00acdf0 con 0x7f3fd00b1e50 2022-06-29T15:42:42.981 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.976+0000 7f3fd4af3700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f3fd00ac8e0 0x7f3fd013fb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59900/0 (socket says 172.21.15.47:59900) 2022-06-29T15:42:42.981 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.976+0000 7f3fd4af3700 1 -- 172.21.15.47:0/1475279945 learned_addr learned my addr 172.21.15.47:0/1475279945 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:42.982 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.976+0000 7f3fd5af5700 1 --2- 172.21.15.47:0/1475279945 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3fd00b1e50 0x7f3fd0140220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:42.982 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.976+0000 7f3fd52f4700 1 --2- 172.21.15.47:0/1475279945 >> v2:172.21.15.64:3300/0 conn(0x7f3fd00b03b0 0x7f3fd0143120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:42.982 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.976+0000 7f3fcffff700 1 -- 172.21.15.47:0/1475279945 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2289067544 0 0) 0x7f3fd00b26b0 con 0x7f3fd00ac8e0 2022-06-29T15:42:42.982 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.976+0000 7f3fcffff700 1 -- 172.21.15.47:0/1475279945 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fbc003260 con 0x7f3fd00ac8e0 2022-06-29T15:42:42.983 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.976+0000 7f3fd5af5700 1 -- 172.21.15.47:0/1475279945 >> v2:172.21.15.64:3300/0 conn(0x7f3fd00b03b0 msgr2=0x7f3fd0143120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:42.983 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.976+0000 7f3fd5af5700 1 --2- 172.21.15.47:0/1475279945 >> v2:172.21.15.64:3300/0 conn(0x7f3fd00b03b0 0x7f3fd0143120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:42.983 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.976+0000 7f3fd5af5700 1 -- 172.21.15.47:0/1475279945 >> v1:172.21.15.47:6789/0 conn(0x7f3fd00ac8e0 legacy=0x7f3fd013fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:42.983 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.976+0000 7f3fd5af5700 1 -- 172.21.15.47:0/1475279945 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd00ad090 con 0x7f3fd00b1e50 2022-06-29T15:42:42.984 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.976+0000 7f3fd5af5700 1 --2- 172.21.15.47:0/1475279945 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3fd00b1e50 0x7f3fd0140220 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f3fc80032f0 tx=0x7f3fc80091d0).ready entity=mon.2 client_cookie=c95a057cc4d7aa83 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:42.984 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.976+0000 7f3fcffff700 1 -- 172.21.15.47:0/1475279945 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3fc800b600 con 0x7f3fd00b1e50 2022-06-29T15:42:42.984 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.976+0000 7f3fcffff700 1 -- 172.21.15.47:0/1475279945 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fc800b760 con 0x7f3fd00b1e50 2022-06-29T15:42:42.984 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.976+0000 7f3fcffff700 1 -- 172.21.15.47:0/1475279945 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3fc800b8d0 con 0x7f3fd00b1e50 2022-06-29T15:42:42.985 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.976+0000 7f3fd62f6700 1 -- 172.21.15.47:0/1475279945 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3fd00b1e50 msgr2=0x7f3fd0140220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:42.985 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.976+0000 7f3fd62f6700 1 --2- 172.21.15.47:0/1475279945 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3fd00b1e50 0x7f3fd0140220 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f3fc80032f0 tx=0x7f3fc80091d0).stop 2022-06-29T15:42:42.985 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.976+0000 7f3fd62f6700 1 -- 172.21.15.47:0/1475279945 shutdown_connections 2022-06-29T15:42:42.986 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.976+0000 7f3fd62f6700 1 --2- 172.21.15.47:0/1475279945 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3fd00b1e50 0x7f3fd0140220 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:42.986 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.976+0000 7f3fd62f6700 1 --2- 172.21.15.47:0/1475279945 >> v2:172.21.15.64:3300/0 conn(0x7f3fd00b03b0 0x7f3fd0143120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:42.986 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.976+0000 7f3fd62f6700 1 -- 172.21.15.47:0/1475279945 >> 172.21.15.47:0/1475279945 conn(0x7f3fd0062560 msgr2=0x7f3fd0062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:42.986 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.976+0000 7f3fd62f6700 1 -- 172.21.15.47:0/1475279945 shutdown_connections 2022-06-29T15:42:42.987 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.976+0000 7f3fd62f6700 1 -- 172.21.15.47:0/1475279945 wait complete. 2022-06-29T15:42:42.990 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.980+0000 7f3fd62f6700 1 Processor -- start 2022-06-29T15:42:42.990 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.980+0000 7f3fd62f6700 1 -- start start 2022-06-29T15:42:42.990 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.980+0000 7f3fd62f6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3fd00ac8e0 0x7f3fd00a3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:42.991 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.980+0000 7f3fd62f6700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3fd00b03b0 0x7f3fd013c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:42.991 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.980+0000 7f3fd62f6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fd00b1e50 0x7f3fd013cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:42.991 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.980+0000 7f3fd62f6700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd013d090 con 0x7f3fd00ac8e0 2022-06-29T15:42:42.991 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.980+0000 7f3fd62f6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd0141d50 con 0x7f3fd00b1e50 2022-06-29T15:42:42.992 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.980+0000 7f3fd62f6700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd0141f80 con 0x7f3fd00b03b0 2022-06-29T15:42:42.992 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.980+0000 7f3fd52f4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3fd00ac8e0 0x7f3fd00a3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:42.992 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.980+0000 7f3fd4af3700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3fd00b03b0 0x7f3fd013c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:42.992 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.980+0000 7f3fd52f4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3fd00ac8e0 0x7f3fd00a3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46120/0 (socket says 172.21.15.47:46120) 2022-06-29T15:42:42.993 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.980+0000 7f3fd52f4700 1 -- 172.21.15.47:0/3561355220 learned_addr learned my addr 172.21.15.47:0/3561355220 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:42.993 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.980+0000 7f3fd5af5700 1 --2- 172.21.15.47:0/3561355220 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fd00b1e50 0x7f3fd013cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:42.993 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.980+0000 7f3fd4af3700 1 -- 172.21.15.47:0/3561355220 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fd00b1e50 msgr2=0x7f3fd013cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:42.993 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.980+0000 7f3fd4af3700 1 --2- 172.21.15.47:0/3561355220 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fd00b1e50 0x7f3fd013cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:42.994 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.980+0000 7f3fd4af3700 1 -- 172.21.15.47:0/3561355220 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3fd00ac8e0 msgr2=0x7f3fd00a3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:42.994 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.980+0000 7f3fd4af3700 1 --2- 172.21.15.47:0/3561355220 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3fd00ac8e0 0x7f3fd00a3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:42.995 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.980+0000 7f3fd4af3700 1 -- 172.21.15.47:0/3561355220 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc8009470 con 0x7f3fd00b03b0 2022-06-29T15:42:42.995 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.980+0000 7f3fd4af3700 1 --2- 172.21.15.47:0/3561355220 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3fd00b03b0 0x7f3fd013c6b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f3fc0001110 tx=0x7f3fc0007a20).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:42.996 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.980+0000 7f3fce7fc700 1 -- 172.21.15.47:0/3561355220 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3fc000ce80 con 0x7f3fd00b03b0 2022-06-29T15:42:42.996 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.980+0000 7f3fce7fc700 1 -- 172.21.15.47:0/3561355220 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fc00070d0 con 0x7f3fd00b03b0 2022-06-29T15:42:42.997 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.980+0000 7f3fd62f6700 1 -- 172.21.15.47:0/3561355220 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fd00a39b0 con 0x7f3fd00b03b0 2022-06-29T15:42:42.997 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.980+0000 7f3fce7fc700 1 -- 172.21.15.47:0/3561355220 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3fc0007430 con 0x7f3fd00b03b0 2022-06-29T15:42:42.997 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.980+0000 7f3fd62f6700 1 -- 172.21.15.47:0/3561355220 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fd01421b0 con 0x7f3fd00b03b0 2022-06-29T15:42:42.998 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.984+0000 7f3fd62f6700 1 -- 172.21.15.47:0/3561355220 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fd00a3bb0 con 0x7f3fd00b03b0 2022-06-29T15:42:42.998 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.984+0000 7f3fce7fc700 1 -- 172.21.15.47:0/3561355220 <== mon.2 v2:172.21.15.64:3301/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3fc001a060 con 0x7f3fd00b03b0 2022-06-29T15:42:42.999 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.984+0000 7f3fce7fc700 1 --2- 172.21.15.47:0/3561355220 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3fb0053a70 0x7f3fb0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:42.999 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.984+0000 7f3fce7fc700 1 -- 172.21.15.47:0/3561355220 <== mon.2 v2:172.21.15.64:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f3fc006c1a0 con 0x7f3fd00b03b0 2022-06-29T15:42:42.999 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.984+0000 7f3fd52f4700 1 --2- 172.21.15.47:0/3561355220 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3fb0053a70 0x7f3fb0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:43.000 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.984+0000 7f3fd52f4700 1 --2- 172.21.15.47:0/3561355220 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3fb0053a70 0x7f3fb0055f30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f3fc4006000 tx=0x7f3fc4006d90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:43.001 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:42.992+0000 7f3fce7fc700 1 -- 172.21.15.47:0/3561355220 <== mon.2 v2:172.21.15.64:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3fc0042ee0 con 0x7f3fd00b03b0 2022-06-29T15:42:43.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.116+0000 7f3fd62f6700 1 -- 172.21.15.47:0/3561355220 --> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3fd0141230 con 0x7f3fb0053a70 2022-06-29T15:42:43.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.124+0000 7f3fce7fc700 1 -- 172.21.15.47:0/3561355220 <== mgr.4102 v2:172.21.15.47:6800/12928 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+82387 (secure 0 0 0) 0x7f3fc4000e90 con 0x7f3fb0053a70 2022-06-29T15:42:43.130 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:42:43.133 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.128+0000 7f3fd62f6700 1 -- 172.21.15.47:0/3561355220 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3fb0053a70 msgr2=0x7f3fb0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:43.133 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.128+0000 7f3fd62f6700 1 --2- 172.21.15.47:0/3561355220 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3fb0053a70 0x7f3fb0055f30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f3fc4006000 tx=0x7f3fc4006d90).stop 2022-06-29T15:42:43.134 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.128+0000 7f3fd62f6700 1 -- 172.21.15.47:0/3561355220 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3fd00b03b0 msgr2=0x7f3fd013c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:43.134 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.128+0000 7f3fd62f6700 1 --2- 172.21.15.47:0/3561355220 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3fd00b03b0 0x7f3fd013c6b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f3fc0001110 tx=0x7f3fc0007a20).stop 2022-06-29T15:42:43.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.128+0000 7f3fd62f6700 1 -- 172.21.15.47:0/3561355220 shutdown_connections 2022-06-29T15:42:43.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.128+0000 7f3fd62f6700 1 --2- 172.21.15.47:0/3561355220 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3fb0053a70 0x7f3fb0055f30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:43.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.128+0000 7f3fd62f6700 1 --2- 172.21.15.47:0/3561355220 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fd00b1e50 0x7f3fd013cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:43.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.128+0000 7f3fd62f6700 1 --2- 172.21.15.47:0/3561355220 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3fd00b03b0 0x7f3fd013c6b0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:43.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.128+0000 7f3fd62f6700 1 --2- 172.21.15.47:0/3561355220 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3fd00ac8e0 0x7f3fd00a3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:43.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.128+0000 7f3fd62f6700 1 -- 172.21.15.47:0/3561355220 >> 172.21.15.47:0/3561355220 conn(0x7f3fd0062560 msgr2=0x7f3fd01430b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:43.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.128+0000 7f3fd62f6700 1 -- 172.21.15.47:0/3561355220 shutdown_connections 2022-06-29T15:42:43.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.128+0000 7f3fd62f6700 1 -- 172.21.15.47:0/3561355220 wait complete. 2022-06-29T15:42:43.138 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2022-06-29T15:42:43.143 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2022-06-29T15:42:42.920037+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2304,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8390880,"kb_used_data":2208,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746583840,"statfs":{"total":773094113280,"available":764501852160,"internally_reserved":8589934592,"allocated":2260992,"data_stored":884800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.444199"},"pg_stats":[{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-29T15:42:33.506940+0000","last_change":"2022-06-29T15:42:33.505292+0000","last_active":"2022-06-29T15:42:33.506940+0000","last_peered":"2022-06-29T15:42:33.506940+0000","last_clean":"2022-06-29T15:42:33.506940+0000","last_became_active":"2022-06-29T15:42:33.504664+0000","last_became_peered":"2022-06-29T15:42:33.504664+0000","last_unstale":"2022-06-29T15:42:33.506940+0000","last_undegraded":"2022-06-29T15:42:33.506940+0000","last_fullsized":"2022-06-29T15:42:33.506940+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:32.123794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:32.123794+0000","last_clean_scrub_stamp":"2022-06-29T15:42:32.123794+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-29T15:42:33.502968+0000","last_change":"2022-06-29T15:42:22.794564+0000","last_active":"2022-06-29T15:42:33.502968+0000","last_peered":"2022-06-29T15:42:33.502968+0000","last_clean":"2022-06-29T15:42:33.502968+0000","last_became_active":"2022-06-29T15:42:22.793734+0000","last_became_peered":"2022-06-29T15:42:22.793734+0000","last_unstale":"2022-06-29T15:42:33.502968+0000","last_undegraded":"2022-06-29T15:42:33.502968+0000","last_fullsized":"2022-06-29T15:42:33.502968+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:17.908903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:17.908903+0000","last_clean_scrub_stamp":"2022-06-29T15:42:17.908903+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-29T15:42:33.491511+0000","last_change":"2022-06-29T15:42:27.002337+0000","last_active":"2022-06-29T15:42:33.491511+0000","last_peered":"2022-06-29T15:42:33.491511+0000","last_clean":"2022-06-29T15:42:33.491511+0000","last_became_active":"2022-06-29T15:42:27.001981+0000","last_became_peered":"2022-06-29T15:42:27.001981+0000","last_unstale":"2022-06-29T15:42:33.491511+0000","last_undegraded":"2022-06-29T15:42:33.491511+0000","last_fullsized":"2022-06-29T15:42:33.491511+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_clean_scrub_stamp":"2022-06-29T15:42:25.967805+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"25'1","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-29T15:42:33.492263+0000","last_change":"2022-06-29T15:42:30.129160+0000","last_active":"2022-06-29T15:42:33.492263+0000","last_peered":"2022-06-29T15:42:33.492263+0000","last_clean":"2022-06-29T15:42:33.492263+0000","last_became_active":"2022-06-29T15:42:30.128708+0000","last_became_peered":"2022-06-29T15:42:30.128708+0000","last_unstale":"2022-06-29T15:42:33.492263+0000","last_undegraded":"2022-06-29T15:42:33.492263+0000","last_fullsized":"2022-06-29T15:42:33.492263+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:29.103452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:29.103452+0000","last_clean_scrub_stamp":"2022-06-29T15:42:29.103452+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-29T15:42:33.504402+0000","last_change":"2022-06-29T15:42:33.503473+0000","last_active":"2022-06-29T15:42:33.504402+0000","last_peered":"2022-06-29T15:42:33.504402+0000","last_clean":"2022-06-29T15:42:33.504402+0000","last_became_active":"2022-06-29T15:42:33.502880+0000","last_became_peered":"2022-06-29T15:42:33.502880+0000","last_unstale":"2022-06-29T15:42:33.504402+0000","last_undegraded":"2022-06-29T15:42:33.504402+0000","last_fullsized":"2022-06-29T15:42:33.504402+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:32.123794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:32.123794+0000","last_clean_scrub_stamp":"2022-06-29T15:42:32.123794+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-29T15:42:33.502872+0000","last_change":"2022-06-29T15:42:27.000351+0000","last_active":"2022-06-29T15:42:33.502872+0000","last_peered":"2022-06-29T15:42:33.502872+0000","last_clean":"2022-06-29T15:42:33.502872+0000","last_became_active":"2022-06-29T15:42:26.999820+0000","last_became_peered":"2022-06-29T15:42:26.999820+0000","last_unstale":"2022-06-29T15:42:33.502872+0000","last_undegraded":"2022-06-29T15:42:33.502872+0000","last_fullsized":"2022-06-29T15:42:33.502872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_clean_scrub_stamp":"2022-06-29T15:42:25.967805+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-29T15:42:33.482558+0000","last_change":"2022-06-29T15:42:30.128532+0000","last_active":"2022-06-29T15:42:33.482558+0000","last_peered":"2022-06-29T15:42:33.482558+0000","last_clean":"2022-06-29T15:42:33.482558+0000","last_became_active":"2022-06-29T15:42:30.128123+0000","last_became_peered":"2022-06-29T15:42:30.128123+0000","last_unstale":"2022-06-29T15:42:33.482558+0000","last_undegraded":"2022-06-29T15:42:33.482558+0000","last_fullsized":"2022-06-29T15:42:33.482558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:29.103452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:29.103452+0000","last_clean_scrub_stamp":"2022-06-29T15:42:29.103452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-29T15:42:33.483765+0000","last_change":"2022-06-29T15:42:33.158808+0000","last_active":"2022-06-29T15:42:33.483765+0000","last_peered":"2022-06-29T15:42:33.483765+0000","last_clean":"2022-06-29T15:42:33.483765+0000","last_became_active":"2022-06-29T15:42:33.158318+0000","last_became_peered":"2022-06-29T15:42:33.158318+0000","last_unstale":"2022-06-29T15:42:33.483765+0000","last_undegraded":"2022-06-29T15:42:33.483765+0000","last_fullsized":"2022-06-29T15:42:33.483765+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:32.123794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:32.123794+0000","last_clean_scrub_stamp":"2022-06-29T15:42:32.123794+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-29T15:42:33.479127+0000","last_change":"2022-06-29T15:42:27.554899+0000","last_active":"2022-06-29T15:42:33.479127+0000","last_peered":"2022-06-29T15:42:33.479127+0000","last_clean":"2022-06-29T15:42:33.479127+0000","last_became_active":"2022-06-29T15:42:27.554328+0000","last_became_peered":"2022-06-29T15:42:27.554328+0000","last_unstale":"2022-06-29T15:42:33.479127+0000","last_undegraded":"2022-06-29T15:42:33.479127+0000","last_fullsized":"2022-06-29T15:42:33.479127+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_clean_scrub_stamp":"2022-06-29T15:42:25.967805+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"25'8","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-29T15:42:33.497662+0000","last_change":"2022-06-29T15:42:30.509276+0000","last_active":"2022-06-29T15:42:33.497662+0000","last_peered":"2022-06-29T15:42:33.497662+0000","last_clean":"2022-06-29T15:42:33.497662+0000","last_became_active":"2022-06-29T15:42:30.508404+0000","last_became_peered":"2022-06-29T15:42:30.508404+0000","last_unstale":"2022-06-29T15:42:33.497662+0000","last_undegraded":"2022-06-29T15:42:33.497662+0000","last_fullsized":"2022-06-29T15:42:33.497662+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:29.103452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:29.103452+0000","last_clean_scrub_stamp":"2022-06-29T15:42:29.103452+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-29T15:42:33.488520+0000","last_change":"2022-06-29T15:42:33.484659+0000","last_active":"2022-06-29T15:42:33.488520+0000","last_peered":"2022-06-29T15:42:33.488520+0000","last_clean":"2022-06-29T15:42:33.488520+0000","last_became_active":"2022-06-29T15:42:33.483803+0000","last_became_peered":"2022-06-29T15:42:33.483803+0000","last_unstale":"2022-06-29T15:42:33.488520+0000","last_undegraded":"2022-06-29T15:42:33.488520+0000","last_fullsized":"2022-06-29T15:42:33.488520+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:32.123794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:32.123794+0000","last_clean_scrub_stamp":"2022-06-29T15:42:32.123794+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-29T15:42:33.491553+0000","last_change":"2022-06-29T15:42:27.001536+0000","last_active":"2022-06-29T15:42:33.491553+0000","last_peered":"2022-06-29T15:42:33.491553+0000","last_clean":"2022-06-29T15:42:33.491553+0000","last_became_active":"2022-06-29T15:42:27.001089+0000","last_became_peered":"2022-06-29T15:42:27.001089+0000","last_unstale":"2022-06-29T15:42:33.491553+0000","last_undegraded":"2022-06-29T15:42:33.491553+0000","last_fullsized":"2022-06-29T15:42:33.491553+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_clean_scrub_stamp":"2022-06-29T15:42:25.967805+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"25'5","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-29T15:42:33.498703+0000","last_change":"2022-06-29T15:42:30.127732+0000","last_active":"2022-06-29T15:42:33.498703+0000","last_peered":"2022-06-29T15:42:33.498703+0000","last_clean":"2022-06-29T15:42:33.498703+0000","last_became_active":"2022-06-29T15:42:30.126907+0000","last_became_peered":"2022-06-29T15:42:30.126907+0000","last_unstale":"2022-06-29T15:42:33.498703+0000","last_undegraded":"2022-06-29T15:42:33.498703+0000","last_fullsized":"2022-06-29T15:42:33.498703+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:29.103452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:29.103452+0000","last_clean_scrub_stamp":"2022-06-29T15:42:29.103452+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-29T15:42:33.483560+0000","last_change":"2022-06-29T15:42:33.159995+0000","last_active":"2022-06-29T15:42:33.483560+0000","last_peered":"2022-06-29T15:42:33.483560+0000","last_clean":"2022-06-29T15:42:33.483560+0000","last_became_active":"2022-06-29T15:42:33.159461+0000","last_became_peered":"2022-06-29T15:42:33.159461+0000","last_unstale":"2022-06-29T15:42:33.483560+0000","last_undegraded":"2022-06-29T15:42:33.483560+0000","last_fullsized":"2022-06-29T15:42:33.483560+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:32.123794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:32.123794+0000","last_clean_scrub_stamp":"2022-06-29T15:42:32.123794+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"25'6","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-29T15:42:33.501329+0000","last_change":"2022-06-29T15:42:30.130523+0000","last_active":"2022-06-29T15:42:33.501329+0000","last_peered":"2022-06-29T15:42:33.501329+0000","last_clean":"2022-06-29T15:42:33.501329+0000","last_became_active":"2022-06-29T15:42:30.130117+0000","last_became_peered":"2022-06-29T15:42:30.130117+0000","last_unstale":"2022-06-29T15:42:33.501329+0000","last_undegraded":"2022-06-29T15:42:33.501329+0000","last_fullsized":"2022-06-29T15:42:33.501329+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:29.103452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:29.103452+0000","last_clean_scrub_stamp":"2022-06-29T15:42:29.103452+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-29T15:42:33.484454+0000","last_change":"2022-06-29T15:42:27.721658+0000","last_active":"2022-06-29T15:42:33.484454+0000","last_peered":"2022-06-29T15:42:33.484454+0000","last_clean":"2022-06-29T15:42:33.484454+0000","last_became_active":"2022-06-29T15:42:27.720770+0000","last_became_peered":"2022-06-29T15:42:27.720770+0000","last_unstale":"2022-06-29T15:42:33.484454+0000","last_undegraded":"2022-06-29T15:42:33.484454+0000","last_fullsized":"2022-06-29T15:42:33.484454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_clean_scrub_stamp":"2022-06-29T15:42:25.967805+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-29T15:42:33.504744+0000","last_change":"2022-06-29T15:42:33.503014+0000","last_active":"2022-06-29T15:42:33.504744+0000","last_peered":"2022-06-29T15:42:33.504744+0000","last_clean":"2022-06-29T15:42:33.504744+0000","last_became_active":"2022-06-29T15:42:33.502453+0000","last_became_peered":"2022-06-29T15:42:33.502453+0000","last_unstale":"2022-06-29T15:42:33.504744+0000","last_undegraded":"2022-06-29T15:42:33.504744+0000","last_fullsized":"2022-06-29T15:42:33.504744+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:32.123794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:32.123794+0000","last_clean_scrub_stamp":"2022-06-29T15:42:32.123794+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"25'8","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-29T15:42:33.501858+0000","last_change":"2022-06-29T15:42:30.509950+0000","last_active":"2022-06-29T15:42:33.501858+0000","last_peered":"2022-06-29T15:42:33.501858+0000","last_clean":"2022-06-29T15:42:33.501858+0000","last_became_active":"2022-06-29T15:42:30.509152+0000","last_became_peered":"2022-06-29T15:42:30.509152+0000","last_unstale":"2022-06-29T15:42:33.501858+0000","last_undegraded":"2022-06-29T15:42:33.501858+0000","last_fullsized":"2022-06-29T15:42:33.501858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:29.103452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:29.103452+0000","last_clean_scrub_stamp":"2022-06-29T15:42:29.103452+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1535,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-29T15:42:33.502709+0000","last_change":"2022-06-29T15:42:27.723852+0000","last_active":"2022-06-29T15:42:33.502709+0000","last_peered":"2022-06-29T15:42:33.502709+0000","last_clean":"2022-06-29T15:42:33.502709+0000","last_became_active":"2022-06-29T15:42:27.722958+0000","last_became_peered":"2022-06-29T15:42:27.722958+0000","last_unstale":"2022-06-29T15:42:33.502709+0000","last_undegraded":"2022-06-29T15:42:33.502709+0000","last_fullsized":"2022-06-29T15:42:33.502709+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_clean_scrub_stamp":"2022-06-29T15:42:25.967805+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-29T15:42:33.504284+0000","last_change":"2022-06-29T15:42:33.503392+0000","last_active":"2022-06-29T15:42:33.504284+0000","last_peered":"2022-06-29T15:42:33.504284+0000","last_clean":"2022-06-29T15:42:33.504284+0000","last_became_active":"2022-06-29T15:42:33.502807+0000","last_became_peered":"2022-06-29T15:42:33.502807+0000","last_unstale":"2022-06-29T15:42:33.504284+0000","last_undegraded":"2022-06-29T15:42:33.504284+0000","last_fullsized":"2022-06-29T15:42:33.504284+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:32.123794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:32.123794+0000","last_clean_scrub_stamp":"2022-06-29T15:42:32.123794+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"25'7","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-29T15:42:33.499523+0000","last_change":"2022-06-29T15:42:30.709758+0000","last_active":"2022-06-29T15:42:33.499523+0000","last_peered":"2022-06-29T15:42:33.499523+0000","last_clean":"2022-06-29T15:42:33.499523+0000","last_became_active":"2022-06-29T15:42:30.708949+0000","last_became_peered":"2022-06-29T15:42:30.708949+0000","last_unstale":"2022-06-29T15:42:33.499523+0000","last_undegraded":"2022-06-29T15:42:33.499523+0000","last_fullsized":"2022-06-29T15:42:33.499523+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:29.103452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:29.103452+0000","last_clean_scrub_stamp":"2022-06-29T15:42:29.103452+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":575,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-29T15:42:33.483851+0000","last_change":"2022-06-29T15:42:26.998189+0000","last_active":"2022-06-29T15:42:33.483851+0000","last_peered":"2022-06-29T15:42:33.483851+0000","last_clean":"2022-06-29T15:42:33.483851+0000","last_became_active":"2022-06-29T15:42:26.997677+0000","last_became_peered":"2022-06-29T15:42:26.997677+0000","last_unstale":"2022-06-29T15:42:33.483851+0000","last_undegraded":"2022-06-29T15:42:33.483851+0000","last_fullsized":"2022-06-29T15:42:33.483851+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_clean_scrub_stamp":"2022-06-29T15:42:25.967805+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-29T15:42:33.506729+0000","last_change":"2022-06-29T15:42:33.505382+0000","last_active":"2022-06-29T15:42:33.506729+0000","last_peered":"2022-06-29T15:42:33.506729+0000","last_clean":"2022-06-29T15:42:33.506729+0000","last_became_active":"2022-06-29T15:42:33.504714+0000","last_became_peered":"2022-06-29T15:42:33.504714+0000","last_unstale":"2022-06-29T15:42:33.506729+0000","last_undegraded":"2022-06-29T15:42:33.506729+0000","last_fullsized":"2022-06-29T15:42:33.506729+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:32.123794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:32.123794+0000","last_clean_scrub_stamp":"2022-06-29T15:42:32.123794+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"26'4","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2022-06-29T15:42:34.798365+0000","last_change":"2022-06-29T15:42:30.125770+0000","last_active":"2022-06-29T15:42:34.798365+0000","last_peered":"2022-06-29T15:42:34.798365+0000","last_clean":"2022-06-29T15:42:34.798365+0000","last_became_active":"2022-06-29T15:42:30.125364+0000","last_became_peered":"2022-06-29T15:42:30.125364+0000","last_unstale":"2022-06-29T15:42:34.798365+0000","last_undegraded":"2022-06-29T15:42:34.798365+0000","last_fullsized":"2022-06-29T15:42:34.798365+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:29.103452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:29.103452+0000","last_clean_scrub_stamp":"2022-06-29T15:42:29.103452+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-06-29T15:42:33.482363+0000","last_change":"2022-06-29T15:42:26.999647+0000","last_active":"2022-06-29T15:42:33.482363+0000","last_peered":"2022-06-29T15:42:33.482363+0000","last_clean":"2022-06-29T15:42:33.482363+0000","last_became_active":"2022-06-29T15:42:26.999119+0000","last_became_peered":"2022-06-29T15:42:26.999119+0000","last_unstale":"2022-06-29T15:42:33.482363+0000","last_undegraded":"2022-06-29T15:42:33.482363+0000","last_fullsized":"2022-06-29T15:42:33.482363+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_clean_scrub_stamp":"2022-06-29T15:42:25.967805+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":32,"acting":32,"num_store_stats":8},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":16,"acting":16,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":14,"seq":60129542151,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048860,"kb_used_data":276,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322980,"statfs":{"total":96636764160,"available":95562731520,"internally_reserved":1073741824,"allocated":282624,"data_stored":110184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81799999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93100000000000005}]}]},{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048860,"kb_used_data":276,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322980,"statfs":{"total":96636764160,"available":95562731520,"internally_reserved":1073741824,"allocated":282624,"data_stored":110599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.115}]}]},{"osd":6,"up_from":14,"seq":60129542151,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048852,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322988,"statfs":{"total":96636764160,"available":95562739712,"internally_reserved":1073741824,"allocated":274432,"data_stored":110024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1020000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0669999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.163}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.224}]}]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048856,"kb_used_data":272,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322984,"statfs":{"total":96636764160,"available":95562735616,"internally_reserved":1073741824,"allocated":278528,"data_stored":110058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95299999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0209999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4330000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.373}]}]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048868,"kb_used_data":284,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322972,"statfs":{"total":96636764160,"available":95562723328,"internally_reserved":1073741824,"allocated":290816,"data_stored":110759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93999999999999995}]}]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048864,"kb_used_data":280,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322976,"statfs":{"total":96636764160,"available":95562727424,"internally_reserved":1073741824,"allocated":286720,"data_stored":111559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.079}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.006}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.127}]}]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048864,"kb_used_data":280,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322976,"statfs":{"total":96636764160,"available":95562727424,"internally_reserved":1073741824,"allocated":286720,"data_stored":111559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1930000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1200000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048856,"kb_used_data":272,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322984,"statfs":{"total":96636764160,"available":95562735616,"internally_reserved":1073741824,"allocated":278528,"data_stored":110058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96799999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92100000000000004}]}]}]}} 2022-06-29T15:42:43.145 INFO:tasks.ceph.ceph_manager.ceph:clean! 2022-06-29T15:42:43.146 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2022-06-29T15:42:43.146 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2022-06-29T15:42:43.147 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2022-06-29T15:42:43.248 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.244+0000 7f09d2e16700 1 Processor -- start 2022-06-29T15:42:43.248 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.244+0000 7f09d2e16700 1 -- start start 2022-06-29T15:42:43.249 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.244+0000 7f09d2e16700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f09cc0afb80 0x7f09cc0b07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:43.249 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.244+0000 7f09d2e16700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f09cc061f40 0x7f09cc0aff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:43.249 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.244+0000 7f09d2e16700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f09cc060e10 0x7f09cc13d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:43.249 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.244+0000 7f09d2e16700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09cc0ae550 con 0x7f09cc061f40 2022-06-29T15:42:43.250 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.244+0000 7f09d2e16700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09cc061800 con 0x7f09cc060e10 2022-06-29T15:42:43.250 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.244+0000 7f09d2e16700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc061aa0 con 0x7f09cc0afb80 2022-06-29T15:42:43.250 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.244+0000 7f09d1613700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f09cc061f40 0x7f09cc0aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:43.250 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.244+0000 7f09d1613700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f09cc061f40 0x7f09cc0aff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46130/0 (socket says 172.21.15.47:46130) 2022-06-29T15:42:43.251 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.244+0000 7f09d1613700 1 -- 172.21.15.47:0/360653622 learned_addr learned my addr 172.21.15.47:0/360653622 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:43.251 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.244+0000 7f09d1e14700 1 --2- 172.21.15.47:0/360653622 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f09cc0afb80 0x7f09cc0b07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:43.251 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.244+0000 7f09d2615700 1 --2- 172.21.15.47:0/360653622 >> v2:172.21.15.64:3300/0 conn(0x7f09cc060e10 0x7f09cc13d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:43.251 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.244+0000 7f09d1613700 1 -- 172.21.15.47:0/360653622 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f09cc0afb80 msgr2=0x7f09cc0b07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:43.251 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.244+0000 7f09d1613700 1 --2- 172.21.15.47:0/360653622 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f09cc0afb80 0x7f09cc0b07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:43.252 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.244+0000 7f09d1613700 1 -- 172.21.15.47:0/360653622 >> v2:172.21.15.64:3300/0 conn(0x7f09cc060e10 msgr2=0x7f09cc13d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:43.252 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.244+0000 7f09d1613700 1 --2- 172.21.15.47:0/360653622 >> v2:172.21.15.64:3300/0 conn(0x7f09cc060e10 0x7f09cc13d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:43.252 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.244+0000 7f09d1613700 1 -- 172.21.15.47:0/360653622 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09cc0ae0e0 con 0x7f09cc061f40 2022-06-29T15:42:43.253 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.244+0000 7f09d1613700 1 --2- 172.21.15.47:0/360653622 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09cc061f40 0x7f09cc0aff60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f09bc002f70 tx=0x7f09bc006eb0).ready entity=mon.0 client_cookie=f07cc45fdc91ac91 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:43.253 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.244+0000 7f09d0e12700 1 -- 172.21.15.47:0/360653622 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09bc00d570 con 0x7f09cc061f40 2022-06-29T15:42:43.253 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.244+0000 7f09d0e12700 1 -- 172.21.15.47:0/360653622 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09bc000e90 con 0x7f09cc061f40 2022-06-29T15:42:43.254 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.244+0000 7f09d0e12700 1 -- 172.21.15.47:0/360653622 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09bc00d900 con 0x7f09cc061f40 2022-06-29T15:42:43.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.244+0000 7f09d2e16700 1 -- 172.21.15.47:0/360653622 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09cc061f40 msgr2=0x7f09cc0aff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:43.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.244+0000 7f09d2e16700 1 --2- 172.21.15.47:0/360653622 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09cc061f40 0x7f09cc0aff60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f09bc002f70 tx=0x7f09bc006eb0).stop 2022-06-29T15:42:43.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.248+0000 7f09d2e16700 1 -- 172.21.15.47:0/360653622 shutdown_connections 2022-06-29T15:42:43.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.248+0000 7f09d2e16700 1 --2- 172.21.15.47:0/360653622 >> v2:172.21.15.64:3300/0 conn(0x7f09cc060e10 0x7f09cc13d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:43.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.248+0000 7f09d2e16700 1 --2- 172.21.15.47:0/360653622 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09cc061f40 0x7f09cc0aff60 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:43.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.248+0000 7f09d2e16700 1 --2- 172.21.15.47:0/360653622 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f09cc0afb80 0x7f09cc0b07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:43.260 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.248+0000 7f09d2e16700 1 -- 172.21.15.47:0/360653622 >> 172.21.15.47:0/360653622 conn(0x7f09cc0b6430 msgr2=0x7f09cc0b6810 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:43.260 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.248+0000 7f09d2e16700 1 -- 172.21.15.47:0/360653622 shutdown_connections 2022-06-29T15:42:43.260 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.248+0000 7f09d2e16700 1 -- 172.21.15.47:0/360653622 wait complete. 2022-06-29T15:42:43.261 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.248+0000 7f09d2e16700 1 Processor -- start 2022-06-29T15:42:43.261 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.248+0000 7f09d2e16700 1 -- start start 2022-06-29T15:42:43.261 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.248+0000 7f09d2e16700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f09cc060e10 0x7f09cc130b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:43.262 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.248+0000 7f09d2e16700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09cc061f40 0x7f09cc0635b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:43.262 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.248+0000 7f09d2e16700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09cc0afb80 0x7f09cc063aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:43.262 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.248+0000 7f09d2e16700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc0ab1d0 con 0x7f09cc0afb80 2022-06-29T15:42:43.262 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.252+0000 7f09d2e16700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc0ac0f0 con 0x7f09cc061f40 2022-06-29T15:42:43.263 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.252+0000 7f09d2e16700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc063f90 con 0x7f09cc060e10 2022-06-29T15:42:43.263 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.252+0000 7f09d2615700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09cc0afb80 0x7f09cc063aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:43.263 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.252+0000 7f09d2615700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09cc0afb80 0x7f09cc063aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46138/0 (socket says 172.21.15.47:46138) 2022-06-29T15:42:43.263 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.252+0000 7f09d2615700 1 -- 172.21.15.47:0/1542932202 learned_addr learned my addr 172.21.15.47:0/1542932202 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:43.264 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.252+0000 7f09d1613700 1 --2- 172.21.15.47:0/1542932202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09cc061f40 0x7f09cc0635b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:43.264 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.252+0000 7f09d1e14700 1 --2- 172.21.15.47:0/1542932202 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f09cc060e10 0x7f09cc130b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:43.264 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.252+0000 7f09d2615700 1 -- 172.21.15.47:0/1542932202 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f09cc060e10 msgr2=0x7f09cc130b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:43.264 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.252+0000 7f09d2615700 1 --2- 172.21.15.47:0/1542932202 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f09cc060e10 0x7f09cc130b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:43.265 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.252+0000 7f09d2615700 1 -- 172.21.15.47:0/1542932202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09cc061f40 msgr2=0x7f09cc0635b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:43.265 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.252+0000 7f09d2615700 1 --2- 172.21.15.47:0/1542932202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09cc061f40 0x7f09cc0635b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:43.266 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.252+0000 7f09d2615700 1 -- 172.21.15.47:0/1542932202 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09bc00b1e0 con 0x7f09cc0afb80 2022-06-29T15:42:43.266 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.252+0000 7f09d1e14700 1 --2- 172.21.15.47:0/1542932202 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f09cc060e10 0x7f09cc130b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:42:43.266 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.252+0000 7f09d2615700 1 --2- 172.21.15.47:0/1542932202 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09cc0afb80 0x7f09cc063aa0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f09c400c000 tx=0x7f09c4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:43.266 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.252+0000 7f09baffd700 1 -- 172.21.15.47:0/1542932202 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09c4011750 con 0x7f09cc0afb80 2022-06-29T15:42:43.267 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.252+0000 7f09baffd700 1 -- 172.21.15.47:0/1542932202 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09c40118b0 con 0x7f09cc0afb80 2022-06-29T15:42:43.267 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.252+0000 7f09d2e16700 1 -- 172.21.15.47:0/1542932202 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09cc0ab400 con 0x7f09cc0afb80 2022-06-29T15:42:43.267 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.252+0000 7f09baffd700 1 -- 172.21.15.47:0/1542932202 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09c4011c10 con 0x7f09cc0afb80 2022-06-29T15:42:43.267 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.252+0000 7f09d2e16700 1 -- 172.21.15.47:0/1542932202 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09cc0ac7f0 con 0x7f09cc0afb80 2022-06-29T15:42:43.267 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.252+0000 7f09d2e16700 1 -- 172.21.15.47:0/1542932202 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09cc130590 con 0x7f09cc0afb80 2022-06-29T15:42:43.268 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.252+0000 7f09baffd700 1 -- 172.21.15.47:0/1542932202 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f09c4011d70 con 0x7f09cc0afb80 2022-06-29T15:42:43.268 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.252+0000 7f09baffd700 1 --2- 172.21.15.47:0/1542932202 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f09ac0539e0 0x7f09ac055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:43.268 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.256+0000 7f09baffd700 1 -- 172.21.15.47:0/1542932202 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f09c406c370 con 0x7f09cc0afb80 2022-06-29T15:42:43.268 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.256+0000 7f09d1e14700 1 --2- 172.21.15.47:0/1542932202 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f09ac0539e0 0x7f09ac055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:43.269 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.256+0000 7f09d1e14700 1 --2- 172.21.15.47:0/1542932202 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f09ac0539e0 0x7f09ac055ea0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f09c800b000 tx=0x7f09c80091d0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:43.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.260+0000 7f09baffd700 1 -- 172.21.15.47:0/1542932202 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f09c4043130 con 0x7f09cc0afb80 2022-06-29T15:42:43.436 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.432+0000 7f09d2e16700 1 -- 172.21.15.47:0/1542932202 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f09cc0573d0 con 0x7f09cc0afb80 2022-06-29T15:42:43.437 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.436+0000 7f09baffd700 1 -- 172.21.15.47:0/1542932202 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+45 (secure 0 0 0) 0x7f09c4046750 con 0x7f09cc0afb80 2022-06-29T15:42:43.437 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:42:43.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.436+0000 7f09d2e16700 1 -- 172.21.15.47:0/1542932202 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f09ac0539e0 msgr2=0x7f09ac055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:43.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.436+0000 7f09d2e16700 1 --2- 172.21.15.47:0/1542932202 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f09ac0539e0 0x7f09ac055ea0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f09c800b000 tx=0x7f09c80091d0).stop 2022-06-29T15:42:43.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.436+0000 7f09d2e16700 1 -- 172.21.15.47:0/1542932202 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09cc0afb80 msgr2=0x7f09cc063aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:43.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.436+0000 7f09d2e16700 1 --2- 172.21.15.47:0/1542932202 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09cc0afb80 0x7f09cc063aa0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f09c400c000 tx=0x7f09c4006df0).stop 2022-06-29T15:42:43.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.436+0000 7f09d2e16700 1 -- 172.21.15.47:0/1542932202 shutdown_connections 2022-06-29T15:42:43.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.436+0000 7f09d2e16700 1 --2- 172.21.15.47:0/1542932202 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09cc0afb80 0x7f09cc063aa0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:43.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.436+0000 7f09d2e16700 1 --2- 172.21.15.47:0/1542932202 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f09ac0539e0 0x7f09ac055ea0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:43.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.436+0000 7f09d2e16700 1 --2- 172.21.15.47:0/1542932202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09cc061f40 0x7f09cc0635b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:43.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.436+0000 7f09d2e16700 1 --2- 172.21.15.47:0/1542932202 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f09cc060e10 0x7f09cc130b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:43.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.436+0000 7f09d2e16700 1 -- 172.21.15.47:0/1542932202 >> 172.21.15.47:0/1542932202 conn(0x7f09cc0b6430 msgr2=0x7f09cc0b1410 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:43.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.436+0000 7f09d2e16700 1 -- 172.21.15.47:0/1542932202 shutdown_connections 2022-06-29T15:42:43.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.436+0000 7f09d2e16700 1 -- 172.21.15.47:0/1542932202 wait complete. 2022-06-29T15:42:43.450 INFO:teuthology.orchestra.run.smithi047.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2022-06-29T15:42:43.451 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2022-06-29T15:42:43.451 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:42:43.553 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.552+0000 7f8e3e7dc700 1 Processor -- start 2022-06-29T15:42:43.554 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.552+0000 7f8e3e7dc700 1 -- start start 2022-06-29T15:42:43.555 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.552+0000 7f8e3e7dc700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8e380b15c0 0x7f8e380b06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:43.555 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.552+0000 7f8e3e7dc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8e380b2a70 0x7f8e38147570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:43.556 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.552+0000 7f8e3e7dc700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e38147a60 con 0x7f8e380b3440 2022-06-29T15:42:43.556 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.552+0000 7f8e3e7dc700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e380aee00 con 0x7f8e380b2a70 2022-06-29T15:42:43.559 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.552+0000 7f8e3e7dc700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e380ad770 con 0x7f8e380b15c0 2022-06-29T15:42:43.559 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.552+0000 7f8e377fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8e380b2a70 0x7f8e38147570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:43.560 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.552+0000 7f8e37fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8e380b15c0 0x7f8e380b06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:43.560 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.552+0000 7f8e3cd79700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f8e380b3440 0x7f8e38067100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:39264/0 (socket says 172.21.15.47:39264) 2022-06-29T15:42:43.561 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.552+0000 7f8e377fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8e380b2a70 0x7f8e38147570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:51846/0 (socket says 172.21.15.47:51846) 2022-06-29T15:42:43.561 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.552+0000 7f8e3cd79700 1 -- 172.21.15.47:0/528418505 learned_addr learned my addr 172.21.15.47:0/528418505 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:43.561 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.552+0000 7f8e37fff700 1 -- 172.21.15.47:0/528418505 >> v2:172.21.15.64:3300/0 conn(0x7f8e380b2a70 msgr2=0x7f8e38147570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:43.562 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.552+0000 7f8e37fff700 1 --2- 172.21.15.47:0/528418505 >> v2:172.21.15.64:3300/0 conn(0x7f8e380b2a70 0x7f8e38147570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:43.562 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.552+0000 7f8e37fff700 1 -- 172.21.15.47:0/528418505 >> v1:172.21.15.64:6789/0 conn(0x7f8e380b3440 legacy=0x7f8e38067100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:43.562 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.552+0000 7f8e37fff700 1 -- 172.21.15.47:0/528418505 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e380b2270 con 0x7f8e380b15c0 2022-06-29T15:42:43.563 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.552+0000 7f8e36ffd700 1 -- 172.21.15.47:0/528418505 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1570255893 0 0) 0x7f8e30001110 con 0x7f8e380b3440 2022-06-29T15:42:43.563 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.552+0000 7f8e377fe700 1 --2- 172.21.15.47:0/528418505 >> v2:172.21.15.64:3300/0 conn(0x7f8e380b2a70 0x7f8e38147570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:42:43.563 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.552+0000 7f8e37fff700 1 --2- 172.21.15.47:0/528418505 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8e380b15c0 0x7f8e380b06f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f8e20002f70 tx=0x7f8e20006df0).ready entity=mon.2 client_cookie=fc79049ae85a395 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:43.563 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.552+0000 7f8e36ffd700 1 -- 172.21.15.47:0/528418505 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e2000d570 con 0x7f8e380b15c0 2022-06-29T15:42:43.564 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.552+0000 7f8e36ffd700 1 -- 172.21.15.47:0/528418505 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e20000e90 con 0x7f8e380b15c0 2022-06-29T15:42:43.564 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.552+0000 7f8e36ffd700 1 -- 172.21.15.47:0/528418505 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e2000d8c0 con 0x7f8e380b15c0 2022-06-29T15:42:43.565 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.556+0000 7f8e3e7dc700 1 -- 172.21.15.47:0/528418505 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8e380b15c0 msgr2=0x7f8e380b06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:43.565 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.556+0000 7f8e3e7dc700 1 --2- 172.21.15.47:0/528418505 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8e380b15c0 0x7f8e380b06f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f8e20002f70 tx=0x7f8e20006df0).stop 2022-06-29T15:42:43.565 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.556+0000 7f8e3e7dc700 1 -- 172.21.15.47:0/528418505 shutdown_connections 2022-06-29T15:42:43.565 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.556+0000 7f8e3e7dc700 1 --2- 172.21.15.47:0/528418505 >> v2:172.21.15.64:3300/0 conn(0x7f8e380b2a70 0x7f8e38147570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:43.566 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.556+0000 7f8e3e7dc700 1 --2- 172.21.15.47:0/528418505 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8e380b15c0 0x7f8e380b06f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:43.566 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.556+0000 7f8e3e7dc700 1 -- 172.21.15.47:0/528418505 >> 172.21.15.47:0/528418505 conn(0x7f8e380b98b0 msgr2=0x7f8e380b9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:43.566 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.556+0000 7f8e3e7dc700 1 -- 172.21.15.47:0/528418505 shutdown_connections 2022-06-29T15:42:43.566 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.556+0000 7f8e3e7dc700 1 -- 172.21.15.47:0/528418505 wait complete. 2022-06-29T15:42:43.567 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.556+0000 7f8e3e7dc700 1 Processor -- start 2022-06-29T15:42:43.567 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.556+0000 7f8e3e7dc700 1 -- start start 2022-06-29T15:42:43.570 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.556+0000 7f8e3e7dc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e380b15c0 0x7f8e380b7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:43.570 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.556+0000 7f8e3e7dc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e380b2a70 0x7f8e38062570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:43.570 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.556+0000 7f8e3e7dc700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8e380b3440 0x7f8e38062a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:43.571 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.556+0000 7f8e3e7dc700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3814cdf0 con 0x7f8e380b15c0 2022-06-29T15:42:43.571 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.556+0000 7f8e3e7dc700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3814b000 con 0x7f8e380b2a70 2022-06-29T15:42:43.571 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.556+0000 7f8e3e7dc700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3814b710 con 0x7f8e380b3440 2022-06-29T15:42:43.571 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.556+0000 7f8e37fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e380b15c0 0x7f8e380b7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:43.572 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.556+0000 7f8e37fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e380b15c0 0x7f8e380b7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46152/0 (socket says 172.21.15.47:46152) 2022-06-29T15:42:43.572 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.556+0000 7f8e37fff700 1 -- 172.21.15.47:0/2080347848 learned_addr learned my addr 172.21.15.47:0/2080347848 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:43.572 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.556+0000 7f8e3cd79700 1 --2- 172.21.15.47:0/2080347848 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8e380b3440 0x7f8e38062a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:43.572 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.556+0000 7f8e377fe700 1 --2- 172.21.15.47:0/2080347848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e380b2a70 0x7f8e38062570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:43.573 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.560+0000 7f8e3cd79700 1 -- 172.21.15.47:0/2080347848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e380b2a70 msgr2=0x7f8e38062570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:43.573 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.560+0000 7f8e3cd79700 1 --2- 172.21.15.47:0/2080347848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e380b2a70 0x7f8e38062570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:43.573 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.560+0000 7f8e3cd79700 1 -- 172.21.15.47:0/2080347848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e380b15c0 msgr2=0x7f8e380b7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:43.573 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.560+0000 7f8e3cd79700 1 --2- 172.21.15.47:0/2080347848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e380b15c0 0x7f8e380b7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:43.574 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.560+0000 7f8e3cd79700 1 -- 172.21.15.47:0/2080347848 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e28009470 con 0x7f8e380b3440 2022-06-29T15:42:43.574 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.560+0000 7f8e37fff700 1 --2- 172.21.15.47:0/2080347848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e380b15c0 0x7f8e380b7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:42:43.574 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.560+0000 7f8e3cd79700 1 --2- 172.21.15.47:0/2080347848 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8e380b3440 0x7f8e38062a60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f8e30000ed0 tx=0x7f8e30007ba0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:43.574 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.560+0000 7f8e357fa700 1 -- 172.21.15.47:0/2080347848 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e30007e30 con 0x7f8e380b3440 2022-06-29T15:42:43.575 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.560+0000 7f8e357fa700 1 -- 172.21.15.47:0/2080347848 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e30007110 con 0x7f8e380b3440 2022-06-29T15:42:43.575 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.560+0000 7f8e3e7dc700 1 -- 172.21.15.47:0/2080347848 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e2000b180 con 0x7f8e380b3440 2022-06-29T15:42:43.575 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.560+0000 7f8e357fa700 1 -- 172.21.15.47:0/2080347848 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e30007470 con 0x7f8e380b3440 2022-06-29T15:42:43.575 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.560+0000 7f8e3e7dc700 1 -- 172.21.15.47:0/2080347848 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e3814cb50 con 0x7f8e380b3440 2022-06-29T15:42:43.576 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.560+0000 7f8e3e7dc700 1 -- 172.21.15.47:0/2080347848 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e380573d0 con 0x7f8e380b3440 2022-06-29T15:42:43.576 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.560+0000 7f8e357fa700 1 -- 172.21.15.47:0/2080347848 <== mon.2 v2:172.21.15.64:3301/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8e30005030 con 0x7f8e380b3440 2022-06-29T15:42:43.576 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.560+0000 7f8e357fa700 1 --2- 172.21.15.47:0/2080347848 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8e10053b30 0x7f8e10055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:43.576 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.560+0000 7f8e37fff700 1 --2- 172.21.15.47:0/2080347848 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8e10053b30 0x7f8e10055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:43.577 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.560+0000 7f8e357fa700 1 -- 172.21.15.47:0/2080347848 <== mon.2 v2:172.21.15.64:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f8e3006b5f0 con 0x7f8e380b3440 2022-06-29T15:42:43.577 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.564+0000 7f8e37fff700 1 --2- 172.21.15.47:0/2080347848 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8e10053b30 0x7f8e10055ff0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f8e2001a000 tx=0x7f8e20014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:43.578 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.568+0000 7f8e357fa700 1 -- 172.21.15.47:0/2080347848 <== mon.2 v2:172.21.15.64:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8e30042310 con 0x7f8e380b3440 2022-06-29T15:42:43.700 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.696+0000 7f8e3e7dc700 1 -- 172.21.15.47:0/2080347848 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8e38060e70 con 0x7f8e380b3440 2022-06-29T15:42:43.702 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.700+0000 7f8e357fa700 1 -- 172.21.15.47:0/2080347848 <== mon.2 v2:172.21.15.64:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f8e30045930 con 0x7f8e380b3440 2022-06-29T15:42:43.708 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:42:43.708 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":26,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:42:34.474438+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:42:28.400446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-06-29T15:42:31.167705+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:42:43.709 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.700+0000 7f8e3e7dc700 1 -- 172.21.15.47:0/2080347848 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8e10053b30 msgr2=0x7f8e10055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:43.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.700+0000 7f8e3e7dc700 1 --2- 172.21.15.47:0/2080347848 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8e10053b30 0x7f8e10055ff0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f8e2001a000 tx=0x7f8e20014000).stop 2022-06-29T15:42:43.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.700+0000 7f8e3e7dc700 1 -- 172.21.15.47:0/2080347848 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8e380b3440 msgr2=0x7f8e38062a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:43.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.700+0000 7f8e3e7dc700 1 --2- 172.21.15.47:0/2080347848 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8e380b3440 0x7f8e38062a60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f8e30000ed0 tx=0x7f8e30007ba0).stop 2022-06-29T15:42:43.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.700+0000 7f8e3e7dc700 1 -- 172.21.15.47:0/2080347848 shutdown_connections 2022-06-29T15:42:43.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.700+0000 7f8e3e7dc700 1 --2- 172.21.15.47:0/2080347848 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8e380b3440 0x7f8e38062a60 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:43.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.700+0000 7f8e3e7dc700 1 --2- 172.21.15.47:0/2080347848 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8e10053b30 0x7f8e10055ff0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:43.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.700+0000 7f8e3e7dc700 1 --2- 172.21.15.47:0/2080347848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e380b2a70 0x7f8e38062570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:43.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.700+0000 7f8e3e7dc700 1 --2- 172.21.15.47:0/2080347848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e380b15c0 0x7f8e380b7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:43.712 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.700+0000 7f8e3e7dc700 1 -- 172.21.15.47:0/2080347848 >> 172.21.15.47:0/2080347848 conn(0x7f8e380b98b0 msgr2=0x7f8e381467c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:43.712 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.700+0000 7f8e3e7dc700 1 -- 172.21.15.47:0/2080347848 shutdown_connections 2022-06-29T15:42:43.712 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.700+0000 7f8e3e7dc700 1 -- 172.21.15.47:0/2080347848 wait complete. 2022-06-29T15:42:43.717 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-29T15:42:16.971431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-29T15:42:25.290884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-29T15:42:28.400446+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2022-06-29T15:42:31.167705+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-29T15:42:43.718 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-29T15:42:43.823 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.820+0000 7f5d325a1700 1 Processor -- start 2022-06-29T15:42:43.823 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.820+0000 7f5d325a1700 1 -- start start 2022-06-29T15:42:43.825 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.820+0000 7f5d325a1700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5d2c1480d0 0x7f5d2c0b5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:43.825 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.820+0000 7f5d325a1700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d2c0b0720 con 0x7f5d2c141580 2022-06-29T15:42:43.825 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.820+0000 7f5d325a1700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d2c1451f0 con 0x7f5d2c060560 2022-06-29T15:42:43.826 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.820+0000 7f5d325a1700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d2c1476f0 con 0x7f5d2c1480d0 2022-06-29T15:42:43.826 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.820+0000 7f5d2bfff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f5d2c141580 0x7f5d2c0afef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59942/0 (socket says 172.21.15.47:59942) 2022-06-29T15:42:43.826 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.820+0000 7f5d2bfff700 1 -- 172.21.15.47:0/396889163 learned_addr learned my addr 172.21.15.47:0/396889163 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:43.826 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.820+0000 7f5d237fe700 1 --2- 172.21.15.47:0/396889163 >> v2:172.21.15.64:3300/0 conn(0x7f5d2c1480d0 0x7f5d2c0b5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:43.827 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.824+0000 7f5d2b7fe700 1 -- 172.21.15.47:0/396889163 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 754664863 0 0) 0x7f5d2c0b0720 con 0x7f5d2c141580 2022-06-29T15:42:43.827 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.824+0000 7f5d2b7fe700 1 -- 172.21.15.47:0/396889163 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d0c003260 con 0x7f5d2c141580 2022-06-29T15:42:43.827 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.824+0000 7f5d237fe700 1 -- 172.21.15.47:0/396889163 >> v1:172.21.15.64:6789/0 conn(0x7f5d2c060560 legacy=0x7f5d2c151ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:43.827 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.824+0000 7f5d237fe700 1 -- 172.21.15.47:0/396889163 >> v1:172.21.15.47:6789/0 conn(0x7f5d2c141580 legacy=0x7f5d2c0afef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:43.827 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.824+0000 7f5d2b7fe700 1 -- 172.21.15.47:0/396889163 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1036397276 0 0) 0x7f5d0c003260 con 0x7f5d2c141580 2022-06-29T15:42:43.828 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.824+0000 7f5d237fe700 1 -- 172.21.15.47:0/396889163 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d2c147990 con 0x7f5d2c1480d0 2022-06-29T15:42:43.828 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.824+0000 7f5d237fe700 1 --2- 172.21.15.47:0/396889163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d2c1480d0 0x7f5d2c0b5460 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f5d1c001800 tx=0x7f5d14006df0).ready entity=mon.1 client_cookie=af237738c6456251 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:43.831 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.824+0000 7f5d2b7fe700 1 -- 172.21.15.47:0/396889163 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d1c001660 con 0x7f5d2c1480d0 2022-06-29T15:42:43.831 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.824+0000 7f5d2b7fe700 1 -- 172.21.15.47:0/396889163 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d14000e90 con 0x7f5d2c1480d0 2022-06-29T15:42:43.832 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.824+0000 7f5d2b7fe700 1 -- 172.21.15.47:0/396889163 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d1400d4f0 con 0x7f5d2c1480d0 2022-06-29T15:42:43.832 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.824+0000 7f5d325a1700 1 -- 172.21.15.47:0/396889163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d2c1480d0 msgr2=0x7f5d2c0b5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:43.832 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.824+0000 7f5d325a1700 1 --2- 172.21.15.47:0/396889163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d2c1480d0 0x7f5d2c0b5460 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f5d1c001800 tx=0x7f5d14006df0).stop 2022-06-29T15:42:43.832 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.824+0000 7f5d325a1700 1 -- 172.21.15.47:0/396889163 shutdown_connections 2022-06-29T15:42:43.833 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.824+0000 7f5d325a1700 1 --2- 172.21.15.47:0/396889163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d2c1480d0 0x7f5d2c0b5460 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:43.833 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.824+0000 7f5d325a1700 1 -- 172.21.15.47:0/396889163 >> 172.21.15.47:0/396889163 conn(0x7f5d2c0626f0 msgr2=0x7f5d2c062ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:43.833 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.824+0000 7f5d325a1700 1 -- 172.21.15.47:0/396889163 shutdown_connections 2022-06-29T15:42:43.833 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.824+0000 7f5d325a1700 1 -- 172.21.15.47:0/396889163 wait complete. 2022-06-29T15:42:43.833 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.824+0000 7f5d325a1700 1 Processor -- start 2022-06-29T15:42:43.834 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.824+0000 7f5d325a1700 1 -- start start 2022-06-29T15:42:43.834 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.824+0000 7f5d325a1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d2c060560 0x7f5d2c14e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:43.834 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.824+0000 7f5d325a1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d2c141580 0x7f5d2c156e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:43.834 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.824+0000 7f5d325a1700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5d2c1480d0 0x7f5d2c157310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:43.835 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.824+0000 7f5d325a1700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c0b5de0 con 0x7f5d2c141580 2022-06-29T15:42:43.835 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.824+0000 7f5d325a1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c157800 con 0x7f5d2c060560 2022-06-29T15:42:43.835 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.824+0000 7f5d325a1700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c063dc0 con 0x7f5d2c1480d0 2022-06-29T15:42:43.835 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.824+0000 7f5d237fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d2c141580 0x7f5d2c156e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:43.835 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.824+0000 7f5d237fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d2c141580 0x7f5d2c156e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46162/0 (socket says 172.21.15.47:46162) 2022-06-29T15:42:43.836 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.824+0000 7f5d237fe700 1 -- 172.21.15.47:0/276438567 learned_addr learned my addr 172.21.15.47:0/276438567 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:43.836 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.824+0000 7f5d2bfff700 1 --2- 172.21.15.47:0/276438567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d2c060560 0x7f5d2c14e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:43.836 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.824+0000 7f5d30b3e700 1 --2- 172.21.15.47:0/276438567 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5d2c1480d0 0x7f5d2c157310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:43.836 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.828+0000 7f5d237fe700 1 -- 172.21.15.47:0/276438567 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5d2c1480d0 msgr2=0x7f5d2c157310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:43.837 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.828+0000 7f5d237fe700 1 --2- 172.21.15.47:0/276438567 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5d2c1480d0 0x7f5d2c157310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:43.837 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.828+0000 7f5d237fe700 1 -- 172.21.15.47:0/276438567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d2c060560 msgr2=0x7f5d2c14e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:43.837 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.828+0000 7f5d237fe700 1 --2- 172.21.15.47:0/276438567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d2c060560 0x7f5d2c14e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:43.837 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.828+0000 7f5d237fe700 1 -- 172.21.15.47:0/276438567 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d1400b180 con 0x7f5d2c141580 2022-06-29T15:42:43.837 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.828+0000 7f5d237fe700 1 --2- 172.21.15.47:0/276438567 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d2c141580 0x7f5d2c156e20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f5d1c001800 tx=0x7f5d14014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:43.838 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.828+0000 7f5d29ffb700 1 -- 172.21.15.47:0/276438567 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d14016070 con 0x7f5d2c141580 2022-06-29T15:42:43.838 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.828+0000 7f5d29ffb700 1 -- 172.21.15.47:0/276438567 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d14002f50 con 0x7f5d2c141580 2022-06-29T15:42:43.838 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.828+0000 7f5d325a1700 1 -- 172.21.15.47:0/276438567 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d2c1492c0 con 0x7f5d2c141580 2022-06-29T15:42:43.838 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.828+0000 7f5d29ffb700 1 -- 172.21.15.47:0/276438567 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d1400d4f0 con 0x7f5d2c141580 2022-06-29T15:42:43.839 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.828+0000 7f5d325a1700 1 -- 172.21.15.47:0/276438567 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d2c156950 con 0x7f5d2c141580 2022-06-29T15:42:43.839 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.828+0000 7f5d29ffb700 1 -- 172.21.15.47:0/276438567 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5d14025030 con 0x7f5d2c141580 2022-06-29T15:42:43.839 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.828+0000 7f5d325a1700 1 -- 172.21.15.47:0/276438567 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d2c0573d0 con 0x7f5d2c141580 2022-06-29T15:42:43.840 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.828+0000 7f5d29ffb700 1 --2- 172.21.15.47:0/276438567 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5d10053b30 0x7f5d10055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:43.840 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.828+0000 7f5d2bfff700 1 --2- 172.21.15.47:0/276438567 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5d10053b30 0x7f5d10055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:43.840 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.828+0000 7f5d29ffb700 1 -- 172.21.15.47:0/276438567 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f5d1406cff0 con 0x7f5d2c141580 2022-06-29T15:42:43.841 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.828+0000 7f5d2bfff700 1 --2- 172.21.15.47:0/276438567 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5d10053b30 0x7f5d10055ff0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f5d1c00d000 tx=0x7f5d1c000bc0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:43.841 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.832+0000 7f5d29ffb700 1 -- 172.21.15.47:0/276438567 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5d14043d00 con 0x7f5d2c141580 2022-06-29T15:42:43.962 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.960+0000 7f5d325a1700 1 -- 172.21.15.47:0/276438567 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f5d2c0b5820 con 0x7f5d2c141580 2022-06-29T15:42:43.963 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.960+0000 7f5d29ffb700 1 -- 172.21.15.47:0/276438567 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v26) v1 ==== 110+0+10 (secure 0 0 0) 0x7f5d14023070 con 0x7f5d2c141580 2022-06-29T15:42:43.963 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2022-06-29T15:42:43.967 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.964+0000 7f5d325a1700 1 -- 172.21.15.47:0/276438567 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5d10053b30 msgr2=0x7f5d10055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:43.968 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.964+0000 7f5d325a1700 1 --2- 172.21.15.47:0/276438567 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5d10053b30 0x7f5d10055ff0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f5d1c00d000 tx=0x7f5d1c000bc0).stop 2022-06-29T15:42:43.968 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.964+0000 7f5d325a1700 1 -- 172.21.15.47:0/276438567 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d2c141580 msgr2=0x7f5d2c156e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:43.969 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.964+0000 7f5d325a1700 1 --2- 172.21.15.47:0/276438567 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d2c141580 0x7f5d2c156e20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f5d1c001800 tx=0x7f5d14014000).stop 2022-06-29T15:42:43.969 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.964+0000 7f5d325a1700 1 -- 172.21.15.47:0/276438567 shutdown_connections 2022-06-29T15:42:43.970 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.964+0000 7f5d325a1700 1 --2- 172.21.15.47:0/276438567 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5d2c1480d0 0x7f5d2c157310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:43.970 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.964+0000 7f5d325a1700 1 --2- 172.21.15.47:0/276438567 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d2c141580 0x7f5d2c156e20 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:43.971 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.964+0000 7f5d325a1700 1 --2- 172.21.15.47:0/276438567 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5d10053b30 0x7f5d10055ff0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:43.971 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.964+0000 7f5d325a1700 1 --2- 172.21.15.47:0/276438567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d2c060560 0x7f5d2c14e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:43.972 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.964+0000 7f5d325a1700 1 -- 172.21.15.47:0/276438567 >> 172.21.15.47:0/276438567 conn(0x7f5d2c0626f0 msgr2=0x7f5d2c0b0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:43.972 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.964+0000 7f5d325a1700 1 -- 172.21.15.47:0/276438567 shutdown_connections 2022-06-29T15:42:43.973 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:43.964+0000 7f5d325a1700 1 -- 172.21.15.47:0/276438567 wait complete. 2022-06-29T15:42:43.981 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-29T15:42:44.089 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.084+0000 7ff31ffff700 1 Processor -- start 2022-06-29T15:42:44.089 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.084+0000 7ff31ffff700 1 -- start start 2022-06-29T15:42:44.100 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.088+0000 7ff31ffff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff3200b5b20 0x7ff32013d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:44.100 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.088+0000 7ff31ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff3200a9c60 0x7ff32013d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:44.101 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.088+0000 7ff31ffff700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3200b0190 con 0x7ff3200a9250 2022-06-29T15:42:44.101 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.088+0000 7ff31ffff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3200aa570 con 0x7ff3200a9c60 2022-06-29T15:42:44.101 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.088+0000 7ff31ffff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3200b03c0 con 0x7ff3200b5b20 2022-06-29T15:42:44.101 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.088+0000 7ff31f7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff3200a9250 0x7ff320140160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59956/0 (socket says 172.21.15.47:59956) 2022-06-29T15:42:44.102 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.088+0000 7ff31f7fe700 1 -- 172.21.15.47:0/3964278403 learned_addr learned my addr 172.21.15.47:0/3964278403 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:44.102 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.088+0000 7ff31e7fc700 1 --2- 172.21.15.47:0/3964278403 >> v2:172.21.15.64:3300/0 conn(0x7ff3200a9c60 0x7ff32013d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:44.102 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.088+0000 7ff31effd700 1 --2- 172.21.15.47:0/3964278403 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff3200b5b20 0x7ff32013d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:44.102 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.088+0000 7ff31dffb700 1 -- 172.21.15.47:0/3964278403 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2798796501 0 0) 0x7ff3200b0190 con 0x7ff3200a9250 2022-06-29T15:42:44.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.088+0000 7ff31dffb700 1 -- 172.21.15.47:0/3964278403 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff300003260 con 0x7ff3200a9250 2022-06-29T15:42:44.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.088+0000 7ff31dffb700 1 -- 172.21.15.47:0/3964278403 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2887230718 0 0) 0x7ff300003260 con 0x7ff3200a9250 2022-06-29T15:42:44.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.088+0000 7ff31dffb700 1 -- 172.21.15.47:0/3964278403 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff3200b5b20 msgr2=0x7ff32013d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:44.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.088+0000 7ff31dffb700 1 --2- 172.21.15.47:0/3964278403 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff3200b5b20 0x7ff32013d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:44.104 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.088+0000 7ff31dffb700 1 -- 172.21.15.47:0/3964278403 >> v2:172.21.15.64:3300/0 conn(0x7ff3200a9c60 msgr2=0x7ff32013d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:44.104 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.088+0000 7ff31dffb700 1 --2- 172.21.15.47:0/3964278403 >> v2:172.21.15.64:3300/0 conn(0x7ff3200a9c60 0x7ff32013d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:44.104 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.088+0000 7ff31dffb700 1 -- 172.21.15.47:0/3964278403 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3200a9670 con 0x7ff3200a9250 2022-06-29T15:42:44.104 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.088+0000 7ff31dffb700 1 -- 172.21.15.47:0/3964278403 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7ff3180019e0 con 0x7ff3200a9250 2022-06-29T15:42:44.105 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.088+0000 7ff31dffb700 1 -- 172.21.15.47:0/3964278403 >> v1:172.21.15.47:6789/0 conn(0x7ff3200a9250 legacy=0x7ff320140160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:44.105 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.088+0000 7ff31dffb700 1 --2- 172.21.15.47:0/3964278403 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff3200b5b20 0x7ff300003750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:44.105 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.088+0000 7ff31dffb700 1 --2- 172.21.15.47:0/3964278403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff300003c80 0x7ff300004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:44.105 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.088+0000 7ff31dffb700 1 --2- 172.21.15.47:0/3964278403 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3000049f0 0x7ff300004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:44.106 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.088+0000 7ff31dffb700 1 -- 172.21.15.47:0/3964278403 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3200aa570 con 0x7ff300003c80 2022-06-29T15:42:44.106 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.088+0000 7ff31dffb700 1 -- 172.21.15.47:0/3964278403 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3200b03c0 con 0x7ff3000049f0 2022-06-29T15:42:44.106 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.088+0000 7ff31dffb700 1 -- 172.21.15.47:0/3964278403 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff30000b750 con 0x7ff3200b5b20 2022-06-29T15:42:44.106 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.088+0000 7ff31e7fc700 1 --2- 172.21.15.47:0/3964278403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff300003c80 0x7ff300004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:44.107 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.088+0000 7ff31effd700 1 --2- 172.21.15.47:0/3964278403 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff3200b5b20 0x7ff300003750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:44.107 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.088+0000 7ff31f7fe700 1 --2- 172.21.15.47:0/3964278403 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3000049f0 0x7ff300004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:44.107 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.088+0000 7ff31e7fc700 1 -- 172.21.15.47:0/3964278403 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff3200b5b20 msgr2=0x7ff300003750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:44.107 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.088+0000 7ff31e7fc700 1 --2- 172.21.15.47:0/3964278403 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff3200b5b20 0x7ff300003750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:44.107 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.088+0000 7ff31e7fc700 1 -- 172.21.15.47:0/3964278403 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3000049f0 msgr2=0x7ff300004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:44.108 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.088+0000 7ff31e7fc700 1 --2- 172.21.15.47:0/3964278403 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3000049f0 0x7ff300004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:44.108 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.088+0000 7ff31e7fc700 1 -- 172.21.15.47:0/3964278403 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff300003260 con 0x7ff300003c80 2022-06-29T15:42:44.108 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.088+0000 7ff31effd700 1 --2- 172.21.15.47:0/3964278403 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff3200b5b20 0x7ff300003750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:42:44.108 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.088+0000 7ff31f7fe700 1 --2- 172.21.15.47:0/3964278403 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3000049f0 0x7ff300004e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:42:44.109 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.088+0000 7ff31e7fc700 1 --2- 172.21.15.47:0/3964278403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff300003c80 0x7ff300004500 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7ff3000054d0 tx=0x7ff308006df0).ready entity=mon.0 client_cookie=b0dd961103dad686 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:44.109 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.088+0000 7ff31dffb700 1 -- 172.21.15.47:0/3964278403 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff30800e830 con 0x7ff300003c80 2022-06-29T15:42:44.109 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.088+0000 7ff31dffb700 1 -- 172.21.15.47:0/3964278403 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff30800be90 con 0x7ff300003c80 2022-06-29T15:42:44.109 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.088+0000 7ff31dffb700 1 -- 172.21.15.47:0/3964278403 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff30800eb80 con 0x7ff300003c80 2022-06-29T15:42:44.110 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.091+0000 7ff31ffff700 1 -- 172.21.15.47:0/3964278403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff300003c80 msgr2=0x7ff300004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:44.110 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.091+0000 7ff31ffff700 1 --2- 172.21.15.47:0/3964278403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff300003c80 0x7ff300004500 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7ff3000054d0 tx=0x7ff308006df0).stop 2022-06-29T15:42:44.110 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.091+0000 7ff31f7fe700 1 -- 172.21.15.47:0/3964278403 reap_dead start 2022-06-29T15:42:44.110 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.091+0000 7ff31ffff700 1 -- 172.21.15.47:0/3964278403 shutdown_connections 2022-06-29T15:42:44.111 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.091+0000 7ff31ffff700 1 -- 172.21.15.47:0/3964278403 >> 172.21.15.47:0/3964278403 conn(0x7ff320062560 msgr2=0x7ff320062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:44.111 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.091+0000 7ff31ffff700 1 -- 172.21.15.47:0/3964278403 shutdown_connections 2022-06-29T15:42:44.111 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.091+0000 7ff31ffff700 1 -- 172.21.15.47:0/3964278403 wait complete. 2022-06-29T15:42:44.111 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.091+0000 7ff31ffff700 1 Processor -- start 2022-06-29T15:42:44.112 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.091+0000 7ff31ffff700 1 -- start start 2022-06-29T15:42:44.112 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.091+0000 7ff31ffff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff300003c80 0x7ff32014b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:44.112 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.091+0000 7ff31ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3200acbb0 0x7ff3201417c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:44.112 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.091+0000 7ff31ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff320141cb0 0x7ff320142840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:44.112 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.091+0000 7ff31ffff700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3200ac7b0 con 0x7ff3200acbb0 2022-06-29T15:42:44.113 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.091+0000 7ff31ffff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff320134380 con 0x7ff320141cb0 2022-06-29T15:42:44.113 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.091+0000 7ff31ffff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff320135510 con 0x7ff300003c80 2022-06-29T15:42:44.113 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.091+0000 7ff31e7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3200acbb0 0x7ff3201417c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:44.113 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.095+0000 7ff31e7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3200acbb0 0x7ff3201417c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46184/0 (socket says 172.21.15.47:46184) 2022-06-29T15:42:44.114 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.095+0000 7ff31e7fc700 1 -- 172.21.15.47:0/4133442595 learned_addr learned my addr 172.21.15.47:0/4133442595 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:44.114 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.095+0000 7ff31effd700 1 --2- 172.21.15.47:0/4133442595 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff300003c80 0x7ff32014b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:44.114 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.095+0000 7ff31f7fe700 1 --2- 172.21.15.47:0/4133442595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff320141cb0 0x7ff320142840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:44.114 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.095+0000 7ff31e7fc700 1 -- 172.21.15.47:0/4133442595 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff300003c80 msgr2=0x7ff32014b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:44.115 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.095+0000 7ff31e7fc700 1 --2- 172.21.15.47:0/4133442595 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff300003c80 0x7ff32014b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:44.115 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.095+0000 7ff31e7fc700 1 -- 172.21.15.47:0/4133442595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff320141cb0 msgr2=0x7ff320142840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:44.115 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.095+0000 7ff31e7fc700 1 --2- 172.21.15.47:0/4133442595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff320141cb0 0x7ff320142840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:44.115 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.095+0000 7ff31e7fc700 1 -- 172.21.15.47:0/4133442595 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff320142130 con 0x7ff3200acbb0 2022-06-29T15:42:44.116 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.095+0000 7ff31f7fe700 1 --2- 172.21.15.47:0/4133442595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff320141cb0 0x7ff320142840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:42:44.116 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.095+0000 7ff31e7fc700 1 --2- 172.21.15.47:0/4133442595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3200acbb0 0x7ff3201417c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7ff3000054d0 tx=0x7ff308019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:44.116 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.095+0000 7ff317fff700 1 -- 172.21.15.47:0/4133442595 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff30801b820 con 0x7ff3200acbb0 2022-06-29T15:42:44.116 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.095+0000 7ff317fff700 1 -- 172.21.15.47:0/4133442595 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff308000e90 con 0x7ff3200acbb0 2022-06-29T15:42:44.116 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.095+0000 7ff31ffff700 1 -- 172.21.15.47:0/4133442595 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3201422c0 con 0x7ff3200acbb0 2022-06-29T15:42:44.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.095+0000 7ff317fff700 1 -- 172.21.15.47:0/4133442595 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff30801bb70 con 0x7ff3200acbb0 2022-06-29T15:42:44.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.095+0000 7ff31ffff700 1 -- 172.21.15.47:0/4133442595 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff320142ed0 con 0x7ff3200acbb0 2022-06-29T15:42:44.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.095+0000 7ff31ffff700 1 -- 172.21.15.47:0/4133442595 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3200573d0 con 0x7ff3200acbb0 2022-06-29T15:42:44.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.095+0000 7ff317fff700 1 -- 172.21.15.47:0/4133442595 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff30801bcd0 con 0x7ff3200acbb0 2022-06-29T15:42:44.118 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.095+0000 7ff317fff700 1 --2- 172.21.15.47:0/4133442595 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff304053a70 0x7ff304055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:44.119 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.099+0000 7ff317fff700 1 -- 172.21.15.47:0/4133442595 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7ff30806d650 con 0x7ff3200acbb0 2022-06-29T15:42:44.119 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.099+0000 7ff31effd700 1 --2- 172.21.15.47:0/4133442595 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff304053a70 0x7ff304055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:44.119 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.099+0000 7ff31effd700 1 --2- 172.21.15.47:0/4133442595 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff304053a70 0x7ff304055f30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7ff31000b000 tx=0x7ff310008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:44.119 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.103+0000 7ff317fff700 1 -- 172.21.15.47:0/4133442595 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff308044550 con 0x7ff3200acbb0 2022-06-29T15:42:44.236 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.231+0000 7ff31ffff700 1 -- 172.21.15.47:0/4133442595 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff320061f60 con 0x7ff3200acbb0 2022-06-29T15:42:44.237 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.235+0000 7ff317fff700 1 -- 172.21.15.47:0/4133442595 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff308047a30 con 0x7ff3200acbb0 2022-06-29T15:42:44.237 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:42:44.240 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.235+0000 7ff31ffff700 1 -- 172.21.15.47:0/4133442595 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff304053a70 msgr2=0x7ff304055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:44.240 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.235+0000 7ff31ffff700 1 --2- 172.21.15.47:0/4133442595 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff304053a70 0x7ff304055f30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7ff31000b000 tx=0x7ff310008040).stop 2022-06-29T15:42:44.241 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.235+0000 7ff31ffff700 1 -- 172.21.15.47:0/4133442595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3200acbb0 msgr2=0x7ff3201417c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:44.241 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.235+0000 7ff31ffff700 1 --2- 172.21.15.47:0/4133442595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3200acbb0 0x7ff3201417c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7ff3000054d0 tx=0x7ff308019000).stop 2022-06-29T15:42:44.241 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.235+0000 7ff31ffff700 1 -- 172.21.15.47:0/4133442595 shutdown_connections 2022-06-29T15:42:44.242 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.235+0000 7ff31ffff700 1 --2- 172.21.15.47:0/4133442595 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff304053a70 0x7ff304055f30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:44.242 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.235+0000 7ff31ffff700 1 --2- 172.21.15.47:0/4133442595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff320141cb0 0x7ff320142840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:44.242 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.235+0000 7ff31ffff700 1 --2- 172.21.15.47:0/4133442595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3200acbb0 0x7ff3201417c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:44.242 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.235+0000 7ff31ffff700 1 --2- 172.21.15.47:0/4133442595 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff300003c80 0x7ff32014b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:44.243 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.235+0000 7ff31ffff700 1 -- 172.21.15.47:0/4133442595 >> 172.21.15.47:0/4133442595 conn(0x7ff320062560 msgr2=0x7ff3200601a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:44.243 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.235+0000 7ff31ffff700 1 -- 172.21.15.47:0/4133442595 shutdown_connections 2022-06-29T15:42:44.243 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.235+0000 7ff31ffff700 1 -- 172.21.15.47:0/4133442595 wait complete. 2022-06-29T15:42:44.255 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-29T15:42:44.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.359+0000 7f5410989700 1 Processor -- start 2022-06-29T15:42:44.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.359+0000 7f5410989700 1 -- start start 2022-06-29T15:42:44.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.363+0000 7f5410989700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f540c0b5d90 0x7f540c143a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:44.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.363+0000 7f5410989700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f540c0676a0 0x7f540c140130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:44.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.363+0000 7f5410989700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f540c05cc30 con 0x7f540c05c850 2022-06-29T15:42:44.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.363+0000 7f5410989700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f540c1430a0 con 0x7f540c0b5d90 2022-06-29T15:42:44.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.363+0000 7f5410989700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f540c143300 con 0x7f540c0676a0 2022-06-29T15:42:44.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.363+0000 7f540ad9d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f540c05c850 0x7f540c0b4860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59976/0 (socket says 172.21.15.47:59976) 2022-06-29T15:42:44.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.363+0000 7f540a59c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f540c0b5d90 0x7f540c143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:44.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.363+0000 7f540ad9d700 1 -- 172.21.15.47:0/1186662401 learned_addr learned my addr 172.21.15.47:0/1186662401 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:44.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.363+0000 7f540b59e700 1 --2- 172.21.15.47:0/1186662401 >> v2:172.21.15.64:3300/0 conn(0x7f540c0676a0 0x7f540c140130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:44.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.363+0000 7f5409d9b700 1 -- 172.21.15.47:0/1186662401 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3262638730 0 0) 0x7f540c05cc30 con 0x7f540c05c850 2022-06-29T15:42:44.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.363+0000 7f5409d9b700 1 -- 172.21.15.47:0/1186662401 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53f0003260 con 0x7f540c05c850 2022-06-29T15:42:44.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.363+0000 7f540a59c700 1 -- 172.21.15.47:0/1186662401 >> v2:172.21.15.64:3300/0 conn(0x7f540c0676a0 msgr2=0x7f540c140130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:44.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.363+0000 7f540a59c700 1 --2- 172.21.15.47:0/1186662401 >> v2:172.21.15.64:3300/0 conn(0x7f540c0676a0 0x7f540c140130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:44.370 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.363+0000 7f540a59c700 1 -- 172.21.15.47:0/1186662401 >> v1:172.21.15.47:6789/0 conn(0x7f540c05c850 legacy=0x7f540c0b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:44.370 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.363+0000 7f540a59c700 1 -- 172.21.15.47:0/1186662401 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f540c1435a0 con 0x7f540c0b5d90 2022-06-29T15:42:44.370 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.363+0000 7f540a59c700 1 --2- 172.21.15.47:0/1186662401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f540c0b5d90 0x7f540c143a40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f5400002f70 tx=0x7f5400006df0).ready entity=mon.0 client_cookie=2753b7a9352f3d7d server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:44.370 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.363+0000 7f5409d9b700 1 -- 172.21.15.47:0/1186662401 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f540000d4f0 con 0x7f540c0b5d90 2022-06-29T15:42:44.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.363+0000 7f5409d9b700 1 -- 172.21.15.47:0/1186662401 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5400000e90 con 0x7f540c0b5d90 2022-06-29T15:42:44.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.363+0000 7f5409d9b700 1 -- 172.21.15.47:0/1186662401 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f540000d840 con 0x7f540c0b5d90 2022-06-29T15:42:44.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.363+0000 7f5410989700 1 -- 172.21.15.47:0/1186662401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f540c0b5d90 msgr2=0x7f540c143a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:44.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.363+0000 7f5410989700 1 --2- 172.21.15.47:0/1186662401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f540c0b5d90 0x7f540c143a40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f5400002f70 tx=0x7f5400006df0).stop 2022-06-29T15:42:44.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.363+0000 7f5410989700 1 -- 172.21.15.47:0/1186662401 shutdown_connections 2022-06-29T15:42:44.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.363+0000 7f5410989700 1 --2- 172.21.15.47:0/1186662401 >> v2:172.21.15.64:3300/0 conn(0x7f540c0676a0 0x7f540c140130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:44.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.363+0000 7f5410989700 1 --2- 172.21.15.47:0/1186662401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f540c0b5d90 0x7f540c143a40 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:44.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.363+0000 7f5410989700 1 -- 172.21.15.47:0/1186662401 >> 172.21.15.47:0/1186662401 conn(0x7f540c060e00 msgr2=0x7f540c0611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:44.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.363+0000 7f5410989700 1 -- 172.21.15.47:0/1186662401 shutdown_connections 2022-06-29T15:42:44.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.363+0000 7f5410989700 1 -- 172.21.15.47:0/1186662401 wait complete. 2022-06-29T15:42:44.379 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.367+0000 7f5410989700 1 Processor -- start 2022-06-29T15:42:44.379 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.367+0000 7f5410989700 1 -- start start 2022-06-29T15:42:44.379 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.367+0000 7f5410989700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f540c05c850 0x7f540c0af240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:44.379 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.367+0000 7f5410989700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f540c0676a0 0x7f540c12bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:44.380 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.367+0000 7f5410989700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f540c0b5d90 0x7f540c12af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:44.380 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.367+0000 7f5410989700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f540c12c2b0 con 0x7f540c0b5d90 2022-06-29T15:42:44.380 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.367+0000 7f5410989700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f540c0b39d0 con 0x7f540c05c850 2022-06-29T15:42:44.380 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.367+0000 7f5410989700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f540c0b2ca0 con 0x7f540c0676a0 2022-06-29T15:42:44.380 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.367+0000 7f540b59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f540c0b5d90 0x7f540c12af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:44.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.367+0000 7f540b59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f540c0b5d90 0x7f540c12af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46200/0 (socket says 172.21.15.47:46200) 2022-06-29T15:42:44.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.367+0000 7f540b59e700 1 -- 172.21.15.47:0/2862866461 learned_addr learned my addr 172.21.15.47:0/2862866461 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:44.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.367+0000 7f540ad9d700 1 --2- 172.21.15.47:0/2862866461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f540c05c850 0x7f540c0af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:44.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.367+0000 7f540a59c700 1 --2- 172.21.15.47:0/2862866461 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f540c0676a0 0x7f540c12bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:44.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.367+0000 7f540b59e700 1 -- 172.21.15.47:0/2862866461 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f540c0676a0 msgr2=0x7f540c12bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:44.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.367+0000 7f540b59e700 1 --2- 172.21.15.47:0/2862866461 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f540c0676a0 0x7f540c12bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:44.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.367+0000 7f540b59e700 1 -- 172.21.15.47:0/2862866461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f540c05c850 msgr2=0x7f540c0af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:44.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.367+0000 7f540b59e700 1 --2- 172.21.15.47:0/2862866461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f540c05c850 0x7f540c0af240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:44.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.371+0000 7f540b59e700 1 -- 172.21.15.47:0/2862866461 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f540000b180 con 0x7f540c0b5d90 2022-06-29T15:42:44.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.371+0000 7f540a59c700 1 --2- 172.21.15.47:0/2862866461 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f540c0676a0 0x7f540c12bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:42:44.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.371+0000 7f540b59e700 1 --2- 172.21.15.47:0/2862866461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f540c0b5d90 0x7f540c12af50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f5404006e00 tx=0x7f54040091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:44.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.371+0000 7f53fbfff700 1 -- 172.21.15.47:0/2862866461 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f540400eeb0 con 0x7f540c0b5d90 2022-06-29T15:42:44.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.371+0000 7f53fbfff700 1 -- 172.21.15.47:0/2862866461 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f540400c850 con 0x7f540c0b5d90 2022-06-29T15:42:44.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.371+0000 7f53fbfff700 1 -- 172.21.15.47:0/2862866461 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f540400cbb0 con 0x7f540c0b5d90 2022-06-29T15:42:44.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.371+0000 7f5410989700 1 -- 172.21.15.47:0/2862866461 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f540c0b8950 con 0x7f540c0b5d90 2022-06-29T15:42:44.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.371+0000 7f5410989700 1 -- 172.21.15.47:0/2862866461 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f540c0b8540 con 0x7f540c0b5d90 2022-06-29T15:42:44.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.371+0000 7f5410989700 1 -- 172.21.15.47:0/2862866461 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f540c0b7550 con 0x7f540c0b5d90 2022-06-29T15:42:44.385 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.371+0000 7f53fbfff700 1 -- 172.21.15.47:0/2862866461 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f540400cd10 con 0x7f540c0b5d90 2022-06-29T15:42:44.385 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.371+0000 7f53fbfff700 1 --2- 172.21.15.47:0/2862866461 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f53e8053b00 0x7f53e8055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:44.385 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.371+0000 7f53fbfff700 1 -- 172.21.15.47:0/2862866461 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f540406c430 con 0x7f540c0b5d90 2022-06-29T15:42:44.385 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.371+0000 7f540ad9d700 1 --2- 172.21.15.47:0/2862866461 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f53e8053b00 0x7f53e8055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:44.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.371+0000 7f540ad9d700 1 --2- 172.21.15.47:0/2862866461 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f53e8053b00 0x7f53e8055fc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f53fc00d000 tx=0x7f53fc009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:44.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.379+0000 7f53fbfff700 1 -- 172.21.15.47:0/2862866461 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f54040430e0 con 0x7f540c0b5d90 2022-06-29T15:42:44.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.507+0000 7f5410989700 1 -- 172.21.15.47:0/2862866461 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f540c0573d0 con 0x7f540c0b5d90 2022-06-29T15:42:44.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.507+0000 7f53fbfff700 1 -- 172.21.15.47:0/2862866461 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+10 (secure 0 0 0) 0x7f5404046700 con 0x7f540c0b5d90 2022-06-29T15:42:44.510 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:42:44.513 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.507+0000 7f5410989700 1 -- 172.21.15.47:0/2862866461 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f53e8053b00 msgr2=0x7f53e8055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:44.514 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.507+0000 7f5410989700 1 --2- 172.21.15.47:0/2862866461 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f53e8053b00 0x7f53e8055fc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f53fc00d000 tx=0x7f53fc009040).stop 2022-06-29T15:42:44.514 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.507+0000 7f5410989700 1 -- 172.21.15.47:0/2862866461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f540c0b5d90 msgr2=0x7f540c12af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:44.514 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.507+0000 7f5410989700 1 --2- 172.21.15.47:0/2862866461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f540c0b5d90 0x7f540c12af50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f5404006e00 tx=0x7f54040091d0).stop 2022-06-29T15:42:44.514 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.511+0000 7f5410989700 1 -- 172.21.15.47:0/2862866461 shutdown_connections 2022-06-29T15:42:44.515 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.511+0000 7f5410989700 1 --2- 172.21.15.47:0/2862866461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f540c0b5d90 0x7f540c12af50 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:44.515 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.511+0000 7f5410989700 1 --2- 172.21.15.47:0/2862866461 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f540c0676a0 0x7f540c12bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:44.515 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.511+0000 7f5410989700 1 --2- 172.21.15.47:0/2862866461 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f53e8053b00 0x7f53e8055fc0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:44.515 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.511+0000 7f5410989700 1 --2- 172.21.15.47:0/2862866461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f540c05c850 0x7f540c0af240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:44.516 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.511+0000 7f5410989700 1 -- 172.21.15.47:0/2862866461 >> 172.21.15.47:0/2862866461 conn(0x7f540c060e00 msgr2=0x7f540c0b0010 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:44.516 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.511+0000 7f5410989700 1 -- 172.21.15.47:0/2862866461 shutdown_connections 2022-06-29T15:42:44.516 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.511+0000 7f5410989700 1 -- 172.21.15.47:0/2862866461 wait complete. 2022-06-29T15:42:44.527 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-29T15:42:44.634 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.631+0000 7f503fda4700 1 Processor -- start 2022-06-29T15:42:44.635 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.631+0000 7f503fda4700 1 -- start start 2022-06-29T15:42:44.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.631+0000 7f503fda4700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5038067470 0x7f5038067850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:44.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.631+0000 7f503fda4700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f50380609d0 0x7f5038143e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:44.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.631+0000 7f503fda4700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50381434e0 con 0x7f5038140700 2022-06-29T15:42:44.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.631+0000 7f503fda4700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5038144370 con 0x7f5038067470 2022-06-29T15:42:44.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.631+0000 7f503fda4700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5038143740 con 0x7f50380609d0 2022-06-29T15:42:44.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.631+0000 7f503db40700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5038067470 0x7f5038067850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:44.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.631+0000 7f503d33f700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f50380609d0 0x7f5038143e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:44.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.631+0000 7f503e341700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f5038140700 0x7f50380b2930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:39326/0 (socket says 172.21.15.47:39326) 2022-06-29T15:42:44.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.631+0000 7f503e341700 1 -- 172.21.15.47:0/1365096052 learned_addr learned my addr 172.21.15.47:0/1365096052 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:44.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.631+0000 7f503d33f700 1 -- 172.21.15.47:0/1365096052 >> v2:172.21.15.64:3300/0 conn(0x7f5038067470 msgr2=0x7f5038067850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:44.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.631+0000 7f503d33f700 1 --2- 172.21.15.47:0/1365096052 >> v2:172.21.15.64:3300/0 conn(0x7f5038067470 0x7f5038067850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:44.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.631+0000 7f503d33f700 1 -- 172.21.15.47:0/1365096052 >> v1:172.21.15.64:6789/0 conn(0x7f5038140700 legacy=0x7f50380b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:44.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.631+0000 7f503d33f700 1 -- 172.21.15.47:0/1365096052 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50381439e0 con 0x7f50380609d0 2022-06-29T15:42:44.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.631+0000 7f503db40700 1 --2- 172.21.15.47:0/1365096052 >> v2:172.21.15.64:3300/0 conn(0x7f5038067470 0x7f5038067850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:42:44.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.635+0000 7f503d33f700 1 --2- 172.21.15.47:0/1365096052 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f50380609d0 0x7f5038143e80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f5034002f70 tx=0x7f5034006df0).ready entity=mon.2 client_cookie=e9dae05d9a18758a server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:44.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.635+0000 7f503cb3e700 1 -- 172.21.15.47:0/1365096052 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f503400d570 con 0x7f50380609d0 2022-06-29T15:42:44.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.635+0000 7f503cb3e700 1 -- 172.21.15.47:0/1365096052 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5034000e90 con 0x7f50380609d0 2022-06-29T15:42:44.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.635+0000 7f503cb3e700 1 -- 172.21.15.47:0/1365096052 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f503400d8c0 con 0x7f50380609d0 2022-06-29T15:42:44.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.635+0000 7f503fda4700 1 -- 172.21.15.47:0/1365096052 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f50380609d0 msgr2=0x7f5038143e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:44.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.635+0000 7f503fda4700 1 --2- 172.21.15.47:0/1365096052 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f50380609d0 0x7f5038143e80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f5034002f70 tx=0x7f5034006df0).stop 2022-06-29T15:42:44.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.635+0000 7f503fda4700 1 -- 172.21.15.47:0/1365096052 shutdown_connections 2022-06-29T15:42:44.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.635+0000 7f503fda4700 1 --2- 172.21.15.47:0/1365096052 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f50380609d0 0x7f5038143e80 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:44.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.635+0000 7f503fda4700 1 --2- 172.21.15.47:0/1365096052 >> v2:172.21.15.64:3300/0 conn(0x7f5038067470 0x7f5038067850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:44.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.635+0000 7f503fda4700 1 -- 172.21.15.47:0/1365096052 >> 172.21.15.47:0/1365096052 conn(0x7f503805f0b0 msgr2=0x7f503805f490 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:44.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.635+0000 7f503fda4700 1 -- 172.21.15.47:0/1365096052 shutdown_connections 2022-06-29T15:42:44.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.635+0000 7f503fda4700 1 -- 172.21.15.47:0/1365096052 wait complete. 2022-06-29T15:42:44.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.635+0000 7f503fda4700 1 Processor -- start 2022-06-29T15:42:44.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.635+0000 7f503fda4700 1 -- start start 2022-06-29T15:42:44.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.635+0000 7f503fda4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50380609d0 0x7f5038148ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:44.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.635+0000 7f503fda4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5038067470 0x7f50381523f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:44.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.635+0000 7f503fda4700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5038140700 0x7f503809b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:44.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.635+0000 7f503fda4700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5038099f90 con 0x7f5038067470 2022-06-29T15:42:44.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.635+0000 7f503fda4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f503809ab00 con 0x7f50380609d0 2022-06-29T15:42:44.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.635+0000 7f503fda4700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50381468c0 con 0x7f5038140700 2022-06-29T15:42:44.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.635+0000 7f503d33f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5038067470 0x7f50381523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:44.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.635+0000 7f503d33f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5038067470 0x7f50381523f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46212/0 (socket says 172.21.15.47:46212) 2022-06-29T15:42:44.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.635+0000 7f503d33f700 1 -- 172.21.15.47:0/2359282504 learned_addr learned my addr 172.21.15.47:0/2359282504 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:44.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.635+0000 7f503db40700 1 --2- 172.21.15.47:0/2359282504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50380609d0 0x7f5038148ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:44.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.635+0000 7f503e341700 1 --2- 172.21.15.47:0/2359282504 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5038140700 0x7f503809b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:44.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.635+0000 7f503d33f700 1 -- 172.21.15.47:0/2359282504 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5038140700 msgr2=0x7f503809b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:44.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.635+0000 7f503d33f700 1 --2- 172.21.15.47:0/2359282504 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5038140700 0x7f503809b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:44.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.635+0000 7f503d33f700 1 -- 172.21.15.47:0/2359282504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50380609d0 msgr2=0x7f5038148ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:44.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.635+0000 7f503d33f700 1 --2- 172.21.15.47:0/2359282504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50380609d0 0x7f5038148ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:44.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.635+0000 7f503d33f700 1 -- 172.21.15.47:0/2359282504 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f502800b180 con 0x7f5038067470 2022-06-29T15:42:44.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.635+0000 7f503d33f700 1 --2- 172.21.15.47:0/2359282504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5038067470 0x7f50381523f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f5034002f70 tx=0x7f5034014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:44.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.635+0000 7f5026ffd700 1 -- 172.21.15.47:0/2359282504 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f503400dea0 con 0x7f5038067470 2022-06-29T15:42:44.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.635+0000 7f5026ffd700 1 -- 172.21.15.47:0/2359282504 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f503401e400 con 0x7f5038067470 2022-06-29T15:42:44.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.635+0000 7f503fda4700 1 -- 172.21.15.47:0/2359282504 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f503400b180 con 0x7f5038067470 2022-06-29T15:42:44.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.635+0000 7f5026ffd700 1 -- 172.21.15.47:0/2359282504 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f503401e760 con 0x7f5038067470 2022-06-29T15:42:44.651 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.639+0000 7f503fda4700 1 -- 172.21.15.47:0/2359282504 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5038146b60 con 0x7f5038067470 2022-06-29T15:42:44.652 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.639+0000 7f5026ffd700 1 -- 172.21.15.47:0/2359282504 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5034025030 con 0x7f5038067470 2022-06-29T15:42:44.652 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.639+0000 7f503fda4700 1 -- 172.21.15.47:0/2359282504 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50380573d0 con 0x7f5038067470 2022-06-29T15:42:44.652 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.639+0000 7f5026ffd700 1 --2- 172.21.15.47:0/2359282504 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5018053b00 0x7f5018055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:44.653 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.639+0000 7f5026ffd700 1 -- 172.21.15.47:0/2359282504 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f503406cdf0 con 0x7f5038067470 2022-06-29T15:42:44.653 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.639+0000 7f503db40700 1 --2- 172.21.15.47:0/2359282504 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5018053b00 0x7f5018055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:44.653 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.639+0000 7f503db40700 1 --2- 172.21.15.47:0/2359282504 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5018053b00 0x7f5018055fc0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f502800b000 tx=0x7f5028008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:44.653 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.647+0000 7f5026ffd700 1 -- 172.21.15.47:0/2359282504 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5034043bb0 con 0x7f5038067470 2022-06-29T15:42:44.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.775+0000 7f503fda4700 1 -- 172.21.15.47:0/2359282504 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5038146320 con 0x7f5038067470 2022-06-29T15:42:44.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.775+0000 7f5026ffd700 1 -- 172.21.15.47:0/2359282504 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+10 (secure 0 0 0) 0x7f5034023020 con 0x7f5038067470 2022-06-29T15:42:44.780 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:42:44.783 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.779+0000 7f503fda4700 1 -- 172.21.15.47:0/2359282504 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5018053b00 msgr2=0x7f5018055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:44.783 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.779+0000 7f503fda4700 1 --2- 172.21.15.47:0/2359282504 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5018053b00 0x7f5018055fc0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f502800b000 tx=0x7f5028008040).stop 2022-06-29T15:42:44.783 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.779+0000 7f503fda4700 1 -- 172.21.15.47:0/2359282504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5038067470 msgr2=0x7f50381523f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:44.783 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.779+0000 7f503fda4700 1 --2- 172.21.15.47:0/2359282504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5038067470 0x7f50381523f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f5034002f70 tx=0x7f5034014000).stop 2022-06-29T15:42:44.784 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.779+0000 7f503fda4700 1 -- 172.21.15.47:0/2359282504 shutdown_connections 2022-06-29T15:42:44.784 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.779+0000 7f503fda4700 1 --2- 172.21.15.47:0/2359282504 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5038140700 0x7f503809b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:44.784 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.779+0000 7f503fda4700 1 --2- 172.21.15.47:0/2359282504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5038067470 0x7f50381523f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:44.784 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.779+0000 7f503fda4700 1 --2- 172.21.15.47:0/2359282504 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5018053b00 0x7f5018055fc0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:44.784 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.779+0000 7f503fda4700 1 --2- 172.21.15.47:0/2359282504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50380609d0 0x7f5038148ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:44.785 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.779+0000 7f503fda4700 1 -- 172.21.15.47:0/2359282504 >> 172.21.15.47:0/2359282504 conn(0x7f503805f0b0 msgr2=0x7f50380b5ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:44.785 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.779+0000 7f503fda4700 1 -- 172.21.15.47:0/2359282504 shutdown_connections 2022-06-29T15:42:44.785 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.779+0000 7f503fda4700 1 -- 172.21.15.47:0/2359282504 wait complete. 2022-06-29T15:42:44.793 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:42:44.896 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.891+0000 7ff5a9031700 1 Processor -- start 2022-06-29T15:42:44.896 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.891+0000 7ff5a9031700 1 -- start start 2022-06-29T15:42:44.900 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.891+0000 7ff5a9031700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff5a40b2d00 0x7ff5a40b3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:44.900 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.891+0000 7ff5a9031700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff5a414a710 0x7ff5a414ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:44.900 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.891+0000 7ff5a9031700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5a40aac20 con 0x7ff5a40b7860 2022-06-29T15:42:44.901 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.891+0000 7ff5a9031700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5a40ab2f0 con 0x7ff5a40b2d00 2022-06-29T15:42:44.901 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.891+0000 7ff5a9031700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5a40af9f0 con 0x7ff5a414a710 2022-06-29T15:42:44.901 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.895+0000 7ff5a37fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff5a40b2d00 0x7ff5a40b3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:44.901 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.895+0000 7ff5a3fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff5a40b7860 0x7ff5a414a000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:60006/0 (socket says 172.21.15.47:60006) 2022-06-29T15:42:44.902 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.895+0000 7ff5a3fff700 1 -- 172.21.15.47:0/1163255762 learned_addr learned my addr 172.21.15.47:0/1163255762 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:44.902 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.895+0000 7ff5a8830700 1 --2- 172.21.15.47:0/1163255762 >> v2:172.21.15.64:3300/0 conn(0x7ff5a414a710 0x7ff5a414ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:44.902 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.895+0000 7ff5a37fe700 1 -- 172.21.15.47:0/1163255762 >> v2:172.21.15.64:3300/0 conn(0x7ff5a414a710 msgr2=0x7ff5a414ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:44.903 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.895+0000 7ff5a37fe700 1 --2- 172.21.15.47:0/1163255762 >> v2:172.21.15.64:3300/0 conn(0x7ff5a414a710 0x7ff5a414ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:44.903 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.895+0000 7ff5a37fe700 1 -- 172.21.15.47:0/1163255762 >> v1:172.21.15.47:6789/0 conn(0x7ff5a40b7860 legacy=0x7ff5a414a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:44.903 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.895+0000 7ff5a2ffd700 1 -- 172.21.15.47:0/1163255762 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4164966633 0 0) 0x7ff5a40aac20 con 0x7ff5a40b7860 2022-06-29T15:42:44.903 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.895+0000 7ff5a37fe700 1 -- 172.21.15.47:0/1163255762 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a40b7c40 con 0x7ff5a40b2d00 2022-06-29T15:42:44.903 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.895+0000 7ff5a37fe700 1 --2- 172.21.15.47:0/1163255762 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5a40b2d00 0x7ff5a40b3450 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7ff5a40afb30 tx=0x7ff594006df0).ready entity=mon.0 client_cookie=b2564cb03632fefe server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:44.904 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.895+0000 7ff5a2ffd700 1 -- 172.21.15.47:0/1163255762 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff59400d5c0 con 0x7ff5a40b2d00 2022-06-29T15:42:44.904 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.895+0000 7ff5a2ffd700 1 -- 172.21.15.47:0/1163255762 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff594000e90 con 0x7ff5a40b2d00 2022-06-29T15:42:44.904 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.895+0000 7ff5a2ffd700 1 -- 172.21.15.47:0/1163255762 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff59400d910 con 0x7ff5a40b2d00 2022-06-29T15:42:44.904 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.895+0000 7ff5a9031700 1 -- 172.21.15.47:0/1163255762 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5a40b2d00 msgr2=0x7ff5a40b3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:44.905 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.895+0000 7ff5a9031700 1 --2- 172.21.15.47:0/1163255762 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5a40b2d00 0x7ff5a40b3450 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7ff5a40afb30 tx=0x7ff594006df0).stop 2022-06-29T15:42:44.905 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.895+0000 7ff5a9031700 1 -- 172.21.15.47:0/1163255762 shutdown_connections 2022-06-29T15:42:44.905 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.895+0000 7ff5a9031700 1 --2- 172.21.15.47:0/1163255762 >> v2:172.21.15.64:3300/0 conn(0x7ff5a414a710 0x7ff5a414ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:44.905 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.895+0000 7ff5a9031700 1 --2- 172.21.15.47:0/1163255762 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5a40b2d00 0x7ff5a40b3450 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:44.906 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.895+0000 7ff5a9031700 1 -- 172.21.15.47:0/1163255762 >> 172.21.15.47:0/1163255762 conn(0x7ff5a405f480 msgr2=0x7ff5a40647d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:44.906 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.895+0000 7ff5a9031700 1 -- 172.21.15.47:0/1163255762 shutdown_connections 2022-06-29T15:42:44.906 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.895+0000 7ff5a9031700 1 -- 172.21.15.47:0/1163255762 wait complete. 2022-06-29T15:42:44.910 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.895+0000 7ff5a9031700 1 Processor -- start 2022-06-29T15:42:44.910 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.895+0000 7ff5a9031700 1 -- start start 2022-06-29T15:42:44.910 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.899+0000 7ff5a9031700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5a40b2d00 0x7ff5a4146740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:44.911 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.899+0000 7ff5a9031700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5a40b7860 0x7ff5a41436f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:44.911 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.899+0000 7ff5a9031700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5a414a710 0x7ff5a4143be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:44.911 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.899+0000 7ff5a9031700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a41440d0 con 0x7ff5a414a710 2022-06-29T15:42:44.911 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.899+0000 7ff5a9031700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a4144480 con 0x7ff5a40b2d00 2022-06-29T15:42:44.912 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.899+0000 7ff5a9031700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a414d720 con 0x7ff5a40b7860 2022-06-29T15:42:44.912 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.899+0000 7ff5a8830700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5a414a710 0x7ff5a4143be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:44.912 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.899+0000 7ff5a8830700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5a414a710 0x7ff5a4143be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46228/0 (socket says 172.21.15.47:46228) 2022-06-29T15:42:44.912 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.899+0000 7ff5a8830700 1 -- 172.21.15.47:0/612431562 learned_addr learned my addr 172.21.15.47:0/612431562 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:44.913 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.899+0000 7ff5a3fff700 1 --2- 172.21.15.47:0/612431562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5a40b2d00 0x7ff5a4146740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:44.913 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.899+0000 7ff5a37fe700 1 --2- 172.21.15.47:0/612431562 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5a40b7860 0x7ff5a41436f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:44.913 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.899+0000 7ff5a8830700 1 -- 172.21.15.47:0/612431562 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5a40b7860 msgr2=0x7ff5a41436f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:44.913 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.899+0000 7ff5a8830700 1 --2- 172.21.15.47:0/612431562 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5a40b7860 0x7ff5a41436f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:44.914 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.899+0000 7ff5a8830700 1 -- 172.21.15.47:0/612431562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5a40b2d00 msgr2=0x7ff5a4146740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:44.914 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.899+0000 7ff5a8830700 1 --2- 172.21.15.47:0/612431562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5a40b2d00 0x7ff5a4146740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:44.914 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.899+0000 7ff5a8830700 1 -- 172.21.15.47:0/612431562 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff59400b180 con 0x7ff5a414a710 2022-06-29T15:42:44.914 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.899+0000 7ff5a8830700 1 --2- 172.21.15.47:0/612431562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5a414a710 0x7ff5a4143be0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7ff59c006e00 tx=0x7ff59c0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:44.914 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.899+0000 7ff5a17fa700 1 -- 172.21.15.47:0/612431562 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff59c012600 con 0x7ff5a414a710 2022-06-29T15:42:44.915 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.899+0000 7ff5a17fa700 1 -- 172.21.15.47:0/612431562 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff59c012760 con 0x7ff5a414a710 2022-06-29T15:42:44.915 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.899+0000 7ff5a17fa700 1 -- 172.21.15.47:0/612431562 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff59c012ac0 con 0x7ff5a414a710 2022-06-29T15:42:44.915 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.899+0000 7ff5a9031700 1 -- 172.21.15.47:0/612431562 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5a4154500 con 0x7ff5a414a710 2022-06-29T15:42:44.915 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.899+0000 7ff5a9031700 1 -- 172.21.15.47:0/612431562 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5a4147170 con 0x7ff5a414a710 2022-06-29T15:42:44.916 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.899+0000 7ff5a9031700 1 -- 172.21.15.47:0/612431562 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5a4066880 con 0x7ff5a414a710 2022-06-29T15:42:44.916 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.903+0000 7ff5a17fa700 1 -- 172.21.15.47:0/612431562 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff59c022070 con 0x7ff5a414a710 2022-06-29T15:42:44.916 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.903+0000 7ff5a17fa700 1 --2- 172.21.15.47:0/612431562 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff584053980 0x7ff584055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:44.916 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.903+0000 7ff5a17fa700 1 -- 172.21.15.47:0/612431562 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7ff59c06c620 con 0x7ff5a414a710 2022-06-29T15:42:44.917 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.903+0000 7ff5a3fff700 1 --2- 172.21.15.47:0/612431562 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff584053980 0x7ff584055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:44.917 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.903+0000 7ff5a3fff700 1 --2- 172.21.15.47:0/612431562 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff584053980 0x7ff584055e40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7ff59800d000 tx=0x7ff598007a00).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:44.917 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:44.907+0000 7ff5a17fa700 1 -- 172.21.15.47:0/612431562 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff5a4066880 con 0x7ff5a414a710 2022-06-29T15:42:45.078 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.075+0000 7ff5a9031700 1 -- 172.21.15.47:0/612431562 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff5a4066880 con 0x7ff5a414a710 2022-06-29T15:42:45.079 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.075+0000 7ff5a17fa700 1 -- 172.21.15.47:0/612431562 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+2483 (secure 0 0 0) 0x7ff59c0169d0 con 0x7ff5a414a710 2022-06-29T15:42:45.079 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:42:45.080 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/438289599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":438289599},{"type":"v1","addr":"172.21.15.47:6835","nonce":438289599}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.64:6833/1472525731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1472525731},{"type":"v1","addr":"172.21.15.64:6833","nonce":1472525731}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":4}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:42:33.471491+0000","modified":"2022-06-29T15:42:35.146757+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":54128},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54128":{"gid":54128,"name":"a","rank":0,"incarnation":5,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6837/354581778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":354581778},{"type":"v1","addr":"172.21.15.47:6837","nonce":354581778}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2022-06-29T15:42:45.082 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.079+0000 7ff5a9031700 1 -- 172.21.15.47:0/612431562 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff584053980 msgr2=0x7ff584055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:45.082 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.079+0000 7ff5a9031700 1 --2- 172.21.15.47:0/612431562 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff584053980 0x7ff584055e40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7ff59800d000 tx=0x7ff598007a00).stop 2022-06-29T15:42:45.082 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.079+0000 7ff5a9031700 1 -- 172.21.15.47:0/612431562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5a414a710 msgr2=0x7ff5a4143be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:45.082 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.079+0000 7ff5a9031700 1 --2- 172.21.15.47:0/612431562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5a414a710 0x7ff5a4143be0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7ff59c006e00 tx=0x7ff59c0091d0).stop 2022-06-29T15:42:45.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.079+0000 7ff5a9031700 1 -- 172.21.15.47:0/612431562 shutdown_connections 2022-06-29T15:42:45.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.079+0000 7ff5a9031700 1 --2- 172.21.15.47:0/612431562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5a414a710 0x7ff5a4143be0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:45.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.079+0000 7ff5a9031700 1 --2- 172.21.15.47:0/612431562 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5a40b7860 0x7ff5a41436f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:45.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.079+0000 7ff5a9031700 1 --2- 172.21.15.47:0/612431562 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff584053980 0x7ff584055e40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:45.084 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.079+0000 7ff5a9031700 1 --2- 172.21.15.47:0/612431562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5a40b2d00 0x7ff5a4146740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:45.084 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.079+0000 7ff5a9031700 1 -- 172.21.15.47:0/612431562 >> 172.21.15.47:0/612431562 conn(0x7ff5a405f480 msgr2=0x7ff5a40aede0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:45.084 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.079+0000 7ff5a9031700 1 -- 172.21.15.47:0/612431562 shutdown_connections 2022-06-29T15:42:45.084 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.079+0000 7ff5a9031700 1 -- 172.21.15.47:0/612431562 wait complete. 2022-06-29T15:42:45.085 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 7 2022-06-29T15:42:45.395 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-29T15:42:33.471491+0000', 'modified': '2022-06-29T15:42:35.146757+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 54128}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_54128': {'gid': 54128, 'name': 'a', 'rank': 0, 'incarnation': 5, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.47:6837/354581778', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6836', 'nonce': 354581778}, {'type': 'v1', 'addr': '172.21.15.47:6837', 'nonce': 354581778}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-29T15:42:45.396 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-29T15:42:45.397 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-06-29T15:42:45.513 INFO:teuthology.orchestra.run.smithi047.stdout:{"cluster_fsid":"9a68894a-53ce-400e-838b-14eab91b834c","whoami":0,"id":54128,"want_state":"up:active","state":"up:active","rank_uptime":12.026164614000001,"mdsmap_epoch":7,"osdmap_epoch":26,"osdmap_epoch_barrier":26,"uptime":18.402741298999999} 2022-06-29T15:42:45.514 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9a68894a-53ce-400e-838b-14eab91b834c", "id": 54128, "mdsmap_epoch": 7, "osdmap_epoch": 26, "osdmap_epoch_barrier": 26, "rank_uptime": 12.026164614, "state": "up:active", "uptime": 18.402741299, "want_state": "up:active", "whoami": 0 } 2022-06-29T15:42:45.514 INFO:teuthology.run_tasks:Running task ceph-fuse... 2022-06-29T15:42:45.533 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2022-06-29T15:42:45.534 INFO:tasks.ceph_fuse:config is None 2022-06-29T15:42:45.534 INFO:tasks.ceph_fuse:new config is {'client.0': None, 'client.1': None} 2022-06-29T15:42:45.535 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2022-06-29T15:42:45.535 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:42:45.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.639+0000 7f85b64b7700 1 Processor -- start 2022-06-29T15:42:45.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.639+0000 7f85b64b7700 1 -- start start 2022-06-29T15:42:45.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.643+0000 7f85b64b7700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f85b005c510 0x7f85b0143bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:45.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.643+0000 7f85b64b7700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f85b005c8f0 0x7f85b00b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:45.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.643+0000 7f85b64b7700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85b0142f90 con 0x7f85b00b2c40 2022-06-29T15:42:45.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.643+0000 7f85b64b7700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85b01431f0 con 0x7f85b005c8f0 2022-06-29T15:42:45.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.643+0000 7f85b64b7700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85b0143490 con 0x7f85b005c510 2022-06-29T15:42:45.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.643+0000 7f85b4cb4700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f85b005c8f0 0x7f85b00b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:45.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.643+0000 7f85b4cb4700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f85b005c8f0 0x7f85b00b4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46232/0 (socket says 172.21.15.47:46232) 2022-06-29T15:42:45.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.643+0000 7f85b4cb4700 1 -- 172.21.15.47:0/3420069518 learned_addr learned my addr 172.21.15.47:0/3420069518 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:45.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.643+0000 7f85b54b5700 1 --2- 172.21.15.47:0/3420069518 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f85b005c510 0x7f85b0143bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:45.651 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.643+0000 7f85b4cb4700 1 -- 172.21.15.47:0/3420069518 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f85b005c510 msgr2=0x7f85b0143bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:45.651 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.643+0000 7f85b4cb4700 1 --2- 172.21.15.47:0/3420069518 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f85b005c510 0x7f85b0143bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:45.651 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.643+0000 7f85b4cb4700 1 -- 172.21.15.47:0/3420069518 >> v1:172.21.15.64:6789/0 conn(0x7f85b00b2c40 legacy=0x7f85b013cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:45.652 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.643+0000 7f85b4cb4700 1 -- 172.21.15.47:0/3420069518 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b0143730 con 0x7f85b005c8f0 2022-06-29T15:42:45.652 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.643+0000 7f85b4cb4700 1 --2- 172.21.15.47:0/3420069518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85b005c8f0 0x7f85b00b4100 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f859c0032f0 tx=0x7f859c009290).ready entity=mon.0 client_cookie=4eecaf316e12505d server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:45.652 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.643+0000 7f85affff700 1 -- 172.21.15.47:0/3420069518 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f859c00b8a0 con 0x7f85b005c8f0 2022-06-29T15:42:45.652 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.643+0000 7f85affff700 1 -- 172.21.15.47:0/3420069518 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f859c00ba00 con 0x7f85b005c8f0 2022-06-29T15:42:45.653 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.643+0000 7f85affff700 1 -- 172.21.15.47:0/3420069518 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f859c00bb70 con 0x7f85b005c8f0 2022-06-29T15:42:45.653 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.643+0000 7f85b64b7700 1 -- 172.21.15.47:0/3420069518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85b005c8f0 msgr2=0x7f85b00b4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:45.653 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.643+0000 7f85b64b7700 1 --2- 172.21.15.47:0/3420069518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85b005c8f0 0x7f85b00b4100 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f859c0032f0 tx=0x7f859c009290).stop 2022-06-29T15:42:45.654 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.643+0000 7f85b64b7700 1 -- 172.21.15.47:0/3420069518 shutdown_connections 2022-06-29T15:42:45.654 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.643+0000 7f85b64b7700 1 --2- 172.21.15.47:0/3420069518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85b005c8f0 0x7f85b00b4100 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:45.654 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.643+0000 7f85b64b7700 1 --2- 172.21.15.47:0/3420069518 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f85b005c510 0x7f85b0143bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:45.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.643+0000 7f85b64b7700 1 -- 172.21.15.47:0/3420069518 >> 172.21.15.47:0/3420069518 conn(0x7f85b0060e00 msgr2=0x7f85b00611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:45.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.643+0000 7f85b64b7700 1 -- 172.21.15.47:0/3420069518 shutdown_connections 2022-06-29T15:42:45.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.643+0000 7f85b64b7700 1 -- 172.21.15.47:0/3420069518 wait complete. 2022-06-29T15:42:45.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.647+0000 7f85b64b7700 1 Processor -- start 2022-06-29T15:42:45.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.647+0000 7f85b64b7700 1 -- start start 2022-06-29T15:42:45.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.647+0000 7f85b64b7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85b005c510 0x7f85b013c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:45.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.647+0000 7f85b64b7700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f85b005c8f0 0x7f85b0152750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:45.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.647+0000 7f85b64b7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85b00b2c40 0x7f85b0152c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:45.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.647+0000 7f85b64b7700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b00afea0 con 0x7f85b005c510 2022-06-29T15:42:45.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.647+0000 7f85b64b7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b0145030 con 0x7f85b00b2c40 2022-06-29T15:42:45.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.647+0000 7f85b64b7700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85b013bed0 con 0x7f85b005c8f0 2022-06-29T15:42:45.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.647+0000 7f85b54b5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85b005c510 0x7f85b013c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:45.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.647+0000 7f85b54b5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85b005c510 0x7f85b013c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46238/0 (socket says 172.21.15.47:46238) 2022-06-29T15:42:45.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.647+0000 7f85b54b5700 1 -- 172.21.15.47:0/118929154 learned_addr learned my addr 172.21.15.47:0/118929154 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:45.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.647+0000 7f85b4cb4700 1 --2- 172.21.15.47:0/118929154 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f85b005c8f0 0x7f85b0152750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:45.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.647+0000 7f85b5cb6700 1 --2- 172.21.15.47:0/118929154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85b00b2c40 0x7f85b0152c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:45.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.647+0000 7f85b54b5700 1 -- 172.21.15.47:0/118929154 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f85b005c8f0 msgr2=0x7f85b0152750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:45.663 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.647+0000 7f85b54b5700 1 --2- 172.21.15.47:0/118929154 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f85b005c8f0 0x7f85b0152750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:45.663 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.647+0000 7f85b54b5700 1 -- 172.21.15.47:0/118929154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85b00b2c40 msgr2=0x7f85b0152c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:45.663 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.647+0000 7f85b54b5700 1 --2- 172.21.15.47:0/118929154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85b00b2c40 0x7f85b0152c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:45.663 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.647+0000 7f85b54b5700 1 -- 172.21.15.47:0/118929154 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f859c009530 con 0x7f85b005c510 2022-06-29T15:42:45.664 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.651+0000 7f85b54b5700 1 --2- 172.21.15.47:0/118929154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85b005c510 0x7f85b013c460 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f85b00afa20 tx=0x7f85a0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:45.664 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.651+0000 7f85ae7fc700 1 -- 172.21.15.47:0/118929154 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85a0011780 con 0x7f85b005c510 2022-06-29T15:42:45.664 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.651+0000 7f85ae7fc700 1 -- 172.21.15.47:0/118929154 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85a00118e0 con 0x7f85b005c510 2022-06-29T15:42:45.664 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.651+0000 7f85ae7fc700 1 -- 172.21.15.47:0/118929154 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85a0011c40 con 0x7f85b005c510 2022-06-29T15:42:45.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.651+0000 7f85b64b7700 1 -- 172.21.15.47:0/118929154 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85b013b990 con 0x7f85b005c510 2022-06-29T15:42:45.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.651+0000 7f85b64b7700 1 -- 172.21.15.47:0/118929154 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85b013ce50 con 0x7f85b005c510 2022-06-29T15:42:45.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.651+0000 7f85b64b7700 1 -- 172.21.15.47:0/118929154 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85b00573d0 con 0x7f85b005c510 2022-06-29T15:42:45.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.651+0000 7f85ae7fc700 1 -- 172.21.15.47:0/118929154 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f85a0012420 con 0x7f85b005c510 2022-06-29T15:42:45.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.651+0000 7f85ae7fc700 1 --2- 172.21.15.47:0/118929154 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f858c053a70 0x7f858c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:45.666 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.651+0000 7f85ae7fc700 1 -- 172.21.15.47:0/118929154 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f85a006bd90 con 0x7f85b005c510 2022-06-29T15:42:45.666 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.651+0000 7f85b4cb4700 1 --2- 172.21.15.47:0/118929154 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f858c053a70 0x7f858c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:45.666 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.655+0000 7f85b4cb4700 1 --2- 172.21.15.47:0/118929154 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f858c053a70 0x7f858c055f30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f859c01b000 tx=0x7f859c016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:45.667 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.659+0000 7f85ae7fc700 1 -- 172.21.15.47:0/118929154 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f85a0042af0 con 0x7f85b005c510 2022-06-29T15:42:45.795 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.791+0000 7f85b64b7700 1 -- 172.21.15.47:0/118929154 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f85b0061b00 con 0x7f85b005c510 2022-06-29T15:42:45.796 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.791+0000 7f85ae7fc700 1 -- 172.21.15.47:0/118929154 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f85a000dea0 con 0x7f85b005c510 2022-06-29T15:42:45.797 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:42:45.798 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":26,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:42:34.474438+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:42:28.400446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-06-29T15:42:31.167705+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:42:45.799 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.795+0000 7f85b64b7700 1 -- 172.21.15.47:0/118929154 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f858c053a70 msgr2=0x7f858c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:45.800 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.795+0000 7f85b64b7700 1 --2- 172.21.15.47:0/118929154 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f858c053a70 0x7f858c055f30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f859c01b000 tx=0x7f859c016000).stop 2022-06-29T15:42:45.800 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.795+0000 7f85b64b7700 1 -- 172.21.15.47:0/118929154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85b005c510 msgr2=0x7f85b013c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:45.800 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.795+0000 7f85b64b7700 1 --2- 172.21.15.47:0/118929154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85b005c510 0x7f85b013c460 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f85b00afa20 tx=0x7f85a0008040).stop 2022-06-29T15:42:45.800 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.795+0000 7f85b64b7700 1 -- 172.21.15.47:0/118929154 shutdown_connections 2022-06-29T15:42:45.801 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.795+0000 7f85b64b7700 1 --2- 172.21.15.47:0/118929154 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f858c053a70 0x7f858c055f30 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:45.801 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.795+0000 7f85b64b7700 1 --2- 172.21.15.47:0/118929154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85b00b2c40 0x7f85b0152c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:45.801 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.795+0000 7f85b64b7700 1 --2- 172.21.15.47:0/118929154 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f85b005c8f0 0x7f85b0152750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:45.801 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.795+0000 7f85b64b7700 1 --2- 172.21.15.47:0/118929154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85b005c510 0x7f85b013c460 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:45.802 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.795+0000 7f85b64b7700 1 -- 172.21.15.47:0/118929154 >> 172.21.15.47:0/118929154 conn(0x7f85b0060e00 msgr2=0x7f85b00b5d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:45.802 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.795+0000 7f85b64b7700 1 -- 172.21.15.47:0/118929154 shutdown_connections 2022-06-29T15:42:45.802 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.795+0000 7f85b64b7700 1 -- 172.21.15.47:0/118929154 wait complete. 2022-06-29T15:42:45.810 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-29T15:42:16.971431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-29T15:42:25.290884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-29T15:42:28.400446+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2022-06-29T15:42:31.167705+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-29T15:42:45.811 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-29T15:42:45.924 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.919+0000 7f7c262c7700 1 Processor -- start 2022-06-29T15:42:45.924 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.919+0000 7f7c262c7700 1 -- start start 2022-06-29T15:42:45.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.923+0000 7f7c262c7700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7c20066fe0 0x7f7c200b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:45.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.923+0000 7f7c262c7700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7c20143150 0x7f7c2012bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:45.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.923+0000 7f7c262c7700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c200b68a0 con 0x7f7c200b64c0 2022-06-29T15:42:45.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.923+0000 7f7c262c7700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c200b13a0 con 0x7f7c20066fe0 2022-06-29T15:42:45.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.923+0000 7f7c262c7700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c201473d0 con 0x7f7c20143150 2022-06-29T15:42:45.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.923+0000 7f7c24ac4700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7c20066fe0 0x7f7c200b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:45.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.923+0000 7f7c252c5700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7c200b64c0 0x7f7c200b4b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:60032/0 (socket says 172.21.15.47:60032) 2022-06-29T15:42:45.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.923+0000 7f7c252c5700 1 -- 172.21.15.47:0/149067570 learned_addr learned my addr 172.21.15.47:0/149067570 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:45.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.923+0000 7f7c24ac4700 1 --2- 172.21.15.47:0/149067570 >> v2:172.21.15.47:3300/0 conn(0x7f7c20066fe0 0x7f7c200b5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46248/0 (socket says 172.21.15.47:46248) 2022-06-29T15:42:45.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.923+0000 7f7c25ac6700 1 --2- 172.21.15.47:0/149067570 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7c20143150 0x7f7c2012bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:45.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.923+0000 7f7c0ffff700 1 -- 172.21.15.47:0/149067570 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3560416064 0 0) 0x7f7c200b68a0 con 0x7f7c200b64c0 2022-06-29T15:42:45.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.923+0000 7f7c0ffff700 1 -- 172.21.15.47:0/149067570 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c08003260 con 0x7f7c200b64c0 2022-06-29T15:42:45.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.923+0000 7f7c24ac4700 1 -- 172.21.15.47:0/149067570 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7c20143150 msgr2=0x7f7c2012bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:45.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.923+0000 7f7c24ac4700 1 --2- 172.21.15.47:0/149067570 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7c20143150 0x7f7c2012bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:45.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.923+0000 7f7c24ac4700 1 -- 172.21.15.47:0/149067570 >> v1:172.21.15.47:6789/0 conn(0x7f7c200b64c0 legacy=0x7f7c200b4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:45.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.923+0000 7f7c24ac4700 1 -- 172.21.15.47:0/149067570 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c200b46a0 con 0x7f7c20066fe0 2022-06-29T15:42:45.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.923+0000 7f7c25ac6700 1 --2- 172.21.15.47:0/149067570 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7c20143150 0x7f7c2012bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:42:45.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.923+0000 7f7c24ac4700 1 --2- 172.21.15.47:0/149067570 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c20066fe0 0x7f7c200b5a20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f7c10002f70 tx=0x7f7c10006d90).ready entity=mon.0 client_cookie=b3107b1531c194b server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:45.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.923+0000 7f7c0ffff700 1 -- 172.21.15.47:0/149067570 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c1000e590 con 0x7f7c20066fe0 2022-06-29T15:42:45.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.923+0000 7f7c0ffff700 1 -- 172.21.15.47:0/149067570 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c10000e90 con 0x7f7c20066fe0 2022-06-29T15:42:45.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.923+0000 7f7c0ffff700 1 -- 172.21.15.47:0/149067570 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c1000e8e0 con 0x7f7c20066fe0 2022-06-29T15:42:45.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.923+0000 7f7c262c7700 1 -- 172.21.15.47:0/149067570 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c20066fe0 msgr2=0x7f7c200b5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:45.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.923+0000 7f7c262c7700 1 --2- 172.21.15.47:0/149067570 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c20066fe0 0x7f7c200b5a20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f7c10002f70 tx=0x7f7c10006d90).stop 2022-06-29T15:42:45.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.923+0000 7f7c262c7700 1 -- 172.21.15.47:0/149067570 shutdown_connections 2022-06-29T15:42:45.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.923+0000 7f7c262c7700 1 --2- 172.21.15.47:0/149067570 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7c20143150 0x7f7c2012bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:45.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.923+0000 7f7c262c7700 1 --2- 172.21.15.47:0/149067570 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c20066fe0 0x7f7c200b5a20 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:45.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.923+0000 7f7c262c7700 1 -- 172.21.15.47:0/149067570 >> 172.21.15.47:0/149067570 conn(0x7f7c20060e00 msgr2=0x7f7c200611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:45.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.923+0000 7f7c262c7700 1 -- 172.21.15.47:0/149067570 shutdown_connections 2022-06-29T15:42:45.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.923+0000 7f7c262c7700 1 -- 172.21.15.47:0/149067570 wait complete. 2022-06-29T15:42:45.939 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.923+0000 7f7c262c7700 1 Processor -- start 2022-06-29T15:42:45.939 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.923+0000 7f7c262c7700 1 -- start start 2022-06-29T15:42:45.939 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.927+0000 7f7c262c7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c20066fe0 0x7f7c2012b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:45.940 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.927+0000 7f7c262c7700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7c200b64c0 0x7f7c20140280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:45.940 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.927+0000 7f7c262c7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c20143150 0x7f7c20140770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:45.940 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.927+0000 7f7c262c7700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c20143000 con 0x7f7c20066fe0 2022-06-29T15:42:45.940 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.927+0000 7f7c262c7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c20145bc0 con 0x7f7c20143150 2022-06-29T15:42:45.941 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.927+0000 7f7c262c7700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c20145e60 con 0x7f7c200b64c0 2022-06-29T15:42:45.941 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.927+0000 7f7c252c5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c20066fe0 0x7f7c2012b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:45.941 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.927+0000 7f7c252c5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c20066fe0 0x7f7c2012b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46252/0 (socket says 172.21.15.47:46252) 2022-06-29T15:42:45.941 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.927+0000 7f7c252c5700 1 -- 172.21.15.47:0/1032196735 learned_addr learned my addr 172.21.15.47:0/1032196735 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:45.942 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.927+0000 7f7c24ac4700 1 --2- 172.21.15.47:0/1032196735 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7c200b64c0 0x7f7c20140280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:45.942 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.927+0000 7f7c25ac6700 1 --2- 172.21.15.47:0/1032196735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c20143150 0x7f7c20140770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:45.942 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.927+0000 7f7c252c5700 1 -- 172.21.15.47:0/1032196735 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7c200b64c0 msgr2=0x7f7c20140280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:45.942 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.927+0000 7f7c252c5700 1 --2- 172.21.15.47:0/1032196735 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7c200b64c0 0x7f7c20140280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:45.942 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.927+0000 7f7c252c5700 1 -- 172.21.15.47:0/1032196735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c20143150 msgr2=0x7f7c20140770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:45.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.927+0000 7f7c252c5700 1 --2- 172.21.15.47:0/1032196735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c20143150 0x7f7c20140770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:45.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.927+0000 7f7c252c5700 1 -- 172.21.15.47:0/1032196735 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c18009470 con 0x7f7c20066fe0 2022-06-29T15:42:45.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.927+0000 7f7c252c5700 1 --2- 172.21.15.47:0/1032196735 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c20066fe0 0x7f7c2012b120 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f7c1c001110 tx=0x7f7c1c007aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:45.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.927+0000 7f7c0e7fc700 1 -- 172.21.15.47:0/1032196735 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c1c0070d0 con 0x7f7c20066fe0 2022-06-29T15:42:45.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.927+0000 7f7c0e7fc700 1 -- 172.21.15.47:0/1032196735 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c1c007230 con 0x7f7c20066fe0 2022-06-29T15:42:45.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.927+0000 7f7c262c7700 1 -- 172.21.15.47:0/1032196735 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c1000b120 con 0x7f7c20066fe0 2022-06-29T15:42:45.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.927+0000 7f7c0e7fc700 1 -- 172.21.15.47:0/1032196735 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c1c005610 con 0x7f7c20066fe0 2022-06-29T15:42:45.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.927+0000 7f7c262c7700 1 -- 172.21.15.47:0/1032196735 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c20144890 con 0x7f7c20066fe0 2022-06-29T15:42:45.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.927+0000 7f7c262c7700 1 -- 172.21.15.47:0/1032196735 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c20144a90 con 0x7f7c20066fe0 2022-06-29T15:42:45.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.927+0000 7f7c0e7fc700 1 -- 172.21.15.47:0/1032196735 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7c1c010070 con 0x7f7c20066fe0 2022-06-29T15:42:45.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.931+0000 7f7c0e7fc700 1 --2- 172.21.15.47:0/1032196735 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7c00053b30 0x7f7c00055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:45.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.931+0000 7f7c0e7fc700 1 -- 172.21.15.47:0/1032196735 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f7c1c06b670 con 0x7f7c20066fe0 2022-06-29T15:42:45.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.931+0000 7f7c24ac4700 1 --2- 172.21.15.47:0/1032196735 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7c00053b30 0x7f7c00055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:45.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.931+0000 7f7c24ac4700 1 --2- 172.21.15.47:0/1032196735 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7c00053b30 0x7f7c00055ff0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f7c1001a000 tx=0x7f7c10014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:45.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:45.935+0000 7f7c0e7fc700 1 -- 172.21.15.47:0/1032196735 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7c20144a90 con 0x7f7c20066fe0 2022-06-29T15:42:46.066 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.063+0000 7f7c262c7700 1 -- 172.21.15.47:0/1032196735 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f7c20144a90 con 0x7f7c20066fe0 2022-06-29T15:42:46.067 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.063+0000 7f7c0e7fc700 1 -- 172.21.15.47:0/1032196735 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v26) v1 ==== 110+0+10 (secure 0 0 0) 0x7f7c20144a90 con 0x7f7c20066fe0 2022-06-29T15:42:46.067 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2022-06-29T15:42:46.070 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.067+0000 7f7c262c7700 1 -- 172.21.15.47:0/1032196735 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7c00053b30 msgr2=0x7f7c00055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:46.071 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.067+0000 7f7c262c7700 1 --2- 172.21.15.47:0/1032196735 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7c00053b30 0x7f7c00055ff0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f7c1001a000 tx=0x7f7c10014000).stop 2022-06-29T15:42:46.071 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.067+0000 7f7c262c7700 1 -- 172.21.15.47:0/1032196735 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c20066fe0 msgr2=0x7f7c2012b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:46.071 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.067+0000 7f7c262c7700 1 --2- 172.21.15.47:0/1032196735 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c20066fe0 0x7f7c2012b120 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f7c1c001110 tx=0x7f7c1c007aa0).stop 2022-06-29T15:42:46.071 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.067+0000 7f7c262c7700 1 -- 172.21.15.47:0/1032196735 shutdown_connections 2022-06-29T15:42:46.072 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.067+0000 7f7c262c7700 1 --2- 172.21.15.47:0/1032196735 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7c00053b30 0x7f7c00055ff0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:46.072 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.067+0000 7f7c262c7700 1 --2- 172.21.15.47:0/1032196735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c20143150 0x7f7c20140770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:46.072 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.067+0000 7f7c262c7700 1 --2- 172.21.15.47:0/1032196735 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7c200b64c0 0x7f7c20140280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:46.072 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.067+0000 7f7c262c7700 1 --2- 172.21.15.47:0/1032196735 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c20066fe0 0x7f7c2012b120 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:46.072 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.067+0000 7f7c262c7700 1 -- 172.21.15.47:0/1032196735 >> 172.21.15.47:0/1032196735 conn(0x7f7c20060e00 msgr2=0x7f7c200b7fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:46.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.067+0000 7f7c262c7700 1 -- 172.21.15.47:0/1032196735 shutdown_connections 2022-06-29T15:42:46.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.067+0000 7f7c262c7700 1 -- 172.21.15.47:0/1032196735 wait complete. 2022-06-29T15:42:46.086 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-29T15:42:46.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.194+0000 7f00e1d9f700 1 Processor -- start 2022-06-29T15:42:46.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.194+0000 7f00e1d9f700 1 -- start start 2022-06-29T15:42:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.198+0000 7f00e1d9f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f00dc05f0b0 0x7f00dc0b1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.198+0000 7f00e1d9f700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00dc061d90 con 0x7f00dc0b1e20 2022-06-29T15:42:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.198+0000 7f00e1d9f700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00dc05f6d0 con 0x7f00dc0619b0 2022-06-29T15:42:46.204 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.198+0000 7f00e1d9f700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00dc146ea0 con 0x7f00dc05f0b0 2022-06-29T15:42:46.204 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.198+0000 7f00daffd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f00dc0b1e20 0x7f00dc0b01b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:60048/0 (socket says 172.21.15.47:60048) 2022-06-29T15:42:46.204 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.198+0000 7f00daffd700 1 -- 172.21.15.47:0/2202028124 learned_addr learned my addr 172.21.15.47:0/2202028124 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:46.205 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.198+0000 7f00dbfff700 1 --2- 172.21.15.47:0/2202028124 >> v2:172.21.15.64:3300/0 conn(0x7f00dc05f0b0 0x7f00dc0b1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:46.205 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.198+0000 7f00da7fc700 1 -- 172.21.15.47:0/2202028124 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1491759559 0 0) 0x7f00dc061d90 con 0x7f00dc0b1e20 2022-06-29T15:42:46.205 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.198+0000 7f00da7fc700 1 -- 172.21.15.47:0/2202028124 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00bc003260 con 0x7f00dc0b1e20 2022-06-29T15:42:46.214 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.198+0000 7f00da7fc700 1 -- 172.21.15.47:0/2202028124 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2686530052 0 0) 0x7f00bc003260 con 0x7f00dc0b1e20 2022-06-29T15:42:46.215 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.198+0000 7f00dbfff700 1 -- 172.21.15.47:0/2202028124 >> v1:172.21.15.64:6789/0 conn(0x7f00dc0619b0 legacy=0x7f00dc146790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:46.215 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.198+0000 7f00dbfff700 1 -- 172.21.15.47:0/2202028124 >> v1:172.21.15.47:6789/0 conn(0x7f00dc0b1e20 legacy=0x7f00dc0b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:46.215 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.198+0000 7f00dbfff700 1 -- 172.21.15.47:0/2202028124 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00dc0b3bf0 con 0x7f00dc05f0b0 2022-06-29T15:42:46.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.198+0000 7f00dbfff700 1 --2- 172.21.15.47:0/2202028124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00dc05f0b0 0x7f00dc0b1360 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f00c4001800 tx=0x7f00d40091d0).ready entity=mon.1 client_cookie=63e2ffd58889e2eb server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:46.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.198+0000 7f00da7fc700 1 -- 172.21.15.47:0/2202028124 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f00c4001660 con 0x7f00dc05f0b0 2022-06-29T15:42:46.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.198+0000 7f00da7fc700 1 -- 172.21.15.47:0/2202028124 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00d400b580 con 0x7f00dc05f0b0 2022-06-29T15:42:46.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.198+0000 7f00da7fc700 1 -- 172.21.15.47:0/2202028124 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f00d400b6f0 con 0x7f00dc05f0b0 2022-06-29T15:42:46.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.198+0000 7f00e1d9f700 1 -- 172.21.15.47:0/2202028124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00dc05f0b0 msgr2=0x7f00dc0b1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:46.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.198+0000 7f00e1d9f700 1 --2- 172.21.15.47:0/2202028124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00dc05f0b0 0x7f00dc0b1360 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f00c4001800 tx=0x7f00d40091d0).stop 2022-06-29T15:42:46.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.198+0000 7f00e1d9f700 1 -- 172.21.15.47:0/2202028124 shutdown_connections 2022-06-29T15:42:46.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.198+0000 7f00e1d9f700 1 --2- 172.21.15.47:0/2202028124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00dc05f0b0 0x7f00dc0b1360 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:46.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.198+0000 7f00e1d9f700 1 -- 172.21.15.47:0/2202028124 >> 172.21.15.47:0/2202028124 conn(0x7f00dc063800 msgr2=0x7f00dc063be0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:46.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.198+0000 7f00e1d9f700 1 -- 172.21.15.47:0/2202028124 shutdown_connections 2022-06-29T15:42:46.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.198+0000 7f00e1d9f700 1 -- 172.21.15.47:0/2202028124 wait complete. 2022-06-29T15:42:46.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.202+0000 7f00e1d9f700 1 Processor -- start 2022-06-29T15:42:46.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.202+0000 7f00e1d9f700 1 -- start start 2022-06-29T15:42:46.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.202+0000 7f00e1d9f700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f00dc05f0b0 0x7f00dc145f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:46.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.202+0000 7f00e1d9f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00dc0619b0 0x7f00dc0abf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:46.222 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.202+0000 7f00e1d9f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00dc0b1e20 0x7f00dc0ac440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:46.222 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.202+0000 7f00e1d9f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00dc066400 con 0x7f00dc0b1e20 2022-06-29T15:42:46.223 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.202+0000 7f00e1d9f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00dc066640 con 0x7f00dc0619b0 2022-06-29T15:42:46.223 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.202+0000 7f00e1d9f700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00dc066f60 con 0x7f00dc05f0b0 2022-06-29T15:42:46.224 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.202+0000 7f00dbfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00dc0b1e20 0x7f00dc0ac440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:46.224 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.202+0000 7f00dbfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00dc0b1e20 0x7f00dc0ac440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46270/0 (socket says 172.21.15.47:46270) 2022-06-29T15:42:46.224 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.202+0000 7f00dbfff700 1 -- 172.21.15.47:0/3150233950 learned_addr learned my addr 172.21.15.47:0/3150233950 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:46.225 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.202+0000 7f00daffd700 1 --2- 172.21.15.47:0/3150233950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00dc0619b0 0x7f00dc0abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:46.225 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.202+0000 7f00db7fe700 1 --2- 172.21.15.47:0/3150233950 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f00dc05f0b0 0x7f00dc145f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:46.225 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.202+0000 7f00dbfff700 1 -- 172.21.15.47:0/3150233950 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f00dc05f0b0 msgr2=0x7f00dc145f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:46.225 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.202+0000 7f00dbfff700 1 --2- 172.21.15.47:0/3150233950 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f00dc05f0b0 0x7f00dc145f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:46.226 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.202+0000 7f00dbfff700 1 -- 172.21.15.47:0/3150233950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00dc0619b0 msgr2=0x7f00dc0abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:46.226 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.202+0000 7f00dbfff700 1 --2- 172.21.15.47:0/3150233950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00dc0619b0 0x7f00dc0abf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:46.226 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.202+0000 7f00dbfff700 1 -- 172.21.15.47:0/3150233950 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d4009470 con 0x7f00dc0b1e20 2022-06-29T15:42:46.226 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.202+0000 7f00daffd700 1 --2- 172.21.15.47:0/3150233950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00dc0619b0 0x7f00dc0abf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:42:46.227 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.202+0000 7f00dbfff700 1 --2- 172.21.15.47:0/3150233950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00dc0b1e20 0x7f00dc0ac440 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f00dc0ab960 tx=0x7f00d4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:46.227 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.202+0000 7f00d8ff9700 1 -- 172.21.15.47:0/3150233950 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f00d40032d0 con 0x7f00dc0b1e20 2022-06-29T15:42:46.227 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.202+0000 7f00d8ff9700 1 -- 172.21.15.47:0/3150233950 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00d4011930 con 0x7f00dc0b1e20 2022-06-29T15:42:46.227 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.202+0000 7f00d8ff9700 1 -- 172.21.15.47:0/3150233950 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f00d4011c90 con 0x7f00dc0b1e20 2022-06-29T15:42:46.228 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.202+0000 7f00e1d9f700 1 -- 172.21.15.47:0/3150233950 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00dc066160 con 0x7f00dc0b1e20 2022-06-29T15:42:46.228 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.202+0000 7f00e1d9f700 1 -- 172.21.15.47:0/3150233950 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00dc065610 con 0x7f00dc0b1e20 2022-06-29T15:42:46.228 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.206+0000 7f00e1d9f700 1 -- 172.21.15.47:0/3150233950 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00dc0673c0 con 0x7f00dc0b1e20 2022-06-29T15:42:46.228 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.206+0000 7f00d8ff9700 1 -- 172.21.15.47:0/3150233950 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f00d4026030 con 0x7f00dc0b1e20 2022-06-29T15:42:46.229 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.206+0000 7f00d8ff9700 1 --2- 172.21.15.47:0/3150233950 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f00c0053a70 0x7f00c0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:46.229 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.206+0000 7f00d8ff9700 1 -- 172.21.15.47:0/3150233950 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f00d406ca30 con 0x7f00dc0b1e20 2022-06-29T15:42:46.229 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.206+0000 7f00db7fe700 1 --2- 172.21.15.47:0/3150233950 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f00c0053a70 0x7f00c0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:46.229 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.206+0000 7f00db7fe700 1 --2- 172.21.15.47:0/3150233950 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f00c0053a70 0x7f00c0055f30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f00cc00d000 tx=0x7f00cc007b90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:46.230 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.210+0000 7f00d8ff9700 1 -- 172.21.15.47:0/3150233950 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f00dc0673c0 con 0x7f00dc0b1e20 2022-06-29T15:42:46.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.338+0000 7f00e1d9f700 1 -- 172.21.15.47:0/3150233950 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f00dc0673c0 con 0x7f00dc0b1e20 2022-06-29T15:42:46.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.338+0000 7f00d8ff9700 1 -- 172.21.15.47:0/3150233950 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7f00dc0673c0 con 0x7f00dc0b1e20 2022-06-29T15:42:46.346 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:42:46.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.342+0000 7f00e1d9f700 1 -- 172.21.15.47:0/3150233950 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f00c0053a70 msgr2=0x7f00c0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:46.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.342+0000 7f00e1d9f700 1 --2- 172.21.15.47:0/3150233950 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f00c0053a70 0x7f00c0055f30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f00cc00d000 tx=0x7f00cc007b90).stop 2022-06-29T15:42:46.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.342+0000 7f00e1d9f700 1 -- 172.21.15.47:0/3150233950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00dc0b1e20 msgr2=0x7f00dc0ac440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:46.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.342+0000 7f00e1d9f700 1 --2- 172.21.15.47:0/3150233950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00dc0b1e20 0x7f00dc0ac440 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f00dc0ab960 tx=0x7f00d4016000).stop 2022-06-29T15:42:46.352 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.342+0000 7f00e1d9f700 1 -- 172.21.15.47:0/3150233950 shutdown_connections 2022-06-29T15:42:46.352 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.342+0000 7f00e1d9f700 1 --2- 172.21.15.47:0/3150233950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00dc0b1e20 0x7f00dc0ac440 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:46.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.342+0000 7f00e1d9f700 1 --2- 172.21.15.47:0/3150233950 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f00c0053a70 0x7f00c0055f30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:46.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.342+0000 7f00e1d9f700 1 --2- 172.21.15.47:0/3150233950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00dc0619b0 0x7f00dc0abf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:46.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.342+0000 7f00e1d9f700 1 --2- 172.21.15.47:0/3150233950 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f00dc05f0b0 0x7f00dc145f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:46.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.342+0000 7f00e1d9f700 1 -- 172.21.15.47:0/3150233950 >> 172.21.15.47:0/3150233950 conn(0x7f00dc063800 msgr2=0x7f00dc0b02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:46.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.342+0000 7f00e1d9f700 1 -- 172.21.15.47:0/3150233950 shutdown_connections 2022-06-29T15:42:46.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.342+0000 7f00e1d9f700 1 -- 172.21.15.47:0/3150233950 wait complete. 2022-06-29T15:42:46.366 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-29T15:42:46.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.474+0000 7f3bb054f700 1 Processor -- start 2022-06-29T15:42:46.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.474+0000 7f3bb054f700 1 -- start start 2022-06-29T15:42:46.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.474+0000 7f3bb054f700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3ba8060ee0 0x7f3ba80612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:46.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.474+0000 7f3bb054f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3ba80b4300 0x7f3ba81471a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:46.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.474+0000 7f3bb054f700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ba8067610 con 0x7f3ba8066f70 2022-06-29T15:42:46.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.474+0000 7f3bb054f700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ba80673a0 con 0x7f3ba80b4300 2022-06-29T15:42:46.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.474+0000 7f3bb054f700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba81467c0 con 0x7f3ba8060ee0 2022-06-29T15:42:46.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.474+0000 7f3badaea700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3ba80b4300 0x7f3ba81471a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:46.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.474+0000 7f3badaea700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3ba80b4300 0x7f3ba81471a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46278/0 (socket says 172.21.15.47:46278) 2022-06-29T15:42:46.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.474+0000 7f3badaea700 1 -- 172.21.15.47:0/320645618 learned_addr learned my addr 172.21.15.47:0/320645618 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:46.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.474+0000 7f3bae2eb700 1 --2- 172.21.15.47:0/320645618 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3ba8060ee0 0x7f3ba80612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:46.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.474+0000 7f3badaea700 1 -- 172.21.15.47:0/320645618 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3ba8060ee0 msgr2=0x7f3ba80612c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:46.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.474+0000 7f3badaea700 1 --2- 172.21.15.47:0/320645618 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3ba8060ee0 0x7f3ba80612c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:46.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.474+0000 7f3badaea700 1 -- 172.21.15.47:0/320645618 >> v1:172.21.15.64:6789/0 conn(0x7f3ba8066f70 legacy=0x7f3ba81430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:46.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.474+0000 7f3badaea700 1 -- 172.21.15.47:0/320645618 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba8146a60 con 0x7f3ba80b4300 2022-06-29T15:42:46.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.474+0000 7f3badaea700 1 --2- 172.21.15.47:0/320645618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ba80b4300 0x7f3ba81471a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f3b98002f70 tx=0x7f3b98006eb0).ready entity=mon.0 client_cookie=e7283b47e7faba0f server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:46.497 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.474+0000 7f3bad2e9700 1 -- 172.21.15.47:0/320645618 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b9800d570 con 0x7f3ba80b4300 2022-06-29T15:42:46.497 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.474+0000 7f3bad2e9700 1 -- 172.21.15.47:0/320645618 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b98000e90 con 0x7f3ba80b4300 2022-06-29T15:42:46.498 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.474+0000 7f3bad2e9700 1 -- 172.21.15.47:0/320645618 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b9800d900 con 0x7f3ba80b4300 2022-06-29T15:42:46.498 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.474+0000 7f3bb054f700 1 -- 172.21.15.47:0/320645618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ba80b4300 msgr2=0x7f3ba81471a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:46.498 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.474+0000 7f3bb054f700 1 --2- 172.21.15.47:0/320645618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ba80b4300 0x7f3ba81471a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f3b98002f70 tx=0x7f3b98006eb0).stop 2022-06-29T15:42:46.499 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.474+0000 7f3bb054f700 1 -- 172.21.15.47:0/320645618 shutdown_connections 2022-06-29T15:42:46.499 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.474+0000 7f3bb054f700 1 --2- 172.21.15.47:0/320645618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ba80b4300 0x7f3ba81471a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:46.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.474+0000 7f3bb054f700 1 --2- 172.21.15.47:0/320645618 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3ba8060ee0 0x7f3ba80612c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:46.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.474+0000 7f3bb054f700 1 -- 172.21.15.47:0/320645618 >> 172.21.15.47:0/320645618 conn(0x7f3ba805c510 msgr2=0x7f3ba805f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:46.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.474+0000 7f3bb054f700 1 -- 172.21.15.47:0/320645618 shutdown_connections 2022-06-29T15:42:46.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.474+0000 7f3bb054f700 1 -- 172.21.15.47:0/320645618 wait complete. 2022-06-29T15:42:46.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.478+0000 7f3bb054f700 1 Processor -- start 2022-06-29T15:42:46.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.478+0000 7f3bb054f700 1 -- start start 2022-06-29T15:42:46.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.478+0000 7f3bb054f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ba8060ee0 0x7f3ba813a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:46.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.478+0000 7f3bb054f700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3ba8066f70 0x7f3ba81354f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:46.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.478+0000 7f3bb054f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ba80b4300 0x7f3ba81359e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:46.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.478+0000 7f3bb054f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba8145310 con 0x7f3ba8060ee0 2022-06-29T15:42:46.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.478+0000 7f3bb054f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba8144dd0 con 0x7f3ba80b4300 2022-06-29T15:42:46.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.478+0000 7f3bb054f700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba8145070 con 0x7f3ba8066f70 2022-06-29T15:42:46.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.478+0000 7f3bae2eb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ba8060ee0 0x7f3ba813a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:46.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.478+0000 7f3bae2eb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ba8060ee0 0x7f3ba813a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46280/0 (socket says 172.21.15.47:46280) 2022-06-29T15:42:46.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.478+0000 7f3bae2eb700 1 -- 172.21.15.47:0/2781576706 learned_addr learned my addr 172.21.15.47:0/2781576706 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:46.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.478+0000 7f3badaea700 1 --2- 172.21.15.47:0/2781576706 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3ba8066f70 0x7f3ba81354f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:46.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.478+0000 7f3baeaec700 1 --2- 172.21.15.47:0/2781576706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ba80b4300 0x7f3ba81359e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:46.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.478+0000 7f3bae2eb700 1 -- 172.21.15.47:0/2781576706 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3ba8066f70 msgr2=0x7f3ba81354f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:46.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.478+0000 7f3bae2eb700 1 --2- 172.21.15.47:0/2781576706 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3ba8066f70 0x7f3ba81354f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:46.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.478+0000 7f3bae2eb700 1 -- 172.21.15.47:0/2781576706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ba80b4300 msgr2=0x7f3ba81359e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:46.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.478+0000 7f3bae2eb700 1 --2- 172.21.15.47:0/2781576706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ba80b4300 0x7f3ba81359e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:46.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.478+0000 7f3bae2eb700 1 -- 172.21.15.47:0/2781576706 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b9800b1e0 con 0x7f3ba8060ee0 2022-06-29T15:42:46.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.478+0000 7f3bae2eb700 1 --2- 172.21.15.47:0/2781576706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ba8060ee0 0x7f3ba813a730 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f3ba400c000 tx=0x7f3ba4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:46.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.478+0000 7f3b977fe700 1 -- 172.21.15.47:0/2781576706 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ba4010780 con 0x7f3ba8060ee0 2022-06-29T15:42:46.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.478+0000 7f3b977fe700 1 -- 172.21.15.47:0/2781576706 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ba40108e0 con 0x7f3ba8060ee0 2022-06-29T15:42:46.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.478+0000 7f3bb054f700 1 -- 172.21.15.47:0/2781576706 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ba8146170 con 0x7f3ba8060ee0 2022-06-29T15:42:46.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.478+0000 7f3b977fe700 1 -- 172.21.15.47:0/2781576706 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ba4010c40 con 0x7f3ba8060ee0 2022-06-29T15:42:46.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.478+0000 7f3bb054f700 1 -- 172.21.15.47:0/2781576706 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ba8143c60 con 0x7f3ba8060ee0 2022-06-29T15:42:46.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.482+0000 7f3bb054f700 1 -- 172.21.15.47:0/2781576706 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ba80573d0 con 0x7f3ba8060ee0 2022-06-29T15:42:46.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.482+0000 7f3b977fe700 1 -- 172.21.15.47:0/2781576706 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3ba401c460 con 0x7f3ba8060ee0 2022-06-29T15:42:46.513 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.482+0000 7f3b977fe700 1 --2- 172.21.15.47:0/2781576706 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3b88053a70 0x7f3b88055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:46.513 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.482+0000 7f3badaea700 1 --2- 172.21.15.47:0/2781576706 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3b88053a70 0x7f3b88055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:46.514 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.482+0000 7f3b977fe700 1 -- 172.21.15.47:0/2781576706 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f3ba406d250 con 0x7f3ba8060ee0 2022-06-29T15:42:46.514 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.482+0000 7f3badaea700 1 --2- 172.21.15.47:0/2781576706 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3b88053a70 0x7f3b88055f30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f3b9801a000 tx=0x7f3b98015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:46.515 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.486+0000 7f3b977fe700 1 -- 172.21.15.47:0/2781576706 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3ba4043ea0 con 0x7f3ba8060ee0 2022-06-29T15:42:46.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.610+0000 7f3bb054f700 1 -- 172.21.15.47:0/2781576706 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3ba80b5e50 con 0x7f3ba8060ee0 2022-06-29T15:42:46.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.610+0000 7f3b977fe700 1 -- 172.21.15.47:0/2781576706 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+10 (secure 0 0 0) 0x7f3ba40474c0 con 0x7f3ba8060ee0 2022-06-29T15:42:46.615 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:42:46.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.610+0000 7f3bb054f700 1 -- 172.21.15.47:0/2781576706 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3b88053a70 msgr2=0x7f3b88055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:46.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.610+0000 7f3bb054f700 1 --2- 172.21.15.47:0/2781576706 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3b88053a70 0x7f3b88055f30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f3b9801a000 tx=0x7f3b98015000).stop 2022-06-29T15:42:46.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.610+0000 7f3bb054f700 1 -- 172.21.15.47:0/2781576706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ba8060ee0 msgr2=0x7f3ba813a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:46.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.610+0000 7f3bb054f700 1 --2- 172.21.15.47:0/2781576706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ba8060ee0 0x7f3ba813a730 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f3ba400c000 tx=0x7f3ba4006df0).stop 2022-06-29T15:42:46.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.610+0000 7f3bb054f700 1 -- 172.21.15.47:0/2781576706 shutdown_connections 2022-06-29T15:42:46.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.610+0000 7f3bb054f700 1 --2- 172.21.15.47:0/2781576706 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3b88053a70 0x7f3b88055f30 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:46.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.610+0000 7f3bb054f700 1 --2- 172.21.15.47:0/2781576706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ba80b4300 0x7f3ba81359e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:46.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.610+0000 7f3bb054f700 1 --2- 172.21.15.47:0/2781576706 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3ba8066f70 0x7f3ba81354f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:46.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.610+0000 7f3bb054f700 1 --2- 172.21.15.47:0/2781576706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ba8060ee0 0x7f3ba813a730 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:46.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.610+0000 7f3bb054f700 1 -- 172.21.15.47:0/2781576706 >> 172.21.15.47:0/2781576706 conn(0x7f3ba805c510 msgr2=0x7f3ba80b7790 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:46.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.610+0000 7f3bb054f700 1 -- 172.21.15.47:0/2781576706 shutdown_connections 2022-06-29T15:42:46.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.610+0000 7f3bb054f700 1 -- 172.21.15.47:0/2781576706 wait complete. 2022-06-29T15:42:46.628 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-29T15:42:46.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.734+0000 7f53d887f700 1 Processor -- start 2022-06-29T15:42:46.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.734+0000 7f53d887f700 1 -- start start 2022-06-29T15:42:46.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.734+0000 7f53d887f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f53d4060ee0 0x7f53d40612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:46.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.734+0000 7f53d887f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f53d40b4300 0x7f53d41471a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:46.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.734+0000 7f53d887f700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53d4067610 con 0x7f53d4066f70 2022-06-29T15:42:46.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.734+0000 7f53d887f700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53d4067400 con 0x7f53d4060ee0 2022-06-29T15:42:46.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.734+0000 7f53d887f700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53d41467c0 con 0x7f53d40b4300 2022-06-29T15:42:46.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.734+0000 7f53d259c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f53d4060ee0 0x7f53d40612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:46.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.734+0000 7f53d259c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f53d4060ee0 0x7f53d40612c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46288/0 (socket says 172.21.15.47:46288) 2022-06-29T15:42:46.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.734+0000 7f53d259c700 1 -- 172.21.15.47:0/1943184967 learned_addr learned my addr 172.21.15.47:0/1943184967 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:46.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.734+0000 7f53d1d9b700 1 --2- 172.21.15.47:0/1943184967 >> v2:172.21.15.64:3300/0 conn(0x7f53d40b4300 0x7f53d41471a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:46.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.734+0000 7f53d259c700 1 -- 172.21.15.47:0/1943184967 >> v2:172.21.15.64:3300/0 conn(0x7f53d40b4300 msgr2=0x7f53d41471a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:46.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.734+0000 7f53d259c700 1 --2- 172.21.15.47:0/1943184967 >> v2:172.21.15.64:3300/0 conn(0x7f53d40b4300 0x7f53d41471a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:46.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.734+0000 7f53d259c700 1 -- 172.21.15.47:0/1943184967 >> v1:172.21.15.64:6789/0 conn(0x7f53d4066f70 legacy=0x7f53d41430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:46.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.734+0000 7f53d259c700 1 -- 172.21.15.47:0/1943184967 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d4146a60 con 0x7f53d4060ee0 2022-06-29T15:42:46.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.734+0000 7f53d259c700 1 --2- 172.21.15.47:0/1943184967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53d4060ee0 0x7f53d40612c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f53d40adb80 tx=0x7f53c4006eb0).ready entity=mon.0 client_cookie=a8e6aec8de221b8b server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:46.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.734+0000 7f53d159a700 1 -- 172.21.15.47:0/1943184967 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f53c400d610 con 0x7f53d4060ee0 2022-06-29T15:42:46.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.734+0000 7f53d159a700 1 -- 172.21.15.47:0/1943184967 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c4000e90 con 0x7f53d4060ee0 2022-06-29T15:42:46.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.734+0000 7f53d159a700 1 -- 172.21.15.47:0/1943184967 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f53c400d960 con 0x7f53d4060ee0 2022-06-29T15:42:46.748 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.738+0000 7f53d887f700 1 -- 172.21.15.47:0/1943184967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53d4060ee0 msgr2=0x7f53d40612c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:46.749 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.738+0000 7f53d887f700 1 --2- 172.21.15.47:0/1943184967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53d4060ee0 0x7f53d40612c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f53d40adb80 tx=0x7f53c4006eb0).stop 2022-06-29T15:42:46.749 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.738+0000 7f53d887f700 1 -- 172.21.15.47:0/1943184967 shutdown_connections 2022-06-29T15:42:46.749 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.738+0000 7f53d887f700 1 --2- 172.21.15.47:0/1943184967 >> v2:172.21.15.64:3300/0 conn(0x7f53d40b4300 0x7f53d41471a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:46.749 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.738+0000 7f53d887f700 1 --2- 172.21.15.47:0/1943184967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53d4060ee0 0x7f53d40612c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:46.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.738+0000 7f53d887f700 1 -- 172.21.15.47:0/1943184967 >> 172.21.15.47:0/1943184967 conn(0x7f53d405c510 msgr2=0x7f53d405f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:46.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.738+0000 7f53d887f700 1 -- 172.21.15.47:0/1943184967 shutdown_connections 2022-06-29T15:42:46.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.738+0000 7f53d887f700 1 -- 172.21.15.47:0/1943184967 wait complete. 2022-06-29T15:42:46.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.738+0000 7f53d887f700 1 Processor -- start 2022-06-29T15:42:46.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.738+0000 7f53d887f700 1 -- start start 2022-06-29T15:42:46.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.738+0000 7f53d887f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53d4060ee0 0x7f53d413b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:46.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.738+0000 7f53d887f700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f53d4066f70 0x7f53d414a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:46.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.738+0000 7f53d887f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53d40b4300 0x7f53d4155f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:46.752 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.738+0000 7f53d887f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d41391a0 con 0x7f53d4060ee0 2022-06-29T15:42:46.752 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.738+0000 7f53d887f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d4144b00 con 0x7f53d40b4300 2022-06-29T15:42:46.752 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.738+0000 7f53d887f700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53d4144d30 con 0x7f53d4066f70 2022-06-29T15:42:46.752 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.738+0000 7f53d259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53d4060ee0 0x7f53d413b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:46.753 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.738+0000 7f53d259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53d4060ee0 0x7f53d413b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46294/0 (socket says 172.21.15.47:46294) 2022-06-29T15:42:46.753 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.738+0000 7f53d259c700 1 -- 172.21.15.47:0/3357310155 learned_addr learned my addr 172.21.15.47:0/3357310155 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:46.753 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.738+0000 7f53d1d9b700 1 --2- 172.21.15.47:0/3357310155 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f53d4066f70 0x7f53d414a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:46.753 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.738+0000 7f53d2d9d700 1 --2- 172.21.15.47:0/3357310155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53d40b4300 0x7f53d4155f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:46.753 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.738+0000 7f53d259c700 1 -- 172.21.15.47:0/3357310155 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f53d4066f70 msgr2=0x7f53d414a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:46.754 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.738+0000 7f53d259c700 1 --2- 172.21.15.47:0/3357310155 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f53d4066f70 0x7f53d414a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:46.754 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.738+0000 7f53d259c700 1 -- 172.21.15.47:0/3357310155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53d40b4300 msgr2=0x7f53d4155f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:46.754 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.738+0000 7f53d259c700 1 --2- 172.21.15.47:0/3357310155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53d40b4300 0x7f53d4155f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:46.754 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.738+0000 7f53d259c700 1 -- 172.21.15.47:0/3357310155 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c400b1e0 con 0x7f53d4060ee0 2022-06-29T15:42:46.755 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.738+0000 7f53d259c700 1 --2- 172.21.15.47:0/3357310155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53d4060ee0 0x7f53d413b8e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f53d40adb80 tx=0x7f53c4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:46.755 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.738+0000 7f53c37fe700 1 -- 172.21.15.47:0/3357310155 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f53c4002f50 con 0x7f53d4060ee0 2022-06-29T15:42:46.755 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.738+0000 7f53c37fe700 1 -- 172.21.15.47:0/3357310155 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c400d990 con 0x7f53d4060ee0 2022-06-29T15:42:46.755 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.738+0000 7f53d887f700 1 -- 172.21.15.47:0/3357310155 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53d4144fd0 con 0x7f53d4060ee0 2022-06-29T15:42:46.755 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.738+0000 7f53c37fe700 1 -- 172.21.15.47:0/3357310155 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f53c4011d30 con 0x7f53d4060ee0 2022-06-29T15:42:46.756 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.738+0000 7f53d887f700 1 -- 172.21.15.47:0/3357310155 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53d41452e0 con 0x7f53d4060ee0 2022-06-29T15:42:46.756 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.742+0000 7f53d887f700 1 -- 172.21.15.47:0/3357310155 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53d4144820 con 0x7f53d4060ee0 2022-06-29T15:42:46.756 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.742+0000 7f53c37fe700 1 -- 172.21.15.47:0/3357310155 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f53c4025030 con 0x7f53d4060ee0 2022-06-29T15:42:46.756 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.742+0000 7f53c37fe700 1 --2- 172.21.15.47:0/3357310155 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f53b0053aa0 0x7f53b0055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:46.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.742+0000 7f53c37fe700 1 -- 172.21.15.47:0/3357310155 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f53c406d1e0 con 0x7f53d4060ee0 2022-06-29T15:42:46.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.742+0000 7f53d1d9b700 1 --2- 172.21.15.47:0/3357310155 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f53b0053aa0 0x7f53b0055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:46.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.742+0000 7f53d1d9b700 1 --2- 172.21.15.47:0/3357310155 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f53b0053aa0 0x7f53b0055f60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f53c800b000 tx=0x7f53c8006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:46.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.746+0000 7f53c37fe700 1 -- 172.21.15.47:0/3357310155 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f53d4144820 con 0x7f53d4060ee0 2022-06-29T15:42:46.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.870+0000 7f53d887f700 1 -- 172.21.15.47:0/3357310155 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f53d4144820 con 0x7f53d4060ee0 2022-06-29T15:42:46.877 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.870+0000 7f53c37fe700 1 -- 172.21.15.47:0/3357310155 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+10 (secure 0 0 0) 0x7f53d4144820 con 0x7f53d4060ee0 2022-06-29T15:42:46.877 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:42:46.880 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.874+0000 7f53d887f700 1 -- 172.21.15.47:0/3357310155 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f53b0053aa0 msgr2=0x7f53b0055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:46.880 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.874+0000 7f53d887f700 1 --2- 172.21.15.47:0/3357310155 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f53b0053aa0 0x7f53b0055f60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f53c800b000 tx=0x7f53c8006df0).stop 2022-06-29T15:42:46.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.874+0000 7f53d887f700 1 -- 172.21.15.47:0/3357310155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53d4060ee0 msgr2=0x7f53d413b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:46.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.874+0000 7f53d887f700 1 --2- 172.21.15.47:0/3357310155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53d4060ee0 0x7f53d413b8e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f53d40adb80 tx=0x7f53c4015000).stop 2022-06-29T15:42:46.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.874+0000 7f53d887f700 1 -- 172.21.15.47:0/3357310155 shutdown_connections 2022-06-29T15:42:46.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.874+0000 7f53d887f700 1 --2- 172.21.15.47:0/3357310155 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f53b0053aa0 0x7f53b0055f60 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:46.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.874+0000 7f53d887f700 1 --2- 172.21.15.47:0/3357310155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53d40b4300 0x7f53d4155f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:46.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.874+0000 7f53d887f700 1 --2- 172.21.15.47:0/3357310155 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f53d4066f70 0x7f53d414a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:46.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.874+0000 7f53d887f700 1 --2- 172.21.15.47:0/3357310155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53d4060ee0 0x7f53d413b8e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:46.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.874+0000 7f53d887f700 1 -- 172.21.15.47:0/3357310155 >> 172.21.15.47:0/3357310155 conn(0x7f53d405c510 msgr2=0x7f53d413ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:46.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.874+0000 7f53d887f700 1 -- 172.21.15.47:0/3357310155 shutdown_connections 2022-06-29T15:42:46.883 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.874+0000 7f53d887f700 1 -- 172.21.15.47:0/3357310155 wait complete. 2022-06-29T15:42:46.895 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-29T15:42:46.895 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:42:47.000 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.994+0000 7f74d3561700 1 Processor -- start 2022-06-29T15:42:47.000 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.994+0000 7f74d3561700 1 -- start start 2022-06-29T15:42:47.004 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.994+0000 7f74d3561700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f74cc0b5c60 0x7f74cc144980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:47.004 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.994+0000 7f74d3561700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f74cc0a72f0 0x7f74cc0b2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:47.004 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.994+0000 7f74d3561700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74cc0a9120 con 0x7f74cc060eb0 2022-06-29T15:42:47.004 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.994+0000 7f74d3561700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74cc0b0b80 con 0x7f74cc0b5c60 2022-06-29T15:42:47.005 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.994+0000 7f74d3561700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc0a89e0 con 0x7f74cc0a72f0 2022-06-29T15:42:47.005 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.994+0000 7f74d1d5e700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f74cc060eb0 0x7f74cc0b1e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:60088/0 (socket says 172.21.15.47:60088) 2022-06-29T15:42:47.005 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.994+0000 7f74d1d5e700 1 -- 172.21.15.47:0/1847694607 learned_addr learned my addr 172.21.15.47:0/1847694607 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:47.006 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.994+0000 7f74d2d60700 1 --2- 172.21.15.47:0/1847694607 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f74cc0a72f0 0x7f74cc0b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:47.006 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.994+0000 7f74d255f700 1 --2- 172.21.15.47:0/1847694607 >> v2:172.21.15.64:3300/0 conn(0x7f74cc0b5c60 0x7f74cc144980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:47.006 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.994+0000 7f74d155d700 1 -- 172.21.15.47:0/1847694607 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1639770600 0 0) 0x7f74cc0a9120 con 0x7f74cc060eb0 2022-06-29T15:42:47.006 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.994+0000 7f74d155d700 1 -- 172.21.15.47:0/1847694607 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74b4003260 con 0x7f74cc060eb0 2022-06-29T15:42:47.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.994+0000 7f74d2d60700 1 -- 172.21.15.47:0/1847694607 >> v2:172.21.15.64:3300/0 conn(0x7f74cc0b5c60 msgr2=0x7f74cc144980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:47.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.994+0000 7f74d2d60700 1 --2- 172.21.15.47:0/1847694607 >> v2:172.21.15.64:3300/0 conn(0x7f74cc0b5c60 0x7f74cc144980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:47.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.994+0000 7f74d2d60700 1 -- 172.21.15.47:0/1847694607 >> v1:172.21.15.47:6789/0 conn(0x7f74cc060eb0 legacy=0x7f74cc0b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:47.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.994+0000 7f74d2d60700 1 -- 172.21.15.47:0/1847694607 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74cc0a8c80 con 0x7f74cc0a72f0 2022-06-29T15:42:47.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.994+0000 7f74d2d60700 1 --2- 172.21.15.47:0/1847694607 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f74cc0a72f0 0x7f74cc0b2540 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f74c4002f70 tx=0x7f74c4006df0).ready entity=mon.2 client_cookie=b0a306d17b9fef6f server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:47.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.998+0000 7f74d155d700 1 -- 172.21.15.47:0/1847694607 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74c400d4a0 con 0x7f74cc0a72f0 2022-06-29T15:42:47.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.998+0000 7f74d155d700 1 -- 172.21.15.47:0/1847694607 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74c4000e90 con 0x7f74cc0a72f0 2022-06-29T15:42:47.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.998+0000 7f74d155d700 1 -- 172.21.15.47:0/1847694607 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74c400d7f0 con 0x7f74cc0a72f0 2022-06-29T15:42:47.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.998+0000 7f74d3561700 1 -- 172.21.15.47:0/1847694607 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f74cc0a72f0 msgr2=0x7f74cc0b2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:47.009 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.998+0000 7f74d3561700 1 --2- 172.21.15.47:0/1847694607 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f74cc0a72f0 0x7f74cc0b2540 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f74c4002f70 tx=0x7f74c4006df0).stop 2022-06-29T15:42:47.009 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.998+0000 7f74d3561700 1 -- 172.21.15.47:0/1847694607 shutdown_connections 2022-06-29T15:42:47.009 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.998+0000 7f74d3561700 1 --2- 172.21.15.47:0/1847694607 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f74cc0a72f0 0x7f74cc0b2540 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:47.009 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.998+0000 7f74d3561700 1 --2- 172.21.15.47:0/1847694607 >> v2:172.21.15.64:3300/0 conn(0x7f74cc0b5c60 0x7f74cc144980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:47.010 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.998+0000 7f74d3561700 1 -- 172.21.15.47:0/1847694607 >> 172.21.15.47:0/1847694607 conn(0x7f74cc062560 msgr2=0x7f74cc062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:47.010 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.998+0000 7f74d3561700 1 -- 172.21.15.47:0/1847694607 shutdown_connections 2022-06-29T15:42:47.010 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.998+0000 7f74d3561700 1 -- 172.21.15.47:0/1847694607 wait complete. 2022-06-29T15:42:47.010 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.998+0000 7f74d3561700 1 Processor -- start 2022-06-29T15:42:47.011 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.998+0000 7f74d3561700 1 -- start start 2022-06-29T15:42:47.013 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.998+0000 7f74d3561700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74cc060eb0 0x7f74cc13e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:47.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.998+0000 7f74d3561700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74cc0a72f0 0x7f74cc0aa9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:47.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.998+0000 7f74d3561700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f74cc0b5c60 0x7f74cc14c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:47.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.998+0000 7f74d3561700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc13cdc0 con 0x7f74cc0a72f0 2022-06-29T15:42:47.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.998+0000 7f74d3561700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc0a9330 con 0x7f74cc060eb0 2022-06-29T15:42:47.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.998+0000 7f74d3561700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc14bce0 con 0x7f74cc0b5c60 2022-06-29T15:42:47.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.998+0000 7f74d1d5e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74cc0a72f0 0x7f74cc0aa9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:47.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.998+0000 7f74d1d5e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74cc0a72f0 0x7f74cc0aa9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46310/0 (socket says 172.21.15.47:46310) 2022-06-29T15:42:47.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.998+0000 7f74d1d5e700 1 -- 172.21.15.47:0/706628751 learned_addr learned my addr 172.21.15.47:0/706628751 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:47.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.998+0000 7f74d2d60700 1 --2- 172.21.15.47:0/706628751 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f74cc0b5c60 0x7f74cc14c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:47.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.998+0000 7f74d255f700 1 --2- 172.21.15.47:0/706628751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74cc060eb0 0x7f74cc13e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:47.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.998+0000 7f74d1d5e700 1 -- 172.21.15.47:0/706628751 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f74cc0b5c60 msgr2=0x7f74cc14c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:47.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.998+0000 7f74d1d5e700 1 --2- 172.21.15.47:0/706628751 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f74cc0b5c60 0x7f74cc14c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:47.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.998+0000 7f74d1d5e700 1 -- 172.21.15.47:0/706628751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74cc060eb0 msgr2=0x7f74cc13e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:47.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.998+0000 7f74d1d5e700 1 --2- 172.21.15.47:0/706628751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74cc060eb0 0x7f74cc13e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:47.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:46.998+0000 7f74d1d5e700 1 -- 172.21.15.47:0/706628751 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c400b180 con 0x7f74cc0a72f0 2022-06-29T15:42:47.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:47.002+0000 7f74d1d5e700 1 --2- 172.21.15.47:0/706628751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74cc0a72f0 0x7f74cc0aa9c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f74c8001490 tx=0x7f74c8007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:47.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:47.002+0000 7f74bb7fe700 1 -- 172.21.15.47:0/706628751 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74c8005ba0 con 0x7f74cc0a72f0 2022-06-29T15:42:47.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:47.002+0000 7f74bb7fe700 1 -- 172.21.15.47:0/706628751 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74c8005d00 con 0x7f74cc0a72f0 2022-06-29T15:42:47.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:47.002+0000 7f74d3561700 1 -- 172.21.15.47:0/706628751 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74cc0a95d0 con 0x7f74cc0a72f0 2022-06-29T15:42:47.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:47.002+0000 7f74bb7fe700 1 -- 172.21.15.47:0/706628751 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74c80072c0 con 0x7f74cc0a72f0 2022-06-29T15:42:47.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:47.002+0000 7f74d3561700 1 -- 172.21.15.47:0/706628751 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74cc14cb20 con 0x7f74cc0a72f0 2022-06-29T15:42:47.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:47.002+0000 7f74d3561700 1 -- 172.21.15.47:0/706628751 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74cc14bea0 con 0x7f74cc0a72f0 2022-06-29T15:42:47.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:47.002+0000 7f74bb7fe700 1 -- 172.21.15.47:0/706628751 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f74c801a070 con 0x7f74cc0a72f0 2022-06-29T15:42:47.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:47.002+0000 7f74bb7fe700 1 --2- 172.21.15.47:0/706628751 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f74ac053b00 0x7f74ac055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:47.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:47.002+0000 7f74bb7fe700 1 -- 172.21.15.47:0/706628751 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f74c806b2c0 con 0x7f74cc0a72f0 2022-06-29T15:42:47.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:47.002+0000 7f74d255f700 1 --2- 172.21.15.47:0/706628751 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f74ac053b00 0x7f74ac055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:47.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:47.006+0000 7f74d255f700 1 --2- 172.21.15.47:0/706628751 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f74ac053b00 0x7f74ac055fc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f74bc006000 tx=0x7f74bc006d90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:47.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:47.010+0000 7f74bb7fe700 1 -- 172.21.15.47:0/706628751 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f74c8042080 con 0x7f74cc0a72f0 2022-06-29T15:42:47.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:47.170+0000 7f74d3561700 1 -- 172.21.15.47:0/706628751 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74cc14d720 con 0x7f74cc0a72f0 2022-06-29T15:42:47.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:47.174+0000 7f74bb7fe700 1 -- 172.21.15.47:0/706628751 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+2483 (secure 0 0 0) 0x7f74c80456a0 con 0x7f74cc0a72f0 2022-06-29T15:42:47.181 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:42:47.181 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/438289599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":438289599},{"type":"v1","addr":"172.21.15.47:6835","nonce":438289599}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.64:6833/1472525731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1472525731},{"type":"v1","addr":"172.21.15.64:6833","nonce":1472525731}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":4}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:42:33.471491+0000","modified":"2022-06-29T15:42:35.146757+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":54128},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54128":{"gid":54128,"name":"a","rank":0,"incarnation":5,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6837/354581778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":354581778},{"type":"v1","addr":"172.21.15.47:6837","nonce":354581778}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2022-06-29T15:42:47.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:47.174+0000 7f74d3561700 1 -- 172.21.15.47:0/706628751 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f74ac053b00 msgr2=0x7f74ac055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:47.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:47.174+0000 7f74d3561700 1 --2- 172.21.15.47:0/706628751 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f74ac053b00 0x7f74ac055fc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f74bc006000 tx=0x7f74bc006d90).stop 2022-06-29T15:42:47.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:47.174+0000 7f74d3561700 1 -- 172.21.15.47:0/706628751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74cc0a72f0 msgr2=0x7f74cc0aa9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:47.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:47.174+0000 7f74d3561700 1 --2- 172.21.15.47:0/706628751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74cc0a72f0 0x7f74cc0aa9c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f74c8001490 tx=0x7f74c8007e20).stop 2022-06-29T15:42:47.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:47.174+0000 7f74d3561700 1 -- 172.21.15.47:0/706628751 shutdown_connections 2022-06-29T15:42:47.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:47.174+0000 7f74d3561700 1 --2- 172.21.15.47:0/706628751 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f74cc0b5c60 0x7f74cc14c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:47.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:47.174+0000 7f74d3561700 1 --2- 172.21.15.47:0/706628751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74cc0a72f0 0x7f74cc0aa9c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:47.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:47.174+0000 7f74d3561700 1 --2- 172.21.15.47:0/706628751 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f74ac053b00 0x7f74ac055fc0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:47.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:47.174+0000 7f74d3561700 1 --2- 172.21.15.47:0/706628751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74cc060eb0 0x7f74cc13e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:47.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:47.174+0000 7f74d3561700 1 -- 172.21.15.47:0/706628751 >> 172.21.15.47:0/706628751 conn(0x7f74cc062560 msgr2=0x7f74cc0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:47.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:47.174+0000 7f74d3561700 1 -- 172.21.15.47:0/706628751 shutdown_connections 2022-06-29T15:42:47.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:47.174+0000 7f74d3561700 1 -- 172.21.15.47:0/706628751 wait complete. 2022-06-29T15:42:47.187 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 7 2022-06-29T15:42:47.496 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-29T15:42:33.471491+0000', 'modified': '2022-06-29T15:42:35.146757+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 54128}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_54128': {'gid': 54128, 'name': 'a', 'rank': 0, 'incarnation': 5, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.47:6837/354581778', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6836', 'nonce': 354581778}, {'type': 'v1', 'addr': '172.21.15.47:6837', 'nonce': 354581778}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-29T15:42:47.496 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-29T15:42:47.497 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-06-29T15:42:47.614 INFO:teuthology.orchestra.run.smithi047.stdout:{"cluster_fsid":"9a68894a-53ce-400e-838b-14eab91b834c","whoami":0,"id":54128,"want_state":"up:active","state":"up:active","rank_uptime":14.125720406999999,"mdsmap_epoch":7,"osdmap_epoch":26,"osdmap_epoch_barrier":26,"uptime":20.502296385000001} 2022-06-29T15:42:47.615 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9a68894a-53ce-400e-838b-14eab91b834c", "id": 54128, "mdsmap_epoch": 7, "osdmap_epoch": 26, "osdmap_epoch_barrier": 26, "rank_uptime": 14.125720407, "state": "up:active", "uptime": 20.502296385, "want_state": "up:active", "whoami": 0 } 2022-06-29T15:42:47.615 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-29T15:42:47.616 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-06-29T15:42:47.616 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi099.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-06-29T15:42:47.616 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:42:47.617 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:42:47.625 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2022-06-29T15:42:47.684 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:42:47.684 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:42:47.699 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:42:47.700 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:42:47.701 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:42:47.701 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2022-06-29T15:42:47.708 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-29T15:42:47.709 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:42:47.757 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2022-06-29T15:42:47.777 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:42:47.778 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:42:47.789 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:42:47.790 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:42:47.791 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:42:47.791 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2022-06-29T15:42:47.839 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:2022-06-29T15:42:47.833+0000 7f7bf1f1f080 -1 init, newargv = 0x55e04c0ff3f0 newargc=9ceph-fuse[12211]: starting ceph client 2022-06-29T15:42:47.839 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr: 2022-06-29T15:42:47.848 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[12211]: starting fuse 2022-06-29T15:42:48.946 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2022-06-29T15:42:48.964 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:42:48.965 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:42:49.023 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:42:49.024 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:42:49.025 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:42:49.025 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2022-06-29T15:42:49.030 INFO:teuthology.orchestra.run.smithi099.stdout:52 2022-06-29T15:42:49.031 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-29T15:42:49.031 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:42:49.031 DEBUG:teuthology.orchestra.run.smithi099:> sudo python3 -c ' 2022-06-29T15:42:49.032 DEBUG:teuthology.orchestra.run.smithi099:> import glob 2022-06-29T15:42:49.032 DEBUG:teuthology.orchestra.run.smithi099:> import re 2022-06-29T15:42:49.033 DEBUG:teuthology.orchestra.run.smithi099:> import os 2022-06-29T15:42:49.033 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2022-06-29T15:42:49.033 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:42:49.034 DEBUG:teuthology.orchestra.run.smithi099:> def find_socket(client_name): 2022-06-29T15:42:49.035 DEBUG:teuthology.orchestra.run.smithi099:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-06-29T15:42:49.035 DEBUG:teuthology.orchestra.run.smithi099:> files = glob.glob(asok_path) 2022-06-29T15:42:49.035 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:42:49.036 DEBUG:teuthology.orchestra.run.smithi099:> # Given a non-glob path, it better be there 2022-06-29T15:42:49.036 DEBUG:teuthology.orchestra.run.smithi099:> if "*" not in asok_path: 2022-06-29T15:42:49.036 DEBUG:teuthology.orchestra.run.smithi099:> assert(len(files) == 1) 2022-06-29T15:42:49.037 DEBUG:teuthology.orchestra.run.smithi099:> return files[0] 2022-06-29T15:42:49.037 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:42:49.037 DEBUG:teuthology.orchestra.run.smithi099:> for f in files: 2022-06-29T15:42:49.037 DEBUG:teuthology.orchestra.run.smithi099:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-29T15:42:49.038 DEBUG:teuthology.orchestra.run.smithi099:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-29T15:42:49.038 DEBUG:teuthology.orchestra.run.smithi099:> return f 2022-06-29T15:42:49.038 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-29T15:42:49.039 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:42:49.039 DEBUG:teuthology.orchestra.run.smithi099:> print(find_socket("client.0")) 2022-06-29T15:42:49.040 DEBUG:teuthology.orchestra.run.smithi099:> ' 2022-06-29T15:42:49.115 INFO:teuthology.orchestra.run.smithi099.stdout:/var/run/ceph/ceph-client.0.12211.asok 2022-06-29T15:42:49.120 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.12211.asok 2022-06-29T15:42:49.121 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:42:49.122 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.12211.asok status 2022-06-29T15:42:49.212 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2022-06-29T15:42:49.212 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": { 2022-06-29T15:42:49.213 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_sha1": "f25ffac72e5a9e63860ff7deb6767a11f0559a16", 2022-06-29T15:42:49.213 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 15.2.16-304-gf25ffac7 (f25ffac72e5a9e63860ff7deb6767a11f0559a16) octopus (stable)", 2022-06-29T15:42:49.213 INFO:teuthology.orchestra.run.smithi099.stdout: "entity_id": "0", 2022-06-29T15:42:49.213 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": "smithi099", 2022-06-29T15:42:49.214 INFO:teuthology.orchestra.run.smithi099.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-06-29T15:42:49.214 INFO:teuthology.orchestra.run.smithi099.stdout: "pid": "12211", 2022-06-29T15:42:49.214 INFO:teuthology.orchestra.run.smithi099.stdout: "root": "/" 2022-06-29T15:42:49.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:42:49.215 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_count": 0, 2022-06-29T15:42:49.216 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_pinned_count": 0, 2022-06-29T15:42:49.216 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 4635, 2022-06-29T15:42:49.216 INFO:teuthology.orchestra.run.smithi099.stdout: "inst": { 2022-06-29T15:42:49.216 INFO:teuthology.orchestra.run.smithi099.stdout: "name": { 2022-06-29T15:42:49.216 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "client", 2022-06-29T15:42:49.217 INFO:teuthology.orchestra.run.smithi099.stdout: "num": 4635 2022-06-29T15:42:49.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:42:49.217 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2022-06-29T15:42:49.217 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2022-06-29T15:42:49.218 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:0", 2022-06-29T15:42:49.218 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 340727388 2022-06-29T15:42:49.218 INFO:teuthology.orchestra.run.smithi099.stdout: } 2022-06-29T15:42:49.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:42:49.219 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2022-06-29T15:42:49.219 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2022-06-29T15:42:49.219 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:0", 2022-06-29T15:42:49.219 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 340727388 2022-06-29T15:42:49.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:42:49.220 INFO:teuthology.orchestra.run.smithi099.stdout: "inst_str": "client.4635 172.21.15.99:0/340727388", 2022-06-29T15:42:49.220 INFO:teuthology.orchestra.run.smithi099.stdout: "addr_str": "172.21.15.99:0/340727388", 2022-06-29T15:42:49.220 INFO:teuthology.orchestra.run.smithi099.stdout: "inode_count": 1, 2022-06-29T15:42:49.221 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_epoch": 7, 2022-06-29T15:42:49.221 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch": 26, 2022-06-29T15:42:49.221 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch_barrier": 0, 2022-06-29T15:42:49.221 INFO:teuthology.orchestra.run.smithi099.stdout: "blacklisted": false, 2022-06-29T15:42:49.222 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs" 2022-06-29T15:42:49.222 INFO:teuthology.orchestra.run.smithi099.stdout:} 2022-06-29T15:42:49.224 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:42:49.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.329+0000 7f7401179700 1 Processor -- start 2022-06-29T15:42:49.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.329+0000 7f7401179700 1 -- start start 2022-06-29T15:42:49.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.329+0000 7f7401179700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f73fc0b5c60 0x7f73fc144980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:49.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.329+0000 7f7401179700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f73fc0a72f0 0x7f73fc0b2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:49.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.329+0000 7f7401179700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73fc0a9120 con 0x7f73fc060eb0 2022-06-29T15:42:49.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.329+0000 7f7401179700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73fc0b0b80 con 0x7f73fc0a72f0 2022-06-29T15:42:49.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.329+0000 7f7401179700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc0a89e0 con 0x7f73fc0b5c60 2022-06-29T15:42:49.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.333+0000 7f7400978700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f73fc0a72f0 0x7f73fc0b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:49.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.333+0000 7f73f37fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f73fc060eb0 0x7f73fc0b1e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:60102/0 (socket says 172.21.15.47:60102) 2022-06-29T15:42:49.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.333+0000 7f73f37fe700 1 -- 172.21.15.47:0/3955922469 learned_addr learned my addr 172.21.15.47:0/3955922469 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:49.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.333+0000 7f73fbfff700 1 --2- 172.21.15.47:0/3955922469 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f73fc0b5c60 0x7f73fc144980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:49.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.333+0000 7f7400978700 1 -- 172.21.15.47:0/3955922469 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f73fc0b5c60 msgr2=0x7f73fc144980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:49.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.333+0000 7f7400978700 1 --2- 172.21.15.47:0/3955922469 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f73fc0b5c60 0x7f73fc144980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:49.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.333+0000 7f7400978700 1 -- 172.21.15.47:0/3955922469 >> v1:172.21.15.47:6789/0 conn(0x7f73fc060eb0 legacy=0x7f73fc0b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:49.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.333+0000 7f73fb7fe700 1 -- 172.21.15.47:0/3955922469 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3238709704 0 0) 0x7f73fc0a9120 con 0x7f73fc060eb0 2022-06-29T15:42:49.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.333+0000 7f7400978700 1 -- 172.21.15.47:0/3955922469 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73fc0a8c80 con 0x7f73fc0a72f0 2022-06-29T15:42:49.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.333+0000 7f7400978700 1 --2- 172.21.15.47:0/3955922469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73fc0a72f0 0x7f73fc0b2540 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f73f40032f0 tx=0x7f73f40091d0).ready entity=mon.0 client_cookie=e41ed700b8f323c6 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:49.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.333+0000 7f73fb7fe700 1 -- 172.21.15.47:0/3955922469 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f73f4009ea0 con 0x7f73fc0a72f0 2022-06-29T15:42:49.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.333+0000 7f73fb7fe700 1 -- 172.21.15.47:0/3955922469 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73f400b8a0 con 0x7f73fc0a72f0 2022-06-29T15:42:49.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.333+0000 7f73fb7fe700 1 -- 172.21.15.47:0/3955922469 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f73f400ba10 con 0x7f73fc0a72f0 2022-06-29T15:42:49.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.333+0000 7f7401179700 1 -- 172.21.15.47:0/3955922469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73fc0a72f0 msgr2=0x7f73fc0b2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:49.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.333+0000 7f7401179700 1 --2- 172.21.15.47:0/3955922469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73fc0a72f0 0x7f73fc0b2540 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f73f40032f0 tx=0x7f73f40091d0).stop 2022-06-29T15:42:49.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.333+0000 7f7401179700 1 -- 172.21.15.47:0/3955922469 shutdown_connections 2022-06-29T15:42:49.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.333+0000 7f7401179700 1 --2- 172.21.15.47:0/3955922469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73fc0a72f0 0x7f73fc0b2540 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:49.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.333+0000 7f7401179700 1 --2- 172.21.15.47:0/3955922469 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f73fc0b5c60 0x7f73fc144980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:49.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.333+0000 7f7401179700 1 -- 172.21.15.47:0/3955922469 >> 172.21.15.47:0/3955922469 conn(0x7f73fc062560 msgr2=0x7f73fc062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:49.352 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.333+0000 7f7401179700 1 -- 172.21.15.47:0/3955922469 shutdown_connections 2022-06-29T15:42:49.352 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.333+0000 7f7401179700 1 -- 172.21.15.47:0/3955922469 wait complete. 2022-06-29T15:42:49.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.337+0000 7f7401179700 1 Processor -- start 2022-06-29T15:42:49.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.337+0000 7f7401179700 1 -- start start 2022-06-29T15:42:49.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.337+0000 7f7401179700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73fc060eb0 0x7f73fc14bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:49.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.337+0000 7f7401179700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f73fc0a72f0 0x7f73fc14b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:49.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.337+0000 7f7401179700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73fc0b5c60 0x7f73fc143580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:49.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.337+0000 7f7401179700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc156f00 con 0x7f73fc060eb0 2022-06-29T15:42:49.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.337+0000 7f7401179700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc157160 con 0x7f73fc0b5c60 2022-06-29T15:42:49.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.337+0000 7f7401179700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc157400 con 0x7f73fc0a72f0 2022-06-29T15:42:49.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.337+0000 7f73fbfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73fc060eb0 0x7f73fc14bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:49.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.337+0000 7f73fbfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73fc060eb0 0x7f73fc14bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46322/0 (socket says 172.21.15.47:46322) 2022-06-29T15:42:49.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.337+0000 7f73fbfff700 1 -- 172.21.15.47:0/1026667029 learned_addr learned my addr 172.21.15.47:0/1026667029 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:49.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.337+0000 7f73f37fe700 1 --2- 172.21.15.47:0/1026667029 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f73fc0a72f0 0x7f73fc14b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:49.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.337+0000 7f7400978700 1 --2- 172.21.15.47:0/1026667029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73fc0b5c60 0x7f73fc143580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:49.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.337+0000 7f73fbfff700 1 -- 172.21.15.47:0/1026667029 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f73fc0a72f0 msgr2=0x7f73fc14b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:49.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.337+0000 7f73fbfff700 1 --2- 172.21.15.47:0/1026667029 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f73fc0a72f0 0x7f73fc14b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:49.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.337+0000 7f73fbfff700 1 -- 172.21.15.47:0/1026667029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73fc0b5c60 msgr2=0x7f73fc143580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:49.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.337+0000 7f73fbfff700 1 --2- 172.21.15.47:0/1026667029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73fc0b5c60 0x7f73fc143580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:49.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.337+0000 7f73fbfff700 1 -- 172.21.15.47:0/1026667029 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f4009470 con 0x7f73fc060eb0 2022-06-29T15:42:49.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.337+0000 7f7400978700 1 --2- 172.21.15.47:0/1026667029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73fc0b5c60 0x7f73fc143580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:42:49.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.341+0000 7f73fbfff700 1 --2- 172.21.15.47:0/1026667029 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73fc060eb0 0x7f73fc14bf10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f73ec00a000 tx=0x7f73ec008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:49.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.341+0000 7f73f9ffb700 1 -- 172.21.15.47:0/1026667029 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f73ec011780 con 0x7f73fc060eb0 2022-06-29T15:42:49.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.341+0000 7f73f9ffb700 1 -- 172.21.15.47:0/1026667029 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73ec0118e0 con 0x7f73fc060eb0 2022-06-29T15:42:49.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.341+0000 7f7401179700 1 -- 172.21.15.47:0/1026667029 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73fc142e40 con 0x7f73fc060eb0 2022-06-29T15:42:49.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.341+0000 7f73f9ffb700 1 -- 172.21.15.47:0/1026667029 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f73ec011c40 con 0x7f73fc060eb0 2022-06-29T15:42:49.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.341+0000 7f7401179700 1 -- 172.21.15.47:0/1026667029 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73fc149aa0 con 0x7f73fc060eb0 2022-06-29T15:42:49.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.341+0000 7f73f9ffb700 1 -- 172.21.15.47:0/1026667029 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f73ec01c460 con 0x7f73fc060eb0 2022-06-29T15:42:49.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.341+0000 7f7401179700 1 -- 172.21.15.47:0/1026667029 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73fc0573d0 con 0x7f73fc060eb0 2022-06-29T15:42:49.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.341+0000 7f73f9ffb700 1 --2- 172.21.15.47:0/1026667029 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f73d4053a70 0x7f73d4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:49.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.341+0000 7f73f9ffb700 1 -- 172.21.15.47:0/1026667029 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f73ec06c600 con 0x7f73fc060eb0 2022-06-29T15:42:49.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.341+0000 7f73f37fe700 1 --2- 172.21.15.47:0/1026667029 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f73d4053a70 0x7f73d4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:49.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.341+0000 7f73f37fe700 1 --2- 172.21.15.47:0/1026667029 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f73d4053a70 0x7f73d4055f30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f73e400d000 tx=0x7f73e4007c20).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:49.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.349+0000 7f73f9ffb700 1 -- 172.21.15.47:0/1026667029 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f73ec043250 con 0x7f73fc060eb0 2022-06-29T15:42:49.484 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.477+0000 7f7401179700 1 -- 172.21.15.47:0/1026667029 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f73fc069e20 con 0x7f73fc060eb0 2022-06-29T15:42:49.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.477+0000 7f73f9ffb700 1 -- 172.21.15.47:0/1026667029 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f73ec046870 con 0x7f73fc060eb0 2022-06-29T15:42:49.487 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:42:49.487 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":26,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:42:34.474438+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:42:28.400446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-06-29T15:42:31.167705+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:42:49.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.481+0000 7f7401179700 1 -- 172.21.15.47:0/1026667029 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f73d4053a70 msgr2=0x7f73d4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:49.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.481+0000 7f7401179700 1 --2- 172.21.15.47:0/1026667029 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f73d4053a70 0x7f73d4055f30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f73e400d000 tx=0x7f73e4007c20).stop 2022-06-29T15:42:49.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.481+0000 7f7401179700 1 -- 172.21.15.47:0/1026667029 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73fc060eb0 msgr2=0x7f73fc14bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:49.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.481+0000 7f7401179700 1 --2- 172.21.15.47:0/1026667029 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73fc060eb0 0x7f73fc14bf10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f73ec00a000 tx=0x7f73ec008040).stop 2022-06-29T15:42:49.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.481+0000 7f7401179700 1 -- 172.21.15.47:0/1026667029 shutdown_connections 2022-06-29T15:42:49.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.481+0000 7f7401179700 1 --2- 172.21.15.47:0/1026667029 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f73d4053a70 0x7f73d4055f30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:49.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.481+0000 7f7401179700 1 --2- 172.21.15.47:0/1026667029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73fc0b5c60 0x7f73fc143580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:49.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.481+0000 7f7401179700 1 --2- 172.21.15.47:0/1026667029 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f73fc0a72f0 0x7f73fc14b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:49.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.481+0000 7f7401179700 1 --2- 172.21.15.47:0/1026667029 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73fc060eb0 0x7f73fc14bf10 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:49.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.481+0000 7f7401179700 1 -- 172.21.15.47:0/1026667029 >> 172.21.15.47:0/1026667029 conn(0x7f73fc062560 msgr2=0x7f73fc14d590 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:49.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.481+0000 7f7401179700 1 -- 172.21.15.47:0/1026667029 shutdown_connections 2022-06-29T15:42:49.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.481+0000 7f7401179700 1 -- 172.21.15.47:0/1026667029 wait complete. 2022-06-29T15:42:49.501 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-29T15:42:16.971431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-29T15:42:25.290884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-29T15:42:28.400446+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2022-06-29T15:42:31.167705+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-29T15:42:49.502 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-29T15:42:49.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.605+0000 7fcee1ef5700 1 Processor -- start 2022-06-29T15:42:49.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.605+0000 7fcee1ef5700 1 -- start start 2022-06-29T15:42:49.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.605+0000 7fcee1ef5700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcedc0ab040 0x7fcedc066a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:49.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.605+0000 7fcee1ef5700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcedc144020 0x7fcedc1414a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:49.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.605+0000 7fcee1ef5700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcedc0676d0 con 0x7fcedc144e30 2022-06-29T15:42:49.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.605+0000 7fcee1ef5700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcedc066f70 con 0x7fcedc144020 2022-06-29T15:42:49.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.605+0000 7fcee1ef5700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcedc0ab420 con 0x7fcedc0ab040 2022-06-29T15:42:49.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.605+0000 7fcedbfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcedc144020 0x7fcedc1414a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:49.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.605+0000 7fcedbfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcedc144020 0x7fcedc1414a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46334/0 (socket says 172.21.15.47:46334) 2022-06-29T15:42:49.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.605+0000 7fcedbfff700 1 -- 172.21.15.47:0/660741653 learned_addr learned my addr 172.21.15.47:0/660741653 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:49.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.605+0000 7fcedb7fe700 1 --2- 172.21.15.47:0/660741653 >> v2:172.21.15.64:3300/0 conn(0x7fcedc0ab040 0x7fcedc066a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:49.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.605+0000 7fcedbfff700 1 -- 172.21.15.47:0/660741653 >> v2:172.21.15.64:3300/0 conn(0x7fcedc0ab040 msgr2=0x7fcedc066a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:49.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.605+0000 7fcedbfff700 1 --2- 172.21.15.47:0/660741653 >> v2:172.21.15.64:3300/0 conn(0x7fcedc0ab040 0x7fcedc066a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:49.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.605+0000 7fcedbfff700 1 -- 172.21.15.47:0/660741653 >> v1:172.21.15.64:6789/0 conn(0x7fcedc144e30 legacy=0x7fcedc143910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:49.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.605+0000 7fcedbfff700 1 -- 172.21.15.47:0/660741653 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcedc067190 con 0x7fcedc144020 2022-06-29T15:42:49.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.605+0000 7fcedbfff700 1 --2- 172.21.15.47:0/660741653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcedc144020 0x7fcedc1414a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7fced40032f0 tx=0x7fced4009290).ready entity=mon.0 client_cookie=cc9027fd6625aa19 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:49.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.609+0000 7fceda7fc700 1 -- 172.21.15.47:0/660741653 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fced400b870 con 0x7fcedc144020 2022-06-29T15:42:49.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.609+0000 7fceda7fc700 1 -- 172.21.15.47:0/660741653 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fced400b9d0 con 0x7fcedc144020 2022-06-29T15:42:49.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.609+0000 7fceda7fc700 1 -- 172.21.15.47:0/660741653 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fced400bb40 con 0x7fcedc144020 2022-06-29T15:42:49.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.609+0000 7fcee1ef5700 1 -- 172.21.15.47:0/660741653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcedc144020 msgr2=0x7fcedc1414a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:49.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.609+0000 7fcee1ef5700 1 --2- 172.21.15.47:0/660741653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcedc144020 0x7fcedc1414a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7fced40032f0 tx=0x7fced4009290).stop 2022-06-29T15:42:49.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.609+0000 7fcee1ef5700 1 -- 172.21.15.47:0/660741653 shutdown_connections 2022-06-29T15:42:49.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.609+0000 7fcee1ef5700 1 --2- 172.21.15.47:0/660741653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcedc144020 0x7fcedc1414a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:49.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.609+0000 7fcee1ef5700 1 --2- 172.21.15.47:0/660741653 >> v2:172.21.15.64:3300/0 conn(0x7fcedc0ab040 0x7fcedc066a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:49.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.609+0000 7fcee1ef5700 1 -- 172.21.15.47:0/660741653 >> 172.21.15.47:0/660741653 conn(0x7fcedc05c510 msgr2=0x7fcedc05c920 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:49.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.609+0000 7fcee1ef5700 1 -- 172.21.15.47:0/660741653 shutdown_connections 2022-06-29T15:42:49.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.609+0000 7fcee1ef5700 1 -- 172.21.15.47:0/660741653 wait complete. 2022-06-29T15:42:49.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.609+0000 7fcee1ef5700 1 Processor -- start 2022-06-29T15:42:49.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.609+0000 7fcee1ef5700 1 -- start start 2022-06-29T15:42:49.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.613+0000 7fcee1ef5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcedc0ab040 0x7fcedc0b28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:49.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.613+0000 7fcee1ef5700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcedc144020 0x7fcedc0b43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:49.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.613+0000 7fcee1ef5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcedc144e30 0x7fcedc13c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:49.626 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.613+0000 7fcee1ef5700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc0af850 con 0x7fcedc144e30 2022-06-29T15:42:49.626 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.613+0000 7fcee1ef5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc0b0eb0 con 0x7fcedc0ab040 2022-06-29T15:42:49.626 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.613+0000 7fcee1ef5700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc0b2080 con 0x7fcedc144020 2022-06-29T15:42:49.626 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.613+0000 7fcedbfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcedc144e30 0x7fcedc13c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:49.627 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.613+0000 7fcedbfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcedc144e30 0x7fcedc13c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46340/0 (socket says 172.21.15.47:46340) 2022-06-29T15:42:49.627 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.613+0000 7fcedbfff700 1 -- 172.21.15.47:0/962338859 learned_addr learned my addr 172.21.15.47:0/962338859 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:49.627 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.613+0000 7fcedb7fe700 1 --2- 172.21.15.47:0/962338859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcedc0ab040 0x7fcedc0b28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:49.627 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.613+0000 7fcedaffd700 1 --2- 172.21.15.47:0/962338859 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcedc144020 0x7fcedc0b43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:49.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.613+0000 7fcedbfff700 1 -- 172.21.15.47:0/962338859 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcedc144020 msgr2=0x7fcedc0b43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:49.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.613+0000 7fcedbfff700 1 --2- 172.21.15.47:0/962338859 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcedc144020 0x7fcedc0b43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:49.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.613+0000 7fcedbfff700 1 -- 172.21.15.47:0/962338859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcedc0ab040 msgr2=0x7fcedc0b28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:49.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.613+0000 7fcedbfff700 1 --2- 172.21.15.47:0/962338859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcedc0ab040 0x7fcedc0b28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:49.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.613+0000 7fcedbfff700 1 -- 172.21.15.47:0/962338859 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced4009530 con 0x7fcedc144e30 2022-06-29T15:42:49.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.613+0000 7fcedaffd700 1 --2- 172.21.15.47:0/962338859 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcedc144020 0x7fcedc0b43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:42:49.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.613+0000 7fcedb7fe700 1 --2- 172.21.15.47:0/962338859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcedc0ab040 0x7fcedc0b28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:42:49.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.613+0000 7fcedbfff700 1 --2- 172.21.15.47:0/962338859 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcedc144e30 0x7fcedc13c580 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7fcedc0aa460 tx=0x7fced4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:49.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.613+0000 7fced8ff9700 1 -- 172.21.15.47:0/962338859 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fced400b870 con 0x7fcedc144e30 2022-06-29T15:42:49.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.613+0000 7fcee1ef5700 1 -- 172.21.15.47:0/962338859 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcedc0b3680 con 0x7fcedc144e30 2022-06-29T15:42:49.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.613+0000 7fced8ff9700 1 -- 172.21.15.47:0/962338859 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fced4021440 con 0x7fcedc144e30 2022-06-29T15:42:49.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.613+0000 7fcee1ef5700 1 -- 172.21.15.47:0/962338859 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcedc0aeed0 con 0x7fcedc144e30 2022-06-29T15:42:49.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.613+0000 7fced8ff9700 1 -- 172.21.15.47:0/962338859 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fced40217a0 con 0x7fcedc144e30 2022-06-29T15:42:49.632 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.617+0000 7fcee1ef5700 1 -- 172.21.15.47:0/962338859 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcedc0b3810 con 0x7fcedc144e30 2022-06-29T15:42:49.632 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.617+0000 7fced8ff9700 1 -- 172.21.15.47:0/962338859 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fced4029070 con 0x7fcedc144e30 2022-06-29T15:42:49.632 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.617+0000 7fced8ff9700 1 --2- 172.21.15.47:0/962338859 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcebc0539e0 0x7fcebc055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:49.632 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.617+0000 7fced8ff9700 1 -- 172.21.15.47:0/962338859 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fced406d270 con 0x7fcedc144e30 2022-06-29T15:42:49.633 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.617+0000 7fcedb7fe700 1 --2- 172.21.15.47:0/962338859 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcebc0539e0 0x7fcebc055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:49.633 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.617+0000 7fcedb7fe700 1 --2- 172.21.15.47:0/962338859 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcebc0539e0 0x7fcebc055ea0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fcecc00b000 tx=0x7fcecc008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:49.633 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.625+0000 7fced8ff9700 1 -- 172.21.15.47:0/962338859 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fced4044090 con 0x7fcedc144e30 2022-06-29T15:42:49.761 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.749+0000 7fcee1ef5700 1 -- 172.21.15.47:0/962338859 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fcedc0573d0 con 0x7fcedc144e30 2022-06-29T15:42:49.761 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.753+0000 7fced8ff9700 1 -- 172.21.15.47:0/962338859 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v26) v1 ==== 110+0+10 (secure 0 0 0) 0x7fced4015020 con 0x7fcedc144e30 2022-06-29T15:42:49.762 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2022-06-29T15:42:49.763 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.753+0000 7fcee1ef5700 1 -- 172.21.15.47:0/962338859 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcebc0539e0 msgr2=0x7fcebc055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:49.763 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.753+0000 7fcee1ef5700 1 --2- 172.21.15.47:0/962338859 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcebc0539e0 0x7fcebc055ea0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fcecc00b000 tx=0x7fcecc008040).stop 2022-06-29T15:42:49.764 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.753+0000 7fcee1ef5700 1 -- 172.21.15.47:0/962338859 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcedc144e30 msgr2=0x7fcedc13c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:49.764 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.753+0000 7fcee1ef5700 1 --2- 172.21.15.47:0/962338859 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcedc144e30 0x7fcedc13c580 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7fcedc0aa460 tx=0x7fced4016000).stop 2022-06-29T15:42:49.764 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.753+0000 7fcee1ef5700 1 -- 172.21.15.47:0/962338859 shutdown_connections 2022-06-29T15:42:49.764 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.753+0000 7fcee1ef5700 1 --2- 172.21.15.47:0/962338859 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcedc144e30 0x7fcedc13c580 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:49.765 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.753+0000 7fcee1ef5700 1 --2- 172.21.15.47:0/962338859 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcedc144020 0x7fcedc0b43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:49.765 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.753+0000 7fcee1ef5700 1 --2- 172.21.15.47:0/962338859 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcebc0539e0 0x7fcebc055ea0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:49.765 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.753+0000 7fcee1ef5700 1 --2- 172.21.15.47:0/962338859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcedc0ab040 0x7fcedc0b28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:49.765 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.753+0000 7fcee1ef5700 1 -- 172.21.15.47:0/962338859 >> 172.21.15.47:0/962338859 conn(0x7fcedc05c510 msgr2=0x7fcedc1403a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:49.766 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.753+0000 7fcee1ef5700 1 -- 172.21.15.47:0/962338859 shutdown_connections 2022-06-29T15:42:49.766 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.753+0000 7fcee1ef5700 1 -- 172.21.15.47:0/962338859 wait complete. 2022-06-29T15:42:49.774 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-29T15:42:49.884 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.877+0000 7f65b92af700 1 Processor -- start 2022-06-29T15:42:49.884 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.877+0000 7f65b92af700 1 -- start start 2022-06-29T15:42:49.885 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.877+0000 7f65b92af700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f65b40b4300 0x7f65b40612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:49.885 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.877+0000 7f65b92af700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f65b41471a0 0x7f65b4066f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:49.886 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.877+0000 7f65b92af700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65b4067610 con 0x7f65b4060ee0 2022-06-29T15:42:49.886 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.877+0000 7f65b92af700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65b41467c0 con 0x7f65b40b4300 2022-06-29T15:42:49.886 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.877+0000 7f65b92af700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65b4146a60 con 0x7f65b41471a0 2022-06-29T15:42:49.886 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.877+0000 7f65b27fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f65b40b4300 0x7f65b40612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:49.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.877+0000 7f65b27fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f65b40b4300 0x7f65b40612c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46346/0 (socket says 172.21.15.47:46346) 2022-06-29T15:42:49.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.877+0000 7f65b37fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f65b41471a0 0x7f65b4066f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:49.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.877+0000 7f65b27fc700 1 -- 172.21.15.47:0/87862904 learned_addr learned my addr 172.21.15.47:0/87862904 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:49.889 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.877+0000 7f65b27fc700 1 -- 172.21.15.47:0/87862904 >> v2:172.21.15.64:3300/0 conn(0x7f65b41471a0 msgr2=0x7f65b4066f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:49.889 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.877+0000 7f65b27fc700 1 --2- 172.21.15.47:0/87862904 >> v2:172.21.15.64:3300/0 conn(0x7f65b41471a0 0x7f65b4066f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:49.889 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.877+0000 7f65b27fc700 1 -- 172.21.15.47:0/87862904 >> v1:172.21.15.64:6789/0 conn(0x7f65b4060ee0 legacy=0x7f65b41430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:49.890 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.877+0000 7f65b27fc700 1 -- 172.21.15.47:0/87862904 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65b4146d00 con 0x7f65b40b4300 2022-06-29T15:42:49.890 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.877+0000 7f65b27fc700 1 --2- 172.21.15.47:0/87862904 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65b40b4300 0x7f65b40612c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f65b40ae790 tx=0x7f659c006eb0).ready entity=mon.0 client_cookie=fc419e3e7ef58e74 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:49.890 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.877+0000 7f65b1ffb700 1 -- 172.21.15.47:0/87862904 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f659c00d610 con 0x7f65b40b4300 2022-06-29T15:42:49.890 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.877+0000 7f65b1ffb700 1 -- 172.21.15.47:0/87862904 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659c000e90 con 0x7f65b40b4300 2022-06-29T15:42:49.891 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.877+0000 7f65b1ffb700 1 -- 172.21.15.47:0/87862904 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f659c00d960 con 0x7f65b40b4300 2022-06-29T15:42:49.891 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.877+0000 7f65b92af700 1 -- 172.21.15.47:0/87862904 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65b40b4300 msgr2=0x7f65b40612c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:49.891 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.877+0000 7f65b92af700 1 --2- 172.21.15.47:0/87862904 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65b40b4300 0x7f65b40612c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f65b40ae790 tx=0x7f659c006eb0).stop 2022-06-29T15:42:49.891 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.877+0000 7f65b92af700 1 -- 172.21.15.47:0/87862904 shutdown_connections 2022-06-29T15:42:49.891 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.877+0000 7f65b92af700 1 --2- 172.21.15.47:0/87862904 >> v2:172.21.15.64:3300/0 conn(0x7f65b41471a0 0x7f65b4066f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:49.892 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.877+0000 7f65b92af700 1 --2- 172.21.15.47:0/87862904 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65b40b4300 0x7f65b40612c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:49.892 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.877+0000 7f65b92af700 1 -- 172.21.15.47:0/87862904 >> 172.21.15.47:0/87862904 conn(0x7f65b405c510 msgr2=0x7f65b405f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:49.892 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.877+0000 7f65b92af700 1 -- 172.21.15.47:0/87862904 shutdown_connections 2022-06-29T15:42:49.892 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.881+0000 7f65b92af700 1 -- 172.21.15.47:0/87862904 wait complete. 2022-06-29T15:42:49.892 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.881+0000 7f65b92af700 1 Processor -- start 2022-06-29T15:42:49.893 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.881+0000 7f65b92af700 1 -- start start 2022-06-29T15:42:49.896 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.881+0000 7f65b92af700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65b4060ee0 0x7f65b413e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:49.896 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.881+0000 7f65b92af700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f65b40b4300 0x7f65b4153340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:49.896 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.881+0000 7f65b92af700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65b41471a0 0x7f65b4112e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:49.896 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.881+0000 7f65b92af700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b40b2b50 con 0x7f65b4060ee0 2022-06-29T15:42:49.897 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.881+0000 7f65b92af700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b40b3a10 con 0x7f65b41471a0 2022-06-29T15:42:49.897 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.881+0000 7f65b92af700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65b413ecf0 con 0x7f65b40b4300 2022-06-29T15:42:49.897 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.881+0000 7f65b2ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65b4060ee0 0x7f65b413e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:49.897 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.881+0000 7f65b2ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65b4060ee0 0x7f65b413e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46350/0 (socket says 172.21.15.47:46350) 2022-06-29T15:42:49.898 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.881+0000 7f65b2ffd700 1 -- 172.21.15.47:0/231680558 learned_addr learned my addr 172.21.15.47:0/231680558 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:49.898 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.881+0000 7f65b37fe700 1 --2- 172.21.15.47:0/231680558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65b41471a0 0x7f65b4112e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:49.898 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.881+0000 7f65b2ffd700 1 -- 172.21.15.47:0/231680558 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f65b40b4300 msgr2=0x7f65b4153340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:49.898 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.881+0000 7f65b27fc700 1 --2- 172.21.15.47:0/231680558 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f65b40b4300 0x7f65b4153340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:49.899 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.881+0000 7f65b2ffd700 1 --2- 172.21.15.47:0/231680558 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f65b40b4300 0x7f65b4153340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:49.899 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.881+0000 7f65b2ffd700 1 -- 172.21.15.47:0/231680558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65b41471a0 msgr2=0x7f65b4112e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:49.899 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.881+0000 7f65b2ffd700 1 --2- 172.21.15.47:0/231680558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65b41471a0 0x7f65b4112e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:49.899 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.881+0000 7f65b2ffd700 1 -- 172.21.15.47:0/231680558 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f659c00b1e0 con 0x7f65b4060ee0 2022-06-29T15:42:49.899 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.881+0000 7f65b37fe700 1 --2- 172.21.15.47:0/231680558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65b41471a0 0x7f65b4112e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:42:49.900 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.881+0000 7f65b27fc700 1 --2- 172.21.15.47:0/231680558 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f65b40b4300 0x7f65b4153340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:42:49.900 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.881+0000 7f65b2ffd700 1 --2- 172.21.15.47:0/231680558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65b4060ee0 0x7f65b413e800 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f65b4066810 tx=0x7f65a4007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:49.900 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.881+0000 7f65abfff700 1 -- 172.21.15.47:0/231680558 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f65a400ce30 con 0x7f65b4060ee0 2022-06-29T15:42:49.900 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.881+0000 7f65abfff700 1 -- 172.21.15.47:0/231680558 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65a400f470 con 0x7f65b4060ee0 2022-06-29T15:42:49.901 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.881+0000 7f65b92af700 1 -- 172.21.15.47:0/231680558 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65b4153830 con 0x7f65b4060ee0 2022-06-29T15:42:49.901 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.881+0000 7f65abfff700 1 -- 172.21.15.47:0/231680558 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f65a400f7d0 con 0x7f65b4060ee0 2022-06-29T15:42:49.901 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.881+0000 7f65b92af700 1 -- 172.21.15.47:0/231680558 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65b4113380 con 0x7f65b4060ee0 2022-06-29T15:42:49.901 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.885+0000 7f65b92af700 1 -- 172.21.15.47:0/231680558 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65b413dd00 con 0x7f65b4060ee0 2022-06-29T15:42:49.901 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.885+0000 7f65abfff700 1 -- 172.21.15.47:0/231680558 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f65a4019030 con 0x7f65b4060ee0 2022-06-29T15:42:49.902 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.885+0000 7f65abfff700 1 --2- 172.21.15.47:0/231680558 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6598053980 0x7f6598055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:49.902 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.885+0000 7f65b27fc700 1 --2- 172.21.15.47:0/231680558 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6598053980 0x7f6598055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:49.902 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.885+0000 7f65abfff700 1 -- 172.21.15.47:0/231680558 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f65a406b500 con 0x7f65b4060ee0 2022-06-29T15:42:49.902 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.885+0000 7f65b27fc700 1 --2- 172.21.15.47:0/231680558 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6598053980 0x7f6598055e40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f659c016000 tx=0x7f659c015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:49.903 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:49.893+0000 7f65abfff700 1 -- 172.21.15.47:0/231680558 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f65a4042240 con 0x7f65b4060ee0 2022-06-29T15:42:50.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.029+0000 7f65b92af700 1 -- 172.21.15.47:0/231680558 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f65b40573d0 con 0x7f65b4060ee0 2022-06-29T15:42:50.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.029+0000 7f65abfff700 1 -- 172.21.15.47:0/231680558 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7f65a4045860 con 0x7f65b4060ee0 2022-06-29T15:42:50.038 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:42:50.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.029+0000 7f65b92af700 1 -- 172.21.15.47:0/231680558 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6598053980 msgr2=0x7f6598055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:50.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.029+0000 7f65b92af700 1 --2- 172.21.15.47:0/231680558 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6598053980 0x7f6598055e40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f659c016000 tx=0x7f659c015000).stop 2022-06-29T15:42:50.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.029+0000 7f65b92af700 1 -- 172.21.15.47:0/231680558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65b4060ee0 msgr2=0x7f65b413e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:50.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.029+0000 7f65b92af700 1 --2- 172.21.15.47:0/231680558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65b4060ee0 0x7f65b413e800 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f65b4066810 tx=0x7f65a4007c50).stop 2022-06-29T15:42:50.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.029+0000 7f65b92af700 1 -- 172.21.15.47:0/231680558 shutdown_connections 2022-06-29T15:42:50.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.029+0000 7f65b92af700 1 --2- 172.21.15.47:0/231680558 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6598053980 0x7f6598055e40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:50.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.029+0000 7f65b92af700 1 --2- 172.21.15.47:0/231680558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65b41471a0 0x7f65b4112e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:50.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.029+0000 7f65b92af700 1 --2- 172.21.15.47:0/231680558 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f65b40b4300 0x7f65b4153340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:50.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.029+0000 7f65b92af700 1 --2- 172.21.15.47:0/231680558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65b4060ee0 0x7f65b413e800 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:50.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.029+0000 7f65b92af700 1 -- 172.21.15.47:0/231680558 >> 172.21.15.47:0/231680558 conn(0x7f65b405c510 msgr2=0x7f65b40b8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:50.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.033+0000 7f65b92af700 1 -- 172.21.15.47:0/231680558 shutdown_connections 2022-06-29T15:42:50.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.033+0000 7f65b92af700 1 -- 172.21.15.47:0/231680558 wait complete. 2022-06-29T15:42:50.050 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-29T15:42:50.154 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.144+0000 7fc271f5d700 1 Processor -- start 2022-06-29T15:42:50.155 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.144+0000 7fc271f5d700 1 -- start start 2022-06-29T15:42:50.156 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.148+0000 7fc271f5d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc26c05f0b0 0x7fc26c0b2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:50.157 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.148+0000 7fc271f5d700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc26c099800 con 0x7fc26c060f20 2022-06-29T15:42:50.157 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.148+0000 7fc271f5d700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc26c09a340 con 0x7fc26c0b2bd0 2022-06-29T15:42:50.157 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.148+0000 7fc271f5d700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc26c0b1560 con 0x7fc26c05f0b0 2022-06-29T15:42:50.157 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.148+0000 7fc27175c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc26c060f20 0x7fc26c1403b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:60146/0 (socket says 172.21.15.47:60146) 2022-06-29T15:42:50.158 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.148+0000 7fc27175c700 1 -- 172.21.15.47:0/1435327346 learned_addr learned my addr 172.21.15.47:0/1435327346 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:50.158 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.148+0000 7fc26bfff700 1 --2- 172.21.15.47:0/1435327346 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc26c05f0b0 0x7fc26c0b2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:50.158 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.148+0000 7fc26b7fe700 1 -- 172.21.15.47:0/1435327346 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1688375309 0 0) 0x7fc26c099800 con 0x7fc26c060f20 2022-06-29T15:42:50.158 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.148+0000 7fc26b7fe700 1 -- 172.21.15.47:0/1435327346 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc258003260 con 0x7fc26c060f20 2022-06-29T15:42:50.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.148+0000 7fc26b7fe700 1 -- 172.21.15.47:0/1435327346 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 103991639 0 0) 0x7fc26c09a340 con 0x7fc26c0b2bd0 2022-06-29T15:42:50.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.148+0000 7fc26b7fe700 1 -- 172.21.15.47:0/1435327346 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc26c099800 con 0x7fc26c0b2bd0 2022-06-29T15:42:50.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.148+0000 7fc26b7fe700 1 -- 172.21.15.47:0/1435327346 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 787360893 0 0) 0x7fc258003260 con 0x7fc26c060f20 2022-06-29T15:42:50.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.148+0000 7fc26b7fe700 1 -- 172.21.15.47:0/1435327346 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc26c05f0b0 msgr2=0x7fc26c0b2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:50.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.148+0000 7fc26b7fe700 1 --2- 172.21.15.47:0/1435327346 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc26c05f0b0 0x7fc26c0b2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:50.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.148+0000 7fc26b7fe700 1 -- 172.21.15.47:0/1435327346 >> v1:172.21.15.64:6789/0 conn(0x7fc26c0b2bd0 legacy=0x7fc26c099c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:50.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.148+0000 7fc26b7fe700 1 -- 172.21.15.47:0/1435327346 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc26c143c90 con 0x7fc26c060f20 2022-06-29T15:42:50.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.148+0000 7fc26b7fe700 1 -- 172.21.15.47:0/1435327346 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7fc2640019e0 con 0x7fc26c060f20 2022-06-29T15:42:50.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.148+0000 7fc26bfff700 1 --2- 172.21.15.47:0/1435327346 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc26c05f0b0 0x7fc26c0b2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:42:50.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.148+0000 7fc26b7fe700 1 -- 172.21.15.47:0/1435327346 >> v1:172.21.15.47:6789/0 conn(0x7fc26c060f20 legacy=0x7fc26c1403b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:50.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.148+0000 7fc26b7fe700 1 --2- 172.21.15.47:0/1435327346 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc26c05f0b0 0x7fc258003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:50.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.148+0000 7fc26b7fe700 1 --2- 172.21.15.47:0/1435327346 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc258003e50 0x7fc258004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:50.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.148+0000 7fc26b7fe700 1 --2- 172.21.15.47:0/1435327346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2580049f0 0x7fc258004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:50.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.148+0000 7fc26b7fe700 1 -- 172.21.15.47:0/1435327346 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc26c0b1560 con 0x7fc258003e50 2022-06-29T15:42:50.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.148+0000 7fc26b7fe700 1 -- 172.21.15.47:0/1435327346 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25800b800 con 0x7fc2580049f0 2022-06-29T15:42:50.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.148+0000 7fc26b7fe700 1 -- 172.21.15.47:0/1435327346 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc25800b9c0 con 0x7fc26c05f0b0 2022-06-29T15:42:50.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.148+0000 7fc26bfff700 1 --2- 172.21.15.47:0/1435327346 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc258003e50 0x7fc258004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:50.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.148+0000 7fc270f5b700 1 --2- 172.21.15.47:0/1435327346 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc26c05f0b0 0x7fc258003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:50.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.148+0000 7fc27175c700 1 --2- 172.21.15.47:0/1435327346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2580049f0 0x7fc258004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:50.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.148+0000 7fc26bfff700 1 -- 172.21.15.47:0/1435327346 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc26c05f0b0 msgr2=0x7fc258003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:50.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.148+0000 7fc26bfff700 1 --2- 172.21.15.47:0/1435327346 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc26c05f0b0 0x7fc258003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:50.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.148+0000 7fc26bfff700 1 -- 172.21.15.47:0/1435327346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2580049f0 msgr2=0x7fc258004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:50.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.148+0000 7fc26bfff700 1 --2- 172.21.15.47:0/1435327346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2580049f0 0x7fc258004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:50.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.148+0000 7fc26bfff700 1 -- 172.21.15.47:0/1435327346 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc25c00b180 con 0x7fc258003e50 2022-06-29T15:42:50.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.148+0000 7fc26bfff700 1 --2- 172.21.15.47:0/1435327346 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc258003e50 0x7fc258004500 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7fc25c00c000 tx=0x7fc25c006df0).ready entity=mon.0 client_cookie=32d933ead85afa63 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:50.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.148+0000 7fc26b7fe700 1 -- 172.21.15.47:0/1435327346 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc25c00e800 con 0x7fc258003e50 2022-06-29T15:42:50.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.148+0000 7fc26b7fe700 1 -- 172.21.15.47:0/1435327346 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc25c00e960 con 0x7fc258003e50 2022-06-29T15:42:50.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.148+0000 7fc26b7fe700 1 -- 172.21.15.47:0/1435327346 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc25c00ecc0 con 0x7fc258003e50 2022-06-29T15:42:50.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.148+0000 7fc271f5d700 1 -- 172.21.15.47:0/1435327346 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc258003e50 msgr2=0x7fc258004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:50.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.148+0000 7fc271f5d700 1 --2- 172.21.15.47:0/1435327346 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc258003e50 0x7fc258004500 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7fc25c00c000 tx=0x7fc25c006df0).stop 2022-06-29T15:42:50.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.148+0000 7fc27175c700 1 -- 172.21.15.47:0/1435327346 reap_dead start 2022-06-29T15:42:50.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.152+0000 7fc271f5d700 1 -- 172.21.15.47:0/1435327346 shutdown_connections 2022-06-29T15:42:50.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.152+0000 7fc271f5d700 1 -- 172.21.15.47:0/1435327346 >> 172.21.15.47:0/1435327346 conn(0x7fc26c062df0 msgr2=0x7fc26c0631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:50.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.152+0000 7fc271f5d700 1 -- 172.21.15.47:0/1435327346 shutdown_connections 2022-06-29T15:42:50.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.152+0000 7fc271f5d700 1 -- 172.21.15.47:0/1435327346 wait complete. 2022-06-29T15:42:50.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.152+0000 7fc271f5d700 1 Processor -- start 2022-06-29T15:42:50.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.152+0000 7fc271f5d700 1 -- start start 2022-06-29T15:42:50.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.152+0000 7fc271f5d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc258003e50 0x7fc26c0ae120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:50.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.152+0000 7fc271f5d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc26c1420a0 0x7fc26c13e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:50.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.152+0000 7fc271f5d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc26c152dc0 0x7fc26c155a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:50.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.152+0000 7fc271f5d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc26c152490 con 0x7fc258003e50 2022-06-29T15:42:50.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.152+0000 7fc271f5d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc26c0af820 con 0x7fc26c152dc0 2022-06-29T15:42:50.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.152+0000 7fc271f5d700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc26c0afac0 con 0x7fc26c1420a0 2022-06-29T15:42:50.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.152+0000 7fc270f5b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc258003e50 0x7fc26c0ae120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:50.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.152+0000 7fc270f5b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc258003e50 0x7fc26c0ae120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46370/0 (socket says 172.21.15.47:46370) 2022-06-29T15:42:50.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.152+0000 7fc270f5b700 1 -- 172.21.15.47:0/811305408 learned_addr learned my addr 172.21.15.47:0/811305408 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:50.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.152+0000 7fc26bfff700 1 --2- 172.21.15.47:0/811305408 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc26c1420a0 0x7fc26c13e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:50.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.152+0000 7fc27175c700 1 --2- 172.21.15.47:0/811305408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc26c152dc0 0x7fc26c155a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:50.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.152+0000 7fc270f5b700 1 -- 172.21.15.47:0/811305408 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc26c1420a0 msgr2=0x7fc26c13e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:50.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.152+0000 7fc270f5b700 1 --2- 172.21.15.47:0/811305408 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc26c1420a0 0x7fc26c13e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:50.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.152+0000 7fc270f5b700 1 -- 172.21.15.47:0/811305408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc26c152dc0 msgr2=0x7fc26c155a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:50.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.152+0000 7fc270f5b700 1 --2- 172.21.15.47:0/811305408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc26c152dc0 0x7fc26c155a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:50.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.152+0000 7fc270f5b700 1 -- 172.21.15.47:0/811305408 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc26c152680 con 0x7fc258003e50 2022-06-29T15:42:50.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.156+0000 7fc270f5b700 1 --2- 172.21.15.47:0/811305408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc258003e50 0x7fc26c0ae120 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7fc26000b000 tx=0x7fc260009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:50.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.156+0000 7fc269ffb700 1 -- 172.21.15.47:0/811305408 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc260001570 con 0x7fc258003e50 2022-06-29T15:42:50.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.156+0000 7fc269ffb700 1 -- 172.21.15.47:0/811305408 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2600016d0 con 0x7fc258003e50 2022-06-29T15:42:50.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.156+0000 7fc271f5d700 1 -- 172.21.15.47:0/811305408 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc26c152920 con 0x7fc258003e50 2022-06-29T15:42:50.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.156+0000 7fc269ffb700 1 -- 172.21.15.47:0/811305408 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc260001a30 con 0x7fc258003e50 2022-06-29T15:42:50.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.156+0000 7fc271f5d700 1 -- 172.21.15.47:0/811305408 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc26c153270 con 0x7fc258003e50 2022-06-29T15:42:50.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.156+0000 7fc271f5d700 1 -- 172.21.15.47:0/811305408 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc26c0afcc0 con 0x7fc258003e50 2022-06-29T15:42:50.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.156+0000 7fc269ffb700 1 -- 172.21.15.47:0/811305408 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc260001b90 con 0x7fc258003e50 2022-06-29T15:42:50.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.156+0000 7fc269ffb700 1 --2- 172.21.15.47:0/811305408 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc24c053b00 0x7fc24c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:50.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.156+0000 7fc26bfff700 1 --2- 172.21.15.47:0/811305408 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc24c053b00 0x7fc24c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:50.182 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.156+0000 7fc269ffb700 1 -- 172.21.15.47:0/811305408 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fc26006c170 con 0x7fc258003e50 2022-06-29T15:42:50.182 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.160+0000 7fc26bfff700 1 --2- 172.21.15.47:0/811305408 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc24c053b00 0x7fc24c055fc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fc25c01c000 tx=0x7fc25c019000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:50.182 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.168+0000 7fc269ffb700 1 -- 172.21.15.47:0/811305408 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc260042f30 con 0x7fc258003e50 2022-06-29T15:42:50.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.284+0000 7fc271f5d700 1 -- 172.21.15.47:0/811305408 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc26c152ab0 con 0x7fc258003e50 2022-06-29T15:42:50.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.288+0000 7fc269ffb700 1 -- 172.21.15.47:0/811305408 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+10 (secure 0 0 0) 0x7fc260046550 con 0x7fc258003e50 2022-06-29T15:42:50.296 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:42:50.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.288+0000 7fc271f5d700 1 -- 172.21.15.47:0/811305408 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc24c053b00 msgr2=0x7fc24c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:50.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.288+0000 7fc271f5d700 1 --2- 172.21.15.47:0/811305408 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc24c053b00 0x7fc24c055fc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fc25c01c000 tx=0x7fc25c019000).stop 2022-06-29T15:42:50.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.288+0000 7fc271f5d700 1 -- 172.21.15.47:0/811305408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc258003e50 msgr2=0x7fc26c0ae120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:50.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.288+0000 7fc271f5d700 1 --2- 172.21.15.47:0/811305408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc258003e50 0x7fc26c0ae120 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7fc26000b000 tx=0x7fc260009040).stop 2022-06-29T15:42:50.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.288+0000 7fc271f5d700 1 -- 172.21.15.47:0/811305408 shutdown_connections 2022-06-29T15:42:50.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.288+0000 7fc271f5d700 1 --2- 172.21.15.47:0/811305408 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc24c053b00 0x7fc24c055fc0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:50.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.288+0000 7fc271f5d700 1 --2- 172.21.15.47:0/811305408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc26c152dc0 0x7fc26c155a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:50.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.288+0000 7fc271f5d700 1 --2- 172.21.15.47:0/811305408 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc26c1420a0 0x7fc26c13e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:50.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.288+0000 7fc271f5d700 1 --2- 172.21.15.47:0/811305408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc258003e50 0x7fc26c0ae120 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:50.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.288+0000 7fc271f5d700 1 -- 172.21.15.47:0/811305408 >> 172.21.15.47:0/811305408 conn(0x7fc26c062df0 msgr2=0x7fc26c0b4930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:50.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.288+0000 7fc271f5d700 1 -- 172.21.15.47:0/811305408 shutdown_connections 2022-06-29T15:42:50.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.288+0000 7fc271f5d700 1 -- 172.21.15.47:0/811305408 wait complete. 2022-06-29T15:42:50.314 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-29T15:42:50.424 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.416+0000 7fd97a4dc700 1 Processor -- start 2022-06-29T15:42:50.425 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.416+0000 7fd97a4dc700 1 -- start start 2022-06-29T15:42:50.427 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.416+0000 7fd97a4dc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd9740b5d90 0x7fd974143a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:50.427 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.416+0000 7fd97a4dc700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd97405cc30 con 0x7fd9740676a0 2022-06-29T15:42:50.427 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.416+0000 7fd97a4dc700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9741430a0 con 0x7fd97405c850 2022-06-29T15:42:50.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.416+0000 7fd97a4dc700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd974143300 con 0x7fd9740b5d90 2022-06-29T15:42:50.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.416+0000 7fd979cdb700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd9740676a0 0x7fd974140130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:60168/0 (socket says 172.21.15.47:60168) 2022-06-29T15:42:50.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.416+0000 7fd978cd9700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd9740b5d90 0x7fd974143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:50.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.416+0000 7fd979cdb700 1 -- 172.21.15.47:0/3041330769 learned_addr learned my addr 172.21.15.47:0/3041330769 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:50.429 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.416+0000 7fd973fff700 1 -- 172.21.15.47:0/3041330769 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 450676163 0 0) 0x7fd97405cc30 con 0x7fd9740676a0 2022-06-29T15:42:50.429 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.416+0000 7fd973fff700 1 -- 172.21.15.47:0/3041330769 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd960003260 con 0x7fd9740676a0 2022-06-29T15:42:50.429 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.416+0000 7fd978cd9700 1 -- 172.21.15.47:0/3041330769 >> v1:172.21.15.64:6789/0 conn(0x7fd97405c850 legacy=0x7fd9740b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:50.429 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.416+0000 7fd978cd9700 1 -- 172.21.15.47:0/3041330769 >> v1:172.21.15.47:6789/0 conn(0x7fd9740676a0 legacy=0x7fd974140130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:50.429 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.416+0000 7fd978cd9700 1 -- 172.21.15.47:0/3041330769 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9741435a0 con 0x7fd9740b5d90 2022-06-29T15:42:50.430 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.416+0000 7fd978cd9700 1 --2- 172.21.15.47:0/3041330769 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9740b5d90 0x7fd974143a40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7fd9680032f0 tx=0x7fd9680091d0).ready entity=mon.0 client_cookie=507fcc3e2a761d17 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:50.430 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.416+0000 7fd973fff700 1 -- 172.21.15.47:0/3041330769 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd96800b6c0 con 0x7fd9740b5d90 2022-06-29T15:42:50.430 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.416+0000 7fd973fff700 1 -- 172.21.15.47:0/3041330769 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd96800b820 con 0x7fd9740b5d90 2022-06-29T15:42:50.430 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.416+0000 7fd973fff700 1 -- 172.21.15.47:0/3041330769 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd96800b990 con 0x7fd9740b5d90 2022-06-29T15:42:50.433 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.420+0000 7fd97a4dc700 1 -- 172.21.15.47:0/3041330769 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9740b5d90 msgr2=0x7fd974143a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:50.433 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.420+0000 7fd97a4dc700 1 --2- 172.21.15.47:0/3041330769 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9740b5d90 0x7fd974143a40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7fd9680032f0 tx=0x7fd9680091d0).stop 2022-06-29T15:42:50.434 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.420+0000 7fd97a4dc700 1 -- 172.21.15.47:0/3041330769 shutdown_connections 2022-06-29T15:42:50.434 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.420+0000 7fd97a4dc700 1 --2- 172.21.15.47:0/3041330769 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9740b5d90 0x7fd974143a40 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:50.434 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.420+0000 7fd97a4dc700 1 -- 172.21.15.47:0/3041330769 >> 172.21.15.47:0/3041330769 conn(0x7fd974060e00 msgr2=0x7fd9740611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:50.434 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.420+0000 7fd97a4dc700 1 -- 172.21.15.47:0/3041330769 shutdown_connections 2022-06-29T15:42:50.435 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.420+0000 7fd97a4dc700 1 -- 172.21.15.47:0/3041330769 wait complete. 2022-06-29T15:42:50.435 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.420+0000 7fd97a4dc700 1 Processor -- start 2022-06-29T15:42:50.435 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.420+0000 7fd97a4dc700 1 -- start start 2022-06-29T15:42:50.435 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.420+0000 7fd97a4dc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd97405c850 0x7fd9740b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:50.436 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.420+0000 7fd97a4dc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9740676a0 0x7fd9740b85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:50.436 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.420+0000 7fd97a4dc700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd9740b5d90 0x7fd9740b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:50.436 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.420+0000 7fd97a4dc700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9740ba010 con 0x7fd9740676a0 2022-06-29T15:42:50.436 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.420+0000 7fd97a4dc700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9740b23e0 con 0x7fd97405c850 2022-06-29T15:42:50.436 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.420+0000 7fd97a4dc700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9740b98d0 con 0x7fd9740b5d90 2022-06-29T15:42:50.437 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.420+0000 7fd978cd9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9740676a0 0x7fd9740b85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:50.437 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.420+0000 7fd978cd9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9740676a0 0x7fd9740b85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46386/0 (socket says 172.21.15.47:46386) 2022-06-29T15:42:50.437 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.420+0000 7fd978cd9700 1 -- 172.21.15.47:0/3268959778 learned_addr learned my addr 172.21.15.47:0/3268959778 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:50.437 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.424+0000 7fd9794da700 1 --2- 172.21.15.47:0/3268959778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd97405c850 0x7fd9740b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:50.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.424+0000 7fd979cdb700 1 --2- 172.21.15.47:0/3268959778 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd9740b5d90 0x7fd9740b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:50.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.424+0000 7fd978cd9700 1 -- 172.21.15.47:0/3268959778 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd9740b5d90 msgr2=0x7fd9740b8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:50.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.424+0000 7fd978cd9700 1 --2- 172.21.15.47:0/3268959778 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd9740b5d90 0x7fd9740b8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:50.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.424+0000 7fd978cd9700 1 -- 172.21.15.47:0/3268959778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd97405c850 msgr2=0x7fd9740b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:50.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.424+0000 7fd978cd9700 1 --2- 172.21.15.47:0/3268959778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd97405c850 0x7fd9740b3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:50.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.424+0000 7fd978cd9700 1 -- 172.21.15.47:0/3268959778 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd968009470 con 0x7fd9740676a0 2022-06-29T15:42:50.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.424+0000 7fd9794da700 1 --2- 172.21.15.47:0/3268959778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd97405c850 0x7fd9740b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:42:50.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.424+0000 7fd979cdb700 1 --2- 172.21.15.47:0/3268959778 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd9740b5d90 0x7fd9740b8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:42:50.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.424+0000 7fd978cd9700 1 --2- 172.21.15.47:0/3268959778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9740676a0 0x7fd9740b85c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7fd9680032f0 tx=0x7fd968016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:50.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.424+0000 7fd9727fc700 1 -- 172.21.15.47:0/3268959778 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd96800b6c0 con 0x7fd9740676a0 2022-06-29T15:42:50.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.424+0000 7fd9727fc700 1 -- 172.21.15.47:0/3268959778 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9680107a0 con 0x7fd9740676a0 2022-06-29T15:42:50.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.424+0000 7fd97a4dc700 1 -- 172.21.15.47:0/3268959778 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9740b9b70 con 0x7fd9740676a0 2022-06-29T15:42:50.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.424+0000 7fd9727fc700 1 -- 172.21.15.47:0/3268959778 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd968010b00 con 0x7fd9740676a0 2022-06-29T15:42:50.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.424+0000 7fd97a4dc700 1 -- 172.21.15.47:0/3268959778 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd97412bdc0 con 0x7fd9740676a0 2022-06-29T15:42:50.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.424+0000 7fd9727fc700 1 -- 172.21.15.47:0/3268959778 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd968027020 con 0x7fd9740676a0 2022-06-29T15:42:50.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.424+0000 7fd97a4dc700 1 -- 172.21.15.47:0/3268959778 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9740b9d00 con 0x7fd9740676a0 2022-06-29T15:42:50.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.424+0000 7fd9727fc700 1 --2- 172.21.15.47:0/3268959778 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd954053950 0x7fd954055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:50.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.424+0000 7fd9727fc700 1 -- 172.21.15.47:0/3268959778 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fd96806d510 con 0x7fd9740676a0 2022-06-29T15:42:50.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.424+0000 7fd9794da700 1 --2- 172.21.15.47:0/3268959778 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd954053950 0x7fd954055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:50.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.424+0000 7fd9794da700 1 --2- 172.21.15.47:0/3268959778 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd954053950 0x7fd954055e10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fd964008000 tx=0x7fd964009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:50.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.432+0000 7fd9727fc700 1 -- 172.21.15.47:0/3268959778 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd9680442d0 con 0x7fd9740676a0 2022-06-29T15:42:50.567 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.556+0000 7fd97a4dc700 1 -- 172.21.15.47:0/3268959778 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd9740b2120 con 0x7fd9740676a0 2022-06-29T15:42:50.568 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.560+0000 7fd9727fc700 1 -- 172.21.15.47:0/3268959778 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+10 (secure 0 0 0) 0x7fd9680478f0 con 0x7fd9740676a0 2022-06-29T15:42:50.568 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:42:50.570 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.560+0000 7fd97a4dc700 1 -- 172.21.15.47:0/3268959778 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd954053950 msgr2=0x7fd954055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:50.571 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.560+0000 7fd97a4dc700 1 --2- 172.21.15.47:0/3268959778 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd954053950 0x7fd954055e10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fd964008000 tx=0x7fd964009040).stop 2022-06-29T15:42:50.571 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.560+0000 7fd97a4dc700 1 -- 172.21.15.47:0/3268959778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9740676a0 msgr2=0x7fd9740b85c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:50.571 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.560+0000 7fd97a4dc700 1 --2- 172.21.15.47:0/3268959778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9740676a0 0x7fd9740b85c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7fd9680032f0 tx=0x7fd968016000).stop 2022-06-29T15:42:50.571 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.560+0000 7fd97a4dc700 1 -- 172.21.15.47:0/3268959778 shutdown_connections 2022-06-29T15:42:50.571 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.560+0000 7fd97a4dc700 1 --2- 172.21.15.47:0/3268959778 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd9740b5d90 0x7fd9740b8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:50.572 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.560+0000 7fd97a4dc700 1 --2- 172.21.15.47:0/3268959778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9740676a0 0x7fd9740b85c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:50.572 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.560+0000 7fd97a4dc700 1 --2- 172.21.15.47:0/3268959778 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd954053950 0x7fd954055e10 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:50.572 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.560+0000 7fd97a4dc700 1 --2- 172.21.15.47:0/3268959778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd97405c850 0x7fd9740b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:50.572 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.560+0000 7fd97a4dc700 1 -- 172.21.15.47:0/3268959778 >> 172.21.15.47:0/3268959778 conn(0x7fd974060e00 msgr2=0x7fd974145000 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:50.573 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.560+0000 7fd97a4dc700 1 -- 172.21.15.47:0/3268959778 shutdown_connections 2022-06-29T15:42:50.573 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.560+0000 7fd97a4dc700 1 -- 172.21.15.47:0/3268959778 wait complete. 2022-06-29T15:42:50.581 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-29T15:42:50.582 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:42:50.689 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.680+0000 7f76f4942700 1 Processor -- start 2022-06-29T15:42:50.689 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.680+0000 7f76f4942700 1 -- start start 2022-06-29T15:42:50.692 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.680+0000 7f76f4942700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f76f00b4420 0x7f76f00b2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:50.692 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.680+0000 7f76f4942700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f76f0067630 0x7f76f0066ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:50.692 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.680+0000 7f76f4942700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f76f0066fb0 0x7f76f0144730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:50.693 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.680+0000 7f76f4942700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76f00b3300 con 0x7f76f0066fb0 2022-06-29T15:42:50.693 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.680+0000 7f76f4942700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76f0067460 con 0x7f76f0067630 2022-06-29T15:42:50.693 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.680+0000 7f76f4942700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76f00b4800 con 0x7f76f00b4420 2022-06-29T15:42:50.693 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.680+0000 7f76eed9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f76f0066fb0 0x7f76f0144730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:50.693 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.680+0000 7f76ee59c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f76f00b4420 0x7f76f00b2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:50.694 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.680+0000 7f76edd9b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f76f0067630 0x7f76f0066ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:50.694 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.680+0000 7f76ee59c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f76f00b4420 0x7f76f00b2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:43494/0 (socket says 172.21.15.47:43494) 2022-06-29T15:42:50.694 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.680+0000 7f76ee59c700 1 -- 172.21.15.47:0/3808873714 learned_addr learned my addr 172.21.15.47:0/3808873714 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:50.694 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.680+0000 7f76eed9d700 1 -- 172.21.15.47:0/3808873714 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f76f00b4420 msgr2=0x7f76f00b2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:50.695 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.680+0000 7f76eed9d700 1 --2- 172.21.15.47:0/3808873714 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f76f00b4420 0x7f76f00b2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:50.695 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.680+0000 7f76eed9d700 1 -- 172.21.15.47:0/3808873714 >> v2:172.21.15.64:3300/0 conn(0x7f76f0067630 msgr2=0x7f76f0066ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:50.695 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.680+0000 7f76eed9d700 1 --2- 172.21.15.47:0/3808873714 >> v2:172.21.15.64:3300/0 conn(0x7f76f0067630 0x7f76f0066ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:50.695 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.680+0000 7f76eed9d700 1 -- 172.21.15.47:0/3808873714 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76f00b34f0 con 0x7f76f0066fb0 2022-06-29T15:42:50.696 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.680+0000 7f76edd9b700 1 --2- 172.21.15.47:0/3808873714 >> v2:172.21.15.64:3300/0 conn(0x7f76f0067630 0x7f76f0066ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:42:50.696 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.680+0000 7f76ee59c700 1 --2- 172.21.15.47:0/3808873714 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f76f00b4420 0x7f76f00b2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:42:50.696 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.680+0000 7f76eed9d700 1 --2- 172.21.15.47:0/3808873714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76f0066fb0 0x7f76f0144730 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f76e0002f70 tx=0x7f76e0006df0).ready entity=mon.0 client_cookie=62fc4792b58e55a0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:50.696 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.680+0000 7f76ed59a700 1 -- 172.21.15.47:0/3808873714 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76e000d4a0 con 0x7f76f0066fb0 2022-06-29T15:42:50.697 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.680+0000 7f76ed59a700 1 -- 172.21.15.47:0/3808873714 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76e0000e90 con 0x7f76f0066fb0 2022-06-29T15:42:50.697 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.680+0000 7f76ed59a700 1 -- 172.21.15.47:0/3808873714 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76e000d7f0 con 0x7f76f0066fb0 2022-06-29T15:42:50.697 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.680+0000 7f76f4942700 1 -- 172.21.15.47:0/3808873714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76f0066fb0 msgr2=0x7f76f0144730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:50.697 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.680+0000 7f76f4942700 1 --2- 172.21.15.47:0/3808873714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76f0066fb0 0x7f76f0144730 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f76e0002f70 tx=0x7f76e0006df0).stop 2022-06-29T15:42:50.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.680+0000 7f76f4942700 1 -- 172.21.15.47:0/3808873714 shutdown_connections 2022-06-29T15:42:50.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.680+0000 7f76f4942700 1 --2- 172.21.15.47:0/3808873714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76f0066fb0 0x7f76f0144730 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:50.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.680+0000 7f76f4942700 1 --2- 172.21.15.47:0/3808873714 >> v2:172.21.15.64:3300/0 conn(0x7f76f0067630 0x7f76f0066ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:50.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.680+0000 7f76f4942700 1 --2- 172.21.15.47:0/3808873714 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f76f00b4420 0x7f76f00b2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:50.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.680+0000 7f76f4942700 1 -- 172.21.15.47:0/3808873714 >> 172.21.15.47:0/3808873714 conn(0x7f76f005c510 msgr2=0x7f76f005c920 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:50.699 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.680+0000 7f76f4942700 1 -- 172.21.15.47:0/3808873714 shutdown_connections 2022-06-29T15:42:50.699 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.680+0000 7f76f4942700 1 -- 172.21.15.47:0/3808873714 wait complete. 2022-06-29T15:42:50.702 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.684+0000 7f76f4942700 1 Processor -- start 2022-06-29T15:42:50.702 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.684+0000 7f76f4942700 1 -- start start 2022-06-29T15:42:50.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.684+0000 7f76f4942700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76f0066fb0 0x7f76f0146110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:50.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.684+0000 7f76f4942700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76f0067630 0x7f76f01570d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:50.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.684+0000 7f76f4942700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f76f00b4420 0x7f76f0060ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:50.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.684+0000 7f76f4942700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f005f0b0 con 0x7f76f0067630 2022-06-29T15:42:50.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.684+0000 7f76f4942700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f005f2e0 con 0x7f76f0066fb0 2022-06-29T15:42:50.704 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.684+0000 7f76f4942700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76f005f580 con 0x7f76f00b4420 2022-06-29T15:42:50.704 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.684+0000 7f76edd9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76f0067630 0x7f76f01570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:50.704 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.684+0000 7f76edd9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76f0067630 0x7f76f01570d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46400/0 (socket says 172.21.15.47:46400) 2022-06-29T15:42:50.704 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.684+0000 7f76edd9b700 1 -- 172.21.15.47:0/3865253624 learned_addr learned my addr 172.21.15.47:0/3865253624 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:50.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.684+0000 7f76eed9d700 1 --2- 172.21.15.47:0/3865253624 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f76f00b4420 0x7f76f0060ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:50.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.684+0000 7f76ee59c700 1 --2- 172.21.15.47:0/3865253624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76f0066fb0 0x7f76f0146110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:50.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.684+0000 7f76edd9b700 1 -- 172.21.15.47:0/3865253624 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f76f00b4420 msgr2=0x7f76f0060ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:50.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.684+0000 7f76edd9b700 1 --2- 172.21.15.47:0/3865253624 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f76f00b4420 0x7f76f0060ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:50.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.684+0000 7f76edd9b700 1 -- 172.21.15.47:0/3865253624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76f0066fb0 msgr2=0x7f76f0146110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:50.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.684+0000 7f76edd9b700 1 --2- 172.21.15.47:0/3865253624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76f0066fb0 0x7f76f0146110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:50.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.684+0000 7f76edd9b700 1 -- 172.21.15.47:0/3865253624 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e8009470 con 0x7f76f0067630 2022-06-29T15:42:50.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.684+0000 7f76ee59c700 1 --2- 172.21.15.47:0/3865253624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76f0066fb0 0x7f76f0146110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:42:50.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.684+0000 7f76edd9b700 1 --2- 172.21.15.47:0/3865253624 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76f0067630 0x7f76f01570d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f76e800c000 tx=0x7f76e80091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:50.707 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.684+0000 7f76e77fe700 1 -- 172.21.15.47:0/3865253624 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76e800c920 con 0x7f76f0067630 2022-06-29T15:42:50.707 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.684+0000 7f76e77fe700 1 -- 172.21.15.47:0/3865253624 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76e800ca80 con 0x7f76f0067630 2022-06-29T15:42:50.707 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.684+0000 7f76f4942700 1 -- 172.21.15.47:0/3865253624 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76d800b1e0 con 0x7f76f0067630 2022-06-29T15:42:50.707 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.684+0000 7f76e77fe700 1 -- 172.21.15.47:0/3865253624 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76e800cde0 con 0x7f76f0067630 2022-06-29T15:42:50.708 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.684+0000 7f76f4942700 1 -- 172.21.15.47:0/3865253624 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76e000b180 con 0x7f76f0067630 2022-06-29T15:42:50.708 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.688+0000 7f76f4942700 1 -- 172.21.15.47:0/3865253624 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76f00573d0 con 0x7f76f0067630 2022-06-29T15:42:50.708 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.688+0000 7f76e77fe700 1 -- 172.21.15.47:0/3865253624 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f76e8022070 con 0x7f76f0067630 2022-06-29T15:42:50.708 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.688+0000 7f76e77fe700 1 --2- 172.21.15.47:0/3865253624 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f76d40539e0 0x7f76d4055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:50.708 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.688+0000 7f76e77fe700 1 -- 172.21.15.47:0/3865253624 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f76e806c870 con 0x7f76f0067630 2022-06-29T15:42:50.709 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.688+0000 7f76ee59c700 1 --2- 172.21.15.47:0/3865253624 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f76d40539e0 0x7f76d4055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:50.709 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.688+0000 7f76ee59c700 1 --2- 172.21.15.47:0/3865253624 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f76d40539e0 0x7f76d4055ea0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f76d800b000 tx=0x7f76d8008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:50.709 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.692+0000 7f76e77fe700 1 -- 172.21.15.47:0/3865253624 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f76e8043630 con 0x7f76f0067630 2022-06-29T15:42:50.869 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.860+0000 7f76f4942700 1 -- 172.21.15.47:0/3865253624 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f76f005fbd0 con 0x7f76f0067630 2022-06-29T15:42:50.869 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.860+0000 7f76e77fe700 1 -- 172.21.15.47:0/3865253624 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+2483 (secure 0 0 0) 0x7f76e8046c50 con 0x7f76f0067630 2022-06-29T15:42:50.870 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:42:50.870 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/438289599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":438289599},{"type":"v1","addr":"172.21.15.47:6835","nonce":438289599}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.64:6833/1472525731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1472525731},{"type":"v1","addr":"172.21.15.64:6833","nonce":1472525731}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":4}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:42:33.471491+0000","modified":"2022-06-29T15:42:35.146757+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":54128},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54128":{"gid":54128,"name":"a","rank":0,"incarnation":5,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6837/354581778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":354581778},{"type":"v1","addr":"172.21.15.47:6837","nonce":354581778}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2022-06-29T15:42:50.873 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.864+0000 7f76f4942700 1 -- 172.21.15.47:0/3865253624 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f76d40539e0 msgr2=0x7f76d4055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:50.874 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.864+0000 7f76f4942700 1 --2- 172.21.15.47:0/3865253624 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f76d40539e0 0x7f76d4055ea0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f76d800b000 tx=0x7f76d8008040).stop 2022-06-29T15:42:50.874 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.864+0000 7f76f4942700 1 -- 172.21.15.47:0/3865253624 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76f0067630 msgr2=0x7f76f01570d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:50.874 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.864+0000 7f76f4942700 1 --2- 172.21.15.47:0/3865253624 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76f0067630 0x7f76f01570d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f76e800c000 tx=0x7f76e80091d0).stop 2022-06-29T15:42:50.874 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.864+0000 7f76f4942700 1 -- 172.21.15.47:0/3865253624 shutdown_connections 2022-06-29T15:42:50.874 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.864+0000 7f76f4942700 1 --2- 172.21.15.47:0/3865253624 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f76f00b4420 0x7f76f0060ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:50.875 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.864+0000 7f76f4942700 1 --2- 172.21.15.47:0/3865253624 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76f0067630 0x7f76f01570d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:50.875 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.864+0000 7f76f4942700 1 --2- 172.21.15.47:0/3865253624 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f76d40539e0 0x7f76d4055ea0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:50.875 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.864+0000 7f76f4942700 1 --2- 172.21.15.47:0/3865253624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76f0066fb0 0x7f76f0146110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:50.875 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.864+0000 7f76f4942700 1 -- 172.21.15.47:0/3865253624 >> 172.21.15.47:0/3865253624 conn(0x7f76f005c510 msgr2=0x7f76f014cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:50.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.864+0000 7f76f4942700 1 -- 172.21.15.47:0/3865253624 shutdown_connections 2022-06-29T15:42:50.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:50.864+0000 7f76f4942700 1 -- 172.21.15.47:0/3865253624 wait complete. 2022-06-29T15:42:50.876 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 7 2022-06-29T15:42:51.192 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-29T15:42:33.471491+0000', 'modified': '2022-06-29T15:42:35.146757+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 54128}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_54128': {'gid': 54128, 'name': 'a', 'rank': 0, 'incarnation': 5, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.47:6837/354581778', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6836', 'nonce': 354581778}, {'type': 'v1', 'addr': '172.21.15.47:6837', 'nonce': 354581778}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-29T15:42:51.192 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-29T15:42:51.193 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-06-29T15:42:51.307 INFO:teuthology.orchestra.run.smithi047.stdout:{"cluster_fsid":"9a68894a-53ce-400e-838b-14eab91b834c","whoami":0,"id":54128,"want_state":"up:active","state":"up:active","rank_uptime":17.818062626,"mdsmap_epoch":7,"osdmap_epoch":26,"osdmap_epoch_barrier":26,"uptime":24.194640901} 2022-06-29T15:42:51.308 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9a68894a-53ce-400e-838b-14eab91b834c", "id": 54128, "mdsmap_epoch": 7, "osdmap_epoch": 26, "osdmap_epoch_barrier": 26, "rank_uptime": 17.818062626, "state": "up:active", "uptime": 24.194640901, "want_state": "up:active", "whoami": 0 } 2022-06-29T15:42:51.308 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-29T15:42:51.309 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2022-06-29T15:42:51.309 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi164.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2022-06-29T15:42:51.309 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:42:51.310 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:42:51.318 DEBUG:teuthology.orchestra.run.smithi164:> sudo modprobe fuse 2022-06-29T15:42:51.337 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:42:51.337 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:42:51.393 INFO:teuthology.orchestra.run.smithi164.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:42:51.394 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:42:51.395 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:42:51.396 DEBUG:teuthology.orchestra.run.smithi164:> ls /sys/fs/fuse/connections 2022-06-29T15:42:51.401 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-29T15:42:51.402 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:42:51.404 DEBUG:teuthology.orchestra.run.smithi164:> sudo modprobe fuse 2022-06-29T15:42:51.420 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:42:51.421 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:42:51.431 INFO:teuthology.orchestra.run.smithi164.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:42:51.432 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:42:51.432 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:42:51.433 DEBUG:teuthology.orchestra.run.smithi164:> ls /sys/fs/fuse/connections 2022-06-29T15:42:51.494 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi164.stderr:2022-06-29T15:42:51.490+0000 7f7a2a87f080 -1 init, newargv = 0x55dfabe7b910 newargc=9ceph-fuse[12274 2022-06-29T15:42:51.495 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi164.stderr:]: starting ceph client 2022-06-29T15:42:51.504 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi164.stderr:ceph-fuse[12274]: starting fuse 2022-06-29T15:42:52.582 DEBUG:teuthology.orchestra.run.smithi164:> sudo modprobe fuse 2022-06-29T15:42:52.601 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:42:52.601 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:42:52.657 INFO:teuthology.orchestra.run.smithi164.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:42:52.658 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:42:52.659 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:42:52.659 DEBUG:teuthology.orchestra.run.smithi164:> ls /sys/fs/fuse/connections 2022-06-29T15:42:52.664 INFO:teuthology.orchestra.run.smithi164.stdout:52 2022-06-29T15:42:52.664 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-29T15:42:52.665 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:42:52.665 DEBUG:teuthology.orchestra.run.smithi164:> sudo python3 -c ' 2022-06-29T15:42:52.665 DEBUG:teuthology.orchestra.run.smithi164:> import glob 2022-06-29T15:42:52.666 DEBUG:teuthology.orchestra.run.smithi164:> import re 2022-06-29T15:42:52.666 DEBUG:teuthology.orchestra.run.smithi164:> import os 2022-06-29T15:42:52.666 DEBUG:teuthology.orchestra.run.smithi164:> import subprocess 2022-06-29T15:42:52.667 DEBUG:teuthology.orchestra.run.smithi164:> 2022-06-29T15:42:52.667 DEBUG:teuthology.orchestra.run.smithi164:> def find_socket(client_name): 2022-06-29T15:42:52.667 DEBUG:teuthology.orchestra.run.smithi164:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2022-06-29T15:42:52.668 DEBUG:teuthology.orchestra.run.smithi164:> files = glob.glob(asok_path) 2022-06-29T15:42:52.668 DEBUG:teuthology.orchestra.run.smithi164:> 2022-06-29T15:42:52.668 DEBUG:teuthology.orchestra.run.smithi164:> # Given a non-glob path, it better be there 2022-06-29T15:42:52.669 DEBUG:teuthology.orchestra.run.smithi164:> if "*" not in asok_path: 2022-06-29T15:42:52.669 DEBUG:teuthology.orchestra.run.smithi164:> assert(len(files) == 1) 2022-06-29T15:42:52.669 DEBUG:teuthology.orchestra.run.smithi164:> return files[0] 2022-06-29T15:42:52.670 DEBUG:teuthology.orchestra.run.smithi164:> 2022-06-29T15:42:52.670 DEBUG:teuthology.orchestra.run.smithi164:> for f in files: 2022-06-29T15:42:52.671 DEBUG:teuthology.orchestra.run.smithi164:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-29T15:42:52.671 DEBUG:teuthology.orchestra.run.smithi164:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-29T15:42:52.671 DEBUG:teuthology.orchestra.run.smithi164:> return f 2022-06-29T15:42:52.672 DEBUG:teuthology.orchestra.run.smithi164:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-29T15:42:52.672 DEBUG:teuthology.orchestra.run.smithi164:> 2022-06-29T15:42:52.672 DEBUG:teuthology.orchestra.run.smithi164:> print(find_socket("client.1")) 2022-06-29T15:42:52.673 DEBUG:teuthology.orchestra.run.smithi164:> ' 2022-06-29T15:42:52.715 INFO:teuthology.orchestra.run.smithi164.stdout:/var/run/ceph/ceph-client.1.12274.asok 2022-06-29T15:42:52.720 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.12274.asok 2022-06-29T15:42:52.721 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:42:52.721 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.12274.asok status 2022-06-29T15:42:52.811 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2022-06-29T15:42:52.812 INFO:teuthology.orchestra.run.smithi164.stdout: "metadata": { 2022-06-29T15:42:52.812 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_sha1": "f25ffac72e5a9e63860ff7deb6767a11f0559a16", 2022-06-29T15:42:52.812 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_version": "ceph version 15.2.16-304-gf25ffac7 (f25ffac72e5a9e63860ff7deb6767a11f0559a16) octopus (stable)", 2022-06-29T15:42:52.812 INFO:teuthology.orchestra.run.smithi164.stdout: "entity_id": "1", 2022-06-29T15:42:52.813 INFO:teuthology.orchestra.run.smithi164.stdout: "hostname": "smithi164", 2022-06-29T15:42:52.813 INFO:teuthology.orchestra.run.smithi164.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2022-06-29T15:42:52.813 INFO:teuthology.orchestra.run.smithi164.stdout: "pid": "12274", 2022-06-29T15:42:52.813 INFO:teuthology.orchestra.run.smithi164.stdout: "root": "/" 2022-06-29T15:42:52.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2022-06-29T15:42:52.814 INFO:teuthology.orchestra.run.smithi164.stdout: "dentry_count": 0, 2022-06-29T15:42:52.814 INFO:teuthology.orchestra.run.smithi164.stdout: "dentry_pinned_count": 0, 2022-06-29T15:42:52.814 INFO:teuthology.orchestra.run.smithi164.stdout: "id": 4666, 2022-06-29T15:42:52.814 INFO:teuthology.orchestra.run.smithi164.stdout: "inst": { 2022-06-29T15:42:52.815 INFO:teuthology.orchestra.run.smithi164.stdout: "name": { 2022-06-29T15:42:52.815 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "client", 2022-06-29T15:42:52.815 INFO:teuthology.orchestra.run.smithi164.stdout: "num": 4666 2022-06-29T15:42:52.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2022-06-29T15:42:52.815 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": { 2022-06-29T15:42:52.816 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "v1", 2022-06-29T15:42:52.816 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": "172.21.15.164:0", 2022-06-29T15:42:52.816 INFO:teuthology.orchestra.run.smithi164.stdout: "nonce": 3640325346 2022-06-29T15:42:52.816 INFO:teuthology.orchestra.run.smithi164.stdout: } 2022-06-29T15:42:52.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2022-06-29T15:42:52.817 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": { 2022-06-29T15:42:52.817 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "v1", 2022-06-29T15:42:52.817 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": "172.21.15.164:0", 2022-06-29T15:42:52.817 INFO:teuthology.orchestra.run.smithi164.stdout: "nonce": 3640325346 2022-06-29T15:42:52.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2022-06-29T15:42:52.818 INFO:teuthology.orchestra.run.smithi164.stdout: "inst_str": "client.4666 172.21.15.164:0/3640325346", 2022-06-29T15:42:52.818 INFO:teuthology.orchestra.run.smithi164.stdout: "addr_str": "172.21.15.164:0/3640325346", 2022-06-29T15:42:52.818 INFO:teuthology.orchestra.run.smithi164.stdout: "inode_count": 1, 2022-06-29T15:42:52.818 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_epoch": 7, 2022-06-29T15:42:52.819 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_epoch": 26, 2022-06-29T15:42:52.819 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_epoch_barrier": 0, 2022-06-29T15:42:52.819 INFO:teuthology.orchestra.run.smithi164.stdout: "blacklisted": false, 2022-06-29T15:42:52.819 INFO:teuthology.orchestra.run.smithi164.stdout: "fs_name": "cephfs" 2022-06-29T15:42:52.820 INFO:teuthology.orchestra.run.smithi164.stdout:} 2022-06-29T15:42:52.821 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:42:52.821 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:42:52.821 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:42:52.830 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2022-06-29T15:42:52.831 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-29T15:42:52.831 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:42:52.832 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:42:52.851 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:42:52.852 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:42:52.852 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:42:52.860 INFO:teuthology.orchestra.run.smithi164.stdout:fuseblk 2022-06-29T15:42:52.861 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-06-29T15:42:52.862 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:42:52.862 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:42:52.885 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2022-06-29T15:42:52.911 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:42:53.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.015+0000 7fa47a36d700 1 Processor -- start 2022-06-29T15:42:53.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.015+0000 7fa47a36d700 1 -- start start 2022-06-29T15:42:53.026 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.015+0000 7fa47a36d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa474061670 0x7fa474143040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:53.026 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.015+0000 7fa47a36d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa474061af0 0x7fa474143530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:53.026 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.015+0000 7fa47a36d700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4740ae370 con 0x7fa47413cfc0 2022-06-29T15:42:53.026 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.015+0000 7fa47a36d700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4740b1b90 con 0x7fa474061af0 2022-06-29T15:42:53.027 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.015+0000 7fa47a36d700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa47405f150 con 0x7fa474061670 2022-06-29T15:42:53.027 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.015+0000 7fa47890a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa474061af0 0x7fa474143530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:53.027 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.015+0000 7fa473fff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa47413cfc0 0x7fa4740b1480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:39524/0 (socket says 172.21.15.47:39524) 2022-06-29T15:42:53.027 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.015+0000 7fa47890a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa474061af0 0x7fa474143530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46408/0 (socket says 172.21.15.47:46408) 2022-06-29T15:42:53.028 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.015+0000 7fa473fff700 1 -- 172.21.15.47:0/2443208125 learned_addr learned my addr 172.21.15.47:0/2443208125 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:53.028 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.015+0000 7fa4737fe700 1 --2- 172.21.15.47:0/2443208125 >> v2:172.21.15.64:3300/0 conn(0x7fa474061670 0x7fa474143040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:53.028 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.019+0000 7fa47890a700 1 -- 172.21.15.47:0/2443208125 >> v2:172.21.15.64:3300/0 conn(0x7fa474061670 msgr2=0x7fa474143040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:53.028 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.019+0000 7fa47890a700 1 --2- 172.21.15.47:0/2443208125 >> v2:172.21.15.64:3300/0 conn(0x7fa474061670 0x7fa474143040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:53.029 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.019+0000 7fa472ffd700 1 -- 172.21.15.47:0/2443208125 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2162219882 0 0) 0x7fa464001110 con 0x7fa47413cfc0 2022-06-29T15:42:53.029 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.019+0000 7fa47890a700 1 -- 172.21.15.47:0/2443208125 >> v1:172.21.15.64:6789/0 conn(0x7fa47413cfc0 legacy=0x7fa4740b1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:53.029 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.019+0000 7fa47890a700 1 -- 172.21.15.47:0/2443208125 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa47405f3f0 con 0x7fa474061af0 2022-06-29T15:42:53.030 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.019+0000 7fa4737fe700 1 --2- 172.21.15.47:0/2443208125 >> v2:172.21.15.64:3300/0 conn(0x7fa474061670 0x7fa474143040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:42:53.030 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.019+0000 7fa47890a700 1 --2- 172.21.15.47:0/2443208125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa474061af0 0x7fa474143530 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7fa46c0032f0 tx=0x7fa46c0091d0).ready entity=mon.0 client_cookie=73098ad0a2cc9003 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:53.031 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.019+0000 7fa472ffd700 1 -- 172.21.15.47:0/2443208125 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa46c009ea0 con 0x7fa474061af0 2022-06-29T15:42:53.031 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.019+0000 7fa472ffd700 1 -- 172.21.15.47:0/2443208125 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa46c00b870 con 0x7fa474061af0 2022-06-29T15:42:53.032 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.019+0000 7fa472ffd700 1 -- 172.21.15.47:0/2443208125 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa46c00b9e0 con 0x7fa474061af0 2022-06-29T15:42:53.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.019+0000 7fa47a36d700 1 -- 172.21.15.47:0/2443208125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa474061af0 msgr2=0x7fa474143530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:53.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.019+0000 7fa47a36d700 1 --2- 172.21.15.47:0/2443208125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa474061af0 0x7fa474143530 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7fa46c0032f0 tx=0x7fa46c0091d0).stop 2022-06-29T15:42:53.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.019+0000 7fa47a36d700 1 -- 172.21.15.47:0/2443208125 shutdown_connections 2022-06-29T15:42:53.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.019+0000 7fa47a36d700 1 --2- 172.21.15.47:0/2443208125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa474061af0 0x7fa474143530 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:53.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.019+0000 7fa47a36d700 1 --2- 172.21.15.47:0/2443208125 >> v2:172.21.15.64:3300/0 conn(0x7fa474061670 0x7fa474143040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:53.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.019+0000 7fa47a36d700 1 -- 172.21.15.47:0/2443208125 >> 172.21.15.47:0/2443208125 conn(0x7fa47405c510 msgr2=0x7fa474063800 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:53.038 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.019+0000 7fa47a36d700 1 -- 172.21.15.47:0/2443208125 shutdown_connections 2022-06-29T15:42:53.038 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.019+0000 7fa47a36d700 1 -- 172.21.15.47:0/2443208125 wait complete. 2022-06-29T15:42:53.038 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.019+0000 7fa47a36d700 1 Processor -- start 2022-06-29T15:42:53.038 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.019+0000 7fa47a36d700 1 -- start start 2022-06-29T15:42:53.038 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.023+0000 7fa47a36d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa474061670 0x7fa474147ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:53.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.023+0000 7fa47a36d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa474061af0 0x7fa474155ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:53.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.023+0000 7fa47a36d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa47413cfc0 0x7fa474155fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:53.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.023+0000 7fa47a36d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa474064790 con 0x7fa47413cfc0 2022-06-29T15:42:53.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.023+0000 7fa47a36d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa474064a30 con 0x7fa474061af0 2022-06-29T15:42:53.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.023+0000 7fa47a36d700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa474064cd0 con 0x7fa474061670 2022-06-29T15:42:53.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.023+0000 7fa47890a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa47413cfc0 0x7fa474155fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:53.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.023+0000 7fa47890a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa47413cfc0 0x7fa474155fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46416/0 (socket says 172.21.15.47:46416) 2022-06-29T15:42:53.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.023+0000 7fa47890a700 1 -- 172.21.15.47:0/3034145620 learned_addr learned my addr 172.21.15.47:0/3034145620 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:53.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.023+0000 7fa4737fe700 1 --2- 172.21.15.47:0/3034145620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa474061af0 0x7fa474155ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:53.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.023+0000 7fa473fff700 1 --2- 172.21.15.47:0/3034145620 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa474061670 0x7fa474147ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:53.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.023+0000 7fa47890a700 1 -- 172.21.15.47:0/3034145620 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa474061670 msgr2=0x7fa474147ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:53.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.023+0000 7fa47890a700 1 --2- 172.21.15.47:0/3034145620 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa474061670 0x7fa474147ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:53.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.023+0000 7fa47890a700 1 -- 172.21.15.47:0/3034145620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa474061af0 msgr2=0x7fa474155ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:53.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.023+0000 7fa47890a700 1 --2- 172.21.15.47:0/3034145620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa474061af0 0x7fa474155ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:53.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.023+0000 7fa47890a700 1 -- 172.21.15.47:0/3034145620 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa46c009470 con 0x7fa47413cfc0 2022-06-29T15:42:53.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.023+0000 7fa47890a700 1 --2- 172.21.15.47:0/3034145620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa47413cfc0 0x7fa474155fa0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7fa46c0032f0 tx=0x7fa46c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:53.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.023+0000 7fa4717fa700 1 -- 172.21.15.47:0/3034145620 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa46c009ea0 con 0x7fa47413cfc0 2022-06-29T15:42:53.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.023+0000 7fa4717fa700 1 -- 172.21.15.47:0/3034145620 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa46c021630 con 0x7fa47413cfc0 2022-06-29T15:42:53.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.023+0000 7fa47a36d700 1 -- 172.21.15.47:0/3034145620 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa474064f70 con 0x7fa47413cfc0 2022-06-29T15:42:53.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.023+0000 7fa4717fa700 1 -- 172.21.15.47:0/3034145620 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa46c021990 con 0x7fa47413cfc0 2022-06-29T15:42:53.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.023+0000 7fa47a36d700 1 -- 172.21.15.47:0/3034145620 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa474143350 con 0x7fa47413cfc0 2022-06-29T15:42:53.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.027+0000 7fa47a36d700 1 -- 172.21.15.47:0/3034145620 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa474065100 con 0x7fa47413cfc0 2022-06-29T15:42:53.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.027+0000 7fa4717fa700 1 -- 172.21.15.47:0/3034145620 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa46c027020 con 0x7fa47413cfc0 2022-06-29T15:42:53.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.027+0000 7fa4717fa700 1 --2- 172.21.15.47:0/3034145620 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa454053b30 0x7fa454055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:53.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.027+0000 7fa4717fa700 1 -- 172.21.15.47:0/3034145620 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fa46c06d4e0 con 0x7fa47413cfc0 2022-06-29T15:42:53.045 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.027+0000 7fa473fff700 1 --2- 172.21.15.47:0/3034145620 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa454053b30 0x7fa454055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:53.045 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.027+0000 7fa473fff700 1 --2- 172.21.15.47:0/3034145620 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa454053b30 0x7fa454055ff0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7fa46400d000 tx=0x7fa464007c00).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:53.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.035+0000 7fa4717fa700 1 -- 172.21.15.47:0/3034145620 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa46c0441d0 con 0x7fa47413cfc0 2022-06-29T15:42:53.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.163+0000 7fa47a36d700 1 -- 172.21.15.47:0/3034145620 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa4740573d0 con 0x7fa47413cfc0 2022-06-29T15:42:53.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.163+0000 7fa4717fa700 1 -- 172.21.15.47:0/3034145620 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+16299 (secure 0 0 0) 0x7fa46c015090 con 0x7fa47413cfc0 2022-06-29T15:42:53.172 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:42:53.173 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":26,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:42:34.474438+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:42:28.400446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-06-29T15:42:31.167705+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:42:53.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.167+0000 7fa47a36d700 1 -- 172.21.15.47:0/3034145620 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa454053b30 msgr2=0x7fa454055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:53.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.167+0000 7fa47a36d700 1 --2- 172.21.15.47:0/3034145620 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa454053b30 0x7fa454055ff0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7fa46400d000 tx=0x7fa464007c00).stop 2022-06-29T15:42:53.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.167+0000 7fa47a36d700 1 -- 172.21.15.47:0/3034145620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa47413cfc0 msgr2=0x7fa474155fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:53.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.167+0000 7fa47a36d700 1 --2- 172.21.15.47:0/3034145620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa47413cfc0 0x7fa474155fa0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7fa46c0032f0 tx=0x7fa46c016000).stop 2022-06-29T15:42:53.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.167+0000 7fa47a36d700 1 -- 172.21.15.47:0/3034145620 shutdown_connections 2022-06-29T15:42:53.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.167+0000 7fa47a36d700 1 --2- 172.21.15.47:0/3034145620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa47413cfc0 0x7fa474155fa0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:53.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.167+0000 7fa47a36d700 1 --2- 172.21.15.47:0/3034145620 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa454053b30 0x7fa454055ff0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:53.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.167+0000 7fa47a36d700 1 --2- 172.21.15.47:0/3034145620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa474061af0 0x7fa474155ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:53.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.167+0000 7fa47a36d700 1 --2- 172.21.15.47:0/3034145620 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa474061670 0x7fa474147ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:53.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.167+0000 7fa47a36d700 1 -- 172.21.15.47:0/3034145620 >> 172.21.15.47:0/3034145620 conn(0x7fa47405c510 msgr2=0x7fa4740b5360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:53.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.167+0000 7fa47a36d700 1 -- 172.21.15.47:0/3034145620 shutdown_connections 2022-06-29T15:42:53.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.167+0000 7fa47a36d700 1 -- 172.21.15.47:0/3034145620 wait complete. 2022-06-29T15:42:53.185 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-29T15:42:16.971431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-29T15:42:25.290884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-29T15:42:28.400446+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2022-06-29T15:42:31.167705+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-29T15:42:53.186 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-29T15:42:53.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.287+0000 7f0568c33700 1 Processor -- start 2022-06-29T15:42:53.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.291+0000 7f0568c33700 1 -- start start 2022-06-29T15:42:53.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.291+0000 7f0568c33700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f05640605d0 0x7f056409a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:53.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.291+0000 7f0568c33700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f056405f0b0 0x7f056409a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:53.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.291+0000 7f0568c33700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f056409b970 con 0x7f05640af250 2022-06-29T15:42:53.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.291+0000 7f0568c33700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f056405fbe0 con 0x7f05640605d0 2022-06-29T15:42:53.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.291+0000 7f0568c33700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f056409ade0 con 0x7f056405f0b0 2022-06-29T15:42:53.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.291+0000 7f0563fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f05640af250 0x7f05640af6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:60210/0 (socket says 172.21.15.47:60210) 2022-06-29T15:42:53.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.291+0000 7f0563fff700 1 -- 172.21.15.47:0/2797793225 learned_addr learned my addr 172.21.15.47:0/2797793225 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:53.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.291+0000 7f055bfff700 1 --2- 172.21.15.47:0/2797793225 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f056405f0b0 0x7f056409a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:53.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.291+0000 7f05637fe700 1 --2- 172.21.15.47:0/2797793225 >> v2:172.21.15.64:3300/0 conn(0x7f05640605d0 0x7f056409a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:53.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.291+0000 7f0562ffd700 1 -- 172.21.15.47:0/2797793225 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1851159700 0 0) 0x7f056409b970 con 0x7f05640af250 2022-06-29T15:42:53.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.291+0000 7f0562ffd700 1 -- 172.21.15.47:0/2797793225 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0548004370 con 0x7f05640af250 2022-06-29T15:42:53.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.291+0000 7f055bfff700 1 -- 172.21.15.47:0/2797793225 >> v2:172.21.15.64:3300/0 conn(0x7f05640605d0 msgr2=0x7f056409a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:53.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.291+0000 7f055bfff700 1 --2- 172.21.15.47:0/2797793225 >> v2:172.21.15.64:3300/0 conn(0x7f05640605d0 0x7f056409a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:53.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.291+0000 7f055bfff700 1 -- 172.21.15.47:0/2797793225 >> v1:172.21.15.47:6789/0 conn(0x7f05640af250 legacy=0x7f05640af6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:53.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.291+0000 7f055bfff700 1 -- 172.21.15.47:0/2797793225 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05640acfd0 con 0x7f056405f0b0 2022-06-29T15:42:53.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.291+0000 7f055bfff700 1 --2- 172.21.15.47:0/2797793225 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f056405f0b0 0x7f056409a8f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f054c002f70 tx=0x7f054c006d90).ready entity=mon.2 client_cookie=c2f373f56eb93c47 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:53.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.291+0000 7f0562ffd700 1 -- 172.21.15.47:0/2797793225 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f054c010990 con 0x7f056405f0b0 2022-06-29T15:42:53.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.291+0000 7f0562ffd700 1 -- 172.21.15.47:0/2797793225 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f054c000e90 con 0x7f056405f0b0 2022-06-29T15:42:53.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.291+0000 7f0562ffd700 1 -- 172.21.15.47:0/2797793225 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f054c010ce0 con 0x7f056405f0b0 2022-06-29T15:42:53.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.295+0000 7f0568c33700 1 -- 172.21.15.47:0/2797793225 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f056405f0b0 msgr2=0x7f056409a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:53.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.295+0000 7f0568c33700 1 --2- 172.21.15.47:0/2797793225 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f056405f0b0 0x7f056409a8f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f054c002f70 tx=0x7f054c006d90).stop 2022-06-29T15:42:53.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.295+0000 7f0568c33700 1 -- 172.21.15.47:0/2797793225 shutdown_connections 2022-06-29T15:42:53.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.295+0000 7f0568c33700 1 --2- 172.21.15.47:0/2797793225 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f056405f0b0 0x7f056409a8f0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:53.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.295+0000 7f0568c33700 1 --2- 172.21.15.47:0/2797793225 >> v2:172.21.15.64:3300/0 conn(0x7f05640605d0 0x7f056409a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:53.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.295+0000 7f0568c33700 1 -- 172.21.15.47:0/2797793225 >> 172.21.15.47:0/2797793225 conn(0x7f0564062df0 msgr2=0x7f05640631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:53.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.295+0000 7f0568c33700 1 -- 172.21.15.47:0/2797793225 shutdown_connections 2022-06-29T15:42:53.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.295+0000 7f0568c33700 1 -- 172.21.15.47:0/2797793225 wait complete. 2022-06-29T15:42:53.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.295+0000 7f0568c33700 1 Processor -- start 2022-06-29T15:42:53.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.295+0000 7f0568c33700 1 -- start start 2022-06-29T15:42:53.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.299+0000 7f0568c33700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f056405f0b0 0x7f0564140060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:53.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.299+0000 7f0568c33700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f05640605d0 0x7f0564063af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:53.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.299+0000 7f0568c33700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05640af250 0x7f0564063fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:53.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.299+0000 7f0568c33700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0564067120 con 0x7f056405f0b0 2022-06-29T15:42:53.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.299+0000 7f0568c33700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05640b0720 con 0x7f05640af250 2022-06-29T15:42:53.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.299+0000 7f0568c33700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05640b09c0 con 0x7f05640605d0 2022-06-29T15:42:53.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.299+0000 7f05637fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f056405f0b0 0x7f0564140060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:53.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.299+0000 7f05637fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f056405f0b0 0x7f0564140060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46426/0 (socket says 172.21.15.47:46426) 2022-06-29T15:42:53.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.299+0000 7f05637fe700 1 -- 172.21.15.47:0/2028062365 learned_addr learned my addr 172.21.15.47:0/2028062365 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:53.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.299+0000 7f055bfff700 1 --2- 172.21.15.47:0/2028062365 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f05640605d0 0x7f0564063af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:53.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.299+0000 7f0563fff700 1 --2- 172.21.15.47:0/2028062365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05640af250 0x7f0564063fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:53.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.299+0000 7f05637fe700 1 -- 172.21.15.47:0/2028062365 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f05640605d0 msgr2=0x7f0564063af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:53.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.299+0000 7f05637fe700 1 --2- 172.21.15.47:0/2028062365 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f05640605d0 0x7f0564063af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:53.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.299+0000 7f05637fe700 1 -- 172.21.15.47:0/2028062365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05640af250 msgr2=0x7f0564063fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:53.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.299+0000 7f05637fe700 1 --2- 172.21.15.47:0/2028062365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05640af250 0x7f0564063fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:53.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.299+0000 7f05637fe700 1 -- 172.21.15.47:0/2028062365 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f054c00b120 con 0x7f056405f0b0 2022-06-29T15:42:53.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.299+0000 7f05637fe700 1 --2- 172.21.15.47:0/2028062365 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f056405f0b0 0x7f0564140060 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f055400c000 tx=0x7f0554006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:53.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.299+0000 7f05617fa700 1 -- 172.21.15.47:0/2028062365 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0554010780 con 0x7f056405f0b0 2022-06-29T15:42:53.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.299+0000 7f05617fa700 1 -- 172.21.15.47:0/2028062365 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05540108e0 con 0x7f056405f0b0 2022-06-29T15:42:53.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.299+0000 7f05617fa700 1 -- 172.21.15.47:0/2028062365 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0554010c40 con 0x7f056405f0b0 2022-06-29T15:42:53.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.299+0000 7f0568c33700 1 -- 172.21.15.47:0/2028062365 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05640b0c60 con 0x7f056405f0b0 2022-06-29T15:42:53.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.299+0000 7f0568c33700 1 -- 172.21.15.47:0/2028062365 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05640b0270 con 0x7f056405f0b0 2022-06-29T15:42:53.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.299+0000 7f0568c33700 1 -- 172.21.15.47:0/2028062365 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05640b0df0 con 0x7f056405f0b0 2022-06-29T15:42:53.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.303+0000 7f05617fa700 1 -- 172.21.15.47:0/2028062365 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f055401c460 con 0x7f056405f0b0 2022-06-29T15:42:53.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.303+0000 7f05617fa700 1 --2- 172.21.15.47:0/2028062365 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f053c053b00 0x7f053c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:53.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.303+0000 7f05617fa700 1 -- 172.21.15.47:0/2028062365 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f055406c3a0 con 0x7f056405f0b0 2022-06-29T15:42:53.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.303+0000 7f055bfff700 1 --2- 172.21.15.47:0/2028062365 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f053c053b00 0x7f053c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:53.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.303+0000 7f055bfff700 1 --2- 172.21.15.47:0/2028062365 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f053c053b00 0x7f053c055fc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f054c01c000 tx=0x7f054c016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:53.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.307+0000 7f05617fa700 1 -- 172.21.15.47:0/2028062365 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0554043100 con 0x7f056405f0b0 2022-06-29T15:42:53.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.431+0000 7f0568c33700 1 -- 172.21.15.47:0/2028062365 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f05640573d0 con 0x7f056405f0b0 2022-06-29T15:42:53.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.431+0000 7f05617fa700 1 -- 172.21.15.47:0/2028062365 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v26) v1 ==== 110+0+10 (secure 0 0 0) 0x7f0554046720 con 0x7f056405f0b0 2022-06-29T15:42:53.439 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2022-06-29T15:42:53.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.431+0000 7f0568c33700 1 -- 172.21.15.47:0/2028062365 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f053c053b00 msgr2=0x7f053c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:53.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.431+0000 7f0568c33700 1 --2- 172.21.15.47:0/2028062365 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f053c053b00 0x7f053c055fc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f054c01c000 tx=0x7f054c016000).stop 2022-06-29T15:42:53.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.431+0000 7f0568c33700 1 -- 172.21.15.47:0/2028062365 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f056405f0b0 msgr2=0x7f0564140060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:53.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.431+0000 7f0568c33700 1 --2- 172.21.15.47:0/2028062365 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f056405f0b0 0x7f0564140060 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f055400c000 tx=0x7f0554006df0).stop 2022-06-29T15:42:53.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.431+0000 7f0568c33700 1 -- 172.21.15.47:0/2028062365 shutdown_connections 2022-06-29T15:42:53.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.431+0000 7f0568c33700 1 --2- 172.21.15.47:0/2028062365 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f053c053b00 0x7f053c055fc0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:53.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.431+0000 7f0568c33700 1 --2- 172.21.15.47:0/2028062365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05640af250 0x7f0564063fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:53.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.431+0000 7f0568c33700 1 --2- 172.21.15.47:0/2028062365 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f05640605d0 0x7f0564063af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:53.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.431+0000 7f0568c33700 1 --2- 172.21.15.47:0/2028062365 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f056405f0b0 0x7f0564140060 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:53.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.431+0000 7f0568c33700 1 -- 172.21.15.47:0/2028062365 >> 172.21.15.47:0/2028062365 conn(0x7f0564062df0 msgr2=0x7f0564140eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:53.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.435+0000 7f0568c33700 1 -- 172.21.15.47:0/2028062365 shutdown_connections 2022-06-29T15:42:53.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.435+0000 7f0568c33700 1 -- 172.21.15.47:0/2028062365 wait complete. 2022-06-29T15:42:53.452 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-29T15:42:53.568 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.559+0000 7f1d28e8b700 1 Processor -- start 2022-06-29T15:42:53.568 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.559+0000 7f1d28e8b700 1 -- start start 2022-06-29T15:42:53.572 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.563+0000 7f1d28e8b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1d2405c850 0x7f1d24143a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:53.572 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.563+0000 7f1d28e8b700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1d240676a0 0x7f1d24140130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:53.572 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.563+0000 7f1d28e8b700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d2405cc30 con 0x7f1d240b5d90 2022-06-29T15:42:53.572 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.563+0000 7f1d28e8b700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d241430a0 con 0x7f1d2405c850 2022-06-29T15:42:53.573 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.563+0000 7f1d28e8b700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d24143300 con 0x7f1d240676a0 2022-06-29T15:42:53.573 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.563+0000 7f1d1affd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1d240b5d90 0x7f1d240b4860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:60224/0 (socket says 172.21.15.47:60224) 2022-06-29T15:42:53.573 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.563+0000 7f1d1affd700 1 -- 172.21.15.47:0/3035177008 learned_addr learned my addr 172.21.15.47:0/3035177008 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:53.573 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.563+0000 7f1d23fff700 1 --2- 172.21.15.47:0/3035177008 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1d240676a0 0x7f1d24140130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:53.573 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.563+0000 7f1d237fe700 1 --2- 172.21.15.47:0/3035177008 >> v2:172.21.15.64:3300/0 conn(0x7f1d2405c850 0x7f1d24143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:53.574 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.563+0000 7f1d22ffd700 1 -- 172.21.15.47:0/3035177008 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1607510966 0 0) 0x7f1d2405cc30 con 0x7f1d240b5d90 2022-06-29T15:42:53.574 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.563+0000 7f1d22ffd700 1 -- 172.21.15.47:0/3035177008 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d08003260 con 0x7f1d240b5d90 2022-06-29T15:42:53.574 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.563+0000 7f1d22ffd700 1 -- 172.21.15.47:0/3035177008 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 735011864 0 0) 0x7f1d08003260 con 0x7f1d240b5d90 2022-06-29T15:42:53.574 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.563+0000 7f1d22ffd700 1 -- 172.21.15.47:0/3035177008 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1d240676a0 msgr2=0x7f1d24140130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:53.575 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.563+0000 7f1d22ffd700 1 --2- 172.21.15.47:0/3035177008 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1d240676a0 0x7f1d24140130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:53.575 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.563+0000 7f1d22ffd700 1 -- 172.21.15.47:0/3035177008 >> v2:172.21.15.64:3300/0 conn(0x7f1d2405c850 msgr2=0x7f1d24143a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:53.575 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.563+0000 7f1d22ffd700 1 --2- 172.21.15.47:0/3035177008 >> v2:172.21.15.64:3300/0 conn(0x7f1d2405c850 0x7f1d24143a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:53.575 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.563+0000 7f1d22ffd700 1 -- 172.21.15.47:0/3035177008 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d241435a0 con 0x7f1d240b5d90 2022-06-29T15:42:53.576 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.563+0000 7f1d22ffd700 1 -- 172.21.15.47:0/3035177008 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7f1d0c001680 con 0x7f1d240b5d90 2022-06-29T15:42:53.576 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.563+0000 7f1d22ffd700 1 -- 172.21.15.47:0/3035177008 >> v1:172.21.15.47:6789/0 conn(0x7f1d240b5d90 legacy=0x7f1d240b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:53.576 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.563+0000 7f1d22ffd700 1 --2- 172.21.15.47:0/3035177008 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d08003750 0x7f1d08003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:53.576 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.563+0000 7f1d22ffd700 1 --2- 172.21.15.47:0/3035177008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d08004500 0x7f1d08004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:53.576 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.563+0000 7f1d22ffd700 1 --2- 172.21.15.47:0/3035177008 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1d240676a0 0x7f1d08004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:53.577 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.563+0000 7f1d22ffd700 1 -- 172.21.15.47:0/3035177008 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d241430a0 con 0x7f1d08003750 2022-06-29T15:42:53.577 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.563+0000 7f1d22ffd700 1 -- 172.21.15.47:0/3035177008 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d24143300 con 0x7f1d08004500 2022-06-29T15:42:53.577 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.563+0000 7f1d22ffd700 1 -- 172.21.15.47:0/3035177008 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0800b750 con 0x7f1d240676a0 2022-06-29T15:42:53.577 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.563+0000 7f1d237fe700 1 --2- 172.21.15.47:0/3035177008 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d08003750 0x7f1d08003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:53.578 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.563+0000 7f1d1affd700 1 --2- 172.21.15.47:0/3035177008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d08004500 0x7f1d08004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:53.578 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.563+0000 7f1d23fff700 1 --2- 172.21.15.47:0/3035177008 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1d240676a0 0x7f1d08004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:53.578 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.563+0000 7f1d237fe700 1 -- 172.21.15.47:0/3035177008 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1d240676a0 msgr2=0x7f1d08004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:53.578 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.563+0000 7f1d237fe700 1 --2- 172.21.15.47:0/3035177008 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1d240676a0 0x7f1d08004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:53.579 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.563+0000 7f1d237fe700 1 -- 172.21.15.47:0/3035177008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d08004500 msgr2=0x7f1d08004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:53.579 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.563+0000 7f1d237fe700 1 --2- 172.21.15.47:0/3035177008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d08004500 0x7f1d08004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:53.579 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.563+0000 7f1d237fe700 1 -- 172.21.15.47:0/3035177008 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d08003260 con 0x7f1d08003750 2022-06-29T15:42:53.579 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.563+0000 7f1d23fff700 1 --2- 172.21.15.47:0/3035177008 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1d240676a0 0x7f1d08004e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:42:53.579 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.563+0000 7f1d237fe700 1 --2- 172.21.15.47:0/3035177008 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d08003750 0x7f1d08003b70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f1d1400c000 tx=0x7f1d14006d90).ready entity=mon.0 client_cookie=94b636fcbad1c038 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:53.580 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.563+0000 7f1d22ffd700 1 -- 172.21.15.47:0/3035177008 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d14013450 con 0x7f1d08003750 2022-06-29T15:42:53.580 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.563+0000 7f1d22ffd700 1 -- 172.21.15.47:0/3035177008 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d140135b0 con 0x7f1d08003750 2022-06-29T15:42:53.580 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.563+0000 7f1d22ffd700 1 -- 172.21.15.47:0/3035177008 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d14013910 con 0x7f1d08003750 2022-06-29T15:42:53.584 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.567+0000 7f1d28e8b700 1 -- 172.21.15.47:0/3035177008 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d08003750 msgr2=0x7f1d08003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:53.584 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.567+0000 7f1d28e8b700 1 --2- 172.21.15.47:0/3035177008 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d08003750 0x7f1d08003b70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f1d1400c000 tx=0x7f1d14006d90).stop 2022-06-29T15:42:53.584 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.567+0000 7f1d23fff700 1 -- 172.21.15.47:0/3035177008 reap_dead start 2022-06-29T15:42:53.585 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.567+0000 7f1d28e8b700 1 -- 172.21.15.47:0/3035177008 shutdown_connections 2022-06-29T15:42:53.585 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.567+0000 7f1d28e8b700 1 -- 172.21.15.47:0/3035177008 >> 172.21.15.47:0/3035177008 conn(0x7f1d24060e00 msgr2=0x7f1d240611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:53.585 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.567+0000 7f1d28e8b700 1 -- 172.21.15.47:0/3035177008 shutdown_connections 2022-06-29T15:42:53.586 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.567+0000 7f1d28e8b700 1 -- 172.21.15.47:0/3035177008 wait complete. 2022-06-29T15:42:53.586 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.567+0000 7f1d28e8b700 1 Processor -- start 2022-06-29T15:42:53.586 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.567+0000 7f1d28e8b700 1 -- start start 2022-06-29T15:42:53.586 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.567+0000 7f1d28e8b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d240afce0 0x7f1d240b1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:53.586 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.567+0000 7f1d28e8b700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1d240b95a0 0x7f1d240b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:53.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.567+0000 7f1d28e8b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d2413f360 0x7f1d2413f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:53.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.567+0000 7f1d28e8b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d240b2160 con 0x7f1d2413f360 2022-06-29T15:42:53.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.567+0000 7f1d28e8b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d240b1350 con 0x7f1d240afce0 2022-06-29T15:42:53.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.567+0000 7f1d28e8b700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d240b00c0 con 0x7f1d240b95a0 2022-06-29T15:42:53.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.567+0000 7f1d23fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d2413f360 0x7f1d2413f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:53.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.567+0000 7f1d23fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d2413f360 0x7f1d2413f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46450/0 (socket says 172.21.15.47:46450) 2022-06-29T15:42:53.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.571+0000 7f1d23fff700 1 -- 172.21.15.47:0/3730476316 learned_addr learned my addr 172.21.15.47:0/3730476316 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:53.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.571+0000 7f1d1affd700 1 --2- 172.21.15.47:0/3730476316 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1d240b95a0 0x7f1d240b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:53.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.571+0000 7f1d237fe700 1 --2- 172.21.15.47:0/3730476316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d240afce0 0x7f1d240b1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:53.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.571+0000 7f1d23fff700 1 -- 172.21.15.47:0/3730476316 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1d240b95a0 msgr2=0x7f1d240b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:53.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.571+0000 7f1d23fff700 1 --2- 172.21.15.47:0/3730476316 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1d240b95a0 0x7f1d240b1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:53.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.571+0000 7f1d23fff700 1 -- 172.21.15.47:0/3730476316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d240afce0 msgr2=0x7f1d240b1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:53.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.571+0000 7f1d23fff700 1 --2- 172.21.15.47:0/3730476316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d240afce0 0x7f1d240b1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:53.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.571+0000 7f1d23fff700 1 -- 172.21.15.47:0/3730476316 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d240b9d60 con 0x7f1d2413f360 2022-06-29T15:42:53.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.571+0000 7f1d1affd700 1 --2- 172.21.15.47:0/3730476316 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1d240b95a0 0x7f1d240b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:42:53.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.571+0000 7f1d23fff700 1 --2- 172.21.15.47:0/3730476316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d2413f360 0x7f1d2413f7f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f1d1c00b000 tx=0x7f1d1c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:53.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.571+0000 7f1d217fa700 1 -- 172.21.15.47:0/3730476316 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d1c011ee0 con 0x7f1d2413f360 2022-06-29T15:42:53.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.571+0000 7f1d217fa700 1 -- 172.21.15.47:0/3730476316 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d1c0145f0 con 0x7f1d2413f360 2022-06-29T15:42:53.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.571+0000 7f1d28e8b700 1 -- 172.21.15.47:0/3730476316 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d240ba000 con 0x7f1d2413f360 2022-06-29T15:42:53.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.571+0000 7f1d217fa700 1 -- 172.21.15.47:0/3730476316 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d1c014950 con 0x7f1d2413f360 2022-06-29T15:42:53.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.571+0000 7f1d28e8b700 1 -- 172.21.15.47:0/3730476316 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d2413fce0 con 0x7f1d2413f360 2022-06-29T15:42:53.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.571+0000 7f1d28e8b700 1 -- 172.21.15.47:0/3730476316 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d240573d0 con 0x7f1d2413f360 2022-06-29T15:42:53.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.575+0000 7f1d217fa700 1 -- 172.21.15.47:0/3730476316 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1d1c012040 con 0x7f1d2413f360 2022-06-29T15:42:53.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.575+0000 7f1d217fa700 1 --2- 172.21.15.47:0/3730476316 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1cfc053b00 0x7f1cfc055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:53.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.575+0000 7f1d217fa700 1 -- 172.21.15.47:0/3730476316 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f1d1c06c620 con 0x7f1d2413f360 2022-06-29T15:42:53.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.575+0000 7f1d237fe700 1 --2- 172.21.15.47:0/3730476316 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1cfc053b00 0x7f1cfc055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:53.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.575+0000 7f1d237fe700 1 --2- 172.21.15.47:0/3730476316 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1cfc053b00 0x7f1cfc055fc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f1d1401c000 tx=0x7f1d14018000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:53.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.583+0000 7f1d217fa700 1 -- 172.21.15.47:0/3730476316 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1d1c043380 con 0x7f1d2413f360 2022-06-29T15:42:53.714 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.707+0000 7f1d28e8b700 1 -- 172.21.15.47:0/3730476316 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1d24061b00 con 0x7f1d2413f360 2022-06-29T15:42:53.714 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.707+0000 7f1d217fa700 1 -- 172.21.15.47:0/3730476316 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1d1c0469a0 con 0x7f1d2413f360 2022-06-29T15:42:53.715 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:42:53.718 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.711+0000 7f1d28e8b700 1 -- 172.21.15.47:0/3730476316 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1cfc053b00 msgr2=0x7f1cfc055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:53.718 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.711+0000 7f1d28e8b700 1 --2- 172.21.15.47:0/3730476316 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1cfc053b00 0x7f1cfc055fc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f1d1401c000 tx=0x7f1d14018000).stop 2022-06-29T15:42:53.719 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.711+0000 7f1d28e8b700 1 -- 172.21.15.47:0/3730476316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d2413f360 msgr2=0x7f1d2413f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:53.719 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.711+0000 7f1d28e8b700 1 --2- 172.21.15.47:0/3730476316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d2413f360 0x7f1d2413f7f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f1d1c00b000 tx=0x7f1d1c008040).stop 2022-06-29T15:42:53.719 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.711+0000 7f1d28e8b700 1 -- 172.21.15.47:0/3730476316 shutdown_connections 2022-06-29T15:42:53.719 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.711+0000 7f1d28e8b700 1 --2- 172.21.15.47:0/3730476316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d2413f360 0x7f1d2413f7f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:53.720 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.711+0000 7f1d28e8b700 1 --2- 172.21.15.47:0/3730476316 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1d240b95a0 0x7f1d240b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:53.720 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.711+0000 7f1d28e8b700 1 --2- 172.21.15.47:0/3730476316 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1cfc053b00 0x7f1cfc055fc0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:53.720 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.711+0000 7f1d28e8b700 1 --2- 172.21.15.47:0/3730476316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d240afce0 0x7f1d240b1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:53.720 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.711+0000 7f1d28e8b700 1 -- 172.21.15.47:0/3730476316 >> 172.21.15.47:0/3730476316 conn(0x7f1d24060e00 msgr2=0x7f1d24146830 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:53.720 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.711+0000 7f1d28e8b700 1 -- 172.21.15.47:0/3730476316 shutdown_connections 2022-06-29T15:42:53.721 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.711+0000 7f1d28e8b700 1 -- 172.21.15.47:0/3730476316 wait complete. 2022-06-29T15:42:53.733 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-29T15:42:53.835 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.827+0000 7ff8f4c55700 1 Processor -- start 2022-06-29T15:42:53.835 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.827+0000 7ff8f4c55700 1 -- start start 2022-06-29T15:42:53.838 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.827+0000 7ff8f4c55700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff8f00612c0 0x7ff8f01471a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:53.839 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.827+0000 7ff8f4c55700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8f0067610 con 0x7ff8f0060ee0 2022-06-29T15:42:53.839 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.827+0000 7ff8f4c55700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8f01467c0 con 0x7ff8f00b4300 2022-06-29T15:42:53.839 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.827+0000 7ff8f4c55700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8f0146a60 con 0x7ff8f00612c0 2022-06-29T15:42:53.840 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.827+0000 7ff8ee59c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff8f0060ee0 0x7ff8f01430e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:60240/0 (socket says 172.21.15.47:60240) 2022-06-29T15:42:53.840 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.827+0000 7ff8ee59c700 1 -- 172.21.15.47:0/2862711580 learned_addr learned my addr 172.21.15.47:0/2862711580 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:53.840 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.827+0000 7ff8eed9d700 1 --2- 172.21.15.47:0/2862711580 >> v2:172.21.15.64:3300/0 conn(0x7ff8f00612c0 0x7ff8f01471a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:53.840 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.827+0000 7ff8ed59a700 1 -- 172.21.15.47:0/2862711580 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 29227897 0 0) 0x7ff8f0067610 con 0x7ff8f0060ee0 2022-06-29T15:42:53.841 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.827+0000 7ff8ed59a700 1 -- 172.21.15.47:0/2862711580 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8d4003260 con 0x7ff8f0060ee0 2022-06-29T15:42:53.841 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.831+0000 7ff8ed59a700 1 -- 172.21.15.47:0/2862711580 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 848625790 0 0) 0x7ff8d4003260 con 0x7ff8f0060ee0 2022-06-29T15:42:53.841 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.831+0000 7ff8ed59a700 1 -- 172.21.15.47:0/2862711580 >> v2:172.21.15.64:3300/0 conn(0x7ff8f00612c0 msgr2=0x7ff8f01471a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:53.841 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.831+0000 7ff8ed59a700 1 --2- 172.21.15.47:0/2862711580 >> v2:172.21.15.64:3300/0 conn(0x7ff8f00612c0 0x7ff8f01471a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:53.841 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.831+0000 7ff8ed59a700 1 -- 172.21.15.47:0/2862711580 >> v1:172.21.15.64:6789/0 conn(0x7ff8f00b4300 legacy=0x7ff8f013cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:53.842 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.831+0000 7ff8ed59a700 1 -- 172.21.15.47:0/2862711580 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f0146d00 con 0x7ff8f0060ee0 2022-06-29T15:42:53.842 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.831+0000 7ff8ed59a700 1 -- 172.21.15.47:0/2862711580 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7ff8e80019e0 con 0x7ff8f0060ee0 2022-06-29T15:42:53.842 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.831+0000 7ff8ed59a700 1 -- 172.21.15.47:0/2862711580 >> v1:172.21.15.47:6789/0 conn(0x7ff8f0060ee0 legacy=0x7ff8f01430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:53.842 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.831+0000 7ff8ed59a700 1 --2- 172.21.15.47:0/2862711580 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff8d4003750 0x7ff8d4003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:53.843 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.831+0000 7ff8ed59a700 1 --2- 172.21.15.47:0/2862711580 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8d4004500 0x7ff8d4004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:53.843 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.831+0000 7ff8ed59a700 1 --2- 172.21.15.47:0/2862711580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8d4004e70 0x7ff8d400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:53.843 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.831+0000 7ff8ed59a700 1 -- 172.21.15.47:0/2862711580 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f0146a60 con 0x7ff8d4004500 2022-06-29T15:42:53.843 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.831+0000 7ff8ed59a700 1 -- 172.21.15.47:0/2862711580 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d400bd20 con 0x7ff8d4004e70 2022-06-29T15:42:53.844 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.831+0000 7ff8ed59a700 1 -- 172.21.15.47:0/2862711580 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d400bee0 con 0x7ff8d4003750 2022-06-29T15:42:53.844 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.831+0000 7ff8edd9b700 1 --2- 172.21.15.47:0/2862711580 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8d4004500 0x7ff8d4004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:53.844 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.831+0000 7ff8ee59c700 1 --2- 172.21.15.47:0/2862711580 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff8d4003750 0x7ff8d4003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:53.845 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.831+0000 7ff8eed9d700 1 --2- 172.21.15.47:0/2862711580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8d4004e70 0x7ff8d400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:53.845 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.831+0000 7ff8edd9b700 1 -- 172.21.15.47:0/2862711580 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff8d4003750 msgr2=0x7ff8d4003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:53.845 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.831+0000 7ff8edd9b700 1 --2- 172.21.15.47:0/2862711580 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff8d4003750 0x7ff8d4003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:53.845 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.831+0000 7ff8edd9b700 1 -- 172.21.15.47:0/2862711580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8d4004e70 msgr2=0x7ff8d400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:53.846 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.831+0000 7ff8edd9b700 1 --2- 172.21.15.47:0/2862711580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8d4004e70 0x7ff8d400b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:53.846 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.831+0000 7ff8edd9b700 1 -- 172.21.15.47:0/2862711580 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d4003260 con 0x7ff8d4004500 2022-06-29T15:42:53.846 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.831+0000 7ff8eed9d700 1 -- 172.21.15.47:0/2862711580 reap_dead start 2022-06-29T15:42:53.847 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.831+0000 7ff8edd9b700 1 --2- 172.21.15.47:0/2862711580 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8d4004500 0x7ff8d4004980 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7ff8f00b9870 tx=0x7ff8e4007a00).ready entity=mon.0 client_cookie=c50eaf7bfa0e8877 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:53.847 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.831+0000 7ff8ed59a700 1 -- 172.21.15.47:0/2862711580 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8e4007d10 con 0x7ff8d4004500 2022-06-29T15:42:53.847 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.831+0000 7ff8ed59a700 1 -- 172.21.15.47:0/2862711580 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e4007030 con 0x7ff8d4004500 2022-06-29T15:42:53.847 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.831+0000 7ff8ed59a700 1 -- 172.21.15.47:0/2862711580 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8e4007390 con 0x7ff8d4004500 2022-06-29T15:42:53.852 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.831+0000 7ff8f4c55700 1 -- 172.21.15.47:0/2862711580 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8d4004500 msgr2=0x7ff8d4004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:53.852 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.831+0000 7ff8f4c55700 1 --2- 172.21.15.47:0/2862711580 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8d4004500 0x7ff8d4004980 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7ff8f00b9870 tx=0x7ff8e4007a00).stop 2022-06-29T15:42:53.852 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.831+0000 7ff8f4c55700 1 -- 172.21.15.47:0/2862711580 shutdown_connections 2022-06-29T15:42:53.853 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.831+0000 7ff8f4c55700 1 --2- 172.21.15.47:0/2862711580 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8d4004500 0x7ff8d4004980 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:53.853 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.831+0000 7ff8f4c55700 1 -- 172.21.15.47:0/2862711580 >> 172.21.15.47:0/2862711580 conn(0x7ff8f005c510 msgr2=0x7ff8f005f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:53.853 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.831+0000 7ff8f4c55700 1 -- 172.21.15.47:0/2862711580 shutdown_connections 2022-06-29T15:42:53.853 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.831+0000 7ff8f4c55700 1 -- 172.21.15.47:0/2862711580 wait complete. 2022-06-29T15:42:53.854 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.831+0000 7ff8f4c55700 1 Processor -- start 2022-06-29T15:42:53.854 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.831+0000 7ff8f4c55700 1 -- start start 2022-06-29T15:42:53.854 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.835+0000 7ff8f4c55700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff8d4004500 0x7ff8f0149e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:53.854 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.835+0000 7ff8f4c55700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8f01444a0 0x7ff8f0155e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:53.854 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.835+0000 7ff8f4c55700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8f0156300 0x7ff8f0144ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:53.855 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.835+0000 7ff8f4c55700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f01429d0 con 0x7ff8f01444a0 2022-06-29T15:42:53.855 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.835+0000 7ff8f4c55700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f0144920 con 0x7ff8f0156300 2022-06-29T15:42:53.855 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.835+0000 7ff8f4c55700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f0138ab0 con 0x7ff8d4004500 2022-06-29T15:42:53.855 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.835+0000 7ff8edd9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8f01444a0 0x7ff8f0155e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:53.856 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.835+0000 7ff8edd9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8f01444a0 0x7ff8f0155e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46468/0 (socket says 172.21.15.47:46468) 2022-06-29T15:42:53.856 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.835+0000 7ff8eed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8f0156300 0x7ff8f0144ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:53.856 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.835+0000 7ff8edd9b700 1 -- 172.21.15.47:0/829195729 learned_addr learned my addr 172.21.15.47:0/829195729 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:53.856 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.835+0000 7ff8ee59c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff8d4004500 0x7ff8f0149e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:53.857 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.835+0000 7ff8edd9b700 1 -- 172.21.15.47:0/829195729 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff8d4004500 msgr2=0x7ff8f0149e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:53.857 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.835+0000 7ff8edd9b700 1 --2- 172.21.15.47:0/829195729 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff8d4004500 0x7ff8f0149e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:53.857 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.835+0000 7ff8edd9b700 1 -- 172.21.15.47:0/829195729 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8f0156300 msgr2=0x7ff8f0144ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:53.857 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.835+0000 7ff8edd9b700 1 --2- 172.21.15.47:0/829195729 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8f0156300 0x7ff8f0144ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:53.858 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.835+0000 7ff8edd9b700 1 -- 172.21.15.47:0/829195729 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f0145d60 con 0x7ff8f01444a0 2022-06-29T15:42:53.858 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.835+0000 7ff8edd9b700 1 --2- 172.21.15.47:0/829195729 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8f01444a0 0x7ff8f0155e10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7ff8f00b9870 tx=0x7ff8e4013000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:53.858 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.835+0000 7ff8df7fe700 1 -- 172.21.15.47:0/829195729 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8e4007d10 con 0x7ff8f01444a0 2022-06-29T15:42:53.858 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.835+0000 7ff8df7fe700 1 -- 172.21.15.47:0/829195729 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e4013450 con 0x7ff8f01444a0 2022-06-29T15:42:53.858 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.835+0000 7ff8df7fe700 1 -- 172.21.15.47:0/829195729 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8e40137b0 con 0x7ff8f01444a0 2022-06-29T15:42:53.859 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.835+0000 7ff8f4c55700 1 -- 172.21.15.47:0/829195729 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8f0138ee0 con 0x7ff8f01444a0 2022-06-29T15:42:53.859 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.835+0000 7ff8f4c55700 1 -- 172.21.15.47:0/829195729 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8f01567b0 con 0x7ff8f01444a0 2022-06-29T15:42:53.859 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.839+0000 7ff8f4c55700 1 -- 172.21.15.47:0/829195729 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8f0138c70 con 0x7ff8f01444a0 2022-06-29T15:42:53.859 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.839+0000 7ff8df7fe700 1 -- 172.21.15.47:0/829195729 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff8e4027070 con 0x7ff8f01444a0 2022-06-29T15:42:53.860 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.839+0000 7ff8df7fe700 1 --2- 172.21.15.47:0/829195729 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff8cc053b30 0x7ff8cc055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:53.860 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.839+0000 7ff8df7fe700 1 -- 172.21.15.47:0/829195729 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7ff8e406c2b0 con 0x7ff8f01444a0 2022-06-29T15:42:53.860 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.839+0000 7ff8ee59c700 1 --2- 172.21.15.47:0/829195729 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff8cc053b30 0x7ff8cc055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:53.860 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.839+0000 7ff8ee59c700 1 --2- 172.21.15.47:0/829195729 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff8cc053b30 0x7ff8cc055ff0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7ff8e8005000 tx=0x7ff8f0146e90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:53.861 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.843+0000 7ff8df7fe700 1 -- 172.21.15.47:0/829195729 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff8e4043070 con 0x7ff8f01444a0 2022-06-29T15:42:53.977 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.971+0000 7ff8f4c55700 1 -- 172.21.15.47:0/829195729 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff8f0145ef0 con 0x7ff8f01444a0 2022-06-29T15:42:53.978 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.971+0000 7ff8df7fe700 1 -- 172.21.15.47:0/829195729 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+10 (secure 0 0 0) 0x7ff8e4046690 con 0x7ff8f01444a0 2022-06-29T15:42:53.978 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:42:53.980 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.971+0000 7ff8f4c55700 1 -- 172.21.15.47:0/829195729 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff8cc053b30 msgr2=0x7ff8cc055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:53.980 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.971+0000 7ff8f4c55700 1 --2- 172.21.15.47:0/829195729 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff8cc053b30 0x7ff8cc055ff0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7ff8e8005000 tx=0x7ff8f0146e90).stop 2022-06-29T15:42:53.980 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.971+0000 7ff8f4c55700 1 -- 172.21.15.47:0/829195729 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8f01444a0 msgr2=0x7ff8f0155e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:53.981 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.971+0000 7ff8f4c55700 1 --2- 172.21.15.47:0/829195729 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8f01444a0 0x7ff8f0155e10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7ff8f00b9870 tx=0x7ff8e4013000).stop 2022-06-29T15:42:53.982 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.971+0000 7ff8f4c55700 1 -- 172.21.15.47:0/829195729 shutdown_connections 2022-06-29T15:42:53.982 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.971+0000 7ff8f4c55700 1 --2- 172.21.15.47:0/829195729 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff8cc053b30 0x7ff8cc055ff0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:53.982 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.971+0000 7ff8f4c55700 1 --2- 172.21.15.47:0/829195729 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8f0156300 0x7ff8f0144ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:53.983 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.971+0000 7ff8f4c55700 1 --2- 172.21.15.47:0/829195729 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8f01444a0 0x7ff8f0155e10 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:53.983 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.971+0000 7ff8f4c55700 1 --2- 172.21.15.47:0/829195729 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff8d4004500 0x7ff8f0149e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:53.983 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.971+0000 7ff8f4c55700 1 -- 172.21.15.47:0/829195729 >> 172.21.15.47:0/829195729 conn(0x7ff8f005c510 msgr2=0x7ff8f013cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:53.983 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.971+0000 7ff8f4c55700 1 -- 172.21.15.47:0/829195729 shutdown_connections 2022-06-29T15:42:53.984 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:53.971+0000 7ff8f4c55700 1 -- 172.21.15.47:0/829195729 wait complete. 2022-06-29T15:42:53.994 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-29T15:42:54.111 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.103+0000 7f919b570700 1 Processor -- start 2022-06-29T15:42:54.111 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.103+0000 7f919b570700 1 -- start start 2022-06-29T15:42:54.114 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.107+0000 7f919b570700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9194144670 0x7f9194061740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:54.115 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.107+0000 7f919b570700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9194061e50 con 0x7f919405f0b0 2022-06-29T15:42:54.115 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.107+0000 7f919b570700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91940620f0 con 0x7f919405f490 2022-06-29T15:42:54.115 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.107+0000 7f919b570700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9194061c30 con 0x7f9194144670 2022-06-29T15:42:54.115 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.107+0000 7f919a56e700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f919405f0b0 0x7f9194143950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:60260/0 (socket says 172.21.15.47:60260) 2022-06-29T15:42:54.115 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.107+0000 7f919a56e700 1 -- 172.21.15.47:0/1986676126 learned_addr learned my addr 172.21.15.47:0/1986676126 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:54.116 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.107+0000 7f919ad6f700 1 --2- 172.21.15.47:0/1986676126 >> v2:172.21.15.64:3300/0 conn(0x7f9194144670 0x7f9194061740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:54.116 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.107+0000 7f919956c700 1 -- 172.21.15.47:0/1986676126 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2157102727 0 0) 0x7f9194061e50 con 0x7f919405f0b0 2022-06-29T15:42:54.116 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.107+0000 7f919956c700 1 -- 172.21.15.47:0/1986676126 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f917c003260 con 0x7f919405f0b0 2022-06-29T15:42:54.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.107+0000 7f919ad6f700 1 -- 172.21.15.47:0/1986676126 >> v1:172.21.15.64:6789/0 conn(0x7f919405f490 legacy=0x7f9194140410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:54.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.107+0000 7f919ad6f700 1 -- 172.21.15.47:0/1986676126 >> v1:172.21.15.47:6789/0 conn(0x7f919405f0b0 legacy=0x7f9194143950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:54.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.107+0000 7f919ad6f700 1 -- 172.21.15.47:0/1986676126 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91941434b0 con 0x7f9194144670 2022-06-29T15:42:54.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.107+0000 7f919ad6f700 1 --2- 172.21.15.47:0/1986676126 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9194144670 0x7f9194061740 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f918c0032f0 tx=0x7f918c0091d0).ready entity=mon.1 client_cookie=5f8f53d37754de41 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:54.118 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.107+0000 7f919956c700 1 -- 172.21.15.47:0/1986676126 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f918c00b6c0 con 0x7f9194144670 2022-06-29T15:42:54.118 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.107+0000 7f919956c700 1 -- 172.21.15.47:0/1986676126 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f918c00b820 con 0x7f9194144670 2022-06-29T15:42:54.118 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.107+0000 7f919956c700 1 -- 172.21.15.47:0/1986676126 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f918c00b990 con 0x7f9194144670 2022-06-29T15:42:54.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.107+0000 7f919b570700 1 -- 172.21.15.47:0/1986676126 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9194144670 msgr2=0x7f9194061740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:54.121 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.107+0000 7f919b570700 1 --2- 172.21.15.47:0/1986676126 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9194144670 0x7f9194061740 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f918c0032f0 tx=0x7f918c0091d0).stop 2022-06-29T15:42:54.121 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.107+0000 7f919b570700 1 -- 172.21.15.47:0/1986676126 shutdown_connections 2022-06-29T15:42:54.121 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.107+0000 7f919b570700 1 --2- 172.21.15.47:0/1986676126 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9194144670 0x7f9194061740 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:54.121 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.107+0000 7f919b570700 1 -- 172.21.15.47:0/1986676126 >> 172.21.15.47:0/1986676126 conn(0x7f9194062570 msgr2=0x7f91940647d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:54.122 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.107+0000 7f919b570700 1 -- 172.21.15.47:0/1986676126 shutdown_connections 2022-06-29T15:42:54.122 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.107+0000 7f919b570700 1 -- 172.21.15.47:0/1986676126 wait complete. 2022-06-29T15:42:54.122 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.107+0000 7f919b570700 1 Processor -- start 2022-06-29T15:42:54.122 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.107+0000 7f919b570700 1 -- start start 2022-06-29T15:42:54.122 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.111+0000 7f919b570700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f919405f0b0 0x7f9194142500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:54.123 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.111+0000 7f919b570700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f919405f490 0x7f91940a6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:54.123 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.111+0000 7f919b570700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9194144670 0x7f91940b6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:54.123 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.111+0000 7f919b570700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91940a6fa0 con 0x7f9194144670 2022-06-29T15:42:54.123 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.111+0000 7f919b570700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9194152f80 con 0x7f919405f490 2022-06-29T15:42:54.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.111+0000 7f919b570700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91940a6610 con 0x7f919405f0b0 2022-06-29T15:42:54.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.111+0000 7f919ad6f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9194144670 0x7f91940b6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:54.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.111+0000 7f919ad6f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9194144670 0x7f91940b6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46484/0 (socket says 172.21.15.47:46484) 2022-06-29T15:42:54.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.111+0000 7f919ad6f700 1 -- 172.21.15.47:0/2038671940 learned_addr learned my addr 172.21.15.47:0/2038671940 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:54.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.111+0000 7f919a56e700 1 --2- 172.21.15.47:0/2038671940 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f919405f0b0 0x7f9194142500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:54.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.111+0000 7f9199d6d700 1 --2- 172.21.15.47:0/2038671940 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f919405f490 0x7f91940a6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:54.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.111+0000 7f919a56e700 1 -- 172.21.15.47:0/2038671940 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f919405f490 msgr2=0x7f91940a6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:54.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.111+0000 7f919a56e700 1 --2- 172.21.15.47:0/2038671940 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f919405f490 0x7f91940a6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:54.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.111+0000 7f919a56e700 1 -- 172.21.15.47:0/2038671940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9194144670 msgr2=0x7f91940b6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:54.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.111+0000 7f919a56e700 1 --2- 172.21.15.47:0/2038671940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9194144670 0x7f91940b6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:54.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.111+0000 7f919a56e700 1 -- 172.21.15.47:0/2038671940 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f918c009470 con 0x7f919405f0b0 2022-06-29T15:42:54.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.111+0000 7f919a56e700 1 --2- 172.21.15.47:0/2038671940 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f919405f0b0 0x7f9194142500 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f9190001110 tx=0x7f9190007a20).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:54.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.115+0000 7f91837fe700 1 -- 172.21.15.47:0/2038671940 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f919000ce80 con 0x7f919405f0b0 2022-06-29T15:42:54.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.115+0000 7f91837fe700 1 -- 172.21.15.47:0/2038671940 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91900070d0 con 0x7f919405f0b0 2022-06-29T15:42:54.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.115+0000 7f919b570700 1 -- 172.21.15.47:0/2038671940 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9194152ca0 con 0x7f919405f0b0 2022-06-29T15:42:54.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.115+0000 7f91837fe700 1 -- 172.21.15.47:0/2038671940 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9190007430 con 0x7f919405f0b0 2022-06-29T15:42:54.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.115+0000 7f919b570700 1 -- 172.21.15.47:0/2038671940 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91940a3350 con 0x7f919405f0b0 2022-06-29T15:42:54.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.119+0000 7f919b570700 1 -- 172.21.15.47:0/2038671940 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9194153180 con 0x7f919405f0b0 2022-06-29T15:42:54.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.119+0000 7f91837fe700 1 -- 172.21.15.47:0/2038671940 <== mon.2 v2:172.21.15.64:3301/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f919001a060 con 0x7f919405f0b0 2022-06-29T15:42:54.130 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.119+0000 7f91837fe700 1 --2- 172.21.15.47:0/2038671940 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9174053aa0 0x7f9174055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:54.130 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.119+0000 7f91837fe700 1 -- 172.21.15.47:0/2038671940 <== mon.2 v2:172.21.15.64:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f919006b780 con 0x7f919405f0b0 2022-06-29T15:42:54.130 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.119+0000 7f9199d6d700 1 --2- 172.21.15.47:0/2038671940 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9174053aa0 0x7f9174055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:54.130 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.119+0000 7f9199d6d700 1 --2- 172.21.15.47:0/2038671940 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9174053aa0 0x7f9174055f60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f918400d000 tx=0x7f9184007b90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:54.131 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.123+0000 7f91837fe700 1 -- 172.21.15.47:0/2038671940 <== mon.2 v2:172.21.15.64:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9190042430 con 0x7f919405f0b0 2022-06-29T15:42:54.262 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.255+0000 7f919b570700 1 -- 172.21.15.47:0/2038671940 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f91940573d0 con 0x7f919405f0b0 2022-06-29T15:42:54.263 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.255+0000 7f91837fe700 1 -- 172.21.15.47:0/2038671940 <== mon.2 v2:172.21.15.64:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+10 (secure 0 0 0) 0x7f9190018070 con 0x7f919405f0b0 2022-06-29T15:42:54.263 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:42:54.265 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.259+0000 7f919b570700 1 -- 172.21.15.47:0/2038671940 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9174053aa0 msgr2=0x7f9174055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:54.266 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.259+0000 7f919b570700 1 --2- 172.21.15.47:0/2038671940 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9174053aa0 0x7f9174055f60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f918400d000 tx=0x7f9184007b90).stop 2022-06-29T15:42:54.266 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.259+0000 7f919b570700 1 -- 172.21.15.47:0/2038671940 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f919405f0b0 msgr2=0x7f9194142500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:54.266 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.259+0000 7f919b570700 1 --2- 172.21.15.47:0/2038671940 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f919405f0b0 0x7f9194142500 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f9190001110 tx=0x7f9190007a20).stop 2022-06-29T15:42:54.266 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.259+0000 7f919b570700 1 -- 172.21.15.47:0/2038671940 shutdown_connections 2022-06-29T15:42:54.267 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.259+0000 7f919b570700 1 --2- 172.21.15.47:0/2038671940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9194144670 0x7f91940b6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:54.267 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.259+0000 7f919b570700 1 --2- 172.21.15.47:0/2038671940 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9174053aa0 0x7f9174055f60 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:54.267 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.259+0000 7f919b570700 1 --2- 172.21.15.47:0/2038671940 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f919405f490 0x7f91940a6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:54.267 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.259+0000 7f919b570700 1 --2- 172.21.15.47:0/2038671940 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f919405f0b0 0x7f9194142500 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:54.267 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.259+0000 7f919b570700 1 -- 172.21.15.47:0/2038671940 >> 172.21.15.47:0/2038671940 conn(0x7f9194062570 msgr2=0x7f919413fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:54.268 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.259+0000 7f919b570700 1 -- 172.21.15.47:0/2038671940 shutdown_connections 2022-06-29T15:42:54.268 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.259+0000 7f919b570700 1 -- 172.21.15.47:0/2038671940 wait complete. 2022-06-29T15:42:54.277 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:42:54.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.375+0000 7efe41d4c700 1 Processor -- start 2022-06-29T15:42:54.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.375+0000 7efe41d4c700 1 -- start start 2022-06-29T15:42:54.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.379+0000 7efe41d4c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7efe3c0b0cf0 0x7efe3c146d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:54.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.379+0000 7efe41d4c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efe3c0b06d0 0x7efe3c13ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:54.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.379+0000 7efe41d4c700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe3c0b4520 con 0x7efe3c0611b0 2022-06-29T15:42:54.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.379+0000 7efe41d4c700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe3c0b4b10 con 0x7efe3c0b0cf0 2022-06-29T15:42:54.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.379+0000 7efe41d4c700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe3c0621d0 con 0x7efe3c0b06d0 2022-06-29T15:42:54.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.379+0000 7efe3affd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7efe3c0b0cf0 0x7efe3c146d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:54.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.379+0000 7efe3affd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7efe3c0b0cf0 0x7efe3c146d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46488/0 (socket says 172.21.15.47:46488) 2022-06-29T15:42:54.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.379+0000 7efe3affd700 1 -- 172.21.15.47:0/2103374421 learned_addr learned my addr 172.21.15.47:0/2103374421 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:54.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.379+0000 7efe3bfff700 1 --2- 172.21.15.47:0/2103374421 >> v2:172.21.15.64:3300/0 conn(0x7efe3c0b06d0 0x7efe3c13ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:54.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.379+0000 7efe3affd700 1 -- 172.21.15.47:0/2103374421 >> v2:172.21.15.64:3300/0 conn(0x7efe3c0b06d0 msgr2=0x7efe3c13ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:54.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.379+0000 7efe3affd700 1 --2- 172.21.15.47:0/2103374421 >> v2:172.21.15.64:3300/0 conn(0x7efe3c0b06d0 0x7efe3c13ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:54.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.379+0000 7efe3affd700 1 -- 172.21.15.47:0/2103374421 >> v1:172.21.15.64:6789/0 conn(0x7efe3c0611b0 legacy=0x7efe3c146640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:54.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.379+0000 7efe3affd700 1 -- 172.21.15.47:0/2103374421 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe3c061590 con 0x7efe3c0b0cf0 2022-06-29T15:42:54.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.379+0000 7efe3affd700 1 --2- 172.21.15.47:0/2103374421 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe3c0b0cf0 0x7efe3c146d50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7efe3c0b1c00 tx=0x7efe24006eb0).ready entity=mon.0 client_cookie=48739da01b18d50 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:54.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.379+0000 7efe3a7fc700 1 -- 172.21.15.47:0/2103374421 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe2400d5c0 con 0x7efe3c0b0cf0 2022-06-29T15:42:54.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.379+0000 7efe3a7fc700 1 -- 172.21.15.47:0/2103374421 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe24000e90 con 0x7efe3c0b0cf0 2022-06-29T15:42:54.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.379+0000 7efe3a7fc700 1 -- 172.21.15.47:0/2103374421 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe2400d950 con 0x7efe3c0b0cf0 2022-06-29T15:42:54.394 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.379+0000 7efe41d4c700 1 -- 172.21.15.47:0/2103374421 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe3c0b0cf0 msgr2=0x7efe3c146d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:54.394 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.379+0000 7efe41d4c700 1 --2- 172.21.15.47:0/2103374421 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe3c0b0cf0 0x7efe3c146d50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7efe3c0b1c00 tx=0x7efe24006eb0).stop 2022-06-29T15:42:54.394 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.379+0000 7efe41d4c700 1 -- 172.21.15.47:0/2103374421 shutdown_connections 2022-06-29T15:42:54.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.379+0000 7efe41d4c700 1 --2- 172.21.15.47:0/2103374421 >> v2:172.21.15.64:3300/0 conn(0x7efe3c0b06d0 0x7efe3c13ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:54.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.379+0000 7efe41d4c700 1 --2- 172.21.15.47:0/2103374421 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe3c0b0cf0 0x7efe3c146d50 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:54.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.379+0000 7efe41d4c700 1 -- 172.21.15.47:0/2103374421 >> 172.21.15.47:0/2103374421 conn(0x7efe3c0b98b0 msgr2=0x7efe3c0b9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:54.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.379+0000 7efe41d4c700 1 -- 172.21.15.47:0/2103374421 shutdown_connections 2022-06-29T15:42:54.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.379+0000 7efe41d4c700 1 -- 172.21.15.47:0/2103374421 wait complete. 2022-06-29T15:42:54.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.383+0000 7efe41d4c700 1 Processor -- start 2022-06-29T15:42:54.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.383+0000 7efe41d4c700 1 -- start start 2022-06-29T15:42:54.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.383+0000 7efe41d4c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7efe3c0611b0 0x7efe3c0b8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:54.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.383+0000 7efe41d4c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe3c0b06d0 0x7efe3c143de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:54.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.383+0000 7efe41d4c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe3c0b0cf0 0x7efe3c1527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:54.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.383+0000 7efe41d4c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3c0af330 con 0x7efe3c0b06d0 2022-06-29T15:42:54.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.383+0000 7efe41d4c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3c0af560 con 0x7efe3c0b0cf0 2022-06-29T15:42:54.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.383+0000 7efe41d4c700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe3c0afaa0 con 0x7efe3c0611b0 2022-06-29T15:42:54.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.383+0000 7efe3affd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe3c0b06d0 0x7efe3c143de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:54.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.383+0000 7efe3affd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe3c0b06d0 0x7efe3c143de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46496/0 (socket says 172.21.15.47:46496) 2022-06-29T15:42:54.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.383+0000 7efe3affd700 1 -- 172.21.15.47:0/3281531997 learned_addr learned my addr 172.21.15.47:0/3281531997 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:54.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.383+0000 7efe3b7fe700 1 --2- 172.21.15.47:0/3281531997 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7efe3c0611b0 0x7efe3c0b8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:54.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.383+0000 7efe3bfff700 1 --2- 172.21.15.47:0/3281531997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe3c0b0cf0 0x7efe3c1527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:54.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.383+0000 7efe3affd700 1 -- 172.21.15.47:0/3281531997 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7efe3c0611b0 msgr2=0x7efe3c0b8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:54.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.383+0000 7efe3affd700 1 --2- 172.21.15.47:0/3281531997 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7efe3c0611b0 0x7efe3c0b8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:54.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.383+0000 7efe3affd700 1 -- 172.21.15.47:0/3281531997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe3c0b0cf0 msgr2=0x7efe3c1527f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:54.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.383+0000 7efe3affd700 1 --2- 172.21.15.47:0/3281531997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe3c0b0cf0 0x7efe3c1527f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:54.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.383+0000 7efe3affd700 1 -- 172.21.15.47:0/3281531997 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe2400b1e0 con 0x7efe3c0b06d0 2022-06-29T15:42:54.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.383+0000 7efe3affd700 1 --2- 172.21.15.47:0/3281531997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe3c0b06d0 0x7efe3c143de0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7efe3c0b1c00 tx=0x7efe24015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:54.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.383+0000 7efe38ff9700 1 -- 172.21.15.47:0/3281531997 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe24021070 con 0x7efe3c0b06d0 2022-06-29T15:42:54.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.383+0000 7efe38ff9700 1 -- 172.21.15.47:0/3281531997 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe24011610 con 0x7efe3c0b06d0 2022-06-29T15:42:54.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.383+0000 7efe41d4c700 1 -- 172.21.15.47:0/3281531997 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe3c0afd40 con 0x7efe3c0b06d0 2022-06-29T15:42:54.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.383+0000 7efe38ff9700 1 -- 172.21.15.47:0/3281531997 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe24011970 con 0x7efe3c0b06d0 2022-06-29T15:42:54.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.383+0000 7efe41d4c700 1 -- 172.21.15.47:0/3281531997 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe3c0aef80 con 0x7efe3c0b06d0 2022-06-29T15:42:54.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.387+0000 7efe41d4c700 1 -- 172.21.15.47:0/3281531997 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe3c0b5e00 con 0x7efe3c0b06d0 2022-06-29T15:42:54.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.387+0000 7efe38ff9700 1 -- 172.21.15.47:0/3281531997 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7efe2400f070 con 0x7efe3c0b06d0 2022-06-29T15:42:54.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.387+0000 7efe38ff9700 1 --2- 172.21.15.47:0/3281531997 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7efe20053980 0x7efe20055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:54.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.387+0000 7efe38ff9700 1 -- 172.21.15.47:0/3281531997 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7efe2406d3f0 con 0x7efe3c0b06d0 2022-06-29T15:42:54.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.387+0000 7efe3b7fe700 1 --2- 172.21.15.47:0/3281531997 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7efe20053980 0x7efe20055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:54.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.387+0000 7efe3b7fe700 1 --2- 172.21.15.47:0/3281531997 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7efe20053980 0x7efe20055e40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7efe2c00d000 tx=0x7efe2c007b90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:54.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.399+0000 7efe38ff9700 1 -- 172.21.15.47:0/3281531997 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7efe24044260 con 0x7efe3c0b06d0 2022-06-29T15:42:54.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.527+0000 7efe41d4c700 1 -- 172.21.15.47:0/3281531997 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe3c0573d0 con 0x7efe3c0b06d0 2022-06-29T15:42:54.537 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.531+0000 7efe38ff9700 1 -- 172.21.15.47:0/3281531997 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+16299 (secure 0 0 0) 0x7efe24047880 con 0x7efe3c0b06d0 2022-06-29T15:42:54.538 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:42:54.538 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":26,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:42:34.474438+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:42:28.400446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-06-29T15:42:31.167705+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:42:54.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.531+0000 7efe41d4c700 1 -- 172.21.15.47:0/3281531997 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7efe20053980 msgr2=0x7efe20055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:54.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.531+0000 7efe41d4c700 1 --2- 172.21.15.47:0/3281531997 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7efe20053980 0x7efe20055e40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7efe2c00d000 tx=0x7efe2c007b90).stop 2022-06-29T15:42:54.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.531+0000 7efe41d4c700 1 -- 172.21.15.47:0/3281531997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe3c0b06d0 msgr2=0x7efe3c143de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:54.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.531+0000 7efe41d4c700 1 --2- 172.21.15.47:0/3281531997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe3c0b06d0 0x7efe3c143de0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7efe3c0b1c00 tx=0x7efe24015000).stop 2022-06-29T15:42:54.541 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.531+0000 7efe41d4c700 1 -- 172.21.15.47:0/3281531997 shutdown_connections 2022-06-29T15:42:54.541 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.531+0000 7efe41d4c700 1 --2- 172.21.15.47:0/3281531997 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7efe20053980 0x7efe20055e40 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:54.541 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.531+0000 7efe41d4c700 1 --2- 172.21.15.47:0/3281531997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe3c0b0cf0 0x7efe3c1527f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:54.541 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.531+0000 7efe41d4c700 1 --2- 172.21.15.47:0/3281531997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe3c0b06d0 0x7efe3c143de0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:54.542 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.531+0000 7efe41d4c700 1 --2- 172.21.15.47:0/3281531997 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7efe3c0611b0 0x7efe3c0b8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:54.542 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.531+0000 7efe41d4c700 1 -- 172.21.15.47:0/3281531997 >> 172.21.15.47:0/3281531997 conn(0x7efe3c0b98b0 msgr2=0x7efe3c0b2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:54.542 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.531+0000 7efe41d4c700 1 -- 172.21.15.47:0/3281531997 shutdown_connections 2022-06-29T15:42:54.542 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.531+0000 7efe41d4c700 1 -- 172.21.15.47:0/3281531997 wait complete. 2022-06-29T15:42:54.550 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-29T15:42:16.971431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-29T15:42:25.290884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-29T15:42:28.400446+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2022-06-29T15:42:31.167705+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-29T15:42:54.551 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-29T15:42:54.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.659+0000 7f832d9b3700 1 Processor -- start 2022-06-29T15:42:54.666 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.659+0000 7f832d9b3700 1 -- start start 2022-06-29T15:42:54.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.659+0000 7f832d9b3700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8328146720 0x7f8328140780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:54.676 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.659+0000 7f832d9b3700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f83280b4320 0x7f8328143230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:54.676 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.659+0000 7f832d9b3700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8328061600 con 0x7f8328061220 2022-06-29T15:42:54.677 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.659+0000 7f832d9b3700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8328146bc0 con 0x7f83280b4320 2022-06-29T15:42:54.677 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.659+0000 7f832d9b3700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8328146e60 con 0x7f8328146720 2022-06-29T15:42:54.677 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.659+0000 7f8326ffd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f8328061220 0x7f8328140070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:60288/0 (socket says 172.21.15.47:60288) 2022-06-29T15:42:54.678 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.659+0000 7f8326ffd700 1 -- 172.21.15.47:0/3067198549 learned_addr learned my addr 172.21.15.47:0/3067198549 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:54.678 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.659+0000 7f83277fe700 1 --2- 172.21.15.47:0/3067198549 >> v2:172.21.15.47:3300/0 conn(0x7f83280b4320 0x7f8328143230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:54.679 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.659+0000 7f831e7fc700 1 --2- 172.21.15.47:0/3067198549 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8328146720 0x7f8328140780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:54.679 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.659+0000 7f83267fc700 1 -- 172.21.15.47:0/3067198549 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1035626621 0 0) 0x7f8328061600 con 0x7f8328061220 2022-06-29T15:42:54.680 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.659+0000 7f83267fc700 1 -- 172.21.15.47:0/3067198549 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8308003260 con 0x7f8328061220 2022-06-29T15:42:54.680 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.659+0000 7f83277fe700 1 -- 172.21.15.47:0/3067198549 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8328146720 msgr2=0x7f8328140780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:54.681 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.659+0000 7f83277fe700 1 --2- 172.21.15.47:0/3067198549 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8328146720 0x7f8328140780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:54.681 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.659+0000 7f83277fe700 1 -- 172.21.15.47:0/3067198549 >> v1:172.21.15.47:6789/0 conn(0x7f8328061220 legacy=0x7f8328140070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:54.682 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.659+0000 7f83267fc700 1 -- 172.21.15.47:0/3067198549 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3254983012 0 0) 0x7f8308003260 con 0x7f8328061220 2022-06-29T15:42:54.682 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.659+0000 7f83277fe700 1 -- 172.21.15.47:0/3067198549 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8328147100 con 0x7f83280b4320 2022-06-29T15:42:54.683 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.659+0000 7f83277fe700 1 --2- 172.21.15.47:0/3067198549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83280b4320 0x7f8328143230 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f8318001800 tx=0x7f83200091d0).ready entity=mon.0 client_cookie=6ae3720d491d54a4 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:54.683 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.659+0000 7f83267fc700 1 -- 172.21.15.47:0/3067198549 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8318001660 con 0x7f83280b4320 2022-06-29T15:42:54.684 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.659+0000 7f83267fc700 1 -- 172.21.15.47:0/3067198549 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f832000f510 con 0x7f83280b4320 2022-06-29T15:42:54.684 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.659+0000 7f83267fc700 1 -- 172.21.15.47:0/3067198549 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f832000f680 con 0x7f83280b4320 2022-06-29T15:42:54.684 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.659+0000 7f832d9b3700 1 -- 172.21.15.47:0/3067198549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83280b4320 msgr2=0x7f8328143230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:54.685 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.659+0000 7f832d9b3700 1 --2- 172.21.15.47:0/3067198549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83280b4320 0x7f8328143230 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f8318001800 tx=0x7f83200091d0).stop 2022-06-29T15:42:54.685 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.659+0000 7f832d9b3700 1 -- 172.21.15.47:0/3067198549 shutdown_connections 2022-06-29T15:42:54.686 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.659+0000 7f832d9b3700 1 --2- 172.21.15.47:0/3067198549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83280b4320 0x7f8328143230 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:54.686 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.659+0000 7f832d9b3700 1 --2- 172.21.15.47:0/3067198549 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8328146720 0x7f8328140780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:54.687 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.659+0000 7f832d9b3700 1 -- 172.21.15.47:0/3067198549 >> 172.21.15.47:0/3067198549 conn(0x7f832805c6b0 msgr2=0x7f832805f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:54.687 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.659+0000 7f832d9b3700 1 -- 172.21.15.47:0/3067198549 shutdown_connections 2022-06-29T15:42:54.688 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.659+0000 7f832d9b3700 1 -- 172.21.15.47:0/3067198549 wait complete. 2022-06-29T15:42:54.688 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.663+0000 7f832d9b3700 1 Processor -- start 2022-06-29T15:42:54.689 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.663+0000 7f832d9b3700 1 -- start start 2022-06-29T15:42:54.689 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.663+0000 7f832d9b3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8328061220 0x7f832813db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:54.690 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.663+0000 7f832d9b3700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f83280b4320 0x7f83280b0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:54.690 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.663+0000 7f832d9b3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8328146720 0x7f83280b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:54.691 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.663+0000 7f832d9b3700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83280b66d0 con 0x7f8328146720 2022-06-29T15:42:54.691 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.663+0000 7f832d9b3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83280b76d0 con 0x7f8328061220 2022-06-29T15:42:54.692 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.663+0000 7f832d9b3700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8328156830 con 0x7f83280b4320 2022-06-29T15:42:54.692 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.663+0000 7f8326ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8328061220 0x7f832813db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:54.693 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.663+0000 7f83277fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8328146720 0x7f83280b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:54.693 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.663+0000 7f831e7fc700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f83280b4320 0x7f83280b0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:54.694 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.663+0000 7f8326ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8328061220 0x7f832813db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:52212/0 (socket says 172.21.15.47:52212) 2022-06-29T15:42:54.694 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.663+0000 7f8326ffd700 1 -- 172.21.15.47:0/3931746480 learned_addr learned my addr 172.21.15.47:0/3931746480 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:54.695 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.663+0000 7f83277fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8328146720 0x7f83280b11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46512/0 (socket says 172.21.15.47:46512) 2022-06-29T15:42:54.695 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.663+0000 7f831e7fc700 1 -- 172.21.15.47:0/3931746480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8328061220 msgr2=0x7f832813db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:54.696 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.663+0000 7f831e7fc700 1 --2- 172.21.15.47:0/3931746480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8328061220 0x7f832813db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:54.696 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.663+0000 7f831e7fc700 1 -- 172.21.15.47:0/3931746480 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8328146720 msgr2=0x7f83280b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:54.697 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.663+0000 7f831e7fc700 1 --2- 172.21.15.47:0/3931746480 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8328146720 0x7f83280b11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:54.697 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.663+0000 7f831e7fc700 1 -- 172.21.15.47:0/3931746480 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8320009470 con 0x7f83280b4320 2022-06-29T15:42:54.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.663+0000 7f831e7fc700 1 --2- 172.21.15.47:0/3931746480 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f83280b4320 0x7f83280b0cd0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f831000c000 tx=0x7f8310006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:54.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.663+0000 7f8324ff9700 1 -- 172.21.15.47:0/3931746480 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8310011750 con 0x7f83280b4320 2022-06-29T15:42:54.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.663+0000 7f8324ff9700 1 -- 172.21.15.47:0/3931746480 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83100118b0 con 0x7f83280b4320 2022-06-29T15:42:54.699 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.663+0000 7f8324ff9700 1 -- 172.21.15.47:0/3931746480 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8310011c10 con 0x7f83280b4320 2022-06-29T15:42:54.699 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.667+0000 7f832d9b3700 1 -- 172.21.15.47:0/3931746480 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83281565f0 con 0x7f83280b4320 2022-06-29T15:42:54.700 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.667+0000 7f832d9b3700 1 -- 172.21.15.47:0/3931746480 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83280b86f0 con 0x7f83280b4320 2022-06-29T15:42:54.700 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.667+0000 7f832d9b3700 1 -- 172.21.15.47:0/3931746480 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83280573d0 con 0x7f83280b4320 2022-06-29T15:42:54.701 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.667+0000 7f8324ff9700 1 -- 172.21.15.47:0/3931746480 <== mon.2 v2:172.21.15.64:3301/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8310011d70 con 0x7f83280b4320 2022-06-29T15:42:54.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.667+0000 7f8324ff9700 1 --2- 172.21.15.47:0/3931746480 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f830c053b30 0x7f830c055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:54.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.667+0000 7f8324ff9700 1 -- 172.21.15.47:0/3931746480 <== mon.2 v2:172.21.15.64:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f831006c2c0 con 0x7f83280b4320 2022-06-29T15:42:54.704 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.667+0000 7f8326ffd700 1 --2- 172.21.15.47:0/3931746480 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f830c053b30 0x7f830c055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:54.704 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.671+0000 7f8326ffd700 1 --2- 172.21.15.47:0/3931746480 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f830c053b30 0x7f830c055ff0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f831800d000 tx=0x7f8318001b30).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:54.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.679+0000 7f8324ff9700 1 -- 172.21.15.47:0/3931746480 <== mon.2 v2:172.21.15.64:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8310042f10 con 0x7f83280b4320 2022-06-29T15:42:54.811 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.803+0000 7f832d9b3700 1 -- 172.21.15.47:0/3931746480 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f83280ad7b0 con 0x7f83280b4320 2022-06-29T15:42:54.812 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.803+0000 7f8324ff9700 1 -- 172.21.15.47:0/3931746480 <== mon.2 v2:172.21.15.64:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v26) v1 ==== 110+0+10 (secure 0 0 0) 0x7f8310046530 con 0x7f83280b4320 2022-06-29T15:42:54.813 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2022-06-29T15:42:54.817 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.807+0000 7f832d9b3700 1 -- 172.21.15.47:0/3931746480 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f830c053b30 msgr2=0x7f830c055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:54.817 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.807+0000 7f832d9b3700 1 --2- 172.21.15.47:0/3931746480 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f830c053b30 0x7f830c055ff0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f831800d000 tx=0x7f8318001b30).stop 2022-06-29T15:42:54.818 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.807+0000 7f832d9b3700 1 -- 172.21.15.47:0/3931746480 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f83280b4320 msgr2=0x7f83280b0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:54.818 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.807+0000 7f832d9b3700 1 --2- 172.21.15.47:0/3931746480 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f83280b4320 0x7f83280b0cd0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f831000c000 tx=0x7f8310006df0).stop 2022-06-29T15:42:54.818 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.807+0000 7f832d9b3700 1 -- 172.21.15.47:0/3931746480 shutdown_connections 2022-06-29T15:42:54.818 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.807+0000 7f832d9b3700 1 --2- 172.21.15.47:0/3931746480 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8328146720 0x7f83280b11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:54.819 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.807+0000 7f832d9b3700 1 --2- 172.21.15.47:0/3931746480 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f83280b4320 0x7f83280b0cd0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:54.819 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.807+0000 7f832d9b3700 1 --2- 172.21.15.47:0/3931746480 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f830c053b30 0x7f830c055ff0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:54.819 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.807+0000 7f832d9b3700 1 --2- 172.21.15.47:0/3931746480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8328061220 0x7f832813db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:54.819 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.807+0000 7f832d9b3700 1 -- 172.21.15.47:0/3931746480 >> 172.21.15.47:0/3931746480 conn(0x7f832805c6b0 msgr2=0x7f832813f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:54.819 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.811+0000 7f832d9b3700 1 -- 172.21.15.47:0/3931746480 shutdown_connections 2022-06-29T15:42:54.820 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.811+0000 7f832d9b3700 1 -- 172.21.15.47:0/3931746480 wait complete. 2022-06-29T15:42:54.833 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-29T15:42:54.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.927+0000 7f17c70d8700 1 Processor -- start 2022-06-29T15:42:54.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.927+0000 7f17c70d8700 1 -- start start 2022-06-29T15:42:54.939 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.931+0000 7f17c70d8700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f17c00b1610 0x7f17c00b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:54.939 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.931+0000 7f17c70d8700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17c0060790 con 0x7f17c00b58d0 2022-06-29T15:42:54.940 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.931+0000 7f17c70d8700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17c014ad80 con 0x7f17c014b4c0 2022-06-29T15:42:54.940 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.931+0000 7f17c70d8700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17c00a99e0 con 0x7f17c00b1610 2022-06-29T15:42:54.940 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.931+0000 7f17c60d6700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f17c00b58d0 0x7f17c01475f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:60302/0 (socket says 172.21.15.47:60302) 2022-06-29T15:42:54.941 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.931+0000 7f17c58d5700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f17c00b1610 0x7f17c00b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:54.941 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.931+0000 7f17c60d6700 1 -- 172.21.15.47:0/3189291487 learned_addr learned my addr 172.21.15.47:0/3189291487 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:54.941 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.931+0000 7f17c58d5700 1 -- 172.21.15.47:0/3189291487 >> v1:172.21.15.64:6789/0 conn(0x7f17c014b4c0 legacy=0x7f17c00aa130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:54.941 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.931+0000 7f17c58d5700 1 -- 172.21.15.47:0/3189291487 >> v1:172.21.15.47:6789/0 conn(0x7f17c00b58d0 legacy=0x7f17c01475f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:54.942 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.931+0000 7f17c50d4700 1 -- 172.21.15.47:0/3189291487 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 43879845 0 0) 0x7f17c0060790 con 0x7f17c00b58d0 2022-06-29T15:42:54.942 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.931+0000 7f17c58d5700 1 -- 172.21.15.47:0/3189291487 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c014b020 con 0x7f17c00b1610 2022-06-29T15:42:54.942 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.931+0000 7f17c58d5700 1 --2- 172.21.15.47:0/3189291487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17c00b1610 0x7f17c00b5cb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f17bc0032f0 tx=0x7f17bc0091d0).ready entity=mon.0 client_cookie=1cdbc34acc7633ef server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:54.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.931+0000 7f17c50d4700 1 -- 172.21.15.47:0/3189291487 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f17bc009ea0 con 0x7f17c00b1610 2022-06-29T15:42:54.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.931+0000 7f17c50d4700 1 -- 172.21.15.47:0/3189291487 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17bc00b8a0 con 0x7f17c00b1610 2022-06-29T15:42:54.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.931+0000 7f17c50d4700 1 -- 172.21.15.47:0/3189291487 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f17bc00ba10 con 0x7f17c00b1610 2022-06-29T15:42:54.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.931+0000 7f17c70d8700 1 -- 172.21.15.47:0/3189291487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17c00b1610 msgr2=0x7f17c00b5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:54.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.931+0000 7f17c70d8700 1 --2- 172.21.15.47:0/3189291487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17c00b1610 0x7f17c00b5cb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f17bc0032f0 tx=0x7f17bc0091d0).stop 2022-06-29T15:42:54.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.931+0000 7f17c70d8700 1 -- 172.21.15.47:0/3189291487 shutdown_connections 2022-06-29T15:42:54.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.931+0000 7f17c70d8700 1 --2- 172.21.15.47:0/3189291487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17c00b1610 0x7f17c00b5cb0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:54.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.931+0000 7f17c70d8700 1 -- 172.21.15.47:0/3189291487 >> 172.21.15.47:0/3189291487 conn(0x7f17c0062560 msgr2=0x7f17c0062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:54.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.931+0000 7f17c70d8700 1 -- 172.21.15.47:0/3189291487 shutdown_connections 2022-06-29T15:42:54.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.931+0000 7f17c70d8700 1 -- 172.21.15.47:0/3189291487 wait complete. 2022-06-29T15:42:54.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.931+0000 7f17c70d8700 1 Processor -- start 2022-06-29T15:42:54.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.935+0000 7f17c70d8700 1 -- start start 2022-06-29T15:42:54.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.935+0000 7f17c70d8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17c00b1610 0x7f17c00b0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:54.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.935+0000 7f17c70d8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17c00b58d0 0x7f17c0141d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:54.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.935+0000 7f17c70d8700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f17c014b4c0 0x7f17c0064f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:54.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.935+0000 7f17c70d8700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c00657b0 con 0x7f17c00b58d0 2022-06-29T15:42:54.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.935+0000 7f17c70d8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c01421f0 con 0x7f17c00b1610 2022-06-29T15:42:54.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.935+0000 7f17c70d8700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17c00659e0 con 0x7f17c014b4c0 2022-06-29T15:42:54.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.935+0000 7f17c58d5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17c00b58d0 0x7f17c0141d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:54.950 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.935+0000 7f17c60d6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17c00b1610 0x7f17c00b0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:54.950 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.935+0000 7f17c58d5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17c00b58d0 0x7f17c0141d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46524/0 (socket says 172.21.15.47:46524) 2022-06-29T15:42:54.950 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.935+0000 7f17c58d5700 1 -- 172.21.15.47:0/2947202019 learned_addr learned my addr 172.21.15.47:0/2947202019 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:54.951 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.935+0000 7f17c68d7700 1 --2- 172.21.15.47:0/2947202019 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f17c014b4c0 0x7f17c0064f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:54.951 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.935+0000 7f17c58d5700 1 -- 172.21.15.47:0/2947202019 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f17c014b4c0 msgr2=0x7f17c0064f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:54.951 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.935+0000 7f17c58d5700 1 --2- 172.21.15.47:0/2947202019 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f17c014b4c0 0x7f17c0064f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:54.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.935+0000 7f17c58d5700 1 -- 172.21.15.47:0/2947202019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17c00b1610 msgr2=0x7f17c00b0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:54.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.935+0000 7f17c58d5700 1 --2- 172.21.15.47:0/2947202019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17c00b1610 0x7f17c00b0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:54.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.935+0000 7f17c58d5700 1 -- 172.21.15.47:0/2947202019 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17bc009470 con 0x7f17c00b58d0 2022-06-29T15:42:54.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.935+0000 7f17c58d5700 1 --2- 172.21.15.47:0/2947202019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17c00b58d0 0x7f17c0141d00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f17bc0032f0 tx=0x7f17bc016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:54.953 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.935+0000 7f17af7fe700 1 -- 172.21.15.47:0/2947202019 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f17bc009ea0 con 0x7f17c00b58d0 2022-06-29T15:42:54.953 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.935+0000 7f17af7fe700 1 -- 172.21.15.47:0/2947202019 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17bc01f3f0 con 0x7f17c00b58d0 2022-06-29T15:42:54.953 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.935+0000 7f17af7fe700 1 -- 172.21.15.47:0/2947202019 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f17bc00bee0 con 0x7f17c00b58d0 2022-06-29T15:42:54.953 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.935+0000 7f17c70d8700 1 -- 172.21.15.47:0/2947202019 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17c0066970 con 0x7f17c00b58d0 2022-06-29T15:42:54.954 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.935+0000 7f17c70d8700 1 -- 172.21.15.47:0/2947202019 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c0065470 con 0x7f17c00b58d0 2022-06-29T15:42:54.954 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.939+0000 7f17c70d8700 1 -- 172.21.15.47:0/2947202019 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17c00663d0 con 0x7f17c00b58d0 2022-06-29T15:42:54.954 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.939+0000 7f17af7fe700 1 -- 172.21.15.47:0/2947202019 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f17bc026030 con 0x7f17c00b58d0 2022-06-29T15:42:54.954 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.939+0000 7f17af7fe700 1 --2- 172.21.15.47:0/2947202019 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f17a0053a70 0x7f17a0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:54.954 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.939+0000 7f17af7fe700 1 -- 172.21.15.47:0/2947202019 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f17bc06cd30 con 0x7f17c00b58d0 2022-06-29T15:42:54.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.939+0000 7f17c60d6700 1 --2- 172.21.15.47:0/2947202019 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f17a0053a70 0x7f17a0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:54.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.939+0000 7f17c60d6700 1 --2- 172.21.15.47:0/2947202019 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f17a0053a70 0x7f17a0055f30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f17b000d000 tx=0x7f17b0007a00).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:54.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:54.943+0000 7f17af7fe700 1 -- 172.21.15.47:0/2947202019 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f17bc043af0 con 0x7f17c00b58d0 2022-06-29T15:42:55.076 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.067+0000 7f17c70d8700 1 -- 172.21.15.47:0/2947202019 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f17c00573d0 con 0x7f17c00b58d0 2022-06-29T15:42:55.077 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.071+0000 7f17af7fe700 1 -- 172.21.15.47:0/2947202019 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7f17bc0240a0 con 0x7f17c00b58d0 2022-06-29T15:42:55.077 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:42:55.079 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.071+0000 7f17c70d8700 1 -- 172.21.15.47:0/2947202019 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f17a0053a70 msgr2=0x7f17a0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:55.080 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.071+0000 7f17c70d8700 1 --2- 172.21.15.47:0/2947202019 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f17a0053a70 0x7f17a0055f30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f17b000d000 tx=0x7f17b0007a00).stop 2022-06-29T15:42:55.080 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.071+0000 7f17c70d8700 1 -- 172.21.15.47:0/2947202019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17c00b58d0 msgr2=0x7f17c0141d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:55.080 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.071+0000 7f17c70d8700 1 --2- 172.21.15.47:0/2947202019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17c00b58d0 0x7f17c0141d00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f17bc0032f0 tx=0x7f17bc016000).stop 2022-06-29T15:42:55.081 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.071+0000 7f17c70d8700 1 -- 172.21.15.47:0/2947202019 shutdown_connections 2022-06-29T15:42:55.081 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.071+0000 7f17c70d8700 1 --2- 172.21.15.47:0/2947202019 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f17c014b4c0 0x7f17c0064f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:55.081 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.071+0000 7f17c70d8700 1 --2- 172.21.15.47:0/2947202019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17c00b58d0 0x7f17c0141d00 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:55.081 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.071+0000 7f17c70d8700 1 --2- 172.21.15.47:0/2947202019 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f17a0053a70 0x7f17a0055f30 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:55.082 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.071+0000 7f17c70d8700 1 --2- 172.21.15.47:0/2947202019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17c00b1610 0x7f17c00b0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:55.082 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.071+0000 7f17c70d8700 1 -- 172.21.15.47:0/2947202019 >> 172.21.15.47:0/2947202019 conn(0x7f17c0062560 msgr2=0x7f17c00ab830 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:55.082 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.071+0000 7f17c70d8700 1 -- 172.21.15.47:0/2947202019 shutdown_connections 2022-06-29T15:42:55.082 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.071+0000 7f17c70d8700 1 -- 172.21.15.47:0/2947202019 wait complete. 2022-06-29T15:42:55.091 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-29T15:42:55.198 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.191+0000 7f40cbc5f700 1 Processor -- start 2022-06-29T15:42:55.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.191+0000 7f40cbc5f700 1 -- start start 2022-06-29T15:42:55.203 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.191+0000 7f40cbc5f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f40c405f0b0 0x7f40c405f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:55.203 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.191+0000 7f40cbc5f700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f40c40b4bb0 0x7f40c40aab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:55.203 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.191+0000 7f40cbc5f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f40c4061bd0 0x7f40c40b7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:55.204 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.191+0000 7f40cbc5f700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40c41485a0 con 0x7f40c405f0b0 2022-06-29T15:42:55.204 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.191+0000 7f40cbc5f700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40c4060190 con 0x7f40c4061bd0 2022-06-29T15:42:55.204 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.191+0000 7f40cbc5f700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c4148060 con 0x7f40c40b4bb0 2022-06-29T15:42:55.204 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.191+0000 7f40c99fb700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f40c405f0b0 0x7f40c405f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:55.205 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.191+0000 7f40c99fb700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f40c405f0b0 0x7f40c405f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46530/0 (socket says 172.21.15.47:46530) 2022-06-29T15:42:55.205 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.191+0000 7f40c99fb700 1 -- 172.21.15.47:0/1081682114 learned_addr learned my addr 172.21.15.47:0/1081682114 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:55.205 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.191+0000 7f40ca1fc700 1 --2- 172.21.15.47:0/1081682114 >> v2:172.21.15.64:3300/0 conn(0x7f40c4061bd0 0x7f40c40b7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:55.205 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.191+0000 7f40c91fa700 1 --2- 172.21.15.47:0/1081682114 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f40c40b4bb0 0x7f40c40aab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:55.206 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.191+0000 7f40c99fb700 1 -- 172.21.15.47:0/1081682114 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f40c40b4bb0 msgr2=0x7f40c40aab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:55.206 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.191+0000 7f40c99fb700 1 --2- 172.21.15.47:0/1081682114 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f40c40b4bb0 0x7f40c40aab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:55.206 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.191+0000 7f40c99fb700 1 -- 172.21.15.47:0/1081682114 >> v2:172.21.15.64:3300/0 conn(0x7f40c4061bd0 msgr2=0x7f40c40b7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:55.206 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.191+0000 7f40c99fb700 1 --2- 172.21.15.47:0/1081682114 >> v2:172.21.15.64:3300/0 conn(0x7f40c4061bd0 0x7f40c40b7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.191+0000 7f40c99fb700 1 -- 172.21.15.47:0/1081682114 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c4148300 con 0x7f40c405f0b0 2022-06-29T15:42:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.191+0000 7f40c99fb700 1 --2- 172.21.15.47:0/1081682114 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40c405f0b0 0x7f40c405f490 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f40c0002f70 tx=0x7f40c0006eb0).ready entity=mon.0 client_cookie=9e0163d1627d7cad server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.191+0000 7f40c89f9700 1 -- 172.21.15.47:0/1081682114 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40c000d570 con 0x7f40c405f0b0 2022-06-29T15:42:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.191+0000 7f40c89f9700 1 -- 172.21.15.47:0/1081682114 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c0000e90 con 0x7f40c405f0b0 2022-06-29T15:42:55.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.195+0000 7f40c89f9700 1 -- 172.21.15.47:0/1081682114 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40c000d900 con 0x7f40c405f0b0 2022-06-29T15:42:55.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.195+0000 7f40cbc5f700 1 -- 172.21.15.47:0/1081682114 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40c405f0b0 msgr2=0x7f40c405f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:55.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.195+0000 7f40cbc5f700 1 --2- 172.21.15.47:0/1081682114 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40c405f0b0 0x7f40c405f490 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f40c0002f70 tx=0x7f40c0006eb0).stop 2022-06-29T15:42:55.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.195+0000 7f40cbc5f700 1 -- 172.21.15.47:0/1081682114 shutdown_connections 2022-06-29T15:42:55.209 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.195+0000 7f40cbc5f700 1 --2- 172.21.15.47:0/1081682114 >> v2:172.21.15.64:3300/0 conn(0x7f40c4061bd0 0x7f40c40b7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:55.209 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.195+0000 7f40cbc5f700 1 --2- 172.21.15.47:0/1081682114 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f40c40b4bb0 0x7f40c40aab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:55.209 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.195+0000 7f40cbc5f700 1 --2- 172.21.15.47:0/1081682114 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40c405f0b0 0x7f40c405f490 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:55.210 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.195+0000 7f40cbc5f700 1 -- 172.21.15.47:0/1081682114 >> 172.21.15.47:0/1081682114 conn(0x7f40c40627d0 msgr2=0x7f40c4062bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:55.210 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.195+0000 7f40cbc5f700 1 -- 172.21.15.47:0/1081682114 shutdown_connections 2022-06-29T15:42:55.210 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.195+0000 7f40cbc5f700 1 -- 172.21.15.47:0/1081682114 wait complete. 2022-06-29T15:42:55.210 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.195+0000 7f40cbc5f700 1 Processor -- start 2022-06-29T15:42:55.210 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.195+0000 7f40cbc5f700 1 -- start start 2022-06-29T15:42:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.195+0000 7f40cbc5f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40c405f0b0 0x7f40c4140910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.195+0000 7f40cbc5f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40c4061bd0 0x7f40c40af380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.195+0000 7f40cbc5f700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f40c40b4bb0 0x7f40c40af870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.195+0000 7f40cbc5f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c40b0c20 con 0x7f40c4061bd0 2022-06-29T15:42:55.215 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.195+0000 7f40cbc5f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c414da00 con 0x7f40c405f0b0 2022-06-29T15:42:55.215 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.195+0000 7f40cbc5f700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c40b1860 con 0x7f40c40b4bb0 2022-06-29T15:42:55.215 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.195+0000 7f40c91fa700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40c4061bd0 0x7f40c40af380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:55.215 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.195+0000 7f40c91fa700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40c4061bd0 0x7f40c40af380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46538/0 (socket says 172.21.15.47:46538) 2022-06-29T15:42:55.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.195+0000 7f40c91fa700 1 -- 172.21.15.47:0/1801803110 learned_addr learned my addr 172.21.15.47:0/1801803110 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:55.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.195+0000 7f40c99fb700 1 --2- 172.21.15.47:0/1801803110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40c405f0b0 0x7f40c4140910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:55.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.195+0000 7f40ca1fc700 1 --2- 172.21.15.47:0/1801803110 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f40c40b4bb0 0x7f40c40af870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:55.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.199+0000 7f40c91fa700 1 -- 172.21.15.47:0/1801803110 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f40c40b4bb0 msgr2=0x7f40c40af870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:55.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.199+0000 7f40c91fa700 1 --2- 172.21.15.47:0/1801803110 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f40c40b4bb0 0x7f40c40af870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:55.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.199+0000 7f40c91fa700 1 -- 172.21.15.47:0/1801803110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40c405f0b0 msgr2=0x7f40c4140910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:55.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.199+0000 7f40c91fa700 1 --2- 172.21.15.47:0/1801803110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40c405f0b0 0x7f40c4140910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:55.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.199+0000 7f40c91fa700 1 -- 172.21.15.47:0/1801803110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c000b1e0 con 0x7f40c4061bd0 2022-06-29T15:42:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.199+0000 7f40ca1fc700 1 --2- 172.21.15.47:0/1801803110 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f40c40b4bb0 0x7f40c40af870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:42:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.199+0000 7f40c91fa700 1 --2- 172.21.15.47:0/1801803110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40c4061bd0 0x7f40c40af380 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f40b400a000 tx=0x7f40b4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.199+0000 7f40b2ffd700 1 -- 172.21.15.47:0/1801803110 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40b4011780 con 0x7f40c4061bd0 2022-06-29T15:42:55.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.199+0000 7f40b2ffd700 1 -- 172.21.15.47:0/1801803110 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b40118e0 con 0x7f40c4061bd0 2022-06-29T15:42:55.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.199+0000 7f40cbc5f700 1 -- 172.21.15.47:0/1801803110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40c414cd40 con 0x7f40c4061bd0 2022-06-29T15:42:55.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.199+0000 7f40b2ffd700 1 -- 172.21.15.47:0/1801803110 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40b4011c40 con 0x7f40c4061bd0 2022-06-29T15:42:55.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.199+0000 7f40cbc5f700 1 -- 172.21.15.47:0/1801803110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40c4141320 con 0x7f40c4061bd0 2022-06-29T15:42:55.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.199+0000 7f40cbc5f700 1 -- 172.21.15.47:0/1801803110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40c40b1a20 con 0x7f40c4061bd0 2022-06-29T15:42:55.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.199+0000 7f40b2ffd700 1 -- 172.21.15.47:0/1801803110 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f40b4012420 con 0x7f40c4061bd0 2022-06-29T15:42:55.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.199+0000 7f40b2ffd700 1 --2- 172.21.15.47:0/1801803110 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f40a4053950 0x7f40a4055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:55.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.199+0000 7f40b2ffd700 1 -- 172.21.15.47:0/1801803110 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f40b406be40 con 0x7f40c4061bd0 2022-06-29T15:42:55.222 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.199+0000 7f40c99fb700 1 --2- 172.21.15.47:0/1801803110 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f40a4053950 0x7f40a4055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:55.222 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.203+0000 7f40c99fb700 1 --2- 172.21.15.47:0/1801803110 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f40a4053950 0x7f40a4055e10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f40c001c000 tx=0x7f40c0015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:55.222 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.203+0000 7f40b2ffd700 1 -- 172.21.15.47:0/1801803110 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f40b4042ba0 con 0x7f40c4061bd0 2022-06-29T15:42:55.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.331+0000 7f40cbc5f700 1 -- 172.21.15.47:0/1801803110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f40c414ced0 con 0x7f40c4061bd0 2022-06-29T15:42:55.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.331+0000 7f40b2ffd700 1 -- 172.21.15.47:0/1801803110 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+10 (secure 0 0 0) 0x7f40b40461c0 con 0x7f40c4061bd0 2022-06-29T15:42:55.339 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:42:55.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.331+0000 7f40cbc5f700 1 -- 172.21.15.47:0/1801803110 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f40a4053950 msgr2=0x7f40a4055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:55.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.331+0000 7f40cbc5f700 1 --2- 172.21.15.47:0/1801803110 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f40a4053950 0x7f40a4055e10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f40c001c000 tx=0x7f40c0015000).stop 2022-06-29T15:42:55.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.331+0000 7f40cbc5f700 1 -- 172.21.15.47:0/1801803110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40c4061bd0 msgr2=0x7f40c40af380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:55.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.331+0000 7f40cbc5f700 1 --2- 172.21.15.47:0/1801803110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40c4061bd0 0x7f40c40af380 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f40b400a000 tx=0x7f40b4008040).stop 2022-06-29T15:42:55.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.331+0000 7f40cbc5f700 1 -- 172.21.15.47:0/1801803110 shutdown_connections 2022-06-29T15:42:55.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.331+0000 7f40cbc5f700 1 --2- 172.21.15.47:0/1801803110 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f40c40b4bb0 0x7f40c40af870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:55.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.331+0000 7f40cbc5f700 1 --2- 172.21.15.47:0/1801803110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40c4061bd0 0x7f40c40af380 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:55.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.331+0000 7f40cbc5f700 1 --2- 172.21.15.47:0/1801803110 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f40a4053950 0x7f40a4055e10 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:55.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.331+0000 7f40cbc5f700 1 --2- 172.21.15.47:0/1801803110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40c405f0b0 0x7f40c4140910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:55.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.331+0000 7f40cbc5f700 1 -- 172.21.15.47:0/1801803110 >> 172.21.15.47:0/1801803110 conn(0x7f40c40627d0 msgr2=0x7f40c414de60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:55.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.335+0000 7f40cbc5f700 1 -- 172.21.15.47:0/1801803110 shutdown_connections 2022-06-29T15:42:55.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.335+0000 7f40cbc5f700 1 -- 172.21.15.47:0/1801803110 wait complete. 2022-06-29T15:42:55.352 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-29T15:42:55.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.455+0000 7f24da12a700 1 Processor -- start 2022-06-29T15:42:55.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.455+0000 7f24da12a700 1 -- start start 2022-06-29T15:42:55.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.455+0000 7f24da12a700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f24d40b6d20 0x7f24d40a9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:55.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.455+0000 7f24da12a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f24d4063100 0x7f24d413e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:55.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.455+0000 7f24da12a700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24d40b7100 con 0x7f24d405f350 2022-06-29T15:42:55.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.455+0000 7f24da12a700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24d405fdc0 con 0x7f24d4063100 2022-06-29T15:42:55.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.455+0000 7f24da12a700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24d4141db0 con 0x7f24d40b6d20 2022-06-29T15:42:55.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.455+0000 7f24d9929700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f24d4063100 0x7f24d413e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:55.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.455+0000 7f24d8927700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f24d405f350 0x7f24d40612b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:60332/0 (socket says 172.21.15.47:60332) 2022-06-29T15:42:55.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.455+0000 7f24d8927700 1 -- 172.21.15.47:0/3132128196 learned_addr learned my addr 172.21.15.47:0/3132128196 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:55.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.455+0000 7f24d9128700 1 --2- 172.21.15.47:0/3132128196 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f24d40b6d20 0x7f24d40a9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:55.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.455+0000 7f24d9929700 1 -- 172.21.15.47:0/3132128196 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f24d40b6d20 msgr2=0x7f24d40a9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:55.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.455+0000 7f24d9929700 1 --2- 172.21.15.47:0/3132128196 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f24d40b6d20 0x7f24d40a9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:55.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.455+0000 7f24d9929700 1 -- 172.21.15.47:0/3132128196 >> v1:172.21.15.47:6789/0 conn(0x7f24d405f350 legacy=0x7f24d40612b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:55.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.455+0000 7f24cbfff700 1 -- 172.21.15.47:0/3132128196 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2890926132 0 0) 0x7f24d40b7100 con 0x7f24d405f350 2022-06-29T15:42:55.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.455+0000 7f24d9929700 1 -- 172.21.15.47:0/3132128196 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d40619c0 con 0x7f24d4063100 2022-06-29T15:42:55.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.455+0000 7f24d9929700 1 --2- 172.21.15.47:0/3132128196 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24d4063100 0x7f24d413e2b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f24cc0032f0 tx=0x7f24cc0091d0).ready entity=mon.0 client_cookie=947d93622eba65d5 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:55.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.455+0000 7f24cbfff700 1 -- 172.21.15.47:0/3132128196 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24cc00b6c0 con 0x7f24d4063100 2022-06-29T15:42:55.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.455+0000 7f24cbfff700 1 -- 172.21.15.47:0/3132128196 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24cc00b820 con 0x7f24d4063100 2022-06-29T15:42:55.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.455+0000 7f24cbfff700 1 -- 172.21.15.47:0/3132128196 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24cc00b990 con 0x7f24d4063100 2022-06-29T15:42:55.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.455+0000 7f24da12a700 1 -- 172.21.15.47:0/3132128196 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24d4063100 msgr2=0x7f24d413e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:55.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.455+0000 7f24da12a700 1 --2- 172.21.15.47:0/3132128196 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24d4063100 0x7f24d413e2b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f24cc0032f0 tx=0x7f24cc0091d0).stop 2022-06-29T15:42:55.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.459+0000 7f24da12a700 1 -- 172.21.15.47:0/3132128196 shutdown_connections 2022-06-29T15:42:55.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.459+0000 7f24da12a700 1 --2- 172.21.15.47:0/3132128196 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24d4063100 0x7f24d413e2b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:55.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.459+0000 7f24da12a700 1 --2- 172.21.15.47:0/3132128196 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f24d40b6d20 0x7f24d40a9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:55.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.459+0000 7f24da12a700 1 -- 172.21.15.47:0/3132128196 >> 172.21.15.47:0/3132128196 conn(0x7f24d4063670 msgr2=0x7f24d4063a50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:55.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.459+0000 7f24da12a700 1 -- 172.21.15.47:0/3132128196 shutdown_connections 2022-06-29T15:42:55.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.459+0000 7f24da12a700 1 -- 172.21.15.47:0/3132128196 wait complete. 2022-06-29T15:42:55.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.459+0000 7f24da12a700 1 Processor -- start 2022-06-29T15:42:55.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.459+0000 7f24da12a700 1 -- start start 2022-06-29T15:42:55.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.463+0000 7f24da12a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24d405f350 0x7f24d4146d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:55.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.463+0000 7f24da12a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24d4063100 0x7f24d40b5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:55.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.463+0000 7f24da12a700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f24d40b6d20 0x7f24d40b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:55.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.463+0000 7f24da12a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d40b6110 con 0x7f24d4063100 2022-06-29T15:42:55.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.463+0000 7f24da12a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d40b5a50 con 0x7f24d405f350 2022-06-29T15:42:55.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.463+0000 7f24da12a700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24d40b49a0 con 0x7f24d40b6d20 2022-06-29T15:42:55.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.463+0000 7f24d8927700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24d4063100 0x7f24d40b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:55.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.463+0000 7f24d8927700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24d4063100 0x7f24d40b5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46552/0 (socket says 172.21.15.47:46552) 2022-06-29T15:42:55.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.463+0000 7f24d8927700 1 -- 172.21.15.47:0/3494953020 learned_addr learned my addr 172.21.15.47:0/3494953020 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:55.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.463+0000 7f24d9128700 1 --2- 172.21.15.47:0/3494953020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24d405f350 0x7f24d4146d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:55.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.463+0000 7f24d9929700 1 --2- 172.21.15.47:0/3494953020 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f24d40b6d20 0x7f24d40b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:55.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.463+0000 7f24d8927700 1 -- 172.21.15.47:0/3494953020 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f24d40b6d20 msgr2=0x7f24d40b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:55.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.463+0000 7f24d8927700 1 --2- 172.21.15.47:0/3494953020 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f24d40b6d20 0x7f24d40b5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:55.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.463+0000 7f24d8927700 1 -- 172.21.15.47:0/3494953020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24d405f350 msgr2=0x7f24d4146d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:55.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.463+0000 7f24d8927700 1 --2- 172.21.15.47:0/3494953020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24d405f350 0x7f24d4146d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:55.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.463+0000 7f24d8927700 1 -- 172.21.15.47:0/3494953020 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24cc009470 con 0x7f24d4063100 2022-06-29T15:42:55.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.463+0000 7f24d8927700 1 --2- 172.21.15.47:0/3494953020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24d4063100 0x7f24d40b5070 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f24d40b5b90 tx=0x7f24c4007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:55.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.463+0000 7f24ca7fc700 1 -- 172.21.15.47:0/3494953020 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24c40070d0 con 0x7f24d4063100 2022-06-29T15:42:55.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.463+0000 7f24ca7fc700 1 -- 172.21.15.47:0/3494953020 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24c4007230 con 0x7f24d4063100 2022-06-29T15:42:55.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.463+0000 7f24da12a700 1 -- 172.21.15.47:0/3494953020 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24d40b4bd0 con 0x7f24d4063100 2022-06-29T15:42:55.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.463+0000 7f24da12a700 1 -- 172.21.15.47:0/3494953020 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24d4150150 con 0x7f24d4063100 2022-06-29T15:42:55.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.463+0000 7f24ca7fc700 1 -- 172.21.15.47:0/3494953020 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24c4005610 con 0x7f24d4063100 2022-06-29T15:42:55.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.467+0000 7f24da12a700 1 -- 172.21.15.47:0/3494953020 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24d40b4d60 con 0x7f24d4063100 2022-06-29T15:42:55.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.467+0000 7f24ca7fc700 1 -- 172.21.15.47:0/3494953020 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f24c4010070 con 0x7f24d4063100 2022-06-29T15:42:55.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.467+0000 7f24ca7fc700 1 --2- 172.21.15.47:0/3494953020 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f24bc053b00 0x7f24bc055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:55.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.467+0000 7f24ca7fc700 1 -- 172.21.15.47:0/3494953020 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f24c406b700 con 0x7f24d4063100 2022-06-29T15:42:55.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.467+0000 7f24d9128700 1 --2- 172.21.15.47:0/3494953020 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f24bc053b00 0x7f24bc055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:55.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.467+0000 7f24d9128700 1 --2- 172.21.15.47:0/3494953020 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f24bc053b00 0x7f24bc055fc0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f24d000b000 tx=0x7f24d0006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:55.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.471+0000 7f24ca7fc700 1 -- 172.21.15.47:0/3494953020 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f24c4042420 con 0x7f24d4063100 2022-06-29T15:42:55.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.599+0000 7f24da12a700 1 -- 172.21.15.47:0/3494953020 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f24d40573d0 con 0x7f24d4063100 2022-06-29T15:42:55.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.599+0000 7f24ca7fc700 1 -- 172.21.15.47:0/3494953020 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+10 (secure 0 0 0) 0x7f24c4045a40 con 0x7f24d4063100 2022-06-29T15:42:55.607 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:42:55.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.603+0000 7f24da12a700 1 -- 172.21.15.47:0/3494953020 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f24bc053b00 msgr2=0x7f24bc055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:55.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.603+0000 7f24da12a700 1 --2- 172.21.15.47:0/3494953020 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f24bc053b00 0x7f24bc055fc0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f24d000b000 tx=0x7f24d0006df0).stop 2022-06-29T15:42:55.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.603+0000 7f24da12a700 1 -- 172.21.15.47:0/3494953020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24d4063100 msgr2=0x7f24d40b5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:55.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.603+0000 7f24da12a700 1 --2- 172.21.15.47:0/3494953020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24d4063100 0x7f24d40b5070 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f24d40b5b90 tx=0x7f24c4007ac0).stop 2022-06-29T15:42:55.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.603+0000 7f24da12a700 1 -- 172.21.15.47:0/3494953020 shutdown_connections 2022-06-29T15:42:55.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.603+0000 7f24da12a700 1 --2- 172.21.15.47:0/3494953020 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f24d40b6d20 0x7f24d40b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:55.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.603+0000 7f24da12a700 1 --2- 172.21.15.47:0/3494953020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24d4063100 0x7f24d40b5070 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:55.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.603+0000 7f24da12a700 1 --2- 172.21.15.47:0/3494953020 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f24bc053b00 0x7f24bc055fc0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:55.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.603+0000 7f24da12a700 1 --2- 172.21.15.47:0/3494953020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24d405f350 0x7f24d4146d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:55.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.603+0000 7f24da12a700 1 -- 172.21.15.47:0/3494953020 >> 172.21.15.47:0/3494953020 conn(0x7f24d4063670 msgr2=0x7f24d4141220 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:55.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.603+0000 7f24da12a700 1 -- 172.21.15.47:0/3494953020 shutdown_connections 2022-06-29T15:42:55.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.603+0000 7f24da12a700 1 -- 172.21.15.47:0/3494953020 wait complete. 2022-06-29T15:42:55.620 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:42:55.725 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.719+0000 7fbe3ffff700 1 Processor -- start 2022-06-29T15:42:55.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.719+0000 7fbe3ffff700 1 -- start start 2022-06-29T15:42:55.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.719+0000 7fbe3ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbe4005fa30 0x7fbe400b1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:55.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.719+0000 7fbe3ffff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbe40060b90 0x7fbe4013f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:55.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.719+0000 7fbe3ffff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe40060fd0 con 0x7fbe400b2770 2022-06-29T15:42:55.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.719+0000 7fbe3ffff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe4005fe10 con 0x7fbe4005fa30 2022-06-29T15:42:55.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.719+0000 7fbe3ffff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe400600e0 con 0x7fbe40060b90 2022-06-29T15:42:55.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.719+0000 7fbe3e7fc700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbe40060b90 0x7fbe4013f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:55.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.719+0000 7fbe3effd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbe4005fa30 0x7fbe400b1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:55.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.719+0000 7fbe3e7fc700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbe40060b90 0x7fbe4013f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:43660/0 (socket says 172.21.15.47:43660) 2022-06-29T15:42:55.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.719+0000 7fbe3e7fc700 1 -- 172.21.15.47:0/391155665 learned_addr learned my addr 172.21.15.47:0/391155665 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:55.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.719+0000 7fbe3f7fe700 1 --1- 172.21.15.47:0/391155665 >> v1:172.21.15.64:6789/0 conn(0x7fbe400b2770 0x7fbe400ae170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:39680/0 (socket says 172.21.15.47:39680) 2022-06-29T15:42:55.732 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.719+0000 7fbe3e7fc700 1 -- 172.21.15.47:0/391155665 >> v2:172.21.15.64:3300/0 conn(0x7fbe4005fa30 msgr2=0x7fbe400b1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:55.732 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.719+0000 7fbe3e7fc700 1 --2- 172.21.15.47:0/391155665 >> v2:172.21.15.64:3300/0 conn(0x7fbe4005fa30 0x7fbe400b1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:55.732 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.719+0000 7fbe3e7fc700 1 -- 172.21.15.47:0/391155665 >> v1:172.21.15.64:6789/0 conn(0x7fbe400b2770 legacy=0x7fbe400ae170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:55.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.719+0000 7fbe3e7fc700 1 -- 172.21.15.47:0/391155665 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe40060380 con 0x7fbe40060b90 2022-06-29T15:42:55.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.723+0000 7fbe3e7fc700 1 --2- 172.21.15.47:0/391155665 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbe40060b90 0x7fbe4013f470 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fbe400b2650 tx=0x7fbe34006eb0).ready entity=mon.2 client_cookie=7a3aa766d89fde65 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:55.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.723+0000 7fbe3dffb700 1 -- 172.21.15.47:0/391155665 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe3400fa60 con 0x7fbe40060b90 2022-06-29T15:42:55.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.723+0000 7fbe3dffb700 1 -- 172.21.15.47:0/391155665 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe34000e90 con 0x7fbe40060b90 2022-06-29T15:42:55.734 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.723+0000 7fbe3dffb700 1 -- 172.21.15.47:0/391155665 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe3400fdf0 con 0x7fbe40060b90 2022-06-29T15:42:55.734 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.723+0000 7fbe3ffff700 1 -- 172.21.15.47:0/391155665 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbe40060b90 msgr2=0x7fbe4013f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:55.734 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.723+0000 7fbe3ffff700 1 --2- 172.21.15.47:0/391155665 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbe40060b90 0x7fbe4013f470 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fbe400b2650 tx=0x7fbe34006eb0).stop 2022-06-29T15:42:55.734 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.723+0000 7fbe3ffff700 1 -- 172.21.15.47:0/391155665 shutdown_connections 2022-06-29T15:42:55.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.723+0000 7fbe3ffff700 1 --2- 172.21.15.47:0/391155665 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbe40060b90 0x7fbe4013f470 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:55.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.723+0000 7fbe3ffff700 1 --2- 172.21.15.47:0/391155665 >> v2:172.21.15.64:3300/0 conn(0x7fbe4005fa30 0x7fbe400b1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:55.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.723+0000 7fbe3ffff700 1 -- 172.21.15.47:0/391155665 >> 172.21.15.47:0/391155665 conn(0x7fbe40062df0 msgr2=0x7fbe400631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:55.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.723+0000 7fbe3ffff700 1 -- 172.21.15.47:0/391155665 shutdown_connections 2022-06-29T15:42:55.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.723+0000 7fbe3ffff700 1 -- 172.21.15.47:0/391155665 wait complete. 2022-06-29T15:42:55.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.727+0000 7fbe3ffff700 1 Processor -- start 2022-06-29T15:42:55.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.727+0000 7fbe3ffff700 1 -- start start 2022-06-29T15:42:55.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.727+0000 7fbe3ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe4005fa30 0x7fbe4009a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:55.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.727+0000 7fbe3ffff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbe40060b90 0x7fbe40151bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:55.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.727+0000 7fbe3ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe400b2770 0x7fbe401413b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:55.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.727+0000 7fbe3ffff700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe401432b0 con 0x7fbe4005fa30 2022-06-29T15:42:55.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.727+0000 7fbe3ffff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe40140940 con 0x7fbe400b2770 2022-06-29T15:42:55.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.727+0000 7fbe3ffff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe40143490 con 0x7fbe40060b90 2022-06-29T15:42:55.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.727+0000 7fbe3effd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe4005fa30 0x7fbe4009a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:55.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.727+0000 7fbe3effd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe4005fa30 0x7fbe4009a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46564/0 (socket says 172.21.15.47:46564) 2022-06-29T15:42:55.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.727+0000 7fbe3effd700 1 -- 172.21.15.47:0/1079563383 learned_addr learned my addr 172.21.15.47:0/1079563383 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:55.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.727+0000 7fbe3e7fc700 1 --2- 172.21.15.47:0/1079563383 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbe40060b90 0x7fbe40151bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:55.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.727+0000 7fbe3f7fe700 1 --2- 172.21.15.47:0/1079563383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe400b2770 0x7fbe401413b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:55.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.727+0000 7fbe3effd700 1 -- 172.21.15.47:0/1079563383 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbe40060b90 msgr2=0x7fbe40151bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:55.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.727+0000 7fbe3effd700 1 --2- 172.21.15.47:0/1079563383 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbe40060b90 0x7fbe40151bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:55.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.727+0000 7fbe3effd700 1 -- 172.21.15.47:0/1079563383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe400b2770 msgr2=0x7fbe401413b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:55.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.727+0000 7fbe3effd700 1 --2- 172.21.15.47:0/1079563383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe400b2770 0x7fbe401413b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:55.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.727+0000 7fbe3effd700 1 -- 172.21.15.47:0/1079563383 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe3400b1e0 con 0x7fbe4005fa30 2022-06-29T15:42:55.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.727+0000 7fbe3effd700 1 --2- 172.21.15.47:0/1079563383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe4005fa30 0x7fbe4009a000 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7fbe3000c000 tx=0x7fbe30006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:55.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.727+0000 7fbe2ffff700 1 -- 172.21.15.47:0/1079563383 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe30010780 con 0x7fbe4005fa30 2022-06-29T15:42:55.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.727+0000 7fbe2ffff700 1 -- 172.21.15.47:0/1079563383 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe300108e0 con 0x7fbe4005fa30 2022-06-29T15:42:55.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.727+0000 7fbe2ffff700 1 -- 172.21.15.47:0/1079563383 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe30010c40 con 0x7fbe4005fa30 2022-06-29T15:42:55.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.727+0000 7fbe3ffff700 1 -- 172.21.15.47:0/1079563383 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe40143730 con 0x7fbe4005fa30 2022-06-29T15:42:55.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.727+0000 7fbe3ffff700 1 -- 172.21.15.47:0/1079563383 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe401520b0 con 0x7fbe4005fa30 2022-06-29T15:42:55.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.731+0000 7fbe3ffff700 1 -- 172.21.15.47:0/1079563383 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe400b16b0 con 0x7fbe4005fa30 2022-06-29T15:42:55.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.731+0000 7fbe2ffff700 1 -- 172.21.15.47:0/1079563383 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fbe3001c460 con 0x7fbe4005fa30 2022-06-29T15:42:55.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.731+0000 7fbe2ffff700 1 --2- 172.21.15.47:0/1079563383 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbe1c053950 0x7fbe1c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:55.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.731+0000 7fbe3e7fc700 1 --2- 172.21.15.47:0/1079563383 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbe1c053950 0x7fbe1c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:55.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.731+0000 7fbe2ffff700 1 -- 172.21.15.47:0/1079563383 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fbe3006c3a0 con 0x7fbe4005fa30 2022-06-29T15:42:55.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.731+0000 7fbe3e7fc700 1 --2- 172.21.15.47:0/1079563383 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbe1c053950 0x7fbe1c055e10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7fbe3401e000 tx=0x7fbe34017000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:55.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.735+0000 7fbe2ffff700 1 -- 172.21.15.47:0/1079563383 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbe30043100 con 0x7fbe4005fa30 2022-06-29T15:42:55.869 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.863+0000 7fbe3ffff700 1 -- 172.21.15.47:0/1079563383 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe400573d0 con 0x7fbe4005fa30 2022-06-29T15:42:55.871 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.863+0000 7fbe2ffff700 1 -- 172.21.15.47:0/1079563383 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+16299 (secure 0 0 0) 0x7fbe30046720 con 0x7fbe4005fa30 2022-06-29T15:42:55.872 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:42:55.872 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":26,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:42:34.474438+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:42:28.400446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-06-29T15:42:31.167705+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:42:55.873 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.867+0000 7fbe3ffff700 1 -- 172.21.15.47:0/1079563383 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbe1c053950 msgr2=0x7fbe1c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:55.873 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.867+0000 7fbe3ffff700 1 --2- 172.21.15.47:0/1079563383 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbe1c053950 0x7fbe1c055e10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7fbe3401e000 tx=0x7fbe34017000).stop 2022-06-29T15:42:55.874 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.867+0000 7fbe3ffff700 1 -- 172.21.15.47:0/1079563383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe4005fa30 msgr2=0x7fbe4009a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:55.875 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.867+0000 7fbe3ffff700 1 --2- 172.21.15.47:0/1079563383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe4005fa30 0x7fbe4009a000 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7fbe3000c000 tx=0x7fbe30006df0).stop 2022-06-29T15:42:55.875 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.867+0000 7fbe3ffff700 1 -- 172.21.15.47:0/1079563383 shutdown_connections 2022-06-29T15:42:55.875 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.867+0000 7fbe3ffff700 1 --2- 172.21.15.47:0/1079563383 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbe1c053950 0x7fbe1c055e10 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:55.875 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.867+0000 7fbe3ffff700 1 --2- 172.21.15.47:0/1079563383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe400b2770 0x7fbe401413b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:55.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.867+0000 7fbe3ffff700 1 --2- 172.21.15.47:0/1079563383 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbe40060b90 0x7fbe40151bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:55.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.867+0000 7fbe3ffff700 1 --2- 172.21.15.47:0/1079563383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe4005fa30 0x7fbe4009a000 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:55.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.867+0000 7fbe3ffff700 1 -- 172.21.15.47:0/1079563383 >> 172.21.15.47:0/1079563383 conn(0x7fbe40062df0 msgr2=0x7fbe400b0250 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:55.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.867+0000 7fbe3ffff700 1 -- 172.21.15.47:0/1079563383 shutdown_connections 2022-06-29T15:42:55.877 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.867+0000 7fbe3ffff700 1 -- 172.21.15.47:0/1079563383 wait complete. 2022-06-29T15:42:55.885 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-29T15:42:16.971431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-29T15:42:25.290884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-29T15:42:28.400446+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2022-06-29T15:42:31.167705+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-29T15:42:55.886 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-29T15:42:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.991+0000 7fc85f59e700 1 Processor -- start 2022-06-29T15:42:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.991+0000 7fc85f59e700 1 -- start start 2022-06-29T15:42:56.001 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.991+0000 7fc85f59e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc860066fe0 0x7fc8600b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:56.001 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.991+0000 7fc85f59e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8600b68a0 con 0x7fc8600b64c0 2022-06-29T15:42:56.001 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.991+0000 7fc85f59e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8601473d0 con 0x7fc860143150 2022-06-29T15:42:56.002 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.991+0000 7fc85f59e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8600b13a0 con 0x7fc860066fe0 2022-06-29T15:42:56.002 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.991+0000 7fc85dd9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc860066fe0 0x7fc8600b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:56.002 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.991+0000 7fc85e59c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc8600b64c0 0x7fc8600b4b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:60358/0 (socket says 172.21.15.47:60358) 2022-06-29T15:42:56.003 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.991+0000 7fc85dd9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc860066fe0 0x7fc8600b5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46574/0 (socket says 172.21.15.47:46574) 2022-06-29T15:42:56.003 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.991+0000 7fc85e59c700 1 -- 172.21.15.47:0/3411918881 learned_addr learned my addr 172.21.15.47:0/3411918881 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:56.003 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.991+0000 7fc85dd9b700 1 -- 172.21.15.47:0/3411918881 >> v1:172.21.15.64:6789/0 conn(0x7fc860143150 legacy=0x7fc86012bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:56.003 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.991+0000 7fc85dd9b700 1 -- 172.21.15.47:0/3411918881 >> v1:172.21.15.47:6789/0 conn(0x7fc8600b64c0 legacy=0x7fc8600b4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:56.003 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.991+0000 7fc85dd9b700 1 -- 172.21.15.47:0/3411918881 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8600b46a0 con 0x7fc860066fe0 2022-06-29T15:42:56.004 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.995+0000 7fc85dd9b700 1 --2- 172.21.15.47:0/3411918881 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc860066fe0 0x7fc8600b5a20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7fc850002f70 tx=0x7fc850006d90).ready entity=mon.0 client_cookie=2b3495eee5ed53b4 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:56.004 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.995+0000 7fc85d59a700 1 -- 172.21.15.47:0/3411918881 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc85000d4a0 con 0x7fc860066fe0 2022-06-29T15:42:56.004 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.995+0000 7fc85d59a700 1 -- 172.21.15.47:0/3411918881 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc850000e90 con 0x7fc860066fe0 2022-06-29T15:42:56.004 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.995+0000 7fc85d59a700 1 -- 172.21.15.47:0/3411918881 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc85000d7f0 con 0x7fc860066fe0 2022-06-29T15:42:56.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.995+0000 7fc85f59e700 1 -- 172.21.15.47:0/3411918881 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc860066fe0 msgr2=0x7fc8600b5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:56.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.995+0000 7fc85f59e700 1 --2- 172.21.15.47:0/3411918881 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc860066fe0 0x7fc8600b5a20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7fc850002f70 tx=0x7fc850006d90).stop 2022-06-29T15:42:56.009 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.995+0000 7fc85f59e700 1 -- 172.21.15.47:0/3411918881 shutdown_connections 2022-06-29T15:42:56.009 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.995+0000 7fc85f59e700 1 --2- 172.21.15.47:0/3411918881 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc860066fe0 0x7fc8600b5a20 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:56.009 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.995+0000 7fc85f59e700 1 -- 172.21.15.47:0/3411918881 >> 172.21.15.47:0/3411918881 conn(0x7fc860060e00 msgr2=0x7fc8600611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:56.009 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.995+0000 7fc85f59e700 1 -- 172.21.15.47:0/3411918881 shutdown_connections 2022-06-29T15:42:56.010 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.995+0000 7fc85f59e700 1 -- 172.21.15.47:0/3411918881 wait complete. 2022-06-29T15:42:56.011 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.995+0000 7fc85f59e700 1 Processor -- start 2022-06-29T15:42:56.012 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.995+0000 7fc85f59e700 1 -- start start 2022-06-29T15:42:56.012 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.995+0000 7fc85f59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc860066fe0 0x7fc8600b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:56.012 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.995+0000 7fc85f59e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc8600b64c0 0x7fc8601562b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:56.013 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.995+0000 7fc85f59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc860143150 0x7fc8600ad560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:56.013 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.995+0000 7fc85f59e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8601567a0 con 0x7fc860066fe0 2022-06-29T15:42:56.013 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.995+0000 7fc85f59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc860155910 con 0x7fc860143150 2022-06-29T15:42:56.013 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.995+0000 7fc85f59e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc860155b70 con 0x7fc8600b64c0 2022-06-29T15:42:56.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.995+0000 7fc85e59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc860066fe0 0x7fc8600b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:56.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.995+0000 7fc85e59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc860066fe0 0x7fc8600b2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46578/0 (socket says 172.21.15.47:46578) 2022-06-29T15:42:56.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.995+0000 7fc85e59c700 1 -- 172.21.15.47:0/2393383994 learned_addr learned my addr 172.21.15.47:0/2393383994 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:56.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.999+0000 7fc85ed9d700 1 --2- 172.21.15.47:0/2393383994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc860143150 0x7fc8600ad560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:56.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.999+0000 7fc85dd9b700 1 --2- 172.21.15.47:0/2393383994 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc8600b64c0 0x7fc8601562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:56.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.999+0000 7fc85e59c700 1 -- 172.21.15.47:0/2393383994 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc8600b64c0 msgr2=0x7fc8601562b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:56.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.999+0000 7fc85e59c700 1 --2- 172.21.15.47:0/2393383994 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc8600b64c0 0x7fc8601562b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:56.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.999+0000 7fc85e59c700 1 -- 172.21.15.47:0/2393383994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc860143150 msgr2=0x7fc8600ad560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:56.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.999+0000 7fc85e59c700 1 --2- 172.21.15.47:0/2393383994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc860143150 0x7fc8600ad560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:56.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.999+0000 7fc85e59c700 1 -- 172.21.15.47:0/2393383994 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc85000b120 con 0x7fc860066fe0 2022-06-29T15:42:56.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.999+0000 7fc85dd9b700 1 --2- 172.21.15.47:0/2393383994 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc8600b64c0 0x7fc8601562b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:42:56.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.999+0000 7fc85e59c700 1 --2- 172.21.15.47:0/2393383994 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc860066fe0 0x7fc8600b2db0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7fc8600b4a20 tx=0x7fc848007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:56.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.999+0000 7fc8577fe700 1 -- 172.21.15.47:0/2393383994 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc8480070d0 con 0x7fc860066fe0 2022-06-29T15:42:56.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.999+0000 7fc8577fe700 1 -- 172.21.15.47:0/2393383994 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc848007230 con 0x7fc860066fe0 2022-06-29T15:42:56.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.999+0000 7fc85f59e700 1 -- 172.21.15.47:0/2393383994 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc860155e10 con 0x7fc860066fe0 2022-06-29T15:42:56.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.999+0000 7fc8577fe700 1 -- 172.21.15.47:0/2393383994 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc848005610 con 0x7fc860066fe0 2022-06-29T15:42:56.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.999+0000 7fc85f59e700 1 -- 172.21.15.47:0/2393383994 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc860145ed0 con 0x7fc860066fe0 2022-06-29T15:42:56.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.999+0000 7fc85f59e700 1 -- 172.21.15.47:0/2393383994 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc860155fa0 con 0x7fc860066fe0 2022-06-29T15:42:56.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:55.999+0000 7fc8577fe700 1 -- 172.21.15.47:0/2393383994 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc848010070 con 0x7fc860066fe0 2022-06-29T15:42:56.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.003+0000 7fc8577fe700 1 --2- 172.21.15.47:0/2393383994 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc844053a70 0x7fc844055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:56.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.003+0000 7fc85dd9b700 1 --2- 172.21.15.47:0/2393383994 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc844053a70 0x7fc844055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:56.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.003+0000 7fc8577fe700 1 -- 172.21.15.47:0/2393383994 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fc84806b550 con 0x7fc860066fe0 2022-06-29T15:42:56.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.003+0000 7fc85dd9b700 1 --2- 172.21.15.47:0/2393383994 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc844053a70 0x7fc844055f30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fc850018000 tx=0x7fc850013000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:56.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.011+0000 7fc8577fe700 1 -- 172.21.15.47:0/2393383994 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc848042310 con 0x7fc860066fe0 2022-06-29T15:42:56.144 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.135+0000 7fc85f59e700 1 -- 172.21.15.47:0/2393383994 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fc8600573d0 con 0x7fc860066fe0 2022-06-29T15:42:56.145 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.139+0000 7fc8577fe700 1 -- 172.21.15.47:0/2393383994 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v26) v1 ==== 110+0+10 (secure 0 0 0) 0x7fc848045930 con 0x7fc860066fe0 2022-06-29T15:42:56.145 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2022-06-29T15:42:56.148 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.139+0000 7fc85f59e700 1 -- 172.21.15.47:0/2393383994 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc844053a70 msgr2=0x7fc844055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:56.149 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.139+0000 7fc85f59e700 1 --2- 172.21.15.47:0/2393383994 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc844053a70 0x7fc844055f30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fc850018000 tx=0x7fc850013000).stop 2022-06-29T15:42:56.149 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.139+0000 7fc85f59e700 1 -- 172.21.15.47:0/2393383994 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc860066fe0 msgr2=0x7fc8600b2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:56.149 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.139+0000 7fc85f59e700 1 --2- 172.21.15.47:0/2393383994 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc860066fe0 0x7fc8600b2db0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7fc8600b4a20 tx=0x7fc848007c40).stop 2022-06-29T15:42:56.149 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.139+0000 7fc85f59e700 1 -- 172.21.15.47:0/2393383994 shutdown_connections 2022-06-29T15:42:56.150 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.139+0000 7fc85f59e700 1 --2- 172.21.15.47:0/2393383994 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc844053a70 0x7fc844055f30 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:56.150 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.139+0000 7fc85f59e700 1 --2- 172.21.15.47:0/2393383994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc860143150 0x7fc8600ad560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:56.150 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.139+0000 7fc85f59e700 1 --2- 172.21.15.47:0/2393383994 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc8600b64c0 0x7fc8601562b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:56.150 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.139+0000 7fc85f59e700 1 --2- 172.21.15.47:0/2393383994 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc860066fe0 0x7fc8600b2db0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:56.150 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.139+0000 7fc85f59e700 1 -- 172.21.15.47:0/2393383994 >> 172.21.15.47:0/2393383994 conn(0x7fc860060e00 msgr2=0x7fc860147860 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:56.151 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.139+0000 7fc85f59e700 1 -- 172.21.15.47:0/2393383994 shutdown_connections 2022-06-29T15:42:56.151 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.139+0000 7fc85f59e700 1 -- 172.21.15.47:0/2393383994 wait complete. 2022-06-29T15:42:56.162 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-29T15:42:56.266 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.259+0000 7f8ffd965700 1 Processor -- start 2022-06-29T15:42:56.266 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.259+0000 7f8ffd965700 1 -- start start 2022-06-29T15:42:56.267 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.259+0000 7f8ffd965700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8ff805f0b0 0x7f8ff814ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:56.268 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.259+0000 7f8ffd965700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ff80b20b0 con 0x7f8ff80619b0 2022-06-29T15:42:56.268 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.259+0000 7f8ffd965700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ff8061d90 con 0x7f8ff80aece0 2022-06-29T15:42:56.268 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.259+0000 7f8ffd965700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ff80b1520 con 0x7f8ff805f0b0 2022-06-29T15:42:56.268 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.259+0000 7f8ff6ffd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f8ff80619b0 0x7f8ff80a9ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:60372/0 (socket says 172.21.15.47:60372) 2022-06-29T15:42:56.269 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.259+0000 7f8ff6ffd700 1 -- 172.21.15.47:0/480034578 learned_addr learned my addr 172.21.15.47:0/480034578 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:56.269 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.259+0000 7f8ff67fc700 1 --2- 172.21.15.47:0/480034578 >> v2:172.21.15.64:3300/0 conn(0x7f8ff805f0b0 0x7f8ff814ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:56.269 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.259+0000 7f8ff5ffb700 1 -- 172.21.15.47:0/480034578 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3433927441 0 0) 0x7f8ff80b20b0 con 0x7f8ff80619b0 2022-06-29T15:42:56.269 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.259+0000 7f8ff5ffb700 1 -- 172.21.15.47:0/480034578 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fdc003260 con 0x7f8ff80619b0 2022-06-29T15:42:56.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.259+0000 7f8ff5ffb700 1 -- 172.21.15.47:0/480034578 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 748207227 0 0) 0x7f8fdc003260 con 0x7f8ff80619b0 2022-06-29T15:42:56.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.259+0000 7f8ff5ffb700 1 -- 172.21.15.47:0/480034578 >> v2:172.21.15.64:3300/0 conn(0x7f8ff805f0b0 msgr2=0x7f8ff814ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:56.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.259+0000 7f8ff5ffb700 1 --2- 172.21.15.47:0/480034578 >> v2:172.21.15.64:3300/0 conn(0x7f8ff805f0b0 0x7f8ff814ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:56.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.259+0000 7f8ff5ffb700 1 -- 172.21.15.47:0/480034578 >> v1:172.21.15.64:6789/0 conn(0x7f8ff80aece0 legacy=0x7f8ff814b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:56.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.259+0000 7f8ff5ffb700 1 -- 172.21.15.47:0/480034578 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff80aea90 con 0x7f8ff80619b0 2022-06-29T15:42:56.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.259+0000 7f8ff5ffb700 1 -- 172.21.15.47:0/480034578 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7f8fe8001660 con 0x7f8ff80619b0 2022-06-29T15:42:56.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.259+0000 7f8ff67fc700 1 --2- 172.21.15.47:0/480034578 >> v2:172.21.15.64:3300/0 conn(0x7f8ff805f0b0 0x7f8ff814ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:42:56.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.259+0000 7f8ff5ffb700 1 -- 172.21.15.47:0/480034578 >> v1:172.21.15.47:6789/0 conn(0x7f8ff80619b0 legacy=0x7f8ff80a9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:56.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.259+0000 7f8ff5ffb700 1 --2- 172.21.15.47:0/480034578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fdc003750 0x7f8fdc003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:56.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.259+0000 7f8ff5ffb700 1 --2- 172.21.15.47:0/480034578 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fdc004500 0x7f8fdc004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:56.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.259+0000 7f8ff5ffb700 1 --2- 172.21.15.47:0/480034578 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8fdc004e70 0x7f8fdc00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:56.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.263+0000 7f8ff5ffb700 1 -- 172.21.15.47:0/480034578 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fdc00bc40 con 0x7f8fdc003750 2022-06-29T15:42:56.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.263+0000 7f8ff5ffb700 1 -- 172.21.15.47:0/480034578 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fdc00bdb0 con 0x7f8fdc004500 2022-06-29T15:42:56.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.263+0000 7f8ff6ffd700 1 --2- 172.21.15.47:0/480034578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fdc003750 0x7f8fdc003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:56.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.263+0000 7f8ff5ffb700 1 -- 172.21.15.47:0/480034578 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fdc00bf70 con 0x7f8fdc004e70 2022-06-29T15:42:56.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.263+0000 7f8ff67fc700 1 --2- 172.21.15.47:0/480034578 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fdc004500 0x7f8fdc004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:56.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.263+0000 7f8ff77fe700 1 --2- 172.21.15.47:0/480034578 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8fdc004e70 0x7f8fdc00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:56.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.263+0000 7f8ff6ffd700 1 -- 172.21.15.47:0/480034578 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8fdc004e70 msgr2=0x7f8fdc00b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:56.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.263+0000 7f8ff6ffd700 1 --2- 172.21.15.47:0/480034578 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8fdc004e70 0x7f8fdc00b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:56.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.263+0000 7f8ff6ffd700 1 -- 172.21.15.47:0/480034578 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fdc004500 msgr2=0x7f8fdc004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:56.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.263+0000 7f8ff6ffd700 1 --2- 172.21.15.47:0/480034578 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fdc004500 0x7f8fdc004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:56.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.263+0000 7f8ff77fe700 1 -- 172.21.15.47:0/480034578 reap_dead start 2022-06-29T15:42:56.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.263+0000 7f8ff6ffd700 1 -- 172.21.15.47:0/480034578 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fdc003260 con 0x7f8fdc003750 2022-06-29T15:42:56.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.263+0000 7f8ff6ffd700 1 --2- 172.21.15.47:0/480034578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fdc003750 0x7f8fdc003b30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f8fe8001370 tx=0x7f8fe8001e30).ready entity=mon.0 client_cookie=5d913b1a5d3b0bbc server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:56.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.263+0000 7f8ff5ffb700 1 -- 172.21.15.47:0/480034578 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fe80071a0 con 0x7f8fdc003750 2022-06-29T15:42:56.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.263+0000 7f8ff5ffb700 1 -- 172.21.15.47:0/480034578 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe8007300 con 0x7f8fdc003750 2022-06-29T15:42:56.282 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.263+0000 7f8ff5ffb700 1 -- 172.21.15.47:0/480034578 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fe800c8f0 con 0x7f8fdc003750 2022-06-29T15:42:56.282 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.263+0000 7f8ffd965700 1 -- 172.21.15.47:0/480034578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fdc003750 msgr2=0x7f8fdc003b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:56.282 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.263+0000 7f8ffd965700 1 --2- 172.21.15.47:0/480034578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fdc003750 0x7f8fdc003b30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f8fe8001370 tx=0x7f8fe8001e30).stop 2022-06-29T15:42:56.282 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.263+0000 7f8ffd965700 1 -- 172.21.15.47:0/480034578 shutdown_connections 2022-06-29T15:42:56.283 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.263+0000 7f8ffd965700 1 --2- 172.21.15.47:0/480034578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fdc003750 0x7f8fdc003b30 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:56.283 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.263+0000 7f8ffd965700 1 -- 172.21.15.47:0/480034578 >> 172.21.15.47:0/480034578 conn(0x7f8ff8063800 msgr2=0x7f8ff8063be0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:56.283 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.263+0000 7f8ffd965700 1 -- 172.21.15.47:0/480034578 shutdown_connections 2022-06-29T15:42:56.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.263+0000 7f8ffd965700 1 -- 172.21.15.47:0/480034578 wait complete. 2022-06-29T15:42:56.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.263+0000 7f8ffd965700 1 Processor -- start 2022-06-29T15:42:56.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.263+0000 7f8ffd965700 1 -- start start 2022-06-29T15:42:56.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.267+0000 7f8ffd965700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8fdc003750 0x7f8ff8149580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:56.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.267+0000 7f8ffd965700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ff8064a20 0x7f8ff80ac460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:56.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.267+0000 7f8ffd965700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ff80ac950 0x7f8ff81475c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:56.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.267+0000 7f8ffd965700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff8066e80 con 0x7f8ff80ac950 2022-06-29T15:42:56.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.267+0000 7f8ffd965700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff80ab820 con 0x7f8ff8064a20 2022-06-29T15:42:56.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.267+0000 7f8ffd965700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff80abac0 con 0x7f8fdc003750 2022-06-29T15:42:56.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.267+0000 7f8ff77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ff80ac950 0x7f8ff81475c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:56.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.267+0000 7f8ff77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ff80ac950 0x7f8ff81475c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46602/0 (socket says 172.21.15.47:46602) 2022-06-29T15:42:56.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.267+0000 7f8ff77fe700 1 -- 172.21.15.47:0/2997827696 learned_addr learned my addr 172.21.15.47:0/2997827696 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:56.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.267+0000 7f8ff6ffd700 1 --2- 172.21.15.47:0/2997827696 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8fdc003750 0x7f8ff8149580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:56.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.267+0000 7f8ff67fc700 1 --2- 172.21.15.47:0/2997827696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ff8064a20 0x7f8ff80ac460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:56.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.267+0000 7f8ff77fe700 1 -- 172.21.15.47:0/2997827696 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8fdc003750 msgr2=0x7f8ff8149580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:56.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.267+0000 7f8ff77fe700 1 --2- 172.21.15.47:0/2997827696 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8fdc003750 0x7f8ff8149580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:56.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.267+0000 7f8ff77fe700 1 -- 172.21.15.47:0/2997827696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ff8064a20 msgr2=0x7f8ff80ac460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:56.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.267+0000 7f8ff77fe700 1 --2- 172.21.15.47:0/2997827696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ff8064a20 0x7f8ff80ac460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:56.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.267+0000 7f8ff77fe700 1 -- 172.21.15.47:0/2997827696 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff80abd60 con 0x7f8ff80ac950 2022-06-29T15:42:56.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.267+0000 7f8ff6ffd700 1 --2- 172.21.15.47:0/2997827696 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8fdc003750 0x7f8ff8149580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:42:56.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.267+0000 7f8ff77fe700 1 --2- 172.21.15.47:0/2997827696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ff80ac950 0x7f8ff81475c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f8ff0001210 tx=0x7f8ff0009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:56.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.267+0000 7f8fe7fff700 1 -- 172.21.15.47:0/2997827696 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ff0003220 con 0x7f8ff80ac950 2022-06-29T15:42:56.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.267+0000 7f8fe7fff700 1 -- 172.21.15.47:0/2997827696 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ff0003380 con 0x7f8ff80ac950 2022-06-29T15:42:56.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.267+0000 7f8ffd965700 1 -- 172.21.15.47:0/2997827696 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ff8064580 con 0x7f8ff80ac950 2022-06-29T15:42:56.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.267+0000 7f8fe7fff700 1 -- 172.21.15.47:0/2997827696 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ff0001b10 con 0x7f8ff80ac950 2022-06-29T15:42:56.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.267+0000 7f8ffd965700 1 -- 172.21.15.47:0/2997827696 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ff80ad2c0 con 0x7f8ff80ac950 2022-06-29T15:42:56.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.267+0000 7f8ffd965700 1 -- 172.21.15.47:0/2997827696 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ff8067390 con 0x7f8ff80ac950 2022-06-29T15:42:56.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.267+0000 7f8fe7fff700 1 -- 172.21.15.47:0/2997827696 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8ff0001c70 con 0x7f8ff80ac950 2022-06-29T15:42:56.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.271+0000 7f8fe7fff700 1 --2- 172.21.15.47:0/2997827696 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8fd4053b30 0x7f8fd4055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:56.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.271+0000 7f8fe7fff700 1 -- 172.21.15.47:0/2997827696 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f8ff006ba20 con 0x7f8ff80ac950 2022-06-29T15:42:56.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.271+0000 7f8ff6ffd700 1 --2- 172.21.15.47:0/2997827696 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8fd4053b30 0x7f8fd4055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:56.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.271+0000 7f8ff6ffd700 1 --2- 172.21.15.47:0/2997827696 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8fd4053b30 0x7f8fd4055ff0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f8fe8018000 tx=0x7f8fe8015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:56.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.275+0000 7f8fe7fff700 1 -- 172.21.15.47:0/2997827696 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8ff8067390 con 0x7f8ff80ac950 2022-06-29T15:42:56.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.403+0000 7f8ffd965700 1 -- 172.21.15.47:0/2997827696 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8ff8067390 con 0x7f8ff80ac950 2022-06-29T15:42:56.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.407+0000 7f8fe7fff700 1 -- 172.21.15.47:0/2997827696 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8ff8067390 con 0x7f8ff80ac950 2022-06-29T15:42:56.415 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:42:56.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.411+0000 7f8ffd965700 1 -- 172.21.15.47:0/2997827696 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8fd4053b30 msgr2=0x7f8fd4055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:56.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.411+0000 7f8ffd965700 1 --2- 172.21.15.47:0/2997827696 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8fd4053b30 0x7f8fd4055ff0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f8fe8018000 tx=0x7f8fe8015000).stop 2022-06-29T15:42:56.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.411+0000 7f8ffd965700 1 -- 172.21.15.47:0/2997827696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ff80ac950 msgr2=0x7f8ff81475c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:56.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.411+0000 7f8ffd965700 1 --2- 172.21.15.47:0/2997827696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ff80ac950 0x7f8ff81475c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f8ff0001210 tx=0x7f8ff0009040).stop 2022-06-29T15:42:56.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.411+0000 7f8ffd965700 1 -- 172.21.15.47:0/2997827696 shutdown_connections 2022-06-29T15:42:56.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.411+0000 7f8ffd965700 1 --2- 172.21.15.47:0/2997827696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ff80ac950 0x7f8ff81475c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:56.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.411+0000 7f8ffd965700 1 --2- 172.21.15.47:0/2997827696 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8fd4053b30 0x7f8fd4055ff0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:56.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.411+0000 7f8ffd965700 1 --2- 172.21.15.47:0/2997827696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ff8064a20 0x7f8ff80ac460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:56.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.411+0000 7f8ffd965700 1 --2- 172.21.15.47:0/2997827696 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8fdc003750 0x7f8ff8149580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:56.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.411+0000 7f8ffd965700 1 -- 172.21.15.47:0/2997827696 >> 172.21.15.47:0/2997827696 conn(0x7f8ff8063800 msgr2=0x7f8ff80b5060 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:56.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.411+0000 7f8ffd965700 1 -- 172.21.15.47:0/2997827696 shutdown_connections 2022-06-29T15:42:56.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.411+0000 7f8ffd965700 1 -- 172.21.15.47:0/2997827696 wait complete. 2022-06-29T15:42:56.432 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-29T15:42:56.537 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.531+0000 7f0e86e3c700 1 Processor -- start 2022-06-29T15:42:56.538 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.531+0000 7f0e86e3c700 1 -- start start 2022-06-29T15:42:56.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.531+0000 7f0e86e3c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0e800b4300 0x7f0e800612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:56.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.531+0000 7f0e86e3c700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e80067610 con 0x7f0e801471a0 2022-06-29T15:42:56.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.531+0000 7f0e86e3c700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e801467c0 con 0x7f0e80060ee0 2022-06-29T15:42:56.541 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.531+0000 7f0e86e3c700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e80146a60 con 0x7f0e800b4300 2022-06-29T15:42:56.541 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.531+0000 7f0e7ffff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0e800b4300 0x7f0e800612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:56.541 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.531+0000 7f0e853d9700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0e801471a0 0x7f0e8013cdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:60396/0 (socket says 172.21.15.47:60396) 2022-06-29T15:42:56.541 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.531+0000 7f0e853d9700 1 -- 172.21.15.47:0/1225489802 learned_addr learned my addr 172.21.15.47:0/1225489802 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:56.542 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.531+0000 7f0e7ffff700 1 -- 172.21.15.47:0/1225489802 >> v1:172.21.15.64:6789/0 conn(0x7f0e80060ee0 legacy=0x7f0e801430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:56.542 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.531+0000 7f0e7ffff700 1 -- 172.21.15.47:0/1225489802 >> v1:172.21.15.47:6789/0 conn(0x7f0e801471a0 legacy=0x7f0e8013cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:56.542 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.531+0000 7f0e7f7fe700 1 -- 172.21.15.47:0/1225489802 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2732422659 0 0) 0x7f0e801467c0 con 0x7f0e80060ee0 2022-06-29T15:42:56.542 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.531+0000 7f0e7ffff700 1 -- 172.21.15.47:0/1225489802 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e80146d00 con 0x7f0e800b4300 2022-06-29T15:42:56.543 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.531+0000 7f0e7ffff700 1 --2- 172.21.15.47:0/1225489802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e800b4300 0x7f0e800612c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f0e800adb80 tx=0x7f0e74006df0).ready entity=mon.0 client_cookie=573bd80656d92575 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:56.543 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.531+0000 7f0e7f7fe700 1 -- 172.21.15.47:0/1225489802 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e7400d5c0 con 0x7f0e800b4300 2022-06-29T15:42:56.543 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.531+0000 7f0e7f7fe700 1 -- 172.21.15.47:0/1225489802 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e74000e90 con 0x7f0e800b4300 2022-06-29T15:42:56.543 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.531+0000 7f0e7f7fe700 1 -- 172.21.15.47:0/1225489802 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e7400d910 con 0x7f0e800b4300 2022-06-29T15:42:56.545 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.535+0000 7f0e86e3c700 1 -- 172.21.15.47:0/1225489802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e800b4300 msgr2=0x7f0e800612c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:56.545 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.535+0000 7f0e86e3c700 1 --2- 172.21.15.47:0/1225489802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e800b4300 0x7f0e800612c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f0e800adb80 tx=0x7f0e74006df0).stop 2022-06-29T15:42:56.545 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.535+0000 7f0e86e3c700 1 -- 172.21.15.47:0/1225489802 shutdown_connections 2022-06-29T15:42:56.545 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.535+0000 7f0e86e3c700 1 --2- 172.21.15.47:0/1225489802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e800b4300 0x7f0e800612c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:56.546 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.535+0000 7f0e86e3c700 1 -- 172.21.15.47:0/1225489802 >> 172.21.15.47:0/1225489802 conn(0x7f0e8005c510 msgr2=0x7f0e8005f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:56.546 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.535+0000 7f0e86e3c700 1 -- 172.21.15.47:0/1225489802 shutdown_connections 2022-06-29T15:42:56.546 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.535+0000 7f0e86e3c700 1 -- 172.21.15.47:0/1225489802 wait complete. 2022-06-29T15:42:56.547 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.535+0000 7f0e86e3c700 1 Processor -- start 2022-06-29T15:42:56.547 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.535+0000 7f0e86e3c700 1 -- start start 2022-06-29T15:42:56.551 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.539+0000 7f0e86e3c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e80060ee0 0x7f0e80156280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:56.551 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.539+0000 7f0e86e3c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e800b4300 0x7f0e80142440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:56.551 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.539+0000 7f0e86e3c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0e801471a0 0x7f0e800b37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:56.551 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.539+0000 7f0e86e3c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e80143090 con 0x7f0e800b4300 2022-06-29T15:42:56.552 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.539+0000 7f0e86e3c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e800b3ca0 con 0x7f0e80060ee0 2022-06-29T15:42:56.552 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.539+0000 7f0e86e3c700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e800b2180 con 0x7f0e801471a0 2022-06-29T15:42:56.552 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.539+0000 7f0e7ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e800b4300 0x7f0e80142440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:56.552 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.539+0000 7f0e84bd8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e80060ee0 0x7f0e80156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:56.552 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.539+0000 7f0e7ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e800b4300 0x7f0e80142440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46614/0 (socket says 172.21.15.47:46614) 2022-06-29T15:42:56.553 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.539+0000 7f0e7ffff700 1 -- 172.21.15.47:0/3347831635 learned_addr learned my addr 172.21.15.47:0/3347831635 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:56.553 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.539+0000 7f0e84bd8700 1 --2- 172.21.15.47:0/3347831635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e80060ee0 0x7f0e80156280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:52316/0 (socket says 172.21.15.47:52316) 2022-06-29T15:42:56.553 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.539+0000 7f0e853d9700 1 --2- 172.21.15.47:0/3347831635 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0e801471a0 0x7f0e800b37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:56.554 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.539+0000 7f0e7ffff700 1 -- 172.21.15.47:0/3347831635 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0e801471a0 msgr2=0x7f0e800b37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:56.554 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.539+0000 7f0e7ffff700 1 --2- 172.21.15.47:0/3347831635 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0e801471a0 0x7f0e800b37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:56.554 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.539+0000 7f0e7ffff700 1 -- 172.21.15.47:0/3347831635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e80060ee0 msgr2=0x7f0e80156280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:56.554 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.539+0000 7f0e7ffff700 1 --2- 172.21.15.47:0/3347831635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e80060ee0 0x7f0e80156280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:56.554 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.539+0000 7f0e7ffff700 1 -- 172.21.15.47:0/3347831635 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e7400b180 con 0x7f0e800b4300 2022-06-29T15:42:56.555 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.539+0000 7f0e7ffff700 1 --2- 172.21.15.47:0/3347831635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e800b4300 0x7f0e80142440 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f0e800adb80 tx=0x7f0e74014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:56.555 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.539+0000 7f0e7dffb700 1 -- 172.21.15.47:0/3347831635 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e74016070 con 0x7f0e800b4300 2022-06-29T15:42:56.555 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.539+0000 7f0e7dffb700 1 -- 172.21.15.47:0/3347831635 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e74013b70 con 0x7f0e800b4300 2022-06-29T15:42:56.555 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.539+0000 7f0e86e3c700 1 -- 172.21.15.47:0/3347831635 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e800b2420 con 0x7f0e800b4300 2022-06-29T15:42:56.556 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.539+0000 7f0e7dffb700 1 -- 172.21.15.47:0/3347831635 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e7401e450 con 0x7f0e800b4300 2022-06-29T15:42:56.556 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.539+0000 7f0e86e3c700 1 -- 172.21.15.47:0/3347831635 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e800b2b20 con 0x7f0e800b4300 2022-06-29T15:42:56.556 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.543+0000 7f0e7dffb700 1 -- 172.21.15.47:0/3347831635 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0e74025030 con 0x7f0e800b4300 2022-06-29T15:42:56.556 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.543+0000 7f0e86e3c700 1 -- 172.21.15.47:0/3347831635 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e800573d0 con 0x7f0e800b4300 2022-06-29T15:42:56.557 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.543+0000 7f0e7dffb700 1 --2- 172.21.15.47:0/3347831635 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0e60053b00 0x7f0e60055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:56.557 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.543+0000 7f0e84bd8700 1 --2- 172.21.15.47:0/3347831635 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0e60053b00 0x7f0e60055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:56.557 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.543+0000 7f0e7dffb700 1 -- 172.21.15.47:0/3347831635 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f0e7406d0d0 con 0x7f0e800b4300 2022-06-29T15:42:56.557 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.543+0000 7f0e84bd8700 1 --2- 172.21.15.47:0/3347831635 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0e60053b00 0x7f0e60055fc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f0e7000d000 tx=0x7f0e70007a60).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:56.558 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.551+0000 7f0e7dffb700 1 -- 172.21.15.47:0/3347831635 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0e74043d80 con 0x7f0e800b4300 2022-06-29T15:42:56.679 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.671+0000 7f0e86e3c700 1 -- 172.21.15.47:0/3347831635 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0e800b5e50 con 0x7f0e800b4300 2022-06-29T15:42:56.680 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.671+0000 7f0e7dffb700 1 -- 172.21.15.47:0/3347831635 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+10 (secure 0 0 0) 0x7f0e740230a0 con 0x7f0e800b4300 2022-06-29T15:42:56.681 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:42:56.683 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.675+0000 7f0e86e3c700 1 -- 172.21.15.47:0/3347831635 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0e60053b00 msgr2=0x7f0e60055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:56.684 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.675+0000 7f0e86e3c700 1 --2- 172.21.15.47:0/3347831635 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0e60053b00 0x7f0e60055fc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f0e7000d000 tx=0x7f0e70007a60).stop 2022-06-29T15:42:56.684 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.675+0000 7f0e86e3c700 1 -- 172.21.15.47:0/3347831635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e800b4300 msgr2=0x7f0e80142440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:56.684 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.675+0000 7f0e86e3c700 1 --2- 172.21.15.47:0/3347831635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e800b4300 0x7f0e80142440 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f0e800adb80 tx=0x7f0e74014000).stop 2022-06-29T15:42:56.685 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.675+0000 7f0e86e3c700 1 -- 172.21.15.47:0/3347831635 shutdown_connections 2022-06-29T15:42:56.685 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.675+0000 7f0e86e3c700 1 --2- 172.21.15.47:0/3347831635 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0e801471a0 0x7f0e800b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:56.685 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.675+0000 7f0e86e3c700 1 --2- 172.21.15.47:0/3347831635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e800b4300 0x7f0e80142440 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:56.685 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.675+0000 7f0e86e3c700 1 --2- 172.21.15.47:0/3347831635 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0e60053b00 0x7f0e60055fc0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:56.686 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.675+0000 7f0e86e3c700 1 --2- 172.21.15.47:0/3347831635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e80060ee0 0x7f0e80156280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:56.686 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.675+0000 7f0e86e3c700 1 -- 172.21.15.47:0/3347831635 >> 172.21.15.47:0/3347831635 conn(0x7f0e8005c510 msgr2=0x7f0e8013c750 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:56.686 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.675+0000 7f0e86e3c700 1 -- 172.21.15.47:0/3347831635 shutdown_connections 2022-06-29T15:42:56.686 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.675+0000 7f0e86e3c700 1 -- 172.21.15.47:0/3347831635 wait complete. 2022-06-29T15:42:56.693 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-29T15:42:56.805 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.795+0000 7f9789816700 1 Processor -- start 2022-06-29T15:42:56.805 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.799+0000 7f9789816700 1 -- start start 2022-06-29T15:42:56.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.799+0000 7f9789816700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f97840b5d90 0x7f9784143a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:56.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.799+0000 7f9789816700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f978405cc30 con 0x7f97840676a0 2022-06-29T15:42:56.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.799+0000 7f9789816700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97841430a0 con 0x7f978405c850 2022-06-29T15:42:56.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.799+0000 7f9789816700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9784143300 con 0x7f97840b5d90 2022-06-29T15:42:56.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.799+0000 7f9789015700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f97840676a0 0x7f9784140130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:60410/0 (socket says 172.21.15.47:60410) 2022-06-29T15:42:56.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.799+0000 7f9789015700 1 -- 172.21.15.47:0/3199490911 learned_addr learned my addr 172.21.15.47:0/3199490911 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:56.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.799+0000 7f9783fff700 1 --2- 172.21.15.47:0/3199490911 >> v2:172.21.15.64:3300/0 conn(0x7f97840b5d90 0x7f9784143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:56.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.799+0000 7f97837fe700 1 -- 172.21.15.47:0/3199490911 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1802427577 0 0) 0x7f978405cc30 con 0x7f97840676a0 2022-06-29T15:42:56.809 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.799+0000 7f97837fe700 1 -- 172.21.15.47:0/3199490911 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9770003260 con 0x7f97840676a0 2022-06-29T15:42:56.810 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.799+0000 7f97837fe700 1 -- 172.21.15.47:0/3199490911 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 883100674 0 0) 0x7f9770003260 con 0x7f97840676a0 2022-06-29T15:42:56.811 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.799+0000 7f9783fff700 1 -- 172.21.15.47:0/3199490911 >> v1:172.21.15.64:6789/0 conn(0x7f978405c850 legacy=0x7f97840b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:56.811 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.799+0000 7f9783fff700 1 -- 172.21.15.47:0/3199490911 >> v1:172.21.15.47:6789/0 conn(0x7f97840676a0 legacy=0x7f9784140130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:56.811 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.799+0000 7f9783fff700 1 -- 172.21.15.47:0/3199490911 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97841435a0 con 0x7f97840b5d90 2022-06-29T15:42:56.811 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.799+0000 7f9783fff700 1 --2- 172.21.15.47:0/3199490911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97840b5d90 0x7f9784143a40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f977c001ba0 tx=0x7f9778006df0).ready entity=mon.1 client_cookie=5dbfadfd2f726a8a server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:56.811 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.799+0000 7f97837fe700 1 -- 172.21.15.47:0/3199490911 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f977c001a00 con 0x7f97840b5d90 2022-06-29T15:42:56.812 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.799+0000 7f97837fe700 1 -- 172.21.15.47:0/3199490911 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9778000e90 con 0x7f97840b5d90 2022-06-29T15:42:56.812 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.799+0000 7f97837fe700 1 -- 172.21.15.47:0/3199490911 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9778011400 con 0x7f97840b5d90 2022-06-29T15:42:56.812 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.803+0000 7f9789816700 1 -- 172.21.15.47:0/3199490911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97840b5d90 msgr2=0x7f9784143a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:56.812 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.803+0000 7f9789816700 1 --2- 172.21.15.47:0/3199490911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97840b5d90 0x7f9784143a40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f977c001ba0 tx=0x7f9778006df0).stop 2022-06-29T15:42:56.813 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.803+0000 7f9789816700 1 -- 172.21.15.47:0/3199490911 shutdown_connections 2022-06-29T15:42:56.813 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.803+0000 7f9789816700 1 --2- 172.21.15.47:0/3199490911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97840b5d90 0x7f9784143a40 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:56.813 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.803+0000 7f9789816700 1 -- 172.21.15.47:0/3199490911 >> 172.21.15.47:0/3199490911 conn(0x7f9784060e00 msgr2=0x7f97840611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:56.813 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.803+0000 7f9789816700 1 -- 172.21.15.47:0/3199490911 shutdown_connections 2022-06-29T15:42:56.813 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.803+0000 7f9789816700 1 -- 172.21.15.47:0/3199490911 wait complete. 2022-06-29T15:42:56.816 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.803+0000 7f9789816700 1 Processor -- start 2022-06-29T15:42:56.816 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.803+0000 7f9789816700 1 -- start start 2022-06-29T15:42:56.817 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.807+0000 7f9789816700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f978405c850 0x7f97840b3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:56.817 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.807+0000 7f9789816700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97840676a0 0x7f97840b9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:56.817 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.807+0000 7f9789816700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f97840b5d90 0x7f97840ba0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:56.817 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.807+0000 7f9789816700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97841419b0 con 0x7f978405c850 2022-06-29T15:42:56.818 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.807+0000 7f9789816700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97840ba5a0 con 0x7f97840676a0 2022-06-29T15:42:56.818 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.807+0000 7f9789816700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97840b7bf0 con 0x7f97840b5d90 2022-06-29T15:42:56.818 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.807+0000 7f9788814700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f978405c850 0x7f97840b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:56.818 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.807+0000 7f9788814700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f978405c850 0x7f97840b3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46626/0 (socket says 172.21.15.47:46626) 2022-06-29T15:42:56.818 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.807+0000 7f9788814700 1 -- 172.21.15.47:0/2432247386 learned_addr learned my addr 172.21.15.47:0/2432247386 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:56.819 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.807+0000 7f9783fff700 1 --2- 172.21.15.47:0/2432247386 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97840676a0 0x7f97840b9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:56.819 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.807+0000 7f9789015700 1 --2- 172.21.15.47:0/2432247386 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f97840b5d90 0x7f97840ba0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:56.819 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.807+0000 7f9788814700 1 -- 172.21.15.47:0/2432247386 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f97840b5d90 msgr2=0x7f97840ba0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:56.819 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.807+0000 7f9788814700 1 --2- 172.21.15.47:0/2432247386 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f97840b5d90 0x7f97840ba0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:56.820 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.807+0000 7f9788814700 1 -- 172.21.15.47:0/2432247386 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97840676a0 msgr2=0x7f97840b9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:56.820 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.807+0000 7f9788814700 1 --2- 172.21.15.47:0/2432247386 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97840676a0 0x7f97840b9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:56.820 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.807+0000 7f9788814700 1 -- 172.21.15.47:0/2432247386 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f977800b180 con 0x7f978405c850 2022-06-29T15:42:56.820 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.807+0000 7f9788814700 1 --2- 172.21.15.47:0/2432247386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f978405c850 0x7f97840b3610 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f97740011a0 tx=0x7f9774007ba0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:56.820 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.807+0000 7f9781ffb700 1 -- 172.21.15.47:0/2432247386 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f977400cea0 con 0x7f978405c850 2022-06-29T15:42:56.821 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.807+0000 7f9781ffb700 1 -- 172.21.15.47:0/2432247386 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f977400f470 con 0x7f978405c850 2022-06-29T15:42:56.821 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.807+0000 7f9789816700 1 -- 172.21.15.47:0/2432247386 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9784141be0 con 0x7f978405c850 2022-06-29T15:42:56.821 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.807+0000 7f9781ffb700 1 -- 172.21.15.47:0/2432247386 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f977400f810 con 0x7f978405c850 2022-06-29T15:42:56.821 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.807+0000 7f9789816700 1 -- 172.21.15.47:0/2432247386 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97840b7870 con 0x7f978405c850 2022-06-29T15:42:56.822 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.807+0000 7f9789816700 1 -- 172.21.15.47:0/2432247386 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9784142080 con 0x7f978405c850 2022-06-29T15:42:56.822 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.807+0000 7f9781ffb700 1 -- 172.21.15.47:0/2432247386 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9774019030 con 0x7f978405c850 2022-06-29T15:42:56.823 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.811+0000 7f9781ffb700 1 --2- 172.21.15.47:0/2432247386 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9764053b00 0x7f9764055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:56.823 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.811+0000 7f9781ffb700 1 -- 172.21.15.47:0/2432247386 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f977406b410 con 0x7f978405c850 2022-06-29T15:42:56.823 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.811+0000 7f9783fff700 1 --2- 172.21.15.47:0/2432247386 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9764053b00 0x7f9764055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:56.823 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.811+0000 7f9783fff700 1 --2- 172.21.15.47:0/2432247386 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9764053b00 0x7f9764055fc0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f9778019000 tx=0x7f97840adf20).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:56.826 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.819+0000 7f9781ffb700 1 -- 172.21.15.47:0/2432247386 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9784142080 con 0x7f978405c850 2022-06-29T15:42:56.951 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.943+0000 7f9789816700 1 -- 172.21.15.47:0/2432247386 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9784142080 con 0x7f978405c850 2022-06-29T15:42:56.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.943+0000 7f9781ffb700 1 -- 172.21.15.47:0/2432247386 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+10 (secure 0 0 0) 0x7f9784142080 con 0x7f978405c850 2022-06-29T15:42:56.953 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:42:56.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.947+0000 7f9789816700 1 -- 172.21.15.47:0/2432247386 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9764053b00 msgr2=0x7f9764055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:56.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.947+0000 7f9789816700 1 --2- 172.21.15.47:0/2432247386 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9764053b00 0x7f9764055fc0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f9778019000 tx=0x7f97840adf20).stop 2022-06-29T15:42:56.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.947+0000 7f9789816700 1 -- 172.21.15.47:0/2432247386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f978405c850 msgr2=0x7f97840b3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:56.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.947+0000 7f9789816700 1 --2- 172.21.15.47:0/2432247386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f978405c850 0x7f97840b3610 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f97740011a0 tx=0x7f9774007ba0).stop 2022-06-29T15:42:56.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.947+0000 7f9789816700 1 -- 172.21.15.47:0/2432247386 shutdown_connections 2022-06-29T15:42:56.958 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.947+0000 7f9789816700 1 --2- 172.21.15.47:0/2432247386 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f97840b5d90 0x7f97840ba0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:56.958 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.947+0000 7f9789816700 1 --2- 172.21.15.47:0/2432247386 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9764053b00 0x7f9764055fc0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:56.958 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.947+0000 7f9789816700 1 --2- 172.21.15.47:0/2432247386 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97840676a0 0x7f97840b9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:56.958 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.947+0000 7f9789816700 1 --2- 172.21.15.47:0/2432247386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f978405c850 0x7f97840b3610 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:56.959 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.947+0000 7f9789816700 1 -- 172.21.15.47:0/2432247386 >> 172.21.15.47:0/2432247386 conn(0x7f9784060e00 msgr2=0x7f9784144a30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:56.959 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.947+0000 7f9789816700 1 -- 172.21.15.47:0/2432247386 shutdown_connections 2022-06-29T15:42:56.959 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:56.947+0000 7f9789816700 1 -- 172.21.15.47:0/2432247386 wait complete. 2022-06-29T15:42:56.971 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:42:57.076 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.067+0000 7f66dc8ac700 1 Processor -- start 2022-06-29T15:42:57.076 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.067+0000 7f66dc8ac700 1 -- start start 2022-06-29T15:42:57.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.067+0000 7f66dc8ac700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f66d80b5c60 0x7f66d8144980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:57.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.067+0000 7f66dc8ac700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66d80a9120 con 0x7f66d80a72f0 2022-06-29T15:42:57.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.067+0000 7f66dc8ac700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66d80b0b80 con 0x7f66d8060eb0 2022-06-29T15:42:57.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.067+0000 7f66dc8ac700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66d80a89e0 con 0x7f66d80b5c60 2022-06-29T15:42:57.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.071+0000 7f66d7fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f66d80a72f0 0x7f66d8151eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:60422/0 (socket says 172.21.15.47:60422) 2022-06-29T15:42:57.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.071+0000 7f66d7fff700 1 -- 172.21.15.47:0/3015393725 learned_addr learned my addr 172.21.15.47:0/3015393725 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:57.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.071+0000 7f66d77fe700 1 --2- 172.21.15.47:0/3015393725 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f66d80b5c60 0x7f66d8144980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:57.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.071+0000 7f66d67fc700 1 -- 172.21.15.47:0/3015393725 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3415210139 0 0) 0x7f66d80a9120 con 0x7f66d80a72f0 2022-06-29T15:42:57.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.071+0000 7f66d67fc700 1 -- 172.21.15.47:0/3015393725 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66c4003260 con 0x7f66d80a72f0 2022-06-29T15:42:57.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.071+0000 7f66d67fc700 1 -- 172.21.15.47:0/3015393725 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2497036828 0 0) 0x7f66d80b0b80 con 0x7f66d8060eb0 2022-06-29T15:42:57.089 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.071+0000 7f66d67fc700 1 -- 172.21.15.47:0/3015393725 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66d80a9120 con 0x7f66d8060eb0 2022-06-29T15:42:57.089 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.071+0000 7f66d77fe700 1 -- 172.21.15.47:0/3015393725 >> v1:172.21.15.64:6789/0 conn(0x7f66d8060eb0 legacy=0x7f66d80b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:57.090 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.071+0000 7f66d77fe700 1 -- 172.21.15.47:0/3015393725 >> v1:172.21.15.47:6789/0 conn(0x7f66d80a72f0 legacy=0x7f66d8151eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:57.090 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.071+0000 7f66d77fe700 1 -- 172.21.15.47:0/3015393725 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66d80a8c80 con 0x7f66d80b5c60 2022-06-29T15:42:57.090 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.071+0000 7f66d77fe700 1 --2- 172.21.15.47:0/3015393725 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f66d80b5c60 0x7f66d8144980 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f66c8002f70 tx=0x7f66c8006d90).ready entity=mon.2 client_cookie=e76264c1bc03e3e9 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:57.090 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.071+0000 7f66d67fc700 1 -- 172.21.15.47:0/3015393725 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66c800d4a0 con 0x7f66d80b5c60 2022-06-29T15:42:57.091 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.071+0000 7f66d67fc700 1 -- 172.21.15.47:0/3015393725 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66c8000e90 con 0x7f66d80b5c60 2022-06-29T15:42:57.091 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.071+0000 7f66d67fc700 1 -- 172.21.15.47:0/3015393725 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66c800d7f0 con 0x7f66d80b5c60 2022-06-29T15:42:57.091 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.071+0000 7f66dc8ac700 1 -- 172.21.15.47:0/3015393725 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f66d80b5c60 msgr2=0x7f66d8144980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:57.091 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.071+0000 7f66dc8ac700 1 --2- 172.21.15.47:0/3015393725 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f66d80b5c60 0x7f66d8144980 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f66c8002f70 tx=0x7f66c8006d90).stop 2022-06-29T15:42:57.092 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.071+0000 7f66dc8ac700 1 -- 172.21.15.47:0/3015393725 shutdown_connections 2022-06-29T15:42:57.092 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.071+0000 7f66dc8ac700 1 --2- 172.21.15.47:0/3015393725 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f66d80b5c60 0x7f66d8144980 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:57.092 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.071+0000 7f66dc8ac700 1 -- 172.21.15.47:0/3015393725 >> 172.21.15.47:0/3015393725 conn(0x7f66d8062560 msgr2=0x7f66d8062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:57.092 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.071+0000 7f66dc8ac700 1 -- 172.21.15.47:0/3015393725 shutdown_connections 2022-06-29T15:42:57.093 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.071+0000 7f66dc8ac700 1 -- 172.21.15.47:0/3015393725 wait complete. 2022-06-29T15:42:57.093 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.071+0000 7f66dc8ac700 1 Processor -- start 2022-06-29T15:42:57.093 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.071+0000 7f66dc8ac700 1 -- start start 2022-06-29T15:42:57.093 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.075+0000 7f66dc8ac700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66d8060eb0 0x7f66d80ae040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:57.094 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.075+0000 7f66dc8ac700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f66d80a72f0 0x7f66d814c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:57.094 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.075+0000 7f66dc8ac700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66d80b5c60 0x7f66d814a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:57.094 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.075+0000 7f66dc8ac700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d8069750 con 0x7f66d80b5c60 2022-06-29T15:42:57.094 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.075+0000 7f66dc8ac700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d814b910 con 0x7f66d8060eb0 2022-06-29T15:42:57.095 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.075+0000 7f66dc8ac700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66d814d350 con 0x7f66d80a72f0 2022-06-29T15:42:57.095 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.075+0000 7f66d7fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66d80b5c60 0x7f66d814a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:57.095 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.075+0000 7f66d77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66d8060eb0 0x7f66d80ae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:57.095 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.075+0000 7f66d7fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66d80b5c60 0x7f66d814a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46644/0 (socket says 172.21.15.47:46644) 2022-06-29T15:42:57.096 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.075+0000 7f66d6ffd700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f66d80a72f0 0x7f66d814c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:57.096 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.075+0000 7f66d7fff700 1 -- 172.21.15.47:0/2404105761 learned_addr learned my addr 172.21.15.47:0/2404105761 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:57.096 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.075+0000 7f66d6ffd700 1 -- 172.21.15.47:0/2404105761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66d8060eb0 msgr2=0x7f66d80ae040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:57.096 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.075+0000 7f66d6ffd700 1 --2- 172.21.15.47:0/2404105761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66d8060eb0 0x7f66d80ae040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:57.097 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.075+0000 7f66d6ffd700 1 -- 172.21.15.47:0/2404105761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66d80b5c60 msgr2=0x7f66d814a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:57.097 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.075+0000 7f66d6ffd700 1 --2- 172.21.15.47:0/2404105761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66d80b5c60 0x7f66d814a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:57.097 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.075+0000 7f66d6ffd700 1 -- 172.21.15.47:0/2404105761 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c800b120 con 0x7f66d80a72f0 2022-06-29T15:42:57.097 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.075+0000 7f66d6ffd700 1 --2- 172.21.15.47:0/2404105761 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f66d80a72f0 0x7f66d814c020 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f66d814a190 tx=0x7f66cc007a20).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:57.098 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.075+0000 7f66d4ff9700 1 -- 172.21.15.47:0/2404105761 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66cc012400 con 0x7f66d80a72f0 2022-06-29T15:42:57.098 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.075+0000 7f66d4ff9700 1 -- 172.21.15.47:0/2404105761 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66cc012560 con 0x7f66d80a72f0 2022-06-29T15:42:57.098 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.075+0000 7f66dc8ac700 1 -- 172.21.15.47:0/2404105761 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66d814ccb0 con 0x7f66d80a72f0 2022-06-29T15:42:57.098 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.075+0000 7f66d4ff9700 1 -- 172.21.15.47:0/2404105761 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66cc0128c0 con 0x7f66d80a72f0 2022-06-29T15:42:57.098 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.075+0000 7f66dc8ac700 1 -- 172.21.15.47:0/2404105761 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66d8148b00 con 0x7f66d80a72f0 2022-06-29T15:42:57.099 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.075+0000 7f66dc8ac700 1 -- 172.21.15.47:0/2404105761 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66d80adba0 con 0x7f66d80a72f0 2022-06-29T15:42:57.100 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.079+0000 7f66d4ff9700 1 -- 172.21.15.47:0/2404105761 <== mon.2 v2:172.21.15.64:3301/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f66cc019030 con 0x7f66d80a72f0 2022-06-29T15:42:57.100 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.079+0000 7f66d4ff9700 1 --2- 172.21.15.47:0/2404105761 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f66b80539e0 0x7f66b8055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:57.101 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.079+0000 7f66d4ff9700 1 -- 172.21.15.47:0/2404105761 <== mon.2 v2:172.21.15.64:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f66cc06b340 con 0x7f66d80a72f0 2022-06-29T15:42:57.101 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.079+0000 7f66d77fe700 1 --2- 172.21.15.47:0/2404105761 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f66b80539e0 0x7f66b8055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:57.101 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.079+0000 7f66d77fe700 1 --2- 172.21.15.47:0/2404105761 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f66b80539e0 0x7f66b8055ea0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f66c801a000 tx=0x7f66c8014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:57.101 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.087+0000 7f66d4ff9700 1 -- 172.21.15.47:0/2404105761 <== mon.2 v2:172.21.15.64:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f66cc042100 con 0x7f66d80a72f0 2022-06-29T15:42:57.226 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.219+0000 7f66dc8ac700 1 -- 172.21.15.47:0/2404105761 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f66d80adde0 con 0x7f66d80a72f0 2022-06-29T15:42:57.228 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.219+0000 7f66d4ff9700 1 -- 172.21.15.47:0/2404105761 <== mon.2 v2:172.21.15.64:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f66cc045720 con 0x7f66d80a72f0 2022-06-29T15:42:57.229 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:42:57.230 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":26,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:42:34.474438+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:42:28.400446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-06-29T15:42:31.167705+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:42:57.232 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.223+0000 7f66dc8ac700 1 -- 172.21.15.47:0/2404105761 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f66b80539e0 msgr2=0x7f66b8055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:57.232 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.223+0000 7f66dc8ac700 1 --2- 172.21.15.47:0/2404105761 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f66b80539e0 0x7f66b8055ea0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f66c801a000 tx=0x7f66c8014000).stop 2022-06-29T15:42:57.233 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.223+0000 7f66dc8ac700 1 -- 172.21.15.47:0/2404105761 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f66d80a72f0 msgr2=0x7f66d814c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:57.233 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.223+0000 7f66dc8ac700 1 --2- 172.21.15.47:0/2404105761 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f66d80a72f0 0x7f66d814c020 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f66d814a190 tx=0x7f66cc007a20).stop 2022-06-29T15:42:57.233 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.223+0000 7f66dc8ac700 1 -- 172.21.15.47:0/2404105761 shutdown_connections 2022-06-29T15:42:57.233 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.223+0000 7f66dc8ac700 1 --2- 172.21.15.47:0/2404105761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66d80b5c60 0x7f66d814a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:57.233 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.223+0000 7f66dc8ac700 1 --2- 172.21.15.47:0/2404105761 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f66d80a72f0 0x7f66d814c020 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:57.234 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.223+0000 7f66dc8ac700 1 --2- 172.21.15.47:0/2404105761 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f66b80539e0 0x7f66b8055ea0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:57.234 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.223+0000 7f66dc8ac700 1 --2- 172.21.15.47:0/2404105761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66d8060eb0 0x7f66d80ae040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:57.234 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.223+0000 7f66dc8ac700 1 -- 172.21.15.47:0/2404105761 >> 172.21.15.47:0/2404105761 conn(0x7f66d8062560 msgr2=0x7f66d80b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:57.234 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.223+0000 7f66dc8ac700 1 -- 172.21.15.47:0/2404105761 shutdown_connections 2022-06-29T15:42:57.235 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.223+0000 7f66dc8ac700 1 -- 172.21.15.47:0/2404105761 wait complete. 2022-06-29T15:42:57.241 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-29T15:42:16.971431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-29T15:42:25.290884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-29T15:42:28.400446+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2022-06-29T15:42:31.167705+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-29T15:42:57.242 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-29T15:42:57.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.343+0000 7fb8db09e700 1 Processor -- start 2022-06-29T15:42:57.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.343+0000 7fb8db09e700 1 -- start start 2022-06-29T15:42:57.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.343+0000 7fb8db09e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb8d40b64c0 0x7fb8d40b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:57.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.343+0000 7fb8db09e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb8d4066fe0 0x7fb8d40b4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:57.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.343+0000 7fb8db09e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb8d4143150 0x7fb8d40b5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:57.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.343+0000 7fb8db09e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8d40b13a0 con 0x7fb8d4066fe0 2022-06-29T15:42:57.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.343+0000 7fb8db09e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8d40b68a0 con 0x7fb8d40b64c0 2022-06-29T15:42:57.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.343+0000 7fb8db09e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d41473d0 con 0x7fb8d4143150 2022-06-29T15:42:57.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.343+0000 7fb8d989b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb8d4066fe0 0x7fb8d40b4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:57.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.343+0000 7fb8d989b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb8d4066fe0 0x7fb8d40b4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46650/0 (socket says 172.21.15.47:46650) 2022-06-29T15:42:57.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.343+0000 7fb8d989b700 1 -- 172.21.15.47:0/2237918845 learned_addr learned my addr 172.21.15.47:0/2237918845 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:57.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.343+0000 7fb8da09c700 1 --2- 172.21.15.47:0/2237918845 >> v2:172.21.15.64:3300/0 conn(0x7fb8d40b64c0 0x7fb8d40b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:57.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.343+0000 7fb8da89d700 1 --2- 172.21.15.47:0/2237918845 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb8d4143150 0x7fb8d40b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:57.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.343+0000 7fb8d989b700 1 -- 172.21.15.47:0/2237918845 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb8d4143150 msgr2=0x7fb8d40b5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:57.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.343+0000 7fb8d989b700 1 --2- 172.21.15.47:0/2237918845 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb8d4143150 0x7fb8d40b5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:57.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.343+0000 7fb8d989b700 1 -- 172.21.15.47:0/2237918845 >> v2:172.21.15.64:3300/0 conn(0x7fb8d40b64c0 msgr2=0x7fb8d40b5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:57.370 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.343+0000 7fb8d989b700 1 --2- 172.21.15.47:0/2237918845 >> v2:172.21.15.64:3300/0 conn(0x7fb8d40b64c0 0x7fb8d40b5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:57.370 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.343+0000 7fb8d989b700 1 -- 172.21.15.47:0/2237918845 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d40b46a0 con 0x7fb8d4066fe0 2022-06-29T15:42:57.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.343+0000 7fb8d989b700 1 --2- 172.21.15.47:0/2237918845 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8d4066fe0 0x7fb8d40b4b40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7fb8c8002f70 tx=0x7fb8c8006e50).ready entity=mon.0 client_cookie=b2afe5f029850bda server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:57.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.343+0000 7fb8d909a700 1 -- 172.21.15.47:0/2237918845 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8c800d570 con 0x7fb8d4066fe0 2022-06-29T15:42:57.372 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.347+0000 7fb8d909a700 1 -- 172.21.15.47:0/2237918845 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c8000e90 con 0x7fb8d4066fe0 2022-06-29T15:42:57.372 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.347+0000 7fb8d909a700 1 -- 172.21.15.47:0/2237918845 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8c800d900 con 0x7fb8d4066fe0 2022-06-29T15:42:57.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.347+0000 7fb8db09e700 1 -- 172.21.15.47:0/2237918845 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8d4066fe0 msgr2=0x7fb8d40b4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:57.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.347+0000 7fb8db09e700 1 --2- 172.21.15.47:0/2237918845 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8d4066fe0 0x7fb8d40b4b40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7fb8c8002f70 tx=0x7fb8c8006e50).stop 2022-06-29T15:42:57.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.347+0000 7fb8db09e700 1 -- 172.21.15.47:0/2237918845 shutdown_connections 2022-06-29T15:42:57.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.347+0000 7fb8db09e700 1 --2- 172.21.15.47:0/2237918845 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb8d4143150 0x7fb8d40b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:57.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.347+0000 7fb8db09e700 1 --2- 172.21.15.47:0/2237918845 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8d4066fe0 0x7fb8d40b4b40 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:57.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.347+0000 7fb8db09e700 1 --2- 172.21.15.47:0/2237918845 >> v2:172.21.15.64:3300/0 conn(0x7fb8d40b64c0 0x7fb8d40b5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:57.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.347+0000 7fb8db09e700 1 -- 172.21.15.47:0/2237918845 >> 172.21.15.47:0/2237918845 conn(0x7fb8d4060e00 msgr2=0x7fb8d40611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:57.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.347+0000 7fb8db09e700 1 -- 172.21.15.47:0/2237918845 shutdown_connections 2022-06-29T15:42:57.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.347+0000 7fb8db09e700 1 -- 172.21.15.47:0/2237918845 wait complete. 2022-06-29T15:42:57.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.347+0000 7fb8db09e700 1 Processor -- start 2022-06-29T15:42:57.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.347+0000 7fb8db09e700 1 -- start start 2022-06-29T15:42:57.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.351+0000 7fb8db09e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8d4066fe0 0x7fb8d40afe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:57.379 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.351+0000 7fb8db09e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb8d40b64c0 0x7fb8d41422c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:57.379 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.351+0000 7fb8db09e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8d4143150 0x7fb8d4151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:57.380 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.351+0000 7fb8db09e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d40af780 con 0x7fb8d4066fe0 2022-06-29T15:42:57.380 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.351+0000 7fb8db09e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d40adf70 con 0x7fb8d4143150 2022-06-29T15:42:57.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.351+0000 7fb8db09e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d40af9b0 con 0x7fb8d40b64c0 2022-06-29T15:42:57.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.351+0000 7fb8da09c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8d4066fe0 0x7fb8d40afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:57.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.351+0000 7fb8da09c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8d4066fe0 0x7fb8d40afe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46654/0 (socket says 172.21.15.47:46654) 2022-06-29T15:42:57.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.351+0000 7fb8da09c700 1 -- 172.21.15.47:0/2886405112 learned_addr learned my addr 172.21.15.47:0/2886405112 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:57.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.351+0000 7fb8da89d700 1 --2- 172.21.15.47:0/2886405112 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8d4143150 0x7fb8d4151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:57.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.351+0000 7fb8d989b700 1 --2- 172.21.15.47:0/2886405112 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb8d40b64c0 0x7fb8d41422c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:57.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.351+0000 7fb8da09c700 1 -- 172.21.15.47:0/2886405112 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb8d40b64c0 msgr2=0x7fb8d41422c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:57.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.351+0000 7fb8da09c700 1 --2- 172.21.15.47:0/2886405112 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb8d40b64c0 0x7fb8d41422c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:57.385 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.351+0000 7fb8da09c700 1 -- 172.21.15.47:0/2886405112 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8d4143150 msgr2=0x7fb8d4151fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:57.385 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.351+0000 7fb8da09c700 1 --2- 172.21.15.47:0/2886405112 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8d4143150 0x7fb8d4151fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:57.385 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.351+0000 7fb8da09c700 1 -- 172.21.15.47:0/2886405112 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8c800b1e0 con 0x7fb8d4066fe0 2022-06-29T15:42:57.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.351+0000 7fb8da09c700 1 --2- 172.21.15.47:0/2886405112 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8d4066fe0 0x7fb8d40afe50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7fb8c400c000 tx=0x7fb8c40091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:57.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.351+0000 7fb8d37fe700 1 -- 172.21.15.47:0/2886405112 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8c400c6a0 con 0x7fb8d4066fe0 2022-06-29T15:42:57.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.351+0000 7fb8d37fe700 1 -- 172.21.15.47:0/2886405112 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c400c800 con 0x7fb8d4066fe0 2022-06-29T15:42:57.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.351+0000 7fb8db09e700 1 -- 172.21.15.47:0/2886405112 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8d40ae210 con 0x7fb8d4066fe0 2022-06-29T15:42:57.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.351+0000 7fb8d37fe700 1 -- 172.21.15.47:0/2886405112 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8c400cb60 con 0x7fb8d4066fe0 2022-06-29T15:42:57.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.351+0000 7fb8db09e700 1 -- 172.21.15.47:0/2886405112 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8d40b05e0 con 0x7fb8d4066fe0 2022-06-29T15:42:57.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.351+0000 7fb8db09e700 1 -- 172.21.15.47:0/2886405112 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8d40afb70 con 0x7fb8d4066fe0 2022-06-29T15:42:57.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.355+0000 7fb8d37fe700 1 -- 172.21.15.47:0/2886405112 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb8c4022070 con 0x7fb8d4066fe0 2022-06-29T15:42:57.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.355+0000 7fb8d37fe700 1 --2- 172.21.15.47:0/2886405112 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb8b4053a70 0x7fb8b4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:57.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.355+0000 7fb8d37fe700 1 -- 172.21.15.47:0/2886405112 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fb8c406c5c0 con 0x7fb8d4066fe0 2022-06-29T15:42:57.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.355+0000 7fb8d989b700 1 --2- 172.21.15.47:0/2886405112 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb8b4053a70 0x7fb8b4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:57.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.355+0000 7fb8d989b700 1 --2- 172.21.15.47:0/2886405112 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb8b4053a70 0x7fb8b4055f30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fb8c801a000 tx=0x7fb8c8014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:57.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.359+0000 7fb8d37fe700 1 -- 172.21.15.47:0/2886405112 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb8c4043380 con 0x7fb8d4066fe0 2022-06-29T15:42:57.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.487+0000 7fb8db09e700 1 -- 172.21.15.47:0/2886405112 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fb8d40573d0 con 0x7fb8d4066fe0 2022-06-29T15:42:57.497 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.491+0000 7fb8d37fe700 1 -- 172.21.15.47:0/2886405112 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v26) v1 ==== 110+0+10 (secure 0 0 0) 0x7fb8c40469a0 con 0x7fb8d4066fe0 2022-06-29T15:42:57.497 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2022-06-29T15:42:57.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.491+0000 7fb8db09e700 1 -- 172.21.15.47:0/2886405112 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb8b4053a70 msgr2=0x7fb8b4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:57.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.491+0000 7fb8db09e700 1 --2- 172.21.15.47:0/2886405112 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb8b4053a70 0x7fb8b4055f30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fb8c801a000 tx=0x7fb8c8014000).stop 2022-06-29T15:42:57.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.491+0000 7fb8db09e700 1 -- 172.21.15.47:0/2886405112 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8d4066fe0 msgr2=0x7fb8d40afe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:57.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.491+0000 7fb8db09e700 1 --2- 172.21.15.47:0/2886405112 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8d4066fe0 0x7fb8d40afe50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7fb8c400c000 tx=0x7fb8c40091d0).stop 2022-06-29T15:42:57.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.491+0000 7fb8db09e700 1 -- 172.21.15.47:0/2886405112 shutdown_connections 2022-06-29T15:42:57.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.491+0000 7fb8db09e700 1 --2- 172.21.15.47:0/2886405112 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb8b4053a70 0x7fb8b4055f30 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:57.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.491+0000 7fb8db09e700 1 --2- 172.21.15.47:0/2886405112 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8d4143150 0x7fb8d4151fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:57.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.491+0000 7fb8db09e700 1 --2- 172.21.15.47:0/2886405112 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb8d40b64c0 0x7fb8d41422c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:57.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.491+0000 7fb8db09e700 1 --2- 172.21.15.47:0/2886405112 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8d4066fe0 0x7fb8d40afe50 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:57.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.491+0000 7fb8db09e700 1 -- 172.21.15.47:0/2886405112 >> 172.21.15.47:0/2886405112 conn(0x7fb8d4060e00 msgr2=0x7fb8d40b7650 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:57.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.491+0000 7fb8db09e700 1 -- 172.21.15.47:0/2886405112 shutdown_connections 2022-06-29T15:42:57.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.491+0000 7fb8db09e700 1 -- 172.21.15.47:0/2886405112 wait complete. 2022-06-29T15:42:57.511 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-29T15:42:57.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.611+0000 7ff883fff700 1 Processor -- start 2022-06-29T15:42:57.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.611+0000 7ff883fff700 1 -- start start 2022-06-29T15:42:57.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.611+0000 7ff883fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff88405c850 0x7ff884143a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:57.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.611+0000 7ff883fff700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff88405cc30 con 0x7ff8840676a0 2022-06-29T15:42:57.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.611+0000 7ff883fff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8841430a0 con 0x7ff8840b5d90 2022-06-29T15:42:57.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.611+0000 7ff883fff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff884143300 con 0x7ff88405c850 2022-06-29T15:42:57.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.611+0000 7ff8837fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff8840676a0 0x7ff884140130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:60452/0 (socket says 172.21.15.47:60452) 2022-06-29T15:42:57.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.611+0000 7ff8837fe700 1 -- 172.21.15.47:0/2873871794 learned_addr learned my addr 172.21.15.47:0/2873871794 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:57.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.611+0000 7ff882ffd700 1 --2- 172.21.15.47:0/2873871794 >> v2:172.21.15.64:3300/0 conn(0x7ff88405c850 0x7ff884143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:57.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.611+0000 7ff881ffb700 1 -- 172.21.15.47:0/2873871794 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3782024383 0 0) 0x7ff88405cc30 con 0x7ff8840676a0 2022-06-29T15:42:57.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.611+0000 7ff881ffb700 1 -- 172.21.15.47:0/2873871794 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff868003260 con 0x7ff8840676a0 2022-06-29T15:42:57.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.615+0000 7ff881ffb700 1 -- 172.21.15.47:0/2873871794 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1516582497 0 0) 0x7ff8841430a0 con 0x7ff8840b5d90 2022-06-29T15:42:57.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.615+0000 7ff881ffb700 1 -- 172.21.15.47:0/2873871794 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff88405cc30 con 0x7ff8840b5d90 2022-06-29T15:42:57.626 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.615+0000 7ff881ffb700 1 -- 172.21.15.47:0/2873871794 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2044955882 0 0) 0x7ff868003260 con 0x7ff8840676a0 2022-06-29T15:42:57.626 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.615+0000 7ff881ffb700 1 -- 172.21.15.47:0/2873871794 >> v2:172.21.15.64:3300/0 conn(0x7ff88405c850 msgr2=0x7ff884143a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:57.626 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.615+0000 7ff881ffb700 1 --2- 172.21.15.47:0/2873871794 >> v2:172.21.15.64:3300/0 conn(0x7ff88405c850 0x7ff884143a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:57.626 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.615+0000 7ff881ffb700 1 -- 172.21.15.47:0/2873871794 >> v1:172.21.15.64:6789/0 conn(0x7ff8840b5d90 legacy=0x7ff8840b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:57.627 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.615+0000 7ff881ffb700 1 -- 172.21.15.47:0/2873871794 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8841435a0 con 0x7ff8840676a0 2022-06-29T15:42:57.627 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.615+0000 7ff881ffb700 1 -- 172.21.15.47:0/2873871794 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7ff87c001a00 con 0x7ff8840676a0 2022-06-29T15:42:57.627 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.615+0000 7ff881ffb700 1 -- 172.21.15.47:0/2873871794 >> v1:172.21.15.47:6789/0 conn(0x7ff8840676a0 legacy=0x7ff884140130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:57.627 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.615+0000 7ff881ffb700 1 --2- 172.21.15.47:0/2873871794 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff8680038e0 0x7ff868003ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:57.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.615+0000 7ff881ffb700 1 --2- 172.21.15.47:0/2873871794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff868004500 0x7ff868004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:57.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.615+0000 7ff881ffb700 1 --2- 172.21.15.47:0/2873871794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff868004e70 0x7ff86800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:57.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.615+0000 7ff881ffb700 1 -- 172.21.15.47:0/2873871794 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff884143300 con 0x7ff868004e70 2022-06-29T15:42:57.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.615+0000 7ff881ffb700 1 -- 172.21.15.47:0/2873871794 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff86800bca0 con 0x7ff868004500 2022-06-29T15:42:57.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.615+0000 7ff881ffb700 1 -- 172.21.15.47:0/2873871794 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff86800be60 con 0x7ff8680038e0 2022-06-29T15:42:57.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.615+0000 7ff8837fe700 1 --2- 172.21.15.47:0/2873871794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff868004e70 0x7ff86800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:57.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.615+0000 7ff882ffd700 1 --2- 172.21.15.47:0/2873871794 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff8680038e0 0x7ff868003ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:57.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.615+0000 7ff8827fc700 1 --2- 172.21.15.47:0/2873871794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff868004500 0x7ff868004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:57.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.615+0000 7ff8837fe700 1 -- 172.21.15.47:0/2873871794 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff8680038e0 msgr2=0x7ff868003ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:57.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.615+0000 7ff8837fe700 1 --2- 172.21.15.47:0/2873871794 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff8680038e0 0x7ff868003ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:57.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.615+0000 7ff8837fe700 1 -- 172.21.15.47:0/2873871794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff868004500 msgr2=0x7ff868004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:57.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.615+0000 7ff8837fe700 1 --2- 172.21.15.47:0/2873871794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff868004500 0x7ff868004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:57.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.615+0000 7ff8837fe700 1 -- 172.21.15.47:0/2873871794 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8841435a0 con 0x7ff868004e70 2022-06-29T15:42:57.631 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.615+0000 7ff882ffd700 1 --2- 172.21.15.47:0/2873871794 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff8680038e0 0x7ff868003ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:42:57.631 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.615+0000 7ff8837fe700 1 -- 172.21.15.47:0/2873871794 reap_dead start 2022-06-29T15:42:57.631 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.615+0000 7ff8837fe700 1 --2- 172.21.15.47:0/2873871794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff868004e70 0x7ff86800b750 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7ff87c0017d0 tx=0x7ff87c0079b0).ready entity=mon.0 client_cookie=385b13b6e56b151 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:57.631 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.615+0000 7ff881ffb700 1 -- 172.21.15.47:0/2873871794 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff87c00cca0 con 0x7ff868004e70 2022-06-29T15:42:57.631 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.615+0000 7ff881ffb700 1 -- 172.21.15.47:0/2873871794 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff87c00ce00 con 0x7ff868004e70 2022-06-29T15:42:57.632 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.615+0000 7ff881ffb700 1 -- 172.21.15.47:0/2873871794 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff87c0072c0 con 0x7ff868004e70 2022-06-29T15:42:57.632 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.615+0000 7ff883fff700 1 -- 172.21.15.47:0/2873871794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff868004e70 msgr2=0x7ff86800b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:57.632 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.615+0000 7ff883fff700 1 --2- 172.21.15.47:0/2873871794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff868004e70 0x7ff86800b750 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7ff87c0017d0 tx=0x7ff87c0079b0).stop 2022-06-29T15:42:57.632 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.615+0000 7ff883fff700 1 -- 172.21.15.47:0/2873871794 shutdown_connections 2022-06-29T15:42:57.633 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.615+0000 7ff883fff700 1 --2- 172.21.15.47:0/2873871794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff868004e70 0x7ff86800b750 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:57.633 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.615+0000 7ff883fff700 1 -- 172.21.15.47:0/2873871794 >> 172.21.15.47:0/2873871794 conn(0x7ff884060e00 msgr2=0x7ff8840611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:57.633 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.615+0000 7ff883fff700 1 -- 172.21.15.47:0/2873871794 shutdown_connections 2022-06-29T15:42:57.633 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.615+0000 7ff883fff700 1 -- 172.21.15.47:0/2873871794 wait complete. 2022-06-29T15:42:57.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.619+0000 7ff883fff700 1 Processor -- start 2022-06-29T15:42:57.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.619+0000 7ff883fff700 1 -- start start 2022-06-29T15:42:57.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.619+0000 7ff883fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff868004e70 0x7ff8840b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:57.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.619+0000 7ff883fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8840b2e40 0x7ff88412c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:57.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.619+0000 7ff883fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff8840b32c0 0x7ff88412c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:57.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.619+0000 7ff883fff700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff88412cbc0 con 0x7ff8840b2e40 2022-06-29T15:42:57.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.619+0000 7ff883fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff884152460 con 0x7ff868004e70 2022-06-29T15:42:57.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.619+0000 7ff883fff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff884152700 con 0x7ff8840b32c0 2022-06-29T15:42:57.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.619+0000 7ff8827fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8840b2e40 0x7ff88412c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:57.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.619+0000 7ff8827fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8840b2e40 0x7ff88412c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46676/0 (socket says 172.21.15.47:46676) 2022-06-29T15:42:57.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.619+0000 7ff8827fc700 1 -- 172.21.15.47:0/636803940 learned_addr learned my addr 172.21.15.47:0/636803940 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:57.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.619+0000 7ff882ffd700 1 --2- 172.21.15.47:0/636803940 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff868004e70 0x7ff8840b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:57.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.619+0000 7ff8837fe700 1 --2- 172.21.15.47:0/636803940 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff8840b32c0 0x7ff88412c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:57.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.619+0000 7ff8827fc700 1 -- 172.21.15.47:0/636803940 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff8840b32c0 msgr2=0x7ff88412c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:57.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.619+0000 7ff8827fc700 1 --2- 172.21.15.47:0/636803940 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff8840b32c0 0x7ff88412c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:57.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.619+0000 7ff8827fc700 1 -- 172.21.15.47:0/636803940 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff868004e70 msgr2=0x7ff8840b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:57.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.619+0000 7ff8827fc700 1 --2- 172.21.15.47:0/636803940 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff868004e70 0x7ff8840b3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:57.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.619+0000 7ff8827fc700 1 -- 172.21.15.47:0/636803940 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8841529a0 con 0x7ff8840b2e40 2022-06-29T15:42:57.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.619+0000 7ff8837fe700 1 --2- 172.21.15.47:0/636803940 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff8840b32c0 0x7ff88412c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:42:57.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.619+0000 7ff882ffd700 1 --2- 172.21.15.47:0/636803940 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff868004e70 0x7ff8840b3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:42:57.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.619+0000 7ff8827fc700 1 --2- 172.21.15.47:0/636803940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8840b2e40 0x7ff88412c1e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7ff87800d000 tx=0x7ff878007a00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:57.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.619+0000 7ff873fff700 1 -- 172.21.15.47:0/636803940 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff878011780 con 0x7ff8840b2e40 2022-06-29T15:42:57.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.619+0000 7ff873fff700 1 -- 172.21.15.47:0/636803940 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8780118e0 con 0x7ff8840b2e40 2022-06-29T15:42:57.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.619+0000 7ff883fff700 1 -- 172.21.15.47:0/636803940 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff884152c40 con 0x7ff8840b2e40 2022-06-29T15:42:57.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.619+0000 7ff873fff700 1 -- 172.21.15.47:0/636803940 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff878011c40 con 0x7ff8840b2e40 2022-06-29T15:42:57.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.619+0000 7ff883fff700 1 -- 172.21.15.47:0/636803940 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff88412f060 con 0x7ff8840b2e40 2022-06-29T15:42:57.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.623+0000 7ff883fff700 1 -- 172.21.15.47:0/636803940 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff884152dd0 con 0x7ff8840b2e40 2022-06-29T15:42:57.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.623+0000 7ff873fff700 1 -- 172.21.15.47:0/636803940 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff87801b030 con 0x7ff8840b2e40 2022-06-29T15:42:57.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.623+0000 7ff873fff700 1 --2- 172.21.15.47:0/636803940 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff860053b30 0x7ff860055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:57.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.623+0000 7ff873fff700 1 -- 172.21.15.47:0/636803940 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7ff87806c140 con 0x7ff8840b2e40 2022-06-29T15:42:57.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.623+0000 7ff882ffd700 1 --2- 172.21.15.47:0/636803940 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff860053b30 0x7ff860055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:57.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.623+0000 7ff882ffd700 1 --2- 172.21.15.47:0/636803940 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff860053b30 0x7ff860055ff0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7ff874006000 tx=0x7ff874008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:57.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.631+0000 7ff873fff700 1 -- 172.21.15.47:0/636803940 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff878042f00 con 0x7ff8840b2e40 2022-06-29T15:42:57.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.751+0000 7ff883fff700 1 -- 172.21.15.47:0/636803940 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff8840573d0 con 0x7ff8840b2e40 2022-06-29T15:42:57.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.751+0000 7ff873fff700 1 -- 172.21.15.47:0/636803940 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff8780190a0 con 0x7ff8840b2e40 2022-06-29T15:42:57.761 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:42:57.763 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.755+0000 7ff883fff700 1 -- 172.21.15.47:0/636803940 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff860053b30 msgr2=0x7ff860055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:57.764 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.755+0000 7ff883fff700 1 --2- 172.21.15.47:0/636803940 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff860053b30 0x7ff860055ff0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7ff874006000 tx=0x7ff874008040).stop 2022-06-29T15:42:57.764 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.755+0000 7ff883fff700 1 -- 172.21.15.47:0/636803940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8840b2e40 msgr2=0x7ff88412c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:57.764 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.755+0000 7ff883fff700 1 --2- 172.21.15.47:0/636803940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8840b2e40 0x7ff88412c1e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7ff87800d000 tx=0x7ff878007a00).stop 2022-06-29T15:42:57.764 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.755+0000 7ff883fff700 1 -- 172.21.15.47:0/636803940 shutdown_connections 2022-06-29T15:42:57.765 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.755+0000 7ff883fff700 1 --2- 172.21.15.47:0/636803940 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff8840b32c0 0x7ff88412c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:57.765 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.755+0000 7ff883fff700 1 --2- 172.21.15.47:0/636803940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8840b2e40 0x7ff88412c1e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:57.765 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.755+0000 7ff883fff700 1 --2- 172.21.15.47:0/636803940 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff860053b30 0x7ff860055ff0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:57.765 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.755+0000 7ff883fff700 1 --2- 172.21.15.47:0/636803940 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff868004e70 0x7ff8840b3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:57.766 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.755+0000 7ff883fff700 1 -- 172.21.15.47:0/636803940 >> 172.21.15.47:0/636803940 conn(0x7ff884060e00 msgr2=0x7ff884142630 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:57.766 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.755+0000 7ff883fff700 1 -- 172.21.15.47:0/636803940 shutdown_connections 2022-06-29T15:42:57.766 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.755+0000 7ff883fff700 1 -- 172.21.15.47:0/636803940 wait complete. 2022-06-29T15:42:57.779 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-29T15:42:57.889 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.883+0000 7efdacc12700 1 Processor -- start 2022-06-29T15:42:57.890 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.883+0000 7efdacc12700 1 -- start start 2022-06-29T15:42:57.892 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.883+0000 7efdacc12700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7efda8060ee0 0x7efda80612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:57.892 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.883+0000 7efdacc12700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7efda81471a0 0x7efda8066f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:57.892 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.883+0000 7efdacc12700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efda8067610 con 0x7efda80b4300 2022-06-29T15:42:57.893 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.883+0000 7efdacc12700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efda81467c0 con 0x7efda8060ee0 2022-06-29T15:42:57.893 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.883+0000 7efdacc12700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efda8146a60 con 0x7efda81471a0 2022-06-29T15:42:57.893 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.883+0000 7efda659c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7efda8060ee0 0x7efda80612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:57.893 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.883+0000 7efda5d9b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7efda80b4300 0x7efda81430e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:60468/0 (socket says 172.21.15.47:60468) 2022-06-29T15:42:57.894 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.883+0000 7efda5d9b700 1 -- 172.21.15.47:0/4051720454 learned_addr learned my addr 172.21.15.47:0/4051720454 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:57.894 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.883+0000 7efda6d9d700 1 --2- 172.21.15.47:0/4051720454 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7efda81471a0 0x7efda8066f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:57.894 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.883+0000 7efda659c700 1 -- 172.21.15.47:0/4051720454 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7efda81471a0 msgr2=0x7efda8066f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:57.894 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.883+0000 7efda659c700 1 --2- 172.21.15.47:0/4051720454 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7efda81471a0 0x7efda8066f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:57.895 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.883+0000 7efda659c700 1 -- 172.21.15.47:0/4051720454 >> v1:172.21.15.47:6789/0 conn(0x7efda80b4300 legacy=0x7efda81430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:57.895 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.883+0000 7efda559a700 1 -- 172.21.15.47:0/4051720454 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3170038224 0 0) 0x7efda8067610 con 0x7efda80b4300 2022-06-29T15:42:57.895 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.883+0000 7efda659c700 1 -- 172.21.15.47:0/4051720454 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda8146d00 con 0x7efda8060ee0 2022-06-29T15:42:57.896 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.883+0000 7efda659c700 1 --2- 172.21.15.47:0/4051720454 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efda8060ee0 0x7efda80612c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7efda80ae790 tx=0x7efd98006df0).ready entity=mon.0 client_cookie=9d27f1da5e512568 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:57.896 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.883+0000 7efda559a700 1 -- 172.21.15.47:0/4051720454 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efd9800d570 con 0x7efda8060ee0 2022-06-29T15:42:57.896 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.883+0000 7efda559a700 1 -- 172.21.15.47:0/4051720454 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd98000e90 con 0x7efda8060ee0 2022-06-29T15:42:57.897 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.883+0000 7efda559a700 1 -- 172.21.15.47:0/4051720454 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efd9800d8c0 con 0x7efda8060ee0 2022-06-29T15:42:57.901 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.887+0000 7efdacc12700 1 -- 172.21.15.47:0/4051720454 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efda8060ee0 msgr2=0x7efda80612c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:57.901 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.887+0000 7efdacc12700 1 --2- 172.21.15.47:0/4051720454 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efda8060ee0 0x7efda80612c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7efda80ae790 tx=0x7efd98006df0).stop 2022-06-29T15:42:57.902 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.887+0000 7efdacc12700 1 -- 172.21.15.47:0/4051720454 shutdown_connections 2022-06-29T15:42:57.902 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.887+0000 7efdacc12700 1 --2- 172.21.15.47:0/4051720454 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7efda81471a0 0x7efda8066f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:57.902 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.887+0000 7efdacc12700 1 --2- 172.21.15.47:0/4051720454 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efda8060ee0 0x7efda80612c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:57.902 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.887+0000 7efdacc12700 1 -- 172.21.15.47:0/4051720454 >> 172.21.15.47:0/4051720454 conn(0x7efda805c510 msgr2=0x7efda805f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:57.903 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.887+0000 7efdacc12700 1 -- 172.21.15.47:0/4051720454 shutdown_connections 2022-06-29T15:42:57.903 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.887+0000 7efdacc12700 1 -- 172.21.15.47:0/4051720454 wait complete. 2022-06-29T15:42:57.903 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.887+0000 7efdacc12700 1 Processor -- start 2022-06-29T15:42:57.903 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.887+0000 7efdacc12700 1 -- start start 2022-06-29T15:42:57.904 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.887+0000 7efdacc12700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efda8060ee0 0x7efda813b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:57.904 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.887+0000 7efdacc12700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efda80b4300 0x7efda81526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:57.904 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.887+0000 7efdacc12700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7efda81471a0 0x7efda8152bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:57.904 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.887+0000 7efdacc12700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda80b13a0 con 0x7efda8060ee0 2022-06-29T15:42:57.905 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.887+0000 7efdacc12700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda8152320 con 0x7efda80b4300 2022-06-29T15:42:57.905 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.887+0000 7efdacc12700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efda80b1080 con 0x7efda81471a0 2022-06-29T15:42:57.905 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.887+0000 7efda659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efda8060ee0 0x7efda813b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:57.906 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.887+0000 7efda659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efda8060ee0 0x7efda813b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46688/0 (socket says 172.21.15.47:46688) 2022-06-29T15:42:57.906 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.887+0000 7efda659c700 1 -- 172.21.15.47:0/3263497115 learned_addr learned my addr 172.21.15.47:0/3263497115 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:57.906 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.887+0000 7efda5d9b700 1 --2- 172.21.15.47:0/3263497115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efda80b4300 0x7efda81526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:57.906 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.887+0000 7efda6d9d700 1 --2- 172.21.15.47:0/3263497115 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7efda81471a0 0x7efda8152bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:57.907 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.891+0000 7efda5d9b700 1 -- 172.21.15.47:0/3263497115 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7efda81471a0 msgr2=0x7efda8152bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:57.907 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.891+0000 7efda5d9b700 1 --2- 172.21.15.47:0/3263497115 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7efda81471a0 0x7efda8152bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:57.907 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.891+0000 7efda5d9b700 1 -- 172.21.15.47:0/3263497115 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efda8060ee0 msgr2=0x7efda813b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:57.907 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.891+0000 7efda5d9b700 1 --2- 172.21.15.47:0/3263497115 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efda8060ee0 0x7efda813b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:57.908 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.891+0000 7efda5d9b700 1 -- 172.21.15.47:0/3263497115 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd9800b180 con 0x7efda80b4300 2022-06-29T15:42:57.908 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.891+0000 7efda5d9b700 1 --2- 172.21.15.47:0/3263497115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efda80b4300 0x7efda81526d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7efd9c000ed0 tx=0x7efd9c007a00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:57.908 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.891+0000 7efd977fe700 1 -- 172.21.15.47:0/3263497115 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efd9c0070d0 con 0x7efda80b4300 2022-06-29T15:42:57.908 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.891+0000 7efd977fe700 1 -- 172.21.15.47:0/3263497115 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd9c007230 con 0x7efda80b4300 2022-06-29T15:42:57.909 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.891+0000 7efdacc12700 1 -- 172.21.15.47:0/3263497115 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efda813de40 con 0x7efda80b4300 2022-06-29T15:42:57.909 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.891+0000 7efd977fe700 1 -- 172.21.15.47:0/3263497115 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efd9c005610 con 0x7efda80b4300 2022-06-29T15:42:57.909 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.891+0000 7efdacc12700 1 -- 172.21.15.47:0/3263497115 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efda8140600 con 0x7efda80b4300 2022-06-29T15:42:57.909 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.891+0000 7efdacc12700 1 -- 172.21.15.47:0/3263497115 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efda8152470 con 0x7efda80b4300 2022-06-29T15:42:57.910 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.891+0000 7efd977fe700 1 -- 172.21.15.47:0/3263497115 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7efd9c01a060 con 0x7efda80b4300 2022-06-29T15:42:57.910 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.891+0000 7efd977fe700 1 --2- 172.21.15.47:0/3263497115 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7efd84053950 0x7efd84055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:57.910 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.891+0000 7efda659c700 1 --2- 172.21.15.47:0/3263497115 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7efd84053950 0x7efd84055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:57.910 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.891+0000 7efd977fe700 1 -- 172.21.15.47:0/3263497115 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7efd9c06b7e0 con 0x7efda80b4300 2022-06-29T15:42:57.911 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.895+0000 7efda659c700 1 --2- 172.21.15.47:0/3263497115 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7efd84053950 0x7efd84055e10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7efd9801a000 tx=0x7efd98014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:57.911 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:57.903+0000 7efd977fe700 1 -- 172.21.15.47:0/3263497115 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7efd9c042490 con 0x7efda80b4300 2022-06-29T15:42:58.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.031+0000 7efdacc12700 1 -- 172.21.15.47:0/3263497115 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efda80573d0 con 0x7efda80b4300 2022-06-29T15:42:58.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.035+0000 7efd977fe700 1 -- 172.21.15.47:0/3263497115 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+10 (secure 0 0 0) 0x7efd9c018020 con 0x7efda80b4300 2022-06-29T15:42:58.041 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:42:58.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.039+0000 7efdacc12700 1 -- 172.21.15.47:0/3263497115 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7efd84053950 msgr2=0x7efd84055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:58.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.039+0000 7efdacc12700 1 --2- 172.21.15.47:0/3263497115 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7efd84053950 0x7efd84055e10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7efd9801a000 tx=0x7efd98014000).stop 2022-06-29T15:42:58.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.039+0000 7efdacc12700 1 -- 172.21.15.47:0/3263497115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efda80b4300 msgr2=0x7efda81526d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:58.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.039+0000 7efdacc12700 1 --2- 172.21.15.47:0/3263497115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efda80b4300 0x7efda81526d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7efd9c000ed0 tx=0x7efd9c007a00).stop 2022-06-29T15:42:58.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.039+0000 7efdacc12700 1 -- 172.21.15.47:0/3263497115 shutdown_connections 2022-06-29T15:42:58.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.039+0000 7efdacc12700 1 --2- 172.21.15.47:0/3263497115 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7efda81471a0 0x7efda8152bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:58.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.039+0000 7efdacc12700 1 --2- 172.21.15.47:0/3263497115 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7efd84053950 0x7efd84055e10 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:58.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.039+0000 7efdacc12700 1 --2- 172.21.15.47:0/3263497115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efda80b4300 0x7efda81526d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:58.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.039+0000 7efdacc12700 1 --2- 172.21.15.47:0/3263497115 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efda8060ee0 0x7efda813b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:58.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.039+0000 7efdacc12700 1 -- 172.21.15.47:0/3263497115 >> 172.21.15.47:0/3263497115 conn(0x7efda805c510 msgr2=0x7efda81444b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:58.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.039+0000 7efdacc12700 1 -- 172.21.15.47:0/3263497115 shutdown_connections 2022-06-29T15:42:58.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.039+0000 7efdacc12700 1 -- 172.21.15.47:0/3263497115 wait complete. 2022-06-29T15:42:58.062 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-29T15:42:58.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.163+0000 7f327bfff700 1 Processor -- start 2022-06-29T15:42:58.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.163+0000 7f327bfff700 1 -- start start 2022-06-29T15:42:58.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.167+0000 7f327bfff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f327c05c850 0x7f327c143a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:58.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.167+0000 7f327bfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f327c0676a0 0x7f327c140130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:58.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.167+0000 7f327bfff700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f327c05cc30 con 0x7f327c0b5d90 2022-06-29T15:42:58.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.167+0000 7f327bfff700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f327c1430a0 con 0x7f327c0676a0 2022-06-29T15:42:58.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.167+0000 7f327bfff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f327c143300 con 0x7f327c05c850 2022-06-29T15:42:58.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.167+0000 7f327a7fc700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f327c0b5d90 0x7f327c0b4860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:60486/0 (socket says 172.21.15.47:60486) 2022-06-29T15:42:58.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.167+0000 7f327b7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f327c0676a0 0x7f327c140130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:58.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.167+0000 7f327a7fc700 1 -- 172.21.15.47:0/2847382906 learned_addr learned my addr 172.21.15.47:0/2847382906 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:58.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.167+0000 7f327affd700 1 --2- 172.21.15.47:0/2847382906 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f327c05c850 0x7f327c143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:58.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.167+0000 7f3279ffb700 1 -- 172.21.15.47:0/2847382906 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 958142935 0 0) 0x7f327c05cc30 con 0x7f327c0b5d90 2022-06-29T15:42:58.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.167+0000 7f3279ffb700 1 -- 172.21.15.47:0/2847382906 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f325c003260 con 0x7f327c0b5d90 2022-06-29T15:42:58.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.167+0000 7f3279ffb700 1 -- 172.21.15.47:0/2847382906 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1789934869 0 0) 0x7f325c003260 con 0x7f327c0b5d90 2022-06-29T15:42:58.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.167+0000 7f3279ffb700 1 -- 172.21.15.47:0/2847382906 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f327c05c850 msgr2=0x7f327c143a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:58.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.167+0000 7f3279ffb700 1 --2- 172.21.15.47:0/2847382906 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f327c05c850 0x7f327c143a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:58.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.167+0000 7f3279ffb700 1 -- 172.21.15.47:0/2847382906 >> v2:172.21.15.47:3300/0 conn(0x7f327c0676a0 msgr2=0x7f327c140130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:58.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.167+0000 7f3279ffb700 1 --2- 172.21.15.47:0/2847382906 >> v2:172.21.15.47:3300/0 conn(0x7f327c0676a0 0x7f327c140130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:58.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.167+0000 7f3279ffb700 1 -- 172.21.15.47:0/2847382906 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f327c1435a0 con 0x7f327c0b5d90 2022-06-29T15:42:58.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.167+0000 7f3279ffb700 1 -- 172.21.15.47:0/2847382906 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7f326c001680 con 0x7f327c0b5d90 2022-06-29T15:42:58.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.167+0000 7f327b7fe700 1 --2- 172.21.15.47:0/2847382906 >> v2:172.21.15.47:3300/0 conn(0x7f327c0676a0 0x7f327c140130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:42:58.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.167+0000 7f3279ffb700 1 -- 172.21.15.47:0/2847382906 >> v1:172.21.15.47:6789/0 conn(0x7f327c0b5d90 legacy=0x7f327c0b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:58.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.167+0000 7f3279ffb700 1 --2- 172.21.15.47:0/2847382906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f325c003750 0x7f325c003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:58.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.167+0000 7f3279ffb700 1 --2- 172.21.15.47:0/2847382906 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f325c0040b0 0x7f325c004580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:58.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.167+0000 7f3279ffb700 1 --2- 172.21.15.47:0/2847382906 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f327c05c850 0x7f325c004c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:58.182 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.167+0000 7f3279ffb700 1 -- 172.21.15.47:0/2847382906 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327c1430a0 con 0x7f325c0040b0 2022-06-29T15:42:58.182 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.167+0000 7f3279ffb700 1 -- 172.21.15.47:0/2847382906 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327c143300 con 0x7f325c003750 2022-06-29T15:42:58.182 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.167+0000 7f3279ffb700 1 -- 172.21.15.47:0/2847382906 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f325c005370 con 0x7f327c05c850 2022-06-29T15:42:58.182 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.167+0000 7f327a7fc700 1 --2- 172.21.15.47:0/2847382906 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f325c0040b0 0x7f325c004580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:58.182 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.167+0000 7f327b7fe700 1 --2- 172.21.15.47:0/2847382906 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f327c05c850 0x7f325c004c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:58.183 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.167+0000 7f327affd700 1 --2- 172.21.15.47:0/2847382906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f325c003750 0x7f325c003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:58.183 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.167+0000 7f327a7fc700 1 -- 172.21.15.47:0/2847382906 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f327c05c850 msgr2=0x7f325c004c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:58.183 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.167+0000 7f327a7fc700 1 --2- 172.21.15.47:0/2847382906 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f327c05c850 0x7f325c004c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:58.183 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.167+0000 7f327a7fc700 1 -- 172.21.15.47:0/2847382906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f325c003750 msgr2=0x7f325c003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:58.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.167+0000 7f327a7fc700 1 --2- 172.21.15.47:0/2847382906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f325c003750 0x7f325c003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:58.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.167+0000 7f327a7fc700 1 -- 172.21.15.47:0/2847382906 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f325c003260 con 0x7f325c0040b0 2022-06-29T15:42:58.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.167+0000 7f327a7fc700 1 --2- 172.21.15.47:0/2847382906 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f325c0040b0 0x7f325c004580 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7f326c001820 tx=0x7f327c0ae9a0).ready entity=mon.0 client_cookie=f55c02ab81fa55e3 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:58.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.167+0000 7f3279ffb700 1 -- 172.21.15.47:0/2847382906 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f326c0080d0 con 0x7f325c0040b0 2022-06-29T15:42:58.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.167+0000 7f3279ffb700 1 -- 172.21.15.47:0/2847382906 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f326c008230 con 0x7f325c0040b0 2022-06-29T15:42:58.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.167+0000 7f3279ffb700 1 -- 172.21.15.47:0/2847382906 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f326c006440 con 0x7f325c0040b0 2022-06-29T15:42:58.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.171+0000 7f327bfff700 1 -- 172.21.15.47:0/2847382906 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f325c0040b0 msgr2=0x7f325c004580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:58.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.171+0000 7f327bfff700 1 --2- 172.21.15.47:0/2847382906 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f325c0040b0 0x7f325c004580 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7f326c001820 tx=0x7f327c0ae9a0).stop 2022-06-29T15:42:58.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.171+0000 7f327b7fe700 1 -- 172.21.15.47:0/2847382906 reap_dead start 2022-06-29T15:42:58.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.171+0000 7f327bfff700 1 -- 172.21.15.47:0/2847382906 shutdown_connections 2022-06-29T15:42:58.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.171+0000 7f327bfff700 1 -- 172.21.15.47:0/2847382906 >> 172.21.15.47:0/2847382906 conn(0x7f327c060e00 msgr2=0x7f327c0611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:58.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.171+0000 7f327bfff700 1 -- 172.21.15.47:0/2847382906 shutdown_connections 2022-06-29T15:42:58.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.171+0000 7f327bfff700 1 -- 172.21.15.47:0/2847382906 wait complete. 2022-06-29T15:42:58.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.171+0000 7f327bfff700 1 Processor -- start 2022-06-29T15:42:58.191 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.171+0000 7f327bfff700 1 -- start start 2022-06-29T15:42:58.191 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.175+0000 7f327bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f325c0040b0 0x7f327c0b2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:58.191 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.175+0000 7f327bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f327c0b3360 0x7f327c0b8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:58.191 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.175+0000 7f327bfff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f327c0b8800 0x7f327c12ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:58.191 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.175+0000 7f327bfff700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327c0b8c80 con 0x7f327c0b3360 2022-06-29T15:42:58.192 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.175+0000 7f327bfff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327c0af4f0 con 0x7f325c0040b0 2022-06-29T15:42:58.192 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.175+0000 7f327bfff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f327c0aee10 con 0x7f327c0b8800 2022-06-29T15:42:58.192 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.175+0000 7f327a7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f327c0b3360 0x7f327c0b8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:58.192 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.175+0000 7f327a7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f327c0b3360 0x7f327c0b8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46710/0 (socket says 172.21.15.47:46710) 2022-06-29T15:42:58.193 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.175+0000 7f327a7fc700 1 -- 172.21.15.47:0/2315714782 learned_addr learned my addr 172.21.15.47:0/2315714782 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:58.193 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.175+0000 7f327affd700 1 --2- 172.21.15.47:0/2315714782 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f325c0040b0 0x7f327c0b2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:58.193 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.175+0000 7f327b7fe700 1 --2- 172.21.15.47:0/2315714782 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f327c0b8800 0x7f327c12ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:58.193 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.175+0000 7f327a7fc700 1 -- 172.21.15.47:0/2315714782 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f327c0b8800 msgr2=0x7f327c12ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:58.193 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.175+0000 7f327a7fc700 1 --2- 172.21.15.47:0/2315714782 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f327c0b8800 0x7f327c12ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:58.194 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.175+0000 7f327a7fc700 1 -- 172.21.15.47:0/2315714782 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f325c0040b0 msgr2=0x7f327c0b2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:58.194 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.175+0000 7f327a7fc700 1 --2- 172.21.15.47:0/2315714782 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f325c0040b0 0x7f327c0b2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:58.194 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.175+0000 7f327a7fc700 1 -- 172.21.15.47:0/2315714782 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f327c0b39b0 con 0x7f327c0b3360 2022-06-29T15:42:58.194 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.175+0000 7f327a7fc700 1 --2- 172.21.15.47:0/2315714782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f327c0b3360 0x7f327c0b8310 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f327c0af630 tx=0x7f326c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:58.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.175+0000 7f3273fff700 1 -- 172.21.15.47:0/2315714782 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f326c023470 con 0x7f327c0b3360 2022-06-29T15:42:58.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.175+0000 7f3273fff700 1 -- 172.21.15.47:0/2315714782 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f326c0235d0 con 0x7f327c0b3360 2022-06-29T15:42:58.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.175+0000 7f327bfff700 1 -- 172.21.15.47:0/2315714782 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f327c0af040 con 0x7f327c0b3360 2022-06-29T15:42:58.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.175+0000 7f3273fff700 1 -- 172.21.15.47:0/2315714782 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f326c023930 con 0x7f327c0b3360 2022-06-29T15:42:58.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.175+0000 7f327bfff700 1 -- 172.21.15.47:0/2315714782 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f327c0afdb0 con 0x7f327c0b3360 2022-06-29T15:42:58.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.179+0000 7f327bfff700 1 -- 172.21.15.47:0/2315714782 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f327c0b37e0 con 0x7f327c0b3360 2022-06-29T15:42:58.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.179+0000 7f3273fff700 1 -- 172.21.15.47:0/2315714782 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f326c02b030 con 0x7f327c0b3360 2022-06-29T15:42:58.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.179+0000 7f3273fff700 1 --2- 172.21.15.47:0/2315714782 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3260053b30 0x7f3260055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:58.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.179+0000 7f3273fff700 1 -- 172.21.15.47:0/2315714782 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f326c01d040 con 0x7f327c0b3360 2022-06-29T15:42:58.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.179+0000 7f327affd700 1 --2- 172.21.15.47:0/2315714782 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3260053b30 0x7f3260055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:58.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.179+0000 7f327affd700 1 --2- 172.21.15.47:0/2315714782 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3260053b30 0x7f3260055ff0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f3264006000 tx=0x7f3264008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:58.198 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.183+0000 7f3273fff700 1 -- 172.21.15.47:0/2315714782 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f326c0445b0 con 0x7f327c0b3360 2022-06-29T15:42:58.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.307+0000 7f327bfff700 1 -- 172.21.15.47:0/2315714782 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f327c0b9b10 con 0x7f327c0b3360 2022-06-29T15:42:58.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.307+0000 7f3273fff700 1 -- 172.21.15.47:0/2315714782 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+10 (secure 0 0 0) 0x7f326c047bd0 con 0x7f327c0b3360 2022-06-29T15:42:58.372 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:42:58.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.307+0000 7f327bfff700 1 -- 172.21.15.47:0/2315714782 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3260053b30 msgr2=0x7f3260055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:58.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.307+0000 7f327bfff700 1 --2- 172.21.15.47:0/2315714782 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3260053b30 0x7f3260055ff0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f3264006000 tx=0x7f3264008040).stop 2022-06-29T15:42:58.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.307+0000 7f327bfff700 1 -- 172.21.15.47:0/2315714782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f327c0b3360 msgr2=0x7f327c0b8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:58.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.307+0000 7f327bfff700 1 --2- 172.21.15.47:0/2315714782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f327c0b3360 0x7f327c0b8310 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f327c0af630 tx=0x7f326c016000).stop 2022-06-29T15:42:58.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.307+0000 7f327bfff700 1 -- 172.21.15.47:0/2315714782 shutdown_connections 2022-06-29T15:42:58.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.307+0000 7f327bfff700 1 --2- 172.21.15.47:0/2315714782 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f327c0b8800 0x7f327c12ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:58.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.307+0000 7f327bfff700 1 --2- 172.21.15.47:0/2315714782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f327c0b3360 0x7f327c0b8310 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:58.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.307+0000 7f327bfff700 1 --2- 172.21.15.47:0/2315714782 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3260053b30 0x7f3260055ff0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:58.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.307+0000 7f327bfff700 1 --2- 172.21.15.47:0/2315714782 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f325c0040b0 0x7f327c0b2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:58.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.307+0000 7f327bfff700 1 -- 172.21.15.47:0/2315714782 >> 172.21.15.47:0/2315714782 conn(0x7f327c060e00 msgr2=0x7f327c146830 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:58.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.307+0000 7f327bfff700 1 -- 172.21.15.47:0/2315714782 shutdown_connections 2022-06-29T15:42:58.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.307+0000 7f327bfff700 1 -- 172.21.15.47:0/2315714782 wait complete. 2022-06-29T15:42:58.382 INFO:tasks.cephfs_test_runner:Starting test: test_drop_cache_command (tasks.cephfs.test_misc.TestCacheDrop) 2022-06-29T15:42:58.383 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command' 2022-06-29T15:42:58.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.483+0000 7fed36b6b700 1 Processor -- start 2022-06-29T15:42:58.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.483+0000 7fed36b6b700 1 -- start start 2022-06-29T15:42:58.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.483+0000 7fed36b6b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fed30060eb0 0x7fed300b1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:58.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.483+0000 7fed36b6b700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fed30145090 0x7fed300b2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:58.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.483+0000 7fed36b6b700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed300a9120 con 0x7fed300b5c60 2022-06-29T15:42:58.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.483+0000 7fed36b6b700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed300b0b80 con 0x7fed30060eb0 2022-06-29T15:42:58.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.483+0000 7fed36b6b700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed300a89e0 con 0x7fed30145090 2022-06-29T15:42:58.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.487+0000 7fed3636a700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fed30145090 0x7fed300b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:58.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.487+0000 7fed35368700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fed30060eb0 0x7fed300b1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:58.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.487+0000 7fed35b69700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fed300b5c60 0x7fed30144980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:39834/0 (socket says 172.21.15.47:39834) 2022-06-29T15:42:58.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.487+0000 7fed35b69700 1 -- 172.21.15.47:0/738177624 learned_addr learned my addr 172.21.15.47:0/738177624 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:58.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.487+0000 7fed34b67700 1 -- 172.21.15.47:0/738177624 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2568696509 0 0) 0x7fed300a9120 con 0x7fed300b5c60 2022-06-29T15:42:58.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.487+0000 7fed3636a700 1 -- 172.21.15.47:0/738177624 >> v2:172.21.15.64:3300/0 conn(0x7fed30060eb0 msgr2=0x7fed300b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:58.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.487+0000 7fed3636a700 1 --2- 172.21.15.47:0/738177624 >> v2:172.21.15.64:3300/0 conn(0x7fed30060eb0 0x7fed300b1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:58.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.487+0000 7fed3636a700 1 -- 172.21.15.47:0/738177624 >> v1:172.21.15.64:6789/0 conn(0x7fed300b5c60 legacy=0x7fed30144980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:58.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.487+0000 7fed3636a700 1 -- 172.21.15.47:0/738177624 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed300a8c80 con 0x7fed30145090 2022-06-29T15:42:58.497 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.487+0000 7fed3636a700 1 --2- 172.21.15.47:0/738177624 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fed30145090 0x7fed300b2320 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fed28002f70 tx=0x7fed28006df0).ready entity=mon.2 client_cookie=7c41f219a9f51138 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:58.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.487+0000 7fed34b67700 1 -- 172.21.15.47:0/738177624 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed2800d570 con 0x7fed30145090 2022-06-29T15:42:58.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.487+0000 7fed34b67700 1 -- 172.21.15.47:0/738177624 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed28000e90 con 0x7fed30145090 2022-06-29T15:42:58.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.487+0000 7fed34b67700 1 -- 172.21.15.47:0/738177624 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed2800d8c0 con 0x7fed30145090 2022-06-29T15:42:58.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.487+0000 7fed36b6b700 1 -- 172.21.15.47:0/738177624 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fed30145090 msgr2=0x7fed300b2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:58.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.487+0000 7fed36b6b700 1 --2- 172.21.15.47:0/738177624 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fed30145090 0x7fed300b2320 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fed28002f70 tx=0x7fed28006df0).stop 2022-06-29T15:42:58.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.487+0000 7fed36b6b700 1 -- 172.21.15.47:0/738177624 shutdown_connections 2022-06-29T15:42:58.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.487+0000 7fed36b6b700 1 --2- 172.21.15.47:0/738177624 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fed30145090 0x7fed300b2320 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:58.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.487+0000 7fed36b6b700 1 --2- 172.21.15.47:0/738177624 >> v2:172.21.15.64:3300/0 conn(0x7fed30060eb0 0x7fed300b1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:58.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.487+0000 7fed36b6b700 1 -- 172.21.15.47:0/738177624 >> 172.21.15.47:0/738177624 conn(0x7fed30062560 msgr2=0x7fed30062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:58.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.487+0000 7fed36b6b700 1 -- 172.21.15.47:0/738177624 shutdown_connections 2022-06-29T15:42:58.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.487+0000 7fed36b6b700 1 -- 172.21.15.47:0/738177624 wait complete. 2022-06-29T15:42:58.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.487+0000 7fed36b6b700 1 Processor -- start 2022-06-29T15:42:58.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.487+0000 7fed36b6b700 1 -- start start 2022-06-29T15:42:58.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.487+0000 7fed36b6b700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fed30060eb0 0x7fed3013ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:58.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.487+0000 7fed36b6b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed300b5c60 0x7fed300b4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:58.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.487+0000 7fed36b6b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed30145090 0x7fed3013ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:58.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.487+0000 7fed36b6b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3013a020 con 0x7fed300b5c60 2022-06-29T15:42:58.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.487+0000 7fed36b6b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed301398c0 con 0x7fed30145090 2022-06-29T15:42:58.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.487+0000 7fed36b6b700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed30138830 con 0x7fed30060eb0 2022-06-29T15:42:58.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.491+0000 7fed35368700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed300b5c60 0x7fed300b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:58.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.491+0000 7fed35368700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed300b5c60 0x7fed300b4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46726/0 (socket says 172.21.15.47:46726) 2022-06-29T15:42:58.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.491+0000 7fed35368700 1 -- 172.21.15.47:0/1902782536 learned_addr learned my addr 172.21.15.47:0/1902782536 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:58.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.491+0000 7fed35b69700 1 --2- 172.21.15.47:0/1902782536 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fed30060eb0 0x7fed3013ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:58.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.491+0000 7fed3636a700 1 --2- 172.21.15.47:0/1902782536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed30145090 0x7fed3013ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:58.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.491+0000 7fed35368700 1 -- 172.21.15.47:0/1902782536 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fed30060eb0 msgr2=0x7fed3013ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:58.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.491+0000 7fed35368700 1 --2- 172.21.15.47:0/1902782536 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fed30060eb0 0x7fed3013ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:58.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.491+0000 7fed35368700 1 -- 172.21.15.47:0/1902782536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed30145090 msgr2=0x7fed3013ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:58.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.491+0000 7fed35368700 1 --2- 172.21.15.47:0/1902782536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed30145090 0x7fed3013ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:58.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.491+0000 7fed35368700 1 -- 172.21.15.47:0/1902782536 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed2800b180 con 0x7fed300b5c60 2022-06-29T15:42:58.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.491+0000 7fed35368700 1 --2- 172.21.15.47:0/1902782536 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed300b5c60 0x7fed300b4590 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7fed2c00c000 tx=0x7fed2c0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:58.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.491+0000 7fed1effd700 1 -- 172.21.15.47:0/1902782536 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed2c012600 con 0x7fed300b5c60 2022-06-29T15:42:58.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.491+0000 7fed1effd700 1 -- 172.21.15.47:0/1902782536 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed2c012760 con 0x7fed300b5c60 2022-06-29T15:42:58.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.491+0000 7fed36b6b700 1 -- 172.21.15.47:0/1902782536 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed30159ab0 con 0x7fed300b5c60 2022-06-29T15:42:58.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.491+0000 7fed1effd700 1 -- 172.21.15.47:0/1902782536 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed2c012ac0 con 0x7fed300b5c60 2022-06-29T15:42:58.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.491+0000 7fed36b6b700 1 -- 172.21.15.47:0/1902782536 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed301393d0 con 0x7fed300b5c60 2022-06-29T15:42:58.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.491+0000 7fed1effd700 1 -- 172.21.15.47:0/1902782536 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fed2c022070 con 0x7fed300b5c60 2022-06-29T15:42:58.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.491+0000 7fed36b6b700 1 -- 172.21.15.47:0/1902782536 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed3013a190 con 0x7fed300b5c60 2022-06-29T15:42:58.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.491+0000 7fed1effd700 1 --2- 172.21.15.47:0/1902782536 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fed10053a70 0x7fed10055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:58.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.491+0000 7fed1effd700 1 -- 172.21.15.47:0/1902782536 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fed2c06c750 con 0x7fed300b5c60 2022-06-29T15:42:58.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.491+0000 7fed35b69700 1 --2- 172.21.15.47:0/1902782536 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fed10053a70 0x7fed10055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:58.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.495+0000 7fed35b69700 1 --2- 172.21.15.47:0/1902782536 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fed10053a70 0x7fed10055f30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fed2000d000 tx=0x7fed20007c10).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:58.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.495+0000 7fed1effd700 1 -- 172.21.15.47:0/1902782536 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fed2c043470 con 0x7fed300b5c60 2022-06-29T15:42:58.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:58.667+0000 7fed36b6b700 1 -- 172.21.15.47:0/1902782536 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command"]} v 0) v1 -- 0x7fed300573d0 con 0x7fed300b5c60 2022-06-29T15:42:59.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.151+0000 7fed1effd700 1 -- 172.21.15.47:0/1902782536 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command"]}]=0 v43) v1 ==== 142+0+0 (secure 0 0 0) 0x7fed2c046a90 con 0x7fed300b5c60 2022-06-29T15:42:59.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.155+0000 7fed36b6b700 1 -- 172.21.15.47:0/1902782536 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fed10053a70 msgr2=0x7fed10055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:59.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.155+0000 7fed36b6b700 1 --2- 172.21.15.47:0/1902782536 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fed10053a70 0x7fed10055f30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fed2000d000 tx=0x7fed20007c10).stop 2022-06-29T15:42:59.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.155+0000 7fed36b6b700 1 -- 172.21.15.47:0/1902782536 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed300b5c60 msgr2=0x7fed300b4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:59.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.155+0000 7fed36b6b700 1 --2- 172.21.15.47:0/1902782536 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed300b5c60 0x7fed300b4590 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7fed2c00c000 tx=0x7fed2c0091d0).stop 2022-06-29T15:42:59.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.155+0000 7fed36b6b700 1 -- 172.21.15.47:0/1902782536 shutdown_connections 2022-06-29T15:42:59.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.155+0000 7fed36b6b700 1 --2- 172.21.15.47:0/1902782536 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fed10053a70 0x7fed10055f30 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:59.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.155+0000 7fed36b6b700 1 --2- 172.21.15.47:0/1902782536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed30145090 0x7fed3013ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:59.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.155+0000 7fed36b6b700 1 --2- 172.21.15.47:0/1902782536 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed300b5c60 0x7fed300b4590 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:59.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.155+0000 7fed36b6b700 1 --2- 172.21.15.47:0/1902782536 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fed30060eb0 0x7fed3013ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:59.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.155+0000 7fed36b6b700 1 -- 172.21.15.47:0/1902782536 >> 172.21.15.47:0/1902782536 conn(0x7fed30062560 msgr2=0x7fed300a7240 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:59.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.155+0000 7fed36b6b700 1 -- 172.21.15.47:0/1902782536 shutdown_connections 2022-06-29T15:42:59.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.155+0000 7fed36b6b700 1 -- 172.21.15.47:0/1902782536 wait complete. 2022-06-29T15:42:59.179 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-06-29T15:42:59.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.283+0000 7f8605d1e700 1 Processor -- start 2022-06-29T15:42:59.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.283+0000 7f8605d1e700 1 -- start start 2022-06-29T15:42:59.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.283+0000 7f8605d1e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f86000b4300 0x7f86000612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:59.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.283+0000 7f8605d1e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f86001471a0 0x7f8600066f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:59.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.283+0000 7f8605d1e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8600067610 con 0x7f8600060ee0 2022-06-29T15:42:59.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.283+0000 7f8605d1e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86001467c0 con 0x7f86000b4300 2022-06-29T15:42:59.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.283+0000 7f8605d1e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8600146a60 con 0x7f86001471a0 2022-06-29T15:42:59.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.287+0000 7f85feffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f86000b4300 0x7f86000612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:59.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.287+0000 7f85feffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f86000b4300 0x7f86000612c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46732/0 (socket says 172.21.15.47:46732) 2022-06-29T15:42:59.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.287+0000 7f85feffd700 1 -- 172.21.15.47:0/4037157052 learned_addr learned my addr 172.21.15.47:0/4037157052 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:59.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.287+0000 7f85fffff700 1 --2- 172.21.15.47:0/4037157052 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f86001471a0 0x7f8600066f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:59.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.287+0000 7f85feffd700 1 -- 172.21.15.47:0/4037157052 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f86001471a0 msgr2=0x7f8600066f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:59.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.287+0000 7f85feffd700 1 --2- 172.21.15.47:0/4037157052 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f86001471a0 0x7f8600066f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:59.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.287+0000 7f85feffd700 1 -- 172.21.15.47:0/4037157052 >> v1:172.21.15.64:6789/0 conn(0x7f8600060ee0 legacy=0x7f86001430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:59.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.287+0000 7f85feffd700 1 -- 172.21.15.47:0/4037157052 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8600146d00 con 0x7f86000b4300 2022-06-29T15:42:59.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.287+0000 7f85feffd700 1 --2- 172.21.15.47:0/4037157052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86000b4300 0x7f86000612c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7f86000ae790 tx=0x7f85f4006eb0).ready entity=mon.0 client_cookie=8ef38895714c59e4 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:59.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.287+0000 7f85fe7fc700 1 -- 172.21.15.47:0/4037157052 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85f400d5c0 con 0x7f86000b4300 2022-06-29T15:42:59.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.287+0000 7f85fe7fc700 1 -- 172.21.15.47:0/4037157052 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f4000e90 con 0x7f86000b4300 2022-06-29T15:42:59.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.287+0000 7f85fe7fc700 1 -- 172.21.15.47:0/4037157052 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85f400d950 con 0x7f86000b4300 2022-06-29T15:42:59.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.287+0000 7f8605d1e700 1 -- 172.21.15.47:0/4037157052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86000b4300 msgr2=0x7f86000612c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:59.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.287+0000 7f8605d1e700 1 --2- 172.21.15.47:0/4037157052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86000b4300 0x7f86000612c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7f86000ae790 tx=0x7f85f4006eb0).stop 2022-06-29T15:42:59.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.287+0000 7f8605d1e700 1 -- 172.21.15.47:0/4037157052 shutdown_connections 2022-06-29T15:42:59.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.287+0000 7f8605d1e700 1 --2- 172.21.15.47:0/4037157052 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f86001471a0 0x7f8600066f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:59.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.287+0000 7f8605d1e700 1 --2- 172.21.15.47:0/4037157052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86000b4300 0x7f86000612c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:59.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.287+0000 7f8605d1e700 1 -- 172.21.15.47:0/4037157052 >> 172.21.15.47:0/4037157052 conn(0x7f860005c510 msgr2=0x7f860005f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:59.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.287+0000 7f8605d1e700 1 -- 172.21.15.47:0/4037157052 shutdown_connections 2022-06-29T15:42:59.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.287+0000 7f8605d1e700 1 -- 172.21.15.47:0/4037157052 wait complete. 2022-06-29T15:42:59.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.291+0000 7f8605d1e700 1 Processor -- start 2022-06-29T15:42:59.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.291+0000 7f8605d1e700 1 -- start start 2022-06-29T15:42:59.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.291+0000 7f8605d1e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8600060ee0 0x7f86001459a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:59.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.291+0000 7f8605d1e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86000b4300 0x7f860013f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:59.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.291+0000 7f8605d1e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f86001471a0 0x7f86001400b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:59.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.291+0000 7f8605d1e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86001130d0 con 0x7f86000b4300 2022-06-29T15:42:59.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.291+0000 7f8605d1e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f860013eb20 con 0x7f8600060ee0 2022-06-29T15:42:59.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.291+0000 7f8605d1e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86000b1380 con 0x7f86001471a0 2022-06-29T15:42:59.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.291+0000 7f85feffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86000b4300 0x7f860013f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:59.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.291+0000 7f85feffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86000b4300 0x7f860013f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46738/0 (socket says 172.21.15.47:46738) 2022-06-29T15:42:59.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.291+0000 7f85feffd700 1 -- 172.21.15.47:0/2576744920 learned_addr learned my addr 172.21.15.47:0/2576744920 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:59.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.291+0000 7f85fffff700 1 --2- 172.21.15.47:0/2576744920 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f86001471a0 0x7f86001400b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:59.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.291+0000 7f85ff7fe700 1 --2- 172.21.15.47:0/2576744920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8600060ee0 0x7f86001459a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:59.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.291+0000 7f85feffd700 1 -- 172.21.15.47:0/2576744920 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f86001471a0 msgr2=0x7f86001400b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:59.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.291+0000 7f85feffd700 1 --2- 172.21.15.47:0/2576744920 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f86001471a0 0x7f86001400b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:59.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.291+0000 7f85feffd700 1 -- 172.21.15.47:0/2576744920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8600060ee0 msgr2=0x7f86001459a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:59.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.291+0000 7f85feffd700 1 --2- 172.21.15.47:0/2576744920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8600060ee0 0x7f86001459a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:59.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.291+0000 7f85feffd700 1 -- 172.21.15.47:0/2576744920 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f400b1e0 con 0x7f86000b4300 2022-06-29T15:42:59.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.291+0000 7f85feffd700 1 --2- 172.21.15.47:0/2576744920 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86000b4300 0x7f860013f7f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f86000ae790 tx=0x7f85f4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:59.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.291+0000 7f85fcff9700 1 -- 172.21.15.47:0/2576744920 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85f4011760 con 0x7f86000b4300 2022-06-29T15:42:59.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.291+0000 7f85fcff9700 1 -- 172.21.15.47:0/2576744920 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f40118c0 con 0x7f86000b4300 2022-06-29T15:42:59.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.291+0000 7f8605d1e700 1 -- 172.21.15.47:0/2576744920 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86000b1620 con 0x7f86000b4300 2022-06-29T15:42:59.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.291+0000 7f85fcff9700 1 -- 172.21.15.47:0/2576744920 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85f4011c20 con 0x7f86000b4300 2022-06-29T15:42:59.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.291+0000 7f8605d1e700 1 -- 172.21.15.47:0/2576744920 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86001405a0 con 0x7f86000b4300 2022-06-29T15:42:59.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.295+0000 7f8605d1e700 1 -- 172.21.15.47:0/2576744920 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8600112e90 con 0x7f86000b4300 2022-06-29T15:42:59.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.295+0000 7f85fcff9700 1 -- 172.21.15.47:0/2576744920 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f85f400f070 con 0x7f86000b4300 2022-06-29T15:42:59.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.295+0000 7f85fcff9700 1 --2- 172.21.15.47:0/2576744920 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f85e0053aa0 0x7f85e0055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:59.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.295+0000 7f85fcff9700 1 -- 172.21.15.47:0/2576744920 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f85f406d860 con 0x7f86000b4300 2022-06-29T15:42:59.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.295+0000 7f85ff7fe700 1 --2- 172.21.15.47:0/2576744920 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f85e0053aa0 0x7f85e0055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:59.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.295+0000 7f85ff7fe700 1 --2- 172.21.15.47:0/2576744920 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f85e0053aa0 0x7f85e0055f60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f85f000d000 tx=0x7f85f0007b90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:59.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.303+0000 7f85fcff9700 1 -- 172.21.15.47:0/2576744920 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8600112e90 con 0x7f86000b4300 2022-06-29T15:42:59.432 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.423+0000 7f8605d1e700 1 -- 172.21.15.47:0/2576744920 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f8600112e90 con 0x7f86000b4300 2022-06-29T15:42:59.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.431+0000 7f85fcff9700 1 -- 172.21.15.47:0/2576744920 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v2)=0 v2) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8600112e90 con 0x7f86000b4300 2022-06-29T15:42:59.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.435+0000 7f8605d1e700 1 -- 172.21.15.47:0/2576744920 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f85e0053aa0 msgr2=0x7f85e0055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:59.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.435+0000 7f8605d1e700 1 --2- 172.21.15.47:0/2576744920 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f85e0053aa0 0x7f85e0055f60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f85f000d000 tx=0x7f85f0007b90).stop 2022-06-29T15:42:59.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.435+0000 7f8605d1e700 1 -- 172.21.15.47:0/2576744920 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86000b4300 msgr2=0x7f860013f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:59.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.435+0000 7f8605d1e700 1 --2- 172.21.15.47:0/2576744920 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86000b4300 0x7f860013f7f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f86000ae790 tx=0x7f85f4015000).stop 2022-06-29T15:42:59.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.435+0000 7f8605d1e700 1 -- 172.21.15.47:0/2576744920 shutdown_connections 2022-06-29T15:42:59.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.435+0000 7f8605d1e700 1 --2- 172.21.15.47:0/2576744920 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f86001471a0 0x7f86001400b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:59.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.435+0000 7f8605d1e700 1 --2- 172.21.15.47:0/2576744920 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86000b4300 0x7f860013f7f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:59.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.435+0000 7f8605d1e700 1 --2- 172.21.15.47:0/2576744920 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f85e0053aa0 0x7f85e0055f60 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:59.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.435+0000 7f8605d1e700 1 --2- 172.21.15.47:0/2576744920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8600060ee0 0x7f86001459a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:59.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.435+0000 7f8605d1e700 1 -- 172.21.15.47:0/2576744920 >> 172.21.15.47:0/2576744920 conn(0x7f860005c510 msgr2=0x7f86000b6110 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:59.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.435+0000 7f8605d1e700 1 -- 172.21.15.47:0/2576744920 shutdown_connections 2022-06-29T15:42:59.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.435+0000 7f8605d1e700 1 -- 172.21.15.47:0/2576744920 wait complete. 2022-06-29T15:42:59.455 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-06-29T15:42:59.456 DEBUG:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:42:59.474 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:42:59.497 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:42:59.518 DEBUG:teuthology.orchestra.run.smithi164:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:42:59.540 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-06-29T15:42:59.540 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:42:59.541 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:42:59.541 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:42:59.549 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2022-06-29T15:42:59.550 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-29T15:42:59.550 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:42:59.551 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:42:59.551 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:42:59.598 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2022-06-29T15:42:59.599 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-29T15:42:59.600 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:42:59.600 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:42:59.601 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:42:59.608 INFO:teuthology.orchestra.run.smithi099.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': Transport endpoint is not connected 2022-06-29T15:42:59.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T15:42:59.609 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.0 2022-06-29T15:42:59.610 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi099.front.sepia.ceph.com... 2022-06-29T15:42:59.610 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-06-29T15:42:59.611 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:42:59.637 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:42:59.638 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:42:59.639 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:42:59.685 INFO:teuthology.orchestra.run.smithi099.stdout:ext2/ext3 2022-06-29T15:42:59.686 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-06-29T15:42:59.687 INFO:teuthology.orchestra.run:waiting for 900 2022-06-29T15:42:59.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-29T15:42:59.688 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:42:59.704 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:42:59.705 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:42:59.705 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:42:59.714 INFO:teuthology.orchestra.run.smithi164.stdout:fuseblk 2022-06-29T15:42:59.715 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-06-29T15:42:59.716 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:42:59.716 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:42:59.717 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:42:59.724 INFO:teuthology.orchestra.run.smithi164.stdout:fuseblk 2022-06-29T15:42:59.726 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-06-29T15:42:59.726 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:42:59.727 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:42:59.727 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:42:59.734 INFO:teuthology.orchestra.run.smithi164.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': Transport endpoint is not connected 2022-06-29T15:42:59.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T15:42:59.736 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.1 2022-06-29T15:42:59.737 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi164.front.sepia.ceph.com... 2022-06-29T15:42:59.737 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-06-29T15:42:59.737 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:42:59.807 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:42:59.808 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:42:59.808 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:42:59.815 INFO:teuthology.orchestra.run.smithi164.stdout:ext2/ext3 2022-06-29T15:42:59.817 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-06-29T15:42:59.817 INFO:teuthology.orchestra.run:waiting for 900 2022-06-29T15:42:59.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-29T15:42:59.818 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:42:59.836 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-06-29T15:42:59.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.935+0000 7fc417cbe700 1 Processor -- start 2022-06-29T15:42:59.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.935+0000 7fc417cbe700 1 -- start start 2022-06-29T15:42:59.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.939+0000 7fc417cbe700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc41005f0b0 0x7fc4100b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:59.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.939+0000 7fc417cbe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc4100b1260 0x7fc41014b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:59.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.939+0000 7fc417cbe700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4100b0740 con 0x7fc4100b2120 2022-06-29T15:42:59.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.939+0000 7fc417cbe700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc410147460 con 0x7fc4100b1260 2022-06-29T15:42:59.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.939+0000 7fc417cbe700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4100b0970 con 0x7fc41005f0b0 2022-06-29T15:42:59.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.939+0000 7fc415a5a700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc4100b2120 0x7fc410147af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:60532/0 (socket says 172.21.15.47:60532) 2022-06-29T15:42:59.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.939+0000 7fc415a5a700 1 -- 172.21.15.47:0/3182650279 learned_addr learned my addr 172.21.15.47:0/3182650279 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:42:59.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.939+0000 7fc415259700 1 --2- 172.21.15.47:0/3182650279 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc41005f0b0 0x7fc4100b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:59.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.939+0000 7fc41625b700 1 --2- 172.21.15.47:0/3182650279 >> v2:172.21.15.64:3300/0 conn(0x7fc4100b1260 0x7fc41014b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:59.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.939+0000 7fc414a58700 1 -- 172.21.15.47:0/3182650279 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1124339508 0 0) 0x7fc4100b0740 con 0x7fc4100b2120 2022-06-29T15:42:59.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.939+0000 7fc414a58700 1 -- 172.21.15.47:0/3182650279 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3f8003260 con 0x7fc4100b2120 2022-06-29T15:42:59.950 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.939+0000 7fc415259700 1 -- 172.21.15.47:0/3182650279 >> v2:172.21.15.64:3300/0 conn(0x7fc4100b1260 msgr2=0x7fc41014b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:42:59.950 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.939+0000 7fc415259700 1 --2- 172.21.15.47:0/3182650279 >> v2:172.21.15.64:3300/0 conn(0x7fc4100b1260 0x7fc41014b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:59.950 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.939+0000 7fc415259700 1 -- 172.21.15.47:0/3182650279 >> v1:172.21.15.47:6789/0 conn(0x7fc4100b2120 legacy=0x7fc410147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:59.950 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.939+0000 7fc415259700 1 -- 172.21.15.47:0/3182650279 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4100b0c10 con 0x7fc41005f0b0 2022-06-29T15:42:59.951 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.939+0000 7fc415259700 1 --2- 172.21.15.47:0/3182650279 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc41005f0b0 0x7fc4100b2500 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fc40c002f70 tx=0x7fc40c006df0).ready entity=mon.2 client_cookie=65dc260b991c928d server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:59.951 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.939+0000 7fc414a58700 1 -- 172.21.15.47:0/3182650279 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc40c00d4a0 con 0x7fc41005f0b0 2022-06-29T15:42:59.951 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.939+0000 7fc414a58700 1 -- 172.21.15.47:0/3182650279 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc40c000e90 con 0x7fc41005f0b0 2022-06-29T15:42:59.951 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.939+0000 7fc414a58700 1 -- 172.21.15.47:0/3182650279 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc40c00d7f0 con 0x7fc41005f0b0 2022-06-29T15:42:59.951 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.939+0000 7fc417cbe700 1 -- 172.21.15.47:0/3182650279 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc41005f0b0 msgr2=0x7fc4100b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:59.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.939+0000 7fc417cbe700 1 --2- 172.21.15.47:0/3182650279 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc41005f0b0 0x7fc4100b2500 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fc40c002f70 tx=0x7fc40c006df0).stop 2022-06-29T15:42:59.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.939+0000 7fc417cbe700 1 -- 172.21.15.47:0/3182650279 shutdown_connections 2022-06-29T15:42:59.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.939+0000 7fc417cbe700 1 --2- 172.21.15.47:0/3182650279 >> v2:172.21.15.64:3300/0 conn(0x7fc4100b1260 0x7fc41014b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:59.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.939+0000 7fc417cbe700 1 --2- 172.21.15.47:0/3182650279 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc41005f0b0 0x7fc4100b2500 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:59.953 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.939+0000 7fc417cbe700 1 -- 172.21.15.47:0/3182650279 >> 172.21.15.47:0/3182650279 conn(0x7fc4100626f0 msgr2=0x7fc410062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:42:59.953 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.939+0000 7fc417cbe700 1 -- 172.21.15.47:0/3182650279 shutdown_connections 2022-06-29T15:42:59.953 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.939+0000 7fc417cbe700 1 -- 172.21.15.47:0/3182650279 wait complete. 2022-06-29T15:42:59.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.939+0000 7fc417cbe700 1 Processor -- start 2022-06-29T15:42:59.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.939+0000 7fc417cbe700 1 -- start start 2022-06-29T15:42:59.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.943+0000 7fc417cbe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4100b1260 0x7fc410149a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:59.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.943+0000 7fc417cbe700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc4100b2120 0x7fc410140920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:59.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.943+0000 7fc417cbe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4100ae560 0x7fc410140e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:59.958 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.943+0000 7fc417cbe700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4100af1b0 con 0x7fc4100ae560 2022-06-29T15:42:59.958 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.943+0000 7fc417cbe700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4100ad530 con 0x7fc4100b1260 2022-06-29T15:42:59.958 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.943+0000 7fc417cbe700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4100af3e0 con 0x7fc4100b2120 2022-06-29T15:42:59.958 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.943+0000 7fc41625b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4100ae560 0x7fc410140e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:59.959 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.943+0000 7fc415259700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc4100b2120 0x7fc410140920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:59.959 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.943+0000 7fc415a5a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4100b1260 0x7fc410149a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:59.959 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.943+0000 7fc41625b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4100ae560 0x7fc410140e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46754/0 (socket says 172.21.15.47:46754) 2022-06-29T15:42:59.959 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.943+0000 7fc41625b700 1 -- 172.21.15.47:0/1137654607 learned_addr learned my addr 172.21.15.47:0/1137654607 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:42:59.959 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.943+0000 7fc415259700 1 -- 172.21.15.47:0/1137654607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4100b1260 msgr2=0x7fc410149a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:59.960 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.943+0000 7fc415259700 1 --2- 172.21.15.47:0/1137654607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4100b1260 0x7fc410149a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:59.960 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.943+0000 7fc415259700 1 -- 172.21.15.47:0/1137654607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4100ae560 msgr2=0x7fc410140e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:42:59.960 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.943+0000 7fc415259700 1 --2- 172.21.15.47:0/1137654607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4100ae560 0x7fc410140e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:42:59.960 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.943+0000 7fc415259700 1 -- 172.21.15.47:0/1137654607 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc40c00b180 con 0x7fc4100b2120 2022-06-29T15:42:59.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.943+0000 7fc41625b700 1 --2- 172.21.15.47:0/1137654607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4100ae560 0x7fc410140e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:42:59.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.943+0000 7fc415259700 1 --2- 172.21.15.47:0/1137654607 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc4100b2120 0x7fc410140920 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fc40c002f70 tx=0x7fc40c014000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:59.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.943+0000 7fc3feffd700 1 -- 172.21.15.47:0/1137654607 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc40c012930 con 0x7fc4100b2120 2022-06-29T15:42:59.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.943+0000 7fc3feffd700 1 -- 172.21.15.47:0/1137654607 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc40c012a90 con 0x7fc4100b2120 2022-06-29T15:42:59.962 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.943+0000 7fc3feffd700 1 -- 172.21.15.47:0/1137654607 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc40c012df0 con 0x7fc4100b2120 2022-06-29T15:42:59.962 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.943+0000 7fc417cbe700 1 -- 172.21.15.47:0/1137654607 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4100af680 con 0x7fc4100b2120 2022-06-29T15:42:59.962 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.943+0000 7fc417cbe700 1 -- 172.21.15.47:0/1137654607 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4100aea50 con 0x7fc4100b2120 2022-06-29T15:42:59.962 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.943+0000 7fc417cbe700 1 -- 172.21.15.47:0/1137654607 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4100ad390 con 0x7fc4100b2120 2022-06-29T15:42:59.963 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.947+0000 7fc3feffd700 1 -- 172.21.15.47:0/1137654607 <== mon.2 v2:172.21.15.64:3301/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc40c01c070 con 0x7fc4100b2120 2022-06-29T15:42:59.963 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.947+0000 7fc3feffd700 1 --2- 172.21.15.47:0/1137654607 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc3f0053a70 0x7fc3f0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:42:59.963 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.947+0000 7fc3feffd700 1 -- 172.21.15.47:0/1137654607 <== mon.2 v2:172.21.15.64:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fc40c06cc40 con 0x7fc4100b2120 2022-06-29T15:42:59.963 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.947+0000 7fc415a5a700 1 --2- 172.21.15.47:0/1137654607 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc3f0053a70 0x7fc3f0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:42:59.963 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.947+0000 7fc415a5a700 1 --2- 172.21.15.47:0/1137654607 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc3f0053a70 0x7fc3f0055f30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7fc40000d000 tx=0x7fc400009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:42:59.964 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:42:59.955+0000 7fc3feffd700 1 -- 172.21.15.47:0/1137654607 <== mon.2 v2:172.21.15.64:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc4100ad390 con 0x7fc4100b2120 2022-06-29T15:43:00.091 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.083+0000 7fc417cbe700 1 -- 172.21.15.47:0/1137654607 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fc4100ad6a0 con 0x7fc4100b2120 2022-06-29T15:43:00.094 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.087+0000 7fc3feffd700 1 -- 172.21.15.47:0/1137654607 <== mon.2 v2:172.21.15.64:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v26) v1 ==== 81+0+34065 (secure 0 0 0) 0x7fc4100ad6a0 con 0x7fc4100b2120 2022-06-29T15:43:00.095 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:43:00.095 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2022-06-29T15:43:00.095 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 26, 2022-06-29T15:43:00.096 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "9a68894a-53ce-400e-838b-14eab91b834c", 2022-06-29T15:43:00.096 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2022-06-29T15:42:14.046361+0000", 2022-06-29T15:43:00.096 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2022-06-29T15:42:34.474438+0000", 2022-06-29T15:43:00.096 INFO:teuthology.orchestra.run.smithi047.stdout: "last_up_change": "2022-06-29T15:42:20.929789+0000", 2022-06-29T15:43:00.097 INFO:teuthology.orchestra.run.smithi047.stdout: "last_in_change": "2022-06-29T15:42:20.929789+0000", 2022-06-29T15:43:00.097 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-06-29T15:43:00.097 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_num": 5799936, 2022-06-29T15:43:00.097 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_set": [ 2022-06-29T15:43:00.098 INFO:teuthology.orchestra.run.smithi047.stdout: "pglog_hardlimit", 2022-06-29T15:43:00.098 INFO:teuthology.orchestra.run.smithi047.stdout: "purged_snapdirs", 2022-06-29T15:43:00.098 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_deletes", 2022-06-29T15:43:00.098 INFO:teuthology.orchestra.run.smithi047.stdout: "sortbitwise" 2022-06-29T15:43:00.099 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:43:00.099 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_version": 5, 2022-06-29T15:43:00.099 INFO:teuthology.orchestra.run.smithi047.stdout: "full_ratio": 0.94999998807907104, 2022-06-29T15:43:00.099 INFO:teuthology.orchestra.run.smithi047.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-06-29T15:43:00.099 INFO:teuthology.orchestra.run.smithi047.stdout: "nearfull_ratio": 0.85000002384185791, 2022-06-29T15:43:00.100 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_snapshot": "", 2022-06-29T15:43:00.100 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_max": 4, 2022-06-29T15:43:00.100 INFO:teuthology.orchestra.run.smithi047.stdout: "max_osd": 8, 2022-06-29T15:43:00.100 INFO:teuthology.orchestra.run.smithi047.stdout: "require_min_compat_client": "jewel", 2022-06-29T15:43:00.101 INFO:teuthology.orchestra.run.smithi047.stdout: "min_compat_client": "jewel", 2022-06-29T15:43:00.101 INFO:teuthology.orchestra.run.smithi047.stdout: "require_osd_release": "octopus", 2022-06-29T15:43:00.101 INFO:teuthology.orchestra.run.smithi047.stdout: "pools": [ 2022-06-29T15:43:00.101 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.101 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 1, 2022-06-29T15:43:00.102 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "device_health_metrics", 2022-06-29T15:43:00.102 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:42:16.971431+0000", 2022-06-29T15:43:00.102 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:43:00.102 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:43:00.103 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:43:00.103 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:43:00.103 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:43:00.103 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:43:00.104 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:43:00.104 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:43:00.104 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 1, 2022-06-29T15:43:00.104 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 1, 2022-06-29T15:43:00.104 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 1, 2022-06-29T15:43:00.105 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 1, 2022-06-29T15:43:00.105 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 1, 2022-06-29T15:43:00.105 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:43:00.105 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:43:00.106 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:43:00.106 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:43:00.106 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:43:00.106 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:43:00.106 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:43:00.107 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.107 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "16", 2022-06-29T15:43:00.107 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:43:00.107 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:43:00.108 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:43:00.108 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:43:00.108 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:43:00.108 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:43:00.109 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:43:00.109 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:43:00.109 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:43:00.109 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:43:00.109 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:43:00.110 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:43:00.110 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:43:00.110 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:43:00.110 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:43:00.111 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:43:00.111 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:43:00.111 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:43:00.111 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:43:00.112 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:43:00.112 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:43:00.112 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:43:00.112 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:43:00.112 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:43:00.113 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:43:00.113 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:43:00.113 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.113 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:43:00.113 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:43:00.114 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:43:00.114 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:43:00.114 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:43:00.114 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:43:00.115 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:43:00.115 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:43:00.115 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:43:00.115 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:43:00.116 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:43:00.116 INFO:teuthology.orchestra.run.smithi047.stdout: "options": { 2022-06-29T15:43:00.116 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_min": 1 2022-06-29T15:43:00.116 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.116 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:43:00.117 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_devicehealth": {} 2022-06-29T15:43:00.117 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:00.117 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.117 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.118 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 2, 2022-06-29T15:43:00.118 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "rbd", 2022-06-29T15:43:00.118 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:42:25.290884+0000", 2022-06-29T15:43:00.119 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:43:00.119 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:43:00.119 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:43:00.119 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:43:00.120 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:43:00.120 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:43:00.120 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:43:00.120 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:43:00.120 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2022-06-29T15:43:00.121 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2022-06-29T15:43:00.121 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2022-06-29T15:43:00.121 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2022-06-29T15:43:00.121 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2022-06-29T15:43:00.122 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:43:00.122 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:43:00.122 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:43:00.122 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:43:00.123 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:43:00.123 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:43:00.123 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:43:00.123 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.123 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "19", 2022-06-29T15:43:00.124 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:43:00.124 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:43:00.124 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:43:00.124 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:43:00.125 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:43:00.125 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:43:00.125 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:43:00.125 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:43:00.126 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:43:00.126 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:43:00.126 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:43:00.126 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:43:00.126 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:43:00.127 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:43:00.127 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:43:00.127 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:43:00.127 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:43:00.128 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:43:00.128 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:43:00.128 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:43:00.128 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:43:00.128 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:43:00.129 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:43:00.129 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:43:00.129 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:43:00.129 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:43:00.130 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.130 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:43:00.130 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:43:00.130 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:43:00.131 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:43:00.131 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:43:00.131 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:43:00.131 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:43:00.131 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:43:00.132 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:43:00.132 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:43:00.132 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:43:00.132 INFO:teuthology.orchestra.run.smithi047.stdout: "options": {}, 2022-06-29T15:43:00.133 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:43:00.133 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd": {} 2022-06-29T15:43:00.133 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:00.133 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.133 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.134 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 3, 2022-06-29T15:43:00.134 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "cephfs_metadata", 2022-06-29T15:43:00.134 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:42:28.400446+0000", 2022-06-29T15:43:00.134 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:43:00.135 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:43:00.135 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:43:00.135 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:43:00.135 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:43:00.136 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:43:00.136 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:43:00.136 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:43:00.136 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2022-06-29T15:43:00.136 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2022-06-29T15:43:00.137 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2022-06-29T15:43:00.137 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2022-06-29T15:43:00.137 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2022-06-29T15:43:00.137 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:43:00.138 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:43:00.138 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:43:00.138 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:43:00.138 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:43:00.139 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:43:00.139 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:43:00.139 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.139 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "25", 2022-06-29T15:43:00.139 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:43:00.140 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:43:00.140 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:43:00.140 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:43:00.140 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:43:00.140 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:43:00.141 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:43:00.141 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:43:00.141 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:43:00.141 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:43:00.142 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:43:00.142 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:43:00.142 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:43:00.142 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:43:00.143 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:43:00.143 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:43:00.143 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:43:00.143 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:43:00.143 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:43:00.144 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:43:00.144 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:43:00.144 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:43:00.144 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:43:00.145 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:43:00.145 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:43:00.145 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:43:00.145 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.145 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:43:00.146 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:43:00.146 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:43:00.146 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:43:00.146 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:43:00.147 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:43:00.147 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:43:00.147 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:43:00.147 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:43:00.148 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:43:00.148 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:43:00.148 INFO:teuthology.orchestra.run.smithi047.stdout: "options": { 2022-06-29T15:43:00.148 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_bias": 4, 2022-06-29T15:43:00.148 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_min": 16, 2022-06-29T15:43:00.149 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_priority": 5 2022-06-29T15:43:00.149 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.149 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:43:00.149 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs": { 2022-06-29T15:43:00.150 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": "cephfs" 2022-06-29T15:43:00.150 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:00.150 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:00.150 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.151 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.151 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 4, 2022-06-29T15:43:00.151 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "cephfs_data", 2022-06-29T15:43:00.151 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:42:31.167705+0000", 2022-06-29T15:43:00.151 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 5, 2022-06-29T15:43:00.152 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool,ec_overwrites", 2022-06-29T15:43:00.152 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 3, 2022-06-29T15:43:00.152 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 4, 2022-06-29T15:43:00.152 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 3, 2022-06-29T15:43:00.153 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 1, 2022-06-29T15:43:00.153 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:43:00.153 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:43:00.153 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2022-06-29T15:43:00.153 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2022-06-29T15:43:00.154 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2022-06-29T15:43:00.154 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2022-06-29T15:43:00.154 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2022-06-29T15:43:00.154 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:43:00.155 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:43:00.155 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:43:00.155 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:43:00.155 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:43:00.155 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:43:00.156 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:43:00.156 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.156 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "26", 2022-06-29T15:43:00.156 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:43:00.157 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:43:00.157 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:43:00.157 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:43:00.157 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:43:00.157 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:43:00.158 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:43:00.158 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:43:00.158 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:43:00.158 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:43:00.159 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:43:00.159 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:43:00.159 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:43:00.159 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:43:00.160 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:43:00.160 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:43:00.160 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:43:00.160 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:43:00.161 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:43:00.161 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:43:00.161 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:43:00.161 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:43:00.161 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:43:00.162 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "cephfs_data", 2022-06-29T15:43:00.162 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:43:00.162 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:43:00.162 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.163 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:43:00.163 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:43:00.163 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:43:00.163 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:43:00.163 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:43:00.164 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:43:00.164 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:43:00.164 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:43:00.164 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 8192, 2022-06-29T15:43:00.165 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:43:00.165 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:43:00.165 INFO:teuthology.orchestra.run.smithi047.stdout: "options": {}, 2022-06-29T15:43:00.165 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:43:00.165 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs": { 2022-06-29T15:43:00.166 INFO:teuthology.orchestra.run.smithi047.stdout: "data": "cephfs" 2022-06-29T15:43:00.166 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:00.166 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:00.166 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:00.167 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:43:00.167 INFO:teuthology.orchestra.run.smithi047.stdout: "osds": [ 2022-06-29T15:43:00.167 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.167 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2022-06-29T15:43:00.167 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "e23eb023-4706-4201-bf72-4bf19c912c5d", 2022-06-29T15:43:00.168 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:43:00.168 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:43:00.168 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:43:00.168 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:43:00.168 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:43:00.169 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:43:00.169 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:43:00.169 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 23, 2022-06-29T15:43:00.169 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:43:00.170 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:43:00.170 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:43:00.170 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:43:00.170 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.171 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:43:00.171 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6818", 2022-06-29T15:43:00.171 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:43:00.171 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.172 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.172 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:43:00.172 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6819", 2022-06-29T15:43:00.172 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:43:00.172 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:00.173 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:43:00.173 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.173 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:43:00.173 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:43:00.174 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.174 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:43:00.174 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6820", 2022-06-29T15:43:00.174 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:43:00.174 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.175 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.175 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:43:00.175 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6821", 2022-06-29T15:43:00.175 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:43:00.176 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:00.176 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:43:00.176 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.176 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:43:00.176 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:43:00.177 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.177 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:43:00.177 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6824", 2022-06-29T15:43:00.177 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:43:00.178 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.178 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.178 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:43:00.178 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6825", 2022-06-29T15:43:00.179 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:43:00.179 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:00.179 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:43:00.179 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.179 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:43:00.180 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:43:00.180 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.180 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:43:00.180 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6822", 2022-06-29T15:43:00.181 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:43:00.181 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.181 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.181 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:43:00.182 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6823", 2022-06-29T15:43:00.182 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:43:00.182 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:00.182 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:43:00.182 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.183 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6819/13096", 2022-06-29T15:43:00.183 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6821/13096", 2022-06-29T15:43:00.183 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6825/13096", 2022-06-29T15:43:00.183 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6823/13096", 2022-06-29T15:43:00.184 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:43:00.184 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:43:00.184 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:43:00.184 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:43:00.184 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.185 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.185 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2022-06-29T15:43:00.185 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "228bb08d-2d64-43d9-9560-bfaeff154452", 2022-06-29T15:43:00.185 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:43:00.186 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:43:00.186 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:43:00.186 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:43:00.186 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:43:00.187 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:43:00.187 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:43:00.187 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 23, 2022-06-29T15:43:00.188 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:43:00.188 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:43:00.188 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:43:00.188 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:43:00.188 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.189 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:43:00.189 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6826", 2022-06-29T15:43:00.189 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:43:00.189 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.190 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.190 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:43:00.190 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6827", 2022-06-29T15:43:00.190 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:43:00.191 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:00.191 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:43:00.191 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.191 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:43:00.191 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:43:00.192 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.192 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:43:00.192 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6828", 2022-06-29T15:43:00.192 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:43:00.193 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.193 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.193 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:43:00.193 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6829", 2022-06-29T15:43:00.193 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:43:00.194 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:00.194 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:43:00.194 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.194 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:43:00.195 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:43:00.195 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.195 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:43:00.196 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6832", 2022-06-29T15:43:00.196 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:43:00.196 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.196 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.197 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:43:00.197 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6833", 2022-06-29T15:43:00.197 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:43:00.197 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:00.198 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:43:00.198 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.198 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:43:00.198 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:43:00.199 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.199 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:43:00.199 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6830", 2022-06-29T15:43:00.199 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:43:00.199 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.200 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.200 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:43:00.200 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6831", 2022-06-29T15:43:00.200 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:43:00.200 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:00.201 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:43:00.201 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.201 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6827/13100", 2022-06-29T15:43:00.201 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6829/13100", 2022-06-29T15:43:00.202 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6833/13100", 2022-06-29T15:43:00.202 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6831/13100", 2022-06-29T15:43:00.202 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:43:00.202 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:43:00.203 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:43:00.203 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:43:00.203 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.203 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.203 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2022-06-29T15:43:00.204 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "797a5452-1510-4640-a111-c9c2fcf707eb", 2022-06-29T15:43:00.204 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:43:00.204 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:43:00.204 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:43:00.204 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:43:00.205 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:43:00.205 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:43:00.205 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:43:00.205 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 17, 2022-06-29T15:43:00.205 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:43:00.206 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:43:00.206 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:43:00.206 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:43:00.206 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.207 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:43:00.207 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6810", 2022-06-29T15:43:00.207 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:43:00.207 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.208 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.208 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:43:00.208 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6811", 2022-06-29T15:43:00.208 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:43:00.208 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:00.209 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:43:00.209 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.209 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:43:00.209 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:43:00.210 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.210 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:43:00.210 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6812", 2022-06-29T15:43:00.210 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:43:00.210 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.211 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.211 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:43:00.211 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6813", 2022-06-29T15:43:00.211 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:43:00.212 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:00.212 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:43:00.212 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.212 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:43:00.212 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:43:00.213 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.213 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:43:00.213 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6816", 2022-06-29T15:43:00.213 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:43:00.213 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.214 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.214 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:43:00.214 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6817", 2022-06-29T15:43:00.214 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:43:00.215 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:00.215 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:43:00.215 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.215 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:43:00.216 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:43:00.216 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.216 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:43:00.216 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6814", 2022-06-29T15:43:00.217 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:43:00.217 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.217 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.217 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:43:00.217 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6815", 2022-06-29T15:43:00.218 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:43:00.218 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:00.218 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:43:00.218 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.219 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6811/13098", 2022-06-29T15:43:00.219 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6813/13098", 2022-06-29T15:43:00.219 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6817/13098", 2022-06-29T15:43:00.219 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6815/13098", 2022-06-29T15:43:00.219 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:43:00.220 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:43:00.220 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:43:00.220 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:43:00.220 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.221 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.221 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2022-06-29T15:43:00.221 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "55b24fd7-bb71-4a01-b3fa-a09caa772a62", 2022-06-29T15:43:00.221 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:43:00.221 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:43:00.222 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:43:00.222 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:43:00.222 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:43:00.222 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:43:00.223 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:43:00.223 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 23, 2022-06-29T15:43:00.223 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:43:00.223 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:43:00.224 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:43:00.224 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:43:00.224 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.224 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:43:00.224 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6802", 2022-06-29T15:43:00.225 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:43:00.225 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.225 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.225 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:43:00.226 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6803", 2022-06-29T15:43:00.226 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:43:00.226 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:00.226 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:43:00.226 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.227 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:43:00.227 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:43:00.227 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.227 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:43:00.228 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6804", 2022-06-29T15:43:00.228 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:43:00.228 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.228 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.228 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:43:00.229 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6805", 2022-06-29T15:43:00.229 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:43:00.229 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:00.229 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:43:00.230 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.230 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:43:00.230 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:43:00.230 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.231 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:43:00.231 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6808", 2022-06-29T15:43:00.231 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:43:00.231 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.231 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.232 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:43:00.232 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6809", 2022-06-29T15:43:00.232 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:43:00.232 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:00.233 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:43:00.233 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.233 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:43:00.233 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:43:00.233 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.234 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:43:00.234 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6806", 2022-06-29T15:43:00.234 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:43:00.234 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.235 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.235 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:43:00.235 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6807", 2022-06-29T15:43:00.235 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:43:00.235 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:00.236 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:43:00.236 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.236 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6803/13097", 2022-06-29T15:43:00.236 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6805/13097", 2022-06-29T15:43:00.237 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6809/13097", 2022-06-29T15:43:00.237 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6807/13097", 2022-06-29T15:43:00.237 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:43:00.237 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:43:00.237 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:43:00.238 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:43:00.238 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.238 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.238 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2022-06-29T15:43:00.239 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3", 2022-06-29T15:43:00.239 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:43:00.239 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:43:00.239 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:43:00.239 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:43:00.240 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:43:00.240 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:43:00.240 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:43:00.240 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 23, 2022-06-29T15:43:00.241 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:43:00.241 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:43:00.241 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:43:00.241 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:43:00.241 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.242 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:43:00.242 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6816", 2022-06-29T15:43:00.242 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:43:00.242 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.243 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.243 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:43:00.243 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6817", 2022-06-29T15:43:00.243 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:43:00.243 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:00.244 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:43:00.244 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.244 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:43:00.244 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:43:00.245 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.245 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:43:00.245 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6818", 2022-06-29T15:43:00.245 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:43:00.245 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.246 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.246 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:43:00.246 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6819", 2022-06-29T15:43:00.246 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:43:00.247 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:00.247 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:43:00.247 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.247 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:43:00.247 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:43:00.248 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.248 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:43:00.248 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6822", 2022-06-29T15:43:00.248 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:43:00.249 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.249 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.249 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:43:00.249 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6823", 2022-06-29T15:43:00.249 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:43:00.250 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:00.250 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:43:00.250 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.250 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:43:00.251 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:43:00.251 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.251 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:43:00.251 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6820", 2022-06-29T15:43:00.251 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:43:00.252 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.252 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.252 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:43:00.252 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6821", 2022-06-29T15:43:00.252 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:43:00.253 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:00.253 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:43:00.253 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.253 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6817/13396", 2022-06-29T15:43:00.254 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6819/13396", 2022-06-29T15:43:00.254 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6823/13396", 2022-06-29T15:43:00.254 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6821/13396", 2022-06-29T15:43:00.254 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:43:00.254 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:43:00.255 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:43:00.255 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:43:00.255 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.255 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.256 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2022-06-29T15:43:00.256 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "f77beebe-6cca-4244-b2d4-677825ebf919", 2022-06-29T15:43:00.256 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:43:00.256 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:43:00.256 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:43:00.257 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:43:00.257 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:43:00.257 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:43:00.257 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:43:00.258 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 20, 2022-06-29T15:43:00.258 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:43:00.258 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:43:00.258 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:43:00.259 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:43:00.259 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.259 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:43:00.259 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6800", 2022-06-29T15:43:00.259 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:43:00.260 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.260 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.260 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:43:00.260 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6801", 2022-06-29T15:43:00.261 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:43:00.261 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:00.261 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:43:00.261 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.261 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:43:00.262 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:43:00.262 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.262 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:43:00.262 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6802", 2022-06-29T15:43:00.263 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:43:00.263 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.263 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.263 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:43:00.263 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6803", 2022-06-29T15:43:00.264 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:43:00.264 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:00.264 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:43:00.264 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.265 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:43:00.265 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:43:00.265 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.265 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:43:00.265 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6806", 2022-06-29T15:43:00.266 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:43:00.266 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.266 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.266 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:43:00.267 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6807", 2022-06-29T15:43:00.267 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:43:00.267 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:00.267 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:43:00.268 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.268 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:43:00.268 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:43:00.268 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.268 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:43:00.269 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6804", 2022-06-29T15:43:00.269 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:43:00.269 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.269 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.270 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:43:00.270 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6805", 2022-06-29T15:43:00.270 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:43:00.270 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:00.271 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:43:00.271 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.271 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6801/13393", 2022-06-29T15:43:00.271 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6803/13393", 2022-06-29T15:43:00.271 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6807/13393", 2022-06-29T15:43:00.272 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6805/13393", 2022-06-29T15:43:00.272 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:43:00.272 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:43:00.272 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:43:00.272 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:43:00.273 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.273 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.273 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2022-06-29T15:43:00.273 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "af0ca9b3-c02c-4910-834b-bf0b8dc6713c", 2022-06-29T15:43:00.274 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:43:00.274 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:43:00.274 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:43:00.274 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:43:00.275 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:43:00.275 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:43:00.275 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:43:00.276 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 23, 2022-06-29T15:43:00.276 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:43:00.276 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:43:00.276 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:43:00.276 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:43:00.277 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.277 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:43:00.277 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6824", 2022-06-29T15:43:00.277 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:43:00.278 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.278 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.278 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:43:00.278 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6825", 2022-06-29T15:43:00.279 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:43:00.279 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:00.279 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:43:00.279 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.279 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:43:00.280 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:43:00.280 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.280 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:43:00.280 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6826", 2022-06-29T15:43:00.281 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:43:00.281 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.281 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.281 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:43:00.281 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6827", 2022-06-29T15:43:00.282 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:43:00.282 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:00.282 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:43:00.282 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.283 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:43:00.283 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:43:00.283 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.283 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:43:00.284 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6830", 2022-06-29T15:43:00.284 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:43:00.284 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.284 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.285 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:43:00.285 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6831", 2022-06-29T15:43:00.285 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:43:00.285 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:00.286 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:43:00.286 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.286 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:43:00.286 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:43:00.287 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.287 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:43:00.287 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6828", 2022-06-29T15:43:00.287 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:43:00.287 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.288 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.288 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:43:00.288 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6829", 2022-06-29T15:43:00.288 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:43:00.289 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:00.289 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:43:00.289 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.289 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6825/13395", 2022-06-29T15:43:00.289 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6827/13395", 2022-06-29T15:43:00.290 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6831/13395", 2022-06-29T15:43:00.290 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6829/13395", 2022-06-29T15:43:00.290 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:43:00.290 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:43:00.291 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:43:00.291 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:43:00.291 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.291 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.291 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2022-06-29T15:43:00.292 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "f19c3ca6-df88-4bde-a150-b51a53f092e1", 2022-06-29T15:43:00.292 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:43:00.292 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:43:00.293 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:43:00.293 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:43:00.293 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:43:00.293 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:43:00.294 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:43:00.294 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 17, 2022-06-29T15:43:00.294 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:43:00.294 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:43:00.295 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:43:00.295 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:43:00.295 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.295 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:43:00.295 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6808", 2022-06-29T15:43:00.296 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:43:00.296 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.296 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.296 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:43:00.297 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6809", 2022-06-29T15:43:00.297 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:43:00.297 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:00.297 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:43:00.297 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.298 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:43:00.298 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:43:00.298 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.298 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:43:00.299 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6810", 2022-06-29T15:43:00.299 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:43:00.299 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.299 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.300 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:43:00.300 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6811", 2022-06-29T15:43:00.300 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:43:00.300 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:00.300 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:43:00.301 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.301 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:43:00.301 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:43:00.301 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.302 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:43:00.302 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6814", 2022-06-29T15:43:00.302 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:43:00.302 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.303 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.303 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:43:00.303 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6815", 2022-06-29T15:43:00.303 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:43:00.304 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:00.304 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:43:00.304 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.304 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:43:00.304 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:43:00.305 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.305 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:43:00.305 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6812", 2022-06-29T15:43:00.305 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:43:00.306 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.306 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.306 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:43:00.306 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6813", 2022-06-29T15:43:00.306 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:43:00.307 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:00.307 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:43:00.307 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.307 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6809/13394", 2022-06-29T15:43:00.308 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6811/13394", 2022-06-29T15:43:00.308 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6815/13394", 2022-06-29T15:43:00.308 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6813/13394", 2022-06-29T15:43:00.308 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:43:00.308 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:43:00.309 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:43:00.309 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:43:00.309 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:00.309 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:43:00.310 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_xinfo": [ 2022-06-29T15:43:00.310 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.310 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2022-06-29T15:43:00.310 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:43:00.311 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:43:00.311 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:43:00.311 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:43:00.311 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:43:00.311 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.720513+0000", 2022-06-29T15:43:00.312 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:43:00.312 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.312 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.312 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2022-06-29T15:43:00.313 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:43:00.313 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:43:00.313 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:43:00.313 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:43:00.313 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:43:00.314 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.772612+0000", 2022-06-29T15:43:00.314 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:43:00.314 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.314 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.315 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2022-06-29T15:43:00.315 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:43:00.315 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:43:00.315 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:43:00.315 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:43:00.316 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:43:00.316 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.450223+0000", 2022-06-29T15:43:00.316 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:43:00.316 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.317 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.317 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2022-06-29T15:43:00.317 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:43:00.317 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:43:00.318 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:43:00.318 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:43:00.318 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:43:00.318 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.477856+0000", 2022-06-29T15:43:00.319 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:43:00.319 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.319 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.319 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2022-06-29T15:43:00.319 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:43:00.320 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:43:00.320 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:43:00.320 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:43:00.320 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:43:00.321 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.539485+0000", 2022-06-29T15:43:00.321 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:43:00.321 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.321 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.321 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2022-06-29T15:43:00.322 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:43:00.322 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:43:00.322 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:43:00.322 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:43:00.323 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:43:00.323 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.487040+0000", 2022-06-29T15:43:00.323 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:43:00.323 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.323 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.324 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2022-06-29T15:43:00.324 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:43:00.324 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:43:00.324 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:43:00.325 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:43:00.325 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:43:00.325 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.748685+0000", 2022-06-29T15:43:00.325 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:43:00.325 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.326 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:00.326 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2022-06-29T15:43:00.326 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:43:00.326 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:43:00.327 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:43:00.327 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:43:00.327 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:43:00.327 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.459516+0000", 2022-06-29T15:43:00.328 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:43:00.328 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:00.328 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:43:00.328 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap": [], 2022-06-29T15:43:00.328 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap_items": [], 2022-06-29T15:43:00.329 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_temp": [], 2022-06-29T15:43:00.329 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_temp": [], 2022-06-29T15:43:00.329 INFO:teuthology.orchestra.run.smithi047.stdout: "blacklist": {}, 2022-06-29T15:43:00.329 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profiles": { 2022-06-29T15:43:00.330 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs_data": { 2022-06-29T15:43:00.330 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-device-class": "", 2022-06-29T15:43:00.330 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2022-06-29T15:43:00.330 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-root": "default", 2022-06-29T15:43:00.330 INFO:teuthology.orchestra.run.smithi047.stdout: "jerasure-per-chunk-alignment": "false", 2022-06-29T15:43:00.331 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2022-06-29T15:43:00.331 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "2", 2022-06-29T15:43:00.331 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2022-06-29T15:43:00.331 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2022-06-29T15:43:00.332 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van", 2022-06-29T15:43:00.332 INFO:teuthology.orchestra.run.smithi047.stdout: "w": "8" 2022-06-29T15:43:00.332 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.332 INFO:teuthology.orchestra.run.smithi047.stdout: "default": { 2022-06-29T15:43:00.332 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2022-06-29T15:43:00.333 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2022-06-29T15:43:00.333 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "1", 2022-06-29T15:43:00.333 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2022-06-29T15:43:00.333 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2022-06-29T15:43:00.334 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van" 2022-06-29T15:43:00.334 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:00.334 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:00.334 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps_queue": [], 2022-06-29T15:43:00.335 INFO:teuthology.orchestra.run.smithi047.stdout: "new_removed_snaps": [], 2022-06-29T15:43:00.335 INFO:teuthology.orchestra.run.smithi047.stdout: "new_purged_snaps": [], 2022-06-29T15:43:00.335 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_node_flags": {}, 2022-06-29T15:43:00.335 INFO:teuthology.orchestra.run.smithi047.stdout: "device_class_flags": {} 2022-06-29T15:43:00.335 INFO:teuthology.orchestra.run.smithi047.stdout:} 2022-06-29T15:43:00.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.091+0000 7fc417cbe700 1 -- 172.21.15.47:0/1137654607 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc3f0053a70 msgr2=0x7fc3f0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:00.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.091+0000 7fc417cbe700 1 --2- 172.21.15.47:0/1137654607 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc3f0053a70 0x7fc3f0055f30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7fc40000d000 tx=0x7fc400009040).stop 2022-06-29T15:43:00.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.091+0000 7fc417cbe700 1 -- 172.21.15.47:0/1137654607 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc4100b2120 msgr2=0x7fc410140920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:00.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.091+0000 7fc417cbe700 1 --2- 172.21.15.47:0/1137654607 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc4100b2120 0x7fc410140920 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fc40c002f70 tx=0x7fc40c014000).stop 2022-06-29T15:43:00.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.091+0000 7fc417cbe700 1 -- 172.21.15.47:0/1137654607 shutdown_connections 2022-06-29T15:43:00.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.091+0000 7fc417cbe700 1 --2- 172.21.15.47:0/1137654607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4100ae560 0x7fc410140e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:00.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.091+0000 7fc417cbe700 1 --2- 172.21.15.47:0/1137654607 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc4100b2120 0x7fc410140920 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:00.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.091+0000 7fc417cbe700 1 --2- 172.21.15.47:0/1137654607 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc3f0053a70 0x7fc3f0055f30 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:00.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.091+0000 7fc417cbe700 1 --2- 172.21.15.47:0/1137654607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4100b1260 0x7fc410149a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:00.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.091+0000 7fc417cbe700 1 -- 172.21.15.47:0/1137654607 >> 172.21.15.47:0/1137654607 conn(0x7fc4100626f0 msgr2=0x7fc410146530 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:00.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.091+0000 7fc417cbe700 1 -- 172.21.15.47:0/1137654607 shutdown_connections 2022-06-29T15:43:00.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.091+0000 7fc417cbe700 1 -- 172.21.15.47:0/1137654607 wait complete. 2022-06-29T15:43:00.341 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:43:00.450 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.443+0000 7fd2637cd700 1 Processor -- start 2022-06-29T15:43:00.451 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.443+0000 7fd2637cd700 1 -- start start 2022-06-29T15:43:00.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.443+0000 7fd2637cd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd25c061600 0x7fd25c061050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:00.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.443+0000 7fd2637cd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd25c0601a0 0x7fd25c140520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:00.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.443+0000 7fd2637cd700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd25c0b2170 con 0x7fd25c0b2a90 2022-06-29T15:43:00.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.443+0000 7fd2637cd700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd25c0b2e70 con 0x7fd25c0601a0 2022-06-29T15:43:00.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.443+0000 7fd2637cd700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd25c0606b0 con 0x7fd25c061600 2022-06-29T15:43:00.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.443+0000 7fd261d6a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd25c0601a0 0x7fd25c140520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:00.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.443+0000 7fd261d6a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd25c0601a0 0x7fd25c140520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46762/0 (socket says 172.21.15.47:46762) 2022-06-29T15:43:00.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.443+0000 7fd261d6a700 1 -- 172.21.15.47:0/228166007 learned_addr learned my addr 172.21.15.47:0/228166007 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:00.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.443+0000 7fd261569700 1 --1- 172.21.15.47:0/228166007 >> v1:172.21.15.47:6789/0 conn(0x7fd25c0b2a90 0x7fd25c146b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:60546/0 (socket says 172.21.15.47:60546) 2022-06-29T15:43:00.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.443+0000 7fd260d68700 1 --2- 172.21.15.47:0/228166007 >> v2:172.21.15.64:3300/0 conn(0x7fd25c061600 0x7fd25c061050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:00.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.443+0000 7fd261d6a700 1 -- 172.21.15.47:0/228166007 >> v2:172.21.15.64:3300/0 conn(0x7fd25c061600 msgr2=0x7fd25c061050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:43:00.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.443+0000 7fd261d6a700 1 --2- 172.21.15.47:0/228166007 >> v2:172.21.15.64:3300/0 conn(0x7fd25c061600 0x7fd25c061050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:00.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.443+0000 7fd261d6a700 1 -- 172.21.15.47:0/228166007 >> v1:172.21.15.47:6789/0 conn(0x7fd25c0b2a90 legacy=0x7fd25c146b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:00.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.443+0000 7fd261d6a700 1 -- 172.21.15.47:0/228166007 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd25c0a9770 con 0x7fd25c0601a0 2022-06-29T15:43:00.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.443+0000 7fd260d68700 1 --2- 172.21.15.47:0/228166007 >> v2:172.21.15.64:3300/0 conn(0x7fd25c061600 0x7fd25c061050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:43:00.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.443+0000 7fd261d6a700 1 --2- 172.21.15.47:0/228166007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd25c0601a0 0x7fd25c140520 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7fd2540032f0 tx=0x7fd254009290).ready entity=mon.0 client_cookie=3a52b6bd2f6c8a67 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:00.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.447+0000 7fd253fff700 1 -- 172.21.15.47:0/228166007 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd25400b870 con 0x7fd25c0601a0 2022-06-29T15:43:00.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.447+0000 7fd253fff700 1 -- 172.21.15.47:0/228166007 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd25400b9d0 con 0x7fd25c0601a0 2022-06-29T15:43:00.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.447+0000 7fd253fff700 1 -- 172.21.15.47:0/228166007 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd25400bb40 con 0x7fd25c0601a0 2022-06-29T15:43:00.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.447+0000 7fd2637cd700 1 -- 172.21.15.47:0/228166007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd25c0601a0 msgr2=0x7fd25c140520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:00.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.447+0000 7fd2637cd700 1 --2- 172.21.15.47:0/228166007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd25c0601a0 0x7fd25c140520 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7fd2540032f0 tx=0x7fd254009290).stop 2022-06-29T15:43:00.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.447+0000 7fd2637cd700 1 -- 172.21.15.47:0/228166007 shutdown_connections 2022-06-29T15:43:00.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.447+0000 7fd2637cd700 1 --2- 172.21.15.47:0/228166007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd25c0601a0 0x7fd25c140520 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:00.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.447+0000 7fd2637cd700 1 --2- 172.21.15.47:0/228166007 >> v2:172.21.15.64:3300/0 conn(0x7fd25c061600 0x7fd25c061050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:00.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.447+0000 7fd2637cd700 1 -- 172.21.15.47:0/228166007 >> 172.21.15.47:0/228166007 conn(0x7fd25c05f0b0 msgr2=0x7fd25c05f490 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:00.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.447+0000 7fd2637cd700 1 -- 172.21.15.47:0/228166007 shutdown_connections 2022-06-29T15:43:00.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.447+0000 7fd2637cd700 1 -- 172.21.15.47:0/228166007 wait complete. 2022-06-29T15:43:00.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.447+0000 7fd2637cd700 1 Processor -- start 2022-06-29T15:43:00.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.447+0000 7fd2637cd700 1 -- start start 2022-06-29T15:43:00.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.447+0000 7fd2637cd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd25c0601a0 0x7fd25c0b63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:00.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.447+0000 7fd2637cd700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd25c061600 0x7fd25c143990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:00.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.447+0000 7fd2637cd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd25c0b2a90 0x7fd25c143e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:00.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.447+0000 7fd2637cd700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c152650 con 0x7fd25c0601a0 2022-06-29T15:43:00.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.447+0000 7fd2637cd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c099f10 con 0x7fd25c0b2a90 2022-06-29T15:43:00.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.447+0000 7fd2637cd700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c099c70 con 0x7fd25c061600 2022-06-29T15:43:00.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.447+0000 7fd261569700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd25c0601a0 0x7fd25c0b63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:00.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.451+0000 7fd261569700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd25c0601a0 0x7fd25c0b63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46764/0 (socket says 172.21.15.47:46764) 2022-06-29T15:43:00.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.451+0000 7fd261569700 1 -- 172.21.15.47:0/1025603588 learned_addr learned my addr 172.21.15.47:0/1025603588 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:00.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.451+0000 7fd261d6a700 1 --2- 172.21.15.47:0/1025603588 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd25c0b2a90 0x7fd25c143e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:00.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.451+0000 7fd260d68700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd25c061600 0x7fd25c143990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:00.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.451+0000 7fd261569700 1 -- 172.21.15.47:0/1025603588 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd25c061600 msgr2=0x7fd25c143990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:00.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.451+0000 7fd261569700 1 --2- 172.21.15.47:0/1025603588 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd25c061600 0x7fd25c143990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:00.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.451+0000 7fd261569700 1 -- 172.21.15.47:0/1025603588 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd25c0b2a90 msgr2=0x7fd25c143e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:00.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.451+0000 7fd261569700 1 --2- 172.21.15.47:0/1025603588 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd25c0b2a90 0x7fd25c143e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:00.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.451+0000 7fd261569700 1 -- 172.21.15.47:0/1025603588 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd254009530 con 0x7fd25c0601a0 2022-06-29T15:43:00.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.451+0000 7fd261d6a700 1 --2- 172.21.15.47:0/1025603588 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd25c0b2a90 0x7fd25c143e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:43:00.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.451+0000 7fd261569700 1 --2- 172.21.15.47:0/1025603588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd25c0601a0 0x7fd25c0b63d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7fd25c09a1a0 tx=0x7fd24c007c60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:00.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.451+0000 7fd2527fc700 1 -- 172.21.15.47:0/1025603588 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd24c0070d0 con 0x7fd25c0601a0 2022-06-29T15:43:00.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.451+0000 7fd2527fc700 1 -- 172.21.15.47:0/1025603588 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd24c007230 con 0x7fd25c0601a0 2022-06-29T15:43:00.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.451+0000 7fd2637cd700 1 -- 172.21.15.47:0/1025603588 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd25c09a280 con 0x7fd25c0601a0 2022-06-29T15:43:00.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.451+0000 7fd2527fc700 1 -- 172.21.15.47:0/1025603588 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd24c005550 con 0x7fd25c0601a0 2022-06-29T15:43:00.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.451+0000 7fd2637cd700 1 -- 172.21.15.47:0/1025603588 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd25c099860 con 0x7fd25c0601a0 2022-06-29T15:43:00.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.451+0000 7fd2637cd700 1 -- 172.21.15.47:0/1025603588 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd238003410 con 0x7fd25c0601a0 2022-06-29T15:43:00.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.455+0000 7fd2527fc700 1 -- 172.21.15.47:0/1025603588 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd24c010070 con 0x7fd25c0601a0 2022-06-29T15:43:00.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.455+0000 7fd2527fc700 1 --2- 172.21.15.47:0/1025603588 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd244053980 0x7fd244055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:00.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.455+0000 7fd2527fc700 1 -- 172.21.15.47:0/1025603588 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fd24c06b390 con 0x7fd25c0601a0 2022-06-29T15:43:00.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.455+0000 7fd260d68700 1 --2- 172.21.15.47:0/1025603588 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd244053980 0x7fd244055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:00.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.455+0000 7fd260d68700 1 --2- 172.21.15.47:0/1025603588 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd244053980 0x7fd244055e40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7fd25800c000 tx=0x7fd258008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:00.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.463+0000 7fd2527fc700 1 -- 172.21.15.47:0/1025603588 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd24c042150 con 0x7fd25c0601a0 2022-06-29T15:43:00.632 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.623+0000 7fd2637cd700 1 -- 172.21.15.47:0/1025603588 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd238003c40 con 0x7fd25c0601a0 2022-06-29T15:43:00.633 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.627+0000 7fd2527fc700 1 -- 172.21.15.47:0/1025603588 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+2483 (secure 0 0 0) 0x7fd24c045770 con 0x7fd25c0601a0 2022-06-29T15:43:00.633 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:43:00.634 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/438289599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":438289599},{"type":"v1","addr":"172.21.15.47:6835","nonce":438289599}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.64:6833/1472525731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1472525731},{"type":"v1","addr":"172.21.15.64:6833","nonce":1472525731}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":4}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:42:33.471491+0000","modified":"2022-06-29T15:42:35.146757+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":54128},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54128":{"gid":54128,"name":"a","rank":0,"incarnation":5,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6837/354581778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":354581778},{"type":"v1","addr":"172.21.15.47:6837","nonce":354581778}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2022-06-29T15:43:00.635 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.627+0000 7fd2637cd700 1 -- 172.21.15.47:0/1025603588 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd244053980 msgr2=0x7fd244055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:00.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.627+0000 7fd2637cd700 1 --2- 172.21.15.47:0/1025603588 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd244053980 0x7fd244055e40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7fd25800c000 tx=0x7fd258008040).stop 2022-06-29T15:43:00.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.627+0000 7fd2637cd700 1 -- 172.21.15.47:0/1025603588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd25c0601a0 msgr2=0x7fd25c0b63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:00.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.627+0000 7fd2637cd700 1 --2- 172.21.15.47:0/1025603588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd25c0601a0 0x7fd25c0b63d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7fd25c09a1a0 tx=0x7fd24c007c60).stop 2022-06-29T15:43:00.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.627+0000 7fd2637cd700 1 -- 172.21.15.47:0/1025603588 shutdown_connections 2022-06-29T15:43:00.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.627+0000 7fd2637cd700 1 --2- 172.21.15.47:0/1025603588 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd244053980 0x7fd244055e40 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:00.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.627+0000 7fd2637cd700 1 --2- 172.21.15.47:0/1025603588 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd25c0b2a90 0x7fd25c143e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:00.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.627+0000 7fd2637cd700 1 --2- 172.21.15.47:0/1025603588 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd25c061600 0x7fd25c143990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:00.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.627+0000 7fd2637cd700 1 --2- 172.21.15.47:0/1025603588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd25c0601a0 0x7fd25c0b63d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:00.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.627+0000 7fd2637cd700 1 -- 172.21.15.47:0/1025603588 >> 172.21.15.47:0/1025603588 conn(0x7fd25c05f0b0 msgr2=0x7fd25c147430 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:00.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.627+0000 7fd2637cd700 1 -- 172.21.15.47:0/1025603588 shutdown_connections 2022-06-29T15:43:00.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.627+0000 7fd2637cd700 1 -- 172.21.15.47:0/1025603588 wait complete. 2022-06-29T15:43:00.639 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 7 2022-06-29T15:43:00.647 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-06-29T15:43:00.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.751+0000 7fe771161700 1 Processor -- start 2022-06-29T15:43:00.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.751+0000 7fe771161700 1 -- start start 2022-06-29T15:43:00.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.751+0000 7fe771161700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe76c145090 0x7fe76c0b2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:00.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.751+0000 7fe771161700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe76c0a9120 con 0x7fe76c060eb0 2022-06-29T15:43:00.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.751+0000 7fe771161700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe76c0b0b80 con 0x7fe76c0b5c60 2022-06-29T15:43:00.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.751+0000 7fe771161700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe76c0a89e0 con 0x7fe76c145090 2022-06-29T15:43:00.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.751+0000 7fe76b7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe76c060eb0 0x7fe76c0b1e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:60560/0 (socket says 172.21.15.47:60560) 2022-06-29T15:43:00.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.751+0000 7fe770960700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe76c145090 0x7fe76c0b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:00.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.751+0000 7fe76b7fe700 1 -- 172.21.15.47:0/691675192 learned_addr learned my addr 172.21.15.47:0/691675192 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:43:00.762 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.751+0000 7fe770960700 1 -- 172.21.15.47:0/691675192 >> v1:172.21.15.64:6789/0 conn(0x7fe76c0b5c60 legacy=0x7fe76c144980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:00.762 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.751+0000 7fe770960700 1 -- 172.21.15.47:0/691675192 >> v1:172.21.15.47:6789/0 conn(0x7fe76c060eb0 legacy=0x7fe76c0b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:00.763 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.751+0000 7fe770960700 1 -- 172.21.15.47:0/691675192 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe76c0a8c80 con 0x7fe76c145090 2022-06-29T15:43:00.763 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.751+0000 7fe770960700 1 --2- 172.21.15.47:0/691675192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe76c145090 0x7fe76c0b2540 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7fe754002f70 tx=0x7fe754006df0).ready entity=mon.0 client_cookie=fe9353367a2ec885 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:00.763 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.751+0000 7fe76affd700 1 -- 172.21.15.47:0/691675192 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe75400d610 con 0x7fe76c145090 2022-06-29T15:43:00.763 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.751+0000 7fe76affd700 1 -- 172.21.15.47:0/691675192 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe754000e90 con 0x7fe76c145090 2022-06-29T15:43:00.763 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.751+0000 7fe76affd700 1 -- 172.21.15.47:0/691675192 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe75400d960 con 0x7fe76c145090 2022-06-29T15:43:00.764 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.755+0000 7fe771161700 1 -- 172.21.15.47:0/691675192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe76c145090 msgr2=0x7fe76c0b2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:00.764 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.755+0000 7fe771161700 1 --2- 172.21.15.47:0/691675192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe76c145090 0x7fe76c0b2540 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7fe754002f70 tx=0x7fe754006df0).stop 2022-06-29T15:43:00.764 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.755+0000 7fe771161700 1 -- 172.21.15.47:0/691675192 shutdown_connections 2022-06-29T15:43:00.764 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.755+0000 7fe771161700 1 --2- 172.21.15.47:0/691675192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe76c145090 0x7fe76c0b2540 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:00.765 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.755+0000 7fe771161700 1 -- 172.21.15.47:0/691675192 >> 172.21.15.47:0/691675192 conn(0x7fe76c062560 msgr2=0x7fe76c062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:00.765 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.755+0000 7fe771161700 1 -- 172.21.15.47:0/691675192 shutdown_connections 2022-06-29T15:43:00.765 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.755+0000 7fe771161700 1 -- 172.21.15.47:0/691675192 wait complete. 2022-06-29T15:43:00.768 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.755+0000 7fe771161700 1 Processor -- start 2022-06-29T15:43:00.768 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.755+0000 7fe771161700 1 -- start start 2022-06-29T15:43:00.768 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.759+0000 7fe771161700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe76c060eb0 0x7fe76c0ad610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:00.769 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.759+0000 7fe771161700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe76c0b5c60 0x7fe76c159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:00.769 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.759+0000 7fe771161700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe76c145090 0x7fe76c15a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:00.769 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.759+0000 7fe771161700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe76c148440 con 0x7fe76c145090 2022-06-29T15:43:00.769 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.759+0000 7fe771161700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe76c0acba0 con 0x7fe76c060eb0 2022-06-29T15:43:00.770 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.759+0000 7fe771161700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe76c145e00 con 0x7fe76c0b5c60 2022-06-29T15:43:00.770 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.759+0000 7fe770960700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe76c145090 0x7fe76c15a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:00.770 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.759+0000 7fe770960700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe76c145090 0x7fe76c15a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46782/0 (socket says 172.21.15.47:46782) 2022-06-29T15:43:00.770 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.759+0000 7fe770960700 1 -- 172.21.15.47:0/1180828561 learned_addr learned my addr 172.21.15.47:0/1180828561 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:00.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.759+0000 7fe76bfff700 1 --2- 172.21.15.47:0/1180828561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe76c060eb0 0x7fe76c0ad610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:00.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.759+0000 7fe76b7fe700 1 --2- 172.21.15.47:0/1180828561 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe76c0b5c60 0x7fe76c159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:00.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.759+0000 7fe770960700 1 -- 172.21.15.47:0/1180828561 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe76c0b5c60 msgr2=0x7fe76c159b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:00.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.759+0000 7fe770960700 1 --2- 172.21.15.47:0/1180828561 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe76c0b5c60 0x7fe76c159b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:00.772 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.759+0000 7fe770960700 1 -- 172.21.15.47:0/1180828561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe76c060eb0 msgr2=0x7fe76c0ad610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:00.772 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.759+0000 7fe770960700 1 --2- 172.21.15.47:0/1180828561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe76c060eb0 0x7fe76c0ad610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:00.772 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.759+0000 7fe770960700 1 -- 172.21.15.47:0/1180828561 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe75400b180 con 0x7fe76c145090 2022-06-29T15:43:00.772 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.759+0000 7fe76b7fe700 1 --2- 172.21.15.47:0/1180828561 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe76c0b5c60 0x7fe76c159b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:43:00.772 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.759+0000 7fe770960700 1 --2- 172.21.15.47:0/1180828561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe76c145090 0x7fe76c15a080 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7fe754002f70 tx=0x7fe754015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:00.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.759+0000 7fe7697fa700 1 -- 172.21.15.47:0/1180828561 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe75400d290 con 0x7fe76c145090 2022-06-29T15:43:00.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.759+0000 7fe7697fa700 1 -- 172.21.15.47:0/1180828561 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe75400d9b0 con 0x7fe76c145090 2022-06-29T15:43:00.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.759+0000 7fe771161700 1 -- 172.21.15.47:0/1180828561 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe76c0ac900 con 0x7fe76c145090 2022-06-29T15:43:00.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.759+0000 7fe7697fa700 1 -- 172.21.15.47:0/1180828561 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe754011e20 con 0x7fe76c145090 2022-06-29T15:43:00.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.759+0000 7fe771161700 1 -- 172.21.15.47:0/1180828561 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe76c14b6d0 con 0x7fe76c145090 2022-06-29T15:43:00.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.759+0000 7fe771161700 1 -- 172.21.15.47:0/1180828561 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe76c0573d0 con 0x7fe76c145090 2022-06-29T15:43:00.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.763+0000 7fe7697fa700 1 -- 172.21.15.47:0/1180828561 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe754025030 con 0x7fe76c145090 2022-06-29T15:43:00.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.763+0000 7fe7697fa700 1 --2- 172.21.15.47:0/1180828561 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe750053a70 0x7fe750055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:00.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.763+0000 7fe7697fa700 1 -- 172.21.15.47:0/1180828561 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fe75406d2b0 con 0x7fe76c145090 2022-06-29T15:43:00.776 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.763+0000 7fe76bfff700 1 --2- 172.21.15.47:0/1180828561 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe750053a70 0x7fe750055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:00.776 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.763+0000 7fe76bfff700 1 --2- 172.21.15.47:0/1180828561 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe750053a70 0x7fe750055f30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fe76400d000 tx=0x7fe764007eb0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:00.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.771+0000 7fe7697fa700 1 -- 172.21.15.47:0/1180828561 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe754044070 con 0x7fe76c145090 2022-06-29T15:43:00.941 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:00.935+0000 7fe771161700 1 -- 172.21.15.47:0/1180828561 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe76c069e20 con 0x7fe76c145090 2022-06-29T15:43:01.160 INFO:tasks.ceph.mon.a.smithi047.stderr:2022-06-29T15:43:01.151+0000 7ff9cf83c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-06-29T15:43:01.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.163+0000 7fe7697fa700 1 -- 172.21.15.47:0/1180828561 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v8) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe754023090 con 0x7fe76c145090 2022-06-29T15:43:01.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.167+0000 7fe771161700 1 -- 172.21.15.47:0/1180828561 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe750053a70 msgr2=0x7fe750055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:01.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.167+0000 7fe771161700 1 --2- 172.21.15.47:0/1180828561 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe750053a70 0x7fe750055f30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fe76400d000 tx=0x7fe764007eb0).stop 2022-06-29T15:43:01.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.167+0000 7fe771161700 1 -- 172.21.15.47:0/1180828561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe76c145090 msgr2=0x7fe76c15a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:01.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.167+0000 7fe771161700 1 --2- 172.21.15.47:0/1180828561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe76c145090 0x7fe76c15a080 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7fe754002f70 tx=0x7fe754015000).stop 2022-06-29T15:43:01.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.171+0000 7fe771161700 1 -- 172.21.15.47:0/1180828561 shutdown_connections 2022-06-29T15:43:01.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.171+0000 7fe771161700 1 --2- 172.21.15.47:0/1180828561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe76c145090 0x7fe76c15a080 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:01.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.171+0000 7fe771161700 1 --2- 172.21.15.47:0/1180828561 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe76c0b5c60 0x7fe76c159b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:01.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.171+0000 7fe771161700 1 --2- 172.21.15.47:0/1180828561 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe750053a70 0x7fe750055f30 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:01.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.171+0000 7fe771161700 1 --2- 172.21.15.47:0/1180828561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe76c060eb0 0x7fe76c0ad610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:01.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.171+0000 7fe771161700 1 -- 172.21.15.47:0/1180828561 >> 172.21.15.47:0/1180828561 conn(0x7fe76c062560 msgr2=0x7fe76c0b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:01.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.171+0000 7fe771161700 1 -- 172.21.15.47:0/1180828561 shutdown_connections 2022-06-29T15:43:01.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.171+0000 7fe771161700 1 -- 172.21.15.47:0/1180828561 wait complete. 2022-06-29T15:43:01.180 INFO:teuthology.orchestra.run.smithi047.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-06-29T15:43:01.195 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:43:01.197 INFO:tasks.ceph.mds.a.smithi047.stdout:starting mds.a at 2022-06-29T15:43:01.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.291+0000 7f527a32b700 1 Processor -- start 2022-06-29T15:43:01.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.291+0000 7f527a32b700 1 -- start start 2022-06-29T15:43:01.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.291+0000 7f527a32b700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f52740b2740 0x7f52740aa9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:01.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.291+0000 7f527a32b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f52741496b0 0x7f527414baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:01.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.291+0000 7f527a32b700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52740ae3e0 con 0x7f52740af990 2022-06-29T15:43:01.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.291+0000 7f527a32b700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52740af110 con 0x7f52741496b0 2022-06-29T15:43:01.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.291+0000 7f527a32b700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52740a9e80 con 0x7f52740b2740 2022-06-29T15:43:01.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.291+0000 7f5279b2a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f52741496b0 0x7f527414baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:01.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.291+0000 7f5279b2a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f52741496b0 0x7f527414baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46802/0 (socket says 172.21.15.47:46802) 2022-06-29T15:43:01.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.291+0000 7f5279329700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f52740b2740 0x7f52740aa9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:01.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.291+0000 7f5279b2a700 1 -- 172.21.15.47:0/586264428 learned_addr learned my addr 172.21.15.47:0/586264428 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:01.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.295+0000 7f5279b2a700 1 -- 172.21.15.47:0/586264428 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f52740b2740 msgr2=0x7f52740aa9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:43:01.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.295+0000 7f5279b2a700 1 --2- 172.21.15.47:0/586264428 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f52740b2740 0x7f52740aa9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:01.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.295+0000 7f5279b2a700 1 -- 172.21.15.47:0/586264428 >> v1:172.21.15.64:6789/0 conn(0x7f52740af990 legacy=0x7f52740ac180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:01.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.295+0000 7f5279b2a700 1 -- 172.21.15.47:0/586264428 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52740b1750 con 0x7f52741496b0 2022-06-29T15:43:01.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.295+0000 7f5279b2a700 1 --2- 172.21.15.47:0/586264428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52741496b0 0x7f527414baa0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f526c0032f0 tx=0x7f526c009290).ready entity=mon.0 client_cookie=a393c768e7c4d2af server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:01.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.295+0000 7f5263fff700 1 -- 172.21.15.47:0/586264428 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f526c00b8a0 con 0x7f52741496b0 2022-06-29T15:43:01.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.295+0000 7f5263fff700 1 -- 172.21.15.47:0/586264428 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f526c00ba00 con 0x7f52741496b0 2022-06-29T15:43:01.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.295+0000 7f5263fff700 1 -- 172.21.15.47:0/586264428 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f526c00bb70 con 0x7f52741496b0 2022-06-29T15:43:01.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.295+0000 7f527a32b700 1 -- 172.21.15.47:0/586264428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52741496b0 msgr2=0x7f527414baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:01.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.295+0000 7f527a32b700 1 --2- 172.21.15.47:0/586264428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52741496b0 0x7f527414baa0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f526c0032f0 tx=0x7f526c009290).stop 2022-06-29T15:43:01.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.295+0000 7f527a32b700 1 -- 172.21.15.47:0/586264428 shutdown_connections 2022-06-29T15:43:01.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.295+0000 7f527a32b700 1 --2- 172.21.15.47:0/586264428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52741496b0 0x7f527414baa0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:01.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.295+0000 7f527a32b700 1 --2- 172.21.15.47:0/586264428 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f52740b2740 0x7f52740aa9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:01.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.295+0000 7f527a32b700 1 -- 172.21.15.47:0/586264428 >> 172.21.15.47:0/586264428 conn(0x7f5274062df0 msgr2=0x7f52740631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:01.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.295+0000 7f527a32b700 1 -- 172.21.15.47:0/586264428 shutdown_connections 2022-06-29T15:43:01.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.295+0000 7f527a32b700 1 -- 172.21.15.47:0/586264428 wait complete. 2022-06-29T15:43:01.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.295+0000 7f527a32b700 1 Processor -- start 2022-06-29T15:43:01.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.295+0000 7f527a32b700 1 -- start start 2022-06-29T15:43:01.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.299+0000 7f527a32b700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f52740af990 0x7f52740674d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:01.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.299+0000 7f527a32b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52740b2740 0x7f5274064f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:01.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.299+0000 7f527a32b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52741496b0 0x7f5274065480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:01.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.299+0000 7f527a32b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5274063e00 con 0x7f52741496b0 2022-06-29T15:43:01.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.299+0000 7f527a32b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5274098230 con 0x7f52740b2740 2022-06-29T15:43:01.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.299+0000 7f527a32b700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52740b6070 con 0x7f52740af990 2022-06-29T15:43:01.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.299+0000 7f5279b2a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52741496b0 0x7f5274065480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:01.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.299+0000 7f5279b2a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52741496b0 0x7f5274065480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46808/0 (socket says 172.21.15.47:46808) 2022-06-29T15:43:01.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.299+0000 7f5279b2a700 1 -- 172.21.15.47:0/2646559388 learned_addr learned my addr 172.21.15.47:0/2646559388 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:01.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.299+0000 7f5279329700 1 --2- 172.21.15.47:0/2646559388 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f52740af990 0x7f52740674d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:01.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.299+0000 7f5278b28700 1 --2- 172.21.15.47:0/2646559388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52740b2740 0x7f5274064f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:01.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.299+0000 7f5279b2a700 1 -- 172.21.15.47:0/2646559388 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f52740af990 msgr2=0x7f52740674d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:01.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.299+0000 7f5279b2a700 1 --2- 172.21.15.47:0/2646559388 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f52740af990 0x7f52740674d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:01.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.299+0000 7f5279b2a700 1 -- 172.21.15.47:0/2646559388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52740b2740 msgr2=0x7f5274064f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:01.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.299+0000 7f5279b2a700 1 --2- 172.21.15.47:0/2646559388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52740b2740 0x7f5274064f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:01.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.299+0000 7f5279b2a700 1 -- 172.21.15.47:0/2646559388 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f526c009530 con 0x7f52741496b0 2022-06-29T15:43:01.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.299+0000 7f5278b28700 1 --2- 172.21.15.47:0/2646559388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52740b2740 0x7f5274064f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:43:01.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.299+0000 7f5279b2a700 1 --2- 172.21.15.47:0/2646559388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52741496b0 0x7f5274065480 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f526c0032f0 tx=0x7f526c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:01.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.299+0000 7f52627fc700 1 -- 172.21.15.47:0/2646559388 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f526c00b8a0 con 0x7f52741496b0 2022-06-29T15:43:01.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.299+0000 7f52627fc700 1 -- 172.21.15.47:0/2646559388 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f526c021440 con 0x7f52741496b0 2022-06-29T15:43:01.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.299+0000 7f527a32b700 1 -- 172.21.15.47:0/2646559388 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52740b6310 con 0x7f52741496b0 2022-06-29T15:43:01.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.299+0000 7f52627fc700 1 -- 172.21.15.47:0/2646559388 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f526c0217a0 con 0x7f52741496b0 2022-06-29T15:43:01.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.299+0000 7f527a32b700 1 -- 172.21.15.47:0/2646559388 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52740b2150 con 0x7f52741496b0 2022-06-29T15:43:01.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.303+0000 7f527a32b700 1 -- 172.21.15.47:0/2646559388 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5274145620 con 0x7f52741496b0 2022-06-29T15:43:01.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.303+0000 7f52627fc700 1 -- 172.21.15.47:0/2646559388 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f526c00f070 con 0x7f52741496b0 2022-06-29T15:43:01.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.303+0000 7f52627fc700 1 --2- 172.21.15.47:0/2646559388 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5254053950 0x7f5254055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:01.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.303+0000 7f52627fc700 1 -- 172.21.15.47:0/2646559388 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6517+0+0 (secure 0 0 0) 0x7f526c06d5b0 con 0x7f52741496b0 2022-06-29T15:43:01.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.303+0000 7f5279329700 1 --2- 172.21.15.47:0/2646559388 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5254053950 0x7f5254055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:01.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.303+0000 7f5279329700 1 --2- 172.21.15.47:0/2646559388 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5254053950 0x7f5254055e10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f527000b000 tx=0x7f5270006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:01.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.307+0000 7f52627fc700 1 -- 172.21.15.47:0/2646559388 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f526c044310 con 0x7f52741496b0 2022-06-29T15:43:01.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.475+0000 7f527a32b700 1 -- 172.21.15.47:0/2646559388 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f52740573d0 con 0x7f52741496b0 2022-06-29T15:43:01.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.475+0000 7f52627fc700 1 -- 172.21.15.47:0/2646559388 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+2123 (secure 0 0 0) 0x7f526c015020 con 0x7f52741496b0 2022-06-29T15:43:01.483 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:43:01.484 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/438289599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":438289599},{"type":"v1","addr":"172.21.15.47:6835","nonce":438289599}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.64:6833/1472525731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1472525731},{"type":"v1","addr":"172.21.15.64:6833","nonce":1472525731}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":4}],"filesystems":[{"mdsmap":{"epoch":8,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:42:33.471491+0000","modified":"2022-06-29T15:43:01.155174+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2022-06-29T15:43:01.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.479+0000 7f527a32b700 1 -- 172.21.15.47:0/2646559388 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5254053950 msgr2=0x7f5254055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:01.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.479+0000 7f527a32b700 1 --2- 172.21.15.47:0/2646559388 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5254053950 0x7f5254055e10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f527000b000 tx=0x7f5270006df0).stop 2022-06-29T15:43:01.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.479+0000 7f527a32b700 1 -- 172.21.15.47:0/2646559388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52741496b0 msgr2=0x7f5274065480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:01.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.479+0000 7f527a32b700 1 --2- 172.21.15.47:0/2646559388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52741496b0 0x7f5274065480 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f526c0032f0 tx=0x7f526c016000).stop 2022-06-29T15:43:01.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.479+0000 7f527a32b700 1 -- 172.21.15.47:0/2646559388 shutdown_connections 2022-06-29T15:43:01.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.479+0000 7f527a32b700 1 --2- 172.21.15.47:0/2646559388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52741496b0 0x7f5274065480 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:01.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.479+0000 7f527a32b700 1 --2- 172.21.15.47:0/2646559388 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5254053950 0x7f5254055e10 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:01.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.479+0000 7f527a32b700 1 --2- 172.21.15.47:0/2646559388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52740b2740 0x7f5274064f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:01.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.479+0000 7f527a32b700 1 --2- 172.21.15.47:0/2646559388 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f52740af990 0x7f52740674d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:01.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.479+0000 7f527a32b700 1 -- 172.21.15.47:0/2646559388 >> 172.21.15.47:0/2646559388 conn(0x7f5274062df0 msgr2=0x7f527409ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:01.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.479+0000 7f527a32b700 1 -- 172.21.15.47:0/2646559388 shutdown_connections 2022-06-29T15:43:01.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.479+0000 7f527a32b700 1 -- 172.21.15.47:0/2646559388 wait complete. 2022-06-29T15:43:01.489 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 8 2022-06-29T15:43:01.497 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-06-29T15:43:01.597 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.591+0000 7fd9d1dab700 1 Processor -- start 2022-06-29T15:43:01.597 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.591+0000 7fd9d1dab700 1 -- start start 2022-06-29T15:43:01.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.591+0000 7fd9d1dab700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd9cc0b5d90 0x7fd9cc143a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:01.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.591+0000 7fd9d1dab700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9cc05cc30 con 0x7fd9cc0676a0 2022-06-29T15:43:01.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.591+0000 7fd9d1dab700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9cc1430a0 con 0x7fd9cc05c850 2022-06-29T15:43:01.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.591+0000 7fd9d1dab700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9cc143300 con 0x7fd9cc0b5d90 2022-06-29T15:43:01.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.591+0000 7fd9d15aa700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd9cc0676a0 0x7fd9cc140130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:60600/0 (socket says 172.21.15.47:60600) 2022-06-29T15:43:01.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.591+0000 7fd9d15aa700 1 -- 172.21.15.47:0/1353866019 learned_addr learned my addr 172.21.15.47:0/1353866019 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:43:01.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.591+0000 7fd9cbfff700 1 --2- 172.21.15.47:0/1353866019 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd9cc0b5d90 0x7fd9cc143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:01.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.591+0000 7fd9cb7fe700 1 -- 172.21.15.47:0/1353866019 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1192928728 0 0) 0x7fd9cc05cc30 con 0x7fd9cc0676a0 2022-06-29T15:43:01.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.591+0000 7fd9cb7fe700 1 -- 172.21.15.47:0/1353866019 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9b0003260 con 0x7fd9cc0676a0 2022-06-29T15:43:01.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.591+0000 7fd9cb7fe700 1 -- 172.21.15.47:0/1353866019 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3431200314 0 0) 0x7fd9cc1430a0 con 0x7fd9cc05c850 2022-06-29T15:43:01.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.591+0000 7fd9cb7fe700 1 -- 172.21.15.47:0/1353866019 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9cc05cc30 con 0x7fd9cc05c850 2022-06-29T15:43:01.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.591+0000 7fd9cbfff700 1 -- 172.21.15.47:0/1353866019 >> v1:172.21.15.64:6789/0 conn(0x7fd9cc05c850 legacy=0x7fd9cc0b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:01.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.591+0000 7fd9cbfff700 1 -- 172.21.15.47:0/1353866019 >> v1:172.21.15.47:6789/0 conn(0x7fd9cc0676a0 legacy=0x7fd9cc140130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:01.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.591+0000 7fd9cbfff700 1 -- 172.21.15.47:0/1353866019 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9cc1435a0 con 0x7fd9cc0b5d90 2022-06-29T15:43:01.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.591+0000 7fd9cbfff700 1 --2- 172.21.15.47:0/1353866019 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd9cc0b5d90 0x7fd9cc143a40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fd9b8002f70 tx=0x7fd9b8006df0).ready entity=mon.2 client_cookie=98321779a0809eed server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:01.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.591+0000 7fd9cb7fe700 1 -- 172.21.15.47:0/1353866019 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd9b800d5c0 con 0x7fd9cc0b5d90 2022-06-29T15:43:01.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.591+0000 7fd9cb7fe700 1 -- 172.21.15.47:0/1353866019 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b8000e90 con 0x7fd9cc0b5d90 2022-06-29T15:43:01.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.591+0000 7fd9cb7fe700 1 -- 172.21.15.47:0/1353866019 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd9b800d910 con 0x7fd9cc0b5d90 2022-06-29T15:43:01.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.595+0000 7fd9d1dab700 1 -- 172.21.15.47:0/1353866019 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd9cc0b5d90 msgr2=0x7fd9cc143a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:01.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.595+0000 7fd9d1dab700 1 --2- 172.21.15.47:0/1353866019 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd9cc0b5d90 0x7fd9cc143a40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fd9b8002f70 tx=0x7fd9b8006df0).stop 2022-06-29T15:43:01.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.595+0000 7fd9d1dab700 1 -- 172.21.15.47:0/1353866019 shutdown_connections 2022-06-29T15:43:01.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.595+0000 7fd9d1dab700 1 --2- 172.21.15.47:0/1353866019 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd9cc0b5d90 0x7fd9cc143a40 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:01.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.595+0000 7fd9d1dab700 1 -- 172.21.15.47:0/1353866019 >> 172.21.15.47:0/1353866019 conn(0x7fd9cc060e00 msgr2=0x7fd9cc0611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:01.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.595+0000 7fd9d1dab700 1 -- 172.21.15.47:0/1353866019 shutdown_connections 2022-06-29T15:43:01.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.595+0000 7fd9d1dab700 1 -- 172.21.15.47:0/1353866019 wait complete. 2022-06-29T15:43:01.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.595+0000 7fd9d1dab700 1 Processor -- start 2022-06-29T15:43:01.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.595+0000 7fd9d1dab700 1 -- start start 2022-06-29T15:43:01.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.595+0000 7fd9d1dab700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd9cc05c850 0x7fd9cc142020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:01.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.595+0000 7fd9d1dab700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9cc0676a0 0x7fd9cc152400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:01.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.595+0000 7fd9d1dab700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9cc0b5d90 0x7fd9cc0b9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:01.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.595+0000 7fd9d1dab700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9cc142810 con 0x7fd9cc0b5d90 2022-06-29T15:43:01.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.595+0000 7fd9d1dab700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9cc0b9dd0 con 0x7fd9cc0676a0 2022-06-29T15:43:01.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.595+0000 7fd9d1dab700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9cc1528f0 con 0x7fd9cc05c850 2022-06-29T15:43:01.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.595+0000 7fd9d15aa700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9cc0b5d90 0x7fd9cc0b9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:01.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.595+0000 7fd9d15aa700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9cc0b5d90 0x7fd9cc0b9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46822/0 (socket says 172.21.15.47:46822) 2022-06-29T15:43:01.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.595+0000 7fd9d15aa700 1 -- 172.21.15.47:0/84762776 learned_addr learned my addr 172.21.15.47:0/84762776 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:01.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.595+0000 7fd9cbfff700 1 --2- 172.21.15.47:0/84762776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9cc0676a0 0x7fd9cc152400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:01.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.595+0000 7fd9d0da9700 1 --2- 172.21.15.47:0/84762776 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd9cc05c850 0x7fd9cc142020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:01.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.599+0000 7fd9d15aa700 1 -- 172.21.15.47:0/84762776 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd9cc05c850 msgr2=0x7fd9cc142020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:01.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.599+0000 7fd9d15aa700 1 --2- 172.21.15.47:0/84762776 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd9cc05c850 0x7fd9cc142020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:01.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.599+0000 7fd9d15aa700 1 -- 172.21.15.47:0/84762776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9cc0676a0 msgr2=0x7fd9cc152400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:01.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.599+0000 7fd9d15aa700 1 --2- 172.21.15.47:0/84762776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9cc0676a0 0x7fd9cc152400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:01.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.599+0000 7fd9d15aa700 1 -- 172.21.15.47:0/84762776 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b800b180 con 0x7fd9cc0b5d90 2022-06-29T15:43:01.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.599+0000 7fd9d15aa700 1 --2- 172.21.15.47:0/84762776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9cc0b5d90 0x7fd9cc0b9300 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7fd9c4001490 tx=0x7fd9c4007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:01.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.599+0000 7fd9c9ffb700 1 -- 172.21.15.47:0/84762776 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd9c4005ba0 con 0x7fd9cc0b5d90 2022-06-29T15:43:01.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.599+0000 7fd9c9ffb700 1 -- 172.21.15.47:0/84762776 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9c4005d00 con 0x7fd9cc0b5d90 2022-06-29T15:43:01.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.599+0000 7fd9c9ffb700 1 -- 172.21.15.47:0/84762776 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd9c40072c0 con 0x7fd9cc0b5d90 2022-06-29T15:43:01.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.599+0000 7fd9d1dab700 1 -- 172.21.15.47:0/84762776 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9cc0b9b30 con 0x7fd9cc0b5d90 2022-06-29T15:43:01.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.599+0000 7fd9d1dab700 1 -- 172.21.15.47:0/84762776 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9cc155c90 con 0x7fd9cc0b5d90 2022-06-29T15:43:01.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.599+0000 7fd9d1dab700 1 -- 172.21.15.47:0/84762776 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9cc0573d0 con 0x7fd9cc0b5d90 2022-06-29T15:43:01.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.599+0000 7fd9c9ffb700 1 -- 172.21.15.47:0/84762776 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd9c4010070 con 0x7fd9cc0b5d90 2022-06-29T15:43:01.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.603+0000 7fd9c9ffb700 1 --2- 172.21.15.47:0/84762776 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd9ac053aa0 0x7fd9ac055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:01.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.603+0000 7fd9c9ffb700 1 -- 172.21.15.47:0/84762776 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6517+0+0 (secure 0 0 0) 0x7fd9c406b9d0 con 0x7fd9cc0b5d90 2022-06-29T15:43:01.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.603+0000 7fd9d0da9700 1 --2- 172.21.15.47:0/84762776 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd9ac053aa0 0x7fd9ac055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:01.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.603+0000 7fd9d0da9700 1 --2- 172.21.15.47:0/84762776 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd9ac053aa0 0x7fd9ac055f60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fd9c000d000 tx=0x7fd9c00079f0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:01.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.611+0000 7fd9c9ffb700 1 -- 172.21.15.47:0/84762776 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd9c4042620 con 0x7fd9cc0b5d90 2022-06-29T15:43:01.784 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:01.775+0000 7fd9d1dab700 1 -- 172.21.15.47:0/84762776 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd9cc0b3e00 con 0x7fd9cc0b5d90 2022-06-29T15:43:02.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.167+0000 7fd9c9ffb700 1 -- 172.21.15.47:0/84762776 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd9c4018020 con 0x7fd9cc0b5d90 2022-06-29T15:43:02.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.171+0000 7fd9d1dab700 1 -- 172.21.15.47:0/84762776 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd9ac053aa0 msgr2=0x7fd9ac055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:02.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.171+0000 7fd9d1dab700 1 --2- 172.21.15.47:0/84762776 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd9ac053aa0 0x7fd9ac055f60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fd9c000d000 tx=0x7fd9c00079f0).stop 2022-06-29T15:43:02.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.171+0000 7fd9d1dab700 1 -- 172.21.15.47:0/84762776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9cc0b5d90 msgr2=0x7fd9cc0b9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:02.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.171+0000 7fd9d1dab700 1 --2- 172.21.15.47:0/84762776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9cc0b5d90 0x7fd9cc0b9300 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7fd9c4001490 tx=0x7fd9c4007e00).stop 2022-06-29T15:43:02.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.171+0000 7fd9d1dab700 1 -- 172.21.15.47:0/84762776 shutdown_connections 2022-06-29T15:43:02.182 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.171+0000 7fd9d1dab700 1 --2- 172.21.15.47:0/84762776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9cc0b5d90 0x7fd9cc0b9300 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:02.182 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.171+0000 7fd9d1dab700 1 --2- 172.21.15.47:0/84762776 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd9ac053aa0 0x7fd9ac055f60 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:02.182 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.171+0000 7fd9d1dab700 1 --2- 172.21.15.47:0/84762776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9cc0676a0 0x7fd9cc152400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:02.183 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.171+0000 7fd9d1dab700 1 --2- 172.21.15.47:0/84762776 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd9cc05c850 0x7fd9cc142020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:02.183 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.171+0000 7fd9d1dab700 1 -- 172.21.15.47:0/84762776 >> 172.21.15.47:0/84762776 conn(0x7fd9cc060e00 msgr2=0x7fd9cc145a80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:02.183 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.171+0000 7fd9d1dab700 1 -- 172.21.15.47:0/84762776 shutdown_connections 2022-06-29T15:43:02.183 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.175+0000 7fd9d1dab700 1 -- 172.21.15.47:0/84762776 wait complete. 2022-06-29T15:43:02.196 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-06-29T15:43:02.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.307+0000 7f2bd6202700 1 Processor -- start 2022-06-29T15:43:02.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.307+0000 7f2bd6202700 1 -- start start 2022-06-29T15:43:02.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.307+0000 7f2bd6202700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2bd00980c0 0x7f2bd00a97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:02.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.307+0000 7f2bd6202700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bd0060370 con 0x7f2bd0060b30 2022-06-29T15:43:02.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.307+0000 7f2bd6202700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bd0095320 con 0x7f2bd0097ce0 2022-06-29T15:43:02.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.307+0000 7f2bd6202700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd00ae5f0 con 0x7f2bd00980c0 2022-06-29T15:43:02.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.307+0000 7f2bceffd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f2bd0060b30 0x7f2bd00b08a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:60614/0 (socket says 172.21.15.47:60614) 2022-06-29T15:43:02.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.307+0000 7f2bceffd700 1 -- 172.21.15.47:0/3825994877 learned_addr learned my addr 172.21.15.47:0/3825994877 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:43:02.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.307+0000 7f2bcffff700 1 --2- 172.21.15.47:0/3825994877 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2bd00980c0 0x7f2bd00a97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:02.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.307+0000 7f2bce7fc700 1 -- 172.21.15.47:0/3825994877 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 330128117 0 0) 0x7f2bd0060370 con 0x7f2bd0060b30 2022-06-29T15:43:02.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.307+0000 7f2bce7fc700 1 -- 172.21.15.47:0/3825994877 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bbc003260 con 0x7f2bd0060b30 2022-06-29T15:43:02.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.307+0000 7f2bce7fc700 1 -- 172.21.15.47:0/3825994877 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3782136516 0 0) 0x7f2bd0095320 con 0x7f2bd0097ce0 2022-06-29T15:43:02.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.307+0000 7f2bce7fc700 1 -- 172.21.15.47:0/3825994877 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bd0060370 con 0x7f2bd0097ce0 2022-06-29T15:43:02.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.307+0000 7f2bcffff700 1 -- 172.21.15.47:0/3825994877 >> v1:172.21.15.64:6789/0 conn(0x7f2bd0097ce0 legacy=0x7f2bd00af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:02.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.307+0000 7f2bcffff700 1 -- 172.21.15.47:0/3825994877 >> v1:172.21.15.47:6789/0 conn(0x7f2bd0060b30 legacy=0x7f2bd00b08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:02.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.307+0000 7f2bcffff700 1 -- 172.21.15.47:0/3825994877 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd00605d0 con 0x7f2bd00980c0 2022-06-29T15:43:02.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.307+0000 7f2bcffff700 1 --2- 172.21.15.47:0/3825994877 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2bd00980c0 0x7f2bd00a97f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f2bc80032f0 tx=0x7f2bc80091d0).ready entity=mon.2 client_cookie=13ec7316413e1e18 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:02.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.307+0000 7f2bce7fc700 1 -- 172.21.15.47:0/3825994877 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2bc800b640 con 0x7f2bd00980c0 2022-06-29T15:43:02.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.307+0000 7f2bce7fc700 1 -- 172.21.15.47:0/3825994877 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc800b7a0 con 0x7f2bd00980c0 2022-06-29T15:43:02.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.307+0000 7f2bce7fc700 1 -- 172.21.15.47:0/3825994877 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2bc800b910 con 0x7f2bd00980c0 2022-06-29T15:43:02.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.311+0000 7f2bd6202700 1 -- 172.21.15.47:0/3825994877 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2bd00980c0 msgr2=0x7f2bd00a97f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:02.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.311+0000 7f2bd6202700 1 --2- 172.21.15.47:0/3825994877 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2bd00980c0 0x7f2bd00a97f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f2bc80032f0 tx=0x7f2bc80091d0).stop 2022-06-29T15:43:02.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.311+0000 7f2bd6202700 1 -- 172.21.15.47:0/3825994877 shutdown_connections 2022-06-29T15:43:02.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.311+0000 7f2bd6202700 1 --2- 172.21.15.47:0/3825994877 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2bd00980c0 0x7f2bd00a97f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:02.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.311+0000 7f2bd6202700 1 -- 172.21.15.47:0/3825994877 >> 172.21.15.47:0/3825994877 conn(0x7f2bd005f0b0 msgr2=0x7f2bd005f490 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:02.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.311+0000 7f2bd6202700 1 -- 172.21.15.47:0/3825994877 shutdown_connections 2022-06-29T15:43:02.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.311+0000 7f2bd6202700 1 -- 172.21.15.47:0/3825994877 wait complete. 2022-06-29T15:43:02.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.311+0000 7f2bd6202700 1 Processor -- start 2022-06-29T15:43:02.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.311+0000 7f2bd6202700 1 -- start start 2022-06-29T15:43:02.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.311+0000 7f2bd6202700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2bd0060b30 0x7f2bd009a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:02.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.311+0000 7f2bd6202700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bd0097ce0 0x7f2bd0064b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:02.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.311+0000 7f2bd6202700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2bd00980c0 0x7f2bd0065040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:02.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.311+0000 7f2bd6202700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd00b4bc0 con 0x7f2bd0060b30 2022-06-29T15:43:02.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.311+0000 7f2bd6202700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd00b5290 con 0x7f2bd0097ce0 2022-06-29T15:43:02.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.311+0000 7f2bd6202700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd00b6b30 con 0x7f2bd00980c0 2022-06-29T15:43:02.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.311+0000 7f2bcf7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2bd0060b30 0x7f2bd009a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:02.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.311+0000 7f2bcf7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2bd0060b30 0x7f2bd009a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46830/0 (socket says 172.21.15.47:46830) 2022-06-29T15:43:02.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.311+0000 7f2bcf7fe700 1 -- 172.21.15.47:0/1877633421 learned_addr learned my addr 172.21.15.47:0/1877633421 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:02.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.311+0000 7f2bcffff700 1 --2- 172.21.15.47:0/1877633421 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2bd00980c0 0x7f2bd0065040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:02.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.311+0000 7f2bceffd700 1 --2- 172.21.15.47:0/1877633421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bd0097ce0 0x7f2bd0064b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:02.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.315+0000 7f2bcf7fe700 1 -- 172.21.15.47:0/1877633421 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2bd00980c0 msgr2=0x7f2bd0065040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:02.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.315+0000 7f2bcf7fe700 1 --2- 172.21.15.47:0/1877633421 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2bd00980c0 0x7f2bd0065040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:02.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.315+0000 7f2bcf7fe700 1 -- 172.21.15.47:0/1877633421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bd0097ce0 msgr2=0x7f2bd0064b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:02.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.315+0000 7f2bcf7fe700 1 --2- 172.21.15.47:0/1877633421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bd0097ce0 0x7f2bd0064b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:02.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.315+0000 7f2bcf7fe700 1 -- 172.21.15.47:0/1877633421 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc8009470 con 0x7f2bd0060b30 2022-06-29T15:43:02.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.315+0000 7f2bcf7fe700 1 --2- 172.21.15.47:0/1877633421 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2bd0060b30 0x7f2bd009a760 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7f2bc0001110 tx=0x7f2bc0007a90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:02.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.315+0000 7f2bccff9700 1 -- 172.21.15.47:0/1877633421 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2bc0012400 con 0x7f2bd0060b30 2022-06-29T15:43:02.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.315+0000 7f2bccff9700 1 -- 172.21.15.47:0/1877633421 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc0012560 con 0x7f2bd0060b30 2022-06-29T15:43:02.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.315+0000 7f2bd6202700 1 -- 172.21.15.47:0/1877633421 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bd00b5650 con 0x7f2bd0060b30 2022-06-29T15:43:02.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.315+0000 7f2bccff9700 1 -- 172.21.15.47:0/1877633421 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2bc00128c0 con 0x7f2bd0060b30 2022-06-29T15:43:02.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.315+0000 7f2bd6202700 1 -- 172.21.15.47:0/1877633421 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bd00639c0 con 0x7f2bd0060b30 2022-06-29T15:43:02.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.315+0000 7f2bd6202700 1 -- 172.21.15.47:0/1877633421 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bd00573d0 con 0x7f2bd0060b30 2022-06-29T15:43:02.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.315+0000 7f2bccff9700 1 -- 172.21.15.47:0/1877633421 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2bc0019030 con 0x7f2bd0060b30 2022-06-29T15:43:02.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.319+0000 7f2bccff9700 1 --2- 172.21.15.47:0/1877633421 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2bb0053a70 0x7f2bb0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:02.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.319+0000 7f2bceffd700 1 --2- 172.21.15.47:0/1877633421 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2bb0053a70 0x7f2bb0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:02.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.319+0000 7f2bccff9700 1 -- 172.21.15.47:0/1877633421 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6517+0+0 (secure 0 0 0) 0x7f2bc006b480 con 0x7f2bd0060b30 2022-06-29T15:43:02.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.319+0000 7f2bceffd700 1 --2- 172.21.15.47:0/1877633421 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2bb0053a70 0x7f2bb0055f30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f2bc400d000 tx=0x7f2bc4009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:02.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.323+0000 7f2bccff9700 1 -- 172.21.15.47:0/1877633421 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2bc0042160 con 0x7f2bd0060b30 2022-06-29T15:43:02.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:02.455+0000 7f2bd6202700 1 -- 172.21.15.47:0/1877633421 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2bd0044710 con 0x7f2bd0060b30 2022-06-29T15:43:03.193 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.187+0000 7f2bccff9700 1 -- 172.21.15.47:0/1877633421 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v28) v1 ==== 185+0+0 (secure 0 0 0) 0x7f2bc0045780 con 0x7f2bd0060b30 2022-06-29T15:43:03.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.187+0000 7f2bd6202700 1 -- 172.21.15.47:0/1877633421 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2bb0053a70 msgr2=0x7f2bb0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:03.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.187+0000 7f2bd6202700 1 --2- 172.21.15.47:0/1877633421 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2bb0053a70 0x7f2bb0055f30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f2bc400d000 tx=0x7f2bc4009040).stop 2022-06-29T15:43:03.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.191+0000 7f2bd6202700 1 -- 172.21.15.47:0/1877633421 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2bd0060b30 msgr2=0x7f2bd009a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:03.198 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.191+0000 7f2bd6202700 1 --2- 172.21.15.47:0/1877633421 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2bd0060b30 0x7f2bd009a760 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7f2bc0001110 tx=0x7f2bc0007a90).stop 2022-06-29T15:43:03.198 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.191+0000 7f2bd6202700 1 -- 172.21.15.47:0/1877633421 shutdown_connections 2022-06-29T15:43:03.198 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.191+0000 7f2bd6202700 1 --2- 172.21.15.47:0/1877633421 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2bd00980c0 0x7f2bd0065040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:03.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.191+0000 7f2bd6202700 1 --2- 172.21.15.47:0/1877633421 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2bb0053a70 0x7f2bb0055f30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:03.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.191+0000 7f2bd6202700 1 --2- 172.21.15.47:0/1877633421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bd0097ce0 0x7f2bd0064b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:03.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.191+0000 7f2bd6202700 1 --2- 172.21.15.47:0/1877633421 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2bd0060b30 0x7f2bd009a760 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:03.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.191+0000 7f2bd6202700 1 -- 172.21.15.47:0/1877633421 >> 172.21.15.47:0/1877633421 conn(0x7f2bd005f0b0 msgr2=0x7f2bd00aacc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:03.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.191+0000 7f2bd6202700 1 -- 172.21.15.47:0/1877633421 shutdown_connections 2022-06-29T15:43:03.200 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.191+0000 7f2bd6202700 1 -- 172.21.15.47:0/1877633421 wait complete. 2022-06-29T15:43:03.200 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' removed 2022-06-29T15:43:03.213 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-06-29T15:43:03.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.323+0000 7f54bb427700 1 Processor -- start 2022-06-29T15:43:03.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.323+0000 7f54bb427700 1 -- start start 2022-06-29T15:43:03.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.323+0000 7f54bb427700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f54b40612c0 0x7f54b41471a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:03.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.323+0000 7f54bb427700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54b4067610 con 0x7f54b40b4300 2022-06-29T15:43:03.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.323+0000 7f54bb427700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54b41467c0 con 0x7f54b4060ee0 2022-06-29T15:43:03.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.323+0000 7f54bb427700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b4146a60 con 0x7f54b40612c0 2022-06-29T15:43:03.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.323+0000 7f54b89c2700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f54b40b4300 0x7f54b413cdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:60626/0 (socket says 172.21.15.47:60626) 2022-06-29T15:43:03.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.327+0000 7f54b89c2700 1 -- 172.21.15.47:0/3843713436 learned_addr learned my addr 172.21.15.47:0/3843713436 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:43:03.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.327+0000 7f54b99c4700 1 --2- 172.21.15.47:0/3843713436 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f54b40612c0 0x7f54b41471a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:03.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.327+0000 7f54a7fff700 1 -- 172.21.15.47:0/3843713436 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4047443446 0 0) 0x7f54b4067610 con 0x7f54b40b4300 2022-06-29T15:43:03.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.327+0000 7f54a7fff700 1 -- 172.21.15.47:0/3843713436 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5498003260 con 0x7f54b40b4300 2022-06-29T15:43:03.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.327+0000 7f54a7fff700 1 -- 172.21.15.47:0/3843713436 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 278674969 0 0) 0x7f54b41467c0 con 0x7f54b4060ee0 2022-06-29T15:43:03.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.327+0000 7f54a7fff700 1 -- 172.21.15.47:0/3843713436 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54b4067610 con 0x7f54b4060ee0 2022-06-29T15:43:03.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.327+0000 7f54b99c4700 1 -- 172.21.15.47:0/3843713436 >> v1:172.21.15.64:6789/0 conn(0x7f54b4060ee0 legacy=0x7f54b41430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:03.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.327+0000 7f54b99c4700 1 -- 172.21.15.47:0/3843713436 >> v1:172.21.15.47:6789/0 conn(0x7f54b40b4300 legacy=0x7f54b413cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:03.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.327+0000 7f54b99c4700 1 -- 172.21.15.47:0/3843713436 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b4146d00 con 0x7f54b40612c0 2022-06-29T15:43:03.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.327+0000 7f54b99c4700 1 --2- 172.21.15.47:0/3843713436 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f54b40612c0 0x7f54b41471a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f54b40ae790 tx=0x7f54ac0091d0).ready entity=mon.2 client_cookie=58c239780b8e6654 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:03.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.327+0000 7f54a7fff700 1 -- 172.21.15.47:0/3843713436 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54ac00b600 con 0x7f54b40612c0 2022-06-29T15:43:03.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.327+0000 7f54a7fff700 1 -- 172.21.15.47:0/3843713436 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54ac00b760 con 0x7f54b40612c0 2022-06-29T15:43:03.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.327+0000 7f54a7fff700 1 -- 172.21.15.47:0/3843713436 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54ac00b8d0 con 0x7f54b40612c0 2022-06-29T15:43:03.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.327+0000 7f54bb427700 1 -- 172.21.15.47:0/3843713436 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f54b40612c0 msgr2=0x7f54b41471a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:03.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.327+0000 7f54bb427700 1 --2- 172.21.15.47:0/3843713436 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f54b40612c0 0x7f54b41471a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f54b40ae790 tx=0x7f54ac0091d0).stop 2022-06-29T15:43:03.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.327+0000 7f54bb427700 1 -- 172.21.15.47:0/3843713436 shutdown_connections 2022-06-29T15:43:03.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.327+0000 7f54bb427700 1 --2- 172.21.15.47:0/3843713436 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f54b40612c0 0x7f54b41471a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:03.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.327+0000 7f54bb427700 1 -- 172.21.15.47:0/3843713436 >> 172.21.15.47:0/3843713436 conn(0x7f54b405c510 msgr2=0x7f54b405f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:03.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.327+0000 7f54bb427700 1 -- 172.21.15.47:0/3843713436 shutdown_connections 2022-06-29T15:43:03.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.327+0000 7f54bb427700 1 -- 172.21.15.47:0/3843713436 wait complete. 2022-06-29T15:43:03.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.331+0000 7f54bb427700 1 Processor -- start 2022-06-29T15:43:03.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.331+0000 7f54bb427700 1 -- start start 2022-06-29T15:43:03.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.331+0000 7f54bb427700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f54b4060ee0 0x7f54b4142050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:03.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.331+0000 7f54bb427700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54b40612c0 0x7f54b40b9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:03.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.331+0000 7f54bb427700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54b40b4300 0x7f54b40b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:03.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.331+0000 7f54bb427700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b40b0cd0 con 0x7f54b40612c0 2022-06-29T15:43:03.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.331+0000 7f54bb427700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b40b1360 con 0x7f54b40b4300 2022-06-29T15:43:03.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.331+0000 7f54bb427700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b40b3ce0 con 0x7f54b4060ee0 2022-06-29T15:43:03.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.331+0000 7f54b89c2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54b40612c0 0x7f54b40b9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:03.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.331+0000 7f54b89c2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54b40612c0 0x7f54b40b9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46846/0 (socket says 172.21.15.47:46846) 2022-06-29T15:43:03.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.331+0000 7f54b89c2700 1 -- 172.21.15.47:0/4176685995 learned_addr learned my addr 172.21.15.47:0/4176685995 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:03.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.331+0000 7f54b91c3700 1 --2- 172.21.15.47:0/4176685995 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f54b4060ee0 0x7f54b4142050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:03.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.331+0000 7f54b99c4700 1 --2- 172.21.15.47:0/4176685995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54b40b4300 0x7f54b40b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:03.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.331+0000 7f54b89c2700 1 -- 172.21.15.47:0/4176685995 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f54b4060ee0 msgr2=0x7f54b4142050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:03.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.331+0000 7f54b89c2700 1 --2- 172.21.15.47:0/4176685995 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f54b4060ee0 0x7f54b4142050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:03.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.331+0000 7f54b89c2700 1 -- 172.21.15.47:0/4176685995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54b40b4300 msgr2=0x7f54b40b15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:03.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.331+0000 7f54b89c2700 1 --2- 172.21.15.47:0/4176685995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54b40b4300 0x7f54b40b15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:03.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.331+0000 7f54b89c2700 1 -- 172.21.15.47:0/4176685995 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54ac009470 con 0x7f54b40612c0 2022-06-29T15:43:03.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.331+0000 7f54b89c2700 1 --2- 172.21.15.47:0/4176685995 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54b40612c0 0x7f54b40b9cc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f54b0001110 tx=0x7f54b0007af0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:03.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.331+0000 7f54a67fc700 1 -- 172.21.15.47:0/4176685995 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54b00070d0 con 0x7f54b40612c0 2022-06-29T15:43:03.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.331+0000 7f54a67fc700 1 -- 172.21.15.47:0/4176685995 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54b0007230 con 0x7f54b40612c0 2022-06-29T15:43:03.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.331+0000 7f54bb427700 1 -- 172.21.15.47:0/4176685995 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54b40b91a0 con 0x7f54b40612c0 2022-06-29T15:43:03.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.331+0000 7f54a67fc700 1 -- 172.21.15.47:0/4176685995 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54b0005610 con 0x7f54b40612c0 2022-06-29T15:43:03.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.331+0000 7f54bb427700 1 -- 172.21.15.47:0/4176685995 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54b40b96e0 con 0x7f54b40612c0 2022-06-29T15:43:03.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.335+0000 7f54a67fc700 1 -- 172.21.15.47:0/4176685995 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f54b001a060 con 0x7f54b40612c0 2022-06-29T15:43:03.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.335+0000 7f54bb427700 1 -- 172.21.15.47:0/4176685995 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54b40b1060 con 0x7f54b40612c0 2022-06-29T15:43:03.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.335+0000 7f54a67fc700 1 --2- 172.21.15.47:0/4176685995 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f549c0539e0 0x7f549c055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:03.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.335+0000 7f54a67fc700 1 -- 172.21.15.47:0/4176685995 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6079+0+0 (secure 0 0 0) 0x7f54b006b680 con 0x7f54b40612c0 2022-06-29T15:43:03.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.335+0000 7f54b91c3700 1 --2- 172.21.15.47:0/4176685995 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f549c0539e0 0x7f549c055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:03.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.335+0000 7f54b91c3700 1 --2- 172.21.15.47:0/4176685995 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f549c0539e0 0x7f549c055ea0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f54a800d000 tx=0x7f54a8007a00).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:03.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.343+0000 7f54a67fc700 1 -- 172.21.15.47:0/4176685995 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f54b0042490 con 0x7f54b40612c0 2022-06-29T15:43:03.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:03.471+0000 7f54bb427700 1 -- 172.21.15.47:0/4176685995 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f54b40573d0 con 0x7f54b40612c0 2022-06-29T15:43:04.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.195+0000 7f54a67fc700 1 -- 172.21.15.47:0/4176685995 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v29) v1 ==== 173+0+0 (secure 0 0 0) 0x7f54b0018020 con 0x7f54b40612c0 2022-06-29T15:43:04.205 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.199+0000 7f54bb427700 1 -- 172.21.15.47:0/4176685995 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f549c0539e0 msgr2=0x7f549c055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:04.205 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.199+0000 7f54bb427700 1 --2- 172.21.15.47:0/4176685995 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f549c0539e0 0x7f549c055ea0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f54a800d000 tx=0x7f54a8007a00).stop 2022-06-29T15:43:04.206 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.199+0000 7f54bb427700 1 -- 172.21.15.47:0/4176685995 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54b40612c0 msgr2=0x7f54b40b9cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:04.206 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.199+0000 7f54bb427700 1 --2- 172.21.15.47:0/4176685995 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54b40612c0 0x7f54b40b9cc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f54b0001110 tx=0x7f54b0007af0).stop 2022-06-29T15:43:04.206 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.199+0000 7f54bb427700 1 -- 172.21.15.47:0/4176685995 shutdown_connections 2022-06-29T15:43:04.206 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.199+0000 7f54bb427700 1 --2- 172.21.15.47:0/4176685995 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f549c0539e0 0x7f549c055ea0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:04.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.199+0000 7f54bb427700 1 --2- 172.21.15.47:0/4176685995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54b40b4300 0x7f54b40b15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:04.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.199+0000 7f54bb427700 1 --2- 172.21.15.47:0/4176685995 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54b40612c0 0x7f54b40b9cc0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:04.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.199+0000 7f54bb427700 1 --2- 172.21.15.47:0/4176685995 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f54b4060ee0 0x7f54b4142050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:04.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.199+0000 7f54bb427700 1 -- 172.21.15.47:0/4176685995 >> 172.21.15.47:0/4176685995 conn(0x7f54b405c510 msgr2=0x7f54b41444e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:04.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.199+0000 7f54bb427700 1 -- 172.21.15.47:0/4176685995 shutdown_connections 2022-06-29T15:43:04.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.199+0000 7f54bb427700 1 -- 172.21.15.47:0/4176685995 wait complete. 2022-06-29T15:43:04.208 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' removed 2022-06-29T15:43:04.221 INFO:tasks.ceph.mds.a:Restarting daemon 2022-06-29T15:43:04.221 INFO:tasks.ceph.mds.a:Stopping old one... 2022-06-29T15:43:04.222 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-06-29T15:43:04.222 INFO:teuthology.orchestra.run:waiting for 300 2022-06-29T15:43:04.223 INFO:tasks.ceph.mds.c:Restarting daemon 2022-06-29T15:43:04.223 INFO:tasks.ceph.mds.c:Stopping old one... 2022-06-29T15:43:04.223 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2022-06-29T15:43:04.223 INFO:teuthology.orchestra.run:waiting for 300 2022-06-29T15:43:04.224 INFO:tasks.ceph.mds.b:Restarting daemon 2022-06-29T15:43:04.224 INFO:tasks.ceph.mds.b:Stopping old one... 2022-06-29T15:43:04.224 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-06-29T15:43:04.225 INFO:teuthology.orchestra.run:waiting for 300 2022-06-29T15:43:04.238 INFO:tasks.ceph.mds.c:Stopped 2022-06-29T15:43:04.239 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2022-06-29T15:43:04.240 INFO:tasks.ceph.mds.a:Stopped 2022-06-29T15:43:04.240 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-06-29T15:43:04.242 INFO:tasks.ceph.mds.c:Started 2022-06-29T15:43:04.242 DEBUG:teuthology.parallel:result is None 2022-06-29T15:43:04.243 INFO:tasks.ceph.mds.b:Stopped 2022-06-29T15:43:04.244 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-06-29T15:43:04.244 INFO:tasks.ceph.mds.a:Started 2022-06-29T15:43:04.245 DEBUG:teuthology.parallel:result is None 2022-06-29T15:43:04.247 INFO:tasks.ceph.mds.b:Started 2022-06-29T15:43:04.247 DEBUG:teuthology.parallel:result is None 2022-06-29T15:43:04.248 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-06-29T15:43:04.319 INFO:tasks.ceph.mds.c.smithi047.stdout:starting mds.c at 2022-06-29T15:43:04.324 INFO:tasks.ceph.mds.b.smithi064.stdout:starting mds.b at 2022-06-29T15:43:04.328 INFO:tasks.ceph.mds.a.smithi047.stdout:starting mds.a at 2022-06-29T15:43:04.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.347+0000 7fb148e20700 1 Processor -- start 2022-06-29T15:43:04.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.347+0000 7fb148e20700 1 -- start start 2022-06-29T15:43:04.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.347+0000 7fb148e20700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb1440619b0 0x7fb1440606e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:04.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.347+0000 7fb148e20700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb1440b0f30 0x7fb1441408e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:04.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.347+0000 7fb148e20700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb144061d90 con 0x7fb1440b1a80 2022-06-29T15:43:04.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.347+0000 7fb148e20700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb144060d80 con 0x7fb1440b0f30 2022-06-29T15:43:04.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.347+0000 7fb148e20700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb14413ff00 con 0x7fb1440619b0 2022-06-29T15:43:04.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.347+0000 7fb141d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb1440b0f30 0x7fb1441408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:04.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.347+0000 7fb141d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb1440b0f30 0x7fb1441408e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46878/0 (socket says 172.21.15.47:46878) 2022-06-29T15:43:04.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.347+0000 7fb141d9b700 1 -- 172.21.15.47:0/1802434844 learned_addr learned my addr 172.21.15.47:0/1802434844 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:04.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.347+0000 7fb14259c700 1 --2- 172.21.15.47:0/1802434844 >> v2:172.21.15.64:3300/0 conn(0x7fb1440619b0 0x7fb1440606e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:04.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.351+0000 7fb141d9b700 1 -- 172.21.15.47:0/1802434844 >> v2:172.21.15.64:3300/0 conn(0x7fb1440619b0 msgr2=0x7fb1440606e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:43:04.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.351+0000 7fb141d9b700 1 --2- 172.21.15.47:0/1802434844 >> v2:172.21.15.64:3300/0 conn(0x7fb1440619b0 0x7fb1440606e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:04.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.351+0000 7fb141d9b700 1 -- 172.21.15.47:0/1802434844 >> v1:172.21.15.64:6789/0 conn(0x7fb1440b1a80 legacy=0x7fb144139a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:04.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.351+0000 7fb141d9b700 1 -- 172.21.15.47:0/1802434844 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1441401a0 con 0x7fb1440b0f30 2022-06-29T15:43:04.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.351+0000 7fb141d9b700 1 --2- 172.21.15.47:0/1802434844 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1440b0f30 0x7fb1441408e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7fb138002f70 tx=0x7fb138006eb0).ready entity=mon.0 client_cookie=91116f195ae68a38 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:04.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.351+0000 7fb14159a700 1 -- 172.21.15.47:0/1802434844 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb13800d5c0 con 0x7fb1440b0f30 2022-06-29T15:43:04.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.351+0000 7fb14159a700 1 -- 172.21.15.47:0/1802434844 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb138000e90 con 0x7fb1440b0f30 2022-06-29T15:43:04.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.351+0000 7fb14159a700 1 -- 172.21.15.47:0/1802434844 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb13800d950 con 0x7fb1440b0f30 2022-06-29T15:43:04.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.351+0000 7fb148e20700 1 -- 172.21.15.47:0/1802434844 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1440b0f30 msgr2=0x7fb1441408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:04.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.351+0000 7fb148e20700 1 --2- 172.21.15.47:0/1802434844 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1440b0f30 0x7fb1441408e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7fb138002f70 tx=0x7fb138006eb0).stop 2022-06-29T15:43:04.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.351+0000 7fb148e20700 1 -- 172.21.15.47:0/1802434844 shutdown_connections 2022-06-29T15:43:04.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.351+0000 7fb148e20700 1 --2- 172.21.15.47:0/1802434844 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1440b0f30 0x7fb1441408e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:04.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.351+0000 7fb148e20700 1 --2- 172.21.15.47:0/1802434844 >> v2:172.21.15.64:3300/0 conn(0x7fb1440619b0 0x7fb1440606e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:04.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.351+0000 7fb148e20700 1 -- 172.21.15.47:0/1802434844 >> 172.21.15.47:0/1802434844 conn(0x7fb14405c510 msgr2=0x7fb144063800 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:04.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.351+0000 7fb148e20700 1 -- 172.21.15.47:0/1802434844 shutdown_connections 2022-06-29T15:43:04.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.351+0000 7fb148e20700 1 -- 172.21.15.47:0/1802434844 wait complete. 2022-06-29T15:43:04.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.351+0000 7fb148e20700 1 Processor -- start 2022-06-29T15:43:04.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.351+0000 7fb148e20700 1 -- start start 2022-06-29T15:43:04.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.355+0000 7fb148e20700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1440619b0 0x7fb14413bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:04.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.355+0000 7fb148e20700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1440b0f30 0x7fb144065a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:04.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.355+0000 7fb148e20700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb1440b1a80 0x7fb144065f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:04.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.355+0000 7fb148e20700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1440663f0 con 0x7fb1440b0f30 2022-06-29T15:43:04.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.355+0000 7fb148e20700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb144064840 con 0x7fb1440619b0 2022-06-29T15:43:04.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.355+0000 7fb148e20700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1440667a0 con 0x7fb1440b1a80 2022-06-29T15:43:04.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.355+0000 7fb141d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1440b0f30 0x7fb144065a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:04.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.355+0000 7fb141d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1440b0f30 0x7fb144065a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46884/0 (socket says 172.21.15.47:46884) 2022-06-29T15:43:04.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.355+0000 7fb141d9b700 1 -- 172.21.15.47:0/3274834851 learned_addr learned my addr 172.21.15.47:0/3274834851 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:04.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.355+0000 7fb14259c700 1 --2- 172.21.15.47:0/3274834851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1440619b0 0x7fb14413bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:04.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.355+0000 7fb142d9d700 1 --2- 172.21.15.47:0/3274834851 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb1440b1a80 0x7fb144065f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:04.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.355+0000 7fb141d9b700 1 -- 172.21.15.47:0/3274834851 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb1440b1a80 msgr2=0x7fb144065f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:04.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.355+0000 7fb141d9b700 1 --2- 172.21.15.47:0/3274834851 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb1440b1a80 0x7fb144065f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:04.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.355+0000 7fb141d9b700 1 -- 172.21.15.47:0/3274834851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1440619b0 msgr2=0x7fb14413bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:04.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.355+0000 7fb141d9b700 1 --2- 172.21.15.47:0/3274834851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1440619b0 0x7fb14413bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:04.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.355+0000 7fb141d9b700 1 -- 172.21.15.47:0/3274834851 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb13800b1e0 con 0x7fb1440b0f30 2022-06-29T15:43:04.370 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.355+0000 7fb141d9b700 1 --2- 172.21.15.47:0/3274834851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1440b0f30 0x7fb144065a10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7fb144066530 tx=0x7fb138015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:04.370 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.355+0000 7fb1337fe700 1 -- 172.21.15.47:0/3274834851 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb13800d940 con 0x7fb1440b0f30 2022-06-29T15:43:04.370 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.355+0000 7fb1337fe700 1 -- 172.21.15.47:0/3274834851 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb138011bf0 con 0x7fb1440b0f30 2022-06-29T15:43:04.370 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.355+0000 7fb1337fe700 1 -- 172.21.15.47:0/3274834851 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb13801e460 con 0x7fb1440b0f30 2022-06-29T15:43:04.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.355+0000 7fb148e20700 1 -- 172.21.15.47:0/3274834851 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1440645a0 con 0x7fb1440b0f30 2022-06-29T15:43:04.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.355+0000 7fb148e20700 1 -- 172.21.15.47:0/3274834851 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb14413c460 con 0x7fb1440b0f30 2022-06-29T15:43:04.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.355+0000 7fb1337fe700 1 -- 172.21.15.47:0/3274834851 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb13801a040 con 0x7fb1440b0f30 2022-06-29T15:43:04.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.355+0000 7fb148e20700 1 -- 172.21.15.47:0/3274834851 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1440573d0 con 0x7fb1440b0f30 2022-06-29T15:43:04.372 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.359+0000 7fb1337fe700 1 --2- 172.21.15.47:0/3274834851 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb120053a70 0x7fb120055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:04.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.359+0000 7fb1337fe700 1 -- 172.21.15.47:0/3274834851 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fb13806cbb0 con 0x7fb1440b0f30 2022-06-29T15:43:04.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.359+0000 7fb14259c700 1 --2- 172.21.15.47:0/3274834851 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb120053a70 0x7fb120055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:04.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.359+0000 7fb14259c700 1 --2- 172.21.15.47:0/3274834851 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb120053a70 0x7fb120055f30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fb13c00b000 tx=0x7fb13c0091d0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:04.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.363+0000 7fb1337fe700 1 -- 172.21.15.47:0/3274834851 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb138073080 con 0x7fb1440b0f30 2022-06-29T15:43:04.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:04.503+0000 7fb148e20700 1 -- 172.21.15.47:0/3274834851 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7fb144065740 con 0x7fb1440b0f30 2022-06-29T15:43:05.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.363+0000 7fb1337fe700 1 -- 172.21.15.47:0/3274834851 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v32) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb138023020 con 0x7fb1440b0f30 2022-06-29T15:43:05.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.367+0000 7fb148e20700 1 -- 172.21.15.47:0/3274834851 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb120053a70 msgr2=0x7fb120055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:05.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.367+0000 7fb148e20700 1 --2- 172.21.15.47:0/3274834851 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb120053a70 0x7fb120055f30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fb13c00b000 tx=0x7fb13c0091d0).stop 2022-06-29T15:43:05.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.367+0000 7fb148e20700 1 -- 172.21.15.47:0/3274834851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1440b0f30 msgr2=0x7fb144065a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:05.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.367+0000 7fb148e20700 1 --2- 172.21.15.47:0/3274834851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1440b0f30 0x7fb144065a10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7fb144066530 tx=0x7fb138015000).stop 2022-06-29T15:43:05.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.367+0000 7fb148e20700 1 -- 172.21.15.47:0/3274834851 shutdown_connections 2022-06-29T15:43:05.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.367+0000 7fb148e20700 1 --2- 172.21.15.47:0/3274834851 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb1440b1a80 0x7fb144065f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.367+0000 7fb148e20700 1 --2- 172.21.15.47:0/3274834851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1440b0f30 0x7fb144065a10 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.367+0000 7fb148e20700 1 --2- 172.21.15.47:0/3274834851 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb120053a70 0x7fb120055f30 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.367+0000 7fb148e20700 1 --2- 172.21.15.47:0/3274834851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1440619b0 0x7fb14413bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:05.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.367+0000 7fb148e20700 1 -- 172.21.15.47:0/3274834851 >> 172.21.15.47:0/3274834851 conn(0x7fb14405c510 msgr2=0x7fb144136670 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:05.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.367+0000 7fb148e20700 1 -- 172.21.15.47:0/3274834851 shutdown_connections 2022-06-29T15:43:05.379 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.367+0000 7fb148e20700 1 -- 172.21.15.47:0/3274834851 wait complete. 2022-06-29T15:43:05.380 INFO:teuthology.orchestra.run.smithi047.stderr: removed all blacklist entries 2022-06-29T15:43:05.388 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2022-06-29T15:43:05.389 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-06-29T15:43:05.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.491+0000 7f1ae359e700 1 Processor -- start 2022-06-29T15:43:05.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.491+0000 7f1ae359e700 1 -- start start 2022-06-29T15:43:05.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.495+0000 7f1ae359e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1adc042560 0x7f1adc0417f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:05.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.495+0000 7f1ae359e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1adc0d88c0 0x7f1adc0d8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:05.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.495+0000 7f1ae359e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1adc041ce0 con 0x7f1adc03da40 2022-06-29T15:43:05.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.495+0000 7f1ae359e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1adc005d40 con 0x7f1adc042560 2022-06-29T15:43:05.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.495+0000 7f1ae359e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1adc0400d0 con 0x7f1adc0d88c0 2022-06-29T15:43:05.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.495+0000 7f1ae259c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1adc03da40 0x7f1adc005630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:60680/0 (socket says 172.21.15.47:60680) 2022-06-29T15:43:05.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.495+0000 7f1ae259c700 1 -- 172.21.15.47:0/2964602415 learned_addr learned my addr 172.21.15.47:0/2964602415 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:43:05.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.495+0000 7f1ae1d9b700 1 --2- 172.21.15.47:0/2964602415 >> v2:172.21.15.47:3300/0 conn(0x7f1adc042560 0x7f1adc0417f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:05.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.495+0000 7f1ae159a700 1 -- 172.21.15.47:0/2964602415 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2290023887 0 0) 0x7f1adc041ce0 con 0x7f1adc03da40 2022-06-29T15:43:05.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.495+0000 7f1ae2d9d700 1 --2- 172.21.15.47:0/2964602415 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1adc0d88c0 0x7f1adc0d8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:05.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.495+0000 7f1ae159a700 1 -- 172.21.15.47:0/2964602415 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ac4003260 con 0x7f1adc03da40 2022-06-29T15:43:05.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.495+0000 7f1ae159a700 1 -- 172.21.15.47:0/2964602415 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3804161270 0 0) 0x7f1ac4003260 con 0x7f1adc03da40 2022-06-29T15:43:05.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.495+0000 7f1ae1d9b700 1 -- 172.21.15.47:0/2964602415 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1adc0d88c0 msgr2=0x7f1adc0d8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:43:05.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.495+0000 7f1ae1d9b700 1 --2- 172.21.15.47:0/2964602415 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1adc0d88c0 0x7f1adc0d8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:05.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.495+0000 7f1ae1d9b700 1 -- 172.21.15.47:0/2964602415 >> v1:172.21.15.47:6789/0 conn(0x7f1adc03da40 legacy=0x7f1adc005630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:05.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.495+0000 7f1ae1d9b700 1 -- 172.21.15.47:0/2964602415 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1adc03de20 con 0x7f1adc042560 2022-06-29T15:43:05.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.495+0000 7f1ae1d9b700 1 --2- 172.21.15.47:0/2964602415 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1adc042560 0x7f1adc0417f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f1acc001820 tx=0x7f1ad4006df0).ready entity=mon.0 client_cookie=b1f01721cb3fd98a server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:05.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.495+0000 7f1ae159a700 1 -- 172.21.15.47:0/2964602415 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1acc001680 con 0x7f1adc042560 2022-06-29T15:43:05.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.495+0000 7f1ae159a700 1 -- 172.21.15.47:0/2964602415 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ad4000e90 con 0x7f1adc042560 2022-06-29T15:43:05.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.495+0000 7f1ae159a700 1 -- 172.21.15.47:0/2964602415 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ad4012460 con 0x7f1adc042560 2022-06-29T15:43:05.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.495+0000 7f1ae359e700 1 -- 172.21.15.47:0/2964602415 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1adc042560 msgr2=0x7f1adc0417f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:05.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.495+0000 7f1ae359e700 1 --2- 172.21.15.47:0/2964602415 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1adc042560 0x7f1adc0417f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f1acc001820 tx=0x7f1ad4006df0).stop 2022-06-29T15:43:05.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.495+0000 7f1ae359e700 1 -- 172.21.15.47:0/2964602415 shutdown_connections 2022-06-29T15:43:05.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.495+0000 7f1ae359e700 1 --2- 172.21.15.47:0/2964602415 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1adc0d88c0 0x7f1adc0d8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:05.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.495+0000 7f1ae359e700 1 --2- 172.21.15.47:0/2964602415 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1adc042560 0x7f1adc0417f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:05.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.495+0000 7f1ae359e700 1 -- 172.21.15.47:0/2964602415 >> 172.21.15.47:0/2964602415 conn(0x7f1adc0462c0 msgr2=0x7f1adc0466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:05.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.495+0000 7f1ae359e700 1 -- 172.21.15.47:0/2964602415 shutdown_connections 2022-06-29T15:43:05.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.495+0000 7f1ae359e700 1 -- 172.21.15.47:0/2964602415 wait complete. 2022-06-29T15:43:05.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.499+0000 7f1ae359e700 1 Processor -- start 2022-06-29T15:43:05.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.499+0000 7f1ae359e700 1 -- start start 2022-06-29T15:43:05.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.499+0000 7f1ae359e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1adc03da40 0x7f1adc0db0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:05.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.499+0000 7f1ae359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1adc042560 0x7f1adc0eb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:05.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.499+0000 7f1ae359e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1adc0d88c0 0x7f1adc0e7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:05.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.499+0000 7f1ae359e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1adc0ebaa0 con 0x7f1adc03da40 2022-06-29T15:43:05.513 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.499+0000 7f1ae359e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1adc039f50 con 0x7f1adc042560 2022-06-29T15:43:05.513 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.499+0000 7f1ae359e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1adc03a1b0 con 0x7f1adc0d88c0 2022-06-29T15:43:05.513 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.499+0000 7f1ae259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1adc03da40 0x7f1adc0db0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:05.513 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.499+0000 7f1ae259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1adc03da40 0x7f1adc0db0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46900/0 (socket says 172.21.15.47:46900) 2022-06-29T15:43:05.514 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.499+0000 7f1ae259c700 1 -- 172.21.15.47:0/3076520892 learned_addr learned my addr 172.21.15.47:0/3076520892 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:05.514 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.499+0000 7f1ae1d9b700 1 --2- 172.21.15.47:0/3076520892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1adc042560 0x7f1adc0eb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:05.514 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.499+0000 7f1ae2d9d700 1 --2- 172.21.15.47:0/3076520892 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1adc0d88c0 0x7f1adc0e7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:05.514 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.503+0000 7f1ae259c700 1 -- 172.21.15.47:0/3076520892 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1adc0d88c0 msgr2=0x7f1adc0e7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:05.515 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.503+0000 7f1ae259c700 1 --2- 172.21.15.47:0/3076520892 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1adc0d88c0 0x7f1adc0e7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:05.515 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.503+0000 7f1ae259c700 1 -- 172.21.15.47:0/3076520892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1adc042560 msgr2=0x7f1adc0eb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:05.515 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.503+0000 7f1ae259c700 1 --2- 172.21.15.47:0/3076520892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1adc042560 0x7f1adc0eb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:05.515 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.503+0000 7f1ae259c700 1 -- 172.21.15.47:0/3076520892 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad400b180 con 0x7f1adc03da40 2022-06-29T15:43:05.516 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.503+0000 7f1ae259c700 1 --2- 172.21.15.47:0/3076520892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1adc03da40 0x7f1adc0db0a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f1acc0018e0 tx=0x7f1acc00c740).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:05.516 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.503+0000 7f1adb7fe700 1 -- 172.21.15.47:0/3076520892 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1acc007170 con 0x7f1adc03da40 2022-06-29T15:43:05.516 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.503+0000 7f1adb7fe700 1 -- 172.21.15.47:0/3076520892 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1acc0072d0 con 0x7f1adc03da40 2022-06-29T15:43:05.517 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.503+0000 7f1adb7fe700 1 -- 172.21.15.47:0/3076520892 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1acc004710 con 0x7f1adc03da40 2022-06-29T15:43:05.517 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.503+0000 7f1ae359e700 1 -- 172.21.15.47:0/3076520892 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1adc03a450 con 0x7f1adc03da40 2022-06-29T15:43:05.517 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.503+0000 7f1ae359e700 1 -- 172.21.15.47:0/3076520892 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1adc03a980 con 0x7f1adc03da40 2022-06-29T15:43:05.518 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.503+0000 7f1adb7fe700 1 -- 172.21.15.47:0/3076520892 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1acc004870 con 0x7f1adc03da40 2022-06-29T15:43:05.518 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.503+0000 7f1ae359e700 1 -- 172.21.15.47:0/3076520892 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1adc0398d0 con 0x7f1adc03da40 2022-06-29T15:43:05.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.503+0000 7f1adb7fe700 1 --2- 172.21.15.47:0/3076520892 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1ac8053a10 0x7f1ac8055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:05.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.503+0000 7f1adb7fe700 1 -- 172.21.15.47:0/3076520892 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f1acc06b6f0 con 0x7f1adc03da40 2022-06-29T15:43:05.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.503+0000 7f1ae1d9b700 1 --2- 172.21.15.47:0/3076520892 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1ac8053a10 0x7f1ac8055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:05.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.503+0000 7f1ae1d9b700 1 --2- 172.21.15.47:0/3076520892 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1ac8053a10 0x7f1ac8055ed0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f1ad401a000 tx=0x7f1ad4014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:05.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.515+0000 7f1adb7fe700 1 -- 172.21.15.47:0/3076520892 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1adc0398d0 con 0x7f1adc03da40 2022-06-29T15:43:05.687 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.679+0000 7f1ae359e700 1 -- 172.21.15.47:0/3076520892 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f1adc0398d0 con 0x7f1adc03da40 2022-06-29T15:43:05.688 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.683+0000 7f1adb7fe700 1 -- 172.21.15.47:0/3076520892 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v7) v1 ==== 80+0+5621 (secure 0 0 0) 0x7f1adc0398d0 con 0x7f1adc03da40 2022-06-29T15:43:05.689 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:43:05.689 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2022-06-29T15:43:05.689 INFO:teuthology.orchestra.run.smithi047.stdout: "auth_dump": [ 2022-06-29T15:43:05.689 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:05.690 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.a", 2022-06-29T15:43:05.690 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxiQJFFMxAA9Tv7FJD9u/BKvbkuKld6Kg==", 2022-06-29T15:43:05.690 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:43:05.690 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:43:05.691 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:43:05.691 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:43:05.691 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:43:05.691 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:05.692 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:05.692 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:05.692 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.b", 2022-06-29T15:43:05.692 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC2crxibt6FARAAEf2bwXKFhXKtqr1nN0rdXQ==", 2022-06-29T15:43:05.692 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:43:05.693 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:43:05.693 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:43:05.693 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:43:05.693 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:43:05.694 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:05.694 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:05.694 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:05.694 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.c", 2022-06-29T15:43:05.694 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxi/4ZMORAANpLbgTm3ehqi80USc3swGg==", 2022-06-29T15:43:05.695 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:43:05.695 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:43:05.695 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:43:05.695 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:43:05.696 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:43:05.696 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:05.696 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:05.696 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:05.696 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.0", 2022-06-29T15:43:05.697 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC2crxiY0YsMxAAZivb+CxL/w7vZwQv37Y4oQ==", 2022-06-29T15:43:05.697 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:43:05.697 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:43:05.697 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:43:05.697 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:43:05.698 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:05.698 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:05.698 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:05.698 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.1", 2022-06-29T15:43:05.699 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC5crxitLkKFxAA8+++LUUKLiO9SAuAfYgT5g==", 2022-06-29T15:43:05.699 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:43:05.699 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:43:05.699 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:43:05.699 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:43:05.700 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:05.700 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:05.700 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:05.700 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.2", 2022-06-29T15:43:05.701 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC7crxi1oM9NxAA2FfLdNt32YPwlyNdGQpaQA==", 2022-06-29T15:43:05.701 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:43:05.701 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:43:05.701 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:43:05.701 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:43:05.702 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:05.702 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:05.702 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:05.702 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.3", 2022-06-29T15:43:05.703 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC+crxiTsBgGhAA+gLBBDaskPbmRNWzgKwcHA==", 2022-06-29T15:43:05.703 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:43:05.703 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:43:05.703 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:43:05.703 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:43:05.704 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:05.704 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:05.704 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:05.704 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.4", 2022-06-29T15:43:05.705 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDDcrxivJebIhAAfwWJa7tK4hamkk1/1CSSfw==", 2022-06-29T15:43:05.705 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:43:05.705 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:43:05.705 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:43:05.705 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:43:05.706 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:05.706 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:05.706 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:05.706 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.5", 2022-06-29T15:43:05.706 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDGcrxijuozBhAAt+Dx3wH3WKp3DrbCbhmh8Q==", 2022-06-29T15:43:05.707 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:43:05.707 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:43:05.707 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:43:05.707 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:43:05.708 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:05.708 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:05.708 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:05.708 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.6", 2022-06-29T15:43:05.708 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDIcrxi91kXJxAA9JX7n98P394bD41Q9LUdvQ==", 2022-06-29T15:43:05.709 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:43:05.709 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:43:05.709 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:43:05.709 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:43:05.709 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:05.710 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:05.710 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:05.710 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.7", 2022-06-29T15:43:05.710 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDLcrxiHj0ZDRAASy6fAix8TmEZbAGtuFCfFg==", 2022-06-29T15:43:05.711 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:43:05.711 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:43:05.711 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:43:05.711 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:43:05.711 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:05.712 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:05.712 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:05.712 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.0", 2022-06-29T15:43:05.712 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC2crxiVDlfBxAAezkUfg4e2g0Rfakaa/Ek1Q==", 2022-06-29T15:43:05.713 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:43:05.713 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:43:05.713 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow r", 2022-06-29T15:43:05.713 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow rw", 2022-06-29T15:43:05.713 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rwx" 2022-06-29T15:43:05.714 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:05.714 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:05.714 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:05.714 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.1", 2022-06-29T15:43:05.715 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC2crxisXJ1CxAAhl07oF7AW66NdzfvvZTJmA==", 2022-06-29T15:43:05.715 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:43:05.715 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:43:05.715 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow r", 2022-06-29T15:43:05.715 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow rw", 2022-06-29T15:43:05.716 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rwx" 2022-06-29T15:43:05.716 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:05.716 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:05.716 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:05.716 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.admin", 2022-06-29T15:43:05.717 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxiW7HSFRAAhYqoGU9GFqj2HfUVGValSA==", 2022-06-29T15:43:05.717 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:43:05.717 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2022-06-29T15:43:05.717 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:43:05.718 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:43:05.718 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:43:05.718 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:05.718 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:05.718 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:05.719 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.bootstrap-mds", 2022-06-29T15:43:05.719 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDWcrxiRHrXAhAAOpausgStY8DSvAsQA3D+cA==", 2022-06-29T15:43:05.719 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:43:05.719 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile bootstrap-mds" 2022-06-29T15:43:05.719 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:05.720 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:05.720 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:05.720 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.bootstrap-mgr", 2022-06-29T15:43:05.720 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDWcrxi45bXAhAADgwZq8KXvVylqtorpe0Igg==", 2022-06-29T15:43:05.721 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:43:05.721 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile bootstrap-mgr" 2022-06-29T15:43:05.721 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:05.721 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:05.721 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:05.722 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.bootstrap-osd", 2022-06-29T15:43:05.722 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDWcrxiTLLXAhAAQAab3u6mS1Jx0h1S07m5jA==", 2022-06-29T15:43:05.722 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:43:05.723 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile bootstrap-osd" 2022-06-29T15:43:05.723 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:05.723 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:05.723 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:05.723 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.bootstrap-rbd", 2022-06-29T15:43:05.724 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDWcrxizMjXAhAAMtyhW7xjsEW5a3m/Gt2daQ==", 2022-06-29T15:43:05.724 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:43:05.724 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile bootstrap-rbd" 2022-06-29T15:43:05.724 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:05.724 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:05.725 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:05.725 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.bootstrap-rbd-mirror", 2022-06-29T15:43:05.725 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDWcrxi0uDXAhAAwmDWeLIZacBWr6tG8XZmTQ==", 2022-06-29T15:43:05.725 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:43:05.726 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2022-06-29T15:43:05.726 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:05.726 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:05.726 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:05.726 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.bootstrap-rgw", 2022-06-29T15:43:05.727 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDWcrxiCfjXAhAAHoIvs+RbLisqL4YduY6pxQ==", 2022-06-29T15:43:05.727 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:43:05.727 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile bootstrap-rgw" 2022-06-29T15:43:05.727 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:05.727 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:05.728 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:05.728 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.x", 2022-06-29T15:43:05.728 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxidmuIMBAAcpNlYVu5+icCHg+yIjcKXw==", 2022-06-29T15:43:05.728 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:43:05.729 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2022-06-29T15:43:05.729 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2022-06-29T15:43:05.729 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:43:05.729 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:05.730 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:05.730 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:05.730 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.y", 2022-06-29T15:43:05.730 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxisSMFLBAAv03r4OoPjOP9nI9G/LK0iw==", 2022-06-29T15:43:05.730 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:43:05.731 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2022-06-29T15:43:05.731 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2022-06-29T15:43:05.731 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:43:05.731 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:05.731 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:05.732 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:43:05.732 INFO:teuthology.orchestra.run.smithi047.stdout:} 2022-06-29T15:43:05.734 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.683+0000 7f1ae359e700 1 -- 172.21.15.47:0/3076520892 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1ac8053a10 msgr2=0x7f1ac8055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:05.734 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.683+0000 7f1ae359e700 1 --2- 172.21.15.47:0/3076520892 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1ac8053a10 0x7f1ac8055ed0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f1ad401a000 tx=0x7f1ad4014000).stop 2022-06-29T15:43:05.734 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.683+0000 7f1ae359e700 1 -- 172.21.15.47:0/3076520892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1adc03da40 msgr2=0x7f1adc0db0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:05.734 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.683+0000 7f1ae359e700 1 --2- 172.21.15.47:0/3076520892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1adc03da40 0x7f1adc0db0a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f1acc0018e0 tx=0x7f1acc00c740).stop 2022-06-29T15:43:05.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.683+0000 7f1ae359e700 1 -- 172.21.15.47:0/3076520892 shutdown_connections 2022-06-29T15:43:05.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.683+0000 7f1ae359e700 1 --2- 172.21.15.47:0/3076520892 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1adc0d88c0 0x7f1adc0e7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:05.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.683+0000 7f1ae359e700 1 --2- 172.21.15.47:0/3076520892 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1ac8053a10 0x7f1ac8055ed0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:05.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.683+0000 7f1ae359e700 1 --2- 172.21.15.47:0/3076520892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1adc042560 0x7f1adc0eb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:05.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.683+0000 7f1ae359e700 1 --2- 172.21.15.47:0/3076520892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1adc03da40 0x7f1adc0db0a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:05.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.683+0000 7f1ae359e700 1 -- 172.21.15.47:0/3076520892 >> 172.21.15.47:0/3076520892 conn(0x7f1adc0462c0 msgr2=0x7f1adc0d8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:05.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.683+0000 7f1ae359e700 1 -- 172.21.15.47:0/3076520892 shutdown_connections 2022-06-29T15:43:05.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.683+0000 7f1ae359e700 1 -- 172.21.15.47:0/3076520892 wait complete. 2022-06-29T15:43:05.737 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2022-06-29T15:43:05.856 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.835+0000 7fad8559f700 1 Processor -- start 2022-06-29T15:43:05.857 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.835+0000 7fad8559f700 1 -- start start 2022-06-29T15:43:05.858 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.839+0000 7fad8559f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fad8005c510 0x7fad80143bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:05.858 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.839+0000 7fad8559f700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fad800b2c40 0x7fad800b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:05.858 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.839+0000 7fad8559f700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad80142f90 con 0x7fad8005c8f0 2022-06-29T15:43:05.859 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.839+0000 7fad8559f700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad801431f0 con 0x7fad8005c510 2022-06-29T15:43:05.859 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.839+0000 7fad8559f700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad80143490 con 0x7fad800b2c40 2022-06-29T15:43:05.860 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.839+0000 7fad7ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fad8005c510 0x7fad80143bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:05.860 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.839+0000 7fad84d9e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fad800b2c40 0x7fad800b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:05.861 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.839+0000 7fad7ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fad8005c510 0x7fad80143bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:52612/0 (socket says 172.21.15.47:52612) 2022-06-29T15:43:05.861 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.839+0000 7fad7f7fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fad8005c8f0 0x7fad800b4100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:40026/0 (socket says 172.21.15.47:40026) 2022-06-29T15:43:05.862 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.839+0000 7fad7ffff700 1 -- 172.21.15.47:0/2735812229 learned_addr learned my addr 172.21.15.47:0/2735812229 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:05.862 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.839+0000 7fad7ffff700 1 -- 172.21.15.47:0/2735812229 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fad800b2c40 msgr2=0x7fad800b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:43:05.863 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.839+0000 7fad7ffff700 1 --2- 172.21.15.47:0/2735812229 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fad800b2c40 0x7fad800b30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:05.863 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.839+0000 7fad7effd700 1 -- 172.21.15.47:0/2735812229 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1892994735 0 0) 0x7fad80142f90 con 0x7fad8005c8f0 2022-06-29T15:43:05.863 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.839+0000 7fad7ffff700 1 -- 172.21.15.47:0/2735812229 >> v1:172.21.15.64:6789/0 conn(0x7fad8005c8f0 legacy=0x7fad800b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:05.864 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.839+0000 7fad7ffff700 1 -- 172.21.15.47:0/2735812229 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad80143730 con 0x7fad8005c510 2022-06-29T15:43:05.864 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.839+0000 7fad84d9e700 1 --2- 172.21.15.47:0/2735812229 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fad800b2c40 0x7fad800b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:43:05.865 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.839+0000 7fad7ffff700 1 --2- 172.21.15.47:0/2735812229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad8005c510 0x7fad80143bd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fad70002f70 tx=0x7fad70006eb0).ready entity=mon.1 client_cookie=a93ae8430fe42725 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:05.865 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.839+0000 7fad7effd700 1 -- 172.21.15.47:0/2735812229 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad7000d570 con 0x7fad8005c510 2022-06-29T15:43:05.866 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.839+0000 7fad7effd700 1 -- 172.21.15.47:0/2735812229 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad70000e90 con 0x7fad8005c510 2022-06-29T15:43:05.866 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.839+0000 7fad7effd700 1 -- 172.21.15.47:0/2735812229 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad7000d8c0 con 0x7fad8005c510 2022-06-29T15:43:05.867 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.839+0000 7fad8559f700 1 -- 172.21.15.47:0/2735812229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad8005c510 msgr2=0x7fad80143bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:05.867 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.839+0000 7fad8559f700 1 --2- 172.21.15.47:0/2735812229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad8005c510 0x7fad80143bd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fad70002f70 tx=0x7fad70006eb0).stop 2022-06-29T15:43:05.867 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.843+0000 7fad8559f700 1 -- 172.21.15.47:0/2735812229 shutdown_connections 2022-06-29T15:43:05.868 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.843+0000 7fad8559f700 1 --2- 172.21.15.47:0/2735812229 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fad800b2c40 0x7fad800b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:05.868 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.843+0000 7fad8559f700 1 --2- 172.21.15.47:0/2735812229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad8005c510 0x7fad80143bd0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:05.869 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.843+0000 7fad8559f700 1 -- 172.21.15.47:0/2735812229 >> 172.21.15.47:0/2735812229 conn(0x7fad80060e00 msgr2=0x7fad800611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:05.869 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.843+0000 7fad8559f700 1 -- 172.21.15.47:0/2735812229 shutdown_connections 2022-06-29T15:43:05.870 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.843+0000 7fad8559f700 1 -- 172.21.15.47:0/2735812229 wait complete. 2022-06-29T15:43:05.870 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.843+0000 7fad8559f700 1 Processor -- start 2022-06-29T15:43:05.870 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.843+0000 7fad8559f700 1 -- start start 2022-06-29T15:43:05.871 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.843+0000 7fad8559f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad8005c510 0x7fad8013fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:05.871 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.843+0000 7fad8559f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad8005c8f0 0x7fad80155b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:05.872 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.843+0000 7fad8559f700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fad800b2c40 0x7fad80144f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:05.872 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.843+0000 7fad8559f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad801472b0 con 0x7fad8005c510 2022-06-29T15:43:05.873 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.843+0000 7fad8559f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad80156700 con 0x7fad8005c8f0 2022-06-29T15:43:05.873 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.843+0000 7fad8559f700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad8013d440 con 0x7fad800b2c40 2022-06-29T15:43:05.874 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.843+0000 7fad7ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad8005c510 0x7fad8013fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:05.874 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.843+0000 7fad7ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad8005c510 0x7fad8013fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46914/0 (socket says 172.21.15.47:46914) 2022-06-29T15:43:05.874 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.843+0000 7fad7ffff700 1 -- 172.21.15.47:0/911772586 learned_addr learned my addr 172.21.15.47:0/911772586 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:05.875 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.843+0000 7fad84d9e700 1 --2- 172.21.15.47:0/911772586 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fad800b2c40 0x7fad80144f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:05.875 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.843+0000 7fad7f7fe700 1 --2- 172.21.15.47:0/911772586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad8005c8f0 0x7fad80155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:05.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.843+0000 7fad7ffff700 1 -- 172.21.15.47:0/911772586 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fad800b2c40 msgr2=0x7fad80144f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:05.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.847+0000 7fad7ffff700 1 --2- 172.21.15.47:0/911772586 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fad800b2c40 0x7fad80144f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:05.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.847+0000 7fad7ffff700 1 -- 172.21.15.47:0/911772586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad8005c8f0 msgr2=0x7fad80155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:05.877 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.847+0000 7fad7ffff700 1 --2- 172.21.15.47:0/911772586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad8005c8f0 0x7fad80155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:05.877 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.847+0000 7fad7ffff700 1 -- 172.21.15.47:0/911772586 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad78009470 con 0x7fad8005c510 2022-06-29T15:43:05.877 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.847+0000 7fad7ffff700 1 --2- 172.21.15.47:0/911772586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad8005c510 0x7fad8013fb50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7fad70002f70 tx=0x7fad70015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:05.877 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.847+0000 7fad7d7fa700 1 -- 172.21.15.47:0/911772586 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad700068b0 con 0x7fad8005c510 2022-06-29T15:43:05.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.847+0000 7fad7d7fa700 1 -- 172.21.15.47:0/911772586 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad7000d8f0 con 0x7fad8005c510 2022-06-29T15:43:05.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.847+0000 7fad8559f700 1 -- 172.21.15.47:0/911772586 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad7000b1e0 con 0x7fad8005c510 2022-06-29T15:43:05.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.847+0000 7fad7d7fa700 1 -- 172.21.15.47:0/911772586 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad70011e20 con 0x7fad8005c510 2022-06-29T15:43:05.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.847+0000 7fad8559f700 1 -- 172.21.15.47:0/911772586 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad801561c0 con 0x7fad8005c510 2022-06-29T15:43:05.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.847+0000 7fad8559f700 1 -- 172.21.15.47:0/911772586 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad80156000 con 0x7fad8005c510 2022-06-29T15:43:05.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.847+0000 7fad7d7fa700 1 -- 172.21.15.47:0/911772586 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fad70025030 con 0x7fad8005c510 2022-06-29T15:43:05.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.847+0000 7fad7d7fa700 1 --2- 172.21.15.47:0/911772586 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fad64053a70 0x7fad64055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:05.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.847+0000 7fad7d7fa700 1 -- 172.21.15.47:0/911772586 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fad7006cdb0 con 0x7fad8005c510 2022-06-29T15:43:05.880 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.847+0000 7fad7f7fe700 1 --2- 172.21.15.47:0/911772586 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fad64053a70 0x7fad64055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:05.880 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.851+0000 7fad7f7fe700 1 --2- 172.21.15.47:0/911772586 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fad64053a70 0x7fad64055f30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7fad6800d000 tx=0x7fad68009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:05.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:05.855+0000 7fad7d7fa700 1 -- 172.21.15.47:0/911772586 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fad70043eb0 con 0x7fad8005c510 2022-06-29T15:43:06.027 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.019+0000 7fad8559f700 1 -- 172.21.15.47:0/911772586 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fad8013f840 con 0x7fad8005c510 2022-06-29T15:43:06.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.027+0000 7fad7d7fa700 1 -- 172.21.15.47:0/911772586 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 updated v8) v1 ==== 97+0+0 (secure 0 0 0) 0x7fad70023020 con 0x7fad8005c510 2022-06-29T15:43:06.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.031+0000 7fad8559f700 1 -- 172.21.15.47:0/911772586 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fad64053a70 msgr2=0x7fad64055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:06.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.031+0000 7fad8559f700 1 --2- 172.21.15.47:0/911772586 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fad64053a70 0x7fad64055f30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7fad6800d000 tx=0x7fad68009040).stop 2022-06-29T15:43:06.038 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.031+0000 7fad8559f700 1 -- 172.21.15.47:0/911772586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad8005c510 msgr2=0x7fad8013fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:06.038 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.031+0000 7fad8559f700 1 --2- 172.21.15.47:0/911772586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad8005c510 0x7fad8013fb50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7fad70002f70 tx=0x7fad70015000).stop 2022-06-29T15:43:06.038 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.031+0000 7fad8559f700 1 -- 172.21.15.47:0/911772586 shutdown_connections 2022-06-29T15:43:06.038 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.031+0000 7fad8559f700 1 --2- 172.21.15.47:0/911772586 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fad800b2c40 0x7fad80144f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:06.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.031+0000 7fad8559f700 1 --2- 172.21.15.47:0/911772586 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fad64053a70 0x7fad64055f30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:06.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.031+0000 7fad8559f700 1 --2- 172.21.15.47:0/911772586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad8005c8f0 0x7fad80155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:06.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.031+0000 7fad8559f700 1 --2- 172.21.15.47:0/911772586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad8005c510 0x7fad8013fb50 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:06.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.031+0000 7fad8559f700 1 -- 172.21.15.47:0/911772586 >> 172.21.15.47:0/911772586 conn(0x7fad80060e00 msgr2=0x7fad800b3620 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:06.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.031+0000 7fad8559f700 1 -- 172.21.15.47:0/911772586 shutdown_connections 2022-06-29T15:43:06.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.031+0000 7fad8559f700 1 -- 172.21.15.47:0/911772586 wait complete. 2022-06-29T15:43:06.040 INFO:teuthology.orchestra.run.smithi047.stderr:updated 2022-06-29T15:43:06.050 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2022-06-29T15:43:06.156 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.147+0000 7f46f19bf700 1 Processor -- start 2022-06-29T15:43:06.156 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.147+0000 7f46f19bf700 1 -- start start 2022-06-29T15:43:06.161 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.151+0000 7f46f19bf700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f46ec0b2120 0x7f46ec0b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:06.161 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.151+0000 7f46f19bf700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f46ec0b1260 0x7f46ec14b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:06.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.151+0000 7f46f19bf700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46ec0b0740 con 0x7f46ec05f0b0 2022-06-29T15:43:06.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.151+0000 7f46f19bf700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46ec147460 con 0x7f46ec0b2120 2022-06-29T15:43:06.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.151+0000 7f46f19bf700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46ec0b0970 con 0x7f46ec0b1260 2022-06-29T15:43:06.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.151+0000 7f46ea7fc700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f46ec05f0b0 0x7f46ec147af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:60708/0 (socket says 172.21.15.47:60708) 2022-06-29T15:43:06.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.151+0000 7f46ea7fc700 1 -- 172.21.15.47:0/3462677147 learned_addr learned my addr 172.21.15.47:0/3462677147 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:43:06.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.151+0000 7f46eaffd700 1 --2- 172.21.15.47:0/3462677147 >> v2:172.21.15.47:3300/0 conn(0x7f46ec0b2120 0x7f46ec0b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:06.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.151+0000 7f46eb7fe700 1 --2- 172.21.15.47:0/3462677147 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f46ec0b1260 0x7f46ec14b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:06.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.151+0000 7f46e9ffb700 1 -- 172.21.15.47:0/3462677147 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 684983749 0 0) 0x7f46ec0b0740 con 0x7f46ec05f0b0 2022-06-29T15:43:06.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.151+0000 7f46e9ffb700 1 -- 172.21.15.47:0/3462677147 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46d0003260 con 0x7f46ec05f0b0 2022-06-29T15:43:06.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.151+0000 7f46e9ffb700 1 -- 172.21.15.47:0/3462677147 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1292206234 0 0) 0x7f46d0003260 con 0x7f46ec05f0b0 2022-06-29T15:43:06.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.151+0000 7f46e9ffb700 1 -- 172.21.15.47:0/3462677147 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f46ec0b1260 msgr2=0x7f46ec14b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:43:06.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.151+0000 7f46e9ffb700 1 --2- 172.21.15.47:0/3462677147 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f46ec0b1260 0x7f46ec14b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:06.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.151+0000 7f46e9ffb700 1 -- 172.21.15.47:0/3462677147 >> v2:172.21.15.47:3300/0 conn(0x7f46ec0b2120 msgr2=0x7f46ec0b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:43:06.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.151+0000 7f46e9ffb700 1 --2- 172.21.15.47:0/3462677147 >> v2:172.21.15.47:3300/0 conn(0x7f46ec0b2120 0x7f46ec0b2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:06.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.151+0000 7f46e9ffb700 1 -- 172.21.15.47:0/3462677147 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46ec0b0c10 con 0x7f46ec05f0b0 2022-06-29T15:43:06.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.151+0000 7f46e9ffb700 1 -- 172.21.15.47:0/3462677147 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7f46dc001660 con 0x7f46ec05f0b0 2022-06-29T15:43:06.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.151+0000 7f46e9ffb700 1 -- 172.21.15.47:0/3462677147 >> v1:172.21.15.47:6789/0 conn(0x7f46ec05f0b0 legacy=0x7f46ec147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:06.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.151+0000 7f46e9ffb700 1 --2- 172.21.15.47:0/3462677147 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f46ec0b1260 0x7f46d0003750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:06.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.151+0000 7f46e9ffb700 1 --2- 172.21.15.47:0/3462677147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46d0003c80 0x7f46d0004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:06.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.151+0000 7f46e9ffb700 1 --2- 172.21.15.47:0/3462677147 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46d00049f0 0x7f46d0004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:06.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.151+0000 7f46e9ffb700 1 -- 172.21.15.47:0/3462677147 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46ec0b0970 con 0x7f46d00049f0 2022-06-29T15:43:06.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.151+0000 7f46e9ffb700 1 -- 172.21.15.47:0/3462677147 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d000b750 con 0x7f46d0003c80 2022-06-29T15:43:06.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.151+0000 7f46e9ffb700 1 -- 172.21.15.47:0/3462677147 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46d000b890 con 0x7f46ec0b1260 2022-06-29T15:43:06.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.151+0000 7f46eb7fe700 1 --2- 172.21.15.47:0/3462677147 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46d00049f0 0x7f46d0004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:06.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.151+0000 7f46eaffd700 1 --2- 172.21.15.47:0/3462677147 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f46ec0b1260 0x7f46d0003750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:06.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.151+0000 7f46ea7fc700 1 --2- 172.21.15.47:0/3462677147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46d0003c80 0x7f46d0004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:06.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.151+0000 7f46eb7fe700 1 -- 172.21.15.47:0/3462677147 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f46ec0b1260 msgr2=0x7f46d0003750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:43:06.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.151+0000 7f46eb7fe700 1 --2- 172.21.15.47:0/3462677147 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f46ec0b1260 0x7f46d0003750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:06.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.151+0000 7f46eb7fe700 1 -- 172.21.15.47:0/3462677147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46d0003c80 msgr2=0x7f46d0004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:43:06.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.151+0000 7f46eb7fe700 1 --2- 172.21.15.47:0/3462677147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46d0003c80 0x7f46d0004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:06.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.151+0000 7f46eb7fe700 1 -- 172.21.15.47:0/3462677147 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d0003260 con 0x7f46d00049f0 2022-06-29T15:43:06.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.151+0000 7f46eb7fe700 1 --2- 172.21.15.47:0/3462677147 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46d00049f0 0x7f46d0004e70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7f46d00054d0 tx=0x7f46e40091d0).ready entity=mon.0 client_cookie=887889bea156a418 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:06.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.151+0000 7f46e9ffb700 1 -- 172.21.15.47:0/3462677147 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f46e400e710 con 0x7f46d00049f0 2022-06-29T15:43:06.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.151+0000 7f46e9ffb700 1 -- 172.21.15.47:0/3462677147 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46e400e870 con 0x7f46d00049f0 2022-06-29T15:43:06.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.151+0000 7f46e9ffb700 1 -- 172.21.15.47:0/3462677147 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f46e400ebd0 con 0x7f46d00049f0 2022-06-29T15:43:06.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.151+0000 7f46f19bf700 1 -- 172.21.15.47:0/3462677147 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46d00049f0 msgr2=0x7f46d0004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:06.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.151+0000 7f46f19bf700 1 --2- 172.21.15.47:0/3462677147 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46d00049f0 0x7f46d0004e70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7f46d00054d0 tx=0x7f46e40091d0).stop 2022-06-29T15:43:06.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.151+0000 7f46eb7fe700 1 -- 172.21.15.47:0/3462677147 reap_dead start 2022-06-29T15:43:06.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.151+0000 7f46f19bf700 1 -- 172.21.15.47:0/3462677147 shutdown_connections 2022-06-29T15:43:06.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.151+0000 7f46f19bf700 1 -- 172.21.15.47:0/3462677147 >> 172.21.15.47:0/3462677147 conn(0x7f46ec0626f0 msgr2=0x7f46ec062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:06.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.151+0000 7f46f19bf700 1 -- 172.21.15.47:0/3462677147 shutdown_connections 2022-06-29T15:43:06.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.151+0000 7f46f19bf700 1 -- 172.21.15.47:0/3462677147 wait complete. 2022-06-29T15:43:06.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.155+0000 7f46f19bf700 1 Processor -- start 2022-06-29T15:43:06.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.155+0000 7f46f19bf700 1 -- start start 2022-06-29T15:43:06.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.155+0000 7f46f19bf700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f46ec05fab0 0x7f46ec157260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:06.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.155+0000 7f46f19bf700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46ec13f7b0 0x7f46ec13fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:06.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.155+0000 7f46f19bf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46ec143e50 0x7f46ec1442d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:06.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.155+0000 7f46f19bf700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46ec0af250 con 0x7f46ec13f7b0 2022-06-29T15:43:06.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.155+0000 7f46f19bf700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46ec05fe90 con 0x7f46ec143e50 2022-06-29T15:43:06.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.155+0000 7f46f19bf700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46ec0af480 con 0x7f46ec05fab0 2022-06-29T15:43:06.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.155+0000 7f46ea7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46ec13f7b0 0x7f46ec13fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:06.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.155+0000 7f46ea7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46ec13f7b0 0x7f46ec13fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46936/0 (socket says 172.21.15.47:46936) 2022-06-29T15:43:06.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.155+0000 7f46ea7fc700 1 -- 172.21.15.47:0/2778612226 learned_addr learned my addr 172.21.15.47:0/2778612226 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:06.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.155+0000 7f46eb7fe700 1 --2- 172.21.15.47:0/2778612226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46ec143e50 0x7f46ec1442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:06.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.155+0000 7f46eaffd700 1 --2- 172.21.15.47:0/2778612226 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f46ec05fab0 0x7f46ec157260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:06.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.155+0000 7f46ea7fc700 1 -- 172.21.15.47:0/2778612226 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f46ec05fab0 msgr2=0x7f46ec157260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:06.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.155+0000 7f46ea7fc700 1 --2- 172.21.15.47:0/2778612226 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f46ec05fab0 0x7f46ec157260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:06.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.155+0000 7f46ea7fc700 1 -- 172.21.15.47:0/2778612226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46ec143e50 msgr2=0x7f46ec1442d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:06.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.155+0000 7f46ea7fc700 1 --2- 172.21.15.47:0/2778612226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46ec143e50 0x7f46ec1442d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:06.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.155+0000 7f46ea7fc700 1 -- 172.21.15.47:0/2778612226 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46ec0af720 con 0x7f46ec13f7b0 2022-06-29T15:43:06.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.155+0000 7f46ea7fc700 1 --2- 172.21.15.47:0/2778612226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46ec13f7b0 0x7f46ec13fc10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f46dc00b000 tx=0x7f46ec0affb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:06.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.155+0000 7f46dbfff700 1 -- 172.21.15.47:0/2778612226 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f46dc015630 con 0x7f46ec13f7b0 2022-06-29T15:43:06.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.155+0000 7f46dbfff700 1 -- 172.21.15.47:0/2778612226 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46dc010040 con 0x7f46ec13f7b0 2022-06-29T15:43:06.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.155+0000 7f46f19bf700 1 -- 172.21.15.47:0/2778612226 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46ec140100 con 0x7f46ec13f7b0 2022-06-29T15:43:06.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.155+0000 7f46dbfff700 1 -- 172.21.15.47:0/2778612226 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f46dc015980 con 0x7f46ec13f7b0 2022-06-29T15:43:06.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.155+0000 7f46f19bf700 1 -- 172.21.15.47:0/2778612226 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46ec142fd0 con 0x7f46ec13f7b0 2022-06-29T15:43:06.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.159+0000 7f46f19bf700 1 -- 172.21.15.47:0/2778612226 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46ec144b00 con 0x7f46ec13f7b0 2022-06-29T15:43:06.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.159+0000 7f46dbfff700 1 -- 172.21.15.47:0/2778612226 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f46dc01c070 con 0x7f46ec13f7b0 2022-06-29T15:43:06.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.159+0000 7f46dbfff700 1 --2- 172.21.15.47:0/2778612226 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f46c80539e0 0x7f46c8055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:06.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.159+0000 7f46dbfff700 1 -- 172.21.15.47:0/2778612226 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f46dc06b420 con 0x7f46ec13f7b0 2022-06-29T15:43:06.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.159+0000 7f46eaffd700 1 --2- 172.21.15.47:0/2778612226 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f46c80539e0 0x7f46c8055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:06.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.159+0000 7f46eaffd700 1 --2- 172.21.15.47:0/2778612226 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f46c80539e0 0x7f46c8055ea0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f46e000b000 tx=0x7f46e0008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:06.182 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.163+0000 7f46dbfff700 1 -- 172.21.15.47:0/2778612226 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f46dc042390 con 0x7f46ec13f7b0 2022-06-29T15:43:06.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.331+0000 7f46f19bf700 1 -- 172.21.15.47:0/2778612226 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f46ec0af8b0 con 0x7f46ec13f7b0 2022-06-29T15:43:06.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.339+0000 7f46dbfff700 1 -- 172.21.15.47:0/2778612226 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 updated v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7f46dc0459b0 con 0x7f46ec13f7b0 2022-06-29T15:43:06.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.343+0000 7f46f19bf700 1 -- 172.21.15.47:0/2778612226 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f46c80539e0 msgr2=0x7f46c8055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:06.352 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.343+0000 7f46f19bf700 1 --2- 172.21.15.47:0/2778612226 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f46c80539e0 0x7f46c8055ea0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f46e000b000 tx=0x7f46e0008040).stop 2022-06-29T15:43:06.352 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.343+0000 7f46f19bf700 1 -- 172.21.15.47:0/2778612226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46ec13f7b0 msgr2=0x7f46ec13fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:06.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.343+0000 7f46f19bf700 1 --2- 172.21.15.47:0/2778612226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46ec13f7b0 0x7f46ec13fc10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f46dc00b000 tx=0x7f46ec0affb0).stop 2022-06-29T15:43:06.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.343+0000 7f46f19bf700 1 -- 172.21.15.47:0/2778612226 shutdown_connections 2022-06-29T15:43:06.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.343+0000 7f46f19bf700 1 --2- 172.21.15.47:0/2778612226 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f46c80539e0 0x7f46c8055ea0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:06.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.343+0000 7f46f19bf700 1 --2- 172.21.15.47:0/2778612226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46ec143e50 0x7f46ec1442d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:06.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.343+0000 7f46f19bf700 1 --2- 172.21.15.47:0/2778612226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46ec13f7b0 0x7f46ec13fc10 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:06.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.343+0000 7f46f19bf700 1 --2- 172.21.15.47:0/2778612226 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f46ec05fab0 0x7f46ec157260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:06.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.343+0000 7f46f19bf700 1 -- 172.21.15.47:0/2778612226 >> 172.21.15.47:0/2778612226 conn(0x7f46ec0626f0 msgr2=0x7f46ec14a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:06.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.343+0000 7f46f19bf700 1 -- 172.21.15.47:0/2778612226 shutdown_connections 2022-06-29T15:43:06.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.343+0000 7f46f19bf700 1 -- 172.21.15.47:0/2778612226 wait complete. 2022-06-29T15:43:06.357 INFO:teuthology.orchestra.run.smithi047.stderr:updated 2022-06-29T15:43:06.364 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2022-06-29T15:43:06.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.463+0000 7ff08b9a2700 1 Processor -- start 2022-06-29T15:43:06.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.467+0000 7ff08b9a2700 1 -- start start 2022-06-29T15:43:06.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.467+0000 7ff08b9a2700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff084095950 0x7ff084097690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:06.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.467+0000 7ff08b9a2700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff084143be0 con 0x7ff0841430f0 2022-06-29T15:43:06.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.467+0000 7ff08b9a2700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff084095dc0 con 0x7ff0840a93c0 2022-06-29T15:43:06.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.467+0000 7ff08b9a2700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff08413d590 con 0x7ff084095950 2022-06-29T15:43:06.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.467+0000 7ff08973e700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff0841430f0 0x7ff0841434d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:60730/0 (socket says 172.21.15.47:60730) 2022-06-29T15:43:06.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.467+0000 7ff08973e700 1 -- 172.21.15.47:0/3161874215 learned_addr learned my addr 172.21.15.47:0/3161874215 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:43:06.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.467+0000 7ff089f3f700 1 --2- 172.21.15.47:0/3161874215 >> v2:172.21.15.64:3300/0 conn(0x7ff084095950 0x7ff084097690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:06.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.467+0000 7ff073fff700 1 -- 172.21.15.47:0/3161874215 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 537785226 0 0) 0x7ff084143be0 con 0x7ff0841430f0 2022-06-29T15:43:06.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.467+0000 7ff073fff700 1 -- 172.21.15.47:0/3161874215 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff06c003260 con 0x7ff0841430f0 2022-06-29T15:43:06.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.467+0000 7ff073fff700 1 -- 172.21.15.47:0/3161874215 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4099616437 0 0) 0x7ff084095dc0 con 0x7ff0840a93c0 2022-06-29T15:43:06.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.467+0000 7ff073fff700 1 -- 172.21.15.47:0/3161874215 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff084143be0 con 0x7ff0840a93c0 2022-06-29T15:43:06.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.467+0000 7ff089f3f700 1 -- 172.21.15.47:0/3161874215 >> v1:172.21.15.64:6789/0 conn(0x7ff0840a93c0 legacy=0x7ff084095240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:06.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.467+0000 7ff089f3f700 1 -- 172.21.15.47:0/3161874215 >> v1:172.21.15.47:6789/0 conn(0x7ff0841430f0 legacy=0x7ff0841434d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:06.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.467+0000 7ff089f3f700 1 -- 172.21.15.47:0/3161874215 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff084060e50 con 0x7ff084095950 2022-06-29T15:43:06.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.467+0000 7ff089f3f700 1 --2- 172.21.15.47:0/3161874215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff084095950 0x7ff084097690 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7ff07c0032f0 tx=0x7ff07c0091d0).ready entity=mon.1 client_cookie=93b163c6f7580f90 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:06.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.471+0000 7ff073fff700 1 -- 172.21.15.47:0/3161874215 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff07c00b600 con 0x7ff084095950 2022-06-29T15:43:06.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.471+0000 7ff073fff700 1 -- 172.21.15.47:0/3161874215 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff07c00b760 con 0x7ff084095950 2022-06-29T15:43:06.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.471+0000 7ff073fff700 1 -- 172.21.15.47:0/3161874215 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff07c00b8d0 con 0x7ff084095950 2022-06-29T15:43:06.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.471+0000 7ff08b9a2700 1 -- 172.21.15.47:0/3161874215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff084095950 msgr2=0x7ff084097690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:06.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.471+0000 7ff08b9a2700 1 --2- 172.21.15.47:0/3161874215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff084095950 0x7ff084097690 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7ff07c0032f0 tx=0x7ff07c0091d0).stop 2022-06-29T15:43:06.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.471+0000 7ff08b9a2700 1 -- 172.21.15.47:0/3161874215 shutdown_connections 2022-06-29T15:43:06.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.471+0000 7ff08b9a2700 1 --2- 172.21.15.47:0/3161874215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff084095950 0x7ff084097690 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:06.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.471+0000 7ff08b9a2700 1 -- 172.21.15.47:0/3161874215 >> 172.21.15.47:0/3161874215 conn(0x7ff08405f0b0 msgr2=0x7ff08405f490 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:06.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.471+0000 7ff08b9a2700 1 -- 172.21.15.47:0/3161874215 shutdown_connections 2022-06-29T15:43:06.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.471+0000 7ff08b9a2700 1 -- 172.21.15.47:0/3161874215 wait complete. 2022-06-29T15:43:06.485 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.471+0000 7ff08b9a2700 1 Processor -- start 2022-06-29T15:43:06.485 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.471+0000 7ff08b9a2700 1 -- start start 2022-06-29T15:43:06.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.475+0000 7ff08b9a2700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff084095950 0x7ff08409aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:06.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.475+0000 7ff08b9a2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0840a93c0 0x7ff08413a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:06.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.475+0000 7ff08b9a2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0841430f0 0x7ff08413a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:06.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.475+0000 7ff08b9a2700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0841399f0 con 0x7ff0840a93c0 2022-06-29T15:43:06.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.475+0000 7ff08b9a2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff084138830 con 0x7ff0841430f0 2022-06-29T15:43:06.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.475+0000 7ff08b9a2700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff084139750 con 0x7ff084095950 2022-06-29T15:43:06.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.475+0000 7ff088f3d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0840a93c0 0x7ff08413a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:06.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.475+0000 7ff08973e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff084095950 0x7ff08409aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:06.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.475+0000 7ff089f3f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0841430f0 0x7ff08413a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:06.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.475+0000 7ff088f3d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0840a93c0 0x7ff08413a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46952/0 (socket says 172.21.15.47:46952) 2022-06-29T15:43:06.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.475+0000 7ff089f3f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0841430f0 0x7ff08413a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:52658/0 (socket says 172.21.15.47:52658) 2022-06-29T15:43:06.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.475+0000 7ff088f3d700 1 -- 172.21.15.47:0/1499860137 learned_addr learned my addr 172.21.15.47:0/1499860137 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:06.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.475+0000 7ff088f3d700 1 -- 172.21.15.47:0/1499860137 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff084095950 msgr2=0x7ff08409aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:06.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.475+0000 7ff088f3d700 1 --2- 172.21.15.47:0/1499860137 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff084095950 0x7ff08409aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:06.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.475+0000 7ff088f3d700 1 -- 172.21.15.47:0/1499860137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0841430f0 msgr2=0x7ff08413a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:06.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.475+0000 7ff088f3d700 1 --2- 172.21.15.47:0/1499860137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0841430f0 0x7ff08413a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:06.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.475+0000 7ff088f3d700 1 -- 172.21.15.47:0/1499860137 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff07c009470 con 0x7ff0840a93c0 2022-06-29T15:43:06.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.475+0000 7ff088f3d700 1 --2- 172.21.15.47:0/1499860137 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0840a93c0 0x7ff08413a450 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7ff074001110 tx=0x7ff074007ae0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:06.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.475+0000 7ff0727fc700 1 -- 172.21.15.47:0/1499860137 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff074012400 con 0x7ff0840a93c0 2022-06-29T15:43:06.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.475+0000 7ff0727fc700 1 -- 172.21.15.47:0/1499860137 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff074012560 con 0x7ff0840a93c0 2022-06-29T15:43:06.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.475+0000 7ff08b9a2700 1 -- 172.21.15.47:0/1499860137 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff08409b0e0 con 0x7ff0840a93c0 2022-06-29T15:43:06.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.475+0000 7ff0727fc700 1 -- 172.21.15.47:0/1499860137 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0740128c0 con 0x7ff0840a93c0 2022-06-29T15:43:06.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.475+0000 7ff08b9a2700 1 -- 172.21.15.47:0/1499860137 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff08413d700 con 0x7ff0840a93c0 2022-06-29T15:43:06.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.475+0000 7ff08b9a2700 1 -- 172.21.15.47:0/1499860137 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff084139d70 con 0x7ff0840a93c0 2022-06-29T15:43:06.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.475+0000 7ff0727fc700 1 -- 172.21.15.47:0/1499860137 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff074019030 con 0x7ff0840a93c0 2022-06-29T15:43:06.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.479+0000 7ff0727fc700 1 --2- 172.21.15.47:0/1499860137 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff064053b30 0x7ff064055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:06.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.479+0000 7ff0727fc700 1 -- 172.21.15.47:0/1499860137 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5600+0+0 (secure 0 0 0) 0x7ff07406b030 con 0x7ff0840a93c0 2022-06-29T15:43:06.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.479+0000 7ff08973e700 1 --2- 172.21.15.47:0/1499860137 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff064053b30 0x7ff064055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:06.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.479+0000 7ff08973e700 1 --2- 172.21.15.47:0/1499860137 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff064053b30 0x7ff064055ff0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7ff08000d000 tx=0x7ff080009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:06.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.483+0000 7ff0727fc700 1 -- 172.21.15.47:0/1499860137 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff074042130 con 0x7ff0840a93c0 2022-06-29T15:43:06.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.655+0000 7ff08b9a2700 1 -- 172.21.15.47:0/1499860137 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7ff084139580 con 0x7ff0840a93c0 2022-06-29T15:43:06.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.663+0000 7ff0727fc700 1 -- 172.21.15.47:0/1499860137 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 updated v10) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff074045750 con 0x7ff0840a93c0 2022-06-29T15:43:06.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.663+0000 7ff08b9a2700 1 -- 172.21.15.47:0/1499860137 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff064053b30 msgr2=0x7ff064055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:06.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.663+0000 7ff08b9a2700 1 --2- 172.21.15.47:0/1499860137 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff064053b30 0x7ff064055ff0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7ff08000d000 tx=0x7ff080009040).stop 2022-06-29T15:43:06.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.663+0000 7ff08b9a2700 1 -- 172.21.15.47:0/1499860137 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0840a93c0 msgr2=0x7ff08413a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:06.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.663+0000 7ff08b9a2700 1 --2- 172.21.15.47:0/1499860137 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0840a93c0 0x7ff08413a450 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7ff074001110 tx=0x7ff074007ae0).stop 2022-06-29T15:43:06.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.667+0000 7ff08b9a2700 1 -- 172.21.15.47:0/1499860137 shutdown_connections 2022-06-29T15:43:06.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.667+0000 7ff08b9a2700 1 --2- 172.21.15.47:0/1499860137 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff064053b30 0x7ff064055ff0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:06.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.667+0000 7ff08b9a2700 1 --2- 172.21.15.47:0/1499860137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0841430f0 0x7ff08413a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:06.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.667+0000 7ff08b9a2700 1 --2- 172.21.15.47:0/1499860137 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0840a93c0 0x7ff08413a450 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:06.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.667+0000 7ff08b9a2700 1 --2- 172.21.15.47:0/1499860137 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff084095950 0x7ff08409aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:06.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.667+0000 7ff08b9a2700 1 -- 172.21.15.47:0/1499860137 >> 172.21.15.47:0/1499860137 conn(0x7ff08405f0b0 msgr2=0x7ff0840b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:06.676 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.667+0000 7ff08b9a2700 1 -- 172.21.15.47:0/1499860137 shutdown_connections 2022-06-29T15:43:06.676 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.667+0000 7ff08b9a2700 1 -- 172.21.15.47:0/1499860137 wait complete. 2022-06-29T15:43:06.677 INFO:teuthology.orchestra.run.smithi047.stderr:updated 2022-06-29T15:43:06.686 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2022-06-29T15:43:06.786 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.779+0000 7f27b759e700 1 Processor -- start 2022-06-29T15:43:06.787 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.779+0000 7f27b759e700 1 -- start start 2022-06-29T15:43:06.790 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.779+0000 7f27b759e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f27b8061c00 0x7f27b813e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:06.790 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.779+0000 7f27b759e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f27b80b63e0 0x7f27b813e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:06.791 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.783+0000 7f27b759e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27b813ec80 con 0x7f27b80b6f10 2022-06-29T15:43:06.791 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.783+0000 7f27b759e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27b80b3e90 con 0x7f27b8061c00 2022-06-29T15:43:06.791 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.783+0000 7f27b759e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27b80a9cf0 con 0x7f27b80b63e0 2022-06-29T15:43:06.791 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.783+0000 7f27b659c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f27b8061c00 0x7f27b813e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:06.792 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.783+0000 7f27b5d9b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f27b80b6f10 0x7f27b805f0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:60746/0 (socket says 172.21.15.47:60746) 2022-06-29T15:43:06.792 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.783+0000 7f27b5d9b700 1 -- 172.21.15.47:0/678733271 learned_addr learned my addr 172.21.15.47:0/678733271 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:43:06.792 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.783+0000 7f27b659c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f27b8061c00 0x7f27b813e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46958/0 (socket says 172.21.15.47:46958) 2022-06-29T15:43:06.792 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.783+0000 7f27b6d9d700 1 --2- 172.21.15.47:0/678733271 >> v2:172.21.15.64:3300/0 conn(0x7f27b80b63e0 0x7f27b813e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:06.793 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.783+0000 7f27b559a700 1 -- 172.21.15.47:0/678733271 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3517273387 0 0) 0x7f27b813ec80 con 0x7f27b80b6f10 2022-06-29T15:43:06.793 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.783+0000 7f27b659c700 1 -- 172.21.15.47:0/678733271 >> v2:172.21.15.64:3300/0 conn(0x7f27b80b63e0 msgr2=0x7f27b813e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:43:06.793 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.783+0000 7f27b659c700 1 --2- 172.21.15.47:0/678733271 >> v2:172.21.15.64:3300/0 conn(0x7f27b80b63e0 0x7f27b813e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:06.793 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.783+0000 7f27b659c700 1 -- 172.21.15.47:0/678733271 >> v1:172.21.15.47:6789/0 conn(0x7f27b80b6f10 legacy=0x7f27b805f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:06.794 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.783+0000 7f27b659c700 1 -- 172.21.15.47:0/678733271 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b80a9f90 con 0x7f27b8061c00 2022-06-29T15:43:06.794 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.783+0000 7f27b659c700 1 --2- 172.21.15.47:0/678733271 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27b8061c00 0x7f27b813e2a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7f27a8002f70 tx=0x7f27a8006df0).ready entity=mon.0 client_cookie=c23121eb4692669c server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:06.794 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.783+0000 7f27b559a700 1 -- 172.21.15.47:0/678733271 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f27a800d570 con 0x7f27b8061c00 2022-06-29T15:43:06.794 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.783+0000 7f27b559a700 1 -- 172.21.15.47:0/678733271 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27a8000e90 con 0x7f27b8061c00 2022-06-29T15:43:06.795 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.783+0000 7f27b559a700 1 -- 172.21.15.47:0/678733271 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f27a800d8c0 con 0x7f27b8061c00 2022-06-29T15:43:06.800 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.783+0000 7f27b759e700 1 -- 172.21.15.47:0/678733271 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27b8061c00 msgr2=0x7f27b813e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:06.800 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.783+0000 7f27b759e700 1 --2- 172.21.15.47:0/678733271 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27b8061c00 0x7f27b813e2a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7f27a8002f70 tx=0x7f27a8006df0).stop 2022-06-29T15:43:06.800 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.783+0000 7f27b759e700 1 -- 172.21.15.47:0/678733271 shutdown_connections 2022-06-29T15:43:06.800 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.783+0000 7f27b759e700 1 --2- 172.21.15.47:0/678733271 >> v2:172.21.15.64:3300/0 conn(0x7f27b80b63e0 0x7f27b813e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:06.801 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.783+0000 7f27b759e700 1 --2- 172.21.15.47:0/678733271 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27b8061c00 0x7f27b813e2a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:06.801 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.783+0000 7f27b759e700 1 -- 172.21.15.47:0/678733271 >> 172.21.15.47:0/678733271 conn(0x7f27b8063670 msgr2=0x7f27b8063a80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:06.801 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.783+0000 7f27b759e700 1 -- 172.21.15.47:0/678733271 shutdown_connections 2022-06-29T15:43:06.801 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.783+0000 7f27b759e700 1 -- 172.21.15.47:0/678733271 wait complete. 2022-06-29T15:43:06.802 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.787+0000 7f27b759e700 1 Processor -- start 2022-06-29T15:43:06.802 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.787+0000 7f27b759e700 1 -- start start 2022-06-29T15:43:06.802 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.787+0000 7f27b759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27b8061c00 0x7f27b8144530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:06.802 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.787+0000 7f27b759e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27b80b63e0 0x7f27b8142610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:06.803 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.787+0000 7f27b759e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f27b80b6f10 0x7f27b8155c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:06.803 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.787+0000 7f27b759e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b8142e30 con 0x7f27b80b63e0 2022-06-29T15:43:06.803 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.787+0000 7f27b759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b80ae160 con 0x7f27b8061c00 2022-06-29T15:43:06.804 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.787+0000 7f27b759e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27b80ae400 con 0x7f27b80b6f10 2022-06-29T15:43:06.804 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.787+0000 7f27b5d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27b80b63e0 0x7f27b8142610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:06.804 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.787+0000 7f27b5d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27b80b63e0 0x7f27b8142610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46964/0 (socket says 172.21.15.47:46964) 2022-06-29T15:43:06.804 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.787+0000 7f27b5d9b700 1 -- 172.21.15.47:0/349692987 learned_addr learned my addr 172.21.15.47:0/349692987 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:06.805 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.787+0000 7f27b6d9d700 1 --2- 172.21.15.47:0/349692987 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f27b80b6f10 0x7f27b8155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:06.805 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.787+0000 7f27b659c700 1 --2- 172.21.15.47:0/349692987 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27b8061c00 0x7f27b8144530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:06.805 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.787+0000 7f27b5d9b700 1 -- 172.21.15.47:0/349692987 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f27b80b6f10 msgr2=0x7f27b8155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:06.806 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.787+0000 7f27b5d9b700 1 --2- 172.21.15.47:0/349692987 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f27b80b6f10 0x7f27b8155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:06.806 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.787+0000 7f27b5d9b700 1 -- 172.21.15.47:0/349692987 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27b8061c00 msgr2=0x7f27b8144530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:06.806 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.787+0000 7f27b5d9b700 1 --2- 172.21.15.47:0/349692987 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27b8061c00 0x7f27b8144530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:06.806 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.787+0000 7f27b5d9b700 1 -- 172.21.15.47:0/349692987 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27a800b180 con 0x7f27b80b63e0 2022-06-29T15:43:06.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.791+0000 7f27b5d9b700 1 --2- 172.21.15.47:0/349692987 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27b80b63e0 0x7f27b8142610 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7f27a4000ed0 tx=0x7f27a4007cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:06.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.791+0000 7f27af7fe700 1 -- 172.21.15.47:0/349692987 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f27a40070d0 con 0x7f27b80b63e0 2022-06-29T15:43:06.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.791+0000 7f27af7fe700 1 -- 172.21.15.47:0/349692987 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27a4007230 con 0x7f27b80b63e0 2022-06-29T15:43:06.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.791+0000 7f27b759e700 1 -- 172.21.15.47:0/349692987 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27b81430d0 con 0x7f27b80b63e0 2022-06-29T15:43:06.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.791+0000 7f27af7fe700 1 -- 172.21.15.47:0/349692987 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f27a4005610 con 0x7f27b80b63e0 2022-06-29T15:43:06.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.791+0000 7f27b759e700 1 -- 172.21.15.47:0/349692987 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27b80ad4e0 con 0x7f27b80b63e0 2022-06-29T15:43:06.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.791+0000 7f27b759e700 1 -- 172.21.15.47:0/349692987 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27b8145370 con 0x7f27b80b63e0 2022-06-29T15:43:06.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.791+0000 7f27af7fe700 1 -- 172.21.15.47:0/349692987 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f27a401a060 con 0x7f27b80b63e0 2022-06-29T15:43:06.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.791+0000 7f27af7fe700 1 --2- 172.21.15.47:0/349692987 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2794053950 0x7f2794055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:06.809 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.791+0000 7f27af7fe700 1 -- 172.21.15.47:0/349692987 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f27a406b410 con 0x7f27b80b63e0 2022-06-29T15:43:06.809 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.791+0000 7f27b659c700 1 --2- 172.21.15.47:0/349692987 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2794053950 0x7f2794055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:06.810 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.795+0000 7f27b659c700 1 --2- 172.21.15.47:0/349692987 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2794053950 0x7f2794055e10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f27a801a000 tx=0x7f27a8014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:06.810 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.795+0000 7f27af7fe700 1 -- 172.21.15.47:0/349692987 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f27b8145370 con 0x7f27b80b63e0 2022-06-29T15:43:06.969 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.963+0000 7f27b759e700 1 -- 172.21.15.47:0/349692987 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f27b8145370 con 0x7f27b80b63e0 2022-06-29T15:43:06.976 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.967+0000 7f27af7fe700 1 -- 172.21.15.47:0/349692987 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 updated v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7f27b8145370 con 0x7f27b80b63e0 2022-06-29T15:43:06.979 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.971+0000 7f27b759e700 1 -- 172.21.15.47:0/349692987 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2794053950 msgr2=0x7f2794055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:06.980 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.971+0000 7f27b759e700 1 --2- 172.21.15.47:0/349692987 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2794053950 0x7f2794055e10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f27a801a000 tx=0x7f27a8014000).stop 2022-06-29T15:43:06.980 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.971+0000 7f27b759e700 1 -- 172.21.15.47:0/349692987 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27b80b63e0 msgr2=0x7f27b8142610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:06.980 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.971+0000 7f27b759e700 1 --2- 172.21.15.47:0/349692987 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27b80b63e0 0x7f27b8142610 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7f27a4000ed0 tx=0x7f27a4007cb0).stop 2022-06-29T15:43:06.980 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.971+0000 7f27b759e700 1 -- 172.21.15.47:0/349692987 shutdown_connections 2022-06-29T15:43:06.980 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.971+0000 7f27b759e700 1 --2- 172.21.15.47:0/349692987 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f27b80b6f10 0x7f27b8155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:06.981 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.971+0000 7f27b759e700 1 --2- 172.21.15.47:0/349692987 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27b80b63e0 0x7f27b8142610 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:06.981 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.971+0000 7f27b759e700 1 --2- 172.21.15.47:0/349692987 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2794053950 0x7f2794055e10 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:06.981 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.971+0000 7f27b759e700 1 --2- 172.21.15.47:0/349692987 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27b8061c00 0x7f27b8144530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:06.981 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.971+0000 7f27b759e700 1 -- 172.21.15.47:0/349692987 >> 172.21.15.47:0/349692987 conn(0x7f27b8063670 msgr2=0x7f27b8140630 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:06.982 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.971+0000 7f27b759e700 1 -- 172.21.15.47:0/349692987 shutdown_connections 2022-06-29T15:43:06.982 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:06.971+0000 7f27b759e700 1 -- 172.21.15.47:0/349692987 wait complete. 2022-06-29T15:43:06.982 INFO:teuthology.orchestra.run.smithi047.stderr:updated 2022-06-29T15:43:06.993 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2022-06-29T15:43:07.099 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.091+0000 7f154b59e700 1 Processor -- start 2022-06-29T15:43:07.099 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.091+0000 7f154b59e700 1 -- start start 2022-06-29T15:43:07.102 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.095+0000 7f154b59e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f154c140f90 0x7f154c064ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:07.102 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.095+0000 7f154b59e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f154c067090 0x7f154c0b21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:07.102 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.095+0000 7f154b59e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f154c0b2e90 con 0x7f154c0646d0 2022-06-29T15:43:07.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.095+0000 7f154b59e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f154c141f50 con 0x7f154c140f90 2022-06-29T15:43:07.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.095+0000 7f154b59e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f154c0b26c0 con 0x7f154c067090 2022-06-29T15:43:07.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.095+0000 7f1549d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f154c140f90 0x7f154c064ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:07.104 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.095+0000 7f1549d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f154c140f90 0x7f154c064ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46974/0 (socket says 172.21.15.47:46974) 2022-06-29T15:43:07.104 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.095+0000 7f1549d9b700 1 -- 172.21.15.47:0/1433519583 learned_addr learned my addr 172.21.15.47:0/1433519583 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:07.104 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.095+0000 7f154a59c700 1 --1- 172.21.15.47:0/1433519583 >> v1:172.21.15.47:6789/0 conn(0x7f154c0646d0 0x7f154c141840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:60758/0 (socket says 172.21.15.47:60758) 2022-06-29T15:43:07.105 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.095+0000 7f154ad9d700 1 --2- 172.21.15.47:0/1433519583 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f154c067090 0x7f154c0b21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:07.105 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.095+0000 7f1549d9b700 1 -- 172.21.15.47:0/1433519583 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f154c067090 msgr2=0x7f154c0b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:43:07.105 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.095+0000 7f1549d9b700 1 --2- 172.21.15.47:0/1433519583 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f154c067090 0x7f154c0b21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:07.106 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.095+0000 7f1549d9b700 1 -- 172.21.15.47:0/1433519583 >> v1:172.21.15.47:6789/0 conn(0x7f154c0646d0 legacy=0x7f154c141840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:07.106 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.095+0000 7f1549d9b700 1 -- 172.21.15.47:0/1433519583 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f154c0b2950 con 0x7f154c140f90 2022-06-29T15:43:07.107 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.095+0000 7f154ad9d700 1 --2- 172.21.15.47:0/1433519583 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f154c067090 0x7f154c0b21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:43:07.107 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.095+0000 7f1549d9b700 1 --2- 172.21.15.47:0/1433519583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f154c140f90 0x7f154c064ab0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7f1540002f70 tx=0x7f1540006eb0).ready entity=mon.0 client_cookie=572a741debb24aae server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:07.107 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.095+0000 7f154959a700 1 -- 172.21.15.47:0/1433519583 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f154000d5c0 con 0x7f154c140f90 2022-06-29T15:43:07.108 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.095+0000 7f154959a700 1 -- 172.21.15.47:0/1433519583 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1540000e90 con 0x7f154c140f90 2022-06-29T15:43:07.108 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.095+0000 7f154959a700 1 -- 172.21.15.47:0/1433519583 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f154000d950 con 0x7f154c140f90 2022-06-29T15:43:07.114 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.095+0000 7f154b59e700 1 -- 172.21.15.47:0/1433519583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f154c140f90 msgr2=0x7f154c064ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:07.114 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.095+0000 7f154b59e700 1 --2- 172.21.15.47:0/1433519583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f154c140f90 0x7f154c064ab0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7f1540002f70 tx=0x7f1540006eb0).stop 2022-06-29T15:43:07.114 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.095+0000 7f154b59e700 1 -- 172.21.15.47:0/1433519583 shutdown_connections 2022-06-29T15:43:07.115 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.095+0000 7f154b59e700 1 --2- 172.21.15.47:0/1433519583 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f154c067090 0x7f154c0b21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:07.115 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.095+0000 7f154b59e700 1 --2- 172.21.15.47:0/1433519583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f154c140f90 0x7f154c064ab0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:07.115 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.095+0000 7f154b59e700 1 -- 172.21.15.47:0/1433519583 >> 172.21.15.47:0/1433519583 conn(0x7f154c05c510 msgr2=0x7f154c05c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:07.116 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.095+0000 7f154b59e700 1 -- 172.21.15.47:0/1433519583 shutdown_connections 2022-06-29T15:43:07.116 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.095+0000 7f154b59e700 1 -- 172.21.15.47:0/1433519583 wait complete. 2022-06-29T15:43:07.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.095+0000 7f154b59e700 1 Processor -- start 2022-06-29T15:43:07.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.095+0000 7f154b59e700 1 -- start start 2022-06-29T15:43:07.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.099+0000 7f154b59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f154c0646d0 0x7f154c13d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:07.118 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.099+0000 7f154b59e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f154c067090 0x7f154c143920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:07.118 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.099+0000 7f154b59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f154c13a2f0 0x7f154c143e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:07.118 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.099+0000 7f154b59e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f154c138520 con 0x7f154c0646d0 2022-06-29T15:43:07.119 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.099+0000 7f154b59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f154c1568f0 con 0x7f154c13a2f0 2022-06-29T15:43:07.119 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.099+0000 7f154b59e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f154c144300 con 0x7f154c067090 2022-06-29T15:43:07.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.099+0000 7f154a59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f154c0646d0 0x7f154c13d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:07.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.099+0000 7f154a59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f154c0646d0 0x7f154c13d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46978/0 (socket says 172.21.15.47:46978) 2022-06-29T15:43:07.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.099+0000 7f154a59c700 1 -- 172.21.15.47:0/206947470 learned_addr learned my addr 172.21.15.47:0/206947470 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:07.121 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.099+0000 7f154ad9d700 1 --2- 172.21.15.47:0/206947470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f154c13a2f0 0x7f154c143e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:07.121 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.099+0000 7f1549d9b700 1 --2- 172.21.15.47:0/206947470 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f154c067090 0x7f154c143920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:07.121 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.099+0000 7f154a59c700 1 -- 172.21.15.47:0/206947470 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f154c067090 msgr2=0x7f154c143920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:07.122 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.099+0000 7f154a59c700 1 --2- 172.21.15.47:0/206947470 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f154c067090 0x7f154c143920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:07.122 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.099+0000 7f154a59c700 1 -- 172.21.15.47:0/206947470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f154c13a2f0 msgr2=0x7f154c143e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:07.123 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.099+0000 7f154a59c700 1 --2- 172.21.15.47:0/206947470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f154c13a2f0 0x7f154c143e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:07.123 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.099+0000 7f154a59c700 1 -- 172.21.15.47:0/206947470 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f154000b1e0 con 0x7f154c0646d0 2022-06-29T15:43:07.123 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.099+0000 7f154ad9d700 1 --2- 172.21.15.47:0/206947470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f154c13a2f0 0x7f154c143e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:43:07.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.099+0000 7f154a59c700 1 --2- 172.21.15.47:0/206947470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f154c0646d0 0x7f154c13d210 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f154c138660 tx=0x7f153c007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:07.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.099+0000 7f153b7fe700 1 -- 172.21.15.47:0/206947470 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f153c0070d0 con 0x7f154c0646d0 2022-06-29T15:43:07.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.099+0000 7f153b7fe700 1 -- 172.21.15.47:0/206947470 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f153c007230 con 0x7f154c0646d0 2022-06-29T15:43:07.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.099+0000 7f154b59e700 1 -- 172.21.15.47:0/206947470 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f154c1559d0 con 0x7f154c0646d0 2022-06-29T15:43:07.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.099+0000 7f153b7fe700 1 -- 172.21.15.47:0/206947470 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f153c005550 con 0x7f154c0646d0 2022-06-29T15:43:07.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.099+0000 7f154b59e700 1 -- 172.21.15.47:0/206947470 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f154c155f10 con 0x7f154c0646d0 2022-06-29T15:43:07.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.099+0000 7f153b7fe700 1 -- 172.21.15.47:0/206947470 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f153c01a070 con 0x7f154c0646d0 2022-06-29T15:43:07.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.099+0000 7f154b59e700 1 -- 172.21.15.47:0/206947470 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f154c1565f0 con 0x7f154c0646d0 2022-06-29T15:43:07.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.099+0000 7f153b7fe700 1 --2- 172.21.15.47:0/206947470 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1528053950 0x7f1528055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:07.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.103+0000 7f153b7fe700 1 -- 172.21.15.47:0/206947470 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f153c06ad60 con 0x7f154c0646d0 2022-06-29T15:43:07.127 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.103+0000 7f1549d9b700 1 --2- 172.21.15.47:0/206947470 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1528053950 0x7f1528055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:07.127 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.103+0000 7f1549d9b700 1 --2- 172.21.15.47:0/206947470 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1528053950 0x7f1528055e10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f154001a000 tx=0x7f1540015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:07.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.111+0000 7f153b7fe700 1 -- 172.21.15.47:0/206947470 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f153c041e60 con 0x7f154c0646d0 2022-06-29T15:43:07.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.275+0000 7f154b59e700 1 -- 172.21.15.47:0/206947470 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f154c0573d0 con 0x7f154c0646d0 2022-06-29T15:43:07.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.279+0000 7f153b7fe700 1 -- 172.21.15.47:0/206947470 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 updated v12) v1 ==== 104+0+0 (secure 0 0 0) 0x7f153c045480 con 0x7f154c0646d0 2022-06-29T15:43:07.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.283+0000 7f154b59e700 1 -- 172.21.15.47:0/206947470 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1528053950 msgr2=0x7f1528055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:07.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.283+0000 7f154b59e700 1 --2- 172.21.15.47:0/206947470 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1528053950 0x7f1528055e10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f154001a000 tx=0x7f1540015000).stop 2022-06-29T15:43:07.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.283+0000 7f154b59e700 1 -- 172.21.15.47:0/206947470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f154c0646d0 msgr2=0x7f154c13d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:07.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.283+0000 7f154b59e700 1 --2- 172.21.15.47:0/206947470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f154c0646d0 0x7f154c13d210 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f154c138660 tx=0x7f153c007c40).stop 2022-06-29T15:43:07.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.283+0000 7f154b59e700 1 -- 172.21.15.47:0/206947470 shutdown_connections 2022-06-29T15:43:07.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.283+0000 7f154b59e700 1 --2- 172.21.15.47:0/206947470 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1528053950 0x7f1528055e10 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:07.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.283+0000 7f154b59e700 1 --2- 172.21.15.47:0/206947470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f154c13a2f0 0x7f154c143e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:07.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.283+0000 7f154b59e700 1 --2- 172.21.15.47:0/206947470 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f154c067090 0x7f154c143920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:07.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.283+0000 7f154b59e700 1 --2- 172.21.15.47:0/206947470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f154c0646d0 0x7f154c13d210 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:07.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.283+0000 7f154b59e700 1 -- 172.21.15.47:0/206947470 >> 172.21.15.47:0/206947470 conn(0x7f154c05c510 msgr2=0x7f154c0b4240 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:07.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.283+0000 7f154b59e700 1 -- 172.21.15.47:0/206947470 shutdown_connections 2022-06-29T15:43:07.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.283+0000 7f154b59e700 1 -- 172.21.15.47:0/206947470 wait complete. 2022-06-29T15:43:07.294 INFO:teuthology.orchestra.run.smithi047.stderr:updated 2022-06-29T15:43:07.305 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2022-06-29T15:43:07.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.403+0000 7f88abfff700 1 Processor -- start 2022-06-29T15:43:07.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.403+0000 7f88abfff700 1 -- start start 2022-06-29T15:43:07.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.403+0000 7f88abfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f88ac0b5c60 0x7f88ac144980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:07.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.403+0000 7f88abfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f88ac0a72f0 0x7f88ac0b2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:07.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.403+0000 7f88abfff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88ac0a9120 con 0x7f88ac060eb0 2022-06-29T15:43:07.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.403+0000 7f88abfff700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88ac0b0b80 con 0x7f88ac0b5c60 2022-06-29T15:43:07.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.403+0000 7f88abfff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88ac0a89e0 con 0x7f88ac0a72f0 2022-06-29T15:43:07.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.407+0000 7f88aaffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f88ac0b5c60 0x7f88ac144980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:07.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.407+0000 7f88aaffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f88ac0b5c60 0x7f88ac144980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46990/0 (socket says 172.21.15.47:46990) 2022-06-29T15:43:07.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.407+0000 7f88ab7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f88ac0a72f0 0x7f88ac0b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:07.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.407+0000 7f88aaffd700 1 -- 172.21.15.47:0/1999142844 learned_addr learned my addr 172.21.15.47:0/1999142844 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:07.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.407+0000 7f88aaffd700 1 -- 172.21.15.47:0/1999142844 >> v2:172.21.15.64:3300/0 conn(0x7f88ac0a72f0 msgr2=0x7f88ac0b2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:43:07.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.407+0000 7f88aaffd700 1 --2- 172.21.15.47:0/1999142844 >> v2:172.21.15.64:3300/0 conn(0x7f88ac0a72f0 0x7f88ac0b2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:07.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.407+0000 7f88aaffd700 1 -- 172.21.15.47:0/1999142844 >> v1:172.21.15.64:6789/0 conn(0x7f88ac060eb0 legacy=0x7f88ac0b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:07.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.407+0000 7f88aaffd700 1 -- 172.21.15.47:0/1999142844 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88ac0a8c80 con 0x7f88ac0b5c60 2022-06-29T15:43:07.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.407+0000 7f88aaffd700 1 --2- 172.21.15.47:0/1999142844 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88ac0b5c60 0x7f88ac144980 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7f88a0002f70 tx=0x7f88a0006eb0).ready entity=mon.0 client_cookie=a60d6f5cdfe13dc1 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:07.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.407+0000 7f88a9ffb700 1 -- 172.21.15.47:0/1999142844 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88a000d610 con 0x7f88ac0b5c60 2022-06-29T15:43:07.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.407+0000 7f88a9ffb700 1 -- 172.21.15.47:0/1999142844 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a0000e90 con 0x7f88ac0b5c60 2022-06-29T15:43:07.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.407+0000 7f88a9ffb700 1 -- 172.21.15.47:0/1999142844 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88a000d9a0 con 0x7f88ac0b5c60 2022-06-29T15:43:07.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.407+0000 7f88abfff700 1 -- 172.21.15.47:0/1999142844 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88ac0b5c60 msgr2=0x7f88ac144980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:07.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.407+0000 7f88abfff700 1 --2- 172.21.15.47:0/1999142844 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88ac0b5c60 0x7f88ac144980 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7f88a0002f70 tx=0x7f88a0006eb0).stop 2022-06-29T15:43:07.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.407+0000 7f88abfff700 1 -- 172.21.15.47:0/1999142844 shutdown_connections 2022-06-29T15:43:07.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.407+0000 7f88abfff700 1 --2- 172.21.15.47:0/1999142844 >> v2:172.21.15.64:3300/0 conn(0x7f88ac0a72f0 0x7f88ac0b2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:07.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.407+0000 7f88abfff700 1 --2- 172.21.15.47:0/1999142844 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88ac0b5c60 0x7f88ac144980 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:07.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.407+0000 7f88abfff700 1 -- 172.21.15.47:0/1999142844 >> 172.21.15.47:0/1999142844 conn(0x7f88ac062560 msgr2=0x7f88ac062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:07.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.407+0000 7f88abfff700 1 -- 172.21.15.47:0/1999142844 shutdown_connections 2022-06-29T15:43:07.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.407+0000 7f88abfff700 1 -- 172.21.15.47:0/1999142844 wait complete. 2022-06-29T15:43:07.425 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.411+0000 7f88abfff700 1 Processor -- start 2022-06-29T15:43:07.425 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.411+0000 7f88abfff700 1 -- start start 2022-06-29T15:43:07.425 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.411+0000 7f88abfff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f88ac060eb0 0x7f88ac157350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:07.425 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.411+0000 7f88abfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88ac0a72f0 0x7f88ac0b3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:07.426 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.411+0000 7f88abfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88ac159dc0 0x7f88ac15a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:07.426 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.411+0000 7f88abfff700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ac0af5b0 con 0x7f88ac159dc0 2022-06-29T15:43:07.426 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.411+0000 7f88abfff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ac156970 con 0x7f88ac0a72f0 2022-06-29T15:43:07.426 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.411+0000 7f88abfff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88ac156c10 con 0x7f88ac060eb0 2022-06-29T15:43:07.427 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.411+0000 7f88ab7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88ac159dc0 0x7f88ac15a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:07.427 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.411+0000 7f88ab7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88ac159dc0 0x7f88ac15a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46996/0 (socket says 172.21.15.47:46996) 2022-06-29T15:43:07.427 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.411+0000 7f88ab7fe700 1 -- 172.21.15.47:0/3098510751 learned_addr learned my addr 172.21.15.47:0/3098510751 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:07.427 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.411+0000 7f88aa7fc700 1 --2- 172.21.15.47:0/3098510751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88ac0a72f0 0x7f88ac0b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:07.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.411+0000 7f88aaffd700 1 --2- 172.21.15.47:0/3098510751 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f88ac060eb0 0x7f88ac157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:07.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.411+0000 7f88ab7fe700 1 -- 172.21.15.47:0/3098510751 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f88ac060eb0 msgr2=0x7f88ac157350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:07.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.411+0000 7f88ab7fe700 1 --2- 172.21.15.47:0/3098510751 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f88ac060eb0 0x7f88ac157350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:07.429 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.411+0000 7f88ab7fe700 1 -- 172.21.15.47:0/3098510751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88ac0a72f0 msgr2=0x7f88ac0b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:07.429 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.411+0000 7f88ab7fe700 1 --2- 172.21.15.47:0/3098510751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88ac0a72f0 0x7f88ac0b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:07.429 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.411+0000 7f88ab7fe700 1 -- 172.21.15.47:0/3098510751 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a000b1e0 con 0x7f88ac159dc0 2022-06-29T15:43:07.430 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.411+0000 7f88aa7fc700 1 --2- 172.21.15.47:0/3098510751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88ac0a72f0 0x7f88ac0b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:43:07.430 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.411+0000 7f88ab7fe700 1 --2- 172.21.15.47:0/3098510751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88ac159dc0 0x7f88ac15a240 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7f88a400c000 tx=0x7f88a4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:07.430 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.411+0000 7f889bfff700 1 -- 172.21.15.47:0/3098510751 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88a4010780 con 0x7f88ac159dc0 2022-06-29T15:43:07.431 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.411+0000 7f889bfff700 1 -- 172.21.15.47:0/3098510751 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a40108e0 con 0x7f88ac159dc0 2022-06-29T15:43:07.431 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.411+0000 7f88abfff700 1 -- 172.21.15.47:0/3098510751 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88ac156eb0 con 0x7f88ac159dc0 2022-06-29T15:43:07.431 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.411+0000 7f889bfff700 1 -- 172.21.15.47:0/3098510751 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88a4010c40 con 0x7f88ac159dc0 2022-06-29T15:43:07.432 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.411+0000 7f88abfff700 1 -- 172.21.15.47:0/3098510751 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88ac0aeed0 con 0x7f88ac159dc0 2022-06-29T15:43:07.432 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.415+0000 7f889bfff700 1 -- 172.21.15.47:0/3098510751 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f88a401c460 con 0x7f88ac159dc0 2022-06-29T15:43:07.432 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.415+0000 7f88abfff700 1 -- 172.21.15.47:0/3098510751 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88ac0573d0 con 0x7f88ac159dc0 2022-06-29T15:43:07.433 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.415+0000 7f889bfff700 1 --2- 172.21.15.47:0/3098510751 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f88880539e0 0x7f8888055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:07.433 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.415+0000 7f889bfff700 1 -- 172.21.15.47:0/3098510751 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f88a406c1a0 con 0x7f88ac159dc0 2022-06-29T15:43:07.434 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.415+0000 7f88aaffd700 1 --2- 172.21.15.47:0/3098510751 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f88880539e0 0x7f8888055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:07.434 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.415+0000 7f88aaffd700 1 --2- 172.21.15.47:0/3098510751 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f88880539e0 0x7f8888055ea0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f88a0018000 tx=0x7f88a0014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:07.435 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.423+0000 7f889bfff700 1 -- 172.21.15.47:0/3098510751 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f88a4043240 con 0x7f88ac159dc0 2022-06-29T15:43:07.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.595+0000 7f88abfff700 1 -- 172.21.15.47:0/3098510751 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f88ac069e20 con 0x7f88ac159dc0 2022-06-29T15:43:07.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.603+0000 7f889bfff700 1 -- 172.21.15.47:0/3098510751 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 updated v13) v1 ==== 97+0+0 (secure 0 0 0) 0x7f88a4046860 con 0x7f88ac159dc0 2022-06-29T15:43:07.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.603+0000 7f88abfff700 1 -- 172.21.15.47:0/3098510751 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f88880539e0 msgr2=0x7f8888055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:07.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.603+0000 7f88abfff700 1 --2- 172.21.15.47:0/3098510751 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f88880539e0 0x7f8888055ea0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f88a0018000 tx=0x7f88a0014000).stop 2022-06-29T15:43:07.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.603+0000 7f88abfff700 1 -- 172.21.15.47:0/3098510751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88ac159dc0 msgr2=0x7f88ac15a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:07.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.603+0000 7f88abfff700 1 --2- 172.21.15.47:0/3098510751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88ac159dc0 0x7f88ac15a240 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7f88a400c000 tx=0x7f88a4006df0).stop 2022-06-29T15:43:07.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.603+0000 7f88abfff700 1 -- 172.21.15.47:0/3098510751 shutdown_connections 2022-06-29T15:43:07.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.603+0000 7f88abfff700 1 --2- 172.21.15.47:0/3098510751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88ac159dc0 0x7f88ac15a240 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:07.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.603+0000 7f88abfff700 1 --2- 172.21.15.47:0/3098510751 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f88880539e0 0x7f8888055ea0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:07.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.603+0000 7f88abfff700 1 --2- 172.21.15.47:0/3098510751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88ac0a72f0 0x7f88ac0b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:07.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.603+0000 7f88abfff700 1 --2- 172.21.15.47:0/3098510751 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f88ac060eb0 0x7f88ac157350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:07.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.603+0000 7f88abfff700 1 -- 172.21.15.47:0/3098510751 >> 172.21.15.47:0/3098510751 conn(0x7f88ac062560 msgr2=0x7f88ac144750 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:07.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.603+0000 7f88abfff700 1 -- 172.21.15.47:0/3098510751 shutdown_connections 2022-06-29T15:43:07.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.603+0000 7f88abfff700 1 -- 172.21.15.47:0/3098510751 wait complete. 2022-06-29T15:43:07.615 INFO:teuthology.orchestra.run.smithi047.stderr:updated 2022-06-29T15:43:07.624 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:43:07.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.719+0000 7f3ffd7a0700 1 Processor -- start 2022-06-29T15:43:07.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.719+0000 7f3ffd7a0700 1 -- start start 2022-06-29T15:43:07.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.723+0000 7f3ffd7a0700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3ff80b19a0 0x7f3ff805fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:07.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.723+0000 7f3ffd7a0700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ff80a97e0 con 0x7f3ff81437d0 2022-06-29T15:43:07.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.723+0000 7f3ffd7a0700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ff80b1d80 con 0x7f3ff80acff0 2022-06-29T15:43:07.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.723+0000 7f3ffd7a0700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ff805f430 con 0x7f3ff80b19a0 2022-06-29T15:43:07.731 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.723+0000 7f3ffcf9f700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f3ff81437d0 0x7f3ff8143c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:60790/0 (socket says 172.21.15.47:60790) 2022-06-29T15:43:07.731 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.723+0000 7f3ffcf9f700 1 -- 172.21.15.47:0/1825574436 learned_addr learned my addr 172.21.15.47:0/1825574436 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:43:07.731 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.723+0000 7f3ff7fff700 1 --2- 172.21.15.47:0/1825574436 >> v2:172.21.15.64:3300/0 conn(0x7f3ff80b19a0 0x7f3ff805fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:07.731 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.723+0000 7f3ff77fe700 1 -- 172.21.15.47:0/1825574436 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1979952075 0 0) 0x7f3ff80a97e0 con 0x7f3ff81437d0 2022-06-29T15:43:07.732 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.723+0000 7f3ff77fe700 1 -- 172.21.15.47:0/1825574436 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fd8003260 con 0x7f3ff81437d0 2022-06-29T15:43:07.732 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.723+0000 7f3ff77fe700 1 -- 172.21.15.47:0/1825574436 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1336452961 0 0) 0x7f3ff80b1d80 con 0x7f3ff80acff0 2022-06-29T15:43:07.732 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.723+0000 7f3ff77fe700 1 -- 172.21.15.47:0/1825574436 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ff80a97e0 con 0x7f3ff80acff0 2022-06-29T15:43:07.732 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.723+0000 7f3ff7fff700 1 -- 172.21.15.47:0/1825574436 >> v1:172.21.15.64:6789/0 conn(0x7f3ff80acff0 legacy=0x7f3ff80ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:07.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.723+0000 7f3ff77fe700 1 -- 172.21.15.47:0/1825574436 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3888027537 0 0) 0x7f3fd8003260 con 0x7f3ff81437d0 2022-06-29T15:43:07.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.723+0000 7f3ff7fff700 1 -- 172.21.15.47:0/1825574436 >> v1:172.21.15.47:6789/0 conn(0x7f3ff81437d0 legacy=0x7f3ff8143c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:07.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.723+0000 7f3ff7fff700 1 -- 172.21.15.47:0/1825574436 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff80a92a0 con 0x7f3ff80b19a0 2022-06-29T15:43:07.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.723+0000 7f3ff7fff700 1 --2- 172.21.15.47:0/1825574436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ff80b19a0 0x7f3ff805fd10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f3ff0001490 tx=0x7f3fe8006df0).ready entity=mon.1 client_cookie=e1449db8ba8eba09 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:07.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.723+0000 7f3ff77fe700 1 -- 172.21.15.47:0/1825574436 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ff00019e0 con 0x7f3ff80b19a0 2022-06-29T15:43:07.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.723+0000 7f3ff77fe700 1 -- 172.21.15.47:0/1825574436 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe8000e90 con 0x7f3ff80b19a0 2022-06-29T15:43:07.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.723+0000 7f3ff77fe700 1 -- 172.21.15.47:0/1825574436 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3fe8012460 con 0x7f3ff80b19a0 2022-06-29T15:43:07.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.723+0000 7f3ffd7a0700 1 -- 172.21.15.47:0/1825574436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ff80b19a0 msgr2=0x7f3ff805fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:07.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.723+0000 7f3ffd7a0700 1 --2- 172.21.15.47:0/1825574436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ff80b19a0 0x7f3ff805fd10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f3ff0001490 tx=0x7f3fe8006df0).stop 2022-06-29T15:43:07.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.723+0000 7f3ffd7a0700 1 -- 172.21.15.47:0/1825574436 shutdown_connections 2022-06-29T15:43:07.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.723+0000 7f3ffd7a0700 1 --2- 172.21.15.47:0/1825574436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ff80b19a0 0x7f3ff805fd10 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:07.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.723+0000 7f3ffd7a0700 1 -- 172.21.15.47:0/1825574436 >> 172.21.15.47:0/1825574436 conn(0x7f3ff8062df0 msgr2=0x7f3ff80631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:07.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.723+0000 7f3ffd7a0700 1 -- 172.21.15.47:0/1825574436 shutdown_connections 2022-06-29T15:43:07.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.723+0000 7f3ffd7a0700 1 -- 172.21.15.47:0/1825574436 wait complete. 2022-06-29T15:43:07.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.727+0000 7f3ffd7a0700 1 Processor -- start 2022-06-29T15:43:07.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.727+0000 7f3ffd7a0700 1 -- start start 2022-06-29T15:43:07.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.731+0000 7f3ffd7a0700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3ff80acff0 0x7f3ff80646a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:07.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.731+0000 7f3ffd7a0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ff80b19a0 0x7f3ff81456c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:07.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.731+0000 7f3ffd7a0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ff81437d0 0x7f3ff8066040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:07.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.731+0000 7f3ffd7a0700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff809b130 con 0x7f3ff81437d0 2022-06-29T15:43:07.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.731+0000 7f3ffd7a0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff809a3a0 con 0x7f3ff80b19a0 2022-06-29T15:43:07.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.731+0000 7f3ffd7a0700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff809a640 con 0x7f3ff80acff0 2022-06-29T15:43:07.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.731+0000 7f3ffcf9f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ff81437d0 0x7f3ff8066040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:07.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.731+0000 7f3ffcf9f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ff81437d0 0x7f3ff8066040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47010/0 (socket says 172.21.15.47:47010) 2022-06-29T15:43:07.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.731+0000 7f3ffcf9f700 1 -- 172.21.15.47:0/1321902489 learned_addr learned my addr 172.21.15.47:0/1321902489 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:07.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.731+0000 7f3ff7fff700 1 --2- 172.21.15.47:0/1321902489 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3ff80acff0 0x7f3ff80646a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:07.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.731+0000 7f3feffff700 1 --2- 172.21.15.47:0/1321902489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ff80b19a0 0x7f3ff81456c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:07.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.731+0000 7f3ffcf9f700 1 -- 172.21.15.47:0/1321902489 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3ff80acff0 msgr2=0x7f3ff80646a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:07.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.731+0000 7f3ffcf9f700 1 --2- 172.21.15.47:0/1321902489 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3ff80acff0 0x7f3ff80646a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:07.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.731+0000 7f3ffcf9f700 1 -- 172.21.15.47:0/1321902489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ff80b19a0 msgr2=0x7f3ff81456c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:07.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.731+0000 7f3ffcf9f700 1 --2- 172.21.15.47:0/1321902489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ff80b19a0 0x7f3ff81456c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:07.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.731+0000 7f3ffcf9f700 1 -- 172.21.15.47:0/1321902489 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe800b180 con 0x7f3ff81437d0 2022-06-29T15:43:07.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.731+0000 7f3ffcf9f700 1 --2- 172.21.15.47:0/1321902489 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ff81437d0 0x7f3ff8066040 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f3ff0001bc0 tx=0x7f3ff0001a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:07.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.731+0000 7f3ff5ffb700 1 -- 172.21.15.47:0/1321902489 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ff0004ab0 con 0x7f3ff81437d0 2022-06-29T15:43:07.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.731+0000 7f3ff5ffb700 1 -- 172.21.15.47:0/1321902489 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ff0004c10 con 0x7f3ff81437d0 2022-06-29T15:43:07.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.731+0000 7f3ff5ffb700 1 -- 172.21.15.47:0/1321902489 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ff0007170 con 0x7f3ff81437d0 2022-06-29T15:43:07.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.731+0000 7f3ffd7a0700 1 -- 172.21.15.47:0/1321902489 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ff809a8e0 con 0x7f3ff81437d0 2022-06-29T15:43:07.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.731+0000 7f3ffd7a0700 1 -- 172.21.15.47:0/1321902489 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ff809adb0 con 0x7f3ff81437d0 2022-06-29T15:43:07.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.731+0000 7f3ffd7a0700 1 -- 172.21.15.47:0/1321902489 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ff8065de0 con 0x7f3ff81437d0 2022-06-29T15:43:07.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.731+0000 7f3ff5ffb700 1 -- 172.21.15.47:0/1321902489 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3ff001b070 con 0x7f3ff81437d0 2022-06-29T15:43:07.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.735+0000 7f3ff5ffb700 1 --2- 172.21.15.47:0/1321902489 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3fdc053b00 0x7f3fdc055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:07.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.735+0000 7f3ff7fff700 1 --2- 172.21.15.47:0/1321902489 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3fdc053b00 0x7f3fdc055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:07.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.735+0000 7f3ff5ffb700 1 -- 172.21.15.47:0/1321902489 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f3ff006b6c0 con 0x7f3ff81437d0 2022-06-29T15:43:07.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.735+0000 7f3ff7fff700 1 --2- 172.21.15.47:0/1321902489 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3fdc053b00 0x7f3fdc055fc0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f3fe801a000 tx=0x7f3fe8014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:07.749 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.743+0000 7f3ff5ffb700 1 -- 172.21.15.47:0/1321902489 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3ff00426b0 con 0x7f3ff81437d0 2022-06-29T15:43:07.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.867+0000 7f3ffd7a0700 1 -- 172.21.15.47:0/1321902489 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ff80573d0 con 0x7f3ff81437d0 2022-06-29T15:43:07.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.871+0000 7f3ff5ffb700 1 -- 172.21.15.47:0/1321902489 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+13414 (secure 0 0 0) 0x7f3ff0045cd0 con 0x7f3ff81437d0 2022-06-29T15:43:07.878 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:43:07.879 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":32,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:43:05.348616+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:43:07.880 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.871+0000 7f3ffd7a0700 1 -- 172.21.15.47:0/1321902489 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3fdc053b00 msgr2=0x7f3fdc055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:07.880 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.871+0000 7f3ffd7a0700 1 --2- 172.21.15.47:0/1321902489 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3fdc053b00 0x7f3fdc055fc0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f3fe801a000 tx=0x7f3fe8014000).stop 2022-06-29T15:43:07.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.871+0000 7f3ffd7a0700 1 -- 172.21.15.47:0/1321902489 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ff81437d0 msgr2=0x7f3ff8066040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:07.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.871+0000 7f3ffd7a0700 1 --2- 172.21.15.47:0/1321902489 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ff81437d0 0x7f3ff8066040 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f3ff0001bc0 tx=0x7f3ff0001a40).stop 2022-06-29T15:43:07.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.871+0000 7f3ffd7a0700 1 -- 172.21.15.47:0/1321902489 shutdown_connections 2022-06-29T15:43:07.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.871+0000 7f3ffd7a0700 1 --2- 172.21.15.47:0/1321902489 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ff81437d0 0x7f3ff8066040 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:07.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.871+0000 7f3ffd7a0700 1 --2- 172.21.15.47:0/1321902489 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3fdc053b00 0x7f3fdc055fc0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:07.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.871+0000 7f3ffd7a0700 1 --2- 172.21.15.47:0/1321902489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ff80b19a0 0x7f3ff81456c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:07.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.871+0000 7f3ffd7a0700 1 --2- 172.21.15.47:0/1321902489 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3ff80acff0 0x7f3ff80646a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:07.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.871+0000 7f3ffd7a0700 1 -- 172.21.15.47:0/1321902489 >> 172.21.15.47:0/1321902489 conn(0x7f3ff8062df0 msgr2=0x7f3ff80605d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:07.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.871+0000 7f3ffd7a0700 1 -- 172.21.15.47:0/1321902489 shutdown_connections 2022-06-29T15:43:07.883 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.875+0000 7f3ffd7a0700 1 -- 172.21.15.47:0/1321902489 wait complete. 2022-06-29T15:43:07.892 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-29T15:42:16.971431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-29T15:42:25.290884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-06-29T15:43:07.893 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-29T15:43:08.002 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.995+0000 7f0e1e580700 1 Processor -- start 2022-06-29T15:43:08.002 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.995+0000 7f0e1e580700 1 -- start start 2022-06-29T15:43:08.004 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.995+0000 7f0e1e580700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0e18060e10 0x7f0e180b07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:08.004 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.995+0000 7f0e1e580700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e180ae550 con 0x7f0e18061f40 2022-06-29T15:43:08.005 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.995+0000 7f0e1e580700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e18061800 con 0x7f0e180afb80 2022-06-29T15:43:08.005 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.995+0000 7f0e1e580700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e18061aa0 con 0x7f0e18060e10 2022-06-29T15:43:08.005 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.999+0000 7f0e1dd7f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0e18060e10 0x7f0e180b07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:08.005 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.999+0000 7f0e1dd7f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0e18060e10 0x7f0e180b07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47018/0 (socket says 172.21.15.47:47018) 2022-06-29T15:43:08.006 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.999+0000 7f0e1dd7f700 1 -- 172.21.15.47:0/211417207 learned_addr learned my addr 172.21.15.47:0/211417207 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:08.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.999+0000 7f0e1dd7f700 1 -- 172.21.15.47:0/211417207 >> v1:172.21.15.64:6789/0 conn(0x7f0e180afb80 legacy=0x7f0e1813d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:08.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.999+0000 7f0e1dd7f700 1 -- 172.21.15.47:0/211417207 >> v1:172.21.15.47:6789/0 conn(0x7f0e18061f40 legacy=0x7f0e1813fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:08.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.999+0000 7f0e1dd7f700 1 -- 172.21.15.47:0/211417207 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e1813d980 con 0x7f0e18060e10 2022-06-29T15:43:08.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.999+0000 7f0e1dd7f700 1 --2- 172.21.15.47:0/211417207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e18060e10 0x7f0e180b07b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f0e180af160 tx=0x7f0e10009290).ready entity=mon.0 client_cookie=d6292fa0fb4a93b4 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:08.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.999+0000 7f0e07fff700 1 -- 172.21.15.47:0/211417207 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e1000b860 con 0x7f0e18060e10 2022-06-29T15:43:08.009 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.999+0000 7f0e07fff700 1 -- 172.21.15.47:0/211417207 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e1000b9c0 con 0x7f0e18060e10 2022-06-29T15:43:08.009 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.999+0000 7f0e07fff700 1 -- 172.21.15.47:0/211417207 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e1000bb30 con 0x7f0e18060e10 2022-06-29T15:43:08.009 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.999+0000 7f0e1e580700 1 -- 172.21.15.47:0/211417207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e18060e10 msgr2=0x7f0e180b07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:08.009 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.999+0000 7f0e1e580700 1 --2- 172.21.15.47:0/211417207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e18060e10 0x7f0e180b07b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f0e180af160 tx=0x7f0e10009290).stop 2022-06-29T15:43:08.010 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.999+0000 7f0e1e580700 1 -- 172.21.15.47:0/211417207 shutdown_connections 2022-06-29T15:43:08.010 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.999+0000 7f0e1e580700 1 --2- 172.21.15.47:0/211417207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e18060e10 0x7f0e180b07b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:08.010 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.999+0000 7f0e1e580700 1 -- 172.21.15.47:0/211417207 >> 172.21.15.47:0/211417207 conn(0x7f0e180b6430 msgr2=0x7f0e180b6810 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:08.010 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.999+0000 7f0e1e580700 1 -- 172.21.15.47:0/211417207 shutdown_connections 2022-06-29T15:43:08.011 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:07.999+0000 7f0e1e580700 1 -- 172.21.15.47:0/211417207 wait complete. 2022-06-29T15:43:08.013 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.003+0000 7f0e1e580700 1 Processor -- start 2022-06-29T15:43:08.013 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.003+0000 7f0e1e580700 1 -- start start 2022-06-29T15:43:08.013 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.003+0000 7f0e1e580700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e18060e10 0x7f0e180b5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:08.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.003+0000 7f0e1e580700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e18061f40 0x7f0e180aa240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:08.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.003+0000 7f0e1e580700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0e180afb80 0x7f0e18139e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:08.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.003+0000 7f0e1e580700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e180aa730 con 0x7f0e18060e10 2022-06-29T15:43:08.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.003+0000 7f0e1e580700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1813ad70 con 0x7f0e18061f40 2022-06-29T15:43:08.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.003+0000 7f0e1e580700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1813b090 con 0x7f0e180afb80 2022-06-29T15:43:08.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.003+0000 7f0e1d57e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e18060e10 0x7f0e180b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:08.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.003+0000 7f0e1d57e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e18060e10 0x7f0e180b5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47020/0 (socket says 172.21.15.47:47020) 2022-06-29T15:43:08.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.003+0000 7f0e1d57e700 1 -- 172.21.15.47:0/74694686 learned_addr learned my addr 172.21.15.47:0/74694686 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:08.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.003+0000 7f0e1cd7d700 1 --2- 172.21.15.47:0/74694686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e18061f40 0x7f0e180aa240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:08.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.003+0000 7f0e1dd7f700 1 --2- 172.21.15.47:0/74694686 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0e180afb80 0x7f0e18139e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:08.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.003+0000 7f0e1d57e700 1 -- 172.21.15.47:0/74694686 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0e180afb80 msgr2=0x7f0e18139e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:08.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.003+0000 7f0e1d57e700 1 --2- 172.21.15.47:0/74694686 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0e180afb80 0x7f0e18139e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:08.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.003+0000 7f0e1d57e700 1 -- 172.21.15.47:0/74694686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e18061f40 msgr2=0x7f0e180aa240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:08.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.003+0000 7f0e1d57e700 1 --2- 172.21.15.47:0/74694686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e18061f40 0x7f0e180aa240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:08.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.003+0000 7f0e1d57e700 1 -- 172.21.15.47:0/74694686 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e10009530 con 0x7f0e18060e10 2022-06-29T15:43:08.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.003+0000 7f0e1d57e700 1 --2- 172.21.15.47:0/74694686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e18060e10 0x7f0e180b5440 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7f0e180b2760 tx=0x7f0e14007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:08.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.003+0000 7f0e067fc700 1 -- 172.21.15.47:0/74694686 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e14012400 con 0x7f0e18060e10 2022-06-29T15:43:08.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.007+0000 7f0e067fc700 1 -- 172.21.15.47:0/74694686 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e14012560 con 0x7f0e18060e10 2022-06-29T15:43:08.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.007+0000 7f0e067fc700 1 -- 172.21.15.47:0/74694686 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e140128c0 con 0x7f0e18060e10 2022-06-29T15:43:08.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.007+0000 7f0e1e580700 1 -- 172.21.15.47:0/74694686 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e1813b330 con 0x7f0e18060e10 2022-06-29T15:43:08.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.007+0000 7f0e1e580700 1 -- 172.21.15.47:0/74694686 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e181394d0 con 0x7f0e18060e10 2022-06-29T15:43:08.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.007+0000 7f0e067fc700 1 -- 172.21.15.47:0/74694686 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0e14019030 con 0x7f0e18060e10 2022-06-29T15:43:08.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.007+0000 7f0e1e580700 1 -- 172.21.15.47:0/74694686 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e180573d0 con 0x7f0e18060e10 2022-06-29T15:43:08.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.007+0000 7f0e067fc700 1 --2- 172.21.15.47:0/74694686 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0df8053950 0x7f0df8055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:08.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.007+0000 7f0e067fc700 1 -- 172.21.15.47:0/74694686 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f0e1406afb0 con 0x7f0e18060e10 2022-06-29T15:43:08.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.007+0000 7f0e1cd7d700 1 --2- 172.21.15.47:0/74694686 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0df8053950 0x7f0df8055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:08.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.007+0000 7f0e1cd7d700 1 --2- 172.21.15.47:0/74694686 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0df8053950 0x7f0df8055e10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f0e0800d000 tx=0x7f0e08007b90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:08.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.011+0000 7f0e067fc700 1 -- 172.21.15.47:0/74694686 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0e140420b0 con 0x7f0e18060e10 2022-06-29T15:43:08.147 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.139+0000 7f0e1e580700 1 -- 172.21.15.47:0/74694686 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f0e1806a0f0 con 0x7f0e18060e10 2022-06-29T15:43:08.148 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.139+0000 7f0e067fc700 1 -- 172.21.15.47:0/74694686 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v32) v1 ==== 110+0+10 (secure 0 0 0) 0x7f0e140456d0 con 0x7f0e18060e10 2022-06-29T15:43:08.148 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2022-06-29T15:43:08.149 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.143+0000 7f0e1e580700 1 -- 172.21.15.47:0/74694686 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0df8053950 msgr2=0x7f0df8055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:08.150 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.143+0000 7f0e1e580700 1 --2- 172.21.15.47:0/74694686 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0df8053950 0x7f0df8055e10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f0e0800d000 tx=0x7f0e08007b90).stop 2022-06-29T15:43:08.150 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.143+0000 7f0e1e580700 1 -- 172.21.15.47:0/74694686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e18060e10 msgr2=0x7f0e180b5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:08.151 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.143+0000 7f0e1e580700 1 --2- 172.21.15.47:0/74694686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e18060e10 0x7f0e180b5440 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7f0e180b2760 tx=0x7f0e14007c50).stop 2022-06-29T15:43:08.151 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.143+0000 7f0e1e580700 1 -- 172.21.15.47:0/74694686 shutdown_connections 2022-06-29T15:43:08.152 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.143+0000 7f0e1e580700 1 --2- 172.21.15.47:0/74694686 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0e180afb80 0x7f0e18139e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:08.152 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.143+0000 7f0e1e580700 1 --2- 172.21.15.47:0/74694686 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0df8053950 0x7f0df8055e10 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:08.152 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.143+0000 7f0e1e580700 1 --2- 172.21.15.47:0/74694686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e18061f40 0x7f0e180aa240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:08.152 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.143+0000 7f0e1e580700 1 --2- 172.21.15.47:0/74694686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e18060e10 0x7f0e180b5440 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:08.153 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.143+0000 7f0e1e580700 1 -- 172.21.15.47:0/74694686 >> 172.21.15.47:0/74694686 conn(0x7f0e180b6430 msgr2=0x7f0e18138bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:08.153 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.143+0000 7f0e1e580700 1 -- 172.21.15.47:0/74694686 shutdown_connections 2022-06-29T15:43:08.153 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.143+0000 7f0e1e580700 1 -- 172.21.15.47:0/74694686 wait complete. 2022-06-29T15:43:08.162 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-29T15:43:08.265 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.259+0000 7f9a6c00a700 1 Processor -- start 2022-06-29T15:43:08.265 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.259+0000 7f9a6c00a700 1 -- start start 2022-06-29T15:43:08.268 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.259+0000 7f9a6c00a700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9a640a72b0 0x7f9a64144e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:08.268 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.259+0000 7f9a6c00a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9a6414b4e0 0x7f9a64147930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:08.269 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.259+0000 7f9a6c00a700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a64060e50 con 0x7f9a64060a70 2022-06-29T15:43:08.269 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.259+0000 7f9a6c00a700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a640a8120 con 0x7f9a6414b4e0 2022-06-29T15:43:08.269 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.259+0000 7f9a6c00a700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a64147540 con 0x7f9a640a72b0 2022-06-29T15:43:08.269 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.259+0000 7f9a6a5a7700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9a6414b4e0 0x7f9a64147930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:08.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.259+0000 7f9a6a5a7700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9a6414b4e0 0x7f9a64147930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47032/0 (socket says 172.21.15.47:47032) 2022-06-29T15:43:08.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.259+0000 7f9a6a5a7700 1 -- 172.21.15.47:0/935316501 learned_addr learned my addr 172.21.15.47:0/935316501 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:08.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.259+0000 7f9a695a5700 1 --2- 172.21.15.47:0/935316501 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9a640a72b0 0x7f9a64144e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:08.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.259+0000 7f9a6a5a7700 1 -- 172.21.15.47:0/935316501 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9a640a72b0 msgr2=0x7f9a64144e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:43:08.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.259+0000 7f9a6a5a7700 1 --2- 172.21.15.47:0/935316501 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9a640a72b0 0x7f9a64144e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:08.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.259+0000 7f9a6a5a7700 1 -- 172.21.15.47:0/935316501 >> v1:172.21.15.64:6789/0 conn(0x7f9a64060a70 legacy=0x7f9a64144750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:08.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.259+0000 7f9a6a5a7700 1 -- 172.21.15.47:0/935316501 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a640a8ac0 con 0x7f9a6414b4e0 2022-06-29T15:43:08.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.259+0000 7f9a6a5a7700 1 --2- 172.21.15.47:0/935316501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a6414b4e0 0x7f9a64147930 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f9a5c0032f0 tx=0x7f9a5c009290).ready entity=mon.0 client_cookie=92bf3db747df1f9d server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:08.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.259+0000 7f9a68da4700 1 -- 172.21.15.47:0/935316501 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a5c00b8a0 con 0x7f9a6414b4e0 2022-06-29T15:43:08.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.259+0000 7f9a68da4700 1 -- 172.21.15.47:0/935316501 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a5c00ba00 con 0x7f9a6414b4e0 2022-06-29T15:43:08.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.259+0000 7f9a68da4700 1 -- 172.21.15.47:0/935316501 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a5c00bb70 con 0x7f9a6414b4e0 2022-06-29T15:43:08.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.259+0000 7f9a6c00a700 1 -- 172.21.15.47:0/935316501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a6414b4e0 msgr2=0x7f9a64147930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:08.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.259+0000 7f9a6c00a700 1 --2- 172.21.15.47:0/935316501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a6414b4e0 0x7f9a64147930 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f9a5c0032f0 tx=0x7f9a5c009290).stop 2022-06-29T15:43:08.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.259+0000 7f9a6c00a700 1 -- 172.21.15.47:0/935316501 shutdown_connections 2022-06-29T15:43:08.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.259+0000 7f9a6c00a700 1 --2- 172.21.15.47:0/935316501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a6414b4e0 0x7f9a64147930 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:08.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.259+0000 7f9a6c00a700 1 --2- 172.21.15.47:0/935316501 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9a640a72b0 0x7f9a64144e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:08.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.259+0000 7f9a6c00a700 1 -- 172.21.15.47:0/935316501 >> 172.21.15.47:0/935316501 conn(0x7f9a640626f0 msgr2=0x7f9a64062ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:08.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.259+0000 7f9a6c00a700 1 -- 172.21.15.47:0/935316501 shutdown_connections 2022-06-29T15:43:08.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.259+0000 7f9a6c00a700 1 -- 172.21.15.47:0/935316501 wait complete. 2022-06-29T15:43:08.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.263+0000 7f9a6c00a700 1 Processor -- start 2022-06-29T15:43:08.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.263+0000 7f9a6c00a700 1 -- start start 2022-06-29T15:43:08.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.263+0000 7f9a6c00a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a64060a70 0x7f9a640acc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:08.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.263+0000 7f9a6c00a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a640a72b0 0x7f9a640aa430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:08.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.263+0000 7f9a6c00a700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9a6414b4e0 0x7f9a640aa920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:08.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.263+0000 7f9a6c00a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a640abc00 con 0x7f9a640a72b0 2022-06-29T15:43:08.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.263+0000 7f9a6c00a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a640abea0 con 0x7f9a64060a70 2022-06-29T15:43:08.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.263+0000 7f9a6c00a700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a64157440 con 0x7f9a6414b4e0 2022-06-29T15:43:08.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.263+0000 7f9a695a5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a640a72b0 0x7f9a640aa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:08.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.263+0000 7f9a695a5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a640a72b0 0x7f9a640aa430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47036/0 (socket says 172.21.15.47:47036) 2022-06-29T15:43:08.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.263+0000 7f9a695a5700 1 -- 172.21.15.47:0/2726374221 learned_addr learned my addr 172.21.15.47:0/2726374221 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:08.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.263+0000 7f9a69da6700 1 --2- 172.21.15.47:0/2726374221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a64060a70 0x7f9a640acc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:08.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.263+0000 7f9a6a5a7700 1 --2- 172.21.15.47:0/2726374221 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9a6414b4e0 0x7f9a640aa920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:08.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.263+0000 7f9a695a5700 1 -- 172.21.15.47:0/2726374221 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9a6414b4e0 msgr2=0x7f9a640aa920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:08.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.263+0000 7f9a695a5700 1 --2- 172.21.15.47:0/2726374221 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9a6414b4e0 0x7f9a640aa920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:08.282 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.263+0000 7f9a695a5700 1 -- 172.21.15.47:0/2726374221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a64060a70 msgr2=0x7f9a640acc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:08.282 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.263+0000 7f9a695a5700 1 --2- 172.21.15.47:0/2726374221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a64060a70 0x7f9a640acc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:08.282 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.263+0000 7f9a695a5700 1 -- 172.21.15.47:0/2726374221 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a5c009530 con 0x7f9a640a72b0 2022-06-29T15:43:08.282 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.263+0000 7f9a695a5700 1 --2- 172.21.15.47:0/2726374221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a640a72b0 0x7f9a640aa430 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7f9a5400c000 tx=0x7f9a54006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:08.283 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.263+0000 7f9a52ffd700 1 -- 172.21.15.47:0/2726374221 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a54010780 con 0x7f9a640a72b0 2022-06-29T15:43:08.283 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.263+0000 7f9a52ffd700 1 -- 172.21.15.47:0/2726374221 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a540108e0 con 0x7f9a640a72b0 2022-06-29T15:43:08.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.263+0000 7f9a6c00a700 1 -- 172.21.15.47:0/2726374221 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a64157160 con 0x7f9a640a72b0 2022-06-29T15:43:08.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.263+0000 7f9a52ffd700 1 -- 172.21.15.47:0/2726374221 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a54010c40 con 0x7f9a640a72b0 2022-06-29T15:43:08.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.263+0000 7f9a6c00a700 1 -- 172.21.15.47:0/2726374221 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a640b6150 con 0x7f9a640a72b0 2022-06-29T15:43:08.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.267+0000 7f9a52ffd700 1 -- 172.21.15.47:0/2726374221 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9a5401c460 con 0x7f9a640a72b0 2022-06-29T15:43:08.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.267+0000 7f9a6c00a700 1 -- 172.21.15.47:0/2726374221 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a64157600 con 0x7f9a640a72b0 2022-06-29T15:43:08.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.267+0000 7f9a52ffd700 1 --2- 172.21.15.47:0/2726374221 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9a440539e0 0x7f9a44055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:08.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.267+0000 7f9a52ffd700 1 -- 172.21.15.47:0/2726374221 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f9a5406c1e0 con 0x7f9a640a72b0 2022-06-29T15:43:08.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.267+0000 7f9a69da6700 1 --2- 172.21.15.47:0/2726374221 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9a440539e0 0x7f9a44055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:08.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.267+0000 7f9a69da6700 1 --2- 172.21.15.47:0/2726374221 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9a440539e0 0x7f9a44055ea0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f9a6000d000 tx=0x7f9a60007b50).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:08.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.275+0000 7f9a52ffd700 1 -- 172.21.15.47:0/2726374221 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9a54043170 con 0x7f9a640a72b0 2022-06-29T15:43:08.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.399+0000 7f9a6c00a700 1 -- 172.21.15.47:0/2726374221 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9a640573d0 con 0x7f9a640a72b0 2022-06-29T15:43:08.409 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.403+0000 7f9a52ffd700 1 -- 172.21.15.47:0/2726374221 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v32) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9a54046790 con 0x7f9a640a72b0 2022-06-29T15:43:08.409 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:43:08.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.403+0000 7f9a6c00a700 1 -- 172.21.15.47:0/2726374221 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9a440539e0 msgr2=0x7f9a44055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:08.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.403+0000 7f9a6c00a700 1 --2- 172.21.15.47:0/2726374221 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9a440539e0 0x7f9a44055ea0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f9a6000d000 tx=0x7f9a60007b50).stop 2022-06-29T15:43:08.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.403+0000 7f9a6c00a700 1 -- 172.21.15.47:0/2726374221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a640a72b0 msgr2=0x7f9a640aa430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:08.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.403+0000 7f9a6c00a700 1 --2- 172.21.15.47:0/2726374221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a640a72b0 0x7f9a640aa430 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7f9a5400c000 tx=0x7f9a54006df0).stop 2022-06-29T15:43:08.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.403+0000 7f9a6c00a700 1 -- 172.21.15.47:0/2726374221 shutdown_connections 2022-06-29T15:43:08.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.403+0000 7f9a6c00a700 1 --2- 172.21.15.47:0/2726374221 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9a6414b4e0 0x7f9a640aa920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:08.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.403+0000 7f9a6c00a700 1 --2- 172.21.15.47:0/2726374221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a640a72b0 0x7f9a640aa430 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:08.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.403+0000 7f9a6c00a700 1 --2- 172.21.15.47:0/2726374221 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9a440539e0 0x7f9a44055ea0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:08.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.403+0000 7f9a6c00a700 1 --2- 172.21.15.47:0/2726374221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a64060a70 0x7f9a640acc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:08.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.403+0000 7f9a6c00a700 1 -- 172.21.15.47:0/2726374221 >> 172.21.15.47:0/2726374221 conn(0x7f9a640626f0 msgr2=0x7f9a6414c440 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:08.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.403+0000 7f9a6c00a700 1 -- 172.21.15.47:0/2726374221 shutdown_connections 2022-06-29T15:43:08.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.403+0000 7f9a6c00a700 1 -- 172.21.15.47:0/2726374221 wait complete. 2022-06-29T15:43:08.423 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-06-29T15:43:08.525 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.519+0000 7f906e64a700 1 Processor -- start 2022-06-29T15:43:08.525 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.519+0000 7f906e64a700 1 -- start start 2022-06-29T15:43:08.527 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.519+0000 7f906e64a700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f90680b7840 0x7f90680b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:08.527 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.519+0000 7f906e64a700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9068062dd0 0x7f9068066c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:08.527 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.519+0000 7f906e64a700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90680b7c20 con 0x7f90680aea80 2022-06-29T15:43:08.527 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.519+0000 7f906e64a700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90680b3d10 con 0x7f90680b7840 2022-06-29T15:43:08.528 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.519+0000 7f906e64a700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90680670f0 con 0x7f9068062dd0 2022-06-29T15:43:08.528 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.519+0000 7f906d648700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f90680b7840 0x7f90680b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:08.528 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.519+0000 7f906ce47700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9068062dd0 0x7f9068066c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:08.528 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.519+0000 7f906d648700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f90680b7840 0x7f90680b3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:52746/0 (socket says 172.21.15.47:52746) 2022-06-29T15:43:08.529 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.519+0000 7f906d648700 1 -- 172.21.15.47:0/2830040770 learned_addr learned my addr 172.21.15.47:0/2830040770 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:08.529 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.519+0000 7f906de49700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f90680aea80 0x7f9068141760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:40162/0 (socket says 172.21.15.47:40162) 2022-06-29T15:43:08.529 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.519+0000 7f906ce47700 1 -- 172.21.15.47:0/2830040770 >> v2:172.21.15.64:3300/0 conn(0x7f90680b7840 msgr2=0x7f90680b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:43:08.529 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.519+0000 7f906ce47700 1 --2- 172.21.15.47:0/2830040770 >> v2:172.21.15.64:3300/0 conn(0x7f90680b7840 0x7f90680b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:08.530 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.519+0000 7f906ce47700 1 -- 172.21.15.47:0/2830040770 >> v1:172.21.15.64:6789/0 conn(0x7f90680aea80 legacy=0x7f9068141760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:08.530 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.519+0000 7f906ce47700 1 -- 172.21.15.47:0/2830040770 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9068067380 con 0x7f9068062dd0 2022-06-29T15:43:08.530 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.519+0000 7f906ce47700 1 --2- 172.21.15.47:0/2830040770 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9068062dd0 0x7f9068066c00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f9058002f70 tx=0x7f9058006df0).ready entity=mon.2 client_cookie=7dca3f5edae5b7bc server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:08.530 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.519+0000 7f9057fff700 1 -- 172.21.15.47:0/2830040770 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f905800d570 con 0x7f9068062dd0 2022-06-29T15:43:08.531 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.519+0000 7f9057fff700 1 -- 172.21.15.47:0/2830040770 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9058000e90 con 0x7f9068062dd0 2022-06-29T15:43:08.531 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.519+0000 7f9057fff700 1 -- 172.21.15.47:0/2830040770 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f905800d8c0 con 0x7f9068062dd0 2022-06-29T15:43:08.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.519+0000 7f906e64a700 1 -- 172.21.15.47:0/2830040770 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9068062dd0 msgr2=0x7f9068066c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:08.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.519+0000 7f906e64a700 1 --2- 172.21.15.47:0/2830040770 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9068062dd0 0x7f9068066c00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f9058002f70 tx=0x7f9058006df0).stop 2022-06-29T15:43:08.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.519+0000 7f906e64a700 1 -- 172.21.15.47:0/2830040770 shutdown_connections 2022-06-29T15:43:08.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.519+0000 7f906e64a700 1 --2- 172.21.15.47:0/2830040770 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9068062dd0 0x7f9068066c00 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:08.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.519+0000 7f906e64a700 1 --2- 172.21.15.47:0/2830040770 >> v2:172.21.15.64:3300/0 conn(0x7f90680b7840 0x7f90680b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:08.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.519+0000 7f906e64a700 1 -- 172.21.15.47:0/2830040770 >> 172.21.15.47:0/2830040770 conn(0x7f906805f480 msgr2=0x7f90680647d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:08.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.519+0000 7f906e64a700 1 -- 172.21.15.47:0/2830040770 shutdown_connections 2022-06-29T15:43:08.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.519+0000 7f906e64a700 1 -- 172.21.15.47:0/2830040770 wait complete. 2022-06-29T15:43:08.536 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.523+0000 7f906e64a700 1 Processor -- start 2022-06-29T15:43:08.536 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.523+0000 7f906e64a700 1 -- start start 2022-06-29T15:43:08.536 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.523+0000 7f906e64a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9068062dd0 0x7f90680af3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:08.536 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.523+0000 7f906e64a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90680aea80 0x7f9068146900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:08.536 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.523+0000 7f906e64a700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f90680b7840 0x7f9068150190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:08.537 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.523+0000 7f906e64a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9068153870 con 0x7f90680aea80 2022-06-29T15:43:08.537 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.523+0000 7f906e64a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9068153b10 con 0x7f9068062dd0 2022-06-29T15:43:08.537 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.523+0000 7f906e64a700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9068153db0 con 0x7f90680b7840 2022-06-29T15:43:08.537 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.523+0000 7f906ce47700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90680aea80 0x7f9068146900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:08.538 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.523+0000 7f906de49700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f90680b7840 0x7f9068150190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:08.538 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.523+0000 7f906d648700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9068062dd0 0x7f90680af3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:08.538 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.523+0000 7f906ce47700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90680aea80 0x7f9068146900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47052/0 (socket says 172.21.15.47:47052) 2022-06-29T15:43:08.538 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.523+0000 7f906ce47700 1 -- 172.21.15.47:0/3905217561 learned_addr learned my addr 172.21.15.47:0/3905217561 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:08.538 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.523+0000 7f906d648700 1 -- 172.21.15.47:0/3905217561 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f90680b7840 msgr2=0x7f9068150190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:08.539 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.523+0000 7f906d648700 1 --2- 172.21.15.47:0/3905217561 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f90680b7840 0x7f9068150190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:08.539 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.523+0000 7f906d648700 1 -- 172.21.15.47:0/3905217561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90680aea80 msgr2=0x7f9068146900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:08.539 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.523+0000 7f906d648700 1 --2- 172.21.15.47:0/3905217561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90680aea80 0x7f9068146900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:08.539 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.523+0000 7f906d648700 1 -- 172.21.15.47:0/3905217561 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f905800b180 con 0x7f9068062dd0 2022-06-29T15:43:08.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.523+0000 7f906d648700 1 --2- 172.21.15.47:0/3905217561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9068062dd0 0x7f90680af3b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f906400c000 tx=0x7f9064006eb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:08.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.523+0000 7f90567fc700 1 -- 172.21.15.47:0/3905217561 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9064012600 con 0x7f9068062dd0 2022-06-29T15:43:08.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.523+0000 7f90567fc700 1 -- 172.21.15.47:0/3905217561 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9064012760 con 0x7f9068062dd0 2022-06-29T15:43:08.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.523+0000 7f90567fc700 1 -- 172.21.15.47:0/3905217561 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9064012ac0 con 0x7f9068062dd0 2022-06-29T15:43:08.541 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.523+0000 7f906e64a700 1 -- 172.21.15.47:0/3905217561 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9068154050 con 0x7f9068062dd0 2022-06-29T15:43:08.541 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.523+0000 7f906e64a700 1 -- 172.21.15.47:0/3905217561 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90681544f0 con 0x7f9068062dd0 2022-06-29T15:43:08.541 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.527+0000 7f906e64a700 1 -- 172.21.15.47:0/3905217561 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90681541e0 con 0x7f9068062dd0 2022-06-29T15:43:08.542 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.527+0000 7f90567fc700 1 -- 172.21.15.47:0/3905217561 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9064022070 con 0x7f9068062dd0 2022-06-29T15:43:08.542 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.527+0000 7f90567fc700 1 --2- 172.21.15.47:0/3905217561 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f90480539e0 0x7f9048055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:08.542 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.527+0000 7f90567fc700 1 -- 172.21.15.47:0/3905217561 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f906406bee0 con 0x7f9068062dd0 2022-06-29T15:43:08.543 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.527+0000 7f906ce47700 1 --2- 172.21.15.47:0/3905217561 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f90480539e0 0x7f9048055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:08.543 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.527+0000 7f906ce47700 1 --2- 172.21.15.47:0/3905217561 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f90480539e0 0x7f9048055ea0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f905801a000 tx=0x7f9058014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:08.545 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.539+0000 7f90567fc700 1 -- 172.21.15.47:0/3905217561 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9064042fe0 con 0x7f9068062dd0 2022-06-29T15:43:08.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.667+0000 7f906e64a700 1 -- 172.21.15.47:0/3905217561 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f90680573d0 con 0x7f9068062dd0 2022-06-29T15:43:08.676 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.667+0000 7f90567fc700 1 -- 172.21.15.47:0/3905217561 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v32) v1 ==== 84+0+143 (secure 0 0 0) 0x7f9064016090 con 0x7f9068062dd0 2022-06-29T15:43:08.677 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:43:08.677 INFO:teuthology.orchestra.run.smithi047.stdout:[ 2022-06-29T15:43:08.678 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:08.678 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 1, 2022-06-29T15:43:08.679 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": "device_health_metrics" 2022-06-29T15:43:08.679 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:43:08.679 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:43:08.680 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 2, 2022-06-29T15:43:08.680 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": "rbd" 2022-06-29T15:43:08.680 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:43:08.681 INFO:teuthology.orchestra.run.smithi047.stdout:] 2022-06-29T15:43:08.683 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.671+0000 7f906e64a700 1 -- 172.21.15.47:0/3905217561 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f90480539e0 msgr2=0x7f9048055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:08.684 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.671+0000 7f906e64a700 1 --2- 172.21.15.47:0/3905217561 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f90480539e0 0x7f9048055ea0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f905801a000 tx=0x7f9058014000).stop 2022-06-29T15:43:08.684 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.671+0000 7f906e64a700 1 -- 172.21.15.47:0/3905217561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9068062dd0 msgr2=0x7f90680af3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:08.685 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.671+0000 7f906e64a700 1 --2- 172.21.15.47:0/3905217561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9068062dd0 0x7f90680af3b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f906400c000 tx=0x7f9064006eb0).stop 2022-06-29T15:43:08.685 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.671+0000 7f906e64a700 1 -- 172.21.15.47:0/3905217561 shutdown_connections 2022-06-29T15:43:08.685 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.671+0000 7f906e64a700 1 --2- 172.21.15.47:0/3905217561 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f90680b7840 0x7f9068150190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:08.685 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.671+0000 7f906e64a700 1 --2- 172.21.15.47:0/3905217561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90680aea80 0x7f9068146900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:08.686 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.671+0000 7f906e64a700 1 --2- 172.21.15.47:0/3905217561 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f90480539e0 0x7f9048055ea0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:08.686 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.671+0000 7f906e64a700 1 --2- 172.21.15.47:0/3905217561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9068062dd0 0x7f90680af3b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:08.687 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.671+0000 7f906e64a700 1 -- 172.21.15.47:0/3905217561 >> 172.21.15.47:0/3905217561 conn(0x7f906805f480 msgr2=0x7f90680b53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:08.687 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.671+0000 7f906e64a700 1 -- 172.21.15.47:0/3905217561 shutdown_connections 2022-06-29T15:43:08.687 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.671+0000 7f906e64a700 1 -- 172.21.15.47:0/3905217561 wait complete. 2022-06-29T15:43:08.695 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-06-29T15:43:08.695 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-06-29T15:43:08.805 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.799+0000 7f222abdd700 1 Processor -- start 2022-06-29T15:43:08.806 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.799+0000 7f222abdd700 1 -- start start 2022-06-29T15:43:08.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.799+0000 7f222abdd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f22240b4300 0x7f22240612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:08.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.799+0000 7f222abdd700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2224067610 con 0x7f2224060ee0 2022-06-29T15:43:08.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.799+0000 7f222abdd700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22241467c0 con 0x7f22241471a0 2022-06-29T15:43:08.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.799+0000 7f222abdd700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2224146a60 con 0x7f22240b4300 2022-06-29T15:43:08.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.799+0000 7f2223fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f22240b4300 0x7f22240612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:08.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.799+0000 7f2223fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f22240b4300 0x7f22240612c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47058/0 (socket says 172.21.15.47:47058) 2022-06-29T15:43:08.809 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.799+0000 7f2223fff700 1 -- 172.21.15.47:0/527165467 learned_addr learned my addr 172.21.15.47:0/527165467 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:08.809 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.799+0000 7f2228979700 1 --1- 172.21.15.47:0/527165467 >> v1:172.21.15.47:6789/0 conn(0x7f2224060ee0 0x7f22241430e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:60842/0 (socket says 172.21.15.47:60842) 2022-06-29T15:43:08.809 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.799+0000 7f2223fff700 1 -- 172.21.15.47:0/527165467 >> v1:172.21.15.64:6789/0 conn(0x7f22241471a0 legacy=0x7f222413cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:08.809 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.799+0000 7f2223fff700 1 -- 172.21.15.47:0/527165467 >> v1:172.21.15.47:6789/0 conn(0x7f2224060ee0 legacy=0x7f22241430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:08.810 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.799+0000 7f2223fff700 1 -- 172.21.15.47:0/527165467 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2224146d00 con 0x7f22240b4300 2022-06-29T15:43:08.810 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.799+0000 7f2223fff700 1 --2- 172.21.15.47:0/527165467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22240b4300 0x7f22240612c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f22240adb80 tx=0x7f220c006eb0).ready entity=mon.0 client_cookie=ac0cccc4487e3a6 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:08.810 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.799+0000 7f22237fe700 1 -- 172.21.15.47:0/527165467 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f220c00d660 con 0x7f22240b4300 2022-06-29T15:43:08.810 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.799+0000 7f22237fe700 1 -- 172.21.15.47:0/527165467 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f220c000e90 con 0x7f22240b4300 2022-06-29T15:43:08.810 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.799+0000 7f22237fe700 1 -- 172.21.15.47:0/527165467 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f220c00d9f0 con 0x7f22240b4300 2022-06-29T15:43:08.812 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.803+0000 7f222abdd700 1 -- 172.21.15.47:0/527165467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22240b4300 msgr2=0x7f22240612c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:08.812 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.803+0000 7f222abdd700 1 --2- 172.21.15.47:0/527165467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22240b4300 0x7f22240612c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f22240adb80 tx=0x7f220c006eb0).stop 2022-06-29T15:43:08.813 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.803+0000 7f222abdd700 1 -- 172.21.15.47:0/527165467 shutdown_connections 2022-06-29T15:43:08.813 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.803+0000 7f222abdd700 1 --2- 172.21.15.47:0/527165467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22240b4300 0x7f22240612c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:08.813 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.803+0000 7f222abdd700 1 -- 172.21.15.47:0/527165467 >> 172.21.15.47:0/527165467 conn(0x7f222405c510 msgr2=0x7f222405f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:08.813 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.803+0000 7f222abdd700 1 -- 172.21.15.47:0/527165467 shutdown_connections 2022-06-29T15:43:08.814 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.803+0000 7f222abdd700 1 -- 172.21.15.47:0/527165467 wait complete. 2022-06-29T15:43:08.814 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.803+0000 7f222abdd700 1 Processor -- start 2022-06-29T15:43:08.814 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.803+0000 7f222abdd700 1 -- start start 2022-06-29T15:43:08.814 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.803+0000 7f222abdd700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2224060ee0 0x7f22240b5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:08.815 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.803+0000 7f222abdd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22240b4300 0x7f2224112c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:08.815 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.803+0000 7f222abdd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22241471a0 0x7f2224113120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:08.815 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.803+0000 7f222abdd700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22240b39e0 con 0x7f22241471a0 2022-06-29T15:43:08.815 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.803+0000 7f222abdd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22240b3d40 con 0x7f22240b4300 2022-06-29T15:43:08.815 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.803+0000 7f222abdd700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22240b3740 con 0x7f2224060ee0 2022-06-29T15:43:08.816 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.807+0000 7f222917a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22241471a0 0x7f2224113120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:08.816 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.807+0000 7f222917a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22241471a0 0x7f2224113120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47066/0 (socket says 172.21.15.47:47066) 2022-06-29T15:43:08.816 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.807+0000 7f222917a700 1 -- 172.21.15.47:0/3693158570 learned_addr learned my addr 172.21.15.47:0/3693158570 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:08.816 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.807+0000 7f2228979700 1 --2- 172.21.15.47:0/3693158570 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2224060ee0 0x7f22240b5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:08.817 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.807+0000 7f2223fff700 1 --2- 172.21.15.47:0/3693158570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22240b4300 0x7f2224112c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:08.818 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.807+0000 7f222917a700 1 -- 172.21.15.47:0/3693158570 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2224060ee0 msgr2=0x7f22240b5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:08.819 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.807+0000 7f222917a700 1 --2- 172.21.15.47:0/3693158570 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2224060ee0 0x7f22240b5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:08.819 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.807+0000 7f222917a700 1 -- 172.21.15.47:0/3693158570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22240b4300 msgr2=0x7f2224112c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:08.819 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.807+0000 7f222917a700 1 --2- 172.21.15.47:0/3693158570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22240b4300 0x7f2224112c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:08.819 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.807+0000 7f222917a700 1 -- 172.21.15.47:0/3693158570 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f220c00b1e0 con 0x7f22241471a0 2022-06-29T15:43:08.820 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.807+0000 7f222917a700 1 --2- 172.21.15.47:0/3693158570 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22241471a0 0x7f2224113120 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f22240b9870 tx=0x7f221c0013a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:08.820 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.807+0000 7f2221ffb700 1 -- 172.21.15.47:0/3693158570 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f221c006aa0 con 0x7f22241471a0 2022-06-29T15:43:08.820 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.807+0000 7f2221ffb700 1 -- 172.21.15.47:0/3693158570 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f221c006c00 con 0x7f22241471a0 2022-06-29T15:43:08.820 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.807+0000 7f2221ffb700 1 -- 172.21.15.47:0/3693158570 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f221c007030 con 0x7f22241471a0 2022-06-29T15:43:08.821 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.807+0000 7f222abdd700 1 -- 172.21.15.47:0/3693158570 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22240b24f0 con 0x7f22241471a0 2022-06-29T15:43:08.821 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.807+0000 7f222abdd700 1 -- 172.21.15.47:0/3693158570 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22241152b0 con 0x7f22241471a0 2022-06-29T15:43:08.821 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.807+0000 7f222abdd700 1 -- 172.21.15.47:0/3693158570 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22240b2680 con 0x7f22241471a0 2022-06-29T15:43:08.821 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.807+0000 7f2221ffb700 1 -- 172.21.15.47:0/3693158570 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f221c010030 con 0x7f22241471a0 2022-06-29T15:43:08.822 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.811+0000 7f2221ffb700 1 --2- 172.21.15.47:0/3693158570 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2208053b00 0x7f2208055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:08.822 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.811+0000 7f2221ffb700 1 -- 172.21.15.47:0/3693158570 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f221c06bbb0 con 0x7f22241471a0 2022-06-29T15:43:08.822 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.811+0000 7f2228979700 1 --2- 172.21.15.47:0/3693158570 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2208053b00 0x7f2208055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:08.822 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.811+0000 7f2228979700 1 --2- 172.21.15.47:0/3693158570 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2208053b00 0x7f2208055fc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f221400d000 tx=0x7f2214007b80).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:08.825 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.819+0000 7f2221ffb700 1 -- 172.21.15.47:0/3693158570 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f221c042ba0 con 0x7f22241471a0 2022-06-29T15:43:08.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:08.939+0000 7f222abdd700 1 -- 172.21.15.47:0/3693158570 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f22240573d0 con 0x7f22241471a0 2022-06-29T15:43:09.234 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.227+0000 7f2221ffb700 1 -- 172.21.15.47:0/3693158570 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v33) v1 ==== 133+0+0 (secure 0 0 0) 0x7f221c0461c0 con 0x7f22241471a0 2022-06-29T15:43:09.237 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.231+0000 7f222abdd700 1 -- 172.21.15.47:0/3693158570 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2208053b00 msgr2=0x7f2208055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:09.238 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.231+0000 7f222abdd700 1 --2- 172.21.15.47:0/3693158570 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2208053b00 0x7f2208055fc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f221400d000 tx=0x7f2214007b80).stop 2022-06-29T15:43:09.238 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.231+0000 7f222abdd700 1 -- 172.21.15.47:0/3693158570 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22241471a0 msgr2=0x7f2224113120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:09.238 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.231+0000 7f222abdd700 1 --2- 172.21.15.47:0/3693158570 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22241471a0 0x7f2224113120 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f22240b9870 tx=0x7f221c0013a0).stop 2022-06-29T15:43:09.239 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.231+0000 7f222abdd700 1 -- 172.21.15.47:0/3693158570 shutdown_connections 2022-06-29T15:43:09.239 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.231+0000 7f222abdd700 1 --2- 172.21.15.47:0/3693158570 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22241471a0 0x7f2224113120 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:09.240 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.231+0000 7f222abdd700 1 --2- 172.21.15.47:0/3693158570 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2208053b00 0x7f2208055fc0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:09.240 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.231+0000 7f222abdd700 1 --2- 172.21.15.47:0/3693158570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22240b4300 0x7f2224112c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:09.240 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.231+0000 7f222abdd700 1 --2- 172.21.15.47:0/3693158570 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2224060ee0 0x7f22240b5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:09.240 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.231+0000 7f222abdd700 1 -- 172.21.15.47:0/3693158570 >> 172.21.15.47:0/3693158570 conn(0x7f222405c510 msgr2=0x7f222413f710 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:09.241 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.231+0000 7f222abdd700 1 -- 172.21.15.47:0/3693158570 shutdown_connections 2022-06-29T15:43:09.241 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.231+0000 7f222abdd700 1 -- 172.21.15.47:0/3693158570 wait complete. 2022-06-29T15:43:09.241 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' created 2022-06-29T15:43:09.253 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-06-29T15:43:09.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.359+0000 7f1132968700 1 Processor -- start 2022-06-29T15:43:09.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.359+0000 7f1132968700 1 -- start start 2022-06-29T15:43:09.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.359+0000 7f1132968700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f112c0609e0 0x7f112c0b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:09.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.359+0000 7f1132968700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f112c147160 0x7f112c0aa130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:09.372 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.359+0000 7f1132968700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f112c0b1380 con 0x7f112c0b58d0 2022-06-29T15:43:09.372 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.359+0000 7f1132968700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f112c1467c0 con 0x7f112c0609e0 2022-06-29T15:43:09.372 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.359+0000 7f1132968700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f112c0b15b0 con 0x7f112c147160 2022-06-29T15:43:09.372 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.363+0000 7f1132167700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f112c147160 0x7f112c0aa130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:09.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.363+0000 7f1131165700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f112c0609e0 0x7f112c0b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:09.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.363+0000 7f1132167700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f112c147160 0x7f112c0aa130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:44174/0 (socket says 172.21.15.47:44174) 2022-06-29T15:43:09.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.363+0000 7f1131165700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f112c0609e0 0x7f112c0b5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:52776/0 (socket says 172.21.15.47:52776) 2022-06-29T15:43:09.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.363+0000 7f1132167700 1 -- 172.21.15.47:0/642976528 learned_addr learned my addr 172.21.15.47:0/642976528 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:09.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.363+0000 7f1131966700 1 --1- 172.21.15.47:0/642976528 >> v1:172.21.15.64:6789/0 conn(0x7f112c0b58d0 0x7f112c1431f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:40188/0 (socket says 172.21.15.47:40188) 2022-06-29T15:43:09.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.363+0000 7f1131165700 1 -- 172.21.15.47:0/642976528 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f112c147160 msgr2=0x7f112c0aa130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:43:09.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.363+0000 7f1131165700 1 --2- 172.21.15.47:0/642976528 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f112c147160 0x7f112c0aa130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:09.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.363+0000 7f1131165700 1 -- 172.21.15.47:0/642976528 >> v1:172.21.15.64:6789/0 conn(0x7f112c0b58d0 legacy=0x7f112c1431f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:09.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.363+0000 7f1131165700 1 -- 172.21.15.47:0/642976528 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f112c146a20 con 0x7f112c0609e0 2022-06-29T15:43:09.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.363+0000 7f1131165700 1 --2- 172.21.15.47:0/642976528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f112c0609e0 0x7f112c0b5cb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f112c0b1850 tx=0x7f1128006df0).ready entity=mon.1 client_cookie=fa5469c10e8827b3 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:09.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.363+0000 7f1130964700 1 -- 172.21.15.47:0/642976528 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f112800d570 con 0x7f112c0609e0 2022-06-29T15:43:09.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.363+0000 7f1130964700 1 -- 172.21.15.47:0/642976528 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1128000e90 con 0x7f112c0609e0 2022-06-29T15:43:09.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.363+0000 7f1130964700 1 -- 172.21.15.47:0/642976528 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f112800d8c0 con 0x7f112c0609e0 2022-06-29T15:43:09.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.363+0000 7f1132968700 1 -- 172.21.15.47:0/642976528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f112c0609e0 msgr2=0x7f112c0b5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:09.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.363+0000 7f1132968700 1 --2- 172.21.15.47:0/642976528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f112c0609e0 0x7f112c0b5cb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f112c0b1850 tx=0x7f1128006df0).stop 2022-06-29T15:43:09.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.363+0000 7f1132968700 1 -- 172.21.15.47:0/642976528 shutdown_connections 2022-06-29T15:43:09.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.363+0000 7f1132968700 1 --2- 172.21.15.47:0/642976528 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f112c147160 0x7f112c0aa130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:09.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.363+0000 7f1132968700 1 --2- 172.21.15.47:0/642976528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f112c0609e0 0x7f112c0b5cb0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:09.379 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.363+0000 7f1132968700 1 -- 172.21.15.47:0/642976528 >> 172.21.15.47:0/642976528 conn(0x7f112c062560 msgr2=0x7f112c062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:09.379 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.363+0000 7f1132968700 1 -- 172.21.15.47:0/642976528 shutdown_connections 2022-06-29T15:43:09.379 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.363+0000 7f1132968700 1 -- 172.21.15.47:0/642976528 wait complete. 2022-06-29T15:43:09.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.367+0000 7f1132968700 1 Processor -- start 2022-06-29T15:43:09.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.367+0000 7f1132968700 1 -- start start 2022-06-29T15:43:09.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.367+0000 7f1132968700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f112c0609e0 0x7f112c066e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:09.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.367+0000 7f1132968700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f112c0b58d0 0x7f112c142cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:09.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.367+0000 7f1132968700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f112c147160 0x7f112c1431a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:09.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.367+0000 7f1132968700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112c0af470 con 0x7f112c0b58d0 2022-06-29T15:43:09.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.367+0000 7f1132968700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112c14a670 con 0x7f112c0609e0 2022-06-29T15:43:09.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.367+0000 7f1132968700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f112c14a3d0 con 0x7f112c147160 2022-06-29T15:43:09.385 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.367+0000 7f1131165700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f112c0b58d0 0x7f112c142cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:09.385 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.367+0000 7f1131165700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f112c0b58d0 0x7f112c142cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47076/0 (socket says 172.21.15.47:47076) 2022-06-29T15:43:09.385 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.367+0000 7f1131966700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f112c0609e0 0x7f112c066e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:09.385 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.367+0000 7f1131165700 1 -- 172.21.15.47:0/2200761208 learned_addr learned my addr 172.21.15.47:0/2200761208 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:09.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.367+0000 7f1132167700 1 --2- 172.21.15.47:0/2200761208 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f112c147160 0x7f112c1431a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:09.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.367+0000 7f1131165700 1 -- 172.21.15.47:0/2200761208 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f112c147160 msgr2=0x7f112c1431a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:09.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.367+0000 7f1131165700 1 --2- 172.21.15.47:0/2200761208 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f112c147160 0x7f112c1431a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:09.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.367+0000 7f1131165700 1 -- 172.21.15.47:0/2200761208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f112c0609e0 msgr2=0x7f112c066e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:09.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.367+0000 7f1131165700 1 --2- 172.21.15.47:0/2200761208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f112c0609e0 0x7f112c066e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:09.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.367+0000 7f1131165700 1 -- 172.21.15.47:0/2200761208 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f112800b180 con 0x7f112c0b58d0 2022-06-29T15:43:09.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.367+0000 7f1131165700 1 --2- 172.21.15.47:0/2200761208 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f112c0b58d0 0x7f112c142cb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7f112c0b1850 tx=0x7f1128014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:09.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.367+0000 7f111affd700 1 -- 172.21.15.47:0/2200761208 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f112800dea0 con 0x7f112c0b58d0 2022-06-29T15:43:09.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.367+0000 7f111affd700 1 -- 172.21.15.47:0/2200761208 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1128013de0 con 0x7f112c0b58d0 2022-06-29T15:43:09.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.367+0000 7f1132968700 1 -- 172.21.15.47:0/2200761208 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f112c0aed30 con 0x7f112c0b58d0 2022-06-29T15:43:09.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.367+0000 7f111affd700 1 -- 172.21.15.47:0/2200761208 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f112801e5f0 con 0x7f112c0b58d0 2022-06-29T15:43:09.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.367+0000 7f1132968700 1 -- 172.21.15.47:0/2200761208 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f112c066840 con 0x7f112c0b58d0 2022-06-29T15:43:09.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.371+0000 7f1132968700 1 -- 172.21.15.47:0/2200761208 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f112c067360 con 0x7f112c0b58d0 2022-06-29T15:43:09.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.371+0000 7f111affd700 1 -- 172.21.15.47:0/2200761208 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f112801c070 con 0x7f112c0b58d0 2022-06-29T15:43:09.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.371+0000 7f111affd700 1 --2- 172.21.15.47:0/2200761208 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f110c053a70 0x7f110c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:09.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.371+0000 7f111affd700 1 -- 172.21.15.47:0/2200761208 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5954+0+0 (secure 0 0 0) 0x7f112806c920 con 0x7f112c0b58d0 2022-06-29T15:43:09.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.371+0000 7f1131966700 1 --2- 172.21.15.47:0/2200761208 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f110c053a70 0x7f110c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:09.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.371+0000 7f1131966700 1 --2- 172.21.15.47:0/2200761208 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f110c053a70 0x7f110c055f30 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f111c00d000 tx=0x7f111c007b80).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:09.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.379+0000 7f111affd700 1 -- 172.21.15.47:0/2200761208 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1128043930 con 0x7f112c0b58d0 2022-06-29T15:43:09.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:09.503+0000 7f1132968700 1 -- 172.21.15.47:0/2200761208 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f112c067360 con 0x7f112c0b58d0 2022-06-29T15:43:10.239 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.231+0000 7f111affd700 1 -- 172.21.15.47:0/2200761208 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v34) v1 ==== 125+0+0 (secure 0 0 0) 0x7f112c067360 con 0x7f112c0b58d0 2022-06-29T15:43:10.246 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.239+0000 7f1132968700 1 -- 172.21.15.47:0/2200761208 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f110c053a70 msgr2=0x7f110c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:10.246 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.239+0000 7f1132968700 1 --2- 172.21.15.47:0/2200761208 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f110c053a70 0x7f110c055f30 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f111c00d000 tx=0x7f111c007b80).stop 2022-06-29T15:43:10.246 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.239+0000 7f1132968700 1 -- 172.21.15.47:0/2200761208 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f112c0b58d0 msgr2=0x7f112c142cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:10.247 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.239+0000 7f1132968700 1 --2- 172.21.15.47:0/2200761208 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f112c0b58d0 0x7f112c142cb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7f112c0b1850 tx=0x7f1128014000).stop 2022-06-29T15:43:10.247 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.239+0000 7f1132968700 1 -- 172.21.15.47:0/2200761208 shutdown_connections 2022-06-29T15:43:10.247 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.239+0000 7f1132968700 1 --2- 172.21.15.47:0/2200761208 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f112c147160 0x7f112c1431a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:10.247 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.239+0000 7f1132968700 1 --2- 172.21.15.47:0/2200761208 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f112c0b58d0 0x7f112c142cb0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:10.248 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.239+0000 7f1132968700 1 --2- 172.21.15.47:0/2200761208 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f110c053a70 0x7f110c055f30 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:10.248 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.239+0000 7f1132968700 1 --2- 172.21.15.47:0/2200761208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f112c0609e0 0x7f112c066e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:10.248 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.239+0000 7f1132968700 1 -- 172.21.15.47:0/2200761208 >> 172.21.15.47:0/2200761208 conn(0x7f112c062560 msgr2=0x7f112c0b1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:10.248 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.239+0000 7f1132968700 1 -- 172.21.15.47:0/2200761208 shutdown_connections 2022-06-29T15:43:10.249 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.239+0000 7f1132968700 1 -- 172.21.15.47:0/2200761208 wait complete. 2022-06-29T15:43:10.252 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' created 2022-06-29T15:43:10.259 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-06-29T15:43:10.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.359+0000 7fce57fff700 1 Processor -- start 2022-06-29T15:43:10.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.359+0000 7fce57fff700 1 -- start start 2022-06-29T15:43:10.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.363+0000 7fce57fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fce58143140 0x7fce5809a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:10.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.363+0000 7fce57fff700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce5805f880 con 0x7fce5809a1d0 2022-06-29T15:43:10.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.363+0000 7fce57fff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce5809ae90 con 0x7fce5805f4a0 2022-06-29T15:43:10.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.363+0000 7fce57fff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce5809ab50 con 0x7fce58143140 2022-06-29T15:43:10.370 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.363+0000 7fce4ffff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fce5809a1d0 0x7fce58142a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:60872/0 (socket says 172.21.15.47:60872) 2022-06-29T15:43:10.370 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.363+0000 7fce4ffff700 1 -- 172.21.15.47:0/2813449011 learned_addr learned my addr 172.21.15.47:0/2813449011 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:43:10.370 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.363+0000 7fce577fe700 1 --2- 172.21.15.47:0/2813449011 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fce58143140 0x7fce5809a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:10.372 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.363+0000 7fce567fc700 1 -- 172.21.15.47:0/2813449011 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3663979834 0 0) 0x7fce5805f880 con 0x7fce5809a1d0 2022-06-29T15:43:10.372 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.363+0000 7fce567fc700 1 -- 172.21.15.47:0/2813449011 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce38003260 con 0x7fce5809a1d0 2022-06-29T15:43:10.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.363+0000 7fce567fc700 1 -- 172.21.15.47:0/2813449011 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4216377995 0 0) 0x7fce5809ae90 con 0x7fce5805f4a0 2022-06-29T15:43:10.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.363+0000 7fce567fc700 1 -- 172.21.15.47:0/2813449011 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce5805f880 con 0x7fce5805f4a0 2022-06-29T15:43:10.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.363+0000 7fce577fe700 1 -- 172.21.15.47:0/2813449011 >> v1:172.21.15.64:6789/0 conn(0x7fce5805f4a0 legacy=0x7fce580b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:10.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.363+0000 7fce577fe700 1 -- 172.21.15.47:0/2813449011 >> v1:172.21.15.47:6789/0 conn(0x7fce5809a1d0 legacy=0x7fce58142a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:10.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.363+0000 7fce577fe700 1 -- 172.21.15.47:0/2813449011 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce5809b130 con 0x7fce58143140 2022-06-29T15:43:10.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.363+0000 7fce577fe700 1 --2- 172.21.15.47:0/2813449011 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fce58143140 0x7fce5809a660 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fce500032f0 tx=0x7fce500091d0).ready entity=mon.2 client_cookie=2420662485a2dc68 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:10.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.363+0000 7fce567fc700 1 -- 172.21.15.47:0/2813449011 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce5000f8a0 con 0x7fce58143140 2022-06-29T15:43:10.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.363+0000 7fce567fc700 1 -- 172.21.15.47:0/2813449011 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce5000fa00 con 0x7fce58143140 2022-06-29T15:43:10.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.363+0000 7fce567fc700 1 -- 172.21.15.47:0/2813449011 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce5000fd60 con 0x7fce58143140 2022-06-29T15:43:10.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.363+0000 7fce57fff700 1 -- 172.21.15.47:0/2813449011 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fce58143140 msgr2=0x7fce5809a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:10.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.363+0000 7fce57fff700 1 --2- 172.21.15.47:0/2813449011 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fce58143140 0x7fce5809a660 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fce500032f0 tx=0x7fce500091d0).stop 2022-06-29T15:43:10.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.363+0000 7fce57fff700 1 -- 172.21.15.47:0/2813449011 shutdown_connections 2022-06-29T15:43:10.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.363+0000 7fce57fff700 1 --2- 172.21.15.47:0/2813449011 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fce58143140 0x7fce5809a660 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:10.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.363+0000 7fce57fff700 1 -- 172.21.15.47:0/2813449011 >> 172.21.15.47:0/2813449011 conn(0x7fce58062df0 msgr2=0x7fce580631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:10.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.363+0000 7fce57fff700 1 -- 172.21.15.47:0/2813449011 shutdown_connections 2022-06-29T15:43:10.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.363+0000 7fce57fff700 1 -- 172.21.15.47:0/2813449011 wait complete. 2022-06-29T15:43:10.379 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.367+0000 7fce57fff700 1 Processor -- start 2022-06-29T15:43:10.380 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.367+0000 7fce57fff700 1 -- start start 2022-06-29T15:43:10.380 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.367+0000 7fce57fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce5805f4a0 0x7fce580b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:10.380 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.367+0000 7fce57fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fce5809a1d0 0x7fce58066120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:10.380 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.367+0000 7fce57fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce58143140 0x7fce5813f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:10.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.367+0000 7fce57fff700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce58065a50 con 0x7fce58143140 2022-06-29T15:43:10.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.367+0000 7fce57fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce58064590 con 0x7fce5805f4a0 2022-06-29T15:43:10.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.367+0000 7fce57fff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce58063e50 con 0x7fce5809a1d0 2022-06-29T15:43:10.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.367+0000 7fce577fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce58143140 0x7fce5813f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:10.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.367+0000 7fce577fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce58143140 0x7fce5813f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47094/0 (socket says 172.21.15.47:47094) 2022-06-29T15:43:10.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.367+0000 7fce577fe700 1 -- 172.21.15.47:0/2462552426 learned_addr learned my addr 172.21.15.47:0/2462552426 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:10.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.371+0000 7fce56ffd700 1 --2- 172.21.15.47:0/2462552426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce5805f4a0 0x7fce580b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:10.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.371+0000 7fce4ffff700 1 --2- 172.21.15.47:0/2462552426 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fce5809a1d0 0x7fce58066120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:10.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.371+0000 7fce577fe700 1 -- 172.21.15.47:0/2462552426 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fce5809a1d0 msgr2=0x7fce58066120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:10.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.371+0000 7fce577fe700 1 --2- 172.21.15.47:0/2462552426 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fce5809a1d0 0x7fce58066120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:10.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.371+0000 7fce577fe700 1 -- 172.21.15.47:0/2462552426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce5805f4a0 msgr2=0x7fce580b50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:10.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.371+0000 7fce577fe700 1 --2- 172.21.15.47:0/2462552426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce5805f4a0 0x7fce580b50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:10.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.371+0000 7fce577fe700 1 -- 172.21.15.47:0/2462552426 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce50009470 con 0x7fce58143140 2022-06-29T15:43:10.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.371+0000 7fce577fe700 1 --2- 172.21.15.47:0/2462552426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce58143140 0x7fce5813f7d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7fce500032f0 tx=0x7fce50018000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:10.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.371+0000 7fce54ff9700 1 -- 172.21.15.47:0/2462552426 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce5000f8a0 con 0x7fce58143140 2022-06-29T15:43:10.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.371+0000 7fce54ff9700 1 -- 172.21.15.47:0/2462552426 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce5001e040 con 0x7fce58143140 2022-06-29T15:43:10.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.371+0000 7fce57fff700 1 -- 172.21.15.47:0/2462552426 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce580640f0 con 0x7fce58143140 2022-06-29T15:43:10.385 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.371+0000 7fce54ff9700 1 -- 172.21.15.47:0/2462552426 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce50023630 con 0x7fce58143140 2022-06-29T15:43:10.385 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.371+0000 7fce57fff700 1 -- 172.21.15.47:0/2462552426 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce580afe80 con 0x7fce58143140 2022-06-29T15:43:10.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.371+0000 7fce54ff9700 1 -- 172.21.15.47:0/2462552426 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fce50029020 con 0x7fce58143140 2022-06-29T15:43:10.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.371+0000 7fce57fff700 1 -- 172.21.15.47:0/2462552426 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce5813e8d0 con 0x7fce58143140 2022-06-29T15:43:10.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.375+0000 7fce54ff9700 1 --2- 172.21.15.47:0/2462552426 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fce3c053980 0x7fce3c055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:10.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.375+0000 7fce54ff9700 1 -- 172.21.15.47:0/2462552426 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6304+0+0 (secure 0 0 0) 0x7fce5006f680 con 0x7fce58143140 2022-06-29T15:43:10.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.375+0000 7fce56ffd700 1 --2- 172.21.15.47:0/2462552426 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fce3c053980 0x7fce3c055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:10.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.375+0000 7fce56ffd700 1 --2- 172.21.15.47:0/2462552426 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fce3c053980 0x7fce3c055e40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fce4800d000 tx=0x7fce48007a20).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:10.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.383+0000 7fce54ff9700 1 -- 172.21.15.47:0/2462552426 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fce500464c0 con 0x7fce58143140 2022-06-29T15:43:10.550 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.543+0000 7fce57fff700 1 -- 172.21.15.47:0/2462552426 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fce580573d0 con 0x7fce58143140 2022-06-29T15:43:10.557 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.551+0000 7fce54ff9700 1 -- 172.21.15.47:0/2462552426 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 5 and data pool 6 v11) v1 ==== 187+0+0 (secure 0 0 0) 0x7fce50049ae0 con 0x7fce58143140 2022-06-29T15:43:10.565 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.555+0000 7fce57fff700 1 -- 172.21.15.47:0/2462552426 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fce3c053980 msgr2=0x7fce3c055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:10.565 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.559+0000 7fce57fff700 1 --2- 172.21.15.47:0/2462552426 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fce3c053980 0x7fce3c055e40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fce4800d000 tx=0x7fce48007a20).stop 2022-06-29T15:43:10.566 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.559+0000 7fce57fff700 1 -- 172.21.15.47:0/2462552426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce58143140 msgr2=0x7fce5813f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:10.566 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.559+0000 7fce57fff700 1 --2- 172.21.15.47:0/2462552426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce58143140 0x7fce5813f7d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7fce500032f0 tx=0x7fce50018000).stop 2022-06-29T15:43:10.566 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.559+0000 7fce57fff700 1 -- 172.21.15.47:0/2462552426 shutdown_connections 2022-06-29T15:43:10.566 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.559+0000 7fce57fff700 1 --2- 172.21.15.47:0/2462552426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce58143140 0x7fce5813f7d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:10.567 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.559+0000 7fce57fff700 1 --2- 172.21.15.47:0/2462552426 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fce5809a1d0 0x7fce58066120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:10.567 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.559+0000 7fce57fff700 1 --2- 172.21.15.47:0/2462552426 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fce3c053980 0x7fce3c055e40 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:10.567 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.559+0000 7fce57fff700 1 --2- 172.21.15.47:0/2462552426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce5805f4a0 0x7fce580b50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:10.567 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.559+0000 7fce57fff700 1 -- 172.21.15.47:0/2462552426 >> 172.21.15.47:0/2462552426 conn(0x7fce58062df0 msgr2=0x7fce580b5b20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:10.568 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.559+0000 7fce57fff700 1 -- 172.21.15.47:0/2462552426 shutdown_connections 2022-06-29T15:43:10.568 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.559+0000 7fce57fff700 1 -- 172.21.15.47:0/2462552426 wait complete. 2022-06-29T15:43:10.569 INFO:teuthology.orchestra.run.smithi047.stderr:new fs with metadata pool 5 and data pool 6 2022-06-29T15:43:10.581 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:43:10.676 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.667+0000 7f8abeb54700 1 Processor -- start 2022-06-29T15:43:10.676 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.667+0000 7f8abeb54700 1 -- start start 2022-06-29T15:43:10.679 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.671+0000 7f8abeb54700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8ab805fa30 0x7f8ab80b1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:10.679 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.671+0000 7f8abeb54700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8ab8060b90 0x7f8ab813f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:10.679 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.671+0000 7f8abeb54700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ab8060fd0 con 0x7f8ab80b2770 2022-06-29T15:43:10.679 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.671+0000 7f8abeb54700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ab805fe10 con 0x7f8ab805fa30 2022-06-29T15:43:10.679 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.671+0000 7f8abeb54700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab80600e0 con 0x7f8ab8060b90 2022-06-29T15:43:10.680 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.671+0000 7f8abe353700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f8ab80b2770 0x7f8ab80ae170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:60900/0 (socket says 172.21.15.47:60900) 2022-06-29T15:43:10.680 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.671+0000 7f8abe353700 1 -- 172.21.15.47:0/1667298311 learned_addr learned my addr 172.21.15.47:0/1667298311 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:43:10.680 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.671+0000 7f8abd351700 1 --2- 172.21.15.47:0/1667298311 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8ab8060b90 0x7f8ab813f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:10.680 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.671+0000 7f8abdb52700 1 --2- 172.21.15.47:0/1667298311 >> v2:172.21.15.64:3300/0 conn(0x7f8ab805fa30 0x7f8ab80b1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:10.681 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.671+0000 7f8abcb50700 1 -- 172.21.15.47:0/1667298311 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3782035477 0 0) 0x7f8ab0001490 con 0x7f8ab80b2770 2022-06-29T15:43:10.681 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.671+0000 7f8abcb50700 1 -- 172.21.15.47:0/1667298311 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8aa0003260 con 0x7f8ab80b2770 2022-06-29T15:43:10.681 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.671+0000 7f8abcb50700 1 -- 172.21.15.47:0/1667298311 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3435453536 0 0) 0x7f8aa0003260 con 0x7f8ab80b2770 2022-06-29T15:43:10.681 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.671+0000 7f8abd351700 1 -- 172.21.15.47:0/1667298311 >> v2:172.21.15.64:3300/0 conn(0x7f8ab805fa30 msgr2=0x7f8ab80b1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:43:10.681 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.671+0000 7f8abd351700 1 --2- 172.21.15.47:0/1667298311 >> v2:172.21.15.64:3300/0 conn(0x7f8ab805fa30 0x7f8ab80b1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:10.682 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.671+0000 7f8abd351700 1 -- 172.21.15.47:0/1667298311 >> v1:172.21.15.47:6789/0 conn(0x7f8ab80b2770 legacy=0x7f8ab80ae170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:10.682 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.671+0000 7f8abd351700 1 -- 172.21.15.47:0/1667298311 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab8060380 con 0x7f8ab8060b90 2022-06-29T15:43:10.682 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.671+0000 7f8abd351700 1 --2- 172.21.15.47:0/1667298311 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8ab8060b90 0x7f8ab813f470 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f8ab80b2650 tx=0x7f8aa8006df0).ready entity=mon.2 client_cookie=a08c42fcaed5445c server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:10.682 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.671+0000 7f8abcb50700 1 -- 172.21.15.47:0/1667298311 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8aa8014070 con 0x7f8ab8060b90 2022-06-29T15:43:10.683 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.671+0000 7f8abcb50700 1 -- 172.21.15.47:0/1667298311 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa8000e90 con 0x7f8ab8060b90 2022-06-29T15:43:10.683 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.671+0000 7f8abcb50700 1 -- 172.21.15.47:0/1667298311 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ab0001b70 con 0x7f8ab8060b90 2022-06-29T15:43:10.687 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.671+0000 7f8abeb54700 1 -- 172.21.15.47:0/1667298311 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8ab8060b90 msgr2=0x7f8ab813f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:10.687 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.671+0000 7f8abeb54700 1 --2- 172.21.15.47:0/1667298311 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8ab8060b90 0x7f8ab813f470 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f8ab80b2650 tx=0x7f8aa8006df0).stop 2022-06-29T15:43:10.687 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.671+0000 7f8abeb54700 1 -- 172.21.15.47:0/1667298311 shutdown_connections 2022-06-29T15:43:10.687 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.671+0000 7f8abeb54700 1 --2- 172.21.15.47:0/1667298311 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8ab8060b90 0x7f8ab813f470 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:10.688 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.671+0000 7f8abeb54700 1 --2- 172.21.15.47:0/1667298311 >> v2:172.21.15.64:3300/0 conn(0x7f8ab805fa30 0x7f8ab80b1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:10.688 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.671+0000 7f8abeb54700 1 -- 172.21.15.47:0/1667298311 >> 172.21.15.47:0/1667298311 conn(0x7f8ab8062df0 msgr2=0x7f8ab80631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:10.688 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.671+0000 7f8abeb54700 1 -- 172.21.15.47:0/1667298311 shutdown_connections 2022-06-29T15:43:10.688 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.671+0000 7f8abeb54700 1 -- 172.21.15.47:0/1667298311 wait complete. 2022-06-29T15:43:10.689 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.675+0000 7f8abeb54700 1 Processor -- start 2022-06-29T15:43:10.689 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.675+0000 7f8abeb54700 1 -- start start 2022-06-29T15:43:10.689 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.675+0000 7f8abeb54700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ab805fa30 0x7f8ab8097710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:10.689 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.675+0000 7f8abeb54700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8ab8060b90 0x7f8ab8142320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:10.689 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.675+0000 7f8abeb54700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ab80b2770 0x7f8ab8065560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:10.690 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.675+0000 7f8abeb54700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab8143760 con 0x7f8ab805fa30 2022-06-29T15:43:10.690 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.675+0000 7f8abeb54700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab80961c0 con 0x7f8ab80b2770 2022-06-29T15:43:10.690 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.675+0000 7f8abeb54700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab8095720 con 0x7f8ab8060b90 2022-06-29T15:43:10.690 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.675+0000 7f8abdb52700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ab805fa30 0x7f8ab8097710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:10.691 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.675+0000 7f8abdb52700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ab805fa30 0x7f8ab8097710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47116/0 (socket says 172.21.15.47:47116) 2022-06-29T15:43:10.691 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.675+0000 7f8abdb52700 1 -- 172.21.15.47:0/406201822 learned_addr learned my addr 172.21.15.47:0/406201822 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:10.691 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.675+0000 7f8abd351700 1 --2- 172.21.15.47:0/406201822 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8ab8060b90 0x7f8ab8142320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:10.691 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.679+0000 7f8abe353700 1 --2- 172.21.15.47:0/406201822 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ab80b2770 0x7f8ab8065560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:10.691 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.679+0000 7f8abdb52700 1 -- 172.21.15.47:0/406201822 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8ab8060b90 msgr2=0x7f8ab8142320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:10.692 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.679+0000 7f8abdb52700 1 --2- 172.21.15.47:0/406201822 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8ab8060b90 0x7f8ab8142320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:10.692 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.679+0000 7f8abdb52700 1 -- 172.21.15.47:0/406201822 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ab80b2770 msgr2=0x7f8ab8065560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:10.692 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.679+0000 7f8abdb52700 1 --2- 172.21.15.47:0/406201822 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ab80b2770 0x7f8ab8065560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:10.693 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.679+0000 7f8abdb52700 1 -- 172.21.15.47:0/406201822 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa800b180 con 0x7f8ab805fa30 2022-06-29T15:43:10.693 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.679+0000 7f8abd351700 1 --2- 172.21.15.47:0/406201822 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8ab8060b90 0x7f8ab8142320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:43:10.693 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.679+0000 7f8abdb52700 1 --2- 172.21.15.47:0/406201822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ab805fa30 0x7f8ab8097710 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f8ab400c000 tx=0x7f8ab4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:10.693 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.679+0000 7f8aa6ffd700 1 -- 172.21.15.47:0/406201822 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ab4010780 con 0x7f8ab805fa30 2022-06-29T15:43:10.694 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.679+0000 7f8aa6ffd700 1 -- 172.21.15.47:0/406201822 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab40108e0 con 0x7f8ab805fa30 2022-06-29T15:43:10.694 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.679+0000 7f8aa6ffd700 1 -- 172.21.15.47:0/406201822 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ab4010c40 con 0x7f8ab805fa30 2022-06-29T15:43:10.694 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.679+0000 7f8abeb54700 1 -- 172.21.15.47:0/406201822 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ab8095480 con 0x7f8ab805fa30 2022-06-29T15:43:10.694 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.679+0000 7f8abeb54700 1 -- 172.21.15.47:0/406201822 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ab8142810 con 0x7f8ab805fa30 2022-06-29T15:43:10.696 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.679+0000 7f8aa6ffd700 1 -- 172.21.15.47:0/406201822 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8ab401c460 con 0x7f8ab805fa30 2022-06-29T15:43:10.696 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.679+0000 7f8abeb54700 1 -- 172.21.15.47:0/406201822 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ab80967f0 con 0x7f8ab805fa30 2022-06-29T15:43:10.696 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.679+0000 7f8aa6ffd700 1 --2- 172.21.15.47:0/406201822 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8a98053950 0x7f8a98055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:10.696 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.679+0000 7f8aa6ffd700 1 -- 172.21.15.47:0/406201822 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f8ab406c380 con 0x7f8ab805fa30 2022-06-29T15:43:10.697 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.683+0000 7f8abd351700 1 --2- 172.21.15.47:0/406201822 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8a98053950 0x7f8a98055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:10.697 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.683+0000 7f8abd351700 1 --2- 172.21.15.47:0/406201822 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8a98053950 0x7f8a98055e10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f8aa801b000 tx=0x7f8aa8016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:10.697 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.687+0000 7f8aa6ffd700 1 -- 172.21.15.47:0/406201822 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8ab80967f0 con 0x7f8ab805fa30 2022-06-29T15:43:10.824 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.815+0000 7f8abeb54700 1 -- 172.21.15.47:0/406201822 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8ab80967f0 con 0x7f8ab805fa30 2022-06-29T15:43:10.826 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.819+0000 7f8aa6ffd700 1 -- 172.21.15.47:0/406201822 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16284 (secure 0 0 0) 0x7f8ab80967f0 con 0x7f8ab805fa30 2022-06-29T15:43:10.827 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:43:10.827 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":35,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:43:10.546635+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:43:08.995574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2022-06-29T15:43:09.557935+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:43:10.828 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.819+0000 7f8abeb54700 1 -- 172.21.15.47:0/406201822 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8a98053950 msgr2=0x7f8a98055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:10.829 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.819+0000 7f8abeb54700 1 --2- 172.21.15.47:0/406201822 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8a98053950 0x7f8a98055e10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f8aa801b000 tx=0x7f8aa8016000).stop 2022-06-29T15:43:10.829 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.819+0000 7f8abeb54700 1 -- 172.21.15.47:0/406201822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ab805fa30 msgr2=0x7f8ab8097710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:10.829 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.819+0000 7f8abeb54700 1 --2- 172.21.15.47:0/406201822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ab805fa30 0x7f8ab8097710 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f8ab400c000 tx=0x7f8ab4006df0).stop 2022-06-29T15:43:10.829 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.819+0000 7f8abeb54700 1 -- 172.21.15.47:0/406201822 shutdown_connections 2022-06-29T15:43:10.830 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.819+0000 7f8abeb54700 1 --2- 172.21.15.47:0/406201822 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8a98053950 0x7f8a98055e10 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:10.830 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.819+0000 7f8abeb54700 1 --2- 172.21.15.47:0/406201822 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ab80b2770 0x7f8ab8065560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:10.830 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.819+0000 7f8abeb54700 1 --2- 172.21.15.47:0/406201822 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8ab8060b90 0x7f8ab8142320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:10.830 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.823+0000 7f8abeb54700 1 --2- 172.21.15.47:0/406201822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ab805fa30 0x7f8ab8097710 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:10.831 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.823+0000 7f8abeb54700 1 -- 172.21.15.47:0/406201822 >> 172.21.15.47:0/406201822 conn(0x7f8ab8062df0 msgr2=0x7f8ab80af220 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:10.831 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.823+0000 7f8abeb54700 1 -- 172.21.15.47:0/406201822 shutdown_connections 2022-06-29T15:43:10.831 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.823+0000 7f8abeb54700 1 -- 172.21.15.47:0/406201822 wait complete. 2022-06-29T15:43:10.839 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:43:10.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.935+0000 7f1301aee700 1 Processor -- start 2022-06-29T15:43:10.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.935+0000 7f1301aee700 1 -- start start 2022-06-29T15:43:10.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.939+0000 7f1301aee700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f12fc0aa9b0 0x7f12fc0ac180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:10.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.939+0000 7f1301aee700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f12fc0ac670 0x7f12fc14b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:10.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.939+0000 7f1301aee700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12fc0ae3e0 con 0x7f12fc0b2740 2022-06-29T15:43:10.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.939+0000 7f1301aee700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12fc0af110 con 0x7f12fc0aa9b0 2022-06-29T15:43:10.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.939+0000 7f1301aee700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12fc0a9e80 con 0x7f12fc0ac670 2022-06-29T15:43:10.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.939+0000 7f1300aec700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f12fc0b2740 0x7f12fc0af990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:60910/0 (socket says 172.21.15.47:60910) 2022-06-29T15:43:10.950 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.939+0000 7f1300aec700 1 -- 172.21.15.47:0/3240954965 learned_addr learned my addr 172.21.15.47:0/3240954965 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:43:10.950 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.939+0000 7f12f37fe700 1 -- 172.21.15.47:0/3240954965 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 590951195 0 0) 0x7f12fc0ae3e0 con 0x7f12fc0b2740 2022-06-29T15:43:10.950 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.939+0000 7f13012ed700 1 --2- 172.21.15.47:0/3240954965 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f12fc0ac670 0x7f12fc14b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:10.950 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.939+0000 7f12f3fff700 1 --2- 172.21.15.47:0/3240954965 >> v2:172.21.15.64:3300/0 conn(0x7f12fc0aa9b0 0x7f12fc0ac180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:10.950 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.939+0000 7f12f37fe700 1 -- 172.21.15.47:0/3240954965 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12e8003260 con 0x7f12fc0b2740 2022-06-29T15:43:10.951 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.939+0000 7f12f37fe700 1 -- 172.21.15.47:0/3240954965 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1186555193 0 0) 0x7f12e8003260 con 0x7f12fc0b2740 2022-06-29T15:43:10.951 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.939+0000 7f12f37fe700 1 -- 172.21.15.47:0/3240954965 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f12fc0ac670 msgr2=0x7f12fc14b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:43:10.951 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.939+0000 7f12f37fe700 1 --2- 172.21.15.47:0/3240954965 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f12fc0ac670 0x7f12fc14b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:10.951 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.939+0000 7f12f37fe700 1 -- 172.21.15.47:0/3240954965 >> v2:172.21.15.64:3300/0 conn(0x7f12fc0aa9b0 msgr2=0x7f12fc0ac180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:43:10.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.939+0000 7f12f37fe700 1 --2- 172.21.15.47:0/3240954965 >> v2:172.21.15.64:3300/0 conn(0x7f12fc0aa9b0 0x7f12fc0ac180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:10.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.939+0000 7f12f37fe700 1 -- 172.21.15.47:0/3240954965 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12fc0b1750 con 0x7f12fc0b2740 2022-06-29T15:43:10.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.939+0000 7f12f37fe700 1 -- 172.21.15.47:0/3240954965 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7f12f8001660 con 0x7f12fc0b2740 2022-06-29T15:43:10.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.939+0000 7f12f37fe700 1 -- 172.21.15.47:0/3240954965 >> v1:172.21.15.47:6789/0 conn(0x7f12fc0b2740 legacy=0x7f12fc0af990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:10.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.939+0000 7f12f37fe700 1 --2- 172.21.15.47:0/3240954965 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12e8003750 0x7f12e8003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:10.953 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.939+0000 7f12f37fe700 1 --2- 172.21.15.47:0/3240954965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12e8004500 0x7f12e8004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:10.953 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.939+0000 7f12f37fe700 1 --2- 172.21.15.47:0/3240954965 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f12fc0ac670 0x7f12e8004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:10.953 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.939+0000 7f12f37fe700 1 -- 172.21.15.47:0/3240954965 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12fc0af110 con 0x7f12e8003750 2022-06-29T15:43:10.953 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.939+0000 7f12f37fe700 1 -- 172.21.15.47:0/3240954965 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e8004320 con 0x7f12e8004500 2022-06-29T15:43:10.954 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.939+0000 7f12f37fe700 1 -- 172.21.15.47:0/3240954965 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12e800b780 con 0x7f12fc0ac670 2022-06-29T15:43:10.954 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.939+0000 7f1300aec700 1 --2- 172.21.15.47:0/3240954965 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12e8003750 0x7f12e8003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:10.954 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.939+0000 7f12f3fff700 1 --2- 172.21.15.47:0/3240954965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12e8004500 0x7f12e8004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:10.954 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.939+0000 7f13012ed700 1 --2- 172.21.15.47:0/3240954965 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f12fc0ac670 0x7f12e8004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:10.954 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.939+0000 7f1300aec700 1 -- 172.21.15.47:0/3240954965 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f12fc0ac670 msgr2=0x7f12e8004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:43:10.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.939+0000 7f1300aec700 1 --2- 172.21.15.47:0/3240954965 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f12fc0ac670 0x7f12e8004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:10.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.939+0000 7f1300aec700 1 -- 172.21.15.47:0/3240954965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12e8004500 msgr2=0x7f12e8004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:43:10.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.939+0000 7f1300aec700 1 --2- 172.21.15.47:0/3240954965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12e8004500 0x7f12e8004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:10.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.939+0000 7f1300aec700 1 -- 172.21.15.47:0/3240954965 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12e8003260 con 0x7f12e8003750 2022-06-29T15:43:10.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.939+0000 7f1300aec700 1 --2- 172.21.15.47:0/3240954965 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12e8003750 0x7f12e8003b70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f12f8001370 tx=0x7f12f8001e50).ready entity=mon.0 client_cookie=36509dff7428255e server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:10.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.939+0000 7f12f37fe700 1 -- 172.21.15.47:0/3240954965 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12f80070d0 con 0x7f12e8003750 2022-06-29T15:43:10.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.939+0000 7f12f37fe700 1 -- 172.21.15.47:0/3240954965 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f8007230 con 0x7f12e8003750 2022-06-29T15:43:10.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.939+0000 7f12f37fe700 1 -- 172.21.15.47:0/3240954965 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12f800c9c0 con 0x7f12e8003750 2022-06-29T15:43:10.960 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.943+0000 7f1301aee700 1 -- 172.21.15.47:0/3240954965 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12e8003750 msgr2=0x7f12e8003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:10.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.943+0000 7f1301aee700 1 --2- 172.21.15.47:0/3240954965 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12e8003750 0x7f12e8003b70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f12f8001370 tx=0x7f12f8001e50).stop 2022-06-29T15:43:10.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.943+0000 7f13012ed700 1 -- 172.21.15.47:0/3240954965 reap_dead start 2022-06-29T15:43:10.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.943+0000 7f1301aee700 1 -- 172.21.15.47:0/3240954965 shutdown_connections 2022-06-29T15:43:10.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.943+0000 7f1301aee700 1 -- 172.21.15.47:0/3240954965 >> 172.21.15.47:0/3240954965 conn(0x7f12fc062df0 msgr2=0x7f12fc0631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:10.962 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.943+0000 7f1301aee700 1 -- 172.21.15.47:0/3240954965 shutdown_connections 2022-06-29T15:43:10.962 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.943+0000 7f1301aee700 1 -- 172.21.15.47:0/3240954965 wait complete. 2022-06-29T15:43:10.962 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.943+0000 7f1301aee700 1 Processor -- start 2022-06-29T15:43:10.962 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.943+0000 7f1301aee700 1 -- start start 2022-06-29T15:43:10.963 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.943+0000 7f1301aee700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12e8003750 0x7f12fc098ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:10.963 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.943+0000 7f1301aee700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12fc151b90 0x7f12fc143610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:10.963 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.943+0000 7f1301aee700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f12fc143b00 0x7f12fc13eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:10.963 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.943+0000 7f1301aee700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12fc143f80 con 0x7f12fc151b90 2022-06-29T15:43:10.963 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.943+0000 7f1301aee700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12fc09a980 con 0x7f12e8003750 2022-06-29T15:43:10.964 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.943+0000 7f1301aee700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12fc09ac20 con 0x7f12fc143b00 2022-06-29T15:43:10.964 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.947+0000 7f12f3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12fc151b90 0x7f12fc143610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:10.964 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.947+0000 7f12f3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12fc151b90 0x7f12fc143610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47138/0 (socket says 172.21.15.47:47138) 2022-06-29T15:43:10.965 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.947+0000 7f12f3fff700 1 -- 172.21.15.47:0/2608696813 learned_addr learned my addr 172.21.15.47:0/2608696813 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:10.965 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.947+0000 7f1300aec700 1 --2- 172.21.15.47:0/2608696813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12e8003750 0x7f12fc098ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:10.965 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.947+0000 7f13012ed700 1 --2- 172.21.15.47:0/2608696813 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f12fc143b00 0x7f12fc13eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:10.965 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.947+0000 7f12f3fff700 1 -- 172.21.15.47:0/2608696813 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f12fc143b00 msgr2=0x7f12fc13eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:10.965 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.947+0000 7f12f3fff700 1 --2- 172.21.15.47:0/2608696813 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f12fc143b00 0x7f12fc13eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:10.966 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.947+0000 7f12f3fff700 1 -- 172.21.15.47:0/2608696813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12e8003750 msgr2=0x7f12fc098ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:10.966 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.947+0000 7f12f3fff700 1 --2- 172.21.15.47:0/2608696813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12e8003750 0x7f12fc098ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:10.966 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.947+0000 7f12f3fff700 1 -- 172.21.15.47:0/2608696813 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12fc09aec0 con 0x7f12fc151b90 2022-06-29T15:43:10.966 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.947+0000 7f12f3fff700 1 --2- 172.21.15.47:0/2608696813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12fc151b90 0x7f12fc143610 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7f12ec00b000 tx=0x7f12ec006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:10.967 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.947+0000 7f12f1ffb700 1 -- 172.21.15.47:0/2608696813 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12ec011e70 con 0x7f12fc151b90 2022-06-29T15:43:10.967 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.947+0000 7f12f1ffb700 1 -- 172.21.15.47:0/2608696813 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12ec00f940 con 0x7f12fc151b90 2022-06-29T15:43:10.967 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.947+0000 7f12f1ffb700 1 -- 172.21.15.47:0/2608696813 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12ec00fca0 con 0x7f12fc151b90 2022-06-29T15:43:10.967 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.947+0000 7f1301aee700 1 -- 172.21.15.47:0/2608696813 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12fc09b160 con 0x7f12fc151b90 2022-06-29T15:43:10.968 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.947+0000 7f1301aee700 1 -- 172.21.15.47:0/2608696813 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12fc0b2030 con 0x7f12fc151b90 2022-06-29T15:43:10.968 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.947+0000 7f1301aee700 1 -- 172.21.15.47:0/2608696813 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12fc09b2f0 con 0x7f12fc151b90 2022-06-29T15:43:10.968 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.951+0000 7f12f1ffb700 1 -- 172.21.15.47:0/2608696813 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f12ec014440 con 0x7f12fc151b90 2022-06-29T15:43:10.968 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.951+0000 7f12f1ffb700 1 --2- 172.21.15.47:0/2608696813 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f12dc053b30 0x7f12dc055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:10.969 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.951+0000 7f12f1ffb700 1 -- 172.21.15.47:0/2608696813 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f12ec06c880 con 0x7f12fc151b90 2022-06-29T15:43:10.969 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.951+0000 7f1300aec700 1 --2- 172.21.15.47:0/2608696813 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f12dc053b30 0x7f12dc055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:10.969 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.951+0000 7f1300aec700 1 --2- 172.21.15.47:0/2608696813 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f12dc053b30 0x7f12dc055ff0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f12f801a000 tx=0x7f12f8015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:10.970 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:10.959+0000 7f12f1ffb700 1 -- 172.21.15.47:0/2608696813 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f12ec0435e0 con 0x7f12fc151b90 2022-06-29T15:43:11.061 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:43:11.064 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:43:11.066 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:43:11.069 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:43:11.091 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.083+0000 7f1301aee700 1 -- 172.21.15.47:0/2608696813 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f12fc0573d0 con 0x7f12fc151b90 2022-06-29T15:43:11.093 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.087+0000 7f12f1ffb700 1 -- 172.21.15.47:0/2608696813 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16284 (secure 0 0 0) 0x7f12ec018020 con 0x7f12fc151b90 2022-06-29T15:43:11.093 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:43:11.094 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":35,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:43:10.546635+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:43:08.995574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2022-06-29T15:43:09.557935+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:43:11.095 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.087+0000 7f1301aee700 1 -- 172.21.15.47:0/2608696813 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f12dc053b30 msgr2=0x7f12dc055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:11.096 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.087+0000 7f1301aee700 1 --2- 172.21.15.47:0/2608696813 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f12dc053b30 0x7f12dc055ff0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f12f801a000 tx=0x7f12f8015000).stop 2022-06-29T15:43:11.096 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.087+0000 7f1301aee700 1 -- 172.21.15.47:0/2608696813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12fc151b90 msgr2=0x7f12fc143610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:11.096 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.087+0000 7f1301aee700 1 --2- 172.21.15.47:0/2608696813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12fc151b90 0x7f12fc143610 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7f12ec00b000 tx=0x7f12ec006df0).stop 2022-06-29T15:43:11.096 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.087+0000 7f1301aee700 1 -- 172.21.15.47:0/2608696813 shutdown_connections 2022-06-29T15:43:11.097 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.087+0000 7f1301aee700 1 --2- 172.21.15.47:0/2608696813 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f12fc143b00 0x7f12fc13eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:11.097 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.087+0000 7f1301aee700 1 --2- 172.21.15.47:0/2608696813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12fc151b90 0x7f12fc143610 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:11.097 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.087+0000 7f1301aee700 1 --2- 172.21.15.47:0/2608696813 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f12dc053b30 0x7f12dc055ff0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:11.097 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.087+0000 7f1301aee700 1 --2- 172.21.15.47:0/2608696813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12e8003750 0x7f12fc098ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:11.097 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.087+0000 7f1301aee700 1 -- 172.21.15.47:0/2608696813 >> 172.21.15.47:0/2608696813 conn(0x7f12fc062df0 msgr2=0x7f12fc05c470 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:11.098 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.087+0000 7f1301aee700 1 -- 172.21.15.47:0/2608696813 shutdown_connections 2022-06-29T15:43:11.098 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.087+0000 7f1301aee700 1 -- 172.21.15.47:0/2608696813 wait complete. 2022-06-29T15:43:11.110 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-06-29T15:43:11.261 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.255+0000 7f56929a8700 1 Processor -- start 2022-06-29T15:43:11.261 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.255+0000 7f56929a8700 1 -- start start 2022-06-29T15:43:11.263 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.255+0000 7f56929a8700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f568c061220 0x7f568c140070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:11.264 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.255+0000 7f56929a8700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f568c0b4320 0x7f568c143230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:11.264 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.255+0000 7f56929a8700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f568c061600 con 0x7f568c146720 2022-06-29T15:43:11.264 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.255+0000 7f56929a8700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f568c146bc0 con 0x7f568c061220 2022-06-29T15:43:11.264 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.255+0000 7f56929a8700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f568c146e60 con 0x7f568c0b4320 2022-06-29T15:43:11.265 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.255+0000 7f568bfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f568c061220 0x7f568c140070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:11.265 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.255+0000 7f568bfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f568c061220 0x7f568c140070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47144/0 (socket says 172.21.15.47:47144) 2022-06-29T15:43:11.265 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.255+0000 7f568bfff700 1 -- 172.21.15.47:0/4150059465 learned_addr learned my addr 172.21.15.47:0/4150059465 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:11.265 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.255+0000 7f5690f45700 1 --2- 172.21.15.47:0/4150059465 >> v2:172.21.15.64:3300/0 conn(0x7f568c0b4320 0x7f568c143230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:11.266 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.255+0000 7f568bfff700 1 -- 172.21.15.47:0/4150059465 >> v2:172.21.15.64:3300/0 conn(0x7f568c0b4320 msgr2=0x7f568c143230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:43:11.266 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.255+0000 7f568bfff700 1 --2- 172.21.15.47:0/4150059465 >> v2:172.21.15.64:3300/0 conn(0x7f568c0b4320 0x7f568c143230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:11.266 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.255+0000 7f568bfff700 1 -- 172.21.15.47:0/4150059465 >> v1:172.21.15.64:6789/0 conn(0x7f568c146720 legacy=0x7f568c140560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:11.266 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.255+0000 7f568bfff700 1 -- 172.21.15.47:0/4150059465 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f568c147100 con 0x7f568c061220 2022-06-29T15:43:11.267 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.255+0000 7f568bfff700 1 --2- 172.21.15.47:0/4150059465 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f568c061220 0x7f568c140070 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7f567c002f70 tx=0x7f567c006eb0).ready entity=mon.0 client_cookie=bd0a13e7cd1c4829 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:11.267 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.255+0000 7f568affd700 1 -- 172.21.15.47:0/4150059465 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f567c00d5c0 con 0x7f568c061220 2022-06-29T15:43:11.267 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.255+0000 7f568affd700 1 -- 172.21.15.47:0/4150059465 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f567c000e90 con 0x7f568c061220 2022-06-29T15:43:11.267 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.255+0000 7f568affd700 1 -- 172.21.15.47:0/4150059465 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f567c00d950 con 0x7f568c061220 2022-06-29T15:43:11.269 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.259+0000 7f56929a8700 1 -- 172.21.15.47:0/4150059465 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f568c061220 msgr2=0x7f568c140070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:11.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.259+0000 7f56929a8700 1 --2- 172.21.15.47:0/4150059465 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f568c061220 0x7f568c140070 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7f567c002f70 tx=0x7f567c006eb0).stop 2022-06-29T15:43:11.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.259+0000 7f56929a8700 1 -- 172.21.15.47:0/4150059465 shutdown_connections 2022-06-29T15:43:11.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.259+0000 7f56929a8700 1 --2- 172.21.15.47:0/4150059465 >> v2:172.21.15.64:3300/0 conn(0x7f568c0b4320 0x7f568c143230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:11.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.259+0000 7f56929a8700 1 --2- 172.21.15.47:0/4150059465 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f568c061220 0x7f568c140070 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:11.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.259+0000 7f56929a8700 1 -- 172.21.15.47:0/4150059465 >> 172.21.15.47:0/4150059465 conn(0x7f568c05c6b0 msgr2=0x7f568c05f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:11.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.259+0000 7f56929a8700 1 -- 172.21.15.47:0/4150059465 shutdown_connections 2022-06-29T15:43:11.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.259+0000 7f56929a8700 1 -- 172.21.15.47:0/4150059465 wait complete. 2022-06-29T15:43:11.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.259+0000 7f56929a8700 1 Processor -- start 2022-06-29T15:43:11.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.259+0000 7f56929a8700 1 -- start start 2022-06-29T15:43:11.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.263+0000 7f56929a8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f568c061220 0x7f568c0b3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:11.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.263+0000 7f56929a8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f568c0b4320 0x7f568c145730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:11.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.263+0000 7f56929a8700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f568c146720 0x7f568c1522c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:11.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.263+0000 7f56929a8700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568c1561c0 con 0x7f568c0b4320 2022-06-29T15:43:11.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.263+0000 7f56929a8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568c0b2dc0 con 0x7f568c061220 2022-06-29T15:43:11.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.263+0000 7f56929a8700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f568c1563c0 con 0x7f568c146720 2022-06-29T15:43:11.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.263+0000 7f568b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f568c0b4320 0x7f568c145730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:11.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.263+0000 7f568b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f568c0b4320 0x7f568c145730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47152/0 (socket says 172.21.15.47:47152) 2022-06-29T15:43:11.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.263+0000 7f568b7fe700 1 -- 172.21.15.47:0/2433541422 learned_addr learned my addr 172.21.15.47:0/2433541422 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:11.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.263+0000 7f568bfff700 1 --2- 172.21.15.47:0/2433541422 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f568c061220 0x7f568c0b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:11.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.263+0000 7f5690f45700 1 --2- 172.21.15.47:0/2433541422 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f568c146720 0x7f568c1522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:11.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.263+0000 7f568b7fe700 1 -- 172.21.15.47:0/2433541422 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f568c146720 msgr2=0x7f568c1522c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:11.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.263+0000 7f568b7fe700 1 --2- 172.21.15.47:0/2433541422 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f568c146720 0x7f568c1522c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:11.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.263+0000 7f568b7fe700 1 -- 172.21.15.47:0/2433541422 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f568c061220 msgr2=0x7f568c0b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:11.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.263+0000 7f568b7fe700 1 --2- 172.21.15.47:0/2433541422 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f568c061220 0x7f568c0b3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:11.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.263+0000 7f568b7fe700 1 -- 172.21.15.47:0/2433541422 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f567c00b1e0 con 0x7f568c0b4320 2022-06-29T15:43:11.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.263+0000 7f568b7fe700 1 --2- 172.21.15.47:0/2433541422 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f568c0b4320 0x7f568c145730 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7f568c0b9030 tx=0x7f5680007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:11.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.263+0000 7f56897fa700 1 -- 172.21.15.47:0/2433541422 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5680012400 con 0x7f568c0b4320 2022-06-29T15:43:11.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.263+0000 7f56897fa700 1 -- 172.21.15.47:0/2433541422 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5680012560 con 0x7f568c0b4320 2022-06-29T15:43:11.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.263+0000 7f56929a8700 1 -- 172.21.15.47:0/2433541422 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f568c0b2ff0 con 0x7f568c0b4320 2022-06-29T15:43:11.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.263+0000 7f56897fa700 1 -- 172.21.15.47:0/2433541422 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f56800128c0 con 0x7f568c0b4320 2022-06-29T15:43:11.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.263+0000 7f56929a8700 1 -- 172.21.15.47:0/2433541422 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f568c156660 con 0x7f568c0b4320 2022-06-29T15:43:11.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.267+0000 7f56929a8700 1 -- 172.21.15.47:0/2433541422 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f568c0b3180 con 0x7f568c0b4320 2022-06-29T15:43:11.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.267+0000 7f56897fa700 1 -- 172.21.15.47:0/2433541422 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5680019030 con 0x7f568c0b4320 2022-06-29T15:43:11.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.267+0000 7f56897fa700 1 --2- 172.21.15.47:0/2433541422 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f566c053b00 0x7f566c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:11.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.267+0000 7f56897fa700 1 -- 172.21.15.47:0/2433541422 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f568006b2b0 con 0x7f568c0b4320 2022-06-29T15:43:11.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.267+0000 7f568bfff700 1 --2- 172.21.15.47:0/2433541422 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f566c053b00 0x7f566c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:11.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.267+0000 7f568bfff700 1 --2- 172.21.15.47:0/2433541422 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f566c053b00 0x7f566c055fc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f567c01c000 tx=0x7f567c015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:11.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.275+0000 7f56897fa700 1 -- 172.21.15.47:0/2433541422 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5680042070 con 0x7f568c0b4320 2022-06-29T15:43:11.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.439+0000 7f56929a8700 1 -- 172.21.15.47:0/2433541422 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f568c0573d0 con 0x7f568c0b4320 2022-06-29T15:43:11.561 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.555+0000 7f56897fa700 1 -- 172.21.15.47:0/2433541422 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v12) v1 ==== 118+0+0 (secure 0 0 0) 0x7f5680045690 con 0x7f568c0b4320 2022-06-29T15:43:11.565 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.559+0000 7f56929a8700 1 -- 172.21.15.47:0/2433541422 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f566c053b00 msgr2=0x7f566c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:11.565 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.559+0000 7f56929a8700 1 --2- 172.21.15.47:0/2433541422 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f566c053b00 0x7f566c055fc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f567c01c000 tx=0x7f567c015000).stop 2022-06-29T15:43:11.566 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.559+0000 7f56929a8700 1 -- 172.21.15.47:0/2433541422 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f568c0b4320 msgr2=0x7f568c145730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:11.566 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.559+0000 7f56929a8700 1 --2- 172.21.15.47:0/2433541422 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f568c0b4320 0x7f568c145730 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7f568c0b9030 tx=0x7f5680007c50).stop 2022-06-29T15:43:11.566 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.559+0000 7f56929a8700 1 -- 172.21.15.47:0/2433541422 shutdown_connections 2022-06-29T15:43:11.566 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.559+0000 7f56929a8700 1 --2- 172.21.15.47:0/2433541422 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f568c146720 0x7f568c1522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:11.567 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.559+0000 7f56929a8700 1 --2- 172.21.15.47:0/2433541422 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f568c0b4320 0x7f568c145730 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:11.567 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.559+0000 7f56929a8700 1 --2- 172.21.15.47:0/2433541422 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f566c053b00 0x7f566c055fc0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:11.567 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.559+0000 7f56929a8700 1 --2- 172.21.15.47:0/2433541422 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f568c061220 0x7f568c0b3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:11.567 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.559+0000 7f56929a8700 1 -- 172.21.15.47:0/2433541422 >> 172.21.15.47:0/2433541422 conn(0x7f568c05c6b0 msgr2=0x7f568c0b6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:11.567 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.559+0000 7f56929a8700 1 -- 172.21.15.47:0/2433541422 shutdown_connections 2022-06-29T15:43:11.568 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.559+0000 7f56929a8700 1 -- 172.21.15.47:0/2433541422 wait complete. 2022-06-29T15:43:11.580 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:43:11.688 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.679+0000 7f566d88d700 1 Processor -- start 2022-06-29T15:43:11.688 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.679+0000 7f566d88d700 1 -- start start 2022-06-29T15:43:11.692 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.683+0000 7f566d88d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f56680b2ee0 0x7f56680b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:11.692 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.683+0000 7f566d88d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f56680ad970 0x7f56681407e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:11.692 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.683+0000 7f566d88d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f56680b0e50 0x7f566813d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:11.692 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.683+0000 7f566d88d700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f566813fef0 con 0x7f56680ad970 2022-06-29T15:43:11.693 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.683+0000 7f566d88d700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5668143810 con 0x7f56680b0e50 2022-06-29T15:43:11.693 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.683+0000 7f566d88d700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56681400e0 con 0x7f56680b2ee0 2022-06-29T15:43:11.693 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.683+0000 7f5667fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f56680ad970 0x7f56681407e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:11.693 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.683+0000 7f5667fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f56680ad970 0x7f56681407e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47158/0 (socket says 172.21.15.47:47158) 2022-06-29T15:43:11.693 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.683+0000 7f5667fff700 1 -- 172.21.15.47:0/4081352105 learned_addr learned my addr 172.21.15.47:0/4081352105 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:11.694 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.683+0000 7f566d08c700 1 --2- 172.21.15.47:0/4081352105 >> v2:172.21.15.64:3300/0 conn(0x7f56680b0e50 0x7f566813d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:11.694 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.683+0000 7f566c88b700 1 --2- 172.21.15.47:0/4081352105 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f56680b2ee0 0x7f56680b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:11.694 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.683+0000 7f5667fff700 1 -- 172.21.15.47:0/4081352105 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f56680b2ee0 msgr2=0x7f56680b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:43:11.695 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.683+0000 7f5667fff700 1 --2- 172.21.15.47:0/4081352105 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f56680b2ee0 0x7f56680b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:11.695 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.683+0000 7f5667fff700 1 -- 172.21.15.47:0/4081352105 >> v2:172.21.15.64:3300/0 conn(0x7f56680b0e50 msgr2=0x7f566813d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:43:11.695 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.683+0000 7f5667fff700 1 --2- 172.21.15.47:0/4081352105 >> v2:172.21.15.64:3300/0 conn(0x7f56680b0e50 0x7f566813d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:11.695 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.683+0000 7f5667fff700 1 -- 172.21.15.47:0/4081352105 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5668143570 con 0x7f56680ad970 2022-06-29T15:43:11.695 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.683+0000 7f5667fff700 1 --2- 172.21.15.47:0/4081352105 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56680ad970 0x7f56681407e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f56680aefc0 tx=0x7f5654006eb0).ready entity=mon.0 client_cookie=4b52d00626648798 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:11.696 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.683+0000 7f56677fe700 1 -- 172.21.15.47:0/4081352105 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f565400e610 con 0x7f56680ad970 2022-06-29T15:43:11.696 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.683+0000 7f56677fe700 1 -- 172.21.15.47:0/4081352105 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5654000e90 con 0x7f56680ad970 2022-06-29T15:43:11.696 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.683+0000 7f56677fe700 1 -- 172.21.15.47:0/4081352105 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f565400e9a0 con 0x7f56680ad970 2022-06-29T15:43:11.696 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.683+0000 7f566d88d700 1 -- 172.21.15.47:0/4081352105 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56680ad970 msgr2=0x7f56681407e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:11.697 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.683+0000 7f566d88d700 1 --2- 172.21.15.47:0/4081352105 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56680ad970 0x7f56681407e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f56680aefc0 tx=0x7f5654006eb0).stop 2022-06-29T15:43:11.697 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.683+0000 7f566d88d700 1 -- 172.21.15.47:0/4081352105 shutdown_connections 2022-06-29T15:43:11.697 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.683+0000 7f566d88d700 1 --2- 172.21.15.47:0/4081352105 >> v2:172.21.15.64:3300/0 conn(0x7f56680b0e50 0x7f566813d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:11.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.683+0000 7f566d88d700 1 --2- 172.21.15.47:0/4081352105 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56680ad970 0x7f56681407e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:11.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.683+0000 7f566d88d700 1 --2- 172.21.15.47:0/4081352105 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f56680b2ee0 0x7f56680b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:11.699 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.683+0000 7f566d88d700 1 -- 172.21.15.47:0/4081352105 >> 172.21.15.47:0/4081352105 conn(0x7f5668063670 msgr2=0x7f5668063a80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:11.699 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.683+0000 7f566d88d700 1 -- 172.21.15.47:0/4081352105 shutdown_connections 2022-06-29T15:43:11.699 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.683+0000 7f566d88d700 1 -- 172.21.15.47:0/4081352105 wait complete. 2022-06-29T15:43:11.702 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.687+0000 7f566d88d700 1 Processor -- start 2022-06-29T15:43:11.702 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.687+0000 7f566d88d700 1 -- start start 2022-06-29T15:43:11.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.687+0000 7f566d88d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f56680ad970 0x7f56680b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:11.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.687+0000 7f566d88d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56680b0e50 0x7f566813c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:11.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.687+0000 7f566d88d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56680b2ee0 0x7f566813cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:11.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.687+0000 7f566d88d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56680ac320 con 0x7f56680b0e50 2022-06-29T15:43:11.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.687+0000 7f566d88d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56680ac5c0 con 0x7f56680b2ee0 2022-06-29T15:43:11.704 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.687+0000 7f566d88d700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56680ac860 con 0x7f56680ad970 2022-06-29T15:43:11.704 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.687+0000 7f5667fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56680b0e50 0x7f566813c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:11.704 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.687+0000 7f5667fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56680b0e50 0x7f566813c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47166/0 (socket says 172.21.15.47:47166) 2022-06-29T15:43:11.704 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.687+0000 7f5667fff700 1 -- 172.21.15.47:0/1427718 learned_addr learned my addr 172.21.15.47:0/1427718 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:11.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.687+0000 7f566c88b700 1 --2- 172.21.15.47:0/1427718 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f56680ad970 0x7f56680b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:11.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.687+0000 7f566d08c700 1 --2- 172.21.15.47:0/1427718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56680b2ee0 0x7f566813cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:11.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.687+0000 7f5667fff700 1 -- 172.21.15.47:0/1427718 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f56680ad970 msgr2=0x7f56680b4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:11.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.687+0000 7f5667fff700 1 --2- 172.21.15.47:0/1427718 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f56680ad970 0x7f56680b4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:11.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.687+0000 7f5667fff700 1 -- 172.21.15.47:0/1427718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56680b2ee0 msgr2=0x7f566813cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:11.707 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.687+0000 7f5667fff700 1 --2- 172.21.15.47:0/1427718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56680b2ee0 0x7f566813cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:11.707 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.687+0000 7f5667fff700 1 -- 172.21.15.47:0/1427718 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f565400b1e0 con 0x7f56680b0e50 2022-06-29T15:43:11.707 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.691+0000 7f5667fff700 1 --2- 172.21.15.47:0/1427718 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56680b0e50 0x7f566813c7b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7f56680aefc0 tx=0x7f5654015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:11.708 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.691+0000 7f5665ffb700 1 -- 172.21.15.47:0/1427718 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5654011760 con 0x7f56680b0e50 2022-06-29T15:43:11.708 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.691+0000 7f5665ffb700 1 -- 172.21.15.47:0/1427718 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56540118c0 con 0x7f56680b0e50 2022-06-29T15:43:11.708 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.691+0000 7f5665ffb700 1 -- 172.21.15.47:0/1427718 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5654011c40 con 0x7f56680b0e50 2022-06-29T15:43:11.708 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.691+0000 7f566d88d700 1 -- 172.21.15.47:0/1427718 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56680acb00 con 0x7f56680b0e50 2022-06-29T15:43:11.708 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.691+0000 7f566d88d700 1 -- 172.21.15.47:0/1427718 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56680abe30 con 0x7f56680b0e50 2022-06-29T15:43:11.709 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.691+0000 7f566d88d700 1 -- 172.21.15.47:0/1427718 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56680573d0 con 0x7f56680b0e50 2022-06-29T15:43:11.709 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.691+0000 7f5665ffb700 1 -- 172.21.15.47:0/1427718 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f565400f070 con 0x7f56680b0e50 2022-06-29T15:43:11.709 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.691+0000 7f5665ffb700 1 --2- 172.21.15.47:0/1427718 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5648053950 0x7f5648055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:11.709 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.691+0000 7f5665ffb700 1 -- 172.21.15.47:0/1427718 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f565406d8e0 con 0x7f56680b0e50 2022-06-29T15:43:11.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.691+0000 7f566c88b700 1 --2- 172.21.15.47:0/1427718 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5648053950 0x7f5648055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:11.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.695+0000 7f566c88b700 1 --2- 172.21.15.47:0/1427718 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5648053950 0x7f5648055e10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f565c00b000 tx=0x7f565c006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:11.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.699+0000 7f5665ffb700 1 -- 172.21.15.47:0/1427718 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5654044640 con 0x7f56680b0e50 2022-06-29T15:43:11.867 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.859+0000 7f566d88d700 1 -- 172.21.15.47:0/1427718 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5668069e20 con 0x7f56680b0e50 2022-06-29T15:43:11.868 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.859+0000 7f5665ffb700 1 -- 172.21.15.47:0/1427718 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+2488 (secure 0 0 0) 0x7f5654047c60 con 0x7f56680b0e50 2022-06-29T15:43:11.868 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:43:11.869 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":12,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4896,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1047560169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1047560169},{"type":"v1","addr":"172.21.15.47:6835","nonce":1047560169}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":10},{"gid":4902,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/804808837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":804808837},{"type":"v1","addr":"172.21.15.64:6833","nonce":804808837}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":10}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:43:10.546977+0000","modified":"2022-06-29T15:43:11.549959+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4905},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4905":{"gid":4905,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6837/2219068343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2219068343},{"type":"v1","addr":"172.21.15.47:6837","nonce":2219068343}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2022-06-29T15:43:11.871 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.863+0000 7f566d88d700 1 -- 172.21.15.47:0/1427718 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5648053950 msgr2=0x7f5648055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:11.871 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.863+0000 7f566d88d700 1 --2- 172.21.15.47:0/1427718 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5648053950 0x7f5648055e10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f565c00b000 tx=0x7f565c006df0).stop 2022-06-29T15:43:11.871 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.863+0000 7f566d88d700 1 -- 172.21.15.47:0/1427718 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56680b0e50 msgr2=0x7f566813c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:11.871 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.863+0000 7f566d88d700 1 --2- 172.21.15.47:0/1427718 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56680b0e50 0x7f566813c7b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7f56680aefc0 tx=0x7f5654015000).stop 2022-06-29T15:43:11.872 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.863+0000 7f566d88d700 1 -- 172.21.15.47:0/1427718 shutdown_connections 2022-06-29T15:43:11.872 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.863+0000 7f566d88d700 1 --2- 172.21.15.47:0/1427718 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5648053950 0x7f5648055e10 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:11.872 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.863+0000 7f566d88d700 1 --2- 172.21.15.47:0/1427718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56680b2ee0 0x7f566813cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:11.872 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.863+0000 7f566d88d700 1 --2- 172.21.15.47:0/1427718 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56680b0e50 0x7f566813c7b0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:11.873 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.863+0000 7f566d88d700 1 --2- 172.21.15.47:0/1427718 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f56680ad970 0x7f56680b4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:11.873 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.863+0000 7f566d88d700 1 -- 172.21.15.47:0/1427718 >> 172.21.15.47:0/1427718 conn(0x7f5668063670 msgr2=0x7f56680b63c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:11.873 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.863+0000 7f566d88d700 1 -- 172.21.15.47:0/1427718 shutdown_connections 2022-06-29T15:43:11.873 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.863+0000 7f566d88d700 1 -- 172.21.15.47:0/1427718 wait complete. 2022-06-29T15:43:11.874 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 12 2022-06-29T15:43:11.882 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:43:11.988 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.983+0000 7ff1b5b07700 1 Processor -- start 2022-06-29T15:43:11.989 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.983+0000 7ff1b5b07700 1 -- start start 2022-06-29T15:43:11.993 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.983+0000 7ff1b5b07700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff1b00b5c60 0x7ff1b0144980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:11.993 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.983+0000 7ff1b5b07700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff1b00a72f0 0x7ff1b00b2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:11.993 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.983+0000 7ff1b5b07700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1b00a9120 con 0x7ff1b0060eb0 2022-06-29T15:43:11.993 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.983+0000 7ff1b5b07700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1b00b0b80 con 0x7ff1b00b5c60 2022-06-29T15:43:11.994 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.983+0000 7ff1b5b07700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b00a89e0 con 0x7ff1b00a72f0 2022-06-29T15:43:11.994 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.983+0000 7ff1affff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff1b0060eb0 0x7ff1b00b1e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:60958/0 (socket says 172.21.15.47:60958) 2022-06-29T15:43:11.994 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.983+0000 7ff1affff700 1 -- 172.21.15.47:0/4223409888 learned_addr learned my addr 172.21.15.47:0/4223409888 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:43:11.995 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.983+0000 7ff1b4b05700 1 --2- 172.21.15.47:0/4223409888 >> v2:172.21.15.64:3300/0 conn(0x7ff1b00b5c60 0x7ff1b0144980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:11.995 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.983+0000 7ff1af7fe700 1 -- 172.21.15.47:0/4223409888 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2989692228 0 0) 0x7ff1b00a9120 con 0x7ff1b0060eb0 2022-06-29T15:43:11.995 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.983+0000 7ff1b5306700 1 --2- 172.21.15.47:0/4223409888 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff1b00a72f0 0x7ff1b00b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:11.996 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.983+0000 7ff1af7fe700 1 -- 172.21.15.47:0/4223409888 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff19c003260 con 0x7ff1b0060eb0 2022-06-29T15:43:11.996 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.983+0000 7ff1af7fe700 1 -- 172.21.15.47:0/4223409888 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3576977264 0 0) 0x7ff19c003260 con 0x7ff1b0060eb0 2022-06-29T15:43:11.996 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.983+0000 7ff1af7fe700 1 -- 172.21.15.47:0/4223409888 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff1b00a72f0 msgr2=0x7ff1b00b2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:43:11.997 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.983+0000 7ff1af7fe700 1 --2- 172.21.15.47:0/4223409888 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff1b00a72f0 0x7ff1b00b2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:11.997 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.983+0000 7ff1af7fe700 1 -- 172.21.15.47:0/4223409888 >> v2:172.21.15.64:3300/0 conn(0x7ff1b00b5c60 msgr2=0x7ff1b0144980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:43:11.997 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.983+0000 7ff1af7fe700 1 --2- 172.21.15.47:0/4223409888 >> v2:172.21.15.64:3300/0 conn(0x7ff1b00b5c60 0x7ff1b0144980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:11.997 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.983+0000 7ff1af7fe700 1 -- 172.21.15.47:0/4223409888 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b00a8c80 con 0x7ff1b0060eb0 2022-06-29T15:43:11.997 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.983+0000 7ff1af7fe700 1 -- 172.21.15.47:0/4223409888 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7ff1a0001660 con 0x7ff1b0060eb0 2022-06-29T15:43:11.998 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.983+0000 7ff1af7fe700 1 -- 172.21.15.47:0/4223409888 >> v1:172.21.15.47:6789/0 conn(0x7ff1b0060eb0 legacy=0x7ff1b00b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:11.998 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.983+0000 7ff1af7fe700 1 --2- 172.21.15.47:0/4223409888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff19c003750 0x7ff19c003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:11.998 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.983+0000 7ff1af7fe700 1 --2- 172.21.15.47:0/4223409888 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff19c004500 0x7ff19c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:11.998 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.983+0000 7ff1af7fe700 1 --2- 172.21.15.47:0/4223409888 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff19c004e70 0x7ff19c00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:11.999 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.983+0000 7ff1af7fe700 1 -- 172.21.15.47:0/4223409888 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b00b0b80 con 0x7ff19c004e70 2022-06-29T15:43:11.999 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.983+0000 7ff1af7fe700 1 -- 172.21.15.47:0/4223409888 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b00a89e0 con 0x7ff19c003750 2022-06-29T15:43:11.999 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.983+0000 7ff1af7fe700 1 -- 172.21.15.47:0/4223409888 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff19c00bc40 con 0x7ff19c004500 2022-06-29T15:43:11.999 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.983+0000 7ff1b5306700 1 --2- 172.21.15.47:0/4223409888 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff1b00a72f0 0x7ff1b00b2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:43:12.000 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.983+0000 7ff1b4b05700 1 --2- 172.21.15.47:0/4223409888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff19c003750 0x7ff19c003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:12.000 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.983+0000 7ff1affff700 1 --2- 172.21.15.47:0/4223409888 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff19c004500 0x7ff19c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:12.000 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.983+0000 7ff1b5306700 1 --2- 172.21.15.47:0/4223409888 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff19c004e70 0x7ff19c00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:12.000 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.983+0000 7ff1b5306700 1 -- 172.21.15.47:0/4223409888 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff19c004500 msgr2=0x7ff19c004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:43:12.000 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.983+0000 7ff1b5306700 1 --2- 172.21.15.47:0/4223409888 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff19c004500 0x7ff19c004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:12.001 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.983+0000 7ff1b5306700 1 -- 172.21.15.47:0/4223409888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff19c003750 msgr2=0x7ff19c003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:43:12.001 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.983+0000 7ff1b5306700 1 --2- 172.21.15.47:0/4223409888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff19c003750 0x7ff19c003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:12.001 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.983+0000 7ff1b5306700 1 -- 172.21.15.47:0/4223409888 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff19c003260 con 0x7ff19c004e70 2022-06-29T15:43:12.001 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.983+0000 7ff1b5306700 1 --2- 172.21.15.47:0/4223409888 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff19c004e70 0x7ff19c00b750 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7ff1a8000b60 tx=0x7ff1a8008040).ready entity=mon.0 client_cookie=7e10606dd215f35a server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:12.001 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.983+0000 7ff1af7fe700 1 -- 172.21.15.47:0/4223409888 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff1a800e5f0 con 0x7ff19c004e70 2022-06-29T15:43:12.002 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.983+0000 7ff1af7fe700 1 -- 172.21.15.47:0/4223409888 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1a800e750 con 0x7ff19c004e70 2022-06-29T15:43:12.002 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.983+0000 7ff1af7fe700 1 -- 172.21.15.47:0/4223409888 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff1a800eab0 con 0x7ff19c004e70 2022-06-29T15:43:12.002 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.983+0000 7ff1b5b07700 1 -- 172.21.15.47:0/4223409888 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff19c004e70 msgr2=0x7ff19c00b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:12.002 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.983+0000 7ff1b5b07700 1 --2- 172.21.15.47:0/4223409888 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff19c004e70 0x7ff19c00b750 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7ff1a8000b60 tx=0x7ff1a8008040).stop 2022-06-29T15:43:12.003 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.983+0000 7ff1b5306700 1 -- 172.21.15.47:0/4223409888 reap_dead start 2022-06-29T15:43:12.003 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.983+0000 7ff1b5b07700 1 -- 172.21.15.47:0/4223409888 shutdown_connections 2022-06-29T15:43:12.003 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.983+0000 7ff1b5b07700 1 -- 172.21.15.47:0/4223409888 >> 172.21.15.47:0/4223409888 conn(0x7ff1b0062560 msgr2=0x7ff1b0062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:12.003 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.987+0000 7ff1b5b07700 1 -- 172.21.15.47:0/4223409888 shutdown_connections 2022-06-29T15:43:12.004 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.987+0000 7ff1b5b07700 1 -- 172.21.15.47:0/4223409888 wait complete. 2022-06-29T15:43:12.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.987+0000 7ff1b5b07700 1 Processor -- start 2022-06-29T15:43:12.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.987+0000 7ff1b5b07700 1 -- start start 2022-06-29T15:43:12.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.987+0000 7ff1b5b07700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff19c004e70 0x7ff1b013ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:12.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.987+0000 7ff1b5b07700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1b00af440 0x7ff1b00ae690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:12.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.987+0000 7ff1b5b07700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1b013d4a0 0x7ff1b014c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:12.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.987+0000 7ff1b5b07700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b00adcb0 con 0x7ff1b013d4a0 2022-06-29T15:43:12.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.987+0000 7ff1b5b07700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b00adf50 con 0x7ff1b00af440 2022-06-29T15:43:12.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.987+0000 7ff1b5b07700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b00ae1f0 con 0x7ff19c004e70 2022-06-29T15:43:12.009 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.987+0000 7ff1b5306700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1b013d4a0 0x7ff1b014c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:12.009 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.987+0000 7ff1b5306700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1b013d4a0 0x7ff1b014c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47188/0 (socket says 172.21.15.47:47188) 2022-06-29T15:43:12.009 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.987+0000 7ff1b5306700 1 -- 172.21.15.47:0/4068819975 learned_addr learned my addr 172.21.15.47:0/4068819975 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:12.009 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.987+0000 7ff1affff700 1 --2- 172.21.15.47:0/4068819975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1b00af440 0x7ff1b00ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:12.010 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.987+0000 7ff1b4b05700 1 --2- 172.21.15.47:0/4068819975 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff19c004e70 0x7ff1b013ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:12.010 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.987+0000 7ff1b5306700 1 -- 172.21.15.47:0/4068819975 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff19c004e70 msgr2=0x7ff1b013ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:12.010 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.987+0000 7ff1b5306700 1 --2- 172.21.15.47:0/4068819975 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff19c004e70 0x7ff1b013ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:12.010 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.987+0000 7ff1b5306700 1 -- 172.21.15.47:0/4068819975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1b00af440 msgr2=0x7ff1b00ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:12.010 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.987+0000 7ff1b5306700 1 --2- 172.21.15.47:0/4068819975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1b00af440 0x7ff1b00ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:12.011 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.987+0000 7ff1b5306700 1 -- 172.21.15.47:0/4068819975 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b014c5a0 con 0x7ff1b013d4a0 2022-06-29T15:43:12.011 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.987+0000 7ff1b5306700 1 --2- 172.21.15.47:0/4068819975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1b013d4a0 0x7ff1b014c9d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7ff1b0061780 tx=0x7ff1a8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:12.011 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.987+0000 7ff1adffb700 1 -- 172.21.15.47:0/4068819975 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff1a8023070 con 0x7ff1b013d4a0 2022-06-29T15:43:12.011 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.987+0000 7ff1adffb700 1 -- 172.21.15.47:0/4068819975 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1a800e550 con 0x7ff1b013d4a0 2022-06-29T15:43:12.012 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.987+0000 7ff1adffb700 1 -- 172.21.15.47:0/4068819975 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff1a8005c60 con 0x7ff1b013d4a0 2022-06-29T15:43:12.012 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.991+0000 7ff1b5b07700 1 -- 172.21.15.47:0/4068819975 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1b014cec0 con 0x7ff1b013d4a0 2022-06-29T15:43:12.012 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.991+0000 7ff1b5b07700 1 -- 172.21.15.47:0/4068819975 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1b013bc00 con 0x7ff1b013d4a0 2022-06-29T15:43:12.012 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.991+0000 7ff1adffb700 1 -- 172.21.15.47:0/4068819975 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff1a8013600 con 0x7ff1b013d4a0 2022-06-29T15:43:12.013 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.991+0000 7ff1b5b07700 1 -- 172.21.15.47:0/4068819975 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1b014d0e0 con 0x7ff1b013d4a0 2022-06-29T15:43:12.013 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.991+0000 7ff1adffb700 1 --2- 172.21.15.47:0/4068819975 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff190053b30 0x7ff190055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:12.013 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.991+0000 7ff1adffb700 1 -- 172.21.15.47:0/4068819975 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff1a806d8c0 con 0x7ff1b013d4a0 2022-06-29T15:43:12.013 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.991+0000 7ff1b4b05700 1 --2- 172.21.15.47:0/4068819975 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff190053b30 0x7ff190055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:12.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.991+0000 7ff1b4b05700 1 --2- 172.21.15.47:0/4068819975 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff190053b30 0x7ff190055ff0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7ff1a4006000 tx=0x7ff1a4006d90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:12.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:11.999+0000 7ff1adffb700 1 -- 172.21.15.47:0/4068819975 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff1a80445c0 con 0x7ff1b013d4a0 2022-06-29T15:43:12.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.123+0000 7ff1b5b07700 1 -- 172.21.15.47:0/4068819975 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff1b00573d0 con 0x7ff1b013d4a0 2022-06-29T15:43:12.131 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.123+0000 7ff1adffb700 1 -- 172.21.15.47:0/4068819975 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+16271 (secure 0 0 0) 0x7ff1a8047ba0 con 0x7ff1b013d4a0 2022-06-29T15:43:12.131 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:43:12.132 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":36,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:43:11.550727+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:43:08.995574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2022-06-29T15:43:09.557935+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:43:12.133 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.127+0000 7ff1b5b07700 1 -- 172.21.15.47:0/4068819975 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff190053b30 msgr2=0x7ff190055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:12.133 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.127+0000 7ff1b5b07700 1 --2- 172.21.15.47:0/4068819975 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff190053b30 0x7ff190055ff0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7ff1a4006000 tx=0x7ff1a4006d90).stop 2022-06-29T15:43:12.134 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.127+0000 7ff1b5b07700 1 -- 172.21.15.47:0/4068819975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1b013d4a0 msgr2=0x7ff1b014c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:12.134 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.127+0000 7ff1b5b07700 1 --2- 172.21.15.47:0/4068819975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1b013d4a0 0x7ff1b014c9d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7ff1b0061780 tx=0x7ff1a8008040).stop 2022-06-29T15:43:12.134 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.127+0000 7ff1b5b07700 1 -- 172.21.15.47:0/4068819975 shutdown_connections 2022-06-29T15:43:12.134 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.127+0000 7ff1b5b07700 1 --2- 172.21.15.47:0/4068819975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1b013d4a0 0x7ff1b014c9d0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:12.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.127+0000 7ff1b5b07700 1 --2- 172.21.15.47:0/4068819975 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff190053b30 0x7ff190055ff0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:12.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.127+0000 7ff1b5b07700 1 --2- 172.21.15.47:0/4068819975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1b00af440 0x7ff1b00ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:12.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.127+0000 7ff1b5b07700 1 --2- 172.21.15.47:0/4068819975 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff19c004e70 0x7ff1b013ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:12.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.127+0000 7ff1b5b07700 1 -- 172.21.15.47:0/4068819975 >> 172.21.15.47:0/4068819975 conn(0x7ff1b0062560 msgr2=0x7ff1b00b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:12.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.127+0000 7ff1b5b07700 1 -- 172.21.15.47:0/4068819975 shutdown_connections 2022-06-29T15:43:12.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.127+0000 7ff1b5b07700 1 -- 172.21.15.47:0/4068819975 wait complete. 2022-06-29T15:43:12.145 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-29T15:43:12.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.251+0000 7fd4d63ca700 1 Processor -- start 2022-06-29T15:43:12.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.251+0000 7fd4d63ca700 1 -- start start 2022-06-29T15:43:12.261 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.251+0000 7fd4d63ca700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd4d00b4300 0x7fd4d00612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:12.261 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.251+0000 7fd4d63ca700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd4d01471a0 0x7fd4d0066f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:12.261 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.251+0000 7fd4d63ca700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4d0067610 con 0x7fd4d0060ee0 2022-06-29T15:43:12.261 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.251+0000 7fd4d63ca700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4d01467c0 con 0x7fd4d00b4300 2022-06-29T15:43:12.262 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.251+0000 7fd4d63ca700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d0146a60 con 0x7fd4d01471a0 2022-06-29T15:43:12.262 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.251+0000 7fd4cffff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd4d0060ee0 0x7fd4d01430e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:60978/0 (socket says 172.21.15.47:60978) 2022-06-29T15:43:12.262 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.251+0000 7fd4cffff700 1 -- 172.21.15.47:0/694156111 learned_addr learned my addr 172.21.15.47:0/694156111 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:43:12.262 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.251+0000 7fd4cf7fe700 1 --2- 172.21.15.47:0/694156111 >> v2:172.21.15.64:3300/0 conn(0x7fd4d00b4300 0x7fd4d00612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:12.263 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.251+0000 7fd4d4967700 1 --2- 172.21.15.47:0/694156111 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd4d01471a0 0x7fd4d0066f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:12.263 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.251+0000 7fd4ceffd700 1 -- 172.21.15.47:0/694156111 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3441951193 0 0) 0x7fd4d0067610 con 0x7fd4d0060ee0 2022-06-29T15:43:12.263 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.251+0000 7fd4ceffd700 1 -- 172.21.15.47:0/694156111 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4b0003260 con 0x7fd4d0060ee0 2022-06-29T15:43:12.263 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.255+0000 7fd4cf7fe700 1 -- 172.21.15.47:0/694156111 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd4d01471a0 msgr2=0x7fd4d0066f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:43:12.264 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.255+0000 7fd4cf7fe700 1 --2- 172.21.15.47:0/694156111 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd4d01471a0 0x7fd4d0066f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:12.264 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.255+0000 7fd4cf7fe700 1 -- 172.21.15.47:0/694156111 >> v1:172.21.15.47:6789/0 conn(0x7fd4d0060ee0 legacy=0x7fd4d01430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:12.264 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.255+0000 7fd4cf7fe700 1 -- 172.21.15.47:0/694156111 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d0146d00 con 0x7fd4d00b4300 2022-06-29T15:43:12.264 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.255+0000 7fd4d4967700 1 --2- 172.21.15.47:0/694156111 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd4d01471a0 0x7fd4d0066f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:43:12.265 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.255+0000 7fd4cf7fe700 1 --2- 172.21.15.47:0/694156111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4d00b4300 0x7fd4d00612c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7fd4d00ae790 tx=0x7fd4b8006df0).ready entity=mon.1 client_cookie=74221216811c7d5f server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:12.265 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.255+0000 7fd4ceffd700 1 -- 172.21.15.47:0/694156111 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd4b800d570 con 0x7fd4d00b4300 2022-06-29T15:43:12.265 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.255+0000 7fd4ceffd700 1 -- 172.21.15.47:0/694156111 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4b8000e90 con 0x7fd4d00b4300 2022-06-29T15:43:12.265 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.255+0000 7fd4ceffd700 1 -- 172.21.15.47:0/694156111 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd4b800d8c0 con 0x7fd4d00b4300 2022-06-29T15:43:12.269 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.255+0000 7fd4d63ca700 1 -- 172.21.15.47:0/694156111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4d00b4300 msgr2=0x7fd4d00612c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:12.269 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.255+0000 7fd4d63ca700 1 --2- 172.21.15.47:0/694156111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4d00b4300 0x7fd4d00612c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7fd4d00ae790 tx=0x7fd4b8006df0).stop 2022-06-29T15:43:12.269 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.255+0000 7fd4d63ca700 1 -- 172.21.15.47:0/694156111 shutdown_connections 2022-06-29T15:43:12.269 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.255+0000 7fd4d63ca700 1 --2- 172.21.15.47:0/694156111 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd4d01471a0 0x7fd4d0066f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:12.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.255+0000 7fd4d63ca700 1 --2- 172.21.15.47:0/694156111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4d00b4300 0x7fd4d00612c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:12.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.255+0000 7fd4d63ca700 1 -- 172.21.15.47:0/694156111 >> 172.21.15.47:0/694156111 conn(0x7fd4d005c510 msgr2=0x7fd4d005f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:12.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.255+0000 7fd4d63ca700 1 -- 172.21.15.47:0/694156111 shutdown_connections 2022-06-29T15:43:12.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.255+0000 7fd4d63ca700 1 -- 172.21.15.47:0/694156111 wait complete. 2022-06-29T15:43:12.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.255+0000 7fd4d63ca700 1 Processor -- start 2022-06-29T15:43:12.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.255+0000 7fd4d63ca700 1 -- start start 2022-06-29T15:43:12.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.255+0000 7fd4d63ca700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4d0060ee0 0x7fd4d0142ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:12.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.255+0000 7fd4d63ca700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4d00b4300 0x7fd4d013f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:12.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.255+0000 7fd4d63ca700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd4d01471a0 0x7fd4d013f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:12.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.255+0000 7fd4d63ca700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d00b3e90 con 0x7fd4d00b4300 2022-06-29T15:43:12.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.255+0000 7fd4d63ca700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d00b9800 con 0x7fd4d0060ee0 2022-06-29T15:43:12.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.255+0000 7fd4d63ca700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d00b9d40 con 0x7fd4d01471a0 2022-06-29T15:43:12.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.255+0000 7fd4cf7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4d00b4300 0x7fd4d013f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:12.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.255+0000 7fd4cf7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4d00b4300 0x7fd4d013f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47202/0 (socket says 172.21.15.47:47202) 2022-06-29T15:43:12.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.255+0000 7fd4cf7fe700 1 -- 172.21.15.47:0/847387609 learned_addr learned my addr 172.21.15.47:0/847387609 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:12.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.255+0000 7fd4cffff700 1 --2- 172.21.15.47:0/847387609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4d0060ee0 0x7fd4d0142ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:12.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.255+0000 7fd4d4967700 1 --2- 172.21.15.47:0/847387609 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd4d01471a0 0x7fd4d013f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:12.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.259+0000 7fd4cf7fe700 1 -- 172.21.15.47:0/847387609 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd4d01471a0 msgr2=0x7fd4d013f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:12.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.259+0000 7fd4cf7fe700 1 --2- 172.21.15.47:0/847387609 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd4d01471a0 0x7fd4d013f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:12.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.259+0000 7fd4cf7fe700 1 -- 172.21.15.47:0/847387609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4d0060ee0 msgr2=0x7fd4d0142ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:12.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.259+0000 7fd4cf7fe700 1 --2- 172.21.15.47:0/847387609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4d0060ee0 0x7fd4d0142ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:12.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.259+0000 7fd4cf7fe700 1 -- 172.21.15.47:0/847387609 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4c8009470 con 0x7fd4d00b4300 2022-06-29T15:43:12.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.259+0000 7fd4cf7fe700 1 --2- 172.21.15.47:0/847387609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4d00b4300 0x7fd4d013f0e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7fd4d00ae790 tx=0x7fd4b8014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:12.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.259+0000 7fd4cd7fa700 1 -- 172.21.15.47:0/847387609 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd4b8013e90 con 0x7fd4d00b4300 2022-06-29T15:43:12.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.259+0000 7fd4cd7fa700 1 -- 172.21.15.47:0/847387609 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4b8002f70 con 0x7fd4d00b4300 2022-06-29T15:43:12.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.259+0000 7fd4d63ca700 1 -- 172.21.15.47:0/847387609 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4b800b180 con 0x7fd4d00b4300 2022-06-29T15:43:12.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.259+0000 7fd4cd7fa700 1 -- 172.21.15.47:0/847387609 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd4b801e5f0 con 0x7fd4d00b4300 2022-06-29T15:43:12.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.259+0000 7fd4d63ca700 1 -- 172.21.15.47:0/847387609 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4d00b94f0 con 0x7fd4d00b4300 2022-06-29T15:43:12.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.259+0000 7fd4d63ca700 1 -- 172.21.15.47:0/847387609 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4d013fc60 con 0x7fd4d00b4300 2022-06-29T15:43:12.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.259+0000 7fd4cd7fa700 1 -- 172.21.15.47:0/847387609 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd4b801c070 con 0x7fd4d00b4300 2022-06-29T15:43:12.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.259+0000 7fd4cd7fa700 1 --2- 172.21.15.47:0/847387609 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd4b4053b00 0x7fd4b4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:12.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.259+0000 7fd4cd7fa700 1 -- 172.21.15.47:0/847387609 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd4b806cb10 con 0x7fd4d00b4300 2022-06-29T15:43:12.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.259+0000 7fd4cffff700 1 --2- 172.21.15.47:0/847387609 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd4b4053b00 0x7fd4b4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:12.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.259+0000 7fd4cffff700 1 --2- 172.21.15.47:0/847387609 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd4b4053b00 0x7fd4b4055fc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7fd4c000d000 tx=0x7fd4c0009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:12.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.267+0000 7fd4cd7fa700 1 -- 172.21.15.47:0/847387609 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd4b80438d0 con 0x7fd4d00b4300 2022-06-29T15:43:12.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.435+0000 7fd4d63ca700 1 -- 172.21.15.47:0/847387609 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fd4d0155960 con 0x7fd4d00b4300 2022-06-29T15:43:12.449 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.443+0000 7fd4cd7fa700 1 -- 172.21.15.47:0/847387609 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v14) v1 ==== 184+0+0 (secure 0 0 0) 0x7fd4b8023020 con 0x7fd4d00b4300 2022-06-29T15:43:12.452 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.443+0000 7fd4d63ca700 1 -- 172.21.15.47:0/847387609 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd4b4053b00 msgr2=0x7fd4b4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:12.452 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.443+0000 7fd4d63ca700 1 --2- 172.21.15.47:0/847387609 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd4b4053b00 0x7fd4b4055fc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7fd4c000d000 tx=0x7fd4c0009040).stop 2022-06-29T15:43:12.452 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.443+0000 7fd4d63ca700 1 -- 172.21.15.47:0/847387609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4d00b4300 msgr2=0x7fd4d013f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:12.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.443+0000 7fd4d63ca700 1 --2- 172.21.15.47:0/847387609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4d00b4300 0x7fd4d013f0e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7fd4d00ae790 tx=0x7fd4b8014000).stop 2022-06-29T15:43:12.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.443+0000 7fd4d63ca700 1 -- 172.21.15.47:0/847387609 shutdown_connections 2022-06-29T15:43:12.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.443+0000 7fd4d63ca700 1 --2- 172.21.15.47:0/847387609 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd4d01471a0 0x7fd4d013f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:12.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.443+0000 7fd4d63ca700 1 --2- 172.21.15.47:0/847387609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4d00b4300 0x7fd4d013f0e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:12.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.443+0000 7fd4d63ca700 1 --2- 172.21.15.47:0/847387609 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd4b4053b00 0x7fd4b4055fc0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:12.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.443+0000 7fd4d63ca700 1 --2- 172.21.15.47:0/847387609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4d0060ee0 0x7fd4d0142ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:12.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.443+0000 7fd4d63ca700 1 -- 172.21.15.47:0/847387609 >> 172.21.15.47:0/847387609 conn(0x7fd4d005c510 msgr2=0x7fd4d00b5290 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:12.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.443+0000 7fd4d63ca700 1 -- 172.21.15.47:0/847387609 shutdown_connections 2022-06-29T15:43:12.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.443+0000 7fd4d63ca700 1 -- 172.21.15.47:0/847387609 wait complete. 2022-06-29T15:43:12.455 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.0 2022-06-29T15:43:12.466 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-29T15:43:12.576 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.571+0000 7fc30c328700 1 Processor -- start 2022-06-29T15:43:12.577 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.571+0000 7fc30c328700 1 -- start start 2022-06-29T15:43:12.578 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.571+0000 7fc30c328700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc304060ee0 0x7fc3040612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:12.579 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.571+0000 7fc30c328700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc3040b4300 0x7fc3041471a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:12.579 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.571+0000 7fc30c328700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc304067610 con 0x7fc304066f70 2022-06-29T15:43:12.579 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.571+0000 7fc30c328700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3040673a0 con 0x7fc3040b4300 2022-06-29T15:43:12.579 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.571+0000 7fc30c328700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3041467c0 con 0x7fc304060ee0 2022-06-29T15:43:12.580 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.571+0000 7fc30a0c4700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc304060ee0 0x7fc3040612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:12.580 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.571+0000 7fc3098c3700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc3040b4300 0x7fc3041471a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:12.580 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.571+0000 7fc30a0c4700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc304060ee0 0x7fc3040612c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:44308/0 (socket says 172.21.15.47:44308) 2022-06-29T15:43:12.580 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.571+0000 7fc30a0c4700 1 -- 172.21.15.47:0/4009120486 learned_addr learned my addr 172.21.15.47:0/4009120486 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:12.582 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.571+0000 7fc3098c3700 1 -- 172.21.15.47:0/4009120486 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc304060ee0 msgr2=0x7fc3040612c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:43:12.582 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.571+0000 7fc3098c3700 1 --2- 172.21.15.47:0/4009120486 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc304060ee0 0x7fc3040612c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:12.582 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.571+0000 7fc3098c3700 1 -- 172.21.15.47:0/4009120486 >> v1:172.21.15.64:6789/0 conn(0x7fc304066f70 legacy=0x7fc3041430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:12.583 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.571+0000 7fc3098c3700 1 -- 172.21.15.47:0/4009120486 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc304146a60 con 0x7fc3040b4300 2022-06-29T15:43:12.583 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.571+0000 7fc3098c3700 1 --2- 172.21.15.47:0/4009120486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3040b4300 0x7fc3041471a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fc2f4002f70 tx=0x7fc2f4006df0).ready entity=mon.1 client_cookie=babb4f14d39f67cd server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:12.583 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.571+0000 7fc3090c2700 1 -- 172.21.15.47:0/4009120486 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc2f400d570 con 0x7fc3040b4300 2022-06-29T15:43:12.583 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.571+0000 7fc3090c2700 1 -- 172.21.15.47:0/4009120486 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f4000e90 con 0x7fc3040b4300 2022-06-29T15:43:12.584 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.571+0000 7fc3090c2700 1 -- 172.21.15.47:0/4009120486 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc2f400d8c0 con 0x7fc3040b4300 2022-06-29T15:43:12.584 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.575+0000 7fc30c328700 1 -- 172.21.15.47:0/4009120486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3040b4300 msgr2=0x7fc3041471a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:12.584 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.575+0000 7fc30c328700 1 --2- 172.21.15.47:0/4009120486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3040b4300 0x7fc3041471a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fc2f4002f70 tx=0x7fc2f4006df0).stop 2022-06-29T15:43:12.584 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.575+0000 7fc30c328700 1 -- 172.21.15.47:0/4009120486 shutdown_connections 2022-06-29T15:43:12.585 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.575+0000 7fc30c328700 1 --2- 172.21.15.47:0/4009120486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3040b4300 0x7fc3041471a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:12.585 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.575+0000 7fc30c328700 1 --2- 172.21.15.47:0/4009120486 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc304060ee0 0x7fc3040612c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:12.585 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.575+0000 7fc30c328700 1 -- 172.21.15.47:0/4009120486 >> 172.21.15.47:0/4009120486 conn(0x7fc30405c510 msgr2=0x7fc30405f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:12.585 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.575+0000 7fc30c328700 1 -- 172.21.15.47:0/4009120486 shutdown_connections 2022-06-29T15:43:12.586 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.575+0000 7fc30c328700 1 -- 172.21.15.47:0/4009120486 wait complete. 2022-06-29T15:43:12.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.575+0000 7fc30c328700 1 Processor -- start 2022-06-29T15:43:12.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.575+0000 7fc30c328700 1 -- start start 2022-06-29T15:43:12.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.579+0000 7fc30c328700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc304060ee0 0x7fc3040af880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:12.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.579+0000 7fc30c328700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc304066f70 0x7fc3040b6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:12.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.579+0000 7fc30c328700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3040b4300 0x7fc3040b6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:12.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.579+0000 7fc30c328700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc304148d70 con 0x7fc304060ee0 2022-06-29T15:43:12.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.579+0000 7fc30c328700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc304149010 con 0x7fc3040b4300 2022-06-29T15:43:12.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.579+0000 7fc30c328700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3041492b0 con 0x7fc304066f70 2022-06-29T15:43:12.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.579+0000 7fc30a0c4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc304060ee0 0x7fc3040af880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:12.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.579+0000 7fc30a0c4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc304060ee0 0x7fc3040af880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47212/0 (socket says 172.21.15.47:47212) 2022-06-29T15:43:12.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.579+0000 7fc30a0c4700 1 -- 172.21.15.47:0/496239781 learned_addr learned my addr 172.21.15.47:0/496239781 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:12.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.579+0000 7fc3098c3700 1 --2- 172.21.15.47:0/496239781 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc304066f70 0x7fc3040b6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:12.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.579+0000 7fc30a8c5700 1 --2- 172.21.15.47:0/496239781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3040b4300 0x7fc3040b6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:12.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.579+0000 7fc30a0c4700 1 -- 172.21.15.47:0/496239781 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc304066f70 msgr2=0x7fc3040b6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:12.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.579+0000 7fc30a0c4700 1 --2- 172.21.15.47:0/496239781 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc304066f70 0x7fc3040b6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:12.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.579+0000 7fc30a0c4700 1 -- 172.21.15.47:0/496239781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3040b4300 msgr2=0x7fc3040b6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:12.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.579+0000 7fc30a0c4700 1 --2- 172.21.15.47:0/496239781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3040b4300 0x7fc3040b6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:12.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.579+0000 7fc30a0c4700 1 -- 172.21.15.47:0/496239781 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f400b180 con 0x7fc304060ee0 2022-06-29T15:43:12.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.579+0000 7fc30a0c4700 1 --2- 172.21.15.47:0/496239781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc304060ee0 0x7fc3040af880 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7fc30000c000 tx=0x7fc300008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:12.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.579+0000 7fc2f37fe700 1 -- 172.21.15.47:0/496239781 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc300013610 con 0x7fc304060ee0 2022-06-29T15:43:12.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.579+0000 7fc2f37fe700 1 -- 172.21.15.47:0/496239781 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc300013770 con 0x7fc304060ee0 2022-06-29T15:43:12.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.579+0000 7fc2f37fe700 1 -- 172.21.15.47:0/496239781 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc300013ad0 con 0x7fc304060ee0 2022-06-29T15:43:12.594 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.579+0000 7fc30c328700 1 -- 172.21.15.47:0/496239781 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc304149550 con 0x7fc304060ee0 2022-06-29T15:43:12.594 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.579+0000 7fc30c328700 1 -- 172.21.15.47:0/496239781 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3041378f0 con 0x7fc304060ee0 2022-06-29T15:43:12.594 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.579+0000 7fc30c328700 1 -- 172.21.15.47:0/496239781 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3041496e0 con 0x7fc304060ee0 2022-06-29T15:43:12.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.583+0000 7fc2f37fe700 1 -- 172.21.15.47:0/496239781 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc300023070 con 0x7fc304060ee0 2022-06-29T15:43:12.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.583+0000 7fc2f37fe700 1 --2- 172.21.15.47:0/496239781 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc2e4053a70 0x7fc2e4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:12.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.583+0000 7fc2f37fe700 1 -- 172.21.15.47:0/496239781 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc30006c3c0 con 0x7fc304060ee0 2022-06-29T15:43:12.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.583+0000 7fc3098c3700 1 --2- 172.21.15.47:0/496239781 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc2e4053a70 0x7fc2e4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:12.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.583+0000 7fc3098c3700 1 --2- 172.21.15.47:0/496239781 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc2e4053a70 0x7fc2e4055f30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7fc2f401a000 tx=0x7fc2f4014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:12.597 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.591+0000 7fc2f37fe700 1 -- 172.21.15.47:0/496239781 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc300043070 con 0x7fc304060ee0 2022-06-29T15:43:12.765 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.759+0000 7fc30c328700 1 -- 172.21.15.47:0/496239781 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fc304137ba0 con 0x7fc304060ee0 2022-06-29T15:43:12.772 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.763+0000 7fc2f37fe700 1 -- 172.21.15.47:0/496239781 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v15) v1 ==== 184+0+0 (secure 0 0 0) 0x7fc300046690 con 0x7fc304060ee0 2022-06-29T15:43:12.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.767+0000 7fc30c328700 1 -- 172.21.15.47:0/496239781 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc2e4053a70 msgr2=0x7fc2e4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:12.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.767+0000 7fc30c328700 1 --2- 172.21.15.47:0/496239781 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc2e4053a70 0x7fc2e4055f30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7fc2f401a000 tx=0x7fc2f4014000).stop 2022-06-29T15:43:12.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.767+0000 7fc30c328700 1 -- 172.21.15.47:0/496239781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc304060ee0 msgr2=0x7fc3040af880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:12.776 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.767+0000 7fc30c328700 1 --2- 172.21.15.47:0/496239781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc304060ee0 0x7fc3040af880 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7fc30000c000 tx=0x7fc300008040).stop 2022-06-29T15:43:12.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.767+0000 7fc30c328700 1 -- 172.21.15.47:0/496239781 shutdown_connections 2022-06-29T15:43:12.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.767+0000 7fc30c328700 1 --2- 172.21.15.47:0/496239781 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc2e4053a70 0x7fc2e4055f30 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:12.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.767+0000 7fc30c328700 1 --2- 172.21.15.47:0/496239781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3040b4300 0x7fc3040b6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:12.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.767+0000 7fc30c328700 1 --2- 172.21.15.47:0/496239781 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc304066f70 0x7fc3040b6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:12.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.767+0000 7fc30c328700 1 --2- 172.21.15.47:0/496239781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc304060ee0 0x7fc3040af880 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:12.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.767+0000 7fc30c328700 1 -- 172.21.15.47:0/496239781 >> 172.21.15.47:0/496239781 conn(0x7fc30405c510 msgr2=0x7fc3040b8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:12.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.767+0000 7fc30c328700 1 -- 172.21.15.47:0/496239781 shutdown_connections 2022-06-29T15:43:12.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.767+0000 7fc30c328700 1 -- 172.21.15.47:0/496239781 wait complete. 2022-06-29T15:43:12.781 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.1 2022-06-29T15:43:12.788 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:43:12.895 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.887+0000 7f67e9fca700 1 Processor -- start 2022-06-29T15:43:12.895 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.887+0000 7f67e9fca700 1 -- start start 2022-06-29T15:43:12.897 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.887+0000 7f67e9fca700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f67e40b2ee0 0x7f67e40b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:12.898 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.887+0000 7f67e9fca700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67e41400e0 con 0x7f67e40b0e50 2022-06-29T15:43:12.898 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.887+0000 7f67e9fca700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67e4143570 con 0x7f67e40ad970 2022-06-29T15:43:12.899 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.887+0000 7f67e9fca700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67e4140ef0 con 0x7f67e40b2ee0 2022-06-29T15:43:12.899 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.887+0000 7f67e97c9700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f67e40b0e50 0x7f67e413d290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:32778/0 (socket says 172.21.15.47:32778) 2022-06-29T15:43:12.899 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.887+0000 7f67e97c9700 1 -- 172.21.15.47:0/1122060538 learned_addr learned my addr 172.21.15.47:0/1122060538 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:43:12.900 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.887+0000 7f67e8fc8700 1 --2- 172.21.15.47:0/1122060538 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f67e40b2ee0 0x7f67e40b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:12.900 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.891+0000 7f67e37fe700 1 -- 172.21.15.47:0/1122060538 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2757527478 0 0) 0x7f67e41400e0 con 0x7f67e40b0e50 2022-06-29T15:43:12.900 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.891+0000 7f67e37fe700 1 -- 172.21.15.47:0/1122060538 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67c4003260 con 0x7f67e40b0e50 2022-06-29T15:43:12.900 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.891+0000 7f67e8fc8700 1 -- 172.21.15.47:0/1122060538 >> v1:172.21.15.64:6789/0 conn(0x7f67e40ad970 legacy=0x7f67e41407e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:12.901 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.891+0000 7f67e37fe700 1 -- 172.21.15.47:0/1122060538 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1372460883 0 0) 0x7f67e4143570 con 0x7f67e40ad970 2022-06-29T15:43:12.901 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.891+0000 7f67e8fc8700 1 -- 172.21.15.47:0/1122060538 >> v1:172.21.15.47:6789/0 conn(0x7f67e40b0e50 legacy=0x7f67e413d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:12.901 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.891+0000 7f67e8fc8700 1 -- 172.21.15.47:0/1122060538 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e4140340 con 0x7f67e40b2ee0 2022-06-29T15:43:12.901 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.891+0000 7f67e8fc8700 1 --2- 172.21.15.47:0/1122060538 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f67e40b2ee0 0x7f67e40b32c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f67d40032f0 tx=0x7f67d40091d0).ready entity=mon.2 client_cookie=64be02afda4e44cf server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:12.902 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.891+0000 7f67e37fe700 1 -- 172.21.15.47:0/1122060538 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f67d400b6c0 con 0x7f67e40b2ee0 2022-06-29T15:43:12.902 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.891+0000 7f67e37fe700 1 -- 172.21.15.47:0/1122060538 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d400b820 con 0x7f67e40b2ee0 2022-06-29T15:43:12.902 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.891+0000 7f67e37fe700 1 -- 172.21.15.47:0/1122060538 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f67d400b990 con 0x7f67e40b2ee0 2022-06-29T15:43:12.906 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.891+0000 7f67e9fca700 1 -- 172.21.15.47:0/1122060538 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f67e40b2ee0 msgr2=0x7f67e40b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:12.907 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.891+0000 7f67e9fca700 1 --2- 172.21.15.47:0/1122060538 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f67e40b2ee0 0x7f67e40b32c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f67d40032f0 tx=0x7f67d40091d0).stop 2022-06-29T15:43:12.907 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.891+0000 7f67e9fca700 1 -- 172.21.15.47:0/1122060538 shutdown_connections 2022-06-29T15:43:12.907 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.891+0000 7f67e9fca700 1 --2- 172.21.15.47:0/1122060538 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f67e40b2ee0 0x7f67e40b32c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:12.907 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.891+0000 7f67e9fca700 1 -- 172.21.15.47:0/1122060538 >> 172.21.15.47:0/1122060538 conn(0x7f67e4063670 msgr2=0x7f67e4063a80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:12.908 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.891+0000 7f67e9fca700 1 -- 172.21.15.47:0/1122060538 shutdown_connections 2022-06-29T15:43:12.908 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.891+0000 7f67e9fca700 1 -- 172.21.15.47:0/1122060538 wait complete. 2022-06-29T15:43:12.908 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.895+0000 7f67e9fca700 1 Processor -- start 2022-06-29T15:43:12.909 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.895+0000 7f67e9fca700 1 -- start start 2022-06-29T15:43:12.909 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.895+0000 7f67e9fca700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67e40ad970 0x7f67e40b5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:12.909 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.895+0000 7f67e9fca700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f67e40b0e50 0x7f67e4142d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:12.909 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.895+0000 7f67e9fca700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67e40b2ee0 0x7f67e4141290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:12.910 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.895+0000 7f67e9fca700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e40ac410 con 0x7f67e40b2ee0 2022-06-29T15:43:12.910 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.895+0000 7f67e9fca700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e414f330 con 0x7f67e40ad970 2022-06-29T15:43:12.910 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.895+0000 7f67e9fca700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67e414f5d0 con 0x7f67e40b0e50 2022-06-29T15:43:12.910 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.895+0000 7f67e97c9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67e40b2ee0 0x7f67e4141290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:12.910 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.895+0000 7f67e8fc8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67e40ad970 0x7f67e40b5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:12.911 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.895+0000 7f67e97c9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67e40b2ee0 0x7f67e4141290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47230/0 (socket says 172.21.15.47:47230) 2022-06-29T15:43:12.911 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.895+0000 7f67e97c9700 1 -- 172.21.15.47:0/1630687021 learned_addr learned my addr 172.21.15.47:0/1630687021 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:12.911 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.895+0000 7f67e3fff700 1 --2- 172.21.15.47:0/1630687021 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f67e40b0e50 0x7f67e4142d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:12.911 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.895+0000 7f67e97c9700 1 -- 172.21.15.47:0/1630687021 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f67e40b0e50 msgr2=0x7f67e4142d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:12.912 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.895+0000 7f67e97c9700 1 --2- 172.21.15.47:0/1630687021 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f67e40b0e50 0x7f67e4142d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:12.912 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.895+0000 7f67e97c9700 1 -- 172.21.15.47:0/1630687021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67e40ad970 msgr2=0x7f67e40b5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:12.912 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.895+0000 7f67e97c9700 1 --2- 172.21.15.47:0/1630687021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67e40ad970 0x7f67e40b5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:12.912 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.899+0000 7f67e97c9700 1 -- 172.21.15.47:0/1630687021 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d4009470 con 0x7f67e40b2ee0 2022-06-29T15:43:12.913 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.899+0000 7f67e3fff700 1 --2- 172.21.15.47:0/1630687021 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f67e40b0e50 0x7f67e4142d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:43:12.913 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.899+0000 7f67e8fc8700 1 --2- 172.21.15.47:0/1630687021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67e40ad970 0x7f67e40b5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:43:12.913 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.899+0000 7f67e97c9700 1 --2- 172.21.15.47:0/1630687021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67e40b2ee0 0x7f67e4141290 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7f67dc001110 tx=0x7f67dc007a80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:12.913 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.899+0000 7f67e1ffb700 1 -- 172.21.15.47:0/1630687021 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f67dc00ce80 con 0x7f67e40b2ee0 2022-06-29T15:43:12.914 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.899+0000 7f67e1ffb700 1 -- 172.21.15.47:0/1630687021 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67dc0070d0 con 0x7f67e40b2ee0 2022-06-29T15:43:12.914 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.899+0000 7f67e9fca700 1 -- 172.21.15.47:0/1630687021 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67e414f8d0 con 0x7f67e40b2ee0 2022-06-29T15:43:12.914 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.899+0000 7f67e1ffb700 1 -- 172.21.15.47:0/1630687021 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f67dc007430 con 0x7f67e40b2ee0 2022-06-29T15:43:12.914 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.899+0000 7f67e9fca700 1 -- 172.21.15.47:0/1630687021 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67e4143200 con 0x7f67e40b2ee0 2022-06-29T15:43:12.915 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.899+0000 7f67e9fca700 1 -- 172.21.15.47:0/1630687021 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67e40573d0 con 0x7f67e40b2ee0 2022-06-29T15:43:12.916 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.899+0000 7f67e1ffb700 1 -- 172.21.15.47:0/1630687021 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f67dc01a060 con 0x7f67e40b2ee0 2022-06-29T15:43:12.916 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.899+0000 7f67e1ffb700 1 --2- 172.21.15.47:0/1630687021 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f67c8053a70 0x7f67c8055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:12.916 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.903+0000 7f67e1ffb700 1 -- 172.21.15.47:0/1630687021 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f67dc06b670 con 0x7f67e40b2ee0 2022-06-29T15:43:12.917 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.903+0000 7f67e8fc8700 1 --2- 172.21.15.47:0/1630687021 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f67c8053a70 0x7f67c8055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:12.917 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.903+0000 7f67e8fc8700 1 --2- 172.21.15.47:0/1630687021 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f67c8053a70 0x7f67c8055f30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f67d401a000 tx=0x7f67d4016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:12.917 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:12.907+0000 7f67e1ffb700 1 -- 172.21.15.47:0/1630687021 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f67dc042320 con 0x7f67e40b2ee0 2022-06-29T15:43:13.082 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.075+0000 7f67e9fca700 1 -- 172.21.15.47:0/1630687021 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f67e40ad590 con 0x7f67e40b2ee0 2022-06-29T15:43:13.082 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.075+0000 7f67e1ffb700 1 -- 172.21.15.47:0/1630687021 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+2488 (secure 0 0 0) 0x7f67dc018070 con 0x7f67e40b2ee0 2022-06-29T15:43:13.083 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:43:13.083 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":12,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4896,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1047560169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1047560169},{"type":"v1","addr":"172.21.15.47:6835","nonce":1047560169}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":10},{"gid":4902,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/804808837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":804808837},{"type":"v1","addr":"172.21.15.64:6833","nonce":804808837}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":10}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:43:10.546977+0000","modified":"2022-06-29T15:43:11.549959+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4905},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4905":{"gid":4905,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6837/2219068343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2219068343},{"type":"v1","addr":"172.21.15.47:6837","nonce":2219068343}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2022-06-29T15:43:13.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.079+0000 7f67e9fca700 1 -- 172.21.15.47:0/1630687021 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f67c8053a70 msgr2=0x7f67c8055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:13.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.079+0000 7f67e9fca700 1 --2- 172.21.15.47:0/1630687021 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f67c8053a70 0x7f67c8055f30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f67d401a000 tx=0x7f67d4016000).stop 2022-06-29T15:43:13.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.079+0000 7f67e9fca700 1 -- 172.21.15.47:0/1630687021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67e40b2ee0 msgr2=0x7f67e4141290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:13.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.079+0000 7f67e9fca700 1 --2- 172.21.15.47:0/1630687021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67e40b2ee0 0x7f67e4141290 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7f67dc001110 tx=0x7f67dc007a80).stop 2022-06-29T15:43:13.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.079+0000 7f67e9fca700 1 -- 172.21.15.47:0/1630687021 shutdown_connections 2022-06-29T15:43:13.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.079+0000 7f67e9fca700 1 --2- 172.21.15.47:0/1630687021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67e40b2ee0 0x7f67e4141290 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:13.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.079+0000 7f67e9fca700 1 --2- 172.21.15.47:0/1630687021 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f67e40b0e50 0x7f67e4142d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:13.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.079+0000 7f67e9fca700 1 --2- 172.21.15.47:0/1630687021 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f67c8053a70 0x7f67c8055f30 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:13.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.079+0000 7f67e9fca700 1 --2- 172.21.15.47:0/1630687021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67e40ad970 0x7f67e40b5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:13.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.079+0000 7f67e9fca700 1 -- 172.21.15.47:0/1630687021 >> 172.21.15.47:0/1630687021 conn(0x7f67e4063670 msgr2=0x7f67e40b0320 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:13.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.079+0000 7f67e9fca700 1 -- 172.21.15.47:0/1630687021 shutdown_connections 2022-06-29T15:43:13.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.079+0000 7f67e9fca700 1 -- 172.21.15.47:0/1630687021 wait complete. 2022-06-29T15:43:13.088 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 12 2022-06-29T15:43:13.399 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 12, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-29T15:43:10.546977+0000', 'modified': '2022-06-29T15:43:11.549959+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4905}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4905': {'gid': 4905, 'name': 'a', 'rank': 0, 'incarnation': 11, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.47:6837/2219068343', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6836', 'nonce': 2219068343}, {'type': 'v1', 'addr': '172.21.15.47:6837', 'nonce': 2219068343}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-29T15:43:13.400 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-29T15:43:13.401 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-06-29T15:43:13.519 INFO:teuthology.orchestra.run.smithi047.stdout:{"cluster_fsid":"9a68894a-53ce-400e-838b-14eab91b834c","whoami":0,"id":4905,"want_state":"up:active","state":"up:active","rank_uptime":2.9546654970000001,"mdsmap_epoch":12,"osdmap_epoch":36,"osdmap_epoch_barrier":35,"uptime":9.1796485729999997} 2022-06-29T15:43:13.519 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9a68894a-53ce-400e-838b-14eab91b834c", "id": 4905, "mdsmap_epoch": 12, "osdmap_epoch": 36, "osdmap_epoch_barrier": 35, "rank_uptime": 2.954665497, "state": "up:active", "uptime": 9.179648573, "want_state": "up:active", "whoami": 0 } 2022-06-29T15:43:13.520 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:43:13.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.615+0000 7fbfc48f8700 1 Processor -- start 2022-06-29T15:43:13.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.615+0000 7fbfc48f8700 1 -- start start 2022-06-29T15:43:13.626 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.619+0000 7fbfc48f8700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbfc0143bd0 0x7fbfc005c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:13.627 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.619+0000 7fbfc48f8700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbfc00b2c40 0x7fbfc00b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:13.627 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.619+0000 7fbfc48f8700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfc0142f90 con 0x7fbfc005c510 2022-06-29T15:43:13.627 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.619+0000 7fbfc48f8700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfc01431f0 con 0x7fbfc0143bd0 2022-06-29T15:43:13.627 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.619+0000 7fbfc48f8700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc0143490 con 0x7fbfc00b2c40 2022-06-29T15:43:13.627 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.619+0000 7fbfbe59c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbfc0143bd0 0x7fbfc005c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:13.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.619+0000 7fbfbf59e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbfc00b2c40 0x7fbfc00b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:13.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.619+0000 7fbfbe59c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbfc0143bd0 0x7fbfc005c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:52938/0 (socket says 172.21.15.47:52938) 2022-06-29T15:43:13.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.619+0000 7fbfbf59e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbfc00b2c40 0x7fbfc00b30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:44340/0 (socket says 172.21.15.47:44340) 2022-06-29T15:43:13.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.619+0000 7fbfbe59c700 1 -- 172.21.15.47:0/2560756906 learned_addr learned my addr 172.21.15.47:0/2560756906 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:13.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.619+0000 7fbfbe59c700 1 -- 172.21.15.47:0/2560756906 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbfc00b2c40 msgr2=0x7fbfc00b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:43:13.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.619+0000 7fbfbe59c700 1 --2- 172.21.15.47:0/2560756906 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbfc00b2c40 0x7fbfc00b30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:13.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.619+0000 7fbfbe59c700 1 -- 172.21.15.47:0/2560756906 >> v1:172.21.15.64:6789/0 conn(0x7fbfc005c510 legacy=0x7fbfc00b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:13.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.619+0000 7fbfbe59c700 1 -- 172.21.15.47:0/2560756906 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc0143730 con 0x7fbfc0143bd0 2022-06-29T15:43:13.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.619+0000 7fbfbe59c700 1 --2- 172.21.15.47:0/2560756906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfc0143bd0 0x7fbfc005c8f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7fbfb0002f70 tx=0x7fbfb0006eb0).ready entity=mon.1 client_cookie=8b6c54223c601262 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:13.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.619+0000 7fbfbdd9b700 1 -- 172.21.15.47:0/2560756906 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfb000d570 con 0x7fbfc0143bd0 2022-06-29T15:43:13.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.619+0000 7fbfbdd9b700 1 -- 172.21.15.47:0/2560756906 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb0000e90 con 0x7fbfc0143bd0 2022-06-29T15:43:13.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.619+0000 7fbfbdd9b700 1 -- 172.21.15.47:0/2560756906 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfb000d900 con 0x7fbfc0143bd0 2022-06-29T15:43:13.633 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.619+0000 7fbfc48f8700 1 -- 172.21.15.47:0/2560756906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfc0143bd0 msgr2=0x7fbfc005c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:13.634 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.619+0000 7fbfc48f8700 1 --2- 172.21.15.47:0/2560756906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfc0143bd0 0x7fbfc005c8f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7fbfb0002f70 tx=0x7fbfb0006eb0).stop 2022-06-29T15:43:13.634 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.619+0000 7fbfc48f8700 1 -- 172.21.15.47:0/2560756906 shutdown_connections 2022-06-29T15:43:13.634 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.619+0000 7fbfc48f8700 1 --2- 172.21.15.47:0/2560756906 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbfc00b2c40 0x7fbfc00b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:13.634 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.619+0000 7fbfc48f8700 1 --2- 172.21.15.47:0/2560756906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfc0143bd0 0x7fbfc005c8f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:13.635 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.619+0000 7fbfc48f8700 1 -- 172.21.15.47:0/2560756906 >> 172.21.15.47:0/2560756906 conn(0x7fbfc0060e00 msgr2=0x7fbfc00611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:13.635 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.619+0000 7fbfc48f8700 1 -- 172.21.15.47:0/2560756906 shutdown_connections 2022-06-29T15:43:13.635 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.619+0000 7fbfc48f8700 1 -- 172.21.15.47:0/2560756906 wait complete. 2022-06-29T15:43:13.635 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.623+0000 7fbfc48f8700 1 Processor -- start 2022-06-29T15:43:13.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.623+0000 7fbfc48f8700 1 -- start start 2022-06-29T15:43:13.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.623+0000 7fbfc48f8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfc005c510 0x7fbfc013dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:13.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.623+0000 7fbfc48f8700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbfc00b2c40 0x7fbfc013e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:13.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.623+0000 7fbfc48f8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfc0143bd0 0x7fbfc013ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:13.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.623+0000 7fbfc48f8700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc0152310 con 0x7fbfc0143bd0 2022-06-29T15:43:13.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.623+0000 7fbfc48f8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc00b6500 con 0x7fbfc005c510 2022-06-29T15:43:13.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.623+0000 7fbfc48f8700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc00b5d60 con 0x7fbfc00b2c40 2022-06-29T15:43:13.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.623+0000 7fbfbf59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfc0143bd0 0x7fbfc013ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:13.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.623+0000 7fbfbf59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfc0143bd0 0x7fbfc013ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47244/0 (socket says 172.21.15.47:47244) 2022-06-29T15:43:13.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.623+0000 7fbfbf59e700 1 -- 172.21.15.47:0/3098414977 learned_addr learned my addr 172.21.15.47:0/3098414977 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:13.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.623+0000 7fbfbe59c700 1 --2- 172.21.15.47:0/3098414977 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbfc00b2c40 0x7fbfc013e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:13.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.623+0000 7fbfbed9d700 1 --2- 172.21.15.47:0/3098414977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfc005c510 0x7fbfc013dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:13.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.623+0000 7fbfbf59e700 1 -- 172.21.15.47:0/3098414977 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbfc00b2c40 msgr2=0x7fbfc013e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:13.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.623+0000 7fbfbf59e700 1 --2- 172.21.15.47:0/3098414977 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbfc00b2c40 0x7fbfc013e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:13.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.623+0000 7fbfbf59e700 1 -- 172.21.15.47:0/3098414977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfc005c510 msgr2=0x7fbfc013dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:13.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.623+0000 7fbfbf59e700 1 --2- 172.21.15.47:0/3098414977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfc005c510 0x7fbfc013dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:13.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.623+0000 7fbfbf59e700 1 -- 172.21.15.47:0/3098414977 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb000b1e0 con 0x7fbfc0143bd0 2022-06-29T15:43:13.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.623+0000 7fbfbf59e700 1 --2- 172.21.15.47:0/3098414977 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfc0143bd0 0x7fbfc013ebc0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7fbfb8006e00 tx=0x7fbfb80091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:13.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.623+0000 7fbfb7fff700 1 -- 172.21.15.47:0/3098414977 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfb800c640 con 0x7fbfc0143bd0 2022-06-29T15:43:13.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.623+0000 7fbfb7fff700 1 -- 172.21.15.47:0/3098414977 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb800c7a0 con 0x7fbfc0143bd0 2022-06-29T15:43:13.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.623+0000 7fbfb7fff700 1 -- 172.21.15.47:0/3098414977 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfb800cb00 con 0x7fbfc0143bd0 2022-06-29T15:43:13.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.623+0000 7fbfc48f8700 1 -- 172.21.15.47:0/3098414977 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfc00b5fc0 con 0x7fbfc0143bd0 2022-06-29T15:43:13.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.623+0000 7fbfc48f8700 1 -- 172.21.15.47:0/3098414977 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfc013b790 con 0x7fbfc0143bd0 2022-06-29T15:43:13.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.627+0000 7fbfc48f8700 1 -- 172.21.15.47:0/3098414977 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfc013e410 con 0x7fbfc0143bd0 2022-06-29T15:43:13.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.627+0000 7fbfb7fff700 1 -- 172.21.15.47:0/3098414977 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fbfb800cc60 con 0x7fbfc0143bd0 2022-06-29T15:43:13.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.627+0000 7fbfb7fff700 1 --2- 172.21.15.47:0/3098414977 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbfa40539e0 0x7fbfa4055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:13.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.627+0000 7fbfb7fff700 1 -- 172.21.15.47:0/3098414977 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fbfb806c250 con 0x7fbfc0143bd0 2022-06-29T15:43:13.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.627+0000 7fbfbed9d700 1 --2- 172.21.15.47:0/3098414977 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbfa40539e0 0x7fbfa4055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:13.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.627+0000 7fbfbed9d700 1 --2- 172.21.15.47:0/3098414977 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbfa40539e0 0x7fbfa4055ea0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7fbfa800d000 tx=0x7fbfa8000bc0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:13.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.635+0000 7fbfb7fff700 1 -- 172.21.15.47:0/3098414977 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbfb8043010 con 0x7fbfc0143bd0 2022-06-29T15:43:13.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.763+0000 7fbfc48f8700 1 -- 172.21.15.47:0/3098414977 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbfc0061b00 con 0x7fbfc0143bd0 2022-06-29T15:43:13.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.767+0000 7fbfb7fff700 1 -- 172.21.15.47:0/3098414977 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+16271 (secure 0 0 0) 0x7fbfb8046630 con 0x7fbfc0143bd0 2022-06-29T15:43:13.775 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:43:13.775 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":36,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:43:11.550727+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:43:08.995574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2022-06-29T15:43:09.557935+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:43:13.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.771+0000 7fbfc48f8700 1 -- 172.21.15.47:0/3098414977 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbfa40539e0 msgr2=0x7fbfa4055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:13.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.771+0000 7fbfc48f8700 1 --2- 172.21.15.47:0/3098414977 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbfa40539e0 0x7fbfa4055ea0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7fbfa800d000 tx=0x7fbfa8000bc0).stop 2022-06-29T15:43:13.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.771+0000 7fbfc48f8700 1 -- 172.21.15.47:0/3098414977 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfc0143bd0 msgr2=0x7fbfc013ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:13.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.771+0000 7fbfc48f8700 1 --2- 172.21.15.47:0/3098414977 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfc0143bd0 0x7fbfc013ebc0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7fbfb8006e00 tx=0x7fbfb80091d0).stop 2022-06-29T15:43:13.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.771+0000 7fbfc48f8700 1 -- 172.21.15.47:0/3098414977 shutdown_connections 2022-06-29T15:43:13.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.771+0000 7fbfc48f8700 1 --2- 172.21.15.47:0/3098414977 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfc0143bd0 0x7fbfc013ebc0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:13.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.771+0000 7fbfc48f8700 1 --2- 172.21.15.47:0/3098414977 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbfc00b2c40 0x7fbfc013e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:13.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.771+0000 7fbfc48f8700 1 --2- 172.21.15.47:0/3098414977 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbfa40539e0 0x7fbfa4055ea0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:13.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.771+0000 7fbfc48f8700 1 --2- 172.21.15.47:0/3098414977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfc005c510 0x7fbfc013dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:13.781 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.771+0000 7fbfc48f8700 1 -- 172.21.15.47:0/3098414977 >> 172.21.15.47:0/3098414977 conn(0x7fbfc0060e00 msgr2=0x7fbfc0066a80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:13.781 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.771+0000 7fbfc48f8700 1 -- 172.21.15.47:0/3098414977 shutdown_connections 2022-06-29T15:43:13.781 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.771+0000 7fbfc48f8700 1 -- 172.21.15.47:0/3098414977 wait complete. 2022-06-29T15:43:13.794 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-29T15:42:16.971431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-29T15:42:25.290884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-29T15:43:08.995574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2022-06-29T15:43:09.557935+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-29T15:43:13.795 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-29T15:43:13.907 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.899+0000 7feced8fd700 1 Processor -- start 2022-06-29T15:43:13.907 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.899+0000 7feced8fd700 1 -- start start 2022-06-29T15:43:13.908 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.899+0000 7feced8fd700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fece80b3c90 0x7fece80a9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:13.908 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.899+0000 7feced8fd700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fece8141b60 con 0x7fece805c510 2022-06-29T15:43:13.909 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.899+0000 7feced8fd700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fece805d160 con 0x7fece8060a70 2022-06-29T15:43:13.909 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.899+0000 7feced8fd700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fece805f7c0 con 0x7fece80b3c90 2022-06-29T15:43:13.909 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.903+0000 7fece67fc700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fece805c510 0x7fece80b3580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:32804/0 (socket says 172.21.15.47:32804) 2022-06-29T15:43:13.909 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.903+0000 7fece67fc700 1 -- 172.21.15.47:0/533421262 learned_addr learned my addr 172.21.15.47:0/533421262 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:43:13.910 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.903+0000 7fece77fe700 1 --2- 172.21.15.47:0/533421262 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fece80b3c90 0x7fece80a9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:13.914 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.903+0000 7fece5ffb700 1 -- 172.21.15.47:0/533421262 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4156825313 0 0) 0x7fece8141b60 con 0x7fece805c510 2022-06-29T15:43:13.914 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.903+0000 7fece5ffb700 1 -- 172.21.15.47:0/533421262 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feccc003260 con 0x7fece805c510 2022-06-29T15:43:13.915 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.903+0000 7fece5ffb700 1 -- 172.21.15.47:0/533421262 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1946413209 0 0) 0x7fece805d160 con 0x7fece8060a70 2022-06-29T15:43:13.915 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.903+0000 7fece5ffb700 1 -- 172.21.15.47:0/533421262 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fece8141b60 con 0x7fece8060a70 2022-06-29T15:43:13.915 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.903+0000 7fece5ffb700 1 -- 172.21.15.47:0/533421262 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 115980405 0 0) 0x7feccc003260 con 0x7fece805c510 2022-06-29T15:43:13.915 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.903+0000 7fece5ffb700 1 -- 172.21.15.47:0/533421262 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fece80b3c90 msgr2=0x7fece80a9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:43:13.916 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.903+0000 7fece5ffb700 1 --2- 172.21.15.47:0/533421262 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fece80b3c90 0x7fece80a9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:13.916 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.903+0000 7fece5ffb700 1 -- 172.21.15.47:0/533421262 >> v1:172.21.15.64:6789/0 conn(0x7fece8060a70 legacy=0x7fece805f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:13.916 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.903+0000 7fece5ffb700 1 -- 172.21.15.47:0/533421262 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fece8141dc0 con 0x7fece805c510 2022-06-29T15:43:13.916 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.903+0000 7fece5ffb700 1 -- 172.21.15.47:0/533421262 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7fecdc001680 con 0x7fece805c510 2022-06-29T15:43:13.916 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.903+0000 7fece5ffb700 1 -- 172.21.15.47:0/533421262 >> v1:172.21.15.47:6789/0 conn(0x7fece805c510 legacy=0x7fece80b3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:13.917 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.903+0000 7fece5ffb700 1 --2- 172.21.15.47:0/533421262 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7feccc003910 0x7feccc003d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:13.917 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.903+0000 7fece5ffb700 1 --2- 172.21.15.47:0/533421262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feccc004500 0x7feccc004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:13.917 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.903+0000 7fece5ffb700 1 --2- 172.21.15.47:0/533421262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feccc004e70 0x7feccc00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:13.917 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.903+0000 7fece5ffb700 1 -- 172.21.15.47:0/533421262 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece805f7c0 con 0x7feccc004e70 2022-06-29T15:43:13.918 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.903+0000 7fece5ffb700 1 -- 172.21.15.47:0/533421262 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feccc00bc40 con 0x7feccc004500 2022-06-29T15:43:13.918 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.903+0000 7fece5ffb700 1 -- 172.21.15.47:0/533421262 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feccc00be00 con 0x7feccc003910 2022-06-29T15:43:13.918 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.903+0000 7fece77fe700 1 --2- 172.21.15.47:0/533421262 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fece80b3c90 0x7fece80a9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:43:13.918 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.903+0000 7fece6ffd700 1 --2- 172.21.15.47:0/533421262 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7feccc003910 0x7feccc003d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:13.919 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.903+0000 7fece77fe700 1 --2- 172.21.15.47:0/533421262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feccc004e70 0x7feccc00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:13.919 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.903+0000 7fece67fc700 1 --2- 172.21.15.47:0/533421262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feccc004500 0x7feccc004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:13.919 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.903+0000 7fece77fe700 1 -- 172.21.15.47:0/533421262 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7feccc003910 msgr2=0x7feccc003d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:43:13.919 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.903+0000 7fece77fe700 1 --2- 172.21.15.47:0/533421262 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7feccc003910 0x7feccc003d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:13.919 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.903+0000 7fece77fe700 1 -- 172.21.15.47:0/533421262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feccc004500 msgr2=0x7feccc004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:43:13.920 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.903+0000 7fece77fe700 1 --2- 172.21.15.47:0/533421262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feccc004500 0x7feccc004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:13.920 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.903+0000 7fece77fe700 1 -- 172.21.15.47:0/533421262 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feccc003260 con 0x7feccc004e70 2022-06-29T15:43:13.920 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.903+0000 7fece77fe700 1 --2- 172.21.15.47:0/533421262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feccc004e70 0x7feccc00b750 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7fece000c000 tx=0x7fece00091d0).ready entity=mon.0 client_cookie=159d7065d007cc5 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:13.920 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.903+0000 7fece5ffb700 1 -- 172.21.15.47:0/533421262 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fece0009700 con 0x7feccc004e70 2022-06-29T15:43:13.921 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.903+0000 7fece5ffb700 1 -- 172.21.15.47:0/533421262 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fece000e6e0 con 0x7feccc004e70 2022-06-29T15:43:13.921 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.903+0000 7fece5ffb700 1 -- 172.21.15.47:0/533421262 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fece000ea40 con 0x7feccc004e70 2022-06-29T15:43:13.921 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.903+0000 7feced8fd700 1 -- 172.21.15.47:0/533421262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feccc004e70 msgr2=0x7feccc00b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:13.921 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.903+0000 7feced8fd700 1 --2- 172.21.15.47:0/533421262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feccc004e70 0x7feccc00b750 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7fece000c000 tx=0x7fece00091d0).stop 2022-06-29T15:43:13.921 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.903+0000 7fece77fe700 1 -- 172.21.15.47:0/533421262 reap_dead start 2022-06-29T15:43:13.922 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.903+0000 7feced8fd700 1 -- 172.21.15.47:0/533421262 shutdown_connections 2022-06-29T15:43:13.922 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.903+0000 7feced8fd700 1 -- 172.21.15.47:0/533421262 >> 172.21.15.47:0/533421262 conn(0x7fece8062110 msgr2=0x7fece80601c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:13.922 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.903+0000 7feced8fd700 1 -- 172.21.15.47:0/533421262 shutdown_connections 2022-06-29T15:43:13.922 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.903+0000 7feced8fd700 1 -- 172.21.15.47:0/533421262 wait complete. 2022-06-29T15:43:13.923 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.907+0000 7feced8fd700 1 Processor -- start 2022-06-29T15:43:13.923 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.907+0000 7feced8fd700 1 -- start start 2022-06-29T15:43:13.923 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.907+0000 7feced8fd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fece80ab6d0 0x7fece80abab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:13.923 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.907+0000 7feced8fd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fece80abfa0 0x7fece8138be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:13.923 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.907+0000 7feced8fd700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fece81390d0 0x7fece80662a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:13.924 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.907+0000 7feced8fd700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece80af3a0 con 0x7fece80abfa0 2022-06-29T15:43:13.924 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.907+0000 7feced8fd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece8064b90 con 0x7fece80ab6d0 2022-06-29T15:43:13.924 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.907+0000 7feced8fd700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fece80af8f0 con 0x7fece81390d0 2022-06-29T15:43:13.924 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.907+0000 7fece67fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fece80abfa0 0x7fece8138be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:13.925 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.907+0000 7fece67fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fece80abfa0 0x7fece8138be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47262/0 (socket says 172.21.15.47:47262) 2022-06-29T15:43:13.925 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.907+0000 7fece67fc700 1 -- 172.21.15.47:0/1586439244 learned_addr learned my addr 172.21.15.47:0/1586439244 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:13.925 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.907+0000 7fece6ffd700 1 --2- 172.21.15.47:0/1586439244 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fece80ab6d0 0x7fece80abab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:13.925 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.907+0000 7fece77fe700 1 --2- 172.21.15.47:0/1586439244 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fece81390d0 0x7fece80662a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:13.925 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.907+0000 7fece67fc700 1 -- 172.21.15.47:0/1586439244 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fece81390d0 msgr2=0x7fece80662a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:13.926 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.907+0000 7fece67fc700 1 --2- 172.21.15.47:0/1586439244 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fece81390d0 0x7fece80662a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:13.926 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.907+0000 7fece67fc700 1 -- 172.21.15.47:0/1586439244 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fece80ab6d0 msgr2=0x7fece80abab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:13.926 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.907+0000 7fece67fc700 1 --2- 172.21.15.47:0/1586439244 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fece80ab6d0 0x7fece80abab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:13.926 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.907+0000 7fece67fc700 1 -- 172.21.15.47:0/1586439244 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fece8065260 con 0x7fece80abfa0 2022-06-29T15:43:13.927 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.907+0000 7fece67fc700 1 --2- 172.21.15.47:0/1586439244 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fece80abfa0 0x7fece8138be0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7fece8156c60 tx=0x7fece8140f60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:13.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.907+0000 7fecd7fff700 1 -- 172.21.15.47:0/1586439244 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fecdc011780 con 0x7fece80abfa0 2022-06-29T15:43:13.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.907+0000 7fecd7fff700 1 -- 172.21.15.47:0/1586439244 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecdc0118e0 con 0x7fece80abfa0 2022-06-29T15:43:13.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.907+0000 7fecd7fff700 1 -- 172.21.15.47:0/1586439244 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fecdc011c40 con 0x7fece80abfa0 2022-06-29T15:43:13.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.907+0000 7feced8fd700 1 -- 172.21.15.47:0/1586439244 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fece8157830 con 0x7fece80abfa0 2022-06-29T15:43:13.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.907+0000 7feced8fd700 1 -- 172.21.15.47:0/1586439244 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fece8064dc0 con 0x7fece80abfa0 2022-06-29T15:43:13.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.911+0000 7feced8fd700 1 -- 172.21.15.47:0/1586439244 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fece8065ed0 con 0x7fece80abfa0 2022-06-29T15:43:13.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.911+0000 7fecd7fff700 1 -- 172.21.15.47:0/1586439244 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fecdc01c360 con 0x7fece80abfa0 2022-06-29T15:43:13.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.911+0000 7fecd7fff700 1 --2- 172.21.15.47:0/1586439244 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fecc4053aa0 0x7fecc4055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:13.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.911+0000 7fecd7fff700 1 -- 172.21.15.47:0/1586439244 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fecdc06bd40 con 0x7fece80abfa0 2022-06-29T15:43:13.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.911+0000 7fece6ffd700 1 --2- 172.21.15.47:0/1586439244 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fecc4053aa0 0x7fecc4055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:13.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.911+0000 7fece6ffd700 1 --2- 172.21.15.47:0/1586439244 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fecc4053aa0 0x7fecc4055f60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7fecd8003000 tx=0x7fecd8009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:13.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:13.919+0000 7fecd7fff700 1 -- 172.21.15.47:0/1586439244 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fece8065ed0 con 0x7fece80abfa0 2022-06-29T15:43:14.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.039+0000 7feced8fd700 1 -- 172.21.15.47:0/1586439244 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fece8065ed0 con 0x7fece80abfa0 2022-06-29T15:43:14.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.043+0000 7fecd7fff700 1 -- 172.21.15.47:0/1586439244 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v36) v1 ==== 110+0+10 (secure 0 0 0) 0x7fece8065ed0 con 0x7fece80abfa0 2022-06-29T15:43:14.049 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2022-06-29T15:43:14.052 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.043+0000 7feced8fd700 1 -- 172.21.15.47:0/1586439244 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fecc4053aa0 msgr2=0x7fecc4055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:14.053 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.043+0000 7feced8fd700 1 --2- 172.21.15.47:0/1586439244 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fecc4053aa0 0x7fecc4055f60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7fecd8003000 tx=0x7fecd8009040).stop 2022-06-29T15:43:14.053 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.043+0000 7feced8fd700 1 -- 172.21.15.47:0/1586439244 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fece80abfa0 msgr2=0x7fece8138be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:14.053 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.043+0000 7feced8fd700 1 --2- 172.21.15.47:0/1586439244 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fece80abfa0 0x7fece8138be0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7fece8156c60 tx=0x7fece8140f60).stop 2022-06-29T15:43:14.054 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.047+0000 7feced8fd700 1 -- 172.21.15.47:0/1586439244 shutdown_connections 2022-06-29T15:43:14.054 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.047+0000 7feced8fd700 1 --2- 172.21.15.47:0/1586439244 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fece81390d0 0x7fece80662a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:14.054 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.047+0000 7feced8fd700 1 --2- 172.21.15.47:0/1586439244 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fece80abfa0 0x7fece8138be0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:14.055 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.047+0000 7feced8fd700 1 --2- 172.21.15.47:0/1586439244 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fecc4053aa0 0x7fecc4055f60 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:14.055 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.047+0000 7feced8fd700 1 --2- 172.21.15.47:0/1586439244 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fece80ab6d0 0x7fece80abab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:14.055 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.047+0000 7feced8fd700 1 -- 172.21.15.47:0/1586439244 >> 172.21.15.47:0/1586439244 conn(0x7fece8062110 msgr2=0x7fece805f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:14.055 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.047+0000 7feced8fd700 1 -- 172.21.15.47:0/1586439244 shutdown_connections 2022-06-29T15:43:14.056 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.047+0000 7feced8fd700 1 -- 172.21.15.47:0/1586439244 wait complete. 2022-06-29T15:43:14.067 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-29T15:43:14.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.175+0000 7ff0f885f700 1 Processor -- start 2022-06-29T15:43:14.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.175+0000 7ff0f885f700 1 -- start start 2022-06-29T15:43:14.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.175+0000 7ff0f885f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff0f40980c0 0x7ff0f40a97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:14.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.175+0000 7ff0f885f700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0f4060370 con 0x7ff0f4097ce0 2022-06-29T15:43:14.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.175+0000 7ff0f885f700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0f4095320 con 0x7ff0f4060b30 2022-06-29T15:43:14.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.175+0000 7ff0f885f700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0f40ae5f0 con 0x7ff0f40980c0 2022-06-29T15:43:14.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.175+0000 7ff0f259c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff0f4097ce0 0x7ff0f40af830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:32822/0 (socket says 172.21.15.47:32822) 2022-06-29T15:43:14.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.175+0000 7ff0f259c700 1 -- 172.21.15.47:0/780299899 learned_addr learned my addr 172.21.15.47:0/780299899 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:43:14.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.175+0000 7ff0f2d9d700 1 --2- 172.21.15.47:0/780299899 >> v2:172.21.15.47:3300/0 conn(0x7ff0f40980c0 0x7ff0f40a97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:14.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.175+0000 7ff0f159a700 1 -- 172.21.15.47:0/780299899 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1290576640 0 0) 0x7ff0f4060370 con 0x7ff0f4097ce0 2022-06-29T15:43:14.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.175+0000 7ff0f159a700 1 -- 172.21.15.47:0/780299899 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0d8003260 con 0x7ff0f4097ce0 2022-06-29T15:43:14.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.175+0000 7ff0f2d9d700 1 -- 172.21.15.47:0/780299899 >> v1:172.21.15.64:6789/0 conn(0x7ff0f4060b30 legacy=0x7ff0f40b08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:14.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.175+0000 7ff0f2d9d700 1 -- 172.21.15.47:0/780299899 >> v1:172.21.15.47:6789/0 conn(0x7ff0f4097ce0 legacy=0x7ff0f40af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:14.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.175+0000 7ff0f2d9d700 1 -- 172.21.15.47:0/780299899 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f40605d0 con 0x7ff0f40980c0 2022-06-29T15:43:14.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.175+0000 7ff0f2d9d700 1 --2- 172.21.15.47:0/780299899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0f40980c0 0x7ff0f40a97f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7ff0ec0032f0 tx=0x7ff0ec0091d0).ready entity=mon.0 client_cookie=a820d040e4102f70 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:14.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.175+0000 7ff0f159a700 1 -- 172.21.15.47:0/780299899 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0ec00b6c0 con 0x7ff0f40980c0 2022-06-29T15:43:14.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.179+0000 7ff0f159a700 1 -- 172.21.15.47:0/780299899 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0ec00b820 con 0x7ff0f40980c0 2022-06-29T15:43:14.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.179+0000 7ff0f159a700 1 -- 172.21.15.47:0/780299899 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0ec00b990 con 0x7ff0f40980c0 2022-06-29T15:43:14.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.179+0000 7ff0f885f700 1 -- 172.21.15.47:0/780299899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0f40980c0 msgr2=0x7ff0f40a97f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:14.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.179+0000 7ff0f885f700 1 --2- 172.21.15.47:0/780299899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0f40980c0 0x7ff0f40a97f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7ff0ec0032f0 tx=0x7ff0ec0091d0).stop 2022-06-29T15:43:14.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.179+0000 7ff0f885f700 1 -- 172.21.15.47:0/780299899 shutdown_connections 2022-06-29T15:43:14.191 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.179+0000 7ff0f885f700 1 --2- 172.21.15.47:0/780299899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0f40980c0 0x7ff0f40a97f0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:14.191 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.179+0000 7ff0f885f700 1 -- 172.21.15.47:0/780299899 >> 172.21.15.47:0/780299899 conn(0x7ff0f405f0b0 msgr2=0x7ff0f405f490 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:14.191 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.179+0000 7ff0f885f700 1 -- 172.21.15.47:0/780299899 shutdown_connections 2022-06-29T15:43:14.191 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.179+0000 7ff0f885f700 1 -- 172.21.15.47:0/780299899 wait complete. 2022-06-29T15:43:14.192 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.179+0000 7ff0f885f700 1 Processor -- start 2022-06-29T15:43:14.192 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.179+0000 7ff0f885f700 1 -- start start 2022-06-29T15:43:14.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.179+0000 7ff0f885f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0f4060b30 0x7ff0f40b0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:14.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.179+0000 7ff0f885f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0f4097ce0 0x7ff0f4097240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:14.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.179+0000 7ff0f885f700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff0f40980c0 0x7ff0f4097730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:14.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.179+0000 7ff0f885f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f40b3f30 con 0x7ff0f4097ce0 2022-06-29T15:43:14.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.179+0000 7ff0f885f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f4096370 con 0x7ff0f4060b30 2022-06-29T15:43:14.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.179+0000 7ff0f885f700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f4096610 con 0x7ff0f40980c0 2022-06-29T15:43:14.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.179+0000 7ff0f1d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0f4097ce0 0x7ff0f4097240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:14.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.179+0000 7ff0f259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0f4060b30 0x7ff0f40b0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:14.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.179+0000 7ff0f1d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0f4097ce0 0x7ff0f4097240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47278/0 (socket says 172.21.15.47:47278) 2022-06-29T15:43:14.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.179+0000 7ff0f1d9b700 1 -- 172.21.15.47:0/2169940810 learned_addr learned my addr 172.21.15.47:0/2169940810 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:14.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.179+0000 7ff0f2d9d700 1 --2- 172.21.15.47:0/2169940810 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff0f40980c0 0x7ff0f4097730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:14.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.179+0000 7ff0f1d9b700 1 -- 172.21.15.47:0/2169940810 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff0f40980c0 msgr2=0x7ff0f4097730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:14.198 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.179+0000 7ff0f1d9b700 1 --2- 172.21.15.47:0/2169940810 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff0f40980c0 0x7ff0f4097730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:14.198 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.179+0000 7ff0f1d9b700 1 -- 172.21.15.47:0/2169940810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0f4060b30 msgr2=0x7ff0f40b0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:14.198 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.179+0000 7ff0f1d9b700 1 --2- 172.21.15.47:0/2169940810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0f4060b30 0x7ff0f40b0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:14.198 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.179+0000 7ff0f1d9b700 1 -- 172.21.15.47:0/2169940810 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0ec009470 con 0x7ff0f4097ce0 2022-06-29T15:43:14.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.179+0000 7ff0f1d9b700 1 --2- 172.21.15.47:0/2169940810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0f4097ce0 0x7ff0f4097240 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7ff0f4095900 tx=0x7ff0e8007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:14.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.179+0000 7ff0e37fe700 1 -- 172.21.15.47:0/2169940810 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0e8012400 con 0x7ff0f4097ce0 2022-06-29T15:43:14.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.183+0000 7ff0e37fe700 1 -- 172.21.15.47:0/2169940810 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0e8012560 con 0x7ff0f4097ce0 2022-06-29T15:43:14.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.183+0000 7ff0f885f700 1 -- 172.21.15.47:0/2169940810 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0f4094cf0 con 0x7ff0f4097ce0 2022-06-29T15:43:14.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.183+0000 7ff0e37fe700 1 -- 172.21.15.47:0/2169940810 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0e80128c0 con 0x7ff0f4097ce0 2022-06-29T15:43:14.200 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.183+0000 7ff0f885f700 1 -- 172.21.15.47:0/2169940810 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0f40b3ca0 con 0x7ff0f4097ce0 2022-06-29T15:43:14.200 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.183+0000 7ff0e37fe700 1 -- 172.21.15.47:0/2169940810 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff0e8019030 con 0x7ff0f4097ce0 2022-06-29T15:43:14.200 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.183+0000 7ff0f885f700 1 -- 172.21.15.47:0/2169940810 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0f40967d0 con 0x7ff0f4097ce0 2022-06-29T15:43:14.200 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.183+0000 7ff0e37fe700 1 --2- 172.21.15.47:0/2169940810 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff0d0053950 0x7ff0d0055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:14.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.183+0000 7ff0e37fe700 1 -- 172.21.15.47:0/2169940810 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff0e806b390 con 0x7ff0f4097ce0 2022-06-29T15:43:14.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.183+0000 7ff0f259c700 1 --2- 172.21.15.47:0/2169940810 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff0d0053950 0x7ff0d0055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:14.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.183+0000 7ff0f259c700 1 --2- 172.21.15.47:0/2169940810 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff0d0053950 0x7ff0d0055e10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7ff0e400d000 tx=0x7ff0e4009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:14.202 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.191+0000 7ff0e37fe700 1 -- 172.21.15.47:0/2169940810 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff0e80420f0 con 0x7ff0f4097ce0 2022-06-29T15:43:14.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.315+0000 7ff0f885f700 1 -- 172.21.15.47:0/2169940810 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff0f40573d0 con 0x7ff0f4097ce0 2022-06-29T15:43:14.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.315+0000 7ff0e37fe700 1 -- 172.21.15.47:0/2169940810 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v36) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff0e8045710 con 0x7ff0f4097ce0 2022-06-29T15:43:14.325 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:43:14.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.319+0000 7ff0f885f700 1 -- 172.21.15.47:0/2169940810 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff0d0053950 msgr2=0x7ff0d0055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:14.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.319+0000 7ff0f885f700 1 --2- 172.21.15.47:0/2169940810 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff0d0053950 0x7ff0d0055e10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7ff0e400d000 tx=0x7ff0e4009040).stop 2022-06-29T15:43:14.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.319+0000 7ff0f885f700 1 -- 172.21.15.47:0/2169940810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0f4097ce0 msgr2=0x7ff0f4097240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:14.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.319+0000 7ff0f885f700 1 --2- 172.21.15.47:0/2169940810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0f4097ce0 0x7ff0f4097240 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7ff0f4095900 tx=0x7ff0e8007c50).stop 2022-06-29T15:43:14.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.319+0000 7ff0f885f700 1 -- 172.21.15.47:0/2169940810 shutdown_connections 2022-06-29T15:43:14.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.319+0000 7ff0f885f700 1 --2- 172.21.15.47:0/2169940810 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff0f40980c0 0x7ff0f4097730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:14.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.319+0000 7ff0f885f700 1 --2- 172.21.15.47:0/2169940810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0f4097ce0 0x7ff0f4097240 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:14.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.319+0000 7ff0f885f700 1 --2- 172.21.15.47:0/2169940810 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff0d0053950 0x7ff0d0055e10 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:14.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.319+0000 7ff0f885f700 1 --2- 172.21.15.47:0/2169940810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0f4060b30 0x7ff0f40b0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:14.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.319+0000 7ff0f885f700 1 -- 172.21.15.47:0/2169940810 >> 172.21.15.47:0/2169940810 conn(0x7ff0f405f0b0 msgr2=0x7ff0f40671d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:14.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.319+0000 7ff0f885f700 1 -- 172.21.15.47:0/2169940810 shutdown_connections 2022-06-29T15:43:14.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.319+0000 7ff0f885f700 1 -- 172.21.15.47:0/2169940810 wait complete. 2022-06-29T15:43:14.338 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-29T15:43:14.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.447+0000 7f403d070700 1 Processor -- start 2022-06-29T15:43:14.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.451+0000 7f403d070700 1 -- start start 2022-06-29T15:43:14.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.451+0000 7f403d070700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f40380b0fe0 0x7f4038061a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:14.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.451+0000 7f403d070700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f40380b0700 0x7f40381436c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:14.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.451+0000 7f403d070700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f403813db90 con 0x7f4038061670 2022-06-29T15:43:14.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.451+0000 7f403d070700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f403813fed0 con 0x7f40380b0fe0 2022-06-29T15:43:14.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.451+0000 7f403d070700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40380af0a0 con 0x7f40380b0700 2022-06-29T15:43:14.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.451+0000 7f4036d9d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4038061670 0x7f403813d480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:32836/0 (socket says 172.21.15.47:32836) 2022-06-29T15:43:14.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.451+0000 7f4036d9d700 1 -- 172.21.15.47:0/3478273696 learned_addr learned my addr 172.21.15.47:0/3478273696 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:43:14.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.451+0000 7f403659c700 1 --2- 172.21.15.47:0/3478273696 >> v2:172.21.15.47:3300/0 conn(0x7f40380b0fe0 0x7f4038061a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:14.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.451+0000 7f403759e700 1 --2- 172.21.15.47:0/3478273696 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f40380b0700 0x7f40381436c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:14.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.451+0000 7f4035d9b700 1 -- 172.21.15.47:0/3478273696 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2172099621 0 0) 0x7f403813db90 con 0x7f4038061670 2022-06-29T15:43:14.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.451+0000 7f403659c700 1 -- 172.21.15.47:0/3478273696 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f40380b0700 msgr2=0x7f40381436c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:43:14.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.451+0000 7f403659c700 1 --2- 172.21.15.47:0/3478273696 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f40380b0700 0x7f40381436c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:14.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.451+0000 7f403659c700 1 -- 172.21.15.47:0/3478273696 >> v1:172.21.15.47:6789/0 conn(0x7f4038061670 legacy=0x7f403813d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:14.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.451+0000 7f403659c700 1 -- 172.21.15.47:0/3478273696 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40380af340 con 0x7f40380b0fe0 2022-06-29T15:43:14.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.451+0000 7f403659c700 1 --2- 172.21.15.47:0/3478273696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40380b0fe0 0x7f4038061a50 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7f402c002f70 tx=0x7f402c006df0).ready entity=mon.0 client_cookie=12d4da1e5d24fd49 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:14.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.451+0000 7f4035d9b700 1 -- 172.21.15.47:0/3478273696 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f402c00d570 con 0x7f40380b0fe0 2022-06-29T15:43:14.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.451+0000 7f4035d9b700 1 -- 172.21.15.47:0/3478273696 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f402c000e90 con 0x7f40380b0fe0 2022-06-29T15:43:14.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.451+0000 7f4035d9b700 1 -- 172.21.15.47:0/3478273696 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f402c00d8c0 con 0x7f40380b0fe0 2022-06-29T15:43:14.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.455+0000 7f403d070700 1 -- 172.21.15.47:0/3478273696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40380b0fe0 msgr2=0x7f4038061a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:14.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.455+0000 7f403d070700 1 --2- 172.21.15.47:0/3478273696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40380b0fe0 0x7f4038061a50 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7f402c002f70 tx=0x7f402c006df0).stop 2022-06-29T15:43:14.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.455+0000 7f403d070700 1 -- 172.21.15.47:0/3478273696 shutdown_connections 2022-06-29T15:43:14.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.455+0000 7f403d070700 1 --2- 172.21.15.47:0/3478273696 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f40380b0700 0x7f40381436c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:14.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.455+0000 7f403d070700 1 --2- 172.21.15.47:0/3478273696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40380b0fe0 0x7f4038061a50 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:14.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.455+0000 7f403d070700 1 -- 172.21.15.47:0/3478273696 >> 172.21.15.47:0/3478273696 conn(0x7f4038063800 msgr2=0x7f4038063be0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:14.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.455+0000 7f403d070700 1 -- 172.21.15.47:0/3478273696 shutdown_connections 2022-06-29T15:43:14.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.455+0000 7f403d070700 1 -- 172.21.15.47:0/3478273696 wait complete. 2022-06-29T15:43:14.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.455+0000 7f403d070700 1 Processor -- start 2022-06-29T15:43:14.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.455+0000 7f403d070700 1 -- start start 2022-06-29T15:43:14.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.459+0000 7f403d070700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4038061670 0x7f4038145b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:14.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.459+0000 7f403d070700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f40380b0700 0x7f4038064490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:14.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.459+0000 7f403d070700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40380b0fe0 0x7f403814f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:14.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.459+0000 7f403d070700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4038146050 con 0x7f4038061670 2022-06-29T15:43:14.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.459+0000 7f4036d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4038061670 0x7f4038145b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:14.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.459+0000 7f4036d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4038061670 0x7f4038145b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47288/0 (socket says 172.21.15.47:47288) 2022-06-29T15:43:14.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.459+0000 7f403d070700 1 -- 172.21.15.47:0/3085784791 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f403814fb10 con 0x7f40380b0fe0 2022-06-29T15:43:14.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.459+0000 7f4036d9d700 1 -- 172.21.15.47:0/3085784791 learned_addr learned my addr 172.21.15.47:0/3085784791 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:14.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.459+0000 7f403d070700 1 -- 172.21.15.47:0/3085784791 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4038145420 con 0x7f40380b0700 2022-06-29T15:43:14.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.459+0000 7f403759e700 1 --2- 172.21.15.47:0/3085784791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40380b0fe0 0x7f403814f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:14.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.459+0000 7f403659c700 1 --2- 172.21.15.47:0/3085784791 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f40380b0700 0x7f4038064490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:14.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.459+0000 7f4036d9d700 1 -- 172.21.15.47:0/3085784791 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f40380b0700 msgr2=0x7f4038064490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:14.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.459+0000 7f4036d9d700 1 --2- 172.21.15.47:0/3085784791 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f40380b0700 0x7f4038064490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:14.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.459+0000 7f4036d9d700 1 -- 172.21.15.47:0/3085784791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40380b0fe0 msgr2=0x7f403814f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:14.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.459+0000 7f4036d9d700 1 --2- 172.21.15.47:0/3085784791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40380b0fe0 0x7f403814f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:14.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.459+0000 7f4036d9d700 1 -- 172.21.15.47:0/3085784791 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f402c00b180 con 0x7f4038061670 2022-06-29T15:43:14.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.459+0000 7f403759e700 1 --2- 172.21.15.47:0/3085784791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40380b0fe0 0x7f403814f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:43:14.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.459+0000 7f4036d9d700 1 --2- 172.21.15.47:0/3085784791 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4038061670 0x7f4038145b60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7f4028000ed0 tx=0x7f4028007aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:14.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.459+0000 7f4027fff700 1 -- 172.21.15.47:0/3085784791 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40280070d0 con 0x7f4038061670 2022-06-29T15:43:14.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.459+0000 7f4027fff700 1 -- 172.21.15.47:0/3085784791 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4028007230 con 0x7f4038061670 2022-06-29T15:43:14.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.459+0000 7f403d070700 1 -- 172.21.15.47:0/3085784791 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40381456c0 con 0x7f4038061670 2022-06-29T15:43:14.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.459+0000 7f4027fff700 1 -- 172.21.15.47:0/3085784791 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4028005590 con 0x7f4038061670 2022-06-29T15:43:14.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.459+0000 7f403d070700 1 -- 172.21.15.47:0/3085784791 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4038064980 con 0x7f4038061670 2022-06-29T15:43:14.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.463+0000 7f403d070700 1 -- 172.21.15.47:0/3085784791 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40380573d0 con 0x7f4038061670 2022-06-29T15:43:14.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.463+0000 7f4027fff700 1 -- 172.21.15.47:0/3085784791 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f402801a060 con 0x7f4038061670 2022-06-29T15:43:14.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.463+0000 7f4027fff700 1 --2- 172.21.15.47:0/3085784791 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4014053b00 0x7f4014055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:14.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.463+0000 7f4027fff700 1 -- 172.21.15.47:0/3085784791 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f402806b880 con 0x7f4038061670 2022-06-29T15:43:14.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.463+0000 7f403659c700 1 --2- 172.21.15.47:0/3085784791 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4014053b00 0x7f4014055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:14.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.463+0000 7f403659c700 1 --2- 172.21.15.47:0/3085784791 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4014053b00 0x7f4014055fc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f402c01a000 tx=0x7f402c014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:14.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.471+0000 7f4027fff700 1 -- 172.21.15.47:0/3085784791 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4028042530 con 0x7f4038061670 2022-06-29T15:43:14.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.599+0000 7f403d070700 1 -- 172.21.15.47:0/3085784791 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f40380b3f80 con 0x7f4038061670 2022-06-29T15:43:14.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.599+0000 7f4027fff700 1 -- 172.21.15.47:0/3085784791 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) v1 ==== 104+0+10 (secure 0 0 0) 0x7f40280180a0 con 0x7f4038061670 2022-06-29T15:43:14.606 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:43:14.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.599+0000 7f403d070700 1 -- 172.21.15.47:0/3085784791 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4014053b00 msgr2=0x7f4014055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:14.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.599+0000 7f403d070700 1 --2- 172.21.15.47:0/3085784791 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4014053b00 0x7f4014055fc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f402c01a000 tx=0x7f402c014000).stop 2022-06-29T15:43:14.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.599+0000 7f403d070700 1 -- 172.21.15.47:0/3085784791 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4038061670 msgr2=0x7f4038145b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:14.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.599+0000 7f403d070700 1 --2- 172.21.15.47:0/3085784791 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4038061670 0x7f4038145b60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7f4028000ed0 tx=0x7f4028007aa0).stop 2022-06-29T15:43:14.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.599+0000 7f403d070700 1 -- 172.21.15.47:0/3085784791 shutdown_connections 2022-06-29T15:43:14.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.599+0000 7f403d070700 1 --2- 172.21.15.47:0/3085784791 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4014053b00 0x7f4014055fc0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:14.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.599+0000 7f403d070700 1 --2- 172.21.15.47:0/3085784791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40380b0fe0 0x7f403814f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:14.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.599+0000 7f403d070700 1 --2- 172.21.15.47:0/3085784791 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f40380b0700 0x7f4038064490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:14.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.599+0000 7f403d070700 1 --2- 172.21.15.47:0/3085784791 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4038061670 0x7f4038145b60 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:14.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.599+0000 7f403d070700 1 -- 172.21.15.47:0/3085784791 >> 172.21.15.47:0/3085784791 conn(0x7f4038063800 msgr2=0x7f4038140810 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:14.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.599+0000 7f403d070700 1 -- 172.21.15.47:0/3085784791 shutdown_connections 2022-06-29T15:43:14.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.599+0000 7f403d070700 1 -- 172.21.15.47:0/3085784791 wait complete. 2022-06-29T15:43:14.618 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-29T15:43:14.734 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.727+0000 7f2a6e6f3700 1 Processor -- start 2022-06-29T15:43:14.734 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.727+0000 7f2a6e6f3700 1 -- start start 2022-06-29T15:43:14.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.727+0000 7f2a6e6f3700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2a680b6d20 0x7f2a680b1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:14.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.727+0000 7f2a6e6f3700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a680b1050 con 0x7f2a681467a0 2022-06-29T15:43:14.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.727+0000 7f2a6e6f3700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a680b7100 con 0x7f2a68140c10 2022-06-29T15:43:14.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.727+0000 7f2a6e6f3700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a680b23c0 con 0x7f2a680b6d20 2022-06-29T15:43:14.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.727+0000 7f2a6cef0700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f2a681467a0 0x7f2a68140500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:32850/0 (socket says 172.21.15.47:32850) 2022-06-29T15:43:14.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.727+0000 7f2a6cef0700 1 -- 172.21.15.47:0/1898409567 learned_addr learned my addr 172.21.15.47:0/1898409567 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:43:14.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.727+0000 7f2a6d6f1700 1 --2- 172.21.15.47:0/1898409567 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2a680b6d20 0x7f2a680b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:14.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.727+0000 7f2a57fff700 1 -- 172.21.15.47:0/1898409567 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1084739789 0 0) 0x7f2a680b1050 con 0x7f2a681467a0 2022-06-29T15:43:14.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.727+0000 7f2a57fff700 1 -- 172.21.15.47:0/1898409567 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a50003260 con 0x7f2a681467a0 2022-06-29T15:43:14.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.731+0000 7f2a57fff700 1 -- 172.21.15.47:0/1898409567 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1053707623 0 0) 0x7f2a680b7100 con 0x7f2a68140c10 2022-06-29T15:43:14.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.731+0000 7f2a57fff700 1 -- 172.21.15.47:0/1898409567 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a680b1050 con 0x7f2a68140c10 2022-06-29T15:43:14.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.731+0000 7f2a6d6f1700 1 -- 172.21.15.47:0/1898409567 >> v1:172.21.15.64:6789/0 conn(0x7f2a68140c10 legacy=0x7f2a6814db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:14.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.731+0000 7f2a57fff700 1 -- 172.21.15.47:0/1898409567 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1321053909 0 0) 0x7f2a50003260 con 0x7f2a681467a0 2022-06-29T15:43:14.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.731+0000 7f2a6d6f1700 1 -- 172.21.15.47:0/1898409567 >> v1:172.21.15.47:6789/0 conn(0x7f2a681467a0 legacy=0x7f2a68140500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:14.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.731+0000 7f2a6d6f1700 1 -- 172.21.15.47:0/1898409567 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a680632f0 con 0x7f2a680b6d20 2022-06-29T15:43:14.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.731+0000 7f2a6d6f1700 1 --2- 172.21.15.47:0/1898409567 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2a680b6d20 0x7f2a680b1480 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f2a58001110 tx=0x7f2a64006df0).ready entity=mon.2 client_cookie=45e9071e7401f8a0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:14.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.731+0000 7f2a57fff700 1 -- 172.21.15.47:0/1898409567 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2a58001660 con 0x7f2a680b6d20 2022-06-29T15:43:14.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.731+0000 7f2a57fff700 1 -- 172.21.15.47:0/1898409567 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a64000e90 con 0x7f2a680b6d20 2022-06-29T15:43:14.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.731+0000 7f2a57fff700 1 -- 172.21.15.47:0/1898409567 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2a64012460 con 0x7f2a680b6d20 2022-06-29T15:43:14.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.731+0000 7f2a6e6f3700 1 -- 172.21.15.47:0/1898409567 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2a680b6d20 msgr2=0x7f2a680b1480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:14.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.731+0000 7f2a6e6f3700 1 --2- 172.21.15.47:0/1898409567 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2a680b6d20 0x7f2a680b1480 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f2a58001110 tx=0x7f2a64006df0).stop 2022-06-29T15:43:14.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.731+0000 7f2a6e6f3700 1 -- 172.21.15.47:0/1898409567 shutdown_connections 2022-06-29T15:43:14.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.731+0000 7f2a6e6f3700 1 --2- 172.21.15.47:0/1898409567 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2a680b6d20 0x7f2a680b1480 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:14.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.731+0000 7f2a6e6f3700 1 -- 172.21.15.47:0/1898409567 >> 172.21.15.47:0/1898409567 conn(0x7f2a68063670 msgr2=0x7f2a68063a50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:14.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.731+0000 7f2a6e6f3700 1 -- 172.21.15.47:0/1898409567 shutdown_connections 2022-06-29T15:43:14.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.731+0000 7f2a6e6f3700 1 -- 172.21.15.47:0/1898409567 wait complete. 2022-06-29T15:43:14.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.731+0000 7f2a6e6f3700 1 Processor -- start 2022-06-29T15:43:14.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.731+0000 7f2a6e6f3700 1 -- start start 2022-06-29T15:43:14.748 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.735+0000 7f2a6e6f3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a680b6d20 0x7f2a68140430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:14.748 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.735+0000 7f2a6e6f3700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2a68140c10 0x7f2a6813b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:14.748 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.735+0000 7f2a6e6f3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a681467a0 0x7f2a6813bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:14.748 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.735+0000 7f2a6e6f3700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a680663a0 con 0x7f2a681467a0 2022-06-29T15:43:14.748 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.735+0000 7f2a6e6f3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a68066100 con 0x7f2a680b6d20 2022-06-29T15:43:14.749 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.735+0000 7f2a6e6f3700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a680ac830 con 0x7f2a68140c10 2022-06-29T15:43:14.749 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.735+0000 7f2a6def2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a681467a0 0x7f2a6813bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:14.749 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.735+0000 7f2a6def2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a681467a0 0x7f2a6813bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47306/0 (socket says 172.21.15.47:47306) 2022-06-29T15:43:14.749 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.735+0000 7f2a6def2700 1 -- 172.21.15.47:0/2094044160 learned_addr learned my addr 172.21.15.47:0/2094044160 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:14.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.735+0000 7f2a6d6f1700 1 --2- 172.21.15.47:0/2094044160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a680b6d20 0x7f2a68140430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:14.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.735+0000 7f2a6cef0700 1 --2- 172.21.15.47:0/2094044160 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2a68140c10 0x7f2a6813b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:14.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.735+0000 7f2a6def2700 1 -- 172.21.15.47:0/2094044160 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2a68140c10 msgr2=0x7f2a6813b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:14.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.735+0000 7f2a6def2700 1 --2- 172.21.15.47:0/2094044160 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2a68140c10 0x7f2a6813b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:14.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.735+0000 7f2a6def2700 1 -- 172.21.15.47:0/2094044160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a680b6d20 msgr2=0x7f2a68140430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:14.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.735+0000 7f2a6def2700 1 --2- 172.21.15.47:0/2094044160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a680b6d20 0x7f2a68140430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:14.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.735+0000 7f2a6def2700 1 -- 172.21.15.47:0/2094044160 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a6400b180 con 0x7f2a681467a0 2022-06-29T15:43:14.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.735+0000 7f2a6cef0700 1 --2- 172.21.15.47:0/2094044160 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2a68140c10 0x7f2a6813b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:43:14.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.735+0000 7f2a6def2700 1 --2- 172.21.15.47:0/2094044160 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a681467a0 0x7f2a6813bbd0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7f2a60001490 tx=0x7f2a60007da0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:14.752 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.735+0000 7f2a567fc700 1 -- 172.21.15.47:0/2094044160 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2a60002310 con 0x7f2a681467a0 2022-06-29T15:43:14.752 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.735+0000 7f2a567fc700 1 -- 172.21.15.47:0/2094044160 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a60002470 con 0x7f2a681467a0 2022-06-29T15:43:14.752 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.735+0000 7f2a6e6f3700 1 -- 172.21.15.47:0/2094044160 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a680acad0 con 0x7f2a681467a0 2022-06-29T15:43:14.753 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.735+0000 7f2a567fc700 1 -- 172.21.15.47:0/2094044160 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2a60001840 con 0x7f2a681467a0 2022-06-29T15:43:14.753 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.735+0000 7f2a6e6f3700 1 -- 172.21.15.47:0/2094044160 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a680ac460 con 0x7f2a681467a0 2022-06-29T15:43:14.753 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.735+0000 7f2a6e6f3700 1 -- 172.21.15.47:0/2094044160 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a680acc60 con 0x7f2a681467a0 2022-06-29T15:43:14.753 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.739+0000 7f2a567fc700 1 -- 172.21.15.47:0/2094044160 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2a60010030 con 0x7f2a681467a0 2022-06-29T15:43:14.754 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.739+0000 7f2a567fc700 1 --2- 172.21.15.47:0/2094044160 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2a48053a70 0x7f2a48055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:14.754 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.739+0000 7f2a567fc700 1 -- 172.21.15.47:0/2094044160 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f2a6006c090 con 0x7f2a681467a0 2022-06-29T15:43:14.754 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.739+0000 7f2a6d6f1700 1 --2- 172.21.15.47:0/2094044160 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2a48053a70 0x7f2a48055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:14.755 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.739+0000 7f2a6d6f1700 1 --2- 172.21.15.47:0/2094044160 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2a48053a70 0x7f2a48055f30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f2a6401a000 tx=0x7f2a64014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:14.755 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.743+0000 7f2a567fc700 1 -- 172.21.15.47:0/2094044160 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2a60042d40 con 0x7f2a681467a0 2022-06-29T15:43:14.877 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.871+0000 7f2a6e6f3700 1 -- 172.21.15.47:0/2094044160 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2a680573d0 con 0x7f2a681467a0 2022-06-29T15:43:14.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.871+0000 7f2a567fc700 1 -- 172.21.15.47:0/2094044160 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) v1 ==== 100+0+10 (secure 0 0 0) 0x7f2a60046360 con 0x7f2a681467a0 2022-06-29T15:43:14.879 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:43:14.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.875+0000 7f2a6e6f3700 1 -- 172.21.15.47:0/2094044160 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2a48053a70 msgr2=0x7f2a48055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:14.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.875+0000 7f2a6e6f3700 1 --2- 172.21.15.47:0/2094044160 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2a48053a70 0x7f2a48055f30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f2a6401a000 tx=0x7f2a64014000).stop 2022-06-29T15:43:14.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.875+0000 7f2a6e6f3700 1 -- 172.21.15.47:0/2094044160 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a681467a0 msgr2=0x7f2a6813bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:14.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.875+0000 7f2a6e6f3700 1 --2- 172.21.15.47:0/2094044160 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a681467a0 0x7f2a6813bbd0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7f2a60001490 tx=0x7f2a60007da0).stop 2022-06-29T15:43:14.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.875+0000 7f2a6e6f3700 1 -- 172.21.15.47:0/2094044160 shutdown_connections 2022-06-29T15:43:14.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.875+0000 7f2a6e6f3700 1 --2- 172.21.15.47:0/2094044160 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a681467a0 0x7f2a6813bbd0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:14.883 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.875+0000 7f2a6e6f3700 1 --2- 172.21.15.47:0/2094044160 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2a68140c10 0x7f2a6813b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:14.883 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.875+0000 7f2a6e6f3700 1 --2- 172.21.15.47:0/2094044160 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2a48053a70 0x7f2a48055f30 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:14.883 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.875+0000 7f2a6e6f3700 1 --2- 172.21.15.47:0/2094044160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a680b6d20 0x7f2a68140430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:14.883 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.875+0000 7f2a6e6f3700 1 -- 172.21.15.47:0/2094044160 >> 172.21.15.47:0/2094044160 conn(0x7f2a68063670 msgr2=0x7f2a681411d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:14.884 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.875+0000 7f2a6e6f3700 1 -- 172.21.15.47:0/2094044160 shutdown_connections 2022-06-29T15:43:14.884 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.875+0000 7f2a6e6f3700 1 -- 172.21.15.47:0/2094044160 wait complete. 2022-06-29T15:43:14.892 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-29T15:43:14.893 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:43:14.993 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.987+0000 7f72a9b9e700 1 Processor -- start 2022-06-29T15:43:14.993 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.987+0000 7f72a9b9e700 1 -- start start 2022-06-29T15:43:14.995 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.987+0000 7f72a9b9e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f72a4060eb0 0x7f72a40b1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:14.996 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.987+0000 7f72a9b9e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72a40a9120 con 0x7f72a40b5c60 2022-06-29T15:43:14.996 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.987+0000 7f72a9b9e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72a40b0b80 con 0x7f72a4145090 2022-06-29T15:43:14.996 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.987+0000 7f72a9b9e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72a40a89e0 con 0x7f72a4060eb0 2022-06-29T15:43:14.996 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.987+0000 7f72a3fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f72a4060eb0 0x7f72a40b1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:14.997 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.987+0000 7f72a8b9c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f72a40b5c60 0x7f72a4144980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:32864/0 (socket says 172.21.15.47:32864) 2022-06-29T15:43:14.997 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.987+0000 7f72a8b9c700 1 -- 172.21.15.47:0/461776628 learned_addr learned my addr 172.21.15.47:0/461776628 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:43:14.997 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.987+0000 7f72a37fe700 1 -- 172.21.15.47:0/461776628 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3953008056 0 0) 0x7f72a40a9120 con 0x7f72a40b5c60 2022-06-29T15:43:14.997 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.987+0000 7f72a37fe700 1 -- 172.21.15.47:0/461776628 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7288003260 con 0x7f72a40b5c60 2022-06-29T15:43:14.997 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.987+0000 7f72a3fff700 1 -- 172.21.15.47:0/461776628 >> v1:172.21.15.64:6789/0 conn(0x7f72a4145090 legacy=0x7f72a40b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:14.998 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.987+0000 7f72a37fe700 1 -- 172.21.15.47:0/461776628 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2014898925 0 0) 0x7f7288003260 con 0x7f72a40b5c60 2022-06-29T15:43:14.998 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.987+0000 7f72a3fff700 1 -- 172.21.15.47:0/461776628 >> v1:172.21.15.47:6789/0 conn(0x7f72a40b5c60 legacy=0x7f72a4144980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:14.998 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.987+0000 7f72a3fff700 1 -- 172.21.15.47:0/461776628 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a40a8c80 con 0x7f72a4060eb0 2022-06-29T15:43:14.998 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.987+0000 7f72a3fff700 1 --2- 172.21.15.47:0/461776628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72a4060eb0 0x7f72a40b1e30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f728c001980 tx=0x7f729c0091d0).ready entity=mon.0 client_cookie=6cadea3de92527a5 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:14.999 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.987+0000 7f72a37fe700 1 -- 172.21.15.47:0/461776628 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f728c0017e0 con 0x7f72a4060eb0 2022-06-29T15:43:14.999 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.987+0000 7f72a37fe700 1 -- 172.21.15.47:0/461776628 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f729c00b580 con 0x7f72a4060eb0 2022-06-29T15:43:14.999 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.987+0000 7f72a37fe700 1 -- 172.21.15.47:0/461776628 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f729c00b6f0 con 0x7f72a4060eb0 2022-06-29T15:43:15.002 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.991+0000 7f72a9b9e700 1 -- 172.21.15.47:0/461776628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72a4060eb0 msgr2=0x7f72a40b1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:15.003 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.991+0000 7f72a9b9e700 1 --2- 172.21.15.47:0/461776628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72a4060eb0 0x7f72a40b1e30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f728c001980 tx=0x7f729c0091d0).stop 2022-06-29T15:43:15.003 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.991+0000 7f72a9b9e700 1 -- 172.21.15.47:0/461776628 shutdown_connections 2022-06-29T15:43:15.003 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.991+0000 7f72a9b9e700 1 --2- 172.21.15.47:0/461776628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72a4060eb0 0x7f72a40b1e30 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:15.003 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.991+0000 7f72a9b9e700 1 -- 172.21.15.47:0/461776628 >> 172.21.15.47:0/461776628 conn(0x7f72a4062560 msgr2=0x7f72a4062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:15.004 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.991+0000 7f72a9b9e700 1 -- 172.21.15.47:0/461776628 shutdown_connections 2022-06-29T15:43:15.004 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.991+0000 7f72a9b9e700 1 -- 172.21.15.47:0/461776628 wait complete. 2022-06-29T15:43:15.004 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.991+0000 7f72a9b9e700 1 Processor -- start 2022-06-29T15:43:15.004 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.991+0000 7f72a9b9e700 1 -- start start 2022-06-29T15:43:15.004 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.991+0000 7f72a9b9e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72a4060eb0 0x7f72a413dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:15.005 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.991+0000 7f72a9b9e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72a40b5c60 0x7f72a414ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:15.005 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.991+0000 7f72a9b9e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f72a4145090 0x7f72a40ade50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:15.005 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.991+0000 7f72a9b9e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a40acdc0 con 0x7f72a40b5c60 2022-06-29T15:43:15.005 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.991+0000 7f72a9b9e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a414bd00 con 0x7f72a4060eb0 2022-06-29T15:43:15.006 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.991+0000 7f72a9b9e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72a414c760 con 0x7f72a4145090 2022-06-29T15:43:15.006 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.991+0000 7f72a3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72a40b5c60 0x7f72a414ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:15.006 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.991+0000 7f72a8b9c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72a4060eb0 0x7f72a413dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:15.006 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.991+0000 7f72a3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72a40b5c60 0x7f72a414ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47318/0 (socket says 172.21.15.47:47318) 2022-06-29T15:43:15.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.991+0000 7f72a3fff700 1 -- 172.21.15.47:0/2729162575 learned_addr learned my addr 172.21.15.47:0/2729162575 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:15.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.991+0000 7f72a939d700 1 --2- 172.21.15.47:0/2729162575 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f72a4145090 0x7f72a40ade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:15.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.991+0000 7f72a3fff700 1 -- 172.21.15.47:0/2729162575 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f72a4145090 msgr2=0x7f72a40ade50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:15.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.991+0000 7f72a3fff700 1 --2- 172.21.15.47:0/2729162575 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f72a4145090 0x7f72a40ade50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:15.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.991+0000 7f72a3fff700 1 -- 172.21.15.47:0/2729162575 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72a4060eb0 msgr2=0x7f72a413dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:15.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.991+0000 7f72a3fff700 1 --2- 172.21.15.47:0/2729162575 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72a4060eb0 0x7f72a413dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:15.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.991+0000 7f72a3fff700 1 -- 172.21.15.47:0/2729162575 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f729c009470 con 0x7f72a40b5c60 2022-06-29T15:43:15.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.991+0000 7f72a3fff700 1 --2- 172.21.15.47:0/2729162575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72a40b5c60 0x7f72a414ca00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7f728c001980 tx=0x7f729c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:15.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.991+0000 7f72a1ffb700 1 -- 172.21.15.47:0/2729162575 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f729c00b6f0 con 0x7f72a40b5c60 2022-06-29T15:43:15.009 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.991+0000 7f72a1ffb700 1 -- 172.21.15.47:0/2729162575 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f729c0032d0 con 0x7f72a40b5c60 2022-06-29T15:43:15.009 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.991+0000 7f72a9b9e700 1 -- 172.21.15.47:0/2729162575 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72a413e870 con 0x7f72a40b5c60 2022-06-29T15:43:15.009 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.991+0000 7f72a1ffb700 1 -- 172.21.15.47:0/2729162575 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f729c00be90 con 0x7f72a40b5c60 2022-06-29T15:43:15.009 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.995+0000 7f72a9b9e700 1 -- 172.21.15.47:0/2729162575 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72a414cef0 con 0x7f72a40b5c60 2022-06-29T15:43:15.010 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.995+0000 7f72a9b9e700 1 -- 172.21.15.47:0/2729162575 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72a414d200 con 0x7f72a40b5c60 2022-06-29T15:43:15.010 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.995+0000 7f72a1ffb700 1 -- 172.21.15.47:0/2729162575 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f729c026030 con 0x7f72a40b5c60 2022-06-29T15:43:15.011 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.995+0000 7f72a1ffb700 1 --2- 172.21.15.47:0/2729162575 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f727c053a70 0x7f727c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:15.011 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.995+0000 7f72a8b9c700 1 --2- 172.21.15.47:0/2729162575 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f727c053a70 0x7f727c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:15.011 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.995+0000 7f72a1ffb700 1 -- 172.21.15.47:0/2729162575 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f729c06c810 con 0x7f72a40b5c60 2022-06-29T15:43:15.012 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:14.995+0000 7f72a8b9c700 1 --2- 172.21.15.47:0/2729162575 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f727c053a70 0x7f727c055f30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f728c00d000 tx=0x7f728c0014f0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:15.012 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:15.003+0000 7f72a1ffb700 1 -- 172.21.15.47:0/2729162575 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f729c0435d0 con 0x7f72a40b5c60 2022-06-29T15:43:15.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:15.167+0000 7f72a9b9e700 1 -- 172.21.15.47:0/2729162575 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f72a414bfd0 con 0x7f72a40b5c60 2022-06-29T15:43:15.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:15.167+0000 7f72a1ffb700 1 -- 172.21.15.47:0/2729162575 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+2488 (secure 0 0 0) 0x7f729c0159d0 con 0x7f72a40b5c60 2022-06-29T15:43:15.174 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:43:15.175 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":12,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4896,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1047560169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1047560169},{"type":"v1","addr":"172.21.15.47:6835","nonce":1047560169}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":10},{"gid":4902,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/804808837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":804808837},{"type":"v1","addr":"172.21.15.64:6833","nonce":804808837}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":10}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:43:10.546977+0000","modified":"2022-06-29T15:43:11.549959+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4905},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4905":{"gid":4905,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6837/2219068343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2219068343},{"type":"v1","addr":"172.21.15.47:6837","nonce":2219068343}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2022-06-29T15:43:15.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:15.171+0000 7f72a9b9e700 1 -- 172.21.15.47:0/2729162575 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f727c053a70 msgr2=0x7f727c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:15.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:15.171+0000 7f72a9b9e700 1 --2- 172.21.15.47:0/2729162575 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f727c053a70 0x7f727c055f30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f728c00d000 tx=0x7f728c0014f0).stop 2022-06-29T15:43:15.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:15.171+0000 7f72a9b9e700 1 -- 172.21.15.47:0/2729162575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72a40b5c60 msgr2=0x7f72a414ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:15.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:15.171+0000 7f72a9b9e700 1 --2- 172.21.15.47:0/2729162575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72a40b5c60 0x7f72a414ca00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7f728c001980 tx=0x7f729c016000).stop 2022-06-29T15:43:15.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:15.171+0000 7f72a9b9e700 1 -- 172.21.15.47:0/2729162575 shutdown_connections 2022-06-29T15:43:15.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:15.171+0000 7f72a9b9e700 1 --2- 172.21.15.47:0/2729162575 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f72a4145090 0x7f72a40ade50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:15.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:15.171+0000 7f72a9b9e700 1 --2- 172.21.15.47:0/2729162575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72a40b5c60 0x7f72a414ca00 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:15.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:15.171+0000 7f72a9b9e700 1 --2- 172.21.15.47:0/2729162575 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f727c053a70 0x7f727c055f30 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:15.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:15.171+0000 7f72a9b9e700 1 --2- 172.21.15.47:0/2729162575 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72a4060eb0 0x7f72a413dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:15.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:15.171+0000 7f72a9b9e700 1 -- 172.21.15.47:0/2729162575 >> 172.21.15.47:0/2729162575 conn(0x7f72a4062560 msgr2=0x7f72a40b1020 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:15.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:15.171+0000 7f72a9b9e700 1 -- 172.21.15.47:0/2729162575 shutdown_connections 2022-06-29T15:43:15.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:15.171+0000 7f72a9b9e700 1 -- 172.21.15.47:0/2729162575 wait complete. 2022-06-29T15:43:15.181 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 12 2022-06-29T15:43:15.496 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 12, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-29T15:43:10.546977+0000', 'modified': '2022-06-29T15:43:11.549959+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4905}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4905': {'gid': 4905, 'name': 'a', 'rank': 0, 'incarnation': 11, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.47:6837/2219068343', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6836', 'nonce': 2219068343}, {'type': 'v1', 'addr': '172.21.15.47:6837', 'nonce': 2219068343}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-29T15:43:15.496 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-29T15:43:15.497 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-06-29T15:43:15.612 INFO:teuthology.orchestra.run.smithi047.stdout:{"cluster_fsid":"9a68894a-53ce-400e-838b-14eab91b834c","whoami":0,"id":4905,"want_state":"up:active","state":"up:active","rank_uptime":5.048315669,"mdsmap_epoch":12,"osdmap_epoch":36,"osdmap_epoch_barrier":35,"uptime":11.273297152} 2022-06-29T15:43:15.613 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9a68894a-53ce-400e-838b-14eab91b834c", "id": 4905, "mdsmap_epoch": 12, "osdmap_epoch": 36, "osdmap_epoch_barrier": 35, "rank_uptime": 5.048315669, "state": "up:active", "uptime": 11.273297152, "want_state": "up:active", "whoami": 0 } 2022-06-29T15:43:15.613 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-29T15:43:15.614 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-06-29T15:43:15.614 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi099.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-06-29T15:43:15.615 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:43:15.615 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:43:15.624 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2022-06-29T15:43:15.684 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:43:15.685 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:43:15.698 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:43:15.699 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:43:15.700 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:43:15.700 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2022-06-29T15:43:15.707 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-29T15:43:15.707 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:43:15.710 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2022-06-29T15:43:15.725 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:43:15.726 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:43:15.736 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:43:15.737 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:43:15.739 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:43:15.740 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2022-06-29T15:43:15.784 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:2022-06-29T15:43:15.777+0000 7fbaba20e080 -1 init, newargv = 0x558c0bc7e800 newargc=9ceph-fuse[12295 2022-06-29T15:43:15.784 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:]: starting ceph client 2022-06-29T15:43:15.797 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[12295]: starting fuse 2022-06-29T15:43:16.891 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2022-06-29T15:43:16.909 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:43:16.910 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:43:16.966 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:43:16.967 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:43:16.968 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:43:16.968 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2022-06-29T15:43:16.975 INFO:teuthology.orchestra.run.smithi099.stdout:52 2022-06-29T15:43:16.975 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-29T15:43:16.976 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:43:16.976 DEBUG:teuthology.orchestra.run.smithi099:> sudo python3 -c ' 2022-06-29T15:43:16.977 DEBUG:teuthology.orchestra.run.smithi099:> import glob 2022-06-29T15:43:16.977 DEBUG:teuthology.orchestra.run.smithi099:> import re 2022-06-29T15:43:16.978 DEBUG:teuthology.orchestra.run.smithi099:> import os 2022-06-29T15:43:16.978 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2022-06-29T15:43:16.979 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:43:16.979 DEBUG:teuthology.orchestra.run.smithi099:> def find_socket(client_name): 2022-06-29T15:43:16.979 DEBUG:teuthology.orchestra.run.smithi099:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-06-29T15:43:16.980 DEBUG:teuthology.orchestra.run.smithi099:> files = glob.glob(asok_path) 2022-06-29T15:43:16.980 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:43:16.981 DEBUG:teuthology.orchestra.run.smithi099:> # Given a non-glob path, it better be there 2022-06-29T15:43:16.981 DEBUG:teuthology.orchestra.run.smithi099:> if "*" not in asok_path: 2022-06-29T15:43:16.981 DEBUG:teuthology.orchestra.run.smithi099:> assert(len(files) == 1) 2022-06-29T15:43:16.982 DEBUG:teuthology.orchestra.run.smithi099:> return files[0] 2022-06-29T15:43:16.982 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:43:16.983 DEBUG:teuthology.orchestra.run.smithi099:> for f in files: 2022-06-29T15:43:16.983 DEBUG:teuthology.orchestra.run.smithi099:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-29T15:43:16.984 DEBUG:teuthology.orchestra.run.smithi099:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-29T15:43:16.984 DEBUG:teuthology.orchestra.run.smithi099:> return f 2022-06-29T15:43:16.984 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-29T15:43:16.985 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:43:16.985 DEBUG:teuthology.orchestra.run.smithi099:> print(find_socket("client.0")) 2022-06-29T15:43:16.986 DEBUG:teuthology.orchestra.run.smithi099:> ' 2022-06-29T15:43:17.032 INFO:teuthology.orchestra.run.smithi099.stdout:/var/run/ceph/ceph-client.0.12295.asok 2022-06-29T15:43:17.037 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.12295.asok 2022-06-29T15:43:17.037 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:43:17.038 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.12295.asok status 2022-06-29T15:43:17.126 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2022-06-29T15:43:17.126 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": { 2022-06-29T15:43:17.127 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_sha1": "f25ffac72e5a9e63860ff7deb6767a11f0559a16", 2022-06-29T15:43:17.127 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 15.2.16-304-gf25ffac7 (f25ffac72e5a9e63860ff7deb6767a11f0559a16) octopus (stable)", 2022-06-29T15:43:17.127 INFO:teuthology.orchestra.run.smithi099.stdout: "entity_id": "0", 2022-06-29T15:43:17.127 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": "smithi099", 2022-06-29T15:43:17.127 INFO:teuthology.orchestra.run.smithi099.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-06-29T15:43:17.128 INFO:teuthology.orchestra.run.smithi099.stdout: "pid": "12295", 2022-06-29T15:43:17.128 INFO:teuthology.orchestra.run.smithi099.stdout: "root": "/" 2022-06-29T15:43:17.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:43:17.128 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_count": 0, 2022-06-29T15:43:17.129 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_pinned_count": 0, 2022-06-29T15:43:17.129 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 5077, 2022-06-29T15:43:17.129 INFO:teuthology.orchestra.run.smithi099.stdout: "inst": { 2022-06-29T15:43:17.129 INFO:teuthology.orchestra.run.smithi099.stdout: "name": { 2022-06-29T15:43:17.129 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "client", 2022-06-29T15:43:17.130 INFO:teuthology.orchestra.run.smithi099.stdout: "num": 5077 2022-06-29T15:43:17.130 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:43:17.130 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2022-06-29T15:43:17.130 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2022-06-29T15:43:17.131 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:0", 2022-06-29T15:43:17.131 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4240866288 2022-06-29T15:43:17.131 INFO:teuthology.orchestra.run.smithi099.stdout: } 2022-06-29T15:43:17.131 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:43:17.132 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2022-06-29T15:43:17.132 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2022-06-29T15:43:17.132 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:0", 2022-06-29T15:43:17.132 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4240866288 2022-06-29T15:43:17.132 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:43:17.133 INFO:teuthology.orchestra.run.smithi099.stdout: "inst_str": "client.5077 172.21.15.99:0/4240866288", 2022-06-29T15:43:17.133 INFO:teuthology.orchestra.run.smithi099.stdout: "addr_str": "172.21.15.99:0/4240866288", 2022-06-29T15:43:17.133 INFO:teuthology.orchestra.run.smithi099.stdout: "inode_count": 1, 2022-06-29T15:43:17.133 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_epoch": 12, 2022-06-29T15:43:17.134 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch": 36, 2022-06-29T15:43:17.134 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch_barrier": 0, 2022-06-29T15:43:17.134 INFO:teuthology.orchestra.run.smithi099.stdout: "blacklisted": false, 2022-06-29T15:43:17.134 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs" 2022-06-29T15:43:17.135 INFO:teuthology.orchestra.run.smithi099.stdout:} 2022-06-29T15:43:17.136 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:43:17.136 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:43:17.137 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:43:17.144 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2022-06-29T15:43:17.145 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-29T15:43:17.145 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:43:17.146 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:43:17.161 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_min_caps_per_client 2022-06-29T15:43:17.272 INFO:teuthology.orchestra.run.smithi047.stdout:{"mds_min_caps_per_client":"100"} 2022-06-29T15:43:17.273 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_min_caps_per_client": "100" } 2022-06-29T15:43:17.273 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:43:17.273 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:43:17.274 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:43:17.282 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2022-06-29T15:43:17.284 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-29T15:43:17.285 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2022-06-29T15:43:17.285 DEBUG:teuthology.orchestra.run.smithi099:> import sys 2022-06-29T15:43:17.285 DEBUG:teuthology.orchestra.run.smithi099:> import time 2022-06-29T15:43:17.286 DEBUG:teuthology.orchestra.run.smithi099:> import os 2022-06-29T15:43:17.286 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:43:17.287 DEBUG:teuthology.orchestra.run.smithi099:> n = 1000 2022-06-29T15:43:17.287 DEBUG:teuthology.orchestra.run.smithi099:> abs_path = "/home/ubuntu/cephtest/mnt.0/dc-dir/dc-file" 2022-06-29T15:43:17.287 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:43:17.288 DEBUG:teuthology.orchestra.run.smithi099:> if not os.path.exists(os.path.dirname(abs_path)): 2022-06-29T15:43:17.288 DEBUG:teuthology.orchestra.run.smithi099:> os.makedirs(os.path.dirname(abs_path)) 2022-06-29T15:43:17.289 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:43:17.289 DEBUG:teuthology.orchestra.run.smithi099:> for i in range(0, n): 2022-06-29T15:43:17.289 DEBUG:teuthology.orchestra.run.smithi099:> fname = "{0}_{1}".format(abs_path, i) 2022-06-29T15:43:17.290 DEBUG:teuthology.orchestra.run.smithi099:> with open(fname, '"'"'w'"'"') as f: 2022-06-29T15:43:17.290 DEBUG:teuthology.orchestra.run.smithi099:> f.write('"'"'content'"'"') 2022-06-29T15:43:17.290 DEBUG:teuthology.orchestra.run.smithi099:> if True: 2022-06-29T15:43:17.291 DEBUG:teuthology.orchestra.run.smithi099:> f.flush() 2022-06-29T15:43:17.291 DEBUG:teuthology.orchestra.run.smithi099:> os.fsync(f.fileno()) 2022-06-29T15:43:17.292 DEBUG:teuthology.orchestra.run.smithi099:> ' 2022-06-29T15:43:28.411 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:43:28.527 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.519+0000 7f9357570700 1 Processor -- start 2022-06-29T15:43:28.528 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.519+0000 7f9357570700 1 -- start start 2022-06-29T15:43:28.528 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.519+0000 7f9357570700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f93500b2120 0x7f93500b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:28.528 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.519+0000 7f9357570700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f935005f0b0 0x7f935014b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:28.528 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.519+0000 7f9357570700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93500b0740 con 0x7f93500b1260 2022-06-29T15:43:28.529 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.519+0000 7f9357570700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9350147460 con 0x7f935005f0b0 2022-06-29T15:43:28.529 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.519+0000 7f9357570700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93500b0970 con 0x7f93500b2120 2022-06-29T15:43:28.529 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.519+0000 7f9354b0b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f935005f0b0 0x7f935014b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:28.530 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.519+0000 7f9354b0b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f935005f0b0 0x7f935014b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47332/0 (socket says 172.21.15.47:47332) 2022-06-29T15:43:28.530 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.519+0000 7f9354b0b700 1 -- 172.21.15.47:0/2249826036 learned_addr learned my addr 172.21.15.47:0/2249826036 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:28.530 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.519+0000 7f935530c700 1 --2- 172.21.15.47:0/2249826036 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f93500b2120 0x7f93500b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:28.531 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.519+0000 7f9354b0b700 1 -- 172.21.15.47:0/2249826036 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f93500b2120 msgr2=0x7f93500b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:43:28.531 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.519+0000 7f9354b0b700 1 --2- 172.21.15.47:0/2249826036 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f93500b2120 0x7f93500b2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:28.531 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.519+0000 7f9354b0b700 1 -- 172.21.15.47:0/2249826036 >> v1:172.21.15.47:6789/0 conn(0x7f93500b1260 legacy=0x7f9350147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:28.531 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.519+0000 7f9354b0b700 1 -- 172.21.15.47:0/2249826036 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93500b0c10 con 0x7f935005f0b0 2022-06-29T15:43:28.532 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.519+0000 7f935530c700 1 --2- 172.21.15.47:0/2249826036 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f93500b2120 0x7f93500b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:43:28.532 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.519+0000 7f9354b0b700 1 --2- 172.21.15.47:0/2249826036 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f935005f0b0 0x7f935014b520 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f934c002f70 tx=0x7f934c006eb0).ready entity=mon.0 client_cookie=7412fb2344018212 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:28.532 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.519+0000 7f9347fff700 1 -- 172.21.15.47:0/2249826036 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f934c00d5c0 con 0x7f935005f0b0 2022-06-29T15:43:28.532 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.519+0000 7f9347fff700 1 -- 172.21.15.47:0/2249826036 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f934c000e90 con 0x7f935005f0b0 2022-06-29T15:43:28.533 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.519+0000 7f9347fff700 1 -- 172.21.15.47:0/2249826036 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f934c00d950 con 0x7f935005f0b0 2022-06-29T15:43:28.533 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.519+0000 7f9357570700 1 -- 172.21.15.47:0/2249826036 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f935005f0b0 msgr2=0x7f935014b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:28.533 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.519+0000 7f9357570700 1 --2- 172.21.15.47:0/2249826036 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f935005f0b0 0x7f935014b520 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f934c002f70 tx=0x7f934c006eb0).stop 2022-06-29T15:43:28.533 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.519+0000 7f9357570700 1 -- 172.21.15.47:0/2249826036 shutdown_connections 2022-06-29T15:43:28.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.519+0000 7f9357570700 1 --2- 172.21.15.47:0/2249826036 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f935005f0b0 0x7f935014b520 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:28.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.519+0000 7f9357570700 1 --2- 172.21.15.47:0/2249826036 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f93500b2120 0x7f93500b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:28.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.519+0000 7f9357570700 1 -- 172.21.15.47:0/2249826036 >> 172.21.15.47:0/2249826036 conn(0x7f93500626f0 msgr2=0x7f9350062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:28.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.519+0000 7f9357570700 1 -- 172.21.15.47:0/2249826036 shutdown_connections 2022-06-29T15:43:28.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.519+0000 7f9357570700 1 -- 172.21.15.47:0/2249826036 wait complete. 2022-06-29T15:43:28.538 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.523+0000 7f9357570700 1 Processor -- start 2022-06-29T15:43:28.538 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.523+0000 7f9357570700 1 -- start start 2022-06-29T15:43:28.539 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.523+0000 7f9357570700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93500b1260 0x7f9350063550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:28.539 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.523+0000 7f9357570700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93500b2120 0x7f9350159de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:28.539 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.523+0000 7f9357570700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f935015a2d0 0x7f935015a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:28.539 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.523+0000 7f9357570700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9350157220 con 0x7f93500b2120 2022-06-29T15:43:28.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.523+0000 7f9357570700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93500af920 con 0x7f93500b1260 2022-06-29T15:43:28.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.523+0000 7f9357570700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93501439b0 con 0x7f935015a2d0 2022-06-29T15:43:28.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.523+0000 7f9354b0b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93500b2120 0x7f9350159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:28.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.523+0000 7f9354b0b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93500b2120 0x7f9350159de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47338/0 (socket says 172.21.15.47:47338) 2022-06-29T15:43:28.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.523+0000 7f9354b0b700 1 -- 172.21.15.47:0/3275593688 learned_addr learned my addr 172.21.15.47:0/3275593688 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:28.541 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.523+0000 7f935530c700 1 --2- 172.21.15.47:0/3275593688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93500b1260 0x7f9350063550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:28.541 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.523+0000 7f9355b0d700 1 --2- 172.21.15.47:0/3275593688 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f935015a2d0 0x7f935015a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:28.541 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.523+0000 7f9354b0b700 1 -- 172.21.15.47:0/3275593688 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f935015a2d0 msgr2=0x7f935015a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:28.542 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.523+0000 7f9354b0b700 1 --2- 172.21.15.47:0/3275593688 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f935015a2d0 0x7f935015a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:28.542 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.523+0000 7f9354b0b700 1 -- 172.21.15.47:0/3275593688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93500b1260 msgr2=0x7f9350063550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:28.542 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.523+0000 7f9354b0b700 1 --2- 172.21.15.47:0/3275593688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93500b1260 0x7f9350063550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:28.542 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.523+0000 7f9354b0b700 1 -- 172.21.15.47:0/3275593688 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f934c00b1e0 con 0x7f93500b2120 2022-06-29T15:43:28.543 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.523+0000 7f9354b0b700 1 --2- 172.21.15.47:0/3275593688 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93500b2120 0x7f9350159de0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7f93500aea10 tx=0x7f934c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:28.543 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.523+0000 7f93467fc700 1 -- 172.21.15.47:0/3275593688 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f934c0118e0 con 0x7f93500b2120 2022-06-29T15:43:28.543 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.523+0000 7f93467fc700 1 -- 172.21.15.47:0/3275593688 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f934c011a40 con 0x7f93500b2120 2022-06-29T15:43:28.544 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.523+0000 7f9357570700 1 -- 172.21.15.47:0/3275593688 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f935015ac50 con 0x7f93500b2120 2022-06-29T15:43:28.544 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.523+0000 7f93467fc700 1 -- 172.21.15.47:0/3275593688 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f934c011da0 con 0x7f93500b2120 2022-06-29T15:43:28.544 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.523+0000 7f9357570700 1 -- 172.21.15.47:0/3275593688 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93500af680 con 0x7f93500b2120 2022-06-29T15:43:28.545 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.527+0000 7f9357570700 1 -- 172.21.15.47:0/3275593688 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93501575b0 con 0x7f93500b2120 2022-06-29T15:43:28.545 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.527+0000 7f93467fc700 1 -- 172.21.15.47:0/3275593688 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f934c025030 con 0x7f93500b2120 2022-06-29T15:43:28.545 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.527+0000 7f93467fc700 1 --2- 172.21.15.47:0/3275593688 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9338053a70 0x7f9338055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:28.545 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.527+0000 7f93467fc700 1 -- 172.21.15.47:0/3275593688 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f934c06d1b0 con 0x7f93500b2120 2022-06-29T15:43:28.545 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.527+0000 7f935530c700 1 --2- 172.21.15.47:0/3275593688 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9338053a70 0x7f9338055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:28.546 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.527+0000 7f935530c700 1 --2- 172.21.15.47:0/3275593688 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9338053a70 0x7f9338055f30 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f934000d000 tx=0x7f9340008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:28.547 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.535+0000 7f93467fc700 1 -- 172.21.15.47:0/3275593688 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f934c047e60 con 0x7f93500b2120 2022-06-29T15:43:28.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.699+0000 7f9357570700 1 -- 172.21.15.47:0/3275593688 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f935015ac50 con 0x7f93500b2120 2022-06-29T15:43:28.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.699+0000 7f93467fc700 1 -- 172.21.15.47:0/3275593688 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+2488 (secure 0 0 0) 0x7f935015ac50 con 0x7f93500b2120 2022-06-29T15:43:28.706 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:43:28.706 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":12,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4896,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1047560169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1047560169},{"type":"v1","addr":"172.21.15.47:6835","nonce":1047560169}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":10},{"gid":4902,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/804808837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":804808837},{"type":"v1","addr":"172.21.15.64:6833","nonce":804808837}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":10}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:43:10.546977+0000","modified":"2022-06-29T15:43:11.549959+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4905},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4905":{"gid":4905,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6837/2219068343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2219068343},{"type":"v1","addr":"172.21.15.47:6837","nonce":2219068343}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2022-06-29T15:43:28.708 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.699+0000 7f9357570700 1 -- 172.21.15.47:0/3275593688 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9338053a70 msgr2=0x7f9338055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:28.708 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.699+0000 7f9357570700 1 --2- 172.21.15.47:0/3275593688 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9338053a70 0x7f9338055f30 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f934000d000 tx=0x7f9340008040).stop 2022-06-29T15:43:28.709 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.699+0000 7f9357570700 1 -- 172.21.15.47:0/3275593688 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93500b2120 msgr2=0x7f9350159de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:28.709 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.699+0000 7f9357570700 1 --2- 172.21.15.47:0/3275593688 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93500b2120 0x7f9350159de0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7f93500aea10 tx=0x7f934c015000).stop 2022-06-29T15:43:28.709 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.699+0000 7f9357570700 1 -- 172.21.15.47:0/3275593688 shutdown_connections 2022-06-29T15:43:28.709 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.699+0000 7f9357570700 1 --2- 172.21.15.47:0/3275593688 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f935015a2d0 0x7f935015a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:28.709 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.699+0000 7f9357570700 1 --2- 172.21.15.47:0/3275593688 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93500b2120 0x7f9350159de0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:28.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.699+0000 7f9357570700 1 --2- 172.21.15.47:0/3275593688 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9338053a70 0x7f9338055f30 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:28.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.699+0000 7f9357570700 1 --2- 172.21.15.47:0/3275593688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93500b1260 0x7f9350063550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:28.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.699+0000 7f9357570700 1 -- 172.21.15.47:0/3275593688 >> 172.21.15.47:0/3275593688 conn(0x7f93500626f0 msgr2=0x7f935014cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:28.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.703+0000 7f9357570700 1 -- 172.21.15.47:0/3275593688 shutdown_connections 2022-06-29T15:43:28.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.703+0000 7f9357570700 1 -- 172.21.15.47:0/3275593688 wait complete. 2022-06-29T15:43:28.711 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 12 2022-06-29T15:43:28.723 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_recall_max_caps 20 2022-06-29T15:43:28.839 INFO:teuthology.orchestra.run.smithi047.stdout:{"success":"mds_recall_max_caps = '20' (not observed, change may require restart) "} 2022-06-29T15:43:28.839 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_caps = '20' (not observed, change may require restart) " } 2022-06-29T15:43:28.840 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:43:28.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.923+0000 7f55ecaa1700 1 Processor -- start 2022-06-29T15:43:28.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.923+0000 7f55ecaa1700 1 -- start start 2022-06-29T15:43:28.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.927+0000 7f55ecaa1700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f55e80630c0 0x7f55e80b33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:28.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.927+0000 7f55ecaa1700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f55e8143f10 0x7f55e80b38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:28.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.927+0000 7f55ecaa1700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55e8062200 con 0x7f55e8061e20 2022-06-29T15:43:28.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.927+0000 7f55ecaa1700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55e805f150 con 0x7f55e80630c0 2022-06-29T15:43:28.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.927+0000 7f55ecaa1700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55e805f3f0 con 0x7f55e8143f10 2022-06-29T15:43:28.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.927+0000 7f55e77fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f55e8061e20 0x7f55e8143800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:32900/0 (socket says 172.21.15.47:32900) 2022-06-29T15:43:28.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.927+0000 7f55e77fe700 1 -- 172.21.15.47:0/4178405160 learned_addr learned my addr 172.21.15.47:0/4178405160 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:43:28.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.927+0000 7f55deffd700 1 --2- 172.21.15.47:0/4178405160 >> v2:172.21.15.64:3300/0 conn(0x7f55e80630c0 0x7f55e80b33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:28.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.927+0000 7f55e7fff700 1 --2- 172.21.15.47:0/4178405160 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f55e8143f10 0x7f55e80b38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:28.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.927+0000 7f55e6ffd700 1 -- 172.21.15.47:0/4178405160 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2818923250 0 0) 0x7f55e8062200 con 0x7f55e8061e20 2022-06-29T15:43:28.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.927+0000 7f55e6ffd700 1 -- 172.21.15.47:0/4178405160 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55cc003260 con 0x7f55e8061e20 2022-06-29T15:43:28.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.927+0000 7f55deffd700 1 -- 172.21.15.47:0/4178405160 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f55e8143f10 msgr2=0x7f55e80b38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:43:28.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.927+0000 7f55deffd700 1 --2- 172.21.15.47:0/4178405160 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f55e8143f10 0x7f55e80b38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:28.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.927+0000 7f55deffd700 1 -- 172.21.15.47:0/4178405160 >> v1:172.21.15.47:6789/0 conn(0x7f55e8061e20 legacy=0x7f55e8143800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:28.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.927+0000 7f55deffd700 1 -- 172.21.15.47:0/4178405160 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e805f690 con 0x7f55e80630c0 2022-06-29T15:43:28.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.927+0000 7f55deffd700 1 --2- 172.21.15.47:0/4178405160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55e80630c0 0x7f55e80b33b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f55d0002f70 tx=0x7f55d0006df0).ready entity=mon.1 client_cookie=1abf782fb5dd83c server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:28.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.927+0000 7f55e6ffd700 1 -- 172.21.15.47:0/4178405160 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55d000d4a0 con 0x7f55e80630c0 2022-06-29T15:43:28.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.927+0000 7f55e6ffd700 1 -- 172.21.15.47:0/4178405160 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55d0000e90 con 0x7f55e80630c0 2022-06-29T15:43:28.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.927+0000 7f55e6ffd700 1 -- 172.21.15.47:0/4178405160 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55d000d7f0 con 0x7f55e80630c0 2022-06-29T15:43:28.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.927+0000 7f55ecaa1700 1 -- 172.21.15.47:0/4178405160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55e80630c0 msgr2=0x7f55e80b33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:28.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.927+0000 7f55ecaa1700 1 --2- 172.21.15.47:0/4178405160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55e80630c0 0x7f55e80b33b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f55d0002f70 tx=0x7f55d0006df0).stop 2022-06-29T15:43:28.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.927+0000 7f55ecaa1700 1 -- 172.21.15.47:0/4178405160 shutdown_connections 2022-06-29T15:43:28.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.927+0000 7f55ecaa1700 1 --2- 172.21.15.47:0/4178405160 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f55e8143f10 0x7f55e80b38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:28.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.927+0000 7f55ecaa1700 1 --2- 172.21.15.47:0/4178405160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55e80630c0 0x7f55e80b33b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:28.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.927+0000 7f55ecaa1700 1 -- 172.21.15.47:0/4178405160 >> 172.21.15.47:0/4178405160 conn(0x7f55e8063670 msgr2=0x7f55e8063a80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:28.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.927+0000 7f55ecaa1700 1 -- 172.21.15.47:0/4178405160 shutdown_connections 2022-06-29T15:43:28.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.927+0000 7f55ecaa1700 1 -- 172.21.15.47:0/4178405160 wait complete. 2022-06-29T15:43:28.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.931+0000 7f55ecaa1700 1 Processor -- start 2022-06-29T15:43:28.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.931+0000 7f55ecaa1700 1 -- start start 2022-06-29T15:43:28.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.931+0000 7f55ecaa1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55e8061e20 0x7f55e8066cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:28.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.931+0000 7f55ecaa1700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f55e80630c0 0x7f55e81411e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:28.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.931+0000 7f55ecaa1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55e8143f10 0x7f55e81416d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:28.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.931+0000 7f55ecaa1700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e813e100 con 0x7f55e8061e20 2022-06-29T15:43:28.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.931+0000 7f55ecaa1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e8065eb0 con 0x7f55e8143f10 2022-06-29T15:43:28.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.931+0000 7f55ecaa1700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55e813d540 con 0x7f55e80630c0 2022-06-29T15:43:28.950 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.931+0000 7f55e77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55e8061e20 0x7f55e8066cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:28.950 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.931+0000 7f55e77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55e8061e20 0x7f55e8066cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47350/0 (socket says 172.21.15.47:47350) 2022-06-29T15:43:28.950 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.931+0000 7f55e77fe700 1 -- 172.21.15.47:0/1871598196 learned_addr learned my addr 172.21.15.47:0/1871598196 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:28.951 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.931+0000 7f55e7fff700 1 --2- 172.21.15.47:0/1871598196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55e8143f10 0x7f55e81416d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:28.951 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.931+0000 7f55deffd700 1 --2- 172.21.15.47:0/1871598196 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f55e80630c0 0x7f55e81411e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:28.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.931+0000 7f55e77fe700 1 -- 172.21.15.47:0/1871598196 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f55e80630c0 msgr2=0x7f55e81411e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:28.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.931+0000 7f55e77fe700 1 --2- 172.21.15.47:0/1871598196 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f55e80630c0 0x7f55e81411e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:28.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.931+0000 7f55e77fe700 1 -- 172.21.15.47:0/1871598196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55e8143f10 msgr2=0x7f55e81416d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:28.953 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.931+0000 7f55e77fe700 1 --2- 172.21.15.47:0/1871598196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55e8143f10 0x7f55e81416d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:28.953 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.931+0000 7f55e77fe700 1 -- 172.21.15.47:0/1871598196 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55d000b180 con 0x7f55e8061e20 2022-06-29T15:43:28.953 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.931+0000 7f55e7fff700 1 --2- 172.21.15.47:0/1871598196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55e8143f10 0x7f55e81416d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:43:28.954 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.931+0000 7f55e77fe700 1 --2- 172.21.15.47:0/1871598196 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55e8061e20 0x7f55e8066cd0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7f55d8001110 tx=0x7f55d8007aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:28.954 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.931+0000 7f55e57fa700 1 -- 172.21.15.47:0/1871598196 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55d80070d0 con 0x7f55e8061e20 2022-06-29T15:43:28.954 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.931+0000 7f55e57fa700 1 -- 172.21.15.47:0/1871598196 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55d8007230 con 0x7f55e8061e20 2022-06-29T15:43:28.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.931+0000 7f55ecaa1700 1 -- 172.21.15.47:0/1871598196 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55e813d7e0 con 0x7f55e8061e20 2022-06-29T15:43:28.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.931+0000 7f55e57fa700 1 -- 172.21.15.47:0/1871598196 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55d8005610 con 0x7f55e8061e20 2022-06-29T15:43:28.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.931+0000 7f55ecaa1700 1 -- 172.21.15.47:0/1871598196 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55e8143010 con 0x7f55e8061e20 2022-06-29T15:43:28.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.935+0000 7f55ecaa1700 1 -- 172.21.15.47:0/1871598196 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55e813efe0 con 0x7f55e8061e20 2022-06-29T15:43:28.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.935+0000 7f55e57fa700 1 -- 172.21.15.47:0/1871598196 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f55d8010070 con 0x7f55e8061e20 2022-06-29T15:43:28.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.935+0000 7f55e57fa700 1 --2- 172.21.15.47:0/1871598196 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f55c0053b00 0x7f55c0055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:28.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.935+0000 7f55e57fa700 1 -- 172.21.15.47:0/1871598196 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f55d806b7d0 con 0x7f55e8061e20 2022-06-29T15:43:28.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.935+0000 7f55deffd700 1 --2- 172.21.15.47:0/1871598196 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f55c0053b00 0x7f55c0055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:28.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.935+0000 7f55deffd700 1 --2- 172.21.15.47:0/1871598196 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f55c0053b00 0x7f55c0055fc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f55d0015000 tx=0x7f55d0014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:28.958 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:28.943+0000 7f55e57fa700 1 -- 172.21.15.47:0/1871598196 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f55d8042480 con 0x7f55e8061e20 2022-06-29T15:43:29.110 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.103+0000 7f55ecaa1700 1 -- 172.21.15.47:0/1871598196 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f55e80573d0 con 0x7f55e8061e20 2022-06-29T15:43:29.112 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.103+0000 7f55e57fa700 1 -- 172.21.15.47:0/1871598196 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+2488 (secure 0 0 0) 0x7f55d8045aa0 con 0x7f55e8061e20 2022-06-29T15:43:29.112 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:43:29.113 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":12,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4896,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1047560169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1047560169},{"type":"v1","addr":"172.21.15.47:6835","nonce":1047560169}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":10},{"gid":4902,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/804808837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":804808837},{"type":"v1","addr":"172.21.15.64:6833","nonce":804808837}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":10}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:43:10.546977+0000","modified":"2022-06-29T15:43:11.549959+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4905},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4905":{"gid":4905,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6837/2219068343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2219068343},{"type":"v1","addr":"172.21.15.47:6837","nonce":2219068343}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2022-06-29T15:43:29.116 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.107+0000 7f55ecaa1700 1 -- 172.21.15.47:0/1871598196 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f55c0053b00 msgr2=0x7f55c0055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:29.116 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.107+0000 7f55ecaa1700 1 --2- 172.21.15.47:0/1871598196 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f55c0053b00 0x7f55c0055fc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f55d0015000 tx=0x7f55d0014000).stop 2022-06-29T15:43:29.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.107+0000 7f55ecaa1700 1 -- 172.21.15.47:0/1871598196 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55e8061e20 msgr2=0x7f55e8066cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:29.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.107+0000 7f55ecaa1700 1 --2- 172.21.15.47:0/1871598196 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55e8061e20 0x7f55e8066cd0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7f55d8001110 tx=0x7f55d8007aa0).stop 2022-06-29T15:43:29.118 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.107+0000 7f55ecaa1700 1 -- 172.21.15.47:0/1871598196 shutdown_connections 2022-06-29T15:43:29.118 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.107+0000 7f55ecaa1700 1 --2- 172.21.15.47:0/1871598196 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f55c0053b00 0x7f55c0055fc0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:29.119 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.107+0000 7f55ecaa1700 1 --2- 172.21.15.47:0/1871598196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55e8143f10 0x7f55e81416d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:29.119 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.107+0000 7f55ecaa1700 1 --2- 172.21.15.47:0/1871598196 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f55e80630c0 0x7f55e81411e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:29.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.107+0000 7f55ecaa1700 1 --2- 172.21.15.47:0/1871598196 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55e8061e20 0x7f55e8066cd0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:29.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.107+0000 7f55ecaa1700 1 -- 172.21.15.47:0/1871598196 >> 172.21.15.47:0/1871598196 conn(0x7f55e8063670 msgr2=0x7f55e8143720 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:29.121 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.107+0000 7f55ecaa1700 1 -- 172.21.15.47:0/1871598196 shutdown_connections 2022-06-29T15:43:29.121 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.107+0000 7f55ecaa1700 1 -- 172.21.15.47:0/1871598196 wait complete. 2022-06-29T15:43:29.122 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 12 2022-06-29T15:43:29.126 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_recall_max_decay_threshold 400 2022-06-29T15:43:29.227 INFO:teuthology.orchestra.run.smithi047.stdout:{"success":"mds_recall_max_decay_threshold = '400' (not observed, change may require restart) "} 2022-06-29T15:43:29.228 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_decay_threshold = '400' (not observed, change may require restart) " } 2022-06-29T15:43:29.228 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:43:29.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.327+0000 7f5e3ad14700 1 Processor -- start 2022-06-29T15:43:29.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.327+0000 7f5e3ad14700 1 -- start start 2022-06-29T15:43:29.335 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.327+0000 7f5e3ad14700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5e340b2ee0 0x7f5e340b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:29.335 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.327+0000 7f5e3ad14700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e341400e0 con 0x7f5e340ad970 2022-06-29T15:43:29.335 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.327+0000 7f5e3ad14700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e34143570 con 0x7f5e340b0e50 2022-06-29T15:43:29.335 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.327+0000 7f5e3ad14700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e34140ef0 con 0x7f5e340b2ee0 2022-06-29T15:43:29.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.327+0000 7f5e39511700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f5e340ad970 0x7f5e341407e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:32912/0 (socket says 172.21.15.47:32912) 2022-06-29T15:43:29.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.327+0000 7f5e39511700 1 -- 172.21.15.47:0/1246242656 learned_addr learned my addr 172.21.15.47:0/1246242656 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:43:29.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.327+0000 7f5e39d12700 1 --2- 172.21.15.47:0/1246242656 >> v2:172.21.15.64:3300/0 conn(0x7f5e340b2ee0 0x7f5e340b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:29.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.327+0000 7f5e38d10700 1 -- 172.21.15.47:0/1246242656 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 213786178 0 0) 0x7f5e34143570 con 0x7f5e340b0e50 2022-06-29T15:43:29.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.327+0000 7f5e38d10700 1 -- 172.21.15.47:0/1246242656 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e1c003260 con 0x7f5e340b0e50 2022-06-29T15:43:29.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.327+0000 7f5e39d12700 1 -- 172.21.15.47:0/1246242656 >> v1:172.21.15.64:6789/0 conn(0x7f5e340b0e50 legacy=0x7f5e3413d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:29.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.327+0000 7f5e39d12700 1 -- 172.21.15.47:0/1246242656 >> v1:172.21.15.47:6789/0 conn(0x7f5e340ad970 legacy=0x7f5e341407e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:29.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.327+0000 7f5e39d12700 1 -- 172.21.15.47:0/1246242656 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e34140340 con 0x7f5e340b2ee0 2022-06-29T15:43:29.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.327+0000 7f5e39d12700 1 --2- 172.21.15.47:0/1246242656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e340b2ee0 0x7f5e340b32c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f5e30002f70 tx=0x7f5e30006e80).ready entity=mon.1 client_cookie=402b348b28619999 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:29.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.327+0000 7f5e38d10700 1 -- 172.21.15.47:0/1246242656 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e3000b7d0 con 0x7f5e340b2ee0 2022-06-29T15:43:29.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.327+0000 7f5e38d10700 1 -- 172.21.15.47:0/1246242656 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e3000b930 con 0x7f5e340b2ee0 2022-06-29T15:43:29.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.327+0000 7f5e38d10700 1 -- 172.21.15.47:0/1246242656 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e3000bc90 con 0x7f5e340b2ee0 2022-06-29T15:43:29.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.327+0000 7f5e3ad14700 1 -- 172.21.15.47:0/1246242656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e340b2ee0 msgr2=0x7f5e340b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:29.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.327+0000 7f5e3ad14700 1 --2- 172.21.15.47:0/1246242656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e340b2ee0 0x7f5e340b32c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f5e30002f70 tx=0x7f5e30006e80).stop 2022-06-29T15:43:29.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.327+0000 7f5e3ad14700 1 -- 172.21.15.47:0/1246242656 shutdown_connections 2022-06-29T15:43:29.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.327+0000 7f5e3ad14700 1 --2- 172.21.15.47:0/1246242656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e340b2ee0 0x7f5e340b32c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:29.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.327+0000 7f5e3ad14700 1 -- 172.21.15.47:0/1246242656 >> 172.21.15.47:0/1246242656 conn(0x7f5e34063670 msgr2=0x7f5e34063a80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:29.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.331+0000 7f5e3ad14700 1 -- 172.21.15.47:0/1246242656 shutdown_connections 2022-06-29T15:43:29.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.331+0000 7f5e3ad14700 1 -- 172.21.15.47:0/1246242656 wait complete. 2022-06-29T15:43:29.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.331+0000 7f5e3ad14700 1 Processor -- start 2022-06-29T15:43:29.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.331+0000 7f5e3ad14700 1 -- start start 2022-06-29T15:43:29.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.331+0000 7f5e3ad14700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5e340ad970 0x7f5e34064ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:29.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.331+0000 7f5e3ad14700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e340b0e50 0x7f5e3409ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:29.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.331+0000 7f5e3ad14700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e340b2ee0 0x7f5e340ab4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:29.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.331+0000 7f5e3ad14700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e340ac4a0 con 0x7f5e340b2ee0 2022-06-29T15:43:29.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.331+0000 7f5e3ad14700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e340674f0 con 0x7f5e340b0e50 2022-06-29T15:43:29.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.331+0000 7f5e3ad14700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e340acf80 con 0x7f5e340ad970 2022-06-29T15:43:29.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.331+0000 7f5e3a513700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e340b2ee0 0x7f5e340ab4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:29.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.331+0000 7f5e3a513700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e340b2ee0 0x7f5e340ab4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47366/0 (socket says 172.21.15.47:47366) 2022-06-29T15:43:29.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.335+0000 7f5e3a513700 1 -- 172.21.15.47:0/233782197 learned_addr learned my addr 172.21.15.47:0/233782197 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:29.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.335+0000 7f5e39d12700 1 --2- 172.21.15.47:0/233782197 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5e340ad970 0x7f5e34064ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:29.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.335+0000 7f5e39511700 1 --2- 172.21.15.47:0/233782197 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e340b0e50 0x7f5e3409ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:29.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.335+0000 7f5e3a513700 1 -- 172.21.15.47:0/233782197 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5e340ad970 msgr2=0x7f5e34064ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:29.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.335+0000 7f5e3a513700 1 --2- 172.21.15.47:0/233782197 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5e340ad970 0x7f5e34064ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:29.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.335+0000 7f5e3a513700 1 -- 172.21.15.47:0/233782197 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e340b0e50 msgr2=0x7f5e3409ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:29.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.335+0000 7f5e3a513700 1 --2- 172.21.15.47:0/233782197 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e340b0e50 0x7f5e3409ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:29.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.335+0000 7f5e3a513700 1 -- 172.21.15.47:0/233782197 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e3000d410 con 0x7f5e340b2ee0 2022-06-29T15:43:29.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.335+0000 7f5e3a513700 1 --2- 172.21.15.47:0/233782197 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e340b2ee0 0x7f5e340ab4b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7f5e2c001490 tx=0x7f5e2c007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:29.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.335+0000 7f5e22ffd700 1 -- 172.21.15.47:0/233782197 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e2c002310 con 0x7f5e340b2ee0 2022-06-29T15:43:29.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.335+0000 7f5e22ffd700 1 -- 172.21.15.47:0/233782197 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e2c002470 con 0x7f5e340b2ee0 2022-06-29T15:43:29.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.335+0000 7f5e22ffd700 1 -- 172.21.15.47:0/233782197 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e2c001840 con 0x7f5e340b2ee0 2022-06-29T15:43:29.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.335+0000 7f5e3ad14700 1 -- 172.21.15.47:0/233782197 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e340abe70 con 0x7f5e340b2ee0 2022-06-29T15:43:29.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.335+0000 7f5e3ad14700 1 -- 172.21.15.47:0/233782197 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e340ab9a0 con 0x7f5e340b2ee0 2022-06-29T15:43:29.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.335+0000 7f5e22ffd700 1 -- 172.21.15.47:0/233782197 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5e2c010030 con 0x7f5e340b2ee0 2022-06-29T15:43:29.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.335+0000 7f5e22ffd700 1 --2- 172.21.15.47:0/233782197 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5e14053950 0x7f5e14055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:29.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.335+0000 7f5e3ad14700 1 -- 172.21.15.47:0/233782197 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e340676f0 con 0x7f5e340b2ee0 2022-06-29T15:43:29.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.335+0000 7f5e22ffd700 1 -- 172.21.15.47:0/233782197 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f5e2c06be50 con 0x7f5e340b2ee0 2022-06-29T15:43:29.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.335+0000 7f5e39d12700 1 --2- 172.21.15.47:0/233782197 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5e14053950 0x7f5e14055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:29.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.339+0000 7f5e39d12700 1 --2- 172.21.15.47:0/233782197 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5e14053950 0x7f5e14055e10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f5e3001e000 tx=0x7f5e34062c90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:29.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.343+0000 7f5e22ffd700 1 -- 172.21.15.47:0/233782197 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5e340676f0 con 0x7f5e340b2ee0 2022-06-29T15:43:29.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.499+0000 7f5e3ad14700 1 -- 172.21.15.47:0/233782197 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e340acde0 con 0x7f5e340b2ee0 2022-06-29T15:43:29.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.499+0000 7f5e22ffd700 1 -- 172.21.15.47:0/233782197 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+2488 (secure 0 0 0) 0x7f5e340acde0 con 0x7f5e340b2ee0 2022-06-29T15:43:29.508 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:43:29.508 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":12,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4896,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1047560169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1047560169},{"type":"v1","addr":"172.21.15.47:6835","nonce":1047560169}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":10},{"gid":4902,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/804808837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":804808837},{"type":"v1","addr":"172.21.15.64:6833","nonce":804808837}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":10}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:43:10.546977+0000","modified":"2022-06-29T15:43:11.549959+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4905},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4905":{"gid":4905,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6837/2219068343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2219068343},{"type":"v1","addr":"172.21.15.47:6837","nonce":2219068343}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2022-06-29T15:43:29.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.503+0000 7f5e3ad14700 1 -- 172.21.15.47:0/233782197 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5e14053950 msgr2=0x7f5e14055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:29.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.503+0000 7f5e3ad14700 1 --2- 172.21.15.47:0/233782197 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5e14053950 0x7f5e14055e10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f5e3001e000 tx=0x7f5e34062c90).stop 2022-06-29T15:43:29.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.503+0000 7f5e3ad14700 1 -- 172.21.15.47:0/233782197 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e340b2ee0 msgr2=0x7f5e340ab4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:29.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.503+0000 7f5e3ad14700 1 --2- 172.21.15.47:0/233782197 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e340b2ee0 0x7f5e340ab4b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7f5e2c001490 tx=0x7f5e2c007dd0).stop 2022-06-29T15:43:29.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.503+0000 7f5e3ad14700 1 -- 172.21.15.47:0/233782197 shutdown_connections 2022-06-29T15:43:29.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.503+0000 7f5e3ad14700 1 --2- 172.21.15.47:0/233782197 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e340b2ee0 0x7f5e340ab4b0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:29.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.503+0000 7f5e3ad14700 1 --2- 172.21.15.47:0/233782197 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5e14053950 0x7f5e14055e10 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:29.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.503+0000 7f5e3ad14700 1 --2- 172.21.15.47:0/233782197 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e340b0e50 0x7f5e3409ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:29.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.503+0000 7f5e3ad14700 1 --2- 172.21.15.47:0/233782197 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5e340ad970 0x7f5e34064ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:29.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.503+0000 7f5e3ad14700 1 -- 172.21.15.47:0/233782197 >> 172.21.15.47:0/233782197 conn(0x7f5e34063670 msgr2=0x7f5e3413e790 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:29.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.503+0000 7f5e3ad14700 1 -- 172.21.15.47:0/233782197 shutdown_connections 2022-06-29T15:43:29.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.503+0000 7f5e3ad14700 1 -- 172.21.15.47:0/233782197 wait complete. 2022-06-29T15:43:29.513 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 12 2022-06-29T15:43:29.521 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a cache drop 2022-06-29T15:43:29.634 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.627+0000 7f339d783700 1 Processor -- start 2022-06-29T15:43:29.634 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.627+0000 7f339d783700 1 -- start start 2022-06-29T15:43:29.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.627+0000 7f339d783700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3398060ee0 0x7f33980612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:29.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.627+0000 7f339d783700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f33981471a0 0x7f3398066f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:29.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.627+0000 7f339d783700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3398067610 con 0x7f33980b4300 2022-06-29T15:43:29.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.627+0000 7f339d783700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33981475d0 con 0x7f33981471a0 2022-06-29T15:43:29.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.627+0000 7f339d783700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33981467c0 con 0x7f3398060ee0 2022-06-29T15:43:29.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.627+0000 7f33977fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f33981471a0 0x7f3398066f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:29.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.627+0000 7f33977fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f33981471a0 0x7f3398066f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47376/0 (socket says 172.21.15.47:47376) 2022-06-29T15:43:29.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.627+0000 7f33977fe700 1 -- 172.21.15.47:0/691603281 learned_addr learned my addr 172.21.15.47:0/691603281 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:29.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.627+0000 7f33967fc700 1 --1- 172.21.15.47:0/691603281 >> v1:172.21.15.47:6789/0 conn(0x7f33980b4300 0x7f33981430e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:32928/0 (socket says 172.21.15.47:32928) 2022-06-29T15:43:29.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.627+0000 7f3396ffd700 1 --2- 172.21.15.47:0/691603281 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3398060ee0 0x7f33980612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:29.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.631+0000 7f33977fe700 1 -- 172.21.15.47:0/691603281 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3398060ee0 msgr2=0x7f33980612c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:43:29.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.631+0000 7f3395ffb700 1 -- 172.21.15.47:0/691603281 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3095330830 0 0) 0x7f3398067610 con 0x7f33980b4300 2022-06-29T15:43:29.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.631+0000 7f33977fe700 1 --2- 172.21.15.47:0/691603281 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3398060ee0 0x7f33980612c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:29.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.631+0000 7f33977fe700 1 -- 172.21.15.47:0/691603281 >> v1:172.21.15.47:6789/0 conn(0x7f33980b4300 legacy=0x7f33981430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:29.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.631+0000 7f33977fe700 1 -- 172.21.15.47:0/691603281 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3398146a60 con 0x7f33981471a0 2022-06-29T15:43:29.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.631+0000 7f33977fe700 1 --2- 172.21.15.47:0/691603281 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33981471a0 0x7f3398066f70 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7f33900032f0 tx=0x7f3390009290).ready entity=mon.0 client_cookie=5afd16354f872fe5 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:29.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.631+0000 7f3395ffb700 1 -- 172.21.15.47:0/691603281 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f339000b8a0 con 0x7f33981471a0 2022-06-29T15:43:29.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.631+0000 7f3395ffb700 1 -- 172.21.15.47:0/691603281 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f339000ba00 con 0x7f33981471a0 2022-06-29T15:43:29.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.631+0000 7f3395ffb700 1 -- 172.21.15.47:0/691603281 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f339000bb70 con 0x7f33981471a0 2022-06-29T15:43:29.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.631+0000 7f339d783700 1 -- 172.21.15.47:0/691603281 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33981471a0 msgr2=0x7f3398066f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:29.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.631+0000 7f339d783700 1 --2- 172.21.15.47:0/691603281 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33981471a0 0x7f3398066f70 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7f33900032f0 tx=0x7f3390009290).stop 2022-06-29T15:43:29.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.631+0000 7f339d783700 1 -- 172.21.15.47:0/691603281 shutdown_connections 2022-06-29T15:43:29.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.631+0000 7f339d783700 1 --2- 172.21.15.47:0/691603281 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33981471a0 0x7f3398066f70 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:29.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.631+0000 7f339d783700 1 --2- 172.21.15.47:0/691603281 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3398060ee0 0x7f33980612c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:29.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.631+0000 7f339d783700 1 -- 172.21.15.47:0/691603281 >> 172.21.15.47:0/691603281 conn(0x7f339805c510 msgr2=0x7f339805f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:29.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.631+0000 7f339d783700 1 -- 172.21.15.47:0/691603281 shutdown_connections 2022-06-29T15:43:29.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.631+0000 7f339d783700 1 -- 172.21.15.47:0/691603281 wait complete. 2022-06-29T15:43:29.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.635+0000 7f339d783700 1 Processor -- start 2022-06-29T15:43:29.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.635+0000 7f339d783700 1 -- start start 2022-06-29T15:43:29.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.635+0000 7f339d783700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3398060ee0 0x7f3398152360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:29.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.635+0000 7f339d783700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f33980b4300 0x7f33980b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:29.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.635+0000 7f339d783700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33981471a0 0x7f33981421e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:29.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.635+0000 7f339d783700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f339813dba0 con 0x7f33981471a0 2022-06-29T15:43:29.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.635+0000 7f339d783700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33980b3a20 con 0x7f3398060ee0 2022-06-29T15:43:29.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.635+0000 7f339d783700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f339813d900 con 0x7f33980b4300 2022-06-29T15:43:29.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.635+0000 7f33977fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33981471a0 0x7f33981421e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:29.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.635+0000 7f33977fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33981471a0 0x7f33981421e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47382/0 (socket says 172.21.15.47:47382) 2022-06-29T15:43:29.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.635+0000 7f33977fe700 1 -- 172.21.15.47:0/2514057403 learned_addr learned my addr 172.21.15.47:0/2514057403 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:29.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.635+0000 7f33967fc700 1 --2- 172.21.15.47:0/2514057403 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f33980b4300 0x7f33980b1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:29.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.635+0000 7f3396ffd700 1 --2- 172.21.15.47:0/2514057403 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3398060ee0 0x7f3398152360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:29.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.635+0000 7f33977fe700 1 -- 172.21.15.47:0/2514057403 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f33980b4300 msgr2=0x7f33980b1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:29.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.635+0000 7f33977fe700 1 --2- 172.21.15.47:0/2514057403 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f33980b4300 0x7f33980b1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:29.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.635+0000 7f33977fe700 1 -- 172.21.15.47:0/2514057403 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3398060ee0 msgr2=0x7f3398152360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:29.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.635+0000 7f33977fe700 1 --2- 172.21.15.47:0/2514057403 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3398060ee0 0x7f3398152360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:29.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.635+0000 7f33977fe700 1 -- 172.21.15.47:0/2514057403 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3390009530 con 0x7f33981471a0 2022-06-29T15:43:29.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.635+0000 7f33977fe700 1 --2- 172.21.15.47:0/2514057403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33981471a0 0x7f33981421e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7f33900032f0 tx=0x7f3390016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:29.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.639+0000 7f3387fff700 1 -- 172.21.15.47:0/2514057403 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f339000bb70 con 0x7f33981471a0 2022-06-29T15:43:29.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.639+0000 7f3387fff700 1 -- 172.21.15.47:0/2514057403 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3390021630 con 0x7f33981471a0 2022-06-29T15:43:29.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.639+0000 7f339d783700 1 -- 172.21.15.47:0/2514057403 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33980b22e0 con 0x7f33981471a0 2022-06-29T15:43:29.651 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.639+0000 7f3387fff700 1 -- 172.21.15.47:0/2514057403 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f339000b8a0 con 0x7f33981471a0 2022-06-29T15:43:29.651 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.639+0000 7f339d783700 1 -- 172.21.15.47:0/2514057403 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33981426d0 con 0x7f33981471a0 2022-06-29T15:43:29.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.639+0000 7f3387fff700 1 -- 172.21.15.47:0/2514057403 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f339000f070 con 0x7f33981471a0 2022-06-29T15:43:29.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.639+0000 7f3387fff700 1 --2- 172.21.15.47:0/2514057403 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3374053a70 0x7f3374055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:29.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.639+0000 7f3396ffd700 1 --2- 172.21.15.47:0/2514057403 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3374053a70 0x7f3374055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:29.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.639+0000 7f3387fff700 1 -- 172.21.15.47:0/2514057403 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f339006d7a0 con 0x7f33981471a0 2022-06-29T15:43:29.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.643+0000 7f3396ffd700 1 --2- 172.21.15.47:0/2514057403 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3374053a70 0x7f3374055f30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f338800b000 tx=0x7f3388006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:29.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.643+0000 7f33a79e5740 1 Processor -- start 2022-06-29T15:43:29.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.643+0000 7f33a79e5740 1 -- start start 2022-06-29T15:43:29.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.643+0000 7f33a79e5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x2494b90 0x24c7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:29.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.643+0000 7f33a79e5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x24c7a40 0x2495590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:29.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.643+0000 7f33a79e5740 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x2495a80 0x2495f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:29.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.643+0000 7f33a79e5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x2405bb0 con 0x2494b90 2022-06-29T15:43:29.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.643+0000 7f33a79e5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x23e6150 con 0x24c7a40 2022-06-29T15:43:29.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.643+0000 7f33a79e5740 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x2496420 con 0x2495a80 2022-06-29T15:43:29.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.643+0000 7f3396ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x2494b90 0x24c7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:29.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.643+0000 7f3396ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x2494b90 0x24c7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47386/0 (socket says 172.21.15.47:47386) 2022-06-29T15:43:29.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.643+0000 7f3396ffd700 1 -- 172.21.15.47:0/1846616193 learned_addr learned my addr 172.21.15.47:0/1846616193 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:29.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.643+0000 7f33977fe700 1 --2- 172.21.15.47:0/1846616193 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x2495a80 0x2495f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:29.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.643+0000 7f33967fc700 1 --2- 172.21.15.47:0/1846616193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x24c7a40 0x2495590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:29.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.643+0000 7f3396ffd700 1 -- 172.21.15.47:0/1846616193 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x2495a80 msgr2=0x2495f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:43:29.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.643+0000 7f3396ffd700 1 --2- 172.21.15.47:0/1846616193 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x2495a80 0x2495f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:29.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.643+0000 7f3396ffd700 1 -- 172.21.15.47:0/1846616193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x24c7a40 msgr2=0x2495590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:43:29.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.643+0000 7f3396ffd700 1 --2- 172.21.15.47:0/1846616193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x24c7a40 0x2495590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:29.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.643+0000 7f3396ffd700 1 -- 172.21.15.47:0/1846616193 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x243d400 con 0x2494b90 2022-06-29T15:43:29.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.643+0000 7f3396ffd700 1 --2- 172.21.15.47:0/1846616193 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x2494b90 0x24c7550 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x24913b0 tx=0x7f3388016000).ready entity=mon.0 client_cookie=79919073a100e27a server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:29.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.643+0000 7f339d783700 1 -- 172.21.15.47:0/1846616193 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3388014ae0 con 0x2494b90 2022-06-29T15:43:29.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.643+0000 7f339d783700 1 -- 172.21.15.47:0/1846616193 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3388014c40 con 0x2494b90 2022-06-29T15:43:29.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.643+0000 7f339d783700 1 -- 172.21.15.47:0/1846616193 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3388014db0 con 0x2494b90 2022-06-29T15:43:29.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.647+0000 7f33a79e5740 1 -- 172.21.15.47:0/1846616193 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x2494b90 msgr2=0x24c7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:29.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.647+0000 7f33a79e5740 1 --2- 172.21.15.47:0/1846616193 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x2494b90 0x24c7550 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x24913b0 tx=0x7f3388016000).stop 2022-06-29T15:43:29.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.647+0000 7f33a79e5740 1 -- 172.21.15.47:0/1846616193 shutdown_connections 2022-06-29T15:43:29.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.647+0000 7f33a79e5740 1 --2- 172.21.15.47:0/1846616193 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x2495a80 0x2495f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:29.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.647+0000 7f33a79e5740 1 --2- 172.21.15.47:0/1846616193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x24c7a40 0x2495590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:29.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.647+0000 7f33a79e5740 1 --2- 172.21.15.47:0/1846616193 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x2494b90 0x24c7550 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:29.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.647+0000 7f33a79e5740 1 -- 172.21.15.47:0/1846616193 >> 172.21.15.47:0/1846616193 conn(0x241d590 msgr2=0x2493e90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:29.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.647+0000 7f33a79e5740 1 -- 172.21.15.47:0/1846616193 shutdown_connections 2022-06-29T15:43:29.663 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.647+0000 7f33a79e5740 1 -- 172.21.15.47:0/1846616193 wait complete. 2022-06-29T15:43:29.663 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.647+0000 7f33a79e5740 1 -- start start 2022-06-29T15:43:29.663 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.647+0000 7f33a79e5740 1 Processor -- start 2022-06-29T15:43:29.668 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.647+0000 7f33a79e5740 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x2494b90 0x249c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:29.668 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.647+0000 7f33a79e5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x2495a80 0x24c6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:29.668 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.647+0000 7f33a79e5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x24c7a40 0x24c6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:29.668 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.647+0000 7f33a79e5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x24d4370 con 0x2495a80 2022-06-29T15:43:29.669 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.647+0000 7f33a79e5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x24daad0 con 0x24c7a40 2022-06-29T15:43:29.669 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.647+0000 7f33a79e5740 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x2498c20 con 0x2494b90 2022-06-29T15:43:29.669 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.647+0000 7f33967fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x2495a80 0x24c6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:29.669 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.647+0000 7f3396ffd700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x2494b90 0x249c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:29.669 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.647+0000 7f33a79e5740 10 client.0 ll_register_callbacks cb 0x244c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2022-06-29T15:43:29.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.647+0000 7f33967fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x2495a80 0x24c6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47394/0 (socket says 172.21.15.47:47394) 2022-06-29T15:43:29.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.647+0000 7f33967fc700 1 -- 172.21.15.47:0/1814081416 learned_addr learned my addr 172.21.15.47:0/1814081416 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:29.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.651+0000 7f33977fe700 1 --2- 172.21.15.47:0/1814081416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x24c7a40 0x24c6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:29.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.651+0000 7f33967fc700 1 -- 172.21.15.47:0/1814081416 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x2494b90 msgr2=0x249c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:43:29.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.651+0000 7f33967fc700 1 --2- 172.21.15.47:0/1814081416 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x2494b90 0x249c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:29.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.651+0000 7f33967fc700 1 -- 172.21.15.47:0/1814081416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x24c7a40 msgr2=0x24c6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:43:29.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.651+0000 7f33967fc700 1 --2- 172.21.15.47:0/1814081416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x24c7a40 0x24c6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:29.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.651+0000 7f33967fc700 1 -- 172.21.15.47:0/1814081416 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33880118a0 con 0x2495a80 2022-06-29T15:43:29.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.651+0000 7f33977fe700 1 --2- 172.21.15.47:0/1814081416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x24c7a40 0x24c6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:43:29.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.651+0000 7f33967fc700 1 --2- 172.21.15.47:0/1814081416 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x2495a80 0x24c6480 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f338c005000 tx=0x7f338c009040).ready entity=mon.0 client_cookie=f6c0e9278b166b29 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:29.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.651+0000 7f3371ffb700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2022-06-29T15:43:29.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.651+0000 7f3371ffb700 1 -- 172.21.15.47:0/1814081416 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f338c001ab0 con 0x2495a80 2022-06-29T15:43:29.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.651+0000 7f3371ffb700 1 -- 172.21.15.47:0/1814081416 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(36..36 src has 1..36) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f338c001c10 con 0x2495a80 2022-06-29T15:43:29.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.651+0000 7f33a79e5740 1 -- 172.21.15.47:0/1814081416 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x23fbaa0 con 0x2495a80 2022-06-29T15:43:29.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.651+0000 7f3371ffb700 1 -- 172.21.15.47:0/1814081416 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=12) v2 ==== 24+0+0 (secure 0 0 0) 0x7f338c0143d0 con 0x2495a80 2022-06-29T15:43:29.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.651+0000 7f33a79e5740 10 client.5130 fetch_fsmap learned FSMap version 12 2022-06-29T15:43:29.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.651+0000 7f33a79e5740 1 -- 172.21.15.47:0/1814081416 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=12,monmap=2+}) v3 -- 0x23f5380 con 0x2495a80 2022-06-29T15:43:29.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.651+0000 7f3371ffb700 1 -- 172.21.15.47:0/1814081416 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x23fbaa0 con 0x2495a80 2022-06-29T15:43:29.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.651+0000 7f3371ffb700 1 -- 172.21.15.47:0/1814081416 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 12) v1 ==== 1434+0+0 (secure 0 0 0) 0x7f338c014b40 con 0x2495a80 2022-06-29T15:43:29.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.651+0000 7f33a79e5740 10 client.5130 fetch_fsmap finished waiting for FSMap version 12 2022-06-29T15:43:29.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.651+0000 7f33a79e5740 10 client.5130 resolve_mds: resolved ID 'a' to GID 4905 2022-06-29T15:43:29.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.651+0000 7f33a79e5740 20 client.5130 populate_metadata read hostname 'smithi047' 2022-06-29T15:43:29.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.651+0000 7f33a79e5740 1 --2- 172.21.15.47:0/1814081416 >> [v2:172.21.15.47:6836/2219068343,v1:172.21.15.47:6837/2219068343] conn(0x2499cf0 0x249a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:29.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.651+0000 7f33a79e5740 4 client.5130 mds_command: new command op to 4905 tid=0[{"prefix": "get_command_descriptions"}] 2022-06-29T15:43:29.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.651+0000 7f33a79e5740 1 -- 172.21.15.47:0/1814081416 --> [v2:172.21.15.47:6836/2219068343,v1:172.21.15.47:6837/2219068343] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x23f3e10 con 0x2499cf0 2022-06-29T15:43:29.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.651+0000 7f3396ffd700 1 --2- 172.21.15.47:0/1814081416 >> [v2:172.21.15.47:6836/2219068343,v1:172.21.15.47:6837/2219068343] conn(0x2499cf0 0x249a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:29.676 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.651+0000 7f3371ffb700 10 client.5130 ms_handle_connect on v2:172.21.15.47:6836/2219068343 2022-06-29T15:43:29.676 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.651+0000 7f3396ffd700 1 --2- 172.21.15.47:0/1814081416 >> [v2:172.21.15.47:6836/2219068343,v1:172.21.15.47:6837/2219068343] conn(0x2499cf0 0x249a0d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=e2231fc3d76bf31b server_cookie=cf945d8eecc69dc0 in_seq=0 out_seq=0 2022-06-29T15:43:29.676 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.651+0000 7f3371ffb700 1 -- 172.21.15.47:0/1814081416 <== mds.0 v2:172.21.15.47:6836/2219068343 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x23f3e10 con 0x2499cf0 2022-06-29T15:43:29.676 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.651+0000 7f3371ffb700 10 client.5130 handle_command_reply: tid=0 2022-06-29T15:43:29.677 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.651+0000 7f33a79e5740 1 client.5130 shutdown 2022-06-29T15:43:29.677 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.651+0000 7f33a79e5740 1 -- 172.21.15.47:0/1814081416 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x2495a80 msgr2=0x24c6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:29.677 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.651+0000 7f33a79e5740 1 --2- 172.21.15.47:0/1814081416 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x2495a80 0x24c6480 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f338c005000 tx=0x7f338c009040).stop 2022-06-29T15:43:29.677 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.651+0000 7f33a79e5740 1 -- 172.21.15.47:0/1814081416 shutdown_connections 2022-06-29T15:43:29.677 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.651+0000 7f33a79e5740 1 --2- 172.21.15.47:0/1814081416 >> [v2:172.21.15.47:6836/2219068343,v1:172.21.15.47:6837/2219068343] conn(0x2499cf0 0x249a0d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:29.678 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.651+0000 7f33a79e5740 1 --2- 172.21.15.47:0/1814081416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x24c7a40 0x24c6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:29.678 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.651+0000 7f33a79e5740 1 --2- 172.21.15.47:0/1814081416 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x2495a80 0x24c6480 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:29.678 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.651+0000 7f33a79e5740 1 --2- 172.21.15.47:0/1814081416 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x2494b90 0x249c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:29.678 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.651+0000 7f33a79e5740 1 -- 172.21.15.47:0/1814081416 >> 172.21.15.47:0/1814081416 conn(0x241d590 msgr2=0x2494f70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:29.679 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.651+0000 7f3371ffb700 0 client.5130 ms_handle_reset on v2:172.21.15.47:6836/2219068343 2022-06-29T15:43:29.679 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.651+0000 7f33a79e5740 1 -- 172.21.15.47:0/1814081416 shutdown_connections 2022-06-29T15:43:29.679 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.651+0000 7f33a79e5740 1 -- 172.21.15.47:0/1814081416 wait complete. 2022-06-29T15:43:29.679 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.651+0000 7f33a79e5740 20 client.5130 trim_cache size 0 max 16384 2022-06-29T15:43:29.686 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.667+0000 7f33a79e5740 1 Processor -- start 2022-06-29T15:43:29.686 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.667+0000 7f33a79e5740 1 -- start start 2022-06-29T15:43:29.686 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.671+0000 7f33a79e5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x2494b90 0x2499cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:29.686 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.671+0000 7f33a79e5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x2495a80 0x249a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:29.686 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.671+0000 7f33a79e5740 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x24c7a40 0x24c6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:29.687 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.671+0000 7f33a79e5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x24934e0 con 0x2494b90 2022-06-29T15:43:29.687 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.671+0000 7f33a79e5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x240bdc0 con 0x2495a80 2022-06-29T15:43:29.687 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.671+0000 7f33a79e5740 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x24c3aa0 con 0x24c7a40 2022-06-29T15:43:29.687 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.671+0000 7f3396ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x2494b90 0x2499cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:29.688 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.671+0000 7f3396ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x2494b90 0x2499cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47400/0 (socket says 172.21.15.47:47400) 2022-06-29T15:43:29.688 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.671+0000 7f3396ffd700 1 -- 172.21.15.47:0/1198051372 learned_addr learned my addr 172.21.15.47:0/1198051372 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:29.688 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.671+0000 7f33977fe700 1 --2- 172.21.15.47:0/1198051372 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x24c7a40 0x24c6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:29.688 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.671+0000 7f33967fc700 1 --2- 172.21.15.47:0/1198051372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x2495a80 0x249a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:29.688 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.671+0000 7f3396ffd700 1 -- 172.21.15.47:0/1198051372 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x24c7a40 msgr2=0x24c6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:43:29.689 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.671+0000 7f3396ffd700 1 --2- 172.21.15.47:0/1198051372 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x24c7a40 0x24c6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:29.689 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.671+0000 7f3396ffd700 1 -- 172.21.15.47:0/1198051372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x2495a80 msgr2=0x249a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:43:29.689 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.671+0000 7f3396ffd700 1 --2- 172.21.15.47:0/1198051372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x2495a80 0x249a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:29.689 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.671+0000 7f3396ffd700 1 -- 172.21.15.47:0/1198051372 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x23fdc80 con 0x2494b90 2022-06-29T15:43:29.690 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.671+0000 7f33977fe700 1 --2- 172.21.15.47:0/1198051372 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x24c7a40 0x24c6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:43:29.690 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.671+0000 7f3396ffd700 1 --2- 172.21.15.47:0/1198051372 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x2494b90 0x2499cf0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x2450cf0 tx=0x7f3388033280).ready entity=mon.0 client_cookie=80bbca22ceba6510 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:29.690 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.671+0000 7f3373fff700 1 -- 172.21.15.47:0/1198051372 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3388032070 con 0x2494b90 2022-06-29T15:43:29.690 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.671+0000 7f3373fff700 1 -- 172.21.15.47:0/1198051372 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3388000e90 con 0x2494b90 2022-06-29T15:43:29.690 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.671+0000 7f3373fff700 1 -- 172.21.15.47:0/1198051372 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3388014750 con 0x2494b90 2022-06-29T15:43:29.691 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.671+0000 7f33a79e5740 1 -- 172.21.15.47:0/1198051372 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x2494b90 msgr2=0x2499cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:43:29.691 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.671+0000 7f33a79e5740 1 --2- 172.21.15.47:0/1198051372 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x2494b90 0x2499cf0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x2450cf0 tx=0x7f3388033280).stop 2022-06-29T15:43:29.691 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.671+0000 7f33a79e5740 1 -- 172.21.15.47:0/1198051372 shutdown_connections 2022-06-29T15:43:29.691 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.671+0000 7f33a79e5740 1 --2- 172.21.15.47:0/1198051372 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x24c7a40 0x24c6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:29.692 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.671+0000 7f33a79e5740 1 --2- 172.21.15.47:0/1198051372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x2495a80 0x249a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:29.692 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.671+0000 7f33a79e5740 1 --2- 172.21.15.47:0/1198051372 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x2494b90 0x2499cf0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:29.692 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.671+0000 7f33a79e5740 1 -- 172.21.15.47:0/1198051372 >> 172.21.15.47:0/1198051372 conn(0x241d590 msgr2=0x2494f70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:43:29.692 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.671+0000 7f33a79e5740 1 -- 172.21.15.47:0/1198051372 shutdown_connections 2022-06-29T15:43:29.692 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.671+0000 7f33a79e5740 1 -- 172.21.15.47:0/1198051372 wait complete. 2022-06-29T15:43:29.693 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.671+0000 7f33a79e5740 1 -- start start 2022-06-29T15:43:29.693 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.671+0000 7f33a79e5740 1 Processor -- start 2022-06-29T15:43:29.693 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.675+0000 7f33a79e5740 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x2494b90 0x24cfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:29.693 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.675+0000 7f33a79e5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x2495a80 0x2529bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:29.694 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.675+0000 7f33a79e5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x24c7a40 0x252a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:29.694 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.675+0000 7f33a79e5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x249a9c0 con 0x2495a80 2022-06-29T15:43:29.694 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.675+0000 7f33a79e5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x24d0470 con 0x24c7a40 2022-06-29T15:43:29.694 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.675+0000 7f33a79e5740 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x24da8f0 con 0x2494b90 2022-06-29T15:43:29.694 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.675+0000 7f33a79e5740 10 client.0 ll_register_callbacks cb 0x249b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2022-06-29T15:43:29.695 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.675+0000 7f33967fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x2495a80 0x2529bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:29.695 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.675+0000 7f33967fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x2495a80 0x2529bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47410/0 (socket says 172.21.15.47:47410) 2022-06-29T15:43:29.695 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.675+0000 7f33977fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x24c7a40 0x252a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:29.695 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.675+0000 7f33967fc700 1 -- 172.21.15.47:0/264983208 learned_addr learned my addr 172.21.15.47:0/264983208 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:43:29.696 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.675+0000 7f3396ffd700 1 --2- 172.21.15.47:0/264983208 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x2494b90 0x24cfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:29.696 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.675+0000 7f33967fc700 1 -- 172.21.15.47:0/264983208 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x2494b90 msgr2=0x24cfe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:43:29.696 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.675+0000 7f33967fc700 1 --2- 172.21.15.47:0/264983208 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x2494b90 0x24cfe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:29.696 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.675+0000 7f33967fc700 1 -- 172.21.15.47:0/264983208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x24c7a40 msgr2=0x252a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:43:29.696 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.675+0000 7f33967fc700 1 --2- 172.21.15.47:0/264983208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x24c7a40 0x252a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:43:29.697 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.675+0000 7f33967fc700 1 -- 172.21.15.47:0/264983208 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33981426d0 con 0x2495a80 2022-06-29T15:43:29.697 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.675+0000 7f33967fc700 1 --2- 172.21.15.47:0/264983208 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x2495a80 0x2529bd0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f338c026000 tx=0x7f338c019000).ready entity=mon.0 client_cookie=d27c934797639190 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:43:29.697 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.675+0000 7f3372ffd700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2022-06-29T15:43:29.697 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.675+0000 7f3372ffd700 1 -- 172.21.15.47:0/264983208 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f338c01f4c0 con 0x2495a80 2022-06-29T15:43:29.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.675+0000 7f33a79e5740 1 -- 172.21.15.47:0/264983208 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3388014c40 con 0x2495a80 2022-06-29T15:43:29.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.675+0000 7f3372ffd700 1 -- 172.21.15.47:0/264983208 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(36..36 src has 1..36) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f338c02cdc0 con 0x2495a80 2022-06-29T15:43:29.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.675+0000 7f3372ffd700 1 -- 172.21.15.47:0/264983208 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=12) v2 ==== 24+0+0 (secure 0 0 0) 0x7f338c01f620 con 0x2495a80 2022-06-29T15:43:29.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.675+0000 7f33a79e5740 10 client.5136 fetch_fsmap learned FSMap version 12 2022-06-29T15:43:29.699 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.675+0000 7f33a79e5740 1 -- 172.21.15.47:0/264983208 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=12,monmap=2+}) v3 -- 0x243d400 con 0x2495a80 2022-06-29T15:43:29.699 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.675+0000 7f3372ffd700 1 -- 172.21.15.47:0/264983208 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3388014c40 con 0x2495a80 2022-06-29T15:43:29.699 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.675+0000 7f3372ffd700 1 -- 172.21.15.47:0/264983208 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 12) v1 ==== 1434+0+0 (secure 0 0 0) 0x7f338c029040 con 0x2495a80 2022-06-29T15:43:29.699 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.675+0000 7f33a79e5740 10 client.5136 fetch_fsmap finished waiting for FSMap version 12 2022-06-29T15:43:29.699 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.675+0000 7f33a79e5740 10 client.5136 resolve_mds: resolved ID 'a' to GID 4905 2022-06-29T15:43:29.700 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.675+0000 7f33a79e5740 20 client.5136 populate_metadata read hostname 'smithi047' 2022-06-29T15:43:29.700 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.675+0000 7f33a79e5740 1 --2- 172.21.15.47:0/264983208 >> [v2:172.21.15.47:6836/2219068343,v1:172.21.15.47:6837/2219068343] conn(0x2530180 0x2530560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:43:29.700 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.675+0000 7f33a79e5740 4 client.5136 mds_command: new command op to 4905 tid=0[{"prefix": "cache drop"}] 2022-06-29T15:43:29.700 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.675+0000 7f33a79e5740 1 -- 172.21.15.47:0/264983208 --> [v2:172.21.15.47:6836/2219068343,v1:172.21.15.47:6837/2219068343] -- command(tid 0: {"prefix": "cache drop"}) v1 -- 0x22e9590 con 0x2530180 2022-06-29T15:43:29.700 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.675+0000 7f3396ffd700 1 --2- 172.21.15.47:0/264983208 >> [v2:172.21.15.47:6836/2219068343,v1:172.21.15.47:6837/2219068343] conn(0x2530180 0x2530560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:43:29.701 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.675+0000 7f3372ffd700 10 client.5136 ms_handle_connect on v2:172.21.15.47:6836/2219068343 2022-06-29T15:43:29.701 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:43:29.675+0000 7f3396ffd700 1 --2- 172.21.15.47:0/264983208 >> [v2:172.21.15.47:6836/2219068343,v1:172.21.15.47:6837/2219068343] conn(0x2530180 0x2530560 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=fdee42b753070626 server_cookie=a547e70c1206a432 in_seq=0 out_seq=0 2022-06-29T15:43:41.133 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:43:41.136 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:43:41.139 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:43:41.141 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:44:11.201 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:44:11.205 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:44:11.208 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:44:11.210 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:44:14.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.015+0000 7f3372ffd700 1 -- 172.21.15.47:0/264983208 <== mds.0 v2:172.21.15.47:6836/2219068343 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+386 (crc 0 0 0) 0x22e9590 con 0x2530180 2022-06-29T15:44:14.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.015+0000 7f3372ffd700 10 client.5136 handle_command_reply: tid=0 2022-06-29T15:44:14.022 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.015+0000 7f33a79e5740 1 client.5136 shutdown 2022-06-29T15:44:14.022 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.015+0000 7f33a79e5740 1 -- 172.21.15.47:0/264983208 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x2495a80 msgr2=0x2529bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:14.022 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.015+0000 7f33a79e5740 1 --2- 172.21.15.47:0/264983208 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x2495a80 0x2529bd0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f338c026000 tx=0x7f338c019000).stop 2022-06-29T15:44:14.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.015+0000 7f33a79e5740 1 -- 172.21.15.47:0/264983208 shutdown_connections 2022-06-29T15:44:14.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.015+0000 7f33a79e5740 1 --2- 172.21.15.47:0/264983208 >> [v2:172.21.15.47:6836/2219068343,v1:172.21.15.47:6837/2219068343] conn(0x2530180 0x2530560 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:14.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.015+0000 7f33a79e5740 1 --2- 172.21.15.47:0/264983208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x24c7a40 0x252a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:14.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.015+0000 7f33a79e5740 1 --2- 172.21.15.47:0/264983208 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x2495a80 0x2529bd0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:14.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.015+0000 7f33a79e5740 1 --2- 172.21.15.47:0/264983208 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x2494b90 0x24cfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:14.024 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.015+0000 7f3372ffd700 0 client.5136 ms_handle_reset on v2:172.21.15.47:6836/2219068343 2022-06-29T15:44:14.024 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.015+0000 7f33a79e5740 1 -- 172.21.15.47:0/264983208 >> 172.21.15.47:0/264983208 conn(0x241d590 msgr2=0x2494f70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:14.024 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.015+0000 7f33a79e5740 1 -- 172.21.15.47:0/264983208 shutdown_connections 2022-06-29T15:44:14.024 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.015+0000 7f33a79e5740 1 -- 172.21.15.47:0/264983208 wait complete. 2022-06-29T15:44:14.025 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.015+0000 7f33a79e5740 20 client.5136 trim_cache size 0 max 16384 2022-06-29T15:44:14.025 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2022-06-29T15:44:14.026 INFO:teuthology.orchestra.run.smithi047.stdout: "client_recall": { 2022-06-29T15:44:14.026 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2022-06-29T15:44:14.026 INFO:teuthology.orchestra.run.smithi047.stdout: "message": "(0) Success", 2022-06-29T15:44:14.026 INFO:teuthology.orchestra.run.smithi047.stdout: "recalled": 920 2022-06-29T15:44:14.027 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.027 INFO:teuthology.orchestra.run.smithi047.stdout: "flush_journal": { 2022-06-29T15:44:14.027 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2022-06-29T15:44:14.027 INFO:teuthology.orchestra.run.smithi047.stdout: "message": "" 2022-06-29T15:44:14.028 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.028 INFO:teuthology.orchestra.run.smithi047.stdout: "trim_cache": { 2022-06-29T15:44:14.028 INFO:teuthology.orchestra.run.smithi047.stdout: "trimmed": 900 2022-06-29T15:44:14.028 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.029 INFO:teuthology.orchestra.run.smithi047.stdout: "cache": { 2022-06-29T15:44:14.029 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": { 2022-06-29T15:44:14.029 INFO:teuthology.orchestra.run.smithi047.stdout: "items": 2531, 2022-06-29T15:44:14.029 INFO:teuthology.orchestra.run.smithi047.stdout: "bytes": 431968 2022-06-29T15:44:14.030 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:14.030 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.030 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 44.336288959000001 2022-06-29T15:44:14.030 INFO:teuthology.orchestra.run.smithi047.stdout:} 2022-06-29T15:44:14.032 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.015+0000 7f339d783700 1 -- 172.21.15.47:0/2514057403 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3374053a70 msgr2=0x7f3374055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:14.032 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.015+0000 7f339d783700 1 --2- 172.21.15.47:0/2514057403 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3374053a70 0x7f3374055f30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f338800b000 tx=0x7f3388006df0).stop 2022-06-29T15:44:14.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.015+0000 7f339d783700 1 -- 172.21.15.47:0/2514057403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33981471a0 msgr2=0x7f33981421e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:14.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.015+0000 7f339d783700 1 --2- 172.21.15.47:0/2514057403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33981471a0 0x7f33981421e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7f33900032f0 tx=0x7f3390016000).stop 2022-06-29T15:44:14.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.015+0000 7f339d783700 1 -- 172.21.15.47:0/2514057403 shutdown_connections 2022-06-29T15:44:14.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.015+0000 7f339d783700 1 --2- 172.21.15.47:0/2514057403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33981471a0 0x7f33981421e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:14.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.015+0000 7f339d783700 1 --2- 172.21.15.47:0/2514057403 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f33980b4300 0x7f33980b1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:14.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.015+0000 7f339d783700 1 --2- 172.21.15.47:0/2514057403 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3374053a70 0x7f3374055f30 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:14.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.015+0000 7f339d783700 1 --2- 172.21.15.47:0/2514057403 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3398060ee0 0x7f3398152360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:14.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.015+0000 7f339d783700 1 -- 172.21.15.47:0/2514057403 >> 172.21.15.47:0/2514057403 conn(0x7f339805c510 msgr2=0x7f339805f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:14.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.015+0000 7f339d783700 1 -- 172.21.15.47:0/2514057403 shutdown_connections 2022-06-29T15:44:14.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.015+0000 7f339d783700 1 -- 172.21.15.47:0/2514057403 wait complete. 2022-06-29T15:44:14.037 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-06-29T15:44:14.038 DEBUG:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:44:14.054 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:44:14.075 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:44:14.096 DEBUG:teuthology.orchestra.run.smithi164:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:44:14.118 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-06-29T15:44:14.119 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:44:14.119 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:44:14.120 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:44:14.147 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2022-06-29T15:44:14.148 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-29T15:44:14.148 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi099.front.sepia.ceph.com... 2022-06-29T15:44:14.149 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-06-29T15:44:14.149 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:44:14.163 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[12295]: fuse finished with error 0 and tester_r 0 2022-06-29T15:44:14.165 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:44:14.166 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:44:14.166 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:44:14.173 INFO:teuthology.orchestra.run.smithi099.stdout:ext2/ext3 2022-06-29T15:44:14.174 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-06-29T15:44:14.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-29T15:44:14.255 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:44:14.263 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:44:14.263 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:44:14.264 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:44:14.271 INFO:teuthology.orchestra.run.smithi164.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-06-29T15:44:14.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T15:44:14.273 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-06-29T15:44:14.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-29T15:44:14.274 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:44:14.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T15:44:14.323 INFO:teuthology.orchestra.run.smithi164.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-06-29T15:44:14.324 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-06-29T15:44:14.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.435+0000 7f1d5abd1700 1 Processor -- start 2022-06-29T15:44:14.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.435+0000 7f1d5abd1700 1 -- start start 2022-06-29T15:44:14.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.435+0000 7f1d5abd1700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1d540619b0 0x7f1d540606e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:14.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.435+0000 7f1d5abd1700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1d541408e0 0x7f1d54139a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:14.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.435+0000 7f1d5abd1700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d54140d50 con 0x7f1d540b0f30 2022-06-29T15:44:14.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.435+0000 7f1d5abd1700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d540b2190 con 0x7f1d541408e0 2022-06-29T15:44:14.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.435+0000 7f1d5abd1700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d54061d90 con 0x7f1d540619b0 2022-06-29T15:44:14.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.435+0000 7f1d5916e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1d541408e0 0x7f1d54139a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:14.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.435+0000 7f1d5896d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1d540619b0 0x7f1d540606e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:14.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.435+0000 7f1d5896d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1d540619b0 0x7f1d540606e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:53120/0 (socket says 172.21.15.47:53120) 2022-06-29T15:44:14.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.435+0000 7f1d5896d700 1 -- 172.21.15.47:0/3955100630 learned_addr learned my addr 172.21.15.47:0/3955100630 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:14.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.435+0000 7f1d4bfff700 1 --1- 172.21.15.47:0/3955100630 >> v1:172.21.15.64:6789/0 conn(0x7f1d540b0f30 0x7f1d540b1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:40536/0 (socket says 172.21.15.47:40536) 2022-06-29T15:44:14.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.439+0000 7f1d5896d700 1 -- 172.21.15.47:0/3955100630 >> v2:172.21.15.47:3300/0 conn(0x7f1d541408e0 msgr2=0x7f1d54139a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:44:14.449 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.439+0000 7f1d5896d700 1 --2- 172.21.15.47:0/3955100630 >> v2:172.21.15.47:3300/0 conn(0x7f1d541408e0 0x7f1d54139a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:14.449 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.439+0000 7f1d5896d700 1 -- 172.21.15.47:0/3955100630 >> v1:172.21.15.64:6789/0 conn(0x7f1d540b0f30 legacy=0x7f1d540b1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:14.449 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.439+0000 7f1d4b7fe700 1 -- 172.21.15.47:0/3955100630 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 138776711 0 0) 0x7f1d54140d50 con 0x7f1d540b0f30 2022-06-29T15:44:14.450 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.439+0000 7f1d5896d700 1 -- 172.21.15.47:0/3955100630 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d54060d80 con 0x7f1d540619b0 2022-06-29T15:44:14.450 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.439+0000 7f1d5896d700 1 --2- 172.21.15.47:0/3955100630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d540619b0 0x7f1d540606e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f1d50002f70 tx=0x7f1d50006f40).ready entity=mon.1 client_cookie=79fe5b3cec385c03 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:14.450 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.439+0000 7f1d4b7fe700 1 -- 172.21.15.47:0/3955100630 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d5000da80 con 0x7f1d540619b0 2022-06-29T15:44:14.451 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.439+0000 7f1d4b7fe700 1 -- 172.21.15.47:0/3955100630 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d50000e90 con 0x7f1d540619b0 2022-06-29T15:44:14.451 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.439+0000 7f1d4b7fe700 1 -- 172.21.15.47:0/3955100630 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d5000ddd0 con 0x7f1d540619b0 2022-06-29T15:44:14.452 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.439+0000 7f1d5abd1700 1 -- 172.21.15.47:0/3955100630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d540619b0 msgr2=0x7f1d540606e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:14.452 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.439+0000 7f1d5abd1700 1 --2- 172.21.15.47:0/3955100630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d540619b0 0x7f1d540606e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f1d50002f70 tx=0x7f1d50006f40).stop 2022-06-29T15:44:14.452 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.439+0000 7f1d5abd1700 1 -- 172.21.15.47:0/3955100630 shutdown_connections 2022-06-29T15:44:14.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.439+0000 7f1d5abd1700 1 --2- 172.21.15.47:0/3955100630 >> v2:172.21.15.47:3300/0 conn(0x7f1d541408e0 0x7f1d54139a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:14.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.439+0000 7f1d5abd1700 1 --2- 172.21.15.47:0/3955100630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d540619b0 0x7f1d540606e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:14.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.439+0000 7f1d5abd1700 1 -- 172.21.15.47:0/3955100630 >> 172.21.15.47:0/3955100630 conn(0x7f1d5405c510 msgr2=0x7f1d54063800 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:14.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.439+0000 7f1d5abd1700 1 -- 172.21.15.47:0/3955100630 shutdown_connections 2022-06-29T15:44:14.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.439+0000 7f1d5abd1700 1 -- 172.21.15.47:0/3955100630 wait complete. 2022-06-29T15:44:14.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.443+0000 7f1d5abd1700 1 Processor -- start 2022-06-29T15:44:14.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.443+0000 7f1d5abd1700 1 -- start start 2022-06-29T15:44:14.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.443+0000 7f1d5abd1700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1d540619b0 0x7f1d54065d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:14.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.443+0000 7f1d5abd1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d540b0f30 0x7f1d540b46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:14.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.443+0000 7f1d5abd1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d541408e0 0x7f1d54066b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:14.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.443+0000 7f1d5abd1700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d540b5bc0 con 0x7f1d541408e0 2022-06-29T15:44:14.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.443+0000 7f1d5abd1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d54143a50 con 0x7f1d540b0f30 2022-06-29T15:44:14.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.443+0000 7f1d5abd1700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d54143cf0 con 0x7f1d540619b0 2022-06-29T15:44:14.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.443+0000 7f1d5916e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d541408e0 0x7f1d54066b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:14.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.443+0000 7f1d5916e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d541408e0 0x7f1d54066b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47426/0 (socket says 172.21.15.47:47426) 2022-06-29T15:44:14.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.443+0000 7f1d5896d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1d540619b0 0x7f1d54065d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:14.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.443+0000 7f1d5916e700 1 -- 172.21.15.47:0/1241488080 learned_addr learned my addr 172.21.15.47:0/1241488080 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:14.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.443+0000 7f1d4bfff700 1 --2- 172.21.15.47:0/1241488080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d540b0f30 0x7f1d540b46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:14.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.443+0000 7f1d5896d700 1 -- 172.21.15.47:0/1241488080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d540b0f30 msgr2=0x7f1d540b46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:14.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.443+0000 7f1d5896d700 1 --2- 172.21.15.47:0/1241488080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d540b0f30 0x7f1d540b46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:14.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.443+0000 7f1d5896d700 1 -- 172.21.15.47:0/1241488080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d541408e0 msgr2=0x7f1d54066b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:14.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.443+0000 7f1d5896d700 1 --2- 172.21.15.47:0/1241488080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d541408e0 0x7f1d54066b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:14.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.443+0000 7f1d5896d700 1 -- 172.21.15.47:0/1241488080 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d5000b460 con 0x7f1d540619b0 2022-06-29T15:44:14.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.443+0000 7f1d5916e700 1 --2- 172.21.15.47:0/1241488080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d541408e0 0x7f1d54066b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:44:14.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.443+0000 7f1d5896d700 1 --2- 172.21.15.47:0/1241488080 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1d540619b0 0x7f1d54065d40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f1d50002f70 tx=0x7f1d50006f40).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:14.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.447+0000 7f1d49ffb700 1 -- 172.21.15.47:0/1241488080 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d5000dec0 con 0x7f1d540619b0 2022-06-29T15:44:14.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.447+0000 7f1d5abd1700 1 -- 172.21.15.47:0/1241488080 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d54142cb0 con 0x7f1d540619b0 2022-06-29T15:44:14.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.447+0000 7f1d49ffb700 1 -- 172.21.15.47:0/1241488080 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d50018400 con 0x7f1d540619b0 2022-06-29T15:44:14.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.447+0000 7f1d5abd1700 1 -- 172.21.15.47:0/1241488080 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d540b4be0 con 0x7f1d540619b0 2022-06-29T15:44:14.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.447+0000 7f1d49ffb700 1 -- 172.21.15.47:0/1241488080 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d50018760 con 0x7f1d540619b0 2022-06-29T15:44:14.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.447+0000 7f1d5abd1700 1 -- 172.21.15.47:0/1241488080 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d540573d0 con 0x7f1d540619b0 2022-06-29T15:44:14.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.447+0000 7f1d49ffb700 1 -- 172.21.15.47:0/1241488080 <== mon.2 v2:172.21.15.64:3301/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1d50021030 con 0x7f1d540619b0 2022-06-29T15:44:14.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.447+0000 7f1d49ffb700 1 --2- 172.21.15.47:0/1241488080 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1d340539e0 0x7f1d34055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:14.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.447+0000 7f1d49ffb700 1 -- 172.21.15.47:0/1241488080 <== mon.2 v2:172.21.15.64:3301/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f1d5006f290 con 0x7f1d540619b0 2022-06-29T15:44:14.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.447+0000 7f1d4bfff700 1 --2- 172.21.15.47:0/1241488080 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1d340539e0 0x7f1d34055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:14.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.447+0000 7f1d4bfff700 1 --2- 172.21.15.47:0/1241488080 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1d340539e0 0x7f1d34055ea0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f1d4400d000 tx=0x7f1d440079e0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:14.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.455+0000 7f1d49ffb700 1 -- 172.21.15.47:0/1241488080 <== mon.2 v2:172.21.15.64:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1d50045fb0 con 0x7f1d540619b0 2022-06-29T15:44:14.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.587+0000 7f1d5abd1700 1 -- 172.21.15.47:0/1241488080 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f1d5406a1a0 con 0x7f1d540619b0 2022-06-29T15:44:14.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.591+0000 7f1d49ffb700 1 -- 172.21.15.47:0/1241488080 <== mon.2 v2:172.21.15.64:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v36) v1 ==== 81+0+34037 (secure 0 0 0) 0x7f1d500495d0 con 0x7f1d540619b0 2022-06-29T15:44:14.596 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:44:14.597 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2022-06-29T15:44:14.597 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 36, 2022-06-29T15:44:14.598 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "9a68894a-53ce-400e-838b-14eab91b834c", 2022-06-29T15:44:14.598 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2022-06-29T15:42:14.046361+0000", 2022-06-29T15:44:14.598 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2022-06-29T15:43:11.550727+0000", 2022-06-29T15:44:14.599 INFO:teuthology.orchestra.run.smithi047.stdout: "last_up_change": "2022-06-29T15:42:20.929789+0000", 2022-06-29T15:44:14.599 INFO:teuthology.orchestra.run.smithi047.stdout: "last_in_change": "2022-06-29T15:42:20.929789+0000", 2022-06-29T15:44:14.600 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-06-29T15:44:14.600 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_num": 5799936, 2022-06-29T15:44:14.600 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_set": [ 2022-06-29T15:44:14.601 INFO:teuthology.orchestra.run.smithi047.stdout: "pglog_hardlimit", 2022-06-29T15:44:14.601 INFO:teuthology.orchestra.run.smithi047.stdout: "purged_snapdirs", 2022-06-29T15:44:14.601 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_deletes", 2022-06-29T15:44:14.602 INFO:teuthology.orchestra.run.smithi047.stdout: "sortbitwise" 2022-06-29T15:44:14.602 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:44:14.602 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_version": 5, 2022-06-29T15:44:14.603 INFO:teuthology.orchestra.run.smithi047.stdout: "full_ratio": 0.94999998807907104, 2022-06-29T15:44:14.603 INFO:teuthology.orchestra.run.smithi047.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-06-29T15:44:14.604 INFO:teuthology.orchestra.run.smithi047.stdout: "nearfull_ratio": 0.85000002384185791, 2022-06-29T15:44:14.604 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_snapshot": "", 2022-06-29T15:44:14.604 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_max": 6, 2022-06-29T15:44:14.605 INFO:teuthology.orchestra.run.smithi047.stdout: "max_osd": 8, 2022-06-29T15:44:14.605 INFO:teuthology.orchestra.run.smithi047.stdout: "require_min_compat_client": "jewel", 2022-06-29T15:44:14.605 INFO:teuthology.orchestra.run.smithi047.stdout: "min_compat_client": "jewel", 2022-06-29T15:44:14.606 INFO:teuthology.orchestra.run.smithi047.stdout: "require_osd_release": "octopus", 2022-06-29T15:44:14.606 INFO:teuthology.orchestra.run.smithi047.stdout: "pools": [ 2022-06-29T15:44:14.606 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.607 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 1, 2022-06-29T15:44:14.607 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "device_health_metrics", 2022-06-29T15:44:14.608 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:42:16.971431+0000", 2022-06-29T15:44:14.608 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:44:14.608 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:44:14.609 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:44:14.609 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:44:14.609 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:44:14.609 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:44:14.610 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:44:14.610 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:44:14.610 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 1, 2022-06-29T15:44:14.610 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 1, 2022-06-29T15:44:14.610 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 1, 2022-06-29T15:44:14.611 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 1, 2022-06-29T15:44:14.611 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 1, 2022-06-29T15:44:14.611 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:44:14.611 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:44:14.612 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:44:14.612 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:44:14.612 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:44:14.612 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:44:14.613 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:44:14.613 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.613 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "16", 2022-06-29T15:44:14.613 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:44:14.613 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:44:14.614 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:44:14.614 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:44:14.614 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:44:14.614 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:44:14.615 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:44:14.615 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:44:14.615 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:44:14.615 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:44:14.616 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:44:14.616 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:44:14.616 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:44:14.616 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:44:14.616 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:44:14.617 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:44:14.617 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:44:14.617 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:44:14.617 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:44:14.618 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:44:14.618 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:44:14.618 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:44:14.619 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:44:14.619 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:44:14.619 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:44:14.620 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:44:14.620 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.621 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:44:14.621 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:44:14.621 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:44:14.622 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:44:14.622 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:44:14.622 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:44:14.623 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:44:14.623 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:44:14.623 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:44:14.624 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:44:14.624 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:44:14.624 INFO:teuthology.orchestra.run.smithi047.stdout: "options": { 2022-06-29T15:44:14.625 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_min": 1 2022-06-29T15:44:14.625 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.626 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:44:14.626 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_devicehealth": {} 2022-06-29T15:44:14.626 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:14.627 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.627 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.627 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 2, 2022-06-29T15:44:14.628 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "rbd", 2022-06-29T15:44:14.628 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:42:25.290884+0000", 2022-06-29T15:44:14.628 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:44:14.629 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:44:14.629 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:44:14.629 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:44:14.630 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:44:14.630 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:44:14.631 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:44:14.631 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:44:14.631 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2022-06-29T15:44:14.632 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2022-06-29T15:44:14.632 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2022-06-29T15:44:14.632 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2022-06-29T15:44:14.633 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2022-06-29T15:44:14.633 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:44:14.633 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:44:14.634 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:44:14.634 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:44:14.634 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:44:14.635 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:44:14.635 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:44:14.636 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.636 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "19", 2022-06-29T15:44:14.636 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:44:14.637 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:44:14.637 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:44:14.637 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:44:14.638 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:44:14.638 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:44:14.638 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:44:14.639 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:44:14.639 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:44:14.639 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:44:14.639 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:44:14.640 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:44:14.640 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:44:14.640 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:44:14.640 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:44:14.641 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:44:14.641 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:44:14.641 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:44:14.641 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:44:14.641 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:44:14.642 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:44:14.642 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:44:14.642 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:44:14.642 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:44:14.643 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:44:14.643 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:44:14.643 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.643 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:44:14.643 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:44:14.644 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:44:14.644 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:44:14.644 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:44:14.644 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:44:14.645 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:44:14.645 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:44:14.645 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:44:14.645 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:44:14.645 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:44:14.646 INFO:teuthology.orchestra.run.smithi047.stdout: "options": {}, 2022-06-29T15:44:14.646 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:44:14.646 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd": {} 2022-06-29T15:44:14.646 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:14.646 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.647 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.647 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 5, 2022-06-29T15:44:14.647 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "cephfs_metadata", 2022-06-29T15:44:14.647 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:43:08.995574+0000", 2022-06-29T15:44:14.648 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:44:14.648 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:44:14.648 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:44:14.649 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:44:14.649 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:44:14.649 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:44:14.650 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:44:14.650 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:44:14.650 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2022-06-29T15:44:14.651 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2022-06-29T15:44:14.651 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2022-06-29T15:44:14.652 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2022-06-29T15:44:14.652 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2022-06-29T15:44:14.652 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:44:14.653 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:44:14.653 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:44:14.653 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:44:14.654 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:44:14.654 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:44:14.654 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:44:14.655 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.655 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "35", 2022-06-29T15:44:14.656 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:44:14.656 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:44:14.656 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:44:14.657 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:44:14.657 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:44:14.657 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:44:14.658 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:44:14.658 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:44:14.658 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:44:14.659 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:44:14.659 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:44:14.659 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:44:14.660 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:44:14.660 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:44:14.660 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:44:14.661 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:44:14.661 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:44:14.661 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:44:14.662 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:44:14.662 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:44:14.662 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:44:14.663 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:44:14.663 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:44:14.664 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:44:14.664 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:44:14.664 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:44:14.665 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.665 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:44:14.665 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:44:14.666 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:44:14.666 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:44:14.666 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:44:14.667 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:44:14.667 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:44:14.667 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:44:14.668 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:44:14.668 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:44:14.668 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:44:14.668 INFO:teuthology.orchestra.run.smithi047.stdout: "options": { 2022-06-29T15:44:14.669 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_bias": 4, 2022-06-29T15:44:14.669 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_min": 16, 2022-06-29T15:44:14.669 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_priority": 5 2022-06-29T15:44:14.669 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.670 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:44:14.670 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs": { 2022-06-29T15:44:14.670 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": "cephfs" 2022-06-29T15:44:14.670 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:14.671 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:14.671 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.671 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.671 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 6, 2022-06-29T15:44:14.672 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "cephfs_data", 2022-06-29T15:44:14.672 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:43:09.557935+0000", 2022-06-29T15:44:14.672 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:44:14.672 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:44:14.672 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:44:14.673 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:44:14.673 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:44:14.673 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:44:14.673 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:44:14.674 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:44:14.674 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2022-06-29T15:44:14.674 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2022-06-29T15:44:14.674 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2022-06-29T15:44:14.674 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2022-06-29T15:44:14.675 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2022-06-29T15:44:14.675 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:44:14.675 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:44:14.675 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:44:14.676 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:44:14.676 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:44:14.676 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:44:14.676 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:44:14.676 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.677 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "36", 2022-06-29T15:44:14.677 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:44:14.677 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:44:14.677 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:44:14.678 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:44:14.678 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:44:14.678 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:44:14.678 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:44:14.679 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:44:14.679 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:44:14.679 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:44:14.679 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:44:14.680 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:44:14.680 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:44:14.680 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:44:14.680 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:44:14.680 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:44:14.681 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:44:14.681 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:44:14.681 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:44:14.681 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:44:14.682 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:44:14.682 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:44:14.682 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:44:14.682 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:44:14.682 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:44:14.683 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:44:14.683 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.683 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:44:14.683 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:44:14.684 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:44:14.684 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:44:14.684 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:44:14.684 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:44:14.684 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:44:14.685 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:44:14.685 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:44:14.685 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:44:14.685 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:44:14.686 INFO:teuthology.orchestra.run.smithi047.stdout: "options": {}, 2022-06-29T15:44:14.686 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:44:14.686 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs": { 2022-06-29T15:44:14.686 INFO:teuthology.orchestra.run.smithi047.stdout: "data": "cephfs" 2022-06-29T15:44:14.686 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:14.687 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:14.687 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:14.687 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:44:14.687 INFO:teuthology.orchestra.run.smithi047.stdout: "osds": [ 2022-06-29T15:44:14.687 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.688 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2022-06-29T15:44:14.688 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "e23eb023-4706-4201-bf72-4bf19c912c5d", 2022-06-29T15:44:14.688 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:44:14.688 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:44:14.689 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:44:14.689 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:44:14.689 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:44:14.689 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:44:14.689 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:44:14.690 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 34, 2022-06-29T15:44:14.690 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:44:14.690 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:44:14.690 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:44:14.691 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:14.691 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.691 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:14.691 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6818", 2022-06-29T15:44:14.691 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:44:14.692 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.692 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.692 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:14.692 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6819", 2022-06-29T15:44:14.693 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:44:14.693 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:14.693 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:14.693 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.693 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:44:14.694 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:14.694 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.694 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:14.694 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6820", 2022-06-29T15:44:14.695 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:44:14.695 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.695 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.695 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:14.696 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6821", 2022-06-29T15:44:14.696 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:44:14.696 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:14.696 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:14.696 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.697 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:44:14.697 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:14.697 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.697 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:14.697 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6824", 2022-06-29T15:44:14.698 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:44:14.698 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.698 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.698 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:14.699 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6825", 2022-06-29T15:44:14.699 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:44:14.699 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:14.699 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:14.699 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.700 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:44:14.700 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:14.700 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.700 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:14.701 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6822", 2022-06-29T15:44:14.701 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:44:14.701 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.701 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.701 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:14.702 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6823", 2022-06-29T15:44:14.702 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:44:14.702 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:14.702 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:14.703 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.703 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6819/13096", 2022-06-29T15:44:14.703 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6821/13096", 2022-06-29T15:44:14.703 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6825/13096", 2022-06-29T15:44:14.703 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6823/13096", 2022-06-29T15:44:14.704 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:44:14.704 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:44:14.704 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:44:14.704 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:14.705 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.705 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.705 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2022-06-29T15:44:14.705 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "228bb08d-2d64-43d9-9560-bfaeff154452", 2022-06-29T15:44:14.705 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:44:14.706 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:44:14.706 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:44:14.706 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:44:14.706 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:44:14.706 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:44:14.707 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:44:14.707 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 34, 2022-06-29T15:44:14.707 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:44:14.707 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:44:14.708 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:44:14.708 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:14.708 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.708 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:14.708 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6826", 2022-06-29T15:44:14.709 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:44:14.709 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.709 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.709 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:14.710 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6827", 2022-06-29T15:44:14.710 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:44:14.710 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:14.710 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:14.710 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.711 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:44:14.711 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:14.711 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.711 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:14.712 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6828", 2022-06-29T15:44:14.712 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:44:14.712 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.712 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.712 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:14.713 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6829", 2022-06-29T15:44:14.713 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:44:14.713 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:14.713 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:14.714 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.714 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:44:14.714 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:14.714 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.714 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:14.715 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6832", 2022-06-29T15:44:14.715 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:44:14.715 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.715 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.716 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:14.716 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6833", 2022-06-29T15:44:14.716 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:44:14.716 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:14.716 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:14.717 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.717 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:44:14.717 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:14.717 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.718 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:14.718 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6830", 2022-06-29T15:44:14.718 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:44:14.718 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.719 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.719 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:14.719 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6831", 2022-06-29T15:44:14.719 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:44:14.719 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:14.720 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:14.720 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.720 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6827/13100", 2022-06-29T15:44:14.720 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6829/13100", 2022-06-29T15:44:14.721 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6833/13100", 2022-06-29T15:44:14.721 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6831/13100", 2022-06-29T15:44:14.721 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:44:14.721 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:44:14.721 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:44:14.722 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:14.722 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.722 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.722 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2022-06-29T15:44:14.722 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "797a5452-1510-4640-a111-c9c2fcf707eb", 2022-06-29T15:44:14.723 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:44:14.723 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:44:14.723 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:44:14.723 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:44:14.724 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:44:14.724 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:44:14.724 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:44:14.724 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 33, 2022-06-29T15:44:14.724 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:44:14.725 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:44:14.725 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:44:14.725 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:14.725 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.726 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:14.726 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6810", 2022-06-29T15:44:14.726 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:44:14.726 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.726 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.727 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:14.727 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6811", 2022-06-29T15:44:14.727 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:44:14.727 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:14.728 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:14.728 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.728 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:44:14.728 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:14.729 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.729 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:14.729 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6812", 2022-06-29T15:44:14.729 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:44:14.729 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.730 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.730 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:14.730 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6813", 2022-06-29T15:44:14.730 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:44:14.730 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:14.731 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:14.731 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.731 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:44:14.731 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:14.732 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.732 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:14.732 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6816", 2022-06-29T15:44:14.732 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:44:14.732 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.733 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.733 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:14.733 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6817", 2022-06-29T15:44:14.733 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:44:14.733 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:14.734 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:14.734 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.734 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:44:14.734 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:14.735 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.735 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:14.735 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6814", 2022-06-29T15:44:14.735 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:44:14.736 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.736 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.736 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:14.736 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6815", 2022-06-29T15:44:14.736 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:44:14.737 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:14.737 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:14.737 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.737 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6811/13098", 2022-06-29T15:44:14.737 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6813/13098", 2022-06-29T15:44:14.738 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6817/13098", 2022-06-29T15:44:14.738 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6815/13098", 2022-06-29T15:44:14.738 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:44:14.738 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:44:14.739 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:44:14.739 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:14.739 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.739 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.739 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2022-06-29T15:44:14.740 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "55b24fd7-bb71-4a01-b3fa-a09caa772a62", 2022-06-29T15:44:14.740 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:44:14.740 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:44:14.740 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:44:14.740 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:44:14.741 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:44:14.741 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:44:14.741 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:44:14.741 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 34, 2022-06-29T15:44:14.742 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:44:14.742 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:44:14.742 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:44:14.742 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:14.742 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.743 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:14.743 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6802", 2022-06-29T15:44:14.743 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:44:14.743 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.744 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.744 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:14.744 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6803", 2022-06-29T15:44:14.744 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:44:14.744 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:14.745 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:14.745 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.745 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:44:14.745 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:14.745 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.746 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:14.746 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6804", 2022-06-29T15:44:14.746 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:44:14.746 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.747 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.747 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:14.747 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6805", 2022-06-29T15:44:14.747 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:44:14.747 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:14.748 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:14.748 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.748 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:44:14.748 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:14.749 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.749 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:14.749 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6808", 2022-06-29T15:44:14.749 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:44:14.749 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.750 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.750 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:14.750 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6809", 2022-06-29T15:44:14.750 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:44:14.751 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:14.751 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:14.751 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.751 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:44:14.751 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:14.752 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.752 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:14.752 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6806", 2022-06-29T15:44:14.752 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:44:14.753 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.753 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.753 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:14.753 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6807", 2022-06-29T15:44:14.754 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:44:14.754 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:14.754 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:14.754 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.754 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6803/13097", 2022-06-29T15:44:14.755 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6805/13097", 2022-06-29T15:44:14.755 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6809/13097", 2022-06-29T15:44:14.755 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6807/13097", 2022-06-29T15:44:14.755 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:44:14.756 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:44:14.756 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:44:14.756 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:14.756 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.756 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.757 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2022-06-29T15:44:14.757 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3", 2022-06-29T15:44:14.757 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:44:14.757 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:44:14.757 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:44:14.758 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:44:14.758 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:44:14.758 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:44:14.758 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:44:14.759 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 34, 2022-06-29T15:44:14.759 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:44:14.759 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:44:14.759 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:44:14.759 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:14.760 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.760 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:14.760 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6816", 2022-06-29T15:44:14.760 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:44:14.761 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.761 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.761 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:14.761 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6817", 2022-06-29T15:44:14.762 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:44:14.762 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:14.762 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:14.762 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.762 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:44:14.763 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:14.763 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.763 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:14.763 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6818", 2022-06-29T15:44:14.763 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:44:14.764 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.764 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.764 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:14.764 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6819", 2022-06-29T15:44:14.765 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:44:14.765 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:14.765 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:14.765 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.765 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:44:14.766 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:14.766 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.766 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:14.766 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6822", 2022-06-29T15:44:14.767 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:44:14.767 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.767 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.767 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:14.767 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6823", 2022-06-29T15:44:14.768 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:44:14.768 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:14.768 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:14.768 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.769 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:44:14.769 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:14.769 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.769 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:14.769 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6820", 2022-06-29T15:44:14.770 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:44:14.770 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.770 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.770 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:14.771 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6821", 2022-06-29T15:44:14.771 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:44:14.771 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:14.771 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:14.771 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.772 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6817/13396", 2022-06-29T15:44:14.772 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6819/13396", 2022-06-29T15:44:14.772 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6823/13396", 2022-06-29T15:44:14.772 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6821/13396", 2022-06-29T15:44:14.772 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:44:14.773 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:44:14.773 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:44:14.773 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:14.773 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.774 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.774 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2022-06-29T15:44:14.774 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "f77beebe-6cca-4244-b2d4-677825ebf919", 2022-06-29T15:44:14.774 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:44:14.774 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:44:14.775 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:44:14.775 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:44:14.775 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:44:14.775 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:44:14.776 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:44:14.776 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 34, 2022-06-29T15:44:14.776 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:44:14.776 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:44:14.776 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:44:14.777 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:14.777 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.777 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:14.777 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6800", 2022-06-29T15:44:14.778 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:44:14.778 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.778 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.778 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:14.778 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6801", 2022-06-29T15:44:14.779 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:44:14.779 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:14.779 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:14.779 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.779 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:44:14.780 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:14.780 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.780 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:14.780 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6802", 2022-06-29T15:44:14.781 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:44:14.781 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.781 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.781 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:14.781 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6803", 2022-06-29T15:44:14.782 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:44:14.782 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:14.782 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:14.782 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.783 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:44:14.783 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:14.783 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.783 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:14.783 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6806", 2022-06-29T15:44:14.784 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:44:14.784 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.784 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.784 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:14.785 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6807", 2022-06-29T15:44:14.785 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:44:14.785 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:14.785 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:14.786 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.786 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:44:14.786 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:14.786 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.787 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:14.787 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6804", 2022-06-29T15:44:14.787 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:44:14.787 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.787 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.788 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:14.788 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6805", 2022-06-29T15:44:14.788 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:44:14.788 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:14.789 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:14.789 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.789 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6801/13393", 2022-06-29T15:44:14.789 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6803/13393", 2022-06-29T15:44:14.789 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6807/13393", 2022-06-29T15:44:14.790 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6805/13393", 2022-06-29T15:44:14.790 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:44:14.790 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:44:14.790 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:44:14.791 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:14.791 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.791 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.791 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2022-06-29T15:44:14.791 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "af0ca9b3-c02c-4910-834b-bf0b8dc6713c", 2022-06-29T15:44:14.792 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:44:14.792 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:44:14.792 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:44:14.792 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:44:14.793 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:44:14.793 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:44:14.793 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:44:14.793 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 33, 2022-06-29T15:44:14.794 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:44:14.794 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:44:14.794 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:44:14.794 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:14.794 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.795 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:14.795 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6824", 2022-06-29T15:44:14.795 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:44:14.795 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.795 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.796 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:14.796 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6825", 2022-06-29T15:44:14.796 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:44:14.796 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:14.796 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:14.797 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.797 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:44:14.797 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:14.797 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.798 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:14.798 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6826", 2022-06-29T15:44:14.798 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:44:14.798 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.798 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.799 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:14.799 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6827", 2022-06-29T15:44:14.799 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:44:14.799 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:14.799 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:14.800 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.800 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:44:14.800 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:14.800 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.801 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:14.801 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6830", 2022-06-29T15:44:14.801 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:44:14.801 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.801 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.802 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:14.802 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6831", 2022-06-29T15:44:14.802 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:44:14.802 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:14.803 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:14.803 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.803 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:44:14.803 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:14.803 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.804 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:14.804 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6828", 2022-06-29T15:44:14.804 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:44:14.804 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.804 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.805 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:14.805 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6829", 2022-06-29T15:44:14.805 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:44:14.805 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:14.806 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:14.806 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.806 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6825/13395", 2022-06-29T15:44:14.806 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6827/13395", 2022-06-29T15:44:14.806 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6831/13395", 2022-06-29T15:44:14.807 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6829/13395", 2022-06-29T15:44:14.807 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:44:14.807 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:44:14.807 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:44:14.808 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:14.808 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.808 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.808 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2022-06-29T15:44:14.808 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "f19c3ca6-df88-4bde-a150-b51a53f092e1", 2022-06-29T15:44:14.809 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:44:14.809 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:44:14.809 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:44:14.809 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:44:14.810 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:44:14.810 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:44:14.810 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:44:14.810 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 34, 2022-06-29T15:44:14.810 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:44:14.811 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:44:14.811 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:44:14.811 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:14.811 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.812 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:14.812 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6808", 2022-06-29T15:44:14.812 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:44:14.812 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.812 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.813 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:14.813 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6809", 2022-06-29T15:44:14.813 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:44:14.813 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:14.814 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:14.814 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.814 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:44:14.814 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:14.814 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.815 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:14.815 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6810", 2022-06-29T15:44:14.815 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:44:14.815 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.816 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.816 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:14.816 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6811", 2022-06-29T15:44:14.816 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:44:14.816 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:14.817 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:14.817 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.817 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:44:14.817 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:14.817 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.818 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:14.818 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6814", 2022-06-29T15:44:14.818 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:44:14.818 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.819 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.819 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:14.819 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6815", 2022-06-29T15:44:14.819 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:44:14.819 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:14.820 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:14.820 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.820 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:44:14.820 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:14.821 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.821 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:14.821 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6812", 2022-06-29T15:44:14.821 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:44:14.821 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.822 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.822 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:14.822 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6813", 2022-06-29T15:44:14.822 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:44:14.823 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:14.823 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:14.823 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.823 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6809/13394", 2022-06-29T15:44:14.823 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6811/13394", 2022-06-29T15:44:14.824 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6815/13394", 2022-06-29T15:44:14.824 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6813/13394", 2022-06-29T15:44:14.824 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:44:14.824 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:44:14.824 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:44:14.825 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:14.825 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:14.825 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:44:14.825 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_xinfo": [ 2022-06-29T15:44:14.825 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.826 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2022-06-29T15:44:14.826 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:44:14.826 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:44:14.826 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:44:14.827 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:44:14.827 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:44:14.827 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.720513+0000", 2022-06-29T15:44:14.827 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:44:14.827 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.828 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.828 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2022-06-29T15:44:14.828 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:44:14.828 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:44:14.829 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:44:14.829 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:44:14.829 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:44:14.829 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.772612+0000", 2022-06-29T15:44:14.829 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:44:14.830 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.830 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.830 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2022-06-29T15:44:14.830 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:44:14.831 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:44:14.831 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:44:14.831 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:44:14.831 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:44:14.831 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.450223+0000", 2022-06-29T15:44:14.832 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:44:14.832 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.832 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.832 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2022-06-29T15:44:14.832 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:44:14.833 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:44:14.833 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:44:14.833 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:44:14.833 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:44:14.834 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.477856+0000", 2022-06-29T15:44:14.834 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:44:14.834 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.834 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.834 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2022-06-29T15:44:14.835 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:44:14.835 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:44:14.835 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:44:14.835 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:44:14.835 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:44:14.836 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.539485+0000", 2022-06-29T15:44:14.836 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:44:14.836 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.836 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.837 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2022-06-29T15:44:14.837 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:44:14.837 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:44:14.837 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:44:14.837 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:44:14.838 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:44:14.838 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.487040+0000", 2022-06-29T15:44:14.838 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:44:14.838 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.839 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.839 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2022-06-29T15:44:14.839 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:44:14.839 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:44:14.839 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:44:14.840 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:44:14.840 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:44:14.840 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.748685+0000", 2022-06-29T15:44:14.840 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:44:14.841 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.841 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:14.841 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2022-06-29T15:44:14.841 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:44:14.841 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:44:14.842 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:44:14.843 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:44:14.843 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:44:14.843 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.459516+0000", 2022-06-29T15:44:14.843 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:44:14.844 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:14.844 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:44:14.844 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap": [], 2022-06-29T15:44:14.844 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap_items": [], 2022-06-29T15:44:14.844 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_temp": [], 2022-06-29T15:44:14.845 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_temp": [], 2022-06-29T15:44:14.845 INFO:teuthology.orchestra.run.smithi047.stdout: "blacklist": {}, 2022-06-29T15:44:14.845 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profiles": { 2022-06-29T15:44:14.845 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs_data": { 2022-06-29T15:44:14.846 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-device-class": "", 2022-06-29T15:44:14.846 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2022-06-29T15:44:14.846 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-root": "default", 2022-06-29T15:44:14.846 INFO:teuthology.orchestra.run.smithi047.stdout: "jerasure-per-chunk-alignment": "false", 2022-06-29T15:44:14.846 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2022-06-29T15:44:14.847 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "2", 2022-06-29T15:44:14.847 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2022-06-29T15:44:14.847 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2022-06-29T15:44:14.847 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van", 2022-06-29T15:44:14.848 INFO:teuthology.orchestra.run.smithi047.stdout: "w": "8" 2022-06-29T15:44:14.848 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.848 INFO:teuthology.orchestra.run.smithi047.stdout: "default": { 2022-06-29T15:44:14.848 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2022-06-29T15:44:14.848 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2022-06-29T15:44:14.849 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "1", 2022-06-29T15:44:14.849 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2022-06-29T15:44:14.849 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2022-06-29T15:44:14.849 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van" 2022-06-29T15:44:14.850 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:14.850 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:14.850 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps_queue": [], 2022-06-29T15:44:14.850 INFO:teuthology.orchestra.run.smithi047.stdout: "new_removed_snaps": [], 2022-06-29T15:44:14.850 INFO:teuthology.orchestra.run.smithi047.stdout: "new_purged_snaps": [], 2022-06-29T15:44:14.851 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_node_flags": {}, 2022-06-29T15:44:14.851 INFO:teuthology.orchestra.run.smithi047.stdout: "device_class_flags": {} 2022-06-29T15:44:14.851 INFO:teuthology.orchestra.run.smithi047.stdout:} 2022-06-29T15:44:14.853 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.591+0000 7f1d5abd1700 1 -- 172.21.15.47:0/1241488080 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1d340539e0 msgr2=0x7f1d34055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:14.853 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.591+0000 7f1d5abd1700 1 --2- 172.21.15.47:0/1241488080 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1d340539e0 0x7f1d34055ea0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f1d4400d000 tx=0x7f1d440079e0).stop 2022-06-29T15:44:14.854 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.591+0000 7f1d5abd1700 1 -- 172.21.15.47:0/1241488080 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1d540619b0 msgr2=0x7f1d54065d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:14.854 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.591+0000 7f1d5abd1700 1 --2- 172.21.15.47:0/1241488080 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1d540619b0 0x7f1d54065d40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f1d50002f70 tx=0x7f1d50006f40).stop 2022-06-29T15:44:14.854 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.591+0000 7f1d5abd1700 1 -- 172.21.15.47:0/1241488080 shutdown_connections 2022-06-29T15:44:14.854 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.591+0000 7f1d5abd1700 1 --2- 172.21.15.47:0/1241488080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d541408e0 0x7f1d54066b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:14.855 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.591+0000 7f1d5abd1700 1 --2- 172.21.15.47:0/1241488080 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1d340539e0 0x7f1d34055ea0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:14.855 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.591+0000 7f1d5abd1700 1 --2- 172.21.15.47:0/1241488080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d540b0f30 0x7f1d540b46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:14.855 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.591+0000 7f1d5abd1700 1 --2- 172.21.15.47:0/1241488080 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1d540619b0 0x7f1d54065d40 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:14.855 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.591+0000 7f1d5abd1700 1 -- 172.21.15.47:0/1241488080 >> 172.21.15.47:0/1241488080 conn(0x7f1d5405c510 msgr2=0x7f1d54139980 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:14.855 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.591+0000 7f1d5abd1700 1 -- 172.21.15.47:0/1241488080 shutdown_connections 2022-06-29T15:44:14.856 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.591+0000 7f1d5abd1700 1 -- 172.21.15.47:0/1241488080 wait complete. 2022-06-29T15:44:14.856 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:44:14.965 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.959+0000 7f13dcde2700 1 Processor -- start 2022-06-29T15:44:14.965 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.959+0000 7f13dcde2700 1 -- start start 2022-06-29T15:44:14.967 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.959+0000 7f13dcde2700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f13d80b1260 0x7f13d805ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:14.967 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.959+0000 7f13dcde2700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13d80b0740 con 0x7f13d80b2120 2022-06-29T15:44:14.967 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.959+0000 7f13dcde2700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13d80b0970 con 0x7f13d805f0b0 2022-06-29T15:44:14.968 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.959+0000 7f13dcde2700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13d8147460 con 0x7f13d80b1260 2022-06-29T15:44:14.968 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.959+0000 7f13d6d9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f13d80b1260 0x7f13d805ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:14.968 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.959+0000 7f13d659c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f13d80b2120 0x7f13d8147af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:32986/0 (socket says 172.21.15.47:32986) 2022-06-29T15:44:14.968 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.959+0000 7f13d659c700 1 -- 172.21.15.47:0/1754844621 learned_addr learned my addr 172.21.15.47:0/1754844621 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:44:14.969 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.959+0000 7f13d6d9d700 1 -- 172.21.15.47:0/1754844621 >> v1:172.21.15.64:6789/0 conn(0x7f13d805f0b0 legacy=0x7f13d805f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:14.969 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.959+0000 7f13d6d9d700 1 -- 172.21.15.47:0/1754844621 >> v1:172.21.15.47:6789/0 conn(0x7f13d80b2120 legacy=0x7f13d8147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:14.969 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.959+0000 7f13d6d9d700 1 -- 172.21.15.47:0/1754844621 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d80b0c10 con 0x7f13d80b1260 2022-06-29T15:44:14.969 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.959+0000 7f13d559a700 1 -- 172.21.15.47:0/1754844621 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2018492753 0 0) 0x7f13d80b0740 con 0x7f13d80b2120 2022-06-29T15:44:14.970 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.963+0000 7f13d6d9d700 1 --2- 172.21.15.47:0/1754844621 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13d80b1260 0x7f13d805ffc0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f13d8141a70 tx=0x7f13d00091d0).ready entity=mon.0 client_cookie=a9f861bbbbd1ee3c server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:14.973 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.963+0000 7f13d559a700 1 -- 172.21.15.47:0/1754844621 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f13d000b820 con 0x7f13d80b1260 2022-06-29T15:44:14.973 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.963+0000 7f13d559a700 1 -- 172.21.15.47:0/1754844621 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13d000b980 con 0x7f13d80b1260 2022-06-29T15:44:14.974 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.963+0000 7f13d559a700 1 -- 172.21.15.47:0/1754844621 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f13d000baf0 con 0x7f13d80b1260 2022-06-29T15:44:14.974 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.963+0000 7f13dcde2700 1 -- 172.21.15.47:0/1754844621 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13d80b1260 msgr2=0x7f13d805ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:14.974 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.963+0000 7f13dcde2700 1 --2- 172.21.15.47:0/1754844621 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13d80b1260 0x7f13d805ffc0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f13d8141a70 tx=0x7f13d00091d0).stop 2022-06-29T15:44:14.974 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.963+0000 7f13dcde2700 1 -- 172.21.15.47:0/1754844621 shutdown_connections 2022-06-29T15:44:14.975 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.963+0000 7f13dcde2700 1 --2- 172.21.15.47:0/1754844621 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13d80b1260 0x7f13d805ffc0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:14.975 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.963+0000 7f13dcde2700 1 -- 172.21.15.47:0/1754844621 >> 172.21.15.47:0/1754844621 conn(0x7f13d80626f0 msgr2=0x7f13d8062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:14.975 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.963+0000 7f13dcde2700 1 -- 172.21.15.47:0/1754844621 shutdown_connections 2022-06-29T15:44:14.975 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.963+0000 7f13dcde2700 1 -- 172.21.15.47:0/1754844621 wait complete. 2022-06-29T15:44:14.976 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.963+0000 7f13dcde2700 1 Processor -- start 2022-06-29T15:44:14.976 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.963+0000 7f13dcde2700 1 -- start start 2022-06-29T15:44:14.976 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.967+0000 7f13dcde2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13d80b1260 0x7f13d8149df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:14.976 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.967+0000 7f13dcde2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13d80b2120 0x7f13d8141ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:14.977 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.967+0000 7f13dcde2700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f13d8141fa0 0x7f13d815a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:14.977 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.967+0000 7f13dcde2700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d80ae640 con 0x7f13d80b2120 2022-06-29T15:44:14.977 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.967+0000 7f13dcde2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d80ae8e0 con 0x7f13d80b1260 2022-06-29T15:44:14.977 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.967+0000 7f13dcde2700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13d80aeb80 con 0x7f13d8141fa0 2022-06-29T15:44:14.977 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.967+0000 7f13d5d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13d80b2120 0x7f13d8141ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:14.978 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.967+0000 7f13d5d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13d80b2120 0x7f13d8141ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47438/0 (socket says 172.21.15.47:47438) 2022-06-29T15:44:14.978 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.967+0000 7f13d5d9b700 1 -- 172.21.15.47:0/3199971280 learned_addr learned my addr 172.21.15.47:0/3199971280 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:14.979 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.967+0000 7f13d6d9d700 1 --2- 172.21.15.47:0/3199971280 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f13d8141fa0 0x7f13d815a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:14.979 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.967+0000 7f13d659c700 1 --2- 172.21.15.47:0/3199971280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13d80b1260 0x7f13d8149df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:14.979 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.967+0000 7f13d5d9b700 1 -- 172.21.15.47:0/3199971280 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f13d8141fa0 msgr2=0x7f13d815a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:14.979 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.967+0000 7f13d5d9b700 1 --2- 172.21.15.47:0/3199971280 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f13d8141fa0 0x7f13d815a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:14.979 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.967+0000 7f13d5d9b700 1 -- 172.21.15.47:0/3199971280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13d80b1260 msgr2=0x7f13d8149df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:14.980 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.967+0000 7f13d5d9b700 1 --2- 172.21.15.47:0/3199971280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13d80b1260 0x7f13d8149df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:14.980 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.967+0000 7f13d5d9b700 1 -- 172.21.15.47:0/3199971280 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d0009470 con 0x7f13d80b2120 2022-06-29T15:44:14.980 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.967+0000 7f13d6d9d700 1 --2- 172.21.15.47:0/3199971280 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f13d8141fa0 0x7f13d815a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:44:14.980 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.967+0000 7f13d659c700 1 --2- 172.21.15.47:0/3199971280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13d80b1260 0x7f13d8149df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:44:14.981 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.967+0000 7f13d5d9b700 1 --2- 172.21.15.47:0/3199971280 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13d80b2120 0x7f13d8141ab0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7f13d80afcc0 tx=0x7f13c8007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:14.981 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.967+0000 7f13c77fe700 1 -- 172.21.15.47:0/3199971280 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f13c800ce30 con 0x7f13d80b2120 2022-06-29T15:44:14.981 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.967+0000 7f13c77fe700 1 -- 172.21.15.47:0/3199971280 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13c800f470 con 0x7f13d80b2120 2022-06-29T15:44:14.981 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.967+0000 7f13dcde2700 1 -- 172.21.15.47:0/3199971280 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13d80adb50 con 0x7f13d80b2120 2022-06-29T15:44:14.982 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.967+0000 7f13c77fe700 1 -- 172.21.15.47:0/3199971280 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f13c800f7d0 con 0x7f13d80b2120 2022-06-29T15:44:14.982 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.967+0000 7f13dcde2700 1 -- 172.21.15.47:0/3199971280 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13d8144820 con 0x7f13d80b2120 2022-06-29T15:44:14.983 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.967+0000 7f13dcde2700 1 -- 172.21.15.47:0/3199971280 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13d8147650 con 0x7f13d80b2120 2022-06-29T15:44:14.983 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.967+0000 7f13c77fe700 1 -- 172.21.15.47:0/3199971280 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f13c8019030 con 0x7f13d80b2120 2022-06-29T15:44:14.984 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.971+0000 7f13c77fe700 1 --2- 172.21.15.47:0/3199971280 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f13b4053a70 0x7f13b4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:14.984 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.971+0000 7f13c77fe700 1 -- 172.21.15.47:0/3199971280 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f13c806b490 con 0x7f13d80b2120 2022-06-29T15:44:14.984 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.971+0000 7f13d659c700 1 --2- 172.21.15.47:0/3199971280 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f13b4053a70 0x7f13b4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:14.984 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.971+0000 7f13d659c700 1 --2- 172.21.15.47:0/3199971280 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f13b4053a70 0x7f13b4055f30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f13cc00b000 tx=0x7f13cc009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:14.985 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:14.975+0000 7f13c77fe700 1 -- 172.21.15.47:0/3199971280 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f13c8042250 con 0x7f13d80b2120 2022-06-29T15:44:15.144 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.139+0000 7f13dcde2700 1 -- 172.21.15.47:0/3199971280 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f13d80573d0 con 0x7f13d80b2120 2022-06-29T15:44:15.149 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.143+0000 7f13c77fe700 1 -- 172.21.15.47:0/3199971280 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+2488 (secure 0 0 0) 0x7f13c8045870 con 0x7f13d80b2120 2022-06-29T15:44:15.149 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:44:15.150 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":12,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4896,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1047560169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1047560169},{"type":"v1","addr":"172.21.15.47:6835","nonce":1047560169}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":10},{"gid":4902,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/804808837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":804808837},{"type":"v1","addr":"172.21.15.64:6833","nonce":804808837}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":10}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:43:10.546977+0000","modified":"2022-06-29T15:43:11.549959+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4905},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4905":{"gid":4905,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6837/2219068343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2219068343},{"type":"v1","addr":"172.21.15.47:6837","nonce":2219068343}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2022-06-29T15:44:15.151 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.143+0000 7f13dcde2700 1 -- 172.21.15.47:0/3199971280 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f13b4053a70 msgr2=0x7f13b4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:15.152 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.143+0000 7f13dcde2700 1 --2- 172.21.15.47:0/3199971280 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f13b4053a70 0x7f13b4055f30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f13cc00b000 tx=0x7f13cc009040).stop 2022-06-29T15:44:15.152 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.143+0000 7f13dcde2700 1 -- 172.21.15.47:0/3199971280 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13d80b2120 msgr2=0x7f13d8141ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:15.152 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.143+0000 7f13dcde2700 1 --2- 172.21.15.47:0/3199971280 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13d80b2120 0x7f13d8141ab0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7f13d80afcc0 tx=0x7f13c8007c20).stop 2022-06-29T15:44:15.152 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.147+0000 7f13dcde2700 1 -- 172.21.15.47:0/3199971280 shutdown_connections 2022-06-29T15:44:15.153 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.147+0000 7f13dcde2700 1 --2- 172.21.15.47:0/3199971280 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f13d8141fa0 0x7f13d815a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:15.153 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.147+0000 7f13dcde2700 1 --2- 172.21.15.47:0/3199971280 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13d80b2120 0x7f13d8141ab0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:15.153 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.147+0000 7f13dcde2700 1 --2- 172.21.15.47:0/3199971280 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f13b4053a70 0x7f13b4055f30 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:15.153 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.147+0000 7f13dcde2700 1 --2- 172.21.15.47:0/3199971280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13d80b1260 0x7f13d8149df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:15.154 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.147+0000 7f13dcde2700 1 -- 172.21.15.47:0/3199971280 >> 172.21.15.47:0/3199971280 conn(0x7f13d80626f0 msgr2=0x7f13d80b44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:15.154 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.147+0000 7f13dcde2700 1 -- 172.21.15.47:0/3199971280 shutdown_connections 2022-06-29T15:44:15.154 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.147+0000 7f13dcde2700 1 -- 172.21.15.47:0/3199971280 wait complete. 2022-06-29T15:44:15.155 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 12 2022-06-29T15:44:15.163 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-06-29T15:44:15.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.267+0000 7f4bcb59e700 1 Processor -- start 2022-06-29T15:44:15.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.267+0000 7f4bcb59e700 1 -- start start 2022-06-29T15:44:15.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.267+0000 7f4bcb59e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4bcc0af590 0x7f4bcc0ad0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:15.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.267+0000 7f4bcb59e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bcc0b1850 con 0x7f4bcc0b04e0 2022-06-29T15:44:15.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.267+0000 7f4bcb59e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bcc142d50 con 0x7f4bcc0b1470 2022-06-29T15:44:15.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.267+0000 7f4bcb59e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bcc0b27c0 con 0x7f4bcc0af590 2022-06-29T15:44:15.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.267+0000 7f4bcad9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4bcc0af590 0x7f4bcc0ad0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:15.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.267+0000 7f4bcad9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4bcc0af590 0x7f4bcc0ad0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47448/0 (socket says 172.21.15.47:47448) 2022-06-29T15:44:15.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.267+0000 7f4bcad9d700 1 -- 172.21.15.47:0/943902523 learned_addr learned my addr 172.21.15.47:0/943902523 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:15.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.267+0000 7f4bc9d9b700 1 --1- 172.21.15.47:0/943902523 >> v1:172.21.15.47:6789/0 conn(0x7f4bcc0b04e0 0x7f4bcc143090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33000/0 (socket says 172.21.15.47:33000) 2022-06-29T15:44:15.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.271+0000 7f4bcad9d700 1 -- 172.21.15.47:0/943902523 >> v1:172.21.15.64:6789/0 conn(0x7f4bcc0b1470 legacy=0x7f4bcc0ac9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:15.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.271+0000 7f4bcad9d700 1 -- 172.21.15.47:0/943902523 >> v1:172.21.15.47:6789/0 conn(0x7f4bcc0b04e0 legacy=0x7f4bcc143090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:15.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.271+0000 7f4bc959a700 1 -- 172.21.15.47:0/943902523 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 112409376 0 0) 0x7f4bcc0b1850 con 0x7f4bcc0b04e0 2022-06-29T15:44:15.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.271+0000 7f4bcad9d700 1 -- 172.21.15.47:0/943902523 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bcc0ae9a0 con 0x7f4bcc0af590 2022-06-29T15:44:15.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.271+0000 7f4bcad9d700 1 --2- 172.21.15.47:0/943902523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bcc0af590 0x7f4bcc0ad0d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f4bc40032f0 tx=0x7f4bc400b530).ready entity=mon.0 client_cookie=7bb5f11edcfc571a server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:15.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.271+0000 7f4bc959a700 1 -- 172.21.15.47:0/943902523 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4bc4010660 con 0x7f4bcc0af590 2022-06-29T15:44:15.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.271+0000 7f4bc959a700 1 -- 172.21.15.47:0/943902523 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bc40107c0 con 0x7f4bcc0af590 2022-06-29T15:44:15.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.271+0000 7f4bc959a700 1 -- 172.21.15.47:0/943902523 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4bc4010b20 con 0x7f4bcc0af590 2022-06-29T15:44:15.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.271+0000 7f4bcb59e700 1 -- 172.21.15.47:0/943902523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bcc0af590 msgr2=0x7f4bcc0ad0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:15.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.271+0000 7f4bcb59e700 1 --2- 172.21.15.47:0/943902523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bcc0af590 0x7f4bcc0ad0d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f4bc40032f0 tx=0x7f4bc400b530).stop 2022-06-29T15:44:15.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.271+0000 7f4bcb59e700 1 -- 172.21.15.47:0/943902523 shutdown_connections 2022-06-29T15:44:15.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.271+0000 7f4bcb59e700 1 --2- 172.21.15.47:0/943902523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bcc0af590 0x7f4bcc0ad0d0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:15.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.271+0000 7f4bcb59e700 1 -- 172.21.15.47:0/943902523 >> 172.21.15.47:0/943902523 conn(0x7f4bcc062560 msgr2=0x7f4bcc062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:15.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.271+0000 7f4bcb59e700 1 -- 172.21.15.47:0/943902523 shutdown_connections 2022-06-29T15:44:15.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.271+0000 7f4bcb59e700 1 -- 172.21.15.47:0/943902523 wait complete. 2022-06-29T15:44:15.282 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.275+0000 7f4bcb59e700 1 Processor -- start 2022-06-29T15:44:15.282 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.275+0000 7f4bcb59e700 1 -- start start 2022-06-29T15:44:15.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.275+0000 7f4bcb59e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4bcc0af590 0x7f4bcc066a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:15.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.275+0000 7f4bcb59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bcc0b04e0 0x7f4bcc1528f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:15.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.275+0000 7f4bcb59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bcc0b1470 0x7f4bcc152de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:15.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.275+0000 7f4bcb59e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bcc1532d0 con 0x7f4bcc0b1470 2022-06-29T15:44:15.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.275+0000 7f4bcb59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bcc1430d0 con 0x7f4bcc0b04e0 2022-06-29T15:44:15.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.275+0000 7f4bcb59e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bcc143370 con 0x7f4bcc0af590 2022-06-29T15:44:15.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.275+0000 7f4bcad9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bcc0b1470 0x7f4bcc152de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:15.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.275+0000 7f4bc9d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bcc0b04e0 0x7f4bcc1528f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:15.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.275+0000 7f4bca59c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4bcc0af590 0x7f4bcc066a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:15.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.275+0000 7f4bc9d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bcc0b04e0 0x7f4bcc1528f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:53156/0 (socket says 172.21.15.47:53156) 2022-06-29T15:44:15.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.275+0000 7f4bc9d9b700 1 -- 172.21.15.47:0/1072779857 learned_addr learned my addr 172.21.15.47:0/1072779857 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:15.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.275+0000 7f4bcad9d700 1 -- 172.21.15.47:0/1072779857 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4bcc0af590 msgr2=0x7f4bcc066a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:15.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.275+0000 7f4bcad9d700 1 --2- 172.21.15.47:0/1072779857 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4bcc0af590 0x7f4bcc066a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:15.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.275+0000 7f4bcad9d700 1 -- 172.21.15.47:0/1072779857 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bcc0b04e0 msgr2=0x7f4bcc1528f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:15.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.275+0000 7f4bcad9d700 1 --2- 172.21.15.47:0/1072779857 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bcc0b04e0 0x7f4bcc1528f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:15.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.275+0000 7f4bcad9d700 1 -- 172.21.15.47:0/1072779857 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc400b7d0 con 0x7f4bcc0b1470 2022-06-29T15:44:15.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.275+0000 7f4bc9d9b700 1 --2- 172.21.15.47:0/1072779857 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bcc0b04e0 0x7f4bcc1528f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:44:15.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.275+0000 7f4bcad9d700 1 --2- 172.21.15.47:0/1072779857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bcc0b1470 0x7f4bcc152de0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f4bc40032f0 tx=0x7f4bcc0b2a80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:15.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.275+0000 7f4bbb7fe700 1 -- 172.21.15.47:0/1072779857 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4bc401d070 con 0x7f4bcc0b1470 2022-06-29T15:44:15.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.275+0000 7f4bbb7fe700 1 -- 172.21.15.47:0/1072779857 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bc4010930 con 0x7f4bcc0b1470 2022-06-29T15:44:15.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.275+0000 7f4bcb59e700 1 -- 172.21.15.47:0/1072779857 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bcc143610 con 0x7f4bcc0b1470 2022-06-29T15:44:15.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.275+0000 7f4bbb7fe700 1 -- 172.21.15.47:0/1072779857 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4bc4010660 con 0x7f4bcc0b1470 2022-06-29T15:44:15.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.275+0000 7f4bcb59e700 1 -- 172.21.15.47:0/1072779857 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bcc1523e0 con 0x7f4bcc0b1470 2022-06-29T15:44:15.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.279+0000 7f4bcb59e700 1 -- 172.21.15.47:0/1072779857 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bcc152630 con 0x7f4bcc0b1470 2022-06-29T15:44:15.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.279+0000 7f4bbb7fe700 1 -- 172.21.15.47:0/1072779857 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4bc400f3d0 con 0x7f4bcc0b1470 2022-06-29T15:44:15.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.279+0000 7f4bbb7fe700 1 --2- 172.21.15.47:0/1072779857 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4ba8053aa0 0x7f4ba8055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:15.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.279+0000 7f4bbb7fe700 1 -- 172.21.15.47:0/1072779857 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f4bc406d520 con 0x7f4bcc0b1470 2022-06-29T15:44:15.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.279+0000 7f4bca59c700 1 --2- 172.21.15.47:0/1072779857 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4ba8053aa0 0x7f4ba8055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:15.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.279+0000 7f4bca59c700 1 --2- 172.21.15.47:0/1072779857 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4ba8053aa0 0x7f4ba8055f60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f4bbc00d000 tx=0x7f4bbc007bc0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:15.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.283+0000 7f4bbb7fe700 1 -- 172.21.15.47:0/1072779857 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4bcc152630 con 0x7f4bcc0b1470 2022-06-29T15:44:15.452 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:15.447+0000 7f4bcb59e700 1 -- 172.21.15.47:0/1072779857 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f4bcc152630 con 0x7f4bcc0b1470 2022-06-29T15:44:16.152 INFO:tasks.ceph.mon.a.smithi047.stderr:2022-06-29T15:44:16.147+0000 7ff9cf83c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-06-29T15:44:16.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.159+0000 7f4bbb7fe700 1 -- 172.21.15.47:0/1072779857 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v13) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4bcc152630 con 0x7f4bcc0b1470 2022-06-29T15:44:16.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.163+0000 7f4bcb59e700 1 -- 172.21.15.47:0/1072779857 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4ba8053aa0 msgr2=0x7f4ba8055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:16.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.163+0000 7f4bcb59e700 1 --2- 172.21.15.47:0/1072779857 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4ba8053aa0 0x7f4ba8055f60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f4bbc00d000 tx=0x7f4bbc007bc0).stop 2022-06-29T15:44:16.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.163+0000 7f4bcb59e700 1 -- 172.21.15.47:0/1072779857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bcc0b1470 msgr2=0x7f4bcc152de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:16.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.163+0000 7f4bcb59e700 1 --2- 172.21.15.47:0/1072779857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bcc0b1470 0x7f4bcc152de0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f4bc40032f0 tx=0x7f4bcc0b2a80).stop 2022-06-29T15:44:16.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.163+0000 7f4bcb59e700 1 -- 172.21.15.47:0/1072779857 shutdown_connections 2022-06-29T15:44:16.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.163+0000 7f4bcb59e700 1 --2- 172.21.15.47:0/1072779857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bcc0b1470 0x7f4bcc152de0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:16.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.163+0000 7f4bcb59e700 1 --2- 172.21.15.47:0/1072779857 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4ba8053aa0 0x7f4ba8055f60 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:16.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.163+0000 7f4bcb59e700 1 --2- 172.21.15.47:0/1072779857 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bcc0b04e0 0x7f4bcc1528f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:16.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.163+0000 7f4bcb59e700 1 --2- 172.21.15.47:0/1072779857 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4bcc0af590 0x7f4bcc066a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:16.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.163+0000 7f4bcb59e700 1 -- 172.21.15.47:0/1072779857 >> 172.21.15.47:0/1072779857 conn(0x7f4bcc062560 msgr2=0x7f4bcc0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:16.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.163+0000 7f4bcb59e700 1 -- 172.21.15.47:0/1072779857 shutdown_connections 2022-06-29T15:44:16.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.163+0000 7f4bcb59e700 1 -- 172.21.15.47:0/1072779857 wait complete. 2022-06-29T15:44:16.171 INFO:teuthology.orchestra.run.smithi047.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-06-29T15:44:16.183 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:44:16.203 INFO:tasks.ceph.mds.a.smithi047.stdout:starting mds.a at 2022-06-29T15:44:16.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.279+0000 7f409a204700 1 Processor -- start 2022-06-29T15:44:16.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.279+0000 7f409a204700 1 -- start start 2022-06-29T15:44:16.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.279+0000 7f409a204700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f40940b0cf0 0x7f4094146d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:16.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.279+0000 7f409a204700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f40940b06d0 0x7f409413ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:16.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.279+0000 7f409a204700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40940b4520 con 0x7f40940611b0 2022-06-29T15:44:16.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.279+0000 7f409a204700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40940b4b10 con 0x7f40940b0cf0 2022-06-29T15:44:16.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.279+0000 7f409a204700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40940621d0 con 0x7f40940b06d0 2022-06-29T15:44:16.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.279+0000 7f40937fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f40940611b0 0x7f4094146640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33024/0 (socket says 172.21.15.47:33024) 2022-06-29T15:44:16.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.279+0000 7f40937fe700 1 -- 172.21.15.47:0/2943819819 learned_addr learned my addr 172.21.15.47:0/2943819819 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:44:16.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.279+0000 7f4092ffd700 1 --2- 172.21.15.47:0/2943819819 >> v2:172.21.15.47:3300/0 conn(0x7f40940b0cf0 0x7f4094146d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:16.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.279+0000 7f4093fff700 1 --2- 172.21.15.47:0/2943819819 >> v2:172.21.15.64:3300/0 conn(0x7f40940b06d0 0x7f409413ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:16.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.283+0000 7f40927fc700 1 -- 172.21.15.47:0/2943819819 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2602424859 0 0) 0x7f40940b4520 con 0x7f40940611b0 2022-06-29T15:44:16.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.283+0000 7f40927fc700 1 -- 172.21.15.47:0/2943819819 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4080003260 con 0x7f40940611b0 2022-06-29T15:44:16.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.283+0000 7f4093fff700 1 -- 172.21.15.47:0/2943819819 >> v2:172.21.15.47:3300/0 conn(0x7f40940b0cf0 msgr2=0x7f4094146d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:44:16.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.283+0000 7f4093fff700 1 --2- 172.21.15.47:0/2943819819 >> v2:172.21.15.47:3300/0 conn(0x7f40940b0cf0 0x7f4094146d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:16.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.283+0000 7f4093fff700 1 -- 172.21.15.47:0/2943819819 >> v1:172.21.15.47:6789/0 conn(0x7f40940611b0 legacy=0x7f4094146640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:16.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.283+0000 7f4093fff700 1 -- 172.21.15.47:0/2943819819 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4094061590 con 0x7f40940b06d0 2022-06-29T15:44:16.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.283+0000 7f4093fff700 1 --2- 172.21.15.47:0/2943819819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40940b06d0 0x7f409413ff00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f40940b1c00 tx=0x7f408c0091d0).ready entity=mon.1 client_cookie=460e4cdc11781a0d server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:16.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.283+0000 7f40927fc700 1 -- 172.21.15.47:0/2943819819 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f408c00b5a0 con 0x7f40940b06d0 2022-06-29T15:44:16.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.283+0000 7f40927fc700 1 -- 172.21.15.47:0/2943819819 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f408c00b700 con 0x7f40940b06d0 2022-06-29T15:44:16.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.283+0000 7f40927fc700 1 -- 172.21.15.47:0/2943819819 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f408c00b870 con 0x7f40940b06d0 2022-06-29T15:44:16.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.283+0000 7f409a204700 1 -- 172.21.15.47:0/2943819819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40940b06d0 msgr2=0x7f409413ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:16.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.283+0000 7f409a204700 1 --2- 172.21.15.47:0/2943819819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40940b06d0 0x7f409413ff00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f40940b1c00 tx=0x7f408c0091d0).stop 2022-06-29T15:44:16.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.283+0000 7f409a204700 1 -- 172.21.15.47:0/2943819819 shutdown_connections 2022-06-29T15:44:16.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.283+0000 7f409a204700 1 --2- 172.21.15.47:0/2943819819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40940b06d0 0x7f409413ff00 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:16.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.283+0000 7f409a204700 1 --2- 172.21.15.47:0/2943819819 >> v2:172.21.15.47:3300/0 conn(0x7f40940b0cf0 0x7f4094146d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:16.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.283+0000 7f409a204700 1 -- 172.21.15.47:0/2943819819 >> 172.21.15.47:0/2943819819 conn(0x7f40940b98b0 msgr2=0x7f40940b9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:16.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.283+0000 7f409a204700 1 -- 172.21.15.47:0/2943819819 shutdown_connections 2022-06-29T15:44:16.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.283+0000 7f409a204700 1 -- 172.21.15.47:0/2943819819 wait complete. 2022-06-29T15:44:16.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.287+0000 7f409a204700 1 Processor -- start 2022-06-29T15:44:16.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.287+0000 7f409a204700 1 -- start start 2022-06-29T15:44:16.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.287+0000 7f409a204700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40940611b0 0x7f4094152450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:16.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.287+0000 7f409a204700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40940b06d0 0x7f40940b8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:16.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.287+0000 7f409a204700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f40940b0cf0 0x7f4094152d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:16.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.287+0000 7f409a204700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40940b7dc0 con 0x7f40940b06d0 2022-06-29T15:44:16.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.287+0000 7f409a204700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409413b430 con 0x7f40940611b0 2022-06-29T15:44:16.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.287+0000 7f409a204700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40940adfb0 con 0x7f40940b0cf0 2022-06-29T15:44:16.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.287+0000 7f4092ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40940b06d0 0x7f40940b8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:16.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.287+0000 7f4092ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40940b06d0 0x7f40940b8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47478/0 (socket says 172.21.15.47:47478) 2022-06-29T15:44:16.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.287+0000 7f4092ffd700 1 -- 172.21.15.47:0/1138986982 learned_addr learned my addr 172.21.15.47:0/1138986982 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:16.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.287+0000 7f40937fe700 1 --2- 172.21.15.47:0/1138986982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40940611b0 0x7f4094152450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:16.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.287+0000 7f4093fff700 1 --2- 172.21.15.47:0/1138986982 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f40940b0cf0 0x7f4094152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:16.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.287+0000 7f4092ffd700 1 -- 172.21.15.47:0/1138986982 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f40940b0cf0 msgr2=0x7f4094152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:16.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.287+0000 7f4092ffd700 1 --2- 172.21.15.47:0/1138986982 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f40940b0cf0 0x7f4094152d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:16.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.287+0000 7f4092ffd700 1 -- 172.21.15.47:0/1138986982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40940611b0 msgr2=0x7f4094152450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:16.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.287+0000 7f4092ffd700 1 --2- 172.21.15.47:0/1138986982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40940611b0 0x7f4094152450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:16.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.287+0000 7f4092ffd700 1 -- 172.21.15.47:0/1138986982 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f408c009470 con 0x7f40940b06d0 2022-06-29T15:44:16.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.287+0000 7f4092ffd700 1 --2- 172.21.15.47:0/1138986982 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40940b06d0 0x7f40940b8580 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f40940ae8e0 tx=0x7f4088006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:16.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.287+0000 7f4090ff9700 1 -- 172.21.15.47:0/1138986982 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4088011750 con 0x7f40940b06d0 2022-06-29T15:44:16.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.287+0000 7f4090ff9700 1 -- 172.21.15.47:0/1138986982 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40880118b0 con 0x7f40940b06d0 2022-06-29T15:44:16.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.287+0000 7f409a204700 1 -- 172.21.15.47:0/1138986982 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40940b8180 con 0x7f40940b06d0 2022-06-29T15:44:16.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.287+0000 7f4090ff9700 1 -- 172.21.15.47:0/1138986982 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4088011c10 con 0x7f40940b06d0 2022-06-29T15:44:16.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.287+0000 7f409a204700 1 -- 172.21.15.47:0/1138986982 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40940b8a70 con 0x7f40940b06d0 2022-06-29T15:44:16.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.291+0000 7f409a204700 1 -- 172.21.15.47:0/1138986982 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40940b5e00 con 0x7f40940b06d0 2022-06-29T15:44:16.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.291+0000 7f4090ff9700 1 -- 172.21.15.47:0/1138986982 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4088011d70 con 0x7f40940b06d0 2022-06-29T15:44:16.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.291+0000 7f4090ff9700 1 --2- 172.21.15.47:0/1138986982 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4074053b00 0x7f4074055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:16.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.291+0000 7f4090ff9700 1 -- 172.21.15.47:0/1138986982 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f408806c340 con 0x7f40940b06d0 2022-06-29T15:44:16.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.291+0000 7f40937fe700 1 --2- 172.21.15.47:0/1138986982 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4074053b00 0x7f4074055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:16.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.291+0000 7f40937fe700 1 --2- 172.21.15.47:0/1138986982 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4074053b00 0x7f4074055fc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f408400d000 tx=0x7f4084009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:16.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.295+0000 7f4090ff9700 1 -- 172.21.15.47:0/1138986982 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f40880430b0 con 0x7f40940b06d0 2022-06-29T15:44:16.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.455+0000 7f409a204700 1 -- 172.21.15.47:0/1138986982 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f40940b8310 con 0x7f40940b06d0 2022-06-29T15:44:16.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.455+0000 7f4090ff9700 1 -- 172.21.15.47:0/1138986982 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+2127 (secure 0 0 0) 0x7f40880466d0 con 0x7f40940b06d0 2022-06-29T15:44:16.462 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:44:16.463 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4896,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1047560169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1047560169},{"type":"v1","addr":"172.21.15.47:6835","nonce":1047560169}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":10},{"gid":4902,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/804808837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":804808837},{"type":"v1","addr":"172.21.15.64:6833","nonce":804808837}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:43:10.546977+0000","modified":"2022-06-29T15:44:16.148801+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2022-06-29T15:44:16.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.459+0000 7f409a204700 1 -- 172.21.15.47:0/1138986982 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4074053b00 msgr2=0x7f4074055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:16.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.459+0000 7f409a204700 1 --2- 172.21.15.47:0/1138986982 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4074053b00 0x7f4074055fc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f408400d000 tx=0x7f4084009040).stop 2022-06-29T15:44:16.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.459+0000 7f409a204700 1 -- 172.21.15.47:0/1138986982 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40940b06d0 msgr2=0x7f40940b8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:16.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.459+0000 7f409a204700 1 --2- 172.21.15.47:0/1138986982 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40940b06d0 0x7f40940b8580 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f40940ae8e0 tx=0x7f4088006df0).stop 2022-06-29T15:44:16.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.459+0000 7f409a204700 1 -- 172.21.15.47:0/1138986982 shutdown_connections 2022-06-29T15:44:16.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.459+0000 7f409a204700 1 --2- 172.21.15.47:0/1138986982 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f40940b0cf0 0x7f4094152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:16.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.459+0000 7f409a204700 1 --2- 172.21.15.47:0/1138986982 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40940b06d0 0x7f40940b8580 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:16.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.459+0000 7f409a204700 1 --2- 172.21.15.47:0/1138986982 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4074053b00 0x7f4074055fc0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:16.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.459+0000 7f409a204700 1 --2- 172.21.15.47:0/1138986982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40940611b0 0x7f4094152450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:16.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.459+0000 7f409a204700 1 -- 172.21.15.47:0/1138986982 >> 172.21.15.47:0/1138986982 conn(0x7f40940b98b0 msgr2=0x7f409413f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:16.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.459+0000 7f409a204700 1 -- 172.21.15.47:0/1138986982 shutdown_connections 2022-06-29T15:44:16.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.459+0000 7f409a204700 1 -- 172.21.15.47:0/1138986982 wait complete. 2022-06-29T15:44:16.468 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 13 2022-06-29T15:44:16.476 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-06-29T15:44:16.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.579+0000 7f5506de8700 1 Processor -- start 2022-06-29T15:44:16.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.579+0000 7f5506de8700 1 -- start start 2022-06-29T15:44:16.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.583+0000 7f5506de8700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5500140940 0x7f550005f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:16.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.583+0000 7f5506de8700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55000b3c30 con 0x7f55000b37f0 2022-06-29T15:44:16.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.583+0000 7f5506de8700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55000b4e50 con 0x7f550005f5a0 2022-06-29T15:44:16.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.583+0000 7f5506de8700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55000621e0 con 0x7f5500140940 2022-06-29T15:44:16.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.583+0000 7f5504b84700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5500140940 0x7f550005f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:16.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.583+0000 7f5505385700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f55000b37f0 0x7f550014da80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33040/0 (socket says 172.21.15.47:33040) 2022-06-29T15:44:16.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.583+0000 7f5505385700 1 -- 172.21.15.47:0/3388843311 learned_addr learned my addr 172.21.15.47:0/3388843311 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:44:16.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.583+0000 7f54ff7fe700 1 -- 172.21.15.47:0/3388843311 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 645233235 0 0) 0x7f55000b3c30 con 0x7f55000b37f0 2022-06-29T15:44:16.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.583+0000 7f54ff7fe700 1 -- 172.21.15.47:0/3388843311 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54ec003260 con 0x7f55000b37f0 2022-06-29T15:44:16.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.583+0000 7f5504b84700 1 -- 172.21.15.47:0/3388843311 >> v1:172.21.15.64:6789/0 conn(0x7f550005f5a0 legacy=0x7f55000b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:16.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.583+0000 7f5504b84700 1 -- 172.21.15.47:0/3388843311 >> v1:172.21.15.47:6789/0 conn(0x7f55000b37f0 legacy=0x7f550014da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:16.594 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.583+0000 7f5504b84700 1 -- 172.21.15.47:0/3388843311 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5500143170 con 0x7f5500140940 2022-06-29T15:44:16.594 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.583+0000 7f5504b84700 1 --2- 172.21.15.47:0/3388843311 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5500140940 0x7f550005f0b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7f54f0002f70 tx=0x7f54f0006df0).ready entity=mon.0 client_cookie=f4ec4d48b7de7f78 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:16.594 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.583+0000 7f54ff7fe700 1 -- 172.21.15.47:0/3388843311 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54f000d610 con 0x7f5500140940 2022-06-29T15:44:16.594 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.583+0000 7f54ff7fe700 1 -- 172.21.15.47:0/3388843311 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54f0000e90 con 0x7f5500140940 2022-06-29T15:44:16.594 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.583+0000 7f54ff7fe700 1 -- 172.21.15.47:0/3388843311 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54f000d960 con 0x7f5500140940 2022-06-29T15:44:16.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.583+0000 7f5506de8700 1 -- 172.21.15.47:0/3388843311 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5500140940 msgr2=0x7f550005f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:16.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.583+0000 7f5506de8700 1 --2- 172.21.15.47:0/3388843311 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5500140940 0x7f550005f0b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7f54f0002f70 tx=0x7f54f0006df0).stop 2022-06-29T15:44:16.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.583+0000 7f5506de8700 1 -- 172.21.15.47:0/3388843311 shutdown_connections 2022-06-29T15:44:16.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.583+0000 7f5506de8700 1 --2- 172.21.15.47:0/3388843311 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5500140940 0x7f550005f0b0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:16.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.583+0000 7f5506de8700 1 -- 172.21.15.47:0/3388843311 >> 172.21.15.47:0/3388843311 conn(0x7f55000627d0 msgr2=0x7f5500062bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:16.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.587+0000 7f5506de8700 1 -- 172.21.15.47:0/3388843311 shutdown_connections 2022-06-29T15:44:16.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.587+0000 7f5506de8700 1 -- 172.21.15.47:0/3388843311 wait complete. 2022-06-29T15:44:16.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.587+0000 7f5506de8700 1 Processor -- start 2022-06-29T15:44:16.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.587+0000 7f5506de8700 1 -- start start 2022-06-29T15:44:16.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.587+0000 7f5506de8700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f55000b37f0 0x7f5500149cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:16.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.587+0000 7f5506de8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5500140940 0x7f550013d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:16.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.587+0000 7f5506de8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f550013baa0 0x7f550013d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:16.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.587+0000 7f5506de8700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55000ad7f0 con 0x7f550013baa0 2022-06-29T15:44:16.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.587+0000 7f5506de8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55000ada20 con 0x7f5500140940 2022-06-29T15:44:16.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.587+0000 7f5506de8700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55000adcc0 con 0x7f55000b37f0 2022-06-29T15:44:16.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.587+0000 7f5505385700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f550013baa0 0x7f550013d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:16.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.587+0000 7f5505385700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f550013baa0 0x7f550013d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47494/0 (socket says 172.21.15.47:47494) 2022-06-29T15:44:16.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.587+0000 7f5505385700 1 -- 172.21.15.47:0/3635925317 learned_addr learned my addr 172.21.15.47:0/3635925317 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:16.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.587+0000 7f54fffff700 1 --2- 172.21.15.47:0/3635925317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5500140940 0x7f550013d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:16.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.587+0000 7f5504b84700 1 --2- 172.21.15.47:0/3635925317 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f55000b37f0 0x7f5500149cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:16.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.587+0000 7f5505385700 1 -- 172.21.15.47:0/3635925317 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f55000b37f0 msgr2=0x7f5500149cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:16.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.587+0000 7f5505385700 1 --2- 172.21.15.47:0/3635925317 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f55000b37f0 0x7f5500149cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:16.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.587+0000 7f5505385700 1 -- 172.21.15.47:0/3635925317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5500140940 msgr2=0x7f550013d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:16.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.587+0000 7f5505385700 1 --2- 172.21.15.47:0/3635925317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5500140940 0x7f550013d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:16.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.587+0000 7f5505385700 1 -- 172.21.15.47:0/3635925317 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f000b180 con 0x7f550013baa0 2022-06-29T15:44:16.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.587+0000 7f5505385700 1 --2- 172.21.15.47:0/3635925317 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f550013baa0 0x7f550013d720 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7f54f8001490 tx=0x7f54f8007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:16.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.587+0000 7f54fdffb700 1 -- 172.21.15.47:0/3635925317 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54f8005ba0 con 0x7f550013baa0 2022-06-29T15:44:16.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.587+0000 7f54fdffb700 1 -- 172.21.15.47:0/3635925317 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54f8005d00 con 0x7f550013baa0 2022-06-29T15:44:16.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.587+0000 7f5506de8700 1 -- 172.21.15.47:0/3635925317 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55000adf60 con 0x7f550013baa0 2022-06-29T15:44:16.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.587+0000 7f54fdffb700 1 -- 172.21.15.47:0/3635925317 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54f80072c0 con 0x7f550013baa0 2022-06-29T15:44:16.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.587+0000 7f5506de8700 1 -- 172.21.15.47:0/3635925317 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f550013b780 con 0x7f550013baa0 2022-06-29T15:44:16.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.591+0000 7f54fdffb700 1 -- 172.21.15.47:0/3635925317 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f54f801a060 con 0x7f550013baa0 2022-06-29T15:44:16.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.591+0000 7f5506de8700 1 -- 172.21.15.47:0/3635925317 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55000573d0 con 0x7f550013baa0 2022-06-29T15:44:16.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.591+0000 7f54fdffb700 1 --2- 172.21.15.47:0/3635925317 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f54e0053aa0 0x7f54e0055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:16.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.591+0000 7f54fdffb700 1 -- 172.21.15.47:0/3635925317 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f54f806bb00 con 0x7f550013baa0 2022-06-29T15:44:16.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.591+0000 7f5504b84700 1 --2- 172.21.15.47:0/3635925317 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f54e0053aa0 0x7f54e0055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:16.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.591+0000 7f5504b84700 1 --2- 172.21.15.47:0/3635925317 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f54e0053aa0 0x7f54e0055f60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f54f0018000 tx=0x7f54f0014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:16.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.595+0000 7f54fdffb700 1 -- 172.21.15.47:0/3635925317 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f54f8042870 con 0x7f550013baa0 2022-06-29T15:44:16.766 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:16.759+0000 7f5506de8700 1 -- 172.21.15.47:0/3635925317 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f550013d030 con 0x7f550013baa0 2022-06-29T15:44:17.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.163+0000 7f54fdffb700 1 -- 172.21.15.47:0/3635925317 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v14) v1 ==== 104+0+0 (secure 0 0 0) 0x7f54f8018020 con 0x7f550013baa0 2022-06-29T15:44:17.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.167+0000 7f5506de8700 1 -- 172.21.15.47:0/3635925317 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f54e0053aa0 msgr2=0x7f54e0055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:17.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.167+0000 7f5506de8700 1 --2- 172.21.15.47:0/3635925317 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f54e0053aa0 0x7f54e0055f60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f54f0018000 tx=0x7f54f0014000).stop 2022-06-29T15:44:17.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.167+0000 7f5506de8700 1 -- 172.21.15.47:0/3635925317 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f550013baa0 msgr2=0x7f550013d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:17.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.167+0000 7f5506de8700 1 --2- 172.21.15.47:0/3635925317 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f550013baa0 0x7f550013d720 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7f54f8001490 tx=0x7f54f8007e00).stop 2022-06-29T15:44:17.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.167+0000 7f5506de8700 1 -- 172.21.15.47:0/3635925317 shutdown_connections 2022-06-29T15:44:17.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.167+0000 7f5506de8700 1 --2- 172.21.15.47:0/3635925317 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f550013baa0 0x7f550013d720 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:17.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.167+0000 7f5506de8700 1 --2- 172.21.15.47:0/3635925317 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f54e0053aa0 0x7f54e0055f60 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:17.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.167+0000 7f5506de8700 1 --2- 172.21.15.47:0/3635925317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5500140940 0x7f550013d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:17.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.167+0000 7f5506de8700 1 --2- 172.21.15.47:0/3635925317 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f55000b37f0 0x7f5500149cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:17.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.167+0000 7f5506de8700 1 -- 172.21.15.47:0/3635925317 >> 172.21.15.47:0/3635925317 conn(0x7f55000627d0 msgr2=0x7f5500062bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:17.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.167+0000 7f5506de8700 1 -- 172.21.15.47:0/3635925317 shutdown_connections 2022-06-29T15:44:17.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.167+0000 7f5506de8700 1 -- 172.21.15.47:0/3635925317 wait complete. 2022-06-29T15:44:17.185 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-06-29T15:44:17.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.291+0000 7f8981c7a700 1 Processor -- start 2022-06-29T15:44:17.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.291+0000 7f8981c7a700 1 -- start start 2022-06-29T15:44:17.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.291+0000 7f8981c7a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f897c05c850 0x7f897c143a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:17.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.291+0000 7f8981c7a700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f897c0676a0 0x7f897c140130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:17.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.291+0000 7f8981c7a700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f897c05cc30 con 0x7f897c0b5d90 2022-06-29T15:44:17.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.295+0000 7f8981c7a700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f897c1430a0 con 0x7f897c05c850 2022-06-29T15:44:17.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.295+0000 7f8981c7a700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f897c143300 con 0x7f897c0676a0 2022-06-29T15:44:17.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.295+0000 7f8980c78700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f897c05c850 0x7f897c143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:17.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.295+0000 7f8980c78700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f897c05c850 0x7f897c143a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47500/0 (socket says 172.21.15.47:47500) 2022-06-29T15:44:17.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.295+0000 7f8980c78700 1 -- 172.21.15.47:0/3067152532 learned_addr learned my addr 172.21.15.47:0/3067152532 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:17.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.295+0000 7f8981479700 1 --2- 172.21.15.47:0/3067152532 >> v2:172.21.15.64:3300/0 conn(0x7f897c0676a0 0x7f897c140130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:17.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.295+0000 7f8980c78700 1 -- 172.21.15.47:0/3067152532 >> v2:172.21.15.64:3300/0 conn(0x7f897c0676a0 msgr2=0x7f897c140130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:44:17.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.295+0000 7f8980c78700 1 --2- 172.21.15.47:0/3067152532 >> v2:172.21.15.64:3300/0 conn(0x7f897c0676a0 0x7f897c140130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:17.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.295+0000 7f8980c78700 1 -- 172.21.15.47:0/3067152532 >> v1:172.21.15.64:6789/0 conn(0x7f897c0b5d90 legacy=0x7f897c0b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:17.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.295+0000 7f897b7fe700 1 -- 172.21.15.47:0/3067152532 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 927336519 0 0) 0x7f897c05cc30 con 0x7f897c0b5d90 2022-06-29T15:44:17.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.295+0000 7f8980c78700 1 -- 172.21.15.47:0/3067152532 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f897c1435a0 con 0x7f897c05c850 2022-06-29T15:44:17.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.295+0000 7f8980c78700 1 --2- 172.21.15.47:0/3067152532 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f897c05c850 0x7f897c143a40 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f8964002f70 tx=0x7f8964006eb0).ready entity=mon.0 client_cookie=4775da4ff6c95b06 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:17.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.295+0000 7f897b7fe700 1 -- 172.21.15.47:0/3067152532 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f896400d610 con 0x7f897c05c850 2022-06-29T15:44:17.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.295+0000 7f897b7fe700 1 -- 172.21.15.47:0/3067152532 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8964000e90 con 0x7f897c05c850 2022-06-29T15:44:17.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.295+0000 7f897b7fe700 1 -- 172.21.15.47:0/3067152532 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f896400d960 con 0x7f897c05c850 2022-06-29T15:44:17.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.295+0000 7f8981c7a700 1 -- 172.21.15.47:0/3067152532 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f897c05c850 msgr2=0x7f897c143a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:17.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.295+0000 7f8981c7a700 1 --2- 172.21.15.47:0/3067152532 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f897c05c850 0x7f897c143a40 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f8964002f70 tx=0x7f8964006eb0).stop 2022-06-29T15:44:17.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.295+0000 7f8981c7a700 1 -- 172.21.15.47:0/3067152532 shutdown_connections 2022-06-29T15:44:17.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.295+0000 7f8981c7a700 1 --2- 172.21.15.47:0/3067152532 >> v2:172.21.15.64:3300/0 conn(0x7f897c0676a0 0x7f897c140130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:17.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.295+0000 7f8981c7a700 1 --2- 172.21.15.47:0/3067152532 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f897c05c850 0x7f897c143a40 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:17.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.295+0000 7f8981c7a700 1 -- 172.21.15.47:0/3067152532 >> 172.21.15.47:0/3067152532 conn(0x7f897c060e00 msgr2=0x7f897c0611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:17.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.295+0000 7f8981c7a700 1 -- 172.21.15.47:0/3067152532 shutdown_connections 2022-06-29T15:44:17.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.295+0000 7f8981c7a700 1 -- 172.21.15.47:0/3067152532 wait complete. 2022-06-29T15:44:17.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.299+0000 7f8981c7a700 1 Processor -- start 2022-06-29T15:44:17.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.299+0000 7f8981c7a700 1 -- start start 2022-06-29T15:44:17.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.299+0000 7f8981c7a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f897c05c850 0x7f897c12e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:17.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.299+0000 7f8981c7a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f897c0676a0 0x7f897c12ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:17.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.299+0000 7f8981c7a700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f897c0b5d90 0x7f897c13f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:17.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.299+0000 7f8981c7a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897c12ef50 con 0x7f897c0676a0 2022-06-29T15:44:17.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.299+0000 7f8981c7a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897c0b2d50 con 0x7f897c05c850 2022-06-29T15:44:17.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.299+0000 7f8981c7a700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f897c0ba5d0 con 0x7f897c0b5d90 2022-06-29T15:44:17.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.299+0000 7f897bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f897c0676a0 0x7f897c12ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:17.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.299+0000 7f897bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f897c0676a0 0x7f897c12ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47506/0 (socket says 172.21.15.47:47506) 2022-06-29T15:44:17.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.299+0000 7f897bfff700 1 -- 172.21.15.47:0/1876760490 learned_addr learned my addr 172.21.15.47:0/1876760490 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:17.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.299+0000 7f8980c78700 1 --2- 172.21.15.47:0/1876760490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f897c05c850 0x7f897c12e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:17.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.299+0000 7f8981479700 1 --2- 172.21.15.47:0/1876760490 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f897c0b5d90 0x7f897c13f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:17.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.299+0000 7f897bfff700 1 -- 172.21.15.47:0/1876760490 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f897c0b5d90 msgr2=0x7f897c13f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:17.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.299+0000 7f897bfff700 1 --2- 172.21.15.47:0/1876760490 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f897c0b5d90 0x7f897c13f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:17.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.299+0000 7f897bfff700 1 -- 172.21.15.47:0/1876760490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f897c05c850 msgr2=0x7f897c12e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:17.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.299+0000 7f897bfff700 1 --2- 172.21.15.47:0/1876760490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f897c05c850 0x7f897c12e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:17.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.299+0000 7f897bfff700 1 -- 172.21.15.47:0/1876760490 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f896400b1e0 con 0x7f897c0676a0 2022-06-29T15:44:17.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.299+0000 7f897bfff700 1 --2- 172.21.15.47:0/1876760490 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f897c0676a0 0x7f897c12ea60 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7f896c000ed0 tx=0x7f896c007a50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:17.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.299+0000 7f8979ffb700 1 -- 172.21.15.47:0/1876760490 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f896c012400 con 0x7f897c0676a0 2022-06-29T15:44:17.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.299+0000 7f8979ffb700 1 -- 172.21.15.47:0/1876760490 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f896c012560 con 0x7f897c0676a0 2022-06-29T15:44:17.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.299+0000 7f8981c7a700 1 -- 172.21.15.47:0/1876760490 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f897c152990 con 0x7f897c0676a0 2022-06-29T15:44:17.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.299+0000 7f8979ffb700 1 -- 172.21.15.47:0/1876760490 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f896c0128c0 con 0x7f897c0676a0 2022-06-29T15:44:17.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.299+0000 7f8981c7a700 1 -- 172.21.15.47:0/1876760490 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f897c0b2f80 con 0x7f897c0676a0 2022-06-29T15:44:17.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.303+0000 7f8981c7a700 1 -- 172.21.15.47:0/1876760490 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f897c0b3bb0 con 0x7f897c0676a0 2022-06-29T15:44:17.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.303+0000 7f8979ffb700 1 -- 172.21.15.47:0/1876760490 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f896c019030 con 0x7f897c0676a0 2022-06-29T15:44:17.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.303+0000 7f8979ffb700 1 --2- 172.21.15.47:0/1876760490 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8960053aa0 0x7f8960055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:17.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.303+0000 7f8979ffb700 1 -- 172.21.15.47:0/1876760490 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f896c06b380 con 0x7f897c0676a0 2022-06-29T15:44:17.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.303+0000 7f8980c78700 1 --2- 172.21.15.47:0/1876760490 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8960053aa0 0x7f8960055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:17.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.303+0000 7f8980c78700 1 --2- 172.21.15.47:0/1876760490 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8960053aa0 0x7f8960055f60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f896401a000 tx=0x7f8964014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:17.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.311+0000 7f8979ffb700 1 -- 172.21.15.47:0/1876760490 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f897c0b3bb0 con 0x7f897c0676a0 2022-06-29T15:44:17.447 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:17.443+0000 7f8981c7a700 1 -- 172.21.15.47:0/1876760490 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f897c0b3bb0 con 0x7f897c0676a0 2022-06-29T15:44:18.224 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.219+0000 7f8979ffb700 1 -- 172.21.15.47:0/1876760490 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v38) v1 ==== 185+0+0 (secure 0 0 0) 0x7f897c0b3bb0 con 0x7f897c0676a0 2022-06-29T15:44:18.228 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.223+0000 7f8981c7a700 1 -- 172.21.15.47:0/1876760490 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8960053aa0 msgr2=0x7f8960055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:18.228 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.223+0000 7f8981c7a700 1 --2- 172.21.15.47:0/1876760490 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8960053aa0 0x7f8960055f60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f896401a000 tx=0x7f8964014000).stop 2022-06-29T15:44:18.228 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.223+0000 7f8981c7a700 1 -- 172.21.15.47:0/1876760490 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f897c0676a0 msgr2=0x7f897c12ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:18.228 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.223+0000 7f8981c7a700 1 --2- 172.21.15.47:0/1876760490 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f897c0676a0 0x7f897c12ea60 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7f896c000ed0 tx=0x7f896c007a50).stop 2022-06-29T15:44:18.230 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.223+0000 7f8981c7a700 1 -- 172.21.15.47:0/1876760490 shutdown_connections 2022-06-29T15:44:18.230 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.223+0000 7f8981c7a700 1 --2- 172.21.15.47:0/1876760490 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f897c0b5d90 0x7f897c13f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:18.230 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.223+0000 7f8981c7a700 1 --2- 172.21.15.47:0/1876760490 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f897c0676a0 0x7f897c12ea60 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:18.230 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.223+0000 7f8981c7a700 1 --2- 172.21.15.47:0/1876760490 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8960053aa0 0x7f8960055f60 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:18.231 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.223+0000 7f8981c7a700 1 --2- 172.21.15.47:0/1876760490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f897c05c850 0x7f897c12e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:18.231 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.223+0000 7f8981c7a700 1 -- 172.21.15.47:0/1876760490 >> 172.21.15.47:0/1876760490 conn(0x7f897c060e00 msgr2=0x7f897c142710 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:18.231 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.223+0000 7f8981c7a700 1 -- 172.21.15.47:0/1876760490 shutdown_connections 2022-06-29T15:44:18.231 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.223+0000 7f8981c7a700 1 -- 172.21.15.47:0/1876760490 wait complete. 2022-06-29T15:44:18.231 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' removed 2022-06-29T15:44:18.243 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-06-29T15:44:18.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.351+0000 7fa384744700 1 Processor -- start 2022-06-29T15:44:18.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.351+0000 7fa384744700 1 -- start start 2022-06-29T15:44:18.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.351+0000 7fa384744700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa37c060ee0 0x7fa37c0612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:18.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.351+0000 7fa384744700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa37c1471a0 0x7fa37c066f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:18.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.351+0000 7fa384744700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa37c067610 con 0x7fa37c0b4300 2022-06-29T15:44:18.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.351+0000 7fa384744700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa37c1467c0 con 0x7fa37c060ee0 2022-06-29T15:44:18.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.351+0000 7fa384744700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa37c146a60 con 0x7fa37c1471a0 2022-06-29T15:44:18.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.355+0000 7fa3824e0700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa37c060ee0 0x7fa37c0612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:18.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.355+0000 7fa3824e0700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa37c060ee0 0x7fa37c0612c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47516/0 (socket says 172.21.15.47:47516) 2022-06-29T15:44:18.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.355+0000 7fa3824e0700 1 -- 172.21.15.47:0/3398268943 learned_addr learned my addr 172.21.15.47:0/3398268943 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:18.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.355+0000 7fa382ce1700 1 --2- 172.21.15.47:0/3398268943 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa37c1471a0 0x7fa37c066f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:18.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.355+0000 7fa3824e0700 1 -- 172.21.15.47:0/3398268943 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa37c1471a0 msgr2=0x7fa37c066f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:44:18.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.355+0000 7fa3824e0700 1 --2- 172.21.15.47:0/3398268943 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa37c1471a0 0x7fa37c066f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:18.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.355+0000 7fa3824e0700 1 -- 172.21.15.47:0/3398268943 >> v1:172.21.15.64:6789/0 conn(0x7fa37c0b4300 legacy=0x7fa37c1430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:18.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.355+0000 7fa3824e0700 1 -- 172.21.15.47:0/3398268943 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa37c146d00 con 0x7fa37c060ee0 2022-06-29T15:44:18.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.355+0000 7fa3824e0700 1 --2- 172.21.15.47:0/3398268943 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa37c060ee0 0x7fa37c0612c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7fa37c0ae790 tx=0x7fa36c006eb0).ready entity=mon.0 client_cookie=6bb613304e74d29a server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:18.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.355+0000 7fa3814de700 1 -- 172.21.15.47:0/3398268943 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa36c00d5c0 con 0x7fa37c060ee0 2022-06-29T15:44:18.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.355+0000 7fa3814de700 1 -- 172.21.15.47:0/3398268943 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa36c000e90 con 0x7fa37c060ee0 2022-06-29T15:44:18.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.355+0000 7fa3814de700 1 -- 172.21.15.47:0/3398268943 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa36c00d950 con 0x7fa37c060ee0 2022-06-29T15:44:18.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.355+0000 7fa384744700 1 -- 172.21.15.47:0/3398268943 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa37c060ee0 msgr2=0x7fa37c0612c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:18.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.355+0000 7fa384744700 1 --2- 172.21.15.47:0/3398268943 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa37c060ee0 0x7fa37c0612c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7fa37c0ae790 tx=0x7fa36c006eb0).stop 2022-06-29T15:44:18.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.355+0000 7fa384744700 1 -- 172.21.15.47:0/3398268943 shutdown_connections 2022-06-29T15:44:18.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.355+0000 7fa384744700 1 --2- 172.21.15.47:0/3398268943 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa37c1471a0 0x7fa37c066f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:18.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.355+0000 7fa384744700 1 --2- 172.21.15.47:0/3398268943 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa37c060ee0 0x7fa37c0612c0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:18.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.355+0000 7fa384744700 1 -- 172.21.15.47:0/3398268943 >> 172.21.15.47:0/3398268943 conn(0x7fa37c05c510 msgr2=0x7fa37c05f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:18.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.355+0000 7fa384744700 1 -- 172.21.15.47:0/3398268943 shutdown_connections 2022-06-29T15:44:18.379 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.355+0000 7fa384744700 1 -- 172.21.15.47:0/3398268943 wait complete. 2022-06-29T15:44:18.379 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.359+0000 7fa384744700 1 Processor -- start 2022-06-29T15:44:18.380 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.359+0000 7fa384744700 1 -- start start 2022-06-29T15:44:18.380 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.359+0000 7fa384744700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa37c060ee0 0x7fa37c0b3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:18.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.359+0000 7fa384744700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa37c0b4300 0x7fa37c1351d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:18.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.359+0000 7fa384744700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa37c1471a0 0x7fa37c1356c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:18.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.359+0000 7fa384744700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37c1376d0 con 0x7fa37c060ee0 2022-06-29T15:44:18.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.359+0000 7fa384744700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37c137970 con 0x7fa37c0b4300 2022-06-29T15:44:18.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.359+0000 7fa384744700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa37c141060 con 0x7fa37c1471a0 2022-06-29T15:44:18.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.359+0000 7fa3824e0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa37c060ee0 0x7fa37c0b3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:18.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.359+0000 7fa3824e0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa37c060ee0 0x7fa37c0b3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47520/0 (socket says 172.21.15.47:47520) 2022-06-29T15:44:18.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.359+0000 7fa3824e0700 1 -- 172.21.15.47:0/3742558588 learned_addr learned my addr 172.21.15.47:0/3742558588 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:18.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.359+0000 7fa382ce1700 1 --2- 172.21.15.47:0/3742558588 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa37c1471a0 0x7fa37c1356c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:18.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.359+0000 7fa381cdf700 1 --2- 172.21.15.47:0/3742558588 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa37c0b4300 0x7fa37c1351d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:18.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.359+0000 7fa3824e0700 1 -- 172.21.15.47:0/3742558588 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa37c1471a0 msgr2=0x7fa37c1356c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:18.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.359+0000 7fa3824e0700 1 --2- 172.21.15.47:0/3742558588 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa37c1471a0 0x7fa37c1356c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:18.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.359+0000 7fa3824e0700 1 -- 172.21.15.47:0/3742558588 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa37c0b4300 msgr2=0x7fa37c1351d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:18.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.359+0000 7fa3824e0700 1 --2- 172.21.15.47:0/3742558588 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa37c0b4300 0x7fa37c1351d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:18.385 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.359+0000 7fa3824e0700 1 -- 172.21.15.47:0/3742558588 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa36c00b1e0 con 0x7fa37c060ee0 2022-06-29T15:44:18.385 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.359+0000 7fa3824e0700 1 --2- 172.21.15.47:0/3742558588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa37c060ee0 0x7fa37c0b3960 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7fa37c0ae790 tx=0x7fa36c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:18.385 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.359+0000 7fa36b7fe700 1 -- 172.21.15.47:0/3742558588 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa36c0117e0 con 0x7fa37c060ee0 2022-06-29T15:44:18.385 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.363+0000 7fa36b7fe700 1 -- 172.21.15.47:0/3742558588 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa36c011940 con 0x7fa37c060ee0 2022-06-29T15:44:18.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.363+0000 7fa384744700 1 -- 172.21.15.47:0/3742558588 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa37c146410 con 0x7fa37c060ee0 2022-06-29T15:44:18.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.363+0000 7fa36b7fe700 1 -- 172.21.15.47:0/3742558588 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa36c011ca0 con 0x7fa37c060ee0 2022-06-29T15:44:18.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.363+0000 7fa384744700 1 -- 172.21.15.47:0/3742558588 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa37c146090 con 0x7fa37c060ee0 2022-06-29T15:44:18.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.363+0000 7fa384744700 1 -- 172.21.15.47:0/3742558588 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa37c145440 con 0x7fa37c060ee0 2022-06-29T15:44:18.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.363+0000 7fa36b7fe700 1 -- 172.21.15.47:0/3742558588 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa36c025030 con 0x7fa37c060ee0 2022-06-29T15:44:18.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.363+0000 7fa36b7fe700 1 --2- 172.21.15.47:0/3742558588 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa35c053b30 0x7fa35c055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:18.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.363+0000 7fa36b7fe700 1 -- 172.21.15.47:0/3742558588 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6068+0+0 (secure 0 0 0) 0x7fa36c06d0b0 con 0x7fa37c060ee0 2022-06-29T15:44:18.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.363+0000 7fa381cdf700 1 --2- 172.21.15.47:0/3742558588 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa35c053b30 0x7fa35c055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:18.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.363+0000 7fa381cdf700 1 --2- 172.21.15.47:0/3742558588 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa35c053b30 0x7fa35c055ff0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fa37800d000 tx=0x7fa378007b90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:18.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.371+0000 7fa36b7fe700 1 -- 172.21.15.47:0/3742558588 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa36c043fd0 con 0x7fa37c060ee0 2022-06-29T15:44:18.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:18.499+0000 7fa384744700 1 -- 172.21.15.47:0/3742558588 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa37c0573d0 con 0x7fa37c060ee0 2022-06-29T15:44:19.237 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.231+0000 7fa36b7fe700 1 -- 172.21.15.47:0/3742558588 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v39) v1 ==== 173+0+0 (secure 0 0 0) 0x7fa36c023070 con 0x7fa37c060ee0 2022-06-29T15:44:19.243 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.235+0000 7fa384744700 1 -- 172.21.15.47:0/3742558588 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa35c053b30 msgr2=0x7fa35c055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:19.243 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.235+0000 7fa384744700 1 --2- 172.21.15.47:0/3742558588 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa35c053b30 0x7fa35c055ff0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fa37800d000 tx=0x7fa378007b90).stop 2022-06-29T15:44:19.243 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.235+0000 7fa384744700 1 -- 172.21.15.47:0/3742558588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa37c060ee0 msgr2=0x7fa37c0b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:19.243 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.235+0000 7fa384744700 1 --2- 172.21.15.47:0/3742558588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa37c060ee0 0x7fa37c0b3960 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7fa37c0ae790 tx=0x7fa36c015000).stop 2022-06-29T15:44:19.244 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.235+0000 7fa384744700 1 -- 172.21.15.47:0/3742558588 shutdown_connections 2022-06-29T15:44:19.244 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.235+0000 7fa384744700 1 --2- 172.21.15.47:0/3742558588 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa37c1471a0 0x7fa37c1356c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:19.244 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.235+0000 7fa384744700 1 --2- 172.21.15.47:0/3742558588 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa35c053b30 0x7fa35c055ff0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:19.244 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.235+0000 7fa384744700 1 --2- 172.21.15.47:0/3742558588 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa37c0b4300 0x7fa37c1351d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:19.245 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.235+0000 7fa384744700 1 --2- 172.21.15.47:0/3742558588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa37c060ee0 0x7fa37c0b3960 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:19.245 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.235+0000 7fa384744700 1 -- 172.21.15.47:0/3742558588 >> 172.21.15.47:0/3742558588 conn(0x7fa37c05c510 msgr2=0x7fa37c13cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:19.245 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.235+0000 7fa384744700 1 -- 172.21.15.47:0/3742558588 shutdown_connections 2022-06-29T15:44:19.245 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.235+0000 7fa384744700 1 -- 172.21.15.47:0/3742558588 wait complete. 2022-06-29T15:44:19.246 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' removed 2022-06-29T15:44:19.257 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-06-29T15:44:19.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.355+0000 7f8b32225700 1 Processor -- start 2022-06-29T15:44:19.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.355+0000 7f8b32225700 1 -- start start 2022-06-29T15:44:19.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.359+0000 7f8b32225700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8b2c0a94f0 0x7f8b2c140ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:19.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.359+0000 7f8b32225700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b2c1472a0 con 0x7f8b2c0b2ad0 2022-06-29T15:44:19.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.359+0000 7f8b32225700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b2c147540 con 0x7f8b2c0608d0 2022-06-29T15:44:19.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.359+0000 7f8b32225700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b2c0b01b0 con 0x7f8b2c0a94f0 2022-06-29T15:44:19.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.359+0000 7f8b2affd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8b2c0a94f0 0x7f8b2c140ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:19.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.359+0000 7f8b2affd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8b2c0a94f0 0x7f8b2c140ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47530/0 (socket says 172.21.15.47:47530) 2022-06-29T15:44:19.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.359+0000 7f8b2bfff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f8b2c0b2ad0 0x7f8b2c143220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33086/0 (socket says 172.21.15.47:33086) 2022-06-29T15:44:19.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.359+0000 7f8b2affd700 1 -- 172.21.15.47:0/3249025819 learned_addr learned my addr 172.21.15.47:0/3249025819 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:19.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.359+0000 7f8b2affd700 1 -- 172.21.15.47:0/3249025819 >> v1:172.21.15.64:6789/0 conn(0x7f8b2c0608d0 legacy=0x7f8b2c1403b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:19.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.359+0000 7f8b2affd700 1 -- 172.21.15.47:0/3249025819 >> v1:172.21.15.47:6789/0 conn(0x7f8b2c0b2ad0 legacy=0x7f8b2c143220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:19.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.359+0000 7f8b2a7fc700 1 -- 172.21.15.47:0/3249025819 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2393447792 0 0) 0x7f8b2c147540 con 0x7f8b2c0608d0 2022-06-29T15:44:19.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.359+0000 7f8b2affd700 1 -- 172.21.15.47:0/3249025819 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b2c0a8d80 con 0x7f8b2c0a94f0 2022-06-29T15:44:19.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.359+0000 7f8b2affd700 1 --2- 172.21.15.47:0/3249025819 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b2c0a94f0 0x7f8b2c140ac0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f8b1c002f70 tx=0x7f8b1c006eb0).ready entity=mon.0 client_cookie=95e6816d48ded775 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:19.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.359+0000 7f8b2a7fc700 1 -- 172.21.15.47:0/3249025819 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b1c00d660 con 0x7f8b2c0a94f0 2022-06-29T15:44:19.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.359+0000 7f8b2a7fc700 1 -- 172.21.15.47:0/3249025819 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b1c000e90 con 0x7f8b2c0a94f0 2022-06-29T15:44:19.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.359+0000 7f8b2a7fc700 1 -- 172.21.15.47:0/3249025819 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b1c00d9b0 con 0x7f8b2c0a94f0 2022-06-29T15:44:19.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.359+0000 7f8b32225700 1 -- 172.21.15.47:0/3249025819 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b2c0a94f0 msgr2=0x7f8b2c140ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:19.370 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.359+0000 7f8b32225700 1 --2- 172.21.15.47:0/3249025819 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b2c0a94f0 0x7f8b2c140ac0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f8b1c002f70 tx=0x7f8b1c006eb0).stop 2022-06-29T15:44:19.370 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.359+0000 7f8b32225700 1 -- 172.21.15.47:0/3249025819 shutdown_connections 2022-06-29T15:44:19.370 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.359+0000 7f8b32225700 1 --2- 172.21.15.47:0/3249025819 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b2c0a94f0 0x7f8b2c140ac0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:19.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.359+0000 7f8b32225700 1 -- 172.21.15.47:0/3249025819 >> 172.21.15.47:0/3249025819 conn(0x7f8b2c0626f0 msgr2=0x7f8b2c062ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:19.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.359+0000 7f8b32225700 1 -- 172.21.15.47:0/3249025819 shutdown_connections 2022-06-29T15:44:19.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.359+0000 7f8b32225700 1 -- 172.21.15.47:0/3249025819 wait complete. 2022-06-29T15:44:19.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.363+0000 7f8b32225700 1 Processor -- start 2022-06-29T15:44:19.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.363+0000 7f8b32225700 1 -- start start 2022-06-29T15:44:19.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.363+0000 7f8b32225700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b2c0608d0 0x7f8b2c13c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:19.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.363+0000 7f8b32225700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8b2c0a94f0 0x7f8b2c152160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:19.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.363+0000 7f8b32225700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b2c0b2ad0 0x7f8b2c152650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:19.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.363+0000 7f8b32225700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2c145de0 con 0x7f8b2c0608d0 2022-06-29T15:44:19.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.363+0000 7f8b32225700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2c13e470 con 0x7f8b2c0b2ad0 2022-06-29T15:44:19.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.367+0000 7f8b32225700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2c1450d0 con 0x7f8b2c0a94f0 2022-06-29T15:44:19.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.367+0000 7f8b2b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b2c0608d0 0x7f8b2c13c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:19.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.367+0000 7f8b2b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b2c0608d0 0x7f8b2c13c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47534/0 (socket says 172.21.15.47:47534) 2022-06-29T15:44:19.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.367+0000 7f8b2b7fe700 1 -- 172.21.15.47:0/2640166839 learned_addr learned my addr 172.21.15.47:0/2640166839 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:19.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.367+0000 7f8b2bfff700 1 --2- 172.21.15.47:0/2640166839 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b2c0b2ad0 0x7f8b2c152650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:19.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.367+0000 7f8b2affd700 1 --2- 172.21.15.47:0/2640166839 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8b2c0a94f0 0x7f8b2c152160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:19.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.367+0000 7f8b2b7fe700 1 -- 172.21.15.47:0/2640166839 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8b2c0a94f0 msgr2=0x7f8b2c152160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:19.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.367+0000 7f8b2b7fe700 1 --2- 172.21.15.47:0/2640166839 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8b2c0a94f0 0x7f8b2c152160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:19.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.367+0000 7f8b2b7fe700 1 -- 172.21.15.47:0/2640166839 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b2c0b2ad0 msgr2=0x7f8b2c152650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:19.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.367+0000 7f8b2b7fe700 1 --2- 172.21.15.47:0/2640166839 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b2c0b2ad0 0x7f8b2c152650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:19.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.367+0000 7f8b2b7fe700 1 -- 172.21.15.47:0/2640166839 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b1c00b1e0 con 0x7f8b2c0608d0 2022-06-29T15:44:19.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.367+0000 7f8b2b7fe700 1 --2- 172.21.15.47:0/2640166839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b2c0608d0 0x7f8b2c13c090 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7f8b20000ed0 tx=0x7f8b20007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:19.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.367+0000 7f8b28ff9700 1 -- 172.21.15.47:0/2640166839 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b20012400 con 0x7f8b2c0608d0 2022-06-29T15:44:19.379 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.367+0000 7f8b28ff9700 1 -- 172.21.15.47:0/2640166839 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b20012560 con 0x7f8b2c0608d0 2022-06-29T15:44:19.379 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.367+0000 7f8b32225700 1 -- 172.21.15.47:0/2640166839 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b2c145370 con 0x7f8b2c0608d0 2022-06-29T15:44:19.379 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.367+0000 7f8b28ff9700 1 -- 172.21.15.47:0/2640166839 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b200128c0 con 0x7f8b2c0608d0 2022-06-29T15:44:19.379 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.367+0000 7f8b32225700 1 -- 172.21.15.47:0/2640166839 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b2c144dd0 con 0x7f8b2c0608d0 2022-06-29T15:44:19.380 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.367+0000 7f8b32225700 1 -- 172.21.15.47:0/2640166839 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b2c0573d0 con 0x7f8b2c0608d0 2022-06-29T15:44:19.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.367+0000 7f8b28ff9700 1 -- 172.21.15.47:0/2640166839 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8b20019030 con 0x7f8b2c0608d0 2022-06-29T15:44:19.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.371+0000 7f8b28ff9700 1 --2- 172.21.15.47:0/2640166839 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8b0c053a70 0x7f8b0c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:19.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.371+0000 7f8b28ff9700 1 -- 172.21.15.47:0/2640166839 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f8b2006afd0 con 0x7f8b2c0608d0 2022-06-29T15:44:19.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.371+0000 7f8b2affd700 1 --2- 172.21.15.47:0/2640166839 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8b0c053a70 0x7f8b0c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:19.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.371+0000 7f8b2affd700 1 --2- 172.21.15.47:0/2640166839 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8b0c053a70 0x7f8b0c055f30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f8b1c01c000 tx=0x7f8b1c015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:19.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.379+0000 7f8b28ff9700 1 -- 172.21.15.47:0/2640166839 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8b20042120 con 0x7f8b2c0608d0 2022-06-29T15:44:19.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.503+0000 7f8b32225700 1 -- 172.21.15.47:0/2640166839 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f8b2c06a210 con 0x7f8b2c0608d0 2022-06-29T15:44:19.517 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.511+0000 7f8b28ff9700 1 -- 172.21.15.47:0/2640166839 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v3) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8b20045740 con 0x7f8b2c0608d0 2022-06-29T15:44:19.520 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.515+0000 7f8b32225700 1 -- 172.21.15.47:0/2640166839 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8b0c053a70 msgr2=0x7f8b0c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:19.520 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.515+0000 7f8b32225700 1 --2- 172.21.15.47:0/2640166839 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8b0c053a70 0x7f8b0c055f30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f8b1c01c000 tx=0x7f8b1c015000).stop 2022-06-29T15:44:19.520 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.515+0000 7f8b32225700 1 -- 172.21.15.47:0/2640166839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b2c0608d0 msgr2=0x7f8b2c13c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:19.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.515+0000 7f8b32225700 1 --2- 172.21.15.47:0/2640166839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b2c0608d0 0x7f8b2c13c090 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7f8b20000ed0 tx=0x7f8b20007ac0).stop 2022-06-29T15:44:19.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.515+0000 7f8b32225700 1 -- 172.21.15.47:0/2640166839 shutdown_connections 2022-06-29T15:44:19.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.515+0000 7f8b32225700 1 --2- 172.21.15.47:0/2640166839 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8b0c053a70 0x7f8b0c055f30 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:19.522 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.515+0000 7f8b32225700 1 --2- 172.21.15.47:0/2640166839 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b2c0b2ad0 0x7f8b2c152650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:19.522 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.515+0000 7f8b32225700 1 --2- 172.21.15.47:0/2640166839 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8b2c0a94f0 0x7f8b2c152160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:19.522 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.515+0000 7f8b32225700 1 --2- 172.21.15.47:0/2640166839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b2c0608d0 0x7f8b2c13c090 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:19.522 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.515+0000 7f8b32225700 1 -- 172.21.15.47:0/2640166839 >> 172.21.15.47:0/2640166839 conn(0x7f8b2c0626f0 msgr2=0x7f8b2c1418a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:19.523 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.515+0000 7f8b32225700 1 -- 172.21.15.47:0/2640166839 shutdown_connections 2022-06-29T15:44:19.523 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.515+0000 7f8b32225700 1 -- 172.21.15.47:0/2640166839 wait complete. 2022-06-29T15:44:19.532 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command' 2022-06-29T15:44:19.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.623+0000 7f766bfff700 1 Processor -- start 2022-06-29T15:44:19.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.623+0000 7f766bfff700 1 -- start start 2022-06-29T15:44:19.632 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.627+0000 7f766bfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f766c13e9b0 0x7f766c05f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:19.632 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.627+0000 7f766bfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f766c0b6f10 0x7f766c05f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:19.633 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.627+0000 7f766bfff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f766c0a9cf0 con 0x7f766c061c00 2022-06-29T15:44:19.633 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.627+0000 7f766bfff700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f766c0b3e90 con 0x7f766c13e9b0 2022-06-29T15:44:19.633 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.627+0000 7f766bfff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f766c0a9f90 con 0x7f766c0b6f10 2022-06-29T15:44:19.633 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.627+0000 7f766a7fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f766c13e9b0 0x7f766c05f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:19.634 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.627+0000 7f766a7fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f766c13e9b0 0x7f766c05f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47544/0 (socket says 172.21.15.47:47544) 2022-06-29T15:44:19.634 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.627+0000 7f766a7fc700 1 -- 172.21.15.47:0/468490275 learned_addr learned my addr 172.21.15.47:0/468490275 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:19.634 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.627+0000 7f766b7fe700 1 --2- 172.21.15.47:0/468490275 >> v2:172.21.15.64:3300/0 conn(0x7f766c0b6f10 0x7f766c05f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:19.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.627+0000 7f766a7fc700 1 -- 172.21.15.47:0/468490275 >> v2:172.21.15.64:3300/0 conn(0x7f766c0b6f10 msgr2=0x7f766c05f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:44:19.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.627+0000 7f766a7fc700 1 --2- 172.21.15.47:0/468490275 >> v2:172.21.15.64:3300/0 conn(0x7f766c0b6f10 0x7f766c05f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:19.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.627+0000 7f766a7fc700 1 -- 172.21.15.47:0/468490275 >> v1:172.21.15.64:6789/0 conn(0x7f766c061c00 legacy=0x7f766c13e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:19.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.627+0000 7f766a7fc700 1 -- 172.21.15.47:0/468490275 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f766c141b70 con 0x7f766c13e9b0 2022-06-29T15:44:19.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.627+0000 7f766b7fe700 1 --2- 172.21.15.47:0/468490275 >> v2:172.21.15.64:3300/0 conn(0x7f766c0b6f10 0x7f766c05f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:44:19.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.627+0000 7f766a7fc700 1 --2- 172.21.15.47:0/468490275 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f766c13e9b0 0x7f766c05f0b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7f7660002f70 tx=0x7f7660006eb0).ready entity=mon.0 client_cookie=64419af60d445c31 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:19.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.627+0000 7f7669ffb700 1 -- 172.21.15.47:0/468490275 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f766000d5c0 con 0x7f766c13e9b0 2022-06-29T15:44:19.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.627+0000 7f7669ffb700 1 -- 172.21.15.47:0/468490275 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7660000e90 con 0x7f766c13e9b0 2022-06-29T15:44:19.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.627+0000 7f7669ffb700 1 -- 172.21.15.47:0/468490275 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f766000d950 con 0x7f766c13e9b0 2022-06-29T15:44:19.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.627+0000 7f766bfff700 1 -- 172.21.15.47:0/468490275 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f766c13e9b0 msgr2=0x7f766c05f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:19.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.627+0000 7f766bfff700 1 --2- 172.21.15.47:0/468490275 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f766c13e9b0 0x7f766c05f0b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7f7660002f70 tx=0x7f7660006eb0).stop 2022-06-29T15:44:19.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.627+0000 7f766bfff700 1 -- 172.21.15.47:0/468490275 shutdown_connections 2022-06-29T15:44:19.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.627+0000 7f766bfff700 1 --2- 172.21.15.47:0/468490275 >> v2:172.21.15.64:3300/0 conn(0x7f766c0b6f10 0x7f766c05f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:19.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.627+0000 7f766bfff700 1 --2- 172.21.15.47:0/468490275 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f766c13e9b0 0x7f766c05f0b0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:19.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.627+0000 7f766bfff700 1 -- 172.21.15.47:0/468490275 >> 172.21.15.47:0/468490275 conn(0x7f766c063670 msgr2=0x7f766c063a80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:19.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.627+0000 7f766bfff700 1 -- 172.21.15.47:0/468490275 shutdown_connections 2022-06-29T15:44:19.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.627+0000 7f766bfff700 1 -- 172.21.15.47:0/468490275 wait complete. 2022-06-29T15:44:19.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.631+0000 7f766bfff700 1 Processor -- start 2022-06-29T15:44:19.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.631+0000 7f766bfff700 1 -- start start 2022-06-29T15:44:19.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.631+0000 7f766bfff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f766c061c00 0x7f766c1562b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:19.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.631+0000 7f766bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f766c0b6f10 0x7f766c0b2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:19.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.631+0000 7f766bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f766c13e9b0 0x7f766c152270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:19.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.631+0000 7f766bfff700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766c0b5710 con 0x7f766c13e9b0 2022-06-29T15:44:19.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.631+0000 7f766bfff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766c0af620 con 0x7f766c0b6f10 2022-06-29T15:44:19.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.631+0000 7f766bfff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f766c0b6460 con 0x7f766c061c00 2022-06-29T15:44:19.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.631+0000 7f766b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f766c13e9b0 0x7f766c152270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:19.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.631+0000 7f766b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f766c13e9b0 0x7f766c152270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47552/0 (socket says 172.21.15.47:47552) 2022-06-29T15:44:19.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.631+0000 7f766affd700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f766c061c00 0x7f766c1562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:19.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.631+0000 7f766b7fe700 1 -- 172.21.15.47:0/3024919932 learned_addr learned my addr 172.21.15.47:0/3024919932 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:19.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.631+0000 7f766a7fc700 1 --2- 172.21.15.47:0/3024919932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f766c0b6f10 0x7f766c0b2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:19.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.635+0000 7f766b7fe700 1 -- 172.21.15.47:0/3024919932 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f766c061c00 msgr2=0x7f766c1562b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:19.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.635+0000 7f766b7fe700 1 --2- 172.21.15.47:0/3024919932 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f766c061c00 0x7f766c1562b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:19.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.635+0000 7f766b7fe700 1 -- 172.21.15.47:0/3024919932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f766c0b6f10 msgr2=0x7f766c0b2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:19.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.635+0000 7f766b7fe700 1 --2- 172.21.15.47:0/3024919932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f766c0b6f10 0x7f766c0b2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:19.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.635+0000 7f766b7fe700 1 -- 172.21.15.47:0/3024919932 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7664009470 con 0x7f766c13e9b0 2022-06-29T15:44:19.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.635+0000 7f766affd700 1 --2- 172.21.15.47:0/3024919932 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f766c061c00 0x7f766c1562b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:44:19.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.635+0000 7f766b7fe700 1 --2- 172.21.15.47:0/3024919932 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f766c13e9b0 0x7f766c152270 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7f766c0aa880 tx=0x7f76640091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:19.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.635+0000 7f765bfff700 1 -- 172.21.15.47:0/3024919932 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f766400c850 con 0x7f766c13e9b0 2022-06-29T15:44:19.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.635+0000 7f765bfff700 1 -- 172.21.15.47:0/3024919932 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f766400c9b0 con 0x7f766c13e9b0 2022-06-29T15:44:19.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.635+0000 7f766bfff700 1 -- 172.21.15.47:0/3024919932 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f766000b1e0 con 0x7f766c13e9b0 2022-06-29T15:44:19.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.635+0000 7f765bfff700 1 -- 172.21.15.47:0/3024919932 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f766400cd10 con 0x7f766c13e9b0 2022-06-29T15:44:19.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.635+0000 7f766bfff700 1 -- 172.21.15.47:0/3024919932 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f766c155e10 con 0x7f766c13e9b0 2022-06-29T15:44:19.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.635+0000 7f766bfff700 1 -- 172.21.15.47:0/3024919932 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f766c0ab3c0 con 0x7f766c13e9b0 2022-06-29T15:44:19.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.635+0000 7f765bfff700 1 -- 172.21.15.47:0/3024919932 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7664022070 con 0x7f766c13e9b0 2022-06-29T15:44:19.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.635+0000 7f765bfff700 1 --2- 172.21.15.47:0/3024919932 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7648053b30 0x7f7648055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:19.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.635+0000 7f765bfff700 1 -- 172.21.15.47:0/3024919932 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f766406c510 con 0x7f766c13e9b0 2022-06-29T15:44:19.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.635+0000 7f766affd700 1 --2- 172.21.15.47:0/3024919932 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7648053b30 0x7f7648055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:19.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.635+0000 7f766affd700 1 --2- 172.21.15.47:0/3024919932 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7648053b30 0x7f7648055ff0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f765c00d000 tx=0x7f765c009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:19.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.643+0000 7f765bfff700 1 -- 172.21.15.47:0/3024919932 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7664043530 con 0x7f766c13e9b0 2022-06-29T15:44:19.816 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:19.811+0000 7f766bfff700 1 -- 172.21.15.47:0/3024919932 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command"]} v 0) v1 -- 0x7f766c0573d0 con 0x7f766c13e9b0 2022-06-29T15:44:20.238 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.231+0000 7f765bfff700 1 -- 172.21.15.47:0/3024919932 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command"]}]=0 v96) v1 ==== 139+0+0 (secure 0 0 0) 0x7f7664016070 con 0x7f766c13e9b0 2022-06-29T15:44:20.242 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.235+0000 7f766bfff700 1 -- 172.21.15.47:0/3024919932 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7648053b30 msgr2=0x7f7648055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:20.242 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.235+0000 7f766bfff700 1 --2- 172.21.15.47:0/3024919932 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7648053b30 0x7f7648055ff0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f765c00d000 tx=0x7f765c009040).stop 2022-06-29T15:44:20.242 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.235+0000 7f766bfff700 1 -- 172.21.15.47:0/3024919932 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f766c13e9b0 msgr2=0x7f766c152270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:20.243 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.235+0000 7f766bfff700 1 --2- 172.21.15.47:0/3024919932 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f766c13e9b0 0x7f766c152270 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7f766c0aa880 tx=0x7f76640091d0).stop 2022-06-29T15:44:20.244 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.235+0000 7f766bfff700 1 -- 172.21.15.47:0/3024919932 shutdown_connections 2022-06-29T15:44:20.244 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.235+0000 7f766bfff700 1 --2- 172.21.15.47:0/3024919932 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f766c13e9b0 0x7f766c152270 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:20.244 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.235+0000 7f766bfff700 1 --2- 172.21.15.47:0/3024919932 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7648053b30 0x7f7648055ff0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:20.245 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.235+0000 7f766bfff700 1 --2- 172.21.15.47:0/3024919932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f766c0b6f10 0x7f766c0b2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:20.245 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.235+0000 7f766bfff700 1 --2- 172.21.15.47:0/3024919932 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f766c061c00 0x7f766c1562b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:20.245 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.235+0000 7f766bfff700 1 -- 172.21.15.47:0/3024919932 >> 172.21.15.47:0/3024919932 conn(0x7f766c063670 msgr2=0x7f766c146710 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:20.245 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.235+0000 7f766bfff700 1 -- 172.21.15.47:0/3024919932 shutdown_connections 2022-06-29T15:44:20.246 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.235+0000 7f766bfff700 1 -- 172.21.15.47:0/3024919932 wait complete. 2022-06-29T15:44:20.259 INFO:tasks.cephfs_test_runner:test_drop_cache_command (tasks.cephfs.test_misc.TestCacheDrop) ... ok 2022-06-29T15:44:20.259 INFO:tasks.cephfs_test_runner:Starting test: test_drop_cache_command_dead (tasks.cephfs.test_misc.TestCacheDrop) 2022-06-29T15:44:20.260 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead' 2022-06-29T15:44:20.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.367+0000 7fd4ba850700 1 Processor -- start 2022-06-29T15:44:20.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.367+0000 7fd4ba850700 1 -- start start 2022-06-29T15:44:20.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.367+0000 7fd4ba850700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd4b40615f0 0x7fd4b41441c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:20.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.367+0000 7fd4ba850700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4b40b19f0 con 0x7fd4b40b0b00 2022-06-29T15:44:20.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.367+0000 7fd4ba850700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4b40619d0 con 0x7fd4b4098200 2022-06-29T15:44:20.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.367+0000 7fd4ba850700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b40b1080 con 0x7fd4b40615f0 2022-06-29T15:44:20.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.367+0000 7fd4abfff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd4b40b0b00 0x7fd4b4097af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33110/0 (socket says 172.21.15.47:33110) 2022-06-29T15:44:20.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.367+0000 7fd4abfff700 1 -- 172.21.15.47:0/4040181822 learned_addr learned my addr 172.21.15.47:0/4040181822 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:44:20.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.367+0000 7fd4b3fff700 1 --2- 172.21.15.47:0/4040181822 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd4b40615f0 0x7fd4b41441c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:20.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.371+0000 7fd4b37fe700 1 -- 172.21.15.47:0/4040181822 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2712523121 0 0) 0x7fd4b40b19f0 con 0x7fd4b40b0b00 2022-06-29T15:44:20.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.371+0000 7fd4b37fe700 1 -- 172.21.15.47:0/4040181822 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd494003260 con 0x7fd4b40b0b00 2022-06-29T15:44:20.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.371+0000 7fd4b37fe700 1 -- 172.21.15.47:0/4040181822 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3315988864 0 0) 0x7fd4b40619d0 con 0x7fd4b4098200 2022-06-29T15:44:20.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.371+0000 7fd4b37fe700 1 -- 172.21.15.47:0/4040181822 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4b40b19f0 con 0x7fd4b4098200 2022-06-29T15:44:20.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.371+0000 7fd4b3fff700 1 -- 172.21.15.47:0/4040181822 >> v1:172.21.15.64:6789/0 conn(0x7fd4b4098200 legacy=0x7fd4b4140520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:20.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.371+0000 7fd4b3fff700 1 -- 172.21.15.47:0/4040181822 >> v1:172.21.15.47:6789/0 conn(0x7fd4b40b0b00 legacy=0x7fd4b4097af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:20.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.371+0000 7fd4b3fff700 1 -- 172.21.15.47:0/4040181822 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4b40b1c20 con 0x7fd4b40615f0 2022-06-29T15:44:20.380 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.371+0000 7fd4b3fff700 1 --2- 172.21.15.47:0/4040181822 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd4b40615f0 0x7fd4b41441c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fd4a4002f70 tx=0x7fd4a4006df0).ready entity=mon.2 client_cookie=1836593fcf61e6fd server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:20.380 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.371+0000 7fd4b37fe700 1 -- 172.21.15.47:0/4040181822 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd4a400d5c0 con 0x7fd4b40615f0 2022-06-29T15:44:20.380 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.371+0000 7fd4b37fe700 1 -- 172.21.15.47:0/4040181822 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a4000e90 con 0x7fd4b40615f0 2022-06-29T15:44:20.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.371+0000 7fd4b37fe700 1 -- 172.21.15.47:0/4040181822 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd4a400d910 con 0x7fd4b40615f0 2022-06-29T15:44:20.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.371+0000 7fd4ba850700 1 -- 172.21.15.47:0/4040181822 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd4b40615f0 msgr2=0x7fd4b41441c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:20.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.371+0000 7fd4ba850700 1 --2- 172.21.15.47:0/4040181822 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd4b40615f0 0x7fd4b41441c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fd4a4002f70 tx=0x7fd4a4006df0).stop 2022-06-29T15:44:20.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.371+0000 7fd4ba850700 1 -- 172.21.15.47:0/4040181822 shutdown_connections 2022-06-29T15:44:20.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.371+0000 7fd4ba850700 1 --2- 172.21.15.47:0/4040181822 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd4b40615f0 0x7fd4b41441c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:20.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.371+0000 7fd4ba850700 1 -- 172.21.15.47:0/4040181822 >> 172.21.15.47:0/4040181822 conn(0x7fd4b405f0b0 msgr2=0x7fd4b405f490 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:20.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.371+0000 7fd4ba850700 1 -- 172.21.15.47:0/4040181822 shutdown_connections 2022-06-29T15:44:20.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.371+0000 7fd4ba850700 1 -- 172.21.15.47:0/4040181822 wait complete. 2022-06-29T15:44:20.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.375+0000 7fd4ba850700 1 Processor -- start 2022-06-29T15:44:20.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.375+0000 7fd4ba850700 1 -- start start 2022-06-29T15:44:20.385 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.375+0000 7fd4ba850700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4b40615f0 0x7fd4b40ade50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:20.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.375+0000 7fd4ba850700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4b4098200 0x7fd4b40af1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:20.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.375+0000 7fd4ba850700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd4b40b0b00 0x7fd4b40af6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:20.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.375+0000 7fd4ba850700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b4142320 con 0x7fd4b4098200 2022-06-29T15:44:20.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.375+0000 7fd4ba850700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b40ae340 con 0x7fd4b40615f0 2022-06-29T15:44:20.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.375+0000 7fd4ba850700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b40997d0 con 0x7fd4b40b0b00 2022-06-29T15:44:20.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.375+0000 7fd4abfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4b4098200 0x7fd4b40af1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:20.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.375+0000 7fd4b3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4b40615f0 0x7fd4b40ade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:20.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.375+0000 7fd4b8ded700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd4b40b0b00 0x7fd4b40af6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:20.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.375+0000 7fd4abfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4b4098200 0x7fd4b40af1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47564/0 (socket says 172.21.15.47:47564) 2022-06-29T15:44:20.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.375+0000 7fd4abfff700 1 -- 172.21.15.47:0/154641193 learned_addr learned my addr 172.21.15.47:0/154641193 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:20.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.379+0000 7fd4abfff700 1 -- 172.21.15.47:0/154641193 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd4b40b0b00 msgr2=0x7fd4b40af6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:20.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.379+0000 7fd4abfff700 1 --2- 172.21.15.47:0/154641193 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd4b40b0b00 0x7fd4b40af6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:20.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.379+0000 7fd4abfff700 1 -- 172.21.15.47:0/154641193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4b40615f0 msgr2=0x7fd4b40ade50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:20.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.379+0000 7fd4abfff700 1 --2- 172.21.15.47:0/154641193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4b40615f0 0x7fd4b40ade50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:20.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.379+0000 7fd4abfff700 1 -- 172.21.15.47:0/154641193 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a400b180 con 0x7fd4b4098200 2022-06-29T15:44:20.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.379+0000 7fd4b8ded700 1 --2- 172.21.15.47:0/154641193 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd4b40b0b00 0x7fd4b40af6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:44:20.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.379+0000 7fd4b3fff700 1 --2- 172.21.15.47:0/154641193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4b40615f0 0x7fd4b40ade50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:44:20.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.379+0000 7fd4abfff700 1 --2- 172.21.15.47:0/154641193 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4b4098200 0x7fd4b40af1e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7fd49c0011d0 tx=0x7fd49c007be0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:20.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.379+0000 7fd4b1ffb700 1 -- 172.21.15.47:0/154641193 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd49c0072e0 con 0x7fd4b4098200 2022-06-29T15:44:20.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.379+0000 7fd4b1ffb700 1 -- 172.21.15.47:0/154641193 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd49c007440 con 0x7fd4b4098200 2022-06-29T15:44:20.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.379+0000 7fd4ba850700 1 -- 172.21.15.47:0/154641193 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4b40994a0 con 0x7fd4b4098200 2022-06-29T15:44:20.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.379+0000 7fd4b1ffb700 1 -- 172.21.15.47:0/154641193 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd49c005800 con 0x7fd4b4098200 2022-06-29T15:44:20.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.379+0000 7fd4ba850700 1 -- 172.21.15.47:0/154641193 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4b4099200 con 0x7fd4b4098200 2022-06-29T15:44:20.394 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.379+0000 7fd4ba850700 1 -- 172.21.15.47:0/154641193 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4b40573d0 con 0x7fd4b4098200 2022-06-29T15:44:20.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.379+0000 7fd4b1ffb700 1 -- 172.21.15.47:0/154641193 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd49c005960 con 0x7fd4b4098200 2022-06-29T15:44:20.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.379+0000 7fd4b1ffb700 1 --2- 172.21.15.47:0/154641193 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd498053b30 0x7fd498055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:20.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.383+0000 7fd4b1ffb700 1 -- 172.21.15.47:0/154641193 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fd49c06e220 con 0x7fd4b4098200 2022-06-29T15:44:20.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.383+0000 7fd4b3fff700 1 --2- 172.21.15.47:0/154641193 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd498053b30 0x7fd498055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:20.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.383+0000 7fd4b3fff700 1 --2- 172.21.15.47:0/154641193 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd498053b30 0x7fd498055ff0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7fd4a4018000 tx=0x7fd4a4014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:20.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.387+0000 7fd4b1ffb700 1 -- 172.21.15.47:0/154641193 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd49c045130 con 0x7fd4b4098200 2022-06-29T15:44:20.562 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:20.555+0000 7fd4ba850700 1 -- 172.21.15.47:0/154641193 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead"]} v 0) v1 -- 0x7fd4b4067c10 con 0x7fd4b4098200 2022-06-29T15:44:21.247 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.239+0000 7fd4b1ffb700 1 -- 172.21.15.47:0/154641193 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead"]}]=0 v97) v1 ==== 147+0+0 (secure 0 0 0) 0x7fd49c018300 con 0x7fd4b4098200 2022-06-29T15:44:21.250 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.243+0000 7fd4ba850700 1 -- 172.21.15.47:0/154641193 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd498053b30 msgr2=0x7fd498055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:21.251 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.243+0000 7fd4ba850700 1 --2- 172.21.15.47:0/154641193 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd498053b30 0x7fd498055ff0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7fd4a4018000 tx=0x7fd4a4014000).stop 2022-06-29T15:44:21.251 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.243+0000 7fd4ba850700 1 -- 172.21.15.47:0/154641193 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4b4098200 msgr2=0x7fd4b40af1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:21.251 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.243+0000 7fd4ba850700 1 --2- 172.21.15.47:0/154641193 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4b4098200 0x7fd4b40af1e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7fd49c0011d0 tx=0x7fd49c007be0).stop 2022-06-29T15:44:21.252 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.247+0000 7fd4ba850700 1 -- 172.21.15.47:0/154641193 shutdown_connections 2022-06-29T15:44:21.252 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.247+0000 7fd4ba850700 1 --2- 172.21.15.47:0/154641193 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd4b40b0b00 0x7fd4b40af6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:21.253 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.247+0000 7fd4ba850700 1 --2- 172.21.15.47:0/154641193 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4b4098200 0x7fd4b40af1e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:21.253 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.247+0000 7fd4ba850700 1 --2- 172.21.15.47:0/154641193 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd498053b30 0x7fd498055ff0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:21.253 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.247+0000 7fd4ba850700 1 --2- 172.21.15.47:0/154641193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4b40615f0 0x7fd4b40ade50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:21.253 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.247+0000 7fd4ba850700 1 -- 172.21.15.47:0/154641193 >> 172.21.15.47:0/154641193 conn(0x7fd4b405f0b0 msgr2=0x7fd4b40b4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:21.254 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.247+0000 7fd4ba850700 1 -- 172.21.15.47:0/154641193 shutdown_connections 2022-06-29T15:44:21.254 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.247+0000 7fd4ba850700 1 -- 172.21.15.47:0/154641193 wait complete. 2022-06-29T15:44:21.267 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-06-29T15:44:21.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.375+0000 7fc9653aa700 1 Processor -- start 2022-06-29T15:44:21.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.375+0000 7fc9653aa700 1 -- start start 2022-06-29T15:44:21.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.379+0000 7fc9653aa700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc96005f0b0 0x7fc96005f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:21.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.379+0000 7fc9653aa700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc960144670 0x7fc960061740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:21.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.379+0000 7fc9653aa700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc960147140 0x7fc960143950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:21.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.379+0000 7fc9653aa700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc960061c30 con 0x7fc96005f0b0 2022-06-29T15:44:21.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.379+0000 7fc9653aa700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc960061e50 con 0x7fc960147140 2022-06-29T15:44:21.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.379+0000 7fc9653aa700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9600620f0 con 0x7fc960144670 2022-06-29T15:44:21.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.379+0000 7fc95ffff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc96005f0b0 0x7fc96005f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:21.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.379+0000 7fc95ffff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc96005f0b0 0x7fc96005f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47570/0 (socket says 172.21.15.47:47570) 2022-06-29T15:44:21.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.379+0000 7fc95ffff700 1 -- 172.21.15.47:0/1805339009 learned_addr learned my addr 172.21.15.47:0/1805339009 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:21.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.379+0000 7fc964ba9700 1 --2- 172.21.15.47:0/1805339009 >> v2:172.21.15.64:3300/0 conn(0x7fc960147140 0x7fc960143950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:21.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.379+0000 7fc9577fe700 1 --2- 172.21.15.47:0/1805339009 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc960144670 0x7fc960061740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:21.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.379+0000 7fc95ffff700 1 -- 172.21.15.47:0/1805339009 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc960144670 msgr2=0x7fc960061740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:44:21.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.379+0000 7fc95ffff700 1 --2- 172.21.15.47:0/1805339009 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc960144670 0x7fc960061740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:21.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.379+0000 7fc95ffff700 1 -- 172.21.15.47:0/1805339009 >> v2:172.21.15.64:3300/0 conn(0x7fc960147140 msgr2=0x7fc960143950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:44:21.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.379+0000 7fc95ffff700 1 --2- 172.21.15.47:0/1805339009 >> v2:172.21.15.64:3300/0 conn(0x7fc960147140 0x7fc960143950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:21.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.379+0000 7fc95ffff700 1 -- 172.21.15.47:0/1805339009 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9601434b0 con 0x7fc96005f0b0 2022-06-29T15:44:21.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.379+0000 7fc964ba9700 1 --2- 172.21.15.47:0/1805339009 >> v2:172.21.15.64:3300/0 conn(0x7fc960147140 0x7fc960143950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:44:21.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.379+0000 7fc9577fe700 1 --2- 172.21.15.47:0/1805339009 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc960144670 0x7fc960061740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:44:21.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.379+0000 7fc95ffff700 1 --2- 172.21.15.47:0/1805339009 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc96005f0b0 0x7fc96005f490 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7fc950002f70 tx=0x7fc950006eb0).ready entity=mon.0 client_cookie=d43cae0ce69e60f3 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:21.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.379+0000 7fc95f7fe700 1 -- 172.21.15.47:0/1805339009 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc95000d570 con 0x7fc96005f0b0 2022-06-29T15:44:21.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.379+0000 7fc95f7fe700 1 -- 172.21.15.47:0/1805339009 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc950000e90 con 0x7fc96005f0b0 2022-06-29T15:44:21.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.379+0000 7fc95f7fe700 1 -- 172.21.15.47:0/1805339009 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc95000d900 con 0x7fc96005f0b0 2022-06-29T15:44:21.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.379+0000 7fc9653aa700 1 -- 172.21.15.47:0/1805339009 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc96005f0b0 msgr2=0x7fc96005f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:21.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.379+0000 7fc9653aa700 1 --2- 172.21.15.47:0/1805339009 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc96005f0b0 0x7fc96005f490 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7fc950002f70 tx=0x7fc950006eb0).stop 2022-06-29T15:44:21.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.379+0000 7fc9653aa700 1 -- 172.21.15.47:0/1805339009 shutdown_connections 2022-06-29T15:44:21.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.379+0000 7fc9653aa700 1 --2- 172.21.15.47:0/1805339009 >> v2:172.21.15.64:3300/0 conn(0x7fc960147140 0x7fc960143950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:21.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.379+0000 7fc9653aa700 1 --2- 172.21.15.47:0/1805339009 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc960144670 0x7fc960061740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:21.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.379+0000 7fc9653aa700 1 --2- 172.21.15.47:0/1805339009 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc96005f0b0 0x7fc96005f490 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:21.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.379+0000 7fc9653aa700 1 -- 172.21.15.47:0/1805339009 >> 172.21.15.47:0/1805339009 conn(0x7fc960062570 msgr2=0x7fc9600647d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:21.394 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.379+0000 7fc9653aa700 1 -- 172.21.15.47:0/1805339009 shutdown_connections 2022-06-29T15:44:21.394 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.379+0000 7fc9653aa700 1 -- 172.21.15.47:0/1805339009 wait complete. 2022-06-29T15:44:21.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.383+0000 7fc9653aa700 1 Processor -- start 2022-06-29T15:44:21.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.383+0000 7fc9653aa700 1 -- start start 2022-06-29T15:44:21.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.383+0000 7fc9653aa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc96005f0b0 0x7fc960144e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:21.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.383+0000 7fc9653aa700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc960144670 0x7fc9600a4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:21.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.383+0000 7fc9653aa700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc960147140 0x7fc9600a4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:21.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.383+0000 7fc9653aa700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9600a5040 con 0x7fc960147140 2022-06-29T15:44:21.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.383+0000 7fc9653aa700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9600b8160 con 0x7fc96005f0b0 2022-06-29T15:44:21.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.383+0000 7fc9653aa700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9600b4060 con 0x7fc960144670 2022-06-29T15:44:21.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.383+0000 7fc964ba9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc960147140 0x7fc9600a4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:21.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.383+0000 7fc964ba9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc960147140 0x7fc9600a4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47580/0 (socket says 172.21.15.47:47580) 2022-06-29T15:44:21.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.383+0000 7fc964ba9700 1 -- 172.21.15.47:0/119177502 learned_addr learned my addr 172.21.15.47:0/119177502 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:21.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.383+0000 7fc95ffff700 1 --2- 172.21.15.47:0/119177502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc96005f0b0 0x7fc960144e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:21.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.383+0000 7fc9577fe700 1 --2- 172.21.15.47:0/119177502 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc960144670 0x7fc9600a4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:21.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.383+0000 7fc964ba9700 1 -- 172.21.15.47:0/119177502 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc960144670 msgr2=0x7fc9600a4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:21.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.383+0000 7fc964ba9700 1 --2- 172.21.15.47:0/119177502 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc960144670 0x7fc9600a4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:21.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.383+0000 7fc964ba9700 1 -- 172.21.15.47:0/119177502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc96005f0b0 msgr2=0x7fc960144e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:21.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.383+0000 7fc964ba9700 1 --2- 172.21.15.47:0/119177502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc96005f0b0 0x7fc960144e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:21.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.383+0000 7fc964ba9700 1 -- 172.21.15.47:0/119177502 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc95000b1e0 con 0x7fc960147140 2022-06-29T15:44:21.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.383+0000 7fc964ba9700 1 --2- 172.21.15.47:0/119177502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc960147140 0x7fc9600a4510 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7fc958006cc0 tx=0x7fc958008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:21.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.383+0000 7fc95dffb700 1 -- 172.21.15.47:0/119177502 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc958011780 con 0x7fc960147140 2022-06-29T15:44:21.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.383+0000 7fc95dffb700 1 -- 172.21.15.47:0/119177502 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9580118e0 con 0x7fc960147140 2022-06-29T15:44:21.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.383+0000 7fc9653aa700 1 -- 172.21.15.47:0/119177502 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9600a3760 con 0x7fc960147140 2022-06-29T15:44:21.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.383+0000 7fc95dffb700 1 -- 172.21.15.47:0/119177502 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc958011c40 con 0x7fc960147140 2022-06-29T15:44:21.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.383+0000 7fc9653aa700 1 -- 172.21.15.47:0/119177502 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9600a3990 con 0x7fc960147140 2022-06-29T15:44:21.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.387+0000 7fc9653aa700 1 -- 172.21.15.47:0/119177502 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9600573d0 con 0x7fc960147140 2022-06-29T15:44:21.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.387+0000 7fc95dffb700 1 -- 172.21.15.47:0/119177502 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc95801c460 con 0x7fc960147140 2022-06-29T15:44:21.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.387+0000 7fc95dffb700 1 --2- 172.21.15.47:0/119177502 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc944053a70 0x7fc944055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:21.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.391+0000 7fc95ffff700 1 --2- 172.21.15.47:0/119177502 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc944053a70 0x7fc944055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:21.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.391+0000 7fc95dffb700 1 -- 172.21.15.47:0/119177502 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fc95806c140 con 0x7fc960147140 2022-06-29T15:44:21.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.391+0000 7fc95ffff700 1 --2- 172.21.15.47:0/119177502 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc944053a70 0x7fc944055f30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7fc95001c000 tx=0x7fc950015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:21.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.395+0000 7fc95dffb700 1 -- 172.21.15.47:0/119177502 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc958043180 con 0x7fc960147140 2022-06-29T15:44:21.532 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.527+0000 7fc9653aa700 1 -- 172.21.15.47:0/119177502 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fc960068830 con 0x7fc960147140 2022-06-29T15:44:21.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.535+0000 7fc95dffb700 1 -- 172.21.15.47:0/119177502 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v4)=0 v4) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc9580467a0 con 0x7fc960147140 2022-06-29T15:44:21.542 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.535+0000 7fc9653aa700 1 -- 172.21.15.47:0/119177502 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc944053a70 msgr2=0x7fc944055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:21.543 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.535+0000 7fc9653aa700 1 --2- 172.21.15.47:0/119177502 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc944053a70 0x7fc944055f30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7fc95001c000 tx=0x7fc950015000).stop 2022-06-29T15:44:21.543 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.535+0000 7fc9653aa700 1 -- 172.21.15.47:0/119177502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc960147140 msgr2=0x7fc9600a4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:21.543 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.535+0000 7fc9653aa700 1 --2- 172.21.15.47:0/119177502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc960147140 0x7fc9600a4510 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7fc958006cc0 tx=0x7fc958008040).stop 2022-06-29T15:44:21.543 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.535+0000 7fc9653aa700 1 -- 172.21.15.47:0/119177502 shutdown_connections 2022-06-29T15:44:21.544 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.535+0000 7fc9653aa700 1 --2- 172.21.15.47:0/119177502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc960147140 0x7fc9600a4510 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:21.544 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.535+0000 7fc9653aa700 1 --2- 172.21.15.47:0/119177502 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc960144670 0x7fc9600a4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:21.544 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.535+0000 7fc9653aa700 1 --2- 172.21.15.47:0/119177502 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc944053a70 0x7fc944055f30 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:21.544 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.535+0000 7fc9653aa700 1 --2- 172.21.15.47:0/119177502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc96005f0b0 0x7fc960144e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:21.544 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.535+0000 7fc9653aa700 1 -- 172.21.15.47:0/119177502 >> 172.21.15.47:0/119177502 conn(0x7fc960062570 msgr2=0x7fc96013fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:21.545 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.535+0000 7fc9653aa700 1 -- 172.21.15.47:0/119177502 shutdown_connections 2022-06-29T15:44:21.545 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.535+0000 7fc9653aa700 1 -- 172.21.15.47:0/119177502 wait complete. 2022-06-29T15:44:21.554 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-06-29T15:44:21.555 DEBUG:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:44:21.573 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:44:21.595 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:44:21.616 DEBUG:teuthology.orchestra.run.smithi164:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:44:21.637 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-06-29T15:44:21.638 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:44:21.638 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:44:21.639 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:44:21.646 INFO:teuthology.orchestra.run.smithi099.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-06-29T15:44:21.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T15:44:21.648 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-06-29T15:44:21.648 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:44:21.649 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:44:21.649 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:44:21.686 INFO:teuthology.orchestra.run.smithi164.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-06-29T15:44:21.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T15:44:21.688 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-06-29T15:44:21.688 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-06-29T15:44:21.800 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.795+0000 7f957cdc6700 1 Processor -- start 2022-06-29T15:44:21.800 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.795+0000 7f957cdc6700 1 -- start start 2022-06-29T15:44:21.803 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.795+0000 7f957cdc6700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f957805f0b0 0x7f95780b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:21.804 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.795+0000 7f957cdc6700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95780b0740 con 0x7f95780b2120 2022-06-29T15:44:21.804 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.795+0000 7f957cdc6700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95780b0970 con 0x7f95780b1260 2022-06-29T15:44:21.804 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.795+0000 7f957cdc6700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9578147460 con 0x7f957805f0b0 2022-06-29T15:44:21.805 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.795+0000 7f9575d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f957805f0b0 0x7f95780b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:21.805 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.795+0000 7f9575d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f957805f0b0 0x7f95780b2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47584/0 (socket says 172.21.15.47:47584) 2022-06-29T15:44:21.805 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.795+0000 7f9575d9b700 1 -- 172.21.15.47:0/4157776510 learned_addr learned my addr 172.21.15.47:0/4157776510 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:21.805 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.795+0000 7f957659c700 1 --1- 172.21.15.47:0/4157776510 >> v1:172.21.15.47:6789/0 conn(0x7f95780b2120 0x7f9578147af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33138/0 (socket says 172.21.15.47:33138) 2022-06-29T15:44:21.806 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.795+0000 7f9575d9b700 1 -- 172.21.15.47:0/4157776510 >> v1:172.21.15.64:6789/0 conn(0x7f95780b1260 legacy=0x7f957805f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:44:21.806 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.795+0000 7f9575d9b700 1 -- 172.21.15.47:0/4157776510 >> v1:172.21.15.47:6789/0 conn(0x7f95780b2120 legacy=0x7f9578147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:21.806 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.795+0000 7f957559a700 1 -- 172.21.15.47:0/4157776510 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2389350222 0 0) 0x7f95780b0740 con 0x7f95780b2120 2022-06-29T15:44:21.806 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.795+0000 7f9575d9b700 1 -- 172.21.15.47:0/4157776510 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95780b0c10 con 0x7f957805f0b0 2022-06-29T15:44:21.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.795+0000 7f9575d9b700 1 --2- 172.21.15.47:0/4157776510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f957805f0b0 0x7f95780b2500 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7f95780affe0 tx=0x7f9568009290).ready entity=mon.0 client_cookie=62911e96c09b885a server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:21.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.795+0000 7f957559a700 1 -- 172.21.15.47:0/4157776510 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f956800b820 con 0x7f957805f0b0 2022-06-29T15:44:21.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.795+0000 7f957559a700 1 -- 172.21.15.47:0/4157776510 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f956800b980 con 0x7f957805f0b0 2022-06-29T15:44:21.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.795+0000 7f957559a700 1 -- 172.21.15.47:0/4157776510 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f956800baf0 con 0x7f957805f0b0 2022-06-29T15:44:21.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.795+0000 7f957cdc6700 1 -- 172.21.15.47:0/4157776510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f957805f0b0 msgr2=0x7f95780b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:21.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.795+0000 7f957cdc6700 1 --2- 172.21.15.47:0/4157776510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f957805f0b0 0x7f95780b2500 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7f95780affe0 tx=0x7f9568009290).stop 2022-06-29T15:44:21.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.799+0000 7f957cdc6700 1 -- 172.21.15.47:0/4157776510 shutdown_connections 2022-06-29T15:44:21.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.799+0000 7f957cdc6700 1 --2- 172.21.15.47:0/4157776510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f957805f0b0 0x7f95780b2500 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:21.809 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.799+0000 7f957cdc6700 1 -- 172.21.15.47:0/4157776510 >> 172.21.15.47:0/4157776510 conn(0x7f95780626f0 msgr2=0x7f9578062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:21.809 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.799+0000 7f957cdc6700 1 -- 172.21.15.47:0/4157776510 shutdown_connections 2022-06-29T15:44:21.809 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.799+0000 7f957cdc6700 1 -- 172.21.15.47:0/4157776510 wait complete. 2022-06-29T15:44:21.813 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.799+0000 7f957cdc6700 1 Processor -- start 2022-06-29T15:44:21.813 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.799+0000 7f957cdc6700 1 -- start start 2022-06-29T15:44:21.814 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.799+0000 7f957cdc6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95780b1260 0x7f9578144800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:21.814 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.799+0000 7f957cdc6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95780b2120 0x7f957814d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:21.814 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.799+0000 7f957cdc6700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f957814d8f0 0x7f95781569f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:21.814 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.799+0000 7f957cdc6700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9578141ab0 con 0x7f95780b1260 2022-06-29T15:44:21.815 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.799+0000 7f957cdc6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9578141ce0 con 0x7f95780b2120 2022-06-29T15:44:21.815 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.799+0000 7f957cdc6700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9578141f80 con 0x7f957814d8f0 2022-06-29T15:44:21.815 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.799+0000 7f957659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95780b1260 0x7f9578144800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:21.815 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.799+0000 7f957659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95780b1260 0x7f9578144800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47590/0 (socket says 172.21.15.47:47590) 2022-06-29T15:44:21.816 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.799+0000 7f957659c700 1 -- 172.21.15.47:0/877565357 learned_addr learned my addr 172.21.15.47:0/877565357 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:21.816 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.799+0000 7f9575d9b700 1 --2- 172.21.15.47:0/877565357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95780b2120 0x7f957814d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:21.816 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.799+0000 7f9576d9d700 1 --2- 172.21.15.47:0/877565357 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f957814d8f0 0x7f95781569f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:21.816 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.799+0000 7f957659c700 1 -- 172.21.15.47:0/877565357 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f957814d8f0 msgr2=0x7f95781569f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:21.817 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.803+0000 7f957659c700 1 --2- 172.21.15.47:0/877565357 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f957814d8f0 0x7f95781569f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:21.817 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.803+0000 7f957659c700 1 -- 172.21.15.47:0/877565357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95780b2120 msgr2=0x7f957814d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:21.817 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.803+0000 7f957659c700 1 --2- 172.21.15.47:0/877565357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95780b2120 0x7f957814d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:21.817 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.803+0000 7f957659c700 1 -- 172.21.15.47:0/877565357 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9568009530 con 0x7f95780b1260 2022-06-29T15:44:21.818 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.803+0000 7f957659c700 1 --2- 172.21.15.47:0/877565357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95780b1260 0x7f9578144800 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7f9560000ed0 tx=0x7f9560007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:21.818 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.803+0000 7f956f7fe700 1 -- 172.21.15.47:0/877565357 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f956000ce50 con 0x7f95780b1260 2022-06-29T15:44:21.818 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.803+0000 7f956f7fe700 1 -- 172.21.15.47:0/877565357 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95600070d0 con 0x7f95780b1260 2022-06-29T15:44:21.819 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.803+0000 7f957cdc6700 1 -- 172.21.15.47:0/877565357 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9578142220 con 0x7f95780b1260 2022-06-29T15:44:21.819 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.803+0000 7f956f7fe700 1 -- 172.21.15.47:0/877565357 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9560007430 con 0x7f95780b1260 2022-06-29T15:44:21.819 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.803+0000 7f957cdc6700 1 -- 172.21.15.47:0/877565357 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f957814eaf0 con 0x7f95780b1260 2022-06-29T15:44:21.819 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.803+0000 7f957cdc6700 1 -- 172.21.15.47:0/877565357 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f957814d0c0 con 0x7f95780b1260 2022-06-29T15:44:21.820 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.803+0000 7f956f7fe700 1 -- 172.21.15.47:0/877565357 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9560010070 con 0x7f95780b1260 2022-06-29T15:44:21.820 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.803+0000 7f956f7fe700 1 --2- 172.21.15.47:0/877565357 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9550053a70 0x7f9550055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:21.820 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.807+0000 7f956f7fe700 1 -- 172.21.15.47:0/877565357 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f956006b370 con 0x7f95780b1260 2022-06-29T15:44:21.820 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.807+0000 7f9575d9b700 1 --2- 172.21.15.47:0/877565357 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9550053a70 0x7f9550055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:21.821 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.807+0000 7f9575d9b700 1 --2- 172.21.15.47:0/877565357 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9550053a70 0x7f9550055f30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f956801b000 tx=0x7f9568016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:21.821 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.811+0000 7f956f7fe700 1 -- 172.21.15.47:0/877565357 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9560042410 con 0x7f95780b1260 2022-06-29T15:44:21.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.939+0000 7f957cdc6700 1 -- 172.21.15.47:0/877565357 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f9578147650 con 0x7f95780b1260 2022-06-29T15:44:21.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.943+0000 7f956f7fe700 1 -- 172.21.15.47:0/877565357 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v39) v1 ==== 81+0+29219 (secure 0 0 0) 0x7f9560045a30 con 0x7f95780b1260 2022-06-29T15:44:21.948 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:44:21.949 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2022-06-29T15:44:21.949 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 39, 2022-06-29T15:44:21.949 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "9a68894a-53ce-400e-838b-14eab91b834c", 2022-06-29T15:44:21.949 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2022-06-29T15:42:14.046361+0000", 2022-06-29T15:44:21.950 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2022-06-29T15:44:19.215956+0000", 2022-06-29T15:44:21.950 INFO:teuthology.orchestra.run.smithi047.stdout: "last_up_change": "2022-06-29T15:42:20.929789+0000", 2022-06-29T15:44:21.950 INFO:teuthology.orchestra.run.smithi047.stdout: "last_in_change": "2022-06-29T15:42:20.929789+0000", 2022-06-29T15:44:21.951 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-06-29T15:44:21.951 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_num": 5799936, 2022-06-29T15:44:21.951 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_set": [ 2022-06-29T15:44:21.951 INFO:teuthology.orchestra.run.smithi047.stdout: "pglog_hardlimit", 2022-06-29T15:44:21.951 INFO:teuthology.orchestra.run.smithi047.stdout: "purged_snapdirs", 2022-06-29T15:44:21.952 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_deletes", 2022-06-29T15:44:21.952 INFO:teuthology.orchestra.run.smithi047.stdout: "sortbitwise" 2022-06-29T15:44:21.952 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:44:21.952 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_version": 5, 2022-06-29T15:44:21.953 INFO:teuthology.orchestra.run.smithi047.stdout: "full_ratio": 0.94999998807907104, 2022-06-29T15:44:21.953 INFO:teuthology.orchestra.run.smithi047.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-06-29T15:44:21.953 INFO:teuthology.orchestra.run.smithi047.stdout: "nearfull_ratio": 0.85000002384185791, 2022-06-29T15:44:21.953 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_snapshot": "", 2022-06-29T15:44:21.953 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_max": 6, 2022-06-29T15:44:21.954 INFO:teuthology.orchestra.run.smithi047.stdout: "max_osd": 8, 2022-06-29T15:44:21.954 INFO:teuthology.orchestra.run.smithi047.stdout: "require_min_compat_client": "jewel", 2022-06-29T15:44:21.954 INFO:teuthology.orchestra.run.smithi047.stdout: "min_compat_client": "jewel", 2022-06-29T15:44:21.954 INFO:teuthology.orchestra.run.smithi047.stdout: "require_osd_release": "octopus", 2022-06-29T15:44:21.955 INFO:teuthology.orchestra.run.smithi047.stdout: "pools": [ 2022-06-29T15:44:21.955 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:21.955 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 1, 2022-06-29T15:44:21.955 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "device_health_metrics", 2022-06-29T15:44:21.956 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:42:16.971431+0000", 2022-06-29T15:44:21.956 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:44:21.956 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:44:21.956 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:44:21.957 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:44:21.957 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:44:21.957 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:44:21.957 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:44:21.958 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:44:21.958 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 1, 2022-06-29T15:44:21.958 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 1, 2022-06-29T15:44:21.958 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 1, 2022-06-29T15:44:21.958 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 1, 2022-06-29T15:44:21.959 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 1, 2022-06-29T15:44:21.959 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:44:21.959 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:44:21.959 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:44:21.960 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:44:21.960 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:44:21.960 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:44:21.960 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:44:21.961 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:21.961 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "16", 2022-06-29T15:44:21.961 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:44:21.961 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:44:21.962 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:44:21.962 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:44:21.962 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:44:21.962 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:44:21.962 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:44:21.963 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:44:21.963 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:44:21.963 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:44:21.963 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:44:21.964 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:44:21.964 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:44:21.964 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:44:21.964 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:44:21.965 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:44:21.965 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:44:21.965 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:44:21.965 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:44:21.965 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:44:21.966 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:44:21.966 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:44:21.966 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:44:21.966 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:44:21.967 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:44:21.967 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:44:21.967 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:21.967 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:44:21.968 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:44:21.968 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:44:21.968 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:44:21.968 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:44:21.969 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:44:21.969 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:44:21.969 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:44:21.969 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:44:21.969 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:44:21.970 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:44:21.970 INFO:teuthology.orchestra.run.smithi047.stdout: "options": { 2022-06-29T15:44:21.970 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_min": 1 2022-06-29T15:44:21.970 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:21.971 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:44:21.971 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_devicehealth": {} 2022-06-29T15:44:21.971 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:21.971 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:21.972 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:21.972 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 2, 2022-06-29T15:44:21.972 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "rbd", 2022-06-29T15:44:21.972 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:42:25.290884+0000", 2022-06-29T15:44:21.973 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:44:21.973 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:44:21.973 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:44:21.973 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:44:21.974 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:44:21.974 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:44:21.974 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:44:21.974 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:44:21.975 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2022-06-29T15:44:21.975 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2022-06-29T15:44:21.975 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2022-06-29T15:44:21.975 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2022-06-29T15:44:21.975 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2022-06-29T15:44:21.976 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:44:21.976 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:44:21.976 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:44:21.976 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:44:21.977 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:44:21.977 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:44:21.977 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:44:21.977 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:21.978 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "19", 2022-06-29T15:44:21.978 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:44:21.978 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:44:21.978 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:44:21.979 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:44:21.979 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:44:21.979 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:44:21.979 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:44:21.980 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:44:21.980 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:44:21.980 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:44:21.980 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:44:21.980 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:44:21.981 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:44:21.981 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:44:21.981 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:44:21.981 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:44:21.982 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:44:21.982 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:44:21.982 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:44:21.982 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:44:21.982 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:44:21.983 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:44:21.983 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:44:21.983 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:44:21.983 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:44:21.984 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:44:21.984 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:21.984 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:44:21.984 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:44:21.985 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:44:21.985 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:44:21.985 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:44:21.985 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:44:21.986 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:44:21.986 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:44:21.986 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:44:21.986 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:44:21.986 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:44:21.987 INFO:teuthology.orchestra.run.smithi047.stdout: "options": {}, 2022-06-29T15:44:21.987 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:44:21.987 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd": {} 2022-06-29T15:44:21.987 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:21.988 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:21.988 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:44:21.988 INFO:teuthology.orchestra.run.smithi047.stdout: "osds": [ 2022-06-29T15:44:21.988 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:21.989 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2022-06-29T15:44:21.989 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "e23eb023-4706-4201-bf72-4bf19c912c5d", 2022-06-29T15:44:21.989 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:44:21.989 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:44:21.990 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:44:21.990 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:44:21.990 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:44:21.990 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:44:21.990 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:44:21.991 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 34, 2022-06-29T15:44:21.991 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:44:21.991 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:44:21.991 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:44:21.992 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:21.992 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:21.992 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:21.992 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6818", 2022-06-29T15:44:21.993 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:44:21.993 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:21.993 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:21.993 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:21.993 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6819", 2022-06-29T15:44:21.994 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:44:21.994 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:21.994 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:21.994 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:21.995 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:44:21.995 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:21.995 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:21.995 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:21.996 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6820", 2022-06-29T15:44:21.996 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:44:21.996 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:21.996 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:21.996 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:21.997 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6821", 2022-06-29T15:44:21.997 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:44:21.997 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:21.997 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:21.998 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:21.998 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:44:21.998 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:21.998 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:21.999 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:21.999 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6824", 2022-06-29T15:44:21.999 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:44:21.999 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:21.999 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.000 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:22.000 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6825", 2022-06-29T15:44:22.000 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:44:22.000 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:22.001 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:22.001 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.001 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:44:22.001 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:22.002 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.002 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:22.002 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6822", 2022-06-29T15:44:22.002 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:44:22.002 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.003 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.003 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:22.003 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6823", 2022-06-29T15:44:22.003 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:44:22.004 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:22.004 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:22.004 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.004 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6819/13096", 2022-06-29T15:44:22.005 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6821/13096", 2022-06-29T15:44:22.005 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6825/13096", 2022-06-29T15:44:22.005 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6823/13096", 2022-06-29T15:44:22.005 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:44:22.005 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:44:22.006 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:44:22.006 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:22.006 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.006 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.007 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2022-06-29T15:44:22.007 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "228bb08d-2d64-43d9-9560-bfaeff154452", 2022-06-29T15:44:22.007 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:44:22.007 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:44:22.008 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:44:22.008 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:44:22.008 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:44:22.008 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:44:22.008 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:44:22.009 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 34, 2022-06-29T15:44:22.009 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:44:22.009 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:44:22.009 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:44:22.010 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:22.010 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.010 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:22.010 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6826", 2022-06-29T15:44:22.011 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:44:22.011 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.011 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.011 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:22.012 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6827", 2022-06-29T15:44:22.012 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:44:22.012 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:22.012 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:22.012 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.013 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:44:22.013 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:22.013 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.013 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:22.014 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6828", 2022-06-29T15:44:22.014 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:44:22.014 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.014 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.015 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:22.015 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6829", 2022-06-29T15:44:22.015 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:44:22.015 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:22.016 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:22.016 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.016 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:44:22.016 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:22.017 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.017 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:22.017 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6832", 2022-06-29T15:44:22.017 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:44:22.018 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.018 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.018 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:22.018 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6833", 2022-06-29T15:44:22.018 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:44:22.019 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:22.019 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:22.019 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.019 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:44:22.020 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:22.020 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.020 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:22.020 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6830", 2022-06-29T15:44:22.021 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:44:22.021 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.021 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.021 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:22.022 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6831", 2022-06-29T15:44:22.022 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:44:22.022 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:22.022 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:22.023 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.023 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6827/13100", 2022-06-29T15:44:22.023 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6829/13100", 2022-06-29T15:44:22.023 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6833/13100", 2022-06-29T15:44:22.024 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6831/13100", 2022-06-29T15:44:22.024 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:44:22.024 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:44:22.024 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:44:22.025 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:22.025 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.025 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.025 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2022-06-29T15:44:22.025 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "797a5452-1510-4640-a111-c9c2fcf707eb", 2022-06-29T15:44:22.026 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:44:22.026 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:44:22.026 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:44:22.026 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:44:22.027 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:44:22.027 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:44:22.027 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:44:22.027 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 33, 2022-06-29T15:44:22.028 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:44:22.028 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:44:22.028 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:44:22.028 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:22.029 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.029 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:22.029 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6810", 2022-06-29T15:44:22.029 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:44:22.029 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.030 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.030 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:22.030 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6811", 2022-06-29T15:44:22.030 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:44:22.031 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:22.031 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:22.031 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.031 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:44:22.032 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:22.032 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.032 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:22.032 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6812", 2022-06-29T15:44:22.033 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:44:22.033 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.033 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.033 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:22.034 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6813", 2022-06-29T15:44:22.034 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:44:22.034 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:22.034 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:22.035 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.035 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:44:22.035 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:22.035 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.035 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:22.036 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6816", 2022-06-29T15:44:22.036 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:44:22.036 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.036 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.037 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:22.037 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6817", 2022-06-29T15:44:22.037 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:44:22.037 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:22.038 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:22.038 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.038 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:44:22.038 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:22.039 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.039 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:22.039 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6814", 2022-06-29T15:44:22.039 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:44:22.039 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.040 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.040 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:22.040 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6815", 2022-06-29T15:44:22.040 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:44:22.041 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:22.041 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:22.041 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.041 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6811/13098", 2022-06-29T15:44:22.041 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6813/13098", 2022-06-29T15:44:22.042 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6817/13098", 2022-06-29T15:44:22.042 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6815/13098", 2022-06-29T15:44:22.042 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:44:22.042 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:44:22.043 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:44:22.043 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:22.043 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.043 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.043 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2022-06-29T15:44:22.044 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "55b24fd7-bb71-4a01-b3fa-a09caa772a62", 2022-06-29T15:44:22.044 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:44:22.044 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:44:22.044 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:44:22.045 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:44:22.045 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:44:22.045 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:44:22.045 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:44:22.045 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 34, 2022-06-29T15:44:22.046 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:44:22.046 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:44:22.046 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:44:22.046 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:22.047 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.047 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:22.047 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6802", 2022-06-29T15:44:22.047 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:44:22.048 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.048 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.048 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:22.048 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6803", 2022-06-29T15:44:22.048 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:44:22.049 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:22.049 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:22.049 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.049 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:44:22.050 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:22.050 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.050 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:22.050 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6804", 2022-06-29T15:44:22.050 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:44:22.051 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.051 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.051 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:22.051 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6805", 2022-06-29T15:44:22.052 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:44:22.052 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:22.052 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:22.052 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.053 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:44:22.053 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:22.053 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.053 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:22.054 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6808", 2022-06-29T15:44:22.054 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:44:22.054 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.054 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.055 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:22.055 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6809", 2022-06-29T15:44:22.055 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:44:22.055 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:22.055 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:22.056 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.056 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:44:22.056 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:22.056 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.057 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:22.057 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6806", 2022-06-29T15:44:22.057 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:44:22.057 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.058 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.058 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:22.058 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6807", 2022-06-29T15:44:22.058 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:44:22.059 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:22.059 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:22.059 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.059 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6803/13097", 2022-06-29T15:44:22.059 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6805/13097", 2022-06-29T15:44:22.060 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6809/13097", 2022-06-29T15:44:22.060 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6807/13097", 2022-06-29T15:44:22.060 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:44:22.060 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:44:22.061 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:44:22.061 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:22.061 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.061 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.061 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2022-06-29T15:44:22.062 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3", 2022-06-29T15:44:22.062 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:44:22.062 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:44:22.062 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:44:22.063 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:44:22.063 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:44:22.063 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:44:22.063 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:44:22.064 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 34, 2022-06-29T15:44:22.064 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:44:22.064 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:44:22.064 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:44:22.064 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:22.065 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.065 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:22.065 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6816", 2022-06-29T15:44:22.065 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:44:22.066 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.066 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.066 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:22.066 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6817", 2022-06-29T15:44:22.066 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:44:22.067 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:22.067 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:22.067 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.068 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:44:22.068 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:22.068 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.068 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:22.068 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6818", 2022-06-29T15:44:22.069 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:44:22.069 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.069 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.069 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:22.070 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6819", 2022-06-29T15:44:22.070 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:44:22.070 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:22.070 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:22.071 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.071 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:44:22.071 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:22.071 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.071 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:22.072 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6822", 2022-06-29T15:44:22.072 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:44:22.072 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.072 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.073 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:22.073 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6823", 2022-06-29T15:44:22.073 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:44:22.073 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:22.074 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:22.074 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.074 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:44:22.074 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:22.075 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.075 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:22.075 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6820", 2022-06-29T15:44:22.075 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:44:22.076 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.076 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.076 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:22.076 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6821", 2022-06-29T15:44:22.077 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:44:22.077 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:22.077 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:22.077 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.077 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6817/13396", 2022-06-29T15:44:22.078 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6819/13396", 2022-06-29T15:44:22.078 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6823/13396", 2022-06-29T15:44:22.078 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6821/13396", 2022-06-29T15:44:22.078 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:44:22.079 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:44:22.079 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:44:22.079 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:22.079 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.080 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.080 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2022-06-29T15:44:22.080 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "f77beebe-6cca-4244-b2d4-677825ebf919", 2022-06-29T15:44:22.080 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:44:22.081 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:44:22.081 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:44:22.081 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:44:22.081 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:44:22.081 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:44:22.082 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:44:22.082 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 34, 2022-06-29T15:44:22.083 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:44:22.083 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:44:22.083 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:44:22.083 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:22.084 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.084 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:22.084 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6800", 2022-06-29T15:44:22.084 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:44:22.085 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.085 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.085 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:22.085 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6801", 2022-06-29T15:44:22.085 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:44:22.086 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:22.086 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:22.086 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.086 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:44:22.087 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:22.087 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.087 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:22.087 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6802", 2022-06-29T15:44:22.087 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:44:22.088 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.088 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.088 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:22.088 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6803", 2022-06-29T15:44:22.089 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:44:22.089 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:22.089 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:22.089 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.089 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:44:22.090 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:22.090 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.090 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:22.090 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6806", 2022-06-29T15:44:22.091 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:44:22.091 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.091 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.091 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:22.091 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6807", 2022-06-29T15:44:22.092 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:44:22.092 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:22.092 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:22.092 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.093 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:44:22.093 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:22.093 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.093 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:22.093 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6804", 2022-06-29T15:44:22.094 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:44:22.094 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.094 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.094 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:22.095 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6805", 2022-06-29T15:44:22.095 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:44:22.095 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:22.095 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:22.095 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.096 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6801/13393", 2022-06-29T15:44:22.096 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6803/13393", 2022-06-29T15:44:22.096 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6807/13393", 2022-06-29T15:44:22.096 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6805/13393", 2022-06-29T15:44:22.097 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:44:22.097 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:44:22.097 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:44:22.097 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:22.097 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.098 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.098 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2022-06-29T15:44:22.098 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "af0ca9b3-c02c-4910-834b-bf0b8dc6713c", 2022-06-29T15:44:22.098 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:44:22.098 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:44:22.099 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:44:22.099 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:44:22.099 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:44:22.099 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:44:22.099 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:44:22.100 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 33, 2022-06-29T15:44:22.100 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:44:22.100 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:44:22.100 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:44:22.101 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:22.101 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.101 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:22.101 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6824", 2022-06-29T15:44:22.101 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:44:22.102 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.102 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.102 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:22.102 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6825", 2022-06-29T15:44:22.103 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:44:22.103 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:22.103 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:22.103 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.103 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:44:22.104 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:22.104 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.104 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:22.104 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6826", 2022-06-29T15:44:22.104 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:44:22.105 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.105 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.105 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:22.105 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6827", 2022-06-29T15:44:22.106 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:44:22.106 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:22.106 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:22.106 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.107 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:44:22.107 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:22.107 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.107 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:22.107 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6830", 2022-06-29T15:44:22.108 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:44:22.108 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.108 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.108 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:22.109 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6831", 2022-06-29T15:44:22.109 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:44:22.109 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:22.109 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:22.109 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.110 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:44:22.110 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:22.110 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.110 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:22.110 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6828", 2022-06-29T15:44:22.111 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:44:22.111 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.111 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.111 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:22.112 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6829", 2022-06-29T15:44:22.112 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:44:22.112 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:22.112 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:22.112 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.113 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6825/13395", 2022-06-29T15:44:22.113 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6827/13395", 2022-06-29T15:44:22.113 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6831/13395", 2022-06-29T15:44:22.113 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6829/13395", 2022-06-29T15:44:22.113 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:44:22.114 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:44:22.114 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:44:22.114 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:22.114 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.115 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.115 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2022-06-29T15:44:22.115 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "f19c3ca6-df88-4bde-a150-b51a53f092e1", 2022-06-29T15:44:22.115 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:44:22.116 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:44:22.116 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:44:22.116 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:44:22.116 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:44:22.117 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:44:22.117 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:44:22.117 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 34, 2022-06-29T15:44:22.117 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:44:22.117 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:44:22.118 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:44:22.118 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:22.118 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.118 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:22.119 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6808", 2022-06-29T15:44:22.119 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:44:22.119 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.119 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.119 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:22.120 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6809", 2022-06-29T15:44:22.120 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:44:22.120 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:22.120 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:22.121 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.121 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:44:22.121 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:22.121 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.121 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:22.122 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6810", 2022-06-29T15:44:22.122 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:44:22.122 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.122 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.123 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:22.123 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6811", 2022-06-29T15:44:22.123 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:44:22.123 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:22.123 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:22.124 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.124 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:44:22.124 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:22.124 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.125 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:22.125 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6814", 2022-06-29T15:44:22.125 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:44:22.125 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.125 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.126 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:22.126 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6815", 2022-06-29T15:44:22.126 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:44:22.126 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:22.127 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:22.127 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.127 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:44:22.127 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:44:22.127 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.128 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:44:22.128 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6812", 2022-06-29T15:44:22.128 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:44:22.128 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.128 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.129 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:44:22.129 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6813", 2022-06-29T15:44:22.129 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:44:22.129 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:22.130 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:22.130 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.130 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6809/13394", 2022-06-29T15:44:22.130 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6811/13394", 2022-06-29T15:44:22.130 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6815/13394", 2022-06-29T15:44:22.131 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6813/13394", 2022-06-29T15:44:22.131 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:44:22.131 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:44:22.131 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:44:22.131 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:22.132 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:22.132 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:44:22.132 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_xinfo": [ 2022-06-29T15:44:22.132 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.133 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2022-06-29T15:44:22.133 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:44:22.133 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:44:22.133 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:44:22.133 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:44:22.134 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:44:22.134 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.720513+0000", 2022-06-29T15:44:22.134 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:44:22.134 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.135 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.135 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2022-06-29T15:44:22.135 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:44:22.135 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:44:22.136 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:44:22.136 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:44:22.136 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:44:22.136 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.772612+0000", 2022-06-29T15:44:22.136 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:44:22.137 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.137 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.137 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2022-06-29T15:44:22.137 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:44:22.138 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:44:22.138 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:44:22.138 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:44:22.138 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:44:22.138 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.450223+0000", 2022-06-29T15:44:22.139 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:44:22.139 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.139 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.139 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2022-06-29T15:44:22.139 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:44:22.140 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:44:22.140 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:44:22.140 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:44:22.140 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:44:22.141 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.477856+0000", 2022-06-29T15:44:22.141 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:44:22.141 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.141 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.142 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2022-06-29T15:44:22.142 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:44:22.142 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:44:22.142 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:44:22.142 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:44:22.143 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:44:22.143 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.539485+0000", 2022-06-29T15:44:22.143 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:44:22.143 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.144 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.144 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2022-06-29T15:44:22.144 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:44:22.144 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:44:22.144 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:44:22.145 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:44:22.145 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:44:22.145 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.487040+0000", 2022-06-29T15:44:22.145 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:44:22.145 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.146 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.146 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2022-06-29T15:44:22.146 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:44:22.146 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:44:22.147 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:44:22.147 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:44:22.147 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:44:22.147 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.748685+0000", 2022-06-29T15:44:22.147 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:44:22.148 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.148 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:22.148 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2022-06-29T15:44:22.148 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:44:22.149 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:44:22.149 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:44:22.149 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:44:22.149 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:44:22.149 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.459516+0000", 2022-06-29T15:44:22.150 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:44:22.150 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:22.150 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:44:22.150 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap": [], 2022-06-29T15:44:22.151 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap_items": [], 2022-06-29T15:44:22.151 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_temp": [], 2022-06-29T15:44:22.151 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_temp": [], 2022-06-29T15:44:22.151 INFO:teuthology.orchestra.run.smithi047.stdout: "blacklist": { 2022-06-29T15:44:22.151 INFO:teuthology.orchestra.run.smithi047.stdout: "172.21.15.47:6837/2219068343": "2022-06-30T15:44:15.449965+0000", 2022-06-29T15:44:22.152 INFO:teuthology.orchestra.run.smithi047.stdout: "172.21.15.47:6836/2219068343": "2022-06-30T15:44:15.449965+0000" 2022-06-29T15:44:22.152 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.152 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profiles": { 2022-06-29T15:44:22.152 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs_data": { 2022-06-29T15:44:22.153 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-device-class": "", 2022-06-29T15:44:22.153 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2022-06-29T15:44:22.153 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-root": "default", 2022-06-29T15:44:22.153 INFO:teuthology.orchestra.run.smithi047.stdout: "jerasure-per-chunk-alignment": "false", 2022-06-29T15:44:22.153 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2022-06-29T15:44:22.154 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "2", 2022-06-29T15:44:22.154 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2022-06-29T15:44:22.154 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2022-06-29T15:44:22.154 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van", 2022-06-29T15:44:22.155 INFO:teuthology.orchestra.run.smithi047.stdout: "w": "8" 2022-06-29T15:44:22.155 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.155 INFO:teuthology.orchestra.run.smithi047.stdout: "default": { 2022-06-29T15:44:22.155 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2022-06-29T15:44:22.155 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2022-06-29T15:44:22.156 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "1", 2022-06-29T15:44:22.156 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2022-06-29T15:44:22.156 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2022-06-29T15:44:22.156 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van" 2022-06-29T15:44:22.157 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:22.157 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:22.157 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps_queue": [], 2022-06-29T15:44:22.157 INFO:teuthology.orchestra.run.smithi047.stdout: "new_removed_snaps": [], 2022-06-29T15:44:22.157 INFO:teuthology.orchestra.run.smithi047.stdout: "new_purged_snaps": [], 2022-06-29T15:44:22.158 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_node_flags": {}, 2022-06-29T15:44:22.158 INFO:teuthology.orchestra.run.smithi047.stdout: "device_class_flags": {} 2022-06-29T15:44:22.158 INFO:teuthology.orchestra.run.smithi047.stdout:} 2022-06-29T15:44:22.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.947+0000 7f957cdc6700 1 -- 172.21.15.47:0/877565357 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9550053a70 msgr2=0x7f9550055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:22.161 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.947+0000 7f957cdc6700 1 --2- 172.21.15.47:0/877565357 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9550053a70 0x7f9550055f30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f956801b000 tx=0x7f9568016000).stop 2022-06-29T15:44:22.161 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.947+0000 7f957cdc6700 1 -- 172.21.15.47:0/877565357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95780b1260 msgr2=0x7f9578144800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:22.161 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.947+0000 7f957cdc6700 1 --2- 172.21.15.47:0/877565357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95780b1260 0x7f9578144800 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7f9560000ed0 tx=0x7f9560007c80).stop 2022-06-29T15:44:22.161 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.947+0000 7f957cdc6700 1 -- 172.21.15.47:0/877565357 shutdown_connections 2022-06-29T15:44:22.161 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.947+0000 7f957cdc6700 1 --2- 172.21.15.47:0/877565357 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f957814d8f0 0x7f95781569f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:22.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.947+0000 7f957cdc6700 1 --2- 172.21.15.47:0/877565357 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9550053a70 0x7f9550055f30 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:22.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.947+0000 7f957cdc6700 1 --2- 172.21.15.47:0/877565357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95780b2120 0x7f957814d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:22.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.947+0000 7f957cdc6700 1 --2- 172.21.15.47:0/877565357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95780b1260 0x7f9578144800 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:22.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.947+0000 7f957cdc6700 1 -- 172.21.15.47:0/877565357 >> 172.21.15.47:0/877565357 conn(0x7f95780626f0 msgr2=0x7f9578146c60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:22.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.947+0000 7f957cdc6700 1 -- 172.21.15.47:0/877565357 shutdown_connections 2022-06-29T15:44:22.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:21.947+0000 7f957cdc6700 1 -- 172.21.15.47:0/877565357 wait complete. 2022-06-29T15:44:22.164 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:44:22.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.271+0000 7fa23d2db700 1 Processor -- start 2022-06-29T15:44:22.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.271+0000 7fa23d2db700 1 -- start start 2022-06-29T15:44:22.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.271+0000 7fa23d2db700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa23805c510 0x7fa238143bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:22.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.271+0000 7fa23d2db700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa23805c8f0 0x7fa2380b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:22.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.271+0000 7fa23d2db700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa238142f90 con 0x7fa2380b2c40 2022-06-29T15:44:22.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.271+0000 7fa23d2db700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2381431f0 con 0x7fa23805c8f0 2022-06-29T15:44:22.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.271+0000 7fa23d2db700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa238143490 con 0x7fa23805c510 2022-06-29T15:44:22.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.271+0000 7fa2377fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa23805c8f0 0x7fa2380b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:22.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.271+0000 7fa237fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa23805c510 0x7fa238143bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:22.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.271+0000 7fa2377fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa23805c8f0 0x7fa2380b4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:53304/0 (socket says 172.21.15.47:53304) 2022-06-29T15:44:22.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.271+0000 7fa2377fe700 1 -- 172.21.15.47:0/2335377308 learned_addr learned my addr 172.21.15.47:0/2335377308 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:22.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.271+0000 7fa2377fe700 1 -- 172.21.15.47:0/2335377308 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa23805c510 msgr2=0x7fa238143bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:44:22.282 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.271+0000 7fa2377fe700 1 --2- 172.21.15.47:0/2335377308 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa23805c510 0x7fa238143bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:22.282 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.271+0000 7fa2377fe700 1 -- 172.21.15.47:0/2335377308 >> v1:172.21.15.64:6789/0 conn(0x7fa2380b2c40 legacy=0x7fa23813cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:22.282 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.271+0000 7fa2377fe700 1 -- 172.21.15.47:0/2335377308 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa238143730 con 0x7fa23805c8f0 2022-06-29T15:44:22.282 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.271+0000 7fa2377fe700 1 --2- 172.21.15.47:0/2335377308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa23805c8f0 0x7fa2380b4100 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7fa22c002f70 tx=0x7fa22c006eb0).ready entity=mon.1 client_cookie=528f940e564592fa server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:22.283 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.275+0000 7fa236ffd700 1 -- 172.21.15.47:0/2335377308 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa22c00d570 con 0x7fa23805c8f0 2022-06-29T15:44:22.283 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.275+0000 7fa236ffd700 1 -- 172.21.15.47:0/2335377308 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa22c000e90 con 0x7fa23805c8f0 2022-06-29T15:44:22.283 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.275+0000 7fa236ffd700 1 -- 172.21.15.47:0/2335377308 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa22c00d900 con 0x7fa23805c8f0 2022-06-29T15:44:22.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.275+0000 7fa23d2db700 1 -- 172.21.15.47:0/2335377308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa23805c8f0 msgr2=0x7fa2380b4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:22.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.275+0000 7fa23d2db700 1 --2- 172.21.15.47:0/2335377308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa23805c8f0 0x7fa2380b4100 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7fa22c002f70 tx=0x7fa22c006eb0).stop 2022-06-29T15:44:22.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.275+0000 7fa23d2db700 1 -- 172.21.15.47:0/2335377308 shutdown_connections 2022-06-29T15:44:22.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.275+0000 7fa23d2db700 1 --2- 172.21.15.47:0/2335377308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa23805c8f0 0x7fa2380b4100 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:22.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.275+0000 7fa23d2db700 1 --2- 172.21.15.47:0/2335377308 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa23805c510 0x7fa238143bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:22.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.275+0000 7fa23d2db700 1 -- 172.21.15.47:0/2335377308 >> 172.21.15.47:0/2335377308 conn(0x7fa238060e00 msgr2=0x7fa2380611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:22.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.275+0000 7fa23d2db700 1 -- 172.21.15.47:0/2335377308 shutdown_connections 2022-06-29T15:44:22.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.275+0000 7fa23d2db700 1 -- 172.21.15.47:0/2335377308 wait complete. 2022-06-29T15:44:22.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.275+0000 7fa23d2db700 1 Processor -- start 2022-06-29T15:44:22.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.275+0000 7fa23d2db700 1 -- start start 2022-06-29T15:44:22.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.275+0000 7fa23d2db700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa23805c510 0x7fa2380af180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:22.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.275+0000 7fa23d2db700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa23805c8f0 0x7fa2380af960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:22.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.275+0000 7fa23d2db700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2380b2c40 0x7fa238152790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:22.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.275+0000 7fa23d2db700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2380b5880 con 0x7fa2380b2c40 2022-06-29T15:44:22.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.275+0000 7fa23d2db700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2380b50a0 con 0x7fa23805c510 2022-06-29T15:44:22.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.275+0000 7fa23d2db700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2380b5340 con 0x7fa23805c8f0 2022-06-29T15:44:22.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.275+0000 7fa23cada700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2380b2c40 0x7fa238152790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:22.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.279+0000 7fa2377fe700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa23805c8f0 0x7fa2380af960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:22.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.279+0000 7fa237fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa23805c510 0x7fa2380af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:22.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.279+0000 7fa2377fe700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa23805c8f0 0x7fa2380af960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:44706/0 (socket says 172.21.15.47:44706) 2022-06-29T15:44:22.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.279+0000 7fa23cada700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2380b2c40 0x7fa238152790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47608/0 (socket says 172.21.15.47:47608) 2022-06-29T15:44:22.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.279+0000 7fa2377fe700 1 -- 172.21.15.47:0/1077231952 learned_addr learned my addr 172.21.15.47:0/1077231952 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:22.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.279+0000 7fa237fff700 1 -- 172.21.15.47:0/1077231952 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa23805c8f0 msgr2=0x7fa2380af960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:22.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.279+0000 7fa237fff700 1 --2- 172.21.15.47:0/1077231952 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa23805c8f0 0x7fa2380af960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:22.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.279+0000 7fa237fff700 1 -- 172.21.15.47:0/1077231952 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2380b2c40 msgr2=0x7fa238152790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:22.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.279+0000 7fa237fff700 1 --2- 172.21.15.47:0/1077231952 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2380b2c40 0x7fa238152790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:22.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.279+0000 7fa237fff700 1 -- 172.21.15.47:0/1077231952 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa22c00b1e0 con 0x7fa23805c510 2022-06-29T15:44:22.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.279+0000 7fa23cada700 1 --2- 172.21.15.47:0/1077231952 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2380b2c40 0x7fa238152790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:44:22.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.279+0000 7fa237fff700 1 --2- 172.21.15.47:0/1077231952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa23805c510 0x7fa2380af180 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7fa22800c000 tx=0x7fa228006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:22.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.279+0000 7fa2357fa700 1 -- 172.21.15.47:0/1077231952 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa228010780 con 0x7fa23805c510 2022-06-29T15:44:22.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.279+0000 7fa23d2db700 1 -- 172.21.15.47:0/1077231952 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2380b55e0 con 0x7fa23805c510 2022-06-29T15:44:22.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.279+0000 7fa2357fa700 1 -- 172.21.15.47:0/1077231952 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2280108e0 con 0x7fa23805c510 2022-06-29T15:44:22.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.279+0000 7fa2357fa700 1 -- 172.21.15.47:0/1077231952 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa228010c40 con 0x7fa23805c510 2022-06-29T15:44:22.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.279+0000 7fa23d2db700 1 -- 172.21.15.47:0/1077231952 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2381471e0 con 0x7fa23805c510 2022-06-29T15:44:22.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.279+0000 7fa23d2db700 1 -- 172.21.15.47:0/1077231952 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2381524f0 con 0x7fa23805c510 2022-06-29T15:44:22.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.283+0000 7fa2357fa700 1 -- 172.21.15.47:0/1077231952 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa22801b400 con 0x7fa23805c510 2022-06-29T15:44:22.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.283+0000 7fa2357fa700 1 --2- 172.21.15.47:0/1077231952 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa218053b30 0x7fa218055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:22.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.283+0000 7fa2357fa700 1 -- 172.21.15.47:0/1077231952 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fa22806bdb0 con 0x7fa23805c510 2022-06-29T15:44:22.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.283+0000 7fa2377fe700 1 --2- 172.21.15.47:0/1077231952 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa218053b30 0x7fa218055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:22.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.283+0000 7fa2377fe700 1 --2- 172.21.15.47:0/1077231952 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa218053b30 0x7fa218055ff0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7fa22c01c000 tx=0x7fa22c015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:22.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.291+0000 7fa2357fa700 1 -- 172.21.15.47:0/1077231952 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa2381524f0 con 0x7fa23805c510 2022-06-29T15:44:22.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.459+0000 7fa23d2db700 1 -- 172.21.15.47:0/1077231952 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa2381524f0 con 0x7fa23805c510 2022-06-29T15:44:22.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.459+0000 7fa2357fa700 1 -- 172.21.15.47:0/1077231952 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+1559 (secure 0 0 0) 0x7fa2381524f0 con 0x7fa23805c510 2022-06-29T15:44:22.467 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:44:22.468 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":14,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4896,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1047560169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1047560169},{"type":"v1","addr":"172.21.15.47:6835","nonce":1047560169}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":10},{"gid":4902,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/804808837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":804808837},{"type":"v1","addr":"172.21.15.64:6833","nonce":804808837}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":10},{"gid":5166,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/4206450314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":4206450314},{"type":"v1","addr":"172.21.15.47:6837","nonce":4206450314}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14}],"filesystems":[]} 2022-06-29T15:44:22.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.463+0000 7fa23d2db700 1 -- 172.21.15.47:0/1077231952 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa218053b30 msgr2=0x7fa218055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:22.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.463+0000 7fa23d2db700 1 --2- 172.21.15.47:0/1077231952 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa218053b30 0x7fa218055ff0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7fa22c01c000 tx=0x7fa22c015000).stop 2022-06-29T15:44:22.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.463+0000 7fa23d2db700 1 -- 172.21.15.47:0/1077231952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa23805c510 msgr2=0x7fa2380af180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:22.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.463+0000 7fa23d2db700 1 --2- 172.21.15.47:0/1077231952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa23805c510 0x7fa2380af180 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7fa22800c000 tx=0x7fa228006df0).stop 2022-06-29T15:44:22.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.463+0000 7fa23d2db700 1 -- 172.21.15.47:0/1077231952 shutdown_connections 2022-06-29T15:44:22.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.463+0000 7fa23d2db700 1 --2- 172.21.15.47:0/1077231952 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2380b2c40 0x7fa238152790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:22.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.463+0000 7fa23d2db700 1 --2- 172.21.15.47:0/1077231952 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa23805c8f0 0x7fa2380af960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:22.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.463+0000 7fa23d2db700 1 --2- 172.21.15.47:0/1077231952 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa218053b30 0x7fa218055ff0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:22.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.463+0000 7fa23d2db700 1 --2- 172.21.15.47:0/1077231952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa23805c510 0x7fa2380af180 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:22.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.463+0000 7fa23d2db700 1 -- 172.21.15.47:0/1077231952 >> 172.21.15.47:0/1077231952 conn(0x7fa238060e00 msgr2=0x7fa238142530 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:22.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.463+0000 7fa23d2db700 1 -- 172.21.15.47:0/1077231952 shutdown_connections 2022-06-29T15:44:22.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.463+0000 7fa23d2db700 1 -- 172.21.15.47:0/1077231952 wait complete. 2022-06-29T15:44:22.475 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 14 2022-06-29T15:44:22.480 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:44:22.585 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.579+0000 7f4f6f3b7700 1 Processor -- start 2022-06-29T15:44:22.585 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.579+0000 7f4f6f3b7700 1 -- start start 2022-06-29T15:44:22.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.579+0000 7f4f6f3b7700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4f680b2ee0 0x7f4f680b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:22.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.579+0000 7f4f6f3b7700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f681400e0 con 0x7f4f680ad970 2022-06-29T15:44:22.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.579+0000 7f4f6f3b7700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f68143570 con 0x7f4f680b0e50 2022-06-29T15:44:22.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.579+0000 7f4f6f3b7700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f68140ef0 con 0x7f4f680b2ee0 2022-06-29T15:44:22.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.579+0000 7f4f6dbb4700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4f680ad970 0x7f4f681407e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33166/0 (socket says 172.21.15.47:33166) 2022-06-29T15:44:22.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.579+0000 7f4f6dbb4700 1 -- 172.21.15.47:0/504955064 learned_addr learned my addr 172.21.15.47:0/504955064 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:44:22.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.579+0000 7f4f6e3b5700 1 --2- 172.21.15.47:0/504955064 >> v2:172.21.15.64:3300/0 conn(0x7f4f680b2ee0 0x7f4f680b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:22.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.579+0000 7f4f6d3b3700 1 -- 172.21.15.47:0/504955064 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3604201692 0 0) 0x7f4f681400e0 con 0x7f4f680ad970 2022-06-29T15:44:22.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.579+0000 7f4f6d3b3700 1 -- 172.21.15.47:0/504955064 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f54003260 con 0x7f4f680ad970 2022-06-29T15:44:22.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.579+0000 7f4f6d3b3700 1 -- 172.21.15.47:0/504955064 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1545788049 0 0) 0x7f4f54003260 con 0x7f4f680ad970 2022-06-29T15:44:22.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.579+0000 7f4f6d3b3700 1 -- 172.21.15.47:0/504955064 >> v2:172.21.15.64:3300/0 conn(0x7f4f680b2ee0 msgr2=0x7f4f680b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:44:22.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.579+0000 7f4f6d3b3700 1 --2- 172.21.15.47:0/504955064 >> v2:172.21.15.64:3300/0 conn(0x7f4f680b2ee0 0x7f4f680b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:22.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.579+0000 7f4f6d3b3700 1 -- 172.21.15.47:0/504955064 >> v1:172.21.15.64:6789/0 conn(0x7f4f680b0e50 legacy=0x7f4f6813d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:22.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.579+0000 7f4f6d3b3700 1 -- 172.21.15.47:0/504955064 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f68140340 con 0x7f4f680ad970 2022-06-29T15:44:22.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.579+0000 7f4f6d3b3700 1 -- 172.21.15.47:0/504955064 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7f4f64001a00 con 0x7f4f680ad970 2022-06-29T15:44:22.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.579+0000 7f4f6d3b3700 1 -- 172.21.15.47:0/504955064 >> v1:172.21.15.47:6789/0 conn(0x7f4f680ad970 legacy=0x7f4f681407e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:22.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.579+0000 7f4f6d3b3700 1 --2- 172.21.15.47:0/504955064 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4f54003750 0x7f4f54003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:22.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.579+0000 7f4f6d3b3700 1 --2- 172.21.15.47:0/504955064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f54004500 0x7f4f54004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:22.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.579+0000 7f4f6d3b3700 1 --2- 172.21.15.47:0/504955064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f54004e70 0x7f4f5400d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:22.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.579+0000 7f4f6d3b3700 1 -- 172.21.15.47:0/504955064 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f68140ef0 con 0x7f4f54004500 2022-06-29T15:44:22.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.579+0000 7f4f6d3b3700 1 -- 172.21.15.47:0/504955064 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5400df30 con 0x7f4f54004e70 2022-06-29T15:44:22.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.579+0000 7f4f6d3b3700 1 -- 172.21.15.47:0/504955064 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5400e0f0 con 0x7f4f54003750 2022-06-29T15:44:22.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.583+0000 7f4f6dbb4700 1 --2- 172.21.15.47:0/504955064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f54004500 0x7f4f54004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:22.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.583+0000 7f4f6e3b5700 1 --2- 172.21.15.47:0/504955064 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4f54003750 0x7f4f54003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:22.594 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.583+0000 7f4f6ebb6700 1 --2- 172.21.15.47:0/504955064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f54004e70 0x7f4f5400d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:22.594 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.583+0000 7f4f6dbb4700 1 -- 172.21.15.47:0/504955064 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4f54003750 msgr2=0x7f4f54003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:44:22.594 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.583+0000 7f4f6dbb4700 1 --2- 172.21.15.47:0/504955064 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4f54003750 0x7f4f54003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:22.594 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.583+0000 7f4f6dbb4700 1 -- 172.21.15.47:0/504955064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f54004e70 msgr2=0x7f4f5400d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:44:22.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.583+0000 7f4f6dbb4700 1 --2- 172.21.15.47:0/504955064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f54004e70 0x7f4f5400d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:22.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.583+0000 7f4f6dbb4700 1 -- 172.21.15.47:0/504955064 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f54003260 con 0x7f4f54004500 2022-06-29T15:44:22.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.583+0000 7f4f6ebb6700 1 --2- 172.21.15.47:0/504955064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f54004e70 0x7f4f5400d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:44:22.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.583+0000 7f4f6ebb6700 1 -- 172.21.15.47:0/504955064 reap_dead start 2022-06-29T15:44:22.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.583+0000 7f4f6dbb4700 1 --2- 172.21.15.47:0/504955064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f54004500 0x7f4f54004980 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7f4f64001710 tx=0x7f4f68143460).ready entity=mon.0 client_cookie=e9346284b7920925 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:22.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.583+0000 7f4f6d3b3700 1 -- 172.21.15.47:0/504955064 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f6400cc90 con 0x7f4f54004500 2022-06-29T15:44:22.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.583+0000 7f4f6d3b3700 1 -- 172.21.15.47:0/504955064 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f6400cdf0 con 0x7f4f54004500 2022-06-29T15:44:22.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.583+0000 7f4f6d3b3700 1 -- 172.21.15.47:0/504955064 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f64004b50 con 0x7f4f54004500 2022-06-29T15:44:22.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.583+0000 7f4f6f3b7700 1 -- 172.21.15.47:0/504955064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f54004500 msgr2=0x7f4f54004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:22.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.583+0000 7f4f6f3b7700 1 --2- 172.21.15.47:0/504955064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f54004500 0x7f4f54004980 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7f4f64001710 tx=0x7f4f68143460).stop 2022-06-29T15:44:22.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.583+0000 7f4f6f3b7700 1 -- 172.21.15.47:0/504955064 shutdown_connections 2022-06-29T15:44:22.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.583+0000 7f4f6f3b7700 1 --2- 172.21.15.47:0/504955064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f54004500 0x7f4f54004980 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:22.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.583+0000 7f4f6f3b7700 1 -- 172.21.15.47:0/504955064 >> 172.21.15.47:0/504955064 conn(0x7f4f68063670 msgr2=0x7f4f68063a80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:22.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.583+0000 7f4f6f3b7700 1 -- 172.21.15.47:0/504955064 shutdown_connections 2022-06-29T15:44:22.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.583+0000 7f4f6f3b7700 1 -- 172.21.15.47:0/504955064 wait complete. 2022-06-29T15:44:22.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.587+0000 7f4f6f3b7700 1 Processor -- start 2022-06-29T15:44:22.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.587+0000 7f4f6f3b7700 1 -- start start 2022-06-29T15:44:22.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.587+0000 7f4f6f3b7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f54004500 0x7f4f68065070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:22.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.587+0000 7f4f6f3b7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f68066aa0 0x7f4f68066f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:22.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.587+0000 7f4f6f3b7700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4f68138670 0x7f4f68138af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:22.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.587+0000 7f4f6f3b7700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f68065c80 con 0x7f4f54004500 2022-06-29T15:44:22.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.587+0000 7f4f6f3b7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f68142e50 con 0x7f4f68066aa0 2022-06-29T15:44:22.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.587+0000 7f4f6f3b7700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f68142030 con 0x7f4f68138670 2022-06-29T15:44:22.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.587+0000 7f4f6e3b5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f54004500 0x7f4f68065070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:22.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.587+0000 7f4f6e3b5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f54004500 0x7f4f68065070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47624/0 (socket says 172.21.15.47:47624) 2022-06-29T15:44:22.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.587+0000 7f4f6e3b5700 1 -- 172.21.15.47:0/1156412154 learned_addr learned my addr 172.21.15.47:0/1156412154 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:22.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.587+0000 7f4f6ebb6700 1 --2- 172.21.15.47:0/1156412154 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4f68138670 0x7f4f68138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:22.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.587+0000 7f4f6dbb4700 1 --2- 172.21.15.47:0/1156412154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f68066aa0 0x7f4f68066f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:22.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.587+0000 7f4f6e3b5700 1 -- 172.21.15.47:0/1156412154 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4f68138670 msgr2=0x7f4f68138af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:22.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.587+0000 7f4f6e3b5700 1 --2- 172.21.15.47:0/1156412154 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4f68138670 0x7f4f68138af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:22.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.587+0000 7f4f6e3b5700 1 -- 172.21.15.47:0/1156412154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f68066aa0 msgr2=0x7f4f68066f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:22.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.587+0000 7f4f6e3b5700 1 --2- 172.21.15.47:0/1156412154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f68066aa0 0x7f4f68066f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:22.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.587+0000 7f4f6e3b5700 1 -- 172.21.15.47:0/1156412154 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f68142bb0 con 0x7f4f54004500 2022-06-29T15:44:22.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.587+0000 7f4f6e3b5700 1 --2- 172.21.15.47:0/1156412154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f54004500 0x7f4f68065070 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7f4f5800b000 tx=0x7f4f58006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:22.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.587+0000 7f4f5f7fe700 1 -- 172.21.15.47:0/1156412154 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f5800f940 con 0x7f4f54004500 2022-06-29T15:44:22.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.587+0000 7f4f5f7fe700 1 -- 172.21.15.47:0/1156412154 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f5800faa0 con 0x7f4f54004500 2022-06-29T15:44:22.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.587+0000 7f4f6f3b7700 1 -- 172.21.15.47:0/1156412154 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f6813a450 con 0x7f4f54004500 2022-06-29T15:44:22.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.587+0000 7f4f5f7fe700 1 -- 172.21.15.47:0/1156412154 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f5800fe00 con 0x7f4f54004500 2022-06-29T15:44:22.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.587+0000 7f4f6f3b7700 1 -- 172.21.15.47:0/1156412154 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f68143320 con 0x7f4f54004500 2022-06-29T15:44:22.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.587+0000 7f4f6f3b7700 1 -- 172.21.15.47:0/1156412154 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f681421a0 con 0x7f4f54004500 2022-06-29T15:44:22.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.591+0000 7f4f5f7fe700 1 -- 172.21.15.47:0/1156412154 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4f58014440 con 0x7f4f54004500 2022-06-29T15:44:22.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.591+0000 7f4f5f7fe700 1 --2- 172.21.15.47:0/1156412154 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4f48053aa0 0x7f4f48055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:22.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.591+0000 7f4f5f7fe700 1 -- 172.21.15.47:0/1156412154 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f4f5806c8b0 con 0x7f4f54004500 2022-06-29T15:44:22.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.591+0000 7f4f6dbb4700 1 --2- 172.21.15.47:0/1156412154 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4f48053aa0 0x7f4f48055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:22.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.591+0000 7f4f6dbb4700 1 --2- 172.21.15.47:0/1156412154 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4f48053aa0 0x7f4f48055f60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f4f64018000 tx=0x7f4f64013000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:22.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.599+0000 7f4f5f7fe700 1 -- 172.21.15.47:0/1156412154 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4f681421a0 con 0x7f4f54004500 2022-06-29T15:44:22.769 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.763+0000 7f4f6f3b7700 1 -- 172.21.15.47:0/1156412154 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4f681421a0 con 0x7f4f54004500 2022-06-29T15:44:22.770 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.763+0000 7f4f5f7fe700 1 -- 172.21.15.47:0/1156412154 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+1559 (secure 0 0 0) 0x7f4f681421a0 con 0x7f4f54004500 2022-06-29T15:44:22.771 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:44:22.771 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":14,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4896,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1047560169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1047560169},{"type":"v1","addr":"172.21.15.47:6835","nonce":1047560169}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":10},{"gid":4902,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/804808837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":804808837},{"type":"v1","addr":"172.21.15.64:6833","nonce":804808837}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":10},{"gid":5166,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/4206450314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":4206450314},{"type":"v1","addr":"172.21.15.47:6837","nonce":4206450314}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14}],"filesystems":[]} 2022-06-29T15:44:22.772 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.767+0000 7f4f6f3b7700 1 -- 172.21.15.47:0/1156412154 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4f48053aa0 msgr2=0x7f4f48055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:22.772 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.767+0000 7f4f6f3b7700 1 --2- 172.21.15.47:0/1156412154 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4f48053aa0 0x7f4f48055f60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f4f64018000 tx=0x7f4f64013000).stop 2022-06-29T15:44:22.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.767+0000 7f4f6f3b7700 1 -- 172.21.15.47:0/1156412154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f54004500 msgr2=0x7f4f68065070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:22.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.767+0000 7f4f6f3b7700 1 --2- 172.21.15.47:0/1156412154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f54004500 0x7f4f68065070 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7f4f5800b000 tx=0x7f4f58006df0).stop 2022-06-29T15:44:22.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.767+0000 7f4f6f3b7700 1 -- 172.21.15.47:0/1156412154 shutdown_connections 2022-06-29T15:44:22.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.767+0000 7f4f6f3b7700 1 --2- 172.21.15.47:0/1156412154 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4f68138670 0x7f4f68138af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:22.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.767+0000 7f4f6f3b7700 1 --2- 172.21.15.47:0/1156412154 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4f48053aa0 0x7f4f48055f60 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:22.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.767+0000 7f4f6f3b7700 1 --2- 172.21.15.47:0/1156412154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f68066aa0 0x7f4f68066f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:22.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.767+0000 7f4f6f3b7700 1 --2- 172.21.15.47:0/1156412154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f54004500 0x7f4f68065070 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:22.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.767+0000 7f4f6f3b7700 1 -- 172.21.15.47:0/1156412154 >> 172.21.15.47:0/1156412154 conn(0x7f4f68063670 msgr2=0x7f4f6805f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:22.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.767+0000 7f4f6f3b7700 1 -- 172.21.15.47:0/1156412154 shutdown_connections 2022-06-29T15:44:22.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.767+0000 7f4f6f3b7700 1 -- 172.21.15.47:0/1156412154 wait complete. 2022-06-29T15:44:22.775 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 14 2022-06-29T15:44:22.784 INFO:tasks.ceph.mds.a:Restarting daemon 2022-06-29T15:44:22.784 INFO:tasks.ceph.mds.a:Stopping old one... 2022-06-29T15:44:22.785 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-06-29T15:44:22.785 INFO:teuthology.orchestra.run:waiting for 300 2022-06-29T15:44:22.785 INFO:tasks.ceph.mds.c:Restarting daemon 2022-06-29T15:44:22.786 INFO:tasks.ceph.mds.c:Stopping old one... 2022-06-29T15:44:22.786 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2022-06-29T15:44:22.786 INFO:teuthology.orchestra.run:waiting for 300 2022-06-29T15:44:22.787 INFO:tasks.ceph.mds.b:Restarting daemon 2022-06-29T15:44:22.787 INFO:tasks.ceph.mds.b:Stopping old one... 2022-06-29T15:44:22.787 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-06-29T15:44:22.788 INFO:teuthology.orchestra.run:waiting for 300 2022-06-29T15:44:22.798 INFO:tasks.ceph.mds.a:Stopped 2022-06-29T15:44:22.798 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-06-29T15:44:22.801 INFO:tasks.ceph.mds.c:Stopped 2022-06-29T15:44:22.801 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2022-06-29T15:44:22.803 INFO:tasks.ceph.mds.b:Stopped 2022-06-29T15:44:22.803 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-06-29T15:44:22.806 INFO:tasks.ceph.mds.b:Started 2022-06-29T15:44:22.806 DEBUG:teuthology.parallel:result is None 2022-06-29T15:44:22.844 INFO:tasks.ceph.mds.a:Started 2022-06-29T15:44:22.845 DEBUG:teuthology.parallel:result is None 2022-06-29T15:44:22.845 INFO:tasks.ceph.mds.c:Started 2022-06-29T15:44:22.846 DEBUG:teuthology.parallel:result is None 2022-06-29T15:44:22.846 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-06-29T15:44:22.882 INFO:tasks.ceph.mds.b.smithi064.stdout:starting mds.b at 2022-06-29T15:44:22.914 INFO:tasks.ceph.mds.a.smithi047.stdout:starting mds.a at 2022-06-29T15:44:22.915 INFO:tasks.ceph.mds.c.smithi047.stdout:starting mds.c at 2022-06-29T15:44:22.954 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.947+0000 7ff0ecdb3700 1 Processor -- start 2022-06-29T15:44:22.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.947+0000 7ff0ecdb3700 1 -- start start 2022-06-29T15:44:22.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.951+0000 7ff0ecdb3700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff0e80b7840 0x7ff0e80b44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:22.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.951+0000 7ff0ecdb3700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0e80b7c20 con 0x7ff0e80b49d0 2022-06-29T15:44:22.958 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.951+0000 7ff0ecdb3700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0e8066840 con 0x7ff0e80b3240 2022-06-29T15:44:22.958 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.951+0000 7ff0ecdb3700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0e80b2ff0 con 0x7ff0e80b7840 2022-06-29T15:44:22.958 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.951+0000 7ff0e7fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff0e80b49d0 0x7ff0e8146fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33214/0 (socket says 172.21.15.47:33214) 2022-06-29T15:44:22.958 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.951+0000 7ff0e7fff700 1 -- 172.21.15.47:0/397671948 learned_addr learned my addr 172.21.15.47:0/397671948 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:44:22.959 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.951+0000 7ff0e77fe700 1 --2- 172.21.15.47:0/397671948 >> v2:172.21.15.64:3300/0 conn(0x7ff0e80b7840 0x7ff0e80b44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:22.959 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.951+0000 7ff0e67fc700 1 -- 172.21.15.47:0/397671948 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4289455749 0 0) 0x7ff0e80b7c20 con 0x7ff0e80b49d0 2022-06-29T15:44:22.959 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.951+0000 7ff0e67fc700 1 -- 172.21.15.47:0/397671948 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0d4003260 con 0x7ff0e80b49d0 2022-06-29T15:44:22.960 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.951+0000 7ff0e77fe700 1 -- 172.21.15.47:0/397671948 >> v1:172.21.15.64:6789/0 conn(0x7ff0e80b3240 legacy=0x7ff0e8146890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:22.960 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.951+0000 7ff0e77fe700 1 -- 172.21.15.47:0/397671948 >> v1:172.21.15.47:6789/0 conn(0x7ff0e80b49d0 legacy=0x7ff0e8146fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:22.960 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.951+0000 7ff0e77fe700 1 -- 172.21.15.47:0/397671948 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e80b4e60 con 0x7ff0e80b7840 2022-06-29T15:44:22.960 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.951+0000 7ff0e77fe700 1 --2- 172.21.15.47:0/397671948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0e80b7840 0x7ff0e80b44e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7ff0e00032f0 tx=0x7ff0e00091d0).ready entity=mon.1 client_cookie=5873da39da2a2418 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:22.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.951+0000 7ff0e67fc700 1 -- 172.21.15.47:0/397671948 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0e000b6c0 con 0x7ff0e80b7840 2022-06-29T15:44:22.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.951+0000 7ff0e67fc700 1 -- 172.21.15.47:0/397671948 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0e000b820 con 0x7ff0e80b7840 2022-06-29T15:44:22.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.951+0000 7ff0e67fc700 1 -- 172.21.15.47:0/397671948 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0e000b990 con 0x7ff0e80b7840 2022-06-29T15:44:22.965 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.951+0000 7ff0ecdb3700 1 -- 172.21.15.47:0/397671948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0e80b7840 msgr2=0x7ff0e80b44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:22.965 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.951+0000 7ff0ecdb3700 1 --2- 172.21.15.47:0/397671948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0e80b7840 0x7ff0e80b44e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7ff0e00032f0 tx=0x7ff0e00091d0).stop 2022-06-29T15:44:22.965 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.951+0000 7ff0ecdb3700 1 -- 172.21.15.47:0/397671948 shutdown_connections 2022-06-29T15:44:22.965 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.951+0000 7ff0ecdb3700 1 --2- 172.21.15.47:0/397671948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0e80b7840 0x7ff0e80b44e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:22.966 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.951+0000 7ff0ecdb3700 1 -- 172.21.15.47:0/397671948 >> 172.21.15.47:0/397671948 conn(0x7ff0e805f480 msgr2=0x7ff0e80647d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:22.966 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.951+0000 7ff0ecdb3700 1 -- 172.21.15.47:0/397671948 shutdown_connections 2022-06-29T15:44:22.966 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.951+0000 7ff0ecdb3700 1 -- 172.21.15.47:0/397671948 wait complete. 2022-06-29T15:44:22.966 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.955+0000 7ff0ecdb3700 1 Processor -- start 2022-06-29T15:44:22.967 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.955+0000 7ff0ecdb3700 1 -- start start 2022-06-29T15:44:22.967 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.955+0000 7ff0ecdb3700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff0e80b3240 0x7ff0e80ae2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:22.967 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.955+0000 7ff0ecdb3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0e80b49d0 0x7ff0e80aa180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:22.967 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.955+0000 7ff0ecdb3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0e80b7840 0x7ff0e80aa670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:22.968 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.955+0000 7ff0ecdb3700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e80626a0 con 0x7ff0e80b49d0 2022-06-29T15:44:22.968 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.955+0000 7ff0ecdb3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e81532d0 con 0x7ff0e80b7840 2022-06-29T15:44:22.968 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.955+0000 7ff0ecdb3700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e80628d0 con 0x7ff0e80b3240 2022-06-29T15:44:22.968 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.955+0000 7ff0e6ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0e80b49d0 0x7ff0e80aa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:22.969 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.955+0000 7ff0e6ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0e80b49d0 0x7ff0e80aa180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47664/0 (socket says 172.21.15.47:47664) 2022-06-29T15:44:22.969 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.955+0000 7ff0e6ffd700 1 -- 172.21.15.47:0/2009341167 learned_addr learned my addr 172.21.15.47:0/2009341167 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:22.969 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.955+0000 7ff0e77fe700 1 --2- 172.21.15.47:0/2009341167 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff0e80b3240 0x7ff0e80ae2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:22.969 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.955+0000 7ff0e7fff700 1 --2- 172.21.15.47:0/2009341167 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0e80b7840 0x7ff0e80aa670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:22.969 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.955+0000 7ff0e6ffd700 1 -- 172.21.15.47:0/2009341167 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff0e80b3240 msgr2=0x7ff0e80ae2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:22.970 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.955+0000 7ff0e6ffd700 1 --2- 172.21.15.47:0/2009341167 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff0e80b3240 0x7ff0e80ae2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:22.970 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.955+0000 7ff0e6ffd700 1 -- 172.21.15.47:0/2009341167 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0e80b7840 msgr2=0x7ff0e80aa670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:22.970 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.955+0000 7ff0e6ffd700 1 --2- 172.21.15.47:0/2009341167 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0e80b7840 0x7ff0e80aa670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:22.970 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.955+0000 7ff0e6ffd700 1 -- 172.21.15.47:0/2009341167 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e0009470 con 0x7ff0e80b49d0 2022-06-29T15:44:22.971 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.955+0000 7ff0e7fff700 1 --2- 172.21.15.47:0/2009341167 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0e80b7840 0x7ff0e80aa670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:44:22.971 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.959+0000 7ff0e6ffd700 1 --2- 172.21.15.47:0/2009341167 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0e80b49d0 0x7ff0e80aa180 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7ff0e80aee10 tx=0x7ff0d8007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:22.971 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.959+0000 7ff0e4ff9700 1 -- 172.21.15.47:0/2009341167 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0d8007030 con 0x7ff0e80b49d0 2022-06-29T15:44:22.971 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.959+0000 7ff0e4ff9700 1 -- 172.21.15.47:0/2009341167 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0d8007190 con 0x7ff0e80b49d0 2022-06-29T15:44:22.972 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.959+0000 7ff0ecdb3700 1 -- 172.21.15.47:0/2009341167 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0e8062b70 con 0x7ff0e80b49d0 2022-06-29T15:44:22.972 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.959+0000 7ff0e4ff9700 1 -- 172.21.15.47:0/2009341167 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0d8006510 con 0x7ff0e80b49d0 2022-06-29T15:44:22.972 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.959+0000 7ff0ecdb3700 1 -- 172.21.15.47:0/2009341167 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0e80aab60 con 0x7ff0e80b49d0 2022-06-29T15:44:22.972 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.959+0000 7ff0ecdb3700 1 -- 172.21.15.47:0/2009341167 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0e8062d50 con 0x7ff0e80b49d0 2022-06-29T15:44:22.974 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.959+0000 7ff0e4ff9700 1 -- 172.21.15.47:0/2009341167 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff0d8006670 con 0x7ff0e80b49d0 2022-06-29T15:44:22.974 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.959+0000 7ff0e4ff9700 1 --2- 172.21.15.47:0/2009341167 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff0c8053b00 0x7ff0c8055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:22.974 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.959+0000 7ff0e4ff9700 1 -- 172.21.15.47:0/2009341167 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5686+0+0 (secure 0 0 0) 0x7ff0d806b670 con 0x7ff0e80b49d0 2022-06-29T15:44:22.975 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.959+0000 7ff0e77fe700 1 --2- 172.21.15.47:0/2009341167 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff0c8053b00 0x7ff0c8055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:22.975 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.959+0000 7ff0e77fe700 1 --2- 172.21.15.47:0/2009341167 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff0c8053b00 0x7ff0c8055fc0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7ff0e001b000 tx=0x7ff0e0016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:22.975 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:22.967+0000 7ff0e4ff9700 1 -- 172.21.15.47:0/2009341167 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff0e8062d50 con 0x7ff0e80b49d0 2022-06-29T15:44:23.113 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:23.107+0000 7ff0ecdb3700 1 -- 172.21.15.47:0/2009341167 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7ff0e8062d50 con 0x7ff0e80b49d0 2022-06-29T15:44:23.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:23.947+0000 7ff0e4ff9700 1 -- 172.21.15.47:0/2009341167 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v43) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff0e8062d50 con 0x7ff0e80b49d0 2022-06-29T15:44:23.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:23.951+0000 7ff0ecdb3700 1 -- 172.21.15.47:0/2009341167 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff0c8053b00 msgr2=0x7ff0c8055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:23.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:23.951+0000 7ff0ecdb3700 1 --2- 172.21.15.47:0/2009341167 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff0c8053b00 0x7ff0c8055fc0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7ff0e001b000 tx=0x7ff0e0016000).stop 2022-06-29T15:44:23.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:23.951+0000 7ff0ecdb3700 1 -- 172.21.15.47:0/2009341167 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0e80b49d0 msgr2=0x7ff0e80aa180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:23.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:23.951+0000 7ff0ecdb3700 1 --2- 172.21.15.47:0/2009341167 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0e80b49d0 0x7ff0e80aa180 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7ff0e80aee10 tx=0x7ff0d8007c00).stop 2022-06-29T15:44:23.958 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:23.951+0000 7ff0ecdb3700 1 -- 172.21.15.47:0/2009341167 shutdown_connections 2022-06-29T15:44:23.958 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:23.951+0000 7ff0ecdb3700 1 --2- 172.21.15.47:0/2009341167 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff0c8053b00 0x7ff0c8055fc0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:23.959 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:23.951+0000 7ff0ecdb3700 1 --2- 172.21.15.47:0/2009341167 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0e80b7840 0x7ff0e80aa670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:23.959 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:23.951+0000 7ff0ecdb3700 1 --2- 172.21.15.47:0/2009341167 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0e80b49d0 0x7ff0e80aa180 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:23.960 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:23.951+0000 7ff0ecdb3700 1 --2- 172.21.15.47:0/2009341167 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff0e80b3240 0x7ff0e80ae2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:23.960 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:23.951+0000 7ff0ecdb3700 1 -- 172.21.15.47:0/2009341167 >> 172.21.15.47:0/2009341167 conn(0x7ff0e805f480 msgr2=0x7ff0e80b6d90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:23.960 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:23.951+0000 7ff0ecdb3700 1 -- 172.21.15.47:0/2009341167 shutdown_connections 2022-06-29T15:44:23.960 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:23.951+0000 7ff0ecdb3700 1 -- 172.21.15.47:0/2009341167 wait complete. 2022-06-29T15:44:23.961 INFO:teuthology.orchestra.run.smithi047.stderr: removed all blacklist entries 2022-06-29T15:44:23.972 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2022-06-29T15:44:23.972 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-06-29T15:44:24.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.079+0000 7f01d534b700 1 Processor -- start 2022-06-29T15:44:24.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.079+0000 7f01d534b700 1 -- start start 2022-06-29T15:44:24.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.079+0000 7f01d534b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f01d00b1ab0 0x7f01d0060b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:24.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.079+0000 7f01d534b700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01d005fe10 con 0x7f01d005fa30 2022-06-29T15:44:24.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.079+0000 7f01d534b700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01d00600e0 con 0x7f01d00b2770 2022-06-29T15:44:24.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.079+0000 7f01d534b700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01d00b1ef0 con 0x7f01d00b1ab0 2022-06-29T15:44:24.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.079+0000 7f01cffff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f01d005fa30 0x7f01d013f470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33228/0 (socket says 172.21.15.47:33228) 2022-06-29T15:44:24.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.079+0000 7f01cffff700 1 -- 172.21.15.47:0/4279210047 learned_addr learned my addr 172.21.15.47:0/4279210047 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:44:24.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.079+0000 7f01cf7fe700 1 --2- 172.21.15.47:0/4279210047 >> v2:172.21.15.64:3300/0 conn(0x7f01d00b1ab0 0x7f01d0060b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:24.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.079+0000 7f01ceffd700 1 -- 172.21.15.47:0/4279210047 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 356739582 0 0) 0x7f01d005fe10 con 0x7f01d005fa30 2022-06-29T15:44:24.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.079+0000 7f01ceffd700 1 -- 172.21.15.47:0/4279210047 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01bc003260 con 0x7f01d005fa30 2022-06-29T15:44:24.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.079+0000 7f01ceffd700 1 -- 172.21.15.47:0/4279210047 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1887130421 0 0) 0x7f01d00600e0 con 0x7f01d00b2770 2022-06-29T15:44:24.089 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.079+0000 7f01ceffd700 1 -- 172.21.15.47:0/4279210047 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01d005fe10 con 0x7f01d00b2770 2022-06-29T15:44:24.089 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.079+0000 7f01ceffd700 1 -- 172.21.15.47:0/4279210047 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 441031386 0 0) 0x7f01bc003260 con 0x7f01d005fa30 2022-06-29T15:44:24.089 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.079+0000 7f01cf7fe700 1 -- 172.21.15.47:0/4279210047 >> v1:172.21.15.64:6789/0 conn(0x7f01d00b2770 legacy=0x7f01d00ae170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:24.089 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.079+0000 7f01cf7fe700 1 -- 172.21.15.47:0/4279210047 >> v1:172.21.15.47:6789/0 conn(0x7f01d005fa30 legacy=0x7f01d013f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:24.089 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.079+0000 7f01cf7fe700 1 -- 172.21.15.47:0/4279210047 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d0060380 con 0x7f01d00b1ab0 2022-06-29T15:44:24.094 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.083+0000 7f01cf7fe700 1 --2- 172.21.15.47:0/4279210047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01d00b1ab0 0x7f01d0060b90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f01c0001110 tx=0x7f01c4006df0).ready entity=mon.1 client_cookie=b586b03b65202b17 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:24.094 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.083+0000 7f01ceffd700 1 -- 172.21.15.47:0/4279210047 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01c0001660 con 0x7f01d00b1ab0 2022-06-29T15:44:24.094 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.083+0000 7f01ceffd700 1 -- 172.21.15.47:0/4279210047 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01c4000e90 con 0x7f01d00b1ab0 2022-06-29T15:44:24.094 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.083+0000 7f01ceffd700 1 -- 172.21.15.47:0/4279210047 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01c4012460 con 0x7f01d00b1ab0 2022-06-29T15:44:24.095 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.083+0000 7f01d534b700 1 -- 172.21.15.47:0/4279210047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01d00b1ab0 msgr2=0x7f01d0060b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:24.095 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.083+0000 7f01d534b700 1 --2- 172.21.15.47:0/4279210047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01d00b1ab0 0x7f01d0060b90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f01c0001110 tx=0x7f01c4006df0).stop 2022-06-29T15:44:24.095 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.083+0000 7f01d534b700 1 -- 172.21.15.47:0/4279210047 shutdown_connections 2022-06-29T15:44:24.095 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.083+0000 7f01d534b700 1 --2- 172.21.15.47:0/4279210047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01d00b1ab0 0x7f01d0060b90 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:24.096 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.083+0000 7f01d534b700 1 -- 172.21.15.47:0/4279210047 >> 172.21.15.47:0/4279210047 conn(0x7f01d0062df0 msgr2=0x7f01d00631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:24.096 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.083+0000 7f01d534b700 1 -- 172.21.15.47:0/4279210047 shutdown_connections 2022-06-29T15:44:24.096 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.083+0000 7f01d534b700 1 -- 172.21.15.47:0/4279210047 wait complete. 2022-06-29T15:44:24.096 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.083+0000 7f01d534b700 1 Processor -- start 2022-06-29T15:44:24.096 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.083+0000 7f01d534b700 1 -- start start 2022-06-29T15:44:24.097 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.083+0000 7f01d534b700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f01d005fa30 0x7f01d013f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:24.097 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.083+0000 7f01d534b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01d00b1ab0 0x7f01d013f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:24.097 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.083+0000 7f01d534b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01d00b2770 0x7f01d0064a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:24.097 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.083+0000 7f01d534b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d0141590 con 0x7f01d00b1ab0 2022-06-29T15:44:24.098 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.083+0000 7f01d534b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d0152fd0 con 0x7f01d00b2770 2022-06-29T15:44:24.098 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.083+0000 7f01d534b700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01d0153270 con 0x7f01d005fa30 2022-06-29T15:44:24.098 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.083+0000 7f01cf7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01d00b1ab0 0x7f01d013f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:24.098 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.083+0000 7f01cf7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01d00b1ab0 0x7f01d013f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47682/0 (socket says 172.21.15.47:47682) 2022-06-29T15:44:24.099 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.083+0000 7f01cf7fe700 1 -- 172.21.15.47:0/459503799 learned_addr learned my addr 172.21.15.47:0/459503799 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:24.099 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.083+0000 7f01cffff700 1 --2- 172.21.15.47:0/459503799 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f01d005fa30 0x7f01d013f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:24.099 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.083+0000 7f01d4b4a700 1 --2- 172.21.15.47:0/459503799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01d00b2770 0x7f01d0064a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:24.099 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.083+0000 7f01cf7fe700 1 -- 172.21.15.47:0/459503799 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f01d005fa30 msgr2=0x7f01d013f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:24.100 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.083+0000 7f01cf7fe700 1 --2- 172.21.15.47:0/459503799 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f01d005fa30 0x7f01d013f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:24.100 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.083+0000 7f01cf7fe700 1 -- 172.21.15.47:0/459503799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01d00b2770 msgr2=0x7f01d0064a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:24.100 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.083+0000 7f01cf7fe700 1 --2- 172.21.15.47:0/459503799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01d00b2770 0x7f01d0064a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:24.100 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.083+0000 7f01cf7fe700 1 -- 172.21.15.47:0/459503799 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c400b180 con 0x7f01d00b1ab0 2022-06-29T15:44:24.100 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.083+0000 7f01cffff700 1 --2- 172.21.15.47:0/459503799 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f01d005fa30 0x7f01d013f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:44:24.101 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.087+0000 7f01cf7fe700 1 --2- 172.21.15.47:0/459503799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01d00b1ab0 0x7f01d013f890 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7f01c0001110 tx=0x7f01c4014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:24.101 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.087+0000 7f01cd7fa700 1 -- 172.21.15.47:0/459503799 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01c400da20 con 0x7f01d00b1ab0 2022-06-29T15:44:24.101 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.087+0000 7f01cd7fa700 1 -- 172.21.15.47:0/459503799 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01c400db80 con 0x7f01d00b1ab0 2022-06-29T15:44:24.101 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.087+0000 7f01cd7fa700 1 -- 172.21.15.47:0/459503799 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01c401e400 con 0x7f01d00b1ab0 2022-06-29T15:44:24.102 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.087+0000 7f01d534b700 1 -- 172.21.15.47:0/459503799 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01d0097060 con 0x7f01d00b1ab0 2022-06-29T15:44:24.102 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.087+0000 7f01d534b700 1 -- 172.21.15.47:0/459503799 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01d01410f0 con 0x7f01d00b1ab0 2022-06-29T15:44:24.102 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.087+0000 7f01d534b700 1 -- 172.21.15.47:0/459503799 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01d0152e10 con 0x7f01d00b1ab0 2022-06-29T15:44:24.102 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.087+0000 7f01cd7fa700 1 -- 172.21.15.47:0/459503799 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f01c401c070 con 0x7f01d00b1ab0 2022-06-29T15:44:24.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.087+0000 7f01cd7fa700 1 --2- 172.21.15.47:0/459503799 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f01b0053b30 0x7f01b0055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:24.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.087+0000 7f01cd7fa700 1 -- 172.21.15.47:0/459503799 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f01c406c810 con 0x7f01d00b1ab0 2022-06-29T15:44:24.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.087+0000 7f01cffff700 1 --2- 172.21.15.47:0/459503799 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f01b0053b30 0x7f01b0055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:24.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.087+0000 7f01cffff700 1 --2- 172.21.15.47:0/459503799 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f01b0053b30 0x7f01b0055ff0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f01c000b000 tx=0x7f01c0000bc0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:24.104 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.091+0000 7f01cd7fa700 1 -- 172.21.15.47:0/459503799 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f01c4043800 con 0x7f01d00b1ab0 2022-06-29T15:44:24.269 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.263+0000 7f01d534b700 1 -- 172.21.15.47:0/459503799 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f01d00573d0 con 0x7f01d00b1ab0 2022-06-29T15:44:24.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.263+0000 7f01cd7fa700 1 -- 172.21.15.47:0/459503799 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+4255 (secure 0 0 0) 0x7f01c4023020 con 0x7f01d00b1ab0 2022-06-29T15:44:24.270 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:44:24.271 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2022-06-29T15:44:24.271 INFO:teuthology.orchestra.run.smithi047.stdout: "auth_dump": [ 2022-06-29T15:44:24.271 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:24.272 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.a", 2022-06-29T15:44:24.272 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxiQJFFMxAA9Tv7FJD9u/BKvbkuKld6Kg==", 2022-06-29T15:44:24.272 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:44:24.272 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:44:24.273 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:44:24.273 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:44:24.273 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:44:24.274 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:24.274 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:24.274 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:24.275 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.b", 2022-06-29T15:44:24.275 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC2crxibt6FARAAEf2bwXKFhXKtqr1nN0rdXQ==", 2022-06-29T15:44:24.276 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:44:24.276 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:44:24.277 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:44:24.277 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:44:24.278 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:44:24.278 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:24.279 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:24.279 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:24.280 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.c", 2022-06-29T15:44:24.280 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxi/4ZMORAANpLbgTm3ehqi80USc3swGg==", 2022-06-29T15:44:24.280 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:44:24.281 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:44:24.281 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:44:24.281 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:44:24.282 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:44:24.282 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:24.283 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:24.283 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:24.283 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.0", 2022-06-29T15:44:24.284 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC2crxiY0YsMxAAZivb+CxL/w7vZwQv37Y4oQ==", 2022-06-29T15:44:24.284 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:44:24.285 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:44:24.285 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:44:24.285 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:44:24.286 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:24.286 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:24.286 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:24.286 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.1", 2022-06-29T15:44:24.287 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC5crxitLkKFxAA8+++LUUKLiO9SAuAfYgT5g==", 2022-06-29T15:44:24.287 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:44:24.287 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:44:24.287 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:44:24.288 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:44:24.288 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:24.288 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:24.288 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:24.289 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.2", 2022-06-29T15:44:24.289 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC7crxi1oM9NxAA2FfLdNt32YPwlyNdGQpaQA==", 2022-06-29T15:44:24.289 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:44:24.289 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:44:24.289 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:44:24.290 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:44:24.290 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:24.290 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:24.290 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:24.291 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.3", 2022-06-29T15:44:24.291 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC+crxiTsBgGhAA+gLBBDaskPbmRNWzgKwcHA==", 2022-06-29T15:44:24.291 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:44:24.291 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:44:24.292 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:44:24.292 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:44:24.292 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:24.292 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:24.292 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:24.293 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.4", 2022-06-29T15:44:24.293 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDDcrxivJebIhAAfwWJa7tK4hamkk1/1CSSfw==", 2022-06-29T15:44:24.293 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:44:24.293 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:44:24.294 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:44:24.294 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:44:24.294 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:24.294 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:24.295 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:24.295 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.5", 2022-06-29T15:44:24.295 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDGcrxijuozBhAAt+Dx3wH3WKp3DrbCbhmh8Q==", 2022-06-29T15:44:24.295 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:44:24.295 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:44:24.296 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:44:24.296 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:44:24.296 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:24.296 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:24.297 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:24.297 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.6", 2022-06-29T15:44:24.297 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDIcrxi91kXJxAA9JX7n98P394bD41Q9LUdvQ==", 2022-06-29T15:44:24.297 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:44:24.298 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:44:24.298 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:44:24.298 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:44:24.298 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:24.299 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:24.299 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:24.299 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.7", 2022-06-29T15:44:24.299 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDLcrxiHj0ZDRAASy6fAix8TmEZbAGtuFCfFg==", 2022-06-29T15:44:24.300 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:44:24.300 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:44:24.300 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:44:24.300 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:44:24.301 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:24.301 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:24.301 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:24.301 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.0", 2022-06-29T15:44:24.302 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC2crxiVDlfBxAAezkUfg4e2g0Rfakaa/Ek1Q==", 2022-06-29T15:44:24.302 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:44:24.302 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:44:24.302 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow r", 2022-06-29T15:44:24.303 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-29T15:44:24.303 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:24.303 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:24.304 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:24.304 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.1", 2022-06-29T15:44:24.304 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC2crxisXJ1CxAAhl07oF7AW66NdzfvvZTJmA==", 2022-06-29T15:44:24.304 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:44:24.305 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:44:24.305 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow r", 2022-06-29T15:44:24.305 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-29T15:44:24.305 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:24.306 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:24.306 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:24.306 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.admin", 2022-06-29T15:44:24.306 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxiW7HSFRAAhYqoGU9GFqj2HfUVGValSA==", 2022-06-29T15:44:24.307 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:44:24.307 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2022-06-29T15:44:24.307 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:44:24.307 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:44:24.308 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:44:24.308 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:24.308 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:24.309 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:24.309 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.x", 2022-06-29T15:44:24.309 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxidmuIMBAAcpNlYVu5+icCHg+yIjcKXw==", 2022-06-29T15:44:24.309 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:44:24.310 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2022-06-29T15:44:24.310 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2022-06-29T15:44:24.310 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:44:24.310 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:24.311 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:24.311 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:24.311 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.y", 2022-06-29T15:44:24.311 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxisSMFLBAAv03r4OoPjOP9nI9G/LK0iw==", 2022-06-29T15:44:24.312 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:44:24.312 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2022-06-29T15:44:24.312 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2022-06-29T15:44:24.312 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:44:24.312 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:24.313 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:24.313 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:44:24.313 INFO:teuthology.orchestra.run.smithi047.stdout:} 2022-06-29T15:44:24.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.267+0000 7f01d534b700 1 -- 172.21.15.47:0/459503799 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f01b0053b30 msgr2=0x7f01b0055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:24.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.267+0000 7f01d534b700 1 --2- 172.21.15.47:0/459503799 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f01b0053b30 0x7f01b0055ff0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f01c000b000 tx=0x7f01c0000bc0).stop 2022-06-29T15:44:24.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.267+0000 7f01d534b700 1 -- 172.21.15.47:0/459503799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01d00b1ab0 msgr2=0x7f01d013f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:24.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.267+0000 7f01d534b700 1 --2- 172.21.15.47:0/459503799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01d00b1ab0 0x7f01d013f890 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7f01c0001110 tx=0x7f01c4014000).stop 2022-06-29T15:44:24.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.267+0000 7f01d534b700 1 -- 172.21.15.47:0/459503799 shutdown_connections 2022-06-29T15:44:24.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.267+0000 7f01d534b700 1 --2- 172.21.15.47:0/459503799 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f01b0053b30 0x7f01b0055ff0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:24.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.267+0000 7f01d534b700 1 --2- 172.21.15.47:0/459503799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01d00b2770 0x7f01d0064a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:24.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.267+0000 7f01d534b700 1 --2- 172.21.15.47:0/459503799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01d00b1ab0 0x7f01d013f890 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:24.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.267+0000 7f01d534b700 1 --2- 172.21.15.47:0/459503799 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f01d005fa30 0x7f01d013f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:24.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.267+0000 7f01d534b700 1 -- 172.21.15.47:0/459503799 >> 172.21.15.47:0/459503799 conn(0x7f01d0062df0 msgr2=0x7f01d00ae060 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:24.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.267+0000 7f01d534b700 1 -- 172.21.15.47:0/459503799 shutdown_connections 2022-06-29T15:44:24.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.267+0000 7f01d534b700 1 -- 172.21.15.47:0/459503799 wait complete. 2022-06-29T15:44:24.319 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:44:24.433 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.427+0000 7f06ca516700 1 Processor -- start 2022-06-29T15:44:24.433 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.427+0000 7f06ca516700 1 -- start start 2022-06-29T15:44:24.437 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.427+0000 7f06ca516700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f06bc0d44b0 0x7f06bc03e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:24.437 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.427+0000 7f06ca516700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f06bc03ed00 0x7f06bc0ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:24.437 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.427+0000 7f06ca516700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06bc040c60 con 0x7f06bc03d700 2022-06-29T15:44:24.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.427+0000 7f06ca516700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06bc03f5d0 con 0x7f06bc03ed00 2022-06-29T15:44:24.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.427+0000 7f06ca516700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06bc040630 con 0x7f06bc0d44b0 2022-06-29T15:44:24.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.427+0000 7f06c8d13700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f06bc03d700 0x7f06bc0cdd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33244/0 (socket says 172.21.15.47:33244) 2022-06-29T15:44:24.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.427+0000 7f06c8d13700 1 -- 172.21.15.47:0/4168371212 learned_addr learned my addr 172.21.15.47:0/4168371212 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:44:24.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.427+0000 7f06c9d15700 1 --2- 172.21.15.47:0/4168371212 >> v2:172.21.15.47:3300/0 conn(0x7f06bc03ed00 0x7f06bc0ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:24.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.427+0000 7f06c9514700 1 --2- 172.21.15.47:0/4168371212 >> v2:172.21.15.64:3300/0 conn(0x7f06bc0d44b0 0x7f06bc03e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:24.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.427+0000 7f06b3fff700 1 -- 172.21.15.47:0/4168371212 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2675629588 0 0) 0x7f06bc040c60 con 0x7f06bc03d700 2022-06-29T15:44:24.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.427+0000 7f06b3fff700 1 -- 172.21.15.47:0/4168371212 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06ac003260 con 0x7f06bc03d700 2022-06-29T15:44:24.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.427+0000 7f06b3fff700 1 -- 172.21.15.47:0/4168371212 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3735986339 0 0) 0x7f06ac003260 con 0x7f06bc03d700 2022-06-29T15:44:24.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.427+0000 7f06c9d15700 1 -- 172.21.15.47:0/4168371212 >> v2:172.21.15.64:3300/0 conn(0x7f06bc0d44b0 msgr2=0x7f06bc03e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:44:24.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.427+0000 7f06c9d15700 1 --2- 172.21.15.47:0/4168371212 >> v2:172.21.15.64:3300/0 conn(0x7f06bc0d44b0 0x7f06bc03e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:24.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.427+0000 7f06c9d15700 1 -- 172.21.15.47:0/4168371212 >> v1:172.21.15.47:6789/0 conn(0x7f06bc03d700 legacy=0x7f06bc0cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:24.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.427+0000 7f06c9d15700 1 -- 172.21.15.47:0/4168371212 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06bc039630 con 0x7f06bc03ed00 2022-06-29T15:44:24.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.431+0000 7f06c9d15700 1 --2- 172.21.15.47:0/4168371212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06bc03ed00 0x7f06bc0ce430 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f06c0001800 tx=0x7f06c4069230).ready entity=mon.0 client_cookie=71d5d4b2d3a8d1c1 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:24.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.431+0000 7f06b3fff700 1 -- 172.21.15.47:0/4168371212 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06c0001660 con 0x7f06bc03ed00 2022-06-29T15:44:24.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.431+0000 7f06b3fff700 1 -- 172.21.15.47:0/4168371212 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06c4043f70 con 0x7f06bc03ed00 2022-06-29T15:44:24.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.431+0000 7f06b3fff700 1 -- 172.21.15.47:0/4168371212 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06c40446d0 con 0x7f06bc03ed00 2022-06-29T15:44:24.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.431+0000 7f06ca516700 1 -- 172.21.15.47:0/4168371212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06bc03ed00 msgr2=0x7f06bc0ce430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:24.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.431+0000 7f06ca516700 1 --2- 172.21.15.47:0/4168371212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06bc03ed00 0x7f06bc0ce430 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f06c0001800 tx=0x7f06c4069230).stop 2022-06-29T15:44:24.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.431+0000 7f06ca516700 1 -- 172.21.15.47:0/4168371212 shutdown_connections 2022-06-29T15:44:24.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.431+0000 7f06ca516700 1 --2- 172.21.15.47:0/4168371212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06bc03ed00 0x7f06bc0ce430 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:24.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.431+0000 7f06ca516700 1 --2- 172.21.15.47:0/4168371212 >> v2:172.21.15.64:3300/0 conn(0x7f06bc0d44b0 0x7f06bc03e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:24.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.431+0000 7f06ca516700 1 -- 172.21.15.47:0/4168371212 >> 172.21.15.47:0/4168371212 conn(0x7f06bc0462c0 msgr2=0x7f06bc0466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:24.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.431+0000 7f06ca516700 1 -- 172.21.15.47:0/4168371212 shutdown_connections 2022-06-29T15:44:24.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.431+0000 7f06ca516700 1 -- 172.21.15.47:0/4168371212 wait complete. 2022-06-29T15:44:24.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.431+0000 7f06ca516700 1 Processor -- start 2022-06-29T15:44:24.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.431+0000 7f06ca516700 1 -- start start 2022-06-29T15:44:24.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.435+0000 7f06ca516700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06bc03d700 0x7f06bc03a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:24.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.435+0000 7f06ca516700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06bc03ed00 0x7f06bc0cde30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:24.449 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.435+0000 7f06ca516700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f06bc0d44b0 0x7f06bc0d0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:24.449 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.435+0000 7f06ca516700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06bc0cf9d0 con 0x7f06bc03ed00 2022-06-29T15:44:24.449 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.435+0000 7f06ca516700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06bc0e4340 con 0x7f06bc03d700 2022-06-29T15:44:24.449 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.435+0000 7f06ca516700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06bc0cfc30 con 0x7f06bc0d44b0 2022-06-29T15:44:24.450 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.435+0000 7f06c8d13700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06bc03ed00 0x7f06bc0cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:24.450 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.435+0000 7f06c8d13700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06bc03ed00 0x7f06bc0cde30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47696/0 (socket says 172.21.15.47:47696) 2022-06-29T15:44:24.450 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.435+0000 7f06c8d13700 1 -- 172.21.15.47:0/1298148048 learned_addr learned my addr 172.21.15.47:0/1298148048 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:24.450 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.435+0000 7f06c9514700 1 --2- 172.21.15.47:0/1298148048 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06bc03d700 0x7f06bc03a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:24.451 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.435+0000 7f06c9d15700 1 --2- 172.21.15.47:0/1298148048 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f06bc0d44b0 0x7f06bc0d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:24.451 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.435+0000 7f06c8d13700 1 -- 172.21.15.47:0/1298148048 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f06bc0d44b0 msgr2=0x7f06bc0d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:24.451 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.435+0000 7f06c8d13700 1 --2- 172.21.15.47:0/1298148048 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f06bc0d44b0 0x7f06bc0d0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:24.451 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.435+0000 7f06c8d13700 1 -- 172.21.15.47:0/1298148048 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06bc03d700 msgr2=0x7f06bc03a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:24.452 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.435+0000 7f06c8d13700 1 --2- 172.21.15.47:0/1298148048 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06bc03d700 0x7f06bc03a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:24.452 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.435+0000 7f06c8d13700 1 -- 172.21.15.47:0/1298148048 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06bc0cfed0 con 0x7f06bc03ed00 2022-06-29T15:44:24.452 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.435+0000 7f06c9d15700 1 --2- 172.21.15.47:0/1298148048 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f06bc0d44b0 0x7f06bc0d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:44:24.452 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.435+0000 7f06c9514700 1 --2- 172.21.15.47:0/1298148048 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06bc03d700 0x7f06bc03a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:44:24.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.435+0000 7f06c8d13700 1 --2- 172.21.15.47:0/1298148048 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06bc03ed00 0x7f06bc0cde30 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7f06c00018c0 tx=0x7f06c0001230).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:24.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.435+0000 7f06b27fc700 1 -- 172.21.15.47:0/1298148048 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06c0007170 con 0x7f06bc03ed00 2022-06-29T15:44:24.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.435+0000 7f06b27fc700 1 -- 172.21.15.47:0/1298148048 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06c00072d0 con 0x7f06bc03ed00 2022-06-29T15:44:24.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.435+0000 7f06ca516700 1 -- 172.21.15.47:0/1298148048 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06bc039d30 con 0x7f06bc03ed00 2022-06-29T15:44:24.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.435+0000 7f06b27fc700 1 -- 172.21.15.47:0/1298148048 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06c0004920 con 0x7f06bc03ed00 2022-06-29T15:44:24.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.435+0000 7f06ca516700 1 -- 172.21.15.47:0/1298148048 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06bc0d0860 con 0x7f06bc03ed00 2022-06-29T15:44:24.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.435+0000 7f06ca516700 1 -- 172.21.15.47:0/1298148048 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06bc03a4a0 con 0x7f06bc03ed00 2022-06-29T15:44:24.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.435+0000 7f06b27fc700 1 -- 172.21.15.47:0/1298148048 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f06c0004a80 con 0x7f06bc03ed00 2022-06-29T15:44:24.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.439+0000 7f06b27fc700 1 --2- 172.21.15.47:0/1298148048 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f06a4053a70 0x7f06a4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:24.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.439+0000 7f06b27fc700 1 -- 172.21.15.47:0/1298148048 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f06c006b980 con 0x7f06bc03ed00 2022-06-29T15:44:24.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.439+0000 7f06c9514700 1 --2- 172.21.15.47:0/1298148048 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f06a4053a70 0x7f06a4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:24.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.439+0000 7f06c9514700 1 --2- 172.21.15.47:0/1298148048 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f06a4053a70 0x7f06a4055f30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f06b400b000 tx=0x7f06b4008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:24.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.443+0000 7f06b27fc700 1 -- 172.21.15.47:0/1298148048 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f06bc03a4a0 con 0x7f06bc03ed00 2022-06-29T15:44:24.569 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.563+0000 7f06ca516700 1 -- 172.21.15.47:0/1298148048 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06bc03a4a0 con 0x7f06bc03ed00 2022-06-29T15:44:24.570 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.563+0000 7f06b27fc700 1 -- 172.21.15.47:0/1298148048 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+13414 (secure 0 0 0) 0x7f06c0025480 con 0x7f06bc03ed00 2022-06-29T15:44:24.571 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:44:24.571 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":43,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:44:23.935990+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:44:24.573 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.567+0000 7f06ca516700 1 -- 172.21.15.47:0/1298148048 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f06a4053a70 msgr2=0x7f06a4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:24.574 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.567+0000 7f06ca516700 1 --2- 172.21.15.47:0/1298148048 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f06a4053a70 0x7f06a4055f30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f06b400b000 tx=0x7f06b4008040).stop 2022-06-29T15:44:24.574 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.567+0000 7f06ca516700 1 -- 172.21.15.47:0/1298148048 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06bc03ed00 msgr2=0x7f06bc0cde30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:24.574 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.567+0000 7f06ca516700 1 --2- 172.21.15.47:0/1298148048 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06bc03ed00 0x7f06bc0cde30 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7f06c00018c0 tx=0x7f06c0001230).stop 2022-06-29T15:44:24.575 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.567+0000 7f06ca516700 1 -- 172.21.15.47:0/1298148048 shutdown_connections 2022-06-29T15:44:24.575 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.567+0000 7f06ca516700 1 --2- 172.21.15.47:0/1298148048 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f06bc0d44b0 0x7f06bc0d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:24.575 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.567+0000 7f06ca516700 1 --2- 172.21.15.47:0/1298148048 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06bc03ed00 0x7f06bc0cde30 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:24.575 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.567+0000 7f06ca516700 1 --2- 172.21.15.47:0/1298148048 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f06a4053a70 0x7f06a4055f30 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:24.575 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.567+0000 7f06ca516700 1 --2- 172.21.15.47:0/1298148048 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06bc03d700 0x7f06bc03a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:24.576 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.567+0000 7f06ca516700 1 -- 172.21.15.47:0/1298148048 >> 172.21.15.47:0/1298148048 conn(0x7f06bc0462c0 msgr2=0x7f06bc0d2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:24.576 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.567+0000 7f06ca516700 1 -- 172.21.15.47:0/1298148048 shutdown_connections 2022-06-29T15:44:24.576 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.567+0000 7f06ca516700 1 -- 172.21.15.47:0/1298148048 wait complete. 2022-06-29T15:44:24.588 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-29T15:42:16.971431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-29T15:42:25.290884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-06-29T15:44:24.589 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-29T15:44:24.692 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.687+0000 7fc99759e700 1 Processor -- start 2022-06-29T15:44:24.693 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.687+0000 7fc99759e700 1 -- start start 2022-06-29T15:44:24.695 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.687+0000 7fc99759e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc9980b2ad0 0x7fc9980ad3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:24.695 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.687+0000 7fc99759e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc99813eaa0 con 0x7fc9980b1bb0 2022-06-29T15:44:24.695 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.687+0000 7fc99759e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc998140f40 con 0x7fc9980acfd0 2022-06-29T15:44:24.696 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.687+0000 7fc99759e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9980a96a0 con 0x7fc9980b2ad0 2022-06-29T15:44:24.696 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.687+0000 7fc995d9b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc9980b1bb0 0x7fc9980b23c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33256/0 (socket says 172.21.15.47:33256) 2022-06-29T15:44:24.696 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.687+0000 7fc995d9b700 1 -- 172.21.15.47:0/3172143838 learned_addr learned my addr 172.21.15.47:0/3172143838 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:44:24.696 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.687+0000 7fc996d9d700 1 --2- 172.21.15.47:0/3172143838 >> v2:172.21.15.64:3300/0 conn(0x7fc9980b2ad0 0x7fc9980ad3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:24.696 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.687+0000 7fc99559a700 1 -- 172.21.15.47:0/3172143838 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4237539056 0 0) 0x7fc99813eaa0 con 0x7fc9980b1bb0 2022-06-29T15:44:24.697 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.687+0000 7fc99559a700 1 -- 172.21.15.47:0/3172143838 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc978003260 con 0x7fc9980b1bb0 2022-06-29T15:44:24.697 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.687+0000 7fc996d9d700 1 -- 172.21.15.47:0/3172143838 >> v1:172.21.15.64:6789/0 conn(0x7fc9980acfd0 legacy=0x7fc998141640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:24.697 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.687+0000 7fc996d9d700 1 -- 172.21.15.47:0/3172143838 >> v1:172.21.15.47:6789/0 conn(0x7fc9980b1bb0 legacy=0x7fc9980b23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:24.697 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.687+0000 7fc996d9d700 1 -- 172.21.15.47:0/3172143838 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9980a9400 con 0x7fc9980b2ad0 2022-06-29T15:44:24.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.687+0000 7fc99559a700 1 -- 172.21.15.47:0/3172143838 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1151193043 0 0) 0x7fc978003260 con 0x7fc9980b1bb0 2022-06-29T15:44:24.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.691+0000 7fc996d9d700 1 --2- 172.21.15.47:0/3172143838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9980b2ad0 0x7fc9980ad3b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7fc9900032f0 tx=0x7fc9900091d0).ready entity=mon.1 client_cookie=af9ea4e8f241a7af server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:24.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.691+0000 7fc99559a700 1 -- 172.21.15.47:0/3172143838 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc99000b6c0 con 0x7fc9980b2ad0 2022-06-29T15:44:24.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.691+0000 7fc99559a700 1 -- 172.21.15.47:0/3172143838 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc99000b820 con 0x7fc9980b2ad0 2022-06-29T15:44:24.699 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.691+0000 7fc99559a700 1 -- 172.21.15.47:0/3172143838 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc99000b990 con 0x7fc9980b2ad0 2022-06-29T15:44:24.702 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.691+0000 7fc99759e700 1 -- 172.21.15.47:0/3172143838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9980b2ad0 msgr2=0x7fc9980ad3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:24.702 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.691+0000 7fc99759e700 1 --2- 172.21.15.47:0/3172143838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9980b2ad0 0x7fc9980ad3b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7fc9900032f0 tx=0x7fc9900091d0).stop 2022-06-29T15:44:24.702 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.691+0000 7fc99759e700 1 -- 172.21.15.47:0/3172143838 shutdown_connections 2022-06-29T15:44:24.702 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.691+0000 7fc99759e700 1 --2- 172.21.15.47:0/3172143838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9980b2ad0 0x7fc9980ad3b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:24.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.691+0000 7fc99759e700 1 -- 172.21.15.47:0/3172143838 >> 172.21.15.47:0/3172143838 conn(0x7fc998062df0 msgr2=0x7fc9980631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:24.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.691+0000 7fc99759e700 1 -- 172.21.15.47:0/3172143838 shutdown_connections 2022-06-29T15:44:24.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.691+0000 7fc99759e700 1 -- 172.21.15.47:0/3172143838 wait complete. 2022-06-29T15:44:24.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.691+0000 7fc99759e700 1 Processor -- start 2022-06-29T15:44:24.704 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.691+0000 7fc99759e700 1 -- start start 2022-06-29T15:44:24.704 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.695+0000 7fc99759e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc9980acfd0 0x7fc9981424e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:24.704 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.695+0000 7fc99759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9980b1bb0 0x7fc9980b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:24.704 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.695+0000 7fc99759e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc9980b2ad0 0x7fc9980b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:24.704 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.695+0000 7fc99759e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc998099ef0 con 0x7fc9980b2ad0 2022-06-29T15:44:24.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.695+0000 7fc99759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9980aa3b0 con 0x7fc9980b1bb0 2022-06-29T15:44:24.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.695+0000 7fc99759e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9980b5ca0 con 0x7fc9980acfd0 2022-06-29T15:44:24.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.695+0000 7fc996d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc9980b2ad0 0x7fc9980b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:24.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.695+0000 7fc996d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc9980b2ad0 0x7fc9980b3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47712/0 (socket says 172.21.15.47:47712) 2022-06-29T15:44:24.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.695+0000 7fc996d9d700 1 -- 172.21.15.47:0/916759601 learned_addr learned my addr 172.21.15.47:0/916759601 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:24.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.695+0000 7fc995d9b700 1 --2- 172.21.15.47:0/916759601 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9980b1bb0 0x7fc9980b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:24.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.695+0000 7fc99659c700 1 --2- 172.21.15.47:0/916759601 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc9980acfd0 0x7fc9981424e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:24.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.695+0000 7fc996d9d700 1 -- 172.21.15.47:0/916759601 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc9980acfd0 msgr2=0x7fc9981424e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:24.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.695+0000 7fc996d9d700 1 --2- 172.21.15.47:0/916759601 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc9980acfd0 0x7fc9981424e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:24.707 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.695+0000 7fc996d9d700 1 -- 172.21.15.47:0/916759601 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9980b1bb0 msgr2=0x7fc9980b3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:24.707 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.695+0000 7fc996d9d700 1 --2- 172.21.15.47:0/916759601 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9980b1bb0 0x7fc9980b3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:24.707 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.695+0000 7fc996d9d700 1 -- 172.21.15.47:0/916759601 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc990009470 con 0x7fc9980b2ad0 2022-06-29T15:44:24.707 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.695+0000 7fc996d9d700 1 --2- 172.21.15.47:0/916759601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc9980b2ad0 0x7fc9980b3f80 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7fc9900032f0 tx=0x7fc990016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:24.708 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.695+0000 7fc98f7fe700 1 -- 172.21.15.47:0/916759601 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc99000b6c0 con 0x7fc9980b2ad0 2022-06-29T15:44:24.708 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.695+0000 7fc98f7fe700 1 -- 172.21.15.47:0/916759601 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc990021630 con 0x7fc9980b2ad0 2022-06-29T15:44:24.708 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.695+0000 7fc99759e700 1 -- 172.21.15.47:0/916759601 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9980b5ed0 con 0x7fc9980b2ad0 2022-06-29T15:44:24.708 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.695+0000 7fc98f7fe700 1 -- 172.21.15.47:0/916759601 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc990021990 con 0x7fc9980b2ad0 2022-06-29T15:44:24.708 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.695+0000 7fc99759e700 1 -- 172.21.15.47:0/916759601 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9980b56a0 con 0x7fc9980b2ad0 2022-06-29T15:44:24.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.695+0000 7fc99759e700 1 -- 172.21.15.47:0/916759601 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9980b6330 con 0x7fc9980b2ad0 2022-06-29T15:44:24.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.699+0000 7fc98f7fe700 1 -- 172.21.15.47:0/916759601 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc990027020 con 0x7fc9980b2ad0 2022-06-29T15:44:24.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.699+0000 7fc98f7fe700 1 --2- 172.21.15.47:0/916759601 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc97c053b30 0x7fc97c055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:24.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.699+0000 7fc98f7fe700 1 -- 172.21.15.47:0/916759601 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fc99006cf20 con 0x7fc9980b2ad0 2022-06-29T15:44:24.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.699+0000 7fc99659c700 1 --2- 172.21.15.47:0/916759601 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc97c053b30 0x7fc97c055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:24.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.699+0000 7fc99659c700 1 --2- 172.21.15.47:0/916759601 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc97c053b30 0x7fc97c055ff0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7fc98800d000 tx=0x7fc988007b50).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:24.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.703+0000 7fc98f7fe700 1 -- 172.21.15.47:0/916759601 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc990044020 con 0x7fc9980b2ad0 2022-06-29T15:44:24.836 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.831+0000 7fc99759e700 1 -- 172.21.15.47:0/916759601 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fc9980573d0 con 0x7fc9980b2ad0 2022-06-29T15:44:24.837 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.831+0000 7fc98f7fe700 1 -- 172.21.15.47:0/916759601 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v43) v1 ==== 110+0+10 (secure 0 0 0) 0x7fc990015020 con 0x7fc9980b2ad0 2022-06-29T15:44:24.838 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2022-06-29T15:44:24.839 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.835+0000 7fc99759e700 1 -- 172.21.15.47:0/916759601 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc97c053b30 msgr2=0x7fc97c055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:24.840 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.835+0000 7fc99759e700 1 --2- 172.21.15.47:0/916759601 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc97c053b30 0x7fc97c055ff0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7fc98800d000 tx=0x7fc988007b50).stop 2022-06-29T15:44:24.840 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.835+0000 7fc99759e700 1 -- 172.21.15.47:0/916759601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc9980b2ad0 msgr2=0x7fc9980b3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:24.840 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.835+0000 7fc99759e700 1 --2- 172.21.15.47:0/916759601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc9980b2ad0 0x7fc9980b3f80 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7fc9900032f0 tx=0x7fc990016000).stop 2022-06-29T15:44:24.840 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.835+0000 7fc99759e700 1 -- 172.21.15.47:0/916759601 shutdown_connections 2022-06-29T15:44:24.841 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.835+0000 7fc99759e700 1 --2- 172.21.15.47:0/916759601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc9980b2ad0 0x7fc9980b3f80 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:24.841 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.835+0000 7fc99759e700 1 --2- 172.21.15.47:0/916759601 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc97c053b30 0x7fc97c055ff0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:24.841 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.835+0000 7fc99759e700 1 --2- 172.21.15.47:0/916759601 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9980b1bb0 0x7fc9980b3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:24.841 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.835+0000 7fc99759e700 1 --2- 172.21.15.47:0/916759601 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc9980acfd0 0x7fc9981424e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:24.842 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.835+0000 7fc99759e700 1 -- 172.21.15.47:0/916759601 >> 172.21.15.47:0/916759601 conn(0x7fc998062df0 msgr2=0x7fc99813c960 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:24.842 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.835+0000 7fc99759e700 1 -- 172.21.15.47:0/916759601 shutdown_connections 2022-06-29T15:44:24.842 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.835+0000 7fc99759e700 1 -- 172.21.15.47:0/916759601 wait complete. 2022-06-29T15:44:24.850 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-29T15:44:24.963 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.959+0000 7f531ea40700 1 Processor -- start 2022-06-29T15:44:24.963 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.959+0000 7f531ea40700 1 -- start start 2022-06-29T15:44:24.965 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.959+0000 7f531ea40700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f53180a8ce0 0x7f53180b05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:24.965 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.959+0000 7f531ea40700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f53181413c0 0x7f5318141870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:24.966 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.959+0000 7f531ea40700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53180b5e40 con 0x7f5318060a70 2022-06-29T15:44:24.966 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.959+0000 7f531ea40700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53180ac710 con 0x7f53180a8ce0 2022-06-29T15:44:24.966 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.959+0000 7f531ea40700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5318060e50 con 0x7f53181413c0 2022-06-29T15:44:24.966 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.959+0000 7f53177fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f53180a8ce0 0x7f53180b05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:24.967 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.959+0000 7f531cfdd700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f53181413c0 0x7f5318141870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:24.967 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.959+0000 7f5317fff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f5318060a70 0x7f53180afed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:40834/0 (socket says 172.21.15.47:40834) 2022-06-29T15:44:24.967 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.959+0000 7f5317fff700 1 -- 172.21.15.47:0/2702894765 learned_addr learned my addr 172.21.15.47:0/2702894765 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:44:24.967 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.959+0000 7f53177fe700 1 -- 172.21.15.47:0/2702894765 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f53181413c0 msgr2=0x7f5318141870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:44:24.968 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.959+0000 7f53177fe700 1 --2- 172.21.15.47:0/2702894765 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f53181413c0 0x7f5318141870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:24.968 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.959+0000 7f53177fe700 1 -- 172.21.15.47:0/2702894765 >> v1:172.21.15.64:6789/0 conn(0x7f5318060a70 legacy=0x7f53180afed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:24.968 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.959+0000 7f53177fe700 1 -- 172.21.15.47:0/2702894765 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53180ac2e0 con 0x7f53180a8ce0 2022-06-29T15:44:24.968 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.959+0000 7f53177fe700 1 --2- 172.21.15.47:0/2702894765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53180a8ce0 0x7f53180b05e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f5308002f70 tx=0x7f5308006df0).ready entity=mon.1 client_cookie=6cbe243d3b83686b server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:24.972 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.959+0000 7f5316ffd700 1 -- 172.21.15.47:0/2702894765 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f530800d570 con 0x7f53180a8ce0 2022-06-29T15:44:24.972 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.959+0000 7f5316ffd700 1 -- 172.21.15.47:0/2702894765 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5308000e90 con 0x7f53180a8ce0 2022-06-29T15:44:24.973 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.959+0000 7f5316ffd700 1 -- 172.21.15.47:0/2702894765 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f530800d8c0 con 0x7f53180a8ce0 2022-06-29T15:44:24.973 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.959+0000 7f531ea40700 1 -- 172.21.15.47:0/2702894765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53180a8ce0 msgr2=0x7f53180b05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:24.973 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.959+0000 7f531ea40700 1 --2- 172.21.15.47:0/2702894765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53180a8ce0 0x7f53180b05e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f5308002f70 tx=0x7f5308006df0).stop 2022-06-29T15:44:24.973 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.959+0000 7f531ea40700 1 -- 172.21.15.47:0/2702894765 shutdown_connections 2022-06-29T15:44:24.974 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.959+0000 7f531ea40700 1 --2- 172.21.15.47:0/2702894765 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f53181413c0 0x7f5318141870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:24.974 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.959+0000 7f531ea40700 1 --2- 172.21.15.47:0/2702894765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53180a8ce0 0x7f53180b05e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:24.974 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.959+0000 7f531ea40700 1 -- 172.21.15.47:0/2702894765 >> 172.21.15.47:0/2702894765 conn(0x7f53180626f0 msgr2=0x7f5318062ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:24.974 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.959+0000 7f531ea40700 1 -- 172.21.15.47:0/2702894765 shutdown_connections 2022-06-29T15:44:24.974 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.959+0000 7f531ea40700 1 -- 172.21.15.47:0/2702894765 wait complete. 2022-06-29T15:44:24.975 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.963+0000 7f531ea40700 1 Processor -- start 2022-06-29T15:44:24.975 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.963+0000 7f531ea40700 1 -- start start 2022-06-29T15:44:24.975 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.963+0000 7f531ea40700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5318060a70 0x7f5318135410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:24.975 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.963+0000 7f531ea40700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53180a8ce0 0x7f5318141870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:24.976 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.963+0000 7f531ea40700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f53181413c0 0x7f5318141d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:24.976 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.963+0000 7f531ea40700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5318136740 con 0x7f53180a8ce0 2022-06-29T15:44:24.976 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.963+0000 7f531ea40700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53181360a0 con 0x7f5318060a70 2022-06-29T15:44:24.976 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.963+0000 7f531ea40700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53180b49e0 con 0x7f53181413c0 2022-06-29T15:44:24.977 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.963+0000 7f53177fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53180a8ce0 0x7f5318141870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:24.977 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.963+0000 7f53177fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53180a8ce0 0x7f5318141870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47722/0 (socket says 172.21.15.47:47722) 2022-06-29T15:44:24.977 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.963+0000 7f53177fe700 1 -- 172.21.15.47:0/3125523038 learned_addr learned my addr 172.21.15.47:0/3125523038 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:24.977 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.963+0000 7f5317fff700 1 --2- 172.21.15.47:0/3125523038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5318060a70 0x7f5318135410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:24.977 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.963+0000 7f531cfdd700 1 --2- 172.21.15.47:0/3125523038 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f53181413c0 0x7f5318141d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:24.978 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.963+0000 7f5317fff700 1 -- 172.21.15.47:0/3125523038 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f53181413c0 msgr2=0x7f5318141d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:24.978 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.963+0000 7f5317fff700 1 --2- 172.21.15.47:0/3125523038 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f53181413c0 0x7f5318141d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:24.978 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.963+0000 7f5317fff700 1 -- 172.21.15.47:0/3125523038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53180a8ce0 msgr2=0x7f5318141870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:24.978 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.963+0000 7f5317fff700 1 --2- 172.21.15.47:0/3125523038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53180a8ce0 0x7f5318141870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:24.979 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.963+0000 7f5317fff700 1 -- 172.21.15.47:0/3125523038 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f530800b180 con 0x7f5318060a70 2022-06-29T15:44:24.979 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.963+0000 7f5317fff700 1 --2- 172.21.15.47:0/3125523038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5318060a70 0x7f5318135410 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f5318061b40 tx=0x7f530c007b50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:24.979 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.963+0000 7f53157fa700 1 -- 172.21.15.47:0/3125523038 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f530c0070d0 con 0x7f5318060a70 2022-06-29T15:44:24.979 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.963+0000 7f53157fa700 1 -- 172.21.15.47:0/3125523038 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f530c007230 con 0x7f5318060a70 2022-06-29T15:44:24.980 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.963+0000 7f531ea40700 1 -- 172.21.15.47:0/3125523038 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53180b59e0 con 0x7f5318060a70 2022-06-29T15:44:24.980 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.963+0000 7f53157fa700 1 -- 172.21.15.47:0/3125523038 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f530c005610 con 0x7f5318060a70 2022-06-29T15:44:24.980 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.967+0000 7f531ea40700 1 -- 172.21.15.47:0/3125523038 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53180b4540 con 0x7f5318060a70 2022-06-29T15:44:24.980 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.967+0000 7f531ea40700 1 -- 172.21.15.47:0/3125523038 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53180b5b70 con 0x7f5318060a70 2022-06-29T15:44:24.981 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.967+0000 7f53157fa700 1 -- 172.21.15.47:0/3125523038 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f530c010070 con 0x7f5318060a70 2022-06-29T15:44:24.982 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.967+0000 7f53157fa700 1 --2- 172.21.15.47:0/3125523038 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f52f80539e0 0x7f52f8055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:24.982 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.967+0000 7f53177fe700 1 --2- 172.21.15.47:0/3125523038 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f52f80539e0 0x7f52f8055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:24.982 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.967+0000 7f53157fa700 1 -- 172.21.15.47:0/3125523038 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f530c06aff0 con 0x7f5318060a70 2022-06-29T15:44:24.982 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.967+0000 7f53177fe700 1 --2- 172.21.15.47:0/3125523038 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f52f80539e0 0x7f52f8055ea0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f530801a000 tx=0x7f5308014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:24.983 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:24.975+0000 7f53157fa700 1 -- 172.21.15.47:0/3125523038 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f530c0420b0 con 0x7f5318060a70 2022-06-29T15:44:25.110 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.103+0000 7f531ea40700 1 -- 172.21.15.47:0/3125523038 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f53180b5e40 con 0x7f5318060a70 2022-06-29T15:44:25.111 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.107+0000 7f53157fa700 1 -- 172.21.15.47:0/3125523038 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v43) v1 ==== 92+0+10 (secure 0 0 0) 0x7f53180b5e40 con 0x7f5318060a70 2022-06-29T15:44:25.111 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:44:25.115 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.107+0000 7f531ea40700 1 -- 172.21.15.47:0/3125523038 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f52f80539e0 msgr2=0x7f52f8055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:25.115 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.107+0000 7f531ea40700 1 --2- 172.21.15.47:0/3125523038 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f52f80539e0 0x7f52f8055ea0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f530801a000 tx=0x7f5308014000).stop 2022-06-29T15:44:25.115 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.107+0000 7f531ea40700 1 -- 172.21.15.47:0/3125523038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5318060a70 msgr2=0x7f5318135410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:25.115 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.107+0000 7f531ea40700 1 --2- 172.21.15.47:0/3125523038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5318060a70 0x7f5318135410 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f5318061b40 tx=0x7f530c007b50).stop 2022-06-29T15:44:25.116 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.107+0000 7f531ea40700 1 -- 172.21.15.47:0/3125523038 shutdown_connections 2022-06-29T15:44:25.116 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.107+0000 7f531ea40700 1 --2- 172.21.15.47:0/3125523038 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f53181413c0 0x7f5318141d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:25.116 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.107+0000 7f531ea40700 1 --2- 172.21.15.47:0/3125523038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53180a8ce0 0x7f5318141870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:25.116 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.107+0000 7f531ea40700 1 --2- 172.21.15.47:0/3125523038 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f52f80539e0 0x7f52f8055ea0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:25.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.107+0000 7f531ea40700 1 --2- 172.21.15.47:0/3125523038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5318060a70 0x7f5318135410 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:25.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.107+0000 7f531ea40700 1 -- 172.21.15.47:0/3125523038 >> 172.21.15.47:0/3125523038 conn(0x7f53180626f0 msgr2=0x7f53180a9420 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:25.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.107+0000 7f531ea40700 1 -- 172.21.15.47:0/3125523038 shutdown_connections 2022-06-29T15:44:25.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.107+0000 7f531ea40700 1 -- 172.21.15.47:0/3125523038 wait complete. 2022-06-29T15:44:25.131 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-06-29T15:44:25.238 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.231+0000 7f4fa0d53700 1 Processor -- start 2022-06-29T15:44:25.239 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.231+0000 7f4fa0d53700 1 -- start start 2022-06-29T15:44:25.241 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.235+0000 7f4fa0d53700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4f9c0b2120 0x7f4f9c0b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:25.241 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.235+0000 7f4fa0d53700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4f9c05f0b0 0x7f4f9c14b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:25.242 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.235+0000 7f4fa0d53700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f9c0b0740 con 0x7f4f9c0b1260 2022-06-29T15:44:25.242 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.235+0000 7f4fa0d53700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f9c147460 con 0x7f4f9c05f0b0 2022-06-29T15:44:25.242 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.235+0000 7f4fa0d53700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9c0b0970 con 0x7f4f9c0b2120 2022-06-29T15:44:25.242 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.235+0000 7f4f99d9b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4f9c05f0b0 0x7f4f9c14b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:25.243 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.235+0000 7f4f99d9b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4f9c05f0b0 0x7f4f9c14b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:53432/0 (socket says 172.21.15.47:53432) 2022-06-29T15:44:25.243 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.235+0000 7f4f99d9b700 1 -- 172.21.15.47:0/439749179 learned_addr learned my addr 172.21.15.47:0/439749179 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:25.243 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.235+0000 7f4f9a59c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4f9c0b2120 0x7f4f9c0b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:25.243 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.235+0000 7f4f9ad9d700 1 --1- 172.21.15.47:0/439749179 >> v1:172.21.15.64:6789/0 conn(0x7f4f9c0b1260 0x7f4f9c147af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:40850/0 (socket says 172.21.15.47:40850) 2022-06-29T15:44:25.244 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.235+0000 7f4f99d9b700 1 -- 172.21.15.47:0/439749179 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4f9c0b2120 msgr2=0x7f4f9c0b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:44:25.244 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.235+0000 7f4f99d9b700 1 --2- 172.21.15.47:0/439749179 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4f9c0b2120 0x7f4f9c0b2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:25.244 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.235+0000 7f4f99d9b700 1 -- 172.21.15.47:0/439749179 >> v1:172.21.15.64:6789/0 conn(0x7f4f9c0b1260 legacy=0x7f4f9c147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:25.245 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.235+0000 7f4f99d9b700 1 -- 172.21.15.47:0/439749179 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f9c0b0c10 con 0x7f4f9c05f0b0 2022-06-29T15:44:25.245 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.235+0000 7f4f99d9b700 1 --2- 172.21.15.47:0/439749179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f9c05f0b0 0x7f4f9c14b520 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f4f8c002f70 tx=0x7f4f8c006eb0).ready entity=mon.1 client_cookie=2c908535d5f1d49f server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:25.245 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.235+0000 7f4f9959a700 1 -- 172.21.15.47:0/439749179 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f8c00d570 con 0x7f4f9c05f0b0 2022-06-29T15:44:25.245 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.235+0000 7f4f9959a700 1 -- 172.21.15.47:0/439749179 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f8c000e90 con 0x7f4f9c05f0b0 2022-06-29T15:44:25.246 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.235+0000 7f4f9959a700 1 -- 172.21.15.47:0/439749179 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f8c00d900 con 0x7f4f9c05f0b0 2022-06-29T15:44:25.250 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.235+0000 7f4fa0d53700 1 -- 172.21.15.47:0/439749179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f9c05f0b0 msgr2=0x7f4f9c14b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:25.250 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.235+0000 7f4fa0d53700 1 --2- 172.21.15.47:0/439749179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f9c05f0b0 0x7f4f9c14b520 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f4f8c002f70 tx=0x7f4f8c006eb0).stop 2022-06-29T15:44:25.251 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.235+0000 7f4fa0d53700 1 -- 172.21.15.47:0/439749179 shutdown_connections 2022-06-29T15:44:25.251 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.235+0000 7f4fa0d53700 1 --2- 172.21.15.47:0/439749179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f9c05f0b0 0x7f4f9c14b520 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:25.251 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.235+0000 7f4fa0d53700 1 --2- 172.21.15.47:0/439749179 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4f9c0b2120 0x7f4f9c0b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:25.252 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.235+0000 7f4fa0d53700 1 -- 172.21.15.47:0/439749179 >> 172.21.15.47:0/439749179 conn(0x7f4f9c0626f0 msgr2=0x7f4f9c062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:25.252 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.235+0000 7f4fa0d53700 1 -- 172.21.15.47:0/439749179 shutdown_connections 2022-06-29T15:44:25.252 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.235+0000 7f4fa0d53700 1 -- 172.21.15.47:0/439749179 wait complete. 2022-06-29T15:44:25.252 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.239+0000 7f4fa0d53700 1 Processor -- start 2022-06-29T15:44:25.252 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.239+0000 7f4fa0d53700 1 -- start start 2022-06-29T15:44:25.253 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.239+0000 7f4fa0d53700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f9c0b1260 0x7f4f9c0637d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:25.253 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.239+0000 7f4fa0d53700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4f9c0b2120 0x7f4f9c1570c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:25.253 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.239+0000 7f4fa0d53700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f9c063cc0 0x7f4f9c1575b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:25.253 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.239+0000 7f4fa0d53700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9c143d50 con 0x7f4f9c063cc0 2022-06-29T15:44:25.254 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.239+0000 7f4fa0d53700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9c156b40 con 0x7f4f9c0b1260 2022-06-29T15:44:25.254 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.239+0000 7f4fa0d53700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9c141b70 con 0x7f4f9c0b2120 2022-06-29T15:44:25.254 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.239+0000 7f4f9ad9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f9c063cc0 0x7f4f9c1575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:25.255 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.239+0000 7f4f99d9b700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4f9c0b2120 0x7f4f9c1570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:25.255 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.239+0000 7f4f9a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f9c0b1260 0x7f4f9c0637d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:25.255 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.239+0000 7f4f9ad9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f9c063cc0 0x7f4f9c1575b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47738/0 (socket says 172.21.15.47:47738) 2022-06-29T15:44:25.255 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.239+0000 7f4f99d9b700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4f9c0b2120 0x7f4f9c1570c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:44838/0 (socket says 172.21.15.47:44838) 2022-06-29T15:44:25.256 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.239+0000 7f4f9ad9d700 1 -- 172.21.15.47:0/655561760 learned_addr learned my addr 172.21.15.47:0/655561760 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:25.256 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.239+0000 7f4f9ad9d700 1 -- 172.21.15.47:0/655561760 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4f9c0b2120 msgr2=0x7f4f9c1570c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:25.256 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.239+0000 7f4f9ad9d700 1 --2- 172.21.15.47:0/655561760 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4f9c0b2120 0x7f4f9c1570c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:25.256 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.239+0000 7f4f9ad9d700 1 -- 172.21.15.47:0/655561760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f9c0b1260 msgr2=0x7f4f9c0637d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:25.257 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.239+0000 7f4f9ad9d700 1 --2- 172.21.15.47:0/655561760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f9c0b1260 0x7f4f9c0637d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:25.257 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.239+0000 7f4f9ad9d700 1 -- 172.21.15.47:0/655561760 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f8c00b1e0 con 0x7f4f9c063cc0 2022-06-29T15:44:25.257 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.239+0000 7f4f99d9b700 1 --2- 172.21.15.47:0/655561760 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4f9c0b2120 0x7f4f9c1570c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:44:25.257 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.239+0000 7f4f9a59c700 1 --2- 172.21.15.47:0/655561760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f9c0b1260 0x7f4f9c0637d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:44:25.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.239+0000 7f4f9ad9d700 1 --2- 172.21.15.47:0/655561760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f9c063cc0 0x7f4f9c1575b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7f4f9c144ae0 tx=0x7f4f94007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:25.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.239+0000 7f4f8b7fe700 1 -- 172.21.15.47:0/655561760 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f94005ae0 con 0x7f4f9c063cc0 2022-06-29T15:44:25.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.239+0000 7f4f8b7fe700 1 -- 172.21.15.47:0/655561760 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f94005c40 con 0x7f4f9c063cc0 2022-06-29T15:44:25.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.239+0000 7f4fa0d53700 1 -- 172.21.15.47:0/655561760 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f9c156d30 con 0x7f4f9c063cc0 2022-06-29T15:44:25.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.239+0000 7f4f8b7fe700 1 -- 172.21.15.47:0/655561760 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f940070d0 con 0x7f4f9c063cc0 2022-06-29T15:44:25.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.239+0000 7f4fa0d53700 1 -- 172.21.15.47:0/655561760 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f9c0ae7e0 con 0x7f4f9c063cc0 2022-06-29T15:44:25.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.243+0000 7f4fa0d53700 1 -- 172.21.15.47:0/655561760 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f9c0573d0 con 0x7f4f9c063cc0 2022-06-29T15:44:25.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.243+0000 7f4f8b7fe700 1 -- 172.21.15.47:0/655561760 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4f9401a060 con 0x7f4f9c063cc0 2022-06-29T15:44:25.260 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.243+0000 7f4f8b7fe700 1 --2- 172.21.15.47:0/655561760 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4f78053b00 0x7f4f78055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:25.260 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.243+0000 7f4f8b7fe700 1 -- 172.21.15.47:0/655561760 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f4f9406b700 con 0x7f4f9c063cc0 2022-06-29T15:44:25.260 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.243+0000 7f4f9a59c700 1 --2- 172.21.15.47:0/655561760 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4f78053b00 0x7f4f78055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:25.260 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.243+0000 7f4f9a59c700 1 --2- 172.21.15.47:0/655561760 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4f78053b00 0x7f4f78055fc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f4f9000b000 tx=0x7f4f90006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:25.261 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.247+0000 7f4f8b7fe700 1 -- 172.21.15.47:0/655561760 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4f94042800 con 0x7f4f9c063cc0 2022-06-29T15:44:25.385 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.379+0000 7f4fa0d53700 1 -- 172.21.15.47:0/655561760 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f4f9c0a2c20 con 0x7f4f9c063cc0 2022-06-29T15:44:25.385 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.379+0000 7f4f8b7fe700 1 -- 172.21.15.47:0/655561760 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v43) v1 ==== 84+0+143 (secure 0 0 0) 0x7f4f94018020 con 0x7f4f9c063cc0 2022-06-29T15:44:25.385 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:44:25.386 INFO:teuthology.orchestra.run.smithi047.stdout:[ 2022-06-29T15:44:25.386 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:25.387 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 1, 2022-06-29T15:44:25.387 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": "device_health_metrics" 2022-06-29T15:44:25.387 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:25.387 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:44:25.388 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 2, 2022-06-29T15:44:25.388 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": "rbd" 2022-06-29T15:44:25.388 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:25.388 INFO:teuthology.orchestra.run.smithi047.stdout:] 2022-06-29T15:44:25.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.379+0000 7f4fa0d53700 1 -- 172.21.15.47:0/655561760 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4f78053b00 msgr2=0x7f4f78055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:25.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.379+0000 7f4fa0d53700 1 --2- 172.21.15.47:0/655561760 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4f78053b00 0x7f4f78055fc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f4f9000b000 tx=0x7f4f90006df0).stop 2022-06-29T15:44:25.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.379+0000 7f4fa0d53700 1 -- 172.21.15.47:0/655561760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f9c063cc0 msgr2=0x7f4f9c1575b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:25.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.379+0000 7f4fa0d53700 1 --2- 172.21.15.47:0/655561760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f9c063cc0 0x7f4f9c1575b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7f4f9c144ae0 tx=0x7f4f94007f40).stop 2022-06-29T15:44:25.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.383+0000 7f4fa0d53700 1 -- 172.21.15.47:0/655561760 shutdown_connections 2022-06-29T15:44:25.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.383+0000 7f4fa0d53700 1 --2- 172.21.15.47:0/655561760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f9c063cc0 0x7f4f9c1575b0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:25.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.383+0000 7f4fa0d53700 1 --2- 172.21.15.47:0/655561760 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4f9c0b2120 0x7f4f9c1570c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:25.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.383+0000 7f4fa0d53700 1 --2- 172.21.15.47:0/655561760 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4f78053b00 0x7f4f78055fc0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:25.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.383+0000 7f4fa0d53700 1 --2- 172.21.15.47:0/655561760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f9c0b1260 0x7f4f9c0637d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:25.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.383+0000 7f4fa0d53700 1 -- 172.21.15.47:0/655561760 >> 172.21.15.47:0/655561760 conn(0x7f4f9c0626f0 msgr2=0x7f4f9c14cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:25.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.383+0000 7f4fa0d53700 1 -- 172.21.15.47:0/655561760 shutdown_connections 2022-06-29T15:44:25.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.383+0000 7f4fa0d53700 1 -- 172.21.15.47:0/655561760 wait complete. 2022-06-29T15:44:25.398 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-06-29T15:44:25.399 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-06-29T15:44:25.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.503+0000 7fdb5f2ed700 1 Processor -- start 2022-06-29T15:44:25.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.503+0000 7fdb5f2ed700 1 -- start start 2022-06-29T15:44:25.513 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.507+0000 7fdb5f2ed700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdb5805f0b0 0x7fdb580b2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:25.513 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.507+0000 7fdb5f2ed700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb58099800 con 0x7fdb58060f20 2022-06-29T15:44:25.514 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.507+0000 7fdb5f2ed700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb5809a340 con 0x7fdb580b2bd0 2022-06-29T15:44:25.514 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.507+0000 7fdb5f2ed700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb580b1560 con 0x7fdb5805f0b0 2022-06-29T15:44:25.514 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.507+0000 7fdb5eaec700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fdb58060f20 0x7fdb581403b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33298/0 (socket says 172.21.15.47:33298) 2022-06-29T15:44:25.514 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.507+0000 7fdb5eaec700 1 -- 172.21.15.47:0/2951423576 learned_addr learned my addr 172.21.15.47:0/2951423576 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:44:25.515 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.507+0000 7fdb5daea700 1 --2- 172.21.15.47:0/2951423576 >> v2:172.21.15.47:3300/0 conn(0x7fdb5805f0b0 0x7fdb580b2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:25.515 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.507+0000 7fdb5d2e9700 1 -- 172.21.15.47:0/2951423576 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 263670641 0 0) 0x7fdb58099800 con 0x7fdb58060f20 2022-06-29T15:44:25.515 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.507+0000 7fdb5d2e9700 1 -- 172.21.15.47:0/2951423576 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb40003260 con 0x7fdb58060f20 2022-06-29T15:44:25.515 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.507+0000 7fdb5daea700 1 -- 172.21.15.47:0/2951423576 >> v1:172.21.15.64:6789/0 conn(0x7fdb580b2bd0 legacy=0x7fdb58099c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:25.516 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.507+0000 7fdb5d2e9700 1 -- 172.21.15.47:0/2951423576 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2078305551 0 0) 0x7fdb40003260 con 0x7fdb58060f20 2022-06-29T15:44:25.516 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.507+0000 7fdb5daea700 1 -- 172.21.15.47:0/2951423576 >> v1:172.21.15.47:6789/0 conn(0x7fdb58060f20 legacy=0x7fdb581403b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:25.516 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.507+0000 7fdb5daea700 1 -- 172.21.15.47:0/2951423576 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb58143c90 con 0x7fdb5805f0b0 2022-06-29T15:44:25.517 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.507+0000 7fdb5daea700 1 --2- 172.21.15.47:0/2951423576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb5805f0b0 0x7fdb580b2fb0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7fdb50001b80 tx=0x7fdb48006df0).ready entity=mon.0 client_cookie=8e0df3f733e247a7 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:25.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.507+0000 7fdb5d2e9700 1 -- 172.21.15.47:0/2951423576 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdb500019e0 con 0x7fdb5805f0b0 2022-06-29T15:44:25.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.507+0000 7fdb5d2e9700 1 -- 172.21.15.47:0/2951423576 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb48000e90 con 0x7fdb5805f0b0 2022-06-29T15:44:25.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.507+0000 7fdb5d2e9700 1 -- 172.21.15.47:0/2951423576 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdb4800d4f0 con 0x7fdb5805f0b0 2022-06-29T15:44:25.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.507+0000 7fdb5f2ed700 1 -- 172.21.15.47:0/2951423576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb5805f0b0 msgr2=0x7fdb580b2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:25.522 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.507+0000 7fdb5f2ed700 1 --2- 172.21.15.47:0/2951423576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb5805f0b0 0x7fdb580b2fb0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7fdb50001b80 tx=0x7fdb48006df0).stop 2022-06-29T15:44:25.522 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.507+0000 7fdb5f2ed700 1 -- 172.21.15.47:0/2951423576 shutdown_connections 2022-06-29T15:44:25.522 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.507+0000 7fdb5f2ed700 1 --2- 172.21.15.47:0/2951423576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb5805f0b0 0x7fdb580b2fb0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:25.522 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.507+0000 7fdb5f2ed700 1 -- 172.21.15.47:0/2951423576 >> 172.21.15.47:0/2951423576 conn(0x7fdb58062df0 msgr2=0x7fdb580631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:25.523 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.507+0000 7fdb5f2ed700 1 -- 172.21.15.47:0/2951423576 shutdown_connections 2022-06-29T15:44:25.523 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.507+0000 7fdb5f2ed700 1 -- 172.21.15.47:0/2951423576 wait complete. 2022-06-29T15:44:25.523 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.511+0000 7fdb5f2ed700 1 Processor -- start 2022-06-29T15:44:25.523 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.511+0000 7fdb5f2ed700 1 -- start start 2022-06-29T15:44:25.524 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.511+0000 7fdb5f2ed700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdb5805f0b0 0x7fdb58094710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:25.524 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.511+0000 7fdb5f2ed700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb58060f20 0x7fdb580961d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:25.524 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.511+0000 7fdb5f2ed700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb580b2bd0 0x7fdb580966c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:25.524 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.511+0000 7fdb5f2ed700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb58146860 con 0x7fdb58060f20 2022-06-29T15:44:25.524 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.511+0000 7fdb5f2ed700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb58146a90 con 0x7fdb580b2bd0 2022-06-29T15:44:25.525 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.511+0000 7fdb5f2ed700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb58146fd0 con 0x7fdb5805f0b0 2022-06-29T15:44:25.525 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.511+0000 7fdb5daea700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb58060f20 0x7fdb580961d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:25.525 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.511+0000 7fdb5daea700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb58060f20 0x7fdb580961d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47748/0 (socket says 172.21.15.47:47748) 2022-06-29T15:44:25.525 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.511+0000 7fdb5daea700 1 -- 172.21.15.47:0/2217608785 learned_addr learned my addr 172.21.15.47:0/2217608785 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:25.526 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.511+0000 7fdb5e2eb700 1 --2- 172.21.15.47:0/2217608785 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdb5805f0b0 0x7fdb58094710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:25.526 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.511+0000 7fdb5eaec700 1 --2- 172.21.15.47:0/2217608785 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb580b2bd0 0x7fdb580966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:25.526 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.511+0000 7fdb5daea700 1 -- 172.21.15.47:0/2217608785 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdb5805f0b0 msgr2=0x7fdb58094710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:25.526 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.511+0000 7fdb5daea700 1 --2- 172.21.15.47:0/2217608785 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdb5805f0b0 0x7fdb58094710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:25.527 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.511+0000 7fdb5daea700 1 -- 172.21.15.47:0/2217608785 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb580b2bd0 msgr2=0x7fdb580966c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:25.527 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.511+0000 7fdb5daea700 1 --2- 172.21.15.47:0/2217608785 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb580b2bd0 0x7fdb580966c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:25.527 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.511+0000 7fdb5daea700 1 -- 172.21.15.47:0/2217608785 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb4800b180 con 0x7fdb58060f20 2022-06-29T15:44:25.527 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.511+0000 7fdb5daea700 1 --2- 172.21.15.47:0/2217608785 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb58060f20 0x7fdb580961d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7fdb50001b80 tx=0x7fdb48014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:25.528 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.511+0000 7fdb477fe700 1 -- 172.21.15.47:0/2217608785 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdb48013de0 con 0x7fdb58060f20 2022-06-29T15:44:25.528 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.511+0000 7fdb477fe700 1 -- 172.21.15.47:0/2217608785 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb48002f50 con 0x7fdb58060f20 2022-06-29T15:44:25.528 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.511+0000 7fdb5f2ed700 1 -- 172.21.15.47:0/2217608785 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb58147270 con 0x7fdb58060f20 2022-06-29T15:44:25.528 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.511+0000 7fdb477fe700 1 -- 172.21.15.47:0/2217608785 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdb4801e5f0 con 0x7fdb58060f20 2022-06-29T15:44:25.529 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.511+0000 7fdb5f2ed700 1 -- 172.21.15.47:0/2217608785 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb580983f0 con 0x7fdb58060f20 2022-06-29T15:44:25.529 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.515+0000 7fdb5f2ed700 1 -- 172.21.15.47:0/2217608785 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb580573d0 con 0x7fdb58060f20 2022-06-29T15:44:25.529 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.515+0000 7fdb477fe700 1 -- 172.21.15.47:0/2217608785 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fdb48025030 con 0x7fdb58060f20 2022-06-29T15:44:25.529 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.515+0000 7fdb477fe700 1 --2- 172.21.15.47:0/2217608785 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdb38053980 0x7fdb38055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:25.530 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.515+0000 7fdb477fe700 1 -- 172.21.15.47:0/2217608785 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fdb4806c8b0 con 0x7fdb58060f20 2022-06-29T15:44:25.530 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.515+0000 7fdb5e2eb700 1 --2- 172.21.15.47:0/2217608785 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdb38053980 0x7fdb38055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:25.530 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.515+0000 7fdb5e2eb700 1 --2- 172.21.15.47:0/2217608785 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdb38053980 0x7fdb38055e40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7fdb5400d000 tx=0x7fdb54007b90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:25.531 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.519+0000 7fdb477fe700 1 -- 172.21.15.47:0/2217608785 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdb480438a0 con 0x7fdb58060f20 2022-06-29T15:44:25.654 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:25.647+0000 7fdb5f2ed700 1 -- 172.21.15.47:0/2217608785 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fdb58146c50 con 0x7fdb58060f20 2022-06-29T15:44:26.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.279+0000 7fdb477fe700 1 -- 172.21.15.47:0/2217608785 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v44) v1 ==== 133+0+0 (secure 0 0 0) 0x7fdb48023070 con 0x7fdb58060f20 2022-06-29T15:44:26.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.279+0000 7fdb5f2ed700 1 -- 172.21.15.47:0/2217608785 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdb38053980 msgr2=0x7fdb38055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:26.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.279+0000 7fdb5f2ed700 1 --2- 172.21.15.47:0/2217608785 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdb38053980 0x7fdb38055e40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7fdb5400d000 tx=0x7fdb54007b90).stop 2022-06-29T15:44:26.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.279+0000 7fdb5f2ed700 1 -- 172.21.15.47:0/2217608785 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb58060f20 msgr2=0x7fdb580961d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:26.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.279+0000 7fdb5f2ed700 1 --2- 172.21.15.47:0/2217608785 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb58060f20 0x7fdb580961d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7fdb50001b80 tx=0x7fdb48014000).stop 2022-06-29T15:44:26.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.283+0000 7fdb5f2ed700 1 -- 172.21.15.47:0/2217608785 shutdown_connections 2022-06-29T15:44:26.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.283+0000 7fdb5f2ed700 1 --2- 172.21.15.47:0/2217608785 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdb38053980 0x7fdb38055e40 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:26.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.283+0000 7fdb5f2ed700 1 --2- 172.21.15.47:0/2217608785 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb580b2bd0 0x7fdb580966c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:26.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.283+0000 7fdb5f2ed700 1 --2- 172.21.15.47:0/2217608785 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb58060f20 0x7fdb580961d0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:26.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.283+0000 7fdb5f2ed700 1 --2- 172.21.15.47:0/2217608785 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdb5805f0b0 0x7fdb58094710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:26.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.283+0000 7fdb5f2ed700 1 -- 172.21.15.47:0/2217608785 >> 172.21.15.47:0/2217608785 conn(0x7fdb58062df0 msgr2=0x7fdb580b0460 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:26.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.283+0000 7fdb5f2ed700 1 -- 172.21.15.47:0/2217608785 shutdown_connections 2022-06-29T15:44:26.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.283+0000 7fdb5f2ed700 1 -- 172.21.15.47:0/2217608785 wait complete. 2022-06-29T15:44:26.290 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' created 2022-06-29T15:44:26.302 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-06-29T15:44:26.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.411+0000 7f60a6bf3700 1 Processor -- start 2022-06-29T15:44:26.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.411+0000 7f60a6bf3700 1 -- start start 2022-06-29T15:44:26.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.415+0000 7f60a6bf3700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f60a0060ee0 0x7f60a00612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:26.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.415+0000 7f60a6bf3700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f60a00b4300 0x7f60a01471a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:26.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.415+0000 7f60a6bf3700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f60a0066f70 0x7f60a01430e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:26.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.415+0000 7f60a6bf3700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60a0067610 con 0x7f60a0066f70 2022-06-29T15:44:26.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.415+0000 7f60a6bf3700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60a00673a0 con 0x7f60a0060ee0 2022-06-29T15:44:26.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.415+0000 7f60a6bf3700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60a01467c0 con 0x7f60a00b4300 2022-06-29T15:44:26.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.415+0000 7f60a5190700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f60a0066f70 0x7f60a01430e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:26.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.415+0000 7f60a5190700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f60a0066f70 0x7f60a01430e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47758/0 (socket says 172.21.15.47:47758) 2022-06-29T15:44:26.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.415+0000 7f60a5190700 1 -- 172.21.15.47:0/972896980 learned_addr learned my addr 172.21.15.47:0/972896980 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:26.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.415+0000 7f60a498f700 1 --2- 172.21.15.47:0/972896980 >> v2:172.21.15.64:3300/0 conn(0x7f60a0060ee0 0x7f60a00612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:26.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.415+0000 7f609ffff700 1 --2- 172.21.15.47:0/972896980 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f60a00b4300 0x7f60a01471a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:26.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.415+0000 7f60a5190700 1 -- 172.21.15.47:0/972896980 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f60a00b4300 msgr2=0x7f60a01471a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:44:26.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.415+0000 7f60a5190700 1 --2- 172.21.15.47:0/972896980 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f60a00b4300 0x7f60a01471a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:26.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.415+0000 7f60a5190700 1 -- 172.21.15.47:0/972896980 >> v2:172.21.15.64:3300/0 conn(0x7f60a0060ee0 msgr2=0x7f60a00612c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:44:26.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.415+0000 7f60a5190700 1 --2- 172.21.15.47:0/972896980 >> v2:172.21.15.64:3300/0 conn(0x7f60a0060ee0 0x7f60a00612c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:26.424 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.415+0000 7f60a5190700 1 -- 172.21.15.47:0/972896980 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60a0146a60 con 0x7f60a0066f70 2022-06-29T15:44:26.424 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.415+0000 7f60a5190700 1 --2- 172.21.15.47:0/972896980 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60a0066f70 0x7f60a01430e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7f60980032f0 tx=0x7f6098009290).ready entity=mon.0 client_cookie=1f6add6fc434b441 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:26.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.415+0000 7f609f7fe700 1 -- 172.21.15.47:0/972896980 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f609800b680 con 0x7f60a0066f70 2022-06-29T15:44:26.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.415+0000 7f609f7fe700 1 -- 172.21.15.47:0/972896980 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f609800b7e0 con 0x7f60a0066f70 2022-06-29T15:44:26.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.415+0000 7f609f7fe700 1 -- 172.21.15.47:0/972896980 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f609800b950 con 0x7f60a0066f70 2022-06-29T15:44:26.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.415+0000 7f60a6bf3700 1 -- 172.21.15.47:0/972896980 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60a0066f70 msgr2=0x7f60a01430e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:26.429 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.415+0000 7f60a6bf3700 1 --2- 172.21.15.47:0/972896980 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60a0066f70 0x7f60a01430e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7f60980032f0 tx=0x7f6098009290).stop 2022-06-29T15:44:26.429 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.415+0000 7f60a6bf3700 1 -- 172.21.15.47:0/972896980 shutdown_connections 2022-06-29T15:44:26.429 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.415+0000 7f60a6bf3700 1 --2- 172.21.15.47:0/972896980 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60a0066f70 0x7f60a01430e0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:26.429 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.415+0000 7f60a6bf3700 1 --2- 172.21.15.47:0/972896980 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f60a00b4300 0x7f60a01471a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:26.430 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.415+0000 7f60a6bf3700 1 --2- 172.21.15.47:0/972896980 >> v2:172.21.15.64:3300/0 conn(0x7f60a0060ee0 0x7f60a00612c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:26.430 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.415+0000 7f60a6bf3700 1 -- 172.21.15.47:0/972896980 >> 172.21.15.47:0/972896980 conn(0x7f60a005c510 msgr2=0x7f60a005f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:26.430 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.415+0000 7f60a6bf3700 1 -- 172.21.15.47:0/972896980 shutdown_connections 2022-06-29T15:44:26.431 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.415+0000 7f60a6bf3700 1 -- 172.21.15.47:0/972896980 wait complete. 2022-06-29T15:44:26.431 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.419+0000 7f60a6bf3700 1 Processor -- start 2022-06-29T15:44:26.431 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.419+0000 7f60a6bf3700 1 -- start start 2022-06-29T15:44:26.431 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.419+0000 7f60a6bf3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60a0060ee0 0x7f60a01457b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:26.431 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.419+0000 7f60a6bf3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60a0066f70 0x7f60a013fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:26.432 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.419+0000 7f60a6bf3700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f60a00b4300 0x7f60a0152250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:26.432 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.419+0000 7f60a6bf3700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a013f3f0 con 0x7f60a0060ee0 2022-06-29T15:44:26.432 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.419+0000 7f60a6bf3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a00b3950 con 0x7f60a0066f70 2022-06-29T15:44:26.432 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.419+0000 7f60a6bf3700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60a00b3bf0 con 0x7f60a00b4300 2022-06-29T15:44:26.433 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.419+0000 7f60a498f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60a0060ee0 0x7f60a01457b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:26.433 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.419+0000 7f60a498f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60a0060ee0 0x7f60a01457b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47760/0 (socket says 172.21.15.47:47760) 2022-06-29T15:44:26.433 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.419+0000 7f60a498f700 1 -- 172.21.15.47:0/456567075 learned_addr learned my addr 172.21.15.47:0/456567075 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:26.433 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.419+0000 7f609ffff700 1 --2- 172.21.15.47:0/456567075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60a0066f70 0x7f60a013fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:26.434 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.423+0000 7f60a5190700 1 --2- 172.21.15.47:0/456567075 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f60a00b4300 0x7f60a0152250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:26.434 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.423+0000 7f60a498f700 1 -- 172.21.15.47:0/456567075 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f60a00b4300 msgr2=0x7f60a0152250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:26.434 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.423+0000 7f60a498f700 1 --2- 172.21.15.47:0/456567075 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f60a00b4300 0x7f60a0152250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:26.434 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.423+0000 7f60a498f700 1 -- 172.21.15.47:0/456567075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60a0066f70 msgr2=0x7f60a013fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:26.435 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.423+0000 7f60a498f700 1 --2- 172.21.15.47:0/456567075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60a0066f70 0x7f60a013fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:26.435 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.423+0000 7f60a498f700 1 -- 172.21.15.47:0/456567075 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6098009530 con 0x7f60a0060ee0 2022-06-29T15:44:26.435 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.423+0000 7f609ffff700 1 --2- 172.21.15.47:0/456567075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60a0066f70 0x7f60a013fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:44:26.435 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.423+0000 7f60a498f700 1 --2- 172.21.15.47:0/456567075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60a0060ee0 0x7f60a01457b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7f609400c000 tx=0x7f6094006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:26.436 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.423+0000 7f609dffb700 1 -- 172.21.15.47:0/456567075 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6094010780 con 0x7f60a0060ee0 2022-06-29T15:44:26.436 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.423+0000 7f609dffb700 1 -- 172.21.15.47:0/456567075 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60940108e0 con 0x7f60a0060ee0 2022-06-29T15:44:26.436 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.423+0000 7f60a6bf3700 1 -- 172.21.15.47:0/456567075 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60a013f620 con 0x7f60a0060ee0 2022-06-29T15:44:26.436 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.423+0000 7f609dffb700 1 -- 172.21.15.47:0/456567075 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6094010c40 con 0x7f60a0060ee0 2022-06-29T15:44:26.436 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.423+0000 7f60a6bf3700 1 -- 172.21.15.47:0/456567075 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60a0152740 con 0x7f60a0060ee0 2022-06-29T15:44:26.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.423+0000 7f60a6bf3700 1 -- 172.21.15.47:0/456567075 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60a00573d0 con 0x7f60a0060ee0 2022-06-29T15:44:26.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.423+0000 7f609dffb700 1 -- 172.21.15.47:0/456567075 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f609401c460 con 0x7f60a0060ee0 2022-06-29T15:44:26.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.423+0000 7f609dffb700 1 --2- 172.21.15.47:0/456567075 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6080053a30 0x7f6080055ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:26.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.423+0000 7f609dffb700 1 -- 172.21.15.47:0/456567075 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5954+0+0 (secure 0 0 0) 0x7f609406c490 con 0x7f60a0060ee0 2022-06-29T15:44:26.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.423+0000 7f609ffff700 1 --2- 172.21.15.47:0/456567075 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6080053a30 0x7f6080055ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:26.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.427+0000 7f609ffff700 1 --2- 172.21.15.47:0/456567075 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6080053a30 0x7f6080055ef0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f608c00b000 tx=0x7f608c008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:26.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.431+0000 7f609dffb700 1 -- 172.21.15.47:0/456567075 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f60940432b0 con 0x7f60a0060ee0 2022-06-29T15:44:26.562 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:26.555+0000 7f60a6bf3700 1 -- 172.21.15.47:0/456567075 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f60a00b5e50 con 0x7f60a0060ee0 2022-06-29T15:44:27.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.291+0000 7f609dffb700 1 -- 172.21.15.47:0/456567075 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v45) v1 ==== 125+0+0 (secure 0 0 0) 0x7f60940468d0 con 0x7f60a0060ee0 2022-06-29T15:44:27.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.295+0000 7f60a6bf3700 1 -- 172.21.15.47:0/456567075 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6080053a30 msgr2=0x7f6080055ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:27.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.295+0000 7f60a6bf3700 1 --2- 172.21.15.47:0/456567075 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6080053a30 0x7f6080055ef0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f608c00b000 tx=0x7f608c008040).stop 2022-06-29T15:44:27.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.295+0000 7f60a6bf3700 1 -- 172.21.15.47:0/456567075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60a0060ee0 msgr2=0x7f60a01457b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:27.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.295+0000 7f60a6bf3700 1 --2- 172.21.15.47:0/456567075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60a0060ee0 0x7f60a01457b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7f609400c000 tx=0x7f6094006df0).stop 2022-06-29T15:44:27.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.295+0000 7f60a6bf3700 1 -- 172.21.15.47:0/456567075 shutdown_connections 2022-06-29T15:44:27.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.295+0000 7f60a6bf3700 1 --2- 172.21.15.47:0/456567075 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f60a00b4300 0x7f60a0152250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:27.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.295+0000 7f60a6bf3700 1 --2- 172.21.15.47:0/456567075 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6080053a30 0x7f6080055ef0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:27.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.295+0000 7f60a6bf3700 1 --2- 172.21.15.47:0/456567075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60a0066f70 0x7f60a013fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:27.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.295+0000 7f60a6bf3700 1 --2- 172.21.15.47:0/456567075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60a0060ee0 0x7f60a01457b0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:27.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.295+0000 7f60a6bf3700 1 -- 172.21.15.47:0/456567075 >> 172.21.15.47:0/456567075 conn(0x7f60a005c510 msgr2=0x7f60a00b8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:27.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.299+0000 7f60a6bf3700 1 -- 172.21.15.47:0/456567075 shutdown_connections 2022-06-29T15:44:27.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.299+0000 7f60a6bf3700 1 -- 172.21.15.47:0/456567075 wait complete. 2022-06-29T15:44:27.306 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' created 2022-06-29T15:44:27.317 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-06-29T15:44:27.430 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.423+0000 7f03c2d16700 1 Processor -- start 2022-06-29T15:44:27.431 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.423+0000 7f03c2d16700 1 -- start start 2022-06-29T15:44:27.432 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.427+0000 7f03c2d16700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f03bc09a2c0 0x7f03bc0b2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:27.432 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.427+0000 7f03c2d16700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f03bc1443a0 0x7f03bc1447f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:27.433 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.427+0000 7f03c2d16700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03bc144ce0 con 0x7f03bc0b2bd0 2022-06-29T15:44:27.433 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.427+0000 7f03c2d16700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03bc0b0020 con 0x7f03bc1443a0 2022-06-29T15:44:27.433 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.427+0000 7f03c2d16700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc144f30 con 0x7f03bc09a2c0 2022-06-29T15:44:27.433 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.427+0000 7f03c2515700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f03bc1443a0 0x7f03bc1447f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:27.434 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.427+0000 7f03c2515700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f03bc1443a0 0x7f03bc1447f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47772/0 (socket says 172.21.15.47:47772) 2022-06-29T15:44:27.434 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.427+0000 7f03c2515700 1 -- 172.21.15.47:0/4074950083 learned_addr learned my addr 172.21.15.47:0/4074950083 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:27.434 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.427+0000 7f03c1513700 1 --2- 172.21.15.47:0/4074950083 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f03bc09a2c0 0x7f03bc0b2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:27.434 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.427+0000 7f03c2515700 1 -- 172.21.15.47:0/4074950083 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f03bc09a2c0 msgr2=0x7f03bc0b2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:44:27.435 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.427+0000 7f03c2515700 1 --2- 172.21.15.47:0/4074950083 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f03bc09a2c0 0x7f03bc0b2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:27.435 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.427+0000 7f03c2515700 1 -- 172.21.15.47:0/4074950083 >> v1:172.21.15.64:6789/0 conn(0x7f03bc0b2bd0 legacy=0x7f03bc098040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:27.435 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.427+0000 7f03c2515700 1 -- 172.21.15.47:0/4074950083 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03bc1451d0 con 0x7f03bc1443a0 2022-06-29T15:44:27.435 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.427+0000 7f03c2515700 1 --2- 172.21.15.47:0/4074950083 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03bc1443a0 0x7f03bc1447f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7f03b4002f70 tx=0x7f03b4006eb0).ready entity=mon.0 client_cookie=e8b4f064db54ab62 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:27.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.427+0000 7f03c0d12700 1 -- 172.21.15.47:0/4074950083 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f03b400d570 con 0x7f03bc1443a0 2022-06-29T15:44:27.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.427+0000 7f03c0d12700 1 -- 172.21.15.47:0/4074950083 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03b4000e90 con 0x7f03bc1443a0 2022-06-29T15:44:27.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.427+0000 7f03c0d12700 1 -- 172.21.15.47:0/4074950083 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f03b400d900 con 0x7f03bc1443a0 2022-06-29T15:44:27.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.427+0000 7f03c2d16700 1 -- 172.21.15.47:0/4074950083 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03bc1443a0 msgr2=0x7f03bc1447f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:27.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.427+0000 7f03c2d16700 1 --2- 172.21.15.47:0/4074950083 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03bc1443a0 0x7f03bc1447f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7f03b4002f70 tx=0x7f03b4006eb0).stop 2022-06-29T15:44:27.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.427+0000 7f03c2d16700 1 -- 172.21.15.47:0/4074950083 shutdown_connections 2022-06-29T15:44:27.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.427+0000 7f03c2d16700 1 --2- 172.21.15.47:0/4074950083 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03bc1443a0 0x7f03bc1447f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:27.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.427+0000 7f03c2d16700 1 --2- 172.21.15.47:0/4074950083 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f03bc09a2c0 0x7f03bc0b2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:27.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.427+0000 7f03c2d16700 1 -- 172.21.15.47:0/4074950083 >> 172.21.15.47:0/4074950083 conn(0x7f03bc062df0 msgr2=0x7f03bc0631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:27.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.427+0000 7f03c2d16700 1 -- 172.21.15.47:0/4074950083 shutdown_connections 2022-06-29T15:44:27.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.427+0000 7f03c2d16700 1 -- 172.21.15.47:0/4074950083 wait complete. 2022-06-29T15:44:27.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.431+0000 7f03c2d16700 1 Processor -- start 2022-06-29T15:44:27.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.431+0000 7f03c2d16700 1 -- start start 2022-06-29T15:44:27.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.431+0000 7f03c2d16700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f03bc09a2c0 0x7f03bc1396d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:27.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.431+0000 7f03c2d16700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03bc0b2bd0 0x7f03bc13dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:27.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.431+0000 7f03c2d16700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03bc1443a0 0x7f03bc13e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:27.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.431+0000 7f03c2d16700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc0af8f0 con 0x7f03bc0b2bd0 2022-06-29T15:44:27.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.431+0000 7f03c2d16700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc099ef0 con 0x7f03bc1443a0 2022-06-29T15:44:27.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.431+0000 7f03c2d16700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc13ed90 con 0x7f03bc09a2c0 2022-06-29T15:44:27.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.431+0000 7f03c1513700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03bc0b2bd0 0x7f03bc13dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:27.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.431+0000 7f03c1513700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03bc0b2bd0 0x7f03bc13dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47776/0 (socket says 172.21.15.47:47776) 2022-06-29T15:44:27.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.431+0000 7f03c1513700 1 -- 172.21.15.47:0/128446496 learned_addr learned my addr 172.21.15.47:0/128446496 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:27.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.431+0000 7f03c1d14700 1 --2- 172.21.15.47:0/128446496 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f03bc09a2c0 0x7f03bc1396d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:27.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.431+0000 7f03c2515700 1 --2- 172.21.15.47:0/128446496 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03bc1443a0 0x7f03bc13e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:27.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.431+0000 7f03c1513700 1 -- 172.21.15.47:0/128446496 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f03bc09a2c0 msgr2=0x7f03bc1396d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:27.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.431+0000 7f03c1513700 1 --2- 172.21.15.47:0/128446496 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f03bc09a2c0 0x7f03bc1396d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:27.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.431+0000 7f03c1513700 1 -- 172.21.15.47:0/128446496 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03bc1443a0 msgr2=0x7f03bc13e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:27.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.431+0000 7f03c1513700 1 --2- 172.21.15.47:0/128446496 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03bc1443a0 0x7f03bc13e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:27.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.431+0000 7f03c1513700 1 -- 172.21.15.47:0/128446496 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b400b1e0 con 0x7f03bc0b2bd0 2022-06-29T15:44:27.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.431+0000 7f03c1513700 1 --2- 172.21.15.47:0/128446496 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03bc0b2bd0 0x7f03bc13dc60 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7f03bc0afa30 tx=0x7f03b0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:27.446 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.431+0000 7f03baffd700 1 -- 172.21.15.47:0/128446496 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f03b0010780 con 0x7f03bc0b2bd0 2022-06-29T15:44:27.446 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.431+0000 7f03baffd700 1 -- 172.21.15.47:0/128446496 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03b00108e0 con 0x7f03bc0b2bd0 2022-06-29T15:44:27.446 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.431+0000 7f03c2d16700 1 -- 172.21.15.47:0/128446496 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03bc13e640 con 0x7f03bc0b2bd0 2022-06-29T15:44:27.446 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.435+0000 7f03baffd700 1 -- 172.21.15.47:0/128446496 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f03b0010c40 con 0x7f03bc0b2bd0 2022-06-29T15:44:27.447 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.435+0000 7f03c2d16700 1 -- 172.21.15.47:0/128446496 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03bc151000 con 0x7f03bc0b2bd0 2022-06-29T15:44:27.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.435+0000 7f03c2d16700 1 -- 172.21.15.47:0/128446496 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03bc141cb0 con 0x7f03bc0b2bd0 2022-06-29T15:44:27.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.435+0000 7f03baffd700 1 -- 172.21.15.47:0/128446496 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f03b001c460 con 0x7f03bc0b2bd0 2022-06-29T15:44:27.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.435+0000 7f03baffd700 1 --2- 172.21.15.47:0/128446496 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f039c053b00 0x7f039c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:27.449 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.435+0000 7f03baffd700 1 -- 172.21.15.47:0/128446496 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f03b006c420 con 0x7f03bc0b2bd0 2022-06-29T15:44:27.449 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.435+0000 7f03c1d14700 1 --2- 172.21.15.47:0/128446496 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f039c053b00 0x7f039c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:27.449 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.439+0000 7f03c1d14700 1 --2- 172.21.15.47:0/128446496 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f039c053b00 0x7f039c055fc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f03ac00d000 tx=0x7f03ac007f10).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:27.450 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.443+0000 7f03baffd700 1 -- 172.21.15.47:0/128446496 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f03b0043160 con 0x7f03bc0b2bd0 2022-06-29T15:44:27.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.611+0000 7f03c2d16700 1 -- 172.21.15.47:0/128446496 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f03bc0573d0 con 0x7f03bc0b2bd0 2022-06-29T15:44:27.627 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.623+0000 7f03baffd700 1 -- 172.21.15.47:0/128446496 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 7 and data pool 8 v16) v1 ==== 187+0+0 (secure 0 0 0) 0x7f03b0046780 con 0x7f03bc0b2bd0 2022-06-29T15:44:27.633 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.627+0000 7f03c2d16700 1 -- 172.21.15.47:0/128446496 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f039c053b00 msgr2=0x7f039c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:27.633 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.627+0000 7f03c2d16700 1 --2- 172.21.15.47:0/128446496 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f039c053b00 0x7f039c055fc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f03ac00d000 tx=0x7f03ac007f10).stop 2022-06-29T15:44:27.633 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.627+0000 7f03c2d16700 1 -- 172.21.15.47:0/128446496 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03bc0b2bd0 msgr2=0x7f03bc13dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:27.633 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.627+0000 7f03c2d16700 1 --2- 172.21.15.47:0/128446496 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03bc0b2bd0 0x7f03bc13dc60 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7f03bc0afa30 tx=0x7f03b0006df0).stop 2022-06-29T15:44:27.634 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.627+0000 7f03c2d16700 1 -- 172.21.15.47:0/128446496 shutdown_connections 2022-06-29T15:44:27.634 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.627+0000 7f03c2d16700 1 --2- 172.21.15.47:0/128446496 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f039c053b00 0x7f039c055fc0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:27.634 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.627+0000 7f03c2d16700 1 --2- 172.21.15.47:0/128446496 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03bc1443a0 0x7f03bc13e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:27.634 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.627+0000 7f03c2d16700 1 --2- 172.21.15.47:0/128446496 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03bc0b2bd0 0x7f03bc13dc60 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:27.635 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.627+0000 7f03c2d16700 1 --2- 172.21.15.47:0/128446496 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f03bc09a2c0 0x7f03bc1396d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:27.635 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.627+0000 7f03c2d16700 1 -- 172.21.15.47:0/128446496 >> 172.21.15.47:0/128446496 conn(0x7f03bc062df0 msgr2=0x7f03bc0b0460 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:27.635 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.627+0000 7f03c2d16700 1 -- 172.21.15.47:0/128446496 shutdown_connections 2022-06-29T15:44:27.635 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.627+0000 7f03c2d16700 1 -- 172.21.15.47:0/128446496 wait complete. 2022-06-29T15:44:27.636 INFO:teuthology.orchestra.run.smithi047.stderr:new fs with metadata pool 7 and data pool 8 2022-06-29T15:44:27.649 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:44:27.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.739+0000 7f1cb201f700 1 Processor -- start 2022-06-29T15:44:27.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.739+0000 7f1cb201f700 1 -- start start 2022-06-29T15:44:27.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.739+0000 7f1cb201f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1cac0630c0 0x7f1cac0b33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:27.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.739+0000 7f1cb201f700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1cac143f10 0x7f1cac0b38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:27.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.739+0000 7f1cb201f700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cac062200 con 0x7f1cac061e20 2022-06-29T15:44:27.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.739+0000 7f1cb201f700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cac05f150 con 0x7f1cac0630c0 2022-06-29T15:44:27.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.739+0000 7f1cb201f700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac05f3f0 con 0x7f1cac143f10 2022-06-29T15:44:27.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.739+0000 7f1cb081c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1cac0630c0 0x7f1cac0b33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:27.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.739+0000 7f1cb081c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1cac0630c0 0x7f1cac0b33b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47784/0 (socket says 172.21.15.47:47784) 2022-06-29T15:44:27.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.739+0000 7f1cb081c700 1 -- 172.21.15.47:0/996822255 learned_addr learned my addr 172.21.15.47:0/996822255 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:27.748 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.739+0000 7f1cb081c700 1 -- 172.21.15.47:0/996822255 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1cac143f10 msgr2=0x7f1cac0b38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:44:27.748 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.739+0000 7f1cb081c700 1 --2- 172.21.15.47:0/996822255 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1cac143f10 0x7f1cac0b38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2022-06-29T15:44:27.748 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.739+0000 7f1cb081c700 1 -- 172.21.15.47:0/996822255 >> v1:172.21.15.47:6789/0 conn(0x7f1cac061e20 legacy=0x7f1cac143800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:27.748 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.739+0000 7f1cb081c700 1 -- 172.21.15.47:0/996822255 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cac05f690 con 0x7f1cac0630c0 2022-06-29T15:44:27.749 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.739+0000 7f1cb081c700 1 --2- 172.21.15.47:0/996822255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1cac0630c0 0x7f1cac0b33b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f1c9c002f70 tx=0x7f1c9c006eb0).ready entity=mon.0 client_cookie=dc8aac32b6375274 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:27.749 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.739+0000 7f1c9bfff700 1 -- 172.21.15.47:0/996822255 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c9c00d5c0 con 0x7f1cac0630c0 2022-06-29T15:44:27.749 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.739+0000 7f1c9bfff700 1 -- 172.21.15.47:0/996822255 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c9c000e90 con 0x7f1cac0630c0 2022-06-29T15:44:27.749 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.739+0000 7f1c9bfff700 1 -- 172.21.15.47:0/996822255 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c9c00d950 con 0x7f1cac0630c0 2022-06-29T15:44:27.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.739+0000 7f1cb201f700 1 -- 172.21.15.47:0/996822255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1cac0630c0 msgr2=0x7f1cac0b33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:27.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.739+0000 7f1cb201f700 1 --2- 172.21.15.47:0/996822255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1cac0630c0 0x7f1cac0b33b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f1c9c002f70 tx=0x7f1c9c006eb0).stop 2022-06-29T15:44:27.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.739+0000 7f1cb201f700 1 -- 172.21.15.47:0/996822255 shutdown_connections 2022-06-29T15:44:27.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.739+0000 7f1cb201f700 1 --2- 172.21.15.47:0/996822255 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1cac143f10 0x7f1cac0b38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2022-06-29T15:44:27.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.739+0000 7f1cb201f700 1 --2- 172.21.15.47:0/996822255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1cac0630c0 0x7f1cac0b33b0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:27.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.739+0000 7f1cb201f700 1 -- 172.21.15.47:0/996822255 >> 172.21.15.47:0/996822255 conn(0x7f1cac063670 msgr2=0x7f1cac063a80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:27.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.739+0000 7f1cb201f700 1 -- 172.21.15.47:0/996822255 shutdown_connections 2022-06-29T15:44:27.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.739+0000 7f1cb201f700 1 -- 172.21.15.47:0/996822255 wait complete. 2022-06-29T15:44:27.755 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.743+0000 7f1cb201f700 1 Processor -- start 2022-06-29T15:44:27.755 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.743+0000 7f1cb201f700 1 -- start start 2022-06-29T15:44:27.755 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.743+0000 7f1cb201f700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1cac061e20 0x7f1cac148ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:27.755 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.743+0000 7f1cb201f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1cac0630c0 0x7f1cac152ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:27.756 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.743+0000 7f1cb201f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cac143f10 0x7f1cac1533c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:27.756 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.743+0000 7f1cb201f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac152840 con 0x7f1cac0630c0 2022-06-29T15:44:27.756 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.743+0000 7f1cb201f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac144ff0 con 0x7f1cac143f10 2022-06-29T15:44:27.756 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.743+0000 7f1cb201f700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac156a40 con 0x7f1cac061e20 2022-06-29T15:44:27.756 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.743+0000 7f1cb081c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1cac0630c0 0x7f1cac152ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:27.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.743+0000 7f1cb081c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1cac0630c0 0x7f1cac152ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47788/0 (socket says 172.21.15.47:47788) 2022-06-29T15:44:27.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.743+0000 7f1cb081c700 1 -- 172.21.15.47:0/1297942016 learned_addr learned my addr 172.21.15.47:0/1297942016 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:27.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.743+0000 7f1cb181e700 1 --2- 172.21.15.47:0/1297942016 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cac143f10 0x7f1cac1533c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:27.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.743+0000 7f1cb101d700 1 --2- 172.21.15.47:0/1297942016 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1cac061e20 0x7f1cac148ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:27.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.743+0000 7f1cb081c700 1 -- 172.21.15.47:0/1297942016 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1cac061e20 msgr2=0x7f1cac148ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:27.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.743+0000 7f1cb081c700 1 --2- 172.21.15.47:0/1297942016 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1cac061e20 0x7f1cac148ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:27.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.743+0000 7f1cb081c700 1 -- 172.21.15.47:0/1297942016 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cac143f10 msgr2=0x7f1cac1533c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:27.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.743+0000 7f1cb081c700 1 --2- 172.21.15.47:0/1297942016 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cac143f10 0x7f1cac1533c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:27.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.743+0000 7f1cb081c700 1 -- 172.21.15.47:0/1297942016 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c9c00b1e0 con 0x7f1cac0630c0 2022-06-29T15:44:27.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.743+0000 7f1cb101d700 1 --2- 172.21.15.47:0/1297942016 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1cac061e20 0x7f1cac148ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:44:27.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.743+0000 7f1cb081c700 1 --2- 172.21.15.47:0/1297942016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1cac0630c0 0x7f1cac152ed0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7f1c9c002f70 tx=0x7f1c9c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:27.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.743+0000 7f1c9a7fc700 1 -- 172.21.15.47:0/1297942016 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c9c006be0 con 0x7f1cac0630c0 2022-06-29T15:44:27.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.743+0000 7f1c9a7fc700 1 -- 172.21.15.47:0/1297942016 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c9c00d940 con 0x7f1cac0630c0 2022-06-29T15:44:27.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.743+0000 7f1cb201f700 1 -- 172.21.15.47:0/1297942016 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cac152a30 con 0x7f1cac0630c0 2022-06-29T15:44:27.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.743+0000 7f1c9a7fc700 1 -- 172.21.15.47:0/1297942016 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c9c01e460 con 0x7f1cac0630c0 2022-06-29T15:44:27.761 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.743+0000 7f1cb201f700 1 -- 172.21.15.47:0/1297942016 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cac1456d0 con 0x7f1cac0630c0 2022-06-29T15:44:27.761 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.747+0000 7f1c9a7fc700 1 -- 172.21.15.47:0/1297942016 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1c9c01a040 con 0x7f1cac0630c0 2022-06-29T15:44:27.761 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.747+0000 7f1cb201f700 1 -- 172.21.15.47:0/1297942016 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cac156c40 con 0x7f1cac0630c0 2022-06-29T15:44:27.761 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.747+0000 7f1c9a7fc700 1 --2- 172.21.15.47:0/1297942016 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1c8c053b00 0x7f1c8c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:27.762 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.747+0000 7f1c9a7fc700 1 -- 172.21.15.47:0/1297942016 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f1c9c06d0d0 con 0x7f1cac0630c0 2022-06-29T15:44:27.762 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.747+0000 7f1cb101d700 1 --2- 172.21.15.47:0/1297942016 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1c8c053b00 0x7f1c8c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:27.762 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.747+0000 7f1cb101d700 1 --2- 172.21.15.47:0/1297942016 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1c8c053b00 0x7f1c8c055fc0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f1ca8008000 tx=0x7f1ca8009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:27.763 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.751+0000 7f1c9a7fc700 1 -- 172.21.15.47:0/1297942016 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1c9c043ef0 con 0x7f1cac0630c0 2022-06-29T15:44:27.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.875+0000 7f1cb201f700 1 -- 172.21.15.47:0/1297942016 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1cac152bc0 con 0x7f1cac0630c0 2022-06-29T15:44:27.884 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.879+0000 7f1c9a7fc700 1 -- 172.21.15.47:0/1297942016 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16284 (secure 0 0 0) 0x7f1c9c047410 con 0x7f1cac0630c0 2022-06-29T15:44:27.884 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:44:27.885 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":46,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:44:27.616367+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:44:25.700617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2022-06-29T15:44:26.610945+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:44:27.886 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.879+0000 7f1cb201f700 1 -- 172.21.15.47:0/1297942016 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1c8c053b00 msgr2=0x7f1c8c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:27.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.879+0000 7f1cb201f700 1 --2- 172.21.15.47:0/1297942016 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1c8c053b00 0x7f1c8c055fc0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f1ca8008000 tx=0x7f1ca8009040).stop 2022-06-29T15:44:27.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.879+0000 7f1cb201f700 1 -- 172.21.15.47:0/1297942016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1cac0630c0 msgr2=0x7f1cac152ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:27.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.879+0000 7f1cb201f700 1 --2- 172.21.15.47:0/1297942016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1cac0630c0 0x7f1cac152ed0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7f1c9c002f70 tx=0x7f1c9c015000).stop 2022-06-29T15:44:27.888 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.879+0000 7f1cb201f700 1 -- 172.21.15.47:0/1297942016 shutdown_connections 2022-06-29T15:44:27.888 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.879+0000 7f1cb201f700 1 --2- 172.21.15.47:0/1297942016 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1c8c053b00 0x7f1c8c055fc0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:27.888 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.879+0000 7f1cb201f700 1 --2- 172.21.15.47:0/1297942016 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cac143f10 0x7f1cac1533c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:27.888 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.879+0000 7f1cb201f700 1 --2- 172.21.15.47:0/1297942016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1cac0630c0 0x7f1cac152ed0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:27.888 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.879+0000 7f1cb201f700 1 --2- 172.21.15.47:0/1297942016 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1cac061e20 0x7f1cac148ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:27.889 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.879+0000 7f1cb201f700 1 -- 172.21.15.47:0/1297942016 >> 172.21.15.47:0/1297942016 conn(0x7f1cac063670 msgr2=0x7f1cac143720 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:27.889 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.879+0000 7f1cb201f700 1 -- 172.21.15.47:0/1297942016 shutdown_connections 2022-06-29T15:44:27.889 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.879+0000 7f1cb201f700 1 -- 172.21.15.47:0/1297942016 wait complete. 2022-06-29T15:44:27.898 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:44:28.003 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.999+0000 7f8d809e8700 1 Processor -- start 2022-06-29T15:44:28.004 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.999+0000 7f8d809e8700 1 -- start start 2022-06-29T15:44:28.006 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.999+0000 7f8d809e8700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8d7c0b5c60 0x7f8d7c144980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:28.006 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.999+0000 7f8d809e8700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8d7c060eb0 0x7f8d7c144e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:28.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.999+0000 7f8d809e8700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d7c0a9120 con 0x7f8d7c0a72f0 2022-06-29T15:44:28.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.999+0000 7f8d809e8700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d7c145360 con 0x7f8d7c060eb0 2022-06-29T15:44:28.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.999+0000 7f8d809e8700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7c0b0b80 con 0x7f8d7c0b5c60 2022-06-29T15:44:28.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.999+0000 7f8d7affd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8d7c060eb0 0x7f8d7c144e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:28.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.999+0000 7f8d7bfff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f8d7c0a72f0 0x7f8d7c0b1e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33354/0 (socket says 172.21.15.47:33354) 2022-06-29T15:44:28.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.999+0000 7f8d7bfff700 1 -- 172.21.15.47:0/3053879292 learned_addr learned my addr 172.21.15.47:0/3053879292 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:44:28.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.999+0000 7f8d7b7fe700 1 --2- 172.21.15.47:0/3053879292 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8d7c0b5c60 0x7f8d7c144980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:28.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.999+0000 7f8d7a7fc700 1 -- 172.21.15.47:0/3053879292 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4285831897 0 0) 0x7f8d7c0a9120 con 0x7f8d7c0a72f0 2022-06-29T15:44:28.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.999+0000 7f8d7affd700 1 -- 172.21.15.47:0/3053879292 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8d7c0b5c60 msgr2=0x7f8d7c144980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:44:28.009 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.999+0000 7f8d7affd700 1 --2- 172.21.15.47:0/3053879292 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8d7c0b5c60 0x7f8d7c144980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:28.009 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.999+0000 7f8d7affd700 1 -- 172.21.15.47:0/3053879292 >> v1:172.21.15.47:6789/0 conn(0x7f8d7c0a72f0 legacy=0x7f8d7c0b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:28.009 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.999+0000 7f8d7affd700 1 -- 172.21.15.47:0/3053879292 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d7c0a89e0 con 0x7f8d7c060eb0 2022-06-29T15:44:28.009 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.999+0000 7f8d7affd700 1 --2- 172.21.15.47:0/3053879292 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d7c060eb0 0x7f8d7c144e70 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7f8d6c002f70 tx=0x7f8d6c006df0).ready entity=mon.0 client_cookie=49b7ad2e4cb7eb9 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:28.010 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.999+0000 7f8d7a7fc700 1 -- 172.21.15.47:0/3053879292 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d6c00d570 con 0x7f8d7c060eb0 2022-06-29T15:44:28.010 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.999+0000 7f8d7a7fc700 1 -- 172.21.15.47:0/3053879292 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d6c000e90 con 0x7f8d7c060eb0 2022-06-29T15:44:28.010 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.999+0000 7f8d7a7fc700 1 -- 172.21.15.47:0/3053879292 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d6c00d8c0 con 0x7f8d7c060eb0 2022-06-29T15:44:28.010 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.999+0000 7f8d809e8700 1 -- 172.21.15.47:0/3053879292 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d7c060eb0 msgr2=0x7f8d7c144e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:28.011 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.999+0000 7f8d809e8700 1 --2- 172.21.15.47:0/3053879292 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d7c060eb0 0x7f8d7c144e70 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7f8d6c002f70 tx=0x7f8d6c006df0).stop 2022-06-29T15:44:28.011 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.999+0000 7f8d809e8700 1 -- 172.21.15.47:0/3053879292 shutdown_connections 2022-06-29T15:44:28.011 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.999+0000 7f8d809e8700 1 --2- 172.21.15.47:0/3053879292 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d7c060eb0 0x7f8d7c144e70 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:28.011 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.999+0000 7f8d809e8700 1 --2- 172.21.15.47:0/3053879292 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8d7c0b5c60 0x7f8d7c144980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:28.011 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.999+0000 7f8d809e8700 1 -- 172.21.15.47:0/3053879292 >> 172.21.15.47:0/3053879292 conn(0x7f8d7c062560 msgr2=0x7f8d7c062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:28.012 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.999+0000 7f8d809e8700 1 -- 172.21.15.47:0/3053879292 shutdown_connections 2022-06-29T15:44:28.012 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:27.999+0000 7f8d809e8700 1 -- 172.21.15.47:0/3053879292 wait complete. 2022-06-29T15:44:28.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.003+0000 7f8d809e8700 1 Processor -- start 2022-06-29T15:44:28.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.003+0000 7f8d809e8700 1 -- start start 2022-06-29T15:44:28.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.003+0000 7f8d809e8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d7c060eb0 0x7f8d7c0b45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:28.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.003+0000 7f8d809e8700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8d7c0a72f0 0x7f8d7c14a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:28.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.003+0000 7f8d809e8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d7c0b5c60 0x7f8d7c14a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:28.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.003+0000 7f8d809e8700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7c14b800 con 0x7f8d7c0b5c60 2022-06-29T15:44:28.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.003+0000 7f8d809e8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7c142600 con 0x7f8d7c060eb0 2022-06-29T15:44:28.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.003+0000 7f8d809e8700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7c0ad330 con 0x7f8d7c0a72f0 2022-06-29T15:44:28.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.003+0000 7f8d7bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d7c0b5c60 0x7f8d7c14a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:28.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.003+0000 7f8d7bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d7c0b5c60 0x7f8d7c14a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47806/0 (socket says 172.21.15.47:47806) 2022-06-29T15:44:28.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.003+0000 7f8d7bfff700 1 -- 172.21.15.47:0/139031129 learned_addr learned my addr 172.21.15.47:0/139031129 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:28.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.003+0000 7f8d7affd700 1 --2- 172.21.15.47:0/139031129 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8d7c0a72f0 0x7f8d7c14a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:28.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.003+0000 7f8d7b7fe700 1 --2- 172.21.15.47:0/139031129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d7c060eb0 0x7f8d7c0b45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:28.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.003+0000 7f8d7bfff700 1 -- 172.21.15.47:0/139031129 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8d7c0a72f0 msgr2=0x7f8d7c14a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:28.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.003+0000 7f8d7bfff700 1 --2- 172.21.15.47:0/139031129 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8d7c0a72f0 0x7f8d7c14a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:28.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.003+0000 7f8d7bfff700 1 -- 172.21.15.47:0/139031129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d7c060eb0 msgr2=0x7f8d7c0b45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:28.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.003+0000 7f8d7bfff700 1 --2- 172.21.15.47:0/139031129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d7c060eb0 0x7f8d7c0b45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:28.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.003+0000 7f8d7bfff700 1 -- 172.21.15.47:0/139031129 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d6c00b180 con 0x7f8d7c0b5c60 2022-06-29T15:44:28.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.003+0000 7f8d7affd700 1 --2- 172.21.15.47:0/139031129 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8d7c0a72f0 0x7f8d7c14a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:44:28.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.003+0000 7f8d7b7fe700 1 --2- 172.21.15.47:0/139031129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d7c060eb0 0x7f8d7c0b45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:44:28.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.003+0000 7f8d7bfff700 1 --2- 172.21.15.47:0/139031129 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d7c0b5c60 0x7f8d7c14a650 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f8d74001250 tx=0x7f8d7400c8a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:28.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.003+0000 7f8d78ff9700 1 -- 172.21.15.47:0/139031129 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d74005ba0 con 0x7f8d7c0b5c60 2022-06-29T15:44:28.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.003+0000 7f8d78ff9700 1 -- 172.21.15.47:0/139031129 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d74005d00 con 0x7f8d7c0b5c60 2022-06-29T15:44:28.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.003+0000 7f8d809e8700 1 -- 172.21.15.47:0/139031129 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d7c0ae380 con 0x7f8d7c0b5c60 2022-06-29T15:44:28.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.003+0000 7f8d78ff9700 1 -- 172.21.15.47:0/139031129 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d74004a90 con 0x7f8d7c0b5c60 2022-06-29T15:44:28.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.003+0000 7f8d809e8700 1 -- 172.21.15.47:0/139031129 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d7c0ad590 con 0x7f8d7c0b5c60 2022-06-29T15:44:28.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.003+0000 7f8d809e8700 1 -- 172.21.15.47:0/139031129 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d7c0ad750 con 0x7f8d7c0b5c60 2022-06-29T15:44:28.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.007+0000 7f8d78ff9700 1 -- 172.21.15.47:0/139031129 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8d74004bf0 con 0x7f8d7c0b5c60 2022-06-29T15:44:28.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.007+0000 7f8d78ff9700 1 --2- 172.21.15.47:0/139031129 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8d600539e0 0x7f8d60055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:28.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.007+0000 7f8d78ff9700 1 -- 172.21.15.47:0/139031129 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f8d7406bc60 con 0x7f8d7c0b5c60 2022-06-29T15:44:28.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.007+0000 7f8d7b7fe700 1 --2- 172.21.15.47:0/139031129 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8d600539e0 0x7f8d60055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:28.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.007+0000 7f8d7b7fe700 1 --2- 172.21.15.47:0/139031129 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8d600539e0 0x7f8d60055ea0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f8d64008000 tx=0x7f8d64008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:28.022 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.011+0000 7f8d78ff9700 1 -- 172.21.15.47:0/139031129 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8d740429c0 con 0x7f8d7c0b5c60 2022-06-29T15:44:28.144 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.139+0000 7f8d809e8700 1 -- 172.21.15.47:0/139031129 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8d7c0573d0 con 0x7f8d7c0b5c60 2022-06-29T15:44:28.146 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.139+0000 7f8d78ff9700 1 -- 172.21.15.47:0/139031129 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16284 (secure 0 0 0) 0x7f8d74025fb0 con 0x7f8d7c0b5c60 2022-06-29T15:44:28.146 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:44:28.147 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":46,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:44:27.616367+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:44:25.700617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2022-06-29T15:44:26.610945+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:44:28.149 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.143+0000 7f8d809e8700 1 -- 172.21.15.47:0/139031129 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8d600539e0 msgr2=0x7f8d60055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:28.149 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.143+0000 7f8d809e8700 1 --2- 172.21.15.47:0/139031129 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8d600539e0 0x7f8d60055ea0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f8d64008000 tx=0x7f8d64008040).stop 2022-06-29T15:44:28.149 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.143+0000 7f8d809e8700 1 -- 172.21.15.47:0/139031129 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d7c0b5c60 msgr2=0x7f8d7c14a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:28.149 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.143+0000 7f8d809e8700 1 --2- 172.21.15.47:0/139031129 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d7c0b5c60 0x7f8d7c14a650 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f8d74001250 tx=0x7f8d7400c8a0).stop 2022-06-29T15:44:28.150 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.143+0000 7f8d809e8700 1 -- 172.21.15.47:0/139031129 shutdown_connections 2022-06-29T15:44:28.150 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.143+0000 7f8d809e8700 1 --2- 172.21.15.47:0/139031129 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d7c0b5c60 0x7f8d7c14a650 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:28.150 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.143+0000 7f8d809e8700 1 --2- 172.21.15.47:0/139031129 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8d7c0a72f0 0x7f8d7c14a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:28.150 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.143+0000 7f8d809e8700 1 --2- 172.21.15.47:0/139031129 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8d600539e0 0x7f8d60055ea0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:28.151 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.143+0000 7f8d809e8700 1 --2- 172.21.15.47:0/139031129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d7c060eb0 0x7f8d7c0b45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:28.151 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.143+0000 7f8d809e8700 1 -- 172.21.15.47:0/139031129 >> 172.21.15.47:0/139031129 conn(0x7f8d7c062560 msgr2=0x7f8d7c0afae0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:28.151 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.143+0000 7f8d809e8700 1 -- 172.21.15.47:0/139031129 shutdown_connections 2022-06-29T15:44:28.151 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.143+0000 7f8d809e8700 1 -- 172.21.15.47:0/139031129 wait complete. 2022-06-29T15:44:28.160 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-06-29T15:44:28.265 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.259+0000 7f0761913700 1 Processor -- start 2022-06-29T15:44:28.265 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.259+0000 7f0761913700 1 -- start start 2022-06-29T15:44:28.268 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.259+0000 7f0761913700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f075c05f0b0 0x7f075c09ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:28.268 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.259+0000 7f0761913700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f075c0af250 0x7f075c0af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:28.268 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.259+0000 7f0761913700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f075c09b970 con 0x7f075c0605d0 2022-06-29T15:44:28.269 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.259+0000 7f0761913700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f075c05fbe0 con 0x7f075c0af250 2022-06-29T15:44:28.269 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.259+0000 7f0761913700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f075c0acfd0 con 0x7f075c05f0b0 2022-06-29T15:44:28.269 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.259+0000 7f0761112700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f075c0af250 0x7f075c0af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:28.269 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.259+0000 7f0761112700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f075c0af250 0x7f075c0af6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47814/0 (socket says 172.21.15.47:47814) 2022-06-29T15:44:28.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.259+0000 7f0761112700 1 -- 172.21.15.47:0/2811288854 learned_addr learned my addr 172.21.15.47:0/2811288854 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:28.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.259+0000 7f075bfff700 1 --2- 172.21.15.47:0/2811288854 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f075c05f0b0 0x7f075c09ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:28.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.259+0000 7f0761112700 1 -- 172.21.15.47:0/2811288854 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f075c05f0b0 msgr2=0x7f075c09ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:44:28.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.259+0000 7f0761112700 1 --2- 172.21.15.47:0/2811288854 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f075c05f0b0 0x7f075c09ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:28.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.259+0000 7f0761112700 1 -- 172.21.15.47:0/2811288854 >> v1:172.21.15.64:6789/0 conn(0x7f075c0605d0 legacy=0x7f075c09a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:28.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.259+0000 7f0761112700 1 -- 172.21.15.47:0/2811288854 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f075c05fe80 con 0x7f075c0af250 2022-06-29T15:44:28.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.263+0000 7f0761112700 1 --2- 172.21.15.47:0/2811288854 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f075c0af250 0x7f075c0af6c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7f07540032f0 tx=0x7f075400b530).ready entity=mon.0 client_cookie=dd9ce8475a3b2efa server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:28.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.263+0000 7f075b7fe700 1 -- 172.21.15.47:0/2811288854 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f075400de20 con 0x7f075c0af250 2022-06-29T15:44:28.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.263+0000 7f075b7fe700 1 -- 172.21.15.47:0/2811288854 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0754010470 con 0x7f075c0af250 2022-06-29T15:44:28.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.263+0000 7f075b7fe700 1 -- 172.21.15.47:0/2811288854 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07540107d0 con 0x7f075c0af250 2022-06-29T15:44:28.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.263+0000 7f0761913700 1 -- 172.21.15.47:0/2811288854 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f075c0af250 msgr2=0x7f075c0af6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:28.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.263+0000 7f0761913700 1 --2- 172.21.15.47:0/2811288854 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f075c0af250 0x7f075c0af6c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7f07540032f0 tx=0x7f075400b530).stop 2022-06-29T15:44:28.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.263+0000 7f0761913700 1 -- 172.21.15.47:0/2811288854 shutdown_connections 2022-06-29T15:44:28.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.263+0000 7f0761913700 1 --2- 172.21.15.47:0/2811288854 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f075c0af250 0x7f075c0af6c0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:28.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.263+0000 7f0761913700 1 --2- 172.21.15.47:0/2811288854 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f075c05f0b0 0x7f075c09ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:28.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.263+0000 7f0761913700 1 -- 172.21.15.47:0/2811288854 >> 172.21.15.47:0/2811288854 conn(0x7f075c062df0 msgr2=0x7f075c0631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:28.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.263+0000 7f0761913700 1 -- 172.21.15.47:0/2811288854 shutdown_connections 2022-06-29T15:44:28.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.263+0000 7f0761913700 1 -- 172.21.15.47:0/2811288854 wait complete. 2022-06-29T15:44:28.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.263+0000 7f0761913700 1 Processor -- start 2022-06-29T15:44:28.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.263+0000 7f0761913700 1 -- start start 2022-06-29T15:44:28.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.267+0000 7f0761913700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f075c05f0b0 0x7f075c0b5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:28.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.267+0000 7f0761913700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f075c0605d0 0x7f075c093610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:28.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.267+0000 7f0761913700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f075c0af250 0x7f075c095b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:28.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.267+0000 7f0761913700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f075c094d30 con 0x7f075c0605d0 2022-06-29T15:44:28.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.267+0000 7f0761913700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f075c0675d0 con 0x7f075c0af250 2022-06-29T15:44:28.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.267+0000 7f0761913700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f075c093460 con 0x7f075c05f0b0 2022-06-29T15:44:28.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.267+0000 7f075bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f075c0605d0 0x7f075c093610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:28.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.267+0000 7f075bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f075c0605d0 0x7f075c093610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47818/0 (socket says 172.21.15.47:47818) 2022-06-29T15:44:28.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.267+0000 7f075bfff700 1 -- 172.21.15.47:0/569369105 learned_addr learned my addr 172.21.15.47:0/569369105 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:28.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.267+0000 7f0760911700 1 --2- 172.21.15.47:0/569369105 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f075c05f0b0 0x7f075c0b5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:28.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.267+0000 7f0761112700 1 --2- 172.21.15.47:0/569369105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f075c0af250 0x7f075c095b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:28.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.267+0000 7f075bfff700 1 -- 172.21.15.47:0/569369105 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f075c05f0b0 msgr2=0x7f075c0b5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:28.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.267+0000 7f075bfff700 1 --2- 172.21.15.47:0/569369105 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f075c05f0b0 0x7f075c0b5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:28.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.267+0000 7f075bfff700 1 -- 172.21.15.47:0/569369105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f075c0af250 msgr2=0x7f075c095b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:28.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.267+0000 7f075bfff700 1 --2- 172.21.15.47:0/569369105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f075c0af250 0x7f075c095b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:28.282 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.267+0000 7f075bfff700 1 -- 172.21.15.47:0/569369105 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f075400b7d0 con 0x7f075c0605d0 2022-06-29T15:44:28.282 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.267+0000 7f075bfff700 1 --2- 172.21.15.47:0/569369105 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f075c0605d0 0x7f075c093610 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7f075c097f30 tx=0x7f074c006d90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:28.282 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.267+0000 7f0759ffb700 1 -- 172.21.15.47:0/569369105 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f074c0104c0 con 0x7f075c0605d0 2022-06-29T15:44:28.283 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.267+0000 7f0759ffb700 1 -- 172.21.15.47:0/569369105 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f074c010620 con 0x7f075c0605d0 2022-06-29T15:44:28.283 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.267+0000 7f0761913700 1 -- 172.21.15.47:0/569369105 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f075c092f20 con 0x7f075c0605d0 2022-06-29T15:44:28.283 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.267+0000 7f0759ffb700 1 -- 172.21.15.47:0/569369105 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f074c010980 con 0x7f075c0605d0 2022-06-29T15:44:28.283 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.267+0000 7f0761913700 1 -- 172.21.15.47:0/569369105 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f075c0956c0 con 0x7f075c0605d0 2022-06-29T15:44:28.283 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.271+0000 7f0761913700 1 -- 172.21.15.47:0/569369105 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f075c0573d0 con 0x7f075c0605d0 2022-06-29T15:44:28.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.271+0000 7f0759ffb700 1 -- 172.21.15.47:0/569369105 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f074c01a030 con 0x7f075c0605d0 2022-06-29T15:44:28.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.271+0000 7f0759ffb700 1 --2- 172.21.15.47:0/569369105 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f073c053a70 0x7f073c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:28.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.271+0000 7f0759ffb700 1 -- 172.21.15.47:0/569369105 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f074c06bb00 con 0x7f075c0605d0 2022-06-29T15:44:28.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.271+0000 7f0760911700 1 --2- 172.21.15.47:0/569369105 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f073c053a70 0x7f073c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:28.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.271+0000 7f0760911700 1 --2- 172.21.15.47:0/569369105 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f073c053a70 0x7f073c055f30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f075000d000 tx=0x7f0750007b90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:28.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.279+0000 7f0759ffb700 1 -- 172.21.15.47:0/569369105 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f074c042910 con 0x7f075c0605d0 2022-06-29T15:44:28.452 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.447+0000 7f0761913700 1 -- 172.21.15.47:0/569369105 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f075c060210 con 0x7f075c0605d0 2022-06-29T15:44:28.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.635+0000 7f0759ffb700 1 -- 172.21.15.47:0/569369105 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7f074c045f30 con 0x7f075c0605d0 2022-06-29T15:44:28.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.639+0000 7f0761913700 1 -- 172.21.15.47:0/569369105 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f073c053a70 msgr2=0x7f073c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:28.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.639+0000 7f0761913700 1 --2- 172.21.15.47:0/569369105 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f073c053a70 0x7f073c055f30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f075000d000 tx=0x7f0750007b90).stop 2022-06-29T15:44:28.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.639+0000 7f0761913700 1 -- 172.21.15.47:0/569369105 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f075c0605d0 msgr2=0x7f075c093610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:28.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.639+0000 7f0761913700 1 --2- 172.21.15.47:0/569369105 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f075c0605d0 0x7f075c093610 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7f075c097f30 tx=0x7f074c006d90).stop 2022-06-29T15:44:28.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.639+0000 7f0761913700 1 -- 172.21.15.47:0/569369105 shutdown_connections 2022-06-29T15:44:28.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.639+0000 7f0761913700 1 --2- 172.21.15.47:0/569369105 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f073c053a70 0x7f073c055f30 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:28.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.639+0000 7f0761913700 1 --2- 172.21.15.47:0/569369105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f075c0af250 0x7f075c095b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:28.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.639+0000 7f0761913700 1 --2- 172.21.15.47:0/569369105 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f075c0605d0 0x7f075c093610 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:28.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.639+0000 7f0761913700 1 --2- 172.21.15.47:0/569369105 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f075c05f0b0 0x7f075c0b5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:28.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.639+0000 7f0761913700 1 -- 172.21.15.47:0/569369105 >> 172.21.15.47:0/569369105 conn(0x7f075c062df0 msgr2=0x7f075c0aea50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:28.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.639+0000 7f0761913700 1 -- 172.21.15.47:0/569369105 shutdown_connections 2022-06-29T15:44:28.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.639+0000 7f0761913700 1 -- 172.21.15.47:0/569369105 wait complete. 2022-06-29T15:44:28.660 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:44:28.767 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.763+0000 7f40f76ce700 1 Processor -- start 2022-06-29T15:44:28.767 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.763+0000 7f40f76ce700 1 -- start start 2022-06-29T15:44:28.769 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.763+0000 7f40f76ce700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f40f0060450 0x7f40f0142820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:28.769 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.763+0000 7f40f76ce700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40f00b35a0 con 0x7f40f00b0fe0 2022-06-29T15:44:28.770 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.763+0000 7f40f76ce700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40f0098310 con 0x7f40f0097f30 2022-06-29T15:44:28.770 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.763+0000 7f40f76ce700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40f00b2790 con 0x7f40f0060450 2022-06-29T15:44:28.770 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.763+0000 7f40f5c6b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f40f00b0fe0 0x7f40f0143d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33382/0 (socket says 172.21.15.47:33382) 2022-06-29T15:44:28.770 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.763+0000 7f40f5c6b700 1 -- 172.21.15.47:0/2389755458 learned_addr learned my addr 172.21.15.47:0/2389755458 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:44:28.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.763+0000 7f40f546a700 1 --1- 172.21.15.47:0/2389755458 >> v1:172.21.15.64:6789/0 conn(0x7f40f0097f30 0x7f40f00b2e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:40942/0 (socket says 172.21.15.47:40942) 2022-06-29T15:44:28.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.763+0000 7f40f4c69700 1 --2- 172.21.15.47:0/2389755458 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f40f0060450 0x7f40f0142820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:28.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.763+0000 7f40dffff700 1 -- 172.21.15.47:0/2389755458 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1659633728 0 0) 0x7f40f00b35a0 con 0x7f40f00b0fe0 2022-06-29T15:44:28.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.763+0000 7f40dffff700 1 -- 172.21.15.47:0/2389755458 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40d8003260 con 0x7f40f00b0fe0 2022-06-29T15:44:28.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.763+0000 7f40dffff700 1 -- 172.21.15.47:0/2389755458 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3727588320 0 0) 0x7f40f0098310 con 0x7f40f0097f30 2022-06-29T15:44:28.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.763+0000 7f40dffff700 1 -- 172.21.15.47:0/2389755458 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40f00b35a0 con 0x7f40f0097f30 2022-06-29T15:44:28.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.763+0000 7f40f4c69700 1 -- 172.21.15.47:0/2389755458 >> v1:172.21.15.64:6789/0 conn(0x7f40f0097f30 legacy=0x7f40f00b2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:28.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.763+0000 7f40f4c69700 1 -- 172.21.15.47:0/2389755458 >> v1:172.21.15.47:6789/0 conn(0x7f40f00b0fe0 legacy=0x7f40f0143d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:28.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.763+0000 7f40f4c69700 1 -- 172.21.15.47:0/2389755458 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40f0061310 con 0x7f40f0060450 2022-06-29T15:44:28.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.763+0000 7f40f4c69700 1 --2- 172.21.15.47:0/2389755458 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f40f0060450 0x7f40f0142820 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f40e0002f70 tx=0x7f40e0006df0).ready entity=mon.2 client_cookie=c5622b974ab2399a server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:28.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.767+0000 7f40dffff700 1 -- 172.21.15.47:0/2389755458 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40e000d5c0 con 0x7f40f0060450 2022-06-29T15:44:28.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.767+0000 7f40dffff700 1 -- 172.21.15.47:0/2389755458 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40e0000e90 con 0x7f40f0060450 2022-06-29T15:44:28.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.767+0000 7f40dffff700 1 -- 172.21.15.47:0/2389755458 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40e000d910 con 0x7f40f0060450 2022-06-29T15:44:28.776 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.767+0000 7f40f76ce700 1 -- 172.21.15.47:0/2389755458 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f40f0060450 msgr2=0x7f40f0142820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:28.776 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.767+0000 7f40f76ce700 1 --2- 172.21.15.47:0/2389755458 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f40f0060450 0x7f40f0142820 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f40e0002f70 tx=0x7f40e0006df0).stop 2022-06-29T15:44:28.776 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.767+0000 7f40f76ce700 1 -- 172.21.15.47:0/2389755458 shutdown_connections 2022-06-29T15:44:28.776 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.767+0000 7f40f76ce700 1 --2- 172.21.15.47:0/2389755458 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f40f0060450 0x7f40f0142820 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:28.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.767+0000 7f40f76ce700 1 -- 172.21.15.47:0/2389755458 >> 172.21.15.47:0/2389755458 conn(0x7f40f005f0b0 msgr2=0x7f40f005f490 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:28.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.767+0000 7f40f76ce700 1 -- 172.21.15.47:0/2389755458 shutdown_connections 2022-06-29T15:44:28.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.767+0000 7f40f76ce700 1 -- 172.21.15.47:0/2389755458 wait complete. 2022-06-29T15:44:28.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.767+0000 7f40f76ce700 1 Processor -- start 2022-06-29T15:44:28.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.767+0000 7f40f76ce700 1 -- start start 2022-06-29T15:44:28.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.771+0000 7f40f76ce700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40f0060450 0x7f40f00aaa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:28.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.771+0000 7f40f76ce700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40f0097f30 0x7f40f00ac4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:28.781 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.771+0000 7f40f76ce700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f40f00b0fe0 0x7f40f00ac9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:28.781 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.771+0000 7f40f76ce700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f00648c0 con 0x7f40f0060450 2022-06-29T15:44:28.781 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.771+0000 7f40f76ce700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f00abdf0 con 0x7f40f0097f30 2022-06-29T15:44:28.781 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.771+0000 7f40f76ce700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40f00ab990 con 0x7f40f00b0fe0 2022-06-29T15:44:28.782 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.771+0000 7f40f546a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40f0060450 0x7f40f00aaa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:28.782 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.771+0000 7f40f546a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40f0060450 0x7f40f00aaa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47830/0 (socket says 172.21.15.47:47830) 2022-06-29T15:44:28.782 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.771+0000 7f40f546a700 1 -- 172.21.15.47:0/2643455065 learned_addr learned my addr 172.21.15.47:0/2643455065 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:28.782 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.771+0000 7f40f4c69700 1 --2- 172.21.15.47:0/2643455065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40f0097f30 0x7f40f00ac4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:28.783 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.771+0000 7f40f5c6b700 1 --2- 172.21.15.47:0/2643455065 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f40f00b0fe0 0x7f40f00ac9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:28.783 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.771+0000 7f40f546a700 1 -- 172.21.15.47:0/2643455065 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f40f00b0fe0 msgr2=0x7f40f00ac9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:28.783 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.771+0000 7f40f546a700 1 --2- 172.21.15.47:0/2643455065 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f40f00b0fe0 0x7f40f00ac9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:28.783 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.771+0000 7f40f546a700 1 -- 172.21.15.47:0/2643455065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40f0097f30 msgr2=0x7f40f00ac4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:28.784 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.771+0000 7f40f546a700 1 --2- 172.21.15.47:0/2643455065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40f0097f30 0x7f40f00ac4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:28.784 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.771+0000 7f40f546a700 1 -- 172.21.15.47:0/2643455065 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e000b180 con 0x7f40f0060450 2022-06-29T15:44:28.784 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.771+0000 7f40f546a700 1 --2- 172.21.15.47:0/2643455065 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40f0060450 0x7f40f00aaa30 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f40f00a9f90 tx=0x7f40ec007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:28.784 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.771+0000 7f40de7fc700 1 -- 172.21.15.47:0/2643455065 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40ec00ce10 con 0x7f40f0060450 2022-06-29T15:44:28.784 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.771+0000 7f40de7fc700 1 -- 172.21.15.47:0/2643455065 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40ec0072e0 con 0x7f40f0060450 2022-06-29T15:44:28.785 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.771+0000 7f40de7fc700 1 -- 172.21.15.47:0/2643455065 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40ec005740 con 0x7f40f0060450 2022-06-29T15:44:28.785 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.771+0000 7f40f76ce700 1 -- 172.21.15.47:0/2643455065 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40f00ab6f0 con 0x7f40f0060450 2022-06-29T15:44:28.785 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.771+0000 7f40f76ce700 1 -- 172.21.15.47:0/2643455065 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40f0065aa0 con 0x7f40f0060450 2022-06-29T15:44:28.785 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.771+0000 7f40f76ce700 1 -- 172.21.15.47:0/2643455065 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40f0063d70 con 0x7f40f0060450 2022-06-29T15:44:28.786 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.775+0000 7f40de7fc700 1 -- 172.21.15.47:0/2643455065 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f40ec010070 con 0x7f40f0060450 2022-06-29T15:44:28.786 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.775+0000 7f40de7fc700 1 --2- 172.21.15.47:0/2643455065 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f40d00539e0 0x7f40d0055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:28.786 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.775+0000 7f40de7fc700 1 -- 172.21.15.47:0/2643455065 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f40ec06b710 con 0x7f40f0060450 2022-06-29T15:44:28.786 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.775+0000 7f40f4c69700 1 --2- 172.21.15.47:0/2643455065 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f40d00539e0 0x7f40d0055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:28.787 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.775+0000 7f40f4c69700 1 --2- 172.21.15.47:0/2643455065 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f40d00539e0 0x7f40d0055ea0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f40e001a000 tx=0x7f40e0014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:28.787 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.779+0000 7f40de7fc700 1 -- 172.21.15.47:0/2643455065 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f40f0063d70 con 0x7f40f0060450 2022-06-29T15:44:28.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.943+0000 7f40f76ce700 1 -- 172.21.15.47:0/2643455065 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f40f0063d70 con 0x7f40f0060450 2022-06-29T15:44:28.950 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.943+0000 7f40de7fc700 1 -- 172.21.15.47:0/2643455065 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+2491 (secure 0 0 0) 0x7f40f0063d70 con 0x7f40f0060450 2022-06-29T15:44:28.951 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:44:28.951 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":17,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5259,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3601815346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3601815346},{"type":"v1","addr":"172.21.15.47:6835","nonce":3601815346}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15},{"gid":5262,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/657177602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":657177602},{"type":"v1","addr":"172.21.15.47:6837","nonce":657177602}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:44:27.616735+0000","modified":"2022-06-29T15:44:28.620601+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":54812},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54812":{"gid":54812,"name":"b","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6833/2497658100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2497658100},{"type":"v1","addr":"172.21.15.64:6833","nonce":2497658100}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2022-06-29T15:44:28.953 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.947+0000 7f40f76ce700 1 -- 172.21.15.47:0/2643455065 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f40d00539e0 msgr2=0x7f40d0055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:28.953 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.947+0000 7f40f76ce700 1 --2- 172.21.15.47:0/2643455065 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f40d00539e0 0x7f40d0055ea0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f40e001a000 tx=0x7f40e0014000).stop 2022-06-29T15:44:28.953 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.947+0000 7f40f76ce700 1 -- 172.21.15.47:0/2643455065 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40f0060450 msgr2=0x7f40f00aaa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:28.954 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.947+0000 7f40f76ce700 1 --2- 172.21.15.47:0/2643455065 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40f0060450 0x7f40f00aaa30 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f40f00a9f90 tx=0x7f40ec007ad0).stop 2022-06-29T15:44:28.954 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.947+0000 7f40f76ce700 1 -- 172.21.15.47:0/2643455065 shutdown_connections 2022-06-29T15:44:28.954 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.947+0000 7f40f76ce700 1 --2- 172.21.15.47:0/2643455065 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f40f00b0fe0 0x7f40f00ac9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:28.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.947+0000 7f40f76ce700 1 --2- 172.21.15.47:0/2643455065 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f40d00539e0 0x7f40d0055ea0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:28.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.947+0000 7f40f76ce700 1 --2- 172.21.15.47:0/2643455065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40f0097f30 0x7f40f00ac4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:28.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.947+0000 7f40f76ce700 1 --2- 172.21.15.47:0/2643455065 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40f0060450 0x7f40f00aaa30 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:28.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.947+0000 7f40f76ce700 1 -- 172.21.15.47:0/2643455065 >> 172.21.15.47:0/2643455065 conn(0x7f40f005f0b0 msgr2=0x7f40f01446a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:28.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.947+0000 7f40f76ce700 1 -- 172.21.15.47:0/2643455065 shutdown_connections 2022-06-29T15:44:28.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:28.947+0000 7f40f76ce700 1 -- 172.21.15.47:0/2643455065 wait complete. 2022-06-29T15:44:28.958 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 17 2022-06-29T15:44:28.965 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:44:29.070 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.063+0000 7fe5dc571700 1 Processor -- start 2022-06-29T15:44:29.071 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.063+0000 7fe5dc571700 1 -- start start 2022-06-29T15:44:29.072 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.067+0000 7fe5dc571700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe5d40b0cf0 0x7fe5d4146d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:29.072 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.067+0000 7fe5dc571700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5d40b4520 con 0x7fe5d40611b0 2022-06-29T15:44:29.072 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.067+0000 7fe5dc571700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5d4061590 con 0x7fe5d40b06d0 2022-06-29T15:44:29.072 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.067+0000 7fe5dc571700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5d40b4b10 con 0x7fe5d40b0cf0 2022-06-29T15:44:29.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.067+0000 7fe5da30d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe5d40611b0 0x7fe5d4146640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33392/0 (socket says 172.21.15.47:33392) 2022-06-29T15:44:29.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.067+0000 7fe5da30d700 1 -- 172.21.15.47:0/3234739006 learned_addr learned my addr 172.21.15.47:0/3234739006 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:44:29.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.067+0000 7fe5d9b0c700 1 --2- 172.21.15.47:0/3234739006 >> v2:172.21.15.64:3300/0 conn(0x7fe5d40b0cf0 0x7fe5d4146d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:29.075 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.067+0000 7fe5d9b0c700 1 -- 172.21.15.47:0/3234739006 >> v1:172.21.15.64:6789/0 conn(0x7fe5d40b06d0 legacy=0x7fe5d413ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:29.075 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.067+0000 7fe5d9b0c700 1 -- 172.21.15.47:0/3234739006 >> v1:172.21.15.47:6789/0 conn(0x7fe5d40611b0 legacy=0x7fe5d4146640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:29.075 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.067+0000 7fe5d9b0c700 1 -- 172.21.15.47:0/3234739006 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d40b5e00 con 0x7fe5d40b0cf0 2022-06-29T15:44:29.076 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.067+0000 7fe5d930b700 1 -- 172.21.15.47:0/3234739006 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2489077735 0 0) 0x7fe5d40b4520 con 0x7fe5d40611b0 2022-06-29T15:44:29.076 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.067+0000 7fe5d9b0c700 1 --2- 172.21.15.47:0/3234739006 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe5d40b0cf0 0x7fe5d4146d50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7fe5d40b1c00 tx=0x7fe5d0006df0).ready entity=mon.1 client_cookie=86169bc75442319b server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:29.076 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.067+0000 7fe5d930b700 1 -- 172.21.15.47:0/3234739006 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5d000d610 con 0x7fe5d40b0cf0 2022-06-29T15:44:29.076 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.067+0000 7fe5d930b700 1 -- 172.21.15.47:0/3234739006 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d0000e90 con 0x7fe5d40b0cf0 2022-06-29T15:44:29.077 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.067+0000 7fe5d930b700 1 -- 172.21.15.47:0/3234739006 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5d000d960 con 0x7fe5d40b0cf0 2022-06-29T15:44:29.077 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.067+0000 7fe5dc571700 1 -- 172.21.15.47:0/3234739006 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe5d40b0cf0 msgr2=0x7fe5d4146d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:29.077 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.067+0000 7fe5dc571700 1 --2- 172.21.15.47:0/3234739006 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe5d40b0cf0 0x7fe5d4146d50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7fe5d40b1c00 tx=0x7fe5d0006df0).stop 2022-06-29T15:44:29.077 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.067+0000 7fe5dc571700 1 -- 172.21.15.47:0/3234739006 shutdown_connections 2022-06-29T15:44:29.078 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.067+0000 7fe5dc571700 1 --2- 172.21.15.47:0/3234739006 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe5d40b0cf0 0x7fe5d4146d50 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:29.078 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.067+0000 7fe5dc571700 1 -- 172.21.15.47:0/3234739006 >> 172.21.15.47:0/3234739006 conn(0x7fe5d40b98b0 msgr2=0x7fe5d40b9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:29.078 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.067+0000 7fe5dc571700 1 -- 172.21.15.47:0/3234739006 shutdown_connections 2022-06-29T15:44:29.078 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.067+0000 7fe5dc571700 1 -- 172.21.15.47:0/3234739006 wait complete. 2022-06-29T15:44:29.082 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.071+0000 7fe5dc571700 1 Processor -- start 2022-06-29T15:44:29.082 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.071+0000 7fe5dc571700 1 -- start start 2022-06-29T15:44:29.082 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.071+0000 7fe5dc571700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5d40611b0 0x7fe5d4152450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:29.082 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.071+0000 7fe5dc571700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe5d40b06d0 0x7fe5d40b85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:29.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.071+0000 7fe5dc571700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe5d40b0cf0 0x7fe5d4145390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:29.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.071+0000 7fe5dc571700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d4152940 con 0x7fe5d40611b0 2022-06-29T15:44:29.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.071+0000 7fe5dc571700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d413f790 con 0x7fe5d40b0cf0 2022-06-29T15:44:29.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.071+0000 7fe5dc571700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d40afcc0 con 0x7fe5d40b06d0 2022-06-29T15:44:29.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.071+0000 7fe5da30d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5d40611b0 0x7fe5d4152450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:29.084 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.071+0000 7fe5da30d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5d40611b0 0x7fe5d4152450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47842/0 (socket says 172.21.15.47:47842) 2022-06-29T15:44:29.084 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.071+0000 7fe5da30d700 1 -- 172.21.15.47:0/3245688938 learned_addr learned my addr 172.21.15.47:0/3245688938 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:29.084 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.071+0000 7fe5d9b0c700 1 --2- 172.21.15.47:0/3245688938 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe5d40b06d0 0x7fe5d40b85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:29.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.071+0000 7fe5dab0e700 1 --2- 172.21.15.47:0/3245688938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe5d40b0cf0 0x7fe5d4145390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:29.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.071+0000 7fe5da30d700 1 -- 172.21.15.47:0/3245688938 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe5d40b06d0 msgr2=0x7fe5d40b85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:29.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.071+0000 7fe5da30d700 1 --2- 172.21.15.47:0/3245688938 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe5d40b06d0 0x7fe5d40b85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:29.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.071+0000 7fe5da30d700 1 -- 172.21.15.47:0/3245688938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe5d40b0cf0 msgr2=0x7fe5d4145390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:29.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.071+0000 7fe5da30d700 1 --2- 172.21.15.47:0/3245688938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe5d40b0cf0 0x7fe5d4145390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:29.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.071+0000 7fe5da30d700 1 -- 172.21.15.47:0/3245688938 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d000b180 con 0x7fe5d40611b0 2022-06-29T15:44:29.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.071+0000 7fe5da30d700 1 --2- 172.21.15.47:0/3245688938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5d40611b0 0x7fe5d4152450 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7fe5c8000bc0 tx=0x7fe5c8007a60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:29.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.071+0000 7fe5c77fe700 1 -- 172.21.15.47:0/3245688938 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5c80070d0 con 0x7fe5d40611b0 2022-06-29T15:44:29.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.071+0000 7fe5c77fe700 1 -- 172.21.15.47:0/3245688938 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c8007230 con 0x7fe5d40611b0 2022-06-29T15:44:29.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.071+0000 7fe5c77fe700 1 -- 172.21.15.47:0/3245688938 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5c8005610 con 0x7fe5d40611b0 2022-06-29T15:44:29.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.071+0000 7fe5dc571700 1 -- 172.21.15.47:0/3245688938 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5d40aff60 con 0x7fe5d40611b0 2022-06-29T15:44:29.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.071+0000 7fe5dc571700 1 -- 172.21.15.47:0/3245688938 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5d4144ef0 con 0x7fe5d40611b0 2022-06-29T15:44:29.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.075+0000 7fe5dc571700 1 -- 172.21.15.47:0/3245688938 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5d40573d0 con 0x7fe5d40611b0 2022-06-29T15:44:29.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.075+0000 7fe5c77fe700 1 -- 172.21.15.47:0/3245688938 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe5c8010070 con 0x7fe5d40611b0 2022-06-29T15:44:29.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.075+0000 7fe5c77fe700 1 --2- 172.21.15.47:0/3245688938 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe5b4053b00 0x7fe5b4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:29.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.075+0000 7fe5c77fe700 1 -- 172.21.15.47:0/3245688938 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe5c806b700 con 0x7fe5d40611b0 2022-06-29T15:44:29.089 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.075+0000 7fe5d9b0c700 1 --2- 172.21.15.47:0/3245688938 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe5b4053b00 0x7fe5b4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:29.089 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.075+0000 7fe5d9b0c700 1 --2- 172.21.15.47:0/3245688938 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe5b4053b00 0x7fe5b4055fc0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7fe5d001c000 tx=0x7fe5d0015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:29.089 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.083+0000 7fe5c77fe700 1 -- 172.21.15.47:0/3245688938 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe5c80424c0 con 0x7fe5d40611b0 2022-06-29T15:44:29.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.215+0000 7fe5dc571700 1 -- 172.21.15.47:0/3245688938 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe5d4061750 con 0x7fe5d40611b0 2022-06-29T15:44:29.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.215+0000 7fe5c77fe700 1 -- 172.21.15.47:0/3245688938 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16271 (secure 0 0 0) 0x7fe5c8045ae0 con 0x7fe5d40611b0 2022-06-29T15:44:29.221 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:44:29.222 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":47,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:44:28.619620+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:44:25.700617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2022-06-29T15:44:26.610945+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:44:29.224 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.219+0000 7fe5dc571700 1 -- 172.21.15.47:0/3245688938 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe5b4053b00 msgr2=0x7fe5b4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:29.224 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.219+0000 7fe5dc571700 1 --2- 172.21.15.47:0/3245688938 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe5b4053b00 0x7fe5b4055fc0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7fe5d001c000 tx=0x7fe5d0015000).stop 2022-06-29T15:44:29.224 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.219+0000 7fe5dc571700 1 -- 172.21.15.47:0/3245688938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5d40611b0 msgr2=0x7fe5d4152450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:29.224 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.219+0000 7fe5dc571700 1 --2- 172.21.15.47:0/3245688938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5d40611b0 0x7fe5d4152450 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7fe5c8000bc0 tx=0x7fe5c8007a60).stop 2022-06-29T15:44:29.225 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.219+0000 7fe5dc571700 1 -- 172.21.15.47:0/3245688938 shutdown_connections 2022-06-29T15:44:29.225 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.219+0000 7fe5dc571700 1 --2- 172.21.15.47:0/3245688938 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe5b4053b00 0x7fe5b4055fc0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:29.225 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.219+0000 7fe5dc571700 1 --2- 172.21.15.47:0/3245688938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe5d40b0cf0 0x7fe5d4145390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:29.225 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.219+0000 7fe5dc571700 1 --2- 172.21.15.47:0/3245688938 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe5d40b06d0 0x7fe5d40b85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:29.226 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.219+0000 7fe5dc571700 1 --2- 172.21.15.47:0/3245688938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5d40611b0 0x7fe5d4152450 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:29.226 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.219+0000 7fe5dc571700 1 -- 172.21.15.47:0/3245688938 >> 172.21.15.47:0/3245688938 conn(0x7fe5d40b98b0 msgr2=0x7fe5d40b6b80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:29.226 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.219+0000 7fe5dc571700 1 -- 172.21.15.47:0/3245688938 shutdown_connections 2022-06-29T15:44:29.226 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.219+0000 7fe5dc571700 1 -- 172.21.15.47:0/3245688938 wait complete. 2022-06-29T15:44:29.234 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-29T15:44:29.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.343+0000 7f159e2f2700 1 Processor -- start 2022-06-29T15:44:29.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.343+0000 7f159e2f2700 1 -- start start 2022-06-29T15:44:29.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.347+0000 7f159e2f2700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f159809a060 0x7f15980b2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:29.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.347+0000 7f159e2f2700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f15980b1e80 0x7f15980600c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:29.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.347+0000 7f159e2f2700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1598140ae0 con 0x7f15980b2bd0 2022-06-29T15:44:29.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.347+0000 7f159e2f2700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1598146900 con 0x7f159809a060 2022-06-29T15:44:29.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.347+0000 7f159e2f2700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1598146ba0 con 0x7f15980b1e80 2022-06-29T15:44:29.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.347+0000 7f159d2f0700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f15980b2bd0 0x7f1598099950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33404/0 (socket says 172.21.15.47:33404) 2022-06-29T15:44:29.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.347+0000 7f159d2f0700 1 -- 172.21.15.47:0/3669872162 learned_addr learned my addr 172.21.15.47:0/3669872162 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:44:29.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.347+0000 7f159daf1700 1 --2- 172.21.15.47:0/3669872162 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f15980b1e80 0x7f15980600c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:29.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.347+0000 7f159caef700 1 --2- 172.21.15.47:0/3669872162 >> v2:172.21.15.64:3300/0 conn(0x7f159809a060 0x7f15980b2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:29.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.347+0000 7f1587fff700 1 -- 172.21.15.47:0/3669872162 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1107324331 0 0) 0x7f1598140ae0 con 0x7f15980b2bd0 2022-06-29T15:44:29.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.347+0000 7f1587fff700 1 -- 172.21.15.47:0/3669872162 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1580003260 con 0x7f15980b2bd0 2022-06-29T15:44:29.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.347+0000 7f159daf1700 1 -- 172.21.15.47:0/3669872162 >> v2:172.21.15.64:3300/0 conn(0x7f159809a060 msgr2=0x7f15980b2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:44:29.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.347+0000 7f159daf1700 1 --2- 172.21.15.47:0/3669872162 >> v2:172.21.15.64:3300/0 conn(0x7f159809a060 0x7f15980b2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:29.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.347+0000 7f1587fff700 1 -- 172.21.15.47:0/3669872162 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1546200774 0 0) 0x7f1580003260 con 0x7f15980b2bd0 2022-06-29T15:44:29.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.347+0000 7f159daf1700 1 -- 172.21.15.47:0/3669872162 >> v1:172.21.15.47:6789/0 conn(0x7f15980b2bd0 legacy=0x7f1598099950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:29.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.347+0000 7f159daf1700 1 -- 172.21.15.47:0/3669872162 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1598146e40 con 0x7f15980b1e80 2022-06-29T15:44:29.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.347+0000 7f159daf1700 1 --2- 172.21.15.47:0/3669872162 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f15980b1e80 0x7f15980600c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f1594001800 tx=0x7f15900091d0).ready entity=mon.2 client_cookie=126c9a03e634e5b5 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:29.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.347+0000 7f1587fff700 1 -- 172.21.15.47:0/3669872162 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1594001660 con 0x7f15980b1e80 2022-06-29T15:44:29.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.347+0000 7f1587fff700 1 -- 172.21.15.47:0/3669872162 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f159000f510 con 0x7f15980b1e80 2022-06-29T15:44:29.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.347+0000 7f1587fff700 1 -- 172.21.15.47:0/3669872162 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f159000f680 con 0x7f15980b1e80 2022-06-29T15:44:29.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.351+0000 7f159e2f2700 1 -- 172.21.15.47:0/3669872162 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f15980b1e80 msgr2=0x7f15980600c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:29.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.351+0000 7f159e2f2700 1 --2- 172.21.15.47:0/3669872162 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f15980b1e80 0x7f15980600c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f1594001800 tx=0x7f15900091d0).stop 2022-06-29T15:44:29.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.351+0000 7f159e2f2700 1 -- 172.21.15.47:0/3669872162 shutdown_connections 2022-06-29T15:44:29.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.351+0000 7f159e2f2700 1 --2- 172.21.15.47:0/3669872162 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f15980b1e80 0x7f15980600c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:29.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.351+0000 7f159e2f2700 1 --2- 172.21.15.47:0/3669872162 >> v2:172.21.15.64:3300/0 conn(0x7f159809a060 0x7f15980b2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:29.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.351+0000 7f159e2f2700 1 -- 172.21.15.47:0/3669872162 >> 172.21.15.47:0/3669872162 conn(0x7f1598062df0 msgr2=0x7f15980631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:29.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.351+0000 7f159e2f2700 1 -- 172.21.15.47:0/3669872162 shutdown_connections 2022-06-29T15:44:29.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.351+0000 7f159e2f2700 1 -- 172.21.15.47:0/3669872162 wait complete. 2022-06-29T15:44:29.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.351+0000 7f159e2f2700 1 Processor -- start 2022-06-29T15:44:29.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.351+0000 7f159e2f2700 1 -- start start 2022-06-29T15:44:29.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.351+0000 7f159e2f2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f159809a060 0x7f159813ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:29.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.351+0000 7f159e2f2700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f15980b1e80 0x7f15980acf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:29.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.351+0000 7f159e2f2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15980b2bd0 0x7f15980b3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:29.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.351+0000 7f159e2f2700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15980b4510 con 0x7f15980b2bd0 2022-06-29T15:44:29.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.351+0000 7f159e2f2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15980ad410 con 0x7f159809a060 2022-06-29T15:44:29.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.351+0000 7f159e2f2700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15980b4270 con 0x7f15980b1e80 2022-06-29T15:44:29.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.351+0000 7f159daf1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15980b2bd0 0x7f15980b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:29.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.355+0000 7f159daf1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15980b2bd0 0x7f15980b3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47860/0 (socket says 172.21.15.47:47860) 2022-06-29T15:44:29.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.355+0000 7f159daf1700 1 -- 172.21.15.47:0/2920946898 learned_addr learned my addr 172.21.15.47:0/2920946898 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:29.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.355+0000 7f159caef700 1 --2- 172.21.15.47:0/2920946898 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f15980b1e80 0x7f15980acf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:29.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.355+0000 7f159d2f0700 1 --2- 172.21.15.47:0/2920946898 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f159809a060 0x7f159813ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:29.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.355+0000 7f159daf1700 1 -- 172.21.15.47:0/2920946898 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f15980b1e80 msgr2=0x7f15980acf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:29.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.355+0000 7f159daf1700 1 --2- 172.21.15.47:0/2920946898 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f15980b1e80 0x7f15980acf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:29.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.355+0000 7f159daf1700 1 -- 172.21.15.47:0/2920946898 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f159809a060 msgr2=0x7f159813ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:29.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.355+0000 7f159daf1700 1 --2- 172.21.15.47:0/2920946898 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f159809a060 0x7f159813ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:29.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.355+0000 7f159daf1700 1 -- 172.21.15.47:0/2920946898 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1590009470 con 0x7f15980b2bd0 2022-06-29T15:44:29.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.355+0000 7f159daf1700 1 --2- 172.21.15.47:0/2920946898 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15980b2bd0 0x7f15980b3890 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f1594001800 tx=0x7f1590016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:29.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.355+0000 7f15867fc700 1 -- 172.21.15.47:0/2920946898 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f15900032d0 con 0x7f15980b2bd0 2022-06-29T15:44:29.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.355+0000 7f15867fc700 1 -- 172.21.15.47:0/2920946898 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1590011c30 con 0x7f15980b2bd0 2022-06-29T15:44:29.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.355+0000 7f159e2f2700 1 -- 172.21.15.47:0/2920946898 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15980afcb0 con 0x7f15980b2bd0 2022-06-29T15:44:29.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.355+0000 7f15867fc700 1 -- 172.21.15.47:0/2920946898 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1590021440 con 0x7f15980b2bd0 2022-06-29T15:44:29.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.355+0000 7f159e2f2700 1 -- 172.21.15.47:0/2920946898 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15980b3d80 con 0x7f15980b2bd0 2022-06-29T15:44:29.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.355+0000 7f159e2f2700 1 -- 172.21.15.47:0/2920946898 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15980ae630 con 0x7f15980b2bd0 2022-06-29T15:44:29.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.359+0000 7f15867fc700 1 -- 172.21.15.47:0/2920946898 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1590027020 con 0x7f15980b2bd0 2022-06-29T15:44:29.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.359+0000 7f15867fc700 1 --2- 172.21.15.47:0/2920946898 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1578053b30 0x7f1578055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:29.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.359+0000 7f15867fc700 1 -- 172.21.15.47:0/2920946898 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f159006d100 con 0x7f15980b2bd0 2022-06-29T15:44:29.372 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.359+0000 7f159d2f0700 1 --2- 172.21.15.47:0/2920946898 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1578053b30 0x7f1578055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:29.372 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.359+0000 7f159d2f0700 1 --2- 172.21.15.47:0/2920946898 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1578053b30 0x7f1578055ff0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f159400d000 tx=0x7f15940019e0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:29.372 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.367+0000 7f15867fc700 1 -- 172.21.15.47:0/2920946898 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1590043e60 con 0x7f15980b2bd0 2022-06-29T15:44:29.541 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.535+0000 7f159e2f2700 1 -- 172.21.15.47:0/2920946898 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f15980573d0 con 0x7f15980b2bd0 2022-06-29T15:44:29.549 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.543+0000 7f15867fc700 1 -- 172.21.15.47:0/2920946898 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v16) v1 ==== 184+0+0 (secure 0 0 0) 0x7f15900150e0 con 0x7f15980b2bd0 2022-06-29T15:44:29.553 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.547+0000 7f159e2f2700 1 -- 172.21.15.47:0/2920946898 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1578053b30 msgr2=0x7f1578055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:29.553 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.547+0000 7f159e2f2700 1 --2- 172.21.15.47:0/2920946898 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1578053b30 0x7f1578055ff0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f159400d000 tx=0x7f15940019e0).stop 2022-06-29T15:44:29.554 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.547+0000 7f159e2f2700 1 -- 172.21.15.47:0/2920946898 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15980b2bd0 msgr2=0x7f15980b3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:29.554 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.547+0000 7f159e2f2700 1 --2- 172.21.15.47:0/2920946898 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15980b2bd0 0x7f15980b3890 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f1594001800 tx=0x7f1590016000).stop 2022-06-29T15:44:29.554 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.547+0000 7f159e2f2700 1 -- 172.21.15.47:0/2920946898 shutdown_connections 2022-06-29T15:44:29.554 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.547+0000 7f159e2f2700 1 --2- 172.21.15.47:0/2920946898 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15980b2bd0 0x7f15980b3890 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:29.555 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.547+0000 7f159e2f2700 1 --2- 172.21.15.47:0/2920946898 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f15980b1e80 0x7f15980acf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:29.555 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.547+0000 7f159e2f2700 1 --2- 172.21.15.47:0/2920946898 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1578053b30 0x7f1578055ff0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:29.555 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.547+0000 7f159e2f2700 1 --2- 172.21.15.47:0/2920946898 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f159809a060 0x7f159813ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:29.555 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.547+0000 7f159e2f2700 1 -- 172.21.15.47:0/2920946898 >> 172.21.15.47:0/2920946898 conn(0x7f1598062df0 msgr2=0x7f159813dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:29.555 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.547+0000 7f159e2f2700 1 -- 172.21.15.47:0/2920946898 shutdown_connections 2022-06-29T15:44:29.556 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.547+0000 7f159e2f2700 1 -- 172.21.15.47:0/2920946898 wait complete. 2022-06-29T15:44:29.556 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.0 2022-06-29T15:44:29.565 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-29T15:44:29.685 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.679+0000 7f8406199700 1 Processor -- start 2022-06-29T15:44:29.686 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.679+0000 7f8406199700 1 -- start start 2022-06-29T15:44:29.689 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.679+0000 7f8406199700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f84000b2bd0 0x7f84000b2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:29.689 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.683+0000 7f8406199700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f840009a2c0 0x7f8400098040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:29.689 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.683+0000 7f8406199700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8400144f30 con 0x7f8400098530 2022-06-29T15:44:29.689 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.683+0000 7f8406199700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84000b0020 con 0x7f84000b2bd0 2022-06-29T15:44:29.690 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.683+0000 7f8406199700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84001451d0 con 0x7f840009a2c0 2022-06-29T15:44:29.690 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.683+0000 7f8405197700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f84000b2bd0 0x7f84000b2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:29.690 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.683+0000 7f8404996700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f840009a2c0 0x7f8400098040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:29.690 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.683+0000 7f8405197700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f84000b2bd0 0x7f84000b2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47864/0 (socket says 172.21.15.47:47864) 2022-06-29T15:44:29.691 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.683+0000 7f8404996700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f840009a2c0 0x7f8400098040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:44968/0 (socket says 172.21.15.47:44968) 2022-06-29T15:44:29.691 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.683+0000 7f8405197700 1 -- 172.21.15.47:0/316332141 learned_addr learned my addr 172.21.15.47:0/316332141 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:29.691 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.683+0000 7f8405998700 1 --1- 172.21.15.47:0/316332141 >> v1:172.21.15.64:6789/0 conn(0x7f8400098530 0x7f84001443a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:40986/0 (socket says 172.21.15.47:40986) 2022-06-29T15:44:29.691 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.683+0000 7f8404996700 1 -- 172.21.15.47:0/316332141 >> v2:172.21.15.47:3300/0 conn(0x7f84000b2bd0 msgr2=0x7f84000b2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:44:29.692 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.683+0000 7f8404996700 1 --2- 172.21.15.47:0/316332141 >> v2:172.21.15.47:3300/0 conn(0x7f84000b2bd0 0x7f84000b2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:29.692 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.683+0000 7f8404996700 1 -- 172.21.15.47:0/316332141 >> v1:172.21.15.64:6789/0 conn(0x7f8400098530 legacy=0x7f84001443a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:29.692 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.683+0000 7f83ebfff700 1 -- 172.21.15.47:0/316332141 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2823400647 0 0) 0x7f8400144f30 con 0x7f8400098530 2022-06-29T15:44:29.692 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.683+0000 7f8404996700 1 -- 172.21.15.47:0/316332141 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84000b1640 con 0x7f840009a2c0 2022-06-29T15:44:29.692 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.683+0000 7f8404996700 1 --2- 172.21.15.47:0/316332141 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f840009a2c0 0x7f8400098040 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f83f4002f70 tx=0x7f83f4006df0).ready entity=mon.2 client_cookie=b678684c5c3013f9 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:29.693 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.683+0000 7f83ebfff700 1 -- 172.21.15.47:0/316332141 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83f400d570 con 0x7f840009a2c0 2022-06-29T15:44:29.693 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.683+0000 7f83ebfff700 1 -- 172.21.15.47:0/316332141 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83f4000e90 con 0x7f840009a2c0 2022-06-29T15:44:29.693 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.683+0000 7f83ebfff700 1 -- 172.21.15.47:0/316332141 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83f400d8c0 con 0x7f840009a2c0 2022-06-29T15:44:29.696 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.683+0000 7f8406199700 1 -- 172.21.15.47:0/316332141 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f840009a2c0 msgr2=0x7f8400098040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:29.696 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.683+0000 7f8406199700 1 --2- 172.21.15.47:0/316332141 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f840009a2c0 0x7f8400098040 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f83f4002f70 tx=0x7f83f4006df0).stop 2022-06-29T15:44:29.697 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.683+0000 7f8406199700 1 -- 172.21.15.47:0/316332141 shutdown_connections 2022-06-29T15:44:29.697 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.683+0000 7f8406199700 1 --2- 172.21.15.47:0/316332141 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f840009a2c0 0x7f8400098040 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:29.697 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.683+0000 7f8406199700 1 --2- 172.21.15.47:0/316332141 >> v2:172.21.15.47:3300/0 conn(0x7f84000b2bd0 0x7f84000b2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:29.697 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.683+0000 7f8406199700 1 -- 172.21.15.47:0/316332141 >> 172.21.15.47:0/316332141 conn(0x7f8400062df0 msgr2=0x7f84000631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:29.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.683+0000 7f8406199700 1 -- 172.21.15.47:0/316332141 shutdown_connections 2022-06-29T15:44:29.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.683+0000 7f8406199700 1 -- 172.21.15.47:0/316332141 wait complete. 2022-06-29T15:44:29.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.687+0000 7f8406199700 1 Processor -- start 2022-06-29T15:44:29.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.687+0000 7f8406199700 1 -- start start 2022-06-29T15:44:29.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.687+0000 7f8406199700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8400098530 0x7f84000ac180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:29.699 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.687+0000 7f8406199700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f840009a2c0 0x7f84000ac670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:29.699 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.687+0000 7f8406199700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84000b2bd0 0x7f84000ab440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:29.699 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.687+0000 7f8406199700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84000ae4f0 con 0x7f84000b2bd0 2022-06-29T15:44:29.699 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.687+0000 7f8406199700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8400143bd0 con 0x7f840009a2c0 2022-06-29T15:44:29.700 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.687+0000 7f8406199700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8400143e00 con 0x7f8400098530 2022-06-29T15:44:29.700 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.687+0000 7f8405998700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84000b2bd0 0x7f84000ab440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:29.700 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.687+0000 7f8405998700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84000b2bd0 0x7f84000ab440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47874/0 (socket says 172.21.15.47:47874) 2022-06-29T15:44:29.700 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.687+0000 7f8405998700 1 -- 172.21.15.47:0/3741936948 learned_addr learned my addr 172.21.15.47:0/3741936948 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:29.700 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.687+0000 7f8405197700 1 --2- 172.21.15.47:0/3741936948 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8400098530 0x7f84000ac180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:29.701 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.691+0000 7f8404996700 1 --2- 172.21.15.47:0/3741936948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f840009a2c0 0x7f84000ac670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:29.701 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.691+0000 7f8405998700 1 -- 172.21.15.47:0/3741936948 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8400098530 msgr2=0x7f84000ac180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:29.701 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.691+0000 7f8405998700 1 --2- 172.21.15.47:0/3741936948 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8400098530 0x7f84000ac180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:29.701 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.691+0000 7f8405998700 1 -- 172.21.15.47:0/3741936948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f840009a2c0 msgr2=0x7f84000ac670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:29.702 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.691+0000 7f8405998700 1 --2- 172.21.15.47:0/3741936948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f840009a2c0 0x7f84000ac670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:29.702 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.691+0000 7f8405998700 1 -- 172.21.15.47:0/3741936948 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f400b180 con 0x7f84000b2bd0 2022-06-29T15:44:29.702 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.691+0000 7f8405998700 1 --2- 172.21.15.47:0/3741936948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84000b2bd0 0x7f84000ab440 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7f83f8001250 tx=0x7f83f8007db0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:29.702 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.691+0000 7f83f2ffd700 1 -- 172.21.15.47:0/3741936948 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83f8005bb0 con 0x7f84000b2bd0 2022-06-29T15:44:29.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.691+0000 7f83f2ffd700 1 -- 172.21.15.47:0/3741936948 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83f8005d10 con 0x7f84000b2bd0 2022-06-29T15:44:29.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.691+0000 7f8406199700 1 -- 172.21.15.47:0/3741936948 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84000a9b10 con 0x7f84000b2bd0 2022-06-29T15:44:29.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.691+0000 7f83f2ffd700 1 -- 172.21.15.47:0/3741936948 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83f8007300 con 0x7f84000b2bd0 2022-06-29T15:44:29.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.691+0000 7f8406199700 1 -- 172.21.15.47:0/3741936948 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f840013e470 con 0x7f84000b2bd0 2022-06-29T15:44:29.704 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.691+0000 7f8406199700 1 -- 172.21.15.47:0/3741936948 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84000a9ca0 con 0x7f84000b2bd0 2022-06-29T15:44:29.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.691+0000 7f83f2ffd700 1 -- 172.21.15.47:0/3741936948 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f83f8010030 con 0x7f84000b2bd0 2022-06-29T15:44:29.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.691+0000 7f83f2ffd700 1 --2- 172.21.15.47:0/3741936948 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f83dc053b30 0x7f83dc055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:29.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.691+0000 7f83f2ffd700 1 -- 172.21.15.47:0/3741936948 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f83f806c020 con 0x7f84000b2bd0 2022-06-29T15:44:29.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.691+0000 7f8405197700 1 --2- 172.21.15.47:0/3741936948 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f83dc053b30 0x7f83dc055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:29.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.695+0000 7f8405197700 1 --2- 172.21.15.47:0/3741936948 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f83dc053b30 0x7f83dc055ff0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f83fc008000 tx=0x7f83fc008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:29.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.699+0000 7f83f2ffd700 1 -- 172.21.15.47:0/3741936948 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f83f8042de0 con 0x7f84000b2bd0 2022-06-29T15:44:29.872 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.867+0000 7f8406199700 1 -- 172.21.15.47:0/3741936948 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f84000573d0 con 0x7f84000b2bd0 2022-06-29T15:44:29.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.875+0000 7f83f2ffd700 1 -- 172.21.15.47:0/3741936948 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v17) v1 ==== 184+0+0 (secure 0 0 0) 0x7f83f8046400 con 0x7f84000b2bd0 2022-06-29T15:44:29.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.875+0000 7f8406199700 1 -- 172.21.15.47:0/3741936948 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f83dc053b30 msgr2=0x7f83dc055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:29.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.875+0000 7f8406199700 1 --2- 172.21.15.47:0/3741936948 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f83dc053b30 0x7f83dc055ff0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f83fc008000 tx=0x7f83fc008040).stop 2022-06-29T15:44:29.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.875+0000 7f8406199700 1 -- 172.21.15.47:0/3741936948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84000b2bd0 msgr2=0x7f84000ab440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:29.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.875+0000 7f8406199700 1 --2- 172.21.15.47:0/3741936948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84000b2bd0 0x7f84000ab440 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7f83f8001250 tx=0x7f83f8007db0).stop 2022-06-29T15:44:29.883 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.875+0000 7f8406199700 1 -- 172.21.15.47:0/3741936948 shutdown_connections 2022-06-29T15:44:29.883 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.875+0000 7f8406199700 1 --2- 172.21.15.47:0/3741936948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84000b2bd0 0x7f84000ab440 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:29.883 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.875+0000 7f8406199700 1 --2- 172.21.15.47:0/3741936948 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f83dc053b30 0x7f83dc055ff0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:29.883 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.875+0000 7f8406199700 1 --2- 172.21.15.47:0/3741936948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f840009a2c0 0x7f84000ac670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:29.884 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.875+0000 7f8406199700 1 --2- 172.21.15.47:0/3741936948 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8400098530 0x7f84000ac180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:29.884 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.875+0000 7f8406199700 1 -- 172.21.15.47:0/3741936948 >> 172.21.15.47:0/3741936948 conn(0x7f8400062df0 msgr2=0x7f8400144570 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:29.884 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.875+0000 7f8406199700 1 -- 172.21.15.47:0/3741936948 shutdown_connections 2022-06-29T15:44:29.885 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:29.875+0000 7f8406199700 1 -- 172.21.15.47:0/3741936948 wait complete. 2022-06-29T15:44:29.886 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.1 2022-06-29T15:44:29.899 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:44:30.009 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.003+0000 7fa2d0d45700 1 Processor -- start 2022-06-29T15:44:30.010 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.003+0000 7fa2d0d45700 1 -- start start 2022-06-29T15:44:30.011 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.007+0000 7fa2d0d45700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa2cc1430f0 0x7fa2cc1434d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:30.012 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.007+0000 7fa2d0d45700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2cc060e50 con 0x7fa2cc095950 2022-06-29T15:44:30.012 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.007+0000 7fa2d0d45700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2cc0610f0 con 0x7fa2cc0a93c0 2022-06-29T15:44:30.012 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.007+0000 7fa2d0d45700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2cc13d590 con 0x7fa2cc1430f0 2022-06-29T15:44:30.013 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.007+0000 7fa2cad9d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa2cc095950 0x7fa2cc097690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33436/0 (socket says 172.21.15.47:33436) 2022-06-29T15:44:30.013 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.007+0000 7fa2cad9d700 1 -- 172.21.15.47:0/2794128745 learned_addr learned my addr 172.21.15.47:0/2794128745 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:44:30.013 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.007+0000 7fa2ca59c700 1 --2- 172.21.15.47:0/2794128745 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa2cc1430f0 0x7fa2cc1434d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:30.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.007+0000 7fa2c9d9b700 1 -- 172.21.15.47:0/2794128745 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4258360075 0 0) 0x7fa2cc060e50 con 0x7fa2cc095950 2022-06-29T15:44:30.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.007+0000 7fa2c9d9b700 1 -- 172.21.15.47:0/2794128745 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2ac003260 con 0x7fa2cc095950 2022-06-29T15:44:30.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.007+0000 7fa2c9d9b700 1 -- 172.21.15.47:0/2794128745 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2490248934 0 0) 0x7fa2cc0610f0 con 0x7fa2cc0a93c0 2022-06-29T15:44:30.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.007+0000 7fa2c9d9b700 1 -- 172.21.15.47:0/2794128745 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2cc060e50 con 0x7fa2cc0a93c0 2022-06-29T15:44:30.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.007+0000 7fa2ca59c700 1 -- 172.21.15.47:0/2794128745 >> v1:172.21.15.64:6789/0 conn(0x7fa2cc0a93c0 legacy=0x7fa2cc095240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:30.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.007+0000 7fa2ca59c700 1 -- 172.21.15.47:0/2794128745 >> v1:172.21.15.47:6789/0 conn(0x7fa2cc095950 legacy=0x7fa2cc097690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:30.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.007+0000 7fa2ca59c700 1 -- 172.21.15.47:0/2794128745 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2cc0971f0 con 0x7fa2cc1430f0 2022-06-29T15:44:30.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.007+0000 7fa2ca59c700 1 --2- 172.21.15.47:0/2794128745 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa2cc1430f0 0x7fa2cc1434d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fa2bc002f70 tx=0x7fa2bc006df0).ready entity=mon.2 client_cookie=90c816983d6afee server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:30.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.007+0000 7fa2c9d9b700 1 -- 172.21.15.47:0/2794128745 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2bc00d4f0 con 0x7fa2cc1430f0 2022-06-29T15:44:30.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.007+0000 7fa2c9d9b700 1 -- 172.21.15.47:0/2794128745 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2bc000e90 con 0x7fa2cc1430f0 2022-06-29T15:44:30.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.007+0000 7fa2c9d9b700 1 -- 172.21.15.47:0/2794128745 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2bc00d840 con 0x7fa2cc1430f0 2022-06-29T15:44:30.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.007+0000 7fa2d0d45700 1 -- 172.21.15.47:0/2794128745 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa2cc1430f0 msgr2=0x7fa2cc1434d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:30.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.007+0000 7fa2d0d45700 1 --2- 172.21.15.47:0/2794128745 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa2cc1430f0 0x7fa2cc1434d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fa2bc002f70 tx=0x7fa2bc006df0).stop 2022-06-29T15:44:30.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.007+0000 7fa2d0d45700 1 -- 172.21.15.47:0/2794128745 shutdown_connections 2022-06-29T15:44:30.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.007+0000 7fa2d0d45700 1 --2- 172.21.15.47:0/2794128745 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa2cc1430f0 0x7fa2cc1434d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:30.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.007+0000 7fa2d0d45700 1 -- 172.21.15.47:0/2794128745 >> 172.21.15.47:0/2794128745 conn(0x7fa2cc05f0b0 msgr2=0x7fa2cc05f490 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:30.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.007+0000 7fa2d0d45700 1 -- 172.21.15.47:0/2794128745 shutdown_connections 2022-06-29T15:44:30.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.007+0000 7fa2d0d45700 1 -- 172.21.15.47:0/2794128745 wait complete. 2022-06-29T15:44:30.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.011+0000 7fa2d0d45700 1 Processor -- start 2022-06-29T15:44:30.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.011+0000 7fa2d0d45700 1 -- start start 2022-06-29T15:44:30.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.011+0000 7fa2d0d45700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2cc095950 0x7fa2cc131dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:30.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.011+0000 7fa2d0d45700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa2cc0a93c0 0x7fa2cc150f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:30.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.011+0000 7fa2d0d45700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2cc1430f0 0x7fa2cc14d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:30.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.011+0000 7fa2d0d45700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2cc153a70 con 0x7fa2cc095950 2022-06-29T15:44:30.024 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.011+0000 7fa2d0d45700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2cc153ca0 con 0x7fa2cc1430f0 2022-06-29T15:44:30.024 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.011+0000 7fa2d0d45700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2cc1507f0 con 0x7fa2cc0a93c0 2022-06-29T15:44:30.024 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.011+0000 7fa2ca59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2cc095950 0x7fa2cc131dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:30.024 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.011+0000 7fa2ca59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2cc095950 0x7fa2cc131dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47884/0 (socket says 172.21.15.47:47884) 2022-06-29T15:44:30.025 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.011+0000 7fa2ca59c700 1 -- 172.21.15.47:0/2809918571 learned_addr learned my addr 172.21.15.47:0/2809918571 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:30.025 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.011+0000 7fa2c1d9b700 1 --2- 172.21.15.47:0/2809918571 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa2cc0a93c0 0x7fa2cc150f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:30.025 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.011+0000 7fa2cad9d700 1 --2- 172.21.15.47:0/2809918571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2cc1430f0 0x7fa2cc14d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:30.025 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.011+0000 7fa2ca59c700 1 -- 172.21.15.47:0/2809918571 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa2cc0a93c0 msgr2=0x7fa2cc150f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:30.025 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.011+0000 7fa2ca59c700 1 --2- 172.21.15.47:0/2809918571 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa2cc0a93c0 0x7fa2cc150f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:30.026 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.011+0000 7fa2ca59c700 1 -- 172.21.15.47:0/2809918571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2cc1430f0 msgr2=0x7fa2cc14d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:30.026 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.011+0000 7fa2ca59c700 1 --2- 172.21.15.47:0/2809918571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2cc1430f0 0x7fa2cc14d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:30.026 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.011+0000 7fa2ca59c700 1 -- 172.21.15.47:0/2809918571 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2bc00b180 con 0x7fa2cc095950 2022-06-29T15:44:30.026 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.011+0000 7fa2cad9d700 1 --2- 172.21.15.47:0/2809918571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2cc1430f0 0x7fa2cc14d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:44:30.027 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.011+0000 7fa2ca59c700 1 --2- 172.21.15.47:0/2809918571 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2cc095950 0x7fa2cc131dc0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7fa2bc002f70 tx=0x7fa2bc014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:30.027 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.011+0000 7fa2c3fff700 1 -- 172.21.15.47:0/2809918571 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2bc00de40 con 0x7fa2cc095950 2022-06-29T15:44:30.027 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.011+0000 7fa2c3fff700 1 -- 172.21.15.47:0/2809918571 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2bc01e400 con 0x7fa2cc095950 2022-06-29T15:44:30.027 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.011+0000 7fa2d0d45700 1 -- 172.21.15.47:0/2809918571 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2cc150a90 con 0x7fa2cc095950 2022-06-29T15:44:30.028 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.011+0000 7fa2c3fff700 1 -- 172.21.15.47:0/2809918571 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2bc01e760 con 0x7fa2cc095950 2022-06-29T15:44:30.028 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.011+0000 7fa2d0d45700 1 -- 172.21.15.47:0/2809918571 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2cc150520 con 0x7fa2cc095950 2022-06-29T15:44:30.028 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.015+0000 7fa2c3fff700 1 -- 172.21.15.47:0/2809918571 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa2bc025030 con 0x7fa2cc095950 2022-06-29T15:44:30.028 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.015+0000 7fa2d0d45700 1 -- 172.21.15.47:0/2809918571 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2cc0573d0 con 0x7fa2cc095950 2022-06-29T15:44:30.028 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.015+0000 7fa2c3fff700 1 --2- 172.21.15.47:0/2809918571 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa2b0053b00 0x7fa2b0055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:30.029 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.015+0000 7fa2c3fff700 1 -- 172.21.15.47:0/2809918571 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa2bc06cd40 con 0x7fa2cc095950 2022-06-29T15:44:30.029 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.015+0000 7fa2c1d9b700 1 --2- 172.21.15.47:0/2809918571 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa2b0053b00 0x7fa2b0055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:30.029 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.015+0000 7fa2c1d9b700 1 --2- 172.21.15.47:0/2809918571 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa2b0053b00 0x7fa2b0055fc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7fa2b400b000 tx=0x7fa2b4009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:30.030 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.023+0000 7fa2c3fff700 1 -- 172.21.15.47:0/2809918571 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa2bc043b00 con 0x7fa2cc095950 2022-06-29T15:44:30.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.179+0000 7fa2d0d45700 1 -- 172.21.15.47:0/2809918571 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa2cc09bdb0 con 0x7fa2cc095950 2022-06-29T15:44:30.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.179+0000 7fa2c3fff700 1 -- 172.21.15.47:0/2809918571 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+2491 (secure 0 0 0) 0x7fa2bc023070 con 0x7fa2cc095950 2022-06-29T15:44:30.186 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:44:30.187 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":17,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5259,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3601815346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3601815346},{"type":"v1","addr":"172.21.15.47:6835","nonce":3601815346}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15},{"gid":5262,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/657177602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":657177602},{"type":"v1","addr":"172.21.15.47:6837","nonce":657177602}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:44:27.616735+0000","modified":"2022-06-29T15:44:28.620601+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":54812},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54812":{"gid":54812,"name":"b","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6833/2497658100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2497658100},{"type":"v1","addr":"172.21.15.64:6833","nonce":2497658100}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2022-06-29T15:44:30.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.183+0000 7fa2d0d45700 1 -- 172.21.15.47:0/2809918571 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa2b0053b00 msgr2=0x7fa2b0055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:30.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.183+0000 7fa2d0d45700 1 --2- 172.21.15.47:0/2809918571 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa2b0053b00 0x7fa2b0055fc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7fa2b400b000 tx=0x7fa2b4009040).stop 2022-06-29T15:44:30.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.183+0000 7fa2d0d45700 1 -- 172.21.15.47:0/2809918571 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2cc095950 msgr2=0x7fa2cc131dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:30.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.183+0000 7fa2d0d45700 1 --2- 172.21.15.47:0/2809918571 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2cc095950 0x7fa2cc131dc0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7fa2bc002f70 tx=0x7fa2bc014000).stop 2022-06-29T15:44:30.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.183+0000 7fa2d0d45700 1 -- 172.21.15.47:0/2809918571 shutdown_connections 2022-06-29T15:44:30.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.183+0000 7fa2d0d45700 1 --2- 172.21.15.47:0/2809918571 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa2b0053b00 0x7fa2b0055fc0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:30.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.183+0000 7fa2d0d45700 1 --2- 172.21.15.47:0/2809918571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2cc1430f0 0x7fa2cc14d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:30.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.183+0000 7fa2d0d45700 1 --2- 172.21.15.47:0/2809918571 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa2cc0a93c0 0x7fa2cc150f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:30.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.183+0000 7fa2d0d45700 1 --2- 172.21.15.47:0/2809918571 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2cc095950 0x7fa2cc131dc0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:30.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.183+0000 7fa2d0d45700 1 -- 172.21.15.47:0/2809918571 >> 172.21.15.47:0/2809918571 conn(0x7fa2cc05f0b0 msgr2=0x7fa2cc0b52a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:30.191 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.183+0000 7fa2d0d45700 1 -- 172.21.15.47:0/2809918571 shutdown_connections 2022-06-29T15:44:30.191 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.183+0000 7fa2d0d45700 1 -- 172.21.15.47:0/2809918571 wait complete. 2022-06-29T15:44:30.191 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 17 2022-06-29T15:44:30.501 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-29T15:44:27.616735+0000', 'modified': '2022-06-29T15:44:28.620601+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 54812}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_54812': {'gid': 54812, 'name': 'b', 'rank': 0, 'incarnation': 16, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.64:6833/2497658100', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6832', 'nonce': 2497658100}, {'type': 'v1', 'addr': '172.21.15.64:6833', 'nonce': 2497658100}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-29T15:44:30.516 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-29T15:44:30.517 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-06-29T15:44:30.633 INFO:teuthology.orchestra.run.smithi064.stdout:{"cluster_fsid":"9a68894a-53ce-400e-838b-14eab91b834c","whoami":0,"id":54812,"want_state":"up:active","state":"up:active","rank_uptime":3.0000333170000002,"mdsmap_epoch":17,"osdmap_epoch":47,"osdmap_epoch_barrier":47,"uptime":7.7393270039999997} 2022-06-29T15:44:30.634 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9a68894a-53ce-400e-838b-14eab91b834c", "id": 54812, "mdsmap_epoch": 17, "osdmap_epoch": 47, "osdmap_epoch_barrier": 47, "rank_uptime": 3.000033317, "state": "up:active", "uptime": 7.739327004, "want_state": "up:active", "whoami": 0 } 2022-06-29T15:44:30.635 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:44:30.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.739+0000 7fde4b150700 1 Processor -- start 2022-06-29T15:44:30.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.739+0000 7fde4b150700 1 -- start start 2022-06-29T15:44:30.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.739+0000 7fde4b150700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fde44060a60 0x7fde440aa080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:30.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.739+0000 7fde4b150700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fde440601a0 0x7fde440ab7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:30.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.739+0000 7fde4b150700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde440b6040 con 0x7fde440b5c60 2022-06-29T15:44:30.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.739+0000 7fde4b150700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde440b11a0 con 0x7fde44060a60 2022-06-29T15:44:30.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.739+0000 7fde4b150700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde440b2900 con 0x7fde440601a0 2022-06-29T15:44:30.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.739+0000 7fde4994d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fde44060a60 0x7fde440aa080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:30.748 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.739+0000 7fde4994d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fde44060a60 0x7fde440aa080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47894/0 (socket says 172.21.15.47:47894) 2022-06-29T15:44:30.748 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.739+0000 7fde4994d700 1 -- 172.21.15.47:0/4043530173 learned_addr learned my addr 172.21.15.47:0/4043530173 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:30.748 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.739+0000 7fde4a94f700 1 --2- 172.21.15.47:0/4043530173 >> v2:172.21.15.64:3300/0 conn(0x7fde440601a0 0x7fde440ab7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:30.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.743+0000 7fde4994d700 1 -- 172.21.15.47:0/4043530173 >> v2:172.21.15.64:3300/0 conn(0x7fde440601a0 msgr2=0x7fde440ab7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:44:30.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.743+0000 7fde4994d700 1 --2- 172.21.15.47:0/4043530173 >> v2:172.21.15.64:3300/0 conn(0x7fde440601a0 0x7fde440ab7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:30.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.743+0000 7fde4994d700 1 -- 172.21.15.47:0/4043530173 >> v1:172.21.15.64:6789/0 conn(0x7fde440b5c60 legacy=0x7fde440afdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:30.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.743+0000 7fde4994d700 1 -- 172.21.15.47:0/4043530173 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde440b2ba0 con 0x7fde44060a60 2022-06-29T15:44:30.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.743+0000 7fde4994d700 1 --2- 172.21.15.47:0/4043530173 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde44060a60 0x7fde440aa080 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7fde38002f70 tx=0x7fde38006eb0).ready entity=mon.0 client_cookie=16b80ba0bf27187f server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:30.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.743+0000 7fde4914c700 1 -- 172.21.15.47:0/4043530173 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde3800d5c0 con 0x7fde44060a60 2022-06-29T15:44:30.752 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.743+0000 7fde4914c700 1 -- 172.21.15.47:0/4043530173 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde38000e90 con 0x7fde44060a60 2022-06-29T15:44:30.752 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.743+0000 7fde4914c700 1 -- 172.21.15.47:0/4043530173 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde3800d950 con 0x7fde44060a60 2022-06-29T15:44:30.752 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.743+0000 7fde4b150700 1 -- 172.21.15.47:0/4043530173 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde44060a60 msgr2=0x7fde440aa080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:30.752 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.743+0000 7fde4b150700 1 --2- 172.21.15.47:0/4043530173 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde44060a60 0x7fde440aa080 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7fde38002f70 tx=0x7fde38006eb0).stop 2022-06-29T15:44:30.752 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.743+0000 7fde4b150700 1 -- 172.21.15.47:0/4043530173 shutdown_connections 2022-06-29T15:44:30.753 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.743+0000 7fde4b150700 1 --2- 172.21.15.47:0/4043530173 >> v2:172.21.15.64:3300/0 conn(0x7fde440601a0 0x7fde440ab7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:30.753 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.743+0000 7fde4b150700 1 --2- 172.21.15.47:0/4043530173 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde44060a60 0x7fde440aa080 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:30.753 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.743+0000 7fde4b150700 1 -- 172.21.15.47:0/4043530173 >> 172.21.15.47:0/4043530173 conn(0x7fde44062560 msgr2=0x7fde44062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:30.753 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.743+0000 7fde4b150700 1 -- 172.21.15.47:0/4043530173 shutdown_connections 2022-06-29T15:44:30.754 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.743+0000 7fde4b150700 1 -- 172.21.15.47:0/4043530173 wait complete. 2022-06-29T15:44:30.754 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.743+0000 7fde4b150700 1 Processor -- start 2022-06-29T15:44:30.754 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.743+0000 7fde4b150700 1 -- start start 2022-06-29T15:44:30.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.747+0000 7fde4b150700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fde440601a0 0x7fde440aa340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:30.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.747+0000 7fde4b150700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde44060a60 0x7fde44155f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:30.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.747+0000 7fde4b150700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde440b5c60 0x7fde44156460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:30.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.747+0000 7fde4b150700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde440670f0 con 0x7fde44060a60 2022-06-29T15:44:30.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.747+0000 7fde4b150700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde44064bc0 con 0x7fde440b5c60 2022-06-29T15:44:30.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.747+0000 7fde4b150700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde44064e60 con 0x7fde440601a0 2022-06-29T15:44:30.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.747+0000 7fde4994d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde44060a60 0x7fde44155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:30.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.747+0000 7fde4994d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde44060a60 0x7fde44155f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47900/0 (socket says 172.21.15.47:47900) 2022-06-29T15:44:30.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.747+0000 7fde4994d700 1 -- 172.21.15.47:0/327667695 learned_addr learned my addr 172.21.15.47:0/327667695 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:30.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.747+0000 7fde4a14e700 1 --2- 172.21.15.47:0/327667695 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fde440601a0 0x7fde440aa340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:30.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.747+0000 7fde4a94f700 1 --2- 172.21.15.47:0/327667695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde440b5c60 0x7fde44156460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:30.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.747+0000 7fde4994d700 1 -- 172.21.15.47:0/327667695 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fde440601a0 msgr2=0x7fde440aa340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:30.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.747+0000 7fde4994d700 1 --2- 172.21.15.47:0/327667695 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fde440601a0 0x7fde440aa340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:30.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.747+0000 7fde4994d700 1 -- 172.21.15.47:0/327667695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde440b5c60 msgr2=0x7fde44156460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:30.761 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.747+0000 7fde4994d700 1 --2- 172.21.15.47:0/327667695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde440b5c60 0x7fde44156460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:30.761 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.747+0000 7fde4994d700 1 -- 172.21.15.47:0/327667695 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde3800b1e0 con 0x7fde44060a60 2022-06-29T15:44:30.761 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.747+0000 7fde4a94f700 1 --2- 172.21.15.47:0/327667695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde440b5c60 0x7fde44156460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:44:30.761 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.747+0000 7fde4a14e700 1 --2- 172.21.15.47:0/327667695 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fde440601a0 0x7fde440aa340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:44:30.761 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.747+0000 7fde4994d700 1 --2- 172.21.15.47:0/327667695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde44060a60 0x7fde44155f70 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7fde38002f70 tx=0x7fde38015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:30.762 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.747+0000 7fde437fe700 1 -- 172.21.15.47:0/327667695 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde38011910 con 0x7fde44060a60 2022-06-29T15:44:30.762 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.747+0000 7fde437fe700 1 -- 172.21.15.47:0/327667695 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde38011a70 con 0x7fde44060a60 2022-06-29T15:44:30.762 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.747+0000 7fde4b150700 1 -- 172.21.15.47:0/327667695 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde44155ad0 con 0x7fde44060a60 2022-06-29T15:44:30.762 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.747+0000 7fde437fe700 1 -- 172.21.15.47:0/327667695 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde38011dd0 con 0x7fde44060a60 2022-06-29T15:44:30.763 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.747+0000 7fde4b150700 1 -- 172.21.15.47:0/327667695 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde440adc90 con 0x7fde44060a60 2022-06-29T15:44:30.763 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.747+0000 7fde4b150700 1 -- 172.21.15.47:0/327667695 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde44064510 con 0x7fde44060a60 2022-06-29T15:44:30.763 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.751+0000 7fde437fe700 1 -- 172.21.15.47:0/327667695 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fde3801a040 con 0x7fde44060a60 2022-06-29T15:44:30.763 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.751+0000 7fde437fe700 1 --2- 172.21.15.47:0/327667695 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fde24053b30 0x7fde24055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:30.764 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.751+0000 7fde437fe700 1 -- 172.21.15.47:0/327667695 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fde3806cf30 con 0x7fde44060a60 2022-06-29T15:44:30.764 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.751+0000 7fde4a14e700 1 --2- 172.21.15.47:0/327667695 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fde24053b30 0x7fde24055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:30.764 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.751+0000 7fde4a14e700 1 --2- 172.21.15.47:0/327667695 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fde24053b30 0x7fde24055ff0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7fde3400b000 tx=0x7fde34009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:30.765 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.755+0000 7fde437fe700 1 -- 172.21.15.47:0/327667695 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fde44064510 con 0x7fde44060a60 2022-06-29T15:44:30.891 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.883+0000 7fde4b150700 1 -- 172.21.15.47:0/327667695 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fde44064510 con 0x7fde44060a60 2022-06-29T15:44:30.893 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.887+0000 7fde437fe700 1 -- 172.21.15.47:0/327667695 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16271 (secure 0 0 0) 0x7fde44064510 con 0x7fde44060a60 2022-06-29T15:44:30.893 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:44:30.894 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":47,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:44:28.619620+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:44:25.700617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2022-06-29T15:44:26.610945+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:44:30.896 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.887+0000 7fde4b150700 1 -- 172.21.15.47:0/327667695 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fde24053b30 msgr2=0x7fde24055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:30.897 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.887+0000 7fde4b150700 1 --2- 172.21.15.47:0/327667695 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fde24053b30 0x7fde24055ff0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7fde3400b000 tx=0x7fde34009040).stop 2022-06-29T15:44:30.897 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.887+0000 7fde4b150700 1 -- 172.21.15.47:0/327667695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde44060a60 msgr2=0x7fde44155f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:30.897 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.887+0000 7fde4b150700 1 --2- 172.21.15.47:0/327667695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde44060a60 0x7fde44155f70 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7fde38002f70 tx=0x7fde38015000).stop 2022-06-29T15:44:30.898 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.891+0000 7fde4b150700 1 -- 172.21.15.47:0/327667695 shutdown_connections 2022-06-29T15:44:30.898 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.891+0000 7fde4b150700 1 --2- 172.21.15.47:0/327667695 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fde24053b30 0x7fde24055ff0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:30.898 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.891+0000 7fde4b150700 1 --2- 172.21.15.47:0/327667695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde440b5c60 0x7fde44156460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:30.899 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.891+0000 7fde4b150700 1 --2- 172.21.15.47:0/327667695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde44060a60 0x7fde44155f70 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:30.899 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.891+0000 7fde4b150700 1 --2- 172.21.15.47:0/327667695 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fde440601a0 0x7fde440aa340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:30.899 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.891+0000 7fde4b150700 1 -- 172.21.15.47:0/327667695 >> 172.21.15.47:0/327667695 conn(0x7fde44062560 msgr2=0x7fde440aac90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:30.900 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.891+0000 7fde4b150700 1 -- 172.21.15.47:0/327667695 shutdown_connections 2022-06-29T15:44:30.900 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:30.891+0000 7fde4b150700 1 -- 172.21.15.47:0/327667695 wait complete. 2022-06-29T15:44:30.911 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-29T15:42:16.971431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-29T15:42:25.290884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-29T15:44:25.700617+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2022-06-29T15:44:26.610945+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '47', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-29T15:44:30.912 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-29T15:44:31.030 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.023+0000 7fbc2552f700 1 Processor -- start 2022-06-29T15:44:31.030 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.023+0000 7fbc2552f700 1 -- start start 2022-06-29T15:44:31.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.023+0000 7fbc2552f700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbc200b64c0 0x7fbc200b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:31.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.023+0000 7fbc2552f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbc20066fe0 0x7fbc200b4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:31.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.023+0000 7fbc2552f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbc20143150 0x7fbc200b5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:31.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.023+0000 7fbc2552f700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc200b13a0 con 0x7fbc20066fe0 2022-06-29T15:44:31.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.023+0000 7fbc2552f700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc200b68a0 con 0x7fbc20143150 2022-06-29T15:44:31.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.023+0000 7fbc2552f700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc201473d0 con 0x7fbc200b64c0 2022-06-29T15:44:31.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.023+0000 7fbc177fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbc20066fe0 0x7fbc200b4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:31.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.023+0000 7fbc177fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbc20066fe0 0x7fbc200b4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47906/0 (socket says 172.21.15.47:47906) 2022-06-29T15:44:31.038 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.023+0000 7fbc177fe700 1 -- 172.21.15.47:0/2994071123 learned_addr learned my addr 172.21.15.47:0/2994071123 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:31.038 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.027+0000 7fbc1ffff700 1 --2- 172.21.15.47:0/2994071123 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbc200b64c0 0x7fbc200b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:31.038 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.027+0000 7fbc24d2e700 1 --2- 172.21.15.47:0/2994071123 >> v2:172.21.15.64:3300/0 conn(0x7fbc20143150 0x7fbc200b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:31.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.027+0000 7fbc177fe700 1 -- 172.21.15.47:0/2994071123 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbc200b64c0 msgr2=0x7fbc200b5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:44:31.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.027+0000 7fbc177fe700 1 --2- 172.21.15.47:0/2994071123 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbc200b64c0 0x7fbc200b5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:31.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.027+0000 7fbc177fe700 1 -- 172.21.15.47:0/2994071123 >> v2:172.21.15.64:3300/0 conn(0x7fbc20143150 msgr2=0x7fbc200b5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:44:31.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.027+0000 7fbc177fe700 1 --2- 172.21.15.47:0/2994071123 >> v2:172.21.15.64:3300/0 conn(0x7fbc20143150 0x7fbc200b5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:31.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.027+0000 7fbc177fe700 1 -- 172.21.15.47:0/2994071123 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc200b46a0 con 0x7fbc20066fe0 2022-06-29T15:44:31.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.027+0000 7fbc1ffff700 1 --2- 172.21.15.47:0/2994071123 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbc200b64c0 0x7fbc200b5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:44:31.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.027+0000 7fbc177fe700 1 --2- 172.21.15.47:0/2994071123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc20066fe0 0x7fbc200b4b40 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7fbc08002f70 tx=0x7fbc08006e50).ready entity=mon.0 client_cookie=e6187a3989a27f47 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:31.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.027+0000 7fbc1f7fe700 1 -- 172.21.15.47:0/2994071123 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbc0800d570 con 0x7fbc20066fe0 2022-06-29T15:44:31.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.027+0000 7fbc1f7fe700 1 -- 172.21.15.47:0/2994071123 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc08000e90 con 0x7fbc20066fe0 2022-06-29T15:44:31.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.027+0000 7fbc1f7fe700 1 -- 172.21.15.47:0/2994071123 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbc0800d900 con 0x7fbc20066fe0 2022-06-29T15:44:31.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.027+0000 7fbc2552f700 1 -- 172.21.15.47:0/2994071123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc20066fe0 msgr2=0x7fbc200b4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:31.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.027+0000 7fbc2552f700 1 --2- 172.21.15.47:0/2994071123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc20066fe0 0x7fbc200b4b40 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7fbc08002f70 tx=0x7fbc08006e50).stop 2022-06-29T15:44:31.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.027+0000 7fbc2552f700 1 -- 172.21.15.47:0/2994071123 shutdown_connections 2022-06-29T15:44:31.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.027+0000 7fbc2552f700 1 --2- 172.21.15.47:0/2994071123 >> v2:172.21.15.64:3300/0 conn(0x7fbc20143150 0x7fbc200b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:31.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.027+0000 7fbc2552f700 1 --2- 172.21.15.47:0/2994071123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc20066fe0 0x7fbc200b4b40 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:31.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.027+0000 7fbc2552f700 1 --2- 172.21.15.47:0/2994071123 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbc200b64c0 0x7fbc200b5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:31.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.027+0000 7fbc2552f700 1 -- 172.21.15.47:0/2994071123 >> 172.21.15.47:0/2994071123 conn(0x7fbc20060e00 msgr2=0x7fbc200611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:31.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.027+0000 7fbc2552f700 1 -- 172.21.15.47:0/2994071123 shutdown_connections 2022-06-29T15:44:31.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.027+0000 7fbc2552f700 1 -- 172.21.15.47:0/2994071123 wait complete. 2022-06-29T15:44:31.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.027+0000 7fbc2552f700 1 Processor -- start 2022-06-29T15:44:31.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.027+0000 7fbc2552f700 1 -- start start 2022-06-29T15:44:31.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.027+0000 7fbc2552f700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbc200b64c0 0x7fbc200b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:31.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.027+0000 7fbc2552f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc20143150 0x7fbc200ad560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:31.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.027+0000 7fbc2552f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc20145c30 0x7fbc20152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:31.045 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.027+0000 7fbc2552f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc201521f0 con 0x7fbc20143150 2022-06-29T15:44:31.045 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.027+0000 7fbc2552f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc200b7fa0 con 0x7fbc20145c30 2022-06-29T15:44:31.045 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.027+0000 7fbc2552f700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc201460d0 con 0x7fbc200b64c0 2022-06-29T15:44:31.045 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.027+0000 7fbc177fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc20143150 0x7fbc200ad560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:31.045 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.031+0000 7fbc177fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc20143150 0x7fbc200ad560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47916/0 (socket says 172.21.15.47:47916) 2022-06-29T15:44:31.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.031+0000 7fbc177fe700 1 -- 172.21.15.47:0/2652529130 learned_addr learned my addr 172.21.15.47:0/2652529130 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:31.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.031+0000 7fbc24d2e700 1 --2- 172.21.15.47:0/2652529130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc20145c30 0x7fbc20152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:31.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.031+0000 7fbc1ffff700 1 --2- 172.21.15.47:0/2652529130 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbc200b64c0 0x7fbc200b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:31.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.031+0000 7fbc177fe700 1 -- 172.21.15.47:0/2652529130 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbc200b64c0 msgr2=0x7fbc200b7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:31.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.031+0000 7fbc177fe700 1 --2- 172.21.15.47:0/2652529130 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbc200b64c0 0x7fbc200b7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:31.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.031+0000 7fbc177fe700 1 -- 172.21.15.47:0/2652529130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc20145c30 msgr2=0x7fbc20152b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:31.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.031+0000 7fbc177fe700 1 --2- 172.21.15.47:0/2652529130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc20145c30 0x7fbc20152b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:31.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.031+0000 7fbc177fe700 1 -- 172.21.15.47:0/2652529130 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc0800b1e0 con 0x7fbc20143150 2022-06-29T15:44:31.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.031+0000 7fbc177fe700 1 --2- 172.21.15.47:0/2652529130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc20143150 0x7fbc200ad560 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7fbc08002f70 tx=0x7fbc08015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:31.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.031+0000 7fbc1dffb700 1 -- 172.21.15.47:0/2652529130 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbc0800d8f0 con 0x7fbc20143150 2022-06-29T15:44:31.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.031+0000 7fbc1dffb700 1 -- 172.21.15.47:0/2652529130 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc08006ae0 con 0x7fbc20143150 2022-06-29T15:44:31.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.031+0000 7fbc2552f700 1 -- 172.21.15.47:0/2652529130 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc201523e0 con 0x7fbc20143150 2022-06-29T15:44:31.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.031+0000 7fbc1dffb700 1 -- 172.21.15.47:0/2652529130 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbc08011d70 con 0x7fbc20143150 2022-06-29T15:44:31.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.031+0000 7fbc2552f700 1 -- 172.21.15.47:0/2652529130 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc200b0310 con 0x7fbc20143150 2022-06-29T15:44:31.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.031+0000 7fbc2552f700 1 -- 172.21.15.47:0/2652529130 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc200b8cd0 con 0x7fbc20143150 2022-06-29T15:44:31.052 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.031+0000 7fbc1dffb700 1 -- 172.21.15.47:0/2652529130 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fbc0801a040 con 0x7fbc20143150 2022-06-29T15:44:31.052 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.035+0000 7fbc1dffb700 1 --2- 172.21.15.47:0/2652529130 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbc04053b00 0x7fbc04055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:31.052 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.035+0000 7fbc1dffb700 1 -- 172.21.15.47:0/2652529130 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fbc0806cd90 con 0x7fbc20143150 2022-06-29T15:44:31.052 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.035+0000 7fbc1ffff700 1 --2- 172.21.15.47:0/2652529130 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbc04053b00 0x7fbc04055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:31.053 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.035+0000 7fbc1ffff700 1 --2- 172.21.15.47:0/2652529130 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbc04053b00 0x7fbc04055fc0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7fbc1000c000 tx=0x7fbc10008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:31.053 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.039+0000 7fbc1dffb700 1 -- 172.21.15.47:0/2652529130 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbc08043ab0 con 0x7fbc20143150 2022-06-29T15:44:31.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.167+0000 7fbc2552f700 1 -- 172.21.15.47:0/2652529130 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fbc20146290 con 0x7fbc20143150 2022-06-29T15:44:31.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.171+0000 7fbc1dffb700 1 -- 172.21.15.47:0/2652529130 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v47) v1 ==== 110+0+10 (secure 0 0 0) 0x7fbc08023020 con 0x7fbc20143150 2022-06-29T15:44:31.175 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2022-06-29T15:44:31.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.171+0000 7fbc2552f700 1 -- 172.21.15.47:0/2652529130 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbc04053b00 msgr2=0x7fbc04055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:31.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.171+0000 7fbc2552f700 1 --2- 172.21.15.47:0/2652529130 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbc04053b00 0x7fbc04055fc0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7fbc1000c000 tx=0x7fbc10008040).stop 2022-06-29T15:44:31.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.171+0000 7fbc2552f700 1 -- 172.21.15.47:0/2652529130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc20143150 msgr2=0x7fbc200ad560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:31.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.171+0000 7fbc2552f700 1 --2- 172.21.15.47:0/2652529130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc20143150 0x7fbc200ad560 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7fbc08002f70 tx=0x7fbc08015000).stop 2022-06-29T15:44:31.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.171+0000 7fbc2552f700 1 -- 172.21.15.47:0/2652529130 shutdown_connections 2022-06-29T15:44:31.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.171+0000 7fbc2552f700 1 --2- 172.21.15.47:0/2652529130 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbc04053b00 0x7fbc04055fc0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:31.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.171+0000 7fbc2552f700 1 --2- 172.21.15.47:0/2652529130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc20145c30 0x7fbc20152b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:31.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.171+0000 7fbc2552f700 1 --2- 172.21.15.47:0/2652529130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc20143150 0x7fbc200ad560 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:31.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.171+0000 7fbc2552f700 1 --2- 172.21.15.47:0/2652529130 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbc200b64c0 0x7fbc200b7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:31.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.171+0000 7fbc2552f700 1 -- 172.21.15.47:0/2652529130 >> 172.21.15.47:0/2652529130 conn(0x7fbc20060e00 msgr2=0x7fbc20148050 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:31.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.171+0000 7fbc2552f700 1 -- 172.21.15.47:0/2652529130 shutdown_connections 2022-06-29T15:44:31.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.171+0000 7fbc2552f700 1 -- 172.21.15.47:0/2652529130 wait complete. 2022-06-29T15:44:31.188 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-29T15:44:31.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.295+0000 7f8fda7a5700 1 Processor -- start 2022-06-29T15:44:31.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.295+0000 7f8fda7a5700 1 -- start start 2022-06-29T15:44:31.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.299+0000 7f8fda7a5700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8fd4063100 0x7f8fd40b1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:31.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.299+0000 7f8fda7a5700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fd40b2ce0 con 0x7f8fd40612b0 2022-06-29T15:44:31.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.299+0000 7f8fda7a5700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fd40b71a0 con 0x7f8fd40b6dc0 2022-06-29T15:44:31.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.299+0000 7f8fda7a5700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fd40b0e00 con 0x7f8fd4063100 2022-06-29T15:44:31.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.299+0000 7f8fd9fa4700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f8fd40612b0 0x7f8fd41402d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33478/0 (socket says 172.21.15.47:33478) 2022-06-29T15:44:31.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.299+0000 7f8fd9fa4700 1 -- 172.21.15.47:0/2259127142 learned_addr learned my addr 172.21.15.47:0/2259127142 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:44:31.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.299+0000 7f8fd8fa2700 1 --2- 172.21.15.47:0/2259127142 >> v2:172.21.15.64:3300/0 conn(0x7f8fd4063100 0x7f8fd40b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:31.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.299+0000 7f8fd8fa2700 1 -- 172.21.15.47:0/2259127142 >> v1:172.21.15.64:6789/0 conn(0x7f8fd40b6dc0 legacy=0x7f8fd40b11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:31.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.299+0000 7f8fd8fa2700 1 -- 172.21.15.47:0/2259127142 >> v1:172.21.15.47:6789/0 conn(0x7f8fd40612b0 legacy=0x7f8fd41402d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:31.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.299+0000 7f8fd8fa2700 1 -- 172.21.15.47:0/2259127142 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd40620e0 con 0x7f8fd4063100 2022-06-29T15:44:31.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.299+0000 7f8fd8fa2700 1 --2- 172.21.15.47:0/2259127142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fd4063100 0x7f8fd40b1900 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f8fc4002f70 tx=0x7f8fc4006df0).ready entity=mon.1 client_cookie=554edf76852799db server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:31.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.299+0000 7f8fcbfff700 1 -- 172.21.15.47:0/2259127142 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fc400d610 con 0x7f8fd4063100 2022-06-29T15:44:31.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.303+0000 7f8fcbfff700 1 -- 172.21.15.47:0/2259127142 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fc4000e90 con 0x7f8fd4063100 2022-06-29T15:44:31.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.303+0000 7f8fcbfff700 1 -- 172.21.15.47:0/2259127142 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fc400d960 con 0x7f8fd4063100 2022-06-29T15:44:31.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.303+0000 7f8fda7a5700 1 -- 172.21.15.47:0/2259127142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fd4063100 msgr2=0x7f8fd40b1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:31.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.303+0000 7f8fda7a5700 1 --2- 172.21.15.47:0/2259127142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fd4063100 0x7f8fd40b1900 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f8fc4002f70 tx=0x7f8fc4006df0).stop 2022-06-29T15:44:31.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.303+0000 7f8fda7a5700 1 -- 172.21.15.47:0/2259127142 shutdown_connections 2022-06-29T15:44:31.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.303+0000 7f8fda7a5700 1 --2- 172.21.15.47:0/2259127142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fd4063100 0x7f8fd40b1900 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:31.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.303+0000 7f8fda7a5700 1 -- 172.21.15.47:0/2259127142 >> 172.21.15.47:0/2259127142 conn(0x7f8fd4063670 msgr2=0x7f8fd4063a50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:31.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.303+0000 7f8fda7a5700 1 -- 172.21.15.47:0/2259127142 shutdown_connections 2022-06-29T15:44:31.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.303+0000 7f8fda7a5700 1 -- 172.21.15.47:0/2259127142 wait complete. 2022-06-29T15:44:31.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.307+0000 7f8fda7a5700 1 Processor -- start 2022-06-29T15:44:31.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.307+0000 7f8fda7a5700 1 -- start start 2022-06-29T15:44:31.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.307+0000 7f8fda7a5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fd40612b0 0x7f8fd40b0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:31.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.307+0000 7f8fda7a5700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8fd4063100 0x7f8fd40adcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:31.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.307+0000 7f8fda7a5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fd40b6dc0 0x7f8fd40ae1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:31.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.307+0000 7f8fda7a5700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd413cc00 con 0x7f8fd40b6dc0 2022-06-29T15:44:31.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.307+0000 7f8fda7a5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd40aecd0 con 0x7f8fd40612b0 2022-06-29T15:44:31.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.307+0000 7f8fda7a5700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd40ae6b0 con 0x7f8fd4063100 2022-06-29T15:44:31.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.307+0000 7f8fd9fa4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fd40b6dc0 0x7f8fd40ae1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:31.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.307+0000 7f8fd9fa4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fd40b6dc0 0x7f8fd40ae1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47930/0 (socket says 172.21.15.47:47930) 2022-06-29T15:44:31.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.307+0000 7f8fd9fa4700 1 -- 172.21.15.47:0/2597565960 learned_addr learned my addr 172.21.15.47:0/2597565960 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:31.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.307+0000 7f8fd8fa2700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8fd4063100 0x7f8fd40adcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:31.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.307+0000 7f8fd97a3700 1 --2- 172.21.15.47:0/2597565960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fd40612b0 0x7f8fd40b0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:31.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.307+0000 7f8fd9fa4700 1 -- 172.21.15.47:0/2597565960 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8fd4063100 msgr2=0x7f8fd40adcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:31.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.307+0000 7f8fd9fa4700 1 --2- 172.21.15.47:0/2597565960 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8fd4063100 0x7f8fd40adcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:31.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.307+0000 7f8fd9fa4700 1 -- 172.21.15.47:0/2597565960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fd40612b0 msgr2=0x7f8fd40b0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:31.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.307+0000 7f8fd9fa4700 1 --2- 172.21.15.47:0/2597565960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fd40612b0 0x7f8fd40b0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:31.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.307+0000 7f8fd9fa4700 1 -- 172.21.15.47:0/2597565960 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc400b180 con 0x7f8fd40b6dc0 2022-06-29T15:44:31.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.307+0000 7f8fd9fa4700 1 --2- 172.21.15.47:0/2597565960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fd40b6dc0 0x7f8fd40ae1c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7f8fd40645d0 tx=0x7f8fcc007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:31.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.307+0000 7f8fca7fc700 1 -- 172.21.15.47:0/2597565960 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fcc005ba0 con 0x7f8fd40b6dc0 2022-06-29T15:44:31.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.307+0000 7f8fda7a5700 1 -- 172.21.15.47:0/2597565960 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fd413db60 con 0x7f8fd40b6dc0 2022-06-29T15:44:31.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.307+0000 7f8fca7fc700 1 -- 172.21.15.47:0/2597565960 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fcc005d00 con 0x7f8fd40b6dc0 2022-06-29T15:44:31.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.307+0000 7f8fca7fc700 1 -- 172.21.15.47:0/2597565960 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fcc0072c0 con 0x7f8fd40b6dc0 2022-06-29T15:44:31.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.307+0000 7f8fda7a5700 1 -- 172.21.15.47:0/2597565960 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fd4064830 con 0x7f8fd40b6dc0 2022-06-29T15:44:31.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.311+0000 7f8fda7a5700 1 -- 172.21.15.47:0/2597565960 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fd40ada10 con 0x7f8fd40b6dc0 2022-06-29T15:44:31.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.311+0000 7f8fca7fc700 1 -- 172.21.15.47:0/2597565960 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8fcc010070 con 0x7f8fd40b6dc0 2022-06-29T15:44:31.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.311+0000 7f8fca7fc700 1 --2- 172.21.15.47:0/2597565960 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8fbc0539e0 0x7f8fbc055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:31.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.311+0000 7f8fca7fc700 1 -- 172.21.15.47:0/2597565960 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f8fcc06b9a0 con 0x7f8fd40b6dc0 2022-06-29T15:44:31.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.311+0000 7f8fd97a3700 1 --2- 172.21.15.47:0/2597565960 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8fbc0539e0 0x7f8fbc055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:31.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.311+0000 7f8fd97a3700 1 --2- 172.21.15.47:0/2597565960 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8fbc0539e0 0x7f8fbc055ea0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f8fd000d000 tx=0x7f8fd0007b90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:31.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.315+0000 7f8fca7fc700 1 -- 172.21.15.47:0/2597565960 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8fd40ada10 con 0x7f8fd40b6dc0 2022-06-29T15:44:31.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.439+0000 7f8fda7a5700 1 -- 172.21.15.47:0/2597565960 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8fd40ada10 con 0x7f8fd40b6dc0 2022-06-29T15:44:31.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.439+0000 7f8fca7fc700 1 -- 172.21.15.47:0/2597565960 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v47) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8fd40ada10 con 0x7f8fd40b6dc0 2022-06-29T15:44:31.446 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:44:31.447 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.439+0000 7f8fda7a5700 1 -- 172.21.15.47:0/2597565960 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8fbc0539e0 msgr2=0x7f8fbc055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:31.447 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.439+0000 7f8fda7a5700 1 --2- 172.21.15.47:0/2597565960 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8fbc0539e0 0x7f8fbc055ea0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f8fd000d000 tx=0x7f8fd0007b90).stop 2022-06-29T15:44:31.447 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.439+0000 7f8fda7a5700 1 -- 172.21.15.47:0/2597565960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fd40b6dc0 msgr2=0x7f8fd40ae1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:31.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.439+0000 7f8fda7a5700 1 --2- 172.21.15.47:0/2597565960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fd40b6dc0 0x7f8fd40ae1c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7f8fd40645d0 tx=0x7f8fcc007f40).stop 2022-06-29T15:44:31.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.443+0000 7f8fda7a5700 1 -- 172.21.15.47:0/2597565960 shutdown_connections 2022-06-29T15:44:31.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.443+0000 7f8fda7a5700 1 --2- 172.21.15.47:0/2597565960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fd40b6dc0 0x7f8fd40ae1c0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:31.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.443+0000 7f8fda7a5700 1 --2- 172.21.15.47:0/2597565960 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8fd4063100 0x7f8fd40adcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:31.449 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.443+0000 7f8fda7a5700 1 --2- 172.21.15.47:0/2597565960 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8fbc0539e0 0x7f8fbc055ea0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:31.449 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.443+0000 7f8fda7a5700 1 --2- 172.21.15.47:0/2597565960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fd40612b0 0x7f8fd40b0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:31.449 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.443+0000 7f8fda7a5700 1 -- 172.21.15.47:0/2597565960 >> 172.21.15.47:0/2597565960 conn(0x7f8fd4063670 msgr2=0x7f8fd40b4510 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:31.450 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.443+0000 7f8fda7a5700 1 -- 172.21.15.47:0/2597565960 shutdown_connections 2022-06-29T15:44:31.450 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.443+0000 7f8fda7a5700 1 -- 172.21.15.47:0/2597565960 wait complete. 2022-06-29T15:44:31.458 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-29T15:44:31.570 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.563+0000 7f9bff59e700 1 Processor -- start 2022-06-29T15:44:31.570 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.563+0000 7f9bff59e700 1 -- start start 2022-06-29T15:44:31.573 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.567+0000 7f9bff59e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9c000b5d90 0x7f9c00143a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:31.573 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.567+0000 7f9bff59e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c0005cc30 con 0x7f9c0005c850 2022-06-29T15:44:31.573 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.567+0000 7f9bff59e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c001430a0 con 0x7f9c000676a0 2022-06-29T15:44:31.574 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.567+0000 7f9bff59e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c00143300 con 0x7f9c000b5d90 2022-06-29T15:44:31.574 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.567+0000 7f9bfdd9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9c000b5d90 0x7f9c00143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:31.574 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.567+0000 7f9bfe59c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9c0005c850 0x7f9c000b4860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33488/0 (socket says 172.21.15.47:33488) 2022-06-29T15:44:31.574 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.567+0000 7f9bfe59c700 1 -- 172.21.15.47:0/1750901562 learned_addr learned my addr 172.21.15.47:0/1750901562 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:44:31.575 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.567+0000 7f9bfdd9b700 1 -- 172.21.15.47:0/1750901562 >> v1:172.21.15.64:6789/0 conn(0x7f9c000676a0 legacy=0x7f9c00140130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:31.575 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.567+0000 7f9bfdd9b700 1 -- 172.21.15.47:0/1750901562 >> v1:172.21.15.47:6789/0 conn(0x7f9c0005c850 legacy=0x7f9c000b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:31.575 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.567+0000 7f9bfdd9b700 1 -- 172.21.15.47:0/1750901562 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c001435a0 con 0x7f9c000b5d90 2022-06-29T15:44:31.575 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.567+0000 7f9bfd59a700 1 -- 172.21.15.47:0/1750901562 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2953724872 0 0) 0x7f9c0005cc30 con 0x7f9c0005c850 2022-06-29T15:44:31.575 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.567+0000 7f9bfdd9b700 1 --2- 172.21.15.47:0/1750901562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c000b5d90 0x7f9c00143a40 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f9be8002f70 tx=0x7f9be8006df0).ready entity=mon.0 client_cookie=4896f2e739efbcd4 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:31.576 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.567+0000 7f9bfd59a700 1 -- 172.21.15.47:0/1750901562 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9be800d610 con 0x7f9c000b5d90 2022-06-29T15:44:31.576 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.567+0000 7f9bfd59a700 1 -- 172.21.15.47:0/1750901562 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9be8000e90 con 0x7f9c000b5d90 2022-06-29T15:44:31.576 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.567+0000 7f9bfd59a700 1 -- 172.21.15.47:0/1750901562 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9be800d960 con 0x7f9c000b5d90 2022-06-29T15:44:31.577 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.567+0000 7f9bff59e700 1 -- 172.21.15.47:0/1750901562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c000b5d90 msgr2=0x7f9c00143a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:31.578 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.567+0000 7f9bff59e700 1 --2- 172.21.15.47:0/1750901562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c000b5d90 0x7f9c00143a40 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f9be8002f70 tx=0x7f9be8006df0).stop 2022-06-29T15:44:31.578 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.567+0000 7f9bff59e700 1 -- 172.21.15.47:0/1750901562 shutdown_connections 2022-06-29T15:44:31.578 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.567+0000 7f9bff59e700 1 --2- 172.21.15.47:0/1750901562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c000b5d90 0x7f9c00143a40 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:31.578 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.567+0000 7f9bff59e700 1 -- 172.21.15.47:0/1750901562 >> 172.21.15.47:0/1750901562 conn(0x7f9c00060e00 msgr2=0x7f9c000611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:31.579 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.567+0000 7f9bff59e700 1 -- 172.21.15.47:0/1750901562 shutdown_connections 2022-06-29T15:44:31.579 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.567+0000 7f9bff59e700 1 -- 172.21.15.47:0/1750901562 wait complete. 2022-06-29T15:44:31.579 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.571+0000 7f9bff59e700 1 Processor -- start 2022-06-29T15:44:31.579 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.571+0000 7f9bff59e700 1 -- start start 2022-06-29T15:44:31.582 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.571+0000 7f9bff59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c0005c850 0x7f9c000b97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:31.582 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.571+0000 7f9bff59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c000676a0 0x7f9c000b13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:31.583 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.571+0000 7f9bff59e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9c000b5d90 0x7f9c000b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:31.583 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.571+0000 7f9bff59e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c000b8c70 con 0x7f9c0005c850 2022-06-29T15:44:31.583 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.571+0000 7f9bff59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c000ba1f0 con 0x7f9c000676a0 2022-06-29T15:44:31.583 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.571+0000 7f9bff59e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c00152d20 con 0x7f9c000b5d90 2022-06-29T15:44:31.584 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.571+0000 7f9bfe59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c0005c850 0x7f9c000b97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:31.584 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.571+0000 7f9bfe59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c0005c850 0x7f9c000b97f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47940/0 (socket says 172.21.15.47:47940) 2022-06-29T15:44:31.584 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.571+0000 7f9bfe59c700 1 -- 172.21.15.47:0/3430733315 learned_addr learned my addr 172.21.15.47:0/3430733315 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:31.584 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.575+0000 7f9bfdd9b700 1 --2- 172.21.15.47:0/3430733315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c000676a0 0x7f9c000b13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:31.585 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.575+0000 7f9bfed9d700 1 --2- 172.21.15.47:0/3430733315 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9c000b5d90 0x7f9c000b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:31.585 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.575+0000 7f9bfe59c700 1 -- 172.21.15.47:0/3430733315 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9c000b5d90 msgr2=0x7f9c000b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:31.585 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.575+0000 7f9bfe59c700 1 --2- 172.21.15.47:0/3430733315 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9c000b5d90 0x7f9c000b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:31.585 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.575+0000 7f9bfe59c700 1 -- 172.21.15.47:0/3430733315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c000676a0 msgr2=0x7f9c000b13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:31.585 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.575+0000 7f9bfe59c700 1 --2- 172.21.15.47:0/3430733315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c000676a0 0x7f9c000b13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:31.586 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.575+0000 7f9bfe59c700 1 -- 172.21.15.47:0/3430733315 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be800b180 con 0x7f9c0005c850 2022-06-29T15:44:31.586 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.575+0000 7f9bfed9d700 1 --2- 172.21.15.47:0/3430733315 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9c000b5d90 0x7f9c000b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:44:31.586 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.575+0000 7f9bfe59c700 1 --2- 172.21.15.47:0/3430733315 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c0005c850 0x7f9c000b97f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f9bf0000bc0 tx=0x7f9bf0001800).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:31.586 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.575+0000 7f9bf77fe700 1 -- 172.21.15.47:0/3430733315 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9bf00070d0 con 0x7f9c0005c850 2022-06-29T15:44:31.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.575+0000 7f9bf77fe700 1 -- 172.21.15.47:0/3430733315 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf0007230 con 0x7f9c0005c850 2022-06-29T15:44:31.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.575+0000 7f9bff59e700 1 -- 172.21.15.47:0/3430733315 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c000b0b50 con 0x7f9c0005c850 2022-06-29T15:44:31.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.575+0000 7f9bf77fe700 1 -- 172.21.15.47:0/3430733315 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9bf00073a0 con 0x7f9c0005c850 2022-06-29T15:44:31.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.575+0000 7f9bff59e700 1 -- 172.21.15.47:0/3430733315 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c000b18b0 con 0x7f9c0005c850 2022-06-29T15:44:31.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.575+0000 7f9bff59e700 1 -- 172.21.15.47:0/3430733315 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c000ba3f0 con 0x7f9c0005c850 2022-06-29T15:44:31.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.575+0000 7f9bf77fe700 1 -- 172.21.15.47:0/3430733315 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9bf0010070 con 0x7f9c0005c850 2022-06-29T15:44:31.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.575+0000 7f9bf77fe700 1 --2- 172.21.15.47:0/3430733315 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9be4053aa0 0x7f9be4055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:31.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.575+0000 7f9bf77fe700 1 -- 172.21.15.47:0/3430733315 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9bf006b880 con 0x7f9c0005c850 2022-06-29T15:44:31.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.575+0000 7f9bfdd9b700 1 --2- 172.21.15.47:0/3430733315 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9be4053aa0 0x7f9be4055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:31.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.579+0000 7f9bfdd9b700 1 --2- 172.21.15.47:0/3430733315 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9be4053aa0 0x7f9be4055f60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f9be801c000 tx=0x7f9be8015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:31.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.587+0000 7f9bf77fe700 1 -- 172.21.15.47:0/3430733315 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9bf00424b0 con 0x7f9c0005c850 2022-06-29T15:44:31.722 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.715+0000 7f9bff59e700 1 -- 172.21.15.47:0/3430733315 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9c000573d0 con 0x7f9c0005c850 2022-06-29T15:44:31.722 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.719+0000 7f9bf77fe700 1 -- 172.21.15.47:0/3430733315 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v47) v1 ==== 104+0+10 (secure 0 0 0) 0x7f9bf0045ad0 con 0x7f9c0005c850 2022-06-29T15:44:31.723 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:44:31.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.719+0000 7f9bff59e700 1 -- 172.21.15.47:0/3430733315 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9be4053aa0 msgr2=0x7f9be4055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:31.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.719+0000 7f9bff59e700 1 --2- 172.21.15.47:0/3430733315 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9be4053aa0 0x7f9be4055f60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f9be801c000 tx=0x7f9be8015000).stop 2022-06-29T15:44:31.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.719+0000 7f9bff59e700 1 -- 172.21.15.47:0/3430733315 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c0005c850 msgr2=0x7f9c000b97f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:31.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.719+0000 7f9bff59e700 1 --2- 172.21.15.47:0/3430733315 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c0005c850 0x7f9c000b97f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f9bf0000bc0 tx=0x7f9bf0001800).stop 2022-06-29T15:44:31.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.719+0000 7f9bff59e700 1 -- 172.21.15.47:0/3430733315 shutdown_connections 2022-06-29T15:44:31.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.719+0000 7f9bff59e700 1 --2- 172.21.15.47:0/3430733315 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9c000b5d90 0x7f9c000b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:31.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.719+0000 7f9bff59e700 1 --2- 172.21.15.47:0/3430733315 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9be4053aa0 0x7f9be4055f60 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:31.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.719+0000 7f9bff59e700 1 --2- 172.21.15.47:0/3430733315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c000676a0 0x7f9c000b13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:31.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.719+0000 7f9bff59e700 1 --2- 172.21.15.47:0/3430733315 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c0005c850 0x7f9c000b97f0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:31.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.719+0000 7f9bff59e700 1 -- 172.21.15.47:0/3430733315 >> 172.21.15.47:0/3430733315 conn(0x7f9c00060e00 msgr2=0x7f9c00144c50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:31.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.719+0000 7f9bff59e700 1 -- 172.21.15.47:0/3430733315 shutdown_connections 2022-06-29T15:44:31.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.719+0000 7f9bff59e700 1 -- 172.21.15.47:0/3430733315 wait complete. 2022-06-29T15:44:31.739 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-29T15:44:31.848 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.843+0000 7fbb82cbf700 1 Processor -- start 2022-06-29T15:44:31.849 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.843+0000 7fbb82cbf700 1 -- start start 2022-06-29T15:44:31.851 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.843+0000 7fbb82cbf700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbb7c0b5d90 0x7fbb7c143a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:31.851 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.843+0000 7fbb82cbf700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbb7c0676a0 0x7fbb7c140130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:31.852 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.843+0000 7fbb82cbf700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb7c05cc30 con 0x7fbb7c05c850 2022-06-29T15:44:31.852 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.843+0000 7fbb82cbf700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb7c1430a0 con 0x7fbb7c0b5d90 2022-06-29T15:44:31.852 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.843+0000 7fbb82cbf700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb7c143300 con 0x7fbb7c0676a0 2022-06-29T15:44:31.852 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.843+0000 7fbb81cbd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fbb7c05c850 0x7fbb7c0b4860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33502/0 (socket says 172.21.15.47:33502) 2022-06-29T15:44:31.853 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.843+0000 7fbb814bc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbb7c0b5d90 0x7fbb7c143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:31.853 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.843+0000 7fbb81cbd700 1 -- 172.21.15.47:0/2907494615 learned_addr learned my addr 172.21.15.47:0/2907494615 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:44:31.853 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.843+0000 7fbb824be700 1 --2- 172.21.15.47:0/2907494615 >> v2:172.21.15.64:3300/0 conn(0x7fbb7c0676a0 0x7fbb7c140130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:31.853 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.843+0000 7fbb80cbb700 1 -- 172.21.15.47:0/2907494615 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2989366346 0 0) 0x7fbb7c05cc30 con 0x7fbb7c05c850 2022-06-29T15:44:31.854 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.843+0000 7fbb814bc700 1 -- 172.21.15.47:0/2907494615 >> v2:172.21.15.64:3300/0 conn(0x7fbb7c0676a0 msgr2=0x7fbb7c140130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:44:31.854 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.843+0000 7fbb814bc700 1 --2- 172.21.15.47:0/2907494615 >> v2:172.21.15.64:3300/0 conn(0x7fbb7c0676a0 0x7fbb7c140130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:31.854 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.843+0000 7fbb814bc700 1 -- 172.21.15.47:0/2907494615 >> v1:172.21.15.47:6789/0 conn(0x7fbb7c05c850 legacy=0x7fbb7c0b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:31.854 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.843+0000 7fbb814bc700 1 -- 172.21.15.47:0/2907494615 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb7c1435a0 con 0x7fbb7c0b5d90 2022-06-29T15:44:31.855 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.847+0000 7fbb814bc700 1 --2- 172.21.15.47:0/2907494615 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb7c0b5d90 0x7fbb7c143a40 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7fbb70002f70 tx=0x7fbb70006df0).ready entity=mon.0 client_cookie=83b595827e0151de server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:31.855 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.847+0000 7fbb80cbb700 1 -- 172.21.15.47:0/2907494615 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb7000d570 con 0x7fbb7c0b5d90 2022-06-29T15:44:31.855 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.847+0000 7fbb80cbb700 1 -- 172.21.15.47:0/2907494615 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb70000e90 con 0x7fbb7c0b5d90 2022-06-29T15:44:31.855 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.847+0000 7fbb80cbb700 1 -- 172.21.15.47:0/2907494615 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb7000d8c0 con 0x7fbb7c0b5d90 2022-06-29T15:44:31.859 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.847+0000 7fbb82cbf700 1 -- 172.21.15.47:0/2907494615 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb7c0b5d90 msgr2=0x7fbb7c143a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:31.859 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.847+0000 7fbb82cbf700 1 --2- 172.21.15.47:0/2907494615 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb7c0b5d90 0x7fbb7c143a40 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7fbb70002f70 tx=0x7fbb70006df0).stop 2022-06-29T15:44:31.860 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.847+0000 7fbb82cbf700 1 -- 172.21.15.47:0/2907494615 shutdown_connections 2022-06-29T15:44:31.860 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.847+0000 7fbb82cbf700 1 --2- 172.21.15.47:0/2907494615 >> v2:172.21.15.64:3300/0 conn(0x7fbb7c0676a0 0x7fbb7c140130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:31.860 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.847+0000 7fbb82cbf700 1 --2- 172.21.15.47:0/2907494615 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb7c0b5d90 0x7fbb7c143a40 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:31.861 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.847+0000 7fbb82cbf700 1 -- 172.21.15.47:0/2907494615 >> 172.21.15.47:0/2907494615 conn(0x7fbb7c060e00 msgr2=0x7fbb7c0611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:31.861 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.847+0000 7fbb82cbf700 1 -- 172.21.15.47:0/2907494615 shutdown_connections 2022-06-29T15:44:31.861 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.847+0000 7fbb82cbf700 1 -- 172.21.15.47:0/2907494615 wait complete. 2022-06-29T15:44:31.861 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.847+0000 7fbb82cbf700 1 Processor -- start 2022-06-29T15:44:31.862 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.847+0000 7fbb82cbf700 1 -- start start 2022-06-29T15:44:31.862 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.851+0000 7fbb82cbf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb7c05c850 0x7fbb7c0b8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:31.862 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.851+0000 7fbb82cbf700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb7c0676a0 0x7fbb7c155650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:31.862 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.851+0000 7fbb82cbf700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbb7c0b5d90 0x7fbb7c155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:31.863 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.851+0000 7fbb82cbf700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c12c110 con 0x7fbb7c0676a0 2022-06-29T15:44:31.863 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.851+0000 7fbb82cbf700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c0b8610 con 0x7fbb7c05c850 2022-06-29T15:44:31.863 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.851+0000 7fbb82cbf700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c0b75b0 con 0x7fbb7c0b5d90 2022-06-29T15:44:31.864 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.851+0000 7fbb814bc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb7c0676a0 0x7fbb7c155650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:31.865 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.851+0000 7fbb814bc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb7c0676a0 0x7fbb7c155650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47956/0 (socket says 172.21.15.47:47956) 2022-06-29T15:44:31.865 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.851+0000 7fbb814bc700 1 -- 172.21.15.47:0/3846885960 learned_addr learned my addr 172.21.15.47:0/3846885960 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:31.865 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.851+0000 7fbb81cbd700 1 --2- 172.21.15.47:0/3846885960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb7c05c850 0x7fbb7c0b8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:31.865 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.851+0000 7fbb824be700 1 --2- 172.21.15.47:0/3846885960 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbb7c0b5d90 0x7fbb7c155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:31.866 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.851+0000 7fbb814bc700 1 -- 172.21.15.47:0/3846885960 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbb7c0b5d90 msgr2=0x7fbb7c155b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:31.866 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.851+0000 7fbb814bc700 1 --2- 172.21.15.47:0/3846885960 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbb7c0b5d90 0x7fbb7c155b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:31.866 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.851+0000 7fbb814bc700 1 -- 172.21.15.47:0/3846885960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb7c05c850 msgr2=0x7fbb7c0b8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:31.867 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.851+0000 7fbb814bc700 1 --2- 172.21.15.47:0/3846885960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb7c05c850 0x7fbb7c0b8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:31.867 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.851+0000 7fbb814bc700 1 -- 172.21.15.47:0/3846885960 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb7000b180 con 0x7fbb7c0676a0 2022-06-29T15:44:31.867 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.851+0000 7fbb814bc700 1 --2- 172.21.15.47:0/3846885960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb7c0676a0 0x7fbb7c155650 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7fbb70002f70 tx=0x7fbb70014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:31.867 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.851+0000 7fbb7affd700 1 -- 172.21.15.47:0/3846885960 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb70016070 con 0x7fbb7c0676a0 2022-06-29T15:44:31.867 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.851+0000 7fbb7affd700 1 -- 172.21.15.47:0/3846885960 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb700138d0 con 0x7fbb7c0676a0 2022-06-29T15:44:31.868 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.851+0000 7fbb82cbf700 1 -- 172.21.15.47:0/3846885960 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb7c156030 con 0x7fbb7c0676a0 2022-06-29T15:44:31.868 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.851+0000 7fbb7affd700 1 -- 172.21.15.47:0/3846885960 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb70013c30 con 0x7fbb7c0676a0 2022-06-29T15:44:31.868 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.851+0000 7fbb82cbf700 1 -- 172.21.15.47:0/3846885960 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb7c0b8370 con 0x7fbb7c0676a0 2022-06-29T15:44:31.868 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.851+0000 7fbb7affd700 1 -- 172.21.15.47:0/3846885960 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fbb70025030 con 0x7fbb7c0676a0 2022-06-29T15:44:31.869 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.851+0000 7fbb82cbf700 1 -- 172.21.15.47:0/3846885960 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb7c0b76f0 con 0x7fbb7c0676a0 2022-06-29T15:44:31.869 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.855+0000 7fbb7affd700 1 --2- 172.21.15.47:0/3846885960 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbb5c053b00 0x7fbb5c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:31.869 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.855+0000 7fbb7affd700 1 -- 172.21.15.47:0/3846885960 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fbb7006ce40 con 0x7fbb7c0676a0 2022-06-29T15:44:31.869 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.855+0000 7fbb81cbd700 1 --2- 172.21.15.47:0/3846885960 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbb5c053b00 0x7fbb5c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:31.870 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.855+0000 7fbb81cbd700 1 --2- 172.21.15.47:0/3846885960 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbb5c053b00 0x7fbb5c055fc0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7fbb6c00d000 tx=0x7fbb6c009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:31.871 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.859+0000 7fbb7affd700 1 -- 172.21.15.47:0/3846885960 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbb70043c00 con 0x7fbb7c0676a0 2022-06-29T15:44:31.988 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.983+0000 7fbb82cbf700 1 -- 172.21.15.47:0/3846885960 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbb7c061b00 con 0x7fbb7c0676a0 2022-06-29T15:44:31.989 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.983+0000 7fbb7affd700 1 -- 172.21.15.47:0/3846885960 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v47) v1 ==== 100+0+10 (secure 0 0 0) 0x7fbb70023070 con 0x7fbb7c0676a0 2022-06-29T15:44:31.990 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:44:31.991 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.987+0000 7fbb82cbf700 1 -- 172.21.15.47:0/3846885960 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbb5c053b00 msgr2=0x7fbb5c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:31.992 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.987+0000 7fbb82cbf700 1 --2- 172.21.15.47:0/3846885960 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbb5c053b00 0x7fbb5c055fc0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7fbb6c00d000 tx=0x7fbb6c009040).stop 2022-06-29T15:44:31.992 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.987+0000 7fbb82cbf700 1 -- 172.21.15.47:0/3846885960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb7c0676a0 msgr2=0x7fbb7c155650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:31.992 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.987+0000 7fbb82cbf700 1 --2- 172.21.15.47:0/3846885960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb7c0676a0 0x7fbb7c155650 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7fbb70002f70 tx=0x7fbb70014000).stop 2022-06-29T15:44:31.993 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.987+0000 7fbb82cbf700 1 -- 172.21.15.47:0/3846885960 shutdown_connections 2022-06-29T15:44:31.993 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.987+0000 7fbb82cbf700 1 --2- 172.21.15.47:0/3846885960 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbb7c0b5d90 0x7fbb7c155b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:31.993 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.987+0000 7fbb82cbf700 1 --2- 172.21.15.47:0/3846885960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb7c0676a0 0x7fbb7c155650 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:31.993 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.987+0000 7fbb82cbf700 1 --2- 172.21.15.47:0/3846885960 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbb5c053b00 0x7fbb5c055fc0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:31.994 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.987+0000 7fbb82cbf700 1 --2- 172.21.15.47:0/3846885960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb7c05c850 0x7fbb7c0b8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:31.994 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.987+0000 7fbb82cbf700 1 -- 172.21.15.47:0/3846885960 >> 172.21.15.47:0/3846885960 conn(0x7fbb7c060e00 msgr2=0x7fbb7c0b0010 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:31.994 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.987+0000 7fbb82cbf700 1 -- 172.21.15.47:0/3846885960 shutdown_connections 2022-06-29T15:44:31.994 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:31.987+0000 7fbb82cbf700 1 -- 172.21.15.47:0/3846885960 wait complete. 2022-06-29T15:44:32.002 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-29T15:44:32.003 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:44:32.105 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.099+0000 7fa14a8ba700 1 Processor -- start 2022-06-29T15:44:32.106 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.099+0000 7fa14a8ba700 1 -- start start 2022-06-29T15:44:32.109 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.099+0000 7fa14a8ba700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa1440b5c60 0x7fa144144980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:32.109 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.099+0000 7fa14a8ba700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa144060eb0 0x7fa144144e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:32.109 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.099+0000 7fa14a8ba700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1440a9120 con 0x7fa1440a72f0 2022-06-29T15:44:32.109 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.099+0000 7fa14a8ba700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa144145360 con 0x7fa1440b5c60 2022-06-29T15:44:32.110 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.099+0000 7fa14a8ba700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1440b0b80 con 0x7fa144060eb0 2022-06-29T15:44:32.110 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.099+0000 7fa14a0b9700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa1440a72f0 0x7fa1440b1e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33518/0 (socket says 172.21.15.47:33518) 2022-06-29T15:44:32.110 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.099+0000 7fa14a0b9700 1 -- 172.21.15.47:0/2776750325 learned_addr learned my addr 172.21.15.47:0/2776750325 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:44:32.110 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.099+0000 7fa1498b8700 1 --2- 172.21.15.47:0/2776750325 >> v2:172.21.15.47:3300/0 conn(0x7fa1440b5c60 0x7fa144144980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:32.111 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.103+0000 7fa1490b7700 1 --2- 172.21.15.47:0/2776750325 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa144060eb0 0x7fa144144e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:32.111 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.103+0000 7fa1488b6700 1 -- 172.21.15.47:0/2776750325 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4049189662 0 0) 0x7fa1440a9120 con 0x7fa1440a72f0 2022-06-29T15:44:32.111 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.103+0000 7fa1488b6700 1 -- 172.21.15.47:0/2776750325 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa12c003260 con 0x7fa1440a72f0 2022-06-29T15:44:32.111 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.103+0000 7fa1498b8700 1 -- 172.21.15.47:0/2776750325 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa144060eb0 msgr2=0x7fa144144e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:44:32.112 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.103+0000 7fa1498b8700 1 --2- 172.21.15.47:0/2776750325 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa144060eb0 0x7fa144144e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:32.112 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.103+0000 7fa1498b8700 1 -- 172.21.15.47:0/2776750325 >> v1:172.21.15.47:6789/0 conn(0x7fa1440a72f0 legacy=0x7fa1440b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:32.112 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.103+0000 7fa1498b8700 1 -- 172.21.15.47:0/2776750325 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1440a89e0 con 0x7fa1440b5c60 2022-06-29T15:44:32.112 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.103+0000 7fa1490b7700 1 --2- 172.21.15.47:0/2776750325 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa144060eb0 0x7fa144144e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:44:32.113 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.103+0000 7fa1498b8700 1 --2- 172.21.15.47:0/2776750325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1440b5c60 0x7fa144144980 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7fa134002f70 tx=0x7fa134006d90).ready entity=mon.0 client_cookie=2e6610f3242df222 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:32.113 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.103+0000 7fa1488b6700 1 -- 172.21.15.47:0/2776750325 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa13400e590 con 0x7fa1440b5c60 2022-06-29T15:44:32.114 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.103+0000 7fa1488b6700 1 -- 172.21.15.47:0/2776750325 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa134000e90 con 0x7fa1440b5c60 2022-06-29T15:44:32.114 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.103+0000 7fa1488b6700 1 -- 172.21.15.47:0/2776750325 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa13400e8e0 con 0x7fa1440b5c60 2022-06-29T15:44:32.115 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.103+0000 7fa14a8ba700 1 -- 172.21.15.47:0/2776750325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1440b5c60 msgr2=0x7fa144144980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:32.115 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.103+0000 7fa14a8ba700 1 --2- 172.21.15.47:0/2776750325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1440b5c60 0x7fa144144980 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7fa134002f70 tx=0x7fa134006d90).stop 2022-06-29T15:44:32.115 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.103+0000 7fa14a8ba700 1 -- 172.21.15.47:0/2776750325 shutdown_connections 2022-06-29T15:44:32.116 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.103+0000 7fa14a8ba700 1 --2- 172.21.15.47:0/2776750325 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa144060eb0 0x7fa144144e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:32.116 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.103+0000 7fa14a8ba700 1 --2- 172.21.15.47:0/2776750325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1440b5c60 0x7fa144144980 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:32.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.103+0000 7fa14a8ba700 1 -- 172.21.15.47:0/2776750325 >> 172.21.15.47:0/2776750325 conn(0x7fa144062560 msgr2=0x7fa144062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:32.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.103+0000 7fa14a8ba700 1 -- 172.21.15.47:0/2776750325 shutdown_connections 2022-06-29T15:44:32.118 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.103+0000 7fa14a8ba700 1 -- 172.21.15.47:0/2776750325 wait complete. 2022-06-29T15:44:32.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.103+0000 7fa14a8ba700 1 Processor -- start 2022-06-29T15:44:32.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.103+0000 7fa14a8ba700 1 -- start start 2022-06-29T15:44:32.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.107+0000 7fa14a8ba700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa144060eb0 0x7fa1440ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:32.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.107+0000 7fa14a8ba700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1440a72f0 0x7fa144159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:32.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.107+0000 7fa14a8ba700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1440b5c60 0x7fa14414ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:32.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.107+0000 7fa14a8ba700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1440a9bc0 con 0x7fa1440a72f0 2022-06-29T15:44:32.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.107+0000 7fa14a8ba700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14414c770 con 0x7fa1440b5c60 2022-06-29T15:44:32.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.107+0000 7fa14a8ba700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa14415a7b0 con 0x7fa144060eb0 2022-06-29T15:44:32.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.107+0000 7fa1490b7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1440a72f0 0x7fa144159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:32.127 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.107+0000 7fa1490b7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1440a72f0 0x7fa144159ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47970/0 (socket says 172.21.15.47:47970) 2022-06-29T15:44:32.127 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.107+0000 7fa1490b7700 1 -- 172.21.15.47:0/1411767311 learned_addr learned my addr 172.21.15.47:0/1411767311 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:32.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.107+0000 7fa1498b8700 1 --2- 172.21.15.47:0/1411767311 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa144060eb0 0x7fa1440ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:32.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.107+0000 7fa14a0b9700 1 --2- 172.21.15.47:0/1411767311 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1440b5c60 0x7fa14414ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:32.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.107+0000 7fa1490b7700 1 -- 172.21.15.47:0/1411767311 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa144060eb0 msgr2=0x7fa1440ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:32.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.107+0000 7fa1490b7700 1 --2- 172.21.15.47:0/1411767311 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa144060eb0 0x7fa1440ad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:32.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.107+0000 7fa1490b7700 1 -- 172.21.15.47:0/1411767311 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1440b5c60 msgr2=0x7fa14414ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:32.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.107+0000 7fa1490b7700 1 --2- 172.21.15.47:0/1411767311 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1440b5c60 0x7fa14414ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:32.130 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.107+0000 7fa1490b7700 1 -- 172.21.15.47:0/1411767311 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa14000b180 con 0x7fa1440a72f0 2022-06-29T15:44:32.130 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.107+0000 7fa1490b7700 1 --2- 172.21.15.47:0/1411767311 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1440a72f0 0x7fa144159ed0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7fa1440aa320 tx=0x7fa140006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:32.131 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.107+0000 7fa132ffd700 1 -- 172.21.15.47:0/1411767311 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa140010590 con 0x7fa1440a72f0 2022-06-29T15:44:32.131 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.107+0000 7fa132ffd700 1 -- 172.21.15.47:0/1411767311 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1400106f0 con 0x7fa1440a72f0 2022-06-29T15:44:32.131 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.107+0000 7fa132ffd700 1 -- 172.21.15.47:0/1411767311 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa140010a50 con 0x7fa1440a72f0 2022-06-29T15:44:32.131 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.107+0000 7fa14a8ba700 1 -- 172.21.15.47:0/1411767311 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa13400b120 con 0x7fa1440a72f0 2022-06-29T15:44:32.132 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.107+0000 7fa14a8ba700 1 -- 172.21.15.47:0/1411767311 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa14414c9d0 con 0x7fa1440a72f0 2022-06-29T15:44:32.132 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.107+0000 7fa14a8ba700 1 -- 172.21.15.47:0/1411767311 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa14414cb90 con 0x7fa1440a72f0 2022-06-29T15:44:32.132 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.111+0000 7fa132ffd700 1 -- 172.21.15.47:0/1411767311 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa140010bb0 con 0x7fa1440a72f0 2022-06-29T15:44:32.132 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.111+0000 7fa132ffd700 1 --2- 172.21.15.47:0/1411767311 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa124053a70 0x7fa124055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:32.132 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.111+0000 7fa132ffd700 1 -- 172.21.15.47:0/1411767311 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa14006cfc0 con 0x7fa1440a72f0 2022-06-29T15:44:32.133 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.111+0000 7fa1498b8700 1 --2- 172.21.15.47:0/1411767311 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa124053a70 0x7fa124055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:32.133 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.111+0000 7fa1498b8700 1 --2- 172.21.15.47:0/1411767311 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa124053a70 0x7fa124055f30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7fa13401a000 tx=0x7fa134014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:32.133 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.115+0000 7fa132ffd700 1 -- 172.21.15.47:0/1411767311 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa140043d80 con 0x7fa1440a72f0 2022-06-29T15:44:32.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.279+0000 7fa14a8ba700 1 -- 172.21.15.47:0/1411767311 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa1440ad650 con 0x7fa1440a72f0 2022-06-29T15:44:32.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.279+0000 7fa132ffd700 1 -- 172.21.15.47:0/1411767311 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+2491 (secure 0 0 0) 0x7fa1400473a0 con 0x7fa1440a72f0 2022-06-29T15:44:32.285 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:44:32.286 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":17,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5259,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3601815346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3601815346},{"type":"v1","addr":"172.21.15.47:6835","nonce":3601815346}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15},{"gid":5262,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/657177602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":657177602},{"type":"v1","addr":"172.21.15.47:6837","nonce":657177602}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:44:27.616735+0000","modified":"2022-06-29T15:44:28.620601+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":54812},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54812":{"gid":54812,"name":"b","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6833/2497658100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2497658100},{"type":"v1","addr":"172.21.15.64:6833","nonce":2497658100}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2022-06-29T15:44:32.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.283+0000 7fa14a8ba700 1 -- 172.21.15.47:0/1411767311 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa124053a70 msgr2=0x7fa124055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:32.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.283+0000 7fa14a8ba700 1 --2- 172.21.15.47:0/1411767311 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa124053a70 0x7fa124055f30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7fa13401a000 tx=0x7fa134014000).stop 2022-06-29T15:44:32.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.283+0000 7fa14a8ba700 1 -- 172.21.15.47:0/1411767311 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1440a72f0 msgr2=0x7fa144159ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:32.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.283+0000 7fa14a8ba700 1 --2- 172.21.15.47:0/1411767311 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1440a72f0 0x7fa144159ed0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7fa1440aa320 tx=0x7fa140006df0).stop 2022-06-29T15:44:32.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.283+0000 7fa14a8ba700 1 -- 172.21.15.47:0/1411767311 shutdown_connections 2022-06-29T15:44:32.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.283+0000 7fa14a8ba700 1 --2- 172.21.15.47:0/1411767311 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa124053a70 0x7fa124055f30 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:32.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.283+0000 7fa14a8ba700 1 --2- 172.21.15.47:0/1411767311 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1440b5c60 0x7fa14414ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:32.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.283+0000 7fa14a8ba700 1 --2- 172.21.15.47:0/1411767311 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1440a72f0 0x7fa144159ed0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:32.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.283+0000 7fa14a8ba700 1 --2- 172.21.15.47:0/1411767311 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa144060eb0 0x7fa1440ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:32.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.283+0000 7fa14a8ba700 1 -- 172.21.15.47:0/1411767311 >> 172.21.15.47:0/1411767311 conn(0x7fa144062560 msgr2=0x7fa144140880 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:32.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.283+0000 7fa14a8ba700 1 -- 172.21.15.47:0/1411767311 shutdown_connections 2022-06-29T15:44:32.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:32.283+0000 7fa14a8ba700 1 -- 172.21.15.47:0/1411767311 wait complete. 2022-06-29T15:44:32.290 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 17 2022-06-29T15:44:32.601 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-29T15:44:27.616735+0000', 'modified': '2022-06-29T15:44:28.620601+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 54812}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_54812': {'gid': 54812, 'name': 'b', 'rank': 0, 'incarnation': 16, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.64:6833/2497658100', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6832', 'nonce': 2497658100}, {'type': 'v1', 'addr': '172.21.15.64:6833', 'nonce': 2497658100}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-29T15:44:32.602 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-29T15:44:32.602 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-06-29T15:44:32.717 INFO:teuthology.orchestra.run.smithi064.stdout:{"cluster_fsid":"9a68894a-53ce-400e-838b-14eab91b834c","whoami":0,"id":54812,"want_state":"up:active","state":"up:active","rank_uptime":5.0844131609999996,"mdsmap_epoch":17,"osdmap_epoch":47,"osdmap_epoch_barrier":47,"uptime":9.8237066689999999} 2022-06-29T15:44:32.718 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9a68894a-53ce-400e-838b-14eab91b834c", "id": 54812, "mdsmap_epoch": 17, "osdmap_epoch": 47, "osdmap_epoch_barrier": 47, "rank_uptime": 5.084413161, "state": "up:active", "uptime": 9.823706669, "want_state": "up:active", "whoami": 0 } 2022-06-29T15:44:32.719 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-29T15:44:32.719 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-06-29T15:44:32.720 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi099.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-06-29T15:44:32.720 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:44:32.721 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:44:32.729 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2022-06-29T15:44:32.747 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:44:32.748 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:44:32.759 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:44:32.760 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:44:32.761 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:44:32.762 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2022-06-29T15:44:32.811 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-29T15:44:32.812 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:44:32.814 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2022-06-29T15:44:32.832 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:44:32.833 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:44:32.844 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:44:32.845 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:44:32.846 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:44:32.846 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2022-06-29T15:44:32.888 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:2022-06-29T15:44:32.881+0000 7f0b425b8080 -1 init, newargv = 0x5615a7929660 newargc=9 2022-06-29T15:44:32.888 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[12394]: starting ceph client 2022-06-29T15:44:32.898 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[12394]: starting fuse 2022-06-29T15:44:33.997 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2022-06-29T15:44:34.016 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:44:34.016 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:44:34.072 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:44:34.073 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:44:34.074 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:44:34.074 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2022-06-29T15:44:34.081 INFO:teuthology.orchestra.run.smithi099.stdout:52 2022-06-29T15:44:34.081 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-29T15:44:34.082 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:44:34.082 DEBUG:teuthology.orchestra.run.smithi099:> sudo python3 -c ' 2022-06-29T15:44:34.083 DEBUG:teuthology.orchestra.run.smithi099:> import glob 2022-06-29T15:44:34.083 DEBUG:teuthology.orchestra.run.smithi099:> import re 2022-06-29T15:44:34.083 DEBUG:teuthology.orchestra.run.smithi099:> import os 2022-06-29T15:44:34.084 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2022-06-29T15:44:34.085 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:44:34.085 DEBUG:teuthology.orchestra.run.smithi099:> def find_socket(client_name): 2022-06-29T15:44:34.085 DEBUG:teuthology.orchestra.run.smithi099:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-06-29T15:44:34.086 DEBUG:teuthology.orchestra.run.smithi099:> files = glob.glob(asok_path) 2022-06-29T15:44:34.086 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:44:34.087 DEBUG:teuthology.orchestra.run.smithi099:> # Given a non-glob path, it better be there 2022-06-29T15:44:34.087 DEBUG:teuthology.orchestra.run.smithi099:> if "*" not in asok_path: 2022-06-29T15:44:34.088 DEBUG:teuthology.orchestra.run.smithi099:> assert(len(files) == 1) 2022-06-29T15:44:34.088 DEBUG:teuthology.orchestra.run.smithi099:> return files[0] 2022-06-29T15:44:34.088 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:44:34.089 DEBUG:teuthology.orchestra.run.smithi099:> for f in files: 2022-06-29T15:44:34.089 DEBUG:teuthology.orchestra.run.smithi099:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-29T15:44:34.090 DEBUG:teuthology.orchestra.run.smithi099:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-29T15:44:34.090 DEBUG:teuthology.orchestra.run.smithi099:> return f 2022-06-29T15:44:34.091 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-29T15:44:34.091 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:44:34.091 DEBUG:teuthology.orchestra.run.smithi099:> print(find_socket("client.0")) 2022-06-29T15:44:34.092 DEBUG:teuthology.orchestra.run.smithi099:> ' 2022-06-29T15:44:34.133 INFO:teuthology.orchestra.run.smithi099.stdout:/var/run/ceph/ceph-client.0.12394.asok 2022-06-29T15:44:34.138 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.12394.asok 2022-06-29T15:44:34.139 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:44:34.139 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.12394.asok status 2022-06-29T15:44:34.225 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2022-06-29T15:44:34.225 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": { 2022-06-29T15:44:34.225 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_sha1": "f25ffac72e5a9e63860ff7deb6767a11f0559a16", 2022-06-29T15:44:34.226 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 15.2.16-304-gf25ffac7 (f25ffac72e5a9e63860ff7deb6767a11f0559a16) octopus (stable)", 2022-06-29T15:44:34.226 INFO:teuthology.orchestra.run.smithi099.stdout: "entity_id": "0", 2022-06-29T15:44:34.226 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": "smithi099", 2022-06-29T15:44:34.226 INFO:teuthology.orchestra.run.smithi099.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-06-29T15:44:34.227 INFO:teuthology.orchestra.run.smithi099.stdout: "pid": "12394", 2022-06-29T15:44:34.227 INFO:teuthology.orchestra.run.smithi099.stdout: "root": "/" 2022-06-29T15:44:34.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:44:34.227 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_count": 0, 2022-06-29T15:44:34.228 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_pinned_count": 0, 2022-06-29T15:44:34.228 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 5374, 2022-06-29T15:44:34.228 INFO:teuthology.orchestra.run.smithi099.stdout: "inst": { 2022-06-29T15:44:34.228 INFO:teuthology.orchestra.run.smithi099.stdout: "name": { 2022-06-29T15:44:34.228 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "client", 2022-06-29T15:44:34.229 INFO:teuthology.orchestra.run.smithi099.stdout: "num": 5374 2022-06-29T15:44:34.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:44:34.229 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2022-06-29T15:44:34.229 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2022-06-29T15:44:34.230 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:0", 2022-06-29T15:44:34.230 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1509445729 2022-06-29T15:44:34.230 INFO:teuthology.orchestra.run.smithi099.stdout: } 2022-06-29T15:44:34.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:44:34.231 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2022-06-29T15:44:34.231 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2022-06-29T15:44:34.231 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:0", 2022-06-29T15:44:34.231 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1509445729 2022-06-29T15:44:34.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:44:34.232 INFO:teuthology.orchestra.run.smithi099.stdout: "inst_str": "client.5374 172.21.15.99:0/1509445729", 2022-06-29T15:44:34.232 INFO:teuthology.orchestra.run.smithi099.stdout: "addr_str": "172.21.15.99:0/1509445729", 2022-06-29T15:44:34.232 INFO:teuthology.orchestra.run.smithi099.stdout: "inode_count": 1, 2022-06-29T15:44:34.233 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_epoch": 17, 2022-06-29T15:44:34.233 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch": 47, 2022-06-29T15:44:34.233 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch_barrier": 0, 2022-06-29T15:44:34.233 INFO:teuthology.orchestra.run.smithi099.stdout: "blacklisted": false, 2022-06-29T15:44:34.234 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs" 2022-06-29T15:44:34.234 INFO:teuthology.orchestra.run.smithi099.stdout:} 2022-06-29T15:44:34.235 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:44:34.236 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:44:34.236 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:44:34.243 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2022-06-29T15:44:34.245 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-29T15:44:34.245 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:44:34.246 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:44:34.307 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:44:34.307 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:44:34.308 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:44:34.316 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2022-06-29T15:44:34.317 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-29T15:44:34.318 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2022-06-29T15:44:34.318 DEBUG:teuthology.orchestra.run.smithi099:> import sys 2022-06-29T15:44:34.319 DEBUG:teuthology.orchestra.run.smithi099:> import time 2022-06-29T15:44:34.319 DEBUG:teuthology.orchestra.run.smithi099:> import os 2022-06-29T15:44:34.319 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:44:34.320 DEBUG:teuthology.orchestra.run.smithi099:> n = 1000 2022-06-29T15:44:34.320 DEBUG:teuthology.orchestra.run.smithi099:> abs_path = "/home/ubuntu/cephtest/mnt.0/dc-dir/dc-file" 2022-06-29T15:44:34.321 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:44:34.321 DEBUG:teuthology.orchestra.run.smithi099:> if not os.path.exists(os.path.dirname(abs_path)): 2022-06-29T15:44:34.321 DEBUG:teuthology.orchestra.run.smithi099:> os.makedirs(os.path.dirname(abs_path)) 2022-06-29T15:44:34.322 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:44:34.322 DEBUG:teuthology.orchestra.run.smithi099:> for i in range(0, n): 2022-06-29T15:44:34.323 DEBUG:teuthology.orchestra.run.smithi099:> fname = "{0}_{1}".format(abs_path, i) 2022-06-29T15:44:34.323 DEBUG:teuthology.orchestra.run.smithi099:> with open(fname, '"'"'w'"'"') as f: 2022-06-29T15:44:34.324 DEBUG:teuthology.orchestra.run.smithi099:> f.write('"'"'content'"'"') 2022-06-29T15:44:34.324 DEBUG:teuthology.orchestra.run.smithi099:> if True: 2022-06-29T15:44:34.325 DEBUG:teuthology.orchestra.run.smithi099:> f.flush() 2022-06-29T15:44:34.325 DEBUG:teuthology.orchestra.run.smithi099:> os.fsync(f.fileno()) 2022-06-29T15:44:34.325 DEBUG:teuthology.orchestra.run.smithi099:> ' 2022-06-29T15:44:41.281 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:44:41.284 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:44:41.286 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:44:41.289 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:44:45.811 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:44:45.923 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.919+0000 7f31a759e700 1 Processor -- start 2022-06-29T15:44:45.924 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.919+0000 7f31a759e700 1 -- start start 2022-06-29T15:44:45.927 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.919+0000 7f31a759e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f31a80b7840 0x7f31a80b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:45.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.919+0000 7f31a759e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f31a8062dd0 0x7f31a8066c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:45.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.919+0000 7f31a759e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f31a80aea80 0x7f31a8141760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:45.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.919+0000 7f31a759e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31a80b7c20 con 0x7f31a8062dd0 2022-06-29T15:44:45.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.919+0000 7f31a759e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31a80b3d10 con 0x7f31a80aea80 2022-06-29T15:44:45.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.919+0000 7f31a759e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31a80670f0 con 0x7f31a80b7840 2022-06-29T15:44:45.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.919+0000 7f31a5d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f31a8062dd0 0x7f31a8066c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:45.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.919+0000 7f31a5d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f31a8062dd0 0x7f31a8066c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47976/0 (socket says 172.21.15.47:47976) 2022-06-29T15:44:45.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.919+0000 7f31a5d9b700 1 -- 172.21.15.47:0/2348109533 learned_addr learned my addr 172.21.15.47:0/2348109533 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:45.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.919+0000 7f31a6d9d700 1 --2- 172.21.15.47:0/2348109533 >> v2:172.21.15.64:3300/0 conn(0x7f31a80aea80 0x7f31a8141760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:45.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.919+0000 7f31a659c700 1 --2- 172.21.15.47:0/2348109533 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f31a80b7840 0x7f31a80b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:45.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.919+0000 7f31a5d9b700 1 -- 172.21.15.47:0/2348109533 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f31a80b7840 msgr2=0x7f31a80b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:44:45.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.919+0000 7f31a5d9b700 1 --2- 172.21.15.47:0/2348109533 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f31a80b7840 0x7f31a80b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:45.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.919+0000 7f31a5d9b700 1 -- 172.21.15.47:0/2348109533 >> v2:172.21.15.64:3300/0 conn(0x7f31a80aea80 msgr2=0x7f31a8141760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:44:45.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.919+0000 7f31a5d9b700 1 --2- 172.21.15.47:0/2348109533 >> v2:172.21.15.64:3300/0 conn(0x7f31a80aea80 0x7f31a8141760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:45.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.919+0000 7f31a5d9b700 1 -- 172.21.15.47:0/2348109533 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a8067380 con 0x7f31a8062dd0 2022-06-29T15:44:45.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.919+0000 7f31a6d9d700 1 --2- 172.21.15.47:0/2348109533 >> v2:172.21.15.64:3300/0 conn(0x7f31a80aea80 0x7f31a8141760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:44:45.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.919+0000 7f31a5d9b700 1 --2- 172.21.15.47:0/2348109533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31a8062dd0 0x7f31a8066c00 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f31980032f0 tx=0x7f3198009290).ready entity=mon.0 client_cookie=1db9626174eeb4d3 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:45.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.919+0000 7f31a559a700 1 -- 172.21.15.47:0/2348109533 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f319800b870 con 0x7f31a8062dd0 2022-06-29T15:44:45.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.919+0000 7f31a559a700 1 -- 172.21.15.47:0/2348109533 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f319800b9d0 con 0x7f31a8062dd0 2022-06-29T15:44:45.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.919+0000 7f31a559a700 1 -- 172.21.15.47:0/2348109533 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f319800bb40 con 0x7f31a8062dd0 2022-06-29T15:44:45.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.919+0000 7f31a759e700 1 -- 172.21.15.47:0/2348109533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31a8062dd0 msgr2=0x7f31a8066c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:45.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.919+0000 7f31a759e700 1 --2- 172.21.15.47:0/2348109533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31a8062dd0 0x7f31a8066c00 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f31980032f0 tx=0x7f3198009290).stop 2022-06-29T15:44:45.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.923+0000 7f31a759e700 1 -- 172.21.15.47:0/2348109533 shutdown_connections 2022-06-29T15:44:45.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.923+0000 7f31a759e700 1 --2- 172.21.15.47:0/2348109533 >> v2:172.21.15.64:3300/0 conn(0x7f31a80aea80 0x7f31a8141760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:45.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.923+0000 7f31a759e700 1 --2- 172.21.15.47:0/2348109533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31a8062dd0 0x7f31a8066c00 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:45.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.923+0000 7f31a759e700 1 --2- 172.21.15.47:0/2348109533 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f31a80b7840 0x7f31a80b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:45.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.923+0000 7f31a759e700 1 -- 172.21.15.47:0/2348109533 >> 172.21.15.47:0/2348109533 conn(0x7f31a805f480 msgr2=0x7f31a80647d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:45.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.923+0000 7f31a759e700 1 -- 172.21.15.47:0/2348109533 shutdown_connections 2022-06-29T15:44:45.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.923+0000 7f31a759e700 1 -- 172.21.15.47:0/2348109533 wait complete. 2022-06-29T15:44:45.938 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.923+0000 7f31a759e700 1 Processor -- start 2022-06-29T15:44:45.938 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.923+0000 7f31a759e700 1 -- start start 2022-06-29T15:44:45.938 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.923+0000 7f31a759e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f31a8062dd0 0x7f31a8145ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:45.939 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.923+0000 7f31a759e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31a80aea80 0x7f31a8146940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:45.939 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.923+0000 7f31a759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31a80b7840 0x7f31a8152180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:45.939 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.923+0000 7f31a759e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a8149ee0 con 0x7f31a80aea80 2022-06-29T15:44:45.939 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.923+0000 7f31a759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a814a180 con 0x7f31a80b7840 2022-06-29T15:44:45.940 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.923+0000 7f31a759e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31a814a420 con 0x7f31a8062dd0 2022-06-29T15:44:45.940 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.923+0000 7f31a5d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31a80aea80 0x7f31a8146940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:45.940 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.923+0000 7f31a659c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f31a8062dd0 0x7f31a8145ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:45.940 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.923+0000 7f31a659c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f31a8062dd0 0x7f31a8145ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:45084/0 (socket says 172.21.15.47:45084) 2022-06-29T15:44:45.941 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.923+0000 7f31a659c700 1 -- 172.21.15.47:0/4219543473 learned_addr learned my addr 172.21.15.47:0/4219543473 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:45.941 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.923+0000 7f31a6d9d700 1 --2- 172.21.15.47:0/4219543473 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31a80b7840 0x7f31a8152180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:45.941 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.923+0000 7f31a5d9b700 1 -- 172.21.15.47:0/4219543473 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f31a8062dd0 msgr2=0x7f31a8145ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:45.942 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.923+0000 7f31a5d9b700 1 --2- 172.21.15.47:0/4219543473 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f31a8062dd0 0x7f31a8145ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:45.942 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.923+0000 7f31a5d9b700 1 -- 172.21.15.47:0/4219543473 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31a80b7840 msgr2=0x7f31a8152180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:45.942 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.923+0000 7f31a5d9b700 1 --2- 172.21.15.47:0/4219543473 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31a80b7840 0x7f31a8152180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:45.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.923+0000 7f31a5d9b700 1 -- 172.21.15.47:0/4219543473 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3198009530 con 0x7f31a80aea80 2022-06-29T15:44:45.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.927+0000 7f31a5d9b700 1 --2- 172.21.15.47:0/4219543473 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31a80aea80 0x7f31a8146940 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7f31980032f0 tx=0x7f3198016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:45.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.927+0000 7f319f7fe700 1 -- 172.21.15.47:0/4219543473 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f319800b870 con 0x7f31a80aea80 2022-06-29T15:44:45.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.927+0000 7f319f7fe700 1 -- 172.21.15.47:0/4219543473 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3198011e60 con 0x7f31a80aea80 2022-06-29T15:44:45.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.927+0000 7f31a759e700 1 -- 172.21.15.47:0/4219543473 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31a8149b60 con 0x7f31a80aea80 2022-06-29T15:44:45.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.927+0000 7f319f7fe700 1 -- 172.21.15.47:0/4219543473 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3198020620 con 0x7f31a80aea80 2022-06-29T15:44:45.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.927+0000 7f31a759e700 1 -- 172.21.15.47:0/4219543473 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31a8147410 con 0x7f31a80aea80 2022-06-29T15:44:45.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.927+0000 7f319f7fe700 1 -- 172.21.15.47:0/4219543473 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3198026020 con 0x7f31a80aea80 2022-06-29T15:44:45.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.927+0000 7f31a759e700 1 -- 172.21.15.47:0/4219543473 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31a814a5e0 con 0x7f31a80aea80 2022-06-29T15:44:45.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.927+0000 7f319f7fe700 1 --2- 172.21.15.47:0/4219543473 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f318c0539e0 0x7f318c055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:45.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.927+0000 7f319f7fe700 1 -- 172.21.15.47:0/4219543473 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f319806cbd0 con 0x7f31a80aea80 2022-06-29T15:44:45.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.927+0000 7f31a659c700 1 --2- 172.21.15.47:0/4219543473 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f318c0539e0 0x7f318c055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:45.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.927+0000 7f31a659c700 1 --2- 172.21.15.47:0/4219543473 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f318c0539e0 0x7f318c055ea0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f3190008000 tx=0x7f3190008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:45.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:45.935+0000 7f319f7fe700 1 -- 172.21.15.47:0/4219543473 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3198043990 con 0x7f31a80aea80 2022-06-29T15:44:46.105 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.099+0000 7f31a759e700 1 -- 172.21.15.47:0/4219543473 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f31a8147410 con 0x7f31a80aea80 2022-06-29T15:44:46.106 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.099+0000 7f319f7fe700 1 -- 172.21.15.47:0/4219543473 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+2491 (secure 0 0 0) 0x7f31a8147410 con 0x7f31a80aea80 2022-06-29T15:44:46.106 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:44:46.106 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":17,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5259,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3601815346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3601815346},{"type":"v1","addr":"172.21.15.47:6835","nonce":3601815346}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15},{"gid":5262,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/657177602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":657177602},{"type":"v1","addr":"172.21.15.47:6837","nonce":657177602}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:44:27.616735+0000","modified":"2022-06-29T15:44:28.620601+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":54812},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54812":{"gid":54812,"name":"b","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6833/2497658100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2497658100},{"type":"v1","addr":"172.21.15.64:6833","nonce":2497658100}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2022-06-29T15:44:46.108 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.103+0000 7f31a759e700 1 -- 172.21.15.47:0/4219543473 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f318c0539e0 msgr2=0x7f318c055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:46.108 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.103+0000 7f31a759e700 1 --2- 172.21.15.47:0/4219543473 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f318c0539e0 0x7f318c055ea0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f3190008000 tx=0x7f3190008040).stop 2022-06-29T15:44:46.109 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.103+0000 7f31a759e700 1 -- 172.21.15.47:0/4219543473 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31a80aea80 msgr2=0x7f31a8146940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:46.110 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.103+0000 7f31a759e700 1 --2- 172.21.15.47:0/4219543473 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31a80aea80 0x7f31a8146940 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7f31980032f0 tx=0x7f3198016000).stop 2022-06-29T15:44:46.110 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.103+0000 7f31a759e700 1 -- 172.21.15.47:0/4219543473 shutdown_connections 2022-06-29T15:44:46.110 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.103+0000 7f31a759e700 1 --2- 172.21.15.47:0/4219543473 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f318c0539e0 0x7f318c055ea0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:46.111 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.103+0000 7f31a759e700 1 --2- 172.21.15.47:0/4219543473 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31a80b7840 0x7f31a8152180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:46.111 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.103+0000 7f31a759e700 1 --2- 172.21.15.47:0/4219543473 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31a80aea80 0x7f31a8146940 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:46.112 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.103+0000 7f31a759e700 1 --2- 172.21.15.47:0/4219543473 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f31a8062dd0 0x7f31a8145ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:46.112 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.103+0000 7f31a759e700 1 -- 172.21.15.47:0/4219543473 >> 172.21.15.47:0/4219543473 conn(0x7f31a805f480 msgr2=0x7f31a80601a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:46.112 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.103+0000 7f31a759e700 1 -- 172.21.15.47:0/4219543473 shutdown_connections 2022-06-29T15:44:46.113 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.103+0000 7f31a759e700 1 -- 172.21.15.47:0/4219543473 wait complete. 2022-06-29T15:44:46.116 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 17 2022-06-29T15:44:46.119 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json config set mds_recall_max_caps 20 2022-06-29T15:44:46.233 INFO:teuthology.orchestra.run.smithi064.stdout:{"success":"mds_recall_max_caps = '20' (not observed, change may require restart) "} 2022-06-29T15:44:46.233 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_caps = '20' (not observed, change may require restart) " } 2022-06-29T15:44:46.234 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:44:46.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.339+0000 7f886501e700 1 Processor -- start 2022-06-29T15:44:46.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.339+0000 7f886501e700 1 -- start start 2022-06-29T15:44:46.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.339+0000 7f886501e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f88600b5c60 0x7f8860144980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:46.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.339+0000 7f886501e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88600a9120 con 0x7f8860060eb0 2022-06-29T15:44:46.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.339+0000 7f886501e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88600b0b80 con 0x7f88600a72f0 2022-06-29T15:44:46.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.339+0000 7f886501e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88600a89e0 con 0x7f88600b5c60 2022-06-29T15:44:46.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.343+0000 7f885f7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f8860060eb0 0x7f88600b1e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33546/0 (socket says 172.21.15.47:33546) 2022-06-29T15:44:46.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.343+0000 7f885f7fe700 1 -- 172.21.15.47:0/2563186948 learned_addr learned my addr 172.21.15.47:0/2563186948 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:44:46.352 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.343+0000 7f885ffff700 1 --2- 172.21.15.47:0/2563186948 >> v2:172.21.15.64:3300/0 conn(0x7f88600b5c60 0x7f8860144980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:46.352 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.343+0000 7f885effd700 1 -- 172.21.15.47:0/2563186948 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3504135309 0 0) 0x7f88600a9120 con 0x7f8860060eb0 2022-06-29T15:44:46.352 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.343+0000 7f885effd700 1 -- 172.21.15.47:0/2563186948 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f884c003260 con 0x7f8860060eb0 2022-06-29T15:44:46.352 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.343+0000 7f885effd700 1 -- 172.21.15.47:0/2563186948 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3371941659 0 0) 0x7f884c003260 con 0x7f8860060eb0 2022-06-29T15:44:46.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.343+0000 7f885effd700 1 -- 172.21.15.47:0/2563186948 >> v2:172.21.15.64:3300/0 conn(0x7f88600b5c60 msgr2=0x7f8860144980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:44:46.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.343+0000 7f885effd700 1 --2- 172.21.15.47:0/2563186948 >> v2:172.21.15.64:3300/0 conn(0x7f88600b5c60 0x7f8860144980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:46.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.343+0000 7f885effd700 1 -- 172.21.15.47:0/2563186948 >> v1:172.21.15.64:6789/0 conn(0x7f88600a72f0 legacy=0x7f8860151eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:46.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.343+0000 7f885effd700 1 -- 172.21.15.47:0/2563186948 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88600a8c80 con 0x7f8860060eb0 2022-06-29T15:44:46.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.343+0000 7f885effd700 1 -- 172.21.15.47:0/2563186948 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7f8854001660 con 0x7f8860060eb0 2022-06-29T15:44:46.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.343+0000 7f885effd700 1 -- 172.21.15.47:0/2563186948 >> v1:172.21.15.47:6789/0 conn(0x7f8860060eb0 legacy=0x7f88600b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:46.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.343+0000 7f885effd700 1 --2- 172.21.15.47:0/2563186948 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f884c003750 0x7f884c003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:46.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.343+0000 7f885effd700 1 --2- 172.21.15.47:0/2563186948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f884c004500 0x7f884c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:46.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.343+0000 7f885effd700 1 --2- 172.21.15.47:0/2563186948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f884c004e70 0x7f884c00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:46.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.343+0000 7f885effd700 1 -- 172.21.15.47:0/2563186948 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88600a89e0 con 0x7f884c004e70 2022-06-29T15:44:46.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.343+0000 7f885effd700 1 -- 172.21.15.47:0/2563186948 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884c00bd20 con 0x7f884c004500 2022-06-29T15:44:46.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.343+0000 7f885effd700 1 -- 172.21.15.47:0/2563186948 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f884c00bee0 con 0x7f884c003750 2022-06-29T15:44:46.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.343+0000 7f885ffff700 1 --2- 172.21.15.47:0/2563186948 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f884c003750 0x7f884c003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:46.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.343+0000 7f885f7fe700 1 --2- 172.21.15.47:0/2563186948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f884c004500 0x7f884c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:46.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.343+0000 7f886481d700 1 --2- 172.21.15.47:0/2563186948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f884c004e70 0x7f884c00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:46.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.343+0000 7f885ffff700 1 -- 172.21.15.47:0/2563186948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f884c004500 msgr2=0x7f884c004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:44:46.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.343+0000 7f885ffff700 1 --2- 172.21.15.47:0/2563186948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f884c004500 0x7f884c004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:46.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.343+0000 7f885ffff700 1 -- 172.21.15.47:0/2563186948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f884c004e70 msgr2=0x7f884c00b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:44:46.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.343+0000 7f885ffff700 1 --2- 172.21.15.47:0/2563186948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f884c004e70 0x7f884c00b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:46.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.343+0000 7f885ffff700 1 -- 172.21.15.47:0/2563186948 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f884c003260 con 0x7f884c003750 2022-06-29T15:44:46.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.343+0000 7f886481d700 1 -- 172.21.15.47:0/2563186948 reap_dead start 2022-06-29T15:44:46.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.343+0000 7f885ffff700 1 --2- 172.21.15.47:0/2563186948 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f884c003750 0x7f884c003b70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f885800c000 tx=0x7f88580091a0).ready entity=mon.2 client_cookie=769a3e12b65b6c65 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:46.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.343+0000 7f885effd700 1 -- 172.21.15.47:0/2563186948 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8858005c60 con 0x7f884c003750 2022-06-29T15:44:46.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.343+0000 7f885effd700 1 -- 172.21.15.47:0/2563186948 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8858005dc0 con 0x7f884c003750 2022-06-29T15:44:46.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.343+0000 7f885effd700 1 -- 172.21.15.47:0/2563186948 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f885800e8d0 con 0x7f884c003750 2022-06-29T15:44:46.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.343+0000 7f886501e700 1 -- 172.21.15.47:0/2563186948 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f884c003750 msgr2=0x7f884c003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:46.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.343+0000 7f886501e700 1 --2- 172.21.15.47:0/2563186948 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f884c003750 0x7f884c003b70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f885800c000 tx=0x7f88580091a0).stop 2022-06-29T15:44:46.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.343+0000 7f886501e700 1 -- 172.21.15.47:0/2563186948 shutdown_connections 2022-06-29T15:44:46.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.343+0000 7f886501e700 1 --2- 172.21.15.47:0/2563186948 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f884c003750 0x7f884c003b70 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:46.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.343+0000 7f886501e700 1 -- 172.21.15.47:0/2563186948 >> 172.21.15.47:0/2563186948 conn(0x7f8860062560 msgr2=0x7f8860062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:46.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.343+0000 7f886501e700 1 -- 172.21.15.47:0/2563186948 shutdown_connections 2022-06-29T15:44:46.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.343+0000 7f886501e700 1 -- 172.21.15.47:0/2563186948 wait complete. 2022-06-29T15:44:46.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.347+0000 7f886501e700 1 Processor -- start 2022-06-29T15:44:46.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.347+0000 7f886501e700 1 -- start start 2022-06-29T15:44:46.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.347+0000 7f886501e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f884c003750 0x7f88601419c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:46.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.347+0000 7f886501e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f886014aa10 0x7f8860156730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:46.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.347+0000 7f886501e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8860148980 0x7f886014a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:46.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.347+0000 7f886501e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8860156c20 con 0x7f884c003750 2022-06-29T15:44:46.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.347+0000 7f886501e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88600ad980 con 0x7f8860148980 2022-06-29T15:44:46.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.347+0000 7f886501e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88601494a0 con 0x7f886014aa10 2022-06-29T15:44:46.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.347+0000 7f885ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f884c003750 0x7f88601419c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:46.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.347+0000 7f885ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f884c003750 0x7f88601419c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48002/0 (socket says 172.21.15.47:48002) 2022-06-29T15:44:46.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.347+0000 7f885ffff700 1 -- 172.21.15.47:0/811717331 learned_addr learned my addr 172.21.15.47:0/811717331 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:46.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.347+0000 7f886481d700 1 --2- 172.21.15.47:0/811717331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8860148980 0x7f886014a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:46.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.347+0000 7f885f7fe700 1 --2- 172.21.15.47:0/811717331 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f886014aa10 0x7f8860156730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:46.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.347+0000 7f885ffff700 1 -- 172.21.15.47:0/811717331 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f886014aa10 msgr2=0x7f8860156730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:46.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.347+0000 7f885ffff700 1 --2- 172.21.15.47:0/811717331 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f886014aa10 0x7f8860156730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:46.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.347+0000 7f885ffff700 1 -- 172.21.15.47:0/811717331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8860148980 msgr2=0x7f886014a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:46.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.347+0000 7f885ffff700 1 --2- 172.21.15.47:0/811717331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8860148980 0x7f886014a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:46.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.347+0000 7f885ffff700 1 -- 172.21.15.47:0/811717331 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8860149740 con 0x7f884c003750 2022-06-29T15:44:46.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.347+0000 7f885ffff700 1 --2- 172.21.15.47:0/811717331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f884c003750 0x7f88601419c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7f885800c000 tx=0x7f8858019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:46.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.347+0000 7f885d7fa700 1 -- 172.21.15.47:0/811717331 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f885801b820 con 0x7f884c003750 2022-06-29T15:44:46.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.347+0000 7f885d7fa700 1 -- 172.21.15.47:0/811717331 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f885801b980 con 0x7f884c003750 2022-06-29T15:44:46.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.347+0000 7f886501e700 1 -- 172.21.15.47:0/811717331 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8860149cb0 con 0x7f884c003750 2022-06-29T15:44:46.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.351+0000 7f885d7fa700 1 -- 172.21.15.47:0/811717331 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f885801bce0 con 0x7f884c003750 2022-06-29T15:44:46.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.351+0000 7f886501e700 1 -- 172.21.15.47:0/811717331 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f886014a640 con 0x7f884c003750 2022-06-29T15:44:46.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.351+0000 7f886501e700 1 -- 172.21.15.47:0/811717331 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8860149e40 con 0x7f884c003750 2022-06-29T15:44:46.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.351+0000 7f885d7fa700 1 -- 172.21.15.47:0/811717331 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8858005f30 con 0x7f884c003750 2022-06-29T15:44:46.370 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.351+0000 7f885d7fa700 1 --2- 172.21.15.47:0/811717331 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8840053b30 0x7f8840055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:46.370 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.351+0000 7f885d7fa700 1 -- 172.21.15.47:0/811717331 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f885806db20 con 0x7f884c003750 2022-06-29T15:44:46.370 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.351+0000 7f885f7fe700 1 --2- 172.21.15.47:0/811717331 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8840053b30 0x7f8840055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:46.370 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.351+0000 7f885f7fe700 1 --2- 172.21.15.47:0/811717331 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8840053b30 0x7f8840055ff0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f8854005000 tx=0x7f88600a87f0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:46.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.359+0000 7f885d7fa700 1 -- 172.21.15.47:0/811717331 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8858044930 con 0x7f884c003750 2022-06-29T15:44:46.529 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.523+0000 7f886501e700 1 -- 172.21.15.47:0/811717331 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f88600573d0 con 0x7f884c003750 2022-06-29T15:44:46.530 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.523+0000 7f885d7fa700 1 -- 172.21.15.47:0/811717331 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+2491 (secure 0 0 0) 0x7f8858047f50 con 0x7f884c003750 2022-06-29T15:44:46.531 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:44:46.531 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":17,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5259,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3601815346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3601815346},{"type":"v1","addr":"172.21.15.47:6835","nonce":3601815346}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15},{"gid":5262,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/657177602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":657177602},{"type":"v1","addr":"172.21.15.47:6837","nonce":657177602}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:44:27.616735+0000","modified":"2022-06-29T15:44:28.620601+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":54812},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54812":{"gid":54812,"name":"b","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6833/2497658100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2497658100},{"type":"v1","addr":"172.21.15.64:6833","nonce":2497658100}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2022-06-29T15:44:46.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.527+0000 7f886501e700 1 -- 172.21.15.47:0/811717331 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8840053b30 msgr2=0x7f8840055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:46.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.527+0000 7f886501e700 1 --2- 172.21.15.47:0/811717331 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8840053b30 0x7f8840055ff0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f8854005000 tx=0x7f88600a87f0).stop 2022-06-29T15:44:46.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.527+0000 7f886501e700 1 -- 172.21.15.47:0/811717331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f884c003750 msgr2=0x7f88601419c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:46.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.527+0000 7f886501e700 1 --2- 172.21.15.47:0/811717331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f884c003750 0x7f88601419c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7f885800c000 tx=0x7f8858019000).stop 2022-06-29T15:44:46.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.527+0000 7f886501e700 1 -- 172.21.15.47:0/811717331 shutdown_connections 2022-06-29T15:44:46.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.527+0000 7f886501e700 1 --2- 172.21.15.47:0/811717331 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8840053b30 0x7f8840055ff0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:46.536 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.527+0000 7f886501e700 1 --2- 172.21.15.47:0/811717331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8860148980 0x7f886014a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:46.536 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.527+0000 7f886501e700 1 --2- 172.21.15.47:0/811717331 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f886014aa10 0x7f8860156730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:46.536 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.527+0000 7f886501e700 1 --2- 172.21.15.47:0/811717331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f884c003750 0x7f88601419c0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:46.536 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.527+0000 7f886501e700 1 -- 172.21.15.47:0/811717331 >> 172.21.15.47:0/811717331 conn(0x7f8860062560 msgr2=0x7f88600b1020 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:46.536 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.527+0000 7f886501e700 1 -- 172.21.15.47:0/811717331 shutdown_connections 2022-06-29T15:44:46.537 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.527+0000 7f886501e700 1 -- 172.21.15.47:0/811717331 wait complete. 2022-06-29T15:44:46.537 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 17 2022-06-29T15:44:46.543 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json config set mds_recall_max_decay_threshold 400 2022-06-29T15:44:46.660 INFO:teuthology.orchestra.run.smithi064.stdout:{"success":"mds_recall_max_decay_threshold = '400' (not observed, change may require restart) "} 2022-06-29T15:44:46.660 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_decay_threshold = '400' (not observed, change may require restart) " } 2022-06-29T15:44:46.661 INFO:tasks.cephfs.fuse_mount:Killing ceph-fuse connection on ubuntu@smithi099.front.sepia.ceph.com... 2022-06-29T15:44:46.680 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:44:46.787 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.783+0000 7f09062cd700 1 Processor -- start 2022-06-29T15:44:46.788 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.783+0000 7f09062cd700 1 -- start start 2022-06-29T15:44:46.789 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.783+0000 7f09062cd700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f09000611b0 0x7f0900146640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:46.789 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.783+0000 7f09062cd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f09000b06d0 0x7f090013ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:46.789 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.783+0000 7f09062cd700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09000b4520 con 0x7f09000b0cf0 2022-06-29T15:44:46.790 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.783+0000 7f09062cd700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09000b4b10 con 0x7f09000b06d0 2022-06-29T15:44:46.790 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.783+0000 7f09062cd700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09000621d0 con 0x7f09000611b0 2022-06-29T15:44:46.790 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.783+0000 7f08ff7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f09000b0cf0 0x7f0900146b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33564/0 (socket says 172.21.15.47:33564) 2022-06-29T15:44:46.790 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.783+0000 7f08ff7fe700 1 -- 172.21.15.47:0/69838497 learned_addr learned my addr 172.21.15.47:0/69838497 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:44:46.791 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.783+0000 7f090486a700 1 --2- 172.21.15.47:0/69838497 >> v2:172.21.15.47:3300/0 conn(0x7f09000b06d0 0x7f090013ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:46.791 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.783+0000 7f08fffff700 1 --2- 172.21.15.47:0/69838497 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f09000611b0 0x7f0900146640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:46.791 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.783+0000 7f08feffd700 1 -- 172.21.15.47:0/69838497 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1266935890 0 0) 0x7f09000b4520 con 0x7f09000b0cf0 2022-06-29T15:44:46.791 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.783+0000 7f08feffd700 1 -- 172.21.15.47:0/69838497 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08ec003260 con 0x7f09000b0cf0 2022-06-29T15:44:46.793 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.783+0000 7f090486a700 1 -- 172.21.15.47:0/69838497 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f09000611b0 msgr2=0x7f0900146640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:44:46.793 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.783+0000 7f090486a700 1 --2- 172.21.15.47:0/69838497 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f09000611b0 0x7f0900146640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:46.794 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.783+0000 7f090486a700 1 -- 172.21.15.47:0/69838497 >> v1:172.21.15.47:6789/0 conn(0x7f09000b0cf0 legacy=0x7f0900146b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:46.794 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.783+0000 7f090486a700 1 -- 172.21.15.47:0/69838497 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0900061590 con 0x7f09000b06d0 2022-06-29T15:44:46.794 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.783+0000 7f08fffff700 1 --2- 172.21.15.47:0/69838497 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f09000611b0 0x7f0900146640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:44:46.794 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.783+0000 7f090486a700 1 --2- 172.21.15.47:0/69838497 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09000b06d0 0x7f090013ff00 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7f08f80032f0 tx=0x7f08f80091d0).ready entity=mon.0 client_cookie=8053312faa3c65f server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:46.795 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.783+0000 7f08feffd700 1 -- 172.21.15.47:0/69838497 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f08f800b600 con 0x7f09000b06d0 2022-06-29T15:44:46.795 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.787+0000 7f08feffd700 1 -- 172.21.15.47:0/69838497 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08f800b760 con 0x7f09000b06d0 2022-06-29T15:44:46.795 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.787+0000 7f08feffd700 1 -- 172.21.15.47:0/69838497 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f08f800b8d0 con 0x7f09000b06d0 2022-06-29T15:44:46.795 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.787+0000 7f09062cd700 1 -- 172.21.15.47:0/69838497 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09000b06d0 msgr2=0x7f090013ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:46.795 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.787+0000 7f09062cd700 1 --2- 172.21.15.47:0/69838497 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09000b06d0 0x7f090013ff00 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7f08f80032f0 tx=0x7f08f80091d0).stop 2022-06-29T15:44:46.796 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.787+0000 7f09062cd700 1 -- 172.21.15.47:0/69838497 shutdown_connections 2022-06-29T15:44:46.796 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.787+0000 7f09062cd700 1 --2- 172.21.15.47:0/69838497 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09000b06d0 0x7f090013ff00 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:46.796 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.787+0000 7f09062cd700 1 --2- 172.21.15.47:0/69838497 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f09000611b0 0x7f0900146640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:46.796 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.787+0000 7f09062cd700 1 -- 172.21.15.47:0/69838497 >> 172.21.15.47:0/69838497 conn(0x7f09000b98b0 msgr2=0x7f09000b9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:46.796 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.787+0000 7f09062cd700 1 -- 172.21.15.47:0/69838497 shutdown_connections 2022-06-29T15:44:46.797 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.787+0000 7f09062cd700 1 -- 172.21.15.47:0/69838497 wait complete. 2022-06-29T15:44:46.800 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.787+0000 7f09062cd700 1 Processor -- start 2022-06-29T15:44:46.800 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.787+0000 7f09062cd700 1 -- start start 2022-06-29T15:44:46.800 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.791+0000 7f09062cd700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f09000611b0 0x7f090013cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:46.801 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.791+0000 7f09062cd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09000b06d0 0x7f0900147ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:46.801 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.791+0000 7f09062cd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09000b0cf0 0x7f09001491c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:46.801 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.791+0000 7f09062cd700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0900136b60 con 0x7f09000b06d0 2022-06-29T15:44:46.801 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.791+0000 7f09062cd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0900148d90 con 0x7f09000b0cf0 2022-06-29T15:44:46.801 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.791+0000 7f09062cd700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0900138da0 con 0x7f09000611b0 2022-06-29T15:44:46.802 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.791+0000 7f08ff7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09000b06d0 0x7f0900147ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:46.802 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.791+0000 7f08ff7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09000b06d0 0x7f0900147ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48018/0 (socket says 172.21.15.47:48018) 2022-06-29T15:44:46.802 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.791+0000 7f08ff7fe700 1 -- 172.21.15.47:0/439015835 learned_addr learned my addr 172.21.15.47:0/439015835 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:46.802 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.791+0000 7f08fffff700 1 --2- 172.21.15.47:0/439015835 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f09000611b0 0x7f090013cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:46.803 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.791+0000 7f090486a700 1 --2- 172.21.15.47:0/439015835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09000b0cf0 0x7f09001491c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:46.803 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.791+0000 7f08ff7fe700 1 -- 172.21.15.47:0/439015835 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f09000611b0 msgr2=0x7f090013cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:46.803 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.791+0000 7f08ff7fe700 1 --2- 172.21.15.47:0/439015835 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f09000611b0 0x7f090013cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:46.803 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.791+0000 7f08ff7fe700 1 -- 172.21.15.47:0/439015835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09000b0cf0 msgr2=0x7f09001491c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:46.804 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.791+0000 7f08ff7fe700 1 --2- 172.21.15.47:0/439015835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09000b0cf0 0x7f09001491c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:46.804 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.791+0000 7f08ff7fe700 1 -- 172.21.15.47:0/439015835 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f000b180 con 0x7f09000b06d0 2022-06-29T15:44:46.804 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.791+0000 7f090486a700 1 --2- 172.21.15.47:0/439015835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09000b0cf0 0x7f09001491c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:44:46.804 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.791+0000 7f08ff7fe700 1 --2- 172.21.15.47:0/439015835 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09000b06d0 0x7f0900147ff0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7f0900136ca0 tx=0x7f08f4007aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:46.804 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.791+0000 7f08fd7fa700 1 -- 172.21.15.47:0/439015835 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f08f400ce70 con 0x7f09000b06d0 2022-06-29T15:44:46.805 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.791+0000 7f08fd7fa700 1 -- 172.21.15.47:0/439015835 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08f40070d0 con 0x7f09000b06d0 2022-06-29T15:44:46.805 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.791+0000 7f09062cd700 1 -- 172.21.15.47:0/439015835 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08f8009470 con 0x7f09000b06d0 2022-06-29T15:44:46.805 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.791+0000 7f08fd7fa700 1 -- 172.21.15.47:0/439015835 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f08f4007430 con 0x7f09000b06d0 2022-06-29T15:44:46.805 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.791+0000 7f09062cd700 1 -- 172.21.15.47:0/439015835 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09001386d0 con 0x7f09000b06d0 2022-06-29T15:44:46.806 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.795+0000 7f08fd7fa700 1 -- 172.21.15.47:0/439015835 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f08f401a070 con 0x7f09000b06d0 2022-06-29T15:44:46.806 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.795+0000 7f09062cd700 1 -- 172.21.15.47:0/439015835 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09000573d0 con 0x7f09000b06d0 2022-06-29T15:44:46.806 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.795+0000 7f08fd7fa700 1 --2- 172.21.15.47:0/439015835 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f08e0053b00 0x7f08e0055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:46.806 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.795+0000 7f08fd7fa700 1 -- 172.21.15.47:0/439015835 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f08f406af80 con 0x7f09000b06d0 2022-06-29T15:44:46.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.795+0000 7f08fffff700 1 --2- 172.21.15.47:0/439015835 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f08e0053b00 0x7f08e0055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:46.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.795+0000 7f08fffff700 1 --2- 172.21.15.47:0/439015835 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f08e0053b00 0x7f08e0055fc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f08f000b000 tx=0x7f08f0006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:46.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.803+0000 7f08fd7fa700 1 -- 172.21.15.47:0/439015835 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f08f4041d40 con 0x7f09000b06d0 2022-06-29T15:44:46.963 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.959+0000 7f09062cd700 1 -- 172.21.15.47:0/439015835 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f09001438e0 con 0x7f09000b06d0 2022-06-29T15:44:46.969 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.959+0000 7f08fd7fa700 1 -- 172.21.15.47:0/439015835 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+2491 (secure 0 0 0) 0x7f08f4045360 con 0x7f09000b06d0 2022-06-29T15:44:46.970 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:44:46.970 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":17,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5259,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3601815346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3601815346},{"type":"v1","addr":"172.21.15.47:6835","nonce":3601815346}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15},{"gid":5262,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/657177602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":657177602},{"type":"v1","addr":"172.21.15.47:6837","nonce":657177602}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:44:27.616735+0000","modified":"2022-06-29T15:44:28.620601+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":54812},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54812":{"gid":54812,"name":"b","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6833/2497658100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2497658100},{"type":"v1","addr":"172.21.15.64:6833","nonce":2497658100}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2022-06-29T15:44:46.972 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.959+0000 7f09062cd700 1 -- 172.21.15.47:0/439015835 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f08e0053b00 msgr2=0x7f08e0055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:46.973 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.963+0000 7f09062cd700 1 --2- 172.21.15.47:0/439015835 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f08e0053b00 0x7f08e0055fc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f08f000b000 tx=0x7f08f0006df0).stop 2022-06-29T15:44:46.973 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.963+0000 7f09062cd700 1 -- 172.21.15.47:0/439015835 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09000b06d0 msgr2=0x7f0900147ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:46.973 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.963+0000 7f09062cd700 1 --2- 172.21.15.47:0/439015835 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09000b06d0 0x7f0900147ff0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7f0900136ca0 tx=0x7f08f4007aa0).stop 2022-06-29T15:44:46.974 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.963+0000 7f09062cd700 1 -- 172.21.15.47:0/439015835 shutdown_connections 2022-06-29T15:44:46.974 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.963+0000 7f09062cd700 1 --2- 172.21.15.47:0/439015835 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f08e0053b00 0x7f08e0055fc0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:46.975 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.963+0000 7f09062cd700 1 --2- 172.21.15.47:0/439015835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09000b0cf0 0x7f09001491c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:46.975 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.963+0000 7f09062cd700 1 --2- 172.21.15.47:0/439015835 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09000b06d0 0x7f0900147ff0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:46.975 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.963+0000 7f09062cd700 1 --2- 172.21.15.47:0/439015835 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f09000611b0 0x7f090013cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:46.975 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.963+0000 7f09062cd700 1 -- 172.21.15.47:0/439015835 >> 172.21.15.47:0/439015835 conn(0x7f09000b98b0 msgr2=0x7f09000667e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:46.976 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.963+0000 7f09062cd700 1 -- 172.21.15.47:0/439015835 shutdown_connections 2022-06-29T15:44:46.976 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:46.963+0000 7f09062cd700 1 -- 172.21.15.47:0/439015835 wait complete. 2022-06-29T15:44:46.976 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 17 2022-06-29T15:44:46.977 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b cache drop 2022-06-29T15:44:47.082 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.075+0000 7f29d954c700 1 Processor -- start 2022-06-29T15:44:47.082 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.075+0000 7f29d954c700 1 -- start start 2022-06-29T15:44:47.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.079+0000 7f29d954c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f29d40a9770 0x7f29d40b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:47.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.079+0000 7f29d954c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f29d4095e30 0x7f29d4098160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:47.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.079+0000 7f29d954c700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29d40b1ac0 con 0x7f29d40b2930 2022-06-29T15:44:47.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.079+0000 7f29d954c700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29d4095c70 con 0x7f29d4095e30 2022-06-29T15:44:47.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.079+0000 7f29d954c700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29d40965b0 con 0x7f29d40a9770 2022-06-29T15:44:47.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.079+0000 7f29d2ffd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f29d40b2930 0x7f29d4097a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33578/0 (socket says 172.21.15.47:33578) 2022-06-29T15:44:47.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.079+0000 7f29d2ffd700 1 -- 172.21.15.47:0/2840360784 learned_addr learned my addr 172.21.15.47:0/2840360784 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:44:47.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.079+0000 7f29d37fe700 1 --2- 172.21.15.47:0/2840360784 >> v2:172.21.15.47:3300/0 conn(0x7f29d4095e30 0x7f29d4098160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:47.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.079+0000 7f29ca7fc700 1 --2- 172.21.15.47:0/2840360784 >> v2:172.21.15.64:3300/0 conn(0x7f29d40a9770 0x7f29d40b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:47.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.079+0000 7f29d27fc700 1 -- 172.21.15.47:0/2840360784 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 332605475 0 0) 0x7f29d40b1ac0 con 0x7f29d40b2930 2022-06-29T15:44:47.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.079+0000 7f29d27fc700 1 -- 172.21.15.47:0/2840360784 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29b4003260 con 0x7f29d40b2930 2022-06-29T15:44:47.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.079+0000 7f29ca7fc700 1 -- 172.21.15.47:0/2840360784 >> v2:172.21.15.47:3300/0 conn(0x7f29d4095e30 msgr2=0x7f29d4098160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:44:47.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.079+0000 7f29ca7fc700 1 --2- 172.21.15.47:0/2840360784 >> v2:172.21.15.47:3300/0 conn(0x7f29d4095e30 0x7f29d4098160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:47.089 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.079+0000 7f29ca7fc700 1 -- 172.21.15.47:0/2840360784 >> v1:172.21.15.47:6789/0 conn(0x7f29d40b2930 legacy=0x7f29d4097a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:47.089 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.079+0000 7f29ca7fc700 1 -- 172.21.15.47:0/2840360784 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d40b1cf0 con 0x7f29d40a9770 2022-06-29T15:44:47.089 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.079+0000 7f29d37fe700 1 --2- 172.21.15.47:0/2840360784 >> v2:172.21.15.47:3300/0 conn(0x7f29d4095e30 0x7f29d4098160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:44:47.089 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.079+0000 7f29ca7fc700 1 --2- 172.21.15.47:0/2840360784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29d40a9770 0x7f29d40b2d10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f29d40975e0 tx=0x7f29bc006df0).ready entity=mon.1 client_cookie=179c3ab3c5336de7 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:47.089 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.079+0000 7f29d27fc700 1 -- 172.21.15.47:0/2840360784 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29bc00d4f0 con 0x7f29d40a9770 2022-06-29T15:44:47.090 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.079+0000 7f29d27fc700 1 -- 172.21.15.47:0/2840360784 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29bc000e90 con 0x7f29d40a9770 2022-06-29T15:44:47.090 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.079+0000 7f29d27fc700 1 -- 172.21.15.47:0/2840360784 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29bc00d840 con 0x7f29d40a9770 2022-06-29T15:44:47.094 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.083+0000 7f29d954c700 1 -- 172.21.15.47:0/2840360784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29d40a9770 msgr2=0x7f29d40b2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:47.095 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.083+0000 7f29d954c700 1 --2- 172.21.15.47:0/2840360784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29d40a9770 0x7f29d40b2d10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f29d40975e0 tx=0x7f29bc006df0).stop 2022-06-29T15:44:47.095 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.083+0000 7f29d954c700 1 -- 172.21.15.47:0/2840360784 shutdown_connections 2022-06-29T15:44:47.095 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.083+0000 7f29d954c700 1 --2- 172.21.15.47:0/2840360784 >> v2:172.21.15.47:3300/0 conn(0x7f29d4095e30 0x7f29d4098160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:47.095 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.083+0000 7f29d954c700 1 --2- 172.21.15.47:0/2840360784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29d40a9770 0x7f29d40b2d10 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:47.096 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.083+0000 7f29d954c700 1 -- 172.21.15.47:0/2840360784 >> 172.21.15.47:0/2840360784 conn(0x7f29d405f0b0 msgr2=0x7f29d405f490 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:47.096 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.083+0000 7f29d954c700 1 -- 172.21.15.47:0/2840360784 shutdown_connections 2022-06-29T15:44:47.096 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.083+0000 7f29d954c700 1 -- 172.21.15.47:0/2840360784 wait complete. 2022-06-29T15:44:47.096 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.083+0000 7f29d954c700 1 Processor -- start 2022-06-29T15:44:47.096 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.083+0000 7f29d954c700 1 -- start start 2022-06-29T15:44:47.097 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.083+0000 7f29d954c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29d4095e30 0x7f29d4141a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:47.097 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.083+0000 7f29d954c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29d40a9770 0x7f29d4155ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:47.098 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.083+0000 7f29d954c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f29d40b2930 0x7f29d41420d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:47.098 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.083+0000 7f29d954c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d40ad250 con 0x7f29d40a9770 2022-06-29T15:44:47.098 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.083+0000 7f29d954c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d40ad440 con 0x7f29d4095e30 2022-06-29T15:44:47.098 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.083+0000 7f29d954c700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29d40afb20 con 0x7f29d40b2930 2022-06-29T15:44:47.098 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.083+0000 7f29ca7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29d40a9770 0x7f29d4155ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:47.099 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.087+0000 7f29ca7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29d40a9770 0x7f29d4155ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48034/0 (socket says 172.21.15.47:48034) 2022-06-29T15:44:47.099 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.087+0000 7f29ca7fc700 1 -- 172.21.15.47:0/1741233969 learned_addr learned my addr 172.21.15.47:0/1741233969 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:47.099 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.087+0000 7f29d2ffd700 1 --2- 172.21.15.47:0/1741233969 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29d4095e30 0x7f29d4141a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:47.099 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.087+0000 7f29d37fe700 1 --2- 172.21.15.47:0/1741233969 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f29d40b2930 0x7f29d41420d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:47.100 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.087+0000 7f29ca7fc700 1 -- 172.21.15.47:0/1741233969 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f29d40b2930 msgr2=0x7f29d41420d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:47.100 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.087+0000 7f29ca7fc700 1 --2- 172.21.15.47:0/1741233969 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f29d40b2930 0x7f29d41420d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:47.100 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.087+0000 7f29ca7fc700 1 -- 172.21.15.47:0/1741233969 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29d4095e30 msgr2=0x7f29d4141a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:47.100 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.087+0000 7f29ca7fc700 1 --2- 172.21.15.47:0/1741233969 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29d4095e30 0x7f29d4141a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:47.101 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.087+0000 7f29ca7fc700 1 -- 172.21.15.47:0/1741233969 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29cc00a610 con 0x7f29d40a9770 2022-06-29T15:44:47.101 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.087+0000 7f29d2ffd700 1 --2- 172.21.15.47:0/1741233969 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29d4095e30 0x7f29d4141a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:44:47.101 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.087+0000 7f29ca7fc700 1 --2- 172.21.15.47:0/1741233969 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29d40a9770 0x7f29d4155ca0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f29d40975e0 tx=0x7f29bc014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:47.101 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.087+0000 7f29d0ff9700 1 -- 172.21.15.47:0/1741233969 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29bc002f70 con 0x7f29d40a9770 2022-06-29T15:44:47.102 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.087+0000 7f29d0ff9700 1 -- 172.21.15.47:0/1741233969 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29bc00de40 con 0x7f29d40a9770 2022-06-29T15:44:47.102 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.087+0000 7f29d954c700 1 -- 172.21.15.47:0/1741233969 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29bc00b180 con 0x7f29d40a9770 2022-06-29T15:44:47.102 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.087+0000 7f29d0ff9700 1 -- 172.21.15.47:0/1741233969 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29bc01e5f0 con 0x7f29d40a9770 2022-06-29T15:44:47.102 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.087+0000 7f29d954c700 1 -- 172.21.15.47:0/1741233969 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29d40b0060 con 0x7f29d40a9770 2022-06-29T15:44:47.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.087+0000 7f29d0ff9700 1 -- 172.21.15.47:0/1741233969 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f29bc01c070 con 0x7f29d40a9770 2022-06-29T15:44:47.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.087+0000 7f29d0ff9700 1 --2- 172.21.15.47:0/1741233969 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f29b8053980 0x7f29b8055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:47.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.091+0000 7f29d0ff9700 1 -- 172.21.15.47:0/1741233969 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f29bc06cca0 con 0x7f29d40a9770 2022-06-29T15:44:47.104 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.091+0000 7f29d2ffd700 1 --2- 172.21.15.47:0/1741233969 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f29b8053980 0x7f29b8055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:47.104 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.091+0000 7f29d2ffd700 1 --2- 172.21.15.47:0/1741233969 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f29b8053980 0x7f29b8055e40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f29c400b000 tx=0x7f29c4009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:47.111 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.091+0000 7f29e37a1740 1 Processor -- start 2022-06-29T15:44:47.111 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.091+0000 7f29e37a1740 1 -- start start 2022-06-29T15:44:47.112 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.091+0000 7f29e37a1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1fd6b90 0x2009550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:47.112 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.091+0000 7f29e37a1740 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x2009a40 0x1fd7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:47.112 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.091+0000 7f29e37a1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x1fd7a80 0x1fd7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:47.112 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.091+0000 7f29e37a1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f47bb0 con 0x1fd7a80 2022-06-29T15:44:47.112 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.091+0000 7f29e37a1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f28150 con 0x1fd6b90 2022-06-29T15:44:47.113 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.091+0000 7f29e37a1740 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x1fd8420 con 0x2009a40 2022-06-29T15:44:47.113 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.091+0000 7f29d37fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x1fd7a80 0x1fd7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:47.113 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.091+0000 7f29ca7fc700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x2009a40 0x1fd7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:47.113 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.091+0000 7f29d2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1fd6b90 0x2009550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:47.114 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.091+0000 7f29d37fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x1fd7a80 0x1fd7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48042/0 (socket says 172.21.15.47:48042) 2022-06-29T15:44:47.114 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.091+0000 7f29d37fe700 1 -- 172.21.15.47:0/1954696095 learned_addr learned my addr 172.21.15.47:0/1954696095 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:47.114 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.091+0000 7f29d37fe700 1 -- 172.21.15.47:0/1954696095 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x2009a40 msgr2=0x1fd7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:44:47.115 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.091+0000 7f29d37fe700 1 --2- 172.21.15.47:0/1954696095 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x2009a40 0x1fd7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:47.115 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.091+0000 7f29d37fe700 1 -- 172.21.15.47:0/1954696095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1fd6b90 msgr2=0x2009550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:44:47.115 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.091+0000 7f29d37fe700 1 --2- 172.21.15.47:0/1954696095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1fd6b90 0x2009550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:47.115 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.091+0000 7f29d37fe700 1 -- 172.21.15.47:0/1954696095 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1f7f400 con 0x1fd7a80 2022-06-29T15:44:47.115 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.091+0000 7f29d37fe700 1 --2- 172.21.15.47:0/1954696095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x1fd7a80 0x1fd7f30 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x1fd33b0 tx=0x7f29cc008040).ready entity=mon.0 client_cookie=1da7e3a37c04d7d5 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:47.116 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.091+0000 7f29d954c700 1 -- 172.21.15.47:0/1954696095 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29cc010b70 con 0x1fd7a80 2022-06-29T15:44:47.116 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.091+0000 7f29d954c700 1 -- 172.21.15.47:0/1954696095 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29cc010cd0 con 0x1fd7a80 2022-06-29T15:44:47.116 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.091+0000 7f29d954c700 1 -- 172.21.15.47:0/1954696095 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29cc013470 con 0x1fd7a80 2022-06-29T15:44:47.116 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.091+0000 7f29e37a1740 1 -- 172.21.15.47:0/1954696095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x1fd7a80 msgr2=0x1fd7f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:47.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.091+0000 7f29e37a1740 1 --2- 172.21.15.47:0/1954696095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x1fd7a80 0x1fd7f30 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x1fd33b0 tx=0x7f29cc008040).stop 2022-06-29T15:44:47.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.091+0000 7f29e37a1740 1 -- 172.21.15.47:0/1954696095 shutdown_connections 2022-06-29T15:44:47.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.091+0000 7f29e37a1740 1 --2- 172.21.15.47:0/1954696095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x1fd7a80 0x1fd7f30 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:47.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.091+0000 7f29e37a1740 1 --2- 172.21.15.47:0/1954696095 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x2009a40 0x1fd7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:47.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.091+0000 7f29e37a1740 1 --2- 172.21.15.47:0/1954696095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1fd6b90 0x2009550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:47.118 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.091+0000 7f29e37a1740 1 -- 172.21.15.47:0/1954696095 >> 172.21.15.47:0/1954696095 conn(0x1f5f590 msgr2=0x1fd5e90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:47.118 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.095+0000 7f29e37a1740 1 -- 172.21.15.47:0/1954696095 shutdown_connections 2022-06-29T15:44:47.119 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.095+0000 7f29e37a1740 1 -- 172.21.15.47:0/1954696095 wait complete. 2022-06-29T15:44:47.119 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.095+0000 7f29e37a1740 1 -- start start 2022-06-29T15:44:47.119 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.095+0000 7f29e37a1740 1 Processor -- start 2022-06-29T15:44:47.119 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.095+0000 7f29e37a1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x1fd6b90 0x1fde0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:47.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.095+0000 7f29e37a1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1fd7a80 0x1fdbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:47.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.095+0000 7f29e37a1740 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x2009a40 0x1fdc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:47.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.095+0000 7f29e37a1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x201c6b0 con 0x1fd6b90 2022-06-29T15:44:47.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.095+0000 7f29e37a1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x201c950 con 0x1fd7a80 2022-06-29T15:44:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.095+0000 7f29e37a1740 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x200f810 con 0x2009a40 2022-06-29T15:44:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.095+0000 7f29e37a1740 10 client.0 ll_register_callbacks cb 0x1f8e9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2022-06-29T15:44:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.095+0000 7f29d2ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x1fd6b90 0x1fde0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.095+0000 7f29d2ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x1fd6b90 0x1fde0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48044/0 (socket says 172.21.15.47:48044) 2022-06-29T15:44:47.122 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.095+0000 7f29d2ffd700 1 -- 172.21.15.47:0/214308252 learned_addr learned my addr 172.21.15.47:0/214308252 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:47.122 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.095+0000 7f29ca7fc700 1 --2- 172.21.15.47:0/214308252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1fd7a80 0x1fdbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:47.122 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.095+0000 7f29d37fe700 1 --2- 172.21.15.47:0/214308252 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x2009a40 0x1fdc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:47.122 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.095+0000 7f29d2ffd700 1 -- 172.21.15.47:0/214308252 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x2009a40 msgr2=0x1fdc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:44:47.123 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.095+0000 7f29d2ffd700 1 --2- 172.21.15.47:0/214308252 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x2009a40 0x1fdc320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:47.123 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.095+0000 7f29d2ffd700 1 -- 172.21.15.47:0/214308252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1fd7a80 msgr2=0x1fdbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:44:47.123 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.095+0000 7f29d2ffd700 1 --2- 172.21.15.47:0/214308252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1fd7a80 0x1fdbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:47.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.095+0000 7f29d2ffd700 1 -- 172.21.15.47:0/214308252 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29cc00ab20 con 0x1fd6b90 2022-06-29T15:44:47.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.095+0000 7f29d2ffd700 1 --2- 172.21.15.47:0/214308252 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x1fd6b90 0x1fde0a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7f29c4017000 tx=0x7f29c401a040).ready entity=mon.0 client_cookie=4750899053adc35f server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:47.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.095+0000 7f29b1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2022-06-29T15:44:47.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.095+0000 7f29b1ffb700 1 -- 172.21.15.47:0/214308252 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29c401d040 con 0x1fd6b90 2022-06-29T15:44:47.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.095+0000 7f29b1ffb700 1 -- 172.21.15.47:0/214308252 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(47..47 src has 1..47) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f29c4018e40 con 0x1fd6b90 2022-06-29T15:44:47.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.095+0000 7f29e37a1740 1 -- 172.21.15.47:0/214308252 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1f3daa0 con 0x1fd6b90 2022-06-29T15:44:47.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.095+0000 7f29b1ffb700 1 -- 172.21.15.47:0/214308252 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29c4007160 con 0x1fd6b90 2022-06-29T15:44:47.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.095+0000 7f29e37a1740 10 client.5454 fetch_fsmap learned FSMap version 17 2022-06-29T15:44:47.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.095+0000 7f29e37a1740 1 -- 172.21.15.47:0/214308252 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x1f37380 con 0x1fd6b90 2022-06-29T15:44:47.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.095+0000 7f29b1ffb700 1 -- 172.21.15.47:0/214308252 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1f3daa0 con 0x1fd6b90 2022-06-29T15:44:47.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.095+0000 7f29b1ffb700 1 -- 172.21.15.47:0/214308252 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 17) v1 ==== 1434+0+0 (secure 0 0 0) 0x7f29c4005220 con 0x1fd6b90 2022-06-29T15:44:47.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.095+0000 7f29e37a1740 10 client.5454 fetch_fsmap finished waiting for FSMap version 17 2022-06-29T15:44:47.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.095+0000 7f29e37a1740 10 client.5454 resolve_mds: resolved ID 'b' to GID 54812 2022-06-29T15:44:47.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.095+0000 7f29e37a1740 20 client.5454 populate_metadata read hostname 'smithi047' 2022-06-29T15:44:47.127 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.095+0000 7f29e37a1740 1 --2- 172.21.15.47:0/214308252 >> [v2:172.21.15.64:6832/2497658100,v1:172.21.15.64:6833/2497658100] conn(0x2010420 0x2010800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:47.127 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.095+0000 7f29e37a1740 4 client.5454 mds_command: new command op to 54812 tid=0[{"prefix": "get_command_descriptions"}] 2022-06-29T15:44:47.127 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.095+0000 7f29e37a1740 1 -- 172.21.15.47:0/214308252 --> [v2:172.21.15.64:6832/2497658100,v1:172.21.15.64:6833/2497658100] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1f35e10 con 0x2010420 2022-06-29T15:44:47.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.095+0000 7f29ca7fc700 1 --2- 172.21.15.47:0/214308252 >> [v2:172.21.15.64:6832/2497658100,v1:172.21.15.64:6833/2497658100] conn(0x2010420 0x2010800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:47.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.099+0000 7f29b1ffb700 10 client.5454 ms_handle_connect on v2:172.21.15.64:6832/2497658100 2022-06-29T15:44:47.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.099+0000 7f29ca7fc700 1 --2- 172.21.15.47:0/214308252 >> [v2:172.21.15.64:6832/2497658100,v1:172.21.15.64:6833/2497658100] conn(0x2010420 0x2010800 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=8624420c5449909b server_cookie=fb97a15f516de24 in_seq=0 out_seq=0 2022-06-29T15:44:47.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.099+0000 7f29b1ffb700 1 -- 172.21.15.47:0/214308252 <== mds.0 v2:172.21.15.64:6832/2497658100 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x1f35e10 con 0x2010420 2022-06-29T15:44:47.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.099+0000 7f29b1ffb700 10 client.5454 handle_command_reply: tid=0 2022-06-29T15:44:47.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.099+0000 7f29e37a1740 1 client.5454 shutdown 2022-06-29T15:44:47.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.099+0000 7f29e37a1740 1 -- 172.21.15.47:0/214308252 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x1fd6b90 msgr2=0x1fde0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:47.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.099+0000 7f29e37a1740 1 --2- 172.21.15.47:0/214308252 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x1fd6b90 0x1fde0a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7f29c4017000 tx=0x7f29c401a040).stop 2022-06-29T15:44:47.130 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.099+0000 7f29e37a1740 1 -- 172.21.15.47:0/214308252 shutdown_connections 2022-06-29T15:44:47.130 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.099+0000 7f29e37a1740 1 --2- 172.21.15.47:0/214308252 >> [v2:172.21.15.64:6832/2497658100,v1:172.21.15.64:6833/2497658100] conn(0x2010420 0x2010800 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:47.130 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.099+0000 7f29e37a1740 1 --2- 172.21.15.47:0/214308252 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x2009a40 0x1fdc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:47.130 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.099+0000 7f29e37a1740 1 --2- 172.21.15.47:0/214308252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1fd7a80 0x1fdbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:47.130 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.099+0000 7f29b1ffb700 0 client.5454 ms_handle_reset on v2:172.21.15.64:6832/2497658100 2022-06-29T15:44:47.131 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.099+0000 7f29e37a1740 1 --2- 172.21.15.47:0/214308252 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x1fd6b90 0x1fde0a0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:47.131 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.099+0000 7f29e37a1740 1 -- 172.21.15.47:0/214308252 >> 172.21.15.47:0/214308252 conn(0x1f5f590 msgr2=0x1fd6f70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:47.131 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.099+0000 7f29e37a1740 1 -- 172.21.15.47:0/214308252 shutdown_connections 2022-06-29T15:44:47.131 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.099+0000 7f29e37a1740 1 -- 172.21.15.47:0/214308252 wait complete. 2022-06-29T15:44:47.132 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.099+0000 7f29e37a1740 20 client.5454 trim_cache size 0 max 16384 2022-06-29T15:44:47.138 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.115+0000 7f29e37a1740 1 Processor -- start 2022-06-29T15:44:47.139 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.115+0000 7f29e37a1740 1 -- start start 2022-06-29T15:44:47.139 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.115+0000 7f29e37a1740 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x1fd6b90 0x1fd5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:47.139 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.115+0000 7f29e37a1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x1fd7a80 0x1fdbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:47.139 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.115+0000 7f29e37a1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x2009a40 0x1fdc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:47.140 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.115+0000 7f29e37a1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x1fd3ac0 con 0x1fd7a80 2022-06-29T15:44:47.140 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.115+0000 7f29e37a1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f2da30 con 0x2009a40 2022-06-29T15:44:47.140 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.115+0000 7f29e37a1740 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x2010ff0 con 0x1fd6b90 2022-06-29T15:44:47.140 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.115+0000 7f29ca7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x1fd7a80 0x1fdbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:47.141 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.115+0000 7f29ca7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x1fd7a80 0x1fdbda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48054/0 (socket says 172.21.15.47:48054) 2022-06-29T15:44:47.141 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.115+0000 7f29d2ffd700 1 --2- 172.21.15.47:0/623422513 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x1fd6b90 0x1fd5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:47.141 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.119+0000 7f29ca7fc700 1 -- 172.21.15.47:0/623422513 learned_addr learned my addr 172.21.15.47:0/623422513 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:47.141 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.119+0000 7f29d37fe700 1 --2- 172.21.15.47:0/623422513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x2009a40 0x1fdc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:47.141 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.119+0000 7f29ca7fc700 1 -- 172.21.15.47:0/623422513 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x1fd6b90 msgr2=0x1fd5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:44:47.142 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.119+0000 7f29ca7fc700 1 --2- 172.21.15.47:0/623422513 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x1fd6b90 0x1fd5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:47.142 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.119+0000 7f29ca7fc700 1 -- 172.21.15.47:0/623422513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x2009a40 msgr2=0x1fdc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:44:47.142 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.119+0000 7f29ca7fc700 1 --2- 172.21.15.47:0/623422513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x2009a40 0x1fdc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:47.142 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.119+0000 7f29ca7fc700 1 -- 172.21.15.47:0/623422513 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1f3fc80 con 0x1fd7a80 2022-06-29T15:44:47.143 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.119+0000 7f29ca7fc700 1 --2- 172.21.15.47:0/623422513 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x1fd7a80 0x1fdbda0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x1f65e20 tx=0x7f29bc033040).ready entity=mon.0 client_cookie=4850ec87d06f0671 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:47.143 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.119+0000 7f29b3fff700 1 -- 172.21.15.47:0/623422513 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29bc0456d0 con 0x1fd7a80 2022-06-29T15:44:47.143 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.119+0000 7f29b3fff700 1 -- 172.21.15.47:0/623422513 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29bc045930 con 0x1fd7a80 2022-06-29T15:44:47.143 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.119+0000 7f29b3fff700 1 -- 172.21.15.47:0/623422513 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29bc046c70 con 0x1fd7a80 2022-06-29T15:44:47.143 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.119+0000 7f29e37a1740 1 -- 172.21.15.47:0/623422513 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x1fd7a80 msgr2=0x1fdbda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:47.144 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.119+0000 7f29e37a1740 1 --2- 172.21.15.47:0/623422513 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x1fd7a80 0x1fdbda0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x1f65e20 tx=0x7f29bc033040).stop 2022-06-29T15:44:47.144 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.119+0000 7f29e37a1740 1 -- 172.21.15.47:0/623422513 shutdown_connections 2022-06-29T15:44:47.144 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.119+0000 7f29e37a1740 1 --2- 172.21.15.47:0/623422513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x2009a40 0x1fdc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:47.144 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.119+0000 7f29e37a1740 1 --2- 172.21.15.47:0/623422513 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x1fd7a80 0x1fdbda0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:47.144 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.119+0000 7f29e37a1740 1 --2- 172.21.15.47:0/623422513 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x1fd6b90 0x1fd5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:47.145 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.119+0000 7f29e37a1740 1 -- 172.21.15.47:0/623422513 >> 172.21.15.47:0/623422513 conn(0x1f5f590 msgr2=0x1fd6f70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:47.145 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.119+0000 7f29e37a1740 1 -- 172.21.15.47:0/623422513 shutdown_connections 2022-06-29T15:44:47.145 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.119+0000 7f29e37a1740 1 -- 172.21.15.47:0/623422513 wait complete. 2022-06-29T15:44:47.145 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.119+0000 7f29e37a1740 1 -- start start 2022-06-29T15:44:47.146 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.119+0000 7f29e37a1740 1 Processor -- start 2022-06-29T15:44:47.146 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.123+0000 7f29e37a1740 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x1fd6b90 0x200b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:47.146 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.123+0000 7f29e37a1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x1fd7a80 0x2018a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:47.146 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.123+0000 7f29e37a1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x2009a40 0x1fdc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:47.147 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.123+0000 7f29e37a1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x200dd40 con 0x1fd7a80 2022-06-29T15:44:47.147 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.123+0000 7f29e37a1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x200f9f0 con 0x2009a40 2022-06-29T15:44:47.147 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.123+0000 7f29e37a1740 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x200df30 con 0x1fd6b90 2022-06-29T15:44:47.148 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.123+0000 7f29e37a1740 10 client.0 ll_register_callbacks cb 0x1fd2f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2022-06-29T15:44:47.148 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.123+0000 7f29ca7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x1fd7a80 0x2018a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:47.148 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.123+0000 7f29d37fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x2009a40 0x1fdc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:47.148 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.123+0000 7f29d37fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x2009a40 0x1fdc1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:53764/0 (socket says 172.21.15.47:53764) 2022-06-29T15:44:47.148 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.123+0000 7f29d37fe700 1 -- 172.21.15.47:0/1753443746 learned_addr learned my addr 172.21.15.47:0/1753443746 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:47.149 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.123+0000 7f29ca7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x1fd7a80 0x2018a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48060/0 (socket says 172.21.15.47:48060) 2022-06-29T15:44:47.149 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.123+0000 7f29d2ffd700 1 --2- 172.21.15.47:0/1753443746 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x1fd6b90 0x200b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:47.149 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.123+0000 7f29d37fe700 1 -- 172.21.15.47:0/1753443746 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x1fd6b90 msgr2=0x200b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:44:47.150 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.123+0000 7f29d37fe700 1 --2- 172.21.15.47:0/1753443746 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x1fd6b90 0x200b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:47.150 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.123+0000 7f29d37fe700 1 -- 172.21.15.47:0/1753443746 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x1fd7a80 msgr2=0x2018a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:44:47.150 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.123+0000 7f29d37fe700 1 --2- 172.21.15.47:0/1753443746 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x1fd7a80 0x2018a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:47.150 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.123+0000 7f29d37fe700 1 -- 172.21.15.47:0/1753443746 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29d40b0060 con 0x2009a40 2022-06-29T15:44:47.151 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.123+0000 7f29ca7fc700 1 --2- 172.21.15.47:0/1753443746 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x1fd7a80 0x2018a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:44:47.151 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.123+0000 7f29d37fe700 1 --2- 172.21.15.47:0/1753443746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x2009a40 0x1fdc1a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f29cc01f000 tx=0x7f29cc01b000).ready entity=mon.1 client_cookie=968cd4871578e9fe server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:47.151 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.123+0000 7f29b2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2022-06-29T15:44:47.151 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.123+0000 7f29b2ffd700 1 -- 172.21.15.47:0/1753443746 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29cc023030 con 0x2009a40 2022-06-29T15:44:47.151 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.123+0000 7f29b2ffd700 1 -- 172.21.15.47:0/1753443746 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(47..47 src has 1..47) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f29cc01d430 con 0x2009a40 2022-06-29T15:44:47.152 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.123+0000 7f29e37a1740 1 -- 172.21.15.47:0/1753443746 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f29cc010cd0 con 0x2009a40 2022-06-29T15:44:47.152 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.123+0000 7f29b2ffd700 1 -- 172.21.15.47:0/1753443746 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29cc010b70 con 0x2009a40 2022-06-29T15:44:47.152 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.123+0000 7f29e37a1740 10 client.5410 fetch_fsmap learned FSMap version 17 2022-06-29T15:44:47.152 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.123+0000 7f29e37a1740 1 -- 172.21.15.47:0/1753443746 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x1f3c670 con 0x2009a40 2022-06-29T15:44:47.153 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.123+0000 7f29b2ffd700 1 -- 172.21.15.47:0/1753443746 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29cc010cd0 con 0x2009a40 2022-06-29T15:44:47.153 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.123+0000 7f29b2ffd700 1 -- 172.21.15.47:0/1753443746 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 17) v1 ==== 1434+0+0 (secure 0 0 0) 0x7f29cc01dc00 con 0x2009a40 2022-06-29T15:44:47.153 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.123+0000 7f29e37a1740 10 client.5410 fetch_fsmap finished waiting for FSMap version 17 2022-06-29T15:44:47.153 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.123+0000 7f29e37a1740 10 client.5410 resolve_mds: resolved ID 'b' to GID 54812 2022-06-29T15:44:47.153 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.123+0000 7f29e37a1740 20 client.5410 populate_metadata read hostname 'smithi047' 2022-06-29T15:44:47.154 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.123+0000 7f29e37a1740 1 --2- 172.21.15.47:0/1753443746 >> [v2:172.21.15.64:6832/2497658100,v1:172.21.15.64:6833/2497658100] conn(0x2010420 0x2072540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:47.154 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.123+0000 7f29e37a1740 4 client.5410 mds_command: new command op to 54812 tid=0[{"prefix": "cache drop"}] 2022-06-29T15:44:47.154 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.123+0000 7f29e37a1740 1 -- 172.21.15.47:0/1753443746 --> [v2:172.21.15.64:6832/2497658100,v1:172.21.15.64:6833/2497658100] -- command(tid 0: {"prefix": "cache drop"}) v1 -- 0x1e2b590 con 0x2010420 2022-06-29T15:44:47.154 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.123+0000 7f29d2ffd700 1 --2- 172.21.15.47:0/1753443746 >> [v2:172.21.15.64:6832/2497658100,v1:172.21.15.64:6833/2497658100] conn(0x2010420 0x2072540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:47.155 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.127+0000 7f29d2ffd700 1 --2- 172.21.15.47:0/1753443746 >> [v2:172.21.15.64:6832/2497658100,v1:172.21.15.64:6833/2497658100] conn(0x2010420 0x2072540 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=855d68da59aa315b server_cookie=3436c70a9fb0a164 in_seq=0 out_seq=0 2022-06-29T15:44:47.155 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:47.127+0000 7f29b2ffd700 10 client.5410 ms_handle_connect on v2:172.21.15.64:6832/2497658100 2022-06-29T15:44:58.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.303+0000 7f29b2ffd700 1 -- 172.21.15.47:0/1753443746 <== mds.0 v2:172.21.15.64:6832/2497658100 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+380 (crc 0 0 0) 0x1e2b590 con 0x2010420 2022-06-29T15:44:58.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.303+0000 7f29b2ffd700 10 client.5410 handle_command_reply: tid=0 2022-06-29T15:44:58.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.303+0000 7f29e37a1740 1 client.5410 shutdown 2022-06-29T15:44:58.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.303+0000 7f29e37a1740 1 -- 172.21.15.47:0/1753443746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x2009a40 msgr2=0x1fdc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:58.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.303+0000 7f29e37a1740 1 --2- 172.21.15.47:0/1753443746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x2009a40 0x1fdc1a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f29cc01f000 tx=0x7f29cc01b000).stop 2022-06-29T15:44:58.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.303+0000 7f29e37a1740 1 -- 172.21.15.47:0/1753443746 shutdown_connections 2022-06-29T15:44:58.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.303+0000 7f29e37a1740 1 --2- 172.21.15.47:0/1753443746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x2009a40 0x1fdc1a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:58.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.303+0000 7f29e37a1740 1 --2- 172.21.15.47:0/1753443746 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x1fd7a80 0x2018a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:58.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.303+0000 7f29e37a1740 1 --2- 172.21.15.47:0/1753443746 >> [v2:172.21.15.64:6832/2497658100,v1:172.21.15.64:6833/2497658100] conn(0x2010420 0x2072540 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:58.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.303+0000 7f29e37a1740 1 --2- 172.21.15.47:0/1753443746 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x1fd6b90 0x200b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:58.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.303+0000 7f29b2ffd700 0 client.5410 ms_handle_reset on v2:172.21.15.64:6832/2497658100 2022-06-29T15:44:58.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.303+0000 7f29e37a1740 1 -- 172.21.15.47:0/1753443746 >> 172.21.15.47:0/1753443746 conn(0x1f5f590 msgr2=0x1fd6f70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:58.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.303+0000 7f29e37a1740 1 -- 172.21.15.47:0/1753443746 shutdown_connections 2022-06-29T15:44:58.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.303+0000 7f29e37a1740 1 -- 172.21.15.47:0/1753443746 wait complete. 2022-06-29T15:44:58.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.303+0000 7f29e37a1740 20 client.5410 trim_cache size 0 max 16384 2022-06-29T15:44:58.315 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2022-06-29T15:44:58.315 INFO:teuthology.orchestra.run.smithi047.stdout: "client_recall": { 2022-06-29T15:44:58.315 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2022-06-29T15:44:58.315 INFO:teuthology.orchestra.run.smithi047.stdout: "message": "(0) Success", 2022-06-29T15:44:58.316 INFO:teuthology.orchestra.run.smithi047.stdout: "recalled": 220 2022-06-29T15:44:58.316 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:58.316 INFO:teuthology.orchestra.run.smithi047.stdout: "flush_journal": { 2022-06-29T15:44:58.316 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2022-06-29T15:44:58.317 INFO:teuthology.orchestra.run.smithi047.stdout: "message": "" 2022-06-29T15:44:58.317 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:58.317 INFO:teuthology.orchestra.run.smithi047.stdout: "trim_cache": { 2022-06-29T15:44:58.317 INFO:teuthology.orchestra.run.smithi047.stdout: "trimmed": 0 2022-06-29T15:44:58.318 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:58.318 INFO:teuthology.orchestra.run.smithi047.stdout: "cache": { 2022-06-29T15:44:58.318 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": { 2022-06-29T15:44:58.318 INFO:teuthology.orchestra.run.smithi047.stdout: "items": 19990, 2022-06-29T15:44:58.319 INFO:teuthology.orchestra.run.smithi047.stdout: "bytes": 3099680 2022-06-29T15:44:58.319 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:44:58.319 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:44:58.319 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 11.177826861 2022-06-29T15:44:58.320 INFO:teuthology.orchestra.run.smithi047.stdout:} 2022-06-29T15:44:58.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.307+0000 7f29d954c700 1 -- 172.21.15.47:0/1741233969 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f29b8053980 msgr2=0x7f29b8055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:58.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.307+0000 7f29d954c700 1 --2- 172.21.15.47:0/1741233969 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f29b8053980 0x7f29b8055e40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f29c400b000 tx=0x7f29c4009040).stop 2022-06-29T15:44:58.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.307+0000 7f29d954c700 1 -- 172.21.15.47:0/1741233969 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29d40a9770 msgr2=0x7f29d4155ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:58.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.307+0000 7f29d954c700 1 --2- 172.21.15.47:0/1741233969 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29d40a9770 0x7f29d4155ca0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f29d40975e0 tx=0x7f29bc014000).stop 2022-06-29T15:44:58.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.307+0000 7f29d954c700 1 -- 172.21.15.47:0/1741233969 shutdown_connections 2022-06-29T15:44:58.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.307+0000 7f29d954c700 1 --2- 172.21.15.47:0/1741233969 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f29d40b2930 0x7f29d41420d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:58.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.307+0000 7f29d954c700 1 --2- 172.21.15.47:0/1741233969 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29d40a9770 0x7f29d4155ca0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:58.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.307+0000 7f29d954c700 1 --2- 172.21.15.47:0/1741233969 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f29b8053980 0x7f29b8055e40 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:58.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.307+0000 7f29d954c700 1 --2- 172.21.15.47:0/1741233969 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29d4095e30 0x7f29d4141a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:58.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.307+0000 7f29d954c700 1 -- 172.21.15.47:0/1741233969 >> 172.21.15.47:0/1741233969 conn(0x7f29d405f0b0 msgr2=0x7f29d4098e90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:58.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.307+0000 7f29d954c700 1 -- 172.21.15.47:0/1741233969 shutdown_connections 2022-06-29T15:44:58.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.307+0000 7f29d954c700 1 -- 172.21.15.47:0/1741233969 wait complete. 2022-06-29T15:44:58.324 INFO:tasks.cephfs.fuse_mount:Cleaning up killed ceph-fuse connection 2022-06-29T15:44:58.325 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:44:58.326 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:44:58.326 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:44:58.334 INFO:teuthology.orchestra.run.smithi099.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': Transport endpoint is not connected 2022-06-29T15:44:58.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T15:44:58.336 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.0 2022-06-29T15:44:58.336 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi099.front.sepia.ceph.com... 2022-06-29T15:44:58.337 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-06-29T15:44:58.337 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:44:58.400 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:44:58.400 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:44:58.401 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:44:58.407 INFO:teuthology.orchestra.run.smithi099.stdout:ext2/ext3 2022-06-29T15:44:58.408 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-06-29T15:44:58.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-29T15:44:58.409 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:44:58.459 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:44:58.574 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.563+0000 7f6853fff700 1 Processor -- start 2022-06-29T15:44:58.575 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.563+0000 7f6853fff700 1 -- start start 2022-06-29T15:44:58.575 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.563+0000 7f6853fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f68540b25c0 0x7f68540b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:58.575 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.563+0000 7f6853fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f68540af440 0x7f68540af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:58.575 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.563+0000 7f6853fff700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6854146510 con 0x7f68540b1fc0 2022-06-29T15:44:58.576 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.563+0000 7f6853fff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6854147430 con 0x7f68540af440 2022-06-29T15:44:58.576 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.563+0000 7f6853fff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68541467b0 con 0x7f68540b25c0 2022-06-29T15:44:58.576 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.563+0000 7f68527fc700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f68540b1fc0 0x7f68541403b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33620/0 (socket says 172.21.15.47:33620) 2022-06-29T15:44:58.576 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.563+0000 7f68527fc700 1 -- 172.21.15.47:0/74495590 learned_addr learned my addr 172.21.15.47:0/74495590 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:44:58.576 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.567+0000 7f6852ffd700 1 --2- 172.21.15.47:0/74495590 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f68540b25c0 0x7f68540b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:58.577 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.567+0000 7f68537fe700 1 --2- 172.21.15.47:0/74495590 >> v2:172.21.15.64:3300/0 conn(0x7f68540af440 0x7f68540af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:58.577 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.567+0000 7f6851ffb700 1 -- 172.21.15.47:0/74495590 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1675554411 0 0) 0x7f6854146510 con 0x7f68540b1fc0 2022-06-29T15:44:58.577 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.567+0000 7f6851ffb700 1 -- 172.21.15.47:0/74495590 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6834003260 con 0x7f68540b1fc0 2022-06-29T15:44:58.577 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.567+0000 7f6851ffb700 1 -- 172.21.15.47:0/74495590 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 738156862 0 0) 0x7f6834003260 con 0x7f68540b1fc0 2022-06-29T15:44:58.578 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.567+0000 7f6852ffd700 1 -- 172.21.15.47:0/74495590 >> v2:172.21.15.64:3300/0 conn(0x7f68540af440 msgr2=0x7f68540af8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:44:58.578 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.567+0000 7f6852ffd700 1 --2- 172.21.15.47:0/74495590 >> v2:172.21.15.64:3300/0 conn(0x7f68540af440 0x7f68540af8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:58.578 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.567+0000 7f6852ffd700 1 -- 172.21.15.47:0/74495590 >> v1:172.21.15.47:6789/0 conn(0x7f68540b1fc0 legacy=0x7f68541403b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:58.578 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.567+0000 7f6852ffd700 1 -- 172.21.15.47:0/74495590 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6854146a50 con 0x7f68540b25c0 2022-06-29T15:44:58.579 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.567+0000 7f68537fe700 1 --2- 172.21.15.47:0/74495590 >> v2:172.21.15.64:3300/0 conn(0x7f68540af440 0x7f68540af8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:44:58.579 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.567+0000 7f6852ffd700 1 --2- 172.21.15.47:0/74495590 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f68540b25c0 0x7f68540b29a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f683c001800 tx=0x7f68440091d0).ready entity=mon.2 client_cookie=bad47e2e3e0111c6 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:58.579 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.567+0000 7f6851ffb700 1 -- 172.21.15.47:0/74495590 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f683c001660 con 0x7f68540b25c0 2022-06-29T15:44:58.579 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.567+0000 7f6851ffb700 1 -- 172.21.15.47:0/74495590 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f684400f510 con 0x7f68540b25c0 2022-06-29T15:44:58.580 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.567+0000 7f6851ffb700 1 -- 172.21.15.47:0/74495590 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f684400f680 con 0x7f68540b25c0 2022-06-29T15:44:58.580 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.567+0000 7f6853fff700 1 -- 172.21.15.47:0/74495590 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f68540b25c0 msgr2=0x7f68540b29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:58.580 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.567+0000 7f6853fff700 1 --2- 172.21.15.47:0/74495590 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f68540b25c0 0x7f68540b29a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f683c001800 tx=0x7f68440091d0).stop 2022-06-29T15:44:58.580 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.567+0000 7f6853fff700 1 -- 172.21.15.47:0/74495590 shutdown_connections 2022-06-29T15:44:58.580 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.567+0000 7f6853fff700 1 --2- 172.21.15.47:0/74495590 >> v2:172.21.15.64:3300/0 conn(0x7f68540af440 0x7f68540af8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:58.581 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.567+0000 7f6853fff700 1 --2- 172.21.15.47:0/74495590 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f68540b25c0 0x7f68540b29a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:58.581 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.567+0000 7f6853fff700 1 -- 172.21.15.47:0/74495590 >> 172.21.15.47:0/74495590 conn(0x7f6854062df0 msgr2=0x7f68540631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:58.581 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.567+0000 7f6853fff700 1 -- 172.21.15.47:0/74495590 shutdown_connections 2022-06-29T15:44:58.581 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.567+0000 7f6853fff700 1 -- 172.21.15.47:0/74495590 wait complete. 2022-06-29T15:44:58.582 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.567+0000 7f6853fff700 1 Processor -- start 2022-06-29T15:44:58.582 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.567+0000 7f6853fff700 1 -- start start 2022-06-29T15:44:58.585 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.571+0000 7f6853fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68540af440 0x7f6854152f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:58.585 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.571+0000 7f6853fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f68540b1fc0 0x7f68541405c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:58.586 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.571+0000 7f6853fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68540b25c0 0x7f68541434d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:58.586 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.571+0000 7f6853fff700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685414fff0 con 0x7f68540af440 2022-06-29T15:44:58.586 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.571+0000 7f6853fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68540ab790 con 0x7f68540b25c0 2022-06-29T15:44:58.586 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.571+0000 7f6853fff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6854153420 con 0x7f68540b1fc0 2022-06-29T15:44:58.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.571+0000 7f6852ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68540af440 0x7f6854152f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:58.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.571+0000 7f6852ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68540af440 0x7f6854152f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48074/0 (socket says 172.21.15.47:48074) 2022-06-29T15:44:58.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.571+0000 7f6852ffd700 1 -- 172.21.15.47:0/341351146 learned_addr learned my addr 172.21.15.47:0/341351146 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:58.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.571+0000 7f68537fe700 1 --2- 172.21.15.47:0/341351146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68540b25c0 0x7f68541434d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:58.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.571+0000 7f68527fc700 1 --2- 172.21.15.47:0/341351146 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f68540b1fc0 0x7f68541405c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:58.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.571+0000 7f6852ffd700 1 -- 172.21.15.47:0/341351146 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f68540b1fc0 msgr2=0x7f68541405c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:58.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.571+0000 7f6852ffd700 1 --2- 172.21.15.47:0/341351146 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f68540b1fc0 0x7f68541405c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:58.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.571+0000 7f6852ffd700 1 -- 172.21.15.47:0/341351146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68540b25c0 msgr2=0x7f68541434d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:58.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.571+0000 7f6852ffd700 1 --2- 172.21.15.47:0/341351146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68540b25c0 0x7f68541434d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:58.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.571+0000 7f6852ffd700 1 -- 172.21.15.47:0/341351146 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f684c00b180 con 0x7f68540af440 2022-06-29T15:44:58.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.571+0000 7f68527fc700 1 --2- 172.21.15.47:0/341351146 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f68540b1fc0 0x7f68541405c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:44:58.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.571+0000 7f6852ffd700 1 --2- 172.21.15.47:0/341351146 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68540af440 0x7f6854152f30 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7f6854142910 tx=0x7f6844016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:58.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.571+0000 7f684bfff700 1 -- 172.21.15.47:0/341351146 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f684400f680 con 0x7f68540af440 2022-06-29T15:44:58.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.571+0000 7f684bfff700 1 -- 172.21.15.47:0/341351146 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68440032f0 con 0x7f68540af440 2022-06-29T15:44:58.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.571+0000 7f6853fff700 1 -- 172.21.15.47:0/341351146 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6844009470 con 0x7f68540af440 2022-06-29T15:44:58.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.571+0000 7f684bfff700 1 -- 172.21.15.47:0/341351146 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f68440205e0 con 0x7f68540af440 2022-06-29T15:44:58.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.571+0000 7f6853fff700 1 -- 172.21.15.47:0/341351146 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6854152a90 con 0x7f68540af440 2022-06-29T15:44:58.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.575+0000 7f6853fff700 1 -- 172.21.15.47:0/341351146 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6854152c50 con 0x7f68540af440 2022-06-29T15:44:58.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.575+0000 7f684bfff700 1 -- 172.21.15.47:0/341351146 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f684400f070 con 0x7f68540af440 2022-06-29T15:44:58.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.575+0000 7f684bfff700 1 --2- 172.21.15.47:0/341351146 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6838053950 0x7f6838055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:58.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.575+0000 7f684bfff700 1 -- 172.21.15.47:0/341351146 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f684406cd20 con 0x7f68540af440 2022-06-29T15:44:58.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.575+0000 7f68527fc700 1 --2- 172.21.15.47:0/341351146 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6838053950 0x7f6838055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:58.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.575+0000 7f68527fc700 1 --2- 172.21.15.47:0/341351146 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6838053950 0x7f6838055e10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f683c00b000 tx=0x7f683c001140).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:58.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.583+0000 7f684bfff700 1 -- 172.21.15.47:0/341351146 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f68440439d0 con 0x7f68540af440 2022-06-29T15:44:58.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.711+0000 7f6853fff700 1 -- 172.21.15.47:0/341351146 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f68540573d0 con 0x7f68540af440 2022-06-29T15:44:58.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.711+0000 7f684bfff700 1 -- 172.21.15.47:0/341351146 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16271 (secure 0 0 0) 0x7f6844015070 con 0x7f68540af440 2022-06-29T15:44:58.717 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:44:58.718 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":47,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:44:28.619620+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:44:25.700617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2022-06-29T15:44:26.610945+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:44:58.719 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.715+0000 7f6853fff700 1 -- 172.21.15.47:0/341351146 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6838053950 msgr2=0x7f6838055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:58.719 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.715+0000 7f6853fff700 1 --2- 172.21.15.47:0/341351146 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6838053950 0x7f6838055e10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f683c00b000 tx=0x7f683c001140).stop 2022-06-29T15:44:58.720 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.715+0000 7f6853fff700 1 -- 172.21.15.47:0/341351146 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68540af440 msgr2=0x7f6854152f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:58.720 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.715+0000 7f6853fff700 1 --2- 172.21.15.47:0/341351146 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68540af440 0x7f6854152f30 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7f6854142910 tx=0x7f6844016000).stop 2022-06-29T15:44:58.720 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.715+0000 7f6853fff700 1 -- 172.21.15.47:0/341351146 shutdown_connections 2022-06-29T15:44:58.720 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.715+0000 7f6853fff700 1 --2- 172.21.15.47:0/341351146 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6838053950 0x7f6838055e10 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:58.721 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.715+0000 7f6853fff700 1 --2- 172.21.15.47:0/341351146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68540b25c0 0x7f68541434d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:58.721 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.715+0000 7f6853fff700 1 --2- 172.21.15.47:0/341351146 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f68540b1fc0 0x7f68541405c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:58.721 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.715+0000 7f6853fff700 1 --2- 172.21.15.47:0/341351146 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68540af440 0x7f6854152f30 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:58.721 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.715+0000 7f6853fff700 1 -- 172.21.15.47:0/341351146 >> 172.21.15.47:0/341351146 conn(0x7f6854062df0 msgr2=0x7f68540b37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:58.722 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.715+0000 7f6853fff700 1 -- 172.21.15.47:0/341351146 shutdown_connections 2022-06-29T15:44:58.722 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.715+0000 7f6853fff700 1 -- 172.21.15.47:0/341351146 wait complete. 2022-06-29T15:44:58.730 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-29T15:42:16.971431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-29T15:42:25.290884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-29T15:44:25.700617+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2022-06-29T15:44:26.610945+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '47', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-29T15:44:58.731 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-29T15:44:58.841 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.835+0000 7f1a4c8e3700 1 Processor -- start 2022-06-29T15:44:58.841 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.835+0000 7f1a4c8e3700 1 -- start start 2022-06-29T15:44:58.844 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.839+0000 7f1a4c8e3700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1a48060a40 0x7f1a48142ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:58.844 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.839+0000 7f1a4c8e3700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1a481409b0 0x7f1a48061620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:58.844 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.839+0000 7f1a4c8e3700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a480af660 con 0x7f1a4805fd00 2022-06-29T15:44:58.845 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.839+0000 7f1a4c8e3700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a48143fc0 con 0x7f1a48060a40 2022-06-29T15:44:58.845 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.839+0000 7f1a4c8e3700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a481437e0 con 0x7f1a481409b0 2022-06-29T15:44:58.845 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.839+0000 7f1a46ffd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1a4805fd00 0x7f1a481402a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33634/0 (socket says 172.21.15.47:33634) 2022-06-29T15:44:58.845 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.839+0000 7f1a46ffd700 1 -- 172.21.15.47:0/1473334772 learned_addr learned my addr 172.21.15.47:0/1473334772 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:44:58.846 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.839+0000 7f1a47fff700 1 --2- 172.21.15.47:0/1473334772 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1a481409b0 0x7f1a48061620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:58.846 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.839+0000 7f1a477fe700 1 --2- 172.21.15.47:0/1473334772 >> v2:172.21.15.64:3300/0 conn(0x7f1a48060a40 0x7f1a48142ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:58.846 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.839+0000 7f1a467fc700 1 -- 172.21.15.47:0/1473334772 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3914029863 0 0) 0x7f1a480af660 con 0x7f1a4805fd00 2022-06-29T15:44:58.847 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.839+0000 7f1a467fc700 1 -- 172.21.15.47:0/1473334772 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a28003260 con 0x7f1a4805fd00 2022-06-29T15:44:58.847 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.839+0000 7f1a47fff700 1 -- 172.21.15.47:0/1473334772 >> v2:172.21.15.64:3300/0 conn(0x7f1a48060a40 msgr2=0x7f1a48142ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:44:58.847 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.839+0000 7f1a47fff700 1 --2- 172.21.15.47:0/1473334772 >> v2:172.21.15.64:3300/0 conn(0x7f1a48060a40 0x7f1a48142ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:58.847 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.839+0000 7f1a47fff700 1 -- 172.21.15.47:0/1473334772 >> v1:172.21.15.47:6789/0 conn(0x7f1a4805fd00 legacy=0x7f1a481402a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:58.848 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.839+0000 7f1a47fff700 1 -- 172.21.15.47:0/1473334772 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a48143a80 con 0x7f1a481409b0 2022-06-29T15:44:58.848 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.839+0000 7f1a47fff700 1 --2- 172.21.15.47:0/1473334772 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1a481409b0 0x7f1a48061620 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f1a400032f0 tx=0x7f1a400091d0).ready entity=mon.2 client_cookie=5bf5c524d8ee509f server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:58.852 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.839+0000 7f1a467fc700 1 -- 172.21.15.47:0/1473334772 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a4000b600 con 0x7f1a481409b0 2022-06-29T15:44:58.853 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.839+0000 7f1a467fc700 1 -- 172.21.15.47:0/1473334772 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a4000b760 con 0x7f1a481409b0 2022-06-29T15:44:58.853 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.839+0000 7f1a467fc700 1 -- 172.21.15.47:0/1473334772 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a4000b8d0 con 0x7f1a481409b0 2022-06-29T15:44:58.853 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.839+0000 7f1a4c8e3700 1 -- 172.21.15.47:0/1473334772 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1a481409b0 msgr2=0x7f1a48061620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:58.854 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.839+0000 7f1a4c8e3700 1 --2- 172.21.15.47:0/1473334772 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1a481409b0 0x7f1a48061620 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f1a400032f0 tx=0x7f1a400091d0).stop 2022-06-29T15:44:58.854 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.839+0000 7f1a4c8e3700 1 -- 172.21.15.47:0/1473334772 shutdown_connections 2022-06-29T15:44:58.854 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.839+0000 7f1a4c8e3700 1 --2- 172.21.15.47:0/1473334772 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1a481409b0 0x7f1a48061620 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:58.854 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.839+0000 7f1a4c8e3700 1 --2- 172.21.15.47:0/1473334772 >> v2:172.21.15.64:3300/0 conn(0x7f1a48060a40 0x7f1a48142ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:58.855 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.839+0000 7f1a4c8e3700 1 -- 172.21.15.47:0/1473334772 >> 172.21.15.47:0/1473334772 conn(0x7f1a480b6430 msgr2=0x7f1a480b6810 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:58.855 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.839+0000 7f1a4c8e3700 1 -- 172.21.15.47:0/1473334772 shutdown_connections 2022-06-29T15:44:58.855 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.839+0000 7f1a4c8e3700 1 -- 172.21.15.47:0/1473334772 wait complete. 2022-06-29T15:44:58.855 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.843+0000 7f1a4c8e3700 1 Processor -- start 2022-06-29T15:44:58.856 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.843+0000 7f1a4c8e3700 1 -- start start 2022-06-29T15:44:58.856 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.843+0000 7f1a4c8e3700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1a4805fd00 0x7f1a48149c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:58.856 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.843+0000 7f1a4c8e3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a48060a40 0x7f1a4813d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:58.856 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.843+0000 7f1a4c8e3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a481409b0 0x7f1a481443f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:58.856 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.843+0000 7f1a4c8e3700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a480abf80 con 0x7f1a481409b0 2022-06-29T15:44:58.857 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.843+0000 7f1a4c8e3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a48156170 con 0x7f1a48060a40 2022-06-29T15:44:58.857 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.843+0000 7f1a4c8e3700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a48147720 con 0x7f1a4805fd00 2022-06-29T15:44:58.857 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.843+0000 7f1a47fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a481409b0 0x7f1a481443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:58.857 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.843+0000 7f1a47fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a481409b0 0x7f1a481443f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48090/0 (socket says 172.21.15.47:48090) 2022-06-29T15:44:58.858 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.843+0000 7f1a47fff700 1 -- 172.21.15.47:0/2433194046 learned_addr learned my addr 172.21.15.47:0/2433194046 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:58.858 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.843+0000 7f1a477fe700 1 --2- 172.21.15.47:0/2433194046 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1a4805fd00 0x7f1a48149c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:58.858 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.847+0000 7f1a46ffd700 1 --2- 172.21.15.47:0/2433194046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a48060a40 0x7f1a4813d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:58.858 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.847+0000 7f1a47fff700 1 -- 172.21.15.47:0/2433194046 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1a4805fd00 msgr2=0x7f1a48149c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:58.859 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.847+0000 7f1a47fff700 1 --2- 172.21.15.47:0/2433194046 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1a4805fd00 0x7f1a48149c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:58.859 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.847+0000 7f1a47fff700 1 -- 172.21.15.47:0/2433194046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a48060a40 msgr2=0x7f1a4813d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:58.859 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.847+0000 7f1a47fff700 1 --2- 172.21.15.47:0/2433194046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a48060a40 0x7f1a4813d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:58.859 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.847+0000 7f1a47fff700 1 -- 172.21.15.47:0/2433194046 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a40009470 con 0x7f1a481409b0 2022-06-29T15:44:58.860 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.847+0000 7f1a47fff700 1 --2- 172.21.15.47:0/2433194046 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a481409b0 0x7f1a481443f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f1a400032f0 tx=0x7f1a40016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:58.860 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.847+0000 7f1a44ff9700 1 -- 172.21.15.47:0/2433194046 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a4000b600 con 0x7f1a481409b0 2022-06-29T15:44:58.860 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.847+0000 7f1a44ff9700 1 -- 172.21.15.47:0/2433194046 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a40011c10 con 0x7f1a481409b0 2022-06-29T15:44:58.860 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.847+0000 7f1a44ff9700 1 -- 172.21.15.47:0/2433194046 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a4001f3f0 con 0x7f1a481409b0 2022-06-29T15:44:58.860 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.847+0000 7f1a4c8e3700 1 -- 172.21.15.47:0/2433194046 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a481479c0 con 0x7f1a481409b0 2022-06-29T15:44:58.861 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.847+0000 7f1a4c8e3700 1 -- 172.21.15.47:0/2433194046 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a48155d10 con 0x7f1a481409b0 2022-06-29T15:44:58.861 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.847+0000 7f1a4c8e3700 1 -- 172.21.15.47:0/2433194046 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a480573d0 con 0x7f1a481409b0 2022-06-29T15:44:58.862 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.847+0000 7f1a44ff9700 1 -- 172.21.15.47:0/2433194046 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1a40026030 con 0x7f1a481409b0 2022-06-29T15:44:58.863 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.851+0000 7f1a44ff9700 1 --2- 172.21.15.47:0/2433194046 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1a2c053950 0x7f1a2c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:58.863 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.851+0000 7f1a477fe700 1 --2- 172.21.15.47:0/2433194046 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1a2c053950 0x7f1a2c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:58.863 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.851+0000 7f1a44ff9700 1 -- 172.21.15.47:0/2433194046 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f1a4006cb80 con 0x7f1a481409b0 2022-06-29T15:44:58.864 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.851+0000 7f1a477fe700 1 --2- 172.21.15.47:0/2433194046 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1a2c053950 0x7f1a2c055e10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f1a3800b000 tx=0x7f1a38006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:58.864 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.855+0000 7f1a44ff9700 1 -- 172.21.15.47:0/2433194046 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1a400438a0 con 0x7f1a481409b0 2022-06-29T15:44:58.988 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.983+0000 7f1a4c8e3700 1 -- 172.21.15.47:0/2433194046 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f1a4806a0f0 con 0x7f1a481409b0 2022-06-29T15:44:58.989 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.983+0000 7f1a44ff9700 1 -- 172.21.15.47:0/2433194046 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v47) v1 ==== 110+0+10 (secure 0 0 0) 0x7f1a400240b0 con 0x7f1a481409b0 2022-06-29T15:44:58.990 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2022-06-29T15:44:58.992 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.987+0000 7f1a4c8e3700 1 -- 172.21.15.47:0/2433194046 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1a2c053950 msgr2=0x7f1a2c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:58.992 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.987+0000 7f1a4c8e3700 1 --2- 172.21.15.47:0/2433194046 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1a2c053950 0x7f1a2c055e10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f1a3800b000 tx=0x7f1a38006df0).stop 2022-06-29T15:44:58.992 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.987+0000 7f1a4c8e3700 1 -- 172.21.15.47:0/2433194046 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a481409b0 msgr2=0x7f1a481443f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:58.992 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.987+0000 7f1a4c8e3700 1 --2- 172.21.15.47:0/2433194046 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a481409b0 0x7f1a481443f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f1a400032f0 tx=0x7f1a40016000).stop 2022-06-29T15:44:58.993 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.987+0000 7f1a4c8e3700 1 -- 172.21.15.47:0/2433194046 shutdown_connections 2022-06-29T15:44:58.993 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.987+0000 7f1a4c8e3700 1 --2- 172.21.15.47:0/2433194046 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a481409b0 0x7f1a481443f0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:58.993 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.987+0000 7f1a4c8e3700 1 --2- 172.21.15.47:0/2433194046 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1a2c053950 0x7f1a2c055e10 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:58.993 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.987+0000 7f1a4c8e3700 1 --2- 172.21.15.47:0/2433194046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a48060a40 0x7f1a4813d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:58.994 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.987+0000 7f1a4c8e3700 1 --2- 172.21.15.47:0/2433194046 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1a4805fd00 0x7f1a48149c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:58.994 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.987+0000 7f1a4c8e3700 1 -- 172.21.15.47:0/2433194046 >> 172.21.15.47:0/2433194046 conn(0x7f1a480b6430 msgr2=0x7f1a4805f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:58.994 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.987+0000 7f1a4c8e3700 1 -- 172.21.15.47:0/2433194046 shutdown_connections 2022-06-29T15:44:58.994 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:58.987+0000 7f1a4c8e3700 1 -- 172.21.15.47:0/2433194046 wait complete. 2022-06-29T15:44:59.003 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-29T15:44:59.112 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.107+0000 7f9cc3497700 1 Processor -- start 2022-06-29T15:44:59.112 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.107+0000 7f9cc3497700 1 -- start start 2022-06-29T15:44:59.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.107+0000 7f9cc3497700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9cbc0b6d30 0x7f9cbc13d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:59.118 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.107+0000 7f9cc3497700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9cbc062f30 0x7f9cbc13d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:59.118 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.107+0000 7f9cc3497700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cbc143f70 con 0x7f9cbc1400c0 2022-06-29T15:44:59.118 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.107+0000 7f9cc3497700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cbc0b1380 con 0x7f9cbc0b6d30 2022-06-29T15:44:59.118 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.107+0000 7f9cc3497700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cbc0a9cf0 con 0x7f9cbc062f30 2022-06-29T15:44:59.119 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.107+0000 7f9cc1c94700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9cbc062f30 0x7f9cbc13d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:59.119 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.107+0000 7f9cc1c94700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9cbc062f30 0x7f9cbc13d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:45196/0 (socket says 172.21.15.47:45196) 2022-06-29T15:44:59.119 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.107+0000 7f9cc1c94700 1 -- 172.21.15.47:0/1404766155 learned_addr learned my addr 172.21.15.47:0/1404766155 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:59.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.107+0000 7f9cc2495700 1 --2- 172.21.15.47:0/1404766155 >> v2:172.21.15.64:3300/0 conn(0x7f9cbc0b6d30 0x7f9cbc13d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:59.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.107+0000 7f9cc1c94700 1 -- 172.21.15.47:0/1404766155 >> v2:172.21.15.64:3300/0 conn(0x7f9cbc0b6d30 msgr2=0x7f9cbc13d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:44:59.121 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.107+0000 7f9cc1493700 1 -- 172.21.15.47:0/1404766155 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1523930710 0 0) 0x7f9cbc143f70 con 0x7f9cbc1400c0 2022-06-29T15:44:59.121 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.107+0000 7f9cc1c94700 1 --2- 172.21.15.47:0/1404766155 >> v2:172.21.15.64:3300/0 conn(0x7f9cbc0b6d30 0x7f9cbc13d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:59.121 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.107+0000 7f9cc1c94700 1 -- 172.21.15.47:0/1404766155 >> v1:172.21.15.64:6789/0 conn(0x7f9cbc1400c0 legacy=0x7f9cbc140550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:59.122 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.107+0000 7f9cc1c94700 1 -- 172.21.15.47:0/1404766155 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cbc13dc80 con 0x7f9cbc062f30 2022-06-29T15:44:59.122 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.111+0000 7f9cc1c94700 1 --2- 172.21.15.47:0/1404766155 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9cbc062f30 0x7f9cbc13d790 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f9cb8002f70 tx=0x7f9cb8006eb0).ready entity=mon.2 client_cookie=fe2e31db1ce192d0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:59.123 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.111+0000 7f9cc1493700 1 -- 172.21.15.47:0/1404766155 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9cb800d570 con 0x7f9cbc062f30 2022-06-29T15:44:59.123 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.111+0000 7f9cc1493700 1 -- 172.21.15.47:0/1404766155 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cb8000e90 con 0x7f9cbc062f30 2022-06-29T15:44:59.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.111+0000 7f9cc1493700 1 -- 172.21.15.47:0/1404766155 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9cb800d8c0 con 0x7f9cbc062f30 2022-06-29T15:44:59.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.111+0000 7f9cc3497700 1 -- 172.21.15.47:0/1404766155 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9cbc062f30 msgr2=0x7f9cbc13d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:59.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.111+0000 7f9cc3497700 1 --2- 172.21.15.47:0/1404766155 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9cbc062f30 0x7f9cbc13d790 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f9cb8002f70 tx=0x7f9cb8006eb0).stop 2022-06-29T15:44:59.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.111+0000 7f9cc3497700 1 -- 172.21.15.47:0/1404766155 shutdown_connections 2022-06-29T15:44:59.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.111+0000 7f9cc3497700 1 --2- 172.21.15.47:0/1404766155 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9cbc062f30 0x7f9cbc13d790 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:59.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.111+0000 7f9cc3497700 1 --2- 172.21.15.47:0/1404766155 >> v2:172.21.15.64:3300/0 conn(0x7f9cbc0b6d30 0x7f9cbc13d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:59.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.111+0000 7f9cc3497700 1 -- 172.21.15.47:0/1404766155 >> 172.21.15.47:0/1404766155 conn(0x7f9cbc063670 msgr2=0x7f9cbc063a50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:59.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.111+0000 7f9cc3497700 1 -- 172.21.15.47:0/1404766155 shutdown_connections 2022-06-29T15:44:59.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.111+0000 7f9cc3497700 1 -- 172.21.15.47:0/1404766155 wait complete. 2022-06-29T15:44:59.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.111+0000 7f9cc3497700 1 Processor -- start 2022-06-29T15:44:59.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.111+0000 7f9cc3497700 1 -- start start 2022-06-29T15:44:59.130 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.115+0000 7f9cc3497700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9cbc062f30 0x7f9cbc142bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:59.131 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.115+0000 7f9cc3497700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cbc0b6d30 0x7f9cbc13b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:59.131 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.115+0000 7f9cc3497700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9cbc1400c0 0x7f9cbc13bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:59.131 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.115+0000 7f9cc3497700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cbc143540 con 0x7f9cbc0b6d30 2022-06-29T15:44:59.131 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.115+0000 7f9cc3497700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cbc141b40 con 0x7f9cbc062f30 2022-06-29T15:44:59.131 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.115+0000 7f9cc3497700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cbc1432a0 con 0x7f9cbc1400c0 2022-06-29T15:44:59.132 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.115+0000 7f9cc1c94700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cbc0b6d30 0x7f9cbc13b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:59.132 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.115+0000 7f9cc1c94700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cbc0b6d30 0x7f9cbc13b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48102/0 (socket says 172.21.15.47:48102) 2022-06-29T15:44:59.133 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.115+0000 7f9cc2495700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9cbc062f30 0x7f9cbc142bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:59.133 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.115+0000 7f9cc1c94700 1 -- 172.21.15.47:0/1168082143 learned_addr learned my addr 172.21.15.47:0/1168082143 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:59.133 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.115+0000 7f9cc2c96700 1 --2- 172.21.15.47:0/1168082143 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9cbc1400c0 0x7f9cbc13bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:59.134 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.115+0000 7f9cc1c94700 1 -- 172.21.15.47:0/1168082143 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9cbc1400c0 msgr2=0x7f9cbc13bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:59.134 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.115+0000 7f9cc1c94700 1 --2- 172.21.15.47:0/1168082143 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9cbc1400c0 0x7f9cbc13bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:59.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.115+0000 7f9cc1c94700 1 -- 172.21.15.47:0/1168082143 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9cbc062f30 msgr2=0x7f9cbc142bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:59.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.115+0000 7f9cc1c94700 1 --2- 172.21.15.47:0/1168082143 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9cbc062f30 0x7f9cbc142bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:59.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.115+0000 7f9cc1c94700 1 -- 172.21.15.47:0/1168082143 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb800b1e0 con 0x7f9cbc0b6d30 2022-06-29T15:44:59.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.115+0000 7f9cc1c94700 1 --2- 172.21.15.47:0/1168082143 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cbc0b6d30 0x7f9cbc13b810 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7f9cb8002f70 tx=0x7f9cb8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:59.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.115+0000 7f9cab7fe700 1 -- 172.21.15.47:0/1168082143 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9cb8011860 con 0x7f9cbc0b6d30 2022-06-29T15:44:59.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.115+0000 7f9cab7fe700 1 -- 172.21.15.47:0/1168082143 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cb80119c0 con 0x7f9cbc0b6d30 2022-06-29T15:44:59.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.115+0000 7f9cc3497700 1 -- 172.21.15.47:0/1168082143 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cbc141da0 con 0x7f9cbc0b6d30 2022-06-29T15:44:59.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.115+0000 7f9cab7fe700 1 -- 172.21.15.47:0/1168082143 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9cb8011d20 con 0x7f9cbc0b6d30 2022-06-29T15:44:59.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.115+0000 7f9cc3497700 1 -- 172.21.15.47:0/1168082143 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cbc13d280 con 0x7f9cbc0b6d30 2022-06-29T15:44:59.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.115+0000 7f9cc3497700 1 -- 172.21.15.47:0/1168082143 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cbc0573d0 con 0x7f9cbc0b6d30 2022-06-29T15:44:59.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.119+0000 7f9cab7fe700 1 -- 172.21.15.47:0/1168082143 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9cb8025030 con 0x7f9cbc0b6d30 2022-06-29T15:44:59.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.119+0000 7f9cab7fe700 1 --2- 172.21.15.47:0/1168082143 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9c9c053950 0x7f9c9c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:59.138 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.119+0000 7f9cab7fe700 1 -- 172.21.15.47:0/1168082143 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9cb806d720 con 0x7f9cbc0b6d30 2022-06-29T15:44:59.138 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.119+0000 7f9cc2495700 1 --2- 172.21.15.47:0/1168082143 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9c9c053950 0x7f9c9c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:59.139 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.119+0000 7f9cc2495700 1 --2- 172.21.15.47:0/1168082143 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9c9c053950 0x7f9c9c055e10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f9cac00b000 tx=0x7f9cac006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:59.139 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.123+0000 7f9cab7fe700 1 -- 172.21.15.47:0/1168082143 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9cb80443d0 con 0x7f9cbc0b6d30 2022-06-29T15:44:59.254 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.251+0000 7f9cc3497700 1 -- 172.21.15.47:0/1168082143 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9cbc069e20 con 0x7f9cbc0b6d30 2022-06-29T15:44:59.255 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.251+0000 7f9cab7fe700 1 -- 172.21.15.47:0/1168082143 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v47) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9cb80230a0 con 0x7f9cbc0b6d30 2022-06-29T15:44:59.256 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:44:59.257 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.251+0000 7f9cc3497700 1 -- 172.21.15.47:0/1168082143 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9c9c053950 msgr2=0x7f9c9c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:59.257 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.251+0000 7f9cc3497700 1 --2- 172.21.15.47:0/1168082143 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9c9c053950 0x7f9c9c055e10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f9cac00b000 tx=0x7f9cac006df0).stop 2022-06-29T15:44:59.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.251+0000 7f9cc3497700 1 -- 172.21.15.47:0/1168082143 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cbc0b6d30 msgr2=0x7f9cbc13b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:59.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.251+0000 7f9cc3497700 1 --2- 172.21.15.47:0/1168082143 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cbc0b6d30 0x7f9cbc13b810 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7f9cb8002f70 tx=0x7f9cb8015000).stop 2022-06-29T15:44:59.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.251+0000 7f9cc3497700 1 -- 172.21.15.47:0/1168082143 shutdown_connections 2022-06-29T15:44:59.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.251+0000 7f9cc3497700 1 --2- 172.21.15.47:0/1168082143 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9cbc1400c0 0x7f9cbc13bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:59.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.251+0000 7f9cc3497700 1 --2- 172.21.15.47:0/1168082143 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cbc0b6d30 0x7f9cbc13b810 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:59.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.251+0000 7f9cc3497700 1 --2- 172.21.15.47:0/1168082143 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9c9c053950 0x7f9c9c055e10 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:59.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.251+0000 7f9cc3497700 1 --2- 172.21.15.47:0/1168082143 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9cbc062f30 0x7f9cbc142bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:59.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.251+0000 7f9cc3497700 1 -- 172.21.15.47:0/1168082143 >> 172.21.15.47:0/1168082143 conn(0x7f9cbc063670 msgr2=0x7f9cbc13f570 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:59.260 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.251+0000 7f9cc3497700 1 -- 172.21.15.47:0/1168082143 shutdown_connections 2022-06-29T15:44:59.260 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.251+0000 7f9cc3497700 1 -- 172.21.15.47:0/1168082143 wait complete. 2022-06-29T15:44:59.269 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-29T15:44:59.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.379+0000 7fd2e359e700 1 Processor -- start 2022-06-29T15:44:59.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.379+0000 7fd2e359e700 1 -- start start 2022-06-29T15:44:59.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.379+0000 7fd2e359e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd2e40b5b20 0x7fd2e413d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:59.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.379+0000 7fd2e359e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd2e40a9250 0x7fd2e4140160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:59.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.379+0000 7fd2e359e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2e40b0190 con 0x7fd2e40a9c60 2022-06-29T15:44:59.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.379+0000 7fd2e359e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2e40aa570 con 0x7fd2e40a9250 2022-06-29T15:44:59.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.379+0000 7fd2e359e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e40b03c0 con 0x7fd2e40b5b20 2022-06-29T15:44:59.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.379+0000 7fd2e2d9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd2e40a9250 0x7fd2e4140160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:59.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.379+0000 7fd2e2d9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd2e40a9250 0x7fd2e4140160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48112/0 (socket says 172.21.15.47:48112) 2022-06-29T15:44:59.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.379+0000 7fd2e2d9d700 1 -- 172.21.15.47:0/2369654075 learned_addr learned my addr 172.21.15.47:0/2369654075 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:59.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.379+0000 7fd2e259c700 1 --2- 172.21.15.47:0/2369654075 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd2e40b5b20 0x7fd2e413d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:59.394 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.379+0000 7fd2e2d9d700 1 -- 172.21.15.47:0/2369654075 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd2e40b5b20 msgr2=0x7fd2e413d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:44:59.394 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.379+0000 7fd2e2d9d700 1 --2- 172.21.15.47:0/2369654075 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd2e40b5b20 0x7fd2e413d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:59.394 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.379+0000 7fd2e2d9d700 1 -- 172.21.15.47:0/2369654075 >> v1:172.21.15.64:6789/0 conn(0x7fd2e40a9c60 legacy=0x7fd2e413d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:44:59.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.379+0000 7fd2e2d9d700 1 -- 172.21.15.47:0/2369654075 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e40a9670 con 0x7fd2e40a9250 2022-06-29T15:44:59.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.379+0000 7fd2e2d9d700 1 --2- 172.21.15.47:0/2369654075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2e40a9250 0x7fd2e4140160 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7fd2e4061e50 tx=0x7fd2dc00b530).ready entity=mon.0 client_cookie=92c895f212b8900f server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:59.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.379+0000 7fd2e159a700 1 -- 172.21.15.47:0/2369654075 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2dc00be70 con 0x7fd2e40a9250 2022-06-29T15:44:59.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.379+0000 7fd2e159a700 1 -- 172.21.15.47:0/2369654075 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2dc010470 con 0x7fd2e40a9250 2022-06-29T15:44:59.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.379+0000 7fd2e159a700 1 -- 172.21.15.47:0/2369654075 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2dc010810 con 0x7fd2e40a9250 2022-06-29T15:44:59.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.383+0000 7fd2e359e700 1 -- 172.21.15.47:0/2369654075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2e40a9250 msgr2=0x7fd2e4140160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:59.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.383+0000 7fd2e359e700 1 --2- 172.21.15.47:0/2369654075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2e40a9250 0x7fd2e4140160 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7fd2e4061e50 tx=0x7fd2dc00b530).stop 2022-06-29T15:44:59.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.383+0000 7fd2e359e700 1 -- 172.21.15.47:0/2369654075 shutdown_connections 2022-06-29T15:44:59.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.383+0000 7fd2e359e700 1 --2- 172.21.15.47:0/2369654075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2e40a9250 0x7fd2e4140160 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:59.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.383+0000 7fd2e359e700 1 --2- 172.21.15.47:0/2369654075 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd2e40b5b20 0x7fd2e413d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:59.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.383+0000 7fd2e359e700 1 -- 172.21.15.47:0/2369654075 >> 172.21.15.47:0/2369654075 conn(0x7fd2e4062560 msgr2=0x7fd2e4062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:59.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.383+0000 7fd2e359e700 1 -- 172.21.15.47:0/2369654075 shutdown_connections 2022-06-29T15:44:59.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.383+0000 7fd2e359e700 1 -- 172.21.15.47:0/2369654075 wait complete. 2022-06-29T15:44:59.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.383+0000 7fd2e359e700 1 Processor -- start 2022-06-29T15:44:59.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.383+0000 7fd2e359e700 1 -- start start 2022-06-29T15:44:59.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.387+0000 7fd2e359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2e40a9250 0x7fd2e40b13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:59.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.387+0000 7fd2e359e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd2e40a9c60 0x7fd2e414e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:59.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.387+0000 7fd2e359e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2e40b5b20 0x7fd2e414ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:59.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.387+0000 7fd2e359e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e4139830 con 0x7fd2e40b5b20 2022-06-29T15:44:59.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.387+0000 7fd2e359e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e413a0a0 con 0x7fd2e40a9250 2022-06-29T15:44:59.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.387+0000 7fd2e359e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e4139a60 con 0x7fd2e40a9c60 2022-06-29T15:44:59.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.387+0000 7fd2e2d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2e40b5b20 0x7fd2e414ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:59.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.387+0000 7fd2e2d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2e40b5b20 0x7fd2e414ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48118/0 (socket says 172.21.15.47:48118) 2022-06-29T15:44:59.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.387+0000 7fd2e2d9d700 1 -- 172.21.15.47:0/1163745390 learned_addr learned my addr 172.21.15.47:0/1163745390 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:59.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.387+0000 7fd2e259c700 1 --2- 172.21.15.47:0/1163745390 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2e40a9250 0x7fd2e40b13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:59.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.387+0000 7fd2e1d9b700 1 --2- 172.21.15.47:0/1163745390 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd2e40a9c60 0x7fd2e414e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:59.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.387+0000 7fd2e2d9d700 1 -- 172.21.15.47:0/1163745390 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd2e40a9c60 msgr2=0x7fd2e414e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:59.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.387+0000 7fd2e2d9d700 1 --2- 172.21.15.47:0/1163745390 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd2e40a9c60 0x7fd2e414e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:59.409 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.387+0000 7fd2e2d9d700 1 -- 172.21.15.47:0/1163745390 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2e40a9250 msgr2=0x7fd2e40b13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:59.409 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.387+0000 7fd2e2d9d700 1 --2- 172.21.15.47:0/1163745390 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2e40a9250 0x7fd2e40b13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:59.409 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.387+0000 7fd2e2d9d700 1 -- 172.21.15.47:0/1163745390 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2dc00b7d0 con 0x7fd2e40b5b20 2022-06-29T15:44:59.409 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.387+0000 7fd2e2d9d700 1 --2- 172.21.15.47:0/1163745390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2e40b5b20 0x7fd2e414ead0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7fd2e4061e50 tx=0x7fd2dc00b530).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:59.410 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.387+0000 7fd2db7fe700 1 -- 172.21.15.47:0/1163745390 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2dc01d070 con 0x7fd2e40b5b20 2022-06-29T15:44:59.410 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.387+0000 7fd2db7fe700 1 -- 172.21.15.47:0/1163745390 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2dc018800 con 0x7fd2e40b5b20 2022-06-29T15:44:59.410 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.387+0000 7fd2e359e700 1 -- 172.21.15.47:0/1163745390 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2e4144040 con 0x7fd2e40b5b20 2022-06-29T15:44:59.410 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.387+0000 7fd2db7fe700 1 -- 172.21.15.47:0/1163745390 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2dc018b60 con 0x7fd2e40b5b20 2022-06-29T15:44:59.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.387+0000 7fd2e359e700 1 -- 172.21.15.47:0/1163745390 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2e413a340 con 0x7fd2e40b5b20 2022-06-29T15:44:59.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.387+0000 7fd2db7fe700 1 -- 172.21.15.47:0/1163745390 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd2dc029070 con 0x7fd2e40b5b20 2022-06-29T15:44:59.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.387+0000 7fd2e359e700 1 -- 172.21.15.47:0/1163745390 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2e4139de0 con 0x7fd2e40b5b20 2022-06-29T15:44:59.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.387+0000 7fd2db7fe700 1 --2- 172.21.15.47:0/1163745390 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd2bc053950 0x7fd2bc055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:59.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.391+0000 7fd2e259c700 1 --2- 172.21.15.47:0/1163745390 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd2bc053950 0x7fd2bc055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:59.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.391+0000 7fd2db7fe700 1 -- 172.21.15.47:0/1163745390 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd2dc06dac0 con 0x7fd2e40b5b20 2022-06-29T15:44:59.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.391+0000 7fd2e259c700 1 --2- 172.21.15.47:0/1163745390 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd2bc053950 0x7fd2bc055e10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7fd2d400c000 tx=0x7fd2d4008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:59.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.395+0000 7fd2db7fe700 1 -- 172.21.15.47:0/1163745390 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd2dc044770 con 0x7fd2e40b5b20 2022-06-29T15:44:59.527 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.523+0000 7fd2e359e700 1 -- 172.21.15.47:0/1163745390 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd2e40573d0 con 0x7fd2e40b5b20 2022-06-29T15:44:59.528 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.523+0000 7fd2db7fe700 1 -- 172.21.15.47:0/1163745390 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v47) v1 ==== 104+0+10 (secure 0 0 0) 0x7fd2dc047d90 con 0x7fd2e40b5b20 2022-06-29T15:44:59.529 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:44:59.531 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.523+0000 7fd2e359e700 1 -- 172.21.15.47:0/1163745390 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd2bc053950 msgr2=0x7fd2bc055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:59.532 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.523+0000 7fd2e359e700 1 --2- 172.21.15.47:0/1163745390 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd2bc053950 0x7fd2bc055e10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7fd2d400c000 tx=0x7fd2d4008040).stop 2022-06-29T15:44:59.532 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.523+0000 7fd2e359e700 1 -- 172.21.15.47:0/1163745390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2e40b5b20 msgr2=0x7fd2e414ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:59.533 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.523+0000 7fd2e359e700 1 --2- 172.21.15.47:0/1163745390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2e40b5b20 0x7fd2e414ead0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7fd2e4061e50 tx=0x7fd2dc00b530).stop 2022-06-29T15:44:59.533 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.523+0000 7fd2e359e700 1 -- 172.21.15.47:0/1163745390 shutdown_connections 2022-06-29T15:44:59.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.523+0000 7fd2e359e700 1 --2- 172.21.15.47:0/1163745390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2e40b5b20 0x7fd2e414ead0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:59.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.523+0000 7fd2e359e700 1 --2- 172.21.15.47:0/1163745390 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd2e40a9c60 0x7fd2e414e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:59.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.523+0000 7fd2e359e700 1 --2- 172.21.15.47:0/1163745390 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd2bc053950 0x7fd2bc055e10 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:59.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.523+0000 7fd2e359e700 1 --2- 172.21.15.47:0/1163745390 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2e40a9250 0x7fd2e40b13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:59.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.523+0000 7fd2e359e700 1 -- 172.21.15.47:0/1163745390 >> 172.21.15.47:0/1163745390 conn(0x7fd2e4062560 msgr2=0x7fd2e413f410 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:59.536 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.523+0000 7fd2e359e700 1 -- 172.21.15.47:0/1163745390 shutdown_connections 2022-06-29T15:44:59.536 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.523+0000 7fd2e359e700 1 -- 172.21.15.47:0/1163745390 wait complete. 2022-06-29T15:44:59.542 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-29T15:44:59.652 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.647+0000 7f0bfa641700 1 Processor -- start 2022-06-29T15:44:59.652 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.647+0000 7f0bfa641700 1 -- start start 2022-06-29T15:44:59.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.647+0000 7f0bfa641700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0bf405c850 0x7f0bf40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:59.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.647+0000 7f0bfa641700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0bf4067630 0x7f0bf4144410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:59.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.647+0000 7f0bfa641700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bf405cc30 con 0x7f0bf4148150 2022-06-29T15:44:59.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.647+0000 7f0bfa641700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bf41474d0 con 0x7f0bf4067630 2022-06-29T15:44:59.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.647+0000 7f0bfa641700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf4147770 con 0x7f0bf405c850 2022-06-29T15:44:59.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.647+0000 7f0bf9e40700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0bf4067630 0x7f0bf4144410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:59.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.647+0000 7f0bf8e3e700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0bf4148150 0x7f0bf40ad740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33678/0 (socket says 172.21.15.47:33678) 2022-06-29T15:44:59.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.647+0000 7f0bf8e3e700 1 -- 172.21.15.47:0/3908354904 learned_addr learned my addr 172.21.15.47:0/3908354904 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:44:59.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.647+0000 7f0bf963f700 1 --2- 172.21.15.47:0/3908354904 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0bf405c850 0x7f0bf40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:59.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.647+0000 7f0bf9e40700 1 -- 172.21.15.47:0/3908354904 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0bf405c850 msgr2=0x7f0bf40b4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:44:59.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.647+0000 7f0bf9e40700 1 --2- 172.21.15.47:0/3908354904 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0bf405c850 0x7f0bf40b4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:59.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.647+0000 7f0bf9e40700 1 -- 172.21.15.47:0/3908354904 >> v1:172.21.15.47:6789/0 conn(0x7f0bf4148150 legacy=0x7f0bf40ad740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:59.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.647+0000 7f0bf9e40700 1 -- 172.21.15.47:0/3908354904 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf4147a10 con 0x7f0bf4067630 2022-06-29T15:44:59.663 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.647+0000 7f0be7fff700 1 -- 172.21.15.47:0/3908354904 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1234566554 0 0) 0x7f0bf405cc30 con 0x7f0bf4148150 2022-06-29T15:44:59.663 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.651+0000 7f0bf9e40700 1 --2- 172.21.15.47:0/3908354904 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bf4067630 0x7f0bf4144410 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7f0bec0032f0 tx=0x7f0bec0091d0).ready entity=mon.0 client_cookie=f375f2093af69f1d server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:59.663 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.651+0000 7f0be7fff700 1 -- 172.21.15.47:0/3908354904 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0bec00b6c0 con 0x7f0bf4067630 2022-06-29T15:44:59.664 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.651+0000 7f0be7fff700 1 -- 172.21.15.47:0/3908354904 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bec00b820 con 0x7f0bf4067630 2022-06-29T15:44:59.664 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.651+0000 7f0be7fff700 1 -- 172.21.15.47:0/3908354904 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0bec00b990 con 0x7f0bf4067630 2022-06-29T15:44:59.664 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.651+0000 7f0bfa641700 1 -- 172.21.15.47:0/3908354904 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bf4067630 msgr2=0x7f0bf4144410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:59.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.651+0000 7f0bfa641700 1 --2- 172.21.15.47:0/3908354904 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bf4067630 0x7f0bf4144410 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7f0bec0032f0 tx=0x7f0bec0091d0).stop 2022-06-29T15:44:59.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.651+0000 7f0bfa641700 1 -- 172.21.15.47:0/3908354904 shutdown_connections 2022-06-29T15:44:59.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.651+0000 7f0bfa641700 1 --2- 172.21.15.47:0/3908354904 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bf4067630 0x7f0bf4144410 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:59.666 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.651+0000 7f0bfa641700 1 --2- 172.21.15.47:0/3908354904 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0bf405c850 0x7f0bf40b4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:59.666 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.651+0000 7f0bfa641700 1 -- 172.21.15.47:0/3908354904 >> 172.21.15.47:0/3908354904 conn(0x7f0bf4060e00 msgr2=0x7f0bf40611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:59.666 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.651+0000 7f0bfa641700 1 -- 172.21.15.47:0/3908354904 shutdown_connections 2022-06-29T15:44:59.667 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.651+0000 7f0bfa641700 1 -- 172.21.15.47:0/3908354904 wait complete. 2022-06-29T15:44:59.667 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.651+0000 7f0bfa641700 1 Processor -- start 2022-06-29T15:44:59.667 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.651+0000 7f0bfa641700 1 -- start start 2022-06-29T15:44:59.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.655+0000 7f0bfa641700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bf405c850 0x7f0bf4140c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:59.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.655+0000 7f0bfa641700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bf4067630 0x7f0bf4156f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:59.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.655+0000 7f0bfa641700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0bf4148150 0x7f0bf413d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:59.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.655+0000 7f0bfa641700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf40b0f60 con 0x7f0bf405c850 2022-06-29T15:44:59.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.655+0000 7f0bfa641700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf4157440 con 0x7f0bf4067630 2022-06-29T15:44:59.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.655+0000 7f0bfa641700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf40b1190 con 0x7f0bf4148150 2022-06-29T15:44:59.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.655+0000 7f0bf963f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bf405c850 0x7f0bf4140c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:59.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.655+0000 7f0bf963f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bf405c850 0x7f0bf4140c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48126/0 (socket says 172.21.15.47:48126) 2022-06-29T15:44:59.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.655+0000 7f0bf963f700 1 -- 172.21.15.47:0/381821764 learned_addr learned my addr 172.21.15.47:0/381821764 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:59.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.655+0000 7f0bf9e40700 1 --2- 172.21.15.47:0/381821764 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0bf4148150 0x7f0bf413d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:59.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.655+0000 7f0bf8e3e700 1 --2- 172.21.15.47:0/381821764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bf4067630 0x7f0bf4156f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:59.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.655+0000 7f0bf963f700 1 -- 172.21.15.47:0/381821764 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0bf4148150 msgr2=0x7f0bf413d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:59.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.655+0000 7f0bf963f700 1 --2- 172.21.15.47:0/381821764 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0bf4148150 0x7f0bf413d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:59.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.655+0000 7f0bf963f700 1 -- 172.21.15.47:0/381821764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bf4067630 msgr2=0x7f0bf4156f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:59.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.655+0000 7f0bf963f700 1 --2- 172.21.15.47:0/381821764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bf4067630 0x7f0bf4156f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:59.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.655+0000 7f0bf963f700 1 -- 172.21.15.47:0/381821764 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bec009470 con 0x7f0bf405c850 2022-06-29T15:44:59.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.655+0000 7f0bf8e3e700 1 --2- 172.21.15.47:0/381821764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bf4067630 0x7f0bf4156f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:44:59.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.655+0000 7f0bf963f700 1 --2- 172.21.15.47:0/381821764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bf405c850 0x7f0bf4140c20 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7f0be800c000 tx=0x7f0be8006d90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:59.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.655+0000 7f0be6ffd700 1 -- 172.21.15.47:0/381821764 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0be80104c0 con 0x7f0bf405c850 2022-06-29T15:44:59.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.655+0000 7f0be6ffd700 1 -- 172.21.15.47:0/381821764 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0be8010620 con 0x7f0bf405c850 2022-06-29T15:44:59.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.655+0000 7f0bfa641700 1 -- 172.21.15.47:0/381821764 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bf40b1430 con 0x7f0bf405c850 2022-06-29T15:44:59.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.655+0000 7f0be6ffd700 1 -- 172.21.15.47:0/381821764 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0be8010980 con 0x7f0bf405c850 2022-06-29T15:44:59.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.655+0000 7f0bfa641700 1 -- 172.21.15.47:0/381821764 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bf40b1970 con 0x7f0bf405c850 2022-06-29T15:44:59.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.655+0000 7f0bfa641700 1 -- 172.21.15.47:0/381821764 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bf40573d0 con 0x7f0bf405c850 2022-06-29T15:44:59.676 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.659+0000 7f0be6ffd700 1 -- 172.21.15.47:0/381821764 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0be801a030 con 0x7f0bf405c850 2022-06-29T15:44:59.676 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.659+0000 7f0be6ffd700 1 --2- 172.21.15.47:0/381821764 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0bd0053a70 0x7f0bd0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:59.676 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.659+0000 7f0be6ffd700 1 -- 172.21.15.47:0/381821764 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f0be806bf10 con 0x7f0bf405c850 2022-06-29T15:44:59.676 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.659+0000 7f0bf8e3e700 1 --2- 172.21.15.47:0/381821764 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0bd0053a70 0x7f0bd0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:59.677 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.659+0000 7f0bf8e3e700 1 --2- 172.21.15.47:0/381821764 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0bd0053a70 0x7f0bd0055f30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f0bf0008000 tx=0x7f0bf0009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:59.677 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.663+0000 7f0be6ffd700 1 -- 172.21.15.47:0/381821764 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0be8042bc0 con 0x7f0bf405c850 2022-06-29T15:44:59.805 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.799+0000 7f0bfa641700 1 -- 172.21.15.47:0/381821764 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0bf40b7630 con 0x7f0bf405c850 2022-06-29T15:44:59.806 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.799+0000 7f0be6ffd700 1 -- 172.21.15.47:0/381821764 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v47) v1 ==== 100+0+10 (secure 0 0 0) 0x7f0be80461e0 con 0x7f0bf405c850 2022-06-29T15:44:59.806 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:44:59.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.803+0000 7f0bfa641700 1 -- 172.21.15.47:0/381821764 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0bd0053a70 msgr2=0x7f0bd0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:59.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.803+0000 7f0bfa641700 1 --2- 172.21.15.47:0/381821764 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0bd0053a70 0x7f0bd0055f30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f0bf0008000 tx=0x7f0bf0009040).stop 2022-06-29T15:44:59.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.803+0000 7f0bfa641700 1 -- 172.21.15.47:0/381821764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bf405c850 msgr2=0x7f0bf4140c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:59.809 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.803+0000 7f0bfa641700 1 --2- 172.21.15.47:0/381821764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bf405c850 0x7f0bf4140c20 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7f0be800c000 tx=0x7f0be8006d90).stop 2022-06-29T15:44:59.809 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.803+0000 7f0bfa641700 1 -- 172.21.15.47:0/381821764 shutdown_connections 2022-06-29T15:44:59.809 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.803+0000 7f0bfa641700 1 --2- 172.21.15.47:0/381821764 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0bf4148150 0x7f0bf413d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:59.809 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.803+0000 7f0bfa641700 1 --2- 172.21.15.47:0/381821764 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0bd0053a70 0x7f0bd0055f30 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:59.810 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.803+0000 7f0bfa641700 1 --2- 172.21.15.47:0/381821764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bf4067630 0x7f0bf4156f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:59.810 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.803+0000 7f0bfa641700 1 --2- 172.21.15.47:0/381821764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bf405c850 0x7f0bf4140c20 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:59.810 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.803+0000 7f0bfa641700 1 -- 172.21.15.47:0/381821764 >> 172.21.15.47:0/381821764 conn(0x7f0bf4060e00 msgr2=0x7f0bf40b5b90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:59.810 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.803+0000 7f0bfa641700 1 -- 172.21.15.47:0/381821764 shutdown_connections 2022-06-29T15:44:59.810 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.803+0000 7f0bfa641700 1 -- 172.21.15.47:0/381821764 wait complete. 2022-06-29T15:44:59.819 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-29T15:44:59.819 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:44:59.924 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.915+0000 7fef6ad38700 1 Processor -- start 2022-06-29T15:44:59.924 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.915+0000 7fef6ad38700 1 -- start start 2022-06-29T15:44:59.925 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.915+0000 7fef6ad38700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fef640b2120 0x7fef640b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:59.925 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.915+0000 7fef6ad38700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fef6405f0b0 0x7fef6414b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:59.926 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.915+0000 7fef6ad38700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef640b0740 con 0x7fef640b1260 2022-06-29T15:44:59.926 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.915+0000 7fef6ad38700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef64147460 con 0x7fef6405f0b0 2022-06-29T15:44:59.927 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.915+0000 7fef6ad38700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef640b0970 con 0x7fef640b2120 2022-06-29T15:44:59.927 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.915+0000 7fef68ad4700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fef640b2120 0x7fef640b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:59.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.915+0000 7fef63fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fef6405f0b0 0x7fef6414b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:59.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.915+0000 7fef68ad4700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fef640b2120 0x7fef640b2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:45236/0 (socket says 172.21.15.47:45236) 2022-06-29T15:44:59.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.915+0000 7fef68ad4700 1 -- 172.21.15.47:0/69479290 learned_addr learned my addr 172.21.15.47:0/69479290 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:59.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.915+0000 7fef68ad4700 1 -- 172.21.15.47:0/69479290 >> v2:172.21.15.64:3300/0 conn(0x7fef6405f0b0 msgr2=0x7fef6414b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:44:59.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.915+0000 7fef68ad4700 1 --2- 172.21.15.47:0/69479290 >> v2:172.21.15.64:3300/0 conn(0x7fef6405f0b0 0x7fef6414b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:59.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.915+0000 7fef68ad4700 1 -- 172.21.15.47:0/69479290 >> v1:172.21.15.64:6789/0 conn(0x7fef640b1260 legacy=0x7fef64147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:59.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.915+0000 7fef68ad4700 1 -- 172.21.15.47:0/69479290 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef640b0c10 con 0x7fef640b2120 2022-06-29T15:44:59.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.915+0000 7fef68ad4700 1 --2- 172.21.15.47:0/69479290 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fef640b2120 0x7fef640b2500 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fef4c002f70 tx=0x7fef4c006eb0).ready entity=mon.2 client_cookie=3cc7682d9bf468f1 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:59.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.915+0000 7fef637fe700 1 -- 172.21.15.47:0/69479290 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef4c00d570 con 0x7fef640b2120 2022-06-29T15:44:59.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.915+0000 7fef637fe700 1 -- 172.21.15.47:0/69479290 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef4c000e90 con 0x7fef640b2120 2022-06-29T15:44:59.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.915+0000 7fef637fe700 1 -- 172.21.15.47:0/69479290 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef4c00d900 con 0x7fef640b2120 2022-06-29T15:44:59.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.915+0000 7fef6ad38700 1 -- 172.21.15.47:0/69479290 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fef640b2120 msgr2=0x7fef640b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:59.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.915+0000 7fef6ad38700 1 --2- 172.21.15.47:0/69479290 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fef640b2120 0x7fef640b2500 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fef4c002f70 tx=0x7fef4c006eb0).stop 2022-06-29T15:44:59.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.915+0000 7fef6ad38700 1 -- 172.21.15.47:0/69479290 shutdown_connections 2022-06-29T15:44:59.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.915+0000 7fef6ad38700 1 --2- 172.21.15.47:0/69479290 >> v2:172.21.15.64:3300/0 conn(0x7fef6405f0b0 0x7fef6414b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:59.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.915+0000 7fef6ad38700 1 --2- 172.21.15.47:0/69479290 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fef640b2120 0x7fef640b2500 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:59.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.915+0000 7fef6ad38700 1 -- 172.21.15.47:0/69479290 >> 172.21.15.47:0/69479290 conn(0x7fef640626f0 msgr2=0x7fef64062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:44:59.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.919+0000 7fef6ad38700 1 -- 172.21.15.47:0/69479290 shutdown_connections 2022-06-29T15:44:59.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.919+0000 7fef6ad38700 1 -- 172.21.15.47:0/69479290 wait complete. 2022-06-29T15:44:59.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.919+0000 7fef6ad38700 1 Processor -- start 2022-06-29T15:44:59.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.919+0000 7fef6ad38700 1 -- start start 2022-06-29T15:44:59.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.919+0000 7fef6ad38700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fef640b1260 0x7fef6414e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:59.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.919+0000 7fef6ad38700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef640b2120 0x7fef6415a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:59.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.919+0000 7fef6ad38700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef64143d10 0x7fef6415a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:59.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.919+0000 7fef6ad38700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6415aa60 con 0x7fef64143d10 2022-06-29T15:44:59.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.919+0000 7fef6ad38700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6414ecd0 con 0x7fef640b2120 2022-06-29T15:44:59.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.919+0000 7fef6ad38700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef64141b50 con 0x7fef640b1260 2022-06-29T15:44:59.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.919+0000 7fef692d5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef64143d10 0x7fef6415a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:59.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.919+0000 7fef68ad4700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fef640b1260 0x7fef6414e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:59.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.919+0000 7fef63fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef640b2120 0x7fef6415a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:59.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.919+0000 7fef692d5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef64143d10 0x7fef6415a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48144/0 (socket says 172.21.15.47:48144) 2022-06-29T15:44:59.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.919+0000 7fef692d5700 1 -- 172.21.15.47:0/2282746313 learned_addr learned my addr 172.21.15.47:0/2282746313 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:44:59.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.919+0000 7fef692d5700 1 -- 172.21.15.47:0/2282746313 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fef640b1260 msgr2=0x7fef6414e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:59.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.919+0000 7fef692d5700 1 --2- 172.21.15.47:0/2282746313 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fef640b1260 0x7fef6414e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:59.950 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.919+0000 7fef692d5700 1 -- 172.21.15.47:0/2282746313 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef640b2120 msgr2=0x7fef6415a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:44:59.951 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.919+0000 7fef692d5700 1 --2- 172.21.15.47:0/2282746313 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef640b2120 0x7fef6415a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:44:59.951 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.919+0000 7fef692d5700 1 -- 172.21.15.47:0/2282746313 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef4c00b1e0 con 0x7fef64143d10 2022-06-29T15:44:59.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.919+0000 7fef692d5700 1 --2- 172.21.15.47:0/2282746313 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef64143d10 0x7fef6415a570 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7fef6406af60 tx=0x7fef5c007bc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:59.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.923+0000 7fef61ffb700 1 -- 172.21.15.47:0/2282746313 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef5c012400 con 0x7fef64143d10 2022-06-29T15:44:59.953 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.923+0000 7fef61ffb700 1 -- 172.21.15.47:0/2282746313 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef5c012560 con 0x7fef64143d10 2022-06-29T15:44:59.953 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.923+0000 7fef61ffb700 1 -- 172.21.15.47:0/2282746313 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef5c0128c0 con 0x7fef64143d10 2022-06-29T15:44:59.954 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.923+0000 7fef6ad38700 1 -- 172.21.15.47:0/2282746313 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef64141df0 con 0x7fef64143d10 2022-06-29T15:44:59.954 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.923+0000 7fef6ad38700 1 -- 172.21.15.47:0/2282746313 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef64142330 con 0x7fef64143d10 2022-06-29T15:44:59.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.923+0000 7fef6ad38700 1 -- 172.21.15.47:0/2282746313 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef641424f0 con 0x7fef64143d10 2022-06-29T15:44:59.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.923+0000 7fef61ffb700 1 -- 172.21.15.47:0/2282746313 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fef5c019030 con 0x7fef64143d10 2022-06-29T15:44:59.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.923+0000 7fef61ffb700 1 --2- 172.21.15.47:0/2282746313 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fef3c053a70 0x7fef3c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:44:59.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.923+0000 7fef61ffb700 1 -- 172.21.15.47:0/2282746313 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fef5c06b2c0 con 0x7fef64143d10 2022-06-29T15:44:59.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.923+0000 7fef68ad4700 1 --2- 172.21.15.47:0/2282746313 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fef3c053a70 0x7fef3c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:44:59.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.927+0000 7fef68ad4700 1 --2- 172.21.15.47:0/2282746313 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fef3c053a70 0x7fef3c055f30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7fef4c01c000 tx=0x7fef4c015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:44:59.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:44:59.931+0000 7fef61ffb700 1 -- 172.21.15.47:0/2282746313 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fef5c042080 con 0x7fef64143d10 2022-06-29T15:45:00.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:00.099+0000 7fef6ad38700 1 -- 172.21.15.47:0/2282746313 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef640573d0 con 0x7fef64143d10 2022-06-29T15:45:00.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:00.099+0000 7fef61ffb700 1 -- 172.21.15.47:0/2282746313 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+2491 (secure 0 0 0) 0x7fef5c0456a0 con 0x7fef64143d10 2022-06-29T15:45:00.104 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:45:00.104 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":17,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5259,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3601815346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3601815346},{"type":"v1","addr":"172.21.15.47:6835","nonce":3601815346}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15},{"gid":5262,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/657177602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":657177602},{"type":"v1","addr":"172.21.15.47:6837","nonce":657177602}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:44:27.616735+0000","modified":"2022-06-29T15:44:28.620601+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":54812},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54812":{"gid":54812,"name":"b","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6833/2497658100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2497658100},{"type":"v1","addr":"172.21.15.64:6833","nonce":2497658100}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2022-06-29T15:45:00.106 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:00.099+0000 7fef6ad38700 1 -- 172.21.15.47:0/2282746313 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fef3c053a70 msgr2=0x7fef3c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:00.107 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:00.099+0000 7fef6ad38700 1 --2- 172.21.15.47:0/2282746313 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fef3c053a70 0x7fef3c055f30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7fef4c01c000 tx=0x7fef4c015000).stop 2022-06-29T15:45:00.107 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:00.099+0000 7fef6ad38700 1 -- 172.21.15.47:0/2282746313 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef64143d10 msgr2=0x7fef6415a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:00.108 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:00.099+0000 7fef6ad38700 1 --2- 172.21.15.47:0/2282746313 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef64143d10 0x7fef6415a570 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7fef6406af60 tx=0x7fef5c007bc0).stop 2022-06-29T15:45:00.108 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:00.099+0000 7fef6ad38700 1 -- 172.21.15.47:0/2282746313 shutdown_connections 2022-06-29T15:45:00.109 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:00.099+0000 7fef6ad38700 1 --2- 172.21.15.47:0/2282746313 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef64143d10 0x7fef6415a570 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:00.109 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:00.099+0000 7fef6ad38700 1 --2- 172.21.15.47:0/2282746313 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fef3c053a70 0x7fef3c055f30 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:00.109 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:00.099+0000 7fef6ad38700 1 --2- 172.21.15.47:0/2282746313 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef640b2120 0x7fef6415a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:00.109 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:00.099+0000 7fef6ad38700 1 --2- 172.21.15.47:0/2282746313 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fef640b1260 0x7fef6414e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:00.110 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:00.099+0000 7fef6ad38700 1 -- 172.21.15.47:0/2282746313 >> 172.21.15.47:0/2282746313 conn(0x7fef640626f0 msgr2=0x7fef64145de0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:45:00.110 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:00.099+0000 7fef6ad38700 1 -- 172.21.15.47:0/2282746313 shutdown_connections 2022-06-29T15:45:00.110 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:00.099+0000 7fef6ad38700 1 -- 172.21.15.47:0/2282746313 wait complete. 2022-06-29T15:45:00.111 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 17 2022-06-29T15:45:00.419 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-29T15:44:27.616735+0000', 'modified': '2022-06-29T15:44:28.620601+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 54812}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_54812': {'gid': 54812, 'name': 'b', 'rank': 0, 'incarnation': 16, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.64:6833/2497658100', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6832', 'nonce': 2497658100}, {'type': 'v1', 'addr': '172.21.15.64:6833', 'nonce': 2497658100}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-29T15:45:00.420 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-29T15:45:00.421 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-06-29T15:45:00.537 INFO:teuthology.orchestra.run.smithi064.stdout:{"cluster_fsid":"9a68894a-53ce-400e-838b-14eab91b834c","whoami":0,"id":54812,"want_state":"up:active","state":"up:active","rank_uptime":32.904928118999997,"mdsmap_epoch":17,"osdmap_epoch":47,"osdmap_epoch_barrier":47,"uptime":37.644230700999998} 2022-06-29T15:45:00.538 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9a68894a-53ce-400e-838b-14eab91b834c", "id": 54812, "mdsmap_epoch": 17, "osdmap_epoch": 47, "osdmap_epoch_barrier": 47, "rank_uptime": 32.904928119, "state": "up:active", "uptime": 37.644230701, "want_state": "up:active", "whoami": 0 } 2022-06-29T15:45:00.539 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-29T15:45:00.539 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-06-29T15:45:00.539 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi099.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-06-29T15:45:00.540 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:45:00.540 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:45:00.549 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2022-06-29T15:45:00.566 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:45:00.567 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:45:00.624 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:45:00.625 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:45:00.626 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:45:00.627 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2022-06-29T15:45:00.633 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-29T15:45:00.634 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:45:00.637 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2022-06-29T15:45:00.657 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:45:00.658 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:45:00.715 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:2022-06-29T15:45:00.709+0000 7fc1b768d080 -1 init, newargv = 0x55837f012840 newargc=9 2022-06-29T15:45:00.715 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[12473]: starting ceph client 2022-06-29T15:45:00.716 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:45:00.717 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:45:00.718 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:45:00.719 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2022-06-29T15:45:00.725 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[12473]: starting fuse 2022-06-29T15:45:01.825 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2022-06-29T15:45:01.844 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:45:01.845 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:45:01.903 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:45:01.904 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:45:01.905 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:45:01.906 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2022-06-29T15:45:01.911 INFO:teuthology.orchestra.run.smithi099.stdout:52 2022-06-29T15:45:01.912 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-29T15:45:01.912 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:45:01.913 DEBUG:teuthology.orchestra.run.smithi099:> sudo python3 -c ' 2022-06-29T15:45:01.913 DEBUG:teuthology.orchestra.run.smithi099:> import glob 2022-06-29T15:45:01.913 DEBUG:teuthology.orchestra.run.smithi099:> import re 2022-06-29T15:45:01.914 DEBUG:teuthology.orchestra.run.smithi099:> import os 2022-06-29T15:45:01.914 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2022-06-29T15:45:01.915 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:45:01.915 DEBUG:teuthology.orchestra.run.smithi099:> def find_socket(client_name): 2022-06-29T15:45:01.916 DEBUG:teuthology.orchestra.run.smithi099:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-06-29T15:45:01.916 DEBUG:teuthology.orchestra.run.smithi099:> files = glob.glob(asok_path) 2022-06-29T15:45:01.916 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:45:01.917 DEBUG:teuthology.orchestra.run.smithi099:> # Given a non-glob path, it better be there 2022-06-29T15:45:01.917 DEBUG:teuthology.orchestra.run.smithi099:> if "*" not in asok_path: 2022-06-29T15:45:01.918 DEBUG:teuthology.orchestra.run.smithi099:> assert(len(files) == 1) 2022-06-29T15:45:01.918 DEBUG:teuthology.orchestra.run.smithi099:> return files[0] 2022-06-29T15:45:01.918 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:45:01.919 DEBUG:teuthology.orchestra.run.smithi099:> for f in files: 2022-06-29T15:45:01.919 DEBUG:teuthology.orchestra.run.smithi099:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-29T15:45:01.920 DEBUG:teuthology.orchestra.run.smithi099:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-29T15:45:01.920 DEBUG:teuthology.orchestra.run.smithi099:> return f 2022-06-29T15:45:01.920 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-29T15:45:01.921 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:45:01.922 DEBUG:teuthology.orchestra.run.smithi099:> print(find_socket("client.0")) 2022-06-29T15:45:01.922 DEBUG:teuthology.orchestra.run.smithi099:> ' 2022-06-29T15:45:01.966 INFO:teuthology.orchestra.run.smithi099.stdout:/var/run/ceph/ceph-client.0.12473.asok 2022-06-29T15:45:01.971 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.12473.asok 2022-06-29T15:45:01.972 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:45:01.972 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.12473.asok status 2022-06-29T15:45:02.103 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2022-06-29T15:45:02.104 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": { 2022-06-29T15:45:02.104 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_sha1": "f25ffac72e5a9e63860ff7deb6767a11f0559a16", 2022-06-29T15:45:02.104 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 15.2.16-304-gf25ffac7 (f25ffac72e5a9e63860ff7deb6767a11f0559a16) octopus (stable)", 2022-06-29T15:45:02.105 INFO:teuthology.orchestra.run.smithi099.stdout: "entity_id": "0", 2022-06-29T15:45:02.106 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": "smithi099", 2022-06-29T15:45:02.106 INFO:teuthology.orchestra.run.smithi099.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-06-29T15:45:02.107 INFO:teuthology.orchestra.run.smithi099.stdout: "pid": "12473", 2022-06-29T15:45:02.107 INFO:teuthology.orchestra.run.smithi099.stdout: "root": "/" 2022-06-29T15:45:02.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:45:02.108 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_count": 0, 2022-06-29T15:45:02.108 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_pinned_count": 0, 2022-06-29T15:45:02.109 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 5502, 2022-06-29T15:45:02.109 INFO:teuthology.orchestra.run.smithi099.stdout: "inst": { 2022-06-29T15:45:02.109 INFO:teuthology.orchestra.run.smithi099.stdout: "name": { 2022-06-29T15:45:02.110 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "client", 2022-06-29T15:45:02.110 INFO:teuthology.orchestra.run.smithi099.stdout: "num": 5502 2022-06-29T15:45:02.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:45:02.111 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2022-06-29T15:45:02.112 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2022-06-29T15:45:02.112 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:0", 2022-06-29T15:45:02.113 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1293343062 2022-06-29T15:45:02.114 INFO:teuthology.orchestra.run.smithi099.stdout: } 2022-06-29T15:45:02.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:45:02.115 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2022-06-29T15:45:02.116 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2022-06-29T15:45:02.116 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:0", 2022-06-29T15:45:02.117 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1293343062 2022-06-29T15:45:02.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:45:02.117 INFO:teuthology.orchestra.run.smithi099.stdout: "inst_str": "client.5502 172.21.15.99:0/1293343062", 2022-06-29T15:45:02.118 INFO:teuthology.orchestra.run.smithi099.stdout: "addr_str": "172.21.15.99:0/1293343062", 2022-06-29T15:45:02.118 INFO:teuthology.orchestra.run.smithi099.stdout: "inode_count": 1, 2022-06-29T15:45:02.118 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_epoch": 17, 2022-06-29T15:45:02.119 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch": 47, 2022-06-29T15:45:02.119 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch_barrier": 0, 2022-06-29T15:45:02.120 INFO:teuthology.orchestra.run.smithi099.stdout: "blacklisted": false, 2022-06-29T15:45:02.120 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs" 2022-06-29T15:45:02.121 INFO:teuthology.orchestra.run.smithi099.stdout:} 2022-06-29T15:45:02.123 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:45:02.124 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:45:02.124 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:45:02.132 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2022-06-29T15:45:02.133 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-29T15:45:02.134 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:45:02.134 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:45:11.347 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:45:11.350 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:45:11.353 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:45:11.356 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:45:41.416 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:45:41.420 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:45:41.423 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:45:41.425 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:45:47.894 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-06-29T15:45:47.895 DEBUG:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:45:47.915 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:45:47.937 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:45:47.959 DEBUG:teuthology.orchestra.run.smithi164:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:45:47.981 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-06-29T15:45:47.982 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:45:47.982 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:45:47.983 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:45:47.991 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2022-06-29T15:45:47.992 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-29T15:45:47.993 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi099.front.sepia.ceph.com... 2022-06-29T15:45:47.994 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-06-29T15:45:47.994 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:45:48.010 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[12473]: fuse finished with error 0 and tester_r 0 2022-06-29T15:45:48.013 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:45:48.013 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:45:48.014 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:45:48.062 INFO:teuthology.orchestra.run.smithi099.stdout:ext2/ext3 2022-06-29T15:45:48.064 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-06-29T15:45:48.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-29T15:45:48.177 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:45:48.184 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:45:48.184 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:45:48.185 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:45:48.192 INFO:teuthology.orchestra.run.smithi164.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-06-29T15:45:48.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T15:45:48.194 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-06-29T15:45:48.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-29T15:45:48.195 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:45:48.202 INFO:teuthology.orchestra.run.smithi164.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-06-29T15:45:48.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T15:45:48.204 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-06-29T15:45:48.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.311+0000 7f92ea864700 1 Processor -- start 2022-06-29T15:45:48.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.311+0000 7f92ea864700 1 -- start start 2022-06-29T15:45:48.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.315+0000 7f92ea864700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f92e40b1470 0x7f92e40af590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:48.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.315+0000 7f92ea864700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92e40b04e0 0x7f92e40ac9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:48.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.315+0000 7f92ea864700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92e40b1850 con 0x7f92e40aceb0 2022-06-29T15:45:48.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.315+0000 7f92ea864700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92e40b27c0 con 0x7f92e40b1470 2022-06-29T15:45:48.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.315+0000 7f92ea864700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e40b6030 con 0x7f92e40b04e0 2022-06-29T15:45:48.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.315+0000 7f92e9862700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f92e40b1470 0x7f92e40af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:48.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.315+0000 7f92e9061700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92e40b04e0 0x7f92e40ac9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:48.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.315+0000 7f92e9061700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92e40b04e0 0x7f92e40ac9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:45252/0 (socket says 172.21.15.47:45252) 2022-06-29T15:45:48.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.315+0000 7f92e9061700 1 -- 172.21.15.47:0/414977865 learned_addr learned my addr 172.21.15.47:0/414977865 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:45:48.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.315+0000 7f92e9862700 1 -- 172.21.15.47:0/414977865 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92e40b04e0 msgr2=0x7f92e40ac9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:45:48.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.315+0000 7f92e9862700 1 --2- 172.21.15.47:0/414977865 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92e40b04e0 0x7f92e40ac9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:48.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.315+0000 7f92e9862700 1 -- 172.21.15.47:0/414977865 >> v1:172.21.15.64:6789/0 conn(0x7f92e40aceb0 legacy=0x7f92e40b5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:48.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.315+0000 7f92e9862700 1 -- 172.21.15.47:0/414977865 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e4142d50 con 0x7f92e40b1470 2022-06-29T15:45:48.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.315+0000 7f92e9862700 1 --2- 172.21.15.47:0/414977865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92e40b1470 0x7f92e40af590 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f92e40b2900 tx=0x7f92e0006d90).ready entity=mon.1 client_cookie=5c35f088c1af07c9 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:48.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.315+0000 7f92e8860700 1 -- 172.21.15.47:0/414977865 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92e000d4a0 con 0x7f92e40b1470 2022-06-29T15:45:48.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.315+0000 7f92e8860700 1 -- 172.21.15.47:0/414977865 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92e0000e90 con 0x7f92e40b1470 2022-06-29T15:45:48.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.315+0000 7f92e8860700 1 -- 172.21.15.47:0/414977865 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92e000d7f0 con 0x7f92e40b1470 2022-06-29T15:45:48.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.315+0000 7f92ea864700 1 -- 172.21.15.47:0/414977865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92e40b1470 msgr2=0x7f92e40af590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:48.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.315+0000 7f92ea864700 1 --2- 172.21.15.47:0/414977865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92e40b1470 0x7f92e40af590 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f92e40b2900 tx=0x7f92e0006d90).stop 2022-06-29T15:45:48.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.315+0000 7f92ea864700 1 -- 172.21.15.47:0/414977865 shutdown_connections 2022-06-29T15:45:48.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.315+0000 7f92ea864700 1 --2- 172.21.15.47:0/414977865 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92e40b04e0 0x7f92e40ac9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:48.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.315+0000 7f92ea864700 1 --2- 172.21.15.47:0/414977865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92e40b1470 0x7f92e40af590 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:48.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.315+0000 7f92ea864700 1 -- 172.21.15.47:0/414977865 >> 172.21.15.47:0/414977865 conn(0x7f92e4062560 msgr2=0x7f92e4062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:45:48.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.315+0000 7f92ea864700 1 -- 172.21.15.47:0/414977865 shutdown_connections 2022-06-29T15:45:48.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.315+0000 7f92ea864700 1 -- 172.21.15.47:0/414977865 wait complete. 2022-06-29T15:45:48.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.319+0000 7f92ea864700 1 Processor -- start 2022-06-29T15:45:48.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.319+0000 7f92ea864700 1 -- start start 2022-06-29T15:45:48.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.319+0000 7f92ea864700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92e40aceb0 0x7f92e40a3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:48.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.319+0000 7f92ea864700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92e40b04e0 0x7f92e4065a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:48.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.319+0000 7f92ea864700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92e40b1470 0x7f92e4065ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:48.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.319+0000 7f92ea864700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e40655d0 con 0x7f92e40b1470 2022-06-29T15:45:48.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.319+0000 7f92ea864700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e4152290 con 0x7f92e40b04e0 2022-06-29T15:45:48.335 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.319+0000 7f92ea864700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e4064710 con 0x7f92e40aceb0 2022-06-29T15:45:48.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.319+0000 7f92ea063700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92e40b1470 0x7f92e4065ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:48.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.319+0000 7f92ea063700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92e40b1470 0x7f92e4065ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48158/0 (socket says 172.21.15.47:48158) 2022-06-29T15:45:48.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.319+0000 7f92e9862700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92e40aceb0 0x7f92e40a3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:48.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.319+0000 7f92ea063700 1 -- 172.21.15.47:0/1736578742 learned_addr learned my addr 172.21.15.47:0/1736578742 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:45:48.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.319+0000 7f92e9061700 1 --2- 172.21.15.47:0/1736578742 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92e40b04e0 0x7f92e4065a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:48.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.319+0000 7f92ea063700 1 -- 172.21.15.47:0/1736578742 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92e40aceb0 msgr2=0x7f92e40a3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:48.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.319+0000 7f92ea063700 1 --2- 172.21.15.47:0/1736578742 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92e40aceb0 0x7f92e40a3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:48.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.319+0000 7f92ea063700 1 -- 172.21.15.47:0/1736578742 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92e40b04e0 msgr2=0x7f92e4065a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:48.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.319+0000 7f92ea063700 1 --2- 172.21.15.47:0/1736578742 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92e40b04e0 0x7f92e4065a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:48.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.319+0000 7f92ea063700 1 -- 172.21.15.47:0/1736578742 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e000b120 con 0x7f92e40b1470 2022-06-29T15:45:48.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.319+0000 7f92ea063700 1 --2- 172.21.15.47:0/1736578742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92e40b1470 0x7f92e4065ef0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7f92e40b6280 tx=0x7f92dc007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:48.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.319+0000 7f92d2ffd700 1 -- 172.21.15.47:0/1736578742 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92dc00e4a0 con 0x7f92e40b1470 2022-06-29T15:45:48.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.319+0000 7f92d2ffd700 1 -- 172.21.15.47:0/1736578742 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92dc00e600 con 0x7f92e40b1470 2022-06-29T15:45:48.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.319+0000 7f92d2ffd700 1 -- 172.21.15.47:0/1736578742 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92dc00e960 con 0x7f92e40b1470 2022-06-29T15:45:48.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.319+0000 7f92ea864700 1 -- 172.21.15.47:0/1736578742 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92e4065260 con 0x7f92e40b1470 2022-06-29T15:45:48.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.319+0000 7f92ea864700 1 -- 172.21.15.47:0/1736578742 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92e413c1a0 con 0x7f92e40b1470 2022-06-29T15:45:48.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.323+0000 7f92ea864700 1 -- 172.21.15.47:0/1736578742 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92e40653f0 con 0x7f92e40b1470 2022-06-29T15:45:48.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.323+0000 7f92d2ffd700 1 -- 172.21.15.47:0/1736578742 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f92dc010030 con 0x7f92e40b1470 2022-06-29T15:45:48.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.323+0000 7f92d2ffd700 1 --2- 172.21.15.47:0/1736578742 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f92c4053b00 0x7f92c4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:48.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.323+0000 7f92d2ffd700 1 -- 172.21.15.47:0/1736578742 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f92dc06c1f0 con 0x7f92e40b1470 2022-06-29T15:45:48.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.323+0000 7f92e9862700 1 --2- 172.21.15.47:0/1736578742 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f92c4053b00 0x7f92c4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:48.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.323+0000 7f92e9862700 1 --2- 172.21.15.47:0/1736578742 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f92c4053b00 0x7f92c4055fc0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f92e001a000 tx=0x7f92e0014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:48.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.331+0000 7f92d2ffd700 1 -- 172.21.15.47:0/1736578742 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f92dc042f10 con 0x7f92e40b1470 2022-06-29T15:45:48.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.451+0000 7f92ea864700 1 -- 172.21.15.47:0/1736578742 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f92e40573d0 con 0x7f92e40b1470 2022-06-29T15:45:48.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.451+0000 7f92d2ffd700 1 -- 172.21.15.47:0/1736578742 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v47) v1 ==== 81+0+34037 (secure 0 0 0) 0x7f92dc046530 con 0x7f92e40b1470 2022-06-29T15:45:48.457 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:45:48.458 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2022-06-29T15:45:48.458 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 47, 2022-06-29T15:45:48.458 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "9a68894a-53ce-400e-838b-14eab91b834c", 2022-06-29T15:45:48.459 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2022-06-29T15:42:14.046361+0000", 2022-06-29T15:45:48.459 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2022-06-29T15:44:28.619620+0000", 2022-06-29T15:45:48.459 INFO:teuthology.orchestra.run.smithi047.stdout: "last_up_change": "2022-06-29T15:42:20.929789+0000", 2022-06-29T15:45:48.460 INFO:teuthology.orchestra.run.smithi047.stdout: "last_in_change": "2022-06-29T15:42:20.929789+0000", 2022-06-29T15:45:48.460 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-06-29T15:45:48.460 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_num": 5799936, 2022-06-29T15:45:48.460 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_set": [ 2022-06-29T15:45:48.461 INFO:teuthology.orchestra.run.smithi047.stdout: "pglog_hardlimit", 2022-06-29T15:45:48.461 INFO:teuthology.orchestra.run.smithi047.stdout: "purged_snapdirs", 2022-06-29T15:45:48.461 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_deletes", 2022-06-29T15:45:48.461 INFO:teuthology.orchestra.run.smithi047.stdout: "sortbitwise" 2022-06-29T15:45:48.462 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:45:48.462 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_version": 5, 2022-06-29T15:45:48.462 INFO:teuthology.orchestra.run.smithi047.stdout: "full_ratio": 0.94999998807907104, 2022-06-29T15:45:48.463 INFO:teuthology.orchestra.run.smithi047.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-06-29T15:45:48.463 INFO:teuthology.orchestra.run.smithi047.stdout: "nearfull_ratio": 0.85000002384185791, 2022-06-29T15:45:48.463 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_snapshot": "", 2022-06-29T15:45:48.463 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_max": 8, 2022-06-29T15:45:48.464 INFO:teuthology.orchestra.run.smithi047.stdout: "max_osd": 8, 2022-06-29T15:45:48.464 INFO:teuthology.orchestra.run.smithi047.stdout: "require_min_compat_client": "jewel", 2022-06-29T15:45:48.464 INFO:teuthology.orchestra.run.smithi047.stdout: "min_compat_client": "jewel", 2022-06-29T15:45:48.465 INFO:teuthology.orchestra.run.smithi047.stdout: "require_osd_release": "octopus", 2022-06-29T15:45:48.465 INFO:teuthology.orchestra.run.smithi047.stdout: "pools": [ 2022-06-29T15:45:48.465 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.466 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 1, 2022-06-29T15:45:48.466 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "device_health_metrics", 2022-06-29T15:45:48.466 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:42:16.971431+0000", 2022-06-29T15:45:48.467 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:45:48.467 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:45:48.467 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:45:48.468 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:45:48.468 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:45:48.469 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:45:48.469 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:45:48.469 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:45:48.470 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 1, 2022-06-29T15:45:48.470 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 1, 2022-06-29T15:45:48.470 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 1, 2022-06-29T15:45:48.471 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 1, 2022-06-29T15:45:48.471 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 1, 2022-06-29T15:45:48.471 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:45:48.472 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:45:48.472 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:45:48.472 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:45:48.472 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:45:48.472 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:45:48.473 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:45:48.473 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.473 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "16", 2022-06-29T15:45:48.473 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:45:48.474 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:45:48.474 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:45:48.474 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:45:48.475 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:45:48.475 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:45:48.475 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:45:48.476 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:45:48.476 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:45:48.476 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:45:48.477 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:45:48.477 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:45:48.477 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:45:48.477 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:45:48.478 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:45:48.478 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:45:48.478 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:45:48.478 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:45:48.479 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:45:48.479 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:45:48.479 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:45:48.479 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:45:48.480 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:45:48.480 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:45:48.480 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:45:48.480 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:45:48.481 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.481 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:45:48.481 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:45:48.481 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:45:48.482 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:45:48.482 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:45:48.482 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:45:48.483 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:45:48.483 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:45:48.483 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:45:48.483 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:45:48.483 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:45:48.484 INFO:teuthology.orchestra.run.smithi047.stdout: "options": { 2022-06-29T15:45:48.484 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_min": 1 2022-06-29T15:45:48.484 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.485 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:45:48.485 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_devicehealth": {} 2022-06-29T15:45:48.485 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:48.485 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.486 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.486 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 2, 2022-06-29T15:45:48.486 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "rbd", 2022-06-29T15:45:48.487 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:42:25.290884+0000", 2022-06-29T15:45:48.487 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:45:48.487 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:45:48.488 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:45:48.488 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:45:48.488 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:45:48.488 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:45:48.489 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:45:48.489 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:45:48.489 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2022-06-29T15:45:48.489 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2022-06-29T15:45:48.490 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2022-06-29T15:45:48.490 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2022-06-29T15:45:48.490 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2022-06-29T15:45:48.491 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:45:48.491 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:45:48.491 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:45:48.491 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:45:48.492 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:45:48.492 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:45:48.492 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:45:48.492 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.492 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "19", 2022-06-29T15:45:48.493 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:45:48.493 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:45:48.493 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:45:48.493 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:45:48.494 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:45:48.494 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:45:48.495 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:45:48.495 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:45:48.495 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:45:48.496 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:45:48.496 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:45:48.496 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:45:48.496 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:45:48.497 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:45:48.497 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:45:48.497 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:45:48.497 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:45:48.498 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:45:48.498 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:45:48.499 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:45:48.499 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:45:48.499 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:45:48.500 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:45:48.500 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:45:48.500 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:45:48.500 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:45:48.501 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.501 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:45:48.501 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:45:48.501 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:45:48.502 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:45:48.502 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:45:48.502 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:45:48.502 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:45:48.503 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:45:48.503 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:45:48.503 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:45:48.503 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:45:48.504 INFO:teuthology.orchestra.run.smithi047.stdout: "options": {}, 2022-06-29T15:45:48.504 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:45:48.504 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd": {} 2022-06-29T15:45:48.504 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:48.505 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.505 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.505 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 7, 2022-06-29T15:45:48.505 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "cephfs_metadata", 2022-06-29T15:45:48.506 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:44:25.700617+0000", 2022-06-29T15:45:48.506 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:45:48.506 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:45:48.507 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:45:48.507 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:45:48.507 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:45:48.507 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:45:48.507 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:45:48.508 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:45:48.508 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2022-06-29T15:45:48.508 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2022-06-29T15:45:48.509 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2022-06-29T15:45:48.509 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2022-06-29T15:45:48.509 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2022-06-29T15:45:48.509 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:45:48.510 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:45:48.510 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:45:48.510 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:45:48.510 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:45:48.511 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:45:48.511 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:45:48.511 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.511 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "46", 2022-06-29T15:45:48.512 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:45:48.512 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:45:48.512 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:45:48.512 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:45:48.512 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:45:48.513 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:45:48.513 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:45:48.513 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:45:48.513 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:45:48.514 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:45:48.514 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:45:48.514 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:45:48.514 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:45:48.515 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:45:48.515 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:45:48.515 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:45:48.515 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:45:48.516 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:45:48.516 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:45:48.516 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:45:48.516 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:45:48.516 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:45:48.517 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:45:48.517 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:45:48.517 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:45:48.518 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:45:48.518 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.518 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:45:48.519 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:45:48.519 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:45:48.520 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:45:48.520 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:45:48.521 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:45:48.521 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:45:48.521 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:45:48.522 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:45:48.522 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:45:48.523 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:45:48.523 INFO:teuthology.orchestra.run.smithi047.stdout: "options": { 2022-06-29T15:45:48.523 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_bias": 4, 2022-06-29T15:45:48.524 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_min": 16, 2022-06-29T15:45:48.524 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_priority": 5 2022-06-29T15:45:48.524 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.525 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:45:48.525 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs": { 2022-06-29T15:45:48.525 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": "cephfs" 2022-06-29T15:45:48.526 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:48.526 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:48.527 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.527 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.527 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 8, 2022-06-29T15:45:48.528 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "cephfs_data", 2022-06-29T15:45:48.528 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:44:26.610945+0000", 2022-06-29T15:45:48.528 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:45:48.528 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:45:48.528 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:45:48.529 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:45:48.529 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:45:48.529 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:45:48.529 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:45:48.530 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:45:48.530 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2022-06-29T15:45:48.530 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2022-06-29T15:45:48.530 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2022-06-29T15:45:48.531 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2022-06-29T15:45:48.531 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2022-06-29T15:45:48.531 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:45:48.531 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:45:48.532 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:45:48.532 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:45:48.532 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:45:48.532 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:45:48.533 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:45:48.533 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.533 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "47", 2022-06-29T15:45:48.533 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:45:48.533 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:45:48.534 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:45:48.534 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:45:48.534 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:45:48.534 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:45:48.535 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:45:48.535 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:45:48.535 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:45:48.535 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:45:48.536 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:45:48.536 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:45:48.536 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:45:48.536 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:45:48.537 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:45:48.537 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:45:48.537 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:45:48.537 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:45:48.538 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:45:48.538 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:45:48.538 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:45:48.538 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:45:48.539 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:45:48.539 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:45:48.539 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:45:48.539 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:45:48.540 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.540 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:45:48.540 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:45:48.540 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:45:48.541 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:45:48.541 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:45:48.541 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:45:48.541 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:45:48.542 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:45:48.542 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:45:48.542 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:45:48.542 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:45:48.543 INFO:teuthology.orchestra.run.smithi047.stdout: "options": {}, 2022-06-29T15:45:48.543 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:45:48.543 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs": { 2022-06-29T15:45:48.543 INFO:teuthology.orchestra.run.smithi047.stdout: "data": "cephfs" 2022-06-29T15:45:48.544 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:48.544 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:48.544 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:48.544 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:45:48.545 INFO:teuthology.orchestra.run.smithi047.stdout: "osds": [ 2022-06-29T15:45:48.545 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.545 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2022-06-29T15:45:48.545 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "e23eb023-4706-4201-bf72-4bf19c912c5d", 2022-06-29T15:45:48.545 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:45:48.546 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:45:48.546 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:45:48.546 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:45:48.546 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:45:48.547 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:45:48.547 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:45:48.547 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 45, 2022-06-29T15:45:48.547 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:45:48.548 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:45:48.548 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:45:48.548 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:48.548 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.549 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:48.549 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6818", 2022-06-29T15:45:48.550 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:45:48.550 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.550 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.551 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:48.551 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6819", 2022-06-29T15:45:48.552 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:45:48.552 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:48.553 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:48.553 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.554 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:45:48.554 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:48.555 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.555 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:48.556 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6820", 2022-06-29T15:45:48.556 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:45:48.556 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.557 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.557 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:48.558 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6821", 2022-06-29T15:45:48.558 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:45:48.558 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:48.559 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:48.559 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.560 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:45:48.560 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:48.561 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.561 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:48.561 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6824", 2022-06-29T15:45:48.562 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:45:48.562 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.562 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.562 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:48.563 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6825", 2022-06-29T15:45:48.563 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:45:48.563 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:48.563 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:48.564 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.564 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:45:48.564 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:48.564 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.565 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:48.565 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6822", 2022-06-29T15:45:48.565 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:45:48.565 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.566 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.566 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:48.566 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6823", 2022-06-29T15:45:48.567 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:45:48.567 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:48.567 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:48.567 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.567 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6819/13096", 2022-06-29T15:45:48.568 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6821/13096", 2022-06-29T15:45:48.568 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6825/13096", 2022-06-29T15:45:48.568 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6823/13096", 2022-06-29T15:45:48.568 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:45:48.569 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:45:48.569 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:45:48.569 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:48.569 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.570 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.570 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2022-06-29T15:45:48.570 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "228bb08d-2d64-43d9-9560-bfaeff154452", 2022-06-29T15:45:48.570 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:45:48.571 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:45:48.571 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:45:48.571 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:45:48.571 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:45:48.571 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:45:48.572 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:45:48.572 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 45, 2022-06-29T15:45:48.572 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:45:48.572 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:45:48.573 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:45:48.573 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:48.573 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.573 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:48.574 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6826", 2022-06-29T15:45:48.574 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:45:48.574 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.575 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.575 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:48.575 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6827", 2022-06-29T15:45:48.575 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:45:48.576 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:48.576 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:48.576 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.576 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:45:48.577 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:48.577 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.577 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:48.577 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6828", 2022-06-29T15:45:48.578 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:45:48.578 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.578 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.578 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:48.579 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6829", 2022-06-29T15:45:48.579 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:45:48.579 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:48.579 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:48.579 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.580 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:45:48.580 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:48.580 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.580 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:48.581 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6832", 2022-06-29T15:45:48.581 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:45:48.581 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.581 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.582 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:48.582 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6833", 2022-06-29T15:45:48.582 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:45:48.582 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:48.583 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:48.583 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.583 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:45:48.583 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:48.583 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.584 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:48.584 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6830", 2022-06-29T15:45:48.584 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:45:48.584 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.585 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.585 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:48.585 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6831", 2022-06-29T15:45:48.585 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:45:48.586 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:48.586 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:48.586 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.586 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6827/13100", 2022-06-29T15:45:48.586 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6829/13100", 2022-06-29T15:45:48.587 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6833/13100", 2022-06-29T15:45:48.587 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6831/13100", 2022-06-29T15:45:48.587 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:45:48.587 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:45:48.588 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:45:48.588 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:48.588 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.588 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.589 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2022-06-29T15:45:48.589 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "797a5452-1510-4640-a111-c9c2fcf707eb", 2022-06-29T15:45:48.589 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:45:48.589 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:45:48.589 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:45:48.590 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:45:48.590 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:45:48.590 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:45:48.590 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:45:48.591 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 33, 2022-06-29T15:45:48.591 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:45:48.591 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:45:48.591 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:45:48.592 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:48.592 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.592 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:48.592 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6810", 2022-06-29T15:45:48.592 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:45:48.593 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.593 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.593 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:48.593 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6811", 2022-06-29T15:45:48.594 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:45:48.594 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:48.594 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:48.594 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.594 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:45:48.595 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:48.595 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.595 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:48.595 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6812", 2022-06-29T15:45:48.596 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:45:48.596 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.596 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.596 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:48.597 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6813", 2022-06-29T15:45:48.597 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:45:48.597 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:48.597 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:48.597 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.598 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:45:48.598 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:48.598 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.598 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:48.599 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6816", 2022-06-29T15:45:48.599 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:45:48.599 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.599 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.600 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:48.600 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6817", 2022-06-29T15:45:48.600 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:45:48.600 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:48.600 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:48.601 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.601 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:45:48.601 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:48.601 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.602 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:48.602 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6814", 2022-06-29T15:45:48.602 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:45:48.602 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.602 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.603 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:48.603 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6815", 2022-06-29T15:45:48.603 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:45:48.603 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:48.604 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:48.604 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.604 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6811/13098", 2022-06-29T15:45:48.604 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6813/13098", 2022-06-29T15:45:48.604 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6817/13098", 2022-06-29T15:45:48.605 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6815/13098", 2022-06-29T15:45:48.605 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:45:48.605 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:45:48.605 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:45:48.606 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:48.606 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.606 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.606 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2022-06-29T15:45:48.607 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "55b24fd7-bb71-4a01-b3fa-a09caa772a62", 2022-06-29T15:45:48.607 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:45:48.607 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:45:48.607 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:45:48.608 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:45:48.608 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:45:48.608 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:45:48.608 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:45:48.609 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 45, 2022-06-29T15:45:48.609 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:45:48.609 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:45:48.609 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:45:48.610 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:48.610 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.610 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:48.610 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6802", 2022-06-29T15:45:48.611 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:45:48.611 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.611 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.611 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:48.611 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6803", 2022-06-29T15:45:48.612 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:45:48.612 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:48.612 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:48.612 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.613 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:45:48.613 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:48.613 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.613 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:48.613 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6804", 2022-06-29T15:45:48.614 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:45:48.614 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.614 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.614 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:48.615 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6805", 2022-06-29T15:45:48.615 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:45:48.615 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:48.615 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:48.616 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.616 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:45:48.616 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:48.616 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.616 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:48.617 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6808", 2022-06-29T15:45:48.617 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:45:48.617 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.618 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.618 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:48.618 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6809", 2022-06-29T15:45:48.618 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:45:48.618 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:48.619 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:48.619 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.619 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:45:48.619 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:48.620 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.620 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:48.620 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6806", 2022-06-29T15:45:48.620 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:45:48.621 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.621 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.621 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:48.621 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6807", 2022-06-29T15:45:48.621 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:45:48.622 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:48.622 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:48.622 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.622 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6803/13097", 2022-06-29T15:45:48.623 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6805/13097", 2022-06-29T15:45:48.623 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6809/13097", 2022-06-29T15:45:48.623 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6807/13097", 2022-06-29T15:45:48.623 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:45:48.624 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:45:48.624 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:45:48.624 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:48.624 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.625 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.625 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2022-06-29T15:45:48.625 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3", 2022-06-29T15:45:48.625 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:45:48.625 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:45:48.626 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:45:48.626 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:45:48.626 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:45:48.626 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:45:48.627 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:45:48.627 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 45, 2022-06-29T15:45:48.627 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:45:48.627 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:45:48.628 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:45:48.628 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:48.628 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.628 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:48.628 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6816", 2022-06-29T15:45:48.629 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:45:48.629 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.629 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.629 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:48.630 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6817", 2022-06-29T15:45:48.630 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:45:48.630 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:48.630 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:48.631 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.631 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:45:48.631 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:48.631 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.632 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:48.632 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6818", 2022-06-29T15:45:48.632 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:45:48.632 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.632 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.633 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:48.633 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6819", 2022-06-29T15:45:48.633 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:45:48.633 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:48.634 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:48.634 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.634 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:45:48.634 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:48.635 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.635 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:48.635 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6822", 2022-06-29T15:45:48.635 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:45:48.635 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.636 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.636 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:48.636 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6823", 2022-06-29T15:45:48.636 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:45:48.637 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:48.637 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:48.637 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.637 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:45:48.637 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:48.638 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.638 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:48.638 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6820", 2022-06-29T15:45:48.638 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:45:48.639 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.639 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.639 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:48.639 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6821", 2022-06-29T15:45:48.639 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:45:48.640 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:48.640 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:48.640 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.640 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6817/13396", 2022-06-29T15:45:48.641 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6819/13396", 2022-06-29T15:45:48.641 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6823/13396", 2022-06-29T15:45:48.641 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6821/13396", 2022-06-29T15:45:48.641 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:45:48.642 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:45:48.642 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:45:48.642 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:48.642 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.642 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.643 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2022-06-29T15:45:48.643 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "f77beebe-6cca-4244-b2d4-677825ebf919", 2022-06-29T15:45:48.643 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:45:48.643 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:45:48.644 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:45:48.644 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:45:48.644 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:45:48.644 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:45:48.645 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:45:48.645 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 45, 2022-06-29T15:45:48.645 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:45:48.645 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:45:48.645 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:45:48.646 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:48.646 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.646 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:48.646 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6800", 2022-06-29T15:45:48.647 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:45:48.647 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.647 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.647 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:48.647 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6801", 2022-06-29T15:45:48.648 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:45:48.648 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:48.648 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:48.648 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.649 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:45:48.649 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:48.649 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.649 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:48.650 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6802", 2022-06-29T15:45:48.650 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:45:48.650 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.650 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.651 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:48.651 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6803", 2022-06-29T15:45:48.651 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:45:48.651 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:48.651 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:48.652 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.652 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:45:48.652 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:48.652 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.653 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:48.653 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6806", 2022-06-29T15:45:48.653 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:45:48.653 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.653 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.654 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:48.654 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6807", 2022-06-29T15:45:48.654 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:45:48.654 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:48.655 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:48.655 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.655 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:45:48.655 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:48.656 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.656 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:48.656 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6804", 2022-06-29T15:45:48.656 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:45:48.656 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.657 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.657 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:48.657 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6805", 2022-06-29T15:45:48.657 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:45:48.658 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:48.658 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:48.658 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.658 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6801/13393", 2022-06-29T15:45:48.659 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6803/13393", 2022-06-29T15:45:48.659 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6807/13393", 2022-06-29T15:45:48.659 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6805/13393", 2022-06-29T15:45:48.659 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:45:48.659 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:45:48.660 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:45:48.660 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:48.660 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.660 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.661 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2022-06-29T15:45:48.661 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "af0ca9b3-c02c-4910-834b-bf0b8dc6713c", 2022-06-29T15:45:48.661 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:45:48.661 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:45:48.661 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:45:48.662 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:45:48.662 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:45:48.662 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:45:48.662 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:45:48.663 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 45, 2022-06-29T15:45:48.663 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:45:48.663 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:45:48.663 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:45:48.664 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:48.664 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.664 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:48.664 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6824", 2022-06-29T15:45:48.665 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:45:48.665 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.665 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.665 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:48.666 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6825", 2022-06-29T15:45:48.666 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:45:48.666 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:48.666 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:48.667 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.667 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:45:48.667 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:48.667 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.668 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:48.668 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6826", 2022-06-29T15:45:48.668 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:45:48.669 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.669 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.669 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:48.669 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6827", 2022-06-29T15:45:48.670 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:45:48.670 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:48.670 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:48.671 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.671 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:45:48.671 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:48.671 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.672 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:48.672 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6830", 2022-06-29T15:45:48.672 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:45:48.672 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.673 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.673 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:48.673 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6831", 2022-06-29T15:45:48.673 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:45:48.674 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:48.674 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:48.674 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.675 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:45:48.675 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:48.675 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.675 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:48.676 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6828", 2022-06-29T15:45:48.676 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:45:48.676 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.676 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.677 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:48.677 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6829", 2022-06-29T15:45:48.677 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:45:48.677 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:48.678 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:48.678 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.678 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6825/13395", 2022-06-29T15:45:48.679 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6827/13395", 2022-06-29T15:45:48.679 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6831/13395", 2022-06-29T15:45:48.679 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6829/13395", 2022-06-29T15:45:48.679 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:45:48.680 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:45:48.680 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:45:48.680 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:48.680 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.681 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.681 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2022-06-29T15:45:48.681 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "f19c3ca6-df88-4bde-a150-b51a53f092e1", 2022-06-29T15:45:48.681 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:45:48.682 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:45:48.682 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:45:48.682 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:45:48.682 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:45:48.683 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:45:48.683 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:45:48.683 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 44, 2022-06-29T15:45:48.683 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:45:48.684 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:45:48.684 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:45:48.684 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:48.684 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.685 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:48.685 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6808", 2022-06-29T15:45:48.685 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:45:48.685 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.686 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.686 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:48.686 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6809", 2022-06-29T15:45:48.686 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:45:48.687 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:48.687 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:48.687 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.687 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:45:48.688 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:48.688 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.688 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:48.688 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6810", 2022-06-29T15:45:48.688 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:45:48.689 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.689 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.689 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:48.690 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6811", 2022-06-29T15:45:48.690 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:45:48.690 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:48.690 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:48.691 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.691 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:45:48.691 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:48.691 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.691 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:48.692 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6814", 2022-06-29T15:45:48.692 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:45:48.692 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.692 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.693 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:48.693 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6815", 2022-06-29T15:45:48.693 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:45:48.693 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:48.694 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:48.694 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.694 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:45:48.694 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:48.695 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.695 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:48.695 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6812", 2022-06-29T15:45:48.695 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:45:48.695 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.696 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.696 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:48.696 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6813", 2022-06-29T15:45:48.696 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:45:48.697 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:48.697 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:48.697 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.697 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6809/13394", 2022-06-29T15:45:48.698 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6811/13394", 2022-06-29T15:45:48.698 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6815/13394", 2022-06-29T15:45:48.698 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6813/13394", 2022-06-29T15:45:48.698 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:45:48.699 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:45:48.699 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:45:48.699 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:48.699 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:48.699 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:45:48.700 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_xinfo": [ 2022-06-29T15:45:48.700 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.700 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2022-06-29T15:45:48.700 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:45:48.701 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:45:48.701 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:45:48.701 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:45:48.702 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:45:48.702 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.720513+0000", 2022-06-29T15:45:48.702 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:45:48.702 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.703 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.703 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2022-06-29T15:45:48.703 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:45:48.703 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:45:48.704 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:45:48.704 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:45:48.704 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:45:48.704 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.772612+0000", 2022-06-29T15:45:48.705 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:45:48.705 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.705 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.705 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2022-06-29T15:45:48.706 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:45:48.706 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:45:48.706 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:45:48.707 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:45:48.707 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:45:48.707 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.450223+0000", 2022-06-29T15:45:48.708 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:45:48.708 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.709 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.709 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2022-06-29T15:45:48.709 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:45:48.709 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:45:48.710 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:45:48.710 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:45:48.710 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:45:48.710 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.477856+0000", 2022-06-29T15:45:48.711 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:45:48.711 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.711 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.712 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2022-06-29T15:45:48.712 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:45:48.712 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:45:48.712 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:45:48.712 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:45:48.713 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:45:48.713 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.539485+0000", 2022-06-29T15:45:48.713 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:45:48.713 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.714 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.714 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2022-06-29T15:45:48.714 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:45:48.714 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:45:48.715 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:45:48.715 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:45:48.715 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:45:48.716 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.487040+0000", 2022-06-29T15:45:48.716 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:45:48.716 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.716 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.717 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2022-06-29T15:45:48.717 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:45:48.717 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:45:48.717 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:45:48.718 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:45:48.718 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:45:48.718 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.748685+0000", 2022-06-29T15:45:48.718 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:45:48.719 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.719 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:48.719 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2022-06-29T15:45:48.720 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:45:48.720 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:45:48.720 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:45:48.720 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:45:48.721 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:45:48.721 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.459516+0000", 2022-06-29T15:45:48.721 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:45:48.721 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:48.722 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:45:48.722 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap": [], 2022-06-29T15:45:48.722 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap_items": [], 2022-06-29T15:45:48.722 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_temp": [], 2022-06-29T15:45:48.723 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_temp": [], 2022-06-29T15:45:48.723 INFO:teuthology.orchestra.run.smithi047.stdout: "blacklist": {}, 2022-06-29T15:45:48.723 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profiles": { 2022-06-29T15:45:48.723 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs_data": { 2022-06-29T15:45:48.724 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-device-class": "", 2022-06-29T15:45:48.724 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2022-06-29T15:45:48.724 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-root": "default", 2022-06-29T15:45:48.725 INFO:teuthology.orchestra.run.smithi047.stdout: "jerasure-per-chunk-alignment": "false", 2022-06-29T15:45:48.725 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2022-06-29T15:45:48.725 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "2", 2022-06-29T15:45:48.725 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2022-06-29T15:45:48.726 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2022-06-29T15:45:48.726 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van", 2022-06-29T15:45:48.726 INFO:teuthology.orchestra.run.smithi047.stdout: "w": "8" 2022-06-29T15:45:48.726 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.727 INFO:teuthology.orchestra.run.smithi047.stdout: "default": { 2022-06-29T15:45:48.727 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2022-06-29T15:45:48.727 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2022-06-29T15:45:48.728 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "1", 2022-06-29T15:45:48.728 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2022-06-29T15:45:48.728 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2022-06-29T15:45:48.728 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van" 2022-06-29T15:45:48.729 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:48.729 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:48.729 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps_queue": [], 2022-06-29T15:45:48.729 INFO:teuthology.orchestra.run.smithi047.stdout: "new_removed_snaps": [], 2022-06-29T15:45:48.730 INFO:teuthology.orchestra.run.smithi047.stdout: "new_purged_snaps": [], 2022-06-29T15:45:48.730 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_node_flags": {}, 2022-06-29T15:45:48.730 INFO:teuthology.orchestra.run.smithi047.stdout: "device_class_flags": {} 2022-06-29T15:45:48.730 INFO:teuthology.orchestra.run.smithi047.stdout:} 2022-06-29T15:45:48.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.455+0000 7f92ea864700 1 -- 172.21.15.47:0/1736578742 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f92c4053b00 msgr2=0x7f92c4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:48.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.455+0000 7f92ea864700 1 --2- 172.21.15.47:0/1736578742 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f92c4053b00 0x7f92c4055fc0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f92e001a000 tx=0x7f92e0014000).stop 2022-06-29T15:45:48.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.455+0000 7f92ea864700 1 -- 172.21.15.47:0/1736578742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92e40b1470 msgr2=0x7f92e4065ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:48.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.455+0000 7f92ea864700 1 --2- 172.21.15.47:0/1736578742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92e40b1470 0x7f92e4065ef0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7f92e40b6280 tx=0x7f92dc007f40).stop 2022-06-29T15:45:48.734 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.455+0000 7f92ea864700 1 -- 172.21.15.47:0/1736578742 shutdown_connections 2022-06-29T15:45:48.734 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.455+0000 7f92ea864700 1 --2- 172.21.15.47:0/1736578742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92e40b1470 0x7f92e4065ef0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:48.734 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.455+0000 7f92ea864700 1 --2- 172.21.15.47:0/1736578742 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f92c4053b00 0x7f92c4055fc0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:48.734 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.455+0000 7f92ea864700 1 --2- 172.21.15.47:0/1736578742 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92e40b04e0 0x7f92e4065a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:48.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.455+0000 7f92ea864700 1 --2- 172.21.15.47:0/1736578742 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92e40aceb0 0x7f92e40a3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:48.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.455+0000 7f92ea864700 1 -- 172.21.15.47:0/1736578742 >> 172.21.15.47:0/1736578742 conn(0x7f92e4062560 msgr2=0x7f92e4060a00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:45:48.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.455+0000 7f92ea864700 1 -- 172.21.15.47:0/1736578742 shutdown_connections 2022-06-29T15:45:48.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.455+0000 7f92ea864700 1 -- 172.21.15.47:0/1736578742 wait complete. 2022-06-29T15:45:48.736 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:45:48.850 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.847+0000 7f36f5e60700 1 Processor -- start 2022-06-29T15:45:48.850 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.847+0000 7f36f5e60700 1 -- start start 2022-06-29T15:45:48.851 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.847+0000 7f36f5e60700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f36f00b2120 0x7f36f00b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:48.852 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.847+0000 7f36f5e60700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f36f005f0b0 0x7f36f014b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:48.852 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.847+0000 7f36f5e60700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36f00b0740 con 0x7f36f00b1260 2022-06-29T15:45:48.852 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.847+0000 7f36f5e60700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36f0147460 con 0x7f36f005f0b0 2022-06-29T15:45:48.852 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.847+0000 7f36f5e60700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36f00b0970 con 0x7f36f00b2120 2022-06-29T15:45:48.853 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.847+0000 7f36e6ffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f36f005f0b0 0x7f36f014b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:48.853 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.847+0000 7f36ef7fe700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f36f00b2120 0x7f36f00b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:48.853 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.847+0000 7f36ef7fe700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f36f00b2120 0x7f36f00b2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:45266/0 (socket says 172.21.15.47:45266) 2022-06-29T15:45:48.853 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.847+0000 7f36ef7fe700 1 -- 172.21.15.47:0/2143797691 learned_addr learned my addr 172.21.15.47:0/2143797691 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:45:48.854 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.847+0000 7f36ef7fe700 1 -- 172.21.15.47:0/2143797691 >> v2:172.21.15.64:3300/0 conn(0x7f36f005f0b0 msgr2=0x7f36f014b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:45:48.854 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.847+0000 7f36ef7fe700 1 --2- 172.21.15.47:0/2143797691 >> v2:172.21.15.64:3300/0 conn(0x7f36f005f0b0 0x7f36f014b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:48.854 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.847+0000 7f36ef7fe700 1 -- 172.21.15.47:0/2143797691 >> v1:172.21.15.64:6789/0 conn(0x7f36f00b1260 legacy=0x7f36f0147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:48.855 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.847+0000 7f36ef7fe700 1 -- 172.21.15.47:0/2143797691 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36f00b0c10 con 0x7f36f00b2120 2022-06-29T15:45:48.855 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.847+0000 7f36e6ffd700 1 --2- 172.21.15.47:0/2143797691 >> v2:172.21.15.64:3300/0 conn(0x7f36f005f0b0 0x7f36f014b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:45:48.855 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.847+0000 7f36ef7fe700 1 --2- 172.21.15.47:0/2143797691 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f36f00b2120 0x7f36f00b2500 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f36e0002f70 tx=0x7f36e0006eb0).ready entity=mon.2 client_cookie=90158ca92d378330 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:48.859 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.847+0000 7f36eeffd700 1 -- 172.21.15.47:0/2143797691 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f36e000d570 con 0x7f36f00b2120 2022-06-29T15:45:48.860 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.847+0000 7f36eeffd700 1 -- 172.21.15.47:0/2143797691 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36e0000e90 con 0x7f36f00b2120 2022-06-29T15:45:48.860 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.847+0000 7f36eeffd700 1 -- 172.21.15.47:0/2143797691 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f36e000d900 con 0x7f36f00b2120 2022-06-29T15:45:48.860 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.847+0000 7f36f5e60700 1 -- 172.21.15.47:0/2143797691 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f36f00b2120 msgr2=0x7f36f00b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:48.860 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.847+0000 7f36f5e60700 1 --2- 172.21.15.47:0/2143797691 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f36f00b2120 0x7f36f00b2500 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f36e0002f70 tx=0x7f36e0006eb0).stop 2022-06-29T15:45:48.861 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.847+0000 7f36f5e60700 1 -- 172.21.15.47:0/2143797691 shutdown_connections 2022-06-29T15:45:48.861 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.847+0000 7f36f5e60700 1 --2- 172.21.15.47:0/2143797691 >> v2:172.21.15.64:3300/0 conn(0x7f36f005f0b0 0x7f36f014b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:48.861 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.847+0000 7f36f5e60700 1 --2- 172.21.15.47:0/2143797691 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f36f00b2120 0x7f36f00b2500 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:48.861 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.847+0000 7f36f5e60700 1 -- 172.21.15.47:0/2143797691 >> 172.21.15.47:0/2143797691 conn(0x7f36f00626f0 msgr2=0x7f36f0062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:45:48.862 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.847+0000 7f36f5e60700 1 -- 172.21.15.47:0/2143797691 shutdown_connections 2022-06-29T15:45:48.862 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.847+0000 7f36f5e60700 1 -- 172.21.15.47:0/2143797691 wait complete. 2022-06-29T15:45:48.863 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.851+0000 7f36f5e60700 1 Processor -- start 2022-06-29T15:45:48.863 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.851+0000 7f36f5e60700 1 -- start start 2022-06-29T15:45:48.863 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.851+0000 7f36f5e60700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f36f00b1260 0x7f36f0149fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:48.863 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.851+0000 7f36f5e60700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36f00b2120 0x7f36f0141c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:48.864 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.851+0000 7f36f5e60700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36f01431c0 0x7f36f0142130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:48.864 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.851+0000 7f36f5e60700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36f00ac990 con 0x7f36f01431c0 2022-06-29T15:45:48.864 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.851+0000 7f36f5e60700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36f00acc30 con 0x7f36f00b2120 2022-06-29T15:45:48.864 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.851+0000 7f36f5e60700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36f00aced0 con 0x7f36f00b1260 2022-06-29T15:45:48.865 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.851+0000 7f36effff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36f01431c0 0x7f36f0142130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:48.865 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.851+0000 7f36e6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36f00b2120 0x7f36f0141c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:48.865 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.851+0000 7f36e6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36f00b2120 0x7f36f0141c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:53874/0 (socket says 172.21.15.47:53874) 2022-06-29T15:45:48.866 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.851+0000 7f36effff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36f01431c0 0x7f36f0142130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48172/0 (socket says 172.21.15.47:48172) 2022-06-29T15:45:48.866 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.851+0000 7f36e6ffd700 1 -- 172.21.15.47:0/1063844830 learned_addr learned my addr 172.21.15.47:0/1063844830 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:45:48.866 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.851+0000 7f36ef7fe700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f36f00b1260 0x7f36f0149fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:48.866 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.851+0000 7f36effff700 1 -- 172.21.15.47:0/1063844830 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f36f00b1260 msgr2=0x7f36f0149fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:48.867 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.851+0000 7f36effff700 1 --2- 172.21.15.47:0/1063844830 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f36f00b1260 0x7f36f0149fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:48.867 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.851+0000 7f36effff700 1 -- 172.21.15.47:0/1063844830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36f00b2120 msgr2=0x7f36f0141c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:48.867 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.851+0000 7f36effff700 1 --2- 172.21.15.47:0/1063844830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36f00b2120 0x7f36f0141c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:48.867 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.851+0000 7f36effff700 1 -- 172.21.15.47:0/1063844830 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d800b180 con 0x7f36f01431c0 2022-06-29T15:45:48.868 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.851+0000 7f36e6ffd700 1 --2- 172.21.15.47:0/1063844830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36f00b2120 0x7f36f0141c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:45:48.868 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.851+0000 7f36effff700 1 --2- 172.21.15.47:0/1063844830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36f01431c0 0x7f36f0142130 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7f36f006af60 tx=0x7f36e8007eb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:48.868 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.851+0000 7f36ed7fa700 1 -- 172.21.15.47:0/1063844830 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f36e800f470 con 0x7f36f01431c0 2022-06-29T15:45:48.868 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.851+0000 7f36ed7fa700 1 -- 172.21.15.47:0/1063844830 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36e800f5d0 con 0x7f36f01431c0 2022-06-29T15:45:48.869 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.851+0000 7f36f5e60700 1 -- 172.21.15.47:0/1063844830 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36e000b1e0 con 0x7f36f01431c0 2022-06-29T15:45:48.869 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.851+0000 7f36ed7fa700 1 -- 172.21.15.47:0/1063844830 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f36e800f930 con 0x7f36f01431c0 2022-06-29T15:45:48.869 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.851+0000 7f36f5e60700 1 -- 172.21.15.47:0/1063844830 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36f0142620 con 0x7f36f01431c0 2022-06-29T15:45:48.869 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.855+0000 7f36f5e60700 1 -- 172.21.15.47:0/1063844830 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36f00573d0 con 0x7f36f01431c0 2022-06-29T15:45:48.870 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.855+0000 7f36ed7fa700 1 -- 172.21.15.47:0/1063844830 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f36e8010070 con 0x7f36f01431c0 2022-06-29T15:45:48.871 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.855+0000 7f36ed7fa700 1 --2- 172.21.15.47:0/1063844830 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f36d4053a10 0x7f36d4055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:48.871 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.855+0000 7f36ef7fe700 1 --2- 172.21.15.47:0/1063844830 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f36d4053a10 0x7f36d4055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:48.872 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.855+0000 7f36ed7fa700 1 -- 172.21.15.47:0/1063844830 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f36e806b5e0 con 0x7f36f01431c0 2022-06-29T15:45:48.872 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.855+0000 7f36ef7fe700 1 --2- 172.21.15.47:0/1063844830 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f36d4053a10 0x7f36d4055ed0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f36e001c000 tx=0x7f36e0015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:48.872 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:48.863+0000 7f36ed7fa700 1 -- 172.21.15.47:0/1063844830 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f36e8042290 con 0x7f36f01431c0 2022-06-29T15:45:49.031 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.027+0000 7f36f5e60700 1 -- 172.21.15.47:0/1063844830 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f36f0062cb0 con 0x7f36f01431c0 2022-06-29T15:45:49.032 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.027+0000 7f36ed7fa700 1 -- 172.21.15.47:0/1063844830 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+2491 (secure 0 0 0) 0x7f36e80458b0 con 0x7f36f01431c0 2022-06-29T15:45:49.032 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:45:49.032 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":17,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5259,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3601815346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3601815346},{"type":"v1","addr":"172.21.15.47:6835","nonce":3601815346}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15},{"gid":5262,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/657177602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":657177602},{"type":"v1","addr":"172.21.15.47:6837","nonce":657177602}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:44:27.616735+0000","modified":"2022-06-29T15:44:28.620601+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":54812},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54812":{"gid":54812,"name":"b","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6833/2497658100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2497658100},{"type":"v1","addr":"172.21.15.64:6833","nonce":2497658100}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2022-06-29T15:45:49.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.031+0000 7f36f5e60700 1 -- 172.21.15.47:0/1063844830 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f36d4053a10 msgr2=0x7f36d4055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:49.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.031+0000 7f36f5e60700 1 --2- 172.21.15.47:0/1063844830 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f36d4053a10 0x7f36d4055ed0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f36e001c000 tx=0x7f36e0015000).stop 2022-06-29T15:45:49.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.031+0000 7f36f5e60700 1 -- 172.21.15.47:0/1063844830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36f01431c0 msgr2=0x7f36f0142130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:49.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.031+0000 7f36f5e60700 1 --2- 172.21.15.47:0/1063844830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36f01431c0 0x7f36f0142130 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7f36f006af60 tx=0x7f36e8007eb0).stop 2022-06-29T15:45:49.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.031+0000 7f36f5e60700 1 -- 172.21.15.47:0/1063844830 shutdown_connections 2022-06-29T15:45:49.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.031+0000 7f36f5e60700 1 --2- 172.21.15.47:0/1063844830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36f01431c0 0x7f36f0142130 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:49.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.031+0000 7f36f5e60700 1 --2- 172.21.15.47:0/1063844830 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f36d4053a10 0x7f36d4055ed0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:49.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.031+0000 7f36f5e60700 1 --2- 172.21.15.47:0/1063844830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36f00b2120 0x7f36f0141c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:49.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.031+0000 7f36f5e60700 1 --2- 172.21.15.47:0/1063844830 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f36f00b1260 0x7f36f0149fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:49.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.031+0000 7f36f5e60700 1 -- 172.21.15.47:0/1063844830 >> 172.21.15.47:0/1063844830 conn(0x7f36f00626f0 msgr2=0x7f36f00b39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:45:49.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.031+0000 7f36f5e60700 1 -- 172.21.15.47:0/1063844830 shutdown_connections 2022-06-29T15:45:49.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.031+0000 7f36f5e60700 1 -- 172.21.15.47:0/1063844830 wait complete. 2022-06-29T15:45:49.038 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 17 2022-06-29T15:45:49.046 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-06-29T15:45:49.152 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.147+0000 7fdcde8af700 1 Processor -- start 2022-06-29T15:45:49.152 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.147+0000 7fdcde8af700 1 -- start start 2022-06-29T15:45:49.155 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.151+0000 7fdcde8af700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdcd81408c0 0x7fdcd8064f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:49.155 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.151+0000 7fdcde8af700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcd80b4800 con 0x7fdcd80b4420 2022-06-29T15:45:49.156 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.151+0000 7fdcde8af700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcd80b2930 con 0x7fdcd80b2490 2022-06-29T15:45:49.156 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.151+0000 7fdcde8af700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcd80b2bd0 con 0x7fdcd81408c0 2022-06-29T15:45:49.156 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.151+0000 7fdcd7fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fdcd80b4420 0x7fdcd81401b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33730/0 (socket says 172.21.15.47:33730) 2022-06-29T15:45:49.156 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.151+0000 7fdcd7fff700 1 -- 172.21.15.47:0/533760368 learned_addr learned my addr 172.21.15.47:0/533760368 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:45:49.157 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.151+0000 7fdcdce4c700 1 --2- 172.21.15.47:0/533760368 >> v2:172.21.15.64:3300/0 conn(0x7fdcd81408c0 0x7fdcd8064f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:49.157 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.151+0000 7fdcd6ffd700 1 -- 172.21.15.47:0/533760368 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3710933743 0 0) 0x7fdcd80b4800 con 0x7fdcd80b4420 2022-06-29T15:45:49.157 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.151+0000 7fdcd6ffd700 1 -- 172.21.15.47:0/533760368 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcb8003260 con 0x7fdcd80b4420 2022-06-29T15:45:49.157 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.151+0000 7fdcd6ffd700 1 -- 172.21.15.47:0/533760368 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3144564720 0 0) 0x7fdcd80b2930 con 0x7fdcd80b2490 2022-06-29T15:45:49.157 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.151+0000 7fdcd6ffd700 1 -- 172.21.15.47:0/533760368 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcd80b4800 con 0x7fdcd80b2490 2022-06-29T15:45:49.158 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.151+0000 7fdcdce4c700 1 -- 172.21.15.47:0/533760368 >> v1:172.21.15.64:6789/0 conn(0x7fdcd80b2490 legacy=0x7fdcd8064820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:49.158 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.151+0000 7fdcdce4c700 1 -- 172.21.15.47:0/533760368 >> v1:172.21.15.47:6789/0 conn(0x7fdcd80b4420 legacy=0x7fdcd81401b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:49.158 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.151+0000 7fdcdce4c700 1 -- 172.21.15.47:0/533760368 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd80b3930 con 0x7fdcd81408c0 2022-06-29T15:45:49.159 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.151+0000 7fdcdce4c700 1 --2- 172.21.15.47:0/533760368 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcd81408c0 0x7fdcd8064f30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7fdcd00032f0 tx=0x7fdcd00091d0).ready entity=mon.1 client_cookie=e0e78d83005ca9b3 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:49.159 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.151+0000 7fdcd6ffd700 1 -- 172.21.15.47:0/533760368 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdcd000b600 con 0x7fdcd81408c0 2022-06-29T15:45:49.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.151+0000 7fdcd6ffd700 1 -- 172.21.15.47:0/533760368 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcd000b760 con 0x7fdcd81408c0 2022-06-29T15:45:49.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.151+0000 7fdcd6ffd700 1 -- 172.21.15.47:0/533760368 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdcd000b8d0 con 0x7fdcd81408c0 2022-06-29T15:45:49.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.151+0000 7fdcde8af700 1 -- 172.21.15.47:0/533760368 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcd81408c0 msgr2=0x7fdcd8064f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:49.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.151+0000 7fdcde8af700 1 --2- 172.21.15.47:0/533760368 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcd81408c0 0x7fdcd8064f30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7fdcd00032f0 tx=0x7fdcd00091d0).stop 2022-06-29T15:45:49.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.151+0000 7fdcde8af700 1 -- 172.21.15.47:0/533760368 shutdown_connections 2022-06-29T15:45:49.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.151+0000 7fdcde8af700 1 --2- 172.21.15.47:0/533760368 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcd81408c0 0x7fdcd8064f30 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:49.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.151+0000 7fdcde8af700 1 -- 172.21.15.47:0/533760368 >> 172.21.15.47:0/533760368 conn(0x7fdcd805c510 msgr2=0x7fdcd805c920 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:45:49.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.151+0000 7fdcde8af700 1 -- 172.21.15.47:0/533760368 shutdown_connections 2022-06-29T15:45:49.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.151+0000 7fdcde8af700 1 -- 172.21.15.47:0/533760368 wait complete. 2022-06-29T15:45:49.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.155+0000 7fdcde8af700 1 Processor -- start 2022-06-29T15:45:49.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.155+0000 7fdcde8af700 1 -- start start 2022-06-29T15:45:49.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.155+0000 7fdcde8af700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdcd80b2490 0x7fdcd813bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:49.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.155+0000 7fdcde8af700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcd80b4420 0x7fdcd8152100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:49.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.155+0000 7fdcde8af700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcd81408c0 0x7fdcd8155e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:49.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.155+0000 7fdcde8af700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd8155770 con 0x7fdcd80b4420 2022-06-29T15:45:49.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.155+0000 7fdcde8af700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd813f4e0 con 0x7fdcd81408c0 2022-06-29T15:45:49.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.155+0000 7fdcde8af700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd81559a0 con 0x7fdcd80b2490 2022-06-29T15:45:49.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.159+0000 7fdcd77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcd80b4420 0x7fdcd8152100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:49.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.159+0000 7fdcd77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcd80b4420 0x7fdcd8152100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48184/0 (socket says 172.21.15.47:48184) 2022-06-29T15:45:49.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.159+0000 7fdcd77fe700 1 -- 172.21.15.47:0/556423740 learned_addr learned my addr 172.21.15.47:0/556423740 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:45:49.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.159+0000 7fdcd7fff700 1 --2- 172.21.15.47:0/556423740 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdcd80b2490 0x7fdcd813bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:49.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.159+0000 7fdcdce4c700 1 --2- 172.21.15.47:0/556423740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcd81408c0 0x7fdcd8155e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:49.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.159+0000 7fdcd77fe700 1 -- 172.21.15.47:0/556423740 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdcd80b2490 msgr2=0x7fdcd813bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:49.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.159+0000 7fdcd77fe700 1 --2- 172.21.15.47:0/556423740 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdcd80b2490 0x7fdcd813bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:49.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.159+0000 7fdcd77fe700 1 -- 172.21.15.47:0/556423740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcd81408c0 msgr2=0x7fdcd8155e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:49.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.159+0000 7fdcd77fe700 1 --2- 172.21.15.47:0/556423740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcd81408c0 0x7fdcd8155e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:49.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.159+0000 7fdcd77fe700 1 -- 172.21.15.47:0/556423740 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd0009470 con 0x7fdcd80b4420 2022-06-29T15:45:49.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.159+0000 7fdcdce4c700 1 --2- 172.21.15.47:0/556423740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcd81408c0 0x7fdcd8155e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:45:49.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.159+0000 7fdcd77fe700 1 --2- 172.21.15.47:0/556423740 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcd80b4420 0x7fdcd8152100 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7fdcc0001110 tx=0x7fdcc0007ae0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:49.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.159+0000 7fdcd57fa700 1 -- 172.21.15.47:0/556423740 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdcc000ce50 con 0x7fdcd80b4420 2022-06-29T15:45:49.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.159+0000 7fdcd57fa700 1 -- 172.21.15.47:0/556423740 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcc000f470 con 0x7fdcd80b4420 2022-06-29T15:45:49.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.159+0000 7fdcde8af700 1 -- 172.21.15.47:0/556423740 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcd80afc10 con 0x7fdcd80b4420 2022-06-29T15:45:49.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.159+0000 7fdcd57fa700 1 -- 172.21.15.47:0/556423740 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdcc000f7d0 con 0x7fdcd80b4420 2022-06-29T15:45:49.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.159+0000 7fdcde8af700 1 -- 172.21.15.47:0/556423740 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcd813e010 con 0x7fdcd80b4420 2022-06-29T15:45:49.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.159+0000 7fdcde8af700 1 -- 172.21.15.47:0/556423740 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcd813f170 con 0x7fdcd80b4420 2022-06-29T15:45:49.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.159+0000 7fdcd57fa700 1 -- 172.21.15.47:0/556423740 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fdcc0019030 con 0x7fdcd80b4420 2022-06-29T15:45:49.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.163+0000 7fdcd57fa700 1 --2- 172.21.15.47:0/556423740 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdcbc053aa0 0x7fdcbc055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:49.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.163+0000 7fdcd57fa700 1 -- 172.21.15.47:0/556423740 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fdcc006b510 con 0x7fdcd80b4420 2022-06-29T15:45:49.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.163+0000 7fdcd7fff700 1 --2- 172.21.15.47:0/556423740 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdcbc053aa0 0x7fdcbc055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:49.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.163+0000 7fdcd7fff700 1 --2- 172.21.15.47:0/556423740 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdcbc053aa0 0x7fdcbc055f60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7fdcc800d000 tx=0x7fdcc8009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:49.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.167+0000 7fdcd57fa700 1 -- 172.21.15.47:0/556423740 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdcc0042230 con 0x7fdcd80b4420 2022-06-29T15:45:49.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:49.335+0000 7fdcde8af700 1 -- 172.21.15.47:0/556423740 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fdcd80573d0 con 0x7fdcd80b4420 2022-06-29T15:45:50.173 INFO:tasks.ceph.mon.a.smithi047.stderr:2022-06-29T15:45:50.171+0000 7ff9cf83c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-06-29T15:45:50.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.179+0000 7fdcd57fa700 1 -- 172.21.15.47:0/556423740 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) v1 ==== 161+0+0 (secure 0 0 0) 0x7fdcc0045850 con 0x7fdcd80b4420 2022-06-29T15:45:50.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.179+0000 7fdcde8af700 1 -- 172.21.15.47:0/556423740 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdcbc053aa0 msgr2=0x7fdcbc055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:50.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.179+0000 7fdcde8af700 1 --2- 172.21.15.47:0/556423740 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdcbc053aa0 0x7fdcbc055f60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7fdcc800d000 tx=0x7fdcc8009040).stop 2022-06-29T15:45:50.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.179+0000 7fdcde8af700 1 -- 172.21.15.47:0/556423740 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcd80b4420 msgr2=0x7fdcd8152100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:50.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.179+0000 7fdcde8af700 1 --2- 172.21.15.47:0/556423740 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcd80b4420 0x7fdcd8152100 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7fdcc0001110 tx=0x7fdcc0007ae0).stop 2022-06-29T15:45:50.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.179+0000 7fdcde8af700 1 -- 172.21.15.47:0/556423740 shutdown_connections 2022-06-29T15:45:50.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.179+0000 7fdcde8af700 1 --2- 172.21.15.47:0/556423740 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdcbc053aa0 0x7fdcbc055f60 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:50.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.183+0000 7fdcde8af700 1 --2- 172.21.15.47:0/556423740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcd81408c0 0x7fdcd8155e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:50.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.183+0000 7fdcde8af700 1 --2- 172.21.15.47:0/556423740 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcd80b4420 0x7fdcd8152100 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:50.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.183+0000 7fdcde8af700 1 --2- 172.21.15.47:0/556423740 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdcd80b2490 0x7fdcd813bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:50.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.183+0000 7fdcde8af700 1 -- 172.21.15.47:0/556423740 >> 172.21.15.47:0/556423740 conn(0x7fdcd805c510 msgr2=0x7fdcd813c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:45:50.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.183+0000 7fdcde8af700 1 -- 172.21.15.47:0/556423740 shutdown_connections 2022-06-29T15:45:50.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.183+0000 7fdcde8af700 1 -- 172.21.15.47:0/556423740 wait complete. 2022-06-29T15:45:50.190 INFO:teuthology.orchestra.run.smithi047.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-06-29T15:45:50.201 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:45:50.223 INFO:tasks.ceph.mds.b.smithi064.stdout:starting mds.b at 2022-06-29T15:45:50.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.307+0000 7f6c1f527700 1 Processor -- start 2022-06-29T15:45:50.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.307+0000 7f6c1f527700 1 -- start start 2022-06-29T15:45:50.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.307+0000 7f6c1f527700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6c18143bd0 0x7f6c1805c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:50.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.307+0000 7f6c1f527700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6c180b2c40 0x7f6c180b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:50.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.307+0000 7f6c1f527700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c18142f90 con 0x7f6c1805c510 2022-06-29T15:45:50.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.307+0000 7f6c1f527700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c181431f0 con 0x7f6c18143bd0 2022-06-29T15:45:50.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.307+0000 7f6c1f527700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c18143490 con 0x7f6c180b2c40 2022-06-29T15:45:50.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.307+0000 7f6c1dd24700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6c18143bd0 0x7f6c1805c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:50.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.307+0000 7f6c1dd24700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6c18143bd0 0x7f6c1805c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48192/0 (socket says 172.21.15.47:48192) 2022-06-29T15:45:50.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.307+0000 7f6c1dd24700 1 -- 172.21.15.47:0/2325600281 learned_addr learned my addr 172.21.15.47:0/2325600281 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:45:50.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.307+0000 7f6c1ed26700 1 --2- 172.21.15.47:0/2325600281 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6c180b2c40 0x7f6c180b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:50.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.307+0000 7f6c1d523700 1 -- 172.21.15.47:0/2325600281 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1952396436 0 0) 0x7f6c18142f90 con 0x7f6c1805c510 2022-06-29T15:45:50.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.307+0000 7f6c1d523700 1 -- 172.21.15.47:0/2325600281 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c00003260 con 0x7f6c1805c510 2022-06-29T15:45:50.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.311+0000 7f6c1dd24700 1 -- 172.21.15.47:0/2325600281 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6c180b2c40 msgr2=0x7f6c180b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:45:50.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.311+0000 7f6c1dd24700 1 --2- 172.21.15.47:0/2325600281 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6c180b2c40 0x7f6c180b30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:50.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.311+0000 7f6c1dd24700 1 -- 172.21.15.47:0/2325600281 >> v1:172.21.15.64:6789/0 conn(0x7f6c1805c510 legacy=0x7f6c180b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:50.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.311+0000 7f6c1dd24700 1 -- 172.21.15.47:0/2325600281 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c18143730 con 0x7f6c18143bd0 2022-06-29T15:45:50.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.311+0000 7f6c1dd24700 1 --2- 172.21.15.47:0/2325600281 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c18143bd0 0x7f6c1805c8f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7f6c14002f70 tx=0x7f6c14006eb0).ready entity=mon.0 client_cookie=e83b062d553c905a server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:50.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.311+0000 7f6c1d523700 1 -- 172.21.15.47:0/2325600281 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c1400d5c0 con 0x7f6c18143bd0 2022-06-29T15:45:50.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.311+0000 7f6c1d523700 1 -- 172.21.15.47:0/2325600281 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c14000e90 con 0x7f6c18143bd0 2022-06-29T15:45:50.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.311+0000 7f6c1d523700 1 -- 172.21.15.47:0/2325600281 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c1400d950 con 0x7f6c18143bd0 2022-06-29T15:45:50.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.311+0000 7f6c1f527700 1 -- 172.21.15.47:0/2325600281 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c18143bd0 msgr2=0x7f6c1805c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:50.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.311+0000 7f6c1f527700 1 --2- 172.21.15.47:0/2325600281 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c18143bd0 0x7f6c1805c8f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7f6c14002f70 tx=0x7f6c14006eb0).stop 2022-06-29T15:45:50.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.311+0000 7f6c1f527700 1 -- 172.21.15.47:0/2325600281 shutdown_connections 2022-06-29T15:45:50.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.311+0000 7f6c1f527700 1 --2- 172.21.15.47:0/2325600281 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6c180b2c40 0x7f6c180b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:50.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.311+0000 7f6c1f527700 1 --2- 172.21.15.47:0/2325600281 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c18143bd0 0x7f6c1805c8f0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:50.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.311+0000 7f6c1f527700 1 -- 172.21.15.47:0/2325600281 >> 172.21.15.47:0/2325600281 conn(0x7f6c18060e00 msgr2=0x7f6c180611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:45:50.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.311+0000 7f6c1f527700 1 -- 172.21.15.47:0/2325600281 shutdown_connections 2022-06-29T15:45:50.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.311+0000 7f6c1f527700 1 -- 172.21.15.47:0/2325600281 wait complete. 2022-06-29T15:45:50.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.311+0000 7f6c1f527700 1 Processor -- start 2022-06-29T15:45:50.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.311+0000 7f6c1f527700 1 -- start start 2022-06-29T15:45:50.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.311+0000 7f6c1f527700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6c1805c510 0x7f6c1813d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:50.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.311+0000 7f6c1f527700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c180b2c40 0x7f6c180b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:50.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.311+0000 7f6c1f527700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c18143bd0 0x7f6c180b51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:50.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.311+0000 7f6c1f527700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1813dd10 con 0x7f6c180b2c40 2022-06-29T15:45:50.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.311+0000 7f6c1f527700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c18144f60 con 0x7f6c18143bd0 2022-06-29T15:45:50.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.311+0000 7f6c1f527700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c180b6640 con 0x7f6c1805c510 2022-06-29T15:45:50.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.315+0000 7f6c1dd24700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c180b2c40 0x7f6c180b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:50.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.315+0000 7f6c1dd24700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c180b2c40 0x7f6c180b4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48198/0 (socket says 172.21.15.47:48198) 2022-06-29T15:45:50.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.315+0000 7f6c1dd24700 1 -- 172.21.15.47:0/2909945243 learned_addr learned my addr 172.21.15.47:0/2909945243 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:45:50.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.315+0000 7f6c1ed26700 1 --2- 172.21.15.47:0/2909945243 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c18143bd0 0x7f6c180b51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:50.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.315+0000 7f6c1e525700 1 --2- 172.21.15.47:0/2909945243 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6c1805c510 0x7f6c1813d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:50.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.315+0000 7f6c1dd24700 1 -- 172.21.15.47:0/2909945243 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6c1805c510 msgr2=0x7f6c1813d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:50.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.315+0000 7f6c1dd24700 1 --2- 172.21.15.47:0/2909945243 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6c1805c510 0x7f6c1813d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:50.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.315+0000 7f6c1dd24700 1 -- 172.21.15.47:0/2909945243 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c18143bd0 msgr2=0x7f6c180b51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:50.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.315+0000 7f6c1dd24700 1 --2- 172.21.15.47:0/2909945243 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c18143bd0 0x7f6c180b51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:50.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.315+0000 7f6c1dd24700 1 -- 172.21.15.47:0/2909945243 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c1400b1e0 con 0x7f6c180b2c40 2022-06-29T15:45:50.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.315+0000 7f6c1ed26700 1 --2- 172.21.15.47:0/2909945243 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c18143bd0 0x7f6c180b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:45:50.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.315+0000 7f6c1dd24700 1 --2- 172.21.15.47:0/2909945243 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c180b2c40 0x7f6c180b4d00 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7f6c14002f70 tx=0x7f6c14015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:50.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.315+0000 7f6c077fe700 1 -- 172.21.15.47:0/2909945243 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c14011860 con 0x7f6c180b2c40 2022-06-29T15:45:50.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.315+0000 7f6c077fe700 1 -- 172.21.15.47:0/2909945243 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c140119c0 con 0x7f6c180b2c40 2022-06-29T15:45:50.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.315+0000 7f6c1f527700 1 -- 172.21.15.47:0/2909945243 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c18144be0 con 0x7f6c180b2c40 2022-06-29T15:45:50.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.315+0000 7f6c077fe700 1 -- 172.21.15.47:0/2909945243 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c14011d20 con 0x7f6c180b2c40 2022-06-29T15:45:50.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.315+0000 7f6c1f527700 1 -- 172.21.15.47:0/2909945243 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c18147280 con 0x7f6c180b2c40 2022-06-29T15:45:50.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.315+0000 7f6c1f527700 1 -- 172.21.15.47:0/2909945243 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c180b6840 con 0x7f6c180b2c40 2022-06-29T15:45:50.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.315+0000 7f6c077fe700 1 -- 172.21.15.47:0/2909945243 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6c14025030 con 0x7f6c180b2c40 2022-06-29T15:45:50.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.315+0000 7f6c077fe700 1 --2- 172.21.15.47:0/2909945243 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6bf8053a10 0x7f6bf8055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:50.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.315+0000 7f6c1e525700 1 --2- 172.21.15.47:0/2909945243 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6bf8053a10 0x7f6bf8055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:50.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.315+0000 7f6c077fe700 1 -- 172.21.15.47:0/2909945243 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f6c1406d2d0 con 0x7f6c180b2c40 2022-06-29T15:45:50.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.319+0000 7f6c1e525700 1 --2- 172.21.15.47:0/2909945243 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6bf8053a10 0x7f6bf8055ed0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f6c0800b000 tx=0x7f6c08009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:50.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.323+0000 7f6c077fe700 1 -- 172.21.15.47:0/2909945243 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6c14044040 con 0x7f6c180b2c40 2022-06-29T15:45:50.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.487+0000 7f6c1f527700 1 -- 172.21.15.47:0/2909945243 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6c181450a0 con 0x7f6c180b2c40 2022-06-29T15:45:50.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.487+0000 7f6c077fe700 1 -- 172.21.15.47:0/2909945243 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+2127 (secure 0 0 0) 0x7f6c14023020 con 0x7f6c180b2c40 2022-06-29T15:45:50.491 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:45:50.491 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":18,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5259,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3601815346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3601815346},{"type":"v1","addr":"172.21.15.47:6835","nonce":3601815346}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15},{"gid":5262,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/657177602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":657177602},{"type":"v1","addr":"172.21.15.47:6837","nonce":657177602}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15}],"filesystems":[{"mdsmap":{"epoch":18,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:44:27.616735+0000","modified":"2022-06-29T15:45:50.172295+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2022-06-29T15:45:50.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.487+0000 7f6c1f527700 1 -- 172.21.15.47:0/2909945243 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6bf8053a10 msgr2=0x7f6bf8055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:50.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.487+0000 7f6c1f527700 1 --2- 172.21.15.47:0/2909945243 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6bf8053a10 0x7f6bf8055ed0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f6c0800b000 tx=0x7f6c08009040).stop 2022-06-29T15:45:50.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.487+0000 7f6c1f527700 1 -- 172.21.15.47:0/2909945243 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c180b2c40 msgr2=0x7f6c180b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:50.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.487+0000 7f6c1f527700 1 --2- 172.21.15.47:0/2909945243 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c180b2c40 0x7f6c180b4d00 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7f6c14002f70 tx=0x7f6c14015000).stop 2022-06-29T15:45:50.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.487+0000 7f6c1f527700 1 -- 172.21.15.47:0/2909945243 shutdown_connections 2022-06-29T15:45:50.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.487+0000 7f6c1f527700 1 --2- 172.21.15.47:0/2909945243 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6bf8053a10 0x7f6bf8055ed0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:50.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.487+0000 7f6c1f527700 1 --2- 172.21.15.47:0/2909945243 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c18143bd0 0x7f6c180b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:50.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.487+0000 7f6c1f527700 1 --2- 172.21.15.47:0/2909945243 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c180b2c40 0x7f6c180b4d00 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:50.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.487+0000 7f6c1f527700 1 --2- 172.21.15.47:0/2909945243 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6c1805c510 0x7f6c1813d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:50.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.487+0000 7f6c1f527700 1 -- 172.21.15.47:0/2909945243 >> 172.21.15.47:0/2909945243 conn(0x7f6c18060e00 msgr2=0x7f6c18142710 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:45:50.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.487+0000 7f6c1f527700 1 -- 172.21.15.47:0/2909945243 shutdown_connections 2022-06-29T15:45:50.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.487+0000 7f6c1f527700 1 -- 172.21.15.47:0/2909945243 wait complete. 2022-06-29T15:45:50.496 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 18 2022-06-29T15:45:50.504 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-06-29T15:45:50.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.607+0000 7f372ab65700 1 Processor -- start 2022-06-29T15:45:50.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.607+0000 7f372ab65700 1 -- start start 2022-06-29T15:45:50.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.607+0000 7f372ab65700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f37240b0fe0 0x7f3724061a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:50.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.607+0000 7f372ab65700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f372413db90 con 0x7f3724061670 2022-06-29T15:45:50.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.607+0000 7f372ab65700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37240af0a0 con 0x7f37240b0700 2022-06-29T15:45:50.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.607+0000 7f372ab65700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f372413fed0 con 0x7f37240b0fe0 2022-06-29T15:45:50.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.607+0000 7f3728901700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f3724061670 0x7f372413d480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33758/0 (socket says 172.21.15.47:33758) 2022-06-29T15:45:50.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.607+0000 7f3728901700 1 -- 172.21.15.47:0/1266691523 learned_addr learned my addr 172.21.15.47:0/1266691523 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:45:50.622 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.607+0000 7f3723fff700 1 --2- 172.21.15.47:0/1266691523 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f37240b0fe0 0x7f3724061a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:50.622 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.607+0000 7f37237fe700 1 -- 172.21.15.47:0/1266691523 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1587740252 0 0) 0x7f372413db90 con 0x7f3724061670 2022-06-29T15:45:50.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.607+0000 7f37237fe700 1 -- 172.21.15.47:0/1266691523 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3710003260 con 0x7f3724061670 2022-06-29T15:45:50.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.611+0000 7f37237fe700 1 -- 172.21.15.47:0/1266691523 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4166810462 0 0) 0x7f37240af0a0 con 0x7f37240b0700 2022-06-29T15:45:50.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.611+0000 7f37237fe700 1 -- 172.21.15.47:0/1266691523 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f372413db90 con 0x7f37240b0700 2022-06-29T15:45:50.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.611+0000 7f37237fe700 1 -- 172.21.15.47:0/1266691523 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 295410312 0 0) 0x7f3710003260 con 0x7f3724061670 2022-06-29T15:45:50.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.611+0000 7f37237fe700 1 -- 172.21.15.47:0/1266691523 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f37240b0fe0 msgr2=0x7f3724061a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:45:50.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.611+0000 7f37237fe700 1 --2- 172.21.15.47:0/1266691523 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f37240b0fe0 0x7f3724061a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:50.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.611+0000 7f37237fe700 1 -- 172.21.15.47:0/1266691523 >> v1:172.21.15.64:6789/0 conn(0x7f37240b0700 legacy=0x7f37241436c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:50.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.611+0000 7f37237fe700 1 -- 172.21.15.47:0/1266691523 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37240af340 con 0x7f3724061670 2022-06-29T15:45:50.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.611+0000 7f37237fe700 1 -- 172.21.15.47:0/1266691523 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7f3714001a00 con 0x7f3724061670 2022-06-29T15:45:50.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.611+0000 7f37237fe700 1 -- 172.21.15.47:0/1266691523 >> v1:172.21.15.47:6789/0 conn(0x7f3724061670 legacy=0x7f372413d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:50.626 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.611+0000 7f37237fe700 1 --2- 172.21.15.47:0/1266691523 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f37240b0fe0 0x7f3710003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:50.626 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.611+0000 7f37237fe700 1 --2- 172.21.15.47:0/1266691523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3710003e70 0x7f3710004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:50.626 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.611+0000 7f37237fe700 1 --2- 172.21.15.47:0/1266691523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37100049f0 0x7f3710004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:50.626 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.611+0000 7f37237fe700 1 -- 172.21.15.47:0/1266691523 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f371000b800 con 0x7f37100049f0 2022-06-29T15:45:50.627 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.611+0000 7f37237fe700 1 -- 172.21.15.47:0/1266691523 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f371000b970 con 0x7f3710003e70 2022-06-29T15:45:50.627 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.611+0000 7f37237fe700 1 -- 172.21.15.47:0/1266691523 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f371000bb30 con 0x7f37240b0fe0 2022-06-29T15:45:50.627 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.611+0000 7f3729102700 1 --2- 172.21.15.47:0/1266691523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37100049f0 0x7f3710004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:50.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.611+0000 7f3728901700 1 --2- 172.21.15.47:0/1266691523 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f37240b0fe0 0x7f3710003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:50.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.611+0000 7f3723fff700 1 --2- 172.21.15.47:0/1266691523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3710003e70 0x7f3710004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:50.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.611+0000 7f3729102700 1 -- 172.21.15.47:0/1266691523 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f37240b0fe0 msgr2=0x7f3710003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:45:50.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.611+0000 7f3729102700 1 --2- 172.21.15.47:0/1266691523 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f37240b0fe0 0x7f3710003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:50.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.611+0000 7f3729102700 1 -- 172.21.15.47:0/1266691523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3710003e70 msgr2=0x7f3710004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:45:50.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.611+0000 7f3729102700 1 --2- 172.21.15.47:0/1266691523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3710003e70 0x7f3710004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:50.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.611+0000 7f3729102700 1 -- 172.21.15.47:0/1266691523 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3710003260 con 0x7f37100049f0 2022-06-29T15:45:50.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.611+0000 7f3729102700 1 --2- 172.21.15.47:0/1266691523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37100049f0 0x7f3710004e70 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7f371c001110 tx=0x7f371c007a00).ready entity=mon.0 client_cookie=8643c494fd316756 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:50.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.611+0000 7f37237fe700 1 -- 172.21.15.47:0/1266691523 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f371c007030 con 0x7f37100049f0 2022-06-29T15:45:50.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.611+0000 7f37237fe700 1 -- 172.21.15.47:0/1266691523 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f371c007190 con 0x7f37100049f0 2022-06-29T15:45:50.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.611+0000 7f37237fe700 1 -- 172.21.15.47:0/1266691523 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f371c006510 con 0x7f37100049f0 2022-06-29T15:45:50.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.611+0000 7f372ab65700 1 -- 172.21.15.47:0/1266691523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37100049f0 msgr2=0x7f3710004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:50.631 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.611+0000 7f372ab65700 1 --2- 172.21.15.47:0/1266691523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37100049f0 0x7f3710004e70 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7f371c001110 tx=0x7f371c007a00).stop 2022-06-29T15:45:50.631 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.611+0000 7f3729102700 1 -- 172.21.15.47:0/1266691523 reap_dead start 2022-06-29T15:45:50.631 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.611+0000 7f372ab65700 1 -- 172.21.15.47:0/1266691523 shutdown_connections 2022-06-29T15:45:50.631 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.611+0000 7f372ab65700 1 -- 172.21.15.47:0/1266691523 >> 172.21.15.47:0/1266691523 conn(0x7f3724063800 msgr2=0x7f3724063be0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:45:50.632 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.611+0000 7f372ab65700 1 -- 172.21.15.47:0/1266691523 shutdown_connections 2022-06-29T15:45:50.632 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.611+0000 7f372ab65700 1 -- 172.21.15.47:0/1266691523 wait complete. 2022-06-29T15:45:50.632 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.611+0000 7f372ab65700 1 Processor -- start 2022-06-29T15:45:50.632 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.611+0000 7f372ab65700 1 -- start start 2022-06-29T15:45:50.633 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.615+0000 7f372ab65700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37100049f0 0x7f37241424c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:50.633 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.615+0000 7f372ab65700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3724152280 0x7f3724155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:50.633 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.615+0000 7f372ab65700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3724146970 0x7f3724156360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:50.633 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.615+0000 7f372ab65700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3724066240 con 0x7f3724146970 2022-06-29T15:45:50.634 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.615+0000 7f372ab65700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37240ad540 con 0x7f37100049f0 2022-06-29T15:45:50.634 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.615+0000 7f372ab65700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3724067000 con 0x7f3724152280 2022-06-29T15:45:50.634 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.615+0000 7f3729102700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3724146970 0x7f3724156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:50.634 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.615+0000 7f3729102700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3724146970 0x7f3724156360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48220/0 (socket says 172.21.15.47:48220) 2022-06-29T15:45:50.635 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.615+0000 7f3723fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3724152280 0x7f3724155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:50.635 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.615+0000 7f3728901700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37100049f0 0x7f37241424c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:50.635 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.615+0000 7f3729102700 1 -- 172.21.15.47:0/3393251228 learned_addr learned my addr 172.21.15.47:0/3393251228 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:45:50.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.615+0000 7f3729102700 1 -- 172.21.15.47:0/3393251228 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3724152280 msgr2=0x7f3724155e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:50.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.615+0000 7f3729102700 1 --2- 172.21.15.47:0/3393251228 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3724152280 0x7f3724155e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:50.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.615+0000 7f3729102700 1 -- 172.21.15.47:0/3393251228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37100049f0 msgr2=0x7f37241424c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:50.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.615+0000 7f3729102700 1 --2- 172.21.15.47:0/3393251228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37100049f0 0x7f37241424c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:50.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.615+0000 7f3729102700 1 -- 172.21.15.47:0/3393251228 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37240672a0 con 0x7f3724146970 2022-06-29T15:45:50.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.615+0000 7f3729102700 1 --2- 172.21.15.47:0/3393251228 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3724146970 0x7f3724156360 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7f3724063260 tx=0x7f371c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:50.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.615+0000 7f3721ffb700 1 -- 172.21.15.47:0/3393251228 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f371c007d10 con 0x7f3724146970 2022-06-29T15:45:50.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.615+0000 7f3721ffb700 1 -- 172.21.15.47:0/3393251228 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f371c01e470 con 0x7f3724146970 2022-06-29T15:45:50.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.615+0000 7f372ab65700 1 -- 172.21.15.47:0/3393251228 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37241559d0 con 0x7f3724146970 2022-06-29T15:45:50.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.615+0000 7f3721ffb700 1 -- 172.21.15.47:0/3393251228 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f371c01e7d0 con 0x7f3724146970 2022-06-29T15:45:50.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.615+0000 7f372ab65700 1 -- 172.21.15.47:0/3393251228 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3724146570 con 0x7f3724146970 2022-06-29T15:45:50.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.615+0000 7f372ab65700 1 -- 172.21.15.47:0/3393251228 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37241557e0 con 0x7f3724146970 2022-06-29T15:45:50.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.619+0000 7f3721ffb700 1 -- 172.21.15.47:0/3393251228 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f371c01b040 con 0x7f3724146970 2022-06-29T15:45:50.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.619+0000 7f3721ffb700 1 --2- 172.21.15.47:0/3393251228 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3704053b00 0x7f3704055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:50.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.619+0000 7f3721ffb700 1 -- 172.21.15.47:0/3393251228 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f371c06c570 con 0x7f3724146970 2022-06-29T15:45:50.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.619+0000 7f3728901700 1 --2- 172.21.15.47:0/3393251228 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3704053b00 0x7f3704055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:50.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.619+0000 7f3728901700 1 --2- 172.21.15.47:0/3393251228 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3704053b00 0x7f3704055fc0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7f3714005000 tx=0x7f37240ae7e0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:50.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.623+0000 7f3721ffb700 1 -- 172.21.15.47:0/3393251228 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f371c0431d0 con 0x7f3724146970 2022-06-29T15:45:50.797 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:50.791+0000 7f372ab65700 1 -- 172.21.15.47:0/3393251228 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f3724155b60 con 0x7f3724146970 2022-06-29T15:45:51.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.187+0000 7f3721ffb700 1 -- 172.21.15.47:0/3393251228 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f371c0467f0 con 0x7f3724146970 2022-06-29T15:45:51.194 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.191+0000 7f372ab65700 1 -- 172.21.15.47:0/3393251228 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3704053b00 msgr2=0x7f3704055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:51.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.191+0000 7f372ab65700 1 --2- 172.21.15.47:0/3393251228 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3704053b00 0x7f3704055fc0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7f3714005000 tx=0x7f37240ae7e0).stop 2022-06-29T15:45:51.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.191+0000 7f372ab65700 1 -- 172.21.15.47:0/3393251228 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3724146970 msgr2=0x7f3724156360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:51.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.191+0000 7f372ab65700 1 --2- 172.21.15.47:0/3393251228 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3724146970 0x7f3724156360 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7f3724063260 tx=0x7f371c016000).stop 2022-06-29T15:45:51.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.191+0000 7f372ab65700 1 -- 172.21.15.47:0/3393251228 shutdown_connections 2022-06-29T15:45:51.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.191+0000 7f372ab65700 1 --2- 172.21.15.47:0/3393251228 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3724146970 0x7f3724156360 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:51.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.191+0000 7f372ab65700 1 --2- 172.21.15.47:0/3393251228 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3724152280 0x7f3724155e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:51.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.191+0000 7f372ab65700 1 --2- 172.21.15.47:0/3393251228 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3704053b00 0x7f3704055fc0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:51.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.191+0000 7f372ab65700 1 --2- 172.21.15.47:0/3393251228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37100049f0 0x7f37241424c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:51.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.191+0000 7f372ab65700 1 -- 172.21.15.47:0/3393251228 >> 172.21.15.47:0/3393251228 conn(0x7f3724063800 msgr2=0x7f37240b6fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:45:51.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.191+0000 7f372ab65700 1 -- 172.21.15.47:0/3393251228 shutdown_connections 2022-06-29T15:45:51.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.191+0000 7f372ab65700 1 -- 172.21.15.47:0/3393251228 wait complete. 2022-06-29T15:45:51.210 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-06-29T15:45:51.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.319+0000 7f4ef27c6700 1 Processor -- start 2022-06-29T15:45:51.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.319+0000 7f4ef27c6700 1 -- start start 2022-06-29T15:45:51.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.323+0000 7f4ef27c6700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4eec061670 0x7f4eec061a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:51.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.323+0000 7f4ef27c6700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4eec0b0fe0 0x7f4eec0b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:51.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.323+0000 7f4ef27c6700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4eec0af0a0 con 0x7f4eec13d480 2022-06-29T15:45:51.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.323+0000 7f4ef27c6700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4eec13fed0 con 0x7f4eec0b0fe0 2022-06-29T15:45:51.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.323+0000 7f4ef27c6700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec0af340 con 0x7f4eec061670 2022-06-29T15:45:51.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.323+0000 7f4ee37fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4eec0b0fe0 0x7f4eec0b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:51.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.323+0000 7f4eebfff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4eec061670 0x7f4eec061a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:51.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.323+0000 7f4ef0d63700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f4eec13d480 0x7f4eec1436c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:41344/0 (socket says 172.21.15.47:41344) 2022-06-29T15:45:51.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.323+0000 7f4ef0d63700 1 -- 172.21.15.47:0/1731387758 learned_addr learned my addr 172.21.15.47:0/1731387758 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:45:51.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.323+0000 7f4ee37fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4eec0b0fe0 0x7f4eec0b0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:53930/0 (socket says 172.21.15.47:53930) 2022-06-29T15:45:51.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.323+0000 7f4ee37fe700 1 -- 172.21.15.47:0/1731387758 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4eec061670 msgr2=0x7f4eec061a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:45:51.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.323+0000 7f4ee37fe700 1 --2- 172.21.15.47:0/1731387758 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4eec061670 0x7f4eec061a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:51.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.323+0000 7f4ee37fe700 1 -- 172.21.15.47:0/1731387758 >> v1:172.21.15.64:6789/0 conn(0x7f4eec13d480 legacy=0x7f4eec1436c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:51.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.323+0000 7f4ee37fe700 1 -- 172.21.15.47:0/1731387758 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eec0af5e0 con 0x7f4eec0b0fe0 2022-06-29T15:45:51.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.323+0000 7f4ee37fe700 1 --2- 172.21.15.47:0/1731387758 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4eec0b0fe0 0x7f4eec0b0700 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f4ee4001490 tx=0x7f4ed4006df0).ready entity=mon.1 client_cookie=7fb092e74438b8a4 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:51.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.323+0000 7f4eeb7fe700 1 -- 172.21.15.47:0/1731387758 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ed400d4a0 con 0x7f4eec0b0fe0 2022-06-29T15:45:51.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.323+0000 7f4eeb7fe700 1 -- 172.21.15.47:0/1731387758 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ed4000e90 con 0x7f4eec0b0fe0 2022-06-29T15:45:51.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.323+0000 7f4eeb7fe700 1 -- 172.21.15.47:0/1731387758 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ed400d7f0 con 0x7f4eec0b0fe0 2022-06-29T15:45:51.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.323+0000 7f4ef27c6700 1 -- 172.21.15.47:0/1731387758 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4eec0b0fe0 msgr2=0x7f4eec0b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:51.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.323+0000 7f4ef27c6700 1 --2- 172.21.15.47:0/1731387758 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4eec0b0fe0 0x7f4eec0b0700 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f4ee4001490 tx=0x7f4ed4006df0).stop 2022-06-29T15:45:51.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.323+0000 7f4ef27c6700 1 -- 172.21.15.47:0/1731387758 shutdown_connections 2022-06-29T15:45:51.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.323+0000 7f4ef27c6700 1 --2- 172.21.15.47:0/1731387758 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4eec0b0fe0 0x7f4eec0b0700 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:51.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.323+0000 7f4ef27c6700 1 --2- 172.21.15.47:0/1731387758 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4eec061670 0x7f4eec061a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:51.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.323+0000 7f4ef27c6700 1 -- 172.21.15.47:0/1731387758 >> 172.21.15.47:0/1731387758 conn(0x7f4eec063800 msgr2=0x7f4eec063be0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:45:51.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.323+0000 7f4ef27c6700 1 -- 172.21.15.47:0/1731387758 shutdown_connections 2022-06-29T15:45:51.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.323+0000 7f4ef27c6700 1 -- 172.21.15.47:0/1731387758 wait complete. 2022-06-29T15:45:51.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.327+0000 7f4ef27c6700 1 Processor -- start 2022-06-29T15:45:51.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.327+0000 7f4ef27c6700 1 -- start start 2022-06-29T15:45:51.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.331+0000 7f4ef27c6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4eec061670 0x7f4eec14afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:51.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.331+0000 7f4ef27c6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4eec0b0fe0 0x7f4eec066730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:51.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.331+0000 7f4ef27c6700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4eec13d480 0x7f4eec066c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:51.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.331+0000 7f4ef27c6700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec0aa990 con 0x7f4eec061670 2022-06-29T15:45:51.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.331+0000 7f4ef27c6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec067750 con 0x7f4eec0b0fe0 2022-06-29T15:45:51.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.331+0000 7f4ef27c6700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec0b7280 con 0x7f4eec13d480 2022-06-29T15:45:51.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.331+0000 7f4eebfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4eec061670 0x7f4eec14afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:51.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.331+0000 7f4eebfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4eec061670 0x7f4eec14afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48230/0 (socket says 172.21.15.47:48230) 2022-06-29T15:45:51.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.331+0000 7f4eebfff700 1 -- 172.21.15.47:0/2993691822 learned_addr learned my addr 172.21.15.47:0/2993691822 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:45:51.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.331+0000 7f4ee37fe700 1 --2- 172.21.15.47:0/2993691822 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4eec0b0fe0 0x7f4eec066730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:51.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.331+0000 7f4ef0d63700 1 --2- 172.21.15.47:0/2993691822 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4eec13d480 0x7f4eec066c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:51.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.331+0000 7f4eebfff700 1 -- 172.21.15.47:0/2993691822 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4eec13d480 msgr2=0x7f4eec066c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:51.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.331+0000 7f4eebfff700 1 --2- 172.21.15.47:0/2993691822 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4eec13d480 0x7f4eec066c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:51.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.331+0000 7f4eebfff700 1 -- 172.21.15.47:0/2993691822 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4eec0b0fe0 msgr2=0x7f4eec066730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:51.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.331+0000 7f4eebfff700 1 --2- 172.21.15.47:0/2993691822 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4eec0b0fe0 0x7f4eec066730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:51.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.331+0000 7f4eebfff700 1 -- 172.21.15.47:0/2993691822 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed400b180 con 0x7f4eec061670 2022-06-29T15:45:51.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.331+0000 7f4eebfff700 1 --2- 172.21.15.47:0/2993691822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4eec061670 0x7f4eec14afb0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7f4edc00c000 tx=0x7f4edc006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:51.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.331+0000 7f4ee9ffb700 1 -- 172.21.15.47:0/2993691822 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4edc010780 con 0x7f4eec061670 2022-06-29T15:45:51.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.331+0000 7f4ee9ffb700 1 -- 172.21.15.47:0/2993691822 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4edc0108e0 con 0x7f4eec061670 2022-06-29T15:45:51.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.331+0000 7f4ee9ffb700 1 -- 172.21.15.47:0/2993691822 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4edc010c40 con 0x7f4eec061670 2022-06-29T15:45:51.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.331+0000 7f4ef27c6700 1 -- 172.21.15.47:0/2993691822 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eec065ff0 con 0x7f4eec061670 2022-06-29T15:45:51.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.331+0000 7f4ef27c6700 1 -- 172.21.15.47:0/2993691822 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eec0abee0 con 0x7f4eec061670 2022-06-29T15:45:51.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.331+0000 7f4ef27c6700 1 -- 172.21.15.47:0/2993691822 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eec0573d0 con 0x7f4eec061670 2022-06-29T15:45:51.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.335+0000 7f4ee9ffb700 1 -- 172.21.15.47:0/2993691822 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4edc01c460 con 0x7f4eec061670 2022-06-29T15:45:51.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.335+0000 7f4ee9ffb700 1 --2- 172.21.15.47:0/2993691822 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4ec4053a70 0x7f4ec4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:51.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.335+0000 7f4ee37fe700 1 --2- 172.21.15.47:0/2993691822 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4ec4053a70 0x7f4ec4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:51.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.335+0000 7f4ee9ffb700 1 -- 172.21.15.47:0/2993691822 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f4edc06c560 con 0x7f4eec061670 2022-06-29T15:45:51.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.335+0000 7f4ee37fe700 1 --2- 172.21.15.47:0/2993691822 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4ec4053a70 0x7f4ec4055f30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f4ed401a000 tx=0x7f4ed4014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:51.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.343+0000 7f4ee9ffb700 1 -- 172.21.15.47:0/2993691822 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4edc043160 con 0x7f4eec061670 2022-06-29T15:45:51.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:51.467+0000 7f4ef27c6700 1 -- 172.21.15.47:0/2993691822 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4eec0b3f80 con 0x7f4eec061670 2022-06-29T15:45:52.194 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.191+0000 7f4ee9ffb700 1 -- 172.21.15.47:0/2993691822 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v49) v1 ==== 185+0+0 (secure 0 0 0) 0x7f4edc046780 con 0x7f4eec061670 2022-06-29T15:45:52.198 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.195+0000 7f4ef27c6700 1 -- 172.21.15.47:0/2993691822 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4ec4053a70 msgr2=0x7f4ec4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:52.198 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.195+0000 7f4ef27c6700 1 --2- 172.21.15.47:0/2993691822 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4ec4053a70 0x7f4ec4055f30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f4ed401a000 tx=0x7f4ed4014000).stop 2022-06-29T15:45:52.198 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.195+0000 7f4ef27c6700 1 -- 172.21.15.47:0/2993691822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4eec061670 msgr2=0x7f4eec14afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:52.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.195+0000 7f4ef27c6700 1 --2- 172.21.15.47:0/2993691822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4eec061670 0x7f4eec14afb0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7f4edc00c000 tx=0x7f4edc006df0).stop 2022-06-29T15:45:52.200 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.195+0000 7f4ef27c6700 1 -- 172.21.15.47:0/2993691822 shutdown_connections 2022-06-29T15:45:52.200 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.195+0000 7f4ef27c6700 1 --2- 172.21.15.47:0/2993691822 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4eec13d480 0x7f4eec066c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:52.200 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.195+0000 7f4ef27c6700 1 --2- 172.21.15.47:0/2993691822 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4ec4053a70 0x7f4ec4055f30 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:52.200 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.195+0000 7f4ef27c6700 1 --2- 172.21.15.47:0/2993691822 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4eec0b0fe0 0x7f4eec066730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:52.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.195+0000 7f4ef27c6700 1 --2- 172.21.15.47:0/2993691822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4eec061670 0x7f4eec14afb0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:52.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.195+0000 7f4ef27c6700 1 -- 172.21.15.47:0/2993691822 >> 172.21.15.47:0/2993691822 conn(0x7f4eec063800 msgr2=0x7f4eec05f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:45:52.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.195+0000 7f4ef27c6700 1 -- 172.21.15.47:0/2993691822 shutdown_connections 2022-06-29T15:45:52.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.195+0000 7f4ef27c6700 1 -- 172.21.15.47:0/2993691822 wait complete. 2022-06-29T15:45:52.202 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' removed 2022-06-29T15:45:52.214 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-06-29T15:45:52.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.327+0000 7f1015e60700 1 Processor -- start 2022-06-29T15:45:52.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.327+0000 7f1015e60700 1 -- start start 2022-06-29T15:45:52.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.327+0000 7f1015e60700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f101005f490 0x7f1010144670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:52.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.327+0000 7f1015e60700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1010061740 0x7f1010147140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:52.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.327+0000 7f1015e60700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1010061e50 con 0x7f101005f0b0 2022-06-29T15:45:52.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.327+0000 7f1015e60700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1010147630 con 0x7f101005f490 2022-06-29T15:45:52.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.327+0000 7f1015e60700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10100620f0 con 0x7f1010061740 2022-06-29T15:45:52.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.331+0000 7f1014e5e700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f101005f0b0 0x7f1010143950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33792/0 (socket says 172.21.15.47:33792) 2022-06-29T15:45:52.335 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.331+0000 7f1014e5e700 1 -- 172.21.15.47:0/4169460355 learned_addr learned my addr 172.21.15.47:0/4169460355 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:45:52.335 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.331+0000 7f101565f700 1 --2- 172.21.15.47:0/4169460355 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1010061740 0x7f1010147140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:52.335 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.331+0000 7f1007fff700 1 --2- 172.21.15.47:0/4169460355 >> v2:172.21.15.64:3300/0 conn(0x7f101005f490 0x7f1010144670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:52.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.331+0000 7f10077fe700 1 -- 172.21.15.47:0/4169460355 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 225143965 0 0) 0x7f1010061e50 con 0x7f101005f0b0 2022-06-29T15:45:52.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.331+0000 7f10077fe700 1 -- 172.21.15.47:0/4169460355 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ffc003260 con 0x7f101005f0b0 2022-06-29T15:45:52.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.331+0000 7f101565f700 1 -- 172.21.15.47:0/4169460355 >> v2:172.21.15.64:3300/0 conn(0x7f101005f490 msgr2=0x7f1010144670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:45:52.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.331+0000 7f101565f700 1 --2- 172.21.15.47:0/4169460355 >> v2:172.21.15.64:3300/0 conn(0x7f101005f490 0x7f1010144670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:52.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.331+0000 7f101565f700 1 -- 172.21.15.47:0/4169460355 >> v1:172.21.15.47:6789/0 conn(0x7f101005f0b0 legacy=0x7f1010143950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:52.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.331+0000 7f101565f700 1 -- 172.21.15.47:0/4169460355 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10101434b0 con 0x7f1010061740 2022-06-29T15:45:52.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.331+0000 7f101565f700 1 --2- 172.21.15.47:0/4169460355 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1010061740 0x7f1010147140 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f10080032f0 tx=0x7f10080091d0).ready entity=mon.2 client_cookie=a6e885a63b9d4c61 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:52.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.331+0000 7f10077fe700 1 -- 172.21.15.47:0/4169460355 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f100800b600 con 0x7f1010061740 2022-06-29T15:45:52.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.331+0000 7f10077fe700 1 -- 172.21.15.47:0/4169460355 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f100800b760 con 0x7f1010061740 2022-06-29T15:45:52.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.331+0000 7f10077fe700 1 -- 172.21.15.47:0/4169460355 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f100800b8d0 con 0x7f1010061740 2022-06-29T15:45:52.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.331+0000 7f1015e60700 1 -- 172.21.15.47:0/4169460355 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1010061740 msgr2=0x7f1010147140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:52.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.331+0000 7f1015e60700 1 --2- 172.21.15.47:0/4169460355 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1010061740 0x7f1010147140 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f10080032f0 tx=0x7f10080091d0).stop 2022-06-29T15:45:52.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.331+0000 7f1015e60700 1 -- 172.21.15.47:0/4169460355 shutdown_connections 2022-06-29T15:45:52.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.331+0000 7f1015e60700 1 --2- 172.21.15.47:0/4169460355 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1010061740 0x7f1010147140 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:52.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.331+0000 7f1015e60700 1 --2- 172.21.15.47:0/4169460355 >> v2:172.21.15.64:3300/0 conn(0x7f101005f490 0x7f1010144670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:52.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.331+0000 7f1015e60700 1 -- 172.21.15.47:0/4169460355 >> 172.21.15.47:0/4169460355 conn(0x7f1010062570 msgr2=0x7f10100647d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:45:52.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.331+0000 7f1015e60700 1 -- 172.21.15.47:0/4169460355 shutdown_connections 2022-06-29T15:45:52.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.331+0000 7f1015e60700 1 -- 172.21.15.47:0/4169460355 wait complete. 2022-06-29T15:45:52.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.335+0000 7f1015e60700 1 Processor -- start 2022-06-29T15:45:52.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.335+0000 7f1015e60700 1 -- start start 2022-06-29T15:45:52.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.335+0000 7f1015e60700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f101005f0b0 0x7f1010152b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:52.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.335+0000 7f1015e60700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f101005f490 0x7f10100b1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:52.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.335+0000 7f1015e60700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1010061740 0x7f10100b1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:52.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.335+0000 7f1015e60700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1010152ff0 con 0x7f1010061740 2022-06-29T15:45:52.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.335+0000 7f1015e60700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10100b5b40 con 0x7f101005f490 2022-06-29T15:45:52.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.335+0000 7f1015e60700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10100b5d70 con 0x7f101005f0b0 2022-06-29T15:45:52.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.335+0000 7f101565f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1010061740 0x7f10100b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:52.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.335+0000 7f101565f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1010061740 0x7f10100b1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48246/0 (socket says 172.21.15.47:48246) 2022-06-29T15:45:52.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.335+0000 7f101565f700 1 -- 172.21.15.47:0/1304669500 learned_addr learned my addr 172.21.15.47:0/1304669500 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:45:52.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.335+0000 7f1014e5e700 1 --2- 172.21.15.47:0/1304669500 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f101005f0b0 0x7f1010152b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:52.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.335+0000 7f1007fff700 1 --2- 172.21.15.47:0/1304669500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f101005f490 0x7f10100b1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:52.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.335+0000 7f101565f700 1 -- 172.21.15.47:0/1304669500 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f101005f0b0 msgr2=0x7f1010152b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:52.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.335+0000 7f101565f700 1 --2- 172.21.15.47:0/1304669500 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f101005f0b0 0x7f1010152b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:52.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.335+0000 7f101565f700 1 -- 172.21.15.47:0/1304669500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f101005f490 msgr2=0x7f10100b1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:52.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.335+0000 7f101565f700 1 --2- 172.21.15.47:0/1304669500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f101005f490 0x7f10100b1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:52.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.335+0000 7f101565f700 1 -- 172.21.15.47:0/1304669500 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1008009470 con 0x7f1010061740 2022-06-29T15:45:52.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.335+0000 7f1014e5e700 1 --2- 172.21.15.47:0/1304669500 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f101005f0b0 0x7f1010152b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:45:52.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.335+0000 7f101565f700 1 --2- 172.21.15.47:0/1304669500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1010061740 0x7f10100b1630 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7f10080032f0 tx=0x7f1008016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:52.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.339+0000 7f1005ffb700 1 -- 172.21.15.47:0/1304669500 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f100800b8d0 con 0x7f1010061740 2022-06-29T15:45:52.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.339+0000 7f1005ffb700 1 -- 172.21.15.47:0/1304669500 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f100801f3f0 con 0x7f1010061740 2022-06-29T15:45:52.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.339+0000 7f1015e60700 1 -- 172.21.15.47:0/1304669500 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10100b6010 con 0x7f1010061740 2022-06-29T15:45:52.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.339+0000 7f1005ffb700 1 -- 172.21.15.47:0/1304669500 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f100800b600 con 0x7f1010061740 2022-06-29T15:45:52.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.339+0000 7f1015e60700 1 -- 172.21.15.47:0/1304669500 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10100b3d70 con 0x7f1010061740 2022-06-29T15:45:52.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.339+0000 7f1015e60700 1 -- 172.21.15.47:0/1304669500 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10100573d0 con 0x7f1010061740 2022-06-29T15:45:52.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.339+0000 7f1005ffb700 1 -- 172.21.15.47:0/1304669500 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1008026030 con 0x7f1010061740 2022-06-29T15:45:52.352 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.339+0000 7f1005ffb700 1 --2- 172.21.15.47:0/1304669500 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0ff0053aa0 0x7f0ff0055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:52.352 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.339+0000 7f1005ffb700 1 -- 172.21.15.47:0/1304669500 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6068+0+0 (secure 0 0 0) 0x7f100806c8c0 con 0x7f1010061740 2022-06-29T15:45:52.352 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.339+0000 7f1014e5e700 1 --2- 172.21.15.47:0/1304669500 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0ff0053aa0 0x7f0ff0055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:52.352 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.343+0000 7f1014e5e700 1 --2- 172.21.15.47:0/1304669500 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0ff0053aa0 0x7f0ff0055f60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f100c00d000 tx=0x7f100c009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:52.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.351+0000 7f1005ffb700 1 -- 172.21.15.47:0/1304669500 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f10080437e0 con 0x7f1010061740 2022-06-29T15:45:52.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:52.483+0000 7f1015e60700 1 -- 172.21.15.47:0/1304669500 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1010061b90 con 0x7f1010061740 2022-06-29T15:45:53.204 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.199+0000 7f1005ffb700 1 -- 172.21.15.47:0/1304669500 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v50) v1 ==== 173+0+0 (secure 0 0 0) 0x7f10080150d0 con 0x7f1010061740 2022-06-29T15:45:53.209 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.203+0000 7f1015e60700 1 -- 172.21.15.47:0/1304669500 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0ff0053aa0 msgr2=0x7f0ff0055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:53.210 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.203+0000 7f1015e60700 1 --2- 172.21.15.47:0/1304669500 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0ff0053aa0 0x7f0ff0055f60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f100c00d000 tx=0x7f100c009040).stop 2022-06-29T15:45:53.210 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.203+0000 7f1015e60700 1 -- 172.21.15.47:0/1304669500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1010061740 msgr2=0x7f10100b1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:53.210 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.203+0000 7f1015e60700 1 --2- 172.21.15.47:0/1304669500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1010061740 0x7f10100b1630 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7f10080032f0 tx=0x7f1008016000).stop 2022-06-29T15:45:53.211 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.207+0000 7f1015e60700 1 -- 172.21.15.47:0/1304669500 shutdown_connections 2022-06-29T15:45:53.211 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.207+0000 7f1015e60700 1 --2- 172.21.15.47:0/1304669500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1010061740 0x7f10100b1630 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:53.211 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.207+0000 7f1015e60700 1 --2- 172.21.15.47:0/1304669500 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0ff0053aa0 0x7f0ff0055f60 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:53.212 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.207+0000 7f1015e60700 1 --2- 172.21.15.47:0/1304669500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f101005f490 0x7f10100b1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:53.212 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.207+0000 7f1015e60700 1 --2- 172.21.15.47:0/1304669500 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f101005f0b0 0x7f1010152b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:53.212 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.207+0000 7f1015e60700 1 -- 172.21.15.47:0/1304669500 >> 172.21.15.47:0/1304669500 conn(0x7f1010062570 msgr2=0x7f1010066e70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:45:53.213 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.207+0000 7f1015e60700 1 -- 172.21.15.47:0/1304669500 shutdown_connections 2022-06-29T15:45:53.213 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.207+0000 7f1015e60700 1 -- 172.21.15.47:0/1304669500 wait complete. 2022-06-29T15:45:53.213 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' removed 2022-06-29T15:45:53.222 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-06-29T15:45:53.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.323+0000 7fe38a886700 1 Processor -- start 2022-06-29T15:45:53.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.327+0000 7fe38a886700 1 -- start start 2022-06-29T15:45:53.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.327+0000 7fe38a886700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe3840b5d90 0x7fe384143a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:53.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.327+0000 7fe38a886700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe3840676a0 0x7fe384140130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:53.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.327+0000 7fe38a886700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe38405cc30 con 0x7fe38405c850 2022-06-29T15:45:53.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.327+0000 7fe38a886700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3841430a0 con 0x7fe3840676a0 2022-06-29T15:45:53.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.327+0000 7fe38a886700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe384143300 con 0x7fe3840b5d90 2022-06-29T15:45:53.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.327+0000 7fe38a085700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe3840676a0 0x7fe384140130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:53.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.327+0000 7fe38a085700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe3840676a0 0x7fe384140130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48252/0 (socket says 172.21.15.47:48252) 2022-06-29T15:45:53.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.327+0000 7fe38a085700 1 -- 172.21.15.47:0/2718865558 learned_addr learned my addr 172.21.15.47:0/2718865558 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:45:53.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.327+0000 7fe389083700 1 --2- 172.21.15.47:0/2718865558 >> v2:172.21.15.64:3300/0 conn(0x7fe3840b5d90 0x7fe384143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:53.335 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.327+0000 7fe38a085700 1 -- 172.21.15.47:0/2718865558 >> v2:172.21.15.64:3300/0 conn(0x7fe3840b5d90 msgr2=0x7fe384143a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:45:53.335 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.327+0000 7fe38a085700 1 --2- 172.21.15.47:0/2718865558 >> v2:172.21.15.64:3300/0 conn(0x7fe3840b5d90 0x7fe384143a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:53.335 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.327+0000 7fe38a085700 1 -- 172.21.15.47:0/2718865558 >> v1:172.21.15.64:6789/0 conn(0x7fe38405c850 legacy=0x7fe3840b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:53.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.327+0000 7fe38a085700 1 -- 172.21.15.47:0/2718865558 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3841435a0 con 0x7fe3840676a0 2022-06-29T15:45:53.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.327+0000 7fe38a085700 1 --2- 172.21.15.47:0/2718865558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3840676a0 0x7fe384140130 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7fe37c002f70 tx=0x7fe37c006eb0).ready entity=mon.0 client_cookie=4967baa6cc807985 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:53.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.327+0000 7fe388882700 1 -- 172.21.15.47:0/2718865558 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe37c00d610 con 0x7fe3840676a0 2022-06-29T15:45:53.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.327+0000 7fe388882700 1 -- 172.21.15.47:0/2718865558 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe37c000e90 con 0x7fe3840676a0 2022-06-29T15:45:53.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.327+0000 7fe388882700 1 -- 172.21.15.47:0/2718865558 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe37c00d9a0 con 0x7fe3840676a0 2022-06-29T15:45:53.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.327+0000 7fe38a886700 1 -- 172.21.15.47:0/2718865558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3840676a0 msgr2=0x7fe384140130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:53.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.327+0000 7fe38a886700 1 --2- 172.21.15.47:0/2718865558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3840676a0 0x7fe384140130 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7fe37c002f70 tx=0x7fe37c006eb0).stop 2022-06-29T15:45:53.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.327+0000 7fe38a886700 1 -- 172.21.15.47:0/2718865558 shutdown_connections 2022-06-29T15:45:53.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.327+0000 7fe38a886700 1 --2- 172.21.15.47:0/2718865558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3840676a0 0x7fe384140130 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:53.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.327+0000 7fe38a886700 1 --2- 172.21.15.47:0/2718865558 >> v2:172.21.15.64:3300/0 conn(0x7fe3840b5d90 0x7fe384143a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:53.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.327+0000 7fe38a886700 1 -- 172.21.15.47:0/2718865558 >> 172.21.15.47:0/2718865558 conn(0x7fe384060e00 msgr2=0x7fe3840611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:45:53.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.327+0000 7fe38a886700 1 -- 172.21.15.47:0/2718865558 shutdown_connections 2022-06-29T15:45:53.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.327+0000 7fe38a886700 1 -- 172.21.15.47:0/2718865558 wait complete. 2022-06-29T15:45:53.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.331+0000 7fe38a886700 1 Processor -- start 2022-06-29T15:45:53.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.331+0000 7fe38a886700 1 -- start start 2022-06-29T15:45:53.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.331+0000 7fe38a886700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe38405c850 0x7fe3840b95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:53.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.331+0000 7fe38a886700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3840676a0 0x7fe384152d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:53.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.331+0000 7fe38a886700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe3840b5d90 0x7fe3840b0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:53.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.331+0000 7fe38a886700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3840ba660 con 0x7fe38405c850 2022-06-29T15:45:53.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.331+0000 7fe38a886700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3840b39a0 con 0x7fe3840676a0 2022-06-29T15:45:53.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.331+0000 7fe38a886700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3840b3c40 con 0x7fe3840b5d90 2022-06-29T15:45:53.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.331+0000 7fe389884700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe38405c850 0x7fe3840b95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:53.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.331+0000 7fe389884700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe38405c850 0x7fe3840b95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48258/0 (socket says 172.21.15.47:48258) 2022-06-29T15:45:53.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.331+0000 7fe389884700 1 -- 172.21.15.47:0/1387081719 learned_addr learned my addr 172.21.15.47:0/1387081719 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:45:53.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.331+0000 7fe389083700 1 --2- 172.21.15.47:0/1387081719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3840676a0 0x7fe384152d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:53.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.331+0000 7fe38a085700 1 --2- 172.21.15.47:0/1387081719 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe3840b5d90 0x7fe3840b0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:53.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.331+0000 7fe389884700 1 -- 172.21.15.47:0/1387081719 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe3840b5d90 msgr2=0x7fe3840b0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:53.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.331+0000 7fe389884700 1 --2- 172.21.15.47:0/1387081719 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe3840b5d90 0x7fe3840b0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:53.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.331+0000 7fe389884700 1 -- 172.21.15.47:0/1387081719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3840676a0 msgr2=0x7fe384152d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:53.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.331+0000 7fe389884700 1 --2- 172.21.15.47:0/1387081719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3840676a0 0x7fe384152d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:53.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.331+0000 7fe389884700 1 -- 172.21.15.47:0/1387081719 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe37c00b1e0 con 0x7fe38405c850 2022-06-29T15:45:53.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.331+0000 7fe38a085700 1 --2- 172.21.15.47:0/1387081719 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe3840b5d90 0x7fe3840b0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:45:53.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.331+0000 7fe389884700 1 --2- 172.21.15.47:0/1387081719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe38405c850 0x7fe3840b95b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7fe3840ae960 tx=0x7fe380001410).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:53.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.331+0000 7fe372ffd700 1 -- 172.21.15.47:0/1387081719 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe3800024e0 con 0x7fe38405c850 2022-06-29T15:45:53.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.331+0000 7fe372ffd700 1 -- 172.21.15.47:0/1387081719 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe380002640 con 0x7fe38405c850 2022-06-29T15:45:53.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.331+0000 7fe38a886700 1 -- 172.21.15.47:0/1387081719 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3840b3ee0 con 0x7fe38405c850 2022-06-29T15:45:53.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.331+0000 7fe372ffd700 1 -- 172.21.15.47:0/1387081719 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe3800029a0 con 0x7fe38405c850 2022-06-29T15:45:53.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.331+0000 7fe38a886700 1 -- 172.21.15.47:0/1387081719 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3841532d0 con 0x7fe38405c850 2022-06-29T15:45:53.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.335+0000 7fe38a886700 1 -- 172.21.15.47:0/1387081719 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3840573d0 con 0x7fe38405c850 2022-06-29T15:45:53.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.335+0000 7fe372ffd700 1 -- 172.21.15.47:0/1387081719 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe380010030 con 0x7fe38405c850 2022-06-29T15:45:53.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.335+0000 7fe372ffd700 1 --2- 172.21.15.47:0/1387081719 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe364053aa0 0x7fe364055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:53.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.335+0000 7fe372ffd700 1 -- 172.21.15.47:0/1387081719 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fe38006c090 con 0x7fe38405c850 2022-06-29T15:45:53.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.335+0000 7fe389083700 1 --2- 172.21.15.47:0/1387081719 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe364053aa0 0x7fe364055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:53.352 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.335+0000 7fe389083700 1 --2- 172.21.15.47:0/1387081719 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe364053aa0 0x7fe364055f60 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7fe37400c000 tx=0x7fe374008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:53.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.339+0000 7fe372ffd700 1 -- 172.21.15.47:0/1387081719 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe380043020 con 0x7fe38405c850 2022-06-29T15:45:53.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.463+0000 7fe38a886700 1 -- 172.21.15.47:0/1387081719 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fe384061b00 con 0x7fe38405c850 2022-06-29T15:45:53.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.471+0000 7fe372ffd700 1 -- 172.21.15.47:0/1387081719 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v5) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe380046640 con 0x7fe38405c850 2022-06-29T15:45:53.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.475+0000 7fe38a886700 1 -- 172.21.15.47:0/1387081719 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe364053aa0 msgr2=0x7fe364055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:53.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.475+0000 7fe38a886700 1 --2- 172.21.15.47:0/1387081719 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe364053aa0 0x7fe364055f60 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7fe37400c000 tx=0x7fe374008040).stop 2022-06-29T15:45:53.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.475+0000 7fe38a886700 1 -- 172.21.15.47:0/1387081719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe38405c850 msgr2=0x7fe3840b95b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:53.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.475+0000 7fe38a886700 1 --2- 172.21.15.47:0/1387081719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe38405c850 0x7fe3840b95b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7fe3840ae960 tx=0x7fe380001410).stop 2022-06-29T15:45:53.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.475+0000 7fe38a886700 1 -- 172.21.15.47:0/1387081719 shutdown_connections 2022-06-29T15:45:53.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.475+0000 7fe38a886700 1 --2- 172.21.15.47:0/1387081719 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe3840b5d90 0x7fe3840b0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:53.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.475+0000 7fe38a886700 1 --2- 172.21.15.47:0/1387081719 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe364053aa0 0x7fe364055f60 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:53.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.475+0000 7fe38a886700 1 --2- 172.21.15.47:0/1387081719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3840676a0 0x7fe384152d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:53.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.475+0000 7fe38a886700 1 --2- 172.21.15.47:0/1387081719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe38405c850 0x7fe3840b95b0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:53.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.475+0000 7fe38a886700 1 -- 172.21.15.47:0/1387081719 >> 172.21.15.47:0/1387081719 conn(0x7fe384060e00 msgr2=0x7fe3840af620 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:45:53.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.475+0000 7fe38a886700 1 -- 172.21.15.47:0/1387081719 shutdown_connections 2022-06-29T15:45:53.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.475+0000 7fe38a886700 1 -- 172.21.15.47:0/1387081719 wait complete. 2022-06-29T15:45:53.491 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead' 2022-06-29T15:45:53.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.591+0000 7fa49e432700 1 Processor -- start 2022-06-29T15:45:53.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.591+0000 7fa49e432700 1 -- start start 2022-06-29T15:45:53.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.591+0000 7fa49e432700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa498145090 0x7fa4980b2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:53.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.591+0000 7fa49e432700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4980a9120 con 0x7fa498060eb0 2022-06-29T15:45:53.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.591+0000 7fa49e432700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4980b0b80 con 0x7fa4980b5c60 2022-06-29T15:45:53.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.591+0000 7fa49e432700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4980a89e0 con 0x7fa498145090 2022-06-29T15:45:53.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.591+0000 7fa49cc2f700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa498060eb0 0x7fa4980b1e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33818/0 (socket says 172.21.15.47:33818) 2022-06-29T15:45:53.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.591+0000 7fa49cc2f700 1 -- 172.21.15.47:0/3866927664 learned_addr learned my addr 172.21.15.47:0/3866927664 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:45:53.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.591+0000 7fa49dc31700 1 --2- 172.21.15.47:0/3866927664 >> v2:172.21.15.64:3300/0 conn(0x7fa498145090 0x7fa4980b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:53.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.591+0000 7fa48ffff700 1 -- 172.21.15.47:0/3866927664 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3295413973 0 0) 0x7fa4980a9120 con 0x7fa498060eb0 2022-06-29T15:45:53.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.591+0000 7fa48ffff700 1 -- 172.21.15.47:0/3866927664 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa47c003260 con 0x7fa498060eb0 2022-06-29T15:45:53.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.591+0000 7fa48ffff700 1 -- 172.21.15.47:0/3866927664 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2967849787 0 0) 0x7fa47c003260 con 0x7fa498060eb0 2022-06-29T15:45:53.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.591+0000 7fa48ffff700 1 -- 172.21.15.47:0/3866927664 >> v2:172.21.15.64:3300/0 conn(0x7fa498145090 msgr2=0x7fa4980b2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:45:53.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.591+0000 7fa48ffff700 1 --2- 172.21.15.47:0/3866927664 >> v2:172.21.15.64:3300/0 conn(0x7fa498145090 0x7fa4980b2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:53.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.591+0000 7fa48ffff700 1 -- 172.21.15.47:0/3866927664 >> v1:172.21.15.64:6789/0 conn(0x7fa4980b5c60 legacy=0x7fa498144980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:53.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.591+0000 7fa48ffff700 1 -- 172.21.15.47:0/3866927664 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4980a8c80 con 0x7fa498060eb0 2022-06-29T15:45:53.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.591+0000 7fa48ffff700 1 -- 172.21.15.47:0/3866927664 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7fa4940019e0 con 0x7fa498060eb0 2022-06-29T15:45:53.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.591+0000 7fa48ffff700 1 -- 172.21.15.47:0/3866927664 >> v1:172.21.15.47:6789/0 conn(0x7fa498060eb0 legacy=0x7fa4980b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:53.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.591+0000 7fa48ffff700 1 --2- 172.21.15.47:0/3866927664 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa47c003750 0x7fa47c003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:53.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.591+0000 7fa48ffff700 1 --2- 172.21.15.47:0/3866927664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa47c004500 0x7fa47c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:53.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.591+0000 7fa48ffff700 1 --2- 172.21.15.47:0/3866927664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa47c004e70 0x7fa47c00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:53.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.595+0000 7fa48ffff700 1 -- 172.21.15.47:0/3866927664 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4980a89e0 con 0x7fa47c004500 2022-06-29T15:45:53.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.595+0000 7fa48ffff700 1 -- 172.21.15.47:0/3866927664 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa47c00bd20 con 0x7fa47c004e70 2022-06-29T15:45:53.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.595+0000 7fa48ffff700 1 -- 172.21.15.47:0/3866927664 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa47c00bee0 con 0x7fa47c003750 2022-06-29T15:45:53.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.595+0000 7fa49cc2f700 1 --2- 172.21.15.47:0/3866927664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa47c004500 0x7fa47c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:53.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.595+0000 7fa49dc31700 1 --2- 172.21.15.47:0/3866927664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa47c004e70 0x7fa47c00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:53.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.595+0000 7fa49d430700 1 --2- 172.21.15.47:0/3866927664 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa47c003750 0x7fa47c003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:53.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.595+0000 7fa49dc31700 1 -- 172.21.15.47:0/3866927664 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa47c003750 msgr2=0x7fa47c003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:45:53.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.595+0000 7fa49dc31700 1 --2- 172.21.15.47:0/3866927664 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa47c003750 0x7fa47c003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:53.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.595+0000 7fa49dc31700 1 -- 172.21.15.47:0/3866927664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa47c004500 msgr2=0x7fa47c004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:45:53.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.595+0000 7fa49dc31700 1 --2- 172.21.15.47:0/3866927664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa47c004500 0x7fa47c004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:53.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.595+0000 7fa49dc31700 1 -- 172.21.15.47:0/3866927664 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa47c003260 con 0x7fa47c004e70 2022-06-29T15:45:53.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.595+0000 7fa49cc2f700 1 --2- 172.21.15.47:0/3866927664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa47c004500 0x7fa47c004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:45:53.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.595+0000 7fa49dc31700 1 -- 172.21.15.47:0/3866927664 reap_dead start 2022-06-29T15:45:53.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.595+0000 7fa49dc31700 1 --2- 172.21.15.47:0/3866927664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa47c004e70 0x7fa47c00b750 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7fa49000c000 tx=0x7fa490006df0).ready entity=mon.1 client_cookie=def80296dd0779d9 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:53.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.595+0000 7fa48ffff700 1 -- 172.21.15.47:0/3866927664 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa49000e830 con 0x7fa47c004e70 2022-06-29T15:45:53.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.595+0000 7fa48ffff700 1 -- 172.21.15.47:0/3866927664 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa49000e990 con 0x7fa47c004e70 2022-06-29T15:45:53.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.595+0000 7fa48ffff700 1 -- 172.21.15.47:0/3866927664 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa49000ecf0 con 0x7fa47c004e70 2022-06-29T15:45:53.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.595+0000 7fa49e432700 1 -- 172.21.15.47:0/3866927664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa47c004e70 msgr2=0x7fa47c00b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:53.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.595+0000 7fa49e432700 1 --2- 172.21.15.47:0/3866927664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa47c004e70 0x7fa47c00b750 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7fa49000c000 tx=0x7fa490006df0).stop 2022-06-29T15:45:53.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.595+0000 7fa49e432700 1 -- 172.21.15.47:0/3866927664 shutdown_connections 2022-06-29T15:45:53.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.595+0000 7fa49e432700 1 --2- 172.21.15.47:0/3866927664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa47c004e70 0x7fa47c00b750 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:53.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.595+0000 7fa49e432700 1 -- 172.21.15.47:0/3866927664 >> 172.21.15.47:0/3866927664 conn(0x7fa498062560 msgr2=0x7fa498062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:45:53.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.595+0000 7fa49e432700 1 -- 172.21.15.47:0/3866927664 shutdown_connections 2022-06-29T15:45:53.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.595+0000 7fa49e432700 1 -- 172.21.15.47:0/3866927664 wait complete. 2022-06-29T15:45:53.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.595+0000 7fa49e432700 1 Processor -- start 2022-06-29T15:45:53.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.595+0000 7fa49e432700 1 -- start start 2022-06-29T15:45:53.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.599+0000 7fa49e432700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa47c004e70 0x7fa49814e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:53.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.599+0000 7fa49e432700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa498148a00 0x7fa4980ab4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:53.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.599+0000 7fa49e432700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4980aa1a0 0x7fa49814a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:53.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.599+0000 7fa49e432700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4980ab9c0 con 0x7fa4980aa1a0 2022-06-29T15:45:53.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.599+0000 7fa49e432700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4980aa7c0 con 0x7fa47c004e70 2022-06-29T15:45:53.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.599+0000 7fa49e432700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4980aaa60 con 0x7fa498148a00 2022-06-29T15:45:53.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.599+0000 7fa49dc31700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4980aa1a0 0x7fa49814a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:53.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.599+0000 7fa49dc31700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4980aa1a0 0x7fa49814a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48280/0 (socket says 172.21.15.47:48280) 2022-06-29T15:45:53.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.599+0000 7fa49dc31700 1 -- 172.21.15.47:0/198861868 learned_addr learned my addr 172.21.15.47:0/198861868 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:45:53.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.599+0000 7fa49d430700 1 --2- 172.21.15.47:0/198861868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa47c004e70 0x7fa49814e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:53.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.599+0000 7fa49cc2f700 1 --2- 172.21.15.47:0/198861868 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa498148a00 0x7fa4980ab4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:53.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.599+0000 7fa49dc31700 1 -- 172.21.15.47:0/198861868 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa498148a00 msgr2=0x7fa4980ab4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:53.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.599+0000 7fa49dc31700 1 --2- 172.21.15.47:0/198861868 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa498148a00 0x7fa4980ab4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:53.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.599+0000 7fa49dc31700 1 -- 172.21.15.47:0/198861868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa47c004e70 msgr2=0x7fa49814e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:53.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.599+0000 7fa49dc31700 1 --2- 172.21.15.47:0/198861868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa47c004e70 0x7fa49814e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:53.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.599+0000 7fa49dc31700 1 -- 172.21.15.47:0/198861868 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4980aad00 con 0x7fa4980aa1a0 2022-06-29T15:45:53.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.599+0000 7fa49d430700 1 --2- 172.21.15.47:0/198861868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa47c004e70 0x7fa49814e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:45:53.622 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.599+0000 7fa49cc2f700 1 --2- 172.21.15.47:0/198861868 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa498148a00 0x7fa4980ab4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:45:53.622 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.599+0000 7fa49dc31700 1 --2- 172.21.15.47:0/198861868 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4980aa1a0 0x7fa49814a640 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7fa49000c000 tx=0x7fa490018000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:53.622 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.599+0000 7fa48e7fc700 1 -- 172.21.15.47:0/198861868 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa490012e10 con 0x7fa4980aa1a0 2022-06-29T15:45:53.622 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.599+0000 7fa48e7fc700 1 -- 172.21.15.47:0/198861868 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa490000e90 con 0x7fa4980aa1a0 2022-06-29T15:45:53.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.599+0000 7fa49e432700 1 -- 172.21.15.47:0/198861868 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa49814a1a0 con 0x7fa4980aa1a0 2022-06-29T15:45:53.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.599+0000 7fa48e7fc700 1 -- 172.21.15.47:0/198861868 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa49001d600 con 0x7fa4980aa1a0 2022-06-29T15:45:53.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.599+0000 7fa49e432700 1 -- 172.21.15.47:0/198861868 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa49814ab30 con 0x7fa4980aa1a0 2022-06-29T15:45:53.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.599+0000 7fa49e432700 1 -- 172.21.15.47:0/198861868 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa49814a360 con 0x7fa4980aa1a0 2022-06-29T15:45:53.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.599+0000 7fa48e7fc700 1 -- 172.21.15.47:0/198861868 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa49001d760 con 0x7fa4980aa1a0 2022-06-29T15:45:53.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.603+0000 7fa48e7fc700 1 --2- 172.21.15.47:0/198861868 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa480053b30 0x7fa480055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:53.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.603+0000 7fa48e7fc700 1 -- 172.21.15.47:0/198861868 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fa49006c860 con 0x7fa4980aa1a0 2022-06-29T15:45:53.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.603+0000 7fa49d430700 1 --2- 172.21.15.47:0/198861868 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa480053b30 0x7fa480055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:53.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.603+0000 7fa49d430700 1 --2- 172.21.15.47:0/198861868 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa480053b30 0x7fa480055ff0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7fa488003000 tx=0x7fa488009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:53.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.607+0000 7fa48e7fc700 1 -- 172.21.15.47:0/198861868 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa490043900 con 0x7fa4980aa1a0 2022-06-29T15:45:53.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:53.771+0000 7fa49e432700 1 -- 172.21.15.47:0/198861868 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead"]} v 0) v1 -- 0x7fa4980573d0 con 0x7fa4980aa1a0 2022-06-29T15:45:54.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.203+0000 7fa48e7fc700 1 -- 172.21.15.47:0/198861868 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead"]}]=0 v156) v1 ==== 144+0+0 (secure 0 0 0) 0x7fa49000be60 con 0x7fa4980aa1a0 2022-06-29T15:45:54.211 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.207+0000 7fa49e432700 1 -- 172.21.15.47:0/198861868 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa480053b30 msgr2=0x7fa480055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:54.212 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.207+0000 7fa49e432700 1 --2- 172.21.15.47:0/198861868 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa480053b30 0x7fa480055ff0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7fa488003000 tx=0x7fa488009040).stop 2022-06-29T15:45:54.212 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.207+0000 7fa49e432700 1 -- 172.21.15.47:0/198861868 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4980aa1a0 msgr2=0x7fa49814a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:54.212 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.207+0000 7fa49e432700 1 --2- 172.21.15.47:0/198861868 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4980aa1a0 0x7fa49814a640 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7fa49000c000 tx=0x7fa490018000).stop 2022-06-29T15:45:54.212 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.207+0000 7fa49e432700 1 -- 172.21.15.47:0/198861868 shutdown_connections 2022-06-29T15:45:54.213 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.207+0000 7fa49e432700 1 --2- 172.21.15.47:0/198861868 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4980aa1a0 0x7fa49814a640 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:54.213 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.207+0000 7fa49e432700 1 --2- 172.21.15.47:0/198861868 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa498148a00 0x7fa4980ab4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:54.213 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.207+0000 7fa49e432700 1 --2- 172.21.15.47:0/198861868 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa480053b30 0x7fa480055ff0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:54.213 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.207+0000 7fa49e432700 1 --2- 172.21.15.47:0/198861868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa47c004e70 0x7fa49814e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:54.213 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.207+0000 7fa49e432700 1 -- 172.21.15.47:0/198861868 >> 172.21.15.47:0/198861868 conn(0x7fa498062560 msgr2=0x7fa498062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:45:54.214 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.207+0000 7fa49e432700 1 -- 172.21.15.47:0/198861868 shutdown_connections 2022-06-29T15:45:54.214 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.207+0000 7fa49e432700 1 -- 172.21.15.47:0/198861868 wait complete. 2022-06-29T15:45:54.226 INFO:tasks.cephfs_test_runner:test_drop_cache_command_dead (tasks.cephfs.test_misc.TestCacheDrop) ... ok 2022-06-29T15:45:54.227 INFO:tasks.cephfs_test_runner:Starting test: test_drop_cache_command_dead_timeout (tasks.cephfs.test_misc.TestCacheDrop) 2022-06-29T15:45:54.228 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout' 2022-06-29T15:45:54.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.331+0000 7f507318e700 1 Processor -- start 2022-06-29T15:45:54.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.331+0000 7f507318e700 1 -- start start 2022-06-29T15:45:54.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.331+0000 7f507318e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f506c0ab040 0x7f506c066a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:54.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.331+0000 7f507318e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f506c144e30 0x7f506c143910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:54.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.331+0000 7f507318e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f506c143e00 0x7f506c1414a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:54.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.335+0000 7f507318e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f506c066f70 con 0x7f506c0ab040 2022-06-29T15:45:54.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.335+0000 7f507318e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f506c0676d0 con 0x7f506c144e30 2022-06-29T15:45:54.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.335+0000 7f507318e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f506c0ab420 con 0x7f506c143e00 2022-06-29T15:45:54.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.335+0000 7f5070f2a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f506c0ab040 0x7f506c066a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:54.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.335+0000 7f507172b700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f506c143e00 0x7f506c1414a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:54.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.335+0000 7f506bfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f506c144e30 0x7f506c143910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:54.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.335+0000 7f5070f2a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f506c0ab040 0x7f506c066a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48286/0 (socket says 172.21.15.47:48286) 2022-06-29T15:45:54.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.335+0000 7f5070f2a700 1 -- 172.21.15.47:0/2301770485 learned_addr learned my addr 172.21.15.47:0/2301770485 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:45:54.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.335+0000 7f5070f2a700 1 -- 172.21.15.47:0/2301770485 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f506c143e00 msgr2=0x7f506c1414a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:45:54.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.335+0000 7f5070f2a700 1 --2- 172.21.15.47:0/2301770485 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f506c143e00 0x7f506c1414a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:54.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.335+0000 7f5070f2a700 1 -- 172.21.15.47:0/2301770485 >> v2:172.21.15.64:3300/0 conn(0x7f506c144e30 msgr2=0x7f506c143910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:45:54.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.335+0000 7f5070f2a700 1 --2- 172.21.15.47:0/2301770485 >> v2:172.21.15.64:3300/0 conn(0x7f506c144e30 0x7f506c143910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:54.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.335+0000 7f5070f2a700 1 -- 172.21.15.47:0/2301770485 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f506c144270 con 0x7f506c0ab040 2022-06-29T15:45:54.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.335+0000 7f5070f2a700 1 --2- 172.21.15.47:0/2301770485 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f506c0ab040 0x7f506c066a80 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7f506c0670b0 tx=0x7f5058006eb0).ready entity=mon.0 client_cookie=766200eddecfd079 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:54.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.335+0000 7f506b7fe700 1 -- 172.21.15.47:0/2301770485 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f505800e610 con 0x7f506c0ab040 2022-06-29T15:45:54.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.335+0000 7f506b7fe700 1 -- 172.21.15.47:0/2301770485 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5058000e90 con 0x7f506c0ab040 2022-06-29T15:45:54.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.335+0000 7f506b7fe700 1 -- 172.21.15.47:0/2301770485 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f505800e9a0 con 0x7f506c0ab040 2022-06-29T15:45:54.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.335+0000 7f507318e700 1 -- 172.21.15.47:0/2301770485 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f506c0ab040 msgr2=0x7f506c066a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:54.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.335+0000 7f507318e700 1 --2- 172.21.15.47:0/2301770485 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f506c0ab040 0x7f506c066a80 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7f506c0670b0 tx=0x7f5058006eb0).stop 2022-06-29T15:45:54.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.335+0000 7f507318e700 1 -- 172.21.15.47:0/2301770485 shutdown_connections 2022-06-29T15:45:54.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.335+0000 7f507318e700 1 --2- 172.21.15.47:0/2301770485 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f506c143e00 0x7f506c1414a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:54.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.335+0000 7f507318e700 1 --2- 172.21.15.47:0/2301770485 >> v2:172.21.15.64:3300/0 conn(0x7f506c144e30 0x7f506c143910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:54.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.335+0000 7f507318e700 1 --2- 172.21.15.47:0/2301770485 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f506c0ab040 0x7f506c066a80 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:54.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.335+0000 7f507318e700 1 -- 172.21.15.47:0/2301770485 >> 172.21.15.47:0/2301770485 conn(0x7f506c05c510 msgr2=0x7f506c05c920 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:45:54.352 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.335+0000 7f507318e700 1 -- 172.21.15.47:0/2301770485 shutdown_connections 2022-06-29T15:45:54.352 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.335+0000 7f507318e700 1 -- 172.21.15.47:0/2301770485 wait complete. 2022-06-29T15:45:54.352 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.339+0000 7f507318e700 1 Processor -- start 2022-06-29T15:45:54.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.339+0000 7f507318e700 1 -- start start 2022-06-29T15:45:54.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.339+0000 7f507318e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f506c0ab040 0x7f506c13f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:54.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.339+0000 7f507318e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f506c143e00 0x7f506c139f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:54.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.339+0000 7f507318e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f506c144e30 0x7f506c13e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:54.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.339+0000 7f507318e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f506c0b5710 con 0x7f506c0ab040 2022-06-29T15:45:54.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.339+0000 7f507318e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f506c0b5db0 con 0x7f506c143e00 2022-06-29T15:45:54.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.339+0000 7f507318e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f506c136170 con 0x7f506c144e30 2022-06-29T15:45:54.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.339+0000 7f5070f2a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f506c0ab040 0x7f506c13f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:54.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.339+0000 7f5070f2a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f506c0ab040 0x7f506c13f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48290/0 (socket says 172.21.15.47:48290) 2022-06-29T15:45:54.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.339+0000 7f5070f2a700 1 -- 172.21.15.47:0/3587644019 learned_addr learned my addr 172.21.15.47:0/3587644019 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:45:54.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.339+0000 7f506bfff700 1 --2- 172.21.15.47:0/3587644019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f506c143e00 0x7f506c139f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:54.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.339+0000 7f507172b700 1 --2- 172.21.15.47:0/3587644019 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f506c144e30 0x7f506c13e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:54.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.339+0000 7f5070f2a700 1 -- 172.21.15.47:0/3587644019 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f506c144e30 msgr2=0x7f506c13e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:54.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.339+0000 7f5070f2a700 1 --2- 172.21.15.47:0/3587644019 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f506c144e30 0x7f506c13e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:54.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.339+0000 7f5070f2a700 1 -- 172.21.15.47:0/3587644019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f506c143e00 msgr2=0x7f506c139f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:54.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.339+0000 7f5070f2a700 1 --2- 172.21.15.47:0/3587644019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f506c143e00 0x7f506c139f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:54.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.339+0000 7f5070f2a700 1 -- 172.21.15.47:0/3587644019 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f505800b1e0 con 0x7f506c0ab040 2022-06-29T15:45:54.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.339+0000 7f507172b700 1 --2- 172.21.15.47:0/3587644019 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f506c144e30 0x7f506c13e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:45:54.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.339+0000 7f5070f2a700 1 --2- 172.21.15.47:0/3587644019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f506c0ab040 0x7f506c13f0d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7f506c0670b0 tx=0x7f5058015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:54.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.339+0000 7f5069ffb700 1 -- 172.21.15.47:0/3587644019 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5058021070 con 0x7f506c0ab040 2022-06-29T15:45:54.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.339+0000 7f5069ffb700 1 -- 172.21.15.47:0/3587644019 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5058002f50 con 0x7f506c0ab040 2022-06-29T15:45:54.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.339+0000 7f507318e700 1 -- 172.21.15.47:0/3587644019 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f506c0b5470 con 0x7f506c0ab040 2022-06-29T15:45:54.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.339+0000 7f5069ffb700 1 -- 172.21.15.47:0/3587644019 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5058011c30 con 0x7f506c0ab040 2022-06-29T15:45:54.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.339+0000 7f507318e700 1 -- 172.21.15.47:0/3587644019 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f506c0b6130 con 0x7f506c0ab040 2022-06-29T15:45:54.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.343+0000 7f507318e700 1 -- 172.21.15.47:0/3587644019 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f506c0573d0 con 0x7f506c0ab040 2022-06-29T15:45:54.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.343+0000 7f5069ffb700 1 -- 172.21.15.47:0/3587644019 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f505800f070 con 0x7f506c0ab040 2022-06-29T15:45:54.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.343+0000 7f5069ffb700 1 --2- 172.21.15.47:0/3587644019 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f504c053aa0 0x7f504c055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:54.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.343+0000 7f5069ffb700 1 -- 172.21.15.47:0/3587644019 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f505806d170 con 0x7f506c0ab040 2022-06-29T15:45:54.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.343+0000 7f506bfff700 1 --2- 172.21.15.47:0/3587644019 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f504c053aa0 0x7f504c055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:54.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.343+0000 7f506bfff700 1 --2- 172.21.15.47:0/3587644019 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f504c053aa0 0x7f504c055f60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f506000b000 tx=0x7f5060006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:54.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.351+0000 7f5069ffb700 1 -- 172.21.15.47:0/3587644019 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5058044170 con 0x7f506c0ab040 2022-06-29T15:45:54.522 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:54.519+0000 7f507318e700 1 -- 172.21.15.47:0/3587644019 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout"]} v 0) v1 -- 0x7f506c1365d0 con 0x7f506c0ab040 2022-06-29T15:45:55.210 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.207+0000 7f5069ffb700 1 -- 172.21.15.47:0/3587644019 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout"]}]=0 v157) v1 ==== 155+0+0 (secure 0 0 0) 0x7f5058047790 con 0x7f506c0ab040 2022-06-29T15:45:55.215 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.211+0000 7f507318e700 1 -- 172.21.15.47:0/3587644019 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f504c053aa0 msgr2=0x7f504c055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:55.215 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.211+0000 7f507318e700 1 --2- 172.21.15.47:0/3587644019 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f504c053aa0 0x7f504c055f60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f506000b000 tx=0x7f5060006df0).stop 2022-06-29T15:45:55.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.211+0000 7f507318e700 1 -- 172.21.15.47:0/3587644019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f506c0ab040 msgr2=0x7f506c13f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:55.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.211+0000 7f507318e700 1 --2- 172.21.15.47:0/3587644019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f506c0ab040 0x7f506c13f0d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7f506c0670b0 tx=0x7f5058015000).stop 2022-06-29T15:45:55.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.211+0000 7f507318e700 1 -- 172.21.15.47:0/3587644019 shutdown_connections 2022-06-29T15:45:55.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.211+0000 7f507318e700 1 --2- 172.21.15.47:0/3587644019 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f506c144e30 0x7f506c13e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:55.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.211+0000 7f507318e700 1 --2- 172.21.15.47:0/3587644019 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f504c053aa0 0x7f504c055f60 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:55.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.211+0000 7f507318e700 1 --2- 172.21.15.47:0/3587644019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f506c143e00 0x7f506c139f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:55.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.211+0000 7f507318e700 1 --2- 172.21.15.47:0/3587644019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f506c0ab040 0x7f506c13f0d0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:55.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.211+0000 7f507318e700 1 -- 172.21.15.47:0/3587644019 >> 172.21.15.47:0/3587644019 conn(0x7f506c05c510 msgr2=0x7f506c140600 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:45:55.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.211+0000 7f507318e700 1 -- 172.21.15.47:0/3587644019 shutdown_connections 2022-06-29T15:45:55.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.211+0000 7f507318e700 1 -- 172.21.15.47:0/3587644019 wait complete. 2022-06-29T15:45:55.230 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-06-29T15:45:55.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.335+0000 7f00074ab700 1 Processor -- start 2022-06-29T15:45:55.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.335+0000 7f00074ab700 1 -- start start 2022-06-29T15:45:55.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.335+0000 7f00074ab700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f000005f3f0 0x7f00001418c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:55.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.335+0000 7f00074ab700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f00000b1320 0x7f0000141db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:55.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.335+0000 7f00074ab700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00001422a0 con 0x7f00000aaf00 2022-06-29T15:45:55.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.335+0000 7f00074ab700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00000b27a0 con 0x7f00000b1320 2022-06-29T15:45:55.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.339+0000 7f00074ab700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f000005f7d0 con 0x7f000005f3f0 2022-06-29T15:45:55.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.339+0000 7f0005ca8700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f00000b1320 0x7f0000141db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:55.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.339+0000 7f0005ca8700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f00000b1320 0x7f0000141db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48300/0 (socket says 172.21.15.47:48300) 2022-06-29T15:45:55.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.339+0000 7f0005ca8700 1 -- 172.21.15.47:0/2659792598 learned_addr learned my addr 172.21.15.47:0/2659792598 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:45:55.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.339+0000 7f00064a9700 1 --2- 172.21.15.47:0/2659792598 >> v2:172.21.15.64:3300/0 conn(0x7f000005f3f0 0x7f00001418c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:55.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.339+0000 7f0005ca8700 1 -- 172.21.15.47:0/2659792598 >> v2:172.21.15.64:3300/0 conn(0x7f000005f3f0 msgr2=0x7f00001418c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:45:55.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.339+0000 7f0005ca8700 1 --2- 172.21.15.47:0/2659792598 >> v2:172.21.15.64:3300/0 conn(0x7f000005f3f0 0x7f00001418c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:55.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.339+0000 7f0005ca8700 1 -- 172.21.15.47:0/2659792598 >> v1:172.21.15.64:6789/0 conn(0x7f00000aaf00 legacy=0x7f00000b2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:55.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.339+0000 7f0005ca8700 1 -- 172.21.15.47:0/2659792598 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00000b4b90 con 0x7f00000b1320 2022-06-29T15:45:55.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.339+0000 7f0005ca8700 1 --2- 172.21.15.47:0/2659792598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00000b1320 0x7f0000141db0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7efff0002f70 tx=0x7efff0006eb0).ready entity=mon.0 client_cookie=f01202021c3f62f7 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:55.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.339+0000 7f00054a7700 1 -- 172.21.15.47:0/2659792598 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efff000d5c0 con 0x7f00000b1320 2022-06-29T15:45:55.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.339+0000 7f00054a7700 1 -- 172.21.15.47:0/2659792598 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efff0000e90 con 0x7f00000b1320 2022-06-29T15:45:55.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.339+0000 7f00054a7700 1 -- 172.21.15.47:0/2659792598 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efff000d950 con 0x7f00000b1320 2022-06-29T15:45:55.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.339+0000 7f00074ab700 1 -- 172.21.15.47:0/2659792598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00000b1320 msgr2=0x7f0000141db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:55.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.339+0000 7f00074ab700 1 --2- 172.21.15.47:0/2659792598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00000b1320 0x7f0000141db0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7efff0002f70 tx=0x7efff0006eb0).stop 2022-06-29T15:45:55.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.339+0000 7f00074ab700 1 -- 172.21.15.47:0/2659792598 shutdown_connections 2022-06-29T15:45:55.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.339+0000 7f00074ab700 1 --2- 172.21.15.47:0/2659792598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00000b1320 0x7f0000141db0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:55.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.339+0000 7f00074ab700 1 --2- 172.21.15.47:0/2659792598 >> v2:172.21.15.64:3300/0 conn(0x7f000005f3f0 0x7f00001418c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:55.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.339+0000 7f00074ab700 1 -- 172.21.15.47:0/2659792598 >> 172.21.15.47:0/2659792598 conn(0x7f0000062570 msgr2=0x7f00000647d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:45:55.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.339+0000 7f00074ab700 1 -- 172.21.15.47:0/2659792598 shutdown_connections 2022-06-29T15:45:55.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.339+0000 7f00074ab700 1 -- 172.21.15.47:0/2659792598 wait complete. 2022-06-29T15:45:55.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.343+0000 7f00074ab700 1 Processor -- start 2022-06-29T15:45:55.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.343+0000 7f00074ab700 1 -- start start 2022-06-29T15:45:55.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.343+0000 7f00074ab700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f000005f3f0 0x7f0000145cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:55.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.343+0000 7f00074ab700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00000aaf00 0x7f0000142780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:55.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.343+0000 7f00074ab700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f00000b1320 0x7f0000142c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:55.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.343+0000 7f00074ab700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0000146af0 con 0x7f00000aaf00 2022-06-29T15:45:55.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.343+0000 7f00074ab700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0000144530 con 0x7f000005f3f0 2022-06-29T15:45:55.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.343+0000 7f00074ab700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0000143ca0 con 0x7f00000b1320 2022-06-29T15:45:55.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.343+0000 7f0005ca8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00000aaf00 0x7f0000142780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:55.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.343+0000 7f00064a9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f000005f3f0 0x7f0000145cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:55.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.343+0000 7f0005ca8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00000aaf00 0x7f0000142780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48308/0 (socket says 172.21.15.47:48308) 2022-06-29T15:45:55.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.343+0000 7f00064a9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f000005f3f0 0x7f0000145cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:54008/0 (socket says 172.21.15.47:54008) 2022-06-29T15:45:55.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.343+0000 7f0005ca8700 1 -- 172.21.15.47:0/3638948061 learned_addr learned my addr 172.21.15.47:0/3638948061 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:45:55.370 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.343+0000 7f0006caa700 1 --2- 172.21.15.47:0/3638948061 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f00000b1320 0x7f0000142c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:55.370 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.343+0000 7f00064a9700 1 -- 172.21.15.47:0/3638948061 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f00000b1320 msgr2=0x7f0000142c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:55.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.343+0000 7f00064a9700 1 --2- 172.21.15.47:0/3638948061 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f00000b1320 0x7f0000142c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:55.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.343+0000 7f00064a9700 1 -- 172.21.15.47:0/3638948061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00000aaf00 msgr2=0x7f0000142780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:55.372 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.343+0000 7f00064a9700 1 --2- 172.21.15.47:0/3638948061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00000aaf00 0x7f0000142780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:55.372 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.343+0000 7f00064a9700 1 -- 172.21.15.47:0/3638948061 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efff000b1e0 con 0x7f000005f3f0 2022-06-29T15:45:55.372 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.343+0000 7f0005ca8700 1 --2- 172.21.15.47:0/3638948061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00000aaf00 0x7f0000142780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:45:55.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.343+0000 7f00064a9700 1 --2- 172.21.15.47:0/3638948061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f000005f3f0 0x7f0000145cb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7efffc00c000 tx=0x7efffc006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:55.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.343+0000 7effef7fe700 1 -- 172.21.15.47:0/3638948061 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efffc010780 con 0x7f000005f3f0 2022-06-29T15:45:55.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.343+0000 7effef7fe700 1 -- 172.21.15.47:0/3638948061 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efffc0108e0 con 0x7f000005f3f0 2022-06-29T15:45:55.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.343+0000 7f00074ab700 1 -- 172.21.15.47:0/3638948061 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0000145090 con 0x7f000005f3f0 2022-06-29T15:45:55.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.343+0000 7effef7fe700 1 -- 172.21.15.47:0/3638948061 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efffc010c40 con 0x7f000005f3f0 2022-06-29T15:45:55.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.343+0000 7f00074ab700 1 -- 172.21.15.47:0/3638948061 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00000ae010 con 0x7f000005f3f0 2022-06-29T15:45:55.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.347+0000 7f00074ab700 1 -- 172.21.15.47:0/3638948061 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00000573d0 con 0x7f000005f3f0 2022-06-29T15:45:55.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.347+0000 7effef7fe700 1 -- 172.21.15.47:0/3638948061 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7efffc01b400 con 0x7f000005f3f0 2022-06-29T15:45:55.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.347+0000 7effef7fe700 1 --2- 172.21.15.47:0/3638948061 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7effe0053a70 0x7effe0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:55.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.347+0000 7effef7fe700 1 -- 172.21.15.47:0/3638948061 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5686+0+0 (secure 0 0 0) 0x7efffc06be20 con 0x7f000005f3f0 2022-06-29T15:45:55.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.347+0000 7f0005ca8700 1 --2- 172.21.15.47:0/3638948061 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7effe0053a70 0x7effe0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:55.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.347+0000 7f0005ca8700 1 --2- 172.21.15.47:0/3638948061 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7effe0053a70 0x7effe0055f30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7efff001c000 tx=0x7efff0015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:55.379 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.359+0000 7effef7fe700 1 -- 172.21.15.47:0/3638948061 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7efffc042dc0 con 0x7f000005f3f0 2022-06-29T15:45:55.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.491+0000 7f00074ab700 1 -- 172.21.15.47:0/3638948061 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0000066040 con 0x7f000005f3f0 2022-06-29T15:45:55.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.499+0000 7effef7fe700 1 -- 172.21.15.47:0/3638948061 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v6)=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7efffc0463e0 con 0x7f000005f3f0 2022-06-29T15:45:55.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.503+0000 7f00074ab700 1 -- 172.21.15.47:0/3638948061 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7effe0053a70 msgr2=0x7effe0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:55.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.503+0000 7f00074ab700 1 --2- 172.21.15.47:0/3638948061 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7effe0053a70 0x7effe0055f30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7efff001c000 tx=0x7efff0015000).stop 2022-06-29T15:45:55.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.503+0000 7f00074ab700 1 -- 172.21.15.47:0/3638948061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f000005f3f0 msgr2=0x7f0000145cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:55.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.503+0000 7f00074ab700 1 --2- 172.21.15.47:0/3638948061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f000005f3f0 0x7f0000145cb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7efffc00c000 tx=0x7efffc006df0).stop 2022-06-29T15:45:55.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.503+0000 7f00074ab700 1 -- 172.21.15.47:0/3638948061 shutdown_connections 2022-06-29T15:45:55.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.503+0000 7f00074ab700 1 --2- 172.21.15.47:0/3638948061 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f00000b1320 0x7f0000142c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:55.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.503+0000 7f00074ab700 1 --2- 172.21.15.47:0/3638948061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00000aaf00 0x7f0000142780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:55.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.503+0000 7f00074ab700 1 --2- 172.21.15.47:0/3638948061 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7effe0053a70 0x7effe0055f30 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:55.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.503+0000 7f00074ab700 1 --2- 172.21.15.47:0/3638948061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f000005f3f0 0x7f0000145cb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:55.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.503+0000 7f00074ab700 1 -- 172.21.15.47:0/3638948061 >> 172.21.15.47:0/3638948061 conn(0x7f0000062570 msgr2=0x7f00000647d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:45:55.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.503+0000 7f00074ab700 1 -- 172.21.15.47:0/3638948061 shutdown_connections 2022-06-29T15:45:55.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.503+0000 7f00074ab700 1 -- 172.21.15.47:0/3638948061 wait complete. 2022-06-29T15:45:55.522 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-06-29T15:45:55.523 DEBUG:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:45:55.552 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:45:55.574 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:45:55.596 DEBUG:teuthology.orchestra.run.smithi164:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:45:55.618 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-06-29T15:45:55.618 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:45:55.619 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:45:55.619 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:45:55.627 INFO:teuthology.orchestra.run.smithi099.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-06-29T15:45:55.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T15:45:55.628 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-06-29T15:45:55.629 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:45:55.630 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:45:55.630 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:45:55.638 INFO:teuthology.orchestra.run.smithi164.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-06-29T15:45:55.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T15:45:55.640 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-06-29T15:45:55.640 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-06-29T15:45:55.753 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.747+0000 7f4069ee0700 1 Processor -- start 2022-06-29T15:45:55.753 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.751+0000 7f4069ee0700 1 -- start start 2022-06-29T15:45:55.756 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.751+0000 7f4069ee0700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f40640b2120 0x7f40640b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:55.756 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.751+0000 7f4069ee0700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40640b0740 con 0x7f40640b1260 2022-06-29T15:45:55.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.751+0000 7f4069ee0700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40640b0970 con 0x7f406405f0b0 2022-06-29T15:45:55.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.751+0000 7f4069ee0700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4064147460 con 0x7f40640b2120 2022-06-29T15:45:55.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.751+0000 7f40637fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f40640b2120 0x7f40640b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:55.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.751+0000 7f4063fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f40640b1260 0x7f406405f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33870/0 (socket says 172.21.15.47:33870) 2022-06-29T15:45:55.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.751+0000 7f4063fff700 1 -- 172.21.15.47:0/961884921 learned_addr learned my addr 172.21.15.47:0/961884921 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:45:55.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.751+0000 7f40627fc700 1 -- 172.21.15.47:0/961884921 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2794191490 0 0) 0x7f40640b0740 con 0x7f40640b1260 2022-06-29T15:45:55.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.751+0000 7f40627fc700 1 -- 172.21.15.47:0/961884921 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4050003260 con 0x7f40640b1260 2022-06-29T15:45:55.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.751+0000 7f40637fe700 1 -- 172.21.15.47:0/961884921 >> v1:172.21.15.64:6789/0 conn(0x7f406405f0b0 legacy=0x7f4064147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:55.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.751+0000 7f40627fc700 1 -- 172.21.15.47:0/961884921 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2459073105 0 0) 0x7f40640b0970 con 0x7f406405f0b0 2022-06-29T15:45:55.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.751+0000 7f40637fe700 1 -- 172.21.15.47:0/961884921 >> v1:172.21.15.47:6789/0 conn(0x7f40640b1260 legacy=0x7f406405f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:55.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.751+0000 7f40637fe700 1 -- 172.21.15.47:0/961884921 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40640b0c10 con 0x7f40640b2120 2022-06-29T15:45:55.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.751+0000 7f40637fe700 1 --2- 172.21.15.47:0/961884921 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40640b2120 0x7f40640b2500 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7f40640affe0 tx=0x7f4058006df0).ready entity=mon.0 client_cookie=9591c52c540b2954 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:55.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.751+0000 7f40627fc700 1 -- 172.21.15.47:0/961884921 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f405800d610 con 0x7f40640b2120 2022-06-29T15:45:55.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.751+0000 7f40627fc700 1 -- 172.21.15.47:0/961884921 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4058000e90 con 0x7f40640b2120 2022-06-29T15:45:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.751+0000 7f40627fc700 1 -- 172.21.15.47:0/961884921 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f405800d960 con 0x7f40640b2120 2022-06-29T15:45:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.751+0000 7f4069ee0700 1 -- 172.21.15.47:0/961884921 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40640b2120 msgr2=0x7f40640b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.751+0000 7f4069ee0700 1 --2- 172.21.15.47:0/961884921 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40640b2120 0x7f40640b2500 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7f40640affe0 tx=0x7f4058006df0).stop 2022-06-29T15:45:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.751+0000 7f4069ee0700 1 -- 172.21.15.47:0/961884921 shutdown_connections 2022-06-29T15:45:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.751+0000 7f4069ee0700 1 --2- 172.21.15.47:0/961884921 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40640b2120 0x7f40640b2500 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.751+0000 7f4069ee0700 1 -- 172.21.15.47:0/961884921 >> 172.21.15.47:0/961884921 conn(0x7f40640626f0 msgr2=0x7f4064062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:45:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.751+0000 7f4069ee0700 1 -- 172.21.15.47:0/961884921 shutdown_connections 2022-06-29T15:45:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.751+0000 7f4069ee0700 1 -- 172.21.15.47:0/961884921 wait complete. 2022-06-29T15:45:55.768 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.755+0000 7f4069ee0700 1 Processor -- start 2022-06-29T15:45:55.768 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.755+0000 7f4069ee0700 1 -- start start 2022-06-29T15:45:55.768 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.755+0000 7f4069ee0700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f40640b1260 0x7f40640aec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:55.769 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.755+0000 7f4069ee0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40640b2120 0x7f4064144d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:55.769 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.755+0000 7f4069ee0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4064159db0 0x7f406415a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:55.769 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.755+0000 7f4069ee0700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406414e560 con 0x7f40640b2120 2022-06-29T15:45:55.770 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.755+0000 7f4069ee0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406414d950 con 0x7f4064159db0 2022-06-29T15:45:55.770 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.755+0000 7f4069ee0700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f406414db80 con 0x7f40640b1260 2022-06-29T15:45:55.770 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.759+0000 7f4062ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40640b2120 0x7f4064144d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:55.770 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.759+0000 7f4062ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40640b2120 0x7f4064144d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48320/0 (socket says 172.21.15.47:48320) 2022-06-29T15:45:55.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.759+0000 7f4062ffd700 1 -- 172.21.15.47:0/1012089555 learned_addr learned my addr 172.21.15.47:0/1012089555 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:45:55.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.759+0000 7f4063fff700 1 --2- 172.21.15.47:0/1012089555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4064159db0 0x7f406415a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:55.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.759+0000 7f40637fe700 1 --2- 172.21.15.47:0/1012089555 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f40640b1260 0x7f40640aec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:55.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.759+0000 7f4062ffd700 1 -- 172.21.15.47:0/1012089555 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f40640b1260 msgr2=0x7f40640aec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:55.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.759+0000 7f4062ffd700 1 --2- 172.21.15.47:0/1012089555 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f40640b1260 0x7f40640aec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:55.772 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.759+0000 7f4062ffd700 1 -- 172.21.15.47:0/1012089555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4064159db0 msgr2=0x7f406415a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:55.772 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.759+0000 7f4062ffd700 1 --2- 172.21.15.47:0/1012089555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4064159db0 0x7f406415a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:55.772 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.759+0000 7f4062ffd700 1 -- 172.21.15.47:0/1012089555 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f405800b180 con 0x7f40640b2120 2022-06-29T15:45:55.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.759+0000 7f4062ffd700 1 --2- 172.21.15.47:0/1012089555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40640b2120 0x7f4064144d90 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7f4054000ed0 tx=0x7f4054007a90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:55.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.759+0000 7f4060ff9700 1 -- 172.21.15.47:0/1012089555 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4054012400 con 0x7f40640b2120 2022-06-29T15:45:55.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.759+0000 7f4060ff9700 1 -- 172.21.15.47:0/1012089555 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4054012560 con 0x7f40640b2120 2022-06-29T15:45:55.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.759+0000 7f4069ee0700 1 -- 172.21.15.47:0/1012089555 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f406414de20 con 0x7f40640b2120 2022-06-29T15:45:55.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.759+0000 7f4060ff9700 1 -- 172.21.15.47:0/1012089555 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40540128c0 con 0x7f40640b2120 2022-06-29T15:45:55.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.759+0000 7f4069ee0700 1 -- 172.21.15.47:0/1012089555 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4064145280 con 0x7f40640b2120 2022-06-29T15:45:55.776 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.759+0000 7f4069ee0700 1 -- 172.21.15.47:0/1012089555 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4064147650 con 0x7f40640b2120 2022-06-29T15:45:55.776 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.759+0000 7f4060ff9700 1 -- 172.21.15.47:0/1012089555 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4054019030 con 0x7f40640b2120 2022-06-29T15:45:55.776 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.759+0000 7f4060ff9700 1 --2- 172.21.15.47:0/1012089555 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4044053b00 0x7f4044055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:55.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.759+0000 7f4060ff9700 1 -- 172.21.15.47:0/1012089555 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f405406b0b0 con 0x7f40640b2120 2022-06-29T15:45:55.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.763+0000 7f40637fe700 1 --2- 172.21.15.47:0/1012089555 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4044053b00 0x7f4044055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:55.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.763+0000 7f40637fe700 1 --2- 172.21.15.47:0/1012089555 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4044053b00 0x7f4044055fc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f4058018000 tx=0x7f4058014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:55.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.767+0000 7f4060ff9700 1 -- 172.21.15.47:0/1012089555 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4054042100 con 0x7f40640b2120 2022-06-29T15:45:55.899 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.895+0000 7f4069ee0700 1 -- 172.21.15.47:0/1012089555 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f40640573d0 con 0x7f40640b2120 2022-06-29T15:45:55.901 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.899+0000 7f4060ff9700 1 -- 172.21.15.47:0/1012089555 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v50) v1 ==== 81+0+29219 (secure 0 0 0) 0x7f4054028240 con 0x7f40640b2120 2022-06-29T15:45:55.903 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:45:55.903 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2022-06-29T15:45:55.904 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 50, 2022-06-29T15:45:55.904 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "9a68894a-53ce-400e-838b-14eab91b834c", 2022-06-29T15:45:55.905 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2022-06-29T15:42:14.046361+0000", 2022-06-29T15:45:55.905 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2022-06-29T15:45:53.188419+0000", 2022-06-29T15:45:55.906 INFO:teuthology.orchestra.run.smithi047.stdout: "last_up_change": "2022-06-29T15:42:20.929789+0000", 2022-06-29T15:45:55.906 INFO:teuthology.orchestra.run.smithi047.stdout: "last_in_change": "2022-06-29T15:42:20.929789+0000", 2022-06-29T15:45:55.907 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-06-29T15:45:55.907 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_num": 5799936, 2022-06-29T15:45:55.908 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_set": [ 2022-06-29T15:45:55.908 INFO:teuthology.orchestra.run.smithi047.stdout: "pglog_hardlimit", 2022-06-29T15:45:55.909 INFO:teuthology.orchestra.run.smithi047.stdout: "purged_snapdirs", 2022-06-29T15:45:55.909 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_deletes", 2022-06-29T15:45:55.910 INFO:teuthology.orchestra.run.smithi047.stdout: "sortbitwise" 2022-06-29T15:45:55.910 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:45:55.910 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_version": 5, 2022-06-29T15:45:55.911 INFO:teuthology.orchestra.run.smithi047.stdout: "full_ratio": 0.94999998807907104, 2022-06-29T15:45:55.911 INFO:teuthology.orchestra.run.smithi047.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-06-29T15:45:55.912 INFO:teuthology.orchestra.run.smithi047.stdout: "nearfull_ratio": 0.85000002384185791, 2022-06-29T15:45:55.912 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_snapshot": "", 2022-06-29T15:45:55.913 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_max": 8, 2022-06-29T15:45:55.913 INFO:teuthology.orchestra.run.smithi047.stdout: "max_osd": 8, 2022-06-29T15:45:55.914 INFO:teuthology.orchestra.run.smithi047.stdout: "require_min_compat_client": "jewel", 2022-06-29T15:45:55.914 INFO:teuthology.orchestra.run.smithi047.stdout: "min_compat_client": "jewel", 2022-06-29T15:45:55.915 INFO:teuthology.orchestra.run.smithi047.stdout: "require_osd_release": "octopus", 2022-06-29T15:45:55.915 INFO:teuthology.orchestra.run.smithi047.stdout: "pools": [ 2022-06-29T15:45:55.915 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:55.916 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 1, 2022-06-29T15:45:55.916 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "device_health_metrics", 2022-06-29T15:45:55.917 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:42:16.971431+0000", 2022-06-29T15:45:55.917 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:45:55.918 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:45:55.918 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:45:55.919 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:45:55.920 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:45:55.920 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:45:55.920 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:45:55.921 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:45:55.921 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 1, 2022-06-29T15:45:55.922 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 1, 2022-06-29T15:45:55.922 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 1, 2022-06-29T15:45:55.923 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 1, 2022-06-29T15:45:55.923 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 1, 2022-06-29T15:45:55.924 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:45:55.924 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:45:55.925 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:45:55.925 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:45:55.925 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:45:55.926 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:45:55.926 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:45:55.927 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:55.927 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "16", 2022-06-29T15:45:55.928 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:45:55.928 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:45:55.929 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:45:55.929 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:45:55.930 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:45:55.930 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:45:55.931 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:45:55.931 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:45:55.931 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:45:55.932 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:45:55.932 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:45:55.933 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:45:55.933 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:45:55.934 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:45:55.934 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:45:55.935 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:45:55.935 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:45:55.936 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:45:55.936 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:45:55.937 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:45:55.937 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:45:55.938 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:45:55.938 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:45:55.939 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:45:55.939 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:45:55.940 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:45:55.940 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:55.941 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:45:55.941 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:45:55.942 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:45:55.942 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:45:55.942 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:45:55.943 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:45:55.943 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:45:55.944 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:45:55.944 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:45:55.945 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:45:55.945 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:45:55.946 INFO:teuthology.orchestra.run.smithi047.stdout: "options": { 2022-06-29T15:45:55.946 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_min": 1 2022-06-29T15:45:55.947 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:55.947 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:45:55.948 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_devicehealth": {} 2022-06-29T15:45:55.948 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:55.949 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:55.949 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:55.949 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 2, 2022-06-29T15:45:55.950 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "rbd", 2022-06-29T15:45:55.950 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:42:25.290884+0000", 2022-06-29T15:45:55.951 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:45:55.951 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:45:55.952 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:45:55.952 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:45:55.953 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:45:55.953 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:45:55.954 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:45:55.954 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:45:55.955 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2022-06-29T15:45:55.955 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2022-06-29T15:45:55.955 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2022-06-29T15:45:55.956 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2022-06-29T15:45:55.956 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2022-06-29T15:45:55.957 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:45:55.957 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:45:55.958 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:45:55.958 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:45:55.959 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:45:55.959 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:45:55.960 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:45:55.960 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:55.960 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "19", 2022-06-29T15:45:55.961 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:45:55.961 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:45:55.962 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:45:55.962 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:45:55.963 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:45:55.963 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:45:55.963 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:45:55.964 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:45:55.964 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:45:55.964 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:45:55.964 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:45:55.965 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:45:55.965 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:45:55.965 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:45:55.965 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:45:55.965 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:45:55.966 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:45:55.966 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:45:55.966 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:45:55.967 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:45:55.967 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:45:55.967 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:45:55.967 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:45:55.968 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:45:55.968 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:45:55.968 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:45:55.968 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:55.968 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:45:55.969 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:45:55.969 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:45:55.969 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:45:55.969 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:45:55.970 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:45:55.970 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:45:55.970 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:45:55.970 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:45:55.971 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:45:55.971 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:45:55.971 INFO:teuthology.orchestra.run.smithi047.stdout: "options": {}, 2022-06-29T15:45:55.971 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:45:55.972 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd": {} 2022-06-29T15:45:55.972 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:55.973 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:55.973 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:45:55.973 INFO:teuthology.orchestra.run.smithi047.stdout: "osds": [ 2022-06-29T15:45:55.973 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:55.974 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2022-06-29T15:45:55.974 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "e23eb023-4706-4201-bf72-4bf19c912c5d", 2022-06-29T15:45:55.974 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:45:55.975 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:45:55.975 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:45:55.976 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:45:55.976 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:45:55.977 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:45:55.977 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:45:55.977 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 45, 2022-06-29T15:45:55.978 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:45:55.978 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:45:55.978 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:45:55.979 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:55.979 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:55.979 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:55.979 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6818", 2022-06-29T15:45:55.980 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:45:55.980 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:55.980 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:55.981 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:55.981 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6819", 2022-06-29T15:45:55.981 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:45:55.981 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:55.982 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:55.982 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:55.982 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:45:55.982 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:55.983 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:55.983 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:55.983 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6820", 2022-06-29T15:45:55.983 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:45:55.983 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:55.984 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:55.984 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:55.984 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6821", 2022-06-29T15:45:55.984 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:45:55.985 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:55.985 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:55.985 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:55.985 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:45:55.986 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:55.986 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:55.986 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:55.986 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6824", 2022-06-29T15:45:55.986 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:45:55.987 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:55.987 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:55.987 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:55.987 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6825", 2022-06-29T15:45:55.987 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:45:55.988 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:55.988 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:55.988 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:55.989 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:45:55.989 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:55.989 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:55.989 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:55.990 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6822", 2022-06-29T15:45:55.990 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:45:55.990 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:55.990 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:55.990 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:55.991 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6823", 2022-06-29T15:45:55.991 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:45:55.991 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:55.991 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:55.992 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:55.992 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6819/13096", 2022-06-29T15:45:55.992 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6821/13096", 2022-06-29T15:45:55.992 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6825/13096", 2022-06-29T15:45:55.992 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6823/13096", 2022-06-29T15:45:55.993 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:45:55.993 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:45:55.993 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:45:55.993 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:55.993 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:55.994 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:55.994 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2022-06-29T15:45:55.994 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "228bb08d-2d64-43d9-9560-bfaeff154452", 2022-06-29T15:45:55.994 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:45:55.995 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:45:55.995 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:45:55.995 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:45:55.995 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:45:55.995 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:45:55.996 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:45:55.996 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 45, 2022-06-29T15:45:55.996 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:45:55.996 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:45:55.997 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:45:55.997 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:55.997 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:55.997 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:55.998 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6826", 2022-06-29T15:45:55.998 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:45:55.998 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:55.998 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:55.999 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:55.999 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6827", 2022-06-29T15:45:55.999 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:45:55.999 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:56.000 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:56.000 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.000 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:45:56.000 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:56.000 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.001 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:56.001 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6828", 2022-06-29T15:45:56.001 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:45:56.001 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.002 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.002 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:56.002 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6829", 2022-06-29T15:45:56.002 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:45:56.002 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:56.003 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:56.003 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.003 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:45:56.003 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:56.004 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.004 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:56.004 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6832", 2022-06-29T15:45:56.004 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:45:56.004 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.005 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.005 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:56.005 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6833", 2022-06-29T15:45:56.005 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:45:56.006 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:56.006 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:56.006 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.006 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:45:56.007 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:56.007 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.007 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:56.007 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6830", 2022-06-29T15:45:56.008 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:45:56.008 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.008 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.008 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:56.008 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6831", 2022-06-29T15:45:56.009 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:45:56.009 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:56.009 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:56.009 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.010 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6827/13100", 2022-06-29T15:45:56.010 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6829/13100", 2022-06-29T15:45:56.010 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6833/13100", 2022-06-29T15:45:56.010 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6831/13100", 2022-06-29T15:45:56.010 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:45:56.011 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:45:56.011 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:45:56.011 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:56.011 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.012 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.012 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2022-06-29T15:45:56.012 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "797a5452-1510-4640-a111-c9c2fcf707eb", 2022-06-29T15:45:56.012 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:45:56.012 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:45:56.013 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:45:56.013 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:45:56.013 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:45:56.013 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:45:56.013 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:45:56.014 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 33, 2022-06-29T15:45:56.014 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:45:56.014 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:45:56.014 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:45:56.015 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:56.015 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.015 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:56.015 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6810", 2022-06-29T15:45:56.016 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:45:56.016 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.016 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.016 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:56.016 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6811", 2022-06-29T15:45:56.017 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:45:56.017 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:56.017 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:56.017 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.018 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:45:56.018 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:56.018 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.018 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:56.018 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6812", 2022-06-29T15:45:56.019 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:45:56.019 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.019 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.019 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:56.020 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6813", 2022-06-29T15:45:56.020 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:45:56.020 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:56.020 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:56.020 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.021 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:45:56.021 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:56.021 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.021 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:56.021 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6816", 2022-06-29T15:45:56.022 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:45:56.022 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.022 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.022 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:56.023 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6817", 2022-06-29T15:45:56.023 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:45:56.023 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:56.023 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:56.024 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.024 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:45:56.024 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:56.024 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.025 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:56.025 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6814", 2022-06-29T15:45:56.025 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:45:56.025 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.025 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.026 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:56.026 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6815", 2022-06-29T15:45:56.026 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:45:56.026 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:56.027 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:56.027 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.027 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6811/13098", 2022-06-29T15:45:56.027 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6813/13098", 2022-06-29T15:45:56.027 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6817/13098", 2022-06-29T15:45:56.028 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6815/13098", 2022-06-29T15:45:56.028 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:45:56.028 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:45:56.028 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:45:56.029 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:56.029 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.029 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.029 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2022-06-29T15:45:56.029 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "55b24fd7-bb71-4a01-b3fa-a09caa772a62", 2022-06-29T15:45:56.030 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:45:56.030 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:45:56.030 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:45:56.030 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:45:56.031 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:45:56.031 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:45:56.031 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:45:56.031 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 45, 2022-06-29T15:45:56.031 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:45:56.032 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:45:56.032 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:45:56.032 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:56.032 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.033 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:56.033 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6802", 2022-06-29T15:45:56.033 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:45:56.033 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.034 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.034 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:56.034 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6803", 2022-06-29T15:45:56.034 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:45:56.035 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:56.035 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:56.035 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.035 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:45:56.035 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:56.036 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.036 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:56.036 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6804", 2022-06-29T15:45:56.036 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:45:56.036 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.037 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.037 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:56.037 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6805", 2022-06-29T15:45:56.037 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:45:56.038 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:56.038 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:56.038 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.038 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:45:56.038 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:56.039 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.039 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:56.039 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6808", 2022-06-29T15:45:56.039 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:45:56.040 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.040 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.040 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:56.040 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6809", 2022-06-29T15:45:56.040 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:45:56.041 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:56.041 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:56.041 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.041 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:45:56.041 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:56.042 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.042 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:56.042 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6806", 2022-06-29T15:45:56.042 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:45:56.043 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.043 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.043 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:56.043 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6807", 2022-06-29T15:45:56.043 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:45:56.044 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:56.044 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:56.044 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.044 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6803/13097", 2022-06-29T15:45:56.045 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6805/13097", 2022-06-29T15:45:56.045 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6809/13097", 2022-06-29T15:45:56.045 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6807/13097", 2022-06-29T15:45:56.045 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:45:56.045 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:45:56.046 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:45:56.046 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:56.046 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.046 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.047 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2022-06-29T15:45:56.047 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3", 2022-06-29T15:45:56.047 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:45:56.047 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:45:56.047 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:45:56.048 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:45:56.048 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:45:56.048 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:45:56.048 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:45:56.048 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 45, 2022-06-29T15:45:56.049 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:45:56.049 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:45:56.049 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:45:56.049 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:56.049 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.050 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:56.050 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6816", 2022-06-29T15:45:56.050 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:45:56.050 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.051 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.051 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:56.051 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6817", 2022-06-29T15:45:56.051 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:45:56.051 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:56.052 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:56.052 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.052 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:45:56.052 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:56.053 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.053 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:56.053 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6818", 2022-06-29T15:45:56.053 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:45:56.053 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.054 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.054 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:56.054 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6819", 2022-06-29T15:45:56.054 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:45:56.055 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:56.055 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:56.055 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.055 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:45:56.055 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:56.056 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.056 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:56.056 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6822", 2022-06-29T15:45:56.056 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:45:56.057 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.057 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.057 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:56.057 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6823", 2022-06-29T15:45:56.057 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:45:56.058 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:56.058 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:56.058 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.058 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:45:56.059 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:56.059 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.059 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:56.059 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6820", 2022-06-29T15:45:56.059 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:45:56.060 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.060 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.060 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:56.060 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6821", 2022-06-29T15:45:56.061 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:45:56.061 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:56.061 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:56.061 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.061 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6817/13396", 2022-06-29T15:45:56.062 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6819/13396", 2022-06-29T15:45:56.062 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6823/13396", 2022-06-29T15:45:56.062 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6821/13396", 2022-06-29T15:45:56.062 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:45:56.063 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:45:56.063 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:45:56.063 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:56.063 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.063 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.064 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2022-06-29T15:45:56.064 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "f77beebe-6cca-4244-b2d4-677825ebf919", 2022-06-29T15:45:56.064 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:45:56.064 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:45:56.065 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:45:56.065 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:45:56.065 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:45:56.065 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:45:56.065 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:45:56.066 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 45, 2022-06-29T15:45:56.066 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:45:56.066 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:45:56.066 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:45:56.066 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:56.067 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.067 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:56.067 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6800", 2022-06-29T15:45:56.067 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:45:56.068 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.068 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.068 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:56.068 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6801", 2022-06-29T15:45:56.068 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:45:56.069 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:56.069 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:56.069 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.069 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:45:56.070 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:56.070 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.070 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:56.070 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6802", 2022-06-29T15:45:56.071 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:45:56.071 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.071 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.071 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:56.071 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6803", 2022-06-29T15:45:56.072 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:45:56.072 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:56.072 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:56.072 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.072 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:45:56.073 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:56.073 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.073 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:56.073 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6806", 2022-06-29T15:45:56.073 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:45:56.074 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.074 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.074 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:56.074 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6807", 2022-06-29T15:45:56.075 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:45:56.075 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:56.075 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:56.075 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.076 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:45:56.076 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:56.076 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.076 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:56.076 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6804", 2022-06-29T15:45:56.077 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:45:56.077 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.077 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.077 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:56.078 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6805", 2022-06-29T15:45:56.078 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:45:56.078 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:56.078 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:56.079 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.079 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6801/13393", 2022-06-29T15:45:56.079 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6803/13393", 2022-06-29T15:45:56.079 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6807/13393", 2022-06-29T15:45:56.080 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6805/13393", 2022-06-29T15:45:56.080 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:45:56.080 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:45:56.080 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:45:56.080 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:56.081 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.081 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.081 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2022-06-29T15:45:56.081 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "af0ca9b3-c02c-4910-834b-bf0b8dc6713c", 2022-06-29T15:45:56.082 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:45:56.082 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:45:56.082 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:45:56.082 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:45:56.082 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:45:56.083 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:45:56.083 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:45:56.083 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 45, 2022-06-29T15:45:56.083 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:45:56.083 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:45:56.084 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:45:56.084 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:56.084 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.084 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:56.085 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6824", 2022-06-29T15:45:56.085 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:45:56.085 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.085 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.085 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:56.086 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6825", 2022-06-29T15:45:56.086 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:45:56.086 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:56.086 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:56.087 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.087 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:45:56.087 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:56.088 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.088 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:56.088 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6826", 2022-06-29T15:45:56.088 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:45:56.089 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.089 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.089 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:56.089 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6827", 2022-06-29T15:45:56.089 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:45:56.090 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:56.090 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:56.090 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.090 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:45:56.091 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:56.091 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.099 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:56.099 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6830", 2022-06-29T15:45:56.099 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:45:56.099 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.100 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.100 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:56.100 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6831", 2022-06-29T15:45:56.100 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:45:56.101 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:56.101 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:56.101 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.101 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:45:56.101 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:56.102 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.102 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:56.102 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6828", 2022-06-29T15:45:56.103 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:45:56.103 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.103 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.103 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:56.103 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6829", 2022-06-29T15:45:56.104 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:45:56.104 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:56.104 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:56.104 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.105 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6825/13395", 2022-06-29T15:45:56.105 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6827/13395", 2022-06-29T15:45:56.105 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6831/13395", 2022-06-29T15:45:56.105 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6829/13395", 2022-06-29T15:45:56.106 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:45:56.106 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:45:56.106 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:45:56.106 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:56.107 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.107 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.107 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2022-06-29T15:45:56.107 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "f19c3ca6-df88-4bde-a150-b51a53f092e1", 2022-06-29T15:45:56.107 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:45:56.108 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:45:56.108 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:45:56.108 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:45:56.108 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:45:56.109 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:45:56.109 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:45:56.109 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 44, 2022-06-29T15:45:56.110 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:45:56.110 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:45:56.110 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:45:56.111 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:56.111 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.111 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:56.111 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6808", 2022-06-29T15:45:56.111 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:45:56.112 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.112 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.112 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:56.112 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6809", 2022-06-29T15:45:56.113 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:45:56.113 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:56.113 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:56.113 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.114 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:45:56.114 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:56.114 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.115 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:56.115 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6810", 2022-06-29T15:45:56.115 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:45:56.115 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.115 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.116 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:56.116 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6811", 2022-06-29T15:45:56.116 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:45:56.116 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:56.116 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:56.117 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.117 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:45:56.117 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:56.117 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.118 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:56.118 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6814", 2022-06-29T15:45:56.118 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:45:56.118 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.118 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.119 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:56.119 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6815", 2022-06-29T15:45:56.119 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:45:56.119 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:56.120 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:56.120 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.120 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:45:56.121 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:45:56.121 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.121 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:45:56.121 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6812", 2022-06-29T15:45:56.122 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:45:56.122 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.122 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.122 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:45:56.123 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6813", 2022-06-29T15:45:56.123 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:45:56.123 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:56.123 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:56.124 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.124 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6809/13394", 2022-06-29T15:45:56.124 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6811/13394", 2022-06-29T15:45:56.124 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6815/13394", 2022-06-29T15:45:56.125 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6813/13394", 2022-06-29T15:45:56.125 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:45:56.125 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:45:56.125 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:45:56.125 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:56.126 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:56.126 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:45:56.126 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_xinfo": [ 2022-06-29T15:45:56.127 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.127 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2022-06-29T15:45:56.127 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:45:56.127 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:45:56.128 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:45:56.128 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:45:56.128 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:45:56.128 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.720513+0000", 2022-06-29T15:45:56.128 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:45:56.129 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.129 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.129 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2022-06-29T15:45:56.129 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:45:56.130 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:45:56.130 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:45:56.130 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:45:56.130 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:45:56.131 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.772612+0000", 2022-06-29T15:45:56.131 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:45:56.131 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.132 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.132 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2022-06-29T15:45:56.132 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:45:56.132 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:45:56.133 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:45:56.133 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:45:56.133 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:45:56.133 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.450223+0000", 2022-06-29T15:45:56.134 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:45:56.134 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.134 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.134 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2022-06-29T15:45:56.135 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:45:56.135 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:45:56.135 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:45:56.135 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:45:56.135 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:45:56.136 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.477856+0000", 2022-06-29T15:45:56.136 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:45:56.136 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.136 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.136 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2022-06-29T15:45:56.137 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:45:56.137 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:45:56.137 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:45:56.137 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:45:56.138 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:45:56.138 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.539485+0000", 2022-06-29T15:45:56.138 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:45:56.138 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.138 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.139 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2022-06-29T15:45:56.139 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:45:56.139 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:45:56.139 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:45:56.140 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:45:56.140 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:45:56.140 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.487040+0000", 2022-06-29T15:45:56.140 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:45:56.141 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.141 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.141 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2022-06-29T15:45:56.142 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:45:56.142 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:45:56.142 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:45:56.142 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:45:56.143 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:45:56.143 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.748685+0000", 2022-06-29T15:45:56.143 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:45:56.143 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.143 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:56.144 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2022-06-29T15:45:56.144 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:45:56.144 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:45:56.145 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:45:56.145 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:45:56.145 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:45:56.145 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.459516+0000", 2022-06-29T15:45:56.145 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:45:56.146 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:56.146 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:45:56.146 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap": [], 2022-06-29T15:45:56.146 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap_items": [], 2022-06-29T15:45:56.147 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_temp": [], 2022-06-29T15:45:56.147 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_temp": [], 2022-06-29T15:45:56.147 INFO:teuthology.orchestra.run.smithi047.stdout: "blacklist": { 2022-06-29T15:45:56.147 INFO:teuthology.orchestra.run.smithi047.stdout: "172.21.15.64:6833/2497658100": "2022-06-30T15:45:49.337800+0000", 2022-06-29T15:45:56.147 INFO:teuthology.orchestra.run.smithi047.stdout: "172.21.15.64:6832/2497658100": "2022-06-30T15:45:49.337800+0000" 2022-06-29T15:45:56.148 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.148 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profiles": { 2022-06-29T15:45:56.148 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs_data": { 2022-06-29T15:45:56.148 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-device-class": "", 2022-06-29T15:45:56.149 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2022-06-29T15:45:56.149 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-root": "default", 2022-06-29T15:45:56.149 INFO:teuthology.orchestra.run.smithi047.stdout: "jerasure-per-chunk-alignment": "false", 2022-06-29T15:45:56.149 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2022-06-29T15:45:56.150 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "2", 2022-06-29T15:45:56.150 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2022-06-29T15:45:56.150 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2022-06-29T15:45:56.150 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van", 2022-06-29T15:45:56.151 INFO:teuthology.orchestra.run.smithi047.stdout: "w": "8" 2022-06-29T15:45:56.151 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.151 INFO:teuthology.orchestra.run.smithi047.stdout: "default": { 2022-06-29T15:45:56.151 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2022-06-29T15:45:56.152 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2022-06-29T15:45:56.152 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "1", 2022-06-29T15:45:56.152 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2022-06-29T15:45:56.152 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2022-06-29T15:45:56.153 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van" 2022-06-29T15:45:56.153 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:56.153 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:56.153 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps_queue": [], 2022-06-29T15:45:56.153 INFO:teuthology.orchestra.run.smithi047.stdout: "new_removed_snaps": [], 2022-06-29T15:45:56.154 INFO:teuthology.orchestra.run.smithi047.stdout: "new_purged_snaps": [], 2022-06-29T15:45:56.154 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_node_flags": {}, 2022-06-29T15:45:56.154 INFO:teuthology.orchestra.run.smithi047.stdout: "device_class_flags": {} 2022-06-29T15:45:56.154 INFO:teuthology.orchestra.run.smithi047.stdout:} 2022-06-29T15:45:56.156 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.899+0000 7f4069ee0700 1 -- 172.21.15.47:0/1012089555 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4044053b00 msgr2=0x7f4044055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:56.156 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.899+0000 7f4069ee0700 1 --2- 172.21.15.47:0/1012089555 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4044053b00 0x7f4044055fc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f4058018000 tx=0x7f4058014000).stop 2022-06-29T15:45:56.157 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.899+0000 7f4069ee0700 1 -- 172.21.15.47:0/1012089555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40640b2120 msgr2=0x7f4064144d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:56.157 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.899+0000 7f4069ee0700 1 --2- 172.21.15.47:0/1012089555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40640b2120 0x7f4064144d90 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7f4054000ed0 tx=0x7f4054007a90).stop 2022-06-29T15:45:56.157 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.899+0000 7f4069ee0700 1 -- 172.21.15.47:0/1012089555 shutdown_connections 2022-06-29T15:45:56.158 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.899+0000 7f4069ee0700 1 --2- 172.21.15.47:0/1012089555 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4044053b00 0x7f4044055fc0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:56.158 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.899+0000 7f4069ee0700 1 --2- 172.21.15.47:0/1012089555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4064159db0 0x7f406415a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:56.158 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.899+0000 7f4069ee0700 1 --2- 172.21.15.47:0/1012089555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40640b2120 0x7f4064144d90 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:56.158 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.899+0000 7f4069ee0700 1 --2- 172.21.15.47:0/1012089555 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f40640b1260 0x7f40640aec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:56.159 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.899+0000 7f4069ee0700 1 -- 172.21.15.47:0/1012089555 >> 172.21.15.47:0/1012089555 conn(0x7f40640626f0 msgr2=0x7f40640b3a70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:45:56.159 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.899+0000 7f4069ee0700 1 -- 172.21.15.47:0/1012089555 shutdown_connections 2022-06-29T15:45:56.159 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:55.899+0000 7f4069ee0700 1 -- 172.21.15.47:0/1012089555 wait complete. 2022-06-29T15:45:56.160 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:45:56.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.271+0000 7fda505ba700 1 Processor -- start 2022-06-29T15:45:56.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.271+0000 7fda505ba700 1 -- start start 2022-06-29T15:45:56.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.271+0000 7fda505ba700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fda4813cfc0 0x7fda480b1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:56.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.271+0000 7fda505ba700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fda48061670 0x7fda48143040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:56.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.271+0000 7fda505ba700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda480ae370 con 0x7fda48061af0 2022-06-29T15:45:56.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.271+0000 7fda505ba700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda4805f150 con 0x7fda4813cfc0 2022-06-29T15:45:56.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.271+0000 7fda505ba700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda4805f3f0 con 0x7fda48061670 2022-06-29T15:45:56.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.271+0000 7fda4e356700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fda4813cfc0 0x7fda480b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:56.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.271+0000 7fda4db55700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fda48061670 0x7fda48143040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:56.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.271+0000 7fda4eb57700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fda48061af0 0x7fda48143530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:41448/0 (socket says 172.21.15.47:41448) 2022-06-29T15:45:56.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.271+0000 7fda4e356700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fda4813cfc0 0x7fda480b1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:54030/0 (socket says 172.21.15.47:54030) 2022-06-29T15:45:56.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.271+0000 7fda4eb57700 1 -- 172.21.15.47:0/2900140966 learned_addr learned my addr 172.21.15.47:0/2900140966 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:45:56.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.271+0000 7fda4e356700 1 -- 172.21.15.47:0/2900140966 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fda48061670 msgr2=0x7fda48143040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:45:56.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.271+0000 7fda4e356700 1 --2- 172.21.15.47:0/2900140966 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fda48061670 0x7fda48143040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:56.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.271+0000 7fda4e356700 1 -- 172.21.15.47:0/2900140966 >> v1:172.21.15.64:6789/0 conn(0x7fda48061af0 legacy=0x7fda48143530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:56.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.271+0000 7fda4e356700 1 -- 172.21.15.47:0/2900140966 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda4805f690 con 0x7fda4813cfc0 2022-06-29T15:45:56.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.271+0000 7fda4e356700 1 --2- 172.21.15.47:0/2900140966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda4813cfc0 0x7fda480b1480 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7fda38002f70 tx=0x7fda38006df0).ready entity=mon.1 client_cookie=86b58756d4bcb839 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:56.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.275+0000 7fda4d354700 1 -- 172.21.15.47:0/2900140966 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda3800d570 con 0x7fda4813cfc0 2022-06-29T15:45:56.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.275+0000 7fda4d354700 1 -- 172.21.15.47:0/2900140966 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda38000e90 con 0x7fda4813cfc0 2022-06-29T15:45:56.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.275+0000 7fda4d354700 1 -- 172.21.15.47:0/2900140966 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda3800d8c0 con 0x7fda4813cfc0 2022-06-29T15:45:56.282 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.275+0000 7fda505ba700 1 -- 172.21.15.47:0/2900140966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda4813cfc0 msgr2=0x7fda480b1480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:56.282 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.275+0000 7fda505ba700 1 --2- 172.21.15.47:0/2900140966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda4813cfc0 0x7fda480b1480 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7fda38002f70 tx=0x7fda38006df0).stop 2022-06-29T15:45:56.282 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.275+0000 7fda505ba700 1 -- 172.21.15.47:0/2900140966 shutdown_connections 2022-06-29T15:45:56.282 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.275+0000 7fda505ba700 1 --2- 172.21.15.47:0/2900140966 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fda48061670 0x7fda48143040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:56.282 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.275+0000 7fda505ba700 1 --2- 172.21.15.47:0/2900140966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda4813cfc0 0x7fda480b1480 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:56.283 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.275+0000 7fda505ba700 1 -- 172.21.15.47:0/2900140966 >> 172.21.15.47:0/2900140966 conn(0x7fda4805c510 msgr2=0x7fda48063800 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:45:56.283 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.275+0000 7fda505ba700 1 -- 172.21.15.47:0/2900140966 shutdown_connections 2022-06-29T15:45:56.283 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.275+0000 7fda505ba700 1 -- 172.21.15.47:0/2900140966 wait complete. 2022-06-29T15:45:56.283 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.275+0000 7fda505ba700 1 Processor -- start 2022-06-29T15:45:56.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.275+0000 7fda505ba700 1 -- start start 2022-06-29T15:45:56.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.275+0000 7fda505ba700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda48061670 0x7fda480afb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:56.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.275+0000 7fda505ba700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda48061af0 0x7fda48066520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:56.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.275+0000 7fda505ba700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fda4813cfc0 0x7fda48066a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:56.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.275+0000 7fda505ba700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda48064db0 con 0x7fda48061670 2022-06-29T15:45:56.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.275+0000 7fda505ba700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda48142970 con 0x7fda48061af0 2022-06-29T15:45:56.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.275+0000 7fda505ba700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda48143990 con 0x7fda4813cfc0 2022-06-29T15:45:56.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.279+0000 7fda4e356700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda48061670 0x7fda480afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:56.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.279+0000 7fda4e356700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda48061670 0x7fda480afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48332/0 (socket says 172.21.15.47:48332) 2022-06-29T15:45:56.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.279+0000 7fda4e356700 1 -- 172.21.15.47:0/3776626888 learned_addr learned my addr 172.21.15.47:0/3776626888 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:45:56.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.279+0000 7fda4db55700 1 --2- 172.21.15.47:0/3776626888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda48061af0 0x7fda48066520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:56.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.279+0000 7fda4eb57700 1 --2- 172.21.15.47:0/3776626888 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fda4813cfc0 0x7fda48066a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:56.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.279+0000 7fda4e356700 1 -- 172.21.15.47:0/3776626888 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fda4813cfc0 msgr2=0x7fda48066a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:56.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.279+0000 7fda4e356700 1 --2- 172.21.15.47:0/3776626888 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fda4813cfc0 0x7fda48066a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:56.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.279+0000 7fda4e356700 1 -- 172.21.15.47:0/3776626888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda48061af0 msgr2=0x7fda48066520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:56.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.279+0000 7fda4e356700 1 --2- 172.21.15.47:0/3776626888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda48061af0 0x7fda48066520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:56.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.279+0000 7fda4e356700 1 -- 172.21.15.47:0/3776626888 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda3800b180 con 0x7fda48061670 2022-06-29T15:45:56.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.279+0000 7fda4e356700 1 --2- 172.21.15.47:0/3776626888 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda48061670 0x7fda480afb90 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7fda38002f70 tx=0x7fda38014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:56.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.279+0000 7fda377fe700 1 -- 172.21.15.47:0/3776626888 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda3800dea0 con 0x7fda48061670 2022-06-29T15:45:56.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.279+0000 7fda377fe700 1 -- 172.21.15.47:0/3776626888 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda38013de0 con 0x7fda48061670 2022-06-29T15:45:56.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.279+0000 7fda377fe700 1 -- 172.21.15.47:0/3776626888 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda3801e5f0 con 0x7fda48061670 2022-06-29T15:45:56.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.279+0000 7fda505ba700 1 -- 172.21.15.47:0/3776626888 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda48065050 con 0x7fda48061670 2022-06-29T15:45:56.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.279+0000 7fda505ba700 1 -- 172.21.15.47:0/3776626888 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda481424f0 con 0x7fda48061670 2022-06-29T15:45:56.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.279+0000 7fda505ba700 1 -- 172.21.15.47:0/3776626888 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda48143b50 con 0x7fda48061670 2022-06-29T15:45:56.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.279+0000 7fda377fe700 1 -- 172.21.15.47:0/3776626888 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fda3801c070 con 0x7fda48061670 2022-06-29T15:45:56.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.283+0000 7fda377fe700 1 --2- 172.21.15.47:0/3776626888 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fda28053b00 0x7fda28055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:56.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.283+0000 7fda377fe700 1 -- 172.21.15.47:0/3776626888 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fda3806c800 con 0x7fda48061670 2022-06-29T15:45:56.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.283+0000 7fda4db55700 1 --2- 172.21.15.47:0/3776626888 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fda28053b00 0x7fda28055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:56.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.283+0000 7fda4db55700 1 --2- 172.21.15.47:0/3776626888 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fda28053b00 0x7fda28055fc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7fda4400b000 tx=0x7fda440091d0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:56.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.287+0000 7fda377fe700 1 -- 172.21.15.47:0/3776626888 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fda38043790 con 0x7fda48061670 2022-06-29T15:45:56.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.455+0000 7fda505ba700 1 -- 172.21.15.47:0/3776626888 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda48142b30 con 0x7fda48061670 2022-06-29T15:45:56.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.455+0000 7fda377fe700 1 -- 172.21.15.47:0/3776626888 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+1559 (secure 0 0 0) 0x7fda38023070 con 0x7fda48061670 2022-06-29T15:45:56.459 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:45:56.459 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":20,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5259,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3601815346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3601815346},{"type":"v1","addr":"172.21.15.47:6835","nonce":3601815346}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15},{"gid":5262,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/657177602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":657177602},{"type":"v1","addr":"172.21.15.47:6837","nonce":657177602}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15},{"gid":5523,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.64:6833/3554527837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3554527837},{"type":"v1","addr":"172.21.15.64:6833","nonce":3554527837}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":20}],"filesystems":[]} 2022-06-29T15:45:56.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.455+0000 7fda505ba700 1 -- 172.21.15.47:0/3776626888 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fda28053b00 msgr2=0x7fda28055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:56.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.455+0000 7fda505ba700 1 --2- 172.21.15.47:0/3776626888 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fda28053b00 0x7fda28055fc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7fda4400b000 tx=0x7fda440091d0).stop 2022-06-29T15:45:56.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.455+0000 7fda505ba700 1 -- 172.21.15.47:0/3776626888 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda48061670 msgr2=0x7fda480afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:56.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.455+0000 7fda505ba700 1 --2- 172.21.15.47:0/3776626888 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda48061670 0x7fda480afb90 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7fda38002f70 tx=0x7fda38014000).stop 2022-06-29T15:45:56.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.455+0000 7fda505ba700 1 -- 172.21.15.47:0/3776626888 shutdown_connections 2022-06-29T15:45:56.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.455+0000 7fda505ba700 1 --2- 172.21.15.47:0/3776626888 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fda4813cfc0 0x7fda48066a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:56.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.455+0000 7fda505ba700 1 --2- 172.21.15.47:0/3776626888 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fda28053b00 0x7fda28055fc0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:56.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.455+0000 7fda505ba700 1 --2- 172.21.15.47:0/3776626888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda48061af0 0x7fda48066520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:56.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.455+0000 7fda505ba700 1 --2- 172.21.15.47:0/3776626888 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda48061670 0x7fda480afb90 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:56.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.455+0000 7fda505ba700 1 -- 172.21.15.47:0/3776626888 >> 172.21.15.47:0/3776626888 conn(0x7fda4805c510 msgr2=0x7fda4813bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:45:56.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.455+0000 7fda505ba700 1 -- 172.21.15.47:0/3776626888 shutdown_connections 2022-06-29T15:45:56.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.455+0000 7fda505ba700 1 -- 172.21.15.47:0/3776626888 wait complete. 2022-06-29T15:45:56.464 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 20 2022-06-29T15:45:56.471 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:45:56.573 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.567+0000 7fe541e29700 1 Processor -- start 2022-06-29T15:45:56.573 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.567+0000 7fe541e29700 1 -- start start 2022-06-29T15:45:56.584 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.571+0000 7fe541e29700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe53c0b2ee0 0x7fe53c0b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:56.584 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.571+0000 7fe541e29700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe53c0ad970 0x7fe53c1407e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:56.584 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.571+0000 7fe541e29700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe53c0b0e50 0x7fe53c13d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:56.585 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.571+0000 7fe541e29700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe53c13fef0 con 0x7fe53c0ad970 2022-06-29T15:45:56.585 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.571+0000 7fe541e29700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe53c143810 con 0x7fe53c0b2ee0 2022-06-29T15:45:56.586 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.571+0000 7fe541e29700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe53c1400e0 con 0x7fe53c0b0e50 2022-06-29T15:45:56.586 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.571+0000 7fe53bfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe53c0ad970 0x7fe53c1407e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:56.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.571+0000 7fe53bfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe53c0ad970 0x7fe53c1407e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48340/0 (socket says 172.21.15.47:48340) 2022-06-29T15:45:56.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.571+0000 7fe53bfff700 1 -- 172.21.15.47:0/1856922918 learned_addr learned my addr 172.21.15.47:0/1856922918 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:45:56.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.571+0000 7fe540e27700 1 --2- 172.21.15.47:0/1856922918 >> v2:172.21.15.64:3300/0 conn(0x7fe53c0b2ee0 0x7fe53c0b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:56.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.571+0000 7fe541628700 1 --2- 172.21.15.47:0/1856922918 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe53c0b0e50 0x7fe53c13d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:56.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.571+0000 7fe53bfff700 1 -- 172.21.15.47:0/1856922918 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe53c0b0e50 msgr2=0x7fe53c13d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:45:56.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.571+0000 7fe53bfff700 1 --2- 172.21.15.47:0/1856922918 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe53c0b0e50 0x7fe53c13d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:56.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.571+0000 7fe53bfff700 1 -- 172.21.15.47:0/1856922918 >> v2:172.21.15.64:3300/0 conn(0x7fe53c0b2ee0 msgr2=0x7fe53c0b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:45:56.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.571+0000 7fe53bfff700 1 --2- 172.21.15.47:0/1856922918 >> v2:172.21.15.64:3300/0 conn(0x7fe53c0b2ee0 0x7fe53c0b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:56.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.571+0000 7fe53bfff700 1 -- 172.21.15.47:0/1856922918 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe53c143570 con 0x7fe53c0ad970 2022-06-29T15:45:56.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.571+0000 7fe53bfff700 1 --2- 172.21.15.47:0/1856922918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe53c0ad970 0x7fe53c1407e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7fe52c002f70 tx=0x7fe52c006eb0).ready entity=mon.0 client_cookie=6776e6fd4093ee81 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:56.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.571+0000 7fe53b7fe700 1 -- 172.21.15.47:0/1856922918 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe52c00d570 con 0x7fe53c0ad970 2022-06-29T15:45:56.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.571+0000 7fe53b7fe700 1 -- 172.21.15.47:0/1856922918 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe52c000e90 con 0x7fe53c0ad970 2022-06-29T15:45:56.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.571+0000 7fe53b7fe700 1 -- 172.21.15.47:0/1856922918 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe52c00d900 con 0x7fe53c0ad970 2022-06-29T15:45:56.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.571+0000 7fe541e29700 1 -- 172.21.15.47:0/1856922918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe53c0ad970 msgr2=0x7fe53c1407e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:56.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.571+0000 7fe541e29700 1 --2- 172.21.15.47:0/1856922918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe53c0ad970 0x7fe53c1407e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7fe52c002f70 tx=0x7fe52c006eb0).stop 2022-06-29T15:45:56.594 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.571+0000 7fe541e29700 1 -- 172.21.15.47:0/1856922918 shutdown_connections 2022-06-29T15:45:56.594 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.571+0000 7fe541e29700 1 --2- 172.21.15.47:0/1856922918 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe53c0b0e50 0x7fe53c13d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:56.594 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.571+0000 7fe541e29700 1 --2- 172.21.15.47:0/1856922918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe53c0ad970 0x7fe53c1407e0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:56.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.571+0000 7fe541e29700 1 --2- 172.21.15.47:0/1856922918 >> v2:172.21.15.64:3300/0 conn(0x7fe53c0b2ee0 0x7fe53c0b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:56.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.571+0000 7fe541e29700 1 -- 172.21.15.47:0/1856922918 >> 172.21.15.47:0/1856922918 conn(0x7fe53c063670 msgr2=0x7fe53c063a80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:45:56.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.571+0000 7fe541e29700 1 -- 172.21.15.47:0/1856922918 shutdown_connections 2022-06-29T15:45:56.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.571+0000 7fe541e29700 1 -- 172.21.15.47:0/1856922918 wait complete. 2022-06-29T15:45:56.597 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.575+0000 7fe541e29700 1 Processor -- start 2022-06-29T15:45:56.597 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.575+0000 7fe541e29700 1 -- start start 2022-06-29T15:45:56.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.575+0000 7fe541e29700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe53c0ad970 0x7fe53c0b4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:56.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.575+0000 7fe541e29700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe53c0b0e50 0x7fe53c145810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:56.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.575+0000 7fe541e29700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe53c0b2ee0 0x7fe53c0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:56.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.575+0000 7fe541e29700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe53c13e770 con 0x7fe53c0ad970 2022-06-29T15:45:56.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.575+0000 7fe541e29700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe53c13bd90 con 0x7fe53c0b2ee0 2022-06-29T15:45:56.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.575+0000 7fe541e29700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe53c0b4260 con 0x7fe53c0b0e50 2022-06-29T15:45:56.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.575+0000 7fe540e27700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe53c0ad970 0x7fe53c0b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:56.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.575+0000 7fe540e27700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe53c0ad970 0x7fe53c0b4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48346/0 (socket says 172.21.15.47:48346) 2022-06-29T15:45:56.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.575+0000 7fe540e27700 1 -- 172.21.15.47:0/2128426770 learned_addr learned my addr 172.21.15.47:0/2128426770 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:45:56.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.575+0000 7fe53bfff700 1 --2- 172.21.15.47:0/2128426770 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe53c0b0e50 0x7fe53c145810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:56.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.575+0000 7fe541628700 1 --2- 172.21.15.47:0/2128426770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe53c0b2ee0 0x7fe53c0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:56.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.575+0000 7fe540e27700 1 -- 172.21.15.47:0/2128426770 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe53c0b0e50 msgr2=0x7fe53c145810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:56.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.575+0000 7fe540e27700 1 --2- 172.21.15.47:0/2128426770 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe53c0b0e50 0x7fe53c145810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:56.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.575+0000 7fe540e27700 1 -- 172.21.15.47:0/2128426770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe53c0b2ee0 msgr2=0x7fe53c0b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:56.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.575+0000 7fe540e27700 1 --2- 172.21.15.47:0/2128426770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe53c0b2ee0 0x7fe53c0b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:56.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.575+0000 7fe540e27700 1 -- 172.21.15.47:0/2128426770 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe52c00b1e0 con 0x7fe53c0ad970 2022-06-29T15:45:56.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.575+0000 7fe541628700 1 --2- 172.21.15.47:0/2128426770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe53c0b2ee0 0x7fe53c0b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:45:56.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.575+0000 7fe540e27700 1 --2- 172.21.15.47:0/2128426770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe53c0ad970 0x7fe53c0b4690 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7fe53000c000 tx=0x7fe530006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:56.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.579+0000 7fe539ffb700 1 -- 172.21.15.47:0/2128426770 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe530010780 con 0x7fe53c0ad970 2022-06-29T15:45:56.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.579+0000 7fe539ffb700 1 -- 172.21.15.47:0/2128426770 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5300108e0 con 0x7fe53c0ad970 2022-06-29T15:45:56.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.579+0000 7fe541e29700 1 -- 172.21.15.47:0/2128426770 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe53c13deb0 con 0x7fe53c0ad970 2022-06-29T15:45:56.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.579+0000 7fe539ffb700 1 -- 172.21.15.47:0/2128426770 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe530010c40 con 0x7fe53c0ad970 2022-06-29T15:45:56.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.579+0000 7fe541e29700 1 -- 172.21.15.47:0/2128426770 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe53c13c260 con 0x7fe53c0ad970 2022-06-29T15:45:56.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.579+0000 7fe541e29700 1 -- 172.21.15.47:0/2128426770 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe53c13c0d0 con 0x7fe53c0ad970 2022-06-29T15:45:56.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.579+0000 7fe539ffb700 1 -- 172.21.15.47:0/2128426770 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe53001c460 con 0x7fe53c0ad970 2022-06-29T15:45:56.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.579+0000 7fe539ffb700 1 --2- 172.21.15.47:0/2128426770 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe51c053a70 0x7fe51c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:56.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.579+0000 7fe539ffb700 1 -- 172.21.15.47:0/2128426770 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fe53006c1f0 con 0x7fe53c0ad970 2022-06-29T15:45:56.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.579+0000 7fe53bfff700 1 --2- 172.21.15.47:0/2128426770 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe51c053a70 0x7fe51c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:56.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.583+0000 7fe53bfff700 1 --2- 172.21.15.47:0/2128426770 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe51c053a70 0x7fe51c055f30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7fe52c01c000 tx=0x7fe52c015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:56.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.587+0000 7fe539ffb700 1 -- 172.21.15.47:0/2128426770 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe530043190 con 0x7fe53c0ad970 2022-06-29T15:45:56.756 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.751+0000 7fe541e29700 1 -- 172.21.15.47:0/2128426770 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe53c13fd40 con 0x7fe53c0ad970 2022-06-29T15:45:56.756 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.751+0000 7fe539ffb700 1 -- 172.21.15.47:0/2128426770 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+1559 (secure 0 0 0) 0x7fe5300467b0 con 0x7fe53c0ad970 2022-06-29T15:45:56.757 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:45:56.757 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":20,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5259,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3601815346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3601815346},{"type":"v1","addr":"172.21.15.47:6835","nonce":3601815346}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15},{"gid":5262,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/657177602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":657177602},{"type":"v1","addr":"172.21.15.47:6837","nonce":657177602}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15},{"gid":5523,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.64:6833/3554527837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3554527837},{"type":"v1","addr":"172.21.15.64:6833","nonce":3554527837}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":20}],"filesystems":[]} 2022-06-29T15:45:56.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.755+0000 7fe541e29700 1 -- 172.21.15.47:0/2128426770 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe51c053a70 msgr2=0x7fe51c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:56.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.755+0000 7fe541e29700 1 --2- 172.21.15.47:0/2128426770 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe51c053a70 0x7fe51c055f30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7fe52c01c000 tx=0x7fe52c015000).stop 2022-06-29T15:45:56.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.755+0000 7fe541e29700 1 -- 172.21.15.47:0/2128426770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe53c0ad970 msgr2=0x7fe53c0b4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:56.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.755+0000 7fe541e29700 1 --2- 172.21.15.47:0/2128426770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe53c0ad970 0x7fe53c0b4690 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7fe53000c000 tx=0x7fe530006df0).stop 2022-06-29T15:45:56.762 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.755+0000 7fe541e29700 1 -- 172.21.15.47:0/2128426770 shutdown_connections 2022-06-29T15:45:56.762 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.755+0000 7fe541e29700 1 --2- 172.21.15.47:0/2128426770 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe51c053a70 0x7fe51c055f30 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:56.762 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.755+0000 7fe541e29700 1 --2- 172.21.15.47:0/2128426770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe53c0b2ee0 0x7fe53c0b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:56.762 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.755+0000 7fe541e29700 1 --2- 172.21.15.47:0/2128426770 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe53c0b0e50 0x7fe53c145810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:56.763 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.755+0000 7fe541e29700 1 --2- 172.21.15.47:0/2128426770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe53c0ad970 0x7fe53c0b4690 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:56.763 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.755+0000 7fe541e29700 1 -- 172.21.15.47:0/2128426770 >> 172.21.15.47:0/2128426770 conn(0x7fe53c063670 msgr2=0x7fe53c0b63c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:45:56.763 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.755+0000 7fe541e29700 1 -- 172.21.15.47:0/2128426770 shutdown_connections 2022-06-29T15:45:56.763 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.755+0000 7fe541e29700 1 -- 172.21.15.47:0/2128426770 wait complete. 2022-06-29T15:45:56.764 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 20 2022-06-29T15:45:56.777 INFO:tasks.ceph.mds.a:Restarting daemon 2022-06-29T15:45:56.778 INFO:tasks.ceph.mds.a:Stopping old one... 2022-06-29T15:45:56.778 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-06-29T15:45:56.778 INFO:teuthology.orchestra.run:waiting for 300 2022-06-29T15:45:56.779 INFO:tasks.ceph.mds.c:Restarting daemon 2022-06-29T15:45:56.779 INFO:tasks.ceph.mds.c:Stopping old one... 2022-06-29T15:45:56.780 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2022-06-29T15:45:56.780 INFO:teuthology.orchestra.run:waiting for 300 2022-06-29T15:45:56.780 INFO:tasks.ceph.mds.b:Restarting daemon 2022-06-29T15:45:56.780 INFO:tasks.ceph.mds.b:Stopping old one... 2022-06-29T15:45:56.781 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-06-29T15:45:56.781 INFO:teuthology.orchestra.run:waiting for 300 2022-06-29T15:45:56.789 INFO:tasks.ceph.mds.a:Stopped 2022-06-29T15:45:56.789 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-06-29T15:45:56.795 INFO:tasks.ceph.mds.c:Stopped 2022-06-29T15:45:56.795 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2022-06-29T15:45:56.796 INFO:tasks.ceph.mds.b:Stopped 2022-06-29T15:45:56.797 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-06-29T15:45:56.838 INFO:tasks.ceph.mds.a:Started 2022-06-29T15:45:56.839 DEBUG:teuthology.parallel:result is None 2022-06-29T15:45:56.840 INFO:tasks.ceph.mds.c:Started 2022-06-29T15:45:56.840 DEBUG:teuthology.parallel:result is None 2022-06-29T15:45:56.841 INFO:tasks.ceph.mds.b:Started 2022-06-29T15:45:56.842 DEBUG:teuthology.parallel:result is None 2022-06-29T15:45:56.842 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-06-29T15:45:56.912 INFO:tasks.ceph.mds.c.smithi047.stdout:starting mds.c at 2022-06-29T15:45:56.919 INFO:tasks.ceph.mds.b.smithi064.stdout:starting mds.b at 2022-06-29T15:45:56.930 INFO:tasks.ceph.mds.a.smithi047.stdout:starting mds.a at 2022-06-29T15:45:56.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.947+0000 7ff58759e700 1 Processor -- start 2022-06-29T15:45:56.953 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.947+0000 7ff58759e700 1 -- start start 2022-06-29T15:45:56.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.951+0000 7ff58759e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff588146890 0x7ff5880b0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:56.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.951+0000 7ff58759e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5880b2d00 0x7ff5880b12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:56.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.951+0000 7ff58759e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5880b3900 con 0x7ff5880b7860 2022-06-29T15:45:56.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.951+0000 7ff58759e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5881477c0 con 0x7ff588146890 2022-06-29T15:45:56.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.951+0000 7ff58759e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5880b7c40 con 0x7ff5880b2d00 2022-06-29T15:45:56.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.951+0000 7ff58659c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff5880b7860 0x7ff5880af160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33932/0 (socket says 172.21.15.47:33932) 2022-06-29T15:45:56.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.951+0000 7ff58659c700 1 -- 172.21.15.47:0/3304148443 learned_addr learned my addr 172.21.15.47:0/3304148443 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:45:56.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.951+0000 7ff586d9d700 1 --2- 172.21.15.47:0/3304148443 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5880b2d00 0x7ff5880b12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:56.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.951+0000 7ff585d9b700 1 --2- 172.21.15.47:0/3304148443 >> v2:172.21.15.64:3300/0 conn(0x7ff588146890 0x7ff5880b0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:56.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.951+0000 7ff586d9d700 1 -- 172.21.15.47:0/3304148443 >> v2:172.21.15.64:3300/0 conn(0x7ff588146890 msgr2=0x7ff5880b0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:45:56.958 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.951+0000 7ff586d9d700 1 --2- 172.21.15.47:0/3304148443 >> v2:172.21.15.64:3300/0 conn(0x7ff588146890 0x7ff5880b0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:56.958 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.951+0000 7ff586d9d700 1 -- 172.21.15.47:0/3304148443 >> v1:172.21.15.47:6789/0 conn(0x7ff5880b7860 legacy=0x7ff5880af160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:56.959 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.951+0000 7ff586d9d700 1 -- 172.21.15.47:0/3304148443 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5880b3660 con 0x7ff5880b2d00 2022-06-29T15:45:56.959 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.951+0000 7ff585d9b700 1 --2- 172.21.15.47:0/3304148443 >> v2:172.21.15.64:3300/0 conn(0x7ff588146890 0x7ff5880b0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:45:56.959 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.951+0000 7ff586d9d700 1 --2- 172.21.15.47:0/3304148443 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5880b2d00 0x7ff5880b12f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7ff5800032f0 tx=0x7ff5800091d0).ready entity=mon.2 client_cookie=42564d77d697d5af server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:56.960 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.955+0000 7ff58559a700 1 -- 172.21.15.47:0/3304148443 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff580009ea0 con 0x7ff5880b2d00 2022-06-29T15:45:56.960 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.955+0000 7ff58559a700 1 -- 172.21.15.47:0/3304148443 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff58000b8a0 con 0x7ff5880b2d00 2022-06-29T15:45:56.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.955+0000 7ff58559a700 1 -- 172.21.15.47:0/3304148443 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff58000ba10 con 0x7ff5880b2d00 2022-06-29T15:45:56.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.955+0000 7ff58759e700 1 -- 172.21.15.47:0/3304148443 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5880b2d00 msgr2=0x7ff5880b12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:56.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.955+0000 7ff58759e700 1 --2- 172.21.15.47:0/3304148443 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5880b2d00 0x7ff5880b12f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7ff5800032f0 tx=0x7ff5800091d0).stop 2022-06-29T15:45:56.962 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.955+0000 7ff58759e700 1 -- 172.21.15.47:0/3304148443 shutdown_connections 2022-06-29T15:45:56.962 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.955+0000 7ff58759e700 1 --2- 172.21.15.47:0/3304148443 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5880b2d00 0x7ff5880b12f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:56.962 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.955+0000 7ff58759e700 1 --2- 172.21.15.47:0/3304148443 >> v2:172.21.15.64:3300/0 conn(0x7ff588146890 0x7ff5880b0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:56.962 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.955+0000 7ff58759e700 1 -- 172.21.15.47:0/3304148443 >> 172.21.15.47:0/3304148443 conn(0x7ff58805f480 msgr2=0x7ff5880647d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:45:56.963 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.955+0000 7ff58759e700 1 -- 172.21.15.47:0/3304148443 shutdown_connections 2022-06-29T15:45:56.963 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.955+0000 7ff58759e700 1 -- 172.21.15.47:0/3304148443 wait complete. 2022-06-29T15:45:56.965 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.959+0000 7ff58759e700 1 Processor -- start 2022-06-29T15:45:56.966 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.959+0000 7ff58759e700 1 -- start start 2022-06-29T15:45:56.966 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.959+0000 7ff58759e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5880b2d00 0x7ff588061cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:56.966 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.959+0000 7ff58759e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5880b7860 0x7ff5880ad8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:56.967 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.959+0000 7ff58759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff588146890 0x7ff5880adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:56.967 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.959+0000 7ff58759e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff588155920 con 0x7ff5880b2d00 2022-06-29T15:45:56.967 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.959+0000 7ff58759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5880629e0 con 0x7ff588146890 2022-06-29T15:45:56.967 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.959+0000 7ff58759e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5880b62f0 con 0x7ff5880b7860 2022-06-29T15:45:56.968 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.959+0000 7ff58659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5880b2d00 0x7ff588061cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:56.968 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.959+0000 7ff58659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5880b2d00 0x7ff588061cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48384/0 (socket says 172.21.15.47:48384) 2022-06-29T15:45:56.968 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.959+0000 7ff58659c700 1 -- 172.21.15.47:0/591616804 learned_addr learned my addr 172.21.15.47:0/591616804 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:45:56.969 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.959+0000 7ff585d9b700 1 --2- 172.21.15.47:0/591616804 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5880b7860 0x7ff5880ad8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:56.969 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.959+0000 7ff586d9d700 1 --2- 172.21.15.47:0/591616804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff588146890 0x7ff5880adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:56.969 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.959+0000 7ff58659c700 1 -- 172.21.15.47:0/591616804 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5880b7860 msgr2=0x7ff5880ad8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:56.970 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.959+0000 7ff58659c700 1 --2- 172.21.15.47:0/591616804 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5880b7860 0x7ff5880ad8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:56.970 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.959+0000 7ff58659c700 1 -- 172.21.15.47:0/591616804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff588146890 msgr2=0x7ff5880adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:56.970 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.959+0000 7ff58659c700 1 --2- 172.21.15.47:0/591616804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff588146890 0x7ff5880adde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:56.971 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.959+0000 7ff58659c700 1 -- 172.21.15.47:0/591616804 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff580009470 con 0x7ff5880b2d00 2022-06-29T15:45:56.971 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.959+0000 7ff58659c700 1 --2- 172.21.15.47:0/591616804 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5880b2d00 0x7ff588061cf0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7ff5880b8080 tx=0x7ff578007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:56.971 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.959+0000 7ff5777fe700 1 -- 172.21.15.47:0/591616804 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff5780070d0 con 0x7ff5880b2d00 2022-06-29T15:45:56.971 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.959+0000 7ff5777fe700 1 -- 172.21.15.47:0/591616804 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff578007230 con 0x7ff5880b2d00 2022-06-29T15:45:56.972 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.959+0000 7ff58759e700 1 -- 172.21.15.47:0/591616804 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5880b6590 con 0x7ff5880b2d00 2022-06-29T15:45:56.972 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.959+0000 7ff5777fe700 1 -- 172.21.15.47:0/591616804 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff578005610 con 0x7ff5880b2d00 2022-06-29T15:45:56.972 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.959+0000 7ff58759e700 1 -- 172.21.15.47:0/591616804 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5880b6f30 con 0x7ff5880b2d00 2022-06-29T15:45:56.974 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.963+0000 7ff58759e700 1 -- 172.21.15.47:0/591616804 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff588062ba0 con 0x7ff5880b2d00 2022-06-29T15:45:56.974 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.963+0000 7ff5777fe700 1 -- 172.21.15.47:0/591616804 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff578010070 con 0x7ff5880b2d00 2022-06-29T15:45:56.974 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.963+0000 7ff5777fe700 1 --2- 172.21.15.47:0/591616804 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff564053a70 0x7ff564055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:56.974 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.963+0000 7ff5777fe700 1 -- 172.21.15.47:0/591616804 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5686+0+0 (secure 0 0 0) 0x7ff57806b350 con 0x7ff5880b2d00 2022-06-29T15:45:56.975 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.963+0000 7ff585d9b700 1 --2- 172.21.15.47:0/591616804 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff564053a70 0x7ff564055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:56.975 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.963+0000 7ff585d9b700 1 --2- 172.21.15.47:0/591616804 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff564053a70 0x7ff564055f30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7ff57c00c000 tx=0x7ff57c008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:56.975 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:56.967+0000 7ff5777fe700 1 -- 172.21.15.47:0/591616804 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff578042350 con 0x7ff5880b2d00 2022-06-29T15:45:57.110 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:57.107+0000 7ff58759e700 1 -- 172.21.15.47:0/591616804 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7ff5880b6720 con 0x7ff5880b2d00 2022-06-29T15:45:57.967 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:57.963+0000 7ff5777fe700 1 -- 172.21.15.47:0/591616804 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v53) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff578018070 con 0x7ff5880b2d00 2022-06-29T15:45:57.971 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:57.967+0000 7ff58759e700 1 -- 172.21.15.47:0/591616804 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff564053a70 msgr2=0x7ff564055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:57.972 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:57.967+0000 7ff58759e700 1 --2- 172.21.15.47:0/591616804 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff564053a70 0x7ff564055f30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7ff57c00c000 tx=0x7ff57c008040).stop 2022-06-29T15:45:57.972 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:57.967+0000 7ff58759e700 1 -- 172.21.15.47:0/591616804 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5880b2d00 msgr2=0x7ff588061cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:57.972 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:57.967+0000 7ff58759e700 1 --2- 172.21.15.47:0/591616804 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5880b2d00 0x7ff588061cf0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7ff5880b8080 tx=0x7ff578007c40).stop 2022-06-29T15:45:57.972 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:57.967+0000 7ff58759e700 1 -- 172.21.15.47:0/591616804 shutdown_connections 2022-06-29T15:45:57.972 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:57.967+0000 7ff58759e700 1 --2- 172.21.15.47:0/591616804 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff564053a70 0x7ff564055f30 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:57.973 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:57.967+0000 7ff58759e700 1 --2- 172.21.15.47:0/591616804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff588146890 0x7ff5880adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:57.973 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:57.967+0000 7ff58759e700 1 --2- 172.21.15.47:0/591616804 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5880b7860 0x7ff5880ad8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:57.973 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:57.967+0000 7ff58759e700 1 --2- 172.21.15.47:0/591616804 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5880b2d00 0x7ff588061cf0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:57.974 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:57.967+0000 7ff58759e700 1 -- 172.21.15.47:0/591616804 >> 172.21.15.47:0/591616804 conn(0x7ff58805f480 msgr2=0x7ff5880b53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:45:57.974 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:57.967+0000 7ff58759e700 1 -- 172.21.15.47:0/591616804 shutdown_connections 2022-06-29T15:45:57.974 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:57.967+0000 7ff58759e700 1 -- 172.21.15.47:0/591616804 wait complete. 2022-06-29T15:45:57.975 INFO:teuthology.orchestra.run.smithi047.stderr: removed all blacklist entries 2022-06-29T15:45:57.986 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2022-06-29T15:45:57.987 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-06-29T15:45:58.100 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.095+0000 7f86cacbd700 1 Processor -- start 2022-06-29T15:45:58.101 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.095+0000 7f86cacbd700 1 -- start start 2022-06-29T15:45:58.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.099+0000 7f86cacbd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f86c40ad970 0x7f86c40b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:58.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.099+0000 7f86cacbd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f86c40b0e50 0x7f86c413d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:58.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.099+0000 7f86cacbd700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86c41400e0 con 0x7f86c40b2ee0 2022-06-29T15:45:58.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.099+0000 7f86cacbd700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86c4140ef0 con 0x7f86c40b0e50 2022-06-29T15:45:58.104 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.099+0000 7f86cacbd700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86c413fef0 con 0x7f86c40ad970 2022-06-29T15:45:58.104 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.099+0000 7f86ca4bc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f86c40b0e50 0x7f86c413d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:58.104 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.099+0000 7f86ca4bc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f86c40b0e50 0x7f86c413d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48398/0 (socket says 172.21.15.47:48398) 2022-06-29T15:45:58.104 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.099+0000 7f86ca4bc700 1 -- 172.21.15.47:0/412702666 learned_addr learned my addr 172.21.15.47:0/412702666 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:45:58.105 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.099+0000 7f86c94ba700 1 --2- 172.21.15.47:0/412702666 >> v2:172.21.15.64:3300/0 conn(0x7f86c40ad970 0x7f86c40b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:58.105 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.099+0000 7f86ca4bc700 1 -- 172.21.15.47:0/412702666 >> v2:172.21.15.64:3300/0 conn(0x7f86c40ad970 msgr2=0x7f86c40b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:45:58.105 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.099+0000 7f86ca4bc700 1 --2- 172.21.15.47:0/412702666 >> v2:172.21.15.64:3300/0 conn(0x7f86c40ad970 0x7f86c40b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:58.105 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.099+0000 7f86ca4bc700 1 -- 172.21.15.47:0/412702666 >> v1:172.21.15.64:6789/0 conn(0x7f86c40b2ee0 legacy=0x7f86c41407e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:58.105 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.099+0000 7f86ca4bc700 1 -- 172.21.15.47:0/412702666 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c4143570 con 0x7f86c40b0e50 2022-06-29T15:45:58.106 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.099+0000 7f86ca4bc700 1 --2- 172.21.15.47:0/412702666 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86c40b0e50 0x7f86c413d290 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7f86bc0032f0 tx=0x7f86bc009290).ready entity=mon.0 client_cookie=f5030e096314b8b4 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:58.115 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.099+0000 7f86c8cb9700 1 -- 172.21.15.47:0/412702666 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f86bc00b870 con 0x7f86c40b0e50 2022-06-29T15:45:58.116 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.099+0000 7f86c8cb9700 1 -- 172.21.15.47:0/412702666 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86bc00b9d0 con 0x7f86c40b0e50 2022-06-29T15:45:58.116 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.099+0000 7f86c8cb9700 1 -- 172.21.15.47:0/412702666 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f86bc00bb40 con 0x7f86c40b0e50 2022-06-29T15:45:58.121 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.099+0000 7f86cacbd700 1 -- 172.21.15.47:0/412702666 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86c40b0e50 msgr2=0x7f86c413d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:58.121 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.099+0000 7f86cacbd700 1 --2- 172.21.15.47:0/412702666 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86c40b0e50 0x7f86c413d290 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7f86bc0032f0 tx=0x7f86bc009290).stop 2022-06-29T15:45:58.121 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.099+0000 7f86cacbd700 1 -- 172.21.15.47:0/412702666 shutdown_connections 2022-06-29T15:45:58.122 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.099+0000 7f86cacbd700 1 --2- 172.21.15.47:0/412702666 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86c40b0e50 0x7f86c413d290 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:58.122 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.099+0000 7f86cacbd700 1 --2- 172.21.15.47:0/412702666 >> v2:172.21.15.64:3300/0 conn(0x7f86c40ad970 0x7f86c40b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:58.122 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.099+0000 7f86cacbd700 1 -- 172.21.15.47:0/412702666 >> 172.21.15.47:0/412702666 conn(0x7f86c4063670 msgr2=0x7f86c4063a80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:45:58.123 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.099+0000 7f86cacbd700 1 -- 172.21.15.47:0/412702666 shutdown_connections 2022-06-29T15:45:58.123 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.099+0000 7f86cacbd700 1 -- 172.21.15.47:0/412702666 wait complete. 2022-06-29T15:45:58.123 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.103+0000 7f86cacbd700 1 Processor -- start 2022-06-29T15:45:58.123 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.103+0000 7f86cacbd700 1 -- start start 2022-06-29T15:45:58.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.103+0000 7f86cacbd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86c40ad970 0x7f86c413d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:58.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.103+0000 7f86cacbd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86c40b0e50 0x7f86c40ac290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:58.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.103+0000 7f86cacbd700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f86c40b2ee0 0x7f86c40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:58.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.103+0000 7f86cacbd700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c40aad40 con 0x7f86c40ad970 2022-06-29T15:45:58.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.103+0000 7f86cacbd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c405f590 con 0x7f86c40b0e50 2022-06-29T15:45:58.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.103+0000 7f86cacbd700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86c40ac780 con 0x7f86c40b2ee0 2022-06-29T15:45:58.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.103+0000 7f86c9cbb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86c40ad970 0x7f86c413d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:58.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.103+0000 7f86c9cbb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86c40ad970 0x7f86c413d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48400/0 (socket says 172.21.15.47:48400) 2022-06-29T15:45:58.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.103+0000 7f86c9cbb700 1 -- 172.21.15.47:0/1091107496 learned_addr learned my addr 172.21.15.47:0/1091107496 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:45:58.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.103+0000 7f86c94ba700 1 --2- 172.21.15.47:0/1091107496 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86c40b0e50 0x7f86c40ac290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:58.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.103+0000 7f86ca4bc700 1 --2- 172.21.15.47:0/1091107496 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f86c40b2ee0 0x7f86c40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:58.127 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.107+0000 7f86c9cbb700 1 -- 172.21.15.47:0/1091107496 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f86c40b2ee0 msgr2=0x7f86c40b4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:58.127 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.107+0000 7f86c9cbb700 1 --2- 172.21.15.47:0/1091107496 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f86c40b2ee0 0x7f86c40b4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:58.127 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.107+0000 7f86c9cbb700 1 -- 172.21.15.47:0/1091107496 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86c40b0e50 msgr2=0x7f86c40ac290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:58.127 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.107+0000 7f86c9cbb700 1 --2- 172.21.15.47:0/1091107496 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86c40b0e50 0x7f86c40ac290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:58.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.107+0000 7f86c9cbb700 1 -- 172.21.15.47:0/1091107496 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86bc009530 con 0x7f86c40ad970 2022-06-29T15:45:58.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.107+0000 7f86c9cbb700 1 --2- 172.21.15.47:0/1091107496 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86c40ad970 0x7f86c413d540 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7f86c405f6d0 tx=0x7f86c0007bd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:58.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.107+0000 7f86b2ffd700 1 -- 172.21.15.47:0/1091107496 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f86c0007030 con 0x7f86c40ad970 2022-06-29T15:45:58.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.107+0000 7f86b2ffd700 1 -- 172.21.15.47:0/1091107496 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86c0007190 con 0x7f86c40ad970 2022-06-29T15:45:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.107+0000 7f86cacbd700 1 -- 172.21.15.47:0/1091107496 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86c40aa3c0 con 0x7f86c40ad970 2022-06-29T15:45:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.107+0000 7f86b2ffd700 1 -- 172.21.15.47:0/1091107496 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f86c0006510 con 0x7f86c40ad970 2022-06-29T15:45:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.107+0000 7f86cacbd700 1 -- 172.21.15.47:0/1091107496 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86c40b4680 con 0x7f86c40ad970 2022-06-29T15:45:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.107+0000 7f86cacbd700 1 -- 172.21.15.47:0/1091107496 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86c40573d0 con 0x7f86c40ad970 2022-06-29T15:45:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.107+0000 7f86b2ffd700 1 -- 172.21.15.47:0/1091107496 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f86c0006670 con 0x7f86c40ad970 2022-06-29T15:45:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.111+0000 7f86b2ffd700 1 --2- 172.21.15.47:0/1091107496 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f86a4053aa0 0x7f86a4055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.111+0000 7f86b2ffd700 1 -- 172.21.15.47:0/1091107496 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f86c006b4b0 con 0x7f86c40ad970 2022-06-29T15:45:58.131 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.111+0000 7f86c94ba700 1 --2- 172.21.15.47:0/1091107496 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f86a4053aa0 0x7f86a4055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:58.131 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.111+0000 7f86c94ba700 1 --2- 172.21.15.47:0/1091107496 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f86a4053aa0 0x7f86a4055f60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f86b400c000 tx=0x7f86b4008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:58.131 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.115+0000 7f86b2ffd700 1 -- 172.21.15.47:0/1091107496 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f86c0042550 con 0x7f86c40ad970 2022-06-29T15:45:58.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.287+0000 7f86cacbd700 1 -- 172.21.15.47:0/1091107496 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f86c4069e20 con 0x7f86c40ad970 2022-06-29T15:45:58.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.287+0000 7f86b2ffd700 1 -- 172.21.15.47:0/1091107496 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+4255 (secure 0 0 0) 0x7f86c0045b70 con 0x7f86c40ad970 2022-06-29T15:45:58.292 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:45:58.292 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2022-06-29T15:45:58.293 INFO:teuthology.orchestra.run.smithi047.stdout: "auth_dump": [ 2022-06-29T15:45:58.293 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:58.293 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.a", 2022-06-29T15:45:58.293 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxiQJFFMxAA9Tv7FJD9u/BKvbkuKld6Kg==", 2022-06-29T15:45:58.294 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:45:58.294 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:45:58.294 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:45:58.294 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:45:58.294 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:45:58.295 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:58.295 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:58.295 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:58.296 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.b", 2022-06-29T15:45:58.296 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC2crxibt6FARAAEf2bwXKFhXKtqr1nN0rdXQ==", 2022-06-29T15:45:58.296 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:45:58.296 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:45:58.296 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:45:58.297 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:45:58.297 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:45:58.297 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:58.297 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:58.298 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:58.298 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.c", 2022-06-29T15:45:58.298 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxi/4ZMORAANpLbgTm3ehqi80USc3swGg==", 2022-06-29T15:45:58.298 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:45:58.299 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:45:58.299 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:45:58.299 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:45:58.299 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:45:58.299 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:58.300 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:58.300 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:58.300 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.0", 2022-06-29T15:45:58.300 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC2crxiY0YsMxAAZivb+CxL/w7vZwQv37Y4oQ==", 2022-06-29T15:45:58.301 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:45:58.301 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:45:58.301 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:45:58.301 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:45:58.302 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:58.302 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:58.302 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:58.302 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.1", 2022-06-29T15:45:58.303 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC5crxitLkKFxAA8+++LUUKLiO9SAuAfYgT5g==", 2022-06-29T15:45:58.303 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:45:58.303 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:45:58.303 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:45:58.303 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:45:58.304 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:58.304 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:58.304 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:58.304 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.2", 2022-06-29T15:45:58.305 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC7crxi1oM9NxAA2FfLdNt32YPwlyNdGQpaQA==", 2022-06-29T15:45:58.305 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:45:58.305 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:45:58.305 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:45:58.306 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:45:58.306 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:58.306 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:58.306 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:58.307 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.3", 2022-06-29T15:45:58.307 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC+crxiTsBgGhAA+gLBBDaskPbmRNWzgKwcHA==", 2022-06-29T15:45:58.307 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:45:58.308 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:45:58.308 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:45:58.308 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:45:58.309 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:58.309 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:58.309 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:58.309 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.4", 2022-06-29T15:45:58.310 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDDcrxivJebIhAAfwWJa7tK4hamkk1/1CSSfw==", 2022-06-29T15:45:58.310 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:45:58.310 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:45:58.310 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:45:58.311 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:45:58.311 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:58.311 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:58.311 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:58.312 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.5", 2022-06-29T15:45:58.312 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDGcrxijuozBhAAt+Dx3wH3WKp3DrbCbhmh8Q==", 2022-06-29T15:45:58.312 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:45:58.312 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:45:58.312 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:45:58.313 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:45:58.313 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:58.313 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:58.313 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:58.314 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.6", 2022-06-29T15:45:58.314 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDIcrxi91kXJxAA9JX7n98P394bD41Q9LUdvQ==", 2022-06-29T15:45:58.314 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:45:58.314 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:45:58.315 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:45:58.315 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:45:58.315 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:58.315 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:58.315 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:58.316 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.7", 2022-06-29T15:45:58.316 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDLcrxiHj0ZDRAASy6fAix8TmEZbAGtuFCfFg==", 2022-06-29T15:45:58.316 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:45:58.316 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:45:58.317 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:45:58.317 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:45:58.317 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:58.317 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:58.317 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:58.318 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.0", 2022-06-29T15:45:58.318 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC2crxiVDlfBxAAezkUfg4e2g0Rfakaa/Ek1Q==", 2022-06-29T15:45:58.318 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:45:58.319 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:45:58.319 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow r", 2022-06-29T15:45:58.319 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-29T15:45:58.320 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:58.320 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:58.320 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:58.320 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.1", 2022-06-29T15:45:58.320 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC2crxisXJ1CxAAhl07oF7AW66NdzfvvZTJmA==", 2022-06-29T15:45:58.321 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:45:58.321 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:45:58.321 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow r", 2022-06-29T15:45:58.321 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-29T15:45:58.322 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:58.322 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:58.322 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:58.322 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.admin", 2022-06-29T15:45:58.323 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxiW7HSFRAAhYqoGU9GFqj2HfUVGValSA==", 2022-06-29T15:45:58.323 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:45:58.323 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2022-06-29T15:45:58.323 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:45:58.323 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:45:58.324 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:45:58.324 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:58.324 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:58.324 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:58.325 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.x", 2022-06-29T15:45:58.325 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxidmuIMBAAcpNlYVu5+icCHg+yIjcKXw==", 2022-06-29T15:45:58.325 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:45:58.325 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2022-06-29T15:45:58.325 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2022-06-29T15:45:58.326 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:45:58.326 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:58.326 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:58.326 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:58.327 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.y", 2022-06-29T15:45:58.327 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxisSMFLBAAv03r4OoPjOP9nI9G/LK0iw==", 2022-06-29T15:45:58.327 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:45:58.327 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2022-06-29T15:45:58.328 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2022-06-29T15:45:58.328 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:45:58.328 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:58.329 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:58.329 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:45:58.329 INFO:teuthology.orchestra.run.smithi047.stdout:} 2022-06-29T15:45:58.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.291+0000 7f86cacbd700 1 -- 172.21.15.47:0/1091107496 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f86a4053aa0 msgr2=0x7f86a4055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:58.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.291+0000 7f86cacbd700 1 --2- 172.21.15.47:0/1091107496 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f86a4053aa0 0x7f86a4055f60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f86b400c000 tx=0x7f86b4008040).stop 2022-06-29T15:45:58.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.291+0000 7f86cacbd700 1 -- 172.21.15.47:0/1091107496 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86c40ad970 msgr2=0x7f86c413d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:58.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.291+0000 7f86cacbd700 1 --2- 172.21.15.47:0/1091107496 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86c40ad970 0x7f86c413d540 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7f86c405f6d0 tx=0x7f86c0007bd0).stop 2022-06-29T15:45:58.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.291+0000 7f86cacbd700 1 -- 172.21.15.47:0/1091107496 shutdown_connections 2022-06-29T15:45:58.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.291+0000 7f86cacbd700 1 --2- 172.21.15.47:0/1091107496 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f86c40b2ee0 0x7f86c40b4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:58.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.291+0000 7f86cacbd700 1 --2- 172.21.15.47:0/1091107496 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f86a4053aa0 0x7f86a4055f60 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:58.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.291+0000 7f86cacbd700 1 --2- 172.21.15.47:0/1091107496 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86c40b0e50 0x7f86c40ac290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:58.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.291+0000 7f86cacbd700 1 --2- 172.21.15.47:0/1091107496 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86c40ad970 0x7f86c413d540 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:58.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.291+0000 7f86cacbd700 1 -- 172.21.15.47:0/1091107496 >> 172.21.15.47:0/1091107496 conn(0x7f86c4063670 msgr2=0x7f86c40b00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:45:58.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.291+0000 7f86cacbd700 1 -- 172.21.15.47:0/1091107496 shutdown_connections 2022-06-29T15:45:58.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.291+0000 7f86cacbd700 1 -- 172.21.15.47:0/1091107496 wait complete. 2022-06-29T15:45:58.334 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:45:58.435 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.431+0000 7f9fe271b700 1 Processor -- start 2022-06-29T15:45:58.436 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.431+0000 7f9fe271b700 1 -- start start 2022-06-29T15:45:58.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.431+0000 7f9fe271b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9fdc0af4a0 0x7f9fdc05f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:58.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.431+0000 7f9fe271b700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fdc0afe20 con 0x7f9fdc0ac180 2022-06-29T15:45:58.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.431+0000 7f9fe271b700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fdc0b2830 con 0x7f9fdc05f0b0 2022-06-29T15:45:58.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.431+0000 7f9fe271b700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fdc0b0ea0 con 0x7f9fdc0af4a0 2022-06-29T15:45:58.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.431+0000 7f9fe0cb8700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9fdc0ac180 0x7f9fdc0b2120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33962/0 (socket says 172.21.15.47:33962) 2022-06-29T15:45:58.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.431+0000 7f9fe0cb8700 1 -- 172.21.15.47:0/3251843097 learned_addr learned my addr 172.21.15.47:0/3251843097 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:45:58.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.431+0000 7f9fdbfff700 1 --2- 172.21.15.47:0/3251843097 >> v2:172.21.15.64:3300/0 conn(0x7f9fdc0af4a0 0x7f9fdc05f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:58.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.431+0000 7f9fdaffd700 1 -- 172.21.15.47:0/3251843097 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2341351692 0 0) 0x7f9fdc0afe20 con 0x7f9fdc0ac180 2022-06-29T15:45:58.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.431+0000 7f9fdaffd700 1 -- 172.21.15.47:0/3251843097 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fc8003260 con 0x7f9fdc0ac180 2022-06-29T15:45:58.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.435+0000 7f9fdaffd700 1 -- 172.21.15.47:0/3251843097 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2374636025 0 0) 0x7f9fdc0b2830 con 0x7f9fdc05f0b0 2022-06-29T15:45:58.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.435+0000 7f9fdaffd700 1 -- 172.21.15.47:0/3251843097 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fdc0afe20 con 0x7f9fdc05f0b0 2022-06-29T15:45:58.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.435+0000 7f9fdbfff700 1 -- 172.21.15.47:0/3251843097 >> v1:172.21.15.64:6789/0 conn(0x7f9fdc05f0b0 legacy=0x7f9fdc0b0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:58.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.435+0000 7f9fdbfff700 1 -- 172.21.15.47:0/3251843097 >> v1:172.21.15.47:6789/0 conn(0x7f9fdc0ac180 legacy=0x7f9fdc0b2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:58.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.435+0000 7f9fdbfff700 1 -- 172.21.15.47:0/3251843097 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fdc143e30 con 0x7f9fdc0af4a0 2022-06-29T15:45:58.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.435+0000 7f9fdbfff700 1 --2- 172.21.15.47:0/3251843097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fdc0af4a0 0x7f9fdc05f9e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f9fcc002f70 tx=0x7f9fcc006df0).ready entity=mon.1 client_cookie=f40dbf4394dca07c server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:58.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.435+0000 7f9fdaffd700 1 -- 172.21.15.47:0/3251843097 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9fcc00d540 con 0x7f9fdc0af4a0 2022-06-29T15:45:58.446 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.435+0000 7f9fdaffd700 1 -- 172.21.15.47:0/3251843097 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fcc000e90 con 0x7f9fdc0af4a0 2022-06-29T15:45:58.446 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.435+0000 7f9fdaffd700 1 -- 172.21.15.47:0/3251843097 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9fcc00d890 con 0x7f9fdc0af4a0 2022-06-29T15:45:58.446 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.435+0000 7f9fe271b700 1 -- 172.21.15.47:0/3251843097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fdc0af4a0 msgr2=0x7f9fdc05f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:58.447 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.435+0000 7f9fe271b700 1 --2- 172.21.15.47:0/3251843097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fdc0af4a0 0x7f9fdc05f9e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f9fcc002f70 tx=0x7f9fcc006df0).stop 2022-06-29T15:45:58.447 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.435+0000 7f9fe271b700 1 -- 172.21.15.47:0/3251843097 shutdown_connections 2022-06-29T15:45:58.447 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.435+0000 7f9fe271b700 1 --2- 172.21.15.47:0/3251843097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fdc0af4a0 0x7f9fdc05f9e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:58.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.435+0000 7f9fe271b700 1 -- 172.21.15.47:0/3251843097 >> 172.21.15.47:0/3251843097 conn(0x7f9fdc0626f0 msgr2=0x7f9fdc062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:45:58.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.435+0000 7f9fe271b700 1 -- 172.21.15.47:0/3251843097 shutdown_connections 2022-06-29T15:45:58.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.435+0000 7f9fe271b700 1 -- 172.21.15.47:0/3251843097 wait complete. 2022-06-29T15:45:58.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.439+0000 7f9fe271b700 1 Processor -- start 2022-06-29T15:45:58.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.439+0000 7f9fe271b700 1 -- start start 2022-06-29T15:45:58.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.439+0000 7f9fe271b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9fdc05f0b0 0x7f9fdc0b4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:58.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.439+0000 7f9fe271b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fdc0ac180 0x7f9fdc0633b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:58.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.439+0000 7f9fe271b700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9fdc0af4a0 0x7f9fdc152b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:58.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.439+0000 7f9fe271b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc140670 con 0x7f9fdc05f0b0 2022-06-29T15:45:58.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.439+0000 7f9fe271b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc156a50 con 0x7f9fdc0ac180 2022-06-29T15:45:58.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.439+0000 7f9fe271b700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc1408a0 con 0x7f9fdc0af4a0 2022-06-29T15:45:58.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.439+0000 7f9fdbfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9fdc05f0b0 0x7f9fdc0b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:58.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.439+0000 7f9fdbfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9fdc05f0b0 0x7f9fdc0b4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48414/0 (socket says 172.21.15.47:48414) 2022-06-29T15:45:58.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.439+0000 7f9fdbfff700 1 -- 172.21.15.47:0/1604476162 learned_addr learned my addr 172.21.15.47:0/1604476162 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:45:58.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.439+0000 7f9fdb7fe700 1 --2- 172.21.15.47:0/1604476162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fdc0ac180 0x7f9fdc0633b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:58.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.439+0000 7f9fe0cb8700 1 --2- 172.21.15.47:0/1604476162 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9fdc0af4a0 0x7f9fdc152b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:58.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.439+0000 7f9fdbfff700 1 -- 172.21.15.47:0/1604476162 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9fdc0af4a0 msgr2=0x7f9fdc152b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:58.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.439+0000 7f9fdbfff700 1 --2- 172.21.15.47:0/1604476162 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9fdc0af4a0 0x7f9fdc152b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:58.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.439+0000 7f9fdbfff700 1 -- 172.21.15.47:0/1604476162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fdc0ac180 msgr2=0x7f9fdc0633b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:58.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.439+0000 7f9fdbfff700 1 --2- 172.21.15.47:0/1604476162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fdc0ac180 0x7f9fdc0633b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:58.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.439+0000 7f9fdbfff700 1 -- 172.21.15.47:0/1604476162 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fcc00b180 con 0x7f9fdc05f0b0 2022-06-29T15:45:58.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.439+0000 7f9fdbfff700 1 --2- 172.21.15.47:0/1604476162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9fdc05f0b0 0x7f9fdc0b4350 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7f9fcc002f70 tx=0x7f9fcc014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:58.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.439+0000 7f9fd97fa700 1 -- 172.21.15.47:0/1604476162 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9fcc016070 con 0x7f9fdc05f0b0 2022-06-29T15:45:58.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.439+0000 7f9fe271b700 1 -- 172.21.15.47:0/1604476162 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fdc1567b0 con 0x7f9fdc05f0b0 2022-06-29T15:45:58.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.439+0000 7f9fd97fa700 1 -- 172.21.15.47:0/1604476162 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fcc013700 con 0x7f9fdc05f0b0 2022-06-29T15:45:58.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.439+0000 7f9fe271b700 1 -- 172.21.15.47:0/1604476162 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fdc1564d0 con 0x7f9fdc05f0b0 2022-06-29T15:45:58.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.439+0000 7f9fd97fa700 1 -- 172.21.15.47:0/1604476162 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9fcc013a60 con 0x7f9fdc05f0b0 2022-06-29T15:45:58.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.443+0000 7f9fe271b700 1 -- 172.21.15.47:0/1604476162 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fdc066fd0 con 0x7f9fdc05f0b0 2022-06-29T15:45:58.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.443+0000 7f9fd97fa700 1 -- 172.21.15.47:0/1604476162 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9fcc025030 con 0x7f9fdc05f0b0 2022-06-29T15:45:58.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.443+0000 7f9fd97fa700 1 --2- 172.21.15.47:0/1604476162 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9fbc053b00 0x7f9fbc055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:58.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.443+0000 7f9fd97fa700 1 -- 172.21.15.47:0/1604476162 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f9fcc06d5c0 con 0x7f9fdc05f0b0 2022-06-29T15:45:58.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.443+0000 7f9fdb7fe700 1 --2- 172.21.15.47:0/1604476162 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9fbc053b00 0x7f9fbc055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:58.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.443+0000 7f9fdb7fe700 1 --2- 172.21.15.47:0/1604476162 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9fbc053b00 0x7f9fbc055fc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f9fd000d000 tx=0x7f9fd0007a00).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:58.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.451+0000 7f9fd97fa700 1 -- 172.21.15.47:0/1604476162 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9fcc0481a0 con 0x7f9fdc05f0b0 2022-06-29T15:45:58.584 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.579+0000 7f9fe271b700 1 -- 172.21.15.47:0/1604476162 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9fdc152780 con 0x7f9fdc05f0b0 2022-06-29T15:45:58.586 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.583+0000 7f9fd97fa700 1 -- 172.21.15.47:0/1604476162 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+13414 (secure 0 0 0) 0x7f9fcc023020 con 0x7f9fdc05f0b0 2022-06-29T15:45:58.586 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:45:58.587 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":53,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:45:57.953552+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:45:58.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.583+0000 7f9fe271b700 1 -- 172.21.15.47:0/1604476162 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9fbc053b00 msgr2=0x7f9fbc055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:58.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.583+0000 7f9fe271b700 1 --2- 172.21.15.47:0/1604476162 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9fbc053b00 0x7f9fbc055fc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f9fd000d000 tx=0x7f9fd0007a00).stop 2022-06-29T15:45:58.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.583+0000 7f9fe271b700 1 -- 172.21.15.47:0/1604476162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9fdc05f0b0 msgr2=0x7f9fdc0b4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:58.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.583+0000 7f9fe271b700 1 --2- 172.21.15.47:0/1604476162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9fdc05f0b0 0x7f9fdc0b4350 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7f9fcc002f70 tx=0x7f9fcc014000).stop 2022-06-29T15:45:58.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.583+0000 7f9fe271b700 1 -- 172.21.15.47:0/1604476162 shutdown_connections 2022-06-29T15:45:58.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.583+0000 7f9fe271b700 1 --2- 172.21.15.47:0/1604476162 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9fdc0af4a0 0x7f9fdc152b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:58.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.583+0000 7f9fe271b700 1 --2- 172.21.15.47:0/1604476162 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9fbc053b00 0x7f9fbc055fc0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:58.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.583+0000 7f9fe271b700 1 --2- 172.21.15.47:0/1604476162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fdc0ac180 0x7f9fdc0633b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:58.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.583+0000 7f9fe271b700 1 --2- 172.21.15.47:0/1604476162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9fdc05f0b0 0x7f9fdc0b4350 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:58.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.583+0000 7f9fe271b700 1 -- 172.21.15.47:0/1604476162 >> 172.21.15.47:0/1604476162 conn(0x7f9fdc0626f0 msgr2=0x7f9fdc05ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:45:58.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.583+0000 7f9fe271b700 1 -- 172.21.15.47:0/1604476162 shutdown_connections 2022-06-29T15:45:58.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.583+0000 7f9fe271b700 1 -- 172.21.15.47:0/1604476162 wait complete. 2022-06-29T15:45:58.600 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-29T15:42:16.971431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-29T15:42:25.290884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-06-29T15:45:58.601 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-29T15:45:58.714 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.711+0000 7f259ac3d700 1 Processor -- start 2022-06-29T15:45:58.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.711+0000 7f259ac3d700 1 -- start start 2022-06-29T15:45:58.719 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.711+0000 7f259ac3d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f25940ade00 0x7f25940acbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:58.719 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.711+0000 7f259ac3d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2594144a90 0x7f2594140da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:58.719 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.711+0000 7f259ac3d700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25940b0850 con 0x7f25941446b0 2022-06-29T15:45:58.720 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.711+0000 7f259ac3d700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25940ad0c0 con 0x7f2594144a90 2022-06-29T15:45:58.720 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.711+0000 7f259ac3d700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25940b0af0 con 0x7f25940ade00 2022-06-29T15:45:58.721 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.711+0000 7f2599c3b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f25941446b0 0x7f25940ad6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33976/0 (socket says 172.21.15.47:33976) 2022-06-29T15:45:58.721 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.711+0000 7f2599c3b700 1 -- 172.21.15.47:0/2494480200 learned_addr learned my addr 172.21.15.47:0/2494480200 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:45:58.721 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.711+0000 7f259a43c700 1 --2- 172.21.15.47:0/2494480200 >> v2:172.21.15.64:3300/0 conn(0x7f2594144a90 0x7f2594140da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:58.721 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.711+0000 7f259943a700 1 --2- 172.21.15.47:0/2494480200 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f25940ade00 0x7f25940acbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:58.722 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.711+0000 7f2598c39700 1 -- 172.21.15.47:0/2494480200 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2663878011 0 0) 0x7f25940b0850 con 0x7f25941446b0 2022-06-29T15:45:58.722 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.711+0000 7f2598c39700 1 -- 172.21.15.47:0/2494480200 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f257c003260 con 0x7f25941446b0 2022-06-29T15:45:58.722 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.711+0000 7f259943a700 1 -- 172.21.15.47:0/2494480200 >> v2:172.21.15.64:3300/0 conn(0x7f2594144a90 msgr2=0x7f2594140da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:45:58.722 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.711+0000 7f259943a700 1 --2- 172.21.15.47:0/2494480200 >> v2:172.21.15.64:3300/0 conn(0x7f2594144a90 0x7f2594140da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:58.723 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.711+0000 7f259943a700 1 -- 172.21.15.47:0/2494480200 >> v1:172.21.15.47:6789/0 conn(0x7f25941446b0 legacy=0x7f25940ad6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:58.723 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.711+0000 7f259943a700 1 -- 172.21.15.47:0/2494480200 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25940b0d90 con 0x7f25940ade00 2022-06-29T15:45:58.723 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.711+0000 7f259a43c700 1 --2- 172.21.15.47:0/2494480200 >> v2:172.21.15.64:3300/0 conn(0x7f2594144a90 0x7f2594140da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:45:58.723 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.715+0000 7f259943a700 1 --2- 172.21.15.47:0/2494480200 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f25940ade00 0x7f25940acbd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f2590002f70 tx=0x7f2590006df0).ready entity=mon.2 client_cookie=deb87ae8c83d3df server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:58.724 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.715+0000 7f2598c39700 1 -- 172.21.15.47:0/2494480200 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f259000d4a0 con 0x7f25940ade00 2022-06-29T15:45:58.724 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.715+0000 7f2598c39700 1 -- 172.21.15.47:0/2494480200 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2590000e90 con 0x7f25940ade00 2022-06-29T15:45:58.724 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.715+0000 7f2598c39700 1 -- 172.21.15.47:0/2494480200 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f259000d7f0 con 0x7f25940ade00 2022-06-29T15:45:58.724 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.715+0000 7f259ac3d700 1 -- 172.21.15.47:0/2494480200 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f25940ade00 msgr2=0x7f25940acbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:58.725 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.715+0000 7f259ac3d700 1 --2- 172.21.15.47:0/2494480200 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f25940ade00 0x7f25940acbd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f2590002f70 tx=0x7f2590006df0).stop 2022-06-29T15:45:58.725 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.715+0000 7f259ac3d700 1 -- 172.21.15.47:0/2494480200 shutdown_connections 2022-06-29T15:45:58.725 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.715+0000 7f259ac3d700 1 --2- 172.21.15.47:0/2494480200 >> v2:172.21.15.64:3300/0 conn(0x7f2594144a90 0x7f2594140da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:58.725 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.715+0000 7f259ac3d700 1 --2- 172.21.15.47:0/2494480200 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f25940ade00 0x7f25940acbd0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:58.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.715+0000 7f259ac3d700 1 -- 172.21.15.47:0/2494480200 >> 172.21.15.47:0/2494480200 conn(0x7f25940b5320 msgr2=0x7f25940b5700 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:45:58.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.715+0000 7f259ac3d700 1 -- 172.21.15.47:0/2494480200 shutdown_connections 2022-06-29T15:45:58.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.715+0000 7f259ac3d700 1 -- 172.21.15.47:0/2494480200 wait complete. 2022-06-29T15:45:58.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.715+0000 7f259ac3d700 1 Processor -- start 2022-06-29T15:45:58.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.715+0000 7f259ac3d700 1 -- start start 2022-06-29T15:45:58.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.719+0000 7f259ac3d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f25940ade00 0x7f25941400c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:58.731 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.719+0000 7f259ac3d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25941446b0 0x7f259413f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:58.731 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.719+0000 7f259ac3d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2594144a90 0x7f2594060c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:58.731 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.719+0000 7f259ac3d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2594133fa0 con 0x7f2594144a90 2022-06-29T15:45:58.731 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.719+0000 7f259ac3d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2594141d70 con 0x7f25941446b0 2022-06-29T15:45:58.732 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.719+0000 7f259ac3d700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25940674c0 con 0x7f25940ade00 2022-06-29T15:45:58.732 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.719+0000 7f259a43c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2594144a90 0x7f2594060c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:58.732 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.719+0000 7f259a43c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2594144a90 0x7f2594060c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48432/0 (socket says 172.21.15.47:48432) 2022-06-29T15:45:58.732 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.719+0000 7f259a43c700 1 -- 172.21.15.47:0/1399421499 learned_addr learned my addr 172.21.15.47:0/1399421499 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:45:58.732 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.719+0000 7f259943a700 1 --2- 172.21.15.47:0/1399421499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25941446b0 0x7f259413f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:58.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.719+0000 7f2599c3b700 1 --2- 172.21.15.47:0/1399421499 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f25940ade00 0x7f25941400c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:58.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.719+0000 7f259a43c700 1 -- 172.21.15.47:0/1399421499 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f25940ade00 msgr2=0x7f25941400c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:58.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.719+0000 7f259a43c700 1 --2- 172.21.15.47:0/1399421499 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f25940ade00 0x7f25941400c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:58.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.719+0000 7f259a43c700 1 -- 172.21.15.47:0/1399421499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25941446b0 msgr2=0x7f259413f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:58.734 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.719+0000 7f259a43c700 1 --2- 172.21.15.47:0/1399421499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25941446b0 0x7f259413f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:58.734 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.719+0000 7f259a43c700 1 -- 172.21.15.47:0/1399421499 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f258c009470 con 0x7f2594144a90 2022-06-29T15:45:58.734 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.719+0000 7f2599c3b700 1 --2- 172.21.15.47:0/1399421499 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f25940ade00 0x7f25941400c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:45:58.734 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.719+0000 7f259a43c700 1 --2- 172.21.15.47:0/1399421499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2594144a90 0x7f2594060c80 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7f2594067600 tx=0x7f258c0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:58.734 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.719+0000 7f2582ffd700 1 -- 172.21.15.47:0/1399421499 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f258c0096b0 con 0x7f2594144a90 2022-06-29T15:45:58.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.719+0000 7f2582ffd700 1 -- 172.21.15.47:0/1399421499 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f258c00c850 con 0x7f2594144a90 2022-06-29T15:45:58.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.719+0000 7f259ac3d700 1 -- 172.21.15.47:0/1399421499 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f259000b180 con 0x7f2594144a90 2022-06-29T15:45:58.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.719+0000 7f2582ffd700 1 -- 172.21.15.47:0/1399421499 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f258c00cbb0 con 0x7f2594144a90 2022-06-29T15:45:58.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.719+0000 7f259ac3d700 1 -- 172.21.15.47:0/1399421499 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f259413fe70 con 0x7f2594144a90 2022-06-29T15:45:58.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.719+0000 7f259ac3d700 1 -- 172.21.15.47:0/1399421499 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25940573d0 con 0x7f2594144a90 2022-06-29T15:45:58.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.719+0000 7f2582ffd700 1 -- 172.21.15.47:0/1399421499 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f258c022070 con 0x7f2594144a90 2022-06-29T15:45:58.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.723+0000 7f2582ffd700 1 --2- 172.21.15.47:0/1399421499 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2574053a70 0x7f2574055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:58.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.723+0000 7f2582ffd700 1 -- 172.21.15.47:0/1399421499 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f258c06c420 con 0x7f2594144a90 2022-06-29T15:45:58.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.723+0000 7f2599c3b700 1 --2- 172.21.15.47:0/1399421499 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2574053a70 0x7f2574055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:58.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.723+0000 7f2599c3b700 1 --2- 172.21.15.47:0/1399421499 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2574053a70 0x7f2574055f30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f258400b000 tx=0x7f2584009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:58.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.727+0000 7f2582ffd700 1 -- 172.21.15.47:0/1399421499 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f258c043520 con 0x7f2594144a90 2022-06-29T15:45:58.855 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.851+0000 7f259ac3d700 1 -- 172.21.15.47:0/1399421499 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f2594061170 con 0x7f2594144a90 2022-06-29T15:45:58.856 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.851+0000 7f2582ffd700 1 -- 172.21.15.47:0/1399421499 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v53) v1 ==== 110+0+10 (secure 0 0 0) 0x7f258c0160b0 con 0x7f2594144a90 2022-06-29T15:45:58.856 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2022-06-29T15:45:58.858 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.855+0000 7f259ac3d700 1 -- 172.21.15.47:0/1399421499 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2574053a70 msgr2=0x7f2574055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:58.858 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.855+0000 7f259ac3d700 1 --2- 172.21.15.47:0/1399421499 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2574053a70 0x7f2574055f30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f258400b000 tx=0x7f2584009040).stop 2022-06-29T15:45:58.859 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.855+0000 7f259ac3d700 1 -- 172.21.15.47:0/1399421499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2594144a90 msgr2=0x7f2594060c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:58.859 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.855+0000 7f259ac3d700 1 --2- 172.21.15.47:0/1399421499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2594144a90 0x7f2594060c80 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7f2594067600 tx=0x7f258c0091d0).stop 2022-06-29T15:45:58.859 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.855+0000 7f259ac3d700 1 -- 172.21.15.47:0/1399421499 shutdown_connections 2022-06-29T15:45:58.859 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.855+0000 7f259ac3d700 1 --2- 172.21.15.47:0/1399421499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2594144a90 0x7f2594060c80 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:58.860 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.855+0000 7f259ac3d700 1 --2- 172.21.15.47:0/1399421499 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2574053a70 0x7f2574055f30 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:58.860 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.855+0000 7f259ac3d700 1 --2- 172.21.15.47:0/1399421499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25941446b0 0x7f259413f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:58.860 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.855+0000 7f259ac3d700 1 --2- 172.21.15.47:0/1399421499 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f25940ade00 0x7f25941400c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:58.860 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.855+0000 7f259ac3d700 1 -- 172.21.15.47:0/1399421499 >> 172.21.15.47:0/1399421499 conn(0x7f25940b5320 msgr2=0x7f2594142c20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:45:58.861 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.855+0000 7f259ac3d700 1 -- 172.21.15.47:0/1399421499 shutdown_connections 2022-06-29T15:45:58.861 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.855+0000 7f259ac3d700 1 -- 172.21.15.47:0/1399421499 wait complete. 2022-06-29T15:45:58.869 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-29T15:45:58.982 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.979+0000 7f2415c78700 1 Processor -- start 2022-06-29T15:45:58.983 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.979+0000 7f2415c78700 1 -- start start 2022-06-29T15:45:58.988 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.979+0000 7f2415c78700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2410063090 0x7f24100b37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:58.988 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.979+0000 7f2415c78700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2410145000 con 0x7f24100b6b80 2022-06-29T15:45:58.989 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.979+0000 7f2415c78700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24101452a0 con 0x7f241005f5c0 2022-06-29T15:45:58.989 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.979+0000 7f2415c78700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24100b2300 con 0x7f2410063090 2022-06-29T15:45:58.989 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.979+0000 7f2414c76700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f24100b6b80 0x7f2410140f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33990/0 (socket says 172.21.15.47:33990) 2022-06-29T15:45:58.989 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.979+0000 7f2414c76700 1 -- 172.21.15.47:0/2268161711 learned_addr learned my addr 172.21.15.47:0/2268161711 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:45:58.990 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.979+0000 7f240ffff700 1 --2- 172.21.15.47:0/2268161711 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2410063090 0x7f24100b37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:58.990 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.979+0000 7f240f7fe700 1 -- 172.21.15.47:0/2268161711 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3797045619 0 0) 0x7f2410145000 con 0x7f24100b6b80 2022-06-29T15:45:58.990 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.979+0000 7f240f7fe700 1 -- 172.21.15.47:0/2268161711 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23f4003260 con 0x7f24100b6b80 2022-06-29T15:45:58.990 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.979+0000 7f240f7fe700 1 -- 172.21.15.47:0/2268161711 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3797242945 0 0) 0x7f24101452a0 con 0x7f241005f5c0 2022-06-29T15:45:58.991 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.979+0000 7f240f7fe700 1 -- 172.21.15.47:0/2268161711 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2410145000 con 0x7f241005f5c0 2022-06-29T15:45:58.991 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.979+0000 7f240f7fe700 1 -- 172.21.15.47:0/2268161711 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2087304702 0 0) 0x7f23f4003260 con 0x7f24100b6b80 2022-06-29T15:45:58.991 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.979+0000 7f240f7fe700 1 -- 172.21.15.47:0/2268161711 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2410063090 msgr2=0x7f24100b37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:45:58.992 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.979+0000 7f240f7fe700 1 --2- 172.21.15.47:0/2268161711 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2410063090 0x7f24100b37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:58.992 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.979+0000 7f240f7fe700 1 -- 172.21.15.47:0/2268161711 >> v1:172.21.15.64:6789/0 conn(0x7f241005f5c0 legacy=0x7f241013e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:58.992 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.979+0000 7f240f7fe700 1 -- 172.21.15.47:0/2268161711 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24100b3ca0 con 0x7f24100b6b80 2022-06-29T15:45:58.992 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.979+0000 7f240f7fe700 1 -- 172.21.15.47:0/2268161711 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7f2404001660 con 0x7f24100b6b80 2022-06-29T15:45:58.993 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.979+0000 7f240f7fe700 1 -- 172.21.15.47:0/2268161711 >> v1:172.21.15.47:6789/0 conn(0x7f24100b6b80 legacy=0x7f2410140f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:58.993 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.983+0000 7f240f7fe700 1 --2- 172.21.15.47:0/2268161711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23f4003910 0x7f23f4003d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:58.993 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.983+0000 7f240f7fe700 1 --2- 172.21.15.47:0/2268161711 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23f4004500 0x7f23f4004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:58.993 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.983+0000 7f240f7fe700 1 --2- 172.21.15.47:0/2268161711 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2410063090 0x7f23f4004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:58.994 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.983+0000 7f240f7fe700 1 -- 172.21.15.47:0/2268161711 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24100b2300 con 0x7f23f4003910 2022-06-29T15:45:58.994 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.983+0000 7f240f7fe700 1 -- 172.21.15.47:0/2268161711 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f400b7b0 con 0x7f23f4004500 2022-06-29T15:45:58.994 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.983+0000 7f240f7fe700 1 -- 172.21.15.47:0/2268161711 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23f400b970 con 0x7f2410063090 2022-06-29T15:45:58.995 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.983+0000 7f2414c76700 1 --2- 172.21.15.47:0/2268161711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23f4003910 0x7f23f4003d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:58.995 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.983+0000 7f240ffff700 1 --2- 172.21.15.47:0/2268161711 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23f4004500 0x7f23f4004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:58.995 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.983+0000 7f2415477700 1 --2- 172.21.15.47:0/2268161711 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2410063090 0x7f23f4004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:58.996 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.983+0000 7f2414c76700 1 -- 172.21.15.47:0/2268161711 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2410063090 msgr2=0x7f23f4004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:45:58.996 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.983+0000 7f2414c76700 1 --2- 172.21.15.47:0/2268161711 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2410063090 0x7f23f4004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:58.996 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.983+0000 7f2414c76700 1 -- 172.21.15.47:0/2268161711 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23f4004500 msgr2=0x7f23f4004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:45:58.996 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.983+0000 7f2414c76700 1 --2- 172.21.15.47:0/2268161711 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23f4004500 0x7f23f4004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:58.997 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.983+0000 7f2414c76700 1 -- 172.21.15.47:0/2268161711 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23f4003260 con 0x7f23f4003910 2022-06-29T15:45:58.997 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.983+0000 7f2414c76700 1 --2- 172.21.15.47:0/2268161711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23f4003910 0x7f23f4003d10 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7f2404001430 tx=0x7f24100b3e30).ready entity=mon.0 client_cookie=9bb60c88ab0c25de server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:58.998 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.983+0000 7f240f7fe700 1 -- 172.21.15.47:0/2268161711 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24040071a0 con 0x7f23f4003910 2022-06-29T15:45:58.998 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.983+0000 7f240f7fe700 1 -- 172.21.15.47:0/2268161711 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2404007300 con 0x7f23f4003910 2022-06-29T15:45:58.998 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.983+0000 7f240f7fe700 1 -- 172.21.15.47:0/2268161711 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f240400c8f0 con 0x7f23f4003910 2022-06-29T15:45:58.998 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.983+0000 7f2415c78700 1 -- 172.21.15.47:0/2268161711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23f4003910 msgr2=0x7f23f4003d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:58.999 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.983+0000 7f2415c78700 1 --2- 172.21.15.47:0/2268161711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23f4003910 0x7f23f4003d10 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7f2404001430 tx=0x7f24100b3e30).stop 2022-06-29T15:45:58.999 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.983+0000 7f2415477700 1 -- 172.21.15.47:0/2268161711 reap_dead start 2022-06-29T15:45:58.999 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.983+0000 7f2415c78700 1 -- 172.21.15.47:0/2268161711 shutdown_connections 2022-06-29T15:45:58.999 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.983+0000 7f2415c78700 1 -- 172.21.15.47:0/2268161711 >> 172.21.15.47:0/2268161711 conn(0x7f2410063670 msgr2=0x7f2410063a50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:45:59.000 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.983+0000 7f2415c78700 1 -- 172.21.15.47:0/2268161711 shutdown_connections 2022-06-29T15:45:59.000 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.983+0000 7f2415c78700 1 -- 172.21.15.47:0/2268161711 wait complete. 2022-06-29T15:45:59.003 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.987+0000 7f2415c78700 1 Processor -- start 2022-06-29T15:45:59.003 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.987+0000 7f2415c78700 1 -- start start 2022-06-29T15:45:59.004 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.987+0000 7f2415c78700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f241009fe50 0x7f24100a0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:59.004 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.987+0000 7f2415c78700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f24100abfb0 0x7f24100aab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:59.004 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.987+0000 7f2415c78700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24101538e0 0x7f2410153d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:59.004 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.987+0000 7f2415c78700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24100ab530 con 0x7f241009fe50 2022-06-29T15:45:59.005 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.987+0000 7f2415c78700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2410150ca0 con 0x7f24101538e0 2022-06-29T15:45:59.005 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.987+0000 7f2415c78700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2410150f10 con 0x7f24100abfb0 2022-06-29T15:45:59.005 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.987+0000 7f2414c76700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f241009fe50 0x7f24100a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:59.005 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.987+0000 7f2414c76700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f241009fe50 0x7f24100a0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48448/0 (socket says 172.21.15.47:48448) 2022-06-29T15:45:59.005 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.987+0000 7f2414c76700 1 -- 172.21.15.47:0/351708164 learned_addr learned my addr 172.21.15.47:0/351708164 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:45:59.006 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.987+0000 7f240ffff700 1 --2- 172.21.15.47:0/351708164 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f24100abfb0 0x7f24100aab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:59.006 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.987+0000 7f2415477700 1 --2- 172.21.15.47:0/351708164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24101538e0 0x7f2410153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:59.006 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.991+0000 7f2414c76700 1 -- 172.21.15.47:0/351708164 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f24100abfb0 msgr2=0x7f24100aab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:59.006 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.991+0000 7f2414c76700 1 --2- 172.21.15.47:0/351708164 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f24100abfb0 0x7f24100aab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:59.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.991+0000 7f2414c76700 1 -- 172.21.15.47:0/351708164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24101538e0 msgr2=0x7f2410153d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:59.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.991+0000 7f2414c76700 1 --2- 172.21.15.47:0/351708164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24101538e0 0x7f2410153d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:59.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.991+0000 7f2414c76700 1 -- 172.21.15.47:0/351708164 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24100ab760 con 0x7f241009fe50 2022-06-29T15:45:59.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.991+0000 7f2415477700 1 --2- 172.21.15.47:0/351708164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24101538e0 0x7f2410153d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:45:59.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.991+0000 7f2414c76700 1 --2- 172.21.15.47:0/351708164 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f241009fe50 0x7f24100a0230 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7f2404001430 tx=0x7f2404013000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:59.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.991+0000 7f240dffb700 1 -- 172.21.15.47:0/351708164 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24040071a0 con 0x7f241009fe50 2022-06-29T15:45:59.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.991+0000 7f2415c78700 1 -- 172.21.15.47:0/351708164 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2410142520 con 0x7f241009fe50 2022-06-29T15:45:59.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.991+0000 7f240dffb700 1 -- 172.21.15.47:0/351708164 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24040045e0 con 0x7f241009fe50 2022-06-29T15:45:59.009 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.991+0000 7f240dffb700 1 -- 172.21.15.47:0/351708164 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2404004940 con 0x7f241009fe50 2022-06-29T15:45:59.009 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.991+0000 7f2415c78700 1 -- 172.21.15.47:0/351708164 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24100ab070 con 0x7f241009fe50 2022-06-29T15:45:59.009 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.991+0000 7f2415c78700 1 -- 172.21.15.47:0/351708164 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24100ab8f0 con 0x7f241009fe50 2022-06-29T15:45:59.009 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.991+0000 7f240dffb700 1 -- 172.21.15.47:0/351708164 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f240401c070 con 0x7f241009fe50 2022-06-29T15:45:59.009 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.991+0000 7f240dffb700 1 --2- 172.21.15.47:0/351708164 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f23f0053a70 0x7f23f0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:59.010 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.991+0000 7f240dffb700 1 -- 172.21.15.47:0/351708164 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f240406bd30 con 0x7f241009fe50 2022-06-29T15:45:59.010 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.991+0000 7f240ffff700 1 --2- 172.21.15.47:0/351708164 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f23f0053a70 0x7f23f0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:59.010 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.991+0000 7f240ffff700 1 --2- 172.21.15.47:0/351708164 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f23f0053a70 0x7f23f0055f30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7f23fc00b000 tx=0x7f23fc006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:59.010 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:58.999+0000 7f240dffb700 1 -- 172.21.15.47:0/351708164 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2404042d90 con 0x7f241009fe50 2022-06-29T15:45:59.134 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.131+0000 7f2415c78700 1 -- 172.21.15.47:0/351708164 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f24100573d0 con 0x7f241009fe50 2022-06-29T15:45:59.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.131+0000 7f240dffb700 1 -- 172.21.15.47:0/351708164 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v53) v1 ==== 92+0+10 (secure 0 0 0) 0x7f24040463b0 con 0x7f241009fe50 2022-06-29T15:45:59.135 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:45:59.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.131+0000 7f2415c78700 1 -- 172.21.15.47:0/351708164 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f23f0053a70 msgr2=0x7f23f0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:59.138 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.135+0000 7f2415c78700 1 --2- 172.21.15.47:0/351708164 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f23f0053a70 0x7f23f0055f30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7f23fc00b000 tx=0x7f23fc006df0).stop 2022-06-29T15:45:59.138 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.135+0000 7f2415c78700 1 -- 172.21.15.47:0/351708164 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f241009fe50 msgr2=0x7f24100a0230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:59.138 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.135+0000 7f2415c78700 1 --2- 172.21.15.47:0/351708164 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f241009fe50 0x7f24100a0230 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7f2404001430 tx=0x7f2404013000).stop 2022-06-29T15:45:59.139 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.135+0000 7f2415c78700 1 -- 172.21.15.47:0/351708164 shutdown_connections 2022-06-29T15:45:59.139 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.135+0000 7f2415c78700 1 --2- 172.21.15.47:0/351708164 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f23f0053a70 0x7f23f0055f30 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:59.139 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.135+0000 7f2415c78700 1 --2- 172.21.15.47:0/351708164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24101538e0 0x7f2410153d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:59.139 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.135+0000 7f2415c78700 1 --2- 172.21.15.47:0/351708164 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f24100abfb0 0x7f24100aab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:59.140 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.135+0000 7f2415c78700 1 --2- 172.21.15.47:0/351708164 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f241009fe50 0x7f24100a0230 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:59.140 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.135+0000 7f2415c78700 1 -- 172.21.15.47:0/351708164 >> 172.21.15.47:0/351708164 conn(0x7f2410063670 msgr2=0x7f24100b5870 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:45:59.140 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.135+0000 7f2415c78700 1 -- 172.21.15.47:0/351708164 shutdown_connections 2022-06-29T15:45:59.140 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.135+0000 7f2415c78700 1 -- 172.21.15.47:0/351708164 wait complete. 2022-06-29T15:45:59.149 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-06-29T15:45:59.249 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.247+0000 7fdbc0dd2700 1 Processor -- start 2022-06-29T15:45:59.250 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.247+0000 7fdbc0dd2700 1 -- start start 2022-06-29T15:45:59.253 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.247+0000 7fdbc0dd2700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdbbc0acff0 0x7fdbbc05fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:59.253 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.247+0000 7fdbc0dd2700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbbc0a97e0 con 0x7fdbbc1437d0 2022-06-29T15:45:59.253 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.247+0000 7fdbc0dd2700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbbc0b1d80 con 0x7fdbbc0b19a0 2022-06-29T15:45:59.254 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.247+0000 7fdbc0dd2700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbbc05f430 con 0x7fdbbc0acff0 2022-06-29T15:45:59.254 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.247+0000 7fdbbaffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdbbc0acff0 0x7fdbbc05fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:59.254 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.247+0000 7fdbbaffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdbbc0acff0 0x7fdbbc05fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48458/0 (socket says 172.21.15.47:48458) 2022-06-29T15:45:59.255 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.247+0000 7fdbbbfff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fdbbc1437d0 0x7fdbbc143c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34014/0 (socket says 172.21.15.47:34014) 2022-06-29T15:45:59.255 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.247+0000 7fdbbaffd700 1 -- 172.21.15.47:0/4028734118 learned_addr learned my addr 172.21.15.47:0/4028734118 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:45:59.255 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.247+0000 7fdbbaffd700 1 -- 172.21.15.47:0/4028734118 >> v1:172.21.15.64:6789/0 conn(0x7fdbbc0b19a0 legacy=0x7fdbbc0ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:59.256 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.247+0000 7fdbbaffd700 1 -- 172.21.15.47:0/4028734118 >> v1:172.21.15.47:6789/0 conn(0x7fdbbc1437d0 legacy=0x7fdbbc143c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:59.256 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.247+0000 7fdbba7fc700 1 -- 172.21.15.47:0/4028734118 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1221337656 0 0) 0x7fdbbc0a97e0 con 0x7fdbbc1437d0 2022-06-29T15:45:59.256 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.247+0000 7fdbbaffd700 1 -- 172.21.15.47:0/4028734118 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbbc0a92a0 con 0x7fdbbc0acff0 2022-06-29T15:45:59.256 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.247+0000 7fdbbaffd700 1 --2- 172.21.15.47:0/4028734118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdbbc0acff0 0x7fdbbc05fd10 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7fdbb0002f70 tx=0x7fdbb0006f40).ready entity=mon.0 client_cookie=645bff431638c18e server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:59.257 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.247+0000 7fdbba7fc700 1 -- 172.21.15.47:0/4028734118 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdbb000dc70 con 0x7fdbbc0acff0 2022-06-29T15:45:59.257 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.247+0000 7fdbba7fc700 1 -- 172.21.15.47:0/4028734118 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbb0000e90 con 0x7fdbbc0acff0 2022-06-29T15:45:59.257 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.247+0000 7fdbba7fc700 1 -- 172.21.15.47:0/4028734118 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdbb000be90 con 0x7fdbbc0acff0 2022-06-29T15:45:59.257 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.247+0000 7fdbc0dd2700 1 -- 172.21.15.47:0/4028734118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdbbc0acff0 msgr2=0x7fdbbc05fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:59.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.247+0000 7fdbc0dd2700 1 --2- 172.21.15.47:0/4028734118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdbbc0acff0 0x7fdbbc05fd10 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7fdbb0002f70 tx=0x7fdbb0006f40).stop 2022-06-29T15:45:59.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.247+0000 7fdbc0dd2700 1 -- 172.21.15.47:0/4028734118 shutdown_connections 2022-06-29T15:45:59.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.247+0000 7fdbc0dd2700 1 --2- 172.21.15.47:0/4028734118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdbbc0acff0 0x7fdbbc05fd10 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:59.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.247+0000 7fdbc0dd2700 1 -- 172.21.15.47:0/4028734118 >> 172.21.15.47:0/4028734118 conn(0x7fdbbc062df0 msgr2=0x7fdbbc0631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:45:59.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.247+0000 7fdbc0dd2700 1 -- 172.21.15.47:0/4028734118 shutdown_connections 2022-06-29T15:45:59.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.247+0000 7fdbc0dd2700 1 -- 172.21.15.47:0/4028734118 wait complete. 2022-06-29T15:45:59.262 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.251+0000 7fdbc0dd2700 1 Processor -- start 2022-06-29T15:45:59.262 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.251+0000 7fdbc0dd2700 1 -- start start 2022-06-29T15:45:59.263 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.251+0000 7fdbc0dd2700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdbbc0acff0 0x7fdbbc0b4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:59.263 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.251+0000 7fdbc0dd2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdbbc0b19a0 0x7fdbbc09b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:59.263 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.251+0000 7fdbc0dd2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbbc1437d0 0x7fdbbc14dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:59.263 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.251+0000 7fdbc0dd2700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbbc067770 con 0x7fdbbc0b19a0 2022-06-29T15:45:59.264 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.251+0000 7fdbc0dd2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbbc067a10 con 0x7fdbbc1437d0 2022-06-29T15:45:59.264 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.251+0000 7fdbc0dd2700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbbc09ae90 con 0x7fdbbc0acff0 2022-06-29T15:45:59.264 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.251+0000 7fdbbaffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdbbc0b19a0 0x7fdbbc09b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:59.264 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.251+0000 7fdbbaffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdbbc0b19a0 0x7fdbbc09b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48464/0 (socket says 172.21.15.47:48464) 2022-06-29T15:45:59.265 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.251+0000 7fdbbaffd700 1 -- 172.21.15.47:0/2153275515 learned_addr learned my addr 172.21.15.47:0/2153275515 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:45:59.265 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.251+0000 7fdbbb7fe700 1 --2- 172.21.15.47:0/2153275515 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdbbc0acff0 0x7fdbbc0b4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:59.265 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.251+0000 7fdbbbfff700 1 --2- 172.21.15.47:0/2153275515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbbc1437d0 0x7fdbbc14dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:59.265 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.251+0000 7fdbbaffd700 1 -- 172.21.15.47:0/2153275515 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdbbc0acff0 msgr2=0x7fdbbc0b4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:59.266 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.251+0000 7fdbbaffd700 1 --2- 172.21.15.47:0/2153275515 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdbbc0acff0 0x7fdbbc0b4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:59.266 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.251+0000 7fdbbaffd700 1 -- 172.21.15.47:0/2153275515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbbc1437d0 msgr2=0x7fdbbc14dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:59.266 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.251+0000 7fdbbaffd700 1 --2- 172.21.15.47:0/2153275515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbbc1437d0 0x7fdbbc14dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:59.266 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.251+0000 7fdbbaffd700 1 -- 172.21.15.47:0/2153275515 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb000b460 con 0x7fdbbc0b19a0 2022-06-29T15:45:59.267 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.251+0000 7fdbbb7fe700 1 --2- 172.21.15.47:0/2153275515 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdbbc0acff0 0x7fdbbc0b4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:45:59.267 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.251+0000 7fdbbbfff700 1 --2- 172.21.15.47:0/2153275515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbbc1437d0 0x7fdbbc14dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:45:59.267 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.251+0000 7fdbbaffd700 1 --2- 172.21.15.47:0/2153275515 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdbbc0b19a0 0x7fdbbc09b5d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7fdbb0002f70 tx=0x7fdbb0006f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:59.267 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.251+0000 7fdbb8ff9700 1 -- 172.21.15.47:0/2153275515 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdbb0024450 con 0x7fdbbc0b19a0 2022-06-29T15:45:59.268 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.251+0000 7fdbb8ff9700 1 -- 172.21.15.47:0/2153275515 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbb00245b0 con 0x7fdbbc0b19a0 2022-06-29T15:45:59.268 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.251+0000 7fdbc0dd2700 1 -- 172.21.15.47:0/2153275515 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbbc09bac0 con 0x7fdbbc0b19a0 2022-06-29T15:45:59.268 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.251+0000 7fdbb8ff9700 1 -- 172.21.15.47:0/2153275515 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdbb0024910 con 0x7fdbbc0b19a0 2022-06-29T15:45:59.268 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.251+0000 7fdbc0dd2700 1 -- 172.21.15.47:0/2153275515 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbbc0982f0 con 0x7fdbbc0b19a0 2022-06-29T15:45:59.269 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.255+0000 7fdbc0dd2700 1 -- 172.21.15.47:0/2153275515 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbbc0573d0 con 0x7fdbbc0b19a0 2022-06-29T15:45:59.269 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.255+0000 7fdbb8ff9700 1 -- 172.21.15.47:0/2153275515 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fdbb0021070 con 0x7fdbbc0b19a0 2022-06-29T15:45:59.269 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.255+0000 7fdbb8ff9700 1 --2- 172.21.15.47:0/2153275515 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdb9c0539e0 0x7fdb9c055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:59.269 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.255+0000 7fdbb8ff9700 1 -- 172.21.15.47:0/2153275515 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fdbb006f840 con 0x7fdbbc0b19a0 2022-06-29T15:45:59.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.255+0000 7fdbbb7fe700 1 --2- 172.21.15.47:0/2153275515 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdb9c0539e0 0x7fdb9c055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:59.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.255+0000 7fdbbb7fe700 1 --2- 172.21.15.47:0/2153275515 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdb9c0539e0 0x7fdb9c055ea0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7fdbac00b000 tx=0x7fdbac009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:59.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.259+0000 7fdbb8ff9700 1 -- 172.21.15.47:0/2153275515 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdbb0046940 con 0x7fdbbc0b19a0 2022-06-29T15:45:59.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.387+0000 7fdbc0dd2700 1 -- 172.21.15.47:0/2153275515 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fdbbc06a0f0 con 0x7fdbbc0b19a0 2022-06-29T15:45:59.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.387+0000 7fdbb8ff9700 1 -- 172.21.15.47:0/2153275515 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v53) v1 ==== 84+0+143 (secure 0 0 0) 0x7fdbb0049f60 con 0x7fdbbc0b19a0 2022-06-29T15:45:59.391 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:45:59.392 INFO:teuthology.orchestra.run.smithi047.stdout:[ 2022-06-29T15:45:59.392 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:59.392 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 1, 2022-06-29T15:45:59.393 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": "device_health_metrics" 2022-06-29T15:45:59.393 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:45:59.393 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:45:59.393 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 2, 2022-06-29T15:45:59.394 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": "rbd" 2022-06-29T15:45:59.394 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:45:59.394 INFO:teuthology.orchestra.run.smithi047.stdout:] 2022-06-29T15:45:59.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.387+0000 7fdbc0dd2700 1 -- 172.21.15.47:0/2153275515 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdb9c0539e0 msgr2=0x7fdb9c055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:59.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.387+0000 7fdbc0dd2700 1 --2- 172.21.15.47:0/2153275515 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdb9c0539e0 0x7fdb9c055ea0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7fdbac00b000 tx=0x7fdbac009040).stop 2022-06-29T15:45:59.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.387+0000 7fdbc0dd2700 1 -- 172.21.15.47:0/2153275515 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdbbc0b19a0 msgr2=0x7fdbbc09b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:59.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.387+0000 7fdbc0dd2700 1 --2- 172.21.15.47:0/2153275515 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdbbc0b19a0 0x7fdbbc09b5d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7fdbb0002f70 tx=0x7fdbb0006f40).stop 2022-06-29T15:45:59.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.391+0000 7fdbc0dd2700 1 -- 172.21.15.47:0/2153275515 shutdown_connections 2022-06-29T15:45:59.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.391+0000 7fdbc0dd2700 1 --2- 172.21.15.47:0/2153275515 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdb9c0539e0 0x7fdb9c055ea0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:59.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.391+0000 7fdbc0dd2700 1 --2- 172.21.15.47:0/2153275515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbbc1437d0 0x7fdbbc14dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:59.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.391+0000 7fdbc0dd2700 1 --2- 172.21.15.47:0/2153275515 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdbbc0b19a0 0x7fdbbc09b5d0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:59.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.391+0000 7fdbc0dd2700 1 --2- 172.21.15.47:0/2153275515 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdbbc0acff0 0x7fdbbc0b4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:59.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.391+0000 7fdbc0dd2700 1 -- 172.21.15.47:0/2153275515 >> 172.21.15.47:0/2153275515 conn(0x7fdbbc062df0 msgr2=0x7fdbbc0b0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:45:59.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.391+0000 7fdbc0dd2700 1 -- 172.21.15.47:0/2153275515 shutdown_connections 2022-06-29T15:45:59.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.391+0000 7fdbc0dd2700 1 -- 172.21.15.47:0/2153275515 wait complete. 2022-06-29T15:45:59.404 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-06-29T15:45:59.405 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-06-29T15:45:59.529 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.511+0000 7fcd31e4c700 1 Processor -- start 2022-06-29T15:45:59.530 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.511+0000 7fcd31e4c700 1 -- start start 2022-06-29T15:45:59.530 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.515+0000 7fcd31e4c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcd2c098590 0x7fcd2c0b2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:59.530 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.515+0000 7fcd31e4c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcd2c05c830 0x7fcd2c0b0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:59.531 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.515+0000 7fcd31e4c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcd2c0b14d0 0x7fcd2c096250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:59.531 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.515+0000 7fcd31e4c700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd2c067160 con 0x7fcd2c0b14d0 2022-06-29T15:45:59.531 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.515+0000 7fcd31e4c700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd2c098970 con 0x7fcd2c098590 2022-06-29T15:45:59.531 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.515+0000 7fcd31e4c700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2c067390 con 0x7fcd2c05c830 2022-06-29T15:45:59.532 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.515+0000 7fcd2bfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcd2c0b14d0 0x7fcd2c096250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:59.532 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.515+0000 7fcd2bfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcd2c0b14d0 0x7fcd2c096250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48474/0 (socket says 172.21.15.47:48474) 2022-06-29T15:45:59.533 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.515+0000 7fcd2bfff700 1 -- 172.21.15.47:0/978987399 learned_addr learned my addr 172.21.15.47:0/978987399 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:45:59.533 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.515+0000 7fcd2b7fe700 1 --2- 172.21.15.47:0/978987399 >> v2:172.21.15.64:3300/0 conn(0x7fcd2c098590 0x7fcd2c0b2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:59.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.515+0000 7fcd2affd700 1 --2- 172.21.15.47:0/978987399 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcd2c05c830 0x7fcd2c0b0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:59.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.515+0000 7fcd2bfff700 1 -- 172.21.15.47:0/978987399 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcd2c05c830 msgr2=0x7fcd2c0b0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:45:59.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.515+0000 7fcd2bfff700 1 --2- 172.21.15.47:0/978987399 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcd2c05c830 0x7fcd2c0b0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:59.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.515+0000 7fcd2bfff700 1 -- 172.21.15.47:0/978987399 >> v2:172.21.15.64:3300/0 conn(0x7fcd2c098590 msgr2=0x7fcd2c0b2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:45:59.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.515+0000 7fcd2bfff700 1 --2- 172.21.15.47:0/978987399 >> v2:172.21.15.64:3300/0 conn(0x7fcd2c098590 0x7fcd2c0b2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:59.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.515+0000 7fcd2bfff700 1 -- 172.21.15.47:0/978987399 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd2c067630 con 0x7fcd2c0b14d0 2022-06-29T15:45:59.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.515+0000 7fcd2b7fe700 1 --2- 172.21.15.47:0/978987399 >> v2:172.21.15.64:3300/0 conn(0x7fcd2c098590 0x7fcd2c0b2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:45:59.536 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.515+0000 7fcd2affd700 1 --2- 172.21.15.47:0/978987399 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcd2c05c830 0x7fcd2c0b0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:45:59.536 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.515+0000 7fcd2bfff700 1 --2- 172.21.15.47:0/978987399 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd2c0b14d0 0x7fcd2c096250 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7fcd240032f0 tx=0x7fcd24009290).ready entity=mon.0 client_cookie=10a3a2c20b96a4cc server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:59.536 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.515+0000 7fcd2a7fc700 1 -- 172.21.15.47:0/978987399 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd2400b870 con 0x7fcd2c0b14d0 2022-06-29T15:45:59.536 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.515+0000 7fcd2a7fc700 1 -- 172.21.15.47:0/978987399 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd2400b9d0 con 0x7fcd2c0b14d0 2022-06-29T15:45:59.537 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.515+0000 7fcd2a7fc700 1 -- 172.21.15.47:0/978987399 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd2400bb40 con 0x7fcd2c0b14d0 2022-06-29T15:45:59.537 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.515+0000 7fcd31e4c700 1 -- 172.21.15.47:0/978987399 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd2c0b14d0 msgr2=0x7fcd2c096250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:59.537 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.515+0000 7fcd31e4c700 1 --2- 172.21.15.47:0/978987399 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd2c0b14d0 0x7fcd2c096250 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7fcd240032f0 tx=0x7fcd24009290).stop 2022-06-29T15:45:59.538 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.515+0000 7fcd31e4c700 1 -- 172.21.15.47:0/978987399 shutdown_connections 2022-06-29T15:45:59.538 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.515+0000 7fcd31e4c700 1 --2- 172.21.15.47:0/978987399 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd2c0b14d0 0x7fcd2c096250 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:59.538 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.515+0000 7fcd31e4c700 1 --2- 172.21.15.47:0/978987399 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcd2c05c830 0x7fcd2c0b0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:59.539 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.515+0000 7fcd31e4c700 1 --2- 172.21.15.47:0/978987399 >> v2:172.21.15.64:3300/0 conn(0x7fcd2c098590 0x7fcd2c0b2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:59.539 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.515+0000 7fcd31e4c700 1 -- 172.21.15.47:0/978987399 >> 172.21.15.47:0/978987399 conn(0x7fcd2c05f0b0 msgr2=0x7fcd2c05f490 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:45:59.539 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.515+0000 7fcd31e4c700 1 -- 172.21.15.47:0/978987399 shutdown_connections 2022-06-29T15:45:59.539 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.515+0000 7fcd31e4c700 1 -- 172.21.15.47:0/978987399 wait complete. 2022-06-29T15:45:59.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.519+0000 7fcd31e4c700 1 Processor -- start 2022-06-29T15:45:59.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.519+0000 7fcd31e4c700 1 -- start start 2022-06-29T15:45:59.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.523+0000 7fcd31e4c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcd2c05c830 0x7fcd2c0ab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:59.541 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.523+0000 7fcd31e4c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd2c098590 0x7fcd2c0aec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:59.541 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.523+0000 7fcd31e4c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd2c0b14d0 0x7fcd2c0af140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:59.541 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.523+0000 7fcd31e4c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2c0b3ed0 con 0x7fcd2c098590 2022-06-29T15:45:59.542 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.523+0000 7fcd31e4c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2c0b0290 con 0x7fcd2c0b14d0 2022-06-29T15:45:59.542 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.523+0000 7fcd31e4c700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2c0abf50 con 0x7fcd2c05c830 2022-06-29T15:45:59.542 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.523+0000 7fcd2affd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd2c098590 0x7fcd2c0aec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:59.542 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.523+0000 7fcd2affd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd2c098590 0x7fcd2c0aec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48478/0 (socket says 172.21.15.47:48478) 2022-06-29T15:45:59.543 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.523+0000 7fcd2affd700 1 -- 172.21.15.47:0/1782307528 learned_addr learned my addr 172.21.15.47:0/1782307528 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:45:59.543 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.523+0000 7fcd2b7fe700 1 --2- 172.21.15.47:0/1782307528 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcd2c05c830 0x7fcd2c0ab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:59.544 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.523+0000 7fcd2bfff700 1 --2- 172.21.15.47:0/1782307528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd2c0b14d0 0x7fcd2c0af140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:59.544 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.523+0000 7fcd2affd700 1 -- 172.21.15.47:0/1782307528 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcd2c05c830 msgr2=0x7fcd2c0ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:59.544 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.523+0000 7fcd2affd700 1 --2- 172.21.15.47:0/1782307528 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcd2c05c830 0x7fcd2c0ab6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:59.545 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.523+0000 7fcd2affd700 1 -- 172.21.15.47:0/1782307528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd2c0b14d0 msgr2=0x7fcd2c0af140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:45:59.545 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.523+0000 7fcd2affd700 1 --2- 172.21.15.47:0/1782307528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd2c0b14d0 0x7fcd2c0af140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:45:59.545 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.523+0000 7fcd2affd700 1 -- 172.21.15.47:0/1782307528 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd24009530 con 0x7fcd2c098590 2022-06-29T15:45:59.545 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.523+0000 7fcd2affd700 1 --2- 172.21.15.47:0/1782307528 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd2c098590 0x7fcd2c0aec50 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7fcd2c0b3880 tx=0x7fcd20008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:59.546 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.523+0000 7fcd28ff9700 1 -- 172.21.15.47:0/1782307528 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd20011780 con 0x7fcd2c098590 2022-06-29T15:45:59.546 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.523+0000 7fcd28ff9700 1 -- 172.21.15.47:0/1782307528 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd200118e0 con 0x7fcd2c098590 2022-06-29T15:45:59.546 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.523+0000 7fcd31e4c700 1 -- 172.21.15.47:0/1782307528 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd2c0ac1f0 con 0x7fcd2c098590 2022-06-29T15:45:59.546 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.523+0000 7fcd28ff9700 1 -- 172.21.15.47:0/1782307528 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd20011c40 con 0x7fcd2c098590 2022-06-29T15:45:59.547 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.523+0000 7fcd31e4c700 1 -- 172.21.15.47:0/1782307528 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd2c09b270 con 0x7fcd2c098590 2022-06-29T15:45:59.547 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.523+0000 7fcd31e4c700 1 -- 172.21.15.47:0/1782307528 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd2c0ae9f0 con 0x7fcd2c098590 2022-06-29T15:45:59.547 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.523+0000 7fcd28ff9700 1 -- 172.21.15.47:0/1782307528 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fcd2001c460 con 0x7fcd2c098590 2022-06-29T15:45:59.547 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.523+0000 7fcd28ff9700 1 --2- 172.21.15.47:0/1782307528 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcd0c053b00 0x7fcd0c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:45:59.548 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.527+0000 7fcd2b7fe700 1 --2- 172.21.15.47:0/1782307528 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcd0c053b00 0x7fcd0c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:45:59.548 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.527+0000 7fcd28ff9700 1 -- 172.21.15.47:0/1782307528 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fcd2006bfc0 con 0x7fcd2c098590 2022-06-29T15:45:59.548 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.527+0000 7fcd2b7fe700 1 --2- 172.21.15.47:0/1782307528 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcd0c053b00 0x7fcd0c055fc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7fcd1c00c000 tx=0x7fcd1c008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:45:59.549 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.531+0000 7fcd28ff9700 1 -- 172.21.15.47:0/1782307528 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcd20043060 con 0x7fcd2c098590 2022-06-29T15:45:59.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:45:59.655+0000 7fcd31e4c700 1 -- 172.21.15.47:0/1782307528 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fcd2c0573d0 con 0x7fcd2c098590 2022-06-29T15:46:00.247 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.243+0000 7fcd28ff9700 1 -- 172.21.15.47:0/1782307528 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v54) v1 ==== 133+0+0 (secure 0 0 0) 0x7fcd20046680 con 0x7fcd2c098590 2022-06-29T15:46:00.253 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.247+0000 7fcd31e4c700 1 -- 172.21.15.47:0/1782307528 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcd0c053b00 msgr2=0x7fcd0c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:00.254 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.247+0000 7fcd31e4c700 1 --2- 172.21.15.47:0/1782307528 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcd0c053b00 0x7fcd0c055fc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7fcd1c00c000 tx=0x7fcd1c008040).stop 2022-06-29T15:46:00.254 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.247+0000 7fcd31e4c700 1 -- 172.21.15.47:0/1782307528 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd2c098590 msgr2=0x7fcd2c0aec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:00.255 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.247+0000 7fcd31e4c700 1 --2- 172.21.15.47:0/1782307528 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd2c098590 0x7fcd2c0aec50 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7fcd2c0b3880 tx=0x7fcd20008040).stop 2022-06-29T15:46:00.255 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.247+0000 7fcd31e4c700 1 -- 172.21.15.47:0/1782307528 shutdown_connections 2022-06-29T15:46:00.256 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.247+0000 7fcd31e4c700 1 --2- 172.21.15.47:0/1782307528 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcd0c053b00 0x7fcd0c055fc0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:00.257 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.247+0000 7fcd31e4c700 1 --2- 172.21.15.47:0/1782307528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd2c0b14d0 0x7fcd2c0af140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:00.257 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.247+0000 7fcd31e4c700 1 --2- 172.21.15.47:0/1782307528 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd2c098590 0x7fcd2c0aec50 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:00.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.247+0000 7fcd31e4c700 1 --2- 172.21.15.47:0/1782307528 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcd2c05c830 0x7fcd2c0ab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:00.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.247+0000 7fcd31e4c700 1 -- 172.21.15.47:0/1782307528 >> 172.21.15.47:0/1782307528 conn(0x7fcd2c05f0b0 msgr2=0x7fcd2c060370 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:00.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.251+0000 7fcd31e4c700 1 -- 172.21.15.47:0/1782307528 shutdown_connections 2022-06-29T15:46:00.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.251+0000 7fcd31e4c700 1 -- 172.21.15.47:0/1782307528 wait complete. 2022-06-29T15:46:00.260 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' created 2022-06-29T15:46:00.268 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-06-29T15:46:00.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.375+0000 7ff1f640e700 1 Processor -- start 2022-06-29T15:46:00.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.375+0000 7ff1f640e700 1 -- start start 2022-06-29T15:46:00.380 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.375+0000 7ff1f640e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff1f00b58d0 0x7ff1f00b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:00.380 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.375+0000 7ff1f640e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff1f00b1610 0x7ff1f01475f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:00.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.375+0000 7ff1f640e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff1f014b4c0 0x7ff1f00aa130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:00.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.375+0000 7ff1f640e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1f00a99e0 con 0x7ff1f00b58d0 2022-06-29T15:46:00.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.375+0000 7ff1f640e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1f0060790 con 0x7ff1f00b1610 2022-06-29T15:46:00.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.375+0000 7ff1f640e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f014ad80 con 0x7ff1f014b4c0 2022-06-29T15:46:00.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.375+0000 7ff1f540c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff1f00b58d0 0x7ff1f00b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:00.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.375+0000 7ff1f540c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff1f00b58d0 0x7ff1f00b5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48484/0 (socket says 172.21.15.47:48484) 2022-06-29T15:46:00.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.375+0000 7ff1f540c700 1 -- 172.21.15.47:0/1177888873 learned_addr learned my addr 172.21.15.47:0/1177888873 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:46:00.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.375+0000 7ff1f4c0b700 1 --2- 172.21.15.47:0/1177888873 >> v2:172.21.15.64:3300/0 conn(0x7ff1f00b1610 0x7ff1f01475f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:00.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.375+0000 7ff1f5c0d700 1 --2- 172.21.15.47:0/1177888873 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff1f014b4c0 0x7ff1f00aa130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:00.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.375+0000 7ff1f540c700 1 -- 172.21.15.47:0/1177888873 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff1f014b4c0 msgr2=0x7ff1f00aa130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:46:00.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.375+0000 7ff1f540c700 1 --2- 172.21.15.47:0/1177888873 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff1f014b4c0 0x7ff1f00aa130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:00.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.375+0000 7ff1f540c700 1 -- 172.21.15.47:0/1177888873 >> v2:172.21.15.64:3300/0 conn(0x7ff1f00b1610 msgr2=0x7ff1f01475f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:46:00.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.375+0000 7ff1f540c700 1 --2- 172.21.15.47:0/1177888873 >> v2:172.21.15.64:3300/0 conn(0x7ff1f00b1610 0x7ff1f01475f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:00.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.375+0000 7ff1f540c700 1 -- 172.21.15.47:0/1177888873 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f014b020 con 0x7ff1f00b58d0 2022-06-29T15:46:00.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.375+0000 7ff1f5c0d700 1 --2- 172.21.15.47:0/1177888873 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff1f014b4c0 0x7ff1f00aa130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:46:00.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.375+0000 7ff1f540c700 1 --2- 172.21.15.47:0/1177888873 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1f00b58d0 0x7ff1f00b5cb0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7ff1ec002f70 tx=0x7ff1ec006eb0).ready entity=mon.0 client_cookie=ce5a1ee9c3e1d026 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:00.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.379+0000 7ff1e3fff700 1 -- 172.21.15.47:0/1177888873 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff1ec00d570 con 0x7ff1f00b58d0 2022-06-29T15:46:00.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.379+0000 7ff1e3fff700 1 -- 172.21.15.47:0/1177888873 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1ec000e90 con 0x7ff1f00b58d0 2022-06-29T15:46:00.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.379+0000 7ff1e3fff700 1 -- 172.21.15.47:0/1177888873 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff1ec00d900 con 0x7ff1f00b58d0 2022-06-29T15:46:00.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.379+0000 7ff1f640e700 1 -- 172.21.15.47:0/1177888873 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1f00b58d0 msgr2=0x7ff1f00b5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:00.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.379+0000 7ff1f640e700 1 --2- 172.21.15.47:0/1177888873 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1f00b58d0 0x7ff1f00b5cb0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7ff1ec002f70 tx=0x7ff1ec006eb0).stop 2022-06-29T15:46:00.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.379+0000 7ff1f640e700 1 -- 172.21.15.47:0/1177888873 shutdown_connections 2022-06-29T15:46:00.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.379+0000 7ff1f640e700 1 --2- 172.21.15.47:0/1177888873 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff1f014b4c0 0x7ff1f00aa130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:00.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.379+0000 7ff1f640e700 1 --2- 172.21.15.47:0/1177888873 >> v2:172.21.15.64:3300/0 conn(0x7ff1f00b1610 0x7ff1f01475f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:00.394 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.379+0000 7ff1f640e700 1 --2- 172.21.15.47:0/1177888873 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1f00b58d0 0x7ff1f00b5cb0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:00.394 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.379+0000 7ff1f640e700 1 -- 172.21.15.47:0/1177888873 >> 172.21.15.47:0/1177888873 conn(0x7ff1f0062560 msgr2=0x7ff1f0062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:00.394 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.379+0000 7ff1f640e700 1 -- 172.21.15.47:0/1177888873 shutdown_connections 2022-06-29T15:46:00.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.379+0000 7ff1f640e700 1 -- 172.21.15.47:0/1177888873 wait complete. 2022-06-29T15:46:00.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.379+0000 7ff1f640e700 1 Processor -- start 2022-06-29T15:46:00.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.379+0000 7ff1f640e700 1 -- start start 2022-06-29T15:46:00.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.379+0000 7ff1f640e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1f00b1610 0x7ff1f00ac6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:00.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.379+0000 7ff1f640e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1f00b58d0 0x7ff1f0063a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:00.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.379+0000 7ff1f640e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff1f014b4c0 0x7ff1f00ae660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:00.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.379+0000 7ff1f640e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f0066e90 con 0x7ff1f00b58d0 2022-06-29T15:46:00.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.379+0000 7ff1f640e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f00acc30 con 0x7ff1f00b1610 2022-06-29T15:46:00.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.379+0000 7ff1f640e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f00ad100 con 0x7ff1f014b4c0 2022-06-29T15:46:00.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.379+0000 7ff1f4c0b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1f00b58d0 0x7ff1f0063a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:00.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.379+0000 7ff1f4c0b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1f00b58d0 0x7ff1f0063a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48492/0 (socket says 172.21.15.47:48492) 2022-06-29T15:46:00.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.379+0000 7ff1f4c0b700 1 -- 172.21.15.47:0/3896899139 learned_addr learned my addr 172.21.15.47:0/3896899139 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:46:00.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.379+0000 7ff1f540c700 1 --2- 172.21.15.47:0/3896899139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1f00b1610 0x7ff1f00ac6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:00.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.379+0000 7ff1f5c0d700 1 --2- 172.21.15.47:0/3896899139 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff1f014b4c0 0x7ff1f00ae660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:00.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.379+0000 7ff1f4c0b700 1 -- 172.21.15.47:0/3896899139 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff1f014b4c0 msgr2=0x7ff1f00ae660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:00.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.379+0000 7ff1f4c0b700 1 --2- 172.21.15.47:0/3896899139 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff1f014b4c0 0x7ff1f00ae660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:00.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.379+0000 7ff1f4c0b700 1 -- 172.21.15.47:0/3896899139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1f00b1610 msgr2=0x7ff1f00ac6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:00.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.379+0000 7ff1f4c0b700 1 --2- 172.21.15.47:0/3896899139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1f00b1610 0x7ff1f00ac6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:00.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.379+0000 7ff1f4c0b700 1 -- 172.21.15.47:0/3896899139 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1ec00b1e0 con 0x7ff1f00b58d0 2022-06-29T15:46:00.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.379+0000 7ff1f540c700 1 --2- 172.21.15.47:0/3896899139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1f00b1610 0x7ff1f00ac6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:46:00.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.379+0000 7ff1f5c0d700 1 --2- 172.21.15.47:0/3896899139 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff1f014b4c0 0x7ff1f00ae660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:46:00.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.383+0000 7ff1f4c0b700 1 --2- 172.21.15.47:0/3896899139 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1f00b58d0 0x7ff1f0063a10 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7ff1f00b4fc0 tx=0x7ff1e4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:00.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.383+0000 7ff1e2ffd700 1 -- 172.21.15.47:0/3896899139 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff1e400c640 con 0x7ff1f00b58d0 2022-06-29T15:46:00.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.383+0000 7ff1e2ffd700 1 -- 172.21.15.47:0/3896899139 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1e400c7a0 con 0x7ff1f00b58d0 2022-06-29T15:46:00.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.383+0000 7ff1f640e700 1 -- 172.21.15.47:0/3896899139 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1f0144d50 con 0x7ff1f00b58d0 2022-06-29T15:46:00.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.383+0000 7ff1e2ffd700 1 -- 172.21.15.47:0/3896899139 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff1e400cb00 con 0x7ff1f00b58d0 2022-06-29T15:46:00.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.383+0000 7ff1f640e700 1 -- 172.21.15.47:0/3896899139 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1f00669f0 con 0x7ff1f00b58d0 2022-06-29T15:46:00.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.383+0000 7ff1f640e700 1 -- 172.21.15.47:0/3896899139 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1f00ad880 con 0x7ff1f00b58d0 2022-06-29T15:46:00.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.383+0000 7ff1e2ffd700 1 -- 172.21.15.47:0/3896899139 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff1e400cc60 con 0x7ff1f00b58d0 2022-06-29T15:46:00.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.387+0000 7ff1e2ffd700 1 --2- 172.21.15.47:0/3896899139 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff1cc053980 0x7ff1cc055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:00.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.387+0000 7ff1e2ffd700 1 -- 172.21.15.47:0/3896899139 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5954+0+0 (secure 0 0 0) 0x7ff1e406bfe0 con 0x7ff1f00b58d0 2022-06-29T15:46:00.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.387+0000 7ff1f540c700 1 --2- 172.21.15.47:0/3896899139 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff1cc053980 0x7ff1cc055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:00.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.387+0000 7ff1f540c700 1 --2- 172.21.15.47:0/3896899139 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff1cc053980 0x7ff1cc055e40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7ff1ec01a000 tx=0x7ff1ec015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:00.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.395+0000 7ff1e2ffd700 1 -- 172.21.15.47:0/3896899139 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff1f00ad880 con 0x7ff1f00b58d0 2022-06-29T15:46:00.526 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.523+0000 7ff1f640e700 1 -- 172.21.15.47:0/3896899139 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7ff1f00ad880 con 0x7ff1f00b58d0 2022-06-29T15:46:00.964 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.959+0000 7ff1e2ffd700 1 -- 172.21.15.47:0/3896899139 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v55) v1 ==== 125+0+0 (secure 0 0 0) 0x7ff1f00ad880 con 0x7ff1f00b58d0 2022-06-29T15:46:00.971 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.967+0000 7ff1f640e700 1 -- 172.21.15.47:0/3896899139 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff1cc053980 msgr2=0x7ff1cc055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:00.971 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.967+0000 7ff1f640e700 1 --2- 172.21.15.47:0/3896899139 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff1cc053980 0x7ff1cc055e40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7ff1ec01a000 tx=0x7ff1ec015000).stop 2022-06-29T15:46:00.972 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.967+0000 7ff1f640e700 1 -- 172.21.15.47:0/3896899139 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1f00b58d0 msgr2=0x7ff1f0063a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:00.972 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.967+0000 7ff1f640e700 1 --2- 172.21.15.47:0/3896899139 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1f00b58d0 0x7ff1f0063a10 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7ff1f00b4fc0 tx=0x7ff1e4006df0).stop 2022-06-29T15:46:00.974 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.971+0000 7ff1f640e700 1 -- 172.21.15.47:0/3896899139 shutdown_connections 2022-06-29T15:46:00.974 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.971+0000 7ff1f640e700 1 --2- 172.21.15.47:0/3896899139 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff1f014b4c0 0x7ff1f00ae660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:00.974 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.971+0000 7ff1f640e700 1 --2- 172.21.15.47:0/3896899139 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1f00b58d0 0x7ff1f0063a10 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:00.975 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.971+0000 7ff1f640e700 1 --2- 172.21.15.47:0/3896899139 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff1cc053980 0x7ff1cc055e40 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:00.975 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.971+0000 7ff1f640e700 1 --2- 172.21.15.47:0/3896899139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1f00b1610 0x7ff1f00ac6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:00.975 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.971+0000 7ff1f640e700 1 -- 172.21.15.47:0/3896899139 >> 172.21.15.47:0/3896899139 conn(0x7ff1f0062560 msgr2=0x7ff1f014a300 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:00.976 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.971+0000 7ff1f640e700 1 -- 172.21.15.47:0/3896899139 shutdown_connections 2022-06-29T15:46:00.976 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:00.971+0000 7ff1f640e700 1 -- 172.21.15.47:0/3896899139 wait complete. 2022-06-29T15:46:00.977 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' created 2022-06-29T15:46:00.987 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-06-29T15:46:01.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.099+0000 7f8c13fff700 1 Processor -- start 2022-06-29T15:46:01.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.099+0000 7f8c13fff700 1 -- start start 2022-06-29T15:46:01.105 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.103+0000 7f8c13fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8c140b2440 0x7f8c1405f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:01.106 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.103+0000 7f8c13fff700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c140b71d0 con 0x7f8c140b2820 2022-06-29T15:46:01.106 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.103+0000 7f8c13fff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c1405f6e0 con 0x7f8c14062f30 2022-06-29T15:46:01.106 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.103+0000 7f8c13fff700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c14141000 con 0x7f8c140b2440 2022-06-29T15:46:01.106 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.103+0000 7f8c12ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8c140b2440 0x7f8c1405f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:01.107 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.103+0000 7f8c12ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8c140b2440 0x7f8c1405f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48498/0 (socket says 172.21.15.47:48498) 2022-06-29T15:46:01.107 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.103+0000 7f8c137fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f8c140b2820 0x7f8c140b6ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34056/0 (socket says 172.21.15.47:34056) 2022-06-29T15:46:01.107 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.103+0000 7f8c12ffd700 1 -- 172.21.15.47:0/738616861 learned_addr learned my addr 172.21.15.47:0/738616861 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:46:01.107 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.103+0000 7f8c12ffd700 1 -- 172.21.15.47:0/738616861 >> v1:172.21.15.64:6789/0 conn(0x7f8c14062f30 legacy=0x7f8c14141930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:46:01.109 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.103+0000 7f8c12ffd700 1 -- 172.21.15.47:0/738616861 >> v1:172.21.15.47:6789/0 conn(0x7f8c140b2820 legacy=0x7f8c140b6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:01.109 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.103+0000 7f8c12ffd700 1 -- 172.21.15.47:0/738616861 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c141411f0 con 0x7f8c140b2440 2022-06-29T15:46:01.109 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.103+0000 7f8c12ffd700 1 --2- 172.21.15.47:0/738616861 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c140b2440 0x7f8c1405f0b0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7f8c04002f70 tx=0x7f8c04006eb0).ready entity=mon.0 client_cookie=4866f5cd1d8c451f server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:01.110 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.103+0000 7f8c127fc700 1 -- 172.21.15.47:0/738616861 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c0400d660 con 0x7f8c140b2440 2022-06-29T15:46:01.110 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.103+0000 7f8c127fc700 1 -- 172.21.15.47:0/738616861 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c04000e90 con 0x7f8c140b2440 2022-06-29T15:46:01.110 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.103+0000 7f8c127fc700 1 -- 172.21.15.47:0/738616861 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c0400d9f0 con 0x7f8c140b2440 2022-06-29T15:46:01.110 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.103+0000 7f8c13fff700 1 -- 172.21.15.47:0/738616861 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c140b2440 msgr2=0x7f8c1405f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:01.111 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.103+0000 7f8c13fff700 1 --2- 172.21.15.47:0/738616861 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c140b2440 0x7f8c1405f0b0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7f8c04002f70 tx=0x7f8c04006eb0).stop 2022-06-29T15:46:01.111 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.103+0000 7f8c13fff700 1 -- 172.21.15.47:0/738616861 shutdown_connections 2022-06-29T15:46:01.111 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.103+0000 7f8c13fff700 1 --2- 172.21.15.47:0/738616861 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c140b2440 0x7f8c1405f0b0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:01.112 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.103+0000 7f8c13fff700 1 -- 172.21.15.47:0/738616861 >> 172.21.15.47:0/738616861 conn(0x7f8c14063670 msgr2=0x7f8c14063a50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:01.112 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.103+0000 7f8c13fff700 1 -- 172.21.15.47:0/738616861 shutdown_connections 2022-06-29T15:46:01.112 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.103+0000 7f8c13fff700 1 -- 172.21.15.47:0/738616861 wait complete. 2022-06-29T15:46:01.115 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.107+0000 7f8c13fff700 1 Processor -- start 2022-06-29T15:46:01.115 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.107+0000 7f8c13fff700 1 -- start start 2022-06-29T15:46:01.115 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.107+0000 7f8c13fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8c14062f30 0x7f8c141462c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:01.115 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.107+0000 7f8c13fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c140b2440 0x7f8c14150600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:01.116 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.107+0000 7f8c13fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c140b2820 0x7f8c14150af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:01.116 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.107+0000 7f8c13fff700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c14150fe0 con 0x7f8c140b2820 2022-06-29T15:46:01.116 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.107+0000 7f8c13fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c141501d0 con 0x7f8c140b2440 2022-06-29T15:46:01.116 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.107+0000 7f8c13fff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c140adf00 con 0x7f8c14062f30 2022-06-29T15:46:01.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.107+0000 7f8c137fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c140b2820 0x7f8c14150af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:01.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.107+0000 7f8c137fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c140b2820 0x7f8c14150af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48508/0 (socket says 172.21.15.47:48508) 2022-06-29T15:46:01.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.107+0000 7f8c137fe700 1 -- 172.21.15.47:0/4031217664 learned_addr learned my addr 172.21.15.47:0/4031217664 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:46:01.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.107+0000 7f8c12ffd700 1 --2- 172.21.15.47:0/4031217664 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8c14062f30 0x7f8c141462c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:01.118 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.107+0000 7f8c0bfff700 1 --2- 172.21.15.47:0/4031217664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c140b2440 0x7f8c14150600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:01.118 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.111+0000 7f8c137fe700 1 -- 172.21.15.47:0/4031217664 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8c14062f30 msgr2=0x7f8c141462c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:01.118 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.111+0000 7f8c137fe700 1 --2- 172.21.15.47:0/4031217664 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8c14062f30 0x7f8c141462c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:01.118 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.111+0000 7f8c137fe700 1 -- 172.21.15.47:0/4031217664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c140b2440 msgr2=0x7f8c14150600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:01.119 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.111+0000 7f8c137fe700 1 --2- 172.21.15.47:0/4031217664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c140b2440 0x7f8c14150600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:01.119 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.111+0000 7f8c137fe700 1 -- 172.21.15.47:0/4031217664 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c0400b1e0 con 0x7f8c140b2820 2022-06-29T15:46:01.119 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.111+0000 7f8c137fe700 1 --2- 172.21.15.47:0/4031217664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c140b2820 0x7f8c14150af0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7f8c141500f0 tx=0x7f8c0c0013a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:01.119 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.111+0000 7f8c10ff9700 1 -- 172.21.15.47:0/4031217664 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c0c005ae0 con 0x7f8c140b2820 2022-06-29T15:46:01.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.111+0000 7f8c10ff9700 1 -- 172.21.15.47:0/4031217664 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c0c005c40 con 0x7f8c140b2820 2022-06-29T15:46:01.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.111+0000 7f8c13fff700 1 -- 172.21.15.47:0/4031217664 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c140ae150 con 0x7f8c140b2820 2022-06-29T15:46:01.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.111+0000 7f8c10ff9700 1 -- 172.21.15.47:0/4031217664 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c0c0070d0 con 0x7f8c140b2820 2022-06-29T15:46:01.121 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.111+0000 7f8c13fff700 1 -- 172.21.15.47:0/4031217664 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c140ae710 con 0x7f8c140b2820 2022-06-29T15:46:01.123 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.111+0000 7f8c13fff700 1 -- 172.21.15.47:0/4031217664 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c14150360 con 0x7f8c140b2820 2022-06-29T15:46:01.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.111+0000 7f8c10ff9700 1 -- 172.21.15.47:0/4031217664 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8c0c01a060 con 0x7f8c140b2820 2022-06-29T15:46:01.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.115+0000 7f8c10ff9700 1 --2- 172.21.15.47:0/4031217664 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8bf80539e0 0x7f8bf8055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:01.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.115+0000 7f8c10ff9700 1 -- 172.21.15.47:0/4031217664 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f8c0c06b920 con 0x7f8c140b2820 2022-06-29T15:46:01.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.115+0000 7f8c12ffd700 1 --2- 172.21.15.47:0/4031217664 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8bf80539e0 0x7f8bf8055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:01.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.115+0000 7f8c12ffd700 1 --2- 172.21.15.47:0/4031217664 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8bf80539e0 0x7f8bf8055ea0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7f8c0401c000 tx=0x7f8c04015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:01.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.119+0000 7f8c10ff9700 1 -- 172.21.15.47:0/4031217664 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8c14150360 con 0x7f8c140b2820 2022-06-29T15:46:01.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.291+0000 7f8c13fff700 1 -- 172.21.15.47:0/4031217664 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f8c14150360 con 0x7f8c140b2820 2022-06-29T15:46:01.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.299+0000 7f8c10ff9700 1 -- 172.21.15.47:0/4031217664 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 9 and data pool 10 v23) v1 ==== 188+0+0 (secure 0 0 0) 0x7f8c14150360 con 0x7f8c140b2820 2022-06-29T15:46:01.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.307+0000 7f8c13fff700 1 -- 172.21.15.47:0/4031217664 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8bf80539e0 msgr2=0x7f8bf8055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:01.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.307+0000 7f8c13fff700 1 --2- 172.21.15.47:0/4031217664 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8bf80539e0 0x7f8bf8055ea0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7f8c0401c000 tx=0x7f8c04015000).stop 2022-06-29T15:46:01.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.307+0000 7f8c13fff700 1 -- 172.21.15.47:0/4031217664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c140b2820 msgr2=0x7f8c14150af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:01.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.307+0000 7f8c13fff700 1 --2- 172.21.15.47:0/4031217664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c140b2820 0x7f8c14150af0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7f8c141500f0 tx=0x7f8c0c0013a0).stop 2022-06-29T15:46:01.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.307+0000 7f8c13fff700 1 -- 172.21.15.47:0/4031217664 shutdown_connections 2022-06-29T15:46:01.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.307+0000 7f8c13fff700 1 --2- 172.21.15.47:0/4031217664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c140b2820 0x7f8c14150af0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:01.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.307+0000 7f8c13fff700 1 --2- 172.21.15.47:0/4031217664 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8bf80539e0 0x7f8bf8055ea0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:01.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.307+0000 7f8c13fff700 1 --2- 172.21.15.47:0/4031217664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c140b2440 0x7f8c14150600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:01.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.307+0000 7f8c13fff700 1 --2- 172.21.15.47:0/4031217664 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8c14062f30 0x7f8c141462c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:01.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.307+0000 7f8c13fff700 1 -- 172.21.15.47:0/4031217664 >> 172.21.15.47:0/4031217664 conn(0x7f8c14063670 msgr2=0x7f8c14144ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:01.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.307+0000 7f8c13fff700 1 -- 172.21.15.47:0/4031217664 shutdown_connections 2022-06-29T15:46:01.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.311+0000 7f8c13fff700 1 -- 172.21.15.47:0/4031217664 wait complete. 2022-06-29T15:46:01.316 INFO:teuthology.orchestra.run.smithi047.stderr:new fs with metadata pool 9 and data pool 10 2022-06-29T15:46:01.327 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:46:01.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.419+0000 7fc3416ea700 1 Processor -- start 2022-06-29T15:46:01.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.419+0000 7fc3416ea700 1 -- start start 2022-06-29T15:46:01.425 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.419+0000 7fc3416ea700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc33c0606e0 0x7fc33c1408e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:01.425 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.419+0000 7fc3416ea700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc33c060b50 con 0x7fc33c0619b0 2022-06-29T15:46:01.426 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.419+0000 7fc3416ea700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc33c061d90 con 0x7fc33c0b0f30 2022-06-29T15:46:01.426 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.419+0000 7fc3416ea700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc33c0b2190 con 0x7fc33c0606e0 2022-06-29T15:46:01.426 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.419+0000 7fc33affd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc33c0619b0 0x7fc33c0b1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34078/0 (socket says 172.21.15.47:34078) 2022-06-29T15:46:01.426 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.419+0000 7fc33affd700 1 -- 172.21.15.47:0/1024052975 learned_addr learned my addr 172.21.15.47:0/1024052975 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:46:01.427 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.419+0000 7fc33b7fe700 1 --2- 172.21.15.47:0/1024052975 >> v2:172.21.15.47:3300/0 conn(0x7fc33c0606e0 0x7fc33c1408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:01.427 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.419+0000 7fc339ffb700 1 -- 172.21.15.47:0/1024052975 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 293436078 0 0) 0x7fc33c060b50 con 0x7fc33c0619b0 2022-06-29T15:46:01.427 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.419+0000 7fc339ffb700 1 -- 172.21.15.47:0/1024052975 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc320003260 con 0x7fc33c0619b0 2022-06-29T15:46:01.427 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.423+0000 7fc33b7fe700 1 -- 172.21.15.47:0/1024052975 >> v1:172.21.15.64:6789/0 conn(0x7fc33c0b0f30 legacy=0x7fc33c139a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:01.427 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.423+0000 7fc33b7fe700 1 -- 172.21.15.47:0/1024052975 >> v1:172.21.15.47:6789/0 conn(0x7fc33c0619b0 legacy=0x7fc33c0b1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:01.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.423+0000 7fc33b7fe700 1 -- 172.21.15.47:0/1024052975 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc33c060d80 con 0x7fc33c0606e0 2022-06-29T15:46:01.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.423+0000 7fc33b7fe700 1 --2- 172.21.15.47:0/1024052975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc33c0606e0 0x7fc33c1408e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7fc3340032f0 tx=0x7fc3340091d0).ready entity=mon.0 client_cookie=71bc9c7e649bb4fa server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:01.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.423+0000 7fc339ffb700 1 -- 172.21.15.47:0/1024052975 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc33400b8b0 con 0x7fc33c0606e0 2022-06-29T15:46:01.429 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.423+0000 7fc339ffb700 1 -- 172.21.15.47:0/1024052975 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc33400ba10 con 0x7fc33c0606e0 2022-06-29T15:46:01.429 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.423+0000 7fc339ffb700 1 -- 172.21.15.47:0/1024052975 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc33400bd70 con 0x7fc33c0606e0 2022-06-29T15:46:01.433 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.423+0000 7fc3416ea700 1 -- 172.21.15.47:0/1024052975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc33c0606e0 msgr2=0x7fc33c1408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:01.433 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.423+0000 7fc3416ea700 1 --2- 172.21.15.47:0/1024052975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc33c0606e0 0x7fc33c1408e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7fc3340032f0 tx=0x7fc3340091d0).stop 2022-06-29T15:46:01.433 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.423+0000 7fc3416ea700 1 -- 172.21.15.47:0/1024052975 shutdown_connections 2022-06-29T15:46:01.433 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.423+0000 7fc3416ea700 1 --2- 172.21.15.47:0/1024052975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc33c0606e0 0x7fc33c1408e0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:01.434 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.423+0000 7fc3416ea700 1 -- 172.21.15.47:0/1024052975 >> 172.21.15.47:0/1024052975 conn(0x7fc33c05c510 msgr2=0x7fc33c063800 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:01.434 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.423+0000 7fc3416ea700 1 -- 172.21.15.47:0/1024052975 shutdown_connections 2022-06-29T15:46:01.434 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.423+0000 7fc3416ea700 1 -- 172.21.15.47:0/1024052975 wait complete. 2022-06-29T15:46:01.434 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.423+0000 7fc3416ea700 1 Processor -- start 2022-06-29T15:46:01.435 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.423+0000 7fc3416ea700 1 -- start start 2022-06-29T15:46:01.435 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.427+0000 7fc3416ea700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc33c0606e0 0x7fc33c0b01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:01.435 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.427+0000 7fc3416ea700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc33c0619b0 0x7fc33c135060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:01.435 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.427+0000 7fc3416ea700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc33c0b0f30 0x7fc33c135550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:01.436 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.427+0000 7fc3416ea700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc33c13bdc0 con 0x7fc33c0606e0 2022-06-29T15:46:01.436 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.427+0000 7fc3416ea700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc33c142320 con 0x7fc33c0b0f30 2022-06-29T15:46:01.436 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.427+0000 7fc3416ea700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc33c13cc80 con 0x7fc33c0619b0 2022-06-29T15:46:01.436 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.427+0000 7fc33affd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc33c0606e0 0x7fc33c0b01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:01.436 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.427+0000 7fc33affd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc33c0606e0 0x7fc33c0b01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48530/0 (socket says 172.21.15.47:48530) 2022-06-29T15:46:01.437 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.427+0000 7fc33affd700 1 -- 172.21.15.47:0/3930797135 learned_addr learned my addr 172.21.15.47:0/3930797135 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:46:01.437 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.427+0000 7fc33a7fc700 1 --2- 172.21.15.47:0/3930797135 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc33c0619b0 0x7fc33c135060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:01.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.427+0000 7fc33b7fe700 1 --2- 172.21.15.47:0/3930797135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc33c0b0f30 0x7fc33c135550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:01.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.427+0000 7fc33affd700 1 -- 172.21.15.47:0/3930797135 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc33c0619b0 msgr2=0x7fc33c135060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:01.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.427+0000 7fc33affd700 1 --2- 172.21.15.47:0/3930797135 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc33c0619b0 0x7fc33c135060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:01.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.427+0000 7fc33affd700 1 -- 172.21.15.47:0/3930797135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc33c0b0f30 msgr2=0x7fc33c135550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:01.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.427+0000 7fc33affd700 1 --2- 172.21.15.47:0/3930797135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc33c0b0f30 0x7fc33c135550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:01.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.427+0000 7fc33affd700 1 -- 172.21.15.47:0/3930797135 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc334009470 con 0x7fc33c0606e0 2022-06-29T15:46:01.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.427+0000 7fc33affd700 1 --2- 172.21.15.47:0/3930797135 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc33c0606e0 0x7fc33c0b01c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7fc32c001110 tx=0x7fc32c007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:01.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.427+0000 7fc32bfff700 1 -- 172.21.15.47:0/3930797135 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc32c0072e0 con 0x7fc33c0606e0 2022-06-29T15:46:01.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.427+0000 7fc32bfff700 1 -- 172.21.15.47:0/3930797135 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc32c007440 con 0x7fc33c0606e0 2022-06-29T15:46:01.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.427+0000 7fc3416ea700 1 -- 172.21.15.47:0/3930797135 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc33c13cf20 con 0x7fc33c0606e0 2022-06-29T15:46:01.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.427+0000 7fc32bfff700 1 -- 172.21.15.47:0/3930797135 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc32c005800 con 0x7fc33c0606e0 2022-06-29T15:46:01.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.427+0000 7fc3416ea700 1 -- 172.21.15.47:0/3930797135 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc33c135a40 con 0x7fc33c0606e0 2022-06-29T15:46:01.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.427+0000 7fc32bfff700 1 -- 172.21.15.47:0/3930797135 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc32c010070 con 0x7fc33c0606e0 2022-06-29T15:46:01.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.427+0000 7fc3416ea700 1 -- 172.21.15.47:0/3930797135 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc33c13cac0 con 0x7fc33c0606e0 2022-06-29T15:46:01.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.431+0000 7fc32bfff700 1 --2- 172.21.15.47:0/3930797135 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc3180539e0 0x7fc318055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:01.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.431+0000 7fc32bfff700 1 -- 172.21.15.47:0/3930797135 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc32c06b7f0 con 0x7fc33c0606e0 2022-06-29T15:46:01.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.431+0000 7fc33a7fc700 1 --2- 172.21.15.47:0/3930797135 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc3180539e0 0x7fc318055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:01.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.431+0000 7fc33a7fc700 1 --2- 172.21.15.47:0/3930797135 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc3180539e0 0x7fc318055ea0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7fc33000b000 tx=0x7fc330009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:01.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.439+0000 7fc32bfff700 1 -- 172.21.15.47:0/3930797135 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc32c0425b0 con 0x7fc33c0606e0 2022-06-29T15:46:01.569 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.563+0000 7fc3416ea700 1 -- 172.21.15.47:0/3930797135 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc33c1424f0 con 0x7fc33c0606e0 2022-06-29T15:46:01.571 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.567+0000 7fc32bfff700 1 -- 172.21.15.47:0/3930797135 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+16299 (secure 0 0 0) 0x7fc32c045bd0 con 0x7fc33c0606e0 2022-06-29T15:46:01.571 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:46:01.572 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":56,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:46:01.293904+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:45:59.710738+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2022-06-29T15:46:00.582445+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:46:01.574 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.567+0000 7fc3416ea700 1 -- 172.21.15.47:0/3930797135 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc3180539e0 msgr2=0x7fc318055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:01.574 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.567+0000 7fc3416ea700 1 --2- 172.21.15.47:0/3930797135 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc3180539e0 0x7fc318055ea0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7fc33000b000 tx=0x7fc330009040).stop 2022-06-29T15:46:01.575 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.567+0000 7fc3416ea700 1 -- 172.21.15.47:0/3930797135 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc33c0606e0 msgr2=0x7fc33c0b01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:01.575 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.567+0000 7fc3416ea700 1 --2- 172.21.15.47:0/3930797135 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc33c0606e0 0x7fc33c0b01c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7fc32c001110 tx=0x7fc32c007ad0).stop 2022-06-29T15:46:01.575 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.567+0000 7fc3416ea700 1 -- 172.21.15.47:0/3930797135 shutdown_connections 2022-06-29T15:46:01.575 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.567+0000 7fc3416ea700 1 --2- 172.21.15.47:0/3930797135 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc3180539e0 0x7fc318055ea0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:01.576 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.567+0000 7fc3416ea700 1 --2- 172.21.15.47:0/3930797135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc33c0b0f30 0x7fc33c135550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:01.576 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.567+0000 7fc3416ea700 1 --2- 172.21.15.47:0/3930797135 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc33c0619b0 0x7fc33c135060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:01.576 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.567+0000 7fc3416ea700 1 --2- 172.21.15.47:0/3930797135 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc33c0606e0 0x7fc33c0b01c0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:01.576 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.567+0000 7fc3416ea700 1 -- 172.21.15.47:0/3930797135 >> 172.21.15.47:0/3930797135 conn(0x7fc33c05c510 msgr2=0x7fc33c139980 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:01.577 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.567+0000 7fc3416ea700 1 -- 172.21.15.47:0/3930797135 shutdown_connections 2022-06-29T15:46:01.577 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.567+0000 7fc3416ea700 1 -- 172.21.15.47:0/3930797135 wait complete. 2022-06-29T15:46:01.584 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:46:01.691 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.683+0000 7f735a87c700 1 Processor -- start 2022-06-29T15:46:01.691 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.683+0000 7f735a87c700 1 -- start start 2022-06-29T15:46:01.691 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.683+0000 7f735a87c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f73540af590 0x7f73540ad0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:01.691 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.687+0000 7f735a87c700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73540b1850 con 0x7f73540b04e0 2022-06-29T15:46:01.692 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.687+0000 7f735a87c700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7354142d50 con 0x7f73540b1470 2022-06-29T15:46:01.692 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.687+0000 7f735a87c700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73540b27c0 con 0x7f73540af590 2022-06-29T15:46:01.693 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.687+0000 7f7359079700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f73540b04e0 0x7f7354143090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34092/0 (socket says 172.21.15.47:34092) 2022-06-29T15:46:01.693 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.687+0000 7f7359079700 1 -- 172.21.15.47:0/335533382 learned_addr learned my addr 172.21.15.47:0/335533382 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:46:01.694 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.687+0000 7f735a07b700 1 --2- 172.21.15.47:0/335533382 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f73540af590 0x7f73540ad0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:01.694 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.687+0000 7f7358878700 1 -- 172.21.15.47:0/335533382 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 429989838 0 0) 0x7f73540b1850 con 0x7f73540b04e0 2022-06-29T15:46:01.694 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.687+0000 7f7358878700 1 -- 172.21.15.47:0/335533382 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f733c003260 con 0x7f73540b04e0 2022-06-29T15:46:01.695 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.687+0000 7f735a07b700 1 -- 172.21.15.47:0/335533382 >> v1:172.21.15.64:6789/0 conn(0x7f73540b1470 legacy=0x7f73540ac9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:01.696 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.687+0000 7f7358878700 1 -- 172.21.15.47:0/335533382 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3351054779 0 0) 0x7f733c003260 con 0x7f73540b04e0 2022-06-29T15:46:01.696 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.687+0000 7f735a07b700 1 -- 172.21.15.47:0/335533382 >> v1:172.21.15.47:6789/0 conn(0x7f73540b04e0 legacy=0x7f7354143090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:01.697 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.687+0000 7f735a07b700 1 -- 172.21.15.47:0/335533382 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73540ae9a0 con 0x7f73540af590 2022-06-29T15:46:01.697 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.687+0000 7f735a07b700 1 --2- 172.21.15.47:0/335533382 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f73540af590 0x7f73540ad0d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f7344001820 tx=0x7f734c0091d0).ready entity=mon.2 client_cookie=ad79786ccdbf55a4 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:01.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.687+0000 7f7358878700 1 -- 172.21.15.47:0/335533382 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7344001680 con 0x7f73540af590 2022-06-29T15:46:01.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.687+0000 7f7358878700 1 -- 172.21.15.47:0/335533382 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f734c010620 con 0x7f73540af590 2022-06-29T15:46:01.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.687+0000 7f7358878700 1 -- 172.21.15.47:0/335533382 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f734c010980 con 0x7f73540af590 2022-06-29T15:46:01.704 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.687+0000 7f735a87c700 1 -- 172.21.15.47:0/335533382 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f73540af590 msgr2=0x7f73540ad0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:01.704 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.687+0000 7f735a87c700 1 --2- 172.21.15.47:0/335533382 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f73540af590 0x7f73540ad0d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f7344001820 tx=0x7f734c0091d0).stop 2022-06-29T15:46:01.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.687+0000 7f735a87c700 1 -- 172.21.15.47:0/335533382 shutdown_connections 2022-06-29T15:46:01.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.687+0000 7f735a87c700 1 --2- 172.21.15.47:0/335533382 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f73540af590 0x7f73540ad0d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:01.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.687+0000 7f735a87c700 1 -- 172.21.15.47:0/335533382 >> 172.21.15.47:0/335533382 conn(0x7f7354062560 msgr2=0x7f7354062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:01.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.687+0000 7f735a87c700 1 -- 172.21.15.47:0/335533382 shutdown_connections 2022-06-29T15:46:01.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.687+0000 7f735a87c700 1 -- 172.21.15.47:0/335533382 wait complete. 2022-06-29T15:46:01.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.691+0000 7f735a87c700 1 Processor -- start 2022-06-29T15:46:01.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.691+0000 7f735a87c700 1 -- start start 2022-06-29T15:46:01.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.691+0000 7f735a87c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73540af590 0x7f7354155b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:01.707 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.691+0000 7f735a87c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73540b04e0 0x7f7354141d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:01.707 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.691+0000 7f735a87c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f73540b1470 0x7f7354142280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:01.707 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.691+0000 7f735a87c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7354064a80 con 0x7f73540af590 2022-06-29T15:46:01.707 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.691+0000 7f735a87c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73541566f0 con 0x7f73540b04e0 2022-06-29T15:46:01.707 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.691+0000 7f735a87c700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7354142770 con 0x7f73540b1470 2022-06-29T15:46:01.708 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.691+0000 7f735987a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73540af590 0x7f7354155b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:01.708 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.691+0000 7f735987a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73540af590 0x7f7354155b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48544/0 (socket says 172.21.15.47:48544) 2022-06-29T15:46:01.708 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.691+0000 7f735987a700 1 -- 172.21.15.47:0/976965116 learned_addr learned my addr 172.21.15.47:0/976965116 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:46:01.708 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.691+0000 7f7359079700 1 --2- 172.21.15.47:0/976965116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73540b04e0 0x7f7354141d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:01.709 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.691+0000 7f735a07b700 1 --2- 172.21.15.47:0/976965116 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f73540b1470 0x7f7354142280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:01.709 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.691+0000 7f735987a700 1 -- 172.21.15.47:0/976965116 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f73540b1470 msgr2=0x7f7354142280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:01.709 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.691+0000 7f735987a700 1 --2- 172.21.15.47:0/976965116 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f73540b1470 0x7f7354142280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:01.709 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.691+0000 7f735987a700 1 -- 172.21.15.47:0/976965116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73540b04e0 msgr2=0x7f7354141d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:01.709 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.691+0000 7f735987a700 1 --2- 172.21.15.47:0/976965116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73540b04e0 0x7f7354141d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:01.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.691+0000 7f735987a700 1 -- 172.21.15.47:0/976965116 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f734c009470 con 0x7f73540af590 2022-06-29T15:46:01.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.691+0000 7f735987a700 1 --2- 172.21.15.47:0/976965116 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73540af590 0x7f7354155b70 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7f73540b6280 tx=0x7f7350007ca0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:01.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.691+0000 7f7342ffd700 1 -- 172.21.15.47:0/976965116 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7350012400 con 0x7f73540af590 2022-06-29T15:46:01.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.691+0000 7f7342ffd700 1 -- 172.21.15.47:0/976965116 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7350012560 con 0x7f73540af590 2022-06-29T15:46:01.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.691+0000 7f7342ffd700 1 -- 172.21.15.47:0/976965116 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f73500128c0 con 0x7f73540af590 2022-06-29T15:46:01.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.691+0000 7f735a87c700 1 -- 172.21.15.47:0/976965116 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7354140b80 con 0x7f73540af590 2022-06-29T15:46:01.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.691+0000 7f735a87c700 1 -- 172.21.15.47:0/976965116 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73541561b0 con 0x7f73540af590 2022-06-29T15:46:01.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.695+0000 7f735a87c700 1 -- 172.21.15.47:0/976965116 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73540573d0 con 0x7f73540af590 2022-06-29T15:46:01.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.695+0000 7f7342ffd700 1 -- 172.21.15.47:0/976965116 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7350019030 con 0x7f73540af590 2022-06-29T15:46:01.712 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.695+0000 7f7342ffd700 1 --2- 172.21.15.47:0/976965116 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7334053a10 0x7f7334055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:01.712 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.695+0000 7f7342ffd700 1 -- 172.21.15.47:0/976965116 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f735006b260 con 0x7f73540af590 2022-06-29T15:46:01.712 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.695+0000 7f7359079700 1 --2- 172.21.15.47:0/976965116 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7334053a10 0x7f7334055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:01.712 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.695+0000 7f7359079700 1 --2- 172.21.15.47:0/976965116 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7334053a10 0x7f7334055ed0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7f734400d000 tx=0x7f7344000bc0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:01.713 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.703+0000 7f7342ffd700 1 -- 172.21.15.47:0/976965116 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7350042020 con 0x7f73540af590 2022-06-29T15:46:01.834 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.831+0000 7f735a87c700 1 -- 172.21.15.47:0/976965116 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7354069e20 con 0x7f73540af590 2022-06-29T15:46:01.836 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.831+0000 7f7342ffd700 1 -- 172.21.15.47:0/976965116 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f7350045640 con 0x7f73540af590 2022-06-29T15:46:01.837 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:46:01.838 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":56,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:46:01.293904+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:45:59.710738+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2022-06-29T15:46:00.582445+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:46:01.839 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.835+0000 7f735a87c700 1 -- 172.21.15.47:0/976965116 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7334053a10 msgr2=0x7f7334055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:01.839 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.835+0000 7f735a87c700 1 --2- 172.21.15.47:0/976965116 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7334053a10 0x7f7334055ed0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7f734400d000 tx=0x7f7344000bc0).stop 2022-06-29T15:46:01.840 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.835+0000 7f735a87c700 1 -- 172.21.15.47:0/976965116 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73540af590 msgr2=0x7f7354155b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:01.840 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.835+0000 7f735a87c700 1 --2- 172.21.15.47:0/976965116 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73540af590 0x7f7354155b70 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7f73540b6280 tx=0x7f7350007ca0).stop 2022-06-29T15:46:01.840 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.835+0000 7f735a87c700 1 -- 172.21.15.47:0/976965116 shutdown_connections 2022-06-29T15:46:01.841 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.835+0000 7f735a87c700 1 --2- 172.21.15.47:0/976965116 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f73540b1470 0x7f7354142280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:01.841 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.835+0000 7f735a87c700 1 --2- 172.21.15.47:0/976965116 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7334053a10 0x7f7334055ed0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:01.841 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.835+0000 7f735a87c700 1 --2- 172.21.15.47:0/976965116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73540b04e0 0x7f7354141d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:01.842 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.835+0000 7f735a87c700 1 --2- 172.21.15.47:0/976965116 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73540af590 0x7f7354155b70 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:01.842 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.835+0000 7f735a87c700 1 -- 172.21.15.47:0/976965116 >> 172.21.15.47:0/976965116 conn(0x7f7354062560 msgr2=0x7f7354062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:01.842 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.835+0000 7f735a87c700 1 -- 172.21.15.47:0/976965116 shutdown_connections 2022-06-29T15:46:01.842 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.835+0000 7f735a87c700 1 -- 172.21.15.47:0/976965116 wait complete. 2022-06-29T15:46:01.855 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-06-29T15:46:01.963 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.959+0000 7f2d9eed7700 1 Processor -- start 2022-06-29T15:46:01.963 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.959+0000 7f2d9eed7700 1 -- start start 2022-06-29T15:46:01.965 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.959+0000 7f2d9eed7700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2d9814acc0 0x7f2d9814b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:01.966 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.959+0000 7f2d9eed7700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2d980b0dd0 0x7f2d980b1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:01.966 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.959+0000 7f2d9eed7700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d98061bf0 con 0x7f2d9814a8e0 2022-06-29T15:46:01.966 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.959+0000 7f2d9eed7700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d9805fca0 con 0x7f2d9814acc0 2022-06-29T15:46:01.966 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.959+0000 7f2d9eed7700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d98061e20 con 0x7f2d980b0dd0 2022-06-29T15:46:01.967 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.959+0000 7f2d9ded5700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f2d9814a8e0 0x7f2d980b1d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34108/0 (socket says 172.21.15.47:34108) 2022-06-29T15:46:01.967 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.959+0000 7f2d9ded5700 1 -- 172.21.15.47:0/610460306 learned_addr learned my addr 172.21.15.47:0/610460306 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:46:01.967 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.959+0000 7f2d9e6d6700 1 --2- 172.21.15.47:0/610460306 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2d980b0dd0 0x7f2d980b1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:01.967 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.959+0000 7f2d9d6d4700 1 --2- 172.21.15.47:0/610460306 >> v2:172.21.15.64:3300/0 conn(0x7f2d9814acc0 0x7f2d9814b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:01.968 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.959+0000 7f2d9ced3700 1 -- 172.21.15.47:0/610460306 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 386535133 0 0) 0x7f2d98061bf0 con 0x7f2d9814a8e0 2022-06-29T15:46:01.968 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.959+0000 7f2d9ced3700 1 -- 172.21.15.47:0/610460306 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d80003260 con 0x7f2d9814a8e0 2022-06-29T15:46:01.968 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.959+0000 7f2d9e6d6700 1 -- 172.21.15.47:0/610460306 >> v2:172.21.15.64:3300/0 conn(0x7f2d9814acc0 msgr2=0x7f2d9814b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:46:01.968 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.959+0000 7f2d9e6d6700 1 --2- 172.21.15.47:0/610460306 >> v2:172.21.15.64:3300/0 conn(0x7f2d9814acc0 0x7f2d9814b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:01.969 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.963+0000 7f2d9e6d6700 1 -- 172.21.15.47:0/610460306 >> v1:172.21.15.47:6789/0 conn(0x7f2d9814a8e0 legacy=0x7f2d980b1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:01.969 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.963+0000 7f2d9e6d6700 1 -- 172.21.15.47:0/610460306 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d9805ff40 con 0x7f2d980b0dd0 2022-06-29T15:46:01.969 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.963+0000 7f2d9e6d6700 1 --2- 172.21.15.47:0/610460306 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2d980b0dd0 0x7f2d980b1220 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f2d900032f0 tx=0x7f2d900091d0).ready entity=mon.2 client_cookie=8cf3abeb6f6b90b3 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:01.969 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.963+0000 7f2d9ced3700 1 -- 172.21.15.47:0/610460306 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d9000b600 con 0x7f2d980b0dd0 2022-06-29T15:46:01.970 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.963+0000 7f2d9ced3700 1 -- 172.21.15.47:0/610460306 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d9000b760 con 0x7f2d980b0dd0 2022-06-29T15:46:01.970 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.963+0000 7f2d9ced3700 1 -- 172.21.15.47:0/610460306 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d9000b8d0 con 0x7f2d980b0dd0 2022-06-29T15:46:01.973 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.963+0000 7f2d9eed7700 1 -- 172.21.15.47:0/610460306 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2d980b0dd0 msgr2=0x7f2d980b1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:01.973 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.963+0000 7f2d9eed7700 1 --2- 172.21.15.47:0/610460306 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2d980b0dd0 0x7f2d980b1220 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f2d900032f0 tx=0x7f2d900091d0).stop 2022-06-29T15:46:01.974 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.963+0000 7f2d9eed7700 1 -- 172.21.15.47:0/610460306 shutdown_connections 2022-06-29T15:46:01.974 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.963+0000 7f2d9eed7700 1 --2- 172.21.15.47:0/610460306 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2d980b0dd0 0x7f2d980b1220 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:01.974 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.963+0000 7f2d9eed7700 1 --2- 172.21.15.47:0/610460306 >> v2:172.21.15.64:3300/0 conn(0x7f2d9814acc0 0x7f2d9814b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:01.974 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.963+0000 7f2d9eed7700 1 -- 172.21.15.47:0/610460306 >> 172.21.15.47:0/610460306 conn(0x7f2d980b6430 msgr2=0x7f2d980b6810 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:01.975 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.963+0000 7f2d9eed7700 1 -- 172.21.15.47:0/610460306 shutdown_connections 2022-06-29T15:46:01.975 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.963+0000 7f2d9eed7700 1 -- 172.21.15.47:0/610460306 wait complete. 2022-06-29T15:46:01.975 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.963+0000 7f2d9eed7700 1 Processor -- start 2022-06-29T15:46:01.975 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.963+0000 7f2d9eed7700 1 -- start start 2022-06-29T15:46:01.976 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.967+0000 7f2d9eed7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d980b0dd0 0x7f2d980b4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:01.976 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.967+0000 7f2d9eed7700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2d9814a8e0 0x7f2d98141e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:01.977 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.967+0000 7f2d9eed7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d9814acc0 0x7f2d98142370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:01.977 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.967+0000 7f2d9eed7700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d980aa2e0 con 0x7f2d980b0dd0 2022-06-29T15:46:01.977 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.967+0000 7f2d9eed7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d980aa580 con 0x7f2d9814acc0 2022-06-29T15:46:01.977 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.967+0000 7f2d9eed7700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d980aa820 con 0x7f2d9814a8e0 2022-06-29T15:46:01.978 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.967+0000 7f2d9ded5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d980b0dd0 0x7f2d980b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:01.978 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.967+0000 7f2d9ded5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d980b0dd0 0x7f2d980b4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48558/0 (socket says 172.21.15.47:48558) 2022-06-29T15:46:01.978 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.967+0000 7f2d9ded5700 1 -- 172.21.15.47:0/1864721778 learned_addr learned my addr 172.21.15.47:0/1864721778 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:46:01.978 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.967+0000 7f2d9d6d4700 1 --2- 172.21.15.47:0/1864721778 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2d9814a8e0 0x7f2d98141e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:01.979 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.967+0000 7f2d9e6d6700 1 --2- 172.21.15.47:0/1864721778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d9814acc0 0x7f2d98142370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:01.979 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.967+0000 7f2d9ded5700 1 -- 172.21.15.47:0/1864721778 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2d9814a8e0 msgr2=0x7f2d98141e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:01.979 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.967+0000 7f2d9ded5700 1 --2- 172.21.15.47:0/1864721778 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2d9814a8e0 0x7f2d98141e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:01.979 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.967+0000 7f2d9ded5700 1 -- 172.21.15.47:0/1864721778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d9814acc0 msgr2=0x7f2d98142370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:01.980 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.967+0000 7f2d9ded5700 1 --2- 172.21.15.47:0/1864721778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d9814acc0 0x7f2d98142370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:01.980 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.967+0000 7f2d9ded5700 1 -- 172.21.15.47:0/1864721778 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d90009470 con 0x7f2d980b0dd0 2022-06-29T15:46:01.980 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.967+0000 7f2d9ded5700 1 --2- 172.21.15.47:0/1864721778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d980b0dd0 0x7f2d980b4370 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7f2d88001110 tx=0x7f2d88007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:01.982 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.971+0000 7f2d86ffd700 1 -- 172.21.15.47:0/1864721778 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d8800ce20 con 0x7f2d980b0dd0 2022-06-29T15:46:01.983 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.971+0000 7f2d86ffd700 1 -- 172.21.15.47:0/1864721778 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d880072e0 con 0x7f2d980b0dd0 2022-06-29T15:46:01.983 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.971+0000 7f2d9eed7700 1 -- 172.21.15.47:0/1864721778 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d980aaac0 con 0x7f2d980b0dd0 2022-06-29T15:46:01.983 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.971+0000 7f2d86ffd700 1 -- 172.21.15.47:0/1864721778 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d88005800 con 0x7f2d980b0dd0 2022-06-29T15:46:01.983 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.971+0000 7f2d9eed7700 1 -- 172.21.15.47:0/1864721778 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d98146750 con 0x7f2d980b0dd0 2022-06-29T15:46:01.983 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.971+0000 7f2d9eed7700 1 -- 172.21.15.47:0/1864721778 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d980573d0 con 0x7f2d980b0dd0 2022-06-29T15:46:01.984 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.971+0000 7f2d86ffd700 1 -- 172.21.15.47:0/1864721778 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2d88010070 con 0x7f2d980b0dd0 2022-06-29T15:46:01.984 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.971+0000 7f2d86ffd700 1 --2- 172.21.15.47:0/1864721778 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2d78053a70 0x7f2d78055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:01.984 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.971+0000 7f2d86ffd700 1 -- 172.21.15.47:0/1864721778 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f2d8806b820 con 0x7f2d980b0dd0 2022-06-29T15:46:01.984 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.971+0000 7f2d9d6d4700 1 --2- 172.21.15.47:0/1864721778 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2d78053a70 0x7f2d78055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:01.985 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.975+0000 7f2d9d6d4700 1 --2- 172.21.15.47:0/1864721778 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2d78053a70 0x7f2d78055f30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f2d9400b000 tx=0x7f2d94006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:01.985 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:01.983+0000 7f2d86ffd700 1 -- 172.21.15.47:0/1864721778 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2d88042540 con 0x7f2d980b0dd0 2022-06-29T15:46:02.150 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.143+0000 7f2d9eed7700 1 -- 172.21.15.47:0/1864721778 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f2d980aac50 con 0x7f2d980b0dd0 2022-06-29T15:46:02.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.299+0000 7f2d86ffd700 1 -- 172.21.15.47:0/1864721778 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v24) v1 ==== 118+0+0 (secure 0 0 0) 0x7f2d88045b60 con 0x7f2d980b0dd0 2022-06-29T15:46:02.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.303+0000 7f2d9eed7700 1 -- 172.21.15.47:0/1864721778 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2d78053a70 msgr2=0x7f2d78055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:02.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.303+0000 7f2d9eed7700 1 --2- 172.21.15.47:0/1864721778 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2d78053a70 0x7f2d78055f30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f2d9400b000 tx=0x7f2d94006df0).stop 2022-06-29T15:46:02.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.303+0000 7f2d9eed7700 1 -- 172.21.15.47:0/1864721778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d980b0dd0 msgr2=0x7f2d980b4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:02.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.303+0000 7f2d9eed7700 1 --2- 172.21.15.47:0/1864721778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d980b0dd0 0x7f2d980b4370 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7f2d88001110 tx=0x7f2d88007ad0).stop 2022-06-29T15:46:02.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.303+0000 7f2d9eed7700 1 -- 172.21.15.47:0/1864721778 shutdown_connections 2022-06-29T15:46:02.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.303+0000 7f2d9eed7700 1 --2- 172.21.15.47:0/1864721778 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2d78053a70 0x7f2d78055f30 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:02.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.303+0000 7f2d9eed7700 1 --2- 172.21.15.47:0/1864721778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d9814acc0 0x7f2d98142370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:02.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.303+0000 7f2d9eed7700 1 --2- 172.21.15.47:0/1864721778 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2d9814a8e0 0x7f2d98141e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:02.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.303+0000 7f2d9eed7700 1 --2- 172.21.15.47:0/1864721778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d980b0dd0 0x7f2d980b4370 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:02.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.303+0000 7f2d9eed7700 1 -- 172.21.15.47:0/1864721778 >> 172.21.15.47:0/1864721778 conn(0x7f2d980b6430 msgr2=0x7f2d980b5cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:02.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.303+0000 7f2d9eed7700 1 -- 172.21.15.47:0/1864721778 shutdown_connections 2022-06-29T15:46:02.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.307+0000 7f2d9eed7700 1 -- 172.21.15.47:0/1864721778 wait complete. 2022-06-29T15:46:02.326 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:46:02.433 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.431+0000 7f23d2c3b700 1 Processor -- start 2022-06-29T15:46:02.434 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.431+0000 7f23d2c3b700 1 -- start start 2022-06-29T15:46:02.436 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.431+0000 7f23d2c3b700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f23cc0b2120 0x7f23cc0b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:02.436 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.431+0000 7f23d2c3b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f23cc0b1260 0x7f23cc14b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:02.436 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.431+0000 7f23d2c3b700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23cc0b0740 con 0x7f23cc05f0b0 2022-06-29T15:46:02.437 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.431+0000 7f23d2c3b700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23cc147460 con 0x7f23cc0b1260 2022-06-29T15:46:02.437 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.431+0000 7f23d2c3b700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc0b0970 con 0x7f23cc0b2120 2022-06-29T15:46:02.437 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.431+0000 7f23d11d8700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f23cc0b1260 0x7f23cc14b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:02.437 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.431+0000 7f23d11d8700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f23cc0b1260 0x7f23cc14b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48568/0 (socket says 172.21.15.47:48568) 2022-06-29T15:46:02.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.431+0000 7f23d09d7700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f23cc0b2120 0x7f23cc0b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:02.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.431+0000 7f23d11d8700 1 -- 172.21.15.47:0/504600532 learned_addr learned my addr 172.21.15.47:0/504600532 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:46:02.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.431+0000 7f23d11d8700 1 -- 172.21.15.47:0/504600532 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f23cc0b2120 msgr2=0x7f23cc0b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:46:02.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.431+0000 7f23d11d8700 1 --2- 172.21.15.47:0/504600532 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f23cc0b2120 0x7f23cc0b2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:02.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.431+0000 7f23d11d8700 1 -- 172.21.15.47:0/504600532 >> v1:172.21.15.64:6789/0 conn(0x7f23cc05f0b0 legacy=0x7f23cc147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:02.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.431+0000 7f23cb7fe700 1 -- 172.21.15.47:0/504600532 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3573311062 0 0) 0x7f23cc0b0740 con 0x7f23cc05f0b0 2022-06-29T15:46:02.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.431+0000 7f23d11d8700 1 -- 172.21.15.47:0/504600532 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23cc0b0c10 con 0x7f23cc0b1260 2022-06-29T15:46:02.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.431+0000 7f23d09d7700 1 --2- 172.21.15.47:0/504600532 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f23cc0b2120 0x7f23cc0b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:46:02.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.431+0000 7f23d11d8700 1 --2- 172.21.15.47:0/504600532 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23cc0b1260 0x7f23cc14b520 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7f23c40032f0 tx=0x7f23c4009290).ready entity=mon.0 client_cookie=aea9fedbfa90fff2 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:02.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.431+0000 7f23cb7fe700 1 -- 172.21.15.47:0/504600532 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23c400b8a0 con 0x7f23cc0b1260 2022-06-29T15:46:02.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.431+0000 7f23cb7fe700 1 -- 172.21.15.47:0/504600532 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c400ba00 con 0x7f23cc0b1260 2022-06-29T15:46:02.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.431+0000 7f23cb7fe700 1 -- 172.21.15.47:0/504600532 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23c400bb70 con 0x7f23cc0b1260 2022-06-29T15:46:02.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.435+0000 7f23d2c3b700 1 -- 172.21.15.47:0/504600532 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23cc0b1260 msgr2=0x7f23cc14b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:02.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.435+0000 7f23d2c3b700 1 --2- 172.21.15.47:0/504600532 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23cc0b1260 0x7f23cc14b520 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7f23c40032f0 tx=0x7f23c4009290).stop 2022-06-29T15:46:02.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.435+0000 7f23d2c3b700 1 -- 172.21.15.47:0/504600532 shutdown_connections 2022-06-29T15:46:02.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.435+0000 7f23d2c3b700 1 --2- 172.21.15.47:0/504600532 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23cc0b1260 0x7f23cc14b520 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:02.446 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.435+0000 7f23d2c3b700 1 --2- 172.21.15.47:0/504600532 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f23cc0b2120 0x7f23cc0b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:02.446 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.435+0000 7f23d2c3b700 1 -- 172.21.15.47:0/504600532 >> 172.21.15.47:0/504600532 conn(0x7f23cc0626f0 msgr2=0x7f23cc062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:02.446 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.435+0000 7f23d2c3b700 1 -- 172.21.15.47:0/504600532 shutdown_connections 2022-06-29T15:46:02.446 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.435+0000 7f23d2c3b700 1 -- 172.21.15.47:0/504600532 wait complete. 2022-06-29T15:46:02.447 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.435+0000 7f23d2c3b700 1 Processor -- start 2022-06-29T15:46:02.447 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.435+0000 7f23d2c3b700 1 -- start start 2022-06-29T15:46:02.447 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.435+0000 7f23d2c3b700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f23cc0b1260 0x7f23cc149ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:02.447 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.435+0000 7f23d2c3b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23cc0b2120 0x7f23cc144d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:02.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.435+0000 7f23d2c3b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23cc0aeb50 0x7f23cc159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:02.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.435+0000 7f23d2c3b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc0ae030 con 0x7f23cc0aeb50 2022-06-29T15:46:02.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.435+0000 7f23d2c3b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc0ae290 con 0x7f23cc0b2120 2022-06-29T15:46:02.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.435+0000 7f23d2c3b700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc0ae530 con 0x7f23cc0b1260 2022-06-29T15:46:02.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.439+0000 7f23d11d8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23cc0aeb50 0x7f23cc159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:02.449 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.439+0000 7f23d09d7700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f23cc0b1260 0x7f23cc149ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:02.449 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.439+0000 7f23d11d8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23cc0aeb50 0x7f23cc159c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48576/0 (socket says 172.21.15.47:48576) 2022-06-29T15:46:02.449 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.439+0000 7f23d11d8700 1 -- 172.21.15.47:0/1634658705 learned_addr learned my addr 172.21.15.47:0/1634658705 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:46:02.450 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.439+0000 7f23cbfff700 1 --2- 172.21.15.47:0/1634658705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23cc0b2120 0x7f23cc144d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:02.450 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.439+0000 7f23d11d8700 1 -- 172.21.15.47:0/1634658705 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f23cc0b1260 msgr2=0x7f23cc149ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:02.450 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.439+0000 7f23d11d8700 1 --2- 172.21.15.47:0/1634658705 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f23cc0b1260 0x7f23cc149ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:02.450 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.439+0000 7f23d11d8700 1 -- 172.21.15.47:0/1634658705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23cc0b2120 msgr2=0x7f23cc144d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:02.451 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.439+0000 7f23d11d8700 1 --2- 172.21.15.47:0/1634658705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23cc0b2120 0x7f23cc144d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:02.451 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.439+0000 7f23d11d8700 1 -- 172.21.15.47:0/1634658705 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c000b180 con 0x7f23cc0aeb50 2022-06-29T15:46:02.451 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.439+0000 7f23d11d8700 1 --2- 172.21.15.47:0/1634658705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23cc0aeb50 0x7f23cc159c90 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7f23c40032f0 tx=0x7f23c4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:02.451 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.439+0000 7f23c9ffb700 1 -- 172.21.15.47:0/1634658705 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23c400b8a0 con 0x7f23cc0aeb50 2022-06-29T15:46:02.451 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.439+0000 7f23c9ffb700 1 -- 172.21.15.47:0/1634658705 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c4021630 con 0x7f23cc0aeb50 2022-06-29T15:46:02.452 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.439+0000 7f23d2c3b700 1 -- 172.21.15.47:0/1634658705 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23c4009530 con 0x7f23cc0aeb50 2022-06-29T15:46:02.452 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.439+0000 7f23c9ffb700 1 -- 172.21.15.47:0/1634658705 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23c4021990 con 0x7f23cc0aeb50 2022-06-29T15:46:02.452 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.439+0000 7f23d2c3b700 1 -- 172.21.15.47:0/1634658705 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23cc0ada50 con 0x7f23cc0aeb50 2022-06-29T15:46:02.452 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.439+0000 7f23d2c3b700 1 -- 172.21.15.47:0/1634658705 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23cc144a30 con 0x7f23cc0aeb50 2022-06-29T15:46:02.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.439+0000 7f23c9ffb700 1 -- 172.21.15.47:0/1634658705 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f23c400f070 con 0x7f23cc0aeb50 2022-06-29T15:46:02.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.443+0000 7f23c9ffb700 1 --2- 172.21.15.47:0/1634658705 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f23ac053b00 0x7f23ac055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:02.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.443+0000 7f23c9ffb700 1 -- 172.21.15.47:0/1634658705 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f23c406d600 con 0x7f23cc0aeb50 2022-06-29T15:46:02.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.443+0000 7f23d09d7700 1 --2- 172.21.15.47:0/1634658705 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f23ac053b00 0x7f23ac055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:02.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.443+0000 7f23d09d7700 1 --2- 172.21.15.47:0/1634658705 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f23ac053b00 0x7f23ac055fc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7f23c000b000 tx=0x7f23c0006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:02.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.447+0000 7f23c9ffb700 1 -- 172.21.15.47:0/1634658705 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f23c40443c0 con 0x7f23cc0aeb50 2022-06-29T15:46:02.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.615+0000 7f23d2c3b700 1 -- 172.21.15.47:0/1634658705 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23cc0573d0 con 0x7f23cc0aeb50 2022-06-29T15:46:02.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.615+0000 7f23c9ffb700 1 -- 172.21.15.47:0/1634658705 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+2492 (secure 0 0 0) 0x7f23c4015090 con 0x7f23cc0aeb50 2022-06-29T15:46:02.620 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:46:02.620 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":24,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5595,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1798995166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1798995166},{"type":"v1","addr":"172.21.15.47:6835","nonce":1798995166}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21},{"gid":5601,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/1641478219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1641478219},{"type":"v1","addr":"172.21.15.64:6833","nonce":1641478219}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:46:01.294125+0000","modified":"2022-06-29T15:46:02.296567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5607":{"gid":5607,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6837/2427298469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2427298469},{"type":"v1","addr":"172.21.15.47:6837","nonce":2427298469}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2022-06-29T15:46:02.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.615+0000 7f23d2c3b700 1 -- 172.21.15.47:0/1634658705 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f23ac053b00 msgr2=0x7f23ac055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:02.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.615+0000 7f23d2c3b700 1 --2- 172.21.15.47:0/1634658705 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f23ac053b00 0x7f23ac055fc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7f23c000b000 tx=0x7f23c0006df0).stop 2022-06-29T15:46:02.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.615+0000 7f23d2c3b700 1 -- 172.21.15.47:0/1634658705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23cc0aeb50 msgr2=0x7f23cc159c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:02.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.615+0000 7f23d2c3b700 1 --2- 172.21.15.47:0/1634658705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23cc0aeb50 0x7f23cc159c90 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7f23c40032f0 tx=0x7f23c4016000).stop 2022-06-29T15:46:02.626 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.615+0000 7f23d2c3b700 1 -- 172.21.15.47:0/1634658705 shutdown_connections 2022-06-29T15:46:02.626 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.615+0000 7f23d2c3b700 1 --2- 172.21.15.47:0/1634658705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23cc0aeb50 0x7f23cc159c90 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:02.626 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.615+0000 7f23d2c3b700 1 --2- 172.21.15.47:0/1634658705 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f23ac053b00 0x7f23ac055fc0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:02.627 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.615+0000 7f23d2c3b700 1 --2- 172.21.15.47:0/1634658705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23cc0b2120 0x7f23cc144d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:02.627 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.615+0000 7f23d2c3b700 1 --2- 172.21.15.47:0/1634658705 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f23cc0b1260 0x7f23cc149ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:02.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.619+0000 7f23d2c3b700 1 -- 172.21.15.47:0/1634658705 >> 172.21.15.47:0/1634658705 conn(0x7f23cc0626f0 msgr2=0x7f23cc145cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:02.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.619+0000 7f23d2c3b700 1 -- 172.21.15.47:0/1634658705 shutdown_connections 2022-06-29T15:46:02.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.619+0000 7f23d2c3b700 1 -- 172.21.15.47:0/1634658705 wait complete. 2022-06-29T15:46:02.629 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 24 2022-06-29T15:46:02.633 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:46:02.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.735+0000 7f7db1729700 1 Processor -- start 2022-06-29T15:46:02.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.735+0000 7f7db1729700 1 -- start start 2022-06-29T15:46:02.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.735+0000 7f7db1729700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7dac0b5c60 0x7f7dac144980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:02.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.735+0000 7f7db1729700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7dac060eb0 0x7f7dac144e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:02.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.735+0000 7f7db1729700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dac0a9120 con 0x7f7dac0a72f0 2022-06-29T15:46:02.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.735+0000 7f7db1729700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dac145360 con 0x7f7dac0b5c60 2022-06-29T15:46:02.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.735+0000 7f7db1729700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dac0b0b80 con 0x7f7dac060eb0 2022-06-29T15:46:02.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.739+0000 7f7dabfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7dac0b5c60 0x7f7dac144980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:02.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.739+0000 7f7dab7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7dac060eb0 0x7f7dac144e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:02.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.739+0000 7f7dab7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7dac060eb0 0x7f7dac144e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:54284/0 (socket says 172.21.15.47:54284) 2022-06-29T15:46:02.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.739+0000 7f7dab7fe700 1 -- 172.21.15.47:0/465143914 learned_addr learned my addr 172.21.15.47:0/465143914 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:46:02.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.739+0000 7f7dabfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7dac0b5c60 0x7f7dac144980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48582/0 (socket says 172.21.15.47:48582) 2022-06-29T15:46:02.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.739+0000 7f7dabfff700 1 -- 172.21.15.47:0/465143914 >> v2:172.21.15.64:3300/0 conn(0x7f7dac060eb0 msgr2=0x7f7dac144e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:46:02.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.739+0000 7f7dabfff700 1 --2- 172.21.15.47:0/465143914 >> v2:172.21.15.64:3300/0 conn(0x7f7dac060eb0 0x7f7dac144e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:02.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.739+0000 7f7dabfff700 1 -- 172.21.15.47:0/465143914 >> v1:172.21.15.64:6789/0 conn(0x7f7dac0a72f0 legacy=0x7f7dac0b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:02.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.739+0000 7f7dabfff700 1 -- 172.21.15.47:0/465143914 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dac0a89e0 con 0x7f7dac0b5c60 2022-06-29T15:46:02.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.739+0000 7f7dab7fe700 1 --2- 172.21.15.47:0/465143914 >> v2:172.21.15.64:3300/0 conn(0x7f7dac060eb0 0x7f7dac144e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:46:02.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.739+0000 7f7dabfff700 1 --2- 172.21.15.47:0/465143914 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7dac0b5c60 0x7f7dac144980 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7f7d94002f70 tx=0x7f7d94006d90).ready entity=mon.0 client_cookie=1ac70d16193ca920 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:02.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.739+0000 7f7daaffd700 1 -- 172.21.15.47:0/465143914 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d9400d4a0 con 0x7f7dac0b5c60 2022-06-29T15:46:02.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.739+0000 7f7daaffd700 1 -- 172.21.15.47:0/465143914 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d94000e90 con 0x7f7dac0b5c60 2022-06-29T15:46:02.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.739+0000 7f7daaffd700 1 -- 172.21.15.47:0/465143914 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d9400d7f0 con 0x7f7dac0b5c60 2022-06-29T15:46:02.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.739+0000 7f7db1729700 1 -- 172.21.15.47:0/465143914 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7dac0b5c60 msgr2=0x7f7dac144980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:02.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.739+0000 7f7db1729700 1 --2- 172.21.15.47:0/465143914 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7dac0b5c60 0x7f7dac144980 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7f7d94002f70 tx=0x7f7d94006d90).stop 2022-06-29T15:46:02.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.739+0000 7f7db1729700 1 -- 172.21.15.47:0/465143914 shutdown_connections 2022-06-29T15:46:02.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.739+0000 7f7db1729700 1 --2- 172.21.15.47:0/465143914 >> v2:172.21.15.64:3300/0 conn(0x7f7dac060eb0 0x7f7dac144e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:02.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.739+0000 7f7db1729700 1 --2- 172.21.15.47:0/465143914 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7dac0b5c60 0x7f7dac144980 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:02.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.739+0000 7f7db1729700 1 -- 172.21.15.47:0/465143914 >> 172.21.15.47:0/465143914 conn(0x7f7dac062560 msgr2=0x7f7dac062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:02.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.739+0000 7f7db1729700 1 -- 172.21.15.47:0/465143914 shutdown_connections 2022-06-29T15:46:02.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.739+0000 7f7db1729700 1 -- 172.21.15.47:0/465143914 wait complete. 2022-06-29T15:46:02.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.743+0000 7f7db1729700 1 Processor -- start 2022-06-29T15:46:02.752 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.743+0000 7f7db1729700 1 -- start start 2022-06-29T15:46:02.752 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.743+0000 7f7db1729700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7dac060eb0 0x7f7dac15a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:02.752 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.743+0000 7f7db1729700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7dac0a72f0 0x7f7dac140330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:02.753 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.743+0000 7f7db1729700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7dac0b5c60 0x7f7dac145580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:02.753 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.743+0000 7f7db1729700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dac140b50 con 0x7f7dac0b5c60 2022-06-29T15:46:02.753 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.743+0000 7f7db1729700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dac0ab580 con 0x7f7dac0a72f0 2022-06-29T15:46:02.754 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.743+0000 7f7db1729700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dac15aa90 con 0x7f7dac060eb0 2022-06-29T15:46:02.754 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.743+0000 7f7db0f28700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7dac0b5c60 0x7f7dac145580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:02.754 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.743+0000 7f7dabfff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7dac060eb0 0x7f7dac15a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:02.754 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.743+0000 7f7db0f28700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7dac0b5c60 0x7f7dac145580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48588/0 (socket says 172.21.15.47:48588) 2022-06-29T15:46:02.755 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.743+0000 7f7db0f28700 1 -- 172.21.15.47:0/964167778 learned_addr learned my addr 172.21.15.47:0/964167778 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:46:02.755 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.743+0000 7f7dab7fe700 1 --2- 172.21.15.47:0/964167778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7dac0a72f0 0x7f7dac140330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:02.755 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.743+0000 7f7db0f28700 1 -- 172.21.15.47:0/964167778 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7dac060eb0 msgr2=0x7f7dac15a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:02.755 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.743+0000 7f7db0f28700 1 --2- 172.21.15.47:0/964167778 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7dac060eb0 0x7f7dac15a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:02.756 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.743+0000 7f7db0f28700 1 -- 172.21.15.47:0/964167778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7dac0a72f0 msgr2=0x7f7dac140330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:02.756 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.743+0000 7f7db0f28700 1 --2- 172.21.15.47:0/964167778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7dac0a72f0 0x7f7dac140330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:02.756 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.743+0000 7f7db0f28700 1 -- 172.21.15.47:0/964167778 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d9c009530 con 0x7f7dac0b5c60 2022-06-29T15:46:02.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.743+0000 7f7db0f28700 1 --2- 172.21.15.47:0/964167778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7dac0b5c60 0x7f7dac145580 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7f7dac062280 tx=0x7f7da4007b80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:02.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.743+0000 7f7da97fa700 1 -- 172.21.15.47:0/964167778 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7da4012400 con 0x7f7dac0b5c60 2022-06-29T15:46:02.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.743+0000 7f7da97fa700 1 -- 172.21.15.47:0/964167778 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7da4012560 con 0x7f7dac0b5c60 2022-06-29T15:46:02.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.743+0000 7f7db1729700 1 -- 172.21.15.47:0/964167778 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d9400b120 con 0x7f7dac0b5c60 2022-06-29T15:46:02.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.743+0000 7f7da97fa700 1 -- 172.21.15.47:0/964167778 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7da40128c0 con 0x7f7dac0b5c60 2022-06-29T15:46:02.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.743+0000 7f7db1729700 1 -- 172.21.15.47:0/964167778 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dac0ac050 con 0x7f7dac0b5c60 2022-06-29T15:46:02.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.747+0000 7f7db1729700 1 -- 172.21.15.47:0/964167778 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dac0a9460 con 0x7f7dac0b5c60 2022-06-29T15:46:02.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.747+0000 7f7da97fa700 1 -- 172.21.15.47:0/964167778 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7da4019030 con 0x7f7dac0b5c60 2022-06-29T15:46:02.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.747+0000 7f7da97fa700 1 --2- 172.21.15.47:0/964167778 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7d90053b00 0x7f7d90055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:02.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.747+0000 7f7da97fa700 1 -- 172.21.15.47:0/964167778 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f7da406b280 con 0x7f7dac0b5c60 2022-06-29T15:46:02.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.747+0000 7f7dabfff700 1 --2- 172.21.15.47:0/964167778 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7d90053b00 0x7f7d90055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:02.767 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.747+0000 7f7dabfff700 1 --2- 172.21.15.47:0/964167778 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7d90053b00 0x7f7d90055fc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f7d9401a000 tx=0x7f7d94014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:02.767 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.755+0000 7f7da97fa700 1 -- 172.21.15.47:0/964167778 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7da4041fe0 con 0x7f7dac0b5c60 2022-06-29T15:46:02.880 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.875+0000 7f7db1729700 1 -- 172.21.15.47:0/964167778 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7dac0573d0 con 0x7f7dac0b5c60 2022-06-29T15:46:02.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.879+0000 7f7da97fa700 1 -- 172.21.15.47:0/964167778 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+16286 (secure 0 0 0) 0x7f7da4017020 con 0x7f7dac0b5c60 2022-06-29T15:46:02.883 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:46:02.883 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":57,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:46:02.297205+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:45:59.710738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2022-06-29T15:46:00.582445+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:46:02.885 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.879+0000 7f7db1729700 1 -- 172.21.15.47:0/964167778 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7d90053b00 msgr2=0x7f7d90055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:02.885 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.879+0000 7f7db1729700 1 --2- 172.21.15.47:0/964167778 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7d90053b00 0x7f7d90055fc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f7d9401a000 tx=0x7f7d94014000).stop 2022-06-29T15:46:02.885 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.879+0000 7f7db1729700 1 -- 172.21.15.47:0/964167778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7dac0b5c60 msgr2=0x7f7dac145580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:02.886 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.879+0000 7f7db1729700 1 --2- 172.21.15.47:0/964167778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7dac0b5c60 0x7f7dac145580 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7f7dac062280 tx=0x7f7da4007b80).stop 2022-06-29T15:46:02.886 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.883+0000 7f7db1729700 1 -- 172.21.15.47:0/964167778 shutdown_connections 2022-06-29T15:46:02.886 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.883+0000 7f7db1729700 1 --2- 172.21.15.47:0/964167778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7dac0b5c60 0x7f7dac145580 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:02.886 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.883+0000 7f7db1729700 1 --2- 172.21.15.47:0/964167778 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7d90053b00 0x7f7d90055fc0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:02.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.883+0000 7f7db1729700 1 --2- 172.21.15.47:0/964167778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7dac0a72f0 0x7f7dac140330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:02.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.883+0000 7f7db1729700 1 --2- 172.21.15.47:0/964167778 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7dac060eb0 0x7f7dac15a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:02.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.883+0000 7f7db1729700 1 -- 172.21.15.47:0/964167778 >> 172.21.15.47:0/964167778 conn(0x7f7dac062560 msgr2=0x7f7dac13d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:02.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.883+0000 7f7db1729700 1 -- 172.21.15.47:0/964167778 shutdown_connections 2022-06-29T15:46:02.888 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:02.883+0000 7f7db1729700 1 -- 172.21.15.47:0/964167778 wait complete. 2022-06-29T15:46:02.897 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-29T15:46:03.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.011+0000 7f81f2d3e700 1 Processor -- start 2022-06-29T15:46:03.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.011+0000 7f81f2d3e700 1 -- start start 2022-06-29T15:46:03.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.011+0000 7f81f2d3e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f81ec060ee0 0x7f81ec0612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:03.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.011+0000 7f81f2d3e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f81ec1471a0 0x7f81ec066f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:03.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.011+0000 7f81f2d3e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81ec067610 con 0x7f81ec0b4300 2022-06-29T15:46:03.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.011+0000 7f81f2d3e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81ec1467c0 con 0x7f81ec060ee0 2022-06-29T15:46:03.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.011+0000 7f81f2d3e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81ec146a60 con 0x7f81ec1471a0 2022-06-29T15:46:03.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.011+0000 7f81f0ada700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f81ec060ee0 0x7f81ec0612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:03.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.011+0000 7f81f12db700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f81ec1471a0 0x7f81ec066f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:03.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.011+0000 7f81f0ada700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f81ec060ee0 0x7f81ec0612c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:54298/0 (socket says 172.21.15.47:54298) 2022-06-29T15:46:03.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.011+0000 7f81f0ada700 1 -- 172.21.15.47:0/18208253 learned_addr learned my addr 172.21.15.47:0/18208253 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:46:03.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.011+0000 7f81f0ada700 1 -- 172.21.15.47:0/18208253 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f81ec1471a0 msgr2=0x7f81ec066f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:46:03.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.011+0000 7f81f0ada700 1 --2- 172.21.15.47:0/18208253 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f81ec1471a0 0x7f81ec066f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:03.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.011+0000 7f81f0ada700 1 -- 172.21.15.47:0/18208253 >> v1:172.21.15.64:6789/0 conn(0x7f81ec0b4300 legacy=0x7f81ec1430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:03.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.011+0000 7f81f0ada700 1 -- 172.21.15.47:0/18208253 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81ec146d00 con 0x7f81ec060ee0 2022-06-29T15:46:03.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.011+0000 7f81f0ada700 1 --2- 172.21.15.47:0/18208253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81ec060ee0 0x7f81ec0612c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f81ec0ae790 tx=0x7f81e0006eb0).ready entity=mon.1 client_cookie=201af8be1515c527 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:03.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.011+0000 7f81eb7fe700 1 -- 172.21.15.47:0/18208253 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81e000d5c0 con 0x7f81ec060ee0 2022-06-29T15:46:03.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.011+0000 7f81eb7fe700 1 -- 172.21.15.47:0/18208253 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81e0000e90 con 0x7f81ec060ee0 2022-06-29T15:46:03.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.011+0000 7f81eb7fe700 1 -- 172.21.15.47:0/18208253 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81e000d950 con 0x7f81ec060ee0 2022-06-29T15:46:03.029 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.015+0000 7f81f2d3e700 1 -- 172.21.15.47:0/18208253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81ec060ee0 msgr2=0x7f81ec0612c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:03.029 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.015+0000 7f81f2d3e700 1 --2- 172.21.15.47:0/18208253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81ec060ee0 0x7f81ec0612c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f81ec0ae790 tx=0x7f81e0006eb0).stop 2022-06-29T15:46:03.029 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.015+0000 7f81f2d3e700 1 -- 172.21.15.47:0/18208253 shutdown_connections 2022-06-29T15:46:03.030 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.015+0000 7f81f2d3e700 1 --2- 172.21.15.47:0/18208253 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f81ec1471a0 0x7f81ec066f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:03.030 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.015+0000 7f81f2d3e700 1 --2- 172.21.15.47:0/18208253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81ec060ee0 0x7f81ec0612c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:03.031 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.015+0000 7f81f2d3e700 1 -- 172.21.15.47:0/18208253 >> 172.21.15.47:0/18208253 conn(0x7f81ec05c510 msgr2=0x7f81ec05f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:03.031 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.015+0000 7f81f2d3e700 1 -- 172.21.15.47:0/18208253 shutdown_connections 2022-06-29T15:46:03.031 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.015+0000 7f81f2d3e700 1 -- 172.21.15.47:0/18208253 wait complete. 2022-06-29T15:46:03.032 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.015+0000 7f81f2d3e700 1 Processor -- start 2022-06-29T15:46:03.032 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.015+0000 7f81f2d3e700 1 -- start start 2022-06-29T15:46:03.032 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.015+0000 7f81f2d3e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81ec060ee0 0x7f81ec148350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:03.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.015+0000 7f81f2d3e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f81ec0b4300 0x7f81ec0b1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:03.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.019+0000 7f81f2d3e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81ec1471a0 0x7f81ec13c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:03.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.019+0000 7f81f2d3e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ec147970 con 0x7f81ec1471a0 2022-06-29T15:46:03.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.019+0000 7f81f2d3e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ec147c10 con 0x7f81ec060ee0 2022-06-29T15:46:03.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.019+0000 7f81f2d3e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81ec147eb0 con 0x7f81ec0b4300 2022-06-29T15:46:03.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.019+0000 7f81f12db700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81ec1471a0 0x7f81ec13c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:03.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.019+0000 7f81f0ada700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81ec060ee0 0x7f81ec148350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:03.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.019+0000 7f81ebfff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f81ec0b4300 0x7f81ec0b1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:03.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.019+0000 7f81f12db700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81ec1471a0 0x7f81ec13c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48602/0 (socket says 172.21.15.47:48602) 2022-06-29T15:46:03.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.019+0000 7f81f12db700 1 -- 172.21.15.47:0/2783566534 learned_addr learned my addr 172.21.15.47:0/2783566534 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:46:03.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.019+0000 7f81f12db700 1 -- 172.21.15.47:0/2783566534 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f81ec0b4300 msgr2=0x7f81ec0b1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:03.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.019+0000 7f81f12db700 1 --2- 172.21.15.47:0/2783566534 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f81ec0b4300 0x7f81ec0b1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:03.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.019+0000 7f81f12db700 1 -- 172.21.15.47:0/2783566534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81ec060ee0 msgr2=0x7f81ec148350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:03.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.019+0000 7f81f12db700 1 --2- 172.21.15.47:0/2783566534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81ec060ee0 0x7f81ec148350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:03.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.019+0000 7f81f12db700 1 -- 172.21.15.47:0/2783566534 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e000b1e0 con 0x7f81ec1471a0 2022-06-29T15:46:03.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.019+0000 7f81ebfff700 1 --2- 172.21.15.47:0/2783566534 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f81ec0b4300 0x7f81ec0b1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:46:03.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.019+0000 7f81f0ada700 1 --2- 172.21.15.47:0/2783566534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81ec060ee0 0x7f81ec148350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:46:03.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.019+0000 7f81f12db700 1 --2- 172.21.15.47:0/2783566534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81ec1471a0 0x7f81ec13c6c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7f81e4006e00 tx=0x7f81e40091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:03.038 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.019+0000 7f81e9ffb700 1 -- 172.21.15.47:0/2783566534 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81e4012600 con 0x7f81ec1471a0 2022-06-29T15:46:03.038 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.019+0000 7f81e9ffb700 1 -- 172.21.15.47:0/2783566534 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81e4012760 con 0x7f81ec1471a0 2022-06-29T15:46:03.038 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.019+0000 7f81f2d3e700 1 -- 172.21.15.47:0/2783566534 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81ec13cbb0 con 0x7f81ec1471a0 2022-06-29T15:46:03.038 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.019+0000 7f81e9ffb700 1 -- 172.21.15.47:0/2783566534 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81e4012ac0 con 0x7f81ec1471a0 2022-06-29T15:46:03.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.019+0000 7f81f2d3e700 1 -- 172.21.15.47:0/2783566534 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81ec146450 con 0x7f81ec1471a0 2022-06-29T15:46:03.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.019+0000 7f81f2d3e700 1 -- 172.21.15.47:0/2783566534 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81ec148070 con 0x7f81ec1471a0 2022-06-29T15:46:03.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.023+0000 7f81e9ffb700 1 -- 172.21.15.47:0/2783566534 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f81e4022070 con 0x7f81ec1471a0 2022-06-29T15:46:03.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.023+0000 7f81e9ffb700 1 --2- 172.21.15.47:0/2783566534 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f81cc053a70 0x7f81cc055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:03.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.023+0000 7f81e9ffb700 1 -- 172.21.15.47:0/2783566534 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f81e406c650 con 0x7f81ec1471a0 2022-06-29T15:46:03.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.023+0000 7f81f0ada700 1 --2- 172.21.15.47:0/2783566534 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f81cc053a70 0x7f81cc055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:03.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.023+0000 7f81f0ada700 1 --2- 172.21.15.47:0/2783566534 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f81cc053a70 0x7f81cc055f30 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f81e001c000 tx=0x7f81e0015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:03.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.031+0000 7f81e9ffb700 1 -- 172.21.15.47:0/2783566534 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f81e4043310 con 0x7f81ec1471a0 2022-06-29T15:46:03.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.203+0000 7f81f2d3e700 1 -- 172.21.15.47:0/2783566534 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f81ec0573d0 con 0x7f81ec1471a0 2022-06-29T15:46:03.213 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.211+0000 7f81e9ffb700 1 -- 172.21.15.47:0/2783566534 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v18) v1 ==== 184+0+0 (secure 0 0 0) 0x7f81e4016020 con 0x7f81ec1471a0 2022-06-29T15:46:03.215 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.211+0000 7f81f2d3e700 1 -- 172.21.15.47:0/2783566534 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f81cc053a70 msgr2=0x7f81cc055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:03.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.211+0000 7f81f2d3e700 1 --2- 172.21.15.47:0/2783566534 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f81cc053a70 0x7f81cc055f30 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f81e001c000 tx=0x7f81e0015000).stop 2022-06-29T15:46:03.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.211+0000 7f81f2d3e700 1 -- 172.21.15.47:0/2783566534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81ec1471a0 msgr2=0x7f81ec13c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:03.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.211+0000 7f81f2d3e700 1 --2- 172.21.15.47:0/2783566534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81ec1471a0 0x7f81ec13c6c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7f81e4006e00 tx=0x7f81e40091d0).stop 2022-06-29T15:46:03.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.211+0000 7f81f2d3e700 1 -- 172.21.15.47:0/2783566534 shutdown_connections 2022-06-29T15:46:03.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.211+0000 7f81f2d3e700 1 --2- 172.21.15.47:0/2783566534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81ec1471a0 0x7f81ec13c6c0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:03.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.211+0000 7f81f2d3e700 1 --2- 172.21.15.47:0/2783566534 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f81ec0b4300 0x7f81ec0b1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:03.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.211+0000 7f81f2d3e700 1 --2- 172.21.15.47:0/2783566534 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f81cc053a70 0x7f81cc055f30 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:03.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.211+0000 7f81f2d3e700 1 --2- 172.21.15.47:0/2783566534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81ec060ee0 0x7f81ec148350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:03.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.211+0000 7f81f2d3e700 1 -- 172.21.15.47:0/2783566534 >> 172.21.15.47:0/2783566534 conn(0x7f81ec05c510 msgr2=0x7f81ec0b8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:03.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.211+0000 7f81f2d3e700 1 -- 172.21.15.47:0/2783566534 shutdown_connections 2022-06-29T15:46:03.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.211+0000 7f81f2d3e700 1 -- 172.21.15.47:0/2783566534 wait complete. 2022-06-29T15:46:03.219 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.0 2022-06-29T15:46:03.227 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-29T15:46:03.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.331+0000 7f58c3fff700 1 Processor -- start 2022-06-29T15:46:03.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.331+0000 7f58c3fff700 1 -- start start 2022-06-29T15:46:03.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.335+0000 7f58c3fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f58c40b1610 0x7f58c40b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:03.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.335+0000 7f58c3fff700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58c4060790 con 0x7f58c40b58d0 2022-06-29T15:46:03.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.335+0000 7f58c3fff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58c414ad80 con 0x7f58c414b4c0 2022-06-29T15:46:03.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.335+0000 7f58c3fff700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58c40a99e0 con 0x7f58c40b1610 2022-06-29T15:46:03.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.335+0000 7f58c27fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f58c40b1610 0x7f58c40b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:03.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.335+0000 7f58c27fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f58c40b1610 0x7f58c40b5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48608/0 (socket says 172.21.15.47:48608) 2022-06-29T15:46:03.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.335+0000 7f58c27fc700 1 -- 172.21.15.47:0/3509977809 learned_addr learned my addr 172.21.15.47:0/3509977809 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:46:03.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.335+0000 7f58c27fc700 1 -- 172.21.15.47:0/3509977809 >> v1:172.21.15.64:6789/0 conn(0x7f58c414b4c0 legacy=0x7f58c40aa130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:03.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.335+0000 7f58c27fc700 1 -- 172.21.15.47:0/3509977809 >> v1:172.21.15.47:6789/0 conn(0x7f58c40b58d0 legacy=0x7f58c41475f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:03.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.335+0000 7f58c27fc700 1 -- 172.21.15.47:0/3509977809 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c414b020 con 0x7f58c40b1610 2022-06-29T15:46:03.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.335+0000 7f58c27fc700 1 --2- 172.21.15.47:0/3509977809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f58c40b1610 0x7f58c40b5cb0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7f58ac002f70 tx=0x7f58ac006eb0).ready entity=mon.0 client_cookie=4eb7635698d9104 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:03.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.335+0000 7f58c1ffb700 1 -- 172.21.15.47:0/3509977809 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f58ac00d660 con 0x7f58c40b1610 2022-06-29T15:46:03.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.335+0000 7f58c1ffb700 1 -- 172.21.15.47:0/3509977809 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58ac000e90 con 0x7f58c40b1610 2022-06-29T15:46:03.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.335+0000 7f58c1ffb700 1 -- 172.21.15.47:0/3509977809 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f58ac00d9f0 con 0x7f58c40b1610 2022-06-29T15:46:03.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.335+0000 7f58c3fff700 1 -- 172.21.15.47:0/3509977809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f58c40b1610 msgr2=0x7f58c40b5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:03.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.335+0000 7f58c3fff700 1 --2- 172.21.15.47:0/3509977809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f58c40b1610 0x7f58c40b5cb0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7f58ac002f70 tx=0x7f58ac006eb0).stop 2022-06-29T15:46:03.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.335+0000 7f58c3fff700 1 -- 172.21.15.47:0/3509977809 shutdown_connections 2022-06-29T15:46:03.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.335+0000 7f58c3fff700 1 --2- 172.21.15.47:0/3509977809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f58c40b1610 0x7f58c40b5cb0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:03.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.335+0000 7f58c3fff700 1 -- 172.21.15.47:0/3509977809 >> 172.21.15.47:0/3509977809 conn(0x7f58c4062560 msgr2=0x7f58c4062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:03.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.335+0000 7f58c3fff700 1 -- 172.21.15.47:0/3509977809 shutdown_connections 2022-06-29T15:46:03.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.335+0000 7f58c3fff700 1 -- 172.21.15.47:0/3509977809 wait complete. 2022-06-29T15:46:03.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.335+0000 7f58c3fff700 1 Processor -- start 2022-06-29T15:46:03.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.335+0000 7f58c3fff700 1 -- start start 2022-06-29T15:46:03.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.339+0000 7f58c3fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f58c40b1610 0x7f58c4066c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:03.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.339+0000 7f58c3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f58c40b58d0 0x7f58c4065c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:03.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.339+0000 7f58c3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58c414b4c0 0x7f58c4066120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:03.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.339+0000 7f58c3fff700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c40b4d20 con 0x7f58c40b58d0 2022-06-29T15:46:03.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.339+0000 7f58c3fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c40ae860 con 0x7f58c414b4c0 2022-06-29T15:46:03.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.339+0000 7f58c3fff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58c40aeb00 con 0x7f58c40b1610 2022-06-29T15:46:03.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.339+0000 7f58c27fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f58c40b58d0 0x7f58c4065c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:03.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.339+0000 7f58c27fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f58c40b58d0 0x7f58c4065c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48612/0 (socket says 172.21.15.47:48612) 2022-06-29T15:46:03.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.339+0000 7f58c27fc700 1 -- 172.21.15.47:0/2576116954 learned_addr learned my addr 172.21.15.47:0/2576116954 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:46:03.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.339+0000 7f58c2ffd700 1 --2- 172.21.15.47:0/2576116954 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f58c40b1610 0x7f58c4066c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:03.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.339+0000 7f58c37fe700 1 --2- 172.21.15.47:0/2576116954 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58c414b4c0 0x7f58c4066120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:03.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.339+0000 7f58c27fc700 1 -- 172.21.15.47:0/2576116954 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f58c40b1610 msgr2=0x7f58c4066c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:03.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.339+0000 7f58c27fc700 1 --2- 172.21.15.47:0/2576116954 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f58c40b1610 0x7f58c4066c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:03.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.339+0000 7f58c27fc700 1 -- 172.21.15.47:0/2576116954 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58c414b4c0 msgr2=0x7f58c4066120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:03.352 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.339+0000 7f58c27fc700 1 --2- 172.21.15.47:0/2576116954 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58c414b4c0 0x7f58c4066120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:03.352 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.339+0000 7f58c27fc700 1 -- 172.21.15.47:0/2576116954 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ac00b1e0 con 0x7f58c40b58d0 2022-06-29T15:46:03.352 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.339+0000 7f58c27fc700 1 --2- 172.21.15.47:0/2576116954 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f58c40b58d0 0x7f58c4065c30 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7f58ac002f70 tx=0x7f58ac015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:03.352 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.339+0000 7f58bbfff700 1 -- 172.21.15.47:0/2576116954 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f58ac0068b0 con 0x7f58c40b58d0 2022-06-29T15:46:03.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.339+0000 7f58bbfff700 1 -- 172.21.15.47:0/2576116954 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58ac011ae0 con 0x7f58c40b58d0 2022-06-29T15:46:03.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.339+0000 7f58c3fff700 1 -- 172.21.15.47:0/2576116954 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58c40aefa0 con 0x7f58c40b58d0 2022-06-29T15:46:03.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.339+0000 7f58bbfff700 1 -- 172.21.15.47:0/2576116954 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f58ac011e40 con 0x7f58c40b58d0 2022-06-29T15:46:03.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.339+0000 7f58c3fff700 1 -- 172.21.15.47:0/2576116954 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58c40adb90 con 0x7f58c40b58d0 2022-06-29T15:46:03.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.343+0000 7f58c3fff700 1 -- 172.21.15.47:0/2576116954 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58c40b4f20 con 0x7f58c40b58d0 2022-06-29T15:46:03.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.343+0000 7f58bbfff700 1 -- 172.21.15.47:0/2576116954 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f58ac025030 con 0x7f58c40b58d0 2022-06-29T15:46:03.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.343+0000 7f58bbfff700 1 --2- 172.21.15.47:0/2576116954 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f58a8053950 0x7f58a8055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:03.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.343+0000 7f58bbfff700 1 -- 172.21.15.47:0/2576116954 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f58ac06d1c0 con 0x7f58c40b58d0 2022-06-29T15:46:03.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.343+0000 7f58c2ffd700 1 --2- 172.21.15.47:0/2576116954 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f58a8053950 0x7f58a8055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:03.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.343+0000 7f58c2ffd700 1 --2- 172.21.15.47:0/2576116954 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f58a8053950 0x7f58a8055e10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f58bc00d000 tx=0x7f58bc007f10).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:03.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.351+0000 7f58bbfff700 1 -- 172.21.15.47:0/2576116954 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f58ac043f80 con 0x7f58c40b58d0 2022-06-29T15:46:03.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.519+0000 7f58c3fff700 1 -- 172.21.15.47:0/2576116954 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f58c40aecc0 con 0x7f58c40b58d0 2022-06-29T15:46:03.529 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.527+0000 7f58bbfff700 1 -- 172.21.15.47:0/2576116954 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v19) v1 ==== 184+0+0 (secure 0 0 0) 0x7f58ac023020 con 0x7f58c40b58d0 2022-06-29T15:46:03.532 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.527+0000 7f58c3fff700 1 -- 172.21.15.47:0/2576116954 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f58a8053950 msgr2=0x7f58a8055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:03.532 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.527+0000 7f58c3fff700 1 --2- 172.21.15.47:0/2576116954 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f58a8053950 0x7f58a8055e10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f58bc00d000 tx=0x7f58bc007f10).stop 2022-06-29T15:46:03.533 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.527+0000 7f58c3fff700 1 -- 172.21.15.47:0/2576116954 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f58c40b58d0 msgr2=0x7f58c4065c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:03.533 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.527+0000 7f58c3fff700 1 --2- 172.21.15.47:0/2576116954 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f58c40b58d0 0x7f58c4065c30 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7f58ac002f70 tx=0x7f58ac015000).stop 2022-06-29T15:46:03.533 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.527+0000 7f58c3fff700 1 -- 172.21.15.47:0/2576116954 shutdown_connections 2022-06-29T15:46:03.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.527+0000 7f58c3fff700 1 --2- 172.21.15.47:0/2576116954 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f58a8053950 0x7f58a8055e10 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:03.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.527+0000 7f58c3fff700 1 --2- 172.21.15.47:0/2576116954 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58c414b4c0 0x7f58c4066120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:03.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.527+0000 7f58c3fff700 1 --2- 172.21.15.47:0/2576116954 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f58c40b58d0 0x7f58c4065c30 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:03.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.527+0000 7f58c3fff700 1 --2- 172.21.15.47:0/2576116954 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f58c40b1610 0x7f58c4066c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:03.536 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.527+0000 7f58c3fff700 1 -- 172.21.15.47:0/2576116954 >> 172.21.15.47:0/2576116954 conn(0x7f58c4062560 msgr2=0x7f58c4149ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:03.536 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.527+0000 7f58c3fff700 1 -- 172.21.15.47:0/2576116954 shutdown_connections 2022-06-29T15:46:03.537 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.527+0000 7f58c3fff700 1 -- 172.21.15.47:0/2576116954 wait complete. 2022-06-29T15:46:03.537 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.1 2022-06-29T15:46:03.545 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:46:03.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.639+0000 7f190b0e0700 1 Processor -- start 2022-06-29T15:46:03.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.639+0000 7f190b0e0700 1 -- start start 2022-06-29T15:46:03.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.639+0000 7f190b0e0700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f19040b1260 0x7f190405ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:03.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.639+0000 7f190b0e0700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19040b0740 con 0x7f19040b2120 2022-06-29T15:46:03.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.639+0000 7f190b0e0700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19040b0970 con 0x7f190405f0b0 2022-06-29T15:46:03.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.639+0000 7f190b0e0700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1904147460 con 0x7f19040b1260 2022-06-29T15:46:03.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.639+0000 7f190967d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f19040b1260 0x7f190405ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:03.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.639+0000 7f190967d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f19040b1260 0x7f190405ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48624/0 (socket says 172.21.15.47:48624) 2022-06-29T15:46:03.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.639+0000 7f1908e7c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f19040b2120 0x7f1904147af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34176/0 (socket says 172.21.15.47:34176) 2022-06-29T15:46:03.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.639+0000 7f190967d700 1 -- 172.21.15.47:0/2995668620 learned_addr learned my addr 172.21.15.47:0/2995668620 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:46:03.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.639+0000 7f190967d700 1 -- 172.21.15.47:0/2995668620 >> v1:172.21.15.64:6789/0 conn(0x7f190405f0b0 legacy=0x7f190405f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:03.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.639+0000 7f190967d700 1 -- 172.21.15.47:0/2995668620 >> v1:172.21.15.47:6789/0 conn(0x7f19040b2120 legacy=0x7f1904147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:03.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.639+0000 7f190967d700 1 -- 172.21.15.47:0/2995668620 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19040b0c10 con 0x7f19040b1260 2022-06-29T15:46:03.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.639+0000 7f19037fe700 1 -- 172.21.15.47:0/2995668620 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2654559793 0 0) 0x7f19040b0740 con 0x7f19040b2120 2022-06-29T15:46:03.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.639+0000 7f190967d700 1 --2- 172.21.15.47:0/2995668620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19040b1260 0x7f190405ffc0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7f1904141a70 tx=0x7f18fc009290).ready entity=mon.0 client_cookie=fff334db705cef6 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:03.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.639+0000 7f19037fe700 1 -- 172.21.15.47:0/2995668620 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18fc00b820 con 0x7f19040b1260 2022-06-29T15:46:03.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.643+0000 7f19037fe700 1 -- 172.21.15.47:0/2995668620 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18fc00b980 con 0x7f19040b1260 2022-06-29T15:46:03.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.643+0000 7f19037fe700 1 -- 172.21.15.47:0/2995668620 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18fc00baf0 con 0x7f19040b1260 2022-06-29T15:46:03.653 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.643+0000 7f190b0e0700 1 -- 172.21.15.47:0/2995668620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19040b1260 msgr2=0x7f190405ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:03.653 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.643+0000 7f190b0e0700 1 --2- 172.21.15.47:0/2995668620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19040b1260 0x7f190405ffc0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7f1904141a70 tx=0x7f18fc009290).stop 2022-06-29T15:46:03.653 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.643+0000 7f190b0e0700 1 -- 172.21.15.47:0/2995668620 shutdown_connections 2022-06-29T15:46:03.654 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.643+0000 7f190b0e0700 1 --2- 172.21.15.47:0/2995668620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19040b1260 0x7f190405ffc0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:03.654 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.643+0000 7f190b0e0700 1 -- 172.21.15.47:0/2995668620 >> 172.21.15.47:0/2995668620 conn(0x7f19040626f0 msgr2=0x7f1904062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:03.654 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.643+0000 7f190b0e0700 1 -- 172.21.15.47:0/2995668620 shutdown_connections 2022-06-29T15:46:03.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.643+0000 7f190b0e0700 1 -- 172.21.15.47:0/2995668620 wait complete. 2022-06-29T15:46:03.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.643+0000 7f190b0e0700 1 Processor -- start 2022-06-29T15:46:03.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.643+0000 7f190b0e0700 1 -- start start 2022-06-29T15:46:03.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.643+0000 7f190b0e0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19040b1260 0x7f1904066e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:03.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.643+0000 7f190b0e0700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f19040b2120 0x7f1904064d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:03.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.643+0000 7f190b0e0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19040a20b0 0x7f190415a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:03.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.643+0000 7f190b0e0700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19040633b0 con 0x7f19040a20b0 2022-06-29T15:46:03.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.643+0000 7f190b0e0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19040651f0 con 0x7f19040b1260 2022-06-29T15:46:03.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.643+0000 7f190b0e0700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1904063e80 con 0x7f19040b2120 2022-06-29T15:46:03.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.647+0000 7f190967d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19040a20b0 0x7f190415a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:03.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.647+0000 7f190967d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19040a20b0 0x7f190415a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48630/0 (socket says 172.21.15.47:48630) 2022-06-29T15:46:03.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.647+0000 7f190967d700 1 -- 172.21.15.47:0/3158026588 learned_addr learned my addr 172.21.15.47:0/3158026588 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:46:03.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.647+0000 7f1908e7c700 1 --2- 172.21.15.47:0/3158026588 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19040b1260 0x7f1904066e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:03.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.647+0000 7f1903fff700 1 --2- 172.21.15.47:0/3158026588 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f19040b2120 0x7f1904064d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:03.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.647+0000 7f190967d700 1 -- 172.21.15.47:0/3158026588 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f19040b2120 msgr2=0x7f1904064d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:03.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.647+0000 7f190967d700 1 --2- 172.21.15.47:0/3158026588 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f19040b2120 0x7f1904064d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:03.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.647+0000 7f190967d700 1 -- 172.21.15.47:0/3158026588 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19040b1260 msgr2=0x7f1904066e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:03.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.647+0000 7f190967d700 1 --2- 172.21.15.47:0/3158026588 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19040b1260 0x7f1904066e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:03.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.647+0000 7f190967d700 1 -- 172.21.15.47:0/3158026588 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18fc009530 con 0x7f19040a20b0 2022-06-29T15:46:03.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.647+0000 7f190967d700 1 --2- 172.21.15.47:0/3158026588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19040a20b0 0x7f190415a1b0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7f1904141a70 tx=0x7f18fc016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:03.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.647+0000 7f1901ffb700 1 -- 172.21.15.47:0/3158026588 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18fc00be90 con 0x7f19040a20b0 2022-06-29T15:46:03.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.647+0000 7f1901ffb700 1 -- 172.21.15.47:0/3158026588 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18fc01f3f0 con 0x7f19040a20b0 2022-06-29T15:46:03.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.647+0000 7f190b0e0700 1 -- 172.21.15.47:0/3158026588 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1904145e50 con 0x7f19040a20b0 2022-06-29T15:46:03.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.647+0000 7f1901ffb700 1 -- 172.21.15.47:0/3158026588 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18fc01f750 con 0x7f19040a20b0 2022-06-29T15:46:03.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.647+0000 7f190b0e0700 1 -- 172.21.15.47:0/3158026588 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19040635e0 con 0x7f19040a20b0 2022-06-29T15:46:03.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.647+0000 7f190b0e0700 1 -- 172.21.15.47:0/3158026588 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1904147650 con 0x7f19040a20b0 2022-06-29T15:46:03.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.647+0000 7f1901ffb700 1 -- 172.21.15.47:0/3158026588 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f18fc00f070 con 0x7f19040a20b0 2022-06-29T15:46:03.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.651+0000 7f1901ffb700 1 --2- 172.21.15.47:0/3158026588 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f18e4053980 0x7f18e4055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:03.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.651+0000 7f1901ffb700 1 -- 172.21.15.47:0/3158026588 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f18fc06cb80 con 0x7f19040a20b0 2022-06-29T15:46:03.663 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.651+0000 7f1908e7c700 1 --2- 172.21.15.47:0/3158026588 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f18e4053980 0x7f18e4055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:03.664 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.651+0000 7f1908e7c700 1 --2- 172.21.15.47:0/3158026588 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f18e4053980 0x7f18e4055e40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7f18f800d000 tx=0x7f18f8007bc0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:03.664 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.655+0000 7f1901ffb700 1 -- 172.21.15.47:0/3158026588 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f18fc0437b0 con 0x7f19040a20b0 2022-06-29T15:46:03.824 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.819+0000 7f190b0e0700 1 -- 172.21.15.47:0/3158026588 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19040573d0 con 0x7f19040a20b0 2022-06-29T15:46:03.825 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.823+0000 7f1901ffb700 1 -- 172.21.15.47:0/3158026588 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+2492 (secure 0 0 0) 0x7f18fc06ce60 con 0x7f19040a20b0 2022-06-29T15:46:03.825 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:46:03.826 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":24,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5595,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1798995166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1798995166},{"type":"v1","addr":"172.21.15.47:6835","nonce":1798995166}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21},{"gid":5601,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/1641478219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1641478219},{"type":"v1","addr":"172.21.15.64:6833","nonce":1641478219}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:46:01.294125+0000","modified":"2022-06-29T15:46:02.296567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5607":{"gid":5607,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6837/2427298469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2427298469},{"type":"v1","addr":"172.21.15.47:6837","nonce":2427298469}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2022-06-29T15:46:03.829 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.823+0000 7f190b0e0700 1 -- 172.21.15.47:0/3158026588 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f18e4053980 msgr2=0x7f18e4055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:03.830 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.823+0000 7f190b0e0700 1 --2- 172.21.15.47:0/3158026588 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f18e4053980 0x7f18e4055e40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7f18f800d000 tx=0x7f18f8007bc0).stop 2022-06-29T15:46:03.830 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.823+0000 7f190b0e0700 1 -- 172.21.15.47:0/3158026588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19040a20b0 msgr2=0x7f190415a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:03.830 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.823+0000 7f190b0e0700 1 --2- 172.21.15.47:0/3158026588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19040a20b0 0x7f190415a1b0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7f1904141a70 tx=0x7f18fc016000).stop 2022-06-29T15:46:03.830 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.823+0000 7f190b0e0700 1 -- 172.21.15.47:0/3158026588 shutdown_connections 2022-06-29T15:46:03.831 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.823+0000 7f190b0e0700 1 --2- 172.21.15.47:0/3158026588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19040a20b0 0x7f190415a1b0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:03.831 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.823+0000 7f190b0e0700 1 --2- 172.21.15.47:0/3158026588 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f19040b2120 0x7f1904064d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:03.831 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.823+0000 7f190b0e0700 1 --2- 172.21.15.47:0/3158026588 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f18e4053980 0x7f18e4055e40 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:03.832 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.823+0000 7f190b0e0700 1 --2- 172.21.15.47:0/3158026588 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19040b1260 0x7f1904066e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:03.832 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.823+0000 7f190b0e0700 1 -- 172.21.15.47:0/3158026588 >> 172.21.15.47:0/3158026588 conn(0x7f19040626f0 msgr2=0x7f1904062ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:03.832 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.827+0000 7f190b0e0700 1 -- 172.21.15.47:0/3158026588 shutdown_connections 2022-06-29T15:46:03.832 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:03.827+0000 7f190b0e0700 1 -- 172.21.15.47:0/3158026588 wait complete. 2022-06-29T15:46:03.833 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 24 2022-06-29T15:46:04.148 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-29T15:46:01.294125+0000', 'modified': '2022-06-29T15:46:02.296567+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5607}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5607': {'gid': 5607, 'name': 'a', 'rank': 0, 'incarnation': 23, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.47:6837/2427298469', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6836', 'nonce': 2427298469}, {'type': 'v1', 'addr': '172.21.15.47:6837', 'nonce': 2427298469}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-29T15:46:04.149 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-29T15:46:04.149 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-06-29T15:46:04.266 INFO:teuthology.orchestra.run.smithi047.stdout:{"cluster_fsid":"9a68894a-53ce-400e-838b-14eab91b834c","whoami":0,"id":5607,"want_state":"up:active","state":"up:active","rank_uptime":2.9594855739999999,"mdsmap_epoch":24,"osdmap_epoch":58,"osdmap_epoch_barrier":56,"uptime":7.3246000090000001} 2022-06-29T15:46:04.267 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9a68894a-53ce-400e-838b-14eab91b834c", "id": 5607, "mdsmap_epoch": 24, "osdmap_epoch": 58, "osdmap_epoch_barrier": 56, "rank_uptime": 2.959485574, "state": "up:active", "uptime": 7.324600009, "want_state": "up:active", "whoami": 0 } 2022-06-29T15:46:04.268 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:46:04.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.363+0000 7fc06eb70700 1 Processor -- start 2022-06-29T15:46:04.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.363+0000 7fc06eb70700 1 -- start start 2022-06-29T15:46:04.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.363+0000 7fc06eb70700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc0680b03b0 0x7fc068143120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:04.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.363+0000 7fc06eb70700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc0680b1e50 0x7fc068140220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:04.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.363+0000 7fc06eb70700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0680b26b0 con 0x7fc0680ac8e0 2022-06-29T15:46:04.372 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.363+0000 7fc06eb70700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0680b0790 con 0x7fc0680b03b0 2022-06-29T15:46:04.372 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.363+0000 7fc06eb70700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0680acdf0 con 0x7fc0680b1e50 2022-06-29T15:46:04.372 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.363+0000 7fc06db6e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc0680b03b0 0x7fc068143120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:04.372 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.363+0000 7fc06db6e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc0680b03b0 0x7fc068143120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48634/0 (socket says 172.21.15.47:48634) 2022-06-29T15:46:04.372 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.363+0000 7fc06db6e700 1 -- 172.21.15.47:0/3733273155 learned_addr learned my addr 172.21.15.47:0/3733273155 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:46:04.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.363+0000 7fc06d36d700 1 --1- 172.21.15.47:0/3733273155 >> v1:172.21.15.47:6789/0 conn(0x7fc0680ac8e0 0x7fc06813fb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34188/0 (socket says 172.21.15.47:34188) 2022-06-29T15:46:04.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.363+0000 7fc06e36f700 1 --2- 172.21.15.47:0/3733273155 >> v2:172.21.15.64:3300/0 conn(0x7fc0680b1e50 0x7fc068140220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:04.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.367+0000 7fc06db6e700 1 -- 172.21.15.47:0/3733273155 >> v2:172.21.15.64:3300/0 conn(0x7fc0680b1e50 msgr2=0x7fc068140220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:46:04.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.367+0000 7fc06db6e700 1 --2- 172.21.15.47:0/3733273155 >> v2:172.21.15.64:3300/0 conn(0x7fc0680b1e50 0x7fc068140220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:04.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.367+0000 7fc06db6e700 1 -- 172.21.15.47:0/3733273155 >> v1:172.21.15.47:6789/0 conn(0x7fc0680ac8e0 legacy=0x7fc06813fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:04.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.367+0000 7fc06db6e700 1 -- 172.21.15.47:0/3733273155 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0680ad090 con 0x7fc0680b03b0 2022-06-29T15:46:04.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.367+0000 7fc06db6e700 1 --2- 172.21.15.47:0/3733273155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0680b03b0 0x7fc068143120 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7fc058002f70 tx=0x7fc058006ee0).ready entity=mon.0 client_cookie=a54feca3af34c885 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:04.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.367+0000 7fc06cb6c700 1 -- 172.21.15.47:0/3733273155 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc058014070 con 0x7fc0680b03b0 2022-06-29T15:46:04.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.367+0000 7fc06cb6c700 1 -- 172.21.15.47:0/3733273155 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc058000e90 con 0x7fc0680b03b0 2022-06-29T15:46:04.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.367+0000 7fc06cb6c700 1 -- 172.21.15.47:0/3733273155 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc05800a8c0 con 0x7fc0680b03b0 2022-06-29T15:46:04.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.367+0000 7fc06eb70700 1 -- 172.21.15.47:0/3733273155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0680b03b0 msgr2=0x7fc068143120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:04.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.367+0000 7fc06eb70700 1 --2- 172.21.15.47:0/3733273155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0680b03b0 0x7fc068143120 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7fc058002f70 tx=0x7fc058006ee0).stop 2022-06-29T15:46:04.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.367+0000 7fc06eb70700 1 -- 172.21.15.47:0/3733273155 shutdown_connections 2022-06-29T15:46:04.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.367+0000 7fc06eb70700 1 --2- 172.21.15.47:0/3733273155 >> v2:172.21.15.64:3300/0 conn(0x7fc0680b1e50 0x7fc068140220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:04.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.367+0000 7fc06eb70700 1 --2- 172.21.15.47:0/3733273155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0680b03b0 0x7fc068143120 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:04.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.367+0000 7fc06eb70700 1 -- 172.21.15.47:0/3733273155 >> 172.21.15.47:0/3733273155 conn(0x7fc068062560 msgr2=0x7fc068062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:04.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.367+0000 7fc06eb70700 1 -- 172.21.15.47:0/3733273155 shutdown_connections 2022-06-29T15:46:04.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.367+0000 7fc06eb70700 1 -- 172.21.15.47:0/3733273155 wait complete. 2022-06-29T15:46:04.380 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.367+0000 7fc06eb70700 1 Processor -- start 2022-06-29T15:46:04.380 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.367+0000 7fc06eb70700 1 -- start start 2022-06-29T15:46:04.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.371+0000 7fc06eb70700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0680ac8e0 0x7fc0681420f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:04.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.371+0000 7fc06eb70700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc0680b03b0 0x7fc0681456e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:04.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.371+0000 7fc06eb70700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0680b1e50 0x7fc068145bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:04.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.371+0000 7fc06eb70700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc068139320 con 0x7fc0680b1e50 2022-06-29T15:46:04.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.371+0000 7fc06eb70700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0681395c0 con 0x7fc0680ac8e0 2022-06-29T15:46:04.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.371+0000 7fc06eb70700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0681525d0 con 0x7fc0680b03b0 2022-06-29T15:46:04.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.371+0000 7fc06e36f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0680b1e50 0x7fc068145bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:04.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.371+0000 7fc06db6e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0680ac8e0 0x7fc0681420f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:04.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.371+0000 7fc06e36f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0680b1e50 0x7fc068145bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48644/0 (socket says 172.21.15.47:48644) 2022-06-29T15:46:04.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.371+0000 7fc06e36f700 1 -- 172.21.15.47:0/3895931939 learned_addr learned my addr 172.21.15.47:0/3895931939 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:46:04.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.371+0000 7fc06d36d700 1 --2- 172.21.15.47:0/3895931939 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc0680b03b0 0x7fc0681456e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:04.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.371+0000 7fc06e36f700 1 -- 172.21.15.47:0/3895931939 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc0680b03b0 msgr2=0x7fc0681456e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:04.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.371+0000 7fc06e36f700 1 --2- 172.21.15.47:0/3895931939 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc0680b03b0 0x7fc0681456e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:04.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.371+0000 7fc06e36f700 1 -- 172.21.15.47:0/3895931939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0680ac8e0 msgr2=0x7fc0681420f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:04.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.371+0000 7fc06e36f700 1 --2- 172.21.15.47:0/3895931939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0680ac8e0 0x7fc0681420f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:04.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.371+0000 7fc06e36f700 1 -- 172.21.15.47:0/3895931939 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc05800c310 con 0x7fc0680b1e50 2022-06-29T15:46:04.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.371+0000 7fc06e36f700 1 --2- 172.21.15.47:0/3895931939 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0680b1e50 0x7fc068145bd0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7fc06000a000 tx=0x7fc060008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:04.385 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.371+0000 7fc056ffd700 1 -- 172.21.15.47:0/3895931939 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc0600127e0 con 0x7fc0680b1e50 2022-06-29T15:46:04.385 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.371+0000 7fc056ffd700 1 -- 172.21.15.47:0/3895931939 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc060012940 con 0x7fc0680b1e50 2022-06-29T15:46:04.385 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.371+0000 7fc06eb70700 1 -- 172.21.15.47:0/3895931939 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc068152870 con 0x7fc0680b1e50 2022-06-29T15:46:04.385 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.371+0000 7fc056ffd700 1 -- 172.21.15.47:0/3895931939 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc060012ca0 con 0x7fc0680b1e50 2022-06-29T15:46:04.385 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.371+0000 7fc06eb70700 1 -- 172.21.15.47:0/3895931939 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0681425e0 con 0x7fc0680b1e50 2022-06-29T15:46:04.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.375+0000 7fc06eb70700 1 -- 172.21.15.47:0/3895931939 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0680573d0 con 0x7fc0680b1e50 2022-06-29T15:46:04.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.375+0000 7fc056ffd700 1 -- 172.21.15.47:0/3895931939 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc060013480 con 0x7fc0680b1e50 2022-06-29T15:46:04.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.375+0000 7fc056ffd700 1 --2- 172.21.15.47:0/3895931939 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc048053950 0x7fc048055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:04.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.375+0000 7fc056ffd700 1 -- 172.21.15.47:0/3895931939 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc06006c470 con 0x7fc0680b1e50 2022-06-29T15:46:04.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.375+0000 7fc06db6e700 1 --2- 172.21.15.47:0/3895931939 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc048053950 0x7fc048055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:04.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.375+0000 7fc06db6e700 1 --2- 172.21.15.47:0/3895931939 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc048053950 0x7fc048055e10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7fc05801b000 tx=0x7fc0680ad220).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:04.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.383+0000 7fc056ffd700 1 -- 172.21.15.47:0/3895931939 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc0600430c0 con 0x7fc0680b1e50 2022-06-29T15:46:04.513 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.511+0000 7fc06eb70700 1 -- 172.21.15.47:0/3895931939 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc0680b50b0 con 0x7fc0680b1e50 2022-06-29T15:46:04.515 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.511+0000 7fc056ffd700 1 -- 172.21.15.47:0/3895931939 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16273 (secure 0 0 0) 0x7fc060012e00 con 0x7fc0680b1e50 2022-06-29T15:46:04.515 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:46:04.516 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":58,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:46:03.307251+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:45:59.710738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2022-06-29T15:46:00.582445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:46:04.518 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.511+0000 7fc06eb70700 1 -- 172.21.15.47:0/3895931939 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc048053950 msgr2=0x7fc048055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:04.518 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.511+0000 7fc06eb70700 1 --2- 172.21.15.47:0/3895931939 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc048053950 0x7fc048055e10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7fc05801b000 tx=0x7fc0680ad220).stop 2022-06-29T15:46:04.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.511+0000 7fc06eb70700 1 -- 172.21.15.47:0/3895931939 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0680b1e50 msgr2=0x7fc068145bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:04.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.511+0000 7fc06eb70700 1 --2- 172.21.15.47:0/3895931939 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0680b1e50 0x7fc068145bd0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7fc06000a000 tx=0x7fc060008040).stop 2022-06-29T15:46:04.520 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.515+0000 7fc06eb70700 1 -- 172.21.15.47:0/3895931939 shutdown_connections 2022-06-29T15:46:04.520 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.515+0000 7fc06eb70700 1 --2- 172.21.15.47:0/3895931939 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0680b1e50 0x7fc068145bd0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:04.520 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.515+0000 7fc06eb70700 1 --2- 172.21.15.47:0/3895931939 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc0680b03b0 0x7fc0681456e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:04.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.515+0000 7fc06eb70700 1 --2- 172.21.15.47:0/3895931939 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc048053950 0x7fc048055e10 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:04.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.515+0000 7fc06eb70700 1 --2- 172.21.15.47:0/3895931939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0680ac8e0 0x7fc0681420f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:04.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.515+0000 7fc06eb70700 1 -- 172.21.15.47:0/3895931939 >> 172.21.15.47:0/3895931939 conn(0x7fc068062560 msgr2=0x7fc0680601a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:04.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.515+0000 7fc06eb70700 1 -- 172.21.15.47:0/3895931939 shutdown_connections 2022-06-29T15:46:04.522 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.515+0000 7fc06eb70700 1 -- 172.21.15.47:0/3895931939 wait complete. 2022-06-29T15:46:04.528 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-29T15:42:16.971431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-29T15:42:25.290884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-29T15:45:59.710738+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2022-06-29T15:46:00.582445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-29T15:46:04.529 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-29T15:46:04.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.635+0000 7f2a2e9dd700 1 Processor -- start 2022-06-29T15:46:04.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.635+0000 7f2a2e9dd700 1 -- start start 2022-06-29T15:46:04.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.635+0000 7f2a2e9dd700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2a28066dc0 0x7f2a280671a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:04.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.635+0000 7f2a2e9dd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2a281472f0 0x7f2a2813cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:04.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.635+0000 7f2a2e9dd700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a28067690 con 0x7f2a2813cfa0 2022-06-29T15:46:04.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.635+0000 7f2a2e9dd700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a281434f0 con 0x7f2a281472f0 2022-06-29T15:46:04.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.635+0000 7f2a2e9dd700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a28143790 con 0x7f2a28066dc0 2022-06-29T15:46:04.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.635+0000 7f2a277fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2a281472f0 0x7f2a2813cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:04.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.635+0000 7f2a27fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2a28066dc0 0x7f2a280671a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:04.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.639+0000 7f2a27fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2a28066dc0 0x7f2a280671a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:45750/0 (socket says 172.21.15.47:45750) 2022-06-29T15:46:04.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.639+0000 7f2a277fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2a281472f0 0x7f2a2813cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:54354/0 (socket says 172.21.15.47:54354) 2022-06-29T15:46:04.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.639+0000 7f2a2cf7a700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f2a2813cfa0 0x7f2a281445d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:41768/0 (socket says 172.21.15.47:41768) 2022-06-29T15:46:04.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.639+0000 7f2a27fff700 1 -- 172.21.15.47:0/1941660691 learned_addr learned my addr 172.21.15.47:0/1941660691 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:46:04.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.639+0000 7f2a27fff700 1 -- 172.21.15.47:0/1941660691 >> v2:172.21.15.64:3300/0 conn(0x7f2a281472f0 msgr2=0x7f2a2813cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:46:04.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.639+0000 7f2a27fff700 1 --2- 172.21.15.47:0/1941660691 >> v2:172.21.15.64:3300/0 conn(0x7f2a281472f0 0x7f2a2813cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:04.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.639+0000 7f2a27fff700 1 -- 172.21.15.47:0/1941660691 >> v1:172.21.15.64:6789/0 conn(0x7f2a2813cfa0 legacy=0x7f2a281445d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:04.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.639+0000 7f2a27fff700 1 -- 172.21.15.47:0/1941660691 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a28143a30 con 0x7f2a28066dc0 2022-06-29T15:46:04.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.639+0000 7f2a277fe700 1 --2- 172.21.15.47:0/1941660691 >> v2:172.21.15.64:3300/0 conn(0x7f2a281472f0 0x7f2a2813cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:46:04.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.639+0000 7f2a27fff700 1 --2- 172.21.15.47:0/1941660691 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2a28066dc0 0x7f2a280671a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f2a1c002f70 tx=0x7f2a1c006eb0).ready entity=mon.2 client_cookie=53ab60a561fffcc3 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:04.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.639+0000 7f2a26ffd700 1 -- 172.21.15.47:0/1941660691 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2a1c00d570 con 0x7f2a28066dc0 2022-06-29T15:46:04.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.639+0000 7f2a26ffd700 1 -- 172.21.15.47:0/1941660691 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a1c000e90 con 0x7f2a28066dc0 2022-06-29T15:46:04.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.639+0000 7f2a26ffd700 1 -- 172.21.15.47:0/1941660691 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2a1c00d900 con 0x7f2a28066dc0 2022-06-29T15:46:04.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.639+0000 7f2a2e9dd700 1 -- 172.21.15.47:0/1941660691 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2a28066dc0 msgr2=0x7f2a280671a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:04.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.639+0000 7f2a2e9dd700 1 --2- 172.21.15.47:0/1941660691 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2a28066dc0 0x7f2a280671a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f2a1c002f70 tx=0x7f2a1c006eb0).stop 2022-06-29T15:46:04.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.639+0000 7f2a2e9dd700 1 -- 172.21.15.47:0/1941660691 shutdown_connections 2022-06-29T15:46:04.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.639+0000 7f2a2e9dd700 1 --2- 172.21.15.47:0/1941660691 >> v2:172.21.15.64:3300/0 conn(0x7f2a281472f0 0x7f2a2813cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:04.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.639+0000 7f2a2e9dd700 1 --2- 172.21.15.47:0/1941660691 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2a28066dc0 0x7f2a280671a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:04.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.639+0000 7f2a2e9dd700 1 -- 172.21.15.47:0/1941660691 >> 172.21.15.47:0/1941660691 conn(0x7f2a2805c510 msgr2=0x7f2a2805c920 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:04.651 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.639+0000 7f2a2e9dd700 1 -- 172.21.15.47:0/1941660691 shutdown_connections 2022-06-29T15:46:04.651 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.639+0000 7f2a2e9dd700 1 -- 172.21.15.47:0/1941660691 wait complete. 2022-06-29T15:46:04.654 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.643+0000 7f2a2e9dd700 1 Processor -- start 2022-06-29T15:46:04.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.643+0000 7f2a2e9dd700 1 -- start start 2022-06-29T15:46:04.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.643+0000 7f2a2e9dd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a28066dc0 0x7f2a28061c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:04.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.643+0000 7f2a2e9dd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a2813cfa0 0x7f2a28141780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:04.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.643+0000 7f2a2e9dd700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2a281472f0 0x7f2a28152720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:04.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.643+0000 7f2a2e9dd700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a280ad790 con 0x7f2a2813cfa0 2022-06-29T15:46:04.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.643+0000 7f2a2e9dd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a28062370 con 0x7f2a28066dc0 2022-06-29T15:46:04.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.643+0000 7f2a2e9dd700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a281522c0 con 0x7f2a281472f0 2022-06-29T15:46:04.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.643+0000 7f2a277fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a2813cfa0 0x7f2a28141780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:04.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.643+0000 7f2a277fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a2813cfa0 0x7f2a28141780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48656/0 (socket says 172.21.15.47:48656) 2022-06-29T15:46:04.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.643+0000 7f2a277fe700 1 -- 172.21.15.47:0/3757387479 learned_addr learned my addr 172.21.15.47:0/3757387479 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:46:04.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.643+0000 7f2a27fff700 1 --2- 172.21.15.47:0/3757387479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a28066dc0 0x7f2a28061c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:04.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.643+0000 7f2a2cf7a700 1 --2- 172.21.15.47:0/3757387479 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2a281472f0 0x7f2a28152720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:04.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.647+0000 7f2a277fe700 1 -- 172.21.15.47:0/3757387479 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2a281472f0 msgr2=0x7f2a28152720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:04.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.647+0000 7f2a277fe700 1 --2- 172.21.15.47:0/3757387479 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2a281472f0 0x7f2a28152720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:04.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.647+0000 7f2a277fe700 1 -- 172.21.15.47:0/3757387479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a28066dc0 msgr2=0x7f2a28061c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:04.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.647+0000 7f2a277fe700 1 --2- 172.21.15.47:0/3757387479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a28066dc0 0x7f2a28061c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:04.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.647+0000 7f2a277fe700 1 -- 172.21.15.47:0/3757387479 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a1800b180 con 0x7f2a2813cfa0 2022-06-29T15:46:04.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.647+0000 7f2a277fe700 1 --2- 172.21.15.47:0/3757387479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a2813cfa0 0x7f2a28141780 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7f2a1800c000 tx=0x7f2a18006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:04.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.647+0000 7f2a257fa700 1 -- 172.21.15.47:0/3757387479 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2a18010780 con 0x7f2a2813cfa0 2022-06-29T15:46:04.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.647+0000 7f2a257fa700 1 -- 172.21.15.47:0/3757387479 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a180108e0 con 0x7f2a2813cfa0 2022-06-29T15:46:04.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.647+0000 7f2a2e9dd700 1 -- 172.21.15.47:0/3757387479 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a1c00b1e0 con 0x7f2a2813cfa0 2022-06-29T15:46:04.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.647+0000 7f2a257fa700 1 -- 172.21.15.47:0/3757387479 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2a18010c40 con 0x7f2a2813cfa0 2022-06-29T15:46:04.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.647+0000 7f2a2e9dd700 1 -- 172.21.15.47:0/3757387479 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a280ae3c0 con 0x7f2a2813cfa0 2022-06-29T15:46:04.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.647+0000 7f2a257fa700 1 -- 172.21.15.47:0/3757387479 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2a1801c460 con 0x7f2a2813cfa0 2022-06-29T15:46:04.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.647+0000 7f2a2e9dd700 1 -- 172.21.15.47:0/3757387479 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a280b2350 con 0x7f2a2813cfa0 2022-06-29T15:46:04.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.647+0000 7f2a257fa700 1 --2- 172.21.15.47:0/3757387479 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2a08053950 0x7f2a08055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:04.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.647+0000 7f2a257fa700 1 -- 172.21.15.47:0/3757387479 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f2a1806c5f0 con 0x7f2a2813cfa0 2022-06-29T15:46:04.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.647+0000 7f2a27fff700 1 --2- 172.21.15.47:0/3757387479 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2a08053950 0x7f2a08055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:04.663 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.651+0000 7f2a27fff700 1 --2- 172.21.15.47:0/3757387479 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2a08053950 0x7f2a08055e10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f2a1c01c000 tx=0x7f2a1c015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:04.663 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.655+0000 7f2a257fa700 1 -- 172.21.15.47:0/3757387479 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2a280b2350 con 0x7f2a2813cfa0 2022-06-29T15:46:04.785 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.783+0000 7f2a2e9dd700 1 -- 172.21.15.47:0/3757387479 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f2a280b2350 con 0x7f2a2813cfa0 2022-06-29T15:46:04.786 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.783+0000 7f2a257fa700 1 -- 172.21.15.47:0/3757387479 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v58) v1 ==== 110+0+10 (secure 0 0 0) 0x7f2a280b2350 con 0x7f2a2813cfa0 2022-06-29T15:46:04.786 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2022-06-29T15:46:04.789 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.787+0000 7f2a2e9dd700 1 -- 172.21.15.47:0/3757387479 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2a08053950 msgr2=0x7f2a08055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:04.790 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.787+0000 7f2a2e9dd700 1 --2- 172.21.15.47:0/3757387479 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2a08053950 0x7f2a08055e10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f2a1c01c000 tx=0x7f2a1c015000).stop 2022-06-29T15:46:04.790 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.787+0000 7f2a2e9dd700 1 -- 172.21.15.47:0/3757387479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a2813cfa0 msgr2=0x7f2a28141780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:04.790 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.787+0000 7f2a2e9dd700 1 --2- 172.21.15.47:0/3757387479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a2813cfa0 0x7f2a28141780 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7f2a1800c000 tx=0x7f2a18006df0).stop 2022-06-29T15:46:04.791 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.787+0000 7f2a2e9dd700 1 -- 172.21.15.47:0/3757387479 shutdown_connections 2022-06-29T15:46:04.791 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.787+0000 7f2a2e9dd700 1 --2- 172.21.15.47:0/3757387479 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2a281472f0 0x7f2a28152720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:04.791 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.787+0000 7f2a2e9dd700 1 --2- 172.21.15.47:0/3757387479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a2813cfa0 0x7f2a28141780 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:04.791 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.787+0000 7f2a2e9dd700 1 --2- 172.21.15.47:0/3757387479 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2a08053950 0x7f2a08055e10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:04.792 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.787+0000 7f2a2e9dd700 1 --2- 172.21.15.47:0/3757387479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a28066dc0 0x7f2a28061c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:04.792 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.787+0000 7f2a2e9dd700 1 -- 172.21.15.47:0/3757387479 >> 172.21.15.47:0/3757387479 conn(0x7f2a2805c510 msgr2=0x7f2a2813c880 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:04.792 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.787+0000 7f2a2e9dd700 1 -- 172.21.15.47:0/3757387479 shutdown_connections 2022-06-29T15:46:04.792 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.787+0000 7f2a2e9dd700 1 -- 172.21.15.47:0/3757387479 wait complete. 2022-06-29T15:46:04.805 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-29T15:46:04.915 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.911+0000 7f4bc687d700 1 Processor -- start 2022-06-29T15:46:04.915 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.911+0000 7f4bc687d700 1 -- start start 2022-06-29T15:46:04.918 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.911+0000 7f4bc687d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4bc005f3f0 0x7f4bc01418c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:04.918 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.911+0000 7f4bc687d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4bc00b1320 0x7f4bc0141db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:04.918 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.911+0000 7f4bc687d700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bc01422a0 con 0x7f4bc00aaf00 2022-06-29T15:46:04.919 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.911+0000 7f4bc687d700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bc00b27a0 con 0x7f4bc005f3f0 2022-06-29T15:46:04.919 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.911+0000 7f4bc687d700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc005f7d0 con 0x7f4bc00b1320 2022-06-29T15:46:04.919 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.911+0000 7f4bc587b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4bc005f3f0 0x7f4bc01418c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:04.919 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.915+0000 7f4bc587b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4bc005f3f0 0x7f4bc01418c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:54366/0 (socket says 172.21.15.47:54366) 2022-06-29T15:46:04.920 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.915+0000 7f4bc607c700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f4bc00aaf00 0x7f4bc00b2ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:41784/0 (socket says 172.21.15.47:41784) 2022-06-29T15:46:04.920 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.915+0000 7f4bc587b700 1 -- 172.21.15.47:0/3187805587 learned_addr learned my addr 172.21.15.47:0/3187805587 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:46:04.920 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.915+0000 7f4bc507a700 1 --2- 172.21.15.47:0/3187805587 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4bc00b1320 0x7f4bc0141db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:04.920 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.915+0000 7f4bc587b700 1 -- 172.21.15.47:0/3187805587 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4bc00b1320 msgr2=0x7f4bc0141db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:46:04.921 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.915+0000 7f4bc587b700 1 --2- 172.21.15.47:0/3187805587 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4bc00b1320 0x7f4bc0141db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:04.921 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.915+0000 7f4bc587b700 1 -- 172.21.15.47:0/3187805587 >> v1:172.21.15.64:6789/0 conn(0x7f4bc00aaf00 legacy=0x7f4bc00b2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:04.921 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.915+0000 7f4bc587b700 1 -- 172.21.15.47:0/3187805587 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc00b4b90 con 0x7f4bc005f3f0 2022-06-29T15:46:04.921 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.915+0000 7f4bc4879700 1 -- 172.21.15.47:0/3187805587 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4099254711 0 0) 0x7f4bc01422a0 con 0x7f4bc00aaf00 2022-06-29T15:46:04.922 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.915+0000 7f4bc587b700 1 --2- 172.21.15.47:0/3187805587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bc005f3f0 0x7f4bc01418c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f4bbc0032f0 tx=0x7f4bbc009290).ready entity=mon.1 client_cookie=5791c83d699cbab3 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:04.922 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.915+0000 7f4bc4879700 1 -- 172.21.15.47:0/3187805587 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4bbc00b8a0 con 0x7f4bc005f3f0 2022-06-29T15:46:04.922 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.915+0000 7f4bc4879700 1 -- 172.21.15.47:0/3187805587 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bbc00ba00 con 0x7f4bc005f3f0 2022-06-29T15:46:04.923 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.915+0000 7f4bc4879700 1 -- 172.21.15.47:0/3187805587 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4bbc00bb70 con 0x7f4bc005f3f0 2022-06-29T15:46:04.927 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.915+0000 7f4bc687d700 1 -- 172.21.15.47:0/3187805587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bc005f3f0 msgr2=0x7f4bc01418c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:04.927 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.915+0000 7f4bc687d700 1 --2- 172.21.15.47:0/3187805587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bc005f3f0 0x7f4bc01418c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f4bbc0032f0 tx=0x7f4bbc009290).stop 2022-06-29T15:46:04.927 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.915+0000 7f4bc687d700 1 -- 172.21.15.47:0/3187805587 shutdown_connections 2022-06-29T15:46:04.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.915+0000 7f4bc687d700 1 --2- 172.21.15.47:0/3187805587 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4bc00b1320 0x7f4bc0141db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:04.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.915+0000 7f4bc687d700 1 --2- 172.21.15.47:0/3187805587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bc005f3f0 0x7f4bc01418c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:04.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.915+0000 7f4bc687d700 1 -- 172.21.15.47:0/3187805587 >> 172.21.15.47:0/3187805587 conn(0x7f4bc0062570 msgr2=0x7f4bc00647d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:04.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.915+0000 7f4bc687d700 1 -- 172.21.15.47:0/3187805587 shutdown_connections 2022-06-29T15:46:04.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.915+0000 7f4bc687d700 1 -- 172.21.15.47:0/3187805587 wait complete. 2022-06-29T15:46:04.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.919+0000 7f4bc687d700 1 Processor -- start 2022-06-29T15:46:04.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.919+0000 7f4bc687d700 1 -- start start 2022-06-29T15:46:04.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.919+0000 7f4bc687d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bc005f3f0 0x7f4bc00b6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:04.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.919+0000 7f4bc687d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4bc00aaf00 0x7f4bc00abec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:04.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.919+0000 7f4bc687d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bc00b1320 0x7f4bc00ac3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:04.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.919+0000 7f4bc687d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc00ac8a0 con 0x7f4bc005f3f0 2022-06-29T15:46:04.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.919+0000 7f4bc687d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc01512c0 con 0x7f4bc00b1320 2022-06-29T15:46:04.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.919+0000 7f4bc687d700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc00ab4e0 con 0x7f4bc00aaf00 2022-06-29T15:46:04.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.919+0000 7f4bc587b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bc005f3f0 0x7f4bc00b6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:04.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.919+0000 7f4bc587b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bc005f3f0 0x7f4bc00b6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48668/0 (socket says 172.21.15.47:48668) 2022-06-29T15:46:04.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.919+0000 7f4bc587b700 1 -- 172.21.15.47:0/3420937499 learned_addr learned my addr 172.21.15.47:0/3420937499 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:46:04.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.919+0000 7f4bc607c700 1 --2- 172.21.15.47:0/3420937499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bc00b1320 0x7f4bc00ac3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:04.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.919+0000 7f4bc507a700 1 --2- 172.21.15.47:0/3420937499 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4bc00aaf00 0x7f4bc00abec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:04.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.919+0000 7f4bc587b700 1 -- 172.21.15.47:0/3420937499 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4bc00aaf00 msgr2=0x7f4bc00abec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:04.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.919+0000 7f4bc587b700 1 --2- 172.21.15.47:0/3420937499 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4bc00aaf00 0x7f4bc00abec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:04.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.919+0000 7f4bc587b700 1 -- 172.21.15.47:0/3420937499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bc00b1320 msgr2=0x7f4bc00ac3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:04.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.919+0000 7f4bc587b700 1 --2- 172.21.15.47:0/3420937499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bc00b1320 0x7f4bc00ac3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:04.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.919+0000 7f4bc587b700 1 -- 172.21.15.47:0/3420937499 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bbc009530 con 0x7f4bc005f3f0 2022-06-29T15:46:04.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.919+0000 7f4bc587b700 1 --2- 172.21.15.47:0/3420937499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bc005f3f0 0x7f4bc00b6690 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7f4bbc0032f0 tx=0x7f4bbc016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:04.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.919+0000 7f4baeffd700 1 -- 172.21.15.47:0/3420937499 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4bbc00b8a0 con 0x7f4bc005f3f0 2022-06-29T15:46:04.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.919+0000 7f4baeffd700 1 -- 172.21.15.47:0/3420937499 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bbc021630 con 0x7f4bc005f3f0 2022-06-29T15:46:04.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.919+0000 7f4bc687d700 1 -- 172.21.15.47:0/3420937499 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bc00ab780 con 0x7f4bc005f3f0 2022-06-29T15:46:04.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.919+0000 7f4baeffd700 1 -- 172.21.15.47:0/3420937499 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4bbc021990 con 0x7f4bc005f3f0 2022-06-29T15:46:04.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.919+0000 7f4bc687d700 1 -- 172.21.15.47:0/3420937499 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bc00acd00 con 0x7f4bc005f3f0 2022-06-29T15:46:04.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.923+0000 7f4baeffd700 1 -- 172.21.15.47:0/3420937499 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4bbc027020 con 0x7f4bc005f3f0 2022-06-29T15:46:04.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.923+0000 7f4bc687d700 1 -- 172.21.15.47:0/3420937499 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bc00573d0 con 0x7f4bc005f3f0 2022-06-29T15:46:04.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.923+0000 7f4baeffd700 1 --2- 172.21.15.47:0/3420937499 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4ba0053980 0x7f4ba0055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:04.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.923+0000 7f4baeffd700 1 -- 172.21.15.47:0/3420937499 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f4bbc06d530 con 0x7f4bc005f3f0 2022-06-29T15:46:04.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.923+0000 7f4bc507a700 1 --2- 172.21.15.47:0/3420937499 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4ba0053980 0x7f4ba0055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:04.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.923+0000 7f4bc507a700 1 --2- 172.21.15.47:0/3420937499 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4ba0053980 0x7f4ba0055e40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f4bb000b000 tx=0x7f4bb0008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:04.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:04.931+0000 7f4baeffd700 1 -- 172.21.15.47:0/3420937499 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4bbc0441e0 con 0x7f4bc005f3f0 2022-06-29T15:46:05.062 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.059+0000 7f4bc687d700 1 -- 172.21.15.47:0/3420937499 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4bc00b3aa0 con 0x7f4bc005f3f0 2022-06-29T15:46:05.063 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.059+0000 7f4baeffd700 1 -- 172.21.15.47:0/3420937499 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v58) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4bbc015020 con 0x7f4bc005f3f0 2022-06-29T15:46:05.064 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:46:05.066 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.063+0000 7f4bc687d700 1 -- 172.21.15.47:0/3420937499 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4ba0053980 msgr2=0x7f4ba0055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:05.066 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.063+0000 7f4bc687d700 1 --2- 172.21.15.47:0/3420937499 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4ba0053980 0x7f4ba0055e40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f4bb000b000 tx=0x7f4bb0008040).stop 2022-06-29T15:46:05.067 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.063+0000 7f4bc687d700 1 -- 172.21.15.47:0/3420937499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bc005f3f0 msgr2=0x7f4bc00b6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:05.067 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.063+0000 7f4bc687d700 1 --2- 172.21.15.47:0/3420937499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bc005f3f0 0x7f4bc00b6690 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7f4bbc0032f0 tx=0x7f4bbc016000).stop 2022-06-29T15:46:05.067 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.063+0000 7f4bc687d700 1 -- 172.21.15.47:0/3420937499 shutdown_connections 2022-06-29T15:46:05.067 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.063+0000 7f4bc687d700 1 --2- 172.21.15.47:0/3420937499 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4ba0053980 0x7f4ba0055e40 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:05.068 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.063+0000 7f4bc687d700 1 --2- 172.21.15.47:0/3420937499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bc00b1320 0x7f4bc00ac3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:05.068 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.063+0000 7f4bc687d700 1 --2- 172.21.15.47:0/3420937499 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4bc00aaf00 0x7f4bc00abec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:05.068 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.063+0000 7f4bc687d700 1 --2- 172.21.15.47:0/3420937499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bc005f3f0 0x7f4bc00b6690 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:05.068 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.063+0000 7f4bc687d700 1 -- 172.21.15.47:0/3420937499 >> 172.21.15.47:0/3420937499 conn(0x7f4bc0062570 msgr2=0x7f4bc013e890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:05.069 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.063+0000 7f4bc687d700 1 -- 172.21.15.47:0/3420937499 shutdown_connections 2022-06-29T15:46:05.069 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.063+0000 7f4bc687d700 1 -- 172.21.15.47:0/3420937499 wait complete. 2022-06-29T15:46:05.077 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-29T15:46:05.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.187+0000 7f6cfd6e9700 1 Processor -- start 2022-06-29T15:46:05.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.187+0000 7f6cfd6e9700 1 -- start start 2022-06-29T15:46:05.192 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.187+0000 7f6cfd6e9700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6cf805f0b0 0x7f6cf805f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:05.192 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.187+0000 7f6cfd6e9700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cf81485a0 con 0x7f6cf8061bd0 2022-06-29T15:46:05.193 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.187+0000 7f6cfd6e9700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cf8060190 con 0x7f6cf80b4bb0 2022-06-29T15:46:05.193 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.187+0000 7f6cfd6e9700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cf8148060 con 0x7f6cf805f0b0 2022-06-29T15:46:05.193 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.187+0000 7f6cf6ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6cf805f0b0 0x7f6cf805f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:05.193 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.187+0000 7f6cf77fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f6cf8061bd0 0x7f6cf80b7620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34234/0 (socket says 172.21.15.47:34234) 2022-06-29T15:46:05.193 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.187+0000 7f6cf77fe700 1 -- 172.21.15.47:0/3191608939 learned_addr learned my addr 172.21.15.47:0/3191608939 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:46:05.194 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.187+0000 7f6cf5ffb700 1 -- 172.21.15.47:0/3191608939 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1929094103 0 0) 0x7f6cf81485a0 con 0x7f6cf8061bd0 2022-06-29T15:46:05.194 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.187+0000 7f6cf5ffb700 1 -- 172.21.15.47:0/3191608939 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cd8003260 con 0x7f6cf8061bd0 2022-06-29T15:46:05.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.187+0000 7f6cf6ffd700 1 -- 172.21.15.47:0/3191608939 >> v1:172.21.15.64:6789/0 conn(0x7f6cf80b4bb0 legacy=0x7f6cf80aab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:05.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.187+0000 7f6cf6ffd700 1 -- 172.21.15.47:0/3191608939 >> v1:172.21.15.47:6789/0 conn(0x7f6cf8061bd0 legacy=0x7f6cf80b7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:05.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.187+0000 7f6cf6ffd700 1 -- 172.21.15.47:0/3191608939 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf8148300 con 0x7f6cf805f0b0 2022-06-29T15:46:05.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.187+0000 7f6cf6ffd700 1 --2- 172.21.15.47:0/3191608939 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cf805f0b0 0x7f6cf805f490 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7f6cf80612d0 tx=0x7f6ce8006df0).ready entity=mon.0 client_cookie=c82a45c5fdefdb0b server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:05.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.187+0000 7f6cf5ffb700 1 -- 172.21.15.47:0/3191608939 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ce800d610 con 0x7f6cf805f0b0 2022-06-29T15:46:05.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.187+0000 7f6cf5ffb700 1 -- 172.21.15.47:0/3191608939 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce8000e90 con 0x7f6cf805f0b0 2022-06-29T15:46:05.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.187+0000 7f6cf5ffb700 1 -- 172.21.15.47:0/3191608939 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ce800d960 con 0x7f6cf805f0b0 2022-06-29T15:46:05.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.191+0000 7f6cfd6e9700 1 -- 172.21.15.47:0/3191608939 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cf805f0b0 msgr2=0x7f6cf805f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:05.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.191+0000 7f6cfd6e9700 1 --2- 172.21.15.47:0/3191608939 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cf805f0b0 0x7f6cf805f490 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7f6cf80612d0 tx=0x7f6ce8006df0).stop 2022-06-29T15:46:05.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.191+0000 7f6cfd6e9700 1 -- 172.21.15.47:0/3191608939 shutdown_connections 2022-06-29T15:46:05.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.191+0000 7f6cfd6e9700 1 --2- 172.21.15.47:0/3191608939 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cf805f0b0 0x7f6cf805f490 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:05.202 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.191+0000 7f6cfd6e9700 1 -- 172.21.15.47:0/3191608939 >> 172.21.15.47:0/3191608939 conn(0x7f6cf80627d0 msgr2=0x7f6cf8062bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:05.202 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.191+0000 7f6cfd6e9700 1 -- 172.21.15.47:0/3191608939 shutdown_connections 2022-06-29T15:46:05.202 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.191+0000 7f6cfd6e9700 1 -- 172.21.15.47:0/3191608939 wait complete. 2022-06-29T15:46:05.202 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.191+0000 7f6cfd6e9700 1 Processor -- start 2022-06-29T15:46:05.203 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.191+0000 7f6cfd6e9700 1 -- start start 2022-06-29T15:46:05.203 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.191+0000 7f6cfd6e9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cf805f0b0 0x7f6cf80b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:05.203 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.191+0000 7f6cfd6e9700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6cf8061bd0 0x7f6cf814d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:05.203 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.191+0000 7f6cfd6e9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cf80b4bb0 0x7f6cf80af5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:05.204 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.191+0000 7f6cfd6e9700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf80ad910 con 0x7f6cf80b4bb0 2022-06-29T15:46:05.204 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.191+0000 7f6cfd6e9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf80adbb0 con 0x7f6cf805f0b0 2022-06-29T15:46:05.204 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.191+0000 7f6cfd6e9700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf80acb60 con 0x7f6cf8061bd0 2022-06-29T15:46:05.204 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.191+0000 7f6cf77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cf80b4bb0 0x7f6cf80af5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:05.204 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.191+0000 7f6cf6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cf805f0b0 0x7f6cf80b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:05.205 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.191+0000 7f6cf77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cf80b4bb0 0x7f6cf80af5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48686/0 (socket says 172.21.15.47:48686) 2022-06-29T15:46:05.205 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.191+0000 7f6cf67fc700 1 --2- 172.21.15.47:0/873654724 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6cf8061bd0 0x7f6cf814d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:05.206 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.191+0000 7f6cf77fe700 1 -- 172.21.15.47:0/873654724 learned_addr learned my addr 172.21.15.47:0/873654724 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:46:05.206 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.191+0000 7f6cf77fe700 1 -- 172.21.15.47:0/873654724 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6cf8061bd0 msgr2=0x7f6cf814d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:05.206 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.191+0000 7f6cf77fe700 1 --2- 172.21.15.47:0/873654724 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6cf8061bd0 0x7f6cf814d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:05.206 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.191+0000 7f6cf77fe700 1 -- 172.21.15.47:0/873654724 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cf805f0b0 msgr2=0x7f6cf80b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:05.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.191+0000 7f6cf77fe700 1 --2- 172.21.15.47:0/873654724 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cf805f0b0 0x7f6cf80b0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:05.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.191+0000 7f6cf77fe700 1 -- 172.21.15.47:0/873654724 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce800b180 con 0x7f6cf80b4bb0 2022-06-29T15:46:05.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.195+0000 7f6cf77fe700 1 --2- 172.21.15.47:0/873654724 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cf80b4bb0 0x7f6cf80af5b0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7f6cf0001490 tx=0x7f6cf0007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:05.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.195+0000 7f6ceffff700 1 -- 172.21.15.47:0/873654724 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6cf0005ba0 con 0x7f6cf80b4bb0 2022-06-29T15:46:05.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.195+0000 7f6ceffff700 1 -- 172.21.15.47:0/873654724 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cf0005d00 con 0x7f6cf80b4bb0 2022-06-29T15:46:05.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.195+0000 7f6cfd6e9700 1 -- 172.21.15.47:0/873654724 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cf8067660 con 0x7f6cf80b4bb0 2022-06-29T15:46:05.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.195+0000 7f6ceffff700 1 -- 172.21.15.47:0/873654724 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6cf00072c0 con 0x7f6cf80b4bb0 2022-06-29T15:46:05.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.195+0000 7f6cfd6e9700 1 -- 172.21.15.47:0/873654724 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cf80aead0 con 0x7f6cf80b4bb0 2022-06-29T15:46:05.209 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.195+0000 7f6cfd6e9700 1 -- 172.21.15.47:0/873654724 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cf80573d0 con 0x7f6cf80b4bb0 2022-06-29T15:46:05.209 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.195+0000 7f6ceffff700 1 -- 172.21.15.47:0/873654724 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6cf0010070 con 0x7f6cf80b4bb0 2022-06-29T15:46:05.209 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.195+0000 7f6ceffff700 1 --2- 172.21.15.47:0/873654724 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6cdc053950 0x7f6cdc055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:05.209 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.195+0000 7f6ceffff700 1 -- 172.21.15.47:0/873654724 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f6cf006b9b0 con 0x7f6cf80b4bb0 2022-06-29T15:46:05.209 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.195+0000 7f6cf6ffd700 1 --2- 172.21.15.47:0/873654724 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6cdc053950 0x7f6cdc055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:05.210 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.195+0000 7f6cf6ffd700 1 --2- 172.21.15.47:0/873654724 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6cdc053950 0x7f6cdc055e10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f6ce801a000 tx=0x7f6ce8014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:05.210 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.199+0000 7f6ceffff700 1 -- 172.21.15.47:0/873654724 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6cf0042660 con 0x7f6cf80b4bb0 2022-06-29T15:46:05.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.327+0000 7f6cfd6e9700 1 -- 172.21.15.47:0/873654724 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6cf80acd30 con 0x7f6cf80b4bb0 2022-06-29T15:46:05.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.331+0000 7f6ceffff700 1 -- 172.21.15.47:0/873654724 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v58) v1 ==== 104+0+10 (secure 0 0 0) 0x7f6cf0018020 con 0x7f6cf80b4bb0 2022-06-29T15:46:05.334 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:46:05.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.331+0000 7f6cfd6e9700 1 -- 172.21.15.47:0/873654724 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6cdc053950 msgr2=0x7f6cdc055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:05.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.331+0000 7f6cfd6e9700 1 --2- 172.21.15.47:0/873654724 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6cdc053950 0x7f6cdc055e10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f6ce801a000 tx=0x7f6ce8014000).stop 2022-06-29T15:46:05.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.331+0000 7f6cfd6e9700 1 -- 172.21.15.47:0/873654724 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cf80b4bb0 msgr2=0x7f6cf80af5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:05.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.331+0000 7f6cfd6e9700 1 --2- 172.21.15.47:0/873654724 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cf80b4bb0 0x7f6cf80af5b0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7f6cf0001490 tx=0x7f6cf0007e00).stop 2022-06-29T15:46:05.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.331+0000 7f6cfd6e9700 1 -- 172.21.15.47:0/873654724 shutdown_connections 2022-06-29T15:46:05.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.331+0000 7f6cfd6e9700 1 --2- 172.21.15.47:0/873654724 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cf80b4bb0 0x7f6cf80af5b0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:05.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.331+0000 7f6cfd6e9700 1 --2- 172.21.15.47:0/873654724 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6cf8061bd0 0x7f6cf814d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:05.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.331+0000 7f6cfd6e9700 1 --2- 172.21.15.47:0/873654724 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6cdc053950 0x7f6cdc055e10 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:05.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.331+0000 7f6cfd6e9700 1 --2- 172.21.15.47:0/873654724 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cf805f0b0 0x7f6cf80b0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:05.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.331+0000 7f6cfd6e9700 1 -- 172.21.15.47:0/873654724 >> 172.21.15.47:0/873654724 conn(0x7f6cf80627d0 msgr2=0x7f6cf8144300 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:05.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.331+0000 7f6cfd6e9700 1 -- 172.21.15.47:0/873654724 shutdown_connections 2022-06-29T15:46:05.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.331+0000 7f6cfd6e9700 1 -- 172.21.15.47:0/873654724 wait complete. 2022-06-29T15:46:05.347 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-29T15:46:05.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.451+0000 7eff36f57700 1 Processor -- start 2022-06-29T15:46:05.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.451+0000 7eff36f57700 1 -- start start 2022-06-29T15:46:05.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.451+0000 7eff36f57700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7eff300b58d0 0x7eff300b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:05.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.451+0000 7eff36f57700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7eff300b1610 0x7eff301475f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:05.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.451+0000 7eff36f57700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff30060790 con 0x7eff3014b4c0 2022-06-29T15:46:05.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.451+0000 7eff36f57700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff300a99e0 con 0x7eff300b58d0 2022-06-29T15:46:05.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.451+0000 7eff36f57700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff3014ad80 con 0x7eff300b1610 2022-06-29T15:46:05.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.451+0000 7eff35f55700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7eff300b58d0 0x7eff300b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:05.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.451+0000 7eff36756700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7eff3014b4c0 0x7eff300aa130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34248/0 (socket says 172.21.15.47:34248) 2022-06-29T15:46:05.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.451+0000 7eff36756700 1 -- 172.21.15.47:0/3054247829 learned_addr learned my addr 172.21.15.47:0/3054247829 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:46:05.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.451+0000 7eff35754700 1 --2- 172.21.15.47:0/3054247829 >> v2:172.21.15.64:3300/0 conn(0x7eff300b1610 0x7eff301475f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:05.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.451+0000 7eff35f55700 1 -- 172.21.15.47:0/3054247829 >> v2:172.21.15.64:3300/0 conn(0x7eff300b1610 msgr2=0x7eff301475f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:46:05.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.451+0000 7eff35f55700 1 --2- 172.21.15.47:0/3054247829 >> v2:172.21.15.64:3300/0 conn(0x7eff300b1610 0x7eff301475f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:05.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.451+0000 7eff35f55700 1 -- 172.21.15.47:0/3054247829 >> v1:172.21.15.47:6789/0 conn(0x7eff3014b4c0 legacy=0x7eff300aa130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:05.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.451+0000 7eff34f53700 1 -- 172.21.15.47:0/3054247829 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 770131825 0 0) 0x7eff30060790 con 0x7eff3014b4c0 2022-06-29T15:46:05.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.451+0000 7eff35f55700 1 -- 172.21.15.47:0/3054247829 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff3014b020 con 0x7eff300b58d0 2022-06-29T15:46:05.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.455+0000 7eff35f55700 1 --2- 172.21.15.47:0/3054247829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff300b58d0 0x7eff300b5cb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7eff20002f70 tx=0x7eff20006df0).ready entity=mon.0 client_cookie=e65962efd37ae625 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:05.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.455+0000 7eff34f53700 1 -- 172.21.15.47:0/3054247829 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff2000d570 con 0x7eff300b58d0 2022-06-29T15:46:05.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.455+0000 7eff34f53700 1 -- 172.21.15.47:0/3054247829 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff20000e90 con 0x7eff300b58d0 2022-06-29T15:46:05.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.455+0000 7eff34f53700 1 -- 172.21.15.47:0/3054247829 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff2000d8c0 con 0x7eff300b58d0 2022-06-29T15:46:05.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.455+0000 7eff36f57700 1 -- 172.21.15.47:0/3054247829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff300b58d0 msgr2=0x7eff300b5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:05.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.455+0000 7eff36f57700 1 --2- 172.21.15.47:0/3054247829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff300b58d0 0x7eff300b5cb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7eff20002f70 tx=0x7eff20006df0).stop 2022-06-29T15:46:05.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.455+0000 7eff36f57700 1 -- 172.21.15.47:0/3054247829 shutdown_connections 2022-06-29T15:46:05.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.455+0000 7eff36f57700 1 --2- 172.21.15.47:0/3054247829 >> v2:172.21.15.64:3300/0 conn(0x7eff300b1610 0x7eff301475f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:05.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.455+0000 7eff36f57700 1 --2- 172.21.15.47:0/3054247829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff300b58d0 0x7eff300b5cb0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:05.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.455+0000 7eff36f57700 1 -- 172.21.15.47:0/3054247829 >> 172.21.15.47:0/3054247829 conn(0x7eff30062560 msgr2=0x7eff30062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:05.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.455+0000 7eff36f57700 1 -- 172.21.15.47:0/3054247829 shutdown_connections 2022-06-29T15:46:05.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.455+0000 7eff36f57700 1 -- 172.21.15.47:0/3054247829 wait complete. 2022-06-29T15:46:05.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.455+0000 7eff36f57700 1 Processor -- start 2022-06-29T15:46:05.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.455+0000 7eff36f57700 1 -- start start 2022-06-29T15:46:05.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.455+0000 7eff36f57700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7eff300b1610 0x7eff30066a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:05.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.455+0000 7eff36f57700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff300b58d0 0x7eff30141ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:05.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.455+0000 7eff36f57700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff3014b4c0 0x7eff30143b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:05.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.455+0000 7eff36f57700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff30066110 con 0x7eff3014b4c0 2022-06-29T15:46:05.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.455+0000 7eff36f57700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff30066f00 con 0x7eff300b58d0 2022-06-29T15:46:05.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.455+0000 7eff36f57700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff30065730 con 0x7eff300b1610 2022-06-29T15:46:05.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.455+0000 7eff36756700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff3014b4c0 0x7eff30143b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:05.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.455+0000 7eff36756700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff3014b4c0 0x7eff30143b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48700/0 (socket says 172.21.15.47:48700) 2022-06-29T15:46:05.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.455+0000 7eff36756700 1 -- 172.21.15.47:0/1309793522 learned_addr learned my addr 172.21.15.47:0/1309793522 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:46:05.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.455+0000 7eff35754700 1 --2- 172.21.15.47:0/1309793522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff300b58d0 0x7eff30141ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:05.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.459+0000 7eff35f55700 1 --2- 172.21.15.47:0/1309793522 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7eff300b1610 0x7eff30066a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:05.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.459+0000 7eff36756700 1 -- 172.21.15.47:0/1309793522 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7eff300b1610 msgr2=0x7eff30066a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:05.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.459+0000 7eff36756700 1 --2- 172.21.15.47:0/1309793522 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7eff300b1610 0x7eff30066a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:05.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.459+0000 7eff36756700 1 -- 172.21.15.47:0/1309793522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff300b58d0 msgr2=0x7eff30141ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:05.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.459+0000 7eff36756700 1 --2- 172.21.15.47:0/1309793522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff300b58d0 0x7eff30141ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:05.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.459+0000 7eff36756700 1 -- 172.21.15.47:0/1309793522 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff2000b180 con 0x7eff3014b4c0 2022-06-29T15:46:05.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.459+0000 7eff35754700 1 --2- 172.21.15.47:0/1309793522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff300b58d0 0x7eff30141ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:46:05.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.459+0000 7eff36756700 1 --2- 172.21.15.47:0/1309793522 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff3014b4c0 0x7eff30143b30 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7eff28000bc0 tx=0x7eff28007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:05.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.459+0000 7eff1effd700 1 -- 172.21.15.47:0/1309793522 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff28001d30 con 0x7eff3014b4c0 2022-06-29T15:46:05.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.459+0000 7eff1effd700 1 -- 172.21.15.47:0/1309793522 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff280072e0 con 0x7eff3014b4c0 2022-06-29T15:46:05.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.459+0000 7eff36f57700 1 -- 172.21.15.47:0/1309793522 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff300659d0 con 0x7eff3014b4c0 2022-06-29T15:46:05.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.459+0000 7eff1effd700 1 -- 172.21.15.47:0/1309793522 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff28005800 con 0x7eff3014b4c0 2022-06-29T15:46:05.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.459+0000 7eff36f57700 1 -- 172.21.15.47:0/1309793522 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff30142390 con 0x7eff3014b4c0 2022-06-29T15:46:05.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.459+0000 7eff36f57700 1 -- 172.21.15.47:0/1309793522 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff300573d0 con 0x7eff3014b4c0 2022-06-29T15:46:05.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.459+0000 7eff1effd700 1 -- 172.21.15.47:0/1309793522 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7eff28010070 con 0x7eff3014b4c0 2022-06-29T15:46:05.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.459+0000 7eff1effd700 1 --2- 172.21.15.47:0/1309793522 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7eff10053950 0x7eff10055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:05.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.459+0000 7eff1effd700 1 -- 172.21.15.47:0/1309793522 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6420+0+0 (secure 0 0 0) 0x7eff2806b9b0 con 0x7eff3014b4c0 2022-06-29T15:46:05.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.459+0000 7eff35f55700 1 --2- 172.21.15.47:0/1309793522 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7eff10053950 0x7eff10055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:05.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.463+0000 7eff35f55700 1 --2- 172.21.15.47:0/1309793522 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7eff10053950 0x7eff10055e10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7eff20018000 tx=0x7eff20014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:05.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.467+0000 7eff1effd700 1 -- 172.21.15.47:0/1309793522 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7eff280425e0 con 0x7eff3014b4c0 2022-06-29T15:46:05.594 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.591+0000 7eff36f57700 1 -- 172.21.15.47:0/1309793522 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7eff300667f0 con 0x7eff3014b4c0 2022-06-29T15:46:05.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.591+0000 7eff1effd700 1 -- 172.21.15.47:0/1309793522 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v58) v1 ==== 100+0+10 (secure 0 0 0) 0x7eff28045c00 con 0x7eff3014b4c0 2022-06-29T15:46:05.595 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:46:05.597 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.591+0000 7eff36f57700 1 -- 172.21.15.47:0/1309793522 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7eff10053950 msgr2=0x7eff10055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:05.597 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.591+0000 7eff36f57700 1 --2- 172.21.15.47:0/1309793522 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7eff10053950 0x7eff10055e10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7eff20018000 tx=0x7eff20014000).stop 2022-06-29T15:46:05.597 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.591+0000 7eff36f57700 1 -- 172.21.15.47:0/1309793522 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff3014b4c0 msgr2=0x7eff30143b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:05.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.591+0000 7eff36f57700 1 --2- 172.21.15.47:0/1309793522 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff3014b4c0 0x7eff30143b30 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7eff28000bc0 tx=0x7eff28007c20).stop 2022-06-29T15:46:05.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.591+0000 7eff36f57700 1 -- 172.21.15.47:0/1309793522 shutdown_connections 2022-06-29T15:46:05.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.591+0000 7eff36f57700 1 --2- 172.21.15.47:0/1309793522 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff3014b4c0 0x7eff30143b30 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:05.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.591+0000 7eff36f57700 1 --2- 172.21.15.47:0/1309793522 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7eff10053950 0x7eff10055e10 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:05.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.591+0000 7eff36f57700 1 --2- 172.21.15.47:0/1309793522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff300b58d0 0x7eff30141ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:05.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.591+0000 7eff36f57700 1 --2- 172.21.15.47:0/1309793522 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7eff300b1610 0x7eff30066a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:05.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.591+0000 7eff36f57700 1 -- 172.21.15.47:0/1309793522 >> 172.21.15.47:0/1309793522 conn(0x7eff30062560 msgr2=0x7eff30149380 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:05.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.595+0000 7eff36f57700 1 -- 172.21.15.47:0/1309793522 shutdown_connections 2022-06-29T15:46:05.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.595+0000 7eff36f57700 1 -- 172.21.15.47:0/1309793522 wait complete. 2022-06-29T15:46:05.608 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-29T15:46:05.609 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:46:05.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.707+0000 7fd27c9c4700 1 Processor -- start 2022-06-29T15:46:05.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.707+0000 7fd27c9c4700 1 -- start start 2022-06-29T15:46:05.712 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.707+0000 7fd27c9c4700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd2781430f0 0x7fd2781434d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:05.713 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.707+0000 7fd27c9c4700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd2780a93c0 0x7fd278095240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:05.713 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.707+0000 7fd27c9c4700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd278095730 0x7fd278097690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:05.713 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.707+0000 7fd27c9c4700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd27813d590 con 0x7fd278095730 2022-06-29T15:46:05.713 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.707+0000 7fd27c9c4700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd278060e50 con 0x7fd2781430f0 2022-06-29T15:46:05.714 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.707+0000 7fd27c9c4700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2780610f0 con 0x7fd2780a93c0 2022-06-29T15:46:05.714 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.707+0000 7fd276d9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd278095730 0x7fd278097690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:05.714 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.707+0000 7fd276d9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd278095730 0x7fd278097690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48708/0 (socket says 172.21.15.47:48708) 2022-06-29T15:46:05.714 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.707+0000 7fd276d9d700 1 -- 172.21.15.47:0/36619074 learned_addr learned my addr 172.21.15.47:0/36619074 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:46:05.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.707+0000 7fd275d9b700 1 --2- 172.21.15.47:0/36619074 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd2780a93c0 0x7fd278095240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:05.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.707+0000 7fd27659c700 1 --2- 172.21.15.47:0/36619074 >> v2:172.21.15.64:3300/0 conn(0x7fd2781430f0 0x7fd2781434d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:05.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.707+0000 7fd276d9d700 1 -- 172.21.15.47:0/36619074 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd2780a93c0 msgr2=0x7fd278095240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:46:05.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.707+0000 7fd276d9d700 1 --2- 172.21.15.47:0/36619074 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd2780a93c0 0x7fd278095240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:05.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.707+0000 7fd276d9d700 1 -- 172.21.15.47:0/36619074 >> v2:172.21.15.64:3300/0 conn(0x7fd2781430f0 msgr2=0x7fd2781434d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:46:05.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.707+0000 7fd276d9d700 1 --2- 172.21.15.47:0/36619074 >> v2:172.21.15.64:3300/0 conn(0x7fd2781430f0 0x7fd2781434d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:05.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.707+0000 7fd276d9d700 1 -- 172.21.15.47:0/36619074 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2780971f0 con 0x7fd278095730 2022-06-29T15:46:05.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.707+0000 7fd27659c700 1 --2- 172.21.15.47:0/36619074 >> v2:172.21.15.64:3300/0 conn(0x7fd2781430f0 0x7fd2781434d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:46:05.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.707+0000 7fd276d9d700 1 --2- 172.21.15.47:0/36619074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd278095730 0x7fd278097690 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7fd2700032f0 tx=0x7fd270009290).ready entity=mon.0 client_cookie=b3122b4b0f6aa98d server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:05.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.707+0000 7fd27559a700 1 -- 172.21.15.47:0/36619074 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd27000b870 con 0x7fd278095730 2022-06-29T15:46:05.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.707+0000 7fd27559a700 1 -- 172.21.15.47:0/36619074 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd27000b9d0 con 0x7fd278095730 2022-06-29T15:46:05.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.707+0000 7fd27559a700 1 -- 172.21.15.47:0/36619074 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd27000bb40 con 0x7fd278095730 2022-06-29T15:46:05.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.707+0000 7fd27c9c4700 1 -- 172.21.15.47:0/36619074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd278095730 msgr2=0x7fd278097690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:05.718 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.707+0000 7fd27c9c4700 1 --2- 172.21.15.47:0/36619074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd278095730 0x7fd278097690 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7fd2700032f0 tx=0x7fd270009290).stop 2022-06-29T15:46:05.718 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.707+0000 7fd27c9c4700 1 -- 172.21.15.47:0/36619074 shutdown_connections 2022-06-29T15:46:05.718 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.707+0000 7fd27c9c4700 1 --2- 172.21.15.47:0/36619074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd278095730 0x7fd278097690 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:05.718 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.707+0000 7fd27c9c4700 1 --2- 172.21.15.47:0/36619074 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd2780a93c0 0x7fd278095240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:05.719 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.707+0000 7fd27c9c4700 1 --2- 172.21.15.47:0/36619074 >> v2:172.21.15.64:3300/0 conn(0x7fd2781430f0 0x7fd2781434d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:05.719 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.707+0000 7fd27c9c4700 1 -- 172.21.15.47:0/36619074 >> 172.21.15.47:0/36619074 conn(0x7fd27805f0b0 msgr2=0x7fd27805f490 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:05.719 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.707+0000 7fd27c9c4700 1 -- 172.21.15.47:0/36619074 shutdown_connections 2022-06-29T15:46:05.719 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.707+0000 7fd27c9c4700 1 -- 172.21.15.47:0/36619074 wait complete. 2022-06-29T15:46:05.723 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.711+0000 7fd27c9c4700 1 Processor -- start 2022-06-29T15:46:05.723 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.711+0000 7fd27c9c4700 1 -- start start 2022-06-29T15:46:05.723 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.711+0000 7fd27c9c4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd278095730 0x7fd2781565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:05.724 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.711+0000 7fd27c9c4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2780a93c0 0x7fd2781525a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:05.724 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.711+0000 7fd27c9c4700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd2781430f0 0x7fd2780b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:05.724 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.711+0000 7fd27c9c4700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd278099300 con 0x7fd2780a93c0 2022-06-29T15:46:05.724 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.711+0000 7fd27c9c4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd278155c10 con 0x7fd278095730 2022-06-29T15:46:05.725 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.711+0000 7fd27c9c4700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd278155eb0 con 0x7fd2781430f0 2022-06-29T15:46:05.725 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.711+0000 7fd275d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2780a93c0 0x7fd2781525a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:05.725 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.711+0000 7fd275d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2780a93c0 0x7fd2781525a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48712/0 (socket says 172.21.15.47:48712) 2022-06-29T15:46:05.725 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.711+0000 7fd275d9b700 1 -- 172.21.15.47:0/1492433222 learned_addr learned my addr 172.21.15.47:0/1492433222 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:46:05.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.711+0000 7fd276d9d700 1 --2- 172.21.15.47:0/1492433222 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd2781430f0 0x7fd2780b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:05.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.711+0000 7fd27659c700 1 --2- 172.21.15.47:0/1492433222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd278095730 0x7fd2781565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:05.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.711+0000 7fd275d9b700 1 -- 172.21.15.47:0/1492433222 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd2781430f0 msgr2=0x7fd2780b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:05.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.711+0000 7fd275d9b700 1 --2- 172.21.15.47:0/1492433222 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd2781430f0 0x7fd2780b4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:05.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.711+0000 7fd275d9b700 1 -- 172.21.15.47:0/1492433222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd278095730 msgr2=0x7fd2781565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:05.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.711+0000 7fd275d9b700 1 --2- 172.21.15.47:0/1492433222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd278095730 0x7fd2781565f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:05.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.711+0000 7fd275d9b700 1 -- 172.21.15.47:0/1492433222 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd270009530 con 0x7fd2780a93c0 2022-06-29T15:46:05.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.711+0000 7fd27659c700 1 --2- 172.21.15.47:0/1492433222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd278095730 0x7fd2781565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:46:05.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.711+0000 7fd275d9b700 1 --2- 172.21.15.47:0/1492433222 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2780a93c0 0x7fd2781525a0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7fd26800a000 tx=0x7fd268008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:05.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.715+0000 7fd2677fe700 1 -- 172.21.15.47:0/1492433222 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2680106e0 con 0x7fd2780a93c0 2022-06-29T15:46:05.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.715+0000 7fd2677fe700 1 -- 172.21.15.47:0/1492433222 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd268010840 con 0x7fd2780a93c0 2022-06-29T15:46:05.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.715+0000 7fd2677fe700 1 -- 172.21.15.47:0/1492433222 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd268010ba0 con 0x7fd2780a93c0 2022-06-29T15:46:05.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.715+0000 7fd27c9c4700 1 -- 172.21.15.47:0/1492433222 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd278156150 con 0x7fd2780a93c0 2022-06-29T15:46:05.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.715+0000 7fd27c9c4700 1 -- 172.21.15.47:0/1492433222 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd278094f00 con 0x7fd2780a93c0 2022-06-29T15:46:05.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.715+0000 7fd27c9c4700 1 -- 172.21.15.47:0/1492433222 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2781562e0 con 0x7fd2780a93c0 2022-06-29T15:46:05.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.715+0000 7fd2677fe700 1 -- 172.21.15.47:0/1492433222 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd26801a030 con 0x7fd2780a93c0 2022-06-29T15:46:05.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.715+0000 7fd2677fe700 1 --2- 172.21.15.47:0/1492433222 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd254053b00 0x7fd254055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:05.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.715+0000 7fd2677fe700 1 -- 172.21.15.47:0/1492433222 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd26806bc40 con 0x7fd2780a93c0 2022-06-29T15:46:05.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.715+0000 7fd27659c700 1 --2- 172.21.15.47:0/1492433222 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd254053b00 0x7fd254055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:05.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.715+0000 7fd27659c700 1 --2- 172.21.15.47:0/1492433222 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd254053b00 0x7fd254055fc0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7fd26c00b000 tx=0x7fd26c008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:05.731 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.723+0000 7fd2677fe700 1 -- 172.21.15.47:0/1492433222 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd2680429a0 con 0x7fd2780a93c0 2022-06-29T15:46:05.890 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.887+0000 7fd27c9c4700 1 -- 172.21.15.47:0/1492433222 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2780573d0 con 0x7fd2780a93c0 2022-06-29T15:46:05.891 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.887+0000 7fd2677fe700 1 -- 172.21.15.47:0/1492433222 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+2492 (secure 0 0 0) 0x7fd26806b030 con 0x7fd2780a93c0 2022-06-29T15:46:05.892 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:46:05.892 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":24,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5595,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1798995166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1798995166},{"type":"v1","addr":"172.21.15.47:6835","nonce":1798995166}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21},{"gid":5601,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/1641478219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1641478219},{"type":"v1","addr":"172.21.15.64:6833","nonce":1641478219}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:46:01.294125+0000","modified":"2022-06-29T15:46:02.296567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5607":{"gid":5607,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6837/2427298469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2427298469},{"type":"v1","addr":"172.21.15.47:6837","nonce":2427298469}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2022-06-29T15:46:05.894 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.891+0000 7fd27c9c4700 1 -- 172.21.15.47:0/1492433222 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd254053b00 msgr2=0x7fd254055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:05.895 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.891+0000 7fd27c9c4700 1 --2- 172.21.15.47:0/1492433222 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd254053b00 0x7fd254055fc0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7fd26c00b000 tx=0x7fd26c008040).stop 2022-06-29T15:46:05.895 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.891+0000 7fd27c9c4700 1 -- 172.21.15.47:0/1492433222 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2780a93c0 msgr2=0x7fd2781525a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:05.895 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.891+0000 7fd27c9c4700 1 --2- 172.21.15.47:0/1492433222 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2780a93c0 0x7fd2781525a0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7fd26800a000 tx=0x7fd268008040).stop 2022-06-29T15:46:05.896 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.891+0000 7fd27c9c4700 1 -- 172.21.15.47:0/1492433222 shutdown_connections 2022-06-29T15:46:05.896 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.891+0000 7fd27c9c4700 1 --2- 172.21.15.47:0/1492433222 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd2781430f0 0x7fd2780b4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:05.896 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.891+0000 7fd27c9c4700 1 --2- 172.21.15.47:0/1492433222 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2780a93c0 0x7fd2781525a0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:05.896 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.891+0000 7fd27c9c4700 1 --2- 172.21.15.47:0/1492433222 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd254053b00 0x7fd254055fc0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:05.897 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.891+0000 7fd27c9c4700 1 --2- 172.21.15.47:0/1492433222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd278095730 0x7fd2781565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:05.897 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.891+0000 7fd27c9c4700 1 -- 172.21.15.47:0/1492433222 >> 172.21.15.47:0/1492433222 conn(0x7fd27805f0b0 msgr2=0x7fd27813af60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:05.897 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.891+0000 7fd27c9c4700 1 -- 172.21.15.47:0/1492433222 shutdown_connections 2022-06-29T15:46:05.898 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:05.891+0000 7fd27c9c4700 1 -- 172.21.15.47:0/1492433222 wait complete. 2022-06-29T15:46:05.898 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 24 2022-06-29T15:46:06.208 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-29T15:46:01.294125+0000', 'modified': '2022-06-29T15:46:02.296567+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5607}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5607': {'gid': 5607, 'name': 'a', 'rank': 0, 'incarnation': 23, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.47:6837/2427298469', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6836', 'nonce': 2427298469}, {'type': 'v1', 'addr': '172.21.15.47:6837', 'nonce': 2427298469}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-29T15:46:06.209 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-29T15:46:06.209 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-06-29T15:46:06.330 INFO:teuthology.orchestra.run.smithi047.stdout:{"cluster_fsid":"9a68894a-53ce-400e-838b-14eab91b834c","whoami":0,"id":5607,"want_state":"up:active","state":"up:active","rank_uptime":5.0232053859999999,"mdsmap_epoch":24,"osdmap_epoch":58,"osdmap_epoch_barrier":56,"uptime":9.3883251800000007} 2022-06-29T15:46:06.331 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9a68894a-53ce-400e-838b-14eab91b834c", "id": 5607, "mdsmap_epoch": 24, "osdmap_epoch": 58, "osdmap_epoch_barrier": 56, "rank_uptime": 5.023205386, "state": "up:active", "uptime": 9.38832518, "want_state": "up:active", "whoami": 0 } 2022-06-29T15:46:06.332 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-29T15:46:06.332 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-06-29T15:46:06.332 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi099.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-06-29T15:46:06.333 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:46:06.334 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:46:06.343 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2022-06-29T15:46:06.356 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:46:06.357 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:46:06.414 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:46:06.414 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:46:06.416 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:46:06.416 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2022-06-29T15:46:06.423 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-29T15:46:06.424 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:46:06.426 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2022-06-29T15:46:06.439 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:46:06.440 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:46:06.451 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:46:06.452 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:46:06.453 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:46:06.453 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2022-06-29T15:46:06.501 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:2022-06-29T15:46:06.497+0000 7f746b94e080 -1 init, newargv = 0x55a8f8e18560 newargc=9ceph-fuse[12572]: starting ceph client 2022-06-29T15:46:06.501 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr: 2022-06-29T15:46:06.511 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[12572]: starting fuse 2022-06-29T15:46:07.604 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2022-06-29T15:46:07.623 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:46:07.623 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:46:07.680 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:46:07.681 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:46:07.683 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:46:07.683 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2022-06-29T15:46:07.689 INFO:teuthology.orchestra.run.smithi099.stdout:52 2022-06-29T15:46:07.690 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-29T15:46:07.690 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:46:07.691 DEBUG:teuthology.orchestra.run.smithi099:> sudo python3 -c ' 2022-06-29T15:46:07.691 DEBUG:teuthology.orchestra.run.smithi099:> import glob 2022-06-29T15:46:07.692 DEBUG:teuthology.orchestra.run.smithi099:> import re 2022-06-29T15:46:07.692 DEBUG:teuthology.orchestra.run.smithi099:> import os 2022-06-29T15:46:07.693 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2022-06-29T15:46:07.693 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:46:07.694 DEBUG:teuthology.orchestra.run.smithi099:> def find_socket(client_name): 2022-06-29T15:46:07.694 DEBUG:teuthology.orchestra.run.smithi099:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-06-29T15:46:07.694 DEBUG:teuthology.orchestra.run.smithi099:> files = glob.glob(asok_path) 2022-06-29T15:46:07.695 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:46:07.695 DEBUG:teuthology.orchestra.run.smithi099:> # Given a non-glob path, it better be there 2022-06-29T15:46:07.696 DEBUG:teuthology.orchestra.run.smithi099:> if "*" not in asok_path: 2022-06-29T15:46:07.696 DEBUG:teuthology.orchestra.run.smithi099:> assert(len(files) == 1) 2022-06-29T15:46:07.696 DEBUG:teuthology.orchestra.run.smithi099:> return files[0] 2022-06-29T15:46:07.697 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:46:07.697 DEBUG:teuthology.orchestra.run.smithi099:> for f in files: 2022-06-29T15:46:07.698 DEBUG:teuthology.orchestra.run.smithi099:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-29T15:46:07.698 DEBUG:teuthology.orchestra.run.smithi099:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-29T15:46:07.699 DEBUG:teuthology.orchestra.run.smithi099:> return f 2022-06-29T15:46:07.699 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-29T15:46:07.700 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:46:07.700 DEBUG:teuthology.orchestra.run.smithi099:> print(find_socket("client.0")) 2022-06-29T15:46:07.701 DEBUG:teuthology.orchestra.run.smithi099:> ' 2022-06-29T15:46:07.744 INFO:teuthology.orchestra.run.smithi099.stdout:/var/run/ceph/ceph-client.0.12572.asok 2022-06-29T15:46:07.749 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.12572.asok 2022-06-29T15:46:07.750 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:46:07.750 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.12572.asok status 2022-06-29T15:46:07.881 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2022-06-29T15:46:07.881 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": { 2022-06-29T15:46:07.882 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_sha1": "f25ffac72e5a9e63860ff7deb6767a11f0559a16", 2022-06-29T15:46:07.882 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 15.2.16-304-gf25ffac7 (f25ffac72e5a9e63860ff7deb6767a11f0559a16) octopus (stable)", 2022-06-29T15:46:07.882 INFO:teuthology.orchestra.run.smithi099.stdout: "entity_id": "0", 2022-06-29T15:46:07.883 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": "smithi099", 2022-06-29T15:46:07.883 INFO:teuthology.orchestra.run.smithi099.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-06-29T15:46:07.883 INFO:teuthology.orchestra.run.smithi099.stdout: "pid": "12572", 2022-06-29T15:46:07.883 INFO:teuthology.orchestra.run.smithi099.stdout: "root": "/" 2022-06-29T15:46:07.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:46:07.884 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_count": 0, 2022-06-29T15:46:07.884 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_pinned_count": 0, 2022-06-29T15:46:07.884 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 55199, 2022-06-29T15:46:07.885 INFO:teuthology.orchestra.run.smithi099.stdout: "inst": { 2022-06-29T15:46:07.885 INFO:teuthology.orchestra.run.smithi099.stdout: "name": { 2022-06-29T15:46:07.885 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "client", 2022-06-29T15:46:07.886 INFO:teuthology.orchestra.run.smithi099.stdout: "num": 55199 2022-06-29T15:46:07.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:46:07.887 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2022-06-29T15:46:07.887 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2022-06-29T15:46:07.887 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:0", 2022-06-29T15:46:07.888 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 938405779 2022-06-29T15:46:07.888 INFO:teuthology.orchestra.run.smithi099.stdout: } 2022-06-29T15:46:07.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:46:07.888 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2022-06-29T15:46:07.889 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2022-06-29T15:46:07.889 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:0", 2022-06-29T15:46:07.889 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 938405779 2022-06-29T15:46:07.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:46:07.889 INFO:teuthology.orchestra.run.smithi099.stdout: "inst_str": "client.55199 172.21.15.99:0/938405779", 2022-06-29T15:46:07.890 INFO:teuthology.orchestra.run.smithi099.stdout: "addr_str": "172.21.15.99:0/938405779", 2022-06-29T15:46:07.890 INFO:teuthology.orchestra.run.smithi099.stdout: "inode_count": 1, 2022-06-29T15:46:07.890 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_epoch": 24, 2022-06-29T15:46:07.891 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch": 58, 2022-06-29T15:46:07.891 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch_barrier": 0, 2022-06-29T15:46:07.891 INFO:teuthology.orchestra.run.smithi099.stdout: "blacklisted": false, 2022-06-29T15:46:07.891 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs" 2022-06-29T15:46:07.891 INFO:teuthology.orchestra.run.smithi099.stdout:} 2022-06-29T15:46:07.893 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:46:07.894 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:46:07.894 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:46:07.901 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2022-06-29T15:46:07.903 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-29T15:46:07.903 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:46:07.904 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:46:07.922 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:46:07.923 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:46:07.923 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:46:07.970 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2022-06-29T15:46:07.971 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-29T15:46:07.972 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2022-06-29T15:46:07.972 DEBUG:teuthology.orchestra.run.smithi099:> import sys 2022-06-29T15:46:07.973 DEBUG:teuthology.orchestra.run.smithi099:> import time 2022-06-29T15:46:07.973 DEBUG:teuthology.orchestra.run.smithi099:> import os 2022-06-29T15:46:07.973 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:46:07.974 DEBUG:teuthology.orchestra.run.smithi099:> n = 1000 2022-06-29T15:46:07.975 DEBUG:teuthology.orchestra.run.smithi099:> abs_path = "/home/ubuntu/cephtest/mnt.0/dc-dir/dc-file" 2022-06-29T15:46:07.975 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:46:07.975 DEBUG:teuthology.orchestra.run.smithi099:> if not os.path.exists(os.path.dirname(abs_path)): 2022-06-29T15:46:07.976 DEBUG:teuthology.orchestra.run.smithi099:> os.makedirs(os.path.dirname(abs_path)) 2022-06-29T15:46:07.976 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:46:07.977 DEBUG:teuthology.orchestra.run.smithi099:> for i in range(0, n): 2022-06-29T15:46:07.978 DEBUG:teuthology.orchestra.run.smithi099:> fname = "{0}_{1}".format(abs_path, i) 2022-06-29T15:46:07.978 DEBUG:teuthology.orchestra.run.smithi099:> with open(fname, '"'"'w'"'"') as f: 2022-06-29T15:46:07.979 DEBUG:teuthology.orchestra.run.smithi099:> f.write('"'"'content'"'"') 2022-06-29T15:46:07.979 DEBUG:teuthology.orchestra.run.smithi099:> if True: 2022-06-29T15:46:07.980 DEBUG:teuthology.orchestra.run.smithi099:> f.flush() 2022-06-29T15:46:07.980 DEBUG:teuthology.orchestra.run.smithi099:> os.fsync(f.fileno()) 2022-06-29T15:46:07.981 DEBUG:teuthology.orchestra.run.smithi099:> ' 2022-06-29T15:46:11.484 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:46:11.487 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:46:11.490 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:46:11.492 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:46:19.064 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:46:19.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.175+0000 7f7d709e6700 1 Processor -- start 2022-06-29T15:46:19.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.175+0000 7f7d709e6700 1 -- start start 2022-06-29T15:46:19.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.175+0000 7f7d709e6700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7d6c0619b0 0x7f7d6c0606e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:19.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.175+0000 7f7d709e6700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7d6c1408e0 0x7f7d6c139a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:19.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.175+0000 7f7d709e6700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d6c140d50 con 0x7f7d6c0b0f30 2022-06-29T15:46:19.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.175+0000 7f7d709e6700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d6c0b2190 con 0x7f7d6c1408e0 2022-06-29T15:46:19.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.175+0000 7f7d709e6700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d6c061d90 con 0x7f7d6c0619b0 2022-06-29T15:46:19.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.175+0000 7f7d69d9b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7d6c0b0f30 0x7f7d6c0b1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34272/0 (socket says 172.21.15.47:34272) 2022-06-29T15:46:19.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.175+0000 7f7d69d9b700 1 -- 172.21.15.47:0/2762030181 learned_addr learned my addr 172.21.15.47:0/2762030181 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:46:19.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.175+0000 7f7d6ad9d700 1 --2- 172.21.15.47:0/2762030181 >> v2:172.21.15.47:3300/0 conn(0x7f7d6c1408e0 0x7f7d6c139a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:19.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.175+0000 7f7d6a59c700 1 --2- 172.21.15.47:0/2762030181 >> v2:172.21.15.64:3300/0 conn(0x7f7d6c0619b0 0x7f7d6c0606e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:19.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.175+0000 7f7d6959a700 1 -- 172.21.15.47:0/2762030181 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3454164328 0 0) 0x7f7d6c140d50 con 0x7f7d6c0b0f30 2022-06-29T15:46:19.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.175+0000 7f7d6959a700 1 -- 172.21.15.47:0/2762030181 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d4c003260 con 0x7f7d6c0b0f30 2022-06-29T15:46:19.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.175+0000 7f7d6a59c700 1 -- 172.21.15.47:0/2762030181 >> v2:172.21.15.47:3300/0 conn(0x7f7d6c1408e0 msgr2=0x7f7d6c139a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:46:19.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.175+0000 7f7d6a59c700 1 --2- 172.21.15.47:0/2762030181 >> v2:172.21.15.47:3300/0 conn(0x7f7d6c1408e0 0x7f7d6c139a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:19.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.175+0000 7f7d6a59c700 1 -- 172.21.15.47:0/2762030181 >> v1:172.21.15.47:6789/0 conn(0x7f7d6c0b0f30 legacy=0x7f7d6c0b1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:19.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.175+0000 7f7d6959a700 1 -- 172.21.15.47:0/2762030181 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1203252180 0 0) 0x7f7d4c003260 con 0x7f7d6c0b0f30 2022-06-29T15:46:19.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.175+0000 7f7d6a59c700 1 -- 172.21.15.47:0/2762030181 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d6c060d80 con 0x7f7d6c0619b0 2022-06-29T15:46:19.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.175+0000 7f7d6a59c700 1 --2- 172.21.15.47:0/2762030181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d6c0619b0 0x7f7d6c0606e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7f7d5c001b80 tx=0x7f7d54006df0).ready entity=mon.1 client_cookie=8d0eb550cd0b38a6 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:19.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.175+0000 7f7d6959a700 1 -- 172.21.15.47:0/2762030181 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d5c0019e0 con 0x7f7d6c0619b0 2022-06-29T15:46:19.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.175+0000 7f7d6959a700 1 -- 172.21.15.47:0/2762030181 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d54000e90 con 0x7f7d6c0619b0 2022-06-29T15:46:19.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.175+0000 7f7d6959a700 1 -- 172.21.15.47:0/2762030181 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d54012460 con 0x7f7d6c0619b0 2022-06-29T15:46:19.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.175+0000 7f7d709e6700 1 -- 172.21.15.47:0/2762030181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d6c0619b0 msgr2=0x7f7d6c0606e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:19.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.175+0000 7f7d709e6700 1 --2- 172.21.15.47:0/2762030181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d6c0619b0 0x7f7d6c0606e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7f7d5c001b80 tx=0x7f7d54006df0).stop 2022-06-29T15:46:19.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.179+0000 7f7d709e6700 1 -- 172.21.15.47:0/2762030181 shutdown_connections 2022-06-29T15:46:19.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.179+0000 7f7d709e6700 1 --2- 172.21.15.47:0/2762030181 >> v2:172.21.15.47:3300/0 conn(0x7f7d6c1408e0 0x7f7d6c139a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:19.191 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.179+0000 7f7d709e6700 1 --2- 172.21.15.47:0/2762030181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d6c0619b0 0x7f7d6c0606e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:19.191 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.179+0000 7f7d709e6700 1 -- 172.21.15.47:0/2762030181 >> 172.21.15.47:0/2762030181 conn(0x7f7d6c05c510 msgr2=0x7f7d6c063800 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:19.192 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.179+0000 7f7d709e6700 1 -- 172.21.15.47:0/2762030181 shutdown_connections 2022-06-29T15:46:19.192 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.179+0000 7f7d709e6700 1 -- 172.21.15.47:0/2762030181 wait complete. 2022-06-29T15:46:19.193 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.179+0000 7f7d709e6700 1 Processor -- start 2022-06-29T15:46:19.193 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.179+0000 7f7d709e6700 1 -- start start 2022-06-29T15:46:19.194 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.179+0000 7f7d709e6700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7d6c0619b0 0x7f7d6c144750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:19.194 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.179+0000 7f7d709e6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d6c0b0f30 0x7f7d6c1351d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:19.194 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.179+0000 7f7d709e6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d6c1408e0 0x7f7d6c1356c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:19.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.179+0000 7f7d709e6700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c0b5e00 con 0x7f7d6c0b0f30 2022-06-29T15:46:19.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.179+0000 7f7d709e6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c138cc0 con 0x7f7d6c1408e0 2022-06-29T15:46:19.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.179+0000 7f7d709e6700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c145050 con 0x7f7d6c0619b0 2022-06-29T15:46:19.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.179+0000 7f7d69d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d6c0b0f30 0x7f7d6c1351d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:19.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.179+0000 7f7d69d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d6c0b0f30 0x7f7d6c1351d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48726/0 (socket says 172.21.15.47:48726) 2022-06-29T15:46:19.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.179+0000 7f7d69d9b700 1 -- 172.21.15.47:0/105945622 learned_addr learned my addr 172.21.15.47:0/105945622 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:46:19.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.179+0000 7f7d6ad9d700 1 --2- 172.21.15.47:0/105945622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d6c1408e0 0x7f7d6c1356c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:19.198 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.179+0000 7f7d6a59c700 1 --2- 172.21.15.47:0/105945622 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7d6c0619b0 0x7f7d6c144750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:19.198 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.179+0000 7f7d69d9b700 1 -- 172.21.15.47:0/105945622 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7d6c0619b0 msgr2=0x7f7d6c144750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:19.198 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.179+0000 7f7d69d9b700 1 --2- 172.21.15.47:0/105945622 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7d6c0619b0 0x7f7d6c144750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:19.198 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.179+0000 7f7d69d9b700 1 -- 172.21.15.47:0/105945622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d6c1408e0 msgr2=0x7f7d6c1356c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:19.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.179+0000 7f7d69d9b700 1 --2- 172.21.15.47:0/105945622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d6c1408e0 0x7f7d6c1356c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:19.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.179+0000 7f7d69d9b700 1 -- 172.21.15.47:0/105945622 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d5400b180 con 0x7f7d6c0b0f30 2022-06-29T15:46:19.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.179+0000 7f7d69d9b700 1 --2- 172.21.15.47:0/105945622 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d6c0b0f30 0x7f7d6c1351d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f7d5c001c40 tx=0x7f7d5c002180).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:19.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.183+0000 7f7d637fe700 1 -- 172.21.15.47:0/105945622 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d5c005ba0 con 0x7f7d6c0b0f30 2022-06-29T15:46:19.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.183+0000 7f7d637fe700 1 -- 172.21.15.47:0/105945622 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d5c005d00 con 0x7f7d6c0b0f30 2022-06-29T15:46:19.202 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.183+0000 7f7d709e6700 1 -- 172.21.15.47:0/105945622 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d6c0b5420 con 0x7f7d6c0b0f30 2022-06-29T15:46:19.202 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.183+0000 7f7d637fe700 1 -- 172.21.15.47:0/105945622 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d5c007300 con 0x7f7d6c0b0f30 2022-06-29T15:46:19.202 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.183+0000 7f7d709e6700 1 -- 172.21.15.47:0/105945622 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d6c0b5960 con 0x7f7d6c0b0f30 2022-06-29T15:46:19.202 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.183+0000 7f7d709e6700 1 -- 172.21.15.47:0/105945622 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d6c0b5b20 con 0x7f7d6c0b0f30 2022-06-29T15:46:19.203 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.183+0000 7f7d637fe700 1 -- 172.21.15.47:0/105945622 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7d5c010030 con 0x7f7d6c0b0f30 2022-06-29T15:46:19.203 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.183+0000 7f7d637fe700 1 --2- 172.21.15.47:0/105945622 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7d50053b00 0x7f7d50055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:19.203 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.183+0000 7f7d6a59c700 1 --2- 172.21.15.47:0/105945622 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7d50053b00 0x7f7d50055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:19.203 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.183+0000 7f7d637fe700 1 -- 172.21.15.47:0/105945622 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f7d5c046a80 con 0x7f7d6c0b0f30 2022-06-29T15:46:19.204 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.187+0000 7f7d6a59c700 1 --2- 172.21.15.47:0/105945622 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7d50053b00 0x7f7d50055fc0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f7d5401a000 tx=0x7f7d54014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:19.204 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.191+0000 7f7d637fe700 1 -- 172.21.15.47:0/105945622 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7d5c042e20 con 0x7f7d6c0b0f30 2022-06-29T15:46:19.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.351+0000 7f7d709e6700 1 -- 172.21.15.47:0/105945622 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d6c0573d0 con 0x7f7d6c0b0f30 2022-06-29T15:46:19.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.351+0000 7f7d637fe700 1 -- 172.21.15.47:0/105945622 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+2492 (secure 0 0 0) 0x7f7d5c046440 con 0x7f7d6c0b0f30 2022-06-29T15:46:19.355 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:46:19.356 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":24,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5595,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1798995166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1798995166},{"type":"v1","addr":"172.21.15.47:6835","nonce":1798995166}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21},{"gid":5601,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/1641478219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1641478219},{"type":"v1","addr":"172.21.15.64:6833","nonce":1641478219}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:46:01.294125+0000","modified":"2022-06-29T15:46:02.296567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5607":{"gid":5607,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6837/2427298469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2427298469},{"type":"v1","addr":"172.21.15.47:6837","nonce":2427298469}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2022-06-29T15:46:19.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.355+0000 7f7d709e6700 1 -- 172.21.15.47:0/105945622 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7d50053b00 msgr2=0x7f7d50055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:19.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.355+0000 7f7d709e6700 1 --2- 172.21.15.47:0/105945622 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7d50053b00 0x7f7d50055fc0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f7d5401a000 tx=0x7f7d54014000).stop 2022-06-29T15:46:19.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.355+0000 7f7d709e6700 1 -- 172.21.15.47:0/105945622 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d6c0b0f30 msgr2=0x7f7d6c1351d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:19.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.355+0000 7f7d709e6700 1 --2- 172.21.15.47:0/105945622 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d6c0b0f30 0x7f7d6c1351d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f7d5c001c40 tx=0x7f7d5c002180).stop 2022-06-29T15:46:19.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.355+0000 7f7d709e6700 1 -- 172.21.15.47:0/105945622 shutdown_connections 2022-06-29T15:46:19.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.355+0000 7f7d709e6700 1 --2- 172.21.15.47:0/105945622 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7d50053b00 0x7f7d50055fc0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:19.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.355+0000 7f7d709e6700 1 --2- 172.21.15.47:0/105945622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d6c1408e0 0x7f7d6c1356c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:19.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.355+0000 7f7d709e6700 1 --2- 172.21.15.47:0/105945622 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d6c0b0f30 0x7f7d6c1351d0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:19.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.355+0000 7f7d709e6700 1 --2- 172.21.15.47:0/105945622 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7d6c0619b0 0x7f7d6c144750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:19.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.355+0000 7f7d709e6700 1 -- 172.21.15.47:0/105945622 >> 172.21.15.47:0/105945622 conn(0x7f7d6c05c510 msgr2=0x7f7d6c13e070 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:19.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.355+0000 7f7d709e6700 1 -- 172.21.15.47:0/105945622 shutdown_connections 2022-06-29T15:46:19.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.355+0000 7f7d709e6700 1 -- 172.21.15.47:0/105945622 wait complete. 2022-06-29T15:46:19.361 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 24 2022-06-29T15:46:19.369 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_recall_max_caps 20 2022-06-29T15:46:19.471 INFO:teuthology.orchestra.run.smithi047.stdout:{"success":"mds_recall_max_caps = '20' (not observed, change may require restart) "} 2022-06-29T15:46:19.472 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_caps = '20' (not observed, change may require restart) " } 2022-06-29T15:46:19.473 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:46:19.578 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.575+0000 7ff551f65700 1 Processor -- start 2022-06-29T15:46:19.579 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.575+0000 7ff551f65700 1 -- start start 2022-06-29T15:46:19.579 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.575+0000 7ff551f65700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff54c0b0cf0 0x7ff54c146d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:19.580 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.575+0000 7ff551f65700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff54c0b06d0 0x7ff54c13ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:19.580 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.575+0000 7ff551f65700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff54c0b4520 con 0x7ff54c0611b0 2022-06-29T15:46:19.581 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.575+0000 7ff551f65700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff54c0b4b10 con 0x7ff54c0b06d0 2022-06-29T15:46:19.581 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.575+0000 7ff551f65700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c0621d0 con 0x7ff54c0b0cf0 2022-06-29T15:46:19.581 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.575+0000 7ff54affd700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff54c0b0cf0 0x7ff54c146d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:19.582 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.575+0000 7ff54bfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff54c0b06d0 0x7ff54c13ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:19.582 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.575+0000 7ff54bfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff54c0b06d0 0x7ff54c13ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:54440/0 (socket says 172.21.15.47:54440) 2022-06-29T15:46:19.582 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.575+0000 7ff54bfff700 1 -- 172.21.15.47:0/506071963 learned_addr learned my addr 172.21.15.47:0/506071963 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:46:19.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.575+0000 7ff54affd700 1 -- 172.21.15.47:0/506071963 >> v2:172.21.15.64:3300/0 conn(0x7ff54c0b06d0 msgr2=0x7ff54c13ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:46:19.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.575+0000 7ff54affd700 1 --2- 172.21.15.47:0/506071963 >> v2:172.21.15.64:3300/0 conn(0x7ff54c0b06d0 0x7ff54c13ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:19.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.575+0000 7ff54affd700 1 -- 172.21.15.47:0/506071963 >> v1:172.21.15.64:6789/0 conn(0x7ff54c0611b0 legacy=0x7ff54c146640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:19.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.575+0000 7ff54affd700 1 -- 172.21.15.47:0/506071963 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff54c061590 con 0x7ff54c0b0cf0 2022-06-29T15:46:19.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.575+0000 7ff54bfff700 1 --2- 172.21.15.47:0/506071963 >> v2:172.21.15.64:3300/0 conn(0x7ff54c0b06d0 0x7ff54c13ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:46:19.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.575+0000 7ff54affd700 1 --2- 172.21.15.47:0/506071963 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff54c0b0cf0 0x7ff54c146d50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7ff540002f70 tx=0x7ff540006df0).ready entity=mon.2 client_cookie=4e92b4f5c903884e server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:19.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.575+0000 7ff54a7fc700 1 -- 172.21.15.47:0/506071963 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff54000d570 con 0x7ff54c0b0cf0 2022-06-29T15:46:19.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.575+0000 7ff54a7fc700 1 -- 172.21.15.47:0/506071963 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff540000e90 con 0x7ff54c0b0cf0 2022-06-29T15:46:19.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.575+0000 7ff54a7fc700 1 -- 172.21.15.47:0/506071963 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff54000d8c0 con 0x7ff54c0b0cf0 2022-06-29T15:46:19.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.575+0000 7ff551f65700 1 -- 172.21.15.47:0/506071963 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff54c0b0cf0 msgr2=0x7ff54c146d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:19.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.575+0000 7ff551f65700 1 --2- 172.21.15.47:0/506071963 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff54c0b0cf0 0x7ff54c146d50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7ff540002f70 tx=0x7ff540006df0).stop 2022-06-29T15:46:19.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.575+0000 7ff551f65700 1 -- 172.21.15.47:0/506071963 shutdown_connections 2022-06-29T15:46:19.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.575+0000 7ff551f65700 1 --2- 172.21.15.47:0/506071963 >> v2:172.21.15.64:3300/0 conn(0x7ff54c0b06d0 0x7ff54c13ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:19.594 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.575+0000 7ff551f65700 1 --2- 172.21.15.47:0/506071963 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff54c0b0cf0 0x7ff54c146d50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:19.594 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.575+0000 7ff551f65700 1 -- 172.21.15.47:0/506071963 >> 172.21.15.47:0/506071963 conn(0x7ff54c0b98b0 msgr2=0x7ff54c0b9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:19.594 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.575+0000 7ff551f65700 1 -- 172.21.15.47:0/506071963 shutdown_connections 2022-06-29T15:46:19.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.575+0000 7ff551f65700 1 -- 172.21.15.47:0/506071963 wait complete. 2022-06-29T15:46:19.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.579+0000 7ff551f65700 1 Processor -- start 2022-06-29T15:46:19.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.579+0000 7ff551f65700 1 -- start start 2022-06-29T15:46:19.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.579+0000 7ff551f65700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff54c0611b0 0x7ff54c14a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:19.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.579+0000 7ff551f65700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff54c0b06d0 0x7ff54c13e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:19.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.579+0000 7ff551f65700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff54c0b0cf0 0x7ff54c13e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:19.597 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.579+0000 7ff551f65700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c13ecc0 con 0x7ff54c0611b0 2022-06-29T15:46:19.597 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.579+0000 7ff551f65700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c0b8660 con 0x7ff54c0b0cf0 2022-06-29T15:46:19.597 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.579+0000 7ff551f65700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c0b83c0 con 0x7ff54c0b06d0 2022-06-29T15:46:19.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.579+0000 7ff54b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff54c0611b0 0x7ff54c14a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:19.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.579+0000 7ff54b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff54c0611b0 0x7ff54c14a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48738/0 (socket says 172.21.15.47:48738) 2022-06-29T15:46:19.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.579+0000 7ff54b7fe700 1 -- 172.21.15.47:0/3141038643 learned_addr learned my addr 172.21.15.47:0/3141038643 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:46:19.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.579+0000 7ff54affd700 1 --2- 172.21.15.47:0/3141038643 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff54c0b06d0 0x7ff54c13e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:19.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.579+0000 7ff54bfff700 1 --2- 172.21.15.47:0/3141038643 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff54c0b0cf0 0x7ff54c13e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:19.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.579+0000 7ff54b7fe700 1 -- 172.21.15.47:0/3141038643 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff54c0b06d0 msgr2=0x7ff54c13e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:19.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.579+0000 7ff54b7fe700 1 --2- 172.21.15.47:0/3141038643 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff54c0b06d0 0x7ff54c13e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:19.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.579+0000 7ff54b7fe700 1 -- 172.21.15.47:0/3141038643 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff54c0b0cf0 msgr2=0x7ff54c13e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:19.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.579+0000 7ff54b7fe700 1 --2- 172.21.15.47:0/3141038643 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff54c0b0cf0 0x7ff54c13e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:19.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.579+0000 7ff54b7fe700 1 -- 172.21.15.47:0/3141038643 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff53c00b1e0 con 0x7ff54c0611b0 2022-06-29T15:46:19.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.579+0000 7ff54b7fe700 1 --2- 172.21.15.47:0/3141038643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff54c0611b0 0x7ff54c14a270 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7ff54c0ad680 tx=0x7ff544007f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:19.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.579+0000 7ff548ff9700 1 -- 172.21.15.47:0/3141038643 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff544006aa0 con 0x7ff54c0611b0 2022-06-29T15:46:19.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.579+0000 7ff548ff9700 1 -- 172.21.15.47:0/3141038643 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff544006c00 con 0x7ff54c0611b0 2022-06-29T15:46:19.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.579+0000 7ff548ff9700 1 -- 172.21.15.47:0/3141038643 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff544007030 con 0x7ff54c0611b0 2022-06-29T15:46:19.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.579+0000 7ff551f65700 1 -- 172.21.15.47:0/3141038643 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff54000b180 con 0x7ff54c0611b0 2022-06-29T15:46:19.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.579+0000 7ff551f65700 1 -- 172.21.15.47:0/3141038643 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff54c0b0030 con 0x7ff54c0611b0 2022-06-29T15:46:19.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.583+0000 7ff551f65700 1 -- 172.21.15.47:0/3141038643 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff54c0573d0 con 0x7ff54c0611b0 2022-06-29T15:46:19.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.583+0000 7ff548ff9700 1 -- 172.21.15.47:0/3141038643 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff544010030 con 0x7ff54c0611b0 2022-06-29T15:46:19.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.583+0000 7ff548ff9700 1 --2- 172.21.15.47:0/3141038643 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff52c053a70 0x7ff52c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:19.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.583+0000 7ff548ff9700 1 -- 172.21.15.47:0/3141038643 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff54406bff0 con 0x7ff54c0611b0 2022-06-29T15:46:19.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.583+0000 7ff54affd700 1 --2- 172.21.15.47:0/3141038643 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff52c053a70 0x7ff52c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:19.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.583+0000 7ff54affd700 1 --2- 172.21.15.47:0/3141038643 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff52c053a70 0x7ff52c055f30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7ff54001a000 tx=0x7ff540014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:19.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.587+0000 7ff548ff9700 1 -- 172.21.15.47:0/3141038643 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff544042d00 con 0x7ff54c0611b0 2022-06-29T15:46:19.754 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.751+0000 7ff551f65700 1 -- 172.21.15.47:0/3141038643 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff54c061750 con 0x7ff54c0611b0 2022-06-29T15:46:19.755 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.751+0000 7ff548ff9700 1 -- 172.21.15.47:0/3141038643 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+2492 (secure 0 0 0) 0x7ff544046320 con 0x7ff54c0611b0 2022-06-29T15:46:19.755 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:46:19.757 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":24,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5595,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1798995166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1798995166},{"type":"v1","addr":"172.21.15.47:6835","nonce":1798995166}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21},{"gid":5601,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/1641478219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1641478219},{"type":"v1","addr":"172.21.15.64:6833","nonce":1641478219}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:46:01.294125+0000","modified":"2022-06-29T15:46:02.296567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5607":{"gid":5607,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6837/2427298469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2427298469},{"type":"v1","addr":"172.21.15.47:6837","nonce":2427298469}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2022-06-29T15:46:19.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.751+0000 7ff551f65700 1 -- 172.21.15.47:0/3141038643 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff52c053a70 msgr2=0x7ff52c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:19.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.751+0000 7ff551f65700 1 --2- 172.21.15.47:0/3141038643 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff52c053a70 0x7ff52c055f30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7ff54001a000 tx=0x7ff540014000).stop 2022-06-29T15:46:19.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.755+0000 7ff551f65700 1 -- 172.21.15.47:0/3141038643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff54c0611b0 msgr2=0x7ff54c14a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:19.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.755+0000 7ff551f65700 1 --2- 172.21.15.47:0/3141038643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff54c0611b0 0x7ff54c14a270 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7ff54c0ad680 tx=0x7ff544007f90).stop 2022-06-29T15:46:19.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.755+0000 7ff551f65700 1 -- 172.21.15.47:0/3141038643 shutdown_connections 2022-06-29T15:46:19.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.755+0000 7ff551f65700 1 --2- 172.21.15.47:0/3141038643 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff52c053a70 0x7ff52c055f30 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:19.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.755+0000 7ff551f65700 1 --2- 172.21.15.47:0/3141038643 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff54c0b0cf0 0x7ff54c13e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:19.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.755+0000 7ff551f65700 1 --2- 172.21.15.47:0/3141038643 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff54c0b06d0 0x7ff54c13e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:19.761 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.755+0000 7ff551f65700 1 --2- 172.21.15.47:0/3141038643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff54c0611b0 0x7ff54c14a270 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:19.761 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.755+0000 7ff551f65700 1 -- 172.21.15.47:0/3141038643 >> 172.21.15.47:0/3141038643 conn(0x7ff54c0b98b0 msgr2=0x7ff54c13f270 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:19.762 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.755+0000 7ff551f65700 1 -- 172.21.15.47:0/3141038643 shutdown_connections 2022-06-29T15:46:19.762 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.755+0000 7ff551f65700 1 -- 172.21.15.47:0/3141038643 wait complete. 2022-06-29T15:46:19.764 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 24 2022-06-29T15:46:19.768 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_recall_max_decay_threshold 400 2022-06-29T15:46:19.875 INFO:teuthology.orchestra.run.smithi047.stdout:{"success":"mds_recall_max_decay_threshold = '400' (not observed, change may require restart) "} 2022-06-29T15:46:19.876 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_decay_threshold = '400' (not observed, change may require restart) " } 2022-06-29T15:46:19.876 INFO:tasks.cephfs.fuse_mount:Killing ceph-fuse connection on ubuntu@smithi099.front.sepia.ceph.com... 2022-06-29T15:46:19.894 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:46:20.006 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.995+0000 7fedccf36700 1 Processor -- start 2022-06-29T15:46:20.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.995+0000 7fedccf36700 1 -- start start 2022-06-29T15:46:20.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.995+0000 7fedccf36700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fedc805f0b0 0x7fedc80b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:20.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.995+0000 7fedccf36700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedc80b0740 con 0x7fedc80b2120 2022-06-29T15:46:20.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.995+0000 7fedccf36700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedc80b0970 con 0x7fedc80b1260 2022-06-29T15:46:20.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.995+0000 7fedccf36700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc8147460 con 0x7fedc805f0b0 2022-06-29T15:46:20.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.999+0000 7fedc659c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fedc80b2120 0x7fedc8147af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34302/0 (socket says 172.21.15.47:34302) 2022-06-29T15:46:20.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.999+0000 7fedc659c700 1 -- 172.21.15.47:0/895089305 learned_addr learned my addr 172.21.15.47:0/895089305 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:46:20.009 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.999+0000 7fedc5d9b700 1 --2- 172.21.15.47:0/895089305 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fedc805f0b0 0x7fedc80b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:20.009 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.999+0000 7fedc559a700 1 -- 172.21.15.47:0/895089305 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2625728302 0 0) 0x7fedc80b0740 con 0x7fedc80b2120 2022-06-29T15:46:20.009 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.999+0000 7fedc559a700 1 -- 172.21.15.47:0/895089305 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedac003260 con 0x7fedc80b2120 2022-06-29T15:46:20.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.999+0000 7fedc559a700 1 -- 172.21.15.47:0/895089305 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1572535590 0 0) 0x7fedc80b0970 con 0x7fedc80b1260 2022-06-29T15:46:20.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.999+0000 7fedc5d9b700 1 -- 172.21.15.47:0/895089305 >> v1:172.21.15.64:6789/0 conn(0x7fedc80b1260 legacy=0x7fedc805f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:20.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.999+0000 7fedc5d9b700 1 -- 172.21.15.47:0/895089305 >> v1:172.21.15.47:6789/0 conn(0x7fedc80b2120 legacy=0x7fedc8147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:20.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.999+0000 7fedc5d9b700 1 -- 172.21.15.47:0/895089305 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc80b0c10 con 0x7fedc805f0b0 2022-06-29T15:46:20.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.999+0000 7fedc5d9b700 1 --2- 172.21.15.47:0/895089305 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fedc805f0b0 0x7fedc80b2500 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fedbc001800 tx=0x7fedb80091d0).ready entity=mon.2 client_cookie=cf27d075413185cc server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:20.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.999+0000 7fedc559a700 1 -- 172.21.15.47:0/895089305 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fedbc001660 con 0x7fedc805f0b0 2022-06-29T15:46:20.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.999+0000 7fedc559a700 1 -- 172.21.15.47:0/895089305 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb800b580 con 0x7fedc805f0b0 2022-06-29T15:46:20.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.999+0000 7fedc559a700 1 -- 172.21.15.47:0/895089305 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fedb800b6f0 con 0x7fedc805f0b0 2022-06-29T15:46:20.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.999+0000 7fedccf36700 1 -- 172.21.15.47:0/895089305 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fedc805f0b0 msgr2=0x7fedc80b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:20.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.999+0000 7fedccf36700 1 --2- 172.21.15.47:0/895089305 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fedc805f0b0 0x7fedc80b2500 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fedbc001800 tx=0x7fedb80091d0).stop 2022-06-29T15:46:20.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.999+0000 7fedccf36700 1 -- 172.21.15.47:0/895089305 shutdown_connections 2022-06-29T15:46:20.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.999+0000 7fedccf36700 1 --2- 172.21.15.47:0/895089305 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fedc805f0b0 0x7fedc80b2500 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:20.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.999+0000 7fedccf36700 1 -- 172.21.15.47:0/895089305 >> 172.21.15.47:0/895089305 conn(0x7fedc80626f0 msgr2=0x7fedc8062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:20.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.999+0000 7fedccf36700 1 -- 172.21.15.47:0/895089305 shutdown_connections 2022-06-29T15:46:20.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:19.999+0000 7fedccf36700 1 -- 172.21.15.47:0/895089305 wait complete. 2022-06-29T15:46:20.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.003+0000 7fedccf36700 1 Processor -- start 2022-06-29T15:46:20.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.003+0000 7fedccf36700 1 -- start start 2022-06-29T15:46:20.022 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.003+0000 7fedccf36700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fedc80b1260 0x7fedc80aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:20.022 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.003+0000 7fedccf36700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fedc80b2120 0x7fedc8156ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:20.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.003+0000 7fedccf36700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedc8157190 0x7fedc815a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:20.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.003+0000 7fedccf36700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc81568b0 con 0x7fedc80b2120 2022-06-29T15:46:20.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.003+0000 7fedccf36700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc8159e40 con 0x7fedc8157190 2022-06-29T15:46:20.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.003+0000 7fedccf36700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc8144630 con 0x7fedc80b1260 2022-06-29T15:46:20.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.003+0000 7fedc5d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fedc80b2120 0x7fedc8156ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:20.024 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.003+0000 7fedc5d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fedc80b2120 0x7fedc8156ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48754/0 (socket says 172.21.15.47:48754) 2022-06-29T15:46:20.024 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.003+0000 7fedc659c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fedc80b1260 0x7fedc80aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:20.024 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.003+0000 7fedc5d9b700 1 -- 172.21.15.47:0/2023462268 learned_addr learned my addr 172.21.15.47:0/2023462268 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:46:20.024 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.003+0000 7fedc6d9d700 1 --2- 172.21.15.47:0/2023462268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedc8157190 0x7fedc815a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:20.025 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.003+0000 7fedc5d9b700 1 -- 172.21.15.47:0/2023462268 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fedc80b1260 msgr2=0x7fedc80aeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:20.025 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.003+0000 7fedc5d9b700 1 --2- 172.21.15.47:0/2023462268 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fedc80b1260 0x7fedc80aeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:20.025 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.003+0000 7fedc5d9b700 1 -- 172.21.15.47:0/2023462268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedc8157190 msgr2=0x7fedc815a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:20.025 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.003+0000 7fedc5d9b700 1 --2- 172.21.15.47:0/2023462268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedc8157190 0x7fedc815a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:20.025 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.003+0000 7fedc5d9b700 1 -- 172.21.15.47:0/2023462268 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb8009470 con 0x7fedc80b2120 2022-06-29T15:46:20.026 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.003+0000 7fedc5d9b700 1 --2- 172.21.15.47:0/2023462268 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fedc80b2120 0x7fedc8156ca0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7fedbc001800 tx=0x7fedc8060d40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:20.026 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.003+0000 7fedb77fe700 1 -- 172.21.15.47:0/2023462268 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fedb8011b20 con 0x7fedc80b2120 2022-06-29T15:46:20.026 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.003+0000 7fedb77fe700 1 -- 172.21.15.47:0/2023462268 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb8011c80 con 0x7fedc80b2120 2022-06-29T15:46:20.026 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.003+0000 7fedccf36700 1 -- 172.21.15.47:0/2023462268 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedc81448d0 con 0x7fedc80b2120 2022-06-29T15:46:20.027 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.003+0000 7fedb77fe700 1 -- 172.21.15.47:0/2023462268 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fedb801d450 con 0x7fedc80b2120 2022-06-29T15:46:20.027 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.007+0000 7fedccf36700 1 -- 172.21.15.47:0/2023462268 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedc814a4c0 con 0x7fedc80b2120 2022-06-29T15:46:20.027 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.007+0000 7fedccf36700 1 -- 172.21.15.47:0/2023462268 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedc8144d70 con 0x7fedc80b2120 2022-06-29T15:46:20.027 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.007+0000 7fedb77fe700 1 -- 172.21.15.47:0/2023462268 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fedb8024030 con 0x7fedc80b2120 2022-06-29T15:46:20.027 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.007+0000 7fedb77fe700 1 --2- 172.21.15.47:0/2023462268 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7feda4053a70 0x7feda4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:20.028 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.007+0000 7fedb77fe700 1 -- 172.21.15.47:0/2023462268 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fedb806d5b0 con 0x7fedc80b2120 2022-06-29T15:46:20.028 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.007+0000 7fedc659c700 1 --2- 172.21.15.47:0/2023462268 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7feda4053a70 0x7feda4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:20.028 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.007+0000 7fedc659c700 1 --2- 172.21.15.47:0/2023462268 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7feda4053a70 0x7feda4055f30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7fedbc00d000 tx=0x7fedbc000bc0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:20.028 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.015+0000 7fedb77fe700 1 -- 172.21.15.47:0/2023462268 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fedb8044370 con 0x7fedc80b2120 2022-06-29T15:46:20.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.183+0000 7fedccf36700 1 -- 172.21.15.47:0/2023462268 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fedc80573d0 con 0x7fedc80b2120 2022-06-29T15:46:20.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.183+0000 7fedb77fe700 1 -- 172.21.15.47:0/2023462268 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+2492 (secure 0 0 0) 0x7fedb8022090 con 0x7fedc80b2120 2022-06-29T15:46:20.189 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:46:20.189 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":24,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5595,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1798995166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1798995166},{"type":"v1","addr":"172.21.15.47:6835","nonce":1798995166}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21},{"gid":5601,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/1641478219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1641478219},{"type":"v1","addr":"172.21.15.64:6833","nonce":1641478219}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:46:01.294125+0000","modified":"2022-06-29T15:46:02.296567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5607":{"gid":5607,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6837/2427298469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2427298469},{"type":"v1","addr":"172.21.15.47:6837","nonce":2427298469}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2022-06-29T15:46:20.191 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.187+0000 7fedccf36700 1 -- 172.21.15.47:0/2023462268 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7feda4053a70 msgr2=0x7feda4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:20.192 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.187+0000 7fedccf36700 1 --2- 172.21.15.47:0/2023462268 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7feda4053a70 0x7feda4055f30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7fedbc00d000 tx=0x7fedbc000bc0).stop 2022-06-29T15:46:20.192 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.187+0000 7fedccf36700 1 -- 172.21.15.47:0/2023462268 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fedc80b2120 msgr2=0x7fedc8156ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:20.193 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.187+0000 7fedccf36700 1 --2- 172.21.15.47:0/2023462268 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fedc80b2120 0x7fedc8156ca0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7fedbc001800 tx=0x7fedc8060d40).stop 2022-06-29T15:46:20.193 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.187+0000 7fedccf36700 1 -- 172.21.15.47:0/2023462268 shutdown_connections 2022-06-29T15:46:20.193 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.187+0000 7fedccf36700 1 --2- 172.21.15.47:0/2023462268 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7feda4053a70 0x7feda4055f30 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:20.194 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.187+0000 7fedccf36700 1 --2- 172.21.15.47:0/2023462268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedc8157190 0x7fedc815a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:20.194 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.187+0000 7fedccf36700 1 --2- 172.21.15.47:0/2023462268 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fedc80b2120 0x7fedc8156ca0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:20.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.187+0000 7fedccf36700 1 --2- 172.21.15.47:0/2023462268 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fedc80b1260 0x7fedc80aeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:20.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.187+0000 7fedccf36700 1 -- 172.21.15.47:0/2023462268 >> 172.21.15.47:0/2023462268 conn(0x7fedc80626f0 msgr2=0x7fedc8145da0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:20.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.187+0000 7fedccf36700 1 -- 172.21.15.47:0/2023462268 shutdown_connections 2022-06-29T15:46:20.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.187+0000 7fedccf36700 1 -- 172.21.15.47:0/2023462268 wait complete. 2022-06-29T15:46:20.198 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 24 2022-06-29T15:46:20.204 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a cache drop 5 2022-06-29T15:46:20.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.311+0000 7f1304af2700 1 Processor -- start 2022-06-29T15:46:20.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.311+0000 7f1304af2700 1 -- start start 2022-06-29T15:46:20.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.311+0000 7f1304af2700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f13000605d0 0x7f130009a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:20.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.311+0000 7f1304af2700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f130009b970 con 0x7f13000af250 2022-06-29T15:46:20.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.311+0000 7f1304af2700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f130005fbe0 con 0x7f130005f0b0 2022-06-29T15:46:20.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.311+0000 7f1304af2700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13000acfd0 con 0x7f13000605d0 2022-06-29T15:46:20.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.311+0000 7f12fffff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f13000af250 0x7f13000af6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34316/0 (socket says 172.21.15.47:34316) 2022-06-29T15:46:20.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.311+0000 7f12fffff700 1 -- 172.21.15.47:0/2543665635 learned_addr learned my addr 172.21.15.47:0/2543665635 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:46:20.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.311+0000 7f12ff7fe700 1 --2- 172.21.15.47:0/2543665635 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f13000605d0 0x7f130009a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:20.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.311+0000 7f12fe7fc700 1 -- 172.21.15.47:0/2543665635 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2537767531 0 0) 0x7f130009b970 con 0x7f13000af250 2022-06-29T15:46:20.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.311+0000 7f12fe7fc700 1 -- 172.21.15.47:0/2543665635 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12ec003260 con 0x7f13000af250 2022-06-29T15:46:20.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.311+0000 7f12fe7fc700 1 -- 172.21.15.47:0/2543665635 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 637104433 0 0) 0x7f130005fbe0 con 0x7f130005f0b0 2022-06-29T15:46:20.335 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.311+0000 7f12fe7fc700 1 -- 172.21.15.47:0/2543665635 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f130009b970 con 0x7f130005f0b0 2022-06-29T15:46:20.335 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.311+0000 7f12fe7fc700 1 -- 172.21.15.47:0/2543665635 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1225768284 0 0) 0x7f12ec003260 con 0x7f13000af250 2022-06-29T15:46:20.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.311+0000 7f12fe7fc700 1 -- 172.21.15.47:0/2543665635 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f13000605d0 msgr2=0x7f130009a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:46:20.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.311+0000 7f12fe7fc700 1 --2- 172.21.15.47:0/2543665635 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f13000605d0 0x7f130009a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:20.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.311+0000 7f12fe7fc700 1 -- 172.21.15.47:0/2543665635 >> v1:172.21.15.64:6789/0 conn(0x7f130005f0b0 legacy=0x7f130009a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:20.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.311+0000 7f12fe7fc700 1 -- 172.21.15.47:0/2543665635 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f130005fe80 con 0x7f13000af250 2022-06-29T15:46:20.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.311+0000 7f12fe7fc700 1 -- 172.21.15.47:0/2543665635 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7f12f80019e0 con 0x7f13000af250 2022-06-29T15:46:20.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.311+0000 7f12ff7fe700 1 --2- 172.21.15.47:0/2543665635 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f13000605d0 0x7f130009a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:46:20.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.311+0000 7f12fe7fc700 1 -- 172.21.15.47:0/2543665635 >> v1:172.21.15.47:6789/0 conn(0x7f13000af250 legacy=0x7f13000af6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:20.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.311+0000 7f12fe7fc700 1 --2- 172.21.15.47:0/2543665635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12ec003910 0x7f12ec003d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:20.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.311+0000 7f12fe7fc700 1 --2- 172.21.15.47:0/2543665635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12ec004500 0x7f12ec004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:20.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.311+0000 7f12fe7fc700 1 --2- 172.21.15.47:0/2543665635 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f13000605d0 0x7f12ec004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:20.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.311+0000 7f12fe7fc700 1 -- 172.21.15.47:0/2543665635 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13000acfd0 con 0x7f12ec003910 2022-06-29T15:46:20.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.311+0000 7f12fe7fc700 1 -- 172.21.15.47:0/2543665635 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12ec00b750 con 0x7f12ec004500 2022-06-29T15:46:20.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.311+0000 7f12fe7fc700 1 -- 172.21.15.47:0/2543665635 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12ec00ca20 con 0x7f13000605d0 2022-06-29T15:46:20.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.315+0000 7f12ff7fe700 1 --2- 172.21.15.47:0/2543665635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12ec003910 0x7f12ec003d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:20.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.315+0000 7f12feffd700 1 --2- 172.21.15.47:0/2543665635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12ec004500 0x7f12ec004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:20.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.315+0000 7f12fffff700 1 --2- 172.21.15.47:0/2543665635 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f13000605d0 0x7f12ec004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:20.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.315+0000 7f12ff7fe700 1 -- 172.21.15.47:0/2543665635 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f13000605d0 msgr2=0x7f12ec004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:46:20.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.315+0000 7f12ff7fe700 1 --2- 172.21.15.47:0/2543665635 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f13000605d0 0x7f12ec004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:20.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.315+0000 7f12ff7fe700 1 -- 172.21.15.47:0/2543665635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12ec004500 msgr2=0x7f12ec004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:46:20.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.315+0000 7f12ff7fe700 1 --2- 172.21.15.47:0/2543665635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12ec004500 0x7f12ec004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:20.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.315+0000 7f12ff7fe700 1 -- 172.21.15.47:0/2543665635 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f000b180 con 0x7f12ec003910 2022-06-29T15:46:20.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.315+0000 7f12fffff700 1 --2- 172.21.15.47:0/2543665635 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f13000605d0 0x7f12ec004e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:46:20.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.315+0000 7f12ff7fe700 1 --2- 172.21.15.47:0/2543665635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12ec003910 0x7f12ec003d10 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7f12f000c000 tx=0x7f12f0006df0).ready entity=mon.0 client_cookie=e3e5ab0b66edce14 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:20.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.315+0000 7f12fe7fc700 1 -- 172.21.15.47:0/2543665635 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12f000e830 con 0x7f12ec003910 2022-06-29T15:46:20.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.315+0000 7f12fe7fc700 1 -- 172.21.15.47:0/2543665635 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f000e990 con 0x7f12ec003910 2022-06-29T15:46:20.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.315+0000 7f12fe7fc700 1 -- 172.21.15.47:0/2543665635 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12f000ecf0 con 0x7f12ec003910 2022-06-29T15:46:20.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.315+0000 7f1304af2700 1 -- 172.21.15.47:0/2543665635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12ec003910 msgr2=0x7f12ec003d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:20.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.315+0000 7f1304af2700 1 --2- 172.21.15.47:0/2543665635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12ec003910 0x7f12ec003d10 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7f12f000c000 tx=0x7f12f0006df0).stop 2022-06-29T15:46:20.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.315+0000 7f12fffff700 1 -- 172.21.15.47:0/2543665635 reap_dead start 2022-06-29T15:46:20.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.315+0000 7f1304af2700 1 -- 172.21.15.47:0/2543665635 shutdown_connections 2022-06-29T15:46:20.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.315+0000 7f1304af2700 1 -- 172.21.15.47:0/2543665635 >> 172.21.15.47:0/2543665635 conn(0x7f1300062df0 msgr2=0x7f13000631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:20.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.315+0000 7f1304af2700 1 -- 172.21.15.47:0/2543665635 shutdown_connections 2022-06-29T15:46:20.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.315+0000 7f1304af2700 1 -- 172.21.15.47:0/2543665635 wait complete. 2022-06-29T15:46:20.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.319+0000 7f1304af2700 1 Processor -- start 2022-06-29T15:46:20.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.319+0000 7f1304af2700 1 -- start start 2022-06-29T15:46:20.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.319+0000 7f1304af2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12ec003910 0x7f1300090940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:20.352 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.319+0000 7f1304af2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13000903c0 0x7f130014e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:20.352 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.319+0000 7f1304af2700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f130014e570 0x7f1300091c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:20.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.319+0000 7f1304af2700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1300092de0 con 0x7f12ec003910 2022-06-29T15:46:20.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.319+0000 7f1304af2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13000940f0 con 0x7f13000903c0 2022-06-29T15:46:20.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.319+0000 7f1304af2700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1300093850 con 0x7f130014e570 2022-06-29T15:46:20.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.319+0000 7f12ff7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12ec003910 0x7f1300090940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:20.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.319+0000 7f12ff7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12ec003910 0x7f1300090940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48772/0 (socket says 172.21.15.47:48772) 2022-06-29T15:46:20.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.319+0000 7f12ff7fe700 1 -- 172.21.15.47:0/2913123753 learned_addr learned my addr 172.21.15.47:0/2913123753 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:46:20.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.319+0000 7f12feffd700 1 --2- 172.21.15.47:0/2913123753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13000903c0 0x7f130014e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:20.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.319+0000 7f12fffff700 1 --2- 172.21.15.47:0/2913123753 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f130014e570 0x7f1300091c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:20.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.319+0000 7f12ff7fe700 1 -- 172.21.15.47:0/2913123753 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f130014e570 msgr2=0x7f1300091c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:20.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.319+0000 7f12ff7fe700 1 --2- 172.21.15.47:0/2913123753 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f130014e570 0x7f1300091c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:20.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.319+0000 7f12ff7fe700 1 -- 172.21.15.47:0/2913123753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13000903c0 msgr2=0x7f130014e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:20.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.319+0000 7f12ff7fe700 1 --2- 172.21.15.47:0/2913123753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13000903c0 0x7f130014e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:20.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.319+0000 7f12ff7fe700 1 -- 172.21.15.47:0/2913123753 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f130014ea00 con 0x7f12ec003910 2022-06-29T15:46:20.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.323+0000 7f12ff7fe700 1 --2- 172.21.15.47:0/2913123753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12ec003910 0x7f1300090940 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7f12f000c000 tx=0x7f12f0019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:20.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.323+0000 7f12fcff9700 1 -- 172.21.15.47:0/2913123753 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12f0022070 con 0x7f12ec003910 2022-06-29T15:46:20.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.323+0000 7f12fcff9700 1 -- 172.21.15.47:0/2913123753 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f0000e90 con 0x7f12ec003910 2022-06-29T15:46:20.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.323+0000 7f1304af2700 1 -- 172.21.15.47:0/2913123753 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1300092b40 con 0x7f12ec003910 2022-06-29T15:46:20.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.323+0000 7f12fcff9700 1 -- 172.21.15.47:0/2913123753 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12f0026600 con 0x7f12ec003910 2022-06-29T15:46:20.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.323+0000 7f1304af2700 1 -- 172.21.15.47:0/2913123753 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1300094580 con 0x7f12ec003910 2022-06-29T15:46:20.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.323+0000 7f12fcff9700 1 -- 172.21.15.47:0/2913123753 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f12f0026760 con 0x7f12ec003910 2022-06-29T15:46:20.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.323+0000 7f12fcff9700 1 --2- 172.21.15.47:0/2913123753 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f12e0053a70 0x7f12e0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:20.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.323+0000 7f12fcff9700 1 -- 172.21.15.47:0/2913123753 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f12f006f870 con 0x7f12ec003910 2022-06-29T15:46:20.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.323+0000 7f12feffd700 1 --2- 172.21.15.47:0/2913123753 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f12e0053a70 0x7f12e0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:20.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.327+0000 7f12feffd700 1 --2- 172.21.15.47:0/2913123753 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f12e0053a70 0x7f12e0055f30 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f12f4003000 tx=0x7f12f4007a50).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:20.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.327+0000 7f130ffa9740 1 Processor -- start 2022-06-29T15:46:20.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.327+0000 7f130ffa9740 1 -- start start 2022-06-29T15:46:20.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.327+0000 7f130ffa9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1161f10 0x119a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:20.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.327+0000 7f130ffa9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x1196960 0x1196de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:20.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.327+0000 7f130ffa9740 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x11972d0 0x115ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:20.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.327+0000 7f130ffa9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x10c12d0 con 0x1196960 2022-06-29T15:46:20.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.327+0000 7f130ffa9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x10d4ef0 con 0x1161f10 2022-06-29T15:46:20.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.327+0000 7f130ffa9740 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x10b51a0 con 0x11972d0 2022-06-29T15:46:20.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.327+0000 7f12feffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x1196960 0x1196de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:20.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.327+0000 7f12feffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x1196960 0x1196de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48780/0 (socket says 172.21.15.47:48780) 2022-06-29T15:46:20.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.327+0000 7f12feffd700 1 -- 172.21.15.47:0/2234460975 learned_addr learned my addr 172.21.15.47:0/2234460975 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:46:20.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.327+0000 7f12ff7fe700 1 --2- 172.21.15.47:0/2234460975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1161f10 0x119a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:20.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.327+0000 7f12fffff700 1 --2- 172.21.15.47:0/2234460975 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x11972d0 0x115ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:20.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.327+0000 7f12feffd700 1 -- 172.21.15.47:0/2234460975 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x11972d0 msgr2=0x115ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:46:20.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.327+0000 7f12feffd700 1 --2- 172.21.15.47:0/2234460975 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x11972d0 0x115ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:20.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.327+0000 7f12feffd700 1 -- 172.21.15.47:0/2234460975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1161f10 msgr2=0x119a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:46:20.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.327+0000 7f12feffd700 1 --2- 172.21.15.47:0/2234460975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1161f10 0x119a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:20.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.327+0000 7f12feffd700 1 -- 172.21.15.47:0/2234460975 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x110cdf0 con 0x1196960 2022-06-29T15:46:20.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.327+0000 7f12feffd700 1 --2- 172.21.15.47:0/2234460975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x1196960 0x1196de0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x1163d80 tx=0x7f12f4016000).ready entity=mon.0 client_cookie=90156e229d8fdccf server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:20.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.331+0000 7f1304af2700 1 -- 172.21.15.47:0/2234460975 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12f4015720 con 0x1196960 2022-06-29T15:46:20.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.331+0000 7f1304af2700 1 -- 172.21.15.47:0/2234460975 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f4015880 con 0x1196960 2022-06-29T15:46:20.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.331+0000 7f1304af2700 1 -- 172.21.15.47:0/2234460975 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12f4015be0 con 0x1196960 2022-06-29T15:46:20.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.331+0000 7f130ffa9740 1 -- 172.21.15.47:0/2234460975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x1196960 msgr2=0x1196de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:20.394 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.331+0000 7f130ffa9740 1 --2- 172.21.15.47:0/2234460975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x1196960 0x1196de0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x1163d80 tx=0x7f12f4016000).stop 2022-06-29T15:46:20.394 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.331+0000 7f130ffa9740 1 -- 172.21.15.47:0/2234460975 shutdown_connections 2022-06-29T15:46:20.394 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.331+0000 7f130ffa9740 1 --2- 172.21.15.47:0/2234460975 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x11972d0 0x115ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:20.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.331+0000 7f130ffa9740 1 --2- 172.21.15.47:0/2234460975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x1196960 0x1196de0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:20.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.331+0000 7f130ffa9740 1 --2- 172.21.15.47:0/2234460975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1161f10 0x119a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:20.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.331+0000 7f130ffa9740 1 -- 172.21.15.47:0/2234460975 >> 172.21.15.47:0/2234460975 conn(0x1160b10 msgr2=0x1162930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:20.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.331+0000 7f130ffa9740 1 -- 172.21.15.47:0/2234460975 shutdown_connections 2022-06-29T15:46:20.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.331+0000 7f130ffa9740 1 -- 172.21.15.47:0/2234460975 wait complete. 2022-06-29T15:46:20.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.331+0000 7f130ffa9740 1 -- start start 2022-06-29T15:46:20.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.331+0000 7f130ffa9740 1 Processor -- start 2022-06-29T15:46:20.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.331+0000 7f130ffa9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1196960 0x11a9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:20.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.331+0000 7f130ffa9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x11972d0 0x118cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:20.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.331+0000 7f130ffa9740 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x119c6a0 0x1169690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:20.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.331+0000 7f130ffa9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x116aba0 con 0x11972d0 2022-06-29T15:46:20.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.331+0000 7f130ffa9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x11a6330 con 0x1196960 2022-06-29T15:46:20.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.331+0000 7f130ffa9740 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x118c650 con 0x119c6a0 2022-06-29T15:46:20.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.331+0000 7f130ffa9740 10 client.0 ll_register_callbacks cb 0x111c3d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2022-06-29T15:46:20.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.331+0000 7f12feffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x11972d0 0x118cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:20.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.331+0000 7f12feffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x11972d0 0x118cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48788/0 (socket says 172.21.15.47:48788) 2022-06-29T15:46:20.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.331+0000 7f12feffd700 1 -- 172.21.15.47:0/4036923996 learned_addr learned my addr 172.21.15.47:0/4036923996 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:46:20.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.331+0000 7f12ff7fe700 1 --2- 172.21.15.47:0/4036923996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1196960 0x11a9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:20.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.331+0000 7f12fffff700 1 --2- 172.21.15.47:0/4036923996 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x119c6a0 0x1169690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:20.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.331+0000 7f12feffd700 1 -- 172.21.15.47:0/4036923996 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x119c6a0 msgr2=0x1169690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:46:20.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.331+0000 7f12feffd700 1 --2- 172.21.15.47:0/4036923996 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x119c6a0 0x1169690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:20.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.331+0000 7f12feffd700 1 -- 172.21.15.47:0/4036923996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1196960 msgr2=0x11a9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:46:20.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.331+0000 7f12feffd700 1 --2- 172.21.15.47:0/4036923996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1196960 0x11a9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:20.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.331+0000 7f12feffd700 1 -- 172.21.15.47:0/4036923996 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f130009b970 con 0x11972d0 2022-06-29T15:46:20.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.331+0000 7f12feffd700 1 --2- 172.21.15.47:0/4036923996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x11972d0 0x118cfc0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x116b300 tx=0x7f12f4023000).ready entity=mon.0 client_cookie=23f0d74ba1ca867d server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:20.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.331+0000 7f12de7fc700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2022-06-29T15:46:20.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.335+0000 7f12de7fc700 1 -- 172.21.15.47:0/4036923996 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12f40036e0 con 0x11972d0 2022-06-29T15:46:20.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.335+0000 7f12de7fc700 1 -- 172.21.15.47:0/4036923996 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f12f401a030 con 0x11972d0 2022-06-29T15:46:20.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.335+0000 7f130ffa9740 1 -- 172.21.15.47:0/4036923996 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x10aa6c0 con 0x11972d0 2022-06-29T15:46:20.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.335+0000 7f12de7fc700 1 -- 172.21.15.47:0/4036923996 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12f4015720 con 0x11972d0 2022-06-29T15:46:20.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.335+0000 7f130ffa9740 10 client.5793 fetch_fsmap learned FSMap version 24 2022-06-29T15:46:20.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.335+0000 7f130ffa9740 1 -- 172.21.15.47:0/4036923996 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x10c43d0 con 0x11972d0 2022-06-29T15:46:20.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.335+0000 7f12de7fc700 1 -- 172.21.15.47:0/4036923996 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x10aa6c0 con 0x11972d0 2022-06-29T15:46:20.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.335+0000 7f12de7fc700 1 -- 172.21.15.47:0/4036923996 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 24) v1 ==== 1434+0+0 (secure 0 0 0) 0x7f12f4028070 con 0x11972d0 2022-06-29T15:46:20.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.335+0000 7f130ffa9740 10 client.5793 fetch_fsmap finished waiting for FSMap version 24 2022-06-29T15:46:20.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.335+0000 7f130ffa9740 10 client.5793 resolve_mds: resolved ID 'a' to GID 5607 2022-06-29T15:46:20.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.335+0000 7f130ffa9740 20 client.5793 populate_metadata read hostname 'smithi047' 2022-06-29T15:46:20.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.335+0000 7f130ffa9740 1 --2- 172.21.15.47:0/4036923996 >> [v2:172.21.15.47:6836/2427298469,v1:172.21.15.47:6837/2427298469] conn(0x11a85e0 0x11a89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:20.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.335+0000 7f130ffa9740 4 client.5793 mds_command: new command op to 5607 tid=0[{"prefix": "get_command_descriptions"}] 2022-06-29T15:46:20.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.335+0000 7f130ffa9740 1 -- 172.21.15.47:0/4036923996 --> [v2:172.21.15.47:6836/2427298469,v1:172.21.15.47:6837/2427298469] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x10cade0 con 0x11a85e0 2022-06-29T15:46:20.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.335+0000 7f12ff7fe700 1 --2- 172.21.15.47:0/4036923996 >> [v2:172.21.15.47:6836/2427298469,v1:172.21.15.47:6837/2427298469] conn(0x11a85e0 0x11a89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:20.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.335+0000 7f12de7fc700 10 client.5793 ms_handle_connect on v2:172.21.15.47:6836/2427298469 2022-06-29T15:46:20.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.335+0000 7f12ff7fe700 1 --2- 172.21.15.47:0/4036923996 >> [v2:172.21.15.47:6836/2427298469,v1:172.21.15.47:6837/2427298469] conn(0x11a85e0 0x11a89c0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=92e114d9a2bdb593 server_cookie=4e260a2b8e3489d5 in_seq=0 out_seq=0 2022-06-29T15:46:20.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.335+0000 7f12de7fc700 1 -- 172.21.15.47:0/4036923996 <== mds.0 v2:172.21.15.47:6836/2427298469 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x10cade0 con 0x11a85e0 2022-06-29T15:46:20.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.335+0000 7f12de7fc700 10 client.5793 handle_command_reply: tid=0 2022-06-29T15:46:20.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.335+0000 7f130ffa9740 1 client.5793 shutdown 2022-06-29T15:46:20.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.335+0000 7f130ffa9740 1 -- 172.21.15.47:0/4036923996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x11972d0 msgr2=0x118cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:20.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.335+0000 7f130ffa9740 1 --2- 172.21.15.47:0/4036923996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x11972d0 0x118cfc0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x116b300 tx=0x7f12f4023000).stop 2022-06-29T15:46:20.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.335+0000 7f130ffa9740 1 -- 172.21.15.47:0/4036923996 shutdown_connections 2022-06-29T15:46:20.409 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.335+0000 7f130ffa9740 1 --2- 172.21.15.47:0/4036923996 >> [v2:172.21.15.47:6836/2427298469,v1:172.21.15.47:6837/2427298469] conn(0x11a85e0 0x11a89c0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:20.409 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.335+0000 7f130ffa9740 1 --2- 172.21.15.47:0/4036923996 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x119c6a0 0x1169690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:20.410 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.335+0000 7f130ffa9740 1 --2- 172.21.15.47:0/4036923996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x11972d0 0x118cfc0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:20.410 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.335+0000 7f130ffa9740 1 --2- 172.21.15.47:0/4036923996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1196960 0x11a9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:20.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.335+0000 7f130ffa9740 1 -- 172.21.15.47:0/4036923996 >> 172.21.15.47:0/4036923996 conn(0x1161f10 msgr2=0x1160f00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:20.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.335+0000 7f12de7fc700 0 client.5793 ms_handle_reset on v2:172.21.15.47:6836/2427298469 2022-06-29T15:46:20.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.335+0000 7f130ffa9740 1 -- 172.21.15.47:0/4036923996 shutdown_connections 2022-06-29T15:46:20.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.335+0000 7f130ffa9740 1 -- 172.21.15.47:0/4036923996 wait complete. 2022-06-29T15:46:20.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.335+0000 7f130ffa9740 20 client.5793 trim_cache size 0 max 16384 2022-06-29T15:46:20.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.351+0000 7f130ffa9740 1 Processor -- start 2022-06-29T15:46:20.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.351+0000 7f130ffa9740 1 -- start start 2022-06-29T15:46:20.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.351+0000 7f130ffa9740 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x1196960 0x1166160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:20.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.351+0000 7f130ffa9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x11972d0 0x1166650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:20.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.351+0000 7f130ffa9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x119c6a0 0x1198e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:20.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.351+0000 7f130ffa9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x10a60b0 con 0x119c6a0 2022-06-29T15:46:20.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.351+0000 7f130ffa9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x10aa6c0 con 0x11972d0 2022-06-29T15:46:20.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.351+0000 7f130ffa9740 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x1162e70 con 0x1196960 2022-06-29T15:46:20.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.351+0000 7f12fffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x119c6a0 0x1198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:20.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.351+0000 7f12feffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x11972d0 0x1166650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:20.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.351+0000 7f12ff7fe700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x1196960 0x1166160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:20.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.351+0000 7f12fffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x119c6a0 0x1198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48796/0 (socket says 172.21.15.47:48796) 2022-06-29T15:46:20.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.351+0000 7f12fffff700 1 -- 172.21.15.47:0/2878403753 learned_addr learned my addr 172.21.15.47:0/2878403753 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:46:20.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.351+0000 7f12fffff700 1 -- 172.21.15.47:0/2878403753 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x1196960 msgr2=0x1166160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:46:20.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.351+0000 7f12fffff700 1 --2- 172.21.15.47:0/2878403753 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x1196960 0x1166160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:20.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.351+0000 7f12fffff700 1 -- 172.21.15.47:0/2878403753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x11972d0 msgr2=0x1166650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:46:20.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.351+0000 7f12fffff700 1 --2- 172.21.15.47:0/2878403753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x11972d0 0x1166650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:20.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.351+0000 7f12fffff700 1 -- 172.21.15.47:0/2878403753 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x10ccfc0 con 0x119c6a0 2022-06-29T15:46:20.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.351+0000 7f12fffff700 1 --2- 172.21.15.47:0/2878403753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x119c6a0 0x1198e90 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7f12f8001210 tx=0x7f12f8009040).ready entity=mon.0 client_cookie=e0b8e39491ea7f12 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:20.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.355+0000 7f12e4ff9700 1 -- 172.21.15.47:0/2878403753 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12f801c070 con 0x119c6a0 2022-06-29T15:46:20.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.355+0000 7f12e4ff9700 1 -- 172.21.15.47:0/2878403753 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f8004a90 con 0x119c6a0 2022-06-29T15:46:20.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.355+0000 7f12e4ff9700 1 -- 172.21.15.47:0/2878403753 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12f8004df0 con 0x119c6a0 2022-06-29T15:46:20.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.355+0000 7f130ffa9740 1 -- 172.21.15.47:0/2878403753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x119c6a0 msgr2=0x1198e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:20.424 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.355+0000 7f130ffa9740 1 --2- 172.21.15.47:0/2878403753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x119c6a0 0x1198e90 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7f12f8001210 tx=0x7f12f8009040).stop 2022-06-29T15:46:20.424 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.355+0000 7f130ffa9740 1 -- 172.21.15.47:0/2878403753 shutdown_connections 2022-06-29T15:46:20.425 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.355+0000 7f130ffa9740 1 --2- 172.21.15.47:0/2878403753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x119c6a0 0x1198e90 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:20.425 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.355+0000 7f130ffa9740 1 --2- 172.21.15.47:0/2878403753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x11972d0 0x1166650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:20.426 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.355+0000 7f130ffa9740 1 --2- 172.21.15.47:0/2878403753 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x1196960 0x1166160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:20.426 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.355+0000 7f130ffa9740 1 -- 172.21.15.47:0/2878403753 >> 172.21.15.47:0/2878403753 conn(0x1161f10 msgr2=0x115f980 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:20.427 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.355+0000 7f130ffa9740 1 -- 172.21.15.47:0/2878403753 shutdown_connections 2022-06-29T15:46:20.427 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.355+0000 7f130ffa9740 1 -- 172.21.15.47:0/2878403753 wait complete. 2022-06-29T15:46:20.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.355+0000 7f130ffa9740 1 -- start start 2022-06-29T15:46:20.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.355+0000 7f130ffa9740 1 Processor -- start 2022-06-29T15:46:20.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.355+0000 7f130ffa9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x1196960 0x1200ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:20.429 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.355+0000 7f130ffa9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x11972d0 0x11f9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:20.429 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.355+0000 7f130ffa9740 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x119c6a0 0x11fa280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:20.430 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.355+0000 7f130ffa9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x11a0fe0 con 0x1196960 2022-06-29T15:46:20.430 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.355+0000 7f130ffa9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x1197040 con 0x11972d0 2022-06-29T15:46:20.431 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.355+0000 7f130ffa9740 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x11604e0 con 0x119c6a0 2022-06-29T15:46:20.431 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.355+0000 7f130ffa9740 10 client.0 ll_register_callbacks cb 0x11ac840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2022-06-29T15:46:20.432 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.359+0000 7f12ff7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x1196960 0x1200ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:20.432 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.359+0000 7f12ff7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x1196960 0x1200ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48800/0 (socket says 172.21.15.47:48800) 2022-06-29T15:46:20.433 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.359+0000 7f12fffff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x119c6a0 0x11fa280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:20.433 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.359+0000 7f12ff7fe700 1 -- 172.21.15.47:0/318075103 learned_addr learned my addr 172.21.15.47:0/318075103 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:46:20.434 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.359+0000 7f12feffd700 1 --2- 172.21.15.47:0/318075103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x11972d0 0x11f9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:20.434 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.359+0000 7f12ff7fe700 1 -- 172.21.15.47:0/318075103 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x119c6a0 msgr2=0x11fa280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:46:20.434 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.359+0000 7f12ff7fe700 1 --2- 172.21.15.47:0/318075103 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x119c6a0 0x11fa280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:20.435 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.359+0000 7f12ff7fe700 1 -- 172.21.15.47:0/318075103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x11972d0 msgr2=0x11f9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:46:20.435 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.359+0000 7f12ff7fe700 1 --2- 172.21.15.47:0/318075103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x11972d0 0x11f9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:20.436 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.359+0000 7f12ff7fe700 1 -- 172.21.15.47:0/318075103 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12f800f040 con 0x1196960 2022-06-29T15:46:20.436 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.359+0000 7f12ff7fe700 1 --2- 172.21.15.47:0/318075103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x1196960 0x1200ac0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x1197180 tx=0x7f12f0049040).ready entity=mon.0 client_cookie=6a59b2fb2485e2e3 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:20.437 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.359+0000 7f12df7fe700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2022-06-29T15:46:20.437 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.359+0000 7f12df7fe700 1 -- 172.21.15.47:0/318075103 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12f003e120 con 0x1196960 2022-06-29T15:46:20.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.359+0000 7f12df7fe700 1 -- 172.21.15.47:0/318075103 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f12f0030d80 con 0x1196960 2022-06-29T15:46:20.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.359+0000 7f130ffa9740 1 -- 172.21.15.47:0/318075103 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f12f4015880 con 0x1196960 2022-06-29T15:46:20.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.359+0000 7f12df7fe700 1 -- 172.21.15.47:0/318075103 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12f0049fc0 con 0x1196960 2022-06-29T15:46:20.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.359+0000 7f130ffa9740 10 client.5799 fetch_fsmap learned FSMap version 24 2022-06-29T15:46:20.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.359+0000 7f130ffa9740 1 -- 172.21.15.47:0/318075103 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x10c99b0 con 0x1196960 2022-06-29T15:46:20.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.359+0000 7f12df7fe700 1 -- 172.21.15.47:0/318075103 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f4015880 con 0x1196960 2022-06-29T15:46:20.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.359+0000 7f12df7fe700 1 -- 172.21.15.47:0/318075103 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 24) v1 ==== 1434+0+0 (secure 0 0 0) 0x7f12f001d0d0 con 0x1196960 2022-06-29T15:46:20.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.359+0000 7f130ffa9740 10 client.5799 fetch_fsmap finished waiting for FSMap version 24 2022-06-29T15:46:20.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.359+0000 7f130ffa9740 10 client.5799 resolve_mds: resolved ID 'a' to GID 5607 2022-06-29T15:46:20.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.359+0000 7f130ffa9740 20 client.5799 populate_metadata read hostname 'smithi047' 2022-06-29T15:46:20.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.359+0000 7f130ffa9740 1 --2- 172.21.15.47:0/318075103 >> [v2:172.21.15.47:6836/2427298469,v1:172.21.15.47:6837/2427298469] conn(0x11a1ea0 0x1203600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:20.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.359+0000 7f130ffa9740 4 client.5799 mds_command: new command op to 5607 tid=0[{"prefix": "cache drop", "timeout": 5}] 2022-06-29T15:46:20.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.359+0000 7f130ffa9740 1 -- 172.21.15.47:0/318075103 --> [v2:172.21.15.47:6836/2427298469,v1:172.21.15.47:6837/2427298469] -- command(tid 0: {"prefix": "cache drop", "timeout": 5}) v1 -- 0x10c2e60 con 0x11a1ea0 2022-06-29T15:46:20.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.359+0000 7f12feffd700 1 --2- 172.21.15.47:0/318075103 >> [v2:172.21.15.47:6836/2427298469,v1:172.21.15.47:6837/2427298469] conn(0x11a1ea0 0x1203600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:20.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.359+0000 7f12feffd700 1 --2- 172.21.15.47:0/318075103 >> [v2:172.21.15.47:6836/2427298469,v1:172.21.15.47:6837/2427298469] conn(0x11a1ea0 0x1203600 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=896fdaabf73dd3f server_cookie=5ea85e1b84d4ae24 in_seq=0 out_seq=0 2022-06-29T15:46:20.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:20.359+0000 7f12df7fe700 10 client.5799 ms_handle_connect on v2:172.21.15.47:6836/2427298469 2022-06-29T15:46:25.556 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.551+0000 7f12df7fe700 1 -- 172.21.15.47:0/318075103 <== mds.0 v2:172.21.15.47:6836/2427298469 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+404 (crc 0 0 0) 0x10c2e60 con 0x11a1ea0 2022-06-29T15:46:25.557 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.551+0000 7f12df7fe700 10 client.5799 handle_command_reply: tid=0 2022-06-29T15:46:25.557 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.551+0000 7f130ffa9740 1 client.5799 shutdown 2022-06-29T15:46:25.557 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.551+0000 7f130ffa9740 1 -- 172.21.15.47:0/318075103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x1196960 msgr2=0x1200ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:25.558 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.551+0000 7f130ffa9740 1 --2- 172.21.15.47:0/318075103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x1196960 0x1200ac0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x1197180 tx=0x7f12f0049040).stop 2022-06-29T15:46:25.558 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.551+0000 7f130ffa9740 1 -- 172.21.15.47:0/318075103 shutdown_connections 2022-06-29T15:46:25.558 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.551+0000 7f130ffa9740 1 --2- 172.21.15.47:0/318075103 >> [v2:172.21.15.47:6836/2427298469,v1:172.21.15.47:6837/2427298469] conn(0x11a1ea0 0x1203600 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:25.559 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.551+0000 7f130ffa9740 1 --2- 172.21.15.47:0/318075103 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x119c6a0 0x11fa280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:25.559 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.551+0000 7f130ffa9740 1 --2- 172.21.15.47:0/318075103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x11972d0 0x11f9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:25.559 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.551+0000 7f12df7fe700 0 client.5799 ms_handle_reset on v2:172.21.15.47:6836/2427298469 2022-06-29T15:46:25.560 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.551+0000 7f130ffa9740 1 --2- 172.21.15.47:0/318075103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x1196960 0x1200ac0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:25.560 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.551+0000 7f130ffa9740 1 -- 172.21.15.47:0/318075103 >> 172.21.15.47:0/318075103 conn(0x1161f10 msgr2=0x1165e20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:25.560 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.551+0000 7f130ffa9740 1 -- 172.21.15.47:0/318075103 shutdown_connections 2022-06-29T15:46:25.560 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.551+0000 7f130ffa9740 1 -- 172.21.15.47:0/318075103 wait complete. 2022-06-29T15:46:25.561 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.551+0000 7f130ffa9740 20 client.5799 trim_cache size 0 max 16384 2022-06-29T15:46:25.561 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2022-06-29T15:46:25.562 INFO:teuthology.orchestra.run.smithi047.stdout: "client_recall": { 2022-06-29T15:46:25.562 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": -110, 2022-06-29T15:46:25.562 INFO:teuthology.orchestra.run.smithi047.stdout: "message": "(110) Connection timed out", 2022-06-29T15:46:25.563 INFO:teuthology.orchestra.run.smithi047.stdout: "recalled": 120 2022-06-29T15:46:25.563 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:46:25.563 INFO:teuthology.orchestra.run.smithi047.stdout: "flush_journal": { 2022-06-29T15:46:25.563 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2022-06-29T15:46:25.564 INFO:teuthology.orchestra.run.smithi047.stdout: "message": "" 2022-06-29T15:46:25.564 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:46:25.564 INFO:teuthology.orchestra.run.smithi047.stdout: "trim_cache": { 2022-06-29T15:46:25.564 INFO:teuthology.orchestra.run.smithi047.stdout: "trimmed": 0 2022-06-29T15:46:25.564 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:46:25.565 INFO:teuthology.orchestra.run.smithi047.stdout: "cache": { 2022-06-29T15:46:25.565 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": { 2022-06-29T15:46:25.565 INFO:teuthology.orchestra.run.smithi047.stdout: "items": 19985, 2022-06-29T15:46:25.565 INFO:teuthology.orchestra.run.smithi047.stdout: "bytes": 3099360 2022-06-29T15:46:25.566 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:46:25.566 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:46:25.566 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 5.1909452280000004 2022-06-29T15:46:25.566 INFO:teuthology.orchestra.run.smithi047.stdout:} 2022-06-29T15:46:25.568 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.551+0000 7f1304af2700 1 -- 172.21.15.47:0/2913123753 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f12e0053a70 msgr2=0x7f12e0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:25.569 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.551+0000 7f1304af2700 1 --2- 172.21.15.47:0/2913123753 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f12e0053a70 0x7f12e0055f30 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f12f4003000 tx=0x7f12f4007a50).stop 2022-06-29T15:46:25.569 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.551+0000 7f1304af2700 1 -- 172.21.15.47:0/2913123753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12ec003910 msgr2=0x7f1300090940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:25.569 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.551+0000 7f1304af2700 1 --2- 172.21.15.47:0/2913123753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12ec003910 0x7f1300090940 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7f12f000c000 tx=0x7f12f0019000).stop 2022-06-29T15:46:25.569 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.551+0000 7f1304af2700 1 -- 172.21.15.47:0/2913123753 shutdown_connections 2022-06-29T15:46:25.570 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.551+0000 7f1304af2700 1 --2- 172.21.15.47:0/2913123753 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f130014e570 0x7f1300091c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:25.570 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.555+0000 7f1304af2700 1 --2- 172.21.15.47:0/2913123753 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f12e0053a70 0x7f12e0055f30 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:25.571 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.555+0000 7f1304af2700 1 --2- 172.21.15.47:0/2913123753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13000903c0 0x7f130014e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:25.571 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.555+0000 7f1304af2700 1 --2- 172.21.15.47:0/2913123753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12ec003910 0x7f1300090940 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:25.571 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.555+0000 7f1304af2700 1 -- 172.21.15.47:0/2913123753 >> 172.21.15.47:0/2913123753 conn(0x7f1300062df0 msgr2=0x7f13000631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:25.571 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.555+0000 7f1304af2700 1 -- 172.21.15.47:0/2913123753 shutdown_connections 2022-06-29T15:46:25.572 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.555+0000 7f1304af2700 1 -- 172.21.15.47:0/2913123753 wait complete. 2022-06-29T15:46:25.572 INFO:tasks.cephfs.fuse_mount:Cleaning up killed ceph-fuse connection 2022-06-29T15:46:25.573 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:46:25.573 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:46:25.574 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:46:25.582 INFO:teuthology.orchestra.run.smithi099.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': Transport endpoint is not connected 2022-06-29T15:46:25.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T15:46:25.584 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.0 2022-06-29T15:46:25.585 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi099.front.sepia.ceph.com... 2022-06-29T15:46:25.586 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-06-29T15:46:25.587 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:46:25.609 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:46:25.610 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:46:25.611 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:46:25.658 INFO:teuthology.orchestra.run.smithi099.stdout:ext2/ext3 2022-06-29T15:46:25.658 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-06-29T15:46:25.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-29T15:46:25.659 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:46:25.668 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:46:25.790 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.779+0000 7f00b716d700 1 Processor -- start 2022-06-29T15:46:25.791 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.779+0000 7f00b716d700 1 -- start start 2022-06-29T15:46:25.792 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.779+0000 7f00b716d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f00b0143bd0 0x7f00b005c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:25.792 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.779+0000 7f00b716d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f00b00b2c40 0x7f00b00b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:25.792 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.779+0000 7f00b716d700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00b0142f90 con 0x7f00b005c510 2022-06-29T15:46:25.793 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.779+0000 7f00b716d700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00b01431f0 con 0x7f00b0143bd0 2022-06-29T15:46:25.793 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.779+0000 7f00b716d700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00b0143490 con 0x7f00b00b2c40 2022-06-29T15:46:25.794 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.779+0000 7f00b596a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f00b0143bd0 0x7f00b005c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:25.794 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.779+0000 7f00b596a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f00b0143bd0 0x7f00b005c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48814/0 (socket says 172.21.15.47:48814) 2022-06-29T15:46:25.795 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.779+0000 7f00b596a700 1 -- 172.21.15.47:0/1603243177 learned_addr learned my addr 172.21.15.47:0/1603243177 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:46:25.795 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.779+0000 7f00b696c700 1 --2- 172.21.15.47:0/1603243177 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f00b00b2c40 0x7f00b00b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:25.796 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.779+0000 7f00b596a700 1 -- 172.21.15.47:0/1603243177 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f00b00b2c40 msgr2=0x7f00b00b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:46:25.796 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.779+0000 7f00b596a700 1 --2- 172.21.15.47:0/1603243177 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f00b00b2c40 0x7f00b00b30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:25.797 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.779+0000 7f00b596a700 1 -- 172.21.15.47:0/1603243177 >> v1:172.21.15.47:6789/0 conn(0x7f00b005c510 legacy=0x7f00b00b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:25.798 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.779+0000 7f00b596a700 1 -- 172.21.15.47:0/1603243177 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00b0143730 con 0x7f00b0143bd0 2022-06-29T15:46:25.798 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.779+0000 7f00b596a700 1 --2- 172.21.15.47:0/1603243177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00b0143bd0 0x7f00b005c8f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7f00ac002f70 tx=0x7f00ac006eb0).ready entity=mon.0 client_cookie=83b9ad5e176589d5 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:25.799 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.779+0000 7f00b5169700 1 -- 172.21.15.47:0/1603243177 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f00ac00d5c0 con 0x7f00b0143bd0 2022-06-29T15:46:25.799 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.779+0000 7f00b5169700 1 -- 172.21.15.47:0/1603243177 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00ac000e90 con 0x7f00b0143bd0 2022-06-29T15:46:25.800 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.779+0000 7f00b5169700 1 -- 172.21.15.47:0/1603243177 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f00ac00d950 con 0x7f00b0143bd0 2022-06-29T15:46:25.800 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.779+0000 7f00b716d700 1 -- 172.21.15.47:0/1603243177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00b0143bd0 msgr2=0x7f00b005c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:25.801 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.779+0000 7f00b716d700 1 --2- 172.21.15.47:0/1603243177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00b0143bd0 0x7f00b005c8f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7f00ac002f70 tx=0x7f00ac006eb0).stop 2022-06-29T15:46:25.801 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.783+0000 7f00b716d700 1 -- 172.21.15.47:0/1603243177 shutdown_connections 2022-06-29T15:46:25.802 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.783+0000 7f00b716d700 1 --2- 172.21.15.47:0/1603243177 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f00b00b2c40 0x7f00b00b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:25.802 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.783+0000 7f00b716d700 1 --2- 172.21.15.47:0/1603243177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00b0143bd0 0x7f00b005c8f0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:25.803 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.783+0000 7f00b716d700 1 -- 172.21.15.47:0/1603243177 >> 172.21.15.47:0/1603243177 conn(0x7f00b0060e00 msgr2=0x7f00b00611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:25.803 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.783+0000 7f00b716d700 1 -- 172.21.15.47:0/1603243177 shutdown_connections 2022-06-29T15:46:25.804 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.783+0000 7f00b716d700 1 -- 172.21.15.47:0/1603243177 wait complete. 2022-06-29T15:46:25.804 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.783+0000 7f00b716d700 1 Processor -- start 2022-06-29T15:46:25.805 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.783+0000 7f00b716d700 1 -- start start 2022-06-29T15:46:25.805 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.783+0000 7f00b716d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00b005c510 0x7f00b00af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:25.806 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.783+0000 7f00b716d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f00b00b2c40 0x7f00b0152ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:25.806 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.783+0000 7f00b716d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00b0143bd0 0x7f00b01559a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:25.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.783+0000 7f00b716d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b01533c0 con 0x7f00b0143bd0 2022-06-29T15:46:25.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.783+0000 7f00b716d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b01442c0 con 0x7f00b005c510 2022-06-29T15:46:25.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.783+0000 7f00b716d700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00b0144560 con 0x7f00b00b2c40 2022-06-29T15:46:25.809 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.783+0000 7f00b696c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00b0143bd0 0x7f00b01559a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:25.809 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.783+0000 7f00b696c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00b0143bd0 0x7f00b01559a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48824/0 (socket says 172.21.15.47:48824) 2022-06-29T15:46:25.809 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.783+0000 7f00b696c700 1 -- 172.21.15.47:0/1199301683 learned_addr learned my addr 172.21.15.47:0/1199301683 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:46:25.810 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.783+0000 7f00b616b700 1 --2- 172.21.15.47:0/1199301683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00b005c510 0x7f00b00af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:25.811 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.783+0000 7f00b596a700 1 --2- 172.21.15.47:0/1199301683 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f00b00b2c40 0x7f00b0152ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:25.811 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.783+0000 7f00b696c700 1 -- 172.21.15.47:0/1199301683 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f00b00b2c40 msgr2=0x7f00b0152ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:25.812 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.783+0000 7f00b696c700 1 --2- 172.21.15.47:0/1199301683 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f00b00b2c40 0x7f00b0152ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:25.812 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.783+0000 7f00b696c700 1 -- 172.21.15.47:0/1199301683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00b005c510 msgr2=0x7f00b00af8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:25.813 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.783+0000 7f00b696c700 1 --2- 172.21.15.47:0/1199301683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00b005c510 0x7f00b00af8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:25.813 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.783+0000 7f00b696c700 1 -- 172.21.15.47:0/1199301683 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00ac00b1e0 con 0x7f00b0143bd0 2022-06-29T15:46:25.814 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.787+0000 7f00b696c700 1 --2- 172.21.15.47:0/1199301683 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00b0143bd0 0x7f00b01559a0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7f00a800a000 tx=0x7f00a8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:25.814 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.787+0000 7f00a77fe700 1 -- 172.21.15.47:0/1199301683 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f00a8013620 con 0x7f00b0143bd0 2022-06-29T15:46:25.815 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.787+0000 7f00a77fe700 1 -- 172.21.15.47:0/1199301683 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00a8013780 con 0x7f00b0143bd0 2022-06-29T15:46:25.815 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.787+0000 7f00a77fe700 1 -- 172.21.15.47:0/1199301683 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f00a8013ae0 con 0x7f00b0143bd0 2022-06-29T15:46:25.816 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.787+0000 7f00b716d700 1 -- 172.21.15.47:0/1199301683 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00b00b7710 con 0x7f00b0143bd0 2022-06-29T15:46:25.816 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.787+0000 7f00b716d700 1 -- 172.21.15.47:0/1199301683 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00b0140280 con 0x7f00b0143bd0 2022-06-29T15:46:25.819 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.787+0000 7f00a77fe700 1 -- 172.21.15.47:0/1199301683 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f00a8013c40 con 0x7f00b0143bd0 2022-06-29T15:46:25.820 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.787+0000 7f00b716d700 1 -- 172.21.15.47:0/1199301683 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00b00b7570 con 0x7f00b0143bd0 2022-06-29T15:46:25.820 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.787+0000 7f00a77fe700 1 --2- 172.21.15.47:0/1199301683 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0090053a70 0x7f0090055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:25.821 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.787+0000 7f00a77fe700 1 -- 172.21.15.47:0/1199301683 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f00a806bd80 con 0x7f00b0143bd0 2022-06-29T15:46:25.822 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.791+0000 7f00b616b700 1 --2- 172.21.15.47:0/1199301683 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0090053a70 0x7f0090055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:25.822 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.791+0000 7f00b616b700 1 --2- 172.21.15.47:0/1199301683 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0090053a70 0x7f0090055f30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f00a000d000 tx=0x7f00a0007b50).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:25.823 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.795+0000 7f00a77fe700 1 -- 172.21.15.47:0/1199301683 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f00b00b7570 con 0x7f00b0143bd0 2022-06-29T15:46:25.927 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.923+0000 7f00b716d700 1 -- 172.21.15.47:0/1199301683 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f00b0144720 con 0x7f00b0143bd0 2022-06-29T15:46:25.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.927+0000 7f00a77fe700 1 -- 172.21.15.47:0/1199301683 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16273 (secure 0 0 0) 0x7f00a806d090 con 0x7f00b0143bd0 2022-06-29T15:46:25.929 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:46:25.930 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":58,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:46:03.307251+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:45:59.710738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2022-06-29T15:46:00.582445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:46:25.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.927+0000 7f00b716d700 1 -- 172.21.15.47:0/1199301683 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0090053a70 msgr2=0x7f0090055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:25.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.927+0000 7f00b716d700 1 --2- 172.21.15.47:0/1199301683 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0090053a70 0x7f0090055f30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f00a000d000 tx=0x7f00a0007b50).stop 2022-06-29T15:46:25.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.927+0000 7f00b716d700 1 -- 172.21.15.47:0/1199301683 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00b0143bd0 msgr2=0x7f00b01559a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:25.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.927+0000 7f00b716d700 1 --2- 172.21.15.47:0/1199301683 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00b0143bd0 0x7f00b01559a0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7f00a800a000 tx=0x7f00a8008040).stop 2022-06-29T15:46:25.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.927+0000 7f00b716d700 1 -- 172.21.15.47:0/1199301683 shutdown_connections 2022-06-29T15:46:25.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.927+0000 7f00b716d700 1 --2- 172.21.15.47:0/1199301683 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00b0143bd0 0x7f00b01559a0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:25.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.927+0000 7f00b716d700 1 --2- 172.21.15.47:0/1199301683 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f00b00b2c40 0x7f00b0152ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:25.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.927+0000 7f00b716d700 1 --2- 172.21.15.47:0/1199301683 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0090053a70 0x7f0090055f30 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:25.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.927+0000 7f00b716d700 1 --2- 172.21.15.47:0/1199301683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00b005c510 0x7f00b00af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:25.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.927+0000 7f00b716d700 1 -- 172.21.15.47:0/1199301683 >> 172.21.15.47:0/1199301683 conn(0x7f00b0060e00 msgr2=0x7f00b00b3620 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:25.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.927+0000 7f00b716d700 1 -- 172.21.15.47:0/1199301683 shutdown_connections 2022-06-29T15:46:25.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:25.927+0000 7f00b716d700 1 -- 172.21.15.47:0/1199301683 wait complete. 2022-06-29T15:46:25.942 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-29T15:42:16.971431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-29T15:42:25.290884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-29T15:45:59.710738+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2022-06-29T15:46:00.582445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-29T15:46:25.943 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-29T15:46:26.056 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.051+0000 7f1b2e6bc700 1 Processor -- start 2022-06-29T15:46:26.056 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.051+0000 7f1b2e6bc700 1 -- start start 2022-06-29T15:46:26.059 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.055+0000 7f1b2e6bc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1b28060e40 0x7f1b2813cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:26.059 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.055+0000 7f1b2e6bc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1b28060890 0x7f1b2813f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:26.060 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.055+0000 7f1b2e6bc700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b280ac990 con 0x7f1b280b0890 2022-06-29T15:46:26.060 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.055+0000 7f1b2e6bc700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b280b0c70 con 0x7f1b28060e40 2022-06-29T15:46:26.060 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.055+0000 7f1b2e6bc700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b280ad7a0 con 0x7f1b28060890 2022-06-29T15:46:26.061 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.055+0000 7f1b2ceb9700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1b28060e40 0x7f1b2813cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:26.061 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.055+0000 7f1b2ceb9700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1b28060e40 0x7f1b2813cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48832/0 (socket says 172.21.15.47:48832) 2022-06-29T15:46:26.061 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.055+0000 7f1b2ceb9700 1 -- 172.21.15.47:0/474568573 learned_addr learned my addr 172.21.15.47:0/474568573 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:46:26.061 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.055+0000 7f1b2debb700 1 --2- 172.21.15.47:0/474568573 >> v2:172.21.15.64:3300/0 conn(0x7f1b28060890 0x7f1b2813f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:26.062 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.055+0000 7f1b2ceb9700 1 -- 172.21.15.47:0/474568573 >> v2:172.21.15.64:3300/0 conn(0x7f1b28060890 msgr2=0x7f1b2813f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:46:26.062 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.055+0000 7f1b2ceb9700 1 --2- 172.21.15.47:0/474568573 >> v2:172.21.15.64:3300/0 conn(0x7f1b28060890 0x7f1b2813f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:26.062 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.055+0000 7f1b2ceb9700 1 -- 172.21.15.47:0/474568573 >> v1:172.21.15.64:6789/0 conn(0x7f1b280b0890 legacy=0x7f1b2813c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:26.062 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.055+0000 7f1b2ceb9700 1 -- 172.21.15.47:0/474568573 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b280aefa0 con 0x7f1b28060e40 2022-06-29T15:46:26.063 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.055+0000 7f1b2ceb9700 1 --2- 172.21.15.47:0/474568573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b28060e40 0x7f1b2813cee0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7f1b18002f70 tx=0x7f1b18006eb0).ready entity=mon.0 client_cookie=90913364174766bd server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:26.063 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.055+0000 7f1b17fff700 1 -- 172.21.15.47:0/474568573 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b1800d5c0 con 0x7f1b28060e40 2022-06-29T15:46:26.063 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.055+0000 7f1b17fff700 1 -- 172.21.15.47:0/474568573 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b18000e90 con 0x7f1b28060e40 2022-06-29T15:46:26.063 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.055+0000 7f1b17fff700 1 -- 172.21.15.47:0/474568573 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b1800d950 con 0x7f1b28060e40 2022-06-29T15:46:26.067 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.055+0000 7f1b2e6bc700 1 -- 172.21.15.47:0/474568573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b28060e40 msgr2=0x7f1b2813cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:26.067 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.055+0000 7f1b2e6bc700 1 --2- 172.21.15.47:0/474568573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b28060e40 0x7f1b2813cee0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7f1b18002f70 tx=0x7f1b18006eb0).stop 2022-06-29T15:46:26.068 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.055+0000 7f1b2e6bc700 1 -- 172.21.15.47:0/474568573 shutdown_connections 2022-06-29T15:46:26.068 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.055+0000 7f1b2e6bc700 1 --2- 172.21.15.47:0/474568573 >> v2:172.21.15.64:3300/0 conn(0x7f1b28060890 0x7f1b2813f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:26.068 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.055+0000 7f1b2e6bc700 1 --2- 172.21.15.47:0/474568573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b28060e40 0x7f1b2813cee0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:26.068 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.055+0000 7f1b2e6bc700 1 -- 172.21.15.47:0/474568573 >> 172.21.15.47:0/474568573 conn(0x7f1b280b5320 msgr2=0x7f1b280b5700 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:26.069 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.055+0000 7f1b2e6bc700 1 -- 172.21.15.47:0/474568573 shutdown_connections 2022-06-29T15:46:26.069 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.055+0000 7f1b2e6bc700 1 -- 172.21.15.47:0/474568573 wait complete. 2022-06-29T15:46:26.069 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.059+0000 7f1b2e6bc700 1 Processor -- start 2022-06-29T15:46:26.069 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.059+0000 7f1b2e6bc700 1 -- start start 2022-06-29T15:46:26.070 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.059+0000 7f1b2e6bc700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1b28060890 0x7f1b28066bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:26.070 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.059+0000 7f1b2e6bc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b28060e40 0x7f1b2812e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:26.070 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.059+0000 7f1b2e6bc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b280b0890 0x7f1b2812ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:26.070 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.059+0000 7f1b2e6bc700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b28130210 con 0x7f1b280b0890 2022-06-29T15:46:26.071 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.059+0000 7f1b2e6bc700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b28130440 con 0x7f1b28060e40 2022-06-29T15:46:26.071 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.059+0000 7f1b2e6bc700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b281306e0 con 0x7f1b28060890 2022-06-29T15:46:26.071 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.059+0000 7f1b2debb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b280b0890 0x7f1b2812ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:26.071 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.059+0000 7f1b2d6ba700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1b28060890 0x7f1b28066bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:26.071 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.059+0000 7f1b2debb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b280b0890 0x7f1b2812ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48838/0 (socket says 172.21.15.47:48838) 2022-06-29T15:46:26.072 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.059+0000 7f1b2ceb9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b28060e40 0x7f1b2812e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:26.072 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.059+0000 7f1b2debb700 1 -- 172.21.15.47:0/303018619 learned_addr learned my addr 172.21.15.47:0/303018619 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:46:26.072 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.059+0000 7f1b2debb700 1 -- 172.21.15.47:0/303018619 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1b28060890 msgr2=0x7f1b28066bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:26.072 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.059+0000 7f1b2debb700 1 --2- 172.21.15.47:0/303018619 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1b28060890 0x7f1b28066bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:26.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.059+0000 7f1b2debb700 1 -- 172.21.15.47:0/303018619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b28060e40 msgr2=0x7f1b2812e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:26.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.059+0000 7f1b2debb700 1 --2- 172.21.15.47:0/303018619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b28060e40 0x7f1b2812e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:26.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.059+0000 7f1b2debb700 1 -- 172.21.15.47:0/303018619 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b1800b1e0 con 0x7f1b280b0890 2022-06-29T15:46:26.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.059+0000 7f1b2debb700 1 --2- 172.21.15.47:0/303018619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b280b0890 0x7f1b2812ecf0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f1b20006e00 tx=0x7f1b200091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:26.074 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.059+0000 7f1b167fc700 1 -- 172.21.15.47:0/303018619 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b2000c640 con 0x7f1b280b0890 2022-06-29T15:46:26.074 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.059+0000 7f1b167fc700 1 -- 172.21.15.47:0/303018619 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b2000c7a0 con 0x7f1b280b0890 2022-06-29T15:46:26.074 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.059+0000 7f1b2e6bc700 1 -- 172.21.15.47:0/303018619 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b28130980 con 0x7f1b280b0890 2022-06-29T15:46:26.074 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.059+0000 7f1b167fc700 1 -- 172.21.15.47:0/303018619 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b2000cb00 con 0x7f1b280b0890 2022-06-29T15:46:26.075 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.059+0000 7f1b2e6bc700 1 -- 172.21.15.47:0/303018619 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b28067260 con 0x7f1b280b0890 2022-06-29T15:46:26.075 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.063+0000 7f1b2e6bc700 1 -- 172.21.15.47:0/303018619 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b280573d0 con 0x7f1b280b0890 2022-06-29T15:46:26.075 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.063+0000 7f1b167fc700 1 -- 172.21.15.47:0/303018619 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1b20022070 con 0x7f1b280b0890 2022-06-29T15:46:26.075 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.063+0000 7f1b167fc700 1 --2- 172.21.15.47:0/303018619 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1b080539e0 0x7f1b08055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:26.076 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.063+0000 7f1b2d6ba700 1 --2- 172.21.15.47:0/303018619 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1b080539e0 0x7f1b08055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:26.076 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.063+0000 7f1b167fc700 1 -- 172.21.15.47:0/303018619 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f1b2006c6f0 con 0x7f1b280b0890 2022-06-29T15:46:26.077 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.063+0000 7f1b2d6ba700 1 --2- 172.21.15.47:0/303018619 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1b080539e0 0x7f1b08055ea0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f1b2400d000 tx=0x7f1b24007b70).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:26.077 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.071+0000 7f1b167fc700 1 -- 172.21.15.47:0/303018619 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1b200433a0 con 0x7f1b280b0890 2022-06-29T15:46:26.202 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.199+0000 7f1b2e6bc700 1 -- 172.21.15.47:0/303018619 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f1b280af6e0 con 0x7f1b280b0890 2022-06-29T15:46:26.203 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.199+0000 7f1b167fc700 1 -- 172.21.15.47:0/303018619 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v58) v1 ==== 110+0+10 (secure 0 0 0) 0x7f1b200469c0 con 0x7f1b280b0890 2022-06-29T15:46:26.203 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2022-06-29T15:46:26.205 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.203+0000 7f1b2e6bc700 1 -- 172.21.15.47:0/303018619 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1b080539e0 msgr2=0x7f1b08055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:26.205 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.203+0000 7f1b2e6bc700 1 --2- 172.21.15.47:0/303018619 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1b080539e0 0x7f1b08055ea0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f1b2400d000 tx=0x7f1b24007b70).stop 2022-06-29T15:46:26.206 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.203+0000 7f1b2e6bc700 1 -- 172.21.15.47:0/303018619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b280b0890 msgr2=0x7f1b2812ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:26.206 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.203+0000 7f1b2e6bc700 1 --2- 172.21.15.47:0/303018619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b280b0890 0x7f1b2812ecf0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f1b20006e00 tx=0x7f1b200091d0).stop 2022-06-29T15:46:26.206 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.203+0000 7f1b2e6bc700 1 -- 172.21.15.47:0/303018619 shutdown_connections 2022-06-29T15:46:26.206 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.203+0000 7f1b2e6bc700 1 --2- 172.21.15.47:0/303018619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b280b0890 0x7f1b2812ecf0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:26.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.203+0000 7f1b2e6bc700 1 --2- 172.21.15.47:0/303018619 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1b080539e0 0x7f1b08055ea0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:26.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.203+0000 7f1b2e6bc700 1 --2- 172.21.15.47:0/303018619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b28060e40 0x7f1b2812e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:26.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.203+0000 7f1b2e6bc700 1 --2- 172.21.15.47:0/303018619 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1b28060890 0x7f1b28066bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:26.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.203+0000 7f1b2e6bc700 1 -- 172.21.15.47:0/303018619 >> 172.21.15.47:0/303018619 conn(0x7f1b280b5320 msgr2=0x7f1b281316c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:26.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.203+0000 7f1b2e6bc700 1 -- 172.21.15.47:0/303018619 shutdown_connections 2022-06-29T15:46:26.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.203+0000 7f1b2e6bc700 1 -- 172.21.15.47:0/303018619 wait complete. 2022-06-29T15:46:26.216 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-29T15:46:26.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.323+0000 7f30cc701700 1 Processor -- start 2022-06-29T15:46:26.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.323+0000 7f30cc701700 1 -- start start 2022-06-29T15:46:26.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.323+0000 7f30cc701700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f30c4060ee0 0x7f30c40612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:26.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.323+0000 7f30cc701700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f30c40b4300 0x7f30c41471a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:26.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.323+0000 7f30cc701700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30c4067610 con 0x7f30c4066f70 2022-06-29T15:46:26.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.323+0000 7f30cc701700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30c40673a0 con 0x7f30c4060ee0 2022-06-29T15:46:26.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.323+0000 7f30cc701700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30c41467c0 con 0x7f30c40b4300 2022-06-29T15:46:26.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.323+0000 7f30ca49d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f30c4060ee0 0x7f30c40612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:26.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.323+0000 7f30ca49d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f30c4060ee0 0x7f30c40612c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48842/0 (socket says 172.21.15.47:48842) 2022-06-29T15:46:26.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.323+0000 7f30ca49d700 1 -- 172.21.15.47:0/1646104994 learned_addr learned my addr 172.21.15.47:0/1646104994 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:46:26.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.323+0000 7f30c9c9c700 1 --2- 172.21.15.47:0/1646104994 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f30c40b4300 0x7f30c41471a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:26.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.323+0000 7f30ca49d700 1 -- 172.21.15.47:0/1646104994 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f30c40b4300 msgr2=0x7f30c41471a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:46:26.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.323+0000 7f30ca49d700 1 --2- 172.21.15.47:0/1646104994 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f30c40b4300 0x7f30c41471a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:26.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.323+0000 7f30ca49d700 1 -- 172.21.15.47:0/1646104994 >> v1:172.21.15.64:6789/0 conn(0x7f30c4066f70 legacy=0x7f30c41430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:26.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.323+0000 7f30ca49d700 1 -- 172.21.15.47:0/1646104994 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c4146a60 con 0x7f30c4060ee0 2022-06-29T15:46:26.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.323+0000 7f30ca49d700 1 --2- 172.21.15.47:0/1646104994 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30c4060ee0 0x7f30c40612c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7f30c0002f70 tx=0x7f30c0006eb0).ready entity=mon.0 client_cookie=208afceb879a4341 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:26.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.323+0000 7f30c949b700 1 -- 172.21.15.47:0/1646104994 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f30c000d570 con 0x7f30c4060ee0 2022-06-29T15:46:26.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.323+0000 7f30c949b700 1 -- 172.21.15.47:0/1646104994 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30c0000e90 con 0x7f30c4060ee0 2022-06-29T15:46:26.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.323+0000 7f30c949b700 1 -- 172.21.15.47:0/1646104994 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f30c000d900 con 0x7f30c4060ee0 2022-06-29T15:46:26.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.323+0000 7f30cc701700 1 -- 172.21.15.47:0/1646104994 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30c4060ee0 msgr2=0x7f30c40612c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:26.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.323+0000 7f30cc701700 1 --2- 172.21.15.47:0/1646104994 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30c4060ee0 0x7f30c40612c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7f30c0002f70 tx=0x7f30c0006eb0).stop 2022-06-29T15:46:26.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.323+0000 7f30cc701700 1 -- 172.21.15.47:0/1646104994 shutdown_connections 2022-06-29T15:46:26.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.323+0000 7f30cc701700 1 --2- 172.21.15.47:0/1646104994 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f30c40b4300 0x7f30c41471a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:26.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.323+0000 7f30cc701700 1 --2- 172.21.15.47:0/1646104994 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30c4060ee0 0x7f30c40612c0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:26.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.323+0000 7f30cc701700 1 -- 172.21.15.47:0/1646104994 >> 172.21.15.47:0/1646104994 conn(0x7f30c405c510 msgr2=0x7f30c405f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:26.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.323+0000 7f30cc701700 1 -- 172.21.15.47:0/1646104994 shutdown_connections 2022-06-29T15:46:26.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.327+0000 7f30cc701700 1 -- 172.21.15.47:0/1646104994 wait complete. 2022-06-29T15:46:26.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.327+0000 7f30cc701700 1 Processor -- start 2022-06-29T15:46:26.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.327+0000 7f30cc701700 1 -- start start 2022-06-29T15:46:26.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.331+0000 7f30cc701700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f30c4060ee0 0x7f30c4145b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:26.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.331+0000 7f30cc701700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30c4066f70 0x7f30c4155a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:26.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.331+0000 7f30cc701700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30c40b4300 0x7f30c4137790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:26.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.331+0000 7f30cc701700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c41452e0 con 0x7f30c4066f70 2022-06-29T15:46:26.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.331+0000 7f30cc701700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c41382f0 con 0x7f30c40b4300 2022-06-29T15:46:26.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.331+0000 7f30cc701700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30c40b1360 con 0x7f30c4060ee0 2022-06-29T15:46:26.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.331+0000 7f30c9c9c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30c4066f70 0x7f30c4155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:26.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.331+0000 7f30ca49d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f30c4060ee0 0x7f30c4145b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:26.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.331+0000 7f30c9c9c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30c4066f70 0x7f30c4155a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48850/0 (socket says 172.21.15.47:48850) 2022-06-29T15:46:26.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.331+0000 7f30c9c9c700 1 -- 172.21.15.47:0/2247480574 learned_addr learned my addr 172.21.15.47:0/2247480574 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:46:26.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.331+0000 7f30cac9e700 1 --2- 172.21.15.47:0/2247480574 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30c40b4300 0x7f30c4137790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:26.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.331+0000 7f30c9c9c700 1 -- 172.21.15.47:0/2247480574 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f30c4060ee0 msgr2=0x7f30c4145b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:26.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.331+0000 7f30c9c9c700 1 --2- 172.21.15.47:0/2247480574 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f30c4060ee0 0x7f30c4145b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:26.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.331+0000 7f30c9c9c700 1 -- 172.21.15.47:0/2247480574 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30c40b4300 msgr2=0x7f30c4137790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:26.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.331+0000 7f30c9c9c700 1 --2- 172.21.15.47:0/2247480574 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30c40b4300 0x7f30c4137790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:26.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.331+0000 7f30c9c9c700 1 -- 172.21.15.47:0/2247480574 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c000b1e0 con 0x7f30c4066f70 2022-06-29T15:46:26.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.331+0000 7f30c9c9c700 1 --2- 172.21.15.47:0/2247480574 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30c4066f70 0x7f30c4155a00 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7f30b400c000 tx=0x7f30b4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:26.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.331+0000 7f30b37fe700 1 -- 172.21.15.47:0/2247480574 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f30b4011750 con 0x7f30c4066f70 2022-06-29T15:46:26.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.331+0000 7f30b37fe700 1 -- 172.21.15.47:0/2247480574 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30b40118b0 con 0x7f30c4066f70 2022-06-29T15:46:26.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.331+0000 7f30cc701700 1 -- 172.21.15.47:0/2247480574 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30c4145540 con 0x7f30c4066f70 2022-06-29T15:46:26.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.331+0000 7f30b37fe700 1 -- 172.21.15.47:0/2247480574 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f30b4011c10 con 0x7f30c4066f70 2022-06-29T15:46:26.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.331+0000 7f30cc701700 1 -- 172.21.15.47:0/2247480574 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30c4146070 con 0x7f30c4066f70 2022-06-29T15:46:26.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.331+0000 7f30cc701700 1 -- 172.21.15.47:0/2247480574 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30c40573d0 con 0x7f30c4066f70 2022-06-29T15:46:26.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.331+0000 7f30b37fe700 1 -- 172.21.15.47:0/2247480574 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f30b4011d70 con 0x7f30c4066f70 2022-06-29T15:46:26.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.335+0000 7f30b37fe700 1 --2- 172.21.15.47:0/2247480574 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f30a4053b00 0x7f30a4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:26.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.335+0000 7f30b37fe700 1 -- 172.21.15.47:0/2247480574 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f30b406c450 con 0x7f30c4066f70 2022-06-29T15:46:26.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.335+0000 7f30ca49d700 1 --2- 172.21.15.47:0/2247480574 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f30a4053b00 0x7f30a4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:26.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.335+0000 7f30ca49d700 1 --2- 172.21.15.47:0/2247480574 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f30a4053b00 0x7f30a4055fc0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f30c001c000 tx=0x7f30c0015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:26.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.339+0000 7f30b37fe700 1 -- 172.21.15.47:0/2247480574 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f30b4043210 con 0x7f30c4066f70 2022-06-29T15:46:26.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.467+0000 7f30cc701700 1 -- 172.21.15.47:0/2247480574 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f30c40b5e50 con 0x7f30c4066f70 2022-06-29T15:46:26.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.467+0000 7f30b37fe700 1 -- 172.21.15.47:0/2247480574 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v58) v1 ==== 92+0+10 (secure 0 0 0) 0x7f30b4046830 con 0x7f30c4066f70 2022-06-29T15:46:26.471 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:46:26.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.467+0000 7f30cc701700 1 -- 172.21.15.47:0/2247480574 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f30a4053b00 msgr2=0x7f30a4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:26.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.467+0000 7f30cc701700 1 --2- 172.21.15.47:0/2247480574 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f30a4053b00 0x7f30a4055fc0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f30c001c000 tx=0x7f30c0015000).stop 2022-06-29T15:46:26.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.467+0000 7f30cc701700 1 -- 172.21.15.47:0/2247480574 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30c4066f70 msgr2=0x7f30c4155a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:26.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.467+0000 7f30cc701700 1 --2- 172.21.15.47:0/2247480574 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30c4066f70 0x7f30c4155a00 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7f30b400c000 tx=0x7f30b4006df0).stop 2022-06-29T15:46:26.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.467+0000 7f30cc701700 1 -- 172.21.15.47:0/2247480574 shutdown_connections 2022-06-29T15:46:26.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.467+0000 7f30cc701700 1 --2- 172.21.15.47:0/2247480574 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f30a4053b00 0x7f30a4055fc0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:26.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.467+0000 7f30cc701700 1 --2- 172.21.15.47:0/2247480574 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30c40b4300 0x7f30c4137790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:26.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.467+0000 7f30cc701700 1 --2- 172.21.15.47:0/2247480574 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30c4066f70 0x7f30c4155a00 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:26.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.467+0000 7f30cc701700 1 --2- 172.21.15.47:0/2247480574 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f30c4060ee0 0x7f30c4145b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:26.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.467+0000 7f30cc701700 1 -- 172.21.15.47:0/2247480574 >> 172.21.15.47:0/2247480574 conn(0x7f30c405c510 msgr2=0x7f30c413cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:26.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.467+0000 7f30cc701700 1 -- 172.21.15.47:0/2247480574 shutdown_connections 2022-06-29T15:46:26.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.471+0000 7f30cc701700 1 -- 172.21.15.47:0/2247480574 wait complete. 2022-06-29T15:46:26.484 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-29T15:46:26.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.595+0000 7f054b59e700 1 Processor -- start 2022-06-29T15:46:26.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.595+0000 7f054b59e700 1 -- start start 2022-06-29T15:46:26.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.595+0000 7f054b59e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f054c1467a0 0x7f054c140500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:26.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.595+0000 7f054b59e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f054c1409f0 0x7f054c14db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:26.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.599+0000 7f054b59e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f054c0b1050 con 0x7f054c0b6d20 2022-06-29T15:46:26.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.599+0000 7f054b59e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f054c0b23c0 con 0x7f054c1467a0 2022-06-29T15:46:26.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.599+0000 7f054b59e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f054c05fab0 con 0x7f054c1409f0 2022-06-29T15:46:26.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.599+0000 7f0549d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f054c1467a0 0x7f054c140500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:26.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.599+0000 7f054a59c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f054c0b6d20 0x7f054c0b1480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34410/0 (socket says 172.21.15.47:34410) 2022-06-29T15:46:26.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.599+0000 7f0549d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f054c1467a0 0x7f054c140500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48858/0 (socket says 172.21.15.47:48858) 2022-06-29T15:46:26.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.599+0000 7f054a59c700 1 -- 172.21.15.47:0/2575761875 learned_addr learned my addr 172.21.15.47:0/2575761875 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:46:26.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.599+0000 7f054ad9d700 1 --2- 172.21.15.47:0/2575761875 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f054c1409f0 0x7f054c14db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:26.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.599+0000 7f0549d9b700 1 -- 172.21.15.47:0/2575761875 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f054c1409f0 msgr2=0x7f054c14db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:46:26.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.599+0000 7f0549d9b700 1 --2- 172.21.15.47:0/2575761875 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f054c1409f0 0x7f054c14db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:26.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.599+0000 7f0549d9b700 1 -- 172.21.15.47:0/2575761875 >> v1:172.21.15.47:6789/0 conn(0x7f054c0b6d20 legacy=0x7f054c0b1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:26.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.599+0000 7f0549d9b700 1 -- 172.21.15.47:0/2575761875 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f054c0b7100 con 0x7f054c1467a0 2022-06-29T15:46:26.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.599+0000 7f0549d9b700 1 --2- 172.21.15.47:0/2575761875 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f054c1467a0 0x7f054c140500 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7f054c0b2700 tx=0x7f0534006df0).ready entity=mon.0 client_cookie=852cb715609eff37 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:26.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.599+0000 7f054959a700 1 -- 172.21.15.47:0/2575761875 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f053400d570 con 0x7f054c1467a0 2022-06-29T15:46:26.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.599+0000 7f054959a700 1 -- 172.21.15.47:0/2575761875 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0534000e90 con 0x7f054c1467a0 2022-06-29T15:46:26.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.599+0000 7f054959a700 1 -- 172.21.15.47:0/2575761875 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f053400d8c0 con 0x7f054c1467a0 2022-06-29T15:46:26.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.599+0000 7f054b59e700 1 -- 172.21.15.47:0/2575761875 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f054c1467a0 msgr2=0x7f054c140500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:26.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.599+0000 7f054b59e700 1 --2- 172.21.15.47:0/2575761875 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f054c1467a0 0x7f054c140500 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7f054c0b2700 tx=0x7f0534006df0).stop 2022-06-29T15:46:26.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.599+0000 7f054b59e700 1 -- 172.21.15.47:0/2575761875 shutdown_connections 2022-06-29T15:46:26.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.599+0000 7f054b59e700 1 --2- 172.21.15.47:0/2575761875 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f054c1409f0 0x7f054c14db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:26.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.599+0000 7f054b59e700 1 --2- 172.21.15.47:0/2575761875 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f054c1467a0 0x7f054c140500 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:26.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.599+0000 7f054b59e700 1 -- 172.21.15.47:0/2575761875 >> 172.21.15.47:0/2575761875 conn(0x7f054c063670 msgr2=0x7f054c063a50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:26.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.599+0000 7f054b59e700 1 -- 172.21.15.47:0/2575761875 shutdown_connections 2022-06-29T15:46:26.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.599+0000 7f054b59e700 1 -- 172.21.15.47:0/2575761875 wait complete. 2022-06-29T15:46:26.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.603+0000 7f054b59e700 1 Processor -- start 2022-06-29T15:46:26.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.603+0000 7f054b59e700 1 -- start start 2022-06-29T15:46:26.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.603+0000 7f054b59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f054c0b6d20 0x7f054c141ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:26.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.603+0000 7f054b59e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f054c1409f0 0x7f054c1526e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:26.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.603+0000 7f054b59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f054c1467a0 0x7f054c152bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:26.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.603+0000 7f054b59e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054c13e060 con 0x7f054c0b6d20 2022-06-29T15:46:26.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.603+0000 7f054b59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054c13e290 con 0x7f054c1467a0 2022-06-29T15:46:26.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.603+0000 7f054b59e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f054c156640 con 0x7f054c1409f0 2022-06-29T15:46:26.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.603+0000 7f054a59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f054c0b6d20 0x7f054c141ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:26.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.603+0000 7f054a59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f054c0b6d20 0x7f054c141ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48862/0 (socket says 172.21.15.47:48862) 2022-06-29T15:46:26.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.603+0000 7f054a59c700 1 -- 172.21.15.47:0/360392093 learned_addr learned my addr 172.21.15.47:0/360392093 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:46:26.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.603+0000 7f054ad9d700 1 --2- 172.21.15.47:0/360392093 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f054c1467a0 0x7f054c152bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:26.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.603+0000 7f0549d9b700 1 --2- 172.21.15.47:0/360392093 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f054c1409f0 0x7f054c1526e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:26.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.603+0000 7f054a59c700 1 -- 172.21.15.47:0/360392093 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f054c1409f0 msgr2=0x7f054c1526e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:26.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.603+0000 7f054a59c700 1 --2- 172.21.15.47:0/360392093 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f054c1409f0 0x7f054c1526e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:26.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.603+0000 7f054a59c700 1 -- 172.21.15.47:0/360392093 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f054c1467a0 msgr2=0x7f054c152bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:26.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.603+0000 7f054a59c700 1 --2- 172.21.15.47:0/360392093 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f054c1467a0 0x7f054c152bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:26.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.603+0000 7f054a59c700 1 -- 172.21.15.47:0/360392093 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f053400b180 con 0x7f054c0b6d20 2022-06-29T15:46:26.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.603+0000 7f054a59c700 1 --2- 172.21.15.47:0/360392093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f054c0b6d20 0x7f054c141ab0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7f054c06b740 tx=0x7f053c007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:26.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.603+0000 7f05437fe700 1 -- 172.21.15.47:0/360392093 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f053c0070d0 con 0x7f054c0b6d20 2022-06-29T15:46:26.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.603+0000 7f05437fe700 1 -- 172.21.15.47:0/360392093 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f053c007230 con 0x7f054c0b6d20 2022-06-29T15:46:26.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.603+0000 7f054b59e700 1 -- 172.21.15.47:0/360392093 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f054c1568e0 con 0x7f054c0b6d20 2022-06-29T15:46:26.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.603+0000 7f05437fe700 1 -- 172.21.15.47:0/360392093 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f053c005610 con 0x7f054c0b6d20 2022-06-29T15:46:26.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.603+0000 7f054b59e700 1 -- 172.21.15.47:0/360392093 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f054c0b0aa0 con 0x7f054c0b6d20 2022-06-29T15:46:26.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.607+0000 7f054b59e700 1 -- 172.21.15.47:0/360392093 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f054c13e490 con 0x7f054c0b6d20 2022-06-29T15:46:26.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.607+0000 7f05437fe700 1 -- 172.21.15.47:0/360392093 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f053c010070 con 0x7f054c0b6d20 2022-06-29T15:46:26.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.607+0000 7f05437fe700 1 --2- 172.21.15.47:0/360392093 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0530053aa0 0x7f0530055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:26.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.607+0000 7f05437fe700 1 -- 172.21.15.47:0/360392093 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f053c06b580 con 0x7f054c0b6d20 2022-06-29T15:46:26.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.607+0000 7f0549d9b700 1 --2- 172.21.15.47:0/360392093 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0530053aa0 0x7f0530055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:26.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.607+0000 7f0549d9b700 1 --2- 172.21.15.47:0/360392093 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0530053aa0 0x7f0530055f60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f053401a000 tx=0x7f0534014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:26.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.615+0000 7f05437fe700 1 -- 172.21.15.47:0/360392093 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f053c0422a0 con 0x7f054c0b6d20 2022-06-29T15:46:26.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.743+0000 7f054b59e700 1 -- 172.21.15.47:0/360392093 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f054c0573d0 con 0x7f054c0b6d20 2022-06-29T15:46:26.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.743+0000 7f05437fe700 1 -- 172.21.15.47:0/360392093 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v58) v1 ==== 104+0+10 (secure 0 0 0) 0x7f053c018020 con 0x7f054c0b6d20 2022-06-29T15:46:26.746 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:46:26.748 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.743+0000 7f054b59e700 1 -- 172.21.15.47:0/360392093 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0530053aa0 msgr2=0x7f0530055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:26.748 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.743+0000 7f054b59e700 1 --2- 172.21.15.47:0/360392093 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0530053aa0 0x7f0530055f60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f053401a000 tx=0x7f0534014000).stop 2022-06-29T15:46:26.749 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.743+0000 7f054b59e700 1 -- 172.21.15.47:0/360392093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f054c0b6d20 msgr2=0x7f054c141ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:26.749 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.743+0000 7f054b59e700 1 --2- 172.21.15.47:0/360392093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f054c0b6d20 0x7f054c141ab0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7f054c06b740 tx=0x7f053c007c40).stop 2022-06-29T15:46:26.749 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.743+0000 7f054b59e700 1 -- 172.21.15.47:0/360392093 shutdown_connections 2022-06-29T15:46:26.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.743+0000 7f054b59e700 1 --2- 172.21.15.47:0/360392093 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0530053aa0 0x7f0530055f60 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:26.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.743+0000 7f054b59e700 1 --2- 172.21.15.47:0/360392093 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f054c1467a0 0x7f054c152bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:26.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.743+0000 7f054b59e700 1 --2- 172.21.15.47:0/360392093 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f054c1409f0 0x7f054c1526e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:26.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.743+0000 7f054b59e700 1 --2- 172.21.15.47:0/360392093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f054c0b6d20 0x7f054c141ab0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:26.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.743+0000 7f054b59e700 1 -- 172.21.15.47:0/360392093 >> 172.21.15.47:0/360392093 conn(0x7f054c063670 msgr2=0x7f054c060a80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:26.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.743+0000 7f054b59e700 1 -- 172.21.15.47:0/360392093 shutdown_connections 2022-06-29T15:46:26.752 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.743+0000 7f054b59e700 1 -- 172.21.15.47:0/360392093 wait complete. 2022-06-29T15:46:26.759 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-29T15:46:26.871 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.867+0000 7f9758b7d700 1 Processor -- start 2022-06-29T15:46:26.872 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.867+0000 7f9758b7d700 1 -- start start 2022-06-29T15:46:26.874 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.871+0000 7f9758b7d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f975405c850 0x7f9754143a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:26.874 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.871+0000 7f9758b7d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f97540b5d90 0x7f97540b4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:26.875 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.871+0000 7f9758b7d700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f975405cc30 con 0x7f97540676a0 2022-06-29T15:46:26.875 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.871+0000 7f9758b7d700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97541430a0 con 0x7f97540b5d90 2022-06-29T15:46:26.875 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.871+0000 7f9758b7d700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9754143300 con 0x7f975405c850 2022-06-29T15:46:26.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.871+0000 7f9752ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f97540b5d90 0x7f97540b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:26.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.871+0000 7f9753fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f97540676a0 0x7f9754140130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34428/0 (socket says 172.21.15.47:34428) 2022-06-29T15:46:26.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.871+0000 7f9753fff700 1 -- 172.21.15.47:0/1576624221 learned_addr learned my addr 172.21.15.47:0/1576624221 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:46:26.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.871+0000 7f97537fe700 1 --2- 172.21.15.47:0/1576624221 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f975405c850 0x7f9754143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:26.877 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.871+0000 7f9752ffd700 1 -- 172.21.15.47:0/1576624221 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f975405c850 msgr2=0x7f9754143a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:46:26.877 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.871+0000 7f9752ffd700 1 --2- 172.21.15.47:0/1576624221 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f975405c850 0x7f9754143a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:26.877 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.871+0000 7f9752ffd700 1 -- 172.21.15.47:0/1576624221 >> v1:172.21.15.47:6789/0 conn(0x7f97540676a0 legacy=0x7f9754140130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:26.877 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.871+0000 7f9752ffd700 1 -- 172.21.15.47:0/1576624221 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97541435a0 con 0x7f97540b5d90 2022-06-29T15:46:26.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.871+0000 7f97527fc700 1 -- 172.21.15.47:0/1576624221 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1297867537 0 0) 0x7f975405cc30 con 0x7f97540676a0 2022-06-29T15:46:26.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.871+0000 7f9752ffd700 1 --2- 172.21.15.47:0/1576624221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97540b5d90 0x7f97540b4860 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7f9744002f70 tx=0x7f9744006df0).ready entity=mon.0 client_cookie=611f64efa3bb6799 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:26.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.871+0000 7f97527fc700 1 -- 172.21.15.47:0/1576624221 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f974400d570 con 0x7f97540b5d90 2022-06-29T15:46:26.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.871+0000 7f97527fc700 1 -- 172.21.15.47:0/1576624221 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9744000e90 con 0x7f97540b5d90 2022-06-29T15:46:26.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.871+0000 7f97527fc700 1 -- 172.21.15.47:0/1576624221 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f974400d8c0 con 0x7f97540b5d90 2022-06-29T15:46:26.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.871+0000 7f9758b7d700 1 -- 172.21.15.47:0/1576624221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97540b5d90 msgr2=0x7f97540b4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:26.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.871+0000 7f9758b7d700 1 --2- 172.21.15.47:0/1576624221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97540b5d90 0x7f97540b4860 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7f9744002f70 tx=0x7f9744006df0).stop 2022-06-29T15:46:26.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.871+0000 7f9758b7d700 1 -- 172.21.15.47:0/1576624221 shutdown_connections 2022-06-29T15:46:26.880 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.871+0000 7f9758b7d700 1 --2- 172.21.15.47:0/1576624221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97540b5d90 0x7f97540b4860 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:26.880 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.871+0000 7f9758b7d700 1 --2- 172.21.15.47:0/1576624221 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f975405c850 0x7f9754143a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:26.880 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.871+0000 7f9758b7d700 1 -- 172.21.15.47:0/1576624221 >> 172.21.15.47:0/1576624221 conn(0x7f9754060e00 msgr2=0x7f97540611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:26.880 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.871+0000 7f9758b7d700 1 -- 172.21.15.47:0/1576624221 shutdown_connections 2022-06-29T15:46:26.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.871+0000 7f9758b7d700 1 -- 172.21.15.47:0/1576624221 wait complete. 2022-06-29T15:46:26.883 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.871+0000 7f9758b7d700 1 Processor -- start 2022-06-29T15:46:26.884 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.871+0000 7f9758b7d700 1 -- start start 2022-06-29T15:46:26.884 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.871+0000 7f9758b7d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f975405c850 0x7f975412c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:26.884 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.871+0000 7f9758b7d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f97540676a0 0x7f9754152d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:26.884 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.871+0000 7f9758b7d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97540b5d90 0x7f975413fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:26.884 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.871+0000 7f9758b7d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97540b3390 con 0x7f975405c850 2022-06-29T15:46:26.885 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.871+0000 7f9758b7d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97540b3630 con 0x7f97540b5d90 2022-06-29T15:46:26.885 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.871+0000 7f9758b7d700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97540b38d0 con 0x7f97540676a0 2022-06-29T15:46:26.885 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.871+0000 7f97537fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f975405c850 0x7f975412c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:26.885 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.871+0000 7f97537fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f975405c850 0x7f975412c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48878/0 (socket says 172.21.15.47:48878) 2022-06-29T15:46:26.886 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.871+0000 7f97537fe700 1 -- 172.21.15.47:0/739515075 learned_addr learned my addr 172.21.15.47:0/739515075 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:46:26.886 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.871+0000 7f9752ffd700 1 --2- 172.21.15.47:0/739515075 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f97540676a0 0x7f9754152d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:26.886 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.875+0000 7f9753fff700 1 --2- 172.21.15.47:0/739515075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97540b5d90 0x7f975413fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:26.886 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.875+0000 7f97537fe700 1 -- 172.21.15.47:0/739515075 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f97540676a0 msgr2=0x7f9754152d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:26.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.875+0000 7f97537fe700 1 --2- 172.21.15.47:0/739515075 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f97540676a0 0x7f9754152d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:26.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.875+0000 7f97537fe700 1 -- 172.21.15.47:0/739515075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97540b5d90 msgr2=0x7f975413fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:26.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.875+0000 7f97537fe700 1 --2- 172.21.15.47:0/739515075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97540b5d90 0x7f975413fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:26.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.875+0000 7f97537fe700 1 -- 172.21.15.47:0/739515075 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f974400b180 con 0x7f975405c850 2022-06-29T15:46:26.888 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.875+0000 7f97537fe700 1 --2- 172.21.15.47:0/739515075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f975405c850 0x7f975412c100 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7f974800a000 tx=0x7f9748008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:26.888 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.875+0000 7f9750ff9700 1 -- 172.21.15.47:0/739515075 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9748011780 con 0x7f975405c850 2022-06-29T15:46:26.888 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.875+0000 7f9750ff9700 1 -- 172.21.15.47:0/739515075 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97480118e0 con 0x7f975405c850 2022-06-29T15:46:26.888 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.875+0000 7f9758b7d700 1 -- 172.21.15.47:0/739515075 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97540b3b70 con 0x7f975405c850 2022-06-29T15:46:26.888 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.875+0000 7f9750ff9700 1 -- 172.21.15.47:0/739515075 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9748011c40 con 0x7f975405c850 2022-06-29T15:46:26.889 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.875+0000 7f9758b7d700 1 -- 172.21.15.47:0/739515075 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9754153210 con 0x7f975405c850 2022-06-29T15:46:26.889 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.875+0000 7f9750ff9700 1 -- 172.21.15.47:0/739515075 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f974801c460 con 0x7f975405c850 2022-06-29T15:46:26.889 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.875+0000 7f9758b7d700 1 -- 172.21.15.47:0/739515075 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97540b3d00 con 0x7f975405c850 2022-06-29T15:46:26.889 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.875+0000 7f9750ff9700 1 --2- 172.21.15.47:0/739515075 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9734053aa0 0x7f9734055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:26.890 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.875+0000 7f9750ff9700 1 -- 172.21.15.47:0/739515075 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f974806c420 con 0x7f975405c850 2022-06-29T15:46:26.890 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.875+0000 7f9752ffd700 1 --2- 172.21.15.47:0/739515075 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9734053aa0 0x7f9734055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:26.890 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.875+0000 7f9752ffd700 1 --2- 172.21.15.47:0/739515075 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9734053aa0 0x7f9734055f60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f974401a000 tx=0x7f9744015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:26.891 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:26.883+0000 7f9750ff9700 1 -- 172.21.15.47:0/739515075 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9748043180 con 0x7f975405c850 2022-06-29T15:46:27.013 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.011+0000 7f9758b7d700 1 -- 172.21.15.47:0/739515075 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f97540573d0 con 0x7f975405c850 2022-06-29T15:46:27.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.011+0000 7f9750ff9700 1 -- 172.21.15.47:0/739515075 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v58) v1 ==== 100+0+10 (secure 0 0 0) 0x7f97480467a0 con 0x7f975405c850 2022-06-29T15:46:27.015 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:46:27.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.011+0000 7f9758b7d700 1 -- 172.21.15.47:0/739515075 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9734053aa0 msgr2=0x7f9734055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:27.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.011+0000 7f9758b7d700 1 --2- 172.21.15.47:0/739515075 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9734053aa0 0x7f9734055f60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f974401a000 tx=0x7f9744015000).stop 2022-06-29T15:46:27.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.011+0000 7f9758b7d700 1 -- 172.21.15.47:0/739515075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f975405c850 msgr2=0x7f975412c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:27.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.011+0000 7f9758b7d700 1 --2- 172.21.15.47:0/739515075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f975405c850 0x7f975412c100 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7f974800a000 tx=0x7f9748008040).stop 2022-06-29T15:46:27.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.011+0000 7f9758b7d700 1 -- 172.21.15.47:0/739515075 shutdown_connections 2022-06-29T15:46:27.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.011+0000 7f9758b7d700 1 --2- 172.21.15.47:0/739515075 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9734053aa0 0x7f9734055f60 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:27.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.011+0000 7f9758b7d700 1 --2- 172.21.15.47:0/739515075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97540b5d90 0x7f975413fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:27.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.015+0000 7f9758b7d700 1 --2- 172.21.15.47:0/739515075 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f97540676a0 0x7f9754152d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:27.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.015+0000 7f9758b7d700 1 --2- 172.21.15.47:0/739515075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f975405c850 0x7f975412c100 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:27.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.015+0000 7f9758b7d700 1 -- 172.21.15.47:0/739515075 >> 172.21.15.47:0/739515075 conn(0x7f9754060e00 msgr2=0x7f97540ae9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:27.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.015+0000 7f9758b7d700 1 -- 172.21.15.47:0/739515075 shutdown_connections 2022-06-29T15:46:27.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.015+0000 7f9758b7d700 1 -- 172.21.15.47:0/739515075 wait complete. 2022-06-29T15:46:27.027 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-29T15:46:27.027 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:46:27.132 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.127+0000 7f349de7f700 1 Processor -- start 2022-06-29T15:46:27.132 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.127+0000 7f349de7f700 1 -- start start 2022-06-29T15:46:27.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.131+0000 7f349de7f700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f349805fa30 0x7f34980b1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:27.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.131+0000 7f349de7f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3498060b90 0x7f349813f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:27.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.131+0000 7f349de7f700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3498060fd0 con 0x7f34980b2770 2022-06-29T15:46:27.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.131+0000 7f349de7f700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f349805fe10 con 0x7f3498060b90 2022-06-29T15:46:27.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.131+0000 7f349de7f700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34980600e0 con 0x7f349805fa30 2022-06-29T15:46:27.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.131+0000 7f3497fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3498060b90 0x7f349813f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:27.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.131+0000 7f3497fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3498060b90 0x7f349813f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48884/0 (socket says 172.21.15.47:48884) 2022-06-29T15:46:27.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.131+0000 7f3497fff700 1 -- 172.21.15.47:0/1677529577 learned_addr learned my addr 172.21.15.47:0/1677529577 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:46:27.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.131+0000 7f349ce7d700 1 --2- 172.21.15.47:0/1677529577 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f349805fa30 0x7f34980b1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:27.138 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.131+0000 7f3497fff700 1 -- 172.21.15.47:0/1677529577 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f349805fa30 msgr2=0x7f34980b1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:46:27.138 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.131+0000 7f3497fff700 1 --2- 172.21.15.47:0/1677529577 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f349805fa30 0x7f34980b1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:27.138 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.131+0000 7f3497fff700 1 -- 172.21.15.47:0/1677529577 >> v1:172.21.15.64:6789/0 conn(0x7f34980b2770 legacy=0x7f34980ae170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:27.138 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.131+0000 7f3497fff700 1 -- 172.21.15.47:0/1677529577 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3498060380 con 0x7f3498060b90 2022-06-29T15:46:27.139 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.131+0000 7f349ce7d700 1 --2- 172.21.15.47:0/1677529577 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f349805fa30 0x7f34980b1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:46:27.139 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.131+0000 7f3497fff700 1 --2- 172.21.15.47:0/1677529577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3498060b90 0x7f349813f470 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7f34980b2650 tx=0x7f348c006eb0).ready entity=mon.0 client_cookie=f39f057bcbc9d5e0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:27.139 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.131+0000 7f34977fe700 1 -- 172.21.15.47:0/1677529577 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f348c00d5c0 con 0x7f3498060b90 2022-06-29T15:46:27.139 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.131+0000 7f34977fe700 1 -- 172.21.15.47:0/1677529577 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f348c000e90 con 0x7f3498060b90 2022-06-29T15:46:27.140 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.131+0000 7f34977fe700 1 -- 172.21.15.47:0/1677529577 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f348c00d950 con 0x7f3498060b90 2022-06-29T15:46:27.140 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.131+0000 7f349de7f700 1 -- 172.21.15.47:0/1677529577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3498060b90 msgr2=0x7f349813f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:27.140 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.131+0000 7f349de7f700 1 --2- 172.21.15.47:0/1677529577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3498060b90 0x7f349813f470 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7f34980b2650 tx=0x7f348c006eb0).stop 2022-06-29T15:46:27.141 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.131+0000 7f349de7f700 1 -- 172.21.15.47:0/1677529577 shutdown_connections 2022-06-29T15:46:27.141 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.131+0000 7f349de7f700 1 --2- 172.21.15.47:0/1677529577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3498060b90 0x7f349813f470 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:27.141 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.131+0000 7f349de7f700 1 --2- 172.21.15.47:0/1677529577 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f349805fa30 0x7f34980b1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:27.141 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.131+0000 7f349de7f700 1 -- 172.21.15.47:0/1677529577 >> 172.21.15.47:0/1677529577 conn(0x7f3498062df0 msgr2=0x7f34980631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:27.142 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.131+0000 7f349de7f700 1 -- 172.21.15.47:0/1677529577 shutdown_connections 2022-06-29T15:46:27.142 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.131+0000 7f349de7f700 1 -- 172.21.15.47:0/1677529577 wait complete. 2022-06-29T15:46:27.142 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.131+0000 7f349de7f700 1 Processor -- start 2022-06-29T15:46:27.142 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.131+0000 7f349de7f700 1 -- start start 2022-06-29T15:46:27.145 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.135+0000 7f349de7f700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f349805fa30 0x7f3498063c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:27.146 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.135+0000 7f349de7f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3498060b90 0x7f3498099b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:27.146 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.135+0000 7f349de7f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34980b2770 0x7f349809a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:27.146 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.135+0000 7f349de7f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349809a550 con 0x7f3498060b90 2022-06-29T15:46:27.146 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.135+0000 7f349de7f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3498151140 con 0x7f34980b2770 2022-06-29T15:46:27.147 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.135+0000 7f349de7f700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f349809b150 con 0x7f349805fa30 2022-06-29T15:46:27.147 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.135+0000 7f3497fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3498060b90 0x7f3498099b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:27.147 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.135+0000 7f349ce7d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f349805fa30 0x7f3498063c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:27.147 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.135+0000 7f3497fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3498060b90 0x7f3498099b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48894/0 (socket says 172.21.15.47:48894) 2022-06-29T15:46:27.148 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.135+0000 7f3497fff700 1 -- 172.21.15.47:0/2119890361 learned_addr learned my addr 172.21.15.47:0/2119890361 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:46:27.148 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.135+0000 7f349d67e700 1 --2- 172.21.15.47:0/2119890361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34980b2770 0x7f349809a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:27.148 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.135+0000 7f349ce7d700 1 -- 172.21.15.47:0/2119890361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34980b2770 msgr2=0x7f349809a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:27.148 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.135+0000 7f349ce7d700 1 --2- 172.21.15.47:0/2119890361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34980b2770 0x7f349809a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:27.148 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.135+0000 7f349ce7d700 1 -- 172.21.15.47:0/2119890361 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3498060b90 msgr2=0x7f3498099b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:27.149 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.135+0000 7f349ce7d700 1 --2- 172.21.15.47:0/2119890361 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3498060b90 0x7f3498099b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:27.149 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.135+0000 7f349ce7d700 1 -- 172.21.15.47:0/2119890361 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f348c00b1e0 con 0x7f349805fa30 2022-06-29T15:46:27.149 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.135+0000 7f3497fff700 1 --2- 172.21.15.47:0/2119890361 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3498060b90 0x7f3498099b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:46:27.149 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.135+0000 7f349ce7d700 1 --2- 172.21.15.47:0/2119890361 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f349805fa30 0x7f3498063c70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f348400a000 tx=0x7f3484008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:27.149 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.135+0000 7f3495ffb700 1 -- 172.21.15.47:0/2119890361 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f34840106e0 con 0x7f349805fa30 2022-06-29T15:46:27.150 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.135+0000 7f349de7f700 1 -- 172.21.15.47:0/2119890361 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3498151370 con 0x7f349805fa30 2022-06-29T15:46:27.150 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.135+0000 7f3495ffb700 1 -- 172.21.15.47:0/2119890361 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3484010840 con 0x7f349805fa30 2022-06-29T15:46:27.150 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.135+0000 7f349de7f700 1 -- 172.21.15.47:0/2119890361 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f349809b3f0 con 0x7f349805fa30 2022-06-29T15:46:27.151 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.135+0000 7f3495ffb700 1 -- 172.21.15.47:0/2119890361 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3484010ba0 con 0x7f349805fa30 2022-06-29T15:46:27.151 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.135+0000 7f349de7f700 1 -- 172.21.15.47:0/2119890361 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3498151550 con 0x7f349805fa30 2022-06-29T15:46:27.151 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.139+0000 7f3495ffb700 1 -- 172.21.15.47:0/2119890361 <== mon.2 v2:172.21.15.64:3301/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f348401a030 con 0x7f349805fa30 2022-06-29T15:46:27.151 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.139+0000 7f3495ffb700 1 --2- 172.21.15.47:0/2119890361 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3478053950 0x7f3478055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:46:27.152 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.139+0000 7f3495ffb700 1 -- 172.21.15.47:0/2119890361 <== mon.2 v2:172.21.15.64:3301/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f348406bbc0 con 0x7f349805fa30 2022-06-29T15:46:27.152 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.139+0000 7f3497fff700 1 --2- 172.21.15.47:0/2119890361 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3478053950 0x7f3478055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:46:27.152 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.139+0000 7f3497fff700 1 --2- 172.21.15.47:0/2119890361 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3478053950 0x7f3478055e10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f348c01c000 tx=0x7f348c015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:46:27.153 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.147+0000 7f3495ffb700 1 -- 172.21.15.47:0/2119890361 <== mon.2 v2:172.21.15.64:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3498151550 con 0x7f349805fa30 2022-06-29T15:46:27.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.315+0000 7f349de7f700 1 -- 172.21.15.47:0/2119890361 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3498151550 con 0x7f349805fa30 2022-06-29T15:46:27.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.319+0000 7f3495ffb700 1 -- 172.21.15.47:0/2119890361 <== mon.2 v2:172.21.15.64:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+2492 (secure 0 0 0) 0x7f3498151550 con 0x7f349805fa30 2022-06-29T15:46:27.321 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:46:27.322 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":24,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5595,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1798995166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1798995166},{"type":"v1","addr":"172.21.15.47:6835","nonce":1798995166}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21},{"gid":5601,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/1641478219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1641478219},{"type":"v1","addr":"172.21.15.64:6833","nonce":1641478219}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:46:01.294125+0000","modified":"2022-06-29T15:46:02.296567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5607":{"gid":5607,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6837/2427298469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2427298469},{"type":"v1","addr":"172.21.15.47:6837","nonce":2427298469}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2022-06-29T15:46:27.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.319+0000 7f349de7f700 1 -- 172.21.15.47:0/2119890361 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3478053950 msgr2=0x7f3478055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:27.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.319+0000 7f349de7f700 1 --2- 172.21.15.47:0/2119890361 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3478053950 0x7f3478055e10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f348c01c000 tx=0x7f348c015000).stop 2022-06-29T15:46:27.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.319+0000 7f349de7f700 1 -- 172.21.15.47:0/2119890361 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f349805fa30 msgr2=0x7f3498063c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:46:27.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.319+0000 7f349de7f700 1 --2- 172.21.15.47:0/2119890361 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f349805fa30 0x7f3498063c70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f348400a000 tx=0x7f3484008040).stop 2022-06-29T15:46:27.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.319+0000 7f349de7f700 1 -- 172.21.15.47:0/2119890361 shutdown_connections 2022-06-29T15:46:27.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.319+0000 7f349de7f700 1 --2- 172.21.15.47:0/2119890361 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3478053950 0x7f3478055e10 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:27.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.319+0000 7f349de7f700 1 --2- 172.21.15.47:0/2119890361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34980b2770 0x7f349809a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:27.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.319+0000 7f349de7f700 1 --2- 172.21.15.47:0/2119890361 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3498060b90 0x7f3498099b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:27.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.319+0000 7f349de7f700 1 --2- 172.21.15.47:0/2119890361 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f349805fa30 0x7f3498063c70 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:46:27.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.319+0000 7f349de7f700 1 -- 172.21.15.47:0/2119890361 >> 172.21.15.47:0/2119890361 conn(0x7f3498062df0 msgr2=0x7f34980ae0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:46:27.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.319+0000 7f349de7f700 1 -- 172.21.15.47:0/2119890361 shutdown_connections 2022-06-29T15:46:27.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:46:27.319+0000 7f349de7f700 1 -- 172.21.15.47:0/2119890361 wait complete. 2022-06-29T15:46:27.327 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 24 2022-06-29T15:46:27.638 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-29T15:46:01.294125+0000', 'modified': '2022-06-29T15:46:02.296567+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5607}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5607': {'gid': 5607, 'name': 'a', 'rank': 0, 'incarnation': 23, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.47:6837/2427298469', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6836', 'nonce': 2427298469}, {'type': 'v1', 'addr': '172.21.15.47:6837', 'nonce': 2427298469}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-29T15:46:27.639 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-29T15:46:27.639 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-06-29T15:46:27.758 INFO:teuthology.orchestra.run.smithi047.stdout:{"cluster_fsid":"9a68894a-53ce-400e-838b-14eab91b834c","whoami":0,"id":5607,"want_state":"up:active","state":"up:active","rank_uptime":26.452004036999998,"mdsmap_epoch":24,"osdmap_epoch":58,"osdmap_epoch_barrier":56,"uptime":30.817135500999999} 2022-06-29T15:46:27.759 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9a68894a-53ce-400e-838b-14eab91b834c", "id": 5607, "mdsmap_epoch": 24, "osdmap_epoch": 58, "osdmap_epoch_barrier": 56, "rank_uptime": 26.452004037, "state": "up:active", "uptime": 30.817135501, "want_state": "up:active", "whoami": 0 } 2022-06-29T15:46:27.760 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-29T15:46:27.760 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-06-29T15:46:27.761 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi099.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-06-29T15:46:27.761 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:46:27.762 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:46:27.771 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2022-06-29T15:46:27.831 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:46:27.831 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:46:27.842 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:46:27.842 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:46:27.843 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:46:27.844 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2022-06-29T15:46:27.850 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-29T15:46:27.851 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:46:27.854 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2022-06-29T15:46:27.872 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:46:27.873 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:46:27.928 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:2022-06-29T15:46:27.925+0000 7f3a7a7d1080 -1 init, newargv = 0x55963322b770 newargc=9ceph-fuse[12651 2022-06-29T15:46:27.929 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:]: starting ceph client 2022-06-29T15:46:27.938 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:46:27.940 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:46:27.941 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:46:27.942 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2022-06-29T15:46:27.942 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[12651]: starting fuse 2022-06-29T15:46:27.949 INFO:teuthology.orchestra.run.smithi099.stdout:52 2022-06-29T15:46:27.950 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-29T15:46:27.952 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:46:27.952 DEBUG:teuthology.orchestra.run.smithi099:> sudo python3 -c ' 2022-06-29T15:46:27.953 DEBUG:teuthology.orchestra.run.smithi099:> import glob 2022-06-29T15:46:27.954 DEBUG:teuthology.orchestra.run.smithi099:> import re 2022-06-29T15:46:27.955 DEBUG:teuthology.orchestra.run.smithi099:> import os 2022-06-29T15:46:27.955 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2022-06-29T15:46:27.956 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:46:27.957 DEBUG:teuthology.orchestra.run.smithi099:> def find_socket(client_name): 2022-06-29T15:46:27.958 DEBUG:teuthology.orchestra.run.smithi099:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-06-29T15:46:27.958 DEBUG:teuthology.orchestra.run.smithi099:> files = glob.glob(asok_path) 2022-06-29T15:46:27.959 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:46:27.960 DEBUG:teuthology.orchestra.run.smithi099:> # Given a non-glob path, it better be there 2022-06-29T15:46:27.960 DEBUG:teuthology.orchestra.run.smithi099:> if "*" not in asok_path: 2022-06-29T15:46:27.961 DEBUG:teuthology.orchestra.run.smithi099:> assert(len(files) == 1) 2022-06-29T15:46:27.962 DEBUG:teuthology.orchestra.run.smithi099:> return files[0] 2022-06-29T15:46:27.962 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:46:27.963 DEBUG:teuthology.orchestra.run.smithi099:> for f in files: 2022-06-29T15:46:27.963 DEBUG:teuthology.orchestra.run.smithi099:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-29T15:46:27.964 DEBUG:teuthology.orchestra.run.smithi099:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-29T15:46:27.964 DEBUG:teuthology.orchestra.run.smithi099:> return f 2022-06-29T15:46:27.964 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-29T15:46:27.965 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:46:27.965 DEBUG:teuthology.orchestra.run.smithi099:> print(find_socket("client.0")) 2022-06-29T15:46:27.966 DEBUG:teuthology.orchestra.run.smithi099:> ' 2022-06-29T15:46:28.011 INFO:teuthology.orchestra.run.smithi099.stdout:/var/run/ceph/ceph-client.0.12651.asok 2022-06-29T15:46:28.015 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.12651.asok 2022-06-29T15:46:28.016 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:46:28.017 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.12651.asok status 2022-06-29T15:46:28.108 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2022-06-29T15:46:28.109 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": { 2022-06-29T15:46:28.109 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_sha1": "f25ffac72e5a9e63860ff7deb6767a11f0559a16", 2022-06-29T15:46:28.109 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 15.2.16-304-gf25ffac7 (f25ffac72e5a9e63860ff7deb6767a11f0559a16) octopus (stable)", 2022-06-29T15:46:28.109 INFO:teuthology.orchestra.run.smithi099.stdout: "entity_id": "0", 2022-06-29T15:46:28.110 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": "smithi099", 2022-06-29T15:46:28.110 INFO:teuthology.orchestra.run.smithi099.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-06-29T15:46:28.110 INFO:teuthology.orchestra.run.smithi099.stdout: "pid": "12651", 2022-06-29T15:46:28.110 INFO:teuthology.orchestra.run.smithi099.stdout: "root": "/" 2022-06-29T15:46:28.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:46:28.111 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_count": 0, 2022-06-29T15:46:28.111 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_pinned_count": 0, 2022-06-29T15:46:28.111 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 5794, 2022-06-29T15:46:28.111 INFO:teuthology.orchestra.run.smithi099.stdout: "inst": { 2022-06-29T15:46:28.112 INFO:teuthology.orchestra.run.smithi099.stdout: "name": { 2022-06-29T15:46:28.112 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "client", 2022-06-29T15:46:28.112 INFO:teuthology.orchestra.run.smithi099.stdout: "num": 5794 2022-06-29T15:46:28.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:46:28.113 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2022-06-29T15:46:28.113 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2022-06-29T15:46:28.113 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:0", 2022-06-29T15:46:28.113 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3800200256 2022-06-29T15:46:28.113 INFO:teuthology.orchestra.run.smithi099.stdout: } 2022-06-29T15:46:28.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:46:28.114 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2022-06-29T15:46:28.114 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2022-06-29T15:46:28.114 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:0", 2022-06-29T15:46:28.115 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3800200256 2022-06-29T15:46:28.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:46:28.115 INFO:teuthology.orchestra.run.smithi099.stdout: "inst_str": "client.5794 172.21.15.99:0/3800200256", 2022-06-29T15:46:28.115 INFO:teuthology.orchestra.run.smithi099.stdout: "addr_str": "172.21.15.99:0/3800200256", 2022-06-29T15:46:28.115 INFO:teuthology.orchestra.run.smithi099.stdout: "inode_count": 1, 2022-06-29T15:46:28.116 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_epoch": 24, 2022-06-29T15:46:28.116 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch": 58, 2022-06-29T15:46:28.116 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch_barrier": 0, 2022-06-29T15:46:28.116 INFO:teuthology.orchestra.run.smithi099.stdout: "blacklisted": false, 2022-06-29T15:46:28.117 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs" 2022-06-29T15:46:28.117 INFO:teuthology.orchestra.run.smithi099.stdout:} 2022-06-29T15:46:28.118 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:46:28.119 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:46:28.119 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:46:28.130 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2022-06-29T15:46:28.131 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-29T15:46:28.132 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:46:28.132 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:46:41.560 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:46:41.564 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:46:41.567 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:46:41.570 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:47:11.729 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:47:11.733 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:47:11.737 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:47:11.740 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:47:21.946 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-06-29T15:47:21.947 DEBUG:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:47:21.969 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:47:21.996 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:47:22.019 DEBUG:teuthology.orchestra.run.smithi164:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:47:22.042 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-06-29T15:47:22.043 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:47:22.044 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:47:22.045 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:47:22.072 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2022-06-29T15:47:22.075 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-29T15:47:22.076 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi099.front.sepia.ceph.com... 2022-06-29T15:47:22.076 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-06-29T15:47:22.077 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:47:22.100 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[12651]: fuse finished with error 0 and tester_r 0 2022-06-29T15:47:22.101 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:47:22.103 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:47:22.103 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:47:22.167 INFO:teuthology.orchestra.run.smithi099.stdout:ext2/ext3 2022-06-29T15:47:22.169 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-06-29T15:47:22.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-29T15:47:22.201 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:47:22.252 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:47:22.253 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:47:22.253 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:47:22.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T15:47:22.263 INFO:teuthology.orchestra.run.smithi164.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-06-29T15:47:22.264 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-06-29T15:47:22.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-29T15:47:22.266 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:47:22.274 INFO:teuthology.orchestra.run.smithi164.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-06-29T15:47:22.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T15:47:22.277 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-06-29T15:47:22.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.387+0000 7fd88ffff700 1 Processor -- start 2022-06-29T15:47:22.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.387+0000 7fd88ffff700 1 -- start start 2022-06-29T15:47:22.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.387+0000 7fd88ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd890060eb0 0x7fd8900b1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:22.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.387+0000 7fd88ffff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd890145090 0x7fd8900b2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:22.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.387+0000 7fd88ffff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8900a9120 con 0x7fd8900b5c60 2022-06-29T15:47:22.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.387+0000 7fd88ffff700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8900b0b80 con 0x7fd890145090 2022-06-29T15:47:22.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.387+0000 7fd88ffff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8900a89e0 con 0x7fd890060eb0 2022-06-29T15:47:22.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.387+0000 7fd88f7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd890145090 0x7fd8900b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:22.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.387+0000 7fd88f7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd890145090 0x7fd8900b2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48902/0 (socket says 172.21.15.47:48902) 2022-06-29T15:47:22.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.387+0000 7fd88f7fe700 1 -- 172.21.15.47:0/251547933 learned_addr learned my addr 172.21.15.47:0/251547933 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:22.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.387+0000 7fd88e7fc700 1 --2- 172.21.15.47:0/251547933 >> v2:172.21.15.64:3300/0 conn(0x7fd890060eb0 0x7fd8900b1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:22.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.387+0000 7fd88f7fe700 1 -- 172.21.15.47:0/251547933 >> v2:172.21.15.64:3300/0 conn(0x7fd890060eb0 msgr2=0x7fd8900b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:22.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.387+0000 7fd88f7fe700 1 --2- 172.21.15.47:0/251547933 >> v2:172.21.15.64:3300/0 conn(0x7fd890060eb0 0x7fd8900b1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:22.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.387+0000 7fd88f7fe700 1 -- 172.21.15.47:0/251547933 >> v1:172.21.15.64:6789/0 conn(0x7fd8900b5c60 legacy=0x7fd890144980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:22.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.387+0000 7fd88f7fe700 1 -- 172.21.15.47:0/251547933 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8900a8c80 con 0x7fd890145090 2022-06-29T15:47:22.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.391+0000 7fd88f7fe700 1 --2- 172.21.15.47:0/251547933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd890145090 0x7fd8900b2320 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7fd890062280 tx=0x7fd888009290).ready entity=mon.0 client_cookie=51196f84756c6396 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:22.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.391+0000 7fd88dffb700 1 -- 172.21.15.47:0/251547933 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd88800b830 con 0x7fd890145090 2022-06-29T15:47:22.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.391+0000 7fd88dffb700 1 -- 172.21.15.47:0/251547933 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd88800b990 con 0x7fd890145090 2022-06-29T15:47:22.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.391+0000 7fd88dffb700 1 -- 172.21.15.47:0/251547933 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd88800bb40 con 0x7fd890145090 2022-06-29T15:47:22.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.391+0000 7fd88ffff700 1 -- 172.21.15.47:0/251547933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd890145090 msgr2=0x7fd8900b2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:22.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.391+0000 7fd88ffff700 1 --2- 172.21.15.47:0/251547933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd890145090 0x7fd8900b2320 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7fd890062280 tx=0x7fd888009290).stop 2022-06-29T15:47:22.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.391+0000 7fd88ffff700 1 -- 172.21.15.47:0/251547933 shutdown_connections 2022-06-29T15:47:22.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.391+0000 7fd88ffff700 1 --2- 172.21.15.47:0/251547933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd890145090 0x7fd8900b2320 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:22.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.391+0000 7fd88ffff700 1 --2- 172.21.15.47:0/251547933 >> v2:172.21.15.64:3300/0 conn(0x7fd890060eb0 0x7fd8900b1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:22.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.391+0000 7fd88ffff700 1 -- 172.21.15.47:0/251547933 >> 172.21.15.47:0/251547933 conn(0x7fd890062560 msgr2=0x7fd890062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:22.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.391+0000 7fd88ffff700 1 -- 172.21.15.47:0/251547933 shutdown_connections 2022-06-29T15:47:22.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.391+0000 7fd88ffff700 1 -- 172.21.15.47:0/251547933 wait complete. 2022-06-29T15:47:22.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.391+0000 7fd88ffff700 1 Processor -- start 2022-06-29T15:47:22.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.395+0000 7fd88ffff700 1 -- start start 2022-06-29T15:47:22.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.395+0000 7fd88ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd890060eb0 0x7fd890148980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:22.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.395+0000 7fd88ffff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd8900b5c60 0x7fd8900b4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:22.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.395+0000 7fd88ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd890145090 0x7fd89013d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:22.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.395+0000 7fd88ffff700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8900ac8a0 con 0x7fd890060eb0 2022-06-29T15:47:22.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.395+0000 7fd88ffff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8900ad750 con 0x7fd890145090 2022-06-29T15:47:22.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.395+0000 7fd88ffff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8900ad9f0 con 0x7fd8900b5c60 2022-06-29T15:47:22.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.395+0000 7fd88effd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd890060eb0 0x7fd890148980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:22.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.395+0000 7fd88e7fc700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd8900b5c60 0x7fd8900b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:22.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.395+0000 7fd88effd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd890060eb0 0x7fd890148980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48906/0 (socket says 172.21.15.47:48906) 2022-06-29T15:47:22.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.395+0000 7fd88e7fc700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd8900b5c60 0x7fd8900b4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:46006/0 (socket says 172.21.15.47:46006) 2022-06-29T15:47:22.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.395+0000 7fd88f7fe700 1 --2- 172.21.15.47:0/871677378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd890145090 0x7fd89013d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:22.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.395+0000 7fd88effd700 1 -- 172.21.15.47:0/871677378 learned_addr learned my addr 172.21.15.47:0/871677378 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:22.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.395+0000 7fd88effd700 1 -- 172.21.15.47:0/871677378 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd8900b5c60 msgr2=0x7fd8900b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:22.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.395+0000 7fd88effd700 1 --2- 172.21.15.47:0/871677378 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd8900b5c60 0x7fd8900b4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:22.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.395+0000 7fd88effd700 1 -- 172.21.15.47:0/871677378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd890145090 msgr2=0x7fd89013d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:22.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.395+0000 7fd88effd700 1 --2- 172.21.15.47:0/871677378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd890145090 0x7fd89013d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:22.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.395+0000 7fd88effd700 1 -- 172.21.15.47:0/871677378 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd888009530 con 0x7fd890060eb0 2022-06-29T15:47:22.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.395+0000 7fd88effd700 1 --2- 172.21.15.47:0/871677378 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd890060eb0 0x7fd890148980 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7fd884001ff0 tx=0x7fd884007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:22.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.395+0000 7fd87ffff700 1 -- 172.21.15.47:0/871677378 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd88400ce50 con 0x7fd890060eb0 2022-06-29T15:47:22.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.395+0000 7fd87ffff700 1 -- 172.21.15.47:0/871677378 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd884007030 con 0x7fd890060eb0 2022-06-29T15:47:22.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.395+0000 7fd88ffff700 1 -- 172.21.15.47:0/871677378 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd89015a660 con 0x7fd890060eb0 2022-06-29T15:47:22.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.395+0000 7fd87ffff700 1 -- 172.21.15.47:0/871677378 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd884007390 con 0x7fd890060eb0 2022-06-29T15:47:22.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.395+0000 7fd88ffff700 1 -- 172.21.15.47:0/871677378 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8900ade60 con 0x7fd890060eb0 2022-06-29T15:47:22.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.399+0000 7fd88ffff700 1 -- 172.21.15.47:0/871677378 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8900573d0 con 0x7fd890060eb0 2022-06-29T15:47:22.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.399+0000 7fd87ffff700 1 -- 172.21.15.47:0/871677378 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd884010030 con 0x7fd890060eb0 2022-06-29T15:47:22.424 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.399+0000 7fd87ffff700 1 --2- 172.21.15.47:0/871677378 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd86c053950 0x7fd86c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:22.424 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.399+0000 7fd87ffff700 1 -- 172.21.15.47:0/871677378 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd88406bc10 con 0x7fd890060eb0 2022-06-29T15:47:22.425 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.399+0000 7fd88e7fc700 1 --2- 172.21.15.47:0/871677378 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd86c053950 0x7fd86c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:22.425 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.399+0000 7fd88e7fc700 1 --2- 172.21.15.47:0/871677378 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd86c053950 0x7fd86c055e10 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7fd88000c000 tx=0x7fd880008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:22.426 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.403+0000 7fd87ffff700 1 -- 172.21.15.47:0/871677378 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd8840429d0 con 0x7fd890060eb0 2022-06-29T15:47:22.533 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.531+0000 7fd88ffff700 1 -- 172.21.15.47:0/871677378 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fd890069e20 con 0x7fd890060eb0 2022-06-29T15:47:22.536 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.535+0000 7fd87ffff700 1 -- 172.21.15.47:0/871677378 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v58) v1 ==== 81+0+34039 (secure 0 0 0) 0x7fd884045ff0 con 0x7fd890060eb0 2022-06-29T15:47:22.537 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:47:22.537 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2022-06-29T15:47:22.538 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 58, 2022-06-29T15:47:22.538 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "9a68894a-53ce-400e-838b-14eab91b834c", 2022-06-29T15:47:22.538 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2022-06-29T15:42:14.046361+0000", 2022-06-29T15:47:22.539 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2022-06-29T15:46:03.307251+0000", 2022-06-29T15:47:22.539 INFO:teuthology.orchestra.run.smithi047.stdout: "last_up_change": "2022-06-29T15:42:20.929789+0000", 2022-06-29T15:47:22.540 INFO:teuthology.orchestra.run.smithi047.stdout: "last_in_change": "2022-06-29T15:42:20.929789+0000", 2022-06-29T15:47:22.540 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-06-29T15:47:22.541 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_num": 5799936, 2022-06-29T15:47:22.541 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_set": [ 2022-06-29T15:47:22.542 INFO:teuthology.orchestra.run.smithi047.stdout: "pglog_hardlimit", 2022-06-29T15:47:22.542 INFO:teuthology.orchestra.run.smithi047.stdout: "purged_snapdirs", 2022-06-29T15:47:22.542 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_deletes", 2022-06-29T15:47:22.542 INFO:teuthology.orchestra.run.smithi047.stdout: "sortbitwise" 2022-06-29T15:47:22.543 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:47:22.543 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_version": 5, 2022-06-29T15:47:22.544 INFO:teuthology.orchestra.run.smithi047.stdout: "full_ratio": 0.94999998807907104, 2022-06-29T15:47:22.544 INFO:teuthology.orchestra.run.smithi047.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-06-29T15:47:22.544 INFO:teuthology.orchestra.run.smithi047.stdout: "nearfull_ratio": 0.85000002384185791, 2022-06-29T15:47:22.544 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_snapshot": "", 2022-06-29T15:47:22.545 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_max": 10, 2022-06-29T15:47:22.545 INFO:teuthology.orchestra.run.smithi047.stdout: "max_osd": 8, 2022-06-29T15:47:22.545 INFO:teuthology.orchestra.run.smithi047.stdout: "require_min_compat_client": "jewel", 2022-06-29T15:47:22.545 INFO:teuthology.orchestra.run.smithi047.stdout: "min_compat_client": "jewel", 2022-06-29T15:47:22.546 INFO:teuthology.orchestra.run.smithi047.stdout: "require_osd_release": "octopus", 2022-06-29T15:47:22.546 INFO:teuthology.orchestra.run.smithi047.stdout: "pools": [ 2022-06-29T15:47:22.546 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.547 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 1, 2022-06-29T15:47:22.547 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "device_health_metrics", 2022-06-29T15:47:22.547 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:42:16.971431+0000", 2022-06-29T15:47:22.547 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:47:22.548 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:47:22.548 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:47:22.548 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:47:22.549 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:47:22.549 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:47:22.549 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:47:22.550 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:47:22.550 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 1, 2022-06-29T15:47:22.550 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 1, 2022-06-29T15:47:22.551 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 1, 2022-06-29T15:47:22.551 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 1, 2022-06-29T15:47:22.551 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 1, 2022-06-29T15:47:22.552 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:47:22.552 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:47:22.552 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:47:22.553 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:47:22.553 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:47:22.553 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:47:22.554 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:47:22.554 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.554 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "16", 2022-06-29T15:47:22.555 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:47:22.555 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:47:22.555 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:47:22.555 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:47:22.556 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:47:22.556 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:47:22.556 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:47:22.556 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:47:22.557 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:47:22.557 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:47:22.557 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:47:22.558 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:47:22.558 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:47:22.558 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:47:22.558 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:47:22.559 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:47:22.559 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:47:22.559 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:47:22.560 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:47:22.560 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:47:22.560 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:47:22.560 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:47:22.561 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:47:22.561 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:47:22.561 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:47:22.561 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:47:22.562 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.562 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:47:22.562 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:47:22.563 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:47:22.563 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:47:22.563 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:47:22.563 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:47:22.564 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:47:22.564 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:47:22.564 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:47:22.564 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:47:22.565 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:47:22.565 INFO:teuthology.orchestra.run.smithi047.stdout: "options": { 2022-06-29T15:47:22.565 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_min": 1 2022-06-29T15:47:22.565 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.566 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:47:22.566 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_devicehealth": {} 2022-06-29T15:47:22.566 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:22.566 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.566 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.567 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 2, 2022-06-29T15:47:22.567 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "rbd", 2022-06-29T15:47:22.567 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:42:25.290884+0000", 2022-06-29T15:47:22.567 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:47:22.568 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:47:22.568 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:47:22.568 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:47:22.568 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:47:22.569 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:47:22.569 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:47:22.569 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:47:22.569 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2022-06-29T15:47:22.570 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2022-06-29T15:47:22.570 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2022-06-29T15:47:22.570 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2022-06-29T15:47:22.570 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2022-06-29T15:47:22.571 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:47:22.571 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:47:22.571 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:47:22.571 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:47:22.572 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:47:22.572 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:47:22.572 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:47:22.572 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.573 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "19", 2022-06-29T15:47:22.573 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:47:22.573 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:47:22.573 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:47:22.574 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:47:22.574 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:47:22.574 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:47:22.574 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:47:22.575 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:47:22.575 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:47:22.575 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:47:22.575 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:47:22.576 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:47:22.576 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:47:22.576 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:47:22.576 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:47:22.576 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:47:22.577 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:47:22.577 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:47:22.577 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:47:22.577 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:47:22.578 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:47:22.578 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:47:22.578 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:47:22.578 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:47:22.579 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:47:22.579 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:47:22.579 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.579 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:47:22.580 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:47:22.580 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:47:22.580 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:47:22.580 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:47:22.580 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:47:22.581 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:47:22.581 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:47:22.581 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:47:22.581 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:47:22.582 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:47:22.582 INFO:teuthology.orchestra.run.smithi047.stdout: "options": {}, 2022-06-29T15:47:22.582 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:47:22.582 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd": {} 2022-06-29T15:47:22.583 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:22.583 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.583 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.583 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 9, 2022-06-29T15:47:22.584 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "cephfs_metadata", 2022-06-29T15:47:22.584 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:45:59.710738+0000", 2022-06-29T15:47:22.584 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:47:22.584 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:47:22.585 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:47:22.585 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:47:22.585 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:47:22.585 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:47:22.586 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:47:22.586 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:47:22.586 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2022-06-29T15:47:22.586 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2022-06-29T15:47:22.587 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2022-06-29T15:47:22.587 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2022-06-29T15:47:22.587 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2022-06-29T15:47:22.587 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:47:22.587 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:47:22.588 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:47:22.588 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:47:22.588 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:47:22.588 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:47:22.589 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:47:22.589 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.589 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "57", 2022-06-29T15:47:22.589 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:47:22.590 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:47:22.590 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:47:22.590 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:47:22.590 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:47:22.591 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:47:22.591 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:47:22.591 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:47:22.591 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:47:22.592 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:47:22.592 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:47:22.592 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:47:22.592 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:47:22.593 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:47:22.593 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:47:22.593 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:47:22.593 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:47:22.594 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:47:22.594 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:47:22.594 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:47:22.594 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:47:22.595 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:47:22.595 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:47:22.595 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:47:22.595 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:47:22.596 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:47:22.596 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.596 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:47:22.596 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:47:22.597 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:47:22.597 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:47:22.597 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:47:22.597 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:47:22.597 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:47:22.598 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:47:22.598 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:47:22.598 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:47:22.598 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:47:22.599 INFO:teuthology.orchestra.run.smithi047.stdout: "options": { 2022-06-29T15:47:22.599 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_bias": 4, 2022-06-29T15:47:22.599 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_min": 16, 2022-06-29T15:47:22.599 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_priority": 5 2022-06-29T15:47:22.600 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.600 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:47:22.600 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs": { 2022-06-29T15:47:22.600 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": "cephfs" 2022-06-29T15:47:22.600 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:22.601 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:22.601 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.601 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.602 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 10, 2022-06-29T15:47:22.602 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "cephfs_data", 2022-06-29T15:47:22.602 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:46:00.582445+0000", 2022-06-29T15:47:22.602 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:47:22.603 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:47:22.603 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:47:22.603 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:47:22.603 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:47:22.604 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:47:22.604 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:47:22.604 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:47:22.604 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2022-06-29T15:47:22.604 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2022-06-29T15:47:22.605 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2022-06-29T15:47:22.605 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2022-06-29T15:47:22.605 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2022-06-29T15:47:22.605 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:47:22.606 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:47:22.606 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:47:22.606 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:47:22.606 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:47:22.607 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:47:22.607 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:47:22.607 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.607 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "58", 2022-06-29T15:47:22.608 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:47:22.608 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:47:22.608 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:47:22.608 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:47:22.608 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:47:22.609 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:47:22.609 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:47:22.609 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:47:22.609 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:47:22.610 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:47:22.610 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:47:22.610 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:47:22.610 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:47:22.611 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:47:22.611 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:47:22.611 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:47:22.611 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:47:22.611 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:47:22.612 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:47:22.612 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:47:22.612 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:47:22.612 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:47:22.613 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:47:22.613 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:47:22.613 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:47:22.613 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:47:22.614 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.614 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:47:22.614 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:47:22.614 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:47:22.615 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:47:22.615 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:47:22.615 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:47:22.615 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:47:22.615 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:47:22.616 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:47:22.616 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:47:22.616 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:47:22.616 INFO:teuthology.orchestra.run.smithi047.stdout: "options": {}, 2022-06-29T15:47:22.617 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:47:22.617 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs": { 2022-06-29T15:47:22.617 INFO:teuthology.orchestra.run.smithi047.stdout: "data": "cephfs" 2022-06-29T15:47:22.617 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:22.618 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:22.618 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:22.618 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:47:22.618 INFO:teuthology.orchestra.run.smithi047.stdout: "osds": [ 2022-06-29T15:47:22.618 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.619 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2022-06-29T15:47:22.619 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "e23eb023-4706-4201-bf72-4bf19c912c5d", 2022-06-29T15:47:22.619 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:47:22.619 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:47:22.620 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:47:22.620 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:47:22.620 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:47:22.620 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:47:22.621 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:47:22.621 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 55, 2022-06-29T15:47:22.621 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:47:22.621 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:47:22.622 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:47:22.622 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:22.622 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.622 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:22.623 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6818", 2022-06-29T15:47:22.623 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:47:22.623 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.623 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.623 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:22.624 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6819", 2022-06-29T15:47:22.624 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:47:22.624 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:22.624 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:22.625 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.625 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:47:22.625 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:22.625 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.626 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:22.626 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6820", 2022-06-29T15:47:22.626 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:47:22.626 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.627 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.627 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:22.627 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6821", 2022-06-29T15:47:22.627 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:47:22.628 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:22.628 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:22.628 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.628 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:47:22.629 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:22.629 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.629 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:22.629 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6824", 2022-06-29T15:47:22.630 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:47:22.630 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.630 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.630 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:22.631 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6825", 2022-06-29T15:47:22.631 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:47:22.631 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:22.631 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:22.631 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.632 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:47:22.632 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:22.632 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.632 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:22.633 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6822", 2022-06-29T15:47:22.633 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:47:22.633 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.633 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.634 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:22.634 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6823", 2022-06-29T15:47:22.634 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:47:22.634 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:22.635 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:22.635 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.635 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6819/13096", 2022-06-29T15:47:22.635 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6821/13096", 2022-06-29T15:47:22.636 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6825/13096", 2022-06-29T15:47:22.636 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6823/13096", 2022-06-29T15:47:22.637 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:47:22.637 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:47:22.637 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:47:22.637 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:22.638 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.638 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.638 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2022-06-29T15:47:22.638 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "228bb08d-2d64-43d9-9560-bfaeff154452", 2022-06-29T15:47:22.639 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:47:22.639 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:47:22.639 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:47:22.639 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:47:22.639 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:47:22.640 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:47:22.640 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:47:22.640 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 55, 2022-06-29T15:47:22.640 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:47:22.641 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:47:22.641 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:47:22.641 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:22.641 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.642 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:22.642 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6826", 2022-06-29T15:47:22.642 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:47:22.642 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.643 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.643 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:22.643 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6827", 2022-06-29T15:47:22.643 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:47:22.644 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:22.644 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:22.644 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.645 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:47:22.645 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:22.645 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.645 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:22.645 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6828", 2022-06-29T15:47:22.646 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:47:22.646 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.646 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.647 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:22.647 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6829", 2022-06-29T15:47:22.647 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:47:22.647 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:22.648 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:22.648 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.648 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:47:22.648 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:22.649 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.649 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:22.649 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6832", 2022-06-29T15:47:22.649 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:47:22.650 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.650 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.650 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:22.650 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6833", 2022-06-29T15:47:22.651 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:47:22.651 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:22.651 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:22.651 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.651 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:47:22.652 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:22.652 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.652 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:22.652 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6830", 2022-06-29T15:47:22.653 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:47:22.653 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.653 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.654 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:22.654 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6831", 2022-06-29T15:47:22.655 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:47:22.655 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:22.656 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:22.656 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.657 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6827/13100", 2022-06-29T15:47:22.657 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6829/13100", 2022-06-29T15:47:22.658 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6833/13100", 2022-06-29T15:47:22.658 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6831/13100", 2022-06-29T15:47:22.659 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:47:22.659 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:47:22.659 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:47:22.660 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:22.660 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.660 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.660 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2022-06-29T15:47:22.661 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "797a5452-1510-4640-a111-c9c2fcf707eb", 2022-06-29T15:47:22.661 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:47:22.661 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:47:22.661 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:47:22.662 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:47:22.662 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:47:22.662 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:47:22.662 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:47:22.663 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 56, 2022-06-29T15:47:22.663 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:47:22.663 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:47:22.664 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:47:22.664 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:22.664 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.664 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:22.665 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6810", 2022-06-29T15:47:22.665 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:47:22.665 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.665 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.666 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:22.666 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6811", 2022-06-29T15:47:22.666 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:47:22.666 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:22.667 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:22.667 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.667 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:47:22.667 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:22.668 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.668 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:22.668 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6812", 2022-06-29T15:47:22.668 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:47:22.669 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.669 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.669 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:22.670 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6813", 2022-06-29T15:47:22.670 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:47:22.670 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:22.670 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:22.671 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.671 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:47:22.671 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:22.671 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.671 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:22.672 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6816", 2022-06-29T15:47:22.672 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:47:22.672 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.672 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.673 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:22.673 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6817", 2022-06-29T15:47:22.673 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:47:22.673 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:22.674 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:22.674 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.674 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:47:22.674 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:22.675 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.675 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:22.675 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6814", 2022-06-29T15:47:22.675 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:47:22.676 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.676 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.676 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:22.676 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6815", 2022-06-29T15:47:22.677 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:47:22.677 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:22.677 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:22.677 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.678 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6811/13098", 2022-06-29T15:47:22.678 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6813/13098", 2022-06-29T15:47:22.678 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6817/13098", 2022-06-29T15:47:22.678 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6815/13098", 2022-06-29T15:47:22.679 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:47:22.679 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:47:22.679 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:47:22.679 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:22.680 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.680 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.680 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2022-06-29T15:47:22.680 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "55b24fd7-bb71-4a01-b3fa-a09caa772a62", 2022-06-29T15:47:22.680 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:47:22.681 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:47:22.681 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:47:22.681 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:47:22.681 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:47:22.682 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:47:22.682 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:47:22.682 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 54, 2022-06-29T15:47:22.683 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:47:22.683 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:47:22.683 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:47:22.683 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:22.683 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.684 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:22.684 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6802", 2022-06-29T15:47:22.684 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:47:22.684 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.685 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.685 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:22.685 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6803", 2022-06-29T15:47:22.686 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:47:22.686 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:22.686 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:22.686 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.687 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:47:22.687 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:22.687 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.687 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:22.688 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6804", 2022-06-29T15:47:22.688 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:47:22.688 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.688 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.689 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:22.689 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6805", 2022-06-29T15:47:22.689 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:47:22.689 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:22.689 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:22.690 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.690 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:47:22.690 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:22.690 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.691 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:22.691 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6808", 2022-06-29T15:47:22.691 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:47:22.691 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.692 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.692 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:22.692 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6809", 2022-06-29T15:47:22.692 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:47:22.693 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:22.693 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:22.693 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.693 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:47:22.694 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:22.694 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.694 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:22.694 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6806", 2022-06-29T15:47:22.695 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:47:22.695 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.695 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.696 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:22.696 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6807", 2022-06-29T15:47:22.696 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:47:22.696 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:22.697 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:22.697 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.697 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6803/13097", 2022-06-29T15:47:22.697 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6805/13097", 2022-06-29T15:47:22.697 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6809/13097", 2022-06-29T15:47:22.698 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6807/13097", 2022-06-29T15:47:22.698 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:47:22.698 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:47:22.698 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:47:22.699 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:22.699 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.699 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.699 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2022-06-29T15:47:22.700 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3", 2022-06-29T15:47:22.700 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:47:22.700 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:47:22.700 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:47:22.701 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:47:22.701 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:47:22.701 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:47:22.701 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:47:22.702 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 55, 2022-06-29T15:47:22.702 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:47:22.702 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:47:22.702 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:47:22.703 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:22.703 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.703 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:22.703 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6816", 2022-06-29T15:47:22.704 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:47:22.704 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.704 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.704 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:22.705 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6817", 2022-06-29T15:47:22.705 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:47:22.705 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:22.705 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:22.706 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.706 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:47:22.706 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:22.706 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.707 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:22.707 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6818", 2022-06-29T15:47:22.707 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:47:22.707 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.708 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.708 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:22.708 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6819", 2022-06-29T15:47:22.708 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:47:22.709 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:22.709 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:22.709 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.709 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:47:22.710 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:22.710 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.710 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:22.710 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6822", 2022-06-29T15:47:22.711 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:47:22.711 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.711 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.711 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:22.712 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6823", 2022-06-29T15:47:22.712 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:47:22.712 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:22.713 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:22.713 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.713 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:47:22.714 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:22.714 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.715 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:22.715 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6820", 2022-06-29T15:47:22.716 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:47:22.716 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.717 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.717 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:22.718 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6821", 2022-06-29T15:47:22.718 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:47:22.719 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:22.719 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:22.720 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.720 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6817/13396", 2022-06-29T15:47:22.721 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6819/13396", 2022-06-29T15:47:22.721 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6823/13396", 2022-06-29T15:47:22.721 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6821/13396", 2022-06-29T15:47:22.722 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:47:22.722 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:47:22.723 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:47:22.723 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:22.723 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.724 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.724 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2022-06-29T15:47:22.724 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "f77beebe-6cca-4244-b2d4-677825ebf919", 2022-06-29T15:47:22.724 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:47:22.725 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:47:22.725 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:47:22.725 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:47:22.725 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:47:22.726 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:47:22.726 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:47:22.726 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 55, 2022-06-29T15:47:22.726 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:47:22.727 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:47:22.727 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:47:22.727 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:22.727 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.728 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:22.728 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6800", 2022-06-29T15:47:22.728 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:47:22.728 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.729 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.729 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:22.729 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6801", 2022-06-29T15:47:22.730 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:47:22.730 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:22.730 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:22.731 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.731 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:47:22.731 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:22.731 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.732 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:22.732 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6802", 2022-06-29T15:47:22.732 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:47:22.732 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.733 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.733 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:22.733 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6803", 2022-06-29T15:47:22.734 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:47:22.734 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:22.734 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:22.734 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.735 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:47:22.735 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:22.735 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.736 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:22.736 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6806", 2022-06-29T15:47:22.736 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:47:22.736 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.736 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.737 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:22.737 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6807", 2022-06-29T15:47:22.737 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:47:22.738 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:22.738 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:22.738 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.738 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:47:22.739 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:22.739 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.739 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:22.739 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6804", 2022-06-29T15:47:22.740 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:47:22.740 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.740 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.740 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:22.741 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6805", 2022-06-29T15:47:22.741 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:47:22.741 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:22.741 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:22.742 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.742 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6801/13393", 2022-06-29T15:47:22.742 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6803/13393", 2022-06-29T15:47:22.742 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6807/13393", 2022-06-29T15:47:22.743 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6805/13393", 2022-06-29T15:47:22.743 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:47:22.743 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:47:22.743 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:47:22.744 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:22.744 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.744 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.744 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2022-06-29T15:47:22.744 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "af0ca9b3-c02c-4910-834b-bf0b8dc6713c", 2022-06-29T15:47:22.745 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:47:22.745 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:47:22.745 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:47:22.745 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:47:22.746 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:47:22.746 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:47:22.746 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:47:22.746 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 55, 2022-06-29T15:47:22.747 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:47:22.747 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:47:22.747 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:47:22.747 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:22.748 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.748 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:22.748 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6824", 2022-06-29T15:47:22.748 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:47:22.749 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.749 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.749 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:22.750 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6825", 2022-06-29T15:47:22.750 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:47:22.750 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:22.750 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:22.751 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.751 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:47:22.751 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:22.751 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.751 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:22.752 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6826", 2022-06-29T15:47:22.752 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:47:22.752 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.752 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.753 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:22.753 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6827", 2022-06-29T15:47:22.753 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:47:22.753 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:22.754 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:22.754 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.754 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:47:22.754 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:22.755 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.755 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:22.755 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6830", 2022-06-29T15:47:22.755 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:47:22.756 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.756 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.756 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:22.756 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6831", 2022-06-29T15:47:22.757 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:47:22.757 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:22.757 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:22.757 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.757 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:47:22.758 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:22.758 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.758 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:22.759 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6828", 2022-06-29T15:47:22.759 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:47:22.759 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.759 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.759 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:22.760 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6829", 2022-06-29T15:47:22.760 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:47:22.760 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:22.760 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:22.761 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.761 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6825/13395", 2022-06-29T15:47:22.761 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6827/13395", 2022-06-29T15:47:22.761 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6831/13395", 2022-06-29T15:47:22.762 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6829/13395", 2022-06-29T15:47:22.762 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:47:22.762 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:47:22.762 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:47:22.763 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:22.763 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.763 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.763 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2022-06-29T15:47:22.764 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "f19c3ca6-df88-4bde-a150-b51a53f092e1", 2022-06-29T15:47:22.764 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:47:22.764 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:47:22.764 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:47:22.765 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:47:22.765 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:47:22.765 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:47:22.765 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:47:22.766 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 54, 2022-06-29T15:47:22.766 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:47:22.766 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:47:22.766 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:47:22.766 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:22.767 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.767 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:22.767 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6808", 2022-06-29T15:47:22.767 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:47:22.768 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.768 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.768 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:22.768 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6809", 2022-06-29T15:47:22.769 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:47:22.769 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:22.769 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:22.769 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.770 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:47:22.770 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:22.770 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.770 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:22.771 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6810", 2022-06-29T15:47:22.771 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:47:22.771 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.771 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.772 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:22.772 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6811", 2022-06-29T15:47:22.772 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:47:22.772 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:22.773 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:22.773 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.773 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:47:22.773 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:22.773 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.774 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:22.774 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6814", 2022-06-29T15:47:22.774 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:47:22.774 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.775 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.775 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:22.775 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6815", 2022-06-29T15:47:22.775 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:47:22.776 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:22.776 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:22.776 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.777 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:47:22.777 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:22.777 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.777 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:22.778 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6812", 2022-06-29T15:47:22.778 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:47:22.778 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.778 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.779 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:22.779 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6813", 2022-06-29T15:47:22.779 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:47:22.780 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:22.780 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:22.780 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.780 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6809/13394", 2022-06-29T15:47:22.781 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6811/13394", 2022-06-29T15:47:22.781 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6815/13394", 2022-06-29T15:47:22.781 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6813/13394", 2022-06-29T15:47:22.782 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:47:22.782 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:47:22.782 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:47:22.782 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:22.783 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:22.783 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:47:22.783 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_xinfo": [ 2022-06-29T15:47:22.783 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.784 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2022-06-29T15:47:22.784 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:47:22.784 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:47:22.784 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:47:22.785 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:47:22.785 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:47:22.785 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.720513+0000", 2022-06-29T15:47:22.786 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:47:22.786 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.786 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.786 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2022-06-29T15:47:22.787 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:47:22.787 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:47:22.787 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:47:22.787 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:47:22.787 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:47:22.788 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.772612+0000", 2022-06-29T15:47:22.788 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:47:22.788 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.788 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.789 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2022-06-29T15:47:22.789 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:47:22.789 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:47:22.789 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:47:22.790 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:47:22.790 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:47:22.790 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.450223+0000", 2022-06-29T15:47:22.791 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:47:22.791 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.791 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.791 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2022-06-29T15:47:22.792 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:47:22.792 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:47:22.792 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:47:22.792 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:47:22.793 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:47:22.793 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.477856+0000", 2022-06-29T15:47:22.793 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:47:22.793 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.794 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.794 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2022-06-29T15:47:22.794 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:47:22.794 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:47:22.795 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:47:22.795 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:47:22.795 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:47:22.795 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.539485+0000", 2022-06-29T15:47:22.796 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:47:22.796 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.796 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.796 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2022-06-29T15:47:22.796 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:47:22.797 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:47:22.797 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:47:22.797 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:47:22.797 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:47:22.798 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.487040+0000", 2022-06-29T15:47:22.798 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:47:22.798 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.798 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.799 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2022-06-29T15:47:22.799 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:47:22.799 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:47:22.799 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:47:22.800 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:47:22.800 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:47:22.800 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.748685+0000", 2022-06-29T15:47:22.800 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:47:22.801 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.801 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:22.801 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2022-06-29T15:47:22.801 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:47:22.801 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:47:22.802 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:47:22.802 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:47:22.802 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:47:22.803 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.459516+0000", 2022-06-29T15:47:22.803 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:47:22.803 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:22.803 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:47:22.803 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap": [], 2022-06-29T15:47:22.804 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap_items": [], 2022-06-29T15:47:22.804 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_temp": [], 2022-06-29T15:47:22.804 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_temp": [], 2022-06-29T15:47:22.804 INFO:teuthology.orchestra.run.smithi047.stdout: "blacklist": {}, 2022-06-29T15:47:22.805 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profiles": { 2022-06-29T15:47:22.805 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs_data": { 2022-06-29T15:47:22.805 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-device-class": "", 2022-06-29T15:47:22.805 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2022-06-29T15:47:22.806 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-root": "default", 2022-06-29T15:47:22.806 INFO:teuthology.orchestra.run.smithi047.stdout: "jerasure-per-chunk-alignment": "false", 2022-06-29T15:47:22.806 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2022-06-29T15:47:22.806 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "2", 2022-06-29T15:47:22.807 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2022-06-29T15:47:22.807 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2022-06-29T15:47:22.807 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van", 2022-06-29T15:47:22.807 INFO:teuthology.orchestra.run.smithi047.stdout: "w": "8" 2022-06-29T15:47:22.807 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.808 INFO:teuthology.orchestra.run.smithi047.stdout: "default": { 2022-06-29T15:47:22.808 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2022-06-29T15:47:22.808 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2022-06-29T15:47:22.808 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "1", 2022-06-29T15:47:22.809 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2022-06-29T15:47:22.809 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2022-06-29T15:47:22.809 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van" 2022-06-29T15:47:22.809 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:22.810 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:22.810 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps_queue": [], 2022-06-29T15:47:22.810 INFO:teuthology.orchestra.run.smithi047.stdout: "new_removed_snaps": [], 2022-06-29T15:47:22.810 INFO:teuthology.orchestra.run.smithi047.stdout: "new_purged_snaps": [], 2022-06-29T15:47:22.811 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_node_flags": {}, 2022-06-29T15:47:22.811 INFO:teuthology.orchestra.run.smithi047.stdout: "device_class_flags": {} 2022-06-29T15:47:22.811 INFO:teuthology.orchestra.run.smithi047.stdout:} 2022-06-29T15:47:22.813 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.535+0000 7fd88ffff700 1 -- 172.21.15.47:0/871677378 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd86c053950 msgr2=0x7fd86c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:22.814 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.535+0000 7fd88ffff700 1 --2- 172.21.15.47:0/871677378 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd86c053950 0x7fd86c055e10 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7fd88000c000 tx=0x7fd880008040).stop 2022-06-29T15:47:22.814 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.535+0000 7fd88ffff700 1 -- 172.21.15.47:0/871677378 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd890060eb0 msgr2=0x7fd890148980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:22.814 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.535+0000 7fd88ffff700 1 --2- 172.21.15.47:0/871677378 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd890060eb0 0x7fd890148980 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7fd884001ff0 tx=0x7fd884007c80).stop 2022-06-29T15:47:22.815 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.535+0000 7fd88ffff700 1 -- 172.21.15.47:0/871677378 shutdown_connections 2022-06-29T15:47:22.815 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.535+0000 7fd88ffff700 1 --2- 172.21.15.47:0/871677378 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd86c053950 0x7fd86c055e10 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:22.815 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.535+0000 7fd88ffff700 1 --2- 172.21.15.47:0/871677378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd890145090 0x7fd89013d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:22.815 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.535+0000 7fd88ffff700 1 --2- 172.21.15.47:0/871677378 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd8900b5c60 0x7fd8900b4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:22.816 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.535+0000 7fd88ffff700 1 --2- 172.21.15.47:0/871677378 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd890060eb0 0x7fd890148980 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:22.816 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.535+0000 7fd88ffff700 1 -- 172.21.15.47:0/871677378 >> 172.21.15.47:0/871677378 conn(0x7fd890062560 msgr2=0x7fd8900b1020 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:22.816 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.535+0000 7fd88ffff700 1 -- 172.21.15.47:0/871677378 shutdown_connections 2022-06-29T15:47:22.816 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.535+0000 7fd88ffff700 1 -- 172.21.15.47:0/871677378 wait complete. 2022-06-29T15:47:22.817 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:47:22.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.927+0000 7f596ad21700 1 Processor -- start 2022-06-29T15:47:22.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.927+0000 7f596ad21700 1 -- start start 2022-06-29T15:47:22.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.927+0000 7f596ad21700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f59640611b0 0x7f5964146640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:22.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.927+0000 7f596ad21700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f59640b0cf0 0x7f5964146b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:22.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.927+0000 7f596ad21700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59640b4520 con 0x7f59640b06d0 2022-06-29T15:47:22.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.927+0000 7f596ad21700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59640b4b10 con 0x7f59640b0cf0 2022-06-29T15:47:22.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.927+0000 7f596ad21700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59640621d0 con 0x7f59640611b0 2022-06-29T15:47:22.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.927+0000 7f5963fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f59640b0cf0 0x7f5964146b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:22.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.927+0000 7f5963fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f59640b0cf0 0x7f5964146b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:54616/0 (socket says 172.21.15.47:54616) 2022-06-29T15:47:22.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.927+0000 7f5963fff700 1 -- 172.21.15.47:0/2336736740 learned_addr learned my addr 172.21.15.47:0/2336736740 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:22.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.927+0000 7f5968abd700 1 --2- 172.21.15.47:0/2336736740 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f59640611b0 0x7f5964146640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:22.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.931+0000 7f5963fff700 1 -- 172.21.15.47:0/2336736740 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f59640611b0 msgr2=0x7f5964146640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:22.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.931+0000 7f5963fff700 1 --2- 172.21.15.47:0/2336736740 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f59640611b0 0x7f5964146640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:22.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.931+0000 7f5963fff700 1 -- 172.21.15.47:0/2336736740 >> v1:172.21.15.64:6789/0 conn(0x7f59640b06d0 legacy=0x7f59640667e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:22.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.931+0000 7f5963fff700 1 -- 172.21.15.47:0/2336736740 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5964061590 con 0x7f59640b0cf0 2022-06-29T15:47:22.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.931+0000 7f5963fff700 1 --2- 172.21.15.47:0/2336736740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59640b0cf0 0x7f5964146b30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f5954002f70 tx=0x7f5954006eb0).ready entity=mon.1 client_cookie=9ea0958206ef6d7b server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:22.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.931+0000 7f59637fe700 1 -- 172.21.15.47:0/2336736740 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f595400d570 con 0x7f59640b0cf0 2022-06-29T15:47:22.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.931+0000 7f59637fe700 1 -- 172.21.15.47:0/2336736740 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5954000e90 con 0x7f59640b0cf0 2022-06-29T15:47:22.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.931+0000 7f59637fe700 1 -- 172.21.15.47:0/2336736740 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f595400d900 con 0x7f59640b0cf0 2022-06-29T15:47:22.938 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.931+0000 7f596ad21700 1 -- 172.21.15.47:0/2336736740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59640b0cf0 msgr2=0x7f5964146b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:22.939 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.931+0000 7f596ad21700 1 --2- 172.21.15.47:0/2336736740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59640b0cf0 0x7f5964146b30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f5954002f70 tx=0x7f5954006eb0).stop 2022-06-29T15:47:22.939 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.931+0000 7f596ad21700 1 -- 172.21.15.47:0/2336736740 shutdown_connections 2022-06-29T15:47:22.939 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.931+0000 7f596ad21700 1 --2- 172.21.15.47:0/2336736740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59640b0cf0 0x7f5964146b30 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:22.939 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.931+0000 7f596ad21700 1 --2- 172.21.15.47:0/2336736740 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f59640611b0 0x7f5964146640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:22.940 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.931+0000 7f596ad21700 1 -- 172.21.15.47:0/2336736740 >> 172.21.15.47:0/2336736740 conn(0x7f59640b98b0 msgr2=0x7f59640b9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:22.940 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.931+0000 7f596ad21700 1 -- 172.21.15.47:0/2336736740 shutdown_connections 2022-06-29T15:47:22.940 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.931+0000 7f596ad21700 1 -- 172.21.15.47:0/2336736740 wait complete. 2022-06-29T15:47:22.940 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.931+0000 7f596ad21700 1 Processor -- start 2022-06-29T15:47:22.941 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.931+0000 7f596ad21700 1 -- start start 2022-06-29T15:47:22.941 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.935+0000 7f596ad21700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59640611b0 0x7f59641446b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:22.941 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.935+0000 7f596ad21700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f59640b06d0 0x7f59640b74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:22.941 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.935+0000 7f596ad21700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59640b0cf0 0x7f59640b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:22.941 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.935+0000 7f596ad21700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59640adf10 con 0x7f59640611b0 2022-06-29T15:47:22.942 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.935+0000 7f596ad21700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5964144ba0 con 0x7f59640b0cf0 2022-06-29T15:47:22.942 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.935+0000 7f596ad21700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5964143df0 con 0x7f59640b06d0 2022-06-29T15:47:22.942 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.935+0000 7f5968abd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59640611b0 0x7f59641446b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:22.942 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.935+0000 7f5963fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f59640b06d0 0x7f59640b74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:22.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.935+0000 7f5968abd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59640611b0 0x7f59641446b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48920/0 (socket says 172.21.15.47:48920) 2022-06-29T15:47:22.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.935+0000 7f5968abd700 1 -- 172.21.15.47:0/2149659144 learned_addr learned my addr 172.21.15.47:0/2149659144 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:22.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.935+0000 7f59692be700 1 --2- 172.21.15.47:0/2149659144 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59640b0cf0 0x7f59640b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:22.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.935+0000 7f5968abd700 1 -- 172.21.15.47:0/2149659144 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f59640b06d0 msgr2=0x7f59640b74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:22.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.935+0000 7f5968abd700 1 --2- 172.21.15.47:0/2149659144 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f59640b06d0 0x7f59640b74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:22.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.935+0000 7f5968abd700 1 -- 172.21.15.47:0/2149659144 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59640b0cf0 msgr2=0x7f59640b79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:22.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.935+0000 7f5968abd700 1 --2- 172.21.15.47:0/2149659144 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59640b0cf0 0x7f59640b79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:22.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.935+0000 7f5968abd700 1 -- 172.21.15.47:0/2149659144 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f595400b1e0 con 0x7f59640611b0 2022-06-29T15:47:22.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.935+0000 7f5963fff700 1 --2- 172.21.15.47:0/2149659144 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f59640b06d0 0x7f59640b74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:47:22.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.935+0000 7f5968abd700 1 --2- 172.21.15.47:0/2149659144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59640611b0 0x7f59641446b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7f5964144ce0 tx=0x7f5958006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:22.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.935+0000 7f5961ffb700 1 -- 172.21.15.47:0/2149659144 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5958010780 con 0x7f59640611b0 2022-06-29T15:47:22.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.935+0000 7f5961ffb700 1 -- 172.21.15.47:0/2149659144 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59580108e0 con 0x7f59640611b0 2022-06-29T15:47:22.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.935+0000 7f5961ffb700 1 -- 172.21.15.47:0/2149659144 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5958010c40 con 0x7f59640611b0 2022-06-29T15:47:22.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.935+0000 7f596ad21700 1 -- 172.21.15.47:0/2149659144 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5964144090 con 0x7f59640611b0 2022-06-29T15:47:22.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.935+0000 7f596ad21700 1 -- 172.21.15.47:0/2149659144 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5964146350 con 0x7f59640611b0 2022-06-29T15:47:22.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.939+0000 7f596ad21700 1 -- 172.21.15.47:0/2149659144 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59640573d0 con 0x7f59640611b0 2022-06-29T15:47:22.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.939+0000 7f5961ffb700 1 -- 172.21.15.47:0/2149659144 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f595801c460 con 0x7f59640611b0 2022-06-29T15:47:22.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.939+0000 7f5961ffb700 1 --2- 172.21.15.47:0/2149659144 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5944053b00 0x7f5944055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:22.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.939+0000 7f5961ffb700 1 -- 172.21.15.47:0/2149659144 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f595806c490 con 0x7f59640611b0 2022-06-29T15:47:22.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.939+0000 7f5963fff700 1 --2- 172.21.15.47:0/2149659144 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5944053b00 0x7f5944055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:22.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.939+0000 7f5963fff700 1 --2- 172.21.15.47:0/2149659144 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5944053b00 0x7f5944055fc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f595401c000 tx=0x7f5954015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:22.950 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:22.943+0000 7f5961ffb700 1 -- 172.21.15.47:0/2149659144 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5958043060 con 0x7f59640611b0 2022-06-29T15:47:23.111 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.107+0000 7f596ad21700 1 -- 172.21.15.47:0/2149659144 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5964061750 con 0x7f59640611b0 2022-06-29T15:47:23.112 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.111+0000 7f5961ffb700 1 -- 172.21.15.47:0/2149659144 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+2492 (secure 0 0 0) 0x7f5958046680 con 0x7f59640611b0 2022-06-29T15:47:23.112 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:47:23.113 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":24,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5595,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1798995166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1798995166},{"type":"v1","addr":"172.21.15.47:6835","nonce":1798995166}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21},{"gid":5601,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/1641478219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1641478219},{"type":"v1","addr":"172.21.15.64:6833","nonce":1641478219}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:46:01.294125+0000","modified":"2022-06-29T15:46:02.296567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5607":{"gid":5607,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6837/2427298469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2427298469},{"type":"v1","addr":"172.21.15.47:6837","nonce":2427298469}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2022-06-29T15:47:23.115 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.111+0000 7f596ad21700 1 -- 172.21.15.47:0/2149659144 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5944053b00 msgr2=0x7f5944055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:23.115 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.111+0000 7f596ad21700 1 --2- 172.21.15.47:0/2149659144 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5944053b00 0x7f5944055fc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f595401c000 tx=0x7f5954015000).stop 2022-06-29T15:47:23.115 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.111+0000 7f596ad21700 1 -- 172.21.15.47:0/2149659144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59640611b0 msgr2=0x7f59641446b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:23.115 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.111+0000 7f596ad21700 1 --2- 172.21.15.47:0/2149659144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59640611b0 0x7f59641446b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7f5964144ce0 tx=0x7f5958006df0).stop 2022-06-29T15:47:23.116 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.111+0000 7f596ad21700 1 -- 172.21.15.47:0/2149659144 shutdown_connections 2022-06-29T15:47:23.116 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.111+0000 7f596ad21700 1 --2- 172.21.15.47:0/2149659144 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5944053b00 0x7f5944055fc0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:23.116 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.111+0000 7f596ad21700 1 --2- 172.21.15.47:0/2149659144 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59640b0cf0 0x7f59640b79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:23.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.111+0000 7f596ad21700 1 --2- 172.21.15.47:0/2149659144 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f59640b06d0 0x7f59640b74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:23.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.111+0000 7f596ad21700 1 --2- 172.21.15.47:0/2149659144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59640611b0 0x7f59641446b0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:23.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.111+0000 7f596ad21700 1 -- 172.21.15.47:0/2149659144 >> 172.21.15.47:0/2149659144 conn(0x7f59640b98b0 msgr2=0x7f59641421a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:23.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.111+0000 7f596ad21700 1 -- 172.21.15.47:0/2149659144 shutdown_connections 2022-06-29T15:47:23.118 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.111+0000 7f596ad21700 1 -- 172.21.15.47:0/2149659144 wait complete. 2022-06-29T15:47:23.118 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 24 2022-06-29T15:47:23.126 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-06-29T15:47:23.228 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.227+0000 7f320305f700 1 Processor -- start 2022-06-29T15:47:23.228 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.227+0000 7f320305f700 1 -- start start 2022-06-29T15:47:23.237 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.227+0000 7f320305f700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f31fc143bd0 0x7f31fc05c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:23.237 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.227+0000 7f320305f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f31fc0b2c40 0x7f31fc0b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:23.238 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.227+0000 7f320305f700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31fc142f90 con 0x7f31fc05c510 2022-06-29T15:47:23.238 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.227+0000 7f320305f700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31fc1431f0 con 0x7f31fc0b2c40 2022-06-29T15:47:23.239 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.227+0000 7f320305f700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31fc143490 con 0x7f31fc143bd0 2022-06-29T15:47:23.239 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.227+0000 7f320285e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f31fc0b2c40 0x7f31fc0b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:23.239 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.227+0000 7f320285e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f31fc0b2c40 0x7f31fc0b30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48928/0 (socket says 172.21.15.47:48928) 2022-06-29T15:47:23.239 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.227+0000 7f320285e700 1 -- 172.21.15.47:0/4017439330 learned_addr learned my addr 172.21.15.47:0/4017439330 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:23.240 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.227+0000 7f320205d700 1 --1- 172.21.15.47:0/4017439330 >> v1:172.21.15.47:6789/0 conn(0x7f31fc05c510 0x7f31fc0b4100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34480/0 (socket says 172.21.15.47:34480) 2022-06-29T15:47:23.240 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.227+0000 7f320185c700 1 --2- 172.21.15.47:0/4017439330 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f31fc143bd0 0x7f31fc05c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:23.240 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.227+0000 7f320285e700 1 -- 172.21.15.47:0/4017439330 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f31fc143bd0 msgr2=0x7f31fc05c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:23.240 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.227+0000 7f320285e700 1 --2- 172.21.15.47:0/4017439330 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f31fc143bd0 0x7f31fc05c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:23.240 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.227+0000 7f320285e700 1 -- 172.21.15.47:0/4017439330 >> v1:172.21.15.47:6789/0 conn(0x7f31fc05c510 legacy=0x7f31fc0b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:23.241 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.227+0000 7f320285e700 1 -- 172.21.15.47:0/4017439330 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31fc143730 con 0x7f31fc0b2c40 2022-06-29T15:47:23.241 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.227+0000 7f320185c700 1 --2- 172.21.15.47:0/4017439330 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f31fc143bd0 0x7f31fc05c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:47:23.241 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.227+0000 7f320285e700 1 --2- 172.21.15.47:0/4017439330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31fc0b2c40 0x7f31fc0b30d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7f31f4002f70 tx=0x7f31f4006eb0).ready entity=mon.0 client_cookie=b5c98f8730b89b54 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:23.241 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.227+0000 7f320105b700 1 -- 172.21.15.47:0/4017439330 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31f400d5c0 con 0x7f31fc0b2c40 2022-06-29T15:47:23.242 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.227+0000 7f320105b700 1 -- 172.21.15.47:0/4017439330 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f4000e90 con 0x7f31fc0b2c40 2022-06-29T15:47:23.242 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.227+0000 7f320105b700 1 -- 172.21.15.47:0/4017439330 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31f400d950 con 0x7f31fc0b2c40 2022-06-29T15:47:23.242 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.227+0000 7f320305f700 1 -- 172.21.15.47:0/4017439330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31fc0b2c40 msgr2=0x7f31fc0b30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:23.242 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.227+0000 7f320305f700 1 --2- 172.21.15.47:0/4017439330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31fc0b2c40 0x7f31fc0b30d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7f31f4002f70 tx=0x7f31f4006eb0).stop 2022-06-29T15:47:23.243 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.231+0000 7f320305f700 1 -- 172.21.15.47:0/4017439330 shutdown_connections 2022-06-29T15:47:23.243 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.231+0000 7f320305f700 1 --2- 172.21.15.47:0/4017439330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31fc0b2c40 0x7f31fc0b30d0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:23.243 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.231+0000 7f320305f700 1 --2- 172.21.15.47:0/4017439330 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f31fc143bd0 0x7f31fc05c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:23.243 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.231+0000 7f320305f700 1 -- 172.21.15.47:0/4017439330 >> 172.21.15.47:0/4017439330 conn(0x7f31fc060e00 msgr2=0x7f31fc0611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:23.244 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.231+0000 7f320305f700 1 -- 172.21.15.47:0/4017439330 shutdown_connections 2022-06-29T15:47:23.244 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.231+0000 7f320305f700 1 -- 172.21.15.47:0/4017439330 wait complete. 2022-06-29T15:47:23.244 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.231+0000 7f320305f700 1 Processor -- start 2022-06-29T15:47:23.244 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.231+0000 7f320305f700 1 -- start start 2022-06-29T15:47:23.244 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.231+0000 7f320305f700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f31fc05c510 0x7f31fc0b0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:23.245 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.231+0000 7f320305f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31fc0b2c40 0x7f31fc0af430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:23.245 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.231+0000 7f320305f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31fc143bd0 0x7f31fc155ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:23.245 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.231+0000 7f320305f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31fc155690 con 0x7f31fc0b2c40 2022-06-29T15:47:23.245 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.231+0000 7f320305f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31fc141270 con 0x7f31fc143bd0 2022-06-29T15:47:23.246 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.231+0000 7f320305f700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31fc141510 con 0x7f31fc05c510 2022-06-29T15:47:23.246 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.231+0000 7f320185c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31fc0b2c40 0x7f31fc0af430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:23.247 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.231+0000 7f320185c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31fc0b2c40 0x7f31fc0af430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48934/0 (socket says 172.21.15.47:48934) 2022-06-29T15:47:23.247 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.231+0000 7f320185c700 1 -- 172.21.15.47:0/20548381 learned_addr learned my addr 172.21.15.47:0/20548381 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:23.248 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.231+0000 7f320285e700 1 --2- 172.21.15.47:0/20548381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31fc143bd0 0x7f31fc155ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:23.248 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.231+0000 7f320205d700 1 --2- 172.21.15.47:0/20548381 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f31fc05c510 0x7f31fc0b0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:23.248 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.231+0000 7f320185c700 1 -- 172.21.15.47:0/20548381 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f31fc05c510 msgr2=0x7f31fc0b0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:23.248 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.231+0000 7f320185c700 1 --2- 172.21.15.47:0/20548381 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f31fc05c510 0x7f31fc0b0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:23.249 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.231+0000 7f320185c700 1 -- 172.21.15.47:0/20548381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31fc143bd0 msgr2=0x7f31fc155ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:23.249 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.231+0000 7f320185c700 1 --2- 172.21.15.47:0/20548381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31fc143bd0 0x7f31fc155ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:23.249 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.231+0000 7f320185c700 1 -- 172.21.15.47:0/20548381 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f400b1e0 con 0x7f31fc0b2c40 2022-06-29T15:47:23.249 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.231+0000 7f320205d700 1 --2- 172.21.15.47:0/20548381 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f31fc05c510 0x7f31fc0b0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:47:23.249 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.231+0000 7f320185c700 1 --2- 172.21.15.47:0/20548381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31fc0b2c40 0x7f31fc0af430 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7f31f8006cc0 tx=0x7f31f8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:23.250 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.231+0000 7f31f37fe700 1 -- 172.21.15.47:0/20548381 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31f800b7f0 con 0x7f31fc0b2c40 2022-06-29T15:47:23.250 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.231+0000 7f320305f700 1 -- 172.21.15.47:0/20548381 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31fc1417b0 con 0x7f31fc0b2c40 2022-06-29T15:47:23.250 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.231+0000 7f31f37fe700 1 -- 172.21.15.47:0/20548381 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f800b950 con 0x7f31fc0b2c40 2022-06-29T15:47:23.251 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.231+0000 7f320305f700 1 -- 172.21.15.47:0/20548381 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31fc1520c0 con 0x7f31fc0b2c40 2022-06-29T15:47:23.251 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.231+0000 7f31f37fe700 1 -- 172.21.15.47:0/20548381 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31f800bcb0 con 0x7f31fc0b2c40 2022-06-29T15:47:23.251 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.235+0000 7f31f37fe700 1 -- 172.21.15.47:0/20548381 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f31f801c460 con 0x7f31fc0b2c40 2022-06-29T15:47:23.251 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.235+0000 7f320305f700 1 -- 172.21.15.47:0/20548381 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31fc0573d0 con 0x7f31fc0b2c40 2022-06-29T15:47:23.252 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.235+0000 7f31f37fe700 1 --2- 172.21.15.47:0/20548381 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f31dc053a70 0x7f31dc055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:23.252 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.235+0000 7f31f37fe700 1 -- 172.21.15.47:0/20548381 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f31f806c790 con 0x7f31fc0b2c40 2022-06-29T15:47:23.252 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.235+0000 7f320205d700 1 --2- 172.21.15.47:0/20548381 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f31dc053a70 0x7f31dc055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:23.253 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.235+0000 7f320205d700 1 --2- 172.21.15.47:0/20548381 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f31dc053a70 0x7f31dc055f30 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f31ec008000 tx=0x7f31ec009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:23.254 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.243+0000 7f31f37fe700 1 -- 172.21.15.47:0/20548381 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f31f8043360 con 0x7f31fc0b2c40 2022-06-29T15:47:23.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:23.403+0000 7f320305f700 1 -- 172.21.15.47:0/20548381 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f31fc069a60 con 0x7f31fc0b2c40 2022-06-29T15:47:24.091 INFO:tasks.ceph.mon.a.smithi047.stderr:2022-06-29T15:47:24.091+0000 7ff9cf83c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-06-29T15:47:24.099 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.099+0000 7f31f37fe700 1 -- 172.21.15.47:0/20548381 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v25) v1 ==== 161+0+0 (secure 0 0 0) 0x7f31f8046980 con 0x7f31fc0b2c40 2022-06-29T15:47:24.109 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.103+0000 7f320305f700 1 -- 172.21.15.47:0/20548381 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f31dc053a70 msgr2=0x7f31dc055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:24.110 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.103+0000 7f320305f700 1 --2- 172.21.15.47:0/20548381 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f31dc053a70 0x7f31dc055f30 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f31ec008000 tx=0x7f31ec009040).stop 2022-06-29T15:47:24.111 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.103+0000 7f320305f700 1 -- 172.21.15.47:0/20548381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31fc0b2c40 msgr2=0x7f31fc0af430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:24.112 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.103+0000 7f320305f700 1 --2- 172.21.15.47:0/20548381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31fc0b2c40 0x7f31fc0af430 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7f31f8006cc0 tx=0x7f31f8008040).stop 2022-06-29T15:47:24.112 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.103+0000 7f320305f700 1 -- 172.21.15.47:0/20548381 shutdown_connections 2022-06-29T15:47:24.113 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.103+0000 7f320305f700 1 --2- 172.21.15.47:0/20548381 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f31dc053a70 0x7f31dc055f30 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:24.114 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.103+0000 7f320305f700 1 --2- 172.21.15.47:0/20548381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31fc143bd0 0x7f31fc155ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:24.115 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.103+0000 7f320305f700 1 --2- 172.21.15.47:0/20548381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31fc0b2c40 0x7f31fc0af430 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:24.115 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.103+0000 7f320305f700 1 --2- 172.21.15.47:0/20548381 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f31fc05c510 0x7f31fc0b0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:24.116 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.103+0000 7f320305f700 1 -- 172.21.15.47:0/20548381 >> 172.21.15.47:0/20548381 conn(0x7f31fc060e00 msgr2=0x7f31fc0b8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:24.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.103+0000 7f320305f700 1 -- 172.21.15.47:0/20548381 shutdown_connections 2022-06-29T15:47:24.118 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.103+0000 7f320305f700 1 -- 172.21.15.47:0/20548381 wait complete. 2022-06-29T15:47:24.118 INFO:teuthology.orchestra.run.smithi047.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-06-29T15:47:24.120 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:47:24.136 INFO:tasks.ceph.mds.a.smithi047.stdout:starting mds.a at 2022-06-29T15:47:24.228 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.227+0000 7f832d5b7700 1 Processor -- start 2022-06-29T15:47:24.229 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.227+0000 7f832d5b7700 1 -- start start 2022-06-29T15:47:24.239 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.227+0000 7f832d5b7700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f83280ad970 0x7f83280b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:24.239 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.227+0000 7f832d5b7700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f83280b0e50 0x7f832813d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:24.240 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.227+0000 7f832d5b7700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83281400e0 con 0x7f83280b2ee0 2022-06-29T15:47:24.240 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.227+0000 7f832d5b7700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8328140ef0 con 0x7f83280ad970 2022-06-29T15:47:24.240 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.227+0000 7f832d5b7700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f832813fef0 con 0x7f83280b0e50 2022-06-29T15:47:24.241 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.227+0000 7f83277fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f83280ad970 0x7f83280b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:24.241 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.227+0000 7f83277fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f83280ad970 0x7f83280b32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48954/0 (socket says 172.21.15.47:48954) 2022-06-29T15:47:24.244 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.227+0000 7f83277fe700 1 -- 172.21.15.47:0/912566584 learned_addr learned my addr 172.21.15.47:0/912566584 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:24.245 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.227+0000 7f8327fff700 1 --1- 172.21.15.47:0/912566584 >> v1:172.21.15.47:6789/0 conn(0x7f83280b2ee0 0x7f83281407e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34506/0 (socket says 172.21.15.47:34506) 2022-06-29T15:47:24.245 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.227+0000 7f832cdb6700 1 --2- 172.21.15.47:0/912566584 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f83280b0e50 0x7f832813d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:24.245 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.227+0000 7f83277fe700 1 -- 172.21.15.47:0/912566584 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f83280b0e50 msgr2=0x7f832813d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:24.246 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.227+0000 7f83277fe700 1 --2- 172.21.15.47:0/912566584 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f83280b0e50 0x7f832813d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:24.246 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.227+0000 7f8326ffd700 1 -- 172.21.15.47:0/912566584 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3689533487 0 0) 0x7f83281400e0 con 0x7f83280b2ee0 2022-06-29T15:47:24.246 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.227+0000 7f83277fe700 1 -- 172.21.15.47:0/912566584 >> v1:172.21.15.47:6789/0 conn(0x7f83280b2ee0 legacy=0x7f83281407e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:24.246 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.227+0000 7f83277fe700 1 -- 172.21.15.47:0/912566584 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8328143570 con 0x7f83280ad970 2022-06-29T15:47:24.247 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.227+0000 7f832cdb6700 1 --2- 172.21.15.47:0/912566584 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f83280b0e50 0x7f832813d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:47:24.247 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.227+0000 7f83277fe700 1 --2- 172.21.15.47:0/912566584 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83280ad970 0x7f83280b32c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7f83180032f0 tx=0x7f831800b530).ready entity=mon.0 client_cookie=ed07d80607d34901 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:24.247 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.227+0000 7f8326ffd700 1 -- 172.21.15.47:0/912566584 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8318010660 con 0x7f83280ad970 2022-06-29T15:47:24.248 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.227+0000 7f8326ffd700 1 -- 172.21.15.47:0/912566584 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83180107c0 con 0x7f83280ad970 2022-06-29T15:47:24.248 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.227+0000 7f8326ffd700 1 -- 172.21.15.47:0/912566584 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8318010b20 con 0x7f83280ad970 2022-06-29T15:47:24.248 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.227+0000 7f832d5b7700 1 -- 172.21.15.47:0/912566584 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83280ad970 msgr2=0x7f83280b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:24.248 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.227+0000 7f832d5b7700 1 --2- 172.21.15.47:0/912566584 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83280ad970 0x7f83280b32c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7f83180032f0 tx=0x7f831800b530).stop 2022-06-29T15:47:24.249 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.227+0000 7f832d5b7700 1 -- 172.21.15.47:0/912566584 shutdown_connections 2022-06-29T15:47:24.249 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.227+0000 7f832d5b7700 1 --2- 172.21.15.47:0/912566584 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f83280b0e50 0x7f832813d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:24.249 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.227+0000 7f832d5b7700 1 --2- 172.21.15.47:0/912566584 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83280ad970 0x7f83280b32c0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:24.249 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.227+0000 7f832d5b7700 1 -- 172.21.15.47:0/912566584 >> 172.21.15.47:0/912566584 conn(0x7f8328063670 msgr2=0x7f8328063a80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:24.250 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.231+0000 7f832d5b7700 1 -- 172.21.15.47:0/912566584 shutdown_connections 2022-06-29T15:47:24.250 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.231+0000 7f832d5b7700 1 -- 172.21.15.47:0/912566584 wait complete. 2022-06-29T15:47:24.250 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.231+0000 7f832d5b7700 1 Processor -- start 2022-06-29T15:47:24.250 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.231+0000 7f832d5b7700 1 -- start start 2022-06-29T15:47:24.251 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.231+0000 7f832d5b7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83280ad970 0x7f8328060380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:24.251 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.231+0000 7f832d5b7700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f83280b0e50 0x7f832813c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:24.251 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.231+0000 7f832d5b7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83280b2ee0 0x7f832813cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:24.252 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.231+0000 7f832d5b7700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8328066bc0 con 0x7f83280ad970 2022-06-29T15:47:24.252 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.231+0000 7f832d5b7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8328065a80 con 0x7f83280b2ee0 2022-06-29T15:47:24.252 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.231+0000 7f832d5b7700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8328065d20 con 0x7f83280b0e50 2022-06-29T15:47:24.253 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.231+0000 7f8327fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83280ad970 0x7f8328060380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:24.253 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.231+0000 7f8327fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83280ad970 0x7f8328060380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48958/0 (socket says 172.21.15.47:48958) 2022-06-29T15:47:24.253 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.231+0000 7f8327fff700 1 -- 172.21.15.47:0/111455820 learned_addr learned my addr 172.21.15.47:0/111455820 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:24.253 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.231+0000 7f832cdb6700 1 --2- 172.21.15.47:0/111455820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83280b2ee0 0x7f832813cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:24.254 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.231+0000 7f83277fe700 1 --2- 172.21.15.47:0/111455820 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f83280b0e50 0x7f832813c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:24.254 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.231+0000 7f8327fff700 1 -- 172.21.15.47:0/111455820 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f83280b0e50 msgr2=0x7f832813c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:24.254 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.231+0000 7f8327fff700 1 --2- 172.21.15.47:0/111455820 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f83280b0e50 0x7f832813c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:24.254 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.231+0000 7f8327fff700 1 -- 172.21.15.47:0/111455820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83280b2ee0 msgr2=0x7f832813cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:24.255 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.231+0000 7f8327fff700 1 --2- 172.21.15.47:0/111455820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83280b2ee0 0x7f832813cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:24.255 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.231+0000 7f8327fff700 1 -- 172.21.15.47:0/111455820 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f831800b7d0 con 0x7f83280ad970 2022-06-29T15:47:24.255 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.231+0000 7f8327fff700 1 --2- 172.21.15.47:0/111455820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83280ad970 0x7f8328060380 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7f8310000ed0 tx=0x7f8310007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:24.256 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.231+0000 7f83257fa700 1 -- 172.21.15.47:0/111455820 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f831000cd20 con 0x7f83280ad970 2022-06-29T15:47:24.256 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.231+0000 7f83257fa700 1 -- 172.21.15.47:0/111455820 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f831000ce80 con 0x7f83280ad970 2022-06-29T15:47:24.256 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.231+0000 7f832d5b7700 1 -- 172.21.15.47:0/111455820 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8328065fc0 con 0x7f83280ad970 2022-06-29T15:47:24.256 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.231+0000 7f83257fa700 1 -- 172.21.15.47:0/111455820 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8310007360 con 0x7f83280ad970 2022-06-29T15:47:24.257 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.235+0000 7f832d5b7700 1 -- 172.21.15.47:0/111455820 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83280aa0a0 con 0x7f83280ad970 2022-06-29T15:47:24.257 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.235+0000 7f832d5b7700 1 -- 172.21.15.47:0/111455820 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83280573d0 con 0x7f83280ad970 2022-06-29T15:47:24.257 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.235+0000 7f83257fa700 1 -- 172.21.15.47:0/111455820 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8310004520 con 0x7f83280ad970 2022-06-29T15:47:24.257 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.235+0000 7f83257fa700 1 --2- 172.21.15.47:0/111455820 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8300053950 0x7f8300055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:24.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.235+0000 7f83257fa700 1 -- 172.21.15.47:0/111455820 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f831006b920 con 0x7f83280ad970 2022-06-29T15:47:24.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.235+0000 7f83277fe700 1 --2- 172.21.15.47:0/111455820 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8300053950 0x7f8300055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:24.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.235+0000 7f83277fe700 1 --2- 172.21.15.47:0/111455820 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8300053950 0x7f8300055e10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f831801e000 tx=0x7f83280ae510).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:24.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.243+0000 7f83257fa700 1 -- 172.21.15.47:0/111455820 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8310042520 con 0x7f83280ad970 2022-06-29T15:47:24.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.407+0000 7f832d5b7700 1 -- 172.21.15.47:0/111455820 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f832805c2f0 con 0x7f83280ad970 2022-06-29T15:47:24.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.407+0000 7f83257fa700 1 -- 172.21.15.47:0/111455820 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+2131 (secure 0 0 0) 0x7f8310045b40 con 0x7f83280ad970 2022-06-29T15:47:24.408 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:47:24.409 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":25,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5595,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1798995166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1798995166},{"type":"v1","addr":"172.21.15.47:6835","nonce":1798995166}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21},{"gid":5601,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/1641478219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1641478219},{"type":"v1","addr":"172.21.15.64:6833","nonce":1641478219}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:46:01.294125+0000","modified":"2022-06-29T15:47:24.092304+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2022-06-29T15:47:24.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.407+0000 7f832d5b7700 1 -- 172.21.15.47:0/111455820 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8300053950 msgr2=0x7f8300055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:24.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.407+0000 7f832d5b7700 1 --2- 172.21.15.47:0/111455820 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8300053950 0x7f8300055e10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f831801e000 tx=0x7f83280ae510).stop 2022-06-29T15:47:24.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.407+0000 7f832d5b7700 1 -- 172.21.15.47:0/111455820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83280ad970 msgr2=0x7f8328060380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:24.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.407+0000 7f832d5b7700 1 --2- 172.21.15.47:0/111455820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83280ad970 0x7f8328060380 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7f8310000ed0 tx=0x7f8310007c50).stop 2022-06-29T15:47:24.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.407+0000 7f832d5b7700 1 -- 172.21.15.47:0/111455820 shutdown_connections 2022-06-29T15:47:24.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.407+0000 7f832d5b7700 1 --2- 172.21.15.47:0/111455820 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8300053950 0x7f8300055e10 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:24.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.407+0000 7f832d5b7700 1 --2- 172.21.15.47:0/111455820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83280b2ee0 0x7f832813cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:24.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.407+0000 7f832d5b7700 1 --2- 172.21.15.47:0/111455820 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f83280b0e50 0x7f832813c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:24.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.407+0000 7f832d5b7700 1 --2- 172.21.15.47:0/111455820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83280ad970 0x7f8328060380 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:24.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.407+0000 7f832d5b7700 1 -- 172.21.15.47:0/111455820 >> 172.21.15.47:0/111455820 conn(0x7f8328063670 msgr2=0x7f832813e370 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:24.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.407+0000 7f832d5b7700 1 -- 172.21.15.47:0/111455820 shutdown_connections 2022-06-29T15:47:24.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.407+0000 7f832d5b7700 1 -- 172.21.15.47:0/111455820 wait complete. 2022-06-29T15:47:24.414 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 25 2022-06-29T15:47:24.422 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-06-29T15:47:24.574 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.571+0000 7f806359e700 1 Processor -- start 2022-06-29T15:47:24.575 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.571+0000 7f806359e700 1 -- start start 2022-06-29T15:47:24.579 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.575+0000 7f806359e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f80640b58d0 0x7f80640b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:24.579 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.575+0000 7f806359e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f806414b4c0 0x7f80640aa130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:24.580 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.575+0000 7f806359e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8064060790 con 0x7f80640b1610 2022-06-29T15:47:24.581 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.575+0000 7f806359e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80640a99e0 con 0x7f806414b4c0 2022-06-29T15:47:24.581 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.575+0000 7f806359e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f806414ad80 con 0x7f80640b58d0 2022-06-29T15:47:24.581 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.575+0000 7f8062d9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f806414b4c0 0x7f80640aa130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:24.582 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.575+0000 7f8062d9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f806414b4c0 0x7f80640aa130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48970/0 (socket says 172.21.15.47:48970) 2022-06-29T15:47:24.582 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.575+0000 7f8062d9d700 1 -- 172.21.15.47:0/3072089897 learned_addr learned my addr 172.21.15.47:0/3072089897 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:24.583 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.575+0000 7f806259c700 1 --2- 172.21.15.47:0/3072089897 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f80640b58d0 0x7f80640b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:24.583 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.575+0000 7f8062d9d700 1 -- 172.21.15.47:0/3072089897 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f80640b58d0 msgr2=0x7f80640b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:24.583 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.575+0000 7f8062d9d700 1 --2- 172.21.15.47:0/3072089897 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f80640b58d0 0x7f80640b5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:24.584 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.575+0000 7f8062d9d700 1 -- 172.21.15.47:0/3072089897 >> v1:172.21.15.64:6789/0 conn(0x7f80640b1610 legacy=0x7f80641475f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:24.584 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.575+0000 7f806159a700 1 -- 172.21.15.47:0/3072089897 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1687842339 0 0) 0x7f8064060790 con 0x7f80640b1610 2022-06-29T15:47:24.584 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.575+0000 7f8062d9d700 1 -- 172.21.15.47:0/3072089897 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f806414b020 con 0x7f806414b4c0 2022-06-29T15:47:24.585 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.575+0000 7f8062d9d700 1 --2- 172.21.15.47:0/3072089897 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f806414b4c0 0x7f80640aa130 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7f805c0032f0 tx=0x7f805c009290).ready entity=mon.0 client_cookie=b1a837fbef48e644 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:24.585 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.575+0000 7f806159a700 1 -- 172.21.15.47:0/3072089897 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f805c00b8a0 con 0x7f806414b4c0 2022-06-29T15:47:24.585 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.575+0000 7f806159a700 1 -- 172.21.15.47:0/3072089897 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f805c00ba00 con 0x7f806414b4c0 2022-06-29T15:47:24.586 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.575+0000 7f806159a700 1 -- 172.21.15.47:0/3072089897 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f805c00bb70 con 0x7f806414b4c0 2022-06-29T15:47:24.586 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.575+0000 7f806359e700 1 -- 172.21.15.47:0/3072089897 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f806414b4c0 msgr2=0x7f80640aa130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:24.586 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.575+0000 7f806359e700 1 --2- 172.21.15.47:0/3072089897 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f806414b4c0 0x7f80640aa130 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7f805c0032f0 tx=0x7f805c009290).stop 2022-06-29T15:47:24.586 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.575+0000 7f806359e700 1 -- 172.21.15.47:0/3072089897 shutdown_connections 2022-06-29T15:47:24.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.575+0000 7f806359e700 1 --2- 172.21.15.47:0/3072089897 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f806414b4c0 0x7f80640aa130 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:24.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.575+0000 7f806359e700 1 --2- 172.21.15.47:0/3072089897 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f80640b58d0 0x7f80640b5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:24.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.575+0000 7f806359e700 1 -- 172.21.15.47:0/3072089897 >> 172.21.15.47:0/3072089897 conn(0x7f8064062560 msgr2=0x7f8064062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:24.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.575+0000 7f806359e700 1 -- 172.21.15.47:0/3072089897 shutdown_connections 2022-06-29T15:47:24.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.575+0000 7f806359e700 1 -- 172.21.15.47:0/3072089897 wait complete. 2022-06-29T15:47:24.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.579+0000 7f806359e700 1 Processor -- start 2022-06-29T15:47:24.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.579+0000 7f806359e700 1 -- start start 2022-06-29T15:47:24.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.579+0000 7f806359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80640b1610 0x7f80640aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:24.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.579+0000 7f806359e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80640b58d0 0x7f8064065a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:24.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.579+0000 7f806359e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f806414b4c0 0x7f8064065ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:24.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.579+0000 7f806359e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806414ee70 con 0x7f80640b58d0 2022-06-29T15:47:24.594 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.579+0000 7f806359e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80640a9060 con 0x7f80640b1610 2022-06-29T15:47:24.594 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.579+0000 7f806359e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f806414e240 con 0x7f806414b4c0 2022-06-29T15:47:24.594 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.579+0000 7f8061d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80640b58d0 0x7f8064065a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:24.594 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.579+0000 7f8061d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80640b58d0 0x7f8064065a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48974/0 (socket says 172.21.15.47:48974) 2022-06-29T15:47:24.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.579+0000 7f8061d9b700 1 -- 172.21.15.47:0/837408816 learned_addr learned my addr 172.21.15.47:0/837408816 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:24.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.583+0000 7f806259c700 1 --2- 172.21.15.47:0/837408816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80640b1610 0x7f80640aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:24.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.583+0000 7f8062d9d700 1 --2- 172.21.15.47:0/837408816 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f806414b4c0 0x7f8064065ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:24.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.583+0000 7f8061d9b700 1 -- 172.21.15.47:0/837408816 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f806414b4c0 msgr2=0x7f8064065ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:24.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.583+0000 7f8061d9b700 1 --2- 172.21.15.47:0/837408816 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f806414b4c0 0x7f8064065ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:24.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.583+0000 7f8061d9b700 1 -- 172.21.15.47:0/837408816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80640b1610 msgr2=0x7f80640aeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:24.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.583+0000 7f8061d9b700 1 --2- 172.21.15.47:0/837408816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80640b1610 0x7f80640aeaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:24.597 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.583+0000 7f8061d9b700 1 -- 172.21.15.47:0/837408816 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f805c009530 con 0x7f80640b58d0 2022-06-29T15:47:24.597 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.583+0000 7f8061d9b700 1 --2- 172.21.15.47:0/837408816 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80640b58d0 0x7f8064065a00 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7f8054000ed0 tx=0x7f8054007a90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:24.597 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.583+0000 7f80537fe700 1 -- 172.21.15.47:0/837408816 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8054012400 con 0x7f80640b58d0 2022-06-29T15:47:24.597 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.583+0000 7f80537fe700 1 -- 172.21.15.47:0/837408816 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8054012560 con 0x7f80640b58d0 2022-06-29T15:47:24.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.583+0000 7f80537fe700 1 -- 172.21.15.47:0/837408816 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f80540128c0 con 0x7f80640b58d0 2022-06-29T15:47:24.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.583+0000 7f806359e700 1 -- 172.21.15.47:0/837408816 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f806414d570 con 0x7f80640b58d0 2022-06-29T15:47:24.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.583+0000 7f806359e700 1 -- 172.21.15.47:0/837408816 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80640642c0 con 0x7f80640b58d0 2022-06-29T15:47:24.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.583+0000 7f806359e700 1 -- 172.21.15.47:0/837408816 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8064064450 con 0x7f80640b58d0 2022-06-29T15:47:24.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.583+0000 7f80537fe700 1 -- 172.21.15.47:0/837408816 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8054019030 con 0x7f80640b58d0 2022-06-29T15:47:24.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.583+0000 7f80537fe700 1 --2- 172.21.15.47:0/837408816 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8040053980 0x7f8040055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:24.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.583+0000 7f80537fe700 1 -- 172.21.15.47:0/837408816 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f805406b3a0 con 0x7f80640b58d0 2022-06-29T15:47:24.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.583+0000 7f806259c700 1 --2- 172.21.15.47:0/837408816 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8040053980 0x7f8040055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:24.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.587+0000 7f806259c700 1 --2- 172.21.15.47:0/837408816 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8040053980 0x7f8040055e40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f805800b000 tx=0x7f8058006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:24.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.591+0000 7f80537fe700 1 -- 172.21.15.47:0/837408816 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f80540420c0 con 0x7f80640b58d0 2022-06-29T15:47:24.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:24.755+0000 7f806359e700 1 -- 172.21.15.47:0/837408816 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f80640573d0 con 0x7f80640b58d0 2022-06-29T15:47:25.113 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.111+0000 7f80537fe700 1 -- 172.21.15.47:0/837408816 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v26) v1 ==== 104+0+0 (secure 0 0 0) 0x7f80540456e0 con 0x7f80640b58d0 2022-06-29T15:47:25.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.115+0000 7f806359e700 1 -- 172.21.15.47:0/837408816 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8040053980 msgr2=0x7f8040055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:25.118 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.115+0000 7f806359e700 1 --2- 172.21.15.47:0/837408816 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8040053980 0x7f8040055e40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f805800b000 tx=0x7f8058006df0).stop 2022-06-29T15:47:25.118 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.115+0000 7f806359e700 1 -- 172.21.15.47:0/837408816 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80640b58d0 msgr2=0x7f8064065a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:25.118 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.115+0000 7f806359e700 1 --2- 172.21.15.47:0/837408816 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80640b58d0 0x7f8064065a00 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7f8054000ed0 tx=0x7f8054007a90).stop 2022-06-29T15:47:25.119 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.115+0000 7f806359e700 1 -- 172.21.15.47:0/837408816 shutdown_connections 2022-06-29T15:47:25.119 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.115+0000 7f806359e700 1 --2- 172.21.15.47:0/837408816 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f806414b4c0 0x7f8064065ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:25.119 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.115+0000 7f806359e700 1 --2- 172.21.15.47:0/837408816 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80640b58d0 0x7f8064065a00 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:25.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.115+0000 7f806359e700 1 --2- 172.21.15.47:0/837408816 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8040053980 0x7f8040055e40 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:25.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.115+0000 7f806359e700 1 --2- 172.21.15.47:0/837408816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80640b1610 0x7f80640aeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:25.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.115+0000 7f806359e700 1 -- 172.21.15.47:0/837408816 >> 172.21.15.47:0/837408816 conn(0x7f8064062560 msgr2=0x7f80640b0620 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:25.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.115+0000 7f806359e700 1 -- 172.21.15.47:0/837408816 shutdown_connections 2022-06-29T15:47:25.121 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.115+0000 7f806359e700 1 -- 172.21.15.47:0/837408816 wait complete. 2022-06-29T15:47:25.137 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-06-29T15:47:25.267 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.255+0000 7f02131d0700 1 Processor -- start 2022-06-29T15:47:25.267 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.255+0000 7f02131d0700 1 -- start start 2022-06-29T15:47:25.268 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.259+0000 7f02131d0700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f020c05c850 0x7f020c143a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:25.268 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.259+0000 7f02131d0700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f020c0676a0 0x7f020c140130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:25.269 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.259+0000 7f02131d0700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f020c05cc30 con 0x7f020c0b5d90 2022-06-29T15:47:25.269 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.259+0000 7f02131d0700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f020c1430a0 con 0x7f020c0676a0 2022-06-29T15:47:25.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.259+0000 7f02131d0700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f020c143300 con 0x7f020c05c850 2022-06-29T15:47:25.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.259+0000 7f02129cf700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f020c0676a0 0x7f020c140130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:25.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.259+0000 7f02119cd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f020c0b5d90 0x7f020c0b4860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34538/0 (socket says 172.21.15.47:34538) 2022-06-29T15:47:25.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.259+0000 7f02119cd700 1 -- 172.21.15.47:0/1633903231 learned_addr learned my addr 172.21.15.47:0/1633903231 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:47:25.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.259+0000 7f02129cf700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f020c0676a0 0x7f020c140130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48984/0 (socket says 172.21.15.47:48984) 2022-06-29T15:47:25.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.259+0000 7f02121ce700 1 --2- 172.21.15.47:0/1633903231 >> v2:172.21.15.64:3300/0 conn(0x7f020c05c850 0x7f020c143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:25.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.259+0000 7f02129cf700 1 -- 172.21.15.47:0/1633903231 >> v2:172.21.15.64:3300/0 conn(0x7f020c05c850 msgr2=0x7f020c143a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:25.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.259+0000 7f02129cf700 1 --2- 172.21.15.47:0/1633903231 >> v2:172.21.15.64:3300/0 conn(0x7f020c05c850 0x7f020c143a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:25.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.259+0000 7f02129cf700 1 -- 172.21.15.47:0/1633903231 >> v1:172.21.15.47:6789/0 conn(0x7f020c0b5d90 legacy=0x7f020c0b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:25.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.259+0000 7f02111cc700 1 -- 172.21.15.47:0/1633903231 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3478877113 0 0) 0x7f020c05cc30 con 0x7f020c0b5d90 2022-06-29T15:47:25.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.259+0000 7f02129cf700 1 -- 172.21.15.47:0/1633903231 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f020c1435a0 con 0x7f020c0676a0 2022-06-29T15:47:25.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.259+0000 7f02129cf700 1 --2- 172.21.15.47:0/1633903231 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f020c0676a0 0x7f020c140130 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7f02040032f0 tx=0x7f02040091d0).ready entity=mon.0 client_cookie=c636ce5d577499b0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:25.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.259+0000 7f02111cc700 1 -- 172.21.15.47:0/1633903231 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f020400b680 con 0x7f020c0676a0 2022-06-29T15:47:25.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.259+0000 7f02111cc700 1 -- 172.21.15.47:0/1633903231 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f020400b7e0 con 0x7f020c0676a0 2022-06-29T15:47:25.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.259+0000 7f02111cc700 1 -- 172.21.15.47:0/1633903231 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f020400b950 con 0x7f020c0676a0 2022-06-29T15:47:25.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.259+0000 7f02131d0700 1 -- 172.21.15.47:0/1633903231 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f020c0676a0 msgr2=0x7f020c140130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:25.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.259+0000 7f02131d0700 1 --2- 172.21.15.47:0/1633903231 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f020c0676a0 0x7f020c140130 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7f02040032f0 tx=0x7f02040091d0).stop 2022-06-29T15:47:25.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.259+0000 7f02131d0700 1 -- 172.21.15.47:0/1633903231 shutdown_connections 2022-06-29T15:47:25.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.259+0000 7f02131d0700 1 --2- 172.21.15.47:0/1633903231 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f020c0676a0 0x7f020c140130 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:25.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.259+0000 7f02131d0700 1 --2- 172.21.15.47:0/1633903231 >> v2:172.21.15.64:3300/0 conn(0x7f020c05c850 0x7f020c143a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:25.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.259+0000 7f02131d0700 1 -- 172.21.15.47:0/1633903231 >> 172.21.15.47:0/1633903231 conn(0x7f020c060e00 msgr2=0x7f020c0611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:25.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.259+0000 7f02131d0700 1 -- 172.21.15.47:0/1633903231 shutdown_connections 2022-06-29T15:47:25.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.259+0000 7f02131d0700 1 -- 172.21.15.47:0/1633903231 wait complete. 2022-06-29T15:47:25.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.263+0000 7f02131d0700 1 Processor -- start 2022-06-29T15:47:25.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.263+0000 7f02131d0700 1 -- start start 2022-06-29T15:47:25.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.263+0000 7f02131d0700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f020c05c850 0x7f020c1418d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:25.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.263+0000 7f02131d0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f020c0676a0 0x7f020c0b9a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:25.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.263+0000 7f02131d0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f020c0b5d90 0x7f020c0b9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:25.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.263+0000 7f02131d0700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020c0b2ae0 con 0x7f020c0676a0 2022-06-29T15:47:25.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.263+0000 7f02131d0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020c0b7f80 con 0x7f020c0b5d90 2022-06-29T15:47:25.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.263+0000 7f02131d0700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f020c0b8220 con 0x7f020c05c850 2022-06-29T15:47:25.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.263+0000 7f02119cd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f020c0676a0 0x7f020c0b9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:25.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.263+0000 7f02119cd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f020c0676a0 0x7f020c0b9a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48990/0 (socket says 172.21.15.47:48990) 2022-06-29T15:47:25.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.263+0000 7f02119cd700 1 -- 172.21.15.47:0/1539286841 learned_addr learned my addr 172.21.15.47:0/1539286841 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:25.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.263+0000 7f02121ce700 1 --2- 172.21.15.47:0/1539286841 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f020c05c850 0x7f020c1418d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:25.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.263+0000 7f02129cf700 1 --2- 172.21.15.47:0/1539286841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f020c0b5d90 0x7f020c0b9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:25.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.263+0000 7f02119cd700 1 -- 172.21.15.47:0/1539286841 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f020c05c850 msgr2=0x7f020c1418d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:25.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.263+0000 7f02119cd700 1 --2- 172.21.15.47:0/1539286841 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f020c05c850 0x7f020c1418d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:25.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.263+0000 7f02119cd700 1 -- 172.21.15.47:0/1539286841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f020c0b5d90 msgr2=0x7f020c0b9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:25.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.263+0000 7f02119cd700 1 --2- 172.21.15.47:0/1539286841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f020c0b5d90 0x7f020c0b9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:25.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.263+0000 7f02119cd700 1 -- 172.21.15.47:0/1539286841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0204009470 con 0x7f020c0676a0 2022-06-29T15:47:25.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.267+0000 7f02119cd700 1 --2- 172.21.15.47:0/1539286841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f020c0676a0 0x7f020c0b9a50 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7f01fc000bc0 tx=0x7f01fc007a00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:25.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.267+0000 7f01fb7fe700 1 -- 172.21.15.47:0/1539286841 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01fc0070d0 con 0x7f020c0676a0 2022-06-29T15:47:25.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.267+0000 7f01fb7fe700 1 -- 172.21.15.47:0/1539286841 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01fc007230 con 0x7f020c0676a0 2022-06-29T15:47:25.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.267+0000 7f02131d0700 1 -- 172.21.15.47:0/1539286841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f020c0b1fd0 con 0x7f020c0676a0 2022-06-29T15:47:25.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.267+0000 7f01fb7fe700 1 -- 172.21.15.47:0/1539286841 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01fc005610 con 0x7f020c0676a0 2022-06-29T15:47:25.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.267+0000 7f02131d0700 1 -- 172.21.15.47:0/1539286841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f020c0b25d0 con 0x7f020c0676a0 2022-06-29T15:47:25.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.267+0000 7f02131d0700 1 -- 172.21.15.47:0/1539286841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f020c0b7de0 con 0x7f020c0676a0 2022-06-29T15:47:25.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.267+0000 7f01fb7fe700 1 -- 172.21.15.47:0/1539286841 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f01fc01a060 con 0x7f020c0676a0 2022-06-29T15:47:25.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.267+0000 7f01fb7fe700 1 --2- 172.21.15.47:0/1539286841 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f01ec0539e0 0x7f01ec055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:25.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.267+0000 7f01fb7fe700 1 -- 172.21.15.47:0/1539286841 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f01fc06b830 con 0x7f020c0676a0 2022-06-29T15:47:25.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.267+0000 7f02121ce700 1 --2- 172.21.15.47:0/1539286841 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f01ec0539e0 0x7f01ec055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:25.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.271+0000 7f02121ce700 1 --2- 172.21.15.47:0/1539286841 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f01ec0539e0 0x7f01ec055ea0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7f0208006000 tx=0x7f0208006d90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:25.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.275+0000 7f01fb7fe700 1 -- 172.21.15.47:0/1539286841 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f020c0b7de0 con 0x7f020c0676a0 2022-06-29T15:47:25.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:25.395+0000 7f02131d0700 1 -- 172.21.15.47:0/1539286841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f020c0b7db0 con 0x7f020c0676a0 2022-06-29T15:47:26.213 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.211+0000 7f01fb7fe700 1 -- 172.21.15.47:0/1539286841 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v60) v1 ==== 185+0+0 (secure 0 0 0) 0x7f01fc018070 con 0x7f020c0676a0 2022-06-29T15:47:26.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.215+0000 7f02131d0700 1 -- 172.21.15.47:0/1539286841 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f01ec0539e0 msgr2=0x7f01ec055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:26.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.215+0000 7f02131d0700 1 --2- 172.21.15.47:0/1539286841 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f01ec0539e0 0x7f01ec055ea0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7f0208006000 tx=0x7f0208006d90).stop 2022-06-29T15:47:26.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.215+0000 7f02131d0700 1 -- 172.21.15.47:0/1539286841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f020c0676a0 msgr2=0x7f020c0b9a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:26.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.215+0000 7f02131d0700 1 --2- 172.21.15.47:0/1539286841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f020c0676a0 0x7f020c0b9a50 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7f01fc000bc0 tx=0x7f01fc007a00).stop 2022-06-29T15:47:26.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.215+0000 7f02131d0700 1 -- 172.21.15.47:0/1539286841 shutdown_connections 2022-06-29T15:47:26.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.215+0000 7f02131d0700 1 --2- 172.21.15.47:0/1539286841 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f01ec0539e0 0x7f01ec055ea0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:26.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.215+0000 7f02131d0700 1 --2- 172.21.15.47:0/1539286841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f020c0b5d90 0x7f020c0b9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:26.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.215+0000 7f02131d0700 1 --2- 172.21.15.47:0/1539286841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f020c0676a0 0x7f020c0b9a50 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:26.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.215+0000 7f02131d0700 1 --2- 172.21.15.47:0/1539286841 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f020c05c850 0x7f020c1418d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:26.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.215+0000 7f02131d0700 1 -- 172.21.15.47:0/1539286841 >> 172.21.15.47:0/1539286841 conn(0x7f020c060e00 msgr2=0x7f020c0af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:26.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.215+0000 7f02131d0700 1 -- 172.21.15.47:0/1539286841 shutdown_connections 2022-06-29T15:47:26.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.215+0000 7f02131d0700 1 -- 172.21.15.47:0/1539286841 wait complete. 2022-06-29T15:47:26.221 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' removed 2022-06-29T15:47:26.233 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-06-29T15:47:26.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.351+0000 7fa5f7fff700 1 Processor -- start 2022-06-29T15:47:26.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.351+0000 7fa5f7fff700 1 -- start start 2022-06-29T15:47:26.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.351+0000 7fa5f7fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa5f805c850 0x7fa5f8143a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:26.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.351+0000 7fa5f7fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa5f80676a0 0x7fa5f8140130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:26.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.351+0000 7fa5f7fff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5f805cc30 con 0x7fa5f80b5d90 2022-06-29T15:47:26.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.351+0000 7fa5f7fff700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5f81430a0 con 0x7fa5f80676a0 2022-06-29T15:47:26.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.351+0000 7fa5f7fff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f8143300 con 0x7fa5f805c850 2022-06-29T15:47:26.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.351+0000 7fa5f77fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa5f80676a0 0x7fa5f8140130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:26.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.351+0000 7fa5f77fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa5f80676a0 0x7fa5f8140130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49000/0 (socket says 172.21.15.47:49000) 2022-06-29T15:47:26.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.351+0000 7fa5f77fe700 1 -- 172.21.15.47:0/2382437305 learned_addr learned my addr 172.21.15.47:0/2382437305 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:26.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.351+0000 7fa5f6ffd700 1 --2- 172.21.15.47:0/2382437305 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa5f805c850 0x7fa5f8143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:26.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.355+0000 7fa5f5ffb700 1 -- 172.21.15.47:0/2382437305 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1498294364 0 0) 0x7fa5f805cc30 con 0x7fa5f80b5d90 2022-06-29T15:47:26.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.355+0000 7fa5f5ffb700 1 -- 172.21.15.47:0/2382437305 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5d8003260 con 0x7fa5f80b5d90 2022-06-29T15:47:26.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.355+0000 7fa5f77fe700 1 -- 172.21.15.47:0/2382437305 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa5f805c850 msgr2=0x7fa5f8143a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:26.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.355+0000 7fa5f77fe700 1 --2- 172.21.15.47:0/2382437305 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa5f805c850 0x7fa5f8143a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:26.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.355+0000 7fa5f77fe700 1 -- 172.21.15.47:0/2382437305 >> v1:172.21.15.64:6789/0 conn(0x7fa5f80b5d90 legacy=0x7fa5f80b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:26.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.355+0000 7fa5f77fe700 1 -- 172.21.15.47:0/2382437305 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f81435a0 con 0x7fa5f80676a0 2022-06-29T15:47:26.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.355+0000 7fa5f77fe700 1 --2- 172.21.15.47:0/2382437305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5f80676a0 0x7fa5f8140130 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7fa5f00032f0 tx=0x7fa5f0009290).ready entity=mon.0 client_cookie=d745314f223f1b23 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:26.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.355+0000 7fa5f5ffb700 1 -- 172.21.15.47:0/2382437305 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa5f000b6c0 con 0x7fa5f80676a0 2022-06-29T15:47:26.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.355+0000 7fa5f5ffb700 1 -- 172.21.15.47:0/2382437305 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5f000b820 con 0x7fa5f80676a0 2022-06-29T15:47:26.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.355+0000 7fa5f5ffb700 1 -- 172.21.15.47:0/2382437305 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa5f000b990 con 0x7fa5f80676a0 2022-06-29T15:47:26.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.355+0000 7fa5f7fff700 1 -- 172.21.15.47:0/2382437305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5f80676a0 msgr2=0x7fa5f8140130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:26.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.355+0000 7fa5f7fff700 1 --2- 172.21.15.47:0/2382437305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5f80676a0 0x7fa5f8140130 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7fa5f00032f0 tx=0x7fa5f0009290).stop 2022-06-29T15:47:26.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.355+0000 7fa5f7fff700 1 -- 172.21.15.47:0/2382437305 shutdown_connections 2022-06-29T15:47:26.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.355+0000 7fa5f7fff700 1 --2- 172.21.15.47:0/2382437305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5f80676a0 0x7fa5f8140130 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:26.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.355+0000 7fa5f7fff700 1 --2- 172.21.15.47:0/2382437305 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa5f805c850 0x7fa5f8143a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:26.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.355+0000 7fa5f7fff700 1 -- 172.21.15.47:0/2382437305 >> 172.21.15.47:0/2382437305 conn(0x7fa5f8060e00 msgr2=0x7fa5f80611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:26.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.355+0000 7fa5f7fff700 1 -- 172.21.15.47:0/2382437305 shutdown_connections 2022-06-29T15:47:26.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.355+0000 7fa5f7fff700 1 -- 172.21.15.47:0/2382437305 wait complete. 2022-06-29T15:47:26.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.355+0000 7fa5f7fff700 1 Processor -- start 2022-06-29T15:47:26.370 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.355+0000 7fa5f7fff700 1 -- start start 2022-06-29T15:47:26.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.359+0000 7fa5f7fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa5f805c850 0x7fa5f80afc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:26.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.359+0000 7fa5f7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5f80676a0 0x7fa5f812af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:26.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.359+0000 7fa5f7fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5f80b5d90 0x7fa5f812b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:26.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.359+0000 7fa5f7fff700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f80b2900 con 0x7fa5f80b5d90 2022-06-29T15:47:26.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.359+0000 7fa5f7fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f813f9d0 con 0x7fa5f80676a0 2022-06-29T15:47:26.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.359+0000 7fa5f7fff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f813fc70 con 0x7fa5f805c850 2022-06-29T15:47:26.379 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.359+0000 7fa5f77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5f80b5d90 0x7fa5f812b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:26.379 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.359+0000 7fa5f77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5f80b5d90 0x7fa5f812b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49006/0 (socket says 172.21.15.47:49006) 2022-06-29T15:47:26.379 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.359+0000 7fa5f77fe700 1 -- 172.21.15.47:0/2024606799 learned_addr learned my addr 172.21.15.47:0/2024606799 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:26.379 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.359+0000 7fa5f6ffd700 1 --2- 172.21.15.47:0/2024606799 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa5f805c850 0x7fa5f80afc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:26.379 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.359+0000 7fa5f67fc700 1 --2- 172.21.15.47:0/2024606799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5f80676a0 0x7fa5f812af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:26.380 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.359+0000 7fa5f77fe700 1 -- 172.21.15.47:0/2024606799 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa5f805c850 msgr2=0x7fa5f80afc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:26.380 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.359+0000 7fa5f77fe700 1 --2- 172.21.15.47:0/2024606799 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa5f805c850 0x7fa5f80afc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:26.380 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.359+0000 7fa5f77fe700 1 -- 172.21.15.47:0/2024606799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5f80676a0 msgr2=0x7fa5f812af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:26.380 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.359+0000 7fa5f77fe700 1 --2- 172.21.15.47:0/2024606799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5f80676a0 0x7fa5f812af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:26.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.359+0000 7fa5f77fe700 1 -- 172.21.15.47:0/2024606799 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f0009530 con 0x7fa5f80b5d90 2022-06-29T15:47:26.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.359+0000 7fa5f6ffd700 1 --2- 172.21.15.47:0/2024606799 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa5f805c850 0x7fa5f80afc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:47:26.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.359+0000 7fa5f77fe700 1 --2- 172.21.15.47:0/2024606799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5f80b5d90 0x7fa5f812b440 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7fa5f00032f0 tx=0x7fa5f0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:26.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.359+0000 7fa5effff700 1 -- 172.21.15.47:0/2024606799 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa5f000b6c0 con 0x7fa5f80b5d90 2022-06-29T15:47:26.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.359+0000 7fa5effff700 1 -- 172.21.15.47:0/2024606799 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5f0011e00 con 0x7fa5f80b5d90 2022-06-29T15:47:26.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.359+0000 7fa5f7fff700 1 -- 172.21.15.47:0/2024606799 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5f80b3350 con 0x7fa5f80b5d90 2022-06-29T15:47:26.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.359+0000 7fa5effff700 1 -- 172.21.15.47:0/2024606799 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa5f0021670 con 0x7fa5f80b5d90 2022-06-29T15:47:26.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.359+0000 7fa5f7fff700 1 -- 172.21.15.47:0/2024606799 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5f812b930 con 0x7fa5f80b5d90 2022-06-29T15:47:26.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.359+0000 7fa5f7fff700 1 -- 172.21.15.47:0/2024606799 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5f80573d0 con 0x7fa5f80b5d90 2022-06-29T15:47:26.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.359+0000 7fa5effff700 1 -- 172.21.15.47:0/2024606799 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa5f000f070 con 0x7fa5f80b5d90 2022-06-29T15:47:26.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.363+0000 7fa5effff700 1 --2- 172.21.15.47:0/2024606799 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa5dc0539e0 0x7fa5dc055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:26.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.363+0000 7fa5effff700 1 -- 172.21.15.47:0/2024606799 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6068+0+0 (secure 0 0 0) 0x7fa5f006d050 con 0x7fa5f80b5d90 2022-06-29T15:47:26.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.363+0000 7fa5f6ffd700 1 --2- 172.21.15.47:0/2024606799 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa5dc0539e0 0x7fa5dc055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:26.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.363+0000 7fa5f6ffd700 1 --2- 172.21.15.47:0/2024606799 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa5dc0539e0 0x7fa5dc055ea0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7fa5e0006000 tx=0x7fa5e0008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:26.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.367+0000 7fa5effff700 1 -- 172.21.15.47:0/2024606799 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa5f0047e60 con 0x7fa5f80b5d90 2022-06-29T15:47:26.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:26.491+0000 7fa5f7fff700 1 -- 172.21.15.47:0/2024606799 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa5f8061b00 con 0x7fa5f80b5d90 2022-06-29T15:47:27.215 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.215+0000 7fa5effff700 1 -- 172.21.15.47:0/2024606799 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v61) v1 ==== 173+0+0 (secure 0 0 0) 0x7fa5f0026030 con 0x7fa5f80b5d90 2022-06-29T15:47:27.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.219+0000 7fa5f7fff700 1 -- 172.21.15.47:0/2024606799 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa5dc0539e0 msgr2=0x7fa5dc055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:27.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.219+0000 7fa5f7fff700 1 --2- 172.21.15.47:0/2024606799 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa5dc0539e0 0x7fa5dc055ea0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7fa5e0006000 tx=0x7fa5e0008040).stop 2022-06-29T15:47:27.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.219+0000 7fa5f7fff700 1 -- 172.21.15.47:0/2024606799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5f80b5d90 msgr2=0x7fa5f812b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:27.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.219+0000 7fa5f7fff700 1 --2- 172.21.15.47:0/2024606799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5f80b5d90 0x7fa5f812b440 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7fa5f00032f0 tx=0x7fa5f0016000).stop 2022-06-29T15:47:27.222 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.219+0000 7fa5f7fff700 1 -- 172.21.15.47:0/2024606799 shutdown_connections 2022-06-29T15:47:27.222 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.219+0000 7fa5f7fff700 1 --2- 172.21.15.47:0/2024606799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5f80b5d90 0x7fa5f812b440 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:27.222 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.219+0000 7fa5f7fff700 1 --2- 172.21.15.47:0/2024606799 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa5dc0539e0 0x7fa5dc055ea0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:27.222 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.219+0000 7fa5f7fff700 1 --2- 172.21.15.47:0/2024606799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5f80676a0 0x7fa5f812af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:27.223 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.219+0000 7fa5f7fff700 1 --2- 172.21.15.47:0/2024606799 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa5f805c850 0x7fa5f80afc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:27.223 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.219+0000 7fa5f7fff700 1 -- 172.21.15.47:0/2024606799 >> 172.21.15.47:0/2024606799 conn(0x7fa5f8060e00 msgr2=0x7fa5f8143960 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:27.223 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.223+0000 7fa5f7fff700 1 -- 172.21.15.47:0/2024606799 shutdown_connections 2022-06-29T15:47:27.227 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.223+0000 7fa5f7fff700 1 -- 172.21.15.47:0/2024606799 wait complete. 2022-06-29T15:47:27.232 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' removed 2022-06-29T15:47:27.246 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-06-29T15:47:27.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.347+0000 7f3413fff700 1 Processor -- start 2022-06-29T15:47:27.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.351+0000 7f3413fff700 1 -- start start 2022-06-29T15:47:27.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.351+0000 7f3413fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f34140b1320 0x7f34140b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:27.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.351+0000 7f3413fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3414141fd0 0x7f34140b34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:27.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.351+0000 7f3413fff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f341405f7d0 con 0x7f341405f3f0 2022-06-29T15:47:27.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.351+0000 7f3413fff700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34140b27a0 con 0x7f3414141fd0 2022-06-29T15:47:27.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.351+0000 7f3413fff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34140b4b90 con 0x7f34140b1320 2022-06-29T15:47:27.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.351+0000 7f34137fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3414141fd0 0x7f34140b34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:27.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.351+0000 7f34137fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3414141fd0 0x7f34140b34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49014/0 (socket says 172.21.15.47:49014) 2022-06-29T15:47:27.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.351+0000 7f34137fe700 1 -- 172.21.15.47:0/799614684 learned_addr learned my addr 172.21.15.47:0/799614684 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:27.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.351+0000 7f34127fc700 1 --2- 172.21.15.47:0/799614684 >> v2:172.21.15.64:3300/0 conn(0x7f34140b1320 0x7f34140b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:27.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.351+0000 7f34137fe700 1 -- 172.21.15.47:0/799614684 >> v2:172.21.15.64:3300/0 conn(0x7f34140b1320 msgr2=0x7f34140b2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:27.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.351+0000 7f34137fe700 1 --2- 172.21.15.47:0/799614684 >> v2:172.21.15.64:3300/0 conn(0x7f34140b1320 0x7f34140b2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:27.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.351+0000 7f34137fe700 1 -- 172.21.15.47:0/799614684 >> v1:172.21.15.64:6789/0 conn(0x7f341405f3f0 legacy=0x7f34141418c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:27.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.351+0000 7f34137fe700 1 -- 172.21.15.47:0/799614684 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34140b2060 con 0x7f3414141fd0 2022-06-29T15:47:27.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.351+0000 7f34137fe700 1 --2- 172.21.15.47:0/799614684 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3414141fd0 0x7f34140b34e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7f340c0032f0 tx=0x7f340c009290).ready entity=mon.0 client_cookie=2d81d88f995ab9f5 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:27.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.351+0000 7f3411ffb700 1 -- 172.21.15.47:0/799614684 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f340c00b870 con 0x7f3414141fd0 2022-06-29T15:47:27.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.351+0000 7f3411ffb700 1 -- 172.21.15.47:0/799614684 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f340c00b9d0 con 0x7f3414141fd0 2022-06-29T15:47:27.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.351+0000 7f3411ffb700 1 -- 172.21.15.47:0/799614684 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f340c00bb40 con 0x7f3414141fd0 2022-06-29T15:47:27.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.351+0000 7f3413fff700 1 -- 172.21.15.47:0/799614684 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3414141fd0 msgr2=0x7f34140b34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:27.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.351+0000 7f3413fff700 1 --2- 172.21.15.47:0/799614684 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3414141fd0 0x7f34140b34e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7f340c0032f0 tx=0x7f340c009290).stop 2022-06-29T15:47:27.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.351+0000 7f3413fff700 1 -- 172.21.15.47:0/799614684 shutdown_connections 2022-06-29T15:47:27.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.351+0000 7f3413fff700 1 --2- 172.21.15.47:0/799614684 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3414141fd0 0x7f34140b34e0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:27.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.351+0000 7f3413fff700 1 --2- 172.21.15.47:0/799614684 >> v2:172.21.15.64:3300/0 conn(0x7f34140b1320 0x7f34140b2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:27.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.351+0000 7f3413fff700 1 -- 172.21.15.47:0/799614684 >> 172.21.15.47:0/799614684 conn(0x7f3414062570 msgr2=0x7f34140647d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:27.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.351+0000 7f3413fff700 1 -- 172.21.15.47:0/799614684 shutdown_connections 2022-06-29T15:47:27.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.351+0000 7f3413fff700 1 -- 172.21.15.47:0/799614684 wait complete. 2022-06-29T15:47:27.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.355+0000 7f3413fff700 1 Processor -- start 2022-06-29T15:47:27.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.355+0000 7f3413fff700 1 -- start start 2022-06-29T15:47:27.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.355+0000 7f3413fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f341405f3f0 0x7f34140b7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:27.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.355+0000 7f3413fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34140b1320 0x7f341413c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:27.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.355+0000 7f3413fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3414141fd0 0x7f34140b54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:27.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.355+0000 7f3413fff700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3414143f10 con 0x7f341405f3f0 2022-06-29T15:47:27.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.355+0000 7f3413fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34141441b0 con 0x7f34140b1320 2022-06-29T15:47:27.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.355+0000 7f3413fff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3414144450 con 0x7f3414141fd0 2022-06-29T15:47:27.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.355+0000 7f3412ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f341405f3f0 0x7f34140b7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:27.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.355+0000 7f3412ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f341405f3f0 0x7f34140b7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49016/0 (socket says 172.21.15.47:49016) 2022-06-29T15:47:27.370 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.355+0000 7f3412ffd700 1 -- 172.21.15.47:0/3715454553 learned_addr learned my addr 172.21.15.47:0/3715454553 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:27.370 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.355+0000 7f34137fe700 1 --2- 172.21.15.47:0/3715454553 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3414141fd0 0x7f34140b54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:27.370 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.355+0000 7f34127fc700 1 --2- 172.21.15.47:0/3715454553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34140b1320 0x7f341413c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:27.370 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.355+0000 7f3412ffd700 1 -- 172.21.15.47:0/3715454553 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3414141fd0 msgr2=0x7f34140b54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:27.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.355+0000 7f3412ffd700 1 --2- 172.21.15.47:0/3715454553 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3414141fd0 0x7f34140b54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:27.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.355+0000 7f3412ffd700 1 -- 172.21.15.47:0/3715454553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34140b1320 msgr2=0x7f341413c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:27.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.355+0000 7f3412ffd700 1 --2- 172.21.15.47:0/3715454553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34140b1320 0x7f341413c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:27.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.355+0000 7f3412ffd700 1 -- 172.21.15.47:0/3715454553 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f340c009530 con 0x7f341405f3f0 2022-06-29T15:47:27.372 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.355+0000 7f3412ffd700 1 --2- 172.21.15.47:0/3715454553 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f341405f3f0 0x7f34140b7d50 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7f3414062e80 tx=0x7f3408007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:27.372 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.355+0000 7f3403fff700 1 -- 172.21.15.47:0/3715454553 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f340800ce30 con 0x7f341405f3f0 2022-06-29T15:47:27.372 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.355+0000 7f3403fff700 1 -- 172.21.15.47:0/3715454553 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3408007240 con 0x7f341405f3f0 2022-06-29T15:47:27.372 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.355+0000 7f3413fff700 1 -- 172.21.15.47:0/3715454553 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34141446f0 con 0x7f341405f3f0 2022-06-29T15:47:27.372 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.355+0000 7f3403fff700 1 -- 172.21.15.47:0/3715454553 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3408006700 con 0x7f341405f3f0 2022-06-29T15:47:27.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.355+0000 7f3413fff700 1 -- 172.21.15.47:0/3715454553 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34140b82b0 con 0x7f341405f3f0 2022-06-29T15:47:27.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.359+0000 7f3413fff700 1 -- 172.21.15.47:0/3715454553 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34140573d0 con 0x7f341405f3f0 2022-06-29T15:47:27.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.359+0000 7f3403fff700 1 -- 172.21.15.47:0/3715454553 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3408010030 con 0x7f341405f3f0 2022-06-29T15:47:27.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.359+0000 7f3403fff700 1 --2- 172.21.15.47:0/3715454553 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f33f0053a70 0x7f33f0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:27.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.359+0000 7f3403fff700 1 -- 172.21.15.47:0/3715454553 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f340806bbe0 con 0x7f341405f3f0 2022-06-29T15:47:27.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.359+0000 7f34127fc700 1 --2- 172.21.15.47:0/3715454553 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f33f0053a70 0x7f33f0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:27.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.359+0000 7f34127fc700 1 --2- 172.21.15.47:0/3715454553 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f33f0053a70 0x7f33f0055f30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7f340400b000 tx=0x7f3404008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:27.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.367+0000 7f3403fff700 1 -- 172.21.15.47:0/3715454553 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3408042af0 con 0x7f341405f3f0 2022-06-29T15:47:27.497 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.495+0000 7f3413fff700 1 -- 172.21.15.47:0/3715454553 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f3414144880 con 0x7f341405f3f0 2022-06-29T15:47:27.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.499+0000 7f3403fff700 1 -- 172.21.15.47:0/3715454553 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v7) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3408046110 con 0x7f341405f3f0 2022-06-29T15:47:27.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.503+0000 7f3413fff700 1 -- 172.21.15.47:0/3715454553 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f33f0053a70 msgr2=0x7f33f0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:27.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.503+0000 7f3413fff700 1 --2- 172.21.15.47:0/3715454553 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f33f0053a70 0x7f33f0055f30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7f340400b000 tx=0x7f3404008040).stop 2022-06-29T15:47:27.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.503+0000 7f3413fff700 1 -- 172.21.15.47:0/3715454553 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f341405f3f0 msgr2=0x7f34140b7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:27.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.503+0000 7f3413fff700 1 --2- 172.21.15.47:0/3715454553 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f341405f3f0 0x7f34140b7d50 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7f3414062e80 tx=0x7f3408007c10).stop 2022-06-29T15:47:27.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.503+0000 7f3413fff700 1 -- 172.21.15.47:0/3715454553 shutdown_connections 2022-06-29T15:47:27.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.503+0000 7f3413fff700 1 --2- 172.21.15.47:0/3715454553 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3414141fd0 0x7f34140b54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:27.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.503+0000 7f3413fff700 1 --2- 172.21.15.47:0/3715454553 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f33f0053a70 0x7f33f0055f30 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:27.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.503+0000 7f3413fff700 1 --2- 172.21.15.47:0/3715454553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34140b1320 0x7f341413c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:27.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.503+0000 7f3413fff700 1 --2- 172.21.15.47:0/3715454553 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f341405f3f0 0x7f34140b7d50 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:27.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.503+0000 7f3413fff700 1 -- 172.21.15.47:0/3715454553 >> 172.21.15.47:0/3715454553 conn(0x7f3414062570 msgr2=0x7f341405cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:27.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.503+0000 7f3413fff700 1 -- 172.21.15.47:0/3715454553 shutdown_connections 2022-06-29T15:47:27.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.503+0000 7f3413fff700 1 -- 172.21.15.47:0/3715454553 wait complete. 2022-06-29T15:47:27.521 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout' 2022-06-29T15:47:27.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.623+0000 7f7bad51c700 1 Processor -- start 2022-06-29T15:47:27.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.623+0000 7f7bad51c700 1 -- start start 2022-06-29T15:47:27.635 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.623+0000 7f7bad51c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7ba80b2ee0 0x7f7ba80b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:27.635 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.623+0000 7f7bad51c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7ba80b0e50 0x7f7ba813d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:27.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.623+0000 7f7bad51c700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ba81400e0 con 0x7f7ba80ad970 2022-06-29T15:47:27.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.623+0000 7f7bad51c700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ba8140ef0 con 0x7f7ba80b0e50 2022-06-29T15:47:27.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.623+0000 7f7bad51c700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ba813fef0 con 0x7f7ba80b2ee0 2022-06-29T15:47:27.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.623+0000 7f7bacd1b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7ba80b0e50 0x7f7ba813d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:27.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.623+0000 7f7bacd1b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7ba80b0e50 0x7f7ba813d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49028/0 (socket says 172.21.15.47:49028) 2022-06-29T15:47:27.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.623+0000 7f7bacd1b700 1 -- 172.21.15.47:0/4099209322 learned_addr learned my addr 172.21.15.47:0/4099209322 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:27.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.623+0000 7f7ba7fff700 1 --2- 172.21.15.47:0/4099209322 >> v2:172.21.15.64:3300/0 conn(0x7f7ba80b2ee0 0x7f7ba80b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:27.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.623+0000 7f7bacd1b700 1 -- 172.21.15.47:0/4099209322 >> v2:172.21.15.64:3300/0 conn(0x7f7ba80b2ee0 msgr2=0x7f7ba80b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:27.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.623+0000 7f7bacd1b700 1 --2- 172.21.15.47:0/4099209322 >> v2:172.21.15.64:3300/0 conn(0x7f7ba80b2ee0 0x7f7ba80b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:27.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.623+0000 7f7bacd1b700 1 -- 172.21.15.47:0/4099209322 >> v1:172.21.15.64:6789/0 conn(0x7f7ba80ad970 legacy=0x7f7ba81407e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:27.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.623+0000 7f7bacd1b700 1 -- 172.21.15.47:0/4099209322 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba8143570 con 0x7f7ba80b0e50 2022-06-29T15:47:27.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.623+0000 7f7bacd1b700 1 --2- 172.21.15.47:0/4099209322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ba80b0e50 0x7f7ba813d290 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7f7ba00032f0 tx=0x7f7ba0009290).ready entity=mon.0 client_cookie=67030b14d58e6355 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:27.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.627+0000 7f7ba6ffd700 1 -- 172.21.15.47:0/4099209322 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ba000b870 con 0x7f7ba80b0e50 2022-06-29T15:47:27.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.627+0000 7f7ba6ffd700 1 -- 172.21.15.47:0/4099209322 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba000b9d0 con 0x7f7ba80b0e50 2022-06-29T15:47:27.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.627+0000 7f7ba6ffd700 1 -- 172.21.15.47:0/4099209322 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ba000bb40 con 0x7f7ba80b0e50 2022-06-29T15:47:27.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.627+0000 7f7bad51c700 1 -- 172.21.15.47:0/4099209322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ba80b0e50 msgr2=0x7f7ba813d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:27.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.627+0000 7f7bad51c700 1 --2- 172.21.15.47:0/4099209322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ba80b0e50 0x7f7ba813d290 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7f7ba00032f0 tx=0x7f7ba0009290).stop 2022-06-29T15:47:27.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.627+0000 7f7bad51c700 1 -- 172.21.15.47:0/4099209322 shutdown_connections 2022-06-29T15:47:27.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.627+0000 7f7bad51c700 1 --2- 172.21.15.47:0/4099209322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ba80b0e50 0x7f7ba813d290 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:27.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.627+0000 7f7bad51c700 1 --2- 172.21.15.47:0/4099209322 >> v2:172.21.15.64:3300/0 conn(0x7f7ba80b2ee0 0x7f7ba80b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:27.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.627+0000 7f7bad51c700 1 -- 172.21.15.47:0/4099209322 >> 172.21.15.47:0/4099209322 conn(0x7f7ba8063670 msgr2=0x7f7ba8063a80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:27.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.627+0000 7f7bad51c700 1 -- 172.21.15.47:0/4099209322 shutdown_connections 2022-06-29T15:47:27.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.627+0000 7f7bad51c700 1 -- 172.21.15.47:0/4099209322 wait complete. 2022-06-29T15:47:27.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.627+0000 7f7bad51c700 1 Processor -- start 2022-06-29T15:47:27.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.627+0000 7f7bad51c700 1 -- start start 2022-06-29T15:47:27.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.631+0000 7f7bad51c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ba80ad970 0x7f7ba80af9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:27.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.631+0000 7f7bad51c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ba80b0e50 0x7f7ba80b5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:27.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.631+0000 7f7bad51c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7ba80b2ee0 0x7f7ba80b5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:27.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.631+0000 7f7bad51c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba80b4260 con 0x7f7ba80ad970 2022-06-29T15:47:27.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.631+0000 7f7bad51c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba80b5c40 con 0x7f7ba80b0e50 2022-06-29T15:47:27.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.631+0000 7f7bad51c700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba80b4490 con 0x7f7ba80b2ee0 2022-06-29T15:47:27.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.631+0000 7f7ba7fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ba80ad970 0x7f7ba80af9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:27.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.631+0000 7f7ba7fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ba80ad970 0x7f7ba80af9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49030/0 (socket says 172.21.15.47:49030) 2022-06-29T15:47:27.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.631+0000 7f7ba7fff700 1 -- 172.21.15.47:0/3113531606 learned_addr learned my addr 172.21.15.47:0/3113531606 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:27.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.631+0000 7f7bacd1b700 1 --2- 172.21.15.47:0/3113531606 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7ba80b2ee0 0x7f7ba80b5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:27.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.631+0000 7f7ba77fe700 1 --2- 172.21.15.47:0/3113531606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ba80b0e50 0x7f7ba80b5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:27.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.631+0000 7f7ba7fff700 1 -- 172.21.15.47:0/3113531606 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7ba80b2ee0 msgr2=0x7f7ba80b5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:27.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.631+0000 7f7ba7fff700 1 --2- 172.21.15.47:0/3113531606 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7ba80b2ee0 0x7f7ba80b5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:27.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.631+0000 7f7ba7fff700 1 -- 172.21.15.47:0/3113531606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ba80b0e50 msgr2=0x7f7ba80b5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:27.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.631+0000 7f7ba7fff700 1 --2- 172.21.15.47:0/3113531606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ba80b0e50 0x7f7ba80b5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:27.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.631+0000 7f7ba7fff700 1 -- 172.21.15.47:0/3113531606 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba0009530 con 0x7f7ba80ad970 2022-06-29T15:47:27.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.631+0000 7f7bacd1b700 1 --2- 172.21.15.47:0/3113531606 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7ba80b2ee0 0x7f7ba80b5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:47:27.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.631+0000 7f7ba77fe700 1 --2- 172.21.15.47:0/3113531606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ba80b0e50 0x7f7ba80b5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:47:27.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.631+0000 7f7ba7fff700 1 --2- 172.21.15.47:0/3113531606 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ba80ad970 0x7f7ba80af9d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7f7b9800a000 tx=0x7f7b98008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:27.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.631+0000 7f7ba57fa700 1 -- 172.21.15.47:0/3113531606 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b98011780 con 0x7f7ba80ad970 2022-06-29T15:47:27.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.631+0000 7f7ba57fa700 1 -- 172.21.15.47:0/3113531606 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b980118e0 con 0x7f7ba80ad970 2022-06-29T15:47:27.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.631+0000 7f7bad51c700 1 -- 172.21.15.47:0/3113531606 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ba80b4730 con 0x7f7ba80ad970 2022-06-29T15:47:27.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.631+0000 7f7ba57fa700 1 -- 172.21.15.47:0/3113531606 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b98011c40 con 0x7f7ba80ad970 2022-06-29T15:47:27.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.631+0000 7f7bad51c700 1 -- 172.21.15.47:0/3113531606 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ba8141290 con 0x7f7ba80ad970 2022-06-29T15:47:27.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.631+0000 7f7bad51c700 1 -- 172.21.15.47:0/3113531606 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ba80573d0 con 0x7f7ba80ad970 2022-06-29T15:47:27.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.635+0000 7f7ba57fa700 1 -- 172.21.15.47:0/3113531606 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7b9801c460 con 0x7f7ba80ad970 2022-06-29T15:47:27.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.635+0000 7f7ba57fa700 1 --2- 172.21.15.47:0/3113531606 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7b8c053950 0x7f7b8c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:27.651 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.635+0000 7f7ba57fa700 1 -- 172.21.15.47:0/3113531606 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f7b9806be60 con 0x7f7ba80ad970 2022-06-29T15:47:27.651 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.635+0000 7f7ba77fe700 1 --2- 172.21.15.47:0/3113531606 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7b8c053950 0x7f7b8c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:27.651 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.635+0000 7f7ba77fe700 1 --2- 172.21.15.47:0/3113531606 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7b8c053950 0x7f7b8c055e10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f7b9000d000 tx=0x7f7b90000bc0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:27.651 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.639+0000 7f7ba57fa700 1 -- 172.21.15.47:0/3113531606 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7b98042ea0 con 0x7f7ba80ad970 2022-06-29T15:47:27.814 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:27.811+0000 7f7bad51c700 1 -- 172.21.15.47:0/3113531606 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout"]} v 0) v1 -- 0x7f7ba8069e20 con 0x7f7ba80ad970 2022-06-29T15:47:28.233 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.231+0000 7f7ba57fa700 1 -- 172.21.15.47:0/3113531606 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout"]}]=0 v217) v1 ==== 152+0+0 (secure 0 0 0) 0x7f7b980464c0 con 0x7f7ba80ad970 2022-06-29T15:47:28.238 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.235+0000 7f7bad51c700 1 -- 172.21.15.47:0/3113531606 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7b8c053950 msgr2=0x7f7b8c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:28.239 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.235+0000 7f7bad51c700 1 --2- 172.21.15.47:0/3113531606 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7b8c053950 0x7f7b8c055e10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f7b9000d000 tx=0x7f7b90000bc0).stop 2022-06-29T15:47:28.239 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.235+0000 7f7bad51c700 1 -- 172.21.15.47:0/3113531606 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ba80ad970 msgr2=0x7f7ba80af9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:28.239 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.235+0000 7f7bad51c700 1 --2- 172.21.15.47:0/3113531606 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ba80ad970 0x7f7ba80af9d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7f7b9800a000 tx=0x7f7b98008040).stop 2022-06-29T15:47:28.241 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.239+0000 7f7bad51c700 1 -- 172.21.15.47:0/3113531606 shutdown_connections 2022-06-29T15:47:28.241 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.239+0000 7f7bad51c700 1 --2- 172.21.15.47:0/3113531606 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7ba80b2ee0 0x7f7ba80b5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:28.241 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.239+0000 7f7bad51c700 1 --2- 172.21.15.47:0/3113531606 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7b8c053950 0x7f7b8c055e10 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:28.242 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.239+0000 7f7bad51c700 1 --2- 172.21.15.47:0/3113531606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ba80b0e50 0x7f7ba80b5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:28.242 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.239+0000 7f7bad51c700 1 --2- 172.21.15.47:0/3113531606 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ba80ad970 0x7f7ba80af9d0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:28.242 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.239+0000 7f7bad51c700 1 -- 172.21.15.47:0/3113531606 >> 172.21.15.47:0/3113531606 conn(0x7f7ba8063670 msgr2=0x7f7ba813d320 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:28.245 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.243+0000 7f7bad51c700 1 -- 172.21.15.47:0/3113531606 shutdown_connections 2022-06-29T15:47:28.245 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.243+0000 7f7bad51c700 1 -- 172.21.15.47:0/3113531606 wait complete. 2022-06-29T15:47:28.259 INFO:tasks.cephfs_test_runner:test_drop_cache_command_dead_timeout (tasks.cephfs.test_misc.TestCacheDrop) ... ok 2022-06-29T15:47:28.260 INFO:tasks.cephfs_test_runner:Starting test: test_drop_cache_command_timeout (tasks.cephfs.test_misc.TestCacheDrop) 2022-06-29T15:47:28.261 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout' 2022-06-29T15:47:28.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.363+0000 7f062ad14700 1 Processor -- start 2022-06-29T15:47:28.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.363+0000 7f062ad14700 1 -- start start 2022-06-29T15:47:28.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.363+0000 7f062ad14700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f06240611b0 0x7f0624146640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:28.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.363+0000 7f062ad14700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f06240b0cf0 0x7f0624146b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:28.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.363+0000 7f062ad14700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06240b4520 con 0x7f06240b06d0 2022-06-29T15:47:28.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.363+0000 7f062ad14700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06240b4b10 con 0x7f06240b0cf0 2022-06-29T15:47:28.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.363+0000 7f062ad14700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06240621d0 con 0x7f06240611b0 2022-06-29T15:47:28.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.363+0000 7f0623fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f06240b0cf0 0x7f0624146b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:28.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.363+0000 7f0623fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f06240b0cf0 0x7f0624146b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49038/0 (socket says 172.21.15.47:49038) 2022-06-29T15:47:28.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.363+0000 7f0623fff700 1 -- 172.21.15.47:0/181396710 learned_addr learned my addr 172.21.15.47:0/181396710 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:28.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.363+0000 7f0628ab0700 1 --2- 172.21.15.47:0/181396710 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f06240611b0 0x7f0624146640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:28.370 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.363+0000 7f0623fff700 1 -- 172.21.15.47:0/181396710 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f06240611b0 msgr2=0x7f0624146640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:28.370 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.363+0000 7f0623fff700 1 --2- 172.21.15.47:0/181396710 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f06240611b0 0x7f0624146640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:28.370 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.363+0000 7f0623fff700 1 -- 172.21.15.47:0/181396710 >> v1:172.21.15.47:6789/0 conn(0x7f06240b06d0 legacy=0x7f06240667e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:28.370 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.363+0000 7f0623fff700 1 -- 172.21.15.47:0/181396710 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0624061590 con 0x7f06240b0cf0 2022-06-29T15:47:28.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.363+0000 7f0628ab0700 1 --2- 172.21.15.47:0/181396710 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f06240611b0 0x7f0624146640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:47:28.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.363+0000 7f0623fff700 1 --2- 172.21.15.47:0/181396710 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06240b0cf0 0x7f0624146b30 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7f06140032f0 tx=0x7f0614009290).ready entity=mon.0 client_cookie=9d757c9622b97410 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:28.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.363+0000 7f06237fe700 1 -- 172.21.15.47:0/181396710 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f061400b8a0 con 0x7f06240b0cf0 2022-06-29T15:47:28.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.363+0000 7f06237fe700 1 -- 172.21.15.47:0/181396710 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f061400ba00 con 0x7f06240b0cf0 2022-06-29T15:47:28.372 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.363+0000 7f06237fe700 1 -- 172.21.15.47:0/181396710 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f061400bb70 con 0x7f06240b0cf0 2022-06-29T15:47:28.372 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.363+0000 7f062ad14700 1 -- 172.21.15.47:0/181396710 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06240b0cf0 msgr2=0x7f0624146b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:28.372 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.363+0000 7f062ad14700 1 --2- 172.21.15.47:0/181396710 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06240b0cf0 0x7f0624146b30 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7f06140032f0 tx=0x7f0614009290).stop 2022-06-29T15:47:28.372 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.363+0000 7f062ad14700 1 -- 172.21.15.47:0/181396710 shutdown_connections 2022-06-29T15:47:28.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.363+0000 7f062ad14700 1 --2- 172.21.15.47:0/181396710 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06240b0cf0 0x7f0624146b30 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:28.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.363+0000 7f062ad14700 1 --2- 172.21.15.47:0/181396710 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f06240611b0 0x7f0624146640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:28.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.363+0000 7f062ad14700 1 -- 172.21.15.47:0/181396710 >> 172.21.15.47:0/181396710 conn(0x7f06240b98b0 msgr2=0x7f06240b9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:28.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.363+0000 7f062ad14700 1 -- 172.21.15.47:0/181396710 shutdown_connections 2022-06-29T15:47:28.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.363+0000 7f062ad14700 1 -- 172.21.15.47:0/181396710 wait complete. 2022-06-29T15:47:28.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.367+0000 7f062ad14700 1 Processor -- start 2022-06-29T15:47:28.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.367+0000 7f062ad14700 1 -- start start 2022-06-29T15:47:28.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.367+0000 7f062ad14700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f06240611b0 0x7f0624141f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:28.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.367+0000 7f062ad14700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06240b06d0 0x7f06240b8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:28.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.367+0000 7f062ad14700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06240b0cf0 0x7f06240b90e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:28.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.367+0000 7f062ad14700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06240b95d0 con 0x7f06240b06d0 2022-06-29T15:47:28.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.367+0000 7f062ad14700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06240af4d0 con 0x7f06240b0cf0 2022-06-29T15:47:28.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.367+0000 7f062ad14700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06240b82d0 con 0x7f06240611b0 2022-06-29T15:47:28.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.367+0000 7f0623fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06240b06d0 0x7f06240b8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:28.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.367+0000 7f0623fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06240b06d0 0x7f06240b8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49046/0 (socket says 172.21.15.47:49046) 2022-06-29T15:47:28.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.367+0000 7f0623fff700 1 -- 172.21.15.47:0/793060045 learned_addr learned my addr 172.21.15.47:0/793060045 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:28.385 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.367+0000 7f0628ab0700 1 --2- 172.21.15.47:0/793060045 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f06240611b0 0x7f0624141f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:28.385 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.367+0000 7f06292b1700 1 --2- 172.21.15.47:0/793060045 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06240b0cf0 0x7f06240b90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:28.385 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.367+0000 7f0623fff700 1 -- 172.21.15.47:0/793060045 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f06240611b0 msgr2=0x7f0624141f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:28.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.367+0000 7f0623fff700 1 --2- 172.21.15.47:0/793060045 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f06240611b0 0x7f0624141f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:28.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.367+0000 7f0623fff700 1 -- 172.21.15.47:0/793060045 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06240b0cf0 msgr2=0x7f06240b90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:28.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.367+0000 7f0623fff700 1 --2- 172.21.15.47:0/793060045 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06240b0cf0 0x7f06240b90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:28.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.367+0000 7f0623fff700 1 -- 172.21.15.47:0/793060045 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0614009530 con 0x7f06240b06d0 2022-06-29T15:47:28.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.367+0000 7f06292b1700 1 --2- 172.21.15.47:0/793060045 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06240b0cf0 0x7f06240b90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:47:28.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.367+0000 7f0623fff700 1 --2- 172.21.15.47:0/793060045 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06240b06d0 0x7f06240b8bf0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7f06140032f0 tx=0x7f0614016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:28.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.367+0000 7f0621ffb700 1 -- 172.21.15.47:0/793060045 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0614021670 con 0x7f06240b06d0 2022-06-29T15:47:28.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.367+0000 7f0621ffb700 1 -- 172.21.15.47:0/793060045 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06140217d0 con 0x7f06240b06d0 2022-06-29T15:47:28.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.367+0000 7f062ad14700 1 -- 172.21.15.47:0/793060045 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06240b7a60 con 0x7f06240b06d0 2022-06-29T15:47:28.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.367+0000 7f0621ffb700 1 -- 172.21.15.47:0/793060045 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0614021b30 con 0x7f06240b06d0 2022-06-29T15:47:28.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.371+0000 7f062ad14700 1 -- 172.21.15.47:0/793060045 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06240af1e0 con 0x7f06240b06d0 2022-06-29T15:47:28.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.371+0000 7f062ad14700 1 -- 172.21.15.47:0/793060045 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06240b5bc0 con 0x7f06240b06d0 2022-06-29T15:47:28.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.371+0000 7f0621ffb700 1 -- 172.21.15.47:0/793060045 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f061400f070 con 0x7f06240b06d0 2022-06-29T15:47:28.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.371+0000 7f0621ffb700 1 --2- 172.21.15.47:0/793060045 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0608053950 0x7f0608055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:28.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.371+0000 7f0621ffb700 1 -- 172.21.15.47:0/793060045 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f061406d630 con 0x7f06240b06d0 2022-06-29T15:47:28.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.371+0000 7f0628ab0700 1 --2- 172.21.15.47:0/793060045 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0608053950 0x7f0608055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:28.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.371+0000 7f0628ab0700 1 --2- 172.21.15.47:0/793060045 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0608053950 0x7f0608055e10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f060c00d000 tx=0x7f060c008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:28.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.379+0000 7f0621ffb700 1 -- 172.21.15.47:0/793060045 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f06240b5bc0 con 0x7f06240b06d0 2022-06-29T15:47:28.546 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:28.543+0000 7f062ad14700 1 -- 172.21.15.47:0/793060045 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout"]} v 0) v1 -- 0x7f06240b5bc0 con 0x7f06240b06d0 2022-06-29T15:47:29.240 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.239+0000 7f0621ffb700 1 -- 172.21.15.47:0/793060045 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout"]}]=0 v218) v1 ==== 150+0+0 (secure 0 0 0) 0x7f06240b5bc0 con 0x7f06240b06d0 2022-06-29T15:47:29.243 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.243+0000 7f062ad14700 1 -- 172.21.15.47:0/793060045 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0608053950 msgr2=0x7f0608055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:29.244 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.243+0000 7f062ad14700 1 --2- 172.21.15.47:0/793060045 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0608053950 0x7f0608055e10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f060c00d000 tx=0x7f060c008040).stop 2022-06-29T15:47:29.244 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.243+0000 7f062ad14700 1 -- 172.21.15.47:0/793060045 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06240b06d0 msgr2=0x7f06240b8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:29.244 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.243+0000 7f062ad14700 1 --2- 172.21.15.47:0/793060045 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06240b06d0 0x7f06240b8bf0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7f06140032f0 tx=0x7f0614016000).stop 2022-06-29T15:47:29.244 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.243+0000 7f062ad14700 1 -- 172.21.15.47:0/793060045 shutdown_connections 2022-06-29T15:47:29.245 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.243+0000 7f062ad14700 1 --2- 172.21.15.47:0/793060045 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0608053950 0x7f0608055e10 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:29.245 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.243+0000 7f062ad14700 1 --2- 172.21.15.47:0/793060045 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06240b0cf0 0x7f06240b90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:29.245 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.243+0000 7f062ad14700 1 --2- 172.21.15.47:0/793060045 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06240b06d0 0x7f06240b8bf0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:29.245 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.243+0000 7f062ad14700 1 --2- 172.21.15.47:0/793060045 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f06240611b0 0x7f0624141f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:29.245 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.243+0000 7f062ad14700 1 -- 172.21.15.47:0/793060045 >> 172.21.15.47:0/793060045 conn(0x7f06240b98b0 msgr2=0x7f062413fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:29.246 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.243+0000 7f062ad14700 1 -- 172.21.15.47:0/793060045 shutdown_connections 2022-06-29T15:47:29.246 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.243+0000 7f062ad14700 1 -- 172.21.15.47:0/793060045 wait complete. 2022-06-29T15:47:29.260 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-06-29T15:47:29.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.367+0000 7faf5097b700 1 Processor -- start 2022-06-29T15:47:29.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.367+0000 7faf5097b700 1 -- start start 2022-06-29T15:47:29.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.367+0000 7faf5097b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7faf4c05f4a0 0x7faf4c0b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:29.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.367+0000 7faf5097b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7faf4c143140 0x7faf4c09a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:29.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.367+0000 7faf5097b700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf4c05f880 con 0x7faf4c09a1d0 2022-06-29T15:47:29.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.367+0000 7faf5097b700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf4c09ab50 con 0x7faf4c05f4a0 2022-06-29T15:47:29.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.367+0000 7faf5097b700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf4c09ae90 con 0x7faf4c143140 2022-06-29T15:47:29.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.367+0000 7faf4ad9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7faf4c05f4a0 0x7faf4c0b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:29.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.367+0000 7faf4a59c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7faf4c09a1d0 0x7faf4c142a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34606/0 (socket says 172.21.15.47:34606) 2022-06-29T15:47:29.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.367+0000 7faf4a59c700 1 -- 172.21.15.47:0/338613816 learned_addr learned my addr 172.21.15.47:0/338613816 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:47:29.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.367+0000 7faf4b59e700 1 --2- 172.21.15.47:0/338613816 >> v2:172.21.15.64:3300/0 conn(0x7faf4c143140 0x7faf4c09a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:29.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.367+0000 7faf4ad9d700 1 -- 172.21.15.47:0/338613816 >> v2:172.21.15.64:3300/0 conn(0x7faf4c143140 msgr2=0x7faf4c09a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:29.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.367+0000 7faf4ad9d700 1 --2- 172.21.15.47:0/338613816 >> v2:172.21.15.64:3300/0 conn(0x7faf4c143140 0x7faf4c09a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:29.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.367+0000 7faf4ad9d700 1 -- 172.21.15.47:0/338613816 >> v1:172.21.15.47:6789/0 conn(0x7faf4c09a1d0 legacy=0x7faf4c142a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:29.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.367+0000 7faf4ad9d700 1 -- 172.21.15.47:0/338613816 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf4c09b130 con 0x7faf4c05f4a0 2022-06-29T15:47:29.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.367+0000 7faf4ad9d700 1 --2- 172.21.15.47:0/338613816 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf4c05f4a0 0x7faf4c0b2cc0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7faf4c13f510 tx=0x7faf40006df0).ready entity=mon.0 client_cookie=8400f528a5bcea27 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:29.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.371+0000 7faf49d9b700 1 -- 172.21.15.47:0/338613816 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf4000fa60 con 0x7faf4c05f4a0 2022-06-29T15:47:29.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.371+0000 7faf49d9b700 1 -- 172.21.15.47:0/338613816 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf40000e90 con 0x7faf4c05f4a0 2022-06-29T15:47:29.379 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.371+0000 7faf49d9b700 1 -- 172.21.15.47:0/338613816 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf4000fdb0 con 0x7faf4c05f4a0 2022-06-29T15:47:29.379 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.371+0000 7faf5097b700 1 -- 172.21.15.47:0/338613816 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf4c05f4a0 msgr2=0x7faf4c0b2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:29.379 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.371+0000 7faf5097b700 1 --2- 172.21.15.47:0/338613816 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf4c05f4a0 0x7faf4c0b2cc0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7faf4c13f510 tx=0x7faf40006df0).stop 2022-06-29T15:47:29.379 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.371+0000 7faf5097b700 1 -- 172.21.15.47:0/338613816 shutdown_connections 2022-06-29T15:47:29.380 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.371+0000 7faf5097b700 1 --2- 172.21.15.47:0/338613816 >> v2:172.21.15.64:3300/0 conn(0x7faf4c143140 0x7faf4c09a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:29.380 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.371+0000 7faf5097b700 1 --2- 172.21.15.47:0/338613816 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf4c05f4a0 0x7faf4c0b2cc0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:29.380 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.371+0000 7faf5097b700 1 -- 172.21.15.47:0/338613816 >> 172.21.15.47:0/338613816 conn(0x7faf4c062df0 msgr2=0x7faf4c0631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:29.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.371+0000 7faf5097b700 1 -- 172.21.15.47:0/338613816 shutdown_connections 2022-06-29T15:47:29.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.371+0000 7faf5097b700 1 -- 172.21.15.47:0/338613816 wait complete. 2022-06-29T15:47:29.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.371+0000 7faf5097b700 1 Processor -- start 2022-06-29T15:47:29.385 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.371+0000 7faf5097b700 1 -- start start 2022-06-29T15:47:29.385 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.371+0000 7faf5097b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf4c05f4a0 0x7faf4c1426a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:29.385 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.371+0000 7faf5097b700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7faf4c09a1d0 0x7faf4c142b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:29.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.375+0000 7faf5097b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf4c143140 0x7faf4c064ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:29.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.375+0000 7faf5097b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c064210 con 0x7faf4c05f4a0 2022-06-29T15:47:29.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.375+0000 7faf5097b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c0b5350 con 0x7faf4c143140 2022-06-29T15:47:29.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.375+0000 7faf5097b700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c067610 con 0x7faf4c09a1d0 2022-06-29T15:47:29.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.375+0000 7faf4ad9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf4c05f4a0 0x7faf4c1426a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:29.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.375+0000 7faf4ad9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf4c05f4a0 0x7faf4c1426a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49058/0 (socket says 172.21.15.47:49058) 2022-06-29T15:47:29.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.375+0000 7faf4ad9d700 1 -- 172.21.15.47:0/3732158648 learned_addr learned my addr 172.21.15.47:0/3732158648 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:29.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.375+0000 7faf4a59c700 1 --2- 172.21.15.47:0/3732158648 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7faf4c09a1d0 0x7faf4c142b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:29.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.375+0000 7faf4b59e700 1 --2- 172.21.15.47:0/3732158648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf4c143140 0x7faf4c064ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:29.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.375+0000 7faf4ad9d700 1 -- 172.21.15.47:0/3732158648 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7faf4c09a1d0 msgr2=0x7faf4c142b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:29.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.375+0000 7faf4ad9d700 1 --2- 172.21.15.47:0/3732158648 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7faf4c09a1d0 0x7faf4c142b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:29.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.375+0000 7faf4ad9d700 1 -- 172.21.15.47:0/3732158648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf4c143140 msgr2=0x7faf4c064ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:29.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.375+0000 7faf4ad9d700 1 --2- 172.21.15.47:0/3732158648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf4c143140 0x7faf4c064ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:29.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.375+0000 7faf4ad9d700 1 -- 172.21.15.47:0/3732158648 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf4000b180 con 0x7faf4c05f4a0 2022-06-29T15:47:29.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.375+0000 7faf4ad9d700 1 --2- 172.21.15.47:0/3732158648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf4c05f4a0 0x7faf4c1426a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7faf4c064070 tx=0x7faf40016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:29.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.375+0000 7faf3bfff700 1 -- 172.21.15.47:0/3732158648 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf4001f070 con 0x7faf4c05f4a0 2022-06-29T15:47:29.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.375+0000 7faf3bfff700 1 -- 172.21.15.47:0/3732158648 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf40024600 con 0x7faf4c05f4a0 2022-06-29T15:47:29.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.375+0000 7faf5097b700 1 -- 172.21.15.47:0/3732158648 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf4c0b58a0 con 0x7faf4c05f4a0 2022-06-29T15:47:29.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.375+0000 7faf3bfff700 1 -- 172.21.15.47:0/3732158648 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf40024960 con 0x7faf4c05f4a0 2022-06-29T15:47:29.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.375+0000 7faf5097b700 1 -- 172.21.15.47:0/3732158648 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf4c1407f0 con 0x7faf4c05f4a0 2022-06-29T15:47:29.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.375+0000 7faf5097b700 1 -- 172.21.15.47:0/3732158648 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf4c0573d0 con 0x7faf4c05f4a0 2022-06-29T15:47:29.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.375+0000 7faf3bfff700 1 -- 172.21.15.47:0/3732158648 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7faf4002b070 con 0x7faf4c05f4a0 2022-06-29T15:47:29.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.379+0000 7faf3bfff700 1 --2- 172.21.15.47:0/3732158648 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7faf28053b00 0x7faf28055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:29.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.379+0000 7faf3bfff700 1 -- 172.21.15.47:0/3732158648 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5686+0+0 (secure 0 0 0) 0x7faf4006f6c0 con 0x7faf4c05f4a0 2022-06-29T15:47:29.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.379+0000 7faf4a59c700 1 --2- 172.21.15.47:0/3732158648 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7faf28053b00 0x7faf28055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:29.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.379+0000 7faf4a59c700 1 --2- 172.21.15.47:0/3732158648 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7faf28053b00 0x7faf28055fc0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7faf3c00d000 tx=0x7faf3c009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:29.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.383+0000 7faf3bfff700 1 -- 172.21.15.47:0/3732158648 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7faf40045bc0 con 0x7faf4c05f4a0 2022-06-29T15:47:29.518 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.515+0000 7faf5097b700 1 -- 172.21.15.47:0/3732158648 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7faf4c067b30 con 0x7faf4c05f4a0 2022-06-29T15:47:29.524 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.523+0000 7faf3bfff700 1 -- 172.21.15.47:0/3732158648 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v8)=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7faf40002f50 con 0x7faf4c05f4a0 2022-06-29T15:47:29.528 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.527+0000 7faf5097b700 1 -- 172.21.15.47:0/3732158648 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7faf28053b00 msgr2=0x7faf28055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:29.528 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.527+0000 7faf5097b700 1 --2- 172.21.15.47:0/3732158648 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7faf28053b00 0x7faf28055fc0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7faf3c00d000 tx=0x7faf3c009040).stop 2022-06-29T15:47:29.529 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.527+0000 7faf5097b700 1 -- 172.21.15.47:0/3732158648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf4c05f4a0 msgr2=0x7faf4c1426a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:29.529 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.527+0000 7faf5097b700 1 --2- 172.21.15.47:0/3732158648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf4c05f4a0 0x7faf4c1426a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7faf4c064070 tx=0x7faf40016000).stop 2022-06-29T15:47:29.529 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.527+0000 7faf5097b700 1 -- 172.21.15.47:0/3732158648 shutdown_connections 2022-06-29T15:47:29.530 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.527+0000 7faf5097b700 1 --2- 172.21.15.47:0/3732158648 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7faf28053b00 0x7faf28055fc0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:29.530 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.527+0000 7faf5097b700 1 --2- 172.21.15.47:0/3732158648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf4c143140 0x7faf4c064ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:29.530 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.527+0000 7faf5097b700 1 --2- 172.21.15.47:0/3732158648 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7faf4c09a1d0 0x7faf4c142b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:29.530 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.527+0000 7faf5097b700 1 --2- 172.21.15.47:0/3732158648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf4c05f4a0 0x7faf4c1426a0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:29.531 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.527+0000 7faf5097b700 1 -- 172.21.15.47:0/3732158648 >> 172.21.15.47:0/3732158648 conn(0x7faf4c062df0 msgr2=0x7faf4c060050 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:29.531 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.527+0000 7faf5097b700 1 -- 172.21.15.47:0/3732158648 shutdown_connections 2022-06-29T15:47:29.531 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.527+0000 7faf5097b700 1 -- 172.21.15.47:0/3732158648 wait complete. 2022-06-29T15:47:29.541 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-06-29T15:47:29.542 DEBUG:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:47:29.563 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:47:29.585 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:47:29.608 DEBUG:teuthology.orchestra.run.smithi164:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:47:29.629 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-06-29T15:47:29.630 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:47:29.630 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:47:29.631 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:47:29.638 INFO:teuthology.orchestra.run.smithi099.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-06-29T15:47:29.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T15:47:29.640 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-06-29T15:47:29.640 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:47:29.641 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:47:29.641 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:47:29.649 INFO:teuthology.orchestra.run.smithi164.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-06-29T15:47:29.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T15:47:29.650 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-06-29T15:47:29.651 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-06-29T15:47:29.762 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.759+0000 7f650280c700 1 Processor -- start 2022-06-29T15:47:29.762 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.759+0000 7f650280c700 1 -- start start 2022-06-29T15:47:29.764 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.759+0000 7f650280c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f64fc0b5ac0 0x7f64fc1401c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:29.765 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.759+0000 7f650280c700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64fc143d20 con 0x7f64fc0afdc0 2022-06-29T15:47:29.765 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.759+0000 7f650280c700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64fc0a8df0 con 0x7f64fc0b02d0 2022-06-29T15:47:29.765 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.759+0000 7f650280c700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc0a9090 con 0x7f64fc0b5ac0 2022-06-29T15:47:29.765 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.763+0000 7f650200b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f64fc0afdc0 0x7f64fc0ae7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34624/0 (socket says 172.21.15.47:34624) 2022-06-29T15:47:29.766 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.763+0000 7f650200b700 1 -- 172.21.15.47:0/2773714913 learned_addr learned my addr 172.21.15.47:0/2773714913 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:47:29.766 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.763+0000 7f650180a700 1 --2- 172.21.15.47:0/2773714913 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f64fc0b5ac0 0x7f64fc1401c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:29.766 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.763+0000 7f6500808700 1 -- 172.21.15.47:0/2773714913 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1478157590 0 0) 0x7f64fc143d20 con 0x7f64fc0afdc0 2022-06-29T15:47:29.766 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.763+0000 7f6500808700 1 -- 172.21.15.47:0/2773714913 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64e4003260 con 0x7f64fc0afdc0 2022-06-29T15:47:29.767 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.763+0000 7f6500808700 1 -- 172.21.15.47:0/2773714913 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3521187105 0 0) 0x7f64fc0a8df0 con 0x7f64fc0b02d0 2022-06-29T15:47:29.767 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.763+0000 7f6500808700 1 -- 172.21.15.47:0/2773714913 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64fc143d20 con 0x7f64fc0b02d0 2022-06-29T15:47:29.767 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.763+0000 7f650180a700 1 -- 172.21.15.47:0/2773714913 >> v1:172.21.15.64:6789/0 conn(0x7f64fc0b02d0 legacy=0x7f64fc0ae0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:29.767 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.763+0000 7f650180a700 1 -- 172.21.15.47:0/2773714913 >> v1:172.21.15.47:6789/0 conn(0x7f64fc0afdc0 legacy=0x7f64fc0ae7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:29.768 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.763+0000 7f650180a700 1 -- 172.21.15.47:0/2773714913 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64fc143f80 con 0x7f64fc0b5ac0 2022-06-29T15:47:29.769 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.763+0000 7f650180a700 1 --2- 172.21.15.47:0/2773714913 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f64fc0b5ac0 0x7f64fc1401c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f64ec002f70 tx=0x7f64ec006d90).ready entity=mon.2 client_cookie=cd0275b004f277a7 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:29.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.763+0000 7f6500808700 1 -- 172.21.15.47:0/2773714913 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f64ec00d4a0 con 0x7f64fc0b5ac0 2022-06-29T15:47:29.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.763+0000 7f6500808700 1 -- 172.21.15.47:0/2773714913 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64ec000e90 con 0x7f64fc0b5ac0 2022-06-29T15:47:29.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.763+0000 7f6500808700 1 -- 172.21.15.47:0/2773714913 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f64ec00d7f0 con 0x7f64fc0b5ac0 2022-06-29T15:47:29.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.763+0000 7f650280c700 1 -- 172.21.15.47:0/2773714913 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f64fc0b5ac0 msgr2=0x7f64fc1401c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:29.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.763+0000 7f650280c700 1 --2- 172.21.15.47:0/2773714913 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f64fc0b5ac0 0x7f64fc1401c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f64ec002f70 tx=0x7f64ec006d90).stop 2022-06-29T15:47:29.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.763+0000 7f650280c700 1 -- 172.21.15.47:0/2773714913 shutdown_connections 2022-06-29T15:47:29.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.763+0000 7f650280c700 1 --2- 172.21.15.47:0/2773714913 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f64fc0b5ac0 0x7f64fc1401c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:29.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.763+0000 7f650280c700 1 -- 172.21.15.47:0/2773714913 >> 172.21.15.47:0/2773714913 conn(0x7f64fc062560 msgr2=0x7f64fc062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:29.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.763+0000 7f650280c700 1 -- 172.21.15.47:0/2773714913 shutdown_connections 2022-06-29T15:47:29.776 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.763+0000 7f650280c700 1 -- 172.21.15.47:0/2773714913 wait complete. 2022-06-29T15:47:29.776 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.767+0000 7f650280c700 1 Processor -- start 2022-06-29T15:47:29.776 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.767+0000 7f650280c700 1 -- start start 2022-06-29T15:47:29.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.767+0000 7f650280c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64fc0afdc0 0x7f64fc0b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:29.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.767+0000 7f650280c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64fc0b02d0 0x7f64fc0b0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:29.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.767+0000 7f650280c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f64fc0b5ac0 0x7f64fc0ab750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:29.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.767+0000 7f650280c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc148d40 con 0x7f64fc0afdc0 2022-06-29T15:47:29.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.767+0000 7f650280c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc147790 con 0x7f64fc0b02d0 2022-06-29T15:47:29.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.767+0000 7f650280c700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc148f70 con 0x7f64fc0b5ac0 2022-06-29T15:47:29.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.767+0000 7f650180a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64fc0afdc0 0x7f64fc0b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:29.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.767+0000 7f650180a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64fc0afdc0 0x7f64fc0b4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49072/0 (socket says 172.21.15.47:49072) 2022-06-29T15:47:29.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.767+0000 7f650180a700 1 -- 172.21.15.47:0/1056803055 learned_addr learned my addr 172.21.15.47:0/1056803055 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:29.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.767+0000 7f6501009700 1 --2- 172.21.15.47:0/1056803055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64fc0b02d0 0x7f64fc0b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:29.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.767+0000 7f650200b700 1 --2- 172.21.15.47:0/1056803055 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f64fc0b5ac0 0x7f64fc0ab750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:29.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.767+0000 7f650180a700 1 -- 172.21.15.47:0/1056803055 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f64fc0b5ac0 msgr2=0x7f64fc0ab750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:29.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.767+0000 7f650180a700 1 --2- 172.21.15.47:0/1056803055 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f64fc0b5ac0 0x7f64fc0ab750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:29.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.767+0000 7f650180a700 1 -- 172.21.15.47:0/1056803055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64fc0b02d0 msgr2=0x7f64fc0b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:29.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.767+0000 7f650180a700 1 --2- 172.21.15.47:0/1056803055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64fc0b02d0 0x7f64fc0b0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:29.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.767+0000 7f650180a700 1 -- 172.21.15.47:0/1056803055 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64ec00b120 con 0x7f64fc0afdc0 2022-06-29T15:47:29.781 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.767+0000 7f650200b700 1 --2- 172.21.15.47:0/1056803055 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f64fc0b5ac0 0x7f64fc0ab750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:47:29.781 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.767+0000 7f650180a700 1 --2- 172.21.15.47:0/1056803055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64fc0afdc0 0x7f64fc0b4ba0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7f64ec002f70 tx=0x7f64ec014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:29.781 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.767+0000 7f64eaffd700 1 -- 172.21.15.47:0/1056803055 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f64ec011d60 con 0x7f64fc0afdc0 2022-06-29T15:47:29.781 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.767+0000 7f64eaffd700 1 -- 172.21.15.47:0/1056803055 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64ec00de90 con 0x7f64fc0afdc0 2022-06-29T15:47:29.782 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.767+0000 7f650280c700 1 -- 172.21.15.47:0/1056803055 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64fc13e510 con 0x7f64fc0afdc0 2022-06-29T15:47:29.782 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.767+0000 7f64eaffd700 1 -- 172.21.15.47:0/1056803055 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f64ec01d600 con 0x7f64fc0afdc0 2022-06-29T15:47:29.782 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.767+0000 7f650280c700 1 -- 172.21.15.47:0/1056803055 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64fc155690 con 0x7f64fc0afdc0 2022-06-29T15:47:29.782 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.771+0000 7f64eaffd700 1 -- 172.21.15.47:0/1056803055 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f64ec00f070 con 0x7f64fc0afdc0 2022-06-29T15:47:29.783 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.771+0000 7f650280c700 1 -- 172.21.15.47:0/1056803055 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64fc0573d0 con 0x7f64fc0afdc0 2022-06-29T15:47:29.783 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.771+0000 7f64eaffd700 1 --2- 172.21.15.47:0/1056803055 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f64dc053aa0 0x7f64dc055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:29.783 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.771+0000 7f64eaffd700 1 -- 172.21.15.47:0/1056803055 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f64ec06c990 con 0x7f64fc0afdc0 2022-06-29T15:47:29.784 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.771+0000 7f6501009700 1 --2- 172.21.15.47:0/1056803055 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f64dc053aa0 0x7f64dc055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:29.784 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.771+0000 7f6501009700 1 --2- 172.21.15.47:0/1056803055 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f64dc053aa0 0x7f64dc055f60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f64f800d000 tx=0x7f64f8007a00).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:29.784 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.779+0000 7f64eaffd700 1 -- 172.21.15.47:0/1056803055 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f64ec0438a0 con 0x7f64fc0afdc0 2022-06-29T15:47:29.910 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.907+0000 7f650280c700 1 -- 172.21.15.47:0/1056803055 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f64fc069e20 con 0x7f64fc0afdc0 2022-06-29T15:47:29.913 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.911+0000 7f64eaffd700 1 -- 172.21.15.47:0/1056803055 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v61) v1 ==== 81+0+29220 (secure 0 0 0) 0x7f64ec022090 con 0x7f64fc0afdc0 2022-06-29T15:47:29.913 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:47:29.914 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2022-06-29T15:47:29.914 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 61, 2022-06-29T15:47:29.915 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "9a68894a-53ce-400e-838b-14eab91b834c", 2022-06-29T15:47:29.915 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2022-06-29T15:42:14.046361+0000", 2022-06-29T15:47:29.916 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2022-06-29T15:47:27.208538+0000", 2022-06-29T15:47:29.916 INFO:teuthology.orchestra.run.smithi047.stdout: "last_up_change": "2022-06-29T15:42:20.929789+0000", 2022-06-29T15:47:29.916 INFO:teuthology.orchestra.run.smithi047.stdout: "last_in_change": "2022-06-29T15:42:20.929789+0000", 2022-06-29T15:47:29.916 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-06-29T15:47:29.917 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_num": 5799936, 2022-06-29T15:47:29.917 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_set": [ 2022-06-29T15:47:29.917 INFO:teuthology.orchestra.run.smithi047.stdout: "pglog_hardlimit", 2022-06-29T15:47:29.974 INFO:teuthology.orchestra.run.smithi047.stdout: "purged_snapdirs", 2022-06-29T15:47:29.974 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_deletes", 2022-06-29T15:47:29.974 INFO:teuthology.orchestra.run.smithi047.stdout: "sortbitwise" 2022-06-29T15:47:29.975 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:47:29.975 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_version": 5, 2022-06-29T15:47:29.975 INFO:teuthology.orchestra.run.smithi047.stdout: "full_ratio": 0.94999998807907104, 2022-06-29T15:47:29.975 INFO:teuthology.orchestra.run.smithi047.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-06-29T15:47:29.976 INFO:teuthology.orchestra.run.smithi047.stdout: "nearfull_ratio": 0.85000002384185791, 2022-06-29T15:47:29.976 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_snapshot": "", 2022-06-29T15:47:29.976 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_max": 10, 2022-06-29T15:47:29.976 INFO:teuthology.orchestra.run.smithi047.stdout: "max_osd": 8, 2022-06-29T15:47:29.977 INFO:teuthology.orchestra.run.smithi047.stdout: "require_min_compat_client": "jewel", 2022-06-29T15:47:29.977 INFO:teuthology.orchestra.run.smithi047.stdout: "min_compat_client": "jewel", 2022-06-29T15:47:29.978 INFO:teuthology.orchestra.run.smithi047.stdout: "require_osd_release": "octopus", 2022-06-29T15:47:29.978 INFO:teuthology.orchestra.run.smithi047.stdout: "pools": [ 2022-06-29T15:47:29.979 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:29.979 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 1, 2022-06-29T15:47:29.980 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "device_health_metrics", 2022-06-29T15:47:29.980 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:42:16.971431+0000", 2022-06-29T15:47:29.981 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:47:29.981 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:47:29.982 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:47:29.982 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:47:29.983 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:47:29.983 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:47:29.983 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:47:29.984 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:47:29.984 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 1, 2022-06-29T15:47:29.985 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 1, 2022-06-29T15:47:29.985 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 1, 2022-06-29T15:47:29.986 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 1, 2022-06-29T15:47:29.986 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 1, 2022-06-29T15:47:29.987 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:47:29.987 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:47:29.988 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:47:29.988 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:47:29.988 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:47:29.989 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:47:29.989 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:47:29.990 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:29.990 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "16", 2022-06-29T15:47:29.991 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:47:29.991 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:47:29.991 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:47:29.992 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:47:29.992 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:47:29.993 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:47:29.993 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:47:29.994 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:47:29.994 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:47:29.994 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:47:29.995 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:47:29.995 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:47:29.996 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:47:29.996 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:47:29.996 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:47:29.996 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:47:29.997 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:47:29.997 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:47:29.997 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:47:29.997 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:47:29.998 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:47:29.998 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:47:29.998 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:47:29.998 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:47:29.999 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:47:29.999 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:47:29.999 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:29.999 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:47:30.000 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:47:30.000 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:47:30.001 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:47:30.001 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:47:30.001 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:47:30.002 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:47:30.002 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:47:30.002 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:47:30.003 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:47:30.003 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:47:30.003 INFO:teuthology.orchestra.run.smithi047.stdout: "options": { 2022-06-29T15:47:30.003 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_min": 1 2022-06-29T15:47:30.004 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.004 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:47:30.004 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_devicehealth": {} 2022-06-29T15:47:30.004 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:30.004 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.005 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.005 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 2, 2022-06-29T15:47:30.005 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "rbd", 2022-06-29T15:47:30.005 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:42:25.290884+0000", 2022-06-29T15:47:30.006 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:47:30.006 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:47:30.006 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:47:30.007 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:47:30.007 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:47:30.007 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:47:30.007 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:47:30.008 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:47:30.008 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2022-06-29T15:47:30.008 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2022-06-29T15:47:30.009 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2022-06-29T15:47:30.009 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2022-06-29T15:47:30.009 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2022-06-29T15:47:30.009 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:47:30.010 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:47:30.010 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:47:30.010 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:47:30.011 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:47:30.011 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:47:30.011 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:47:30.011 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.012 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "19", 2022-06-29T15:47:30.012 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:47:30.012 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:47:30.012 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:47:30.012 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:47:30.013 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:47:30.013 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:47:30.013 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:47:30.013 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:47:30.014 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:47:30.014 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:47:30.014 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:47:30.014 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:47:30.015 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:47:30.015 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:47:30.015 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:47:30.015 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:47:30.015 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:47:30.016 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:47:30.016 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:47:30.016 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:47:30.016 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:47:30.017 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:47:30.017 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:47:30.017 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:47:30.017 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:47:30.017 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:47:30.018 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.018 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:47:30.018 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:47:30.019 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:47:30.019 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:47:30.019 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:47:30.019 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:47:30.020 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:47:30.020 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:47:30.020 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:47:30.020 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:47:30.021 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:47:30.021 INFO:teuthology.orchestra.run.smithi047.stdout: "options": {}, 2022-06-29T15:47:30.021 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:47:30.022 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd": {} 2022-06-29T15:47:30.022 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:30.022 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:30.022 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:47:30.023 INFO:teuthology.orchestra.run.smithi047.stdout: "osds": [ 2022-06-29T15:47:30.023 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.023 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2022-06-29T15:47:30.023 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "e23eb023-4706-4201-bf72-4bf19c912c5d", 2022-06-29T15:47:30.024 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:47:30.024 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:47:30.024 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:47:30.024 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:47:30.025 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:47:30.025 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:47:30.025 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:47:30.025 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 55, 2022-06-29T15:47:30.026 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:47:30.026 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:47:30.026 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:47:30.026 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:30.027 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.027 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:30.027 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6818", 2022-06-29T15:47:30.027 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:47:30.028 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.028 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.028 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:30.029 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6819", 2022-06-29T15:47:30.029 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:47:30.029 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:30.029 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:30.030 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.030 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:47:30.030 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:30.030 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.031 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:30.031 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6820", 2022-06-29T15:47:30.031 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:47:30.031 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.032 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.032 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:30.032 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6821", 2022-06-29T15:47:30.032 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:47:30.032 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:30.033 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:30.033 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.033 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:47:30.033 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:30.033 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.034 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:30.034 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6824", 2022-06-29T15:47:30.034 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:47:30.034 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.035 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.035 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:30.035 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6825", 2022-06-29T15:47:30.036 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:47:30.036 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:30.036 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:30.036 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.037 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:47:30.037 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:30.037 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.037 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:30.038 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6822", 2022-06-29T15:47:30.038 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:47:30.038 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.038 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.039 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:30.039 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6823", 2022-06-29T15:47:30.039 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:47:30.039 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:30.040 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:30.040 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.040 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6819/13096", 2022-06-29T15:47:30.041 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6821/13096", 2022-06-29T15:47:30.041 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6825/13096", 2022-06-29T15:47:30.042 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6823/13096", 2022-06-29T15:47:30.042 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:47:30.043 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:47:30.043 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:47:30.044 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:30.044 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.045 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.045 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2022-06-29T15:47:30.045 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "228bb08d-2d64-43d9-9560-bfaeff154452", 2022-06-29T15:47:30.046 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:47:30.046 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:47:30.047 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:47:30.047 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:47:30.047 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:47:30.048 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:47:30.048 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:47:30.048 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 55, 2022-06-29T15:47:30.049 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:47:30.049 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:47:30.049 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:47:30.049 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:30.050 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.050 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:30.050 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6826", 2022-06-29T15:47:30.050 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:47:30.051 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.051 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.051 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:30.052 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6827", 2022-06-29T15:47:30.052 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:47:30.052 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:30.052 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:30.053 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.053 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:47:30.053 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:30.053 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.054 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:30.054 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6828", 2022-06-29T15:47:30.054 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:47:30.054 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.055 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.055 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:30.056 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6829", 2022-06-29T15:47:30.056 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:47:30.056 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:30.057 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:30.057 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.057 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:47:30.058 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:30.058 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.058 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:30.058 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6832", 2022-06-29T15:47:30.059 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:47:30.059 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.059 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.060 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:30.060 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6833", 2022-06-29T15:47:30.060 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:47:30.060 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:30.061 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:30.061 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.061 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:47:30.061 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:30.062 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.062 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:30.062 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6830", 2022-06-29T15:47:30.062 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:47:30.063 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.063 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.063 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:30.064 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6831", 2022-06-29T15:47:30.064 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:47:30.064 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:30.064 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:30.065 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.065 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6827/13100", 2022-06-29T15:47:30.065 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6829/13100", 2022-06-29T15:47:30.065 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6833/13100", 2022-06-29T15:47:30.065 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6831/13100", 2022-06-29T15:47:30.066 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:47:30.066 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:47:30.066 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:47:30.066 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:30.067 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.067 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.067 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2022-06-29T15:47:30.067 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "797a5452-1510-4640-a111-c9c2fcf707eb", 2022-06-29T15:47:30.068 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:47:30.068 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:47:30.068 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:47:30.068 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:47:30.068 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:47:30.069 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:47:30.069 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:47:30.070 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 56, 2022-06-29T15:47:30.070 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:47:30.070 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:47:30.070 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:47:30.071 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:30.071 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.071 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:30.071 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6810", 2022-06-29T15:47:30.072 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:47:30.072 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.072 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.072 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:30.073 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6811", 2022-06-29T15:47:30.073 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:47:30.073 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:30.073 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:30.073 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.074 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:47:30.074 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:30.074 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.074 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:30.075 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6812", 2022-06-29T15:47:30.075 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:47:30.075 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.075 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.076 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:30.076 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6813", 2022-06-29T15:47:30.076 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:47:30.076 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:30.076 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:30.077 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.077 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:47:30.077 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:30.077 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.078 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:30.078 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6816", 2022-06-29T15:47:30.078 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:47:30.078 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.079 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.079 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:30.079 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6817", 2022-06-29T15:47:30.079 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:47:30.079 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:30.080 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:30.080 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.081 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:47:30.081 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:30.081 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.081 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:30.081 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6814", 2022-06-29T15:47:30.082 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:47:30.082 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.082 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.082 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:30.083 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6815", 2022-06-29T15:47:30.083 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:47:30.083 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:30.083 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:30.084 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.084 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6811/13098", 2022-06-29T15:47:30.084 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6813/13098", 2022-06-29T15:47:30.085 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6817/13098", 2022-06-29T15:47:30.085 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6815/13098", 2022-06-29T15:47:30.085 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:47:30.086 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:47:30.086 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:47:30.087 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:30.087 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.087 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.088 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2022-06-29T15:47:30.088 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "55b24fd7-bb71-4a01-b3fa-a09caa772a62", 2022-06-29T15:47:30.088 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:47:30.088 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:47:30.089 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:47:30.089 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:47:30.089 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:47:30.089 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:47:30.090 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:47:30.090 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 54, 2022-06-29T15:47:30.090 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:47:30.090 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:47:30.090 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:47:30.091 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:30.091 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.091 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:30.091 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6802", 2022-06-29T15:47:30.091 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:47:30.092 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.092 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.092 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:30.092 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6803", 2022-06-29T15:47:30.093 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:47:30.093 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:30.093 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:30.093 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.093 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:47:30.094 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:30.094 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.094 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:30.094 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6804", 2022-06-29T15:47:30.095 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:47:30.095 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.095 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.096 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:30.096 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6805", 2022-06-29T15:47:30.096 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:47:30.096 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:30.097 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:30.097 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.097 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:47:30.097 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:30.098 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.098 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:30.098 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6808", 2022-06-29T15:47:30.098 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:47:30.099 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.099 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.099 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:30.099 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6809", 2022-06-29T15:47:30.100 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:47:30.100 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:30.100 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:30.100 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.100 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:47:30.101 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:30.101 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.101 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:30.101 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6806", 2022-06-29T15:47:30.102 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:47:30.102 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.102 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.102 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:30.103 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6807", 2022-06-29T15:47:30.103 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:47:30.103 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:30.103 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:30.104 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.104 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6803/13097", 2022-06-29T15:47:30.104 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6805/13097", 2022-06-29T15:47:30.104 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6809/13097", 2022-06-29T15:47:30.105 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6807/13097", 2022-06-29T15:47:30.105 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:47:30.105 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:47:30.105 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:47:30.106 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:30.106 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.106 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.106 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2022-06-29T15:47:30.107 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3", 2022-06-29T15:47:30.107 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:47:30.107 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:47:30.107 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:47:30.108 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:47:30.108 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:47:30.108 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:47:30.108 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:47:30.108 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 55, 2022-06-29T15:47:30.109 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:47:30.109 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:47:30.109 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:47:30.109 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:30.109 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.110 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:30.110 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6816", 2022-06-29T15:47:30.110 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:47:30.110 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.111 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.111 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:30.111 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6817", 2022-06-29T15:47:30.111 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:47:30.112 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:30.112 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:30.112 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.112 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:47:30.112 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:30.113 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.113 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:30.113 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6818", 2022-06-29T15:47:30.113 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:47:30.114 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.114 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.114 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:30.114 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6819", 2022-06-29T15:47:30.114 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:47:30.115 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:30.115 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:30.115 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.115 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:47:30.116 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:30.116 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.116 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:30.116 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6822", 2022-06-29T15:47:30.116 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:47:30.117 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.117 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.117 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:30.117 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6823", 2022-06-29T15:47:30.117 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:47:30.118 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:30.118 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:30.118 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.118 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:47:30.119 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:30.119 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.119 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:30.119 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6820", 2022-06-29T15:47:30.120 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:47:30.120 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.120 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.120 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:30.120 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6821", 2022-06-29T15:47:30.121 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:47:30.121 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:30.121 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:30.121 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.121 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6817/13396", 2022-06-29T15:47:30.122 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6819/13396", 2022-06-29T15:47:30.122 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6823/13396", 2022-06-29T15:47:30.122 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6821/13396", 2022-06-29T15:47:30.122 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:47:30.123 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:47:30.123 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:47:30.123 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:30.123 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.124 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.124 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2022-06-29T15:47:30.124 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "f77beebe-6cca-4244-b2d4-677825ebf919", 2022-06-29T15:47:30.124 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:47:30.124 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:47:30.125 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:47:30.125 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:47:30.125 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:47:30.125 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:47:30.125 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:47:30.126 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 55, 2022-06-29T15:47:30.126 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:47:30.126 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:47:30.126 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:47:30.127 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:30.127 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.127 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:30.127 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6800", 2022-06-29T15:47:30.127 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:47:30.128 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.128 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.128 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:30.128 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6801", 2022-06-29T15:47:30.129 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:47:30.129 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:30.129 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:30.129 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.130 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:47:30.130 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:30.130 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.130 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:30.131 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6802", 2022-06-29T15:47:30.131 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:47:30.131 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.131 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.132 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:30.132 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6803", 2022-06-29T15:47:30.132 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:47:30.132 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:30.133 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:30.133 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.133 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:47:30.133 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:30.134 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.134 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:30.134 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6806", 2022-06-29T15:47:30.134 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:47:30.135 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.135 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.135 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:30.135 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6807", 2022-06-29T15:47:30.136 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:47:30.136 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:30.136 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:30.136 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.137 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:47:30.137 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:30.137 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.137 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:30.138 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6804", 2022-06-29T15:47:30.138 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:47:30.138 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.138 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.139 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:30.139 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6805", 2022-06-29T15:47:30.139 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:47:30.139 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:30.139 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:30.140 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.140 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6801/13393", 2022-06-29T15:47:30.140 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6803/13393", 2022-06-29T15:47:30.140 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6807/13393", 2022-06-29T15:47:30.141 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6805/13393", 2022-06-29T15:47:30.141 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:47:30.141 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:47:30.141 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:47:30.141 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:30.142 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.142 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.142 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2022-06-29T15:47:30.142 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "af0ca9b3-c02c-4910-834b-bf0b8dc6713c", 2022-06-29T15:47:30.143 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:47:30.143 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:47:30.143 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:47:30.143 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:47:30.143 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:47:30.144 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:47:30.144 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:47:30.144 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 55, 2022-06-29T15:47:30.144 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:47:30.144 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:47:30.145 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:47:30.145 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:30.145 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.145 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:30.146 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6824", 2022-06-29T15:47:30.146 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:47:30.146 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.146 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.146 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:30.147 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6825", 2022-06-29T15:47:30.147 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:47:30.147 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:30.147 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:30.148 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.148 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:47:30.148 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:30.148 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.148 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:30.149 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6826", 2022-06-29T15:47:30.149 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:47:30.149 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.149 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.150 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:30.150 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6827", 2022-06-29T15:47:30.150 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:47:30.150 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:30.150 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:30.151 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.151 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:47:30.151 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:30.151 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.152 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:30.152 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6830", 2022-06-29T15:47:30.152 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:47:30.152 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.152 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.153 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:30.153 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6831", 2022-06-29T15:47:30.153 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:47:30.153 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:30.154 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:30.154 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.154 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:47:30.154 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:30.154 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.155 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:30.155 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6828", 2022-06-29T15:47:30.155 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:47:30.155 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.156 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.156 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:30.156 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6829", 2022-06-29T15:47:30.156 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:47:30.157 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:30.157 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:30.157 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.157 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6825/13395", 2022-06-29T15:47:30.157 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6827/13395", 2022-06-29T15:47:30.158 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6831/13395", 2022-06-29T15:47:30.158 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6829/13395", 2022-06-29T15:47:30.158 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:47:30.158 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:47:30.159 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:47:30.159 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:30.159 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.159 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.160 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2022-06-29T15:47:30.160 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "f19c3ca6-df88-4bde-a150-b51a53f092e1", 2022-06-29T15:47:30.160 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:47:30.160 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:47:30.161 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:47:30.161 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:47:30.161 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:47:30.161 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:47:30.161 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:47:30.162 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 54, 2022-06-29T15:47:30.162 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:47:30.162 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:47:30.162 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:47:30.163 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:30.163 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.163 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:30.163 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6808", 2022-06-29T15:47:30.163 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:47:30.164 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.164 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.164 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:30.164 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6809", 2022-06-29T15:47:30.165 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:47:30.165 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:30.165 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:30.165 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.165 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:47:30.166 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:30.166 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.166 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:30.166 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6810", 2022-06-29T15:47:30.167 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:47:30.167 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.167 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.167 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:30.168 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6811", 2022-06-29T15:47:30.168 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:47:30.168 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:30.168 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:30.169 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.169 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:47:30.169 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:30.169 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.170 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:30.170 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6814", 2022-06-29T15:47:30.170 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:47:30.170 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.171 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.171 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:30.171 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6815", 2022-06-29T15:47:30.171 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:47:30.172 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:30.172 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:30.172 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.172 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:47:30.173 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:47:30.173 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.173 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:47:30.173 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6812", 2022-06-29T15:47:30.174 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:47:30.174 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.174 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.174 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:47:30.175 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6813", 2022-06-29T15:47:30.175 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:47:30.175 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:30.175 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:30.175 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.176 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6809/13394", 2022-06-29T15:47:30.176 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6811/13394", 2022-06-29T15:47:30.176 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6815/13394", 2022-06-29T15:47:30.176 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6813/13394", 2022-06-29T15:47:30.177 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:47:30.177 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:47:30.177 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:47:30.177 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:30.177 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:30.178 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:47:30.178 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_xinfo": [ 2022-06-29T15:47:30.178 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.178 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2022-06-29T15:47:30.179 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:47:30.179 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:47:30.179 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:47:30.179 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:47:30.179 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:47:30.180 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.720513+0000", 2022-06-29T15:47:30.180 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:47:30.180 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.180 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.181 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2022-06-29T15:47:30.181 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:47:30.181 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:47:30.181 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:47:30.181 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:47:30.182 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:47:30.182 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.772612+0000", 2022-06-29T15:47:30.182 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:47:30.182 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.183 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.183 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2022-06-29T15:47:30.183 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:47:30.183 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:47:30.183 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:47:30.184 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:47:30.184 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:47:30.184 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.450223+0000", 2022-06-29T15:47:30.184 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:47:30.185 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.185 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.185 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2022-06-29T15:47:30.185 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:47:30.185 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:47:30.186 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:47:30.186 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:47:30.186 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:47:30.186 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.477856+0000", 2022-06-29T15:47:30.187 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:47:30.187 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.187 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.187 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2022-06-29T15:47:30.187 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:47:30.188 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:47:30.188 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:47:30.188 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:47:30.188 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:47:30.189 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.539485+0000", 2022-06-29T15:47:30.189 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:47:30.189 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.189 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.189 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2022-06-29T15:47:30.190 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:47:30.190 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:47:30.190 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:47:30.190 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:47:30.191 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:47:30.191 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.487040+0000", 2022-06-29T15:47:30.191 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:47:30.191 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.191 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.192 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2022-06-29T15:47:30.192 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:47:30.192 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:47:30.192 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:47:30.192 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:47:30.193 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:47:30.193 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.748685+0000", 2022-06-29T15:47:30.193 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:47:30.193 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.194 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:30.194 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2022-06-29T15:47:30.194 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:47:30.194 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:47:30.194 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:47:30.195 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:47:30.195 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:47:30.195 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.459516+0000", 2022-06-29T15:47:30.195 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:47:30.196 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:30.196 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:47:30.196 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap": [], 2022-06-29T15:47:30.196 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap_items": [], 2022-06-29T15:47:30.196 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_temp": [], 2022-06-29T15:47:30.197 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_temp": [], 2022-06-29T15:47:30.197 INFO:teuthology.orchestra.run.smithi047.stdout: "blacklist": { 2022-06-29T15:47:30.197 INFO:teuthology.orchestra.run.smithi047.stdout: "172.21.15.47:6837/2427298469": "2022-06-30T15:47:23.406630+0000", 2022-06-29T15:47:30.197 INFO:teuthology.orchestra.run.smithi047.stdout: "172.21.15.47:6836/2427298469": "2022-06-30T15:47:23.406630+0000" 2022-06-29T15:47:30.197 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.198 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profiles": { 2022-06-29T15:47:30.198 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs_data": { 2022-06-29T15:47:30.198 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-device-class": "", 2022-06-29T15:47:30.198 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2022-06-29T15:47:30.199 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-root": "default", 2022-06-29T15:47:30.199 INFO:teuthology.orchestra.run.smithi047.stdout: "jerasure-per-chunk-alignment": "false", 2022-06-29T15:47:30.199 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2022-06-29T15:47:30.199 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "2", 2022-06-29T15:47:30.199 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2022-06-29T15:47:30.200 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2022-06-29T15:47:30.200 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van", 2022-06-29T15:47:30.200 INFO:teuthology.orchestra.run.smithi047.stdout: "w": "8" 2022-06-29T15:47:30.200 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.201 INFO:teuthology.orchestra.run.smithi047.stdout: "default": { 2022-06-29T15:47:30.201 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2022-06-29T15:47:30.201 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2022-06-29T15:47:30.201 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "1", 2022-06-29T15:47:30.201 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2022-06-29T15:47:30.202 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2022-06-29T15:47:30.202 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van" 2022-06-29T15:47:30.202 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:30.203 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:30.203 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps_queue": [], 2022-06-29T15:47:30.203 INFO:teuthology.orchestra.run.smithi047.stdout: "new_removed_snaps": [], 2022-06-29T15:47:30.203 INFO:teuthology.orchestra.run.smithi047.stdout: "new_purged_snaps": [], 2022-06-29T15:47:30.204 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_node_flags": {}, 2022-06-29T15:47:30.204 INFO:teuthology.orchestra.run.smithi047.stdout: "device_class_flags": {} 2022-06-29T15:47:30.204 INFO:teuthology.orchestra.run.smithi047.stdout:} 2022-06-29T15:47:30.206 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.911+0000 7f650280c700 1 -- 172.21.15.47:0/1056803055 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f64dc053aa0 msgr2=0x7f64dc055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:30.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.911+0000 7f650280c700 1 --2- 172.21.15.47:0/1056803055 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f64dc053aa0 0x7f64dc055f60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f64f800d000 tx=0x7f64f8007a00).stop 2022-06-29T15:47:30.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.911+0000 7f650280c700 1 -- 172.21.15.47:0/1056803055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64fc0afdc0 msgr2=0x7f64fc0b4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:30.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.911+0000 7f650280c700 1 --2- 172.21.15.47:0/1056803055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64fc0afdc0 0x7f64fc0b4ba0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7f64ec002f70 tx=0x7f64ec014000).stop 2022-06-29T15:47:30.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.911+0000 7f650280c700 1 -- 172.21.15.47:0/1056803055 shutdown_connections 2022-06-29T15:47:30.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.911+0000 7f650280c700 1 --2- 172.21.15.47:0/1056803055 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f64fc0b5ac0 0x7f64fc0ab750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:30.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.911+0000 7f650280c700 1 --2- 172.21.15.47:0/1056803055 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f64dc053aa0 0x7f64dc055f60 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:30.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.911+0000 7f650280c700 1 --2- 172.21.15.47:0/1056803055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64fc0b02d0 0x7f64fc0b0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:30.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.911+0000 7f650280c700 1 --2- 172.21.15.47:0/1056803055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64fc0afdc0 0x7f64fc0b4ba0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:30.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.911+0000 7f650280c700 1 -- 172.21.15.47:0/1056803055 >> 172.21.15.47:0/1056803055 conn(0x7f64fc062560 msgr2=0x7f64fc13ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:30.209 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.911+0000 7f650280c700 1 -- 172.21.15.47:0/1056803055 shutdown_connections 2022-06-29T15:47:30.209 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:29.911+0000 7f650280c700 1 -- 172.21.15.47:0/1056803055 wait complete. 2022-06-29T15:47:30.209 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:47:30.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.319+0000 7f04c2d21700 1 Processor -- start 2022-06-29T15:47:30.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.319+0000 7f04c2d21700 1 -- start start 2022-06-29T15:47:30.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.323+0000 7f04c2d21700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f04bc097a80 0x7f04bc097ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:30.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.323+0000 7f04c2d21700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f04bc0983e0 0x7f04bc140520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:30.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.323+0000 7f04c2d21700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04bc0b11d0 con 0x7f04bc0615f0 2022-06-29T15:47:30.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.323+0000 7f04c2d21700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04bc0619d0 con 0x7f04bc097a80 2022-06-29T15:47:30.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.323+0000 7f04c2d21700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04bc0b0f30 con 0x7f04bc0983e0 2022-06-29T15:47:30.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.323+0000 7f04c0abd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f04bc0615f0 0x7f04bc1441c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34634/0 (socket says 172.21.15.47:34634) 2022-06-29T15:47:30.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.323+0000 7f04c0abd700 1 -- 172.21.15.47:0/1556305567 learned_addr learned my addr 172.21.15.47:0/1556305567 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:47:30.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.323+0000 7f04bbfff700 1 --2- 172.21.15.47:0/1556305567 >> v2:172.21.15.47:3300/0 conn(0x7f04bc097a80 0x7f04bc097ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:30.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.323+0000 7f04c12be700 1 --2- 172.21.15.47:0/1556305567 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f04bc0983e0 0x7f04bc140520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:30.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.323+0000 7f04bb7fe700 1 -- 172.21.15.47:0/1556305567 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1832957217 0 0) 0x7f04bc0b11d0 con 0x7f04bc0615f0 2022-06-29T15:47:30.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.323+0000 7f04bb7fe700 1 -- 172.21.15.47:0/1556305567 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04a0003260 con 0x7f04bc0615f0 2022-06-29T15:47:30.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.323+0000 7f04bbfff700 1 -- 172.21.15.47:0/1556305567 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f04bc0983e0 msgr2=0x7f04bc140520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:30.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.323+0000 7f04bbfff700 1 --2- 172.21.15.47:0/1556305567 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f04bc0983e0 0x7f04bc140520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:30.335 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.323+0000 7f04bbfff700 1 -- 172.21.15.47:0/1556305567 >> v1:172.21.15.47:6789/0 conn(0x7f04bc0615f0 legacy=0x7f04bc1441c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:30.335 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.323+0000 7f04bbfff700 1 -- 172.21.15.47:0/1556305567 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04bc0b1b00 con 0x7f04bc097a80 2022-06-29T15:47:30.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.323+0000 7f04bbfff700 1 --2- 172.21.15.47:0/1556305567 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04bc097a80 0x7f04bc097ef0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7f04a8002f70 tx=0x7f04a8006df0).ready entity=mon.0 client_cookie=cb382197f76ddee1 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:30.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.323+0000 7f04bb7fe700 1 -- 172.21.15.47:0/1556305567 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f04a800d4a0 con 0x7f04bc097a80 2022-06-29T15:47:30.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.323+0000 7f04bb7fe700 1 -- 172.21.15.47:0/1556305567 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04a8000e90 con 0x7f04bc097a80 2022-06-29T15:47:30.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.323+0000 7f04bb7fe700 1 -- 172.21.15.47:0/1556305567 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f04a800d7f0 con 0x7f04bc097a80 2022-06-29T15:47:30.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.327+0000 7f04c2d21700 1 -- 172.21.15.47:0/1556305567 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04bc097a80 msgr2=0x7f04bc097ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:30.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.327+0000 7f04c2d21700 1 --2- 172.21.15.47:0/1556305567 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04bc097a80 0x7f04bc097ef0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7f04a8002f70 tx=0x7f04a8006df0).stop 2022-06-29T15:47:30.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.327+0000 7f04c2d21700 1 -- 172.21.15.47:0/1556305567 shutdown_connections 2022-06-29T15:47:30.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.327+0000 7f04c2d21700 1 --2- 172.21.15.47:0/1556305567 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f04bc0983e0 0x7f04bc140520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:30.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.327+0000 7f04c2d21700 1 --2- 172.21.15.47:0/1556305567 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04bc097a80 0x7f04bc097ef0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:30.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.327+0000 7f04c2d21700 1 -- 172.21.15.47:0/1556305567 >> 172.21.15.47:0/1556305567 conn(0x7f04bc05f0b0 msgr2=0x7f04bc05f490 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:30.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.327+0000 7f04c2d21700 1 -- 172.21.15.47:0/1556305567 shutdown_connections 2022-06-29T15:47:30.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.327+0000 7f04c2d21700 1 -- 172.21.15.47:0/1556305567 wait complete. 2022-06-29T15:47:30.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.327+0000 7f04c2d21700 1 Processor -- start 2022-06-29T15:47:30.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.327+0000 7f04c2d21700 1 -- start start 2022-06-29T15:47:30.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.327+0000 7f04c2d21700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04bc0615f0 0x7f04bc09b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:30.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.327+0000 7f04c2d21700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04bc097a80 0x7f04bc0af690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:30.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.327+0000 7f04c2d21700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f04bc0983e0 0x7f04bc0aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:30.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.327+0000 7f04c2d21700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04bc155710 con 0x7f04bc0615f0 2022-06-29T15:47:30.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.327+0000 7f04c2d21700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04bc099f70 con 0x7f04bc097a80 2022-06-29T15:47:30.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.327+0000 7f04c2d21700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04bc0a9b30 con 0x7f04bc0983e0 2022-06-29T15:47:30.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.327+0000 7f04c0abd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04bc0615f0 0x7f04bc09b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:30.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.327+0000 7f04c0abd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04bc0615f0 0x7f04bc09b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49088/0 (socket says 172.21.15.47:49088) 2022-06-29T15:47:30.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.327+0000 7f04c0abd700 1 -- 172.21.15.47:0/3883690593 learned_addr learned my addr 172.21.15.47:0/3883690593 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:30.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.327+0000 7f04bbfff700 1 --2- 172.21.15.47:0/3883690593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04bc097a80 0x7f04bc0af690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:30.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.327+0000 7f04c12be700 1 --2- 172.21.15.47:0/3883690593 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f04bc0983e0 0x7f04bc0aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:30.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.331+0000 7f04c0abd700 1 -- 172.21.15.47:0/3883690593 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f04bc0983e0 msgr2=0x7f04bc0aebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:30.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.331+0000 7f04c0abd700 1 --2- 172.21.15.47:0/3883690593 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f04bc0983e0 0x7f04bc0aebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:30.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.331+0000 7f04c0abd700 1 -- 172.21.15.47:0/3883690593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04bc097a80 msgr2=0x7f04bc0af690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:30.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.331+0000 7f04c0abd700 1 --2- 172.21.15.47:0/3883690593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04bc097a80 0x7f04bc0af690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:30.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.331+0000 7f04c0abd700 1 -- 172.21.15.47:0/3883690593 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04a800b180 con 0x7f04bc0615f0 2022-06-29T15:47:30.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.331+0000 7f04c0abd700 1 --2- 172.21.15.47:0/3883690593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04bc0615f0 0x7f04bc09b890 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7f04b0001110 tx=0x7f04b0007aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:30.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.331+0000 7f04b9ffb700 1 -- 172.21.15.47:0/3883690593 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f04b000ce80 con 0x7f04bc0615f0 2022-06-29T15:47:30.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.331+0000 7f04b9ffb700 1 -- 172.21.15.47:0/3883690593 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b00070d0 con 0x7f04bc0615f0 2022-06-29T15:47:30.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.331+0000 7f04c2d21700 1 -- 172.21.15.47:0/3883690593 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04bc09a1a0 con 0x7f04bc0615f0 2022-06-29T15:47:30.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.331+0000 7f04b9ffb700 1 -- 172.21.15.47:0/3883690593 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f04b0007430 con 0x7f04bc0615f0 2022-06-29T15:47:30.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.331+0000 7f04c2d21700 1 -- 172.21.15.47:0/3883690593 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04bc0a9dd0 con 0x7f04bc0615f0 2022-06-29T15:47:30.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.331+0000 7f04c2d21700 1 -- 172.21.15.47:0/3883690593 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04bc0573d0 con 0x7f04bc0615f0 2022-06-29T15:47:30.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.331+0000 7f04b9ffb700 1 -- 172.21.15.47:0/3883690593 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f04b0010070 con 0x7f04bc0615f0 2022-06-29T15:47:30.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.331+0000 7f04b9ffb700 1 --2- 172.21.15.47:0/3883690593 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f049c053950 0x7f049c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:30.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.331+0000 7f04b9ffb700 1 -- 172.21.15.47:0/3883690593 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f04b006b340 con 0x7f04bc0615f0 2022-06-29T15:47:30.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.331+0000 7f04bbfff700 1 --2- 172.21.15.47:0/3883690593 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f049c053950 0x7f049c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:30.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.335+0000 7f04bbfff700 1 --2- 172.21.15.47:0/3883690593 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f049c053950 0x7f049c055e10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7f04a801a000 tx=0x7f04a8014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:30.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.339+0000 7f04b9ffb700 1 -- 172.21.15.47:0/3883690593 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f04b00423e0 con 0x7f04bc0615f0 2022-06-29T15:47:30.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.503+0000 7f04c2d21700 1 -- 172.21.15.47:0/3883690593 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f04bc069e20 con 0x7f04bc0615f0 2022-06-29T15:47:30.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.503+0000 7f04b9ffb700 1 -- 172.21.15.47:0/3883690593 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+1562 (secure 0 0 0) 0x7f04b0018070 con 0x7f04bc0615f0 2022-06-29T15:47:30.507 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:47:30.507 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":26,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5595,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1798995166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1798995166},{"type":"v1","addr":"172.21.15.47:6835","nonce":1798995166}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21},{"gid":5601,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/1641478219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1641478219},{"type":"v1","addr":"172.21.15.64:6833","nonce":1641478219}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21},{"gid":5874,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2599480984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2599480984},{"type":"v1","addr":"172.21.15.47:6837","nonce":2599480984}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":26}],"filesystems":[]} 2022-06-29T15:47:30.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.507+0000 7f04c2d21700 1 -- 172.21.15.47:0/3883690593 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f049c053950 msgr2=0x7f049c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:30.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.507+0000 7f04c2d21700 1 --2- 172.21.15.47:0/3883690593 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f049c053950 0x7f049c055e10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7f04a801a000 tx=0x7f04a8014000).stop 2022-06-29T15:47:30.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.507+0000 7f04c2d21700 1 -- 172.21.15.47:0/3883690593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04bc0615f0 msgr2=0x7f04bc09b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:30.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.507+0000 7f04c2d21700 1 --2- 172.21.15.47:0/3883690593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04bc0615f0 0x7f04bc09b890 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7f04b0001110 tx=0x7f04b0007aa0).stop 2022-06-29T15:47:30.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.507+0000 7f04c2d21700 1 -- 172.21.15.47:0/3883690593 shutdown_connections 2022-06-29T15:47:30.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.507+0000 7f04c2d21700 1 --2- 172.21.15.47:0/3883690593 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f04bc0983e0 0x7f04bc0aebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:30.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.507+0000 7f04c2d21700 1 --2- 172.21.15.47:0/3883690593 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f049c053950 0x7f049c055e10 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:30.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.507+0000 7f04c2d21700 1 --2- 172.21.15.47:0/3883690593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04bc097a80 0x7f04bc0af690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:30.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.507+0000 7f04c2d21700 1 --2- 172.21.15.47:0/3883690593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04bc0615f0 0x7f04bc09b890 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:30.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.507+0000 7f04c2d21700 1 -- 172.21.15.47:0/3883690593 >> 172.21.15.47:0/3883690593 conn(0x7f04bc05f0b0 msgr2=0x7f04bc05f490 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:30.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.507+0000 7f04c2d21700 1 -- 172.21.15.47:0/3883690593 shutdown_connections 2022-06-29T15:47:30.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.507+0000 7f04c2d21700 1 -- 172.21.15.47:0/3883690593 wait complete. 2022-06-29T15:47:30.512 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 26 2022-06-29T15:47:30.521 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:47:30.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.623+0000 7ff01ffff700 1 Processor -- start 2022-06-29T15:47:30.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.623+0000 7ff01ffff700 1 -- start start 2022-06-29T15:47:30.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.623+0000 7ff01ffff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff0200b5c60 0x7ff020144980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:30.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.623+0000 7ff01ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff0200a72f0 0x7ff0200b2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:30.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.623+0000 7ff01ffff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0200a9120 con 0x7ff020060eb0 2022-06-29T15:47:30.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.623+0000 7ff01ffff700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0200b0b80 con 0x7ff0200b5c60 2022-06-29T15:47:30.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.623+0000 7ff01ffff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0200a89e0 con 0x7ff0200a72f0 2022-06-29T15:47:30.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.623+0000 7ff01effd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff0200b5c60 0x7ff020144980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:30.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.623+0000 7ff01f7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff0200a72f0 0x7ff0200b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:30.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.623+0000 7ff01e7fc700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ff020060eb0 0x7ff0200b1e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:42212/0 (socket says 172.21.15.47:42212) 2022-06-29T15:47:30.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.623+0000 7ff01e7fc700 1 -- 172.21.15.47:0/2025506090 learned_addr learned my addr 172.21.15.47:0/2025506090 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:47:30.631 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.623+0000 7ff01effd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff0200b5c60 0x7ff020144980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49096/0 (socket says 172.21.15.47:49096) 2022-06-29T15:47:30.631 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.623+0000 7ff01f7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff0200a72f0 0x7ff0200b2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:54802/0 (socket says 172.21.15.47:54802) 2022-06-29T15:47:30.631 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.623+0000 7ff01dffb700 1 -- 172.21.15.47:0/2025506090 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2145909805 0 0) 0x7ff0200a9120 con 0x7ff020060eb0 2022-06-29T15:47:30.631 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.623+0000 7ff01dffb700 1 -- 172.21.15.47:0/2025506090 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff004003260 con 0x7ff020060eb0 2022-06-29T15:47:30.632 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.623+0000 7ff01f7fe700 1 -- 172.21.15.47:0/2025506090 >> v2:172.21.15.47:3300/0 conn(0x7ff0200b5c60 msgr2=0x7ff020144980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:30.632 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.623+0000 7ff01f7fe700 1 --2- 172.21.15.47:0/2025506090 >> v2:172.21.15.47:3300/0 conn(0x7ff0200b5c60 0x7ff020144980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:30.632 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.623+0000 7ff01f7fe700 1 -- 172.21.15.47:0/2025506090 >> v1:172.21.15.64:6789/0 conn(0x7ff020060eb0 legacy=0x7ff0200b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:30.632 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.627+0000 7ff01f7fe700 1 -- 172.21.15.47:0/2025506090 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0200a8c80 con 0x7ff0200a72f0 2022-06-29T15:47:30.632 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.627+0000 7ff01effd700 1 --2- 172.21.15.47:0/2025506090 >> v2:172.21.15.47:3300/0 conn(0x7ff0200b5c60 0x7ff020144980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:47:30.633 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.627+0000 7ff01f7fe700 1 --2- 172.21.15.47:0/2025506090 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0200a72f0 0x7ff0200b2540 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7ff018002f70 tx=0x7ff018006df0).ready entity=mon.1 client_cookie=7b135c9ffc006cbd server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:30.633 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.627+0000 7ff01dffb700 1 -- 172.21.15.47:0/2025506090 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff01800d4f0 con 0x7ff0200a72f0 2022-06-29T15:47:30.633 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.627+0000 7ff01dffb700 1 -- 172.21.15.47:0/2025506090 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff018000e90 con 0x7ff0200a72f0 2022-06-29T15:47:30.634 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.627+0000 7ff01dffb700 1 -- 172.21.15.47:0/2025506090 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff01800d840 con 0x7ff0200a72f0 2022-06-29T15:47:30.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.627+0000 7ff01ffff700 1 -- 172.21.15.47:0/2025506090 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0200a72f0 msgr2=0x7ff0200b2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:30.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.627+0000 7ff01ffff700 1 --2- 172.21.15.47:0/2025506090 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0200a72f0 0x7ff0200b2540 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7ff018002f70 tx=0x7ff018006df0).stop 2022-06-29T15:47:30.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.627+0000 7ff01ffff700 1 -- 172.21.15.47:0/2025506090 shutdown_connections 2022-06-29T15:47:30.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.627+0000 7ff01ffff700 1 --2- 172.21.15.47:0/2025506090 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0200a72f0 0x7ff0200b2540 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:30.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.627+0000 7ff01ffff700 1 --2- 172.21.15.47:0/2025506090 >> v2:172.21.15.47:3300/0 conn(0x7ff0200b5c60 0x7ff020144980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:30.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.627+0000 7ff01ffff700 1 -- 172.21.15.47:0/2025506090 >> 172.21.15.47:0/2025506090 conn(0x7ff020062560 msgr2=0x7ff020062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:30.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.627+0000 7ff01ffff700 1 -- 172.21.15.47:0/2025506090 shutdown_connections 2022-06-29T15:47:30.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.627+0000 7ff01ffff700 1 -- 172.21.15.47:0/2025506090 wait complete. 2022-06-29T15:47:30.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.631+0000 7ff01ffff700 1 Processor -- start 2022-06-29T15:47:30.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.631+0000 7ff01ffff700 1 -- start start 2022-06-29T15:47:30.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.631+0000 7ff01ffff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff020060eb0 0x7ff0200b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:30.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.631+0000 7ff01ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0200a72f0 0x7ff02013c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:30.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.631+0000 7ff01ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0200b5c60 0x7ff02013c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:30.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.631+0000 7ff01ffff700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0200acde0 con 0x7ff0200b5c60 2022-06-29T15:47:30.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.631+0000 7ff01ffff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0200ac970 con 0x7ff0200a72f0 2022-06-29T15:47:30.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.631+0000 7ff01ffff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff02015a710 con 0x7ff020060eb0 2022-06-29T15:47:30.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.631+0000 7ff01f7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0200b5c60 0x7ff02013c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:30.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.631+0000 7ff01f7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0200b5c60 0x7ff02013c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49104/0 (socket says 172.21.15.47:49104) 2022-06-29T15:47:30.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.631+0000 7ff01effd700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff020060eb0 0x7ff0200b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:30.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.631+0000 7ff01f7fe700 1 -- 172.21.15.47:0/2547771915 learned_addr learned my addr 172.21.15.47:0/2547771915 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:30.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.631+0000 7ff01e7fc700 1 --2- 172.21.15.47:0/2547771915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0200a72f0 0x7ff02013c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:30.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.631+0000 7ff01f7fe700 1 -- 172.21.15.47:0/2547771915 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff020060eb0 msgr2=0x7ff0200b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:30.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.631+0000 7ff01f7fe700 1 --2- 172.21.15.47:0/2547771915 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff020060eb0 0x7ff0200b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:30.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.631+0000 7ff01f7fe700 1 -- 172.21.15.47:0/2547771915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0200a72f0 msgr2=0x7ff02013c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:30.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.631+0000 7ff01f7fe700 1 --2- 172.21.15.47:0/2547771915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0200a72f0 0x7ff02013c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:30.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.631+0000 7ff01f7fe700 1 -- 172.21.15.47:0/2547771915 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff01400b120 con 0x7ff0200b5c60 2022-06-29T15:47:30.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.631+0000 7ff01f7fe700 1 --2- 172.21.15.47:0/2547771915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0200b5c60 0x7ff02013c8a0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7ff018002f70 tx=0x7ff018014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:30.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.635+0000 7ff00ffff700 1 -- 172.21.15.47:0/2547771915 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff018016070 con 0x7ff0200b5c60 2022-06-29T15:47:30.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.635+0000 7ff00ffff700 1 -- 172.21.15.47:0/2547771915 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff018012670 con 0x7ff0200b5c60 2022-06-29T15:47:30.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.635+0000 7ff01ffff700 1 -- 172.21.15.47:0/2547771915 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff01800b180 con 0x7ff0200b5c60 2022-06-29T15:47:30.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.635+0000 7ff00ffff700 1 -- 172.21.15.47:0/2547771915 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0180129d0 con 0x7ff0200b5c60 2022-06-29T15:47:30.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.635+0000 7ff01ffff700 1 -- 172.21.15.47:0/2547771915 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff02013cd90 con 0x7ff0200b5c60 2022-06-29T15:47:30.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.635+0000 7ff01ffff700 1 -- 172.21.15.47:0/2547771915 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff020062df0 con 0x7ff0200b5c60 2022-06-29T15:47:30.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.635+0000 7ff00ffff700 1 -- 172.21.15.47:0/2547771915 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff018024030 con 0x7ff0200b5c60 2022-06-29T15:47:30.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.635+0000 7ff00ffff700 1 --2- 172.21.15.47:0/2547771915 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7feffc053950 0x7feffc055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:30.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.635+0000 7ff01effd700 1 --2- 172.21.15.47:0/2547771915 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7feffc053950 0x7feffc055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:30.651 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.635+0000 7ff00ffff700 1 -- 172.21.15.47:0/2547771915 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5686+0+0 (secure 0 0 0) 0x7ff01806c700 con 0x7ff0200b5c60 2022-06-29T15:47:30.651 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.639+0000 7ff01effd700 1 --2- 172.21.15.47:0/2547771915 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7feffc053950 0x7feffc055e10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7ff014008000 tx=0x7ff014008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:30.651 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.643+0000 7ff00ffff700 1 -- 172.21.15.47:0/2547771915 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff0180437a0 con 0x7ff0200b5c60 2022-06-29T15:47:30.826 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.803+0000 7ff01ffff700 1 -- 172.21.15.47:0/2547771915 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff02015a8e0 con 0x7ff0200b5c60 2022-06-29T15:47:30.827 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.803+0000 7ff00ffff700 1 -- 172.21.15.47:0/2547771915 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+1562 (secure 0 0 0) 0x7ff018022090 con 0x7ff0200b5c60 2022-06-29T15:47:30.828 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:47:30.828 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":26,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5595,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1798995166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1798995166},{"type":"v1","addr":"172.21.15.47:6835","nonce":1798995166}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21},{"gid":5601,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/1641478219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1641478219},{"type":"v1","addr":"172.21.15.64:6833","nonce":1641478219}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21},{"gid":5874,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2599480984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2599480984},{"type":"v1","addr":"172.21.15.47:6837","nonce":2599480984}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":26}],"filesystems":[]} 2022-06-29T15:47:30.831 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.807+0000 7ff01ffff700 1 -- 172.21.15.47:0/2547771915 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7feffc053950 msgr2=0x7feffc055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:30.832 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.807+0000 7ff01ffff700 1 --2- 172.21.15.47:0/2547771915 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7feffc053950 0x7feffc055e10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7ff014008000 tx=0x7ff014008040).stop 2022-06-29T15:47:30.832 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.807+0000 7ff01ffff700 1 -- 172.21.15.47:0/2547771915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0200b5c60 msgr2=0x7ff02013c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:30.832 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.807+0000 7ff01ffff700 1 --2- 172.21.15.47:0/2547771915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0200b5c60 0x7ff02013c8a0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7ff018002f70 tx=0x7ff018014000).stop 2022-06-29T15:47:30.833 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.807+0000 7ff01ffff700 1 -- 172.21.15.47:0/2547771915 shutdown_connections 2022-06-29T15:47:30.833 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.807+0000 7ff01ffff700 1 --2- 172.21.15.47:0/2547771915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0200b5c60 0x7ff02013c8a0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:30.833 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.807+0000 7ff01ffff700 1 --2- 172.21.15.47:0/2547771915 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7feffc053950 0x7feffc055e10 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:30.834 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.807+0000 7ff01ffff700 1 --2- 172.21.15.47:0/2547771915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0200a72f0 0x7ff02013c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:30.834 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.807+0000 7ff01ffff700 1 --2- 172.21.15.47:0/2547771915 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff020060eb0 0x7ff0200b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:30.835 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.807+0000 7ff01ffff700 1 -- 172.21.15.47:0/2547771915 >> 172.21.15.47:0/2547771915 conn(0x7ff020062560 msgr2=0x7ff020141a50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:30.835 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.807+0000 7ff01ffff700 1 -- 172.21.15.47:0/2547771915 shutdown_connections 2022-06-29T15:47:30.835 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:30.807+0000 7ff01ffff700 1 -- 172.21.15.47:0/2547771915 wait complete. 2022-06-29T15:47:30.836 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 26 2022-06-29T15:47:30.836 INFO:tasks.ceph.mds.a:Restarting daemon 2022-06-29T15:47:30.837 INFO:tasks.ceph.mds.a:Stopping old one... 2022-06-29T15:47:30.837 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-06-29T15:47:30.837 INFO:teuthology.orchestra.run:waiting for 300 2022-06-29T15:47:30.838 INFO:tasks.ceph.mds.c:Restarting daemon 2022-06-29T15:47:30.838 INFO:tasks.ceph.mds.c:Stopping old one... 2022-06-29T15:47:30.839 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2022-06-29T15:47:30.839 INFO:teuthology.orchestra.run:waiting for 300 2022-06-29T15:47:30.839 INFO:tasks.ceph.mds.b:Restarting daemon 2022-06-29T15:47:30.839 INFO:tasks.ceph.mds.b:Stopping old one... 2022-06-29T15:47:30.840 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-06-29T15:47:30.840 INFO:teuthology.orchestra.run:waiting for 300 2022-06-29T15:47:30.855 INFO:tasks.ceph.mds.a:Stopped 2022-06-29T15:47:30.856 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-06-29T15:47:30.857 INFO:tasks.ceph.mds.c:Stopped 2022-06-29T15:47:30.857 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2022-06-29T15:47:30.859 INFO:tasks.ceph.mds.b:Stopped 2022-06-29T15:47:30.859 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-06-29T15:47:30.905 INFO:tasks.ceph.mds.a:Started 2022-06-29T15:47:30.906 DEBUG:teuthology.parallel:result is None 2022-06-29T15:47:30.909 INFO:tasks.ceph.mds.b:Started 2022-06-29T15:47:30.909 INFO:tasks.ceph.mds.c:Started 2022-06-29T15:47:30.910 DEBUG:teuthology.parallel:result is None 2022-06-29T15:47:30.920 DEBUG:teuthology.parallel:result is None 2022-06-29T15:47:30.921 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-06-29T15:47:30.976 INFO:tasks.ceph.mds.c.smithi047.stdout:starting mds.c at 2022-06-29T15:47:30.982 INFO:tasks.ceph.mds.b.smithi064.stdout:starting mds.b at 2022-06-29T15:47:30.986 INFO:tasks.ceph.mds.a.smithi047.stdout:starting mds.a at 2022-06-29T15:47:31.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.035+0000 7ff118ce0700 1 Processor -- start 2022-06-29T15:47:31.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.035+0000 7ff118ce0700 1 -- start start 2022-06-29T15:47:31.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.035+0000 7ff118ce0700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff11405f760 0x7ff1140ac210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:31.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.035+0000 7ff118ce0700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1140af710 con 0x7ff1140af190 2022-06-29T15:47:31.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.035+0000 7ff118ce0700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff114060d40 con 0x7ff1140aca80 2022-06-29T15:47:31.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.035+0000 7ff118ce0700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1140aeec0 con 0x7ff11405f760 2022-06-29T15:47:31.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.035+0000 7ff111d9b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff1140af190 0x7ff1140abb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34686/0 (socket says 172.21.15.47:34686) 2022-06-29T15:47:31.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.035+0000 7ff111d9b700 1 -- 172.21.15.47:0/2085044374 learned_addr learned my addr 172.21.15.47:0/2085044374 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:47:31.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.035+0000 7ff112d9d700 1 --2- 172.21.15.47:0/2085044374 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff11405f760 0x7ff1140ac210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:31.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.035+0000 7ff11159a700 1 -- 172.21.15.47:0/2085044374 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3899443018 0 0) 0x7ff1140af710 con 0x7ff1140af190 2022-06-29T15:47:31.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.035+0000 7ff11159a700 1 -- 172.21.15.47:0/2085044374 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0f8003260 con 0x7ff1140af190 2022-06-29T15:47:31.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.035+0000 7ff11159a700 1 -- 172.21.15.47:0/2085044374 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3141289450 0 0) 0x7ff114060d40 con 0x7ff1140aca80 2022-06-29T15:47:31.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.035+0000 7ff11159a700 1 -- 172.21.15.47:0/2085044374 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1140af710 con 0x7ff1140aca80 2022-06-29T15:47:31.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.035+0000 7ff11159a700 1 -- 172.21.15.47:0/2085044374 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2852358652 0 0) 0x7ff0f8003260 con 0x7ff1140af190 2022-06-29T15:47:31.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.035+0000 7ff11159a700 1 -- 172.21.15.47:0/2085044374 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff11405f760 msgr2=0x7ff1140ac210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:31.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.035+0000 7ff11159a700 1 --2- 172.21.15.47:0/2085044374 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff11405f760 0x7ff1140ac210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:31.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.035+0000 7ff11159a700 1 -- 172.21.15.47:0/2085044374 >> v1:172.21.15.64:6789/0 conn(0x7ff1140aca80 legacy=0x7ff11413f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:31.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.035+0000 7ff11159a700 1 -- 172.21.15.47:0/2085044374 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff11413fce0 con 0x7ff1140af190 2022-06-29T15:47:31.045 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.035+0000 7ff11159a700 1 -- 172.21.15.47:0/2085044374 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7ff104001660 con 0x7ff1140af190 2022-06-29T15:47:31.045 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.035+0000 7ff11159a700 1 -- 172.21.15.47:0/2085044374 >> v1:172.21.15.47:6789/0 conn(0x7ff1140af190 legacy=0x7ff1140abb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:31.045 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.035+0000 7ff11159a700 1 --2- 172.21.15.47:0/2085044374 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff11405f760 0x7ff0f8003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:31.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.035+0000 7ff11159a700 1 --2- 172.21.15.47:0/2085044374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0f8003e70 0x7ff0f8004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:31.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.035+0000 7ff11159a700 1 --2- 172.21.15.47:0/2085044374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0f80049f0 0x7ff0f8004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:31.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.035+0000 7ff11159a700 1 -- 172.21.15.47:0/2085044374 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1140aeec0 con 0x7ff0f80049f0 2022-06-29T15:47:31.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.035+0000 7ff11159a700 1 -- 172.21.15.47:0/2085044374 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f800b830 con 0x7ff0f8003e70 2022-06-29T15:47:31.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.035+0000 7ff11159a700 1 -- 172.21.15.47:0/2085044374 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f800b9f0 con 0x7ff11405f760 2022-06-29T15:47:31.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.035+0000 7ff112d9d700 1 --2- 172.21.15.47:0/2085044374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0f80049f0 0x7ff0f8004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:31.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.035+0000 7ff111d9b700 1 --2- 172.21.15.47:0/2085044374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0f8003e70 0x7ff0f8004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:31.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.035+0000 7ff11259c700 1 --2- 172.21.15.47:0/2085044374 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff11405f760 0x7ff0f8003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:31.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.035+0000 7ff112d9d700 1 -- 172.21.15.47:0/2085044374 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff11405f760 msgr2=0x7ff0f8003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:31.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.035+0000 7ff112d9d700 1 --2- 172.21.15.47:0/2085044374 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff11405f760 0x7ff0f8003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:31.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.035+0000 7ff112d9d700 1 -- 172.21.15.47:0/2085044374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0f8003e70 msgr2=0x7ff0f8004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:31.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.035+0000 7ff112d9d700 1 --2- 172.21.15.47:0/2085044374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0f8003e70 0x7ff0f8004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:31.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.035+0000 7ff112d9d700 1 -- 172.21.15.47:0/2085044374 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f8003260 con 0x7ff0f80049f0 2022-06-29T15:47:31.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.035+0000 7ff112d9d700 1 --2- 172.21.15.47:0/2085044374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0f80049f0 0x7ff0f8004e70 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7ff10c00c000 tx=0x7ff10c0091d0).ready entity=mon.0 client_cookie=f7435b37f129d5f4 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:31.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.035+0000 7ff11159a700 1 -- 172.21.15.47:0/2085044374 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff10c00e6e0 con 0x7ff0f80049f0 2022-06-29T15:47:31.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.035+0000 7ff11159a700 1 -- 172.21.15.47:0/2085044374 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff10c00e840 con 0x7ff0f80049f0 2022-06-29T15:47:31.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.035+0000 7ff11159a700 1 -- 172.21.15.47:0/2085044374 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff10c00eba0 con 0x7ff0f80049f0 2022-06-29T15:47:31.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.039+0000 7ff118ce0700 1 -- 172.21.15.47:0/2085044374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0f80049f0 msgr2=0x7ff0f8004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:31.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.039+0000 7ff118ce0700 1 --2- 172.21.15.47:0/2085044374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0f80049f0 0x7ff0f8004e70 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7ff10c00c000 tx=0x7ff10c0091d0).stop 2022-06-29T15:47:31.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.039+0000 7ff112d9d700 1 -- 172.21.15.47:0/2085044374 reap_dead start 2022-06-29T15:47:31.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.039+0000 7ff118ce0700 1 -- 172.21.15.47:0/2085044374 shutdown_connections 2022-06-29T15:47:31.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.039+0000 7ff118ce0700 1 -- 172.21.15.47:0/2085044374 >> 172.21.15.47:0/2085044374 conn(0x7ff1140626f0 msgr2=0x7ff114062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:31.052 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.039+0000 7ff118ce0700 1 -- 172.21.15.47:0/2085044374 shutdown_connections 2022-06-29T15:47:31.052 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.039+0000 7ff118ce0700 1 -- 172.21.15.47:0/2085044374 wait complete. 2022-06-29T15:47:31.052 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.039+0000 7ff118ce0700 1 Processor -- start 2022-06-29T15:47:31.053 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.039+0000 7ff118ce0700 1 -- start start 2022-06-29T15:47:31.059 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.039+0000 7ff118ce0700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff114142ad0 0x7ff114144420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:31.059 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.039+0000 7ff118ce0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1140aaec0 0x7ff1140ab340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:31.060 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.039+0000 7ff118ce0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff114151930 0x7ff114151db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:31.060 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.039+0000 7ff118ce0700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff114144c50 con 0x7ff114151930 2022-06-29T15:47:31.061 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.039+0000 7ff118ce0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff114142eb0 con 0x7ff1140aaec0 2022-06-29T15:47:31.061 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.039+0000 7ff118ce0700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff114143970 con 0x7ff114142ad0 2022-06-29T15:47:31.062 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.039+0000 7ff112d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff114151930 0x7ff114151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:31.062 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.039+0000 7ff112d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff114151930 0x7ff114151db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49148/0 (socket says 172.21.15.47:49148) 2022-06-29T15:47:31.063 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.039+0000 7ff112d9d700 1 -- 172.21.15.47:0/1493688311 learned_addr learned my addr 172.21.15.47:0/1493688311 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:31.063 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.039+0000 7ff11259c700 1 --2- 172.21.15.47:0/1493688311 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff114142ad0 0x7ff114144420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:31.063 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.039+0000 7ff111d9b700 1 --2- 172.21.15.47:0/1493688311 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1140aaec0 0x7ff1140ab340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:31.064 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.039+0000 7ff112d9d700 1 -- 172.21.15.47:0/1493688311 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff114142ad0 msgr2=0x7ff114144420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:31.064 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.039+0000 7ff112d9d700 1 --2- 172.21.15.47:0/1493688311 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff114142ad0 0x7ff114144420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:31.065 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.039+0000 7ff112d9d700 1 -- 172.21.15.47:0/1493688311 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1140aaec0 msgr2=0x7ff1140ab340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:31.065 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.039+0000 7ff112d9d700 1 --2- 172.21.15.47:0/1493688311 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1140aaec0 0x7ff1140ab340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:31.066 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.039+0000 7ff112d9d700 1 -- 172.21.15.47:0/1493688311 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff114143c10 con 0x7ff114151930 2022-06-29T15:47:31.066 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.039+0000 7ff112d9d700 1 --2- 172.21.15.47:0/1493688311 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff114151930 0x7ff114151db0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7ff104001840 tx=0x7ff10c01a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:31.067 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.039+0000 7ff1037fe700 1 -- 172.21.15.47:0/1493688311 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff10c00e6e0 con 0x7ff114151930 2022-06-29T15:47:31.067 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.039+0000 7ff1037fe700 1 -- 172.21.15.47:0/1493688311 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff10c00ee50 con 0x7ff114151930 2022-06-29T15:47:31.068 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.039+0000 7ff1037fe700 1 -- 172.21.15.47:0/1493688311 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff10c01b640 con 0x7ff114151930 2022-06-29T15:47:31.068 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.039+0000 7ff118ce0700 1 -- 172.21.15.47:0/1493688311 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff114143eb0 con 0x7ff114151930 2022-06-29T15:47:31.069 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.039+0000 7ff118ce0700 1 -- 172.21.15.47:0/1493688311 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff114142390 con 0x7ff114151930 2022-06-29T15:47:31.069 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.043+0000 7ff118ce0700 1 -- 172.21.15.47:0/1493688311 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff114144070 con 0x7ff114151930 2022-06-29T15:47:31.069 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.043+0000 7ff1037fe700 1 -- 172.21.15.47:0/1493688311 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff10c01b7a0 con 0x7ff114151930 2022-06-29T15:47:31.069 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.043+0000 7ff1037fe700 1 --2- 172.21.15.47:0/1493688311 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff0f0053b00 0x7ff0f0055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:31.070 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.043+0000 7ff1037fe700 1 -- 172.21.15.47:0/1493688311 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5686+0+0 (secure 0 0 0) 0x7ff10c06d810 con 0x7ff114151930 2022-06-29T15:47:31.070 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.043+0000 7ff11259c700 1 --2- 172.21.15.47:0/1493688311 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff0f0053b00 0x7ff0f0055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:31.070 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.043+0000 7ff11259c700 1 --2- 172.21.15.47:0/1493688311 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff0f0053b00 0x7ff0f0055fc0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7ff10800d000 tx=0x7ff10800c040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:31.070 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.051+0000 7ff1037fe700 1 -- 172.21.15.47:0/1493688311 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff10c0447a0 con 0x7ff114151930 2022-06-29T15:47:31.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:31.187+0000 7ff118ce0700 1 -- 172.21.15.47:0/1493688311 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7ff1140573d0 con 0x7ff114151930 2022-06-29T15:47:32.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.015+0000 7ff1037fe700 1 -- 172.21.15.47:0/1493688311 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v64) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff10c016070 con 0x7ff114151930 2022-06-29T15:47:32.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.019+0000 7ff118ce0700 1 -- 172.21.15.47:0/1493688311 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff0f0053b00 msgr2=0x7ff0f0055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:32.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.019+0000 7ff118ce0700 1 --2- 172.21.15.47:0/1493688311 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff0f0053b00 0x7ff0f0055fc0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7ff10800d000 tx=0x7ff10800c040).stop 2022-06-29T15:47:32.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.019+0000 7ff118ce0700 1 -- 172.21.15.47:0/1493688311 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff114151930 msgr2=0x7ff114151db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:32.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.019+0000 7ff118ce0700 1 --2- 172.21.15.47:0/1493688311 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff114151930 0x7ff114151db0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7ff104001840 tx=0x7ff10c01a000).stop 2022-06-29T15:47:32.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.019+0000 7ff118ce0700 1 -- 172.21.15.47:0/1493688311 shutdown_connections 2022-06-29T15:47:32.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.019+0000 7ff118ce0700 1 --2- 172.21.15.47:0/1493688311 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff114151930 0x7ff114151db0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:32.022 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.019+0000 7ff118ce0700 1 --2- 172.21.15.47:0/1493688311 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff0f0053b00 0x7ff0f0055fc0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:32.022 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.019+0000 7ff118ce0700 1 --2- 172.21.15.47:0/1493688311 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1140aaec0 0x7ff1140ab340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:32.022 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.019+0000 7ff118ce0700 1 --2- 172.21.15.47:0/1493688311 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff114142ad0 0x7ff114144420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:32.022 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.019+0000 7ff118ce0700 1 -- 172.21.15.47:0/1493688311 >> 172.21.15.47:0/1493688311 conn(0x7ff1140626f0 msgr2=0x7ff1140b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:32.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.019+0000 7ff118ce0700 1 -- 172.21.15.47:0/1493688311 shutdown_connections 2022-06-29T15:47:32.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.019+0000 7ff118ce0700 1 -- 172.21.15.47:0/1493688311 wait complete. 2022-06-29T15:47:32.024 INFO:teuthology.orchestra.run.smithi047.stderr: removed all blacklist entries 2022-06-29T15:47:32.035 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2022-06-29T15:47:32.036 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-06-29T15:47:32.147 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.147+0000 7fe62f59e700 1 Processor -- start 2022-06-29T15:47:32.148 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.147+0000 7fe62f59e700 1 -- start start 2022-06-29T15:47:32.152 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.147+0000 7fe62f59e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe630060eb0 0x7fe6300b1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:32.152 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.147+0000 7fe62f59e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe630145090 0x7fe6300b2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:32.152 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.147+0000 7fe62f59e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6300a9120 con 0x7fe6300b5c60 2022-06-29T15:47:32.153 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.147+0000 7fe62f59e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6300b0b80 con 0x7fe630145090 2022-06-29T15:47:32.153 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.147+0000 7fe62f59e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6300a89e0 con 0x7fe630060eb0 2022-06-29T15:47:32.153 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.147+0000 7fe62e59c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe6300b5c60 0x7fe630144980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34708/0 (socket says 172.21.15.47:34708) 2022-06-29T15:47:32.153 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.147+0000 7fe62e59c700 1 -- 172.21.15.47:0/192525535 learned_addr learned my addr 172.21.15.47:0/192525535 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:47:32.153 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.147+0000 7fe62ed9d700 1 --2- 172.21.15.47:0/192525535 >> v2:172.21.15.47:3300/0 conn(0x7fe630145090 0x7fe6300b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:32.154 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.147+0000 7fe627fff700 1 --2- 172.21.15.47:0/192525535 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe630060eb0 0x7fe6300b1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:32.154 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.147+0000 7fe62ed9d700 1 -- 172.21.15.47:0/192525535 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe630060eb0 msgr2=0x7fe6300b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:32.154 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.147+0000 7fe62ed9d700 1 --2- 172.21.15.47:0/192525535 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe630060eb0 0x7fe6300b1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:32.154 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.147+0000 7fe62dd9b700 1 -- 172.21.15.47:0/192525535 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3032918219 0 0) 0x7fe6300a9120 con 0x7fe6300b5c60 2022-06-29T15:47:32.155 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.147+0000 7fe62ed9d700 1 -- 172.21.15.47:0/192525535 >> v1:172.21.15.47:6789/0 conn(0x7fe6300b5c60 legacy=0x7fe630144980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:32.155 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.147+0000 7fe62ed9d700 1 -- 172.21.15.47:0/192525535 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6300a8c80 con 0x7fe630145090 2022-06-29T15:47:32.155 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.147+0000 7fe62ed9d700 1 --2- 172.21.15.47:0/192525535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe630145090 0x7fe6300b2320 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7fe6280032f0 tx=0x7fe6280091d0).ready entity=mon.0 client_cookie=89c3a139ee0fa19f server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:32.156 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.147+0000 7fe62dd9b700 1 -- 172.21.15.47:0/192525535 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe62800b6c0 con 0x7fe630145090 2022-06-29T15:47:32.156 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.147+0000 7fe62dd9b700 1 -- 172.21.15.47:0/192525535 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe62800b820 con 0x7fe630145090 2022-06-29T15:47:32.156 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.147+0000 7fe62dd9b700 1 -- 172.21.15.47:0/192525535 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe62800b990 con 0x7fe630145090 2022-06-29T15:47:32.156 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.147+0000 7fe62f59e700 1 -- 172.21.15.47:0/192525535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe630145090 msgr2=0x7fe6300b2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:32.156 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.147+0000 7fe62f59e700 1 --2- 172.21.15.47:0/192525535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe630145090 0x7fe6300b2320 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7fe6280032f0 tx=0x7fe6280091d0).stop 2022-06-29T15:47:32.157 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.147+0000 7fe62f59e700 1 -- 172.21.15.47:0/192525535 shutdown_connections 2022-06-29T15:47:32.157 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.147+0000 7fe62f59e700 1 --2- 172.21.15.47:0/192525535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe630145090 0x7fe6300b2320 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:32.157 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.147+0000 7fe62f59e700 1 --2- 172.21.15.47:0/192525535 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe630060eb0 0x7fe6300b1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:32.158 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.147+0000 7fe62f59e700 1 -- 172.21.15.47:0/192525535 >> 172.21.15.47:0/192525535 conn(0x7fe630062560 msgr2=0x7fe630062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:32.158 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.147+0000 7fe62f59e700 1 -- 172.21.15.47:0/192525535 shutdown_connections 2022-06-29T15:47:32.159 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.147+0000 7fe62f59e700 1 -- 172.21.15.47:0/192525535 wait complete. 2022-06-29T15:47:32.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.151+0000 7fe62f59e700 1 Processor -- start 2022-06-29T15:47:32.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.151+0000 7fe62f59e700 1 -- start start 2022-06-29T15:47:32.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.151+0000 7fe62f59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe630060eb0 0x7fe63013cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:32.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.151+0000 7fe62f59e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe6300b5c60 0x7fe630140140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:32.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.151+0000 7fe62f59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe630145090 0x7fe63014bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:32.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.151+0000 7fe62f59e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe63013bc60 con 0x7fe630060eb0 2022-06-29T15:47:32.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.151+0000 7fe62f59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe63013d2e0 con 0x7fe630145090 2022-06-29T15:47:32.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.151+0000 7fe62f59e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe63013d580 con 0x7fe6300b5c60 2022-06-29T15:47:32.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.151+0000 7fe62e59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe630060eb0 0x7fe63013cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:32.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.151+0000 7fe62e59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe630060eb0 0x7fe63013cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49162/0 (socket says 172.21.15.47:49162) 2022-06-29T15:47:32.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.151+0000 7fe62e59c700 1 -- 172.21.15.47:0/2418967547 learned_addr learned my addr 172.21.15.47:0/2418967547 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:32.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.151+0000 7fe627fff700 1 --2- 172.21.15.47:0/2418967547 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe6300b5c60 0x7fe630140140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:32.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.151+0000 7fe62ed9d700 1 --2- 172.21.15.47:0/2418967547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe630145090 0x7fe63014bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:32.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.151+0000 7fe62e59c700 1 -- 172.21.15.47:0/2418967547 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe6300b5c60 msgr2=0x7fe630140140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:32.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.151+0000 7fe62e59c700 1 --2- 172.21.15.47:0/2418967547 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe6300b5c60 0x7fe630140140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:32.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.151+0000 7fe62e59c700 1 -- 172.21.15.47:0/2418967547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe630145090 msgr2=0x7fe63014bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:32.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.151+0000 7fe62e59c700 1 --2- 172.21.15.47:0/2418967547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe630145090 0x7fe63014bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:32.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.151+0000 7fe62e59c700 1 -- 172.21.15.47:0/2418967547 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe628009470 con 0x7fe630060eb0 2022-06-29T15:47:32.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.151+0000 7fe62e59c700 1 --2- 172.21.15.47:0/2418967547 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe630060eb0 0x7fe63013cbb0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7fe618000f70 tx=0x7fe618007b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:32.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.151+0000 7fe6277fe700 1 -- 172.21.15.47:0/2418967547 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6180070d0 con 0x7fe630060eb0 2022-06-29T15:47:32.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.155+0000 7fe6277fe700 1 -- 172.21.15.47:0/2418967547 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe618007230 con 0x7fe630060eb0 2022-06-29T15:47:32.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.155+0000 7fe62f59e700 1 -- 172.21.15.47:0/2418967547 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe630139f60 con 0x7fe630060eb0 2022-06-29T15:47:32.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.155+0000 7fe6277fe700 1 -- 172.21.15.47:0/2418967547 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6180073a0 con 0x7fe630060eb0 2022-06-29T15:47:32.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.155+0000 7fe62f59e700 1 -- 172.21.15.47:0/2418967547 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6301385e0 con 0x7fe630060eb0 2022-06-29T15:47:32.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.155+0000 7fe62f59e700 1 -- 172.21.15.47:0/2418967547 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6300573d0 con 0x7fe630060eb0 2022-06-29T15:47:32.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.155+0000 7fe6277fe700 1 -- 172.21.15.47:0/2418967547 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe61801a060 con 0x7fe630060eb0 2022-06-29T15:47:32.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.155+0000 7fe6277fe700 1 --2- 172.21.15.47:0/2418967547 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe608053aa0 0x7fe608055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:32.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.155+0000 7fe6277fe700 1 -- 172.21.15.47:0/2418967547 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fe61806b010 con 0x7fe630060eb0 2022-06-29T15:47:32.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.155+0000 7fe627fff700 1 --2- 172.21.15.47:0/2418967547 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe608053aa0 0x7fe608055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:32.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.155+0000 7fe627fff700 1 --2- 172.21.15.47:0/2418967547 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe608053aa0 0x7fe608055f60 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7fe62000b000 tx=0x7fe620006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:32.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.167+0000 7fe6277fe700 1 -- 172.21.15.47:0/2418967547 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe618042110 con 0x7fe630060eb0 2022-06-29T15:47:32.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.339+0000 7fe62f59e700 1 -- 172.21.15.47:0/2418967547 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fe63013a0f0 con 0x7fe630060eb0 2022-06-29T15:47:32.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.339+0000 7fe6277fe700 1 -- 172.21.15.47:0/2418967547 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+4255 (secure 0 0 0) 0x7fe63013a0f0 con 0x7fe630060eb0 2022-06-29T15:47:32.341 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:47:32.341 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2022-06-29T15:47:32.341 INFO:teuthology.orchestra.run.smithi047.stdout: "auth_dump": [ 2022-06-29T15:47:32.342 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:32.342 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.a", 2022-06-29T15:47:32.342 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxiQJFFMxAA9Tv7FJD9u/BKvbkuKld6Kg==", 2022-06-29T15:47:32.343 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:47:32.343 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:47:32.343 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:47:32.343 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:47:32.343 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:47:32.344 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:32.344 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:32.344 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:32.344 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.b", 2022-06-29T15:47:32.345 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC2crxibt6FARAAEf2bwXKFhXKtqr1nN0rdXQ==", 2022-06-29T15:47:32.345 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:47:32.345 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:47:32.346 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:47:32.346 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:47:32.347 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:47:32.347 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:32.348 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:32.348 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:32.348 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.c", 2022-06-29T15:47:32.348 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxi/4ZMORAANpLbgTm3ehqi80USc3swGg==", 2022-06-29T15:47:32.349 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:47:32.349 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:47:32.349 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:47:32.349 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:47:32.350 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:47:32.350 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:32.350 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:32.351 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:32.351 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.0", 2022-06-29T15:47:32.351 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC2crxiY0YsMxAAZivb+CxL/w7vZwQv37Y4oQ==", 2022-06-29T15:47:32.351 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:47:32.352 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:47:32.352 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:47:32.352 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:47:32.353 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:32.353 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:32.353 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:32.353 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.1", 2022-06-29T15:47:32.354 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC5crxitLkKFxAA8+++LUUKLiO9SAuAfYgT5g==", 2022-06-29T15:47:32.354 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:47:32.354 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:47:32.355 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:47:32.355 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:47:32.355 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:32.355 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:32.356 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:32.356 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.2", 2022-06-29T15:47:32.356 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC7crxi1oM9NxAA2FfLdNt32YPwlyNdGQpaQA==", 2022-06-29T15:47:32.356 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:47:32.357 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:47:32.357 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:47:32.357 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:47:32.357 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:32.358 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:32.358 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:32.358 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.3", 2022-06-29T15:47:32.359 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC+crxiTsBgGhAA+gLBBDaskPbmRNWzgKwcHA==", 2022-06-29T15:47:32.359 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:47:32.359 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:47:32.359 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:47:32.360 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:47:32.360 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:32.360 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:32.360 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:32.361 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.4", 2022-06-29T15:47:32.361 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDDcrxivJebIhAAfwWJa7tK4hamkk1/1CSSfw==", 2022-06-29T15:47:32.361 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:47:32.361 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:47:32.362 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:47:32.362 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:47:32.362 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:32.362 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:32.363 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:32.363 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.5", 2022-06-29T15:47:32.363 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDGcrxijuozBhAAt+Dx3wH3WKp3DrbCbhmh8Q==", 2022-06-29T15:47:32.363 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:47:32.363 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:47:32.364 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:47:32.364 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:47:32.364 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:32.364 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:32.364 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:32.365 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.6", 2022-06-29T15:47:32.365 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDIcrxi91kXJxAA9JX7n98P394bD41Q9LUdvQ==", 2022-06-29T15:47:32.365 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:47:32.365 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:47:32.366 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:47:32.366 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:47:32.366 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:32.366 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:32.366 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:32.367 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.7", 2022-06-29T15:47:32.367 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDLcrxiHj0ZDRAASy6fAix8TmEZbAGtuFCfFg==", 2022-06-29T15:47:32.367 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:47:32.367 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:47:32.367 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:47:32.368 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:47:32.368 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:32.368 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:32.368 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:32.369 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.0", 2022-06-29T15:47:32.369 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC2crxiVDlfBxAAezkUfg4e2g0Rfakaa/Ek1Q==", 2022-06-29T15:47:32.369 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:47:32.369 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:47:32.370 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow r", 2022-06-29T15:47:32.370 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-29T15:47:32.370 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:32.370 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:32.371 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:32.371 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.1", 2022-06-29T15:47:32.371 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC2crxisXJ1CxAAhl07oF7AW66NdzfvvZTJmA==", 2022-06-29T15:47:32.371 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:47:32.372 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:47:32.372 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow r", 2022-06-29T15:47:32.372 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-29T15:47:32.372 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:32.372 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:32.373 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:32.373 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.admin", 2022-06-29T15:47:32.373 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxiW7HSFRAAhYqoGU9GFqj2HfUVGValSA==", 2022-06-29T15:47:32.373 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:47:32.374 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2022-06-29T15:47:32.374 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:47:32.374 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:47:32.374 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:47:32.374 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:32.375 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:32.375 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:32.375 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.x", 2022-06-29T15:47:32.375 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxidmuIMBAAcpNlYVu5+icCHg+yIjcKXw==", 2022-06-29T15:47:32.376 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:47:32.376 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2022-06-29T15:47:32.376 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2022-06-29T15:47:32.376 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:47:32.376 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:32.377 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:32.377 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:32.377 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.y", 2022-06-29T15:47:32.378 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxisSMFLBAAv03r4OoPjOP9nI9G/LK0iw==", 2022-06-29T15:47:32.378 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:47:32.378 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2022-06-29T15:47:32.378 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2022-06-29T15:47:32.379 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:47:32.379 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:32.379 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:32.379 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:47:32.379 INFO:teuthology.orchestra.run.smithi047.stdout:} 2022-06-29T15:47:32.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.339+0000 7fe62f59e700 1 -- 172.21.15.47:0/2418967547 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe608053aa0 msgr2=0x7fe608055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:32.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.339+0000 7fe62f59e700 1 --2- 172.21.15.47:0/2418967547 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe608053aa0 0x7fe608055f60 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7fe62000b000 tx=0x7fe620006df0).stop 2022-06-29T15:47:32.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.339+0000 7fe62f59e700 1 -- 172.21.15.47:0/2418967547 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe630060eb0 msgr2=0x7fe63013cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:32.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.339+0000 7fe62f59e700 1 --2- 172.21.15.47:0/2418967547 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe630060eb0 0x7fe63013cbb0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7fe618000f70 tx=0x7fe618007b50).stop 2022-06-29T15:47:32.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.339+0000 7fe62f59e700 1 -- 172.21.15.47:0/2418967547 shutdown_connections 2022-06-29T15:47:32.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.339+0000 7fe62f59e700 1 --2- 172.21.15.47:0/2418967547 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe608053aa0 0x7fe608055f60 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:32.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.339+0000 7fe62f59e700 1 --2- 172.21.15.47:0/2418967547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe630145090 0x7fe63014bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:32.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.339+0000 7fe62f59e700 1 --2- 172.21.15.47:0/2418967547 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe6300b5c60 0x7fe630140140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:32.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.339+0000 7fe62f59e700 1 --2- 172.21.15.47:0/2418967547 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe630060eb0 0x7fe63013cbb0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:32.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.339+0000 7fe62f59e700 1 -- 172.21.15.47:0/2418967547 >> 172.21.15.47:0/2418967547 conn(0x7fe630062560 msgr2=0x7fe630141260 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:32.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.339+0000 7fe62f59e700 1 -- 172.21.15.47:0/2418967547 shutdown_connections 2022-06-29T15:47:32.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.339+0000 7fe62f59e700 1 -- 172.21.15.47:0/2418967547 wait complete. 2022-06-29T15:47:32.385 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:47:32.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.491+0000 7f1ce5e3a700 1 Processor -- start 2022-06-29T15:47:32.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.495+0000 7f1ce5e3a700 1 -- start start 2022-06-29T15:47:32.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.495+0000 7f1ce5e3a700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1ce00b2120 0x7f1ce00b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:32.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.495+0000 7f1ce5e3a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1ce00b1260 0x7f1ce014b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:32.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.495+0000 7f1ce5e3a700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce00b0740 con 0x7f1ce005f0b0 2022-06-29T15:47:32.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.495+0000 7f1ce5e3a700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce0147460 con 0x7f1ce00b1260 2022-06-29T15:47:32.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.495+0000 7f1ce5e3a700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce00b0970 con 0x7f1ce00b2120 2022-06-29T15:47:32.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.495+0000 7f1cdffff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1ce00b1260 0x7f1ce014b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:32.513 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.495+0000 7f1cdffff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1ce00b1260 0x7f1ce014b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49170/0 (socket says 172.21.15.47:49170) 2022-06-29T15:47:32.513 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.495+0000 7f1cdffff700 1 -- 172.21.15.47:0/3245965526 learned_addr learned my addr 172.21.15.47:0/3245965526 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:32.513 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.495+0000 7f1cdf7fe700 1 --2- 172.21.15.47:0/3245965526 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1ce00b2120 0x7f1ce00b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:32.514 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.495+0000 7f1cdffff700 1 -- 172.21.15.47:0/3245965526 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1ce00b2120 msgr2=0x7f1ce00b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:32.514 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.495+0000 7f1cdffff700 1 --2- 172.21.15.47:0/3245965526 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1ce00b2120 0x7f1ce00b2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:32.514 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.495+0000 7f1cdffff700 1 -- 172.21.15.47:0/3245965526 >> v1:172.21.15.64:6789/0 conn(0x7f1ce005f0b0 legacy=0x7f1ce0147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:32.515 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.495+0000 7f1cdffff700 1 -- 172.21.15.47:0/3245965526 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce00b0c10 con 0x7f1ce00b1260 2022-06-29T15:47:32.515 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.495+0000 7f1cdf7fe700 1 --2- 172.21.15.47:0/3245965526 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1ce00b2120 0x7f1ce00b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:47:32.515 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.495+0000 7f1cdffff700 1 --2- 172.21.15.47:0/3245965526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ce00b1260 0x7f1ce014b520 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f1cd8002f70 tx=0x7f1cd8006eb0).ready entity=mon.0 client_cookie=a4f92daeb26a365a server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:32.516 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.495+0000 7f1cde7fc700 1 -- 172.21.15.47:0/3245965526 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1cd800d5c0 con 0x7f1ce00b1260 2022-06-29T15:47:32.516 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.495+0000 7f1cde7fc700 1 -- 172.21.15.47:0/3245965526 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cd8000e90 con 0x7f1ce00b1260 2022-06-29T15:47:32.516 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.495+0000 7f1cde7fc700 1 -- 172.21.15.47:0/3245965526 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1cd800d950 con 0x7f1ce00b1260 2022-06-29T15:47:32.516 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.495+0000 7f1ce5e3a700 1 -- 172.21.15.47:0/3245965526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ce00b1260 msgr2=0x7f1ce014b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:32.517 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.495+0000 7f1ce5e3a700 1 --2- 172.21.15.47:0/3245965526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ce00b1260 0x7f1ce014b520 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f1cd8002f70 tx=0x7f1cd8006eb0).stop 2022-06-29T15:47:32.517 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.495+0000 7f1ce5e3a700 1 -- 172.21.15.47:0/3245965526 shutdown_connections 2022-06-29T15:47:32.517 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.495+0000 7f1ce5e3a700 1 --2- 172.21.15.47:0/3245965526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ce00b1260 0x7f1ce014b520 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:32.518 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.495+0000 7f1ce5e3a700 1 --2- 172.21.15.47:0/3245965526 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1ce00b2120 0x7f1ce00b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:32.518 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.495+0000 7f1ce5e3a700 1 -- 172.21.15.47:0/3245965526 >> 172.21.15.47:0/3245965526 conn(0x7f1ce00626f0 msgr2=0x7f1ce0062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:32.518 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.495+0000 7f1ce5e3a700 1 -- 172.21.15.47:0/3245965526 shutdown_connections 2022-06-29T15:47:32.518 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.495+0000 7f1ce5e3a700 1 -- 172.21.15.47:0/3245965526 wait complete. 2022-06-29T15:47:32.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.499+0000 7f1ce5e3a700 1 Processor -- start 2022-06-29T15:47:32.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.499+0000 7f1ce5e3a700 1 -- start start 2022-06-29T15:47:32.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.499+0000 7f1ce5e3a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ce00b1260 0x7f1ce0141da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:32.520 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.499+0000 7f1ce5e3a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ce00b2120 0x7f1ce0143e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:32.520 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.499+0000 7f1ce5e3a700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1ce00adf00 0x7f1ce0140830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:32.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.499+0000 7f1ce5e3a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce014a700 con 0x7f1ce00b2120 2022-06-29T15:47:32.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.499+0000 7f1ce5e3a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce00af0c0 con 0x7f1ce00b1260 2022-06-29T15:47:32.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.499+0000 7f1ce5e3a700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce01437b0 con 0x7f1ce00adf00 2022-06-29T15:47:32.522 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.499+0000 7f1cdeffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ce00b2120 0x7f1ce0143e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:32.522 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.499+0000 7f1cdeffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ce00b2120 0x7f1ce0143e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49176/0 (socket says 172.21.15.47:49176) 2022-06-29T15:47:32.523 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.499+0000 7f1cdeffd700 1 -- 172.21.15.47:0/1708413356 learned_addr learned my addr 172.21.15.47:0/1708413356 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:32.524 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.499+0000 7f1cdf7fe700 1 --2- 172.21.15.47:0/1708413356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ce00b1260 0x7f1ce0141da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:32.524 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.499+0000 7f1cdffff700 1 --2- 172.21.15.47:0/1708413356 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1ce00adf00 0x7f1ce0140830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:32.525 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.499+0000 7f1cdeffd700 1 -- 172.21.15.47:0/1708413356 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1ce00adf00 msgr2=0x7f1ce0140830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:32.525 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.499+0000 7f1cdeffd700 1 --2- 172.21.15.47:0/1708413356 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1ce00adf00 0x7f1ce0140830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:32.526 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.499+0000 7f1cdeffd700 1 -- 172.21.15.47:0/1708413356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ce00b1260 msgr2=0x7f1ce0141da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:32.526 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.499+0000 7f1cdeffd700 1 --2- 172.21.15.47:0/1708413356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ce00b1260 0x7f1ce0141da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:32.527 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.499+0000 7f1cdeffd700 1 -- 172.21.15.47:0/1708413356 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd800b1e0 con 0x7f1ce00b2120 2022-06-29T15:47:32.527 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.499+0000 7f1cdeffd700 1 --2- 172.21.15.47:0/1708413356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ce00b2120 0x7f1ce0143e80 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7f1ce0143700 tx=0x7f1cd4007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:32.527 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.499+0000 7f1cdcff9700 1 -- 172.21.15.47:0/1708413356 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1cd4012400 con 0x7f1ce00b2120 2022-06-29T15:47:32.528 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.499+0000 7f1cdcff9700 1 -- 172.21.15.47:0/1708413356 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cd4012560 con 0x7f1ce00b2120 2022-06-29T15:47:32.528 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.499+0000 7f1ce5e3a700 1 -- 172.21.15.47:0/1708413356 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ce00af2b0 con 0x7f1ce00b2120 2022-06-29T15:47:32.528 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.499+0000 7f1cdcff9700 1 -- 172.21.15.47:0/1708413356 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1cd40128c0 con 0x7f1ce00b2120 2022-06-29T15:47:32.529 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.499+0000 7f1ce5e3a700 1 -- 172.21.15.47:0/1708413356 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ce01444b0 con 0x7f1ce00b2120 2022-06-29T15:47:32.529 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.503+0000 7f1ce5e3a700 1 -- 172.21.15.47:0/1708413356 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ce00573d0 con 0x7f1ce00b2120 2022-06-29T15:47:32.529 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.503+0000 7f1cdcff9700 1 -- 172.21.15.47:0/1708413356 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1cd4019030 con 0x7f1ce00b2120 2022-06-29T15:47:32.529 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.503+0000 7f1cdcff9700 1 --2- 172.21.15.47:0/1708413356 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1cc0053a70 0x7f1cc0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:32.530 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.503+0000 7f1cdcff9700 1 -- 172.21.15.47:0/1708413356 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f1cd406b020 con 0x7f1ce00b2120 2022-06-29T15:47:32.530 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.503+0000 7f1cdf7fe700 1 --2- 172.21.15.47:0/1708413356 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1cc0053a70 0x7f1cc0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:32.530 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.503+0000 7f1cdf7fe700 1 --2- 172.21.15.47:0/1708413356 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1cc0053a70 0x7f1cc0055f30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7f1cd000c000 tx=0x7f1cd0008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:32.531 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.507+0000 7f1cdcff9700 1 -- 172.21.15.47:0/1708413356 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1cd4042120 con 0x7f1ce00b2120 2022-06-29T15:47:32.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.639+0000 7f1ce5e3a700 1 -- 172.21.15.47:0/1708413356 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1ce00a2c20 con 0x7f1ce00b2120 2022-06-29T15:47:32.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.639+0000 7f1cdcff9700 1 -- 172.21.15.47:0/1708413356 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+13415 (secure 0 0 0) 0x7f1cd4045740 con 0x7f1ce00b2120 2022-06-29T15:47:32.641 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:47:32.642 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":64,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:47:32.010127+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:47:32.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.639+0000 7f1ce5e3a700 1 -- 172.21.15.47:0/1708413356 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1cc0053a70 msgr2=0x7f1cc0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:32.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.639+0000 7f1ce5e3a700 1 --2- 172.21.15.47:0/1708413356 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1cc0053a70 0x7f1cc0055f30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7f1cd000c000 tx=0x7f1cd0008040).stop 2022-06-29T15:47:32.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.639+0000 7f1ce5e3a700 1 -- 172.21.15.47:0/1708413356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ce00b2120 msgr2=0x7f1ce0143e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:32.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.639+0000 7f1ce5e3a700 1 --2- 172.21.15.47:0/1708413356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ce00b2120 0x7f1ce0143e80 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7f1ce0143700 tx=0x7f1cd4007c50).stop 2022-06-29T15:47:32.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.643+0000 7f1ce5e3a700 1 -- 172.21.15.47:0/1708413356 shutdown_connections 2022-06-29T15:47:32.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.643+0000 7f1ce5e3a700 1 --2- 172.21.15.47:0/1708413356 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1ce00adf00 0x7f1ce0140830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:32.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.643+0000 7f1ce5e3a700 1 --2- 172.21.15.47:0/1708413356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ce00b2120 0x7f1ce0143e80 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:32.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.643+0000 7f1ce5e3a700 1 --2- 172.21.15.47:0/1708413356 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1cc0053a70 0x7f1cc0055f30 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:32.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.643+0000 7f1ce5e3a700 1 --2- 172.21.15.47:0/1708413356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ce00b1260 0x7f1ce0141da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:32.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.643+0000 7f1ce5e3a700 1 -- 172.21.15.47:0/1708413356 >> 172.21.15.47:0/1708413356 conn(0x7f1ce00626f0 msgr2=0x7f1ce00b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:32.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.643+0000 7f1ce5e3a700 1 -- 172.21.15.47:0/1708413356 shutdown_connections 2022-06-29T15:47:32.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.643+0000 7f1ce5e3a700 1 -- 172.21.15.47:0/1708413356 wait complete. 2022-06-29T15:47:32.659 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-29T15:42:16.971431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-29T15:42:25.290884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-06-29T15:47:32.660 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-29T15:47:32.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.767+0000 7f0acb59e700 1 Processor -- start 2022-06-29T15:47:32.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.767+0000 7f0acb59e700 1 -- start start 2022-06-29T15:47:32.772 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.771+0000 7f0acb59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0acc1409b0 0x7f0acc061620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:32.772 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.771+0000 7f0acb59e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0acc0af660 con 0x7f0acc05fd00 2022-06-29T15:47:32.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.771+0000 7f0acb59e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0acc1437e0 con 0x7f0acc060a40 2022-06-29T15:47:32.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.771+0000 7f0acb59e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0acc143fc0 con 0x7f0acc1409b0 2022-06-29T15:47:32.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.771+0000 7f0ac9d9b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0acc05fd00 0x7f0acc1402a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34736/0 (socket says 172.21.15.47:34736) 2022-06-29T15:47:32.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.771+0000 7f0ac9d9b700 1 -- 172.21.15.47:0/1154765427 learned_addr learned my addr 172.21.15.47:0/1154765427 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:47:32.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.771+0000 7f0acad9d700 1 --2- 172.21.15.47:0/1154765427 >> v2:172.21.15.64:3300/0 conn(0x7f0acc1409b0 0x7f0acc061620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:32.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.771+0000 7f0ac959a700 1 -- 172.21.15.47:0/1154765427 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4069027245 0 0) 0x7f0acc0af660 con 0x7f0acc05fd00 2022-06-29T15:47:32.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.771+0000 7f0ac959a700 1 -- 172.21.15.47:0/1154765427 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0aac003260 con 0x7f0acc05fd00 2022-06-29T15:47:32.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.771+0000 7f0acad9d700 1 -- 172.21.15.47:0/1154765427 >> v1:172.21.15.64:6789/0 conn(0x7f0acc060a40 legacy=0x7f0acc142ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:32.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.771+0000 7f0acad9d700 1 -- 172.21.15.47:0/1154765427 >> v1:172.21.15.47:6789/0 conn(0x7f0acc05fd00 legacy=0x7f0acc1402a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:32.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.771+0000 7f0acad9d700 1 -- 172.21.15.47:0/1154765427 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0acc143a80 con 0x7f0acc1409b0 2022-06-29T15:47:32.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.771+0000 7f0acad9d700 1 --2- 172.21.15.47:0/1154765427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0acc1409b0 0x7f0acc061620 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f0ac40032f0 tx=0x7f0ac40091d0).ready entity=mon.1 client_cookie=6095b31c05eec569 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:32.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.771+0000 7f0ac959a700 1 -- 172.21.15.47:0/1154765427 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ac400b6c0 con 0x7f0acc1409b0 2022-06-29T15:47:32.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.771+0000 7f0ac959a700 1 -- 172.21.15.47:0/1154765427 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ac400b820 con 0x7f0acc1409b0 2022-06-29T15:47:32.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.771+0000 7f0ac959a700 1 -- 172.21.15.47:0/1154765427 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ac400b990 con 0x7f0acc1409b0 2022-06-29T15:47:32.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.771+0000 7f0acb59e700 1 -- 172.21.15.47:0/1154765427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0acc1409b0 msgr2=0x7f0acc061620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:32.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.771+0000 7f0acb59e700 1 --2- 172.21.15.47:0/1154765427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0acc1409b0 0x7f0acc061620 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f0ac40032f0 tx=0x7f0ac40091d0).stop 2022-06-29T15:47:32.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.771+0000 7f0acb59e700 1 -- 172.21.15.47:0/1154765427 shutdown_connections 2022-06-29T15:47:32.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.771+0000 7f0acb59e700 1 --2- 172.21.15.47:0/1154765427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0acc1409b0 0x7f0acc061620 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:32.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.771+0000 7f0acb59e700 1 -- 172.21.15.47:0/1154765427 >> 172.21.15.47:0/1154765427 conn(0x7f0acc0b6430 msgr2=0x7f0acc0b6810 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:32.781 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.771+0000 7f0acb59e700 1 -- 172.21.15.47:0/1154765427 shutdown_connections 2022-06-29T15:47:32.781 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.775+0000 7f0acb59e700 1 -- 172.21.15.47:0/1154765427 wait complete. 2022-06-29T15:47:32.781 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.775+0000 7f0acb59e700 1 Processor -- start 2022-06-29T15:47:32.781 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.775+0000 7f0acb59e700 1 -- start start 2022-06-29T15:47:32.785 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.775+0000 7f0acb59e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0acc05fd00 0x7f0acc141940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:32.786 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.775+0000 7f0acb59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0acc060a40 0x7f0acc1524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:32.786 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.775+0000 7f0acb59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0acc1409b0 0x7f0acc1529c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:32.787 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.775+0000 7f0acb59e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0acc1520a0 con 0x7f0acc1409b0 2022-06-29T15:47:32.787 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.775+0000 7f0acb59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0acc144270 con 0x7f0acc060a40 2022-06-29T15:47:32.787 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.775+0000 7f0acb59e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0acc143230 con 0x7f0acc05fd00 2022-06-29T15:47:32.788 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.775+0000 7f0acad9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0acc1409b0 0x7f0acc1529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:32.788 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.775+0000 7f0acad9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0acc1409b0 0x7f0acc1529c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49192/0 (socket says 172.21.15.47:49192) 2022-06-29T15:47:32.788 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.775+0000 7f0acad9d700 1 -- 172.21.15.47:0/560931662 learned_addr learned my addr 172.21.15.47:0/560931662 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:32.788 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.775+0000 7f0ac9d9b700 1 --2- 172.21.15.47:0/560931662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0acc060a40 0x7f0acc1524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:32.789 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.775+0000 7f0aca59c700 1 --2- 172.21.15.47:0/560931662 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0acc05fd00 0x7f0acc141940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:32.789 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.779+0000 7f0ac9d9b700 1 -- 172.21.15.47:0/560931662 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0acc05fd00 msgr2=0x7f0acc141940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:32.789 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.779+0000 7f0ac9d9b700 1 --2- 172.21.15.47:0/560931662 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0acc05fd00 0x7f0acc141940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:32.789 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.779+0000 7f0ac9d9b700 1 -- 172.21.15.47:0/560931662 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0acc1409b0 msgr2=0x7f0acc1529c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:32.789 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.779+0000 7f0ac9d9b700 1 --2- 172.21.15.47:0/560931662 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0acc1409b0 0x7f0acc1529c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:32.790 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.779+0000 7f0ac9d9b700 1 -- 172.21.15.47:0/560931662 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac4009470 con 0x7f0acc060a40 2022-06-29T15:47:32.790 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.779+0000 7f0acad9d700 1 --2- 172.21.15.47:0/560931662 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0acc1409b0 0x7f0acc1529c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:47:32.790 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.779+0000 7f0ac9d9b700 1 --2- 172.21.15.47:0/560931662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0acc060a40 0x7f0acc1524d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f0ab4001110 tx=0x7f0ab4007a20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:32.790 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.779+0000 7f0ac37fe700 1 -- 172.21.15.47:0/560931662 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ab400ce80 con 0x7f0acc060a40 2022-06-29T15:47:32.791 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.779+0000 7f0ac37fe700 1 -- 172.21.15.47:0/560931662 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab40070d0 con 0x7f0acc060a40 2022-06-29T15:47:32.791 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.779+0000 7f0acb59e700 1 -- 172.21.15.47:0/560931662 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0acc1425e0 con 0x7f0acc060a40 2022-06-29T15:47:32.791 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.779+0000 7f0ac37fe700 1 -- 172.21.15.47:0/560931662 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ab4007430 con 0x7f0acc060a40 2022-06-29T15:47:32.791 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.779+0000 7f0acb59e700 1 -- 172.21.15.47:0/560931662 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0acc0aa240 con 0x7f0acc060a40 2022-06-29T15:47:32.792 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.779+0000 7f0acb59e700 1 -- 172.21.15.47:0/560931662 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0acc152210 con 0x7f0acc060a40 2022-06-29T15:47:32.792 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.779+0000 7f0ac37fe700 1 -- 172.21.15.47:0/560931662 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0ab401a060 con 0x7f0acc060a40 2022-06-29T15:47:32.792 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.779+0000 7f0ac37fe700 1 --2- 172.21.15.47:0/560931662 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0ab0053950 0x7f0ab0055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:32.792 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.779+0000 7f0ac37fe700 1 -- 172.21.15.47:0/560931662 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f0ab406b360 con 0x7f0acc060a40 2022-06-29T15:47:32.792 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.779+0000 7f0aca59c700 1 --2- 172.21.15.47:0/560931662 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0ab0053950 0x7f0ab0055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:32.793 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.783+0000 7f0aca59c700 1 --2- 172.21.15.47:0/560931662 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0ab0053950 0x7f0ab0055e10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f0abc00d000 tx=0x7f0abc009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:32.793 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.791+0000 7f0ac37fe700 1 -- 172.21.15.47:0/560931662 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0acc152210 con 0x7f0acc060a40 2022-06-29T15:47:32.921 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.919+0000 7f0acb59e700 1 -- 172.21.15.47:0/560931662 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f0acc152210 con 0x7f0acc060a40 2022-06-29T15:47:32.922 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.919+0000 7f0ac37fe700 1 -- 172.21.15.47:0/560931662 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v64) v1 ==== 110+0+10 (secure 0 0 0) 0x7f0acc152210 con 0x7f0acc060a40 2022-06-29T15:47:32.922 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2022-06-29T15:47:32.925 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.923+0000 7f0acb59e700 1 -- 172.21.15.47:0/560931662 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0ab0053950 msgr2=0x7f0ab0055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:32.926 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.923+0000 7f0acb59e700 1 --2- 172.21.15.47:0/560931662 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0ab0053950 0x7f0ab0055e10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f0abc00d000 tx=0x7f0abc009040).stop 2022-06-29T15:47:32.926 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.923+0000 7f0acb59e700 1 -- 172.21.15.47:0/560931662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0acc060a40 msgr2=0x7f0acc1524d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:32.926 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.923+0000 7f0acb59e700 1 --2- 172.21.15.47:0/560931662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0acc060a40 0x7f0acc1524d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f0ab4001110 tx=0x7f0ab4007a20).stop 2022-06-29T15:47:32.926 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.923+0000 7f0acb59e700 1 -- 172.21.15.47:0/560931662 shutdown_connections 2022-06-29T15:47:32.927 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.923+0000 7f0acb59e700 1 --2- 172.21.15.47:0/560931662 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0acc1409b0 0x7f0acc1529c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:32.927 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.923+0000 7f0acb59e700 1 --2- 172.21.15.47:0/560931662 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0ab0053950 0x7f0ab0055e10 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:32.927 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.923+0000 7f0acb59e700 1 --2- 172.21.15.47:0/560931662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0acc060a40 0x7f0acc1524d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:32.927 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.923+0000 7f0acb59e700 1 --2- 172.21.15.47:0/560931662 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0acc05fd00 0x7f0acc141940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:32.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.923+0000 7f0acb59e700 1 -- 172.21.15.47:0/560931662 >> 172.21.15.47:0/560931662 conn(0x7f0acc0b6430 msgr2=0x7f0acc0b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:32.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.923+0000 7f0acb59e700 1 -- 172.21.15.47:0/560931662 shutdown_connections 2022-06-29T15:47:32.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:32.923+0000 7f0acb59e700 1 -- 172.21.15.47:0/560931662 wait complete. 2022-06-29T15:47:32.940 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-29T15:47:33.058 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.055+0000 7fe8051a5700 1 Processor -- start 2022-06-29T15:47:33.058 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.055+0000 7fe8051a5700 1 -- start start 2022-06-29T15:47:33.059 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.055+0000 7fe8051a5700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe8000b58d0 0x7fe8000b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:33.060 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.055+0000 7fe8051a5700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe80014b4c0 0x7fe8000aa130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:33.060 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.055+0000 7fe8051a5700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe800060790 con 0x7fe8000b1610 2022-06-29T15:47:33.060 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.055+0000 7fe8051a5700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8000a99e0 con 0x7fe80014b4c0 2022-06-29T15:47:33.060 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.055+0000 7fe8051a5700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe80014ad80 con 0x7fe8000b58d0 2022-06-29T15:47:33.061 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.059+0000 7fe8049a4700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe80014b4c0 0x7fe8000aa130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:33.061 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.059+0000 7fe7fffff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe8000b58d0 0x7fe8000b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:33.061 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.059+0000 7fe8049a4700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe80014b4c0 0x7fe8000aa130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:54902/0 (socket says 172.21.15.47:54902) 2022-06-29T15:47:33.061 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.059+0000 7fe8049a4700 1 -- 172.21.15.47:0/2337217655 learned_addr learned my addr 172.21.15.47:0/2337217655 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:33.062 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.059+0000 7fe7ff7fe700 1 --1- 172.21.15.47:0/2337217655 >> v1:172.21.15.64:6789/0 conn(0x7fe8000b1610 0x7fe8001475f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:42314/0 (socket says 172.21.15.47:42314) 2022-06-29T15:47:33.062 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.059+0000 7fe8049a4700 1 -- 172.21.15.47:0/2337217655 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe8000b58d0 msgr2=0x7fe8000b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:33.062 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.059+0000 7fe8049a4700 1 --2- 172.21.15.47:0/2337217655 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe8000b58d0 0x7fe8000b5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:33.062 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.059+0000 7fe8049a4700 1 -- 172.21.15.47:0/2337217655 >> v1:172.21.15.64:6789/0 conn(0x7fe8000b1610 legacy=0x7fe8001475f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:33.062 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.059+0000 7fe8049a4700 1 -- 172.21.15.47:0/2337217655 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe80014b020 con 0x7fe80014b4c0 2022-06-29T15:47:33.064 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.059+0000 7fe8049a4700 1 --2- 172.21.15.47:0/2337217655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe80014b4c0 0x7fe8000aa130 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7fe7f80032f0 tx=0x7fe7f8009290).ready entity=mon.1 client_cookie=991cfd2297ec3c12 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:33.064 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.059+0000 7fe7feffd700 1 -- 172.21.15.47:0/2337217655 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe7f800b8a0 con 0x7fe80014b4c0 2022-06-29T15:47:33.065 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.059+0000 7fe7feffd700 1 -- 172.21.15.47:0/2337217655 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7f800ba00 con 0x7fe80014b4c0 2022-06-29T15:47:33.066 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.059+0000 7fe7feffd700 1 -- 172.21.15.47:0/2337217655 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe7f800bb70 con 0x7fe80014b4c0 2022-06-29T15:47:33.066 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.059+0000 7fe8051a5700 1 -- 172.21.15.47:0/2337217655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe80014b4c0 msgr2=0x7fe8000aa130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:33.066 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.059+0000 7fe8051a5700 1 --2- 172.21.15.47:0/2337217655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe80014b4c0 0x7fe8000aa130 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7fe7f80032f0 tx=0x7fe7f8009290).stop 2022-06-29T15:47:33.067 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.059+0000 7fe8051a5700 1 -- 172.21.15.47:0/2337217655 shutdown_connections 2022-06-29T15:47:33.067 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.059+0000 7fe8051a5700 1 --2- 172.21.15.47:0/2337217655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe80014b4c0 0x7fe8000aa130 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:33.067 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.059+0000 7fe8051a5700 1 --2- 172.21.15.47:0/2337217655 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe8000b58d0 0x7fe8000b5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:33.067 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.059+0000 7fe8051a5700 1 -- 172.21.15.47:0/2337217655 >> 172.21.15.47:0/2337217655 conn(0x7fe800062560 msgr2=0x7fe800062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:33.068 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.059+0000 7fe8051a5700 1 -- 172.21.15.47:0/2337217655 shutdown_connections 2022-06-29T15:47:33.068 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.059+0000 7fe8051a5700 1 -- 172.21.15.47:0/2337217655 wait complete. 2022-06-29T15:47:33.068 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.063+0000 7fe8051a5700 1 Processor -- start 2022-06-29T15:47:33.068 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.063+0000 7fe8051a5700 1 -- start start 2022-06-29T15:47:33.071 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.063+0000 7fe8051a5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8000b1610 0x7fe80015a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:33.071 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.063+0000 7fe8051a5700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe8000b58d0 0x7fe8000b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:33.071 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.063+0000 7fe8051a5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe80014b4c0 0x7fe8001574d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:33.072 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.063+0000 7fe8051a5700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe80015b0b0 con 0x7fe8000b1610 2022-06-29T15:47:33.072 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.063+0000 7fe8051a5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe800156ba0 con 0x7fe80014b4c0 2022-06-29T15:47:33.072 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.063+0000 7fe8051a5700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe80015a640 con 0x7fe8000b58d0 2022-06-29T15:47:33.072 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.063+0000 7fe7fffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8000b1610 0x7fe80015a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:33.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.063+0000 7fe7fffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8000b1610 0x7fe80015a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49202/0 (socket says 172.21.15.47:49202) 2022-06-29T15:47:33.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.063+0000 7fe7fffff700 1 -- 172.21.15.47:0/4154385445 learned_addr learned my addr 172.21.15.47:0/4154385445 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:33.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.063+0000 7fe8049a4700 1 --2- 172.21.15.47:0/4154385445 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe80014b4c0 0x7fe8001574d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:33.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.063+0000 7fe7ff7fe700 1 --2- 172.21.15.47:0/4154385445 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe8000b58d0 0x7fe8000b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:33.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.063+0000 7fe7fffff700 1 -- 172.21.15.47:0/4154385445 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe8000b58d0 msgr2=0x7fe8000b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:33.074 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.063+0000 7fe7fffff700 1 --2- 172.21.15.47:0/4154385445 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe8000b58d0 0x7fe8000b45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:33.074 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.063+0000 7fe7fffff700 1 -- 172.21.15.47:0/4154385445 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe80014b4c0 msgr2=0x7fe8001574d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:33.074 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.063+0000 7fe7fffff700 1 --2- 172.21.15.47:0/4154385445 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe80014b4c0 0x7fe8001574d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:33.074 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.063+0000 7fe7fffff700 1 -- 172.21.15.47:0/4154385445 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f8009530 con 0x7fe8000b1610 2022-06-29T15:47:33.075 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.063+0000 7fe7fffff700 1 --2- 172.21.15.47:0/4154385445 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8000b1610 0x7fe80015a150 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7fe7f400c000 tx=0x7fe7f4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:33.075 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.063+0000 7fe7fd7fa700 1 -- 172.21.15.47:0/4154385445 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe7f4010780 con 0x7fe8000b1610 2022-06-29T15:47:33.075 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.063+0000 7fe7fd7fa700 1 -- 172.21.15.47:0/4154385445 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7f40108e0 con 0x7fe8000b1610 2022-06-29T15:47:33.076 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.063+0000 7fe8051a5700 1 -- 172.21.15.47:0/4154385445 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8000accd0 con 0x7fe8000b1610 2022-06-29T15:47:33.077 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.063+0000 7fe7fd7fa700 1 -- 172.21.15.47:0/4154385445 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe7f4010c40 con 0x7fe8000b1610 2022-06-29T15:47:33.077 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.063+0000 7fe8051a5700 1 -- 172.21.15.47:0/4154385445 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8000b4ad0 con 0x7fe8000b1610 2022-06-29T15:47:33.077 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.067+0000 7fe7fd7fa700 1 -- 172.21.15.47:0/4154385445 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe7f401c460 con 0x7fe8000b1610 2022-06-29T15:47:33.077 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.067+0000 7fe8051a5700 1 -- 172.21.15.47:0/4154385445 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8000573d0 con 0x7fe8000b1610 2022-06-29T15:47:33.078 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.067+0000 7fe7fd7fa700 1 --2- 172.21.15.47:0/4154385445 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe7e00539e0 0x7fe7e0055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:33.078 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.067+0000 7fe7fd7fa700 1 -- 172.21.15.47:0/4154385445 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fe7f406c150 con 0x7fe8000b1610 2022-06-29T15:47:33.078 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.067+0000 7fe7ff7fe700 1 --2- 172.21.15.47:0/4154385445 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe7e00539e0 0x7fe7e0055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:33.078 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.067+0000 7fe7ff7fe700 1 --2- 172.21.15.47:0/4154385445 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe7e00539e0 0x7fe7e0055ea0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7fe7f000d000 tx=0x7fe7f0007b80).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:33.079 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.071+0000 7fe7fd7fa700 1 -- 172.21.15.47:0/4154385445 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe7f4043170 con 0x7fe8000b1610 2022-06-29T15:47:33.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.199+0000 7fe8051a5700 1 -- 172.21.15.47:0/4154385445 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe800069e20 con 0x7fe8000b1610 2022-06-29T15:47:33.202 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.199+0000 7fe7fd7fa700 1 -- 172.21.15.47:0/4154385445 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v64) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe7f4046790 con 0x7fe8000b1610 2022-06-29T15:47:33.202 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:47:33.204 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.203+0000 7fe8051a5700 1 -- 172.21.15.47:0/4154385445 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe7e00539e0 msgr2=0x7fe7e0055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:33.205 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.203+0000 7fe8051a5700 1 --2- 172.21.15.47:0/4154385445 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe7e00539e0 0x7fe7e0055ea0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7fe7f000d000 tx=0x7fe7f0007b80).stop 2022-06-29T15:47:33.205 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.203+0000 7fe8051a5700 1 -- 172.21.15.47:0/4154385445 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8000b1610 msgr2=0x7fe80015a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:33.205 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.203+0000 7fe8051a5700 1 --2- 172.21.15.47:0/4154385445 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8000b1610 0x7fe80015a150 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7fe7f400c000 tx=0x7fe7f4006df0).stop 2022-06-29T15:47:33.205 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.203+0000 7fe8051a5700 1 -- 172.21.15.47:0/4154385445 shutdown_connections 2022-06-29T15:47:33.206 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.203+0000 7fe8051a5700 1 --2- 172.21.15.47:0/4154385445 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe7e00539e0 0x7fe7e0055ea0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:33.206 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.203+0000 7fe8051a5700 1 --2- 172.21.15.47:0/4154385445 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe80014b4c0 0x7fe8001574d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:33.206 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.203+0000 7fe8051a5700 1 --2- 172.21.15.47:0/4154385445 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe8000b58d0 0x7fe8000b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:33.206 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.203+0000 7fe8051a5700 1 --2- 172.21.15.47:0/4154385445 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8000b1610 0x7fe80015a150 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:33.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.203+0000 7fe8051a5700 1 -- 172.21.15.47:0/4154385445 >> 172.21.15.47:0/4154385445 conn(0x7fe800062560 msgr2=0x7fe80014a440 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:33.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.203+0000 7fe8051a5700 1 -- 172.21.15.47:0/4154385445 shutdown_connections 2022-06-29T15:47:33.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.203+0000 7fe8051a5700 1 -- 172.21.15.47:0/4154385445 wait complete. 2022-06-29T15:47:33.216 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-06-29T15:47:33.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.315+0000 7f400e203700 1 Processor -- start 2022-06-29T15:47:33.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.315+0000 7f400e203700 1 -- start start 2022-06-29T15:47:33.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.315+0000 7f400e203700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f40080b5c60 0x7f4008144980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:33.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.315+0000 7f400e203700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f40080a72f0 0x7f40080b2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:33.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.315+0000 7f400e203700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40080a9120 con 0x7f4008060eb0 2022-06-29T15:47:33.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.315+0000 7f400e203700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40080b0b80 con 0x7f40080a72f0 2022-06-29T15:47:33.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.315+0000 7f400e203700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40080a89e0 con 0x7f40080b5c60 2022-06-29T15:47:33.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.315+0000 7f400da02700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f40080a72f0 0x7f40080b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:33.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.319+0000 7f400da02700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f40080a72f0 0x7f40080b2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49210/0 (socket says 172.21.15.47:49210) 2022-06-29T15:47:33.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.319+0000 7f400da02700 1 -- 172.21.15.47:0/246989579 learned_addr learned my addr 172.21.15.47:0/246989579 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:33.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.319+0000 7f400ca00700 1 --1- 172.21.15.47:0/246989579 >> v1:172.21.15.47:6789/0 conn(0x7f4008060eb0 0x7f40080b1e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34762/0 (socket says 172.21.15.47:34762) 2022-06-29T15:47:33.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.319+0000 7f400d201700 1 --2- 172.21.15.47:0/246989579 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f40080b5c60 0x7f4008144980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:33.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.319+0000 7f400da02700 1 -- 172.21.15.47:0/246989579 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f40080b5c60 msgr2=0x7f4008144980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:33.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.319+0000 7f400da02700 1 --2- 172.21.15.47:0/246989579 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f40080b5c60 0x7f4008144980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:33.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.319+0000 7f400da02700 1 -- 172.21.15.47:0/246989579 >> v1:172.21.15.47:6789/0 conn(0x7f4008060eb0 legacy=0x7f40080b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:33.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.319+0000 7f4007fff700 1 -- 172.21.15.47:0/246989579 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1051603745 0 0) 0x7f40080a9120 con 0x7f4008060eb0 2022-06-29T15:47:33.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.319+0000 7f400da02700 1 -- 172.21.15.47:0/246989579 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40080a8c80 con 0x7f40080a72f0 2022-06-29T15:47:33.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.319+0000 7f400da02700 1 --2- 172.21.15.47:0/246989579 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40080a72f0 0x7f40080b2540 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7f4000002f70 tx=0x7f4000006eb0).ready entity=mon.0 client_cookie=88713ab6e1c8f566 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:33.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.319+0000 7f4007fff700 1 -- 172.21.15.47:0/246989579 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f400000d5c0 con 0x7f40080a72f0 2022-06-29T15:47:33.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.319+0000 7f4007fff700 1 -- 172.21.15.47:0/246989579 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4000000e90 con 0x7f40080a72f0 2022-06-29T15:47:33.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.319+0000 7f4007fff700 1 -- 172.21.15.47:0/246989579 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f400000d910 con 0x7f40080a72f0 2022-06-29T15:47:33.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.319+0000 7f400e203700 1 -- 172.21.15.47:0/246989579 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40080a72f0 msgr2=0x7f40080b2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:33.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.319+0000 7f400e203700 1 --2- 172.21.15.47:0/246989579 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40080a72f0 0x7f40080b2540 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7f4000002f70 tx=0x7f4000006eb0).stop 2022-06-29T15:47:33.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.319+0000 7f400e203700 1 -- 172.21.15.47:0/246989579 shutdown_connections 2022-06-29T15:47:33.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.319+0000 7f400e203700 1 --2- 172.21.15.47:0/246989579 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40080a72f0 0x7f40080b2540 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:33.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.319+0000 7f400e203700 1 --2- 172.21.15.47:0/246989579 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f40080b5c60 0x7f4008144980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:33.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.319+0000 7f400e203700 1 -- 172.21.15.47:0/246989579 >> 172.21.15.47:0/246989579 conn(0x7f4008062560 msgr2=0x7f4008062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:33.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.319+0000 7f400e203700 1 -- 172.21.15.47:0/246989579 shutdown_connections 2022-06-29T15:47:33.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.319+0000 7f400e203700 1 -- 172.21.15.47:0/246989579 wait complete. 2022-06-29T15:47:33.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.323+0000 7f400e203700 1 Processor -- start 2022-06-29T15:47:33.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.323+0000 7f400e203700 1 -- start start 2022-06-29T15:47:33.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.323+0000 7f400e203700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4008060eb0 0x7f400814e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:33.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.323+0000 7f400e203700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f40080a72f0 0x7f400814af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:33.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.323+0000 7f400e203700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40080b5c60 0x7f400814b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:33.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.323+0000 7f400e203700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400814b970 con 0x7f40080b5c60 2022-06-29T15:47:33.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.323+0000 7f400e203700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4008156a90 con 0x7f4008060eb0 2022-06-29T15:47:33.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.323+0000 7f400e203700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4008156d30 con 0x7f40080a72f0 2022-06-29T15:47:33.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.323+0000 7f400da02700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40080b5c60 0x7f400814b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:33.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.323+0000 7f400da02700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40080b5c60 0x7f400814b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49218/0 (socket says 172.21.15.47:49218) 2022-06-29T15:47:33.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.323+0000 7f400da02700 1 -- 172.21.15.47:0/2426384560 learned_addr learned my addr 172.21.15.47:0/2426384560 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:33.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.323+0000 7f400ca00700 1 --2- 172.21.15.47:0/2426384560 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f40080a72f0 0x7f400814af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:33.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.323+0000 7f400d201700 1 --2- 172.21.15.47:0/2426384560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4008060eb0 0x7f400814e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:33.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.323+0000 7f400da02700 1 -- 172.21.15.47:0/2426384560 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f40080a72f0 msgr2=0x7f400814af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:33.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.323+0000 7f400da02700 1 --2- 172.21.15.47:0/2426384560 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f40080a72f0 0x7f400814af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:33.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.323+0000 7f400da02700 1 -- 172.21.15.47:0/2426384560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4008060eb0 msgr2=0x7f400814e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:33.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.323+0000 7f400da02700 1 --2- 172.21.15.47:0/2426384560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4008060eb0 0x7f400814e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:33.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.323+0000 7f400da02700 1 -- 172.21.15.47:0/2426384560 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f400000b1e0 con 0x7f40080b5c60 2022-06-29T15:47:33.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.323+0000 7f400da02700 1 --2- 172.21.15.47:0/2426384560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40080b5c60 0x7f400814b480 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7f4000002f70 tx=0x7f4000015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:33.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.323+0000 7f40067fc700 1 -- 172.21.15.47:0/2426384560 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4000006b60 con 0x7f40080b5c60 2022-06-29T15:47:33.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.323+0000 7f40067fc700 1 -- 172.21.15.47:0/2426384560 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f400000d940 con 0x7f40080b5c60 2022-06-29T15:47:33.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.323+0000 7f40067fc700 1 -- 172.21.15.47:0/2426384560 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4000011d30 con 0x7f40080b5c60 2022-06-29T15:47:33.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.327+0000 7f400e203700 1 -- 172.21.15.47:0/2426384560 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4008156fd0 con 0x7f40080b5c60 2022-06-29T15:47:33.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.327+0000 7f400e203700 1 -- 172.21.15.47:0/2426384560 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4008149ad0 con 0x7f40080b5c60 2022-06-29T15:47:33.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.327+0000 7f400e203700 1 -- 172.21.15.47:0/2426384560 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40080573d0 con 0x7f40080b5c60 2022-06-29T15:47:33.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.327+0000 7f40067fc700 1 -- 172.21.15.47:0/2426384560 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4000025030 con 0x7f40080b5c60 2022-06-29T15:47:33.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.327+0000 7f40067fc700 1 --2- 172.21.15.47:0/2426384560 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3fe8053950 0x7f3fe8055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:33.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.327+0000 7f40067fc700 1 -- 172.21.15.47:0/2426384560 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f400006cda0 con 0x7f40080b5c60 2022-06-29T15:47:33.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.327+0000 7f400d201700 1 --2- 172.21.15.47:0/2426384560 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3fe8053950 0x7f3fe8055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:33.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.327+0000 7f400d201700 1 --2- 172.21.15.47:0/2426384560 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3fe8053950 0x7f3fe8055e10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f3ffc00c000 tx=0x7f3ffc008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:33.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.335+0000 7f40067fc700 1 -- 172.21.15.47:0/2426384560 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4000043d90 con 0x7f40080b5c60 2022-06-29T15:47:33.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.459+0000 7f400e203700 1 -- 172.21.15.47:0/2426384560 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f4008069e20 con 0x7f40080b5c60 2022-06-29T15:47:33.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.463+0000 7f40067fc700 1 -- 172.21.15.47:0/2426384560 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v64) v1 ==== 84+0+143 (secure 0 0 0) 0x7f4000023020 con 0x7f40080b5c60 2022-06-29T15:47:33.464 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:47:33.464 INFO:teuthology.orchestra.run.smithi047.stdout:[ 2022-06-29T15:47:33.464 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:33.465 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 1, 2022-06-29T15:47:33.465 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": "device_health_metrics" 2022-06-29T15:47:33.465 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:47:33.465 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:47:33.466 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 2, 2022-06-29T15:47:33.466 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": "rbd" 2022-06-29T15:47:33.466 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:47:33.466 INFO:teuthology.orchestra.run.smithi047.stdout:] 2022-06-29T15:47:33.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.463+0000 7f400e203700 1 -- 172.21.15.47:0/2426384560 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3fe8053950 msgr2=0x7f3fe8055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:33.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.463+0000 7f400e203700 1 --2- 172.21.15.47:0/2426384560 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3fe8053950 0x7f3fe8055e10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f3ffc00c000 tx=0x7f3ffc008040).stop 2022-06-29T15:47:33.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.463+0000 7f400e203700 1 -- 172.21.15.47:0/2426384560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40080b5c60 msgr2=0x7f400814b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:33.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.463+0000 7f400e203700 1 --2- 172.21.15.47:0/2426384560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40080b5c60 0x7f400814b480 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7f4000002f70 tx=0x7f4000015000).stop 2022-06-29T15:47:33.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.463+0000 7f400e203700 1 -- 172.21.15.47:0/2426384560 shutdown_connections 2022-06-29T15:47:33.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.463+0000 7f400e203700 1 --2- 172.21.15.47:0/2426384560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40080b5c60 0x7f400814b480 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:33.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.463+0000 7f400e203700 1 --2- 172.21.15.47:0/2426384560 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f40080a72f0 0x7f400814af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:33.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.463+0000 7f400e203700 1 --2- 172.21.15.47:0/2426384560 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3fe8053950 0x7f3fe8055e10 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:33.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.463+0000 7f400e203700 1 --2- 172.21.15.47:0/2426384560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4008060eb0 0x7f400814e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:33.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.463+0000 7f400e203700 1 -- 172.21.15.47:0/2426384560 >> 172.21.15.47:0/2426384560 conn(0x7f4008062560 msgr2=0x7f40080afae0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:33.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.463+0000 7f400e203700 1 -- 172.21.15.47:0/2426384560 shutdown_connections 2022-06-29T15:47:33.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.463+0000 7f400e203700 1 -- 172.21.15.47:0/2426384560 wait complete. 2022-06-29T15:47:33.476 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-06-29T15:47:33.477 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-06-29T15:47:33.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.587+0000 7f9ff8732700 1 Processor -- start 2022-06-29T15:47:33.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.587+0000 7f9ff8732700 1 -- start start 2022-06-29T15:47:33.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.587+0000 7f9ff8732700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9ff00b4300 0x7f9ff00612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:33.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.587+0000 7f9ff8732700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9ff01471a0 0x7f9ff0066f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:33.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.587+0000 7f9ff8732700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ff0067610 con 0x7f9ff0060ee0 2022-06-29T15:47:33.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.587+0000 7f9ff8732700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ff01467c0 con 0x7f9ff01471a0 2022-06-29T15:47:33.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.587+0000 7f9ff8732700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ff0146a60 con 0x7f9ff00b4300 2022-06-29T15:47:33.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.587+0000 7f9ff6ccf700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9ff01471a0 0x7f9ff0066f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:33.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.587+0000 7f9ff6ccf700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9ff01471a0 0x7f9ff0066f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49226/0 (socket says 172.21.15.47:49226) 2022-06-29T15:47:33.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.587+0000 7f9ff6ccf700 1 -- 172.21.15.47:0/2803532816 learned_addr learned my addr 172.21.15.47:0/2803532816 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:33.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.587+0000 7f9ff5ccd700 1 --2- 172.21.15.47:0/2803532816 >> v2:172.21.15.64:3300/0 conn(0x7f9ff00b4300 0x7f9ff00612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:33.594 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.591+0000 7f9ff6ccf700 1 -- 172.21.15.47:0/2803532816 >> v2:172.21.15.64:3300/0 conn(0x7f9ff00b4300 msgr2=0x7f9ff00612c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:33.594 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.591+0000 7f9ff6ccf700 1 --2- 172.21.15.47:0/2803532816 >> v2:172.21.15.64:3300/0 conn(0x7f9ff00b4300 0x7f9ff00612c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:33.594 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.591+0000 7f9ff6ccf700 1 -- 172.21.15.47:0/2803532816 >> v1:172.21.15.64:6789/0 conn(0x7f9ff0060ee0 legacy=0x7f9ff01430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:33.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.591+0000 7f9ff6ccf700 1 -- 172.21.15.47:0/2803532816 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff0146d00 con 0x7f9ff01471a0 2022-06-29T15:47:33.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.591+0000 7f9ff5ccd700 1 --2- 172.21.15.47:0/2803532816 >> v2:172.21.15.64:3300/0 conn(0x7f9ff00b4300 0x7f9ff00612c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:47:33.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.591+0000 7f9ff6ccf700 1 --2- 172.21.15.47:0/2803532816 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ff01471a0 0x7f9ff0066f70 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7f9ff00adb80 tx=0x7f9fe8009290).ready entity=mon.0 client_cookie=f9802bdb3581d203 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:33.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.591+0000 7f9ff54cc700 1 -- 172.21.15.47:0/2803532816 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9fe800b820 con 0x7f9ff01471a0 2022-06-29T15:47:33.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.591+0000 7f9ff54cc700 1 -- 172.21.15.47:0/2803532816 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fe800b980 con 0x7f9ff01471a0 2022-06-29T15:47:33.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.591+0000 7f9ff54cc700 1 -- 172.21.15.47:0/2803532816 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9fe800baf0 con 0x7f9ff01471a0 2022-06-29T15:47:33.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.591+0000 7f9ff8732700 1 -- 172.21.15.47:0/2803532816 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ff01471a0 msgr2=0x7f9ff0066f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:33.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.591+0000 7f9ff8732700 1 --2- 172.21.15.47:0/2803532816 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ff01471a0 0x7f9ff0066f70 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7f9ff00adb80 tx=0x7f9fe8009290).stop 2022-06-29T15:47:33.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.591+0000 7f9ff8732700 1 -- 172.21.15.47:0/2803532816 shutdown_connections 2022-06-29T15:47:33.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.591+0000 7f9ff8732700 1 --2- 172.21.15.47:0/2803532816 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ff01471a0 0x7f9ff0066f70 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:33.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.591+0000 7f9ff8732700 1 --2- 172.21.15.47:0/2803532816 >> v2:172.21.15.64:3300/0 conn(0x7f9ff00b4300 0x7f9ff00612c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:33.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.591+0000 7f9ff8732700 1 -- 172.21.15.47:0/2803532816 >> 172.21.15.47:0/2803532816 conn(0x7f9ff005c510 msgr2=0x7f9ff005f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:33.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.591+0000 7f9ff8732700 1 -- 172.21.15.47:0/2803532816 shutdown_connections 2022-06-29T15:47:33.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.591+0000 7f9ff8732700 1 -- 172.21.15.47:0/2803532816 wait complete. 2022-06-29T15:47:33.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.591+0000 7f9ff8732700 1 Processor -- start 2022-06-29T15:47:33.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.591+0000 7f9ff8732700 1 -- start start 2022-06-29T15:47:33.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.591+0000 7f9ff8732700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ff0060ee0 0x7f9ff01455d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:33.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.591+0000 7f9ff8732700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ff00b4300 0x7f9ff013b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:33.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.591+0000 7f9ff8732700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9ff01471a0 0x7f9ff0156190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:33.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.591+0000 7f9ff8732700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff013c380 con 0x7f9ff00b4300 2022-06-29T15:47:33.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.591+0000 7f9ff8732700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff0153000 con 0x7f9ff0060ee0 2022-06-29T15:47:33.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.591+0000 7f9ff8732700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff01532a0 con 0x7f9ff01471a0 2022-06-29T15:47:33.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.591+0000 7f9ff5ccd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ff00b4300 0x7f9ff013b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:33.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.591+0000 7f9ff5ccd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ff00b4300 0x7f9ff013b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49230/0 (socket says 172.21.15.47:49230) 2022-06-29T15:47:33.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.591+0000 7f9ff5ccd700 1 -- 172.21.15.47:0/444138631 learned_addr learned my addr 172.21.15.47:0/444138631 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:33.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.591+0000 7f9ff6ccf700 1 --2- 172.21.15.47:0/444138631 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9ff01471a0 0x7f9ff0156190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:33.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.591+0000 7f9ff64ce700 1 --2- 172.21.15.47:0/444138631 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ff0060ee0 0x7f9ff01455d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:33.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.591+0000 7f9ff5ccd700 1 -- 172.21.15.47:0/444138631 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9ff01471a0 msgr2=0x7f9ff0156190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:33.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.591+0000 7f9ff5ccd700 1 --2- 172.21.15.47:0/444138631 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9ff01471a0 0x7f9ff0156190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:33.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.591+0000 7f9ff5ccd700 1 -- 172.21.15.47:0/444138631 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ff0060ee0 msgr2=0x7f9ff01455d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:33.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.591+0000 7f9ff5ccd700 1 --2- 172.21.15.47:0/444138631 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ff0060ee0 0x7f9ff01455d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:33.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.591+0000 7f9ff5ccd700 1 -- 172.21.15.47:0/444138631 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fec00b180 con 0x7f9ff00b4300 2022-06-29T15:47:33.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.591+0000 7f9ff5ccd700 1 --2- 172.21.15.47:0/444138631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ff00b4300 0x7f9ff013b7b0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7f9fec00c000 tx=0x7f9fec006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:33.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.595+0000 7f9fe37fe700 1 -- 172.21.15.47:0/444138631 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9fec00c890 con 0x7f9ff00b4300 2022-06-29T15:47:33.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.595+0000 7f9fe37fe700 1 -- 172.21.15.47:0/444138631 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fec00c9f0 con 0x7f9ff00b4300 2022-06-29T15:47:33.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.595+0000 7f9ff8732700 1 -- 172.21.15.47:0/444138631 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fe8009530 con 0x7f9ff00b4300 2022-06-29T15:47:33.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.595+0000 7f9fe37fe700 1 -- 172.21.15.47:0/444138631 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9fec00cd50 con 0x7f9ff00b4300 2022-06-29T15:47:33.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.595+0000 7f9ff8732700 1 -- 172.21.15.47:0/444138631 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ff0155cf0 con 0x7f9ff00b4300 2022-06-29T15:47:33.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.595+0000 7f9ff8732700 1 -- 172.21.15.47:0/444138631 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ff0155ef0 con 0x7f9ff00b4300 2022-06-29T15:47:33.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.595+0000 7f9fe37fe700 1 -- 172.21.15.47:0/444138631 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9fec01c460 con 0x7f9ff00b4300 2022-06-29T15:47:33.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.595+0000 7f9fe37fe700 1 --2- 172.21.15.47:0/444138631 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9fd0053b30 0x7f9fd0055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:33.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.595+0000 7f9fe37fe700 1 -- 172.21.15.47:0/444138631 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f9fec06c3d0 con 0x7f9ff00b4300 2022-06-29T15:47:33.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.595+0000 7f9ff64ce700 1 --2- 172.21.15.47:0/444138631 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9fd0053b30 0x7f9fd0055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:33.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.595+0000 7f9ff64ce700 1 --2- 172.21.15.47:0/444138631 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9fd0053b30 0x7f9fd0055ff0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f9fe400b000 tx=0x7f9fe4009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:33.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.599+0000 7f9fe37fe700 1 -- 172.21.15.47:0/444138631 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9ff0155ef0 con 0x7f9ff00b4300 2022-06-29T15:47:33.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:33.727+0000 7f9ff8732700 1 -- 172.21.15.47:0/444138631 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f9ff0155ef0 con 0x7f9ff00b4300 2022-06-29T15:47:34.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.271+0000 7f9fe37fe700 1 -- 172.21.15.47:0/444138631 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v65) v1 ==== 133+0+0 (secure 0 0 0) 0x7f9ff0155ef0 con 0x7f9ff00b4300 2022-06-29T15:47:34.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.275+0000 7f9ff8732700 1 -- 172.21.15.47:0/444138631 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9fd0053b30 msgr2=0x7f9fd0055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:34.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.275+0000 7f9ff8732700 1 --2- 172.21.15.47:0/444138631 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9fd0053b30 0x7f9fd0055ff0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f9fe400b000 tx=0x7f9fe4009040).stop 2022-06-29T15:47:34.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.275+0000 7f9ff8732700 1 -- 172.21.15.47:0/444138631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ff00b4300 msgr2=0x7f9ff013b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:34.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.275+0000 7f9ff8732700 1 --2- 172.21.15.47:0/444138631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ff00b4300 0x7f9ff013b7b0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7f9fec00c000 tx=0x7f9fec006df0).stop 2022-06-29T15:47:34.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.275+0000 7f9ff8732700 1 -- 172.21.15.47:0/444138631 shutdown_connections 2022-06-29T15:47:34.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.275+0000 7f9ff8732700 1 --2- 172.21.15.47:0/444138631 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9ff01471a0 0x7f9ff0156190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:34.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.275+0000 7f9ff8732700 1 --2- 172.21.15.47:0/444138631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ff00b4300 0x7f9ff013b7b0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:34.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.275+0000 7f9ff8732700 1 --2- 172.21.15.47:0/444138631 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9fd0053b30 0x7f9fd0055ff0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:34.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.275+0000 7f9ff8732700 1 --2- 172.21.15.47:0/444138631 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ff0060ee0 0x7f9ff01455d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:34.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.275+0000 7f9ff8732700 1 -- 172.21.15.47:0/444138631 >> 172.21.15.47:0/444138631 conn(0x7f9ff005c510 msgr2=0x7f9ff013ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:34.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.275+0000 7f9ff8732700 1 -- 172.21.15.47:0/444138631 shutdown_connections 2022-06-29T15:47:34.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.275+0000 7f9ff8732700 1 -- 172.21.15.47:0/444138631 wait complete. 2022-06-29T15:47:34.282 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' created 2022-06-29T15:47:34.295 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-06-29T15:47:34.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.411+0000 7fd75a9ad700 1 Processor -- start 2022-06-29T15:47:34.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.411+0000 7fd75a9ad700 1 -- start start 2022-06-29T15:47:34.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.411+0000 7fd75a9ad700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd7540b2c60 0x7fd75405f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:34.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.411+0000 7fd75a9ad700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd7540611f0 0x7fd754142e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:34.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.411+0000 7fd75a9ad700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7540aa830 con 0x7fd75413ef00 2022-06-29T15:47:34.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.411+0000 7fd75a9ad700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd75405fad0 con 0x7fd7540b2c60 2022-06-29T15:47:34.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.411+0000 7fd75a9ad700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7540b3040 con 0x7fd7540611f0 2022-06-29T15:47:34.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.411+0000 7fd75a1ac700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd75413ef00 0x7fd7540ac160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34794/0 (socket says 172.21.15.47:34794) 2022-06-29T15:47:34.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.411+0000 7fd75a1ac700 1 -- 172.21.15.47:0/1235150943 learned_addr learned my addr 172.21.15.47:0/1235150943 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:47:34.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.411+0000 7fd7591aa700 1 --2- 172.21.15.47:0/1235150943 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd7540611f0 0x7fd754142e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:34.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.411+0000 7fd7599ab700 1 --2- 172.21.15.47:0/1235150943 >> v2:172.21.15.64:3300/0 conn(0x7fd7540b2c60 0x7fd75405f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:34.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.411+0000 7fd7589a9700 1 -- 172.21.15.47:0/1235150943 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 688219259 0 0) 0x7fd7540aa830 con 0x7fd75413ef00 2022-06-29T15:47:34.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.411+0000 7fd7589a9700 1 -- 172.21.15.47:0/1235150943 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd740003260 con 0x7fd75413ef00 2022-06-29T15:47:34.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.411+0000 7fd7591aa700 1 -- 172.21.15.47:0/1235150943 >> v2:172.21.15.64:3300/0 conn(0x7fd7540b2c60 msgr2=0x7fd75405f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:34.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.411+0000 7fd7591aa700 1 --2- 172.21.15.47:0/1235150943 >> v2:172.21.15.64:3300/0 conn(0x7fd7540b2c60 0x7fd75405f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:34.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.411+0000 7fd7591aa700 1 -- 172.21.15.47:0/1235150943 >> v1:172.21.15.47:6789/0 conn(0x7fd75413ef00 legacy=0x7fd7540ac160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:34.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.411+0000 7fd7591aa700 1 -- 172.21.15.47:0/1235150943 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7541424d0 con 0x7fd7540611f0 2022-06-29T15:47:34.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.411+0000 7fd7591aa700 1 --2- 172.21.15.47:0/1235150943 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd7540611f0 0x7fd754142e40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fd748002f70 tx=0x7fd748006df0).ready entity=mon.2 client_cookie=bf2f2cbbe789f3a4 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:34.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.415+0000 7fd7589a9700 1 -- 172.21.15.47:0/1235150943 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd74800d4a0 con 0x7fd7540611f0 2022-06-29T15:47:34.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.415+0000 7fd7589a9700 1 -- 172.21.15.47:0/1235150943 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd748000e90 con 0x7fd7540611f0 2022-06-29T15:47:34.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.415+0000 7fd7589a9700 1 -- 172.21.15.47:0/1235150943 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd74800d7f0 con 0x7fd7540611f0 2022-06-29T15:47:34.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.415+0000 7fd75a9ad700 1 -- 172.21.15.47:0/1235150943 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd7540611f0 msgr2=0x7fd754142e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:34.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.415+0000 7fd75a9ad700 1 --2- 172.21.15.47:0/1235150943 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd7540611f0 0x7fd754142e40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fd748002f70 tx=0x7fd748006df0).stop 2022-06-29T15:47:34.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.415+0000 7fd75a9ad700 1 -- 172.21.15.47:0/1235150943 shutdown_connections 2022-06-29T15:47:34.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.415+0000 7fd75a9ad700 1 --2- 172.21.15.47:0/1235150943 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd7540611f0 0x7fd754142e40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:34.424 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.415+0000 7fd75a9ad700 1 --2- 172.21.15.47:0/1235150943 >> v2:172.21.15.64:3300/0 conn(0x7fd7540b2c60 0x7fd75405f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:34.424 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.415+0000 7fd75a9ad700 1 -- 172.21.15.47:0/1235150943 >> 172.21.15.47:0/1235150943 conn(0x7fd7540b6240 msgr2=0x7fd7540b2210 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:34.424 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.415+0000 7fd75a9ad700 1 -- 172.21.15.47:0/1235150943 shutdown_connections 2022-06-29T15:47:34.424 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.415+0000 7fd75a9ad700 1 -- 172.21.15.47:0/1235150943 wait complete. 2022-06-29T15:47:34.425 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.415+0000 7fd75a9ad700 1 Processor -- start 2022-06-29T15:47:34.425 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.415+0000 7fd75a9ad700 1 -- start start 2022-06-29T15:47:34.427 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.415+0000 7fd75a9ad700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd7540611f0 0x7fd75413e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:34.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.415+0000 7fd75a9ad700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7540b2c60 0x7fd754063480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:34.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.415+0000 7fd75a9ad700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd75413ef00 0x7fd754063970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:34.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.415+0000 7fd75a9ad700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7540ac2f0 con 0x7fd75413ef00 2022-06-29T15:47:34.429 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.415+0000 7fd75a9ad700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7540ad210 con 0x7fd7540b2c60 2022-06-29T15:47:34.429 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.415+0000 7fd75a9ad700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7540ac520 con 0x7fd7540611f0 2022-06-29T15:47:34.429 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.415+0000 7fd75a1ac700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd75413ef00 0x7fd754063970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:34.429 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.415+0000 7fd7599ab700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd7540611f0 0x7fd75413e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:34.430 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.415+0000 7fd75a1ac700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd75413ef00 0x7fd754063970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49246/0 (socket says 172.21.15.47:49246) 2022-06-29T15:47:34.430 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.419+0000 7fd75a1ac700 1 -- 172.21.15.47:0/2004365850 learned_addr learned my addr 172.21.15.47:0/2004365850 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:34.430 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.419+0000 7fd7591aa700 1 --2- 172.21.15.47:0/2004365850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7540b2c60 0x7fd754063480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:34.430 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.419+0000 7fd7599ab700 1 -- 172.21.15.47:0/2004365850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7540b2c60 msgr2=0x7fd754063480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:34.431 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.419+0000 7fd7599ab700 1 --2- 172.21.15.47:0/2004365850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7540b2c60 0x7fd754063480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:34.431 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.419+0000 7fd7599ab700 1 -- 172.21.15.47:0/2004365850 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd75413ef00 msgr2=0x7fd754063970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:34.431 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.419+0000 7fd7599ab700 1 --2- 172.21.15.47:0/2004365850 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd75413ef00 0x7fd754063970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:34.431 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.419+0000 7fd7599ab700 1 -- 172.21.15.47:0/2004365850 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd74800b180 con 0x7fd7540611f0 2022-06-29T15:47:34.432 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.419+0000 7fd7599ab700 1 --2- 172.21.15.47:0/2004365850 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd7540611f0 0x7fd75413e610 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7fd75413c150 tx=0x7fd744006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:34.432 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.419+0000 7fd752ffd700 1 -- 172.21.15.47:0/2004365850 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd744010780 con 0x7fd7540611f0 2022-06-29T15:47:34.432 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.419+0000 7fd752ffd700 1 -- 172.21.15.47:0/2004365850 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7440108e0 con 0x7fd7540611f0 2022-06-29T15:47:34.432 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.419+0000 7fd75a9ad700 1 -- 172.21.15.47:0/2004365850 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7540ad470 con 0x7fd7540611f0 2022-06-29T15:47:34.433 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.419+0000 7fd752ffd700 1 -- 172.21.15.47:0/2004365850 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd744010c40 con 0x7fd7540611f0 2022-06-29T15:47:34.433 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.419+0000 7fd75a9ad700 1 -- 172.21.15.47:0/2004365850 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd75413ec40 con 0x7fd7540611f0 2022-06-29T15:47:34.433 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.419+0000 7fd75a9ad700 1 -- 172.21.15.47:0/2004365850 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7540573d0 con 0x7fd7540611f0 2022-06-29T15:47:34.433 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.419+0000 7fd752ffd700 1 -- 172.21.15.47:0/2004365850 <== mon.2 v2:172.21.15.64:3301/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd74401b400 con 0x7fd7540611f0 2022-06-29T15:47:34.434 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.423+0000 7fd752ffd700 1 --2- 172.21.15.47:0/2004365850 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd734053950 0x7fd734055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:34.434 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.423+0000 7fd752ffd700 1 -- 172.21.15.47:0/2004365850 <== mon.2 v2:172.21.15.64:3301/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5954+0+0 (secure 0 0 0) 0x7fd74406bf70 con 0x7fd7540611f0 2022-06-29T15:47:34.434 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.423+0000 7fd7591aa700 1 --2- 172.21.15.47:0/2004365850 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd734053950 0x7fd734055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:34.434 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.423+0000 7fd7591aa700 1 --2- 172.21.15.47:0/2004365850 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd734053950 0x7fd734055e10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7fd74801a000 tx=0x7fd748014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:34.435 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.431+0000 7fd752ffd700 1 -- 172.21.15.47:0/2004365850 <== mon.2 v2:172.21.15.64:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd744042e00 con 0x7fd7540611f0 2022-06-29T15:47:34.560 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:34.559+0000 7fd75a9ad700 1 -- 172.21.15.47:0/2004365850 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7fd75406b700 con 0x7fd7540611f0 2022-06-29T15:47:35.024 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.023+0000 7fd752ffd700 1 -- 172.21.15.47:0/2004365850 <== mon.2 v2:172.21.15.64:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v66) v1 ==== 125+0+0 (secure 0 0 0) 0x7fd744046420 con 0x7fd7540611f0 2022-06-29T15:47:35.031 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.031+0000 7fd75a9ad700 1 -- 172.21.15.47:0/2004365850 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd734053950 msgr2=0x7fd734055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:35.031 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.031+0000 7fd75a9ad700 1 --2- 172.21.15.47:0/2004365850 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd734053950 0x7fd734055e10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7fd74801a000 tx=0x7fd748014000).stop 2022-06-29T15:47:35.032 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.031+0000 7fd75a9ad700 1 -- 172.21.15.47:0/2004365850 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd7540611f0 msgr2=0x7fd75413e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:35.032 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.031+0000 7fd75a9ad700 1 --2- 172.21.15.47:0/2004365850 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd7540611f0 0x7fd75413e610 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7fd75413c150 tx=0x7fd744006df0).stop 2022-06-29T15:47:35.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.031+0000 7fd75a9ad700 1 -- 172.21.15.47:0/2004365850 shutdown_connections 2022-06-29T15:47:35.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.031+0000 7fd75a9ad700 1 --2- 172.21.15.47:0/2004365850 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd75413ef00 0x7fd754063970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:35.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.031+0000 7fd75a9ad700 1 --2- 172.21.15.47:0/2004365850 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd734053950 0x7fd734055e10 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:35.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.031+0000 7fd75a9ad700 1 --2- 172.21.15.47:0/2004365850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7540b2c60 0x7fd754063480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:35.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.031+0000 7fd75a9ad700 1 --2- 172.21.15.47:0/2004365850 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd7540611f0 0x7fd75413e610 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:35.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.031+0000 7fd75a9ad700 1 -- 172.21.15.47:0/2004365850 >> 172.21.15.47:0/2004365850 conn(0x7fd7540b6240 msgr2=0x7fd7540b4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:35.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.031+0000 7fd75a9ad700 1 -- 172.21.15.47:0/2004365850 shutdown_connections 2022-06-29T15:47:35.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.031+0000 7fd75a9ad700 1 -- 172.21.15.47:0/2004365850 wait complete. 2022-06-29T15:47:35.036 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' created 2022-06-29T15:47:35.047 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-06-29T15:47:35.154 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.151+0000 7f7c97304700 1 Processor -- start 2022-06-29T15:47:35.154 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.151+0000 7f7c97304700 1 -- start start 2022-06-29T15:47:35.158 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.151+0000 7f7c97304700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7c900b4300 0x7f7c900612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:35.158 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.151+0000 7f7c97304700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7c901471a0 0x7f7c90066f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:35.159 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.151+0000 7f7c97304700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c90067610 con 0x7f7c90060ee0 2022-06-29T15:47:35.159 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.151+0000 7f7c97304700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c901467c0 con 0x7f7c901471a0 2022-06-29T15:47:35.159 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.151+0000 7f7c97304700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c90146a60 con 0x7f7c900b4300 2022-06-29T15:47:35.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.151+0000 7f7c958a1700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7c901471a0 0x7f7c90066f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:35.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.155+0000 7f7c958a1700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7c901471a0 0x7f7c90066f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49254/0 (socket says 172.21.15.47:49254) 2022-06-29T15:47:35.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.155+0000 7f7c958a1700 1 -- 172.21.15.47:0/1942080591 learned_addr learned my addr 172.21.15.47:0/1942080591 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:35.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.155+0000 7f7c9489f700 1 --2- 172.21.15.47:0/1942080591 >> v2:172.21.15.64:3300/0 conn(0x7f7c900b4300 0x7f7c900612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:35.161 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.155+0000 7f7c958a1700 1 -- 172.21.15.47:0/1942080591 >> v2:172.21.15.64:3300/0 conn(0x7f7c900b4300 msgr2=0x7f7c900612c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:35.161 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.155+0000 7f7c958a1700 1 --2- 172.21.15.47:0/1942080591 >> v2:172.21.15.64:3300/0 conn(0x7f7c900b4300 0x7f7c900612c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:35.161 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.155+0000 7f7c958a1700 1 -- 172.21.15.47:0/1942080591 >> v1:172.21.15.64:6789/0 conn(0x7f7c90060ee0 legacy=0x7f7c901430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:35.161 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.155+0000 7f7c958a1700 1 -- 172.21.15.47:0/1942080591 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c90146d00 con 0x7f7c901471a0 2022-06-29T15:47:35.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.155+0000 7f7c958a1700 1 --2- 172.21.15.47:0/1942080591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c901471a0 0x7f7c90066f70 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7f7c900adb80 tx=0x7f7c88009290).ready entity=mon.0 client_cookie=20aa74b0ab0006d server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:35.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.155+0000 7f7c87fff700 1 -- 172.21.15.47:0/1942080591 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c8800b820 con 0x7f7c901471a0 2022-06-29T15:47:35.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.155+0000 7f7c87fff700 1 -- 172.21.15.47:0/1942080591 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c8800b980 con 0x7f7c901471a0 2022-06-29T15:47:35.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.155+0000 7f7c87fff700 1 -- 172.21.15.47:0/1942080591 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c8800baf0 con 0x7f7c901471a0 2022-06-29T15:47:35.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.155+0000 7f7c97304700 1 -- 172.21.15.47:0/1942080591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c901471a0 msgr2=0x7f7c90066f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:35.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.155+0000 7f7c97304700 1 --2- 172.21.15.47:0/1942080591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c901471a0 0x7f7c90066f70 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7f7c900adb80 tx=0x7f7c88009290).stop 2022-06-29T15:47:35.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.155+0000 7f7c97304700 1 -- 172.21.15.47:0/1942080591 shutdown_connections 2022-06-29T15:47:35.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.155+0000 7f7c97304700 1 --2- 172.21.15.47:0/1942080591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c901471a0 0x7f7c90066f70 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:35.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.155+0000 7f7c97304700 1 --2- 172.21.15.47:0/1942080591 >> v2:172.21.15.64:3300/0 conn(0x7f7c900b4300 0x7f7c900612c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:35.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.155+0000 7f7c97304700 1 -- 172.21.15.47:0/1942080591 >> 172.21.15.47:0/1942080591 conn(0x7f7c9005c510 msgr2=0x7f7c9005f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:35.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.155+0000 7f7c97304700 1 -- 172.21.15.47:0/1942080591 shutdown_connections 2022-06-29T15:47:35.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.155+0000 7f7c97304700 1 -- 172.21.15.47:0/1942080591 wait complete. 2022-06-29T15:47:35.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.155+0000 7f7c97304700 1 Processor -- start 2022-06-29T15:47:35.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.155+0000 7f7c97304700 1 -- start start 2022-06-29T15:47:35.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.159+0000 7f7c97304700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7c90060ee0 0x7f7c900b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:35.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.159+0000 7f7c97304700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c900b4300 0x7f7c901354d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:35.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.159+0000 7f7c97304700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c901471a0 0x7f7c901359c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:35.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.159+0000 7f7c97304700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9013f300 con 0x7f7c901471a0 2022-06-29T15:47:35.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.159+0000 7f7c97304700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c90144eb0 con 0x7f7c900b4300 2022-06-29T15:47:35.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.159+0000 7f7c97304700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c90145cc0 con 0x7f7c90060ee0 2022-06-29T15:47:35.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.159+0000 7f7c958a1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c901471a0 0x7f7c901359c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:35.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.159+0000 7f7c9489f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c900b4300 0x7f7c901354d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:35.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.159+0000 7f7c958a1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c901471a0 0x7f7c901359c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49260/0 (socket says 172.21.15.47:49260) 2022-06-29T15:47:35.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.159+0000 7f7c958a1700 1 -- 172.21.15.47:0/242238383 learned_addr learned my addr 172.21.15.47:0/242238383 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:35.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.159+0000 7f7c950a0700 1 --2- 172.21.15.47:0/242238383 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7c90060ee0 0x7f7c900b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:35.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.159+0000 7f7c958a1700 1 -- 172.21.15.47:0/242238383 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7c90060ee0 msgr2=0x7f7c900b7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:35.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.159+0000 7f7c958a1700 1 --2- 172.21.15.47:0/242238383 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7c90060ee0 0x7f7c900b7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:35.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.159+0000 7f7c958a1700 1 -- 172.21.15.47:0/242238383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c900b4300 msgr2=0x7f7c901354d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:35.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.159+0000 7f7c958a1700 1 --2- 172.21.15.47:0/242238383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c900b4300 0x7f7c901354d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:35.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.159+0000 7f7c958a1700 1 -- 172.21.15.47:0/242238383 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c88009530 con 0x7f7c901471a0 2022-06-29T15:47:35.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.159+0000 7f7c958a1700 1 --2- 172.21.15.47:0/242238383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c901471a0 0x7f7c901359c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7f7c900adb80 tx=0x7f7c88016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:35.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.159+0000 7f7c867fc700 1 -- 172.21.15.47:0/242238383 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c880032f0 con 0x7f7c901471a0 2022-06-29T15:47:35.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.159+0000 7f7c867fc700 1 -- 172.21.15.47:0/242238383 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c88021440 con 0x7f7c901471a0 2022-06-29T15:47:35.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.159+0000 7f7c97304700 1 -- 172.21.15.47:0/242238383 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c901450e0 con 0x7f7c901471a0 2022-06-29T15:47:35.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.159+0000 7f7c867fc700 1 -- 172.21.15.47:0/242238383 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c880217a0 con 0x7f7c901471a0 2022-06-29T15:47:35.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.159+0000 7f7c97304700 1 -- 172.21.15.47:0/242238383 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c901443d0 con 0x7f7c901471a0 2022-06-29T15:47:35.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.163+0000 7f7c97304700 1 -- 172.21.15.47:0/242238383 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c90145580 con 0x7f7c901471a0 2022-06-29T15:47:35.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.163+0000 7f7c867fc700 1 -- 172.21.15.47:0/242238383 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7c8800f070 con 0x7f7c901471a0 2022-06-29T15:47:35.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.163+0000 7f7c867fc700 1 --2- 172.21.15.47:0/242238383 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7c78053a70 0x7f7c78055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:35.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.163+0000 7f7c867fc700 1 -- 172.21.15.47:0/242238383 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f7c8806d7a0 con 0x7f7c901471a0 2022-06-29T15:47:35.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.163+0000 7f7c950a0700 1 --2- 172.21.15.47:0/242238383 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7c78053a70 0x7f7c78055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:35.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.163+0000 7f7c950a0700 1 --2- 172.21.15.47:0/242238383 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7c78053a70 0x7f7c78055f30 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f7c8000d000 tx=0x7f7c80009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:35.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.171+0000 7f7c867fc700 1 -- 172.21.15.47:0/242238383 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7c90145580 con 0x7f7c901471a0 2022-06-29T15:47:35.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.335+0000 7f7c97304700 1 -- 172.21.15.47:0/242238383 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f7c90145580 con 0x7f7c901471a0 2022-06-29T15:47:35.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.347+0000 7f7c867fc700 1 -- 172.21.15.47:0/242238383 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 11 and data pool 12 v29) v1 ==== 189+0+0 (secure 0 0 0) 0x7f7c90145580 con 0x7f7c901471a0 2022-06-29T15:47:35.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.355+0000 7f7c97304700 1 -- 172.21.15.47:0/242238383 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7c78053a70 msgr2=0x7f7c78055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:35.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.355+0000 7f7c97304700 1 --2- 172.21.15.47:0/242238383 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7c78053a70 0x7f7c78055f30 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f7c8000d000 tx=0x7f7c80009040).stop 2022-06-29T15:47:35.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.355+0000 7f7c97304700 1 -- 172.21.15.47:0/242238383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c901471a0 msgr2=0x7f7c901359c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:35.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.355+0000 7f7c97304700 1 --2- 172.21.15.47:0/242238383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c901471a0 0x7f7c901359c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7f7c900adb80 tx=0x7f7c88016000).stop 2022-06-29T15:47:35.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.355+0000 7f7c97304700 1 -- 172.21.15.47:0/242238383 shutdown_connections 2022-06-29T15:47:35.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.355+0000 7f7c97304700 1 --2- 172.21.15.47:0/242238383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c901471a0 0x7f7c901359c0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:35.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.355+0000 7f7c97304700 1 --2- 172.21.15.47:0/242238383 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7c78053a70 0x7f7c78055f30 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:35.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.355+0000 7f7c97304700 1 --2- 172.21.15.47:0/242238383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c900b4300 0x7f7c901354d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:35.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.355+0000 7f7c97304700 1 --2- 172.21.15.47:0/242238383 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7c90060ee0 0x7f7c900b7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:35.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.355+0000 7f7c97304700 1 -- 172.21.15.47:0/242238383 >> 172.21.15.47:0/242238383 conn(0x7f7c9005c510 msgr2=0x7f7c90138bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:35.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.355+0000 7f7c97304700 1 -- 172.21.15.47:0/242238383 shutdown_connections 2022-06-29T15:47:35.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.355+0000 7f7c97304700 1 -- 172.21.15.47:0/242238383 wait complete. 2022-06-29T15:47:35.359 INFO:teuthology.orchestra.run.smithi047.stderr:new fs with metadata pool 11 and data pool 12 2022-06-29T15:47:35.369 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:47:35.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.463+0000 7f72fc6ee700 1 Processor -- start 2022-06-29T15:47:35.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.463+0000 7f72fc6ee700 1 -- start start 2022-06-29T15:47:35.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.463+0000 7f72fc6ee700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f72f4095950 0x7f72f4097690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:35.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.463+0000 7f72fc6ee700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72f4143be0 con 0x7f72f41430f0 2022-06-29T15:47:35.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.463+0000 7f72fc6ee700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72f4095dc0 con 0x7f72f40a93c0 2022-06-29T15:47:35.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.463+0000 7f72fc6ee700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72f413d590 con 0x7f72f4095950 2022-06-29T15:47:35.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.463+0000 7f72fa48a700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f72f41430f0 0x7f72f41434d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34832/0 (socket says 172.21.15.47:34832) 2022-06-29T15:47:35.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.463+0000 7f72fa48a700 1 -- 172.21.15.47:0/2753458312 learned_addr learned my addr 172.21.15.47:0/2753458312 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:47:35.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.463+0000 7f72fac8b700 1 --2- 172.21.15.47:0/2753458312 >> v2:172.21.15.64:3300/0 conn(0x7f72f4095950 0x7f72f4097690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:35.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.463+0000 7f72f9488700 1 -- 172.21.15.47:0/2753458312 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4064847496 0 0) 0x7f72f4143be0 con 0x7f72f41430f0 2022-06-29T15:47:35.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.463+0000 7f72f9488700 1 -- 172.21.15.47:0/2753458312 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72dc003260 con 0x7f72f41430f0 2022-06-29T15:47:35.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.463+0000 7f72f9488700 1 -- 172.21.15.47:0/2753458312 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 15619993 0 0) 0x7f72dc003260 con 0x7f72f41430f0 2022-06-29T15:47:35.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.463+0000 7f72f9488700 1 -- 172.21.15.47:0/2753458312 >> v2:172.21.15.64:3300/0 conn(0x7f72f4095950 msgr2=0x7f72f4097690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:35.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.463+0000 7f72f9488700 1 --2- 172.21.15.47:0/2753458312 >> v2:172.21.15.64:3300/0 conn(0x7f72f4095950 0x7f72f4097690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:35.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.463+0000 7f72f9488700 1 -- 172.21.15.47:0/2753458312 >> v1:172.21.15.64:6789/0 conn(0x7f72f40a93c0 legacy=0x7f72f4095240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:35.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.463+0000 7f72f9488700 1 -- 172.21.15.47:0/2753458312 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f4060e50 con 0x7f72f41430f0 2022-06-29T15:47:35.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.463+0000 7f72f9488700 1 -- 172.21.15.47:0/2753458312 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7f72e4001660 con 0x7f72f41430f0 2022-06-29T15:47:35.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.463+0000 7f72fac8b700 1 --2- 172.21.15.47:0/2753458312 >> v2:172.21.15.64:3300/0 conn(0x7f72f4095950 0x7f72f4097690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:47:35.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.463+0000 7f72f9488700 1 -- 172.21.15.47:0/2753458312 >> v1:172.21.15.47:6789/0 conn(0x7f72f41430f0 legacy=0x7f72f41434d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:35.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.463+0000 7f72f9488700 1 --2- 172.21.15.47:0/2753458312 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f72dc003750 0x7f72dc003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:35.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.463+0000 7f72f9488700 1 --2- 172.21.15.47:0/2753458312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72dc004500 0x7f72dc004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:35.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.463+0000 7f72f9488700 1 --2- 172.21.15.47:0/2753458312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72dc004ec0 0x7f72dc00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:35.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.463+0000 7f72f9488700 1 -- 172.21.15.47:0/2753458312 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f413d590 con 0x7f72dc004ec0 2022-06-29T15:47:35.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.463+0000 7f72f9488700 1 -- 172.21.15.47:0/2753458312 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc00bda0 con 0x7f72dc004500 2022-06-29T15:47:35.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.463+0000 7f72f9488700 1 -- 172.21.15.47:0/2753458312 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc00bf60 con 0x7f72dc003750 2022-06-29T15:47:35.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.463+0000 7f72fac8b700 1 --2- 172.21.15.47:0/2753458312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72dc004ec0 0x7f72dc00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:35.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.463+0000 7f72f9c89700 1 --2- 172.21.15.47:0/2753458312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72dc004500 0x7f72dc004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:35.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.463+0000 7f72fa48a700 1 --2- 172.21.15.47:0/2753458312 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f72dc003750 0x7f72dc003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:35.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.463+0000 7f72fac8b700 1 -- 172.21.15.47:0/2753458312 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f72dc003750 msgr2=0x7f72dc003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:35.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.463+0000 7f72fac8b700 1 --2- 172.21.15.47:0/2753458312 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f72dc003750 0x7f72dc003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:35.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.463+0000 7f72fac8b700 1 -- 172.21.15.47:0/2753458312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72dc004500 msgr2=0x7f72dc004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:35.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.463+0000 7f72fac8b700 1 --2- 172.21.15.47:0/2753458312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72dc004500 0x7f72dc004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:35.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.463+0000 7f72fac8b700 1 -- 172.21.15.47:0/2753458312 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72dc003260 con 0x7f72dc004ec0 2022-06-29T15:47:35.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.463+0000 7f72fa48a700 1 --2- 172.21.15.47:0/2753458312 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f72dc003750 0x7f72dc003b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:47:35.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.463+0000 7f72fac8b700 1 -- 172.21.15.47:0/2753458312 reap_dead start 2022-06-29T15:47:35.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.463+0000 7f72fac8b700 1 --2- 172.21.15.47:0/2753458312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72dc004ec0 0x7f72dc00b750 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7f72ec00a000 tx=0x7f72ec008040).ready entity=mon.0 client_cookie=3c087695af21ca3d server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:35.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.463+0000 7f72f9488700 1 -- 172.21.15.47:0/2753458312 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72ec00c070 con 0x7f72dc004ec0 2022-06-29T15:47:35.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.463+0000 7f72f9488700 1 -- 172.21.15.47:0/2753458312 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72ec00e620 con 0x7f72dc004ec0 2022-06-29T15:47:35.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.463+0000 7f72f9488700 1 -- 172.21.15.47:0/2753458312 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72ec00e980 con 0x7f72dc004ec0 2022-06-29T15:47:35.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.467+0000 7f72fc6ee700 1 -- 172.21.15.47:0/2753458312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72dc004ec0 msgr2=0x7f72dc00b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:35.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.467+0000 7f72fc6ee700 1 --2- 172.21.15.47:0/2753458312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72dc004ec0 0x7f72dc00b750 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7f72ec00a000 tx=0x7f72ec008040).stop 2022-06-29T15:47:35.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.467+0000 7f72fc6ee700 1 -- 172.21.15.47:0/2753458312 shutdown_connections 2022-06-29T15:47:35.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.467+0000 7f72fc6ee700 1 --2- 172.21.15.47:0/2753458312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72dc004ec0 0x7f72dc00b750 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:35.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.467+0000 7f72fc6ee700 1 -- 172.21.15.47:0/2753458312 >> 172.21.15.47:0/2753458312 conn(0x7f72f405f0b0 msgr2=0x7f72f405f490 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:35.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.467+0000 7f72fc6ee700 1 -- 172.21.15.47:0/2753458312 shutdown_connections 2022-06-29T15:47:35.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.467+0000 7f72fc6ee700 1 -- 172.21.15.47:0/2753458312 wait complete. 2022-06-29T15:47:35.485 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.467+0000 7f72fc6ee700 1 Processor -- start 2022-06-29T15:47:35.485 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.467+0000 7f72fc6ee700 1 -- start start 2022-06-29T15:47:35.485 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.467+0000 7f72fc6ee700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f72dc004ec0 0x7f72f40b54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:35.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.467+0000 7f72fc6ee700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72f41306e0 0x7f72f412fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:35.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.467+0000 7f72fc6ee700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72f4130230 0x7f72f40b1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:35.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.467+0000 7f72fc6ee700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f409b560 con 0x7f72f41306e0 2022-06-29T15:47:35.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.467+0000 7f72fc6ee700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f4098d80 con 0x7f72f4130230 2022-06-29T15:47:35.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.467+0000 7f72fc6ee700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72f412f710 con 0x7f72dc004ec0 2022-06-29T15:47:35.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.467+0000 7f72f9c89700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72f41306e0 0x7f72f412fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:35.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.467+0000 7f72f9c89700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72f41306e0 0x7f72f412fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49292/0 (socket says 172.21.15.47:49292) 2022-06-29T15:47:35.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.467+0000 7f72f9c89700 1 -- 172.21.15.47:0/3699118671 learned_addr learned my addr 172.21.15.47:0/3699118671 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:35.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.471+0000 7f72fac8b700 1 --2- 172.21.15.47:0/3699118671 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72f4130230 0x7f72f40b1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:35.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.471+0000 7f72fa48a700 1 --2- 172.21.15.47:0/3699118671 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f72dc004ec0 0x7f72f40b54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:35.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.471+0000 7f72f9c89700 1 -- 172.21.15.47:0/3699118671 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f72dc004ec0 msgr2=0x7f72f40b54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:35.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.471+0000 7f72f9c89700 1 --2- 172.21.15.47:0/3699118671 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f72dc004ec0 0x7f72f40b54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:35.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.471+0000 7f72f9c89700 1 -- 172.21.15.47:0/3699118671 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72f4130230 msgr2=0x7f72f40b1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:35.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.471+0000 7f72f9c89700 1 --2- 172.21.15.47:0/3699118671 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72f4130230 0x7f72f40b1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:35.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.471+0000 7f72f9c89700 1 -- 172.21.15.47:0/3699118671 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f409b790 con 0x7f72f41306e0 2022-06-29T15:47:35.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.471+0000 7f72f9c89700 1 --2- 172.21.15.47:0/3699118671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72f41306e0 0x7f72f412fd40 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7f72f000d000 tx=0x7f72f0007a00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:35.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.471+0000 7f72e37fe700 1 -- 172.21.15.47:0/3699118671 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72f00133f0 con 0x7f72f41306e0 2022-06-29T15:47:35.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.471+0000 7f72e37fe700 1 -- 172.21.15.47:0/3699118671 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72f0013550 con 0x7f72f41306e0 2022-06-29T15:47:35.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.471+0000 7f72fc6ee700 1 -- 172.21.15.47:0/3699118671 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72f40696e0 con 0x7f72f41306e0 2022-06-29T15:47:35.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.471+0000 7f72e37fe700 1 -- 172.21.15.47:0/3699118671 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72f00138b0 con 0x7f72f41306e0 2022-06-29T15:47:35.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.471+0000 7f72fc6ee700 1 -- 172.21.15.47:0/3699118671 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72f40b59a0 con 0x7f72f41306e0 2022-06-29T15:47:35.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.471+0000 7f72fc6ee700 1 -- 172.21.15.47:0/3699118671 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72f409b920 con 0x7f72f41306e0 2022-06-29T15:47:35.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.471+0000 7f72e37fe700 1 -- 172.21.15.47:0/3699118671 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f72f001a030 con 0x7f72f41306e0 2022-06-29T15:47:35.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.475+0000 7f72e37fe700 1 --2- 172.21.15.47:0/3699118671 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f72d4053b00 0x7f72d4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:35.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.475+0000 7f72e37fe700 1 -- 172.21.15.47:0/3699118671 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f72f006ba60 con 0x7f72f41306e0 2022-06-29T15:47:35.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.475+0000 7f72fa48a700 1 --2- 172.21.15.47:0/3699118671 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f72d4053b00 0x7f72d4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:35.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.475+0000 7f72fa48a700 1 --2- 172.21.15.47:0/3699118671 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f72d4053b00 0x7f72d4055fc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7f72e400d000 tx=0x7f72e4007040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:35.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.483+0000 7f72e37fe700 1 -- 172.21.15.47:0/3699118671 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f72f00428d0 con 0x7f72f41306e0 2022-06-29T15:47:35.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.603+0000 7f72fc6ee700 1 -- 172.21.15.47:0/3699118671 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f72f40573d0 con 0x7f72f41306e0 2022-06-29T15:47:35.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.603+0000 7f72e37fe700 1 -- 172.21.15.47:0/3699118671 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+16287 (secure 0 0 0) 0x7f72f0025fb0 con 0x7f72f41306e0 2022-06-29T15:47:35.607 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:47:35.608 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":67,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:47:35.340057+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:47:33.782330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2022-06-29T15:47:34.614730+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:47:35.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.607+0000 7f72fc6ee700 1 -- 172.21.15.47:0/3699118671 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f72d4053b00 msgr2=0x7f72d4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:35.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.607+0000 7f72fc6ee700 1 --2- 172.21.15.47:0/3699118671 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f72d4053b00 0x7f72d4055fc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7f72e400d000 tx=0x7f72e4007040).stop 2022-06-29T15:47:35.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.607+0000 7f72fc6ee700 1 -- 172.21.15.47:0/3699118671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72f41306e0 msgr2=0x7f72f412fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:35.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.607+0000 7f72fc6ee700 1 --2- 172.21.15.47:0/3699118671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72f41306e0 0x7f72f412fd40 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7f72f000d000 tx=0x7f72f0007a00).stop 2022-06-29T15:47:35.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.607+0000 7f72fc6ee700 1 -- 172.21.15.47:0/3699118671 shutdown_connections 2022-06-29T15:47:35.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.607+0000 7f72fc6ee700 1 --2- 172.21.15.47:0/3699118671 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f72d4053b00 0x7f72d4055fc0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:35.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.607+0000 7f72fc6ee700 1 --2- 172.21.15.47:0/3699118671 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72f4130230 0x7f72f40b1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:35.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.607+0000 7f72fc6ee700 1 --2- 172.21.15.47:0/3699118671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72f41306e0 0x7f72f412fd40 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:35.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.607+0000 7f72fc6ee700 1 --2- 172.21.15.47:0/3699118671 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f72dc004ec0 0x7f72f40b54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:35.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.607+0000 7f72fc6ee700 1 -- 172.21.15.47:0/3699118671 >> 172.21.15.47:0/3699118671 conn(0x7f72f405f0b0 msgr2=0x7f72f405bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:35.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.607+0000 7f72fc6ee700 1 -- 172.21.15.47:0/3699118671 shutdown_connections 2022-06-29T15:47:35.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.607+0000 7f72fc6ee700 1 -- 172.21.15.47:0/3699118671 wait complete. 2022-06-29T15:47:35.620 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:47:35.723 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.719+0000 7fb70bfff700 1 Processor -- start 2022-06-29T15:47:35.723 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.719+0000 7fb70bfff700 1 -- start start 2022-06-29T15:47:35.725 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.723+0000 7fb70bfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb70c05c510 0x7fb70c143bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:35.725 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.723+0000 7fb70bfff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb70c05c8f0 0x7fb70c0b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:35.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.723+0000 7fb70bfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb70c0b2c40 0x7fb70c0b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:35.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.723+0000 7fb70bfff700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb70c142f90 con 0x7fb70c0b2c40 2022-06-29T15:47:35.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.723+0000 7fb70bfff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb70c1431f0 con 0x7fb70c05c510 2022-06-29T15:47:35.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.723+0000 7fb70bfff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb70c143490 con 0x7fb70c05c8f0 2022-06-29T15:47:35.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.723+0000 7fb70b7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb70c0b2c40 0x7fb70c0b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:35.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.723+0000 7fb70b7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb70c0b2c40 0x7fb70c0b30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49302/0 (socket says 172.21.15.47:49302) 2022-06-29T15:47:35.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.723+0000 7fb70b7fe700 1 -- 172.21.15.47:0/1939549514 learned_addr learned my addr 172.21.15.47:0/1939549514 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:35.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.723+0000 7fb703fff700 1 --2- 172.21.15.47:0/1939549514 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb70c05c8f0 0x7fb70c0b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:35.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.723+0000 7fb70affd700 1 --2- 172.21.15.47:0/1939549514 >> v2:172.21.15.64:3300/0 conn(0x7fb70c05c510 0x7fb70c143bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:35.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.723+0000 7fb70b7fe700 1 -- 172.21.15.47:0/1939549514 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb70c05c8f0 msgr2=0x7fb70c0b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:35.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.723+0000 7fb70b7fe700 1 --2- 172.21.15.47:0/1939549514 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb70c05c8f0 0x7fb70c0b4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:35.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.723+0000 7fb70b7fe700 1 -- 172.21.15.47:0/1939549514 >> v2:172.21.15.64:3300/0 conn(0x7fb70c05c510 msgr2=0x7fb70c143bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:35.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.723+0000 7fb70b7fe700 1 --2- 172.21.15.47:0/1939549514 >> v2:172.21.15.64:3300/0 conn(0x7fb70c05c510 0x7fb70c143bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:35.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.723+0000 7fb70b7fe700 1 -- 172.21.15.47:0/1939549514 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb70c143730 con 0x7fb70c0b2c40 2022-06-29T15:47:35.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.723+0000 7fb70affd700 1 --2- 172.21.15.47:0/1939549514 >> v2:172.21.15.64:3300/0 conn(0x7fb70c05c510 0x7fb70c143bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:47:35.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.723+0000 7fb70b7fe700 1 --2- 172.21.15.47:0/1939549514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb70c0b2c40 0x7fb70c0b30d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7fb7040032f0 tx=0x7fb704009290).ready entity=mon.0 client_cookie=a230526f9c299649 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:35.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.723+0000 7fb70a7fc700 1 -- 172.21.15.47:0/1939549514 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb70400b870 con 0x7fb70c0b2c40 2022-06-29T15:47:35.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.723+0000 7fb70a7fc700 1 -- 172.21.15.47:0/1939549514 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb70400b9d0 con 0x7fb70c0b2c40 2022-06-29T15:47:35.731 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.723+0000 7fb70a7fc700 1 -- 172.21.15.47:0/1939549514 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb70400bb40 con 0x7fb70c0b2c40 2022-06-29T15:47:35.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.723+0000 7fb70bfff700 1 -- 172.21.15.47:0/1939549514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb70c0b2c40 msgr2=0x7fb70c0b30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:35.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.723+0000 7fb70bfff700 1 --2- 172.21.15.47:0/1939549514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb70c0b2c40 0x7fb70c0b30d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7fb7040032f0 tx=0x7fb704009290).stop 2022-06-29T15:47:35.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.723+0000 7fb70bfff700 1 -- 172.21.15.47:0/1939549514 shutdown_connections 2022-06-29T15:47:35.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.723+0000 7fb70bfff700 1 --2- 172.21.15.47:0/1939549514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb70c0b2c40 0x7fb70c0b30d0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:35.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.723+0000 7fb70bfff700 1 --2- 172.21.15.47:0/1939549514 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb70c05c8f0 0x7fb70c0b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:35.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.723+0000 7fb70bfff700 1 --2- 172.21.15.47:0/1939549514 >> v2:172.21.15.64:3300/0 conn(0x7fb70c05c510 0x7fb70c143bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:35.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.723+0000 7fb70bfff700 1 -- 172.21.15.47:0/1939549514 >> 172.21.15.47:0/1939549514 conn(0x7fb70c060e00 msgr2=0x7fb70c0611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:35.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.723+0000 7fb70bfff700 1 -- 172.21.15.47:0/1939549514 shutdown_connections 2022-06-29T15:47:35.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.723+0000 7fb70bfff700 1 -- 172.21.15.47:0/1939549514 wait complete. 2022-06-29T15:47:35.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.727+0000 7fb70bfff700 1 Processor -- start 2022-06-29T15:47:35.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.727+0000 7fb70bfff700 1 -- start start 2022-06-29T15:47:35.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.727+0000 7fb70bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb70c05c510 0x7fb70c0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:35.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.727+0000 7fb70bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb70c05c8f0 0x7fb70c0b7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:35.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.727+0000 7fb70bfff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb70c0b2c40 0x7fb70c140e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:35.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.727+0000 7fb70bfff700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb70c0ad670 con 0x7fb70c05c8f0 2022-06-29T15:47:35.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.727+0000 7fb70bfff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb70c0aede0 con 0x7fb70c05c510 2022-06-29T15:47:35.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.727+0000 7fb70bfff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb70c0ad8a0 con 0x7fb70c0b2c40 2022-06-29T15:47:35.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.727+0000 7fb703fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb70c05c8f0 0x7fb70c0b7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:35.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.727+0000 7fb703fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb70c05c8f0 0x7fb70c0b7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49306/0 (socket says 172.21.15.47:49306) 2022-06-29T15:47:35.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.727+0000 7fb703fff700 1 -- 172.21.15.47:0/370170665 learned_addr learned my addr 172.21.15.47:0/370170665 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:35.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.727+0000 7fb70b7fe700 1 --2- 172.21.15.47:0/370170665 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb70c0b2c40 0x7fb70c140e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:35.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.727+0000 7fb70affd700 1 --2- 172.21.15.47:0/370170665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb70c05c510 0x7fb70c0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:35.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.727+0000 7fb703fff700 1 -- 172.21.15.47:0/370170665 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb70c0b2c40 msgr2=0x7fb70c140e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:35.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.727+0000 7fb703fff700 1 --2- 172.21.15.47:0/370170665 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb70c0b2c40 0x7fb70c140e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:35.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.727+0000 7fb703fff700 1 -- 172.21.15.47:0/370170665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb70c05c510 msgr2=0x7fb70c0b4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:35.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.727+0000 7fb703fff700 1 --2- 172.21.15.47:0/370170665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb70c05c510 0x7fb70c0b4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:35.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.727+0000 7fb703fff700 1 -- 172.21.15.47:0/370170665 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb704009530 con 0x7fb70c05c8f0 2022-06-29T15:47:35.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.727+0000 7fb703fff700 1 --2- 172.21.15.47:0/370170665 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb70c05c8f0 0x7fb70c0b7b70 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7fb6f400c000 tx=0x7fb6f4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:35.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.727+0000 7fb708ff9700 1 -- 172.21.15.47:0/370170665 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6f4010780 con 0x7fb70c05c8f0 2022-06-29T15:47:35.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.731+0000 7fb70bfff700 1 -- 172.21.15.47:0/370170665 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb70c0af040 con 0x7fb70c05c8f0 2022-06-29T15:47:35.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.731+0000 7fb708ff9700 1 -- 172.21.15.47:0/370170665 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6f400b3b0 con 0x7fb70c05c8f0 2022-06-29T15:47:35.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.731+0000 7fb708ff9700 1 -- 172.21.15.47:0/370170665 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6f4010ad0 con 0x7fb70c05c8f0 2022-06-29T15:47:35.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.731+0000 7fb70bfff700 1 -- 172.21.15.47:0/370170665 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb70c1422f0 con 0x7fb70c05c8f0 2022-06-29T15:47:35.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.731+0000 7fb708ff9700 1 -- 172.21.15.47:0/370170665 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb6f4010c30 con 0x7fb70c05c8f0 2022-06-29T15:47:35.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.731+0000 7fb70bfff700 1 -- 172.21.15.47:0/370170665 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb70c0ada60 con 0x7fb70c05c8f0 2022-06-29T15:47:35.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.731+0000 7fb708ff9700 1 --2- 172.21.15.47:0/370170665 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb6f0053aa0 0x7fb6f0055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:35.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.731+0000 7fb70affd700 1 --2- 172.21.15.47:0/370170665 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb6f0053aa0 0x7fb6f0055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:35.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.731+0000 7fb708ff9700 1 -- 172.21.15.47:0/370170665 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb6f406c5e0 con 0x7fb70c05c8f0 2022-06-29T15:47:35.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.731+0000 7fb70affd700 1 --2- 172.21.15.47:0/370170665 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb6f0053aa0 0x7fb6f0055f60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7fb6fc00c000 tx=0x7fb6fc008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:35.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.739+0000 7fb708ff9700 1 -- 172.21.15.47:0/370170665 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb6f4043210 con 0x7fb70c05c8f0 2022-06-29T15:47:35.872 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.867+0000 7fb70bfff700 1 -- 172.21.15.47:0/370170665 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb70c0573d0 con 0x7fb70c05c8f0 2022-06-29T15:47:35.872 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.871+0000 7fb708ff9700 1 -- 172.21.15.47:0/370170665 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+16287 (secure 0 0 0) 0x7fb6f4046830 con 0x7fb70c05c8f0 2022-06-29T15:47:35.873 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:47:35.873 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":67,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:47:35.340057+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:47:33.782330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2022-06-29T15:47:34.614730+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:47:35.874 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.871+0000 7fb70bfff700 1 -- 172.21.15.47:0/370170665 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb6f0053aa0 msgr2=0x7fb6f0055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:35.875 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.871+0000 7fb70bfff700 1 --2- 172.21.15.47:0/370170665 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb6f0053aa0 0x7fb6f0055f60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7fb6fc00c000 tx=0x7fb6fc008040).stop 2022-06-29T15:47:35.875 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.871+0000 7fb70bfff700 1 -- 172.21.15.47:0/370170665 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb70c05c8f0 msgr2=0x7fb70c0b7b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:35.875 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.871+0000 7fb70bfff700 1 --2- 172.21.15.47:0/370170665 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb70c05c8f0 0x7fb70c0b7b70 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7fb6f400c000 tx=0x7fb6f4006df0).stop 2022-06-29T15:47:35.875 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.871+0000 7fb70bfff700 1 -- 172.21.15.47:0/370170665 shutdown_connections 2022-06-29T15:47:35.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.871+0000 7fb70bfff700 1 --2- 172.21.15.47:0/370170665 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb70c0b2c40 0x7fb70c140e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:35.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.871+0000 7fb70bfff700 1 --2- 172.21.15.47:0/370170665 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb70c05c8f0 0x7fb70c0b7b70 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:35.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.871+0000 7fb70bfff700 1 --2- 172.21.15.47:0/370170665 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb6f0053aa0 0x7fb6f0055f60 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:35.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.871+0000 7fb70bfff700 1 --2- 172.21.15.47:0/370170665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb70c05c510 0x7fb70c0b4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:35.877 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.871+0000 7fb70bfff700 1 -- 172.21.15.47:0/370170665 >> 172.21.15.47:0/370170665 conn(0x7fb70c060e00 msgr2=0x7fb70c0611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:35.877 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.871+0000 7fb70bfff700 1 -- 172.21.15.47:0/370170665 shutdown_connections 2022-06-29T15:47:35.877 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.871+0000 7fb70bfff700 1 -- 172.21.15.47:0/370170665 wait complete. 2022-06-29T15:47:35.885 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-06-29T15:47:35.994 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.991+0000 7f8923469700 1 Processor -- start 2022-06-29T15:47:35.994 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.991+0000 7f8923469700 1 -- start start 2022-06-29T15:47:35.996 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.995+0000 7f8923469700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f891c0acfd0 0x7f891c0ad3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:35.997 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.995+0000 7f8923469700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f891c0b1bb0 0x7f891c141640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:35.997 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.995+0000 7f8923469700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f891c09a3a0 0x7f891c0b23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:35.997 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.995+0000 7f8923469700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f891c13eaa0 con 0x7f891c0b1bb0 2022-06-29T15:47:35.997 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.995+0000 7f8923469700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f891c140f40 con 0x7f891c0acfd0 2022-06-29T15:47:35.998 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.995+0000 7f8923469700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f891c0a96a0 con 0x7f891c09a3a0 2022-06-29T15:47:35.998 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.995+0000 7f8921c66700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f891c0b1bb0 0x7f891c141640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:35.998 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.995+0000 7f8921c66700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f891c0b1bb0 0x7f891c141640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49316/0 (socket says 172.21.15.47:49316) 2022-06-29T15:47:35.998 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.995+0000 7f8921c66700 1 -- 172.21.15.47:0/2852886102 learned_addr learned my addr 172.21.15.47:0/2852886102 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:35.999 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.995+0000 7f8922c68700 1 --2- 172.21.15.47:0/2852886102 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f891c09a3a0 0x7f891c0b23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:35.999 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.995+0000 7f8922467700 1 --2- 172.21.15.47:0/2852886102 >> v2:172.21.15.64:3300/0 conn(0x7f891c0acfd0 0x7f891c0ad3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:35.999 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.995+0000 7f8921c66700 1 -- 172.21.15.47:0/2852886102 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f891c09a3a0 msgr2=0x7f891c0b23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:35.999 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.995+0000 7f8921c66700 1 --2- 172.21.15.47:0/2852886102 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f891c09a3a0 0x7f891c0b23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:36.000 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.995+0000 7f8921c66700 1 -- 172.21.15.47:0/2852886102 >> v2:172.21.15.64:3300/0 conn(0x7f891c0acfd0 msgr2=0x7f891c0ad3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:36.000 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.995+0000 7f8921c66700 1 --2- 172.21.15.47:0/2852886102 >> v2:172.21.15.64:3300/0 conn(0x7f891c0acfd0 0x7f891c0ad3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:36.000 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.995+0000 7f8921c66700 1 -- 172.21.15.47:0/2852886102 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f891c0a9400 con 0x7f891c0b1bb0 2022-06-29T15:47:36.000 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.995+0000 7f8922467700 1 --2- 172.21.15.47:0/2852886102 >> v2:172.21.15.64:3300/0 conn(0x7f891c0acfd0 0x7f891c0ad3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:47:36.000 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.995+0000 7f8921c66700 1 --2- 172.21.15.47:0/2852886102 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f891c0b1bb0 0x7f891c141640 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7f8918002f70 tx=0x7f8918006eb0).ready entity=mon.0 client_cookie=cf64031f2085292e server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:36.001 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.995+0000 7f8921465700 1 -- 172.21.15.47:0/2852886102 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f891800d570 con 0x7f891c0b1bb0 2022-06-29T15:47:36.001 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.995+0000 7f8921465700 1 -- 172.21.15.47:0/2852886102 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8918000e90 con 0x7f891c0b1bb0 2022-06-29T15:47:36.001 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.995+0000 7f8921465700 1 -- 172.21.15.47:0/2852886102 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f891800d900 con 0x7f891c0b1bb0 2022-06-29T15:47:36.005 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.995+0000 7f8923469700 1 -- 172.21.15.47:0/2852886102 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f891c0b1bb0 msgr2=0x7f891c141640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:36.005 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.995+0000 7f8923469700 1 --2- 172.21.15.47:0/2852886102 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f891c0b1bb0 0x7f891c141640 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7f8918002f70 tx=0x7f8918006eb0).stop 2022-06-29T15:47:36.005 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.995+0000 7f8923469700 1 -- 172.21.15.47:0/2852886102 shutdown_connections 2022-06-29T15:47:36.006 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.995+0000 7f8923469700 1 --2- 172.21.15.47:0/2852886102 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f891c09a3a0 0x7f891c0b23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:36.006 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.995+0000 7f8923469700 1 --2- 172.21.15.47:0/2852886102 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f891c0b1bb0 0x7f891c141640 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:36.006 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.995+0000 7f8923469700 1 --2- 172.21.15.47:0/2852886102 >> v2:172.21.15.64:3300/0 conn(0x7f891c0acfd0 0x7f891c0ad3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:36.006 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.995+0000 7f8923469700 1 -- 172.21.15.47:0/2852886102 >> 172.21.15.47:0/2852886102 conn(0x7f891c062df0 msgr2=0x7f891c0631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:36.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.995+0000 7f8923469700 1 -- 172.21.15.47:0/2852886102 shutdown_connections 2022-06-29T15:47:36.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.995+0000 7f8923469700 1 -- 172.21.15.47:0/2852886102 wait complete. 2022-06-29T15:47:36.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.999+0000 7f8923469700 1 Processor -- start 2022-06-29T15:47:36.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.999+0000 7f8923469700 1 -- start start 2022-06-29T15:47:36.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.999+0000 7f8923469700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f891c09a3a0 0x7f891c0b5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.999+0000 7f8923469700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f891c0acfd0 0x7f891c0a9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.999+0000 7f8923469700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f891c0b1bb0 0x7f891c0aa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:36.010 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.999+0000 7f8923469700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891c0b6a90 con 0x7f891c0acfd0 2022-06-29T15:47:36.010 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.999+0000 7f8923469700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891c0b5840 con 0x7f891c09a3a0 2022-06-29T15:47:36.010 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.999+0000 7f8923469700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f891c0b5a70 con 0x7f891c0b1bb0 2022-06-29T15:47:36.011 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.999+0000 7f8921c66700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f891c0acfd0 0x7f891c0a9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:36.011 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.999+0000 7f8921c66700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f891c0acfd0 0x7f891c0a9fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49318/0 (socket says 172.21.15.47:49318) 2022-06-29T15:47:36.012 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:35.999+0000 7f8921c66700 1 -- 172.21.15.47:0/424680515 learned_addr learned my addr 172.21.15.47:0/424680515 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:36.012 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.003+0000 7f8922467700 1 --2- 172.21.15.47:0/424680515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f891c09a3a0 0x7f891c0b5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:36.013 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.003+0000 7f8922c68700 1 --2- 172.21.15.47:0/424680515 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f891c0b1bb0 0x7f891c0aa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:36.013 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.003+0000 7f8921c66700 1 -- 172.21.15.47:0/424680515 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f891c0b1bb0 msgr2=0x7f891c0aa4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:36.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.003+0000 7f8921c66700 1 --2- 172.21.15.47:0/424680515 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f891c0b1bb0 0x7f891c0aa4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:36.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.003+0000 7f8921c66700 1 -- 172.21.15.47:0/424680515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f891c09a3a0 msgr2=0x7f891c0b5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:36.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.003+0000 7f8921c66700 1 --2- 172.21.15.47:0/424680515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f891c09a3a0 0x7f891c0b5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:36.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.003+0000 7f8921c66700 1 -- 172.21.15.47:0/424680515 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8910009470 con 0x7f891c0acfd0 2022-06-29T15:47:36.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.003+0000 7f8922c68700 1 --2- 172.21.15.47:0/424680515 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f891c0b1bb0 0x7f891c0aa4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:47:36.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.003+0000 7f8921c66700 1 --2- 172.21.15.47:0/424680515 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f891c0acfd0 0x7f891c0a9fc0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7f8918002f70 tx=0x7f8918014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:36.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.003+0000 7f890f7fe700 1 -- 172.21.15.47:0/424680515 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8918013e90 con 0x7f891c0acfd0 2022-06-29T15:47:36.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.003+0000 7f890f7fe700 1 -- 172.21.15.47:0/424680515 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89180068b0 con 0x7f891c0acfd0 2022-06-29T15:47:36.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.003+0000 7f8923469700 1 -- 172.21.15.47:0/424680515 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f891800b1e0 con 0x7f891c0acfd0 2022-06-29T15:47:36.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.003+0000 7f890f7fe700 1 -- 172.21.15.47:0/424680515 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f891801e5f0 con 0x7f891c0acfd0 2022-06-29T15:47:36.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.003+0000 7f8923469700 1 -- 172.21.15.47:0/424680515 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f891c0b6240 con 0x7f891c0acfd0 2022-06-29T15:47:36.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.003+0000 7f890f7fe700 1 -- 172.21.15.47:0/424680515 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8918025030 con 0x7f891c0acfd0 2022-06-29T15:47:36.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.007+0000 7f890f7fe700 1 --2- 172.21.15.47:0/424680515 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f88fc053980 0x7f88fc055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:36.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.007+0000 7f890f7fe700 1 -- 172.21.15.47:0/424680515 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f891806cb40 con 0x7f891c0acfd0 2022-06-29T15:47:36.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.007+0000 7f8923469700 1 -- 172.21.15.47:0/424680515 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f891c140d10 con 0x7f891c0acfd0 2022-06-29T15:47:36.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.007+0000 7f8922467700 1 --2- 172.21.15.47:0/424680515 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f88fc053980 0x7f88fc055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:36.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.007+0000 7f8922467700 1 --2- 172.21.15.47:0/424680515 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f88fc053980 0x7f88fc055e40 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f891000b000 tx=0x7f89100091d0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:36.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.015+0000 7f890f7fe700 1 -- 172.21.15.47:0/424680515 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8918023020 con 0x7f891c0acfd0 2022-06-29T15:47:36.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.171+0000 7f8923469700 1 -- 172.21.15.47:0/424680515 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f891c150b50 con 0x7f891c0acfd0 2022-06-29T15:47:36.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.355+0000 7f890f7fe700 1 -- 172.21.15.47:0/424680515 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v30) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8918023020 con 0x7f891c0acfd0 2022-06-29T15:47:36.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.359+0000 7f8923469700 1 -- 172.21.15.47:0/424680515 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f88fc053980 msgr2=0x7f88fc055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:36.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.359+0000 7f8923469700 1 --2- 172.21.15.47:0/424680515 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f88fc053980 0x7f88fc055e40 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f891000b000 tx=0x7f89100091d0).stop 2022-06-29T15:47:36.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.359+0000 7f8923469700 1 -- 172.21.15.47:0/424680515 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f891c0acfd0 msgr2=0x7f891c0a9fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:36.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.359+0000 7f8923469700 1 --2- 172.21.15.47:0/424680515 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f891c0acfd0 0x7f891c0a9fc0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7f8918002f70 tx=0x7f8918014000).stop 2022-06-29T15:47:36.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.359+0000 7f8923469700 1 -- 172.21.15.47:0/424680515 shutdown_connections 2022-06-29T15:47:36.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.359+0000 7f8923469700 1 --2- 172.21.15.47:0/424680515 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f891c0b1bb0 0x7f891c0aa4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:36.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.359+0000 7f8923469700 1 --2- 172.21.15.47:0/424680515 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f891c0acfd0 0x7f891c0a9fc0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:36.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.359+0000 7f8923469700 1 --2- 172.21.15.47:0/424680515 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f88fc053980 0x7f88fc055e40 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:36.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.359+0000 7f8923469700 1 --2- 172.21.15.47:0/424680515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f891c09a3a0 0x7f891c0b5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:36.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.359+0000 7f8923469700 1 -- 172.21.15.47:0/424680515 >> 172.21.15.47:0/424680515 conn(0x7f891c062df0 msgr2=0x7f891c13cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:36.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.359+0000 7f8923469700 1 -- 172.21.15.47:0/424680515 shutdown_connections 2022-06-29T15:47:36.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.359+0000 7f8923469700 1 -- 172.21.15.47:0/424680515 wait complete. 2022-06-29T15:47:36.376 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:47:36.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.483+0000 7f2362f12700 1 Processor -- start 2022-06-29T15:47:36.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.483+0000 7f2362f12700 1 -- start start 2022-06-29T15:47:36.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.483+0000 7f2362f12700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f235c0b2ee0 0x7f235c0b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:36.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.483+0000 7f2362f12700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f235c0ad970 0x7f235c1407e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:36.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.483+0000 7f2362f12700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f235c1400e0 con 0x7f235c0b0e50 2022-06-29T15:47:36.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.483+0000 7f2362f12700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f235c13fef0 con 0x7f235c0ad970 2022-06-29T15:47:36.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.483+0000 7f2362f12700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f235c143810 con 0x7f235c0b2ee0 2022-06-29T15:47:36.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.483+0000 7f236170f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f235c0ad970 0x7f235c1407e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:36.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.487+0000 7f236170f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f235c0ad970 0x7f235c1407e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49326/0 (socket says 172.21.15.47:49326) 2022-06-29T15:47:36.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.487+0000 7f236170f700 1 -- 172.21.15.47:0/2912907162 learned_addr learned my addr 172.21.15.47:0/2912907162 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:36.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.487+0000 7f2361f10700 1 --2- 172.21.15.47:0/2912907162 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f235c0b2ee0 0x7f235c0b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:36.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.487+0000 7f236170f700 1 -- 172.21.15.47:0/2912907162 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f235c0b2ee0 msgr2=0x7f235c0b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:36.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.487+0000 7f236170f700 1 --2- 172.21.15.47:0/2912907162 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f235c0b2ee0 0x7f235c0b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:36.497 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.487+0000 7f236170f700 1 -- 172.21.15.47:0/2912907162 >> v1:172.21.15.64:6789/0 conn(0x7f235c0b0e50 legacy=0x7f235c13d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:36.497 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.487+0000 7f236170f700 1 -- 172.21.15.47:0/2912907162 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f235c143570 con 0x7f235c0ad970 2022-06-29T15:47:36.498 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.487+0000 7f236170f700 1 --2- 172.21.15.47:0/2912907162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f235c0ad970 0x7f235c1407e0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f2358002f70 tx=0x7f2358006eb0).ready entity=mon.0 client_cookie=8c6e22e5f3b42413 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:36.498 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.487+0000 7f2360f0e700 1 -- 172.21.15.47:0/2912907162 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f235800f9c0 con 0x7f235c0ad970 2022-06-29T15:47:36.498 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.487+0000 7f2360f0e700 1 -- 172.21.15.47:0/2912907162 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2358000e90 con 0x7f235c0ad970 2022-06-29T15:47:36.499 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.487+0000 7f2360f0e700 1 -- 172.21.15.47:0/2912907162 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f235800fd50 con 0x7f235c0ad970 2022-06-29T15:47:36.499 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.487+0000 7f2362f12700 1 -- 172.21.15.47:0/2912907162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f235c0ad970 msgr2=0x7f235c1407e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:36.499 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.487+0000 7f2362f12700 1 --2- 172.21.15.47:0/2912907162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f235c0ad970 0x7f235c1407e0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f2358002f70 tx=0x7f2358006eb0).stop 2022-06-29T15:47:36.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.487+0000 7f2362f12700 1 -- 172.21.15.47:0/2912907162 shutdown_connections 2022-06-29T15:47:36.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.487+0000 7f2362f12700 1 --2- 172.21.15.47:0/2912907162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f235c0ad970 0x7f235c1407e0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:36.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.487+0000 7f2362f12700 1 --2- 172.21.15.47:0/2912907162 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f235c0b2ee0 0x7f235c0b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:36.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.487+0000 7f2362f12700 1 -- 172.21.15.47:0/2912907162 >> 172.21.15.47:0/2912907162 conn(0x7f235c063670 msgr2=0x7f235c063a80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:36.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.487+0000 7f2362f12700 1 -- 172.21.15.47:0/2912907162 shutdown_connections 2022-06-29T15:47:36.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.487+0000 7f2362f12700 1 -- 172.21.15.47:0/2912907162 wait complete. 2022-06-29T15:47:36.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.491+0000 7f2362f12700 1 Processor -- start 2022-06-29T15:47:36.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.491+0000 7f2362f12700 1 -- start start 2022-06-29T15:47:36.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.491+0000 7f2362f12700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f235c0ad970 0x7f235c1415d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:36.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.491+0000 7f2362f12700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f235c0b0e50 0x7f235c064350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:36.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.491+0000 7f2362f12700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f235c0b2ee0 0x7f235c064840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:36.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.491+0000 7f2362f12700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235c0a9dd0 con 0x7f235c0b0e50 2022-06-29T15:47:36.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.491+0000 7f2362f12700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235c0acf90 con 0x7f235c0b2ee0 2022-06-29T15:47:36.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.491+0000 7f2362f12700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f235c0b5d40 con 0x7f235c0ad970 2022-06-29T15:47:36.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.491+0000 7f236170f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f235c0b0e50 0x7f235c064350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:36.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.491+0000 7f236170f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f235c0b0e50 0x7f235c064350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49334/0 (socket says 172.21.15.47:49334) 2022-06-29T15:47:36.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.491+0000 7f236170f700 1 -- 172.21.15.47:0/715434219 learned_addr learned my addr 172.21.15.47:0/715434219 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:36.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.491+0000 7f2361f10700 1 --2- 172.21.15.47:0/715434219 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f235c0ad970 0x7f235c1415d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:36.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.491+0000 7f2362711700 1 --2- 172.21.15.47:0/715434219 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f235c0b2ee0 0x7f235c064840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:36.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.491+0000 7f236170f700 1 -- 172.21.15.47:0/715434219 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f235c0ad970 msgr2=0x7f235c1415d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:36.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.491+0000 7f236170f700 1 --2- 172.21.15.47:0/715434219 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f235c0ad970 0x7f235c1415d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:36.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.491+0000 7f236170f700 1 -- 172.21.15.47:0/715434219 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f235c0b2ee0 msgr2=0x7f235c064840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:36.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.491+0000 7f236170f700 1 --2- 172.21.15.47:0/715434219 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f235c0b2ee0 0x7f235c064840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:36.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.491+0000 7f236170f700 1 -- 172.21.15.47:0/715434219 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f235800b1e0 con 0x7f235c0b0e50 2022-06-29T15:47:36.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.491+0000 7f236170f700 1 --2- 172.21.15.47:0/715434219 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f235c0b0e50 0x7f235c064350 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7f2358002f70 tx=0x7f2358017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:36.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.491+0000 7f234affd700 1 -- 172.21.15.47:0/715434219 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23580185f0 con 0x7f235c0b0e50 2022-06-29T15:47:36.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.491+0000 7f234affd700 1 -- 172.21.15.47:0/715434219 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2358018750 con 0x7f235c0b0e50 2022-06-29T15:47:36.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.491+0000 7f2362f12700 1 -- 172.21.15.47:0/715434219 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f235c064d30 con 0x7f235c0b0e50 2022-06-29T15:47:36.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.491+0000 7f234affd700 1 -- 172.21.15.47:0/715434219 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2358018ab0 con 0x7f235c0b0e50 2022-06-29T15:47:36.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.491+0000 7f2362f12700 1 -- 172.21.15.47:0/715434219 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f235c0afe30 con 0x7f235c0b0e50 2022-06-29T15:47:36.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.495+0000 7f2362f12700 1 -- 172.21.15.47:0/715434219 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f235c0ad150 con 0x7f235c0b0e50 2022-06-29T15:47:36.513 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.495+0000 7f234affd700 1 -- 172.21.15.47:0/715434219 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f235802b070 con 0x7f235c0b0e50 2022-06-29T15:47:36.513 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.495+0000 7f234affd700 1 --2- 172.21.15.47:0/715434219 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f233c053950 0x7f233c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:36.513 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.495+0000 7f234affd700 1 -- 172.21.15.47:0/715434219 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f235806d3b0 con 0x7f235c0b0e50 2022-06-29T15:47:36.513 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.495+0000 7f2361f10700 1 --2- 172.21.15.47:0/715434219 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f233c053950 0x7f233c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:36.514 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.495+0000 7f2361f10700 1 --2- 172.21.15.47:0/715434219 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f233c053950 0x7f233c055e10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7f234c00c000 tx=0x7f234c008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:36.514 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.499+0000 7f234affd700 1 -- 172.21.15.47:0/715434219 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2358044060 con 0x7f235c0b0e50 2022-06-29T15:47:36.668 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.667+0000 7f2362f12700 1 -- 172.21.15.47:0/715434219 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f235c0b5f00 con 0x7f235c0b0e50 2022-06-29T15:47:36.669 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.667+0000 7f234affd700 1 -- 172.21.15.47:0/715434219 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+2490 (secure 0 0 0) 0x7f2358047680 con 0x7f235c0b0e50 2022-06-29T15:47:36.669 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:47:36.670 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":30,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5887,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/929350416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":929350416},{"type":"v1","addr":"172.21.15.47:6835","nonce":929350416}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":27},{"gid":5970,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2425218664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2425218664},{"type":"v1","addr":"172.21.15.64:6833","nonce":2425218664}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":27}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:47:35.340236+0000","modified":"2022-06-29T15:47:36.343849+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5973":{"gid":5973,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6837/1166335240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1166335240},{"type":"v1","addr":"172.21.15.47:6837","nonce":1166335240}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2022-06-29T15:47:36.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.671+0000 7f2362f12700 1 -- 172.21.15.47:0/715434219 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f233c053950 msgr2=0x7f233c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:36.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.671+0000 7f2362f12700 1 --2- 172.21.15.47:0/715434219 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f233c053950 0x7f233c055e10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7f234c00c000 tx=0x7f234c008040).stop 2022-06-29T15:47:36.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.671+0000 7f2362f12700 1 -- 172.21.15.47:0/715434219 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f235c0b0e50 msgr2=0x7f235c064350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:36.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.671+0000 7f2362f12700 1 --2- 172.21.15.47:0/715434219 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f235c0b0e50 0x7f235c064350 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7f2358002f70 tx=0x7f2358017000).stop 2022-06-29T15:47:36.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.671+0000 7f2362f12700 1 -- 172.21.15.47:0/715434219 shutdown_connections 2022-06-29T15:47:36.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.671+0000 7f2362f12700 1 --2- 172.21.15.47:0/715434219 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f233c053950 0x7f233c055e10 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:36.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.671+0000 7f2362f12700 1 --2- 172.21.15.47:0/715434219 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f235c0b2ee0 0x7f235c064840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:36.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.671+0000 7f2362f12700 1 --2- 172.21.15.47:0/715434219 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f235c0b0e50 0x7f235c064350 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:36.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.671+0000 7f2362f12700 1 --2- 172.21.15.47:0/715434219 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f235c0ad970 0x7f235c1415d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:36.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.671+0000 7f2362f12700 1 -- 172.21.15.47:0/715434219 >> 172.21.15.47:0/715434219 conn(0x7f235c063670 msgr2=0x7f235c13eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:36.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.671+0000 7f2362f12700 1 -- 172.21.15.47:0/715434219 shutdown_connections 2022-06-29T15:47:36.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.671+0000 7f2362f12700 1 -- 172.21.15.47:0/715434219 wait complete. 2022-06-29T15:47:36.675 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 30 2022-06-29T15:47:36.683 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:47:36.784 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.783+0000 7fb8d2dec700 1 Processor -- start 2022-06-29T15:47:36.785 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.783+0000 7fb8d2dec700 1 -- start start 2022-06-29T15:47:36.785 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.783+0000 7fb8d2dec700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb8cc0b1260 0x7fb8cc05ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:36.785 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.783+0000 7fb8d2dec700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8cc0b0740 con 0x7fb8cc0b2120 2022-06-29T15:47:36.786 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.783+0000 7fb8d2dec700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8cc0b0970 con 0x7fb8cc05f0b0 2022-06-29T15:47:36.786 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.783+0000 7fb8d2dec700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8cc147460 con 0x7fb8cc0b1260 2022-06-29T15:47:36.786 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.783+0000 7fb8d1389700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb8cc0b1260 0x7fb8cc05ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:36.787 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.783+0000 7fb8d0b88700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb8cc0b2120 0x7fb8cc147af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34894/0 (socket says 172.21.15.47:34894) 2022-06-29T15:47:36.787 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.783+0000 7fb8d0b88700 1 -- 172.21.15.47:0/1762274691 learned_addr learned my addr 172.21.15.47:0/1762274691 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:47:36.797 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.783+0000 7fb8d1389700 1 -- 172.21.15.47:0/1762274691 >> v1:172.21.15.64:6789/0 conn(0x7fb8cc05f0b0 legacy=0x7fb8cc05f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:36.797 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.783+0000 7fb8d1389700 1 -- 172.21.15.47:0/1762274691 >> v1:172.21.15.47:6789/0 conn(0x7fb8cc0b2120 legacy=0x7fb8cc147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:36.797 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.783+0000 7fb8d1389700 1 -- 172.21.15.47:0/1762274691 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8cc0b0c10 con 0x7fb8cc0b1260 2022-06-29T15:47:36.797 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.783+0000 7fb8cb7fe700 1 -- 172.21.15.47:0/1762274691 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 229117954 0 0) 0x7fb8cc0b0740 con 0x7fb8cc0b2120 2022-06-29T15:47:36.798 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.783+0000 7fb8d1389700 1 --2- 172.21.15.47:0/1762274691 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8cc0b1260 0x7fb8cc05ffc0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7fb8cc141a70 tx=0x7fb8c40091d0).ready entity=mon.0 client_cookie=aa42dce86e61908b server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:36.798 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.783+0000 7fb8cb7fe700 1 -- 172.21.15.47:0/1762274691 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8c400b820 con 0x7fb8cc0b1260 2022-06-29T15:47:36.798 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.783+0000 7fb8cb7fe700 1 -- 172.21.15.47:0/1762274691 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c400b980 con 0x7fb8cc0b1260 2022-06-29T15:47:36.798 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.783+0000 7fb8cb7fe700 1 -- 172.21.15.47:0/1762274691 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8c400baf0 con 0x7fb8cc0b1260 2022-06-29T15:47:36.799 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.783+0000 7fb8d2dec700 1 -- 172.21.15.47:0/1762274691 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8cc0b1260 msgr2=0x7fb8cc05ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:36.799 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.783+0000 7fb8d2dec700 1 --2- 172.21.15.47:0/1762274691 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8cc0b1260 0x7fb8cc05ffc0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7fb8cc141a70 tx=0x7fb8c40091d0).stop 2022-06-29T15:47:36.799 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.783+0000 7fb8d2dec700 1 -- 172.21.15.47:0/1762274691 shutdown_connections 2022-06-29T15:47:36.799 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.783+0000 7fb8d2dec700 1 --2- 172.21.15.47:0/1762274691 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8cc0b1260 0x7fb8cc05ffc0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:36.799 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.783+0000 7fb8d2dec700 1 -- 172.21.15.47:0/1762274691 >> 172.21.15.47:0/1762274691 conn(0x7fb8cc0626f0 msgr2=0x7fb8cc062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:36.800 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.783+0000 7fb8d2dec700 1 -- 172.21.15.47:0/1762274691 shutdown_connections 2022-06-29T15:47:36.800 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.783+0000 7fb8d2dec700 1 -- 172.21.15.47:0/1762274691 wait complete. 2022-06-29T15:47:36.800 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.787+0000 7fb8d2dec700 1 Processor -- start 2022-06-29T15:47:36.800 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.787+0000 7fb8d2dec700 1 -- start start 2022-06-29T15:47:36.801 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.787+0000 7fb8d2dec700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8cc0b1260 0x7fb8cc149df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:36.801 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.787+0000 7fb8d2dec700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8cc0b2120 0x7fb8cc141ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:36.801 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.787+0000 7fb8d2dec700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb8cc141fa0 0x7fb8cc15a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:36.801 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.787+0000 7fb8d2dec700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8cc0ae640 con 0x7fb8cc0b1260 2022-06-29T15:47:36.801 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.787+0000 7fb8d2dec700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8cc0ae8e0 con 0x7fb8cc0b2120 2022-06-29T15:47:36.802 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.787+0000 7fb8d2dec700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8cc0aeb80 con 0x7fb8cc141fa0 2022-06-29T15:47:36.802 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.787+0000 7fb8d0b88700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8cc0b1260 0x7fb8cc149df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:36.802 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.787+0000 7fb8d0b88700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8cc0b1260 0x7fb8cc149df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49346/0 (socket says 172.21.15.47:49346) 2022-06-29T15:47:36.803 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.787+0000 7fb8d0b88700 1 -- 172.21.15.47:0/3054562834 learned_addr learned my addr 172.21.15.47:0/3054562834 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:36.803 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.787+0000 7fb8d1389700 1 --2- 172.21.15.47:0/3054562834 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb8cc141fa0 0x7fb8cc15a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:36.803 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.787+0000 7fb8cbfff700 1 --2- 172.21.15.47:0/3054562834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8cc0b2120 0x7fb8cc141ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:36.803 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.787+0000 7fb8d0b88700 1 -- 172.21.15.47:0/3054562834 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb8cc141fa0 msgr2=0x7fb8cc15a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:36.804 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.787+0000 7fb8d0b88700 1 --2- 172.21.15.47:0/3054562834 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb8cc141fa0 0x7fb8cc15a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:36.804 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.787+0000 7fb8d0b88700 1 -- 172.21.15.47:0/3054562834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8cc0b2120 msgr2=0x7fb8cc141ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:36.804 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.787+0000 7fb8d0b88700 1 --2- 172.21.15.47:0/3054562834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8cc0b2120 0x7fb8cc141ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:36.804 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.787+0000 7fb8d0b88700 1 -- 172.21.15.47:0/3054562834 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8c4009470 con 0x7fb8cc0b1260 2022-06-29T15:47:36.805 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.787+0000 7fb8d0b88700 1 --2- 172.21.15.47:0/3054562834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8cc0b1260 0x7fb8cc149df0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7fb8c0000bc0 tx=0x7fb8c0007c90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:36.805 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.787+0000 7fb8c9ffb700 1 -- 172.21.15.47:0/3054562834 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8c000ce00 con 0x7fb8cc0b1260 2022-06-29T15:47:36.805 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.787+0000 7fb8c9ffb700 1 -- 172.21.15.47:0/3054562834 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c00070d0 con 0x7fb8cc0b1260 2022-06-29T15:47:36.805 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.787+0000 7fb8d2dec700 1 -- 172.21.15.47:0/3054562834 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8cc0adb50 con 0x7fb8cc0b1260 2022-06-29T15:47:36.806 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.787+0000 7fb8c9ffb700 1 -- 172.21.15.47:0/3054562834 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8c0007430 con 0x7fb8cc0b1260 2022-06-29T15:47:36.806 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.787+0000 7fb8d2dec700 1 -- 172.21.15.47:0/3054562834 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8cc144820 con 0x7fb8cc0b1260 2022-06-29T15:47:36.806 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.791+0000 7fb8d2dec700 1 -- 172.21.15.47:0/3054562834 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8cc0aed80 con 0x7fb8cc0b1260 2022-06-29T15:47:36.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.791+0000 7fb8c9ffb700 1 -- 172.21.15.47:0/3054562834 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb8c001a060 con 0x7fb8cc0b1260 2022-06-29T15:47:36.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.791+0000 7fb8c9ffb700 1 --2- 172.21.15.47:0/3054562834 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb8ac053a70 0x7fb8ac055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:36.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.791+0000 7fb8cbfff700 1 --2- 172.21.15.47:0/3054562834 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb8ac053a70 0x7fb8ac055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:36.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.791+0000 7fb8c9ffb700 1 -- 172.21.15.47:0/3054562834 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb8c006b260 con 0x7fb8cc0b1260 2022-06-29T15:47:36.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.791+0000 7fb8cbfff700 1 --2- 172.21.15.47:0/3054562834 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb8ac053a70 0x7fb8ac055f30 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7fb8bc00d000 tx=0x7fb8bc007b90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:36.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.799+0000 7fb8c9ffb700 1 -- 172.21.15.47:0/3054562834 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb8cc0aed80 con 0x7fb8cc0b1260 2022-06-29T15:47:36.925 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.923+0000 7fb8d2dec700 1 -- 172.21.15.47:0/3054562834 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb8cc0aed80 con 0x7fb8cc0b1260 2022-06-29T15:47:36.926 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.923+0000 7fb8c9ffb700 1 -- 172.21.15.47:0/3054562834 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+16274 (secure 0 0 0) 0x7fb8cc0aed80 con 0x7fb8cc0b1260 2022-06-29T15:47:36.926 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:47:36.927 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":68,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:47:36.342870+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:47:33.782330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2022-06-29T15:47:34.614730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:47:36.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.927+0000 7fb8d2dec700 1 -- 172.21.15.47:0/3054562834 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb8ac053a70 msgr2=0x7fb8ac055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:36.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.927+0000 7fb8d2dec700 1 --2- 172.21.15.47:0/3054562834 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb8ac053a70 0x7fb8ac055f30 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7fb8bc00d000 tx=0x7fb8bc007b90).stop 2022-06-29T15:47:36.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.927+0000 7fb8d2dec700 1 -- 172.21.15.47:0/3054562834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8cc0b1260 msgr2=0x7fb8cc149df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:36.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.927+0000 7fb8d2dec700 1 --2- 172.21.15.47:0/3054562834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8cc0b1260 0x7fb8cc149df0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7fb8c0000bc0 tx=0x7fb8c0007c90).stop 2022-06-29T15:47:36.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.927+0000 7fb8d2dec700 1 -- 172.21.15.47:0/3054562834 shutdown_connections 2022-06-29T15:47:36.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.927+0000 7fb8d2dec700 1 --2- 172.21.15.47:0/3054562834 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb8cc141fa0 0x7fb8cc15a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:36.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.927+0000 7fb8d2dec700 1 --2- 172.21.15.47:0/3054562834 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb8ac053a70 0x7fb8ac055f30 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:36.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.927+0000 7fb8d2dec700 1 --2- 172.21.15.47:0/3054562834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8cc0b2120 0x7fb8cc141ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:36.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.927+0000 7fb8d2dec700 1 --2- 172.21.15.47:0/3054562834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8cc0b1260 0x7fb8cc149df0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:36.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.927+0000 7fb8d2dec700 1 -- 172.21.15.47:0/3054562834 >> 172.21.15.47:0/3054562834 conn(0x7fb8cc0626f0 msgr2=0x7fb8cc0b44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:36.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.927+0000 7fb8d2dec700 1 -- 172.21.15.47:0/3054562834 shutdown_connections 2022-06-29T15:47:36.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:36.927+0000 7fb8d2dec700 1 -- 172.21.15.47:0/3054562834 wait complete. 2022-06-29T15:47:36.941 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-29T15:47:37.064 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.063+0000 7f9247ffa700 1 Processor -- start 2022-06-29T15:47:37.065 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.063+0000 7f9247ffa700 1 -- start start 2022-06-29T15:47:37.067 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.063+0000 7f9247ffa700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9240060ee0 0x7f92400612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:37.067 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.063+0000 7f9247ffa700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92400b4300 0x7f92401471a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:37.067 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.063+0000 7f9247ffa700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9240067610 con 0x7f9240066f70 2022-06-29T15:47:37.067 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.063+0000 7f9247ffa700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92400673a0 con 0x7f9240060ee0 2022-06-29T15:47:37.068 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.063+0000 7f9247ffa700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92401467c0 con 0x7f92400b4300 2022-06-29T15:47:37.068 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.063+0000 7f9246597700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9240066f70 0x7f92401430e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34910/0 (socket says 172.21.15.47:34910) 2022-06-29T15:47:37.068 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.063+0000 7f9245d96700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9240060ee0 0x7f92400612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:37.068 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.063+0000 7f9246597700 1 -- 172.21.15.47:0/496429993 learned_addr learned my addr 172.21.15.47:0/496429993 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:47:37.069 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.063+0000 7f9245595700 1 --2- 172.21.15.47:0/496429993 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92400b4300 0x7f92401471a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:37.069 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.063+0000 7f9244d94700 1 -- 172.21.15.47:0/496429993 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2141240230 0 0) 0x7f9240067610 con 0x7f9240066f70 2022-06-29T15:47:37.069 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.063+0000 7f9244d94700 1 -- 172.21.15.47:0/496429993 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9228003260 con 0x7f9240066f70 2022-06-29T15:47:37.069 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.063+0000 7f9245595700 1 -- 172.21.15.47:0/496429993 >> v2:172.21.15.47:3300/0 conn(0x7f9240060ee0 msgr2=0x7f92400612c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:37.070 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.063+0000 7f9245595700 1 --2- 172.21.15.47:0/496429993 >> v2:172.21.15.47:3300/0 conn(0x7f9240060ee0 0x7f92400612c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:37.070 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.063+0000 7f9245595700 1 -- 172.21.15.47:0/496429993 >> v1:172.21.15.47:6789/0 conn(0x7f9240066f70 legacy=0x7f92401430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:37.070 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.063+0000 7f9245595700 1 -- 172.21.15.47:0/496429993 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9240146a60 con 0x7f92400b4300 2022-06-29T15:47:37.070 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.063+0000 7f9245d96700 1 --2- 172.21.15.47:0/496429993 >> v2:172.21.15.47:3300/0 conn(0x7f9240060ee0 0x7f92400612c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:47:37.071 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.067+0000 7f9245595700 1 --2- 172.21.15.47:0/496429993 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92400b4300 0x7f92401471a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f923c002f70 tx=0x7f923c006df0).ready entity=mon.2 client_cookie=2d6b05fe6501a8f4 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:37.074 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.067+0000 7f9244d94700 1 -- 172.21.15.47:0/496429993 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f923c00d4a0 con 0x7f92400b4300 2022-06-29T15:47:37.074 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.067+0000 7f9244d94700 1 -- 172.21.15.47:0/496429993 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f923c000e90 con 0x7f92400b4300 2022-06-29T15:47:37.075 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.067+0000 7f9244d94700 1 -- 172.21.15.47:0/496429993 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f923c00d7f0 con 0x7f92400b4300 2022-06-29T15:47:37.075 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.067+0000 7f9247ffa700 1 -- 172.21.15.47:0/496429993 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92400b4300 msgr2=0x7f92401471a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:37.076 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.067+0000 7f9247ffa700 1 --2- 172.21.15.47:0/496429993 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92400b4300 0x7f92401471a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f923c002f70 tx=0x7f923c006df0).stop 2022-06-29T15:47:37.076 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.067+0000 7f9247ffa700 1 -- 172.21.15.47:0/496429993 shutdown_connections 2022-06-29T15:47:37.076 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.067+0000 7f9247ffa700 1 --2- 172.21.15.47:0/496429993 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92400b4300 0x7f92401471a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:37.076 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.067+0000 7f9247ffa700 1 --2- 172.21.15.47:0/496429993 >> v2:172.21.15.47:3300/0 conn(0x7f9240060ee0 0x7f92400612c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:37.077 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.067+0000 7f9247ffa700 1 -- 172.21.15.47:0/496429993 >> 172.21.15.47:0/496429993 conn(0x7f924005c510 msgr2=0x7f924005f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:37.077 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.067+0000 7f9247ffa700 1 -- 172.21.15.47:0/496429993 shutdown_connections 2022-06-29T15:47:37.077 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.067+0000 7f9247ffa700 1 -- 172.21.15.47:0/496429993 wait complete. 2022-06-29T15:47:37.078 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.071+0000 7f9247ffa700 1 Processor -- start 2022-06-29T15:47:37.078 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.071+0000 7f9247ffa700 1 -- start start 2022-06-29T15:47:37.078 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.071+0000 7f9247ffa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9240060ee0 0x7f924013b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:37.078 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.071+0000 7f9247ffa700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9240066f70 0x7f92401374a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:37.079 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.071+0000 7f9247ffa700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92400b4300 0x7f9240137990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:37.079 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.071+0000 7f9247ffa700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9240137040 con 0x7f9240066f70 2022-06-29T15:47:37.079 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.071+0000 7f9247ffa700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92401386e0 con 0x7f9240060ee0 2022-06-29T15:47:37.079 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.071+0000 7f9247ffa700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9240138980 con 0x7f92400b4300 2022-06-29T15:47:37.080 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.071+0000 7f9245595700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9240066f70 0x7f92401374a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:37.080 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.071+0000 7f9246597700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92400b4300 0x7f9240137990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:37.080 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.071+0000 7f9245595700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9240066f70 0x7f92401374a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49364/0 (socket says 172.21.15.47:49364) 2022-06-29T15:47:37.080 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.071+0000 7f9245d96700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9240060ee0 0x7f924013b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:37.081 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.071+0000 7f9245595700 1 -- 172.21.15.47:0/589293478 learned_addr learned my addr 172.21.15.47:0/589293478 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:37.081 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.071+0000 7f9245595700 1 -- 172.21.15.47:0/589293478 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92400b4300 msgr2=0x7f9240137990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:37.081 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.071+0000 7f9245595700 1 --2- 172.21.15.47:0/589293478 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92400b4300 0x7f9240137990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:37.082 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.071+0000 7f9245595700 1 -- 172.21.15.47:0/589293478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9240060ee0 msgr2=0x7f924013b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:37.082 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.071+0000 7f9245595700 1 --2- 172.21.15.47:0/589293478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9240060ee0 0x7f924013b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:37.082 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.071+0000 7f9245595700 1 -- 172.21.15.47:0/589293478 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f923400b180 con 0x7f9240066f70 2022-06-29T15:47:37.082 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.071+0000 7f9245595700 1 --2- 172.21.15.47:0/589293478 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9240066f70 0x7f92401374a0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7f923c002f70 tx=0x7f923c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:37.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.071+0000 7f9232ffd700 1 -- 172.21.15.47:0/589293478 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f923c016070 con 0x7f9240066f70 2022-06-29T15:47:37.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.071+0000 7f9232ffd700 1 -- 172.21.15.47:0/589293478 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f923c0137d0 con 0x7f9240066f70 2022-06-29T15:47:37.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.071+0000 7f9247ffa700 1 -- 172.21.15.47:0/589293478 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f923c00b180 con 0x7f9240066f70 2022-06-29T15:47:37.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.071+0000 7f9232ffd700 1 -- 172.21.15.47:0/589293478 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f923c013b30 con 0x7f9240066f70 2022-06-29T15:47:37.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.071+0000 7f9247ffa700 1 -- 172.21.15.47:0/589293478 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9240146460 con 0x7f9240066f70 2022-06-29T15:47:37.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.075+0000 7f9247ffa700 1 -- 172.21.15.47:0/589293478 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92401371b0 con 0x7f9240066f70 2022-06-29T15:47:37.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.075+0000 7f9232ffd700 1 -- 172.21.15.47:0/589293478 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f923c025030 con 0x7f9240066f70 2022-06-29T15:47:37.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.075+0000 7f9232ffd700 1 --2- 172.21.15.47:0/589293478 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9220053aa0 0x7f9220055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:37.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.075+0000 7f9232ffd700 1 -- 172.21.15.47:0/589293478 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f923c06cf70 con 0x7f9240066f70 2022-06-29T15:47:37.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.075+0000 7f9245d96700 1 --2- 172.21.15.47:0/589293478 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9220053aa0 0x7f9220055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:37.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.075+0000 7f9245d96700 1 --2- 172.21.15.47:0/589293478 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9220053aa0 0x7f9220055f60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7f923400b000 tx=0x7f9234006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:37.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.083+0000 7f9232ffd700 1 -- 172.21.15.47:0/589293478 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f923c043d30 con 0x7f9240066f70 2022-06-29T15:47:37.246 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.243+0000 7f9247ffa700 1 -- 172.21.15.47:0/589293478 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f92400573d0 con 0x7f9240066f70 2022-06-29T15:47:37.254 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.251+0000 7f9232ffd700 1 -- 172.21.15.47:0/589293478 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v20) v1 ==== 184+0+0 (secure 0 0 0) 0x7f923c023070 con 0x7f9240066f70 2022-06-29T15:47:37.257 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.255+0000 7f9247ffa700 1 -- 172.21.15.47:0/589293478 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9220053aa0 msgr2=0x7f9220055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:37.257 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.255+0000 7f9247ffa700 1 --2- 172.21.15.47:0/589293478 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9220053aa0 0x7f9220055f60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7f923400b000 tx=0x7f9234006df0).stop 2022-06-29T15:47:37.257 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.255+0000 7f9247ffa700 1 -- 172.21.15.47:0/589293478 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9240066f70 msgr2=0x7f92401374a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:37.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.255+0000 7f9247ffa700 1 --2- 172.21.15.47:0/589293478 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9240066f70 0x7f92401374a0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7f923c002f70 tx=0x7f923c014000).stop 2022-06-29T15:47:37.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.255+0000 7f9247ffa700 1 -- 172.21.15.47:0/589293478 shutdown_connections 2022-06-29T15:47:37.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.255+0000 7f9247ffa700 1 --2- 172.21.15.47:0/589293478 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92400b4300 0x7f9240137990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:37.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.255+0000 7f9247ffa700 1 --2- 172.21.15.47:0/589293478 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9240066f70 0x7f92401374a0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:37.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.255+0000 7f9247ffa700 1 --2- 172.21.15.47:0/589293478 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9220053aa0 0x7f9220055f60 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:37.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.255+0000 7f9247ffa700 1 --2- 172.21.15.47:0/589293478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9240060ee0 0x7f924013b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:37.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.255+0000 7f9247ffa700 1 -- 172.21.15.47:0/589293478 >> 172.21.15.47:0/589293478 conn(0x7f924005c510 msgr2=0x7f92400b8830 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:37.260 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.255+0000 7f9247ffa700 1 -- 172.21.15.47:0/589293478 shutdown_connections 2022-06-29T15:47:37.260 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.255+0000 7f9247ffa700 1 -- 172.21.15.47:0/589293478 wait complete. 2022-06-29T15:47:37.260 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.0 2022-06-29T15:47:37.270 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-29T15:47:37.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.383+0000 7f8ce3bdf700 1 Processor -- start 2022-06-29T15:47:37.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.383+0000 7f8ce3bdf700 1 -- start start 2022-06-29T15:47:37.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.383+0000 7f8ce3bdf700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8cdc067500 0x7f8cdc140f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:37.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.383+0000 7f8ce3bdf700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8cdc13e930 0x7f8cdc066ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:37.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.383+0000 7f8ce3bdf700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cdc05f7c0 con 0x7f8cdc13e550 2022-06-29T15:47:37.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.383+0000 7f8ce3bdf700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cdc0b3a80 con 0x7f8cdc13e930 2022-06-29T15:47:37.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.383+0000 7f8ce3bdf700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cdc0b4ca0 con 0x7f8cdc067500 2022-06-29T15:47:37.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.383+0000 7f8ce217c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8cdc13e930 0x7f8cdc066ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:37.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.383+0000 7f8ce217c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8cdc13e930 0x7f8cdc066ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49370/0 (socket says 172.21.15.47:49370) 2022-06-29T15:47:37.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.383+0000 7f8ce217c700 1 -- 172.21.15.47:0/3401783917 learned_addr learned my addr 172.21.15.47:0/3401783917 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:37.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.383+0000 7f8ce117a700 1 --2- 172.21.15.47:0/3401783917 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8cdc067500 0x7f8cdc140f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:37.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.383+0000 7f8ce217c700 1 -- 172.21.15.47:0/3401783917 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8cdc067500 msgr2=0x7f8cdc140f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:37.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.383+0000 7f8ce217c700 1 --2- 172.21.15.47:0/3401783917 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8cdc067500 0x7f8cdc140f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:37.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.383+0000 7f8ce217c700 1 -- 172.21.15.47:0/3401783917 >> v1:172.21.15.64:6789/0 conn(0x7f8cdc13e550 legacy=0x7f8cdc05f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:37.394 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.383+0000 7f8ce217c700 1 -- 172.21.15.47:0/3401783917 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cdc0b37e0 con 0x7f8cdc13e930 2022-06-29T15:47:37.394 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.383+0000 7f8ce217c700 1 --2- 172.21.15.47:0/3401783917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8cdc13e930 0x7f8cdc066ba0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7f8cdc066a80 tx=0x7f8cd4009290).ready entity=mon.0 client_cookie=84b435ff93bbae05 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:37.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.383+0000 7f8ce0979700 1 -- 172.21.15.47:0/3401783917 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8cd400b860 con 0x7f8cdc13e930 2022-06-29T15:47:37.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.383+0000 7f8ce0979700 1 -- 172.21.15.47:0/3401783917 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cd400b9c0 con 0x7f8cdc13e930 2022-06-29T15:47:37.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.383+0000 7f8ce0979700 1 -- 172.21.15.47:0/3401783917 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8cd400bb70 con 0x7f8cdc13e930 2022-06-29T15:47:37.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.387+0000 7f8ce3bdf700 1 -- 172.21.15.47:0/3401783917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8cdc13e930 msgr2=0x7f8cdc066ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:37.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.387+0000 7f8ce3bdf700 1 --2- 172.21.15.47:0/3401783917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8cdc13e930 0x7f8cdc066ba0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7f8cdc066a80 tx=0x7f8cd4009290).stop 2022-06-29T15:47:37.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.387+0000 7f8ce3bdf700 1 -- 172.21.15.47:0/3401783917 shutdown_connections 2022-06-29T15:47:37.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.387+0000 7f8ce3bdf700 1 --2- 172.21.15.47:0/3401783917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8cdc13e930 0x7f8cdc066ba0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:37.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.387+0000 7f8ce3bdf700 1 --2- 172.21.15.47:0/3401783917 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8cdc067500 0x7f8cdc140f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:37.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.387+0000 7f8ce3bdf700 1 -- 172.21.15.47:0/3401783917 >> 172.21.15.47:0/3401783917 conn(0x7f8cdc0627d0 msgr2=0x7f8cdc062bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:37.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.387+0000 7f8ce3bdf700 1 -- 172.21.15.47:0/3401783917 shutdown_connections 2022-06-29T15:47:37.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.387+0000 7f8ce3bdf700 1 -- 172.21.15.47:0/3401783917 wait complete. 2022-06-29T15:47:37.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.387+0000 7f8ce3bdf700 1 Processor -- start 2022-06-29T15:47:37.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.387+0000 7f8ce3bdf700 1 -- start start 2022-06-29T15:47:37.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.391+0000 7f8ce3bdf700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8cdc067500 0x7f8cdc0ab520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:37.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.391+0000 7f8ce3bdf700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8cdc13e550 0x7f8cdc13ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:37.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.391+0000 7f8ce3bdf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cdc13e930 0x7f8cdc0b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:37.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.391+0000 7f8ce3bdf700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cdc0b7fa0 con 0x7f8cdc13e550 2022-06-29T15:47:37.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.391+0000 7f8ce3bdf700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cdc0b5ac0 con 0x7f8cdc13e930 2022-06-29T15:47:37.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.391+0000 7f8ce3bdf700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cdc0b5d50 con 0x7f8cdc067500 2022-06-29T15:47:37.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.391+0000 7f8ce117a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8cdc13e550 0x7f8cdc13ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:37.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.391+0000 7f8ce117a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8cdc13e550 0x7f8cdc13ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49376/0 (socket says 172.21.15.47:49376) 2022-06-29T15:47:37.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.391+0000 7f8ce117a700 1 -- 172.21.15.47:0/4063682054 learned_addr learned my addr 172.21.15.47:0/4063682054 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:37.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.391+0000 7f8ce217c700 1 --2- 172.21.15.47:0/4063682054 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cdc13e930 0x7f8cdc0b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:37.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.391+0000 7f8ce197b700 1 --2- 172.21.15.47:0/4063682054 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8cdc067500 0x7f8cdc0ab520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:37.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.391+0000 7f8ce117a700 1 -- 172.21.15.47:0/4063682054 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8cdc067500 msgr2=0x7f8cdc0ab520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:37.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.391+0000 7f8ce117a700 1 --2- 172.21.15.47:0/4063682054 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8cdc067500 0x7f8cdc0ab520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:37.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.391+0000 7f8ce117a700 1 -- 172.21.15.47:0/4063682054 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cdc13e930 msgr2=0x7f8cdc0b55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:37.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.391+0000 7f8ce117a700 1 --2- 172.21.15.47:0/4063682054 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cdc13e930 0x7f8cdc0b55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:37.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.391+0000 7f8ce117a700 1 -- 172.21.15.47:0/4063682054 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd4009530 con 0x7f8cdc13e550 2022-06-29T15:47:37.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.391+0000 7f8ce117a700 1 --2- 172.21.15.47:0/4063682054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8cdc13e550 0x7f8cdc13ddf0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7f8ccc00a000 tx=0x7f8ccc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:37.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.391+0000 7f8ccaffd700 1 -- 172.21.15.47:0/4063682054 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ccc011780 con 0x7f8cdc13e550 2022-06-29T15:47:37.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.391+0000 7f8ccaffd700 1 -- 172.21.15.47:0/4063682054 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ccc0118e0 con 0x7f8cdc13e550 2022-06-29T15:47:37.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.391+0000 7f8ce3bdf700 1 -- 172.21.15.47:0/4063682054 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cdc152520 con 0x7f8cdc13e550 2022-06-29T15:47:37.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.391+0000 7f8ccaffd700 1 -- 172.21.15.47:0/4063682054 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ccc011c40 con 0x7f8cdc13e550 2022-06-29T15:47:37.409 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.391+0000 7f8ce3bdf700 1 -- 172.21.15.47:0/4063682054 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cdc0b5ff0 con 0x7f8cdc13e550 2022-06-29T15:47:37.409 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.395+0000 7f8ce3bdf700 1 -- 172.21.15.47:0/4063682054 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cdc0573d0 con 0x7f8cdc13e550 2022-06-29T15:47:37.409 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.395+0000 7f8ccaffd700 1 -- 172.21.15.47:0/4063682054 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8ccc012420 con 0x7f8cdc13e550 2022-06-29T15:47:37.409 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.395+0000 7f8ccaffd700 1 --2- 172.21.15.47:0/4063682054 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8cbc053a70 0x7f8cbc055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:37.409 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.395+0000 7f8ccaffd700 1 -- 172.21.15.47:0/4063682054 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f8ccc06bdf0 con 0x7f8cdc13e550 2022-06-29T15:47:37.410 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.395+0000 7f8ce197b700 1 --2- 172.21.15.47:0/4063682054 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8cbc053a70 0x7f8cbc055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:37.410 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.395+0000 7f8ce197b700 1 --2- 172.21.15.47:0/4063682054 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8cbc053a70 0x7f8cbc055f30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f8cd800d000 tx=0x7f8cd8007b70).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:37.410 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.399+0000 7f8ccaffd700 1 -- 172.21.15.47:0/4063682054 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8ccc042b50 con 0x7f8cdc13e550 2022-06-29T15:47:37.571 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.567+0000 7f8ce3bdf700 1 -- 172.21.15.47:0/4063682054 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f8cdc0b8410 con 0x7f8cdc13e550 2022-06-29T15:47:37.579 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.579+0000 7f8ccaffd700 1 -- 172.21.15.47:0/4063682054 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v21) v1 ==== 184+0+0 (secure 0 0 0) 0x7f8ccc046170 con 0x7f8cdc13e550 2022-06-29T15:47:37.582 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.579+0000 7f8ce3bdf700 1 -- 172.21.15.47:0/4063682054 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8cbc053a70 msgr2=0x7f8cbc055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:37.582 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.579+0000 7f8ce3bdf700 1 --2- 172.21.15.47:0/4063682054 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8cbc053a70 0x7f8cbc055f30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f8cd800d000 tx=0x7f8cd8007b70).stop 2022-06-29T15:47:37.583 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.579+0000 7f8ce3bdf700 1 -- 172.21.15.47:0/4063682054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8cdc13e550 msgr2=0x7f8cdc13ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:37.583 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.579+0000 7f8ce3bdf700 1 --2- 172.21.15.47:0/4063682054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8cdc13e550 0x7f8cdc13ddf0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7f8ccc00a000 tx=0x7f8ccc008040).stop 2022-06-29T15:47:37.583 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.579+0000 7f8ce3bdf700 1 -- 172.21.15.47:0/4063682054 shutdown_connections 2022-06-29T15:47:37.583 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.579+0000 7f8ce3bdf700 1 --2- 172.21.15.47:0/4063682054 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8cbc053a70 0x7f8cbc055f30 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:37.584 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.579+0000 7f8ce3bdf700 1 --2- 172.21.15.47:0/4063682054 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cdc13e930 0x7f8cdc0b55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:37.585 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.579+0000 7f8ce3bdf700 1 --2- 172.21.15.47:0/4063682054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8cdc13e550 0x7f8cdc13ddf0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:37.585 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.579+0000 7f8ce3bdf700 1 --2- 172.21.15.47:0/4063682054 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8cdc067500 0x7f8cdc0ab520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:37.585 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.579+0000 7f8ce3bdf700 1 -- 172.21.15.47:0/4063682054 >> 172.21.15.47:0/4063682054 conn(0x7f8cdc0627d0 msgr2=0x7f8cdc05cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:37.586 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.579+0000 7f8ce3bdf700 1 -- 172.21.15.47:0/4063682054 shutdown_connections 2022-06-29T15:47:37.586 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.579+0000 7f8ce3bdf700 1 -- 172.21.15.47:0/4063682054 wait complete. 2022-06-29T15:47:37.587 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.1 2022-06-29T15:47:37.595 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:47:37.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.691+0000 7efd90e8c700 1 Processor -- start 2022-06-29T15:47:37.699 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.691+0000 7efd90e8c700 1 -- start start 2022-06-29T15:47:37.699 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.695+0000 7efd90e8c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efd8c0630d0 0x7efd8c0b2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:37.699 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.695+0000 7efd90e8c700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd8c05f5c0 con 0x7efd8c0afca0 2022-06-29T15:47:37.700 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.695+0000 7efd90e8c700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd8c0b0840 con 0x7efd8c061d00 2022-06-29T15:47:37.700 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.695+0000 7efd90e8c700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd8c0aeac0 con 0x7efd8c0630d0 2022-06-29T15:47:37.701 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.695+0000 7efd8bfff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7efd8c0afca0 0x7efd8c0b0130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34940/0 (socket says 172.21.15.47:34940) 2022-06-29T15:47:37.701 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.695+0000 7efd8bfff700 1 -- 172.21.15.47:0/3965217633 learned_addr learned my addr 172.21.15.47:0/3965217633 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:47:37.701 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.695+0000 7efd8affd700 1 --2- 172.21.15.47:0/3965217633 >> v2:172.21.15.64:3300/0 conn(0x7efd8c0630d0 0x7efd8c0b2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:37.702 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.695+0000 7efd8a7fc700 1 -- 172.21.15.47:0/3965217633 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1821891752 0 0) 0x7efd8c05f5c0 con 0x7efd8c0afca0 2022-06-29T15:47:37.702 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.695+0000 7efd8a7fc700 1 -- 172.21.15.47:0/3965217633 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd78003260 con 0x7efd8c0afca0 2022-06-29T15:47:37.702 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.695+0000 7efd8affd700 1 -- 172.21.15.47:0/3965217633 >> v1:172.21.15.64:6789/0 conn(0x7efd8c061d00 legacy=0x7efd8c143a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:37.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.695+0000 7efd8affd700 1 -- 172.21.15.47:0/3965217633 >> v1:172.21.15.47:6789/0 conn(0x7efd8c0afca0 legacy=0x7efd8c0b0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:37.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.695+0000 7efd8affd700 1 -- 172.21.15.47:0/3965217633 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd8c05f7f0 con 0x7efd8c0630d0 2022-06-29T15:47:37.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.695+0000 7efd8affd700 1 --2- 172.21.15.47:0/3965217633 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd8c0630d0 0x7efd8c0b2d20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7efd8c05f4a0 tx=0x7efd80006df0).ready entity=mon.1 client_cookie=3193c9bb74f3f8da server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:37.704 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.695+0000 7efd8a7fc700 1 -- 172.21.15.47:0/3965217633 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efd8000d610 con 0x7efd8c0630d0 2022-06-29T15:47:37.704 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.695+0000 7efd8a7fc700 1 -- 172.21.15.47:0/3965217633 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd80000e90 con 0x7efd8c0630d0 2022-06-29T15:47:37.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.695+0000 7efd8a7fc700 1 -- 172.21.15.47:0/3965217633 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efd8000d960 con 0x7efd8c0630d0 2022-06-29T15:47:37.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.695+0000 7efd90e8c700 1 -- 172.21.15.47:0/3965217633 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd8c0630d0 msgr2=0x7efd8c0b2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:37.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.695+0000 7efd90e8c700 1 --2- 172.21.15.47:0/3965217633 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd8c0630d0 0x7efd8c0b2d20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7efd8c05f4a0 tx=0x7efd80006df0).stop 2022-06-29T15:47:37.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.695+0000 7efd90e8c700 1 -- 172.21.15.47:0/3965217633 shutdown_connections 2022-06-29T15:47:37.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.695+0000 7efd90e8c700 1 --2- 172.21.15.47:0/3965217633 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd8c0630d0 0x7efd8c0b2d20 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:37.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.695+0000 7efd90e8c700 1 -- 172.21.15.47:0/3965217633 >> 172.21.15.47:0/3965217633 conn(0x7efd8c063670 msgr2=0x7efd8c063a80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:37.707 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.695+0000 7efd90e8c700 1 -- 172.21.15.47:0/3965217633 shutdown_connections 2022-06-29T15:47:37.707 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.695+0000 7efd90e8c700 1 -- 172.21.15.47:0/3965217633 wait complete. 2022-06-29T15:47:37.712 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.699+0000 7efd90e8c700 1 Processor -- start 2022-06-29T15:47:37.712 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.699+0000 7efd90e8c700 1 -- start start 2022-06-29T15:47:37.713 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.699+0000 7efd90e8c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd8c061d00 0x7efd8c149f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:37.713 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.699+0000 7efd90e8c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd8c0630d0 0x7efd8c066a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:37.713 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.699+0000 7efd90e8c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7efd8c0afca0 0x7efd8c066f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:37.714 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.699+0000 7efd90e8c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd8c13afe0 con 0x7efd8c0630d0 2022-06-29T15:47:37.714 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.699+0000 7efd90e8c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd8c067430 con 0x7efd8c061d00 2022-06-29T15:47:37.714 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.699+0000 7efd90e8c700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd8c13a5c0 con 0x7efd8c0afca0 2022-06-29T15:47:37.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.699+0000 7efd8affd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd8c0630d0 0x7efd8c066a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:37.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.699+0000 7efd8affd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd8c0630d0 0x7efd8c066a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49390/0 (socket says 172.21.15.47:49390) 2022-06-29T15:47:37.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.699+0000 7efd8b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd8c061d00 0x7efd8c149f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:37.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.699+0000 7efd8affd700 1 -- 172.21.15.47:0/191401253 learned_addr learned my addr 172.21.15.47:0/191401253 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:37.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.699+0000 7efd8bfff700 1 --2- 172.21.15.47:0/191401253 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7efd8c0afca0 0x7efd8c066f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:37.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.703+0000 7efd8b7fe700 1 -- 172.21.15.47:0/191401253 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7efd8c0afca0 msgr2=0x7efd8c066f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:37.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.703+0000 7efd8b7fe700 1 --2- 172.21.15.47:0/191401253 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7efd8c0afca0 0x7efd8c066f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:37.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.703+0000 7efd8b7fe700 1 -- 172.21.15.47:0/191401253 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd8c0630d0 msgr2=0x7efd8c066a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:37.718 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.703+0000 7efd8b7fe700 1 --2- 172.21.15.47:0/191401253 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd8c0630d0 0x7efd8c066a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:37.718 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.703+0000 7efd8b7fe700 1 -- 172.21.15.47:0/191401253 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd8000b180 con 0x7efd8c061d00 2022-06-29T15:47:37.719 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.703+0000 7efd8affd700 1 --2- 172.21.15.47:0/191401253 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd8c0630d0 0x7efd8c066a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:47:37.719 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.703+0000 7efd8b7fe700 1 --2- 172.21.15.47:0/191401253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd8c061d00 0x7efd8c149f30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7efd8c13d1f0 tx=0x7efd7c007b50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:37.720 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.703+0000 7efd88ff9700 1 -- 172.21.15.47:0/191401253 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efd7c007030 con 0x7efd8c061d00 2022-06-29T15:47:37.720 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.703+0000 7efd88ff9700 1 -- 172.21.15.47:0/191401253 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd7c007190 con 0x7efd8c061d00 2022-06-29T15:47:37.720 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.703+0000 7efd90e8c700 1 -- 172.21.15.47:0/191401253 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd8c13ca30 con 0x7efd8c061d00 2022-06-29T15:47:37.721 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.703+0000 7efd88ff9700 1 -- 172.21.15.47:0/191401253 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efd7c006510 con 0x7efd8c061d00 2022-06-29T15:47:37.721 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.703+0000 7efd90e8c700 1 -- 172.21.15.47:0/191401253 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd8c13cc90 con 0x7efd8c061d00 2022-06-29T15:47:37.721 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.703+0000 7efd90e8c700 1 -- 172.21.15.47:0/191401253 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd8c13ce50 con 0x7efd8c061d00 2022-06-29T15:47:37.722 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.703+0000 7efd88ff9700 1 -- 172.21.15.47:0/191401253 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7efd7c010030 con 0x7efd8c061d00 2022-06-29T15:47:37.722 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.707+0000 7efd88ff9700 1 --2- 172.21.15.47:0/191401253 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7efd6c053a70 0x7efd6c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:37.722 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.707+0000 7efd88ff9700 1 -- 172.21.15.47:0/191401253 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6420+0+0 (secure 0 0 0) 0x7efd7c06bab0 con 0x7efd8c061d00 2022-06-29T15:47:37.723 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.707+0000 7efd8affd700 1 --2- 172.21.15.47:0/191401253 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7efd6c053a70 0x7efd6c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:37.723 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.707+0000 7efd8affd700 1 --2- 172.21.15.47:0/191401253 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7efd6c053a70 0x7efd6c055f30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7efd8001a000 tx=0x7efd80014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:37.724 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.715+0000 7efd88ff9700 1 -- 172.21.15.47:0/191401253 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7efd7c042870 con 0x7efd8c061d00 2022-06-29T15:47:37.880 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.879+0000 7efd90e8c700 1 -- 172.21.15.47:0/191401253 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efd8c0573d0 con 0x7efd8c061d00 2022-06-29T15:47:37.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.879+0000 7efd88ff9700 1 -- 172.21.15.47:0/191401253 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+2490 (secure 0 0 0) 0x7efd7c045e90 con 0x7efd8c061d00 2022-06-29T15:47:37.882 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:47:37.882 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":30,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5887,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/929350416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":929350416},{"type":"v1","addr":"172.21.15.47:6835","nonce":929350416}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":27},{"gid":5970,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2425218664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2425218664},{"type":"v1","addr":"172.21.15.64:6833","nonce":2425218664}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":27}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:47:35.340236+0000","modified":"2022-06-29T15:47:36.343849+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5973":{"gid":5973,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6837/1166335240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1166335240},{"type":"v1","addr":"172.21.15.47:6837","nonce":1166335240}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2022-06-29T15:47:37.884 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.883+0000 7efd90e8c700 1 -- 172.21.15.47:0/191401253 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7efd6c053a70 msgr2=0x7efd6c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:37.884 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.883+0000 7efd90e8c700 1 --2- 172.21.15.47:0/191401253 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7efd6c053a70 0x7efd6c055f30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7efd8001a000 tx=0x7efd80014000).stop 2022-06-29T15:47:37.884 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.883+0000 7efd90e8c700 1 -- 172.21.15.47:0/191401253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd8c061d00 msgr2=0x7efd8c149f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:37.884 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.883+0000 7efd90e8c700 1 --2- 172.21.15.47:0/191401253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd8c061d00 0x7efd8c149f30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7efd8c13d1f0 tx=0x7efd7c007b50).stop 2022-06-29T15:47:37.885 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.883+0000 7efd90e8c700 1 -- 172.21.15.47:0/191401253 shutdown_connections 2022-06-29T15:47:37.885 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.883+0000 7efd90e8c700 1 --2- 172.21.15.47:0/191401253 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7efd8c0afca0 0x7efd8c066f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:37.885 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.883+0000 7efd90e8c700 1 --2- 172.21.15.47:0/191401253 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd8c0630d0 0x7efd8c066a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:37.885 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.883+0000 7efd90e8c700 1 --2- 172.21.15.47:0/191401253 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7efd6c053a70 0x7efd6c055f30 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:37.885 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.883+0000 7efd90e8c700 1 --2- 172.21.15.47:0/191401253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd8c061d00 0x7efd8c149f30 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:37.886 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.883+0000 7efd90e8c700 1 -- 172.21.15.47:0/191401253 >> 172.21.15.47:0/191401253 conn(0x7efd8c063670 msgr2=0x7efd8c0b1800 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:37.886 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.883+0000 7efd90e8c700 1 -- 172.21.15.47:0/191401253 shutdown_connections 2022-06-29T15:47:37.886 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:37.883+0000 7efd90e8c700 1 -- 172.21.15.47:0/191401253 wait complete. 2022-06-29T15:47:37.887 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 30 2022-06-29T15:47:38.202 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 30, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-29T15:47:35.340236+0000', 'modified': '2022-06-29T15:47:36.343849+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5973}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5973': {'gid': 5973, 'name': 'a', 'rank': 0, 'incarnation': 29, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.47:6837/1166335240', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6836', 'nonce': 1166335240}, {'type': 'v1', 'addr': '172.21.15.47:6837', 'nonce': 1166335240}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-29T15:47:38.203 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-29T15:47:38.206 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-06-29T15:47:38.327 INFO:teuthology.orchestra.run.smithi047.stdout:{"cluster_fsid":"9a68894a-53ce-400e-838b-14eab91b834c","whoami":0,"id":5973,"want_state":"up:active","state":"up:active","rank_uptime":2.9742553539999999,"mdsmap_epoch":30,"osdmap_epoch":68,"osdmap_epoch_barrier":68,"uptime":7.3285818489999999} 2022-06-29T15:47:38.328 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9a68894a-53ce-400e-838b-14eab91b834c", "id": 5973, "mdsmap_epoch": 30, "osdmap_epoch": 68, "osdmap_epoch_barrier": 68, "rank_uptime": 2.974255354, "state": "up:active", "uptime": 7.328581849, "want_state": "up:active", "whoami": 0 } 2022-06-29T15:47:38.329 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:47:38.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.439+0000 7ff26bfff700 1 Processor -- start 2022-06-29T15:47:38.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.439+0000 7ff26bfff700 1 -- start start 2022-06-29T15:47:38.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.439+0000 7ff26bfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff26c05c510 0x7ff26c143bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:38.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.439+0000 7ff26bfff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff26c05c8f0 0x7ff26c0b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:38.446 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.439+0000 7ff26bfff700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff26c142f90 con 0x7ff26c0b2c40 2022-06-29T15:47:38.446 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.439+0000 7ff26bfff700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff26c1431f0 con 0x7ff26c05c510 2022-06-29T15:47:38.446 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.439+0000 7ff26bfff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c143490 con 0x7ff26c05c8f0 2022-06-29T15:47:38.446 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.439+0000 7ff26b7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff26c0b2c40 0x7ff26c13cbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34952/0 (socket says 172.21.15.47:34952) 2022-06-29T15:47:38.447 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.439+0000 7ff26affd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff26c05c510 0x7ff26c143bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:38.447 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.439+0000 7ff26b7fe700 1 -- 172.21.15.47:0/1241259363 learned_addr learned my addr 172.21.15.47:0/1241259363 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:47:38.447 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.443+0000 7ff26a7fc700 1 --2- 172.21.15.47:0/1241259363 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff26c05c8f0 0x7ff26c0b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:38.447 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.443+0000 7ff269ffb700 1 -- 172.21.15.47:0/1241259363 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 796758432 0 0) 0x7ff26c142f90 con 0x7ff26c0b2c40 2022-06-29T15:47:38.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.443+0000 7ff269ffb700 1 -- 172.21.15.47:0/1241259363 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff250003260 con 0x7ff26c0b2c40 2022-06-29T15:47:38.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.443+0000 7ff26a7fc700 1 -- 172.21.15.47:0/1241259363 >> v2:172.21.15.47:3300/0 conn(0x7ff26c05c510 msgr2=0x7ff26c143bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:38.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.443+0000 7ff26a7fc700 1 --2- 172.21.15.47:0/1241259363 >> v2:172.21.15.47:3300/0 conn(0x7ff26c05c510 0x7ff26c143bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:38.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.443+0000 7ff26a7fc700 1 -- 172.21.15.47:0/1241259363 >> v1:172.21.15.47:6789/0 conn(0x7ff26c0b2c40 legacy=0x7ff26c13cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:38.449 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.443+0000 7ff26a7fc700 1 -- 172.21.15.47:0/1241259363 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff26c143730 con 0x7ff26c05c8f0 2022-06-29T15:47:38.449 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.443+0000 7ff26a7fc700 1 --2- 172.21.15.47:0/1241259363 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff26c05c8f0 0x7ff26c0b4100 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7ff260002f70 tx=0x7ff260006df0).ready entity=mon.2 client_cookie=ed78f82b0bf98327 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:38.449 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.443+0000 7ff269ffb700 1 -- 172.21.15.47:0/1241259363 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff26000d4a0 con 0x7ff26c05c8f0 2022-06-29T15:47:38.450 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.443+0000 7ff269ffb700 1 -- 172.21.15.47:0/1241259363 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff260000e90 con 0x7ff26c05c8f0 2022-06-29T15:47:38.450 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.443+0000 7ff269ffb700 1 -- 172.21.15.47:0/1241259363 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff26000d7f0 con 0x7ff26c05c8f0 2022-06-29T15:47:38.450 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.443+0000 7ff26bfff700 1 -- 172.21.15.47:0/1241259363 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff26c05c8f0 msgr2=0x7ff26c0b4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:38.450 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.443+0000 7ff26bfff700 1 --2- 172.21.15.47:0/1241259363 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff26c05c8f0 0x7ff26c0b4100 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7ff260002f70 tx=0x7ff260006df0).stop 2022-06-29T15:47:38.451 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.443+0000 7ff26bfff700 1 -- 172.21.15.47:0/1241259363 shutdown_connections 2022-06-29T15:47:38.451 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.443+0000 7ff26bfff700 1 --2- 172.21.15.47:0/1241259363 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff26c05c8f0 0x7ff26c0b4100 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:38.451 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.443+0000 7ff26bfff700 1 --2- 172.21.15.47:0/1241259363 >> v2:172.21.15.47:3300/0 conn(0x7ff26c05c510 0x7ff26c143bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:38.451 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.443+0000 7ff26bfff700 1 -- 172.21.15.47:0/1241259363 >> 172.21.15.47:0/1241259363 conn(0x7ff26c060e00 msgr2=0x7ff26c0611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:38.452 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.443+0000 7ff26bfff700 1 -- 172.21.15.47:0/1241259363 shutdown_connections 2022-06-29T15:47:38.452 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.443+0000 7ff26bfff700 1 -- 172.21.15.47:0/1241259363 wait complete. 2022-06-29T15:47:38.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.443+0000 7ff26bfff700 1 Processor -- start 2022-06-29T15:47:38.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.443+0000 7ff26bfff700 1 -- start start 2022-06-29T15:47:38.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.447+0000 7ff26bfff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff26c05c510 0x7ff26c1482e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:38.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.447+0000 7ff26bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff26c05c8f0 0x7ff26c14a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:38.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.447+0000 7ff26bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff26c0b2c40 0x7ff26c146c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:38.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.447+0000 7ff26bfff700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c138ff0 con 0x7ff26c05c8f0 2022-06-29T15:47:38.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.447+0000 7ff26bfff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c13c120 con 0x7ff26c0b2c40 2022-06-29T15:47:38.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.447+0000 7ff26bfff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c13c600 con 0x7ff26c05c510 2022-06-29T15:47:38.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.447+0000 7ff26a7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff26c05c8f0 0x7ff26c14a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:38.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.447+0000 7ff26a7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff26c05c8f0 0x7ff26c14a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49406/0 (socket says 172.21.15.47:49406) 2022-06-29T15:47:38.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.447+0000 7ff26a7fc700 1 -- 172.21.15.47:0/2997900199 learned_addr learned my addr 172.21.15.47:0/2997900199 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:38.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.447+0000 7ff26affd700 1 --2- 172.21.15.47:0/2997900199 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff26c05c510 0x7ff26c1482e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:38.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.447+0000 7ff26b7fe700 1 --2- 172.21.15.47:0/2997900199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff26c0b2c40 0x7ff26c146c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:38.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.447+0000 7ff26a7fc700 1 -- 172.21.15.47:0/2997900199 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff26c05c510 msgr2=0x7ff26c1482e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:38.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.447+0000 7ff26a7fc700 1 --2- 172.21.15.47:0/2997900199 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff26c05c510 0x7ff26c1482e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:38.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.447+0000 7ff26a7fc700 1 -- 172.21.15.47:0/2997900199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff26c0b2c40 msgr2=0x7ff26c146c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:38.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.447+0000 7ff26a7fc700 1 --2- 172.21.15.47:0/2997900199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff26c0b2c40 0x7ff26c146c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:38.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.447+0000 7ff26a7fc700 1 -- 172.21.15.47:0/2997900199 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff26000b180 con 0x7ff26c05c8f0 2022-06-29T15:47:38.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.447+0000 7ff26a7fc700 1 --2- 172.21.15.47:0/2997900199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff26c05c8f0 0x7ff26c14a1c0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7ff260002f70 tx=0x7ff260014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:38.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.447+0000 7ff25bfff700 1 -- 172.21.15.47:0/2997900199 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff2600139c0 con 0x7ff26c05c8f0 2022-06-29T15:47:38.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.447+0000 7ff25bfff700 1 -- 172.21.15.47:0/2997900199 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff260013b20 con 0x7ff26c05c8f0 2022-06-29T15:47:38.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.447+0000 7ff26bfff700 1 -- 172.21.15.47:0/2997900199 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff26c139210 con 0x7ff26c05c8f0 2022-06-29T15:47:38.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.447+0000 7ff25bfff700 1 -- 172.21.15.47:0/2997900199 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff260013e80 con 0x7ff26c05c8f0 2022-06-29T15:47:38.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.447+0000 7ff26bfff700 1 -- 172.21.15.47:0/2997900199 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff26c140980 con 0x7ff26c05c8f0 2022-06-29T15:47:38.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.447+0000 7ff26bfff700 1 -- 172.21.15.47:0/2997900199 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff26c149ba0 con 0x7ff26c05c8f0 2022-06-29T15:47:38.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.451+0000 7ff25bfff700 1 -- 172.21.15.47:0/2997900199 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff260025030 con 0x7ff26c05c8f0 2022-06-29T15:47:38.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.451+0000 7ff25bfff700 1 --2- 172.21.15.47:0/2997900199 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff2480539e0 0x7ff248055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:38.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.451+0000 7ff25bfff700 1 -- 172.21.15.47:0/2997900199 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff26006cd80 con 0x7ff26c05c8f0 2022-06-29T15:47:38.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.451+0000 7ff26affd700 1 --2- 172.21.15.47:0/2997900199 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff2480539e0 0x7ff248055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:38.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.451+0000 7ff26affd700 1 --2- 172.21.15.47:0/2997900199 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff2480539e0 0x7ff248055ea0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7ff26400b000 tx=0x7ff2640091d0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:38.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.455+0000 7ff25bfff700 1 -- 172.21.15.47:0/2997900199 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff260043a30 con 0x7ff26c05c8f0 2022-06-29T15:47:38.585 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.583+0000 7ff26bfff700 1 -- 172.21.15.47:0/2997900199 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff26c140b40 con 0x7ff26c05c8f0 2022-06-29T15:47:38.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.587+0000 7ff25bfff700 1 -- 172.21.15.47:0/2997900199 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+16274 (secure 0 0 0) 0x7ff260023020 con 0x7ff26c05c8f0 2022-06-29T15:47:38.587 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:47:38.588 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":68,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:47:36.342870+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:47:33.782330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2022-06-29T15:47:34.614730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:47:38.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.587+0000 7ff26bfff700 1 -- 172.21.15.47:0/2997900199 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff2480539e0 msgr2=0x7ff248055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:38.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.587+0000 7ff26bfff700 1 --2- 172.21.15.47:0/2997900199 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff2480539e0 0x7ff248055ea0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7ff26400b000 tx=0x7ff2640091d0).stop 2022-06-29T15:47:38.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.587+0000 7ff26bfff700 1 -- 172.21.15.47:0/2997900199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff26c05c8f0 msgr2=0x7ff26c14a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:38.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.587+0000 7ff26bfff700 1 --2- 172.21.15.47:0/2997900199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff26c05c8f0 0x7ff26c14a1c0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7ff260002f70 tx=0x7ff260014000).stop 2022-06-29T15:47:38.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.587+0000 7ff26bfff700 1 -- 172.21.15.47:0/2997900199 shutdown_connections 2022-06-29T15:47:38.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.587+0000 7ff26bfff700 1 --2- 172.21.15.47:0/2997900199 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff2480539e0 0x7ff248055ea0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:38.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.587+0000 7ff26bfff700 1 --2- 172.21.15.47:0/2997900199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff26c0b2c40 0x7ff26c146c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:38.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.587+0000 7ff26bfff700 1 --2- 172.21.15.47:0/2997900199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff26c05c8f0 0x7ff26c14a1c0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:38.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.587+0000 7ff26bfff700 1 --2- 172.21.15.47:0/2997900199 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff26c05c510 0x7ff26c1482e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:38.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.587+0000 7ff26bfff700 1 -- 172.21.15.47:0/2997900199 >> 172.21.15.47:0/2997900199 conn(0x7ff26c060e00 msgr2=0x7ff26c1427c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:38.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.587+0000 7ff26bfff700 1 -- 172.21.15.47:0/2997900199 shutdown_connections 2022-06-29T15:47:38.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.587+0000 7ff26bfff700 1 -- 172.21.15.47:0/2997900199 wait complete. 2022-06-29T15:47:38.601 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-29T15:42:16.971431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-29T15:42:25.290884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-29T15:47:33.782330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2022-06-29T15:47:34.614730+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-29T15:47:38.602 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-29T15:47:38.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.711+0000 7fc294fad700 1 Processor -- start 2022-06-29T15:47:38.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.711+0000 7fc294fad700 1 -- start start 2022-06-29T15:47:38.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.715+0000 7fc294fad700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc290061220 0x7fc290140070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:38.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.715+0000 7fc294fad700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc290146720 0x7fc290140560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:38.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.715+0000 7fc294fad700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc290061600 con 0x7fc2900b4320 2022-06-29T15:47:38.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.715+0000 7fc294fad700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc290146bc0 con 0x7fc290146720 2022-06-29T15:47:38.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.715+0000 7fc294fad700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc290146e60 con 0x7fc290061220 2022-06-29T15:47:38.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.715+0000 7fc28dd9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc290146720 0x7fc290140560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:38.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.715+0000 7fc28dd9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc290146720 0x7fc290140560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49410/0 (socket says 172.21.15.47:49410) 2022-06-29T15:47:38.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.715+0000 7fc28dd9b700 1 -- 172.21.15.47:0/1391920009 learned_addr learned my addr 172.21.15.47:0/1391920009 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:38.732 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.715+0000 7fc28ed9d700 1 --1- 172.21.15.47:0/1391920009 >> v1:172.21.15.47:6789/0 conn(0x7fc2900b4320 0x7fc290143230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34966/0 (socket says 172.21.15.47:34966) 2022-06-29T15:47:38.732 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.715+0000 7fc28e59c700 1 --2- 172.21.15.47:0/1391920009 >> v2:172.21.15.64:3300/0 conn(0x7fc290061220 0x7fc290140070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:38.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.715+0000 7fc28dd9b700 1 -- 172.21.15.47:0/1391920009 >> v2:172.21.15.64:3300/0 conn(0x7fc290061220 msgr2=0x7fc290140070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:38.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.715+0000 7fc28dd9b700 1 --2- 172.21.15.47:0/1391920009 >> v2:172.21.15.64:3300/0 conn(0x7fc290061220 0x7fc290140070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:38.734 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.715+0000 7fc28dd9b700 1 -- 172.21.15.47:0/1391920009 >> v1:172.21.15.47:6789/0 conn(0x7fc2900b4320 legacy=0x7fc290143230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:38.734 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.715+0000 7fc28dd9b700 1 -- 172.21.15.47:0/1391920009 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc290147100 con 0x7fc290146720 2022-06-29T15:47:38.734 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.715+0000 7fc28dd9b700 1 --2- 172.21.15.47:0/1391920009 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc290146720 0x7fc290140560 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7fc280002f70 tx=0x7fc280006eb0).ready entity=mon.0 client_cookie=4a436592f476e844 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:38.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.715+0000 7fc28d59a700 1 -- 172.21.15.47:0/1391920009 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc28000d610 con 0x7fc290146720 2022-06-29T15:47:38.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.715+0000 7fc28d59a700 1 -- 172.21.15.47:0/1391920009 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc280000e90 con 0x7fc290146720 2022-06-29T15:47:38.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.715+0000 7fc28d59a700 1 -- 172.21.15.47:0/1391920009 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc28000d9a0 con 0x7fc290146720 2022-06-29T15:47:38.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.715+0000 7fc294fad700 1 -- 172.21.15.47:0/1391920009 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc290146720 msgr2=0x7fc290140560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:38.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.715+0000 7fc294fad700 1 --2- 172.21.15.47:0/1391920009 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc290146720 0x7fc290140560 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7fc280002f70 tx=0x7fc280006eb0).stop 2022-06-29T15:47:38.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.715+0000 7fc294fad700 1 -- 172.21.15.47:0/1391920009 shutdown_connections 2022-06-29T15:47:38.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.715+0000 7fc294fad700 1 --2- 172.21.15.47:0/1391920009 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc290146720 0x7fc290140560 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:38.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.715+0000 7fc294fad700 1 --2- 172.21.15.47:0/1391920009 >> v2:172.21.15.64:3300/0 conn(0x7fc290061220 0x7fc290140070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:38.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.715+0000 7fc294fad700 1 -- 172.21.15.47:0/1391920009 >> 172.21.15.47:0/1391920009 conn(0x7fc29005c6b0 msgr2=0x7fc29005f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:38.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.715+0000 7fc294fad700 1 -- 172.21.15.47:0/1391920009 shutdown_connections 2022-06-29T15:47:38.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.715+0000 7fc294fad700 1 -- 172.21.15.47:0/1391920009 wait complete. 2022-06-29T15:47:38.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.719+0000 7fc294fad700 1 Processor -- start 2022-06-29T15:47:38.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.719+0000 7fc294fad700 1 -- start start 2022-06-29T15:47:38.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.719+0000 7fc294fad700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc290061220 0x7fc2900b8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:38.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.719+0000 7fc294fad700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2900b4320 0x7fc290155cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:38.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.719+0000 7fc294fad700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc290146720 0x7fc2901561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:38.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.719+0000 7fc294fad700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc290156690 con 0x7fc290061220 2022-06-29T15:47:38.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.719+0000 7fc294fad700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2900b64d0 con 0x7fc2900b4320 2022-06-29T15:47:38.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.719+0000 7fc294fad700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2900b6770 con 0x7fc290146720 2022-06-29T15:47:38.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.719+0000 7fc28e59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc290061220 0x7fc2900b8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:38.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.719+0000 7fc28e59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc290061220 0x7fc2900b8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49416/0 (socket says 172.21.15.47:49416) 2022-06-29T15:47:38.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.719+0000 7fc28e59c700 1 -- 172.21.15.47:0/3298703934 learned_addr learned my addr 172.21.15.47:0/3298703934 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:38.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.719+0000 7fc28dd9b700 1 --2- 172.21.15.47:0/3298703934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2900b4320 0x7fc290155cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:38.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.719+0000 7fc28ed9d700 1 --2- 172.21.15.47:0/3298703934 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc290146720 0x7fc2901561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:38.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.719+0000 7fc28e59c700 1 -- 172.21.15.47:0/3298703934 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc290146720 msgr2=0x7fc2901561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:38.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.719+0000 7fc28e59c700 1 --2- 172.21.15.47:0/3298703934 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc290146720 0x7fc2901561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:38.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.719+0000 7fc28e59c700 1 -- 172.21.15.47:0/3298703934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2900b4320 msgr2=0x7fc290155cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:38.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.719+0000 7fc28e59c700 1 --2- 172.21.15.47:0/3298703934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2900b4320 0x7fc290155cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:38.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.719+0000 7fc28e59c700 1 -- 172.21.15.47:0/3298703934 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc28000b1e0 con 0x7fc290061220 2022-06-29T15:47:38.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.719+0000 7fc28e59c700 1 --2- 172.21.15.47:0/3298703934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc290061220 0x7fc2900b8e50 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7fc2780030b0 tx=0x7fc278008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:38.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.719+0000 7fc2877fe700 1 -- 172.21.15.47:0/3298703934 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc278011780 con 0x7fc290061220 2022-06-29T15:47:38.748 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.719+0000 7fc2877fe700 1 -- 172.21.15.47:0/3298703934 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2780118e0 con 0x7fc290061220 2022-06-29T15:47:38.748 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.719+0000 7fc294fad700 1 -- 172.21.15.47:0/3298703934 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2900b6a10 con 0x7fc290061220 2022-06-29T15:47:38.749 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.719+0000 7fc2877fe700 1 -- 172.21.15.47:0/3298703934 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc278011c40 con 0x7fc290061220 2022-06-29T15:47:38.749 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.719+0000 7fc294fad700 1 -- 172.21.15.47:0/3298703934 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2900b5870 con 0x7fc290061220 2022-06-29T15:47:38.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.723+0000 7fc2877fe700 1 -- 172.21.15.47:0/3298703934 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc27801c460 con 0x7fc290061220 2022-06-29T15:47:38.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.723+0000 7fc294fad700 1 -- 172.21.15.47:0/3298703934 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2900573d0 con 0x7fc290061220 2022-06-29T15:47:38.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.723+0000 7fc2877fe700 1 --2- 172.21.15.47:0/3298703934 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc2740539e0 0x7fc274055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:38.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.723+0000 7fc2877fe700 1 -- 172.21.15.47:0/3298703934 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc27806c460 con 0x7fc290061220 2022-06-29T15:47:38.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.723+0000 7fc28dd9b700 1 --2- 172.21.15.47:0/3298703934 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc2740539e0 0x7fc274055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:38.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.723+0000 7fc28dd9b700 1 --2- 172.21.15.47:0/3298703934 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc2740539e0 0x7fc274055ea0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7fc28001c000 tx=0x7fc280015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:38.752 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.727+0000 7fc2877fe700 1 -- 172.21.15.47:0/3298703934 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc278043120 con 0x7fc290061220 2022-06-29T15:47:38.857 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.855+0000 7fc294fad700 1 -- 172.21.15.47:0/3298703934 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fc29006a210 con 0x7fc290061220 2022-06-29T15:47:38.858 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.855+0000 7fc2877fe700 1 -- 172.21.15.47:0/3298703934 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v68) v1 ==== 110+0+10 (secure 0 0 0) 0x7fc278046740 con 0x7fc290061220 2022-06-29T15:47:38.859 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2022-06-29T15:47:38.861 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.859+0000 7fc294fad700 1 -- 172.21.15.47:0/3298703934 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc2740539e0 msgr2=0x7fc274055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:38.861 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.859+0000 7fc294fad700 1 --2- 172.21.15.47:0/3298703934 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc2740539e0 0x7fc274055ea0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7fc28001c000 tx=0x7fc280015000).stop 2022-06-29T15:47:38.861 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.859+0000 7fc294fad700 1 -- 172.21.15.47:0/3298703934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc290061220 msgr2=0x7fc2900b8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:38.861 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.859+0000 7fc294fad700 1 --2- 172.21.15.47:0/3298703934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc290061220 0x7fc2900b8e50 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7fc2780030b0 tx=0x7fc278008040).stop 2022-06-29T15:47:38.862 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.859+0000 7fc294fad700 1 -- 172.21.15.47:0/3298703934 shutdown_connections 2022-06-29T15:47:38.862 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.859+0000 7fc294fad700 1 --2- 172.21.15.47:0/3298703934 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc290146720 0x7fc2901561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:38.862 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.859+0000 7fc294fad700 1 --2- 172.21.15.47:0/3298703934 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc2740539e0 0x7fc274055ea0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:38.862 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.859+0000 7fc294fad700 1 --2- 172.21.15.47:0/3298703934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2900b4320 0x7fc290155cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:38.863 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.859+0000 7fc294fad700 1 --2- 172.21.15.47:0/3298703934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc290061220 0x7fc2900b8e50 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:38.863 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.859+0000 7fc294fad700 1 -- 172.21.15.47:0/3298703934 >> 172.21.15.47:0/3298703934 conn(0x7fc29005c6b0 msgr2=0x7fc2901457d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:38.863 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.859+0000 7fc294fad700 1 -- 172.21.15.47:0/3298703934 shutdown_connections 2022-06-29T15:47:38.863 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.859+0000 7fc294fad700 1 -- 172.21.15.47:0/3298703934 wait complete. 2022-06-29T15:47:38.872 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-29T15:47:38.999 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.983+0000 7fd4d521d700 1 Processor -- start 2022-06-29T15:47:39.000 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.983+0000 7fd4d521d700 1 -- start start 2022-06-29T15:47:39.000 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.983+0000 7fd4d521d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd4d00b5b20 0x7fd4d013d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:39.001 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.983+0000 7fd4d521d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd4d00a9250 0x7fd4d0140160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:39.001 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.983+0000 7fd4d521d700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4d00b0190 con 0x7fd4d00a9c60 2022-06-29T15:47:39.001 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.983+0000 7fd4d521d700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4d00aa570 con 0x7fd4d00b5b20 2022-06-29T15:47:39.002 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.983+0000 7fd4d521d700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4d00b03c0 con 0x7fd4d00a9250 2022-06-29T15:47:39.002 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.983+0000 7fd4cffff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd4d00b5b20 0x7fd4d013d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:39.003 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.983+0000 7fd4cf7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd4d00a9c60 0x7fd4d013d690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34980/0 (socket says 172.21.15.47:34980) 2022-06-29T15:47:39.003 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.983+0000 7fd4cf7fe700 1 -- 172.21.15.47:0/838468617 learned_addr learned my addr 172.21.15.47:0/838468617 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:47:39.004 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.983+0000 7fd4d4a1c700 1 --2- 172.21.15.47:0/838468617 >> v2:172.21.15.64:3300/0 conn(0x7fd4d00a9250 0x7fd4d0140160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:39.004 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.983+0000 7fd4ceffd700 1 -- 172.21.15.47:0/838468617 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1331457559 0 0) 0x7fd4d00b0190 con 0x7fd4d00a9c60 2022-06-29T15:47:39.005 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.983+0000 7fd4ceffd700 1 -- 172.21.15.47:0/838468617 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4bc003260 con 0x7fd4d00a9c60 2022-06-29T15:47:39.005 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.983+0000 7fd4cffff700 1 -- 172.21.15.47:0/838468617 >> v2:172.21.15.64:3300/0 conn(0x7fd4d00a9250 msgr2=0x7fd4d0140160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:39.006 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.983+0000 7fd4cffff700 1 --2- 172.21.15.47:0/838468617 >> v2:172.21.15.64:3300/0 conn(0x7fd4d00a9250 0x7fd4d0140160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:39.006 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.983+0000 7fd4cffff700 1 -- 172.21.15.47:0/838468617 >> v1:172.21.15.47:6789/0 conn(0x7fd4d00a9c60 legacy=0x7fd4d013d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:39.006 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.983+0000 7fd4cffff700 1 -- 172.21.15.47:0/838468617 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d00a9670 con 0x7fd4d00b5b20 2022-06-29T15:47:39.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.983+0000 7fd4cffff700 1 --2- 172.21.15.47:0/838468617 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4d00b5b20 0x7fd4d013d1a0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7fd4c0002f70 tx=0x7fd4c0006df0).ready entity=mon.0 client_cookie=3ed04716bc961561 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:39.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.983+0000 7fd4ceffd700 1 -- 172.21.15.47:0/838468617 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd4c000d4f0 con 0x7fd4d00b5b20 2022-06-29T15:47:39.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.983+0000 7fd4ceffd700 1 -- 172.21.15.47:0/838468617 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4c0000e90 con 0x7fd4d00b5b20 2022-06-29T15:47:39.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.983+0000 7fd4ceffd700 1 -- 172.21.15.47:0/838468617 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd4c000d840 con 0x7fd4d00b5b20 2022-06-29T15:47:39.013 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.983+0000 7fd4d521d700 1 -- 172.21.15.47:0/838468617 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4d00b5b20 msgr2=0x7fd4d013d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:39.013 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.983+0000 7fd4d521d700 1 --2- 172.21.15.47:0/838468617 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4d00b5b20 0x7fd4d013d1a0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7fd4c0002f70 tx=0x7fd4c0006df0).stop 2022-06-29T15:47:39.013 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.983+0000 7fd4d521d700 1 -- 172.21.15.47:0/838468617 shutdown_connections 2022-06-29T15:47:39.013 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.983+0000 7fd4d521d700 1 --2- 172.21.15.47:0/838468617 >> v2:172.21.15.64:3300/0 conn(0x7fd4d00a9250 0x7fd4d0140160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:39.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.983+0000 7fd4d521d700 1 --2- 172.21.15.47:0/838468617 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4d00b5b20 0x7fd4d013d1a0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:39.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.983+0000 7fd4d521d700 1 -- 172.21.15.47:0/838468617 >> 172.21.15.47:0/838468617 conn(0x7fd4d0062560 msgr2=0x7fd4d0062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:39.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.987+0000 7fd4d521d700 1 -- 172.21.15.47:0/838468617 shutdown_connections 2022-06-29T15:47:39.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.987+0000 7fd4d521d700 1 -- 172.21.15.47:0/838468617 wait complete. 2022-06-29T15:47:39.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.987+0000 7fd4d521d700 1 Processor -- start 2022-06-29T15:47:39.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.987+0000 7fd4d521d700 1 -- start start 2022-06-29T15:47:39.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.987+0000 7fd4d521d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd4d00a9250 0x7fd4d013a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:39.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.987+0000 7fd4d521d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4d00a9c60 0x7fd4d014ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:39.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.987+0000 7fd4d521d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4d00b5b20 0x7fd4d00ac6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:39.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.987+0000 7fd4d521d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d014e320 con 0x7fd4d00a9c60 2022-06-29T15:47:39.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.987+0000 7fd4d521d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d01525a0 con 0x7fd4d00b5b20 2022-06-29T15:47:39.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.987+0000 7fd4d521d700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d014e580 con 0x7fd4d00a9250 2022-06-29T15:47:39.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.987+0000 7fd4cf7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4d00a9c60 0x7fd4d014ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:39.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.987+0000 7fd4cf7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4d00a9c60 0x7fd4d014ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49432/0 (socket says 172.21.15.47:49432) 2022-06-29T15:47:39.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.987+0000 7fd4cf7fe700 1 -- 172.21.15.47:0/2024337694 learned_addr learned my addr 172.21.15.47:0/2024337694 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:39.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.987+0000 7fd4cffff700 1 --2- 172.21.15.47:0/2024337694 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd4d00a9250 0x7fd4d013a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:39.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.987+0000 7fd4d4a1c700 1 --2- 172.21.15.47:0/2024337694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4d00b5b20 0x7fd4d00ac6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:39.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.991+0000 7fd4cf7fe700 1 -- 172.21.15.47:0/2024337694 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd4d00a9250 msgr2=0x7fd4d013a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:39.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.991+0000 7fd4cf7fe700 1 --2- 172.21.15.47:0/2024337694 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd4d00a9250 0x7fd4d013a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:39.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.991+0000 7fd4cf7fe700 1 -- 172.21.15.47:0/2024337694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4d00b5b20 msgr2=0x7fd4d00ac6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:39.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.991+0000 7fd4cf7fe700 1 --2- 172.21.15.47:0/2024337694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4d00b5b20 0x7fd4d00ac6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:39.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.991+0000 7fd4cf7fe700 1 -- 172.21.15.47:0/2024337694 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4c000b180 con 0x7fd4d00a9c60 2022-06-29T15:47:39.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.991+0000 7fd4cffff700 1 --2- 172.21.15.47:0/2024337694 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd4d00a9250 0x7fd4d013a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:47:39.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.991+0000 7fd4cf7fe700 1 --2- 172.21.15.47:0/2024337694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4d00a9c60 0x7fd4d014ecc0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7fd4c4001110 tx=0x7fd4c4007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:39.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.991+0000 7fd4cd7fa700 1 -- 172.21.15.47:0/2024337694 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd4c40072e0 con 0x7fd4d00a9c60 2022-06-29T15:47:39.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.991+0000 7fd4cd7fa700 1 -- 172.21.15.47:0/2024337694 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4c4007440 con 0x7fd4d00a9c60 2022-06-29T15:47:39.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.991+0000 7fd4cd7fa700 1 -- 172.21.15.47:0/2024337694 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd4c4005800 con 0x7fd4d00a9c60 2022-06-29T15:47:39.024 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.991+0000 7fd4d521d700 1 -- 172.21.15.47:0/2024337694 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4d014e820 con 0x7fd4d00a9c60 2022-06-29T15:47:39.024 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.991+0000 7fd4d521d700 1 -- 172.21.15.47:0/2024337694 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4d00acb90 con 0x7fd4d00a9c60 2022-06-29T15:47:39.025 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.991+0000 7fd4d521d700 1 -- 172.21.15.47:0/2024337694 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4d014e9b0 con 0x7fd4d00a9c60 2022-06-29T15:47:39.025 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.991+0000 7fd4cd7fa700 1 -- 172.21.15.47:0/2024337694 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd4c4010070 con 0x7fd4d00a9c60 2022-06-29T15:47:39.026 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.995+0000 7fd4cd7fa700 1 --2- 172.21.15.47:0/2024337694 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd4b0053950 0x7fd4b0055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:39.026 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.995+0000 7fd4cd7fa700 1 -- 172.21.15.47:0/2024337694 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd4c406b7a0 con 0x7fd4d00a9c60 2022-06-29T15:47:39.026 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.995+0000 7fd4cffff700 1 --2- 172.21.15.47:0/2024337694 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd4b0053950 0x7fd4b0055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:39.027 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.995+0000 7fd4cffff700 1 --2- 172.21.15.47:0/2024337694 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd4b0053950 0x7fd4b0055e10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7fd4c0018000 tx=0x7fd4c0014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:39.028 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:38.999+0000 7fd4cd7fa700 1 -- 172.21.15.47:0/2024337694 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd4c4042560 con 0x7fd4d00a9c60 2022-06-29T15:47:39.130 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.127+0000 7fd4d521d700 1 -- 172.21.15.47:0/2024337694 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd4d00573d0 con 0x7fd4d00a9c60 2022-06-29T15:47:39.131 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.131+0000 7fd4cd7fa700 1 -- 172.21.15.47:0/2024337694 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v68) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd4c4018020 con 0x7fd4d00a9c60 2022-06-29T15:47:39.132 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:47:39.134 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.131+0000 7fd4d521d700 1 -- 172.21.15.47:0/2024337694 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd4b0053950 msgr2=0x7fd4b0055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:39.134 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.131+0000 7fd4d521d700 1 --2- 172.21.15.47:0/2024337694 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd4b0053950 0x7fd4b0055e10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7fd4c0018000 tx=0x7fd4c0014000).stop 2022-06-29T15:47:39.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.131+0000 7fd4d521d700 1 -- 172.21.15.47:0/2024337694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4d00a9c60 msgr2=0x7fd4d014ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:39.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.131+0000 7fd4d521d700 1 --2- 172.21.15.47:0/2024337694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4d00a9c60 0x7fd4d014ecc0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7fd4c4001110 tx=0x7fd4c4007ad0).stop 2022-06-29T15:47:39.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.131+0000 7fd4d521d700 1 -- 172.21.15.47:0/2024337694 shutdown_connections 2022-06-29T15:47:39.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.131+0000 7fd4d521d700 1 --2- 172.21.15.47:0/2024337694 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd4b0053950 0x7fd4b0055e10 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:39.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.131+0000 7fd4d521d700 1 --2- 172.21.15.47:0/2024337694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4d00b5b20 0x7fd4d00ac6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:39.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.131+0000 7fd4d521d700 1 --2- 172.21.15.47:0/2024337694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4d00a9c60 0x7fd4d014ecc0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:39.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.131+0000 7fd4d521d700 1 --2- 172.21.15.47:0/2024337694 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd4d00a9250 0x7fd4d013a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:39.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.131+0000 7fd4d521d700 1 -- 172.21.15.47:0/2024337694 >> 172.21.15.47:0/2024337694 conn(0x7fd4d0062560 msgr2=0x7fd4d0144ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:39.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.131+0000 7fd4d521d700 1 -- 172.21.15.47:0/2024337694 shutdown_connections 2022-06-29T15:47:39.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.131+0000 7fd4d521d700 1 -- 172.21.15.47:0/2024337694 wait complete. 2022-06-29T15:47:39.146 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-29T15:47:39.265 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.263+0000 7fc5d666f700 1 Processor -- start 2022-06-29T15:47:39.265 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.263+0000 7fc5d666f700 1 -- start start 2022-06-29T15:47:39.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.263+0000 7fc5d666f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc5d0060ee0 0x7fc5d00612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:39.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.263+0000 7fc5d666f700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc5d01471a0 0x7fc5d0066f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:39.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.263+0000 7fc5d666f700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5d0067610 con 0x7fc5d00b4300 2022-06-29T15:47:39.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.263+0000 7fc5d666f700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5d01467c0 con 0x7fc5d0060ee0 2022-06-29T15:47:39.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.263+0000 7fc5d666f700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d0146a60 con 0x7fc5d01471a0 2022-06-29T15:47:39.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.267+0000 7fc5d4c0c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc5d01471a0 0x7fc5d0066f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:39.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.267+0000 7fc5cffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc5d0060ee0 0x7fc5d00612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:39.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.267+0000 7fc5d4c0c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc5d01471a0 0x7fc5d0066f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:46544/0 (socket says 172.21.15.47:46544) 2022-06-29T15:47:39.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.267+0000 7fc5d4c0c700 1 -- 172.21.15.47:0/748163588 learned_addr learned my addr 172.21.15.47:0/748163588 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:39.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.267+0000 7fc5c77fe700 1 --1- 172.21.15.47:0/748163588 >> v1:172.21.15.64:6789/0 conn(0x7fc5d00b4300 0x7fc5d01430e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:42558/0 (socket says 172.21.15.47:42558) 2022-06-29T15:47:39.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.267+0000 7fc5d4c0c700 1 -- 172.21.15.47:0/748163588 >> v2:172.21.15.64:3300/0 conn(0x7fc5d0060ee0 msgr2=0x7fc5d00612c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:39.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.267+0000 7fc5d4c0c700 1 --2- 172.21.15.47:0/748163588 >> v2:172.21.15.64:3300/0 conn(0x7fc5d0060ee0 0x7fc5d00612c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:39.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.267+0000 7fc5d4c0c700 1 -- 172.21.15.47:0/748163588 >> v1:172.21.15.64:6789/0 conn(0x7fc5d00b4300 legacy=0x7fc5d01430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:39.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.267+0000 7fc5d4c0c700 1 -- 172.21.15.47:0/748163588 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d0146d00 con 0x7fc5d01471a0 2022-06-29T15:47:39.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.267+0000 7fc5d4c0c700 1 --2- 172.21.15.47:0/748163588 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc5d01471a0 0x7fc5d0066f70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fc5d00ae790 tx=0x7fc5c8009290).ready entity=mon.2 client_cookie=6ec42128e1ab4ab7 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:39.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.267+0000 7fc5cf7fe700 1 -- 172.21.15.47:0/748163588 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc5c800b860 con 0x7fc5d01471a0 2022-06-29T15:47:39.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.267+0000 7fc5cf7fe700 1 -- 172.21.15.47:0/748163588 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5c800b9c0 con 0x7fc5d01471a0 2022-06-29T15:47:39.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.267+0000 7fc5cf7fe700 1 -- 172.21.15.47:0/748163588 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc5c800bb30 con 0x7fc5d01471a0 2022-06-29T15:47:39.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.267+0000 7fc5d666f700 1 -- 172.21.15.47:0/748163588 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc5d01471a0 msgr2=0x7fc5d0066f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:39.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.267+0000 7fc5d666f700 1 --2- 172.21.15.47:0/748163588 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc5d01471a0 0x7fc5d0066f70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fc5d00ae790 tx=0x7fc5c8009290).stop 2022-06-29T15:47:39.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.267+0000 7fc5d666f700 1 -- 172.21.15.47:0/748163588 shutdown_connections 2022-06-29T15:47:39.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.267+0000 7fc5d666f700 1 --2- 172.21.15.47:0/748163588 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc5d01471a0 0x7fc5d0066f70 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:39.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.267+0000 7fc5d666f700 1 --2- 172.21.15.47:0/748163588 >> v2:172.21.15.64:3300/0 conn(0x7fc5d0060ee0 0x7fc5d00612c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:39.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.267+0000 7fc5d666f700 1 -- 172.21.15.47:0/748163588 >> 172.21.15.47:0/748163588 conn(0x7fc5d005c510 msgr2=0x7fc5d005f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:39.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.267+0000 7fc5d666f700 1 -- 172.21.15.47:0/748163588 shutdown_connections 2022-06-29T15:47:39.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.267+0000 7fc5d666f700 1 -- 172.21.15.47:0/748163588 wait complete. 2022-06-29T15:47:39.283 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.271+0000 7fc5d666f700 1 Processor -- start 2022-06-29T15:47:39.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.271+0000 7fc5d666f700 1 -- start start 2022-06-29T15:47:39.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.271+0000 7fc5d666f700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc5d0060ee0 0x7fc5d01363a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:39.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.271+0000 7fc5d666f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5d00b4300 0x7fc5d0156790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:39.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.271+0000 7fc5d666f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5d01471a0 0x7fc5d00b13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:39.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.271+0000 7fc5d666f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d0141270 con 0x7fc5d00b4300 2022-06-29T15:47:39.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.271+0000 7fc5d666f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d00b2220 con 0x7fc5d01471a0 2022-06-29T15:47:39.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.271+0000 7fc5d666f700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d0156c80 con 0x7fc5d0060ee0 2022-06-29T15:47:39.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.271+0000 7fc5c77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5d00b4300 0x7fc5d0156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:39.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.271+0000 7fc5c77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5d00b4300 0x7fc5d0156790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49446/0 (socket says 172.21.15.47:49446) 2022-06-29T15:47:39.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.271+0000 7fc5c77fe700 1 -- 172.21.15.47:0/2549719117 learned_addr learned my addr 172.21.15.47:0/2549719117 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:39.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.271+0000 7fc5cffff700 1 --2- 172.21.15.47:0/2549719117 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc5d0060ee0 0x7fc5d01363a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:39.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.271+0000 7fc5d4c0c700 1 --2- 172.21.15.47:0/2549719117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5d01471a0 0x7fc5d00b13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:39.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.275+0000 7fc5c77fe700 1 -- 172.21.15.47:0/2549719117 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc5d0060ee0 msgr2=0x7fc5d01363a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:39.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.275+0000 7fc5c77fe700 1 --2- 172.21.15.47:0/2549719117 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc5d0060ee0 0x7fc5d01363a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:39.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.275+0000 7fc5c77fe700 1 -- 172.21.15.47:0/2549719117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5d01471a0 msgr2=0x7fc5d00b13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:39.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.275+0000 7fc5c77fe700 1 --2- 172.21.15.47:0/2549719117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5d01471a0 0x7fc5d00b13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:39.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.275+0000 7fc5c77fe700 1 -- 172.21.15.47:0/2549719117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5c8009530 con 0x7fc5d00b4300 2022-06-29T15:47:39.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.275+0000 7fc5c77fe700 1 --2- 172.21.15.47:0/2549719117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5d00b4300 0x7fc5d0156790 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7fc5d00b2360 tx=0x7fc5b8007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:39.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.275+0000 7fc5cdffb700 1 -- 172.21.15.47:0/2549719117 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc5b80070d0 con 0x7fc5d00b4300 2022-06-29T15:47:39.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.275+0000 7fc5cdffb700 1 -- 172.21.15.47:0/2549719117 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5b8007230 con 0x7fc5d00b4300 2022-06-29T15:47:39.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.275+0000 7fc5cdffb700 1 -- 172.21.15.47:0/2549719117 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc5b8005550 con 0x7fc5d00b4300 2022-06-29T15:47:39.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.275+0000 7fc5d666f700 1 -- 172.21.15.47:0/2549719117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5d0140fd0 con 0x7fc5d00b4300 2022-06-29T15:47:39.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.275+0000 7fc5d666f700 1 -- 172.21.15.47:0/2549719117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5d0140a30 con 0x7fc5d00b4300 2022-06-29T15:47:39.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.275+0000 7fc5d666f700 1 -- 172.21.15.47:0/2549719117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5d00573d0 con 0x7fc5d00b4300 2022-06-29T15:47:39.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.275+0000 7fc5cdffb700 1 -- 172.21.15.47:0/2549719117 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc5b801a070 con 0x7fc5d00b4300 2022-06-29T15:47:39.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.279+0000 7fc5cdffb700 1 --2- 172.21.15.47:0/2549719117 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc5a8053a70 0x7fc5a8055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:39.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.279+0000 7fc5cdffb700 1 -- 172.21.15.47:0/2549719117 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc5b806b2c0 con 0x7fc5d00b4300 2022-06-29T15:47:39.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.279+0000 7fc5cffff700 1 --2- 172.21.15.47:0/2549719117 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc5a8053a70 0x7fc5a8055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:39.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.279+0000 7fc5cffff700 1 --2- 172.21.15.47:0/2549719117 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc5a8053a70 0x7fc5a8055f30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7fc5c000b000 tx=0x7fc5c0006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:39.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.283+0000 7fc5cdffb700 1 -- 172.21.15.47:0/2549719117 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc5b8042020 con 0x7fc5d00b4300 2022-06-29T15:47:39.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.411+0000 7fc5d666f700 1 -- 172.21.15.47:0/2549719117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc5d0066a50 con 0x7fc5d00b4300 2022-06-29T15:47:39.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.415+0000 7fc5cdffb700 1 -- 172.21.15.47:0/2549719117 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v68) v1 ==== 104+0+10 (secure 0 0 0) 0x7fc5b8045640 con 0x7fc5d00b4300 2022-06-29T15:47:39.415 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:47:39.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.415+0000 7fc5d666f700 1 -- 172.21.15.47:0/2549719117 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc5a8053a70 msgr2=0x7fc5a8055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:39.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.415+0000 7fc5d666f700 1 --2- 172.21.15.47:0/2549719117 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc5a8053a70 0x7fc5a8055f30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7fc5c000b000 tx=0x7fc5c0006df0).stop 2022-06-29T15:47:39.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.415+0000 7fc5d666f700 1 -- 172.21.15.47:0/2549719117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5d00b4300 msgr2=0x7fc5d0156790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:39.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.415+0000 7fc5d666f700 1 --2- 172.21.15.47:0/2549719117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5d00b4300 0x7fc5d0156790 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7fc5d00b2360 tx=0x7fc5b8007c10).stop 2022-06-29T15:47:39.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.415+0000 7fc5d666f700 1 -- 172.21.15.47:0/2549719117 shutdown_connections 2022-06-29T15:47:39.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.415+0000 7fc5d666f700 1 --2- 172.21.15.47:0/2549719117 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc5a8053a70 0x7fc5a8055f30 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:39.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.415+0000 7fc5d666f700 1 --2- 172.21.15.47:0/2549719117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5d01471a0 0x7fc5d00b13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:39.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.415+0000 7fc5d666f700 1 --2- 172.21.15.47:0/2549719117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5d00b4300 0x7fc5d0156790 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:39.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.415+0000 7fc5d666f700 1 --2- 172.21.15.47:0/2549719117 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc5d0060ee0 0x7fc5d01363a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:39.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.415+0000 7fc5d666f700 1 -- 172.21.15.47:0/2549719117 >> 172.21.15.47:0/2549719117 conn(0x7fc5d005c510 msgr2=0x7fc5d0138ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:39.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.415+0000 7fc5d666f700 1 -- 172.21.15.47:0/2549719117 shutdown_connections 2022-06-29T15:47:39.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.415+0000 7fc5d666f700 1 -- 172.21.15.47:0/2549719117 wait complete. 2022-06-29T15:47:39.429 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-29T15:47:39.533 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.531+0000 7fc706d10700 1 Processor -- start 2022-06-29T15:47:39.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.531+0000 7fc706d10700 1 -- start start 2022-06-29T15:47:39.538 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.535+0000 7fc706d10700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc70005f3f0 0x7fc7001418c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:39.538 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.535+0000 7fc706d10700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc7000aaf00 0x7fc7000b3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:39.538 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.535+0000 7fc706d10700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc70005f7d0 con 0x7fc7000b1320 2022-06-29T15:47:39.539 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.535+0000 7fc706d10700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7000b27a0 con 0x7fc70005f3f0 2022-06-29T15:47:39.539 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.535+0000 7fc706d10700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7000b4b90 con 0x7fc7000aaf00 2022-06-29T15:47:39.539 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.535+0000 7fc70650f700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc7000aaf00 0x7fc7000b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:39.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.535+0000 7fc705d0e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc70005f3f0 0x7fc7001418c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:39.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.535+0000 7fc70650f700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc7000aaf00 0x7fc7000b3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:46556/0 (socket says 172.21.15.47:46556) 2022-06-29T15:47:39.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.535+0000 7fc70650f700 1 -- 172.21.15.47:0/758226183 learned_addr learned my addr 172.21.15.47:0/758226183 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:39.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.535+0000 7fc70650f700 1 -- 172.21.15.47:0/758226183 >> v2:172.21.15.64:3300/0 conn(0x7fc70005f3f0 msgr2=0x7fc7001418c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:39.541 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.535+0000 7fc70650f700 1 --2- 172.21.15.47:0/758226183 >> v2:172.21.15.64:3300/0 conn(0x7fc70005f3f0 0x7fc7001418c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:39.541 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.535+0000 7fc70650f700 1 -- 172.21.15.47:0/758226183 >> v1:172.21.15.64:6789/0 conn(0x7fc7000b1320 legacy=0x7fc7000b2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:39.541 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.535+0000 7fc704d0c700 1 -- 172.21.15.47:0/758226183 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3695437903 0 0) 0x7fc70005f7d0 con 0x7fc7000b1320 2022-06-29T15:47:39.541 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.535+0000 7fc70650f700 1 -- 172.21.15.47:0/758226183 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7000b2060 con 0x7fc7000aaf00 2022-06-29T15:47:39.542 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.535+0000 7fc70650f700 1 --2- 172.21.15.47:0/758226183 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc7000aaf00 0x7fc7000b3700 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7fc6f80032f0 tx=0x7fc6f8009290).ready entity=mon.2 client_cookie=3e18ae2a94431c6d server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:39.542 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.535+0000 7fc704d0c700 1 -- 172.21.15.47:0/758226183 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6f800b8a0 con 0x7fc7000aaf00 2022-06-29T15:47:39.542 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.535+0000 7fc704d0c700 1 -- 172.21.15.47:0/758226183 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6f800ba00 con 0x7fc7000aaf00 2022-06-29T15:47:39.543 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.535+0000 7fc704d0c700 1 -- 172.21.15.47:0/758226183 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6f800bb70 con 0x7fc7000aaf00 2022-06-29T15:47:39.543 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.535+0000 7fc706d10700 1 -- 172.21.15.47:0/758226183 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc7000aaf00 msgr2=0x7fc7000b3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:39.543 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.535+0000 7fc706d10700 1 --2- 172.21.15.47:0/758226183 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc7000aaf00 0x7fc7000b3700 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7fc6f80032f0 tx=0x7fc6f8009290).stop 2022-06-29T15:47:39.543 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.535+0000 7fc706d10700 1 -- 172.21.15.47:0/758226183 shutdown_connections 2022-06-29T15:47:39.544 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.535+0000 7fc706d10700 1 --2- 172.21.15.47:0/758226183 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc7000aaf00 0x7fc7000b3700 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:39.544 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.535+0000 7fc706d10700 1 --2- 172.21.15.47:0/758226183 >> v2:172.21.15.64:3300/0 conn(0x7fc70005f3f0 0x7fc7001418c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:39.544 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.535+0000 7fc706d10700 1 -- 172.21.15.47:0/758226183 >> 172.21.15.47:0/758226183 conn(0x7fc700062570 msgr2=0x7fc7000647d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:39.544 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.535+0000 7fc706d10700 1 -- 172.21.15.47:0/758226183 shutdown_connections 2022-06-29T15:47:39.545 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.535+0000 7fc706d10700 1 -- 172.21.15.47:0/758226183 wait complete. 2022-06-29T15:47:39.548 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.539+0000 7fc706d10700 1 Processor -- start 2022-06-29T15:47:39.548 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.539+0000 7fc706d10700 1 -- start start 2022-06-29T15:47:39.548 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.539+0000 7fc706d10700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc70005f3f0 0x7fc7000b6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:39.548 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.539+0000 7fc706d10700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7000aaf00 0x7fc7000ac600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:39.549 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.539+0000 7fc706d10700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc7000b1320 0x7fc7000acaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:39.549 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.539+0000 7fc706d10700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7000abeb0 con 0x7fc70005f3f0 2022-06-29T15:47:39.549 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.539+0000 7fc706d10700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7000ab4b0 con 0x7fc7000aaf00 2022-06-29T15:47:39.550 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.539+0000 7fc706d10700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc700061480 con 0x7fc7000b1320 2022-06-29T15:47:39.550 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.539+0000 7fc705d0e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc70005f3f0 0x7fc7000b6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:39.550 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.539+0000 7fc705d0e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc70005f3f0 0x7fc7000b6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49458/0 (socket says 172.21.15.47:49458) 2022-06-29T15:47:39.550 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.539+0000 7fc70550d700 1 --2- 172.21.15.47:0/2223953963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7000aaf00 0x7fc7000ac600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:39.551 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.539+0000 7fc705d0e700 1 -- 172.21.15.47:0/2223953963 learned_addr learned my addr 172.21.15.47:0/2223953963 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:39.551 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.539+0000 7fc70650f700 1 --2- 172.21.15.47:0/2223953963 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc7000b1320 0x7fc7000acaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:39.551 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.539+0000 7fc705d0e700 1 -- 172.21.15.47:0/2223953963 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc7000b1320 msgr2=0x7fc7000acaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:39.552 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.539+0000 7fc705d0e700 1 --2- 172.21.15.47:0/2223953963 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc7000b1320 0x7fc7000acaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:39.552 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.539+0000 7fc705d0e700 1 -- 172.21.15.47:0/2223953963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7000aaf00 msgr2=0x7fc7000ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:39.552 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.543+0000 7fc705d0e700 1 --2- 172.21.15.47:0/2223953963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7000aaf00 0x7fc7000ac600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:39.552 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.543+0000 7fc705d0e700 1 -- 172.21.15.47:0/2223953963 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f8009530 con 0x7fc70005f3f0 2022-06-29T15:47:39.553 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.543+0000 7fc705d0e700 1 --2- 172.21.15.47:0/2223953963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc70005f3f0 0x7fc7000b6690 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7fc6f400c000 tx=0x7fc6f4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:39.553 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.543+0000 7fc6f2ffd700 1 -- 172.21.15.47:0/2223953963 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6f4010780 con 0x7fc70005f3f0 2022-06-29T15:47:39.553 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.543+0000 7fc706d10700 1 -- 172.21.15.47:0/2223953963 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc700061e50 con 0x7fc70005f3f0 2022-06-29T15:47:39.553 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.543+0000 7fc6f2ffd700 1 -- 172.21.15.47:0/2223953963 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6f40108e0 con 0x7fc70005f3f0 2022-06-29T15:47:39.554 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.543+0000 7fc6f2ffd700 1 -- 172.21.15.47:0/2223953963 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6f4010c40 con 0x7fc70005f3f0 2022-06-29T15:47:39.554 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.543+0000 7fc706d10700 1 -- 172.21.15.47:0/2223953963 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7000b7250 con 0x7fc70005f3f0 2022-06-29T15:47:39.554 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.543+0000 7fc706d10700 1 -- 172.21.15.47:0/2223953963 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7000573d0 con 0x7fc70005f3f0 2022-06-29T15:47:39.554 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.543+0000 7fc6f2ffd700 1 -- 172.21.15.47:0/2223953963 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc6f401c460 con 0x7fc70005f3f0 2022-06-29T15:47:39.555 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.543+0000 7fc6f2ffd700 1 --2- 172.21.15.47:0/2223953963 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc6e0053b30 0x7fc6e0055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:39.555 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.547+0000 7fc6f2ffd700 1 -- 172.21.15.47:0/2223953963 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc6f406c540 con 0x7fc70005f3f0 2022-06-29T15:47:39.555 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.547+0000 7fc70550d700 1 --2- 172.21.15.47:0/2223953963 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc6e0053b30 0x7fc6e0055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:39.555 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.547+0000 7fc70550d700 1 --2- 172.21.15.47:0/2223953963 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc6e0053b30 0x7fc6e0055ff0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7fc6fc00d000 tx=0x7fc6fc007b90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:39.556 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.551+0000 7fc6f2ffd700 1 -- 172.21.15.47:0/2223953963 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc6f4043110 con 0x7fc70005f3f0 2022-06-29T15:47:39.677 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.675+0000 7fc706d10700 1 -- 172.21.15.47:0/2223953963 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc700068830 con 0x7fc70005f3f0 2022-06-29T15:47:39.678 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.675+0000 7fc6f2ffd700 1 -- 172.21.15.47:0/2223953963 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v68) v1 ==== 100+0+10 (secure 0 0 0) 0x7fc6f4046730 con 0x7fc70005f3f0 2022-06-29T15:47:39.679 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:47:39.683 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.679+0000 7fc706d10700 1 -- 172.21.15.47:0/2223953963 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc6e0053b30 msgr2=0x7fc6e0055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:39.683 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.679+0000 7fc706d10700 1 --2- 172.21.15.47:0/2223953963 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc6e0053b30 0x7fc6e0055ff0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7fc6fc00d000 tx=0x7fc6fc007b90).stop 2022-06-29T15:47:39.683 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.679+0000 7fc706d10700 1 -- 172.21.15.47:0/2223953963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc70005f3f0 msgr2=0x7fc7000b6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:39.684 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.679+0000 7fc706d10700 1 --2- 172.21.15.47:0/2223953963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc70005f3f0 0x7fc7000b6690 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7fc6f400c000 tx=0x7fc6f4006df0).stop 2022-06-29T15:47:39.684 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.679+0000 7fc706d10700 1 -- 172.21.15.47:0/2223953963 shutdown_connections 2022-06-29T15:47:39.685 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.679+0000 7fc706d10700 1 --2- 172.21.15.47:0/2223953963 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc7000b1320 0x7fc7000acaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:39.685 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.679+0000 7fc706d10700 1 --2- 172.21.15.47:0/2223953963 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc6e0053b30 0x7fc6e0055ff0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:39.686 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.679+0000 7fc706d10700 1 --2- 172.21.15.47:0/2223953963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7000aaf00 0x7fc7000ac600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:39.686 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.679+0000 7fc706d10700 1 --2- 172.21.15.47:0/2223953963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc70005f3f0 0x7fc7000b6690 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:39.687 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.679+0000 7fc706d10700 1 -- 172.21.15.47:0/2223953963 >> 172.21.15.47:0/2223953963 conn(0x7fc700062570 msgr2=0x7fc7000aea80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:39.687 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.679+0000 7fc706d10700 1 -- 172.21.15.47:0/2223953963 shutdown_connections 2022-06-29T15:47:39.688 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.679+0000 7fc706d10700 1 -- 172.21.15.47:0/2223953963 wait complete. 2022-06-29T15:47:39.693 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-29T15:47:39.694 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:47:39.798 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.795+0000 7f6483d3d700 1 Processor -- start 2022-06-29T15:47:39.799 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.795+0000 7f6483d3d700 1 -- start start 2022-06-29T15:47:39.809 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.799+0000 7f6483d3d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f647c05f0b0 0x7f647c0b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:39.810 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.799+0000 7f6483d3d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f647c0b1260 0x7f647c14b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:39.810 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.799+0000 7f6483d3d700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f647c0b0740 con 0x7f647c0b2120 2022-06-29T15:47:39.810 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.799+0000 7f6483d3d700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f647c147460 con 0x7f647c0b1260 2022-06-29T15:47:39.811 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.799+0000 7f6483d3d700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f647c0b0970 con 0x7f647c05f0b0 2022-06-29T15:47:39.811 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.799+0000 7f64822da700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f647c0b1260 0x7f647c14b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:39.812 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.799+0000 7f6481ad9700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f647c0b2120 0x7f647c147af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35022/0 (socket says 172.21.15.47:35022) 2022-06-29T15:47:39.812 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.799+0000 7f6481ad9700 1 -- 172.21.15.47:0/647790936 learned_addr learned my addr 172.21.15.47:0/647790936 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:47:39.812 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.799+0000 7f64812d8700 1 --2- 172.21.15.47:0/647790936 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f647c05f0b0 0x7f647c0b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:39.813 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.799+0000 7f64812d8700 1 -- 172.21.15.47:0/647790936 >> v2:172.21.15.47:3300/0 conn(0x7f647c0b1260 msgr2=0x7f647c14b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:39.813 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.799+0000 7f64812d8700 1 --2- 172.21.15.47:0/647790936 >> v2:172.21.15.47:3300/0 conn(0x7f647c0b1260 0x7f647c14b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:39.814 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.799+0000 7f64812d8700 1 -- 172.21.15.47:0/647790936 >> v1:172.21.15.47:6789/0 conn(0x7f647c0b2120 legacy=0x7f647c147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:39.814 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.799+0000 7f64812d8700 1 -- 172.21.15.47:0/647790936 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f647c0b0c10 con 0x7f647c05f0b0 2022-06-29T15:47:39.814 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.799+0000 7f64822da700 1 --2- 172.21.15.47:0/647790936 >> v2:172.21.15.47:3300/0 conn(0x7f647c0b1260 0x7f647c14b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:47:39.815 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.799+0000 7f64812d8700 1 --2- 172.21.15.47:0/647790936 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f647c05f0b0 0x7f647c0b2500 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f6478002f70 tx=0x7f6478006df0).ready entity=mon.2 client_cookie=966564bfdbdfc550 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:39.815 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.799+0000 7f6480ad7700 1 -- 172.21.15.47:0/647790936 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f647800d570 con 0x7f647c05f0b0 2022-06-29T15:47:39.816 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.799+0000 7f6480ad7700 1 -- 172.21.15.47:0/647790936 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6478000e90 con 0x7f647c05f0b0 2022-06-29T15:47:39.816 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.799+0000 7f6480ad7700 1 -- 172.21.15.47:0/647790936 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f647800d8c0 con 0x7f647c05f0b0 2022-06-29T15:47:39.816 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.799+0000 7f6483d3d700 1 -- 172.21.15.47:0/647790936 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f647c05f0b0 msgr2=0x7f647c0b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:39.817 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.799+0000 7f6483d3d700 1 --2- 172.21.15.47:0/647790936 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f647c05f0b0 0x7f647c0b2500 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f6478002f70 tx=0x7f6478006df0).stop 2022-06-29T15:47:39.817 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.799+0000 7f6483d3d700 1 -- 172.21.15.47:0/647790936 shutdown_connections 2022-06-29T15:47:39.817 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.799+0000 7f6483d3d700 1 --2- 172.21.15.47:0/647790936 >> v2:172.21.15.47:3300/0 conn(0x7f647c0b1260 0x7f647c14b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:39.819 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.799+0000 7f6483d3d700 1 --2- 172.21.15.47:0/647790936 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f647c05f0b0 0x7f647c0b2500 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:39.819 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.799+0000 7f6483d3d700 1 -- 172.21.15.47:0/647790936 >> 172.21.15.47:0/647790936 conn(0x7f647c0626f0 msgr2=0x7f647c062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:39.819 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.799+0000 7f6483d3d700 1 -- 172.21.15.47:0/647790936 shutdown_connections 2022-06-29T15:47:39.820 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.799+0000 7f6483d3d700 1 -- 172.21.15.47:0/647790936 wait complete. 2022-06-29T15:47:39.820 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.803+0000 7f6483d3d700 1 Processor -- start 2022-06-29T15:47:39.821 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.803+0000 7f6483d3d700 1 -- start start 2022-06-29T15:47:39.821 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.803+0000 7f6483d3d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f647c0b1260 0x7f647c149ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:39.821 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.803+0000 7f6483d3d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f647c0b2120 0x7f647c140660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:39.822 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.803+0000 7f6483d3d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f647c140b50 0x7f647c15a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:39.822 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.803+0000 7f6483d3d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f647c141ab0 con 0x7f647c0b1260 2022-06-29T15:47:39.823 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.803+0000 7f6483d3d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f647c141d10 con 0x7f647c140b50 2022-06-29T15:47:39.823 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.803+0000 7f6483d3d700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f647c141fb0 con 0x7f647c0b2120 2022-06-29T15:47:39.823 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.803+0000 7f6481ad9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f647c0b1260 0x7f647c149ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:39.824 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.803+0000 7f6481ad9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f647c0b1260 0x7f647c149ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49472/0 (socket says 172.21.15.47:49472) 2022-06-29T15:47:39.824 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.803+0000 7f6481ad9700 1 -- 172.21.15.47:0/3483357836 learned_addr learned my addr 172.21.15.47:0/3483357836 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:39.825 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.803+0000 7f64812d8700 1 --2- 172.21.15.47:0/3483357836 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f647c0b2120 0x7f647c140660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:39.825 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.803+0000 7f64822da700 1 --2- 172.21.15.47:0/3483357836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f647c140b50 0x7f647c15a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:39.825 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.803+0000 7f6481ad9700 1 -- 172.21.15.47:0/3483357836 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f647c0b2120 msgr2=0x7f647c140660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:39.826 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.803+0000 7f6481ad9700 1 --2- 172.21.15.47:0/3483357836 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f647c0b2120 0x7f647c140660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:39.826 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.803+0000 7f6481ad9700 1 -- 172.21.15.47:0/3483357836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f647c140b50 msgr2=0x7f647c15a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:39.827 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.803+0000 7f6481ad9700 1 --2- 172.21.15.47:0/3483357836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f647c140b50 0x7f647c15a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:39.827 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.803+0000 7f6481ad9700 1 -- 172.21.15.47:0/3483357836 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6474009470 con 0x7f647c0b1260 2022-06-29T15:47:39.827 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.803+0000 7f6481ad9700 1 --2- 172.21.15.47:0/3483357836 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f647c0b1260 0x7f647c149ac0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7f647c06af60 tx=0x7f646c007ca0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:39.828 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.803+0000 7f646affd700 1 -- 172.21.15.47:0/3483357836 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f646c0070d0 con 0x7f647c0b1260 2022-06-29T15:47:39.828 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.803+0000 7f646affd700 1 -- 172.21.15.47:0/3483357836 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f646c007230 con 0x7f647c0b1260 2022-06-29T15:47:39.829 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.803+0000 7f6483d3d700 1 -- 172.21.15.47:0/3483357836 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f647800b180 con 0x7f647c0b1260 2022-06-29T15:47:39.829 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.803+0000 7f646affd700 1 -- 172.21.15.47:0/3483357836 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f646c005610 con 0x7f647c0b1260 2022-06-29T15:47:39.829 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.803+0000 7f6483d3d700 1 -- 172.21.15.47:0/3483357836 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f647c1424f0 con 0x7f647c0b1260 2022-06-29T15:47:39.830 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.807+0000 7f6483d3d700 1 -- 172.21.15.47:0/3483357836 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f647c0573d0 con 0x7f647c0b1260 2022-06-29T15:47:39.830 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.807+0000 7f646affd700 1 -- 172.21.15.47:0/3483357836 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f646c010070 con 0x7f647c0b1260 2022-06-29T15:47:39.831 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.807+0000 7f646affd700 1 --2- 172.21.15.47:0/3483357836 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f645c053b30 0x7f645c055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:39.831 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.807+0000 7f64812d8700 1 --2- 172.21.15.47:0/3483357836 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f645c053b30 0x7f645c055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:39.831 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.807+0000 7f646affd700 1 -- 172.21.15.47:0/3483357836 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f646c06b590 con 0x7f647c0b1260 2022-06-29T15:47:39.833 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.807+0000 7f64812d8700 1 --2- 172.21.15.47:0/3483357836 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f645c053b30 0x7f645c055ff0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f647801a000 tx=0x7f6478014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:39.833 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.815+0000 7f646affd700 1 -- 172.21.15.47:0/3483357836 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f646c042350 con 0x7f647c0b1260 2022-06-29T15:47:39.983 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.979+0000 7f6483d3d700 1 -- 172.21.15.47:0/3483357836 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f647c159ea0 con 0x7f647c0b1260 2022-06-29T15:47:39.983 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.983+0000 7f646affd700 1 -- 172.21.15.47:0/3483357836 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+2490 (secure 0 0 0) 0x7f646c045970 con 0x7f647c0b1260 2022-06-29T15:47:39.984 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:47:39.984 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":30,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5887,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/929350416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":929350416},{"type":"v1","addr":"172.21.15.47:6835","nonce":929350416}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":27},{"gid":5970,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2425218664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2425218664},{"type":"v1","addr":"172.21.15.64:6833","nonce":2425218664}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":27}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:47:35.340236+0000","modified":"2022-06-29T15:47:36.343849+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5973":{"gid":5973,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6837/1166335240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1166335240},{"type":"v1","addr":"172.21.15.47:6837","nonce":1166335240}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2022-06-29T15:47:39.985 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.983+0000 7f6483d3d700 1 -- 172.21.15.47:0/3483357836 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f645c053b30 msgr2=0x7f645c055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:39.986 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.983+0000 7f6483d3d700 1 --2- 172.21.15.47:0/3483357836 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f645c053b30 0x7f645c055ff0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f647801a000 tx=0x7f6478014000).stop 2022-06-29T15:47:39.986 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.983+0000 7f6483d3d700 1 -- 172.21.15.47:0/3483357836 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f647c0b1260 msgr2=0x7f647c149ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:39.986 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.983+0000 7f6483d3d700 1 --2- 172.21.15.47:0/3483357836 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f647c0b1260 0x7f647c149ac0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7f647c06af60 tx=0x7f646c007ca0).stop 2022-06-29T15:47:39.986 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.983+0000 7f6483d3d700 1 -- 172.21.15.47:0/3483357836 shutdown_connections 2022-06-29T15:47:39.987 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.983+0000 7f6483d3d700 1 --2- 172.21.15.47:0/3483357836 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f645c053b30 0x7f645c055ff0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:39.987 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.983+0000 7f6483d3d700 1 --2- 172.21.15.47:0/3483357836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f647c140b50 0x7f647c15a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:39.987 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.983+0000 7f6483d3d700 1 --2- 172.21.15.47:0/3483357836 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f647c0b2120 0x7f647c140660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:39.987 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.983+0000 7f6483d3d700 1 --2- 172.21.15.47:0/3483357836 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f647c0b1260 0x7f647c149ac0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:39.987 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.983+0000 7f6483d3d700 1 -- 172.21.15.47:0/3483357836 >> 172.21.15.47:0/3483357836 conn(0x7f647c0626f0 msgr2=0x7f647c0b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:39.988 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.983+0000 7f6483d3d700 1 -- 172.21.15.47:0/3483357836 shutdown_connections 2022-06-29T15:47:39.988 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:39.983+0000 7f6483d3d700 1 -- 172.21.15.47:0/3483357836 wait complete. 2022-06-29T15:47:39.988 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 30 2022-06-29T15:47:40.299 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 30, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-29T15:47:35.340236+0000', 'modified': '2022-06-29T15:47:36.343849+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5973}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5973': {'gid': 5973, 'name': 'a', 'rank': 0, 'incarnation': 29, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.47:6837/1166335240', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6836', 'nonce': 1166335240}, {'type': 'v1', 'addr': '172.21.15.47:6837', 'nonce': 1166335240}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-29T15:47:40.300 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-29T15:47:40.300 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-06-29T15:47:40.419 INFO:teuthology.orchestra.run.smithi047.stdout:{"cluster_fsid":"9a68894a-53ce-400e-838b-14eab91b834c","whoami":0,"id":5973,"want_state":"up:active","state":"up:active","rank_uptime":5.0670367890000003,"mdsmap_epoch":30,"osdmap_epoch":68,"osdmap_epoch_barrier":68,"uptime":9.4213615239999999} 2022-06-29T15:47:40.420 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9a68894a-53ce-400e-838b-14eab91b834c", "id": 5973, "mdsmap_epoch": 30, "osdmap_epoch": 68, "osdmap_epoch_barrier": 68, "rank_uptime": 5.067036789, "state": "up:active", "uptime": 9.421361524, "want_state": "up:active", "whoami": 0 } 2022-06-29T15:47:40.421 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-29T15:47:40.421 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-06-29T15:47:40.422 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi099.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-06-29T15:47:40.422 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:47:40.423 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:47:40.431 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2022-06-29T15:47:40.449 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:47:40.449 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:47:40.463 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:47:40.464 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:47:40.466 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:47:40.467 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2022-06-29T15:47:40.519 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-29T15:47:40.519 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:47:40.523 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2022-06-29T15:47:40.539 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:47:40.540 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:47:40.549 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:47:40.550 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:47:40.551 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:47:40.551 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2022-06-29T15:47:40.596 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:2022-06-29T15:47:40.593+0000 7f0cf898c080 -1 init, newargv = 0x555f809367a0 newargc=9ceph-fuse[12745 2022-06-29T15:47:40.597 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:]: starting ceph client 2022-06-29T15:47:40.606 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[12745]: starting fuse 2022-06-29T15:47:41.705 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2022-06-29T15:47:41.724 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:47:41.725 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:47:41.786 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:47:41.787 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:47:41.788 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:47:41.789 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:47:41.789 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2022-06-29T15:47:41.792 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:47:41.796 INFO:teuthology.orchestra.run.smithi099.stdout:52 2022-06-29T15:47:41.796 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-29T15:47:41.797 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:47:41.797 DEBUG:teuthology.orchestra.run.smithi099:> sudo python3 -c ' 2022-06-29T15:47:41.798 DEBUG:teuthology.orchestra.run.smithi099:> import glob 2022-06-29T15:47:41.798 DEBUG:teuthology.orchestra.run.smithi099:> import re 2022-06-29T15:47:41.799 DEBUG:teuthology.orchestra.run.smithi099:> import os 2022-06-29T15:47:41.800 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2022-06-29T15:47:41.801 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:47:41.802 DEBUG:teuthology.orchestra.run.smithi099:> def find_socket(client_name): 2022-06-29T15:47:41.802 DEBUG:teuthology.orchestra.run.smithi099:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-06-29T15:47:41.803 DEBUG:teuthology.orchestra.run.smithi099:> files = glob.glob(asok_path) 2022-06-29T15:47:41.804 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:47:41.805 DEBUG:teuthology.orchestra.run.smithi099:> # Given a non-glob path, it better be there 2022-06-29T15:47:41.806 DEBUG:teuthology.orchestra.run.smithi099:> if "*" not in asok_path: 2022-06-29T15:47:41.807 DEBUG:teuthology.orchestra.run.smithi099:> assert(len(files) == 1) 2022-06-29T15:47:41.808 DEBUG:teuthology.orchestra.run.smithi099:> return files[0] 2022-06-29T15:47:41.809 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:47:41.810 DEBUG:teuthology.orchestra.run.smithi099:> for f in files: 2022-06-29T15:47:41.810 DEBUG:teuthology.orchestra.run.smithi099:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-29T15:47:41.811 DEBUG:teuthology.orchestra.run.smithi099:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-29T15:47:41.812 DEBUG:teuthology.orchestra.run.smithi099:> return f 2022-06-29T15:47:41.812 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-29T15:47:41.812 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:47:41.813 DEBUG:teuthology.orchestra.run.smithi099:> print(find_socket("client.0")) 2022-06-29T15:47:41.813 DEBUG:teuthology.orchestra.run.smithi099:> ' 2022-06-29T15:47:41.815 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:47:41.843 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:47:41.886 INFO:teuthology.orchestra.run.smithi099.stdout:/var/run/ceph/ceph-client.0.12745.asok 2022-06-29T15:47:41.891 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.12745.asok 2022-06-29T15:47:41.892 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:47:41.892 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.12745.asok status 2022-06-29T15:47:41.984 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2022-06-29T15:47:41.984 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": { 2022-06-29T15:47:41.985 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_sha1": "f25ffac72e5a9e63860ff7deb6767a11f0559a16", 2022-06-29T15:47:41.985 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 15.2.16-304-gf25ffac7 (f25ffac72e5a9e63860ff7deb6767a11f0559a16) octopus (stable)", 2022-06-29T15:47:41.985 INFO:teuthology.orchestra.run.smithi099.stdout: "entity_id": "0", 2022-06-29T15:47:41.985 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": "smithi099", 2022-06-29T15:47:41.986 INFO:teuthology.orchestra.run.smithi099.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-06-29T15:47:41.986 INFO:teuthology.orchestra.run.smithi099.stdout: "pid": "12745", 2022-06-29T15:47:41.986 INFO:teuthology.orchestra.run.smithi099.stdout: "root": "/" 2022-06-29T15:47:41.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:47:41.987 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_count": 0, 2022-06-29T15:47:41.987 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_pinned_count": 0, 2022-06-29T15:47:41.987 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 6135, 2022-06-29T15:47:41.987 INFO:teuthology.orchestra.run.smithi099.stdout: "inst": { 2022-06-29T15:47:41.988 INFO:teuthology.orchestra.run.smithi099.stdout: "name": { 2022-06-29T15:47:41.988 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "client", 2022-06-29T15:47:41.988 INFO:teuthology.orchestra.run.smithi099.stdout: "num": 6135 2022-06-29T15:47:41.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:47:41.988 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2022-06-29T15:47:41.989 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2022-06-29T15:47:41.989 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:0", 2022-06-29T15:47:41.989 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3346200054 2022-06-29T15:47:41.989 INFO:teuthology.orchestra.run.smithi099.stdout: } 2022-06-29T15:47:41.990 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:47:41.990 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2022-06-29T15:47:41.990 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2022-06-29T15:47:41.990 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:0", 2022-06-29T15:47:41.991 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3346200054 2022-06-29T15:47:41.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:47:41.991 INFO:teuthology.orchestra.run.smithi099.stdout: "inst_str": "client.6135 172.21.15.99:0/3346200054", 2022-06-29T15:47:41.991 INFO:teuthology.orchestra.run.smithi099.stdout: "addr_str": "172.21.15.99:0/3346200054", 2022-06-29T15:47:41.991 INFO:teuthology.orchestra.run.smithi099.stdout: "inode_count": 1, 2022-06-29T15:47:41.992 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_epoch": 30, 2022-06-29T15:47:41.992 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch": 68, 2022-06-29T15:47:41.992 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch_barrier": 0, 2022-06-29T15:47:41.992 INFO:teuthology.orchestra.run.smithi099.stdout: "blacklisted": false, 2022-06-29T15:47:41.993 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs" 2022-06-29T15:47:41.993 INFO:teuthology.orchestra.run.smithi099.stdout:} 2022-06-29T15:47:41.995 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:47:41.995 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:47:41.996 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:47:42.002 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2022-06-29T15:47:42.004 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-29T15:47:42.005 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:47:42.006 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:47:42.025 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:47:42.026 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:47:42.027 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:47:42.035 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2022-06-29T15:47:42.037 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-29T15:47:42.038 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2022-06-29T15:47:42.039 DEBUG:teuthology.orchestra.run.smithi099:> import sys 2022-06-29T15:47:42.039 DEBUG:teuthology.orchestra.run.smithi099:> import time 2022-06-29T15:47:42.040 DEBUG:teuthology.orchestra.run.smithi099:> import os 2022-06-29T15:47:42.040 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:47:42.040 DEBUG:teuthology.orchestra.run.smithi099:> n = 1000 2022-06-29T15:47:42.041 DEBUG:teuthology.orchestra.run.smithi099:> abs_path = "/home/ubuntu/cephtest/mnt.0/dc-dir/dc-file" 2022-06-29T15:47:42.041 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:47:42.042 DEBUG:teuthology.orchestra.run.smithi099:> if not os.path.exists(os.path.dirname(abs_path)): 2022-06-29T15:47:42.042 DEBUG:teuthology.orchestra.run.smithi099:> os.makedirs(os.path.dirname(abs_path)) 2022-06-29T15:47:42.043 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:47:42.043 DEBUG:teuthology.orchestra.run.smithi099:> for i in range(0, n): 2022-06-29T15:47:42.044 DEBUG:teuthology.orchestra.run.smithi099:> fname = "{0}_{1}".format(abs_path, i) 2022-06-29T15:47:42.044 DEBUG:teuthology.orchestra.run.smithi099:> with open(fname, '"'"'w'"'"') as f: 2022-06-29T15:47:42.044 DEBUG:teuthology.orchestra.run.smithi099:> f.write('"'"'content'"'"') 2022-06-29T15:47:42.045 DEBUG:teuthology.orchestra.run.smithi099:> if True: 2022-06-29T15:47:42.045 DEBUG:teuthology.orchestra.run.smithi099:> f.flush() 2022-06-29T15:47:42.046 DEBUG:teuthology.orchestra.run.smithi099:> os.fsync(f.fileno()) 2022-06-29T15:47:42.046 DEBUG:teuthology.orchestra.run.smithi099:> ' 2022-06-29T15:47:53.763 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:47:53.874 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.871+0000 7fc036ec2700 1 Processor -- start 2022-06-29T15:47:53.874 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.871+0000 7fc036ec2700 1 -- start start 2022-06-29T15:47:53.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.875+0000 7fc036ec2700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc0301430f0 0x7fc0301434d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:53.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.875+0000 7fc036ec2700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc030095dc0 con 0x7fc030095950 2022-06-29T15:47:53.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.875+0000 7fc036ec2700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc030060e50 con 0x7fc0300a93c0 2022-06-29T15:47:53.877 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.875+0000 7fc036ec2700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc03013d590 con 0x7fc0301430f0 2022-06-29T15:47:53.877 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.875+0000 7fc034c5e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc0301430f0 0x7fc0301434d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:53.877 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.875+0000 7fc03545f700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc030095950 0x7fc030097690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35038/0 (socket says 172.21.15.47:35038) 2022-06-29T15:47:53.877 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.875+0000 7fc034c5e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc0301430f0 0x7fc0301434d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49480/0 (socket says 172.21.15.47:49480) 2022-06-29T15:47:53.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.875+0000 7fc03545f700 1 -- 172.21.15.47:0/90994664 learned_addr learned my addr 172.21.15.47:0/90994664 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:47:53.880 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.875+0000 7fc034c5e700 1 -- 172.21.15.47:0/90994664 >> v1:172.21.15.64:6789/0 conn(0x7fc0300a93c0 legacy=0x7fc030095240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:53.880 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.875+0000 7fc034c5e700 1 -- 172.21.15.47:0/90994664 >> v1:172.21.15.47:6789/0 conn(0x7fc030095950 legacy=0x7fc030097690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:53.880 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.875+0000 7fc02f7fe700 1 -- 172.21.15.47:0/90994664 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 15275092 0 0) 0x7fc030095dc0 con 0x7fc030095950 2022-06-29T15:47:53.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.875+0000 7fc034c5e700 1 -- 172.21.15.47:0/90994664 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0300610f0 con 0x7fc0301430f0 2022-06-29T15:47:53.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.875+0000 7fc034c5e700 1 --2- 172.21.15.47:0/90994664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0301430f0 0x7fc0301434d0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7fc024002f70 tx=0x7fc024006df0).ready entity=mon.0 client_cookie=fadd981968e6d755 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:53.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.875+0000 7fc02f7fe700 1 -- 172.21.15.47:0/90994664 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc02400d610 con 0x7fc0301430f0 2022-06-29T15:47:53.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.875+0000 7fc02f7fe700 1 -- 172.21.15.47:0/90994664 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc024000e90 con 0x7fc0301430f0 2022-06-29T15:47:53.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.875+0000 7fc02f7fe700 1 -- 172.21.15.47:0/90994664 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc02400d960 con 0x7fc0301430f0 2022-06-29T15:47:53.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.875+0000 7fc036ec2700 1 -- 172.21.15.47:0/90994664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0301430f0 msgr2=0x7fc0301434d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:53.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.875+0000 7fc036ec2700 1 --2- 172.21.15.47:0/90994664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0301430f0 0x7fc0301434d0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7fc024002f70 tx=0x7fc024006df0).stop 2022-06-29T15:47:53.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.879+0000 7fc036ec2700 1 -- 172.21.15.47:0/90994664 shutdown_connections 2022-06-29T15:47:53.883 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.879+0000 7fc036ec2700 1 --2- 172.21.15.47:0/90994664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0301430f0 0x7fc0301434d0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:53.883 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.879+0000 7fc036ec2700 1 -- 172.21.15.47:0/90994664 >> 172.21.15.47:0/90994664 conn(0x7fc03005f0b0 msgr2=0x7fc03005f490 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:53.883 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.879+0000 7fc036ec2700 1 -- 172.21.15.47:0/90994664 shutdown_connections 2022-06-29T15:47:53.883 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.879+0000 7fc036ec2700 1 -- 172.21.15.47:0/90994664 wait complete. 2022-06-29T15:47:53.886 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.879+0000 7fc036ec2700 1 Processor -- start 2022-06-29T15:47:53.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.879+0000 7fc036ec2700 1 -- start start 2022-06-29T15:47:53.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.883+0000 7fc036ec2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc030095950 0x7fc0300b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:53.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.883+0000 7fc036ec2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0300a93c0 0x7fc0300a8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:53.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.883+0000 7fc036ec2700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc0301430f0 0x7fc0300a8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:53.888 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.883+0000 7fc036ec2700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03013dc80 con 0x7fc030095950 2022-06-29T15:47:53.888 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.883+0000 7fc036ec2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03014ef30 con 0x7fc0300a93c0 2022-06-29T15:47:53.888 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.883+0000 7fc036ec2700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc03014f1d0 con 0x7fc0301430f0 2022-06-29T15:47:53.888 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.883+0000 7fc034c5e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc030095950 0x7fc0300b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:53.889 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.883+0000 7fc034c5e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc030095950 0x7fc0300b20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49486/0 (socket says 172.21.15.47:49486) 2022-06-29T15:47:53.889 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.883+0000 7fc034c5e700 1 -- 172.21.15.47:0/3407798008 learned_addr learned my addr 172.21.15.47:0/3407798008 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:53.889 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.883+0000 7fc02ffff700 1 --2- 172.21.15.47:0/3407798008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0300a93c0 0x7fc0300a8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:53.889 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.883+0000 7fc03545f700 1 --2- 172.21.15.47:0/3407798008 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc0301430f0 0x7fc0300a8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:53.889 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.883+0000 7fc034c5e700 1 -- 172.21.15.47:0/3407798008 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc0301430f0 msgr2=0x7fc0300a8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:53.890 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.883+0000 7fc034c5e700 1 --2- 172.21.15.47:0/3407798008 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc0301430f0 0x7fc0300a8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:53.890 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.883+0000 7fc034c5e700 1 -- 172.21.15.47:0/3407798008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0300a93c0 msgr2=0x7fc0300a8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:53.890 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.883+0000 7fc034c5e700 1 --2- 172.21.15.47:0/3407798008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0300a93c0 0x7fc0300a8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:53.891 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.883+0000 7fc034c5e700 1 -- 172.21.15.47:0/3407798008 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc02400b180 con 0x7fc030095950 2022-06-29T15:47:53.891 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.883+0000 7fc034c5e700 1 --2- 172.21.15.47:0/3407798008 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc030095950 0x7fc0300b20b0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7fc024002f70 tx=0x7fc024015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:53.891 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.883+0000 7fc02dffb700 1 -- 172.21.15.47:0/3407798008 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc02400d290 con 0x7fc030095950 2022-06-29T15:47:53.891 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.883+0000 7fc02dffb700 1 -- 172.21.15.47:0/3407798008 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc02401dc70 con 0x7fc030095950 2022-06-29T15:47:53.891 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.883+0000 7fc02dffb700 1 -- 172.21.15.47:0/3407798008 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc024011400 con 0x7fc030095950 2022-06-29T15:47:53.892 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.883+0000 7fc036ec2700 1 -- 172.21.15.47:0/3407798008 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0300b28f0 con 0x7fc030095950 2022-06-29T15:47:53.892 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.883+0000 7fc036ec2700 1 -- 172.21.15.47:0/3407798008 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0300a9060 con 0x7fc030095950 2022-06-29T15:47:53.893 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.887+0000 7fc036ec2700 1 -- 172.21.15.47:0/3407798008 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0300573d0 con 0x7fc030095950 2022-06-29T15:47:53.894 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.887+0000 7fc02dffb700 1 -- 172.21.15.47:0/3407798008 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc02402d070 con 0x7fc030095950 2022-06-29T15:47:53.894 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.887+0000 7fc02dffb700 1 --2- 172.21.15.47:0/3407798008 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc010053b00 0x7fc010055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:53.894 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.887+0000 7fc02dffb700 1 -- 172.21.15.47:0/3407798008 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc02406f320 con 0x7fc030095950 2022-06-29T15:47:53.894 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.887+0000 7fc02ffff700 1 --2- 172.21.15.47:0/3407798008 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc010053b00 0x7fc010055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:53.895 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.887+0000 7fc02ffff700 1 --2- 172.21.15.47:0/3407798008 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc010053b00 0x7fc010055fc0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7fc01c00d000 tx=0x7fc01c007b90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:53.895 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:53.891+0000 7fc02dffb700 1 -- 172.21.15.47:0/3407798008 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc024046040 con 0x7fc030095950 2022-06-29T15:47:54.054 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.051+0000 7fc036ec2700 1 -- 172.21.15.47:0/3407798008 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc03009bdb0 con 0x7fc030095950 2022-06-29T15:47:54.055 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.055+0000 7fc02dffb700 1 -- 172.21.15.47:0/3407798008 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+2490 (secure 0 0 0) 0x7fc024049660 con 0x7fc030095950 2022-06-29T15:47:54.056 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:47:54.056 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":30,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5887,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/929350416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":929350416},{"type":"v1","addr":"172.21.15.47:6835","nonce":929350416}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":27},{"gid":5970,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2425218664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2425218664},{"type":"v1","addr":"172.21.15.64:6833","nonce":2425218664}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":27}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:47:35.340236+0000","modified":"2022-06-29T15:47:36.343849+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5973":{"gid":5973,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6837/1166335240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1166335240},{"type":"v1","addr":"172.21.15.47:6837","nonce":1166335240}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2022-06-29T15:47:54.058 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.055+0000 7fc036ec2700 1 -- 172.21.15.47:0/3407798008 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc010053b00 msgr2=0x7fc010055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:54.058 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.055+0000 7fc036ec2700 1 --2- 172.21.15.47:0/3407798008 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc010053b00 0x7fc010055fc0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7fc01c00d000 tx=0x7fc01c007b90).stop 2022-06-29T15:47:54.058 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.055+0000 7fc036ec2700 1 -- 172.21.15.47:0/3407798008 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc030095950 msgr2=0x7fc0300b20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:54.059 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.055+0000 7fc036ec2700 1 --2- 172.21.15.47:0/3407798008 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc030095950 0x7fc0300b20b0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7fc024002f70 tx=0x7fc024015000).stop 2022-06-29T15:47:54.059 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.055+0000 7fc036ec2700 1 -- 172.21.15.47:0/3407798008 shutdown_connections 2022-06-29T15:47:54.059 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.055+0000 7fc036ec2700 1 --2- 172.21.15.47:0/3407798008 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc0301430f0 0x7fc0300a8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:54.059 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.055+0000 7fc036ec2700 1 --2- 172.21.15.47:0/3407798008 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc010053b00 0x7fc010055fc0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:54.060 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.055+0000 7fc036ec2700 1 --2- 172.21.15.47:0/3407798008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0300a93c0 0x7fc0300a8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:54.060 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.055+0000 7fc036ec2700 1 --2- 172.21.15.47:0/3407798008 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc030095950 0x7fc0300b20b0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:54.060 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.055+0000 7fc036ec2700 1 -- 172.21.15.47:0/3407798008 >> 172.21.15.47:0/3407798008 conn(0x7fc03005f0b0 msgr2=0x7fc0300b5470 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:54.060 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.055+0000 7fc036ec2700 1 -- 172.21.15.47:0/3407798008 shutdown_connections 2022-06-29T15:47:54.061 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.055+0000 7fc036ec2700 1 -- 172.21.15.47:0/3407798008 wait complete. 2022-06-29T15:47:54.061 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 30 2022-06-29T15:47:54.069 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_recall_max_caps 20 2022-06-29T15:47:54.175 INFO:teuthology.orchestra.run.smithi047.stdout:{"success":"mds_recall_max_caps = '20' (not observed, change may require restart) "} 2022-06-29T15:47:54.175 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_caps = '20' (not observed, change may require restart) " } 2022-06-29T15:47:54.176 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:47:54.283 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.283+0000 7f4a16f81700 1 Processor -- start 2022-06-29T15:47:54.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.283+0000 7f4a16f81700 1 -- start start 2022-06-29T15:47:54.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.283+0000 7f4a16f81700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4a1005c510 0x7f4a10143bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:54.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.283+0000 7f4a16f81700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4a100b2c40 0x7f4a100b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:54.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.283+0000 7f4a16f81700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a10142f90 con 0x7f4a1005c8f0 2022-06-29T15:47:54.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.283+0000 7f4a16f81700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a101431f0 con 0x7f4a100b2c40 2022-06-29T15:47:54.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.283+0000 7f4a16f81700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a10143490 con 0x7f4a1005c510 2022-06-29T15:47:54.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.283+0000 7f4a1577e700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4a1005c8f0 0x7f4a100b4100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35050/0 (socket says 172.21.15.47:35050) 2022-06-29T15:47:54.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.283+0000 7f4a1577e700 1 -- 172.21.15.47:0/3661450734 learned_addr learned my addr 172.21.15.47:0/3661450734 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:47:54.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.283+0000 7f4a15f7f700 1 --2- 172.21.15.47:0/3661450734 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4a1005c510 0x7f4a10143bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:54.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.283+0000 7f4a16780700 1 --2- 172.21.15.47:0/3661450734 >> v2:172.21.15.64:3300/0 conn(0x7f4a100b2c40 0x7f4a100b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:54.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.283+0000 7f4a14f7d700 1 -- 172.21.15.47:0/3661450734 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 340567482 0 0) 0x7f4a10142f90 con 0x7f4a1005c8f0 2022-06-29T15:47:54.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.283+0000 7f4a14f7d700 1 -- 172.21.15.47:0/3661450734 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49f8003260 con 0x7f4a1005c8f0 2022-06-29T15:47:54.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.283+0000 7f4a15f7f700 1 -- 172.21.15.47:0/3661450734 >> v2:172.21.15.64:3300/0 conn(0x7f4a100b2c40 msgr2=0x7f4a100b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:54.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.283+0000 7f4a15f7f700 1 --2- 172.21.15.47:0/3661450734 >> v2:172.21.15.64:3300/0 conn(0x7f4a100b2c40 0x7f4a100b30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:54.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.283+0000 7f4a15f7f700 1 -- 172.21.15.47:0/3661450734 >> v1:172.21.15.47:6789/0 conn(0x7f4a1005c8f0 legacy=0x7f4a100b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:54.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.283+0000 7f4a15f7f700 1 -- 172.21.15.47:0/3661450734 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a10143730 con 0x7f4a1005c510 2022-06-29T15:47:54.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.283+0000 7f4a15f7f700 1 --2- 172.21.15.47:0/3661450734 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4a1005c510 0x7f4a10143bd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f4a0c002f70 tx=0x7f4a0c006df0).ready entity=mon.2 client_cookie=38a24446e7d8ceea server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:54.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.287+0000 7f4a14f7d700 1 -- 172.21.15.47:0/3661450734 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4a0c00d4a0 con 0x7f4a1005c510 2022-06-29T15:47:54.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.287+0000 7f4a14f7d700 1 -- 172.21.15.47:0/3661450734 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a0c000e90 con 0x7f4a1005c510 2022-06-29T15:47:54.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.287+0000 7f4a14f7d700 1 -- 172.21.15.47:0/3661450734 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4a0c00d7f0 con 0x7f4a1005c510 2022-06-29T15:47:54.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.287+0000 7f4a16f81700 1 -- 172.21.15.47:0/3661450734 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4a1005c510 msgr2=0x7f4a10143bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:54.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.287+0000 7f4a16f81700 1 --2- 172.21.15.47:0/3661450734 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4a1005c510 0x7f4a10143bd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f4a0c002f70 tx=0x7f4a0c006df0).stop 2022-06-29T15:47:54.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.287+0000 7f4a16f81700 1 -- 172.21.15.47:0/3661450734 shutdown_connections 2022-06-29T15:47:54.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.287+0000 7f4a16f81700 1 --2- 172.21.15.47:0/3661450734 >> v2:172.21.15.64:3300/0 conn(0x7f4a100b2c40 0x7f4a100b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:54.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.287+0000 7f4a16f81700 1 --2- 172.21.15.47:0/3661450734 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4a1005c510 0x7f4a10143bd0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:54.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.287+0000 7f4a16f81700 1 -- 172.21.15.47:0/3661450734 >> 172.21.15.47:0/3661450734 conn(0x7f4a10060e00 msgr2=0x7f4a100611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:54.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.287+0000 7f4a16f81700 1 -- 172.21.15.47:0/3661450734 shutdown_connections 2022-06-29T15:47:54.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.287+0000 7f4a16f81700 1 -- 172.21.15.47:0/3661450734 wait complete. 2022-06-29T15:47:54.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.287+0000 7f4a16f81700 1 Processor -- start 2022-06-29T15:47:54.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.287+0000 7f4a16f81700 1 -- start start 2022-06-29T15:47:54.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.287+0000 7f4a16f81700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a1005c510 0x7f4a100ad9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:54.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.287+0000 7f4a16f81700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4a1005c8f0 0x7f4a10146560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:54.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.287+0000 7f4a16f81700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a100b2c40 0x7f4a10146a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:54.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.287+0000 7f4a16f81700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1013dee0 con 0x7f4a1005c510 2022-06-29T15:47:54.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.287+0000 7f4a16f81700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a10152da0 con 0x7f4a100b2c40 2022-06-29T15:47:54.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.287+0000 7f4a16f81700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a10153040 con 0x7f4a1005c8f0 2022-06-29T15:47:54.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.287+0000 7f4a15f7f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a1005c510 0x7f4a100ad9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:54.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.291+0000 7f4a15f7f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a1005c510 0x7f4a100ad9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49500/0 (socket says 172.21.15.47:49500) 2022-06-29T15:47:54.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.291+0000 7f4a15f7f700 1 -- 172.21.15.47:0/2740347293 learned_addr learned my addr 172.21.15.47:0/2740347293 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:54.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.291+0000 7f4a16780700 1 --2- 172.21.15.47:0/2740347293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a100b2c40 0x7f4a10146a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:54.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.291+0000 7f4a1577e700 1 --2- 172.21.15.47:0/2740347293 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4a1005c8f0 0x7f4a10146560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:54.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.291+0000 7f4a15f7f700 1 -- 172.21.15.47:0/2740347293 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4a1005c8f0 msgr2=0x7f4a10146560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:54.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.291+0000 7f4a15f7f700 1 --2- 172.21.15.47:0/2740347293 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4a1005c8f0 0x7f4a10146560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:54.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.291+0000 7f4a15f7f700 1 -- 172.21.15.47:0/2740347293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a100b2c40 msgr2=0x7f4a10146a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:54.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.291+0000 7f4a15f7f700 1 --2- 172.21.15.47:0/2740347293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a100b2c40 0x7f4a10146a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:54.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.291+0000 7f4a15f7f700 1 -- 172.21.15.47:0/2740347293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a0c00b180 con 0x7f4a1005c510 2022-06-29T15:47:54.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.291+0000 7f4a15f7f700 1 --2- 172.21.15.47:0/2740347293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a1005c510 0x7f4a100ad9b0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7f4a0c002f70 tx=0x7f4a0c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:54.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.291+0000 7f49feffd700 1 -- 172.21.15.47:0/2740347293 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4a0c012930 con 0x7f4a1005c510 2022-06-29T15:47:54.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.291+0000 7f49feffd700 1 -- 172.21.15.47:0/2740347293 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a0c012a90 con 0x7f4a1005c510 2022-06-29T15:47:54.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.291+0000 7f4a16f81700 1 -- 172.21.15.47:0/2740347293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a1013e0d0 con 0x7f4a1005c510 2022-06-29T15:47:54.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.291+0000 7f49feffd700 1 -- 172.21.15.47:0/2740347293 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4a0c012df0 con 0x7f4a1005c510 2022-06-29T15:47:54.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.291+0000 7f4a16f81700 1 -- 172.21.15.47:0/2740347293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a10146f40 con 0x7f4a1005c510 2022-06-29T15:47:54.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.291+0000 7f4a16f81700 1 -- 172.21.15.47:0/2740347293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a100573d0 con 0x7f4a1005c510 2022-06-29T15:47:54.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.291+0000 7f49feffd700 1 -- 172.21.15.47:0/2740347293 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4a0c01c070 con 0x7f4a1005c510 2022-06-29T15:47:54.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.291+0000 7f49feffd700 1 --2- 172.21.15.47:0/2740347293 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f49f00539e0 0x7f49f0055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:54.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.291+0000 7f49feffd700 1 -- 172.21.15.47:0/2740347293 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f4a0c06cb90 con 0x7f4a1005c510 2022-06-29T15:47:54.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.291+0000 7f4a1577e700 1 --2- 172.21.15.47:0/2740347293 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f49f00539e0 0x7f49f0055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:54.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.295+0000 7f4a1577e700 1 --2- 172.21.15.47:0/2740347293 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f49f00539e0 0x7f49f0055ea0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f4a0000d000 tx=0x7f4a00009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:54.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.299+0000 7f49feffd700 1 -- 172.21.15.47:0/2740347293 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4a0c0438b0 con 0x7f4a1005c510 2022-06-29T15:47:54.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.459+0000 7f4a16f81700 1 -- 172.21.15.47:0/2740347293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4a1013e260 con 0x7f4a1005c510 2022-06-29T15:47:54.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.463+0000 7f49feffd700 1 -- 172.21.15.47:0/2740347293 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+2490 (secure 0 0 0) 0x7f4a0c019070 con 0x7f4a1005c510 2022-06-29T15:47:54.463 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:47:54.463 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":30,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5887,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/929350416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":929350416},{"type":"v1","addr":"172.21.15.47:6835","nonce":929350416}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":27},{"gid":5970,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2425218664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2425218664},{"type":"v1","addr":"172.21.15.64:6833","nonce":2425218664}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":27}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:47:35.340236+0000","modified":"2022-06-29T15:47:36.343849+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5973":{"gid":5973,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6837/1166335240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1166335240},{"type":"v1","addr":"172.21.15.47:6837","nonce":1166335240}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2022-06-29T15:47:54.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.463+0000 7f4a16f81700 1 -- 172.21.15.47:0/2740347293 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f49f00539e0 msgr2=0x7f49f0055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:54.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.463+0000 7f4a16f81700 1 --2- 172.21.15.47:0/2740347293 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f49f00539e0 0x7f49f0055ea0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f4a0000d000 tx=0x7f4a00009040).stop 2022-06-29T15:47:54.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.463+0000 7f4a16f81700 1 -- 172.21.15.47:0/2740347293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a1005c510 msgr2=0x7f4a100ad9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:54.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.463+0000 7f4a16f81700 1 --2- 172.21.15.47:0/2740347293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a1005c510 0x7f4a100ad9b0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7f4a0c002f70 tx=0x7f4a0c014000).stop 2022-06-29T15:47:54.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.463+0000 7f4a16f81700 1 -- 172.21.15.47:0/2740347293 shutdown_connections 2022-06-29T15:47:54.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.463+0000 7f4a16f81700 1 --2- 172.21.15.47:0/2740347293 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f49f00539e0 0x7f49f0055ea0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:54.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.463+0000 7f4a16f81700 1 --2- 172.21.15.47:0/2740347293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a100b2c40 0x7f4a10146a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:54.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.463+0000 7f4a16f81700 1 --2- 172.21.15.47:0/2740347293 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4a1005c8f0 0x7f4a10146560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:54.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.463+0000 7f4a16f81700 1 --2- 172.21.15.47:0/2740347293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a1005c510 0x7f4a100ad9b0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:54.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.463+0000 7f4a16f81700 1 -- 172.21.15.47:0/2740347293 >> 172.21.15.47:0/2740347293 conn(0x7f4a10060e00 msgr2=0x7f4a100b3640 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:54.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.463+0000 7f4a16f81700 1 -- 172.21.15.47:0/2740347293 shutdown_connections 2022-06-29T15:47:54.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.463+0000 7f4a16f81700 1 -- 172.21.15.47:0/2740347293 wait complete. 2022-06-29T15:47:54.468 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 30 2022-06-29T15:47:54.480 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_recall_max_decay_threshold 400 2022-06-29T15:47:54.587 INFO:teuthology.orchestra.run.smithi047.stdout:{"success":"mds_recall_max_decay_threshold = '400' (not observed, change may require restart) "} 2022-06-29T15:47:54.587 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_decay_threshold = '400' (not observed, change may require restart) " } 2022-06-29T15:47:54.588 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:47:54.691 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.691+0000 7f57d211a700 1 Processor -- start 2022-06-29T15:47:54.691 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.691+0000 7f57d211a700 1 -- start start 2022-06-29T15:47:54.695 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.691+0000 7f57d211a700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f57cc0b5c60 0x7f57cc144980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:54.695 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.691+0000 7f57d211a700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57cc0a9120 con 0x7f57cc0a72f0 2022-06-29T15:47:54.696 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.691+0000 7f57d211a700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57cc0b0b80 con 0x7f57cc060eb0 2022-06-29T15:47:54.696 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.691+0000 7f57d211a700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57cc0a89e0 con 0x7f57cc0b5c60 2022-06-29T15:47:54.696 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.691+0000 7f57d1919700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f57cc0a72f0 0x7f57cc151eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35066/0 (socket says 172.21.15.47:35066) 2022-06-29T15:47:54.696 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.691+0000 7f57d1919700 1 -- 172.21.15.47:0/50107682 learned_addr learned my addr 172.21.15.47:0/50107682 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:47:54.697 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.691+0000 7f57d1118700 1 --2- 172.21.15.47:0/50107682 >> v2:172.21.15.64:3300/0 conn(0x7f57cc0b5c60 0x7f57cc144980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:54.697 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.691+0000 7f57c3fff700 1 -- 172.21.15.47:0/50107682 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3514185746 0 0) 0x7f57cc0a9120 con 0x7f57cc0a72f0 2022-06-29T15:47:54.697 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.691+0000 7f57c3fff700 1 -- 172.21.15.47:0/50107682 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57b0003260 con 0x7f57cc0a72f0 2022-06-29T15:47:54.697 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.691+0000 7f57c3fff700 1 -- 172.21.15.47:0/50107682 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 195263438 0 0) 0x7f57b0003260 con 0x7f57cc0a72f0 2022-06-29T15:47:54.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.691+0000 7f57c3fff700 1 -- 172.21.15.47:0/50107682 >> v2:172.21.15.64:3300/0 conn(0x7f57cc0b5c60 msgr2=0x7f57cc144980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:54.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.691+0000 7f57c3fff700 1 --2- 172.21.15.47:0/50107682 >> v2:172.21.15.64:3300/0 conn(0x7f57cc0b5c60 0x7f57cc144980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:54.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.691+0000 7f57c3fff700 1 -- 172.21.15.47:0/50107682 >> v1:172.21.15.64:6789/0 conn(0x7f57cc060eb0 legacy=0x7f57cc0b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:54.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.691+0000 7f57c3fff700 1 -- 172.21.15.47:0/50107682 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57cc0a8c80 con 0x7f57cc0a72f0 2022-06-29T15:47:54.699 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.691+0000 7f57c3fff700 1 -- 172.21.15.47:0/50107682 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7f57c40019e0 con 0x7f57cc0a72f0 2022-06-29T15:47:54.699 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.691+0000 7f57c3fff700 1 -- 172.21.15.47:0/50107682 >> v1:172.21.15.47:6789/0 conn(0x7f57cc0a72f0 legacy=0x7f57cc151eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:54.699 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.691+0000 7f57c3fff700 1 --2- 172.21.15.47:0/50107682 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f57b0003750 0x7f57b0003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:54.699 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.691+0000 7f57c3fff700 1 --2- 172.21.15.47:0/50107682 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57b0004500 0x7f57b0004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:54.700 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.691+0000 7f57c3fff700 1 --2- 172.21.15.47:0/50107682 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57b0004e70 0x7f57b000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:54.700 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.691+0000 7f57c3fff700 1 -- 172.21.15.47:0/50107682 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc0a89e0 con 0x7f57b0004e70 2022-06-29T15:47:54.700 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.691+0000 7f57c3fff700 1 -- 172.21.15.47:0/50107682 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b000bd20 con 0x7f57b0004500 2022-06-29T15:47:54.700 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.691+0000 7f57c3fff700 1 -- 172.21.15.47:0/50107682 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57b000bee0 con 0x7f57b0003750 2022-06-29T15:47:54.700 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.691+0000 7f57d1118700 1 --2- 172.21.15.47:0/50107682 >> v2:172.21.15.64:3300/0 conn(0x7f57cc0b5c60 0x7f57cc144980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:47:54.701 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.691+0000 7f57d1919700 1 --2- 172.21.15.47:0/50107682 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57b0004e70 0x7f57b000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:54.701 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.691+0000 7f57d0917700 1 --2- 172.21.15.47:0/50107682 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57b0004500 0x7f57b0004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:54.701 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.691+0000 7f57d1919700 1 -- 172.21.15.47:0/50107682 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f57b0003750 msgr2=0x7f57b0003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:54.701 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.691+0000 7f57d1118700 1 --2- 172.21.15.47:0/50107682 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f57b0003750 0x7f57b0003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:54.702 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.691+0000 7f57d1919700 1 --2- 172.21.15.47:0/50107682 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f57b0003750 0x7f57b0003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:54.702 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.691+0000 7f57d1919700 1 -- 172.21.15.47:0/50107682 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57b0004500 msgr2=0x7f57b0004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:54.702 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.691+0000 7f57d1919700 1 --2- 172.21.15.47:0/50107682 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57b0004500 0x7f57b0004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:54.702 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.691+0000 7f57d1919700 1 -- 172.21.15.47:0/50107682 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b0003260 con 0x7f57b0004e70 2022-06-29T15:47:54.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.691+0000 7f57d0917700 1 --2- 172.21.15.47:0/50107682 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57b0004500 0x7f57b0004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:47:54.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.691+0000 7f57d1118700 1 --2- 172.21.15.47:0/50107682 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f57b0003750 0x7f57b0003b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:47:54.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.691+0000 7f57d1919700 1 -- 172.21.15.47:0/50107682 reap_dead start 2022-06-29T15:47:54.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.691+0000 7f57d1919700 1 --2- 172.21.15.47:0/50107682 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57b0004e70 0x7f57b000b750 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7f57c40016f0 tx=0x7f57cc0a87f0).ready entity=mon.0 client_cookie=2aeff981b60c1346 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:54.704 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.691+0000 7f57c3fff700 1 -- 172.21.15.47:0/50107682 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f57c400cc70 con 0x7f57b0004e70 2022-06-29T15:47:54.704 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.691+0000 7f57c3fff700 1 -- 172.21.15.47:0/50107682 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57c400cdd0 con 0x7f57b0004e70 2022-06-29T15:47:54.704 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.691+0000 7f57c3fff700 1 -- 172.21.15.47:0/50107682 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f57c4005b80 con 0x7f57b0004e70 2022-06-29T15:47:54.704 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.691+0000 7f57d211a700 1 -- 172.21.15.47:0/50107682 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57b0004e70 msgr2=0x7f57b000b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:54.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.691+0000 7f57d211a700 1 --2- 172.21.15.47:0/50107682 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57b0004e70 0x7f57b000b750 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7f57c40016f0 tx=0x7f57cc0a87f0).stop 2022-06-29T15:47:54.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.691+0000 7f57d211a700 1 -- 172.21.15.47:0/50107682 shutdown_connections 2022-06-29T15:47:54.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.691+0000 7f57d211a700 1 --2- 172.21.15.47:0/50107682 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57b0004e70 0x7f57b000b750 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:54.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.691+0000 7f57d211a700 1 -- 172.21.15.47:0/50107682 >> 172.21.15.47:0/50107682 conn(0x7f57cc062560 msgr2=0x7f57cc062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:54.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.695+0000 7f57d211a700 1 -- 172.21.15.47:0/50107682 shutdown_connections 2022-06-29T15:47:54.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.695+0000 7f57d211a700 1 -- 172.21.15.47:0/50107682 wait complete. 2022-06-29T15:47:54.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.695+0000 7f57d211a700 1 Processor -- start 2022-06-29T15:47:54.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.695+0000 7f57d211a700 1 -- start start 2022-06-29T15:47:54.709 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.695+0000 7f57d211a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57b0004e70 0x7f57cc13db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:54.709 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.695+0000 7f57d211a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57cc142410 0x7f57cc147670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:54.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.695+0000 7f57d211a700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f57cc147b60 0x7f57cc14bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:54.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.695+0000 7f57d211a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc147190 con 0x7f57cc142410 2022-06-29T15:47:54.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.695+0000 7f57d211a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc13e6d0 con 0x7f57b0004e70 2022-06-29T15:47:54.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.695+0000 7f57d211a700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc13e080 con 0x7f57cc147b60 2022-06-29T15:47:54.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.695+0000 7f57d0917700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57cc142410 0x7f57cc147670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:54.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.695+0000 7f57d0917700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57cc142410 0x7f57cc147670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49520/0 (socket says 172.21.15.47:49520) 2022-06-29T15:47:54.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.695+0000 7f57d0917700 1 -- 172.21.15.47:0/1483579683 learned_addr learned my addr 172.21.15.47:0/1483579683 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:54.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.695+0000 7f57d1118700 1 --2- 172.21.15.47:0/1483579683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57b0004e70 0x7f57cc13db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:54.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.695+0000 7f57d1919700 1 --2- 172.21.15.47:0/1483579683 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f57cc147b60 0x7f57cc14bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:54.712 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.695+0000 7f57d0917700 1 -- 172.21.15.47:0/1483579683 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f57cc147b60 msgr2=0x7f57cc14bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:54.712 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.695+0000 7f57d0917700 1 --2- 172.21.15.47:0/1483579683 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f57cc147b60 0x7f57cc14bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:54.712 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.695+0000 7f57d0917700 1 -- 172.21.15.47:0/1483579683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57b0004e70 msgr2=0x7f57cc13db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:54.712 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.695+0000 7f57d0917700 1 --2- 172.21.15.47:0/1483579683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57b0004e70 0x7f57cc13db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:54.713 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.695+0000 7f57d0917700 1 -- 172.21.15.47:0/1483579683 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57cc13e930 con 0x7f57cc142410 2022-06-29T15:47:54.713 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.695+0000 7f57d1919700 1 --2- 172.21.15.47:0/1483579683 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f57cc147b60 0x7f57cc14bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:47:54.713 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.695+0000 7f57d0917700 1 --2- 172.21.15.47:0/1483579683 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57cc142410 0x7f57cc147670 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7f57cc13e210 tx=0x7f57bc009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:54.713 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.695+0000 7f57c27fc700 1 -- 172.21.15.47:0/1483579683 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f57bc015630 con 0x7f57cc142410 2022-06-29T15:47:54.714 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.695+0000 7f57c27fc700 1 -- 172.21.15.47:0/1483579683 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57bc010040 con 0x7f57cc142410 2022-06-29T15:47:54.714 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.695+0000 7f57d211a700 1 -- 172.21.15.47:0/1483579683 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57cc14e450 con 0x7f57cc142410 2022-06-29T15:47:54.714 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.695+0000 7f57c27fc700 1 -- 172.21.15.47:0/1483579683 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f57bc015980 con 0x7f57cc142410 2022-06-29T15:47:54.714 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.695+0000 7f57d211a700 1 -- 172.21.15.47:0/1483579683 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57cc146270 con 0x7f57cc142410 2022-06-29T15:47:54.714 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.695+0000 7f57d211a700 1 -- 172.21.15.47:0/1483579683 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57cc142a30 con 0x7f57cc142410 2022-06-29T15:47:54.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.699+0000 7f57c27fc700 1 -- 172.21.15.47:0/1483579683 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f57bc01c060 con 0x7f57cc142410 2022-06-29T15:47:54.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.699+0000 7f57c27fc700 1 --2- 172.21.15.47:0/1483579683 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f57b4053aa0 0x7f57b4055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:54.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.699+0000 7f57c27fc700 1 -- 172.21.15.47:0/1483579683 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f57bc06b960 con 0x7f57cc142410 2022-06-29T15:47:54.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.699+0000 7f57d1118700 1 --2- 172.21.15.47:0/1483579683 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f57b4053aa0 0x7f57b4055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:54.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.699+0000 7f57d1118700 1 --2- 172.21.15.47:0/1483579683 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f57b4053aa0 0x7f57b4055f60 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f57c8008000 tx=0x7f57c8008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:54.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.707+0000 7f57c27fc700 1 -- 172.21.15.47:0/1483579683 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f57bc042720 con 0x7f57cc142410 2022-06-29T15:47:54.871 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.871+0000 7f57d211a700 1 -- 172.21.15.47:0/1483579683 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f57cc0573d0 con 0x7f57cc142410 2022-06-29T15:47:54.872 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.871+0000 7f57c27fc700 1 -- 172.21.15.47:0/1483579683 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+2490 (secure 0 0 0) 0x7f57bc01a020 con 0x7f57cc142410 2022-06-29T15:47:54.873 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:47:54.873 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":30,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5887,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/929350416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":929350416},{"type":"v1","addr":"172.21.15.47:6835","nonce":929350416}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":27},{"gid":5970,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2425218664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2425218664},{"type":"v1","addr":"172.21.15.64:6833","nonce":2425218664}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":27}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:47:35.340236+0000","modified":"2022-06-29T15:47:36.343849+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5973":{"gid":5973,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6837/1166335240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1166335240},{"type":"v1","addr":"172.21.15.47:6837","nonce":1166335240}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2022-06-29T15:47:54.875 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.871+0000 7f57d211a700 1 -- 172.21.15.47:0/1483579683 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f57b4053aa0 msgr2=0x7f57b4055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:54.875 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.871+0000 7f57d211a700 1 --2- 172.21.15.47:0/1483579683 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f57b4053aa0 0x7f57b4055f60 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f57c8008000 tx=0x7f57c8008040).stop 2022-06-29T15:47:54.875 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.871+0000 7f57d211a700 1 -- 172.21.15.47:0/1483579683 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57cc142410 msgr2=0x7f57cc147670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:54.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.871+0000 7f57d211a700 1 --2- 172.21.15.47:0/1483579683 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57cc142410 0x7f57cc147670 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7f57cc13e210 tx=0x7f57bc009040).stop 2022-06-29T15:47:54.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.871+0000 7f57d211a700 1 -- 172.21.15.47:0/1483579683 shutdown_connections 2022-06-29T15:47:54.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.871+0000 7f57d211a700 1 --2- 172.21.15.47:0/1483579683 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f57cc147b60 0x7f57cc14bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:54.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.871+0000 7f57d211a700 1 --2- 172.21.15.47:0/1483579683 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57cc142410 0x7f57cc147670 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:54.877 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.871+0000 7f57d211a700 1 --2- 172.21.15.47:0/1483579683 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f57b4053aa0 0x7f57b4055f60 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:54.877 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.871+0000 7f57d211a700 1 --2- 172.21.15.47:0/1483579683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57b0004e70 0x7f57cc13db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:54.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.875+0000 7f57d211a700 1 -- 172.21.15.47:0/1483579683 >> 172.21.15.47:0/1483579683 conn(0x7f57cc062560 msgr2=0x7f57cc144750 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:54.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.875+0000 7f57d211a700 1 -- 172.21.15.47:0/1483579683 shutdown_connections 2022-06-29T15:47:54.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.875+0000 7f57d211a700 1 -- 172.21.15.47:0/1483579683 wait complete. 2022-06-29T15:47:54.879 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 30 2022-06-29T15:47:54.886 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a cache drop 10 2022-06-29T15:47:54.995 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.995+0000 7ff80d29d700 1 Processor -- start 2022-06-29T15:47:54.996 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.995+0000 7ff80d29d700 1 -- start start 2022-06-29T15:47:54.999 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.995+0000 7ff80d29d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff8080b64c0 0x7ff8080b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:54.999 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.995+0000 7ff80d29d700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8080b68a0 con 0x7ff808143150 2022-06-29T15:47:55.000 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.995+0000 7ff80d29d700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8081473d0 con 0x7ff808066fe0 2022-06-29T15:47:55.000 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.995+0000 7ff80d29d700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8080b13a0 con 0x7ff8080b64c0 2022-06-29T15:47:55.001 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.995+0000 7ff80ca9c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff808143150 0x7ff80812bec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35084/0 (socket says 172.21.15.47:35084) 2022-06-29T15:47:55.001 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.995+0000 7ff80ca9c700 1 -- 172.21.15.47:0/1294005404 learned_addr learned my addr 172.21.15.47:0/1294005404 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:47:55.001 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.995+0000 7ff807fff700 1 --2- 172.21.15.47:0/1294005404 >> v2:172.21.15.64:3300/0 conn(0x7ff8080b64c0 0x7ff8080b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:55.001 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.995+0000 7ff806ffd700 1 -- 172.21.15.47:0/1294005404 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2221805911 0 0) 0x7ff8080b68a0 con 0x7ff808143150 2022-06-29T15:47:55.002 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.995+0000 7ff806ffd700 1 -- 172.21.15.47:0/1294005404 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7f4003260 con 0x7ff808143150 2022-06-29T15:47:55.002 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.995+0000 7ff806ffd700 1 -- 172.21.15.47:0/1294005404 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3929582075 0 0) 0x7ff8081473d0 con 0x7ff808066fe0 2022-06-29T15:47:55.002 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.995+0000 7ff806ffd700 1 -- 172.21.15.47:0/1294005404 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8080b68a0 con 0x7ff808066fe0 2022-06-29T15:47:55.002 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.995+0000 7ff807fff700 1 -- 172.21.15.47:0/1294005404 >> v1:172.21.15.64:6789/0 conn(0x7ff808066fe0 legacy=0x7ff8080b4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:55.003 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.995+0000 7ff807fff700 1 -- 172.21.15.47:0/1294005404 >> v1:172.21.15.47:6789/0 conn(0x7ff808143150 legacy=0x7ff80812bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:55.003 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.995+0000 7ff807fff700 1 -- 172.21.15.47:0/1294005404 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8080b46a0 con 0x7ff8080b64c0 2022-06-29T15:47:55.003 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.995+0000 7ff807fff700 1 --2- 172.21.15.47:0/1294005404 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8080b64c0 0x7ff8080b5a20 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7ff7f8002f70 tx=0x7ff7f8006df0).ready entity=mon.1 client_cookie=c8e586b30649e74d server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:55.003 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.995+0000 7ff806ffd700 1 -- 172.21.15.47:0/1294005404 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff7f800d540 con 0x7ff8080b64c0 2022-06-29T15:47:55.004 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.995+0000 7ff806ffd700 1 -- 172.21.15.47:0/1294005404 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7f8000e90 con 0x7ff8080b64c0 2022-06-29T15:47:55.004 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.995+0000 7ff806ffd700 1 -- 172.21.15.47:0/1294005404 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff7f800d890 con 0x7ff8080b64c0 2022-06-29T15:47:55.009 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.999+0000 7ff80d29d700 1 -- 172.21.15.47:0/1294005404 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8080b64c0 msgr2=0x7ff8080b5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:55.010 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.999+0000 7ff80d29d700 1 --2- 172.21.15.47:0/1294005404 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8080b64c0 0x7ff8080b5a20 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7ff7f8002f70 tx=0x7ff7f8006df0).stop 2022-06-29T15:47:55.010 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.999+0000 7ff80d29d700 1 -- 172.21.15.47:0/1294005404 shutdown_connections 2022-06-29T15:47:55.010 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.999+0000 7ff80d29d700 1 --2- 172.21.15.47:0/1294005404 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8080b64c0 0x7ff8080b5a20 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:55.011 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.999+0000 7ff80d29d700 1 -- 172.21.15.47:0/1294005404 >> 172.21.15.47:0/1294005404 conn(0x7ff808060e00 msgr2=0x7ff8080611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:55.011 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.999+0000 7ff80d29d700 1 -- 172.21.15.47:0/1294005404 shutdown_connections 2022-06-29T15:47:55.011 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.999+0000 7ff80d29d700 1 -- 172.21.15.47:0/1294005404 wait complete. 2022-06-29T15:47:55.011 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.999+0000 7ff80d29d700 1 Processor -- start 2022-06-29T15:47:55.011 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:54.999+0000 7ff80d29d700 1 -- start start 2022-06-29T15:47:55.012 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.003+0000 7ff80d29d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff808066fe0 0x7ff80814a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:55.012 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.003+0000 7ff80d29d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8080b64c0 0x7ff8081520b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:55.012 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.003+0000 7ff80d29d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff808143150 0x7ff808127a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:55.012 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.003+0000 7ff80d29d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8080b3460 con 0x7ff808143150 2022-06-29T15:47:55.013 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.003+0000 7ff80d29d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8080b2640 con 0x7ff8080b64c0 2022-06-29T15:47:55.013 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.003+0000 7ff80d29d700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8080b23a0 con 0x7ff808066fe0 2022-06-29T15:47:55.013 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.003+0000 7ff80ca9c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff808143150 0x7ff808127a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:55.013 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.003+0000 7ff80ca9c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff808143150 0x7ff808127a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49536/0 (socket says 172.21.15.47:49536) 2022-06-29T15:47:55.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.003+0000 7ff80ca9c700 1 -- 172.21.15.47:0/1335008474 learned_addr learned my addr 172.21.15.47:0/1335008474 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:55.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.003+0000 7ff807fff700 1 --2- 172.21.15.47:0/1335008474 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff808066fe0 0x7ff80814a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:55.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.003+0000 7ff8077fe700 1 --2- 172.21.15.47:0/1335008474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8080b64c0 0x7ff8081520b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:55.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.003+0000 7ff80ca9c700 1 -- 172.21.15.47:0/1335008474 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff808066fe0 msgr2=0x7ff80814a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:55.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.003+0000 7ff80ca9c700 1 --2- 172.21.15.47:0/1335008474 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff808066fe0 0x7ff80814a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:55.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.003+0000 7ff80ca9c700 1 -- 172.21.15.47:0/1335008474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8080b64c0 msgr2=0x7ff8081520b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:55.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.003+0000 7ff80ca9c700 1 --2- 172.21.15.47:0/1335008474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8080b64c0 0x7ff8081520b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:55.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.003+0000 7ff80ca9c700 1 -- 172.21.15.47:0/1335008474 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f800b180 con 0x7ff808143150 2022-06-29T15:47:55.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.003+0000 7ff80ca9c700 1 --2- 172.21.15.47:0/1335008474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff808143150 0x7ff808127a90 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7ff800001490 tx=0x7ff800007de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:55.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.003+0000 7ff8057fa700 1 -- 172.21.15.47:0/1335008474 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff800005ba0 con 0x7ff808143150 2022-06-29T15:47:55.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.003+0000 7ff8057fa700 1 -- 172.21.15.47:0/1335008474 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff800005d00 con 0x7ff808143150 2022-06-29T15:47:55.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.003+0000 7ff8057fa700 1 -- 172.21.15.47:0/1335008474 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8000072c0 con 0x7ff808143150 2022-06-29T15:47:55.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.003+0000 7ff80d29d700 1 -- 172.21.15.47:0/1335008474 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff80814a660 con 0x7ff808143150 2022-06-29T15:47:55.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.003+0000 7ff80d29d700 1 -- 172.21.15.47:0/1335008474 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff808148320 con 0x7ff808143150 2022-06-29T15:47:55.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.007+0000 7ff8057fa700 1 -- 172.21.15.47:0/1335008474 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff800010070 con 0x7ff808143150 2022-06-29T15:47:55.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.007+0000 7ff8057fa700 1 --2- 172.21.15.47:0/1335008474 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff7e8053a70 0x7ff7e8055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:55.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.007+0000 7ff8057fa700 1 -- 172.21.15.47:0/1335008474 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff80006b8f0 con 0x7ff808143150 2022-06-29T15:47:55.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.007+0000 7ff807fff700 1 --2- 172.21.15.47:0/1335008474 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff7e8053a70 0x7ff7e8055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:55.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.007+0000 7ff807fff700 1 --2- 172.21.15.47:0/1335008474 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff7e8053a70 0x7ff7e8055f30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7ff7f8018000 tx=0x7ff7f8014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:55.029 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.007+0000 7ff818754740 1 Processor -- start 2022-06-29T15:47:55.030 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.011+0000 7ff818754740 1 -- start start 2022-06-29T15:47:55.030 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.011+0000 7ff818754740 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x187df10 0x18b6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:55.031 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.011+0000 7ff818754740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x18b2960 0x18b2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:55.031 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.011+0000 7ff818754740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x18b32d0 0x187ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:55.032 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.011+0000 7ff818754740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x17dd2d0 con 0x18b32d0 2022-06-29T15:47:55.032 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.011+0000 7ff818754740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x17f0ef0 con 0x18b2960 2022-06-29T15:47:55.032 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.011+0000 7ff818754740 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x17d11a0 con 0x187df10 2022-06-29T15:47:55.032 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.011+0000 7ff80ca9c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x18b32d0 0x187ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:55.032 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.011+0000 7ff8077fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x18b2960 0x18b2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:55.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.011+0000 7ff8077fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x18b2960 0x18b2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:55246/0 (socket says 172.21.15.47:55246) 2022-06-29T15:47:55.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.011+0000 7ff807fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x187df10 0x18b6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:55.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.011+0000 7ff8077fe700 1 -- 172.21.15.47:0/1816851958 learned_addr learned my addr 172.21.15.47:0/1816851958 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:55.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.011+0000 7ff80ca9c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x18b32d0 0x187ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49544/0 (socket says 172.21.15.47:49544) 2022-06-29T15:47:55.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.011+0000 7ff80ca9c700 1 -- 172.21.15.47:0/1816851958 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x187df10 msgr2=0x18b6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:55.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.011+0000 7ff80ca9c700 1 --2- 172.21.15.47:0/1816851958 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x187df10 0x18b6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:55.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.011+0000 7ff80ca9c700 1 -- 172.21.15.47:0/1816851958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x18b2960 msgr2=0x18b2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:55.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.011+0000 7ff80ca9c700 1 --2- 172.21.15.47:0/1816851958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x18b2960 0x18b2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:55.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.011+0000 7ff80ca9c700 1 -- 172.21.15.47:0/1816851958 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1828df0 con 0x18b32d0 2022-06-29T15:47:55.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.011+0000 7ff80ca9c700 1 --2- 172.21.15.47:0/1816851958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x18b32d0 0x187ae00 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x187fd80 tx=0x7ff800010000).ready entity=mon.0 client_cookie=4955ae2a8ae5699b server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:55.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.011+0000 7ff80d29d700 1 -- 172.21.15.47:0/1816851958 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff800046310 con 0x18b32d0 2022-06-29T15:47:55.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.011+0000 7ff80d29d700 1 -- 172.21.15.47:0/1816851958 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff800044710 con 0x18b32d0 2022-06-29T15:47:55.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.011+0000 7ff80d29d700 1 -- 172.21.15.47:0/1816851958 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff800042160 con 0x18b32d0 2022-06-29T15:47:55.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.011+0000 7ff818754740 1 -- 172.21.15.47:0/1816851958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x18b32d0 msgr2=0x187ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:55.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.011+0000 7ff818754740 1 --2- 172.21.15.47:0/1816851958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x18b32d0 0x187ae00 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x187fd80 tx=0x7ff800010000).stop 2022-06-29T15:47:55.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.011+0000 7ff818754740 1 -- 172.21.15.47:0/1816851958 shutdown_connections 2022-06-29T15:47:55.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.011+0000 7ff818754740 1 --2- 172.21.15.47:0/1816851958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x18b32d0 0x187ae00 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:55.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.011+0000 7ff818754740 1 --2- 172.21.15.47:0/1816851958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x18b2960 0x18b2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:55.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.011+0000 7ff818754740 1 --2- 172.21.15.47:0/1816851958 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x187df10 0x18b6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:55.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.011+0000 7ff818754740 1 -- 172.21.15.47:0/1816851958 >> 172.21.15.47:0/1816851958 conn(0x187cb10 msgr2=0x187e930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:55.038 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.011+0000 7ff818754740 1 -- 172.21.15.47:0/1816851958 shutdown_connections 2022-06-29T15:47:55.038 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.011+0000 7ff818754740 1 -- 172.21.15.47:0/1816851958 wait complete. 2022-06-29T15:47:55.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.015+0000 7ff818754740 1 -- start start 2022-06-29T15:47:55.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.015+0000 7ff818754740 1 Processor -- start 2022-06-29T15:47:55.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.015+0000 7ff818754740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x18b2960 0x18b1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:55.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.015+0000 7ff818754740 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x18b32d0 0x18b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:55.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.015+0000 7ff818754740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x1885cc0 0x18c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:55.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.015+0000 7ff818754740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x18775e0 con 0x1885cc0 2022-06-29T15:47:55.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.015+0000 7ff818754740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x18c50a0 con 0x18b2960 2022-06-29T15:47:55.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.015+0000 7ff818754740 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x18c5340 con 0x18b32d0 2022-06-29T15:47:55.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.015+0000 7ff818754740 10 client.0 ll_register_callbacks cb 0x18383d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2022-06-29T15:47:55.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.015+0000 7ff80ca9c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x1885cc0 0x18c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:55.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.015+0000 7ff8077fe700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x18b32d0 0x18b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:55.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.015+0000 7ff807fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x18b2960 0x18b1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:55.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.015+0000 7ff80ca9c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x1885cc0 0x18c5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49550/0 (socket says 172.21.15.47:49550) 2022-06-29T15:47:55.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.015+0000 7ff80ca9c700 1 -- 172.21.15.47:0/750892690 learned_addr learned my addr 172.21.15.47:0/750892690 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:55.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.015+0000 7ff80ca9c700 1 -- 172.21.15.47:0/750892690 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x18b32d0 msgr2=0x18b21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:55.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.015+0000 7ff80ca9c700 1 --2- 172.21.15.47:0/750892690 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x18b32d0 0x18b21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:55.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.015+0000 7ff80ca9c700 1 -- 172.21.15.47:0/750892690 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x18b2960 msgr2=0x18b1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:55.045 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.015+0000 7ff80ca9c700 1 --2- 172.21.15.47:0/750892690 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x18b2960 0x18b1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:55.045 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.015+0000 7ff80ca9c700 1 -- 172.21.15.47:0/750892690 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff808148320 con 0x1885cc0 2022-06-29T15:47:55.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.015+0000 7ff80ca9c700 1 --2- 172.21.15.47:0/750892690 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x1885cc0 0x18c5bd0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x187fd80 tx=0x7ff80006a000).ready entity=mon.0 client_cookie=45ba33a270422e3 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:55.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.015+0000 7ff7e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2022-06-29T15:47:55.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.015+0000 7ff7e6ffd700 1 -- 172.21.15.47:0/750892690 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff800007a90 con 0x1885cc0 2022-06-29T15:47:55.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.015+0000 7ff7e6ffd700 1 -- 172.21.15.47:0/750892690 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(68..68 src has 1..68) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff800046310 con 0x1885cc0 2022-06-29T15:47:55.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.015+0000 7ff818754740 1 -- 172.21.15.47:0/750892690 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x17c66c0 con 0x1885cc0 2022-06-29T15:47:55.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.015+0000 7ff7e6ffd700 1 -- 172.21.15.47:0/750892690 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff800007630 con 0x1885cc0 2022-06-29T15:47:55.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.015+0000 7ff818754740 10 client.6168 fetch_fsmap learned FSMap version 30 2022-06-29T15:47:55.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.015+0000 7ff818754740 1 -- 172.21.15.47:0/750892690 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) v3 -- 0x17e03d0 con 0x1885cc0 2022-06-29T15:47:55.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.019+0000 7ff7e6ffd700 1 -- 172.21.15.47:0/750892690 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x17c66c0 con 0x1885cc0 2022-06-29T15:47:55.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.019+0000 7ff7e6ffd700 1 -- 172.21.15.47:0/750892690 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) v1 ==== 1434+0+0 (secure 0 0 0) 0x7ff80005a020 con 0x1885cc0 2022-06-29T15:47:55.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.019+0000 7ff818754740 10 client.6168 fetch_fsmap finished waiting for FSMap version 30 2022-06-29T15:47:55.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.019+0000 7ff818754740 10 client.6168 resolve_mds: resolved ID 'a' to GID 5973 2022-06-29T15:47:55.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.019+0000 7ff818754740 20 client.6168 populate_metadata read hostname 'smithi047' 2022-06-29T15:47:55.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.019+0000 7ff818754740 1 --2- 172.21.15.47:0/750892690 >> [v2:172.21.15.47:6836/1166335240,v1:172.21.15.47:6837/1166335240] conn(0x18c6590 0x18c7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:55.052 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.019+0000 7ff818754740 4 client.6168 mds_command: new command op to 5973 tid=0[{"prefix": "get_command_descriptions"}] 2022-06-29T15:47:55.052 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.019+0000 7ff818754740 1 -- 172.21.15.47:0/750892690 --> [v2:172.21.15.47:6836/1166335240,v1:172.21.15.47:6837/1166335240] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x17e6de0 con 0x18c6590 2022-06-29T15:47:55.053 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.019+0000 7ff807fff700 1 --2- 172.21.15.47:0/750892690 >> [v2:172.21.15.47:6836/1166335240,v1:172.21.15.47:6837/1166335240] conn(0x18c6590 0x18c7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:55.053 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.019+0000 7ff7e6ffd700 10 client.6168 ms_handle_connect on v2:172.21.15.47:6836/1166335240 2022-06-29T15:47:55.053 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.019+0000 7ff807fff700 1 --2- 172.21.15.47:0/750892690 >> [v2:172.21.15.47:6836/1166335240,v1:172.21.15.47:6837/1166335240] conn(0x18c6590 0x18c7bb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=d94231942c84c1c5 server_cookie=780c0f90df75764c in_seq=0 out_seq=0 2022-06-29T15:47:55.054 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.019+0000 7ff7e6ffd700 1 -- 172.21.15.47:0/750892690 <== mds.0 v2:172.21.15.47:6836/1166335240 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x17e6de0 con 0x18c6590 2022-06-29T15:47:55.054 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.019+0000 7ff7e6ffd700 10 client.6168 handle_command_reply: tid=0 2022-06-29T15:47:55.055 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.019+0000 7ff818754740 1 client.6168 shutdown 2022-06-29T15:47:55.055 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.019+0000 7ff818754740 1 -- 172.21.15.47:0/750892690 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x1885cc0 msgr2=0x18c5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:55.055 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.019+0000 7ff818754740 1 --2- 172.21.15.47:0/750892690 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x1885cc0 0x18c5bd0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x187fd80 tx=0x7ff80006a000).stop 2022-06-29T15:47:55.056 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.019+0000 7ff818754740 1 -- 172.21.15.47:0/750892690 shutdown_connections 2022-06-29T15:47:55.056 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.019+0000 7ff818754740 1 --2- 172.21.15.47:0/750892690 >> [v2:172.21.15.47:6836/1166335240,v1:172.21.15.47:6837/1166335240] conn(0x18c6590 0x18c7bb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:55.056 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.019+0000 7ff818754740 1 --2- 172.21.15.47:0/750892690 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x1885cc0 0x18c5bd0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:55.057 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.019+0000 7ff818754740 1 --2- 172.21.15.47:0/750892690 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x18b32d0 0x18b21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:55.057 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.019+0000 7ff7e6ffd700 0 client.6168 ms_handle_reset on v2:172.21.15.47:6836/1166335240 2022-06-29T15:47:55.058 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.019+0000 7ff818754740 1 --2- 172.21.15.47:0/750892690 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x18b2960 0x18b1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:55.058 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.019+0000 7ff818754740 1 -- 172.21.15.47:0/750892690 >> 172.21.15.47:0/750892690 conn(0x187df10 msgr2=0x18b9600 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:55.058 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.019+0000 7ff818754740 1 -- 172.21.15.47:0/750892690 shutdown_connections 2022-06-29T15:47:55.059 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.019+0000 7ff818754740 1 -- 172.21.15.47:0/750892690 wait complete. 2022-06-29T15:47:55.059 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.019+0000 7ff818754740 20 client.6168 trim_cache size 0 max 16384 2022-06-29T15:47:55.070 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.035+0000 7ff818754740 1 Processor -- start 2022-06-29T15:47:55.071 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.035+0000 7ff818754740 1 -- start start 2022-06-29T15:47:55.071 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.035+0000 7ff818754740 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x1885cc0 0x18b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:55.072 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.035+0000 7ff818754740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x18b2960 0x18c1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:55.072 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.035+0000 7ff818754740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x18b32d0 0x18c1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:55.072 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.035+0000 7ff818754740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x17e6de0 con 0x18b32d0 2022-06-29T15:47:55.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.035+0000 7ff818754740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x1844770 con 0x18b2960 2022-06-29T15:47:55.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.035+0000 7ff818754740 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x187ec00 con 0x1885cc0 2022-06-29T15:47:55.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.035+0000 7ff80ca9c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x18b32d0 0x18c1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:55.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.035+0000 7ff80ca9c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x18b32d0 0x18c1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49558/0 (socket says 172.21.15.47:49558) 2022-06-29T15:47:55.074 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.035+0000 7ff80ca9c700 1 -- 172.21.15.47:0/393673692 learned_addr learned my addr 172.21.15.47:0/393673692 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:55.074 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.035+0000 7ff8077fe700 1 --2- 172.21.15.47:0/393673692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x18b2960 0x18c1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:55.074 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.035+0000 7ff807fff700 1 --2- 172.21.15.47:0/393673692 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x1885cc0 0x18b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:55.074 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.039+0000 7ff80ca9c700 1 -- 172.21.15.47:0/393673692 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x1885cc0 msgr2=0x18b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:55.075 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.039+0000 7ff80ca9c700 1 --2- 172.21.15.47:0/393673692 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x1885cc0 0x18b1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:55.075 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.039+0000 7ff80ca9c700 1 -- 172.21.15.47:0/393673692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x18b2960 msgr2=0x18c1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:55.075 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.039+0000 7ff80ca9c700 1 --2- 172.21.15.47:0/393673692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x18b2960 0x18c1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:55.075 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.039+0000 7ff80ca9c700 1 -- 172.21.15.47:0/393673692 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x17e8fc0 con 0x18b32d0 2022-06-29T15:47:55.076 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.039+0000 7ff80ca9c700 1 --2- 172.21.15.47:0/393673692 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x18b32d0 0x18c1f70 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x17f8030 tx=0x7ff800055040).ready entity=mon.0 client_cookie=78f2e07bbf161c27 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:55.076 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.039+0000 7ff7ed7fa700 1 -- 172.21.15.47:0/393673692 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff800007a90 con 0x18b32d0 2022-06-29T15:47:55.076 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.039+0000 7ff7ed7fa700 1 -- 172.21.15.47:0/393673692 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff800046310 con 0x18b32d0 2022-06-29T15:47:55.076 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.039+0000 7ff7ed7fa700 1 -- 172.21.15.47:0/393673692 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff80006b420 con 0x18b32d0 2022-06-29T15:47:55.076 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.039+0000 7ff818754740 1 -- 172.21.15.47:0/393673692 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x18b32d0 msgr2=0x18c1f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:47:55.077 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.039+0000 7ff818754740 1 --2- 172.21.15.47:0/393673692 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x18b32d0 0x18c1f70 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x17f8030 tx=0x7ff800055040).stop 2022-06-29T15:47:55.077 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.039+0000 7ff818754740 1 -- 172.21.15.47:0/393673692 shutdown_connections 2022-06-29T15:47:55.077 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.039+0000 7ff818754740 1 --2- 172.21.15.47:0/393673692 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x18b32d0 0x18c1f70 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:55.077 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.039+0000 7ff818754740 1 --2- 172.21.15.47:0/393673692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x18b2960 0x18c1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:55.078 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.039+0000 7ff818754740 1 --2- 172.21.15.47:0/393673692 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x1885cc0 0x18b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:55.078 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.039+0000 7ff818754740 1 -- 172.21.15.47:0/393673692 >> 172.21.15.47:0/393673692 conn(0x187df10 msgr2=0x18b9480 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:47:55.078 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.039+0000 7ff818754740 1 -- 172.21.15.47:0/393673692 shutdown_connections 2022-06-29T15:47:55.078 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.039+0000 7ff818754740 1 -- 172.21.15.47:0/393673692 wait complete. 2022-06-29T15:47:55.079 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.039+0000 7ff818754740 1 -- start start 2022-06-29T15:47:55.079 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.039+0000 7ff818754740 1 Processor -- start 2022-06-29T15:47:55.079 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.039+0000 7ff818754740 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x1885cc0 0x1911a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:55.079 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.039+0000 7ff818754740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x18b2960 0x1911f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:55.079 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.039+0000 7ff818754740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x18b32d0 0x1919250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:55.080 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.039+0000 7ff818754740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x1918830 con 0x18b32d0 2022-06-29T15:47:55.080 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.039+0000 7ff818754740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x18b8550 con 0x18b2960 2022-06-29T15:47:55.080 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.039+0000 7ff818754740 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x1915980 con 0x1885cc0 2022-06-29T15:47:55.080 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.039+0000 7ff818754740 10 client.0 ll_register_callbacks cb 0x18b4760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2022-06-29T15:47:55.081 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.039+0000 7ff80ca9c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x18b32d0 0x1919250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:55.081 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.039+0000 7ff80ca9c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x18b32d0 0x1919250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49564/0 (socket says 172.21.15.47:49564) 2022-06-29T15:47:55.081 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.039+0000 7ff80ca9c700 1 -- 172.21.15.47:0/2149667242 learned_addr learned my addr 172.21.15.47:0/2149667242 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:47:55.081 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.039+0000 7ff8077fe700 1 --2- 172.21.15.47:0/2149667242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x18b2960 0x1911f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:55.082 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.039+0000 7ff807fff700 1 --2- 172.21.15.47:0/2149667242 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x1885cc0 0x1911a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:55.082 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.043+0000 7ff80ca9c700 1 -- 172.21.15.47:0/2149667242 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x1885cc0 msgr2=0x1911a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:55.082 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.043+0000 7ff80ca9c700 1 --2- 172.21.15.47:0/2149667242 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x1885cc0 0x1911a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:55.082 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.043+0000 7ff80ca9c700 1 -- 172.21.15.47:0/2149667242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x18b2960 msgr2=0x1911f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:47:55.082 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.043+0000 7ff80ca9c700 1 --2- 172.21.15.47:0/2149667242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x18b2960 0x1911f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:47:55.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.043+0000 7ff80ca9c700 1 -- 172.21.15.47:0/2149667242 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x17e03d0 con 0x18b32d0 2022-06-29T15:47:55.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.043+0000 7ff80ca9c700 1 --2- 172.21.15.47:0/2149667242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x18b32d0 0x1919250 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x17f8030 tx=0x7ff800054040).ready entity=mon.0 client_cookie=845db708b7c29546 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:47:55.084 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.043+0000 7ff7e7fff700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2022-06-29T15:47:55.084 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.043+0000 7ff7e7fff700 1 -- 172.21.15.47:0/2149667242 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff800007a90 con 0x18b32d0 2022-06-29T15:47:55.084 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.043+0000 7ff7e7fff700 1 -- 172.21.15.47:0/2149667242 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(68..68 src has 1..68) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff80005ddb0 con 0x18b32d0 2022-06-29T15:47:55.084 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.043+0000 7ff818754740 1 -- 172.21.15.47:0/2149667242 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff800044710 con 0x18b32d0 2022-06-29T15:47:55.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.043+0000 7ff7e7fff700 1 -- 172.21.15.47:0/2149667242 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff800045cd0 con 0x18b32d0 2022-06-29T15:47:55.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.043+0000 7ff818754740 10 client.6174 fetch_fsmap learned FSMap version 30 2022-06-29T15:47:55.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.043+0000 7ff818754740 1 -- 172.21.15.47:0/2149667242 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) v3 -- 0x17e59b0 con 0x18b32d0 2022-06-29T15:47:55.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.043+0000 7ff7e7fff700 1 -- 172.21.15.47:0/2149667242 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff800044710 con 0x18b32d0 2022-06-29T15:47:55.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.043+0000 7ff7e7fff700 1 -- 172.21.15.47:0/2149667242 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) v1 ==== 1434+0+0 (secure 0 0 0) 0x7ff80005a040 con 0x18b32d0 2022-06-29T15:47:55.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.043+0000 7ff818754740 10 client.6174 fetch_fsmap finished waiting for FSMap version 30 2022-06-29T15:47:55.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.043+0000 7ff818754740 10 client.6174 resolve_mds: resolved ID 'a' to GID 5973 2022-06-29T15:47:55.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.043+0000 7ff818754740 20 client.6174 populate_metadata read hostname 'smithi047' 2022-06-29T15:47:55.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.043+0000 7ff818754740 1 --2- 172.21.15.47:0/2149667242 >> [v2:172.21.15.47:6836/1166335240,v1:172.21.15.47:6837/1166335240] conn(0x1919740 0x1919b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:47:55.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.043+0000 7ff818754740 4 client.6174 mds_command: new command op to 5973 tid=0[{"prefix": "cache drop", "timeout": 10}] 2022-06-29T15:47:55.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.043+0000 7ff818754740 1 -- 172.21.15.47:0/2149667242 --> [v2:172.21.15.47:6836/1166335240,v1:172.21.15.47:6837/1166335240] -- command(tid 0: {"prefix": "cache drop", "timeout": 10}) v1 -- 0x17dee60 con 0x1919740 2022-06-29T15:47:55.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.043+0000 7ff807fff700 1 --2- 172.21.15.47:0/2149667242 >> [v2:172.21.15.47:6836/1166335240,v1:172.21.15.47:6837/1166335240] conn(0x1919740 0x1919b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:47:55.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.043+0000 7ff7e7fff700 10 client.6174 ms_handle_connect on v2:172.21.15.47:6836/1166335240 2022-06-29T15:47:55.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:47:55.043+0000 7ff807fff700 1 --2- 172.21.15.47:0/2149667242 >> [v2:172.21.15.47:6836/1166335240,v1:172.21.15.47:6837/1166335240] conn(0x1919740 0x1919b20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=bcb7e6dd284ee34c server_cookie=29d61d0fa8c40331 in_seq=0 out_seq=0 2022-06-29T15:48:05.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.267+0000 7ff7e7fff700 1 -- 172.21.15.47:0/2149667242 <== mds.0 v2:172.21.15.47:6836/1166335240 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+400 (crc 0 0 0) 0x17dee60 con 0x1919740 2022-06-29T15:48:05.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.267+0000 7ff7e7fff700 10 client.6174 handle_command_reply: tid=0 2022-06-29T15:48:05.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.267+0000 7ff818754740 1 client.6174 shutdown 2022-06-29T15:48:05.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.267+0000 7ff818754740 1 -- 172.21.15.47:0/2149667242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x18b32d0 msgr2=0x1919250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:05.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.267+0000 7ff818754740 1 --2- 172.21.15.47:0/2149667242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x18b32d0 0x1919250 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x17f8030 tx=0x7ff800054040).stop 2022-06-29T15:48:05.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.267+0000 7ff818754740 1 -- 172.21.15.47:0/2149667242 shutdown_connections 2022-06-29T15:48:05.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.267+0000 7ff818754740 1 --2- 172.21.15.47:0/2149667242 >> [v2:172.21.15.47:6836/1166335240,v1:172.21.15.47:6837/1166335240] conn(0x1919740 0x1919b20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:05.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.267+0000 7ff818754740 1 --2- 172.21.15.47:0/2149667242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x18b32d0 0x1919250 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:05.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.267+0000 7ff818754740 1 --2- 172.21.15.47:0/2149667242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x18b2960 0x1911f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:05.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.267+0000 7ff818754740 1 --2- 172.21.15.47:0/2149667242 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x1885cc0 0x1911a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:05.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.267+0000 7ff818754740 1 -- 172.21.15.47:0/2149667242 >> 172.21.15.47:0/2149667242 conn(0x187df10 msgr2=0x18b1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:05.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.267+0000 7ff7e7fff700 0 client.6174 ms_handle_reset on v2:172.21.15.47:6836/1166335240 2022-06-29T15:48:05.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.267+0000 7ff818754740 1 -- 172.21.15.47:0/2149667242 shutdown_connections 2022-06-29T15:48:05.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.267+0000 7ff818754740 1 -- 172.21.15.47:0/2149667242 wait complete. 2022-06-29T15:48:05.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.267+0000 7ff818754740 20 client.6174 trim_cache size 0 max 16384 2022-06-29T15:48:05.274 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2022-06-29T15:48:05.274 INFO:teuthology.orchestra.run.smithi047.stdout: "client_recall": { 2022-06-29T15:48:05.275 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": -110, 2022-06-29T15:48:05.275 INFO:teuthology.orchestra.run.smithi047.stdout: "message": "(110) Connection timed out", 2022-06-29T15:48:05.275 INFO:teuthology.orchestra.run.smithi047.stdout: "recalled": 240 2022-06-29T15:48:05.275 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:05.276 INFO:teuthology.orchestra.run.smithi047.stdout: "flush_journal": { 2022-06-29T15:48:05.276 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2022-06-29T15:48:05.276 INFO:teuthology.orchestra.run.smithi047.stdout: "message": "" 2022-06-29T15:48:05.276 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:05.277 INFO:teuthology.orchestra.run.smithi047.stdout: "trim_cache": { 2022-06-29T15:48:05.277 INFO:teuthology.orchestra.run.smithi047.stdout: "trimmed": 220 2022-06-29T15:48:05.277 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:05.277 INFO:teuthology.orchestra.run.smithi047.stdout: "cache": { 2022-06-29T15:48:05.278 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": { 2022-06-29T15:48:05.278 INFO:teuthology.orchestra.run.smithi047.stdout: "items": 15436, 2022-06-29T15:48:05.278 INFO:teuthology.orchestra.run.smithi047.stdout: "bytes": 2428368 2022-06-29T15:48:05.278 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:05.279 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:05.279 INFO:teuthology.orchestra.run.smithi047.stdout: "duration": 10.224316341 2022-06-29T15:48:05.279 INFO:teuthology.orchestra.run.smithi047.stdout:} 2022-06-29T15:48:05.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.271+0000 7ff80d29d700 1 -- 172.21.15.47:0/1335008474 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff7e8053a70 msgr2=0x7ff7e8055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:05.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.271+0000 7ff80d29d700 1 --2- 172.21.15.47:0/1335008474 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff7e8053a70 0x7ff7e8055f30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7ff7f8018000 tx=0x7ff7f8014000).stop 2022-06-29T15:48:05.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.271+0000 7ff80d29d700 1 -- 172.21.15.47:0/1335008474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff808143150 msgr2=0x7ff808127a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:05.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.271+0000 7ff80d29d700 1 --2- 172.21.15.47:0/1335008474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff808143150 0x7ff808127a90 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7ff800001490 tx=0x7ff800007de0).stop 2022-06-29T15:48:05.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.271+0000 7ff80d29d700 1 -- 172.21.15.47:0/1335008474 shutdown_connections 2022-06-29T15:48:05.282 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.271+0000 7ff80d29d700 1 --2- 172.21.15.47:0/1335008474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff808143150 0x7ff808127a90 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:05.282 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.271+0000 7ff80d29d700 1 --2- 172.21.15.47:0/1335008474 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff7e8053a70 0x7ff7e8055f30 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:05.282 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.271+0000 7ff80d29d700 1 --2- 172.21.15.47:0/1335008474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8080b64c0 0x7ff8081520b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:05.282 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.271+0000 7ff80d29d700 1 --2- 172.21.15.47:0/1335008474 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff808066fe0 0x7ff80814a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:05.283 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.271+0000 7ff80d29d700 1 -- 172.21.15.47:0/1335008474 >> 172.21.15.47:0/1335008474 conn(0x7ff808060e00 msgr2=0x7ff8080611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:05.283 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.271+0000 7ff80d29d700 1 -- 172.21.15.47:0/1335008474 shutdown_connections 2022-06-29T15:48:05.283 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.271+0000 7ff80d29d700 1 -- 172.21.15.47:0/1335008474 wait complete. 2022-06-29T15:48:05.285 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-06-29T15:48:05.285 DEBUG:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:48:05.301 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:48:05.321 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:48:05.344 DEBUG:teuthology.orchestra.run.smithi164:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:48:05.366 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-06-29T15:48:05.366 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:48:05.367 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:48:05.367 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:48:05.395 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2022-06-29T15:48:05.397 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-29T15:48:05.397 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi099.front.sepia.ceph.com... 2022-06-29T15:48:05.398 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-06-29T15:48:05.399 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:48:05.416 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[12745]: fuse finished with error 0 and tester_r 0 2022-06-29T15:48:05.418 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:48:05.418 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:48:05.419 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:48:05.469 INFO:teuthology.orchestra.run.smithi099.stdout:ext2/ext3 2022-06-29T15:48:05.470 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-06-29T15:48:05.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-29T15:48:05.588 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:48:05.596 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:48:05.596 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:48:05.597 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:48:05.604 INFO:teuthology.orchestra.run.smithi164.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-06-29T15:48:05.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T15:48:05.606 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-06-29T15:48:05.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-29T15:48:05.607 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:48:05.654 INFO:teuthology.orchestra.run.smithi164.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-06-29T15:48:05.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T15:48:05.657 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-06-29T15:48:05.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.771+0000 7feb0759e700 1 Processor -- start 2022-06-29T15:48:05.772 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.771+0000 7feb0759e700 1 -- start start 2022-06-29T15:48:05.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.771+0000 7feb0759e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7feb080ad970 0x7feb080b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:05.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.771+0000 7feb0759e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7feb080b0e50 0x7feb0813d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:05.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.771+0000 7feb0759e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb081400e0 con 0x7feb080b2ee0 2022-06-29T15:48:05.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.771+0000 7feb0759e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb08140ef0 con 0x7feb080ad970 2022-06-29T15:48:05.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.771+0000 7feb0759e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb0813fef0 con 0x7feb080b0e50 2022-06-29T15:48:05.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.771+0000 7feb05d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7feb080ad970 0x7feb080b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:05.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.771+0000 7feb05d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7feb080ad970 0x7feb080b32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49572/0 (socket says 172.21.15.47:49572) 2022-06-29T15:48:05.776 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.771+0000 7feb05d9b700 1 -- 172.21.15.47:0/979822168 learned_addr learned my addr 172.21.15.47:0/979822168 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:05.776 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.771+0000 7feb06d9d700 1 --2- 172.21.15.47:0/979822168 >> v2:172.21.15.64:3300/0 conn(0x7feb080b0e50 0x7feb0813d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:05.776 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.771+0000 7feb05d9b700 1 -- 172.21.15.47:0/979822168 >> v2:172.21.15.64:3300/0 conn(0x7feb080b0e50 msgr2=0x7feb0813d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:05.776 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.771+0000 7feb05d9b700 1 --2- 172.21.15.47:0/979822168 >> v2:172.21.15.64:3300/0 conn(0x7feb080b0e50 0x7feb0813d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:05.776 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.771+0000 7feb05d9b700 1 -- 172.21.15.47:0/979822168 >> v1:172.21.15.64:6789/0 conn(0x7feb080b2ee0 legacy=0x7feb081407e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:05.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.771+0000 7feb05d9b700 1 -- 172.21.15.47:0/979822168 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb08143570 con 0x7feb080ad970 2022-06-29T15:48:05.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.771+0000 7feb05d9b700 1 --2- 172.21.15.47:0/979822168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb080ad970 0x7feb080b32c0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7feaf00032f0 tx=0x7feaf0009290).ready entity=mon.0 client_cookie=7c79f7b6e50607d3 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:05.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.771+0000 7feb0559a700 1 -- 172.21.15.47:0/979822168 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feaf000b870 con 0x7feb080ad970 2022-06-29T15:48:05.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.771+0000 7feb0559a700 1 -- 172.21.15.47:0/979822168 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf000b9d0 con 0x7feb080ad970 2022-06-29T15:48:05.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.771+0000 7feb0559a700 1 -- 172.21.15.47:0/979822168 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feaf000bb40 con 0x7feb080ad970 2022-06-29T15:48:05.781 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.771+0000 7feb0759e700 1 -- 172.21.15.47:0/979822168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb080ad970 msgr2=0x7feb080b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:05.782 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.771+0000 7feb0759e700 1 --2- 172.21.15.47:0/979822168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb080ad970 0x7feb080b32c0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7feaf00032f0 tx=0x7feaf0009290).stop 2022-06-29T15:48:05.782 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.771+0000 7feb0759e700 1 -- 172.21.15.47:0/979822168 shutdown_connections 2022-06-29T15:48:05.782 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.771+0000 7feb0759e700 1 --2- 172.21.15.47:0/979822168 >> v2:172.21.15.64:3300/0 conn(0x7feb080b0e50 0x7feb0813d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:05.782 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.771+0000 7feb0759e700 1 --2- 172.21.15.47:0/979822168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb080ad970 0x7feb080b32c0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:05.783 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.771+0000 7feb0759e700 1 -- 172.21.15.47:0/979822168 >> 172.21.15.47:0/979822168 conn(0x7feb08063670 msgr2=0x7feb08063a80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:05.783 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.771+0000 7feb0759e700 1 -- 172.21.15.47:0/979822168 shutdown_connections 2022-06-29T15:48:05.783 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.771+0000 7feb0759e700 1 -- 172.21.15.47:0/979822168 wait complete. 2022-06-29T15:48:05.783 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.775+0000 7feb0759e700 1 Processor -- start 2022-06-29T15:48:05.783 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.775+0000 7feb0759e700 1 -- start start 2022-06-29T15:48:05.784 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.775+0000 7feb0759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb080ad970 0x7feb0805f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:05.784 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.775+0000 7feb0759e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7feb080b0e50 0x7feb0805fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:05.784 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.775+0000 7feb0759e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb080b2ee0 0x7feb081468b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:05.784 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.775+0000 7feb0759e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb08060010 con 0x7feb080b2ee0 2022-06-29T15:48:05.785 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.775+0000 7feb0759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb08142f70 con 0x7feb080ad970 2022-06-29T15:48:05.785 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.775+0000 7feb0759e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb08143210 con 0x7feb080b0e50 2022-06-29T15:48:05.785 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.775+0000 7feb06d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb080b2ee0 0x7feb081468b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:05.786 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.775+0000 7feb0659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb080ad970 0x7feb0805f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:05.786 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.775+0000 7feb06d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb080b2ee0 0x7feb081468b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49580/0 (socket says 172.21.15.47:49580) 2022-06-29T15:48:05.786 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.775+0000 7feb0659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb080ad970 0x7feb0805f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:55280/0 (socket says 172.21.15.47:55280) 2022-06-29T15:48:05.786 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.775+0000 7feb06d9d700 1 -- 172.21.15.47:0/1148963637 learned_addr learned my addr 172.21.15.47:0/1148963637 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:05.787 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.775+0000 7feb05d9b700 1 --2- 172.21.15.47:0/1148963637 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7feb080b0e50 0x7feb0805fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:05.787 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.775+0000 7feb0659c700 1 -- 172.21.15.47:0/1148963637 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7feb080b0e50 msgr2=0x7feb0805fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:05.787 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.775+0000 7feb0659c700 1 --2- 172.21.15.47:0/1148963637 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7feb080b0e50 0x7feb0805fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:05.787 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.775+0000 7feb0659c700 1 -- 172.21.15.47:0/1148963637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb080b2ee0 msgr2=0x7feb081468b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:05.788 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.775+0000 7feb0659c700 1 --2- 172.21.15.47:0/1148963637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb080b2ee0 0x7feb081468b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:05.788 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.775+0000 7feb0659c700 1 -- 172.21.15.47:0/1148963637 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf0009530 con 0x7feb080ad970 2022-06-29T15:48:05.788 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.775+0000 7feb0659c700 1 --2- 172.21.15.47:0/1148963637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb080ad970 0x7feb0805f3f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7feb08061bb0 tx=0x7feaf8007b90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:05.788 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.775+0000 7feaff7fe700 1 -- 172.21.15.47:0/1148963637 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feaf8013450 con 0x7feb080ad970 2022-06-29T15:48:05.788 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.775+0000 7feaff7fe700 1 -- 172.21.15.47:0/1148963637 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf80135b0 con 0x7feb080ad970 2022-06-29T15:48:05.789 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.775+0000 7feb0759e700 1 -- 172.21.15.47:0/1148963637 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb080b5430 con 0x7feb080ad970 2022-06-29T15:48:05.789 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.775+0000 7feaff7fe700 1 -- 172.21.15.47:0/1148963637 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feaf8013910 con 0x7feb080ad970 2022-06-29T15:48:05.789 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.775+0000 7feb0759e700 1 -- 172.21.15.47:0/1148963637 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb080b5970 con 0x7feb080ad970 2022-06-29T15:48:05.789 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.779+0000 7feb0759e700 1 -- 172.21.15.47:0/1148963637 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb080573d0 con 0x7feb080ad970 2022-06-29T15:48:05.789 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.779+0000 7feaff7fe700 1 -- 172.21.15.47:0/1148963637 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7feaf8010070 con 0x7feb080ad970 2022-06-29T15:48:05.790 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.779+0000 7feaff7fe700 1 --2- 172.21.15.47:0/1148963637 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7feae0053a70 0x7feae0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:05.790 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.779+0000 7feaff7fe700 1 -- 172.21.15.47:0/1148963637 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6420+0+0 (secure 0 0 0) 0x7feaf806c080 con 0x7feb080ad970 2022-06-29T15:48:05.790 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.779+0000 7feb05d9b700 1 --2- 172.21.15.47:0/1148963637 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7feae0053a70 0x7feae0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:05.791 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.779+0000 7feb05d9b700 1 --2- 172.21.15.47:0/1148963637 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7feae0053a70 0x7feae0055f30 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7feaf001b000 tx=0x7feaf0016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:05.791 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.787+0000 7feaff7fe700 1 -- 172.21.15.47:0/1148963637 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7feaf8042d30 con 0x7feb080ad970 2022-06-29T15:48:05.917 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.915+0000 7feb0759e700 1 -- 172.21.15.47:0/1148963637 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7feb08069e20 con 0x7feb080ad970 2022-06-29T15:48:05.920 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.919+0000 7feaff7fe700 1 -- 172.21.15.47:0/1148963637 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v68) v1 ==== 81+0+34040 (secure 0 0 0) 0x7feaf8046350 con 0x7feb080ad970 2022-06-29T15:48:05.922 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:48:05.923 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2022-06-29T15:48:05.923 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 68, 2022-06-29T15:48:05.923 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "9a68894a-53ce-400e-838b-14eab91b834c", 2022-06-29T15:48:05.924 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2022-06-29T15:42:14.046361+0000", 2022-06-29T15:48:05.924 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2022-06-29T15:47:36.342870+0000", 2022-06-29T15:48:05.925 INFO:teuthology.orchestra.run.smithi047.stdout: "last_up_change": "2022-06-29T15:42:20.929789+0000", 2022-06-29T15:48:05.925 INFO:teuthology.orchestra.run.smithi047.stdout: "last_in_change": "2022-06-29T15:42:20.929789+0000", 2022-06-29T15:48:05.926 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-06-29T15:48:05.926 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_num": 5799936, 2022-06-29T15:48:05.927 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_set": [ 2022-06-29T15:48:05.927 INFO:teuthology.orchestra.run.smithi047.stdout: "pglog_hardlimit", 2022-06-29T15:48:05.928 INFO:teuthology.orchestra.run.smithi047.stdout: "purged_snapdirs", 2022-06-29T15:48:05.928 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_deletes", 2022-06-29T15:48:05.929 INFO:teuthology.orchestra.run.smithi047.stdout: "sortbitwise" 2022-06-29T15:48:05.929 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:48:05.930 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_version": 5, 2022-06-29T15:48:05.930 INFO:teuthology.orchestra.run.smithi047.stdout: "full_ratio": 0.94999998807907104, 2022-06-29T15:48:05.931 INFO:teuthology.orchestra.run.smithi047.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-06-29T15:48:05.931 INFO:teuthology.orchestra.run.smithi047.stdout: "nearfull_ratio": 0.85000002384185791, 2022-06-29T15:48:05.932 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_snapshot": "", 2022-06-29T15:48:05.932 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_max": 12, 2022-06-29T15:48:05.933 INFO:teuthology.orchestra.run.smithi047.stdout: "max_osd": 8, 2022-06-29T15:48:05.934 INFO:teuthology.orchestra.run.smithi047.stdout: "require_min_compat_client": "jewel", 2022-06-29T15:48:05.934 INFO:teuthology.orchestra.run.smithi047.stdout: "min_compat_client": "jewel", 2022-06-29T15:48:05.935 INFO:teuthology.orchestra.run.smithi047.stdout: "require_osd_release": "octopus", 2022-06-29T15:48:05.935 INFO:teuthology.orchestra.run.smithi047.stdout: "pools": [ 2022-06-29T15:48:05.935 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:05.936 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 1, 2022-06-29T15:48:05.936 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "device_health_metrics", 2022-06-29T15:48:05.937 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:42:16.971431+0000", 2022-06-29T15:48:05.937 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:48:05.938 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:48:05.938 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:48:05.938 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:48:05.939 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:48:05.939 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:48:05.939 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:48:05.939 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:48:05.940 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 1, 2022-06-29T15:48:05.940 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 1, 2022-06-29T15:48:05.940 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 1, 2022-06-29T15:48:05.940 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 1, 2022-06-29T15:48:05.941 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 1, 2022-06-29T15:48:05.941 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:48:05.941 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:48:05.941 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:48:05.942 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:48:05.942 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:48:05.942 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:48:05.942 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:48:05.943 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:05.943 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "16", 2022-06-29T15:48:05.943 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:48:05.944 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:48:05.944 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:48:05.944 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:48:05.944 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:48:05.944 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:48:05.945 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:48:05.945 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:48:05.945 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:48:05.945 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:48:05.946 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:48:05.946 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:48:05.946 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:48:05.947 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:48:05.947 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:48:05.947 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:48:05.947 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:48:05.948 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:48:05.948 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:48:05.948 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:48:05.948 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:48:05.948 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:48:05.949 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:48:05.949 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:48:05.949 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:48:05.949 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:48:05.950 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:05.950 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:48:05.950 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:48:05.950 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:48:05.950 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:48:05.951 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:48:05.951 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:48:05.951 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:48:05.951 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:48:05.952 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:48:05.952 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:48:05.952 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:48:05.952 INFO:teuthology.orchestra.run.smithi047.stdout: "options": { 2022-06-29T15:48:05.952 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_min": 1 2022-06-29T15:48:05.953 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:05.953 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:48:05.953 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_devicehealth": {} 2022-06-29T15:48:05.953 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:05.954 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:05.954 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:05.954 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 2, 2022-06-29T15:48:05.954 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "rbd", 2022-06-29T15:48:05.955 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:42:25.290884+0000", 2022-06-29T15:48:05.955 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:48:05.955 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:48:05.955 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:48:05.955 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:48:05.956 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:48:05.956 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:48:05.956 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:48:05.956 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:48:05.957 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2022-06-29T15:48:05.957 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2022-06-29T15:48:05.957 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2022-06-29T15:48:05.957 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2022-06-29T15:48:05.957 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2022-06-29T15:48:05.958 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:48:05.958 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:48:05.958 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:48:05.958 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:48:05.959 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:48:05.959 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:48:05.959 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:48:05.959 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:05.960 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "19", 2022-06-29T15:48:05.960 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:48:05.960 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:48:05.960 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:48:05.960 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:48:05.961 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:48:05.961 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:48:05.961 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:48:05.961 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:48:05.962 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:48:05.962 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:48:05.962 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:48:05.962 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:48:05.963 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:48:05.963 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:48:05.963 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:48:05.963 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:48:05.963 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:48:05.964 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:48:05.964 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:48:05.964 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:48:05.964 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:48:05.965 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:48:05.965 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:48:05.965 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:48:05.965 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:48:05.965 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:48:05.966 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:05.966 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:48:05.966 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:48:05.966 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:48:05.967 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:48:05.967 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:48:05.967 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:48:05.967 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:48:05.967 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:48:05.968 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:48:05.968 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:48:05.968 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:48:05.968 INFO:teuthology.orchestra.run.smithi047.stdout: "options": {}, 2022-06-29T15:48:05.969 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:48:05.969 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd": {} 2022-06-29T15:48:05.969 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:05.969 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:05.970 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:05.970 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 11, 2022-06-29T15:48:05.971 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "cephfs_metadata", 2022-06-29T15:48:05.971 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:47:33.782330+0000", 2022-06-29T15:48:05.972 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:48:05.972 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:48:05.972 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:48:05.972 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:48:05.973 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:48:05.973 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:48:05.973 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:48:05.974 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:48:05.974 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2022-06-29T15:48:05.975 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2022-06-29T15:48:05.975 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2022-06-29T15:48:05.975 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2022-06-29T15:48:05.976 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2022-06-29T15:48:05.976 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:48:05.976 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:48:05.976 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:48:05.976 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:48:05.977 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:48:05.977 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:48:05.977 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:48:05.977 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:05.978 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "67", 2022-06-29T15:48:05.978 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:48:05.978 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:48:05.978 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:48:05.979 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:48:05.979 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:48:05.979 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:48:05.979 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:48:05.979 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:48:05.980 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:48:05.980 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:48:05.980 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:48:05.980 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:48:05.981 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:48:05.981 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:48:05.981 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:48:05.981 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:48:05.981 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:48:05.982 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:48:05.982 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:48:05.982 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:48:05.982 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:48:05.983 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:48:05.983 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:48:05.983 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:48:05.983 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:48:05.984 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:48:05.984 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:05.984 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:48:05.984 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:48:05.984 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:48:05.985 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:48:05.985 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:48:05.985 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:48:05.985 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:48:05.986 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:48:05.986 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:48:05.986 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:48:05.986 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:48:05.986 INFO:teuthology.orchestra.run.smithi047.stdout: "options": { 2022-06-29T15:48:05.987 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_bias": 4, 2022-06-29T15:48:05.987 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_min": 16, 2022-06-29T15:48:05.987 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_priority": 5 2022-06-29T15:48:05.987 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:05.988 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:48:05.988 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs": { 2022-06-29T15:48:05.988 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": "cephfs" 2022-06-29T15:48:05.988 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:05.988 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:05.989 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:05.989 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:05.989 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 12, 2022-06-29T15:48:05.989 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "cephfs_data", 2022-06-29T15:48:05.990 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:47:34.614730+0000", 2022-06-29T15:48:05.990 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:48:05.990 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:48:05.990 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:48:05.991 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:48:05.991 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:48:05.991 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:48:05.991 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:48:05.991 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:48:05.992 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2022-06-29T15:48:05.992 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2022-06-29T15:48:05.992 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2022-06-29T15:48:05.992 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2022-06-29T15:48:05.993 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2022-06-29T15:48:05.993 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:48:05.993 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:48:05.993 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:48:05.993 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:48:05.994 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:48:05.994 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:48:05.994 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:48:05.994 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:05.995 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "68", 2022-06-29T15:48:05.995 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:48:05.995 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:48:05.995 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:48:05.995 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:48:05.996 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:48:05.996 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:48:05.996 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:48:05.996 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:48:05.997 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:48:05.997 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:48:05.997 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:48:05.997 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:48:05.998 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:48:05.998 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:48:05.998 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:48:05.998 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:48:05.998 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:48:05.999 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:48:05.999 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:48:05.999 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:48:05.999 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:48:06.000 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:48:06.000 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:48:06.000 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:48:06.000 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:48:06.000 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:48:06.001 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.001 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:48:06.001 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:48:06.001 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:48:06.002 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:48:06.002 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:48:06.002 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:48:06.002 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:48:06.002 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:48:06.003 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:48:06.003 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:48:06.003 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:48:06.003 INFO:teuthology.orchestra.run.smithi047.stdout: "options": {}, 2022-06-29T15:48:06.004 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:48:06.004 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs": { 2022-06-29T15:48:06.004 INFO:teuthology.orchestra.run.smithi047.stdout: "data": "cephfs" 2022-06-29T15:48:06.004 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:06.005 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:06.005 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:06.005 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:48:06.005 INFO:teuthology.orchestra.run.smithi047.stdout: "osds": [ 2022-06-29T15:48:06.006 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.006 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2022-06-29T15:48:06.006 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "e23eb023-4706-4201-bf72-4bf19c912c5d", 2022-06-29T15:48:06.006 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:48:06.007 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:48:06.007 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:48:06.007 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:48:06.007 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:48:06.007 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:48:06.008 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:48:06.008 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 66, 2022-06-29T15:48:06.008 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:48:06.008 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:48:06.009 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:48:06.009 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:06.009 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.009 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:06.010 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6818", 2022-06-29T15:48:06.010 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:48:06.010 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.010 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.011 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:06.011 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6819", 2022-06-29T15:48:06.011 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:48:06.011 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:06.011 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:06.012 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.012 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:48:06.012 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:06.012 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.013 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:06.013 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6820", 2022-06-29T15:48:06.013 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:48:06.013 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.013 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.014 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:06.014 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6821", 2022-06-29T15:48:06.014 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:48:06.014 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:06.015 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:06.015 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.015 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:48:06.015 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:06.016 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.016 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:06.016 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6824", 2022-06-29T15:48:06.016 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:48:06.017 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.017 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.017 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:06.017 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6825", 2022-06-29T15:48:06.017 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:48:06.018 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:06.018 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:06.018 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.018 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:48:06.019 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:06.019 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.019 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:06.019 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6822", 2022-06-29T15:48:06.020 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:48:06.020 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.020 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.020 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:06.021 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6823", 2022-06-29T15:48:06.021 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:48:06.021 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:06.021 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:06.021 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.022 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6819/13096", 2022-06-29T15:48:06.022 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6821/13096", 2022-06-29T15:48:06.022 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6825/13096", 2022-06-29T15:48:06.022 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6823/13096", 2022-06-29T15:48:06.023 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:48:06.023 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:48:06.023 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:48:06.023 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:06.024 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.024 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.024 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2022-06-29T15:48:06.024 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "228bb08d-2d64-43d9-9560-bfaeff154452", 2022-06-29T15:48:06.025 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:48:06.025 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:48:06.025 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:48:06.025 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:48:06.025 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:48:06.026 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:48:06.026 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:48:06.026 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 66, 2022-06-29T15:48:06.026 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:48:06.027 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:48:06.027 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:48:06.027 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:06.027 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.027 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:06.028 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6826", 2022-06-29T15:48:06.028 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:48:06.028 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.028 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.029 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:06.029 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6827", 2022-06-29T15:48:06.029 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:48:06.029 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:06.029 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:06.030 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.030 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:48:06.030 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:06.030 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.031 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:06.031 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6828", 2022-06-29T15:48:06.031 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:48:06.031 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.032 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.032 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:06.032 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6829", 2022-06-29T15:48:06.032 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:48:06.032 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:06.033 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:06.033 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.033 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:48:06.033 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:06.034 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.034 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:06.034 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6832", 2022-06-29T15:48:06.034 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:48:06.035 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.035 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.035 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:06.036 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6833", 2022-06-29T15:48:06.036 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:48:06.036 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:06.037 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:06.037 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.037 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:48:06.037 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:06.038 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.038 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:06.038 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6830", 2022-06-29T15:48:06.038 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:48:06.038 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.039 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.039 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:06.039 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6831", 2022-06-29T15:48:06.039 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:48:06.040 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:06.040 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:06.040 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.040 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6827/13100", 2022-06-29T15:48:06.041 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6829/13100", 2022-06-29T15:48:06.041 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6833/13100", 2022-06-29T15:48:06.041 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6831/13100", 2022-06-29T15:48:06.041 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:48:06.041 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:48:06.042 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:48:06.042 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:06.042 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.042 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.043 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2022-06-29T15:48:06.043 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "797a5452-1510-4640-a111-c9c2fcf707eb", 2022-06-29T15:48:06.043 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:48:06.043 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:48:06.043 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:48:06.044 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:48:06.044 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:48:06.044 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:48:06.044 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:48:06.045 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 65, 2022-06-29T15:48:06.045 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:48:06.045 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:48:06.045 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:48:06.045 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:06.046 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.046 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:06.046 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6810", 2022-06-29T15:48:06.046 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:48:06.047 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.047 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.047 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:06.048 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6811", 2022-06-29T15:48:06.048 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:48:06.048 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:06.049 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:06.049 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.049 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:48:06.049 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:06.050 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.050 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:06.050 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6812", 2022-06-29T15:48:06.051 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:48:06.051 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.051 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.052 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:06.052 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6813", 2022-06-29T15:48:06.052 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:48:06.052 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:06.053 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:06.053 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.053 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:48:06.054 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:06.054 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.054 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:06.054 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6816", 2022-06-29T15:48:06.055 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:48:06.055 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.055 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.055 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:06.056 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6817", 2022-06-29T15:48:06.056 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:48:06.056 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:06.056 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:06.057 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.057 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:48:06.057 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:06.057 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.058 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:06.058 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6814", 2022-06-29T15:48:06.058 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:48:06.058 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.059 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.059 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:06.059 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6815", 2022-06-29T15:48:06.059 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:48:06.060 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:06.060 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:06.060 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.060 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6811/13098", 2022-06-29T15:48:06.060 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6813/13098", 2022-06-29T15:48:06.061 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6817/13098", 2022-06-29T15:48:06.061 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6815/13098", 2022-06-29T15:48:06.061 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:48:06.061 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:48:06.062 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:48:06.062 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:06.062 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.062 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.062 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2022-06-29T15:48:06.063 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "55b24fd7-bb71-4a01-b3fa-a09caa772a62", 2022-06-29T15:48:06.063 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:48:06.063 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:48:06.063 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:48:06.064 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:48:06.064 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:48:06.064 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:48:06.064 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:48:06.064 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 54, 2022-06-29T15:48:06.065 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:48:06.065 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:48:06.065 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:48:06.065 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:06.066 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.066 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:06.066 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6802", 2022-06-29T15:48:06.066 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:48:06.066 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.067 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.067 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:06.067 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6803", 2022-06-29T15:48:06.067 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:48:06.068 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:06.068 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:06.068 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.068 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:48:06.069 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:06.069 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.069 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:06.070 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6804", 2022-06-29T15:48:06.070 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:48:06.070 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.070 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.071 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:06.071 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6805", 2022-06-29T15:48:06.071 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:48:06.071 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:06.071 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:06.072 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.072 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:48:06.072 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:06.072 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.073 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:06.073 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6808", 2022-06-29T15:48:06.073 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:48:06.073 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.073 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.074 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:06.074 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6809", 2022-06-29T15:48:06.074 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:48:06.074 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:06.075 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:06.075 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.075 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:48:06.075 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:06.075 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.076 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:06.076 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6806", 2022-06-29T15:48:06.076 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:48:06.076 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.077 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.077 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:06.077 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6807", 2022-06-29T15:48:06.077 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:48:06.077 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:06.078 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:06.078 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.078 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6803/13097", 2022-06-29T15:48:06.079 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6805/13097", 2022-06-29T15:48:06.079 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6809/13097", 2022-06-29T15:48:06.079 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6807/13097", 2022-06-29T15:48:06.079 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:48:06.080 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:48:06.080 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:48:06.080 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:06.081 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.081 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.081 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2022-06-29T15:48:06.081 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3", 2022-06-29T15:48:06.082 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:48:06.082 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:48:06.082 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:48:06.082 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:48:06.083 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:48:06.083 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:48:06.083 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:48:06.083 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 66, 2022-06-29T15:48:06.083 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:48:06.084 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:48:06.084 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:48:06.084 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:06.084 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.085 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:06.085 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6816", 2022-06-29T15:48:06.085 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:48:06.085 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.086 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.086 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:06.086 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6817", 2022-06-29T15:48:06.086 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:48:06.086 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:06.087 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:06.087 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.087 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:48:06.087 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:06.088 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.088 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:06.088 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6818", 2022-06-29T15:48:06.088 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:48:06.089 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.089 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.089 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:06.089 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6819", 2022-06-29T15:48:06.089 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:48:06.090 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:06.090 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:06.090 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.090 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:48:06.091 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:06.091 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.091 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:06.091 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6822", 2022-06-29T15:48:06.092 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:48:06.092 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.092 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.092 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:06.092 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6823", 2022-06-29T15:48:06.093 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:48:06.093 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:06.093 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:06.093 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.094 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:48:06.094 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:06.094 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.094 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:06.095 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6820", 2022-06-29T15:48:06.095 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:48:06.095 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.095 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.095 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:06.096 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6821", 2022-06-29T15:48:06.096 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:48:06.096 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:06.096 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:06.097 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.097 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6817/13396", 2022-06-29T15:48:06.097 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6819/13396", 2022-06-29T15:48:06.097 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6823/13396", 2022-06-29T15:48:06.098 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6821/13396", 2022-06-29T15:48:06.098 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:48:06.098 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:48:06.099 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:48:06.099 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:06.099 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.100 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.100 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2022-06-29T15:48:06.100 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "f77beebe-6cca-4244-b2d4-677825ebf919", 2022-06-29T15:48:06.101 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:48:06.101 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:48:06.101 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:48:06.102 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:48:06.102 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:48:06.102 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:48:06.103 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:48:06.103 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 65, 2022-06-29T15:48:06.103 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:48:06.104 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:48:06.104 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:48:06.104 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:06.105 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.105 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:06.105 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6800", 2022-06-29T15:48:06.105 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:48:06.106 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.106 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.106 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:06.106 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6801", 2022-06-29T15:48:06.107 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:48:06.107 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:06.107 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:06.107 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.107 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:48:06.108 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:06.108 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.108 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:06.108 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6802", 2022-06-29T15:48:06.109 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:48:06.109 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.109 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.109 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:06.110 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6803", 2022-06-29T15:48:06.110 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:48:06.110 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:06.111 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:06.111 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.111 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:48:06.112 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:06.112 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.112 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:06.113 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6806", 2022-06-29T15:48:06.113 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:48:06.113 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.114 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.114 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:06.114 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6807", 2022-06-29T15:48:06.115 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:48:06.115 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:06.115 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:06.115 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.115 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:48:06.116 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:06.116 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.116 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:06.116 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6804", 2022-06-29T15:48:06.117 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:48:06.117 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.117 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.117 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:06.118 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6805", 2022-06-29T15:48:06.118 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:48:06.118 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:06.118 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:06.119 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.119 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6801/13393", 2022-06-29T15:48:06.119 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6803/13393", 2022-06-29T15:48:06.120 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6807/13393", 2022-06-29T15:48:06.120 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6805/13393", 2022-06-29T15:48:06.120 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:48:06.120 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:48:06.121 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:48:06.121 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:06.121 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.121 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.121 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2022-06-29T15:48:06.122 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "af0ca9b3-c02c-4910-834b-bf0b8dc6713c", 2022-06-29T15:48:06.122 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:48:06.122 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:48:06.122 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:48:06.123 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:48:06.123 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:48:06.123 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:48:06.123 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:48:06.123 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 66, 2022-06-29T15:48:06.124 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:48:06.124 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:48:06.124 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:48:06.124 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:06.125 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.125 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:06.125 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6824", 2022-06-29T15:48:06.125 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:48:06.125 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.126 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.126 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:06.126 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6825", 2022-06-29T15:48:06.126 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:48:06.127 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:06.127 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:06.127 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.127 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:48:06.127 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:06.128 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.128 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:06.128 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6826", 2022-06-29T15:48:06.128 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:48:06.129 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.129 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.129 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:06.129 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6827", 2022-06-29T15:48:06.129 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:48:06.130 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:06.130 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:06.130 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.131 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:48:06.131 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:06.131 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.131 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:06.132 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6830", 2022-06-29T15:48:06.132 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:48:06.132 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.132 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.133 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:06.133 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6831", 2022-06-29T15:48:06.133 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:48:06.133 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:06.133 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:06.134 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.134 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:48:06.134 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:06.134 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.135 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:06.135 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6828", 2022-06-29T15:48:06.135 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:48:06.135 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.135 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.136 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:06.136 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6829", 2022-06-29T15:48:06.136 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:48:06.136 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:06.137 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:06.137 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.137 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6825/13395", 2022-06-29T15:48:06.137 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6827/13395", 2022-06-29T15:48:06.137 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6831/13395", 2022-06-29T15:48:06.138 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6829/13395", 2022-06-29T15:48:06.138 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:48:06.138 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:48:06.139 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:48:06.139 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:06.139 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.140 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.140 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2022-06-29T15:48:06.140 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "f19c3ca6-df88-4bde-a150-b51a53f092e1", 2022-06-29T15:48:06.140 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:48:06.141 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:48:06.141 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:48:06.141 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:48:06.141 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:48:06.141 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:48:06.142 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:48:06.142 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 65, 2022-06-29T15:48:06.142 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:48:06.142 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:48:06.143 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:48:06.143 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:06.143 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.143 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:06.143 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6808", 2022-06-29T15:48:06.144 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:48:06.144 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.144 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.144 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:06.145 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6809", 2022-06-29T15:48:06.145 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:48:06.145 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:06.145 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:06.145 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.146 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:48:06.146 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:06.146 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.146 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:06.147 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6810", 2022-06-29T15:48:06.147 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:48:06.147 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.147 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.148 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:06.148 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6811", 2022-06-29T15:48:06.148 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:48:06.148 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:06.148 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:06.149 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.149 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:48:06.149 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:06.149 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.149 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:06.150 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6814", 2022-06-29T15:48:06.150 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:48:06.150 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.150 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.151 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:06.151 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6815", 2022-06-29T15:48:06.151 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:48:06.151 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:06.152 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:06.152 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.152 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:48:06.152 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:06.152 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.153 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:06.153 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6812", 2022-06-29T15:48:06.153 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:48:06.153 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.154 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.154 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:06.154 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6813", 2022-06-29T15:48:06.154 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:48:06.154 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:06.155 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:06.155 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.155 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6809/13394", 2022-06-29T15:48:06.155 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6811/13394", 2022-06-29T15:48:06.156 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6815/13394", 2022-06-29T15:48:06.156 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6813/13394", 2022-06-29T15:48:06.156 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:48:06.156 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:48:06.156 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:48:06.157 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:06.157 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:06.157 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:48:06.157 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_xinfo": [ 2022-06-29T15:48:06.158 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.158 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2022-06-29T15:48:06.158 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:48:06.158 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:48:06.159 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:48:06.159 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:48:06.159 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:48:06.159 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.720513+0000", 2022-06-29T15:48:06.159 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:48:06.160 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.160 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.160 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2022-06-29T15:48:06.160 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:48:06.161 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:48:06.161 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:48:06.161 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:48:06.161 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:48:06.161 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.772612+0000", 2022-06-29T15:48:06.162 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:48:06.162 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.162 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.163 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2022-06-29T15:48:06.163 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:48:06.163 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:48:06.164 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:48:06.164 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:48:06.164 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:48:06.164 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.450223+0000", 2022-06-29T15:48:06.165 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:48:06.165 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.165 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.165 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2022-06-29T15:48:06.165 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:48:06.166 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:48:06.166 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:48:06.166 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:48:06.166 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:48:06.167 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.477856+0000", 2022-06-29T15:48:06.167 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:48:06.167 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.167 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.167 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2022-06-29T15:48:06.168 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:48:06.168 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:48:06.168 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:48:06.168 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:48:06.169 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:48:06.169 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.539485+0000", 2022-06-29T15:48:06.169 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:48:06.169 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.169 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.170 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2022-06-29T15:48:06.170 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:48:06.170 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:48:06.170 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:48:06.171 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:48:06.171 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:48:06.171 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.487040+0000", 2022-06-29T15:48:06.171 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:48:06.172 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.172 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.172 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2022-06-29T15:48:06.172 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:48:06.172 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:48:06.173 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:48:06.173 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:48:06.173 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:48:06.173 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.748685+0000", 2022-06-29T15:48:06.174 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:48:06.174 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.174 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:06.174 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2022-06-29T15:48:06.175 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:48:06.175 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:48:06.175 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:48:06.175 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:48:06.175 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:48:06.176 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.459516+0000", 2022-06-29T15:48:06.176 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:48:06.176 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:06.176 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:48:06.176 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap": [], 2022-06-29T15:48:06.177 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap_items": [], 2022-06-29T15:48:06.177 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_temp": [], 2022-06-29T15:48:06.177 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_temp": [], 2022-06-29T15:48:06.177 INFO:teuthology.orchestra.run.smithi047.stdout: "blacklist": {}, 2022-06-29T15:48:06.178 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profiles": { 2022-06-29T15:48:06.178 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs_data": { 2022-06-29T15:48:06.178 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-device-class": "", 2022-06-29T15:48:06.179 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2022-06-29T15:48:06.179 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-root": "default", 2022-06-29T15:48:06.179 INFO:teuthology.orchestra.run.smithi047.stdout: "jerasure-per-chunk-alignment": "false", 2022-06-29T15:48:06.180 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2022-06-29T15:48:06.180 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "2", 2022-06-29T15:48:06.181 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2022-06-29T15:48:06.181 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2022-06-29T15:48:06.181 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van", 2022-06-29T15:48:06.182 INFO:teuthology.orchestra.run.smithi047.stdout: "w": "8" 2022-06-29T15:48:06.182 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.182 INFO:teuthology.orchestra.run.smithi047.stdout: "default": { 2022-06-29T15:48:06.183 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2022-06-29T15:48:06.183 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2022-06-29T15:48:06.183 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "1", 2022-06-29T15:48:06.184 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2022-06-29T15:48:06.184 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2022-06-29T15:48:06.184 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van" 2022-06-29T15:48:06.184 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:06.185 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:06.185 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps_queue": [], 2022-06-29T15:48:06.185 INFO:teuthology.orchestra.run.smithi047.stdout: "new_removed_snaps": [], 2022-06-29T15:48:06.185 INFO:teuthology.orchestra.run.smithi047.stdout: "new_purged_snaps": [], 2022-06-29T15:48:06.186 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_node_flags": {}, 2022-06-29T15:48:06.186 INFO:teuthology.orchestra.run.smithi047.stdout: "device_class_flags": {} 2022-06-29T15:48:06.186 INFO:teuthology.orchestra.run.smithi047.stdout:} 2022-06-29T15:48:06.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.919+0000 7feb0759e700 1 -- 172.21.15.47:0/1148963637 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7feae0053a70 msgr2=0x7feae0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:06.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.919+0000 7feb0759e700 1 --2- 172.21.15.47:0/1148963637 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7feae0053a70 0x7feae0055f30 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7feaf001b000 tx=0x7feaf0016000).stop 2022-06-29T15:48:06.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.919+0000 7feb0759e700 1 -- 172.21.15.47:0/1148963637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb080ad970 msgr2=0x7feb0805f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:06.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.919+0000 7feb0759e700 1 --2- 172.21.15.47:0/1148963637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb080ad970 0x7feb0805f3f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7feb08061bb0 tx=0x7feaf8007b90).stop 2022-06-29T15:48:06.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.919+0000 7feb0759e700 1 -- 172.21.15.47:0/1148963637 shutdown_connections 2022-06-29T15:48:06.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.919+0000 7feb0759e700 1 --2- 172.21.15.47:0/1148963637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb080b2ee0 0x7feb081468b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:06.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.919+0000 7feb0759e700 1 --2- 172.21.15.47:0/1148963637 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7feb080b0e50 0x7feb0805fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:06.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.919+0000 7feb0759e700 1 --2- 172.21.15.47:0/1148963637 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7feae0053a70 0x7feae0055f30 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:06.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.919+0000 7feb0759e700 1 --2- 172.21.15.47:0/1148963637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb080ad970 0x7feb0805f3f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:06.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.919+0000 7feb0759e700 1 -- 172.21.15.47:0/1148963637 >> 172.21.15.47:0/1148963637 conn(0x7feb08063670 msgr2=0x7feb0813dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:06.191 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.919+0000 7feb0759e700 1 -- 172.21.15.47:0/1148963637 shutdown_connections 2022-06-29T15:48:06.191 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:05.919+0000 7feb0759e700 1 -- 172.21.15.47:0/1148963637 wait complete. 2022-06-29T15:48:06.192 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:48:06.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.303+0000 7fab71828700 1 Processor -- start 2022-06-29T15:48:06.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.303+0000 7fab71828700 1 -- start start 2022-06-29T15:48:06.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.303+0000 7fab71828700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fab6c097f30 0x7fab6c142820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:06.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.303+0000 7fab71828700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fab6c060450 0x7fab6c0b0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:06.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.303+0000 7fab71828700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab6c098310 con 0x7fab6c0b2f00 2022-06-29T15:48:06.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.303+0000 7fab71828700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab6c0b1a10 con 0x7fab6c060450 2022-06-29T15:48:06.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.303+0000 7fab71828700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c0b2800 con 0x7fab6c097f30 2022-06-29T15:48:06.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.303+0000 7fab6b7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fab6c0b2f00 0x7fab6c143de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35142/0 (socket says 172.21.15.47:35142) 2022-06-29T15:48:06.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.303+0000 7fab6b7fe700 1 -- 172.21.15.47:0/1675374053 learned_addr learned my addr 172.21.15.47:0/1675374053 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:48:06.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.303+0000 7fab6affd700 1 --2- 172.21.15.47:0/1675374053 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fab6c097f30 0x7fab6c142820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:06.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.303+0000 7fab6a7fc700 1 --2- 172.21.15.47:0/1675374053 >> v2:172.21.15.64:3300/0 conn(0x7fab6c060450 0x7fab6c0b0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:06.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.307+0000 7fab69ffb700 1 -- 172.21.15.47:0/1675374053 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2490526196 0 0) 0x7fab6c098310 con 0x7fab6c0b2f00 2022-06-29T15:48:06.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.307+0000 7fab69ffb700 1 -- 172.21.15.47:0/1675374053 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab50003260 con 0x7fab6c0b2f00 2022-06-29T15:48:06.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.307+0000 7fab6affd700 1 -- 172.21.15.47:0/1675374053 >> v2:172.21.15.64:3300/0 conn(0x7fab6c060450 msgr2=0x7fab6c0b0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:06.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.307+0000 7fab6affd700 1 --2- 172.21.15.47:0/1675374053 >> v2:172.21.15.64:3300/0 conn(0x7fab6c060450 0x7fab6c0b0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:06.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.307+0000 7fab6affd700 1 -- 172.21.15.47:0/1675374053 >> v1:172.21.15.47:6789/0 conn(0x7fab6c0b2f00 legacy=0x7fab6c143de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:06.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.307+0000 7fab6affd700 1 -- 172.21.15.47:0/1675374053 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab6c061310 con 0x7fab6c097f30 2022-06-29T15:48:06.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.307+0000 7fab6affd700 1 --2- 172.21.15.47:0/1675374053 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fab6c097f30 0x7fab6c142820 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7fab5c0032f0 tx=0x7fab5c0091d0).ready entity=mon.2 client_cookie=45ea16217efed87f server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:06.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.307+0000 7fab69ffb700 1 -- 172.21.15.47:0/1675374053 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab5c00b600 con 0x7fab6c097f30 2022-06-29T15:48:06.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.307+0000 7fab69ffb700 1 -- 172.21.15.47:0/1675374053 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab5c00b760 con 0x7fab6c097f30 2022-06-29T15:48:06.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.307+0000 7fab69ffb700 1 -- 172.21.15.47:0/1675374053 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab5c00b8d0 con 0x7fab6c097f30 2022-06-29T15:48:06.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.307+0000 7fab71828700 1 -- 172.21.15.47:0/1675374053 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fab6c097f30 msgr2=0x7fab6c142820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:06.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.307+0000 7fab71828700 1 --2- 172.21.15.47:0/1675374053 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fab6c097f30 0x7fab6c142820 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7fab5c0032f0 tx=0x7fab5c0091d0).stop 2022-06-29T15:48:06.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.307+0000 7fab71828700 1 -- 172.21.15.47:0/1675374053 shutdown_connections 2022-06-29T15:48:06.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.307+0000 7fab71828700 1 --2- 172.21.15.47:0/1675374053 >> v2:172.21.15.64:3300/0 conn(0x7fab6c060450 0x7fab6c0b0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:06.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.307+0000 7fab71828700 1 --2- 172.21.15.47:0/1675374053 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fab6c097f30 0x7fab6c142820 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:06.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.307+0000 7fab71828700 1 -- 172.21.15.47:0/1675374053 >> 172.21.15.47:0/1675374053 conn(0x7fab6c05f0b0 msgr2=0x7fab6c05f490 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:06.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.307+0000 7fab71828700 1 -- 172.21.15.47:0/1675374053 shutdown_connections 2022-06-29T15:48:06.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.307+0000 7fab71828700 1 -- 172.21.15.47:0/1675374053 wait complete. 2022-06-29T15:48:06.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.311+0000 7fab71828700 1 Processor -- start 2022-06-29T15:48:06.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.311+0000 7fab71828700 1 -- start start 2022-06-29T15:48:06.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.311+0000 7fab71828700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fab6c060450 0x7fab6c09a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:06.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.311+0000 7fab71828700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab6c097f30 0x7fab6c150190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:06.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.311+0000 7fab71828700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab6c0b2f00 0x7fab6c150680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:06.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.311+0000 7fab71828700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c0aa070 con 0x7fab6c097f30 2022-06-29T15:48:06.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.311+0000 7fab71828700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c0a9b30 con 0x7fab6c0b2f00 2022-06-29T15:48:06.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.311+0000 7fab71828700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c0a9dd0 con 0x7fab6c060450 2022-06-29T15:48:06.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.311+0000 7fab6a7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab6c097f30 0x7fab6c150190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:06.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.311+0000 7fab6a7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab6c097f30 0x7fab6c150190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49592/0 (socket says 172.21.15.47:49592) 2022-06-29T15:48:06.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.311+0000 7fab6a7fc700 1 -- 172.21.15.47:0/3822915691 learned_addr learned my addr 172.21.15.47:0/3822915691 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:06.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.311+0000 7fab6affd700 1 --2- 172.21.15.47:0/3822915691 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fab6c060450 0x7fab6c09a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:06.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.311+0000 7fab6b7fe700 1 --2- 172.21.15.47:0/3822915691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab6c0b2f00 0x7fab6c150680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:06.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.311+0000 7fab6a7fc700 1 -- 172.21.15.47:0/3822915691 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fab6c060450 msgr2=0x7fab6c09a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:06.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.311+0000 7fab6a7fc700 1 --2- 172.21.15.47:0/3822915691 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fab6c060450 0x7fab6c09a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:06.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.311+0000 7fab6a7fc700 1 -- 172.21.15.47:0/3822915691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab6c0b2f00 msgr2=0x7fab6c150680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:06.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.311+0000 7fab6a7fc700 1 --2- 172.21.15.47:0/3822915691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab6c0b2f00 0x7fab6c150680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:06.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.311+0000 7fab6a7fc700 1 -- 172.21.15.47:0/3822915691 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab5c009470 con 0x7fab6c097f30 2022-06-29T15:48:06.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.311+0000 7fab6b7fe700 1 --2- 172.21.15.47:0/3822915691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab6c0b2f00 0x7fab6c150680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:48:06.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.311+0000 7fab6a7fc700 1 --2- 172.21.15.47:0/3822915691 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab6c097f30 0x7fab6c150190 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7fab6000c000 tx=0x7fab60006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:06.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.311+0000 7fab5bfff700 1 -- 172.21.15.47:0/3822915691 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab60010590 con 0x7fab6c097f30 2022-06-29T15:48:06.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.311+0000 7fab5bfff700 1 -- 172.21.15.47:0/3822915691 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab600106f0 con 0x7fab6c097f30 2022-06-29T15:48:06.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.311+0000 7fab71828700 1 -- 172.21.15.47:0/3822915691 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab6c0ab570 con 0x7fab6c097f30 2022-06-29T15:48:06.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.311+0000 7fab5bfff700 1 -- 172.21.15.47:0/3822915691 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab60010a90 con 0x7fab6c097f30 2022-06-29T15:48:06.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.311+0000 7fab71828700 1 -- 172.21.15.47:0/3822915691 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab6c05c730 con 0x7fab6c097f30 2022-06-29T15:48:06.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.315+0000 7fab71828700 1 -- 172.21.15.47:0/3822915691 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab6c0573d0 con 0x7fab6c097f30 2022-06-29T15:48:06.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.315+0000 7fab5bfff700 1 -- 172.21.15.47:0/3822915691 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fab60010bf0 con 0x7fab6c097f30 2022-06-29T15:48:06.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.315+0000 7fab5bfff700 1 --2- 172.21.15.47:0/3822915691 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fab48053980 0x7fab48055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:06.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.315+0000 7fab5bfff700 1 -- 172.21.15.47:0/3822915691 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fab6006c290 con 0x7fab6c097f30 2022-06-29T15:48:06.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.315+0000 7fab6affd700 1 --2- 172.21.15.47:0/3822915691 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fab48053980 0x7fab48055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:06.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.315+0000 7fab6affd700 1 --2- 172.21.15.47:0/3822915691 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fab48053980 0x7fab48055e40 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7fab5c01b000 tx=0x7fab5c016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:06.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.323+0000 7fab5bfff700 1 -- 172.21.15.47:0/3822915691 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fab60042ec0 con 0x7fab6c097f30 2022-06-29T15:48:06.485 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.483+0000 7fab71828700 1 -- 172.21.15.47:0/3822915691 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab6c060230 con 0x7fab6c097f30 2022-06-29T15:48:06.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.483+0000 7fab5bfff700 1 -- 172.21.15.47:0/3822915691 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+2490 (secure 0 0 0) 0x7fab600464e0 con 0x7fab6c097f30 2022-06-29T15:48:06.486 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:48:06.486 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":30,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5887,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/929350416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":929350416},{"type":"v1","addr":"172.21.15.47:6835","nonce":929350416}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":27},{"gid":5970,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2425218664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2425218664},{"type":"v1","addr":"172.21.15.64:6833","nonce":2425218664}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":27}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:47:35.340236+0000","modified":"2022-06-29T15:47:36.343849+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5973":{"gid":5973,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6837/1166335240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1166335240},{"type":"v1","addr":"172.21.15.47:6837","nonce":1166335240}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2022-06-29T15:48:06.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.487+0000 7fab71828700 1 -- 172.21.15.47:0/3822915691 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fab48053980 msgr2=0x7fab48055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:06.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.487+0000 7fab71828700 1 --2- 172.21.15.47:0/3822915691 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fab48053980 0x7fab48055e40 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7fab5c01b000 tx=0x7fab5c016000).stop 2022-06-29T15:48:06.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.487+0000 7fab71828700 1 -- 172.21.15.47:0/3822915691 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab6c097f30 msgr2=0x7fab6c150190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:06.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.487+0000 7fab71828700 1 --2- 172.21.15.47:0/3822915691 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab6c097f30 0x7fab6c150190 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7fab6000c000 tx=0x7fab60006df0).stop 2022-06-29T15:48:06.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.487+0000 7fab71828700 1 -- 172.21.15.47:0/3822915691 shutdown_connections 2022-06-29T15:48:06.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.487+0000 7fab71828700 1 --2- 172.21.15.47:0/3822915691 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fab48053980 0x7fab48055e40 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:06.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.487+0000 7fab71828700 1 --2- 172.21.15.47:0/3822915691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab6c0b2f00 0x7fab6c150680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:06.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.487+0000 7fab71828700 1 --2- 172.21.15.47:0/3822915691 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab6c097f30 0x7fab6c150190 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:06.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.487+0000 7fab71828700 1 --2- 172.21.15.47:0/3822915691 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fab6c060450 0x7fab6c09a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:06.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.487+0000 7fab71828700 1 -- 172.21.15.47:0/3822915691 >> 172.21.15.47:0/3822915691 conn(0x7fab6c05f0b0 msgr2=0x7fab6c0b65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:06.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.487+0000 7fab71828700 1 -- 172.21.15.47:0/3822915691 shutdown_connections 2022-06-29T15:48:06.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.487+0000 7fab71828700 1 -- 172.21.15.47:0/3822915691 wait complete. 2022-06-29T15:48:06.492 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 30 2022-06-29T15:48:06.499 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-06-29T15:48:06.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.603+0000 7fc46d871700 1 Processor -- start 2022-06-29T15:48:06.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.603+0000 7fc46d871700 1 -- start start 2022-06-29T15:48:06.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.603+0000 7fc46d871700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc4680b2120 0x7fc4680b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:06.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.603+0000 7fc46d871700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc4680b1260 0x7fc46814b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:06.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.603+0000 7fc46d871700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4680b0740 con 0x7fc46805f0b0 2022-06-29T15:48:06.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.603+0000 7fc46d871700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc468147460 con 0x7fc4680b2120 2022-06-29T15:48:06.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.603+0000 7fc46d871700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4680b0970 con 0x7fc4680b1260 2022-06-29T15:48:06.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.603+0000 7fc466ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc4680b2120 0x7fc4680b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:06.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.603+0000 7fc466ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc4680b2120 0x7fc4680b2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49600/0 (socket says 172.21.15.47:49600) 2022-06-29T15:48:06.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.603+0000 7fc466ffd700 1 -- 172.21.15.47:0/2110515456 learned_addr learned my addr 172.21.15.47:0/2110515456 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:06.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.603+0000 7fc4677fe700 1 --2- 172.21.15.47:0/2110515456 >> v2:172.21.15.64:3300/0 conn(0x7fc4680b1260 0x7fc46814b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:06.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.603+0000 7fc466ffd700 1 -- 172.21.15.47:0/2110515456 >> v2:172.21.15.64:3300/0 conn(0x7fc4680b1260 msgr2=0x7fc46814b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:06.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.603+0000 7fc466ffd700 1 --2- 172.21.15.47:0/2110515456 >> v2:172.21.15.64:3300/0 conn(0x7fc4680b1260 0x7fc46814b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:06.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.603+0000 7fc466ffd700 1 -- 172.21.15.47:0/2110515456 >> v1:172.21.15.64:6789/0 conn(0x7fc46805f0b0 legacy=0x7fc468147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:06.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.603+0000 7fc466ffd700 1 -- 172.21.15.47:0/2110515456 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4680b0c10 con 0x7fc4680b2120 2022-06-29T15:48:06.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.603+0000 7fc466ffd700 1 --2- 172.21.15.47:0/2110515456 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4680b2120 0x7fc4680b2500 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7fc468141a70 tx=0x7fc45c006eb0).ready entity=mon.0 client_cookie=24b0dc759bf08366 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:06.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.607+0000 7fc465ffb700 1 -- 172.21.15.47:0/2110515456 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc45c00d610 con 0x7fc4680b2120 2022-06-29T15:48:06.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.607+0000 7fc465ffb700 1 -- 172.21.15.47:0/2110515456 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc45c000e90 con 0x7fc4680b2120 2022-06-29T15:48:06.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.607+0000 7fc465ffb700 1 -- 172.21.15.47:0/2110515456 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc45c00d960 con 0x7fc4680b2120 2022-06-29T15:48:06.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.607+0000 7fc46d871700 1 -- 172.21.15.47:0/2110515456 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4680b2120 msgr2=0x7fc4680b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:06.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.607+0000 7fc46d871700 1 --2- 172.21.15.47:0/2110515456 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4680b2120 0x7fc4680b2500 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7fc468141a70 tx=0x7fc45c006eb0).stop 2022-06-29T15:48:06.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.607+0000 7fc46d871700 1 -- 172.21.15.47:0/2110515456 shutdown_connections 2022-06-29T15:48:06.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.607+0000 7fc46d871700 1 --2- 172.21.15.47:0/2110515456 >> v2:172.21.15.64:3300/0 conn(0x7fc4680b1260 0x7fc46814b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:06.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.607+0000 7fc46d871700 1 --2- 172.21.15.47:0/2110515456 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4680b2120 0x7fc4680b2500 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:06.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.607+0000 7fc46d871700 1 -- 172.21.15.47:0/2110515456 >> 172.21.15.47:0/2110515456 conn(0x7fc4680626f0 msgr2=0x7fc468062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:06.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.607+0000 7fc46d871700 1 -- 172.21.15.47:0/2110515456 shutdown_connections 2022-06-29T15:48:06.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.607+0000 7fc46d871700 1 -- 172.21.15.47:0/2110515456 wait complete. 2022-06-29T15:48:06.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.607+0000 7fc46d871700 1 Processor -- start 2022-06-29T15:48:06.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.607+0000 7fc46d871700 1 -- start start 2022-06-29T15:48:06.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.611+0000 7fc46d871700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4680b1260 0x7fc4680af380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:06.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.611+0000 7fc46d871700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4680b2120 0x7fc468156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:06.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.611+0000 7fc46d871700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc4680641c0 0x7fc468156e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:06.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.611+0000 7fc46d871700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc468065f80 con 0x7fc4680b1260 2022-06-29T15:48:06.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.611+0000 7fc46d871700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4680ae7e0 con 0x7fc4680b2120 2022-06-29T15:48:06.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.611+0000 7fc46d871700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc468157310 con 0x7fc4680641c0 2022-06-29T15:48:06.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.611+0000 7fc466ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4680b1260 0x7fc4680af380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:06.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.611+0000 7fc4667fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4680b2120 0x7fc468156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:06.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.611+0000 7fc466ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4680b1260 0x7fc4680af380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49606/0 (socket says 172.21.15.47:49606) 2022-06-29T15:48:06.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.611+0000 7fc4677fe700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc4680641c0 0x7fc468156e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:06.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.611+0000 7fc466ffd700 1 -- 172.21.15.47:0/1229271720 learned_addr learned my addr 172.21.15.47:0/1229271720 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:06.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.611+0000 7fc466ffd700 1 -- 172.21.15.47:0/1229271720 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc4680641c0 msgr2=0x7fc468156e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:06.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.611+0000 7fc466ffd700 1 --2- 172.21.15.47:0/1229271720 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc4680641c0 0x7fc468156e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:06.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.611+0000 7fc466ffd700 1 -- 172.21.15.47:0/1229271720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4680b2120 msgr2=0x7fc468156930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:06.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.611+0000 7fc466ffd700 1 --2- 172.21.15.47:0/1229271720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4680b2120 0x7fc468156930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:06.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.611+0000 7fc466ffd700 1 -- 172.21.15.47:0/1229271720 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc45c00b1e0 con 0x7fc4680b1260 2022-06-29T15:48:06.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.611+0000 7fc466ffd700 1 --2- 172.21.15.47:0/1229271720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4680b1260 0x7fc4680af380 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7fc468141a70 tx=0x7fc45c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:06.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.611+0000 7fc457fff700 1 -- 172.21.15.47:0/1229271720 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc45c002f50 con 0x7fc4680b1260 2022-06-29T15:48:06.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.611+0000 7fc457fff700 1 -- 172.21.15.47:0/1229271720 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc45c00d990 con 0x7fc4680b1260 2022-06-29T15:48:06.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.611+0000 7fc46d871700 1 -- 172.21.15.47:0/1229271720 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc468064780 con 0x7fc4680b1260 2022-06-29T15:48:06.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.611+0000 7fc457fff700 1 -- 172.21.15.47:0/1229271720 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc45c011d30 con 0x7fc4680b1260 2022-06-29T15:48:06.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.611+0000 7fc46d871700 1 -- 172.21.15.47:0/1229271720 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc468064cc0 con 0x7fc4680b1260 2022-06-29T15:48:06.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.615+0000 7fc46d871700 1 -- 172.21.15.47:0/1229271720 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc468064e80 con 0x7fc4680b1260 2022-06-29T15:48:06.622 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.615+0000 7fc457fff700 1 -- 172.21.15.47:0/1229271720 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc45c025030 con 0x7fc4680b1260 2022-06-29T15:48:06.622 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.615+0000 7fc457fff700 1 --2- 172.21.15.47:0/1229271720 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc444053b00 0x7fc444055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:06.622 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.615+0000 7fc4667fc700 1 --2- 172.21.15.47:0/1229271720 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc444053b00 0x7fc444055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:06.622 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.615+0000 7fc457fff700 1 -- 172.21.15.47:0/1229271720 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc45c06d340 con 0x7fc4680b1260 2022-06-29T15:48:06.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.615+0000 7fc4667fc700 1 --2- 172.21.15.47:0/1229271720 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc444053b00 0x7fc444055fc0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7fc45800d000 tx=0x7fc458007b90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:06.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.619+0000 7fc457fff700 1 -- 172.21.15.47:0/1229271720 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc45c043ff0 con 0x7fc4680b1260 2022-06-29T15:48:06.790 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:06.787+0000 7fc46d871700 1 -- 172.21.15.47:0/1229271720 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc468063760 con 0x7fc4680b1260 2022-06-29T15:48:07.231 INFO:tasks.ceph.mon.a.smithi047.stderr:2022-06-29T15:48:07.231+0000 7ff9cf83c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-06-29T15:48:07.246 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.243+0000 7fc457fff700 1 -- 172.21.15.47:0/1229271720 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v31) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc45c023020 con 0x7fc4680b1260 2022-06-29T15:48:07.249 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.247+0000 7fc46d871700 1 -- 172.21.15.47:0/1229271720 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc444053b00 msgr2=0x7fc444055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:07.250 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.247+0000 7fc46d871700 1 --2- 172.21.15.47:0/1229271720 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc444053b00 0x7fc444055fc0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7fc45800d000 tx=0x7fc458007b90).stop 2022-06-29T15:48:07.250 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.247+0000 7fc46d871700 1 -- 172.21.15.47:0/1229271720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4680b1260 msgr2=0x7fc4680af380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:07.251 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.247+0000 7fc46d871700 1 --2- 172.21.15.47:0/1229271720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4680b1260 0x7fc4680af380 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7fc468141a70 tx=0x7fc45c015000).stop 2022-06-29T15:48:07.251 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.247+0000 7fc46d871700 1 -- 172.21.15.47:0/1229271720 shutdown_connections 2022-06-29T15:48:07.251 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.247+0000 7fc46d871700 1 --2- 172.21.15.47:0/1229271720 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc4680641c0 0x7fc468156e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:07.251 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.247+0000 7fc46d871700 1 --2- 172.21.15.47:0/1229271720 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc444053b00 0x7fc444055fc0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:07.252 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.247+0000 7fc46d871700 1 --2- 172.21.15.47:0/1229271720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4680b2120 0x7fc468156930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:07.252 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.247+0000 7fc46d871700 1 --2- 172.21.15.47:0/1229271720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4680b1260 0x7fc4680af380 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:07.252 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.247+0000 7fc46d871700 1 -- 172.21.15.47:0/1229271720 >> 172.21.15.47:0/1229271720 conn(0x7fc4680626f0 msgr2=0x7fc468146af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:07.252 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.247+0000 7fc46d871700 1 -- 172.21.15.47:0/1229271720 shutdown_connections 2022-06-29T15:48:07.253 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.247+0000 7fc46d871700 1 -- 172.21.15.47:0/1229271720 wait complete. 2022-06-29T15:48:07.253 INFO:teuthology.orchestra.run.smithi047.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-06-29T15:48:07.265 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:48:07.284 INFO:tasks.ceph.mds.a.smithi047.stdout:starting mds.a at 2022-06-29T15:48:07.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.367+0000 7fa544907700 1 Processor -- start 2022-06-29T15:48:07.370 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.367+0000 7fa544907700 1 -- start start 2022-06-29T15:48:07.372 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.367+0000 7fa544907700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa5400611b0 0x7fa540146640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:07.372 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.367+0000 7fa544907700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa5400b0cf0 0x7fa540146b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:07.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.367+0000 7fa544907700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5400b4520 con 0x7fa5400b06d0 2022-06-29T15:48:07.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.367+0000 7fa544907700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5400b4b10 con 0x7fa5400b0cf0 2022-06-29T15:48:07.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.367+0000 7fa544907700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5400621d0 con 0x7fa5400611b0 2022-06-29T15:48:07.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.371+0000 7fa53e59c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa5400611b0 0x7fa540146640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:07.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.371+0000 7fa53dd9b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa5400b0cf0 0x7fa540146b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:07.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.371+0000 7fa53e59c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa5400611b0 0x7fa540146640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:46726/0 (socket says 172.21.15.47:46726) 2022-06-29T15:48:07.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.371+0000 7fa53e59c700 1 -- 172.21.15.47:0/3532533093 learned_addr learned my addr 172.21.15.47:0/3532533093 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:07.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.371+0000 7fa53e59c700 1 -- 172.21.15.47:0/3532533093 >> v2:172.21.15.64:3300/0 conn(0x7fa5400b0cf0 msgr2=0x7fa540146b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:07.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.371+0000 7fa53e59c700 1 --2- 172.21.15.47:0/3532533093 >> v2:172.21.15.64:3300/0 conn(0x7fa5400b0cf0 0x7fa540146b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:07.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.371+0000 7fa53e59c700 1 -- 172.21.15.47:0/3532533093 >> v1:172.21.15.64:6789/0 conn(0x7fa5400b06d0 legacy=0x7fa5400667e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:07.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.371+0000 7fa53e59c700 1 -- 172.21.15.47:0/3532533093 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa540061590 con 0x7fa5400611b0 2022-06-29T15:48:07.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.371+0000 7fa53e59c700 1 --2- 172.21.15.47:0/3532533093 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa5400611b0 0x7fa540146640 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fa5300032f0 tx=0x7fa530009290).ready entity=mon.2 client_cookie=89312a7bd0f5d277 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:07.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.371+0000 7fa53d59a700 1 -- 172.21.15.47:0/3532533093 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa53000b8a0 con 0x7fa5400611b0 2022-06-29T15:48:07.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.371+0000 7fa53d59a700 1 -- 172.21.15.47:0/3532533093 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa53000ba00 con 0x7fa5400611b0 2022-06-29T15:48:07.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.371+0000 7fa53d59a700 1 -- 172.21.15.47:0/3532533093 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa53000bb70 con 0x7fa5400611b0 2022-06-29T15:48:07.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.371+0000 7fa544907700 1 -- 172.21.15.47:0/3532533093 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa5400611b0 msgr2=0x7fa540146640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:07.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.371+0000 7fa544907700 1 --2- 172.21.15.47:0/3532533093 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa5400611b0 0x7fa540146640 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fa5300032f0 tx=0x7fa530009290).stop 2022-06-29T15:48:07.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.371+0000 7fa544907700 1 -- 172.21.15.47:0/3532533093 shutdown_connections 2022-06-29T15:48:07.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.371+0000 7fa544907700 1 --2- 172.21.15.47:0/3532533093 >> v2:172.21.15.64:3300/0 conn(0x7fa5400b0cf0 0x7fa540146b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:07.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.371+0000 7fa544907700 1 --2- 172.21.15.47:0/3532533093 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa5400611b0 0x7fa540146640 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:07.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.371+0000 7fa544907700 1 -- 172.21.15.47:0/3532533093 >> 172.21.15.47:0/3532533093 conn(0x7fa5400b98b0 msgr2=0x7fa5400b9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:07.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.371+0000 7fa544907700 1 -- 172.21.15.47:0/3532533093 shutdown_connections 2022-06-29T15:48:07.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.371+0000 7fa544907700 1 -- 172.21.15.47:0/3532533093 wait complete. 2022-06-29T15:48:07.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.371+0000 7fa544907700 1 Processor -- start 2022-06-29T15:48:07.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.371+0000 7fa544907700 1 -- start start 2022-06-29T15:48:07.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.375+0000 7fa544907700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5400611b0 0x7fa5400afaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:07.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.375+0000 7fa544907700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5400b06d0 0x7fa540145690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:07.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.375+0000 7fa544907700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa5400b0cf0 0x7fa540145b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:07.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.375+0000 7fa544907700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5400af030 con 0x7fa5400b06d0 2022-06-29T15:48:07.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.375+0000 7fa544907700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5401449b0 con 0x7fa5400611b0 2022-06-29T15:48:07.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.375+0000 7fa544907700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa540144c50 con 0x7fa5400b0cf0 2022-06-29T15:48:07.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.375+0000 7fa53dd9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5400b06d0 0x7fa540145690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:07.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.375+0000 7fa53e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5400611b0 0x7fa5400afaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:07.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.375+0000 7fa53dd9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5400b06d0 0x7fa540145690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49632/0 (socket says 172.21.15.47:49632) 2022-06-29T15:48:07.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.375+0000 7fa53dd9b700 1 -- 172.21.15.47:0/3644286367 learned_addr learned my addr 172.21.15.47:0/3644286367 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:07.385 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.375+0000 7fa53ed9d700 1 --2- 172.21.15.47:0/3644286367 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa5400b0cf0 0x7fa540145b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:07.385 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.375+0000 7fa53dd9b700 1 -- 172.21.15.47:0/3644286367 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa5400b0cf0 msgr2=0x7fa540145b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:07.385 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.375+0000 7fa53dd9b700 1 --2- 172.21.15.47:0/3644286367 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa5400b0cf0 0x7fa540145b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:07.385 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.375+0000 7fa53dd9b700 1 -- 172.21.15.47:0/3644286367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5400611b0 msgr2=0x7fa5400afaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:07.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.375+0000 7fa53dd9b700 1 --2- 172.21.15.47:0/3644286367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5400611b0 0x7fa5400afaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:07.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.375+0000 7fa53dd9b700 1 -- 172.21.15.47:0/3644286367 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa530009530 con 0x7fa5400b06d0 2022-06-29T15:48:07.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.375+0000 7fa53e59c700 1 --2- 172.21.15.47:0/3644286367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5400611b0 0x7fa5400afaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:48:07.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.375+0000 7fa53ed9d700 1 --2- 172.21.15.47:0/3644286367 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa5400b0cf0 0x7fa540145b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:48:07.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.375+0000 7fa53dd9b700 1 --2- 172.21.15.47:0/3644286367 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5400b06d0 0x7fa540145690 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7fa52800c000 tx=0x7fa528006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:07.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.375+0000 7fa5377fe700 1 -- 172.21.15.47:0/3644286367 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa528010780 con 0x7fa5400b06d0 2022-06-29T15:48:07.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.375+0000 7fa5377fe700 1 -- 172.21.15.47:0/3644286367 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5280108e0 con 0x7fa5400b06d0 2022-06-29T15:48:07.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.375+0000 7fa544907700 1 -- 172.21.15.47:0/3644286367 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa540144ef0 con 0x7fa5400b06d0 2022-06-29T15:48:07.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.375+0000 7fa5377fe700 1 -- 172.21.15.47:0/3644286367 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa528010c40 con 0x7fa5400b06d0 2022-06-29T15:48:07.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.375+0000 7fa544907700 1 -- 172.21.15.47:0/3644286367 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa540145390 con 0x7fa5400b06d0 2022-06-29T15:48:07.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.375+0000 7fa544907700 1 -- 172.21.15.47:0/3644286367 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5400573d0 con 0x7fa5400b06d0 2022-06-29T15:48:07.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.379+0000 7fa5377fe700 1 -- 172.21.15.47:0/3644286367 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa52801c460 con 0x7fa5400b06d0 2022-06-29T15:48:07.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.379+0000 7fa5377fe700 1 --2- 172.21.15.47:0/3644286367 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa524053b00 0x7fa524055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:07.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.379+0000 7fa5377fe700 1 -- 172.21.15.47:0/3644286367 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fa52806c4e0 con 0x7fa5400b06d0 2022-06-29T15:48:07.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.379+0000 7fa53e59c700 1 --2- 172.21.15.47:0/3644286367 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa524053b00 0x7fa524055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:07.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.379+0000 7fa53e59c700 1 --2- 172.21.15.47:0/3644286367 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa524053b00 0x7fa524055fc0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7fa53001b000 tx=0x7fa530016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:07.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.383+0000 7fa5377fe700 1 -- 172.21.15.47:0/3644286367 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa5280431f0 con 0x7fa5400b06d0 2022-06-29T15:48:07.550 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.551+0000 7fa544907700 1 -- 172.21.15.47:0/3644286367 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa540140b80 con 0x7fa5400b06d0 2022-06-29T15:48:07.551 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.551+0000 7fa5377fe700 1 -- 172.21.15.47:0/3644286367 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+2129 (secure 0 0 0) 0x7fa528046810 con 0x7fa5400b06d0 2022-06-29T15:48:07.552 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:48:07.552 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":31,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5887,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/929350416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":929350416},{"type":"v1","addr":"172.21.15.47:6835","nonce":929350416}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":27},{"gid":5970,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2425218664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2425218664},{"type":"v1","addr":"172.21.15.64:6833","nonce":2425218664}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":27}],"filesystems":[{"mdsmap":{"epoch":31,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:47:35.340236+0000","modified":"2022-06-29T15:48:07.233417+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2022-06-29T15:48:07.553 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.551+0000 7fa544907700 1 -- 172.21.15.47:0/3644286367 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa524053b00 msgr2=0x7fa524055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:07.554 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.551+0000 7fa544907700 1 --2- 172.21.15.47:0/3644286367 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa524053b00 0x7fa524055fc0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7fa53001b000 tx=0x7fa530016000).stop 2022-06-29T15:48:07.554 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.551+0000 7fa544907700 1 -- 172.21.15.47:0/3644286367 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5400b06d0 msgr2=0x7fa540145690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:07.554 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.551+0000 7fa544907700 1 --2- 172.21.15.47:0/3644286367 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5400b06d0 0x7fa540145690 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7fa52800c000 tx=0x7fa528006df0).stop 2022-06-29T15:48:07.554 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.551+0000 7fa544907700 1 -- 172.21.15.47:0/3644286367 shutdown_connections 2022-06-29T15:48:07.555 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.551+0000 7fa544907700 1 --2- 172.21.15.47:0/3644286367 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa5400b0cf0 0x7fa540145b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:07.555 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.551+0000 7fa544907700 1 --2- 172.21.15.47:0/3644286367 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5400b06d0 0x7fa540145690 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:07.555 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.551+0000 7fa544907700 1 --2- 172.21.15.47:0/3644286367 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa524053b00 0x7fa524055fc0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:07.555 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.551+0000 7fa544907700 1 --2- 172.21.15.47:0/3644286367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5400611b0 0x7fa5400afaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:07.556 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.551+0000 7fa544907700 1 -- 172.21.15.47:0/3644286367 >> 172.21.15.47:0/3644286367 conn(0x7fa5400b98b0 msgr2=0x7fa5400b5060 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:07.556 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.551+0000 7fa544907700 1 -- 172.21.15.47:0/3644286367 shutdown_connections 2022-06-29T15:48:07.556 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.551+0000 7fa544907700 1 -- 172.21.15.47:0/3644286367 wait complete. 2022-06-29T15:48:07.557 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 31 2022-06-29T15:48:07.569 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-06-29T15:48:07.681 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.679+0000 7fe19e7f5700 1 Processor -- start 2022-06-29T15:48:07.682 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.679+0000 7fe19e7f5700 1 -- start start 2022-06-29T15:48:07.683 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.683+0000 7fe19e7f5700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe1980b58d0 0x7fe1980b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:07.683 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.683+0000 7fe19e7f5700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe1980b1610 0x7fe1981475f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:07.684 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.683+0000 7fe19e7f5700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe198060790 con 0x7fe19814b4c0 2022-06-29T15:48:07.684 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.683+0000 7fe19e7f5700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1980a99e0 con 0x7fe1980b58d0 2022-06-29T15:48:07.684 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.683+0000 7fe19e7f5700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe19814ad80 con 0x7fe1980b1610 2022-06-29T15:48:07.684 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.683+0000 7fe19d7f3700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe1980b58d0 0x7fe1980b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:07.685 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.683+0000 7fe19d7f3700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe1980b58d0 0x7fe1980b5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49642/0 (socket says 172.21.15.47:49642) 2022-06-29T15:48:07.685 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.683+0000 7fe19d7f3700 1 -- 172.21.15.47:0/4112300388 learned_addr learned my addr 172.21.15.47:0/4112300388 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:07.685 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.683+0000 7fe19cff2700 1 --2- 172.21.15.47:0/4112300388 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe1980b1610 0x7fe1981475f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:07.685 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.683+0000 7fe19d7f3700 1 -- 172.21.15.47:0/4112300388 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe1980b1610 msgr2=0x7fe1981475f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:07.686 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.683+0000 7fe19d7f3700 1 --2- 172.21.15.47:0/4112300388 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe1980b1610 0x7fe1981475f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:07.686 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.683+0000 7fe19d7f3700 1 -- 172.21.15.47:0/4112300388 >> v1:172.21.15.64:6789/0 conn(0x7fe19814b4c0 legacy=0x7fe1980aa130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:07.686 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.683+0000 7fe19d7f3700 1 -- 172.21.15.47:0/4112300388 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe19814b020 con 0x7fe1980b58d0 2022-06-29T15:48:07.686 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.683+0000 7fe19d7f3700 1 --2- 172.21.15.47:0/4112300388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1980b58d0 0x7fe1980b5cb0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7fe1980623e0 tx=0x7fe188006eb0).ready entity=mon.0 client_cookie=1bd7a73fe6ec76b6 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:07.688 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.683+0000 7fe18ffff700 1 -- 172.21.15.47:0/4112300388 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe18800d5c0 con 0x7fe1980b58d0 2022-06-29T15:48:07.688 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.683+0000 7fe18ffff700 1 -- 172.21.15.47:0/4112300388 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe188000e90 con 0x7fe1980b58d0 2022-06-29T15:48:07.688 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.683+0000 7fe18ffff700 1 -- 172.21.15.47:0/4112300388 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe18800d950 con 0x7fe1980b58d0 2022-06-29T15:48:07.689 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.683+0000 7fe19e7f5700 1 -- 172.21.15.47:0/4112300388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1980b58d0 msgr2=0x7fe1980b5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:07.689 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.683+0000 7fe19e7f5700 1 --2- 172.21.15.47:0/4112300388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1980b58d0 0x7fe1980b5cb0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7fe1980623e0 tx=0x7fe188006eb0).stop 2022-06-29T15:48:07.689 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.683+0000 7fe19e7f5700 1 -- 172.21.15.47:0/4112300388 shutdown_connections 2022-06-29T15:48:07.689 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.683+0000 7fe19e7f5700 1 --2- 172.21.15.47:0/4112300388 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe1980b1610 0x7fe1981475f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:07.689 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.683+0000 7fe19e7f5700 1 --2- 172.21.15.47:0/4112300388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1980b58d0 0x7fe1980b5cb0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:07.690 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.683+0000 7fe19e7f5700 1 -- 172.21.15.47:0/4112300388 >> 172.21.15.47:0/4112300388 conn(0x7fe198062560 msgr2=0x7fe198062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:07.690 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.683+0000 7fe19e7f5700 1 -- 172.21.15.47:0/4112300388 shutdown_connections 2022-06-29T15:48:07.690 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.683+0000 7fe19e7f5700 1 -- 172.21.15.47:0/4112300388 wait complete. 2022-06-29T15:48:07.690 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.687+0000 7fe19e7f5700 1 Processor -- start 2022-06-29T15:48:07.691 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.687+0000 7fe19e7f5700 1 -- start start 2022-06-29T15:48:07.693 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.687+0000 7fe19e7f5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1980b1610 0x7fe198141e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:07.694 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.687+0000 7fe19e7f5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1980b58d0 0x7fe1980abc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:07.694 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.687+0000 7fe19e7f5700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe19814b4c0 0x7fe198142f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:07.694 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.687+0000 7fe19e7f5700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1980af5f0 con 0x7fe1980b1610 2022-06-29T15:48:07.694 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.687+0000 7fe19e7f5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe198144ca0 con 0x7fe1980b58d0 2022-06-29T15:48:07.694 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.687+0000 7fe19e7f5700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe198142920 con 0x7fe19814b4c0 2022-06-29T15:48:07.695 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.687+0000 7fe19d7f3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1980b1610 0x7fe198141e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:07.695 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.687+0000 7fe19d7f3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1980b1610 0x7fe198141e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49644/0 (socket says 172.21.15.47:49644) 2022-06-29T15:48:07.695 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.687+0000 7fe19d7f3700 1 -- 172.21.15.47:0/3323907397 learned_addr learned my addr 172.21.15.47:0/3323907397 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:07.695 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.687+0000 7fe19cff2700 1 --2- 172.21.15.47:0/3323907397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1980b58d0 0x7fe1980abc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:07.696 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.687+0000 7fe19dff4700 1 --2- 172.21.15.47:0/3323907397 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe19814b4c0 0x7fe198142f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:07.696 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.687+0000 7fe19d7f3700 1 -- 172.21.15.47:0/3323907397 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe19814b4c0 msgr2=0x7fe198142f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:07.696 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.687+0000 7fe19d7f3700 1 --2- 172.21.15.47:0/3323907397 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe19814b4c0 0x7fe198142f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:07.696 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.691+0000 7fe19d7f3700 1 -- 172.21.15.47:0/3323907397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1980b58d0 msgr2=0x7fe1980abc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:07.697 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.691+0000 7fe19d7f3700 1 --2- 172.21.15.47:0/3323907397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1980b58d0 0x7fe1980abc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:07.697 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.691+0000 7fe19d7f3700 1 -- 172.21.15.47:0/3323907397 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe18800b1e0 con 0x7fe1980b1610 2022-06-29T15:48:07.697 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.691+0000 7fe19d7f3700 1 --2- 172.21.15.47:0/3323907397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1980b1610 0x7fe198141e20 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7fe1980634b0 tx=0x7fe188015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:07.697 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.691+0000 7fe18e7fc700 1 -- 172.21.15.47:0/3323907397 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe1880117e0 con 0x7fe1980b1610 2022-06-29T15:48:07.697 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.691+0000 7fe18e7fc700 1 -- 172.21.15.47:0/3323907397 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe188011940 con 0x7fe1980b1610 2022-06-29T15:48:07.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.691+0000 7fe19e7f5700 1 -- 172.21.15.47:0/3323907397 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe19815a240 con 0x7fe1980b1610 2022-06-29T15:48:07.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.691+0000 7fe18e7fc700 1 -- 172.21.15.47:0/3323907397 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe188011ca0 con 0x7fe1980b1610 2022-06-29T15:48:07.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.691+0000 7fe19e7f5700 1 -- 172.21.15.47:0/3323907397 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe198142bc0 con 0x7fe1980b1610 2022-06-29T15:48:07.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.691+0000 7fe19e7f5700 1 -- 172.21.15.47:0/3323907397 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1980573d0 con 0x7fe1980b1610 2022-06-29T15:48:07.699 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.691+0000 7fe18e7fc700 1 -- 172.21.15.47:0/3323907397 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe188025030 con 0x7fe1980b1610 2022-06-29T15:48:07.699 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.695+0000 7fe18e7fc700 1 --2- 172.21.15.47:0/3323907397 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe180053950 0x7fe180055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:07.700 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.695+0000 7fe18e7fc700 1 -- 172.21.15.47:0/3323907397 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fe18806d360 con 0x7fe1980b1610 2022-06-29T15:48:07.700 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.695+0000 7fe19cff2700 1 --2- 172.21.15.47:0/3323907397 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe180053950 0x7fe180055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:07.700 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.695+0000 7fe19cff2700 1 --2- 172.21.15.47:0/3323907397 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe180053950 0x7fe180055e10 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7fe19400b000 tx=0x7fe194006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:07.700 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.699+0000 7fe18e7fc700 1 -- 172.21.15.47:0/3323907397 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe188043fd0 con 0x7fe1980b1610 2022-06-29T15:48:07.861 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:07.859+0000 7fe19e7f5700 1 -- 172.21.15.47:0/3323907397 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe1980a8f20 con 0x7fe1980b1610 2022-06-29T15:48:08.252 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.251+0000 7fe18e7fc700 1 -- 172.21.15.47:0/3323907397 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v32) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe1880230a0 con 0x7fe1980b1610 2022-06-29T15:48:08.256 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.255+0000 7fe19e7f5700 1 -- 172.21.15.47:0/3323907397 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe180053950 msgr2=0x7fe180055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:08.256 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.255+0000 7fe19e7f5700 1 --2- 172.21.15.47:0/3323907397 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe180053950 0x7fe180055e10 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7fe19400b000 tx=0x7fe194006df0).stop 2022-06-29T15:48:08.257 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.255+0000 7fe19e7f5700 1 -- 172.21.15.47:0/3323907397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1980b1610 msgr2=0x7fe198141e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:08.257 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.255+0000 7fe19e7f5700 1 --2- 172.21.15.47:0/3323907397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1980b1610 0x7fe198141e20 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7fe1980634b0 tx=0x7fe188015000).stop 2022-06-29T15:48:08.257 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.255+0000 7fe19e7f5700 1 -- 172.21.15.47:0/3323907397 shutdown_connections 2022-06-29T15:48:08.257 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.255+0000 7fe19e7f5700 1 --2- 172.21.15.47:0/3323907397 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe19814b4c0 0x7fe198142f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:08.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.255+0000 7fe19e7f5700 1 --2- 172.21.15.47:0/3323907397 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe180053950 0x7fe180055e10 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:08.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.255+0000 7fe19e7f5700 1 --2- 172.21.15.47:0/3323907397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1980b58d0 0x7fe1980abc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:08.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.255+0000 7fe19e7f5700 1 --2- 172.21.15.47:0/3323907397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1980b1610 0x7fe198141e20 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:08.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.255+0000 7fe19e7f5700 1 -- 172.21.15.47:0/3323907397 >> 172.21.15.47:0/3323907397 conn(0x7fe198062560 msgr2=0x7fe1980b3210 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:08.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.255+0000 7fe19e7f5700 1 -- 172.21.15.47:0/3323907397 shutdown_connections 2022-06-29T15:48:08.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.255+0000 7fe19e7f5700 1 -- 172.21.15.47:0/3323907397 wait complete. 2022-06-29T15:48:08.271 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-06-29T15:48:08.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.383+0000 7f6ec5d40700 1 Processor -- start 2022-06-29T15:48:08.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.383+0000 7f6ec5d40700 1 -- start start 2022-06-29T15:48:08.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.383+0000 7f6ec5d40700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6ec00b6e20 0x7f6ec01447c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:08.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.383+0000 7f6ec5d40700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6ec00630f0 0x7f6ec01412e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:08.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.383+0000 7f6ec5d40700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ec00b7200 con 0x7f6ec0144cb0 2022-06-29T15:48:08.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.383+0000 7f6ec5d40700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ec005f5c0 con 0x7f6ec00b6e20 2022-06-29T15:48:08.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.383+0000 7f6ec5d40700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec005f860 con 0x7f6ec00630f0 2022-06-29T15:48:08.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.383+0000 7f6ec553f700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f6ec0144cb0 0x7f6ec01417d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35210/0 (socket says 172.21.15.47:35210) 2022-06-29T15:48:08.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.383+0000 7f6ec4d3e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6ec00b6e20 0x7f6ec01447c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:08.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.383+0000 7f6ec553f700 1 -- 172.21.15.47:0/1586705174 learned_addr learned my addr 172.21.15.47:0/1586705174 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:48:08.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.383+0000 7f6ebffff700 1 --2- 172.21.15.47:0/1586705174 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6ec00630f0 0x7f6ec01412e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:08.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.383+0000 7f6ec4d3e700 1 -- 172.21.15.47:0/1586705174 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6ec00630f0 msgr2=0x7f6ec01412e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:08.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.383+0000 7f6ec4d3e700 1 --2- 172.21.15.47:0/1586705174 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6ec00630f0 0x7f6ec01412e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:08.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.383+0000 7f6ec4d3e700 1 -- 172.21.15.47:0/1586705174 >> v1:172.21.15.47:6789/0 conn(0x7f6ec0144cb0 legacy=0x7f6ec01417d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:08.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.383+0000 7f6ec4d3e700 1 -- 172.21.15.47:0/1586705174 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec00afcb0 con 0x7f6ec00b6e20 2022-06-29T15:48:08.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.383+0000 7f6ebf7fe700 1 -- 172.21.15.47:0/1586705174 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2827508422 0 0) 0x7f6ec00b7200 con 0x7f6ec0144cb0 2022-06-29T15:48:08.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.383+0000 7f6ebffff700 1 --2- 172.21.15.47:0/1586705174 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6ec00630f0 0x7f6ec01412e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:48:08.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.383+0000 7f6ec4d3e700 1 --2- 172.21.15.47:0/1586705174 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ec00b6e20 0x7f6ec01447c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7f6eb4002f70 tx=0x7f6eb4006df0).ready entity=mon.0 client_cookie=4c64e3b8b5b291ca server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:08.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.387+0000 7f6ebf7fe700 1 -- 172.21.15.47:0/1586705174 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6eb400d570 con 0x7f6ec00b6e20 2022-06-29T15:48:08.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.387+0000 7f6ebf7fe700 1 -- 172.21.15.47:0/1586705174 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb4000e90 con 0x7f6ec00b6e20 2022-06-29T15:48:08.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.387+0000 7f6ebf7fe700 1 -- 172.21.15.47:0/1586705174 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6eb400d8c0 con 0x7f6ec00b6e20 2022-06-29T15:48:08.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.387+0000 7f6ec5d40700 1 -- 172.21.15.47:0/1586705174 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ec00b6e20 msgr2=0x7f6ec01447c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:08.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.387+0000 7f6ec5d40700 1 --2- 172.21.15.47:0/1586705174 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ec00b6e20 0x7f6ec01447c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7f6eb4002f70 tx=0x7f6eb4006df0).stop 2022-06-29T15:48:08.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.387+0000 7f6ec5d40700 1 -- 172.21.15.47:0/1586705174 shutdown_connections 2022-06-29T15:48:08.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.387+0000 7f6ec5d40700 1 --2- 172.21.15.47:0/1586705174 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6ec00630f0 0x7f6ec01412e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:08.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.387+0000 7f6ec5d40700 1 --2- 172.21.15.47:0/1586705174 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ec00b6e20 0x7f6ec01447c0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:08.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.387+0000 7f6ec5d40700 1 -- 172.21.15.47:0/1586705174 >> 172.21.15.47:0/1586705174 conn(0x7f6ec0063670 msgr2=0x7f6ec0063a50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:08.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.387+0000 7f6ec5d40700 1 -- 172.21.15.47:0/1586705174 shutdown_connections 2022-06-29T15:48:08.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.387+0000 7f6ec5d40700 1 -- 172.21.15.47:0/1586705174 wait complete. 2022-06-29T15:48:08.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.387+0000 7f6ec5d40700 1 Processor -- start 2022-06-29T15:48:08.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.387+0000 7f6ec5d40700 1 -- start start 2022-06-29T15:48:08.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.391+0000 7f6ec5d40700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ec00630f0 0x7f6ec00adc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:08.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.391+0000 7f6ec5d40700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6ec00b6e20 0x7f6ec0143060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:08.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.391+0000 7f6ec5d40700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ec0144cb0 0x7f6ec0143b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:08.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.391+0000 7f6ec5d40700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec0143550 con 0x7f6ec0144cb0 2022-06-29T15:48:08.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.391+0000 7f6ec5d40700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec00aa450 con 0x7f6ec00630f0 2022-06-29T15:48:08.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.391+0000 7f6ec5d40700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec00aa6f0 con 0x7f6ec00b6e20 2022-06-29T15:48:08.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.391+0000 7f6ec553f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ec0144cb0 0x7f6ec0143b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:08.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.391+0000 7f6ec553f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ec0144cb0 0x7f6ec0143b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49664/0 (socket says 172.21.15.47:49664) 2022-06-29T15:48:08.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.391+0000 7f6ec553f700 1 -- 172.21.15.47:0/1541003383 learned_addr learned my addr 172.21.15.47:0/1541003383 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:08.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.391+0000 7f6ebffff700 1 --2- 172.21.15.47:0/1541003383 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6ec00b6e20 0x7f6ec0143060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:08.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.391+0000 7f6ec4d3e700 1 --2- 172.21.15.47:0/1541003383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ec00630f0 0x7f6ec00adc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:08.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.391+0000 7f6ec553f700 1 -- 172.21.15.47:0/1541003383 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6ec00b6e20 msgr2=0x7f6ec0143060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:08.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.391+0000 7f6ec553f700 1 --2- 172.21.15.47:0/1541003383 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6ec00b6e20 0x7f6ec0143060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:08.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.391+0000 7f6ec553f700 1 -- 172.21.15.47:0/1541003383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ec00630f0 msgr2=0x7f6ec00adc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:08.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.391+0000 7f6ec553f700 1 --2- 172.21.15.47:0/1541003383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ec00630f0 0x7f6ec00adc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:08.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.391+0000 7f6ec553f700 1 -- 172.21.15.47:0/1541003383 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb400b180 con 0x7f6ec0144cb0 2022-06-29T15:48:08.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.391+0000 7f6ebffff700 1 --2- 172.21.15.47:0/1541003383 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6ec00b6e20 0x7f6ec0143060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:48:08.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.391+0000 7f6ec553f700 1 --2- 172.21.15.47:0/1541003383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ec0144cb0 0x7f6ec0143b80 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7f6eb8001250 tx=0x7f6eb8001360).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:08.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.391+0000 7f6ebdffb700 1 -- 172.21.15.47:0/1541003383 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6eb8005ba0 con 0x7f6ec0144cb0 2022-06-29T15:48:08.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.391+0000 7f6ebdffb700 1 -- 172.21.15.47:0/1541003383 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb8005d00 con 0x7f6ec0144cb0 2022-06-29T15:48:08.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.391+0000 7f6ec5d40700 1 -- 172.21.15.47:0/1541003383 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ec00aa990 con 0x7f6ec0144cb0 2022-06-29T15:48:08.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.391+0000 7f6ebdffb700 1 -- 172.21.15.47:0/1541003383 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6eb8004a90 con 0x7f6ec0144cb0 2022-06-29T15:48:08.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.391+0000 7f6ec5d40700 1 -- 172.21.15.47:0/1541003383 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ec00ad570 con 0x7f6ec0144cb0 2022-06-29T15:48:08.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.391+0000 7f6ec5d40700 1 -- 172.21.15.47:0/1541003383 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ec00aab20 con 0x7f6ec0144cb0 2022-06-29T15:48:08.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.395+0000 7f6ebdffb700 1 -- 172.21.15.47:0/1541003383 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6eb8004bf0 con 0x7f6ec0144cb0 2022-06-29T15:48:08.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.395+0000 7f6ebdffb700 1 --2- 172.21.15.47:0/1541003383 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6ea00539e0 0x7f6ea0055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:08.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.395+0000 7f6ebdffb700 1 -- 172.21.15.47:0/1541003383 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f6eb806bd40 con 0x7f6ec0144cb0 2022-06-29T15:48:08.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.395+0000 7f6ec4d3e700 1 --2- 172.21.15.47:0/1541003383 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6ea00539e0 0x7f6ea0055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:08.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.395+0000 7f6ec4d3e700 1 --2- 172.21.15.47:0/1541003383 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6ea00539e0 0x7f6ea0055ea0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7f6eb4018000 tx=0x7f6eb4014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:08.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.399+0000 7f6ebdffb700 1 -- 172.21.15.47:0/1541003383 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6eb8042ab0 con 0x7f6ec0144cb0 2022-06-29T15:48:08.528 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:08.527+0000 7f6ec5d40700 1 -- 172.21.15.47:0/1541003383 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6ec00573d0 con 0x7f6ec0144cb0 2022-06-29T15:48:09.265 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.263+0000 7f6ebdffb700 1 -- 172.21.15.47:0/1541003383 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v70) v1 ==== 185+0+0 (secure 0 0 0) 0x7f6eb80460d0 con 0x7f6ec0144cb0 2022-06-29T15:48:09.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.267+0000 7f6ec5d40700 1 -- 172.21.15.47:0/1541003383 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6ea00539e0 msgr2=0x7f6ea0055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:09.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.267+0000 7f6ec5d40700 1 --2- 172.21.15.47:0/1541003383 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6ea00539e0 0x7f6ea0055ea0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7f6eb4018000 tx=0x7f6eb4014000).stop 2022-06-29T15:48:09.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.267+0000 7f6ec5d40700 1 -- 172.21.15.47:0/1541003383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ec0144cb0 msgr2=0x7f6ec0143b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:09.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.267+0000 7f6ec5d40700 1 --2- 172.21.15.47:0/1541003383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ec0144cb0 0x7f6ec0143b80 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7f6eb8001250 tx=0x7f6eb8001360).stop 2022-06-29T15:48:09.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.271+0000 7f6ec5d40700 1 -- 172.21.15.47:0/1541003383 shutdown_connections 2022-06-29T15:48:09.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.271+0000 7f6ec5d40700 1 --2- 172.21.15.47:0/1541003383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ec0144cb0 0x7f6ec0143b80 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:09.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.271+0000 7f6ec5d40700 1 --2- 172.21.15.47:0/1541003383 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6ec00b6e20 0x7f6ec0143060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:09.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.271+0000 7f6ec5d40700 1 --2- 172.21.15.47:0/1541003383 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6ea00539e0 0x7f6ea0055ea0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:09.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.271+0000 7f6ec5d40700 1 --2- 172.21.15.47:0/1541003383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ec00630f0 0x7f6ec00adc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:09.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.271+0000 7f6ec5d40700 1 -- 172.21.15.47:0/1541003383 >> 172.21.15.47:0/1541003383 conn(0x7f6ec0063670 msgr2=0x7f6ec00b3590 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:09.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.271+0000 7f6ec5d40700 1 -- 172.21.15.47:0/1541003383 shutdown_connections 2022-06-29T15:48:09.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.271+0000 7f6ec5d40700 1 -- 172.21.15.47:0/1541003383 wait complete. 2022-06-29T15:48:09.275 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' removed 2022-06-29T15:48:09.290 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-06-29T15:48:09.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.399+0000 7f4a33fff700 1 Processor -- start 2022-06-29T15:48:09.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.399+0000 7f4a33fff700 1 -- start start 2022-06-29T15:48:09.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.403+0000 7f4a33fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4a3405c850 0x7f4a34143a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:09.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.403+0000 7f4a33fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4a340b5d90 0x7f4a340b4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:09.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.403+0000 7f4a33fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4a340676a0 0x7f4a34140130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:09.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.403+0000 7f4a33fff700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a3405cc30 con 0x7f4a340b5d90 2022-06-29T15:48:09.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.403+0000 7f4a33fff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a341430a0 con 0x7f4a3405c850 2022-06-29T15:48:09.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.403+0000 7f4a33fff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a34143300 con 0x7f4a340676a0 2022-06-29T15:48:09.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.403+0000 7f4a327fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4a340b5d90 0x7f4a340b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:09.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.403+0000 7f4a327fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4a340b5d90 0x7f4a340b4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49672/0 (socket says 172.21.15.47:49672) 2022-06-29T15:48:09.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.403+0000 7f4a327fc700 1 -- 172.21.15.47:0/228622801 learned_addr learned my addr 172.21.15.47:0/228622801 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:09.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.403+0000 7f4a337fe700 1 --2- 172.21.15.47:0/228622801 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4a340676a0 0x7f4a34140130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:09.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.403+0000 7f4a32ffd700 1 --2- 172.21.15.47:0/228622801 >> v2:172.21.15.64:3300/0 conn(0x7f4a3405c850 0x7f4a34143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:09.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.403+0000 7f4a327fc700 1 -- 172.21.15.47:0/228622801 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4a340676a0 msgr2=0x7f4a34140130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:09.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.403+0000 7f4a327fc700 1 --2- 172.21.15.47:0/228622801 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4a340676a0 0x7f4a34140130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:09.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.403+0000 7f4a327fc700 1 -- 172.21.15.47:0/228622801 >> v2:172.21.15.64:3300/0 conn(0x7f4a3405c850 msgr2=0x7f4a34143a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:09.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.403+0000 7f4a327fc700 1 --2- 172.21.15.47:0/228622801 >> v2:172.21.15.64:3300/0 conn(0x7f4a3405c850 0x7f4a34143a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:09.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.403+0000 7f4a327fc700 1 -- 172.21.15.47:0/228622801 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a341435a0 con 0x7f4a340b5d90 2022-06-29T15:48:09.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.403+0000 7f4a327fc700 1 --2- 172.21.15.47:0/228622801 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a340b5d90 0x7f4a340b4860 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7f4a1c002f70 tx=0x7f4a1c006eb0).ready entity=mon.0 client_cookie=977aded5298e1124 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:09.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.403+0000 7f4a31ffb700 1 -- 172.21.15.47:0/228622801 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4a1c00d570 con 0x7f4a340b5d90 2022-06-29T15:48:09.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.403+0000 7f4a31ffb700 1 -- 172.21.15.47:0/228622801 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a1c000e90 con 0x7f4a340b5d90 2022-06-29T15:48:09.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.403+0000 7f4a31ffb700 1 -- 172.21.15.47:0/228622801 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4a1c00d900 con 0x7f4a340b5d90 2022-06-29T15:48:09.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.403+0000 7f4a33fff700 1 -- 172.21.15.47:0/228622801 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a340b5d90 msgr2=0x7f4a340b4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:09.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.403+0000 7f4a33fff700 1 --2- 172.21.15.47:0/228622801 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a340b5d90 0x7f4a340b4860 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7f4a1c002f70 tx=0x7f4a1c006eb0).stop 2022-06-29T15:48:09.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.403+0000 7f4a33fff700 1 -- 172.21.15.47:0/228622801 shutdown_connections 2022-06-29T15:48:09.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.403+0000 7f4a33fff700 1 --2- 172.21.15.47:0/228622801 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4a340676a0 0x7f4a34140130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:09.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.403+0000 7f4a33fff700 1 --2- 172.21.15.47:0/228622801 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a340b5d90 0x7f4a340b4860 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:09.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.403+0000 7f4a33fff700 1 --2- 172.21.15.47:0/228622801 >> v2:172.21.15.64:3300/0 conn(0x7f4a3405c850 0x7f4a34143a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:09.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.403+0000 7f4a33fff700 1 -- 172.21.15.47:0/228622801 >> 172.21.15.47:0/228622801 conn(0x7f4a34060e00 msgr2=0x7f4a340611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:09.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.403+0000 7f4a33fff700 1 -- 172.21.15.47:0/228622801 shutdown_connections 2022-06-29T15:48:09.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.403+0000 7f4a33fff700 1 -- 172.21.15.47:0/228622801 wait complete. 2022-06-29T15:48:09.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.407+0000 7f4a33fff700 1 Processor -- start 2022-06-29T15:48:09.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.407+0000 7f4a33fff700 1 -- start start 2022-06-29T15:48:09.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.407+0000 7f4a33fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a3405c850 0x7f4a340b88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:09.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.407+0000 7f4a33fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a340676a0 0x7f4a340b8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:09.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.407+0000 7f4a33fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4a340b5d90 0x7f4a34155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:09.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.407+0000 7f4a33fff700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a340b9770 con 0x7f4a3405c850 2022-06-29T15:48:09.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.407+0000 7f4a33fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a341560c0 con 0x7f4a340676a0 2022-06-29T15:48:09.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.407+0000 7f4a33fff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a340b9e10 con 0x7f4a340b5d90 2022-06-29T15:48:09.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.407+0000 7f4a32ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a3405c850 0x7f4a340b88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:09.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.407+0000 7f4a32ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a3405c850 0x7f4a340b88a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49674/0 (socket says 172.21.15.47:49674) 2022-06-29T15:48:09.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.407+0000 7f4a32ffd700 1 -- 172.21.15.47:0/832937562 learned_addr learned my addr 172.21.15.47:0/832937562 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:09.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.407+0000 7f4a327fc700 1 --2- 172.21.15.47:0/832937562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a340676a0 0x7f4a340b8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:09.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.407+0000 7f4a337fe700 1 --2- 172.21.15.47:0/832937562 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4a340b5d90 0x7f4a34155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:09.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.411+0000 7f4a32ffd700 1 -- 172.21.15.47:0/832937562 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4a340b5d90 msgr2=0x7f4a34155bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:09.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.411+0000 7f4a32ffd700 1 --2- 172.21.15.47:0/832937562 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4a340b5d90 0x7f4a34155bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:09.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.411+0000 7f4a32ffd700 1 -- 172.21.15.47:0/832937562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a340676a0 msgr2=0x7f4a340b8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:09.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.411+0000 7f4a32ffd700 1 --2- 172.21.15.47:0/832937562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a340676a0 0x7f4a340b8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:09.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.411+0000 7f4a32ffd700 1 -- 172.21.15.47:0/832937562 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a1c00b1e0 con 0x7f4a3405c850 2022-06-29T15:48:09.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.411+0000 7f4a32ffd700 1 --2- 172.21.15.47:0/832937562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a3405c850 0x7f4a340b88a0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7f4a2400c000 tx=0x7f4a24006d90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:09.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.411+0000 7f4a2bfff700 1 -- 172.21.15.47:0/832937562 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4a24011750 con 0x7f4a3405c850 2022-06-29T15:48:09.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.411+0000 7f4a2bfff700 1 -- 172.21.15.47:0/832937562 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a240118b0 con 0x7f4a3405c850 2022-06-29T15:48:09.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.411+0000 7f4a33fff700 1 -- 172.21.15.47:0/832937562 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a341562f0 con 0x7f4a3405c850 2022-06-29T15:48:09.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.411+0000 7f4a2bfff700 1 -- 172.21.15.47:0/832937562 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4a24011c10 con 0x7f4a3405c850 2022-06-29T15:48:09.426 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.411+0000 7f4a33fff700 1 -- 172.21.15.47:0/832937562 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a340b9970 con 0x7f4a3405c850 2022-06-29T15:48:09.426 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.411+0000 7f4a33fff700 1 -- 172.21.15.47:0/832937562 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a340573d0 con 0x7f4a3405c850 2022-06-29T15:48:09.427 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.411+0000 7f4a2bfff700 1 -- 172.21.15.47:0/832937562 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4a24011d70 con 0x7f4a3405c850 2022-06-29T15:48:09.427 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.415+0000 7f4a2bfff700 1 --2- 172.21.15.47:0/832937562 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4a18053950 0x7f4a18055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:09.427 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.415+0000 7f4a2bfff700 1 -- 172.21.15.47:0/832937562 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6068+0+0 (secure 0 0 0) 0x7f4a2406c1a0 con 0x7f4a3405c850 2022-06-29T15:48:09.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.415+0000 7f4a327fc700 1 --2- 172.21.15.47:0/832937562 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4a18053950 0x7f4a18055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:09.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.415+0000 7f4a327fc700 1 --2- 172.21.15.47:0/832937562 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4a18053950 0x7f4a18055e10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7f4a1c01c000 tx=0x7f4a1c015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:09.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.419+0000 7f4a2bfff700 1 -- 172.21.15.47:0/832937562 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4a24042fb0 con 0x7f4a3405c850 2022-06-29T15:48:09.542 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:09.539+0000 7f4a33fff700 1 -- 172.21.15.47:0/832937562 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4a34061b00 con 0x7f4a3405c850 2022-06-29T15:48:10.268 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.267+0000 7f4a2bfff700 1 -- 172.21.15.47:0/832937562 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v71) v1 ==== 173+0+0 (secure 0 0 0) 0x7f4a240465d0 con 0x7f4a3405c850 2022-06-29T15:48:10.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.271+0000 7f4a33fff700 1 -- 172.21.15.47:0/832937562 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4a18053950 msgr2=0x7f4a18055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:10.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.271+0000 7f4a33fff700 1 --2- 172.21.15.47:0/832937562 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4a18053950 0x7f4a18055e10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7f4a1c01c000 tx=0x7f4a1c015000).stop 2022-06-29T15:48:10.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.271+0000 7f4a33fff700 1 -- 172.21.15.47:0/832937562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a3405c850 msgr2=0x7f4a340b88a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:10.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.271+0000 7f4a33fff700 1 --2- 172.21.15.47:0/832937562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a3405c850 0x7f4a340b88a0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7f4a2400c000 tx=0x7f4a24006d90).stop 2022-06-29T15:48:10.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.271+0000 7f4a33fff700 1 -- 172.21.15.47:0/832937562 shutdown_connections 2022-06-29T15:48:10.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.271+0000 7f4a33fff700 1 --2- 172.21.15.47:0/832937562 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4a340b5d90 0x7f4a34155bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:10.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.271+0000 7f4a33fff700 1 --2- 172.21.15.47:0/832937562 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4a18053950 0x7f4a18055e10 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:10.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.271+0000 7f4a33fff700 1 --2- 172.21.15.47:0/832937562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a340676a0 0x7f4a340b8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:10.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.271+0000 7f4a33fff700 1 --2- 172.21.15.47:0/832937562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a3405c850 0x7f4a340b88a0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:10.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.271+0000 7f4a33fff700 1 -- 172.21.15.47:0/832937562 >> 172.21.15.47:0/832937562 conn(0x7f4a34060e00 msgr2=0x7f4a341464f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:10.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.271+0000 7f4a33fff700 1 -- 172.21.15.47:0/832937562 shutdown_connections 2022-06-29T15:48:10.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.271+0000 7f4a33fff700 1 -- 172.21.15.47:0/832937562 wait complete. 2022-06-29T15:48:10.277 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' removed 2022-06-29T15:48:10.289 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-06-29T15:48:10.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.399+0000 7fcb5f59e700 1 Processor -- start 2022-06-29T15:48:10.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.399+0000 7fcb5f59e700 1 -- start start 2022-06-29T15:48:10.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.399+0000 7fcb5f59e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcb600b5c10 0x7fcb6013e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:10.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.399+0000 7fcb5f59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcb600ab8d0 0x7fcb600b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:10.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.399+0000 7fcb5f59e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb600b5ff0 con 0x7fcb600601a0 2022-06-29T15:48:10.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.399+0000 7fcb5f59e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb600ac910 con 0x7fcb600b5c10 2022-06-29T15:48:10.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.399+0000 7fcb5f59e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb600acbb0 con 0x7fcb600ab8d0 2022-06-29T15:48:10.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.399+0000 7fcb5e59c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcb600b5c10 0x7fcb6013e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:10.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.399+0000 7fcb5e59c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcb600b5c10 0x7fcb6013e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49682/0 (socket says 172.21.15.47:49682) 2022-06-29T15:48:10.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.399+0000 7fcb5e59c700 1 -- 172.21.15.47:0/3448496155 learned_addr learned my addr 172.21.15.47:0/3448496155 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:10.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.399+0000 7fcb5ed9d700 1 --2- 172.21.15.47:0/3448496155 >> v2:172.21.15.64:3300/0 conn(0x7fcb600ab8d0 0x7fcb600b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:10.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.399+0000 7fcb5e59c700 1 -- 172.21.15.47:0/3448496155 >> v2:172.21.15.64:3300/0 conn(0x7fcb600ab8d0 msgr2=0x7fcb600b0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:10.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.399+0000 7fcb5e59c700 1 --2- 172.21.15.47:0/3448496155 >> v2:172.21.15.64:3300/0 conn(0x7fcb600ab8d0 0x7fcb600b0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:10.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.399+0000 7fcb5e59c700 1 -- 172.21.15.47:0/3448496155 >> v1:172.21.15.64:6789/0 conn(0x7fcb600601a0 legacy=0x7fcb600afd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:10.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.399+0000 7fcb5e59c700 1 -- 172.21.15.47:0/3448496155 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb600ace50 con 0x7fcb600b5c10 2022-06-29T15:48:10.410 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.399+0000 7fcb5e59c700 1 --2- 172.21.15.47:0/3448496155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb600b5c10 0x7fcb6013e290 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 rx=0x7fcb50002f70 tx=0x7fcb50006eb0).ready entity=mon.0 client_cookie=4721c24f5631e765 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:10.410 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.399+0000 7fcb5d59a700 1 -- 172.21.15.47:0/3448496155 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb5000d5c0 con 0x7fcb600b5c10 2022-06-29T15:48:10.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.403+0000 7fcb5d59a700 1 -- 172.21.15.47:0/3448496155 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb50000e90 con 0x7fcb600b5c10 2022-06-29T15:48:10.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.403+0000 7fcb5d59a700 1 -- 172.21.15.47:0/3448496155 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb5000d950 con 0x7fcb600b5c10 2022-06-29T15:48:10.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.403+0000 7fcb5f59e700 1 -- 172.21.15.47:0/3448496155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb600b5c10 msgr2=0x7fcb6013e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:10.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.403+0000 7fcb5f59e700 1 --2- 172.21.15.47:0/3448496155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb600b5c10 0x7fcb6013e290 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 rx=0x7fcb50002f70 tx=0x7fcb50006eb0).stop 2022-06-29T15:48:10.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.403+0000 7fcb5f59e700 1 -- 172.21.15.47:0/3448496155 shutdown_connections 2022-06-29T15:48:10.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.403+0000 7fcb5f59e700 1 --2- 172.21.15.47:0/3448496155 >> v2:172.21.15.64:3300/0 conn(0x7fcb600ab8d0 0x7fcb600b0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:10.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.403+0000 7fcb5f59e700 1 --2- 172.21.15.47:0/3448496155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb600b5c10 0x7fcb6013e290 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:10.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.403+0000 7fcb5f59e700 1 -- 172.21.15.47:0/3448496155 >> 172.21.15.47:0/3448496155 conn(0x7fcb60062560 msgr2=0x7fcb60062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:10.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.403+0000 7fcb5f59e700 1 -- 172.21.15.47:0/3448496155 shutdown_connections 2022-06-29T15:48:10.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.403+0000 7fcb5f59e700 1 -- 172.21.15.47:0/3448496155 wait complete. 2022-06-29T15:48:10.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.403+0000 7fcb5f59e700 1 Processor -- start 2022-06-29T15:48:10.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.403+0000 7fcb5f59e700 1 -- start start 2022-06-29T15:48:10.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.403+0000 7fcb5f59e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcb600601a0 0x7fcb600b3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:10.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.403+0000 7fcb5f59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb600ab8d0 0x7fcb601440f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:10.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.403+0000 7fcb5f59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb600b5c10 0x7fcb60144bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:10.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.403+0000 7fcb5f59e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb600aac50 con 0x7fcb600ab8d0 2022-06-29T15:48:10.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.403+0000 7fcb5f59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb600b3510 con 0x7fcb600b5c10 2022-06-29T15:48:10.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.403+0000 7fcb5f59e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb600b41e0 con 0x7fcb600601a0 2022-06-29T15:48:10.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.403+0000 7fcb5dd9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb600ab8d0 0x7fcb601440f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:10.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.403+0000 7fcb5dd9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb600ab8d0 0x7fcb601440f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49690/0 (socket says 172.21.15.47:49690) 2022-06-29T15:48:10.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.403+0000 7fcb5dd9b700 1 -- 172.21.15.47:0/568410915 learned_addr learned my addr 172.21.15.47:0/568410915 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:10.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.403+0000 7fcb5e59c700 1 --2- 172.21.15.47:0/568410915 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcb600601a0 0x7fcb600b3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:10.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.403+0000 7fcb5ed9d700 1 --2- 172.21.15.47:0/568410915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb600b5c10 0x7fcb60144bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:10.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.403+0000 7fcb5dd9b700 1 -- 172.21.15.47:0/568410915 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcb600601a0 msgr2=0x7fcb600b3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:10.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.403+0000 7fcb5dd9b700 1 --2- 172.21.15.47:0/568410915 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcb600601a0 0x7fcb600b3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:10.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.403+0000 7fcb5dd9b700 1 -- 172.21.15.47:0/568410915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb600b5c10 msgr2=0x7fcb60144bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:10.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.403+0000 7fcb5dd9b700 1 --2- 172.21.15.47:0/568410915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb600b5c10 0x7fcb60144bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:10.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.403+0000 7fcb5dd9b700 1 -- 172.21.15.47:0/568410915 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb5000b1e0 con 0x7fcb600ab8d0 2022-06-29T15:48:10.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.403+0000 7fcb5dd9b700 1 --2- 172.21.15.47:0/568410915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb600ab8d0 0x7fcb601440f0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7fcb60062140 tx=0x7fcb48007f30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:10.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.403+0000 7fcb577fe700 1 -- 172.21.15.47:0/568410915 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb48006aa0 con 0x7fcb600ab8d0 2022-06-29T15:48:10.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.403+0000 7fcb577fe700 1 -- 172.21.15.47:0/568410915 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb48006c00 con 0x7fcb600ab8d0 2022-06-29T15:48:10.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.403+0000 7fcb577fe700 1 -- 172.21.15.47:0/568410915 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb48007030 con 0x7fcb600ab8d0 2022-06-29T15:48:10.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.403+0000 7fcb5f59e700 1 -- 172.21.15.47:0/568410915 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb600b3270 con 0x7fcb600ab8d0 2022-06-29T15:48:10.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.407+0000 7fcb5f59e700 1 -- 172.21.15.47:0/568410915 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb6013b8c0 con 0x7fcb600ab8d0 2022-06-29T15:48:10.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.407+0000 7fcb5f59e700 1 -- 172.21.15.47:0/568410915 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb600573d0 con 0x7fcb600ab8d0 2022-06-29T15:48:10.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.407+0000 7fcb577fe700 1 -- 172.21.15.47:0/568410915 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fcb48010070 con 0x7fcb600ab8d0 2022-06-29T15:48:10.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.407+0000 7fcb577fe700 1 --2- 172.21.15.47:0/568410915 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcb440539e0 0x7fcb44055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:10.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.407+0000 7fcb577fe700 1 -- 172.21.15.47:0/568410915 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fcb4806b9b0 con 0x7fcb600ab8d0 2022-06-29T15:48:10.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.407+0000 7fcb5e59c700 1 --2- 172.21.15.47:0/568410915 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcb440539e0 0x7fcb44055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:10.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.407+0000 7fcb5e59c700 1 --2- 172.21.15.47:0/568410915 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcb440539e0 0x7fcb44055ea0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7fcb5001c000 tx=0x7fcb50015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:10.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.419+0000 7fcb577fe700 1 -- 172.21.15.47:0/568410915 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcb48042a50 con 0x7fcb600ab8d0 2022-06-29T15:48:10.546 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.543+0000 7fcb5f59e700 1 -- 172.21.15.47:0/568410915 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fcb600a8c10 con 0x7fcb600ab8d0 2022-06-29T15:48:10.551 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.551+0000 7fcb577fe700 1 -- 172.21.15.47:0/568410915 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7fcb48046070 con 0x7fcb600ab8d0 2022-06-29T15:48:10.553 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.551+0000 7fcb5f59e700 1 -- 172.21.15.47:0/568410915 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcb440539e0 msgr2=0x7fcb44055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:10.554 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.551+0000 7fcb5f59e700 1 --2- 172.21.15.47:0/568410915 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcb440539e0 0x7fcb44055ea0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7fcb5001c000 tx=0x7fcb50015000).stop 2022-06-29T15:48:10.554 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.551+0000 7fcb5f59e700 1 -- 172.21.15.47:0/568410915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb600ab8d0 msgr2=0x7fcb601440f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:10.554 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.551+0000 7fcb5f59e700 1 --2- 172.21.15.47:0/568410915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb600ab8d0 0x7fcb601440f0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7fcb60062140 tx=0x7fcb48007f30).stop 2022-06-29T15:48:10.555 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.551+0000 7fcb5f59e700 1 -- 172.21.15.47:0/568410915 shutdown_connections 2022-06-29T15:48:10.555 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.551+0000 7fcb5f59e700 1 --2- 172.21.15.47:0/568410915 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcb440539e0 0x7fcb44055ea0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:10.555 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.551+0000 7fcb5f59e700 1 --2- 172.21.15.47:0/568410915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb600b5c10 0x7fcb60144bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:10.555 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.551+0000 7fcb5f59e700 1 --2- 172.21.15.47:0/568410915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb600ab8d0 0x7fcb601440f0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:10.555 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.551+0000 7fcb5f59e700 1 --2- 172.21.15.47:0/568410915 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcb600601a0 0x7fcb600b3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:10.556 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.551+0000 7fcb5f59e700 1 -- 172.21.15.47:0/568410915 >> 172.21.15.47:0/568410915 conn(0x7fcb60062560 msgr2=0x7fcb6013be10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:10.556 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.551+0000 7fcb5f59e700 1 -- 172.21.15.47:0/568410915 shutdown_connections 2022-06-29T15:48:10.556 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.551+0000 7fcb5f59e700 1 -- 172.21.15.47:0/568410915 wait complete. 2022-06-29T15:48:10.568 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout' 2022-06-29T15:48:10.676 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.675+0000 7f4040c48700 1 Processor -- start 2022-06-29T15:48:10.676 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.675+0000 7f4040c48700 1 -- start start 2022-06-29T15:48:10.679 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.675+0000 7f4040c48700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f403c05f0b0 0x7f403c0b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:10.681 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.675+0000 7f4040c48700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f403c0b1260 0x7f403c14b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:10.681 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.675+0000 7f4040c48700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f403c0b0740 con 0x7f403c0b2120 2022-06-29T15:48:10.681 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.675+0000 7f4040c48700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f403c147460 con 0x7f403c0b1260 2022-06-29T15:48:10.681 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.675+0000 7f4040c48700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f403c0b0970 con 0x7f403c05f0b0 2022-06-29T15:48:10.682 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.675+0000 7f403ad9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f403c0b1260 0x7f403c14b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:10.682 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.675+0000 7f403a59c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f403c0b2120 0x7f403c147af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35250/0 (socket says 172.21.15.47:35250) 2022-06-29T15:48:10.682 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.675+0000 7f403a59c700 1 -- 172.21.15.47:0/851431419 learned_addr learned my addr 172.21.15.47:0/851431419 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:48:10.683 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.675+0000 7f4039d9b700 1 --2- 172.21.15.47:0/851431419 >> v2:172.21.15.64:3300/0 conn(0x7f403c05f0b0 0x7f403c0b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:10.683 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.675+0000 7f403959a700 1 -- 172.21.15.47:0/851431419 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2675835847 0 0) 0x7f403c0b0740 con 0x7f403c0b2120 2022-06-29T15:48:10.683 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.675+0000 7f403959a700 1 -- 172.21.15.47:0/851431419 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f401c003260 con 0x7f403c0b2120 2022-06-29T15:48:10.683 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.675+0000 7f403ad9d700 1 -- 172.21.15.47:0/851431419 >> v2:172.21.15.64:3300/0 conn(0x7f403c05f0b0 msgr2=0x7f403c0b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:10.684 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.675+0000 7f403ad9d700 1 --2- 172.21.15.47:0/851431419 >> v2:172.21.15.64:3300/0 conn(0x7f403c05f0b0 0x7f403c0b2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:10.684 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.675+0000 7f403ad9d700 1 -- 172.21.15.47:0/851431419 >> v1:172.21.15.47:6789/0 conn(0x7f403c0b2120 legacy=0x7f403c147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:10.684 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.675+0000 7f403ad9d700 1 -- 172.21.15.47:0/851431419 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f403c0b0c10 con 0x7f403c0b1260 2022-06-29T15:48:10.685 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.675+0000 7f403ad9d700 1 --2- 172.21.15.47:0/851431419 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f403c0b1260 0x7f403c14b520 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7f403c060d70 tx=0x7f4034006df0).ready entity=mon.0 client_cookie=c1de5a3fd277ec34 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:10.685 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.675+0000 7f403959a700 1 -- 172.21.15.47:0/851431419 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f403400d4a0 con 0x7f403c0b1260 2022-06-29T15:48:10.685 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.675+0000 7f403959a700 1 -- 172.21.15.47:0/851431419 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4034000e90 con 0x7f403c0b1260 2022-06-29T15:48:10.685 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.675+0000 7f403959a700 1 -- 172.21.15.47:0/851431419 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f403400d7f0 con 0x7f403c0b1260 2022-06-29T15:48:10.686 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.679+0000 7f4040c48700 1 -- 172.21.15.47:0/851431419 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f403c0b1260 msgr2=0x7f403c14b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:10.686 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.679+0000 7f4040c48700 1 --2- 172.21.15.47:0/851431419 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f403c0b1260 0x7f403c14b520 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7f403c060d70 tx=0x7f4034006df0).stop 2022-06-29T15:48:10.686 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.679+0000 7f4040c48700 1 -- 172.21.15.47:0/851431419 shutdown_connections 2022-06-29T15:48:10.686 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.679+0000 7f4040c48700 1 --2- 172.21.15.47:0/851431419 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f403c0b1260 0x7f403c14b520 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:10.687 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.679+0000 7f4040c48700 1 --2- 172.21.15.47:0/851431419 >> v2:172.21.15.64:3300/0 conn(0x7f403c05f0b0 0x7f403c0b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:10.687 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.679+0000 7f4040c48700 1 -- 172.21.15.47:0/851431419 >> 172.21.15.47:0/851431419 conn(0x7f403c0626f0 msgr2=0x7f403c062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:10.687 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.679+0000 7f4040c48700 1 -- 172.21.15.47:0/851431419 shutdown_connections 2022-06-29T15:48:10.687 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.679+0000 7f4040c48700 1 -- 172.21.15.47:0/851431419 wait complete. 2022-06-29T15:48:10.690 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.679+0000 7f4040c48700 1 Processor -- start 2022-06-29T15:48:10.691 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.679+0000 7f4040c48700 1 -- start start 2022-06-29T15:48:10.691 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.679+0000 7f4040c48700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f403c0b1260 0x7f403c0ada60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:10.691 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.679+0000 7f4040c48700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f403c0b2120 0x7f403c0aec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:10.692 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.679+0000 7f4040c48700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f403c0af120 0x7f403c156720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:10.692 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.679+0000 7f4040c48700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f403c0af580 con 0x7f403c0af120 2022-06-29T15:48:10.692 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.679+0000 7f4040c48700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f403c159d50 con 0x7f403c0b1260 2022-06-29T15:48:10.692 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.679+0000 7f4040c48700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f403c159ff0 con 0x7f403c0b2120 2022-06-29T15:48:10.693 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.679+0000 7f403ad9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f403c0af120 0x7f403c156720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:10.693 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.679+0000 7f403ad9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f403c0af120 0x7f403c156720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49706/0 (socket says 172.21.15.47:49706) 2022-06-29T15:48:10.693 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.679+0000 7f4039d9b700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f403c0b2120 0x7f403c0aec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:10.693 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.679+0000 7f403ad9d700 1 -- 172.21.15.47:0/888313859 learned_addr learned my addr 172.21.15.47:0/888313859 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:10.693 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.679+0000 7f403a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f403c0b1260 0x7f403c0ada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:10.694 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.683+0000 7f403ad9d700 1 -- 172.21.15.47:0/888313859 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f403c0b2120 msgr2=0x7f403c0aec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:10.694 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.683+0000 7f403ad9d700 1 --2- 172.21.15.47:0/888313859 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f403c0b2120 0x7f403c0aec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:10.694 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.683+0000 7f403ad9d700 1 -- 172.21.15.47:0/888313859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f403c0b1260 msgr2=0x7f403c0ada60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:10.695 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.683+0000 7f403ad9d700 1 --2- 172.21.15.47:0/888313859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f403c0b1260 0x7f403c0ada60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:10.695 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.683+0000 7f403ad9d700 1 -- 172.21.15.47:0/888313859 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f403400b180 con 0x7f403c0af120 2022-06-29T15:48:10.695 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.683+0000 7f403ad9d700 1 --2- 172.21.15.47:0/888313859 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f403c0af120 0x7f403c156720 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7f403c060d70 tx=0x7f4034014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:10.696 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.683+0000 7f40337fe700 1 -- 172.21.15.47:0/888313859 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4034016070 con 0x7f403c0af120 2022-06-29T15:48:10.696 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.683+0000 7f40337fe700 1 -- 172.21.15.47:0/888313859 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40340137d0 con 0x7f403c0af120 2022-06-29T15:48:10.696 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.683+0000 7f4040c48700 1 -- 172.21.15.47:0/888313859 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f403c15a290 con 0x7f403c0af120 2022-06-29T15:48:10.697 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.683+0000 7f40337fe700 1 -- 172.21.15.47:0/888313859 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4034013b30 con 0x7f403c0af120 2022-06-29T15:48:10.697 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.683+0000 7f4040c48700 1 -- 172.21.15.47:0/888313859 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f403c1492b0 con 0x7f403c0af120 2022-06-29T15:48:10.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.683+0000 7f4040c48700 1 -- 172.21.15.47:0/888313859 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f403c0573d0 con 0x7f403c0af120 2022-06-29T15:48:10.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.683+0000 7f40337fe700 1 -- 172.21.15.47:0/888313859 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4034025030 con 0x7f403c0af120 2022-06-29T15:48:10.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.683+0000 7f40337fe700 1 --2- 172.21.15.47:0/888313859 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4020053b00 0x7f4020055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:10.699 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.683+0000 7f40337fe700 1 -- 172.21.15.47:0/888313859 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f403406cb60 con 0x7f403c0af120 2022-06-29T15:48:10.699 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.683+0000 7f403a59c700 1 --2- 172.21.15.47:0/888313859 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4020053b00 0x7f4020055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:10.699 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.687+0000 7f403a59c700 1 --2- 172.21.15.47:0/888313859 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4020053b00 0x7f4020055fc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7f402400d000 tx=0x7f4024009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:10.700 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.691+0000 7f40337fe700 1 -- 172.21.15.47:0/888313859 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4034043c00 con 0x7f403c0af120 2022-06-29T15:48:10.857 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:10.855+0000 7f4040c48700 1 -- 172.21.15.47:0/888313859 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout"]} v 0) v1 -- 0x7f403c15a730 con 0x7f403c0af120 2022-06-29T15:48:11.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.283+0000 7f40337fe700 1 -- 172.21.15.47:0/888313859 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout"]}]=0 v248) v1 ==== 147+0+0 (secure 0 0 0) 0x7f4034023070 con 0x7f403c0af120 2022-06-29T15:48:11.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.287+0000 7f4040c48700 1 -- 172.21.15.47:0/888313859 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4020053b00 msgr2=0x7f4020055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:11.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.287+0000 7f4040c48700 1 --2- 172.21.15.47:0/888313859 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4020053b00 0x7f4020055fc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7f402400d000 tx=0x7f4024009040).stop 2022-06-29T15:48:11.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.287+0000 7f4040c48700 1 -- 172.21.15.47:0/888313859 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f403c0af120 msgr2=0x7f403c156720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:11.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.287+0000 7f4040c48700 1 --2- 172.21.15.47:0/888313859 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f403c0af120 0x7f403c156720 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7f403c060d70 tx=0x7f4034014000).stop 2022-06-29T15:48:11.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.287+0000 7f4040c48700 1 -- 172.21.15.47:0/888313859 shutdown_connections 2022-06-29T15:48:11.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.287+0000 7f4040c48700 1 --2- 172.21.15.47:0/888313859 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f403c0af120 0x7f403c156720 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:11.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.287+0000 7f4040c48700 1 --2- 172.21.15.47:0/888313859 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f403c0b2120 0x7f403c0aec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:11.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.287+0000 7f4040c48700 1 --2- 172.21.15.47:0/888313859 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4020053b00 0x7f4020055fc0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:11.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.287+0000 7f4040c48700 1 --2- 172.21.15.47:0/888313859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f403c0b1260 0x7f403c0ada60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:11.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.287+0000 7f4040c48700 1 -- 172.21.15.47:0/888313859 >> 172.21.15.47:0/888313859 conn(0x7f403c0626f0 msgr2=0x7f403c0b5480 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:11.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.287+0000 7f4040c48700 1 -- 172.21.15.47:0/888313859 shutdown_connections 2022-06-29T15:48:11.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.287+0000 7f4040c48700 1 -- 172.21.15.47:0/888313859 wait complete. 2022-06-29T15:48:11.305 INFO:tasks.cephfs_test_runner:test_drop_cache_command_timeout (tasks.cephfs.test_misc.TestCacheDrop) ... ok 2022-06-29T15:48:11.306 INFO:tasks.cephfs_test_runner:Starting test: test_cap_revoke_nonresponder (tasks.cephfs.test_misc.TestMisc) 2022-06-29T15:48:11.307 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder' 2022-06-29T15:48:11.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.415+0000 7ff5c18f5700 1 Processor -- start 2022-06-29T15:48:11.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.415+0000 7ff5c18f5700 1 -- start start 2022-06-29T15:48:11.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.419+0000 7ff5c18f5700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff5bc05c510 0x7ff5bc143bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:11.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.419+0000 7ff5c18f5700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5bc05c8f0 0x7ff5bc0b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:11.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.419+0000 7ff5c18f5700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5bc142f90 con 0x7ff5bc0b2c40 2022-06-29T15:48:11.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.419+0000 7ff5c18f5700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5bc1431f0 con 0x7ff5bc05c510 2022-06-29T15:48:11.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.419+0000 7ff5c18f5700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5bc143490 con 0x7ff5bc05c8f0 2022-06-29T15:48:11.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.419+0000 7ff5c10f4700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff5bc0b2c40 0x7ff5bc13cbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35268/0 (socket says 172.21.15.47:35268) 2022-06-29T15:48:11.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.419+0000 7ff5c10f4700 1 -- 172.21.15.47:0/3664911866 learned_addr learned my addr 172.21.15.47:0/3664911866 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:48:11.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.419+0000 7ff5c08f3700 1 --2- 172.21.15.47:0/3664911866 >> v2:172.21.15.47:3300/0 conn(0x7ff5bc05c510 0x7ff5bc143bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:11.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.419+0000 7ff5bbfff700 1 --2- 172.21.15.47:0/3664911866 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5bc05c8f0 0x7ff5bc0b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:11.424 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.419+0000 7ff5bb7fe700 1 -- 172.21.15.47:0/3664911866 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3621119684 0 0) 0x7ff5bc142f90 con 0x7ff5bc0b2c40 2022-06-29T15:48:11.424 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.419+0000 7ff5bb7fe700 1 -- 172.21.15.47:0/3664911866 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5a8003260 con 0x7ff5bc0b2c40 2022-06-29T15:48:11.424 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.419+0000 7ff5c08f3700 1 -- 172.21.15.47:0/3664911866 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5bc05c8f0 msgr2=0x7ff5bc0b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:11.425 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.419+0000 7ff5c08f3700 1 --2- 172.21.15.47:0/3664911866 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5bc05c8f0 0x7ff5bc0b4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:11.425 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.419+0000 7ff5c08f3700 1 -- 172.21.15.47:0/3664911866 >> v1:172.21.15.47:6789/0 conn(0x7ff5bc0b2c40 legacy=0x7ff5bc13cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:11.425 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.419+0000 7ff5c08f3700 1 -- 172.21.15.47:0/3664911866 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5bc143730 con 0x7ff5bc05c510 2022-06-29T15:48:11.426 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.419+0000 7ff5c08f3700 1 --2- 172.21.15.47:0/3664911866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5bc05c510 0x7ff5bc143bd0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7ff5b0002f70 tx=0x7ff5b0006df0).ready entity=mon.0 client_cookie=cdcea274badff6c5 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:11.426 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.419+0000 7ff5bb7fe700 1 -- 172.21.15.47:0/3664911866 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff5b000d4a0 con 0x7ff5bc05c510 2022-06-29T15:48:11.426 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.419+0000 7ff5bb7fe700 1 -- 172.21.15.47:0/3664911866 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b0000e90 con 0x7ff5bc05c510 2022-06-29T15:48:11.427 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.419+0000 7ff5bb7fe700 1 -- 172.21.15.47:0/3664911866 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff5b000d7f0 con 0x7ff5bc05c510 2022-06-29T15:48:11.431 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.419+0000 7ff5c18f5700 1 -- 172.21.15.47:0/3664911866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5bc05c510 msgr2=0x7ff5bc143bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:11.431 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.419+0000 7ff5c18f5700 1 --2- 172.21.15.47:0/3664911866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5bc05c510 0x7ff5bc143bd0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7ff5b0002f70 tx=0x7ff5b0006df0).stop 2022-06-29T15:48:11.431 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.419+0000 7ff5c18f5700 1 -- 172.21.15.47:0/3664911866 shutdown_connections 2022-06-29T15:48:11.431 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.419+0000 7ff5c18f5700 1 --2- 172.21.15.47:0/3664911866 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5bc05c8f0 0x7ff5bc0b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:11.432 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.419+0000 7ff5c18f5700 1 --2- 172.21.15.47:0/3664911866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5bc05c510 0x7ff5bc143bd0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:11.432 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.419+0000 7ff5c18f5700 1 -- 172.21.15.47:0/3664911866 >> 172.21.15.47:0/3664911866 conn(0x7ff5bc060e00 msgr2=0x7ff5bc0611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:11.432 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.423+0000 7ff5c18f5700 1 -- 172.21.15.47:0/3664911866 shutdown_connections 2022-06-29T15:48:11.433 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.423+0000 7ff5c18f5700 1 -- 172.21.15.47:0/3664911866 wait complete. 2022-06-29T15:48:11.433 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.423+0000 7ff5c18f5700 1 Processor -- start 2022-06-29T15:48:11.433 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.423+0000 7ff5c18f5700 1 -- start start 2022-06-29T15:48:11.434 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.427+0000 7ff5c18f5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5bc05c510 0x7ff5bc14a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:11.434 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.427+0000 7ff5c18f5700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5bc05c8f0 0x7ff5bc152260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:11.434 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.427+0000 7ff5c18f5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5bc0b2c40 0x7ff5bc152750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:11.435 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.427+0000 7ff5c18f5700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5bc1498d0 con 0x7ff5bc05c510 2022-06-29T15:48:11.435 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.427+0000 7ff5c18f5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5bc149b00 con 0x7ff5bc0b2c40 2022-06-29T15:48:11.435 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.427+0000 7ff5c18f5700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5bc149da0 con 0x7ff5bc05c8f0 2022-06-29T15:48:11.435 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.427+0000 7ff5c08f3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5bc05c510 0x7ff5bc14a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:11.435 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.427+0000 7ff5c08f3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5bc05c510 0x7ff5bc14a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49716/0 (socket says 172.21.15.47:49716) 2022-06-29T15:48:11.436 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.427+0000 7ff5c08f3700 1 -- 172.21.15.47:0/1749536166 learned_addr learned my addr 172.21.15.47:0/1749536166 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:11.436 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.427+0000 7ff5bbfff700 1 --2- 172.21.15.47:0/1749536166 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5bc05c8f0 0x7ff5bc152260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:11.436 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.427+0000 7ff5c10f4700 1 --2- 172.21.15.47:0/1749536166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5bc0b2c40 0x7ff5bc152750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:11.436 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.427+0000 7ff5c08f3700 1 -- 172.21.15.47:0/1749536166 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5bc05c8f0 msgr2=0x7ff5bc152260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:11.437 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.427+0000 7ff5c08f3700 1 --2- 172.21.15.47:0/1749536166 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5bc05c8f0 0x7ff5bc152260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:11.437 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.427+0000 7ff5c08f3700 1 -- 172.21.15.47:0/1749536166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5bc0b2c40 msgr2=0x7ff5bc152750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:11.437 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.427+0000 7ff5c08f3700 1 --2- 172.21.15.47:0/1749536166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5bc0b2c40 0x7ff5bc152750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:11.437 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.427+0000 7ff5c08f3700 1 -- 172.21.15.47:0/1749536166 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b000b180 con 0x7ff5bc05c510 2022-06-29T15:48:11.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.427+0000 7ff5c08f3700 1 --2- 172.21.15.47:0/1749536166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5bc05c510 0x7ff5bc14a3a0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7ff5b0002f70 tx=0x7ff5b0014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:11.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.427+0000 7ff5b9ffb700 1 -- 172.21.15.47:0/1749536166 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff5b0016070 con 0x7ff5bc05c510 2022-06-29T15:48:11.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.427+0000 7ff5b9ffb700 1 -- 172.21.15.47:0/1749536166 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b00138d0 con 0x7ff5bc05c510 2022-06-29T15:48:11.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.427+0000 7ff5c18f5700 1 -- 172.21.15.47:0/1749536166 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5bc0b5f90 con 0x7ff5bc05c510 2022-06-29T15:48:11.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.427+0000 7ff5b9ffb700 1 -- 172.21.15.47:0/1749536166 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff5b0013c30 con 0x7ff5bc05c510 2022-06-29T15:48:11.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.427+0000 7ff5c18f5700 1 -- 172.21.15.47:0/1749536166 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5bc1471d0 con 0x7ff5bc05c510 2022-06-29T15:48:11.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.427+0000 7ff5c18f5700 1 -- 172.21.15.47:0/1749536166 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5bc0573d0 con 0x7ff5bc05c510 2022-06-29T15:48:11.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.427+0000 7ff5b9ffb700 1 -- 172.21.15.47:0/1749536166 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff5b0025030 con 0x7ff5bc05c510 2022-06-29T15:48:11.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.431+0000 7ff5b9ffb700 1 --2- 172.21.15.47:0/1749536166 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff59c053b30 0x7ff59c055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:11.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.431+0000 7ff5b9ffb700 1 -- 172.21.15.47:0/1749536166 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5686+0+0 (secure 0 0 0) 0x7ff5b006cd10 con 0x7ff5bc05c510 2022-06-29T15:48:11.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.431+0000 7ff5bbfff700 1 --2- 172.21.15.47:0/1749536166 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff59c053b30 0x7ff59c055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:11.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.431+0000 7ff5bbfff700 1 --2- 172.21.15.47:0/1749536166 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff59c053b30 0x7ff59c055ff0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7ff5ac00b000 tx=0x7ff5ac006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:11.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.435+0000 7ff5b9ffb700 1 -- 172.21.15.47:0/1749536166 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff5b0043db0 con 0x7ff5bc05c510 2022-06-29T15:48:11.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:11.603+0000 7ff5c18f5700 1 -- 172.21.15.47:0/1749536166 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder"]} v 0) v1 -- 0x7ff5bc061b00 con 0x7ff5bc05c510 2022-06-29T15:48:11.962 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:48:11.965 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:48:11.967 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:48:11.970 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:48:12.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.283+0000 7ff5b9ffb700 1 -- 172.21.15.47:0/1749536166 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder"]}]=0 v249) v1 ==== 142+0+0 (secure 0 0 0) 0x7ff5b0023070 con 0x7ff5bc05c510 2022-06-29T15:48:12.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.287+0000 7ff5c18f5700 1 -- 172.21.15.47:0/1749536166 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff59c053b30 msgr2=0x7ff59c055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:12.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.287+0000 7ff5c18f5700 1 --2- 172.21.15.47:0/1749536166 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff59c053b30 0x7ff59c055ff0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7ff5ac00b000 tx=0x7ff5ac006df0).stop 2022-06-29T15:48:12.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.287+0000 7ff5c18f5700 1 -- 172.21.15.47:0/1749536166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5bc05c510 msgr2=0x7ff5bc14a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:12.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.287+0000 7ff5c18f5700 1 --2- 172.21.15.47:0/1749536166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5bc05c510 0x7ff5bc14a3a0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7ff5b0002f70 tx=0x7ff5b0014000).stop 2022-06-29T15:48:12.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.287+0000 7ff5c18f5700 1 -- 172.21.15.47:0/1749536166 shutdown_connections 2022-06-29T15:48:12.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.287+0000 7ff5c18f5700 1 --2- 172.21.15.47:0/1749536166 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff59c053b30 0x7ff59c055ff0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:12.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.287+0000 7ff5c18f5700 1 --2- 172.21.15.47:0/1749536166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5bc0b2c40 0x7ff5bc152750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:12.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.287+0000 7ff5c18f5700 1 --2- 172.21.15.47:0/1749536166 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5bc05c8f0 0x7ff5bc152260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:12.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.287+0000 7ff5c18f5700 1 --2- 172.21.15.47:0/1749536166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5bc05c510 0x7ff5bc14a3a0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:12.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.287+0000 7ff5c18f5700 1 -- 172.21.15.47:0/1749536166 >> 172.21.15.47:0/1749536166 conn(0x7ff5bc060e00 msgr2=0x7ff5bc13b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:12.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.287+0000 7ff5c18f5700 1 -- 172.21.15.47:0/1749536166 shutdown_connections 2022-06-29T15:48:12.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.287+0000 7ff5c18f5700 1 -- 172.21.15.47:0/1749536166 wait complete. 2022-06-29T15:48:12.305 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-06-29T15:48:12.426 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.423+0000 7fe967e1c700 1 Processor -- start 2022-06-29T15:48:12.426 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.423+0000 7fe967e1c700 1 -- start start 2022-06-29T15:48:12.427 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.427+0000 7fe967e1c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe960140e10 0x7fe960067850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:12.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.427+0000 7fe967e1c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe9600609d0 0x7fe960143e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:12.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.427+0000 7fe967e1c700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9601434e0 con 0x7fe960067470 2022-06-29T15:48:12.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.427+0000 7fe967e1c700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe960144370 con 0x7fe9600609d0 2022-06-29T15:48:12.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.427+0000 7fe967e1c700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe960143740 con 0x7fe960140e10 2022-06-29T15:48:12.429 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.427+0000 7fe965bb8700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe960067470 0x7fe960140700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35278/0 (socket says 172.21.15.47:35278) 2022-06-29T15:48:12.429 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.427+0000 7fe965bb8700 1 -- 172.21.15.47:0/1283668042 learned_addr learned my addr 172.21.15.47:0/1283668042 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:48:12.429 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.427+0000 7fe9663b9700 1 --2- 172.21.15.47:0/1283668042 >> v2:172.21.15.64:3300/0 conn(0x7fe9600609d0 0x7fe960143e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:12.429 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.427+0000 7fe9653b7700 1 --2- 172.21.15.47:0/1283668042 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe960140e10 0x7fe960067850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:12.430 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.427+0000 7fe964bb6700 1 -- 172.21.15.47:0/1283668042 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3083830095 0 0) 0x7fe9601434e0 con 0x7fe960067470 2022-06-29T15:48:12.430 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.427+0000 7fe964bb6700 1 -- 172.21.15.47:0/1283668042 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe948003260 con 0x7fe960067470 2022-06-29T15:48:12.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.427+0000 7fe964bb6700 1 -- 172.21.15.47:0/1283668042 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 223833966 0 0) 0x7fe948003260 con 0x7fe960067470 2022-06-29T15:48:12.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.427+0000 7fe964bb6700 1 -- 172.21.15.47:0/1283668042 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe960140e10 msgr2=0x7fe960067850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:12.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.427+0000 7fe964bb6700 1 --2- 172.21.15.47:0/1283668042 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe960140e10 0x7fe960067850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:12.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.427+0000 7fe964bb6700 1 -- 172.21.15.47:0/1283668042 >> v2:172.21.15.64:3300/0 conn(0x7fe9600609d0 msgr2=0x7fe960143e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:12.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.427+0000 7fe964bb6700 1 --2- 172.21.15.47:0/1283668042 >> v2:172.21.15.64:3300/0 conn(0x7fe9600609d0 0x7fe960143e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:12.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.427+0000 7fe964bb6700 1 -- 172.21.15.47:0/1283668042 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9601439e0 con 0x7fe960067470 2022-06-29T15:48:12.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.427+0000 7fe964bb6700 1 -- 172.21.15.47:0/1283668042 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7fe950001660 con 0x7fe960067470 2022-06-29T15:48:12.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.427+0000 7fe964bb6700 1 -- 172.21.15.47:0/1283668042 >> v1:172.21.15.47:6789/0 conn(0x7fe960067470 legacy=0x7fe960140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:12.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.427+0000 7fe964bb6700 1 --2- 172.21.15.47:0/1283668042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe948003750 0x7fe948003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:12.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.427+0000 7fe964bb6700 1 --2- 172.21.15.47:0/1283668042 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe960140e10 0x7fe948004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:12.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.427+0000 7fe964bb6700 1 --2- 172.21.15.47:0/1283668042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9480049f0 0x7fe948004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:12.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.427+0000 7fe964bb6700 1 -- 172.21.15.47:0/1283668042 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe960144370 con 0x7fe9480049f0 2022-06-29T15:48:12.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.427+0000 7fe964bb6700 1 -- 172.21.15.47:0/1283668042 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe960143740 con 0x7fe948003750 2022-06-29T15:48:12.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.427+0000 7fe964bb6700 1 -- 172.21.15.47:0/1283668042 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe94800b750 con 0x7fe960140e10 2022-06-29T15:48:12.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.427+0000 7fe9663b9700 1 --2- 172.21.15.47:0/1283668042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9480049f0 0x7fe948004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:12.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.427+0000 7fe965bb8700 1 --2- 172.21.15.47:0/1283668042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe948003750 0x7fe948003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:12.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.427+0000 7fe9653b7700 1 --2- 172.21.15.47:0/1283668042 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe960140e10 0x7fe948004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:12.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.427+0000 7fe9663b9700 1 -- 172.21.15.47:0/1283668042 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe960140e10 msgr2=0x7fe948004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:12.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.427+0000 7fe9663b9700 1 --2- 172.21.15.47:0/1283668042 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe960140e10 0x7fe948004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:12.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.427+0000 7fe9663b9700 1 -- 172.21.15.47:0/1283668042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe948003750 msgr2=0x7fe948003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:12.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.427+0000 7fe9663b9700 1 --2- 172.21.15.47:0/1283668042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe948003750 0x7fe948003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:12.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.427+0000 7fe9663b9700 1 -- 172.21.15.47:0/1283668042 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe948003260 con 0x7fe9480049f0 2022-06-29T15:48:12.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.427+0000 7fe9663b9700 1 --2- 172.21.15.47:0/1283668042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9480049f0 0x7fe948004e70 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7fe95800c000 tx=0x7fe9580091d0).ready entity=mon.0 client_cookie=eccbaf9e2dc262b3 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:12.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.427+0000 7fe964bb6700 1 -- 172.21.15.47:0/1283668042 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe95800e6e0 con 0x7fe9480049f0 2022-06-29T15:48:12.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.427+0000 7fe964bb6700 1 -- 172.21.15.47:0/1283668042 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe95800e840 con 0x7fe9480049f0 2022-06-29T15:48:12.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.427+0000 7fe964bb6700 1 -- 172.21.15.47:0/1283668042 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe95800eba0 con 0x7fe9480049f0 2022-06-29T15:48:12.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.431+0000 7fe967e1c700 1 -- 172.21.15.47:0/1283668042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9480049f0 msgr2=0x7fe948004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:12.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.431+0000 7fe967e1c700 1 --2- 172.21.15.47:0/1283668042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9480049f0 0x7fe948004e70 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7fe95800c000 tx=0x7fe9580091d0).stop 2022-06-29T15:48:12.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.431+0000 7fe9663b9700 1 -- 172.21.15.47:0/1283668042 reap_dead start 2022-06-29T15:48:12.446 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.431+0000 7fe967e1c700 1 -- 172.21.15.47:0/1283668042 shutdown_connections 2022-06-29T15:48:12.446 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.431+0000 7fe967e1c700 1 -- 172.21.15.47:0/1283668042 >> 172.21.15.47:0/1283668042 conn(0x7fe96005f0b0 msgr2=0x7fe96005f490 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:12.446 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.431+0000 7fe967e1c700 1 -- 172.21.15.47:0/1283668042 shutdown_connections 2022-06-29T15:48:12.446 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.431+0000 7fe967e1c700 1 -- 172.21.15.47:0/1283668042 wait complete. 2022-06-29T15:48:12.447 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.431+0000 7fe967e1c700 1 Processor -- start 2022-06-29T15:48:12.447 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.431+0000 7fe967e1c700 1 -- start start 2022-06-29T15:48:12.447 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.431+0000 7fe967e1c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe9480049f0 0x7fe9601463f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:12.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.431+0000 7fe967e1c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe96009b6e0 0x7fe96013fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:12.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.431+0000 7fe967e1c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe96013ffb0 0x7fe9600ad450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:12.449 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.431+0000 7fe967e1c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe960152250 con 0x7fe96009b6e0 2022-06-29T15:48:12.449 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.431+0000 7fe967e1c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9600af400 con 0x7fe96013ffb0 2022-06-29T15:48:12.449 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.431+0000 7fe967e1c700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9600af630 con 0x7fe9480049f0 2022-06-29T15:48:12.450 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.435+0000 7fe9653b7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe96009b6e0 0x7fe96013fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:12.450 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.435+0000 7fe9653b7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe96009b6e0 0x7fe96013fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49738/0 (socket says 172.21.15.47:49738) 2022-06-29T15:48:12.451 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.435+0000 7fe9653b7700 1 -- 172.21.15.47:0/3626779554 learned_addr learned my addr 172.21.15.47:0/3626779554 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:12.451 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.435+0000 7fe965bb8700 1 --2- 172.21.15.47:0/3626779554 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe9480049f0 0x7fe9601463f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:12.452 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.435+0000 7fe9663b9700 1 --2- 172.21.15.47:0/3626779554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe96013ffb0 0x7fe9600ad450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:12.452 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.435+0000 7fe9653b7700 1 -- 172.21.15.47:0/3626779554 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe9480049f0 msgr2=0x7fe9601463f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:12.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.435+0000 7fe9653b7700 1 --2- 172.21.15.47:0/3626779554 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe9480049f0 0x7fe9601463f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:12.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.435+0000 7fe9653b7700 1 -- 172.21.15.47:0/3626779554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe96013ffb0 msgr2=0x7fe9600ad450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:12.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.435+0000 7fe9653b7700 1 --2- 172.21.15.47:0/3626779554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe96013ffb0 0x7fe9600ad450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:12.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.435+0000 7fe9653b7700 1 -- 172.21.15.47:0/3626779554 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9600af8d0 con 0x7fe96009b6e0 2022-06-29T15:48:12.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.435+0000 7fe9653b7700 1 --2- 172.21.15.47:0/3626779554 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe96009b6e0 0x7fe96013fac0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7fe95c00b000 tx=0x7fe95c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:12.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.435+0000 7fe94effd700 1 -- 172.21.15.47:0/3626779554 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe95c00f760 con 0x7fe96009b6e0 2022-06-29T15:48:12.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.435+0000 7fe94effd700 1 -- 172.21.15.47:0/3626779554 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe95c00f8c0 con 0x7fe96009b6e0 2022-06-29T15:48:12.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.435+0000 7fe967e1c700 1 -- 172.21.15.47:0/3626779554 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9600afb70 con 0x7fe96009b6e0 2022-06-29T15:48:12.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.435+0000 7fe94effd700 1 -- 172.21.15.47:0/3626779554 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe95c00fc60 con 0x7fe96009b6e0 2022-06-29T15:48:12.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.435+0000 7fe967e1c700 1 -- 172.21.15.47:0/3626779554 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe96009bb40 con 0x7fe96009b6e0 2022-06-29T15:48:12.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.435+0000 7fe94effd700 1 -- 172.21.15.47:0/3626779554 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe95c014440 con 0x7fe96009b6e0 2022-06-29T15:48:12.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.435+0000 7fe967e1c700 1 -- 172.21.15.47:0/3626779554 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9600573d0 con 0x7fe96009b6e0 2022-06-29T15:48:12.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.435+0000 7fe94effd700 1 --2- 172.21.15.47:0/3626779554 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe940053b00 0x7fe940055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:12.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.439+0000 7fe94effd700 1 -- 172.21.15.47:0/3626779554 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fe95c06c800 con 0x7fe96009b6e0 2022-06-29T15:48:12.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.439+0000 7fe965bb8700 1 --2- 172.21.15.47:0/3626779554 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe940053b00 0x7fe940055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:12.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.439+0000 7fe965bb8700 1 --2- 172.21.15.47:0/3626779554 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe940053b00 0x7fe940055fc0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7fe950005000 tx=0x7fe960095db0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:12.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.443+0000 7fe94effd700 1 -- 172.21.15.47:0/3626779554 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe95c043770 con 0x7fe96009b6e0 2022-06-29T15:48:12.569 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.567+0000 7fe967e1c700 1 -- 172.21.15.47:0/3626779554 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fe960044710 con 0x7fe96009b6e0 2022-06-29T15:48:12.575 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.575+0000 7fe94effd700 1 -- 172.21.15.47:0/3626779554 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe95c018070 con 0x7fe96009b6e0 2022-06-29T15:48:12.577 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.575+0000 7fe967e1c700 1 -- 172.21.15.47:0/3626779554 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe940053b00 msgr2=0x7fe940055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:12.577 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.575+0000 7fe967e1c700 1 --2- 172.21.15.47:0/3626779554 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe940053b00 0x7fe940055fc0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7fe950005000 tx=0x7fe960095db0).stop 2022-06-29T15:48:12.577 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.575+0000 7fe967e1c700 1 -- 172.21.15.47:0/3626779554 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe96009b6e0 msgr2=0x7fe96013fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:12.578 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.575+0000 7fe967e1c700 1 --2- 172.21.15.47:0/3626779554 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe96009b6e0 0x7fe96013fac0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7fe95c00b000 tx=0x7fe95c006df0).stop 2022-06-29T15:48:12.578 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.575+0000 7fe967e1c700 1 -- 172.21.15.47:0/3626779554 shutdown_connections 2022-06-29T15:48:12.578 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.575+0000 7fe967e1c700 1 --2- 172.21.15.47:0/3626779554 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe940053b00 0x7fe940055fc0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:12.578 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.575+0000 7fe967e1c700 1 --2- 172.21.15.47:0/3626779554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe96013ffb0 0x7fe9600ad450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:12.579 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.575+0000 7fe967e1c700 1 --2- 172.21.15.47:0/3626779554 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe96009b6e0 0x7fe96013fac0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:12.579 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.575+0000 7fe967e1c700 1 --2- 172.21.15.47:0/3626779554 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe9480049f0 0x7fe9601463f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:12.579 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.575+0000 7fe967e1c700 1 -- 172.21.15.47:0/3626779554 >> 172.21.15.47:0/3626779554 conn(0x7fe96005f0b0 msgr2=0x7fe960141ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:12.579 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.575+0000 7fe967e1c700 1 -- 172.21.15.47:0/3626779554 shutdown_connections 2022-06-29T15:48:12.579 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.575+0000 7fe967e1c700 1 -- 172.21.15.47:0/3626779554 wait complete. 2022-06-29T15:48:12.591 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-06-29T15:48:12.592 DEBUG:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:48:12.607 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:48:12.628 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:48:12.649 DEBUG:teuthology.orchestra.run.smithi164:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:48:12.670 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-06-29T15:48:12.670 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:48:12.671 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:48:12.671 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:48:12.698 INFO:teuthology.orchestra.run.smithi099.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-06-29T15:48:12.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T15:48:12.700 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-06-29T15:48:12.700 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:48:12.701 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:48:12.701 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:48:12.718 INFO:teuthology.orchestra.run.smithi164.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-06-29T15:48:12.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T15:48:12.720 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-06-29T15:48:12.721 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-06-29T15:48:12.835 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.835+0000 7f73f2ff9700 1 Processor -- start 2022-06-29T15:48:12.836 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.835+0000 7f73f2ff9700 1 -- start start 2022-06-29T15:48:12.839 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.835+0000 7f73f2ff9700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f73ec0630f0 0x7f73ec0b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:12.839 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.835+0000 7f73f2ff9700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f73ec05f0b0 0x7f73ec0b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:12.840 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.835+0000 7f73f2ff9700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73ec062180 con 0x7f73ec061da0 2022-06-29T15:48:12.840 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.835+0000 7f73f2ff9700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73ec05fae0 con 0x7f73ec0630f0 2022-06-29T15:48:12.840 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.835+0000 7f73f2ff9700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73ec0b2ea0 con 0x7f73ec05f0b0 2022-06-29T15:48:12.840 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.835+0000 7f73f1ff7700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f73ec061da0 0x7f73ec0b3880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35298/0 (socket says 172.21.15.47:35298) 2022-06-29T15:48:12.840 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.835+0000 7f73f1ff7700 1 -- 172.21.15.47:0/3306312558 learned_addr learned my addr 172.21.15.47:0/3306312558 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:48:12.841 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.835+0000 7f73f17f6700 1 --2- 172.21.15.47:0/3306312558 >> v2:172.21.15.47:3300/0 conn(0x7f73ec0630f0 0x7f73ec0b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:12.841 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.835+0000 7f73f27f8700 1 --2- 172.21.15.47:0/3306312558 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f73ec05f0b0 0x7f73ec0b17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:12.841 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.835+0000 7f73f0ff5700 1 -- 172.21.15.47:0/3306312558 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2214333662 0 0) 0x7f73ec062180 con 0x7f73ec061da0 2022-06-29T15:48:12.841 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.835+0000 7f73f0ff5700 1 -- 172.21.15.47:0/3306312558 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73d8003260 con 0x7f73ec061da0 2022-06-29T15:48:12.842 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.835+0000 7f73f17f6700 1 -- 172.21.15.47:0/3306312558 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f73ec05f0b0 msgr2=0x7f73ec0b17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:12.842 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.835+0000 7f73f17f6700 1 --2- 172.21.15.47:0/3306312558 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f73ec05f0b0 0x7f73ec0b17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:12.842 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.835+0000 7f73f17f6700 1 -- 172.21.15.47:0/3306312558 >> v1:172.21.15.47:6789/0 conn(0x7f73ec061da0 legacy=0x7f73ec0b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:12.842 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.835+0000 7f73f17f6700 1 -- 172.21.15.47:0/3306312558 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73ec0b3140 con 0x7f73ec0630f0 2022-06-29T15:48:12.843 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.835+0000 7f73f27f8700 1 --2- 172.21.15.47:0/3306312558 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f73ec05f0b0 0x7f73ec0b17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:48:12.843 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.835+0000 7f73f17f6700 1 --2- 172.21.15.47:0/3306312558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73ec0630f0 0x7f73ec0b0e50 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7f73e0002f70 tx=0x7f73e0006df0).ready entity=mon.0 client_cookie=fe8c5fb46990deea server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:12.843 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.835+0000 7f73f0ff5700 1 -- 172.21.15.47:0/3306312558 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f73e000d570 con 0x7f73ec0630f0 2022-06-29T15:48:12.843 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.835+0000 7f73f0ff5700 1 -- 172.21.15.47:0/3306312558 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73e0000e90 con 0x7f73ec0630f0 2022-06-29T15:48:12.844 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.835+0000 7f73f0ff5700 1 -- 172.21.15.47:0/3306312558 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f73e000d8c0 con 0x7f73ec0630f0 2022-06-29T15:48:12.844 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.835+0000 7f73f2ff9700 1 -- 172.21.15.47:0/3306312558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73ec0630f0 msgr2=0x7f73ec0b0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:12.844 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.835+0000 7f73f2ff9700 1 --2- 172.21.15.47:0/3306312558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73ec0630f0 0x7f73ec0b0e50 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7f73e0002f70 tx=0x7f73e0006df0).stop 2022-06-29T15:48:12.844 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.835+0000 7f73f2ff9700 1 -- 172.21.15.47:0/3306312558 shutdown_connections 2022-06-29T15:48:12.845 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.835+0000 7f73f2ff9700 1 --2- 172.21.15.47:0/3306312558 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f73ec05f0b0 0x7f73ec0b17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:12.845 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.835+0000 7f73f2ff9700 1 --2- 172.21.15.47:0/3306312558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73ec0630f0 0x7f73ec0b0e50 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:12.845 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.835+0000 7f73f2ff9700 1 -- 172.21.15.47:0/3306312558 >> 172.21.15.47:0/3306312558 conn(0x7f73ec063670 msgr2=0x7f73ec063a80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:12.845 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.839+0000 7f73f2ff9700 1 -- 172.21.15.47:0/3306312558 shutdown_connections 2022-06-29T15:48:12.846 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.839+0000 7f73f2ff9700 1 -- 172.21.15.47:0/3306312558 wait complete. 2022-06-29T15:48:12.849 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.839+0000 7f73f2ff9700 1 Processor -- start 2022-06-29T15:48:12.850 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.839+0000 7f73f2ff9700 1 -- start start 2022-06-29T15:48:12.850 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.839+0000 7f73f2ff9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73ec05f0b0 0x7f73ec14e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:12.850 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.839+0000 7f73f2ff9700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f73ec061da0 0x7f73ec09ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:12.850 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.839+0000 7f73f2ff9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73ec0630f0 0x7f73ec09f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:12.851 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.839+0000 7f73f2ff9700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73ec09ea30 con 0x7f73ec0630f0 2022-06-29T15:48:12.851 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.839+0000 7f73f2ff9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73ec14bba0 con 0x7f73ec05f0b0 2022-06-29T15:48:12.857 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.839+0000 7f73f2ff9700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73ec0abf00 con 0x7f73ec061da0 2022-06-29T15:48:12.857 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.839+0000 7f73f27f8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73ec0630f0 0x7f73ec09f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:12.857 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.839+0000 7f73f27f8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73ec0630f0 0x7f73ec09f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49754/0 (socket says 172.21.15.47:49754) 2022-06-29T15:48:12.857 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.839+0000 7f73f27f8700 1 -- 172.21.15.47:0/1235639177 learned_addr learned my addr 172.21.15.47:0/1235639177 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:12.857 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.839+0000 7f73f17f6700 1 --2- 172.21.15.47:0/1235639177 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f73ec061da0 0x7f73ec09ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:12.858 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.839+0000 7f73f1ff7700 1 --2- 172.21.15.47:0/1235639177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73ec05f0b0 0x7f73ec14e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:12.860 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.839+0000 7f73f17f6700 1 -- 172.21.15.47:0/1235639177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73ec05f0b0 msgr2=0x7f73ec14e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:12.860 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.839+0000 7f73f17f6700 1 --2- 172.21.15.47:0/1235639177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73ec05f0b0 0x7f73ec14e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:12.861 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.839+0000 7f73f17f6700 1 -- 172.21.15.47:0/1235639177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73ec0630f0 msgr2=0x7f73ec09f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:12.861 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.839+0000 7f73f17f6700 1 --2- 172.21.15.47:0/1235639177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73ec0630f0 0x7f73ec09f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:12.861 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.839+0000 7f73f17f6700 1 -- 172.21.15.47:0/1235639177 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e000b180 con 0x7f73ec061da0 2022-06-29T15:48:12.861 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.839+0000 7f73f27f8700 1 --2- 172.21.15.47:0/1235639177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73ec0630f0 0x7f73ec09f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:48:12.862 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.839+0000 7f73f17f6700 1 --2- 172.21.15.47:0/1235639177 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f73ec061da0 0x7f73ec09ee20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f73e0002f70 tx=0x7f73e0014000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:12.862 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.843+0000 7f73eaffd700 1 -- 172.21.15.47:0/1235639177 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f73e000dea0 con 0x7f73ec061da0 2022-06-29T15:48:12.862 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.843+0000 7f73eaffd700 1 -- 172.21.15.47:0/1235639177 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73e0013de0 con 0x7f73ec061da0 2022-06-29T15:48:12.862 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.843+0000 7f73f2ff9700 1 -- 172.21.15.47:0/1235639177 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73ec09f800 con 0x7f73ec061da0 2022-06-29T15:48:12.862 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.843+0000 7f73eaffd700 1 -- 172.21.15.47:0/1235639177 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f73e001e5f0 con 0x7f73ec061da0 2022-06-29T15:48:12.863 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.843+0000 7f73f2ff9700 1 -- 172.21.15.47:0/1235639177 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73ec0ac6b0 con 0x7f73ec061da0 2022-06-29T15:48:12.863 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.843+0000 7f73f2ff9700 1 -- 172.21.15.47:0/1235639177 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73ec0573d0 con 0x7f73ec061da0 2022-06-29T15:48:12.863 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.843+0000 7f73eaffd700 1 -- 172.21.15.47:0/1235639177 <== mon.2 v2:172.21.15.64:3301/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f73e001c070 con 0x7f73ec061da0 2022-06-29T15:48:12.863 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.847+0000 7f73eaffd700 1 --2- 172.21.15.47:0/1235639177 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f73cc0539e0 0x7f73cc055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:12.864 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.847+0000 7f73eaffd700 1 -- 172.21.15.47:0/1235639177 <== mon.2 v2:172.21.15.64:3301/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f73e006c740 con 0x7f73ec061da0 2022-06-29T15:48:12.864 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.847+0000 7f73f1ff7700 1 --2- 172.21.15.47:0/1235639177 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f73cc0539e0 0x7f73cc055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:12.864 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.847+0000 7f73f1ff7700 1 --2- 172.21.15.47:0/1235639177 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f73cc0539e0 0x7f73cc055ea0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7f73dc00d000 tx=0x7f73dc009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:12.865 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.855+0000 7f73eaffd700 1 -- 172.21.15.47:0/1235639177 <== mon.2 v2:172.21.15.64:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f73e00437e0 con 0x7f73ec061da0 2022-06-29T15:48:12.983 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.983+0000 7f73f2ff9700 1 -- 172.21.15.47:0/1235639177 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f73ec0675d0 con 0x7f73ec061da0 2022-06-29T15:48:12.986 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.983+0000 7f73eaffd700 1 -- 172.21.15.47:0/1235639177 <== mon.2 v2:172.21.15.64:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v71) v1 ==== 81+0+29220 (secure 0 0 0) 0x7f73e0023020 con 0x7f73ec061da0 2022-06-29T15:48:12.987 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:48:12.987 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2022-06-29T15:48:12.987 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 71, 2022-06-29T15:48:12.988 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "9a68894a-53ce-400e-838b-14eab91b834c", 2022-06-29T15:48:12.988 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2022-06-29T15:42:14.046361+0000", 2022-06-29T15:48:12.988 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2022-06-29T15:48:10.262581+0000", 2022-06-29T15:48:12.988 INFO:teuthology.orchestra.run.smithi047.stdout: "last_up_change": "2022-06-29T15:42:20.929789+0000", 2022-06-29T15:48:12.988 INFO:teuthology.orchestra.run.smithi047.stdout: "last_in_change": "2022-06-29T15:42:20.929789+0000", 2022-06-29T15:48:12.989 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-06-29T15:48:12.989 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_num": 5799936, 2022-06-29T15:48:12.989 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_set": [ 2022-06-29T15:48:12.989 INFO:teuthology.orchestra.run.smithi047.stdout: "pglog_hardlimit", 2022-06-29T15:48:12.990 INFO:teuthology.orchestra.run.smithi047.stdout: "purged_snapdirs", 2022-06-29T15:48:12.990 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_deletes", 2022-06-29T15:48:12.990 INFO:teuthology.orchestra.run.smithi047.stdout: "sortbitwise" 2022-06-29T15:48:12.990 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:48:12.991 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_version": 5, 2022-06-29T15:48:12.991 INFO:teuthology.orchestra.run.smithi047.stdout: "full_ratio": 0.94999998807907104, 2022-06-29T15:48:12.991 INFO:teuthology.orchestra.run.smithi047.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-06-29T15:48:12.991 INFO:teuthology.orchestra.run.smithi047.stdout: "nearfull_ratio": 0.85000002384185791, 2022-06-29T15:48:12.991 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_snapshot": "", 2022-06-29T15:48:12.992 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_max": 12, 2022-06-29T15:48:12.993 INFO:teuthology.orchestra.run.smithi047.stdout: "max_osd": 8, 2022-06-29T15:48:12.993 INFO:teuthology.orchestra.run.smithi047.stdout: "require_min_compat_client": "jewel", 2022-06-29T15:48:12.993 INFO:teuthology.orchestra.run.smithi047.stdout: "min_compat_client": "jewel", 2022-06-29T15:48:12.994 INFO:teuthology.orchestra.run.smithi047.stdout: "require_osd_release": "octopus", 2022-06-29T15:48:12.994 INFO:teuthology.orchestra.run.smithi047.stdout: "pools": [ 2022-06-29T15:48:12.995 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:12.995 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 1, 2022-06-29T15:48:12.996 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "device_health_metrics", 2022-06-29T15:48:12.996 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:42:16.971431+0000", 2022-06-29T15:48:12.997 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:48:12.997 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:48:12.998 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:48:12.998 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:48:12.998 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:48:12.999 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:48:12.999 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:48:13.000 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:48:13.000 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 1, 2022-06-29T15:48:13.001 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 1, 2022-06-29T15:48:13.001 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 1, 2022-06-29T15:48:13.002 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 1, 2022-06-29T15:48:13.002 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 1, 2022-06-29T15:48:13.003 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:48:13.003 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:48:13.004 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:48:13.004 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:48:13.004 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:48:13.005 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:48:13.005 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:48:13.006 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.006 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "16", 2022-06-29T15:48:13.007 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:48:13.007 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:48:13.008 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:48:13.008 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:48:13.009 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:48:13.009 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:48:13.010 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:48:13.010 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:48:13.010 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:48:13.011 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:48:13.011 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:48:13.012 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:48:13.012 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:48:13.013 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:48:13.013 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:48:13.014 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:48:13.014 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:48:13.015 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:48:13.015 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:48:13.016 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:48:13.016 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:48:13.017 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:48:13.017 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:48:13.018 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:48:13.018 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:48:13.019 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:48:13.019 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.020 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:48:13.020 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:48:13.021 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:48:13.021 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:48:13.021 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:48:13.022 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:48:13.022 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:48:13.023 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:48:13.023 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:48:13.023 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:48:13.024 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:48:13.024 INFO:teuthology.orchestra.run.smithi047.stdout: "options": { 2022-06-29T15:48:13.024 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_min": 1 2022-06-29T15:48:13.024 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.025 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:48:13.025 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_devicehealth": {} 2022-06-29T15:48:13.026 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:13.026 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.026 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.027 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 2, 2022-06-29T15:48:13.027 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "rbd", 2022-06-29T15:48:13.027 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:42:25.290884+0000", 2022-06-29T15:48:13.027 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:48:13.028 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:48:13.028 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:48:13.028 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:48:13.028 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:48:13.029 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:48:13.029 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:48:13.029 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:48:13.029 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2022-06-29T15:48:13.030 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2022-06-29T15:48:13.030 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2022-06-29T15:48:13.030 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2022-06-29T15:48:13.031 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2022-06-29T15:48:13.031 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:48:13.031 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:48:13.032 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:48:13.032 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:48:13.033 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:48:13.033 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:48:13.033 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:48:13.034 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.034 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "19", 2022-06-29T15:48:13.034 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:48:13.035 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:48:13.035 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:48:13.035 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:48:13.035 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:48:13.036 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:48:13.036 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:48:13.036 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:48:13.036 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:48:13.036 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:48:13.037 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:48:13.037 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:48:13.037 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:48:13.037 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:48:13.038 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:48:13.038 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:48:13.039 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:48:13.039 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:48:13.039 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:48:13.039 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:48:13.040 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:48:13.040 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:48:13.040 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:48:13.041 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:48:13.041 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:48:13.041 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:48:13.041 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.042 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:48:13.042 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:48:13.042 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:48:13.043 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:48:13.043 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:48:13.043 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:48:13.043 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:48:13.043 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:48:13.044 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:48:13.044 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:48:13.044 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:48:13.044 INFO:teuthology.orchestra.run.smithi047.stdout: "options": {}, 2022-06-29T15:48:13.045 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:48:13.045 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd": {} 2022-06-29T15:48:13.045 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:13.045 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:13.046 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:48:13.046 INFO:teuthology.orchestra.run.smithi047.stdout: "osds": [ 2022-06-29T15:48:13.046 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.046 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2022-06-29T15:48:13.046 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "e23eb023-4706-4201-bf72-4bf19c912c5d", 2022-06-29T15:48:13.047 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:48:13.047 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:48:13.047 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:48:13.047 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:48:13.048 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:48:13.048 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:48:13.048 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:48:13.048 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 66, 2022-06-29T15:48:13.048 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:48:13.049 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:48:13.049 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:48:13.049 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:13.050 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.050 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:13.050 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6818", 2022-06-29T15:48:13.050 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:48:13.051 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.051 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.051 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:13.051 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6819", 2022-06-29T15:48:13.052 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:48:13.052 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:13.052 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:13.052 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.052 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:48:13.053 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:13.053 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.053 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:13.053 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6820", 2022-06-29T15:48:13.054 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:48:13.054 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.054 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.054 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:13.055 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6821", 2022-06-29T15:48:13.055 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:48:13.055 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:13.055 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:13.055 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.056 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:48:13.056 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:13.056 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.056 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:13.056 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6824", 2022-06-29T15:48:13.057 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:48:13.057 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.057 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.057 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:13.058 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6825", 2022-06-29T15:48:13.058 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:48:13.058 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:13.058 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:13.058 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.059 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:48:13.059 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:13.059 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.059 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:13.059 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6822", 2022-06-29T15:48:13.060 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:48:13.060 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.060 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.060 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:13.061 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6823", 2022-06-29T15:48:13.061 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:48:13.061 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:13.061 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:13.061 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.062 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6819/13096", 2022-06-29T15:48:13.062 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6821/13096", 2022-06-29T15:48:13.062 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6825/13096", 2022-06-29T15:48:13.062 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6823/13096", 2022-06-29T15:48:13.062 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:48:13.063 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:48:13.063 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:48:13.063 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:13.063 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.064 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.064 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2022-06-29T15:48:13.064 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "228bb08d-2d64-43d9-9560-bfaeff154452", 2022-06-29T15:48:13.064 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:48:13.064 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:48:13.065 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:48:13.065 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:48:13.065 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:48:13.065 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:48:13.066 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:48:13.066 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 66, 2022-06-29T15:48:13.066 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:48:13.066 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:48:13.066 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:48:13.067 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:13.067 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.067 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:13.067 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6826", 2022-06-29T15:48:13.067 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:48:13.068 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.068 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.068 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:13.068 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6827", 2022-06-29T15:48:13.069 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:48:13.069 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:13.069 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:13.069 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.069 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:48:13.070 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:13.070 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.070 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:13.070 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6828", 2022-06-29T15:48:13.070 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:48:13.071 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.071 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.071 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:13.071 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6829", 2022-06-29T15:48:13.072 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:48:13.072 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:13.072 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:13.072 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.073 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:48:13.073 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:13.073 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.073 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:13.073 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6832", 2022-06-29T15:48:13.074 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:48:13.074 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.074 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.074 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:13.075 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6833", 2022-06-29T15:48:13.075 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:48:13.075 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:13.075 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:13.075 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.076 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:48:13.076 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:13.076 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.076 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:13.077 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6830", 2022-06-29T15:48:13.077 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:48:13.077 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.077 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.078 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:13.078 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6831", 2022-06-29T15:48:13.078 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:48:13.078 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:13.078 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:13.079 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.079 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6827/13100", 2022-06-29T15:48:13.079 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6829/13100", 2022-06-29T15:48:13.079 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6833/13100", 2022-06-29T15:48:13.080 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6831/13100", 2022-06-29T15:48:13.080 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:48:13.080 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:48:13.080 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:48:13.080 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:13.081 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.081 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.081 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2022-06-29T15:48:13.081 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "797a5452-1510-4640-a111-c9c2fcf707eb", 2022-06-29T15:48:13.081 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:48:13.082 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:48:13.082 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:48:13.082 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:48:13.082 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:48:13.083 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:48:13.083 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:48:13.083 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 65, 2022-06-29T15:48:13.083 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:48:13.083 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:48:13.084 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:48:13.084 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:13.084 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.084 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:13.084 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6810", 2022-06-29T15:48:13.085 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:48:13.085 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.085 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.085 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:13.086 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6811", 2022-06-29T15:48:13.086 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:48:13.086 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:13.086 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:13.086 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.087 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:48:13.087 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:13.087 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.087 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:13.088 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6812", 2022-06-29T15:48:13.088 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:48:13.088 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.088 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.089 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:13.089 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6813", 2022-06-29T15:48:13.089 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:48:13.089 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:13.089 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:13.090 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.090 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:48:13.090 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:13.090 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.091 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:13.091 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6816", 2022-06-29T15:48:13.091 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:48:13.091 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.091 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.092 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:13.092 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6817", 2022-06-29T15:48:13.092 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:48:13.092 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:13.092 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:13.093 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.093 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:48:13.093 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:13.093 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.094 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:13.094 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6814", 2022-06-29T15:48:13.094 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:48:13.094 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.094 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.095 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:13.095 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6815", 2022-06-29T15:48:13.095 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:48:13.095 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:13.096 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:13.096 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.096 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6811/13098", 2022-06-29T15:48:13.096 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6813/13098", 2022-06-29T15:48:13.096 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6817/13098", 2022-06-29T15:48:13.097 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6815/13098", 2022-06-29T15:48:13.097 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:48:13.097 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:48:13.097 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:48:13.098 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:13.098 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.098 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.098 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2022-06-29T15:48:13.098 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "55b24fd7-bb71-4a01-b3fa-a09caa772a62", 2022-06-29T15:48:13.099 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:48:13.099 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:48:13.099 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:48:13.099 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:48:13.099 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:48:13.100 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:48:13.100 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:48:13.100 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 54, 2022-06-29T15:48:13.100 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:48:13.101 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:48:13.101 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:48:13.101 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:13.101 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.101 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:13.102 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6802", 2022-06-29T15:48:13.102 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:48:13.102 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.102 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.103 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:13.103 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6803", 2022-06-29T15:48:13.103 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:48:13.103 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:13.103 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:13.104 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.104 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:48:13.104 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:13.104 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.105 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:13.105 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6804", 2022-06-29T15:48:13.105 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:48:13.105 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.105 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.106 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:13.106 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6805", 2022-06-29T15:48:13.106 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:48:13.106 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:13.107 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:13.107 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.107 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:48:13.107 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:13.108 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.108 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:13.108 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6808", 2022-06-29T15:48:13.108 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:48:13.109 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.109 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.109 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:13.109 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6809", 2022-06-29T15:48:13.109 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:48:13.110 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:13.110 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:13.110 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.110 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:48:13.110 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:13.111 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.111 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:13.111 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6806", 2022-06-29T15:48:13.111 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:48:13.112 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.112 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.112 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:13.112 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6807", 2022-06-29T15:48:13.112 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:48:13.113 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:13.113 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:13.113 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.113 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6803/13097", 2022-06-29T15:48:13.113 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6805/13097", 2022-06-29T15:48:13.114 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6809/13097", 2022-06-29T15:48:13.114 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6807/13097", 2022-06-29T15:48:13.114 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:48:13.114 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:48:13.115 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:48:13.115 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:13.115 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.115 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.115 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2022-06-29T15:48:13.116 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3", 2022-06-29T15:48:13.116 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:48:13.116 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:48:13.116 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:48:13.117 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:48:13.117 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:48:13.117 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:48:13.117 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:48:13.117 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 66, 2022-06-29T15:48:13.118 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:48:13.118 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:48:13.118 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:48:13.118 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:13.119 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.119 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:13.119 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6816", 2022-06-29T15:48:13.119 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:48:13.119 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.120 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.120 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:13.120 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6817", 2022-06-29T15:48:13.120 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:48:13.121 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:13.121 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:13.121 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.121 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:48:13.121 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:13.122 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.122 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:13.122 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6818", 2022-06-29T15:48:13.122 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:48:13.123 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.123 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.123 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:13.123 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6819", 2022-06-29T15:48:13.124 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:48:13.124 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:13.124 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:13.124 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.124 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:48:13.125 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:13.125 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.125 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:13.125 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6822", 2022-06-29T15:48:13.126 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:48:13.126 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.126 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.126 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:13.127 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6823", 2022-06-29T15:48:13.127 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:48:13.127 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:13.127 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:13.127 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.128 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:48:13.128 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:13.128 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.128 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:13.128 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6820", 2022-06-29T15:48:13.129 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:48:13.129 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.129 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.129 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:13.130 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6821", 2022-06-29T15:48:13.130 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:48:13.130 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:13.130 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:13.131 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.131 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6817/13396", 2022-06-29T15:48:13.131 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6819/13396", 2022-06-29T15:48:13.131 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6823/13396", 2022-06-29T15:48:13.131 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6821/13396", 2022-06-29T15:48:13.132 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:48:13.132 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:48:13.132 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:48:13.132 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:13.133 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.133 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.133 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2022-06-29T15:48:13.133 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "f77beebe-6cca-4244-b2d4-677825ebf919", 2022-06-29T15:48:13.134 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:48:13.134 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:48:13.134 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:48:13.134 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:48:13.134 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:48:13.135 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:48:13.135 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:48:13.135 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 65, 2022-06-29T15:48:13.135 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:48:13.136 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:48:13.136 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:48:13.136 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:13.136 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.136 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:13.137 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6800", 2022-06-29T15:48:13.137 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:48:13.137 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.137 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.138 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:13.138 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6801", 2022-06-29T15:48:13.138 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:48:13.138 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:13.138 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:13.139 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.139 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:48:13.139 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:13.139 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.140 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:13.140 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6802", 2022-06-29T15:48:13.140 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:48:13.140 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.140 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.141 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:13.141 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6803", 2022-06-29T15:48:13.141 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:48:13.141 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:13.142 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:13.142 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.142 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:48:13.142 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:13.143 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.143 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:13.143 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6806", 2022-06-29T15:48:13.143 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:48:13.143 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.144 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.144 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:13.144 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6807", 2022-06-29T15:48:13.144 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:48:13.144 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:13.145 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:13.145 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.145 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:48:13.145 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:13.146 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.146 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:13.146 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6804", 2022-06-29T15:48:13.146 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:48:13.147 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.147 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.147 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:13.147 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6805", 2022-06-29T15:48:13.147 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:48:13.148 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:13.148 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:13.148 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.149 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6801/13393", 2022-06-29T15:48:13.149 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6803/13393", 2022-06-29T15:48:13.149 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6807/13393", 2022-06-29T15:48:13.149 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6805/13393", 2022-06-29T15:48:13.150 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:48:13.150 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:48:13.150 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:48:13.150 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:13.151 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.151 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.151 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2022-06-29T15:48:13.151 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "af0ca9b3-c02c-4910-834b-bf0b8dc6713c", 2022-06-29T15:48:13.152 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:48:13.152 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:48:13.152 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:48:13.152 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:48:13.153 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:48:13.153 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:48:13.153 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:48:13.153 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 66, 2022-06-29T15:48:13.154 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:48:13.154 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:48:13.154 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:48:13.154 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:13.155 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.155 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:13.155 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6824", 2022-06-29T15:48:13.155 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:48:13.156 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.156 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.156 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:13.156 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6825", 2022-06-29T15:48:13.157 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:48:13.157 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:13.157 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:13.158 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.158 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:48:13.158 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:13.158 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.159 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:13.159 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6826", 2022-06-29T15:48:13.159 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:48:13.159 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.160 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.160 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:13.160 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6827", 2022-06-29T15:48:13.160 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:48:13.161 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:13.161 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:13.161 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.161 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:48:13.162 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:13.162 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.162 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:13.162 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6830", 2022-06-29T15:48:13.163 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:48:13.163 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.163 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.163 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:13.164 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6831", 2022-06-29T15:48:13.164 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:48:13.164 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:13.164 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:13.165 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.165 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:48:13.165 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:13.165 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.166 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:13.166 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6828", 2022-06-29T15:48:13.166 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:48:13.167 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.167 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.167 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:13.167 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6829", 2022-06-29T15:48:13.167 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:48:13.168 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:13.168 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:13.168 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.168 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6825/13395", 2022-06-29T15:48:13.169 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6827/13395", 2022-06-29T15:48:13.169 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6831/13395", 2022-06-29T15:48:13.169 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6829/13395", 2022-06-29T15:48:13.169 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:48:13.170 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:48:13.170 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:48:13.170 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:13.170 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.171 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.171 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2022-06-29T15:48:13.171 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "f19c3ca6-df88-4bde-a150-b51a53f092e1", 2022-06-29T15:48:13.171 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:48:13.172 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:48:13.172 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:48:13.172 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:48:13.172 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:48:13.172 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:48:13.173 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:48:13.173 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 65, 2022-06-29T15:48:13.173 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:48:13.173 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:48:13.174 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:48:13.174 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:13.174 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.174 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:13.175 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6808", 2022-06-29T15:48:13.175 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:48:13.175 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.175 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.176 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:13.176 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6809", 2022-06-29T15:48:13.176 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:48:13.176 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:13.177 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:13.177 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.177 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:48:13.177 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:13.178 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.178 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:13.178 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6810", 2022-06-29T15:48:13.178 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:48:13.179 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.179 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.179 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:13.179 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6811", 2022-06-29T15:48:13.179 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:48:13.180 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:13.180 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:13.180 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.180 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:48:13.181 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:13.181 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.181 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:13.181 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6814", 2022-06-29T15:48:13.182 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:48:13.182 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.182 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.182 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:13.183 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6815", 2022-06-29T15:48:13.183 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:48:13.183 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:13.183 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:13.184 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.184 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:48:13.184 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:48:13.184 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.185 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:48:13.185 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6812", 2022-06-29T15:48:13.185 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:48:13.185 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.185 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.186 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:48:13.186 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6813", 2022-06-29T15:48:13.186 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:48:13.186 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:13.187 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:13.187 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.187 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6809/13394", 2022-06-29T15:48:13.187 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6811/13394", 2022-06-29T15:48:13.188 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6815/13394", 2022-06-29T15:48:13.188 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6813/13394", 2022-06-29T15:48:13.188 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:48:13.188 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:48:13.189 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:48:13.189 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:13.189 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:13.189 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:48:13.190 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_xinfo": [ 2022-06-29T15:48:13.190 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.190 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2022-06-29T15:48:13.190 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:48:13.190 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:48:13.191 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:48:13.191 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:48:13.191 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:48:13.191 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.720513+0000", 2022-06-29T15:48:13.192 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:48:13.192 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.192 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.192 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2022-06-29T15:48:13.193 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:48:13.193 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:48:13.193 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:48:13.193 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:48:13.193 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:48:13.194 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.772612+0000", 2022-06-29T15:48:13.194 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:48:13.194 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.195 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.195 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2022-06-29T15:48:13.195 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:48:13.195 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:48:13.196 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:48:13.196 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:48:13.196 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:48:13.196 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.450223+0000", 2022-06-29T15:48:13.196 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:48:13.197 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.197 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.197 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2022-06-29T15:48:13.197 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:48:13.198 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:48:13.198 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:48:13.198 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:48:13.198 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:48:13.199 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.477856+0000", 2022-06-29T15:48:13.199 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:48:13.199 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.200 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.200 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2022-06-29T15:48:13.200 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:48:13.200 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:48:13.200 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:48:13.201 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:48:13.201 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:48:13.201 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.539485+0000", 2022-06-29T15:48:13.201 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:48:13.202 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.202 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.202 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2022-06-29T15:48:13.202 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:48:13.203 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:48:13.203 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:48:13.203 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:48:13.203 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:48:13.204 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.487040+0000", 2022-06-29T15:48:13.204 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:48:13.204 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.204 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.204 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2022-06-29T15:48:13.205 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:48:13.205 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:48:13.205 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:48:13.205 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:48:13.206 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:48:13.206 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.748685+0000", 2022-06-29T15:48:13.206 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:48:13.206 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.207 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:13.207 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2022-06-29T15:48:13.207 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:48:13.207 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:48:13.208 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:48:13.208 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:48:13.208 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:48:13.208 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.459516+0000", 2022-06-29T15:48:13.209 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:48:13.209 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:13.209 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:48:13.209 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap": [], 2022-06-29T15:48:13.210 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap_items": [], 2022-06-29T15:48:13.210 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_temp": [], 2022-06-29T15:48:13.210 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_temp": [], 2022-06-29T15:48:13.210 INFO:teuthology.orchestra.run.smithi047.stdout: "blacklist": { 2022-06-29T15:48:13.211 INFO:teuthology.orchestra.run.smithi047.stdout: "172.21.15.47:6837/1166335240": "2022-06-30T15:48:06.792466+0000", 2022-06-29T15:48:13.211 INFO:teuthology.orchestra.run.smithi047.stdout: "172.21.15.47:6836/1166335240": "2022-06-30T15:48:06.792466+0000" 2022-06-29T15:48:13.211 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.211 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profiles": { 2022-06-29T15:48:13.211 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs_data": { 2022-06-29T15:48:13.212 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-device-class": "", 2022-06-29T15:48:13.212 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2022-06-29T15:48:13.212 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-root": "default", 2022-06-29T15:48:13.212 INFO:teuthology.orchestra.run.smithi047.stdout: "jerasure-per-chunk-alignment": "false", 2022-06-29T15:48:13.213 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2022-06-29T15:48:13.213 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "2", 2022-06-29T15:48:13.213 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2022-06-29T15:48:13.213 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2022-06-29T15:48:13.214 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van", 2022-06-29T15:48:13.214 INFO:teuthology.orchestra.run.smithi047.stdout: "w": "8" 2022-06-29T15:48:13.214 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.215 INFO:teuthology.orchestra.run.smithi047.stdout: "default": { 2022-06-29T15:48:13.215 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2022-06-29T15:48:13.215 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2022-06-29T15:48:13.215 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "1", 2022-06-29T15:48:13.216 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2022-06-29T15:48:13.216 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2022-06-29T15:48:13.216 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van" 2022-06-29T15:48:13.216 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:13.216 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:13.217 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps_queue": [], 2022-06-29T15:48:13.217 INFO:teuthology.orchestra.run.smithi047.stdout: "new_removed_snaps": [], 2022-06-29T15:48:13.217 INFO:teuthology.orchestra.run.smithi047.stdout: "new_purged_snaps": [], 2022-06-29T15:48:13.217 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_node_flags": {}, 2022-06-29T15:48:13.218 INFO:teuthology.orchestra.run.smithi047.stdout: "device_class_flags": {} 2022-06-29T15:48:13.218 INFO:teuthology.orchestra.run.smithi047.stdout:} 2022-06-29T15:48:13.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.987+0000 7f73f2ff9700 1 -- 172.21.15.47:0/1235639177 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f73cc0539e0 msgr2=0x7f73cc055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:13.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.987+0000 7f73f2ff9700 1 --2- 172.21.15.47:0/1235639177 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f73cc0539e0 0x7f73cc055ea0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7f73dc00d000 tx=0x7f73dc009040).stop 2022-06-29T15:48:13.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.987+0000 7f73f2ff9700 1 -- 172.21.15.47:0/1235639177 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f73ec061da0 msgr2=0x7f73ec09ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:13.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.987+0000 7f73f2ff9700 1 --2- 172.21.15.47:0/1235639177 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f73ec061da0 0x7f73ec09ee20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f73e0002f70 tx=0x7f73e0014000).stop 2022-06-29T15:48:13.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.987+0000 7f73f2ff9700 1 -- 172.21.15.47:0/1235639177 shutdown_connections 2022-06-29T15:48:13.222 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.987+0000 7f73f2ff9700 1 --2- 172.21.15.47:0/1235639177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73ec0630f0 0x7f73ec09f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:13.222 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.987+0000 7f73f2ff9700 1 --2- 172.21.15.47:0/1235639177 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f73ec061da0 0x7f73ec09ee20 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:13.222 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.987+0000 7f73f2ff9700 1 --2- 172.21.15.47:0/1235639177 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f73cc0539e0 0x7f73cc055ea0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:13.222 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.987+0000 7f73f2ff9700 1 --2- 172.21.15.47:0/1235639177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73ec05f0b0 0x7f73ec14e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:13.223 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.987+0000 7f73f2ff9700 1 -- 172.21.15.47:0/1235639177 >> 172.21.15.47:0/1235639177 conn(0x7f73ec063670 msgr2=0x7f73ec14bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:13.223 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.987+0000 7f73f2ff9700 1 -- 172.21.15.47:0/1235639177 shutdown_connections 2022-06-29T15:48:13.223 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:12.987+0000 7f73f2ff9700 1 -- 172.21.15.47:0/1235639177 wait complete. 2022-06-29T15:48:13.224 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:48:13.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.331+0000 7f51d759e700 1 Processor -- start 2022-06-29T15:48:13.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.331+0000 7f51d759e700 1 -- start start 2022-06-29T15:48:13.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.331+0000 7f51d759e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f51d8143bd0 0x7f51d805c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:13.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.331+0000 7f51d759e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f51d80b2c40 0x7f51d80b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:13.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.331+0000 7f51d759e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51d8142f90 con 0x7f51d805c510 2022-06-29T15:48:13.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.331+0000 7f51d759e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51d81431f0 con 0x7f51d80b2c40 2022-06-29T15:48:13.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.331+0000 7f51d759e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51d8143490 con 0x7f51d8143bd0 2022-06-29T15:48:13.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.335+0000 7f51d6d9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f51d80b2c40 0x7f51d80b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:13.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.335+0000 7f51d5d9b700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f51d8143bd0 0x7f51d805c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:13.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.335+0000 7f51d659c700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f51d805c510 0x7f51d80b4100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:42876/0 (socket says 172.21.15.47:42876) 2022-06-29T15:48:13.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.335+0000 7f51d659c700 1 -- 172.21.15.47:0/2420360995 learned_addr learned my addr 172.21.15.47:0/2420360995 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:48:13.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.335+0000 7f51d5d9b700 1 -- 172.21.15.47:0/2420360995 >> v2:172.21.15.47:3300/0 conn(0x7f51d80b2c40 msgr2=0x7f51d80b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:13.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.335+0000 7f51d5d9b700 1 --2- 172.21.15.47:0/2420360995 >> v2:172.21.15.47:3300/0 conn(0x7f51d80b2c40 0x7f51d80b30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:13.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.335+0000 7f51d5d9b700 1 -- 172.21.15.47:0/2420360995 >> v1:172.21.15.64:6789/0 conn(0x7f51d805c510 legacy=0x7f51d80b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:13.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.335+0000 7f51d5d9b700 1 -- 172.21.15.47:0/2420360995 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d8143730 con 0x7f51d8143bd0 2022-06-29T15:48:13.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.335+0000 7f51d559a700 1 -- 172.21.15.47:0/2420360995 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1913335494 0 0) 0x7f51d8142f90 con 0x7f51d805c510 2022-06-29T15:48:13.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.335+0000 7f51d5d9b700 1 --2- 172.21.15.47:0/2420360995 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f51d8143bd0 0x7f51d805c8f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f51c00032f0 tx=0x7f51c00091d0).ready entity=mon.2 client_cookie=d160f3a322b63d60 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:13.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.335+0000 7f51d559a700 1 -- 172.21.15.47:0/2420360995 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f51c000b6c0 con 0x7f51d8143bd0 2022-06-29T15:48:13.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.335+0000 7f51d559a700 1 -- 172.21.15.47:0/2420360995 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51c000b820 con 0x7f51d8143bd0 2022-06-29T15:48:13.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.335+0000 7f51d559a700 1 -- 172.21.15.47:0/2420360995 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f51c000b990 con 0x7f51d8143bd0 2022-06-29T15:48:13.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.335+0000 7f51d759e700 1 -- 172.21.15.47:0/2420360995 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f51d8143bd0 msgr2=0x7f51d805c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:13.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.335+0000 7f51d759e700 1 --2- 172.21.15.47:0/2420360995 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f51d8143bd0 0x7f51d805c8f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f51c00032f0 tx=0x7f51c00091d0).stop 2022-06-29T15:48:13.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.335+0000 7f51d759e700 1 -- 172.21.15.47:0/2420360995 shutdown_connections 2022-06-29T15:48:13.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.335+0000 7f51d759e700 1 --2- 172.21.15.47:0/2420360995 >> v2:172.21.15.47:3300/0 conn(0x7f51d80b2c40 0x7f51d80b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:13.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.335+0000 7f51d759e700 1 --2- 172.21.15.47:0/2420360995 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f51d8143bd0 0x7f51d805c8f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:13.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.335+0000 7f51d759e700 1 -- 172.21.15.47:0/2420360995 >> 172.21.15.47:0/2420360995 conn(0x7f51d8060e00 msgr2=0x7f51d80611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:13.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.335+0000 7f51d759e700 1 -- 172.21.15.47:0/2420360995 shutdown_connections 2022-06-29T15:48:13.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.335+0000 7f51d759e700 1 -- 172.21.15.47:0/2420360995 wait complete. 2022-06-29T15:48:13.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.339+0000 7f51d759e700 1 Processor -- start 2022-06-29T15:48:13.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.339+0000 7f51d759e700 1 -- start start 2022-06-29T15:48:13.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.339+0000 7f51d759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51d805c510 0x7f51d8152db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:13.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.339+0000 7f51d759e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f51d80b2c40 0x7f51d813e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:13.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.339+0000 7f51d759e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51d8143bd0 0x7f51d813ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:13.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.339+0000 7f51d759e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d813fb60 con 0x7f51d8143bd0 2022-06-29T15:48:13.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.339+0000 7f51d759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d80ad670 con 0x7f51d805c510 2022-06-29T15:48:13.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.339+0000 7f51d759e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51d81473c0 con 0x7f51d80b2c40 2022-06-29T15:48:13.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.339+0000 7f51d6d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51d8143bd0 0x7f51d813ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:13.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.339+0000 7f51d6d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51d8143bd0 0x7f51d813ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49766/0 (socket says 172.21.15.47:49766) 2022-06-29T15:48:13.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.339+0000 7f51d6d9d700 1 -- 172.21.15.47:0/260245764 learned_addr learned my addr 172.21.15.47:0/260245764 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:13.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.339+0000 7f51d5d9b700 1 --2- 172.21.15.47:0/260245764 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f51d80b2c40 0x7f51d813e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:13.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.339+0000 7f51d659c700 1 --2- 172.21.15.47:0/260245764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51d805c510 0x7f51d8152db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:13.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.339+0000 7f51d6d9d700 1 -- 172.21.15.47:0/260245764 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f51d80b2c40 msgr2=0x7f51d813e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:13.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.339+0000 7f51d6d9d700 1 --2- 172.21.15.47:0/260245764 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f51d80b2c40 0x7f51d813e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:13.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.339+0000 7f51d6d9d700 1 -- 172.21.15.47:0/260245764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51d805c510 msgr2=0x7f51d8152db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:13.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.339+0000 7f51d6d9d700 1 --2- 172.21.15.47:0/260245764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51d805c510 0x7f51d8152db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:13.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.339+0000 7f51d6d9d700 1 -- 172.21.15.47:0/260245764 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c0009470 con 0x7f51d8143bd0 2022-06-29T15:48:13.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.339+0000 7f51d659c700 1 --2- 172.21.15.47:0/260245764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51d805c510 0x7f51d8152db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:48:13.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.339+0000 7f51d6d9d700 1 --2- 172.21.15.47:0/260245764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51d8143bd0 0x7f51d813ea00 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 rx=0x7f51d8145d90 tx=0x7f51d0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:13.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.339+0000 7f51cf7fe700 1 -- 172.21.15.47:0/260245764 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f51d0010780 con 0x7f51d8143bd0 2022-06-29T15:48:13.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.339+0000 7f51cf7fe700 1 -- 172.21.15.47:0/260245764 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d000b3b0 con 0x7f51d8143bd0 2022-06-29T15:48:13.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.339+0000 7f51d759e700 1 -- 172.21.15.47:0/260245764 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51d8146bc0 con 0x7f51d8143bd0 2022-06-29T15:48:13.352 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.339+0000 7f51cf7fe700 1 -- 172.21.15.47:0/260245764 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f51d0010ad0 con 0x7f51d8143bd0 2022-06-29T15:48:13.352 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.339+0000 7f51d759e700 1 -- 172.21.15.47:0/260245764 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51d81404f0 con 0x7f51d8143bd0 2022-06-29T15:48:13.352 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.343+0000 7f51d759e700 1 -- 172.21.15.47:0/260245764 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51d80573d0 con 0x7f51d8143bd0 2022-06-29T15:48:13.352 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.343+0000 7f51cf7fe700 1 -- 172.21.15.47:0/260245764 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f51d0010c30 con 0x7f51d8143bd0 2022-06-29T15:48:13.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.343+0000 7f51cf7fe700 1 --2- 172.21.15.47:0/260245764 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f51bc053aa0 0x7f51bc055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:13.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.343+0000 7f51cf7fe700 1 -- 172.21.15.47:0/260245764 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f51d006c110 con 0x7f51d8143bd0 2022-06-29T15:48:13.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.343+0000 7f51d659c700 1 --2- 172.21.15.47:0/260245764 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f51bc053aa0 0x7f51bc055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:13.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.343+0000 7f51d659c700 1 --2- 172.21.15.47:0/260245764 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f51bc053aa0 0x7f51bc055f60 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7f51c8008000 tx=0x7f51c8009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:13.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.351+0000 7f51cf7fe700 1 -- 172.21.15.47:0/260245764 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f51d0043110 con 0x7f51d8143bd0 2022-06-29T15:48:13.522 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.519+0000 7f51d759e700 1 -- 172.21.15.47:0/260245764 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f51d8069a60 con 0x7f51d8143bd0 2022-06-29T15:48:13.523 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.523+0000 7f51cf7fe700 1 -- 172.21.15.47:0/260245764 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+1556 (secure 0 0 0) 0x7f51d0046730 con 0x7f51d8143bd0 2022-06-29T15:48:13.523 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:48:13.524 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":32,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5887,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/929350416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":929350416},{"type":"v1","addr":"172.21.15.47:6835","nonce":929350416}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":27},{"gid":5970,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2425218664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2425218664},{"type":"v1","addr":"172.21.15.64:6833","nonce":2425218664}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":27},{"gid":6201,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/236814902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":236814902},{"type":"v1","addr":"172.21.15.47:6837","nonce":236814902}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":32}],"filesystems":[]} 2022-06-29T15:48:13.528 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.527+0000 7f51d759e700 1 -- 172.21.15.47:0/260245764 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f51bc053aa0 msgr2=0x7f51bc055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:13.528 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.527+0000 7f51d759e700 1 --2- 172.21.15.47:0/260245764 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f51bc053aa0 0x7f51bc055f60 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7f51c8008000 tx=0x7f51c8009040).stop 2022-06-29T15:48:13.528 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.527+0000 7f51d759e700 1 -- 172.21.15.47:0/260245764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51d8143bd0 msgr2=0x7f51d813ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:13.528 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.527+0000 7f51d759e700 1 --2- 172.21.15.47:0/260245764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51d8143bd0 0x7f51d813ea00 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 rx=0x7f51d8145d90 tx=0x7f51d0006df0).stop 2022-06-29T15:48:13.529 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.527+0000 7f51d759e700 1 -- 172.21.15.47:0/260245764 shutdown_connections 2022-06-29T15:48:13.529 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.527+0000 7f51d759e700 1 --2- 172.21.15.47:0/260245764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51d8143bd0 0x7f51d813ea00 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:13.529 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.527+0000 7f51d759e700 1 --2- 172.21.15.47:0/260245764 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f51d80b2c40 0x7f51d813e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:13.529 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.527+0000 7f51d759e700 1 --2- 172.21.15.47:0/260245764 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f51bc053aa0 0x7f51bc055f60 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:13.530 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.527+0000 7f51d759e700 1 --2- 172.21.15.47:0/260245764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51d805c510 0x7f51d8152db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:13.530 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.527+0000 7f51d759e700 1 -- 172.21.15.47:0/260245764 >> 172.21.15.47:0/260245764 conn(0x7f51d8060e00 msgr2=0x7f51d81427c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:13.530 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.527+0000 7f51d759e700 1 -- 172.21.15.47:0/260245764 shutdown_connections 2022-06-29T15:48:13.530 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.527+0000 7f51d759e700 1 -- 172.21.15.47:0/260245764 wait complete. 2022-06-29T15:48:13.531 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 32 2022-06-29T15:48:13.543 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:48:13.651 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.651+0000 7fe7a9c5f700 1 Processor -- start 2022-06-29T15:48:13.652 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.651+0000 7fe7a9c5f700 1 -- start start 2022-06-29T15:48:13.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.651+0000 7fe7a9c5f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe7a4060eb0 0x7fe7a40b1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:13.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.651+0000 7fe7a9c5f700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7a40a9120 con 0x7fe7a4145090 2022-06-29T15:48:13.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.651+0000 7fe7a9c5f700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7a40b0b80 con 0x7fe7a40b5c60 2022-06-29T15:48:13.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.651+0000 7fe7a9c5f700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7a40a89e0 con 0x7fe7a4060eb0 2022-06-29T15:48:13.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.651+0000 7fe7a3fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe7a4060eb0 0x7fe7a40b1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:13.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.651+0000 7fe7a3fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe7a4060eb0 0x7fe7a40b1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49774/0 (socket says 172.21.15.47:49774) 2022-06-29T15:48:13.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.651+0000 7fe7a3fff700 1 -- 172.21.15.47:0/3918926313 learned_addr learned my addr 172.21.15.47:0/3918926313 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:13.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.651+0000 7fe7a3fff700 1 -- 172.21.15.47:0/3918926313 >> v1:172.21.15.64:6789/0 conn(0x7fe7a40b5c60 legacy=0x7fe7a4144980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:13.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.651+0000 7fe7a3fff700 1 -- 172.21.15.47:0/3918926313 >> v1:172.21.15.47:6789/0 conn(0x7fe7a4145090 legacy=0x7fe7a40b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:13.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.651+0000 7fe7a3fff700 1 -- 172.21.15.47:0/3918926313 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a40a8c80 con 0x7fe7a4060eb0 2022-06-29T15:48:13.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.651+0000 7fe7a3fff700 1 --2- 172.21.15.47:0/3918926313 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7a4060eb0 0x7fe7a40b1e30 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7fe7a406ac40 tx=0x7fe790006eb0).ready entity=mon.0 client_cookie=e27dd577dca11f7 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:13.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.651+0000 7fe7a37fe700 1 -- 172.21.15.47:0/3918926313 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe79000d610 con 0x7fe7a4060eb0 2022-06-29T15:48:13.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.651+0000 7fe7a37fe700 1 -- 172.21.15.47:0/3918926313 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe790000e90 con 0x7fe7a4060eb0 2022-06-29T15:48:13.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.651+0000 7fe7a37fe700 1 -- 172.21.15.47:0/3918926313 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe79000d960 con 0x7fe7a4060eb0 2022-06-29T15:48:13.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.651+0000 7fe7a9c5f700 1 -- 172.21.15.47:0/3918926313 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7a4060eb0 msgr2=0x7fe7a40b1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:13.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.651+0000 7fe7a9c5f700 1 --2- 172.21.15.47:0/3918926313 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7a4060eb0 0x7fe7a40b1e30 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7fe7a406ac40 tx=0x7fe790006eb0).stop 2022-06-29T15:48:13.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.651+0000 7fe7a9c5f700 1 -- 172.21.15.47:0/3918926313 shutdown_connections 2022-06-29T15:48:13.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.651+0000 7fe7a9c5f700 1 --2- 172.21.15.47:0/3918926313 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7a4060eb0 0x7fe7a40b1e30 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:13.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.651+0000 7fe7a9c5f700 1 -- 172.21.15.47:0/3918926313 >> 172.21.15.47:0/3918926313 conn(0x7fe7a4062560 msgr2=0x7fe7a4062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:13.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.651+0000 7fe7a9c5f700 1 -- 172.21.15.47:0/3918926313 shutdown_connections 2022-06-29T15:48:13.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.651+0000 7fe7a9c5f700 1 -- 172.21.15.47:0/3918926313 wait complete. 2022-06-29T15:48:13.666 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.655+0000 7fe7a9c5f700 1 Processor -- start 2022-06-29T15:48:13.666 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.655+0000 7fe7a9c5f700 1 -- start start 2022-06-29T15:48:13.666 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.655+0000 7fe7a9c5f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7a4060eb0 0x7fe7a40ace50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:13.666 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.659+0000 7fe7a9c5f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7a40b5c60 0x7fe7a4156860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:13.667 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.659+0000 7fe7a9c5f700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe7a4145090 0x7fe7a4156d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:13.667 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.659+0000 7fe7a9c5f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a41454a0 con 0x7fe7a40b5c60 2022-06-29T15:48:13.667 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.659+0000 7fe7a9c5f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a415a6b0 con 0x7fe7a4060eb0 2022-06-29T15:48:13.668 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.659+0000 7fe7a9c5f700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a41456d0 con 0x7fe7a4145090 2022-06-29T15:48:13.668 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.659+0000 7fe7a3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7a40b5c60 0x7fe7a4156860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:13.668 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.659+0000 7fe7a3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7a40b5c60 0x7fe7a4156860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49780/0 (socket says 172.21.15.47:49780) 2022-06-29T15:48:13.668 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.659+0000 7fe7a3fff700 1 -- 172.21.15.47:0/2798758481 learned_addr learned my addr 172.21.15.47:0/2798758481 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:13.669 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.659+0000 7fe7a945e700 1 --2- 172.21.15.47:0/2798758481 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe7a4145090 0x7fe7a4156d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:13.669 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.659+0000 7fe7a8c5d700 1 --2- 172.21.15.47:0/2798758481 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7a4060eb0 0x7fe7a40ace50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:13.669 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.659+0000 7fe7a3fff700 1 -- 172.21.15.47:0/2798758481 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe7a4145090 msgr2=0x7fe7a4156d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:13.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.659+0000 7fe7a3fff700 1 --2- 172.21.15.47:0/2798758481 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe7a4145090 0x7fe7a4156d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:13.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.659+0000 7fe7a3fff700 1 -- 172.21.15.47:0/2798758481 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7a4060eb0 msgr2=0x7fe7a40ace50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:13.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.659+0000 7fe7a3fff700 1 --2- 172.21.15.47:0/2798758481 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7a4060eb0 0x7fe7a40ace50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:13.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.659+0000 7fe7a3fff700 1 -- 172.21.15.47:0/2798758481 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe79000b1e0 con 0x7fe7a40b5c60 2022-06-29T15:48:13.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.659+0000 7fe7a3fff700 1 --2- 172.21.15.47:0/2798758481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7a40b5c60 0x7fe7a4156860 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 rx=0x7fe7a406ac40 tx=0x7fe790015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:13.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.659+0000 7fe7a1ffb700 1 -- 172.21.15.47:0/2798758481 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe7900067e0 con 0x7fe7a40b5c60 2022-06-29T15:48:13.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.659+0000 7fe7a1ffb700 1 -- 172.21.15.47:0/2798758481 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe790002f50 con 0x7fe7a40b5c60 2022-06-29T15:48:13.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.659+0000 7fe7a9c5f700 1 -- 172.21.15.47:0/2798758481 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7a415a940 con 0x7fe7a40b5c60 2022-06-29T15:48:13.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.659+0000 7fe7a1ffb700 1 -- 172.21.15.47:0/2798758481 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe790011c30 con 0x7fe7a40b5c60 2022-06-29T15:48:13.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.659+0000 7fe7a9c5f700 1 -- 172.21.15.47:0/2798758481 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7a4147710 con 0x7fe7a40b5c60 2022-06-29T15:48:13.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.659+0000 7fe7a9c5f700 1 -- 172.21.15.47:0/2798758481 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7a40573d0 con 0x7fe7a40b5c60 2022-06-29T15:48:13.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.663+0000 7fe7a1ffb700 1 -- 172.21.15.47:0/2798758481 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe790025030 con 0x7fe7a40b5c60 2022-06-29T15:48:13.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.663+0000 7fe7a1ffb700 1 --2- 172.21.15.47:0/2798758481 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe784053980 0x7fe784055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:13.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.663+0000 7fe7a1ffb700 1 -- 172.21.15.47:0/2798758481 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fe79006cf30 con 0x7fe7a40b5c60 2022-06-29T15:48:13.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.663+0000 7fe7a8c5d700 1 --2- 172.21.15.47:0/2798758481 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe784053980 0x7fe784055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:13.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.663+0000 7fe7a8c5d700 1 --2- 172.21.15.47:0/2798758481 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe784053980 0x7fe784055e40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7fe79800b000 tx=0x7fe798009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:13.676 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.671+0000 7fe7a1ffb700 1 -- 172.21.15.47:0/2798758481 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe790043fd0 con 0x7fe7a40b5c60 2022-06-29T15:48:13.833 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.831+0000 7fe7a9c5f700 1 -- 172.21.15.47:0/2798758481 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe7a4069e20 con 0x7fe7a40b5c60 2022-06-29T15:48:13.834 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.831+0000 7fe7a1ffb700 1 -- 172.21.15.47:0/2798758481 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+1556 (secure 0 0 0) 0x7fe790023020 con 0x7fe7a40b5c60 2022-06-29T15:48:13.834 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:48:13.835 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":32,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5887,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/929350416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":929350416},{"type":"v1","addr":"172.21.15.47:6835","nonce":929350416}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":27},{"gid":5970,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2425218664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2425218664},{"type":"v1","addr":"172.21.15.64:6833","nonce":2425218664}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":27},{"gid":6201,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/236814902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":236814902},{"type":"v1","addr":"172.21.15.47:6837","nonce":236814902}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":32}],"filesystems":[]} 2022-06-29T15:48:13.837 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.835+0000 7fe7a9c5f700 1 -- 172.21.15.47:0/2798758481 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe784053980 msgr2=0x7fe784055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:13.837 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.835+0000 7fe7a9c5f700 1 --2- 172.21.15.47:0/2798758481 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe784053980 0x7fe784055e40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7fe79800b000 tx=0x7fe798009040).stop 2022-06-29T15:48:13.838 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.835+0000 7fe7a9c5f700 1 -- 172.21.15.47:0/2798758481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7a40b5c60 msgr2=0x7fe7a4156860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:13.838 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.835+0000 7fe7a9c5f700 1 --2- 172.21.15.47:0/2798758481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7a40b5c60 0x7fe7a4156860 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 rx=0x7fe7a406ac40 tx=0x7fe790015000).stop 2022-06-29T15:48:13.838 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.835+0000 7fe7a9c5f700 1 -- 172.21.15.47:0/2798758481 shutdown_connections 2022-06-29T15:48:13.838 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.835+0000 7fe7a9c5f700 1 --2- 172.21.15.47:0/2798758481 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe7a4145090 0x7fe7a4156d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:13.839 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.835+0000 7fe7a9c5f700 1 --2- 172.21.15.47:0/2798758481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7a40b5c60 0x7fe7a4156860 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:13.839 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.835+0000 7fe7a9c5f700 1 --2- 172.21.15.47:0/2798758481 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe784053980 0x7fe784055e40 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:13.839 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.835+0000 7fe7a9c5f700 1 --2- 172.21.15.47:0/2798758481 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7a4060eb0 0x7fe7a40ace50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:13.839 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.835+0000 7fe7a9c5f700 1 -- 172.21.15.47:0/2798758481 >> 172.21.15.47:0/2798758481 conn(0x7fe7a4062560 msgr2=0x7fe7a4062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:13.839 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.835+0000 7fe7a9c5f700 1 -- 172.21.15.47:0/2798758481 shutdown_connections 2022-06-29T15:48:13.840 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:13.835+0000 7fe7a9c5f700 1 -- 172.21.15.47:0/2798758481 wait complete. 2022-06-29T15:48:13.840 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 32 2022-06-29T15:48:13.848 INFO:tasks.ceph.mds.a:Restarting daemon 2022-06-29T15:48:13.848 INFO:tasks.ceph.mds.a:Stopping old one... 2022-06-29T15:48:13.849 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-06-29T15:48:13.849 INFO:teuthology.orchestra.run:waiting for 300 2022-06-29T15:48:13.849 INFO:tasks.ceph.mds.c:Restarting daemon 2022-06-29T15:48:13.850 INFO:tasks.ceph.mds.c:Stopping old one... 2022-06-29T15:48:13.850 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2022-06-29T15:48:13.850 INFO:teuthology.orchestra.run:waiting for 300 2022-06-29T15:48:13.851 INFO:tasks.ceph.mds.b:Restarting daemon 2022-06-29T15:48:13.851 INFO:tasks.ceph.mds.b:Stopping old one... 2022-06-29T15:48:13.851 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-06-29T15:48:13.852 INFO:teuthology.orchestra.run:waiting for 300 2022-06-29T15:48:13.862 INFO:tasks.ceph.mds.a:Stopped 2022-06-29T15:48:13.863 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-06-29T15:48:13.865 INFO:tasks.ceph.mds.c:Stopped 2022-06-29T15:48:13.866 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2022-06-29T15:48:13.867 INFO:tasks.ceph.mds.b:Stopped 2022-06-29T15:48:13.868 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-06-29T15:48:13.870 INFO:tasks.ceph.mds.b:Started 2022-06-29T15:48:13.871 DEBUG:teuthology.parallel:result is None 2022-06-29T15:48:13.907 INFO:tasks.ceph.mds.a:Started 2022-06-29T15:48:13.908 DEBUG:teuthology.parallel:result is None 2022-06-29T15:48:13.909 INFO:tasks.ceph.mds.c:Started 2022-06-29T15:48:13.909 DEBUG:teuthology.parallel:result is None 2022-06-29T15:48:13.910 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-06-29T15:48:13.944 INFO:tasks.ceph.mds.b.smithi064.stdout:starting mds.b at 2022-06-29T15:48:13.981 INFO:tasks.ceph.mds.c.smithi047.stdout:starting mds.c at 2022-06-29T15:48:13.998 INFO:tasks.ceph.mds.a.smithi047.stdout:starting mds.a at 2022-06-29T15:48:14.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.039+0000 7f0ecbfff700 1 Processor -- start 2022-06-29T15:48:14.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.039+0000 7f0ecbfff700 1 -- start start 2022-06-29T15:48:14.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.039+0000 7f0ecbfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0ecc0acff0 0x7f0ecc05fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:14.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.039+0000 7f0ecbfff700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ecc0a97e0 con 0x7f0ecc1437d0 2022-06-29T15:48:14.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.039+0000 7f0ecbfff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ecc0b1d80 con 0x7f0ecc0b19a0 2022-06-29T15:48:14.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.039+0000 7f0ecbfff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ecc05f430 con 0x7f0ecc0acff0 2022-06-29T15:48:14.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.039+0000 7f0ecb7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0ecc1437d0 0x7f0ecc143c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35366/0 (socket says 172.21.15.47:35366) 2022-06-29T15:48:14.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.039+0000 7f0ecb7fe700 1 -- 172.21.15.47:0/185743639 learned_addr learned my addr 172.21.15.47:0/185743639 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:48:14.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.039+0000 7f0ec3fff700 1 --2- 172.21.15.47:0/185743639 >> v2:172.21.15.64:3300/0 conn(0x7f0ecc0acff0 0x7f0ecc05fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:14.045 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.039+0000 7f0eca7fc700 1 -- 172.21.15.47:0/185743639 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 782977496 0 0) 0x7f0ecc0a97e0 con 0x7f0ecc1437d0 2022-06-29T15:48:14.045 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.039+0000 7f0eca7fc700 1 -- 172.21.15.47:0/185743639 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0eac003260 con 0x7f0ecc1437d0 2022-06-29T15:48:14.045 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.039+0000 7f0eca7fc700 1 -- 172.21.15.47:0/185743639 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 676362658 0 0) 0x7f0ecc0b1d80 con 0x7f0ecc0b19a0 2022-06-29T15:48:14.045 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.039+0000 7f0eca7fc700 1 -- 172.21.15.47:0/185743639 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ecc0a97e0 con 0x7f0ecc0b19a0 2022-06-29T15:48:14.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.039+0000 7f0eca7fc700 1 -- 172.21.15.47:0/185743639 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 296582570 0 0) 0x7f0eac003260 con 0x7f0ecc1437d0 2022-06-29T15:48:14.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.039+0000 7f0eca7fc700 1 -- 172.21.15.47:0/185743639 >> v2:172.21.15.64:3300/0 conn(0x7f0ecc0acff0 msgr2=0x7f0ecc05fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:14.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.039+0000 7f0eca7fc700 1 --2- 172.21.15.47:0/185743639 >> v2:172.21.15.64:3300/0 conn(0x7f0ecc0acff0 0x7f0ecc05fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:14.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.039+0000 7f0eca7fc700 1 -- 172.21.15.47:0/185743639 >> v1:172.21.15.64:6789/0 conn(0x7f0ecc0b19a0 legacy=0x7f0ecc0ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:14.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.039+0000 7f0eca7fc700 1 -- 172.21.15.47:0/185743639 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ecc0a92a0 con 0x7f0ecc1437d0 2022-06-29T15:48:14.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.039+0000 7f0eca7fc700 1 -- 172.21.15.47:0/185743639 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7f0ec40019e0 con 0x7f0ecc1437d0 2022-06-29T15:48:14.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.039+0000 7f0eca7fc700 1 -- 172.21.15.47:0/185743639 >> v1:172.21.15.47:6789/0 conn(0x7f0ecc1437d0 legacy=0x7f0ecc143c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:14.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.039+0000 7f0eca7fc700 1 --2- 172.21.15.47:0/185743639 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0eac0038e0 0x7f0eac003ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:14.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.039+0000 7f0eca7fc700 1 --2- 172.21.15.47:0/185743639 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0eac004500 0x7f0eac004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:14.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.039+0000 7f0eca7fc700 1 --2- 172.21.15.47:0/185743639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0eac004e70 0x7f0eac00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:14.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.039+0000 7f0eca7fc700 1 -- 172.21.15.47:0/185743639 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc05f430 con 0x7f0eac004500 2022-06-29T15:48:14.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.039+0000 7f0eca7fc700 1 -- 172.21.15.47:0/185743639 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eac00bcf0 con 0x7f0eac004e70 2022-06-29T15:48:14.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.039+0000 7f0eca7fc700 1 -- 172.21.15.47:0/185743639 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eac00beb0 con 0x7f0eac0038e0 2022-06-29T15:48:14.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.039+0000 7f0ec3fff700 1 --2- 172.21.15.47:0/185743639 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0eac004500 0x7f0eac004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:14.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.039+0000 7f0ecb7fe700 1 --2- 172.21.15.47:0/185743639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0eac004e70 0x7f0eac00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:14.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.039+0000 7f0ecaffd700 1 --2- 172.21.15.47:0/185743639 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0eac0038e0 0x7f0eac003ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:14.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.039+0000 7f0ec3fff700 1 -- 172.21.15.47:0/185743639 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0eac0038e0 msgr2=0x7f0eac003ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:14.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.039+0000 7f0ec3fff700 1 --2- 172.21.15.47:0/185743639 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0eac0038e0 0x7f0eac003ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:14.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.039+0000 7f0ec3fff700 1 -- 172.21.15.47:0/185743639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0eac004e70 msgr2=0x7f0eac00b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:14.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.039+0000 7f0ec3fff700 1 --2- 172.21.15.47:0/185743639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0eac004e70 0x7f0eac00b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:14.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.039+0000 7f0ecb7fe700 1 -- 172.21.15.47:0/185743639 reap_dead start 2022-06-29T15:48:14.052 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.039+0000 7f0ec3fff700 1 -- 172.21.15.47:0/185743639 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eac003260 con 0x7f0eac004500 2022-06-29T15:48:14.052 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.039+0000 7f0ec3fff700 1 --2- 172.21.15.47:0/185743639 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0eac004500 0x7f0eac004980 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7f0eb400c000 tx=0x7f0eb4006df0).ready entity=mon.0 client_cookie=56b3d0db13dbcfd6 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:14.052 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.043+0000 7f0eca7fc700 1 -- 172.21.15.47:0/185743639 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0eb400e800 con 0x7f0eac004500 2022-06-29T15:48:14.052 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.043+0000 7f0eca7fc700 1 -- 172.21.15.47:0/185743639 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eb400e960 con 0x7f0eac004500 2022-06-29T15:48:14.053 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.043+0000 7f0eca7fc700 1 -- 172.21.15.47:0/185743639 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0eb400ecc0 con 0x7f0eac004500 2022-06-29T15:48:14.053 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.043+0000 7f0ecbfff700 1 -- 172.21.15.47:0/185743639 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0eac004500 msgr2=0x7f0eac004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:14.053 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.043+0000 7f0ecbfff700 1 --2- 172.21.15.47:0/185743639 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0eac004500 0x7f0eac004980 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7f0eb400c000 tx=0x7f0eb4006df0).stop 2022-06-29T15:48:14.058 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.043+0000 7f0ecbfff700 1 -- 172.21.15.47:0/185743639 shutdown_connections 2022-06-29T15:48:14.058 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.043+0000 7f0ecbfff700 1 --2- 172.21.15.47:0/185743639 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0eac004500 0x7f0eac004980 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:14.059 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.043+0000 7f0ecbfff700 1 -- 172.21.15.47:0/185743639 >> 172.21.15.47:0/185743639 conn(0x7f0ecc062df0 msgr2=0x7f0ecc0631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:14.059 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.043+0000 7f0ecbfff700 1 -- 172.21.15.47:0/185743639 shutdown_connections 2022-06-29T15:48:14.059 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.043+0000 7f0ecbfff700 1 -- 172.21.15.47:0/185743639 wait complete. 2022-06-29T15:48:14.060 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.043+0000 7f0ecbfff700 1 Processor -- start 2022-06-29T15:48:14.060 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.043+0000 7f0ecbfff700 1 -- start start 2022-06-29T15:48:14.061 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.043+0000 7f0ecbfff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0eac004500 0x7f0ecc138030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:14.061 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.043+0000 7f0ecbfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ecc1425d0 0x7f0ecc0982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:14.062 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.043+0000 7f0ecbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ecc142a20 0x7f0ecc155910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:14.062 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.043+0000 7f0ecbfff700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc0a99c0 con 0x7f0ecc1425d0 2022-06-29T15:48:14.062 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.043+0000 7f0ecbfff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc0ac8e0 con 0x7f0ecc142a20 2022-06-29T15:48:14.062 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.043+0000 7f0ecbfff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc0a9bf0 con 0x7f0eac004500 2022-06-29T15:48:14.063 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.043+0000 7f0ec3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ecc1425d0 0x7f0ecc0982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:14.063 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.043+0000 7f0ecaffd700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0eac004500 0x7f0ecc138030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:14.063 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.043+0000 7f0ecb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ecc142a20 0x7f0ecc155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:14.063 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.043+0000 7f0ec3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ecc1425d0 0x7f0ecc0982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49826/0 (socket says 172.21.15.47:49826) 2022-06-29T15:48:14.064 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.043+0000 7f0ec3fff700 1 -- 172.21.15.47:0/3465605746 learned_addr learned my addr 172.21.15.47:0/3465605746 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:14.064 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.043+0000 7f0ec3fff700 1 -- 172.21.15.47:0/3465605746 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0eac004500 msgr2=0x7f0ecc138030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:14.064 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.043+0000 7f0ec3fff700 1 --2- 172.21.15.47:0/3465605746 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0eac004500 0x7f0ecc138030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:14.064 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.043+0000 7f0ec3fff700 1 -- 172.21.15.47:0/3465605746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ecc142a20 msgr2=0x7f0ecc155910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:14.065 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.043+0000 7f0ec3fff700 1 --2- 172.21.15.47:0/3465605746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ecc142a20 0x7f0ecc155910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:14.065 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.047+0000 7f0ec3fff700 1 -- 172.21.15.47:0/3465605746 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ecc0aa3f0 con 0x7f0ecc1425d0 2022-06-29T15:48:14.065 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.047+0000 7f0ec3fff700 1 --2- 172.21.15.47:0/3465605746 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ecc1425d0 0x7f0ecc0982e0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0x7f0eb400c000 tx=0x7f0eb4019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:14.066 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.047+0000 7f0ec8ff9700 1 -- 172.21.15.47:0/3465605746 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0eb401cda0 con 0x7f0ecc1425d0 2022-06-29T15:48:14.066 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.047+0000 7f0ec8ff9700 1 -- 172.21.15.47:0/3465605746 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eb4000e90 con 0x7f0ecc1425d0 2022-06-29T15:48:14.066 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.047+0000 7f0ecbfff700 1 -- 172.21.15.47:0/3465605746 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ecc138c40 con 0x7f0ecc1425d0 2022-06-29T15:48:14.066 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.047+0000 7f0ec8ff9700 1 -- 172.21.15.47:0/3465605746 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0eb40245e0 con 0x7f0ecc1425d0 2022-06-29T15:48:14.067 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.047+0000 7f0ecbfff700 1 -- 172.21.15.47:0/3465605746 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ecc0ac4a0 con 0x7f0ecc1425d0 2022-06-29T15:48:14.067 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.047+0000 7f0ecbfff700 1 -- 172.21.15.47:0/3465605746 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ecc0aa580 con 0x7f0ecc1425d0 2022-06-29T15:48:14.067 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.047+0000 7f0ec8ff9700 1 -- 172.21.15.47:0/3465605746 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0eb4024740 con 0x7f0ecc1425d0 2022-06-29T15:48:14.067 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.051+0000 7f0ec8ff9700 1 --2- 172.21.15.47:0/3465605746 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0eb0053980 0x7f0eb0055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:14.068 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.051+0000 7f0ec8ff9700 1 -- 172.21.15.47:0/3465605746 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f0eb406d360 con 0x7f0ecc1425d0 2022-06-29T15:48:14.068 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.051+0000 7f0ecaffd700 1 --2- 172.21.15.47:0/3465605746 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0eb0053980 0x7f0eb0055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:14.069 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.051+0000 7f0ecaffd700 1 --2- 172.21.15.47:0/3465605746 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0eb0053980 0x7f0eb0055e40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f0ebc003000 tx=0x7f0ebc007a00).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:14.069 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.055+0000 7f0ec8ff9700 1 -- 172.21.15.47:0/3465605746 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0eb4047e30 con 0x7f0ecc1425d0 2022-06-29T15:48:14.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:14.199+0000 7f0ecbfff700 1 -- 172.21.15.47:0/3465605746 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f0ecc0573d0 con 0x7f0ecc1425d0 2022-06-29T15:48:15.022 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.019+0000 7f0ec8ff9700 1 -- 172.21.15.47:0/3465605746 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v74) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0eb400be60 con 0x7f0ecc1425d0 2022-06-29T15:48:15.027 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.023+0000 7f0ecbfff700 1 -- 172.21.15.47:0/3465605746 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0eb0053980 msgr2=0x7f0eb0055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:15.027 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.023+0000 7f0ecbfff700 1 --2- 172.21.15.47:0/3465605746 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0eb0053980 0x7f0eb0055e40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f0ebc003000 tx=0x7f0ebc007a00).stop 2022-06-29T15:48:15.027 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.023+0000 7f0ecbfff700 1 -- 172.21.15.47:0/3465605746 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ecc1425d0 msgr2=0x7f0ecc0982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:15.028 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.023+0000 7f0ecbfff700 1 --2- 172.21.15.47:0/3465605746 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ecc1425d0 0x7f0ecc0982e0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0x7f0eb400c000 tx=0x7f0eb4019000).stop 2022-06-29T15:48:15.028 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.027+0000 7f0ecbfff700 1 -- 172.21.15.47:0/3465605746 shutdown_connections 2022-06-29T15:48:15.028 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.027+0000 7f0ecbfff700 1 --2- 172.21.15.47:0/3465605746 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0eb0053980 0x7f0eb0055e40 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:15.028 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.027+0000 7f0ecbfff700 1 --2- 172.21.15.47:0/3465605746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ecc142a20 0x7f0ecc155910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:15.029 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.027+0000 7f0ecbfff700 1 --2- 172.21.15.47:0/3465605746 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ecc1425d0 0x7f0ecc0982e0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:15.029 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.027+0000 7f0ecbfff700 1 --2- 172.21.15.47:0/3465605746 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0eac004500 0x7f0ecc138030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:15.029 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.027+0000 7f0ecbfff700 1 -- 172.21.15.47:0/3465605746 >> 172.21.15.47:0/3465605746 conn(0x7f0ecc062df0 msgr2=0x7f0ecc0ae760 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:15.029 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.027+0000 7f0ecbfff700 1 -- 172.21.15.47:0/3465605746 shutdown_connections 2022-06-29T15:48:15.030 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.027+0000 7f0ecbfff700 1 -- 172.21.15.47:0/3465605746 wait complete. 2022-06-29T15:48:15.030 INFO:teuthology.orchestra.run.smithi047.stderr: removed all blacklist entries 2022-06-29T15:48:15.042 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2022-06-29T15:48:15.043 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-06-29T15:48:15.150 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.147+0000 7f1b4f3af700 1 Processor -- start 2022-06-29T15:48:15.150 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.147+0000 7f1b4f3af700 1 -- start start 2022-06-29T15:48:15.152 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.147+0000 7f1b4f3af700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1b480b2ee0 0x7f1b480b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:15.152 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.147+0000 7f1b4f3af700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b481400e0 con 0x7f1b480ad970 2022-06-29T15:48:15.153 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.151+0000 7f1b4f3af700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b48143570 con 0x7f1b480b0e50 2022-06-29T15:48:15.153 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.151+0000 7f1b4f3af700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b48140ef0 con 0x7f1b480b2ee0 2022-06-29T15:48:15.153 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.151+0000 7f1b4dbac700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1b480ad970 0x7f1b481407e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35386/0 (socket says 172.21.15.47:35386) 2022-06-29T15:48:15.153 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.151+0000 7f1b4dbac700 1 -- 172.21.15.47:0/2083785495 learned_addr learned my addr 172.21.15.47:0/2083785495 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:48:15.153 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.151+0000 7f1b4e3ad700 1 --2- 172.21.15.47:0/2083785495 >> v2:172.21.15.47:3300/0 conn(0x7f1b480b2ee0 0x7f1b480b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:15.154 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.151+0000 7f1b4d3ab700 1 -- 172.21.15.47:0/2083785495 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1347248938 0 0) 0x7f1b481400e0 con 0x7f1b480ad970 2022-06-29T15:48:15.154 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.151+0000 7f1b4d3ab700 1 -- 172.21.15.47:0/2083785495 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b30003260 con 0x7f1b480ad970 2022-06-29T15:48:15.154 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.151+0000 7f1b4d3ab700 1 -- 172.21.15.47:0/2083785495 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4015917822 0 0) 0x7f1b30003260 con 0x7f1b480ad970 2022-06-29T15:48:15.154 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.151+0000 7f1b4e3ad700 1 -- 172.21.15.47:0/2083785495 >> v1:172.21.15.64:6789/0 conn(0x7f1b480b0e50 legacy=0x7f1b4813d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:15.155 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.151+0000 7f1b4e3ad700 1 -- 172.21.15.47:0/2083785495 >> v1:172.21.15.47:6789/0 conn(0x7f1b480ad970 legacy=0x7f1b481407e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:15.155 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.151+0000 7f1b4e3ad700 1 -- 172.21.15.47:0/2083785495 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b48140340 con 0x7f1b480b2ee0 2022-06-29T15:48:15.155 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.151+0000 7f1b4e3ad700 1 --2- 172.21.15.47:0/2083785495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b480b2ee0 0x7f1b480b32c0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0x7f1b44001820 tx=0x7f1b3c006df0).ready entity=mon.0 client_cookie=2a49f8b893a8fd82 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:15.155 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.151+0000 7f1b4d3ab700 1 -- 172.21.15.47:0/2083785495 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b44001680 con 0x7f1b480b2ee0 2022-06-29T15:48:15.156 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.151+0000 7f1b4d3ab700 1 -- 172.21.15.47:0/2083785495 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b3c000e90 con 0x7f1b480b2ee0 2022-06-29T15:48:15.156 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.151+0000 7f1b4d3ab700 1 -- 172.21.15.47:0/2083785495 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b3c00d4f0 con 0x7f1b480b2ee0 2022-06-29T15:48:15.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.151+0000 7f1b4f3af700 1 -- 172.21.15.47:0/2083785495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b480b2ee0 msgr2=0x7f1b480b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:15.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.151+0000 7f1b4f3af700 1 --2- 172.21.15.47:0/2083785495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b480b2ee0 0x7f1b480b32c0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0x7f1b44001820 tx=0x7f1b3c006df0).stop 2022-06-29T15:48:15.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.151+0000 7f1b4f3af700 1 -- 172.21.15.47:0/2083785495 shutdown_connections 2022-06-29T15:48:15.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.151+0000 7f1b4f3af700 1 --2- 172.21.15.47:0/2083785495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b480b2ee0 0x7f1b480b32c0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:15.161 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.151+0000 7f1b4f3af700 1 -- 172.21.15.47:0/2083785495 >> 172.21.15.47:0/2083785495 conn(0x7f1b48063670 msgr2=0x7f1b48063a80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:15.161 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.151+0000 7f1b4f3af700 1 -- 172.21.15.47:0/2083785495 shutdown_connections 2022-06-29T15:48:15.161 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.151+0000 7f1b4f3af700 1 -- 172.21.15.47:0/2083785495 wait complete. 2022-06-29T15:48:15.161 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.151+0000 7f1b4f3af700 1 Processor -- start 2022-06-29T15:48:15.161 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.155+0000 7f1b4f3af700 1 -- start start 2022-06-29T15:48:15.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.155+0000 7f1b4f3af700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b480ad970 0x7f1b48066ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:15.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.155+0000 7f1b4f3af700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b480b0e50 0x7f1b48142ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:15.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.155+0000 7f1b4f3af700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1b480b2ee0 0x7f1b48064350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:15.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.155+0000 7f1b4f3af700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b480ac5d0 con 0x7f1b480ad970 2022-06-29T15:48:15.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.155+0000 7f1b4f3af700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b48066510 con 0x7f1b480b0e50 2022-06-29T15:48:15.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.155+0000 7f1b4f3af700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b480acb50 con 0x7f1b480b2ee0 2022-06-29T15:48:15.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.155+0000 7f1b4e3ad700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b480ad970 0x7f1b48066ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:15.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.155+0000 7f1b4e3ad700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b480ad970 0x7f1b48066ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49838/0 (socket says 172.21.15.47:49838) 2022-06-29T15:48:15.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.155+0000 7f1b4e3ad700 1 -- 172.21.15.47:0/1624221975 learned_addr learned my addr 172.21.15.47:0/1624221975 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:15.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.155+0000 7f1b4ebae700 1 --2- 172.21.15.47:0/1624221975 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1b480b2ee0 0x7f1b48064350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:15.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.155+0000 7f1b4dbac700 1 --2- 172.21.15.47:0/1624221975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b480b0e50 0x7f1b48142ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:15.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.155+0000 7f1b4e3ad700 1 -- 172.21.15.47:0/1624221975 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1b480b2ee0 msgr2=0x7f1b48064350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:15.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.155+0000 7f1b4e3ad700 1 --2- 172.21.15.47:0/1624221975 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1b480b2ee0 0x7f1b48064350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:15.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.155+0000 7f1b4e3ad700 1 -- 172.21.15.47:0/1624221975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b480b0e50 msgr2=0x7f1b48142ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:15.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.155+0000 7f1b4e3ad700 1 --2- 172.21.15.47:0/1624221975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b480b0e50 0x7f1b48142ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:15.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.155+0000 7f1b4e3ad700 1 -- 172.21.15.47:0/1624221975 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b3c00b180 con 0x7f1b480ad970 2022-06-29T15:48:15.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.155+0000 7f1b4ebae700 1 --2- 172.21.15.47:0/1624221975 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1b480b2ee0 0x7f1b48064350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:48:15.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.155+0000 7f1b4e3ad700 1 --2- 172.21.15.47:0/1624221975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b480ad970 0x7f1b48066ff0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 rx=0x7f1b48066090 tx=0x7f1b3c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:15.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.155+0000 7f1b3b7fe700 1 -- 172.21.15.47:0/1624221975 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b3c002f50 con 0x7f1b480ad970 2022-06-29T15:48:15.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.155+0000 7f1b3b7fe700 1 -- 172.21.15.47:0/1624221975 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b3c00de40 con 0x7f1b480ad970 2022-06-29T15:48:15.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.155+0000 7f1b4f3af700 1 -- 172.21.15.47:0/1624221975 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b48148950 con 0x7f1b480ad970 2022-06-29T15:48:15.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.155+0000 7f1b3b7fe700 1 -- 172.21.15.47:0/1624221975 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b3c01e5f0 con 0x7f1b480ad970 2022-06-29T15:48:15.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.155+0000 7f1b4f3af700 1 -- 172.21.15.47:0/1624221975 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b481567b0 con 0x7f1b480ad970 2022-06-29T15:48:15.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.159+0000 7f1b4f3af700 1 -- 172.21.15.47:0/1624221975 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b480573d0 con 0x7f1b480ad970 2022-06-29T15:48:15.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.159+0000 7f1b3b7fe700 1 -- 172.21.15.47:0/1624221975 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1b3c025030 con 0x7f1b480ad970 2022-06-29T15:48:15.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.159+0000 7f1b3b7fe700 1 --2- 172.21.15.47:0/1624221975 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1b28053950 0x7f1b28055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:15.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.159+0000 7f1b3b7fe700 1 -- 172.21.15.47:0/1624221975 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f1b3c06cb30 con 0x7f1b480ad970 2022-06-29T15:48:15.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.159+0000 7f1b4dbac700 1 --2- 172.21.15.47:0/1624221975 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1b28053950 0x7f1b28055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:15.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.159+0000 7f1b4dbac700 1 --2- 172.21.15.47:0/1624221975 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1b28053950 0x7f1b28055e10 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f1b4400b000 tx=0x7f1b440015f0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:15.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.163+0000 7f1b3b7fe700 1 -- 172.21.15.47:0/1624221975 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1b3c043cb0 con 0x7f1b480ad970 2022-06-29T15:48:15.335 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.335+0000 7f1b4f3af700 1 -- 172.21.15.47:0/1624221975 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f1b48069e20 con 0x7f1b480ad970 2022-06-29T15:48:15.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.335+0000 7f1b3b7fe700 1 -- 172.21.15.47:0/1624221975 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) v1 ==== 80+0+4255 (secure 0 0 0) 0x7f1b3c047250 con 0x7f1b480ad970 2022-06-29T15:48:15.337 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:48:15.337 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2022-06-29T15:48:15.337 INFO:teuthology.orchestra.run.smithi047.stdout: "auth_dump": [ 2022-06-29T15:48:15.337 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:15.338 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.a", 2022-06-29T15:48:15.338 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxiQJFFMxAA9Tv7FJD9u/BKvbkuKld6Kg==", 2022-06-29T15:48:15.338 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:48:15.339 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:48:15.339 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:48:15.339 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:48:15.339 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:48:15.340 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:15.340 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:15.340 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:15.340 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.b", 2022-06-29T15:48:15.341 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC2crxibt6FARAAEf2bwXKFhXKtqr1nN0rdXQ==", 2022-06-29T15:48:15.341 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:48:15.341 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:48:15.341 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:48:15.341 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:48:15.342 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:48:15.342 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:15.342 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:15.342 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:15.343 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.c", 2022-06-29T15:48:15.343 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxi/4ZMORAANpLbgTm3ehqi80USc3swGg==", 2022-06-29T15:48:15.343 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:48:15.343 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:48:15.343 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:48:15.344 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:48:15.344 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:48:15.344 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:15.344 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:15.345 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:15.345 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.0", 2022-06-29T15:48:15.345 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC2crxiY0YsMxAAZivb+CxL/w7vZwQv37Y4oQ==", 2022-06-29T15:48:15.345 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:48:15.345 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:48:15.346 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:48:15.346 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:48:15.346 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:15.346 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:15.347 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:15.347 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.1", 2022-06-29T15:48:15.347 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC5crxitLkKFxAA8+++LUUKLiO9SAuAfYgT5g==", 2022-06-29T15:48:15.347 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:48:15.348 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:48:15.348 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:48:15.348 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:48:15.348 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:15.349 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:15.349 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:15.349 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.2", 2022-06-29T15:48:15.349 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC7crxi1oM9NxAA2FfLdNt32YPwlyNdGQpaQA==", 2022-06-29T15:48:15.350 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:48:15.350 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:48:15.350 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:48:15.350 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:48:15.351 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:15.351 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:15.351 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:15.351 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.3", 2022-06-29T15:48:15.352 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC+crxiTsBgGhAA+gLBBDaskPbmRNWzgKwcHA==", 2022-06-29T15:48:15.352 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:48:15.352 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:48:15.352 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:48:15.352 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:48:15.353 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:15.353 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:15.353 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:15.353 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.4", 2022-06-29T15:48:15.354 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDDcrxivJebIhAAfwWJa7tK4hamkk1/1CSSfw==", 2022-06-29T15:48:15.354 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:48:15.354 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:48:15.354 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:48:15.355 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:48:15.355 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:15.355 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:15.355 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:15.355 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.5", 2022-06-29T15:48:15.356 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDGcrxijuozBhAAt+Dx3wH3WKp3DrbCbhmh8Q==", 2022-06-29T15:48:15.356 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:48:15.356 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:48:15.356 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:48:15.357 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:48:15.357 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:15.357 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:15.357 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:15.358 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.6", 2022-06-29T15:48:15.358 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDIcrxi91kXJxAA9JX7n98P394bD41Q9LUdvQ==", 2022-06-29T15:48:15.358 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:48:15.358 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:48:15.358 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:48:15.359 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:48:15.359 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:15.359 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:15.359 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:15.360 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.7", 2022-06-29T15:48:15.360 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDLcrxiHj0ZDRAASy6fAix8TmEZbAGtuFCfFg==", 2022-06-29T15:48:15.360 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:48:15.360 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:48:15.360 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:48:15.361 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:48:15.361 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:15.361 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:15.361 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:15.362 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.0", 2022-06-29T15:48:15.362 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC2crxiVDlfBxAAezkUfg4e2g0Rfakaa/Ek1Q==", 2022-06-29T15:48:15.362 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:48:15.362 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:48:15.363 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow r", 2022-06-29T15:48:15.363 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-29T15:48:15.363 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:15.363 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:15.364 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:15.364 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.1", 2022-06-29T15:48:15.364 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC2crxisXJ1CxAAhl07oF7AW66NdzfvvZTJmA==", 2022-06-29T15:48:15.364 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:48:15.365 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:48:15.365 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow r", 2022-06-29T15:48:15.365 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-29T15:48:15.365 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:15.365 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:15.366 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:15.366 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.admin", 2022-06-29T15:48:15.366 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxiW7HSFRAAhYqoGU9GFqj2HfUVGValSA==", 2022-06-29T15:48:15.366 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:48:15.367 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2022-06-29T15:48:15.367 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:48:15.367 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:48:15.367 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:48:15.368 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:15.368 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:15.368 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:15.368 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.x", 2022-06-29T15:48:15.369 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxidmuIMBAAcpNlYVu5+icCHg+yIjcKXw==", 2022-06-29T15:48:15.369 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:48:15.369 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2022-06-29T15:48:15.369 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2022-06-29T15:48:15.369 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:48:15.370 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:15.370 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:15.370 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:15.370 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.y", 2022-06-29T15:48:15.371 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxisSMFLBAAv03r4OoPjOP9nI9G/LK0iw==", 2022-06-29T15:48:15.371 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:48:15.371 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2022-06-29T15:48:15.371 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2022-06-29T15:48:15.372 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:48:15.372 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:15.372 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:15.372 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:48:15.372 INFO:teuthology.orchestra.run.smithi047.stdout:} 2022-06-29T15:48:15.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.335+0000 7f1b4f3af700 1 -- 172.21.15.47:0/1624221975 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1b28053950 msgr2=0x7f1b28055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:15.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.335+0000 7f1b4f3af700 1 --2- 172.21.15.47:0/1624221975 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1b28053950 0x7f1b28055e10 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f1b4400b000 tx=0x7f1b440015f0).stop 2022-06-29T15:48:15.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.335+0000 7f1b4f3af700 1 -- 172.21.15.47:0/1624221975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b480ad970 msgr2=0x7f1b48066ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:15.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.335+0000 7f1b4f3af700 1 --2- 172.21.15.47:0/1624221975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b480ad970 0x7f1b48066ff0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 rx=0x7f1b48066090 tx=0x7f1b3c014000).stop 2022-06-29T15:48:15.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.335+0000 7f1b4f3af700 1 -- 172.21.15.47:0/1624221975 shutdown_connections 2022-06-29T15:48:15.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.335+0000 7f1b4f3af700 1 --2- 172.21.15.47:0/1624221975 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1b480b2ee0 0x7f1b48064350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:15.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.335+0000 7f1b4f3af700 1 --2- 172.21.15.47:0/1624221975 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1b28053950 0x7f1b28055e10 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:15.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.335+0000 7f1b4f3af700 1 --2- 172.21.15.47:0/1624221975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b480b0e50 0x7f1b48142ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:15.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.335+0000 7f1b4f3af700 1 --2- 172.21.15.47:0/1624221975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b480ad970 0x7f1b48066ff0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:15.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.335+0000 7f1b4f3af700 1 -- 172.21.15.47:0/1624221975 >> 172.21.15.47:0/1624221975 conn(0x7f1b48063670 msgr2=0x7f1b4805bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:15.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.339+0000 7f1b4f3af700 1 -- 172.21.15.47:0/1624221975 shutdown_connections 2022-06-29T15:48:15.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.339+0000 7f1b4f3af700 1 -- 172.21.15.47:0/1624221975 wait complete. 2022-06-29T15:48:15.378 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:48:15.485 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.483+0000 7f3cb9f61700 1 Processor -- start 2022-06-29T15:48:15.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.483+0000 7f3cb9f61700 1 -- start start 2022-06-29T15:48:15.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.483+0000 7f3cb9f61700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3cb405c6b0 0x7f3cb40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:15.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.483+0000 7f3cb9f61700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3cb40b5c50 0x7f3cb414b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:15.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.483+0000 7f3cb9f61700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3cb4147360 0x7f3cb40b33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:15.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.483+0000 7f3cb9f61700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cb40b2910 con 0x7f3cb40b5c50 2022-06-29T15:48:15.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.483+0000 7f3cb9f61700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cb414a930 con 0x7f3cb405c6b0 2022-06-29T15:48:15.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.483+0000 7f3cb9f61700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb414b7c0 con 0x7f3cb4147360 2022-06-29T15:48:15.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.483+0000 7f3cb3fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3cb40b5c50 0x7f3cb414b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:15.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.483+0000 7f3cb3fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3cb40b5c50 0x7f3cb414b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49846/0 (socket says 172.21.15.47:49846) 2022-06-29T15:48:15.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.483+0000 7f3cb3fff700 1 -- 172.21.15.47:0/1563091484 learned_addr learned my addr 172.21.15.47:0/1563091484 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:15.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.483+0000 7f3cb9760700 1 --2- 172.21.15.47:0/1563091484 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3cb4147360 0x7f3cb40b33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:15.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.483+0000 7f3cb8f5f700 1 --2- 172.21.15.47:0/1563091484 >> v2:172.21.15.64:3300/0 conn(0x7f3cb405c6b0 0x7f3cb40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:15.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.487+0000 7f3cb3fff700 1 -- 172.21.15.47:0/1563091484 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3cb4147360 msgr2=0x7f3cb40b33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:15.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.487+0000 7f3cb3fff700 1 --2- 172.21.15.47:0/1563091484 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3cb4147360 0x7f3cb40b33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:15.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.487+0000 7f3cb3fff700 1 -- 172.21.15.47:0/1563091484 >> v2:172.21.15.64:3300/0 conn(0x7f3cb405c6b0 msgr2=0x7f3cb40b4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:15.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.487+0000 7f3cb3fff700 1 --2- 172.21.15.47:0/1563091484 >> v2:172.21.15.64:3300/0 conn(0x7f3cb405c6b0 0x7f3cb40b4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:15.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.487+0000 7f3cb3fff700 1 -- 172.21.15.47:0/1563091484 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb40b5440 con 0x7f3cb40b5c50 2022-06-29T15:48:15.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.487+0000 7f3cb8f5f700 1 --2- 172.21.15.47:0/1563091484 >> v2:172.21.15.64:3300/0 conn(0x7f3cb405c6b0 0x7f3cb40b4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:48:15.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.487+0000 7f3cb3fff700 1 --2- 172.21.15.47:0/1563091484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cb40b5c50 0x7f3cb414b2d0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 rx=0x7f3cb40b6080 tx=0x7f3c9c006eb0).ready entity=mon.0 client_cookie=98625b13e10bf7a2 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:15.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.487+0000 7f3cb37fe700 1 -- 172.21.15.47:0/1563091484 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c9c00d570 con 0x7f3cb40b5c50 2022-06-29T15:48:15.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.487+0000 7f3cb37fe700 1 -- 172.21.15.47:0/1563091484 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c9c000e90 con 0x7f3cb40b5c50 2022-06-29T15:48:15.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.487+0000 7f3cb37fe700 1 -- 172.21.15.47:0/1563091484 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c9c00d900 con 0x7f3cb40b5c50 2022-06-29T15:48:15.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.487+0000 7f3cb9f61700 1 -- 172.21.15.47:0/1563091484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cb40b5c50 msgr2=0x7f3cb414b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:15.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.487+0000 7f3cb9f61700 1 --2- 172.21.15.47:0/1563091484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cb40b5c50 0x7f3cb414b2d0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 rx=0x7f3cb40b6080 tx=0x7f3c9c006eb0).stop 2022-06-29T15:48:15.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.487+0000 7f3cb9f61700 1 -- 172.21.15.47:0/1563091484 shutdown_connections 2022-06-29T15:48:15.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.487+0000 7f3cb9f61700 1 --2- 172.21.15.47:0/1563091484 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3cb4147360 0x7f3cb40b33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:15.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.487+0000 7f3cb9f61700 1 --2- 172.21.15.47:0/1563091484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cb40b5c50 0x7f3cb414b2d0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:15.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.487+0000 7f3cb9f61700 1 --2- 172.21.15.47:0/1563091484 >> v2:172.21.15.64:3300/0 conn(0x7f3cb405c6b0 0x7f3cb40b4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:15.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.487+0000 7f3cb9f61700 1 -- 172.21.15.47:0/1563091484 >> 172.21.15.47:0/1563091484 conn(0x7f3cb4060e00 msgr2=0x7f3cb40611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:15.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.487+0000 7f3cb9f61700 1 -- 172.21.15.47:0/1563091484 shutdown_connections 2022-06-29T15:48:15.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.487+0000 7f3cb9f61700 1 -- 172.21.15.47:0/1563091484 wait complete. 2022-06-29T15:48:15.499 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.487+0000 7f3cb9f61700 1 Processor -- start 2022-06-29T15:48:15.499 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.487+0000 7f3cb9f61700 1 -- start start 2022-06-29T15:48:15.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.491+0000 7f3cb9f61700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3cb405c6b0 0x7f3cb4145860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:15.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.491+0000 7f3cb9f61700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cb40b5c50 0x7f3cb4148d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:15.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.491+0000 7f3cb9f61700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cb4147360 0x7f3cb413b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:15.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.491+0000 7f3cb9f61700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb4144790 con 0x7f3cb4147360 2022-06-29T15:48:15.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.491+0000 7f3cb9f61700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb4149eb0 con 0x7f3cb40b5c50 2022-06-29T15:48:15.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.491+0000 7f3cb9f61700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb4149c10 con 0x7f3cb405c6b0 2022-06-29T15:48:15.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.491+0000 7f3cb9760700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cb4147360 0x7f3cb413b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:15.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.491+0000 7f3cb9760700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cb4147360 0x7f3cb413b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49856/0 (socket says 172.21.15.47:49856) 2022-06-29T15:48:15.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.491+0000 7f3cb9760700 1 -- 172.21.15.47:0/2408009196 learned_addr learned my addr 172.21.15.47:0/2408009196 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:15.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.491+0000 7f3cb3fff700 1 --2- 172.21.15.47:0/2408009196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cb40b5c50 0x7f3cb4148d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:15.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.491+0000 7f3cb8f5f700 1 --2- 172.21.15.47:0/2408009196 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3cb405c6b0 0x7f3cb4145860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:15.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.491+0000 7f3cb9760700 1 -- 172.21.15.47:0/2408009196 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3cb405c6b0 msgr2=0x7f3cb4145860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:15.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.491+0000 7f3cb9760700 1 --2- 172.21.15.47:0/2408009196 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3cb405c6b0 0x7f3cb4145860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:15.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.491+0000 7f3cb9760700 1 -- 172.21.15.47:0/2408009196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cb40b5c50 msgr2=0x7f3cb4148d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:15.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.491+0000 7f3cb9760700 1 --2- 172.21.15.47:0/2408009196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cb40b5c50 0x7f3cb4148d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:15.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.491+0000 7f3cb9760700 1 -- 172.21.15.47:0/2408009196 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c9c00b1e0 con 0x7f3cb4147360 2022-06-29T15:48:15.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.491+0000 7f3cb8f5f700 1 --2- 172.21.15.47:0/2408009196 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3cb405c6b0 0x7f3cb4145860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:48:15.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.491+0000 7f3cb3fff700 1 --2- 172.21.15.47:0/2408009196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cb40b5c50 0x7f3cb4148d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:48:15.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.491+0000 7f3cb9760700 1 --2- 172.21.15.47:0/2408009196 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cb4147360 0x7f3cb413b980 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x7f3cac006e00 tx=0x7f3cac0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:15.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.491+0000 7f3cb1ffb700 1 -- 172.21.15.47:0/2408009196 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3cac00c7f0 con 0x7f3cb4147360 2022-06-29T15:48:15.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.491+0000 7f3cb1ffb700 1 -- 172.21.15.47:0/2408009196 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cac00c950 con 0x7f3cb4147360 2022-06-29T15:48:15.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.491+0000 7f3cb1ffb700 1 -- 172.21.15.47:0/2408009196 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3cac00ccb0 con 0x7f3cb4147360 2022-06-29T15:48:15.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.491+0000 7f3cb9f61700 1 -- 172.21.15.47:0/2408009196 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cb41449f0 con 0x7f3cb4147360 2022-06-29T15:48:15.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.491+0000 7f3cb9f61700 1 -- 172.21.15.47:0/2408009196 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cb40b0ae0 con 0x7f3cb4147360 2022-06-29T15:48:15.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.491+0000 7f3cb9f61700 1 -- 172.21.15.47:0/2408009196 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cb40573d0 con 0x7f3cb4147360 2022-06-29T15:48:15.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.491+0000 7f3cb1ffb700 1 -- 172.21.15.47:0/2408009196 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3cac0123f0 con 0x7f3cb4147360 2022-06-29T15:48:15.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.491+0000 7f3cb1ffb700 1 --2- 172.21.15.47:0/2408009196 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3c8c053a10 0x7f3c8c055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:15.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.491+0000 7f3cb1ffb700 1 -- 172.21.15.47:0/2408009196 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f3cac06c0b0 con 0x7f3cb4147360 2022-06-29T15:48:15.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.495+0000 7f3cb8f5f700 1 --2- 172.21.15.47:0/2408009196 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3c8c053a10 0x7f3c8c055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:15.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.495+0000 7f3cb8f5f700 1 --2- 172.21.15.47:0/2408009196 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3c8c053a10 0x7f3c8c055ed0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7f3ca400c000 tx=0x7f3ca4008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:15.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.499+0000 7f3cb1ffb700 1 -- 172.21.15.47:0/2408009196 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3cac0430b0 con 0x7f3cb4147360 2022-06-29T15:48:15.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.627+0000 7f3cb9f61700 1 -- 172.21.15.47:0/2408009196 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3cb40b1ef0 con 0x7f3cb4147360 2022-06-29T15:48:15.632 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.631+0000 7f3cb1ffb700 1 -- 172.21.15.47:0/2408009196 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+13415 (secure 0 0 0) 0x7f3cac0466d0 con 0x7f3cb4147360 2022-06-29T15:48:15.633 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:48:15.633 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":74,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:48:15.015557+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:48:15.634 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.631+0000 7f3cb9f61700 1 -- 172.21.15.47:0/2408009196 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3c8c053a10 msgr2=0x7f3c8c055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:15.635 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.631+0000 7f3cb9f61700 1 --2- 172.21.15.47:0/2408009196 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3c8c053a10 0x7f3c8c055ed0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7f3ca400c000 tx=0x7f3ca4008040).stop 2022-06-29T15:48:15.635 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.631+0000 7f3cb9f61700 1 -- 172.21.15.47:0/2408009196 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cb4147360 msgr2=0x7f3cb413b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:15.635 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.631+0000 7f3cb9f61700 1 --2- 172.21.15.47:0/2408009196 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cb4147360 0x7f3cb413b980 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x7f3cac006e00 tx=0x7f3cac0091d0).stop 2022-06-29T15:48:15.635 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.631+0000 7f3cb9f61700 1 -- 172.21.15.47:0/2408009196 shutdown_connections 2022-06-29T15:48:15.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.631+0000 7f3cb9f61700 1 --2- 172.21.15.47:0/2408009196 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cb4147360 0x7f3cb413b980 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:15.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.631+0000 7f3cb9f61700 1 --2- 172.21.15.47:0/2408009196 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3c8c053a10 0x7f3c8c055ed0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:15.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.631+0000 7f3cb9f61700 1 --2- 172.21.15.47:0/2408009196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cb40b5c50 0x7f3cb4148d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:15.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.631+0000 7f3cb9f61700 1 --2- 172.21.15.47:0/2408009196 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3cb405c6b0 0x7f3cb4145860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:15.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.631+0000 7f3cb9f61700 1 -- 172.21.15.47:0/2408009196 >> 172.21.15.47:0/2408009196 conn(0x7f3cb4060e00 msgr2=0x7f3cb413eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:15.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.631+0000 7f3cb9f61700 1 -- 172.21.15.47:0/2408009196 shutdown_connections 2022-06-29T15:48:15.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.631+0000 7f3cb9f61700 1 -- 172.21.15.47:0/2408009196 wait complete. 2022-06-29T15:48:15.646 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-29T15:42:16.971431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-29T15:42:25.290884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-06-29T15:48:15.647 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-29T15:48:15.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.759+0000 7f72e9157700 1 Processor -- start 2022-06-29T15:48:15.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.759+0000 7f72e9157700 1 -- start start 2022-06-29T15:48:15.762 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.759+0000 7f72e9157700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f72e4146720 0x7f72e4140780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:15.763 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.759+0000 7f72e9157700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f72e40b4320 0x7f72e4143230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:15.763 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.759+0000 7f72e9157700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72e4061600 con 0x7f72e4061220 2022-06-29T15:48:15.763 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.759+0000 7f72e9157700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72e4146bc0 con 0x7f72e40b4320 2022-06-29T15:48:15.763 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.759+0000 7f72e9157700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72e4146e60 con 0x7f72e4146720 2022-06-29T15:48:15.764 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.759+0000 7f72e2d9d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f72e4061220 0x7f72e4140070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35414/0 (socket says 172.21.15.47:35414) 2022-06-29T15:48:15.764 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.759+0000 7f72e2d9d700 1 -- 172.21.15.47:0/4164385543 learned_addr learned my addr 172.21.15.47:0/4164385543 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:48:15.764 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.759+0000 7f72da59c700 1 --2- 172.21.15.47:0/4164385543 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f72e4146720 0x7f72e4140780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:15.764 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.759+0000 7f72e359e700 1 --2- 172.21.15.47:0/4164385543 >> v2:172.21.15.64:3300/0 conn(0x7f72e40b4320 0x7f72e4143230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:15.765 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.759+0000 7f72e259c700 1 -- 172.21.15.47:0/4164385543 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 464415204 0 0) 0x7f72e4061600 con 0x7f72e4061220 2022-06-29T15:48:15.765 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.759+0000 7f72e259c700 1 -- 172.21.15.47:0/4164385543 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72c4003260 con 0x7f72e4061220 2022-06-29T15:48:15.765 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.759+0000 7f72da59c700 1 -- 172.21.15.47:0/4164385543 >> v2:172.21.15.64:3300/0 conn(0x7f72e40b4320 msgr2=0x7f72e4143230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:15.765 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.759+0000 7f72da59c700 1 --2- 172.21.15.47:0/4164385543 >> v2:172.21.15.64:3300/0 conn(0x7f72e40b4320 0x7f72e4143230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:15.766 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.759+0000 7f72da59c700 1 -- 172.21.15.47:0/4164385543 >> v1:172.21.15.47:6789/0 conn(0x7f72e4061220 legacy=0x7f72e4140070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:15.766 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.759+0000 7f72da59c700 1 -- 172.21.15.47:0/4164385543 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e4147100 con 0x7f72e4146720 2022-06-29T15:48:15.766 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.759+0000 7f72da59c700 1 --2- 172.21.15.47:0/4164385543 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f72e4146720 0x7f72e4140780 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f72cc002f70 tx=0x7f72cc006df0).ready entity=mon.2 client_cookie=9974e72d2e19849b server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:15.767 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.759+0000 7f72e259c700 1 -- 172.21.15.47:0/4164385543 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72cc00d4a0 con 0x7f72e4146720 2022-06-29T15:48:15.767 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.759+0000 7f72e259c700 1 -- 172.21.15.47:0/4164385543 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72cc000e90 con 0x7f72e4146720 2022-06-29T15:48:15.767 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.759+0000 7f72e259c700 1 -- 172.21.15.47:0/4164385543 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72cc00d7f0 con 0x7f72e4146720 2022-06-29T15:48:15.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.763+0000 7f72e9157700 1 -- 172.21.15.47:0/4164385543 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f72e4146720 msgr2=0x7f72e4140780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:15.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.763+0000 7f72e9157700 1 --2- 172.21.15.47:0/4164385543 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f72e4146720 0x7f72e4140780 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f72cc002f70 tx=0x7f72cc006df0).stop 2022-06-29T15:48:15.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.763+0000 7f72e9157700 1 -- 172.21.15.47:0/4164385543 shutdown_connections 2022-06-29T15:48:15.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.763+0000 7f72e9157700 1 --2- 172.21.15.47:0/4164385543 >> v2:172.21.15.64:3300/0 conn(0x7f72e40b4320 0x7f72e4143230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:15.772 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.763+0000 7f72e9157700 1 --2- 172.21.15.47:0/4164385543 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f72e4146720 0x7f72e4140780 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:15.772 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.763+0000 7f72e9157700 1 -- 172.21.15.47:0/4164385543 >> 172.21.15.47:0/4164385543 conn(0x7f72e405c6b0 msgr2=0x7f72e405f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:15.772 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.763+0000 7f72e9157700 1 -- 172.21.15.47:0/4164385543 shutdown_connections 2022-06-29T15:48:15.772 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.763+0000 7f72e9157700 1 -- 172.21.15.47:0/4164385543 wait complete. 2022-06-29T15:48:15.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.763+0000 7f72e9157700 1 Processor -- start 2022-06-29T15:48:15.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.763+0000 7f72e9157700 1 -- start start 2022-06-29T15:48:15.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.763+0000 7f72e9157700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72e4061220 0x7f72e40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:15.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.763+0000 7f72e9157700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f72e40b4320 0x7f72e40b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:15.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.763+0000 7f72e9157700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72e4146720 0x7f72e40b36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:15.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.763+0000 7f72e9157700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e40b78a0 con 0x7f72e4146720 2022-06-29T15:48:15.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.763+0000 7f72e9157700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e40b0db0 con 0x7f72e4061220 2022-06-29T15:48:15.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.763+0000 7f72e9157700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72e40b7a90 con 0x7f72e40b4320 2022-06-29T15:48:15.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.763+0000 7f72e359e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72e4146720 0x7f72e40b36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:15.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.763+0000 7f72e359e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72e4146720 0x7f72e40b36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49870/0 (socket says 172.21.15.47:49870) 2022-06-29T15:48:15.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.763+0000 7f72e359e700 1 -- 172.21.15.47:0/3642212655 learned_addr learned my addr 172.21.15.47:0/3642212655 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:15.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.763+0000 7f72da59c700 1 --2- 172.21.15.47:0/3642212655 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f72e40b4320 0x7f72e40b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:15.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.763+0000 7f72e2d9d700 1 --2- 172.21.15.47:0/3642212655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72e4061220 0x7f72e40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:15.776 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.763+0000 7f72e359e700 1 -- 172.21.15.47:0/3642212655 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f72e40b4320 msgr2=0x7f72e40b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:15.776 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.763+0000 7f72e359e700 1 --2- 172.21.15.47:0/3642212655 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f72e40b4320 0x7f72e40b31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:15.776 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.763+0000 7f72e359e700 1 -- 172.21.15.47:0/3642212655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72e4061220 msgr2=0x7f72e40b4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:15.776 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.763+0000 7f72e359e700 1 --2- 172.21.15.47:0/3642212655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72e4061220 0x7f72e40b4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:15.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.763+0000 7f72e359e700 1 -- 172.21.15.47:0/3642212655 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72cc00b180 con 0x7f72e4146720 2022-06-29T15:48:15.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.767+0000 7f72e2d9d700 1 --2- 172.21.15.47:0/3642212655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72e4061220 0x7f72e40b4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:48:15.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.767+0000 7f72e359e700 1 --2- 172.21.15.47:0/3642212655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72e4146720 0x7f72e40b36d0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 rx=0x7f72dc006e00 tx=0x7f72dc0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:15.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.767+0000 7f72e0d99700 1 -- 172.21.15.47:0/3642212655 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72dc00eeb0 con 0x7f72e4146720 2022-06-29T15:48:15.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.767+0000 7f72e0d99700 1 -- 172.21.15.47:0/3642212655 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72dc00c850 con 0x7f72e4146720 2022-06-29T15:48:15.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.767+0000 7f72e9157700 1 -- 172.21.15.47:0/3642212655 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72e40b7d30 con 0x7f72e4146720 2022-06-29T15:48:15.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.767+0000 7f72e0d99700 1 -- 172.21.15.47:0/3642212655 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72dc00cbb0 con 0x7f72e4146720 2022-06-29T15:48:15.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.767+0000 7f72e9157700 1 -- 172.21.15.47:0/3642212655 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72e40b5280 con 0x7f72e4146720 2022-06-29T15:48:15.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.767+0000 7f72e9157700 1 -- 172.21.15.47:0/3642212655 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72e40b8fa0 con 0x7f72e4146720 2022-06-29T15:48:15.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.767+0000 7f72e0d99700 1 -- 172.21.15.47:0/3642212655 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f72dc00cd10 con 0x7f72e4146720 2022-06-29T15:48:15.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.767+0000 7f72e0d99700 1 --2- 172.21.15.47:0/3642212655 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f72c8053a70 0x7f72c8055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:15.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.767+0000 7f72e0d99700 1 -- 172.21.15.47:0/3642212655 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f72dc06c0c0 con 0x7f72e4146720 2022-06-29T15:48:15.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.767+0000 7f72e2d9d700 1 --2- 172.21.15.47:0/3642212655 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f72c8053a70 0x7f72c8055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:15.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.771+0000 7f72e2d9d700 1 --2- 172.21.15.47:0/3642212655 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f72c8053a70 0x7f72c8055f30 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7f72d400b000 tx=0x7f72d4009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:15.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.775+0000 7f72e0d99700 1 -- 172.21.15.47:0/3642212655 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f72e40b8fa0 con 0x7f72e4146720 2022-06-29T15:48:15.902 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.899+0000 7f72e9157700 1 -- 172.21.15.47:0/3642212655 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f72e40b8fa0 con 0x7f72e4146720 2022-06-29T15:48:15.903 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.903+0000 7f72e0d99700 1 -- 172.21.15.47:0/3642212655 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v74) v1 ==== 110+0+10 (secure 0 0 0) 0x7f72e40b8fa0 con 0x7f72e4146720 2022-06-29T15:48:15.903 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2022-06-29T15:48:15.905 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.903+0000 7f72e9157700 1 -- 172.21.15.47:0/3642212655 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f72c8053a70 msgr2=0x7f72c8055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:15.905 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.903+0000 7f72e9157700 1 --2- 172.21.15.47:0/3642212655 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f72c8053a70 0x7f72c8055f30 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7f72d400b000 tx=0x7f72d4009040).stop 2022-06-29T15:48:15.906 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.903+0000 7f72e9157700 1 -- 172.21.15.47:0/3642212655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72e4146720 msgr2=0x7f72e40b36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:15.906 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.903+0000 7f72e9157700 1 --2- 172.21.15.47:0/3642212655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72e4146720 0x7f72e40b36d0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 rx=0x7f72dc006e00 tx=0x7f72dc0091d0).stop 2022-06-29T15:48:15.906 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.903+0000 7f72e9157700 1 -- 172.21.15.47:0/3642212655 shutdown_connections 2022-06-29T15:48:15.906 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.903+0000 7f72e9157700 1 --2- 172.21.15.47:0/3642212655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72e4146720 0x7f72e40b36d0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:15.907 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.903+0000 7f72e9157700 1 --2- 172.21.15.47:0/3642212655 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f72e40b4320 0x7f72e40b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:15.907 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.903+0000 7f72e9157700 1 --2- 172.21.15.47:0/3642212655 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f72c8053a70 0x7f72c8055f30 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:15.907 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.903+0000 7f72e9157700 1 --2- 172.21.15.47:0/3642212655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72e4061220 0x7f72e40b4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:15.907 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.903+0000 7f72e9157700 1 -- 172.21.15.47:0/3642212655 >> 172.21.15.47:0/3642212655 conn(0x7f72e405c6b0 msgr2=0x7f72e413eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:15.908 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.903+0000 7f72e9157700 1 -- 172.21.15.47:0/3642212655 shutdown_connections 2022-06-29T15:48:15.908 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:15.903+0000 7f72e9157700 1 -- 172.21.15.47:0/3642212655 wait complete. 2022-06-29T15:48:15.916 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-29T15:48:16.026 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.023+0000 7f9a80625700 1 Processor -- start 2022-06-29T15:48:16.027 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.023+0000 7f9a80625700 1 -- start start 2022-06-29T15:48:16.029 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.027+0000 7f9a80625700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9a780b2930 0x7f9a780b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:16.029 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.027+0000 7f9a80625700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9a78095e30 0x7f9a78098160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:16.029 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.027+0000 7f9a80625700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a780b1ac0 con 0x7f9a780a9770 2022-06-29T15:48:16.031 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.027+0000 7f9a80625700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a78095c70 con 0x7f9a780b2930 2022-06-29T15:48:16.031 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.027+0000 7f9a80625700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a780965b0 con 0x7f9a78095e30 2022-06-29T15:48:16.031 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.027+0000 7f9a7ebc2700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9a78095e30 0x7f9a78098160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:16.031 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.027+0000 7f9a7e3c1700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9a780b2930 0x7f9a780b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:16.032 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.027+0000 7f9a7ebc2700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9a78095e30 0x7f9a78098160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:46980/0 (socket says 172.21.15.47:46980) 2022-06-29T15:48:16.032 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.027+0000 7f9a7e3c1700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9a780b2930 0x7f9a780b2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:55578/0 (socket says 172.21.15.47:55578) 2022-06-29T15:48:16.032 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.027+0000 7f9a7ebc2700 1 -- 172.21.15.47:0/1890889903 learned_addr learned my addr 172.21.15.47:0/1890889903 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:16.032 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.027+0000 7f9a7dbc0700 1 --1- 172.21.15.47:0/1890889903 >> v1:172.21.15.64:6789/0 conn(0x7f9a780a9770 0x7f9a78097a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:42994/0 (socket says 172.21.15.47:42994) 2022-06-29T15:48:16.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.027+0000 7f9a7ebc2700 1 -- 172.21.15.47:0/1890889903 >> v2:172.21.15.64:3300/0 conn(0x7f9a780b2930 msgr2=0x7f9a780b2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:16.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.027+0000 7f9a7ebc2700 1 --2- 172.21.15.47:0/1890889903 >> v2:172.21.15.64:3300/0 conn(0x7f9a780b2930 0x7f9a780b2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:16.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.027+0000 7f9a7ebc2700 1 -- 172.21.15.47:0/1890889903 >> v1:172.21.15.64:6789/0 conn(0x7f9a780a9770 legacy=0x7f9a78097a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:16.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.027+0000 7f9a7ebc2700 1 -- 172.21.15.47:0/1890889903 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a780b1cf0 con 0x7f9a78095e30 2022-06-29T15:48:16.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.027+0000 7f9a7ebc2700 1 --2- 172.21.15.47:0/1890889903 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9a78095e30 0x7f9a78098160 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f9a700032f0 tx=0x7f9a70009290).ready entity=mon.2 client_cookie=41b16b98c75c3fc server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:16.038 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.027+0000 7f9a7d3bf700 1 -- 172.21.15.47:0/1890889903 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a7000b8a0 con 0x7f9a78095e30 2022-06-29T15:48:16.038 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.027+0000 7f9a7d3bf700 1 -- 172.21.15.47:0/1890889903 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a7000ba00 con 0x7f9a78095e30 2022-06-29T15:48:16.038 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.027+0000 7f9a7d3bf700 1 -- 172.21.15.47:0/1890889903 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a7000bb70 con 0x7f9a78095e30 2022-06-29T15:48:16.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.031+0000 7f9a80625700 1 -- 172.21.15.47:0/1890889903 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9a78095e30 msgr2=0x7f9a78098160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:16.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.031+0000 7f9a80625700 1 --2- 172.21.15.47:0/1890889903 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9a78095e30 0x7f9a78098160 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f9a700032f0 tx=0x7f9a70009290).stop 2022-06-29T15:48:16.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.031+0000 7f9a80625700 1 -- 172.21.15.47:0/1890889903 shutdown_connections 2022-06-29T15:48:16.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.031+0000 7f9a80625700 1 --2- 172.21.15.47:0/1890889903 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9a78095e30 0x7f9a78098160 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:16.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.031+0000 7f9a80625700 1 --2- 172.21.15.47:0/1890889903 >> v2:172.21.15.64:3300/0 conn(0x7f9a780b2930 0x7f9a780b2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:16.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.031+0000 7f9a80625700 1 -- 172.21.15.47:0/1890889903 >> 172.21.15.47:0/1890889903 conn(0x7f9a7805f0b0 msgr2=0x7f9a7805f490 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:16.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.031+0000 7f9a80625700 1 -- 172.21.15.47:0/1890889903 shutdown_connections 2022-06-29T15:48:16.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.031+0000 7f9a80625700 1 -- 172.21.15.47:0/1890889903 wait complete. 2022-06-29T15:48:16.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.031+0000 7f9a80625700 1 Processor -- start 2022-06-29T15:48:16.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.031+0000 7f9a80625700 1 -- start start 2022-06-29T15:48:16.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.035+0000 7f9a80625700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a78095e30 0x7f9a780ae5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:16.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.035+0000 7f9a80625700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a780a9770 0x7f9a780ac150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:16.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.035+0000 7f9a80625700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9a780b2930 0x7f9a780ac640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:16.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.035+0000 7f9a80625700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a780acb30 con 0x7f9a78095e30 2022-06-29T15:48:16.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.035+0000 7f9a80625700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a781407e0 con 0x7f9a780a9770 2022-06-29T15:48:16.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.035+0000 7f9a80625700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a78154270 con 0x7f9a780b2930 2022-06-29T15:48:16.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.035+0000 7f9a7e3c1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a78095e30 0x7f9a780ae5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:16.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.035+0000 7f9a7e3c1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a78095e30 0x7f9a780ae5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49880/0 (socket says 172.21.15.47:49880) 2022-06-29T15:48:16.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.035+0000 7f9a7e3c1700 1 -- 172.21.15.47:0/4038213044 learned_addr learned my addr 172.21.15.47:0/4038213044 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:16.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.035+0000 7f9a7ebc2700 1 --2- 172.21.15.47:0/4038213044 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9a780b2930 0x7f9a780ac640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:16.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.035+0000 7f9a7dbc0700 1 --2- 172.21.15.47:0/4038213044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a780a9770 0x7f9a780ac150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:16.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.035+0000 7f9a7e3c1700 1 -- 172.21.15.47:0/4038213044 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9a780b2930 msgr2=0x7f9a780ac640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:16.045 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.035+0000 7f9a7e3c1700 1 --2- 172.21.15.47:0/4038213044 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9a780b2930 0x7f9a780ac640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:16.045 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.035+0000 7f9a7e3c1700 1 -- 172.21.15.47:0/4038213044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a780a9770 msgr2=0x7f9a780ac150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:16.045 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.035+0000 7f9a7e3c1700 1 --2- 172.21.15.47:0/4038213044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a780a9770 0x7f9a780ac150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:16.045 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.035+0000 7f9a7e3c1700 1 -- 172.21.15.47:0/4038213044 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a70009530 con 0x7f9a78095e30 2022-06-29T15:48:16.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.035+0000 7f9a7e3c1700 1 --2- 172.21.15.47:0/4038213044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a78095e30 0x7f9a780ae5c0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7f9a7400d000 tx=0x7f9a74008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:16.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.035+0000 7f9a677fe700 1 -- 172.21.15.47:0/4038213044 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a74010820 con 0x7f9a78095e30 2022-06-29T15:48:16.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.035+0000 7f9a677fe700 1 -- 172.21.15.47:0/4038213044 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a74010980 con 0x7f9a78095e30 2022-06-29T15:48:16.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.035+0000 7f9a677fe700 1 -- 172.21.15.47:0/4038213044 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a74010ce0 con 0x7f9a78095e30 2022-06-29T15:48:16.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.035+0000 7f9a80625700 1 -- 172.21.15.47:0/4038213044 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a78154510 con 0x7f9a78095e30 2022-06-29T15:48:16.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.035+0000 7f9a80625700 1 -- 172.21.15.47:0/4038213044 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a78150190 con 0x7f9a78095e30 2022-06-29T15:48:16.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.039+0000 7f9a80625700 1 -- 172.21.15.47:0/4038213044 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a781409e0 con 0x7f9a78095e30 2022-06-29T15:48:16.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.039+0000 7f9a677fe700 1 -- 172.21.15.47:0/4038213044 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9a74017030 con 0x7f9a78095e30 2022-06-29T15:48:16.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.039+0000 7f9a677fe700 1 --2- 172.21.15.47:0/4038213044 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9a580539e0 0x7f9a58055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:16.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.039+0000 7f9a677fe700 1 -- 172.21.15.47:0/4038213044 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f9a7406bdd0 con 0x7f9a78095e30 2022-06-29T15:48:16.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.039+0000 7f9a7dbc0700 1 --2- 172.21.15.47:0/4038213044 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9a580539e0 0x7f9a58055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:16.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.039+0000 7f9a7dbc0700 1 --2- 172.21.15.47:0/4038213044 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9a580539e0 0x7f9a58055ea0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f9a6800d000 tx=0x7f9a68007b80).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:16.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.047+0000 7f9a677fe700 1 -- 172.21.15.47:0/4038213044 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9a74042dc0 con 0x7f9a78095e30 2022-06-29T15:48:16.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.175+0000 7f9a80625700 1 -- 172.21.15.47:0/4038213044 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9a781540b0 con 0x7f9a78095e30 2022-06-29T15:48:16.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.175+0000 7f9a677fe700 1 -- 172.21.15.47:0/4038213044 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v74) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9a740463e0 con 0x7f9a78095e30 2022-06-29T15:48:16.176 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:48:16.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.175+0000 7f9a80625700 1 -- 172.21.15.47:0/4038213044 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9a580539e0 msgr2=0x7f9a58055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:16.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.175+0000 7f9a80625700 1 --2- 172.21.15.47:0/4038213044 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9a580539e0 0x7f9a58055ea0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f9a6800d000 tx=0x7f9a68007b80).stop 2022-06-29T15:48:16.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.175+0000 7f9a80625700 1 -- 172.21.15.47:0/4038213044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a78095e30 msgr2=0x7f9a780ae5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:16.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.175+0000 7f9a80625700 1 --2- 172.21.15.47:0/4038213044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a78095e30 0x7f9a780ae5c0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7f9a7400d000 tx=0x7f9a74008040).stop 2022-06-29T15:48:16.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.175+0000 7f9a80625700 1 -- 172.21.15.47:0/4038213044 shutdown_connections 2022-06-29T15:48:16.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.175+0000 7f9a80625700 1 --2- 172.21.15.47:0/4038213044 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9a780b2930 0x7f9a780ac640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:16.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.175+0000 7f9a80625700 1 --2- 172.21.15.47:0/4038213044 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9a580539e0 0x7f9a58055ea0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:16.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.175+0000 7f9a80625700 1 --2- 172.21.15.47:0/4038213044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a780a9770 0x7f9a780ac150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:16.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.175+0000 7f9a80625700 1 --2- 172.21.15.47:0/4038213044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a78095e30 0x7f9a780ae5c0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:16.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.175+0000 7f9a80625700 1 -- 172.21.15.47:0/4038213044 >> 172.21.15.47:0/4038213044 conn(0x7f9a7805f0b0 msgr2=0x7f9a78144440 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:16.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.175+0000 7f9a80625700 1 -- 172.21.15.47:0/4038213044 shutdown_connections 2022-06-29T15:48:16.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.175+0000 7f9a80625700 1 -- 172.21.15.47:0/4038213044 wait complete. 2022-06-29T15:48:16.194 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-06-29T15:48:16.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.283+0000 7f1beb59e700 1 Processor -- start 2022-06-29T15:48:16.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.283+0000 7f1beb59e700 1 -- start start 2022-06-29T15:48:16.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.283+0000 7f1beb59e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1bec0b5c60 0x7f1bec144980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:16.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.283+0000 7f1beb59e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1bec0a72f0 0x7f1bec0b2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:16.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.283+0000 7f1beb59e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bec0a9120 con 0x7f1bec060eb0 2022-06-29T15:48:16.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.283+0000 7f1beb59e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bec0b0b80 con 0x7f1bec0b5c60 2022-06-29T15:48:16.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.283+0000 7f1beb59e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec0a89e0 con 0x7f1bec0a72f0 2022-06-29T15:48:16.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.283+0000 7f1be9d9b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1bec060eb0 0x7f1bec0b1e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35442/0 (socket says 172.21.15.47:35442) 2022-06-29T15:48:16.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.283+0000 7f1be9d9b700 1 -- 172.21.15.47:0/1340371849 learned_addr learned my addr 172.21.15.47:0/1340371849 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:48:16.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.283+0000 7f1bea59c700 1 --2- 172.21.15.47:0/1340371849 >> v2:172.21.15.47:3300/0 conn(0x7f1bec0b5c60 0x7f1bec144980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:16.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.283+0000 7f1bead9d700 1 --2- 172.21.15.47:0/1340371849 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1bec0a72f0 0x7f1bec0b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:16.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.283+0000 7f1be959a700 1 -- 172.21.15.47:0/1340371849 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4212845369 0 0) 0x7f1bec0a9120 con 0x7f1bec060eb0 2022-06-29T15:48:16.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.283+0000 7f1be959a700 1 -- 172.21.15.47:0/1340371849 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1bd0003260 con 0x7f1bec060eb0 2022-06-29T15:48:16.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.283+0000 7f1be959a700 1 -- 172.21.15.47:0/1340371849 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2288735545 0 0) 0x7f1bd0003260 con 0x7f1bec060eb0 2022-06-29T15:48:16.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.283+0000 7f1be959a700 1 -- 172.21.15.47:0/1340371849 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1bec0a72f0 msgr2=0x7f1bec0b2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:16.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.283+0000 7f1be959a700 1 --2- 172.21.15.47:0/1340371849 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1bec0a72f0 0x7f1bec0b2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:16.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.283+0000 7f1be959a700 1 -- 172.21.15.47:0/1340371849 >> v2:172.21.15.47:3300/0 conn(0x7f1bec0b5c60 msgr2=0x7f1bec144980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:16.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.283+0000 7f1be959a700 1 --2- 172.21.15.47:0/1340371849 >> v2:172.21.15.47:3300/0 conn(0x7f1bec0b5c60 0x7f1bec144980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:16.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.283+0000 7f1be959a700 1 -- 172.21.15.47:0/1340371849 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bec0a8c80 con 0x7f1bec060eb0 2022-06-29T15:48:16.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.283+0000 7f1be959a700 1 -- 172.21.15.47:0/1340371849 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7f1bdc0019e0 con 0x7f1bec060eb0 2022-06-29T15:48:16.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.283+0000 7f1be959a700 1 -- 172.21.15.47:0/1340371849 >> v1:172.21.15.47:6789/0 conn(0x7f1bec060eb0 legacy=0x7f1bec0b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:16.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.283+0000 7f1be959a700 1 --2- 172.21.15.47:0/1340371849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1bd0003750 0x7f1bd0003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:16.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.283+0000 7f1be959a700 1 --2- 172.21.15.47:0/1340371849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bd0004500 0x7f1bd0004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:16.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.283+0000 7f1be959a700 1 --2- 172.21.15.47:0/1340371849 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1bec0a72f0 0x7f1bd0004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:16.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.283+0000 7f1be959a700 1 -- 172.21.15.47:0/1340371849 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec0b0b80 con 0x7f1bd0004500 2022-06-29T15:48:16.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.283+0000 7f1be959a700 1 -- 172.21.15.47:0/1340371849 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec0a89e0 con 0x7f1bd0003750 2022-06-29T15:48:16.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.283+0000 7f1be959a700 1 -- 172.21.15.47:0/1340371849 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd000b750 con 0x7f1bec0a72f0 2022-06-29T15:48:16.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.283+0000 7f1bea59c700 1 --2- 172.21.15.47:0/1340371849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1bd0003750 0x7f1bd0003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:16.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.283+0000 7f1be9d9b700 1 --2- 172.21.15.47:0/1340371849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bd0004500 0x7f1bd0004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:16.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.283+0000 7f1bead9d700 1 --2- 172.21.15.47:0/1340371849 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1bec0a72f0 0x7f1bd0004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:16.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.287+0000 7f1be9d9b700 1 -- 172.21.15.47:0/1340371849 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1bec0a72f0 msgr2=0x7f1bd0004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:16.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.287+0000 7f1be9d9b700 1 --2- 172.21.15.47:0/1340371849 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1bec0a72f0 0x7f1bd0004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:16.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.287+0000 7f1be9d9b700 1 -- 172.21.15.47:0/1340371849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1bd0003750 msgr2=0x7f1bd0003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:16.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.287+0000 7f1be9d9b700 1 --2- 172.21.15.47:0/1340371849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1bd0003750 0x7f1bd0003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:16.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.287+0000 7f1be9d9b700 1 -- 172.21.15.47:0/1340371849 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bd0003260 con 0x7f1bd0004500 2022-06-29T15:48:16.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.287+0000 7f1be9d9b700 1 --2- 172.21.15.47:0/1340371849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bd0004500 0x7f1bd0004980 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7f1bdc0016f0 tx=0x7f1bdc0021d0).ready entity=mon.0 client_cookie=9a25d19ae9335177 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:16.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.287+0000 7f1be959a700 1 -- 172.21.15.47:0/1340371849 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1bdc00cc80 con 0x7f1bd0004500 2022-06-29T15:48:16.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.287+0000 7f1be959a700 1 -- 172.21.15.47:0/1340371849 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bdc00cde0 con 0x7f1bd0004500 2022-06-29T15:48:16.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.287+0000 7f1be959a700 1 -- 172.21.15.47:0/1340371849 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1bdc005b80 con 0x7f1bd0004500 2022-06-29T15:48:16.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.287+0000 7f1beb59e700 1 -- 172.21.15.47:0/1340371849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bd0004500 msgr2=0x7f1bd0004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:16.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.287+0000 7f1beb59e700 1 --2- 172.21.15.47:0/1340371849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bd0004500 0x7f1bd0004980 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7f1bdc0016f0 tx=0x7f1bdc0021d0).stop 2022-06-29T15:48:16.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.287+0000 7f1bead9d700 1 -- 172.21.15.47:0/1340371849 reap_dead start 2022-06-29T15:48:16.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.287+0000 7f1beb59e700 1 -- 172.21.15.47:0/1340371849 shutdown_connections 2022-06-29T15:48:16.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.287+0000 7f1beb59e700 1 -- 172.21.15.47:0/1340371849 >> 172.21.15.47:0/1340371849 conn(0x7f1bec062560 msgr2=0x7f1bec062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:16.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.287+0000 7f1beb59e700 1 -- 172.21.15.47:0/1340371849 shutdown_connections 2022-06-29T15:48:16.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.287+0000 7f1beb59e700 1 -- 172.21.15.47:0/1340371849 wait complete. 2022-06-29T15:48:16.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.287+0000 7f1beb59e700 1 Processor -- start 2022-06-29T15:48:16.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.287+0000 7f1beb59e700 1 -- start start 2022-06-29T15:48:16.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.291+0000 7f1beb59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1bd0004500 0x7f1bec1454a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:16.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.291+0000 7f1beb59e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1bec145990 0x7f1bec13e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:16.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.291+0000 7f1beb59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bec13e5d0 0x7f1bec13ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:16.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.291+0000 7f1beb59e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec1574e0 con 0x7f1bec13e5d0 2022-06-29T15:48:16.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.291+0000 7f1beb59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec143740 con 0x7f1bd0004500 2022-06-29T15:48:16.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.291+0000 7f1beb59e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec1439e0 con 0x7f1bec145990 2022-06-29T15:48:16.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.291+0000 7f1bead9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bec13e5d0 0x7f1bec13ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:16.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.291+0000 7f1bead9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bec13e5d0 0x7f1bec13ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49904/0 (socket says 172.21.15.47:49904) 2022-06-29T15:48:16.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.291+0000 7f1bead9d700 1 -- 172.21.15.47:0/2676458084 learned_addr learned my addr 172.21.15.47:0/2676458084 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:16.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.291+0000 7f1be9d9b700 1 --2- 172.21.15.47:0/2676458084 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1bec145990 0x7f1bec13e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:16.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.291+0000 7f1bea59c700 1 --2- 172.21.15.47:0/2676458084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1bd0004500 0x7f1bec1454a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:16.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.291+0000 7f1bead9d700 1 -- 172.21.15.47:0/2676458084 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1bec145990 msgr2=0x7f1bec13e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:16.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.291+0000 7f1bead9d700 1 --2- 172.21.15.47:0/2676458084 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1bec145990 0x7f1bec13e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:16.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.291+0000 7f1bead9d700 1 -- 172.21.15.47:0/2676458084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1bd0004500 msgr2=0x7f1bec1454a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:16.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.291+0000 7f1bead9d700 1 --2- 172.21.15.47:0/2676458084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1bd0004500 0x7f1bec1454a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:16.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.291+0000 7f1bead9d700 1 -- 172.21.15.47:0/2676458084 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bec143c80 con 0x7f1bec13e5d0 2022-06-29T15:48:16.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.291+0000 7f1bead9d700 1 --2- 172.21.15.47:0/2676458084 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bec13e5d0 0x7f1bec13ff10 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7f1be400b000 tx=0x7f1be4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:16.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.291+0000 7f1bdb7fe700 1 -- 172.21.15.47:0/2676458084 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1be4013680 con 0x7f1bec13e5d0 2022-06-29T15:48:16.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.291+0000 7f1bdb7fe700 1 -- 172.21.15.47:0/2676458084 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1be40137e0 con 0x7f1bec13e5d0 2022-06-29T15:48:16.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.291+0000 7f1beb59e700 1 -- 172.21.15.47:0/2676458084 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bec144120 con 0x7f1bec13e5d0 2022-06-29T15:48:16.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.291+0000 7f1bdb7fe700 1 -- 172.21.15.47:0/2676458084 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1be4013b40 con 0x7f1bec13e5d0 2022-06-29T15:48:16.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.291+0000 7f1beb59e700 1 -- 172.21.15.47:0/2676458084 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bec156fd0 con 0x7f1bec13e5d0 2022-06-29T15:48:16.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.291+0000 7f1beb59e700 1 -- 172.21.15.47:0/2676458084 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bec143e10 con 0x7f1bec13e5d0 2022-06-29T15:48:16.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.291+0000 7f1bdb7fe700 1 -- 172.21.15.47:0/2676458084 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1be4013ca0 con 0x7f1bec13e5d0 2022-06-29T15:48:16.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.295+0000 7f1bdb7fe700 1 --2- 172.21.15.47:0/2676458084 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1bc80539e0 0x7f1bc8055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:16.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.295+0000 7f1bea59c700 1 --2- 172.21.15.47:0/2676458084 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1bc80539e0 0x7f1bc8055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:16.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.295+0000 7f1bdb7fe700 1 -- 172.21.15.47:0/2676458084 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f1be406c410 con 0x7f1bec13e5d0 2022-06-29T15:48:16.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.295+0000 7f1bea59c700 1 --2- 172.21.15.47:0/2676458084 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1bc80539e0 0x7f1bc8055ea0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7f1be0006000 tx=0x7f1be0006d90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:16.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.299+0000 7f1bdb7fe700 1 -- 172.21.15.47:0/2676458084 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1be40434b0 con 0x7f1bec13e5d0 2022-06-29T15:48:16.429 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.427+0000 7f1beb59e700 1 -- 172.21.15.47:0/2676458084 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f1bec0573d0 con 0x7f1bec13e5d0 2022-06-29T15:48:16.430 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.427+0000 7f1bdb7fe700 1 -- 172.21.15.47:0/2676458084 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v74) v1 ==== 84+0+143 (secure 0 0 0) 0x7f1be4046ad0 con 0x7f1bec13e5d0 2022-06-29T15:48:16.430 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:48:16.430 INFO:teuthology.orchestra.run.smithi047.stdout:[ 2022-06-29T15:48:16.431 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:16.431 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 1, 2022-06-29T15:48:16.431 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": "device_health_metrics" 2022-06-29T15:48:16.431 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:48:16.431 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:48:16.432 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 2, 2022-06-29T15:48:16.432 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": "rbd" 2022-06-29T15:48:16.432 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:48:16.432 INFO:teuthology.orchestra.run.smithi047.stdout:] 2022-06-29T15:48:16.434 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.431+0000 7f1beb59e700 1 -- 172.21.15.47:0/2676458084 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1bc80539e0 msgr2=0x7f1bc8055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:16.434 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.431+0000 7f1beb59e700 1 --2- 172.21.15.47:0/2676458084 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1bc80539e0 0x7f1bc8055ea0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7f1be0006000 tx=0x7f1be0006d90).stop 2022-06-29T15:48:16.435 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.431+0000 7f1beb59e700 1 -- 172.21.15.47:0/2676458084 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bec13e5d0 msgr2=0x7f1bec13ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:16.435 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.431+0000 7f1beb59e700 1 --2- 172.21.15.47:0/2676458084 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bec13e5d0 0x7f1bec13ff10 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7f1be400b000 tx=0x7f1be4006df0).stop 2022-06-29T15:48:16.435 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.431+0000 7f1beb59e700 1 -- 172.21.15.47:0/2676458084 shutdown_connections 2022-06-29T15:48:16.435 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.431+0000 7f1beb59e700 1 --2- 172.21.15.47:0/2676458084 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bec13e5d0 0x7f1bec13ff10 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:16.436 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.431+0000 7f1beb59e700 1 --2- 172.21.15.47:0/2676458084 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1bec145990 0x7f1bec13e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:16.436 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.431+0000 7f1beb59e700 1 --2- 172.21.15.47:0/2676458084 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1bc80539e0 0x7f1bc8055ea0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:16.436 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.431+0000 7f1beb59e700 1 --2- 172.21.15.47:0/2676458084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1bd0004500 0x7f1bec1454a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:16.436 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.431+0000 7f1beb59e700 1 -- 172.21.15.47:0/2676458084 >> 172.21.15.47:0/2676458084 conn(0x7f1bec062560 msgr2=0x7f1bec0601a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:16.437 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.431+0000 7f1beb59e700 1 -- 172.21.15.47:0/2676458084 shutdown_connections 2022-06-29T15:48:16.437 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.431+0000 7f1beb59e700 1 -- 172.21.15.47:0/2676458084 wait complete. 2022-06-29T15:48:16.444 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-06-29T15:48:16.445 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-06-29T15:48:16.550 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.547+0000 7f6141b74700 1 Processor -- start 2022-06-29T15:48:16.550 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.547+0000 7f6141b74700 1 -- start start 2022-06-29T15:48:16.555 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.551+0000 7f6141b74700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f613c05c850 0x7f613c143a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:16.555 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.551+0000 7f6141b74700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f613c0676a0 0x7f613c140130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:16.555 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.551+0000 7f6141b74700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f613c05cc30 con 0x7f613c0b5d90 2022-06-29T15:48:16.556 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.551+0000 7f6141b74700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f613c1430a0 con 0x7f613c0676a0 2022-06-29T15:48:16.556 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.551+0000 7f6141b74700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f613c143300 con 0x7f613c05c850 2022-06-29T15:48:16.556 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.551+0000 7f6141373700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f613c0676a0 0x7f613c140130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:16.556 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.551+0000 7f613bfff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f613c0b5d90 0x7f613c0b4860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35464/0 (socket says 172.21.15.47:35464) 2022-06-29T15:48:16.557 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.551+0000 7f613bfff700 1 -- 172.21.15.47:0/430510291 learned_addr learned my addr 172.21.15.47:0/430510291 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:48:16.557 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.551+0000 7f6140b72700 1 --2- 172.21.15.47:0/430510291 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f613c05c850 0x7f613c143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:16.557 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.551+0000 7f613b7fe700 1 -- 172.21.15.47:0/430510291 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1087327490 0 0) 0x7f613c05cc30 con 0x7f613c0b5d90 2022-06-29T15:48:16.557 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.551+0000 7f613b7fe700 1 -- 172.21.15.47:0/430510291 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f611c003260 con 0x7f613c0b5d90 2022-06-29T15:48:16.557 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.551+0000 7f6141373700 1 -- 172.21.15.47:0/430510291 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f613c05c850 msgr2=0x7f613c143a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:16.558 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.551+0000 7f6141373700 1 --2- 172.21.15.47:0/430510291 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f613c05c850 0x7f613c143a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:16.558 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.551+0000 7f6141373700 1 -- 172.21.15.47:0/430510291 >> v1:172.21.15.47:6789/0 conn(0x7f613c0b5d90 legacy=0x7f613c0b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:16.558 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.551+0000 7f613b7fe700 1 -- 172.21.15.47:0/430510291 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 957109261 0 0) 0x7f611c003260 con 0x7f613c0b5d90 2022-06-29T15:48:16.558 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.551+0000 7f6141373700 1 -- 172.21.15.47:0/430510291 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f613c1435a0 con 0x7f613c0676a0 2022-06-29T15:48:16.559 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.551+0000 7f6140b72700 1 --2- 172.21.15.47:0/430510291 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f613c05c850 0x7f613c143a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:48:16.559 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.551+0000 7f6141373700 1 --2- 172.21.15.47:0/430510291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f613c0676a0 0x7f613c140130 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7f6124001820 tx=0x7f61340091d0).ready entity=mon.0 client_cookie=404838d409f631d1 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:16.559 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.551+0000 7f613b7fe700 1 -- 172.21.15.47:0/430510291 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6124001680 con 0x7f613c0676a0 2022-06-29T15:48:16.559 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.551+0000 7f613b7fe700 1 -- 172.21.15.47:0/430510291 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f613400f510 con 0x7f613c0676a0 2022-06-29T15:48:16.560 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.551+0000 7f613b7fe700 1 -- 172.21.15.47:0/430510291 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f613400f680 con 0x7f613c0676a0 2022-06-29T15:48:16.560 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.551+0000 7f6141b74700 1 -- 172.21.15.47:0/430510291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f613c0676a0 msgr2=0x7f613c140130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:16.560 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.551+0000 7f6141b74700 1 --2- 172.21.15.47:0/430510291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f613c0676a0 0x7f613c140130 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7f6124001820 tx=0x7f61340091d0).stop 2022-06-29T15:48:16.560 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.551+0000 7f6141b74700 1 -- 172.21.15.47:0/430510291 shutdown_connections 2022-06-29T15:48:16.560 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.551+0000 7f6141b74700 1 --2- 172.21.15.47:0/430510291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f613c0676a0 0x7f613c140130 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:16.561 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.551+0000 7f6141b74700 1 --2- 172.21.15.47:0/430510291 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f613c05c850 0x7f613c143a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:16.561 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.551+0000 7f6141b74700 1 -- 172.21.15.47:0/430510291 >> 172.21.15.47:0/430510291 conn(0x7f613c060e00 msgr2=0x7f613c0611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:16.561 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.551+0000 7f6141b74700 1 -- 172.21.15.47:0/430510291 shutdown_connections 2022-06-29T15:48:16.561 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.551+0000 7f6141b74700 1 -- 172.21.15.47:0/430510291 wait complete. 2022-06-29T15:48:16.564 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.555+0000 7f6141b74700 1 Processor -- start 2022-06-29T15:48:16.565 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.555+0000 7f6141b74700 1 -- start start 2022-06-29T15:48:16.565 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.555+0000 7f6141b74700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f613c05c850 0x7f613c12e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:16.565 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.555+0000 7f6141b74700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f613c0676a0 0x7f613c13f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:16.565 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.555+0000 7f6141b74700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f613c0b5d90 0x7f613c13f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:16.566 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.555+0000 7f6141b74700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613c0b2d20 con 0x7f613c0b5d90 2022-06-29T15:48:16.566 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.555+0000 7f6141b74700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613c140d00 con 0x7f613c0676a0 2022-06-29T15:48:16.566 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.555+0000 7f6141b74700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f613c0b23a0 con 0x7f613c05c850 2022-06-29T15:48:16.567 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.555+0000 7f6141373700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f613c0b5d90 0x7f613c13f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:16.567 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.555+0000 7f6141373700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f613c0b5d90 0x7f613c13f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49918/0 (socket says 172.21.15.47:49918) 2022-06-29T15:48:16.567 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.555+0000 7f6141373700 1 -- 172.21.15.47:0/2875232815 learned_addr learned my addr 172.21.15.47:0/2875232815 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:16.567 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.559+0000 7f613bfff700 1 --2- 172.21.15.47:0/2875232815 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f613c0676a0 0x7f613c13f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:16.568 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.559+0000 7f6140b72700 1 --2- 172.21.15.47:0/2875232815 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f613c05c850 0x7f613c12e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:16.568 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.559+0000 7f6141373700 1 -- 172.21.15.47:0/2875232815 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f613c05c850 msgr2=0x7f613c12e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:16.568 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.559+0000 7f6141373700 1 --2- 172.21.15.47:0/2875232815 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f613c05c850 0x7f613c12e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:16.568 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.559+0000 7f6141373700 1 -- 172.21.15.47:0/2875232815 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f613c0676a0 msgr2=0x7f613c13f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:16.569 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.559+0000 7f6141373700 1 --2- 172.21.15.47:0/2875232815 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f613c0676a0 0x7f613c13f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:16.569 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.559+0000 7f6141373700 1 -- 172.21.15.47:0/2875232815 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f612c00b120 con 0x7f613c0b5d90 2022-06-29T15:48:16.569 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.559+0000 7f613bfff700 1 --2- 172.21.15.47:0/2875232815 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f613c0676a0 0x7f613c13f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:48:16.569 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.559+0000 7f6141373700 1 --2- 172.21.15.47:0/2875232815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f613c0b5d90 0x7f613c13f6e0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7f6124001820 tx=0x7f6134016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:16.570 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.559+0000 7f6139ffb700 1 -- 172.21.15.47:0/2875232815 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6134006b60 con 0x7f613c0b5d90 2022-06-29T15:48:16.570 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.559+0000 7f6139ffb700 1 -- 172.21.15.47:0/2875232815 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61340032d0 con 0x7f613c0b5d90 2022-06-29T15:48:16.570 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.559+0000 7f6141b74700 1 -- 172.21.15.47:0/2875232815 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6134009470 con 0x7f613c0b5d90 2022-06-29T15:48:16.570 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.559+0000 7f6139ffb700 1 -- 172.21.15.47:0/2875232815 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6134021440 con 0x7f613c0b5d90 2022-06-29T15:48:16.570 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.559+0000 7f6141b74700 1 -- 172.21.15.47:0/2875232815 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f613c0b2f50 con 0x7f613c0b5d90 2022-06-29T15:48:16.571 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.559+0000 7f6141b74700 1 -- 172.21.15.47:0/2875232815 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f613c0b3150 con 0x7f613c0b5d90 2022-06-29T15:48:16.571 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.559+0000 7f6139ffb700 1 -- 172.21.15.47:0/2875232815 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6134027020 con 0x7f613c0b5d90 2022-06-29T15:48:16.571 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.559+0000 7f6139ffb700 1 --2- 172.21.15.47:0/2875232815 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6120053a70 0x7f6120055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:16.571 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.559+0000 7f6139ffb700 1 -- 172.21.15.47:0/2875232815 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f613406cce0 con 0x7f613c0b5d90 2022-06-29T15:48:16.572 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.559+0000 7f6140b72700 1 --2- 172.21.15.47:0/2875232815 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6120053a70 0x7f6120055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:16.572 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.563+0000 7f6140b72700 1 --2- 172.21.15.47:0/2875232815 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6120053a70 0x7f6120055f30 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7f612c008000 tx=0x7f612c008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:16.572 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.567+0000 7f6139ffb700 1 -- 172.21.15.47:0/2875232815 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f613c0b3150 con 0x7f613c0b5d90 2022-06-29T15:48:16.700 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:16.699+0000 7f6141b74700 1 -- 172.21.15.47:0/2875232815 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f613c0b3150 con 0x7f613c0b5d90 2022-06-29T15:48:17.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.598+0000 7f6139ffb700 1 -- 172.21.15.47:0/2875232815 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v75) v1 ==== 133+0+0 (secure 0 0 0) 0x7f613c0b3150 con 0x7f613c0b5d90 2022-06-29T15:48:17.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.602+0000 7f6141b74700 1 -- 172.21.15.47:0/2875232815 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6120053a70 msgr2=0x7f6120055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:17.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.602+0000 7f6141b74700 1 --2- 172.21.15.47:0/2875232815 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6120053a70 0x7f6120055f30 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7f612c008000 tx=0x7f612c008040).stop 2022-06-29T15:48:17.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.602+0000 7f6141b74700 1 -- 172.21.15.47:0/2875232815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f613c0b5d90 msgr2=0x7f613c13f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:17.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.602+0000 7f6141b74700 1 --2- 172.21.15.47:0/2875232815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f613c0b5d90 0x7f613c13f6e0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7f6124001820 tx=0x7f6134016000).stop 2022-06-29T15:48:17.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.602+0000 7f6141b74700 1 -- 172.21.15.47:0/2875232815 shutdown_connections 2022-06-29T15:48:17.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.602+0000 7f6141b74700 1 --2- 172.21.15.47:0/2875232815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f613c0b5d90 0x7f613c13f6e0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:17.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.602+0000 7f6141b74700 1 --2- 172.21.15.47:0/2875232815 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6120053a70 0x7f6120055f30 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:17.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.602+0000 7f6141b74700 1 --2- 172.21.15.47:0/2875232815 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f613c0676a0 0x7f613c13f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:17.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.602+0000 7f6141b74700 1 --2- 172.21.15.47:0/2875232815 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f613c05c850 0x7f613c12e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:17.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.602+0000 7f6141b74700 1 -- 172.21.15.47:0/2875232815 >> 172.21.15.47:0/2875232815 conn(0x7f613c060e00 msgr2=0x7f613c143960 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:17.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.602+0000 7f6141b74700 1 -- 172.21.15.47:0/2875232815 shutdown_connections 2022-06-29T15:48:17.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.602+0000 7f6141b74700 1 -- 172.21.15.47:0/2875232815 wait complete. 2022-06-29T15:48:17.612 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' created 2022-06-29T15:48:17.621 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-06-29T15:48:17.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.730+0000 7fc285738700 1 Processor -- start 2022-06-29T15:48:17.734 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.730+0000 7fc285738700 1 -- start start 2022-06-29T15:48:17.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.730+0000 7fc285738700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc2800b1610 0x7fc2800b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:17.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.730+0000 7fc285738700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc28014b4c0 0x7fc2800aa130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:17.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.730+0000 7fc285738700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc280060790 con 0x7fc2800b58d0 2022-06-29T15:48:17.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.730+0000 7fc285738700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2800a99e0 con 0x7fc28014b4c0 2022-06-29T15:48:17.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.730+0000 7fc285738700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc28014ad80 con 0x7fc2800b1610 2022-06-29T15:48:17.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.730+0000 7fc284f37700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc28014b4c0 0x7fc2800aa130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:17.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.730+0000 7fc277fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc2800b1610 0x7fc2800b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:17.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.730+0000 7fc27ffff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fc2800b58d0 0x7fc2801475f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:43040/0 (socket says 172.21.15.47:43040) 2022-06-29T15:48:17.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.730+0000 7fc284f37700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc28014b4c0 0x7fc2800aa130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:55630/0 (socket says 172.21.15.47:55630) 2022-06-29T15:48:17.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.730+0000 7fc27ffff700 1 -- 172.21.15.47:0/878089088 learned_addr learned my addr 172.21.15.47:0/878089088 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:48:17.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.730+0000 7fc277fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc2800b1610 0x7fc2800b5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:47026/0 (socket says 172.21.15.47:47026) 2022-06-29T15:48:17.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.734+0000 7fc27f7fe700 1 -- 172.21.15.47:0/878089088 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2686920240 0 0) 0x7fc280060790 con 0x7fc2800b58d0 2022-06-29T15:48:17.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.734+0000 7fc27f7fe700 1 -- 172.21.15.47:0/878089088 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc260003260 con 0x7fc2800b58d0 2022-06-29T15:48:17.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.734+0000 7fc284f37700 1 -- 172.21.15.47:0/878089088 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc2800b1610 msgr2=0x7fc2800b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:17.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.734+0000 7fc284f37700 1 --2- 172.21.15.47:0/878089088 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc2800b1610 0x7fc2800b5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:17.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.734+0000 7fc284f37700 1 -- 172.21.15.47:0/878089088 >> v1:172.21.15.64:6789/0 conn(0x7fc2800b58d0 legacy=0x7fc2801475f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:17.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.734+0000 7fc284f37700 1 -- 172.21.15.47:0/878089088 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc28014b020 con 0x7fc28014b4c0 2022-06-29T15:48:17.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.734+0000 7fc284f37700 1 --2- 172.21.15.47:0/878089088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc28014b4c0 0x7fc2800aa130 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7fc2780032f0 tx=0x7fc2780091d0).ready entity=mon.1 client_cookie=c25b80c5015e73a server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:17.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.734+0000 7fc27f7fe700 1 -- 172.21.15.47:0/878089088 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc27800b600 con 0x7fc28014b4c0 2022-06-29T15:48:17.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.734+0000 7fc27f7fe700 1 -- 172.21.15.47:0/878089088 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc27800b760 con 0x7fc28014b4c0 2022-06-29T15:48:17.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.734+0000 7fc27f7fe700 1 -- 172.21.15.47:0/878089088 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc27800b8d0 con 0x7fc28014b4c0 2022-06-29T15:48:17.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.734+0000 7fc285738700 1 -- 172.21.15.47:0/878089088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc28014b4c0 msgr2=0x7fc2800aa130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:17.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.734+0000 7fc285738700 1 --2- 172.21.15.47:0/878089088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc28014b4c0 0x7fc2800aa130 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7fc2780032f0 tx=0x7fc2780091d0).stop 2022-06-29T15:48:17.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.734+0000 7fc285738700 1 -- 172.21.15.47:0/878089088 shutdown_connections 2022-06-29T15:48:17.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.734+0000 7fc285738700 1 --2- 172.21.15.47:0/878089088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc28014b4c0 0x7fc2800aa130 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:17.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.734+0000 7fc285738700 1 --2- 172.21.15.47:0/878089088 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc2800b1610 0x7fc2800b5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:17.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.734+0000 7fc285738700 1 -- 172.21.15.47:0/878089088 >> 172.21.15.47:0/878089088 conn(0x7fc280062560 msgr2=0x7fc280062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:17.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.734+0000 7fc285738700 1 -- 172.21.15.47:0/878089088 shutdown_connections 2022-06-29T15:48:17.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.734+0000 7fc285738700 1 -- 172.21.15.47:0/878089088 wait complete. 2022-06-29T15:48:17.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.738+0000 7fc285738700 1 Processor -- start 2022-06-29T15:48:17.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.738+0000 7fc285738700 1 -- start start 2022-06-29T15:48:17.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.738+0000 7fc285738700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2800b1610 0x7fc280064240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:17.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.738+0000 7fc285738700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc2800b58d0 0x7fc280066c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:17.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.738+0000 7fc285738700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc28014b4c0 0x7fc2800b4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:17.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.738+0000 7fc285738700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc280066780 con 0x7fc28014b4c0 2022-06-29T15:48:17.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.738+0000 7fc285738700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2800664a0 con 0x7fc2800b1610 2022-06-29T15:48:17.748 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.738+0000 7fc285738700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2800ae460 con 0x7fc2800b58d0 2022-06-29T15:48:17.748 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.738+0000 7fc284f37700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc28014b4c0 0x7fc2800b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:17.748 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.738+0000 7fc284f37700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc28014b4c0 0x7fc2800b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49930/0 (socket says 172.21.15.47:49930) 2022-06-29T15:48:17.748 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.738+0000 7fc284f37700 1 -- 172.21.15.47:0/1547371388 learned_addr learned my addr 172.21.15.47:0/1547371388 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:17.749 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.738+0000 7fc27ffff700 1 --2- 172.21.15.47:0/1547371388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2800b1610 0x7fc280064240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:17.749 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.738+0000 7fc277fff700 1 --2- 172.21.15.47:0/1547371388 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc2800b58d0 0x7fc280066c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:17.749 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.738+0000 7fc284f37700 1 -- 172.21.15.47:0/1547371388 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc2800b58d0 msgr2=0x7fc280066c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:17.749 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.738+0000 7fc284f37700 1 --2- 172.21.15.47:0/1547371388 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc2800b58d0 0x7fc280066c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:17.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.738+0000 7fc284f37700 1 -- 172.21.15.47:0/1547371388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2800b1610 msgr2=0x7fc280064240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:17.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.738+0000 7fc284f37700 1 --2- 172.21.15.47:0/1547371388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2800b1610 0x7fc280064240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:17.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.738+0000 7fc284f37700 1 -- 172.21.15.47:0/1547371388 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc278009470 con 0x7fc28014b4c0 2022-06-29T15:48:17.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.738+0000 7fc284f37700 1 --2- 172.21.15.47:0/1547371388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc28014b4c0 0x7fc2800b4460 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7fc280065880 tx=0x7fc278016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:17.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.738+0000 7fc27dffb700 1 -- 172.21.15.47:0/1547371388 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc27800b600 con 0x7fc28014b4c0 2022-06-29T15:48:17.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.738+0000 7fc27dffb700 1 -- 172.21.15.47:0/1547371388 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc278011c00 con 0x7fc28014b4c0 2022-06-29T15:48:17.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.738+0000 7fc285738700 1 -- 172.21.15.47:0/1547371388 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2800a8e60 con 0x7fc28014b4c0 2022-06-29T15:48:17.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.738+0000 7fc27dffb700 1 -- 172.21.15.47:0/1547371388 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc27801f3f0 con 0x7fc28014b4c0 2022-06-29T15:48:17.752 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.738+0000 7fc285738700 1 -- 172.21.15.47:0/1547371388 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc280148260 con 0x7fc28014b4c0 2022-06-29T15:48:17.752 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.742+0000 7fc285738700 1 -- 172.21.15.47:0/1547371388 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2800ae780 con 0x7fc28014b4c0 2022-06-29T15:48:17.752 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.742+0000 7fc27dffb700 1 -- 172.21.15.47:0/1547371388 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc278026030 con 0x7fc28014b4c0 2022-06-29T15:48:17.753 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.742+0000 7fc27dffb700 1 --2- 172.21.15.47:0/1547371388 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc264053a70 0x7fc264055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:17.753 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.742+0000 7fc27dffb700 1 -- 172.21.15.47:0/1547371388 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5954+0+0 (secure 0 0 0) 0x7fc27806cac0 con 0x7fc28014b4c0 2022-06-29T15:48:17.754 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.742+0000 7fc27ffff700 1 --2- 172.21.15.47:0/1547371388 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc264053a70 0x7fc264055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:17.754 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.742+0000 7fc27ffff700 1 --2- 172.21.15.47:0/1547371388 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc264053a70 0x7fc264055f30 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7fc27000d000 tx=0x7fc270009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:17.754 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.750+0000 7fc27dffb700 1 -- 172.21.15.47:0/1547371388 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc2800ae780 con 0x7fc28014b4c0 2022-06-29T15:48:17.883 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:17.878+0000 7fc285738700 1 -- 172.21.15.47:0/1547371388 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7fc2800ae780 con 0x7fc28014b4c0 2022-06-29T15:48:18.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.030+0000 7fc27dffb700 1 -- 172.21.15.47:0/1547371388 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v76) v1 ==== 125+0+0 (secure 0 0 0) 0x7fc2800ae780 con 0x7fc28014b4c0 2022-06-29T15:48:18.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.038+0000 7fc285738700 1 -- 172.21.15.47:0/1547371388 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc264053a70 msgr2=0x7fc264055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:18.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.038+0000 7fc285738700 1 --2- 172.21.15.47:0/1547371388 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc264053a70 0x7fc264055f30 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7fc27000d000 tx=0x7fc270009040).stop 2022-06-29T15:48:18.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.038+0000 7fc285738700 1 -- 172.21.15.47:0/1547371388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc28014b4c0 msgr2=0x7fc2800b4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:18.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.038+0000 7fc285738700 1 --2- 172.21.15.47:0/1547371388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc28014b4c0 0x7fc2800b4460 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7fc280065880 tx=0x7fc278016000).stop 2022-06-29T15:48:18.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.042+0000 7fc285738700 1 -- 172.21.15.47:0/1547371388 shutdown_connections 2022-06-29T15:48:18.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.042+0000 7fc285738700 1 --2- 172.21.15.47:0/1547371388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc28014b4c0 0x7fc2800b4460 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:18.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.042+0000 7fc285738700 1 --2- 172.21.15.47:0/1547371388 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc2800b58d0 0x7fc280066c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:18.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.042+0000 7fc285738700 1 --2- 172.21.15.47:0/1547371388 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc264053a70 0x7fc264055f30 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:18.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.042+0000 7fc285738700 1 --2- 172.21.15.47:0/1547371388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2800b1610 0x7fc280064240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:18.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.042+0000 7fc285738700 1 -- 172.21.15.47:0/1547371388 >> 172.21.15.47:0/1547371388 conn(0x7fc280062560 msgr2=0x7fc2800ab700 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:18.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.042+0000 7fc285738700 1 -- 172.21.15.47:0/1547371388 shutdown_connections 2022-06-29T15:48:18.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.042+0000 7fc285738700 1 -- 172.21.15.47:0/1547371388 wait complete. 2022-06-29T15:48:18.048 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' created 2022-06-29T15:48:18.058 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-06-29T15:48:18.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.162+0000 7f4115172700 1 Processor -- start 2022-06-29T15:48:18.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.162+0000 7f4115172700 1 -- start start 2022-06-29T15:48:18.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.166+0000 7f4115172700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f41100b4300 0x7f41100612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:18.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.166+0000 7f4115172700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f41101471a0 0x7f4110066f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:18.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.166+0000 7f4115172700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4110067610 con 0x7f4110060ee0 2022-06-29T15:48:18.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.166+0000 7f4115172700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41101467c0 con 0x7f41100b4300 2022-06-29T15:48:18.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.166+0000 7f4115172700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4110146a60 con 0x7f41101471a0 2022-06-29T15:48:18.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.166+0000 7f410659c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f41100b4300 0x7f41100612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:18.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.166+0000 7f410659c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f41100b4300 0x7f41100612c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49936/0 (socket says 172.21.15.47:49936) 2022-06-29T15:48:18.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.166+0000 7f410ed9d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4110060ee0 0x7f41101430e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35490/0 (socket says 172.21.15.47:35490) 2022-06-29T15:48:18.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.166+0000 7f410659c700 1 -- 172.21.15.47:0/401109427 learned_addr learned my addr 172.21.15.47:0/401109427 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:18.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.166+0000 7f410f59e700 1 --2- 172.21.15.47:0/401109427 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f41101471a0 0x7f4110066f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:18.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.166+0000 7f410659c700 1 -- 172.21.15.47:0/401109427 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f41101471a0 msgr2=0x7f4110066f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:18.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.166+0000 7f410659c700 1 --2- 172.21.15.47:0/401109427 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f41101471a0 0x7f4110066f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:18.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.166+0000 7f410659c700 1 -- 172.21.15.47:0/401109427 >> v1:172.21.15.47:6789/0 conn(0x7f4110060ee0 legacy=0x7f41101430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:18.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.166+0000 7f410e59c700 1 -- 172.21.15.47:0/401109427 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1423825906 0 0) 0x7f4110067610 con 0x7f4110060ee0 2022-06-29T15:48:18.182 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.166+0000 7f410659c700 1 -- 172.21.15.47:0/401109427 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4110146d00 con 0x7f41100b4300 2022-06-29T15:48:18.182 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.166+0000 7f410f59e700 1 --2- 172.21.15.47:0/401109427 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f41101471a0 0x7f4110066f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:48:18.183 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.166+0000 7f410659c700 1 --2- 172.21.15.47:0/401109427 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41100b4300 0x7f41100612c0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7f41100ae790 tx=0x7f40f8006eb0).ready entity=mon.0 client_cookie=e3a6e3bc34cac3fe server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:18.183 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.166+0000 7f410e59c700 1 -- 172.21.15.47:0/401109427 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40f800d5c0 con 0x7f41100b4300 2022-06-29T15:48:18.183 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.166+0000 7f410e59c700 1 -- 172.21.15.47:0/401109427 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40f8000e90 con 0x7f41100b4300 2022-06-29T15:48:18.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.166+0000 7f410e59c700 1 -- 172.21.15.47:0/401109427 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40f800d910 con 0x7f41100b4300 2022-06-29T15:48:18.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.166+0000 7f4115172700 1 -- 172.21.15.47:0/401109427 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41100b4300 msgr2=0x7f41100612c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:18.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.166+0000 7f4115172700 1 --2- 172.21.15.47:0/401109427 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41100b4300 0x7f41100612c0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7f41100ae790 tx=0x7f40f8006eb0).stop 2022-06-29T15:48:18.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.166+0000 7f4115172700 1 -- 172.21.15.47:0/401109427 shutdown_connections 2022-06-29T15:48:18.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.166+0000 7f4115172700 1 --2- 172.21.15.47:0/401109427 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f41101471a0 0x7f4110066f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:18.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.166+0000 7f4115172700 1 --2- 172.21.15.47:0/401109427 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41100b4300 0x7f41100612c0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:18.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.166+0000 7f4115172700 1 -- 172.21.15.47:0/401109427 >> 172.21.15.47:0/401109427 conn(0x7f411005c510 msgr2=0x7f411005f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:18.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.166+0000 7f4115172700 1 -- 172.21.15.47:0/401109427 shutdown_connections 2022-06-29T15:48:18.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.166+0000 7f4115172700 1 -- 172.21.15.47:0/401109427 wait complete. 2022-06-29T15:48:18.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.166+0000 7f4115172700 1 Processor -- start 2022-06-29T15:48:18.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.170+0000 7f4115172700 1 -- start start 2022-06-29T15:48:18.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.170+0000 7f4115172700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4110060ee0 0x7f41100b2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:18.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.170+0000 7f4115172700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41100b4300 0x7f4110152550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:18.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.170+0000 7f4115172700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f41101471a0 0x7f4110152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:18.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.170+0000 7f4115172700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41100b9150 con 0x7f41100b4300 2022-06-29T15:48:18.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.170+0000 7f4115172700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411013e1d0 con 0x7f4110060ee0 2022-06-29T15:48:18.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.170+0000 7f4115172700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41100b8c10 con 0x7f41101471a0 2022-06-29T15:48:18.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.170+0000 7f410659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41100b4300 0x7f4110152550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:18.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.170+0000 7f410659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41100b4300 0x7f4110152550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49944/0 (socket says 172.21.15.47:49944) 2022-06-29T15:48:18.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.170+0000 7f410659c700 1 -- 172.21.15.47:0/3562529284 learned_addr learned my addr 172.21.15.47:0/3562529284 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:18.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.170+0000 7f410f59e700 1 --2- 172.21.15.47:0/3562529284 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f41101471a0 0x7f4110152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:18.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.170+0000 7f410ed9d700 1 --2- 172.21.15.47:0/3562529284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4110060ee0 0x7f41100b2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:18.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.170+0000 7f410659c700 1 -- 172.21.15.47:0/3562529284 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f41101471a0 msgr2=0x7f4110152a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:18.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.170+0000 7f410659c700 1 --2- 172.21.15.47:0/3562529284 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f41101471a0 0x7f4110152a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:18.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.170+0000 7f410659c700 1 -- 172.21.15.47:0/3562529284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4110060ee0 msgr2=0x7f41100b2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:18.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.170+0000 7f410659c700 1 --2- 172.21.15.47:0/3562529284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4110060ee0 0x7f41100b2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:18.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.170+0000 7f410659c700 1 -- 172.21.15.47:0/3562529284 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4108009470 con 0x7f41100b4300 2022-06-29T15:48:18.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.170+0000 7f410659c700 1 --2- 172.21.15.47:0/3562529284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41100b4300 0x7f4110152550 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0x7f41100ae790 tx=0x7f40f8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:18.191 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.170+0000 7f410cd99700 1 -- 172.21.15.47:0/3562529284 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40f8011860 con 0x7f41100b4300 2022-06-29T15:48:18.191 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.170+0000 7f410cd99700 1 -- 172.21.15.47:0/3562529284 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40f80119c0 con 0x7f41100b4300 2022-06-29T15:48:18.191 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.170+0000 7f4115172700 1 -- 172.21.15.47:0/3562529284 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40f800b1e0 con 0x7f41100b4300 2022-06-29T15:48:18.191 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.170+0000 7f410cd99700 1 -- 172.21.15.47:0/3562529284 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40f8011d20 con 0x7f41100b4300 2022-06-29T15:48:18.192 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.170+0000 7f4115172700 1 -- 172.21.15.47:0/3562529284 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41100b3700 con 0x7f41100b4300 2022-06-29T15:48:18.192 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.174+0000 7f4115172700 1 -- 172.21.15.47:0/3562529284 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41100573d0 con 0x7f41100b4300 2022-06-29T15:48:18.193 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.174+0000 7f410cd99700 1 -- 172.21.15.47:0/3562529284 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f40f800f070 con 0x7f41100b4300 2022-06-29T15:48:18.193 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.174+0000 7f410cd99700 1 --2- 172.21.15.47:0/3562529284 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f40f40539e0 0x7f40f4055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:18.194 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.174+0000 7f410cd99700 1 -- 172.21.15.47:0/3562529284 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f40f806d970 con 0x7f41100b4300 2022-06-29T15:48:18.194 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.174+0000 7f410ed9d700 1 --2- 172.21.15.47:0/3562529284 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f40f40539e0 0x7f40f4055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:18.194 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.174+0000 7f410ed9d700 1 --2- 172.21.15.47:0/3562529284 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f40f40539e0 0x7f40f4055ea0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7f410000d000 tx=0x7f4100007a00).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:18.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.182+0000 7f410cd99700 1 -- 172.21.15.47:0/3562529284 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f40f80446a0 con 0x7f41100b4300 2022-06-29T15:48:18.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.342+0000 7f4115172700 1 -- 172.21.15.47:0/3562529284 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f41100b5e50 con 0x7f41100b4300 2022-06-29T15:48:18.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.350+0000 7f410cd99700 1 -- 172.21.15.47:0/3562529284 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 13 and data pool 14 v34) v1 ==== 189+0+0 (secure 0 0 0) 0x7f40f8047cc0 con 0x7f41100b4300 2022-06-29T15:48:18.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.354+0000 7f4115172700 1 -- 172.21.15.47:0/3562529284 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f40f40539e0 msgr2=0x7f40f4055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:18.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.354+0000 7f4115172700 1 --2- 172.21.15.47:0/3562529284 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f40f40539e0 0x7f40f4055ea0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7f410000d000 tx=0x7f4100007a00).stop 2022-06-29T15:48:18.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.354+0000 7f4115172700 1 -- 172.21.15.47:0/3562529284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41100b4300 msgr2=0x7f4110152550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:18.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.354+0000 7f4115172700 1 --2- 172.21.15.47:0/3562529284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41100b4300 0x7f4110152550 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0x7f41100ae790 tx=0x7f40f8015000).stop 2022-06-29T15:48:18.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.358+0000 7f4115172700 1 -- 172.21.15.47:0/3562529284 shutdown_connections 2022-06-29T15:48:18.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.358+0000 7f4115172700 1 --2- 172.21.15.47:0/3562529284 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f41101471a0 0x7f4110152a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:18.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.358+0000 7f4115172700 1 --2- 172.21.15.47:0/3562529284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41100b4300 0x7f4110152550 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:18.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.358+0000 7f4115172700 1 --2- 172.21.15.47:0/3562529284 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f40f40539e0 0x7f40f4055ea0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:18.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.358+0000 7f4115172700 1 --2- 172.21.15.47:0/3562529284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4110060ee0 0x7f41100b2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:18.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.358+0000 7f4115172700 1 -- 172.21.15.47:0/3562529284 >> 172.21.15.47:0/3562529284 conn(0x7f411005c510 msgr2=0x7f411013c730 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:18.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.358+0000 7f4115172700 1 -- 172.21.15.47:0/3562529284 shutdown_connections 2022-06-29T15:48:18.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.358+0000 7f4115172700 1 -- 172.21.15.47:0/3562529284 wait complete. 2022-06-29T15:48:18.364 INFO:teuthology.orchestra.run.smithi047.stderr:new fs with metadata pool 13 and data pool 14 2022-06-29T15:48:18.376 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:48:18.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.462+0000 7fe7e88e7700 1 Processor -- start 2022-06-29T15:48:18.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.462+0000 7fe7e88e7700 1 -- start start 2022-06-29T15:48:18.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.466+0000 7fe7e88e7700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe7e405c510 0x7fe7e4143bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:18.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.466+0000 7fe7e88e7700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe7e40b2c40 0x7fe7e40b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:18.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.466+0000 7fe7e88e7700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7e4142f90 con 0x7fe7e405c8f0 2022-06-29T15:48:18.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.466+0000 7fe7e88e7700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7e41431f0 con 0x7fe7e40b2c40 2022-06-29T15:48:18.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.466+0000 7fe7e88e7700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7e4143490 con 0x7fe7e405c510 2022-06-29T15:48:18.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.466+0000 7fe7e359e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe7e40b2c40 0x7fe7e40b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:18.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.466+0000 7fe7e359e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe7e40b2c40 0x7fe7e40b30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49956/0 (socket says 172.21.15.47:49956) 2022-06-29T15:48:18.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.466+0000 7fe7e359e700 1 -- 172.21.15.47:0/3270789016 learned_addr learned my addr 172.21.15.47:0/3270789016 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:18.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.466+0000 7fe7da59c700 1 --1- 172.21.15.47:0/3270789016 >> v1:172.21.15.47:6789/0 conn(0x7fe7e405c8f0 0x7fe7e40b4100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35508/0 (socket says 172.21.15.47:35508) 2022-06-29T15:48:18.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.466+0000 7fe7e2d9d700 1 --2- 172.21.15.47:0/3270789016 >> v2:172.21.15.64:3300/0 conn(0x7fe7e405c510 0x7fe7e4143bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:18.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.466+0000 7fe7e359e700 1 -- 172.21.15.47:0/3270789016 >> v2:172.21.15.64:3300/0 conn(0x7fe7e405c510 msgr2=0x7fe7e4143bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:18.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.466+0000 7fe7e359e700 1 --2- 172.21.15.47:0/3270789016 >> v2:172.21.15.64:3300/0 conn(0x7fe7e405c510 0x7fe7e4143bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:18.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.466+0000 7fe7e359e700 1 -- 172.21.15.47:0/3270789016 >> v1:172.21.15.47:6789/0 conn(0x7fe7e405c8f0 legacy=0x7fe7e40b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:18.484 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.466+0000 7fe7e359e700 1 -- 172.21.15.47:0/3270789016 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e4143730 con 0x7fe7e40b2c40 2022-06-29T15:48:18.484 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.466+0000 7fe7e2d9d700 1 --2- 172.21.15.47:0/3270789016 >> v2:172.21.15.64:3300/0 conn(0x7fe7e405c510 0x7fe7e4143bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:48:18.484 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.466+0000 7fe7e359e700 1 --2- 172.21.15.47:0/3270789016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7e40b2c40 0x7fe7e40b30d0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0x7fe7dc0032f0 tx=0x7fe7dc009290).ready entity=mon.0 client_cookie=a0e5a494859c42a8 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:18.484 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.466+0000 7fe7e259c700 1 -- 172.21.15.47:0/3270789016 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe7dc00b870 con 0x7fe7e40b2c40 2022-06-29T15:48:18.485 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.466+0000 7fe7e259c700 1 -- 172.21.15.47:0/3270789016 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7dc00b9d0 con 0x7fe7e40b2c40 2022-06-29T15:48:18.485 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.466+0000 7fe7e259c700 1 -- 172.21.15.47:0/3270789016 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe7dc00bb40 con 0x7fe7e40b2c40 2022-06-29T15:48:18.485 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.466+0000 7fe7e88e7700 1 -- 172.21.15.47:0/3270789016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7e40b2c40 msgr2=0x7fe7e40b30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:18.485 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.466+0000 7fe7e88e7700 1 --2- 172.21.15.47:0/3270789016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7e40b2c40 0x7fe7e40b30d0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0x7fe7dc0032f0 tx=0x7fe7dc009290).stop 2022-06-29T15:48:18.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.466+0000 7fe7e88e7700 1 -- 172.21.15.47:0/3270789016 shutdown_connections 2022-06-29T15:48:18.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.466+0000 7fe7e88e7700 1 --2- 172.21.15.47:0/3270789016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7e40b2c40 0x7fe7e40b30d0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:18.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.466+0000 7fe7e88e7700 1 --2- 172.21.15.47:0/3270789016 >> v2:172.21.15.64:3300/0 conn(0x7fe7e405c510 0x7fe7e4143bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:18.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.466+0000 7fe7e88e7700 1 -- 172.21.15.47:0/3270789016 >> 172.21.15.47:0/3270789016 conn(0x7fe7e4060e00 msgr2=0x7fe7e40611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:18.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.466+0000 7fe7e88e7700 1 -- 172.21.15.47:0/3270789016 shutdown_connections 2022-06-29T15:48:18.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.466+0000 7fe7e88e7700 1 -- 172.21.15.47:0/3270789016 wait complete. 2022-06-29T15:48:18.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.470+0000 7fe7e88e7700 1 Processor -- start 2022-06-29T15:48:18.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.470+0000 7fe7e88e7700 1 -- start start 2022-06-29T15:48:18.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.470+0000 7fe7e88e7700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe7e405c510 0x7fe7e40ad9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:18.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.470+0000 7fe7e88e7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7e405c8f0 0x7fe7e40b0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:18.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.470+0000 7fe7e88e7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7e40b2c40 0x7fe7e413efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:18.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.470+0000 7fe7e88e7700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e40afb70 con 0x7fe7e405c8f0 2022-06-29T15:48:18.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.470+0000 7fe7e88e7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e4153380 con 0x7fe7e40b2c40 2022-06-29T15:48:18.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.470+0000 7fe7e88e7700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e4141d20 con 0x7fe7e405c510 2022-06-29T15:48:18.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.470+0000 7fe7da59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7e405c8f0 0x7fe7e40b0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:18.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.470+0000 7fe7da59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7e405c8f0 0x7fe7e40b0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49960/0 (socket says 172.21.15.47:49960) 2022-06-29T15:48:18.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.470+0000 7fe7da59c700 1 -- 172.21.15.47:0/2575674333 learned_addr learned my addr 172.21.15.47:0/2575674333 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:18.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.470+0000 7fe7e2d9d700 1 --2- 172.21.15.47:0/2575674333 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe7e405c510 0x7fe7e40ad9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:18.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.470+0000 7fe7e359e700 1 --2- 172.21.15.47:0/2575674333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7e40b2c40 0x7fe7e413efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:18.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.470+0000 7fe7da59c700 1 -- 172.21.15.47:0/2575674333 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe7e405c510 msgr2=0x7fe7e40ad9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:18.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.470+0000 7fe7da59c700 1 --2- 172.21.15.47:0/2575674333 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe7e405c510 0x7fe7e40ad9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:18.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.470+0000 7fe7da59c700 1 -- 172.21.15.47:0/2575674333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7e40b2c40 msgr2=0x7fe7e413efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:18.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.470+0000 7fe7da59c700 1 --2- 172.21.15.47:0/2575674333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7e40b2c40 0x7fe7e413efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:18.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.470+0000 7fe7da59c700 1 -- 172.21.15.47:0/2575674333 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7dc009530 con 0x7fe7e405c8f0 2022-06-29T15:48:18.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.470+0000 7fe7da59c700 1 --2- 172.21.15.47:0/2575674333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7e405c8f0 0x7fe7e40b0520 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 rx=0x7fe7e40b7300 tx=0x7fe7cc007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:18.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.470+0000 7fe7e0d99700 1 -- 172.21.15.47:0/2575674333 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe7cc0070d0 con 0x7fe7e405c8f0 2022-06-29T15:48:18.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.470+0000 7fe7e0d99700 1 -- 172.21.15.47:0/2575674333 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7cc007230 con 0x7fe7e405c8f0 2022-06-29T15:48:18.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.470+0000 7fe7e88e7700 1 -- 172.21.15.47:0/2575674333 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7e4152e40 con 0x7fe7e405c8f0 2022-06-29T15:48:18.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.470+0000 7fe7e0d99700 1 -- 172.21.15.47:0/2575674333 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe7cc005740 con 0x7fe7e405c8f0 2022-06-29T15:48:18.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.470+0000 7fe7e88e7700 1 -- 172.21.15.47:0/2575674333 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7e4146880 con 0x7fe7e405c8f0 2022-06-29T15:48:18.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.474+0000 7fe7e0d99700 1 -- 172.21.15.47:0/2575674333 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe7cc010070 con 0x7fe7e405c8f0 2022-06-29T15:48:18.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.474+0000 7fe7e88e7700 1 -- 172.21.15.47:0/2575674333 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7e4112af0 con 0x7fe7e405c8f0 2022-06-29T15:48:18.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.474+0000 7fe7e0d99700 1 --2- 172.21.15.47:0/2575674333 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe7c8053b30 0x7fe7c8055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:18.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.474+0000 7fe7e0d99700 1 -- 172.21.15.47:0/2575674333 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe7cc06b460 con 0x7fe7e405c8f0 2022-06-29T15:48:18.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.474+0000 7fe7e2d9d700 1 --2- 172.21.15.47:0/2575674333 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe7c8053b30 0x7fe7c8055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:18.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.474+0000 7fe7e2d9d700 1 --2- 172.21.15.47:0/2575674333 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe7c8053b30 0x7fe7c8055ff0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7fe7d400c000 tx=0x7fe7d4008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:18.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.478+0000 7fe7e0d99700 1 -- 172.21.15.47:0/2575674333 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe7cc042220 con 0x7fe7e405c8f0 2022-06-29T15:48:18.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.606+0000 7fe7e88e7700 1 -- 172.21.15.47:0/2575674333 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe7e4146a40 con 0x7fe7e405c8f0 2022-06-29T15:48:18.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.606+0000 7fe7e0d99700 1 -- 172.21.15.47:0/2575674333 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+16287 (secure 0 0 0) 0x7fe7cc045840 con 0x7fe7e405c8f0 2022-06-29T15:48:18.608 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:48:18.609 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":77,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:48:18.345670+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:48:16.753707+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2022-06-29T15:48:17.935585+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:48:18.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.606+0000 7fe7e88e7700 1 -- 172.21.15.47:0/2575674333 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe7c8053b30 msgr2=0x7fe7c8055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:18.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.606+0000 7fe7e88e7700 1 --2- 172.21.15.47:0/2575674333 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe7c8053b30 0x7fe7c8055ff0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7fe7d400c000 tx=0x7fe7d4008040).stop 2022-06-29T15:48:18.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.606+0000 7fe7e88e7700 1 -- 172.21.15.47:0/2575674333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7e405c8f0 msgr2=0x7fe7e40b0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:18.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.606+0000 7fe7e88e7700 1 --2- 172.21.15.47:0/2575674333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7e405c8f0 0x7fe7e40b0520 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 rx=0x7fe7e40b7300 tx=0x7fe7cc007c00).stop 2022-06-29T15:48:18.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.606+0000 7fe7e88e7700 1 -- 172.21.15.47:0/2575674333 shutdown_connections 2022-06-29T15:48:18.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.606+0000 7fe7e88e7700 1 --2- 172.21.15.47:0/2575674333 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe7c8053b30 0x7fe7c8055ff0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:18.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.606+0000 7fe7e88e7700 1 --2- 172.21.15.47:0/2575674333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7e40b2c40 0x7fe7e413efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:18.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.606+0000 7fe7e88e7700 1 --2- 172.21.15.47:0/2575674333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7e405c8f0 0x7fe7e40b0520 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:18.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.606+0000 7fe7e88e7700 1 --2- 172.21.15.47:0/2575674333 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe7e405c510 0x7fe7e40ad9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:18.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.606+0000 7fe7e88e7700 1 -- 172.21.15.47:0/2575674333 >> 172.21.15.47:0/2575674333 conn(0x7fe7e4060e00 msgr2=0x7fe7e413fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:18.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.606+0000 7fe7e88e7700 1 -- 172.21.15.47:0/2575674333 shutdown_connections 2022-06-29T15:48:18.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.606+0000 7fe7e88e7700 1 -- 172.21.15.47:0/2575674333 wait complete. 2022-06-29T15:48:18.622 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:48:18.724 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.722+0000 7fdf756c9700 1 Processor -- start 2022-06-29T15:48:18.724 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.722+0000 7fdf756c9700 1 -- start start 2022-06-29T15:48:18.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.722+0000 7fdf756c9700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdf700b2120 0x7fdf700b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:18.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.722+0000 7fdf756c9700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdf700b1260 0x7fdf7014b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:18.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.722+0000 7fdf756c9700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf700b0740 con 0x7fdf7005f0b0 2022-06-29T15:48:18.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.722+0000 7fdf756c9700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf70147460 con 0x7fdf700b2120 2022-06-29T15:48:18.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.722+0000 7fdf756c9700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf700b0970 con 0x7fdf700b1260 2022-06-29T15:48:18.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.722+0000 7fdf6e7fc700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fdf7005f0b0 0x7fdf70147af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35520/0 (socket says 172.21.15.47:35520) 2022-06-29T15:48:18.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.722+0000 7fdf6e7fc700 1 -- 172.21.15.47:0/296345613 learned_addr learned my addr 172.21.15.47:0/296345613 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:48:18.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.722+0000 7fdf6effd700 1 --2- 172.21.15.47:0/296345613 >> v2:172.21.15.64:3300/0 conn(0x7fdf700b2120 0x7fdf700b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:18.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.722+0000 7fdf6f7fe700 1 --2- 172.21.15.47:0/296345613 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdf700b1260 0x7fdf7014b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:18.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.722+0000 7fdf6dffb700 1 -- 172.21.15.47:0/296345613 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 336850467 0 0) 0x7fdf700b0740 con 0x7fdf7005f0b0 2022-06-29T15:48:18.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.722+0000 7fdf6dffb700 1 -- 172.21.15.47:0/296345613 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf54003260 con 0x7fdf7005f0b0 2022-06-29T15:48:18.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.722+0000 7fdf6dffb700 1 -- 172.21.15.47:0/296345613 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1878880285 0 0) 0x7fdf54003260 con 0x7fdf7005f0b0 2022-06-29T15:48:18.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.722+0000 7fdf6effd700 1 -- 172.21.15.47:0/296345613 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdf700b1260 msgr2=0x7fdf7014b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:18.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.722+0000 7fdf6effd700 1 --2- 172.21.15.47:0/296345613 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdf700b1260 0x7fdf7014b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:18.731 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.722+0000 7fdf6effd700 1 -- 172.21.15.47:0/296345613 >> v1:172.21.15.47:6789/0 conn(0x7fdf7005f0b0 legacy=0x7fdf70147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:18.731 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.722+0000 7fdf6effd700 1 -- 172.21.15.47:0/296345613 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf700b0c10 con 0x7fdf700b2120 2022-06-29T15:48:18.731 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.722+0000 7fdf6effd700 1 --2- 172.21.15.47:0/296345613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf700b2120 0x7fdf700b2500 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7fdf70141a70 tx=0x7fdf60006df0).ready entity=mon.1 client_cookie=18ca4586ac0c891b server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:18.731 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.722+0000 7fdf6dffb700 1 -- 172.21.15.47:0/296345613 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf680019e0 con 0x7fdf700b2120 2022-06-29T15:48:18.732 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.722+0000 7fdf6dffb700 1 -- 172.21.15.47:0/296345613 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf60000e90 con 0x7fdf700b2120 2022-06-29T15:48:18.732 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.722+0000 7fdf6dffb700 1 -- 172.21.15.47:0/296345613 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf60012460 con 0x7fdf700b2120 2022-06-29T15:48:18.732 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.726+0000 7fdf756c9700 1 -- 172.21.15.47:0/296345613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf700b2120 msgr2=0x7fdf700b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:18.732 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.726+0000 7fdf756c9700 1 --2- 172.21.15.47:0/296345613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf700b2120 0x7fdf700b2500 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7fdf70141a70 tx=0x7fdf60006df0).stop 2022-06-29T15:48:18.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.726+0000 7fdf756c9700 1 -- 172.21.15.47:0/296345613 shutdown_connections 2022-06-29T15:48:18.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.726+0000 7fdf756c9700 1 --2- 172.21.15.47:0/296345613 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdf700b1260 0x7fdf7014b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:18.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.726+0000 7fdf756c9700 1 --2- 172.21.15.47:0/296345613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf700b2120 0x7fdf700b2500 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:18.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.726+0000 7fdf756c9700 1 -- 172.21.15.47:0/296345613 >> 172.21.15.47:0/296345613 conn(0x7fdf700626f0 msgr2=0x7fdf70062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:18.734 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.726+0000 7fdf756c9700 1 -- 172.21.15.47:0/296345613 shutdown_connections 2022-06-29T15:48:18.734 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.726+0000 7fdf756c9700 1 -- 172.21.15.47:0/296345613 wait complete. 2022-06-29T15:48:18.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.726+0000 7fdf756c9700 1 Processor -- start 2022-06-29T15:48:18.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.726+0000 7fdf756c9700 1 -- start start 2022-06-29T15:48:18.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.726+0000 7fdf756c9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf700b1260 0x7fdf700ae490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:18.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.726+0000 7fdf756c9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf700b2120 0x7fdf7014b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:18.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.726+0000 7fdf756c9700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdf700af4a0 0x7fdf7014a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:18.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.726+0000 7fdf756c9700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf70156770 con 0x7fdf700b2120 2022-06-29T15:48:18.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.726+0000 7fdf756c9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf70145980 con 0x7fdf700b1260 2022-06-29T15:48:18.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.726+0000 7fdf756c9700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf700aea60 con 0x7fdf700af4a0 2022-06-29T15:48:18.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.726+0000 7fdf6e7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf700b2120 0x7fdf7014b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:18.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.726+0000 7fdf6e7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf700b2120 0x7fdf7014b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49976/0 (socket says 172.21.15.47:49976) 2022-06-29T15:48:18.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.726+0000 7fdf6e7fc700 1 -- 172.21.15.47:0/3417723950 learned_addr learned my addr 172.21.15.47:0/3417723950 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:18.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.726+0000 7fdf6effd700 1 --2- 172.21.15.47:0/3417723950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf700b1260 0x7fdf700ae490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:18.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.726+0000 7fdf6f7fe700 1 --2- 172.21.15.47:0/3417723950 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdf700af4a0 0x7fdf7014a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:18.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.726+0000 7fdf6e7fc700 1 -- 172.21.15.47:0/3417723950 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdf700af4a0 msgr2=0x7fdf7014a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:18.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.726+0000 7fdf6e7fc700 1 --2- 172.21.15.47:0/3417723950 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdf700af4a0 0x7fdf7014a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:18.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.726+0000 7fdf6e7fc700 1 -- 172.21.15.47:0/3417723950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf700b1260 msgr2=0x7fdf700ae490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:18.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.726+0000 7fdf6e7fc700 1 --2- 172.21.15.47:0/3417723950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf700b1260 0x7fdf700ae490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:18.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.726+0000 7fdf6e7fc700 1 -- 172.21.15.47:0/3417723950 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf6000b180 con 0x7fdf700b2120 2022-06-29T15:48:18.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.730+0000 7fdf6e7fc700 1 --2- 172.21.15.47:0/3417723950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf700b2120 0x7fdf7014b3d0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 rx=0x7fdf68001c40 tx=0x7fdf68002180).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:18.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.730+0000 7fdf67fff700 1 -- 172.21.15.47:0/3417723950 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf68005ba0 con 0x7fdf700b2120 2022-06-29T15:48:18.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.730+0000 7fdf67fff700 1 -- 172.21.15.47:0/3417723950 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf68005d00 con 0x7fdf700b2120 2022-06-29T15:48:18.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.730+0000 7fdf756c9700 1 -- 172.21.15.47:0/3417723950 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf70144b80 con 0x7fdf700b2120 2022-06-29T15:48:18.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.730+0000 7fdf67fff700 1 -- 172.21.15.47:0/3417723950 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf68007300 con 0x7fdf700b2120 2022-06-29T15:48:18.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.730+0000 7fdf756c9700 1 -- 172.21.15.47:0/3417723950 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf701450c0 con 0x7fdf700b2120 2022-06-29T15:48:18.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.730+0000 7fdf756c9700 1 -- 172.21.15.47:0/3417723950 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf70145280 con 0x7fdf700b2120 2022-06-29T15:48:18.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.730+0000 7fdf67fff700 1 -- 172.21.15.47:0/3417723950 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fdf68010030 con 0x7fdf700b2120 2022-06-29T15:48:18.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.730+0000 7fdf67fff700 1 --2- 172.21.15.47:0/3417723950 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdf48053a10 0x7fdf48055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:18.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.730+0000 7fdf67fff700 1 -- 172.21.15.47:0/3417723950 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fdf6806c190 con 0x7fdf700b2120 2022-06-29T15:48:18.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.730+0000 7fdf6effd700 1 --2- 172.21.15.47:0/3417723950 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdf48053a10 0x7fdf48055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:18.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.734+0000 7fdf6effd700 1 --2- 172.21.15.47:0/3417723950 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdf48053a10 0x7fdf48055ed0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7fdf6001a000 tx=0x7fdf60014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:18.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.738+0000 7fdf67fff700 1 -- 172.21.15.47:0/3417723950 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdf68042e50 con 0x7fdf700b2120 2022-06-29T15:48:18.866 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.862+0000 7fdf756c9700 1 -- 172.21.15.47:0/3417723950 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf700573d0 con 0x7fdf700b2120 2022-06-29T15:48:18.868 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.866+0000 7fdf67fff700 1 -- 172.21.15.47:0/3417723950 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+16287 (secure 0 0 0) 0x7fdf68046470 con 0x7fdf700b2120 2022-06-29T15:48:18.869 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:48:18.869 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":77,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:48:18.345670+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:48:16.753707+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2022-06-29T15:48:17.935585+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:48:18.870 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.866+0000 7fdf756c9700 1 -- 172.21.15.47:0/3417723950 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdf48053a10 msgr2=0x7fdf48055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:18.871 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.866+0000 7fdf756c9700 1 --2- 172.21.15.47:0/3417723950 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdf48053a10 0x7fdf48055ed0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7fdf6001a000 tx=0x7fdf60014000).stop 2022-06-29T15:48:18.871 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.866+0000 7fdf756c9700 1 -- 172.21.15.47:0/3417723950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf700b2120 msgr2=0x7fdf7014b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:18.871 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.866+0000 7fdf756c9700 1 --2- 172.21.15.47:0/3417723950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf700b2120 0x7fdf7014b3d0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 rx=0x7fdf68001c40 tx=0x7fdf68002180).stop 2022-06-29T15:48:18.871 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.866+0000 7fdf756c9700 1 -- 172.21.15.47:0/3417723950 shutdown_connections 2022-06-29T15:48:18.872 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.866+0000 7fdf756c9700 1 --2- 172.21.15.47:0/3417723950 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdf700af4a0 0x7fdf7014a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:18.872 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.866+0000 7fdf756c9700 1 --2- 172.21.15.47:0/3417723950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf700b2120 0x7fdf7014b3d0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:18.872 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.866+0000 7fdf756c9700 1 --2- 172.21.15.47:0/3417723950 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdf48053a10 0x7fdf48055ed0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:18.872 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.866+0000 7fdf756c9700 1 --2- 172.21.15.47:0/3417723950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf700b1260 0x7fdf700ae490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:18.873 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.866+0000 7fdf756c9700 1 -- 172.21.15.47:0/3417723950 >> 172.21.15.47:0/3417723950 conn(0x7fdf700626f0 msgr2=0x7fdf70146970 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:18.873 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.866+0000 7fdf756c9700 1 -- 172.21.15.47:0/3417723950 shutdown_connections 2022-06-29T15:48:18.873 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.866+0000 7fdf756c9700 1 -- 172.21.15.47:0/3417723950 wait complete. 2022-06-29T15:48:18.882 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-06-29T15:48:18.993 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.990+0000 7fc908ec9700 1 Processor -- start 2022-06-29T15:48:18.993 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.990+0000 7fc908ec9700 1 -- start start 2022-06-29T15:48:18.996 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.990+0000 7fc908ec9700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc9040b64c0 0x7fc9040b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:18.997 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.990+0000 7fc908ec9700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc904066fe0 0x7fc9040b4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:18.997 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.990+0000 7fc908ec9700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9040b68a0 con 0x7fc904143150 2022-06-29T15:48:18.997 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.990+0000 7fc908ec9700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9040b13a0 con 0x7fc904066fe0 2022-06-29T15:48:18.997 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.990+0000 7fc908ec9700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9041473d0 con 0x7fc9040b64c0 2022-06-29T15:48:18.998 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.990+0000 7fc902ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc904066fe0 0x7fc9040b4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:18.998 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.990+0000 7fc902ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc904066fe0 0x7fc9040b4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49980/0 (socket says 172.21.15.47:49980) 2022-06-29T15:48:18.998 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.990+0000 7fc902ffd700 1 -- 172.21.15.47:0/1024551614 learned_addr learned my addr 172.21.15.47:0/1024551614 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:18.998 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.990+0000 7fc9037fe700 1 --2- 172.21.15.47:0/1024551614 >> v2:172.21.15.64:3300/0 conn(0x7fc9040b64c0 0x7fc9040b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:18.999 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.990+0000 7fc902ffd700 1 -- 172.21.15.47:0/1024551614 >> v2:172.21.15.64:3300/0 conn(0x7fc9040b64c0 msgr2=0x7fc9040b5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:18.999 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.990+0000 7fc902ffd700 1 --2- 172.21.15.47:0/1024551614 >> v2:172.21.15.64:3300/0 conn(0x7fc9040b64c0 0x7fc9040b5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:18.999 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.990+0000 7fc902ffd700 1 -- 172.21.15.47:0/1024551614 >> v1:172.21.15.64:6789/0 conn(0x7fc904143150 legacy=0x7fc90412bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:19.000 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.990+0000 7fc902ffd700 1 -- 172.21.15.47:0/1024551614 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9040b46a0 con 0x7fc904066fe0 2022-06-29T15:48:19.000 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.990+0000 7fc902ffd700 1 --2- 172.21.15.47:0/1024551614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc904066fe0 0x7fc9040b4b40 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 rx=0x7fc8f4002f70 tx=0x7fc8f4006e50).ready entity=mon.0 client_cookie=45eaa85049f6e570 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:19.000 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.994+0000 7fc9027fc700 1 -- 172.21.15.47:0/1024551614 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc8f400d610 con 0x7fc904066fe0 2022-06-29T15:48:19.000 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.994+0000 7fc9027fc700 1 -- 172.21.15.47:0/1024551614 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8f4000e90 con 0x7fc904066fe0 2022-06-29T15:48:19.001 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.994+0000 7fc9027fc700 1 -- 172.21.15.47:0/1024551614 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc8f400d9a0 con 0x7fc904066fe0 2022-06-29T15:48:19.001 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.994+0000 7fc908ec9700 1 -- 172.21.15.47:0/1024551614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc904066fe0 msgr2=0x7fc9040b4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:19.001 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.994+0000 7fc908ec9700 1 --2- 172.21.15.47:0/1024551614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc904066fe0 0x7fc9040b4b40 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 rx=0x7fc8f4002f70 tx=0x7fc8f4006e50).stop 2022-06-29T15:48:19.001 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.994+0000 7fc908ec9700 1 -- 172.21.15.47:0/1024551614 shutdown_connections 2022-06-29T15:48:19.001 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.994+0000 7fc908ec9700 1 --2- 172.21.15.47:0/1024551614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc904066fe0 0x7fc9040b4b40 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:19.002 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.994+0000 7fc908ec9700 1 --2- 172.21.15.47:0/1024551614 >> v2:172.21.15.64:3300/0 conn(0x7fc9040b64c0 0x7fc9040b5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:19.002 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.994+0000 7fc908ec9700 1 -- 172.21.15.47:0/1024551614 >> 172.21.15.47:0/1024551614 conn(0x7fc904060e00 msgr2=0x7fc9040611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:19.002 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.994+0000 7fc908ec9700 1 -- 172.21.15.47:0/1024551614 shutdown_connections 2022-06-29T15:48:19.002 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.994+0000 7fc908ec9700 1 -- 172.21.15.47:0/1024551614 wait complete. 2022-06-29T15:48:19.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.994+0000 7fc908ec9700 1 Processor -- start 2022-06-29T15:48:19.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.994+0000 7fc908ec9700 1 -- start start 2022-06-29T15:48:19.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.994+0000 7fc908ec9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc904066fe0 0x7fc9040b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:19.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.994+0000 7fc908ec9700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc9040b64c0 0x7fc9040ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:19.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.994+0000 7fc908ec9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc904143150 0x7fc9040add60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:19.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.994+0000 7fc908ec9700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9040b3990 con 0x7fc904143150 2022-06-29T15:48:19.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.994+0000 7fc908ec9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9041466a0 con 0x7fc904066fe0 2022-06-29T15:48:19.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.994+0000 7fc908ec9700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc904145520 con 0x7fc9040b64c0 2022-06-29T15:48:19.009 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.998+0000 7fc903fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc904143150 0x7fc9040add60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:19.009 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.998+0000 7fc902ffd700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc9040b64c0 0x7fc9040ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:19.009 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.998+0000 7fc903fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc904143150 0x7fc9040add60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49990/0 (socket says 172.21.15.47:49990) 2022-06-29T15:48:19.009 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.998+0000 7fc903fff700 1 -- 172.21.15.47:0/2256973269 learned_addr learned my addr 172.21.15.47:0/2256973269 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:19.010 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.998+0000 7fc9037fe700 1 --2- 172.21.15.47:0/2256973269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc904066fe0 0x7fc9040b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:19.010 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.998+0000 7fc903fff700 1 -- 172.21.15.47:0/2256973269 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc9040b64c0 msgr2=0x7fc9040ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:19.010 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.998+0000 7fc903fff700 1 --2- 172.21.15.47:0/2256973269 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc9040b64c0 0x7fc9040ad870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:19.010 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.998+0000 7fc903fff700 1 -- 172.21.15.47:0/2256973269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc904066fe0 msgr2=0x7fc9040b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:19.011 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.998+0000 7fc903fff700 1 --2- 172.21.15.47:0/2256973269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc904066fe0 0x7fc9040b2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:19.011 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.998+0000 7fc903fff700 1 -- 172.21.15.47:0/2256973269 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f400b1e0 con 0x7fc904143150 2022-06-29T15:48:19.011 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.998+0000 7fc903fff700 1 --2- 172.21.15.47:0/2256973269 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc904143150 0x7fc9040add60 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 rx=0x7fc9040b4a20 tx=0x7fc8fc007f70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:19.011 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.998+0000 7fc900ff9700 1 -- 172.21.15.47:0/2256973269 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc8fc006aa0 con 0x7fc904143150 2022-06-29T15:48:19.012 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.998+0000 7fc900ff9700 1 -- 172.21.15.47:0/2256973269 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8fc006c00 con 0x7fc904143150 2022-06-29T15:48:19.012 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.998+0000 7fc908ec9700 1 -- 172.21.15.47:0/2256973269 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9040b00f0 con 0x7fc904143150 2022-06-29T15:48:19.012 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.998+0000 7fc900ff9700 1 -- 172.21.15.47:0/2256973269 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc8fc007030 con 0x7fc904143150 2022-06-29T15:48:19.012 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.998+0000 7fc908ec9700 1 -- 172.21.15.47:0/2256973269 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9040ae250 con 0x7fc904143150 2022-06-29T15:48:19.013 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.998+0000 7fc900ff9700 1 -- 172.21.15.47:0/2256973269 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc8fc010030 con 0x7fc904143150 2022-06-29T15:48:19.013 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.998+0000 7fc908ec9700 1 -- 172.21.15.47:0/2256973269 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9041468a0 con 0x7fc904143150 2022-06-29T15:48:19.013 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.998+0000 7fc900ff9700 1 --2- 172.21.15.47:0/2256973269 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc8e8053a70 0x7fc8e8055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:19.013 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.998+0000 7fc900ff9700 1 -- 172.21.15.47:0/2256973269 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc8fc06bfe0 con 0x7fc904143150 2022-06-29T15:48:19.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:18.998+0000 7fc9037fe700 1 --2- 172.21.15.47:0/2256973269 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc8e8053a70 0x7fc8e8055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:19.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.002+0000 7fc9037fe700 1 --2- 172.21.15.47:0/2256973269 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc8e8053a70 0x7fc8e8055f30 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7fc8ec00b000 tx=0x7fc8ec006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:19.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.006+0000 7fc900ff9700 1 -- 172.21.15.47:0/2256973269 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc8fc042cb0 con 0x7fc904143150 2022-06-29T15:48:19.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.174+0000 7fc908ec9700 1 -- 172.21.15.47:0/2256973269 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc904145360 con 0x7fc904143150 2022-06-29T15:48:19.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.358+0000 7fc900ff9700 1 -- 172.21.15.47:0/2256973269 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v35) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc8fc0462d0 con 0x7fc904143150 2022-06-29T15:48:19.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.362+0000 7fc908ec9700 1 -- 172.21.15.47:0/2256973269 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc8e8053a70 msgr2=0x7fc8e8055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:19.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.362+0000 7fc908ec9700 1 --2- 172.21.15.47:0/2256973269 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc8e8053a70 0x7fc8e8055f30 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7fc8ec00b000 tx=0x7fc8ec006df0).stop 2022-06-29T15:48:19.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.362+0000 7fc908ec9700 1 -- 172.21.15.47:0/2256973269 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc904143150 msgr2=0x7fc9040add60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:19.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.362+0000 7fc908ec9700 1 --2- 172.21.15.47:0/2256973269 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc904143150 0x7fc9040add60 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 rx=0x7fc9040b4a20 tx=0x7fc8fc007f70).stop 2022-06-29T15:48:19.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.362+0000 7fc908ec9700 1 -- 172.21.15.47:0/2256973269 shutdown_connections 2022-06-29T15:48:19.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.362+0000 7fc908ec9700 1 --2- 172.21.15.47:0/2256973269 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc904143150 0x7fc9040add60 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:19.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.362+0000 7fc908ec9700 1 --2- 172.21.15.47:0/2256973269 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc9040b64c0 0x7fc9040ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:19.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.362+0000 7fc908ec9700 1 --2- 172.21.15.47:0/2256973269 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc8e8053a70 0x7fc8e8055f30 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:19.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.362+0000 7fc908ec9700 1 --2- 172.21.15.47:0/2256973269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc904066fe0 0x7fc9040b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:19.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.362+0000 7fc908ec9700 1 -- 172.21.15.47:0/2256973269 >> 172.21.15.47:0/2256973269 conn(0x7fc904060e00 msgr2=0x7fc9041477e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:19.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.362+0000 7fc908ec9700 1 -- 172.21.15.47:0/2256973269 shutdown_connections 2022-06-29T15:48:19.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.362+0000 7fc908ec9700 1 -- 172.21.15.47:0/2256973269 wait complete. 2022-06-29T15:48:19.378 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:48:19.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.474+0000 7ffbdca23700 1 Processor -- start 2022-06-29T15:48:19.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.474+0000 7ffbdca23700 1 -- start start 2022-06-29T15:48:19.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.478+0000 7ffbdca23700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffbd8143bd0 0x7ffbd805c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:19.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.478+0000 7ffbdca23700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ffbd80b2c40 0x7ffbd80b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:19.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.478+0000 7ffbdca23700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbd8142f90 con 0x7ffbd805c510 2022-06-29T15:48:19.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.478+0000 7ffbdca23700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbd81431f0 con 0x7ffbd80b2c40 2022-06-29T15:48:19.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.478+0000 7ffbdca23700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd8143490 con 0x7ffbd8143bd0 2022-06-29T15:48:19.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.478+0000 7ffbd77fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ffbd805c510 0x7ffbd80b4100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35548/0 (socket says 172.21.15.47:35548) 2022-06-29T15:48:19.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.478+0000 7ffbd77fe700 1 -- 172.21.15.47:0/694495427 learned_addr learned my addr 172.21.15.47:0/694495427 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:48:19.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.478+0000 7ffbd7fff700 1 --2- 172.21.15.47:0/694495427 >> v2:172.21.15.64:3300/0 conn(0x7ffbd80b2c40 0x7ffbd80b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:19.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.478+0000 7ffbd6ffd700 1 --2- 172.21.15.47:0/694495427 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffbd8143bd0 0x7ffbd805c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:19.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.478+0000 7ffbd67fc700 1 -- 172.21.15.47:0/694495427 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2916906059 0 0) 0x7ffbd8142f90 con 0x7ffbd805c510 2022-06-29T15:48:19.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.478+0000 7ffbd67fc700 1 -- 172.21.15.47:0/694495427 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffbc4003260 con 0x7ffbd805c510 2022-06-29T15:48:19.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.478+0000 7ffbd7fff700 1 -- 172.21.15.47:0/694495427 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffbd8143bd0 msgr2=0x7ffbd805c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:19.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.478+0000 7ffbd7fff700 1 --2- 172.21.15.47:0/694495427 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffbd8143bd0 0x7ffbd805c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:19.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.478+0000 7ffbd7fff700 1 -- 172.21.15.47:0/694495427 >> v1:172.21.15.47:6789/0 conn(0x7ffbd805c510 legacy=0x7ffbd80b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:19.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.478+0000 7ffbd7fff700 1 -- 172.21.15.47:0/694495427 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd8143730 con 0x7ffbd80b2c40 2022-06-29T15:48:19.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.478+0000 7ffbd7fff700 1 --2- 172.21.15.47:0/694495427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffbd80b2c40 0x7ffbd80b30d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7ffbd00032f0 tx=0x7ffbd00091d0).ready entity=mon.1 client_cookie=b228111e005a109b server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:19.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.478+0000 7ffbd67fc700 1 -- 172.21.15.47:0/694495427 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffbd000b600 con 0x7ffbd80b2c40 2022-06-29T15:48:19.497 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.478+0000 7ffbd67fc700 1 -- 172.21.15.47:0/694495427 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbd000b760 con 0x7ffbd80b2c40 2022-06-29T15:48:19.497 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.478+0000 7ffbd67fc700 1 -- 172.21.15.47:0/694495427 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffbd000b8d0 con 0x7ffbd80b2c40 2022-06-29T15:48:19.498 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.478+0000 7ffbdca23700 1 -- 172.21.15.47:0/694495427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffbd80b2c40 msgr2=0x7ffbd80b30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:19.498 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.478+0000 7ffbdca23700 1 --2- 172.21.15.47:0/694495427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffbd80b2c40 0x7ffbd80b30d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7ffbd00032f0 tx=0x7ffbd00091d0).stop 2022-06-29T15:48:19.499 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.478+0000 7ffbdca23700 1 -- 172.21.15.47:0/694495427 shutdown_connections 2022-06-29T15:48:19.499 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.478+0000 7ffbdca23700 1 --2- 172.21.15.47:0/694495427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffbd80b2c40 0x7ffbd80b30d0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:19.499 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.478+0000 7ffbdca23700 1 --2- 172.21.15.47:0/694495427 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffbd8143bd0 0x7ffbd805c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:19.499 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.478+0000 7ffbdca23700 1 -- 172.21.15.47:0/694495427 >> 172.21.15.47:0/694495427 conn(0x7ffbd8060e00 msgr2=0x7ffbd80611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:19.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.478+0000 7ffbdca23700 1 -- 172.21.15.47:0/694495427 shutdown_connections 2022-06-29T15:48:19.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.478+0000 7ffbdca23700 1 -- 172.21.15.47:0/694495427 wait complete. 2022-06-29T15:48:19.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.482+0000 7ffbdca23700 1 Processor -- start 2022-06-29T15:48:19.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.482+0000 7ffbdca23700 1 -- start start 2022-06-29T15:48:19.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.482+0000 7ffbdca23700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffbd805c510 0x7ffbd80b8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:19.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.482+0000 7ffbdca23700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffbd80b2c40 0x7ffbd8155b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:19.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.482+0000 7ffbdca23700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffbd8143bd0 0x7ffbd8156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:19.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.482+0000 7ffbdca23700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd813ed60 con 0x7ffbd8143bd0 2022-06-29T15:48:19.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.482+0000 7ffbdca23700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd813ef90 con 0x7ffbd80b2c40 2022-06-29T15:48:19.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.482+0000 7ffbdca23700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd813f230 con 0x7ffbd805c510 2022-06-29T15:48:19.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.482+0000 7ffbd7fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffbd8143bd0 0x7ffbd8156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:19.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.482+0000 7ffbd7fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffbd8143bd0 0x7ffbd8156060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50002/0 (socket says 172.21.15.47:50002) 2022-06-29T15:48:19.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.482+0000 7ffbd7fff700 1 -- 172.21.15.47:0/269953424 learned_addr learned my addr 172.21.15.47:0/269953424 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:19.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.482+0000 7ffbd77fe700 1 --2- 172.21.15.47:0/269953424 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffbd805c510 0x7ffbd80b8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:19.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.482+0000 7ffbd6ffd700 1 --2- 172.21.15.47:0/269953424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffbd80b2c40 0x7ffbd8155b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:19.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.482+0000 7ffbd7fff700 1 -- 172.21.15.47:0/269953424 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffbd805c510 msgr2=0x7ffbd80b8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:19.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.482+0000 7ffbd7fff700 1 --2- 172.21.15.47:0/269953424 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffbd805c510 0x7ffbd80b8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:19.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.482+0000 7ffbd7fff700 1 -- 172.21.15.47:0/269953424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffbd80b2c40 msgr2=0x7ffbd8155b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:19.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.482+0000 7ffbd7fff700 1 --2- 172.21.15.47:0/269953424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffbd80b2c40 0x7ffbd8155b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:19.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.482+0000 7ffbd7fff700 1 -- 172.21.15.47:0/269953424 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd0009470 con 0x7ffbd8143bd0 2022-06-29T15:48:19.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.482+0000 7ffbd7fff700 1 --2- 172.21.15.47:0/269953424 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffbd8143bd0 0x7ffbd8156060 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 rx=0x7ffbd00032f0 tx=0x7ffbd0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:19.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.482+0000 7ffbd4ff9700 1 -- 172.21.15.47:0/269953424 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffbd000b600 con 0x7ffbd8143bd0 2022-06-29T15:48:19.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.482+0000 7ffbd4ff9700 1 -- 172.21.15.47:0/269953424 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbd0011b60 con 0x7ffbd8143bd0 2022-06-29T15:48:19.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.482+0000 7ffbdca23700 1 -- 172.21.15.47:0/269953424 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbd81556d0 con 0x7ffbd8143bd0 2022-06-29T15:48:19.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.482+0000 7ffbd4ff9700 1 -- 172.21.15.47:0/269953424 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffbd001f3f0 con 0x7ffbd8143bd0 2022-06-29T15:48:19.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.486+0000 7ffbdca23700 1 -- 172.21.15.47:0/269953424 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbd81521e0 con 0x7ffbd8143bd0 2022-06-29T15:48:19.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.486+0000 7ffbdca23700 1 -- 172.21.15.47:0/269953424 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbd8140350 con 0x7ffbd8143bd0 2022-06-29T15:48:19.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.486+0000 7ffbd4ff9700 1 -- 172.21.15.47:0/269953424 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ffbd0026030 con 0x7ffbd8143bd0 2022-06-29T15:48:19.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.486+0000 7ffbd4ff9700 1 --2- 172.21.15.47:0/269953424 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ffbb80539e0 0x7ffbb8055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:19.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.486+0000 7ffbd4ff9700 1 -- 172.21.15.47:0/269953424 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ffbd006caf0 con 0x7ffbd8143bd0 2022-06-29T15:48:19.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.486+0000 7ffbd77fe700 1 --2- 172.21.15.47:0/269953424 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ffbb80539e0 0x7ffbb8055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:19.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.486+0000 7ffbd77fe700 1 --2- 172.21.15.47:0/269953424 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ffbb80539e0 0x7ffbb8055ea0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7ffbc800d000 tx=0x7ffbc8009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:19.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.498+0000 7ffbd4ff9700 1 -- 172.21.15.47:0/269953424 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ffbd8140350 con 0x7ffbd8143bd0 2022-06-29T15:48:19.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.662+0000 7ffbdca23700 1 -- 172.21.15.47:0/269953424 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffbd8140350 con 0x7ffbd8143bd0 2022-06-29T15:48:19.666 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.662+0000 7ffbd4ff9700 1 -- 172.21.15.47:0/269953424 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+2493 (secure 0 0 0) 0x7ffbd8140350 con 0x7ffbd8143bd0 2022-06-29T15:48:19.666 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:48:19.667 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":35,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6291,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.47:6835/2910037081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2910037081},{"type":"v1","addr":"172.21.15.47:6835","nonce":2910037081}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":34},{"gid":6300,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/359532891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":359532891},{"type":"v1","addr":"172.21.15.47:6837","nonce":359532891}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":33}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:48:18.345845+0000","modified":"2022-06-29T15:48:19.349718+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":55562},"failed":[],"damaged":[],"stopped":[],"info":{"gid_55562":{"gid":55562,"name":"b","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6833/3871453337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3871453337},{"type":"v1","addr":"172.21.15.64:6833","nonce":3871453337}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2022-06-29T15:48:19.668 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.666+0000 7ffbdca23700 1 -- 172.21.15.47:0/269953424 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ffbb80539e0 msgr2=0x7ffbb8055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:19.668 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.666+0000 7ffbdca23700 1 --2- 172.21.15.47:0/269953424 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ffbb80539e0 0x7ffbb8055ea0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7ffbc800d000 tx=0x7ffbc8009040).stop 2022-06-29T15:48:19.669 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.666+0000 7ffbdca23700 1 -- 172.21.15.47:0/269953424 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffbd8143bd0 msgr2=0x7ffbd8156060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:19.669 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.666+0000 7ffbdca23700 1 --2- 172.21.15.47:0/269953424 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffbd8143bd0 0x7ffbd8156060 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 rx=0x7ffbd00032f0 tx=0x7ffbd0016000).stop 2022-06-29T15:48:19.669 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.666+0000 7ffbdca23700 1 -- 172.21.15.47:0/269953424 shutdown_connections 2022-06-29T15:48:19.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.666+0000 7ffbdca23700 1 --2- 172.21.15.47:0/269953424 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffbd8143bd0 0x7ffbd8156060 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:19.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.666+0000 7ffbdca23700 1 --2- 172.21.15.47:0/269953424 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ffbb80539e0 0x7ffbb8055ea0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:19.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.666+0000 7ffbdca23700 1 --2- 172.21.15.47:0/269953424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffbd80b2c40 0x7ffbd8155b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:19.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.666+0000 7ffbdca23700 1 --2- 172.21.15.47:0/269953424 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffbd805c510 0x7ffbd80b8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:19.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.666+0000 7ffbdca23700 1 -- 172.21.15.47:0/269953424 >> 172.21.15.47:0/269953424 conn(0x7ffbd8060e00 msgr2=0x7ffbd81427c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:19.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.666+0000 7ffbdca23700 1 -- 172.21.15.47:0/269953424 shutdown_connections 2022-06-29T15:48:19.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.666+0000 7ffbdca23700 1 -- 172.21.15.47:0/269953424 wait complete. 2022-06-29T15:48:19.672 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 35 2022-06-29T15:48:19.679 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:48:19.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.778+0000 7f5dd6860700 1 Processor -- start 2022-06-29T15:48:19.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.778+0000 7f5dd6860700 1 -- start start 2022-06-29T15:48:19.784 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.778+0000 7f5dd6860700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5dd00619b0 0x7f5dd00606e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:19.785 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.778+0000 7f5dd6860700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5dd00b0f30 0x7f5dd01408e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:19.785 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.778+0000 7f5dd6860700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dd0061d90 con 0x7f5dd00b1a80 2022-06-29T15:48:19.785 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.778+0000 7f5dd6860700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dd0060d80 con 0x7f5dd00b0f30 2022-06-29T15:48:19.786 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.778+0000 7f5dd6860700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dd013ff00 con 0x7f5dd00619b0 2022-06-29T15:48:19.786 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.778+0000 7f5dcf7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5dd00b0f30 0x7f5dd01408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:19.786 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.778+0000 7f5dcf7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5dd00b0f30 0x7f5dd01408e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50008/0 (socket says 172.21.15.47:50008) 2022-06-29T15:48:19.787 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.778+0000 7f5dcf7fe700 1 -- 172.21.15.47:0/1419012410 learned_addr learned my addr 172.21.15.47:0/1419012410 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:19.787 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.778+0000 7f5dcffff700 1 --2- 172.21.15.47:0/1419012410 >> v2:172.21.15.64:3300/0 conn(0x7f5dd00619b0 0x7f5dd00606e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:19.787 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.778+0000 7f5dcf7fe700 1 -- 172.21.15.47:0/1419012410 >> v2:172.21.15.64:3300/0 conn(0x7f5dd00619b0 msgr2=0x7f5dd00606e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:19.787 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.778+0000 7f5dcf7fe700 1 --2- 172.21.15.47:0/1419012410 >> v2:172.21.15.64:3300/0 conn(0x7f5dd00619b0 0x7f5dd00606e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:19.788 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.778+0000 7f5dcf7fe700 1 -- 172.21.15.47:0/1419012410 >> v1:172.21.15.64:6789/0 conn(0x7f5dd00b1a80 legacy=0x7f5dd0139a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:19.788 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.778+0000 7f5dcf7fe700 1 -- 172.21.15.47:0/1419012410 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd01401a0 con 0x7f5dd00b0f30 2022-06-29T15:48:19.788 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.778+0000 7f5dcf7fe700 1 --2- 172.21.15.47:0/1419012410 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5dd00b0f30 0x7f5dd01408e0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 rx=0x7f5dc80032f0 tx=0x7f5dc800b530).ready entity=mon.0 client_cookie=3f37f89dea677bd1 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:19.788 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.778+0000 7f5dceffd700 1 -- 172.21.15.47:0/1419012410 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5dc800ddf0 con 0x7f5dd00b0f30 2022-06-29T15:48:19.788 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.778+0000 7f5dceffd700 1 -- 172.21.15.47:0/1419012410 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc8010470 con 0x7f5dd00b0f30 2022-06-29T15:48:19.789 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.778+0000 7f5dceffd700 1 -- 172.21.15.47:0/1419012410 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5dc80107d0 con 0x7f5dd00b0f30 2022-06-29T15:48:19.789 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.778+0000 7f5dd6860700 1 -- 172.21.15.47:0/1419012410 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5dd00b0f30 msgr2=0x7f5dd01408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:19.790 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.778+0000 7f5dd6860700 1 --2- 172.21.15.47:0/1419012410 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5dd00b0f30 0x7f5dd01408e0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 rx=0x7f5dc80032f0 tx=0x7f5dc800b530).stop 2022-06-29T15:48:19.791 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.778+0000 7f5dd6860700 1 -- 172.21.15.47:0/1419012410 shutdown_connections 2022-06-29T15:48:19.791 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.778+0000 7f5dd6860700 1 --2- 172.21.15.47:0/1419012410 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5dd00b0f30 0x7f5dd01408e0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:19.791 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.778+0000 7f5dd6860700 1 --2- 172.21.15.47:0/1419012410 >> v2:172.21.15.64:3300/0 conn(0x7f5dd00619b0 0x7f5dd00606e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:19.792 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.778+0000 7f5dd6860700 1 -- 172.21.15.47:0/1419012410 >> 172.21.15.47:0/1419012410 conn(0x7f5dd005c510 msgr2=0x7f5dd0063800 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:19.792 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.778+0000 7f5dd6860700 1 -- 172.21.15.47:0/1419012410 shutdown_connections 2022-06-29T15:48:19.793 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.778+0000 7f5dd6860700 1 -- 172.21.15.47:0/1419012410 wait complete. 2022-06-29T15:48:19.798 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.782+0000 7f5dd6860700 1 Processor -- start 2022-06-29T15:48:19.799 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.782+0000 7f5dd6860700 1 -- start start 2022-06-29T15:48:19.799 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.782+0000 7f5dd6860700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5dd00619b0 0x7f5dd00b5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:19.799 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.782+0000 7f5dd6860700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5dd00b0f30 0x7f5dd01352f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:19.799 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.782+0000 7f5dd6860700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5dd00b1a80 0x7f5dd01357e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:19.800 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.782+0000 7f5dd6860700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd00b1ef0 con 0x7f5dd00b0f30 2022-06-29T15:48:19.800 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.782+0000 7f5dd6860700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd00b5b20 con 0x7f5dd00619b0 2022-06-29T15:48:19.800 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.786+0000 7f5dd6860700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd013ac60 con 0x7f5dd00b1a80 2022-06-29T15:48:19.800 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.786+0000 7f5dcf7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5dd00b0f30 0x7f5dd01352f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:19.801 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.786+0000 7f5dcf7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5dd00b0f30 0x7f5dd01352f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50016/0 (socket says 172.21.15.47:50016) 2022-06-29T15:48:19.801 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.786+0000 7f5dcf7fe700 1 -- 172.21.15.47:0/1504843515 learned_addr learned my addr 172.21.15.47:0/1504843515 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:19.801 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.786+0000 7f5dd4dfd700 1 --2- 172.21.15.47:0/1504843515 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5dd00b1a80 0x7f5dd01357e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:19.802 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.786+0000 7f5dcffff700 1 --2- 172.21.15.47:0/1504843515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5dd00619b0 0x7f5dd00b5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:19.802 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.786+0000 7f5dcf7fe700 1 -- 172.21.15.47:0/1504843515 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5dd00b1a80 msgr2=0x7f5dd01357e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:19.802 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.786+0000 7f5dcf7fe700 1 --2- 172.21.15.47:0/1504843515 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5dd00b1a80 0x7f5dd01357e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:19.802 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.786+0000 7f5dcf7fe700 1 -- 172.21.15.47:0/1504843515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5dd00619b0 msgr2=0x7f5dd00b5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:19.803 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.786+0000 7f5dcf7fe700 1 --2- 172.21.15.47:0/1504843515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5dd00619b0 0x7f5dd00b5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:19.803 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.786+0000 7f5dcf7fe700 1 -- 172.21.15.47:0/1504843515 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc800b7d0 con 0x7f5dd00b0f30 2022-06-29T15:48:19.803 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.786+0000 7f5dcf7fe700 1 --2- 172.21.15.47:0/1504843515 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5dd00b0f30 0x7f5dd01352f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 rx=0x7f5dc80032f0 tx=0x7f5dd00ae8b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:19.803 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.786+0000 7f5dcd7fa700 1 -- 172.21.15.47:0/1504843515 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5dc8018610 con 0x7f5dd00b0f30 2022-06-29T15:48:19.804 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.786+0000 7f5dcd7fa700 1 -- 172.21.15.47:0/1504843515 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc8018770 con 0x7f5dd00b0f30 2022-06-29T15:48:19.804 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.786+0000 7f5dd6860700 1 -- 172.21.15.47:0/1504843515 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dd013cce0 con 0x7f5dd00b0f30 2022-06-29T15:48:19.804 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.786+0000 7f5dcd7fa700 1 -- 172.21.15.47:0/1504843515 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5dc8018ad0 con 0x7f5dd00b0f30 2022-06-29T15:48:19.804 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.786+0000 7f5dd6860700 1 -- 172.21.15.47:0/1504843515 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dd013d220 con 0x7f5dd00b0f30 2022-06-29T15:48:19.805 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.786+0000 7f5dd6860700 1 -- 172.21.15.47:0/1504843515 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dd00573d0 con 0x7f5dd00b0f30 2022-06-29T15:48:19.805 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.786+0000 7f5dcd7fa700 1 -- 172.21.15.47:0/1504843515 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5dc8021020 con 0x7f5dd00b0f30 2022-06-29T15:48:19.806 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.790+0000 7f5dcd7fa700 1 --2- 172.21.15.47:0/1504843515 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5db4053a70 0x7f5db4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:19.806 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.790+0000 7f5dcffff700 1 --2- 172.21.15.47:0/1504843515 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5db4053a70 0x7f5db4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:19.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.790+0000 7f5dcd7fa700 1 -- 172.21.15.47:0/1504843515 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f5dc8070180 con 0x7f5dd00b0f30 2022-06-29T15:48:19.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.790+0000 7f5dcffff700 1 --2- 172.21.15.47:0/1504843515 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5db4053a70 0x7f5db4055f30 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f5db800b000 tx=0x7f5db8006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:19.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.798+0000 7f5dcd7fa700 1 -- 172.21.15.47:0/1504843515 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5dc8046ea0 con 0x7f5dd00b0f30 2022-06-29T15:48:19.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.926+0000 7f5dd6860700 1 -- 172.21.15.47:0/1504843515 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5dd0142f20 con 0x7f5dd00b0f30 2022-06-29T15:48:19.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.926+0000 7f5dcd7fa700 1 -- 172.21.15.47:0/1504843515 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+16274 (secure 0 0 0) 0x7f5dc804a4c0 con 0x7f5dd00b0f30 2022-06-29T15:48:19.930 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:48:19.931 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":78,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:48:19.348792+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:48:16.753707+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2022-06-29T15:48:17.935585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:48:19.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.930+0000 7f5dd6860700 1 -- 172.21.15.47:0/1504843515 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5db4053a70 msgr2=0x7f5db4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:19.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.930+0000 7f5dd6860700 1 --2- 172.21.15.47:0/1504843515 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5db4053a70 0x7f5db4055f30 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f5db800b000 tx=0x7f5db8006df0).stop 2022-06-29T15:48:19.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.930+0000 7f5dd6860700 1 -- 172.21.15.47:0/1504843515 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5dd00b0f30 msgr2=0x7f5dd01352f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:19.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.930+0000 7f5dd6860700 1 --2- 172.21.15.47:0/1504843515 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5dd00b0f30 0x7f5dd01352f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 rx=0x7f5dc80032f0 tx=0x7f5dd00ae8b0).stop 2022-06-29T15:48:19.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.930+0000 7f5dd6860700 1 -- 172.21.15.47:0/1504843515 shutdown_connections 2022-06-29T15:48:19.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.930+0000 7f5dd6860700 1 --2- 172.21.15.47:0/1504843515 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5dd00b1a80 0x7f5dd01357e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:19.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.930+0000 7f5dd6860700 1 --2- 172.21.15.47:0/1504843515 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5dd00b0f30 0x7f5dd01352f0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:19.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.930+0000 7f5dd6860700 1 --2- 172.21.15.47:0/1504843515 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5db4053a70 0x7f5db4055f30 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:19.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.930+0000 7f5dd6860700 1 --2- 172.21.15.47:0/1504843515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5dd00619b0 0x7f5dd00b5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:19.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.930+0000 7f5dd6860700 1 -- 172.21.15.47:0/1504843515 >> 172.21.15.47:0/1504843515 conn(0x7f5dd005c510 msgr2=0x7f5dd0139a70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:19.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.930+0000 7f5dd6860700 1 -- 172.21.15.47:0/1504843515 shutdown_connections 2022-06-29T15:48:19.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:19.930+0000 7f5dd6860700 1 -- 172.21.15.47:0/1504843515 wait complete. 2022-06-29T15:48:19.944 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-29T15:48:20.059 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.054+0000 7f4294804700 1 Processor -- start 2022-06-29T15:48:20.059 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.054+0000 7f4294804700 1 -- start start 2022-06-29T15:48:20.064 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.058+0000 7f4294804700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4290060ee0 0x7f42900612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:20.064 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.058+0000 7f4294804700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f42900b4300 0x7f42901471a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:20.064 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.058+0000 7f4294804700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4290066f70 0x7f42901430e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:20.064 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.058+0000 7f4294804700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4290067400 con 0x7f4290060ee0 2022-06-29T15:48:20.065 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.058+0000 7f4294804700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4290067610 con 0x7f42900b4300 2022-06-29T15:48:20.065 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.058+0000 7f4294804700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42901467c0 con 0x7f4290066f70 2022-06-29T15:48:20.065 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.058+0000 7f428e59c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4290060ee0 0x7f42900612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:20.065 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.058+0000 7f428e59c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4290060ee0 0x7f42900612c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50024/0 (socket says 172.21.15.47:50024) 2022-06-29T15:48:20.066 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.058+0000 7f428e59c700 1 -- 172.21.15.47:0/1643312155 learned_addr learned my addr 172.21.15.47:0/1643312155 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:20.066 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.058+0000 7f428ed9d700 1 --2- 172.21.15.47:0/1643312155 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4290066f70 0x7f42901430e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:20.066 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.058+0000 7f428dd9b700 1 --2- 172.21.15.47:0/1643312155 >> v2:172.21.15.64:3300/0 conn(0x7f42900b4300 0x7f42901471a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:20.067 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.058+0000 7f428e59c700 1 -- 172.21.15.47:0/1643312155 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4290066f70 msgr2=0x7f42901430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:20.067 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.058+0000 7f428e59c700 1 --2- 172.21.15.47:0/1643312155 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4290066f70 0x7f42901430e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:20.067 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.058+0000 7f428e59c700 1 -- 172.21.15.47:0/1643312155 >> v2:172.21.15.64:3300/0 conn(0x7f42900b4300 msgr2=0x7f42901471a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:20.067 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.058+0000 7f428e59c700 1 --2- 172.21.15.47:0/1643312155 >> v2:172.21.15.64:3300/0 conn(0x7f42900b4300 0x7f42901471a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:20.068 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.058+0000 7f428e59c700 1 -- 172.21.15.47:0/1643312155 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4290146a60 con 0x7f4290060ee0 2022-06-29T15:48:20.068 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.058+0000 7f428dd9b700 1 --2- 172.21.15.47:0/1643312155 >> v2:172.21.15.64:3300/0 conn(0x7f42900b4300 0x7f42901471a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:48:20.068 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.058+0000 7f428e59c700 1 --2- 172.21.15.47:0/1643312155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4290060ee0 0x7f42900612c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 rx=0x7f42900ae790 tx=0x7f4280006eb0).ready entity=mon.0 client_cookie=f649645c3f962701 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:20.068 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.058+0000 7f428d59a700 1 -- 172.21.15.47:0/1643312155 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f428000d570 con 0x7f4290060ee0 2022-06-29T15:48:20.068 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.058+0000 7f428d59a700 1 -- 172.21.15.47:0/1643312155 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4280000e90 con 0x7f4290060ee0 2022-06-29T15:48:20.069 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.058+0000 7f428d59a700 1 -- 172.21.15.47:0/1643312155 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f428000d900 con 0x7f4290060ee0 2022-06-29T15:48:20.069 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.058+0000 7f4294804700 1 -- 172.21.15.47:0/1643312155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4290060ee0 msgr2=0x7f42900612c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:20.069 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.058+0000 7f4294804700 1 --2- 172.21.15.47:0/1643312155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4290060ee0 0x7f42900612c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 rx=0x7f42900ae790 tx=0x7f4280006eb0).stop 2022-06-29T15:48:20.070 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.058+0000 7f4294804700 1 -- 172.21.15.47:0/1643312155 shutdown_connections 2022-06-29T15:48:20.070 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.058+0000 7f4294804700 1 --2- 172.21.15.47:0/1643312155 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4290066f70 0x7f42901430e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:20.070 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.058+0000 7f4294804700 1 --2- 172.21.15.47:0/1643312155 >> v2:172.21.15.64:3300/0 conn(0x7f42900b4300 0x7f42901471a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:20.071 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.058+0000 7f4294804700 1 --2- 172.21.15.47:0/1643312155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4290060ee0 0x7f42900612c0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:20.071 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.058+0000 7f4294804700 1 -- 172.21.15.47:0/1643312155 >> 172.21.15.47:0/1643312155 conn(0x7f429005c510 msgr2=0x7f429005f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:20.071 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.058+0000 7f4294804700 1 -- 172.21.15.47:0/1643312155 shutdown_connections 2022-06-29T15:48:20.071 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.058+0000 7f4294804700 1 -- 172.21.15.47:0/1643312155 wait complete. 2022-06-29T15:48:20.075 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.062+0000 7f4294804700 1 Processor -- start 2022-06-29T15:48:20.075 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.062+0000 7f4294804700 1 -- start start 2022-06-29T15:48:20.075 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.062+0000 7f4294804700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4290060ee0 0x7f429013fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:20.076 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.066+0000 7f4294804700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4290066f70 0x7f4290145a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:20.076 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.066+0000 7f4294804700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f42900b4300 0x7f4290145f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:20.076 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.066+0000 7f4294804700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42901453a0 con 0x7f4290060ee0 2022-06-29T15:48:20.076 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.066+0000 7f4294804700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4290146470 con 0x7f4290066f70 2022-06-29T15:48:20.076 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.066+0000 7f4294804700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4290140520 con 0x7f42900b4300 2022-06-29T15:48:20.077 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.066+0000 7f428e59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4290060ee0 0x7f429013fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:20.077 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.066+0000 7f428e59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4290060ee0 0x7f429013fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50028/0 (socket says 172.21.15.47:50028) 2022-06-29T15:48:20.077 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.066+0000 7f428e59c700 1 -- 172.21.15.47:0/121444770 learned_addr learned my addr 172.21.15.47:0/121444770 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:20.078 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.066+0000 7f428ed9d700 1 --2- 172.21.15.47:0/121444770 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f42900b4300 0x7f4290145f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:20.078 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.066+0000 7f428dd9b700 1 --2- 172.21.15.47:0/121444770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4290066f70 0x7f4290145a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:20.078 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.066+0000 7f428e59c700 1 -- 172.21.15.47:0/121444770 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f42900b4300 msgr2=0x7f4290145f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:20.079 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.066+0000 7f428e59c700 1 --2- 172.21.15.47:0/121444770 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f42900b4300 0x7f4290145f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:20.079 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.066+0000 7f428e59c700 1 -- 172.21.15.47:0/121444770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4290066f70 msgr2=0x7f4290145a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:20.080 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.066+0000 7f428e59c700 1 --2- 172.21.15.47:0/121444770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4290066f70 0x7f4290145a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:20.080 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.066+0000 7f428e59c700 1 -- 172.21.15.47:0/121444770 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f428000b1e0 con 0x7f4290060ee0 2022-06-29T15:48:20.081 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.066+0000 7f428e59c700 1 --2- 172.21.15.47:0/121444770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4290060ee0 0x7f429013fb60 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 rx=0x7f42900ae790 tx=0x7f4280015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:20.081 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.066+0000 7f42877fe700 1 -- 172.21.15.47:0/121444770 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f42800116e0 con 0x7f4290060ee0 2022-06-29T15:48:20.082 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.066+0000 7f42877fe700 1 -- 172.21.15.47:0/121444770 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4280011840 con 0x7f4290060ee0 2022-06-29T15:48:20.082 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.066+0000 7f4294804700 1 -- 172.21.15.47:0/121444770 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42901455d0 con 0x7f4290060ee0 2022-06-29T15:48:20.082 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.066+0000 7f42877fe700 1 -- 172.21.15.47:0/121444770 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4280011ba0 con 0x7f4290060ee0 2022-06-29T15:48:20.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.066+0000 7f4294804700 1 -- 172.21.15.47:0/121444770 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4290152430 con 0x7f4290060ee0 2022-06-29T15:48:20.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.066+0000 7f4294804700 1 -- 172.21.15.47:0/121444770 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42900573d0 con 0x7f4290060ee0 2022-06-29T15:48:20.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.066+0000 7f42877fe700 1 -- 172.21.15.47:0/121444770 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4280025030 con 0x7f4290060ee0 2022-06-29T15:48:20.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.070+0000 7f42877fe700 1 --2- 172.21.15.47:0/121444770 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f42740539e0 0x7f4274055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:20.084 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.070+0000 7f42877fe700 1 -- 172.21.15.47:0/121444770 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f428006d150 con 0x7f4290060ee0 2022-06-29T15:48:20.084 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.070+0000 7f428dd9b700 1 --2- 172.21.15.47:0/121444770 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f42740539e0 0x7f4274055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:20.084 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.070+0000 7f428dd9b700 1 --2- 172.21.15.47:0/121444770 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f42740539e0 0x7f4274055ea0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f427800c000 tx=0x7f4278008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:20.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.074+0000 7f42877fe700 1 -- 172.21.15.47:0/121444770 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4280043f10 con 0x7f4290060ee0 2022-06-29T15:48:20.244 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.242+0000 7f4294804700 1 -- 172.21.15.47:0/121444770 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f4290144190 con 0x7f4290060ee0 2022-06-29T15:48:20.251 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.246+0000 7f42877fe700 1 -- 172.21.15.47:0/121444770 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v22) v1 ==== 184+0+0 (secure 0 0 0) 0x7f42800230e0 con 0x7f4290060ee0 2022-06-29T15:48:20.253 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.250+0000 7f4294804700 1 -- 172.21.15.47:0/121444770 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f42740539e0 msgr2=0x7f4274055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:20.254 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.250+0000 7f4294804700 1 --2- 172.21.15.47:0/121444770 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f42740539e0 0x7f4274055ea0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f427800c000 tx=0x7f4278008040).stop 2022-06-29T15:48:20.254 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.250+0000 7f4294804700 1 -- 172.21.15.47:0/121444770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4290060ee0 msgr2=0x7f429013fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:20.255 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.250+0000 7f4294804700 1 --2- 172.21.15.47:0/121444770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4290060ee0 0x7f429013fb60 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 rx=0x7f42900ae790 tx=0x7f4280015000).stop 2022-06-29T15:48:20.255 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.250+0000 7f4294804700 1 -- 172.21.15.47:0/121444770 shutdown_connections 2022-06-29T15:48:20.255 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.250+0000 7f4294804700 1 --2- 172.21.15.47:0/121444770 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f42900b4300 0x7f4290145f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:20.256 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.250+0000 7f4294804700 1 --2- 172.21.15.47:0/121444770 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f42740539e0 0x7f4274055ea0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:20.256 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.250+0000 7f4294804700 1 --2- 172.21.15.47:0/121444770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4290066f70 0x7f4290145a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:20.256 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.250+0000 7f4294804700 1 --2- 172.21.15.47:0/121444770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4290060ee0 0x7f429013fb60 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:20.256 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.250+0000 7f4294804700 1 -- 172.21.15.47:0/121444770 >> 172.21.15.47:0/121444770 conn(0x7f429005c510 msgr2=0x7f42900b89c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:20.257 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.250+0000 7f4294804700 1 -- 172.21.15.47:0/121444770 shutdown_connections 2022-06-29T15:48:20.257 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.250+0000 7f4294804700 1 -- 172.21.15.47:0/121444770 wait complete. 2022-06-29T15:48:20.258 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.0 2022-06-29T15:48:20.267 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-29T15:48:20.380 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.378+0000 7f2af57a4700 1 Processor -- start 2022-06-29T15:48:20.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.378+0000 7f2af57a4700 1 -- start start 2022-06-29T15:48:20.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.378+0000 7f2af57a4700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2af00b58d0 0x7f2af00b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:20.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.378+0000 7f2af57a4700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2af014b4c0 0x7f2af00aa130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:20.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.378+0000 7f2af57a4700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2af0060790 con 0x7f2af00b1610 2022-06-29T15:48:20.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.378+0000 7f2af57a4700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2af00a99e0 con 0x7f2af00b58d0 2022-06-29T15:48:20.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.378+0000 7f2af57a4700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2af014ad80 con 0x7f2af014b4c0 2022-06-29T15:48:20.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.378+0000 7f2aeffff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2af00b58d0 0x7f2af00b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:20.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.378+0000 7f2aeffff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2af00b58d0 0x7f2af00b5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50036/0 (socket says 172.21.15.47:50036) 2022-06-29T15:48:20.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.378+0000 7f2aeffff700 1 -- 172.21.15.47:0/896213288 learned_addr learned my addr 172.21.15.47:0/896213288 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:20.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.378+0000 7f2af4fa3700 1 --2- 172.21.15.47:0/896213288 >> v2:172.21.15.64:3300/0 conn(0x7f2af014b4c0 0x7f2af00aa130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:20.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.378+0000 7f2af4fa3700 1 -- 172.21.15.47:0/896213288 >> v2:172.21.15.47:3300/0 conn(0x7f2af00b58d0 msgr2=0x7f2af00b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:20.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.378+0000 7f2af4fa3700 1 --2- 172.21.15.47:0/896213288 >> v2:172.21.15.47:3300/0 conn(0x7f2af00b58d0 0x7f2af00b5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:20.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.378+0000 7f2af4fa3700 1 -- 172.21.15.47:0/896213288 >> v1:172.21.15.64:6789/0 conn(0x7f2af00b1610 legacy=0x7f2af01475f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:20.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.378+0000 7f2af4fa3700 1 -- 172.21.15.47:0/896213288 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af014b020 con 0x7f2af014b4c0 2022-06-29T15:48:20.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.378+0000 7f2af4fa3700 1 --2- 172.21.15.47:0/896213288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2af014b4c0 0x7f2af00aa130 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f2ae80032f0 tx=0x7f2ae80091d0).ready entity=mon.1 client_cookie=836bc5d323647a57 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:20.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.378+0000 7f2aeeffd700 1 -- 172.21.15.47:0/896213288 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2ae800b680 con 0x7f2af014b4c0 2022-06-29T15:48:20.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.378+0000 7f2aeeffd700 1 -- 172.21.15.47:0/896213288 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ae800b7e0 con 0x7f2af014b4c0 2022-06-29T15:48:20.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.378+0000 7f2aeeffd700 1 -- 172.21.15.47:0/896213288 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2ae800b950 con 0x7f2af014b4c0 2022-06-29T15:48:20.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.382+0000 7f2af57a4700 1 -- 172.21.15.47:0/896213288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2af014b4c0 msgr2=0x7f2af00aa130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:20.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.382+0000 7f2af57a4700 1 --2- 172.21.15.47:0/896213288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2af014b4c0 0x7f2af00aa130 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f2ae80032f0 tx=0x7f2ae80091d0).stop 2022-06-29T15:48:20.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.382+0000 7f2af57a4700 1 -- 172.21.15.47:0/896213288 shutdown_connections 2022-06-29T15:48:20.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.382+0000 7f2af57a4700 1 --2- 172.21.15.47:0/896213288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2af014b4c0 0x7f2af00aa130 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:20.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.382+0000 7f2af57a4700 1 --2- 172.21.15.47:0/896213288 >> v2:172.21.15.47:3300/0 conn(0x7f2af00b58d0 0x7f2af00b5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:20.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.382+0000 7f2af57a4700 1 -- 172.21.15.47:0/896213288 >> 172.21.15.47:0/896213288 conn(0x7f2af0062560 msgr2=0x7f2af0062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:20.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.382+0000 7f2af57a4700 1 -- 172.21.15.47:0/896213288 shutdown_connections 2022-06-29T15:48:20.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.382+0000 7f2af57a4700 1 -- 172.21.15.47:0/896213288 wait complete. 2022-06-29T15:48:20.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.382+0000 7f2af57a4700 1 Processor -- start 2022-06-29T15:48:20.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.382+0000 7f2af57a4700 1 -- start start 2022-06-29T15:48:20.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.382+0000 7f2af57a4700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2af00b1610 0x7f2af00aef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:20.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.382+0000 7f2af57a4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2af00b58d0 0x7f2af014e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:20.394 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.386+0000 7f2af57a4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2af014b4c0 0x7f2af00ac6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:20.394 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.386+0000 7f2af57a4700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af0066330 con 0x7f2af00b58d0 2022-06-29T15:48:20.394 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.386+0000 7f2af57a4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af00a8ce0 con 0x7f2af014b4c0 2022-06-29T15:48:20.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.386+0000 7f2af57a4700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af0063540 con 0x7f2af00b1610 2022-06-29T15:48:20.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.386+0000 7f2aef7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2af00b58d0 0x7f2af014e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:20.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.386+0000 7f2aeffff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2af00b1610 0x7f2af00aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:20.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.386+0000 7f2af4fa3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2af014b4c0 0x7f2af00ac6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:20.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.386+0000 7f2aef7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2af00b58d0 0x7f2af014e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50044/0 (socket says 172.21.15.47:50044) 2022-06-29T15:48:20.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.386+0000 7f2aeffff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2af00b1610 0x7f2af00aef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:47144/0 (socket says 172.21.15.47:47144) 2022-06-29T15:48:20.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.386+0000 7f2aef7fe700 1 -- 172.21.15.47:0/980720340 learned_addr learned my addr 172.21.15.47:0/980720340 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:20.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.386+0000 7f2aeffff700 1 -- 172.21.15.47:0/980720340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2af014b4c0 msgr2=0x7f2af00ac6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:20.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.386+0000 7f2aeffff700 1 --2- 172.21.15.47:0/980720340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2af014b4c0 0x7f2af00ac6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:20.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.386+0000 7f2aeffff700 1 -- 172.21.15.47:0/980720340 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2af00b58d0 msgr2=0x7f2af014e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:20.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.386+0000 7f2aeffff700 1 --2- 172.21.15.47:0/980720340 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2af00b58d0 0x7f2af014e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:20.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.386+0000 7f2aeffff700 1 -- 172.21.15.47:0/980720340 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ae8009470 con 0x7f2af00b1610 2022-06-29T15:48:20.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.386+0000 7f2af4fa3700 1 --2- 172.21.15.47:0/980720340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2af014b4c0 0x7f2af00ac6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:48:20.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.386+0000 7f2aeffff700 1 --2- 172.21.15.47:0/980720340 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2af00b1610 0x7f2af00aef90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f2ae000c000 tx=0x7f2ae0008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:20.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.386+0000 7f2aed7fa700 1 -- 172.21.15.47:0/980720340 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2ae0013400 con 0x7f2af00b1610 2022-06-29T15:48:20.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.386+0000 7f2af57a4700 1 -- 172.21.15.47:0/980720340 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2af00ae340 con 0x7f2af00b1610 2022-06-29T15:48:20.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.386+0000 7f2aed7fa700 1 -- 172.21.15.47:0/980720340 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ae0013560 con 0x7f2af00b1610 2022-06-29T15:48:20.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.386+0000 7f2aed7fa700 1 -- 172.21.15.47:0/980720340 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2ae00138c0 con 0x7f2af00b1610 2022-06-29T15:48:20.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.386+0000 7f2af57a4700 1 -- 172.21.15.47:0/980720340 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2af00a8f10 con 0x7f2af00b1610 2022-06-29T15:48:20.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.386+0000 7f2af57a4700 1 -- 172.21.15.47:0/980720340 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2af00573d0 con 0x7f2af00b1610 2022-06-29T15:48:20.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.390+0000 7f2aed7fa700 1 -- 172.21.15.47:0/980720340 <== mon.2 v2:172.21.15.64:3301/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2ae0022020 con 0x7f2af00b1610 2022-06-29T15:48:20.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.390+0000 7f2aed7fa700 1 --2- 172.21.15.47:0/980720340 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2ad0053980 0x7f2ad0055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:20.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.390+0000 7f2aed7fa700 1 -- 172.21.15.47:0/980720340 <== mon.2 v2:172.21.15.64:3301/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f2ae006bd70 con 0x7f2af00b1610 2022-06-29T15:48:20.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.390+0000 7f2aef7fe700 1 --2- 172.21.15.47:0/980720340 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2ad0053980 0x7f2ad0055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:20.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.390+0000 7f2aef7fe700 1 --2- 172.21.15.47:0/980720340 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2ad0053980 0x7f2ad0055e40 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f2ae400d000 tx=0x7f2ae4000bc0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:20.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.398+0000 7f2aed7fa700 1 -- 172.21.15.47:0/980720340 <== mon.2 v2:172.21.15.64:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2ae0042a90 con 0x7f2af00b1610 2022-06-29T15:48:20.574 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.570+0000 7f2af57a4700 1 -- 172.21.15.47:0/980720340 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f2af0069e20 con 0x7f2af00b1610 2022-06-29T15:48:20.583 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.578+0000 7f2aed7fa700 1 -- 172.21.15.47:0/980720340 <== mon.2 v2:172.21.15.64:3301/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v23) v1 ==== 184+0+0 (secure 0 0 0) 0x7f2ae00460b0 con 0x7f2af00b1610 2022-06-29T15:48:20.586 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.582+0000 7f2af57a4700 1 -- 172.21.15.47:0/980720340 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2ad0053980 msgr2=0x7f2ad0055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:20.586 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.582+0000 7f2af57a4700 1 --2- 172.21.15.47:0/980720340 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2ad0053980 0x7f2ad0055e40 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f2ae400d000 tx=0x7f2ae4000bc0).stop 2022-06-29T15:48:20.586 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.582+0000 7f2af57a4700 1 -- 172.21.15.47:0/980720340 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2af00b1610 msgr2=0x7f2af00aef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:20.586 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.582+0000 7f2af57a4700 1 --2- 172.21.15.47:0/980720340 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2af00b1610 0x7f2af00aef90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f2ae000c000 tx=0x7f2ae0008040).stop 2022-06-29T15:48:20.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.582+0000 7f2af57a4700 1 -- 172.21.15.47:0/980720340 shutdown_connections 2022-06-29T15:48:20.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.582+0000 7f2af57a4700 1 --2- 172.21.15.47:0/980720340 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2ad0053980 0x7f2ad0055e40 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:20.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.582+0000 7f2af57a4700 1 --2- 172.21.15.47:0/980720340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2af014b4c0 0x7f2af00ac6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:20.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.582+0000 7f2af57a4700 1 --2- 172.21.15.47:0/980720340 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2af00b58d0 0x7f2af014e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:20.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.582+0000 7f2af57a4700 1 --2- 172.21.15.47:0/980720340 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2af00b1610 0x7f2af00aef90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:20.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.582+0000 7f2af57a4700 1 -- 172.21.15.47:0/980720340 >> 172.21.15.47:0/980720340 conn(0x7f2af0062560 msgr2=0x7f2af00b08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:20.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.582+0000 7f2af57a4700 1 -- 172.21.15.47:0/980720340 shutdown_connections 2022-06-29T15:48:20.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.582+0000 7f2af57a4700 1 -- 172.21.15.47:0/980720340 wait complete. 2022-06-29T15:48:20.589 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.1 2022-06-29T15:48:20.599 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:48:20.708 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.702+0000 7f0a9cd10700 1 Processor -- start 2022-06-29T15:48:20.708 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.702+0000 7f0a9cd10700 1 -- start start 2022-06-29T15:48:20.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.706+0000 7f0a9cd10700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0a980b2740 0x7f0a980aa9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:20.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.706+0000 7f0a9cd10700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0a981496b0 0x7f0a9814baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:20.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.706+0000 7f0a9cd10700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a980ae3e0 con 0x7f0a980af990 2022-06-29T15:48:20.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.706+0000 7f0a9cd10700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a980af110 con 0x7f0a980b2740 2022-06-29T15:48:20.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.706+0000 7f0a9cd10700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a980a9e80 con 0x7f0a981496b0 2022-06-29T15:48:20.712 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.706+0000 7f0a977fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0a980b2740 0x7f0a980aa9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:20.712 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.706+0000 7f0a977fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0a980b2740 0x7f0a980aa9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50050/0 (socket says 172.21.15.47:50050) 2022-06-29T15:48:20.712 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.706+0000 7f0a977fe700 1 -- 172.21.15.47:0/1531299893 learned_addr learned my addr 172.21.15.47:0/1531299893 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:20.713 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.706+0000 7f0a97fff700 1 --2- 172.21.15.47:0/1531299893 >> v2:172.21.15.64:3300/0 conn(0x7f0a981496b0 0x7f0a9814baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:20.713 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.706+0000 7f0a977fe700 1 -- 172.21.15.47:0/1531299893 >> v2:172.21.15.64:3300/0 conn(0x7f0a981496b0 msgr2=0x7f0a9814baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:20.713 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.706+0000 7f0a977fe700 1 --2- 172.21.15.47:0/1531299893 >> v2:172.21.15.64:3300/0 conn(0x7f0a981496b0 0x7f0a9814baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:20.713 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.706+0000 7f0a977fe700 1 -- 172.21.15.47:0/1531299893 >> v1:172.21.15.64:6789/0 conn(0x7f0a980af990 legacy=0x7f0a980ac180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:20.714 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.706+0000 7f0a977fe700 1 -- 172.21.15.47:0/1531299893 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a980b1750 con 0x7f0a980b2740 2022-06-29T15:48:20.714 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.706+0000 7f0a977fe700 1 --2- 172.21.15.47:0/1531299893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a980b2740 0x7f0a980aa9b0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 rx=0x7f0a98067e60 tx=0x7f0a88006eb0).ready entity=mon.0 client_cookie=a3657fa92bad16a5 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:20.714 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.706+0000 7f0a96ffd700 1 -- 172.21.15.47:0/1531299893 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a8800d5c0 con 0x7f0a980b2740 2022-06-29T15:48:20.714 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.706+0000 7f0a96ffd700 1 -- 172.21.15.47:0/1531299893 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a88000e90 con 0x7f0a980b2740 2022-06-29T15:48:20.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.706+0000 7f0a96ffd700 1 -- 172.21.15.47:0/1531299893 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a8800d910 con 0x7f0a980b2740 2022-06-29T15:48:20.719 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.710+0000 7f0a9cd10700 1 -- 172.21.15.47:0/1531299893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a980b2740 msgr2=0x7f0a980aa9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:20.720 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.710+0000 7f0a9cd10700 1 --2- 172.21.15.47:0/1531299893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a980b2740 0x7f0a980aa9b0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 rx=0x7f0a98067e60 tx=0x7f0a88006eb0).stop 2022-06-29T15:48:20.720 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.710+0000 7f0a9cd10700 1 -- 172.21.15.47:0/1531299893 shutdown_connections 2022-06-29T15:48:20.720 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.710+0000 7f0a9cd10700 1 --2- 172.21.15.47:0/1531299893 >> v2:172.21.15.64:3300/0 conn(0x7f0a981496b0 0x7f0a9814baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:20.721 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.710+0000 7f0a9cd10700 1 --2- 172.21.15.47:0/1531299893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a980b2740 0x7f0a980aa9b0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:20.721 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.710+0000 7f0a9cd10700 1 -- 172.21.15.47:0/1531299893 >> 172.21.15.47:0/1531299893 conn(0x7f0a98062df0 msgr2=0x7f0a980631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:20.721 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.710+0000 7f0a9cd10700 1 -- 172.21.15.47:0/1531299893 shutdown_connections 2022-06-29T15:48:20.721 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.710+0000 7f0a9cd10700 1 -- 172.21.15.47:0/1531299893 wait complete. 2022-06-29T15:48:20.721 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.710+0000 7f0a9cd10700 1 Processor -- start 2022-06-29T15:48:20.722 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.710+0000 7f0a9cd10700 1 -- start start 2022-06-29T15:48:20.722 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.714+0000 7f0a9cd10700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a980af990 0x7f0a98147ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:20.723 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.714+0000 7f0a9cd10700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a980b2740 0x7f0a980ac190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:20.723 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.714+0000 7f0a9cd10700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0a981496b0 0x7f0a981535e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:20.723 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.714+0000 7f0a9cd10700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a980b2300 con 0x7f0a980af990 2022-06-29T15:48:20.724 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.714+0000 7f0a9cd10700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a981492c0 con 0x7f0a980b2740 2022-06-29T15:48:20.724 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.714+0000 7f0a9cd10700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a98148920 con 0x7f0a981496b0 2022-06-29T15:48:20.724 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.714+0000 7f0a977fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a980af990 0x7f0a98147ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:20.724 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.714+0000 7f0a977fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a980af990 0x7f0a98147ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50056/0 (socket says 172.21.15.47:50056) 2022-06-29T15:48:20.725 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.714+0000 7f0a977fe700 1 -- 172.21.15.47:0/1633524123 learned_addr learned my addr 172.21.15.47:0/1633524123 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:20.725 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.714+0000 7f0a97fff700 1 --2- 172.21.15.47:0/1633524123 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0a981496b0 0x7f0a981535e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:20.725 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.714+0000 7f0a8ffff700 1 --2- 172.21.15.47:0/1633524123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a980b2740 0x7f0a980ac190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:20.725 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.714+0000 7f0a977fe700 1 -- 172.21.15.47:0/1633524123 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0a981496b0 msgr2=0x7f0a981535e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:20.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.714+0000 7f0a977fe700 1 --2- 172.21.15.47:0/1633524123 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0a981496b0 0x7f0a981535e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:20.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.714+0000 7f0a977fe700 1 -- 172.21.15.47:0/1633524123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a980b2740 msgr2=0x7f0a980ac190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:20.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.714+0000 7f0a977fe700 1 --2- 172.21.15.47:0/1633524123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a980b2740 0x7f0a980ac190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:20.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.714+0000 7f0a977fe700 1 -- 172.21.15.47:0/1633524123 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a8800b1e0 con 0x7f0a980af990 2022-06-29T15:48:20.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.714+0000 7f0a8ffff700 1 --2- 172.21.15.47:0/1633524123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a980b2740 0x7f0a980ac190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:48:20.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.714+0000 7f0a977fe700 1 --2- 172.21.15.47:0/1633524123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a980af990 0x7f0a98147ad0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 rx=0x7f0a98149400 tx=0x7f0a88015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:20.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.714+0000 7f0a957fa700 1 -- 172.21.15.47:0/1633524123 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a880117e0 con 0x7f0a980af990 2022-06-29T15:48:20.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.714+0000 7f0a957fa700 1 -- 172.21.15.47:0/1633524123 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a88011940 con 0x7f0a980af990 2022-06-29T15:48:20.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.714+0000 7f0a9cd10700 1 -- 172.21.15.47:0/1633524123 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a98148100 con 0x7f0a980af990 2022-06-29T15:48:20.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.714+0000 7f0a957fa700 1 -- 172.21.15.47:0/1633524123 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a88011ca0 con 0x7f0a980af990 2022-06-29T15:48:20.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.714+0000 7f0a9cd10700 1 -- 172.21.15.47:0/1633524123 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a98148e20 con 0x7f0a980af990 2022-06-29T15:48:20.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.714+0000 7f0a9cd10700 1 -- 172.21.15.47:0/1633524123 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a980573d0 con 0x7f0a980af990 2022-06-29T15:48:20.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.714+0000 7f0a957fa700 1 -- 172.21.15.47:0/1633524123 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0a88025030 con 0x7f0a980af990 2022-06-29T15:48:20.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.718+0000 7f0a957fa700 1 --2- 172.21.15.47:0/1633524123 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0a7c0539e0 0x7f0a7c055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:20.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.718+0000 7f0a957fa700 1 -- 172.21.15.47:0/1633524123 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f0a8806d3a0 con 0x7f0a980af990 2022-06-29T15:48:20.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.718+0000 7f0a8ffff700 1 --2- 172.21.15.47:0/1633524123 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0a7c0539e0 0x7f0a7c055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:20.731 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.718+0000 7f0a8ffff700 1 --2- 172.21.15.47:0/1633524123 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0a7c0539e0 0x7f0a7c055ea0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7f0a8000b000 tx=0x7f0a80009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:20.731 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.722+0000 7f0a957fa700 1 -- 172.21.15.47:0/1633524123 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0a88044100 con 0x7f0a980af990 2022-06-29T15:48:20.908 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.906+0000 7f0a9cd10700 1 -- 172.21.15.47:0/1633524123 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a98064970 con 0x7f0a980af990 2022-06-29T15:48:20.909 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.906+0000 7f0a957fa700 1 -- 172.21.15.47:0/1633524123 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+2493 (secure 0 0 0) 0x7f0a88023020 con 0x7f0a980af990 2022-06-29T15:48:20.910 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:48:20.910 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":35,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6291,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.47:6835/2910037081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2910037081},{"type":"v1","addr":"172.21.15.47:6835","nonce":2910037081}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":34},{"gid":6300,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/359532891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":359532891},{"type":"v1","addr":"172.21.15.47:6837","nonce":359532891}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":33}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:48:18.345845+0000","modified":"2022-06-29T15:48:19.349718+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":55562},"failed":[],"damaged":[],"stopped":[],"info":{"gid_55562":{"gid":55562,"name":"b","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6833/3871453337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3871453337},{"type":"v1","addr":"172.21.15.64:6833","nonce":3871453337}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2022-06-29T15:48:20.913 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.910+0000 7f0a9cd10700 1 -- 172.21.15.47:0/1633524123 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0a7c0539e0 msgr2=0x7f0a7c055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:20.913 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.910+0000 7f0a9cd10700 1 --2- 172.21.15.47:0/1633524123 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0a7c0539e0 0x7f0a7c055ea0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7f0a8000b000 tx=0x7f0a80009040).stop 2022-06-29T15:48:20.913 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.910+0000 7f0a9cd10700 1 -- 172.21.15.47:0/1633524123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a980af990 msgr2=0x7f0a98147ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:20.914 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.910+0000 7f0a9cd10700 1 --2- 172.21.15.47:0/1633524123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a980af990 0x7f0a98147ad0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 rx=0x7f0a98149400 tx=0x7f0a88015000).stop 2022-06-29T15:48:20.914 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.910+0000 7f0a9cd10700 1 -- 172.21.15.47:0/1633524123 shutdown_connections 2022-06-29T15:48:20.914 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.910+0000 7f0a9cd10700 1 --2- 172.21.15.47:0/1633524123 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0a981496b0 0x7f0a981535e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:20.914 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.910+0000 7f0a9cd10700 1 --2- 172.21.15.47:0/1633524123 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0a7c0539e0 0x7f0a7c055ea0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:20.915 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.910+0000 7f0a9cd10700 1 --2- 172.21.15.47:0/1633524123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a980b2740 0x7f0a980ac190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:20.915 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.910+0000 7f0a9cd10700 1 --2- 172.21.15.47:0/1633524123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a980af990 0x7f0a98147ad0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:20.915 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.910+0000 7f0a9cd10700 1 -- 172.21.15.47:0/1633524123 >> 172.21.15.47:0/1633524123 conn(0x7f0a98062df0 msgr2=0x7f0a980631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:20.915 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.910+0000 7f0a9cd10700 1 -- 172.21.15.47:0/1633524123 shutdown_connections 2022-06-29T15:48:20.916 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:20.910+0000 7f0a9cd10700 1 -- 172.21.15.47:0/1633524123 wait complete. 2022-06-29T15:48:20.916 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 35 2022-06-29T15:48:21.231 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 35, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-29T15:48:18.345845+0000', 'modified': '2022-06-29T15:48:19.349718+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 55562}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_55562': {'gid': 55562, 'name': 'b', 'rank': 0, 'incarnation': 34, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.64:6833/3871453337', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6832', 'nonce': 3871453337}, {'type': 'v1', 'addr': '172.21.15.64:6833', 'nonce': 3871453337}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-29T15:48:21.231 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-29T15:48:21.232 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-06-29T15:48:21.352 INFO:teuthology.orchestra.run.smithi064.stdout:{"cluster_fsid":"9a68894a-53ce-400e-838b-14eab91b834c","whoami":0,"id":55562,"want_state":"up:active","state":"up:active","rank_uptime":2.99154498,"mdsmap_epoch":35,"osdmap_epoch":78,"osdmap_epoch_barrier":78,"uptime":7.3957138750000002} 2022-06-29T15:48:21.353 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9a68894a-53ce-400e-838b-14eab91b834c", "id": 55562, "mdsmap_epoch": 35, "osdmap_epoch": 78, "osdmap_epoch_barrier": 78, "rank_uptime": 2.99154498, "state": "up:active", "uptime": 7.395713875, "want_state": "up:active", "whoami": 0 } 2022-06-29T15:48:21.354 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:48:21.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.462+0000 7f122f59e700 1 Processor -- start 2022-06-29T15:48:21.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.462+0000 7f122f59e700 1 -- start start 2022-06-29T15:48:21.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.462+0000 7f122f59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f12300b5c60 0x7f1230144980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:21.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.462+0000 7f122f59e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12300a9120 con 0x7f1230060eb0 2022-06-29T15:48:21.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.462+0000 7f122f59e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12300b0b80 con 0x7f12300a72f0 2022-06-29T15:48:21.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.462+0000 7f122f59e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12300a89e0 con 0x7f12300b5c60 2022-06-29T15:48:21.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.462+0000 7f122dd9b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1230060eb0 0x7f12300b1e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35618/0 (socket says 172.21.15.47:35618) 2022-06-29T15:48:21.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.462+0000 7f122dd9b700 1 -- 172.21.15.47:0/226778080 learned_addr learned my addr 172.21.15.47:0/226778080 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:48:21.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.462+0000 7f122e59c700 1 --2- 172.21.15.47:0/226778080 >> v2:172.21.15.64:3300/0 conn(0x7f12300b5c60 0x7f1230144980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:21.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.462+0000 7f122d59a700 1 -- 172.21.15.47:0/226778080 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 130023341 0 0) 0x7f12300a9120 con 0x7f1230060eb0 2022-06-29T15:48:21.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.462+0000 7f122d59a700 1 -- 172.21.15.47:0/226778080 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1214003260 con 0x7f1230060eb0 2022-06-29T15:48:21.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.462+0000 7f122d59a700 1 -- 172.21.15.47:0/226778080 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 499368718 0 0) 0x7f12300b0b80 con 0x7f12300a72f0 2022-06-29T15:48:21.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.462+0000 7f122d59a700 1 -- 172.21.15.47:0/226778080 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12300a9120 con 0x7f12300a72f0 2022-06-29T15:48:21.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.462+0000 7f122d59a700 1 -- 172.21.15.47:0/226778080 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1067281128 0 0) 0x7f1214003260 con 0x7f1230060eb0 2022-06-29T15:48:21.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.462+0000 7f122d59a700 1 -- 172.21.15.47:0/226778080 >> v2:172.21.15.64:3300/0 conn(0x7f12300b5c60 msgr2=0x7f1230144980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:21.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.462+0000 7f122d59a700 1 --2- 172.21.15.47:0/226778080 >> v2:172.21.15.64:3300/0 conn(0x7f12300b5c60 0x7f1230144980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:21.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.462+0000 7f122d59a700 1 -- 172.21.15.47:0/226778080 >> v1:172.21.15.64:6789/0 conn(0x7f12300a72f0 legacy=0x7f1230151eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:21.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.462+0000 7f122d59a700 1 -- 172.21.15.47:0/226778080 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12300a8c80 con 0x7f1230060eb0 2022-06-29T15:48:21.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.462+0000 7f122d59a700 1 -- 172.21.15.47:0/226778080 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7f1220001660 con 0x7f1230060eb0 2022-06-29T15:48:21.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.462+0000 7f122e59c700 1 --2- 172.21.15.47:0/226778080 >> v2:172.21.15.64:3300/0 conn(0x7f12300b5c60 0x7f1230144980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:48:21.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.462+0000 7f122d59a700 1 -- 172.21.15.47:0/226778080 >> v1:172.21.15.47:6789/0 conn(0x7f1230060eb0 legacy=0x7f12300b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:21.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.462+0000 7f122d59a700 1 --2- 172.21.15.47:0/226778080 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1214003930 0x7f1214003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:21.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.462+0000 7f122d59a700 1 --2- 172.21.15.47:0/226778080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1214004500 0x7f1214004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:21.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.466+0000 7f122d59a700 1 --2- 172.21.15.47:0/226778080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1214004e70 0x7f121400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:21.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.466+0000 7f122d59a700 1 -- 172.21.15.47:0/226778080 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12300a89e0 con 0x7f1214004500 2022-06-29T15:48:21.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.466+0000 7f122d59a700 1 -- 172.21.15.47:0/226778080 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f121400bd20 con 0x7f1214004e70 2022-06-29T15:48:21.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.466+0000 7f122d59a700 1 -- 172.21.15.47:0/226778080 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f121400bee0 con 0x7f1214003930 2022-06-29T15:48:21.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.466+0000 7f122dd9b700 1 --2- 172.21.15.47:0/226778080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1214004500 0x7f1214004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:21.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.466+0000 7f122e59c700 1 --2- 172.21.15.47:0/226778080 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1214003930 0x7f1214003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:21.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.466+0000 7f122ed9d700 1 --2- 172.21.15.47:0/226778080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1214004e70 0x7f121400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:21.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.466+0000 7f122dd9b700 1 -- 172.21.15.47:0/226778080 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1214003930 msgr2=0x7f1214003d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:21.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.466+0000 7f122dd9b700 1 --2- 172.21.15.47:0/226778080 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1214003930 0x7f1214003d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:21.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.466+0000 7f122dd9b700 1 -- 172.21.15.47:0/226778080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1214004e70 msgr2=0x7f121400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:21.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.466+0000 7f122dd9b700 1 --2- 172.21.15.47:0/226778080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1214004e70 0x7f121400b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:21.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.466+0000 7f122dd9b700 1 -- 172.21.15.47:0/226778080 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1214003260 con 0x7f1214004500 2022-06-29T15:48:21.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.466+0000 7f122ed9d700 1 --2- 172.21.15.47:0/226778080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1214004e70 0x7f121400b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:48:21.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.466+0000 7f122ed9d700 1 -- 172.21.15.47:0/226778080 reap_dead start 2022-06-29T15:48:21.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.466+0000 7f122dd9b700 1 --2- 172.21.15.47:0/226778080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1214004500 0x7f1214004980 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 rx=0x7f1220001430 tx=0x7f12300a87f0).ready entity=mon.0 client_cookie=d6dd674167da8f5 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:21.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.466+0000 7f122d59a700 1 -- 172.21.15.47:0/226778080 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12200070d0 con 0x7f1214004500 2022-06-29T15:48:21.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.466+0000 7f122d59a700 1 -- 172.21.15.47:0/226778080 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1220007230 con 0x7f1214004500 2022-06-29T15:48:21.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.466+0000 7f122d59a700 1 -- 172.21.15.47:0/226778080 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f122000c8f0 con 0x7f1214004500 2022-06-29T15:48:21.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.466+0000 7f122f59e700 1 -- 172.21.15.47:0/226778080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1214004500 msgr2=0x7f1214004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:21.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.466+0000 7f122f59e700 1 --2- 172.21.15.47:0/226778080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1214004500 0x7f1214004980 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 rx=0x7f1220001430 tx=0x7f12300a87f0).stop 2022-06-29T15:48:21.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.466+0000 7f122f59e700 1 -- 172.21.15.47:0/226778080 shutdown_connections 2022-06-29T15:48:21.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.466+0000 7f122f59e700 1 --2- 172.21.15.47:0/226778080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1214004500 0x7f1214004980 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:21.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.466+0000 7f122f59e700 1 -- 172.21.15.47:0/226778080 >> 172.21.15.47:0/226778080 conn(0x7f1230062560 msgr2=0x7f1230062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:21.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.466+0000 7f122f59e700 1 -- 172.21.15.47:0/226778080 shutdown_connections 2022-06-29T15:48:21.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.466+0000 7f122f59e700 1 -- 172.21.15.47:0/226778080 wait complete. 2022-06-29T15:48:21.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.470+0000 7f122f59e700 1 Processor -- start 2022-06-29T15:48:21.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.470+0000 7f122f59e700 1 -- start start 2022-06-29T15:48:21.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.470+0000 7f122f59e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1214004500 0x7f12300b35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:21.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.470+0000 7f122f59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12301435c0 0x7f12300af220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:21.484 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.470+0000 7f122f59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12301417f0 0x7f1230141c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:21.484 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.470+0000 7f122f59e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1230148f90 con 0x7f12301435c0 2022-06-29T15:48:21.484 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.470+0000 7f122f59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1230148a90 con 0x7f12301417f0 2022-06-29T15:48:21.484 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.470+0000 7f122f59e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1230148cf0 con 0x7f1214004500 2022-06-29T15:48:21.485 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.470+0000 7f122dd9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12301435c0 0x7f12300af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:21.485 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.470+0000 7f122dd9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12301435c0 0x7f12300af220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50076/0 (socket says 172.21.15.47:50076) 2022-06-29T15:48:21.485 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.470+0000 7f122dd9b700 1 -- 172.21.15.47:0/2968470588 learned_addr learned my addr 172.21.15.47:0/2968470588 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:21.485 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.470+0000 7f122e59c700 1 --2- 172.21.15.47:0/2968470588 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1214004500 0x7f12300b35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:21.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.470+0000 7f122ed9d700 1 --2- 172.21.15.47:0/2968470588 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12301417f0 0x7f1230141c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:21.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.470+0000 7f122dd9b700 1 -- 172.21.15.47:0/2968470588 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1214004500 msgr2=0x7f12300b35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:21.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.470+0000 7f122dd9b700 1 --2- 172.21.15.47:0/2968470588 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1214004500 0x7f12300b35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:21.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.470+0000 7f122dd9b700 1 -- 172.21.15.47:0/2968470588 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12301417f0 msgr2=0x7f1230141c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:21.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.470+0000 7f122dd9b700 1 --2- 172.21.15.47:0/2968470588 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12301417f0 0x7f1230141c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:21.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.470+0000 7f122dd9b700 1 -- 172.21.15.47:0/2968470588 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1230141350 con 0x7f12301435c0 2022-06-29T15:48:21.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.470+0000 7f122dd9b700 1 --2- 172.21.15.47:0/2968470588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12301435c0 0x7f12300af220 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 rx=0x7f1220001430 tx=0x7f1220013000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:21.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.470+0000 7f121f7fe700 1 -- 172.21.15.47:0/2968470588 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1220017020 con 0x7f12301435c0 2022-06-29T15:48:21.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.470+0000 7f121f7fe700 1 -- 172.21.15.47:0/2968470588 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1220004920 con 0x7f12301435c0 2022-06-29T15:48:21.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.470+0000 7f122f59e700 1 -- 172.21.15.47:0/2968470588 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1230159b60 con 0x7f12301435c0 2022-06-29T15:48:21.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.470+0000 7f121f7fe700 1 -- 172.21.15.47:0/2968470588 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1220004c80 con 0x7f12301435c0 2022-06-29T15:48:21.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.470+0000 7f122f59e700 1 -- 172.21.15.47:0/2968470588 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f123015a010 con 0x7f12301435c0 2022-06-29T15:48:21.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.474+0000 7f121f7fe700 1 -- 172.21.15.47:0/2968470588 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1220025070 con 0x7f12301435c0 2022-06-29T15:48:21.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.474+0000 7f122f59e700 1 -- 172.21.15.47:0/2968470588 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12301414e0 con 0x7f12301435c0 2022-06-29T15:48:21.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.474+0000 7f121f7fe700 1 --2- 172.21.15.47:0/2968470588 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f120c053a70 0x7f120c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:21.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.474+0000 7f121f7fe700 1 -- 172.21.15.47:0/2968470588 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f122001a040 con 0x7f12301435c0 2022-06-29T15:48:21.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.474+0000 7f122e59c700 1 --2- 172.21.15.47:0/2968470588 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f120c053a70 0x7f120c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:21.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.474+0000 7f122e59c700 1 --2- 172.21.15.47:0/2968470588 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f120c053a70 0x7f120c055f30 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f1224008000 tx=0x7f1224008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:21.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.482+0000 7f121f7fe700 1 -- 172.21.15.47:0/2968470588 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1220043390 con 0x7f12301435c0 2022-06-29T15:48:21.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.610+0000 7f122f59e700 1 -- 172.21.15.47:0/2968470588 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f12300573d0 con 0x7f12301435c0 2022-06-29T15:48:21.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.614+0000 7f121f7fe700 1 -- 172.21.15.47:0/2968470588 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+16274 (secure 0 0 0) 0x7f12200469b0 con 0x7f12301435c0 2022-06-29T15:48:21.617 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:48:21.617 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":78,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:48:19.348792+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:48:16.753707+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2022-06-29T15:48:17.935585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:48:21.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.614+0000 7f122f59e700 1 -- 172.21.15.47:0/2968470588 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f120c053a70 msgr2=0x7f120c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:21.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.614+0000 7f122f59e700 1 --2- 172.21.15.47:0/2968470588 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f120c053a70 0x7f120c055f30 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f1224008000 tx=0x7f1224008040).stop 2022-06-29T15:48:21.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.614+0000 7f122f59e700 1 -- 172.21.15.47:0/2968470588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12301435c0 msgr2=0x7f12300af220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:21.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.614+0000 7f122f59e700 1 --2- 172.21.15.47:0/2968470588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12301435c0 0x7f12300af220 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 rx=0x7f1220001430 tx=0x7f1220013000).stop 2022-06-29T15:48:21.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.614+0000 7f122f59e700 1 -- 172.21.15.47:0/2968470588 shutdown_connections 2022-06-29T15:48:21.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.614+0000 7f122f59e700 1 --2- 172.21.15.47:0/2968470588 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f120c053a70 0x7f120c055f30 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:21.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.614+0000 7f122f59e700 1 --2- 172.21.15.47:0/2968470588 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12301417f0 0x7f1230141c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:21.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.614+0000 7f122f59e700 1 --2- 172.21.15.47:0/2968470588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12301435c0 0x7f12300af220 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:21.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.614+0000 7f122f59e700 1 --2- 172.21.15.47:0/2968470588 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1214004500 0x7f12300b35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:21.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.614+0000 7f122f59e700 1 -- 172.21.15.47:0/2968470588 >> 172.21.15.47:0/2968470588 conn(0x7f1230062560 msgr2=0x7f12300b4b20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:21.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.614+0000 7f122f59e700 1 -- 172.21.15.47:0/2968470588 shutdown_connections 2022-06-29T15:48:21.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.614+0000 7f122f59e700 1 -- 172.21.15.47:0/2968470588 wait complete. 2022-06-29T15:48:21.637 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-29T15:42:16.971431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-29T15:42:25.290884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-29T15:48:16.753707+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2022-06-29T15:48:17.935585+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-29T15:48:21.638 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-29T15:48:21.752 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.750+0000 7fa1023cb700 1 Processor -- start 2022-06-29T15:48:21.752 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.750+0000 7fa1023cb700 1 -- start start 2022-06-29T15:48:21.755 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.750+0000 7fa1023cb700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa0fc146720 0x7fa0fc140780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:21.755 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.750+0000 7fa1023cb700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa0fc0b4320 0x7fa0fc143230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:21.755 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.750+0000 7fa1023cb700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0fc061600 con 0x7fa0fc061220 2022-06-29T15:48:21.756 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.750+0000 7fa1023cb700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0fc146bc0 con 0x7fa0fc146720 2022-06-29T15:48:21.756 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.750+0000 7fa1023cb700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc146e60 con 0x7fa0fc0b4320 2022-06-29T15:48:21.756 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.750+0000 7fa0fbfff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa0fc061220 0x7fa0fc140070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35636/0 (socket says 172.21.15.47:35636) 2022-06-29T15:48:21.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.750+0000 7fa0fbfff700 1 -- 172.21.15.47:0/1841673326 learned_addr learned my addr 172.21.15.47:0/1841673326 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:48:21.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.750+0000 7fa0fb7fe700 1 --2- 172.21.15.47:0/1841673326 >> v2:172.21.15.64:3300/0 conn(0x7fa0fc146720 0x7fa0fc140780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:21.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.750+0000 7fa100968700 1 --2- 172.21.15.47:0/1841673326 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa0fc0b4320 0x7fa0fc143230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:21.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.750+0000 7fa0faffd700 1 -- 172.21.15.47:0/1841673326 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3223678271 0 0) 0x7fa0fc061600 con 0x7fa0fc061220 2022-06-29T15:48:21.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.750+0000 7fa0faffd700 1 -- 172.21.15.47:0/1841673326 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0e8003260 con 0x7fa0fc061220 2022-06-29T15:48:21.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.750+0000 7fa0fb7fe700 1 -- 172.21.15.47:0/1841673326 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa0fc0b4320 msgr2=0x7fa0fc143230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:21.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.750+0000 7fa0fb7fe700 1 --2- 172.21.15.47:0/1841673326 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa0fc0b4320 0x7fa0fc143230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:21.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.750+0000 7fa0fb7fe700 1 -- 172.21.15.47:0/1841673326 >> v1:172.21.15.47:6789/0 conn(0x7fa0fc061220 legacy=0x7fa0fc140070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:21.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.750+0000 7fa0fb7fe700 1 -- 172.21.15.47:0/1841673326 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0fc147100 con 0x7fa0fc146720 2022-06-29T15:48:21.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.750+0000 7fa0fb7fe700 1 --2- 172.21.15.47:0/1841673326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0fc146720 0x7fa0fc140780 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7fa0f0002f70 tx=0x7fa0f0006df0).ready entity=mon.1 client_cookie=ab2d16968d9a47c7 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:21.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.750+0000 7fa0faffd700 1 -- 172.21.15.47:0/1841673326 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0f000d4a0 con 0x7fa0fc146720 2022-06-29T15:48:21.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.750+0000 7fa0faffd700 1 -- 172.21.15.47:0/1841673326 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0f0000e90 con 0x7fa0fc146720 2022-06-29T15:48:21.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.750+0000 7fa0faffd700 1 -- 172.21.15.47:0/1841673326 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0f000d7f0 con 0x7fa0fc146720 2022-06-29T15:48:21.763 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.754+0000 7fa1023cb700 1 -- 172.21.15.47:0/1841673326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0fc146720 msgr2=0x7fa0fc140780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:21.763 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.754+0000 7fa1023cb700 1 --2- 172.21.15.47:0/1841673326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0fc146720 0x7fa0fc140780 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7fa0f0002f70 tx=0x7fa0f0006df0).stop 2022-06-29T15:48:21.764 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.754+0000 7fa1023cb700 1 -- 172.21.15.47:0/1841673326 shutdown_connections 2022-06-29T15:48:21.764 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.754+0000 7fa1023cb700 1 --2- 172.21.15.47:0/1841673326 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa0fc0b4320 0x7fa0fc143230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:21.764 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.754+0000 7fa1023cb700 1 --2- 172.21.15.47:0/1841673326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0fc146720 0x7fa0fc140780 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:21.765 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.754+0000 7fa1023cb700 1 -- 172.21.15.47:0/1841673326 >> 172.21.15.47:0/1841673326 conn(0x7fa0fc05c6b0 msgr2=0x7fa0fc05f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:21.765 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.754+0000 7fa1023cb700 1 -- 172.21.15.47:0/1841673326 shutdown_connections 2022-06-29T15:48:21.765 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.754+0000 7fa1023cb700 1 -- 172.21.15.47:0/1841673326 wait complete. 2022-06-29T15:48:21.765 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.754+0000 7fa1023cb700 1 Processor -- start 2022-06-29T15:48:21.766 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.754+0000 7fa1023cb700 1 -- start start 2022-06-29T15:48:21.766 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.754+0000 7fa1023cb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0fc061220 0x7fa0fc0b1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:21.766 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.754+0000 7fa1023cb700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa0fc0b4320 0x7fa0fc0b3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:21.766 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.754+0000 7fa1023cb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0fc146720 0x7fa0fc152540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:21.767 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.754+0000 7fa1023cb700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc0b1800 con 0x7fa0fc061220 2022-06-29T15:48:21.767 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.754+0000 7fa1023cb700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc0b1a40 con 0x7fa0fc146720 2022-06-29T15:48:21.767 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.754+0000 7fa1023cb700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc0b04c0 con 0x7fa0fc0b4320 2022-06-29T15:48:21.767 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.758+0000 7fa0fbfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0fc061220 0x7fa0fc0b1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:21.767 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.758+0000 7fa0fbfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0fc061220 0x7fa0fc0b1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50086/0 (socket says 172.21.15.47:50086) 2022-06-29T15:48:21.768 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.758+0000 7fa0fbfff700 1 -- 172.21.15.47:0/722835560 learned_addr learned my addr 172.21.15.47:0/722835560 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:21.768 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.758+0000 7fa100968700 1 --2- 172.21.15.47:0/722835560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0fc146720 0x7fa0fc152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:21.768 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.758+0000 7fa0fb7fe700 1 --2- 172.21.15.47:0/722835560 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa0fc0b4320 0x7fa0fc0b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:21.768 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.758+0000 7fa0fbfff700 1 -- 172.21.15.47:0/722835560 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa0fc0b4320 msgr2=0x7fa0fc0b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:21.769 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.758+0000 7fa0fbfff700 1 --2- 172.21.15.47:0/722835560 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa0fc0b4320 0x7fa0fc0b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:21.769 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.758+0000 7fa0fbfff700 1 -- 172.21.15.47:0/722835560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0fc146720 msgr2=0x7fa0fc152540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:21.769 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.758+0000 7fa0fbfff700 1 --2- 172.21.15.47:0/722835560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0fc146720 0x7fa0fc152540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:21.769 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.758+0000 7fa0fbfff700 1 -- 172.21.15.47:0/722835560 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f000b180 con 0x7fa0fc061220 2022-06-29T15:48:21.769 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.758+0000 7fa100968700 1 --2- 172.21.15.47:0/722835560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0fc146720 0x7fa0fc152540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:48:21.770 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.758+0000 7fa0fbfff700 1 --2- 172.21.15.47:0/722835560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0fc061220 0x7fa0fc0b1ce0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 rx=0x7fa0ec001110 tx=0x7fa0ec007aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:21.770 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.758+0000 7fa0f97fa700 1 -- 172.21.15.47:0/722835560 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0ec00ce80 con 0x7fa0fc061220 2022-06-29T15:48:21.770 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.758+0000 7fa0f97fa700 1 -- 172.21.15.47:0/722835560 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0ec0070d0 con 0x7fa0fc061220 2022-06-29T15:48:21.770 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.758+0000 7fa1023cb700 1 -- 172.21.15.47:0/722835560 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0fc0b0960 con 0x7fa0fc061220 2022-06-29T15:48:21.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.758+0000 7fa0f97fa700 1 -- 172.21.15.47:0/722835560 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0ec007430 con 0x7fa0fc061220 2022-06-29T15:48:21.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.758+0000 7fa1023cb700 1 -- 172.21.15.47:0/722835560 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0fc0af9a0 con 0x7fa0fc061220 2022-06-29T15:48:21.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.758+0000 7fa1023cb700 1 -- 172.21.15.47:0/722835560 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0fc0b0300 con 0x7fa0fc061220 2022-06-29T15:48:21.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.762+0000 7fa0f97fa700 1 -- 172.21.15.47:0/722835560 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa0ec010070 con 0x7fa0fc061220 2022-06-29T15:48:21.772 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.762+0000 7fa0f97fa700 1 --2- 172.21.15.47:0/722835560 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa0dc053b00 0x7fa0dc055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:21.772 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.762+0000 7fa0f97fa700 1 -- 172.21.15.47:0/722835560 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa0ec06b610 con 0x7fa0fc061220 2022-06-29T15:48:21.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.762+0000 7fa0fb7fe700 1 --2- 172.21.15.47:0/722835560 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa0dc053b00 0x7fa0dc055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:21.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.762+0000 7fa0fb7fe700 1 --2- 172.21.15.47:0/722835560 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa0dc053b00 0x7fa0dc055fc0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7fa0f001a000 tx=0x7fa0f0014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:21.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.770+0000 7fa0f97fa700 1 -- 172.21.15.47:0/722835560 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa0ec042370 con 0x7fa0fc061220 2022-06-29T15:48:21.899 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.894+0000 7fa1023cb700 1 -- 172.21.15.47:0/722835560 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fa0fc142660 con 0x7fa0fc061220 2022-06-29T15:48:21.900 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.898+0000 7fa0f97fa700 1 -- 172.21.15.47:0/722835560 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v78) v1 ==== 110+0+10 (secure 0 0 0) 0x7fa0ec045990 con 0x7fa0fc061220 2022-06-29T15:48:21.900 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2022-06-29T15:48:21.903 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.898+0000 7fa1023cb700 1 -- 172.21.15.47:0/722835560 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa0dc053b00 msgr2=0x7fa0dc055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:21.903 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.898+0000 7fa1023cb700 1 --2- 172.21.15.47:0/722835560 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa0dc053b00 0x7fa0dc055fc0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7fa0f001a000 tx=0x7fa0f0014000).stop 2022-06-29T15:48:21.903 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.898+0000 7fa1023cb700 1 -- 172.21.15.47:0/722835560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0fc061220 msgr2=0x7fa0fc0b1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:21.903 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.898+0000 7fa1023cb700 1 --2- 172.21.15.47:0/722835560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0fc061220 0x7fa0fc0b1ce0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 rx=0x7fa0ec001110 tx=0x7fa0ec007aa0).stop 2022-06-29T15:48:21.904 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.898+0000 7fa1023cb700 1 -- 172.21.15.47:0/722835560 shutdown_connections 2022-06-29T15:48:21.904 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.898+0000 7fa1023cb700 1 --2- 172.21.15.47:0/722835560 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa0dc053b00 0x7fa0dc055fc0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:21.904 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.898+0000 7fa1023cb700 1 --2- 172.21.15.47:0/722835560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0fc146720 0x7fa0fc152540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:21.904 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.898+0000 7fa1023cb700 1 --2- 172.21.15.47:0/722835560 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa0fc0b4320 0x7fa0fc0b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:21.905 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.898+0000 7fa1023cb700 1 --2- 172.21.15.47:0/722835560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0fc061220 0x7fa0fc0b1ce0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:21.905 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.898+0000 7fa1023cb700 1 -- 172.21.15.47:0/722835560 >> 172.21.15.47:0/722835560 conn(0x7fa0fc05c6b0 msgr2=0x7fa0fc1452a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:21.905 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.898+0000 7fa1023cb700 1 -- 172.21.15.47:0/722835560 shutdown_connections 2022-06-29T15:48:21.905 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:21.898+0000 7fa1023cb700 1 -- 172.21.15.47:0/722835560 wait complete. 2022-06-29T15:48:21.914 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-29T15:48:22.026 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.022+0000 7f605e2a2700 1 Processor -- start 2022-06-29T15:48:22.027 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.022+0000 7f605e2a2700 1 -- start start 2022-06-29T15:48:22.029 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.026+0000 7f605e2a2700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f605805f0b0 0x7f605805f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:22.029 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.026+0000 7f605e2a2700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6058061740 0x7f6058147140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:22.030 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.026+0000 7f605e2a2700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6058061e50 con 0x7f6058144670 2022-06-29T15:48:22.030 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.026+0000 7f605e2a2700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6058147630 con 0x7f605805f0b0 2022-06-29T15:48:22.030 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.026+0000 7f605e2a2700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60580620f0 con 0x7f6058061740 2022-06-29T15:48:22.031 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.026+0000 7f605d2a0700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f605805f0b0 0x7f605805f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:22.031 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.026+0000 7f605ca9f700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f6058144670 0x7f6058143950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35650/0 (socket says 172.21.15.47:35650) 2022-06-29T15:48:22.031 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.026+0000 7f605ca9f700 1 -- 172.21.15.47:0/1427427114 learned_addr learned my addr 172.21.15.47:0/1427427114 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:48:22.031 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.026+0000 7f605daa1700 1 --2- 172.21.15.47:0/1427427114 >> v2:172.21.15.64:3300/0 conn(0x7f6058061740 0x7f6058147140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:22.032 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.026+0000 7f605d2a0700 1 -- 172.21.15.47:0/1427427114 >> v2:172.21.15.64:3300/0 conn(0x7f6058061740 msgr2=0x7f6058147140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:22.032 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.026+0000 7f605d2a0700 1 --2- 172.21.15.47:0/1427427114 >> v2:172.21.15.64:3300/0 conn(0x7f6058061740 0x7f6058147140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:22.032 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.026+0000 7f605d2a0700 1 -- 172.21.15.47:0/1427427114 >> v1:172.21.15.47:6789/0 conn(0x7f6058144670 legacy=0x7f6058143950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:22.032 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.026+0000 7f605d2a0700 1 -- 172.21.15.47:0/1427427114 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60581434b0 con 0x7f605805f0b0 2022-06-29T15:48:22.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.026+0000 7f604ffff700 1 -- 172.21.15.47:0/1427427114 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2552394760 0 0) 0x7f6058061e50 con 0x7f6058144670 2022-06-29T15:48:22.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.026+0000 7f605d2a0700 1 --2- 172.21.15.47:0/1427427114 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f605805f0b0 0x7f605805f490 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 rx=0x7f6054002f70 tx=0x7f6054006df0).ready entity=mon.0 client_cookie=c084dda6a3d79b98 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:22.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.026+0000 7f604ffff700 1 -- 172.21.15.47:0/1427427114 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f605400d570 con 0x7f605805f0b0 2022-06-29T15:48:22.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.026+0000 7f604ffff700 1 -- 172.21.15.47:0/1427427114 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6054000e90 con 0x7f605805f0b0 2022-06-29T15:48:22.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.026+0000 7f604ffff700 1 -- 172.21.15.47:0/1427427114 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f605400d8c0 con 0x7f605805f0b0 2022-06-29T15:48:22.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.026+0000 7f605e2a2700 1 -- 172.21.15.47:0/1427427114 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f605805f0b0 msgr2=0x7f605805f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:22.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.026+0000 7f605e2a2700 1 --2- 172.21.15.47:0/1427427114 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f605805f0b0 0x7f605805f490 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 rx=0x7f6054002f70 tx=0x7f6054006df0).stop 2022-06-29T15:48:22.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.026+0000 7f605e2a2700 1 -- 172.21.15.47:0/1427427114 shutdown_connections 2022-06-29T15:48:22.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.026+0000 7f605e2a2700 1 --2- 172.21.15.47:0/1427427114 >> v2:172.21.15.64:3300/0 conn(0x7f6058061740 0x7f6058147140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:22.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.026+0000 7f605e2a2700 1 --2- 172.21.15.47:0/1427427114 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f605805f0b0 0x7f605805f490 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:22.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.026+0000 7f605e2a2700 1 -- 172.21.15.47:0/1427427114 >> 172.21.15.47:0/1427427114 conn(0x7f6058062570 msgr2=0x7f60580647d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:22.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.026+0000 7f605e2a2700 1 -- 172.21.15.47:0/1427427114 shutdown_connections 2022-06-29T15:48:22.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.026+0000 7f605e2a2700 1 -- 172.21.15.47:0/1427427114 wait complete. 2022-06-29T15:48:22.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.030+0000 7f605e2a2700 1 Processor -- start 2022-06-29T15:48:22.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.030+0000 7f605e2a2700 1 -- start start 2022-06-29T15:48:22.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.030+0000 7f605e2a2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f605805f0b0 0x7f60580b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:22.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.030+0000 7f605e2a2700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6058061740 0x7f6058145e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:22.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.030+0000 7f605e2a2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6058144670 0x7f605813d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:22.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.030+0000 7f605e2a2700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60580b68a0 con 0x7f6058144670 2022-06-29T15:48:22.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.030+0000 7f605e2a2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60580b6f70 con 0x7f605805f0b0 2022-06-29T15:48:22.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.030+0000 7f605e2a2700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60580b74d0 con 0x7f6058061740 2022-06-29T15:48:22.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.030+0000 7f605daa1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6058144670 0x7f605813d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:22.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.030+0000 7f605daa1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6058144670 0x7f605813d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50104/0 (socket says 172.21.15.47:50104) 2022-06-29T15:48:22.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.030+0000 7f605ca9f700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6058061740 0x7f6058145e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:22.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.030+0000 7f605d2a0700 1 --2- 172.21.15.47:0/2302613328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f605805f0b0 0x7f60580b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:22.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.030+0000 7f605daa1700 1 -- 172.21.15.47:0/2302613328 learned_addr learned my addr 172.21.15.47:0/2302613328 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:22.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.030+0000 7f605daa1700 1 -- 172.21.15.47:0/2302613328 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6058061740 msgr2=0x7f6058145e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:22.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.030+0000 7f605daa1700 1 --2- 172.21.15.47:0/2302613328 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6058061740 0x7f6058145e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:22.045 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.030+0000 7f605daa1700 1 -- 172.21.15.47:0/2302613328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f605805f0b0 msgr2=0x7f60580b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:22.045 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.030+0000 7f605daa1700 1 --2- 172.21.15.47:0/2302613328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f605805f0b0 0x7f60580b3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:22.045 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.030+0000 7f605daa1700 1 -- 172.21.15.47:0/2302613328 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f605400b180 con 0x7f6058144670 2022-06-29T15:48:22.045 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.030+0000 7f605ca9f700 1 --2- 172.21.15.47:0/2302613328 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6058061740 0x7f6058145e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:48:22.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.030+0000 7f605daa1700 1 --2- 172.21.15.47:0/2302613328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6058144670 0x7f605813d8a0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 rx=0x7f6050006e00 tx=0x7f60500091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:22.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.030+0000 7f604e7fc700 1 -- 172.21.15.47:0/2302613328 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f605000c640 con 0x7f6058144670 2022-06-29T15:48:22.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.030+0000 7f604e7fc700 1 -- 172.21.15.47:0/2302613328 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f605000c7a0 con 0x7f6058144670 2022-06-29T15:48:22.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.030+0000 7f605e2a2700 1 -- 172.21.15.47:0/2302613328 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60580b7770 con 0x7f6058144670 2022-06-29T15:48:22.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.030+0000 7f604e7fc700 1 -- 172.21.15.47:0/2302613328 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f605000cb00 con 0x7f6058144670 2022-06-29T15:48:22.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.030+0000 7f605e2a2700 1 -- 172.21.15.47:0/2302613328 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f605813f060 con 0x7f6058144670 2022-06-29T15:48:22.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.034+0000 7f605e2a2700 1 -- 172.21.15.47:0/2302613328 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60580573d0 con 0x7f6058144670 2022-06-29T15:48:22.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.034+0000 7f604e7fc700 1 -- 172.21.15.47:0/2302613328 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6050022070 con 0x7f6058144670 2022-06-29T15:48:22.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.034+0000 7f604e7fc700 1 --2- 172.21.15.47:0/2302613328 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f60400539e0 0x7f6040055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:22.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.034+0000 7f604e7fc700 1 -- 172.21.15.47:0/2302613328 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f605006c6d0 con 0x7f6058144670 2022-06-29T15:48:22.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.034+0000 7f605d2a0700 1 --2- 172.21.15.47:0/2302613328 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f60400539e0 0x7f6040055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:22.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.034+0000 7f605d2a0700 1 --2- 172.21.15.47:0/2302613328 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f60400539e0 0x7f6040055ea0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7f605401a000 tx=0x7f6054014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:22.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.042+0000 7f604e7fc700 1 -- 172.21.15.47:0/2302613328 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6050043490 con 0x7f6058144670 2022-06-29T15:48:22.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.170+0000 7f605e2a2700 1 -- 172.21.15.47:0/2302613328 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6058068830 con 0x7f6058144670 2022-06-29T15:48:22.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.170+0000 7f604e7fc700 1 -- 172.21.15.47:0/2302613328 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v78) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6050046ab0 con 0x7f6058144670 2022-06-29T15:48:22.176 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:48:22.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.174+0000 7f605e2a2700 1 -- 172.21.15.47:0/2302613328 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f60400539e0 msgr2=0x7f6040055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:22.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.174+0000 7f605e2a2700 1 --2- 172.21.15.47:0/2302613328 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f60400539e0 0x7f6040055ea0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7f605401a000 tx=0x7f6054014000).stop 2022-06-29T15:48:22.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.174+0000 7f605e2a2700 1 -- 172.21.15.47:0/2302613328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6058144670 msgr2=0x7f605813d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:22.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.174+0000 7f605e2a2700 1 --2- 172.21.15.47:0/2302613328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6058144670 0x7f605813d8a0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 rx=0x7f6050006e00 tx=0x7f60500091d0).stop 2022-06-29T15:48:22.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.174+0000 7f605e2a2700 1 -- 172.21.15.47:0/2302613328 shutdown_connections 2022-06-29T15:48:22.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.174+0000 7f605e2a2700 1 --2- 172.21.15.47:0/2302613328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6058144670 0x7f605813d8a0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:22.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.174+0000 7f605e2a2700 1 --2- 172.21.15.47:0/2302613328 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6058061740 0x7f6058145e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:22.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.174+0000 7f605e2a2700 1 --2- 172.21.15.47:0/2302613328 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f60400539e0 0x7f6040055ea0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:22.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.174+0000 7f605e2a2700 1 --2- 172.21.15.47:0/2302613328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f605805f0b0 0x7f60580b3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:22.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.174+0000 7f605e2a2700 1 -- 172.21.15.47:0/2302613328 >> 172.21.15.47:0/2302613328 conn(0x7f6058062570 msgr2=0x7f605813fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:22.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.174+0000 7f605e2a2700 1 -- 172.21.15.47:0/2302613328 shutdown_connections 2022-06-29T15:48:22.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.174+0000 7f605e2a2700 1 -- 172.21.15.47:0/2302613328 wait complete. 2022-06-29T15:48:22.186 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-29T15:48:22.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.290+0000 7fc4d6816700 1 Processor -- start 2022-06-29T15:48:22.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.290+0000 7fc4d6816700 1 -- start start 2022-06-29T15:48:22.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.290+0000 7fc4d6816700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc4d00b5c10 0x7fc4d013e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:22.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.290+0000 7fc4d6816700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc4d00601a0 0x7fc4d013e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:22.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.290+0000 7fc4d6816700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc4d00ab8d0 0x7fc4d00afd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:22.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.290+0000 7fc4d6816700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4d00b5ff0 con 0x7fc4d00601a0 2022-06-29T15:48:22.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.290+0000 7fc4d6816700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4d00ac910 con 0x7fc4d00ab8d0 2022-06-29T15:48:22.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.290+0000 7fc4d6816700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d00acbb0 con 0x7fc4d00b5c10 2022-06-29T15:48:22.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.290+0000 7fc4d5013700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc4d00601a0 0x7fc4d013e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:22.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.290+0000 7fc4d5013700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc4d00601a0 0x7fc4d013e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50110/0 (socket says 172.21.15.47:50110) 2022-06-29T15:48:22.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.290+0000 7fc4d5013700 1 -- 172.21.15.47:0/4224757999 learned_addr learned my addr 172.21.15.47:0/4224757999 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:22.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.294+0000 7fc4d5814700 1 --2- 172.21.15.47:0/4224757999 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc4d00b5c10 0x7fc4d013e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:22.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.294+0000 7fc4d6015700 1 --2- 172.21.15.47:0/4224757999 >> v2:172.21.15.64:3300/0 conn(0x7fc4d00ab8d0 0x7fc4d00afd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:22.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.294+0000 7fc4d5013700 1 -- 172.21.15.47:0/4224757999 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc4d00b5c10 msgr2=0x7fc4d013e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:22.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.294+0000 7fc4d5013700 1 --2- 172.21.15.47:0/4224757999 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc4d00b5c10 0x7fc4d013e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:22.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.294+0000 7fc4d5013700 1 -- 172.21.15.47:0/4224757999 >> v2:172.21.15.64:3300/0 conn(0x7fc4d00ab8d0 msgr2=0x7fc4d00afd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:22.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.294+0000 7fc4d5013700 1 --2- 172.21.15.47:0/4224757999 >> v2:172.21.15.64:3300/0 conn(0x7fc4d00ab8d0 0x7fc4d00afd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:22.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.294+0000 7fc4d5013700 1 -- 172.21.15.47:0/4224757999 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d00ace50 con 0x7fc4d00601a0 2022-06-29T15:48:22.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.294+0000 7fc4d6015700 1 --2- 172.21.15.47:0/4224757999 >> v2:172.21.15.64:3300/0 conn(0x7fc4d00ab8d0 0x7fc4d00afd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:48:22.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.294+0000 7fc4d5013700 1 --2- 172.21.15.47:0/4224757999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4d00601a0 0x7fc4d013e780 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 rx=0x7fc4c4002f70 tx=0x7fc4c4006eb0).ready entity=mon.0 client_cookie=5ddd15f8c4282bdb server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:22.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.294+0000 7fc4d4812700 1 -- 172.21.15.47:0/4224757999 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc4c400d570 con 0x7fc4d00601a0 2022-06-29T15:48:22.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.294+0000 7fc4d4812700 1 -- 172.21.15.47:0/4224757999 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4c4000e90 con 0x7fc4d00601a0 2022-06-29T15:48:22.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.294+0000 7fc4d4812700 1 -- 172.21.15.47:0/4224757999 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc4c400d900 con 0x7fc4d00601a0 2022-06-29T15:48:22.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.294+0000 7fc4d6816700 1 -- 172.21.15.47:0/4224757999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4d00601a0 msgr2=0x7fc4d013e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:22.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.294+0000 7fc4d6816700 1 --2- 172.21.15.47:0/4224757999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4d00601a0 0x7fc4d013e780 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 rx=0x7fc4c4002f70 tx=0x7fc4c4006eb0).stop 2022-06-29T15:48:22.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.294+0000 7fc4d6816700 1 -- 172.21.15.47:0/4224757999 shutdown_connections 2022-06-29T15:48:22.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.294+0000 7fc4d6816700 1 --2- 172.21.15.47:0/4224757999 >> v2:172.21.15.64:3300/0 conn(0x7fc4d00ab8d0 0x7fc4d00afd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:22.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.294+0000 7fc4d6816700 1 --2- 172.21.15.47:0/4224757999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4d00601a0 0x7fc4d013e780 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:22.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.294+0000 7fc4d6816700 1 --2- 172.21.15.47:0/4224757999 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc4d00b5c10 0x7fc4d013e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:22.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.294+0000 7fc4d6816700 1 -- 172.21.15.47:0/4224757999 >> 172.21.15.47:0/4224757999 conn(0x7fc4d0062560 msgr2=0x7fc4d0062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:22.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.294+0000 7fc4d6816700 1 -- 172.21.15.47:0/4224757999 shutdown_connections 2022-06-29T15:48:22.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.294+0000 7fc4d6816700 1 -- 172.21.15.47:0/4224757999 wait complete. 2022-06-29T15:48:22.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.294+0000 7fc4d6816700 1 Processor -- start 2022-06-29T15:48:22.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.294+0000 7fc4d6816700 1 -- start start 2022-06-29T15:48:22.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.298+0000 7fc4d6816700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc4d00601a0 0x7fc4d00aea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:22.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.298+0000 7fc4d6816700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4d00ab8d0 0x7fc4d0137530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:22.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.298+0000 7fc4d6816700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4d00b5c10 0x7fc4d0137a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:22.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.298+0000 7fc4d6816700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d01443f0 con 0x7fc4d00ab8d0 2022-06-29T15:48:22.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.298+0000 7fc4d6816700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d0144620 con 0x7fc4d00b5c10 2022-06-29T15:48:22.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.298+0000 7fc4d6816700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d01448c0 con 0x7fc4d00601a0 2022-06-29T15:48:22.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.298+0000 7fc4d5013700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4d00ab8d0 0x7fc4d0137530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:22.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.298+0000 7fc4d5013700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4d00ab8d0 0x7fc4d0137530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50114/0 (socket says 172.21.15.47:50114) 2022-06-29T15:48:22.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.298+0000 7fc4d5013700 1 -- 172.21.15.47:0/1115034122 learned_addr learned my addr 172.21.15.47:0/1115034122 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:22.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.298+0000 7fc4d5814700 1 --2- 172.21.15.47:0/1115034122 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc4d00601a0 0x7fc4d00aea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:22.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.298+0000 7fc4d6015700 1 --2- 172.21.15.47:0/1115034122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4d00b5c10 0x7fc4d0137a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:22.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.298+0000 7fc4d5013700 1 -- 172.21.15.47:0/1115034122 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc4d00601a0 msgr2=0x7fc4d00aea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:22.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.298+0000 7fc4d5013700 1 --2- 172.21.15.47:0/1115034122 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc4d00601a0 0x7fc4d00aea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:22.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.298+0000 7fc4d5013700 1 -- 172.21.15.47:0/1115034122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4d00b5c10 msgr2=0x7fc4d0137a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:22.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.298+0000 7fc4d5013700 1 --2- 172.21.15.47:0/1115034122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4d00b5c10 0x7fc4d0137a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:22.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.298+0000 7fc4d5013700 1 -- 172.21.15.47:0/1115034122 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c400b1e0 con 0x7fc4d00ab8d0 2022-06-29T15:48:22.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.298+0000 7fc4d6015700 1 --2- 172.21.15.47:0/1115034122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4d00b5c10 0x7fc4d0137a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:48:22.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.298+0000 7fc4d5013700 1 --2- 172.21.15.47:0/1115034122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4d00ab8d0 0x7fc4d0137530 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 rx=0x7fc4c4002f70 tx=0x7fc4c4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:22.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.298+0000 7fc4c2ffd700 1 -- 172.21.15.47:0/1115034122 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc4c4011760 con 0x7fc4d00ab8d0 2022-06-29T15:48:22.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.298+0000 7fc4c2ffd700 1 -- 172.21.15.47:0/1115034122 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4c40118c0 con 0x7fc4d00ab8d0 2022-06-29T15:48:22.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.298+0000 7fc4d6816700 1 -- 172.21.15.47:0/1115034122 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4d0144b60 con 0x7fc4d00ab8d0 2022-06-29T15:48:22.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.298+0000 7fc4c2ffd700 1 -- 172.21.15.47:0/1115034122 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc4c4011c20 con 0x7fc4d00ab8d0 2022-06-29T15:48:22.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.298+0000 7fc4d6816700 1 -- 172.21.15.47:0/1115034122 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4d00b3bd0 con 0x7fc4d00ab8d0 2022-06-29T15:48:22.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.298+0000 7fc4d6816700 1 -- 172.21.15.47:0/1115034122 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4d00573d0 con 0x7fc4d00ab8d0 2022-06-29T15:48:22.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.302+0000 7fc4c2ffd700 1 -- 172.21.15.47:0/1115034122 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc4c4025030 con 0x7fc4d00ab8d0 2022-06-29T15:48:22.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.302+0000 7fc4c2ffd700 1 --2- 172.21.15.47:0/1115034122 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc4b00539e0 0x7fc4b0055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:22.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.302+0000 7fc4c2ffd700 1 -- 172.21.15.47:0/1115034122 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc4c406d590 con 0x7fc4d00ab8d0 2022-06-29T15:48:22.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.302+0000 7fc4d5814700 1 --2- 172.21.15.47:0/1115034122 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc4b00539e0 0x7fc4b0055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:22.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.302+0000 7fc4d5814700 1 --2- 172.21.15.47:0/1115034122 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc4b00539e0 0x7fc4b0055ea0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7fc4cc00c000 tx=0x7fc4cc008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:22.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.310+0000 7fc4c2ffd700 1 -- 172.21.15.47:0/1115034122 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc4c4044350 con 0x7fc4d00ab8d0 2022-06-29T15:48:22.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.438+0000 7fc4d6816700 1 -- 172.21.15.47:0/1115034122 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc4d0062220 con 0x7fc4d00ab8d0 2022-06-29T15:48:22.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.438+0000 7fc4c2ffd700 1 -- 172.21.15.47:0/1115034122 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v78) v1 ==== 104+0+10 (secure 0 0 0) 0x7fc4c40230a0 con 0x7fc4d00ab8d0 2022-06-29T15:48:22.441 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:48:22.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.438+0000 7fc4d6816700 1 -- 172.21.15.47:0/1115034122 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc4b00539e0 msgr2=0x7fc4b0055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:22.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.438+0000 7fc4d6816700 1 --2- 172.21.15.47:0/1115034122 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc4b00539e0 0x7fc4b0055ea0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7fc4cc00c000 tx=0x7fc4cc008040).stop 2022-06-29T15:48:22.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.438+0000 7fc4d6816700 1 -- 172.21.15.47:0/1115034122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4d00ab8d0 msgr2=0x7fc4d0137530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:22.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.438+0000 7fc4d6816700 1 --2- 172.21.15.47:0/1115034122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4d00ab8d0 0x7fc4d0137530 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 rx=0x7fc4c4002f70 tx=0x7fc4c4015000).stop 2022-06-29T15:48:22.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.438+0000 7fc4d6816700 1 -- 172.21.15.47:0/1115034122 shutdown_connections 2022-06-29T15:48:22.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.438+0000 7fc4d6816700 1 --2- 172.21.15.47:0/1115034122 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc4b00539e0 0x7fc4b0055ea0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:22.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.438+0000 7fc4d6816700 1 --2- 172.21.15.47:0/1115034122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4d00b5c10 0x7fc4d0137a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:22.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.438+0000 7fc4d6816700 1 --2- 172.21.15.47:0/1115034122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4d00ab8d0 0x7fc4d0137530 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:22.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.438+0000 7fc4d6816700 1 --2- 172.21.15.47:0/1115034122 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc4d00601a0 0x7fc4d00aea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:22.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.438+0000 7fc4d6816700 1 -- 172.21.15.47:0/1115034122 >> 172.21.15.47:0/1115034122 conn(0x7fc4d0062560 msgr2=0x7fc4d0139690 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:22.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.438+0000 7fc4d6816700 1 -- 172.21.15.47:0/1115034122 shutdown_connections 2022-06-29T15:48:22.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.438+0000 7fc4d6816700 1 -- 172.21.15.47:0/1115034122 wait complete. 2022-06-29T15:48:22.454 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-29T15:48:22.564 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.562+0000 7fd2ee735700 1 Processor -- start 2022-06-29T15:48:22.565 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.562+0000 7fd2ee735700 1 -- start start 2022-06-29T15:48:22.570 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.562+0000 7fd2ee735700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd2e8061670 0x7fd2e8061a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:22.571 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.562+0000 7fd2ee735700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd2e80b0700 0x7fd2e81436c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:22.571 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.562+0000 7fd2ee735700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2e813db90 con 0x7fd2e80b0fe0 2022-06-29T15:48:22.571 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.562+0000 7fd2ee735700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2e813fed0 con 0x7fd2e80b0700 2022-06-29T15:48:22.572 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.562+0000 7fd2ee735700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e80af0a0 con 0x7fd2e8061670 2022-06-29T15:48:22.572 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.562+0000 7fd2eccd2700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd2e80b0700 0x7fd2e81436c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:22.573 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.562+0000 7fd2e77fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd2e80b0fe0 0x7fd2e813d480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35674/0 (socket says 172.21.15.47:35674) 2022-06-29T15:48:22.573 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.562+0000 7fd2e77fe700 1 -- 172.21.15.47:0/216148684 learned_addr learned my addr 172.21.15.47:0/216148684 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:48:22.574 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.562+0000 7fd2e7fff700 1 --2- 172.21.15.47:0/216148684 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd2e8061670 0x7fd2e8061a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:22.574 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.562+0000 7fd2eccd2700 1 -- 172.21.15.47:0/216148684 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd2e8061670 msgr2=0x7fd2e8061a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:22.575 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.562+0000 7fd2eccd2700 1 --2- 172.21.15.47:0/216148684 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd2e8061670 0x7fd2e8061a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:22.575 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.562+0000 7fd2eccd2700 1 -- 172.21.15.47:0/216148684 >> v1:172.21.15.47:6789/0 conn(0x7fd2e80b0fe0 legacy=0x7fd2e813d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:22.576 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.562+0000 7fd2eccd2700 1 -- 172.21.15.47:0/216148684 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e80af340 con 0x7fd2e80b0700 2022-06-29T15:48:22.576 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.562+0000 7fd2eccd2700 1 --2- 172.21.15.47:0/216148684 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2e80b0700 0x7fd2e81436c0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 rx=0x7fd2e00032f0 tx=0x7fd2e00091d0).ready entity=mon.0 client_cookie=2a33a6bba140d349 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:22.576 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.562+0000 7fd2e6ffd700 1 -- 172.21.15.47:0/216148684 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2e000b6c0 con 0x7fd2e80b0700 2022-06-29T15:48:22.577 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.562+0000 7fd2e6ffd700 1 -- 172.21.15.47:0/216148684 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2e000b820 con 0x7fd2e80b0700 2022-06-29T15:48:22.577 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.562+0000 7fd2e6ffd700 1 -- 172.21.15.47:0/216148684 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2e000b990 con 0x7fd2e80b0700 2022-06-29T15:48:22.578 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.562+0000 7fd2ee735700 1 -- 172.21.15.47:0/216148684 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2e80b0700 msgr2=0x7fd2e81436c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:22.578 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.566+0000 7fd2ee735700 1 --2- 172.21.15.47:0/216148684 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2e80b0700 0x7fd2e81436c0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 rx=0x7fd2e00032f0 tx=0x7fd2e00091d0).stop 2022-06-29T15:48:22.578 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.566+0000 7fd2ee735700 1 -- 172.21.15.47:0/216148684 shutdown_connections 2022-06-29T15:48:22.579 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.566+0000 7fd2ee735700 1 --2- 172.21.15.47:0/216148684 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2e80b0700 0x7fd2e81436c0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:22.579 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.566+0000 7fd2ee735700 1 --2- 172.21.15.47:0/216148684 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd2e8061670 0x7fd2e8061a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:22.580 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.566+0000 7fd2ee735700 1 -- 172.21.15.47:0/216148684 >> 172.21.15.47:0/216148684 conn(0x7fd2e8063800 msgr2=0x7fd2e8063be0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:22.580 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.566+0000 7fd2ee735700 1 -- 172.21.15.47:0/216148684 shutdown_connections 2022-06-29T15:48:22.580 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.566+0000 7fd2ee735700 1 -- 172.21.15.47:0/216148684 wait complete. 2022-06-29T15:48:22.581 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.566+0000 7fd2ee735700 1 Processor -- start 2022-06-29T15:48:22.581 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.566+0000 7fd2ee735700 1 -- start start 2022-06-29T15:48:22.585 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.566+0000 7fd2ee735700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd2e8061670 0x7fd2e813f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:22.585 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.570+0000 7fd2ee735700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2e80b0700 0x7fd2e80ac290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:22.586 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.570+0000 7fd2ee735700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2e80b0fe0 0x7fd2e80ac780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:22.586 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.570+0000 7fd2ee735700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e80aa510 con 0x7fd2e80b0700 2022-06-29T15:48:22.586 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.570+0000 7fd2ee735700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e80acc70 con 0x7fd2e80b0fe0 2022-06-29T15:48:22.586 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.570+0000 7fd2ee735700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e80aa740 con 0x7fd2e8061670 2022-06-29T15:48:22.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.570+0000 7fd2e77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2e80b0700 0x7fd2e80ac290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:22.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.570+0000 7fd2e77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2e80b0700 0x7fd2e80ac290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50128/0 (socket says 172.21.15.47:50128) 2022-06-29T15:48:22.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.570+0000 7fd2e77fe700 1 -- 172.21.15.47:0/888199409 learned_addr learned my addr 172.21.15.47:0/888199409 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:22.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.570+0000 7fd2e7fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd2e8061670 0x7fd2e813f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:22.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.570+0000 7fd2eccd2700 1 --2- 172.21.15.47:0/888199409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2e80b0fe0 0x7fd2e80ac780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:22.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.570+0000 7fd2e77fe700 1 -- 172.21.15.47:0/888199409 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd2e8061670 msgr2=0x7fd2e813f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:22.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.570+0000 7fd2e77fe700 1 --2- 172.21.15.47:0/888199409 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd2e8061670 0x7fd2e813f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:22.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.570+0000 7fd2e77fe700 1 -- 172.21.15.47:0/888199409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2e80b0fe0 msgr2=0x7fd2e80ac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:22.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.570+0000 7fd2e77fe700 1 --2- 172.21.15.47:0/888199409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2e80b0fe0 0x7fd2e80ac780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:22.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.570+0000 7fd2e77fe700 1 -- 172.21.15.47:0/888199409 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e0009470 con 0x7fd2e80b0700 2022-06-29T15:48:22.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.570+0000 7fd2e77fe700 1 --2- 172.21.15.47:0/888199409 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2e80b0700 0x7fd2e80ac290 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 rx=0x7fd2e81435a0 tx=0x7fd2d8007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:22.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.570+0000 7fd2e57fa700 1 -- 172.21.15.47:0/888199409 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2d800ce50 con 0x7fd2e80b0700 2022-06-29T15:48:22.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.570+0000 7fd2e57fa700 1 -- 172.21.15.47:0/888199409 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2d8007110 con 0x7fd2e80b0700 2022-06-29T15:48:22.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.570+0000 7fd2e57fa700 1 -- 172.21.15.47:0/888199409 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2d8007470 con 0x7fd2e80b0700 2022-06-29T15:48:22.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.570+0000 7fd2ee735700 1 -- 172.21.15.47:0/888199409 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2e80b5630 con 0x7fd2e80b0700 2022-06-29T15:48:22.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.570+0000 7fd2ee735700 1 -- 172.21.15.47:0/888199409 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2e80b5b70 con 0x7fd2e80b0700 2022-06-29T15:48:22.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.570+0000 7fd2ee735700 1 -- 172.21.15.47:0/888199409 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2e80b5d30 con 0x7fd2e80b0700 2022-06-29T15:48:22.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.574+0000 7fd2e57fa700 1 -- 172.21.15.47:0/888199409 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd2d801b070 con 0x7fd2e80b0700 2022-06-29T15:48:22.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.574+0000 7fd2e57fa700 1 --2- 172.21.15.47:0/888199409 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd2cc053a10 0x7fd2cc055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:22.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.574+0000 7fd2e57fa700 1 -- 172.21.15.47:0/888199409 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd2d806b3b0 con 0x7fd2e80b0700 2022-06-29T15:48:22.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.574+0000 7fd2e7fff700 1 --2- 172.21.15.47:0/888199409 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd2cc053a10 0x7fd2cc055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:22.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.574+0000 7fd2e7fff700 1 --2- 172.21.15.47:0/888199409 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd2cc053a10 0x7fd2cc055ed0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7fd2d000b000 tx=0x7fd2d0006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:22.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.578+0000 7fd2e57fa700 1 -- 172.21.15.47:0/888199409 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd2d8042170 con 0x7fd2e80b0700 2022-06-29T15:48:22.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.706+0000 7fd2ee735700 1 -- 172.21.15.47:0/888199409 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd2e80573d0 con 0x7fd2e80b0700 2022-06-29T15:48:22.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.706+0000 7fd2e57fa700 1 -- 172.21.15.47:0/888199409 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v78) v1 ==== 100+0+10 (secure 0 0 0) 0x7fd2d8045790 con 0x7fd2e80b0700 2022-06-29T15:48:22.711 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:48:22.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.710+0000 7fd2ee735700 1 -- 172.21.15.47:0/888199409 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd2cc053a10 msgr2=0x7fd2cc055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:22.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.710+0000 7fd2ee735700 1 --2- 172.21.15.47:0/888199409 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd2cc053a10 0x7fd2cc055ed0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7fd2d000b000 tx=0x7fd2d0006df0).stop 2022-06-29T15:48:22.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.710+0000 7fd2ee735700 1 -- 172.21.15.47:0/888199409 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2e80b0700 msgr2=0x7fd2e80ac290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:22.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.710+0000 7fd2ee735700 1 --2- 172.21.15.47:0/888199409 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2e80b0700 0x7fd2e80ac290 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 rx=0x7fd2e81435a0 tx=0x7fd2d8007c00).stop 2022-06-29T15:48:22.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.710+0000 7fd2ee735700 1 -- 172.21.15.47:0/888199409 shutdown_connections 2022-06-29T15:48:22.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.710+0000 7fd2ee735700 1 --2- 172.21.15.47:0/888199409 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd2cc053a10 0x7fd2cc055ed0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:22.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.710+0000 7fd2ee735700 1 --2- 172.21.15.47:0/888199409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2e80b0fe0 0x7fd2e80ac780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:22.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.710+0000 7fd2ee735700 1 --2- 172.21.15.47:0/888199409 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2e80b0700 0x7fd2e80ac290 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:22.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.710+0000 7fd2ee735700 1 --2- 172.21.15.47:0/888199409 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd2e8061670 0x7fd2e813f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:22.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.710+0000 7fd2ee735700 1 -- 172.21.15.47:0/888199409 >> 172.21.15.47:0/888199409 conn(0x7fd2e8063800 msgr2=0x7fd2e813e630 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:22.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.710+0000 7fd2ee735700 1 -- 172.21.15.47:0/888199409 shutdown_connections 2022-06-29T15:48:22.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.710+0000 7fd2ee735700 1 -- 172.21.15.47:0/888199409 wait complete. 2022-06-29T15:48:22.729 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-29T15:48:22.730 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:48:22.833 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.830+0000 7fe688874700 1 Processor -- start 2022-06-29T15:48:22.833 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.830+0000 7fe688874700 1 -- start start 2022-06-29T15:48:22.836 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.830+0000 7fe688874700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe6840630f0 0x7fe6840b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:22.836 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.830+0000 7fe688874700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe68405f0b0 0x7fe6840b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:22.837 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.830+0000 7fe688874700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe684062180 con 0x7fe684061da0 2022-06-29T15:48:22.837 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.830+0000 7fe688874700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe68405fae0 con 0x7fe6840630f0 2022-06-29T15:48:22.837 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.830+0000 7fe688874700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6840b2ea0 con 0x7fe68405f0b0 2022-06-29T15:48:22.837 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.830+0000 7fe682ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe6840630f0 0x7fe6840b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:22.838 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.830+0000 7fe682ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe6840630f0 0x7fe6840b0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50136/0 (socket says 172.21.15.47:50136) 2022-06-29T15:48:22.838 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.830+0000 7fe682ffd700 1 -- 172.21.15.47:0/272398191 learned_addr learned my addr 172.21.15.47:0/272398191 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:22.839 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.830+0000 7fe683fff700 1 --2- 172.21.15.47:0/272398191 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe68405f0b0 0x7fe6840b17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:22.839 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.830+0000 7fe682ffd700 1 -- 172.21.15.47:0/272398191 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe68405f0b0 msgr2=0x7fe6840b17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:22.840 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.830+0000 7fe682ffd700 1 --2- 172.21.15.47:0/272398191 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe68405f0b0 0x7fe6840b17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:22.840 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.830+0000 7fe682ffd700 1 -- 172.21.15.47:0/272398191 >> v1:172.21.15.64:6789/0 conn(0x7fe684061da0 legacy=0x7fe6840b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:22.840 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.830+0000 7fe682ffd700 1 -- 172.21.15.47:0/272398191 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6840b3140 con 0x7fe6840630f0 2022-06-29T15:48:22.841 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.830+0000 7fe682ffd700 1 --2- 172.21.15.47:0/272398191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6840630f0 0x7fe6840b0e50 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 rx=0x7fe678002f70 tx=0x7fe678006eb0).ready entity=mon.0 client_cookie=1ccff4eebe73bb0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:22.841 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.830+0000 7fe6827fc700 1 -- 172.21.15.47:0/272398191 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe67800d570 con 0x7fe6840630f0 2022-06-29T15:48:22.842 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.830+0000 7fe6827fc700 1 -- 172.21.15.47:0/272398191 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe678000e90 con 0x7fe6840630f0 2022-06-29T15:48:22.842 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.830+0000 7fe6827fc700 1 -- 172.21.15.47:0/272398191 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe67800d900 con 0x7fe6840630f0 2022-06-29T15:48:22.843 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.834+0000 7fe688874700 1 -- 172.21.15.47:0/272398191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6840630f0 msgr2=0x7fe6840b0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:22.843 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.834+0000 7fe688874700 1 --2- 172.21.15.47:0/272398191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6840630f0 0x7fe6840b0e50 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 rx=0x7fe678002f70 tx=0x7fe678006eb0).stop 2022-06-29T15:48:22.843 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.834+0000 7fe688874700 1 -- 172.21.15.47:0/272398191 shutdown_connections 2022-06-29T15:48:22.843 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.834+0000 7fe688874700 1 --2- 172.21.15.47:0/272398191 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe68405f0b0 0x7fe6840b17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:22.844 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.834+0000 7fe688874700 1 --2- 172.21.15.47:0/272398191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6840630f0 0x7fe6840b0e50 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:22.844 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.834+0000 7fe688874700 1 -- 172.21.15.47:0/272398191 >> 172.21.15.47:0/272398191 conn(0x7fe684063670 msgr2=0x7fe684063a80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:22.844 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.834+0000 7fe688874700 1 -- 172.21.15.47:0/272398191 shutdown_connections 2022-06-29T15:48:22.844 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.834+0000 7fe688874700 1 -- 172.21.15.47:0/272398191 wait complete. 2022-06-29T15:48:22.847 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.834+0000 7fe688874700 1 Processor -- start 2022-06-29T15:48:22.847 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.834+0000 7fe688874700 1 -- start start 2022-06-29T15:48:22.848 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.834+0000 7fe688874700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe68405f0b0 0x7fe6840a2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:22.848 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.834+0000 7fe688874700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe684061da0 0x7fe6840b55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:22.848 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.834+0000 7fe688874700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe6840630f0 0x7fe6840a32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:22.848 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.834+0000 7fe688874700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6840b17f0 con 0x7fe684061da0 2022-06-29T15:48:22.849 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.834+0000 7fe688874700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6840a23b0 con 0x7fe68405f0b0 2022-06-29T15:48:22.849 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.834+0000 7fe688874700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6840a59b0 con 0x7fe6840630f0 2022-06-29T15:48:22.849 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.834+0000 7fe682ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe684061da0 0x7fe6840b55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:22.849 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.838+0000 7fe682ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe684061da0 0x7fe6840b55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50142/0 (socket says 172.21.15.47:50142) 2022-06-29T15:48:22.850 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.838+0000 7fe682ffd700 1 -- 172.21.15.47:0/686864022 learned_addr learned my addr 172.21.15.47:0/686864022 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:22.850 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.838+0000 7fe683fff700 1 --2- 172.21.15.47:0/686864022 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe6840630f0 0x7fe6840a32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:22.850 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.838+0000 7fe6837fe700 1 --2- 172.21.15.47:0/686864022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe68405f0b0 0x7fe6840a2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:22.850 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.838+0000 7fe682ffd700 1 -- 172.21.15.47:0/686864022 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe6840630f0 msgr2=0x7fe6840a32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:22.850 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.838+0000 7fe682ffd700 1 --2- 172.21.15.47:0/686864022 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe6840630f0 0x7fe6840a32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:22.851 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.838+0000 7fe682ffd700 1 -- 172.21.15.47:0/686864022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe68405f0b0 msgr2=0x7fe6840a2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:22.851 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.838+0000 7fe682ffd700 1 --2- 172.21.15.47:0/686864022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe68405f0b0 0x7fe6840a2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:22.851 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.838+0000 7fe682ffd700 1 -- 172.21.15.47:0/686864022 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe67800b1e0 con 0x7fe684061da0 2022-06-29T15:48:22.851 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.838+0000 7fe683fff700 1 --2- 172.21.15.47:0/686864022 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe6840630f0 0x7fe6840a32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:48:22.852 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.838+0000 7fe682ffd700 1 --2- 172.21.15.47:0/686864022 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe684061da0 0x7fe6840b55b0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0x7fe678002f70 tx=0x7fe678015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:22.852 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.838+0000 7fe680ff9700 1 -- 172.21.15.47:0/686864022 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6780116e0 con 0x7fe684061da0 2022-06-29T15:48:22.852 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.838+0000 7fe680ff9700 1 -- 172.21.15.47:0/686864022 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe678011840 con 0x7fe684061da0 2022-06-29T15:48:22.852 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.838+0000 7fe680ff9700 1 -- 172.21.15.47:0/686864022 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe678011ba0 con 0x7fe684061da0 2022-06-29T15:48:22.852 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.838+0000 7fe688874700 1 -- 172.21.15.47:0/686864022 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6840a1e70 con 0x7fe684061da0 2022-06-29T15:48:22.853 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.838+0000 7fe688874700 1 -- 172.21.15.47:0/686864022 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6840ad360 con 0x7fe684061da0 2022-06-29T15:48:22.853 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.838+0000 7fe680ff9700 1 -- 172.21.15.47:0/686864022 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe678025030 con 0x7fe684061da0 2022-06-29T15:48:22.853 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.838+0000 7fe688874700 1 -- 172.21.15.47:0/686864022 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6840573d0 con 0x7fe684061da0 2022-06-29T15:48:22.853 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.838+0000 7fe680ff9700 1 --2- 172.21.15.47:0/686864022 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe664053a70 0x7fe664055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:22.854 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.842+0000 7fe680ff9700 1 -- 172.21.15.47:0/686864022 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe67806d010 con 0x7fe684061da0 2022-06-29T15:48:22.854 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.842+0000 7fe6837fe700 1 --2- 172.21.15.47:0/686864022 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe664053a70 0x7fe664055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:22.854 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.842+0000 7fe6837fe700 1 --2- 172.21.15.47:0/686864022 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe664053a70 0x7fe664055f30 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7fe67400d000 tx=0x7fe674007b90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:22.855 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:22.850+0000 7fe680ff9700 1 -- 172.21.15.47:0/686864022 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe678043e80 con 0x7fe684061da0 2022-06-29T15:48:23.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:23.010+0000 7fe688874700 1 -- 172.21.15.47:0/686864022 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe6840b1210 con 0x7fe684061da0 2022-06-29T15:48:23.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:23.014+0000 7fe680ff9700 1 -- 172.21.15.47:0/686864022 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+2493 (secure 0 0 0) 0x7fe678023070 con 0x7fe684061da0 2022-06-29T15:48:23.017 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:48:23.017 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":35,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6291,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.47:6835/2910037081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2910037081},{"type":"v1","addr":"172.21.15.47:6835","nonce":2910037081}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":34},{"gid":6300,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/359532891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":359532891},{"type":"v1","addr":"172.21.15.47:6837","nonce":359532891}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":33}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:48:18.345845+0000","modified":"2022-06-29T15:48:19.349718+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":55562},"failed":[],"damaged":[],"stopped":[],"info":{"gid_55562":{"gid":55562,"name":"b","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6833/3871453337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3871453337},{"type":"v1","addr":"172.21.15.64:6833","nonce":3871453337}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2022-06-29T15:48:23.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:23.014+0000 7fe688874700 1 -- 172.21.15.47:0/686864022 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe664053a70 msgr2=0x7fe664055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:23.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:23.014+0000 7fe688874700 1 --2- 172.21.15.47:0/686864022 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe664053a70 0x7fe664055f30 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7fe67400d000 tx=0x7fe674007b90).stop 2022-06-29T15:48:23.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:23.014+0000 7fe688874700 1 -- 172.21.15.47:0/686864022 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe684061da0 msgr2=0x7fe6840b55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:23.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:23.014+0000 7fe688874700 1 --2- 172.21.15.47:0/686864022 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe684061da0 0x7fe6840b55b0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0x7fe678002f70 tx=0x7fe678015000).stop 2022-06-29T15:48:23.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:23.014+0000 7fe688874700 1 -- 172.21.15.47:0/686864022 shutdown_connections 2022-06-29T15:48:23.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:23.014+0000 7fe688874700 1 --2- 172.21.15.47:0/686864022 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe6840630f0 0x7fe6840a32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:23.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:23.014+0000 7fe688874700 1 --2- 172.21.15.47:0/686864022 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe684061da0 0x7fe6840b55b0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:23.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:23.014+0000 7fe688874700 1 --2- 172.21.15.47:0/686864022 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe664053a70 0x7fe664055f30 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:23.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:23.014+0000 7fe688874700 1 --2- 172.21.15.47:0/686864022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe68405f0b0 0x7fe6840a2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:23.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:23.014+0000 7fe688874700 1 -- 172.21.15.47:0/686864022 >> 172.21.15.47:0/686864022 conn(0x7fe684063670 msgr2=0x7fe6840ae9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:23.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:23.014+0000 7fe688874700 1 -- 172.21.15.47:0/686864022 shutdown_connections 2022-06-29T15:48:23.022 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:23.014+0000 7fe688874700 1 -- 172.21.15.47:0/686864022 wait complete. 2022-06-29T15:48:23.022 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 35 2022-06-29T15:48:23.333 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 35, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-29T15:48:18.345845+0000', 'modified': '2022-06-29T15:48:19.349718+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 55562}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_55562': {'gid': 55562, 'name': 'b', 'rank': 0, 'incarnation': 34, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.64:6833/3871453337', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6832', 'nonce': 3871453337}, {'type': 'v1', 'addr': '172.21.15.64:6833', 'nonce': 3871453337}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-29T15:48:23.334 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-29T15:48:23.337 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-06-29T15:48:23.454 INFO:teuthology.orchestra.run.smithi064.stdout:{"cluster_fsid":"9a68894a-53ce-400e-838b-14eab91b834c","whoami":0,"id":55562,"want_state":"up:active","state":"up:active","rank_uptime":5.0932955440000001,"mdsmap_epoch":35,"osdmap_epoch":78,"osdmap_epoch_barrier":78,"uptime":9.4974615849999999} 2022-06-29T15:48:23.455 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9a68894a-53ce-400e-838b-14eab91b834c", "id": 55562, "mdsmap_epoch": 35, "osdmap_epoch": 78, "osdmap_epoch_barrier": 78, "rank_uptime": 5.093295544, "state": "up:active", "uptime": 9.497461585, "want_state": "up:active", "whoami": 0 } 2022-06-29T15:48:23.456 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-29T15:48:23.456 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-06-29T15:48:23.457 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi099.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-06-29T15:48:23.457 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:48:23.458 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:48:23.467 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2022-06-29T15:48:23.526 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:48:23.527 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:48:23.537 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:48:23.538 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:48:23.539 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:48:23.540 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2022-06-29T15:48:23.545 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-29T15:48:23.546 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:48:23.548 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2022-06-29T15:48:23.563 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:48:23.563 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:48:23.618 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:48:23.620 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:48:23.622 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:48:23.623 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2022-06-29T15:48:23.626 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[12842]: starting ceph client 2022-06-29T15:48:23.626 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:2022-06-29T15:48:23.623+0000 7f47a418c080 -1 init, newargv = 0x559bac6da4b0 newargc=9 2022-06-29T15:48:23.636 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[12842]: starting fuse 2022-06-29T15:48:24.731 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2022-06-29T15:48:24.750 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:48:24.750 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:48:24.808 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:48:24.809 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:48:24.810 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:48:24.811 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2022-06-29T15:48:24.817 INFO:teuthology.orchestra.run.smithi099.stdout:52 2022-06-29T15:48:24.818 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-29T15:48:24.818 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:48:24.819 DEBUG:teuthology.orchestra.run.smithi099:> sudo python3 -c ' 2022-06-29T15:48:24.819 DEBUG:teuthology.orchestra.run.smithi099:> import glob 2022-06-29T15:48:24.820 DEBUG:teuthology.orchestra.run.smithi099:> import re 2022-06-29T15:48:24.820 DEBUG:teuthology.orchestra.run.smithi099:> import os 2022-06-29T15:48:24.821 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2022-06-29T15:48:24.821 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:48:24.822 DEBUG:teuthology.orchestra.run.smithi099:> def find_socket(client_name): 2022-06-29T15:48:24.822 DEBUG:teuthology.orchestra.run.smithi099:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-06-29T15:48:24.822 DEBUG:teuthology.orchestra.run.smithi099:> files = glob.glob(asok_path) 2022-06-29T15:48:24.823 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:48:24.823 DEBUG:teuthology.orchestra.run.smithi099:> # Given a non-glob path, it better be there 2022-06-29T15:48:24.824 DEBUG:teuthology.orchestra.run.smithi099:> if "*" not in asok_path: 2022-06-29T15:48:24.824 DEBUG:teuthology.orchestra.run.smithi099:> assert(len(files) == 1) 2022-06-29T15:48:24.825 DEBUG:teuthology.orchestra.run.smithi099:> return files[0] 2022-06-29T15:48:24.825 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:48:24.825 DEBUG:teuthology.orchestra.run.smithi099:> for f in files: 2022-06-29T15:48:24.826 DEBUG:teuthology.orchestra.run.smithi099:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-29T15:48:24.827 DEBUG:teuthology.orchestra.run.smithi099:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-29T15:48:24.827 DEBUG:teuthology.orchestra.run.smithi099:> return f 2022-06-29T15:48:24.827 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-29T15:48:24.828 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:48:24.828 DEBUG:teuthology.orchestra.run.smithi099:> print(find_socket("client.0")) 2022-06-29T15:48:24.829 DEBUG:teuthology.orchestra.run.smithi099:> ' 2022-06-29T15:48:24.872 INFO:teuthology.orchestra.run.smithi099.stdout:/var/run/ceph/ceph-client.0.12842.asok 2022-06-29T15:48:24.878 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.12842.asok 2022-06-29T15:48:24.879 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:48:24.879 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.12842.asok status 2022-06-29T15:48:24.970 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2022-06-29T15:48:24.970 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": { 2022-06-29T15:48:24.971 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_sha1": "f25ffac72e5a9e63860ff7deb6767a11f0559a16", 2022-06-29T15:48:24.971 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 15.2.16-304-gf25ffac7 (f25ffac72e5a9e63860ff7deb6767a11f0559a16) octopus (stable)", 2022-06-29T15:48:24.971 INFO:teuthology.orchestra.run.smithi099.stdout: "entity_id": "0", 2022-06-29T15:48:24.971 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": "smithi099", 2022-06-29T15:48:24.972 INFO:teuthology.orchestra.run.smithi099.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-06-29T15:48:24.972 INFO:teuthology.orchestra.run.smithi099.stdout: "pid": "12842", 2022-06-29T15:48:24.972 INFO:teuthology.orchestra.run.smithi099.stdout: "root": "/" 2022-06-29T15:48:24.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:48:24.973 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_count": 0, 2022-06-29T15:48:24.973 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_pinned_count": 0, 2022-06-29T15:48:24.973 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 6343, 2022-06-29T15:48:24.973 INFO:teuthology.orchestra.run.smithi099.stdout: "inst": { 2022-06-29T15:48:24.974 INFO:teuthology.orchestra.run.smithi099.stdout: "name": { 2022-06-29T15:48:24.974 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "client", 2022-06-29T15:48:24.974 INFO:teuthology.orchestra.run.smithi099.stdout: "num": 6343 2022-06-29T15:48:24.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:48:24.975 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2022-06-29T15:48:24.975 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2022-06-29T15:48:24.975 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:0", 2022-06-29T15:48:24.975 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2790422795 2022-06-29T15:48:24.976 INFO:teuthology.orchestra.run.smithi099.stdout: } 2022-06-29T15:48:24.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:48:24.976 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2022-06-29T15:48:24.976 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2022-06-29T15:48:24.977 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:0", 2022-06-29T15:48:24.977 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2790422795 2022-06-29T15:48:24.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:48:24.977 INFO:teuthology.orchestra.run.smithi099.stdout: "inst_str": "client.6343 172.21.15.99:0/2790422795", 2022-06-29T15:48:24.978 INFO:teuthology.orchestra.run.smithi099.stdout: "addr_str": "172.21.15.99:0/2790422795", 2022-06-29T15:48:24.978 INFO:teuthology.orchestra.run.smithi099.stdout: "inode_count": 1, 2022-06-29T15:48:24.978 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_epoch": 35, 2022-06-29T15:48:24.978 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch": 78, 2022-06-29T15:48:24.979 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch_barrier": 0, 2022-06-29T15:48:24.979 INFO:teuthology.orchestra.run.smithi099.stdout: "blacklisted": false, 2022-06-29T15:48:24.979 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs" 2022-06-29T15:48:24.979 INFO:teuthology.orchestra.run.smithi099.stdout:} 2022-06-29T15:48:24.981 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:48:24.982 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:48:24.982 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:48:24.988 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2022-06-29T15:48:24.990 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-29T15:48:24.990 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:48:24.991 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:48:25.007 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:48:25.116 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.113+0000 7fdf3bfdd700 1 Processor -- start 2022-06-29T15:48:25.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.113+0000 7fdf3bfdd700 1 -- start start 2022-06-29T15:48:25.118 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.113+0000 7fdf3bfdd700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdf340b15c0 0x7fdf340b06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:25.118 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.113+0000 7fdf3bfdd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdf340b2a70 0x7fdf34147570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:25.118 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.113+0000 7fdf3bfdd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdf340b3440 0x7fdf34067100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:25.118 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.113+0000 7fdf3bfdd700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf340aee00 con 0x7fdf340b3440 2022-06-29T15:48:25.119 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.113+0000 7fdf3bfdd700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf340ad770 con 0x7fdf340b2a70 2022-06-29T15:48:25.119 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.113+0000 7fdf3bfdd700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf34147a60 con 0x7fdf340b15c0 2022-06-29T15:48:25.119 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.113+0000 7fdf3a57a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdf340b3440 0x7fdf34067100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:25.119 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.113+0000 7fdf39d79700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdf340b15c0 0x7fdf340b06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:25.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.113+0000 7fdf39d79700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdf340b15c0 0x7fdf340b06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:47250/0 (socket says 172.21.15.47:47250) 2022-06-29T15:48:25.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.113+0000 7fdf39578700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdf340b2a70 0x7fdf34147570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:25.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.113+0000 7fdf3a57a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdf340b3440 0x7fdf34067100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50152/0 (socket says 172.21.15.47:50152) 2022-06-29T15:48:25.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.113+0000 7fdf39d79700 1 -- 172.21.15.47:0/2591763747 learned_addr learned my addr 172.21.15.47:0/2591763747 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:25.122 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.113+0000 7fdf39d79700 1 -- 172.21.15.47:0/2591763747 >> v2:172.21.15.64:3300/0 conn(0x7fdf340b2a70 msgr2=0x7fdf34147570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:25.123 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.113+0000 7fdf39d79700 1 --2- 172.21.15.47:0/2591763747 >> v2:172.21.15.64:3300/0 conn(0x7fdf340b2a70 0x7fdf34147570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:25.123 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.113+0000 7fdf39d79700 1 -- 172.21.15.47:0/2591763747 >> v2:172.21.15.47:3300/0 conn(0x7fdf340b3440 msgr2=0x7fdf34067100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:25.123 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.113+0000 7fdf39d79700 1 --2- 172.21.15.47:0/2591763747 >> v2:172.21.15.47:3300/0 conn(0x7fdf340b3440 0x7fdf34067100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:25.123 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.113+0000 7fdf39d79700 1 -- 172.21.15.47:0/2591763747 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf340b2270 con 0x7fdf340b15c0 2022-06-29T15:48:25.123 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.113+0000 7fdf39d79700 1 --2- 172.21.15.47:0/2591763747 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdf340b15c0 0x7fdf340b06f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fdf30002f70 tx=0x7fdf30006eb0).ready entity=mon.2 client_cookie=2cf13542ebfb5fc4 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:25.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.113+0000 7fdf38d77700 1 -- 172.21.15.47:0/2591763747 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf3000d570 con 0x7fdf340b15c0 2022-06-29T15:48:25.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.113+0000 7fdf38d77700 1 -- 172.21.15.47:0/2591763747 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf30000e90 con 0x7fdf340b15c0 2022-06-29T15:48:25.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.113+0000 7fdf38d77700 1 -- 172.21.15.47:0/2591763747 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf3000d900 con 0x7fdf340b15c0 2022-06-29T15:48:25.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.117+0000 7fdf3bfdd700 1 -- 172.21.15.47:0/2591763747 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdf340b15c0 msgr2=0x7fdf340b06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:25.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.117+0000 7fdf3bfdd700 1 --2- 172.21.15.47:0/2591763747 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdf340b15c0 0x7fdf340b06f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fdf30002f70 tx=0x7fdf30006eb0).stop 2022-06-29T15:48:25.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.117+0000 7fdf3bfdd700 1 -- 172.21.15.47:0/2591763747 shutdown_connections 2022-06-29T15:48:25.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.117+0000 7fdf3bfdd700 1 --2- 172.21.15.47:0/2591763747 >> v2:172.21.15.47:3300/0 conn(0x7fdf340b3440 0x7fdf34067100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:25.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.117+0000 7fdf3bfdd700 1 --2- 172.21.15.47:0/2591763747 >> v2:172.21.15.64:3300/0 conn(0x7fdf340b2a70 0x7fdf34147570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:25.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.117+0000 7fdf3bfdd700 1 --2- 172.21.15.47:0/2591763747 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdf340b15c0 0x7fdf340b06f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:25.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.117+0000 7fdf3bfdd700 1 -- 172.21.15.47:0/2591763747 >> 172.21.15.47:0/2591763747 conn(0x7fdf340b98b0 msgr2=0x7fdf340b9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:25.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.117+0000 7fdf3bfdd700 1 -- 172.21.15.47:0/2591763747 shutdown_connections 2022-06-29T15:48:25.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.117+0000 7fdf3bfdd700 1 -- 172.21.15.47:0/2591763747 wait complete. 2022-06-29T15:48:25.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.117+0000 7fdf3bfdd700 1 Processor -- start 2022-06-29T15:48:25.127 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.117+0000 7fdf3bfdd700 1 -- start start 2022-06-29T15:48:25.130 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.117+0000 7fdf3bfdd700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdf340b15c0 0x7fdf340626a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:25.130 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.117+0000 7fdf3bfdd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf340b2a70 0x7fdf34159c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:25.130 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.117+0000 7fdf3bfdd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf340b3440 0x7fdf3415a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:25.130 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.117+0000 7fdf3bfdd700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3414bf70 con 0x7fdf340b2a70 2022-06-29T15:48:25.131 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.117+0000 7fdf3bfdd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf34062b90 con 0x7fdf340b3440 2022-06-29T15:48:25.131 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.117+0000 7fdf3bfdd700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3414cba0 con 0x7fdf340b15c0 2022-06-29T15:48:25.131 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.117+0000 7fdf39578700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf340b2a70 0x7fdf34159c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:25.131 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.117+0000 7fdf39578700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf340b2a70 0x7fdf34159c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50154/0 (socket says 172.21.15.47:50154) 2022-06-29T15:48:25.132 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.117+0000 7fdf39578700 1 -- 172.21.15.47:0/2277404280 learned_addr learned my addr 172.21.15.47:0/2277404280 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:25.132 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.117+0000 7fdf39d79700 1 --2- 172.21.15.47:0/2277404280 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdf340b15c0 0x7fdf340626a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:25.132 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.117+0000 7fdf3a57a700 1 --2- 172.21.15.47:0/2277404280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf340b3440 0x7fdf3415a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:25.132 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.121+0000 7fdf39578700 1 -- 172.21.15.47:0/2277404280 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdf340b15c0 msgr2=0x7fdf340626a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:25.132 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.121+0000 7fdf39578700 1 --2- 172.21.15.47:0/2277404280 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdf340b15c0 0x7fdf340626a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:25.133 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.121+0000 7fdf39578700 1 -- 172.21.15.47:0/2277404280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf340b3440 msgr2=0x7fdf3415a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:25.133 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.121+0000 7fdf39578700 1 --2- 172.21.15.47:0/2277404280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf340b3440 0x7fdf3415a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:25.133 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.121+0000 7fdf39578700 1 -- 172.21.15.47:0/2277404280 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf3000b1e0 con 0x7fdf340b2a70 2022-06-29T15:48:25.133 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.121+0000 7fdf39d79700 1 --2- 172.21.15.47:0/2277404280 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdf340b15c0 0x7fdf340626a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:48:25.134 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.121+0000 7fdf39578700 1 --2- 172.21.15.47:0/2277404280 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf340b2a70 0x7fdf34159c30 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 rx=0x7fdf2400c000 tx=0x7fdf24006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:25.134 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.121+0000 7fdf22ffd700 1 -- 172.21.15.47:0/2277404280 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf24010780 con 0x7fdf340b2a70 2022-06-29T15:48:25.134 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.121+0000 7fdf22ffd700 1 -- 172.21.15.47:0/2277404280 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf2400b3b0 con 0x7fdf340b2a70 2022-06-29T15:48:25.134 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.121+0000 7fdf3bfdd700 1 -- 172.21.15.47:0/2277404280 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf3414d8c0 con 0x7fdf340b2a70 2022-06-29T15:48:25.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.121+0000 7fdf22ffd700 1 -- 172.21.15.47:0/2277404280 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf24010ad0 con 0x7fdf340b2a70 2022-06-29T15:48:25.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.121+0000 7fdf3bfdd700 1 -- 172.21.15.47:0/2277404280 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf3414a780 con 0x7fdf340b2a70 2022-06-29T15:48:25.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.121+0000 7fdf3bfdd700 1 -- 172.21.15.47:0/2277404280 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf3414da50 con 0x7fdf340b2a70 2022-06-29T15:48:25.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.125+0000 7fdf22ffd700 1 -- 172.21.15.47:0/2277404280 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fdf24010c30 con 0x7fdf340b2a70 2022-06-29T15:48:25.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.125+0000 7fdf22ffd700 1 --2- 172.21.15.47:0/2277404280 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdf14053950 0x7fdf14055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:25.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.125+0000 7fdf22ffd700 1 -- 172.21.15.47:0/2277404280 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fdf2406c430 con 0x7fdf340b2a70 2022-06-29T15:48:25.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.125+0000 7fdf39d79700 1 --2- 172.21.15.47:0/2277404280 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdf14053950 0x7fdf14055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:25.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.125+0000 7fdf39d79700 1 --2- 172.21.15.47:0/2277404280 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdf14053950 0x7fdf14055e10 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7fdf3001a000 tx=0x7fdf30015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:25.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.133+0000 7fdf22ffd700 1 -- 172.21.15.47:0/2277404280 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdf240430f0 con 0x7fdf340b2a70 2022-06-29T15:48:25.266 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.261+0000 7fdf3bfdd700 1 -- 172.21.15.47:0/2277404280 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf340b2450 con 0x7fdf340b2a70 2022-06-29T15:48:25.268 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.265+0000 7fdf22ffd700 1 -- 172.21.15.47:0/2277404280 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+16274 (secure 0 0 0) 0x7fdf340b2450 con 0x7fdf340b2a70 2022-06-29T15:48:25.268 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:48:25.269 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":78,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:48:19.348792+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:48:16.753707+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2022-06-29T15:48:17.935585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:48:25.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.265+0000 7fdf3bfdd700 1 -- 172.21.15.47:0/2277404280 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdf14053950 msgr2=0x7fdf14055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:25.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.265+0000 7fdf3bfdd700 1 --2- 172.21.15.47:0/2277404280 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdf14053950 0x7fdf14055e10 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7fdf3001a000 tx=0x7fdf30015000).stop 2022-06-29T15:48:25.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.265+0000 7fdf3bfdd700 1 -- 172.21.15.47:0/2277404280 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf340b2a70 msgr2=0x7fdf34159c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:25.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.265+0000 7fdf3bfdd700 1 --2- 172.21.15.47:0/2277404280 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf340b2a70 0x7fdf34159c30 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 rx=0x7fdf2400c000 tx=0x7fdf24006df0).stop 2022-06-29T15:48:25.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.265+0000 7fdf3bfdd700 1 -- 172.21.15.47:0/2277404280 shutdown_connections 2022-06-29T15:48:25.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.265+0000 7fdf3bfdd700 1 --2- 172.21.15.47:0/2277404280 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdf14053950 0x7fdf14055e10 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:25.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.265+0000 7fdf3bfdd700 1 --2- 172.21.15.47:0/2277404280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf340b3440 0x7fdf3415a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:25.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.265+0000 7fdf3bfdd700 1 --2- 172.21.15.47:0/2277404280 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf340b2a70 0x7fdf34159c30 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:25.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.265+0000 7fdf3bfdd700 1 --2- 172.21.15.47:0/2277404280 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdf340b15c0 0x7fdf340626a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:25.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.265+0000 7fdf3bfdd700 1 -- 172.21.15.47:0/2277404280 >> 172.21.15.47:0/2277404280 conn(0x7fdf340b98b0 msgr2=0x7fdf34146820 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:25.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.265+0000 7fdf3bfdd700 1 -- 172.21.15.47:0/2277404280 shutdown_connections 2022-06-29T15:48:25.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.265+0000 7fdf3bfdd700 1 -- 172.21.15.47:0/2277404280 wait complete. 2022-06-29T15:48:25.282 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-29T15:42:16.971431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-29T15:42:25.290884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-29T15:48:16.753707+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2022-06-29T15:48:17.935585+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-29T15:48:25.283 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-29T15:48:25.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.385+0000 7f606759e700 1 Processor -- start 2022-06-29T15:48:25.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.385+0000 7f606759e700 1 -- start start 2022-06-29T15:48:25.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.385+0000 7f606759e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f60680b64c0 0x7f60680b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:25.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.385+0000 7f606759e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6068066fe0 0x7f60680b4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:25.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.385+0000 7f606759e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60680b68a0 con 0x7f6068143150 2022-06-29T15:48:25.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.385+0000 7f606759e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60680b13a0 con 0x7f6068066fe0 2022-06-29T15:48:25.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.385+0000 7f606759e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60681473d0 con 0x7f60680b64c0 2022-06-29T15:48:25.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.385+0000 7f6066d9d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f6068143150 0x7f606812bec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35716/0 (socket says 172.21.15.47:35716) 2022-06-29T15:48:25.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.385+0000 7f6066d9d700 1 -- 172.21.15.47:0/1709245688 learned_addr learned my addr 172.21.15.47:0/1709245688 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:48:25.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.385+0000 7f606659c700 1 --2- 172.21.15.47:0/1709245688 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f60680b64c0 0x7f60680b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:25.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.385+0000 7f6065d9b700 1 --2- 172.21.15.47:0/1709245688 >> v2:172.21.15.64:3300/0 conn(0x7f6068066fe0 0x7f60680b4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:25.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.385+0000 7f606559a700 1 -- 172.21.15.47:0/1709245688 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3183825525 0 0) 0x7f60680b68a0 con 0x7f6068143150 2022-06-29T15:48:25.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.385+0000 7f606559a700 1 -- 172.21.15.47:0/1709245688 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f604c003260 con 0x7f6068143150 2022-06-29T15:48:25.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.385+0000 7f606559a700 1 -- 172.21.15.47:0/1709245688 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2993782592 0 0) 0x7f604c003260 con 0x7f6068143150 2022-06-29T15:48:25.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.385+0000 7f606559a700 1 -- 172.21.15.47:0/1709245688 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f60680b64c0 msgr2=0x7f60680b5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:25.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.385+0000 7f606559a700 1 --2- 172.21.15.47:0/1709245688 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f60680b64c0 0x7f60680b5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:25.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.385+0000 7f606559a700 1 -- 172.21.15.47:0/1709245688 >> v2:172.21.15.64:3300/0 conn(0x7f6068066fe0 msgr2=0x7f60680b4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:25.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.385+0000 7f606559a700 1 --2- 172.21.15.47:0/1709245688 >> v2:172.21.15.64:3300/0 conn(0x7f6068066fe0 0x7f60680b4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:25.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.385+0000 7f606559a700 1 -- 172.21.15.47:0/1709245688 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60680b46a0 con 0x7f6068143150 2022-06-29T15:48:25.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.385+0000 7f606559a700 1 -- 172.21.15.47:0/1709245688 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7f60600019e0 con 0x7f6068143150 2022-06-29T15:48:25.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.385+0000 7f606559a700 1 -- 172.21.15.47:0/1709245688 >> v1:172.21.15.47:6789/0 conn(0x7f6068143150 legacy=0x7f606812bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:25.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.385+0000 7f606559a700 1 --2- 172.21.15.47:0/1709245688 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f60680b64c0 0x7f604c003750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:25.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.385+0000 7f606559a700 1 --2- 172.21.15.47:0/1709245688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f604c003c80 0x7f604c004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:25.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.385+0000 7f606559a700 1 --2- 172.21.15.47:0/1709245688 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f604c0049f0 0x7f604c004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:25.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.385+0000 7f606559a700 1 -- 172.21.15.47:0/1709245688 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60681473d0 con 0x7f604c0049f0 2022-06-29T15:48:25.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.385+0000 7f606559a700 1 -- 172.21.15.47:0/1709245688 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f604c00b750 con 0x7f604c003c80 2022-06-29T15:48:25.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.385+0000 7f606559a700 1 -- 172.21.15.47:0/1709245688 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f604c00b890 con 0x7f60680b64c0 2022-06-29T15:48:25.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.385+0000 7f6066d9d700 1 --2- 172.21.15.47:0/1709245688 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f604c0049f0 0x7f604c004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:25.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.385+0000 7f606659c700 1 --2- 172.21.15.47:0/1709245688 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f60680b64c0 0x7f604c003750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:25.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.385+0000 7f6065d9b700 1 --2- 172.21.15.47:0/1709245688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f604c003c80 0x7f604c004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:25.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.385+0000 7f6066d9d700 1 -- 172.21.15.47:0/1709245688 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f60680b64c0 msgr2=0x7f604c003750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:25.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.385+0000 7f6066d9d700 1 --2- 172.21.15.47:0/1709245688 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f60680b64c0 0x7f604c003750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:25.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.385+0000 7f6066d9d700 1 -- 172.21.15.47:0/1709245688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f604c003c80 msgr2=0x7f604c004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:25.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.385+0000 7f6066d9d700 1 --2- 172.21.15.47:0/1709245688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f604c003c80 0x7f604c004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:25.409 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.385+0000 7f6066d9d700 1 -- 172.21.15.47:0/1709245688 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f604c003260 con 0x7f604c0049f0 2022-06-29T15:48:25.409 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.385+0000 7f6066d9d700 1 --2- 172.21.15.47:0/1709245688 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f604c0049f0 0x7f604c004e70 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 rx=0x7f604c0054d0 tx=0x7f60600021d0).ready entity=mon.0 client_cookie=42d5fc3451eace2f server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:25.409 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.389+0000 7f606559a700 1 -- 172.21.15.47:0/1709245688 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f606000cc40 con 0x7f604c0049f0 2022-06-29T15:48:25.409 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.389+0000 7f606559a700 1 -- 172.21.15.47:0/1709245688 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f606000cda0 con 0x7f604c0049f0 2022-06-29T15:48:25.410 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.389+0000 7f606559a700 1 -- 172.21.15.47:0/1709245688 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6060005b80 con 0x7f604c0049f0 2022-06-29T15:48:25.410 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.389+0000 7f606759e700 1 -- 172.21.15.47:0/1709245688 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f604c0049f0 msgr2=0x7f604c004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:25.410 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.389+0000 7f606759e700 1 --2- 172.21.15.47:0/1709245688 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f604c0049f0 0x7f604c004e70 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 rx=0x7f604c0054d0 tx=0x7f60600021d0).stop 2022-06-29T15:48:25.410 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.389+0000 7f6066d9d700 1 -- 172.21.15.47:0/1709245688 reap_dead start 2022-06-29T15:48:25.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.389+0000 7f606759e700 1 -- 172.21.15.47:0/1709245688 shutdown_connections 2022-06-29T15:48:25.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.389+0000 7f606759e700 1 -- 172.21.15.47:0/1709245688 >> 172.21.15.47:0/1709245688 conn(0x7f6068060e00 msgr2=0x7f60680611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:25.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.389+0000 7f606759e700 1 -- 172.21.15.47:0/1709245688 shutdown_connections 2022-06-29T15:48:25.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.389+0000 7f606759e700 1 -- 172.21.15.47:0/1709245688 wait complete. 2022-06-29T15:48:25.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.389+0000 7f606759e700 1 Processor -- start 2022-06-29T15:48:25.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.389+0000 7f606759e700 1 -- start start 2022-06-29T15:48:25.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.389+0000 7f606759e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f604c0049f0 0x7f6068145a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:25.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.389+0000 7f606759e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60680b79d0 0x7f6068149850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:25.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.389+0000 7f606759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6068149d40 0x7f6068126f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:25.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.389+0000 7f606759e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6068127f40 con 0x7f60680b79d0 2022-06-29T15:48:25.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.389+0000 7f606759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60680b7e50 con 0x7f6068149d40 2022-06-29T15:48:25.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.389+0000 7f606759e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6068149310 con 0x7f604c0049f0 2022-06-29T15:48:25.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.389+0000 7f6065d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60680b79d0 0x7f6068149850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:25.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.389+0000 7f6065d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60680b79d0 0x7f6068149850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50174/0 (socket says 172.21.15.47:50174) 2022-06-29T15:48:25.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.389+0000 7f6065d9b700 1 -- 172.21.15.47:0/3134289726 learned_addr learned my addr 172.21.15.47:0/3134289726 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:25.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.389+0000 7f606659c700 1 --2- 172.21.15.47:0/3134289726 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f604c0049f0 0x7f6068145a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:25.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.389+0000 7f6066d9d700 1 --2- 172.21.15.47:0/3134289726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6068149d40 0x7f6068126f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:25.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.389+0000 7f6065d9b700 1 -- 172.21.15.47:0/3134289726 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f604c0049f0 msgr2=0x7f6068145a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:25.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.389+0000 7f6065d9b700 1 --2- 172.21.15.47:0/3134289726 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f604c0049f0 0x7f6068145a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:25.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.389+0000 7f6065d9b700 1 -- 172.21.15.47:0/3134289726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6068149d40 msgr2=0x7f6068126f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:25.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.389+0000 7f6065d9b700 1 --2- 172.21.15.47:0/3134289726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6068149d40 0x7f6068126f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:25.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.389+0000 7f6065d9b700 1 -- 172.21.15.47:0/3134289726 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60681495b0 con 0x7f60680b79d0 2022-06-29T15:48:25.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.393+0000 7f6065d9b700 1 --2- 172.21.15.47:0/3134289726 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60680b79d0 0x7f6068149850 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 rx=0x7f6050006000 tx=0x7f6050006d90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:25.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.393+0000 7f605f7fe700 1 -- 172.21.15.47:0/3134289726 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6050014650 con 0x7f60680b79d0 2022-06-29T15:48:25.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.393+0000 7f605f7fe700 1 -- 172.21.15.47:0/3134289726 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60500147b0 con 0x7f60680b79d0 2022-06-29T15:48:25.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.393+0000 7f606759e700 1 -- 172.21.15.47:0/3134289726 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60680b9ca0 con 0x7f60680b79d0 2022-06-29T15:48:25.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.393+0000 7f605f7fe700 1 -- 172.21.15.47:0/3134289726 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6050014920 con 0x7f60680b79d0 2022-06-29T15:48:25.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.393+0000 7f606759e700 1 -- 172.21.15.47:0/3134289726 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6068127bc0 con 0x7f60680b79d0 2022-06-29T15:48:25.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.393+0000 7f606759e700 1 -- 172.21.15.47:0/3134289726 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60680573d0 con 0x7f60680b79d0 2022-06-29T15:48:25.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.393+0000 7f605f7fe700 1 -- 172.21.15.47:0/3134289726 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6050024070 con 0x7f60680b79d0 2022-06-29T15:48:25.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.393+0000 7f605f7fe700 1 --2- 172.21.15.47:0/3134289726 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6040053b30 0x7f6040055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:25.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.393+0000 7f605f7fe700 1 -- 172.21.15.47:0/3134289726 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f605006c6f0 con 0x7f60680b79d0 2022-06-29T15:48:25.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.393+0000 7f606659c700 1 --2- 172.21.15.47:0/3134289726 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6040053b30 0x7f6040055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:25.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.393+0000 7f606659c700 1 --2- 172.21.15.47:0/3134289726 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6040053b30 0x7f6040055ff0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f605800b000 tx=0x7f6058008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:25.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.401+0000 7f605f7fe700 1 -- 172.21.15.47:0/3134289726 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6050043450 con 0x7f60680b79d0 2022-06-29T15:48:25.533 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.529+0000 7f606759e700 1 -- 172.21.15.47:0/3134289726 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f606806a0f0 con 0x7f60680b79d0 2022-06-29T15:48:25.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.529+0000 7f605f7fe700 1 -- 172.21.15.47:0/3134289726 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v78) v1 ==== 110+0+10 (secure 0 0 0) 0x7f6050046a70 con 0x7f60680b79d0 2022-06-29T15:48:25.535 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2022-06-29T15:48:25.537 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.533+0000 7f606759e700 1 -- 172.21.15.47:0/3134289726 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6040053b30 msgr2=0x7f6040055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:25.538 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.533+0000 7f606759e700 1 --2- 172.21.15.47:0/3134289726 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6040053b30 0x7f6040055ff0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f605800b000 tx=0x7f6058008040).stop 2022-06-29T15:48:25.538 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.533+0000 7f606759e700 1 -- 172.21.15.47:0/3134289726 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60680b79d0 msgr2=0x7f6068149850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:25.538 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.533+0000 7f606759e700 1 --2- 172.21.15.47:0/3134289726 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60680b79d0 0x7f6068149850 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 rx=0x7f6050006000 tx=0x7f6050006d90).stop 2022-06-29T15:48:25.538 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.533+0000 7f606759e700 1 -- 172.21.15.47:0/3134289726 shutdown_connections 2022-06-29T15:48:25.539 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.533+0000 7f606759e700 1 --2- 172.21.15.47:0/3134289726 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6040053b30 0x7f6040055ff0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:25.539 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.533+0000 7f606759e700 1 --2- 172.21.15.47:0/3134289726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6068149d40 0x7f6068126f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:25.539 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.533+0000 7f606759e700 1 --2- 172.21.15.47:0/3134289726 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60680b79d0 0x7f6068149850 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:25.539 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.533+0000 7f606759e700 1 --2- 172.21.15.47:0/3134289726 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f604c0049f0 0x7f6068145a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:25.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.533+0000 7f606759e700 1 -- 172.21.15.47:0/3134289726 >> 172.21.15.47:0/3134289726 conn(0x7f6068060e00 msgr2=0x7f60680b59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:25.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.533+0000 7f606759e700 1 -- 172.21.15.47:0/3134289726 shutdown_connections 2022-06-29T15:48:25.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.533+0000 7f606759e700 1 -- 172.21.15.47:0/3134289726 wait complete. 2022-06-29T15:48:25.552 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-29T15:48:25.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.657+0000 7fbee3b06700 1 Processor -- start 2022-06-29T15:48:25.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.657+0000 7fbee3b06700 1 -- start start 2022-06-29T15:48:25.664 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.657+0000 7fbee3b06700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbedc0b4300 0x7fbedc0612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:25.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.657+0000 7fbee3b06700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbedc067610 con 0x7fbedc060ee0 2022-06-29T15:48:25.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.657+0000 7fbee3b06700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbedc1467c0 con 0x7fbedc1471a0 2022-06-29T15:48:25.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.657+0000 7fbee3b06700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbedc146a60 con 0x7fbedc0b4300 2022-06-29T15:48:25.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.657+0000 7fbee18a2700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fbedc060ee0 0x7fbedc1430e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35734/0 (socket says 172.21.15.47:35734) 2022-06-29T15:48:25.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.657+0000 7fbee18a2700 1 -- 172.21.15.47:0/1215301548 learned_addr learned my addr 172.21.15.47:0/1215301548 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:48:25.666 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.657+0000 7fbee10a1700 1 --2- 172.21.15.47:0/1215301548 >> v2:172.21.15.64:3300/0 conn(0x7fbedc0b4300 0x7fbedc0612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:25.666 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.661+0000 7fbee08a0700 1 -- 172.21.15.47:0/1215301548 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 234661560 0 0) 0x7fbedc067610 con 0x7fbedc060ee0 2022-06-29T15:48:25.666 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.661+0000 7fbee08a0700 1 -- 172.21.15.47:0/1215301548 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbec4003260 con 0x7fbedc060ee0 2022-06-29T15:48:25.666 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.661+0000 7fbee08a0700 1 -- 172.21.15.47:0/1215301548 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2087695840 0 0) 0x7fbedc1467c0 con 0x7fbedc1471a0 2022-06-29T15:48:25.667 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.661+0000 7fbee08a0700 1 -- 172.21.15.47:0/1215301548 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbedc067610 con 0x7fbedc1471a0 2022-06-29T15:48:25.667 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.661+0000 7fbee10a1700 1 -- 172.21.15.47:0/1215301548 >> v1:172.21.15.64:6789/0 conn(0x7fbedc1471a0 legacy=0x7fbedc13cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:25.667 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.661+0000 7fbee10a1700 1 -- 172.21.15.47:0/1215301548 >> v1:172.21.15.47:6789/0 conn(0x7fbedc060ee0 legacy=0x7fbedc1430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:25.667 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.661+0000 7fbee10a1700 1 -- 172.21.15.47:0/1215301548 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbedc146d00 con 0x7fbedc0b4300 2022-06-29T15:48:25.668 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.661+0000 7fbee10a1700 1 --2- 172.21.15.47:0/1215301548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbedc0b4300 0x7fbedc0612c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7fbedc0adb80 tx=0x7fbed8006df0).ready entity=mon.1 client_cookie=1520c8896218ab92 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:25.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.661+0000 7fbee08a0700 1 -- 172.21.15.47:0/1215301548 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbed800d540 con 0x7fbedc0b4300 2022-06-29T15:48:25.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.661+0000 7fbee08a0700 1 -- 172.21.15.47:0/1215301548 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbed8000e90 con 0x7fbedc0b4300 2022-06-29T15:48:25.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.661+0000 7fbee08a0700 1 -- 172.21.15.47:0/1215301548 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbed800d890 con 0x7fbedc0b4300 2022-06-29T15:48:25.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.661+0000 7fbee3b06700 1 -- 172.21.15.47:0/1215301548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbedc0b4300 msgr2=0x7fbedc0612c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:25.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.661+0000 7fbee3b06700 1 --2- 172.21.15.47:0/1215301548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbedc0b4300 0x7fbedc0612c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7fbedc0adb80 tx=0x7fbed8006df0).stop 2022-06-29T15:48:25.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.661+0000 7fbee3b06700 1 -- 172.21.15.47:0/1215301548 shutdown_connections 2022-06-29T15:48:25.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.661+0000 7fbee3b06700 1 --2- 172.21.15.47:0/1215301548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbedc0b4300 0x7fbedc0612c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:25.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.661+0000 7fbee3b06700 1 -- 172.21.15.47:0/1215301548 >> 172.21.15.47:0/1215301548 conn(0x7fbedc05c510 msgr2=0x7fbedc05f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:25.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.661+0000 7fbee3b06700 1 -- 172.21.15.47:0/1215301548 shutdown_connections 2022-06-29T15:48:25.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.661+0000 7fbee3b06700 1 -- 172.21.15.47:0/1215301548 wait complete. 2022-06-29T15:48:25.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.665+0000 7fbee3b06700 1 Processor -- start 2022-06-29T15:48:25.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.665+0000 7fbee3b06700 1 -- start start 2022-06-29T15:48:25.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.665+0000 7fbee3b06700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbedc060ee0 0x7fbedc13f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:25.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.665+0000 7fbee3b06700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbedc0b4300 0x7fbedc1446d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:25.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.665+0000 7fbee3b06700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbedc1471a0 0x7fbedc13db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:25.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.665+0000 7fbee3b06700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbedc0b8bb0 con 0x7fbedc1471a0 2022-06-29T15:48:25.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.665+0000 7fbee3b06700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbedc0b8e50 con 0x7fbedc0b4300 2022-06-29T15:48:25.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.665+0000 7fbee3b06700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbedc0b90f0 con 0x7fbedc060ee0 2022-06-29T15:48:25.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.665+0000 7fbee20a3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbedc1471a0 0x7fbedc13db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:25.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.665+0000 7fbee20a3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbedc1471a0 0x7fbedc13db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50190/0 (socket says 172.21.15.47:50190) 2022-06-29T15:48:25.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.665+0000 7fbee20a3700 1 -- 172.21.15.47:0/2543929362 learned_addr learned my addr 172.21.15.47:0/2543929362 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:25.676 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.665+0000 7fbee10a1700 1 --2- 172.21.15.47:0/2543929362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbedc0b4300 0x7fbedc1446d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:25.676 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.665+0000 7fbee18a2700 1 --2- 172.21.15.47:0/2543929362 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbedc060ee0 0x7fbedc13f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:25.676 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.665+0000 7fbee20a3700 1 -- 172.21.15.47:0/2543929362 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbedc060ee0 msgr2=0x7fbedc13f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:25.676 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.665+0000 7fbee20a3700 1 --2- 172.21.15.47:0/2543929362 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbedc060ee0 0x7fbedc13f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:25.676 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.665+0000 7fbee20a3700 1 -- 172.21.15.47:0/2543929362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbedc0b4300 msgr2=0x7fbedc1446d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:25.677 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.665+0000 7fbee20a3700 1 --2- 172.21.15.47:0/2543929362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbedc0b4300 0x7fbedc1446d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:25.677 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.665+0000 7fbee20a3700 1 -- 172.21.15.47:0/2543929362 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed800b180 con 0x7fbedc1471a0 2022-06-29T15:48:25.677 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.665+0000 7fbee10a1700 1 --2- 172.21.15.47:0/2543929362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbedc0b4300 0x7fbedc1446d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:48:25.677 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.665+0000 7fbee20a3700 1 --2- 172.21.15.47:0/2543929362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbedc1471a0 0x7fbedc13db70 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 rx=0x7fbed4001490 tx=0x7fbed4007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:25.677 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.665+0000 7fbecaffd700 1 -- 172.21.15.47:0/2543929362 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbed4002240 con 0x7fbedc1471a0 2022-06-29T15:48:25.678 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.665+0000 7fbecaffd700 1 -- 172.21.15.47:0/2543929362 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbed40023a0 con 0x7fbedc1471a0 2022-06-29T15:48:25.678 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.665+0000 7fbee3b06700 1 -- 172.21.15.47:0/2543929362 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbedc144f50 con 0x7fbedc1471a0 2022-06-29T15:48:25.678 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.665+0000 7fbecaffd700 1 -- 172.21.15.47:0/2543929362 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbed40017a0 con 0x7fbedc1471a0 2022-06-29T15:48:25.678 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.665+0000 7fbee3b06700 1 -- 172.21.15.47:0/2543929362 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbedc140620 con 0x7fbedc1471a0 2022-06-29T15:48:25.680 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.669+0000 7fbee3b06700 1 -- 172.21.15.47:0/2543929362 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbedc0b92b0 con 0x7fbedc1471a0 2022-06-29T15:48:25.680 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.669+0000 7fbecaffd700 1 -- 172.21.15.47:0/2543929362 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fbed4010030 con 0x7fbedc1471a0 2022-06-29T15:48:25.680 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.669+0000 7fbecaffd700 1 --2- 172.21.15.47:0/2543929362 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbebc0539e0 0x7fbebc055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:25.680 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.669+0000 7fbee18a2700 1 --2- 172.21.15.47:0/2543929362 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbebc0539e0 0x7fbebc055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:25.681 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.669+0000 7fbecaffd700 1 -- 172.21.15.47:0/2543929362 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fbed406bf30 con 0x7fbedc1471a0 2022-06-29T15:48:25.681 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.669+0000 7fbee18a2700 1 --2- 172.21.15.47:0/2543929362 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbebc0539e0 0x7fbebc055ea0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7fbecc00b000 tx=0x7fbecc009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:25.681 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.677+0000 7fbecaffd700 1 -- 172.21.15.47:0/2543929362 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbed4042be0 con 0x7fbedc1471a0 2022-06-29T15:48:25.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.801+0000 7fbee3b06700 1 -- 172.21.15.47:0/2543929362 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbedc140620 con 0x7fbedc1471a0 2022-06-29T15:48:25.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.805+0000 7fbecaffd700 1 -- 172.21.15.47:0/2543929362 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v78) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbedc140620 con 0x7fbedc1471a0 2022-06-29T15:48:25.808 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:48:25.810 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.805+0000 7fbee3b06700 1 -- 172.21.15.47:0/2543929362 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbebc0539e0 msgr2=0x7fbebc055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:25.811 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.805+0000 7fbee3b06700 1 --2- 172.21.15.47:0/2543929362 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbebc0539e0 0x7fbebc055ea0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7fbecc00b000 tx=0x7fbecc009040).stop 2022-06-29T15:48:25.811 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.805+0000 7fbee3b06700 1 -- 172.21.15.47:0/2543929362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbedc1471a0 msgr2=0x7fbedc13db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:25.811 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.805+0000 7fbee3b06700 1 --2- 172.21.15.47:0/2543929362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbedc1471a0 0x7fbedc13db70 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 rx=0x7fbed4001490 tx=0x7fbed4007dd0).stop 2022-06-29T15:48:25.811 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.805+0000 7fbee3b06700 1 -- 172.21.15.47:0/2543929362 shutdown_connections 2022-06-29T15:48:25.811 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.805+0000 7fbee3b06700 1 --2- 172.21.15.47:0/2543929362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbedc1471a0 0x7fbedc13db70 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:25.812 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.805+0000 7fbee3b06700 1 --2- 172.21.15.47:0/2543929362 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbebc0539e0 0x7fbebc055ea0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:25.812 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.805+0000 7fbee3b06700 1 --2- 172.21.15.47:0/2543929362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbedc0b4300 0x7fbedc1446d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:25.812 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.805+0000 7fbee3b06700 1 --2- 172.21.15.47:0/2543929362 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbedc060ee0 0x7fbedc13f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:25.812 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.805+0000 7fbee3b06700 1 -- 172.21.15.47:0/2543929362 >> 172.21.15.47:0/2543929362 conn(0x7fbedc05c510 msgr2=0x7fbedc13c6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:25.813 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.805+0000 7fbee3b06700 1 -- 172.21.15.47:0/2543929362 shutdown_connections 2022-06-29T15:48:25.813 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.805+0000 7fbee3b06700 1 -- 172.21.15.47:0/2543929362 wait complete. 2022-06-29T15:48:25.821 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-29T15:48:25.939 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.933+0000 7f6b2507a700 1 Processor -- start 2022-06-29T15:48:25.940 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.933+0000 7f6b2507a700 1 -- start start 2022-06-29T15:48:25.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.933+0000 7f6b2507a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6b200630f0 0x7f6b200b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:25.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.933+0000 7f6b2507a700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6b200b03f0 0x7f6b200b3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:25.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.937+0000 7f6b2507a700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b200b7200 con 0x7f6b200b6e20 2022-06-29T15:48:25.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.937+0000 7f6b2507a700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b2005f5c0 con 0x7f6b200630f0 2022-06-29T15:48:25.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.937+0000 7f6b2507a700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2005f860 con 0x7f6b200b03f0 2022-06-29T15:48:25.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.937+0000 7f6b1f7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6b200630f0 0x7f6b200b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:25.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.937+0000 7f6b1ffff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f6b200b6e20 0x7f6b201447c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35748/0 (socket says 172.21.15.47:35748) 2022-06-29T15:48:25.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.937+0000 7f6b1f7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6b200630f0 0x7f6b200b2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50194/0 (socket says 172.21.15.47:50194) 2022-06-29T15:48:25.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.937+0000 7f6b1ffff700 1 -- 172.21.15.47:0/4216879683 learned_addr learned my addr 172.21.15.47:0/4216879683 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:48:25.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.937+0000 7f6b24879700 1 --2- 172.21.15.47:0/4216879683 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6b200b03f0 0x7f6b200b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:25.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.937+0000 7f6b1f7fe700 1 -- 172.21.15.47:0/4216879683 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6b200b03f0 msgr2=0x7f6b200b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:25.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.937+0000 7f6b1f7fe700 1 --2- 172.21.15.47:0/4216879683 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6b200b03f0 0x7f6b200b3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:25.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.937+0000 7f6b1f7fe700 1 -- 172.21.15.47:0/4216879683 >> v1:172.21.15.47:6789/0 conn(0x7f6b200b6e20 legacy=0x7f6b201447c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:25.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.937+0000 7f6b1f7fe700 1 -- 172.21.15.47:0/4216879683 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b200afcb0 con 0x7f6b200630f0 2022-06-29T15:48:25.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.937+0000 7f6b1f7fe700 1 --2- 172.21.15.47:0/4216879683 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b200630f0 0x7f6b200b2e60 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 rx=0x7f6b08002f70 tx=0x7f6b08006df0).ready entity=mon.0 client_cookie=e72fa07332e2f984 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:25.950 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.937+0000 7f6b1effd700 1 -- 172.21.15.47:0/4216879683 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b0800d570 con 0x7f6b200630f0 2022-06-29T15:48:25.951 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.937+0000 7f6b1effd700 1 -- 172.21.15.47:0/4216879683 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b08000e90 con 0x7f6b200630f0 2022-06-29T15:48:25.951 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.937+0000 7f6b1effd700 1 -- 172.21.15.47:0/4216879683 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b0800d8c0 con 0x7f6b200630f0 2022-06-29T15:48:25.951 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.937+0000 7f6b2507a700 1 -- 172.21.15.47:0/4216879683 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b200630f0 msgr2=0x7f6b200b2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:25.951 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.937+0000 7f6b2507a700 1 --2- 172.21.15.47:0/4216879683 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b200630f0 0x7f6b200b2e60 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 rx=0x7f6b08002f70 tx=0x7f6b08006df0).stop 2022-06-29T15:48:25.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.937+0000 7f6b2507a700 1 -- 172.21.15.47:0/4216879683 shutdown_connections 2022-06-29T15:48:25.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.937+0000 7f6b2507a700 1 --2- 172.21.15.47:0/4216879683 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6b200b03f0 0x7f6b200b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:25.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.937+0000 7f6b2507a700 1 --2- 172.21.15.47:0/4216879683 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b200630f0 0x7f6b200b2e60 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:25.953 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.937+0000 7f6b2507a700 1 -- 172.21.15.47:0/4216879683 >> 172.21.15.47:0/4216879683 conn(0x7f6b20063670 msgr2=0x7f6b20063a50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:25.953 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.937+0000 7f6b2507a700 1 -- 172.21.15.47:0/4216879683 shutdown_connections 2022-06-29T15:48:25.953 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.937+0000 7f6b2507a700 1 -- 172.21.15.47:0/4216879683 wait complete. 2022-06-29T15:48:25.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.941+0000 7f6b2507a700 1 Processor -- start 2022-06-29T15:48:25.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.941+0000 7f6b2507a700 1 -- start start 2022-06-29T15:48:25.958 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.941+0000 7f6b2507a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b200630f0 0x7f6b201449d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:25.958 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.941+0000 7f6b2507a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b200b03f0 0x7f6b2013c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:25.958 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.941+0000 7f6b2507a700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6b200b6e20 0x7f6b2013cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:25.959 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.941+0000 7f6b2507a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b20143380 con 0x7f6b200630f0 2022-06-29T15:48:25.959 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.941+0000 7f6b2507a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b200648c0 con 0x7f6b200b03f0 2022-06-29T15:48:25.959 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.941+0000 7f6b2507a700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b20152530 con 0x7f6b200b6e20 2022-06-29T15:48:25.960 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.941+0000 7f6b1ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b200630f0 0x7f6b201449d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:25.960 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.941+0000 7f6b1ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b200630f0 0x7f6b201449d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50202/0 (socket says 172.21.15.47:50202) 2022-06-29T15:48:25.960 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.941+0000 7f6b1ffff700 1 -- 172.21.15.47:0/1043810215 learned_addr learned my addr 172.21.15.47:0/1043810215 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:25.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.941+0000 7f6b24879700 1 --2- 172.21.15.47:0/1043810215 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6b200b6e20 0x7f6b2013cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:25.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.941+0000 7f6b1f7fe700 1 --2- 172.21.15.47:0/1043810215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b200b03f0 0x7f6b2013c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:25.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.941+0000 7f6b1ffff700 1 -- 172.21.15.47:0/1043810215 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6b200b6e20 msgr2=0x7f6b2013cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:25.962 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.941+0000 7f6b1ffff700 1 --2- 172.21.15.47:0/1043810215 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6b200b6e20 0x7f6b2013cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:25.962 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.941+0000 7f6b1ffff700 1 -- 172.21.15.47:0/1043810215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b200b03f0 msgr2=0x7f6b2013c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:25.962 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.941+0000 7f6b1ffff700 1 --2- 172.21.15.47:0/1043810215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b200b03f0 0x7f6b2013c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:25.963 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.941+0000 7f6b1ffff700 1 -- 172.21.15.47:0/1043810215 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b0800b180 con 0x7f6b200630f0 2022-06-29T15:48:25.963 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.941+0000 7f6b1f7fe700 1 --2- 172.21.15.47:0/1043810215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b200b03f0 0x7f6b2013c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:48:25.963 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.941+0000 7f6b1ffff700 1 --2- 172.21.15.47:0/1043810215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b200630f0 0x7f6b201449d0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 rx=0x7f6b201442d0 tx=0x7f6b10007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:25.963 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.941+0000 7f6b1d7fa700 1 -- 172.21.15.47:0/1043810215 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b100070d0 con 0x7f6b200630f0 2022-06-29T15:48:25.964 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.941+0000 7f6b1d7fa700 1 -- 172.21.15.47:0/1043810215 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b10007230 con 0x7f6b200630f0 2022-06-29T15:48:25.964 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.941+0000 7f6b1d7fa700 1 -- 172.21.15.47:0/1043810215 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b10005610 con 0x7f6b200630f0 2022-06-29T15:48:25.964 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.941+0000 7f6b2507a700 1 -- 172.21.15.47:0/1043810215 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b2013d130 con 0x7f6b200630f0 2022-06-29T15:48:25.964 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.941+0000 7f6b2507a700 1 -- 172.21.15.47:0/1043810215 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b20064d50 con 0x7f6b200630f0 2022-06-29T15:48:25.965 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.945+0000 7f6b2507a700 1 -- 172.21.15.47:0/1043810215 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b20152680 con 0x7f6b200630f0 2022-06-29T15:48:25.965 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.945+0000 7f6b1d7fa700 1 -- 172.21.15.47:0/1043810215 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6b10010070 con 0x7f6b200630f0 2022-06-29T15:48:25.965 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.945+0000 7f6b1d7fa700 1 --2- 172.21.15.47:0/1043810215 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6b04053a70 0x7f6b04055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:25.966 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.945+0000 7f6b1d7fa700 1 -- 172.21.15.47:0/1043810215 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f6b1006b5c0 con 0x7f6b200630f0 2022-06-29T15:48:25.966 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.945+0000 7f6b1f7fe700 1 --2- 172.21.15.47:0/1043810215 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6b04053a70 0x7f6b04055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:25.966 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.949+0000 7f6b1f7fe700 1 --2- 172.21.15.47:0/1043810215 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6b04053a70 0x7f6b04055f30 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7f6b08018000 tx=0x7f6b08014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:25.967 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:25.953+0000 7f6b1d7fa700 1 -- 172.21.15.47:0/1043810215 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6b10042380 con 0x7f6b200630f0 2022-06-29T15:48:26.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.081+0000 7f6b2507a700 1 -- 172.21.15.47:0/1043810215 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6b200573d0 con 0x7f6b200630f0 2022-06-29T15:48:26.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.085+0000 7f6b1d7fa700 1 -- 172.21.15.47:0/1043810215 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v78) v1 ==== 104+0+10 (secure 0 0 0) 0x7f6b100180a0 con 0x7f6b200630f0 2022-06-29T15:48:26.089 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:48:26.091 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.085+0000 7f6b2507a700 1 -- 172.21.15.47:0/1043810215 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6b04053a70 msgr2=0x7f6b04055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:26.091 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.085+0000 7f6b2507a700 1 --2- 172.21.15.47:0/1043810215 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6b04053a70 0x7f6b04055f30 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7f6b08018000 tx=0x7f6b08014000).stop 2022-06-29T15:48:26.091 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.085+0000 7f6b2507a700 1 -- 172.21.15.47:0/1043810215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b200630f0 msgr2=0x7f6b201449d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:26.092 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.085+0000 7f6b2507a700 1 --2- 172.21.15.47:0/1043810215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b200630f0 0x7f6b201449d0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 rx=0x7f6b201442d0 tx=0x7f6b10007c40).stop 2022-06-29T15:48:26.092 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.085+0000 7f6b2507a700 1 -- 172.21.15.47:0/1043810215 shutdown_connections 2022-06-29T15:48:26.092 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.085+0000 7f6b2507a700 1 --2- 172.21.15.47:0/1043810215 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6b200b6e20 0x7f6b2013cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:26.092 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.085+0000 7f6b2507a700 1 --2- 172.21.15.47:0/1043810215 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6b04053a70 0x7f6b04055f30 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:26.093 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.085+0000 7f6b2507a700 1 --2- 172.21.15.47:0/1043810215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b200b03f0 0x7f6b2013c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:26.093 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.085+0000 7f6b2507a700 1 --2- 172.21.15.47:0/1043810215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b200630f0 0x7f6b201449d0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:26.093 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.085+0000 7f6b2507a700 1 -- 172.21.15.47:0/1043810215 >> 172.21.15.47:0/1043810215 conn(0x7f6b20063670 msgr2=0x7f6b200612b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:26.093 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.085+0000 7f6b2507a700 1 -- 172.21.15.47:0/1043810215 shutdown_connections 2022-06-29T15:48:26.093 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.085+0000 7f6b2507a700 1 -- 172.21.15.47:0/1043810215 wait complete. 2022-06-29T15:48:26.101 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-29T15:48:26.210 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.205+0000 7fad3759e700 1 Processor -- start 2022-06-29T15:48:26.210 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.205+0000 7fad3759e700 1 -- start start 2022-06-29T15:48:26.213 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.205+0000 7fad3759e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fad380b1610 0x7fad380b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:26.214 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.205+0000 7fad3759e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fad3814b4c0 0x7fad380aa130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:26.214 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.205+0000 7fad3759e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad38060790 con 0x7fad380b58d0 2022-06-29T15:48:26.214 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.205+0000 7fad3759e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad380a99e0 con 0x7fad3814b4c0 2022-06-29T15:48:26.214 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.205+0000 7fad3759e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad3814ad80 con 0x7fad380b1610 2022-06-29T15:48:26.215 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.205+0000 7fad3659c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fad380b58d0 0x7fad381475f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35762/0 (socket says 172.21.15.47:35762) 2022-06-29T15:48:26.215 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.205+0000 7fad3659c700 1 -- 172.21.15.47:0/852702670 learned_addr learned my addr 172.21.15.47:0/852702670 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:48:26.215 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.205+0000 7fad36d9d700 1 --2- 172.21.15.47:0/852702670 >> v2:172.21.15.64:3300/0 conn(0x7fad3814b4c0 0x7fad380aa130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:26.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.205+0000 7fad35d9b700 1 --2- 172.21.15.47:0/852702670 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fad380b1610 0x7fad380b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:26.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.205+0000 7fad3559a700 1 -- 172.21.15.47:0/852702670 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2159399529 0 0) 0x7fad38060790 con 0x7fad380b58d0 2022-06-29T15:48:26.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.205+0000 7fad3559a700 1 -- 172.21.15.47:0/852702670 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad1c003260 con 0x7fad380b58d0 2022-06-29T15:48:26.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.205+0000 7fad3559a700 1 -- 172.21.15.47:0/852702670 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2477428318 0 0) 0x7fad1c003260 con 0x7fad380b58d0 2022-06-29T15:48:26.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.205+0000 7fad3559a700 1 -- 172.21.15.47:0/852702670 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fad380b1610 msgr2=0x7fad380b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:26.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.205+0000 7fad3559a700 1 --2- 172.21.15.47:0/852702670 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fad380b1610 0x7fad380b5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:26.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.205+0000 7fad3559a700 1 -- 172.21.15.47:0/852702670 >> v2:172.21.15.64:3300/0 conn(0x7fad3814b4c0 msgr2=0x7fad380aa130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:26.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.205+0000 7fad3559a700 1 --2- 172.21.15.47:0/852702670 >> v2:172.21.15.64:3300/0 conn(0x7fad3814b4c0 0x7fad380aa130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:26.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.205+0000 7fad3559a700 1 -- 172.21.15.47:0/852702670 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad3814b020 con 0x7fad380b58d0 2022-06-29T15:48:26.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.205+0000 7fad3559a700 1 -- 172.21.15.47:0/852702670 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7fad2c001660 con 0x7fad380b58d0 2022-06-29T15:48:26.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.205+0000 7fad3559a700 1 -- 172.21.15.47:0/852702670 >> v1:172.21.15.47:6789/0 conn(0x7fad380b58d0 legacy=0x7fad381475f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:26.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.209+0000 7fad3559a700 1 --2- 172.21.15.47:0/852702670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad1c003750 0x7fad1c003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:26.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.209+0000 7fad3559a700 1 --2- 172.21.15.47:0/852702670 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad1c004500 0x7fad1c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:26.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.209+0000 7fad3559a700 1 --2- 172.21.15.47:0/852702670 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fad380b1610 0x7fad1c004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:26.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.209+0000 7fad3559a700 1 -- 172.21.15.47:0/852702670 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad380a99e0 con 0x7fad1c004500 2022-06-29T15:48:26.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.209+0000 7fad3559a700 1 -- 172.21.15.47:0/852702670 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3814ad80 con 0x7fad1c003750 2022-06-29T15:48:26.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.209+0000 7fad3559a700 1 -- 172.21.15.47:0/852702670 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c00b750 con 0x7fad380b1610 2022-06-29T15:48:26.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.209+0000 7fad35d9b700 1 --2- 172.21.15.47:0/852702670 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad1c004500 0x7fad1c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:26.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.209+0000 7fad36d9d700 1 --2- 172.21.15.47:0/852702670 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fad380b1610 0x7fad1c004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:26.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.209+0000 7fad3659c700 1 --2- 172.21.15.47:0/852702670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad1c003750 0x7fad1c003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:26.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.209+0000 7fad35d9b700 1 -- 172.21.15.47:0/852702670 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fad380b1610 msgr2=0x7fad1c004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:26.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.209+0000 7fad35d9b700 1 --2- 172.21.15.47:0/852702670 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fad380b1610 0x7fad1c004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:26.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.209+0000 7fad35d9b700 1 -- 172.21.15.47:0/852702670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad1c003750 msgr2=0x7fad1c003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:26.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.209+0000 7fad35d9b700 1 --2- 172.21.15.47:0/852702670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad1c003750 0x7fad1c003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:26.222 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.209+0000 7fad35d9b700 1 -- 172.21.15.47:0/852702670 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad1c003260 con 0x7fad1c004500 2022-06-29T15:48:26.222 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.209+0000 7fad3659c700 1 --2- 172.21.15.47:0/852702670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad1c003750 0x7fad1c003b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:48:26.222 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.209+0000 7fad35d9b700 1 --2- 172.21.15.47:0/852702670 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad1c004500 0x7fad1c004980 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 rx=0x7fad2800c000 tx=0x7fad28006df0).ready entity=mon.0 client_cookie=1aeded3a892bdfaf server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:26.222 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.209+0000 7fad3559a700 1 -- 172.21.15.47:0/852702670 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad2800e5f0 con 0x7fad1c004500 2022-06-29T15:48:26.223 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.209+0000 7fad3559a700 1 -- 172.21.15.47:0/852702670 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad2800e750 con 0x7fad1c004500 2022-06-29T15:48:26.223 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.209+0000 7fad3559a700 1 -- 172.21.15.47:0/852702670 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad2800eab0 con 0x7fad1c004500 2022-06-29T15:48:26.227 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.209+0000 7fad3759e700 1 -- 172.21.15.47:0/852702670 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad1c004500 msgr2=0x7fad1c004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:26.227 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.209+0000 7fad3759e700 1 --2- 172.21.15.47:0/852702670 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad1c004500 0x7fad1c004980 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 rx=0x7fad2800c000 tx=0x7fad28006df0).stop 2022-06-29T15:48:26.227 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.209+0000 7fad36d9d700 1 -- 172.21.15.47:0/852702670 reap_dead start 2022-06-29T15:48:26.228 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.209+0000 7fad3759e700 1 -- 172.21.15.47:0/852702670 shutdown_connections 2022-06-29T15:48:26.228 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.209+0000 7fad3759e700 1 -- 172.21.15.47:0/852702670 >> 172.21.15.47:0/852702670 conn(0x7fad38062560 msgr2=0x7fad38062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:26.228 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.209+0000 7fad3759e700 1 -- 172.21.15.47:0/852702670 shutdown_connections 2022-06-29T15:48:26.228 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.209+0000 7fad3759e700 1 -- 172.21.15.47:0/852702670 wait complete. 2022-06-29T15:48:26.229 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.213+0000 7fad3759e700 1 Processor -- start 2022-06-29T15:48:26.229 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.213+0000 7fad3759e700 1 -- start start 2022-06-29T15:48:26.229 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.213+0000 7fad3759e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fad1c004500 0x7fad38149bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:26.229 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.213+0000 7fad3759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad381420b0 0x7fad38140d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:26.230 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.213+0000 7fad3759e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad38142800 0x7fad38141240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:26.230 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.213+0000 7fad3759e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad380af5f0 con 0x7fad38142800 2022-06-29T15:48:26.230 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.213+0000 7fad3759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad380ae1c0 con 0x7fad381420b0 2022-06-29T15:48:26.230 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.213+0000 7fad3759e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad380ae460 con 0x7fad1c004500 2022-06-29T15:48:26.231 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.213+0000 7fad36d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad38142800 0x7fad38141240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:26.231 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.213+0000 7fad36d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad38142800 0x7fad38141240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50222/0 (socket says 172.21.15.47:50222) 2022-06-29T15:48:26.231 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.213+0000 7fad36d9d700 1 -- 172.21.15.47:0/2644795500 learned_addr learned my addr 172.21.15.47:0/2644795500 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:26.231 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.213+0000 7fad35d9b700 1 --2- 172.21.15.47:0/2644795500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad381420b0 0x7fad38140d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:26.232 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.213+0000 7fad3659c700 1 --2- 172.21.15.47:0/2644795500 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fad1c004500 0x7fad38149bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:26.232 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.213+0000 7fad36d9d700 1 -- 172.21.15.47:0/2644795500 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fad1c004500 msgr2=0x7fad38149bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:26.232 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.213+0000 7fad36d9d700 1 --2- 172.21.15.47:0/2644795500 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fad1c004500 0x7fad38149bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:26.232 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.213+0000 7fad36d9d700 1 -- 172.21.15.47:0/2644795500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad381420b0 msgr2=0x7fad38140d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:26.233 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.213+0000 7fad36d9d700 1 --2- 172.21.15.47:0/2644795500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad381420b0 0x7fad38140d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:26.233 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.213+0000 7fad36d9d700 1 -- 172.21.15.47:0/2644795500 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad380aeeb0 con 0x7fad38142800 2022-06-29T15:48:26.233 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.213+0000 7fad35d9b700 1 --2- 172.21.15.47:0/2644795500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad381420b0 0x7fad38140d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:48:26.233 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.213+0000 7fad36d9d700 1 --2- 172.21.15.47:0/2644795500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad38142800 0x7fad38141240 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 rx=0x7fad3000b000 tx=0x7fad30008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:26.234 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.213+0000 7fad277fe700 1 -- 172.21.15.47:0/2644795500 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad300145f0 con 0x7fad38142800 2022-06-29T15:48:26.234 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.213+0000 7fad277fe700 1 -- 172.21.15.47:0/2644795500 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad30014750 con 0x7fad38142800 2022-06-29T15:48:26.234 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.213+0000 7fad3759e700 1 -- 172.21.15.47:0/2644795500 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad380af150 con 0x7fad38142800 2022-06-29T15:48:26.234 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.213+0000 7fad277fe700 1 -- 172.21.15.47:0/2644795500 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad30014ab0 con 0x7fad38142800 2022-06-29T15:48:26.235 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.213+0000 7fad3759e700 1 -- 172.21.15.47:0/2644795500 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad38141730 con 0x7fad38142800 2022-06-29T15:48:26.235 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.217+0000 7fad3759e700 1 -- 172.21.15.47:0/2644795500 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad380af2e0 con 0x7fad38142800 2022-06-29T15:48:26.235 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.217+0000 7fad277fe700 1 -- 172.21.15.47:0/2644795500 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fad30012040 con 0x7fad38142800 2022-06-29T15:48:26.235 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.217+0000 7fad277fe700 1 --2- 172.21.15.47:0/2644795500 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fad14053a70 0x7fad14055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:26.236 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.217+0000 7fad277fe700 1 -- 172.21.15.47:0/2644795500 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fad3006c750 con 0x7fad38142800 2022-06-29T15:48:26.236 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.217+0000 7fad3659c700 1 --2- 172.21.15.47:0/2644795500 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fad14053a70 0x7fad14055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:26.236 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.217+0000 7fad3659c700 1 --2- 172.21.15.47:0/2644795500 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fad14053a70 0x7fad14055f30 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7fad2c00d000 tx=0x7fad2c009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:26.237 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.225+0000 7fad277fe700 1 -- 172.21.15.47:0/2644795500 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fad30043510 con 0x7fad38142800 2022-06-29T15:48:26.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.353+0000 7fad3759e700 1 -- 172.21.15.47:0/2644795500 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fad380573d0 con 0x7fad38142800 2022-06-29T15:48:26.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.353+0000 7fad277fe700 1 -- 172.21.15.47:0/2644795500 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v78) v1 ==== 100+0+10 (secure 0 0 0) 0x7fad30046b30 con 0x7fad38142800 2022-06-29T15:48:26.360 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:48:26.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.357+0000 7fad3759e700 1 -- 172.21.15.47:0/2644795500 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fad14053a70 msgr2=0x7fad14055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:26.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.357+0000 7fad3759e700 1 --2- 172.21.15.47:0/2644795500 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fad14053a70 0x7fad14055f30 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7fad2c00d000 tx=0x7fad2c009040).stop 2022-06-29T15:48:26.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.357+0000 7fad3759e700 1 -- 172.21.15.47:0/2644795500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad38142800 msgr2=0x7fad38141240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:26.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.357+0000 7fad3759e700 1 --2- 172.21.15.47:0/2644795500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad38142800 0x7fad38141240 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 rx=0x7fad3000b000 tx=0x7fad30008040).stop 2022-06-29T15:48:26.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.357+0000 7fad3759e700 1 -- 172.21.15.47:0/2644795500 shutdown_connections 2022-06-29T15:48:26.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.357+0000 7fad3759e700 1 --2- 172.21.15.47:0/2644795500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad38142800 0x7fad38141240 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:26.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.357+0000 7fad3759e700 1 --2- 172.21.15.47:0/2644795500 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fad14053a70 0x7fad14055f30 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:26.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.357+0000 7fad3759e700 1 --2- 172.21.15.47:0/2644795500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad381420b0 0x7fad38140d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:26.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.357+0000 7fad3759e700 1 --2- 172.21.15.47:0/2644795500 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fad1c004500 0x7fad38149bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:26.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.357+0000 7fad3759e700 1 -- 172.21.15.47:0/2644795500 >> 172.21.15.47:0/2644795500 conn(0x7fad38062560 msgr2=0x7fad380b3a70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:26.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.357+0000 7fad3759e700 1 -- 172.21.15.47:0/2644795500 shutdown_connections 2022-06-29T15:48:26.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.357+0000 7fad3759e700 1 -- 172.21.15.47:0/2644795500 wait complete. 2022-06-29T15:48:26.373 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-29T15:48:26.374 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:48:26.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.469+0000 7f3ce5a4e700 1 Processor -- start 2022-06-29T15:48:26.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.469+0000 7f3ce5a4e700 1 -- start start 2022-06-29T15:48:26.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.469+0000 7f3ce5a4e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3ce00b5c60 0x7f3ce0144980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:26.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.469+0000 7f3ce5a4e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3ce00a72f0 0x7f3ce00b2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:26.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.469+0000 7f3ce5a4e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ce00a9120 con 0x7f3ce0060eb0 2022-06-29T15:48:26.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.469+0000 7f3ce5a4e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ce00b0b80 con 0x7f3ce00a72f0 2022-06-29T15:48:26.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.469+0000 7f3ce5a4e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce00a89e0 con 0x7f3ce00b5c60 2022-06-29T15:48:26.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.469+0000 7f3ce524d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3ce00a72f0 0x7f3ce00b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:26.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.469+0000 7f3ce4a4c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3ce00b5c60 0x7f3ce0144980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:26.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.469+0000 7f3cdffff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f3ce0060eb0 0x7f3ce00b1e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:43346/0 (socket says 172.21.15.47:43346) 2022-06-29T15:48:26.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.469+0000 7f3cdffff700 1 -- 172.21.15.47:0/3947677713 learned_addr learned my addr 172.21.15.47:0/3947677713 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:48:26.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.469+0000 7f3cdf7fe700 1 -- 172.21.15.47:0/3947677713 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3033285144 0 0) 0x7f3ce00a9120 con 0x7f3ce0060eb0 2022-06-29T15:48:26.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.469+0000 7f3cdf7fe700 1 -- 172.21.15.47:0/3947677713 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cc4003260 con 0x7f3ce0060eb0 2022-06-29T15:48:26.484 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.473+0000 7f3ce524d700 1 -- 172.21.15.47:0/3947677713 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3ce00b5c60 msgr2=0x7f3ce0144980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:26.484 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.473+0000 7f3ce524d700 1 --2- 172.21.15.47:0/3947677713 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3ce00b5c60 0x7f3ce0144980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:26.485 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.473+0000 7f3ce524d700 1 -- 172.21.15.47:0/3947677713 >> v1:172.21.15.64:6789/0 conn(0x7f3ce0060eb0 legacy=0x7f3ce00b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:26.485 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.473+0000 7f3ce524d700 1 -- 172.21.15.47:0/3947677713 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce00a8c80 con 0x7f3ce00a72f0 2022-06-29T15:48:26.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.473+0000 7f3ce4a4c700 1 --2- 172.21.15.47:0/3947677713 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3ce00b5c60 0x7f3ce0144980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:48:26.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.473+0000 7f3ce524d700 1 --2- 172.21.15.47:0/3947677713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ce00a72f0 0x7f3ce00b2540 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7f3cd80032f0 tx=0x7f3cd80091d0).ready entity=mon.1 client_cookie=5b988c3a89dbdcf6 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:26.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.473+0000 7f3cdf7fe700 1 -- 172.21.15.47:0/3947677713 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3cd800b600 con 0x7f3ce00a72f0 2022-06-29T15:48:26.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.473+0000 7f3cdf7fe700 1 -- 172.21.15.47:0/3947677713 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd800b760 con 0x7f3ce00a72f0 2022-06-29T15:48:26.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.473+0000 7f3cdf7fe700 1 -- 172.21.15.47:0/3947677713 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3cd800b8d0 con 0x7f3ce00a72f0 2022-06-29T15:48:26.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.473+0000 7f3ce5a4e700 1 -- 172.21.15.47:0/3947677713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ce00a72f0 msgr2=0x7f3ce00b2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:26.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.473+0000 7f3ce5a4e700 1 --2- 172.21.15.47:0/3947677713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ce00a72f0 0x7f3ce00b2540 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7f3cd80032f0 tx=0x7f3cd80091d0).stop 2022-06-29T15:48:26.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.473+0000 7f3ce5a4e700 1 -- 172.21.15.47:0/3947677713 shutdown_connections 2022-06-29T15:48:26.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.473+0000 7f3ce5a4e700 1 --2- 172.21.15.47:0/3947677713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ce00a72f0 0x7f3ce00b2540 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:26.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.473+0000 7f3ce5a4e700 1 --2- 172.21.15.47:0/3947677713 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3ce00b5c60 0x7f3ce0144980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:26.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.473+0000 7f3ce5a4e700 1 -- 172.21.15.47:0/3947677713 >> 172.21.15.47:0/3947677713 conn(0x7f3ce0062560 msgr2=0x7f3ce0062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:26.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.473+0000 7f3ce5a4e700 1 -- 172.21.15.47:0/3947677713 shutdown_connections 2022-06-29T15:48:26.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.473+0000 7f3ce5a4e700 1 -- 172.21.15.47:0/3947677713 wait complete. 2022-06-29T15:48:26.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.477+0000 7f3ce5a4e700 1 Processor -- start 2022-06-29T15:48:26.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.477+0000 7f3ce5a4e700 1 -- start start 2022-06-29T15:48:26.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.477+0000 7f3ce5a4e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ce0060eb0 0x7f3ce0143200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:26.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.477+0000 7f3ce5a4e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ce00a72f0 0x7f3ce014c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:26.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.477+0000 7f3ce5a4e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3ce00b5c60 0x7f3ce014c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:26.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.477+0000 7f3ce5a4e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce00ac360 con 0x7f3ce0060eb0 2022-06-29T15:48:26.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.477+0000 7f3ce5a4e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce00b6040 con 0x7f3ce00a72f0 2022-06-29T15:48:26.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.477+0000 7f3ce5a4e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce0143e40 con 0x7f3ce00b5c60 2022-06-29T15:48:26.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.477+0000 7f3ce4a4c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ce0060eb0 0x7f3ce0143200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:26.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.477+0000 7f3ce4a4c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ce0060eb0 0x7f3ce0143200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50234/0 (socket says 172.21.15.47:50234) 2022-06-29T15:48:26.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.477+0000 7f3ce4a4c700 1 -- 172.21.15.47:0/2499309668 learned_addr learned my addr 172.21.15.47:0/2499309668 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:26.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.477+0000 7f3cdffff700 1 --2- 172.21.15.47:0/2499309668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ce00a72f0 0x7f3ce014c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:26.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.477+0000 7f3ce524d700 1 --2- 172.21.15.47:0/2499309668 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3ce00b5c60 0x7f3ce014c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:26.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.477+0000 7f3ce4a4c700 1 -- 172.21.15.47:0/2499309668 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3ce00b5c60 msgr2=0x7f3ce014c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:26.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.477+0000 7f3ce4a4c700 1 --2- 172.21.15.47:0/2499309668 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3ce00b5c60 0x7f3ce014c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:26.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.477+0000 7f3ce4a4c700 1 -- 172.21.15.47:0/2499309668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ce00a72f0 msgr2=0x7f3ce014c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:26.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.477+0000 7f3ce4a4c700 1 --2- 172.21.15.47:0/2499309668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ce00a72f0 0x7f3ce014c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:26.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.477+0000 7f3ce4a4c700 1 -- 172.21.15.47:0/2499309668 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd400b120 con 0x7f3ce0060eb0 2022-06-29T15:48:26.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.477+0000 7f3ce4a4c700 1 --2- 172.21.15.47:0/2499309668 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ce0060eb0 0x7f3ce0143200 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 rx=0x7f3cd400c000 tx=0x7f3cd4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:26.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.477+0000 7f3cddffb700 1 -- 172.21.15.47:0/2499309668 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3cd4012600 con 0x7f3ce0060eb0 2022-06-29T15:48:26.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.477+0000 7f3cddffb700 1 -- 172.21.15.47:0/2499309668 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd4012760 con 0x7f3ce0060eb0 2022-06-29T15:48:26.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.477+0000 7f3ce5a4e700 1 -- 172.21.15.47:0/2499309668 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cd8009470 con 0x7f3ce0060eb0 2022-06-29T15:48:26.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.477+0000 7f3cddffb700 1 -- 172.21.15.47:0/2499309668 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3cd4012ac0 con 0x7f3ce0060eb0 2022-06-29T15:48:26.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.477+0000 7f3ce5a4e700 1 -- 172.21.15.47:0/2499309668 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ce0144380 con 0x7f3ce0060eb0 2022-06-29T15:48:26.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.481+0000 7f3ce5a4e700 1 -- 172.21.15.47:0/2499309668 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ce00ac1a0 con 0x7f3ce0060eb0 2022-06-29T15:48:26.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.481+0000 7f3cddffb700 1 -- 172.21.15.47:0/2499309668 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3cd4022070 con 0x7f3ce0060eb0 2022-06-29T15:48:26.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.481+0000 7f3cddffb700 1 --2- 172.21.15.47:0/2499309668 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3cc00539e0 0x7f3cc0055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:26.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.481+0000 7f3cddffb700 1 -- 172.21.15.47:0/2499309668 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f3cd406c580 con 0x7f3ce0060eb0 2022-06-29T15:48:26.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.481+0000 7f3cdffff700 1 --2- 172.21.15.47:0/2499309668 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3cc00539e0 0x7f3cc0055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:26.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.481+0000 7f3cdffff700 1 --2- 172.21.15.47:0/2499309668 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3cc00539e0 0x7f3cc0055ea0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7f3ccc00d000 tx=0x7f3ccc009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:26.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.489+0000 7f3cddffb700 1 -- 172.21.15.47:0/2499309668 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3cd40432a0 con 0x7f3ce0060eb0 2022-06-29T15:48:26.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.653+0000 7f3ce5a4e700 1 -- 172.21.15.47:0/2499309668 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ce014be10 con 0x7f3ce0060eb0 2022-06-29T15:48:26.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.653+0000 7f3cddffb700 1 -- 172.21.15.47:0/2499309668 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+2493 (secure 0 0 0) 0x7f3cd40468c0 con 0x7f3ce0060eb0 2022-06-29T15:48:26.659 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:48:26.659 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":35,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6291,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.47:6835/2910037081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2910037081},{"type":"v1","addr":"172.21.15.47:6835","nonce":2910037081}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":34},{"gid":6300,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/359532891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":359532891},{"type":"v1","addr":"172.21.15.47:6837","nonce":359532891}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":33}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:48:18.345845+0000","modified":"2022-06-29T15:48:19.349718+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":55562},"failed":[],"damaged":[],"stopped":[],"info":{"gid_55562":{"gid":55562,"name":"b","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6833/3871453337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3871453337},{"type":"v1","addr":"172.21.15.64:6833","nonce":3871453337}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2022-06-29T15:48:26.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.657+0000 7f3ce5a4e700 1 -- 172.21.15.47:0/2499309668 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3cc00539e0 msgr2=0x7f3cc0055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:26.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.657+0000 7f3ce5a4e700 1 --2- 172.21.15.47:0/2499309668 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3cc00539e0 0x7f3cc0055ea0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7f3ccc00d000 tx=0x7f3ccc009040).stop 2022-06-29T15:48:26.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.657+0000 7f3ce5a4e700 1 -- 172.21.15.47:0/2499309668 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ce0060eb0 msgr2=0x7f3ce0143200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:26.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.657+0000 7f3ce5a4e700 1 --2- 172.21.15.47:0/2499309668 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ce0060eb0 0x7f3ce0143200 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 rx=0x7f3cd400c000 tx=0x7f3cd4008040).stop 2022-06-29T15:48:26.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.657+0000 7f3ce5a4e700 1 -- 172.21.15.47:0/2499309668 shutdown_connections 2022-06-29T15:48:26.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.657+0000 7f3ce5a4e700 1 --2- 172.21.15.47:0/2499309668 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3ce00b5c60 0x7f3ce014c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:26.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.657+0000 7f3ce5a4e700 1 --2- 172.21.15.47:0/2499309668 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3cc00539e0 0x7f3cc0055ea0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:26.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.657+0000 7f3ce5a4e700 1 --2- 172.21.15.47:0/2499309668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ce00a72f0 0x7f3ce014c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:26.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.657+0000 7f3ce5a4e700 1 --2- 172.21.15.47:0/2499309668 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ce0060eb0 0x7f3ce0143200 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:26.663 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.657+0000 7f3ce5a4e700 1 -- 172.21.15.47:0/2499309668 >> 172.21.15.47:0/2499309668 conn(0x7f3ce0062560 msgr2=0x7f3ce00afae0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:26.663 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.657+0000 7f3ce5a4e700 1 -- 172.21.15.47:0/2499309668 shutdown_connections 2022-06-29T15:48:26.663 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:26.657+0000 7f3ce5a4e700 1 -- 172.21.15.47:0/2499309668 wait complete. 2022-06-29T15:48:26.664 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 35 2022-06-29T15:48:26.974 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 35, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-29T15:48:18.345845+0000', 'modified': '2022-06-29T15:48:19.349718+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 55562}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_55562': {'gid': 55562, 'name': 'b', 'rank': 0, 'incarnation': 34, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.64:6833/3871453337', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6832', 'nonce': 3871453337}, {'type': 'v1', 'addr': '172.21.15.64:6833', 'nonce': 3871453337}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-29T15:48:26.974 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-29T15:48:26.975 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-06-29T15:48:27.092 INFO:teuthology.orchestra.run.smithi064.stdout:{"cluster_fsid":"9a68894a-53ce-400e-838b-14eab91b834c","whoami":0,"id":55562,"want_state":"up:active","state":"up:active","rank_uptime":8.7309812260000008,"mdsmap_epoch":35,"osdmap_epoch":78,"osdmap_epoch_barrier":78,"uptime":13.135139968000001} 2022-06-29T15:48:27.092 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9a68894a-53ce-400e-838b-14eab91b834c", "id": 55562, "mdsmap_epoch": 35, "osdmap_epoch": 78, "osdmap_epoch_barrier": 78, "rank_uptime": 8.730981226, "state": "up:active", "uptime": 13.135139968, "want_state": "up:active", "whoami": 0 } 2022-06-29T15:48:27.093 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-29T15:48:27.093 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2022-06-29T15:48:27.094 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi164.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2022-06-29T15:48:27.094 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:48:27.095 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:48:27.104 DEBUG:teuthology.orchestra.run.smithi164:> sudo modprobe fuse 2022-06-29T15:48:27.163 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:48:27.164 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:48:27.178 INFO:teuthology.orchestra.run.smithi164.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:48:27.179 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:48:27.180 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:48:27.181 DEBUG:teuthology.orchestra.run.smithi164:> ls /sys/fs/fuse/connections 2022-06-29T15:48:27.187 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-29T15:48:27.188 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:48:27.191 DEBUG:teuthology.orchestra.run.smithi164:> sudo modprobe fuse 2022-06-29T15:48:27.207 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:48:27.208 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:48:27.217 INFO:teuthology.orchestra.run.smithi164.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:48:27.218 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:48:27.219 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:48:27.220 DEBUG:teuthology.orchestra.run.smithi164:> ls /sys/fs/fuse/connections 2022-06-29T15:48:27.263 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi164.stderr:ceph-fuse[12464]: starting ceph client 2022-06-29T15:48:27.263 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi164.stderr:2022-06-29T15:48:27.260+0000 7f8b3a7cc080 -1 init, newargv = 0x564b0079c2c0 newargc=9 2022-06-29T15:48:27.273 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi164.stderr:ceph-fuse[12464]: starting fuse 2022-06-29T15:48:28.326 DEBUG:teuthology.orchestra.run.smithi164:> sudo modprobe fuse 2022-06-29T15:48:28.345 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:48:28.346 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:48:28.401 INFO:teuthology.orchestra.run.smithi164.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:48:28.403 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:48:28.404 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:48:28.405 DEBUG:teuthology.orchestra.run.smithi164:> ls /sys/fs/fuse/connections 2022-06-29T15:48:28.413 INFO:teuthology.orchestra.run.smithi164.stdout:52 2022-06-29T15:48:28.413 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-29T15:48:28.414 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:48:28.415 DEBUG:teuthology.orchestra.run.smithi164:> sudo python3 -c ' 2022-06-29T15:48:28.416 DEBUG:teuthology.orchestra.run.smithi164:> import glob 2022-06-29T15:48:28.417 DEBUG:teuthology.orchestra.run.smithi164:> import re 2022-06-29T15:48:28.417 DEBUG:teuthology.orchestra.run.smithi164:> import os 2022-06-29T15:48:28.418 DEBUG:teuthology.orchestra.run.smithi164:> import subprocess 2022-06-29T15:48:28.419 DEBUG:teuthology.orchestra.run.smithi164:> 2022-06-29T15:48:28.420 DEBUG:teuthology.orchestra.run.smithi164:> def find_socket(client_name): 2022-06-29T15:48:28.421 DEBUG:teuthology.orchestra.run.smithi164:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2022-06-29T15:48:28.421 DEBUG:teuthology.orchestra.run.smithi164:> files = glob.glob(asok_path) 2022-06-29T15:48:28.422 DEBUG:teuthology.orchestra.run.smithi164:> 2022-06-29T15:48:28.423 DEBUG:teuthology.orchestra.run.smithi164:> # Given a non-glob path, it better be there 2022-06-29T15:48:28.424 DEBUG:teuthology.orchestra.run.smithi164:> if "*" not in asok_path: 2022-06-29T15:48:28.425 DEBUG:teuthology.orchestra.run.smithi164:> assert(len(files) == 1) 2022-06-29T15:48:28.425 DEBUG:teuthology.orchestra.run.smithi164:> return files[0] 2022-06-29T15:48:28.426 DEBUG:teuthology.orchestra.run.smithi164:> 2022-06-29T15:48:28.427 DEBUG:teuthology.orchestra.run.smithi164:> for f in files: 2022-06-29T15:48:28.428 DEBUG:teuthology.orchestra.run.smithi164:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-29T15:48:28.428 DEBUG:teuthology.orchestra.run.smithi164:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-29T15:48:28.429 DEBUG:teuthology.orchestra.run.smithi164:> return f 2022-06-29T15:48:28.430 DEBUG:teuthology.orchestra.run.smithi164:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-29T15:48:28.431 DEBUG:teuthology.orchestra.run.smithi164:> 2022-06-29T15:48:28.431 DEBUG:teuthology.orchestra.run.smithi164:> print(find_socket("client.1")) 2022-06-29T15:48:28.432 DEBUG:teuthology.orchestra.run.smithi164:> ' 2022-06-29T15:48:28.480 INFO:teuthology.orchestra.run.smithi164.stdout:/var/run/ceph/ceph-client.1.12464.asok 2022-06-29T15:48:28.485 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.12464.asok 2022-06-29T15:48:28.486 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:48:28.486 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.12464.asok status 2022-06-29T15:48:28.578 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2022-06-29T15:48:28.579 INFO:teuthology.orchestra.run.smithi164.stdout: "metadata": { 2022-06-29T15:48:28.579 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_sha1": "f25ffac72e5a9e63860ff7deb6767a11f0559a16", 2022-06-29T15:48:28.579 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_version": "ceph version 15.2.16-304-gf25ffac7 (f25ffac72e5a9e63860ff7deb6767a11f0559a16) octopus (stable)", 2022-06-29T15:48:28.580 INFO:teuthology.orchestra.run.smithi164.stdout: "entity_id": "1", 2022-06-29T15:48:28.580 INFO:teuthology.orchestra.run.smithi164.stdout: "hostname": "smithi164", 2022-06-29T15:48:28.580 INFO:teuthology.orchestra.run.smithi164.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2022-06-29T15:48:28.580 INFO:teuthology.orchestra.run.smithi164.stdout: "pid": "12464", 2022-06-29T15:48:28.581 INFO:teuthology.orchestra.run.smithi164.stdout: "root": "/" 2022-06-29T15:48:28.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2022-06-29T15:48:28.581 INFO:teuthology.orchestra.run.smithi164.stdout: "dentry_count": 0, 2022-06-29T15:48:28.581 INFO:teuthology.orchestra.run.smithi164.stdout: "dentry_pinned_count": 0, 2022-06-29T15:48:28.582 INFO:teuthology.orchestra.run.smithi164.stdout: "id": 55730, 2022-06-29T15:48:28.582 INFO:teuthology.orchestra.run.smithi164.stdout: "inst": { 2022-06-29T15:48:28.582 INFO:teuthology.orchestra.run.smithi164.stdout: "name": { 2022-06-29T15:48:28.582 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "client", 2022-06-29T15:48:28.583 INFO:teuthology.orchestra.run.smithi164.stdout: "num": 55730 2022-06-29T15:48:28.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2022-06-29T15:48:28.583 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": { 2022-06-29T15:48:28.583 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "v1", 2022-06-29T15:48:28.584 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": "172.21.15.164:0", 2022-06-29T15:48:28.584 INFO:teuthology.orchestra.run.smithi164.stdout: "nonce": 3865029161 2022-06-29T15:48:28.584 INFO:teuthology.orchestra.run.smithi164.stdout: } 2022-06-29T15:48:28.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2022-06-29T15:48:28.585 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": { 2022-06-29T15:48:28.585 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "v1", 2022-06-29T15:48:28.585 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": "172.21.15.164:0", 2022-06-29T15:48:28.585 INFO:teuthology.orchestra.run.smithi164.stdout: "nonce": 3865029161 2022-06-29T15:48:28.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2022-06-29T15:48:28.586 INFO:teuthology.orchestra.run.smithi164.stdout: "inst_str": "client.55730 172.21.15.164:0/3865029161", 2022-06-29T15:48:28.586 INFO:teuthology.orchestra.run.smithi164.stdout: "addr_str": "172.21.15.164:0/3865029161", 2022-06-29T15:48:28.586 INFO:teuthology.orchestra.run.smithi164.stdout: "inode_count": 1, 2022-06-29T15:48:28.586 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_epoch": 35, 2022-06-29T15:48:28.587 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_epoch": 78, 2022-06-29T15:48:28.587 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_epoch_barrier": 0, 2022-06-29T15:48:28.587 INFO:teuthology.orchestra.run.smithi164.stdout: "blacklisted": false, 2022-06-29T15:48:28.587 INFO:teuthology.orchestra.run.smithi164.stdout: "fs_name": "cephfs" 2022-06-29T15:48:28.588 INFO:teuthology.orchestra.run.smithi164.stdout:} 2022-06-29T15:48:28.589 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:48:28.590 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:48:28.590 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:48:28.598 INFO:teuthology.orchestra.run.smithi164.stdout:fuseblk 2022-06-29T15:48:28.599 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-06-29T15:48:28.600 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:48:28.600 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:48:28.618 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:48:28.731 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.725+0000 7f0918a3a700 1 Processor -- start 2022-06-29T15:48:28.732 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.725+0000 7f0918a3a700 1 -- start start 2022-06-29T15:48:28.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.729+0000 7f0918a3a700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0914145090 0x7f09140b2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:28.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.729+0000 7f0918a3a700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09140a9120 con 0x7f0914060eb0 2022-06-29T15:48:28.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.729+0000 7f0918a3a700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09140b0b80 con 0x7f09140b5c60 2022-06-29T15:48:28.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.729+0000 7f0918a3a700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09140a89e0 con 0x7f0914145090 2022-06-29T15:48:28.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.729+0000 7f091259c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0914060eb0 0x7f09140b1e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35796/0 (socket says 172.21.15.47:35796) 2022-06-29T15:48:28.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.729+0000 7f091259c700 1 -- 172.21.15.47:0/3084688413 learned_addr learned my addr 172.21.15.47:0/3084688413 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:48:28.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.729+0000 7f091359e700 1 --2- 172.21.15.47:0/3084688413 >> v2:172.21.15.64:3300/0 conn(0x7f0914145090 0x7f09140b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:28.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.729+0000 7f0911d9b700 1 -- 172.21.15.47:0/3084688413 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 276445240 0 0) 0x7f09140a9120 con 0x7f0914060eb0 2022-06-29T15:48:28.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.729+0000 7f0911d9b700 1 -- 172.21.15.47:0/3084688413 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08f8003260 con 0x7f0914060eb0 2022-06-29T15:48:28.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.729+0000 7f091359e700 1 -- 172.21.15.47:0/3084688413 >> v1:172.21.15.64:6789/0 conn(0x7f09140b5c60 legacy=0x7f0914144980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:28.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.729+0000 7f091359e700 1 -- 172.21.15.47:0/3084688413 >> v1:172.21.15.47:6789/0 conn(0x7f0914060eb0 legacy=0x7f09140b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:28.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.729+0000 7f091359e700 1 -- 172.21.15.47:0/3084688413 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09140a8c80 con 0x7f0914145090 2022-06-29T15:48:28.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.729+0000 7f091359e700 1 --2- 172.21.15.47:0/3084688413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0914145090 0x7f09140b2540 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f090c0032f0 tx=0x7f090c0091d0).ready entity=mon.1 client_cookie=8eac6ce27991bedd server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:28.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.729+0000 7f0911d9b700 1 -- 172.21.15.47:0/3084688413 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f090c00b6c0 con 0x7f0914145090 2022-06-29T15:48:28.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.729+0000 7f0911d9b700 1 -- 172.21.15.47:0/3084688413 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f090c00b820 con 0x7f0914145090 2022-06-29T15:48:28.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.729+0000 7f0911d9b700 1 -- 172.21.15.47:0/3084688413 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f090c00b990 con 0x7f0914145090 2022-06-29T15:48:28.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.729+0000 7f0918a3a700 1 -- 172.21.15.47:0/3084688413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0914145090 msgr2=0x7f09140b2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:28.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.729+0000 7f0918a3a700 1 --2- 172.21.15.47:0/3084688413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0914145090 0x7f09140b2540 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f090c0032f0 tx=0x7f090c0091d0).stop 2022-06-29T15:48:28.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.729+0000 7f0918a3a700 1 -- 172.21.15.47:0/3084688413 shutdown_connections 2022-06-29T15:48:28.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.729+0000 7f0918a3a700 1 --2- 172.21.15.47:0/3084688413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0914145090 0x7f09140b2540 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:28.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.729+0000 7f0918a3a700 1 -- 172.21.15.47:0/3084688413 >> 172.21.15.47:0/3084688413 conn(0x7f0914062560 msgr2=0x7f0914062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:28.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.729+0000 7f0918a3a700 1 -- 172.21.15.47:0/3084688413 shutdown_connections 2022-06-29T15:48:28.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.729+0000 7f0918a3a700 1 -- 172.21.15.47:0/3084688413 wait complete. 2022-06-29T15:48:28.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.733+0000 7f0918a3a700 1 Processor -- start 2022-06-29T15:48:28.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.733+0000 7f0918a3a700 1 -- start start 2022-06-29T15:48:28.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.733+0000 7f0918a3a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0914060eb0 0x7f09140b3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:28.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.733+0000 7f0918a3a700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f09140b5c60 0x7f0914145c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:28.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.733+0000 7f0918a3a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0914145090 0x7f0914146170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:28.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.733+0000 7f0918a3a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09141494b0 con 0x7f0914060eb0 2022-06-29T15:48:28.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.733+0000 7f0918a3a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0914149cc0 con 0x7f0914145090 2022-06-29T15:48:28.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.733+0000 7f0918a3a700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0914149010 con 0x7f09140b5c60 2022-06-29T15:48:28.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.733+0000 7f0912d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0914060eb0 0x7f09140b3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:28.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.733+0000 7f0912d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0914060eb0 0x7f09140b3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50248/0 (socket says 172.21.15.47:50248) 2022-06-29T15:48:28.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.733+0000 7f0912d9d700 1 -- 172.21.15.47:0/4100265168 learned_addr learned my addr 172.21.15.47:0/4100265168 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:28.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.733+0000 7f091359e700 1 --2- 172.21.15.47:0/4100265168 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0914145090 0x7f0914146170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:28.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.733+0000 7f091259c700 1 --2- 172.21.15.47:0/4100265168 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f09140b5c60 0x7f0914145c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:28.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.733+0000 7f0912d9d700 1 -- 172.21.15.47:0/4100265168 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f09140b5c60 msgr2=0x7f0914145c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:28.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.733+0000 7f0912d9d700 1 --2- 172.21.15.47:0/4100265168 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f09140b5c60 0x7f0914145c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:28.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.733+0000 7f0912d9d700 1 -- 172.21.15.47:0/4100265168 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0914145090 msgr2=0x7f0914146170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:28.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.733+0000 7f0912d9d700 1 --2- 172.21.15.47:0/4100265168 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0914145090 0x7f0914146170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:28.748 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.733+0000 7f0912d9d700 1 -- 172.21.15.47:0/4100265168 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f090c009470 con 0x7f0914060eb0 2022-06-29T15:48:28.748 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.733+0000 7f091259c700 1 --2- 172.21.15.47:0/4100265168 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f09140b5c60 0x7f0914145c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:48:28.748 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.733+0000 7f0912d9d700 1 --2- 172.21.15.47:0/4100265168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0914060eb0 0x7f09140b3650 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 rx=0x7f091406ac40 tx=0x7f0908007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:28.748 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.733+0000 7f0903fff700 1 -- 172.21.15.47:0/4100265168 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0908007020 con 0x7f0914060eb0 2022-06-29T15:48:28.748 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.733+0000 7f0903fff700 1 -- 172.21.15.47:0/4100265168 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0908007180 con 0x7f0914060eb0 2022-06-29T15:48:28.749 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.733+0000 7f0918a3a700 1 -- 172.21.15.47:0/4100265168 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0914146660 con 0x7f0914060eb0 2022-06-29T15:48:28.749 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.733+0000 7f0903fff700 1 -- 172.21.15.47:0/4100265168 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0908006560 con 0x7f0914060eb0 2022-06-29T15:48:28.749 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.733+0000 7f0918a3a700 1 -- 172.21.15.47:0/4100265168 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09141441f0 con 0x7f0914060eb0 2022-06-29T15:48:28.749 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.737+0000 7f0918a3a700 1 -- 172.21.15.47:0/4100265168 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09141491d0 con 0x7f0914060eb0 2022-06-29T15:48:28.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.737+0000 7f0903fff700 1 -- 172.21.15.47:0/4100265168 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0908010070 con 0x7f0914060eb0 2022-06-29T15:48:28.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.737+0000 7f0903fff700 1 --2- 172.21.15.47:0/4100265168 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f08f0053a70 0x7f08f0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:28.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.737+0000 7f0903fff700 1 -- 172.21.15.47:0/4100265168 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f090806b380 con 0x7f0914060eb0 2022-06-29T15:48:28.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.737+0000 7f091259c700 1 --2- 172.21.15.47:0/4100265168 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f08f0053a70 0x7f08f0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:28.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.737+0000 7f091259c700 1 --2- 172.21.15.47:0/4100265168 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f08f0053a70 0x7f08f0055f30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f090400b000 tx=0x7f0904009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:28.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.741+0000 7f0903fff700 1 -- 172.21.15.47:0/4100265168 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f09080420a0 con 0x7f0914060eb0 2022-06-29T15:48:28.913 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.909+0000 7f0918a3a700 1 -- 172.21.15.47:0/4100265168 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f09140573d0 con 0x7f0914060eb0 2022-06-29T15:48:28.914 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.909+0000 7f0903fff700 1 -- 172.21.15.47:0/4100265168 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+2493 (secure 0 0 0) 0x7f09080456c0 con 0x7f0914060eb0 2022-06-29T15:48:28.915 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:48:28.915 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":35,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6291,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.47:6835/2910037081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2910037081},{"type":"v1","addr":"172.21.15.47:6835","nonce":2910037081}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":34},{"gid":6300,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/359532891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":359532891},{"type":"v1","addr":"172.21.15.47:6837","nonce":359532891}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":33}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:48:18.345845+0000","modified":"2022-06-29T15:48:19.349718+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":55562},"failed":[],"damaged":[],"stopped":[],"info":{"gid_55562":{"gid":55562,"name":"b","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6833/3871453337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3871453337},{"type":"v1","addr":"172.21.15.64:6833","nonce":3871453337}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2022-06-29T15:48:28.916 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.913+0000 7f0918a3a700 1 -- 172.21.15.47:0/4100265168 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f08f0053a70 msgr2=0x7f08f0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:28.917 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.913+0000 7f0918a3a700 1 --2- 172.21.15.47:0/4100265168 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f08f0053a70 0x7f08f0055f30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f090400b000 tx=0x7f0904009040).stop 2022-06-29T15:48:28.917 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.913+0000 7f0918a3a700 1 -- 172.21.15.47:0/4100265168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0914060eb0 msgr2=0x7f09140b3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:28.917 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.913+0000 7f0918a3a700 1 --2- 172.21.15.47:0/4100265168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0914060eb0 0x7f09140b3650 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 rx=0x7f091406ac40 tx=0x7f0908007c40).stop 2022-06-29T15:48:28.917 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.913+0000 7f0918a3a700 1 -- 172.21.15.47:0/4100265168 shutdown_connections 2022-06-29T15:48:28.918 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.913+0000 7f0918a3a700 1 --2- 172.21.15.47:0/4100265168 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f08f0053a70 0x7f08f0055f30 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:28.918 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.913+0000 7f0918a3a700 1 --2- 172.21.15.47:0/4100265168 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0914145090 0x7f0914146170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:28.918 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.913+0000 7f0918a3a700 1 --2- 172.21.15.47:0/4100265168 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f09140b5c60 0x7f0914145c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:28.918 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.913+0000 7f0918a3a700 1 --2- 172.21.15.47:0/4100265168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0914060eb0 0x7f09140b3650 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:28.919 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.913+0000 7f0918a3a700 1 -- 172.21.15.47:0/4100265168 >> 172.21.15.47:0/4100265168 conn(0x7f0914062560 msgr2=0x7f09140afae0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:28.919 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.913+0000 7f0918a3a700 1 -- 172.21.15.47:0/4100265168 shutdown_connections 2022-06-29T15:48:28.919 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:28.913+0000 7f0918a3a700 1 -- 172.21.15.47:0/4100265168 wait complete. 2022-06-29T15:48:28.920 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 35 2022-06-29T15:48:28.928 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:48:29.028 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.021+0000 7fab1bfff700 1 Processor -- start 2022-06-29T15:48:29.028 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.021+0000 7fab1bfff700 1 -- start start 2022-06-29T15:48:29.032 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.025+0000 7fab1bfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fab1c05c510 0x7fab1c143bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:29.032 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.025+0000 7fab1bfff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fab1c05c8f0 0x7fab1c0b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:29.032 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.025+0000 7fab1bfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fab1c0b2c40 0x7fab1c0b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:29.032 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.025+0000 7fab1bfff700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab1c142f90 con 0x7fab1c0b2c40 2022-06-29T15:48:29.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.025+0000 7fab1bfff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab1c1431f0 con 0x7fab1c05c510 2022-06-29T15:48:29.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.025+0000 7fab1bfff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c143490 con 0x7fab1c05c8f0 2022-06-29T15:48:29.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.025+0000 7fab1b7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fab1c0b2c40 0x7fab1c0b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:29.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.025+0000 7fab1b7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fab1c0b2c40 0x7fab1c0b30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50260/0 (socket says 172.21.15.47:50260) 2022-06-29T15:48:29.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.025+0000 7fab1b7fe700 1 -- 172.21.15.47:0/1500884035 learned_addr learned my addr 172.21.15.47:0/1500884035 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:29.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.025+0000 7fab1a7fc700 1 --2- 172.21.15.47:0/1500884035 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fab1c05c8f0 0x7fab1c0b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:29.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.025+0000 7fab1affd700 1 --2- 172.21.15.47:0/1500884035 >> v2:172.21.15.64:3300/0 conn(0x7fab1c05c510 0x7fab1c143bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:29.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.025+0000 7fab1b7fe700 1 -- 172.21.15.47:0/1500884035 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fab1c05c8f0 msgr2=0x7fab1c0b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:29.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.025+0000 7fab1b7fe700 1 --2- 172.21.15.47:0/1500884035 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fab1c05c8f0 0x7fab1c0b4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:29.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.025+0000 7fab1b7fe700 1 -- 172.21.15.47:0/1500884035 >> v2:172.21.15.64:3300/0 conn(0x7fab1c05c510 msgr2=0x7fab1c143bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:48:29.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.025+0000 7fab1b7fe700 1 --2- 172.21.15.47:0/1500884035 >> v2:172.21.15.64:3300/0 conn(0x7fab1c05c510 0x7fab1c143bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:29.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.025+0000 7fab1b7fe700 1 -- 172.21.15.47:0/1500884035 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab1c143730 con 0x7fab1c0b2c40 2022-06-29T15:48:29.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.025+0000 7fab1affd700 1 --2- 172.21.15.47:0/1500884035 >> v2:172.21.15.64:3300/0 conn(0x7fab1c05c510 0x7fab1c143bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:48:29.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.025+0000 7fab1b7fe700 1 --2- 172.21.15.47:0/1500884035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab1c0b2c40 0x7fab1c0b30d0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 rx=0x7fab140032f0 tx=0x7fab14009290).ready entity=mon.0 client_cookie=c0441956eca9ec35 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:29.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.025+0000 7fab19ffb700 1 -- 172.21.15.47:0/1500884035 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab1400b870 con 0x7fab1c0b2c40 2022-06-29T15:48:29.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.025+0000 7fab19ffb700 1 -- 172.21.15.47:0/1500884035 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab1400b9d0 con 0x7fab1c0b2c40 2022-06-29T15:48:29.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.025+0000 7fab19ffb700 1 -- 172.21.15.47:0/1500884035 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab1400bb40 con 0x7fab1c0b2c40 2022-06-29T15:48:29.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.025+0000 7fab1bfff700 1 -- 172.21.15.47:0/1500884035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab1c0b2c40 msgr2=0x7fab1c0b30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:29.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.025+0000 7fab1bfff700 1 --2- 172.21.15.47:0/1500884035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab1c0b2c40 0x7fab1c0b30d0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 rx=0x7fab140032f0 tx=0x7fab14009290).stop 2022-06-29T15:48:29.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.025+0000 7fab1bfff700 1 -- 172.21.15.47:0/1500884035 shutdown_connections 2022-06-29T15:48:29.038 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.025+0000 7fab1bfff700 1 --2- 172.21.15.47:0/1500884035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab1c0b2c40 0x7fab1c0b30d0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:29.038 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.025+0000 7fab1bfff700 1 --2- 172.21.15.47:0/1500884035 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fab1c05c8f0 0x7fab1c0b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:29.038 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.025+0000 7fab1bfff700 1 --2- 172.21.15.47:0/1500884035 >> v2:172.21.15.64:3300/0 conn(0x7fab1c05c510 0x7fab1c143bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:29.038 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.025+0000 7fab1bfff700 1 -- 172.21.15.47:0/1500884035 >> 172.21.15.47:0/1500884035 conn(0x7fab1c060e00 msgr2=0x7fab1c0611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:29.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.025+0000 7fab1bfff700 1 -- 172.21.15.47:0/1500884035 shutdown_connections 2022-06-29T15:48:29.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.025+0000 7fab1bfff700 1 -- 172.21.15.47:0/1500884035 wait complete. 2022-06-29T15:48:29.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.029+0000 7fab1bfff700 1 Processor -- start 2022-06-29T15:48:29.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.029+0000 7fab1bfff700 1 -- start start 2022-06-29T15:48:29.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.029+0000 7fab1bfff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fab1c05c510 0x7fab1c1446d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:29.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.029+0000 7fab1bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab1c05c8f0 0x7fab1c146580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:29.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.029+0000 7fab1bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab1c0b2c40 0x7fab1c146a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:29.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.029+0000 7fab1bfff700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c1521d0 con 0x7fab1c05c8f0 2022-06-29T15:48:29.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.029+0000 7fab1bfff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c141510 con 0x7fab1c0b2c40 2022-06-29T15:48:29.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.029+0000 7fab1bfff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c144f90 con 0x7fab1c05c510 2022-06-29T15:48:29.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.029+0000 7fab1a7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab1c05c8f0 0x7fab1c146580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:29.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.029+0000 7fab1a7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab1c05c8f0 0x7fab1c146580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50262/0 (socket says 172.21.15.47:50262) 2022-06-29T15:48:29.045 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.029+0000 7fab1a7fc700 1 -- 172.21.15.47:0/3289725874 learned_addr learned my addr 172.21.15.47:0/3289725874 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:48:29.045 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.029+0000 7fab1b7fe700 1 --2- 172.21.15.47:0/3289725874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab1c0b2c40 0x7fab1c146a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:29.045 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.029+0000 7fab1affd700 1 --2- 172.21.15.47:0/3289725874 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fab1c05c510 0x7fab1c1446d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:29.045 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.029+0000 7fab1a7fc700 1 -- 172.21.15.47:0/3289725874 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fab1c05c510 msgr2=0x7fab1c1446d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:29.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.029+0000 7fab1a7fc700 1 --2- 172.21.15.47:0/3289725874 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fab1c05c510 0x7fab1c1446d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:29.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.029+0000 7fab1a7fc700 1 -- 172.21.15.47:0/3289725874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab1c0b2c40 msgr2=0x7fab1c146a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:29.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.029+0000 7fab1a7fc700 1 --2- 172.21.15.47:0/3289725874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab1c0b2c40 0x7fab1c146a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:29.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.029+0000 7fab1a7fc700 1 -- 172.21.15.47:0/3289725874 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab14009530 con 0x7fab1c05c8f0 2022-06-29T15:48:29.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.029+0000 7fab1affd700 1 --2- 172.21.15.47:0/3289725874 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fab1c05c510 0x7fab1c1446d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:48:29.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.029+0000 7fab1a7fc700 1 --2- 172.21.15.47:0/3289725874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab1c05c8f0 0x7fab1c146580 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 rx=0x7fab1000c000 tx=0x7fab10006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:29.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.029+0000 7fab0bfff700 1 -- 172.21.15.47:0/3289725874 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab1000c830 con 0x7fab1c05c8f0 2022-06-29T15:48:29.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.029+0000 7fab0bfff700 1 -- 172.21.15.47:0/3289725874 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab1000c990 con 0x7fab1c05c8f0 2022-06-29T15:48:29.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.029+0000 7fab1bfff700 1 -- 172.21.15.47:0/3289725874 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab1c141980 con 0x7fab1c05c8f0 2022-06-29T15:48:29.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.029+0000 7fab0bfff700 1 -- 172.21.15.47:0/3289725874 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab1000ccf0 con 0x7fab1c05c8f0 2022-06-29T15:48:29.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.029+0000 7fab1bfff700 1 -- 172.21.15.47:0/3289725874 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab1c146f60 con 0x7fab1c05c8f0 2022-06-29T15:48:29.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.033+0000 7fab1bfff700 1 -- 172.21.15.47:0/3289725874 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab1c141b10 con 0x7fab1c05c8f0 2022-06-29T15:48:29.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.033+0000 7fab0bfff700 1 -- 172.21.15.47:0/3289725874 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fab1001c460 con 0x7fab1c05c8f0 2022-06-29T15:48:29.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.033+0000 7fab0bfff700 1 --2- 172.21.15.47:0/3289725874 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7faaf8053a10 0x7faaf8055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:48:29.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.033+0000 7fab0bfff700 1 -- 172.21.15.47:0/3289725874 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fab1006c460 con 0x7fab1c05c8f0 2022-06-29T15:48:29.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.033+0000 7fab1affd700 1 --2- 172.21.15.47:0/3289725874 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7faaf8053a10 0x7faaf8055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:48:29.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.033+0000 7fab1affd700 1 --2- 172.21.15.47:0/3289725874 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7faaf8053a10 0x7faaf8055ed0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7fab0c008000 tx=0x7fab0c008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:48:29.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.041+0000 7fab0bfff700 1 -- 172.21.15.47:0/3289725874 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fab100431c0 con 0x7fab1c05c8f0 2022-06-29T15:48:29.212 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.205+0000 7fab1bfff700 1 -- 172.21.15.47:0/3289725874 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab1c0573d0 con 0x7fab1c05c8f0 2022-06-29T15:48:29.212 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.209+0000 7fab0bfff700 1 -- 172.21.15.47:0/3289725874 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+2493 (secure 0 0 0) 0x7fab100467e0 con 0x7fab1c05c8f0 2022-06-29T15:48:29.213 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:48:29.213 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":35,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6291,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.47:6835/2910037081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2910037081},{"type":"v1","addr":"172.21.15.47:6835","nonce":2910037081}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":34},{"gid":6300,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/359532891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":359532891},{"type":"v1","addr":"172.21.15.47:6837","nonce":359532891}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":33}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:48:18.345845+0000","modified":"2022-06-29T15:48:19.349718+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":55562},"failed":[],"damaged":[],"stopped":[],"info":{"gid_55562":{"gid":55562,"name":"b","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6833/3871453337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3871453337},{"type":"v1","addr":"172.21.15.64:6833","nonce":3871453337}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2022-06-29T15:48:29.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.209+0000 7fab1bfff700 1 -- 172.21.15.47:0/3289725874 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7faaf8053a10 msgr2=0x7faaf8055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:29.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.209+0000 7fab1bfff700 1 --2- 172.21.15.47:0/3289725874 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7faaf8053a10 0x7faaf8055ed0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7fab0c008000 tx=0x7fab0c008040).stop 2022-06-29T15:48:29.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.209+0000 7fab1bfff700 1 -- 172.21.15.47:0/3289725874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab1c05c8f0 msgr2=0x7fab1c146580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:48:29.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.209+0000 7fab1bfff700 1 --2- 172.21.15.47:0/3289725874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab1c05c8f0 0x7fab1c146580 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 rx=0x7fab1000c000 tx=0x7fab10006df0).stop 2022-06-29T15:48:29.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.209+0000 7fab1bfff700 1 -- 172.21.15.47:0/3289725874 shutdown_connections 2022-06-29T15:48:29.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.209+0000 7fab1bfff700 1 --2- 172.21.15.47:0/3289725874 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7faaf8053a10 0x7faaf8055ed0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:29.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.209+0000 7fab1bfff700 1 --2- 172.21.15.47:0/3289725874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab1c0b2c40 0x7fab1c146a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:29.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.209+0000 7fab1bfff700 1 --2- 172.21.15.47:0/3289725874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab1c05c8f0 0x7fab1c146580 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:29.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.209+0000 7fab1bfff700 1 --2- 172.21.15.47:0/3289725874 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fab1c05c510 0x7fab1c1446d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:48:29.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.209+0000 7fab1bfff700 1 -- 172.21.15.47:0/3289725874 >> 172.21.15.47:0/3289725874 conn(0x7fab1c060e00 msgr2=0x7fab1c0b8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:48:29.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.209+0000 7fab1bfff700 1 -- 172.21.15.47:0/3289725874 shutdown_connections 2022-06-29T15:48:29.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:48:29.213+0000 7fab1bfff700 1 -- 172.21.15.47:0/3289725874 wait complete. 2022-06-29T15:48:29.220 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 35 2022-06-29T15:48:29.233 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json config set mds_cap_revoke_eviction_timeout 30.0 2022-06-29T15:48:29.345 INFO:teuthology.orchestra.run.smithi064.stdout:{"success":"mds_cap_revoke_eviction_timeout = '30.000000' "} 2022-06-29T15:48:29.346 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_cap_revoke_eviction_timeout = '30.000000' " } 2022-06-29T15:48:29.347 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:48:29.347 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:48:29.347 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:48:29.356 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2022-06-29T15:48:29.358 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-29T15:48:29.359 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2022-06-29T15:48:29.359 DEBUG:teuthology.orchestra.run.smithi099:> import time 2022-06-29T15:48:29.360 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:48:29.360 DEBUG:teuthology.orchestra.run.smithi099:> with open("/home/ubuntu/cephtest/mnt.0/background_file", '"'"'w'"'"') as f: 2022-06-29T15:48:29.360 DEBUG:teuthology.orchestra.run.smithi099:> f.write('"'"'content'"'"') 2022-06-29T15:48:29.361 DEBUG:teuthology.orchestra.run.smithi099:> f.flush() 2022-06-29T15:48:29.361 DEBUG:teuthology.orchestra.run.smithi099:> f.write('"'"'content2'"'"') 2022-06-29T15:48:29.362 DEBUG:teuthology.orchestra.run.smithi099:> while True: 2022-06-29T15:48:29.362 DEBUG:teuthology.orchestra.run.smithi099:> time.sleep(1) 2022-06-29T15:48:29.362 DEBUG:teuthology.orchestra.run.smithi099:> ' 2022-06-29T15:48:29.365 DEBUG:teuthology.orchestra.run.smithi099:> sudo ls /home/ubuntu/cephtest/mnt.0/background_file 2022-06-29T15:48:29.383 INFO:teuthology.orchestra.run.smithi099.stderr:ls: cannot access '/home/ubuntu/cephtest/mnt.0/background_file': No such file or directory 2022-06-29T15:48:29.383 DEBUG:teuthology.orchestra.run:got remote process result: 2 2022-06-29T15:48:30.385 DEBUG:teuthology.orchestra.run.smithi099:> sudo ls /home/ubuntu/cephtest/mnt.0/background_file 2022-06-29T15:48:30.403 INFO:teuthology.orchestra.run.smithi099.stdout:/home/ubuntu/cephtest/mnt.0/background_file 2022-06-29T15:48:30.405 DEBUG:tasks.cephfs.mount:File background_file became visible from 0 after 1s 2022-06-29T15:48:30.405 DEBUG:teuthology.orchestra.run.smithi164:> sudo ls /home/ubuntu/cephtest/mnt.1/background_file 2022-06-29T15:48:30.446 INFO:teuthology.orchestra.run.smithi164.stdout:/home/ubuntu/cephtest/mnt.1/background_file 2022-06-29T15:48:30.447 DEBUG:tasks.cephfs.mount:File background_file became visible from 1 after 0s 2022-06-29T15:48:30.448 INFO:tasks.cephfs.fuse_mount:Killing ceph-fuse connection on ubuntu@smithi099.front.sepia.ceph.com... 2022-06-29T15:48:30.465 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:48:30.465 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:48:30.466 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:48:30.474 INFO:teuthology.orchestra.run.smithi164.stdout:fuseblk 2022-06-29T15:48:30.475 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-06-29T15:48:30.476 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2022-06-29T15:48:30.476 DEBUG:teuthology.orchestra.run.smithi164:> import os 2022-06-29T15:48:30.477 DEBUG:teuthology.orchestra.run.smithi164:> import time 2022-06-29T15:48:30.477 DEBUG:teuthology.orchestra.run.smithi164:> 2022-06-29T15:48:30.477 DEBUG:teuthology.orchestra.run.smithi164:> fd = os.open("/home/ubuntu/cephtest/mnt.1/background_file", os.O_RDWR | os.O_CREAT, 0o644) 2022-06-29T15:48:30.478 DEBUG:teuthology.orchestra.run.smithi164:> try: 2022-06-29T15:48:30.478 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2022-06-29T15:48:30.479 DEBUG:teuthology.orchestra.run.smithi164:> os.write(fd, b'"'"'content'"'"') 2022-06-29T15:48:30.479 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(1) 2022-06-29T15:48:30.480 DEBUG:teuthology.orchestra.run.smithi164:> if not False: 2022-06-29T15:48:30.480 DEBUG:teuthology.orchestra.run.smithi164:> break 2022-06-29T15:48:30.481 DEBUG:teuthology.orchestra.run.smithi164:> except IOError as e: 2022-06-29T15:48:30.481 DEBUG:teuthology.orchestra.run.smithi164:> pass 2022-06-29T15:48:30.482 DEBUG:teuthology.orchestra.run.smithi164:> os.close(fd) 2022-06-29T15:48:30.482 DEBUG:teuthology.orchestra.run.smithi164:> ' 2022-06-29T15:48:42.129 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:48:42.133 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:48:42.135 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:48:42.138 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:49:10.011 INFO:tasks.cephfs.test_misc:cap_waiter waited 39.52635192871094s 2022-06-29T15:49:10.012 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist ls --format=json 2022-06-29T15:49:10.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.122+0000 7feab3acc700 1 Processor -- start 2022-06-29T15:49:10.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.122+0000 7feab3acc700 1 -- start start 2022-06-29T15:49:10.132 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.122+0000 7feab3acc700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7feaac0a8ec0 0x7feaac1467f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:10.133 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.122+0000 7feab3acc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7feaac0b29c0 0x7feaac143400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:10.133 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.122+0000 7feab3acc700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaac060c80 con 0x7feaac0608a0 2022-06-29T15:49:10.133 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.122+0000 7feab3acc700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaac0b2dd0 con 0x7feaac0b29c0 2022-06-29T15:49:10.134 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.122+0000 7feab3acc700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaac0b0000 con 0x7feaac0a8ec0 2022-06-29T15:49:10.134 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.122+0000 7feab2069700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7feaac0b29c0 0x7feaac143400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:10.134 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.122+0000 7feab2069700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7feaac0b29c0 0x7feaac143400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50272/0 (socket says 172.21.15.47:50272) 2022-06-29T15:49:10.134 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.122+0000 7feab2069700 1 -- 172.21.15.47:0/3318628346 learned_addr learned my addr 172.21.15.47:0/3318628346 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:10.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.122+0000 7feab1067700 1 --2- 172.21.15.47:0/3318628346 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7feaac0a8ec0 0x7feaac1467f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:10.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.126+0000 7feab1067700 1 -- 172.21.15.47:0/3318628346 >> v2:172.21.15.47:3300/0 conn(0x7feaac0b29c0 msgr2=0x7feaac143400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:10.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.126+0000 7feab1067700 1 --2- 172.21.15.47:0/3318628346 >> v2:172.21.15.47:3300/0 conn(0x7feaac0b29c0 0x7feaac143400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:10.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.126+0000 7feab1067700 1 -- 172.21.15.47:0/3318628346 >> v1:172.21.15.64:6789/0 conn(0x7feaac0608a0 legacy=0x7feaac0ae250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:10.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.126+0000 7feab1067700 1 -- 172.21.15.47:0/3318628346 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaac0b0260 con 0x7feaac0a8ec0 2022-06-29T15:49:10.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.126+0000 7feab2069700 1 --2- 172.21.15.47:0/3318628346 >> v2:172.21.15.47:3300/0 conn(0x7feaac0b29c0 0x7feaac143400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:49:10.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.126+0000 7feab1067700 1 --2- 172.21.15.47:0/3318628346 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7feaac0a8ec0 0x7feaac1467f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7feaa8002f70 tx=0x7feaa8006df0).ready entity=mon.2 client_cookie=f4f4e928a21eec9b server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:10.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.126+0000 7feab0866700 1 -- 172.21.15.47:0/3318628346 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feaa800d570 con 0x7feaac0a8ec0 2022-06-29T15:49:10.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.126+0000 7feab0866700 1 -- 172.21.15.47:0/3318628346 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaa8000e90 con 0x7feaac0a8ec0 2022-06-29T15:49:10.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.126+0000 7feab0866700 1 -- 172.21.15.47:0/3318628346 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feaa800d900 con 0x7feaac0a8ec0 2022-06-29T15:49:10.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.126+0000 7feab3acc700 1 -- 172.21.15.47:0/3318628346 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7feaac0a8ec0 msgr2=0x7feaac1467f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:10.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.126+0000 7feab3acc700 1 --2- 172.21.15.47:0/3318628346 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7feaac0a8ec0 0x7feaac1467f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7feaa8002f70 tx=0x7feaa8006df0).stop 2022-06-29T15:49:10.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.126+0000 7feab3acc700 1 -- 172.21.15.47:0/3318628346 shutdown_connections 2022-06-29T15:49:10.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.126+0000 7feab3acc700 1 --2- 172.21.15.47:0/3318628346 >> v2:172.21.15.47:3300/0 conn(0x7feaac0b29c0 0x7feaac143400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:10.138 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.126+0000 7feab3acc700 1 --2- 172.21.15.47:0/3318628346 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7feaac0a8ec0 0x7feaac1467f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:10.138 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.126+0000 7feab3acc700 1 -- 172.21.15.47:0/3318628346 >> 172.21.15.47:0/3318628346 conn(0x7feaac0626f0 msgr2=0x7feaac062ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:10.139 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.126+0000 7feab3acc700 1 -- 172.21.15.47:0/3318628346 shutdown_connections 2022-06-29T15:49:10.139 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.126+0000 7feab3acc700 1 -- 172.21.15.47:0/3318628346 wait complete. 2022-06-29T15:49:10.143 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.126+0000 7feab3acc700 1 Processor -- start 2022-06-29T15:49:10.143 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.126+0000 7feab3acc700 1 -- start start 2022-06-29T15:49:10.143 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.130+0000 7feab3acc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feaac0608a0 0x7feaac13b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:10.144 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.130+0000 7feab3acc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feaac0a8ec0 0x7feaac156000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:10.144 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.130+0000 7feab3acc700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7feaac0b29c0 0x7feaac1564f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:10.144 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.130+0000 7feab3acc700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaac1402a0 con 0x7feaac0a8ec0 2022-06-29T15:49:10.144 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.130+0000 7feab3acc700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaac140040 con 0x7feaac0608a0 2022-06-29T15:49:10.144 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.130+0000 7feab3acc700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaac1569e0 con 0x7feaac0b29c0 2022-06-29T15:49:10.145 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.130+0000 7feab1067700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feaac0a8ec0 0x7feaac156000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:10.145 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.130+0000 7feab2069700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7feaac0b29c0 0x7feaac1564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:10.145 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.130+0000 7feab1067700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feaac0a8ec0 0x7feaac156000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50278/0 (socket says 172.21.15.47:50278) 2022-06-29T15:49:10.145 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.130+0000 7feab1868700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feaac0608a0 0x7feaac13b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:10.146 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.130+0000 7feab2069700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7feaac0b29c0 0x7feaac1564f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:47382/0 (socket says 172.21.15.47:47382) 2022-06-29T15:49:10.146 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.130+0000 7feab1067700 1 -- 172.21.15.47:0/195997138 learned_addr learned my addr 172.21.15.47:0/195997138 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:10.146 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.130+0000 7feab1868700 1 -- 172.21.15.47:0/195997138 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7feaac0b29c0 msgr2=0x7feaac1564f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:10.146 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.130+0000 7feab1868700 1 --2- 172.21.15.47:0/195997138 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7feaac0b29c0 0x7feaac1564f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:10.147 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.130+0000 7feab1868700 1 -- 172.21.15.47:0/195997138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feaac0a8ec0 msgr2=0x7feaac156000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:10.147 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.130+0000 7feab1868700 1 --2- 172.21.15.47:0/195997138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feaac0a8ec0 0x7feaac156000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:10.147 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.130+0000 7feab1868700 1 -- 172.21.15.47:0/195997138 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaa4009530 con 0x7feaac0608a0 2022-06-29T15:49:10.147 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.130+0000 7feab2069700 1 --2- 172.21.15.47:0/195997138 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7feaac0b29c0 0x7feaac1564f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:49:10.147 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.130+0000 7feab1067700 1 --2- 172.21.15.47:0/195997138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feaac0a8ec0 0x7feaac156000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:49:10.148 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.130+0000 7feab1868700 1 --2- 172.21.15.47:0/195997138 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feaac0608a0 0x7feaac13b900 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7feaac06afc0 tx=0x7fea9c007b90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:10.148 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.130+0000 7fea9affd700 1 -- 172.21.15.47:0/195997138 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea9c00ccd0 con 0x7feaac0608a0 2022-06-29T15:49:10.148 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.130+0000 7feab3acc700 1 -- 172.21.15.47:0/195997138 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaa800b180 con 0x7feaac0608a0 2022-06-29T15:49:10.148 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.130+0000 7fea9affd700 1 -- 172.21.15.47:0/195997138 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea9c00ce30 con 0x7feaac0608a0 2022-06-29T15:49:10.149 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.130+0000 7fea9affd700 1 -- 172.21.15.47:0/195997138 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea9c0125f0 con 0x7feaac0608a0 2022-06-29T15:49:10.149 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.130+0000 7feab3acc700 1 -- 172.21.15.47:0/195997138 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaac155b60 con 0x7feaac0608a0 2022-06-29T15:49:10.149 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.134+0000 7feab3acc700 1 -- 172.21.15.47:0/195997138 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaac155d20 con 0x7feaac0608a0 2022-06-29T15:49:10.150 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.134+0000 7fea9affd700 1 -- 172.21.15.47:0/195997138 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fea9c019030 con 0x7feaac0608a0 2022-06-29T15:49:10.150 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.134+0000 7fea9affd700 1 --2- 172.21.15.47:0/195997138 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fea8c053a10 0x7fea8c055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:10.150 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.134+0000 7fea9affd700 1 -- 172.21.15.47:0/195997138 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6463+0+0 (secure 0 0 0) 0x7fea9c06b290 con 0x7feaac0608a0 2022-06-29T15:49:10.150 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.134+0000 7feab1067700 1 --2- 172.21.15.47:0/195997138 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fea8c053a10 0x7fea8c055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:10.151 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.134+0000 7feab1067700 1 --2- 172.21.15.47:0/195997138 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fea8c053a10 0x7fea8c055ed0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7feaa801a000 tx=0x7feaa8014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:10.152 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.146+0000 7fea9affd700 1 -- 172.21.15.47:0/195997138 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fea9c041f90 con 0x7feaac0608a0 2022-06-29T15:49:10.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.286+0000 7feab3acc700 1 -- 172.21.15.47:0/195997138 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blacklist ls", "format": "json"} v 0) v1 -- 0x7feaac0573d0 con 0x7feaac0608a0 2022-06-29T15:49:10.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.286+0000 7fea9affd700 1 -- 172.21.15.47:0/195997138 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist ls", "format": "json"}]=0 listed 1 entries v79) v1 ==== 98+0+80 (secure 0 0 0) 0x7fea9c0455b0 con 0x7feaac0608a0 2022-06-29T15:49:10.292 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:49:10.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.290+0000 7feab3acc700 1 -- 172.21.15.47:0/195997138 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fea8c053a10 msgr2=0x7fea8c055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:10.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.290+0000 7feab3acc700 1 --2- 172.21.15.47:0/195997138 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fea8c053a10 0x7fea8c055ed0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7feaa801a000 tx=0x7feaa8014000).stop 2022-06-29T15:49:10.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.290+0000 7feab3acc700 1 -- 172.21.15.47:0/195997138 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feaac0608a0 msgr2=0x7feaac13b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:10.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.290+0000 7feab3acc700 1 --2- 172.21.15.47:0/195997138 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feaac0608a0 0x7feaac13b900 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7feaac06afc0 tx=0x7fea9c007b90).stop 2022-06-29T15:49:10.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.290+0000 7feab3acc700 1 -- 172.21.15.47:0/195997138 shutdown_connections 2022-06-29T15:49:10.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.290+0000 7feab3acc700 1 --2- 172.21.15.47:0/195997138 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7feaac0b29c0 0x7feaac1564f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:10.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.290+0000 7feab3acc700 1 --2- 172.21.15.47:0/195997138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feaac0a8ec0 0x7feaac156000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:10.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.290+0000 7feab3acc700 1 --2- 172.21.15.47:0/195997138 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fea8c053a10 0x7fea8c055ed0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:10.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.290+0000 7feab3acc700 1 --2- 172.21.15.47:0/195997138 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feaac0608a0 0x7feaac13b900 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:10.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.290+0000 7feab3acc700 1 -- 172.21.15.47:0/195997138 >> 172.21.15.47:0/195997138 conn(0x7feaac0626f0 msgr2=0x7feaac145cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:10.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.290+0000 7feab3acc700 1 -- 172.21.15.47:0/195997138 shutdown_connections 2022-06-29T15:49:10.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.290+0000 7feab3acc700 1 -- 172.21.15.47:0/195997138 wait complete. 2022-06-29T15:49:10.301 INFO:teuthology.orchestra.run.smithi047.stderr:listed 1 entries 2022-06-29T15:49:10.310 INFO:teuthology.orchestra.run.smithi047.stdout:[{"addr":"172.21.15.99:0/2790422795","until":"2022-06-29T16:49:03.956107+0000"}] 2022-06-29T15:49:10.326 INFO:tasks.cephfs.fuse_mount:Cleaning up killed ceph-fuse connection 2022-06-29T15:49:10.327 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:49:10.328 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:49:10.329 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:49:10.336 INFO:teuthology.orchestra.run.smithi099.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': Transport endpoint is not connected 2022-06-29T15:49:10.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T15:49:10.338 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.0 2022-06-29T15:49:10.339 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi099.front.sepia.ceph.com... 2022-06-29T15:49:10.339 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-06-29T15:49:10.340 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:49:10.397 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:49:10.397 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:49:10.398 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:49:10.405 INFO:teuthology.orchestra.run.smithi099.stdout:ext2/ext3 2022-06-29T15:49:10.406 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-06-29T15:49:10.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-29T15:49:10.407 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:49:10.415 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:49:10.526 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.518+0000 7fe1dbfff700 1 Processor -- start 2022-06-29T15:49:10.526 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.518+0000 7fe1dbfff700 1 -- start start 2022-06-29T15:49:10.529 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.522+0000 7fe1dbfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe1dc0b5c60 0x7fe1dc144980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:10.529 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.522+0000 7fe1dbfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe1dc0a72f0 0x7fe1dc0b2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:10.529 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.522+0000 7fe1dbfff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1dc0a9120 con 0x7fe1dc060eb0 2022-06-29T15:49:10.530 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.522+0000 7fe1dbfff700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1dc0b0b80 con 0x7fe1dc0b5c60 2022-06-29T15:49:10.530 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.522+0000 7fe1dbfff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1dc0a89e0 con 0x7fe1dc0a72f0 2022-06-29T15:49:10.530 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.522+0000 7fe1daffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe1dc0b5c60 0x7fe1dc144980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:10.530 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.522+0000 7fe1da7fc700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fe1dc060eb0 0x7fe1dc0b1e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:43402/0 (socket says 172.21.15.47:43402) 2022-06-29T15:49:10.531 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.522+0000 7fe1da7fc700 1 -- 172.21.15.47:0/2372047095 learned_addr learned my addr 172.21.15.47:0/2372047095 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:49:10.531 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.522+0000 7fe1db7fe700 1 --2- 172.21.15.47:0/2372047095 >> v2:172.21.15.64:3300/0 conn(0x7fe1dc0a72f0 0x7fe1dc0b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:10.531 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.522+0000 7fe1daffd700 1 -- 172.21.15.47:0/2372047095 >> v2:172.21.15.64:3300/0 conn(0x7fe1dc0a72f0 msgr2=0x7fe1dc0b2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:10.531 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.522+0000 7fe1d9ffb700 1 -- 172.21.15.47:0/2372047095 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3070736200 0 0) 0x7fe1dc0a9120 con 0x7fe1dc060eb0 2022-06-29T15:49:10.532 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.522+0000 7fe1daffd700 1 --2- 172.21.15.47:0/2372047095 >> v2:172.21.15.64:3300/0 conn(0x7fe1dc0a72f0 0x7fe1dc0b2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:10.532 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.522+0000 7fe1daffd700 1 -- 172.21.15.47:0/2372047095 >> v1:172.21.15.64:6789/0 conn(0x7fe1dc060eb0 legacy=0x7fe1dc0b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:10.532 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.522+0000 7fe1daffd700 1 -- 172.21.15.47:0/2372047095 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1dc0a8c80 con 0x7fe1dc0b5c60 2022-06-29T15:49:10.532 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.522+0000 7fe1daffd700 1 --2- 172.21.15.47:0/2372047095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1dc0b5c60 0x7fe1dc144980 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 rx=0x7fe1c4002f70 tx=0x7fe1c4006d90).ready entity=mon.0 client_cookie=f9f8502052cb9d70 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:10.533 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.522+0000 7fe1d9ffb700 1 -- 172.21.15.47:0/2372047095 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe1c400d4a0 con 0x7fe1dc0b5c60 2022-06-29T15:49:10.533 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.522+0000 7fe1d9ffb700 1 -- 172.21.15.47:0/2372047095 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1c4000e90 con 0x7fe1dc0b5c60 2022-06-29T15:49:10.533 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.522+0000 7fe1d9ffb700 1 -- 172.21.15.47:0/2372047095 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe1c400d7f0 con 0x7fe1dc0b5c60 2022-06-29T15:49:10.533 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.522+0000 7fe1dbfff700 1 -- 172.21.15.47:0/2372047095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1dc0b5c60 msgr2=0x7fe1dc144980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:10.533 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.522+0000 7fe1dbfff700 1 --2- 172.21.15.47:0/2372047095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1dc0b5c60 0x7fe1dc144980 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 rx=0x7fe1c4002f70 tx=0x7fe1c4006d90).stop 2022-06-29T15:49:10.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.522+0000 7fe1dbfff700 1 -- 172.21.15.47:0/2372047095 shutdown_connections 2022-06-29T15:49:10.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.522+0000 7fe1dbfff700 1 --2- 172.21.15.47:0/2372047095 >> v2:172.21.15.64:3300/0 conn(0x7fe1dc0a72f0 0x7fe1dc0b2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:10.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.522+0000 7fe1dbfff700 1 --2- 172.21.15.47:0/2372047095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1dc0b5c60 0x7fe1dc144980 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:10.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.522+0000 7fe1dbfff700 1 -- 172.21.15.47:0/2372047095 >> 172.21.15.47:0/2372047095 conn(0x7fe1dc062560 msgr2=0x7fe1dc062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:10.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.522+0000 7fe1dbfff700 1 -- 172.21.15.47:0/2372047095 shutdown_connections 2022-06-29T15:49:10.536 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.522+0000 7fe1dbfff700 1 -- 172.21.15.47:0/2372047095 wait complete. 2022-06-29T15:49:10.542 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.526+0000 7fe1dbfff700 1 Processor -- start 2022-06-29T15:49:10.543 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.526+0000 7fe1dbfff700 1 -- start start 2022-06-29T15:49:10.543 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.526+0000 7fe1dbfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1dc060eb0 0x7fe1dc143970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:10.543 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.526+0000 7fe1dbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1dc0a72f0 0x7fe1dc15a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:10.544 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.526+0000 7fe1dbfff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe1dc0b5c60 0x7fe1dc0acab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:10.544 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.526+0000 7fe1dbfff700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1dc13c710 con 0x7fe1dc060eb0 2022-06-29T15:49:10.544 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.526+0000 7fe1dbfff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1dc14c990 con 0x7fe1dc0a72f0 2022-06-29T15:49:10.544 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.526+0000 7fe1dbfff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1dc159df0 con 0x7fe1dc0b5c60 2022-06-29T15:49:10.544 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.526+0000 7fe1daffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1dc060eb0 0x7fe1dc143970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.526+0000 7fe1daffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1dc060eb0 0x7fe1dc143970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50290/0 (socket says 172.21.15.47:50290) 2022-06-29T15:49:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.526+0000 7fe1daffd700 1 -- 172.21.15.47:0/4178220763 learned_addr learned my addr 172.21.15.47:0/4178220763 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.526+0000 7fe1db7fe700 1 --2- 172.21.15.47:0/4178220763 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe1dc0b5c60 0x7fe1dc0acab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.526+0000 7fe1da7fc700 1 --2- 172.21.15.47:0/4178220763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1dc0a72f0 0x7fe1dc15a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:10.546 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.526+0000 7fe1daffd700 1 -- 172.21.15.47:0/4178220763 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe1dc0b5c60 msgr2=0x7fe1dc0acab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:10.546 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.526+0000 7fe1daffd700 1 --2- 172.21.15.47:0/4178220763 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe1dc0b5c60 0x7fe1dc0acab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:10.546 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.526+0000 7fe1daffd700 1 -- 172.21.15.47:0/4178220763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1dc0a72f0 msgr2=0x7fe1dc15a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:10.546 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.526+0000 7fe1daffd700 1 --2- 172.21.15.47:0/4178220763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1dc0a72f0 0x7fe1dc15a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:10.547 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.526+0000 7fe1daffd700 1 -- 172.21.15.47:0/4178220763 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c400b120 con 0x7fe1dc060eb0 2022-06-29T15:49:10.547 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.526+0000 7fe1da7fc700 1 --2- 172.21.15.47:0/4178220763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1dc0a72f0 0x7fe1dc15a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:49:10.547 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.526+0000 7fe1daffd700 1 --2- 172.21.15.47:0/4178220763 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1dc060eb0 0x7fe1dc143970 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 rx=0x7fe1c4002f70 tx=0x7fe1c4014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:10.547 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.526+0000 7fe1d3fff700 1 -- 172.21.15.47:0/4178220763 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe1c40116f0 con 0x7fe1dc060eb0 2022-06-29T15:49:10.547 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.526+0000 7fe1d3fff700 1 -- 172.21.15.47:0/4178220763 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1c4011850 con 0x7fe1dc060eb0 2022-06-29T15:49:10.548 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.526+0000 7fe1dbfff700 1 -- 172.21.15.47:0/4178220763 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1dc14cc70 con 0x7fe1dc060eb0 2022-06-29T15:49:10.548 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.526+0000 7fe1d3fff700 1 -- 172.21.15.47:0/4178220763 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe1c4011bb0 con 0x7fe1dc060eb0 2022-06-29T15:49:10.548 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.526+0000 7fe1dbfff700 1 -- 172.21.15.47:0/4178220763 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1dc14d1b0 con 0x7fe1dc060eb0 2022-06-29T15:49:10.548 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.530+0000 7fe1d3fff700 1 -- 172.21.15.47:0/4178220763 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe1c400f070 con 0x7fe1dc060eb0 2022-06-29T15:49:10.549 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.530+0000 7fe1dbfff700 1 -- 172.21.15.47:0/4178220763 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1dc0573d0 con 0x7fe1dc060eb0 2022-06-29T15:49:10.549 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.530+0000 7fe1d3fff700 1 --2- 172.21.15.47:0/4178220763 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe1c0053a70 0x7fe1c0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:10.549 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.530+0000 7fe1d3fff700 1 -- 172.21.15.47:0/4178220763 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6463+0+0 (secure 0 0 0) 0x7fe1c406d2a0 con 0x7fe1dc060eb0 2022-06-29T15:49:10.549 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.530+0000 7fe1da7fc700 1 --2- 172.21.15.47:0/4178220763 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe1c0053a70 0x7fe1c0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:10.550 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.530+0000 7fe1da7fc700 1 --2- 172.21.15.47:0/4178220763 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe1c0053a70 0x7fe1c0055f30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7fe1cc008000 tx=0x7fe1cc009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:10.550 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.534+0000 7fe1d3fff700 1 -- 172.21.15.47:0/4178220763 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe1c4043fe0 con 0x7fe1dc060eb0 2022-06-29T15:49:10.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.666+0000 7fe1dbfff700 1 -- 172.21.15.47:0/4178220763 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe1dc1442f0 con 0x7fe1dc060eb0 2022-06-29T15:49:10.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.666+0000 7fe1d3fff700 1 -- 172.21.15.47:0/4178220763 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+16335 (secure 0 0 0) 0x7fe1c40475c0 con 0x7fe1dc060eb0 2022-06-29T15:49:10.673 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:49:10.674 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":79,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:49:04.104902+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:48:16.753707+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2022-06-29T15:48:17.935585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{"172.21.15.99:0/2790422795":"2022-06-29T16:49:03.956107+0000"},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:49:10.677 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.670+0000 7fe1dbfff700 1 -- 172.21.15.47:0/4178220763 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe1c0053a70 msgr2=0x7fe1c0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:10.677 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.670+0000 7fe1dbfff700 1 --2- 172.21.15.47:0/4178220763 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe1c0053a70 0x7fe1c0055f30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7fe1cc008000 tx=0x7fe1cc009040).stop 2022-06-29T15:49:10.677 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.670+0000 7fe1dbfff700 1 -- 172.21.15.47:0/4178220763 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1dc060eb0 msgr2=0x7fe1dc143970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:10.678 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.670+0000 7fe1dbfff700 1 --2- 172.21.15.47:0/4178220763 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1dc060eb0 0x7fe1dc143970 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 rx=0x7fe1c4002f70 tx=0x7fe1c4014000).stop 2022-06-29T15:49:10.678 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.670+0000 7fe1dbfff700 1 -- 172.21.15.47:0/4178220763 shutdown_connections 2022-06-29T15:49:10.678 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.670+0000 7fe1dbfff700 1 --2- 172.21.15.47:0/4178220763 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe1dc0b5c60 0x7fe1dc0acab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:10.678 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.670+0000 7fe1dbfff700 1 --2- 172.21.15.47:0/4178220763 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe1c0053a70 0x7fe1c0055f30 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:10.679 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.670+0000 7fe1dbfff700 1 --2- 172.21.15.47:0/4178220763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1dc0a72f0 0x7fe1dc15a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:10.679 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.670+0000 7fe1dbfff700 1 --2- 172.21.15.47:0/4178220763 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1dc060eb0 0x7fe1dc143970 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:10.679 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.670+0000 7fe1dbfff700 1 -- 172.21.15.47:0/4178220763 >> 172.21.15.47:0/4178220763 conn(0x7fe1dc062560 msgr2=0x7fe1dc14de80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:10.680 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.670+0000 7fe1dbfff700 1 -- 172.21.15.47:0/4178220763 shutdown_connections 2022-06-29T15:49:10.680 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.670+0000 7fe1dbfff700 1 -- 172.21.15.47:0/4178220763 wait complete. 2022-06-29T15:49:10.686 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-29T15:42:16.971431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-29T15:42:25.290884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-29T15:48:16.753707+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2022-06-29T15:48:17.935585+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-29T15:49:10.688 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-29T15:49:10.798 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.790+0000 7fe3cc958700 1 Processor -- start 2022-06-29T15:49:10.798 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.790+0000 7fe3cc958700 1 -- start start 2022-06-29T15:49:10.800 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.794+0000 7fe3cc958700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe3c80a9d80 0x7fe3c805f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:10.801 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.794+0000 7fe3cc958700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe3c80b27c0 0x7fe3c8144c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:10.801 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.794+0000 7fe3cc958700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3c80add80 con 0x7fe3c805f0b0 2022-06-29T15:49:10.801 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.794+0000 7fe3cc958700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3c80ae020 con 0x7fe3c80b27c0 2022-06-29T15:49:10.802 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.794+0000 7fe3cc958700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c81444f0 con 0x7fe3c80a9d80 2022-06-29T15:49:10.802 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.794+0000 7fe3c7fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe3c80b27c0 0x7fe3c8144c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:10.802 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.794+0000 7fe3c7fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe3c80b27c0 0x7fe3c8144c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50302/0 (socket says 172.21.15.47:50302) 2022-06-29T15:49:10.802 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.794+0000 7fe3c7fff700 1 -- 172.21.15.47:0/979673546 learned_addr learned my addr 172.21.15.47:0/979673546 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:10.803 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.794+0000 7fe3c6ffd700 1 --2- 172.21.15.47:0/979673546 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe3c80a9d80 0x7fe3c805f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:10.803 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.794+0000 7fe3c7fff700 1 -- 172.21.15.47:0/979673546 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe3c80a9d80 msgr2=0x7fe3c805f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:10.803 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.794+0000 7fe3c7fff700 1 --2- 172.21.15.47:0/979673546 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe3c80a9d80 0x7fe3c805f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:10.803 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.794+0000 7fe3c7fff700 1 -- 172.21.15.47:0/979673546 >> v1:172.21.15.64:6789/0 conn(0x7fe3c805f0b0 legacy=0x7fe3c8140da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:10.804 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.794+0000 7fe3c7fff700 1 -- 172.21.15.47:0/979673546 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c8144790 con 0x7fe3c80b27c0 2022-06-29T15:49:10.804 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.794+0000 7fe3c7fff700 1 --2- 172.21.15.47:0/979673546 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3c80b27c0 0x7fe3c8144c30 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 rx=0x7fe3c00032f0 tx=0x7fe3c0009290).ready entity=mon.0 client_cookie=b917931f111c2f02 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:10.804 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.794+0000 7fe3c67fc700 1 -- 172.21.15.47:0/979673546 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe3c000b8a0 con 0x7fe3c80b27c0 2022-06-29T15:49:10.804 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.794+0000 7fe3c67fc700 1 -- 172.21.15.47:0/979673546 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3c000ba00 con 0x7fe3c80b27c0 2022-06-29T15:49:10.805 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.794+0000 7fe3c67fc700 1 -- 172.21.15.47:0/979673546 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe3c000bb70 con 0x7fe3c80b27c0 2022-06-29T15:49:10.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.794+0000 7fe3cc958700 1 -- 172.21.15.47:0/979673546 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3c80b27c0 msgr2=0x7fe3c8144c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:10.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.794+0000 7fe3cc958700 1 --2- 172.21.15.47:0/979673546 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3c80b27c0 0x7fe3c8144c30 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 rx=0x7fe3c00032f0 tx=0x7fe3c0009290).stop 2022-06-29T15:49:10.809 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.794+0000 7fe3cc958700 1 -- 172.21.15.47:0/979673546 shutdown_connections 2022-06-29T15:49:10.809 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.794+0000 7fe3cc958700 1 --2- 172.21.15.47:0/979673546 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3c80b27c0 0x7fe3c8144c30 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:10.809 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.794+0000 7fe3cc958700 1 --2- 172.21.15.47:0/979673546 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe3c80a9d80 0x7fe3c805f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:10.809 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.794+0000 7fe3cc958700 1 -- 172.21.15.47:0/979673546 >> 172.21.15.47:0/979673546 conn(0x7fe3c80b6430 msgr2=0x7fe3c80b6810 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:10.810 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.794+0000 7fe3cc958700 1 -- 172.21.15.47:0/979673546 shutdown_connections 2022-06-29T15:49:10.810 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.794+0000 7fe3cc958700 1 -- 172.21.15.47:0/979673546 wait complete. 2022-06-29T15:49:10.810 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.798+0000 7fe3cc958700 1 Processor -- start 2022-06-29T15:49:10.810 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.798+0000 7fe3cc958700 1 -- start start 2022-06-29T15:49:10.811 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.798+0000 7fe3cc958700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3c805f0b0 0x7fe3c813f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:10.811 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.798+0000 7fe3cc958700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3c80a9d80 0x7fe3c813c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:10.811 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.798+0000 7fe3cc958700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe3c80b27c0 0x7fe3c8153bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:10.811 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.798+0000 7fe3cc958700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c813fd80 con 0x7fe3c805f0b0 2022-06-29T15:49:10.812 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.798+0000 7fe3cc958700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c8063890 con 0x7fe3c80a9d80 2022-06-29T15:49:10.812 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.798+0000 7fe3cc958700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c813f190 con 0x7fe3c80b27c0 2022-06-29T15:49:10.812 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.798+0000 7fe3c77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3c805f0b0 0x7fe3c813f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:10.812 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.798+0000 7fe3c77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3c805f0b0 0x7fe3c813f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50304/0 (socket says 172.21.15.47:50304) 2022-06-29T15:49:10.813 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.798+0000 7fe3c77fe700 1 -- 172.21.15.47:0/1846986398 learned_addr learned my addr 172.21.15.47:0/1846986398 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:10.813 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.798+0000 7fe3c6ffd700 1 --2- 172.21.15.47:0/1846986398 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3c80a9d80 0x7fe3c813c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:10.813 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.798+0000 7fe3c7fff700 1 --2- 172.21.15.47:0/1846986398 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe3c80b27c0 0x7fe3c8153bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:10.813 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.798+0000 7fe3c77fe700 1 -- 172.21.15.47:0/1846986398 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe3c80b27c0 msgr2=0x7fe3c8153bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:10.813 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.798+0000 7fe3c77fe700 1 --2- 172.21.15.47:0/1846986398 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe3c80b27c0 0x7fe3c8153bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:10.814 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.798+0000 7fe3c77fe700 1 -- 172.21.15.47:0/1846986398 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3c80a9d80 msgr2=0x7fe3c813c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:10.814 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.798+0000 7fe3c77fe700 1 --2- 172.21.15.47:0/1846986398 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3c80a9d80 0x7fe3c813c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:10.814 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.798+0000 7fe3c77fe700 1 -- 172.21.15.47:0/1846986398 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c0009530 con 0x7fe3c805f0b0 2022-06-29T15:49:10.814 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.798+0000 7fe3c77fe700 1 --2- 172.21.15.47:0/1846986398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3c805f0b0 0x7fe3c813f630 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 rx=0x7fe3c805d4b0 tx=0x7fe3b8007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:10.815 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.798+0000 7fe3c4ff9700 1 -- 172.21.15.47:0/1846986398 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe3b800ce30 con 0x7fe3c805f0b0 2022-06-29T15:49:10.815 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.798+0000 7fe3c4ff9700 1 -- 172.21.15.47:0/1846986398 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3b800f470 con 0x7fe3c805f0b0 2022-06-29T15:49:10.815 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.798+0000 7fe3cc958700 1 -- 172.21.15.47:0/1846986398 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3c813ec90 con 0x7fe3c805f0b0 2022-06-29T15:49:10.815 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.798+0000 7fe3c4ff9700 1 -- 172.21.15.47:0/1846986398 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe3b800f7d0 con 0x7fe3c805f0b0 2022-06-29T15:49:10.816 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.798+0000 7fe3cc958700 1 -- 172.21.15.47:0/1846986398 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3c8153720 con 0x7fe3c805f0b0 2022-06-29T15:49:10.816 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.798+0000 7fe3cc958700 1 -- 172.21.15.47:0/1846986398 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3c8150250 con 0x7fe3c805f0b0 2022-06-29T15:49:10.816 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.802+0000 7fe3c4ff9700 1 -- 172.21.15.47:0/1846986398 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe3b8010070 con 0x7fe3c805f0b0 2022-06-29T15:49:10.816 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.802+0000 7fe3c4ff9700 1 --2- 172.21.15.47:0/1846986398 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe3ac053b30 0x7fe3ac055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:10.816 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.802+0000 7fe3c4ff9700 1 -- 172.21.15.47:0/1846986398 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6463+0+0 (secure 0 0 0) 0x7fe3b806b150 con 0x7fe3c805f0b0 2022-06-29T15:49:10.817 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.802+0000 7fe3c6ffd700 1 --2- 172.21.15.47:0/1846986398 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe3ac053b30 0x7fe3ac055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:10.817 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.802+0000 7fe3c6ffd700 1 --2- 172.21.15.47:0/1846986398 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe3ac053b30 0x7fe3ac055ff0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7fe3b000b000 tx=0x7fe3b0008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:10.818 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.806+0000 7fe3c4ff9700 1 -- 172.21.15.47:0/1846986398 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe3b8041e50 con 0x7fe3c805f0b0 2022-06-29T15:49:10.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.930+0000 7fe3cc958700 1 -- 172.21.15.47:0/1846986398 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fe3c80573d0 con 0x7fe3c805f0b0 2022-06-29T15:49:10.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.930+0000 7fe3c4ff9700 1 -- 172.21.15.47:0/1846986398 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v79) v1 ==== 110+0+10 (secure 0 0 0) 0x7fe3b8045470 con 0x7fe3c805f0b0 2022-06-29T15:49:10.937 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2022-06-29T15:49:10.939 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.934+0000 7fe3cc958700 1 -- 172.21.15.47:0/1846986398 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe3ac053b30 msgr2=0x7fe3ac055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:10.940 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.934+0000 7fe3cc958700 1 --2- 172.21.15.47:0/1846986398 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe3ac053b30 0x7fe3ac055ff0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7fe3b000b000 tx=0x7fe3b0008040).stop 2022-06-29T15:49:10.940 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.934+0000 7fe3cc958700 1 -- 172.21.15.47:0/1846986398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3c805f0b0 msgr2=0x7fe3c813f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:10.940 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.934+0000 7fe3cc958700 1 --2- 172.21.15.47:0/1846986398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3c805f0b0 0x7fe3c813f630 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 rx=0x7fe3c805d4b0 tx=0x7fe3b8007c50).stop 2022-06-29T15:49:10.941 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.934+0000 7fe3cc958700 1 -- 172.21.15.47:0/1846986398 shutdown_connections 2022-06-29T15:49:10.941 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.934+0000 7fe3cc958700 1 --2- 172.21.15.47:0/1846986398 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe3c80b27c0 0x7fe3c8153bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:10.941 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.934+0000 7fe3cc958700 1 --2- 172.21.15.47:0/1846986398 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe3ac053b30 0x7fe3ac055ff0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:10.941 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.934+0000 7fe3cc958700 1 --2- 172.21.15.47:0/1846986398 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3c80a9d80 0x7fe3c813c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:10.942 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.934+0000 7fe3cc958700 1 --2- 172.21.15.47:0/1846986398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3c805f0b0 0x7fe3c813f630 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:10.942 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.934+0000 7fe3cc958700 1 -- 172.21.15.47:0/1846986398 >> 172.21.15.47:0/1846986398 conn(0x7fe3c80b6430 msgr2=0x7fe3c813e290 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:10.942 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.934+0000 7fe3cc958700 1 -- 172.21.15.47:0/1846986398 shutdown_connections 2022-06-29T15:49:10.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:10.934+0000 7fe3cc958700 1 -- 172.21.15.47:0/1846986398 wait complete. 2022-06-29T15:49:10.950 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-29T15:49:11.057 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.050+0000 7f8e35bcd700 1 Processor -- start 2022-06-29T15:49:11.058 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.050+0000 7f8e35bcd700 1 -- start start 2022-06-29T15:49:11.060 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.054+0000 7f8e35bcd700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8e300b2930 0x7f8e300b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:11.060 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.054+0000 7f8e35bcd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8e30095e30 0x7f8e30098160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:11.061 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.054+0000 7f8e35bcd700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e300b1ac0 con 0x7f8e300a9770 2022-06-29T15:49:11.061 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.054+0000 7f8e35bcd700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e30095c70 con 0x7f8e30095e30 2022-06-29T15:49:11.061 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.054+0000 7f8e35bcd700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e300965b0 con 0x7f8e300b2930 2022-06-29T15:49:11.061 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.054+0000 7f8e2ffff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8e30095e30 0x7f8e30098160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:11.062 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.054+0000 7f8e2ffff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8e30095e30 0x7f8e30098160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50316/0 (socket says 172.21.15.47:50316) 2022-06-29T15:49:11.062 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.054+0000 7f8e2ffff700 1 -- 172.21.15.47:0/3595841219 learned_addr learned my addr 172.21.15.47:0/3595841219 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:11.062 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.054+0000 7f8e2f7fe700 1 --2- 172.21.15.47:0/3595841219 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8e300b2930 0x7f8e300b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:11.063 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.054+0000 7f8e2ffff700 1 -- 172.21.15.47:0/3595841219 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8e300b2930 msgr2=0x7f8e300b2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:11.063 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.054+0000 7f8e2ffff700 1 --2- 172.21.15.47:0/3595841219 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8e300b2930 0x7f8e300b2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:11.063 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.054+0000 7f8e2ffff700 1 -- 172.21.15.47:0/3595841219 >> v1:172.21.15.64:6789/0 conn(0x7f8e300a9770 legacy=0x7f8e30097a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:11.063 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.054+0000 7f8e2ffff700 1 -- 172.21.15.47:0/3595841219 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e300b1cf0 con 0x7f8e30095e30 2022-06-29T15:49:11.064 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.054+0000 7f8e2ffff700 1 --2- 172.21.15.47:0/3595841219 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e30095e30 0x7f8e30098160 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 rx=0x7f8e280032f0 tx=0x7f8e28009290).ready entity=mon.0 client_cookie=ffada46cc35926e8 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:11.064 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.054+0000 7f8e2e7fc700 1 -- 172.21.15.47:0/3595841219 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e2800b8a0 con 0x7f8e30095e30 2022-06-29T15:49:11.064 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.054+0000 7f8e2e7fc700 1 -- 172.21.15.47:0/3595841219 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e2800ba00 con 0x7f8e30095e30 2022-06-29T15:49:11.064 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.054+0000 7f8e2e7fc700 1 -- 172.21.15.47:0/3595841219 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e2800bb70 con 0x7f8e30095e30 2022-06-29T15:49:11.068 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.054+0000 7f8e35bcd700 1 -- 172.21.15.47:0/3595841219 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e30095e30 msgr2=0x7f8e30098160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:11.068 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.054+0000 7f8e35bcd700 1 --2- 172.21.15.47:0/3595841219 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e30095e30 0x7f8e30098160 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 rx=0x7f8e280032f0 tx=0x7f8e28009290).stop 2022-06-29T15:49:11.069 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.054+0000 7f8e35bcd700 1 -- 172.21.15.47:0/3595841219 shutdown_connections 2022-06-29T15:49:11.069 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.054+0000 7f8e35bcd700 1 --2- 172.21.15.47:0/3595841219 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e30095e30 0x7f8e30098160 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:11.069 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.054+0000 7f8e35bcd700 1 --2- 172.21.15.47:0/3595841219 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8e300b2930 0x7f8e300b2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:11.069 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.054+0000 7f8e35bcd700 1 -- 172.21.15.47:0/3595841219 >> 172.21.15.47:0/3595841219 conn(0x7f8e3005f0b0 msgr2=0x7f8e3005f490 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:11.070 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.054+0000 7f8e35bcd700 1 -- 172.21.15.47:0/3595841219 shutdown_connections 2022-06-29T15:49:11.070 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.054+0000 7f8e35bcd700 1 -- 172.21.15.47:0/3595841219 wait complete. 2022-06-29T15:49:11.070 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.054+0000 7f8e35bcd700 1 Processor -- start 2022-06-29T15:49:11.070 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.054+0000 7f8e35bcd700 1 -- start start 2022-06-29T15:49:11.071 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.058+0000 7f8e35bcd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e30095e30 0x7f8e30148c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:11.071 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.058+0000 7f8e35bcd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e300a9770 0x7f8e3009b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:11.071 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.058+0000 7f8e35bcd700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8e300b2930 0x7f8e3009b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:11.071 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.058+0000 7f8e35bcd700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3009a2c0 con 0x7f8e30095e30 2022-06-29T15:49:11.072 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.058+0000 7f8e35bcd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e300af5d0 con 0x7f8e300a9770 2022-06-29T15:49:11.072 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.058+0000 7f8e35bcd700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3009a020 con 0x7f8e300b2930 2022-06-29T15:49:11.072 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.058+0000 7f8e2f7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e30095e30 0x7f8e30148c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:11.072 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.058+0000 7f8e2f7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e30095e30 0x7f8e30148c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50318/0 (socket says 172.21.15.47:50318) 2022-06-29T15:49:11.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.058+0000 7f8e2f7fe700 1 -- 172.21.15.47:0/1240945050 learned_addr learned my addr 172.21.15.47:0/1240945050 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:11.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.058+0000 7f8e2ffff700 1 --2- 172.21.15.47:0/1240945050 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8e300b2930 0x7f8e3009b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:11.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.058+0000 7f8e2effd700 1 --2- 172.21.15.47:0/1240945050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e300a9770 0x7f8e3009b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:11.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.058+0000 7f8e2f7fe700 1 -- 172.21.15.47:0/1240945050 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8e300b2930 msgr2=0x7f8e3009b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:11.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.058+0000 7f8e2f7fe700 1 --2- 172.21.15.47:0/1240945050 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8e300b2930 0x7f8e3009b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:11.074 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.058+0000 7f8e2f7fe700 1 -- 172.21.15.47:0/1240945050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e300a9770 msgr2=0x7f8e3009b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:11.074 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.058+0000 7f8e2f7fe700 1 --2- 172.21.15.47:0/1240945050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e300a9770 0x7f8e3009b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:11.074 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.058+0000 7f8e2f7fe700 1 -- 172.21.15.47:0/1240945050 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e28009530 con 0x7f8e30095e30 2022-06-29T15:49:11.075 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.058+0000 7f8e2f7fe700 1 --2- 172.21.15.47:0/1240945050 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e30095e30 0x7f8e30148c60 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 rx=0x7f8e2000d000 tx=0x7f8e20008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:11.075 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.058+0000 7f8e2cff9700 1 -- 172.21.15.47:0/1240945050 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e20010820 con 0x7f8e30095e30 2022-06-29T15:49:11.075 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.058+0000 7f8e2cff9700 1 -- 172.21.15.47:0/1240945050 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e20010980 con 0x7f8e30095e30 2022-06-29T15:49:11.075 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.058+0000 7f8e35bcd700 1 -- 172.21.15.47:0/1240945050 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e300ad250 con 0x7f8e30095e30 2022-06-29T15:49:11.076 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.058+0000 7f8e2cff9700 1 -- 172.21.15.47:0/1240945050 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e20010ce0 con 0x7f8e30095e30 2022-06-29T15:49:11.076 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.058+0000 7f8e35bcd700 1 -- 172.21.15.47:0/1240945050 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e300af0e0 con 0x7f8e30095e30 2022-06-29T15:49:11.076 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.058+0000 7f8e35bcd700 1 -- 172.21.15.47:0/1240945050 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e300af770 con 0x7f8e30095e30 2022-06-29T15:49:11.076 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.062+0000 7f8e2cff9700 1 -- 172.21.15.47:0/1240945050 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8e20021070 con 0x7f8e30095e30 2022-06-29T15:49:11.077 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.062+0000 7f8e2cff9700 1 --2- 172.21.15.47:0/1240945050 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8e10053950 0x7f8e10055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:11.077 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.062+0000 7f8e2cff9700 1 -- 172.21.15.47:0/1240945050 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6463+0+0 (secure 0 0 0) 0x7f8e2006c180 con 0x7f8e30095e30 2022-06-29T15:49:11.077 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.062+0000 7f8e2effd700 1 --2- 172.21.15.47:0/1240945050 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8e10053950 0x7f8e10055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:11.077 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.062+0000 7f8e2effd700 1 --2- 172.21.15.47:0/1240945050 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8e10053950 0x7f8e10055e10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f8e2400d000 tx=0x7f8e24000bc0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:11.078 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.066+0000 7f8e2cff9700 1 -- 172.21.15.47:0/1240945050 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8e20042ec0 con 0x7f8e30095e30 2022-06-29T15:49:11.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.194+0000 7f8e35bcd700 1 -- 172.21.15.47:0/1240945050 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8e300573d0 con 0x7f8e30095e30 2022-06-29T15:49:11.200 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.194+0000 7f8e2cff9700 1 -- 172.21.15.47:0/1240945050 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v79) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8e2001f0a0 con 0x7f8e30095e30 2022-06-29T15:49:11.200 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:49:11.203 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.194+0000 7f8e35bcd700 1 -- 172.21.15.47:0/1240945050 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8e10053950 msgr2=0x7f8e10055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:11.203 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.194+0000 7f8e35bcd700 1 --2- 172.21.15.47:0/1240945050 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8e10053950 0x7f8e10055e10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f8e2400d000 tx=0x7f8e24000bc0).stop 2022-06-29T15:49:11.204 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.194+0000 7f8e35bcd700 1 -- 172.21.15.47:0/1240945050 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e30095e30 msgr2=0x7f8e30148c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:11.204 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.194+0000 7f8e35bcd700 1 --2- 172.21.15.47:0/1240945050 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e30095e30 0x7f8e30148c60 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 rx=0x7f8e2000d000 tx=0x7f8e20008040).stop 2022-06-29T15:49:11.204 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.194+0000 7f8e35bcd700 1 -- 172.21.15.47:0/1240945050 shutdown_connections 2022-06-29T15:49:11.204 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.194+0000 7f8e35bcd700 1 --2- 172.21.15.47:0/1240945050 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8e300b2930 0x7f8e3009b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:11.205 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.194+0000 7f8e35bcd700 1 --2- 172.21.15.47:0/1240945050 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8e10053950 0x7f8e10055e10 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:11.205 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.194+0000 7f8e35bcd700 1 --2- 172.21.15.47:0/1240945050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e300a9770 0x7f8e3009b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:11.205 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.194+0000 7f8e35bcd700 1 --2- 172.21.15.47:0/1240945050 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e30095e30 0x7f8e30148c60 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:11.205 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.194+0000 7f8e35bcd700 1 -- 172.21.15.47:0/1240945050 >> 172.21.15.47:0/1240945050 conn(0x7f8e3005f0b0 msgr2=0x7f8e3005c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:11.206 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.194+0000 7f8e35bcd700 1 -- 172.21.15.47:0/1240945050 shutdown_connections 2022-06-29T15:49:11.206 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.194+0000 7f8e35bcd700 1 -- 172.21.15.47:0/1240945050 wait complete. 2022-06-29T15:49:11.213 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-29T15:49:11.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.314+0000 7f22bc81d700 1 Processor -- start 2022-06-29T15:49:11.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.314+0000 7f22bc81d700 1 -- start start 2022-06-29T15:49:11.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.314+0000 7f22bc81d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f22b80b1320 0x7f22b80b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:11.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.314+0000 7f22bc81d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f22b8141fd0 0x7f22b80b34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:11.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.314+0000 7f22bc81d700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22b805f7d0 con 0x7f22b805f3f0 2022-06-29T15:49:11.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.314+0000 7f22bc81d700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22b80b27a0 con 0x7f22b80b1320 2022-06-29T15:49:11.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.318+0000 7f22bc81d700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22b80b4b90 con 0x7f22b8141fd0 2022-06-29T15:49:11.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.318+0000 7f22b6ffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f22b80b1320 0x7f22b80b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:11.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.318+0000 7f22b7fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f22b8141fd0 0x7f22b80b34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:11.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.318+0000 7f22b7fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f22b8141fd0 0x7f22b80b34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:47432/0 (socket says 172.21.15.47:47432) 2022-06-29T15:49:11.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.318+0000 7f22b77fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f22b805f3f0 0x7f22b81418c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:43444/0 (socket says 172.21.15.47:43444) 2022-06-29T15:49:11.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.318+0000 7f22b6ffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f22b80b1320 0x7f22b80b2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:56032/0 (socket says 172.21.15.47:56032) 2022-06-29T15:49:11.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.318+0000 7f22b7fff700 1 -- 172.21.15.47:0/4172699372 learned_addr learned my addr 172.21.15.47:0/4172699372 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:11.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.318+0000 7f22b6ffd700 1 -- 172.21.15.47:0/4172699372 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f22b8141fd0 msgr2=0x7f22b80b34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:11.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.318+0000 7f22b6ffd700 1 --2- 172.21.15.47:0/4172699372 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f22b8141fd0 0x7f22b80b34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:11.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.318+0000 7f22b6ffd700 1 -- 172.21.15.47:0/4172699372 >> v1:172.21.15.64:6789/0 conn(0x7f22b805f3f0 legacy=0x7f22b81418c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:11.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.318+0000 7f22b6ffd700 1 -- 172.21.15.47:0/4172699372 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b80b2060 con 0x7f22b80b1320 2022-06-29T15:49:11.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.318+0000 7f22b67fc700 1 -- 172.21.15.47:0/4172699372 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3199076278 0 0) 0x7f22b805f7d0 con 0x7f22b805f3f0 2022-06-29T15:49:11.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.318+0000 7f22b7fff700 1 --2- 172.21.15.47:0/4172699372 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f22b8141fd0 0x7f22b80b34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:49:11.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.318+0000 7f22b6ffd700 1 --2- 172.21.15.47:0/4172699372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22b80b1320 0x7f22b80b2ff0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7f22a80032f0 tx=0x7f22a80091d0).ready entity=mon.1 client_cookie=e96649f44f6cfe13 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:11.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.318+0000 7f22b67fc700 1 -- 172.21.15.47:0/4172699372 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f22a800b6c0 con 0x7f22b80b1320 2022-06-29T15:49:11.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.318+0000 7f22b67fc700 1 -- 172.21.15.47:0/4172699372 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22a800b820 con 0x7f22b80b1320 2022-06-29T15:49:11.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.318+0000 7f22b67fc700 1 -- 172.21.15.47:0/4172699372 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f22a800b990 con 0x7f22b80b1320 2022-06-29T15:49:11.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.318+0000 7f22bc81d700 1 -- 172.21.15.47:0/4172699372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22b80b1320 msgr2=0x7f22b80b2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:11.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.318+0000 7f22bc81d700 1 --2- 172.21.15.47:0/4172699372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22b80b1320 0x7f22b80b2ff0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7f22a80032f0 tx=0x7f22a80091d0).stop 2022-06-29T15:49:11.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.318+0000 7f22bc81d700 1 -- 172.21.15.47:0/4172699372 shutdown_connections 2022-06-29T15:49:11.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.318+0000 7f22bc81d700 1 --2- 172.21.15.47:0/4172699372 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f22b8141fd0 0x7f22b80b34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:11.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.318+0000 7f22bc81d700 1 --2- 172.21.15.47:0/4172699372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22b80b1320 0x7f22b80b2ff0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:11.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.318+0000 7f22bc81d700 1 -- 172.21.15.47:0/4172699372 >> 172.21.15.47:0/4172699372 conn(0x7f22b8062570 msgr2=0x7f22b80647d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:11.335 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.318+0000 7f22bc81d700 1 -- 172.21.15.47:0/4172699372 shutdown_connections 2022-06-29T15:49:11.335 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.318+0000 7f22bc81d700 1 -- 172.21.15.47:0/4172699372 wait complete. 2022-06-29T15:49:11.335 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.322+0000 7f22bc81d700 1 Processor -- start 2022-06-29T15:49:11.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.322+0000 7f22bc81d700 1 -- start start 2022-06-29T15:49:11.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.322+0000 7f22bc81d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22b805f3f0 0x7f22b8146b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:11.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.322+0000 7f22bc81d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f22b80b1320 0x7f22b80ad9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:11.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.322+0000 7f22bc81d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22b8141fd0 0x7f22b80aded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:11.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.322+0000 7f22bc81d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b8061c50 con 0x7f22b8141fd0 2022-06-29T15:49:11.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.322+0000 7f22bc81d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b80ab990 con 0x7f22b805f3f0 2022-06-29T15:49:11.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.322+0000 7f22bc81d700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22b8061e80 con 0x7f22b80b1320 2022-06-29T15:49:11.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.322+0000 7f22b7fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22b8141fd0 0x7f22b80aded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:11.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.322+0000 7f22b7fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22b8141fd0 0x7f22b80aded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50336/0 (socket says 172.21.15.47:50336) 2022-06-29T15:49:11.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.322+0000 7f22b7fff700 1 -- 172.21.15.47:0/196310648 learned_addr learned my addr 172.21.15.47:0/196310648 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:11.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.322+0000 7f22b77fe700 1 --2- 172.21.15.47:0/196310648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22b805f3f0 0x7f22b8146b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:11.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.322+0000 7f22b6ffd700 1 --2- 172.21.15.47:0/196310648 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f22b80b1320 0x7f22b80ad9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:11.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.322+0000 7f22b7fff700 1 -- 172.21.15.47:0/196310648 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f22b80b1320 msgr2=0x7f22b80ad9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:11.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.322+0000 7f22b7fff700 1 --2- 172.21.15.47:0/196310648 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f22b80b1320 0x7f22b80ad9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:11.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.322+0000 7f22b7fff700 1 -- 172.21.15.47:0/196310648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22b805f3f0 msgr2=0x7f22b8146b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:11.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.322+0000 7f22b7fff700 1 --2- 172.21.15.47:0/196310648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22b805f3f0 0x7f22b8146b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:11.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.322+0000 7f22b7fff700 1 -- 172.21.15.47:0/196310648 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b000b1e0 con 0x7f22b8141fd0 2022-06-29T15:49:11.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.322+0000 7f22b7fff700 1 --2- 172.21.15.47:0/196310648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22b8141fd0 0x7f22b80aded0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 rx=0x7f22b000c000 tx=0x7f22b0006eb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:11.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.322+0000 7f22b4ff9700 1 -- 172.21.15.47:0/196310648 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f22b0012600 con 0x7f22b8141fd0 2022-06-29T15:49:11.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.322+0000 7f22b4ff9700 1 -- 172.21.15.47:0/196310648 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22b0012760 con 0x7f22b8141fd0 2022-06-29T15:49:11.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.322+0000 7f22bc81d700 1 -- 172.21.15.47:0/196310648 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22a8009470 con 0x7f22b8141fd0 2022-06-29T15:49:11.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.322+0000 7f22b4ff9700 1 -- 172.21.15.47:0/196310648 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f22b0012ac0 con 0x7f22b8141fd0 2022-06-29T15:49:11.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.322+0000 7f22bc81d700 1 -- 172.21.15.47:0/196310648 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22b81559a0 con 0x7f22b8141fd0 2022-06-29T15:49:11.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.326+0000 7f22bc81d700 1 -- 172.21.15.47:0/196310648 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22b80573d0 con 0x7f22b8141fd0 2022-06-29T15:49:11.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.326+0000 7f22b4ff9700 1 -- 172.21.15.47:0/196310648 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f22b0022070 con 0x7f22b8141fd0 2022-06-29T15:49:11.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.326+0000 7f22b4ff9700 1 --2- 172.21.15.47:0/196310648 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2298053a70 0x7f2298055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:11.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.326+0000 7f22b4ff9700 1 -- 172.21.15.47:0/196310648 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6463+0+0 (secure 0 0 0) 0x7f22b006c4d0 con 0x7f22b8141fd0 2022-06-29T15:49:11.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.326+0000 7f22b77fe700 1 --2- 172.21.15.47:0/196310648 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2298053a70 0x7f2298055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:11.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.326+0000 7f22b77fe700 1 --2- 172.21.15.47:0/196310648 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2298053a70 0x7f2298055f30 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7f22ac00d000 tx=0x7f22ac007a00).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:11.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.334+0000 7f22b4ff9700 1 -- 172.21.15.47:0/196310648 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f22b0043280 con 0x7f22b8141fd0 2022-06-29T15:49:11.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.462+0000 7f22bc81d700 1 -- 172.21.15.47:0/196310648 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f22b8068830 con 0x7f22b8141fd0 2022-06-29T15:49:11.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.462+0000 7f22b4ff9700 1 -- 172.21.15.47:0/196310648 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v79) v1 ==== 104+0+10 (secure 0 0 0) 0x7f22b0016020 con 0x7f22b8141fd0 2022-06-29T15:49:11.469 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:49:11.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.462+0000 7f22bc81d700 1 -- 172.21.15.47:0/196310648 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2298053a70 msgr2=0x7f2298055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:11.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.462+0000 7f22bc81d700 1 --2- 172.21.15.47:0/196310648 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2298053a70 0x7f2298055f30 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7f22ac00d000 tx=0x7f22ac007a00).stop 2022-06-29T15:49:11.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.462+0000 7f22bc81d700 1 -- 172.21.15.47:0/196310648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22b8141fd0 msgr2=0x7f22b80aded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:11.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.462+0000 7f22bc81d700 1 --2- 172.21.15.47:0/196310648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22b8141fd0 0x7f22b80aded0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 rx=0x7f22b000c000 tx=0x7f22b0006eb0).stop 2022-06-29T15:49:11.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.462+0000 7f22bc81d700 1 -- 172.21.15.47:0/196310648 shutdown_connections 2022-06-29T15:49:11.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.462+0000 7f22bc81d700 1 --2- 172.21.15.47:0/196310648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22b8141fd0 0x7f22b80aded0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:11.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.462+0000 7f22bc81d700 1 --2- 172.21.15.47:0/196310648 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f22b80b1320 0x7f22b80ad9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:11.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.462+0000 7f22bc81d700 1 --2- 172.21.15.47:0/196310648 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2298053a70 0x7f2298055f30 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:11.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.462+0000 7f22bc81d700 1 --2- 172.21.15.47:0/196310648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22b805f3f0 0x7f22b8146b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:11.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.462+0000 7f22bc81d700 1 -- 172.21.15.47:0/196310648 >> 172.21.15.47:0/196310648 conn(0x7f22b8062570 msgr2=0x7f22b805cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:11.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.466+0000 7f22bc81d700 1 -- 172.21.15.47:0/196310648 shutdown_connections 2022-06-29T15:49:11.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.466+0000 7f22bc81d700 1 -- 172.21.15.47:0/196310648 wait complete. 2022-06-29T15:49:11.481 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-29T15:49:11.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.586+0000 7fc04759e700 1 Processor -- start 2022-06-29T15:49:11.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.586+0000 7fc04759e700 1 -- start start 2022-06-29T15:49:11.594 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.586+0000 7fc04759e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc0480b58d0 0x7fc0480b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:11.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.586+0000 7fc04759e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc048060790 con 0x7fc04814b4c0 2022-06-29T15:49:11.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.586+0000 7fc04759e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc04814ad80 con 0x7fc0480b1610 2022-06-29T15:49:11.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.586+0000 7fc04759e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0480a99e0 con 0x7fc0480b58d0 2022-06-29T15:49:11.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.586+0000 7fc046d9d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc04814b4c0 0x7fc0480aa130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35898/0 (socket says 172.21.15.47:35898) 2022-06-29T15:49:11.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.586+0000 7fc046d9d700 1 -- 172.21.15.47:0/38323555 learned_addr learned my addr 172.21.15.47:0/38323555 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:49:11.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.586+0000 7fc04659c700 1 --2- 172.21.15.47:0/38323555 >> v2:172.21.15.64:3300/0 conn(0x7fc0480b58d0 0x7fc0480b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:11.597 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.586+0000 7fc04559a700 1 -- 172.21.15.47:0/38323555 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1257060110 0 0) 0x7fc048060790 con 0x7fc04814b4c0 2022-06-29T15:49:11.597 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.586+0000 7fc04559a700 1 -- 172.21.15.47:0/38323555 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc02c003260 con 0x7fc04814b4c0 2022-06-29T15:49:11.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.586+0000 7fc04659c700 1 -- 172.21.15.47:0/38323555 >> v1:172.21.15.64:6789/0 conn(0x7fc0480b1610 legacy=0x7fc0481475f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:11.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.586+0000 7fc04659c700 1 -- 172.21.15.47:0/38323555 >> v1:172.21.15.47:6789/0 conn(0x7fc04814b4c0 legacy=0x7fc0480aa130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:11.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.586+0000 7fc04659c700 1 -- 172.21.15.47:0/38323555 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc04814b020 con 0x7fc0480b58d0 2022-06-29T15:49:11.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.586+0000 7fc04659c700 1 --2- 172.21.15.47:0/38323555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0480b58d0 0x7fc0480b5cb0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7fc038002f70 tx=0x7fc038006df0).ready entity=mon.1 client_cookie=9f4a45cf8d340d10 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:11.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.586+0000 7fc04559a700 1 -- 172.21.15.47:0/38323555 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc03800d610 con 0x7fc0480b58d0 2022-06-29T15:49:11.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.586+0000 7fc04559a700 1 -- 172.21.15.47:0/38323555 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc038000e90 con 0x7fc0480b58d0 2022-06-29T15:49:11.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.586+0000 7fc04559a700 1 -- 172.21.15.47:0/38323555 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc03800d960 con 0x7fc0480b58d0 2022-06-29T15:49:11.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.586+0000 7fc04759e700 1 -- 172.21.15.47:0/38323555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0480b58d0 msgr2=0x7fc0480b5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:11.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.586+0000 7fc04759e700 1 --2- 172.21.15.47:0/38323555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0480b58d0 0x7fc0480b5cb0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7fc038002f70 tx=0x7fc038006df0).stop 2022-06-29T15:49:11.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.586+0000 7fc04759e700 1 -- 172.21.15.47:0/38323555 shutdown_connections 2022-06-29T15:49:11.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.586+0000 7fc04759e700 1 --2- 172.21.15.47:0/38323555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0480b58d0 0x7fc0480b5cb0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:11.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.586+0000 7fc04759e700 1 -- 172.21.15.47:0/38323555 >> 172.21.15.47:0/38323555 conn(0x7fc048062560 msgr2=0x7fc048062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:11.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.586+0000 7fc04759e700 1 -- 172.21.15.47:0/38323555 shutdown_connections 2022-06-29T15:49:11.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.586+0000 7fc04759e700 1 -- 172.21.15.47:0/38323555 wait complete. 2022-06-29T15:49:11.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.590+0000 7fc04759e700 1 Processor -- start 2022-06-29T15:49:11.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.590+0000 7fc04759e700 1 -- start start 2022-06-29T15:49:11.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.590+0000 7fc04759e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc0480b1610 0x7fc048066620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:11.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.590+0000 7fc04759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0480b58d0 0x7fc048156c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:11.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.590+0000 7fc04759e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc04814b4c0 0x7fc048157180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:11.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.590+0000 7fc04759e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0481568a0 con 0x7fc04814b4c0 2022-06-29T15:49:11.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.590+0000 7fc04759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc048064390 con 0x7fc0480b58d0 2022-06-29T15:49:11.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.590+0000 7fc04759e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc048065540 con 0x7fc0480b1610 2022-06-29T15:49:11.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.590+0000 7fc046d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc04814b4c0 0x7fc048157180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:11.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.590+0000 7fc046d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc04814b4c0 0x7fc048157180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50350/0 (socket says 172.21.15.47:50350) 2022-06-29T15:49:11.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.590+0000 7fc046d9d700 1 -- 172.21.15.47:0/1170291761 learned_addr learned my addr 172.21.15.47:0/1170291761 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:11.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.590+0000 7fc04659c700 1 --2- 172.21.15.47:0/1170291761 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc0480b1610 0x7fc048066620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:11.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.590+0000 7fc045d9b700 1 --2- 172.21.15.47:0/1170291761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0480b58d0 0x7fc048156c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:11.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.590+0000 7fc046d9d700 1 -- 172.21.15.47:0/1170291761 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc0480b1610 msgr2=0x7fc048066620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:11.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.590+0000 7fc046d9d700 1 --2- 172.21.15.47:0/1170291761 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc0480b1610 0x7fc048066620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:11.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.590+0000 7fc046d9d700 1 -- 172.21.15.47:0/1170291761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0480b58d0 msgr2=0x7fc048156c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:11.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.590+0000 7fc046d9d700 1 --2- 172.21.15.47:0/1170291761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0480b58d0 0x7fc048156c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:11.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.590+0000 7fc046d9d700 1 -- 172.21.15.47:0/1170291761 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc03800b180 con 0x7fc04814b4c0 2022-06-29T15:49:11.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.590+0000 7fc04659c700 1 --2- 172.21.15.47:0/1170291761 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc0480b1610 0x7fc048066620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:49:11.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.590+0000 7fc046d9d700 1 --2- 172.21.15.47:0/1170291761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc04814b4c0 0x7fc048157180 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 rx=0x7fc040001490 tx=0x7fc040007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:11.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.590+0000 7fc0377fe700 1 -- 172.21.15.47:0/1170291761 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc040005ba0 con 0x7fc04814b4c0 2022-06-29T15:49:11.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.590+0000 7fc0377fe700 1 -- 172.21.15.47:0/1170291761 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc040005d00 con 0x7fc04814b4c0 2022-06-29T15:49:11.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.590+0000 7fc04759e700 1 -- 172.21.15.47:0/1170291761 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc048065770 con 0x7fc04814b4c0 2022-06-29T15:49:11.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.590+0000 7fc0377fe700 1 -- 172.21.15.47:0/1170291761 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc0400072c0 con 0x7fc04814b4c0 2022-06-29T15:49:11.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.590+0000 7fc04759e700 1 -- 172.21.15.47:0/1170291761 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc048064ca0 con 0x7fc04814b4c0 2022-06-29T15:49:11.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.594+0000 7fc04759e700 1 -- 172.21.15.47:0/1170291761 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0480573d0 con 0x7fc04814b4c0 2022-06-29T15:49:11.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.594+0000 7fc0377fe700 1 -- 172.21.15.47:0/1170291761 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc04001a060 con 0x7fc04814b4c0 2022-06-29T15:49:11.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.594+0000 7fc0377fe700 1 --2- 172.21.15.47:0/1170291761 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc0240539e0 0x7fc024055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:11.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.594+0000 7fc0377fe700 1 -- 172.21.15.47:0/1170291761 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6463+0+0 (secure 0 0 0) 0x7fc04006ba80 con 0x7fc04814b4c0 2022-06-29T15:49:11.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.594+0000 7fc04659c700 1 --2- 172.21.15.47:0/1170291761 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc0240539e0 0x7fc024055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:11.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.594+0000 7fc04659c700 1 --2- 172.21.15.47:0/1170291761 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc0240539e0 0x7fc024055ea0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7fc038018000 tx=0x7fc038014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:11.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.598+0000 7fc0377fe700 1 -- 172.21.15.47:0/1170291761 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc040042820 con 0x7fc04814b4c0 2022-06-29T15:49:11.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.722+0000 7fc04759e700 1 -- 172.21.15.47:0/1170291761 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc048069e20 con 0x7fc04814b4c0 2022-06-29T15:49:11.731 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.722+0000 7fc0377fe700 1 -- 172.21.15.47:0/1170291761 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v79) v1 ==== 100+0+10 (secure 0 0 0) 0x7fc040018070 con 0x7fc04814b4c0 2022-06-29T15:49:11.731 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:49:11.734 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.726+0000 7fc04759e700 1 -- 172.21.15.47:0/1170291761 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc0240539e0 msgr2=0x7fc024055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:11.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.726+0000 7fc04759e700 1 --2- 172.21.15.47:0/1170291761 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc0240539e0 0x7fc024055ea0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7fc038018000 tx=0x7fc038014000).stop 2022-06-29T15:49:11.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.726+0000 7fc04759e700 1 -- 172.21.15.47:0/1170291761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc04814b4c0 msgr2=0x7fc048157180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:11.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.726+0000 7fc04759e700 1 --2- 172.21.15.47:0/1170291761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc04814b4c0 0x7fc048157180 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 rx=0x7fc040001490 tx=0x7fc040007e00).stop 2022-06-29T15:49:11.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.726+0000 7fc04759e700 1 -- 172.21.15.47:0/1170291761 shutdown_connections 2022-06-29T15:49:11.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.726+0000 7fc04759e700 1 --2- 172.21.15.47:0/1170291761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc04814b4c0 0x7fc048157180 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:11.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.726+0000 7fc04759e700 1 --2- 172.21.15.47:0/1170291761 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc0240539e0 0x7fc024055ea0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:11.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.726+0000 7fc04759e700 1 --2- 172.21.15.47:0/1170291761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0480b58d0 0x7fc048156c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:11.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.726+0000 7fc04759e700 1 --2- 172.21.15.47:0/1170291761 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc0480b1610 0x7fc048066620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:11.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.726+0000 7fc04759e700 1 -- 172.21.15.47:0/1170291761 >> 172.21.15.47:0/1170291761 conn(0x7fc048062560 msgr2=0x7fc0481494a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:11.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.726+0000 7fc04759e700 1 -- 172.21.15.47:0/1170291761 shutdown_connections 2022-06-29T15:49:11.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.726+0000 7fc04759e700 1 -- 172.21.15.47:0/1170291761 wait complete. 2022-06-29T15:49:11.747 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-29T15:49:11.748 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:49:11.848 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.842+0000 7f768535b700 1 Processor -- start 2022-06-29T15:49:11.849 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.842+0000 7f768535b700 1 -- start start 2022-06-29T15:49:11.851 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.842+0000 7f768535b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f76800afdc0 0x7f76800ae7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:11.852 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.842+0000 7f768535b700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7680143d20 con 0x7f76800b02d0 2022-06-29T15:49:11.852 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.842+0000 7f768535b700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76800a8df0 con 0x7f76800b5ac0 2022-06-29T15:49:11.852 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.842+0000 7f768535b700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76800a9090 con 0x7f76800afdc0 2022-06-29T15:49:11.853 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.842+0000 7f7684b5a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f76800afdc0 0x7f76800ae7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:11.853 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.842+0000 7f7684b5a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f76800afdc0 0x7f76800ae7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50358/0 (socket says 172.21.15.47:50358) 2022-06-29T15:49:11.854 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.842+0000 7f7684b5a700 1 -- 172.21.15.47:0/870203155 learned_addr learned my addr 172.21.15.47:0/870203155 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:11.854 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.842+0000 7f767f7fe700 1 --1- 172.21.15.47:0/870203155 >> v1:172.21.15.47:6789/0 conn(0x7f76800b02d0 0x7f76800ae0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35910/0 (socket says 172.21.15.47:35910) 2022-06-29T15:49:11.855 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.842+0000 7f7684b5a700 1 -- 172.21.15.47:0/870203155 >> v1:172.21.15.64:6789/0 conn(0x7f76800b5ac0 legacy=0x7f76801401c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:11.855 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.842+0000 7f7684b5a700 1 -- 172.21.15.47:0/870203155 >> v1:172.21.15.47:6789/0 conn(0x7f76800b02d0 legacy=0x7f76800ae0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:11.856 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.842+0000 7f7684b5a700 1 -- 172.21.15.47:0/870203155 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7680143f80 con 0x7f76800afdc0 2022-06-29T15:49:11.856 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.842+0000 7f7684b5a700 1 --2- 172.21.15.47:0/870203155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76800afdc0 0x7f76800ae7d0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 rx=0x7f7678002f70 tx=0x7f7678006eb0).ready entity=mon.0 client_cookie=fe71aeec39636cff server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:11.856 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.846+0000 7f767effd700 1 -- 172.21.15.47:0/870203155 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f767800d660 con 0x7f76800afdc0 2022-06-29T15:49:11.857 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.846+0000 7f767effd700 1 -- 172.21.15.47:0/870203155 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7678000e90 con 0x7f76800afdc0 2022-06-29T15:49:11.857 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.846+0000 7f767effd700 1 -- 172.21.15.47:0/870203155 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f767800d9f0 con 0x7f76800afdc0 2022-06-29T15:49:11.858 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.846+0000 7f768535b700 1 -- 172.21.15.47:0/870203155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76800afdc0 msgr2=0x7f76800ae7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:11.858 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.846+0000 7f768535b700 1 --2- 172.21.15.47:0/870203155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76800afdc0 0x7f76800ae7d0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 rx=0x7f7678002f70 tx=0x7f7678006eb0).stop 2022-06-29T15:49:11.858 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.846+0000 7f768535b700 1 -- 172.21.15.47:0/870203155 shutdown_connections 2022-06-29T15:49:11.859 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.846+0000 7f768535b700 1 --2- 172.21.15.47:0/870203155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76800afdc0 0x7f76800ae7d0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:11.859 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.846+0000 7f768535b700 1 -- 172.21.15.47:0/870203155 >> 172.21.15.47:0/870203155 conn(0x7f7680062560 msgr2=0x7f7680062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:11.859 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.846+0000 7f768535b700 1 -- 172.21.15.47:0/870203155 shutdown_connections 2022-06-29T15:49:11.860 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.846+0000 7f768535b700 1 -- 172.21.15.47:0/870203155 wait complete. 2022-06-29T15:49:11.863 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.846+0000 7f768535b700 1 Processor -- start 2022-06-29T15:49:11.863 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.846+0000 7f768535b700 1 -- start start 2022-06-29T15:49:11.864 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.846+0000 7f768535b700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f76800afdc0 0x7f76800ae270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:11.864 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.846+0000 7f768535b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76800b02d0 0x7f76800a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:11.864 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.846+0000 7f768535b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76800b5ac0 0x7f76800a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:11.864 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.846+0000 7f768535b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76800aae80 con 0x7f76800b02d0 2022-06-29T15:49:11.865 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.846+0000 7f768535b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f768009ebb0 con 0x7f76800b5ac0 2022-06-29T15:49:11.865 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.846+0000 7f768535b700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76800aabe0 con 0x7f76800afdc0 2022-06-29T15:49:11.865 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.850+0000 7f767f7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76800b02d0 0x7f76800a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:11.865 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.850+0000 7f767f7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76800b02d0 0x7f76800a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50362/0 (socket says 172.21.15.47:50362) 2022-06-29T15:49:11.866 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.850+0000 7f767f7fe700 1 -- 172.21.15.47:0/2237878695 learned_addr learned my addr 172.21.15.47:0/2237878695 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:11.866 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.850+0000 7f7684b5a700 1 --2- 172.21.15.47:0/2237878695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76800b5ac0 0x7f76800a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:11.866 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.850+0000 7f767ffff700 1 --2- 172.21.15.47:0/2237878695 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f76800afdc0 0x7f76800ae270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:11.866 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.850+0000 7f767f7fe700 1 -- 172.21.15.47:0/2237878695 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f76800afdc0 msgr2=0x7f76800ae270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:11.867 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.850+0000 7f767f7fe700 1 --2- 172.21.15.47:0/2237878695 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f76800afdc0 0x7f76800ae270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:11.867 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.850+0000 7f767f7fe700 1 -- 172.21.15.47:0/2237878695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76800b5ac0 msgr2=0x7f76800a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:11.867 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.850+0000 7f767f7fe700 1 --2- 172.21.15.47:0/2237878695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76800b5ac0 0x7f76800a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:11.867 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.850+0000 7f767f7fe700 1 -- 172.21.15.47:0/2237878695 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f767800b1e0 con 0x7f76800b02d0 2022-06-29T15:49:11.868 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.850+0000 7f767f7fe700 1 --2- 172.21.15.47:0/2237878695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76800b02d0 0x7f76800a06a0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 rx=0x7f7680143150 tx=0x7f7668007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:11.868 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.850+0000 7f767d7fa700 1 -- 172.21.15.47:0/2237878695 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76680070d0 con 0x7f76800b02d0 2022-06-29T15:49:11.868 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.850+0000 7f767d7fa700 1 -- 172.21.15.47:0/2237878695 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7668007230 con 0x7f76800b02d0 2022-06-29T15:49:11.868 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.850+0000 7f768535b700 1 -- 172.21.15.47:0/2237878695 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f768009ee10 con 0x7f76800b02d0 2022-06-29T15:49:11.868 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.850+0000 7f767d7fa700 1 -- 172.21.15.47:0/2237878695 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7668005550 con 0x7f76800b02d0 2022-06-29T15:49:11.869 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.850+0000 7f768535b700 1 -- 172.21.15.47:0/2237878695 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76800a1080 con 0x7f76800b02d0 2022-06-29T15:49:11.869 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.850+0000 7f768535b700 1 -- 172.21.15.47:0/2237878695 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f768009efa0 con 0x7f76800b02d0 2022-06-29T15:49:11.869 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.854+0000 7f767d7fa700 1 -- 172.21.15.47:0/2237878695 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7668010070 con 0x7f76800b02d0 2022-06-29T15:49:11.869 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.854+0000 7f767d7fa700 1 --2- 172.21.15.47:0/2237878695 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7664053a70 0x7f7664055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:11.870 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.854+0000 7f767d7fa700 1 -- 172.21.15.47:0/2237878695 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6463+0+0 (secure 0 0 0) 0x7f766806b600 con 0x7f76800b02d0 2022-06-29T15:49:11.870 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.854+0000 7f767ffff700 1 --2- 172.21.15.47:0/2237878695 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7664053a70 0x7f7664055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:11.870 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.854+0000 7f767ffff700 1 --2- 172.21.15.47:0/2237878695 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7664053a70 0x7f7664055f30 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7f767000d000 tx=0x7f7670007f60).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:11.871 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:11.858+0000 7f767d7fa700 1 -- 172.21.15.47:0/2237878695 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7668042290 con 0x7f76800b02d0 2022-06-29T15:49:12.025 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:12.018+0000 7f768535b700 1 -- 172.21.15.47:0/2237878695 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f76800573d0 con 0x7f76800b02d0 2022-06-29T15:49:12.026 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:12.018+0000 7f767d7fa700 1 -- 172.21.15.47:0/2237878695 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+2493 (secure 0 0 0) 0x7f76680458b0 con 0x7f76800b02d0 2022-06-29T15:49:12.026 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:49:12.026 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":35,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6291,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.47:6835/2910037081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2910037081},{"type":"v1","addr":"172.21.15.47:6835","nonce":2910037081}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":34},{"gid":6300,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/359532891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":359532891},{"type":"v1","addr":"172.21.15.47:6837","nonce":359532891}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":33}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:48:18.345845+0000","modified":"2022-06-29T15:48:19.349718+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":55562},"failed":[],"damaged":[],"stopped":[],"info":{"gid_55562":{"gid":55562,"name":"b","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6833/3871453337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3871453337},{"type":"v1","addr":"172.21.15.64:6833","nonce":3871453337}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2022-06-29T15:49:12.028 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:12.022+0000 7f768535b700 1 -- 172.21.15.47:0/2237878695 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7664053a70 msgr2=0x7f7664055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:12.028 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:12.022+0000 7f768535b700 1 --2- 172.21.15.47:0/2237878695 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7664053a70 0x7f7664055f30 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7f767000d000 tx=0x7f7670007f60).stop 2022-06-29T15:49:12.028 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:12.022+0000 7f768535b700 1 -- 172.21.15.47:0/2237878695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76800b02d0 msgr2=0x7f76800a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:12.029 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:12.022+0000 7f768535b700 1 --2- 172.21.15.47:0/2237878695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76800b02d0 0x7f76800a06a0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 rx=0x7f7680143150 tx=0x7f7668007c10).stop 2022-06-29T15:49:12.029 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:12.022+0000 7f768535b700 1 -- 172.21.15.47:0/2237878695 shutdown_connections 2022-06-29T15:49:12.029 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:12.022+0000 7f768535b700 1 --2- 172.21.15.47:0/2237878695 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7664053a70 0x7f7664055f30 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:12.029 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:12.022+0000 7f768535b700 1 --2- 172.21.15.47:0/2237878695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76800b5ac0 0x7f76800a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:12.030 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:12.022+0000 7f768535b700 1 --2- 172.21.15.47:0/2237878695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76800b02d0 0x7f76800a06a0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:12.030 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:12.022+0000 7f768535b700 1 --2- 172.21.15.47:0/2237878695 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f76800afdc0 0x7f76800ae270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:12.030 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:12.022+0000 7f768535b700 1 -- 172.21.15.47:0/2237878695 >> 172.21.15.47:0/2237878695 conn(0x7f7680062560 msgr2=0x7f76801429c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:12.031 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:12.022+0000 7f768535b700 1 -- 172.21.15.47:0/2237878695 shutdown_connections 2022-06-29T15:49:12.031 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:12.022+0000 7f768535b700 1 -- 172.21.15.47:0/2237878695 wait complete. 2022-06-29T15:49:12.032 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 35 2022-06-29T15:49:12.189 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:49:12.192 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:49:12.194 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:49:12.198 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:49:12.342 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 35, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-29T15:48:18.345845+0000', 'modified': '2022-06-29T15:48:19.349718+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 55562}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_55562': {'gid': 55562, 'name': 'b', 'rank': 0, 'incarnation': 34, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.64:6833/3871453337', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6832', 'nonce': 3871453337}, {'type': 'v1', 'addr': '172.21.15.64:6833', 'nonce': 3871453337}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-29T15:49:12.397 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-29T15:49:12.398 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-06-29T15:49:12.517 INFO:teuthology.orchestra.run.smithi064.stdout:{"cluster_fsid":"9a68894a-53ce-400e-838b-14eab91b834c","whoami":0,"id":55562,"want_state":"up:active","state":"up:active","rank_uptime":54.150415781,"mdsmap_epoch":35,"osdmap_epoch":79,"osdmap_epoch_barrier":79,"uptime":58.554578692} 2022-06-29T15:49:12.518 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9a68894a-53ce-400e-838b-14eab91b834c", "id": 55562, "mdsmap_epoch": 35, "osdmap_epoch": 79, "osdmap_epoch_barrier": 79, "rank_uptime": 54.150415781, "state": "up:active", "uptime": 58.554578692, "want_state": "up:active", "whoami": 0 } 2022-06-29T15:49:12.518 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-29T15:49:12.519 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-06-29T15:49:12.520 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi099.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-06-29T15:49:12.521 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:49:12.521 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:49:12.531 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2022-06-29T15:49:12.589 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:49:12.590 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:49:12.601 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:49:12.602 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:49:12.604 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:49:12.605 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2022-06-29T15:49:12.611 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-29T15:49:12.612 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:49:12.658 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2022-06-29T15:49:12.678 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:49:12.678 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:49:12.689 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:49:12.690 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:49:12.691 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:49:12.692 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2022-06-29T15:49:12.732 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:2022-06-29T15:49:12.727+0000 7f093e98e080 -1 init, newargv = 0x55fe5bb34460 newargc=9ceph-fuse[12931]: starting ceph client 2022-06-29T15:49:12.733 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr: 2022-06-29T15:49:12.743 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[12931]: starting fuse 2022-06-29T15:49:13.838 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2022-06-29T15:49:13.857 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:49:13.860 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:49:13.872 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:49:13.873 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:49:13.874 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:49:13.875 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2022-06-29T15:49:13.923 INFO:teuthology.orchestra.run.smithi099.stdout:52 2022-06-29T15:49:13.924 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-29T15:49:13.924 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:49:13.925 DEBUG:teuthology.orchestra.run.smithi099:> sudo python3 -c ' 2022-06-29T15:49:13.926 DEBUG:teuthology.orchestra.run.smithi099:> import glob 2022-06-29T15:49:13.927 DEBUG:teuthology.orchestra.run.smithi099:> import re 2022-06-29T15:49:13.927 DEBUG:teuthology.orchestra.run.smithi099:> import os 2022-06-29T15:49:13.928 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2022-06-29T15:49:13.928 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:49:13.929 DEBUG:teuthology.orchestra.run.smithi099:> def find_socket(client_name): 2022-06-29T15:49:13.929 DEBUG:teuthology.orchestra.run.smithi099:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-06-29T15:49:13.929 DEBUG:teuthology.orchestra.run.smithi099:> files = glob.glob(asok_path) 2022-06-29T15:49:13.930 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:49:13.930 DEBUG:teuthology.orchestra.run.smithi099:> # Given a non-glob path, it better be there 2022-06-29T15:49:13.931 DEBUG:teuthology.orchestra.run.smithi099:> if "*" not in asok_path: 2022-06-29T15:49:13.931 DEBUG:teuthology.orchestra.run.smithi099:> assert(len(files) == 1) 2022-06-29T15:49:13.932 DEBUG:teuthology.orchestra.run.smithi099:> return files[0] 2022-06-29T15:49:13.932 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:49:13.932 DEBUG:teuthology.orchestra.run.smithi099:> for f in files: 2022-06-29T15:49:13.933 DEBUG:teuthology.orchestra.run.smithi099:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-29T15:49:13.933 DEBUG:teuthology.orchestra.run.smithi099:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-29T15:49:13.934 DEBUG:teuthology.orchestra.run.smithi099:> return f 2022-06-29T15:49:13.934 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-29T15:49:13.934 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:49:13.935 DEBUG:teuthology.orchestra.run.smithi099:> print(find_socket("client.0")) 2022-06-29T15:49:13.935 DEBUG:teuthology.orchestra.run.smithi099:> ' 2022-06-29T15:49:13.979 INFO:teuthology.orchestra.run.smithi099.stdout:/var/run/ceph/ceph-client.0.12931.asok 2022-06-29T15:49:13.985 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.12931.asok 2022-06-29T15:49:13.985 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:49:13.986 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.12931.asok status 2022-06-29T15:49:14.074 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2022-06-29T15:49:14.075 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": { 2022-06-29T15:49:14.075 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_sha1": "f25ffac72e5a9e63860ff7deb6767a11f0559a16", 2022-06-29T15:49:14.075 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 15.2.16-304-gf25ffac7 (f25ffac72e5a9e63860ff7deb6767a11f0559a16) octopus (stable)", 2022-06-29T15:49:14.075 INFO:teuthology.orchestra.run.smithi099.stdout: "entity_id": "0", 2022-06-29T15:49:14.075 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": "smithi099", 2022-06-29T15:49:14.076 INFO:teuthology.orchestra.run.smithi099.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-06-29T15:49:14.076 INFO:teuthology.orchestra.run.smithi099.stdout: "pid": "12931", 2022-06-29T15:49:14.076 INFO:teuthology.orchestra.run.smithi099.stdout: "root": "/" 2022-06-29T15:49:14.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:49:14.077 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_count": 0, 2022-06-29T15:49:14.077 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_pinned_count": 0, 2022-06-29T15:49:14.077 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 6457, 2022-06-29T15:49:14.077 INFO:teuthology.orchestra.run.smithi099.stdout: "inst": { 2022-06-29T15:49:14.077 INFO:teuthology.orchestra.run.smithi099.stdout: "name": { 2022-06-29T15:49:14.078 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "client", 2022-06-29T15:49:14.078 INFO:teuthology.orchestra.run.smithi099.stdout: "num": 6457 2022-06-29T15:49:14.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:49:14.078 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2022-06-29T15:49:14.079 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2022-06-29T15:49:14.079 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:0", 2022-06-29T15:49:14.079 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3683028961 2022-06-29T15:49:14.079 INFO:teuthology.orchestra.run.smithi099.stdout: } 2022-06-29T15:49:14.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:49:14.080 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2022-06-29T15:49:14.080 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2022-06-29T15:49:14.080 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:0", 2022-06-29T15:49:14.080 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3683028961 2022-06-29T15:49:14.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:49:14.081 INFO:teuthology.orchestra.run.smithi099.stdout: "inst_str": "client.6457 172.21.15.99:0/3683028961", 2022-06-29T15:49:14.081 INFO:teuthology.orchestra.run.smithi099.stdout: "addr_str": "172.21.15.99:0/3683028961", 2022-06-29T15:49:14.081 INFO:teuthology.orchestra.run.smithi099.stdout: "inode_count": 1, 2022-06-29T15:49:14.081 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_epoch": 35, 2022-06-29T15:49:14.082 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch": 79, 2022-06-29T15:49:14.082 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch_barrier": 0, 2022-06-29T15:49:14.082 INFO:teuthology.orchestra.run.smithi099.stdout: "blacklisted": false, 2022-06-29T15:49:14.082 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs" 2022-06-29T15:49:14.083 INFO:teuthology.orchestra.run.smithi099.stdout:} 2022-06-29T15:49:14.085 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:49:14.085 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:49:14.086 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:49:14.093 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2022-06-29T15:49:14.095 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-29T15:49:14.096 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:49:14.097 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:49:14.159 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-06-29T15:49:14.160 DEBUG:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:49:14.179 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:49:14.203 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:49:14.224 DEBUG:teuthology.orchestra.run.smithi164:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:49:14.246 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-06-29T15:49:14.247 INFO:tasks.cephfs.mount:Terminating background process 2022-06-29T15:49:14.247 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:49:14.248 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:49:14.248 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:49:14.256 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2022-06-29T15:49:14.257 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-29T15:49:14.258 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi099.front.sepia.ceph.com... 2022-06-29T15:49:14.258 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-06-29T15:49:14.259 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:49:14.315 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[12931]: fuse finished with error 0 and tester_r 0 2022-06-29T15:49:14.317 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:49:14.317 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:49:14.318 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:49:14.323 INFO:teuthology.orchestra.run.smithi099.stdout:ext2/ext3 2022-06-29T15:49:14.325 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-06-29T15:49:14.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-29T15:49:14.443 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:49:14.492 INFO:tasks.cephfs.mount:Terminating background process 2022-06-29T15:49:14.492 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:49:14.493 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:49:14.493 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:49:14.502 INFO:teuthology.orchestra.run.smithi164.stdout:fuseblk 2022-06-29T15:49:14.503 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-06-29T15:49:14.504 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi164.front.sepia.ceph.com... 2022-06-29T15:49:14.504 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-06-29T15:49:14.505 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:49:14.520 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi164.stderr:ceph-fuse[12464]: fuse finished with error 0 and tester_r 0 2022-06-29T15:49:14.523 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:49:14.524 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:49:14.524 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:49:14.532 INFO:teuthology.orchestra.run.smithi164.stdout:ext2/ext3 2022-06-29T15:49:14.534 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-06-29T15:49:14.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-29T15:49:14.653 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:49:14.661 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-06-29T15:49:14.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.766+0000 7f47dede2700 1 Processor -- start 2022-06-29T15:49:14.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.766+0000 7f47dede2700 1 -- start start 2022-06-29T15:49:14.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.770+0000 7f47dede2700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47d80b7860 0x7f47d80af160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:14.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.770+0000 7f47dede2700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f47d8146890 0x7f47d80af650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:14.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.770+0000 7f47dede2700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47d80b3900 con 0x7f47d80b2d00 2022-06-29T15:49:14.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.770+0000 7f47dede2700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47d81477c0 con 0x7f47d8146890 2022-06-29T15:49:14.776 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.770+0000 7f47dede2700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47d80b7c40 con 0x7f47d80b7860 2022-06-29T15:49:14.776 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.770+0000 7f47dd5df700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f47d8146890 0x7f47d80af650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:14.776 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.770+0000 7f47ddde0700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47d80b7860 0x7f47d80af160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:14.776 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.770+0000 7f47dd5df700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f47d8146890 0x7f47d80af650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:56072/0 (socket says 172.21.15.47:56072) 2022-06-29T15:49:14.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.770+0000 7f47dd5df700 1 -- 172.21.15.47:0/3018525145 learned_addr learned my addr 172.21.15.47:0/3018525145 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:14.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.770+0000 7f47dd5df700 1 -- 172.21.15.47:0/3018525145 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47d80b7860 msgr2=0x7f47d80af160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:14.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.770+0000 7f47dd5df700 1 --2- 172.21.15.47:0/3018525145 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47d80b7860 0x7f47d80af160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:14.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.770+0000 7f47dd5df700 1 -- 172.21.15.47:0/3018525145 >> v1:172.21.15.64:6789/0 conn(0x7f47d80b2d00 legacy=0x7f47d80b0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:14.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.770+0000 7f47dd5df700 1 -- 172.21.15.47:0/3018525145 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d80b3660 con 0x7f47d8146890 2022-06-29T15:49:14.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.770+0000 7f47ddde0700 1 --2- 172.21.15.47:0/3018525145 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47d80b7860 0x7f47d80af160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:49:14.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.770+0000 7f47dd5df700 1 --2- 172.21.15.47:0/3018525145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47d8146890 0x7f47d80af650 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7f47d80b1b60 tx=0x7f47c8006eb0).ready entity=mon.1 client_cookie=d558d1551ece6ce6 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:14.781 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.770+0000 7f47dcdde700 1 -- 172.21.15.47:0/3018525145 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47c800d5c0 con 0x7f47d8146890 2022-06-29T15:49:14.781 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.770+0000 7f47dcdde700 1 -- 172.21.15.47:0/3018525145 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c8000e90 con 0x7f47d8146890 2022-06-29T15:49:14.781 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.770+0000 7f47dcdde700 1 -- 172.21.15.47:0/3018525145 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47c800d950 con 0x7f47d8146890 2022-06-29T15:49:14.782 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.770+0000 7f47dede2700 1 -- 172.21.15.47:0/3018525145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47d8146890 msgr2=0x7f47d80af650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:14.782 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.770+0000 7f47dede2700 1 --2- 172.21.15.47:0/3018525145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47d8146890 0x7f47d80af650 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7f47d80b1b60 tx=0x7f47c8006eb0).stop 2022-06-29T15:49:14.783 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.770+0000 7f47dede2700 1 -- 172.21.15.47:0/3018525145 shutdown_connections 2022-06-29T15:49:14.783 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.770+0000 7f47dede2700 1 --2- 172.21.15.47:0/3018525145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47d8146890 0x7f47d80af650 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:14.784 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.770+0000 7f47dede2700 1 --2- 172.21.15.47:0/3018525145 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47d80b7860 0x7f47d80af160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:14.784 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.770+0000 7f47dede2700 1 -- 172.21.15.47:0/3018525145 >> 172.21.15.47:0/3018525145 conn(0x7f47d805f480 msgr2=0x7f47d80647d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:14.785 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.770+0000 7f47dede2700 1 -- 172.21.15.47:0/3018525145 shutdown_connections 2022-06-29T15:49:14.786 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.770+0000 7f47dede2700 1 -- 172.21.15.47:0/3018525145 wait complete. 2022-06-29T15:49:14.792 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.774+0000 7f47dede2700 1 Processor -- start 2022-06-29T15:49:14.792 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.774+0000 7f47dede2700 1 -- start start 2022-06-29T15:49:14.792 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.774+0000 7f47dede2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47d80b2d00 0x7f47d80b56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:14.793 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.774+0000 7f47dede2700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47d80b7860 0x7f47d80b5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:14.793 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.774+0000 7f47dede2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47d8146890 0x7f47d813dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:14.793 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.774+0000 7f47dede2700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d8152d10 con 0x7f47d80b2d00 2022-06-29T15:49:14.793 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.774+0000 7f47dede2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d813f980 con 0x7f47d8146890 2022-06-29T15:49:14.793 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.774+0000 7f47dede2700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47d8152f40 con 0x7f47d80b7860 2022-06-29T15:49:14.794 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.778+0000 7f47ddde0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47d80b2d00 0x7f47d80b56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:14.794 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.778+0000 7f47ddde0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47d80b2d00 0x7f47d80b56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50374/0 (socket says 172.21.15.47:50374) 2022-06-29T15:49:14.794 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.778+0000 7f47ddde0700 1 -- 172.21.15.47:0/875002013 learned_addr learned my addr 172.21.15.47:0/875002013 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:14.794 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.778+0000 7f47dd5df700 1 --2- 172.21.15.47:0/875002013 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47d80b7860 0x7f47d80b5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:14.795 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.778+0000 7f47de5e1700 1 --2- 172.21.15.47:0/875002013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47d8146890 0x7f47d813dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:14.795 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.778+0000 7f47ddde0700 1 -- 172.21.15.47:0/875002013 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47d80b7860 msgr2=0x7f47d80b5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:14.795 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.778+0000 7f47ddde0700 1 --2- 172.21.15.47:0/875002013 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47d80b7860 0x7f47d80b5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:14.795 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.778+0000 7f47ddde0700 1 -- 172.21.15.47:0/875002013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47d8146890 msgr2=0x7f47d813dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:14.795 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.778+0000 7f47ddde0700 1 --2- 172.21.15.47:0/875002013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47d8146890 0x7f47d813dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:14.796 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.778+0000 7f47ddde0700 1 -- 172.21.15.47:0/875002013 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d400b180 con 0x7f47d80b2d00 2022-06-29T15:49:14.796 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.778+0000 7f47ddde0700 1 --2- 172.21.15.47:0/875002013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47d80b2d00 0x7f47d80b56e0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 rx=0x7f47d400c000 tx=0x7f47d4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:14.796 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.778+0000 7f47c6ffd700 1 -- 172.21.15.47:0/875002013 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47d400c7e0 con 0x7f47d80b2d00 2022-06-29T15:49:14.796 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.778+0000 7f47c6ffd700 1 -- 172.21.15.47:0/875002013 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47d400c940 con 0x7f47d80b2d00 2022-06-29T15:49:14.797 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.778+0000 7f47c6ffd700 1 -- 172.21.15.47:0/875002013 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47d400cca0 con 0x7f47d80b2d00 2022-06-29T15:49:14.797 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.778+0000 7f47dede2700 1 -- 172.21.15.47:0/875002013 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47c800b1e0 con 0x7f47d80b2d00 2022-06-29T15:49:14.797 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.778+0000 7f47dede2700 1 -- 172.21.15.47:0/875002013 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47d8143a70 con 0x7f47d80b2d00 2022-06-29T15:49:14.798 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.778+0000 7f47dede2700 1 -- 172.21.15.47:0/875002013 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47d813f610 con 0x7f47d80b2d00 2022-06-29T15:49:14.798 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.778+0000 7f47c6ffd700 1 -- 172.21.15.47:0/875002013 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f47d401c460 con 0x7f47d80b2d00 2022-06-29T15:49:14.798 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.782+0000 7f47c6ffd700 1 --2- 172.21.15.47:0/875002013 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f47b80539e0 0x7f47b8055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:14.799 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.782+0000 7f47c6ffd700 1 -- 172.21.15.47:0/875002013 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6463+0+0 (secure 0 0 0) 0x7f47d406c600 con 0x7f47d80b2d00 2022-06-29T15:49:14.799 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.782+0000 7f47dd5df700 1 --2- 172.21.15.47:0/875002013 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f47b80539e0 0x7f47b8055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:14.800 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.782+0000 7f47dd5df700 1 --2- 172.21.15.47:0/875002013 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f47b80539e0 0x7f47b8055ea0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7f47c801c000 tx=0x7f47c8015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:14.801 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.786+0000 7f47c6ffd700 1 -- 172.21.15.47:0/875002013 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f47d813f610 con 0x7f47d80b2d00 2022-06-29T15:49:14.923 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.918+0000 7f47dede2700 1 -- 172.21.15.47:0/875002013 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f47d813fb40 con 0x7f47d80b2d00 2022-06-29T15:49:14.926 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.918+0000 7f47c6ffd700 1 -- 172.21.15.47:0/875002013 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v79) v1 ==== 81+0+34116 (secure 0 0 0) 0x7f47d813fb40 con 0x7f47d80b2d00 2022-06-29T15:49:14.927 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:49:14.927 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2022-06-29T15:49:14.928 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 79, 2022-06-29T15:49:14.928 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "9a68894a-53ce-400e-838b-14eab91b834c", 2022-06-29T15:49:14.928 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2022-06-29T15:42:14.046361+0000", 2022-06-29T15:49:14.928 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2022-06-29T15:49:04.104902+0000", 2022-06-29T15:49:14.929 INFO:teuthology.orchestra.run.smithi047.stdout: "last_up_change": "2022-06-29T15:42:20.929789+0000", 2022-06-29T15:49:14.929 INFO:teuthology.orchestra.run.smithi047.stdout: "last_in_change": "2022-06-29T15:42:20.929789+0000", 2022-06-29T15:49:14.930 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-06-29T15:49:14.930 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_num": 5799936, 2022-06-29T15:49:14.930 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_set": [ 2022-06-29T15:49:14.931 INFO:teuthology.orchestra.run.smithi047.stdout: "pglog_hardlimit", 2022-06-29T15:49:14.931 INFO:teuthology.orchestra.run.smithi047.stdout: "purged_snapdirs", 2022-06-29T15:49:14.931 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_deletes", 2022-06-29T15:49:14.932 INFO:teuthology.orchestra.run.smithi047.stdout: "sortbitwise" 2022-06-29T15:49:14.932 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:49:14.932 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_version": 5, 2022-06-29T15:49:14.932 INFO:teuthology.orchestra.run.smithi047.stdout: "full_ratio": 0.94999998807907104, 2022-06-29T15:49:14.933 INFO:teuthology.orchestra.run.smithi047.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-06-29T15:49:14.933 INFO:teuthology.orchestra.run.smithi047.stdout: "nearfull_ratio": 0.85000002384185791, 2022-06-29T15:49:14.933 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_snapshot": "", 2022-06-29T15:49:14.933 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_max": 14, 2022-06-29T15:49:14.934 INFO:teuthology.orchestra.run.smithi047.stdout: "max_osd": 8, 2022-06-29T15:49:14.934 INFO:teuthology.orchestra.run.smithi047.stdout: "require_min_compat_client": "jewel", 2022-06-29T15:49:14.934 INFO:teuthology.orchestra.run.smithi047.stdout: "min_compat_client": "jewel", 2022-06-29T15:49:14.934 INFO:teuthology.orchestra.run.smithi047.stdout: "require_osd_release": "octopus", 2022-06-29T15:49:14.935 INFO:teuthology.orchestra.run.smithi047.stdout: "pools": [ 2022-06-29T15:49:14.935 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:14.935 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 1, 2022-06-29T15:49:14.935 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "device_health_metrics", 2022-06-29T15:49:14.935 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:42:16.971431+0000", 2022-06-29T15:49:14.936 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:49:14.936 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:49:14.936 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:49:14.936 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:49:14.937 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:49:14.937 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:49:14.937 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:49:14.937 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:49:14.937 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 1, 2022-06-29T15:49:14.938 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 1, 2022-06-29T15:49:14.938 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 1, 2022-06-29T15:49:14.938 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 1, 2022-06-29T15:49:14.938 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 1, 2022-06-29T15:49:14.939 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:49:14.939 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:49:14.939 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:49:14.939 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:49:14.939 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:49:14.940 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:49:14.940 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:49:14.941 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:14.941 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "16", 2022-06-29T15:49:14.941 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:49:14.942 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:49:14.942 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:49:14.943 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:49:14.943 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:49:14.943 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:49:14.944 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:49:14.944 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:49:14.944 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:49:14.944 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:49:14.945 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:49:14.945 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:49:14.945 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:49:14.945 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:49:14.945 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:49:14.946 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:49:14.946 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:49:14.947 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:49:14.947 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:49:14.948 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:49:14.948 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:49:14.948 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:49:14.949 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:49:14.949 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:49:14.950 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:49:14.950 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:49:14.950 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:14.951 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:49:14.951 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:49:14.952 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:49:14.952 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:49:14.953 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:49:14.953 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:49:14.953 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:49:14.954 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:49:14.954 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:49:14.955 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:49:14.955 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:49:14.956 INFO:teuthology.orchestra.run.smithi047.stdout: "options": { 2022-06-29T15:49:14.956 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_min": 1 2022-06-29T15:49:14.956 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:14.956 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:49:14.957 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_devicehealth": {} 2022-06-29T15:49:14.957 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:14.957 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:14.957 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:14.958 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 2, 2022-06-29T15:49:14.958 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "rbd", 2022-06-29T15:49:14.958 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:42:25.290884+0000", 2022-06-29T15:49:14.958 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:49:14.959 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:49:14.959 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:49:14.959 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:49:14.959 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:49:14.959 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:49:14.960 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:49:14.960 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:49:14.960 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2022-06-29T15:49:14.960 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2022-06-29T15:49:14.961 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2022-06-29T15:49:14.961 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2022-06-29T15:49:14.961 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2022-06-29T15:49:14.961 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:49:14.962 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:49:14.962 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:49:14.962 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:49:14.962 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:49:14.962 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:49:14.963 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:49:14.963 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:14.963 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "19", 2022-06-29T15:49:14.963 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:49:14.964 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:49:14.964 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:49:14.964 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:49:14.964 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:49:14.964 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:49:14.965 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:49:14.965 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:49:14.965 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:49:14.966 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:49:14.966 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:49:14.966 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:49:14.967 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:49:14.967 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:49:14.968 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:49:14.968 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:49:14.969 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:49:14.969 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:49:14.970 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:49:14.970 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:49:14.970 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:49:14.971 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:49:14.971 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:49:14.971 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:49:14.972 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:49:14.972 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:49:14.972 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:14.973 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:49:14.973 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:49:14.974 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:49:14.974 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:49:14.974 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:49:14.975 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:49:14.975 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:49:14.975 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:49:14.976 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:49:14.976 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:49:14.977 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:49:14.977 INFO:teuthology.orchestra.run.smithi047.stdout: "options": {}, 2022-06-29T15:49:14.978 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:49:14.978 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd": {} 2022-06-29T15:49:14.978 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:14.979 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:14.979 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:14.979 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 13, 2022-06-29T15:49:14.979 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "cephfs_metadata", 2022-06-29T15:49:14.980 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:48:16.753707+0000", 2022-06-29T15:49:14.980 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:49:14.980 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:49:14.980 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:49:14.981 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:49:14.981 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:49:14.981 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:49:14.981 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:49:14.981 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:49:14.982 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2022-06-29T15:49:14.982 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2022-06-29T15:49:14.982 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2022-06-29T15:49:14.983 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2022-06-29T15:49:14.983 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2022-06-29T15:49:14.983 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:49:14.983 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:49:14.984 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:49:14.984 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:49:14.984 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:49:14.984 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:49:14.984 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:49:14.985 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:14.985 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "77", 2022-06-29T15:49:14.985 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:49:14.985 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:49:14.986 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:49:14.986 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:49:14.986 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:49:14.986 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:49:14.986 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:49:14.987 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:49:14.987 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:49:14.987 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:49:14.987 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:49:14.988 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:49:14.988 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:49:14.988 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:49:14.988 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:49:14.989 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:49:14.989 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:49:14.989 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:49:14.990 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:49:14.990 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:49:14.990 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:49:14.991 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:49:14.991 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:49:14.991 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:49:14.991 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:49:14.991 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:49:14.992 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:14.992 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:49:14.992 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:49:14.992 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:49:14.993 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:49:14.993 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:49:14.993 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:49:14.993 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:49:14.993 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:49:14.994 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:49:14.994 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:49:14.994 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:49:14.994 INFO:teuthology.orchestra.run.smithi047.stdout: "options": { 2022-06-29T15:49:14.995 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_bias": 4, 2022-06-29T15:49:14.995 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_min": 16, 2022-06-29T15:49:14.995 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_priority": 5 2022-06-29T15:49:14.995 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:14.996 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:49:14.996 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs": { 2022-06-29T15:49:14.996 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": "cephfs" 2022-06-29T15:49:14.996 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:14.996 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:14.997 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:14.997 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:14.997 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 14, 2022-06-29T15:49:14.997 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "cephfs_data", 2022-06-29T15:49:14.998 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:48:17.935585+0000", 2022-06-29T15:49:14.998 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:49:14.998 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:49:14.998 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:49:14.999 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:49:14.999 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:49:14.999 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:49:14.999 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:49:14.999 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:49:15.000 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2022-06-29T15:49:15.000 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2022-06-29T15:49:15.000 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2022-06-29T15:49:15.001 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2022-06-29T15:49:15.001 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2022-06-29T15:49:15.001 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:49:15.001 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:49:15.002 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:49:15.002 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:49:15.002 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:49:15.002 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:49:15.003 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:49:15.003 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.003 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "78", 2022-06-29T15:49:15.003 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:49:15.003 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:49:15.004 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:49:15.004 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:49:15.004 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:49:15.004 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:49:15.005 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:49:15.005 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:49:15.005 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:49:15.005 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:49:15.005 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:49:15.006 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:49:15.006 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:49:15.006 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:49:15.006 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:49:15.007 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:49:15.007 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:49:15.007 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:49:15.007 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:49:15.008 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:49:15.008 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:49:15.008 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:49:15.009 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:49:15.009 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:49:15.009 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:49:15.009 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:49:15.009 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.010 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:49:15.010 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:49:15.010 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:49:15.011 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:49:15.011 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:49:15.011 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:49:15.012 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:49:15.012 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:49:15.013 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:49:15.013 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:49:15.013 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:49:15.014 INFO:teuthology.orchestra.run.smithi047.stdout: "options": {}, 2022-06-29T15:49:15.014 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:49:15.014 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs": { 2022-06-29T15:49:15.014 INFO:teuthology.orchestra.run.smithi047.stdout: "data": "cephfs" 2022-06-29T15:49:15.015 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:15.015 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:15.015 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:15.015 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:49:15.015 INFO:teuthology.orchestra.run.smithi047.stdout: "osds": [ 2022-06-29T15:49:15.016 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.016 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2022-06-29T15:49:15.016 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "e23eb023-4706-4201-bf72-4bf19c912c5d", 2022-06-29T15:49:15.016 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:49:15.017 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:49:15.017 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:49:15.017 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:49:15.017 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:49:15.017 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:49:15.018 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:49:15.018 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 76, 2022-06-29T15:49:15.018 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:49:15.018 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:49:15.019 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:49:15.019 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:15.019 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.019 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:15.020 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6818", 2022-06-29T15:49:15.020 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:49:15.020 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.020 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.020 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:15.021 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6819", 2022-06-29T15:49:15.021 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:49:15.021 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:15.021 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:15.022 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.022 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:49:15.022 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:15.022 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.023 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:15.023 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6820", 2022-06-29T15:49:15.023 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:49:15.023 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.024 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.024 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:15.025 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6821", 2022-06-29T15:49:15.025 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:49:15.026 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:15.026 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:15.026 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.026 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:49:15.027 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:15.027 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.027 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:15.027 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6824", 2022-06-29T15:49:15.028 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:49:15.028 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.028 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.028 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:15.028 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6825", 2022-06-29T15:49:15.029 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:49:15.029 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:15.029 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:15.029 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.030 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:49:15.030 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:15.030 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.030 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:15.030 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6822", 2022-06-29T15:49:15.031 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:49:15.031 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.031 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.031 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:15.032 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6823", 2022-06-29T15:49:15.032 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:49:15.032 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:15.032 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:15.032 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.033 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6819/13096", 2022-06-29T15:49:15.033 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6821/13096", 2022-06-29T15:49:15.033 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6825/13096", 2022-06-29T15:49:15.033 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6823/13096", 2022-06-29T15:49:15.033 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:49:15.034 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:49:15.034 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:49:15.034 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:15.034 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.035 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.035 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2022-06-29T15:49:15.035 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "228bb08d-2d64-43d9-9560-bfaeff154452", 2022-06-29T15:49:15.035 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:49:15.035 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:49:15.036 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:49:15.036 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:49:15.036 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:49:15.036 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:49:15.037 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:49:15.037 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 66, 2022-06-29T15:49:15.037 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:49:15.037 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:49:15.037 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:49:15.038 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:15.038 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.038 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:15.038 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6826", 2022-06-29T15:49:15.039 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:49:15.039 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.039 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.039 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:15.039 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6827", 2022-06-29T15:49:15.040 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:49:15.040 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:15.040 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:15.040 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.041 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:49:15.041 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:15.041 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.041 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:15.041 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6828", 2022-06-29T15:49:15.042 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:49:15.042 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.042 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.042 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:15.043 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6829", 2022-06-29T15:49:15.043 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:49:15.043 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:15.043 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:15.043 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.044 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:49:15.044 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:15.044 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.044 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:15.045 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6832", 2022-06-29T15:49:15.045 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:49:15.045 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.045 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.046 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:15.046 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6833", 2022-06-29T15:49:15.046 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:49:15.046 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:15.046 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:15.047 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.047 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:49:15.047 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:15.047 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.048 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:15.048 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6830", 2022-06-29T15:49:15.048 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:49:15.048 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.048 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.049 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:15.049 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6831", 2022-06-29T15:49:15.049 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:49:15.049 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:15.050 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:15.050 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.050 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6827/13100", 2022-06-29T15:49:15.050 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6829/13100", 2022-06-29T15:49:15.050 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6833/13100", 2022-06-29T15:49:15.051 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6831/13100", 2022-06-29T15:49:15.051 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:49:15.051 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:49:15.051 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:49:15.052 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:15.052 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.052 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.052 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2022-06-29T15:49:15.052 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "797a5452-1510-4640-a111-c9c2fcf707eb", 2022-06-29T15:49:15.053 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:49:15.053 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:49:15.053 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:49:15.053 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:49:15.053 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:49:15.054 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:49:15.054 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:49:15.054 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 76, 2022-06-29T15:49:15.054 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:49:15.055 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:49:15.055 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:49:15.055 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:15.055 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.055 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:15.056 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6810", 2022-06-29T15:49:15.056 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:49:15.056 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.056 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.057 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:15.057 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6811", 2022-06-29T15:49:15.057 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:49:15.057 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:15.057 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:15.058 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.058 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:49:15.058 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:15.058 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.059 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:15.059 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6812", 2022-06-29T15:49:15.059 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:49:15.059 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.059 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.060 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:15.060 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6813", 2022-06-29T15:49:15.060 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:49:15.060 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:15.061 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:15.061 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.061 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:49:15.061 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:15.061 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.062 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:15.062 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6816", 2022-06-29T15:49:15.062 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:49:15.062 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.063 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.063 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:15.063 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6817", 2022-06-29T15:49:15.063 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:49:15.063 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:15.064 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:15.064 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.064 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:49:15.064 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:15.065 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.065 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:15.065 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6814", 2022-06-29T15:49:15.065 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:49:15.065 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.066 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.066 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:15.066 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6815", 2022-06-29T15:49:15.066 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:49:15.067 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:15.067 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:15.067 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.067 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6811/13098", 2022-06-29T15:49:15.068 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6813/13098", 2022-06-29T15:49:15.068 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6817/13098", 2022-06-29T15:49:15.068 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6815/13098", 2022-06-29T15:49:15.068 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:49:15.068 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:49:15.069 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:49:15.069 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:15.069 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.069 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.069 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2022-06-29T15:49:15.070 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "55b24fd7-bb71-4a01-b3fa-a09caa772a62", 2022-06-29T15:49:15.070 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:49:15.070 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:49:15.070 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:49:15.071 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:49:15.071 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:49:15.071 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:49:15.071 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:49:15.072 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 76, 2022-06-29T15:49:15.072 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:49:15.072 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:49:15.072 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:49:15.072 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:15.073 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.073 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:15.073 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6802", 2022-06-29T15:49:15.073 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:49:15.073 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.074 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.074 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:15.074 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6803", 2022-06-29T15:49:15.074 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:49:15.075 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:15.075 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:15.075 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.075 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:49:15.076 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:15.076 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.076 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:15.076 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6804", 2022-06-29T15:49:15.076 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:49:15.077 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.077 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.077 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:15.077 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6805", 2022-06-29T15:49:15.078 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:49:15.078 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:15.078 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:15.078 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.079 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:49:15.079 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:15.079 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.079 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:15.079 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6808", 2022-06-29T15:49:15.080 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:49:15.080 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.080 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.080 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:15.081 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6809", 2022-06-29T15:49:15.081 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:49:15.081 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:15.081 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:15.081 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.082 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:49:15.082 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:15.082 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.083 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:15.083 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6806", 2022-06-29T15:49:15.083 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:49:15.083 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.083 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.084 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:15.084 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6807", 2022-06-29T15:49:15.084 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:49:15.084 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:15.085 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:15.085 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.085 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6803/13097", 2022-06-29T15:49:15.085 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6805/13097", 2022-06-29T15:49:15.085 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6809/13097", 2022-06-29T15:49:15.086 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6807/13097", 2022-06-29T15:49:15.086 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:49:15.086 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:49:15.086 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:49:15.087 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:15.087 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.087 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.087 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2022-06-29T15:49:15.087 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3", 2022-06-29T15:49:15.088 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:49:15.088 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:49:15.088 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:49:15.088 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:49:15.088 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:49:15.089 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:49:15.089 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:49:15.089 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 75, 2022-06-29T15:49:15.089 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:49:15.090 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:49:15.090 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:49:15.090 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:15.090 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.090 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:15.091 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6816", 2022-06-29T15:49:15.091 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:49:15.091 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.091 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.092 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:15.092 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6817", 2022-06-29T15:49:15.092 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:49:15.092 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:15.092 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:15.093 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.093 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:49:15.093 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:15.093 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.093 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:15.094 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6818", 2022-06-29T15:49:15.094 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:49:15.094 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.094 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.095 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:15.095 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6819", 2022-06-29T15:49:15.095 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:49:15.095 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:15.095 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:15.096 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.096 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:49:15.096 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:15.096 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.097 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:15.097 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6822", 2022-06-29T15:49:15.097 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:49:15.097 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.097 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.098 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:15.098 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6823", 2022-06-29T15:49:15.098 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:49:15.098 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:15.099 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:15.099 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.099 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:49:15.099 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:15.099 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.100 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:15.100 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6820", 2022-06-29T15:49:15.100 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:49:15.100 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.101 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.101 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:15.101 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6821", 2022-06-29T15:49:15.101 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:49:15.101 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:15.102 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:15.102 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.102 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6817/13396", 2022-06-29T15:49:15.102 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6819/13396", 2022-06-29T15:49:15.103 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6823/13396", 2022-06-29T15:49:15.103 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6821/13396", 2022-06-29T15:49:15.103 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:49:15.103 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:49:15.103 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:49:15.104 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:15.104 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.104 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.104 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2022-06-29T15:49:15.104 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "f77beebe-6cca-4244-b2d4-677825ebf919", 2022-06-29T15:49:15.105 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:49:15.105 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:49:15.105 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:49:15.105 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:49:15.106 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:49:15.106 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:49:15.106 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:49:15.106 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 76, 2022-06-29T15:49:15.107 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:49:15.107 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:49:15.107 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:49:15.107 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:15.107 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.108 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:15.108 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6800", 2022-06-29T15:49:15.108 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:49:15.108 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.109 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.109 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:15.109 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6801", 2022-06-29T15:49:15.109 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:49:15.109 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:15.110 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:15.110 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.110 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:49:15.110 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:15.111 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.111 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:15.111 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6802", 2022-06-29T15:49:15.111 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:49:15.111 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.112 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.112 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:15.112 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6803", 2022-06-29T15:49:15.112 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:49:15.112 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:15.113 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:15.113 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.113 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:49:15.113 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:15.114 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.114 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:15.114 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6806", 2022-06-29T15:49:15.114 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:49:15.114 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.115 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.115 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:15.115 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6807", 2022-06-29T15:49:15.115 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:49:15.116 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:15.116 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:15.116 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.116 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:49:15.116 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:15.117 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.117 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:15.117 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6804", 2022-06-29T15:49:15.117 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:49:15.117 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.118 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.118 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:15.118 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6805", 2022-06-29T15:49:15.118 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:49:15.119 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:15.119 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:15.119 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.119 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6801/13393", 2022-06-29T15:49:15.120 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6803/13393", 2022-06-29T15:49:15.120 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6807/13393", 2022-06-29T15:49:15.120 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6805/13393", 2022-06-29T15:49:15.120 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:49:15.120 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:49:15.121 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:49:15.121 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:15.121 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.121 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.121 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2022-06-29T15:49:15.122 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "af0ca9b3-c02c-4910-834b-bf0b8dc6713c", 2022-06-29T15:49:15.122 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:49:15.122 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:49:15.122 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:49:15.123 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:49:15.123 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:49:15.123 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:49:15.123 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:49:15.123 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 76, 2022-06-29T15:49:15.124 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:49:15.124 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:49:15.124 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:49:15.124 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:15.125 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.125 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:15.125 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6824", 2022-06-29T15:49:15.125 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:49:15.125 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.126 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.126 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:15.126 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6825", 2022-06-29T15:49:15.126 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:49:15.127 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:15.127 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:15.127 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.127 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:49:15.127 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:15.128 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.128 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:15.128 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6826", 2022-06-29T15:49:15.128 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:49:15.128 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.129 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.129 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:15.129 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6827", 2022-06-29T15:49:15.129 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:49:15.130 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:15.130 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:15.130 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.130 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:49:15.131 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:15.131 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.131 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:15.131 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6830", 2022-06-29T15:49:15.131 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:49:15.132 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.132 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.132 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:15.132 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6831", 2022-06-29T15:49:15.133 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:49:15.133 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:15.133 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:15.133 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.133 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:49:15.134 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:15.134 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.134 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:15.135 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6828", 2022-06-29T15:49:15.135 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:49:15.135 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.135 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.136 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:15.136 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6829", 2022-06-29T15:49:15.136 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:49:15.136 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:15.137 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:15.137 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.137 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6825/13395", 2022-06-29T15:49:15.137 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6827/13395", 2022-06-29T15:49:15.138 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6831/13395", 2022-06-29T15:49:15.138 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6829/13395", 2022-06-29T15:49:15.138 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:49:15.138 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:49:15.139 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:49:15.139 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:15.139 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.139 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.139 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2022-06-29T15:49:15.140 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "f19c3ca6-df88-4bde-a150-b51a53f092e1", 2022-06-29T15:49:15.140 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:49:15.140 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:49:15.140 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:49:15.141 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:49:15.141 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:49:15.141 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:49:15.141 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:49:15.141 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 75, 2022-06-29T15:49:15.142 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:49:15.142 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:49:15.142 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:49:15.142 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:15.142 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.143 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:15.143 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6808", 2022-06-29T15:49:15.143 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:49:15.143 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.144 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.144 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:15.144 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6809", 2022-06-29T15:49:15.144 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:49:15.144 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:15.145 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:15.145 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.145 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:49:15.145 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:15.145 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.146 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:15.146 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6810", 2022-06-29T15:49:15.146 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:49:15.146 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.147 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.147 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:15.147 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6811", 2022-06-29T15:49:15.147 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:49:15.148 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:15.148 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:15.148 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.148 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:49:15.148 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:15.149 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.149 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:15.149 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6814", 2022-06-29T15:49:15.149 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:49:15.150 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.150 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.150 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:15.150 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6815", 2022-06-29T15:49:15.151 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:49:15.151 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:15.151 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:15.151 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.151 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:49:15.152 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:15.152 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.152 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:15.152 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6812", 2022-06-29T15:49:15.153 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:49:15.153 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.153 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.153 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:15.153 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6813", 2022-06-29T15:49:15.154 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:49:15.154 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:15.154 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:15.154 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.155 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6809/13394", 2022-06-29T15:49:15.155 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6811/13394", 2022-06-29T15:49:15.155 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6815/13394", 2022-06-29T15:49:15.155 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6813/13394", 2022-06-29T15:49:15.155 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:49:15.156 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:49:15.156 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:49:15.156 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:15.156 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:15.156 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:49:15.157 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_xinfo": [ 2022-06-29T15:49:15.157 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.157 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2022-06-29T15:49:15.157 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:49:15.158 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:49:15.158 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:49:15.158 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:49:15.158 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:49:15.158 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.720513+0000", 2022-06-29T15:49:15.159 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:49:15.159 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.159 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.159 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2022-06-29T15:49:15.160 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:49:15.160 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:49:15.160 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:49:15.160 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:49:15.160 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:49:15.161 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.772612+0000", 2022-06-29T15:49:15.161 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:49:15.161 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.161 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.162 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2022-06-29T15:49:15.162 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:49:15.162 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:49:15.162 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:49:15.163 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:49:15.163 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:49:15.163 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.450223+0000", 2022-06-29T15:49:15.163 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:49:15.163 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.164 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.164 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2022-06-29T15:49:15.164 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:49:15.164 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:49:15.165 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:49:15.165 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:49:15.165 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:49:15.165 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.477856+0000", 2022-06-29T15:49:15.165 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:49:15.166 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.166 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.166 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2022-06-29T15:49:15.166 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:49:15.167 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:49:15.167 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:49:15.167 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:49:15.167 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:49:15.167 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.539485+0000", 2022-06-29T15:49:15.168 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:49:15.168 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.168 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.168 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2022-06-29T15:49:15.169 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:49:15.169 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:49:15.169 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:49:15.169 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:49:15.169 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:49:15.170 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.487040+0000", 2022-06-29T15:49:15.170 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:49:15.170 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.170 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.171 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2022-06-29T15:49:15.171 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:49:15.171 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:49:15.171 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:49:15.171 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:49:15.172 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:49:15.172 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.748685+0000", 2022-06-29T15:49:15.172 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:49:15.172 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.173 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:15.173 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2022-06-29T15:49:15.173 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:49:15.173 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:49:15.173 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:49:15.174 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:49:15.174 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:49:15.174 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.459516+0000", 2022-06-29T15:49:15.174 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:49:15.175 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:15.175 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:49:15.175 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap": [], 2022-06-29T15:49:15.175 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap_items": [], 2022-06-29T15:49:15.175 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_temp": [], 2022-06-29T15:49:15.176 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_temp": [], 2022-06-29T15:49:15.176 INFO:teuthology.orchestra.run.smithi047.stdout: "blacklist": { 2022-06-29T15:49:15.176 INFO:teuthology.orchestra.run.smithi047.stdout: "172.21.15.99:0/2790422795": "2022-06-29T16:49:03.956107+0000" 2022-06-29T15:49:15.176 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.176 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profiles": { 2022-06-29T15:49:15.177 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs_data": { 2022-06-29T15:49:15.177 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-device-class": "", 2022-06-29T15:49:15.177 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2022-06-29T15:49:15.177 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-root": "default", 2022-06-29T15:49:15.178 INFO:teuthology.orchestra.run.smithi047.stdout: "jerasure-per-chunk-alignment": "false", 2022-06-29T15:49:15.178 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2022-06-29T15:49:15.178 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "2", 2022-06-29T15:49:15.178 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2022-06-29T15:49:15.178 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2022-06-29T15:49:15.179 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van", 2022-06-29T15:49:15.179 INFO:teuthology.orchestra.run.smithi047.stdout: "w": "8" 2022-06-29T15:49:15.179 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.179 INFO:teuthology.orchestra.run.smithi047.stdout: "default": { 2022-06-29T15:49:15.180 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2022-06-29T15:49:15.180 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2022-06-29T15:49:15.180 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "1", 2022-06-29T15:49:15.180 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2022-06-29T15:49:15.180 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2022-06-29T15:49:15.181 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van" 2022-06-29T15:49:15.181 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:15.181 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:15.181 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps_queue": [], 2022-06-29T15:49:15.182 INFO:teuthology.orchestra.run.smithi047.stdout: "new_removed_snaps": [], 2022-06-29T15:49:15.182 INFO:teuthology.orchestra.run.smithi047.stdout: "new_purged_snaps": [], 2022-06-29T15:49:15.182 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_node_flags": {}, 2022-06-29T15:49:15.182 INFO:teuthology.orchestra.run.smithi047.stdout: "device_class_flags": {} 2022-06-29T15:49:15.183 INFO:teuthology.orchestra.run.smithi047.stdout:} 2022-06-29T15:49:15.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.922+0000 7f47dede2700 1 -- 172.21.15.47:0/875002013 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f47b80539e0 msgr2=0x7f47b8055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:15.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.922+0000 7f47dede2700 1 --2- 172.21.15.47:0/875002013 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f47b80539e0 0x7f47b8055ea0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7f47c801c000 tx=0x7f47c8015000).stop 2022-06-29T15:49:15.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.922+0000 7f47dede2700 1 -- 172.21.15.47:0/875002013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47d80b2d00 msgr2=0x7f47d80b56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:15.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.922+0000 7f47dede2700 1 --2- 172.21.15.47:0/875002013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47d80b2d00 0x7f47d80b56e0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 rx=0x7f47d400c000 tx=0x7f47d4006df0).stop 2022-06-29T15:49:15.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.922+0000 7f47dede2700 1 -- 172.21.15.47:0/875002013 shutdown_connections 2022-06-29T15:49:15.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.922+0000 7f47dede2700 1 --2- 172.21.15.47:0/875002013 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f47b80539e0 0x7f47b8055ea0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:15.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.922+0000 7f47dede2700 1 --2- 172.21.15.47:0/875002013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47d8146890 0x7f47d813dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:15.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.922+0000 7f47dede2700 1 --2- 172.21.15.47:0/875002013 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47d80b7860 0x7f47d80b5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:15.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.922+0000 7f47dede2700 1 --2- 172.21.15.47:0/875002013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47d80b2d00 0x7f47d80b56e0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:15.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.922+0000 7f47dede2700 1 -- 172.21.15.47:0/875002013 >> 172.21.15.47:0/875002013 conn(0x7f47d805f480 msgr2=0x7f47d80b0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:15.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.922+0000 7f47dede2700 1 -- 172.21.15.47:0/875002013 shutdown_connections 2022-06-29T15:49:15.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:14.922+0000 7f47dede2700 1 -- 172.21.15.47:0/875002013 wait complete. 2022-06-29T15:49:15.188 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:49:15.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.294+0000 7f7f0d44a700 1 Processor -- start 2022-06-29T15:49:15.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.294+0000 7f7f0d44a700 1 -- start start 2022-06-29T15:49:15.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.298+0000 7f7f0d44a700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7f080acff0 0x7f7f0805fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:15.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.298+0000 7f7f0d44a700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7f081437d0 0x7f7f08143c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:15.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.298+0000 7f7f0d44a700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f080a97e0 con 0x7f7f080b19a0 2022-06-29T15:49:15.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.298+0000 7f7f0d44a700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f080b1d80 con 0x7f7f081437d0 2022-06-29T15:49:15.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.298+0000 7f7f0d44a700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0805f430 con 0x7f7f080acff0 2022-06-29T15:49:15.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.298+0000 7f7efffff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7f080acff0 0x7f7f0805fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:15.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.298+0000 7f7f0cc49700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7f081437d0 0x7f7f08143c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:15.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.298+0000 7f7f07fff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f7f080b19a0 0x7f7f080ae840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:43500/0 (socket says 172.21.15.47:43500) 2022-06-29T15:49:15.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.298+0000 7f7f07fff700 1 -- 172.21.15.47:0/2568841788 learned_addr learned my addr 172.21.15.47:0/2568841788 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:49:15.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.298+0000 7f7f077fe700 1 -- 172.21.15.47:0/2568841788 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2337808242 0 0) 0x7f7f080a97e0 con 0x7f7f080b19a0 2022-06-29T15:49:15.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.298+0000 7f7f077fe700 1 -- 172.21.15.47:0/2568841788 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ee8003260 con 0x7f7f080b19a0 2022-06-29T15:49:15.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.298+0000 7f7efffff700 1 -- 172.21.15.47:0/2568841788 >> v2:172.21.15.64:3300/0 conn(0x7f7f081437d0 msgr2=0x7f7f08143c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:15.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.298+0000 7f7efffff700 1 --2- 172.21.15.47:0/2568841788 >> v2:172.21.15.64:3300/0 conn(0x7f7f081437d0 0x7f7f08143c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:15.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.298+0000 7f7efffff700 1 -- 172.21.15.47:0/2568841788 >> v1:172.21.15.64:6789/0 conn(0x7f7f080b19a0 legacy=0x7f7f080ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:15.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.298+0000 7f7efffff700 1 -- 172.21.15.47:0/2568841788 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f080a92a0 con 0x7f7f080acff0 2022-06-29T15:49:15.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.298+0000 7f7efffff700 1 --2- 172.21.15.47:0/2568841788 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7f080acff0 0x7f7f0805fd10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f7ef0002f70 tx=0x7f7ef0006df0).ready entity=mon.2 client_cookie=8d8613ab84d4cec9 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:15.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.298+0000 7f7f077fe700 1 -- 172.21.15.47:0/2568841788 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ef000d4a0 con 0x7f7f080acff0 2022-06-29T15:49:15.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.298+0000 7f7f077fe700 1 -- 172.21.15.47:0/2568841788 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef0000e90 con 0x7f7f080acff0 2022-06-29T15:49:15.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.298+0000 7f7f077fe700 1 -- 172.21.15.47:0/2568841788 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ef000d7f0 con 0x7f7f080acff0 2022-06-29T15:49:15.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.298+0000 7f7f0d44a700 1 -- 172.21.15.47:0/2568841788 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7f080acff0 msgr2=0x7f7f0805fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:15.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.298+0000 7f7f0d44a700 1 --2- 172.21.15.47:0/2568841788 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7f080acff0 0x7f7f0805fd10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f7ef0002f70 tx=0x7f7ef0006df0).stop 2022-06-29T15:49:15.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.298+0000 7f7f0d44a700 1 -- 172.21.15.47:0/2568841788 shutdown_connections 2022-06-29T15:49:15.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.298+0000 7f7f0d44a700 1 --2- 172.21.15.47:0/2568841788 >> v2:172.21.15.64:3300/0 conn(0x7f7f081437d0 0x7f7f08143c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:15.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.298+0000 7f7f0d44a700 1 --2- 172.21.15.47:0/2568841788 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7f080acff0 0x7f7f0805fd10 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:15.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.298+0000 7f7f0d44a700 1 -- 172.21.15.47:0/2568841788 >> 172.21.15.47:0/2568841788 conn(0x7f7f08062df0 msgr2=0x7f7f080631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:15.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.302+0000 7f7f0d44a700 1 -- 172.21.15.47:0/2568841788 shutdown_connections 2022-06-29T15:49:15.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.302+0000 7f7f0d44a700 1 -- 172.21.15.47:0/2568841788 wait complete. 2022-06-29T15:49:15.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.302+0000 7f7f0d44a700 1 Processor -- start 2022-06-29T15:49:15.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.302+0000 7f7f0d44a700 1 -- start start 2022-06-29T15:49:15.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.306+0000 7f7f0d44a700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7f080acff0 0x7f7f08099820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:15.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.306+0000 7f7f0d44a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f080b19a0 0x7f7f080642b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:15.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.306+0000 7f7f0d44a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f081437d0 0x7f7f080abf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:15.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.306+0000 7f7f0d44a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f08138ea0 con 0x7f7f081437d0 2022-06-29T15:49:15.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.306+0000 7f7f0d44a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f080647a0 con 0x7f7f080b19a0 2022-06-29T15:49:15.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.306+0000 7f7f0d44a700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f08098ce0 con 0x7f7f080acff0 2022-06-29T15:49:15.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.306+0000 7f7f0cc49700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f081437d0 0x7f7f080abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:15.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.306+0000 7f7efffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f080b19a0 0x7f7f080642b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:15.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.306+0000 7f7f07fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7f080acff0 0x7f7f08099820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:15.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.306+0000 7f7f0cc49700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f081437d0 0x7f7f080abf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50392/0 (socket says 172.21.15.47:50392) 2022-06-29T15:49:15.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.306+0000 7f7f0cc49700 1 -- 172.21.15.47:0/716727011 learned_addr learned my addr 172.21.15.47:0/716727011 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:15.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.306+0000 7f7f0cc49700 1 -- 172.21.15.47:0/716727011 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7f080acff0 msgr2=0x7f7f08099820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:15.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.306+0000 7f7f0cc49700 1 --2- 172.21.15.47:0/716727011 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7f080acff0 0x7f7f08099820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:15.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.306+0000 7f7f0cc49700 1 -- 172.21.15.47:0/716727011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f080b19a0 msgr2=0x7f7f080642b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:15.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.306+0000 7f7f0cc49700 1 --2- 172.21.15.47:0/716727011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f080b19a0 0x7f7f080642b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:15.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.306+0000 7f7f0cc49700 1 -- 172.21.15.47:0/716727011 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef000b180 con 0x7f7f081437d0 2022-06-29T15:49:15.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.306+0000 7f7f0cc49700 1 --2- 172.21.15.47:0/716727011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f081437d0 0x7f7f080abf50 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 rx=0x7f7f00006e00 tx=0x7f7f000091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:15.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.306+0000 7f7f05ffb700 1 -- 172.21.15.47:0/716727011 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f00012600 con 0x7f7f081437d0 2022-06-29T15:49:15.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.306+0000 7f7f05ffb700 1 -- 172.21.15.47:0/716727011 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f00012760 con 0x7f7f081437d0 2022-06-29T15:49:15.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.306+0000 7f7f0d44a700 1 -- 172.21.15.47:0/716727011 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f0809add0 con 0x7f7f081437d0 2022-06-29T15:49:15.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.306+0000 7f7f05ffb700 1 -- 172.21.15.47:0/716727011 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f00012ac0 con 0x7f7f081437d0 2022-06-29T15:49:15.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.306+0000 7f7f0d44a700 1 -- 172.21.15.47:0/716727011 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f08098430 con 0x7f7f081437d0 2022-06-29T15:49:15.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.306+0000 7f7f0d44a700 1 -- 172.21.15.47:0/716727011 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f0809af60 con 0x7f7f081437d0 2022-06-29T15:49:15.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.306+0000 7f7f05ffb700 1 -- 172.21.15.47:0/716727011 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7f00022070 con 0x7f7f081437d0 2022-06-29T15:49:15.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.310+0000 7f7f05ffb700 1 --2- 172.21.15.47:0/716727011 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7eec053950 0x7f7eec055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:15.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.310+0000 7f7f05ffb700 1 -- 172.21.15.47:0/716727011 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6463+0+0 (secure 0 0 0) 0x7f7f0006c600 con 0x7f7f081437d0 2022-06-29T15:49:15.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.310+0000 7f7f07fff700 1 --2- 172.21.15.47:0/716727011 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7eec053950 0x7f7eec055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:15.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.310+0000 7f7f07fff700 1 --2- 172.21.15.47:0/716727011 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7eec053950 0x7f7eec055e10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7f7ef800d000 tx=0x7f7ef8009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:15.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.314+0000 7f7f05ffb700 1 -- 172.21.15.47:0/716727011 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7f00043300 con 0x7f7f081437d0 2022-06-29T15:49:15.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.478+0000 7f7f0d44a700 1 -- 172.21.15.47:0/716727011 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f080573d0 con 0x7f7f081437d0 2022-06-29T15:49:15.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.482+0000 7f7f05ffb700 1 -- 172.21.15.47:0/716727011 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+2493 (secure 0 0 0) 0x7f7f00016090 con 0x7f7f081437d0 2022-06-29T15:49:15.488 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:49:15.488 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":35,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6291,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.47:6835/2910037081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2910037081},{"type":"v1","addr":"172.21.15.47:6835","nonce":2910037081}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":34},{"gid":6300,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/359532891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":359532891},{"type":"v1","addr":"172.21.15.47:6837","nonce":359532891}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":33}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:48:18.345845+0000","modified":"2022-06-29T15:48:19.349718+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":55562},"failed":[],"damaged":[],"stopped":[],"info":{"gid_55562":{"gid":55562,"name":"b","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6833/3871453337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3871453337},{"type":"v1","addr":"172.21.15.64:6833","nonce":3871453337}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2022-06-29T15:49:15.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.482+0000 7f7f0d44a700 1 -- 172.21.15.47:0/716727011 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7eec053950 msgr2=0x7f7eec055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:15.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.482+0000 7f7f0d44a700 1 --2- 172.21.15.47:0/716727011 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7eec053950 0x7f7eec055e10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7f7ef800d000 tx=0x7f7ef8009040).stop 2022-06-29T15:49:15.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.482+0000 7f7f0d44a700 1 -- 172.21.15.47:0/716727011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f081437d0 msgr2=0x7f7f080abf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:15.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.482+0000 7f7f0d44a700 1 --2- 172.21.15.47:0/716727011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f081437d0 0x7f7f080abf50 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 rx=0x7f7f00006e00 tx=0x7f7f000091d0).stop 2022-06-29T15:49:15.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.482+0000 7f7f0d44a700 1 -- 172.21.15.47:0/716727011 shutdown_connections 2022-06-29T15:49:15.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.482+0000 7f7f0d44a700 1 --2- 172.21.15.47:0/716727011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f081437d0 0x7f7f080abf50 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:15.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.482+0000 7f7f0d44a700 1 --2- 172.21.15.47:0/716727011 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7eec053950 0x7f7eec055e10 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:15.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.482+0000 7f7f0d44a700 1 --2- 172.21.15.47:0/716727011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f080b19a0 0x7f7f080642b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:15.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.482+0000 7f7f0d44a700 1 --2- 172.21.15.47:0/716727011 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7f080acff0 0x7f7f08099820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:15.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.482+0000 7f7f0d44a700 1 -- 172.21.15.47:0/716727011 >> 172.21.15.47:0/716727011 conn(0x7f7f08062df0 msgr2=0x7f7f080b24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:15.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.482+0000 7f7f0d44a700 1 -- 172.21.15.47:0/716727011 shutdown_connections 2022-06-29T15:49:15.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.482+0000 7f7f0d44a700 1 -- 172.21.15.47:0/716727011 wait complete. 2022-06-29T15:49:15.493 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 35 2022-06-29T15:49:15.501 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-06-29T15:49:15.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.602+0000 7f29cafad700 1 Processor -- start 2022-06-29T15:49:15.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.602+0000 7f29cafad700 1 -- start start 2022-06-29T15:49:15.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.606+0000 7f29cafad700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f29c40b0f30 0x7f29c40606e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:15.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.606+0000 7f29cafad700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f29c41408e0 0x7f29c4139a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:15.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.606+0000 7f29cafad700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29c4140d50 con 0x7f29c40619b0 2022-06-29T15:49:15.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.606+0000 7f29cafad700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29c40b2190 con 0x7f29c41408e0 2022-06-29T15:49:15.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.606+0000 7f29cafad700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c4061d90 con 0x7f29c40b0f30 2022-06-29T15:49:15.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.606+0000 7f29c8d49700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f29c40619b0 0x7f29c40b1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35952/0 (socket says 172.21.15.47:35952) 2022-06-29T15:49:15.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.606+0000 7f29c8d49700 1 -- 172.21.15.47:0/3116573515 learned_addr learned my addr 172.21.15.47:0/3116573515 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:49:15.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.606+0000 7f29c954a700 1 --2- 172.21.15.47:0/3116573515 >> v2:172.21.15.64:3300/0 conn(0x7f29c41408e0 0x7f29c4139a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:15.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.606+0000 7f29c3fff700 1 --2- 172.21.15.47:0/3116573515 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f29c40b0f30 0x7f29c40606e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:15.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.606+0000 7f29c37fe700 1 -- 172.21.15.47:0/3116573515 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4264562439 0 0) 0x7f29c4140d50 con 0x7f29c40619b0 2022-06-29T15:49:15.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.606+0000 7f29c37fe700 1 -- 172.21.15.47:0/3116573515 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29b0003260 con 0x7f29c40619b0 2022-06-29T15:49:15.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.606+0000 7f29c954a700 1 -- 172.21.15.47:0/3116573515 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f29c40b0f30 msgr2=0x7f29c40606e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:15.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.606+0000 7f29c954a700 1 --2- 172.21.15.47:0/3116573515 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f29c40b0f30 0x7f29c40606e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:15.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.606+0000 7f29c954a700 1 -- 172.21.15.47:0/3116573515 >> v1:172.21.15.47:6789/0 conn(0x7f29c40619b0 legacy=0x7f29c40b1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:15.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.606+0000 7f29c954a700 1 -- 172.21.15.47:0/3116573515 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c4060d80 con 0x7f29c41408e0 2022-06-29T15:49:15.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.606+0000 7f29c3fff700 1 --2- 172.21.15.47:0/3116573515 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f29c40b0f30 0x7f29c40606e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:49:15.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.606+0000 7f29c954a700 1 --2- 172.21.15.47:0/3116573515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29c41408e0 0x7f29c4139a20 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7f29bc0032f0 tx=0x7f29bc0091d0).ready entity=mon.1 client_cookie=f8eed5b42c68f528 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:15.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.606+0000 7f29c37fe700 1 -- 172.21.15.47:0/3116573515 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29bc00b600 con 0x7f29c41408e0 2022-06-29T15:49:15.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.606+0000 7f29c37fe700 1 -- 172.21.15.47:0/3116573515 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29bc00b760 con 0x7f29c41408e0 2022-06-29T15:49:15.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.606+0000 7f29c37fe700 1 -- 172.21.15.47:0/3116573515 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29bc00b8d0 con 0x7f29c41408e0 2022-06-29T15:49:15.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.606+0000 7f29cafad700 1 -- 172.21.15.47:0/3116573515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29c41408e0 msgr2=0x7f29c4139a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:15.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.606+0000 7f29cafad700 1 --2- 172.21.15.47:0/3116573515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29c41408e0 0x7f29c4139a20 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7f29bc0032f0 tx=0x7f29bc0091d0).stop 2022-06-29T15:49:15.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.606+0000 7f29cafad700 1 -- 172.21.15.47:0/3116573515 shutdown_connections 2022-06-29T15:49:15.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.606+0000 7f29cafad700 1 --2- 172.21.15.47:0/3116573515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29c41408e0 0x7f29c4139a20 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:15.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.606+0000 7f29cafad700 1 --2- 172.21.15.47:0/3116573515 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f29c40b0f30 0x7f29c40606e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:15.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.606+0000 7f29cafad700 1 -- 172.21.15.47:0/3116573515 >> 172.21.15.47:0/3116573515 conn(0x7f29c405c510 msgr2=0x7f29c4063800 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:15.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.606+0000 7f29cafad700 1 -- 172.21.15.47:0/3116573515 shutdown_connections 2022-06-29T15:49:15.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.606+0000 7f29cafad700 1 -- 172.21.15.47:0/3116573515 wait complete. 2022-06-29T15:49:15.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.610+0000 7f29cafad700 1 Processor -- start 2022-06-29T15:49:15.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.610+0000 7f29cafad700 1 -- start start 2022-06-29T15:49:15.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.610+0000 7f29cafad700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29c40619b0 0x7f29c4143d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:15.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.610+0000 7f29cafad700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f29c40b0f30 0x7f29c4142ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:15.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.610+0000 7f29cafad700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29c41408e0 0x7f29c4143190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:15.626 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.610+0000 7f29cafad700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c4141d00 con 0x7f29c41408e0 2022-06-29T15:49:15.626 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.610+0000 7f29cafad700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c40b5120 con 0x7f29c40619b0 2022-06-29T15:49:15.626 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.610+0000 7f29cafad700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c40b5a90 con 0x7f29c40b0f30 2022-06-29T15:49:15.626 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.610+0000 7f29c954a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29c41408e0 0x7f29c4143190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:15.627 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.610+0000 7f29c954a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29c41408e0 0x7f29c4143190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50404/0 (socket says 172.21.15.47:50404) 2022-06-29T15:49:15.627 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.610+0000 7f29c954a700 1 -- 172.21.15.47:0/4034731706 learned_addr learned my addr 172.21.15.47:0/4034731706 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:15.627 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.610+0000 7f29c3fff700 1 --2- 172.21.15.47:0/4034731706 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f29c40b0f30 0x7f29c4142ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:15.627 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.610+0000 7f29c8d49700 1 --2- 172.21.15.47:0/4034731706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29c40619b0 0x7f29c4143d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:15.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.610+0000 7f29c954a700 1 -- 172.21.15.47:0/4034731706 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f29c40b0f30 msgr2=0x7f29c4142ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:15.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.610+0000 7f29c954a700 1 --2- 172.21.15.47:0/4034731706 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f29c40b0f30 0x7f29c4142ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:15.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.610+0000 7f29c954a700 1 -- 172.21.15.47:0/4034731706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29c40619b0 msgr2=0x7f29c4143d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:15.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.610+0000 7f29c954a700 1 --2- 172.21.15.47:0/4034731706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29c40619b0 0x7f29c4143d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:15.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.610+0000 7f29c954a700 1 -- 172.21.15.47:0/4034731706 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b400b180 con 0x7f29c41408e0 2022-06-29T15:49:15.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.610+0000 7f29c954a700 1 --2- 172.21.15.47:0/4034731706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29c41408e0 0x7f29c4143190 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 rx=0x7f29bc0032f0 tx=0x7f29bc016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:15.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.610+0000 7f29c1ffb700 1 -- 172.21.15.47:0/4034731706 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29bc00b600 con 0x7f29c41408e0 2022-06-29T15:49:15.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.610+0000 7f29c1ffb700 1 -- 172.21.15.47:0/4034731706 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29bc011b60 con 0x7f29c41408e0 2022-06-29T15:49:15.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.610+0000 7f29cafad700 1 -- 172.21.15.47:0/4034731706 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29bc009470 con 0x7f29c41408e0 2022-06-29T15:49:15.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.610+0000 7f29c1ffb700 1 -- 172.21.15.47:0/4034731706 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29bc01f3f0 con 0x7f29c41408e0 2022-06-29T15:49:15.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.610+0000 7f29cafad700 1 -- 172.21.15.47:0/4034731706 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29c40b5350 con 0x7f29c41408e0 2022-06-29T15:49:15.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.614+0000 7f29cafad700 1 -- 172.21.15.47:0/4034731706 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29c40b5790 con 0x7f29c41408e0 2022-06-29T15:49:15.631 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.614+0000 7f29c1ffb700 1 -- 172.21.15.47:0/4034731706 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f29bc026030 con 0x7f29c41408e0 2022-06-29T15:49:15.631 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.614+0000 7f29c1ffb700 1 --2- 172.21.15.47:0/4034731706 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f29a4053b00 0x7f29a4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:15.631 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.614+0000 7f29c1ffb700 1 -- 172.21.15.47:0/4034731706 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6463+0+0 (secure 0 0 0) 0x7f29bc06ca40 con 0x7f29c41408e0 2022-06-29T15:49:15.631 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.614+0000 7f29c8d49700 1 --2- 172.21.15.47:0/4034731706 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f29a4053b00 0x7f29a4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:15.631 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.614+0000 7f29c8d49700 1 --2- 172.21.15.47:0/4034731706 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f29a4053b00 0x7f29a4055fc0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7f29b800d000 tx=0x7f29b8009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:15.632 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.622+0000 7f29c1ffb700 1 -- 172.21.15.47:0/4034731706 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f29bc0437e0 con 0x7f29c41408e0 2022-06-29T15:49:15.793 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:15.786+0000 7f29cafad700 1 -- 172.21.15.47:0/4034731706 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f29c40573d0 con 0x7f29c41408e0 2022-06-29T15:49:16.251 INFO:tasks.ceph.mon.a.smithi047.stderr:2022-06-29T15:49:16.246+0000 7ff9cf83c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-06-29T15:49:16.265 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.258+0000 7f29c1ffb700 1 -- 172.21.15.47:0/4034731706 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v36) v1 ==== 161+0+0 (secure 0 0 0) 0x7f29bc046e00 con 0x7f29c41408e0 2022-06-29T15:49:16.268 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.262+0000 7f29cafad700 1 -- 172.21.15.47:0/4034731706 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f29a4053b00 msgr2=0x7f29a4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:16.269 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.262+0000 7f29cafad700 1 --2- 172.21.15.47:0/4034731706 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f29a4053b00 0x7f29a4055fc0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7f29b800d000 tx=0x7f29b8009040).stop 2022-06-29T15:49:16.269 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.262+0000 7f29cafad700 1 -- 172.21.15.47:0/4034731706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29c41408e0 msgr2=0x7f29c4143190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:16.269 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.262+0000 7f29cafad700 1 --2- 172.21.15.47:0/4034731706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29c41408e0 0x7f29c4143190 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 rx=0x7f29bc0032f0 tx=0x7f29bc016000).stop 2022-06-29T15:49:16.269 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.262+0000 7f29cafad700 1 -- 172.21.15.47:0/4034731706 shutdown_connections 2022-06-29T15:49:16.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.262+0000 7f29cafad700 1 --2- 172.21.15.47:0/4034731706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29c41408e0 0x7f29c4143190 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:16.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.262+0000 7f29cafad700 1 --2- 172.21.15.47:0/4034731706 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f29c40b0f30 0x7f29c4142ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:16.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.262+0000 7f29cafad700 1 --2- 172.21.15.47:0/4034731706 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f29a4053b00 0x7f29a4055fc0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:16.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.262+0000 7f29cafad700 1 --2- 172.21.15.47:0/4034731706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29c40619b0 0x7f29c4143d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:16.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.262+0000 7f29cafad700 1 -- 172.21.15.47:0/4034731706 >> 172.21.15.47:0/4034731706 conn(0x7f29c405c510 msgr2=0x7f29c40b3780 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:16.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.262+0000 7f29cafad700 1 -- 172.21.15.47:0/4034731706 shutdown_connections 2022-06-29T15:49:16.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.262+0000 7f29cafad700 1 -- 172.21.15.47:0/4034731706 wait complete. 2022-06-29T15:49:16.272 INFO:teuthology.orchestra.run.smithi047.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-06-29T15:49:16.284 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:49:16.298 INFO:tasks.ceph.mds.b.smithi064.stdout:starting mds.b at 2022-06-29T15:49:16.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.378+0000 7f5e7df82700 1 Processor -- start 2022-06-29T15:49:16.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.378+0000 7f5e7df82700 1 -- start start 2022-06-29T15:49:16.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.378+0000 7f5e7df82700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5e781430f0 0x7f5e781434d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:16.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.378+0000 7f5e7df82700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5e780a93c0 0x7f5e78095240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:16.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.378+0000 7f5e7df82700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e78060e50 con 0x7f5e78095730 2022-06-29T15:49:16.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.378+0000 7f5e7df82700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e7813d590 con 0x7f5e781430f0 2022-06-29T15:49:16.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.378+0000 7f5e7df82700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e780610f0 con 0x7f5e780a93c0 2022-06-29T15:49:16.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.378+0000 7f5e777fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5e781430f0 0x7f5e781434d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:16.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.378+0000 7f5e76ffd700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5e780a93c0 0x7f5e78095240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:16.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.378+0000 7f5e777fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5e781430f0 0x7f5e781434d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:56112/0 (socket says 172.21.15.47:56112) 2022-06-29T15:49:16.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.378+0000 7f5e77fff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f5e78095730 0x7f5e78097690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:43528/0 (socket says 172.21.15.47:43528) 2022-06-29T15:49:16.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.378+0000 7f5e777fe700 1 -- 172.21.15.47:0/1450966354 learned_addr learned my addr 172.21.15.47:0/1450966354 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:16.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.378+0000 7f5e777fe700 1 -- 172.21.15.47:0/1450966354 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5e780a93c0 msgr2=0x7f5e78095240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:16.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.378+0000 7f5e777fe700 1 --2- 172.21.15.47:0/1450966354 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5e780a93c0 0x7f5e78095240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:16.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.378+0000 7f5e777fe700 1 -- 172.21.15.47:0/1450966354 >> v1:172.21.15.64:6789/0 conn(0x7f5e78095730 legacy=0x7f5e78097690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:16.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.378+0000 7f5e777fe700 1 -- 172.21.15.47:0/1450966354 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e780971f0 con 0x7f5e781430f0 2022-06-29T15:49:16.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.378+0000 7f5e777fe700 1 --2- 172.21.15.47:0/1450966354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e781430f0 0x7f5e781434d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7f5e60002f70 tx=0x7f5e60006eb0).ready entity=mon.1 client_cookie=5990c3b377fecedb server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:16.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.378+0000 7f5e767fc700 1 -- 172.21.15.47:0/1450966354 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e6000d570 con 0x7f5e781430f0 2022-06-29T15:49:16.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.378+0000 7f5e767fc700 1 -- 172.21.15.47:0/1450966354 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e60000e90 con 0x7f5e781430f0 2022-06-29T15:49:16.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.378+0000 7f5e767fc700 1 -- 172.21.15.47:0/1450966354 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e6000d900 con 0x7f5e781430f0 2022-06-29T15:49:16.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.382+0000 7f5e7df82700 1 -- 172.21.15.47:0/1450966354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e781430f0 msgr2=0x7f5e781434d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:16.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.382+0000 7f5e7df82700 1 --2- 172.21.15.47:0/1450966354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e781430f0 0x7f5e781434d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7f5e60002f70 tx=0x7f5e60006eb0).stop 2022-06-29T15:49:16.394 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.382+0000 7f5e7df82700 1 -- 172.21.15.47:0/1450966354 shutdown_connections 2022-06-29T15:49:16.394 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.382+0000 7f5e7df82700 1 --2- 172.21.15.47:0/1450966354 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5e780a93c0 0x7f5e78095240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:16.394 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.382+0000 7f5e7df82700 1 --2- 172.21.15.47:0/1450966354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e781430f0 0x7f5e781434d0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:16.394 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.382+0000 7f5e7df82700 1 -- 172.21.15.47:0/1450966354 >> 172.21.15.47:0/1450966354 conn(0x7f5e7805f0b0 msgr2=0x7f5e7805f490 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:16.394 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.382+0000 7f5e7df82700 1 -- 172.21.15.47:0/1450966354 shutdown_connections 2022-06-29T15:49:16.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.382+0000 7f5e7df82700 1 -- 172.21.15.47:0/1450966354 wait complete. 2022-06-29T15:49:16.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.382+0000 7f5e7df82700 1 Processor -- start 2022-06-29T15:49:16.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.382+0000 7f5e7df82700 1 -- start start 2022-06-29T15:49:16.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.382+0000 7f5e7df82700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e78095730 0x7f5e781402e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:16.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.382+0000 7f5e7df82700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e780a93c0 0x7f5e7809a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:16.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.382+0000 7f5e7df82700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5e781430f0 0x7f5e780b1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:16.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.382+0000 7f5e7df82700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7813faf0 con 0x7f5e78095730 2022-06-29T15:49:16.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.382+0000 7f5e7df82700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7813eb20 con 0x7f5e780a93c0 2022-06-29T15:49:16.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.382+0000 7f5e7df82700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e780639a0 con 0x7f5e781430f0 2022-06-29T15:49:16.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.382+0000 7f5e777fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e78095730 0x7f5e781402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:16.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.382+0000 7f5e777fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e78095730 0x7f5e781402e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50414/0 (socket says 172.21.15.47:50414) 2022-06-29T15:49:16.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.382+0000 7f5e777fe700 1 -- 172.21.15.47:0/1830234638 learned_addr learned my addr 172.21.15.47:0/1830234638 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:16.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.382+0000 7f5e77fff700 1 --2- 172.21.15.47:0/1830234638 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5e781430f0 0x7f5e780b1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:16.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.382+0000 7f5e76ffd700 1 --2- 172.21.15.47:0/1830234638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e780a93c0 0x7f5e7809a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:16.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.386+0000 7f5e777fe700 1 -- 172.21.15.47:0/1830234638 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5e781430f0 msgr2=0x7f5e780b1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:16.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.386+0000 7f5e777fe700 1 --2- 172.21.15.47:0/1830234638 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5e781430f0 0x7f5e780b1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:16.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.386+0000 7f5e777fe700 1 -- 172.21.15.47:0/1830234638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e780a93c0 msgr2=0x7f5e7809a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:16.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.386+0000 7f5e777fe700 1 --2- 172.21.15.47:0/1830234638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e780a93c0 0x7f5e7809a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:16.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.386+0000 7f5e777fe700 1 -- 172.21.15.47:0/1830234638 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e6000b1e0 con 0x7f5e78095730 2022-06-29T15:49:16.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.386+0000 7f5e777fe700 1 --2- 172.21.15.47:0/1830234638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e78095730 0x7f5e781402e0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 rx=0x7f5e7813eea0 tx=0x7f5e60015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:16.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.386+0000 7f5e74ff9700 1 -- 172.21.15.47:0/1830234638 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e600068b0 con 0x7f5e78095730 2022-06-29T15:49:16.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.386+0000 7f5e74ff9700 1 -- 172.21.15.47:0/1830234638 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e6000de30 con 0x7f5e78095730 2022-06-29T15:49:16.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.386+0000 7f5e7df82700 1 -- 172.21.15.47:0/1830234638 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e7813d170 con 0x7f5e78095730 2022-06-29T15:49:16.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.386+0000 7f5e74ff9700 1 -- 172.21.15.47:0/1830234638 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e60011d30 con 0x7f5e78095730 2022-06-29T15:49:16.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.386+0000 7f5e7df82700 1 -- 172.21.15.47:0/1830234638 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e780b32b0 con 0x7f5e78095730 2022-06-29T15:49:16.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.386+0000 7f5e7df82700 1 -- 172.21.15.47:0/1830234638 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e7809a040 con 0x7f5e78095730 2022-06-29T15:49:16.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.390+0000 7f5e74ff9700 1 -- 172.21.15.47:0/1830234638 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5e60025030 con 0x7f5e78095730 2022-06-29T15:49:16.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.390+0000 7f5e74ff9700 1 --2- 172.21.15.47:0/1830234638 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5e5c053a70 0x7f5e5c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:16.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.390+0000 7f5e74ff9700 1 -- 172.21.15.47:0/1830234638 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6549+0+0 (secure 0 0 0) 0x7f5e6006d530 con 0x7f5e78095730 2022-06-29T15:49:16.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.390+0000 7f5e76ffd700 1 --2- 172.21.15.47:0/1830234638 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5e5c053a70 0x7f5e5c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:16.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.390+0000 7f5e76ffd700 1 --2- 172.21.15.47:0/1830234638 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5e5c053a70 0x7f5e5c055f30 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7f5e68008000 tx=0x7f5e68008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:16.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.398+0000 7f5e74ff9700 1 -- 172.21.15.47:0/1830234638 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5e60044270 con 0x7f5e78095730 2022-06-29T15:49:16.566 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.562+0000 7f5e7df82700 1 -- 172.21.15.47:0/1830234638 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e780573d0 con 0x7f5e78095730 2022-06-29T15:49:16.567 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.562+0000 7f5e74ff9700 1 -- 172.21.15.47:0/1830234638 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+2129 (secure 0 0 0) 0x7f5e60023090 con 0x7f5e78095730 2022-06-29T15:49:16.568 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:49:16.568 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":36,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6291,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.47:6835/2910037081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2910037081},{"type":"v1","addr":"172.21.15.47:6835","nonce":2910037081}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":34},{"gid":6300,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/359532891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":359532891},{"type":"v1","addr":"172.21.15.47:6837","nonce":359532891}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":33}],"filesystems":[{"mdsmap":{"epoch":36,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:48:18.345845+0000","modified":"2022-06-29T15:49:16.248746+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2022-06-29T15:49:16.570 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.562+0000 7f5e7df82700 1 -- 172.21.15.47:0/1830234638 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5e5c053a70 msgr2=0x7f5e5c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:16.570 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.562+0000 7f5e7df82700 1 --2- 172.21.15.47:0/1830234638 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5e5c053a70 0x7f5e5c055f30 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7f5e68008000 tx=0x7f5e68008040).stop 2022-06-29T15:49:16.570 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.562+0000 7f5e7df82700 1 -- 172.21.15.47:0/1830234638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e78095730 msgr2=0x7f5e781402e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:16.570 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.562+0000 7f5e7df82700 1 --2- 172.21.15.47:0/1830234638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e78095730 0x7f5e781402e0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 rx=0x7f5e7813eea0 tx=0x7f5e60015000).stop 2022-06-29T15:49:16.571 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.562+0000 7f5e7df82700 1 -- 172.21.15.47:0/1830234638 shutdown_connections 2022-06-29T15:49:16.571 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.562+0000 7f5e7df82700 1 --2- 172.21.15.47:0/1830234638 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5e781430f0 0x7f5e780b1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:16.571 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.562+0000 7f5e7df82700 1 --2- 172.21.15.47:0/1830234638 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5e5c053a70 0x7f5e5c055f30 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:16.571 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.562+0000 7f5e7df82700 1 --2- 172.21.15.47:0/1830234638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e780a93c0 0x7f5e7809a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:16.572 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.562+0000 7f5e7df82700 1 --2- 172.21.15.47:0/1830234638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e78095730 0x7f5e781402e0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:16.572 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.562+0000 7f5e7df82700 1 -- 172.21.15.47:0/1830234638 >> 172.21.15.47:0/1830234638 conn(0x7f5e7805f0b0 msgr2=0x7f5e78137cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:16.572 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.562+0000 7f5e7df82700 1 -- 172.21.15.47:0/1830234638 shutdown_connections 2022-06-29T15:49:16.572 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.562+0000 7f5e7df82700 1 -- 172.21.15.47:0/1830234638 wait complete. 2022-06-29T15:49:16.573 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 36 2022-06-29T15:49:16.585 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-06-29T15:49:16.693 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.686+0000 7f41a4901700 1 Processor -- start 2022-06-29T15:49:16.693 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.686+0000 7f41a4901700 1 -- start start 2022-06-29T15:49:16.696 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.686+0000 7f41a4901700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f41a00b58d0 0x7f41a00b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:16.696 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.686+0000 7f41a4901700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f41a00b1610 0x7f41a01475f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:16.697 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.686+0000 7f41a4901700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41a0060790 con 0x7f41a014b4c0 2022-06-29T15:49:16.697 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.686+0000 7f41a4901700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41a00a99e0 con 0x7f41a00b1610 2022-06-29T15:49:16.697 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.686+0000 7f41a4901700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41a014ad80 con 0x7f41a00b58d0 2022-06-29T15:49:16.697 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.690+0000 7f419f59e700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f41a014b4c0 0x7f41a00aa130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35980/0 (socket says 172.21.15.47:35980) 2022-06-29T15:49:16.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.690+0000 7f419f59e700 1 -- 172.21.15.47:0/779907775 learned_addr learned my addr 172.21.15.47:0/779907775 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:49:16.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.690+0000 7f419e59c700 1 --2- 172.21.15.47:0/779907775 >> v2:172.21.15.64:3300/0 conn(0x7f41a00b1610 0x7f41a01475f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:16.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.690+0000 7f419ed9d700 1 --2- 172.21.15.47:0/779907775 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f41a00b58d0 0x7f41a00b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:16.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.690+0000 7f419dd9b700 1 -- 172.21.15.47:0/779907775 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3119847220 0 0) 0x7f41a0060790 con 0x7f41a014b4c0 2022-06-29T15:49:16.699 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.690+0000 7f419dd9b700 1 -- 172.21.15.47:0/779907775 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4184003260 con 0x7f41a014b4c0 2022-06-29T15:49:16.699 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.690+0000 7f419e59c700 1 -- 172.21.15.47:0/779907775 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f41a00b58d0 msgr2=0x7f41a00b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:16.699 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.690+0000 7f419e59c700 1 --2- 172.21.15.47:0/779907775 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f41a00b58d0 0x7f41a00b5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:16.699 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.690+0000 7f419e59c700 1 -- 172.21.15.47:0/779907775 >> v1:172.21.15.47:6789/0 conn(0x7f41a014b4c0 legacy=0x7f41a00aa130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:16.700 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.690+0000 7f419e59c700 1 -- 172.21.15.47:0/779907775 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a014b020 con 0x7f41a00b1610 2022-06-29T15:49:16.700 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.690+0000 7f419e59c700 1 --2- 172.21.15.47:0/779907775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41a00b1610 0x7f41a01475f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f41a00623e0 tx=0x7f41900091d0).ready entity=mon.1 client_cookie=6ea1b0d5383bd64b server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:16.700 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.690+0000 7f419dd9b700 1 -- 172.21.15.47:0/779907775 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f419000b600 con 0x7f41a00b1610 2022-06-29T15:49:16.700 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.690+0000 7f419dd9b700 1 -- 172.21.15.47:0/779907775 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f419000b760 con 0x7f41a00b1610 2022-06-29T15:49:16.701 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.690+0000 7f419dd9b700 1 -- 172.21.15.47:0/779907775 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f419000b8d0 con 0x7f41a00b1610 2022-06-29T15:49:16.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.690+0000 7f41a4901700 1 -- 172.21.15.47:0/779907775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41a00b1610 msgr2=0x7f41a01475f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:16.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.690+0000 7f41a4901700 1 --2- 172.21.15.47:0/779907775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41a00b1610 0x7f41a01475f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f41a00623e0 tx=0x7f41900091d0).stop 2022-06-29T15:49:16.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.690+0000 7f41a4901700 1 -- 172.21.15.47:0/779907775 shutdown_connections 2022-06-29T15:49:16.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.690+0000 7f41a4901700 1 --2- 172.21.15.47:0/779907775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41a00b1610 0x7f41a01475f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:16.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.690+0000 7f41a4901700 1 --2- 172.21.15.47:0/779907775 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f41a00b58d0 0x7f41a00b5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:16.707 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.690+0000 7f41a4901700 1 -- 172.21.15.47:0/779907775 >> 172.21.15.47:0/779907775 conn(0x7f41a0062560 msgr2=0x7f41a0062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:16.707 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.690+0000 7f41a4901700 1 -- 172.21.15.47:0/779907775 shutdown_connections 2022-06-29T15:49:16.707 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.690+0000 7f41a4901700 1 -- 172.21.15.47:0/779907775 wait complete. 2022-06-29T15:49:16.707 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.694+0000 7f41a4901700 1 Processor -- start 2022-06-29T15:49:16.708 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.694+0000 7f41a4901700 1 -- start start 2022-06-29T15:49:16.708 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.694+0000 7f41a4901700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41a00b1610 0x7f41a00b4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:16.708 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.694+0000 7f41a4901700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41a00b58d0 0x7f41a00aed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:16.708 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.694+0000 7f41a4901700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f41a014b4c0 0x7f41a0063130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:16.709 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.694+0000 7f41a4901700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a00a8e60 con 0x7f41a00b1610 2022-06-29T15:49:16.709 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.694+0000 7f41a4901700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a00ae6a0 con 0x7f41a00b58d0 2022-06-29T15:49:16.709 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.694+0000 7f41a4901700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41a00ae940 con 0x7f41a014b4c0 2022-06-29T15:49:16.709 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.694+0000 7f419ed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41a00b1610 0x7f41a00b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:16.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.694+0000 7f419ed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41a00b1610 0x7f41a00b4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50428/0 (socket says 172.21.15.47:50428) 2022-06-29T15:49:16.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.694+0000 7f419ed9d700 1 -- 172.21.15.47:0/643446102 learned_addr learned my addr 172.21.15.47:0/643446102 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:16.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.694+0000 7f419e59c700 1 --2- 172.21.15.47:0/643446102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41a00b58d0 0x7f41a00aed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:16.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.694+0000 7f419f59e700 1 --2- 172.21.15.47:0/643446102 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f41a014b4c0 0x7f41a0063130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:16.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.694+0000 7f419ed9d700 1 -- 172.21.15.47:0/643446102 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f41a014b4c0 msgr2=0x7f41a0063130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:16.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.694+0000 7f419ed9d700 1 --2- 172.21.15.47:0/643446102 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f41a014b4c0 0x7f41a0063130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:16.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.694+0000 7f419ed9d700 1 -- 172.21.15.47:0/643446102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41a00b58d0 msgr2=0x7f41a00aed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:16.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.694+0000 7f419ed9d700 1 --2- 172.21.15.47:0/643446102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41a00b58d0 0x7f41a00aed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:16.712 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.694+0000 7f419ed9d700 1 -- 172.21.15.47:0/643446102 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4190009470 con 0x7f41a00b1610 2022-06-29T15:49:16.712 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.694+0000 7f419ed9d700 1 --2- 172.21.15.47:0/643446102 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41a00b1610 0x7f41a00b4630 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 rx=0x7f419400c000 tx=0x7f4194006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:16.712 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.694+0000 7f418ffff700 1 -- 172.21.15.47:0/643446102 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4194010780 con 0x7f41a00b1610 2022-06-29T15:49:16.713 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.694+0000 7f418ffff700 1 -- 172.21.15.47:0/643446102 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41940108e0 con 0x7f41a00b1610 2022-06-29T15:49:16.713 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.694+0000 7f41a4901700 1 -- 172.21.15.47:0/643446102 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41a00af1f0 con 0x7f41a00b1610 2022-06-29T15:49:16.713 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.694+0000 7f418ffff700 1 -- 172.21.15.47:0/643446102 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4194010c40 con 0x7f41a00b1610 2022-06-29T15:49:16.713 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.694+0000 7f41a4901700 1 -- 172.21.15.47:0/643446102 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41a00ae120 con 0x7f41a00b1610 2022-06-29T15:49:16.714 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.698+0000 7f41a4901700 1 -- 172.21.15.47:0/643446102 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41a00a8fd0 con 0x7f41a00b1610 2022-06-29T15:49:16.714 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.698+0000 7f418ffff700 1 -- 172.21.15.47:0/643446102 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f419401c460 con 0x7f41a00b1610 2022-06-29T15:49:16.714 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.698+0000 7f418ffff700 1 --2- 172.21.15.47:0/643446102 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f417c053a70 0x7f417c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:16.714 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.698+0000 7f418ffff700 1 -- 172.21.15.47:0/643446102 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6549+0+0 (secure 0 0 0) 0x7f419406c670 con 0x7f41a00b1610 2022-06-29T15:49:16.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.698+0000 7f419e59c700 1 --2- 172.21.15.47:0/643446102 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f417c053a70 0x7f417c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:16.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.698+0000 7f419e59c700 1 --2- 172.21.15.47:0/643446102 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f417c053a70 0x7f417c055f30 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f419001b000 tx=0x7f4190016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:16.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.706+0000 7f418ffff700 1 -- 172.21.15.47:0/643446102 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f41a00a8fd0 con 0x7f41a00b1610 2022-06-29T15:49:16.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:16.874+0000 7f41a4901700 1 -- 172.21.15.47:0/643446102 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f41a00a8fa0 con 0x7f41a00b1610 2022-06-29T15:49:17.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.266+0000 7f418ffff700 1 -- 172.21.15.47:0/643446102 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v37) v1 ==== 104+0+0 (secure 0 0 0) 0x7f41940432b0 con 0x7f41a00b1610 2022-06-29T15:49:17.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.270+0000 7f41a4901700 1 -- 172.21.15.47:0/643446102 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f417c053a70 msgr2=0x7f417c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:17.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.270+0000 7f41a4901700 1 --2- 172.21.15.47:0/643446102 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f417c053a70 0x7f417c055f30 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f419001b000 tx=0x7f4190016000).stop 2022-06-29T15:49:17.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.270+0000 7f41a4901700 1 -- 172.21.15.47:0/643446102 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41a00b1610 msgr2=0x7f41a00b4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:17.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.270+0000 7f41a4901700 1 --2- 172.21.15.47:0/643446102 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41a00b1610 0x7f41a00b4630 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 rx=0x7f419400c000 tx=0x7f4194006df0).stop 2022-06-29T15:49:17.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.270+0000 7f41a4901700 1 -- 172.21.15.47:0/643446102 shutdown_connections 2022-06-29T15:49:17.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.270+0000 7f41a4901700 1 --2- 172.21.15.47:0/643446102 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f41a014b4c0 0x7f41a0063130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:17.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.270+0000 7f41a4901700 1 --2- 172.21.15.47:0/643446102 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f417c053a70 0x7f417c055f30 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:17.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.270+0000 7f41a4901700 1 --2- 172.21.15.47:0/643446102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41a00b58d0 0x7f41a00aed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:17.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.270+0000 7f41a4901700 1 --2- 172.21.15.47:0/643446102 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41a00b1610 0x7f41a00b4630 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:17.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.270+0000 7f41a4901700 1 -- 172.21.15.47:0/643446102 >> 172.21.15.47:0/643446102 conn(0x7f41a0062560 msgr2=0x7f41a00b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:17.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.270+0000 7f41a4901700 1 -- 172.21.15.47:0/643446102 shutdown_connections 2022-06-29T15:49:17.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.270+0000 7f41a4901700 1 -- 172.21.15.47:0/643446102 wait complete. 2022-06-29T15:49:17.293 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-06-29T15:49:17.409 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.402+0000 7ffb5ea16700 1 Processor -- start 2022-06-29T15:49:17.409 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.402+0000 7ffb5ea16700 1 -- start start 2022-06-29T15:49:17.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.402+0000 7ffb5ea16700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ffb580b6d20 0x7ffb580a9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:17.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.402+0000 7ffb5ea16700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffb5805f350 0x7ffb580612b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:17.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.402+0000 7ffb5ea16700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb580b7100 con 0x7ffb58063100 2022-06-29T15:49:17.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.402+0000 7ffb5ea16700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb5805fdc0 con 0x7ffb580b6d20 2022-06-29T15:49:17.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.402+0000 7ffb5ea16700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb58141db0 con 0x7ffb5805f350 2022-06-29T15:49:17.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.406+0000 7ffb5d213700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffb5805f350 0x7ffb580612b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:17.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.406+0000 7ffb5da14700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ffb580b6d20 0x7ffb580a9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:17.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.406+0000 7ffb5d213700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffb5805f350 0x7ffb580612b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:47540/0 (socket says 172.21.15.47:47540) 2022-06-29T15:49:17.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.406+0000 7ffb5e215700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ffb58063100 0x7ffb5813e2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:43556/0 (socket says 172.21.15.47:43556) 2022-06-29T15:49:17.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.406+0000 7ffb5d213700 1 -- 172.21.15.47:0/3179077821 learned_addr learned my addr 172.21.15.47:0/3179077821 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:17.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.406+0000 7ffb5d213700 1 -- 172.21.15.47:0/3179077821 >> v2:172.21.15.64:3300/0 conn(0x7ffb580b6d20 msgr2=0x7ffb580a9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:17.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.406+0000 7ffb5d213700 1 --2- 172.21.15.47:0/3179077821 >> v2:172.21.15.64:3300/0 conn(0x7ffb580b6d20 0x7ffb580a9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:17.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.406+0000 7ffb5d213700 1 -- 172.21.15.47:0/3179077821 >> v1:172.21.15.64:6789/0 conn(0x7ffb58063100 legacy=0x7ffb5813e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:17.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.406+0000 7ffb5ca12700 1 -- 172.21.15.47:0/3179077821 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2145481558 0 0) 0x7ffb580b7100 con 0x7ffb58063100 2022-06-29T15:49:17.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.406+0000 7ffb5d213700 1 -- 172.21.15.47:0/3179077821 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb5805faf0 con 0x7ffb5805f350 2022-06-29T15:49:17.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.406+0000 7ffb5d213700 1 --2- 172.21.15.47:0/3179077821 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffb5805f350 0x7ffb580612b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7ffb54002f70 tx=0x7ffb54006eb0).ready entity=mon.2 client_cookie=70e9559c0a434f28 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:17.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.406+0000 7ffb5ca12700 1 -- 172.21.15.47:0/3179077821 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb5400d570 con 0x7ffb5805f350 2022-06-29T15:49:17.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.406+0000 7ffb5ca12700 1 -- 172.21.15.47:0/3179077821 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb54000e90 con 0x7ffb5805f350 2022-06-29T15:49:17.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.406+0000 7ffb5ca12700 1 -- 172.21.15.47:0/3179077821 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb5400d8c0 con 0x7ffb5805f350 2022-06-29T15:49:17.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.406+0000 7ffb5ea16700 1 -- 172.21.15.47:0/3179077821 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffb5805f350 msgr2=0x7ffb580612b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:17.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.406+0000 7ffb5ea16700 1 --2- 172.21.15.47:0/3179077821 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffb5805f350 0x7ffb580612b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7ffb54002f70 tx=0x7ffb54006eb0).stop 2022-06-29T15:49:17.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.406+0000 7ffb5ea16700 1 -- 172.21.15.47:0/3179077821 shutdown_connections 2022-06-29T15:49:17.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.406+0000 7ffb5ea16700 1 --2- 172.21.15.47:0/3179077821 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffb5805f350 0x7ffb580612b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:17.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.406+0000 7ffb5ea16700 1 --2- 172.21.15.47:0/3179077821 >> v2:172.21.15.64:3300/0 conn(0x7ffb580b6d20 0x7ffb580a9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:17.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.406+0000 7ffb5ea16700 1 -- 172.21.15.47:0/3179077821 >> 172.21.15.47:0/3179077821 conn(0x7ffb58063670 msgr2=0x7ffb58063a50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:17.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.406+0000 7ffb5ea16700 1 -- 172.21.15.47:0/3179077821 shutdown_connections 2022-06-29T15:49:17.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.406+0000 7ffb5ea16700 1 -- 172.21.15.47:0/3179077821 wait complete. 2022-06-29T15:49:17.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.410+0000 7ffb5ea16700 1 Processor -- start 2022-06-29T15:49:17.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.410+0000 7ffb5ea16700 1 -- start start 2022-06-29T15:49:17.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.410+0000 7ffb5ea16700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffb5805f350 0x7ffb5813bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:17.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.410+0000 7ffb5ea16700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb58063100 0x7ffb58150ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:17.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.410+0000 7ffb5ea16700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb580b6d20 0x7ffb58153b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:17.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.410+0000 7ffb5ea16700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb58150570 con 0x7ffb58063100 2022-06-29T15:49:17.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.410+0000 7ffb5ea16700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5813d0f0 con 0x7ffb580b6d20 2022-06-29T15:49:17.424 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.410+0000 7ffb5ea16700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5813d390 con 0x7ffb5805f350 2022-06-29T15:49:17.424 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.410+0000 7ffb5d213700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb58063100 0x7ffb58150ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:17.424 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.410+0000 7ffb5d213700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb58063100 0x7ffb58150ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50446/0 (socket says 172.21.15.47:50446) 2022-06-29T15:49:17.424 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.410+0000 7ffb5d213700 1 -- 172.21.15.47:0/3466397012 learned_addr learned my addr 172.21.15.47:0/3466397012 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:17.425 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.410+0000 7ffb5da14700 1 --2- 172.21.15.47:0/3466397012 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffb5805f350 0x7ffb5813bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:17.425 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.410+0000 7ffb5e215700 1 --2- 172.21.15.47:0/3466397012 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb580b6d20 0x7ffb58153b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:17.425 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.410+0000 7ffb5d213700 1 -- 172.21.15.47:0/3466397012 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffb5805f350 msgr2=0x7ffb5813bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:17.425 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.410+0000 7ffb5d213700 1 --2- 172.21.15.47:0/3466397012 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffb5805f350 0x7ffb5813bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:17.425 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.410+0000 7ffb5d213700 1 -- 172.21.15.47:0/3466397012 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb580b6d20 msgr2=0x7ffb58153b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:17.426 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.410+0000 7ffb5d213700 1 --2- 172.21.15.47:0/3466397012 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb580b6d20 0x7ffb58153b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:17.426 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.410+0000 7ffb5d213700 1 -- 172.21.15.47:0/3466397012 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb5400b1e0 con 0x7ffb58063100 2022-06-29T15:49:17.426 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.410+0000 7ffb5e215700 1 --2- 172.21.15.47:0/3466397012 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb580b6d20 0x7ffb58153b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:49:17.426 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.410+0000 7ffb5d213700 1 --2- 172.21.15.47:0/3466397012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb58063100 0x7ffb58150ea0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 rx=0x7ffb54002f70 tx=0x7ffb54015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:17.427 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.410+0000 7ffb4affd700 1 -- 172.21.15.47:0/3466397012 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb54011860 con 0x7ffb58063100 2022-06-29T15:49:17.427 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.410+0000 7ffb4affd700 1 -- 172.21.15.47:0/3466397012 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb540119c0 con 0x7ffb58063100 2022-06-29T15:49:17.427 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.410+0000 7ffb4affd700 1 -- 172.21.15.47:0/3466397012 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb54011d20 con 0x7ffb58063100 2022-06-29T15:49:17.427 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.410+0000 7ffb5ea16700 1 -- 172.21.15.47:0/3466397012 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb58150760 con 0x7ffb58063100 2022-06-29T15:49:17.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.410+0000 7ffb5ea16700 1 -- 172.21.15.47:0/3466397012 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb5813c270 con 0x7ffb58063100 2022-06-29T15:49:17.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.414+0000 7ffb5ea16700 1 -- 172.21.15.47:0/3466397012 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb5813d590 con 0x7ffb58063100 2022-06-29T15:49:17.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.414+0000 7ffb4affd700 1 -- 172.21.15.47:0/3466397012 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ffb5401a040 con 0x7ffb58063100 2022-06-29T15:49:17.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.414+0000 7ffb4affd700 1 --2- 172.21.15.47:0/3466397012 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ffb38053aa0 0x7ffb38055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:17.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.414+0000 7ffb4affd700 1 -- 172.21.15.47:0/3466397012 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6549+0+0 (secure 0 0 0) 0x7ffb5406cf20 con 0x7ffb58063100 2022-06-29T15:49:17.429 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.414+0000 7ffb5da14700 1 --2- 172.21.15.47:0/3466397012 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ffb38053aa0 0x7ffb38055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:17.430 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.414+0000 7ffb5da14700 1 --2- 172.21.15.47:0/3466397012 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ffb38053aa0 0x7ffb38055f60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7ffb4c00b000 tx=0x7ffb4c006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:17.430 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.422+0000 7ffb4affd700 1 -- 172.21.15.47:0/3466397012 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ffb54043c60 con 0x7ffb58063100 2022-06-29T15:49:17.555 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:17.550+0000 7ffb5ea16700 1 -- 172.21.15.47:0/3466397012 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ffb580573d0 con 0x7ffb58063100 2022-06-29T15:49:18.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.330+0000 7ffb4affd700 1 -- 172.21.15.47:0/3466397012 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v81) v1 ==== 185+0+0 (secure 0 0 0) 0x7ffb54023020 con 0x7ffb58063100 2022-06-29T15:49:18.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.334+0000 7ffb5ea16700 1 -- 172.21.15.47:0/3466397012 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ffb38053aa0 msgr2=0x7ffb38055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:18.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.334+0000 7ffb5ea16700 1 --2- 172.21.15.47:0/3466397012 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ffb38053aa0 0x7ffb38055f60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7ffb4c00b000 tx=0x7ffb4c006df0).stop 2022-06-29T15:49:18.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.334+0000 7ffb5ea16700 1 -- 172.21.15.47:0/3466397012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb58063100 msgr2=0x7ffb58150ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:18.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.334+0000 7ffb5ea16700 1 --2- 172.21.15.47:0/3466397012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb58063100 0x7ffb58150ea0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 rx=0x7ffb54002f70 tx=0x7ffb54015000).stop 2022-06-29T15:49:18.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.338+0000 7ffb5ea16700 1 -- 172.21.15.47:0/3466397012 shutdown_connections 2022-06-29T15:49:18.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.338+0000 7ffb5ea16700 1 --2- 172.21.15.47:0/3466397012 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ffb38053aa0 0x7ffb38055f60 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:18.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.338+0000 7ffb5ea16700 1 --2- 172.21.15.47:0/3466397012 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb580b6d20 0x7ffb58153b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:18.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.338+0000 7ffb5ea16700 1 --2- 172.21.15.47:0/3466397012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb58063100 0x7ffb58150ea0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:18.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.338+0000 7ffb5ea16700 1 --2- 172.21.15.47:0/3466397012 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffb5805f350 0x7ffb5813bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:18.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.338+0000 7ffb5ea16700 1 -- 172.21.15.47:0/3466397012 >> 172.21.15.47:0/3466397012 conn(0x7ffb58063670 msgr2=0x7ffb58140130 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:18.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.338+0000 7ffb5ea16700 1 -- 172.21.15.47:0/3466397012 shutdown_connections 2022-06-29T15:49:18.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.338+0000 7ffb5ea16700 1 -- 172.21.15.47:0/3466397012 wait complete. 2022-06-29T15:49:18.346 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' removed 2022-06-29T15:49:18.358 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-06-29T15:49:18.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.470+0000 7f4c622e3700 1 Processor -- start 2022-06-29T15:49:18.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.470+0000 7f4c622e3700 1 -- start start 2022-06-29T15:49:18.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.474+0000 7f4c622e3700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4c5c0601a0 0x7f4c5c0afd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:18.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.474+0000 7f4c622e3700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c5c0b5ff0 con 0x7f4c5c0b5c10 2022-06-29T15:49:18.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.474+0000 7f4c622e3700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c5c0ac910 con 0x7f4c5c13e9a0 2022-06-29T15:49:18.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.474+0000 7f4c622e3700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c5c0acbb0 con 0x7f4c5c0601a0 2022-06-29T15:49:18.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.474+0000 7f4c60ae0700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4c5c0601a0 0x7f4c5c0afd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:18.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.474+0000 7f4c612e1700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4c5c0b5c10 0x7f4c5c13e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36004/0 (socket says 172.21.15.47:36004) 2022-06-29T15:49:18.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.474+0000 7f4c612e1700 1 -- 172.21.15.47:0/1697905858 learned_addr learned my addr 172.21.15.47:0/1697905858 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:49:18.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.474+0000 7f4c60ae0700 1 -- 172.21.15.47:0/1697905858 >> v1:172.21.15.64:6789/0 conn(0x7f4c5c13e9a0 legacy=0x7f4c5c0b0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:18.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.474+0000 7f4c60ae0700 1 -- 172.21.15.47:0/1697905858 >> v1:172.21.15.47:6789/0 conn(0x7f4c5c0b5c10 legacy=0x7f4c5c13e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:18.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.474+0000 7f4c60ae0700 1 -- 172.21.15.47:0/1697905858 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c5c0ace50 con 0x7f4c5c0601a0 2022-06-29T15:49:18.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.474+0000 7f4c47fff700 1 -- 172.21.15.47:0/1697905858 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 904936010 0 0) 0x7f4c5c0b5ff0 con 0x7f4c5c0b5c10 2022-06-29T15:49:18.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.474+0000 7f4c60ae0700 1 --2- 172.21.15.47:0/1697905858 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c5c0601a0 0x7f4c5c0afd60 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 rx=0x7f4c50002f70 tx=0x7f4c50006df0).ready entity=mon.0 client_cookie=4da309ce5f26d76d server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:18.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.474+0000 7f4c47fff700 1 -- 172.21.15.47:0/1697905858 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c5000d610 con 0x7f4c5c0601a0 2022-06-29T15:49:18.484 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.474+0000 7f4c47fff700 1 -- 172.21.15.47:0/1697905858 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c50000e90 con 0x7f4c5c0601a0 2022-06-29T15:49:18.484 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.474+0000 7f4c47fff700 1 -- 172.21.15.47:0/1697905858 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c5000d960 con 0x7f4c5c0601a0 2022-06-29T15:49:18.484 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.474+0000 7f4c622e3700 1 -- 172.21.15.47:0/1697905858 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c5c0601a0 msgr2=0x7f4c5c0afd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:18.484 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.474+0000 7f4c622e3700 1 --2- 172.21.15.47:0/1697905858 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c5c0601a0 0x7f4c5c0afd60 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 rx=0x7f4c50002f70 tx=0x7f4c50006df0).stop 2022-06-29T15:49:18.485 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.474+0000 7f4c622e3700 1 -- 172.21.15.47:0/1697905858 shutdown_connections 2022-06-29T15:49:18.485 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.474+0000 7f4c622e3700 1 --2- 172.21.15.47:0/1697905858 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c5c0601a0 0x7f4c5c0afd60 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:18.485 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.474+0000 7f4c622e3700 1 -- 172.21.15.47:0/1697905858 >> 172.21.15.47:0/1697905858 conn(0x7f4c5c062560 msgr2=0x7f4c5c062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:18.485 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.474+0000 7f4c622e3700 1 -- 172.21.15.47:0/1697905858 shutdown_connections 2022-06-29T15:49:18.485 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.474+0000 7f4c622e3700 1 -- 172.21.15.47:0/1697905858 wait complete. 2022-06-29T15:49:18.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.478+0000 7f4c622e3700 1 Processor -- start 2022-06-29T15:49:18.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.478+0000 7f4c622e3700 1 -- start start 2022-06-29T15:49:18.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.478+0000 7f4c622e3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c5c0601a0 0x7f4c5c1413d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:18.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.478+0000 7f4c622e3700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4c5c0b5c10 0x7f4c5c141a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:18.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.478+0000 7f4c622e3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c5c13e9a0 0x7f4c5c138680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:18.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.478+0000 7f4c622e3700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5c144ce0 con 0x7f4c5c13e9a0 2022-06-29T15:49:18.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.478+0000 7f4c622e3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5c144780 con 0x7f4c5c0601a0 2022-06-29T15:49:18.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.478+0000 7f4c622e3700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5c144a40 con 0x7f4c5c0b5c10 2022-06-29T15:49:18.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.478+0000 7f4c61ae2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c5c13e9a0 0x7f4c5c138680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:18.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.478+0000 7f4c61ae2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c5c13e9a0 0x7f4c5c138680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50460/0 (socket says 172.21.15.47:50460) 2022-06-29T15:49:18.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.478+0000 7f4c61ae2700 1 -- 172.21.15.47:0/767279595 learned_addr learned my addr 172.21.15.47:0/767279595 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:18.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.478+0000 7f4c60ae0700 1 --2- 172.21.15.47:0/767279595 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4c5c0b5c10 0x7f4c5c141a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:18.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.478+0000 7f4c612e1700 1 --2- 172.21.15.47:0/767279595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c5c0601a0 0x7f4c5c1413d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:18.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.478+0000 7f4c61ae2700 1 -- 172.21.15.47:0/767279595 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4c5c0b5c10 msgr2=0x7f4c5c141a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:18.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.478+0000 7f4c61ae2700 1 --2- 172.21.15.47:0/767279595 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4c5c0b5c10 0x7f4c5c141a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:18.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.478+0000 7f4c61ae2700 1 -- 172.21.15.47:0/767279595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c5c0601a0 msgr2=0x7f4c5c1413d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:18.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.478+0000 7f4c61ae2700 1 --2- 172.21.15.47:0/767279595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c5c0601a0 0x7f4c5c1413d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:18.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.478+0000 7f4c61ae2700 1 -- 172.21.15.47:0/767279595 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c5000b180 con 0x7f4c5c13e9a0 2022-06-29T15:49:18.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.478+0000 7f4c61ae2700 1 --2- 172.21.15.47:0/767279595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c5c13e9a0 0x7f4c5c138680 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7f4c5c0621e0 tx=0x7f4c540013a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:18.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.478+0000 7f4c4effd700 1 -- 172.21.15.47:0/767279595 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c54006aa0 con 0x7f4c5c13e9a0 2022-06-29T15:49:18.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.478+0000 7f4c4effd700 1 -- 172.21.15.47:0/767279595 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c54006c00 con 0x7f4c5c13e9a0 2022-06-29T15:49:18.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.478+0000 7f4c622e3700 1 -- 172.21.15.47:0/767279595 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c5c13fa70 con 0x7f4c5c13e9a0 2022-06-29T15:49:18.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.478+0000 7f4c4effd700 1 -- 172.21.15.47:0/767279595 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c54007030 con 0x7f4c5c13e9a0 2022-06-29T15:49:18.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.478+0000 7f4c622e3700 1 -- 172.21.15.47:0/767279595 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c5c0b50f0 con 0x7f4c5c13e9a0 2022-06-29T15:49:18.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.482+0000 7f4c622e3700 1 -- 172.21.15.47:0/767279595 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c5c1404f0 con 0x7f4c5c13e9a0 2022-06-29T15:49:18.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.482+0000 7f4c4effd700 1 -- 172.21.15.47:0/767279595 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4c54010030 con 0x7f4c5c13e9a0 2022-06-29T15:49:18.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.482+0000 7f4c4effd700 1 --2- 172.21.15.47:0/767279595 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4c38053a70 0x7f4c38055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:18.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.482+0000 7f4c4effd700 1 -- 172.21.15.47:0/767279595 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6111+0+0 (secure 0 0 0) 0x7f4c5406bcf0 con 0x7f4c5c13e9a0 2022-06-29T15:49:18.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.482+0000 7f4c612e1700 1 --2- 172.21.15.47:0/767279595 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4c38053a70 0x7f4c38055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:18.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.482+0000 7f4c612e1700 1 --2- 172.21.15.47:0/767279595 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4c38053a70 0x7f4c38055f30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7f4c5800d000 tx=0x7f4c58007a00).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:18.499 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.494+0000 7f4c4effd700 1 -- 172.21.15.47:0/767279595 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4c5c1404f0 con 0x7f4c5c13e9a0 2022-06-29T15:49:18.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:18.622+0000 7f4c622e3700 1 -- 172.21.15.47:0/767279595 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4c5c1404f0 con 0x7f4c5c13e9a0 2022-06-29T15:49:19.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.342+0000 7f4c4effd700 1 -- 172.21.15.47:0/767279595 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v82) v1 ==== 173+0+0 (secure 0 0 0) 0x7f4c5c1404f0 con 0x7f4c5c13e9a0 2022-06-29T15:49:19.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.346+0000 7f4c622e3700 1 -- 172.21.15.47:0/767279595 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4c38053a70 msgr2=0x7f4c38055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:19.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.346+0000 7f4c622e3700 1 --2- 172.21.15.47:0/767279595 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4c38053a70 0x7f4c38055f30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7f4c5800d000 tx=0x7f4c58007a00).stop 2022-06-29T15:49:19.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.346+0000 7f4c622e3700 1 -- 172.21.15.47:0/767279595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c5c13e9a0 msgr2=0x7f4c5c138680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:19.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.346+0000 7f4c622e3700 1 --2- 172.21.15.47:0/767279595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c5c13e9a0 0x7f4c5c138680 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7f4c5c0621e0 tx=0x7f4c540013a0).stop 2022-06-29T15:49:19.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.346+0000 7f4c622e3700 1 -- 172.21.15.47:0/767279595 shutdown_connections 2022-06-29T15:49:19.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.346+0000 7f4c622e3700 1 --2- 172.21.15.47:0/767279595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c5c13e9a0 0x7f4c5c138680 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:19.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.346+0000 7f4c622e3700 1 --2- 172.21.15.47:0/767279595 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4c5c0b5c10 0x7f4c5c141a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:19.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.346+0000 7f4c622e3700 1 --2- 172.21.15.47:0/767279595 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4c38053a70 0x7f4c38055f30 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:19.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.346+0000 7f4c622e3700 1 --2- 172.21.15.47:0/767279595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c5c0601a0 0x7f4c5c1413d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:19.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.346+0000 7f4c622e3700 1 -- 172.21.15.47:0/767279595 >> 172.21.15.47:0/767279595 conn(0x7f4c5c062560 msgr2=0x7f4c5c0b1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:19.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.346+0000 7f4c622e3700 1 -- 172.21.15.47:0/767279595 shutdown_connections 2022-06-29T15:49:19.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.346+0000 7f4c622e3700 1 -- 172.21.15.47:0/767279595 wait complete. 2022-06-29T15:49:19.357 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' removed 2022-06-29T15:49:19.368 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-06-29T15:49:19.484 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.478+0000 7fd38d6b8700 1 Processor -- start 2022-06-29T15:49:19.485 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.478+0000 7fd38d6b8700 1 -- start start 2022-06-29T15:49:19.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.478+0000 7fd38d6b8700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd3880612b0 0x7fd388148dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:19.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.478+0000 7fd38d6b8700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd388062f30 0x7fd388066860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:19.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.478+0000 7fd38d6b8700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3880b3d30 con 0x7fd3880b6d20 2022-06-29T15:49:19.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.478+0000 7fd38d6b8700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3880b17f0 con 0x7fd3880612b0 2022-06-29T15:49:19.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.478+0000 7fd38d6b8700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3880b7100 con 0x7fd388062f30 2022-06-29T15:49:19.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.478+0000 7fd3877fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd3880612b0 0x7fd388148dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:19.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.478+0000 7fd3877fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd3880612b0 0x7fd388148dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50466/0 (socket says 172.21.15.47:50466) 2022-06-29T15:49:19.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.478+0000 7fd3877fe700 1 -- 172.21.15.47:0/3475666580 learned_addr learned my addr 172.21.15.47:0/3475666580 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:19.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.478+0000 7fd38ceb7700 1 --2- 172.21.15.47:0/3475666580 >> v2:172.21.15.64:3300/0 conn(0x7fd388062f30 0x7fd388066860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:19.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.478+0000 7fd3877fe700 1 -- 172.21.15.47:0/3475666580 >> v2:172.21.15.64:3300/0 conn(0x7fd388062f30 msgr2=0x7fd388066860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:19.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.478+0000 7fd3877fe700 1 --2- 172.21.15.47:0/3475666580 >> v2:172.21.15.64:3300/0 conn(0x7fd388062f30 0x7fd388066860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:19.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.478+0000 7fd3877fe700 1 -- 172.21.15.47:0/3475666580 >> v1:172.21.15.64:6789/0 conn(0x7fd3880b6d20 legacy=0x7fd3880b3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:19.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.478+0000 7fd3877fe700 1 -- 172.21.15.47:0/3475666580 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3880b1e80 con 0x7fd3880612b0 2022-06-29T15:49:19.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.478+0000 7fd38ceb7700 1 --2- 172.21.15.47:0/3475666580 >> v2:172.21.15.64:3300/0 conn(0x7fd388062f30 0x7fd388066860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:49:19.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.478+0000 7fd3877fe700 1 --2- 172.21.15.47:0/3475666580 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3880612b0 0x7fd388148dc0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 rx=0x7fd378002f70 tx=0x7fd378006eb0).ready entity=mon.0 client_cookie=b6a34036d50f7d6e server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:19.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.482+0000 7fd386ffd700 1 -- 172.21.15.47:0/3475666580 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd37800d610 con 0x7fd3880612b0 2022-06-29T15:49:19.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.482+0000 7fd386ffd700 1 -- 172.21.15.47:0/3475666580 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd378000e90 con 0x7fd3880612b0 2022-06-29T15:49:19.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.482+0000 7fd386ffd700 1 -- 172.21.15.47:0/3475666580 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd37800d9a0 con 0x7fd3880612b0 2022-06-29T15:49:19.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.482+0000 7fd38d6b8700 1 -- 172.21.15.47:0/3475666580 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3880612b0 msgr2=0x7fd388148dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:19.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.482+0000 7fd38d6b8700 1 --2- 172.21.15.47:0/3475666580 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3880612b0 0x7fd388148dc0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 rx=0x7fd378002f70 tx=0x7fd378006eb0).stop 2022-06-29T15:49:19.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.482+0000 7fd38d6b8700 1 -- 172.21.15.47:0/3475666580 shutdown_connections 2022-06-29T15:49:19.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.482+0000 7fd38d6b8700 1 --2- 172.21.15.47:0/3475666580 >> v2:172.21.15.64:3300/0 conn(0x7fd388062f30 0x7fd388066860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:19.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.482+0000 7fd38d6b8700 1 --2- 172.21.15.47:0/3475666580 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3880612b0 0x7fd388148dc0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:19.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.482+0000 7fd38d6b8700 1 -- 172.21.15.47:0/3475666580 >> 172.21.15.47:0/3475666580 conn(0x7fd388063670 msgr2=0x7fd388063a50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:19.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.482+0000 7fd38d6b8700 1 -- 172.21.15.47:0/3475666580 shutdown_connections 2022-06-29T15:49:19.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.482+0000 7fd38d6b8700 1 -- 172.21.15.47:0/3475666580 wait complete. 2022-06-29T15:49:19.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.482+0000 7fd38d6b8700 1 Processor -- start 2022-06-29T15:49:19.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.482+0000 7fd38d6b8700 1 -- start start 2022-06-29T15:49:19.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.486+0000 7fd38d6b8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3880612b0 0x7fd38814e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:19.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.486+0000 7fd38d6b8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd388062f30 0x7fd38815abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:19.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.486+0000 7fd38d6b8700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd3880b6d20 0x7fd3880ae910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:19.497 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.486+0000 7fd38d6b8700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd388156d60 con 0x7fd3880612b0 2022-06-29T15:49:19.497 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.486+0000 7fd38d6b8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd388156f50 con 0x7fd388062f30 2022-06-29T15:49:19.497 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.486+0000 7fd38d6b8700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd38815a470 con 0x7fd3880b6d20 2022-06-29T15:49:19.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.486+0000 7fd387fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3880612b0 0x7fd38814e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:19.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.486+0000 7fd3877fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd388062f30 0x7fd38815abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:19.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.486+0000 7fd387fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3880612b0 0x7fd38814e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50472/0 (socket says 172.21.15.47:50472) 2022-06-29T15:49:19.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.486+0000 7fd387fff700 1 -- 172.21.15.47:0/3772255314 learned_addr learned my addr 172.21.15.47:0/3772255314 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:19.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.486+0000 7fd38ceb7700 1 --2- 172.21.15.47:0/3772255314 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd3880b6d20 0x7fd3880ae910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:19.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.486+0000 7fd3877fe700 1 -- 172.21.15.47:0/3772255314 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd3880b6d20 msgr2=0x7fd3880ae910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:19.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.486+0000 7fd3877fe700 1 --2- 172.21.15.47:0/3772255314 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd3880b6d20 0x7fd3880ae910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:19.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.486+0000 7fd3877fe700 1 -- 172.21.15.47:0/3772255314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3880612b0 msgr2=0x7fd38814e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:19.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.486+0000 7fd3877fe700 1 --2- 172.21.15.47:0/3772255314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3880612b0 0x7fd38814e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:19.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.486+0000 7fd3877fe700 1 -- 172.21.15.47:0/3772255314 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd37800b1e0 con 0x7fd388062f30 2022-06-29T15:49:19.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.486+0000 7fd3877fe700 1 --2- 172.21.15.47:0/3772255314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd388062f30 0x7fd38815abb0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7fd378002f70 tx=0x7fd378015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:19.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.486+0000 7fd3857fa700 1 -- 172.21.15.47:0/3772255314 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd378011860 con 0x7fd388062f30 2022-06-29T15:49:19.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.486+0000 7fd3857fa700 1 -- 172.21.15.47:0/3772255314 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3780119c0 con 0x7fd388062f30 2022-06-29T15:49:19.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.486+0000 7fd3857fa700 1 -- 172.21.15.47:0/3772255314 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd378011d20 con 0x7fd388062f30 2022-06-29T15:49:19.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.486+0000 7fd38d6b8700 1 -- 172.21.15.47:0/3772255314 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd38815a710 con 0x7fd388062f30 2022-06-29T15:49:19.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.486+0000 7fd38d6b8700 1 -- 172.21.15.47:0/3772255314 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd38814c8b0 con 0x7fd388062f30 2022-06-29T15:49:19.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.486+0000 7fd38d6b8700 1 -- 172.21.15.47:0/3772255314 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd38815a8a0 con 0x7fd388062f30 2022-06-29T15:49:19.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.490+0000 7fd3857fa700 1 -- 172.21.15.47:0/3772255314 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd37801a040 con 0x7fd388062f30 2022-06-29T15:49:19.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.490+0000 7fd3857fa700 1 --2- 172.21.15.47:0/3772255314 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd368053950 0x7fd368055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:19.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.490+0000 7fd3857fa700 1 -- 172.21.15.47:0/3772255314 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5729+0+0 (secure 0 0 0) 0x7fd37806cee0 con 0x7fd388062f30 2022-06-29T15:49:19.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.490+0000 7fd387fff700 1 --2- 172.21.15.47:0/3772255314 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd368053950 0x7fd368055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:19.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.490+0000 7fd387fff700 1 --2- 172.21.15.47:0/3772255314 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd368053950 0x7fd368055e10 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7fd37c00d000 tx=0x7fd37c007c50).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:19.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.498+0000 7fd3857fa700 1 -- 172.21.15.47:0/3772255314 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd378043e80 con 0x7fd388062f30 2022-06-29T15:49:19.635 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.630+0000 7fd38d6b8700 1 -- 172.21.15.47:0/3772255314 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fd3880573d0 con 0x7fd388062f30 2022-06-29T15:49:19.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.642+0000 7fd3857fa700 1 -- 172.21.15.47:0/3772255314 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd378047460 con 0x7fd388062f30 2022-06-29T15:49:19.652 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.646+0000 7fd38d6b8700 1 -- 172.21.15.47:0/3772255314 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd368053950 msgr2=0x7fd368055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:19.653 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.646+0000 7fd38d6b8700 1 --2- 172.21.15.47:0/3772255314 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd368053950 0x7fd368055e10 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7fd37c00d000 tx=0x7fd37c007c50).stop 2022-06-29T15:49:19.653 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.646+0000 7fd38d6b8700 1 -- 172.21.15.47:0/3772255314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd388062f30 msgr2=0x7fd38815abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:19.654 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.646+0000 7fd38d6b8700 1 --2- 172.21.15.47:0/3772255314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd388062f30 0x7fd38815abb0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7fd378002f70 tx=0x7fd378015000).stop 2022-06-29T15:49:19.654 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.646+0000 7fd38d6b8700 1 -- 172.21.15.47:0/3772255314 shutdown_connections 2022-06-29T15:49:19.654 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.646+0000 7fd38d6b8700 1 --2- 172.21.15.47:0/3772255314 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd3880b6d20 0x7fd3880ae910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:19.654 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.646+0000 7fd38d6b8700 1 --2- 172.21.15.47:0/3772255314 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd368053950 0x7fd368055e10 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:19.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.646+0000 7fd38d6b8700 1 --2- 172.21.15.47:0/3772255314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd388062f30 0x7fd38815abb0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:19.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.646+0000 7fd38d6b8700 1 --2- 172.21.15.47:0/3772255314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3880612b0 0x7fd38814e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:19.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.646+0000 7fd38d6b8700 1 -- 172.21.15.47:0/3772255314 >> 172.21.15.47:0/3772255314 conn(0x7fd388063670 msgr2=0x7fd388065e40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:19.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.646+0000 7fd38d6b8700 1 -- 172.21.15.47:0/3772255314 shutdown_connections 2022-06-29T15:49:19.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.646+0000 7fd38d6b8700 1 -- 172.21.15.47:0/3772255314 wait complete. 2022-06-29T15:49:19.665 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder' 2022-06-29T15:49:19.770 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.762+0000 7fd6c4937700 1 Processor -- start 2022-06-29T15:49:19.770 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.762+0000 7fd6c4937700 1 -- start start 2022-06-29T15:49:19.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.766+0000 7fd6c4937700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd6c00b29a0 0x7fd6c00afb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:19.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.766+0000 7fd6c4937700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6c0146510 con 0x7fd6c00b25c0 2022-06-29T15:49:19.783 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.766+0000 7fd6c4937700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6c01467b0 con 0x7fd6c00b1fc0 2022-06-29T15:49:19.784 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.766+0000 7fd6c4937700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6c00b0040 con 0x7fd6c00b29a0 2022-06-29T15:49:19.784 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.766+0000 7fd6bf7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd6c00b25c0 0x7fd6c01403b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36032/0 (socket says 172.21.15.47:36032) 2022-06-29T15:49:19.785 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.766+0000 7fd6bf7fe700 1 -- 172.21.15.47:0/3120302670 learned_addr learned my addr 172.21.15.47:0/3120302670 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:49:19.785 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.766+0000 7fd6bffff700 1 --2- 172.21.15.47:0/3120302670 >> v2:172.21.15.64:3300/0 conn(0x7fd6c00b29a0 0x7fd6c00afb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:19.785 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.766+0000 7fd6be7fc700 1 -- 172.21.15.47:0/3120302670 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4220126273 0 0) 0x7fd6c0146510 con 0x7fd6c00b25c0 2022-06-29T15:49:19.786 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.766+0000 7fd6be7fc700 1 -- 172.21.15.47:0/3120302670 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6ac003260 con 0x7fd6c00b25c0 2022-06-29T15:49:19.786 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.766+0000 7fd6be7fc700 1 -- 172.21.15.47:0/3120302670 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1903058216 0 0) 0x7fd6ac003260 con 0x7fd6c00b25c0 2022-06-29T15:49:19.786 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.766+0000 7fd6bffff700 1 -- 172.21.15.47:0/3120302670 >> v1:172.21.15.64:6789/0 conn(0x7fd6c00b1fc0 legacy=0x7fd6c00af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:19.786 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.766+0000 7fd6bffff700 1 -- 172.21.15.47:0/3120302670 >> v1:172.21.15.47:6789/0 conn(0x7fd6c00b25c0 legacy=0x7fd6c01403b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:19.787 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.766+0000 7fd6bffff700 1 -- 172.21.15.47:0/3120302670 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c0146a50 con 0x7fd6c00b29a0 2022-06-29T15:49:19.787 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.766+0000 7fd6bffff700 1 --2- 172.21.15.47:0/3120302670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6c00b29a0 0x7fd6c00afb50 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7fd6b0001800 tx=0x7fd6b80091d0).ready entity=mon.1 client_cookie=23822663dd44aafb server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:19.787 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.766+0000 7fd6be7fc700 1 -- 172.21.15.47:0/3120302670 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd6b0001660 con 0x7fd6c00b29a0 2022-06-29T15:49:19.787 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.766+0000 7fd6be7fc700 1 -- 172.21.15.47:0/3120302670 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6b800b580 con 0x7fd6c00b29a0 2022-06-29T15:49:19.788 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.766+0000 7fd6be7fc700 1 -- 172.21.15.47:0/3120302670 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd6b800b6f0 con 0x7fd6c00b29a0 2022-06-29T15:49:19.788 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.766+0000 7fd6c4937700 1 -- 172.21.15.47:0/3120302670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6c00b29a0 msgr2=0x7fd6c00afb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:19.788 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.766+0000 7fd6c4937700 1 --2- 172.21.15.47:0/3120302670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6c00b29a0 0x7fd6c00afb50 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7fd6b0001800 tx=0x7fd6b80091d0).stop 2022-06-29T15:49:19.788 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.766+0000 7fd6c4937700 1 -- 172.21.15.47:0/3120302670 shutdown_connections 2022-06-29T15:49:19.789 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.766+0000 7fd6c4937700 1 --2- 172.21.15.47:0/3120302670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6c00b29a0 0x7fd6c00afb50 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:19.789 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.766+0000 7fd6c4937700 1 -- 172.21.15.47:0/3120302670 >> 172.21.15.47:0/3120302670 conn(0x7fd6c0062df0 msgr2=0x7fd6c00631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:19.789 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.766+0000 7fd6c4937700 1 -- 172.21.15.47:0/3120302670 shutdown_connections 2022-06-29T15:49:19.790 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.766+0000 7fd6c4937700 1 -- 172.21.15.47:0/3120302670 wait complete. 2022-06-29T15:49:19.793 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.770+0000 7fd6c4937700 1 Processor -- start 2022-06-29T15:49:19.793 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.770+0000 7fd6c4937700 1 -- start start 2022-06-29T15:49:19.794 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.770+0000 7fd6c4937700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6c00b1fc0 0x7fd6c0142400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:19.794 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.770+0000 7fd6c4937700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd6c00b25c0 0x7fd6c0098ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:19.794 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.770+0000 7fd6c4937700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6c00b29a0 0x7fd6c0099390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:19.795 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.770+0000 7fd6c4937700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c00ab9a0 con 0x7fd6c00b29a0 2022-06-29T15:49:19.795 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.770+0000 7fd6c4937700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c00ab200 con 0x7fd6c00b1fc0 2022-06-29T15:49:19.795 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.770+0000 7fd6c4937700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c0099880 con 0x7fd6c00b25c0 2022-06-29T15:49:19.795 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.770+0000 7fd6bffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6c00b29a0 0x7fd6c0099390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:19.796 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.770+0000 7fd6bffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6c00b29a0 0x7fd6c0099390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50486/0 (socket says 172.21.15.47:50486) 2022-06-29T15:49:19.796 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.770+0000 7fd6bffff700 1 -- 172.21.15.47:0/3838441915 learned_addr learned my addr 172.21.15.47:0/3838441915 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:19.796 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.770+0000 7fd6beffd700 1 --2- 172.21.15.47:0/3838441915 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd6c00b25c0 0x7fd6c0098ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:19.796 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.770+0000 7fd6bffff700 1 -- 172.21.15.47:0/3838441915 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd6c00b25c0 msgr2=0x7fd6c0098ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:19.797 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.770+0000 7fd6bffff700 1 --2- 172.21.15.47:0/3838441915 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd6c00b25c0 0x7fd6c0098ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:19.797 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.770+0000 7fd6bffff700 1 -- 172.21.15.47:0/3838441915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6c00b1fc0 msgr2=0x7fd6c0142400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:19.797 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.770+0000 7fd6bffff700 1 --2- 172.21.15.47:0/3838441915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6c00b1fc0 0x7fd6c0142400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2022-06-29T15:49:19.797 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.770+0000 7fd6bffff700 1 -- 172.21.15.47:0/3838441915 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b8009470 con 0x7fd6c00b29a0 2022-06-29T15:49:19.798 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.770+0000 7fd6beffd700 1 --2- 172.21.15.47:0/3838441915 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd6c00b25c0 0x7fd6c0098ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:49:19.798 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.770+0000 7fd6bffff700 1 --2- 172.21.15.47:0/3838441915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6c00b29a0 0x7fd6c0099390 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 rx=0x7fd6b0001800 tx=0x7fd6b8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:19.798 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.774+0000 7fd6bcff9700 1 -- 172.21.15.47:0/3838441915 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd6b800b6f0 con 0x7fd6c00b29a0 2022-06-29T15:49:19.798 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.774+0000 7fd6bcff9700 1 -- 172.21.15.47:0/3838441915 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6b80032f0 con 0x7fd6c00b29a0 2022-06-29T15:49:19.799 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.774+0000 7fd6c4937700 1 -- 172.21.15.47:0/3838441915 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6c0155ca0 con 0x7fd6c00b29a0 2022-06-29T15:49:19.799 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.774+0000 7fd6bcff9700 1 -- 172.21.15.47:0/3838441915 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd6b8011e50 con 0x7fd6c00b29a0 2022-06-29T15:49:19.799 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.774+0000 7fd6c4937700 1 -- 172.21.15.47:0/3838441915 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6c00ab700 con 0x7fd6c00b29a0 2022-06-29T15:49:19.800 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.774+0000 7fd6c4937700 1 -- 172.21.15.47:0/3838441915 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6c00573d0 con 0x7fd6c00b29a0 2022-06-29T15:49:19.800 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.774+0000 7fd6bcff9700 1 -- 172.21.15.47:0/3838441915 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd6b800f070 con 0x7fd6c00b29a0 2022-06-29T15:49:19.800 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.774+0000 7fd6bcff9700 1 --2- 172.21.15.47:0/3838441915 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd6a00539a0 0x7fd6a0055e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:19.801 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.774+0000 7fd6bcff9700 1 -- 172.21.15.47:0/3838441915 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5729+0+0 (secure 0 0 0) 0x7fd6b806d110 con 0x7fd6c00b29a0 2022-06-29T15:49:19.801 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.774+0000 7fd6bf7fe700 1 --2- 172.21.15.47:0/3838441915 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd6a00539a0 0x7fd6a0055e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:19.801 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.774+0000 7fd6bf7fe700 1 --2- 172.21.15.47:0/3838441915 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd6a00539a0 0x7fd6a0055e60 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7fd6b00018c0 tx=0x7fd6b0001400).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:19.802 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.778+0000 7fd6bcff9700 1 -- 172.21.15.47:0/3838441915 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd6b8044180 con 0x7fd6c00b29a0 2022-06-29T15:49:19.951 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:19.946+0000 7fd6c4937700 1 -- 172.21.15.47:0/3838441915 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder"]} v 0) v1 -- 0x7fd6c00b58e0 con 0x7fd6c00b29a0 2022-06-29T15:49:20.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.342+0000 7fd6bcff9700 1 -- 172.21.15.47:0/3838441915 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder"]}]=0 v295) v1 ==== 139+0+0 (secure 0 0 0) 0x7fd6b80477a0 con 0x7fd6c00b29a0 2022-06-29T15:49:20.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.346+0000 7fd6c4937700 1 -- 172.21.15.47:0/3838441915 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd6a00539a0 msgr2=0x7fd6a0055e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:20.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.346+0000 7fd6c4937700 1 --2- 172.21.15.47:0/3838441915 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd6a00539a0 0x7fd6a0055e60 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7fd6b00018c0 tx=0x7fd6b0001400).stop 2022-06-29T15:49:20.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.346+0000 7fd6c4937700 1 -- 172.21.15.47:0/3838441915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6c00b29a0 msgr2=0x7fd6c0099390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:20.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.346+0000 7fd6c4937700 1 --2- 172.21.15.47:0/3838441915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6c00b29a0 0x7fd6c0099390 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 rx=0x7fd6b0001800 tx=0x7fd6b8016000).stop 2022-06-29T15:49:20.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.346+0000 7fd6c4937700 1 -- 172.21.15.47:0/3838441915 shutdown_connections 2022-06-29T15:49:20.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.346+0000 7fd6c4937700 1 --2- 172.21.15.47:0/3838441915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6c00b29a0 0x7fd6c0099390 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:20.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.346+0000 7fd6c4937700 1 --2- 172.21.15.47:0/3838441915 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd6c00b25c0 0x7fd6c0098ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:20.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.346+0000 7fd6c4937700 1 --2- 172.21.15.47:0/3838441915 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd6a00539a0 0x7fd6a0055e60 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:20.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.346+0000 7fd6c4937700 1 --2- 172.21.15.47:0/3838441915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6c00b1fc0 0x7fd6c0142400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2022-06-29T15:49:20.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.346+0000 7fd6c4937700 1 -- 172.21.15.47:0/3838441915 >> 172.21.15.47:0/3838441915 conn(0x7fd6c0062df0 msgr2=0x7fd6c00b4f00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:20.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.346+0000 7fd6c4937700 1 -- 172.21.15.47:0/3838441915 shutdown_connections 2022-06-29T15:49:20.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.350+0000 7fd6c4937700 1 -- 172.21.15.47:0/3838441915 wait complete. 2022-06-29T15:49:20.370 INFO:tasks.cephfs_test_runner:test_cap_revoke_nonresponder (tasks.cephfs.test_misc.TestMisc) ... ok 2022-06-29T15:49:20.371 INFO:tasks.cephfs_test_runner:Starting test: test_dump_inode (tasks.cephfs.test_misc.TestMisc) 2022-06-29T15:49:20.372 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode' 2022-06-29T15:49:20.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.474+0000 7f6273fff700 1 Processor -- start 2022-06-29T15:49:20.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.474+0000 7f6273fff700 1 -- start start 2022-06-29T15:49:20.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.474+0000 7f6273fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6274145090 0x7f62740b2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:20.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.474+0000 7f6273fff700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62740a9120 con 0x7f6274060eb0 2022-06-29T15:49:20.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.474+0000 7f6273fff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62740b0b80 con 0x7f62740b5c60 2022-06-29T15:49:20.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.474+0000 7f6273fff700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62740a89e0 con 0x7f6274145090 2022-06-29T15:49:20.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.474+0000 7f62737fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6274145090 0x7f62740b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:20.484 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.474+0000 7f62727fc700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f6274060eb0 0x7f62740b1e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36046/0 (socket says 172.21.15.47:36046) 2022-06-29T15:49:20.484 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.474+0000 7f62727fc700 1 -- 172.21.15.47:0/580107663 learned_addr learned my addr 172.21.15.47:0/580107663 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:49:20.484 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.474+0000 7f62737fe700 1 -- 172.21.15.47:0/580107663 >> v1:172.21.15.64:6789/0 conn(0x7f62740b5c60 legacy=0x7f6274144980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:20.484 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.474+0000 7f62737fe700 1 -- 172.21.15.47:0/580107663 >> v1:172.21.15.47:6789/0 conn(0x7f6274060eb0 legacy=0x7f62740b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:20.484 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.474+0000 7f6271ffb700 1 -- 172.21.15.47:0/580107663 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3912471088 0 0) 0x7f62740a9120 con 0x7f6274060eb0 2022-06-29T15:49:20.485 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.474+0000 7f62737fe700 1 -- 172.21.15.47:0/580107663 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62740a8c80 con 0x7f6274145090 2022-06-29T15:49:20.485 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.474+0000 7f62737fe700 1 --2- 172.21.15.47:0/580107663 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6274145090 0x7f62740b2540 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 rx=0x7f626c002f70 tx=0x7f626c006df0).ready entity=mon.0 client_cookie=6d9463b71bed308b server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:20.485 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.478+0000 7f6271ffb700 1 -- 172.21.15.47:0/580107663 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f626c00d610 con 0x7f6274145090 2022-06-29T15:49:20.485 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.478+0000 7f6271ffb700 1 -- 172.21.15.47:0/580107663 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f626c000e90 con 0x7f6274145090 2022-06-29T15:49:20.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.478+0000 7f6271ffb700 1 -- 172.21.15.47:0/580107663 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f626c00d960 con 0x7f6274145090 2022-06-29T15:49:20.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.478+0000 7f6273fff700 1 -- 172.21.15.47:0/580107663 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6274145090 msgr2=0x7f62740b2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:20.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.478+0000 7f6273fff700 1 --2- 172.21.15.47:0/580107663 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6274145090 0x7f62740b2540 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 rx=0x7f626c002f70 tx=0x7f626c006df0).stop 2022-06-29T15:49:20.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.478+0000 7f6273fff700 1 -- 172.21.15.47:0/580107663 shutdown_connections 2022-06-29T15:49:20.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.478+0000 7f6273fff700 1 --2- 172.21.15.47:0/580107663 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6274145090 0x7f62740b2540 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:20.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.478+0000 7f6273fff700 1 -- 172.21.15.47:0/580107663 >> 172.21.15.47:0/580107663 conn(0x7f6274062560 msgr2=0x7f6274062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:20.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.478+0000 7f6273fff700 1 -- 172.21.15.47:0/580107663 shutdown_connections 2022-06-29T15:49:20.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.478+0000 7f6273fff700 1 -- 172.21.15.47:0/580107663 wait complete. 2022-06-29T15:49:20.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.478+0000 7f6273fff700 1 Processor -- start 2022-06-29T15:49:20.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.478+0000 7f6273fff700 1 -- start start 2022-06-29T15:49:20.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.478+0000 7f6273fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6274060eb0 0x7f62740aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:20.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.478+0000 7f6273fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f62740b5c60 0x7f6274147ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:20.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.478+0000 7f6273fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6274145090 0x7f6274147fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:20.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.478+0000 7f6273fff700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62741577a0 con 0x7f6274060eb0 2022-06-29T15:49:20.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.478+0000 7f6273fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6274156880 con 0x7f6274145090 2022-06-29T15:49:20.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.478+0000 7f6273fff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6274156b20 con 0x7f62740b5c60 2022-06-29T15:49:20.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.478+0000 7f6272ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6274060eb0 0x7f62740aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:20.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.478+0000 7f6272ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6274060eb0 0x7f62740aeb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50502/0 (socket says 172.21.15.47:50502) 2022-06-29T15:49:20.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.478+0000 7f6272ffd700 1 -- 172.21.15.47:0/2626876125 learned_addr learned my addr 172.21.15.47:0/2626876125 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:20.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.478+0000 7f62737fe700 1 --2- 172.21.15.47:0/2626876125 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6274145090 0x7f6274147fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:20.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.478+0000 7f62727fc700 1 --2- 172.21.15.47:0/2626876125 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f62740b5c60 0x7f6274147ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:20.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.482+0000 7f6272ffd700 1 -- 172.21.15.47:0/2626876125 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f62740b5c60 msgr2=0x7f6274147ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:20.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.482+0000 7f6272ffd700 1 --2- 172.21.15.47:0/2626876125 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f62740b5c60 0x7f6274147ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:20.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.482+0000 7f6272ffd700 1 -- 172.21.15.47:0/2626876125 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6274145090 msgr2=0x7f6274147fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:20.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.482+0000 7f6272ffd700 1 --2- 172.21.15.47:0/2626876125 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6274145090 0x7f6274147fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:20.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.482+0000 7f6272ffd700 1 -- 172.21.15.47:0/2626876125 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f626c00b180 con 0x7f6274060eb0 2022-06-29T15:49:20.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.482+0000 7f6272ffd700 1 --2- 172.21.15.47:0/2626876125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6274060eb0 0x7f62740aeb40 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 rx=0x7f627406ac40 tx=0x7f6268007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:20.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.482+0000 7f6263fff700 1 -- 172.21.15.47:0/2626876125 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6268007020 con 0x7f6274060eb0 2022-06-29T15:49:20.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.482+0000 7f6263fff700 1 -- 172.21.15.47:0/2626876125 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6268007180 con 0x7f6274060eb0 2022-06-29T15:49:20.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.482+0000 7f6263fff700 1 -- 172.21.15.47:0/2626876125 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6268006560 con 0x7f6274060eb0 2022-06-29T15:49:20.497 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.482+0000 7f6273fff700 1 -- 172.21.15.47:0/2626876125 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6274156dc0 con 0x7f6274060eb0 2022-06-29T15:49:20.497 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.482+0000 7f6273fff700 1 -- 172.21.15.47:0/2626876125 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6274157300 con 0x7f6274060eb0 2022-06-29T15:49:20.497 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.482+0000 7f6273fff700 1 -- 172.21.15.47:0/2626876125 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6274146860 con 0x7f6274060eb0 2022-06-29T15:49:20.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.482+0000 7f6263fff700 1 -- 172.21.15.47:0/2626876125 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6268010070 con 0x7f6274060eb0 2022-06-29T15:49:20.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.482+0000 7f6263fff700 1 --2- 172.21.15.47:0/2626876125 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f62500539e0 0x7f6250055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:20.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.482+0000 7f6263fff700 1 -- 172.21.15.47:0/2626876125 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5729+0+0 (secure 0 0 0) 0x7f626806af10 con 0x7f6274060eb0 2022-06-29T15:49:20.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.482+0000 7f62727fc700 1 --2- 172.21.15.47:0/2626876125 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f62500539e0 0x7f6250055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:20.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.486+0000 7f62727fc700 1 --2- 172.21.15.47:0/2626876125 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f62500539e0 0x7f6250055ea0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f626400d000 tx=0x7f6264001520).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:20.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.490+0000 7f6263fff700 1 -- 172.21.15.47:0/2626876125 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6268041f80 con 0x7f6274060eb0 2022-06-29T15:49:20.669 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:20.662+0000 7f6273fff700 1 -- 172.21.15.47:0/2626876125 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode"]} v 0) v1 -- 0x7f62740573d0 con 0x7f6274060eb0 2022-06-29T15:49:21.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.349+0000 7f6263fff700 1 -- 172.21.15.47:0/2626876125 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode"]}]=0 v296) v1 ==== 129+0+0 (secure 0 0 0) 0x7f62680455a0 con 0x7f6274060eb0 2022-06-29T15:49:21.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.353+0000 7f6273fff700 1 -- 172.21.15.47:0/2626876125 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f62500539e0 msgr2=0x7f6250055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:21.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.353+0000 7f6273fff700 1 --2- 172.21.15.47:0/2626876125 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f62500539e0 0x7f6250055ea0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f626400d000 tx=0x7f6264001520).stop 2022-06-29T15:49:21.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.353+0000 7f6273fff700 1 -- 172.21.15.47:0/2626876125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6274060eb0 msgr2=0x7f62740aeb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:21.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.353+0000 7f6273fff700 1 --2- 172.21.15.47:0/2626876125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6274060eb0 0x7f62740aeb40 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 rx=0x7f627406ac40 tx=0x7f6268007c40).stop 2022-06-29T15:49:21.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.353+0000 7f6273fff700 1 -- 172.21.15.47:0/2626876125 shutdown_connections 2022-06-29T15:49:21.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.353+0000 7f6273fff700 1 --2- 172.21.15.47:0/2626876125 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f62500539e0 0x7f6250055ea0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:21.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.353+0000 7f6273fff700 1 --2- 172.21.15.47:0/2626876125 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6274145090 0x7f6274147fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:21.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.353+0000 7f6273fff700 1 --2- 172.21.15.47:0/2626876125 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f62740b5c60 0x7f6274147ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:21.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.353+0000 7f6273fff700 1 --2- 172.21.15.47:0/2626876125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6274060eb0 0x7f62740aeb40 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:21.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.353+0000 7f6273fff700 1 -- 172.21.15.47:0/2626876125 >> 172.21.15.47:0/2626876125 conn(0x7f6274062560 msgr2=0x7f62740b21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:21.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.353+0000 7f6273fff700 1 -- 172.21.15.47:0/2626876125 shutdown_connections 2022-06-29T15:49:21.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.357+0000 7f6273fff700 1 -- 172.21.15.47:0/2626876125 wait complete. 2022-06-29T15:49:21.377 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-06-29T15:49:21.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.485+0000 7f6abbdb7700 1 Processor -- start 2022-06-29T15:49:21.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.485+0000 7f6abbdb7700 1 -- start start 2022-06-29T15:49:21.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.489+0000 7f6abbdb7700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6ab4060a70 0x7f6ab4143be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:21.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.489+0000 7f6abbdb7700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6ab40b2bb0 0x7f6ab40b5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:21.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.489+0000 7f6abbdb7700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ab4140e40 con 0x7f6ab40a8ce0 2022-06-29T15:49:21.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.489+0000 7f6abbdb7700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ab4060e50 con 0x7f6ab40b2bb0 2022-06-29T15:49:21.497 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.489+0000 7f6abbdb7700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ab40b1cf0 con 0x7f6ab4060a70 2022-06-29T15:49:21.497 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.489+0000 7f6aba354700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f6ab40a8ce0 0x7f6ab40b5970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36064/0 (socket says 172.21.15.47:36064) 2022-06-29T15:49:21.497 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.489+0000 7f6aba354700 1 -- 172.21.15.47:0/739812949 learned_addr learned my addr 172.21.15.47:0/739812949 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:49:21.497 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.489+0000 7f6ab9352700 1 --2- 172.21.15.47:0/739812949 >> v2:172.21.15.47:3300/0 conn(0x7f6ab40b2bb0 0x7f6ab40b5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:21.498 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.489+0000 7f6ab9b53700 1 --2- 172.21.15.47:0/739812949 >> v2:172.21.15.64:3300/0 conn(0x7f6ab4060a70 0x7f6ab4143be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:21.498 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.489+0000 7f6ab8b51700 1 -- 172.21.15.47:0/739812949 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1158720986 0 0) 0x7f6aac001490 con 0x7f6ab40a8ce0 2022-06-29T15:49:21.498 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.489+0000 7f6ab8b51700 1 -- 172.21.15.47:0/739812949 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a9c003260 con 0x7f6ab40a8ce0 2022-06-29T15:49:21.498 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.489+0000 7f6ab9352700 1 -- 172.21.15.47:0/739812949 >> v2:172.21.15.64:3300/0 conn(0x7f6ab4060a70 msgr2=0x7f6ab4143be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:21.499 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.489+0000 7f6ab9352700 1 --2- 172.21.15.47:0/739812949 >> v2:172.21.15.64:3300/0 conn(0x7f6ab4060a70 0x7f6ab4143be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:21.499 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.489+0000 7f6ab9352700 1 -- 172.21.15.47:0/739812949 >> v1:172.21.15.47:6789/0 conn(0x7f6ab40a8ce0 legacy=0x7f6ab40b5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:21.499 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.489+0000 7f6ab9352700 1 -- 172.21.15.47:0/739812949 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ab40b1f90 con 0x7f6ab40b2bb0 2022-06-29T15:49:21.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.489+0000 7f6ab9352700 1 --2- 172.21.15.47:0/739812949 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ab40b2bb0 0x7f6ab40b5480 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 rx=0x7f6aa4002f70 tx=0x7f6aa4006df0).ready entity=mon.0 client_cookie=bde4e8f30763029 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:21.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.489+0000 7f6ab8b51700 1 -- 172.21.15.47:0/739812949 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6aa400d4f0 con 0x7f6ab40b2bb0 2022-06-29T15:49:21.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.489+0000 7f6ab8b51700 1 -- 172.21.15.47:0/739812949 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aa4000e90 con 0x7f6ab40b2bb0 2022-06-29T15:49:21.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.489+0000 7f6ab8b51700 1 -- 172.21.15.47:0/739812949 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6aa400d840 con 0x7f6ab40b2bb0 2022-06-29T15:49:21.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.489+0000 7f6abbdb7700 1 -- 172.21.15.47:0/739812949 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ab40b2bb0 msgr2=0x7f6ab40b5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:21.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.489+0000 7f6abbdb7700 1 --2- 172.21.15.47:0/739812949 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ab40b2bb0 0x7f6ab40b5480 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 rx=0x7f6aa4002f70 tx=0x7f6aa4006df0).stop 2022-06-29T15:49:21.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.489+0000 7f6abbdb7700 1 -- 172.21.15.47:0/739812949 shutdown_connections 2022-06-29T15:49:21.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.489+0000 7f6abbdb7700 1 --2- 172.21.15.47:0/739812949 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ab40b2bb0 0x7f6ab40b5480 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:21.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.489+0000 7f6abbdb7700 1 --2- 172.21.15.47:0/739812949 >> v2:172.21.15.64:3300/0 conn(0x7f6ab4060a70 0x7f6ab4143be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:21.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.489+0000 7f6abbdb7700 1 -- 172.21.15.47:0/739812949 >> 172.21.15.47:0/739812949 conn(0x7f6ab40626f0 msgr2=0x7f6ab4062ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:21.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.489+0000 7f6abbdb7700 1 -- 172.21.15.47:0/739812949 shutdown_connections 2022-06-29T15:49:21.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.489+0000 7f6abbdb7700 1 -- 172.21.15.47:0/739812949 wait complete. 2022-06-29T15:49:21.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.493+0000 7f6abbdb7700 1 Processor -- start 2022-06-29T15:49:21.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.493+0000 7f6abbdb7700 1 -- start start 2022-06-29T15:49:21.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.493+0000 7f6abbdb7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ab4060a70 0x7f6ab40af420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:21.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.493+0000 7f6abbdb7700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6ab40a8ce0 0x7f6ab413b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:21.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.493+0000 7f6abbdb7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ab40b2bb0 0x7f6ab413b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:21.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.493+0000 7f6abbdb7700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab413d110 con 0x7f6ab4060a70 2022-06-29T15:49:21.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.493+0000 7f6abbdb7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab413c830 con 0x7f6ab40b2bb0 2022-06-29T15:49:21.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.493+0000 7f6abbdb7700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab413cad0 con 0x7f6ab40a8ce0 2022-06-29T15:49:21.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.493+0000 7f6ab9b53700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ab4060a70 0x7f6ab40af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:21.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.493+0000 7f6ab9b53700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ab4060a70 0x7f6ab40af420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50514/0 (socket says 172.21.15.47:50514) 2022-06-29T15:49:21.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.493+0000 7f6ab9352700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6ab40a8ce0 0x7f6ab413b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:21.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.493+0000 7f6aba354700 1 --2- 172.21.15.47:0/4064659961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ab40b2bb0 0x7f6ab413b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:21.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.493+0000 7f6ab9b53700 1 -- 172.21.15.47:0/4064659961 learned_addr learned my addr 172.21.15.47:0/4064659961 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:21.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.493+0000 7f6ab9b53700 1 -- 172.21.15.47:0/4064659961 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6ab40a8ce0 msgr2=0x7f6ab413b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:21.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.493+0000 7f6ab9b53700 1 --2- 172.21.15.47:0/4064659961 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6ab40a8ce0 0x7f6ab413b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:21.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.493+0000 7f6ab9b53700 1 -- 172.21.15.47:0/4064659961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ab40b2bb0 msgr2=0x7f6ab413b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:21.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.493+0000 7f6ab9b53700 1 --2- 172.21.15.47:0/4064659961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ab40b2bb0 0x7f6ab413b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:21.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.493+0000 7f6ab9b53700 1 -- 172.21.15.47:0/4064659961 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa400b180 con 0x7f6ab4060a70 2022-06-29T15:49:21.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.493+0000 7f6aba354700 1 --2- 172.21.15.47:0/4064659961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ab40b2bb0 0x7f6ab413b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:49:21.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.493+0000 7f6ab9b53700 1 --2- 172.21.15.47:0/4064659961 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ab4060a70 0x7f6ab40af420 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 rx=0x7f6ab000c000 tx=0x7f6ab0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:21.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.493+0000 7f6aa2ffd700 1 -- 172.21.15.47:0/4064659961 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ab0010780 con 0x7f6ab4060a70 2022-06-29T15:49:21.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.493+0000 7f6aa2ffd700 1 -- 172.21.15.47:0/4064659961 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ab00108e0 con 0x7f6ab4060a70 2022-06-29T15:49:21.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.493+0000 7f6abbdb7700 1 -- 172.21.15.47:0/4064659961 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ab413af30 con 0x7f6ab4060a70 2022-06-29T15:49:21.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.493+0000 7f6aa2ffd700 1 -- 172.21.15.47:0/4064659961 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ab0010c40 con 0x7f6ab4060a70 2022-06-29T15:49:21.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.493+0000 7f6abbdb7700 1 -- 172.21.15.47:0/4064659961 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ab413ce00 con 0x7f6ab4060a70 2022-06-29T15:49:21.513 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.497+0000 7f6abbdb7700 1 -- 172.21.15.47:0/4064659961 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ab413bf50 con 0x7f6ab4060a70 2022-06-29T15:49:21.513 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.497+0000 7f6aa2ffd700 1 -- 172.21.15.47:0/4064659961 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6ab001c460 con 0x7f6ab4060a70 2022-06-29T15:49:21.514 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.497+0000 7f6aa2ffd700 1 --2- 172.21.15.47:0/4064659961 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6a94053950 0x7f6a94055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:21.514 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.497+0000 7f6ab9352700 1 --2- 172.21.15.47:0/4064659961 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6a94053950 0x7f6a94055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:21.514 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.497+0000 7f6aa2ffd700 1 -- 172.21.15.47:0/4064659961 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5729+0+0 (secure 0 0 0) 0x7f6ab006c270 con 0x7f6ab4060a70 2022-06-29T15:49:21.514 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.497+0000 7f6ab9352700 1 --2- 172.21.15.47:0/4064659961 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6a94053950 0x7f6a94055e10 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7f6aa401a000 tx=0x7f6aa4014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:21.515 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.505+0000 7f6aa2ffd700 1 -- 172.21.15.47:0/4064659961 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6ab0043170 con 0x7f6ab4060a70 2022-06-29T15:49:21.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.637+0000 7f6abbdb7700 1 -- 172.21.15.47:0/4064659961 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f6ab40573d0 con 0x7f6ab4060a70 2022-06-29T15:49:21.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.645+0000 7f6aa2ffd700 1 -- 172.21.15.47:0/4064659961 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6ab0046790 con 0x7f6ab4060a70 2022-06-29T15:49:21.653 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.645+0000 7f6abbdb7700 1 -- 172.21.15.47:0/4064659961 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6a94053950 msgr2=0x7f6a94055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:21.654 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.645+0000 7f6abbdb7700 1 --2- 172.21.15.47:0/4064659961 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6a94053950 0x7f6a94055e10 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7f6aa401a000 tx=0x7f6aa4014000).stop 2022-06-29T15:49:21.654 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.645+0000 7f6abbdb7700 1 -- 172.21.15.47:0/4064659961 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ab4060a70 msgr2=0x7f6ab40af420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:21.654 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.645+0000 7f6abbdb7700 1 --2- 172.21.15.47:0/4064659961 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ab4060a70 0x7f6ab40af420 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 rx=0x7f6ab000c000 tx=0x7f6ab0006df0).stop 2022-06-29T15:49:21.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.645+0000 7f6abbdb7700 1 -- 172.21.15.47:0/4064659961 shutdown_connections 2022-06-29T15:49:21.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.645+0000 7f6abbdb7700 1 --2- 172.21.15.47:0/4064659961 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6a94053950 0x7f6a94055e10 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:21.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.645+0000 7f6abbdb7700 1 --2- 172.21.15.47:0/4064659961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ab40b2bb0 0x7f6ab413b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:21.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.645+0000 7f6abbdb7700 1 --2- 172.21.15.47:0/4064659961 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6ab40a8ce0 0x7f6ab413b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:21.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.645+0000 7f6abbdb7700 1 --2- 172.21.15.47:0/4064659961 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ab4060a70 0x7f6ab40af420 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:21.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.645+0000 7f6abbdb7700 1 -- 172.21.15.47:0/4064659961 >> 172.21.15.47:0/4064659961 conn(0x7f6ab40626f0 msgr2=0x7f6ab4139760 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:21.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.645+0000 7f6abbdb7700 1 -- 172.21.15.47:0/4064659961 shutdown_connections 2022-06-29T15:49:21.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.645+0000 7f6abbdb7700 1 -- 172.21.15.47:0/4064659961 wait complete. 2022-06-29T15:49:21.666 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-06-29T15:49:21.666 DEBUG:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:49:21.680 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:49:21.702 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:49:21.723 DEBUG:teuthology.orchestra.run.smithi164:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:49:21.745 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-06-29T15:49:21.746 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:49:21.746 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:49:21.747 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:49:21.754 INFO:teuthology.orchestra.run.smithi099.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-06-29T15:49:21.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T15:49:21.756 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-06-29T15:49:21.757 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:49:21.757 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:49:21.758 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:49:21.799 INFO:teuthology.orchestra.run.smithi164.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-06-29T15:49:21.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T15:49:21.802 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-06-29T15:49:21.803 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-06-29T15:49:21.916 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.909+0000 7f8496302700 1 Processor -- start 2022-06-29T15:49:21.917 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.909+0000 7f8496302700 1 -- start start 2022-06-29T15:49:21.917 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.909+0000 7f8496302700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f84900b2120 0x7f84900b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:21.917 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.909+0000 7f8496302700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f84900b1260 0x7f849014b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:21.917 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.909+0000 7f8496302700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84900b0740 con 0x7f849005f0b0 2022-06-29T15:49:21.918 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.909+0000 7f8496302700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8490147460 con 0x7f84900b2120 2022-06-29T15:49:21.918 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.909+0000 7f8496302700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84900b0970 con 0x7f84900b1260 2022-06-29T15:49:21.918 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.909+0000 7f848ffff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f84900b2120 0x7f84900b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:21.918 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.909+0000 7f848ffff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f84900b2120 0x7f84900b2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50522/0 (socket says 172.21.15.47:50522) 2022-06-29T15:49:21.919 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.909+0000 7f848ffff700 1 -- 172.21.15.47:0/4258116900 learned_addr learned my addr 172.21.15.47:0/4258116900 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:21.919 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.909+0000 7f849489f700 1 --2- 172.21.15.47:0/4258116900 >> v2:172.21.15.64:3300/0 conn(0x7f84900b1260 0x7f849014b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:21.919 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.909+0000 7f848ffff700 1 -- 172.21.15.47:0/4258116900 >> v2:172.21.15.64:3300/0 conn(0x7f84900b1260 msgr2=0x7f849014b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:21.919 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.909+0000 7f848ffff700 1 --2- 172.21.15.47:0/4258116900 >> v2:172.21.15.64:3300/0 conn(0x7f84900b1260 0x7f849014b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:21.920 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.909+0000 7f848ffff700 1 -- 172.21.15.47:0/4258116900 >> v1:172.21.15.64:6789/0 conn(0x7f849005f0b0 legacy=0x7f8490147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:21.920 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.909+0000 7f848ffff700 1 -- 172.21.15.47:0/4258116900 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84900b0c10 con 0x7f84900b2120 2022-06-29T15:49:21.920 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.909+0000 7f849489f700 1 --2- 172.21.15.47:0/4258116900 >> v2:172.21.15.64:3300/0 conn(0x7f84900b1260 0x7f849014b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:49:21.920 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.909+0000 7f848ffff700 1 --2- 172.21.15.47:0/4258116900 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84900b2120 0x7f84900b2500 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 rx=0x7f8490141a70 tx=0x7f8484006eb0).ready entity=mon.0 client_cookie=617a92ee669fef07 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:21.924 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.909+0000 7f848effd700 1 -- 172.21.15.47:0/4258116900 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f848400d5c0 con 0x7f84900b2120 2022-06-29T15:49:21.925 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.909+0000 7f848effd700 1 -- 172.21.15.47:0/4258116900 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8484000e90 con 0x7f84900b2120 2022-06-29T15:49:21.925 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.909+0000 7f848effd700 1 -- 172.21.15.47:0/4258116900 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f848400d950 con 0x7f84900b2120 2022-06-29T15:49:21.925 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.909+0000 7f8496302700 1 -- 172.21.15.47:0/4258116900 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84900b2120 msgr2=0x7f84900b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:21.925 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.909+0000 7f8496302700 1 --2- 172.21.15.47:0/4258116900 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84900b2120 0x7f84900b2500 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 rx=0x7f8490141a70 tx=0x7f8484006eb0).stop 2022-06-29T15:49:21.926 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.909+0000 7f8496302700 1 -- 172.21.15.47:0/4258116900 shutdown_connections 2022-06-29T15:49:21.926 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.909+0000 7f8496302700 1 --2- 172.21.15.47:0/4258116900 >> v2:172.21.15.64:3300/0 conn(0x7f84900b1260 0x7f849014b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:21.926 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.909+0000 7f8496302700 1 --2- 172.21.15.47:0/4258116900 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84900b2120 0x7f84900b2500 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:21.926 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.909+0000 7f8496302700 1 -- 172.21.15.47:0/4258116900 >> 172.21.15.47:0/4258116900 conn(0x7f84900626f0 msgr2=0x7f8490062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:21.927 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.909+0000 7f8496302700 1 -- 172.21.15.47:0/4258116900 shutdown_connections 2022-06-29T15:49:21.927 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.909+0000 7f8496302700 1 -- 172.21.15.47:0/4258116900 wait complete. 2022-06-29T15:49:21.927 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.913+0000 7f8496302700 1 Processor -- start 2022-06-29T15:49:21.927 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.913+0000 7f8496302700 1 -- start start 2022-06-29T15:49:21.927 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.913+0000 7f8496302700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84900b1260 0x7f84901422c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:21.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.913+0000 7f8496302700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84900b2120 0x7f8490156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:21.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.913+0000 7f8496302700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f84900af5f0 0x7f8490143f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:21.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.913+0000 7f8496302700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8490142eb0 con 0x7f84900b1260 2022-06-29T15:49:21.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.913+0000 7f8496302700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8490141bc0 con 0x7f84900b2120 2022-06-29T15:49:21.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.913+0000 7f8496302700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8490145010 con 0x7f84900af5f0 2022-06-29T15:49:21.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.913+0000 7f848ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84900b1260 0x7f84901422c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:21.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.913+0000 7f848ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84900b1260 0x7f84901422c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50526/0 (socket says 172.21.15.47:50526) 2022-06-29T15:49:21.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.913+0000 7f848ffff700 1 -- 172.21.15.47:0/3987785951 learned_addr learned my addr 172.21.15.47:0/3987785951 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:21.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.913+0000 7f848f7fe700 1 --2- 172.21.15.47:0/3987785951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84900b2120 0x7f8490156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:21.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.913+0000 7f849489f700 1 --2- 172.21.15.47:0/3987785951 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f84900af5f0 0x7f8490143f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:21.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.913+0000 7f848ffff700 1 -- 172.21.15.47:0/3987785951 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f84900af5f0 msgr2=0x7f8490143f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:21.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.913+0000 7f848ffff700 1 --2- 172.21.15.47:0/3987785951 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f84900af5f0 0x7f8490143f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:21.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.913+0000 7f848ffff700 1 -- 172.21.15.47:0/3987785951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84900b2120 msgr2=0x7f8490156930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:21.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.913+0000 7f848ffff700 1 --2- 172.21.15.47:0/3987785951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84900b2120 0x7f8490156930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:21.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.913+0000 7f848ffff700 1 -- 172.21.15.47:0/3987785951 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f848400b1e0 con 0x7f84900b1260 2022-06-29T15:49:21.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.913+0000 7f848ffff700 1 --2- 172.21.15.47:0/3987785951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84900b1260 0x7f84901422c0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 rx=0x7f8490141a70 tx=0x7f8484015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:21.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.913+0000 7f848d7fa700 1 -- 172.21.15.47:0/3987785951 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8484011790 con 0x7f84900b1260 2022-06-29T15:49:21.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.913+0000 7f848d7fa700 1 -- 172.21.15.47:0/3987785951 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84840118f0 con 0x7f84900b1260 2022-06-29T15:49:21.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.913+0000 7f8496302700 1 -- 172.21.15.47:0/3987785951 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84901427b0 con 0x7f84900b1260 2022-06-29T15:49:21.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.913+0000 7f848d7fa700 1 -- 172.21.15.47:0/3987785951 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8484011c50 con 0x7f84900b1260 2022-06-29T15:49:21.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.913+0000 7f8496302700 1 -- 172.21.15.47:0/3987785951 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8490141e20 con 0x7f84900b1260 2022-06-29T15:49:21.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.917+0000 7f8496302700 1 -- 172.21.15.47:0/3987785951 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8490156fc0 con 0x7f84900b1260 2022-06-29T15:49:21.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.917+0000 7f848d7fa700 1 -- 172.21.15.47:0/3987785951 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8484025030 con 0x7f84900b1260 2022-06-29T15:49:21.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.917+0000 7f848d7fa700 1 --2- 172.21.15.47:0/3987785951 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8470053b00 0x7f8470055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:21.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.917+0000 7f848d7fa700 1 -- 172.21.15.47:0/3987785951 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5729+0+0 (secure 0 0 0) 0x7f848406cf60 con 0x7f84900b1260 2022-06-29T15:49:21.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.917+0000 7f848f7fe700 1 --2- 172.21.15.47:0/3987785951 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8470053b00 0x7f8470055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:21.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.917+0000 7f848f7fe700 1 --2- 172.21.15.47:0/3987785951 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8470053b00 0x7f8470055fc0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f848000d000 tx=0x7f8480007b90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:21.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:21.925+0000 7f848d7fa700 1 -- 172.21.15.47:0/3987785951 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8484043fd0 con 0x7f84900b1260 2022-06-29T15:49:22.061 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.053+0000 7f8496302700 1 -- 172.21.15.47:0/3987785951 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f8490142020 con 0x7f84900b1260 2022-06-29T15:49:22.063 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.057+0000 7f848d7fa700 1 -- 172.21.15.47:0/3987785951 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v82) v1 ==== 81+0+29292 (secure 0 0 0) 0x7f8490142020 con 0x7f84900b1260 2022-06-29T15:49:22.064 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:49:22.064 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2022-06-29T15:49:22.065 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 82, 2022-06-29T15:49:22.065 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "9a68894a-53ce-400e-838b-14eab91b834c", 2022-06-29T15:49:22.065 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2022-06-29T15:42:14.046361+0000", 2022-06-29T15:49:22.065 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2022-06-29T15:49:19.330828+0000", 2022-06-29T15:49:22.066 INFO:teuthology.orchestra.run.smithi047.stdout: "last_up_change": "2022-06-29T15:42:20.929789+0000", 2022-06-29T15:49:22.066 INFO:teuthology.orchestra.run.smithi047.stdout: "last_in_change": "2022-06-29T15:42:20.929789+0000", 2022-06-29T15:49:22.066 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-06-29T15:49:22.066 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_num": 5799936, 2022-06-29T15:49:22.066 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_set": [ 2022-06-29T15:49:22.067 INFO:teuthology.orchestra.run.smithi047.stdout: "pglog_hardlimit", 2022-06-29T15:49:22.067 INFO:teuthology.orchestra.run.smithi047.stdout: "purged_snapdirs", 2022-06-29T15:49:22.067 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_deletes", 2022-06-29T15:49:22.067 INFO:teuthology.orchestra.run.smithi047.stdout: "sortbitwise" 2022-06-29T15:49:22.068 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:49:22.068 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_version": 5, 2022-06-29T15:49:22.068 INFO:teuthology.orchestra.run.smithi047.stdout: "full_ratio": 0.94999998807907104, 2022-06-29T15:49:22.068 INFO:teuthology.orchestra.run.smithi047.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-06-29T15:49:22.068 INFO:teuthology.orchestra.run.smithi047.stdout: "nearfull_ratio": 0.85000002384185791, 2022-06-29T15:49:22.069 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_snapshot": "", 2022-06-29T15:49:22.069 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_max": 14, 2022-06-29T15:49:22.069 INFO:teuthology.orchestra.run.smithi047.stdout: "max_osd": 8, 2022-06-29T15:49:22.069 INFO:teuthology.orchestra.run.smithi047.stdout: "require_min_compat_client": "jewel", 2022-06-29T15:49:22.070 INFO:teuthology.orchestra.run.smithi047.stdout: "min_compat_client": "jewel", 2022-06-29T15:49:22.070 INFO:teuthology.orchestra.run.smithi047.stdout: "require_osd_release": "octopus", 2022-06-29T15:49:22.070 INFO:teuthology.orchestra.run.smithi047.stdout: "pools": [ 2022-06-29T15:49:22.070 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.071 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 1, 2022-06-29T15:49:22.071 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "device_health_metrics", 2022-06-29T15:49:22.071 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:42:16.971431+0000", 2022-06-29T15:49:22.071 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:49:22.071 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:49:22.072 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:49:22.072 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:49:22.072 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:49:22.072 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:49:22.073 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:49:22.073 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:49:22.073 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 1, 2022-06-29T15:49:22.073 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 1, 2022-06-29T15:49:22.073 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 1, 2022-06-29T15:49:22.074 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 1, 2022-06-29T15:49:22.074 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 1, 2022-06-29T15:49:22.075 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:49:22.075 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:49:22.075 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:49:22.075 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:49:22.076 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:49:22.076 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:49:22.076 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:49:22.076 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.076 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "16", 2022-06-29T15:49:22.077 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:49:22.077 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:49:22.077 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:49:22.077 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:49:22.078 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:49:22.078 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:49:22.078 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:49:22.078 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:49:22.079 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:49:22.079 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:49:22.079 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:49:22.079 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:49:22.079 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:49:22.080 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:49:22.080 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:49:22.080 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:49:22.080 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:49:22.081 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:49:22.081 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:49:22.081 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:49:22.081 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:49:22.081 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:49:22.082 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:49:22.082 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:49:22.082 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:49:22.082 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:49:22.083 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.083 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:49:22.083 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:49:22.083 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:49:22.083 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:49:22.084 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:49:22.084 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:49:22.084 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:49:22.085 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:49:22.085 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:49:22.086 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:49:22.086 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:49:22.086 INFO:teuthology.orchestra.run.smithi047.stdout: "options": { 2022-06-29T15:49:22.087 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_min": 1 2022-06-29T15:49:22.087 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.087 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:49:22.087 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_devicehealth": {} 2022-06-29T15:49:22.088 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:22.088 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.088 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.088 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 2, 2022-06-29T15:49:22.088 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "rbd", 2022-06-29T15:49:22.089 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:42:25.290884+0000", 2022-06-29T15:49:22.089 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:49:22.089 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:49:22.089 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:49:22.090 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:49:22.090 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:49:22.090 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:49:22.091 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:49:22.091 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:49:22.091 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2022-06-29T15:49:22.092 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2022-06-29T15:49:22.092 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2022-06-29T15:49:22.092 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2022-06-29T15:49:22.092 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2022-06-29T15:49:22.093 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:49:22.093 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:49:22.093 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:49:22.093 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:49:22.094 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:49:22.094 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:49:22.094 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:49:22.094 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.095 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "19", 2022-06-29T15:49:22.095 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:49:22.095 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:49:22.095 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:49:22.096 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:49:22.096 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:49:22.096 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:49:22.096 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:49:22.096 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:49:22.097 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:49:22.097 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:49:22.097 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:49:22.097 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:49:22.098 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:49:22.098 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:49:22.098 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:49:22.098 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:49:22.098 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:49:22.099 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:49:22.099 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:49:22.099 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:49:22.099 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:49:22.100 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:49:22.100 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:49:22.100 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:49:22.100 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:49:22.100 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:49:22.101 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.101 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:49:22.101 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:49:22.101 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:49:22.102 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:49:22.102 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:49:22.102 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:49:22.102 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:49:22.103 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:49:22.103 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:49:22.103 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:49:22.103 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:49:22.103 INFO:teuthology.orchestra.run.smithi047.stdout: "options": {}, 2022-06-29T15:49:22.104 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:49:22.104 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd": {} 2022-06-29T15:49:22.104 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:22.104 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:22.104 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:49:22.105 INFO:teuthology.orchestra.run.smithi047.stdout: "osds": [ 2022-06-29T15:49:22.105 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.105 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2022-06-29T15:49:22.105 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "e23eb023-4706-4201-bf72-4bf19c912c5d", 2022-06-29T15:49:22.106 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:49:22.106 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:49:22.106 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:49:22.106 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:49:22.106 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:49:22.107 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:49:22.107 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:49:22.107 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 76, 2022-06-29T15:49:22.107 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:49:22.108 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:49:22.108 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:49:22.108 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:22.108 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.108 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:22.109 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6818", 2022-06-29T15:49:22.109 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:49:22.109 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.109 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.110 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:22.110 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6819", 2022-06-29T15:49:22.110 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:49:22.110 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:22.110 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:22.111 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.111 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:49:22.111 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:22.111 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.112 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:22.112 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6820", 2022-06-29T15:49:22.112 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:49:22.112 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.112 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.113 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:22.113 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6821", 2022-06-29T15:49:22.113 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:49:22.113 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:22.114 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:22.114 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.114 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:49:22.114 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:22.114 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.115 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:22.115 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6824", 2022-06-29T15:49:22.115 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:49:22.115 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.116 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.116 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:22.116 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6825", 2022-06-29T15:49:22.116 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:49:22.116 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:22.117 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:22.117 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.117 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:49:22.117 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:22.118 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.118 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:22.118 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6822", 2022-06-29T15:49:22.118 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:49:22.118 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.119 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.119 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:22.119 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6823", 2022-06-29T15:49:22.119 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:49:22.120 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:22.120 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:22.120 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.120 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6819/13096", 2022-06-29T15:49:22.120 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6821/13096", 2022-06-29T15:49:22.121 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6825/13096", 2022-06-29T15:49:22.121 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6823/13096", 2022-06-29T15:49:22.121 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:49:22.121 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:49:22.122 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:49:22.122 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:22.122 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.122 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.122 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2022-06-29T15:49:22.123 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "228bb08d-2d64-43d9-9560-bfaeff154452", 2022-06-29T15:49:22.123 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:49:22.123 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:49:22.123 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:49:22.124 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:49:22.124 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:49:22.124 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:49:22.124 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:49:22.124 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 66, 2022-06-29T15:49:22.125 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:49:22.125 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:49:22.125 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:49:22.125 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:22.126 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.126 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:22.126 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6826", 2022-06-29T15:49:22.126 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:49:22.126 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.127 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.127 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:22.127 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6827", 2022-06-29T15:49:22.127 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:49:22.128 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:22.128 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:22.128 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.128 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:49:22.128 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:22.129 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.129 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:22.129 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6828", 2022-06-29T15:49:22.129 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:49:22.130 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.130 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.130 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:22.130 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6829", 2022-06-29T15:49:22.131 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:49:22.131 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:22.131 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:22.131 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.132 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:49:22.132 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:22.132 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.132 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:22.132 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6832", 2022-06-29T15:49:22.133 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:49:22.133 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.133 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.133 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:22.134 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6833", 2022-06-29T15:49:22.134 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:49:22.134 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:22.134 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:22.135 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.135 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:49:22.135 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:22.135 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.136 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:22.136 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6830", 2022-06-29T15:49:22.136 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:49:22.136 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.136 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.137 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:22.137 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6831", 2022-06-29T15:49:22.137 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:49:22.137 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:22.138 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:22.138 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.138 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6827/13100", 2022-06-29T15:49:22.138 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6829/13100", 2022-06-29T15:49:22.138 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6833/13100", 2022-06-29T15:49:22.139 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6831/13100", 2022-06-29T15:49:22.139 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:49:22.139 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:49:22.139 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:49:22.140 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:22.140 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.140 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.140 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2022-06-29T15:49:22.140 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "797a5452-1510-4640-a111-c9c2fcf707eb", 2022-06-29T15:49:22.141 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:49:22.141 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:49:22.141 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:49:22.141 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:49:22.142 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:49:22.142 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:49:22.142 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:49:22.142 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 76, 2022-06-29T15:49:22.142 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:49:22.143 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:49:22.143 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:49:22.143 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:22.143 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.143 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:22.144 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6810", 2022-06-29T15:49:22.144 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:49:22.144 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.144 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.145 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:22.145 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6811", 2022-06-29T15:49:22.145 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:49:22.145 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:22.145 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:22.146 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.146 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:49:22.146 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:22.146 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.147 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:22.147 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6812", 2022-06-29T15:49:22.147 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:49:22.147 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.148 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.148 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:22.148 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6813", 2022-06-29T15:49:22.148 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:49:22.149 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:22.149 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:22.149 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.149 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:49:22.149 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:22.150 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.150 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:22.150 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6816", 2022-06-29T15:49:22.151 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:49:22.151 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.151 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.151 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:22.152 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6817", 2022-06-29T15:49:22.152 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:49:22.152 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:22.152 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:22.153 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.153 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:49:22.153 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:22.153 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.154 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:22.154 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6814", 2022-06-29T15:49:22.154 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:49:22.154 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.155 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.155 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:22.155 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6815", 2022-06-29T15:49:22.155 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:49:22.155 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:22.156 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:22.156 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.156 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6811/13098", 2022-06-29T15:49:22.156 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6813/13098", 2022-06-29T15:49:22.157 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6817/13098", 2022-06-29T15:49:22.157 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6815/13098", 2022-06-29T15:49:22.157 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:49:22.157 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:49:22.158 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:49:22.158 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:22.158 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.158 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.158 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2022-06-29T15:49:22.159 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "55b24fd7-bb71-4a01-b3fa-a09caa772a62", 2022-06-29T15:49:22.159 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:49:22.159 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:49:22.159 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:49:22.160 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:49:22.160 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:49:22.160 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:49:22.160 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:49:22.161 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 76, 2022-06-29T15:49:22.161 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:49:22.161 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:49:22.161 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:49:22.161 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:22.162 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.162 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:22.162 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6802", 2022-06-29T15:49:22.162 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:49:22.163 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.163 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.163 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:22.163 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6803", 2022-06-29T15:49:22.164 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:49:22.164 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:22.164 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:22.164 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.165 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:49:22.165 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:22.165 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.165 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:22.165 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6804", 2022-06-29T15:49:22.166 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:49:22.166 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.166 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.166 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:22.167 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6805", 2022-06-29T15:49:22.167 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:49:22.167 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:22.167 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:22.168 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.168 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:49:22.168 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:22.168 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.168 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:22.169 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6808", 2022-06-29T15:49:22.169 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:49:22.169 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.169 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.170 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:22.170 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6809", 2022-06-29T15:49:22.170 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:49:22.170 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:22.171 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:22.171 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.171 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:49:22.171 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:22.171 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.172 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:22.172 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6806", 2022-06-29T15:49:22.172 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:49:22.172 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.173 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.173 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:22.173 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6807", 2022-06-29T15:49:22.173 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:49:22.173 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:22.174 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:22.174 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.174 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6803/13097", 2022-06-29T15:49:22.174 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6805/13097", 2022-06-29T15:49:22.175 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6809/13097", 2022-06-29T15:49:22.175 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6807/13097", 2022-06-29T15:49:22.175 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:49:22.175 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:49:22.176 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:49:22.176 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:22.176 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.176 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.177 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2022-06-29T15:49:22.177 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3", 2022-06-29T15:49:22.177 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:49:22.177 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:49:22.177 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:49:22.178 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:49:22.178 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:49:22.178 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:49:22.178 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:49:22.179 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 75, 2022-06-29T15:49:22.179 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:49:22.179 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:49:22.179 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:49:22.180 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:22.180 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.180 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:22.180 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6816", 2022-06-29T15:49:22.181 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:49:22.181 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.181 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.181 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:22.181 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6817", 2022-06-29T15:49:22.182 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:49:22.182 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:22.182 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:22.182 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.183 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:49:22.183 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:22.183 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.183 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:22.184 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6818", 2022-06-29T15:49:22.184 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:49:22.184 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.184 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.185 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:22.185 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6819", 2022-06-29T15:49:22.185 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:49:22.185 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:22.185 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:22.186 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.186 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:49:22.186 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:22.186 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.187 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:22.187 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6822", 2022-06-29T15:49:22.187 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:49:22.187 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.188 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.188 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:22.188 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6823", 2022-06-29T15:49:22.188 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:49:22.188 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:22.189 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:22.189 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.189 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:49:22.189 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:22.190 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.190 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:22.190 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6820", 2022-06-29T15:49:22.190 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:49:22.191 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.191 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.191 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:22.191 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6821", 2022-06-29T15:49:22.191 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:49:22.192 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:22.192 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:22.192 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.192 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6817/13396", 2022-06-29T15:49:22.193 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6819/13396", 2022-06-29T15:49:22.193 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6823/13396", 2022-06-29T15:49:22.193 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6821/13396", 2022-06-29T15:49:22.193 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:49:22.194 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:49:22.194 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:49:22.194 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:22.194 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.194 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.195 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2022-06-29T15:49:22.195 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "f77beebe-6cca-4244-b2d4-677825ebf919", 2022-06-29T15:49:22.195 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:49:22.195 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:49:22.196 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:49:22.196 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:49:22.196 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:49:22.196 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:49:22.197 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:49:22.197 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 76, 2022-06-29T15:49:22.197 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:49:22.197 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:49:22.198 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:49:22.198 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:22.198 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.198 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:22.198 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6800", 2022-06-29T15:49:22.199 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:49:22.199 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.199 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.199 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:22.200 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6801", 2022-06-29T15:49:22.200 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:49:22.200 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:22.200 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:22.201 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.201 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:49:22.201 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:22.201 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.202 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:22.202 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6802", 2022-06-29T15:49:22.202 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:49:22.202 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.203 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.203 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:22.203 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6803", 2022-06-29T15:49:22.203 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:49:22.204 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:22.204 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:22.204 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.204 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:49:22.205 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:22.205 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.205 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:22.205 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6806", 2022-06-29T15:49:22.206 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:49:22.206 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.206 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.206 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:22.206 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6807", 2022-06-29T15:49:22.207 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:49:22.207 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:22.207 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:22.207 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.208 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:49:22.208 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:22.208 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.208 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:22.208 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6804", 2022-06-29T15:49:22.209 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:49:22.209 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.209 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.209 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:22.210 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6805", 2022-06-29T15:49:22.210 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:49:22.210 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:22.210 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:22.211 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.211 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6801/13393", 2022-06-29T15:49:22.211 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6803/13393", 2022-06-29T15:49:22.211 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6807/13393", 2022-06-29T15:49:22.211 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6805/13393", 2022-06-29T15:49:22.212 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:49:22.212 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:49:22.212 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:49:22.212 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:22.213 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.213 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.213 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2022-06-29T15:49:22.213 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "af0ca9b3-c02c-4910-834b-bf0b8dc6713c", 2022-06-29T15:49:22.214 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:49:22.214 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:49:22.214 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:49:22.214 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:49:22.215 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:49:22.215 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:49:22.215 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:49:22.215 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 76, 2022-06-29T15:49:22.215 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:49:22.216 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:49:22.216 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:49:22.216 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:22.217 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.217 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:22.217 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6824", 2022-06-29T15:49:22.217 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:49:22.218 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.218 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.218 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:22.218 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6825", 2022-06-29T15:49:22.219 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:49:22.219 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:22.219 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:22.219 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.219 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:49:22.220 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:22.220 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.220 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:22.220 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6826", 2022-06-29T15:49:22.221 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:49:22.221 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.221 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.221 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:22.222 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6827", 2022-06-29T15:49:22.222 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:49:22.222 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:22.222 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:22.222 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.223 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:49:22.223 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:22.223 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.223 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:22.224 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6830", 2022-06-29T15:49:22.224 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:49:22.224 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.224 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.225 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:22.225 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6831", 2022-06-29T15:49:22.225 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:49:22.225 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:22.225 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:22.226 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.226 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:49:22.226 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:22.226 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.227 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:22.227 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6828", 2022-06-29T15:49:22.227 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:49:22.227 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.227 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.228 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:22.228 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6829", 2022-06-29T15:49:22.228 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:49:22.228 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:22.229 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:22.229 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.229 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6825/13395", 2022-06-29T15:49:22.229 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6827/13395", 2022-06-29T15:49:22.230 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6831/13395", 2022-06-29T15:49:22.230 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6829/13395", 2022-06-29T15:49:22.230 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:49:22.230 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:49:22.230 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:49:22.231 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:22.231 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.231 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.231 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2022-06-29T15:49:22.232 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "f19c3ca6-df88-4bde-a150-b51a53f092e1", 2022-06-29T15:49:22.232 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:49:22.232 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:49:22.232 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:49:22.233 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:49:22.233 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:49:22.233 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:49:22.233 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:49:22.233 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 75, 2022-06-29T15:49:22.234 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:49:22.234 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:49:22.234 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:49:22.234 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:22.235 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.235 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:22.235 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6808", 2022-06-29T15:49:22.235 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:49:22.236 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.236 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.236 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:22.236 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6809", 2022-06-29T15:49:22.236 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:49:22.237 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:22.237 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:22.237 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.237 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:49:22.238 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:22.238 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.238 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:22.238 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6810", 2022-06-29T15:49:22.238 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:49:22.239 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.239 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.239 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:22.239 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6811", 2022-06-29T15:49:22.240 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:49:22.240 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:22.240 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:22.240 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.241 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:49:22.241 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:22.241 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.241 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:22.241 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6814", 2022-06-29T15:49:22.242 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:49:22.242 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.242 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.242 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:22.243 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6815", 2022-06-29T15:49:22.243 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:49:22.243 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:22.243 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:22.244 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.244 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:49:22.244 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:22.244 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.245 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:22.245 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6812", 2022-06-29T15:49:22.245 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:49:22.245 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.246 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.246 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:22.246 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6813", 2022-06-29T15:49:22.246 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:49:22.247 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:22.247 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:22.247 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.247 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6809/13394", 2022-06-29T15:49:22.247 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6811/13394", 2022-06-29T15:49:22.248 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6815/13394", 2022-06-29T15:49:22.248 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6813/13394", 2022-06-29T15:49:22.248 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:49:22.248 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:49:22.249 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:49:22.249 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:22.249 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:22.249 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:49:22.250 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_xinfo": [ 2022-06-29T15:49:22.250 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.250 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2022-06-29T15:49:22.250 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:49:22.251 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:49:22.251 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:49:22.251 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:49:22.251 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:49:22.252 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.720513+0000", 2022-06-29T15:49:22.252 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:49:22.252 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.252 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.252 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2022-06-29T15:49:22.253 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:49:22.253 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:49:22.253 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:49:22.253 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:49:22.254 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:49:22.254 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.772612+0000", 2022-06-29T15:49:22.254 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:49:22.254 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.255 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.255 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2022-06-29T15:49:22.255 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:49:22.255 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:49:22.255 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:49:22.256 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:49:22.256 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:49:22.256 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.450223+0000", 2022-06-29T15:49:22.256 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:49:22.257 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.257 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.257 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2022-06-29T15:49:22.257 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:49:22.258 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:49:22.258 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:49:22.258 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:49:22.258 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:49:22.258 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.477856+0000", 2022-06-29T15:49:22.259 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:49:22.259 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.259 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.259 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2022-06-29T15:49:22.260 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:49:22.260 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:49:22.260 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:49:22.260 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:49:22.261 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:49:22.261 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.539485+0000", 2022-06-29T15:49:22.261 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:49:22.261 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.261 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.262 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2022-06-29T15:49:22.262 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:49:22.262 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:49:22.262 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:49:22.263 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:49:22.263 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:49:22.263 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.487040+0000", 2022-06-29T15:49:22.263 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:49:22.264 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.264 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.264 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2022-06-29T15:49:22.264 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:49:22.264 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:49:22.265 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:49:22.265 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:49:22.265 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:49:22.265 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.748685+0000", 2022-06-29T15:49:22.266 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:49:22.266 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.266 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:22.266 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2022-06-29T15:49:22.267 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:49:22.267 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:49:22.267 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:49:22.267 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:49:22.268 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:49:22.268 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.459516+0000", 2022-06-29T15:49:22.268 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:49:22.268 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:22.268 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:49:22.269 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap": [], 2022-06-29T15:49:22.269 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap_items": [], 2022-06-29T15:49:22.269 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_temp": [], 2022-06-29T15:49:22.269 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_temp": [], 2022-06-29T15:49:22.270 INFO:teuthology.orchestra.run.smithi047.stdout: "blacklist": { 2022-06-29T15:49:22.270 INFO:teuthology.orchestra.run.smithi047.stdout: "172.21.15.64:6833/3871453337": "2022-06-30T15:49:15.792052+0000", 2022-06-29T15:49:22.270 INFO:teuthology.orchestra.run.smithi047.stdout: "172.21.15.64:6832/3871453337": "2022-06-30T15:49:15.792052+0000", 2022-06-29T15:49:22.270 INFO:teuthology.orchestra.run.smithi047.stdout: "172.21.15.99:0/2790422795": "2022-06-29T16:49:03.956107+0000" 2022-06-29T15:49:22.271 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.271 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profiles": { 2022-06-29T15:49:22.271 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs_data": { 2022-06-29T15:49:22.271 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-device-class": "", 2022-06-29T15:49:22.271 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2022-06-29T15:49:22.272 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-root": "default", 2022-06-29T15:49:22.272 INFO:teuthology.orchestra.run.smithi047.stdout: "jerasure-per-chunk-alignment": "false", 2022-06-29T15:49:22.272 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2022-06-29T15:49:22.272 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "2", 2022-06-29T15:49:22.273 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2022-06-29T15:49:22.273 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2022-06-29T15:49:22.273 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van", 2022-06-29T15:49:22.273 INFO:teuthology.orchestra.run.smithi047.stdout: "w": "8" 2022-06-29T15:49:22.273 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.274 INFO:teuthology.orchestra.run.smithi047.stdout: "default": { 2022-06-29T15:49:22.274 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2022-06-29T15:49:22.274 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2022-06-29T15:49:22.274 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "1", 2022-06-29T15:49:22.275 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2022-06-29T15:49:22.275 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2022-06-29T15:49:22.275 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van" 2022-06-29T15:49:22.275 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:22.276 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:22.276 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps_queue": [], 2022-06-29T15:49:22.276 INFO:teuthology.orchestra.run.smithi047.stdout: "new_removed_snaps": [], 2022-06-29T15:49:22.276 INFO:teuthology.orchestra.run.smithi047.stdout: "new_purged_snaps": [], 2022-06-29T15:49:22.276 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_node_flags": {}, 2022-06-29T15:49:22.277 INFO:teuthology.orchestra.run.smithi047.stdout: "device_class_flags": {} 2022-06-29T15:49:22.277 INFO:teuthology.orchestra.run.smithi047.stdout:} 2022-06-29T15:49:22.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.057+0000 7f8496302700 1 -- 172.21.15.47:0/3987785951 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8470053b00 msgr2=0x7f8470055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:22.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.057+0000 7f8496302700 1 --2- 172.21.15.47:0/3987785951 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8470053b00 0x7f8470055fc0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f848000d000 tx=0x7f8480007b90).stop 2022-06-29T15:49:22.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.057+0000 7f8496302700 1 -- 172.21.15.47:0/3987785951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84900b1260 msgr2=0x7f84901422c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:22.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.057+0000 7f8496302700 1 --2- 172.21.15.47:0/3987785951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84900b1260 0x7f84901422c0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 rx=0x7f8490141a70 tx=0x7f8484015000).stop 2022-06-29T15:49:22.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.057+0000 7f8496302700 1 -- 172.21.15.47:0/3987785951 shutdown_connections 2022-06-29T15:49:22.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.057+0000 7f8496302700 1 --2- 172.21.15.47:0/3987785951 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f84900af5f0 0x7f8490143f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:22.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.057+0000 7f8496302700 1 --2- 172.21.15.47:0/3987785951 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8470053b00 0x7f8470055fc0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:22.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.057+0000 7f8496302700 1 --2- 172.21.15.47:0/3987785951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84900b2120 0x7f8490156930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:22.282 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.057+0000 7f8496302700 1 --2- 172.21.15.47:0/3987785951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84900b1260 0x7f84901422c0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:22.282 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.057+0000 7f8496302700 1 -- 172.21.15.47:0/3987785951 >> 172.21.15.47:0/3987785951 conn(0x7f84900626f0 msgr2=0x7f849014b340 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:22.282 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.057+0000 7f8496302700 1 -- 172.21.15.47:0/3987785951 shutdown_connections 2022-06-29T15:49:22.282 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.057+0000 7f8496302700 1 -- 172.21.15.47:0/3987785951 wait complete. 2022-06-29T15:49:22.283 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:49:22.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.389+0000 7f19cd6f4700 1 Processor -- start 2022-06-29T15:49:22.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.389+0000 7f19cd6f4700 1 -- start start 2022-06-29T15:49:22.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.389+0000 7f19cd6f4700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f19c80615f0 0x7f19c81441c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:22.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.389+0000 7f19cd6f4700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f19c8098200 0x7f19c8140520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:22.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.389+0000 7f19cd6f4700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19c80b19f0 con 0x7f19c80b0b00 2022-06-29T15:49:22.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.389+0000 7f19cd6f4700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19c80619d0 con 0x7f19c8098200 2022-06-29T15:49:22.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.389+0000 7f19cd6f4700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19c80b1080 con 0x7f19c80615f0 2022-06-29T15:49:22.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.393+0000 7f19c6ffd700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f19c80615f0 0x7f19c81441c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:22.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.393+0000 7f19c77fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f19c8098200 0x7f19c8140520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:22.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.393+0000 7f19c6ffd700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f19c80615f0 0x7f19c81441c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:47636/0 (socket says 172.21.15.47:47636) 2022-06-29T15:49:22.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.393+0000 7f19c6ffd700 1 -- 172.21.15.47:0/1574181615 learned_addr learned my addr 172.21.15.47:0/1574181615 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:22.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.393+0000 7f19c77fe700 1 -- 172.21.15.47:0/1574181615 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f19c80615f0 msgr2=0x7f19c81441c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:22.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.393+0000 7f19c77fe700 1 --2- 172.21.15.47:0/1574181615 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f19c80615f0 0x7f19c81441c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:22.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.393+0000 7f19c77fe700 1 -- 172.21.15.47:0/1574181615 >> v1:172.21.15.64:6789/0 conn(0x7f19c80b0b00 legacy=0x7f19c8097af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:22.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.393+0000 7f19c77fe700 1 -- 172.21.15.47:0/1574181615 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c80b1c20 con 0x7f19c8098200 2022-06-29T15:49:22.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.393+0000 7f19c77fe700 1 --2- 172.21.15.47:0/1574181615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19c8098200 0x7f19c8140520 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f19c00032f0 tx=0x7f19c00091d0).ready entity=mon.1 client_cookie=b7b5934784dcf2cc server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:22.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.393+0000 7f19c5ffb700 1 -- 172.21.15.47:0/1574181615 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f19c000b6c0 con 0x7f19c8098200 2022-06-29T15:49:22.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.393+0000 7f19c5ffb700 1 -- 172.21.15.47:0/1574181615 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19c000b820 con 0x7f19c8098200 2022-06-29T15:49:22.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.393+0000 7f19c5ffb700 1 -- 172.21.15.47:0/1574181615 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f19c000b990 con 0x7f19c8098200 2022-06-29T15:49:22.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.393+0000 7f19cd6f4700 1 -- 172.21.15.47:0/1574181615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19c8098200 msgr2=0x7f19c8140520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:22.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.393+0000 7f19cd6f4700 1 --2- 172.21.15.47:0/1574181615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19c8098200 0x7f19c8140520 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f19c00032f0 tx=0x7f19c00091d0).stop 2022-06-29T15:49:22.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.393+0000 7f19cd6f4700 1 -- 172.21.15.47:0/1574181615 shutdown_connections 2022-06-29T15:49:22.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.393+0000 7f19cd6f4700 1 --2- 172.21.15.47:0/1574181615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19c8098200 0x7f19c8140520 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:22.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.393+0000 7f19cd6f4700 1 --2- 172.21.15.47:0/1574181615 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f19c80615f0 0x7f19c81441c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:22.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.393+0000 7f19cd6f4700 1 -- 172.21.15.47:0/1574181615 >> 172.21.15.47:0/1574181615 conn(0x7f19c805f0b0 msgr2=0x7f19c805f490 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:22.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.393+0000 7f19cd6f4700 1 -- 172.21.15.47:0/1574181615 shutdown_connections 2022-06-29T15:49:22.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.393+0000 7f19cd6f4700 1 -- 172.21.15.47:0/1574181615 wait complete. 2022-06-29T15:49:22.410 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.397+0000 7f19cd6f4700 1 Processor -- start 2022-06-29T15:49:22.410 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.397+0000 7f19cd6f4700 1 -- start start 2022-06-29T15:49:22.410 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.397+0000 7f19cd6f4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19c80615f0 0x7f19c8060340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:22.410 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.397+0000 7f19cd6f4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19c8098200 0x7f19c8092700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:22.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.397+0000 7f19cd6f4700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f19c80b0b00 0x7f19c8092bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:22.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.397+0000 7f19cd6f4700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c8067300 con 0x7f19c8098200 2022-06-29T15:49:22.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.397+0000 7f19cd6f4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c8066480 con 0x7f19c80615f0 2022-06-29T15:49:22.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.397+0000 7f19cd6f4700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19c8066720 con 0x7f19c80b0b00 2022-06-29T15:49:22.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.397+0000 7f19c67fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19c8098200 0x7f19c8092700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:22.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.401+0000 7f19c6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19c80615f0 0x7f19c8060340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:22.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.401+0000 7f19c6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19c80615f0 0x7f19c8060340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:56244/0 (socket says 172.21.15.47:56244) 2022-06-29T15:49:22.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.401+0000 7f19c6ffd700 1 -- 172.21.15.47:0/4046636819 learned_addr learned my addr 172.21.15.47:0/4046636819 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:22.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.401+0000 7f19c67fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19c8098200 0x7f19c8092700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50542/0 (socket says 172.21.15.47:50542) 2022-06-29T15:49:22.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.401+0000 7f19c77fe700 1 --2- 172.21.15.47:0/4046636819 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f19c80b0b00 0x7f19c8092bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:22.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.401+0000 7f19c67fc700 1 -- 172.21.15.47:0/4046636819 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f19c80b0b00 msgr2=0x7f19c8092bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:22.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.401+0000 7f19c67fc700 1 --2- 172.21.15.47:0/4046636819 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f19c80b0b00 0x7f19c8092bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:22.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.401+0000 7f19c67fc700 1 -- 172.21.15.47:0/4046636819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19c80615f0 msgr2=0x7f19c8060340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:22.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.401+0000 7f19c67fc700 1 --2- 172.21.15.47:0/4046636819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19c80615f0 0x7f19c8060340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:22.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.401+0000 7f19c67fc700 1 -- 172.21.15.47:0/4046636819 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c0009470 con 0x7f19c8098200 2022-06-29T15:49:22.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.401+0000 7f19c67fc700 1 --2- 172.21.15.47:0/4046636819 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19c8098200 0x7f19c8092700 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 rx=0x7f19c805fd70 tx=0x7f19bc007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:22.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.401+0000 7f19b7fff700 1 -- 172.21.15.47:0/4046636819 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f19bc013070 con 0x7f19c8098200 2022-06-29T15:49:22.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.401+0000 7f19b7fff700 1 -- 172.21.15.47:0/4046636819 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19bc00c640 con 0x7f19c8098200 2022-06-29T15:49:22.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.401+0000 7f19cd6f4700 1 -- 172.21.15.47:0/4046636819 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19c80669c0 con 0x7f19c8098200 2022-06-29T15:49:22.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.401+0000 7f19b7fff700 1 -- 172.21.15.47:0/4046636819 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f19bc00c9a0 con 0x7f19c8098200 2022-06-29T15:49:22.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.401+0000 7f19cd6f4700 1 -- 172.21.15.47:0/4046636819 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19c8066fd0 con 0x7f19c8098200 2022-06-29T15:49:22.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.401+0000 7f19cd6f4700 1 -- 172.21.15.47:0/4046636819 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19c80924a0 con 0x7f19c8098200 2022-06-29T15:49:22.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.401+0000 7f19b7fff700 1 -- 172.21.15.47:0/4046636819 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f19bc00cb00 con 0x7f19c8098200 2022-06-29T15:49:22.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.401+0000 7f19b7fff700 1 --2- 172.21.15.47:0/4046636819 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f19a40539e0 0x7f19a4055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:22.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.401+0000 7f19b7fff700 1 -- 172.21.15.47:0/4046636819 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5729+0+0 (secure 0 0 0) 0x7f19bc06f810 con 0x7f19c8098200 2022-06-29T15:49:22.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.401+0000 7f19c6ffd700 1 --2- 172.21.15.47:0/4046636819 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f19a40539e0 0x7f19a4055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:22.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.405+0000 7f19c6ffd700 1 --2- 172.21.15.47:0/4046636819 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f19a40539e0 0x7f19a4055ea0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f19b8008000 tx=0x7f19b8008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:22.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.409+0000 7f19b7fff700 1 -- 172.21.15.47:0/4046636819 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f19bc046880 con 0x7f19c8098200 2022-06-29T15:49:22.578 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.573+0000 7f19cd6f4700 1 -- 172.21.15.47:0/4046636819 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19c80573d0 con 0x7f19c8098200 2022-06-29T15:49:22.579 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.573+0000 7f19b7fff700 1 -- 172.21.15.47:0/4046636819 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+1559 (secure 0 0 0) 0x7f19bc049ea0 con 0x7f19c8098200 2022-06-29T15:49:22.580 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:49:22.580 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":37,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6291,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.47:6835/2910037081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2910037081},{"type":"v1","addr":"172.21.15.47:6835","nonce":2910037081}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":34},{"gid":6300,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/359532891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":359532891},{"type":"v1","addr":"172.21.15.47:6837","nonce":359532891}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":33},{"gid":6490,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/3453185762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3453185762},{"type":"v1","addr":"172.21.15.64:6833","nonce":3453185762}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":37}],"filesystems":[]} 2022-06-29T15:49:22.582 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.573+0000 7f19cd6f4700 1 -- 172.21.15.47:0/4046636819 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f19a40539e0 msgr2=0x7f19a4055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:22.582 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.573+0000 7f19cd6f4700 1 --2- 172.21.15.47:0/4046636819 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f19a40539e0 0x7f19a4055ea0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f19b8008000 tx=0x7f19b8008040).stop 2022-06-29T15:49:22.582 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.573+0000 7f19cd6f4700 1 -- 172.21.15.47:0/4046636819 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19c8098200 msgr2=0x7f19c8092700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:22.582 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.573+0000 7f19cd6f4700 1 --2- 172.21.15.47:0/4046636819 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19c8098200 0x7f19c8092700 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 rx=0x7f19c805fd70 tx=0x7f19bc007c20).stop 2022-06-29T15:49:22.583 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.573+0000 7f19cd6f4700 1 -- 172.21.15.47:0/4046636819 shutdown_connections 2022-06-29T15:49:22.583 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.573+0000 7f19cd6f4700 1 --2- 172.21.15.47:0/4046636819 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f19c80b0b00 0x7f19c8092bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:22.583 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.573+0000 7f19cd6f4700 1 --2- 172.21.15.47:0/4046636819 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19c8098200 0x7f19c8092700 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:22.583 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.573+0000 7f19cd6f4700 1 --2- 172.21.15.47:0/4046636819 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f19a40539e0 0x7f19a4055ea0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:22.584 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.573+0000 7f19cd6f4700 1 --2- 172.21.15.47:0/4046636819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19c80615f0 0x7f19c8060340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:22.584 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.573+0000 7f19cd6f4700 1 -- 172.21.15.47:0/4046636819 >> 172.21.15.47:0/4046636819 conn(0x7f19c805f0b0 msgr2=0x7f19c809b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:22.584 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.573+0000 7f19cd6f4700 1 -- 172.21.15.47:0/4046636819 shutdown_connections 2022-06-29T15:49:22.584 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.573+0000 7f19cd6f4700 1 -- 172.21.15.47:0/4046636819 wait complete. 2022-06-29T15:49:22.585 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 37 2022-06-29T15:49:22.598 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:49:22.700 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.693+0000 7f3fed0f6700 1 Processor -- start 2022-06-29T15:49:22.701 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.693+0000 7f3fed0f6700 1 -- start start 2022-06-29T15:49:22.704 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.693+0000 7f3fed0f6700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3fe8060eb0 0x7f3fe80b1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:22.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.693+0000 7f3fed0f6700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3fe8145090 0x7f3fe80b2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:22.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.693+0000 7f3fed0f6700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fe80a9120 con 0x7f3fe80b5c60 2022-06-29T15:49:22.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.693+0000 7f3fed0f6700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fe80b0b80 con 0x7f3fe8145090 2022-06-29T15:49:22.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.693+0000 7f3fed0f6700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe80a89e0 con 0x7f3fe8060eb0 2022-06-29T15:49:22.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.693+0000 7f3fe7fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f3fe80b5c60 0x7f3fe8144980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36104/0 (socket says 172.21.15.47:36104) 2022-06-29T15:49:22.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.693+0000 7f3fe7fff700 1 -- 172.21.15.47:0/1474128878 learned_addr learned my addr 172.21.15.47:0/1474128878 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:49:22.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.693+0000 7f3fe77fe700 1 --2- 172.21.15.47:0/1474128878 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3fe8060eb0 0x7f3fe80b1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:22.707 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.693+0000 7f3fec8f5700 1 --2- 172.21.15.47:0/1474128878 >> v2:172.21.15.64:3300/0 conn(0x7f3fe8145090 0x7f3fe80b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:22.707 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.693+0000 7f3fe6ffd700 1 -- 172.21.15.47:0/1474128878 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2124951145 0 0) 0x7f3fe80a9120 con 0x7f3fe80b5c60 2022-06-29T15:49:22.707 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.693+0000 7f3fe6ffd700 1 -- 172.21.15.47:0/1474128878 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fd4003260 con 0x7f3fe80b5c60 2022-06-29T15:49:22.707 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.693+0000 7f3fe77fe700 1 -- 172.21.15.47:0/1474128878 >> v2:172.21.15.64:3300/0 conn(0x7f3fe8145090 msgr2=0x7f3fe80b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:22.708 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.693+0000 7f3fe77fe700 1 --2- 172.21.15.47:0/1474128878 >> v2:172.21.15.64:3300/0 conn(0x7f3fe8145090 0x7f3fe80b2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:22.708 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.693+0000 7f3fe77fe700 1 -- 172.21.15.47:0/1474128878 >> v1:172.21.15.47:6789/0 conn(0x7f3fe80b5c60 legacy=0x7f3fe8144980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:22.708 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.693+0000 7f3fe77fe700 1 -- 172.21.15.47:0/1474128878 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe80a8c80 con 0x7f3fe8060eb0 2022-06-29T15:49:22.708 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.693+0000 7f3fe77fe700 1 --2- 172.21.15.47:0/1474128878 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3fe8060eb0 0x7f3fe80b1e30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f3fd80032f0 tx=0x7f3fd80091d0).ready entity=mon.2 client_cookie=753cd6bb0579838e server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:22.709 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.697+0000 7f3fe6ffd700 1 -- 172.21.15.47:0/1474128878 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3fd800b600 con 0x7f3fe8060eb0 2022-06-29T15:49:22.709 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.697+0000 7f3fe6ffd700 1 -- 172.21.15.47:0/1474128878 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd800b760 con 0x7f3fe8060eb0 2022-06-29T15:49:22.709 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.697+0000 7f3fe6ffd700 1 -- 172.21.15.47:0/1474128878 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3fd800b8d0 con 0x7f3fe8060eb0 2022-06-29T15:49:22.709 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.697+0000 7f3fed0f6700 1 -- 172.21.15.47:0/1474128878 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3fe8060eb0 msgr2=0x7f3fe80b1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:22.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.697+0000 7f3fed0f6700 1 --2- 172.21.15.47:0/1474128878 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3fe8060eb0 0x7f3fe80b1e30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f3fd80032f0 tx=0x7f3fd80091d0).stop 2022-06-29T15:49:22.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.697+0000 7f3fed0f6700 1 -- 172.21.15.47:0/1474128878 shutdown_connections 2022-06-29T15:49:22.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.697+0000 7f3fed0f6700 1 --2- 172.21.15.47:0/1474128878 >> v2:172.21.15.64:3300/0 conn(0x7f3fe8145090 0x7f3fe80b2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:22.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.697+0000 7f3fed0f6700 1 --2- 172.21.15.47:0/1474128878 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3fe8060eb0 0x7f3fe80b1e30 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:22.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.697+0000 7f3fed0f6700 1 -- 172.21.15.47:0/1474128878 >> 172.21.15.47:0/1474128878 conn(0x7f3fe8062560 msgr2=0x7f3fe8062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:22.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.697+0000 7f3fed0f6700 1 -- 172.21.15.47:0/1474128878 shutdown_connections 2022-06-29T15:49:22.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.697+0000 7f3fed0f6700 1 -- 172.21.15.47:0/1474128878 wait complete. 2022-06-29T15:49:22.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.697+0000 7f3fed0f6700 1 Processor -- start 2022-06-29T15:49:22.712 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.697+0000 7f3fed0f6700 1 -- start start 2022-06-29T15:49:22.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.697+0000 7f3fed0f6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3fe8060eb0 0x7f3fe80b4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:22.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.697+0000 7f3fed0f6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fe80b5c60 0x7f3fe80ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:22.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.697+0000 7f3fed0f6700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3fe8145090 0x7f3fe80aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:22.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.697+0000 7f3fed0f6700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe80a9d40 con 0x7f3fe8060eb0 2022-06-29T15:49:22.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.697+0000 7f3fed0f6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe80ab320 con 0x7f3fe80b5c60 2022-06-29T15:49:22.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.697+0000 7f3fed0f6700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe80ad3a0 con 0x7f3fe8145090 2022-06-29T15:49:22.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.697+0000 7f3fe7fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3fe8060eb0 0x7f3fe80b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:22.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.697+0000 7f3fe7fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3fe8060eb0 0x7f3fe80b4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50554/0 (socket says 172.21.15.47:50554) 2022-06-29T15:49:22.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.697+0000 7f3fe7fff700 1 -- 172.21.15.47:0/2710424179 learned_addr learned my addr 172.21.15.47:0/2710424179 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:22.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.697+0000 7f3fe77fe700 1 --2- 172.21.15.47:0/2710424179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fe80b5c60 0x7f3fe80ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:22.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.697+0000 7f3fec8f5700 1 --2- 172.21.15.47:0/2710424179 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3fe8145090 0x7f3fe80aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:22.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.701+0000 7f3fe7fff700 1 -- 172.21.15.47:0/2710424179 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3fe8145090 msgr2=0x7f3fe80aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:22.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.701+0000 7f3fe7fff700 1 --2- 172.21.15.47:0/2710424179 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3fe8145090 0x7f3fe80aedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:22.718 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.701+0000 7f3fe7fff700 1 -- 172.21.15.47:0/2710424179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fe80b5c60 msgr2=0x7f3fe80ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:22.718 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.701+0000 7f3fe7fff700 1 --2- 172.21.15.47:0/2710424179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fe80b5c60 0x7f3fe80ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:22.718 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.701+0000 7f3fe7fff700 1 -- 172.21.15.47:0/2710424179 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd8009470 con 0x7f3fe8060eb0 2022-06-29T15:49:22.718 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.701+0000 7f3fe7fff700 1 --2- 172.21.15.47:0/2710424179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3fe8060eb0 0x7f3fe80b4640 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 rx=0x7f3fe80ab460 tx=0x7f3fdc007a20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:22.719 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.701+0000 7f3fe57fa700 1 -- 172.21.15.47:0/2710424179 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3fdc0070d0 con 0x7f3fe8060eb0 2022-06-29T15:49:22.719 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.701+0000 7f3fe57fa700 1 -- 172.21.15.47:0/2710424179 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fdc007230 con 0x7f3fe8060eb0 2022-06-29T15:49:22.719 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.701+0000 7f3fe57fa700 1 -- 172.21.15.47:0/2710424179 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3fdc005610 con 0x7f3fe8060eb0 2022-06-29T15:49:22.720 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.701+0000 7f3fed0f6700 1 -- 172.21.15.47:0/2710424179 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fe80ad640 con 0x7f3fe8060eb0 2022-06-29T15:49:22.720 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.701+0000 7f3fed0f6700 1 -- 172.21.15.47:0/2710424179 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fe80ade50 con 0x7f3fe8060eb0 2022-06-29T15:49:22.720 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.701+0000 7f3fed0f6700 1 -- 172.21.15.47:0/2710424179 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fe80573d0 con 0x7f3fe8060eb0 2022-06-29T15:49:22.720 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.701+0000 7f3fe57fa700 1 -- 172.21.15.47:0/2710424179 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3fdc010070 con 0x7f3fe8060eb0 2022-06-29T15:49:22.721 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.701+0000 7f3fe57fa700 1 --2- 172.21.15.47:0/2710424179 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3fc8053a70 0x7f3fc8055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:22.721 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.701+0000 7f3fe57fa700 1 -- 172.21.15.47:0/2710424179 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5729+0+0 (secure 0 0 0) 0x7f3fdc06b3f0 con 0x7f3fe8060eb0 2022-06-29T15:49:22.721 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.705+0000 7f3fe77fe700 1 --2- 172.21.15.47:0/2710424179 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3fc8053a70 0x7f3fc8055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:22.721 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.705+0000 7f3fe77fe700 1 --2- 172.21.15.47:0/2710424179 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3fc8053a70 0x7f3fc8055f30 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f3fd801b000 tx=0x7f3fd8016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:22.722 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.709+0000 7f3fe57fa700 1 -- 172.21.15.47:0/2710424179 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3fdc042460 con 0x7f3fe8060eb0 2022-06-29T15:49:22.880 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.873+0000 7f3fed0f6700 1 -- 172.21.15.47:0/2710424179 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3fe80ad7d0 con 0x7f3fe8060eb0 2022-06-29T15:49:22.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.873+0000 7f3fe57fa700 1 -- 172.21.15.47:0/2710424179 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+1559 (secure 0 0 0) 0x7f3fdc045a80 con 0x7f3fe8060eb0 2022-06-29T15:49:22.882 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:49:22.882 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":37,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6291,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.47:6835/2910037081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2910037081},{"type":"v1","addr":"172.21.15.47:6835","nonce":2910037081}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":34},{"gid":6300,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/359532891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":359532891},{"type":"v1","addr":"172.21.15.47:6837","nonce":359532891}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":33},{"gid":6490,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/3453185762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3453185762},{"type":"v1","addr":"172.21.15.64:6833","nonce":3453185762}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":37}],"filesystems":[]} 2022-06-29T15:49:22.885 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.877+0000 7f3fed0f6700 1 -- 172.21.15.47:0/2710424179 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3fc8053a70 msgr2=0x7f3fc8055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:22.885 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.877+0000 7f3fed0f6700 1 --2- 172.21.15.47:0/2710424179 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3fc8053a70 0x7f3fc8055f30 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f3fd801b000 tx=0x7f3fd8016000).stop 2022-06-29T15:49:22.886 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.877+0000 7f3fed0f6700 1 -- 172.21.15.47:0/2710424179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3fe8060eb0 msgr2=0x7f3fe80b4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:22.886 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.877+0000 7f3fed0f6700 1 --2- 172.21.15.47:0/2710424179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3fe8060eb0 0x7f3fe80b4640 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 rx=0x7f3fe80ab460 tx=0x7f3fdc007a20).stop 2022-06-29T15:49:22.886 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.877+0000 7f3fed0f6700 1 -- 172.21.15.47:0/2710424179 shutdown_connections 2022-06-29T15:49:22.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.877+0000 7f3fed0f6700 1 --2- 172.21.15.47:0/2710424179 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3fe8145090 0x7f3fe80aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:22.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.877+0000 7f3fed0f6700 1 --2- 172.21.15.47:0/2710424179 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3fc8053a70 0x7f3fc8055f30 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:22.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.877+0000 7f3fed0f6700 1 --2- 172.21.15.47:0/2710424179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fe80b5c60 0x7f3fe80ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:22.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.877+0000 7f3fed0f6700 1 --2- 172.21.15.47:0/2710424179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3fe8060eb0 0x7f3fe80b4640 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:22.888 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.877+0000 7f3fed0f6700 1 -- 172.21.15.47:0/2710424179 >> 172.21.15.47:0/2710424179 conn(0x7f3fe8062560 msgr2=0x7f3fe80a7240 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:22.888 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.877+0000 7f3fed0f6700 1 -- 172.21.15.47:0/2710424179 shutdown_connections 2022-06-29T15:49:22.888 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:22.877+0000 7f3fed0f6700 1 -- 172.21.15.47:0/2710424179 wait complete. 2022-06-29T15:49:22.889 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 37 2022-06-29T15:49:22.897 INFO:tasks.ceph.mds.a:Restarting daemon 2022-06-29T15:49:22.897 INFO:tasks.ceph.mds.a:Stopping old one... 2022-06-29T15:49:22.898 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-06-29T15:49:22.898 INFO:teuthology.orchestra.run:waiting for 300 2022-06-29T15:49:22.898 INFO:tasks.ceph.mds.c:Restarting daemon 2022-06-29T15:49:22.899 INFO:tasks.ceph.mds.c:Stopping old one... 2022-06-29T15:49:22.899 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2022-06-29T15:49:22.899 INFO:teuthology.orchestra.run:waiting for 300 2022-06-29T15:49:22.900 INFO:tasks.ceph.mds.b:Restarting daemon 2022-06-29T15:49:22.900 INFO:tasks.ceph.mds.b:Stopping old one... 2022-06-29T15:49:22.900 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-06-29T15:49:22.901 INFO:teuthology.orchestra.run:waiting for 300 2022-06-29T15:49:22.909 INFO:tasks.ceph.mds.c:Stopped 2022-06-29T15:49:22.910 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2022-06-29T15:49:22.913 INFO:tasks.ceph.mds.a:Stopped 2022-06-29T15:49:22.913 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-06-29T15:49:22.917 INFO:tasks.ceph.mds.b:Stopped 2022-06-29T15:49:22.917 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-06-29T15:49:22.920 INFO:tasks.ceph.mds.b:Started 2022-06-29T15:49:22.920 DEBUG:teuthology.parallel:result is None 2022-06-29T15:49:22.955 INFO:tasks.ceph.mds.c:Started 2022-06-29T15:49:22.956 DEBUG:teuthology.parallel:result is None 2022-06-29T15:49:22.957 INFO:tasks.ceph.mds.a:Started 2022-06-29T15:49:22.958 DEBUG:teuthology.parallel:result is None 2022-06-29T15:49:22.958 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-06-29T15:49:22.995 INFO:tasks.ceph.mds.b.smithi064.stdout:starting mds.b at 2022-06-29T15:49:23.024 INFO:tasks.ceph.mds.c.smithi047.stdout:starting mds.c at 2022-06-29T15:49:23.031 INFO:tasks.ceph.mds.a.smithi047.stdout:starting mds.a at 2022-06-29T15:49:23.071 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.065+0000 7fef9d222700 1 Processor -- start 2022-06-29T15:49:23.072 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.065+0000 7fef9d222700 1 -- start start 2022-06-29T15:49:23.075 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.065+0000 7fef9d222700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fef980b1470 0x7fef980af590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:23.075 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.065+0000 7fef9d222700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fef980b04e0 0x7fef980ac9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:23.075 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.065+0000 7fef9d222700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fef980aceb0 0x7fef98143090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:23.076 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.065+0000 7fef9d222700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef980b27c0 con 0x7fef980b04e0 2022-06-29T15:49:23.076 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.065+0000 7fef9d222700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef980b1850 con 0x7fef980aceb0 2022-06-29T15:49:23.076 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.065+0000 7fef9d222700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef98142d50 con 0x7fef980b1470 2022-06-29T15:49:23.076 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.065+0000 7fef977fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fef980b04e0 0x7fef980ac9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:23.076 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.065+0000 7fef977fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fef980b04e0 0x7fef980ac9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50588/0 (socket says 172.21.15.47:50588) 2022-06-29T15:49:23.077 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.065+0000 7fef977fe700 1 -- 172.21.15.47:0/3101229924 learned_addr learned my addr 172.21.15.47:0/3101229924 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:23.077 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.065+0000 7fef9ca21700 1 --2- 172.21.15.47:0/3101229924 >> v2:172.21.15.64:3300/0 conn(0x7fef980aceb0 0x7fef98143090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:23.077 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.065+0000 7fef97fff700 1 --2- 172.21.15.47:0/3101229924 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fef980b1470 0x7fef980af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:23.077 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.065+0000 7fef977fe700 1 -- 172.21.15.47:0/3101229924 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fef980b1470 msgr2=0x7fef980af590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:23.078 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.065+0000 7fef977fe700 1 --2- 172.21.15.47:0/3101229924 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fef980b1470 0x7fef980af590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:23.078 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.065+0000 7fef977fe700 1 -- 172.21.15.47:0/3101229924 >> v2:172.21.15.64:3300/0 conn(0x7fef980aceb0 msgr2=0x7fef98143090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:23.078 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.065+0000 7fef977fe700 1 --2- 172.21.15.47:0/3101229924 >> v2:172.21.15.64:3300/0 conn(0x7fef980aceb0 0x7fef98143090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:23.078 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.065+0000 7fef977fe700 1 -- 172.21.15.47:0/3101229924 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef980ae9a0 con 0x7fef980b04e0 2022-06-29T15:49:23.078 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.065+0000 7fef977fe700 1 --2- 172.21.15.47:0/3101229924 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef980b04e0 0x7fef980ac9c0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 rx=0x7fef88002f70 tx=0x7fef88006eb0).ready entity=mon.0 client_cookie=32f83976c12938fb server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:23.079 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.065+0000 7fef96ffd700 1 -- 172.21.15.47:0/3101229924 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef8800d570 con 0x7fef980b04e0 2022-06-29T15:49:23.079 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.065+0000 7fef96ffd700 1 -- 172.21.15.47:0/3101229924 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef88000e90 con 0x7fef980b04e0 2022-06-29T15:49:23.079 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.065+0000 7fef96ffd700 1 -- 172.21.15.47:0/3101229924 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef8800d900 con 0x7fef980b04e0 2022-06-29T15:49:23.079 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.065+0000 7fef9d222700 1 -- 172.21.15.47:0/3101229924 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef980b04e0 msgr2=0x7fef980ac9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:23.080 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.065+0000 7fef9d222700 1 --2- 172.21.15.47:0/3101229924 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef980b04e0 0x7fef980ac9c0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 rx=0x7fef88002f70 tx=0x7fef88006eb0).stop 2022-06-29T15:49:23.080 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.065+0000 7fef9d222700 1 -- 172.21.15.47:0/3101229924 shutdown_connections 2022-06-29T15:49:23.080 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.065+0000 7fef9d222700 1 --2- 172.21.15.47:0/3101229924 >> v2:172.21.15.64:3300/0 conn(0x7fef980aceb0 0x7fef98143090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:23.081 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.065+0000 7fef9d222700 1 --2- 172.21.15.47:0/3101229924 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef980b04e0 0x7fef980ac9c0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:23.081 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.065+0000 7fef9d222700 1 --2- 172.21.15.47:0/3101229924 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fef980b1470 0x7fef980af590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:23.081 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.065+0000 7fef9d222700 1 -- 172.21.15.47:0/3101229924 >> 172.21.15.47:0/3101229924 conn(0x7fef98062560 msgr2=0x7fef98062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:23.081 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.065+0000 7fef9d222700 1 -- 172.21.15.47:0/3101229924 shutdown_connections 2022-06-29T15:49:23.082 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.065+0000 7fef9d222700 1 -- 172.21.15.47:0/3101229924 wait complete. 2022-06-29T15:49:23.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.069+0000 7fef9d222700 1 Processor -- start 2022-06-29T15:49:23.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.069+0000 7fef9d222700 1 -- start start 2022-06-29T15:49:23.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.069+0000 7fef9d222700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fef980aceb0 0x7fef98139b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:23.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.069+0000 7fef9d222700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef980b04e0 0x7fef980abb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:23.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.069+0000 7fef9d222700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef980b1470 0x7fef980abff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:23.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.069+0000 7fef9d222700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9813b890 con 0x7fef980b04e0 2022-06-29T15:49:23.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.069+0000 7fef9d222700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef980ac4e0 con 0x7fef980b1470 2022-06-29T15:49:23.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.069+0000 7fef9d222700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef9813bac0 con 0x7fef980aceb0 2022-06-29T15:49:23.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.069+0000 7fef977fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef980b04e0 0x7fef980abb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:23.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.069+0000 7fef97fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fef980aceb0 0x7fef98139b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:23.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.069+0000 7fef97fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fef980aceb0 0x7fef98139b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:47694/0 (socket says 172.21.15.47:47694) 2022-06-29T15:49:23.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.069+0000 7fef97fff700 1 -- 172.21.15.47:0/840238363 learned_addr learned my addr 172.21.15.47:0/840238363 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:23.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.069+0000 7fef9ca21700 1 --2- 172.21.15.47:0/840238363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef980b1470 0x7fef980abff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:23.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.069+0000 7fef97fff700 1 -- 172.21.15.47:0/840238363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef980b1470 msgr2=0x7fef980abff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:23.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.069+0000 7fef97fff700 1 --2- 172.21.15.47:0/840238363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef980b1470 0x7fef980abff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:23.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.069+0000 7fef97fff700 1 -- 172.21.15.47:0/840238363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef980b04e0 msgr2=0x7fef980abb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:23.089 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.069+0000 7fef97fff700 1 --2- 172.21.15.47:0/840238363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef980b04e0 0x7fef980abb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:23.089 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.069+0000 7fef97fff700 1 -- 172.21.15.47:0/840238363 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef8800b1e0 con 0x7fef980aceb0 2022-06-29T15:49:23.089 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.069+0000 7fef977fe700 1 --2- 172.21.15.47:0/840238363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef980b04e0 0x7fef980abb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:49:23.089 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.069+0000 7fef97fff700 1 --2- 172.21.15.47:0/840238363 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fef980aceb0 0x7fef98139b80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fef980ac670 tx=0x7fef80006d90).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:23.090 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.069+0000 7fef957fa700 1 -- 172.21.15.47:0/840238363 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef80011920 con 0x7fef980aceb0 2022-06-29T15:49:23.090 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.073+0000 7fef957fa700 1 -- 172.21.15.47:0/840238363 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef80011a80 con 0x7fef980aceb0 2022-06-29T15:49:23.090 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.073+0000 7fef9d222700 1 -- 172.21.15.47:0/840238363 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef9813aaf0 con 0x7fef980aceb0 2022-06-29T15:49:23.090 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.073+0000 7fef957fa700 1 -- 172.21.15.47:0/840238363 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef80011de0 con 0x7fef980aceb0 2022-06-29T15:49:23.091 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.073+0000 7fef9d222700 1 -- 172.21.15.47:0/840238363 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef9813b290 con 0x7fef980aceb0 2022-06-29T15:49:23.091 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.073+0000 7fef9d222700 1 -- 172.21.15.47:0/840238363 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef980573d0 con 0x7fef980aceb0 2022-06-29T15:49:23.091 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.073+0000 7fef957fa700 1 -- 172.21.15.47:0/840238363 <== mon.2 v2:172.21.15.64:3301/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fef80021070 con 0x7fef980aceb0 2022-06-29T15:49:23.091 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.073+0000 7fef957fa700 1 --2- 172.21.15.47:0/840238363 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fef7c053980 0x7fef7c055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:23.092 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.073+0000 7fef957fa700 1 -- 172.21.15.47:0/840238363 <== mon.2 v2:172.21.15.64:3301/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5729+0+0 (secure 0 0 0) 0x7fef8006dbe0 con 0x7fef980aceb0 2022-06-29T15:49:23.092 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.073+0000 7fef977fe700 1 --2- 172.21.15.47:0/840238363 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fef7c053980 0x7fef7c055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:23.092 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.073+0000 7fef977fe700 1 --2- 172.21.15.47:0/840238363 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fef7c053980 0x7fef7c055e40 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7fef8801c000 tx=0x7fef88015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:23.093 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.077+0000 7fef957fa700 1 -- 172.21.15.47:0/840238363 <== mon.2 v2:172.21.15.64:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fef80044ae0 con 0x7fef980aceb0 2022-06-29T15:49:23.226 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:23.217+0000 7fef9d222700 1 -- 172.21.15.47:0/840238363 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7fef98069e20 con 0x7fef980aceb0 2022-06-29T15:49:24.074 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.065+0000 7fef957fa700 1 -- 172.21.15.47:0/840238363 <== mon.2 v2:172.21.15.64:3301/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v86) v1 ==== 97+0+0 (secure 0 0 0) 0x7fef80048100 con 0x7fef980aceb0 2022-06-29T15:49:24.080 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.073+0000 7fef9d222700 1 -- 172.21.15.47:0/840238363 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fef7c053980 msgr2=0x7fef7c055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:24.080 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.073+0000 7fef9d222700 1 --2- 172.21.15.47:0/840238363 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fef7c053980 0x7fef7c055e40 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7fef8801c000 tx=0x7fef88015000).stop 2022-06-29T15:49:24.081 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.073+0000 7fef9d222700 1 -- 172.21.15.47:0/840238363 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fef980aceb0 msgr2=0x7fef98139b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:24.081 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.073+0000 7fef9d222700 1 --2- 172.21.15.47:0/840238363 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fef980aceb0 0x7fef98139b80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fef980ac670 tx=0x7fef80006d90).stop 2022-06-29T15:49:24.081 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.073+0000 7fef9d222700 1 -- 172.21.15.47:0/840238363 shutdown_connections 2022-06-29T15:49:24.081 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.073+0000 7fef9d222700 1 --2- 172.21.15.47:0/840238363 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fef7c053980 0x7fef7c055e40 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:24.082 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.073+0000 7fef9d222700 1 --2- 172.21.15.47:0/840238363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef980b1470 0x7fef980abff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:24.082 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.073+0000 7fef9d222700 1 --2- 172.21.15.47:0/840238363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef980b04e0 0x7fef980abb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:24.082 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.073+0000 7fef9d222700 1 --2- 172.21.15.47:0/840238363 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fef980aceb0 0x7fef98139b80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:24.082 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.073+0000 7fef9d222700 1 -- 172.21.15.47:0/840238363 >> 172.21.15.47:0/840238363 conn(0x7fef98062560 msgr2=0x7fef980b5a90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:24.082 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.073+0000 7fef9d222700 1 -- 172.21.15.47:0/840238363 shutdown_connections 2022-06-29T15:49:24.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.073+0000 7fef9d222700 1 -- 172.21.15.47:0/840238363 wait complete. 2022-06-29T15:49:24.083 INFO:teuthology.orchestra.run.smithi047.stderr: removed all blacklist entries 2022-06-29T15:49:24.094 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2022-06-29T15:49:24.094 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-06-29T15:49:24.200 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.193+0000 7fc4b48c2700 1 Processor -- start 2022-06-29T15:49:24.200 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.193+0000 7fc4b48c2700 1 -- start start 2022-06-29T15:49:24.203 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.193+0000 7fc4b48c2700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc4b00611b0 0x7fc4b0146640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:24.203 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.193+0000 7fc4b48c2700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc4b00b06d0 0x7fc4b013ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:24.204 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.193+0000 7fc4b48c2700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4b00b4520 con 0x7fc4b00b0cf0 2022-06-29T15:49:24.204 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.193+0000 7fc4b48c2700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4b00b4b10 con 0x7fc4b00b06d0 2022-06-29T15:49:24.204 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.193+0000 7fc4b48c2700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4b00621d0 con 0x7fc4b00611b0 2022-06-29T15:49:24.204 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.193+0000 7fc4aed9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc4b00b06d0 0x7fc4b013ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:24.205 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.193+0000 7fc4ae59c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc4b00611b0 0x7fc4b0146640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:24.205 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.193+0000 7fc4ae59c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc4b00611b0 0x7fc4b0146640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:56308/0 (socket says 172.21.15.47:56308) 2022-06-29T15:49:24.205 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.193+0000 7fc4ae59c700 1 -- 172.21.15.47:0/672474538 learned_addr learned my addr 172.21.15.47:0/672474538 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:24.205 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.193+0000 7fc4aed9d700 1 -- 172.21.15.47:0/672474538 >> v2:172.21.15.64:3300/0 conn(0x7fc4b00611b0 msgr2=0x7fc4b0146640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:24.206 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.193+0000 7fc4aed9d700 1 --2- 172.21.15.47:0/672474538 >> v2:172.21.15.64:3300/0 conn(0x7fc4b00611b0 0x7fc4b0146640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:24.206 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.193+0000 7fc4aed9d700 1 -- 172.21.15.47:0/672474538 >> v1:172.21.15.64:6789/0 conn(0x7fc4b00b0cf0 legacy=0x7fc4b0146b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:24.206 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.193+0000 7fc4aed9d700 1 -- 172.21.15.47:0/672474538 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b0061590 con 0x7fc4b00b06d0 2022-06-29T15:49:24.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.193+0000 7fc4ae59c700 1 --2- 172.21.15.47:0/672474538 >> v2:172.21.15.64:3300/0 conn(0x7fc4b00611b0 0x7fc4b0146640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:49:24.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.193+0000 7fc4aed9d700 1 --2- 172.21.15.47:0/672474538 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4b00b06d0 0x7fc4b013ff00 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 rx=0x7fc4b00b1c00 tx=0x7fc4a80091d0).ready entity=mon.0 client_cookie=8482965520af4709 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:24.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.193+0000 7fc4add9b700 1 -- 172.21.15.47:0/672474538 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc4a800b640 con 0x7fc4b00b06d0 2022-06-29T15:49:24.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.193+0000 7fc4add9b700 1 -- 172.21.15.47:0/672474538 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4a800b7a0 con 0x7fc4b00b06d0 2022-06-29T15:49:24.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.193+0000 7fc4add9b700 1 -- 172.21.15.47:0/672474538 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc4a800b910 con 0x7fc4b00b06d0 2022-06-29T15:49:24.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.197+0000 7fc4b48c2700 1 -- 172.21.15.47:0/672474538 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4b00b06d0 msgr2=0x7fc4b013ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:24.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.197+0000 7fc4b48c2700 1 --2- 172.21.15.47:0/672474538 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4b00b06d0 0x7fc4b013ff00 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 rx=0x7fc4b00b1c00 tx=0x7fc4a80091d0).stop 2022-06-29T15:49:24.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.197+0000 7fc4b48c2700 1 -- 172.21.15.47:0/672474538 shutdown_connections 2022-06-29T15:49:24.209 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.197+0000 7fc4b48c2700 1 --2- 172.21.15.47:0/672474538 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4b00b06d0 0x7fc4b013ff00 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:24.209 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.197+0000 7fc4b48c2700 1 --2- 172.21.15.47:0/672474538 >> v2:172.21.15.64:3300/0 conn(0x7fc4b00611b0 0x7fc4b0146640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:24.209 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.197+0000 7fc4b48c2700 1 -- 172.21.15.47:0/672474538 >> 172.21.15.47:0/672474538 conn(0x7fc4b00b98b0 msgr2=0x7fc4b00b9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:24.209 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.197+0000 7fc4b48c2700 1 -- 172.21.15.47:0/672474538 shutdown_connections 2022-06-29T15:49:24.210 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.197+0000 7fc4b48c2700 1 -- 172.21.15.47:0/672474538 wait complete. 2022-06-29T15:49:24.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.197+0000 7fc4b48c2700 1 Processor -- start 2022-06-29T15:49:24.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.197+0000 7fc4b48c2700 1 -- start start 2022-06-29T15:49:24.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.197+0000 7fc4b48c2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4b00611b0 0x7fc4b00b8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:24.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.197+0000 7fc4b48c2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4b00b06d0 0x7fc4b0146eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:24.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.197+0000 7fc4b48c2700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc4b00b0cf0 0x7fc4b01526a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:24.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.197+0000 7fc4b48c2700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b01473a0 con 0x7fc4b00611b0 2022-06-29T15:49:24.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.197+0000 7fc4b48c2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b0146790 con 0x7fc4b00b06d0 2022-06-29T15:49:24.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.197+0000 7fc4b48c2700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b00af140 con 0x7fc4b00b0cf0 2022-06-29T15:49:24.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.197+0000 7fc4ae59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4b00611b0 0x7fc4b00b8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:24.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.197+0000 7fc4ae59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4b00611b0 0x7fc4b00b8970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50610/0 (socket says 172.21.15.47:50610) 2022-06-29T15:49:24.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.197+0000 7fc4ae59c700 1 -- 172.21.15.47:0/1663763681 learned_addr learned my addr 172.21.15.47:0/1663763681 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:24.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.197+0000 7fc4a5d9b700 1 --2- 172.21.15.47:0/1663763681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4b00b06d0 0x7fc4b0146eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:24.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.197+0000 7fc4aed9d700 1 --2- 172.21.15.47:0/1663763681 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc4b00b0cf0 0x7fc4b01526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:24.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.197+0000 7fc4ae59c700 1 -- 172.21.15.47:0/1663763681 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc4b00b0cf0 msgr2=0x7fc4b01526a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:24.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.197+0000 7fc4ae59c700 1 --2- 172.21.15.47:0/1663763681 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc4b00b0cf0 0x7fc4b01526a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:24.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.197+0000 7fc4ae59c700 1 -- 172.21.15.47:0/1663763681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4b00b06d0 msgr2=0x7fc4b0146eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:24.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.197+0000 7fc4ae59c700 1 --2- 172.21.15.47:0/1663763681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4b00b06d0 0x7fc4b0146eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:24.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.197+0000 7fc4ae59c700 1 -- 172.21.15.47:0/1663763681 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4a000b1e0 con 0x7fc4b00611b0 2022-06-29T15:49:24.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.201+0000 7fc4ae59c700 1 --2- 172.21.15.47:0/1663763681 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4b00611b0 0x7fc4b00b8970 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 rx=0x7fc4a000c000 tx=0x7fc4a0006eb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:24.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.201+0000 7fc4a7fff700 1 -- 172.21.15.47:0/1663763681 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc4a0012600 con 0x7fc4b00611b0 2022-06-29T15:49:24.222 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.201+0000 7fc4a7fff700 1 -- 172.21.15.47:0/1663763681 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4a0012760 con 0x7fc4b00611b0 2022-06-29T15:49:24.222 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.201+0000 7fc4b48c2700 1 -- 172.21.15.47:0/1663763681 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4a8009470 con 0x7fc4b00611b0 2022-06-29T15:49:24.222 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.201+0000 7fc4a7fff700 1 -- 172.21.15.47:0/1663763681 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc4a0012ac0 con 0x7fc4b00611b0 2022-06-29T15:49:24.222 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.201+0000 7fc4b48c2700 1 -- 172.21.15.47:0/1663763681 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4b0146a10 con 0x7fc4b00611b0 2022-06-29T15:49:24.223 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.201+0000 7fc4b48c2700 1 -- 172.21.15.47:0/1663763681 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4b00b5e00 con 0x7fc4b00611b0 2022-06-29T15:49:24.223 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.201+0000 7fc4a7fff700 1 -- 172.21.15.47:0/1663763681 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc4a0022070 con 0x7fc4b00611b0 2022-06-29T15:49:24.223 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.201+0000 7fc4a7fff700 1 --2- 172.21.15.47:0/1663763681 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc4940539e0 0x7fc494055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:24.223 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.201+0000 7fc4a7fff700 1 -- 172.21.15.47:0/1663763681 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fc4a006c2b0 con 0x7fc4b00611b0 2022-06-29T15:49:24.223 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.201+0000 7fc4a5d9b700 1 --2- 172.21.15.47:0/1663763681 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc4940539e0 0x7fc494055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:24.224 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.201+0000 7fc4a5d9b700 1 --2- 172.21.15.47:0/1663763681 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc4940539e0 0x7fc494055ea0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7fc49800d000 tx=0x7fc498007b70).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:24.224 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.205+0000 7fc4a7fff700 1 -- 172.21.15.47:0/1663763681 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc4a00432a0 con 0x7fc4b00611b0 2022-06-29T15:49:24.380 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.373+0000 7fc4b48c2700 1 -- 172.21.15.47:0/1663763681 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fc4b00573d0 con 0x7fc4b00611b0 2022-06-29T15:49:24.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.373+0000 7fc4a7fff700 1 -- 172.21.15.47:0/1663763681 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v23) v1 ==== 80+0+4255 (secure 0 0 0) 0x7fc4a00468c0 con 0x7fc4b00611b0 2022-06-29T15:49:24.381 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:49:24.381 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2022-06-29T15:49:24.382 INFO:teuthology.orchestra.run.smithi047.stdout: "auth_dump": [ 2022-06-29T15:49:24.382 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:24.382 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.a", 2022-06-29T15:49:24.382 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxiQJFFMxAA9Tv7FJD9u/BKvbkuKld6Kg==", 2022-06-29T15:49:24.383 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:49:24.383 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:49:24.383 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:49:24.383 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:49:24.384 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:49:24.384 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:24.384 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:24.384 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:24.384 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.b", 2022-06-29T15:49:24.385 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC2crxibt6FARAAEf2bwXKFhXKtqr1nN0rdXQ==", 2022-06-29T15:49:24.385 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:49:24.385 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:49:24.386 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:49:24.386 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:49:24.386 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:49:24.386 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:24.386 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:24.387 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:24.387 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.c", 2022-06-29T15:49:24.387 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxi/4ZMORAANpLbgTm3ehqi80USc3swGg==", 2022-06-29T15:49:24.387 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:49:24.388 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:49:24.388 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:49:24.388 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:49:24.388 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:49:24.388 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:24.389 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:24.389 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:24.389 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.0", 2022-06-29T15:49:24.389 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC2crxiY0YsMxAAZivb+CxL/w7vZwQv37Y4oQ==", 2022-06-29T15:49:24.390 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:49:24.390 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:49:24.390 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:49:24.390 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:49:24.391 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:24.391 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:24.391 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:24.391 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.1", 2022-06-29T15:49:24.391 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC5crxitLkKFxAA8+++LUUKLiO9SAuAfYgT5g==", 2022-06-29T15:49:24.392 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:49:24.392 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:49:24.392 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:49:24.392 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:49:24.393 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:24.393 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:24.393 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:24.393 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.2", 2022-06-29T15:49:24.394 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC7crxi1oM9NxAA2FfLdNt32YPwlyNdGQpaQA==", 2022-06-29T15:49:24.394 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:49:24.394 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:49:24.394 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:49:24.395 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:49:24.395 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:24.395 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:24.395 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:24.395 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.3", 2022-06-29T15:49:24.396 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC+crxiTsBgGhAA+gLBBDaskPbmRNWzgKwcHA==", 2022-06-29T15:49:24.396 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:49:24.396 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:49:24.396 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:49:24.397 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:49:24.397 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:24.397 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:24.397 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:24.397 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.4", 2022-06-29T15:49:24.398 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDDcrxivJebIhAAfwWJa7tK4hamkk1/1CSSfw==", 2022-06-29T15:49:24.398 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:49:24.398 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:49:24.398 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:49:24.399 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:49:24.399 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:24.399 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:24.399 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:24.399 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.5", 2022-06-29T15:49:24.400 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDGcrxijuozBhAAt+Dx3wH3WKp3DrbCbhmh8Q==", 2022-06-29T15:49:24.400 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:49:24.400 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:49:24.400 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:49:24.401 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:49:24.401 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:24.401 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:24.401 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:24.401 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.6", 2022-06-29T15:49:24.402 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDIcrxi91kXJxAA9JX7n98P394bD41Q9LUdvQ==", 2022-06-29T15:49:24.402 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:49:24.402 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:49:24.402 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:49:24.403 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:49:24.403 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:24.403 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:24.403 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:24.403 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.7", 2022-06-29T15:49:24.404 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDLcrxiHj0ZDRAASy6fAix8TmEZbAGtuFCfFg==", 2022-06-29T15:49:24.404 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:49:24.404 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:49:24.404 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:49:24.405 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:49:24.405 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:24.405 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:24.405 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:24.405 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.0", 2022-06-29T15:49:24.406 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC2crxiVDlfBxAAezkUfg4e2g0Rfakaa/Ek1Q==", 2022-06-29T15:49:24.406 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:49:24.406 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:49:24.406 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow r", 2022-06-29T15:49:24.407 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-29T15:49:24.407 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:24.407 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:24.407 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:24.408 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.1", 2022-06-29T15:49:24.408 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC2crxisXJ1CxAAhl07oF7AW66NdzfvvZTJmA==", 2022-06-29T15:49:24.408 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:49:24.408 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:49:24.409 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow r", 2022-06-29T15:49:24.409 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-29T15:49:24.409 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:24.409 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:24.409 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:24.410 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.admin", 2022-06-29T15:49:24.410 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxiW7HSFRAAhYqoGU9GFqj2HfUVGValSA==", 2022-06-29T15:49:24.410 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:49:24.410 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2022-06-29T15:49:24.411 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:49:24.411 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:49:24.411 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:49:24.411 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:24.411 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:24.412 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:24.412 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.x", 2022-06-29T15:49:24.412 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxidmuIMBAAcpNlYVu5+icCHg+yIjcKXw==", 2022-06-29T15:49:24.412 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:49:24.413 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2022-06-29T15:49:24.413 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2022-06-29T15:49:24.413 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:49:24.413 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:24.413 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:24.414 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:24.414 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.y", 2022-06-29T15:49:24.414 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxisSMFLBAAv03r4OoPjOP9nI9G/LK0iw==", 2022-06-29T15:49:24.414 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:49:24.415 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2022-06-29T15:49:24.415 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2022-06-29T15:49:24.415 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:49:24.415 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:24.415 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:24.416 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:24.416 INFO:teuthology.orchestra.run.smithi047.stdout:} 2022-06-29T15:49:24.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.377+0000 7fc4b48c2700 1 -- 172.21.15.47:0/1663763681 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc4940539e0 msgr2=0x7fc494055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:24.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.377+0000 7fc4b48c2700 1 --2- 172.21.15.47:0/1663763681 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc4940539e0 0x7fc494055ea0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7fc49800d000 tx=0x7fc498007b70).stop 2022-06-29T15:49:24.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.377+0000 7fc4b48c2700 1 -- 172.21.15.47:0/1663763681 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4b00611b0 msgr2=0x7fc4b00b8970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:24.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.377+0000 7fc4b48c2700 1 --2- 172.21.15.47:0/1663763681 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4b00611b0 0x7fc4b00b8970 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 rx=0x7fc4a000c000 tx=0x7fc4a0006eb0).stop 2022-06-29T15:49:24.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.377+0000 7fc4b48c2700 1 -- 172.21.15.47:0/1663763681 shutdown_connections 2022-06-29T15:49:24.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.377+0000 7fc4b48c2700 1 --2- 172.21.15.47:0/1663763681 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc4b00b0cf0 0x7fc4b01526a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:24.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.377+0000 7fc4b48c2700 1 --2- 172.21.15.47:0/1663763681 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc4940539e0 0x7fc494055ea0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:24.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.377+0000 7fc4b48c2700 1 --2- 172.21.15.47:0/1663763681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4b00b06d0 0x7fc4b0146eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:24.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.377+0000 7fc4b48c2700 1 --2- 172.21.15.47:0/1663763681 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4b00611b0 0x7fc4b00b8970 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:24.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.377+0000 7fc4b48c2700 1 -- 172.21.15.47:0/1663763681 >> 172.21.15.47:0/1663763681 conn(0x7fc4b00b98b0 msgr2=0x7fc4b0141f10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:24.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.377+0000 7fc4b48c2700 1 -- 172.21.15.47:0/1663763681 shutdown_connections 2022-06-29T15:49:24.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.377+0000 7fc4b48c2700 1 -- 172.21.15.47:0/1663763681 wait complete. 2022-06-29T15:49:24.421 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:49:24.529 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.521+0000 7f4e5522d700 1 Processor -- start 2022-06-29T15:49:24.529 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.521+0000 7f4e5522d700 1 -- start start 2022-06-29T15:49:24.532 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.521+0000 7f4e5522d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4e50060eb0 0x7f4e500b1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:24.533 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.521+0000 7f4e5522d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4e50145090 0x7f4e500b2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:24.533 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.521+0000 7f4e5522d700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e500a9120 con 0x7f4e500b5c60 2022-06-29T15:49:24.533 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.521+0000 7f4e5522d700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e500b0b80 con 0x7f4e50060eb0 2022-06-29T15:49:24.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.521+0000 7f4e5522d700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e500a89e0 con 0x7f4e50145090 2022-06-29T15:49:24.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.521+0000 7f4e4f7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4e50060eb0 0x7f4e500b1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:24.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.521+0000 7f4e4f7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4e50060eb0 0x7f4e500b1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50618/0 (socket says 172.21.15.47:50618) 2022-06-29T15:49:24.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.521+0000 7f4e4f7fe700 1 -- 172.21.15.47:0/2387780058 learned_addr learned my addr 172.21.15.47:0/2387780058 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:24.536 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.521+0000 7f4e4ffff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4e500b5c60 0x7f4e50144980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36172/0 (socket says 172.21.15.47:36172) 2022-06-29T15:49:24.536 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.521+0000 7f4e54a2c700 1 --2- 172.21.15.47:0/2387780058 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4e50145090 0x7f4e500b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:24.536 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.521+0000 7f4e4f7fe700 1 -- 172.21.15.47:0/2387780058 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4e50145090 msgr2=0x7f4e500b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:24.537 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.521+0000 7f4e4f7fe700 1 --2- 172.21.15.47:0/2387780058 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4e50145090 0x7f4e500b2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:24.537 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.521+0000 7f4e4f7fe700 1 -- 172.21.15.47:0/2387780058 >> v1:172.21.15.47:6789/0 conn(0x7f4e500b5c60 legacy=0x7f4e50144980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:24.537 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.521+0000 7f4e4f7fe700 1 -- 172.21.15.47:0/2387780058 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e500a8c80 con 0x7f4e50060eb0 2022-06-29T15:49:24.538 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.521+0000 7f4e54a2c700 1 --2- 172.21.15.47:0/2387780058 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4e50145090 0x7f4e500b2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:49:24.538 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.521+0000 7f4e4f7fe700 1 --2- 172.21.15.47:0/2387780058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e50060eb0 0x7f4e500b1e30 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 rx=0x7f4e400032f0 tx=0x7f4e40009290).ready entity=mon.0 client_cookie=72d8cd9c23e40124 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:24.539 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.525+0000 7f4e4effd700 1 -- 172.21.15.47:0/2387780058 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e4000b8a0 con 0x7f4e50060eb0 2022-06-29T15:49:24.539 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.525+0000 7f4e4effd700 1 -- 172.21.15.47:0/2387780058 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e4000ba00 con 0x7f4e50060eb0 2022-06-29T15:49:24.539 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.525+0000 7f4e4effd700 1 -- 172.21.15.47:0/2387780058 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e4000bb70 con 0x7f4e50060eb0 2022-06-29T15:49:24.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.525+0000 7f4e5522d700 1 -- 172.21.15.47:0/2387780058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e50060eb0 msgr2=0x7f4e500b1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:24.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.525+0000 7f4e5522d700 1 --2- 172.21.15.47:0/2387780058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e50060eb0 0x7f4e500b1e30 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 rx=0x7f4e400032f0 tx=0x7f4e40009290).stop 2022-06-29T15:49:24.541 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.525+0000 7f4e5522d700 1 -- 172.21.15.47:0/2387780058 shutdown_connections 2022-06-29T15:49:24.541 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.525+0000 7f4e5522d700 1 --2- 172.21.15.47:0/2387780058 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4e50145090 0x7f4e500b2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:24.541 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.525+0000 7f4e5522d700 1 --2- 172.21.15.47:0/2387780058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e50060eb0 0x7f4e500b1e30 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:24.541 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.525+0000 7f4e5522d700 1 -- 172.21.15.47:0/2387780058 >> 172.21.15.47:0/2387780058 conn(0x7f4e50062560 msgr2=0x7f4e50062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:24.542 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.525+0000 7f4e5522d700 1 -- 172.21.15.47:0/2387780058 shutdown_connections 2022-06-29T15:49:24.542 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.525+0000 7f4e5522d700 1 -- 172.21.15.47:0/2387780058 wait complete. 2022-06-29T15:49:24.542 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.525+0000 7f4e5522d700 1 Processor -- start 2022-06-29T15:49:24.542 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.525+0000 7f4e5522d700 1 -- start start 2022-06-29T15:49:24.546 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.525+0000 7f4e5522d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4e50060eb0 0x7f4e50157310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:24.546 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.525+0000 7f4e5522d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e500b5c60 0x7f4e500b4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:24.546 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.525+0000 7f4e5522d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e50145090 0x7f4e50148b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:24.547 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.525+0000 7f4e5522d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e50156740 con 0x7f4e50145090 2022-06-29T15:49:24.547 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.525+0000 7f4e5522d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e50156930 con 0x7f4e500b5c60 2022-06-29T15:49:24.547 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.525+0000 7f4e5522d700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e50156bd0 con 0x7f4e50060eb0 2022-06-29T15:49:24.547 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.525+0000 7f4e54a2c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e50145090 0x7f4e50148b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:24.548 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.525+0000 7f4e54a2c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e50145090 0x7f4e50148b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50628/0 (socket says 172.21.15.47:50628) 2022-06-29T15:49:24.548 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.525+0000 7f4e54a2c700 1 -- 172.21.15.47:0/3794019228 learned_addr learned my addr 172.21.15.47:0/3794019228 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:24.548 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.525+0000 7f4e4ffff700 1 --2- 172.21.15.47:0/3794019228 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4e50060eb0 0x7f4e50157310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:24.548 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.525+0000 7f4e4f7fe700 1 --2- 172.21.15.47:0/3794019228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e500b5c60 0x7f4e500b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:24.549 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.525+0000 7f4e54a2c700 1 -- 172.21.15.47:0/3794019228 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4e50060eb0 msgr2=0x7f4e50157310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:24.549 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.525+0000 7f4e54a2c700 1 --2- 172.21.15.47:0/3794019228 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4e50060eb0 0x7f4e50157310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:24.549 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.525+0000 7f4e54a2c700 1 -- 172.21.15.47:0/3794019228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e500b5c60 msgr2=0x7f4e500b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:24.549 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.525+0000 7f4e54a2c700 1 --2- 172.21.15.47:0/3794019228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e500b5c60 0x7f4e500b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:24.550 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.525+0000 7f4e54a2c700 1 -- 172.21.15.47:0/3794019228 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e40009530 con 0x7f4e50145090 2022-06-29T15:49:24.550 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.525+0000 7f4e4f7fe700 1 --2- 172.21.15.47:0/3794019228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e500b5c60 0x7f4e500b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:49:24.550 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.525+0000 7f4e54a2c700 1 --2- 172.21.15.47:0/3794019228 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e50145090 0x7f4e50148b70 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 rx=0x7f4e48007000 tx=0x7f4e48008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:24.550 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.529+0000 7f4e4d7fa700 1 -- 172.21.15.47:0/3794019228 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e48010580 con 0x7f4e50145090 2022-06-29T15:49:24.551 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.529+0000 7f4e4d7fa700 1 -- 172.21.15.47:0/3794019228 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e480106e0 con 0x7f4e50145090 2022-06-29T15:49:24.551 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.529+0000 7f4e5522d700 1 -- 172.21.15.47:0/3794019228 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e50156e70 con 0x7f4e50145090 2022-06-29T15:49:24.551 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.529+0000 7f4e4d7fa700 1 -- 172.21.15.47:0/3794019228 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e48010a40 con 0x7f4e50145090 2022-06-29T15:49:24.551 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.529+0000 7f4e5522d700 1 -- 172.21.15.47:0/3794019228 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e50149060 con 0x7f4e50145090 2022-06-29T15:49:24.552 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.529+0000 7f4e5522d700 1 -- 172.21.15.47:0/3794019228 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e50149310 con 0x7f4e50145090 2022-06-29T15:49:24.552 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.529+0000 7f4e4d7fa700 1 -- 172.21.15.47:0/3794019228 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4e48010ba0 con 0x7f4e50145090 2022-06-29T15:49:24.552 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.529+0000 7f4e4d7fa700 1 --2- 172.21.15.47:0/3794019228 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4e30053a70 0x7f4e30055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:24.552 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.529+0000 7f4e4d7fa700 1 -- 172.21.15.47:0/3794019228 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f4e4806be00 con 0x7f4e50145090 2022-06-29T15:49:24.553 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.529+0000 7f4e4ffff700 1 --2- 172.21.15.47:0/3794019228 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4e30053a70 0x7f4e30055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:24.553 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.529+0000 7f4e4ffff700 1 --2- 172.21.15.47:0/3794019228 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4e30053a70 0x7f4e30055f30 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f4e4400b000 tx=0x7f4e44009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:24.553 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.537+0000 7f4e4d7fa700 1 -- 172.21.15.47:0/3794019228 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4e48042ea0 con 0x7f4e50145090 2022-06-29T15:49:24.664 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.657+0000 7f4e5522d700 1 -- 172.21.15.47:0/3794019228 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4e500573d0 con 0x7f4e50145090 2022-06-29T15:49:24.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.657+0000 7f4e4d7fa700 1 -- 172.21.15.47:0/3794019228 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+13415 (secure 0 0 0) 0x7f4e480464c0 con 0x7f4e50145090 2022-06-29T15:49:24.666 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:49:24.666 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":86,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:49:24.063334+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:49:24.668 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.661+0000 7f4e5522d700 1 -- 172.21.15.47:0/3794019228 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4e30053a70 msgr2=0x7f4e30055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:24.668 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.661+0000 7f4e5522d700 1 --2- 172.21.15.47:0/3794019228 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4e30053a70 0x7f4e30055f30 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f4e4400b000 tx=0x7f4e44009040).stop 2022-06-29T15:49:24.669 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.661+0000 7f4e5522d700 1 -- 172.21.15.47:0/3794019228 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e50145090 msgr2=0x7f4e50148b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:24.669 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.661+0000 7f4e5522d700 1 --2- 172.21.15.47:0/3794019228 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e50145090 0x7f4e50148b70 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 rx=0x7f4e48007000 tx=0x7f4e48008040).stop 2022-06-29T15:49:24.669 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.661+0000 7f4e5522d700 1 -- 172.21.15.47:0/3794019228 shutdown_connections 2022-06-29T15:49:24.669 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.661+0000 7f4e5522d700 1 --2- 172.21.15.47:0/3794019228 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e50145090 0x7f4e50148b70 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:24.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.661+0000 7f4e5522d700 1 --2- 172.21.15.47:0/3794019228 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4e30053a70 0x7f4e30055f30 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:24.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.661+0000 7f4e5522d700 1 --2- 172.21.15.47:0/3794019228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e500b5c60 0x7f4e500b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:24.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.661+0000 7f4e5522d700 1 --2- 172.21.15.47:0/3794019228 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4e50060eb0 0x7f4e50157310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:24.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.661+0000 7f4e5522d700 1 -- 172.21.15.47:0/3794019228 >> 172.21.15.47:0/3794019228 conn(0x7f4e50062560 msgr2=0x7f4e501424e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:24.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.661+0000 7f4e5522d700 1 -- 172.21.15.47:0/3794019228 shutdown_connections 2022-06-29T15:49:24.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.661+0000 7f4e5522d700 1 -- 172.21.15.47:0/3794019228 wait complete. 2022-06-29T15:49:24.679 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-29T15:42:16.971431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-29T15:42:25.290884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-06-29T15:49:24.681 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-29T15:49:24.793 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.785+0000 7fae0f59e700 1 Processor -- start 2022-06-29T15:49:24.794 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.785+0000 7fae0f59e700 1 -- start start 2022-06-29T15:49:24.806 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.785+0000 7fae0f59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fae10143150 0x7fae1012c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:24.806 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.785+0000 7fae0f59e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae100b68a0 con 0x7fae100b64c0 2022-06-29T15:49:24.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.785+0000 7fae0f59e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae101473d0 con 0x7fae10066fe0 2022-06-29T15:49:24.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.785+0000 7fae0f59e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae100b13a0 con 0x7fae10143150 2022-06-29T15:49:24.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.785+0000 7fae0e59c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fae100b64c0 0x7fae100b4b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36186/0 (socket says 172.21.15.47:36186) 2022-06-29T15:49:24.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.785+0000 7fae0e59c700 1 -- 172.21.15.47:0/4036819807 learned_addr learned my addr 172.21.15.47:0/4036819807 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:49:24.809 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.785+0000 7fae0ed9d700 1 --2- 172.21.15.47:0/4036819807 >> v2:172.21.15.64:3300/0 conn(0x7fae10143150 0x7fae1012c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:24.809 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.789+0000 7fae0dd9b700 1 -- 172.21.15.47:0/4036819807 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4181319029 0 0) 0x7fae100b68a0 con 0x7fae100b64c0 2022-06-29T15:49:24.810 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.789+0000 7fae0dd9b700 1 -- 172.21.15.47:0/4036819807 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fadf0003260 con 0x7fae100b64c0 2022-06-29T15:49:24.810 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.789+0000 7fae0ed9d700 1 -- 172.21.15.47:0/4036819807 >> v1:172.21.15.64:6789/0 conn(0x7fae10066fe0 legacy=0x7fae1012bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:24.811 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.789+0000 7fae0dd9b700 1 -- 172.21.15.47:0/4036819807 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1698490879 0 0) 0x7fae101473d0 con 0x7fae10066fe0 2022-06-29T15:49:24.811 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.789+0000 7fae0ed9d700 1 -- 172.21.15.47:0/4036819807 >> v1:172.21.15.47:6789/0 conn(0x7fae100b64c0 legacy=0x7fae100b4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:24.812 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.789+0000 7fae0ed9d700 1 -- 172.21.15.47:0/4036819807 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae100b46a0 con 0x7fae10143150 2022-06-29T15:49:24.812 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.789+0000 7fae0ed9d700 1 --2- 172.21.15.47:0/4036819807 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae10143150 0x7fae1012c5d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7fae080032f0 tx=0x7fae080091d0).ready entity=mon.1 client_cookie=8bc4f8e5324f7c1b server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:24.813 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.789+0000 7fae0dd9b700 1 -- 172.21.15.47:0/4036819807 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae0800b6c0 con 0x7fae10143150 2022-06-29T15:49:24.813 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.789+0000 7fae0dd9b700 1 -- 172.21.15.47:0/4036819807 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae0800b820 con 0x7fae10143150 2022-06-29T15:49:24.813 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.789+0000 7fae0dd9b700 1 -- 172.21.15.47:0/4036819807 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae0800b990 con 0x7fae10143150 2022-06-29T15:49:24.814 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.789+0000 7fae0f59e700 1 -- 172.21.15.47:0/4036819807 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae10143150 msgr2=0x7fae1012c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:24.815 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.789+0000 7fae0f59e700 1 --2- 172.21.15.47:0/4036819807 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae10143150 0x7fae1012c5d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7fae080032f0 tx=0x7fae080091d0).stop 2022-06-29T15:49:24.815 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.789+0000 7fae0f59e700 1 -- 172.21.15.47:0/4036819807 shutdown_connections 2022-06-29T15:49:24.815 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.789+0000 7fae0f59e700 1 --2- 172.21.15.47:0/4036819807 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae10143150 0x7fae1012c5d0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:24.816 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.789+0000 7fae0f59e700 1 -- 172.21.15.47:0/4036819807 >> 172.21.15.47:0/4036819807 conn(0x7fae10060e00 msgr2=0x7fae100611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:24.816 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.789+0000 7fae0f59e700 1 -- 172.21.15.47:0/4036819807 shutdown_connections 2022-06-29T15:49:24.817 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.789+0000 7fae0f59e700 1 -- 172.21.15.47:0/4036819807 wait complete. 2022-06-29T15:49:24.817 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.789+0000 7fae0f59e700 1 Processor -- start 2022-06-29T15:49:24.818 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.793+0000 7fae0f59e700 1 -- start start 2022-06-29T15:49:24.818 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.793+0000 7fae0f59e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fae10066fe0 0x7fae10149c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:24.819 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.793+0000 7fae0f59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae100b64c0 0x7fae10152030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:24.819 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.793+0000 7fae0f59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae10143150 0x7fae10142230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:24.820 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.793+0000 7fae0f59e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae100b2660 con 0x7fae10143150 2022-06-29T15:49:24.820 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.793+0000 7fae0f59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae10149570 con 0x7fae100b64c0 2022-06-29T15:49:24.821 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.793+0000 7fae0f59e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae100b2890 con 0x7fae10066fe0 2022-06-29T15:49:24.821 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.793+0000 7fae0ed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae10143150 0x7fae10142230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:24.821 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.793+0000 7fae0ed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae10143150 0x7fae10142230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50640/0 (socket says 172.21.15.47:50640) 2022-06-29T15:49:24.822 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.793+0000 7fae0ed9d700 1 -- 172.21.15.47:0/1336356731 learned_addr learned my addr 172.21.15.47:0/1336356731 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:24.822 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.793+0000 7fae0e59c700 1 --2- 172.21.15.47:0/1336356731 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fae10066fe0 0x7fae10149c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:24.823 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.793+0000 7fae05d9b700 1 --2- 172.21.15.47:0/1336356731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae100b64c0 0x7fae10152030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:24.823 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.793+0000 7fae0ed9d700 1 -- 172.21.15.47:0/1336356731 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fae10066fe0 msgr2=0x7fae10149c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:24.824 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.793+0000 7fae0ed9d700 1 --2- 172.21.15.47:0/1336356731 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fae10066fe0 0x7fae10149c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:24.824 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.793+0000 7fae0ed9d700 1 -- 172.21.15.47:0/1336356731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae100b64c0 msgr2=0x7fae10152030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:24.825 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.793+0000 7fae0ed9d700 1 --2- 172.21.15.47:0/1336356731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae100b64c0 0x7fae10152030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:24.825 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.793+0000 7fae0ed9d700 1 -- 172.21.15.47:0/1336356731 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae08009470 con 0x7fae10143150 2022-06-29T15:49:24.826 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.793+0000 7fae0ed9d700 1 --2- 172.21.15.47:0/1336356731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae10143150 0x7fae10142230 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 rx=0x7fae080032f0 tx=0x7fae08016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:24.826 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.793+0000 7fae07fff700 1 -- 172.21.15.47:0/1336356731 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae0800b6c0 con 0x7fae10143150 2022-06-29T15:49:24.827 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.793+0000 7fae07fff700 1 -- 172.21.15.47:0/1336356731 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae08011e60 con 0x7fae10143150 2022-06-29T15:49:24.827 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.793+0000 7fae0f59e700 1 -- 172.21.15.47:0/1336356731 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae101497d0 con 0x7fae10143150 2022-06-29T15:49:24.827 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.793+0000 7fae07fff700 1 -- 172.21.15.47:0/1336356731 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae0801f5e0 con 0x7fae10143150 2022-06-29T15:49:24.828 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.793+0000 7fae0f59e700 1 -- 172.21.15.47:0/1336356731 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae100b1d50 con 0x7fae10143150 2022-06-29T15:49:24.828 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.793+0000 7fae0f59e700 1 -- 172.21.15.47:0/1336356731 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae101499b0 con 0x7fae10143150 2022-06-29T15:49:24.829 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.797+0000 7fae07fff700 1 -- 172.21.15.47:0/1336356731 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fae0800f070 con 0x7fae10143150 2022-06-29T15:49:24.829 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.797+0000 7fae07fff700 1 --2- 172.21.15.47:0/1336356731 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fadf40539e0 0x7fadf4055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:24.829 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.797+0000 7fae07fff700 1 -- 172.21.15.47:0/1336356731 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fae0806c780 con 0x7fae10143150 2022-06-29T15:49:24.830 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.797+0000 7fae0e59c700 1 --2- 172.21.15.47:0/1336356731 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fadf40539e0 0x7fadf4055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:24.830 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.797+0000 7fae0e59c700 1 --2- 172.21.15.47:0/1336356731 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fadf40539e0 0x7fadf4055ea0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7fae0000d000 tx=0x7fae00009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:24.831 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.805+0000 7fae07fff700 1 -- 172.21.15.47:0/1336356731 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fae101499b0 con 0x7fae10143150 2022-06-29T15:49:24.942 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.933+0000 7fae0f59e700 1 -- 172.21.15.47:0/1336356731 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fae10149960 con 0x7fae10143150 2022-06-29T15:49:24.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.937+0000 7fae07fff700 1 -- 172.21.15.47:0/1336356731 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v86) v1 ==== 110+0+10 (secure 0 0 0) 0x7fae08043880 con 0x7fae10143150 2022-06-29T15:49:24.944 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2022-06-29T15:49:24.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.937+0000 7fae0f59e700 1 -- 172.21.15.47:0/1336356731 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fadf40539e0 msgr2=0x7fadf4055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:24.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.937+0000 7fae0f59e700 1 --2- 172.21.15.47:0/1336356731 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fadf40539e0 0x7fadf4055ea0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7fae0000d000 tx=0x7fae00009040).stop 2022-06-29T15:49:24.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.937+0000 7fae0f59e700 1 -- 172.21.15.47:0/1336356731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae10143150 msgr2=0x7fae10142230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:24.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.937+0000 7fae0f59e700 1 --2- 172.21.15.47:0/1336356731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae10143150 0x7fae10142230 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 rx=0x7fae080032f0 tx=0x7fae08016000).stop 2022-06-29T15:49:24.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.937+0000 7fae0f59e700 1 -- 172.21.15.47:0/1336356731 shutdown_connections 2022-06-29T15:49:24.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.937+0000 7fae0f59e700 1 --2- 172.21.15.47:0/1336356731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae10143150 0x7fae10142230 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:24.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.937+0000 7fae0f59e700 1 --2- 172.21.15.47:0/1336356731 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fadf40539e0 0x7fadf4055ea0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:24.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.937+0000 7fae0f59e700 1 --2- 172.21.15.47:0/1336356731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae100b64c0 0x7fae10152030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:24.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.937+0000 7fae0f59e700 1 --2- 172.21.15.47:0/1336356731 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fae10066fe0 0x7fae10149c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:24.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.937+0000 7fae0f59e700 1 -- 172.21.15.47:0/1336356731 >> 172.21.15.47:0/1336356731 conn(0x7fae10060e00 msgr2=0x7fae100b7650 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:24.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.937+0000 7fae0f59e700 1 -- 172.21.15.47:0/1336356731 shutdown_connections 2022-06-29T15:49:24.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:24.937+0000 7fae0f59e700 1 -- 172.21.15.47:0/1336356731 wait complete. 2022-06-29T15:49:24.956 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-29T15:49:25.065 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.057+0000 7f7467885700 1 Processor -- start 2022-06-29T15:49:25.066 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.057+0000 7f7467885700 1 -- start start 2022-06-29T15:49:25.069 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.057+0000 7f7467885700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7460060af0 0x7f74600b00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:25.069 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.057+0000 7f7467885700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f74600ab700 0x7f74601413c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:25.070 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.057+0000 7f7467885700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7460060ed0 con 0x7f74600a8ce0 2022-06-29T15:49:25.070 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.057+0000 7f7467885700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74600ac4d0 con 0x7f74600ab700 2022-06-29T15:49:25.070 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.057+0000 7f7467885700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74600b6010 con 0x7f7460060af0 2022-06-29T15:49:25.071 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.057+0000 7f7465e22700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f74600ab700 0x7f74601413c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:25.071 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.057+0000 7f7465e22700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f74600ab700 0x7f74601413c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50648/0 (socket says 172.21.15.47:50648) 2022-06-29T15:49:25.071 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.057+0000 7f7465621700 1 --2- 172.21.15.47:0/67078192 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7460060af0 0x7f74600b00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:25.072 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.057+0000 7f7465e22700 1 -- 172.21.15.47:0/67078192 learned_addr learned my addr 172.21.15.47:0/67078192 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:25.072 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.057+0000 7f7464e20700 1 --1- 172.21.15.47:0/67078192 >> v1:172.21.15.64:6789/0 conn(0x7f74600a8ce0 0x7f74600b05c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:43762/0 (socket says 172.21.15.47:43762) 2022-06-29T15:49:25.072 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.061+0000 7f7465e22700 1 -- 172.21.15.47:0/67078192 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7460060af0 msgr2=0x7f74600b00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:25.072 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.061+0000 7f7465e22700 1 --2- 172.21.15.47:0/67078192 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7460060af0 0x7f74600b00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:25.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.061+0000 7f7465e22700 1 -- 172.21.15.47:0/67078192 >> v1:172.21.15.64:6789/0 conn(0x7f74600a8ce0 legacy=0x7f74600b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:25.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.061+0000 7f7457fff700 1 -- 172.21.15.47:0/67078192 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2625964286 0 0) 0x7f7460060ed0 con 0x7f74600a8ce0 2022-06-29T15:49:25.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.061+0000 7f7465e22700 1 -- 172.21.15.47:0/67078192 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74600ac230 con 0x7f74600ab700 2022-06-29T15:49:25.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.061+0000 7f7465e22700 1 --2- 172.21.15.47:0/67078192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74600ab700 0x7f74601413c0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 rx=0x7f74580032f0 tx=0x7f7458009290).ready entity=mon.0 client_cookie=48bfc3c3289cb096 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:25.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.061+0000 7f7457fff700 1 -- 172.21.15.47:0/67078192 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f745800b8a0 con 0x7f74600ab700 2022-06-29T15:49:25.074 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.061+0000 7f7457fff700 1 -- 172.21.15.47:0/67078192 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f745800ba00 con 0x7f74600ab700 2022-06-29T15:49:25.074 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.061+0000 7f7457fff700 1 -- 172.21.15.47:0/67078192 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f745800bb70 con 0x7f74600ab700 2022-06-29T15:49:25.074 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.061+0000 7f7467885700 1 -- 172.21.15.47:0/67078192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74600ab700 msgr2=0x7f74601413c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:25.074 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.061+0000 7f7467885700 1 --2- 172.21.15.47:0/67078192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74600ab700 0x7f74601413c0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 rx=0x7f74580032f0 tx=0x7f7458009290).stop 2022-06-29T15:49:25.075 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.061+0000 7f7467885700 1 -- 172.21.15.47:0/67078192 shutdown_connections 2022-06-29T15:49:25.075 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.061+0000 7f7467885700 1 --2- 172.21.15.47:0/67078192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74600ab700 0x7f74601413c0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:25.076 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.061+0000 7f7467885700 1 --2- 172.21.15.47:0/67078192 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7460060af0 0x7f74600b00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:25.076 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.061+0000 7f7467885700 1 -- 172.21.15.47:0/67078192 >> 172.21.15.47:0/67078192 conn(0x7f74600626f0 msgr2=0x7f7460062ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:25.077 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.061+0000 7f7467885700 1 -- 172.21.15.47:0/67078192 shutdown_connections 2022-06-29T15:49:25.077 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.061+0000 7f7467885700 1 -- 172.21.15.47:0/67078192 wait complete. 2022-06-29T15:49:25.082 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.061+0000 7f7467885700 1 Processor -- start 2022-06-29T15:49:25.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.061+0000 7f7467885700 1 -- start start 2022-06-29T15:49:25.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.065+0000 7f7467885700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7460060af0 0x7f74600b5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:25.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.065+0000 7f7467885700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74600a8ce0 0x7f7460066910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:25.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.065+0000 7f7467885700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74600ab700 0x7f7460063cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:25.084 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.065+0000 7f7467885700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7460066e00 con 0x7f74600a8ce0 2022-06-29T15:49:25.084 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.065+0000 7f7467885700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74601480b0 con 0x7f74600ab700 2022-06-29T15:49:25.084 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.065+0000 7f7467885700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7460148350 con 0x7f7460060af0 2022-06-29T15:49:25.084 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.065+0000 7f7464e20700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74600a8ce0 0x7f7460066910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:25.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.065+0000 7f7464e20700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74600a8ce0 0x7f7460066910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50652/0 (socket says 172.21.15.47:50652) 2022-06-29T15:49:25.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.065+0000 7f7464e20700 1 -- 172.21.15.47:0/3400973609 learned_addr learned my addr 172.21.15.47:0/3400973609 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:25.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.065+0000 7f7465621700 1 --2- 172.21.15.47:0/3400973609 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7460060af0 0x7f74600b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:25.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.065+0000 7f7465e22700 1 --2- 172.21.15.47:0/3400973609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74600ab700 0x7f7460063cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:25.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.065+0000 7f7464e20700 1 -- 172.21.15.47:0/3400973609 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7460060af0 msgr2=0x7f74600b5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:25.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.065+0000 7f7464e20700 1 --2- 172.21.15.47:0/3400973609 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7460060af0 0x7f74600b5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:25.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.065+0000 7f7464e20700 1 -- 172.21.15.47:0/3400973609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74600ab700 msgr2=0x7f7460063cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:25.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.065+0000 7f7464e20700 1 --2- 172.21.15.47:0/3400973609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74600ab700 0x7f7460063cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:25.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.065+0000 7f7464e20700 1 -- 172.21.15.47:0/3400973609 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7458009530 con 0x7f74600a8ce0 2022-06-29T15:49:25.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.065+0000 7f7465e22700 1 --2- 172.21.15.47:0/3400973609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74600ab700 0x7f7460063cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:49:25.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.065+0000 7f7464e20700 1 --2- 172.21.15.47:0/3400973609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74600a8ce0 0x7f7460066910 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 rx=0x7f7450000ed0 tx=0x7f7450007ab0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:25.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.065+0000 7f74567fc700 1 -- 172.21.15.47:0/3400973609 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f745000ce50 con 0x7f74600a8ce0 2022-06-29T15:49:25.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.065+0000 7f74567fc700 1 -- 172.21.15.47:0/3400973609 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7450007170 con 0x7f74600a8ce0 2022-06-29T15:49:25.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.065+0000 7f7467885700 1 -- 172.21.15.47:0/3400973609 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74601485f0 con 0x7f74600a8ce0 2022-06-29T15:49:25.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.065+0000 7f74567fc700 1 -- 172.21.15.47:0/3400973609 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7450004730 con 0x7f74600a8ce0 2022-06-29T15:49:25.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.065+0000 7f7467885700 1 -- 172.21.15.47:0/3400973609 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74600641e0 con 0x7f74600a8ce0 2022-06-29T15:49:25.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.069+0000 7f74567fc700 1 -- 172.21.15.47:0/3400973609 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7450004890 con 0x7f74600a8ce0 2022-06-29T15:49:25.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.069+0000 7f7467885700 1 -- 172.21.15.47:0/3400973609 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74600573d0 con 0x7f74600a8ce0 2022-06-29T15:49:25.089 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.069+0000 7f74567fc700 1 --2- 172.21.15.47:0/3400973609 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7448053a70 0x7f7448055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:25.089 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.069+0000 7f74567fc700 1 -- 172.21.15.47:0/3400973609 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f745006b710 con 0x7f74600a8ce0 2022-06-29T15:49:25.089 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.069+0000 7f7465621700 1 --2- 172.21.15.47:0/3400973609 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7448053a70 0x7f7448055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:25.089 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.069+0000 7f7465621700 1 --2- 172.21.15.47:0/3400973609 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7448053a70 0x7f7448055f30 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7f745c00b000 tx=0x7f745c006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:25.090 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.073+0000 7f74567fc700 1 -- 172.21.15.47:0/3400973609 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f74500427b0 con 0x7f74600a8ce0 2022-06-29T15:49:25.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.201+0000 7f7467885700 1 -- 172.21.15.47:0/3400973609 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f746006a210 con 0x7f74600a8ce0 2022-06-29T15:49:25.209 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.201+0000 7f74567fc700 1 -- 172.21.15.47:0/3400973609 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v86) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7450045dd0 con 0x7f74600a8ce0 2022-06-29T15:49:25.209 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:49:25.211 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.201+0000 7f7467885700 1 -- 172.21.15.47:0/3400973609 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7448053a70 msgr2=0x7f7448055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:25.212 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.201+0000 7f7467885700 1 --2- 172.21.15.47:0/3400973609 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7448053a70 0x7f7448055f30 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7f745c00b000 tx=0x7f745c006df0).stop 2022-06-29T15:49:25.212 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.205+0000 7f7467885700 1 -- 172.21.15.47:0/3400973609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74600a8ce0 msgr2=0x7f7460066910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:25.212 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.205+0000 7f7467885700 1 --2- 172.21.15.47:0/3400973609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74600a8ce0 0x7f7460066910 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 rx=0x7f7450000ed0 tx=0x7f7450007ab0).stop 2022-06-29T15:49:25.213 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.205+0000 7f7467885700 1 -- 172.21.15.47:0/3400973609 shutdown_connections 2022-06-29T15:49:25.213 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.205+0000 7f7467885700 1 --2- 172.21.15.47:0/3400973609 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7448053a70 0x7f7448055f30 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:25.213 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.205+0000 7f7467885700 1 --2- 172.21.15.47:0/3400973609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74600ab700 0x7f7460063cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:25.213 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.205+0000 7f7467885700 1 --2- 172.21.15.47:0/3400973609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74600a8ce0 0x7f7460066910 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:25.214 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.205+0000 7f7467885700 1 --2- 172.21.15.47:0/3400973609 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7460060af0 0x7f74600b5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:25.214 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.205+0000 7f7467885700 1 -- 172.21.15.47:0/3400973609 >> 172.21.15.47:0/3400973609 conn(0x7f74600626f0 msgr2=0x7f7460142ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:25.214 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.205+0000 7f7467885700 1 -- 172.21.15.47:0/3400973609 shutdown_connections 2022-06-29T15:49:25.214 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.205+0000 7f7467885700 1 -- 172.21.15.47:0/3400973609 wait complete. 2022-06-29T15:49:25.223 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-06-29T15:49:25.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.321+0000 7f84fd25e700 1 Processor -- start 2022-06-29T15:49:25.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.321+0000 7f84fd25e700 1 -- start start 2022-06-29T15:49:25.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.321+0000 7f84fd25e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f84f80b5c60 0x7f84f8144980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:25.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.321+0000 7f84fd25e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f84f8060eb0 0x7f84f8144e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:25.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.321+0000 7f84fd25e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84f80a9120 con 0x7f84f80a72f0 2022-06-29T15:49:25.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.321+0000 7f84fd25e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84f8145360 con 0x7f84f80b5c60 2022-06-29T15:49:25.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.321+0000 7f84fd25e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84f80b0b80 con 0x7f84f8060eb0 2022-06-29T15:49:25.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.321+0000 7f84f7fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f84f80b5c60 0x7f84f8144980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:25.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.321+0000 7f84f7fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f84f80b5c60 0x7f84f8144980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50660/0 (socket says 172.21.15.47:50660) 2022-06-29T15:49:25.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.321+0000 7f84f7fff700 1 -- 172.21.15.47:0/2892966755 learned_addr learned my addr 172.21.15.47:0/2892966755 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:25.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.321+0000 7f84f77fe700 1 --2- 172.21.15.47:0/2892966755 >> v2:172.21.15.64:3300/0 conn(0x7f84f8060eb0 0x7f84f8144e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:25.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.321+0000 7f84f7fff700 1 -- 172.21.15.47:0/2892966755 >> v2:172.21.15.64:3300/0 conn(0x7f84f8060eb0 msgr2=0x7f84f8144e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:25.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.325+0000 7f84f7fff700 1 --2- 172.21.15.47:0/2892966755 >> v2:172.21.15.64:3300/0 conn(0x7f84f8060eb0 0x7f84f8144e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:25.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.325+0000 7f84f7fff700 1 -- 172.21.15.47:0/2892966755 >> v1:172.21.15.64:6789/0 conn(0x7f84f80a72f0 legacy=0x7f84f80b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:25.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.325+0000 7f84f6ffd700 1 -- 172.21.15.47:0/2892966755 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1373682472 0 0) 0x7f84f80a9120 con 0x7f84f80a72f0 2022-06-29T15:49:25.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.325+0000 7f84f7fff700 1 -- 172.21.15.47:0/2892966755 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f80a89e0 con 0x7f84f80b5c60 2022-06-29T15:49:25.335 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.325+0000 7f84f77fe700 1 --2- 172.21.15.47:0/2892966755 >> v2:172.21.15.64:3300/0 conn(0x7f84f8060eb0 0x7f84f8144e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:49:25.335 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.325+0000 7f84f7fff700 1 --2- 172.21.15.47:0/2892966755 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84f80b5c60 0x7f84f8144980 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 rx=0x7f84ec002f70 tx=0x7f84ec006e50).ready entity=mon.0 client_cookie=c3855d48c34c6980 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:25.335 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.325+0000 7f84f6ffd700 1 -- 172.21.15.47:0/2892966755 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f84ec00d610 con 0x7f84f80b5c60 2022-06-29T15:49:25.335 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.325+0000 7f84f6ffd700 1 -- 172.21.15.47:0/2892966755 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84ec000e90 con 0x7f84f80b5c60 2022-06-29T15:49:25.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.325+0000 7f84f6ffd700 1 -- 172.21.15.47:0/2892966755 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f84ec00d960 con 0x7f84f80b5c60 2022-06-29T15:49:25.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.325+0000 7f84fd25e700 1 -- 172.21.15.47:0/2892966755 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84f80b5c60 msgr2=0x7f84f8144980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:25.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.325+0000 7f84fd25e700 1 --2- 172.21.15.47:0/2892966755 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84f80b5c60 0x7f84f8144980 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 rx=0x7f84ec002f70 tx=0x7f84ec006e50).stop 2022-06-29T15:49:25.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.325+0000 7f84fd25e700 1 -- 172.21.15.47:0/2892966755 shutdown_connections 2022-06-29T15:49:25.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.325+0000 7f84fd25e700 1 --2- 172.21.15.47:0/2892966755 >> v2:172.21.15.64:3300/0 conn(0x7f84f8060eb0 0x7f84f8144e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:25.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.325+0000 7f84fd25e700 1 --2- 172.21.15.47:0/2892966755 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84f80b5c60 0x7f84f8144980 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:25.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.325+0000 7f84fd25e700 1 -- 172.21.15.47:0/2892966755 >> 172.21.15.47:0/2892966755 conn(0x7f84f8062560 msgr2=0x7f84f8062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:25.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.325+0000 7f84fd25e700 1 -- 172.21.15.47:0/2892966755 shutdown_connections 2022-06-29T15:49:25.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.325+0000 7f84fd25e700 1 -- 172.21.15.47:0/2892966755 wait complete. 2022-06-29T15:49:25.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.329+0000 7f84fd25e700 1 Processor -- start 2022-06-29T15:49:25.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.329+0000 7f84fd25e700 1 -- start start 2022-06-29T15:49:25.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.329+0000 7f84fd25e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84f8060eb0 0x7f84f814a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:25.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.329+0000 7f84fd25e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f84f80a72f0 0x7f84f80adea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:25.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.329+0000 7f84fd25e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84f80b5c60 0x7f84f80ae390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:25.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.329+0000 7f84fd25e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f814ad70 con 0x7f84f8060eb0 2022-06-29T15:49:25.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.329+0000 7f84fd25e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f813f1f0 con 0x7f84f80b5c60 2022-06-29T15:49:25.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.329+0000 7f84fd25e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84f813ed50 con 0x7f84f80a72f0 2022-06-29T15:49:25.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.329+0000 7f84f7fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84f8060eb0 0x7f84f814a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:25.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.329+0000 7f84f7fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84f8060eb0 0x7f84f814a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50664/0 (socket says 172.21.15.47:50664) 2022-06-29T15:49:25.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.329+0000 7f84f7fff700 1 -- 172.21.15.47:0/3746239681 learned_addr learned my addr 172.21.15.47:0/3746239681 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:25.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.329+0000 7f84f77fe700 1 --2- 172.21.15.47:0/3746239681 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f84f80a72f0 0x7f84f80adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:25.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.329+0000 7f84fca5d700 1 --2- 172.21.15.47:0/3746239681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84f80b5c60 0x7f84f80ae390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:25.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.329+0000 7f84f7fff700 1 -- 172.21.15.47:0/3746239681 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f84f80a72f0 msgr2=0x7f84f80adea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:25.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.329+0000 7f84f7fff700 1 --2- 172.21.15.47:0/3746239681 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f84f80a72f0 0x7f84f80adea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:25.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.329+0000 7f84f7fff700 1 -- 172.21.15.47:0/3746239681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84f80b5c60 msgr2=0x7f84f80ae390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:25.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.329+0000 7f84f7fff700 1 --2- 172.21.15.47:0/3746239681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84f80b5c60 0x7f84f80ae390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:25.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.329+0000 7f84f7fff700 1 -- 172.21.15.47:0/3746239681 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e800b180 con 0x7f84f8060eb0 2022-06-29T15:49:25.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.329+0000 7f84fca5d700 1 --2- 172.21.15.47:0/3746239681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84f80b5c60 0x7f84f80ae390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:49:25.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.329+0000 7f84f7fff700 1 --2- 172.21.15.47:0/3746239681 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84f8060eb0 0x7f84f814a230 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 rx=0x7f84ec002f70 tx=0x7f84ec014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:25.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.329+0000 7f84f57fa700 1 -- 172.21.15.47:0/3746239681 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f84ec016070 con 0x7f84f8060eb0 2022-06-29T15:49:25.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.329+0000 7f84f57fa700 1 -- 172.21.15.47:0/3746239681 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84ec00d290 con 0x7f84f8060eb0 2022-06-29T15:49:25.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.329+0000 7f84fd25e700 1 -- 172.21.15.47:0/3746239681 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84ec00b1e0 con 0x7f84f8060eb0 2022-06-29T15:49:25.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.329+0000 7f84f57fa700 1 -- 172.21.15.47:0/3746239681 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f84ec013d50 con 0x7f84f8060eb0 2022-06-29T15:49:25.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.329+0000 7f84fd25e700 1 -- 172.21.15.47:0/3746239681 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84f80ae880 con 0x7f84f8060eb0 2022-06-29T15:49:25.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.333+0000 7f84fd25e700 1 -- 172.21.15.47:0/3746239681 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84f80573d0 con 0x7f84f8060eb0 2022-06-29T15:49:25.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.333+0000 7f84f57fa700 1 -- 172.21.15.47:0/3746239681 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f84ec024030 con 0x7f84f8060eb0 2022-06-29T15:49:25.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.333+0000 7f84f57fa700 1 --2- 172.21.15.47:0/3746239681 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f84d8053a70 0x7f84d8055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:25.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.333+0000 7f84f57fa700 1 -- 172.21.15.47:0/3746239681 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f84ec06c830 con 0x7f84f8060eb0 2022-06-29T15:49:25.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.333+0000 7f84f77fe700 1 --2- 172.21.15.47:0/3746239681 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f84d8053a70 0x7f84d8055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:25.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.333+0000 7f84f77fe700 1 --2- 172.21.15.47:0/3746239681 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f84d8053a70 0x7f84d8055f30 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f84e800b000 tx=0x7f84e8008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:25.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.341+0000 7f84f57fa700 1 -- 172.21.15.47:0/3746239681 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f84ec043930 con 0x7f84f8060eb0 2022-06-29T15:49:25.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.469+0000 7f84fd25e700 1 -- 172.21.15.47:0/3746239681 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f84f814aeb0 con 0x7f84f8060eb0 2022-06-29T15:49:25.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.469+0000 7f84f57fa700 1 -- 172.21.15.47:0/3746239681 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v86) v1 ==== 84+0+143 (secure 0 0 0) 0x7f84f814aeb0 con 0x7f84f8060eb0 2022-06-29T15:49:25.477 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:49:25.477 INFO:teuthology.orchestra.run.smithi047.stdout:[ 2022-06-29T15:49:25.477 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:25.477 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 1, 2022-06-29T15:49:25.478 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": "device_health_metrics" 2022-06-29T15:49:25.478 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:25.478 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:25.478 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 2, 2022-06-29T15:49:25.478 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": "rbd" 2022-06-29T15:49:25.479 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:25.479 INFO:teuthology.orchestra.run.smithi047.stdout:] 2022-06-29T15:49:25.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.469+0000 7f84fd25e700 1 -- 172.21.15.47:0/3746239681 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f84d8053a70 msgr2=0x7f84d8055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:25.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.469+0000 7f84fd25e700 1 --2- 172.21.15.47:0/3746239681 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f84d8053a70 0x7f84d8055f30 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f84e800b000 tx=0x7f84e8008040).stop 2022-06-29T15:49:25.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.469+0000 7f84fd25e700 1 -- 172.21.15.47:0/3746239681 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84f8060eb0 msgr2=0x7f84f814a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:25.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.469+0000 7f84fd25e700 1 --2- 172.21.15.47:0/3746239681 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84f8060eb0 0x7f84f814a230 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 rx=0x7f84ec002f70 tx=0x7f84ec014000).stop 2022-06-29T15:49:25.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.469+0000 7f84fd25e700 1 -- 172.21.15.47:0/3746239681 shutdown_connections 2022-06-29T15:49:25.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.469+0000 7f84fd25e700 1 --2- 172.21.15.47:0/3746239681 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f84d8053a70 0x7f84d8055f30 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:25.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.469+0000 7f84fd25e700 1 --2- 172.21.15.47:0/3746239681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84f80b5c60 0x7f84f80ae390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:25.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.469+0000 7f84fd25e700 1 --2- 172.21.15.47:0/3746239681 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f84f80a72f0 0x7f84f80adea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:25.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.469+0000 7f84fd25e700 1 --2- 172.21.15.47:0/3746239681 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84f8060eb0 0x7f84f814a230 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:25.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.469+0000 7f84fd25e700 1 -- 172.21.15.47:0/3746239681 >> 172.21.15.47:0/3746239681 conn(0x7f84f8062560 msgr2=0x7f84f80afae0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:25.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.469+0000 7f84fd25e700 1 -- 172.21.15.47:0/3746239681 shutdown_connections 2022-06-29T15:49:25.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.469+0000 7f84fd25e700 1 -- 172.21.15.47:0/3746239681 wait complete. 2022-06-29T15:49:25.490 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-06-29T15:49:25.490 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-06-29T15:49:25.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.585+0000 7f66688d5700 1 Processor -- start 2022-06-29T15:49:25.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.585+0000 7f66688d5700 1 -- start start 2022-06-29T15:49:25.594 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.585+0000 7f66688d5700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6664144be0 0x7f66640b1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:25.594 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.585+0000 7f66688d5700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f66640b2440 0x7f666405f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:25.594 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.585+0000 7f66688d5700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66641476e0 con 0x7f666405f5a0 2022-06-29T15:49:25.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.585+0000 7f66688d5700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6664147980 con 0x7f66640b2440 2022-06-29T15:49:25.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.585+0000 7f66688d5700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6664147c20 con 0x7f6664144be0 2022-06-29T15:49:25.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.585+0000 7f6662ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f66640b2440 0x7f666405f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:25.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.585+0000 7f6662ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f66640b2440 0x7f666405f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50676/0 (socket says 172.21.15.47:50676) 2022-06-29T15:49:25.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.585+0000 7f6663fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f666405f5a0 0x7f666414ab80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36228/0 (socket says 172.21.15.47:36228) 2022-06-29T15:49:25.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.585+0000 7f6662ffd700 1 -- 172.21.15.47:0/3917601369 learned_addr learned my addr 172.21.15.47:0/3917601369 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:25.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.585+0000 7f66637fe700 1 --2- 172.21.15.47:0/3917601369 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6664144be0 0x7f66640b1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:25.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.585+0000 7f6662ffd700 1 -- 172.21.15.47:0/3917601369 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6664144be0 msgr2=0x7f66640b1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:25.597 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.585+0000 7f6662ffd700 1 --2- 172.21.15.47:0/3917601369 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6664144be0 0x7f66640b1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:25.597 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.585+0000 7f6662ffd700 1 -- 172.21.15.47:0/3917601369 >> v1:172.21.15.47:6789/0 conn(0x7f666405f5a0 legacy=0x7f666414ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:25.597 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.585+0000 7f6662ffd700 1 -- 172.21.15.47:0/3917601369 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6664147ec0 con 0x7f66640b2440 2022-06-29T15:49:25.597 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.585+0000 7f66627fc700 1 -- 172.21.15.47:0/3917601369 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 84893778 0 0) 0x7f66641476e0 con 0x7f666405f5a0 2022-06-29T15:49:25.597 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.585+0000 7f6662ffd700 1 --2- 172.21.15.47:0/3917601369 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66640b2440 0x7f666405f0b0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 rx=0x7f6658002f70 tx=0x7f6658006eb0).ready entity=mon.0 client_cookie=e682f228a3c877ca server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:25.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.585+0000 7f66627fc700 1 -- 172.21.15.47:0/3917601369 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f665800d5c0 con 0x7f66640b2440 2022-06-29T15:49:25.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.585+0000 7f66627fc700 1 -- 172.21.15.47:0/3917601369 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6658000e90 con 0x7f66640b2440 2022-06-29T15:49:25.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.585+0000 7f66627fc700 1 -- 172.21.15.47:0/3917601369 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f665800d910 con 0x7f66640b2440 2022-06-29T15:49:25.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.589+0000 7f66688d5700 1 -- 172.21.15.47:0/3917601369 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66640b2440 msgr2=0x7f666405f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:25.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.589+0000 7f66688d5700 1 --2- 172.21.15.47:0/3917601369 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66640b2440 0x7f666405f0b0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 rx=0x7f6658002f70 tx=0x7f6658006eb0).stop 2022-06-29T15:49:25.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.589+0000 7f66688d5700 1 -- 172.21.15.47:0/3917601369 shutdown_connections 2022-06-29T15:49:25.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.589+0000 7f66688d5700 1 --2- 172.21.15.47:0/3917601369 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66640b2440 0x7f666405f0b0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:25.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.589+0000 7f66688d5700 1 --2- 172.21.15.47:0/3917601369 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6664144be0 0x7f66640b1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:25.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.589+0000 7f66688d5700 1 -- 172.21.15.47:0/3917601369 >> 172.21.15.47:0/3917601369 conn(0x7f6664062570 msgr2=0x7f66640647d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:25.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.589+0000 7f66688d5700 1 -- 172.21.15.47:0/3917601369 shutdown_connections 2022-06-29T15:49:25.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.589+0000 7f66688d5700 1 -- 172.21.15.47:0/3917601369 wait complete. 2022-06-29T15:49:25.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.589+0000 7f66688d5700 1 Processor -- start 2022-06-29T15:49:25.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.589+0000 7f66688d5700 1 -- start start 2022-06-29T15:49:25.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.589+0000 7f66688d5700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6664144be0 0x7f6664066710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:25.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.589+0000 7f66688d5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6664149b80 0x7f666415a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:25.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.589+0000 7f66688d5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f666414a000 0x7f666415a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:25.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.589+0000 7f66688d5700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6664066fa0 con 0x7f6664149b80 2022-06-29T15:49:25.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.589+0000 7f66688d5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66640ad150 con 0x7f666414a000 2022-06-29T15:49:25.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.589+0000 7f66688d5700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66640ad380 con 0x7f6664144be0 2022-06-29T15:49:25.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.589+0000 7f6662ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6664149b80 0x7f666415a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:25.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.589+0000 7f6662ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6664149b80 0x7f666415a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50682/0 (socket says 172.21.15.47:50682) 2022-06-29T15:49:25.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.589+0000 7f6662ffd700 1 -- 172.21.15.47:0/1262042618 learned_addr learned my addr 172.21.15.47:0/1262042618 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:25.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.589+0000 7f66637fe700 1 --2- 172.21.15.47:0/1262042618 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6664144be0 0x7f6664066710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:25.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.589+0000 7f6663fff700 1 --2- 172.21.15.47:0/1262042618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f666414a000 0x7f666415a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:25.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.593+0000 7f6662ffd700 1 -- 172.21.15.47:0/1262042618 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6664144be0 msgr2=0x7f6664066710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:25.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.593+0000 7f6662ffd700 1 --2- 172.21.15.47:0/1262042618 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6664144be0 0x7f6664066710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:25.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.593+0000 7f6662ffd700 1 -- 172.21.15.47:0/1262042618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f666414a000 msgr2=0x7f666415a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:25.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.593+0000 7f6662ffd700 1 --2- 172.21.15.47:0/1262042618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f666414a000 0x7f666415a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:25.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.593+0000 7f6662ffd700 1 -- 172.21.15.47:0/1262042618 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f665800b1e0 con 0x7f6664149b80 2022-06-29T15:49:25.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.593+0000 7f6663fff700 1 --2- 172.21.15.47:0/1262042618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f666414a000 0x7f666415a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:49:25.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.593+0000 7f6662ffd700 1 --2- 172.21.15.47:0/1262042618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6664149b80 0x7f666415a020 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 rx=0x7f6658002f70 tx=0x7f6658015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:25.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.593+0000 7f6660ff9700 1 -- 172.21.15.47:0/1262042618 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f665800de40 con 0x7f6664149b80 2022-06-29T15:49:25.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.593+0000 7f6660ff9700 1 -- 172.21.15.47:0/1262042618 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6658011b80 con 0x7f6664149b80 2022-06-29T15:49:25.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.593+0000 7f66688d5700 1 -- 172.21.15.47:0/1262042618 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66640ad620 con 0x7f6664149b80 2022-06-29T15:49:25.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.593+0000 7f6660ff9700 1 -- 172.21.15.47:0/1262042618 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f665801e460 con 0x7f6664149b80 2022-06-29T15:49:25.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.593+0000 7f66688d5700 1 -- 172.21.15.47:0/1262042618 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66640adb60 con 0x7f6664149b80 2022-06-29T15:49:25.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.593+0000 7f66688d5700 1 -- 172.21.15.47:0/1262042618 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66640573d0 con 0x7f6664149b80 2022-06-29T15:49:25.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.593+0000 7f6660ff9700 1 -- 172.21.15.47:0/1262042618 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6658025030 con 0x7f6664149b80 2022-06-29T15:49:25.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.593+0000 7f6660ff9700 1 --2- 172.21.15.47:0/1262042618 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6644053b50 0x7f6644056010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:25.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.597+0000 7f6660ff9700 1 -- 172.21.15.47:0/1262042618 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f665806cfe0 con 0x7f6664149b80 2022-06-29T15:49:25.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.597+0000 7f66637fe700 1 --2- 172.21.15.47:0/1262042618 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6644053b50 0x7f6644056010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:25.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.597+0000 7f66637fe700 1 --2- 172.21.15.47:0/1262042618 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6644053b50 0x7f6644056010 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7f665400c000 tx=0x7f6654008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:25.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.601+0000 7f6660ff9700 1 -- 172.21.15.47:0/1262042618 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6658044030 con 0x7f6664149b80 2022-06-29T15:49:25.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:25.729+0000 7f66688d5700 1 -- 172.21.15.47:0/1262042618 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f6664068830 con 0x7f6664149b80 2022-06-29T15:49:26.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.389+0000 7f6660ff9700 1 -- 172.21.15.47:0/1262042618 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v87) v1 ==== 133+0+0 (secure 0 0 0) 0x7f66580230a0 con 0x7f6664149b80 2022-06-29T15:49:26.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.393+0000 7f66688d5700 1 -- 172.21.15.47:0/1262042618 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6644053b50 msgr2=0x7f6644056010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:26.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.393+0000 7f66688d5700 1 --2- 172.21.15.47:0/1262042618 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6644053b50 0x7f6644056010 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7f665400c000 tx=0x7f6654008040).stop 2022-06-29T15:49:26.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.393+0000 7f66688d5700 1 -- 172.21.15.47:0/1262042618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6664149b80 msgr2=0x7f666415a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:26.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.393+0000 7f66688d5700 1 --2- 172.21.15.47:0/1262042618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6664149b80 0x7f666415a020 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 rx=0x7f6658002f70 tx=0x7f6658015000).stop 2022-06-29T15:49:26.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.393+0000 7f66688d5700 1 -- 172.21.15.47:0/1262042618 shutdown_connections 2022-06-29T15:49:26.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.393+0000 7f66688d5700 1 --2- 172.21.15.47:0/1262042618 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6644053b50 0x7f6644056010 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:26.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.393+0000 7f66688d5700 1 --2- 172.21.15.47:0/1262042618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f666414a000 0x7f666415a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:26.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.393+0000 7f66688d5700 1 --2- 172.21.15.47:0/1262042618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6664149b80 0x7f666415a020 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:26.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.393+0000 7f66688d5700 1 --2- 172.21.15.47:0/1262042618 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6664144be0 0x7f6664066710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:26.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.393+0000 7f66688d5700 1 -- 172.21.15.47:0/1262042618 >> 172.21.15.47:0/1262042618 conn(0x7f6664062570 msgr2=0x7f6664064740 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:26.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.393+0000 7f66688d5700 1 -- 172.21.15.47:0/1262042618 shutdown_connections 2022-06-29T15:49:26.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.393+0000 7f66688d5700 1 -- 172.21.15.47:0/1262042618 wait complete. 2022-06-29T15:49:26.404 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' created 2022-06-29T15:49:26.417 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-06-29T15:49:26.529 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.521+0000 7f7901912700 1 Processor -- start 2022-06-29T15:49:26.529 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.521+0000 7f7901912700 1 -- start start 2022-06-29T15:49:26.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.521+0000 7f7901912700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f78fc05c850 0x7f78fc143a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:26.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.521+0000 7f7901912700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f78fc0b5d90 0x7f78fc0b4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:26.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.521+0000 7f7901912700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78fc05cc30 con 0x7f78fc0676a0 2022-06-29T15:49:26.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.521+0000 7f7901912700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78fc1430a0 con 0x7f78fc0b5d90 2022-06-29T15:49:26.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.521+0000 7f7901912700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc143300 con 0x7f78fc05c850 2022-06-29T15:49:26.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.521+0000 7f7901111700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f78fc0676a0 0x7f78fc140130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36242/0 (socket says 172.21.15.47:36242) 2022-06-29T15:49:26.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.521+0000 7f7901111700 1 -- 172.21.15.47:0/2494360731 learned_addr learned my addr 172.21.15.47:0/2494360731 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:49:26.536 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.525+0000 7f78f3fff700 1 --2- 172.21.15.47:0/2494360731 >> v2:172.21.15.64:3300/0 conn(0x7f78fc0b5d90 0x7f78fc0b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:26.536 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.525+0000 7f7900910700 1 --2- 172.21.15.47:0/2494360731 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f78fc05c850 0x7f78fc143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:26.536 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.525+0000 7f78f37fe700 1 -- 172.21.15.47:0/2494360731 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3695510666 0 0) 0x7f78fc05cc30 con 0x7f78fc0676a0 2022-06-29T15:49:26.537 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.525+0000 7f78f37fe700 1 -- 172.21.15.47:0/2494360731 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78e8003260 con 0x7f78fc0676a0 2022-06-29T15:49:26.537 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.525+0000 7f78f37fe700 1 -- 172.21.15.47:0/2494360731 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 628813631 0 0) 0x7f78e8003260 con 0x7f78fc0676a0 2022-06-29T15:49:26.537 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.525+0000 7f78f37fe700 1 -- 172.21.15.47:0/2494360731 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f78fc05c850 msgr2=0x7f78fc143a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:26.538 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.525+0000 7f78f37fe700 1 --2- 172.21.15.47:0/2494360731 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f78fc05c850 0x7f78fc143a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:26.538 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.525+0000 7f78f37fe700 1 -- 172.21.15.47:0/2494360731 >> v2:172.21.15.64:3300/0 conn(0x7f78fc0b5d90 msgr2=0x7f78fc0b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:26.539 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.525+0000 7f78f37fe700 1 --2- 172.21.15.47:0/2494360731 >> v2:172.21.15.64:3300/0 conn(0x7f78fc0b5d90 0x7f78fc0b4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:26.539 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.525+0000 7f78f37fe700 1 -- 172.21.15.47:0/2494360731 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78fc1435a0 con 0x7f78fc0676a0 2022-06-29T15:49:26.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.525+0000 7f78f37fe700 1 -- 172.21.15.47:0/2494360731 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7f78f4001a00 con 0x7f78fc0676a0 2022-06-29T15:49:26.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.525+0000 7f78f37fe700 1 -- 172.21.15.47:0/2494360731 >> v1:172.21.15.47:6789/0 conn(0x7f78fc0676a0 legacy=0x7f78fc140130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:26.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.525+0000 7f78f37fe700 1 --2- 172.21.15.47:0/2494360731 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f78fc05c850 0x7f78e8003750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:26.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.525+0000 7f78f37fe700 1 --2- 172.21.15.47:0/2494360731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78e8003c80 0x7f78e8004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:26.541 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.525+0000 7f78f37fe700 1 --2- 172.21.15.47:0/2494360731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78e80049f0 0x7f78e8004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:26.541 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.525+0000 7f78f37fe700 1 -- 172.21.15.47:0/2494360731 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc1430a0 con 0x7f78e8003c80 2022-06-29T15:49:26.541 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.525+0000 7f78f37fe700 1 -- 172.21.15.47:0/2494360731 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc143300 con 0x7f78e80049f0 2022-06-29T15:49:26.541 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.525+0000 7f78f37fe700 1 -- 172.21.15.47:0/2494360731 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78e800b750 con 0x7f78fc05c850 2022-06-29T15:49:26.542 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.525+0000 7f7900910700 1 --2- 172.21.15.47:0/2494360731 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f78fc05c850 0x7f78e8003750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:26.542 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.525+0000 7f78f3fff700 1 --2- 172.21.15.47:0/2494360731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78e8003c80 0x7f78e8004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:26.542 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.525+0000 7f7901111700 1 --2- 172.21.15.47:0/2494360731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78e80049f0 0x7f78e8004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:26.542 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.525+0000 7f78f3fff700 1 -- 172.21.15.47:0/2494360731 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f78fc05c850 msgr2=0x7f78e8003750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:26.543 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.525+0000 7f78f3fff700 1 --2- 172.21.15.47:0/2494360731 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f78fc05c850 0x7f78e8003750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:26.543 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.525+0000 7f78f3fff700 1 -- 172.21.15.47:0/2494360731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78e80049f0 msgr2=0x7f78e8004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:26.543 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.525+0000 7f78f3fff700 1 --2- 172.21.15.47:0/2494360731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78e80049f0 0x7f78e8004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:26.543 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.525+0000 7f78f3fff700 1 -- 172.21.15.47:0/2494360731 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e8003260 con 0x7f78e8003c80 2022-06-29T15:49:26.544 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.525+0000 7f78f3fff700 1 --2- 172.21.15.47:0/2494360731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78e8003c80 0x7f78e8004500 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 rx=0x7f78e80054d0 tx=0x7f78ec006df0).ready entity=mon.0 client_cookie=2f805d106abb0c4b server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:26.544 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.525+0000 7f78f37fe700 1 -- 172.21.15.47:0/2494360731 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f78ec00e830 con 0x7f78e8003c80 2022-06-29T15:49:26.544 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.525+0000 7f78f37fe700 1 -- 172.21.15.47:0/2494360731 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78ec00be90 con 0x7f78e8003c80 2022-06-29T15:49:26.544 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.525+0000 7f78f37fe700 1 -- 172.21.15.47:0/2494360731 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f78ec00eb80 con 0x7f78e8003c80 2022-06-29T15:49:26.545 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.525+0000 7f7901912700 1 -- 172.21.15.47:0/2494360731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78e8003c80 msgr2=0x7f78e8004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:26.545 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.525+0000 7f7901912700 1 --2- 172.21.15.47:0/2494360731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78e8003c80 0x7f78e8004500 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 rx=0x7f78e80054d0 tx=0x7f78ec006df0).stop 2022-06-29T15:49:26.545 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.525+0000 7f7901111700 1 -- 172.21.15.47:0/2494360731 reap_dead start 2022-06-29T15:49:26.545 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.525+0000 7f7901912700 1 -- 172.21.15.47:0/2494360731 shutdown_connections 2022-06-29T15:49:26.545 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.525+0000 7f7901912700 1 -- 172.21.15.47:0/2494360731 >> 172.21.15.47:0/2494360731 conn(0x7f78fc060e00 msgr2=0x7f78fc0611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:26.546 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.525+0000 7f7901912700 1 -- 172.21.15.47:0/2494360731 shutdown_connections 2022-06-29T15:49:26.546 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.525+0000 7f7901912700 1 -- 172.21.15.47:0/2494360731 wait complete. 2022-06-29T15:49:26.550 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.529+0000 7f7901912700 1 Processor -- start 2022-06-29T15:49:26.550 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.529+0000 7f7901912700 1 -- start start 2022-06-29T15:49:26.550 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.529+0000 7f7901912700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78e8003c80 0x7f78fc0b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:26.550 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.529+0000 7f7901912700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78fc1401d0 0x7f78fc12bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:26.551 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.529+0000 7f7901912700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f78fc0b7800 0x7f78fc152250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:26.551 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.529+0000 7f7901912700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc0ba2b0 con 0x7f78fc1401d0 2022-06-29T15:49:26.551 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.529+0000 7f7901912700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc156250 con 0x7f78e8003c80 2022-06-29T15:49:26.551 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.529+0000 7f7901912700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc0b7d00 con 0x7f78fc0b7800 2022-06-29T15:49:26.552 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.529+0000 7f78f3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78fc1401d0 0x7f78fc12bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:26.552 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.529+0000 7f78f3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78fc1401d0 0x7f78fc12bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50700/0 (socket says 172.21.15.47:50700) 2022-06-29T15:49:26.552 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.529+0000 7f78f3fff700 1 -- 172.21.15.47:0/514148595 learned_addr learned my addr 172.21.15.47:0/514148595 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:26.552 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.529+0000 7f7900910700 1 --2- 172.21.15.47:0/514148595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78e8003c80 0x7f78fc0b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:26.553 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.529+0000 7f7901111700 1 --2- 172.21.15.47:0/514148595 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f78fc0b7800 0x7f78fc152250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:26.553 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.529+0000 7f78f3fff700 1 -- 172.21.15.47:0/514148595 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f78fc0b7800 msgr2=0x7f78fc152250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:26.553 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.529+0000 7f78f3fff700 1 --2- 172.21.15.47:0/514148595 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f78fc0b7800 0x7f78fc152250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:26.553 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.529+0000 7f78f3fff700 1 -- 172.21.15.47:0/514148595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78e8003c80 msgr2=0x7f78fc0b32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:26.553 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.529+0000 7f78f3fff700 1 --2- 172.21.15.47:0/514148595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78e8003c80 0x7f78fc0b32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:26.554 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.529+0000 7f78f3fff700 1 -- 172.21.15.47:0/514148595 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78fc0ba4e0 con 0x7f78fc1401d0 2022-06-29T15:49:26.554 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.529+0000 7f78f3fff700 1 --2- 172.21.15.47:0/514148595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78fc1401d0 0x7f78fc12bea0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 rx=0x7f78e80054d0 tx=0x7f78ec019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:26.554 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.529+0000 7f78f1ffb700 1 -- 172.21.15.47:0/514148595 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f78ec01d8e0 con 0x7f78fc1401d0 2022-06-29T15:49:26.554 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.529+0000 7f78f1ffb700 1 -- 172.21.15.47:0/514148595 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78ec000e90 con 0x7f78fc1401d0 2022-06-29T15:49:26.555 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.529+0000 7f7901912700 1 -- 172.21.15.47:0/514148595 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78fc152740 con 0x7f78fc1401d0 2022-06-29T15:49:26.555 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.529+0000 7f7901912700 1 -- 172.21.15.47:0/514148595 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78fc0b1d00 con 0x7f78fc1401d0 2022-06-29T15:49:26.555 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.529+0000 7f78f1ffb700 1 -- 172.21.15.47:0/514148595 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f78ec01dc30 con 0x7f78fc1401d0 2022-06-29T15:49:26.556 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.533+0000 7f7901912700 1 -- 172.21.15.47:0/514148595 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78fc0573d0 con 0x7f78fc1401d0 2022-06-29T15:49:26.556 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.533+0000 7f78f1ffb700 1 -- 172.21.15.47:0/514148595 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f78ec02d070 con 0x7f78fc1401d0 2022-06-29T15:49:26.556 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.533+0000 7f78f1ffb700 1 --2- 172.21.15.47:0/514148595 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f78dc053950 0x7f78dc055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:26.556 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.533+0000 7f78f1ffb700 1 -- 172.21.15.47:0/514148595 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5954+0+0 (secure 0 0 0) 0x7f78ec06dda0 con 0x7f78fc1401d0 2022-06-29T15:49:26.556 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.533+0000 7f7900910700 1 --2- 172.21.15.47:0/514148595 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f78dc053950 0x7f78dc055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:26.557 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.533+0000 7f7900910700 1 --2- 172.21.15.47:0/514148595 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f78dc053950 0x7f78dc055e10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7f78f8006000 tx=0x7f78f8006d90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:26.557 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.541+0000 7f78f1ffb700 1 -- 172.21.15.47:0/514148595 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f78ec048760 con 0x7f78fc1401d0 2022-06-29T15:49:26.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:26.665+0000 7f7901912700 1 -- 172.21.15.47:0/514148595 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f78fc061b00 con 0x7f78fc1401d0 2022-06-29T15:49:27.409 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.401+0000 7f78f1ffb700 1 -- 172.21.15.47:0/514148595 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v88) v1 ==== 125+0+0 (secure 0 0 0) 0x7f78ec02b070 con 0x7f78fc1401d0 2022-06-29T15:49:27.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.405+0000 7f7901912700 1 -- 172.21.15.47:0/514148595 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f78dc053950 msgr2=0x7f78dc055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:27.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.405+0000 7f7901912700 1 --2- 172.21.15.47:0/514148595 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f78dc053950 0x7f78dc055e10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7f78f8006000 tx=0x7f78f8006d90).stop 2022-06-29T15:49:27.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.405+0000 7f7901912700 1 -- 172.21.15.47:0/514148595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78fc1401d0 msgr2=0x7f78fc12bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:27.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.405+0000 7f7901912700 1 --2- 172.21.15.47:0/514148595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78fc1401d0 0x7f78fc12bea0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 rx=0x7f78e80054d0 tx=0x7f78ec019000).stop 2022-06-29T15:49:27.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.409+0000 7f7901912700 1 -- 172.21.15.47:0/514148595 shutdown_connections 2022-06-29T15:49:27.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.409+0000 7f7901912700 1 --2- 172.21.15.47:0/514148595 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f78fc0b7800 0x7f78fc152250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:27.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.409+0000 7f7901912700 1 --2- 172.21.15.47:0/514148595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78fc1401d0 0x7f78fc12bea0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:27.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.409+0000 7f7901912700 1 --2- 172.21.15.47:0/514148595 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f78dc053950 0x7f78dc055e10 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:27.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.409+0000 7f7901912700 1 --2- 172.21.15.47:0/514148595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78e8003c80 0x7f78fc0b32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:27.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.409+0000 7f7901912700 1 -- 172.21.15.47:0/514148595 >> 172.21.15.47:0/514148595 conn(0x7f78fc060e00 msgr2=0x7f78fc0aff90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:27.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.409+0000 7f7901912700 1 -- 172.21.15.47:0/514148595 shutdown_connections 2022-06-29T15:49:27.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.409+0000 7f7901912700 1 -- 172.21.15.47:0/514148595 wait complete. 2022-06-29T15:49:27.417 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' created 2022-06-29T15:49:27.429 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-06-29T15:49:27.537 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.529+0000 7ff26890e700 1 Processor -- start 2022-06-29T15:49:27.538 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.529+0000 7ff26890e700 1 -- start start 2022-06-29T15:49:27.543 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.529+0000 7ff26890e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff2640b2bd0 0x7ff2640b2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:27.543 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.529+0000 7ff26890e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff26405f0b0 0x7ff264099c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:27.543 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.529+0000 7ff26890e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff264099800 con 0x7ff26409a120 2022-06-29T15:49:27.544 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.529+0000 7ff26890e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2640b1560 con 0x7ff2640b2bd0 2022-06-29T15:49:27.544 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.529+0000 7ff26890e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff264143c90 con 0x7ff26405f0b0 2022-06-29T15:49:27.544 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.529+0000 7ff262d9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff2640b2bd0 0x7ff2640b2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:27.545 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.533+0000 7ff262d9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff2640b2bd0 0x7ff2640b2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50706/0 (socket says 172.21.15.47:50706) 2022-06-29T15:49:27.545 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.533+0000 7ff262d9d700 1 -- 172.21.15.47:0/1076545781 learned_addr learned my addr 172.21.15.47:0/1076545781 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:27.545 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.533+0000 7ff26259c700 1 --2- 172.21.15.47:0/1076545781 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff26405f0b0 0x7ff264099c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:27.546 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.533+0000 7ff262d9d700 1 -- 172.21.15.47:0/1076545781 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff26405f0b0 msgr2=0x7ff264099c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:27.546 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.533+0000 7ff262d9d700 1 --2- 172.21.15.47:0/1076545781 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff26405f0b0 0x7ff264099c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:27.546 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.533+0000 7ff262d9d700 1 -- 172.21.15.47:0/1076545781 >> v1:172.21.15.64:6789/0 conn(0x7ff26409a120 legacy=0x7ff2641403b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:27.547 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.533+0000 7ff262d9d700 1 -- 172.21.15.47:0/1076545781 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff264143f30 con 0x7ff2640b2bd0 2022-06-29T15:49:27.547 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.533+0000 7ff262d9d700 1 --2- 172.21.15.47:0/1076545781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2640b2bd0 0x7ff2640b2fb0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 rx=0x7ff254002f70 tx=0x7ff254006eb0).ready entity=mon.0 client_cookie=4ec2c341ee388e77 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:27.547 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.533+0000 7ff261d9b700 1 -- 172.21.15.47:0/1076545781 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff25400d5c0 con 0x7ff2640b2bd0 2022-06-29T15:49:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.533+0000 7ff261d9b700 1 -- 172.21.15.47:0/1076545781 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff254000e90 con 0x7ff2640b2bd0 2022-06-29T15:49:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.533+0000 7ff261d9b700 1 -- 172.21.15.47:0/1076545781 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff25400d950 con 0x7ff2640b2bd0 2022-06-29T15:49:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.533+0000 7ff26890e700 1 -- 172.21.15.47:0/1076545781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2640b2bd0 msgr2=0x7ff2640b2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:27.549 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.533+0000 7ff26890e700 1 --2- 172.21.15.47:0/1076545781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2640b2bd0 0x7ff2640b2fb0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 rx=0x7ff254002f70 tx=0x7ff254006eb0).stop 2022-06-29T15:49:27.549 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.533+0000 7ff26890e700 1 -- 172.21.15.47:0/1076545781 shutdown_connections 2022-06-29T15:49:27.549 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.533+0000 7ff26890e700 1 --2- 172.21.15.47:0/1076545781 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff26405f0b0 0x7ff264099c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:27.550 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.533+0000 7ff26890e700 1 --2- 172.21.15.47:0/1076545781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2640b2bd0 0x7ff2640b2fb0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:27.550 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.533+0000 7ff26890e700 1 -- 172.21.15.47:0/1076545781 >> 172.21.15.47:0/1076545781 conn(0x7ff264062df0 msgr2=0x7ff2640631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:27.550 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.533+0000 7ff26890e700 1 -- 172.21.15.47:0/1076545781 shutdown_connections 2022-06-29T15:49:27.551 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.533+0000 7ff26890e700 1 -- 172.21.15.47:0/1076545781 wait complete. 2022-06-29T15:49:27.551 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.533+0000 7ff26890e700 1 Processor -- start 2022-06-29T15:49:27.551 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.533+0000 7ff26890e700 1 -- start start 2022-06-29T15:49:27.556 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.537+0000 7ff26890e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff26405f0b0 0x7ff264140600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:27.557 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.537+0000 7ff26890e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff26409a120 0x7ff2640acee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:27.557 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.537+0000 7ff26890e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff2640b2bd0 0x7ff2640ad3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:27.557 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.537+0000 7ff26890e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2640affd0 con 0x7ff26405f0b0 2022-06-29T15:49:27.558 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.537+0000 7ff26890e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2640ae7a0 con 0x7ff26409a120 2022-06-29T15:49:27.558 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.537+0000 7ff26890e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2640ad8c0 con 0x7ff2640b2bd0 2022-06-29T15:49:27.559 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.537+0000 7ff262d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff26405f0b0 0x7ff264140600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:27.559 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.537+0000 7ff262d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff26405f0b0 0x7ff264140600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50712/0 (socket says 172.21.15.47:50712) 2022-06-29T15:49:27.560 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.537+0000 7ff262d9d700 1 -- 172.21.15.47:0/1953696401 learned_addr learned my addr 172.21.15.47:0/1953696401 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:27.560 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.537+0000 7ff26259c700 1 --2- 172.21.15.47:0/1953696401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff26409a120 0x7ff2640acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:27.561 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.537+0000 7ff26359e700 1 --2- 172.21.15.47:0/1953696401 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff2640b2bd0 0x7ff2640ad3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:27.561 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.537+0000 7ff262d9d700 1 -- 172.21.15.47:0/1953696401 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff2640b2bd0 msgr2=0x7ff2640ad3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:27.561 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.537+0000 7ff262d9d700 1 --2- 172.21.15.47:0/1953696401 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff2640b2bd0 0x7ff2640ad3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:27.562 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.537+0000 7ff262d9d700 1 -- 172.21.15.47:0/1953696401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff26409a120 msgr2=0x7ff2640acee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:27.562 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.537+0000 7ff262d9d700 1 --2- 172.21.15.47:0/1953696401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff26409a120 0x7ff2640acee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:27.563 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.537+0000 7ff262d9d700 1 -- 172.21.15.47:0/1953696401 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff25400b1e0 con 0x7ff26405f0b0 2022-06-29T15:49:27.563 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.537+0000 7ff26259c700 1 --2- 172.21.15.47:0/1953696401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff26409a120 0x7ff2640acee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:49:27.564 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.537+0000 7ff262d9d700 1 --2- 172.21.15.47:0/1953696401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff26405f0b0 0x7ff264140600 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 rx=0x7ff254002f70 tx=0x7ff254015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:27.564 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.537+0000 7ff253fff700 1 -- 172.21.15.47:0/1953696401 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff254011950 con 0x7ff26405f0b0 2022-06-29T15:49:27.565 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.537+0000 7ff253fff700 1 -- 172.21.15.47:0/1953696401 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff254011ab0 con 0x7ff26405f0b0 2022-06-29T15:49:27.565 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.537+0000 7ff26890e700 1 -- 172.21.15.47:0/1953696401 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff26413eb90 con 0x7ff26405f0b0 2022-06-29T15:49:27.566 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.537+0000 7ff253fff700 1 -- 172.21.15.47:0/1953696401 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff254011e10 con 0x7ff26405f0b0 2022-06-29T15:49:27.566 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.537+0000 7ff26890e700 1 -- 172.21.15.47:0/1953696401 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff264140af0 con 0x7ff26405f0b0 2022-06-29T15:49:27.567 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.537+0000 7ff253fff700 1 -- 172.21.15.47:0/1953696401 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff254025030 con 0x7ff26405f0b0 2022-06-29T15:49:27.567 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.541+0000 7ff253fff700 1 --2- 172.21.15.47:0/1953696401 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff240053b00 0x7ff240055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:27.568 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.541+0000 7ff253fff700 1 -- 172.21.15.47:0/1953696401 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6304+0+0 (secure 0 0 0) 0x7ff25406d2f0 con 0x7ff26405f0b0 2022-06-29T15:49:27.568 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.541+0000 7ff26890e700 1 -- 172.21.15.47:0/1953696401 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2640573d0 con 0x7ff26405f0b0 2022-06-29T15:49:27.568 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.541+0000 7ff26259c700 1 --2- 172.21.15.47:0/1953696401 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff240053b00 0x7ff240055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:27.569 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.541+0000 7ff26259c700 1 --2- 172.21.15.47:0/1953696401 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff240053b00 0x7ff240055fc0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7ff25800c000 tx=0x7ff258008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:27.569 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.545+0000 7ff253fff700 1 -- 172.21.15.47:0/1953696401 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff254023020 con 0x7ff26405f0b0 2022-06-29T15:49:27.719 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.709+0000 7ff26890e700 1 -- 172.21.15.47:0/1953696401 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7ff264067b30 con 0x7ff26405f0b0 2022-06-29T15:49:27.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.721+0000 7ff253fff700 1 -- 172.21.15.47:0/1953696401 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 15 and data pool 16 v39) v1 ==== 189+0+0 (secure 0 0 0) 0x7ff254044130 con 0x7ff26405f0b0 2022-06-29T15:49:27.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.725+0000 7ff26890e700 1 -- 172.21.15.47:0/1953696401 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff240053b00 msgr2=0x7ff240055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:27.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.725+0000 7ff26890e700 1 --2- 172.21.15.47:0/1953696401 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff240053b00 0x7ff240055fc0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7ff25800c000 tx=0x7ff258008040).stop 2022-06-29T15:49:27.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.729+0000 7ff26890e700 1 -- 172.21.15.47:0/1953696401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff26405f0b0 msgr2=0x7ff264140600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:27.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.729+0000 7ff26890e700 1 --2- 172.21.15.47:0/1953696401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff26405f0b0 0x7ff264140600 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 rx=0x7ff254002f70 tx=0x7ff254015000).stop 2022-06-29T15:49:27.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.729+0000 7ff26890e700 1 -- 172.21.15.47:0/1953696401 shutdown_connections 2022-06-29T15:49:27.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.729+0000 7ff26890e700 1 --2- 172.21.15.47:0/1953696401 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff2640b2bd0 0x7ff2640ad3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:27.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.729+0000 7ff26890e700 1 --2- 172.21.15.47:0/1953696401 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff240053b00 0x7ff240055fc0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:27.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.729+0000 7ff26890e700 1 --2- 172.21.15.47:0/1953696401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff26409a120 0x7ff2640acee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:27.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.729+0000 7ff26890e700 1 --2- 172.21.15.47:0/1953696401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff26405f0b0 0x7ff264140600 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:27.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.729+0000 7ff26890e700 1 -- 172.21.15.47:0/1953696401 >> 172.21.15.47:0/1953696401 conn(0x7ff264062df0 msgr2=0x7ff264148280 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:27.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.729+0000 7ff26890e700 1 -- 172.21.15.47:0/1953696401 shutdown_connections 2022-06-29T15:49:27.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.729+0000 7ff26890e700 1 -- 172.21.15.47:0/1953696401 wait complete. 2022-06-29T15:49:27.739 INFO:teuthology.orchestra.run.smithi047.stderr:new fs with metadata pool 15 and data pool 16 2022-06-29T15:49:27.752 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:49:27.847 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.841+0000 7f5d7c8f3700 1 Processor -- start 2022-06-29T15:49:27.848 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.841+0000 7f5d7c8f3700 1 -- start start 2022-06-29T15:49:27.855 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.841+0000 7f5d7c8f3700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5d78060b90 0x7f5d780ae170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:27.855 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.841+0000 7f5d7c8f3700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d7805fe10 con 0x7f5d7805fa30 2022-06-29T15:49:27.856 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.841+0000 7f5d7c8f3700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d780600e0 con 0x7f5d780b1ab0 2022-06-29T15:49:27.856 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.841+0000 7f5d7c8f3700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d780b1ef0 con 0x7f5d78060b90 2022-06-29T15:49:27.857 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.841+0000 7f5d777fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f5d7805fa30 0x7f5d7813f470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36284/0 (socket says 172.21.15.47:36284) 2022-06-29T15:49:27.857 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.841+0000 7f5d777fe700 1 -- 172.21.15.47:0/338228401 learned_addr learned my addr 172.21.15.47:0/338228401 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:49:27.858 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.841+0000 7f5d77fff700 1 --2- 172.21.15.47:0/338228401 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5d78060b90 0x7f5d780ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:27.858 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.841+0000 7f5d767fc700 1 -- 172.21.15.47:0/338228401 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2413492567 0 0) 0x7f5d7805fe10 con 0x7f5d7805fa30 2022-06-29T15:49:27.859 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.841+0000 7f5d767fc700 1 -- 172.21.15.47:0/338228401 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d64003260 con 0x7f5d7805fa30 2022-06-29T15:49:27.859 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.841+0000 7f5d767fc700 1 -- 172.21.15.47:0/338228401 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2205422462 0 0) 0x7f5d780600e0 con 0x7f5d780b1ab0 2022-06-29T15:49:27.859 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.841+0000 7f5d767fc700 1 -- 172.21.15.47:0/338228401 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d7805fe10 con 0x7f5d780b1ab0 2022-06-29T15:49:27.860 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.841+0000 7f5d767fc700 1 -- 172.21.15.47:0/338228401 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 511238210 0 0) 0x7f5d64003260 con 0x7f5d7805fa30 2022-06-29T15:49:27.860 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.841+0000 7f5d77fff700 1 -- 172.21.15.47:0/338228401 >> v1:172.21.15.64:6789/0 conn(0x7f5d780b1ab0 legacy=0x7f5d780b2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:27.861 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.841+0000 7f5d77fff700 1 -- 172.21.15.47:0/338228401 >> v1:172.21.15.47:6789/0 conn(0x7f5d7805fa30 legacy=0x7f5d7813f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:27.861 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.841+0000 7f5d77fff700 1 -- 172.21.15.47:0/338228401 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d78060380 con 0x7f5d78060b90 2022-06-29T15:49:27.862 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.841+0000 7f5d77fff700 1 --2- 172.21.15.47:0/338228401 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5d78060b90 0x7f5d780ae170 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f5d68001110 tx=0x7f5d7000a370).ready entity=mon.2 client_cookie=da0b7f2d696a7bcd server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:27.862 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.845+0000 7f5d767fc700 1 -- 172.21.15.47:0/338228401 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d68001660 con 0x7f5d78060b90 2022-06-29T15:49:27.863 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.845+0000 7f5d767fc700 1 -- 172.21.15.47:0/338228401 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d7000c870 con 0x7f5d78060b90 2022-06-29T15:49:27.863 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.845+0000 7f5d767fc700 1 -- 172.21.15.47:0/338228401 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d7000cbd0 con 0x7f5d78060b90 2022-06-29T15:49:27.864 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.845+0000 7f5d7c8f3700 1 -- 172.21.15.47:0/338228401 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5d78060b90 msgr2=0x7f5d780ae170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:27.864 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.845+0000 7f5d7c8f3700 1 --2- 172.21.15.47:0/338228401 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5d78060b90 0x7f5d780ae170 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f5d68001110 tx=0x7f5d7000a370).stop 2022-06-29T15:49:27.864 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.845+0000 7f5d7c8f3700 1 -- 172.21.15.47:0/338228401 shutdown_connections 2022-06-29T15:49:27.865 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.845+0000 7f5d7c8f3700 1 --2- 172.21.15.47:0/338228401 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5d78060b90 0x7f5d780ae170 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:27.865 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.845+0000 7f5d7c8f3700 1 -- 172.21.15.47:0/338228401 >> 172.21.15.47:0/338228401 conn(0x7f5d78062df0 msgr2=0x7f5d780631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:27.866 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.845+0000 7f5d7c8f3700 1 -- 172.21.15.47:0/338228401 shutdown_connections 2022-06-29T15:49:27.866 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.845+0000 7f5d7c8f3700 1 -- 172.21.15.47:0/338228401 wait complete. 2022-06-29T15:49:27.867 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.845+0000 7f5d7c8f3700 1 Processor -- start 2022-06-29T15:49:27.867 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.845+0000 7f5d7c8f3700 1 -- start start 2022-06-29T15:49:27.873 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.849+0000 7f5d7c8f3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d7805fa30 0x7f5d780b0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:27.873 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.849+0000 7f5d7c8f3700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5d78060b90 0x7f5d780b1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:27.874 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.849+0000 7f5d7c8f3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d780b1ab0 0x7f5d780ae510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:27.874 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.849+0000 7f5d7c8f3700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d781430a0 con 0x7f5d7805fa30 2022-06-29T15:49:27.874 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.849+0000 7f5d7c8f3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d781562a0 con 0x7f5d780b1ab0 2022-06-29T15:49:27.875 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.849+0000 7f5d7c8f3700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d78156520 con 0x7f5d78060b90 2022-06-29T15:49:27.875 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.849+0000 7f5d777fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d7805fa30 0x7f5d780b0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:27.875 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.849+0000 7f5d777fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d7805fa30 0x7f5d780b0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50736/0 (socket says 172.21.15.47:50736) 2022-06-29T15:49:27.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.849+0000 7f5d777fe700 1 -- 172.21.15.47:0/2914269912 learned_addr learned my addr 172.21.15.47:0/2914269912 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:27.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.849+0000 7f5d77fff700 1 --2- 172.21.15.47:0/2914269912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d780b1ab0 0x7f5d780ae510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:27.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.849+0000 7f5d76ffd700 1 --2- 172.21.15.47:0/2914269912 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5d78060b90 0x7f5d780b1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:27.877 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.849+0000 7f5d777fe700 1 -- 172.21.15.47:0/2914269912 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5d78060b90 msgr2=0x7f5d780b1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:27.877 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.849+0000 7f5d777fe700 1 --2- 172.21.15.47:0/2914269912 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5d78060b90 0x7f5d780b1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:27.877 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.849+0000 7f5d777fe700 1 -- 172.21.15.47:0/2914269912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d780b1ab0 msgr2=0x7f5d780ae510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:27.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.849+0000 7f5d777fe700 1 --2- 172.21.15.47:0/2914269912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d780b1ab0 0x7f5d780ae510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:27.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.849+0000 7f5d777fe700 1 -- 172.21.15.47:0/2914269912 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d7000a610 con 0x7f5d7805fa30 2022-06-29T15:49:27.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.849+0000 7f5d777fe700 1 --2- 172.21.15.47:0/2914269912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d7805fa30 0x7f5d780b0b80 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 rx=0x7f5d68001840 tx=0x7f5d68001720).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:27.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.849+0000 7f5d74ff9700 1 -- 172.21.15.47:0/2914269912 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d68007170 con 0x7f5d7805fa30 2022-06-29T15:49:27.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.849+0000 7f5d74ff9700 1 -- 172.21.15.47:0/2914269912 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d680072d0 con 0x7f5d7805fa30 2022-06-29T15:49:27.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.849+0000 7f5d7c8f3700 1 -- 172.21.15.47:0/2914269912 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d781567c0 con 0x7f5d7805fa30 2022-06-29T15:49:27.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.849+0000 7f5d74ff9700 1 -- 172.21.15.47:0/2914269912 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d68004710 con 0x7f5d7805fa30 2022-06-29T15:49:27.880 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.849+0000 7f5d7c8f3700 1 -- 172.21.15.47:0/2914269912 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d780ac090 con 0x7f5d7805fa30 2022-06-29T15:49:27.880 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.853+0000 7f5d7c8f3700 1 -- 172.21.15.47:0/2914269912 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d78143670 con 0x7f5d7805fa30 2022-06-29T15:49:27.880 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.853+0000 7f5d74ff9700 1 -- 172.21.15.47:0/2914269912 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5d6801b070 con 0x7f5d7805fa30 2022-06-29T15:49:27.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.853+0000 7f5d74ff9700 1 --2- 172.21.15.47:0/2914269912 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5d58053980 0x7f5d58055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:27.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.853+0000 7f5d74ff9700 1 -- 172.21.15.47:0/2914269912 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f5d6806b6d0 con 0x7f5d7805fa30 2022-06-29T15:49:27.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.853+0000 7f5d76ffd700 1 --2- 172.21.15.47:0/2914269912 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5d58053980 0x7f5d58055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:27.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.853+0000 7f5d76ffd700 1 --2- 172.21.15.47:0/2914269912 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5d58053980 0x7f5d58055e40 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7f5d6c00d000 tx=0x7f5d6c007a20).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:27.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.861+0000 7f5d74ff9700 1 -- 172.21.15.47:0/2914269912 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5d68042380 con 0x7f5d7805fa30 2022-06-29T15:49:27.996 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.989+0000 7f5d7c8f3700 1 -- 172.21.15.47:0/2914269912 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d780573d0 con 0x7f5d7805fa30 2022-06-29T15:49:27.998 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.989+0000 7f5d74ff9700 1 -- 172.21.15.47:0/2914269912 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+16287 (secure 0 0 0) 0x7f5d680459a0 con 0x7f5d7805fa30 2022-06-29T15:49:27.999 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:49:27.999 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":89,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:49:27.716889+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:49:25.785974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2022-06-29T15:49:26.726660+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:49:28.001 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.993+0000 7f5d7c8f3700 1 -- 172.21.15.47:0/2914269912 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5d58053980 msgr2=0x7f5d58055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:28.001 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.993+0000 7f5d7c8f3700 1 --2- 172.21.15.47:0/2914269912 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5d58053980 0x7f5d58055e40 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7f5d6c00d000 tx=0x7f5d6c007a20).stop 2022-06-29T15:49:28.001 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.993+0000 7f5d7c8f3700 1 -- 172.21.15.47:0/2914269912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d7805fa30 msgr2=0x7f5d780b0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:28.001 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.993+0000 7f5d7c8f3700 1 --2- 172.21.15.47:0/2914269912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d7805fa30 0x7f5d780b0b80 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 rx=0x7f5d68001840 tx=0x7f5d68001720).stop 2022-06-29T15:49:28.002 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.993+0000 7f5d7c8f3700 1 -- 172.21.15.47:0/2914269912 shutdown_connections 2022-06-29T15:49:28.002 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.993+0000 7f5d7c8f3700 1 --2- 172.21.15.47:0/2914269912 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5d58053980 0x7f5d58055e40 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:28.002 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.993+0000 7f5d7c8f3700 1 --2- 172.21.15.47:0/2914269912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d780b1ab0 0x7f5d780ae510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:28.002 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.993+0000 7f5d7c8f3700 1 --2- 172.21.15.47:0/2914269912 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5d78060b90 0x7f5d780b1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:28.003 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.993+0000 7f5d7c8f3700 1 --2- 172.21.15.47:0/2914269912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d7805fa30 0x7f5d780b0b80 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:28.003 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.993+0000 7f5d7c8f3700 1 -- 172.21.15.47:0/2914269912 >> 172.21.15.47:0/2914269912 conn(0x7f5d78062df0 msgr2=0x7f5d7805c470 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:28.003 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.993+0000 7f5d7c8f3700 1 -- 172.21.15.47:0/2914269912 shutdown_connections 2022-06-29T15:49:28.003 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:27.993+0000 7f5d7c8f3700 1 -- 172.21.15.47:0/2914269912 wait complete. 2022-06-29T15:49:28.012 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:49:28.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.113+0000 7f4010ac8700 1 Processor -- start 2022-06-29T15:49:28.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.113+0000 7f4010ac8700 1 -- start start 2022-06-29T15:49:28.123 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.113+0000 7f4010ac8700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f400c0b5c60 0x7f400c144980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:28.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.113+0000 7f4010ac8700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f400c0a9120 con 0x7f400c060eb0 2022-06-29T15:49:28.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.113+0000 7f4010ac8700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f400c0b0b80 con 0x7f400c0a72f0 2022-06-29T15:49:28.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.113+0000 7f4010ac8700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f400c0a89e0 con 0x7f400c0b5c60 2022-06-29T15:49:28.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.113+0000 7f400affd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f400c060eb0 0x7f400c0b1e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36300/0 (socket says 172.21.15.47:36300) 2022-06-29T15:49:28.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.113+0000 7f400affd700 1 -- 172.21.15.47:0/1895578372 learned_addr learned my addr 172.21.15.47:0/1895578372 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:49:28.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.113+0000 7f400b7fe700 1 --2- 172.21.15.47:0/1895578372 >> v2:172.21.15.64:3300/0 conn(0x7f400c0b5c60 0x7f400c144980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:28.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.113+0000 7f400a7fc700 1 -- 172.21.15.47:0/1895578372 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 923433247 0 0) 0x7f400c0a9120 con 0x7f400c060eb0 2022-06-29T15:49:28.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.113+0000 7f400a7fc700 1 -- 172.21.15.47:0/1895578372 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ff0003260 con 0x7f400c060eb0 2022-06-29T15:49:28.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.113+0000 7f400b7fe700 1 -- 172.21.15.47:0/1895578372 >> v1:172.21.15.64:6789/0 conn(0x7f400c0a72f0 legacy=0x7f400c151eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:28.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.113+0000 7f400b7fe700 1 -- 172.21.15.47:0/1895578372 >> v1:172.21.15.47:6789/0 conn(0x7f400c060eb0 legacy=0x7f400c0b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:28.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.113+0000 7f400b7fe700 1 -- 172.21.15.47:0/1895578372 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f400c0a8c80 con 0x7f400c0b5c60 2022-06-29T15:49:28.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.113+0000 7f400b7fe700 1 --2- 172.21.15.47:0/1895578372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f400c0b5c60 0x7f400c144980 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7f3ff4002f70 tx=0x7f3ff4006d90).ready entity=mon.1 client_cookie=f9738159b6984407 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:28.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.113+0000 7f400a7fc700 1 -- 172.21.15.47:0/1895578372 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ff400d4a0 con 0x7f400c0b5c60 2022-06-29T15:49:28.127 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.113+0000 7f400a7fc700 1 -- 172.21.15.47:0/1895578372 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ff4000e90 con 0x7f400c0b5c60 2022-06-29T15:49:28.127 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.113+0000 7f400a7fc700 1 -- 172.21.15.47:0/1895578372 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ff400d7f0 con 0x7f400c0b5c60 2022-06-29T15:49:28.127 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.113+0000 7f4010ac8700 1 -- 172.21.15.47:0/1895578372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f400c0b5c60 msgr2=0x7f400c144980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:28.127 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.113+0000 7f4010ac8700 1 --2- 172.21.15.47:0/1895578372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f400c0b5c60 0x7f400c144980 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7f3ff4002f70 tx=0x7f3ff4006d90).stop 2022-06-29T15:49:28.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.113+0000 7f4010ac8700 1 -- 172.21.15.47:0/1895578372 shutdown_connections 2022-06-29T15:49:28.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.113+0000 7f4010ac8700 1 --2- 172.21.15.47:0/1895578372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f400c0b5c60 0x7f400c144980 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:28.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.113+0000 7f4010ac8700 1 -- 172.21.15.47:0/1895578372 >> 172.21.15.47:0/1895578372 conn(0x7f400c062560 msgr2=0x7f400c062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:28.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.113+0000 7f4010ac8700 1 -- 172.21.15.47:0/1895578372 shutdown_connections 2022-06-29T15:49:28.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.113+0000 7f4010ac8700 1 -- 172.21.15.47:0/1895578372 wait complete. 2022-06-29T15:49:28.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.117+0000 7f4010ac8700 1 Processor -- start 2022-06-29T15:49:28.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.117+0000 7f4010ac8700 1 -- start start 2022-06-29T15:49:28.132 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.117+0000 7f4010ac8700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f400c060eb0 0x7f400c13eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:28.132 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.117+0000 7f4010ac8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f400c0a72f0 0x7f400c0aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:28.132 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.117+0000 7f4010ac8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f400c0b5c60 0x7f400c148950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:28.132 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.117+0000 7f4010ac8700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400c0af440 con 0x7f400c0b5c60 2022-06-29T15:49:28.133 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.117+0000 7f4010ac8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400c14b6d0 con 0x7f400c0a72f0 2022-06-29T15:49:28.133 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.117+0000 7f4010ac8700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f400c13f6f0 con 0x7f400c060eb0 2022-06-29T15:49:28.133 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.117+0000 7f400bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f400c0b5c60 0x7f400c148950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:28.133 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.117+0000 7f400bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f400c0b5c60 0x7f400c148950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50754/0 (socket says 172.21.15.47:50754) 2022-06-29T15:49:28.133 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.117+0000 7f400bfff700 1 -- 172.21.15.47:0/4188442470 learned_addr learned my addr 172.21.15.47:0/4188442470 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:28.134 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.117+0000 7f400b7fe700 1 --2- 172.21.15.47:0/4188442470 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f400c060eb0 0x7f400c13eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:28.134 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.117+0000 7f400affd700 1 --2- 172.21.15.47:0/4188442470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f400c0a72f0 0x7f400c0aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:28.134 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.117+0000 7f400bfff700 1 -- 172.21.15.47:0/4188442470 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f400c060eb0 msgr2=0x7f400c13eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:28.134 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.117+0000 7f400bfff700 1 --2- 172.21.15.47:0/4188442470 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f400c060eb0 0x7f400c13eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:28.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.117+0000 7f400bfff700 1 -- 172.21.15.47:0/4188442470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f400c0a72f0 msgr2=0x7f400c0aeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:28.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.117+0000 7f400bfff700 1 --2- 172.21.15.47:0/4188442470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f400c0a72f0 0x7f400c0aeaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:28.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.117+0000 7f400bfff700 1 -- 172.21.15.47:0/4188442470 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff400b120 con 0x7f400c0b5c60 2022-06-29T15:49:28.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.117+0000 7f400affd700 1 --2- 172.21.15.47:0/4188442470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f400c0a72f0 0x7f400c0aeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:49:28.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.117+0000 7f400b7fe700 1 --2- 172.21.15.47:0/4188442470 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f400c060eb0 0x7f400c13eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:49:28.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.117+0000 7f400bfff700 1 --2- 172.21.15.47:0/4188442470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f400c0b5c60 0x7f400c148950 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 rx=0x7f400c06ac40 tx=0x7f3ffc007bc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:28.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.117+0000 7f4008ff9700 1 -- 172.21.15.47:0/4188442470 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ffc00ce30 con 0x7f400c0b5c60 2022-06-29T15:49:28.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.117+0000 7f4008ff9700 1 -- 172.21.15.47:0/4188442470 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ffc00f470 con 0x7f400c0b5c60 2022-06-29T15:49:28.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.117+0000 7f4010ac8700 1 -- 172.21.15.47:0/4188442470 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f400c0aded0 con 0x7f400c0b5c60 2022-06-29T15:49:28.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.117+0000 7f4008ff9700 1 -- 172.21.15.47:0/4188442470 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ffc00f7d0 con 0x7f400c0b5c60 2022-06-29T15:49:28.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.117+0000 7f4010ac8700 1 -- 172.21.15.47:0/4188442470 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f400c148fe0 con 0x7f400c0b5c60 2022-06-29T15:49:28.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.117+0000 7f4010ac8700 1 -- 172.21.15.47:0/4188442470 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f400c0ae850 con 0x7f400c0b5c60 2022-06-29T15:49:28.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.121+0000 7f4008ff9700 1 -- 172.21.15.47:0/4188442470 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3ffc019030 con 0x7f400c0b5c60 2022-06-29T15:49:28.138 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.121+0000 7f4008ff9700 1 --2- 172.21.15.47:0/4188442470 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3fe40539e0 0x7f3fe4055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:28.138 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.121+0000 7f4008ff9700 1 -- 172.21.15.47:0/4188442470 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f3ffc06b430 con 0x7f400c0b5c60 2022-06-29T15:49:28.138 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.121+0000 7f400b7fe700 1 --2- 172.21.15.47:0/4188442470 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3fe40539e0 0x7f3fe4055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:28.138 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.121+0000 7f400b7fe700 1 --2- 172.21.15.47:0/4188442470 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3fe40539e0 0x7f3fe4055ea0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7f3ff4018000 tx=0x7f3ff4014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:28.139 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.125+0000 7f4008ff9700 1 -- 172.21.15.47:0/4188442470 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f400c0ae850 con 0x7f400c0b5c60 2022-06-29T15:49:28.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.253+0000 7f4010ac8700 1 -- 172.21.15.47:0/4188442470 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f400c0ae850 con 0x7f400c0b5c60 2022-06-29T15:49:28.260 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.253+0000 7f4008ff9700 1 -- 172.21.15.47:0/4188442470 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+16287 (secure 0 0 0) 0x7f400c0ae850 con 0x7f400c0b5c60 2022-06-29T15:49:28.261 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:49:28.262 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":89,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:49:27.716889+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:49:25.785974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2022-06-29T15:49:26.726660+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:49:28.265 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.257+0000 7f4010ac8700 1 -- 172.21.15.47:0/4188442470 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3fe40539e0 msgr2=0x7f3fe4055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:28.266 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.257+0000 7f4010ac8700 1 --2- 172.21.15.47:0/4188442470 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3fe40539e0 0x7f3fe4055ea0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7f3ff4018000 tx=0x7f3ff4014000).stop 2022-06-29T15:49:28.266 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.257+0000 7f4010ac8700 1 -- 172.21.15.47:0/4188442470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f400c0b5c60 msgr2=0x7f400c148950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:28.267 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.257+0000 7f4010ac8700 1 --2- 172.21.15.47:0/4188442470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f400c0b5c60 0x7f400c148950 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 rx=0x7f400c06ac40 tx=0x7f3ffc007bc0).stop 2022-06-29T15:49:28.267 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.257+0000 7f4010ac8700 1 -- 172.21.15.47:0/4188442470 shutdown_connections 2022-06-29T15:49:28.267 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.257+0000 7f4010ac8700 1 --2- 172.21.15.47:0/4188442470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f400c0b5c60 0x7f400c148950 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:28.268 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.257+0000 7f4010ac8700 1 --2- 172.21.15.47:0/4188442470 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3fe40539e0 0x7f3fe4055ea0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:28.268 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.257+0000 7f4010ac8700 1 --2- 172.21.15.47:0/4188442470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f400c0a72f0 0x7f400c0aeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:28.268 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.257+0000 7f4010ac8700 1 --2- 172.21.15.47:0/4188442470 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f400c060eb0 0x7f400c13eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:28.268 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.257+0000 7f4010ac8700 1 -- 172.21.15.47:0/4188442470 >> 172.21.15.47:0/4188442470 conn(0x7f400c062560 msgr2=0x7f400c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:28.269 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.257+0000 7f4010ac8700 1 -- 172.21.15.47:0/4188442470 shutdown_connections 2022-06-29T15:49:28.269 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.257+0000 7f4010ac8700 1 -- 172.21.15.47:0/4188442470 wait complete. 2022-06-29T15:49:28.275 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-06-29T15:49:28.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.377+0000 7f97f4c60700 1 Processor -- start 2022-06-29T15:49:28.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.377+0000 7f97f4c60700 1 -- start start 2022-06-29T15:49:28.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.377+0000 7f97f4c60700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f97f005f3f0 0x7f97f01445b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:28.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.377+0000 7f97f4c60700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f97f005fc50 0x7f97f0144aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:28.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.377+0000 7f97f4c60700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97f00b02c0 con 0x7f97f00b17a0 2022-06-29T15:49:28.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.377+0000 7f97f4c60700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97f005f7d0 con 0x7f97f005f3f0 2022-06-29T15:49:28.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.377+0000 7f97f4c60700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f01481e0 con 0x7f97f005fc50 2022-06-29T15:49:28.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.381+0000 7f97eeffd700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f97f005fc50 0x7f97f0144aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:28.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.381+0000 7f97ef7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f97f005f3f0 0x7f97f01445b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:28.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.381+0000 7f97eeffd700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f97f005fc50 0x7f97f0144aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:47864/0 (socket says 172.21.15.47:47864) 2022-06-29T15:49:28.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.381+0000 7f97eeffd700 1 -- 172.21.15.47:0/3161079541 learned_addr learned my addr 172.21.15.47:0/3161079541 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:28.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.381+0000 7f97effff700 1 --1- 172.21.15.47:0/3161079541 >> v1:172.21.15.64:6789/0 conn(0x7f97f00b17a0 0x7f97f00a7320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:43878/0 (socket says 172.21.15.47:43878) 2022-06-29T15:49:28.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.381+0000 7f97eeffd700 1 -- 172.21.15.47:0/3161079541 >> v2:172.21.15.64:3300/0 conn(0x7f97f005f3f0 msgr2=0x7f97f01445b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:28.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.381+0000 7f97eeffd700 1 --2- 172.21.15.47:0/3161079541 >> v2:172.21.15.64:3300/0 conn(0x7f97f005f3f0 0x7f97f01445b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:28.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.381+0000 7f97ee7fc700 1 -- 172.21.15.47:0/3161079541 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2972878655 0 0) 0x7f97f00b02c0 con 0x7f97f00b17a0 2022-06-29T15:49:28.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.381+0000 7f97eeffd700 1 -- 172.21.15.47:0/3161079541 >> v1:172.21.15.64:6789/0 conn(0x7f97f00b17a0 legacy=0x7f97f00a7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:28.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.381+0000 7f97eeffd700 1 -- 172.21.15.47:0/3161079541 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f00b04f0 con 0x7f97f005fc50 2022-06-29T15:49:28.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.381+0000 7f97ef7fe700 1 --2- 172.21.15.47:0/3161079541 >> v2:172.21.15.64:3300/0 conn(0x7f97f005f3f0 0x7f97f01445b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:49:28.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.381+0000 7f97eeffd700 1 --2- 172.21.15.47:0/3161079541 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f97f005fc50 0x7f97f0144aa0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f97e4002f70 tx=0x7f97e4006eb0).ready entity=mon.2 client_cookie=fb5033d1afb744a0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:28.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.381+0000 7f97ee7fc700 1 -- 172.21.15.47:0/3161079541 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97e400d570 con 0x7f97f005fc50 2022-06-29T15:49:28.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.381+0000 7f97ee7fc700 1 -- 172.21.15.47:0/3161079541 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e4000e90 con 0x7f97f005fc50 2022-06-29T15:49:28.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.381+0000 7f97ee7fc700 1 -- 172.21.15.47:0/3161079541 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97e400d8c0 con 0x7f97f005fc50 2022-06-29T15:49:28.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.381+0000 7f97f4c60700 1 -- 172.21.15.47:0/3161079541 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f97f005fc50 msgr2=0x7f97f0144aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:28.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.381+0000 7f97f4c60700 1 --2- 172.21.15.47:0/3161079541 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f97f005fc50 0x7f97f0144aa0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f97e4002f70 tx=0x7f97e4006eb0).stop 2022-06-29T15:49:28.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.381+0000 7f97f4c60700 1 -- 172.21.15.47:0/3161079541 shutdown_connections 2022-06-29T15:49:28.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.381+0000 7f97f4c60700 1 --2- 172.21.15.47:0/3161079541 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f97f005fc50 0x7f97f0144aa0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:28.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.381+0000 7f97f4c60700 1 --2- 172.21.15.47:0/3161079541 >> v2:172.21.15.64:3300/0 conn(0x7f97f005f3f0 0x7f97f01445b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:28.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.381+0000 7f97f4c60700 1 -- 172.21.15.47:0/3161079541 >> 172.21.15.47:0/3161079541 conn(0x7f97f00b5320 msgr2=0x7f97f00b5700 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:28.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.381+0000 7f97f4c60700 1 -- 172.21.15.47:0/3161079541 shutdown_connections 2022-06-29T15:49:28.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.381+0000 7f97f4c60700 1 -- 172.21.15.47:0/3161079541 wait complete. 2022-06-29T15:49:28.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.385+0000 7f97f4c60700 1 Processor -- start 2022-06-29T15:49:28.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.385+0000 7f97f4c60700 1 -- start start 2022-06-29T15:49:28.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.385+0000 7f97f4c60700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f97f005f3f0 0x7f97f0140240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:28.409 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.385+0000 7f97f4c60700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97f005fc50 0x7f97f01487b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:28.409 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.385+0000 7f97f4c60700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97f00b17a0 0x7f97f0148ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:28.409 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.385+0000 7f97f4c60700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f013e660 con 0x7f97f005fc50 2022-06-29T15:49:28.410 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.385+0000 7f97f4c60700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f013fa10 con 0x7f97f00b17a0 2022-06-29T15:49:28.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.385+0000 7f97f4c60700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f013fcb0 con 0x7f97f005f3f0 2022-06-29T15:49:28.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.385+0000 7f97eeffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97f005fc50 0x7f97f01487b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:28.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.385+0000 7f97effff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97f00b17a0 0x7f97f0148ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:28.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.385+0000 7f97eeffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97f005fc50 0x7f97f01487b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50768/0 (socket says 172.21.15.47:50768) 2022-06-29T15:49:28.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.385+0000 7f97effff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97f00b17a0 0x7f97f0148ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:56470/0 (socket says 172.21.15.47:56470) 2022-06-29T15:49:28.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.385+0000 7f97eeffd700 1 -- 172.21.15.47:0/1610127848 learned_addr learned my addr 172.21.15.47:0/1610127848 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:28.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.385+0000 7f97ef7fe700 1 --2- 172.21.15.47:0/1610127848 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f97f005f3f0 0x7f97f0140240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:28.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.385+0000 7f97eeffd700 1 -- 172.21.15.47:0/1610127848 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f97f005f3f0 msgr2=0x7f97f0140240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:28.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.385+0000 7f97eeffd700 1 --2- 172.21.15.47:0/1610127848 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f97f005f3f0 0x7f97f0140240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:28.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.385+0000 7f97eeffd700 1 -- 172.21.15.47:0/1610127848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97f00b17a0 msgr2=0x7f97f0148ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:28.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.385+0000 7f97eeffd700 1 --2- 172.21.15.47:0/1610127848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97f00b17a0 0x7f97f0148ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:28.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.385+0000 7f97eeffd700 1 -- 172.21.15.47:0/1610127848 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e000b180 con 0x7f97f005fc50 2022-06-29T15:49:28.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.385+0000 7f97effff700 1 --2- 172.21.15.47:0/1610127848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97f00b17a0 0x7f97f0148ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:49:28.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.385+0000 7f97eeffd700 1 --2- 172.21.15.47:0/1610127848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97f005fc50 0x7f97f01487b0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 rx=0x7f97e4002f70 tx=0x7f97e4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:28.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.385+0000 7f97ecff9700 1 -- 172.21.15.47:0/1610127848 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97e4011760 con 0x7f97f005fc50 2022-06-29T15:49:28.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.385+0000 7f97ecff9700 1 -- 172.21.15.47:0/1610127848 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e40118c0 con 0x7f97f005fc50 2022-06-29T15:49:28.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.385+0000 7f97ecff9700 1 -- 172.21.15.47:0/1610127848 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97e4011c20 con 0x7f97f005fc50 2022-06-29T15:49:28.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.385+0000 7f97f4c60700 1 -- 172.21.15.47:0/1610127848 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97e400b1e0 con 0x7f97f005fc50 2022-06-29T15:49:28.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.385+0000 7f97f4c60700 1 -- 172.21.15.47:0/1610127848 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97f014b290 con 0x7f97f005fc50 2022-06-29T15:49:28.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.389+0000 7f97f4c60700 1 -- 172.21.15.47:0/1610127848 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97f00573d0 con 0x7f97f005fc50 2022-06-29T15:49:28.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.389+0000 7f97ecff9700 1 -- 172.21.15.47:0/1610127848 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f97e401a040 con 0x7f97f005fc50 2022-06-29T15:49:28.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.389+0000 7f97ecff9700 1 --2- 172.21.15.47:0/1610127848 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f97d00539e0 0x7f97d0055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:28.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.389+0000 7f97ecff9700 1 -- 172.21.15.47:0/1610127848 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f97e406cf20 con 0x7f97f005fc50 2022-06-29T15:49:28.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.389+0000 7f97ef7fe700 1 --2- 172.21.15.47:0/1610127848 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f97d00539e0 0x7f97d0055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:28.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.389+0000 7f97ef7fe700 1 --2- 172.21.15.47:0/1610127848 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f97d00539e0 0x7f97d0055ea0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7f97e000b000 tx=0x7f97e0006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:28.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.397+0000 7f97ecff9700 1 -- 172.21.15.47:0/1610127848 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f97e4043c00 con 0x7f97f005fc50 2022-06-29T15:49:28.563 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.557+0000 7f97f4c60700 1 -- 172.21.15.47:0/1610127848 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f97f013e370 con 0x7f97f005fc50 2022-06-29T15:49:28.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.721+0000 7f97ecff9700 1 -- 172.21.15.47:0/1610127848 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v40) v1 ==== 118+0+0 (secure 0 0 0) 0x7f97e4023070 con 0x7f97f005fc50 2022-06-29T15:49:28.734 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.725+0000 7f97f4c60700 1 -- 172.21.15.47:0/1610127848 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f97d00539e0 msgr2=0x7f97d0055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:28.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.725+0000 7f97f4c60700 1 --2- 172.21.15.47:0/1610127848 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f97d00539e0 0x7f97d0055ea0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7f97e000b000 tx=0x7f97e0006df0).stop 2022-06-29T15:49:28.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.725+0000 7f97f4c60700 1 -- 172.21.15.47:0/1610127848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97f005fc50 msgr2=0x7f97f01487b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:28.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.725+0000 7f97f4c60700 1 --2- 172.21.15.47:0/1610127848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97f005fc50 0x7f97f01487b0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 rx=0x7f97e4002f70 tx=0x7f97e4015000).stop 2022-06-29T15:49:28.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.725+0000 7f97f4c60700 1 -- 172.21.15.47:0/1610127848 shutdown_connections 2022-06-29T15:49:28.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.725+0000 7f97f4c60700 1 --2- 172.21.15.47:0/1610127848 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f97d00539e0 0x7f97d0055ea0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:28.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.725+0000 7f97f4c60700 1 --2- 172.21.15.47:0/1610127848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97f00b17a0 0x7f97f0148ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:28.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.725+0000 7f97f4c60700 1 --2- 172.21.15.47:0/1610127848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97f005fc50 0x7f97f01487b0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:28.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.725+0000 7f97f4c60700 1 --2- 172.21.15.47:0/1610127848 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f97f005f3f0 0x7f97f0140240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:28.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.725+0000 7f97f4c60700 1 -- 172.21.15.47:0/1610127848 >> 172.21.15.47:0/1610127848 conn(0x7f97f00b5320 msgr2=0x7f97f00a7d40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:28.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.725+0000 7f97f4c60700 1 -- 172.21.15.47:0/1610127848 shutdown_connections 2022-06-29T15:49:28.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.725+0000 7f97f4c60700 1 -- 172.21.15.47:0/1610127848 wait complete. 2022-06-29T15:49:28.750 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:49:28.854 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.845+0000 7fadbce11700 1 Processor -- start 2022-06-29T15:49:28.854 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.845+0000 7fadbce11700 1 -- start start 2022-06-29T15:49:28.856 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.849+0000 7fadbce11700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fadb80acff0 0x7fadb805fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:28.857 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.849+0000 7fadbce11700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fadb81437d0 0x7fadb8143c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:28.857 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.849+0000 7fadbce11700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadb80a97e0 con 0x7fadb80b19a0 2022-06-29T15:49:28.858 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.849+0000 7fadbce11700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadb80b1d80 con 0x7fadb80acff0 2022-06-29T15:49:28.858 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.849+0000 7fadbce11700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadb805f430 con 0x7fadb81437d0 2022-06-29T15:49:28.859 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.849+0000 7fadb6ffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fadb80acff0 0x7fadb805fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:28.859 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.849+0000 7fadb77fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fadb80b19a0 0x7fadb80ae840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:43890/0 (socket says 172.21.15.47:43890) 2022-06-29T15:49:28.860 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.849+0000 7fadb77fe700 1 -- 172.21.15.47:0/475407294 learned_addr learned my addr 172.21.15.47:0/475407294 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:49:28.860 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.849+0000 7fadb7fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fadb81437d0 0x7fadb8143c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:28.861 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.849+0000 7fadb6ffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fadb80acff0 0x7fadb805fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:56480/0 (socket says 172.21.15.47:56480) 2022-06-29T15:49:28.861 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.849+0000 7fadb7fff700 1 -- 172.21.15.47:0/475407294 >> v2:172.21.15.64:3300/0 conn(0x7fadb80acff0 msgr2=0x7fadb805fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:28.861 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.849+0000 7fadb7fff700 1 --2- 172.21.15.47:0/475407294 >> v2:172.21.15.64:3300/0 conn(0x7fadb80acff0 0x7fadb805fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:28.862 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.849+0000 7fadb7fff700 1 -- 172.21.15.47:0/475407294 >> v1:172.21.15.64:6789/0 conn(0x7fadb80b19a0 legacy=0x7fadb80ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:28.862 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.849+0000 7fadb7fff700 1 -- 172.21.15.47:0/475407294 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb80a92a0 con 0x7fadb81437d0 2022-06-29T15:49:28.865 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.849+0000 7fadb7fff700 1 --2- 172.21.15.47:0/475407294 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fadb81437d0 0x7fadb8143c60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fadb00032f0 tx=0x7fadb00091d0).ready entity=mon.2 client_cookie=a235b48d5eed0b55 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:28.866 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.849+0000 7fadb67fc700 1 -- 172.21.15.47:0/475407294 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fadb000b6c0 con 0x7fadb81437d0 2022-06-29T15:49:28.866 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.849+0000 7fadb67fc700 1 -- 172.21.15.47:0/475407294 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb000b820 con 0x7fadb81437d0 2022-06-29T15:49:28.866 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.849+0000 7fadb67fc700 1 -- 172.21.15.47:0/475407294 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fadb000b990 con 0x7fadb81437d0 2022-06-29T15:49:28.866 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.849+0000 7fadbce11700 1 -- 172.21.15.47:0/475407294 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fadb81437d0 msgr2=0x7fadb8143c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:28.867 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.849+0000 7fadbce11700 1 --2- 172.21.15.47:0/475407294 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fadb81437d0 0x7fadb8143c60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fadb00032f0 tx=0x7fadb00091d0).stop 2022-06-29T15:49:28.867 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.849+0000 7fadbce11700 1 -- 172.21.15.47:0/475407294 shutdown_connections 2022-06-29T15:49:28.867 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.849+0000 7fadbce11700 1 --2- 172.21.15.47:0/475407294 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fadb81437d0 0x7fadb8143c60 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:28.867 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.849+0000 7fadbce11700 1 --2- 172.21.15.47:0/475407294 >> v2:172.21.15.64:3300/0 conn(0x7fadb80acff0 0x7fadb805fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:28.868 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.849+0000 7fadbce11700 1 -- 172.21.15.47:0/475407294 >> 172.21.15.47:0/475407294 conn(0x7fadb8062df0 msgr2=0x7fadb80631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:28.868 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.849+0000 7fadbce11700 1 -- 172.21.15.47:0/475407294 shutdown_connections 2022-06-29T15:49:28.868 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.853+0000 7fadbce11700 1 -- 172.21.15.47:0/475407294 wait complete. 2022-06-29T15:49:28.868 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.853+0000 7fadbce11700 1 Processor -- start 2022-06-29T15:49:28.868 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.853+0000 7fadbce11700 1 -- start start 2022-06-29T15:49:28.869 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.853+0000 7fadbce11700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadb80acff0 0x7fadb8140730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:28.869 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.853+0000 7fadbce11700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fadb80b19a0 0x7fadb813dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:28.869 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.853+0000 7fadbce11700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fadb81437d0 0x7fadb81422d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:28.869 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.853+0000 7fadbce11700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb80ad760 con 0x7fadb81437d0 2022-06-29T15:49:28.870 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.853+0000 7fadbce11700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb8099540 con 0x7fadb80acff0 2022-06-29T15:49:28.870 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.853+0000 7fadbce11700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadb80b32e0 con 0x7fadb80b19a0 2022-06-29T15:49:28.870 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.853+0000 7fadb7fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fadb81437d0 0x7fadb81422d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:28.870 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.853+0000 7fadb7fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fadb81437d0 0x7fadb81422d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50782/0 (socket says 172.21.15.47:50782) 2022-06-29T15:49:28.871 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.853+0000 7fadb7fff700 1 -- 172.21.15.47:0/4248290408 learned_addr learned my addr 172.21.15.47:0/4248290408 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:28.871 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.853+0000 7fadb6ffd700 1 --2- 172.21.15.47:0/4248290408 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fadb80b19a0 0x7fadb813dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:28.871 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.853+0000 7fadb77fe700 1 --2- 172.21.15.47:0/4248290408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadb80acff0 0x7fadb8140730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:28.871 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.857+0000 7fadb7fff700 1 -- 172.21.15.47:0/4248290408 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fadb80b19a0 msgr2=0x7fadb813dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:28.872 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.857+0000 7fadb7fff700 1 --2- 172.21.15.47:0/4248290408 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fadb80b19a0 0x7fadb813dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:28.872 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.857+0000 7fadb7fff700 1 -- 172.21.15.47:0/4248290408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadb80acff0 msgr2=0x7fadb8140730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:28.872 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.857+0000 7fadb7fff700 1 --2- 172.21.15.47:0/4248290408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadb80acff0 0x7fadb8140730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:28.872 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.857+0000 7fadb7fff700 1 -- 172.21.15.47:0/4248290408 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb0009470 con 0x7fadb81437d0 2022-06-29T15:49:28.873 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.857+0000 7fadb7fff700 1 --2- 172.21.15.47:0/4248290408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fadb81437d0 0x7fadb81422d0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 rx=0x7fadb00032f0 tx=0x7fadb0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:28.873 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.857+0000 7fadb4ff9700 1 -- 172.21.15.47:0/4248290408 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fadb0011e20 con 0x7fadb81437d0 2022-06-29T15:49:28.873 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.857+0000 7fadb4ff9700 1 -- 172.21.15.47:0/4248290408 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb0020400 con 0x7fadb81437d0 2022-06-29T15:49:28.873 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.857+0000 7fadb4ff9700 1 -- 172.21.15.47:0/4248290408 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fadb0020760 con 0x7fadb81437d0 2022-06-29T15:49:28.873 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.857+0000 7fadbce11700 1 -- 172.21.15.47:0/4248290408 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadb813f260 con 0x7fadb81437d0 2022-06-29T15:49:28.874 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.857+0000 7fadbce11700 1 -- 172.21.15.47:0/4248290408 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadb813eb90 con 0x7fadb81437d0 2022-06-29T15:49:28.874 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.857+0000 7fadbce11700 1 -- 172.21.15.47:0/4248290408 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadb8145e40 con 0x7fadb81437d0 2022-06-29T15:49:28.875 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.857+0000 7fadb4ff9700 1 -- 172.21.15.47:0/4248290408 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fadb001e070 con 0x7fadb81437d0 2022-06-29T15:49:28.875 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.857+0000 7fadb4ff9700 1 --2- 172.21.15.47:0/4248290408 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fad9c053a10 0x7fad9c055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:28.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.857+0000 7fadb4ff9700 1 -- 172.21.15.47:0/4248290408 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fadb006cf90 con 0x7fadb81437d0 2022-06-29T15:49:28.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.861+0000 7fadb77fe700 1 --2- 172.21.15.47:0/4248290408 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fad9c053a10 0x7fad9c055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:28.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.861+0000 7fadb77fe700 1 --2- 172.21.15.47:0/4248290408 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fad9c053a10 0x7fad9c055ed0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7fada800d000 tx=0x7fada8009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:28.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:28.869+0000 7fadb4ff9700 1 -- 172.21.15.47:0/4248290408 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fadb0043d50 con 0x7fadb81437d0 2022-06-29T15:49:29.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.025+0000 7fadbce11700 1 -- 172.21.15.47:0/4248290408 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fadb80573d0 con 0x7fadb81437d0 2022-06-29T15:49:29.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.025+0000 7fadb4ff9700 1 -- 172.21.15.47:0/4248290408 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+2493 (secure 0 0 0) 0x7fadb0015070 con 0x7fadb81437d0 2022-06-29T15:49:29.034 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:49:29.035 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":40,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6577,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/4286932616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":4286932616},{"type":"v1","addr":"172.21.15.64:6833","nonce":4286932616}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":38},{"gid":6657,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3528102962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3528102962},{"type":"v1","addr":"172.21.15.47:6835","nonce":3528102962}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":38}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:49:27.717245+0000","modified":"2022-06-29T15:49:28.719969+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6660},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6660":{"gid":6660,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6837/4146905199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":4146905199},{"type":"v1","addr":"172.21.15.47:6837","nonce":4146905199}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2022-06-29T15:49:29.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.029+0000 7fadbce11700 1 -- 172.21.15.47:0/4248290408 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fad9c053a10 msgr2=0x7fad9c055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:29.038 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.029+0000 7fadbce11700 1 --2- 172.21.15.47:0/4248290408 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fad9c053a10 0x7fad9c055ed0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7fada800d000 tx=0x7fada8009040).stop 2022-06-29T15:49:29.038 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.029+0000 7fadbce11700 1 -- 172.21.15.47:0/4248290408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fadb81437d0 msgr2=0x7fadb81422d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:29.038 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.029+0000 7fadbce11700 1 --2- 172.21.15.47:0/4248290408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fadb81437d0 0x7fadb81422d0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 rx=0x7fadb00032f0 tx=0x7fadb0016000).stop 2022-06-29T15:49:29.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.029+0000 7fadbce11700 1 -- 172.21.15.47:0/4248290408 shutdown_connections 2022-06-29T15:49:29.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.029+0000 7fadbce11700 1 --2- 172.21.15.47:0/4248290408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fadb81437d0 0x7fadb81422d0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:29.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.029+0000 7fadbce11700 1 --2- 172.21.15.47:0/4248290408 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fadb80b19a0 0x7fadb813dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:29.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.029+0000 7fadbce11700 1 --2- 172.21.15.47:0/4248290408 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fad9c053a10 0x7fad9c055ed0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:29.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.029+0000 7fadbce11700 1 --2- 172.21.15.47:0/4248290408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadb80acff0 0x7fadb8140730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:29.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.029+0000 7fadbce11700 1 -- 172.21.15.47:0/4248290408 >> 172.21.15.47:0/4248290408 conn(0x7fadb8062df0 msgr2=0x7fadb81487a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:29.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.029+0000 7fadbce11700 1 -- 172.21.15.47:0/4248290408 shutdown_connections 2022-06-29T15:49:29.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.029+0000 7fadbce11700 1 -- 172.21.15.47:0/4248290408 wait complete. 2022-06-29T15:49:29.042 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 40 2022-06-29T15:49:29.047 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:49:29.152 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.145+0000 7fa98c727700 1 Processor -- start 2022-06-29T15:49:29.152 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.145+0000 7fa98c727700 1 -- start start 2022-06-29T15:49:29.155 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.145+0000 7fa98c727700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa9840b19a0 0x7fa9840ad560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:29.155 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.145+0000 7fa98c727700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa98413fd90 0x7fa9841401f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:29.157 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.145+0000 7fa98c727700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9840b2450 con 0x7fa9840ad180 2022-06-29T15:49:29.157 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.145+0000 7fa98c727700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa984061340 con 0x7fa98413fd90 2022-06-29T15:49:29.157 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.145+0000 7fa98c727700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa984060980 con 0x7fa9840b19a0 2022-06-29T15:49:29.157 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.145+0000 7fa98acc4700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa98413fd90 0x7fa9841401f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:29.158 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.145+0000 7fa98a4c3700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa9840ad180 0x7fa98413d2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36342/0 (socket says 172.21.15.47:36342) 2022-06-29T15:49:29.158 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.145+0000 7fa98acc4700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa98413fd90 0x7fa9841401f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50790/0 (socket says 172.21.15.47:50790) 2022-06-29T15:49:29.158 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.145+0000 7fa98a4c3700 1 -- 172.21.15.47:0/2639093986 learned_addr learned my addr 172.21.15.47:0/2639093986 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:49:29.158 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.145+0000 7fa989cc2700 1 --2- 172.21.15.47:0/2639093986 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa9840b19a0 0x7fa9840ad560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:29.159 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.145+0000 7fa98acc4700 1 -- 172.21.15.47:0/2639093986 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa9840b19a0 msgr2=0x7fa9840ad560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:29.159 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.145+0000 7fa98acc4700 1 --2- 172.21.15.47:0/2639093986 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa9840b19a0 0x7fa9840ad560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:29.159 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.145+0000 7fa98acc4700 1 -- 172.21.15.47:0/2639093986 >> v1:172.21.15.47:6789/0 conn(0x7fa9840ad180 legacy=0x7fa98413d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:29.159 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.145+0000 7fa9894c1700 1 -- 172.21.15.47:0/2639093986 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1094676531 0 0) 0x7fa9840b2450 con 0x7fa9840ad180 2022-06-29T15:49:29.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.145+0000 7fa98acc4700 1 -- 172.21.15.47:0/2639093986 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9840606e0 con 0x7fa98413fd90 2022-06-29T15:49:29.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.145+0000 7fa98acc4700 1 --2- 172.21.15.47:0/2639093986 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa98413fd90 0x7fa9841401f0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 rx=0x7fa97c0032f0 tx=0x7fa97c0091d0).ready entity=mon.0 client_cookie=a330fd50bdcc1f5a server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:29.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.145+0000 7fa9894c1700 1 -- 172.21.15.47:0/2639093986 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa97c009ea0 con 0x7fa98413fd90 2022-06-29T15:49:29.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.145+0000 7fa9894c1700 1 -- 172.21.15.47:0/2639093986 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa97c00b8a0 con 0x7fa98413fd90 2022-06-29T15:49:29.161 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.145+0000 7fa9894c1700 1 -- 172.21.15.47:0/2639093986 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa97c00ba10 con 0x7fa98413fd90 2022-06-29T15:49:29.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.149+0000 7fa98c727700 1 -- 172.21.15.47:0/2639093986 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa98413fd90 msgr2=0x7fa9841401f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:29.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.149+0000 7fa98c727700 1 --2- 172.21.15.47:0/2639093986 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa98413fd90 0x7fa9841401f0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 rx=0x7fa97c0032f0 tx=0x7fa97c0091d0).stop 2022-06-29T15:49:29.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.149+0000 7fa98c727700 1 -- 172.21.15.47:0/2639093986 shutdown_connections 2022-06-29T15:49:29.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.149+0000 7fa98c727700 1 --2- 172.21.15.47:0/2639093986 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa98413fd90 0x7fa9841401f0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:29.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.149+0000 7fa98c727700 1 --2- 172.21.15.47:0/2639093986 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa9840b19a0 0x7fa9840ad560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:29.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.149+0000 7fa98c727700 1 -- 172.21.15.47:0/2639093986 >> 172.21.15.47:0/2639093986 conn(0x7fa9840626f0 msgr2=0x7fa984062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:29.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.149+0000 7fa98c727700 1 -- 172.21.15.47:0/2639093986 shutdown_connections 2022-06-29T15:49:29.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.149+0000 7fa98c727700 1 -- 172.21.15.47:0/2639093986 wait complete. 2022-06-29T15:49:29.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.149+0000 7fa98c727700 1 Processor -- start 2022-06-29T15:49:29.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.149+0000 7fa98c727700 1 -- start start 2022-06-29T15:49:29.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.149+0000 7fa98c727700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9840ad180 0x7fa984139650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:29.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.149+0000 7fa98c727700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9840b19a0 0x7fa984134240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:29.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.149+0000 7fa98c727700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa98413fd90 0x7fa984134730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:29.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.149+0000 7fa98c727700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa984135fd0 con 0x7fa9840ad180 2022-06-29T15:49:29.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.149+0000 7fa98c727700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98413aa50 con 0x7fa9840b19a0 2022-06-29T15:49:29.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.149+0000 7fa98c727700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa984135d30 con 0x7fa98413fd90 2022-06-29T15:49:29.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.149+0000 7fa98a4c3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9840ad180 0x7fa984139650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:29.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.149+0000 7fa98a4c3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9840ad180 0x7fa984139650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50792/0 (socket says 172.21.15.47:50792) 2022-06-29T15:49:29.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.149+0000 7fa98a4c3700 1 -- 172.21.15.47:0/655556448 learned_addr learned my addr 172.21.15.47:0/655556448 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:29.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.149+0000 7fa989cc2700 1 --2- 172.21.15.47:0/655556448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9840b19a0 0x7fa984134240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:29.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.149+0000 7fa98acc4700 1 --2- 172.21.15.47:0/655556448 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa98413fd90 0x7fa984134730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:29.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.153+0000 7fa98a4c3700 1 -- 172.21.15.47:0/655556448 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa98413fd90 msgr2=0x7fa984134730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:29.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.153+0000 7fa98a4c3700 1 --2- 172.21.15.47:0/655556448 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa98413fd90 0x7fa984134730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:29.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.153+0000 7fa98a4c3700 1 -- 172.21.15.47:0/655556448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9840b19a0 msgr2=0x7fa984134240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:29.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.153+0000 7fa98a4c3700 1 --2- 172.21.15.47:0/655556448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9840b19a0 0x7fa984134240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:29.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.153+0000 7fa98a4c3700 1 -- 172.21.15.47:0/655556448 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa97c009470 con 0x7fa9840ad180 2022-06-29T15:49:29.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.153+0000 7fa989cc2700 1 --2- 172.21.15.47:0/655556448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9840b19a0 0x7fa984134240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:49:29.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.153+0000 7fa98a4c3700 1 --2- 172.21.15.47:0/655556448 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9840ad180 0x7fa984139650 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 rx=0x7fa984137580 tx=0x7fa974007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:29.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.153+0000 7fa9737fe700 1 -- 172.21.15.47:0/655556448 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa97400ce40 con 0x7fa9840ad180 2022-06-29T15:49:29.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.153+0000 7fa9737fe700 1 -- 172.21.15.47:0/655556448 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9740072e0 con 0x7fa9840ad180 2022-06-29T15:49:29.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.153+0000 7fa98c727700 1 -- 172.21.15.47:0/655556448 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa984139b40 con 0x7fa9840ad180 2022-06-29T15:49:29.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.153+0000 7fa9737fe700 1 -- 172.21.15.47:0/655556448 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa974005610 con 0x7fa9840ad180 2022-06-29T15:49:29.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.153+0000 7fa98c727700 1 -- 172.21.15.47:0/655556448 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9841357e0 con 0x7fa9840ad180 2022-06-29T15:49:29.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.153+0000 7fa98c727700 1 -- 172.21.15.47:0/655556448 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa98413ad20 con 0x7fa9840ad180 2022-06-29T15:49:29.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.153+0000 7fa9737fe700 1 -- 172.21.15.47:0/655556448 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa97401a060 con 0x7fa9840ad180 2022-06-29T15:49:29.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.153+0000 7fa9737fe700 1 --2- 172.21.15.47:0/655556448 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa964053950 0x7fa964055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:29.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.157+0000 7fa9737fe700 1 -- 172.21.15.47:0/655556448 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa97406b940 con 0x7fa9840ad180 2022-06-29T15:49:29.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.157+0000 7fa989cc2700 1 --2- 172.21.15.47:0/655556448 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa964053950 0x7fa964055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:29.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.157+0000 7fa989cc2700 1 --2- 172.21.15.47:0/655556448 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa964053950 0x7fa964055e10 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7fa98000c000 tx=0x7fa980008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:29.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.161+0000 7fa9737fe700 1 -- 172.21.15.47:0/655556448 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa974042700 con 0x7fa9840ad180 2022-06-29T15:49:29.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.285+0000 7fa98c727700 1 -- 172.21.15.47:0/655556448 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9840573d0 con 0x7fa9840ad180 2022-06-29T15:49:29.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.285+0000 7fa9737fe700 1 -- 172.21.15.47:0/655556448 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+16274 (secure 0 0 0) 0x7fa974018020 con 0x7fa9840ad180 2022-06-29T15:49:29.295 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:49:29.295 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":90,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:49:28.720705+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:49:25.785974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2022-06-29T15:49:26.726660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:49:29.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.289+0000 7fa98c727700 1 -- 172.21.15.47:0/655556448 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa964053950 msgr2=0x7fa964055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:29.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.289+0000 7fa98c727700 1 --2- 172.21.15.47:0/655556448 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa964053950 0x7fa964055e10 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7fa98000c000 tx=0x7fa980008040).stop 2022-06-29T15:49:29.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.289+0000 7fa98c727700 1 -- 172.21.15.47:0/655556448 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9840ad180 msgr2=0x7fa984139650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:29.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.289+0000 7fa98c727700 1 --2- 172.21.15.47:0/655556448 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9840ad180 0x7fa984139650 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 rx=0x7fa984137580 tx=0x7fa974007ac0).stop 2022-06-29T15:49:29.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.289+0000 7fa98c727700 1 -- 172.21.15.47:0/655556448 shutdown_connections 2022-06-29T15:49:29.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.289+0000 7fa98c727700 1 --2- 172.21.15.47:0/655556448 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa98413fd90 0x7fa984134730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:29.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.289+0000 7fa98c727700 1 --2- 172.21.15.47:0/655556448 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa964053950 0x7fa964055e10 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:29.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.289+0000 7fa98c727700 1 --2- 172.21.15.47:0/655556448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9840b19a0 0x7fa984134240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:29.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.289+0000 7fa98c727700 1 --2- 172.21.15.47:0/655556448 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9840ad180 0x7fa984139650 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:29.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.289+0000 7fa98c727700 1 -- 172.21.15.47:0/655556448 >> 172.21.15.47:0/655556448 conn(0x7fa9840626f0 msgr2=0x7fa98413dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:29.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.289+0000 7fa98c727700 1 -- 172.21.15.47:0/655556448 shutdown_connections 2022-06-29T15:49:29.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.289+0000 7fa98c727700 1 -- 172.21.15.47:0/655556448 wait complete. 2022-06-29T15:49:29.309 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-29T15:49:29.424 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.417+0000 7f13da9ca700 1 Processor -- start 2022-06-29T15:49:29.425 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.417+0000 7f13da9ca700 1 -- start start 2022-06-29T15:49:29.426 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.417+0000 7f13da9ca700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f13d414ca80 0x7f13d414ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:29.427 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.417+0000 7f13da9ca700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13d405f7c0 con 0x7f13d4140940 2022-06-29T15:49:29.427 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.417+0000 7f13da9ca700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13d40b4e50 con 0x7f13d40b30e0 2022-06-29T15:49:29.427 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.417+0000 7f13da9ca700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13d40621e0 con 0x7f13d414ca80 2022-06-29T15:49:29.427 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.417+0000 7f13d3fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f13d4140940 0x7f13d405f0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36354/0 (socket says 172.21.15.47:36354) 2022-06-29T15:49:29.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.417+0000 7f13d3fff700 1 -- 172.21.15.47:0/414345474 learned_addr learned my addr 172.21.15.47:0/414345474 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:49:29.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.417+0000 7f13d8f67700 1 --2- 172.21.15.47:0/414345474 >> v2:172.21.15.64:3300/0 conn(0x7f13d414ca80 0x7f13d414ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:29.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.417+0000 7f13d2ffd700 1 -- 172.21.15.47:0/414345474 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2240890200 0 0) 0x7f13d405f7c0 con 0x7f13d4140940 2022-06-29T15:49:29.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.417+0000 7f13d2ffd700 1 -- 172.21.15.47:0/414345474 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13c0003260 con 0x7f13d4140940 2022-06-29T15:49:29.430 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.421+0000 7f13d2ffd700 1 -- 172.21.15.47:0/414345474 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3700722965 0 0) 0x7f13c0003260 con 0x7f13d4140940 2022-06-29T15:49:29.431 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.421+0000 7f13d8f67700 1 -- 172.21.15.47:0/414345474 >> v1:172.21.15.64:6789/0 conn(0x7f13d40b30e0 legacy=0x7f13d40b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:29.431 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.421+0000 7f13d8f67700 1 -- 172.21.15.47:0/414345474 >> v1:172.21.15.47:6789/0 conn(0x7f13d4140940 legacy=0x7f13d405f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:29.431 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.421+0000 7f13d8f67700 1 -- 172.21.15.47:0/414345474 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d4143170 con 0x7f13d414ca80 2022-06-29T15:49:29.431 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.421+0000 7f13d8f67700 1 --2- 172.21.15.47:0/414345474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13d414ca80 0x7f13d414ee70 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7f13c4001800 tx=0x7f13cc0091d0).ready entity=mon.1 client_cookie=bb73485f2d5e9793 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:29.432 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.421+0000 7f13d2ffd700 1 -- 172.21.15.47:0/414345474 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f13c4001660 con 0x7f13d414ca80 2022-06-29T15:49:29.432 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.421+0000 7f13d2ffd700 1 -- 172.21.15.47:0/414345474 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13cc00b580 con 0x7f13d414ca80 2022-06-29T15:49:29.432 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.421+0000 7f13d2ffd700 1 -- 172.21.15.47:0/414345474 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f13cc00b6f0 con 0x7f13d414ca80 2022-06-29T15:49:29.433 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.421+0000 7f13da9ca700 1 -- 172.21.15.47:0/414345474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13d414ca80 msgr2=0x7f13d414ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:29.433 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.421+0000 7f13da9ca700 1 --2- 172.21.15.47:0/414345474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13d414ca80 0x7f13d414ee70 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7f13c4001800 tx=0x7f13cc0091d0).stop 2022-06-29T15:49:29.433 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.421+0000 7f13da9ca700 1 -- 172.21.15.47:0/414345474 shutdown_connections 2022-06-29T15:49:29.433 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.421+0000 7f13da9ca700 1 --2- 172.21.15.47:0/414345474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13d414ca80 0x7f13d414ee70 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:29.434 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.421+0000 7f13da9ca700 1 -- 172.21.15.47:0/414345474 >> 172.21.15.47:0/414345474 conn(0x7f13d40627d0 msgr2=0x7f13d4062bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:29.434 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.421+0000 7f13da9ca700 1 -- 172.21.15.47:0/414345474 shutdown_connections 2022-06-29T15:49:29.434 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.421+0000 7f13da9ca700 1 -- 172.21.15.47:0/414345474 wait complete. 2022-06-29T15:49:29.437 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.425+0000 7f13da9ca700 1 Processor -- start 2022-06-29T15:49:29.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.425+0000 7f13da9ca700 1 -- start start 2022-06-29T15:49:29.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.425+0000 7f13da9ca700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13d40b30e0 0x7f13d40b6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:29.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.425+0000 7f13da9ca700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f13d4140940 0x7f13d40b7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:29.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.425+0000 7f13da9ca700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13d414ca80 0x7f13d413faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:29.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.425+0000 7f13da9ca700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d4137bf0 con 0x7f13d40b30e0 2022-06-29T15:49:29.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.425+0000 7f13da9ca700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d41393f0 con 0x7f13d414ca80 2022-06-29T15:49:29.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.425+0000 7f13da9ca700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13d4139690 con 0x7f13d4140940 2022-06-29T15:49:29.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.425+0000 7f13d3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13d40b30e0 0x7f13d40b6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:29.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.425+0000 7f13d3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13d40b30e0 0x7f13d40b6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50806/0 (socket says 172.21.15.47:50806) 2022-06-29T15:49:29.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.425+0000 7f13d3fff700 1 -- 172.21.15.47:0/3980102395 learned_addr learned my addr 172.21.15.47:0/3980102395 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:29.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.425+0000 7f13d37fe700 1 --2- 172.21.15.47:0/3980102395 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f13d4140940 0x7f13d40b7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:29.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.425+0000 7f13d8f67700 1 --2- 172.21.15.47:0/3980102395 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13d414ca80 0x7f13d413faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:29.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.425+0000 7f13d37fe700 1 -- 172.21.15.47:0/3980102395 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13d414ca80 msgr2=0x7f13d413faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:29.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.425+0000 7f13d37fe700 1 --2- 172.21.15.47:0/3980102395 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13d414ca80 0x7f13d413faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:29.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.425+0000 7f13d37fe700 1 -- 172.21.15.47:0/3980102395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13d40b30e0 msgr2=0x7f13d40b6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:29.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.425+0000 7f13d37fe700 1 --2- 172.21.15.47:0/3980102395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13d40b30e0 0x7f13d40b6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:29.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.425+0000 7f13d37fe700 1 -- 172.21.15.47:0/3980102395 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13cc009470 con 0x7f13d4140940 2022-06-29T15:49:29.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.425+0000 7f13d3fff700 1 --2- 172.21.15.47:0/3980102395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13d40b30e0 0x7f13d40b6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:49:29.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.425+0000 7f13d37fe700 1 --2- 172.21.15.47:0/3980102395 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f13d4140940 0x7f13d40b7800 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f13d40b7750 tx=0x7f13c8007b70).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:29.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.425+0000 7f13d17fa700 1 -- 172.21.15.47:0/3980102395 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f13c8007030 con 0x7f13d4140940 2022-06-29T15:49:29.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.425+0000 7f13d17fa700 1 -- 172.21.15.47:0/3980102395 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13c8007190 con 0x7f13d4140940 2022-06-29T15:49:29.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.429+0000 7f13da9ca700 1 -- 172.21.15.47:0/3980102395 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13d4139930 con 0x7f13d4140940 2022-06-29T15:49:29.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.429+0000 7f13d17fa700 1 -- 172.21.15.47:0/3980102395 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f13c8006510 con 0x7f13d4140940 2022-06-29T15:49:29.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.429+0000 7f13da9ca700 1 -- 172.21.15.47:0/3980102395 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13d4138800 con 0x7f13d4140940 2022-06-29T15:49:29.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.429+0000 7f13d17fa700 1 -- 172.21.15.47:0/3980102395 <== mon.2 v2:172.21.15.64:3301/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f13c8010030 con 0x7f13d4140940 2022-06-29T15:49:29.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.429+0000 7f13da9ca700 1 -- 172.21.15.47:0/3980102395 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13d40573d0 con 0x7f13d4140940 2022-06-29T15:49:29.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.429+0000 7f13d17fa700 1 --2- 172.21.15.47:0/3980102395 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f13b40539e0 0x7f13b4055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:29.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.429+0000 7f13d17fa700 1 -- 172.21.15.47:0/3980102395 <== mon.2 v2:172.21.15.64:3301/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f13c806bd10 con 0x7f13d4140940 2022-06-29T15:49:29.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.429+0000 7f13d3fff700 1 --2- 172.21.15.47:0/3980102395 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f13b40539e0 0x7f13b4055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:29.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.429+0000 7f13d3fff700 1 --2- 172.21.15.47:0/3980102395 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f13b40539e0 0x7f13b4055ea0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f13c400d000 tx=0x7f13c4000bc0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:29.447 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.441+0000 7f13d17fa700 1 -- 172.21.15.47:0/3980102395 <== mon.2 v2:172.21.15.64:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f13c8042a30 con 0x7f13d4140940 2022-06-29T15:49:29.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.613+0000 7f13da9ca700 1 -- 172.21.15.47:0/3980102395 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f13d406a0f0 con 0x7f13d4140940 2022-06-29T15:49:29.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.621+0000 7f13d17fa700 1 -- 172.21.15.47:0/3980102395 <== mon.2 v2:172.21.15.64:3301/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v24) v1 ==== 184+0+0 (secure 0 0 0) 0x7f13c8046050 con 0x7f13d4140940 2022-06-29T15:49:29.634 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.625+0000 7f13da9ca700 1 -- 172.21.15.47:0/3980102395 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f13b40539e0 msgr2=0x7f13b4055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:29.634 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.625+0000 7f13da9ca700 1 --2- 172.21.15.47:0/3980102395 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f13b40539e0 0x7f13b4055ea0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f13c400d000 tx=0x7f13c4000bc0).stop 2022-06-29T15:49:29.634 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.625+0000 7f13da9ca700 1 -- 172.21.15.47:0/3980102395 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f13d4140940 msgr2=0x7f13d40b7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:29.635 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.625+0000 7f13da9ca700 1 --2- 172.21.15.47:0/3980102395 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f13d4140940 0x7f13d40b7800 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f13d40b7750 tx=0x7f13c8007b70).stop 2022-06-29T15:49:29.635 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.625+0000 7f13da9ca700 1 -- 172.21.15.47:0/3980102395 shutdown_connections 2022-06-29T15:49:29.635 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.625+0000 7f13da9ca700 1 --2- 172.21.15.47:0/3980102395 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f13b40539e0 0x7f13b4055ea0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:29.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.625+0000 7f13da9ca700 1 --2- 172.21.15.47:0/3980102395 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13d414ca80 0x7f13d413faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:29.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.625+0000 7f13da9ca700 1 --2- 172.21.15.47:0/3980102395 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f13d4140940 0x7f13d40b7800 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:29.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.625+0000 7f13da9ca700 1 --2- 172.21.15.47:0/3980102395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13d40b30e0 0x7f13d40b6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:29.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.625+0000 7f13da9ca700 1 -- 172.21.15.47:0/3980102395 >> 172.21.15.47:0/3980102395 conn(0x7f13d40627d0 msgr2=0x7f13d4065bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:29.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.625+0000 7f13da9ca700 1 -- 172.21.15.47:0/3980102395 shutdown_connections 2022-06-29T15:49:29.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.625+0000 7f13da9ca700 1 -- 172.21.15.47:0/3980102395 wait complete. 2022-06-29T15:49:29.637 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.0 2022-06-29T15:49:29.647 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-29T15:49:29.765 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.757+0000 7f15b1c64700 1 Processor -- start 2022-06-29T15:49:29.766 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.757+0000 7f15b1c64700 1 -- start start 2022-06-29T15:49:29.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.757+0000 7f15b1c64700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f15ac05c850 0x7f15ac143a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:29.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.757+0000 7f15b1c64700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f15ac0676a0 0x7f15ac140130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:29.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.757+0000 7f15b1c64700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15ac05cc30 con 0x7f15ac0b5d90 2022-06-29T15:49:29.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.757+0000 7f15b1c64700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15ac1430a0 con 0x7f15ac0676a0 2022-06-29T15:49:29.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.757+0000 7f15b1c64700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15ac143300 con 0x7f15ac05c850 2022-06-29T15:49:29.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.757+0000 7f15b1463700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f15ac0676a0 0x7f15ac140130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:29.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.757+0000 7f15b1463700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f15ac0676a0 0x7f15ac140130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50818/0 (socket says 172.21.15.47:50818) 2022-06-29T15:49:29.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.757+0000 7f15b1463700 1 -- 172.21.15.47:0/287787642 learned_addr learned my addr 172.21.15.47:0/287787642 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:29.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.757+0000 7f15b0c62700 1 --2- 172.21.15.47:0/287787642 >> v2:172.21.15.64:3300/0 conn(0x7f15ac05c850 0x7f15ac143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:29.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.757+0000 7f15b1463700 1 -- 172.21.15.47:0/287787642 >> v2:172.21.15.64:3300/0 conn(0x7f15ac05c850 msgr2=0x7f15ac143a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:29.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.757+0000 7f15b1463700 1 --2- 172.21.15.47:0/287787642 >> v2:172.21.15.64:3300/0 conn(0x7f15ac05c850 0x7f15ac143a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:29.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.757+0000 7f15b1463700 1 -- 172.21.15.47:0/287787642 >> v1:172.21.15.64:6789/0 conn(0x7f15ac0b5d90 legacy=0x7f15ac0b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:29.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.757+0000 7f15b1463700 1 -- 172.21.15.47:0/287787642 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15ac1435a0 con 0x7f15ac0676a0 2022-06-29T15:49:29.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.761+0000 7f15b1463700 1 --2- 172.21.15.47:0/287787642 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15ac0676a0 0x7f15ac140130 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 rx=0x7f15a4002f70 tx=0x7f15a4006eb0).ready entity=mon.0 client_cookie=c0fafa4db6a55957 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:29.781 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.761+0000 7f15ab7fe700 1 -- 172.21.15.47:0/287787642 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f15a400d5c0 con 0x7f15ac0676a0 2022-06-29T15:49:29.781 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.761+0000 7f15ab7fe700 1 -- 172.21.15.47:0/287787642 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15a4000e90 con 0x7f15ac0676a0 2022-06-29T15:49:29.781 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.761+0000 7f15ab7fe700 1 -- 172.21.15.47:0/287787642 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f15a400d950 con 0x7f15ac0676a0 2022-06-29T15:49:29.782 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.761+0000 7f15b1c64700 1 -- 172.21.15.47:0/287787642 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15ac0676a0 msgr2=0x7f15ac140130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:29.782 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.761+0000 7f15b1c64700 1 --2- 172.21.15.47:0/287787642 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15ac0676a0 0x7f15ac140130 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 rx=0x7f15a4002f70 tx=0x7f15a4006eb0).stop 2022-06-29T15:49:29.782 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.761+0000 7f15b1c64700 1 -- 172.21.15.47:0/287787642 shutdown_connections 2022-06-29T15:49:29.782 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.761+0000 7f15b1c64700 1 --2- 172.21.15.47:0/287787642 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15ac0676a0 0x7f15ac140130 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:29.783 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.761+0000 7f15b1c64700 1 --2- 172.21.15.47:0/287787642 >> v2:172.21.15.64:3300/0 conn(0x7f15ac05c850 0x7f15ac143a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:29.784 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.761+0000 7f15b1c64700 1 -- 172.21.15.47:0/287787642 >> 172.21.15.47:0/287787642 conn(0x7f15ac060e00 msgr2=0x7f15ac0611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:29.784 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.761+0000 7f15b1c64700 1 -- 172.21.15.47:0/287787642 shutdown_connections 2022-06-29T15:49:29.784 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.761+0000 7f15b1c64700 1 -- 172.21.15.47:0/287787642 wait complete. 2022-06-29T15:49:29.784 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.761+0000 7f15b1c64700 1 Processor -- start 2022-06-29T15:49:29.784 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.761+0000 7f15b1c64700 1 -- start start 2022-06-29T15:49:29.785 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.761+0000 7f15b1c64700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15ac05c850 0x7f15ac1531b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:29.785 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.761+0000 7f15b1c64700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15ac0676a0 0x7f15ac0b3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:29.785 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.761+0000 7f15b1c64700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f15ac0b5d90 0x7f15ac13f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:29.785 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.761+0000 7f15b1c64700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ac0b9fa0 con 0x7f15ac0676a0 2022-06-29T15:49:29.786 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.761+0000 7f15b1c64700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ac12bf80 con 0x7f15ac05c850 2022-06-29T15:49:29.786 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.761+0000 7f15b1c64700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15ac1536a0 con 0x7f15ac0b5d90 2022-06-29T15:49:29.786 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.761+0000 7f15abfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15ac0676a0 0x7f15ac0b3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:29.787 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.761+0000 7f15abfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15ac0676a0 0x7f15ac0b3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50820/0 (socket says 172.21.15.47:50820) 2022-06-29T15:49:29.787 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.761+0000 7f15abfff700 1 -- 172.21.15.47:0/2960702132 learned_addr learned my addr 172.21.15.47:0/2960702132 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:29.787 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.761+0000 7f15b0c62700 1 --2- 172.21.15.47:0/2960702132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15ac05c850 0x7f15ac1531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:29.787 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.761+0000 7f15b1463700 1 --2- 172.21.15.47:0/2960702132 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f15ac0b5d90 0x7f15ac13f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:29.788 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.761+0000 7f15abfff700 1 -- 172.21.15.47:0/2960702132 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f15ac0b5d90 msgr2=0x7f15ac13f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:29.788 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.761+0000 7f15abfff700 1 --2- 172.21.15.47:0/2960702132 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f15ac0b5d90 0x7f15ac13f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:29.788 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.761+0000 7f15abfff700 1 -- 172.21.15.47:0/2960702132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15ac05c850 msgr2=0x7f15ac1531b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:29.788 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.761+0000 7f15abfff700 1 --2- 172.21.15.47:0/2960702132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15ac05c850 0x7f15ac1531b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:29.789 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.761+0000 7f15abfff700 1 -- 172.21.15.47:0/2960702132 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a400b1e0 con 0x7f15ac0676a0 2022-06-29T15:49:29.789 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.765+0000 7f15abfff700 1 --2- 172.21.15.47:0/2960702132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15ac0676a0 0x7f15ac0b3a30 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 rx=0x7f15ac0b8e60 tx=0x7f1594007ba0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:29.789 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.765+0000 7f15a9ffb700 1 -- 172.21.15.47:0/2960702132 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1594007030 con 0x7f15ac0676a0 2022-06-29T15:49:29.789 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.765+0000 7f15a9ffb700 1 -- 172.21.15.47:0/2960702132 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1594007190 con 0x7f15ac0676a0 2022-06-29T15:49:29.790 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.765+0000 7f15b1c64700 1 -- 172.21.15.47:0/2960702132 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15ac0b1000 con 0x7f15ac0676a0 2022-06-29T15:49:29.790 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.765+0000 7f15a9ffb700 1 -- 172.21.15.47:0/2960702132 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1594006510 con 0x7f15ac0676a0 2022-06-29T15:49:29.791 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.765+0000 7f15b1c64700 1 -- 172.21.15.47:0/2960702132 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15ac0b9d00 con 0x7f15ac0676a0 2022-06-29T15:49:29.791 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.765+0000 7f15b1c64700 1 -- 172.21.15.47:0/2960702132 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15ac0b41c0 con 0x7f15ac0676a0 2022-06-29T15:49:29.791 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.765+0000 7f15a9ffb700 1 -- 172.21.15.47:0/2960702132 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f159401b070 con 0x7f15ac0676a0 2022-06-29T15:49:29.792 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.765+0000 7f15a9ffb700 1 --2- 172.21.15.47:0/2960702132 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1590053b30 0x7f1590055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:29.792 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.765+0000 7f15a9ffb700 1 -- 172.21.15.47:0/2960702132 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f159406b640 con 0x7f15ac0676a0 2022-06-29T15:49:29.792 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.765+0000 7f15b0c62700 1 --2- 172.21.15.47:0/2960702132 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1590053b30 0x7f1590055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:29.792 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.765+0000 7f15b0c62700 1 --2- 172.21.15.47:0/2960702132 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1590053b30 0x7f1590055ff0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f159c00b000 tx=0x7f159c008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:29.793 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.773+0000 7f15a9ffb700 1 -- 172.21.15.47:0/2960702132 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1594042400 con 0x7f15ac0676a0 2022-06-29T15:49:29.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.941+0000 7f15b1c64700 1 -- 172.21.15.47:0/2960702132 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f15ac0b1190 con 0x7f15ac0676a0 2022-06-29T15:49:29.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.949+0000 7f15a9ffb700 1 -- 172.21.15.47:0/2960702132 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v25) v1 ==== 184+0+0 (secure 0 0 0) 0x7f1594045a20 con 0x7f15ac0676a0 2022-06-29T15:49:29.958 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.949+0000 7f15b1c64700 1 -- 172.21.15.47:0/2960702132 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1590053b30 msgr2=0x7f1590055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:29.958 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.949+0000 7f15b1c64700 1 --2- 172.21.15.47:0/2960702132 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1590053b30 0x7f1590055ff0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f159c00b000 tx=0x7f159c008040).stop 2022-06-29T15:49:29.958 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.949+0000 7f15b1c64700 1 -- 172.21.15.47:0/2960702132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15ac0676a0 msgr2=0x7f15ac0b3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:29.959 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.949+0000 7f15b1c64700 1 --2- 172.21.15.47:0/2960702132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15ac0676a0 0x7f15ac0b3a30 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 rx=0x7f15ac0b8e60 tx=0x7f1594007ba0).stop 2022-06-29T15:49:29.960 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.949+0000 7f15b1c64700 1 -- 172.21.15.47:0/2960702132 shutdown_connections 2022-06-29T15:49:29.960 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.949+0000 7f15b1c64700 1 --2- 172.21.15.47:0/2960702132 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f15ac0b5d90 0x7f15ac13f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:29.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.949+0000 7f15b1c64700 1 --2- 172.21.15.47:0/2960702132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15ac0676a0 0x7f15ac0b3a30 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:29.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.949+0000 7f15b1c64700 1 --2- 172.21.15.47:0/2960702132 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1590053b30 0x7f1590055ff0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:29.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.949+0000 7f15b1c64700 1 --2- 172.21.15.47:0/2960702132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15ac05c850 0x7f15ac1531b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:29.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.953+0000 7f15b1c64700 1 -- 172.21.15.47:0/2960702132 >> 172.21.15.47:0/2960702132 conn(0x7f15ac060e00 msgr2=0x7f15ac0af3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:29.962 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.953+0000 7f15b1c64700 1 -- 172.21.15.47:0/2960702132 shutdown_connections 2022-06-29T15:49:29.962 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:29.953+0000 7f15b1c64700 1 -- 172.21.15.47:0/2960702132 wait complete. 2022-06-29T15:49:29.962 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.1 2022-06-29T15:49:29.972 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:49:30.077 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.069+0000 7f00affff700 1 Processor -- start 2022-06-29T15:49:30.078 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.069+0000 7f00affff700 1 -- start start 2022-06-29T15:49:30.082 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.069+0000 7f00affff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f00b00b2690 0x7f00b00b5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:30.082 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.069+0000 7f00affff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f00b00b5e10 0x7f00b0140350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:30.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.069+0000 7f00affff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00b00b0be0 con 0x7f00b00b1720 2022-06-29T15:49:30.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.069+0000 7f00affff700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00b013dac0 con 0x7f00b00b2690 2022-06-29T15:49:30.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.069+0000 7f00affff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00b00b0e80 con 0x7f00b00b5e10 2022-06-29T15:49:30.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.069+0000 7f00aeffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f00b00b2690 0x7f00b00b5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:30.084 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.069+0000 7f00aeffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f00b00b2690 0x7f00b00b5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50828/0 (socket says 172.21.15.47:50828) 2022-06-29T15:49:30.084 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.069+0000 7f00aeffd700 1 -- 172.21.15.47:0/2211610752 learned_addr learned my addr 172.21.15.47:0/2211610752 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:30.084 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.069+0000 7f00a7fff700 1 --2- 172.21.15.47:0/2211610752 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f00b00b5e10 0x7f00b0140350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:30.084 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.073+0000 7f00aeffd700 1 -- 172.21.15.47:0/2211610752 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f00b00b5e10 msgr2=0x7f00b0140350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:30.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.073+0000 7f00aeffd700 1 --2- 172.21.15.47:0/2211610752 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f00b00b5e10 0x7f00b0140350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:30.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.073+0000 7f00aeffd700 1 -- 172.21.15.47:0/2211610752 >> v1:172.21.15.64:6789/0 conn(0x7f00b00b1720 legacy=0x7f00b0143370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:30.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.073+0000 7f00aeffd700 1 -- 172.21.15.47:0/2211610752 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00b00b1120 con 0x7f00b00b2690 2022-06-29T15:49:30.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.073+0000 7f00aeffd700 1 --2- 172.21.15.47:0/2211610752 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00b00b2690 0x7f00b00b5920 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 rx=0x7f00a0002f70 tx=0x7f00a0006eb0).ready entity=mon.0 client_cookie=cb62ef77eb2701cd server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:30.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.073+0000 7f00ae7fc700 1 -- 172.21.15.47:0/2211610752 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f00a000d5c0 con 0x7f00b00b2690 2022-06-29T15:49:30.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.073+0000 7f00ae7fc700 1 -- 172.21.15.47:0/2211610752 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00a0000e90 con 0x7f00b00b2690 2022-06-29T15:49:30.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.073+0000 7f00ae7fc700 1 -- 172.21.15.47:0/2211610752 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f00a000d950 con 0x7f00b00b2690 2022-06-29T15:49:30.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.073+0000 7f00affff700 1 -- 172.21.15.47:0/2211610752 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00b00b2690 msgr2=0x7f00b00b5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:30.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.073+0000 7f00affff700 1 --2- 172.21.15.47:0/2211610752 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00b00b2690 0x7f00b00b5920 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 rx=0x7f00a0002f70 tx=0x7f00a0006eb0).stop 2022-06-29T15:49:30.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.073+0000 7f00affff700 1 -- 172.21.15.47:0/2211610752 shutdown_connections 2022-06-29T15:49:30.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.073+0000 7f00affff700 1 --2- 172.21.15.47:0/2211610752 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f00b00b5e10 0x7f00b0140350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:30.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.073+0000 7f00affff700 1 --2- 172.21.15.47:0/2211610752 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00b00b2690 0x7f00b00b5920 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:30.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.073+0000 7f00affff700 1 -- 172.21.15.47:0/2211610752 >> 172.21.15.47:0/2211610752 conn(0x7f00b0062560 msgr2=0x7f00b0062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:30.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.073+0000 7f00affff700 1 -- 172.21.15.47:0/2211610752 shutdown_connections 2022-06-29T15:49:30.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.073+0000 7f00affff700 1 -- 172.21.15.47:0/2211610752 wait complete. 2022-06-29T15:49:30.095 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.077+0000 7f00affff700 1 Processor -- start 2022-06-29T15:49:30.096 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.077+0000 7f00affff700 1 -- start start 2022-06-29T15:49:30.096 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.077+0000 7f00affff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f00b00b1720 0x7f00b0066660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:30.096 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.077+0000 7f00affff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00b00b2690 0x7f00b0143960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:30.097 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.077+0000 7f00affff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00b00a9610 0x7f00b00a9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:30.097 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.077+0000 7f00affff700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b00a9f90 con 0x7f00b00b2690 2022-06-29T15:49:30.097 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.077+0000 7f00affff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b00ab520 con 0x7f00b00a9610 2022-06-29T15:49:30.097 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.077+0000 7f00affff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00b00ab750 con 0x7f00b00b1720 2022-06-29T15:49:30.097 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.077+0000 7f00a7fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00b00b2690 0x7f00b0143960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:30.098 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.077+0000 7f00a7fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00b00b2690 0x7f00b0143960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50836/0 (socket says 172.21.15.47:50836) 2022-06-29T15:49:30.098 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.077+0000 7f00a7fff700 1 -- 172.21.15.47:0/3367563087 learned_addr learned my addr 172.21.15.47:0/3367563087 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:30.098 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.077+0000 7f00aeffd700 1 --2- 172.21.15.47:0/3367563087 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f00b00b1720 0x7f00b0066660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:30.099 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.077+0000 7f00af7fe700 1 --2- 172.21.15.47:0/3367563087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00b00a9610 0x7f00b00a9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:30.099 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.077+0000 7f00a7fff700 1 -- 172.21.15.47:0/3367563087 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f00b00b1720 msgr2=0x7f00b0066660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:30.099 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.077+0000 7f00a7fff700 1 --2- 172.21.15.47:0/3367563087 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f00b00b1720 0x7f00b0066660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:30.099 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.077+0000 7f00a7fff700 1 -- 172.21.15.47:0/3367563087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00b00a9610 msgr2=0x7f00b00a9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:30.100 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.077+0000 7f00a7fff700 1 --2- 172.21.15.47:0/3367563087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00b00a9610 0x7f00b00a9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:30.100 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.077+0000 7f00a7fff700 1 -- 172.21.15.47:0/3367563087 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a000b1e0 con 0x7f00b00b2690 2022-06-29T15:49:30.100 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.077+0000 7f00aeffd700 1 --2- 172.21.15.47:0/3367563087 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f00b00b1720 0x7f00b0066660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:49:30.100 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.077+0000 7f00a7fff700 1 --2- 172.21.15.47:0/3367563087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00b00b2690 0x7f00b0143960 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 rx=0x7f009800c000 tx=0x7f0098006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:30.101 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.077+0000 7f00acff9700 1 -- 172.21.15.47:0/3367563087 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0098010780 con 0x7f00b00b2690 2022-06-29T15:49:30.101 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.077+0000 7f00acff9700 1 -- 172.21.15.47:0/3367563087 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00980108e0 con 0x7f00b00b2690 2022-06-29T15:49:30.101 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.077+0000 7f00affff700 1 -- 172.21.15.47:0/3367563087 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00b00ab9f0 con 0x7f00b00b2690 2022-06-29T15:49:30.101 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.077+0000 7f00acff9700 1 -- 172.21.15.47:0/3367563087 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0098010c40 con 0x7f00b00b2690 2022-06-29T15:49:30.102 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.077+0000 7f00affff700 1 -- 172.21.15.47:0/3367563087 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00b0066400 con 0x7f00b00b2690 2022-06-29T15:49:30.102 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.081+0000 7f00acff9700 1 -- 172.21.15.47:0/3367563087 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f009801c460 con 0x7f00b00b2690 2022-06-29T15:49:30.102 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.081+0000 7f00affff700 1 -- 172.21.15.47:0/3367563087 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00b00573d0 con 0x7f00b00b2690 2022-06-29T15:49:30.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.081+0000 7f00acff9700 1 --2- 172.21.15.47:0/3367563087 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0088053950 0x7f0088055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:30.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.081+0000 7f00acff9700 1 -- 172.21.15.47:0/3367563087 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f009806c610 con 0x7f00b00b2690 2022-06-29T15:49:30.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.081+0000 7f00aeffd700 1 --2- 172.21.15.47:0/3367563087 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0088053950 0x7f0088055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:30.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.081+0000 7f00aeffd700 1 --2- 172.21.15.47:0/3367563087 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0088053950 0x7f0088055e10 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7f00a0018000 tx=0x7f00a0014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:30.104 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.085+0000 7f00acff9700 1 -- 172.21.15.47:0/3367563087 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0098043260 con 0x7f00b00b2690 2022-06-29T15:49:30.255 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.249+0000 7f00affff700 1 -- 172.21.15.47:0/3367563087 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f00b0069e20 con 0x7f00b00b2690 2022-06-29T15:49:30.256 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.249+0000 7f00acff9700 1 -- 172.21.15.47:0/3367563087 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+2493 (secure 0 0 0) 0x7f0098046880 con 0x7f00b00b2690 2022-06-29T15:49:30.257 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:49:30.257 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":40,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6577,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/4286932616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":4286932616},{"type":"v1","addr":"172.21.15.64:6833","nonce":4286932616}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":38},{"gid":6657,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3528102962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3528102962},{"type":"v1","addr":"172.21.15.47:6835","nonce":3528102962}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":38}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:49:27.717245+0000","modified":"2022-06-29T15:49:28.719969+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6660},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6660":{"gid":6660,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6837/4146905199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":4146905199},{"type":"v1","addr":"172.21.15.47:6837","nonce":4146905199}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2022-06-29T15:49:30.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.249+0000 7f00affff700 1 -- 172.21.15.47:0/3367563087 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0088053950 msgr2=0x7f0088055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:30.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.249+0000 7f00affff700 1 --2- 172.21.15.47:0/3367563087 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0088053950 0x7f0088055e10 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7f00a0018000 tx=0x7f00a0014000).stop 2022-06-29T15:49:30.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.249+0000 7f00affff700 1 -- 172.21.15.47:0/3367563087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00b00b2690 msgr2=0x7f00b0143960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:30.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.249+0000 7f00affff700 1 --2- 172.21.15.47:0/3367563087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00b00b2690 0x7f00b0143960 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 rx=0x7f009800c000 tx=0x7f0098006df0).stop 2022-06-29T15:49:30.260 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.249+0000 7f00affff700 1 -- 172.21.15.47:0/3367563087 shutdown_connections 2022-06-29T15:49:30.260 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.249+0000 7f00affff700 1 --2- 172.21.15.47:0/3367563087 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0088053950 0x7f0088055e10 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:30.260 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.249+0000 7f00affff700 1 --2- 172.21.15.47:0/3367563087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00b00a9610 0x7f00b00a9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:30.260 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.249+0000 7f00affff700 1 --2- 172.21.15.47:0/3367563087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00b00b2690 0x7f00b0143960 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:30.261 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.249+0000 7f00affff700 1 --2- 172.21.15.47:0/3367563087 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f00b00b1720 0x7f00b0066660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:30.261 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.249+0000 7f00affff700 1 -- 172.21.15.47:0/3367563087 >> 172.21.15.47:0/3367563087 conn(0x7f00b0062560 msgr2=0x7f00b0140270 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:30.261 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.249+0000 7f00affff700 1 -- 172.21.15.47:0/3367563087 shutdown_connections 2022-06-29T15:49:30.261 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.249+0000 7f00affff700 1 -- 172.21.15.47:0/3367563087 wait complete. 2022-06-29T15:49:30.262 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 40 2022-06-29T15:49:30.572 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 40, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-29T15:49:27.717245+0000', 'modified': '2022-06-29T15:49:28.719969+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6660}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6660': {'gid': 6660, 'name': 'a', 'rank': 0, 'incarnation': 39, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.47:6837/4146905199', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6836', 'nonce': 4146905199}, {'type': 'v1', 'addr': '172.21.15.47:6837', 'nonce': 4146905199}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-29T15:49:30.572 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-29T15:49:30.573 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-06-29T15:49:30.692 INFO:teuthology.orchestra.run.smithi047.stdout:{"cluster_fsid":"9a68894a-53ce-400e-838b-14eab91b834c","whoami":0,"id":6660,"want_state":"up:active","state":"up:active","rank_uptime":2.9580964999999999,"mdsmap_epoch":40,"osdmap_epoch":90,"osdmap_epoch_barrier":89,"uptime":7.6485027949999997} 2022-06-29T15:49:30.692 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9a68894a-53ce-400e-838b-14eab91b834c", "id": 6660, "mdsmap_epoch": 40, "osdmap_epoch": 90, "osdmap_epoch_barrier": 89, "rank_uptime": 2.9580965, "state": "up:active", "uptime": 7.648502795, "want_state": "up:active", "whoami": 0 } 2022-06-29T15:49:30.694 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:49:30.799 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.793+0000 7f375813a700 1 Processor -- start 2022-06-29T15:49:30.800 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.793+0000 7f375813a700 1 -- start start 2022-06-29T15:49:30.803 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.793+0000 7f375813a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f37500b2120 0x7f37500b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:30.803 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.793+0000 7f375813a700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f375005f0b0 0x7f375014b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:30.803 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.793+0000 7f375813a700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37500b0740 con 0x7f37500b1260 2022-06-29T15:49:30.803 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.793+0000 7f375813a700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3750147460 con 0x7f37500b2120 2022-06-29T15:49:30.804 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.793+0000 7f375813a700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37500b0970 con 0x7f375005f0b0 2022-06-29T15:49:30.804 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.793+0000 7f3755ed6700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f37500b2120 0x7f37500b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:30.804 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.793+0000 7f3755ed6700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f37500b2120 0x7f37500b2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50844/0 (socket says 172.21.15.47:50844) 2022-06-29T15:49:30.804 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.793+0000 7f3755ed6700 1 -- 172.21.15.47:0/3815007142 learned_addr learned my addr 172.21.15.47:0/3815007142 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:30.804 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.793+0000 7f37556d5700 1 --2- 172.21.15.47:0/3815007142 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f375005f0b0 0x7f375014b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:30.805 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.793+0000 7f3755ed6700 1 -- 172.21.15.47:0/3815007142 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f375005f0b0 msgr2=0x7f375014b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:30.805 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.793+0000 7f3755ed6700 1 --2- 172.21.15.47:0/3815007142 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f375005f0b0 0x7f375014b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:30.805 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.793+0000 7f3755ed6700 1 -- 172.21.15.47:0/3815007142 >> v1:172.21.15.64:6789/0 conn(0x7f37500b1260 legacy=0x7f3750147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:30.805 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.793+0000 7f3755ed6700 1 -- 172.21.15.47:0/3815007142 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37500b0c10 con 0x7f37500b2120 2022-06-29T15:49:30.806 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.793+0000 7f3754ed4700 1 -- 172.21.15.47:0/3815007142 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3973969651 0 0) 0x7f37500b0740 con 0x7f37500b1260 2022-06-29T15:49:30.806 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.793+0000 7f3755ed6700 1 --2- 172.21.15.47:0/3815007142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37500b2120 0x7f37500b2500 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 rx=0x7f3744002f70 tx=0x7f3744006eb0).ready entity=mon.0 client_cookie=fd4afd89176e7c33 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:30.806 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.793+0000 7f3754ed4700 1 -- 172.21.15.47:0/3815007142 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f374400d570 con 0x7f37500b2120 2022-06-29T15:49:30.806 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.793+0000 7f3754ed4700 1 -- 172.21.15.47:0/3815007142 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3744000e90 con 0x7f37500b2120 2022-06-29T15:49:30.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.793+0000 7f3754ed4700 1 -- 172.21.15.47:0/3815007142 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f374400d8c0 con 0x7f37500b2120 2022-06-29T15:49:30.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.793+0000 7f375813a700 1 -- 172.21.15.47:0/3815007142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37500b2120 msgr2=0x7f37500b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:30.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.793+0000 7f375813a700 1 --2- 172.21.15.47:0/3815007142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37500b2120 0x7f37500b2500 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 rx=0x7f3744002f70 tx=0x7f3744006eb0).stop 2022-06-29T15:49:30.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.793+0000 7f375813a700 1 -- 172.21.15.47:0/3815007142 shutdown_connections 2022-06-29T15:49:30.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.793+0000 7f375813a700 1 --2- 172.21.15.47:0/3815007142 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f375005f0b0 0x7f375014b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:30.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.793+0000 7f375813a700 1 --2- 172.21.15.47:0/3815007142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37500b2120 0x7f37500b2500 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:30.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.793+0000 7f375813a700 1 -- 172.21.15.47:0/3815007142 >> 172.21.15.47:0/3815007142 conn(0x7f37500626f0 msgr2=0x7f3750062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:30.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.793+0000 7f375813a700 1 -- 172.21.15.47:0/3815007142 shutdown_connections 2022-06-29T15:49:30.809 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.797+0000 7f375813a700 1 -- 172.21.15.47:0/3815007142 wait complete. 2022-06-29T15:49:30.813 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.797+0000 7f375813a700 1 Processor -- start 2022-06-29T15:49:30.813 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.797+0000 7f375813a700 1 -- start start 2022-06-29T15:49:30.813 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.797+0000 7f375813a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37500b1260 0x7f37500b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:30.813 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.797+0000 7f375813a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37500b2120 0x7f37501435c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:30.814 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.797+0000 7f375813a700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f37500adcb0 0x7f37500af310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:30.814 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.797+0000 7f375813a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37501422a0 con 0x7f37500b1260 2022-06-29T15:49:30.814 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.797+0000 7f375813a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3750141b00 con 0x7f37500b2120 2022-06-29T15:49:30.815 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.797+0000 7f375813a700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3750143ab0 con 0x7f37500adcb0 2022-06-29T15:49:30.815 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.797+0000 7f3755ed6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37500b1260 0x7f37500b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:30.815 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.797+0000 7f3755ed6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37500b1260 0x7f37500b3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50848/0 (socket says 172.21.15.47:50848) 2022-06-29T15:49:30.815 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.797+0000 7f3755ed6700 1 -- 172.21.15.47:0/1326238076 learned_addr learned my addr 172.21.15.47:0/1326238076 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:30.816 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.797+0000 7f37566d7700 1 --2- 172.21.15.47:0/1326238076 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f37500adcb0 0x7f37500af310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:30.816 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.797+0000 7f37556d5700 1 --2- 172.21.15.47:0/1326238076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37500b2120 0x7f37501435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:30.816 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.797+0000 7f3755ed6700 1 -- 172.21.15.47:0/1326238076 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f37500adcb0 msgr2=0x7f37500af310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:30.816 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.797+0000 7f3755ed6700 1 --2- 172.21.15.47:0/1326238076 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f37500adcb0 0x7f37500af310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:30.817 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.797+0000 7f3755ed6700 1 -- 172.21.15.47:0/1326238076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37500b2120 msgr2=0x7f37501435c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:30.817 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.797+0000 7f3755ed6700 1 --2- 172.21.15.47:0/1326238076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37500b2120 0x7f37501435c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:30.817 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.797+0000 7f3755ed6700 1 -- 172.21.15.47:0/1326238076 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f374400b1e0 con 0x7f37500b1260 2022-06-29T15:49:30.817 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.797+0000 7f3755ed6700 1 --2- 172.21.15.47:0/1326238076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37500b1260 0x7f37500b3cc0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 rx=0x7f3744002f70 tx=0x7f3744015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:30.818 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.797+0000 7f374effd700 1 -- 172.21.15.47:0/1326238076 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f37440068b0 con 0x7f37500b1260 2022-06-29T15:49:30.818 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.797+0000 7f374effd700 1 -- 172.21.15.47:0/1326238076 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f374400d8f0 con 0x7f37500b1260 2022-06-29T15:49:30.818 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.797+0000 7f374effd700 1 -- 172.21.15.47:0/1326238076 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f374401e460 con 0x7f37500b1260 2022-06-29T15:49:30.818 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.797+0000 7f375813a700 1 -- 172.21.15.47:0/1326238076 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3750141d60 con 0x7f37500b1260 2022-06-29T15:49:30.818 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.797+0000 7f375813a700 1 -- 172.21.15.47:0/1326238076 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37500af800 con 0x7f37500b1260 2022-06-29T15:49:30.819 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.801+0000 7f375813a700 1 -- 172.21.15.47:0/1326238076 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37501442c0 con 0x7f37500b1260 2022-06-29T15:49:30.819 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.801+0000 7f374effd700 1 -- 172.21.15.47:0/1326238076 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f374401a040 con 0x7f37500b1260 2022-06-29T15:49:30.819 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.801+0000 7f374effd700 1 --2- 172.21.15.47:0/1326238076 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3730053a70 0x7f3730055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:30.819 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.801+0000 7f374effd700 1 -- 172.21.15.47:0/1326238076 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f374406cf40 con 0x7f37500b1260 2022-06-29T15:49:30.820 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.801+0000 7f37556d5700 1 --2- 172.21.15.47:0/1326238076 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3730053a70 0x7f3730055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:30.820 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.801+0000 7f37556d5700 1 --2- 172.21.15.47:0/1326238076 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3730053a70 0x7f3730055f30 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7f374800b000 tx=0x7f3748008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:30.821 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.809+0000 7f374effd700 1 -- 172.21.15.47:0/1326238076 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3744043d80 con 0x7f37500b1260 2022-06-29T15:49:30.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.941+0000 7f375813a700 1 -- 172.21.15.47:0/1326238076 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f37500573d0 con 0x7f37500b1260 2022-06-29T15:49:30.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.941+0000 7f374effd700 1 -- 172.21.15.47:0/1326238076 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+16274 (secure 0 0 0) 0x7f3744023070 con 0x7f37500b1260 2022-06-29T15:49:30.950 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:49:30.950 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":90,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:49:28.720705+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:49:25.785974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2022-06-29T15:49:26.726660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:49:30.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.945+0000 7f375813a700 1 -- 172.21.15.47:0/1326238076 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3730053a70 msgr2=0x7f3730055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:30.953 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.945+0000 7f375813a700 1 --2- 172.21.15.47:0/1326238076 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3730053a70 0x7f3730055f30 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7f374800b000 tx=0x7f3748008040).stop 2022-06-29T15:49:30.953 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.945+0000 7f375813a700 1 -- 172.21.15.47:0/1326238076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37500b1260 msgr2=0x7f37500b3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:30.953 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.945+0000 7f375813a700 1 --2- 172.21.15.47:0/1326238076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37500b1260 0x7f37500b3cc0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 rx=0x7f3744002f70 tx=0x7f3744015000).stop 2022-06-29T15:49:30.953 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.945+0000 7f375813a700 1 -- 172.21.15.47:0/1326238076 shutdown_connections 2022-06-29T15:49:30.954 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.945+0000 7f375813a700 1 --2- 172.21.15.47:0/1326238076 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f37500adcb0 0x7f37500af310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:30.954 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.945+0000 7f375813a700 1 --2- 172.21.15.47:0/1326238076 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3730053a70 0x7f3730055f30 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:30.954 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.945+0000 7f375813a700 1 --2- 172.21.15.47:0/1326238076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37500b2120 0x7f37501435c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:30.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.945+0000 7f375813a700 1 --2- 172.21.15.47:0/1326238076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37500b1260 0x7f37500b3cc0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:30.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.945+0000 7f375813a700 1 -- 172.21.15.47:0/1326238076 >> 172.21.15.47:0/1326238076 conn(0x7f37500626f0 msgr2=0x7f375014a820 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:30.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.945+0000 7f375813a700 1 -- 172.21.15.47:0/1326238076 shutdown_connections 2022-06-29T15:49:30.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:30.945+0000 7f375813a700 1 -- 172.21.15.47:0/1326238076 wait complete. 2022-06-29T15:49:30.968 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-29T15:42:16.971431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-29T15:42:25.290884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-29T15:49:25.785974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2022-06-29T15:49:26.726660+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-29T15:49:30.970 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-29T15:49:31.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.073+0000 7f85dffff700 1 Processor -- start 2022-06-29T15:49:31.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.073+0000 7f85dffff700 1 -- start start 2022-06-29T15:49:31.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.077+0000 7f85dffff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f85e01409b0 0x7f85e0061620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:31.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.077+0000 7f85dffff700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85e00af660 con 0x7f85e005fd00 2022-06-29T15:49:31.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.077+0000 7f85dffff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85e01437e0 con 0x7f85e0060a40 2022-06-29T15:49:31.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.077+0000 7f85dffff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85e0143fc0 con 0x7f85e01409b0 2022-06-29T15:49:31.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.077+0000 7f85d7fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f85e005fd00 0x7f85e01402a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36410/0 (socket says 172.21.15.47:36410) 2022-06-29T15:49:31.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.077+0000 7f85d7fff700 1 -- 172.21.15.47:0/3367100280 learned_addr learned my addr 172.21.15.47:0/3367100280 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:49:31.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.077+0000 7f85df7fe700 1 --2- 172.21.15.47:0/3367100280 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f85e01409b0 0x7f85e0061620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:31.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.077+0000 7f85de7fc700 1 -- 172.21.15.47:0/3367100280 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3413698575 0 0) 0x7f85e00af660 con 0x7f85e005fd00 2022-06-29T15:49:31.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.077+0000 7f85de7fc700 1 -- 172.21.15.47:0/3367100280 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85c4003260 con 0x7f85e005fd00 2022-06-29T15:49:31.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.077+0000 7f85de7fc700 1 -- 172.21.15.47:0/3367100280 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1148046832 0 0) 0x7f85e01437e0 con 0x7f85e0060a40 2022-06-29T15:49:31.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.077+0000 7f85df7fe700 1 -- 172.21.15.47:0/3367100280 >> v1:172.21.15.64:6789/0 conn(0x7f85e0060a40 legacy=0x7f85e0142ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:31.089 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.077+0000 7f85df7fe700 1 -- 172.21.15.47:0/3367100280 >> v1:172.21.15.47:6789/0 conn(0x7f85e005fd00 legacy=0x7f85e01402a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:31.089 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.077+0000 7f85df7fe700 1 -- 172.21.15.47:0/3367100280 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e0143a80 con 0x7f85e01409b0 2022-06-29T15:49:31.089 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.077+0000 7f85df7fe700 1 --2- 172.21.15.47:0/3367100280 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f85e01409b0 0x7f85e0061620 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f85c8001800 tx=0x7f85d80091d0).ready entity=mon.2 client_cookie=176224827b0daedc server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:31.089 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.077+0000 7f85de7fc700 1 -- 172.21.15.47:0/3367100280 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85c8001660 con 0x7f85e01409b0 2022-06-29T15:49:31.089 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.077+0000 7f85de7fc700 1 -- 172.21.15.47:0/3367100280 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85d800b580 con 0x7f85e01409b0 2022-06-29T15:49:31.090 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.077+0000 7f85de7fc700 1 -- 172.21.15.47:0/3367100280 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85d800b6f0 con 0x7f85e01409b0 2022-06-29T15:49:31.093 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.077+0000 7f85dffff700 1 -- 172.21.15.47:0/3367100280 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f85e01409b0 msgr2=0x7f85e0061620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:31.094 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.077+0000 7f85dffff700 1 --2- 172.21.15.47:0/3367100280 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f85e01409b0 0x7f85e0061620 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f85c8001800 tx=0x7f85d80091d0).stop 2022-06-29T15:49:31.094 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.077+0000 7f85dffff700 1 -- 172.21.15.47:0/3367100280 shutdown_connections 2022-06-29T15:49:31.094 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.077+0000 7f85dffff700 1 --2- 172.21.15.47:0/3367100280 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f85e01409b0 0x7f85e0061620 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:31.094 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.077+0000 7f85dffff700 1 -- 172.21.15.47:0/3367100280 >> 172.21.15.47:0/3367100280 conn(0x7f85e00b6430 msgr2=0x7f85e00b6810 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:31.095 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.077+0000 7f85dffff700 1 -- 172.21.15.47:0/3367100280 shutdown_connections 2022-06-29T15:49:31.095 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.081+0000 7f85dffff700 1 -- 172.21.15.47:0/3367100280 wait complete. 2022-06-29T15:49:31.095 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.081+0000 7f85dffff700 1 Processor -- start 2022-06-29T15:49:31.095 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.081+0000 7f85dffff700 1 -- start start 2022-06-29T15:49:31.096 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.081+0000 7f85dffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85e005fd00 0x7f85e00b4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:31.096 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.081+0000 7f85dffff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f85e0060a40 0x7f85e0152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:31.096 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.081+0000 7f85dffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85e01409b0 0x7f85e013e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:31.096 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.081+0000 7f85dffff700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e01452f0 con 0x7f85e005fd00 2022-06-29T15:49:31.096 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.081+0000 7f85dffff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e0149e60 con 0x7f85e01409b0 2022-06-29T15:49:31.097 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.081+0000 7f85dffff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85e014a100 con 0x7f85e0060a40 2022-06-29T15:49:31.097 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.081+0000 7f85deffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85e005fd00 0x7f85e00b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:31.097 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.081+0000 7f85deffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85e005fd00 0x7f85e00b4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50862/0 (socket says 172.21.15.47:50862) 2022-06-29T15:49:31.097 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.081+0000 7f85deffd700 1 -- 172.21.15.47:0/856319128 learned_addr learned my addr 172.21.15.47:0/856319128 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:31.098 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.081+0000 7f85d7fff700 1 --2- 172.21.15.47:0/856319128 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f85e0060a40 0x7f85e0152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:31.098 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.081+0000 7f85df7fe700 1 --2- 172.21.15.47:0/856319128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85e01409b0 0x7f85e013e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:31.098 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.081+0000 7f85deffd700 1 -- 172.21.15.47:0/856319128 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f85e0060a40 msgr2=0x7f85e0152610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:31.098 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.081+0000 7f85deffd700 1 --2- 172.21.15.47:0/856319128 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f85e0060a40 0x7f85e0152610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:31.098 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.081+0000 7f85deffd700 1 -- 172.21.15.47:0/856319128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85e01409b0 msgr2=0x7f85e013e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:31.099 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.081+0000 7f85deffd700 1 --2- 172.21.15.47:0/856319128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85e01409b0 0x7f85e013e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:31.099 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.081+0000 7f85deffd700 1 -- 172.21.15.47:0/856319128 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85d8009470 con 0x7f85e005fd00 2022-06-29T15:49:31.099 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.085+0000 7f85deffd700 1 --2- 172.21.15.47:0/856319128 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85e005fd00 0x7f85e00b4350 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 rx=0x7f85d0000ed0 tx=0x7f85d0007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:31.099 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.085+0000 7f85dcff9700 1 -- 172.21.15.47:0/856319128 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85d0012400 con 0x7f85e005fd00 2022-06-29T15:49:31.100 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.085+0000 7f85dcff9700 1 -- 172.21.15.47:0/856319128 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85d0012560 con 0x7f85e005fd00 2022-06-29T15:49:31.100 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.085+0000 7f85dffff700 1 -- 172.21.15.47:0/856319128 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85e0151ed0 con 0x7f85e005fd00 2022-06-29T15:49:31.100 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.085+0000 7f85dcff9700 1 -- 172.21.15.47:0/856319128 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85d00128c0 con 0x7f85e005fd00 2022-06-29T15:49:31.100 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.085+0000 7f85dffff700 1 -- 172.21.15.47:0/856319128 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85e00ad000 con 0x7f85e005fd00 2022-06-29T15:49:31.100 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.085+0000 7f85dffff700 1 -- 172.21.15.47:0/856319128 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85e00573d0 con 0x7f85e005fd00 2022-06-29T15:49:31.101 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.085+0000 7f85dcff9700 1 -- 172.21.15.47:0/856319128 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f85d0019030 con 0x7f85e005fd00 2022-06-29T15:49:31.102 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.085+0000 7f85dcff9700 1 --2- 172.21.15.47:0/856319128 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f85b8053980 0x7f85b8055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:31.102 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.085+0000 7f85dcff9700 1 -- 172.21.15.47:0/856319128 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f85d006b310 con 0x7f85e005fd00 2022-06-29T15:49:31.102 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.085+0000 7f85d7fff700 1 --2- 172.21.15.47:0/856319128 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f85b8053980 0x7f85b8055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:31.102 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.089+0000 7f85d7fff700 1 --2- 172.21.15.47:0/856319128 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f85b8053980 0x7f85b8055e40 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7f85c800d000 tx=0x7f85c8000bc0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:31.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.093+0000 7f85dcff9700 1 -- 172.21.15.47:0/856319128 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f85d0042010 con 0x7f85e005fd00 2022-06-29T15:49:31.228 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.221+0000 7f85dffff700 1 -- 172.21.15.47:0/856319128 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f85e00b70c0 con 0x7f85e005fd00 2022-06-29T15:49:31.229 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.221+0000 7f85dcff9700 1 -- 172.21.15.47:0/856319128 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v90) v1 ==== 110+0+10 (secure 0 0 0) 0x7f85d0045630 con 0x7f85e005fd00 2022-06-29T15:49:31.229 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2022-06-29T15:49:31.231 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.225+0000 7f85dffff700 1 -- 172.21.15.47:0/856319128 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f85b8053980 msgr2=0x7f85b8055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:31.231 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.225+0000 7f85dffff700 1 --2- 172.21.15.47:0/856319128 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f85b8053980 0x7f85b8055e40 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7f85c800d000 tx=0x7f85c8000bc0).stop 2022-06-29T15:49:31.232 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.225+0000 7f85dffff700 1 -- 172.21.15.47:0/856319128 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85e005fd00 msgr2=0x7f85e00b4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:31.232 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.225+0000 7f85dffff700 1 --2- 172.21.15.47:0/856319128 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85e005fd00 0x7f85e00b4350 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 rx=0x7f85d0000ed0 tx=0x7f85d0007ac0).stop 2022-06-29T15:49:31.232 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.225+0000 7f85dffff700 1 -- 172.21.15.47:0/856319128 shutdown_connections 2022-06-29T15:49:31.233 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.225+0000 7f85dffff700 1 --2- 172.21.15.47:0/856319128 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f85b8053980 0x7f85b8055e40 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:31.233 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.225+0000 7f85dffff700 1 --2- 172.21.15.47:0/856319128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85e01409b0 0x7f85e013e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:31.233 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.225+0000 7f85dffff700 1 --2- 172.21.15.47:0/856319128 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f85e0060a40 0x7f85e0152610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:31.233 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.225+0000 7f85dffff700 1 --2- 172.21.15.47:0/856319128 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85e005fd00 0x7f85e00b4350 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:31.234 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.225+0000 7f85dffff700 1 -- 172.21.15.47:0/856319128 >> 172.21.15.47:0/856319128 conn(0x7f85e00b6430 msgr2=0x7f85e00b5bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:31.234 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.225+0000 7f85dffff700 1 -- 172.21.15.47:0/856319128 shutdown_connections 2022-06-29T15:49:31.234 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.225+0000 7f85dffff700 1 -- 172.21.15.47:0/856319128 wait complete. 2022-06-29T15:49:31.244 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-29T15:49:31.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.349+0000 7f615b7e5700 1 Processor -- start 2022-06-29T15:49:31.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.349+0000 7f615b7e5700 1 -- start start 2022-06-29T15:49:31.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.349+0000 7f615b7e5700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6154060ee0 0x7f61540612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:31.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.349+0000 7f615b7e5700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f61541471a0 0x7f6154066f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:31.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.349+0000 7f615b7e5700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6154067610 con 0x7f61540b4300 2022-06-29T15:49:31.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.349+0000 7f615b7e5700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61541467c0 con 0x7f6154060ee0 2022-06-29T15:49:31.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.349+0000 7f615b7e5700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6154146a60 con 0x7f61541471a0 2022-06-29T15:49:31.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.349+0000 7f6159581700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6154060ee0 0x7f61540612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:31.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.349+0000 7f6158d80700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f61540b4300 0x7f61541430e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36424/0 (socket says 172.21.15.47:36424) 2022-06-29T15:49:31.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.349+0000 7f6158d80700 1 -- 172.21.15.47:0/113987771 learned_addr learned my addr 172.21.15.47:0/113987771 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:49:31.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.353+0000 7f6159d82700 1 --2- 172.21.15.47:0/113987771 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f61541471a0 0x7f6154066f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:31.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.353+0000 7f613ffff700 1 -- 172.21.15.47:0/113987771 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 276766395 0 0) 0x7f6154067610 con 0x7f61540b4300 2022-06-29T15:49:31.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.353+0000 7f613ffff700 1 -- 172.21.15.47:0/113987771 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6138003260 con 0x7f61540b4300 2022-06-29T15:49:31.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.353+0000 7f6159581700 1 -- 172.21.15.47:0/113987771 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f61541471a0 msgr2=0x7f6154066f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:31.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.353+0000 7f6159581700 1 --2- 172.21.15.47:0/113987771 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f61541471a0 0x7f6154066f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:31.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.353+0000 7f6159581700 1 -- 172.21.15.47:0/113987771 >> v1:172.21.15.47:6789/0 conn(0x7f61540b4300 legacy=0x7f61541430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:31.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.353+0000 7f613ffff700 1 -- 172.21.15.47:0/113987771 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 88717886 0 0) 0x7f6138003260 con 0x7f61540b4300 2022-06-29T15:49:31.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.353+0000 7f6159581700 1 -- 172.21.15.47:0/113987771 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6154146d00 con 0x7f6154060ee0 2022-06-29T15:49:31.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.353+0000 7f6159d82700 1 --2- 172.21.15.47:0/113987771 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f61541471a0 0x7f6154066f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:49:31.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.353+0000 7f6159581700 1 --2- 172.21.15.47:0/113987771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6154060ee0 0x7f61540612c0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 rx=0x7f61540ae790 tx=0x7f6150006df0).ready entity=mon.0 client_cookie=2474687a9b1d8b9a server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:31.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.353+0000 7f613ffff700 1 -- 172.21.15.47:0/113987771 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6148001660 con 0x7f6154060ee0 2022-06-29T15:49:31.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.353+0000 7f613ffff700 1 -- 172.21.15.47:0/113987771 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6150000e90 con 0x7f6154060ee0 2022-06-29T15:49:31.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.353+0000 7f613ffff700 1 -- 172.21.15.47:0/113987771 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6150012460 con 0x7f6154060ee0 2022-06-29T15:49:31.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.353+0000 7f615b7e5700 1 -- 172.21.15.47:0/113987771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6154060ee0 msgr2=0x7f61540612c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:31.370 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.353+0000 7f615b7e5700 1 --2- 172.21.15.47:0/113987771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6154060ee0 0x7f61540612c0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 rx=0x7f61540ae790 tx=0x7f6150006df0).stop 2022-06-29T15:49:31.370 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.353+0000 7f615b7e5700 1 -- 172.21.15.47:0/113987771 shutdown_connections 2022-06-29T15:49:31.370 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.353+0000 7f615b7e5700 1 --2- 172.21.15.47:0/113987771 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f61541471a0 0x7f6154066f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:31.370 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.353+0000 7f615b7e5700 1 --2- 172.21.15.47:0/113987771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6154060ee0 0x7f61540612c0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:31.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.353+0000 7f615b7e5700 1 -- 172.21.15.47:0/113987771 >> 172.21.15.47:0/113987771 conn(0x7f615405c510 msgr2=0x7f615405f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:31.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.353+0000 7f615b7e5700 1 -- 172.21.15.47:0/113987771 shutdown_connections 2022-06-29T15:49:31.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.353+0000 7f615b7e5700 1 -- 172.21.15.47:0/113987771 wait complete. 2022-06-29T15:49:31.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.353+0000 7f615b7e5700 1 Processor -- start 2022-06-29T15:49:31.372 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.357+0000 7f615b7e5700 1 -- start start 2022-06-29T15:49:31.372 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.357+0000 7f615b7e5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6154060ee0 0x7f6154139dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:31.372 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.357+0000 7f615b7e5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61540b4300 0x7f61540b8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:31.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.357+0000 7f615b7e5700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f61541471a0 0x7f61540b9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:31.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.357+0000 7f615b7e5700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61540ba200 con 0x7f6154060ee0 2022-06-29T15:49:31.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.357+0000 7f615b7e5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61540b83c0 con 0x7f61540b4300 2022-06-29T15:49:31.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.357+0000 7f615b7e5700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61540b9c00 con 0x7f61541471a0 2022-06-29T15:49:31.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.357+0000 7f6159581700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6154060ee0 0x7f6154139dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:31.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.357+0000 7f6159581700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6154060ee0 0x7f6154139dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50876/0 (socket says 172.21.15.47:50876) 2022-06-29T15:49:31.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.357+0000 7f6159581700 1 -- 172.21.15.47:0/1978987831 learned_addr learned my addr 172.21.15.47:0/1978987831 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:31.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.357+0000 7f6159d82700 1 --2- 172.21.15.47:0/1978987831 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f61541471a0 0x7f61540b9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:31.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.357+0000 7f6158d80700 1 --2- 172.21.15.47:0/1978987831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61540b4300 0x7f61540b8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:31.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.357+0000 7f6159581700 1 -- 172.21.15.47:0/1978987831 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f61541471a0 msgr2=0x7f61540b9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:31.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.357+0000 7f6159581700 1 --2- 172.21.15.47:0/1978987831 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f61541471a0 0x7f61540b9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:31.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.357+0000 7f6159581700 1 -- 172.21.15.47:0/1978987831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61540b4300 msgr2=0x7f61540b8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:31.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.357+0000 7f6159581700 1 --2- 172.21.15.47:0/1978987831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61540b4300 0x7f61540b8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:31.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.357+0000 7f6159581700 1 -- 172.21.15.47:0/1978987831 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f614c009470 con 0x7f6154060ee0 2022-06-29T15:49:31.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.357+0000 7f6159581700 1 --2- 172.21.15.47:0/1978987831 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6154060ee0 0x7f6154139dc0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 rx=0x7f61540ae790 tx=0x7f6150014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:31.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.357+0000 7f6146ffd700 1 -- 172.21.15.47:0/1978987831 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6150016070 con 0x7f6154060ee0 2022-06-29T15:49:31.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.357+0000 7f6146ffd700 1 -- 172.21.15.47:0/1978987831 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6150013e20 con 0x7f6154060ee0 2022-06-29T15:49:31.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.357+0000 7f615b7e5700 1 -- 172.21.15.47:0/1978987831 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f615000b180 con 0x7f6154060ee0 2022-06-29T15:49:31.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.357+0000 7f6146ffd700 1 -- 172.21.15.47:0/1978987831 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6150002f50 con 0x7f6154060ee0 2022-06-29T15:49:31.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.357+0000 7f615b7e5700 1 -- 172.21.15.47:0/1978987831 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61540b7f20 con 0x7f6154060ee0 2022-06-29T15:49:31.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.361+0000 7f615b7e5700 1 -- 172.21.15.47:0/1978987831 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61540b78e0 con 0x7f6154060ee0 2022-06-29T15:49:31.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.361+0000 7f6146ffd700 1 -- 172.21.15.47:0/1978987831 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6150025030 con 0x7f6154060ee0 2022-06-29T15:49:31.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.361+0000 7f6146ffd700 1 --2- 172.21.15.47:0/1978987831 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6130053950 0x7f6130055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:31.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.361+0000 7f6146ffd700 1 -- 172.21.15.47:0/1978987831 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f615006d020 con 0x7f6154060ee0 2022-06-29T15:49:31.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.361+0000 7f6158d80700 1 --2- 172.21.15.47:0/1978987831 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6130053950 0x7f6130055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:31.379 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.361+0000 7f6158d80700 1 --2- 172.21.15.47:0/1978987831 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6130053950 0x7f6130055e10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f614800d000 tx=0x7f6148001660).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:31.379 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.369+0000 7f6146ffd700 1 -- 172.21.15.47:0/1978987831 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6150043de0 con 0x7f6154060ee0 2022-06-29T15:49:31.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.493+0000 7f615b7e5700 1 -- 172.21.15.47:0/1978987831 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f61540b80e0 con 0x7f6154060ee0 2022-06-29T15:49:31.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.497+0000 7f6146ffd700 1 -- 172.21.15.47:0/1978987831 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v90) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6150023070 con 0x7f6154060ee0 2022-06-29T15:49:31.503 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:49:31.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.497+0000 7f615b7e5700 1 -- 172.21.15.47:0/1978987831 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6130053950 msgr2=0x7f6130055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:31.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.497+0000 7f615b7e5700 1 --2- 172.21.15.47:0/1978987831 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6130053950 0x7f6130055e10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f614800d000 tx=0x7f6148001660).stop 2022-06-29T15:49:31.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.497+0000 7f615b7e5700 1 -- 172.21.15.47:0/1978987831 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6154060ee0 msgr2=0x7f6154139dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:31.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.497+0000 7f615b7e5700 1 --2- 172.21.15.47:0/1978987831 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6154060ee0 0x7f6154139dc0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 rx=0x7f61540ae790 tx=0x7f6150014000).stop 2022-06-29T15:49:31.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.497+0000 7f615b7e5700 1 -- 172.21.15.47:0/1978987831 shutdown_connections 2022-06-29T15:49:31.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.497+0000 7f615b7e5700 1 --2- 172.21.15.47:0/1978987831 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f61541471a0 0x7f61540b9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:31.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.497+0000 7f615b7e5700 1 --2- 172.21.15.47:0/1978987831 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6130053950 0x7f6130055e10 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:31.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.497+0000 7f615b7e5700 1 --2- 172.21.15.47:0/1978987831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61540b4300 0x7f61540b8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:31.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.497+0000 7f615b7e5700 1 --2- 172.21.15.47:0/1978987831 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6154060ee0 0x7f6154139dc0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:31.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.497+0000 7f615b7e5700 1 -- 172.21.15.47:0/1978987831 >> 172.21.15.47:0/1978987831 conn(0x7f615405c510 msgr2=0x7f615413d020 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:31.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.497+0000 7f615b7e5700 1 -- 172.21.15.47:0/1978987831 shutdown_connections 2022-06-29T15:49:31.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.497+0000 7f615b7e5700 1 -- 172.21.15.47:0/1978987831 wait complete. 2022-06-29T15:49:31.517 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-29T15:49:31.635 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.625+0000 7f947359e700 1 Processor -- start 2022-06-29T15:49:31.635 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.625+0000 7f947359e700 1 -- start start 2022-06-29T15:49:31.635 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.625+0000 7f947359e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f947405f4a0 0x7f94740b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:31.635 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.625+0000 7f947359e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9474143140 0x7f947409a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:31.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.625+0000 7f947359e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f947405f880 con 0x7f947409a1d0 2022-06-29T15:49:31.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.625+0000 7f947359e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f947409ab50 con 0x7f9474143140 2022-06-29T15:49:31.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.625+0000 7f947359e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f947409ae90 con 0x7f947405f4a0 2022-06-29T15:49:31.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.625+0000 7f9472d9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9474143140 0x7f947409a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:31.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.625+0000 7f9472d9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9474143140 0x7f947409a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50886/0 (socket says 172.21.15.47:50886) 2022-06-29T15:49:31.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.625+0000 7f9472d9d700 1 -- 172.21.15.47:0/599537666 learned_addr learned my addr 172.21.15.47:0/599537666 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:31.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.625+0000 7f9471d9b700 1 --1- 172.21.15.47:0/599537666 >> v1:172.21.15.47:6789/0 conn(0x7f947409a1d0 0x7f9474142a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36442/0 (socket says 172.21.15.47:36442) 2022-06-29T15:49:31.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.625+0000 7f947259c700 1 --2- 172.21.15.47:0/599537666 >> v2:172.21.15.64:3300/0 conn(0x7f947405f4a0 0x7f94740b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:31.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.625+0000 7f9472d9d700 1 -- 172.21.15.47:0/599537666 >> v2:172.21.15.64:3300/0 conn(0x7f947405f4a0 msgr2=0x7f94740b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:31.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.625+0000 7f9472d9d700 1 --2- 172.21.15.47:0/599537666 >> v2:172.21.15.64:3300/0 conn(0x7f947405f4a0 0x7f94740b2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:31.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.625+0000 7f9472d9d700 1 -- 172.21.15.47:0/599537666 >> v1:172.21.15.47:6789/0 conn(0x7f947409a1d0 legacy=0x7f9474142a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:31.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.625+0000 7f9472d9d700 1 -- 172.21.15.47:0/599537666 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f947409b130 con 0x7f9474143140 2022-06-29T15:49:31.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.625+0000 7f9472d9d700 1 --2- 172.21.15.47:0/599537666 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9474143140 0x7f947409a660 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 rx=0x7f947413f510 tx=0x7f946c00b530).ready entity=mon.0 client_cookie=4def0428563943bf server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:31.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.625+0000 7f947159a700 1 -- 172.21.15.47:0/599537666 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f946c00be70 con 0x7f9474143140 2022-06-29T15:49:31.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.625+0000 7f947159a700 1 -- 172.21.15.47:0/599537666 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f946c010470 con 0x7f9474143140 2022-06-29T15:49:31.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.625+0000 7f947159a700 1 -- 172.21.15.47:0/599537666 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f946c0107d0 con 0x7f9474143140 2022-06-29T15:49:31.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.625+0000 7f947359e700 1 -- 172.21.15.47:0/599537666 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9474143140 msgr2=0x7f947409a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:31.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.625+0000 7f947359e700 1 --2- 172.21.15.47:0/599537666 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9474143140 0x7f947409a660 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 rx=0x7f947413f510 tx=0x7f946c00b530).stop 2022-06-29T15:49:31.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.625+0000 7f947359e700 1 -- 172.21.15.47:0/599537666 shutdown_connections 2022-06-29T15:49:31.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.625+0000 7f947359e700 1 --2- 172.21.15.47:0/599537666 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9474143140 0x7f947409a660 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:31.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.625+0000 7f947359e700 1 --2- 172.21.15.47:0/599537666 >> v2:172.21.15.64:3300/0 conn(0x7f947405f4a0 0x7f94740b2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:31.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.625+0000 7f947359e700 1 -- 172.21.15.47:0/599537666 >> 172.21.15.47:0/599537666 conn(0x7f9474062df0 msgr2=0x7f94740631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:31.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.625+0000 7f947359e700 1 -- 172.21.15.47:0/599537666 shutdown_connections 2022-06-29T15:49:31.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.625+0000 7f947359e700 1 -- 172.21.15.47:0/599537666 wait complete. 2022-06-29T15:49:31.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.625+0000 7f947359e700 1 Processor -- start 2022-06-29T15:49:31.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.629+0000 7f947359e700 1 -- start start 2022-06-29T15:49:31.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.629+0000 7f947359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f947405f4a0 0x7f94740ac6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:31.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.629+0000 7f947359e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f947409a1d0 0x7f94740b5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:31.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.629+0000 7f947359e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9474143140 0x7f9474138cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:31.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.629+0000 7f947359e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947414e9f0 con 0x7f947409a1d0 2022-06-29T15:49:31.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.629+0000 7f947359e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94740aff50 con 0x7f947405f4a0 2022-06-29T15:49:31.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.629+0000 7f947359e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f947414ec50 con 0x7f9474143140 2022-06-29T15:49:31.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.629+0000 7f9471d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f947409a1d0 0x7f94740b5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:31.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.629+0000 7f947259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f947405f4a0 0x7f94740ac6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:31.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.629+0000 7f9471d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f947409a1d0 0x7f94740b5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50894/0 (socket says 172.21.15.47:50894) 2022-06-29T15:49:31.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.629+0000 7f9472d9d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9474143140 0x7f9474138cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:31.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.629+0000 7f9471d9b700 1 -- 172.21.15.47:0/1519374127 learned_addr learned my addr 172.21.15.47:0/1519374127 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:31.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.629+0000 7f9471d9b700 1 -- 172.21.15.47:0/1519374127 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9474143140 msgr2=0x7f9474138cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:31.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.629+0000 7f9471d9b700 1 --2- 172.21.15.47:0/1519374127 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9474143140 0x7f9474138cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:31.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.629+0000 7f9471d9b700 1 -- 172.21.15.47:0/1519374127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f947405f4a0 msgr2=0x7f94740ac6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:31.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.629+0000 7f9471d9b700 1 --2- 172.21.15.47:0/1519374127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f947405f4a0 0x7f94740ac6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:31.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.629+0000 7f9471d9b700 1 -- 172.21.15.47:0/1519374127 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f946c00b7d0 con 0x7f947409a1d0 2022-06-29T15:49:31.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.629+0000 7f9471d9b700 1 --2- 172.21.15.47:0/1519374127 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f947409a1d0 0x7f94740b5f00 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 rx=0x7f94740989f0 tx=0x7f9468007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:31.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.629+0000 7f94637fe700 1 -- 172.21.15.47:0/1519374127 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94680070d0 con 0x7f947409a1d0 2022-06-29T15:49:31.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.629+0000 7f94637fe700 1 -- 172.21.15.47:0/1519374127 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9468007230 con 0x7f947409a1d0 2022-06-29T15:49:31.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.629+0000 7f947359e700 1 -- 172.21.15.47:0/1519374127 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94740afab0 con 0x7f947409a1d0 2022-06-29T15:49:31.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.629+0000 7f94637fe700 1 -- 172.21.15.47:0/1519374127 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9468005740 con 0x7f947409a1d0 2022-06-29T15:49:31.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.629+0000 7f947359e700 1 -- 172.21.15.47:0/1519374127 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94740b63f0 con 0x7f947409a1d0 2022-06-29T15:49:31.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.633+0000 7f947359e700 1 -- 172.21.15.47:0/1519374127 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94740af910 con 0x7f947409a1d0 2022-06-29T15:49:31.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.633+0000 7f94637fe700 1 -- 172.21.15.47:0/1519374127 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9468010070 con 0x7f947409a1d0 2022-06-29T15:49:31.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.633+0000 7f94637fe700 1 --2- 172.21.15.47:0/1519374127 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9450053950 0x7f9450055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:31.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.633+0000 7f94637fe700 1 -- 172.21.15.47:0/1519374127 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f946806b5d0 con 0x7f947409a1d0 2022-06-29T15:49:31.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.633+0000 7f947259c700 1 --2- 172.21.15.47:0/1519374127 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9450053950 0x7f9450055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:31.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.633+0000 7f947259c700 1 --2- 172.21.15.47:0/1519374127 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9450053950 0x7f9450055e10 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7f946400c000 tx=0x7f9464008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:31.651 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.637+0000 7f94637fe700 1 -- 172.21.15.47:0/1519374127 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9468068e90 con 0x7f947409a1d0 2022-06-29T15:49:31.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.761+0000 7f947359e700 1 -- 172.21.15.47:0/1519374127 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f947414ee50 con 0x7f947409a1d0 2022-06-29T15:49:31.772 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.765+0000 7f94637fe700 1 -- 172.21.15.47:0/1519374127 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+10 (secure 0 0 0) 0x7f9468018020 con 0x7f947409a1d0 2022-06-29T15:49:31.773 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:49:31.776 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.765+0000 7f947359e700 1 -- 172.21.15.47:0/1519374127 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9450053950 msgr2=0x7f9450055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:31.776 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.765+0000 7f947359e700 1 --2- 172.21.15.47:0/1519374127 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9450053950 0x7f9450055e10 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7f946400c000 tx=0x7f9464008040).stop 2022-06-29T15:49:31.776 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.765+0000 7f947359e700 1 -- 172.21.15.47:0/1519374127 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f947409a1d0 msgr2=0x7f94740b5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:31.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.765+0000 7f947359e700 1 --2- 172.21.15.47:0/1519374127 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f947409a1d0 0x7f94740b5f00 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 rx=0x7f94740989f0 tx=0x7f9468007c00).stop 2022-06-29T15:49:31.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.765+0000 7f947359e700 1 -- 172.21.15.47:0/1519374127 shutdown_connections 2022-06-29T15:49:31.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.765+0000 7f947359e700 1 --2- 172.21.15.47:0/1519374127 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9474143140 0x7f9474138cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:31.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.765+0000 7f947359e700 1 --2- 172.21.15.47:0/1519374127 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f947409a1d0 0x7f94740b5f00 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:31.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.765+0000 7f947359e700 1 --2- 172.21.15.47:0/1519374127 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9450053950 0x7f9450055e10 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:31.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.765+0000 7f947359e700 1 --2- 172.21.15.47:0/1519374127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f947405f4a0 0x7f94740ac6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:31.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.765+0000 7f947359e700 1 -- 172.21.15.47:0/1519374127 >> 172.21.15.47:0/1519374127 conn(0x7f9474062df0 msgr2=0x7f94740b0460 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:31.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.765+0000 7f947359e700 1 -- 172.21.15.47:0/1519374127 shutdown_connections 2022-06-29T15:49:31.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.765+0000 7f947359e700 1 -- 172.21.15.47:0/1519374127 wait complete. 2022-06-29T15:49:31.786 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-29T15:49:31.911 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.889+0000 7f000962f700 1 Processor -- start 2022-06-29T15:49:31.912 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.889+0000 7f000962f700 1 -- start start 2022-06-29T15:49:31.912 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.889+0000 7f000962f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f00040b5cb0 0x7f00040aa130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:31.913 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.889+0000 7f000962f700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0004060790 con 0x7f00040b1610 2022-06-29T15:49:31.913 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.889+0000 7f000962f700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f000414ad80 con 0x7f00040b58d0 2022-06-29T15:49:31.914 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.889+0000 7f000962f700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00040a99e0 con 0x7f00040b5cb0 2022-06-29T15:49:31.914 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.889+0000 7f00037fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f00040b1610 0x7f000414b4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36452/0 (socket says 172.21.15.47:36452) 2022-06-29T15:49:31.914 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.889+0000 7f00037fe700 1 -- 172.21.15.47:0/580282604 learned_addr learned my addr 172.21.15.47:0/580282604 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:49:31.915 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.893+0000 7f0008e2e700 1 --2- 172.21.15.47:0/580282604 >> v2:172.21.15.64:3300/0 conn(0x7f00040b5cb0 0x7f00040aa130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:31.915 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.893+0000 7f0002ffd700 1 -- 172.21.15.47:0/580282604 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2089154545 0 0) 0x7f0004060790 con 0x7f00040b1610 2022-06-29T15:49:31.916 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.893+0000 7f0002ffd700 1 -- 172.21.15.47:0/580282604 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effe4003260 con 0x7f00040b1610 2022-06-29T15:49:31.916 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.893+0000 7f0002ffd700 1 -- 172.21.15.47:0/580282604 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3885953257 0 0) 0x7effe4003260 con 0x7f00040b1610 2022-06-29T15:49:31.917 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.893+0000 7f0002ffd700 1 -- 172.21.15.47:0/580282604 >> v2:172.21.15.64:3300/0 conn(0x7f00040b5cb0 msgr2=0x7f00040aa130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:31.917 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.893+0000 7f0002ffd700 1 --2- 172.21.15.47:0/580282604 >> v2:172.21.15.64:3300/0 conn(0x7f00040b5cb0 0x7f00040aa130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:31.918 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.893+0000 7f0002ffd700 1 -- 172.21.15.47:0/580282604 >> v1:172.21.15.64:6789/0 conn(0x7f00040b58d0 legacy=0x7f00041475f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:31.918 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.893+0000 7f0002ffd700 1 -- 172.21.15.47:0/580282604 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f000414b020 con 0x7f00040b1610 2022-06-29T15:49:31.919 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.893+0000 7f0002ffd700 1 -- 172.21.15.47:0/580282604 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7efff4001660 con 0x7f00040b1610 2022-06-29T15:49:31.919 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.893+0000 7f0002ffd700 1 -- 172.21.15.47:0/580282604 >> v1:172.21.15.47:6789/0 conn(0x7f00040b1610 legacy=0x7f000414b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:31.920 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.893+0000 7f0002ffd700 1 --2- 172.21.15.47:0/580282604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effe4003750 0x7effe4003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:31.920 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.893+0000 7f0002ffd700 1 --2- 172.21.15.47:0/580282604 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7effe4004500 0x7effe4004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:31.920 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.893+0000 7f0002ffd700 1 --2- 172.21.15.47:0/580282604 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effe4004e70 0x7effe400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:31.921 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.893+0000 7f0002ffd700 1 -- 172.21.15.47:0/580282604 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00040a99e0 con 0x7effe4003750 2022-06-29T15:49:31.921 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.893+0000 7f0002ffd700 1 -- 172.21.15.47:0/580282604 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe400bd20 con 0x7effe4004e70 2022-06-29T15:49:31.922 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.893+0000 7f0002ffd700 1 -- 172.21.15.47:0/580282604 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effe400bee0 con 0x7effe4004500 2022-06-29T15:49:31.922 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.893+0000 7f0003fff700 1 --2- 172.21.15.47:0/580282604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effe4003750 0x7effe4003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:31.923 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.893+0000 7f0008e2e700 1 --2- 172.21.15.47:0/580282604 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effe4004e70 0x7effe400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:31.923 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.893+0000 7f00037fe700 1 --2- 172.21.15.47:0/580282604 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7effe4004500 0x7effe4004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:31.924 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.893+0000 7f0003fff700 1 -- 172.21.15.47:0/580282604 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7effe4004500 msgr2=0x7effe4004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:31.924 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.893+0000 7f0003fff700 1 --2- 172.21.15.47:0/580282604 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7effe4004500 0x7effe4004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:31.924 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.893+0000 7f0003fff700 1 -- 172.21.15.47:0/580282604 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effe4004e70 msgr2=0x7effe400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:31.925 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.893+0000 7f0003fff700 1 --2- 172.21.15.47:0/580282604 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effe4004e70 0x7effe400b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:31.925 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.893+0000 7f0003fff700 1 -- 172.21.15.47:0/580282604 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe4003260 con 0x7effe4003750 2022-06-29T15:49:31.925 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.893+0000 7f0008e2e700 1 --2- 172.21.15.47:0/580282604 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effe4004e70 0x7effe400b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:49:31.925 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.893+0000 7f0008e2e700 1 -- 172.21.15.47:0/580282604 reap_dead start 2022-06-29T15:49:31.926 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.893+0000 7f0003fff700 1 --2- 172.21.15.47:0/580282604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effe4003750 0x7effe4003b70 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 rx=0x7f0004062140 tx=0x7effec007a00).ready entity=mon.0 client_cookie=241272f4de290079 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:31.926 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.893+0000 7f0002ffd700 1 -- 172.21.15.47:0/580282604 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effec007d10 con 0x7effe4003750 2022-06-29T15:49:31.926 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.893+0000 7f0002ffd700 1 -- 172.21.15.47:0/580282604 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effec007030 con 0x7effe4003750 2022-06-29T15:49:31.926 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.893+0000 7f0002ffd700 1 -- 172.21.15.47:0/580282604 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effec007390 con 0x7effe4003750 2022-06-29T15:49:31.927 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.893+0000 7f000962f700 1 -- 172.21.15.47:0/580282604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effe4003750 msgr2=0x7effe4003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:31.927 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.893+0000 7f000962f700 1 --2- 172.21.15.47:0/580282604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effe4003750 0x7effe4003b70 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 rx=0x7f0004062140 tx=0x7effec007a00).stop 2022-06-29T15:49:31.927 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.893+0000 7f000962f700 1 -- 172.21.15.47:0/580282604 shutdown_connections 2022-06-29T15:49:31.927 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.893+0000 7f000962f700 1 --2- 172.21.15.47:0/580282604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effe4003750 0x7effe4003b70 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:31.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.893+0000 7f000962f700 1 -- 172.21.15.47:0/580282604 >> 172.21.15.47:0/580282604 conn(0x7f0004062560 msgr2=0x7f0004062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:31.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.893+0000 7f000962f700 1 -- 172.21.15.47:0/580282604 shutdown_connections 2022-06-29T15:49:31.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.893+0000 7f000962f700 1 -- 172.21.15.47:0/580282604 wait complete. 2022-06-29T15:49:31.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.893+0000 7f000962f700 1 Processor -- start 2022-06-29T15:49:31.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.893+0000 7f000962f700 1 -- start start 2022-06-29T15:49:31.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.897+0000 7f000962f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effe4003750 0x7f0004064110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:31.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.897+0000 7f000962f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00040aedc0 0x7f0004066850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:31.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.897+0000 7f000962f700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0004063160 0x7f0004063610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:31.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.897+0000 7f000962f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0004064d60 con 0x7f00040aedc0 2022-06-29T15:49:31.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.897+0000 7f000962f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0004066330 con 0x7effe4003750 2022-06-29T15:49:31.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.897+0000 7f000962f700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0004147070 con 0x7f0004063160 2022-06-29T15:49:31.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.897+0000 7f00037fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00040aedc0 0x7f0004066850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:31.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.897+0000 7f00037fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00040aedc0 0x7f0004066850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50912/0 (socket says 172.21.15.47:50912) 2022-06-29T15:49:31.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.897+0000 7f00037fe700 1 -- 172.21.15.47:0/866594821 learned_addr learned my addr 172.21.15.47:0/866594821 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:31.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.897+0000 7f0003fff700 1 --2- 172.21.15.47:0/866594821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effe4003750 0x7f0004064110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:31.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.897+0000 7f0008e2e700 1 --2- 172.21.15.47:0/866594821 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0004063160 0x7f0004063610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:31.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.897+0000 7f00037fe700 1 -- 172.21.15.47:0/866594821 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0004063160 msgr2=0x7f0004063610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:31.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.897+0000 7f00037fe700 1 --2- 172.21.15.47:0/866594821 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0004063160 0x7f0004063610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:31.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.897+0000 7f00037fe700 1 -- 172.21.15.47:0/866594821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effe4003750 msgr2=0x7f0004064110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:31.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.897+0000 7f00037fe700 1 --2- 172.21.15.47:0/866594821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effe4003750 0x7f0004064110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:31.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.897+0000 7f00037fe700 1 -- 172.21.15.47:0/866594821 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0004147310 con 0x7f00040aedc0 2022-06-29T15:49:31.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.897+0000 7f00037fe700 1 --2- 172.21.15.47:0/866594821 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00040aedc0 0x7f0004066850 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 rx=0x7efff400b000 tx=0x7f0004060920).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:31.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.897+0000 7f00017fa700 1 -- 172.21.15.47:0/866594821 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efff4010710 con 0x7f00040aedc0 2022-06-29T15:49:31.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.897+0000 7f00017fa700 1 -- 172.21.15.47:0/866594821 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efff4010870 con 0x7f00040aedc0 2022-06-29T15:49:31.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.897+0000 7f000962f700 1 -- 172.21.15.47:0/866594821 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00040ae310 con 0x7f00040aedc0 2022-06-29T15:49:31.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.897+0000 7f00017fa700 1 -- 172.21.15.47:0/866594821 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efff4010bd0 con 0x7f00040aedc0 2022-06-29T15:49:31.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.897+0000 7f000962f700 1 -- 172.21.15.47:0/866594821 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00040ae7b0 con 0x7f00040aedc0 2022-06-29T15:49:31.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.897+0000 7f000962f700 1 -- 172.21.15.47:0/866594821 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00041474a0 con 0x7f00040aedc0 2022-06-29T15:49:31.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.901+0000 7f00017fa700 1 -- 172.21.15.47:0/866594821 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7efff4010d30 con 0x7f00040aedc0 2022-06-29T15:49:31.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.901+0000 7f00017fa700 1 --2- 172.21.15.47:0/866594821 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7effe8053b30 0x7effe8055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:31.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.901+0000 7f00017fa700 1 -- 172.21.15.47:0/866594821 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6420+0+0 (secure 0 0 0) 0x7efff406c1b0 con 0x7f00040aedc0 2022-06-29T15:49:31.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.901+0000 7f0003fff700 1 --2- 172.21.15.47:0/866594821 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7effe8053b30 0x7effe8055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:31.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.901+0000 7f0003fff700 1 --2- 172.21.15.47:0/866594821 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7effe8053b30 0x7effe8055ff0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7effec01b000 tx=0x7effec015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:31.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:31.905+0000 7f00017fa700 1 -- 172.21.15.47:0/866594821 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7efff4042e60 con 0x7f00040aedc0 2022-06-29T15:49:32.038 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.029+0000 7f000962f700 1 -- 172.21.15.47:0/866594821 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f00040573d0 con 0x7f00040aedc0 2022-06-29T15:49:32.038 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.029+0000 7f00017fa700 1 -- 172.21.15.47:0/866594821 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+10 (secure 0 0 0) 0x7efff4046480 con 0x7f00040aedc0 2022-06-29T15:49:32.039 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:49:32.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.033+0000 7f000962f700 1 -- 172.21.15.47:0/866594821 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7effe8053b30 msgr2=0x7effe8055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:32.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.033+0000 7f000962f700 1 --2- 172.21.15.47:0/866594821 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7effe8053b30 0x7effe8055ff0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7effec01b000 tx=0x7effec015000).stop 2022-06-29T15:49:32.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.033+0000 7f000962f700 1 -- 172.21.15.47:0/866594821 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00040aedc0 msgr2=0x7f0004066850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:32.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.033+0000 7f000962f700 1 --2- 172.21.15.47:0/866594821 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00040aedc0 0x7f0004066850 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 rx=0x7efff400b000 tx=0x7f0004060920).stop 2022-06-29T15:49:32.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.033+0000 7f000962f700 1 -- 172.21.15.47:0/866594821 shutdown_connections 2022-06-29T15:49:32.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.033+0000 7f000962f700 1 --2- 172.21.15.47:0/866594821 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0004063160 0x7f0004063610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:32.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.033+0000 7f000962f700 1 --2- 172.21.15.47:0/866594821 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00040aedc0 0x7f0004066850 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:32.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.033+0000 7f000962f700 1 --2- 172.21.15.47:0/866594821 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7effe8053b30 0x7effe8055ff0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:32.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.033+0000 7f000962f700 1 --2- 172.21.15.47:0/866594821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effe4003750 0x7f0004064110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:32.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.033+0000 7f000962f700 1 -- 172.21.15.47:0/866594821 >> 172.21.15.47:0/866594821 conn(0x7f0004062560 msgr2=0x7f00041412c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:32.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.033+0000 7f000962f700 1 -- 172.21.15.47:0/866594821 shutdown_connections 2022-06-29T15:49:32.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.033+0000 7f000962f700 1 -- 172.21.15.47:0/866594821 wait complete. 2022-06-29T15:49:32.052 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-29T15:49:32.053 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:49:32.154 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.145+0000 7f9391f83700 1 Processor -- start 2022-06-29T15:49:32.154 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.145+0000 7f9391f83700 1 -- start start 2022-06-29T15:49:32.156 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.145+0000 7f9391f83700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f938c0619b0 0x7f938c0606e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:32.157 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.145+0000 7f9391f83700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f938c1408e0 0x7f938c139a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:32.157 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.145+0000 7f9391f83700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f938c140d50 con 0x7f938c0b0f30 2022-06-29T15:49:32.157 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.145+0000 7f9391f83700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f938c0b2190 con 0x7f938c1408e0 2022-06-29T15:49:32.157 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.145+0000 7f9391f83700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f938c061d90 con 0x7f938c0619b0 2022-06-29T15:49:32.158 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.149+0000 7f938bfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f938c1408e0 0x7f938c139a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:32.158 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.149+0000 7f938bfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f938c1408e0 0x7f938c139a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50922/0 (socket says 172.21.15.47:50922) 2022-06-29T15:49:32.158 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.149+0000 7f938bfff700 1 -- 172.21.15.47:0/1574214216 learned_addr learned my addr 172.21.15.47:0/1574214216 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:32.158 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.149+0000 7f938b7fe700 1 --2- 172.21.15.47:0/1574214216 >> v2:172.21.15.64:3300/0 conn(0x7f938c0619b0 0x7f938c0606e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:32.159 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.149+0000 7f938bfff700 1 -- 172.21.15.47:0/1574214216 >> v2:172.21.15.64:3300/0 conn(0x7f938c0619b0 msgr2=0x7f938c0606e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:32.159 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.149+0000 7f938bfff700 1 --2- 172.21.15.47:0/1574214216 >> v2:172.21.15.64:3300/0 conn(0x7f938c0619b0 0x7f938c0606e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:32.159 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.149+0000 7f938bfff700 1 -- 172.21.15.47:0/1574214216 >> v1:172.21.15.47:6789/0 conn(0x7f938c0b0f30 legacy=0x7f938c0b1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:32.159 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.149+0000 7f938bfff700 1 -- 172.21.15.47:0/1574214216 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f938c060d80 con 0x7f938c1408e0 2022-06-29T15:49:32.159 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.149+0000 7f938bfff700 1 --2- 172.21.15.47:0/1574214216 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f938c1408e0 0x7f938c139a20 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 rx=0x7f9384002f70 tx=0x7f9384006eb0).ready entity=mon.0 client_cookie=8aa79867d7719fd5 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:32.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.149+0000 7f938a7fc700 1 -- 172.21.15.47:0/1574214216 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f938400d5c0 con 0x7f938c1408e0 2022-06-29T15:49:32.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.149+0000 7f938a7fc700 1 -- 172.21.15.47:0/1574214216 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9384000e90 con 0x7f938c1408e0 2022-06-29T15:49:32.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.149+0000 7f938a7fc700 1 -- 172.21.15.47:0/1574214216 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f938400d950 con 0x7f938c1408e0 2022-06-29T15:49:32.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.149+0000 7f9391f83700 1 -- 172.21.15.47:0/1574214216 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f938c1408e0 msgr2=0x7f938c139a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:32.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.149+0000 7f9391f83700 1 --2- 172.21.15.47:0/1574214216 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f938c1408e0 0x7f938c139a20 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 rx=0x7f9384002f70 tx=0x7f9384006eb0).stop 2022-06-29T15:49:32.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.149+0000 7f9391f83700 1 -- 172.21.15.47:0/1574214216 shutdown_connections 2022-06-29T15:49:32.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.149+0000 7f9391f83700 1 --2- 172.21.15.47:0/1574214216 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f938c1408e0 0x7f938c139a20 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:32.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.149+0000 7f9391f83700 1 --2- 172.21.15.47:0/1574214216 >> v2:172.21.15.64:3300/0 conn(0x7f938c0619b0 0x7f938c0606e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:32.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.149+0000 7f9391f83700 1 -- 172.21.15.47:0/1574214216 >> 172.21.15.47:0/1574214216 conn(0x7f938c05c510 msgr2=0x7f938c063800 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:32.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.149+0000 7f9391f83700 1 -- 172.21.15.47:0/1574214216 shutdown_connections 2022-06-29T15:49:32.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.149+0000 7f9391f83700 1 -- 172.21.15.47:0/1574214216 wait complete. 2022-06-29T15:49:32.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.153+0000 7f9391f83700 1 Processor -- start 2022-06-29T15:49:32.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.153+0000 7f9391f83700 1 -- start start 2022-06-29T15:49:32.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.153+0000 7f9391f83700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f938c0619b0 0x7f938c0af710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:32.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.153+0000 7f9391f83700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f938c0b0f30 0x7f938c1530b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:32.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.153+0000 7f9391f83700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f938c1408e0 0x7f938c0b6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:32.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.153+0000 7f9391f83700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938c0b47b0 con 0x7f938c0b0f30 2022-06-29T15:49:32.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.153+0000 7f9391f83700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938c1535a0 con 0x7f938c1408e0 2022-06-29T15:49:32.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.153+0000 7f9391f83700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f938c143780 con 0x7f938c0619b0 2022-06-29T15:49:32.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.153+0000 7f938affd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f938c0b0f30 0x7f938c1530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:32.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.153+0000 7f938affd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f938c0b0f30 0x7f938c1530b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50924/0 (socket says 172.21.15.47:50924) 2022-06-29T15:49:32.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.153+0000 7f938affd700 1 -- 172.21.15.47:0/3474340068 learned_addr learned my addr 172.21.15.47:0/3474340068 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:32.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.153+0000 7f938bfff700 1 --2- 172.21.15.47:0/3474340068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f938c1408e0 0x7f938c0b6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:32.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.153+0000 7f938b7fe700 1 --2- 172.21.15.47:0/3474340068 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f938c0619b0 0x7f938c0af710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:32.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.153+0000 7f938affd700 1 -- 172.21.15.47:0/3474340068 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f938c0619b0 msgr2=0x7f938c0af710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:32.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.153+0000 7f938affd700 1 --2- 172.21.15.47:0/3474340068 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f938c0619b0 0x7f938c0af710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:32.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.157+0000 7f938affd700 1 -- 172.21.15.47:0/3474340068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f938c1408e0 msgr2=0x7f938c0b6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:32.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.157+0000 7f938affd700 1 --2- 172.21.15.47:0/3474340068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f938c1408e0 0x7f938c0b6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:32.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.157+0000 7f938affd700 1 -- 172.21.15.47:0/3474340068 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f938400b1e0 con 0x7f938c0b0f30 2022-06-29T15:49:32.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.157+0000 7f938affd700 1 --2- 172.21.15.47:0/3474340068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f938c0b0f30 0x7f938c1530b0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 rx=0x7f938c0b0e20 tx=0x7f9374007f60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:32.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.157+0000 7f9388ff9700 1 -- 172.21.15.47:0/3474340068 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9374006c90 con 0x7f938c0b0f30 2022-06-29T15:49:32.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.157+0000 7f9388ff9700 1 -- 172.21.15.47:0/3474340068 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9374006df0 con 0x7f938c0b0f30 2022-06-29T15:49:32.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.157+0000 7f9391f83700 1 -- 172.21.15.47:0/3474340068 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f938c143a20 con 0x7f938c0b0f30 2022-06-29T15:49:32.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.157+0000 7f9388ff9700 1 -- 172.21.15.47:0/3474340068 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9374007220 con 0x7f938c0b0f30 2022-06-29T15:49:32.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.157+0000 7f9391f83700 1 -- 172.21.15.47:0/3474340068 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f938c0b71b0 con 0x7f938c0b0f30 2022-06-29T15:49:32.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.157+0000 7f9391f83700 1 -- 172.21.15.47:0/3474340068 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f938c0573d0 con 0x7f938c0b0f30 2022-06-29T15:49:32.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.157+0000 7f9388ff9700 1 -- 172.21.15.47:0/3474340068 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9374007380 con 0x7f938c0b0f30 2022-06-29T15:49:32.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.161+0000 7f9388ff9700 1 --2- 172.21.15.47:0/3474340068 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9370053b30 0x7f9370055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:32.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.161+0000 7f9388ff9700 1 -- 172.21.15.47:0/3474340068 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f937406e310 con 0x7f938c0b0f30 2022-06-29T15:49:32.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.161+0000 7f938b7fe700 1 --2- 172.21.15.47:0/3474340068 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9370053b30 0x7f9370055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:32.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.161+0000 7f938b7fe700 1 --2- 172.21.15.47:0/3474340068 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9370053b30 0x7f9370055ff0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7f937c00b000 tx=0x7f937c006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:32.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.165+0000 7f9388ff9700 1 -- 172.21.15.47:0/3474340068 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9374044f40 con 0x7f938c0b0f30 2022-06-29T15:49:32.335 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.325+0000 7f9391f83700 1 -- 172.21.15.47:0/3474340068 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f938c143bb0 con 0x7f938c0b0f30 2022-06-29T15:49:32.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.329+0000 7f9388ff9700 1 -- 172.21.15.47:0/3474340068 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+2493 (secure 0 0 0) 0x7f9374048560 con 0x7f938c0b0f30 2022-06-29T15:49:32.336 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:49:32.336 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":40,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6577,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/4286932616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":4286932616},{"type":"v1","addr":"172.21.15.64:6833","nonce":4286932616}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":38},{"gid":6657,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3528102962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3528102962},{"type":"v1","addr":"172.21.15.47:6835","nonce":3528102962}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":38}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:49:27.717245+0000","modified":"2022-06-29T15:49:28.719969+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6660},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6660":{"gid":6660,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6837/4146905199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":4146905199},{"type":"v1","addr":"172.21.15.47:6837","nonce":4146905199}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2022-06-29T15:49:32.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.329+0000 7f9391f83700 1 -- 172.21.15.47:0/3474340068 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9370053b30 msgr2=0x7f9370055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:32.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.329+0000 7f9391f83700 1 --2- 172.21.15.47:0/3474340068 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9370053b30 0x7f9370055ff0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7f937c00b000 tx=0x7f937c006df0).stop 2022-06-29T15:49:32.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.329+0000 7f9391f83700 1 -- 172.21.15.47:0/3474340068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f938c0b0f30 msgr2=0x7f938c1530b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:32.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.329+0000 7f9391f83700 1 --2- 172.21.15.47:0/3474340068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f938c0b0f30 0x7f938c1530b0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 rx=0x7f938c0b0e20 tx=0x7f9374007f60).stop 2022-06-29T15:49:32.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.333+0000 7f9391f83700 1 -- 172.21.15.47:0/3474340068 shutdown_connections 2022-06-29T15:49:32.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.333+0000 7f9391f83700 1 --2- 172.21.15.47:0/3474340068 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9370053b30 0x7f9370055ff0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:32.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.333+0000 7f9391f83700 1 --2- 172.21.15.47:0/3474340068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f938c1408e0 0x7f938c0b6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:32.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.333+0000 7f9391f83700 1 --2- 172.21.15.47:0/3474340068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f938c0b0f30 0x7f938c1530b0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:32.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.333+0000 7f9391f83700 1 --2- 172.21.15.47:0/3474340068 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f938c0619b0 0x7f938c0af710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:32.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.333+0000 7f9391f83700 1 -- 172.21.15.47:0/3474340068 >> 172.21.15.47:0/3474340068 conn(0x7f938c05c510 msgr2=0x7f938c13f500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:32.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.333+0000 7f9391f83700 1 -- 172.21.15.47:0/3474340068 shutdown_connections 2022-06-29T15:49:32.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:32.333+0000 7f9391f83700 1 -- 172.21.15.47:0/3474340068 wait complete. 2022-06-29T15:49:32.343 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 40 2022-06-29T15:49:32.658 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 40, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-29T15:49:27.717245+0000', 'modified': '2022-06-29T15:49:28.719969+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6660}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6660': {'gid': 6660, 'name': 'a', 'rank': 0, 'incarnation': 39, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.47:6837/4146905199', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6836', 'nonce': 4146905199}, {'type': 'v1', 'addr': '172.21.15.47:6837', 'nonce': 4146905199}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-29T15:49:32.658 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-29T15:49:32.659 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-06-29T15:49:32.780 INFO:teuthology.orchestra.run.smithi047.stdout:{"cluster_fsid":"9a68894a-53ce-400e-838b-14eab91b834c","whoami":0,"id":6660,"want_state":"up:active","state":"up:active","rank_uptime":5.0474122149999996,"mdsmap_epoch":40,"osdmap_epoch":90,"osdmap_epoch_barrier":89,"uptime":9.7378169369999998} 2022-06-29T15:49:32.781 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9a68894a-53ce-400e-838b-14eab91b834c", "id": 6660, "mdsmap_epoch": 40, "osdmap_epoch": 90, "osdmap_epoch_barrier": 89, "rank_uptime": 5.047412215, "state": "up:active", "uptime": 9.737816937, "want_state": "up:active", "whoami": 0 } 2022-06-29T15:49:32.781 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-29T15:49:32.782 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-06-29T15:49:32.783 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi099.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-06-29T15:49:32.784 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:49:32.785 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:49:32.794 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2022-06-29T15:49:32.812 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:49:32.879 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:49:32.896 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:49:32.897 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:49:32.898 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:49:32.899 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2022-06-29T15:49:32.948 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-29T15:49:32.949 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:49:32.953 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2022-06-29T15:49:32.971 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:49:32.972 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:49:33.027 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:2022-06-29T15:49:33.022+0000 7ffb8281c080 -1 init, newargv = 0x55cb7a6cf710 newargc=9ceph-fuse[13011]: starting ceph client 2022-06-29T15:49:33.027 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr: 2022-06-29T15:49:33.033 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:49:33.033 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:49:33.035 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:49:33.035 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2022-06-29T15:49:33.038 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[13011]: starting fuse 2022-06-29T15:49:33.042 INFO:teuthology.orchestra.run.smithi099.stdout:52 2022-06-29T15:49:33.043 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-29T15:49:33.043 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:49:33.044 DEBUG:teuthology.orchestra.run.smithi099:> sudo python3 -c ' 2022-06-29T15:49:33.044 DEBUG:teuthology.orchestra.run.smithi099:> import glob 2022-06-29T15:49:33.044 DEBUG:teuthology.orchestra.run.smithi099:> import re 2022-06-29T15:49:33.045 DEBUG:teuthology.orchestra.run.smithi099:> import os 2022-06-29T15:49:33.045 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2022-06-29T15:49:33.046 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:49:33.046 DEBUG:teuthology.orchestra.run.smithi099:> def find_socket(client_name): 2022-06-29T15:49:33.047 DEBUG:teuthology.orchestra.run.smithi099:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-06-29T15:49:33.047 DEBUG:teuthology.orchestra.run.smithi099:> files = glob.glob(asok_path) 2022-06-29T15:49:33.047 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:49:33.048 DEBUG:teuthology.orchestra.run.smithi099:> # Given a non-glob path, it better be there 2022-06-29T15:49:33.048 DEBUG:teuthology.orchestra.run.smithi099:> if "*" not in asok_path: 2022-06-29T15:49:33.049 DEBUG:teuthology.orchestra.run.smithi099:> assert(len(files) == 1) 2022-06-29T15:49:33.049 DEBUG:teuthology.orchestra.run.smithi099:> return files[0] 2022-06-29T15:49:33.050 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:49:33.050 DEBUG:teuthology.orchestra.run.smithi099:> for f in files: 2022-06-29T15:49:33.050 DEBUG:teuthology.orchestra.run.smithi099:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-29T15:49:33.051 DEBUG:teuthology.orchestra.run.smithi099:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-29T15:49:33.051 DEBUG:teuthology.orchestra.run.smithi099:> return f 2022-06-29T15:49:33.052 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-29T15:49:33.052 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:49:33.053 DEBUG:teuthology.orchestra.run.smithi099:> print(find_socket("client.0")) 2022-06-29T15:49:33.053 DEBUG:teuthology.orchestra.run.smithi099:> ' 2022-06-29T15:49:33.093 INFO:teuthology.orchestra.run.smithi099.stdout:/var/run/ceph/ceph-client.0.13011.asok 2022-06-29T15:49:33.098 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.13011.asok 2022-06-29T15:49:33.098 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:49:33.099 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.13011.asok status 2022-06-29T15:49:33.186 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2022-06-29T15:49:33.186 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": { 2022-06-29T15:49:33.186 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_sha1": "f25ffac72e5a9e63860ff7deb6767a11f0559a16", 2022-06-29T15:49:33.186 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 15.2.16-304-gf25ffac7 (f25ffac72e5a9e63860ff7deb6767a11f0559a16) octopus (stable)", 2022-06-29T15:49:33.187 INFO:teuthology.orchestra.run.smithi099.stdout: "entity_id": "0", 2022-06-29T15:49:33.187 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": "smithi099", 2022-06-29T15:49:33.187 INFO:teuthology.orchestra.run.smithi099.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-06-29T15:49:33.187 INFO:teuthology.orchestra.run.smithi099.stdout: "pid": "13011", 2022-06-29T15:49:33.188 INFO:teuthology.orchestra.run.smithi099.stdout: "root": "/" 2022-06-29T15:49:33.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:49:33.188 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_count": 0, 2022-06-29T15:49:33.188 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_pinned_count": 0, 2022-06-29T15:49:33.188 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 6745, 2022-06-29T15:49:33.189 INFO:teuthology.orchestra.run.smithi099.stdout: "inst": { 2022-06-29T15:49:33.189 INFO:teuthology.orchestra.run.smithi099.stdout: "name": { 2022-06-29T15:49:33.189 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "client", 2022-06-29T15:49:33.189 INFO:teuthology.orchestra.run.smithi099.stdout: "num": 6745 2022-06-29T15:49:33.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:49:33.190 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2022-06-29T15:49:33.190 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2022-06-29T15:49:33.190 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:0", 2022-06-29T15:49:33.191 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2437750181 2022-06-29T15:49:33.191 INFO:teuthology.orchestra.run.smithi099.stdout: } 2022-06-29T15:49:33.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:49:33.191 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2022-06-29T15:49:33.191 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2022-06-29T15:49:33.192 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:0", 2022-06-29T15:49:33.192 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2437750181 2022-06-29T15:49:33.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:49:33.192 INFO:teuthology.orchestra.run.smithi099.stdout: "inst_str": "client.6745 172.21.15.99:0/2437750181", 2022-06-29T15:49:33.192 INFO:teuthology.orchestra.run.smithi099.stdout: "addr_str": "172.21.15.99:0/2437750181", 2022-06-29T15:49:33.193 INFO:teuthology.orchestra.run.smithi099.stdout: "inode_count": 1, 2022-06-29T15:49:33.193 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_epoch": 40, 2022-06-29T15:49:33.194 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch": 90, 2022-06-29T15:49:33.195 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch_barrier": 0, 2022-06-29T15:49:33.195 INFO:teuthology.orchestra.run.smithi099.stdout: "blacklisted": false, 2022-06-29T15:49:33.195 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs" 2022-06-29T15:49:33.196 INFO:teuthology.orchestra.run.smithi099.stdout:} 2022-06-29T15:49:33.199 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:49:33.200 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:49:33.200 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:49:33.209 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2022-06-29T15:49:33.211 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-29T15:49:33.212 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:49:33.213 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:49:33.233 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:49:33.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.337+0000 7f864d085700 1 Processor -- start 2022-06-29T15:49:33.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.337+0000 7f864d085700 1 -- start start 2022-06-29T15:49:33.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.337+0000 7f864d085700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f86480b5c60 0x7f8648144980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:33.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.337+0000 7f864d085700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86480a9120 con 0x7f86480a72f0 2022-06-29T15:49:33.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.337+0000 7f864d085700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86480b0b80 con 0x7f8648060eb0 2022-06-29T15:49:33.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.337+0000 7f864d085700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86480a89e0 con 0x7f86480b5c60 2022-06-29T15:49:33.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.341+0000 7f864c884700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f86480a72f0 0x7f8648151eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36490/0 (socket says 172.21.15.47:36490) 2022-06-29T15:49:33.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.341+0000 7f864c884700 1 -- 172.21.15.47:0/2261683652 learned_addr learned my addr 172.21.15.47:0/2261683652 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:49:33.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.341+0000 7f8647fff700 1 --2- 172.21.15.47:0/2261683652 >> v2:172.21.15.64:3300/0 conn(0x7f86480b5c60 0x7f8648144980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:33.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.341+0000 7f8646ffd700 1 -- 172.21.15.47:0/2261683652 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2461176742 0 0) 0x7f86480a9120 con 0x7f86480a72f0 2022-06-29T15:49:33.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.341+0000 7f8646ffd700 1 -- 172.21.15.47:0/2261683652 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8628003260 con 0x7f86480a72f0 2022-06-29T15:49:33.352 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.341+0000 7f8647fff700 1 -- 172.21.15.47:0/2261683652 >> v1:172.21.15.64:6789/0 conn(0x7f8648060eb0 legacy=0x7f86480b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:33.352 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.341+0000 7f8647fff700 1 -- 172.21.15.47:0/2261683652 >> v1:172.21.15.47:6789/0 conn(0x7f86480a72f0 legacy=0x7f8648151eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:33.352 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.341+0000 7f8647fff700 1 -- 172.21.15.47:0/2261683652 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86480a8c80 con 0x7f86480b5c60 2022-06-29T15:49:33.352 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.341+0000 7f8647fff700 1 --2- 172.21.15.47:0/2261683652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86480b5c60 0x7f8648144980 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f86400032f0 tx=0x7f86400091a0).ready entity=mon.1 client_cookie=ecdfd592830178f5 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:33.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.341+0000 7f8646ffd700 1 -- 172.21.15.47:0/2261683652 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f864000b680 con 0x7f86480b5c60 2022-06-29T15:49:33.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.341+0000 7f8646ffd700 1 -- 172.21.15.47:0/2261683652 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f864000b7e0 con 0x7f86480b5c60 2022-06-29T15:49:33.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.341+0000 7f8646ffd700 1 -- 172.21.15.47:0/2261683652 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f864000b950 con 0x7f86480b5c60 2022-06-29T15:49:33.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.341+0000 7f864d085700 1 -- 172.21.15.47:0/2261683652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86480b5c60 msgr2=0x7f8648144980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:33.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.341+0000 7f864d085700 1 --2- 172.21.15.47:0/2261683652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86480b5c60 0x7f8648144980 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f86400032f0 tx=0x7f86400091a0).stop 2022-06-29T15:49:33.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.341+0000 7f864d085700 1 -- 172.21.15.47:0/2261683652 shutdown_connections 2022-06-29T15:49:33.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.341+0000 7f864d085700 1 --2- 172.21.15.47:0/2261683652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86480b5c60 0x7f8648144980 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:33.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.341+0000 7f864d085700 1 -- 172.21.15.47:0/2261683652 >> 172.21.15.47:0/2261683652 conn(0x7f8648062560 msgr2=0x7f8648062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:33.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.341+0000 7f864d085700 1 -- 172.21.15.47:0/2261683652 shutdown_connections 2022-06-29T15:49:33.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.341+0000 7f864d085700 1 -- 172.21.15.47:0/2261683652 wait complete. 2022-06-29T15:49:33.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.345+0000 7f864d085700 1 Processor -- start 2022-06-29T15:49:33.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.345+0000 7f864d085700 1 -- start start 2022-06-29T15:49:33.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.345+0000 7f864d085700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8648060eb0 0x7f864814d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:33.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.345+0000 7f864d085700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86480a72f0 0x7f864813a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:33.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.345+0000 7f864d085700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86480b5c60 0x7f864813bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:33.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.345+0000 7f864d085700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8648139a10 con 0x7f86480b5c60 2022-06-29T15:49:33.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.345+0000 7f864d085700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f864813b3d0 con 0x7f86480a72f0 2022-06-29T15:49:33.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.345+0000 7f864d085700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8648139c90 con 0x7f8648060eb0 2022-06-29T15:49:33.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.345+0000 7f864c884700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86480b5c60 0x7f864813bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:33.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.345+0000 7f864c884700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86480b5c60 0x7f864813bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50942/0 (socket says 172.21.15.47:50942) 2022-06-29T15:49:33.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.345+0000 7f864c884700 1 -- 172.21.15.47:0/608730333 learned_addr learned my addr 172.21.15.47:0/608730333 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:33.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.345+0000 7f86477fe700 1 --2- 172.21.15.47:0/608730333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86480a72f0 0x7f864813a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:33.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.345+0000 7f8647fff700 1 --2- 172.21.15.47:0/608730333 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8648060eb0 0x7f864814d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:33.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.345+0000 7f864c884700 1 -- 172.21.15.47:0/608730333 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8648060eb0 msgr2=0x7f864814d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:33.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.345+0000 7f864c884700 1 --2- 172.21.15.47:0/608730333 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8648060eb0 0x7f864814d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:33.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.345+0000 7f864c884700 1 -- 172.21.15.47:0/608730333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86480a72f0 msgr2=0x7f864813a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:33.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.345+0000 7f864c884700 1 --2- 172.21.15.47:0/608730333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86480a72f0 0x7f864813a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:33.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.345+0000 7f864c884700 1 -- 172.21.15.47:0/608730333 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8640009420 con 0x7f86480b5c60 2022-06-29T15:49:33.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.345+0000 7f864c884700 1 --2- 172.21.15.47:0/608730333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86480b5c60 0x7f864813bce0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 rx=0x7f8638001110 tx=0x7f8638007a80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:33.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.345+0000 7f86457fa700 1 -- 172.21.15.47:0/608730333 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f863800ce80 con 0x7f86480b5c60 2022-06-29T15:49:33.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.345+0000 7f86457fa700 1 -- 172.21.15.47:0/608730333 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86380070d0 con 0x7f86480b5c60 2022-06-29T15:49:33.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.345+0000 7f864d085700 1 -- 172.21.15.47:0/608730333 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8648139f30 con 0x7f86480b5c60 2022-06-29T15:49:33.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.345+0000 7f86457fa700 1 -- 172.21.15.47:0/608730333 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8638007430 con 0x7f86480b5c60 2022-06-29T15:49:33.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.345+0000 7f864d085700 1 -- 172.21.15.47:0/608730333 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f864813c1d0 con 0x7f86480b5c60 2022-06-29T15:49:33.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.349+0000 7f864d085700 1 -- 172.21.15.47:0/608730333 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f864813a0c0 con 0x7f86480b5c60 2022-06-29T15:49:33.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.349+0000 7f86457fa700 1 -- 172.21.15.47:0/608730333 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f863801a060 con 0x7f86480b5c60 2022-06-29T15:49:33.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.349+0000 7f86457fa700 1 --2- 172.21.15.47:0/608730333 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f862c0539e0 0x7f862c055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:33.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.349+0000 7f8647fff700 1 --2- 172.21.15.47:0/608730333 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f862c0539e0 0x7f862c055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:33.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.349+0000 7f86457fa700 1 -- 172.21.15.47:0/608730333 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f863806b820 con 0x7f86480b5c60 2022-06-29T15:49:33.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.349+0000 7f8647fff700 1 --2- 172.21.15.47:0/608730333 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f862c0539e0 0x7f862c055ea0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f864001b000 tx=0x7f8640016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:33.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.357+0000 7f86457fa700 1 -- 172.21.15.47:0/608730333 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f86380424d0 con 0x7f86480b5c60 2022-06-29T15:49:33.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.485+0000 7f864d085700 1 -- 172.21.15.47:0/608730333 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86480573d0 con 0x7f86480b5c60 2022-06-29T15:49:33.497 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.489+0000 7f86457fa700 1 -- 172.21.15.47:0/608730333 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+16274 (secure 0 0 0) 0x7f8638018070 con 0x7f86480b5c60 2022-06-29T15:49:33.497 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:49:33.498 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":90,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:49:28.720705+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:49:25.785974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2022-06-29T15:49:26.726660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:49:33.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.493+0000 7f864d085700 1 -- 172.21.15.47:0/608730333 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f862c0539e0 msgr2=0x7f862c055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:33.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.493+0000 7f864d085700 1 --2- 172.21.15.47:0/608730333 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f862c0539e0 0x7f862c055ea0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f864001b000 tx=0x7f8640016000).stop 2022-06-29T15:49:33.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.493+0000 7f864d085700 1 -- 172.21.15.47:0/608730333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86480b5c60 msgr2=0x7f864813bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:33.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.493+0000 7f864d085700 1 --2- 172.21.15.47:0/608730333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86480b5c60 0x7f864813bce0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 rx=0x7f8638001110 tx=0x7f8638007a80).stop 2022-06-29T15:49:33.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.493+0000 7f864d085700 1 -- 172.21.15.47:0/608730333 shutdown_connections 2022-06-29T15:49:33.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.493+0000 7f864d085700 1 --2- 172.21.15.47:0/608730333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86480b5c60 0x7f864813bce0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:33.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.493+0000 7f864d085700 1 --2- 172.21.15.47:0/608730333 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f862c0539e0 0x7f862c055ea0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:33.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.493+0000 7f864d085700 1 --2- 172.21.15.47:0/608730333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86480a72f0 0x7f864813a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:33.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.493+0000 7f864d085700 1 --2- 172.21.15.47:0/608730333 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8648060eb0 0x7f864814d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:33.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.493+0000 7f864d085700 1 -- 172.21.15.47:0/608730333 >> 172.21.15.47:0/608730333 conn(0x7f8648062560 msgr2=0x7f86480b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:33.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.493+0000 7f864d085700 1 -- 172.21.15.47:0/608730333 shutdown_connections 2022-06-29T15:49:33.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.493+0000 7f864d085700 1 -- 172.21.15.47:0/608730333 wait complete. 2022-06-29T15:49:33.516 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-29T15:42:16.971431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-29T15:42:25.290884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-29T15:49:25.785974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2022-06-29T15:49:26.726660+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-29T15:49:33.517 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-29T15:49:33.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.613+0000 7f370bfff700 1 Processor -- start 2022-06-29T15:49:33.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.613+0000 7f370bfff700 1 -- start start 2022-06-29T15:49:33.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.613+0000 7f370bfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f370c0603d0 0x7f370c142f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:33.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.613+0000 7f370bfff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f370c060de0 0x7f370c1402a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:33.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.613+0000 7f370bfff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f370c061220 con 0x7f370c140790 2022-06-29T15:49:33.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.613+0000 7f370bfff700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f370c140be0 con 0x7f370c0603d0 2022-06-29T15:49:33.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.613+0000 7f370bfff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f370c061480 con 0x7f370c060de0 2022-06-29T15:49:33.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.613+0000 7f370affd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f370c0603d0 0x7f370c142f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:33.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.613+0000 7f370affd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f370c0603d0 0x7f370c142f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50946/0 (socket says 172.21.15.47:50946) 2022-06-29T15:49:33.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.613+0000 7f370affd700 1 -- 172.21.15.47:0/3062190648 learned_addr learned my addr 172.21.15.47:0/3062190648 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:33.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.613+0000 7f370a7fc700 1 --2- 172.21.15.47:0/3062190648 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f370c060de0 0x7f370c1402a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:33.626 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.613+0000 7f370affd700 1 -- 172.21.15.47:0/3062190648 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f370c060de0 msgr2=0x7f370c1402a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:33.626 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.613+0000 7f370affd700 1 --2- 172.21.15.47:0/3062190648 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f370c060de0 0x7f370c1402a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:33.626 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.613+0000 7f370affd700 1 -- 172.21.15.47:0/3062190648 >> v1:172.21.15.64:6789/0 conn(0x7f370c140790 legacy=0x7f370c0b0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:33.626 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.617+0000 7f370affd700 1 -- 172.21.15.47:0/3062190648 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f370c061720 con 0x7f370c0603d0 2022-06-29T15:49:33.627 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.617+0000 7f370affd700 1 --2- 172.21.15.47:0/3062190648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f370c0603d0 0x7f370c142f00 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 rx=0x7f36fc002f70 tx=0x7f36fc006eb0).ready entity=mon.0 client_cookie=f8ae64f92019313 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:33.627 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.617+0000 7f3709ffb700 1 -- 172.21.15.47:0/3062190648 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f36fc00d570 con 0x7f370c0603d0 2022-06-29T15:49:33.627 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.617+0000 7f3709ffb700 1 -- 172.21.15.47:0/3062190648 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36fc000e90 con 0x7f370c0603d0 2022-06-29T15:49:33.627 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.617+0000 7f3709ffb700 1 -- 172.21.15.47:0/3062190648 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f36fc00d900 con 0x7f370c0603d0 2022-06-29T15:49:33.631 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.617+0000 7f370bfff700 1 -- 172.21.15.47:0/3062190648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f370c0603d0 msgr2=0x7f370c142f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:33.631 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.617+0000 7f370bfff700 1 --2- 172.21.15.47:0/3062190648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f370c0603d0 0x7f370c142f00 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 rx=0x7f36fc002f70 tx=0x7f36fc006eb0).stop 2022-06-29T15:49:33.631 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.617+0000 7f370bfff700 1 -- 172.21.15.47:0/3062190648 shutdown_connections 2022-06-29T15:49:33.631 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.617+0000 7f370bfff700 1 --2- 172.21.15.47:0/3062190648 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f370c060de0 0x7f370c1402a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:33.632 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.617+0000 7f370bfff700 1 --2- 172.21.15.47:0/3062190648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f370c0603d0 0x7f370c142f00 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:33.632 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.617+0000 7f370bfff700 1 -- 172.21.15.47:0/3062190648 >> 172.21.15.47:0/3062190648 conn(0x7f370c0b6430 msgr2=0x7f370c0b6810 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:33.632 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.617+0000 7f370bfff700 1 -- 172.21.15.47:0/3062190648 shutdown_connections 2022-06-29T15:49:33.632 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.617+0000 7f370bfff700 1 -- 172.21.15.47:0/3062190648 wait complete. 2022-06-29T15:49:33.633 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.621+0000 7f370bfff700 1 Processor -- start 2022-06-29T15:49:33.633 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.621+0000 7f370bfff700 1 -- start start 2022-06-29T15:49:33.633 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.621+0000 7f370bfff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f370c0603d0 0x7f370c144620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:33.633 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.621+0000 7f370bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f370c060de0 0x7f370c155f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:33.634 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.621+0000 7f370bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f370c140790 0x7f370c0ac7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:33.634 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.621+0000 7f370bfff700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370c146e40 con 0x7f370c140790 2022-06-29T15:49:33.634 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.621+0000 7f370bfff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370c156430 con 0x7f370c060de0 2022-06-29T15:49:33.634 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.621+0000 7f370bfff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f370c0aae20 con 0x7f370c0603d0 2022-06-29T15:49:33.634 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.621+0000 7f370b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f370c140790 0x7f370c0ac7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:33.635 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.621+0000 7f370b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f370c140790 0x7f370c0ac7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50956/0 (socket says 172.21.15.47:50956) 2022-06-29T15:49:33.635 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.621+0000 7f370b7fe700 1 -- 172.21.15.47:0/4161483179 learned_addr learned my addr 172.21.15.47:0/4161483179 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:33.635 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.621+0000 7f370a7fc700 1 --2- 172.21.15.47:0/4161483179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f370c060de0 0x7f370c155f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:33.635 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.621+0000 7f370affd700 1 --2- 172.21.15.47:0/4161483179 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f370c0603d0 0x7f370c144620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:33.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.621+0000 7f370b7fe700 1 -- 172.21.15.47:0/4161483179 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f370c0603d0 msgr2=0x7f370c144620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:33.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.621+0000 7f370b7fe700 1 --2- 172.21.15.47:0/4161483179 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f370c0603d0 0x7f370c144620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:33.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.621+0000 7f370b7fe700 1 -- 172.21.15.47:0/4161483179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f370c060de0 msgr2=0x7f370c155f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:33.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.621+0000 7f370b7fe700 1 --2- 172.21.15.47:0/4161483179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f370c060de0 0x7f370c155f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:33.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.621+0000 7f370b7fe700 1 -- 172.21.15.47:0/4161483179 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36fc00b1e0 con 0x7f370c140790 2022-06-29T15:49:33.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.621+0000 7f370a7fc700 1 --2- 172.21.15.47:0/4161483179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f370c060de0 0x7f370c155f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:49:33.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.621+0000 7f370b7fe700 1 --2- 172.21.15.47:0/4161483179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f370c140790 0x7f370c0ac7e0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 rx=0x7f370c0b7490 tx=0x7f3704007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:33.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.621+0000 7f36fbfff700 1 -- 172.21.15.47:0/4161483179 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f370400e4a0 con 0x7f370c140790 2022-06-29T15:49:33.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.621+0000 7f36fbfff700 1 -- 172.21.15.47:0/4161483179 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f370400e600 con 0x7f370c140790 2022-06-29T15:49:33.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.621+0000 7f36fbfff700 1 -- 172.21.15.47:0/4161483179 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f370400e960 con 0x7f370c140790 2022-06-29T15:49:33.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.621+0000 7f370bfff700 1 -- 172.21.15.47:0/4161483179 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f370c0ab0c0 con 0x7f370c140790 2022-06-29T15:49:33.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.621+0000 7f370bfff700 1 -- 172.21.15.47:0/4161483179 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f370c0aa8a0 con 0x7f370c140790 2022-06-29T15:49:33.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.625+0000 7f36fbfff700 1 -- 172.21.15.47:0/4161483179 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3704010030 con 0x7f370c140790 2022-06-29T15:49:33.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.625+0000 7f370bfff700 1 -- 172.21.15.47:0/4161483179 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f370c0573d0 con 0x7f370c140790 2022-06-29T15:49:33.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.625+0000 7f36fbfff700 1 --2- 172.21.15.47:0/4161483179 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f36e8053b00 0x7f36e8055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:33.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.625+0000 7f36fbfff700 1 -- 172.21.15.47:0/4161483179 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f370406c250 con 0x7f370c140790 2022-06-29T15:49:33.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.625+0000 7f370affd700 1 --2- 172.21.15.47:0/4161483179 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f36e8053b00 0x7f36e8055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:33.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.625+0000 7f370affd700 1 --2- 172.21.15.47:0/4161483179 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f36e8053b00 0x7f36e8055fc0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f36fc01c000 tx=0x7f36fc015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:33.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.629+0000 7f36fbfff700 1 -- 172.21.15.47:0/4161483179 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3704046e60 con 0x7f370c140790 2022-06-29T15:49:33.763 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.757+0000 7f370bfff700 1 -- 172.21.15.47:0/4161483179 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f370c06a0f0 con 0x7f370c140790 2022-06-29T15:49:33.764 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.757+0000 7f36fbfff700 1 -- 172.21.15.47:0/4161483179 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v90) v1 ==== 110+0+10 (secure 0 0 0) 0x7f3704042f70 con 0x7f370c140790 2022-06-29T15:49:33.764 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2022-06-29T15:49:33.766 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.757+0000 7f370bfff700 1 -- 172.21.15.47:0/4161483179 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f36e8053b00 msgr2=0x7f36e8055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:33.767 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.757+0000 7f370bfff700 1 --2- 172.21.15.47:0/4161483179 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f36e8053b00 0x7f36e8055fc0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f36fc01c000 tx=0x7f36fc015000).stop 2022-06-29T15:49:33.767 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.757+0000 7f370bfff700 1 -- 172.21.15.47:0/4161483179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f370c140790 msgr2=0x7f370c0ac7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:33.767 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.757+0000 7f370bfff700 1 --2- 172.21.15.47:0/4161483179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f370c140790 0x7f370c0ac7e0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 rx=0x7f370c0b7490 tx=0x7f3704007f40).stop 2022-06-29T15:49:33.767 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.757+0000 7f370bfff700 1 -- 172.21.15.47:0/4161483179 shutdown_connections 2022-06-29T15:49:33.768 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.757+0000 7f370bfff700 1 --2- 172.21.15.47:0/4161483179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f370c140790 0x7f370c0ac7e0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:33.768 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.757+0000 7f370bfff700 1 --2- 172.21.15.47:0/4161483179 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f36e8053b00 0x7f36e8055fc0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:33.768 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.757+0000 7f370bfff700 1 --2- 172.21.15.47:0/4161483179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f370c060de0 0x7f370c155f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:33.768 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.757+0000 7f370bfff700 1 --2- 172.21.15.47:0/4161483179 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f370c0603d0 0x7f370c144620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:33.768 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.757+0000 7f370bfff700 1 -- 172.21.15.47:0/4161483179 >> 172.21.15.47:0/4161483179 conn(0x7f370c0b6430 msgr2=0x7f370c0b5270 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:33.769 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.757+0000 7f370bfff700 1 -- 172.21.15.47:0/4161483179 shutdown_connections 2022-06-29T15:49:33.769 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.757+0000 7f370bfff700 1 -- 172.21.15.47:0/4161483179 wait complete. 2022-06-29T15:49:33.778 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-29T15:49:33.891 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.881+0000 7f259f0b3700 1 Processor -- start 2022-06-29T15:49:33.891 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.881+0000 7f259f0b3700 1 -- start start 2022-06-29T15:49:33.891 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.881+0000 7f259f0b3700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f259805c850 0x7f2598143a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:33.892 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.881+0000 7f259f0b3700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f25980676a0 0x7f2598140130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:33.892 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.881+0000 7f259f0b3700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f259805cc30 con 0x7f25980b5d90 2022-06-29T15:49:33.892 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.881+0000 7f259f0b3700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25981430a0 con 0x7f25980676a0 2022-06-29T15:49:33.893 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.881+0000 7f259f0b3700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2598143300 con 0x7f259805c850 2022-06-29T15:49:33.893 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.881+0000 7f259e8b2700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f25980676a0 0x7f2598140130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:33.893 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.881+0000 7f259e8b2700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f25980676a0 0x7f2598140130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50964/0 (socket says 172.21.15.47:50964) 2022-06-29T15:49:33.893 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.881+0000 7f259e8b2700 1 -- 172.21.15.47:0/765648360 learned_addr learned my addr 172.21.15.47:0/765648360 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:33.894 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.881+0000 7f259e0b1700 1 --2- 172.21.15.47:0/765648360 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f259805c850 0x7f2598143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:33.894 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.881+0000 7f259e8b2700 1 -- 172.21.15.47:0/765648360 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f259805c850 msgr2=0x7f2598143a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:33.894 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.881+0000 7f259e8b2700 1 --2- 172.21.15.47:0/765648360 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f259805c850 0x7f2598143a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:33.894 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.881+0000 7f259e8b2700 1 -- 172.21.15.47:0/765648360 >> v1:172.21.15.47:6789/0 conn(0x7f25980b5d90 legacy=0x7f25980b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:33.895 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.881+0000 7f259e8b2700 1 -- 172.21.15.47:0/765648360 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25981435a0 con 0x7f25980676a0 2022-06-29T15:49:33.895 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.881+0000 7f259e8b2700 1 --2- 172.21.15.47:0/765648360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25980676a0 0x7f2598140130 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 rx=0x7f25900032f0 tx=0x7f2590009290).ready entity=mon.0 client_cookie=75f277f6ab7bd3a1 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:33.895 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.881+0000 7f259d0af700 1 -- 172.21.15.47:0/765648360 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f259000b8a0 con 0x7f25980676a0 2022-06-29T15:49:33.895 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.881+0000 7f259d0af700 1 -- 172.21.15.47:0/765648360 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f259000ba00 con 0x7f25980676a0 2022-06-29T15:49:33.896 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.881+0000 7f259d0af700 1 -- 172.21.15.47:0/765648360 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f259000bb70 con 0x7f25980676a0 2022-06-29T15:49:33.896 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.881+0000 7f259f0b3700 1 -- 172.21.15.47:0/765648360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25980676a0 msgr2=0x7f2598140130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:33.896 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.881+0000 7f259f0b3700 1 --2- 172.21.15.47:0/765648360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25980676a0 0x7f2598140130 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 rx=0x7f25900032f0 tx=0x7f2590009290).stop 2022-06-29T15:49:33.896 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.881+0000 7f259f0b3700 1 -- 172.21.15.47:0/765648360 shutdown_connections 2022-06-29T15:49:33.897 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.881+0000 7f259f0b3700 1 --2- 172.21.15.47:0/765648360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25980676a0 0x7f2598140130 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:33.897 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.881+0000 7f259f0b3700 1 --2- 172.21.15.47:0/765648360 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f259805c850 0x7f2598143a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:33.897 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.881+0000 7f259f0b3700 1 -- 172.21.15.47:0/765648360 >> 172.21.15.47:0/765648360 conn(0x7f2598060e00 msgr2=0x7f25980611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:33.897 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.881+0000 7f259f0b3700 1 -- 172.21.15.47:0/765648360 shutdown_connections 2022-06-29T15:49:33.898 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.881+0000 7f259f0b3700 1 -- 172.21.15.47:0/765648360 wait complete. 2022-06-29T15:49:33.902 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.885+0000 7f259f0b3700 1 Processor -- start 2022-06-29T15:49:33.902 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.885+0000 7f259f0b3700 1 -- start start 2022-06-29T15:49:33.902 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.885+0000 7f259f0b3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f259805c850 0x7f25980b3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:33.903 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.885+0000 7f259f0b3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25980676a0 0x7f2598155970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:33.903 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.885+0000 7f259f0b3700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f25980b5d90 0x7f2598155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:33.903 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.885+0000 7f259f0b3700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25980b42c0 con 0x7f259805c850 2022-06-29T15:49:33.903 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.885+0000 7f259f0b3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25980b2170 con 0x7f25980676a0 2022-06-29T15:49:33.904 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.885+0000 7f259f0b3700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25980b9ca0 con 0x7f25980b5d90 2022-06-29T15:49:33.904 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.885+0000 7f259e0b1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f259805c850 0x7f25980b3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:33.904 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.885+0000 7f259e0b1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f259805c850 0x7f25980b3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50966/0 (socket says 172.21.15.47:50966) 2022-06-29T15:49:33.904 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.885+0000 7f259e0b1700 1 -- 172.21.15.47:0/2603323259 learned_addr learned my addr 172.21.15.47:0/2603323259 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:33.905 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.885+0000 7f259d8b0700 1 --2- 172.21.15.47:0/2603323259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25980676a0 0x7f2598155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:33.905 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.885+0000 7f259e8b2700 1 --2- 172.21.15.47:0/2603323259 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f25980b5d90 0x7f2598155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:33.905 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.885+0000 7f259e0b1700 1 -- 172.21.15.47:0/2603323259 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f25980b5d90 msgr2=0x7f2598155e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:33.905 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.885+0000 7f259e0b1700 1 --2- 172.21.15.47:0/2603323259 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f25980b5d90 0x7f2598155e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:33.906 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.885+0000 7f259e0b1700 1 -- 172.21.15.47:0/2603323259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25980676a0 msgr2=0x7f2598155970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:33.906 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.885+0000 7f259e0b1700 1 --2- 172.21.15.47:0/2603323259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25980676a0 0x7f2598155970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:33.906 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.885+0000 7f259e0b1700 1 -- 172.21.15.47:0/2603323259 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2590009530 con 0x7f259805c850 2022-06-29T15:49:33.906 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.885+0000 7f259e8b2700 1 --2- 172.21.15.47:0/2603323259 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f25980b5d90 0x7f2598155e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:49:33.907 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.885+0000 7f259e0b1700 1 --2- 172.21.15.47:0/2603323259 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f259805c850 0x7f25980b3a80 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 rx=0x7f259400c000 tx=0x7f2594006d90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:33.907 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.885+0000 7f25877fe700 1 -- 172.21.15.47:0/2603323259 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2594011750 con 0x7f259805c850 2022-06-29T15:49:33.907 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.885+0000 7f25877fe700 1 -- 172.21.15.47:0/2603323259 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25940118b0 con 0x7f259805c850 2022-06-29T15:49:33.907 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.885+0000 7f259f0b3700 1 -- 172.21.15.47:0/2603323259 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25980b0ab0 con 0x7f259805c850 2022-06-29T15:49:33.908 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.885+0000 7f25877fe700 1 -- 172.21.15.47:0/2603323259 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2594011c10 con 0x7f259805c850 2022-06-29T15:49:33.908 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.885+0000 7f259f0b3700 1 -- 172.21.15.47:0/2603323259 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25980b1c80 con 0x7f259805c850 2022-06-29T15:49:33.908 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.885+0000 7f259f0b3700 1 -- 172.21.15.47:0/2603323259 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25980b9e60 con 0x7f259805c850 2022-06-29T15:49:33.909 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.889+0000 7f25877fe700 1 -- 172.21.15.47:0/2603323259 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2594011d70 con 0x7f259805c850 2022-06-29T15:49:33.909 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.889+0000 7f25877fe700 1 --2- 172.21.15.47:0/2603323259 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2578053b30 0x7f2578055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:33.909 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.889+0000 7f25877fe700 1 -- 172.21.15.47:0/2603323259 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f259406c0a0 con 0x7f259805c850 2022-06-29T15:49:33.910 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.889+0000 7f259d8b0700 1 --2- 172.21.15.47:0/2603323259 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2578053b30 0x7f2578055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:33.910 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.889+0000 7f259d8b0700 1 --2- 172.21.15.47:0/2603323259 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2578053b30 0x7f2578055ff0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7f258800b000 tx=0x7f2588009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:33.911 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:33.897+0000 7f25877fe700 1 -- 172.21.15.47:0/2603323259 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2594042e00 con 0x7f259805c850 2022-06-29T15:49:34.032 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.025+0000 7f259f0b3700 1 -- 172.21.15.47:0/2603323259 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f25980573d0 con 0x7f259805c850 2022-06-29T15:49:34.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.025+0000 7f25877fe700 1 -- 172.21.15.47:0/2603323259 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v90) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2594021250 con 0x7f259805c850 2022-06-29T15:49:34.033 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:49:34.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.025+0000 7f259f0b3700 1 -- 172.21.15.47:0/2603323259 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2578053b30 msgr2=0x7f2578055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:34.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.025+0000 7f259f0b3700 1 --2- 172.21.15.47:0/2603323259 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2578053b30 0x7f2578055ff0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7f258800b000 tx=0x7f2588009040).stop 2022-06-29T15:49:34.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.025+0000 7f259f0b3700 1 -- 172.21.15.47:0/2603323259 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f259805c850 msgr2=0x7f25980b3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:34.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.025+0000 7f259f0b3700 1 --2- 172.21.15.47:0/2603323259 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f259805c850 0x7f25980b3a80 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 rx=0x7f259400c000 tx=0x7f2594006d90).stop 2022-06-29T15:49:34.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.029+0000 7f259f0b3700 1 -- 172.21.15.47:0/2603323259 shutdown_connections 2022-06-29T15:49:34.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.029+0000 7f259f0b3700 1 --2- 172.21.15.47:0/2603323259 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f25980b5d90 0x7f2598155e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:34.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.029+0000 7f259f0b3700 1 --2- 172.21.15.47:0/2603323259 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2578053b30 0x7f2578055ff0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:34.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.029+0000 7f259f0b3700 1 --2- 172.21.15.47:0/2603323259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25980676a0 0x7f2598155970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:34.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.029+0000 7f259f0b3700 1 --2- 172.21.15.47:0/2603323259 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f259805c850 0x7f25980b3a80 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:34.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.029+0000 7f259f0b3700 1 -- 172.21.15.47:0/2603323259 >> 172.21.15.47:0/2603323259 conn(0x7f2598060e00 msgr2=0x7f2598146d80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:34.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.029+0000 7f259f0b3700 1 -- 172.21.15.47:0/2603323259 shutdown_connections 2022-06-29T15:49:34.038 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.029+0000 7f259f0b3700 1 -- 172.21.15.47:0/2603323259 wait complete. 2022-06-29T15:49:34.046 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-29T15:49:34.157 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.149+0000 7f3ec9f4d700 1 Processor -- start 2022-06-29T15:49:34.157 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.149+0000 7f3ec9f4d700 1 -- start start 2022-06-29T15:49:34.161 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.149+0000 7f3ec9f4d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3ec405f0b0 0x7f3ec405f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:34.161 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.149+0000 7f3ec9f4d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3ec4144670 0x7f3ec4061740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:34.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.149+0000 7f3ec9f4d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3ec4147140 0x7f3ec4143950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:34.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.149+0000 7f3ec9f4d700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ec4061c30 con 0x7f3ec405f0b0 2022-06-29T15:49:34.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.149+0000 7f3ec9f4d700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ec4061e50 con 0x7f3ec4144670 2022-06-29T15:49:34.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.149+0000 7f3ec9f4d700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec40620f0 con 0x7f3ec4147140 2022-06-29T15:49:34.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.149+0000 7f3ec8f4b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3ec405f0b0 0x7f3ec405f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:34.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.149+0000 7f3ec8f4b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3ec405f0b0 0x7f3ec405f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50974/0 (socket says 172.21.15.47:50974) 2022-06-29T15:49:34.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.149+0000 7f3ec8f4b700 1 -- 172.21.15.47:0/3238424286 learned_addr learned my addr 172.21.15.47:0/3238424286 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:34.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.149+0000 7f3ebbfff700 1 --2- 172.21.15.47:0/3238424286 >> v2:172.21.15.64:3300/0 conn(0x7f3ec4144670 0x7f3ec4061740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:34.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.149+0000 7f3ec974c700 1 --2- 172.21.15.47:0/3238424286 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3ec4147140 0x7f3ec4143950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:34.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.153+0000 7f3ec8f4b700 1 -- 172.21.15.47:0/3238424286 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3ec4147140 msgr2=0x7f3ec4143950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:34.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.153+0000 7f3ec8f4b700 1 --2- 172.21.15.47:0/3238424286 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3ec4147140 0x7f3ec4143950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:34.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.153+0000 7f3ec8f4b700 1 -- 172.21.15.47:0/3238424286 >> v2:172.21.15.64:3300/0 conn(0x7f3ec4144670 msgr2=0x7f3ec4061740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:34.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.153+0000 7f3ec8f4b700 1 --2- 172.21.15.47:0/3238424286 >> v2:172.21.15.64:3300/0 conn(0x7f3ec4144670 0x7f3ec4061740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:34.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.153+0000 7f3ec8f4b700 1 -- 172.21.15.47:0/3238424286 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ec41434b0 con 0x7f3ec405f0b0 2022-06-29T15:49:34.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.153+0000 7f3ec8f4b700 1 --2- 172.21.15.47:0/3238424286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ec405f0b0 0x7f3ec405f490 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 rx=0x7f3ec0002f70 tx=0x7f3ec0006eb0).ready entity=mon.0 client_cookie=63b190beae670112 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:34.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.153+0000 7f3ebb7fe700 1 -- 172.21.15.47:0/3238424286 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ec000d570 con 0x7f3ec405f0b0 2022-06-29T15:49:34.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.153+0000 7f3ebb7fe700 1 -- 172.21.15.47:0/3238424286 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ec0000e90 con 0x7f3ec405f0b0 2022-06-29T15:49:34.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.153+0000 7f3ebb7fe700 1 -- 172.21.15.47:0/3238424286 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ec000d900 con 0x7f3ec405f0b0 2022-06-29T15:49:34.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.153+0000 7f3ec9f4d700 1 -- 172.21.15.47:0/3238424286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ec405f0b0 msgr2=0x7f3ec405f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:34.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.153+0000 7f3ec9f4d700 1 --2- 172.21.15.47:0/3238424286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ec405f0b0 0x7f3ec405f490 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 rx=0x7f3ec0002f70 tx=0x7f3ec0006eb0).stop 2022-06-29T15:49:34.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.153+0000 7f3ec9f4d700 1 -- 172.21.15.47:0/3238424286 shutdown_connections 2022-06-29T15:49:34.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.153+0000 7f3ec9f4d700 1 --2- 172.21.15.47:0/3238424286 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3ec4147140 0x7f3ec4143950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:34.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.153+0000 7f3ec9f4d700 1 --2- 172.21.15.47:0/3238424286 >> v2:172.21.15.64:3300/0 conn(0x7f3ec4144670 0x7f3ec4061740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:34.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.153+0000 7f3ec9f4d700 1 --2- 172.21.15.47:0/3238424286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ec405f0b0 0x7f3ec405f490 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:34.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.153+0000 7f3ec9f4d700 1 -- 172.21.15.47:0/3238424286 >> 172.21.15.47:0/3238424286 conn(0x7f3ec4062570 msgr2=0x7f3ec40647d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:34.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.153+0000 7f3ec9f4d700 1 -- 172.21.15.47:0/3238424286 shutdown_connections 2022-06-29T15:49:34.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.153+0000 7f3ec9f4d700 1 -- 172.21.15.47:0/3238424286 wait complete. 2022-06-29T15:49:34.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.157+0000 7f3ec9f4d700 1 Processor -- start 2022-06-29T15:49:34.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.157+0000 7f3ec9f4d700 1 -- start start 2022-06-29T15:49:34.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.157+0000 7f3ec9f4d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3ec405f0b0 0x7f3ec40b6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:34.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.157+0000 7f3ec9f4d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ec4144670 0x7f3ec4152080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:34.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.157+0000 7f3ec9f4d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ec4147140 0x7f3ec4152570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:34.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.157+0000 7f3ec9f4d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec40a2f90 con 0x7f3ec4147140 2022-06-29T15:49:34.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.157+0000 7f3ec9f4d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec4152a60 con 0x7f3ec4144670 2022-06-29T15:49:34.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.157+0000 7f3ec9f4d700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec4155670 con 0x7f3ec405f0b0 2022-06-29T15:49:34.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.157+0000 7f3ec974c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ec4147140 0x7f3ec4152570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:34.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.157+0000 7f3ebbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ec4144670 0x7f3ec4152080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:34.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.157+0000 7f3ec8f4b700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3ec405f0b0 0x7f3ec40b6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:34.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.157+0000 7f3ebbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ec4144670 0x7f3ec4152080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:56684/0 (socket says 172.21.15.47:56684) 2022-06-29T15:49:34.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.157+0000 7f3ebbfff700 1 -- 172.21.15.47:0/1584871240 learned_addr learned my addr 172.21.15.47:0/1584871240 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:34.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.157+0000 7f3ec974c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ec4147140 0x7f3ec4152570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50982/0 (socket says 172.21.15.47:50982) 2022-06-29T15:49:34.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.157+0000 7f3ec974c700 1 -- 172.21.15.47:0/1584871240 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3ec405f0b0 msgr2=0x7f3ec40b6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:34.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.157+0000 7f3ec974c700 1 --2- 172.21.15.47:0/1584871240 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3ec405f0b0 0x7f3ec40b6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:34.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.161+0000 7f3ec974c700 1 -- 172.21.15.47:0/1584871240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ec4144670 msgr2=0x7f3ec4152080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:34.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.161+0000 7f3ec974c700 1 --2- 172.21.15.47:0/1584871240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ec4144670 0x7f3ec4152080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:34.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.161+0000 7f3ec974c700 1 -- 172.21.15.47:0/1584871240 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ec000b1e0 con 0x7f3ec4147140 2022-06-29T15:49:34.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.161+0000 7f3ebbfff700 1 --2- 172.21.15.47:0/1584871240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ec4144670 0x7f3ec4152080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:49:34.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.161+0000 7f3ec8f4b700 1 --2- 172.21.15.47:0/1584871240 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3ec405f0b0 0x7f3ec40b6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:49:34.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.161+0000 7f3ec974c700 1 --2- 172.21.15.47:0/1584871240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ec4147140 0x7f3ec4152570 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 rx=0x7f3ebc00c000 tx=0x7f3ebc0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:34.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.161+0000 7f3eb9ffb700 1 -- 172.21.15.47:0/1584871240 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ebc012600 con 0x7f3ec4147140 2022-06-29T15:49:34.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.161+0000 7f3eb9ffb700 1 -- 172.21.15.47:0/1584871240 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ebc012760 con 0x7f3ec4147140 2022-06-29T15:49:34.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.161+0000 7f3eb9ffb700 1 -- 172.21.15.47:0/1584871240 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ebc012ac0 con 0x7f3ec4147140 2022-06-29T15:49:34.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.161+0000 7f3ec9f4d700 1 -- 172.21.15.47:0/1584871240 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ec4155910 con 0x7f3ec4147140 2022-06-29T15:49:34.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.161+0000 7f3ec9f4d700 1 -- 172.21.15.47:0/1584871240 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ec4155e50 con 0x7f3ec4147140 2022-06-29T15:49:34.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.161+0000 7f3ec9f4d700 1 -- 172.21.15.47:0/1584871240 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ec4156010 con 0x7f3ec4147140 2022-06-29T15:49:34.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.161+0000 7f3eb9ffb700 1 -- 172.21.15.47:0/1584871240 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3ebc022070 con 0x7f3ec4147140 2022-06-29T15:49:34.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.165+0000 7f3eb9ffb700 1 --2- 172.21.15.47:0/1584871240 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3ea40539e0 0x7f3ea4055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:34.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.165+0000 7f3eb9ffb700 1 -- 172.21.15.47:0/1584871240 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f3ebc06c790 con 0x7f3ec4147140 2022-06-29T15:49:34.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.165+0000 7f3ec8f4b700 1 --2- 172.21.15.47:0/1584871240 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3ea40539e0 0x7f3ea4055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:34.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.165+0000 7f3ec8f4b700 1 --2- 172.21.15.47:0/1584871240 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3ea40539e0 0x7f3ea4055ea0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7f3ec001c000 tx=0x7f3ec0015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:34.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.173+0000 7f3eb9ffb700 1 -- 172.21.15.47:0/1584871240 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3ebc043440 con 0x7f3ec4147140 2022-06-29T15:49:34.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.297+0000 7f3ec9f4d700 1 -- 172.21.15.47:0/1584871240 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3ec40a4dd0 con 0x7f3ec4147140 2022-06-29T15:49:34.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.301+0000 7f3eb9ffb700 1 -- 172.21.15.47:0/1584871240 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+10 (secure 0 0 0) 0x7f3ebc0160a0 con 0x7f3ec4147140 2022-06-29T15:49:34.307 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:49:34.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.301+0000 7f3ec9f4d700 1 -- 172.21.15.47:0/1584871240 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3ea40539e0 msgr2=0x7f3ea4055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:34.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.301+0000 7f3ec9f4d700 1 --2- 172.21.15.47:0/1584871240 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3ea40539e0 0x7f3ea4055ea0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7f3ec001c000 tx=0x7f3ec0015000).stop 2022-06-29T15:49:34.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.301+0000 7f3ec9f4d700 1 -- 172.21.15.47:0/1584871240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ec4147140 msgr2=0x7f3ec4152570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:34.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.301+0000 7f3ec9f4d700 1 --2- 172.21.15.47:0/1584871240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ec4147140 0x7f3ec4152570 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 rx=0x7f3ebc00c000 tx=0x7f3ebc0091d0).stop 2022-06-29T15:49:34.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.301+0000 7f3ec9f4d700 1 -- 172.21.15.47:0/1584871240 shutdown_connections 2022-06-29T15:49:34.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.301+0000 7f3ec9f4d700 1 --2- 172.21.15.47:0/1584871240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ec4147140 0x7f3ec4152570 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:34.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.301+0000 7f3ec9f4d700 1 --2- 172.21.15.47:0/1584871240 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3ea40539e0 0x7f3ea4055ea0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:34.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.301+0000 7f3ec9f4d700 1 --2- 172.21.15.47:0/1584871240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ec4144670 0x7f3ec4152080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:34.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.301+0000 7f3ec9f4d700 1 --2- 172.21.15.47:0/1584871240 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3ec405f0b0 0x7f3ec40b6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:34.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.301+0000 7f3ec9f4d700 1 -- 172.21.15.47:0/1584871240 >> 172.21.15.47:0/1584871240 conn(0x7f3ec4062570 msgr2=0x7f3ec4142120 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:34.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.301+0000 7f3ec9f4d700 1 -- 172.21.15.47:0/1584871240 shutdown_connections 2022-06-29T15:49:34.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.301+0000 7f3ec9f4d700 1 -- 172.21.15.47:0/1584871240 wait complete. 2022-06-29T15:49:34.321 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-29T15:49:34.437 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.429+0000 7f574bfff700 1 Processor -- start 2022-06-29T15:49:34.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.429+0000 7f574bfff700 1 -- start start 2022-06-29T15:49:34.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.433+0000 7f574bfff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f574c0b6b80 0x7f574c140f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:34.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.433+0000 7f574bfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f574c13e9c0 0x7f574c0b37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:34.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.433+0000 7f574bfff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f574c145000 con 0x7f574c063090 2022-06-29T15:49:34.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.433+0000 7f574bfff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f574c1452a0 con 0x7f574c13e9c0 2022-06-29T15:49:34.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.433+0000 7f574bfff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f574c0b2300 con 0x7f574c0b6b80 2022-06-29T15:49:34.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.433+0000 7f574affd700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f574c0b6b80 0x7f574c140f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:34.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.433+0000 7f574a7fc700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f574c063090 0x7f574c13e2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:44106/0 (socket says 172.21.15.47:44106) 2022-06-29T15:49:34.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.433+0000 7f574a7fc700 1 -- 172.21.15.47:0/272657708 learned_addr learned my addr 172.21.15.47:0/272657708 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:49:34.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.433+0000 7f574b7fe700 1 --2- 172.21.15.47:0/272657708 >> v2:172.21.15.64:3300/0 conn(0x7f574c13e9c0 0x7f574c0b37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:34.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.433+0000 7f574affd700 1 -- 172.21.15.47:0/272657708 >> v2:172.21.15.64:3300/0 conn(0x7f574c13e9c0 msgr2=0x7f574c0b37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:34.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.433+0000 7f574affd700 1 --2- 172.21.15.47:0/272657708 >> v2:172.21.15.64:3300/0 conn(0x7f574c13e9c0 0x7f574c0b37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:34.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.433+0000 7f574affd700 1 -- 172.21.15.47:0/272657708 >> v1:172.21.15.64:6789/0 conn(0x7f574c063090 legacy=0x7f574c13e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:34.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.433+0000 7f574affd700 1 -- 172.21.15.47:0/272657708 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f574c0b3ca0 con 0x7f574c0b6b80 2022-06-29T15:49:34.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.433+0000 7f574affd700 1 --2- 172.21.15.47:0/272657708 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f574c0b6b80 0x7f574c140f80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f573c002f70 tx=0x7f573c006df0).ready entity=mon.2 client_cookie=b07a070574e09a4d server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:34.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.437+0000 7f5749ffb700 1 -- 172.21.15.47:0/272657708 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f573c00d570 con 0x7f574c0b6b80 2022-06-29T15:49:34.446 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.437+0000 7f5749ffb700 1 -- 172.21.15.47:0/272657708 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f573c000e90 con 0x7f574c0b6b80 2022-06-29T15:49:34.446 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.437+0000 7f5749ffb700 1 -- 172.21.15.47:0/272657708 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f573c00d8c0 con 0x7f574c0b6b80 2022-06-29T15:49:34.446 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.437+0000 7f574bfff700 1 -- 172.21.15.47:0/272657708 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f574c0b6b80 msgr2=0x7f574c140f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:34.446 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.437+0000 7f574bfff700 1 --2- 172.21.15.47:0/272657708 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f574c0b6b80 0x7f574c140f80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f573c002f70 tx=0x7f573c006df0).stop 2022-06-29T15:49:34.447 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.437+0000 7f574bfff700 1 -- 172.21.15.47:0/272657708 shutdown_connections 2022-06-29T15:49:34.447 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.437+0000 7f574bfff700 1 --2- 172.21.15.47:0/272657708 >> v2:172.21.15.64:3300/0 conn(0x7f574c13e9c0 0x7f574c0b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:34.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.437+0000 7f574bfff700 1 --2- 172.21.15.47:0/272657708 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f574c0b6b80 0x7f574c140f80 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:34.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.437+0000 7f574bfff700 1 -- 172.21.15.47:0/272657708 >> 172.21.15.47:0/272657708 conn(0x7f574c063670 msgr2=0x7f574c063a50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:34.449 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.437+0000 7f574bfff700 1 -- 172.21.15.47:0/272657708 shutdown_connections 2022-06-29T15:49:34.449 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.437+0000 7f574bfff700 1 -- 172.21.15.47:0/272657708 wait complete. 2022-06-29T15:49:34.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.437+0000 7f574bfff700 1 Processor -- start 2022-06-29T15:49:34.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.437+0000 7f574bfff700 1 -- start start 2022-06-29T15:49:34.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.441+0000 7f574bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f574c063090 0x7f574c152840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:34.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.441+0000 7f574bfff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f574c0b6b80 0x7f574c152d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:34.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.441+0000 7f574bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f574c13e9c0 0x7f574c064370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:34.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.441+0000 7f574bfff700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574c144350 con 0x7f574c13e9c0 2022-06-29T15:49:34.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.441+0000 7f574bfff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574c155810 con 0x7f574c063090 2022-06-29T15:49:34.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.441+0000 7f574bfff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f574c155ab0 con 0x7f574c0b6b80 2022-06-29T15:49:34.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.441+0000 7f574b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f574c13e9c0 0x7f574c064370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:34.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.441+0000 7f574b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f574c13e9c0 0x7f574c064370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50996/0 (socket says 172.21.15.47:50996) 2022-06-29T15:49:34.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.441+0000 7f574b7fe700 1 -- 172.21.15.47:0/3750034883 learned_addr learned my addr 172.21.15.47:0/3750034883 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:34.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.441+0000 7f574a7fc700 1 --2- 172.21.15.47:0/3750034883 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f574c0b6b80 0x7f574c152d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:34.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.441+0000 7f574affd700 1 --2- 172.21.15.47:0/3750034883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f574c063090 0x7f574c152840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:34.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.441+0000 7f574b7fe700 1 -- 172.21.15.47:0/3750034883 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f574c0b6b80 msgr2=0x7f574c152d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:34.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.441+0000 7f574b7fe700 1 --2- 172.21.15.47:0/3750034883 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f574c0b6b80 0x7f574c152d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:34.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.441+0000 7f574b7fe700 1 -- 172.21.15.47:0/3750034883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f574c063090 msgr2=0x7f574c152840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:34.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.441+0000 7f574b7fe700 1 --2- 172.21.15.47:0/3750034883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f574c063090 0x7f574c152840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:34.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.441+0000 7f574b7fe700 1 -- 172.21.15.47:0/3750034883 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f573c00b180 con 0x7f574c13e9c0 2022-06-29T15:49:34.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.441+0000 7f574b7fe700 1 --2- 172.21.15.47:0/3750034883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f574c13e9c0 0x7f574c064370 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 rx=0x7f574400c000 tx=0x7f57440091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:34.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.441+0000 7f573bfff700 1 -- 172.21.15.47:0/3750034883 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5744012600 con 0x7f574c13e9c0 2022-06-29T15:49:34.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.441+0000 7f573bfff700 1 -- 172.21.15.47:0/3750034883 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5744012760 con 0x7f574c13e9c0 2022-06-29T15:49:34.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.441+0000 7f574bfff700 1 -- 172.21.15.47:0/3750034883 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f574c155d50 con 0x7f574c13e9c0 2022-06-29T15:49:34.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.441+0000 7f573bfff700 1 -- 172.21.15.47:0/3750034883 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5744012ac0 con 0x7f574c13e9c0 2022-06-29T15:49:34.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.441+0000 7f574bfff700 1 -- 172.21.15.47:0/3750034883 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f574c147110 con 0x7f574c13e9c0 2022-06-29T15:49:34.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.441+0000 7f574bfff700 1 -- 172.21.15.47:0/3750034883 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f574c0573d0 con 0x7f574c13e9c0 2022-06-29T15:49:34.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.445+0000 7f573bfff700 1 -- 172.21.15.47:0/3750034883 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5744022070 con 0x7f574c13e9c0 2022-06-29T15:49:34.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.445+0000 7f573bfff700 1 --2- 172.21.15.47:0/3750034883 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5728053a70 0x7f5728055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:34.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.445+0000 7f573bfff700 1 -- 172.21.15.47:0/3750034883 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f574406c710 con 0x7f574c13e9c0 2022-06-29T15:49:34.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.445+0000 7f574affd700 1 --2- 172.21.15.47:0/3750034883 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5728053a70 0x7f5728055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:34.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.445+0000 7f574affd700 1 --2- 172.21.15.47:0/3750034883 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5728053a70 0x7f5728055f30 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f573c01a000 tx=0x7f573c014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:34.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.453+0000 7f573bfff700 1 -- 172.21.15.47:0/3750034883 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5744043390 con 0x7f574c13e9c0 2022-06-29T15:49:34.586 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.577+0000 7f574bfff700 1 -- 172.21.15.47:0/3750034883 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f574c069e20 con 0x7f574c13e9c0 2022-06-29T15:49:34.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.577+0000 7f573bfff700 1 -- 172.21.15.47:0/3750034883 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+10 (secure 0 0 0) 0x7f57440160a0 con 0x7f574c13e9c0 2022-06-29T15:49:34.588 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:49:34.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.581+0000 7f574bfff700 1 -- 172.21.15.47:0/3750034883 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5728053a70 msgr2=0x7f5728055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:34.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.581+0000 7f574bfff700 1 --2- 172.21.15.47:0/3750034883 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5728053a70 0x7f5728055f30 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f573c01a000 tx=0x7f573c014000).stop 2022-06-29T15:49:34.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.581+0000 7f574bfff700 1 -- 172.21.15.47:0/3750034883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f574c13e9c0 msgr2=0x7f574c064370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:34.597 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.581+0000 7f574bfff700 1 --2- 172.21.15.47:0/3750034883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f574c13e9c0 0x7f574c064370 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 rx=0x7f574400c000 tx=0x7f57440091d0).stop 2022-06-29T15:49:34.597 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.581+0000 7f574bfff700 1 -- 172.21.15.47:0/3750034883 shutdown_connections 2022-06-29T15:49:34.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.581+0000 7f574bfff700 1 --2- 172.21.15.47:0/3750034883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f574c13e9c0 0x7f574c064370 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:34.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.581+0000 7f574bfff700 1 --2- 172.21.15.47:0/3750034883 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f574c0b6b80 0x7f574c152d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:34.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.581+0000 7f574bfff700 1 --2- 172.21.15.47:0/3750034883 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5728053a70 0x7f5728055f30 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:34.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.581+0000 7f574bfff700 1 --2- 172.21.15.47:0/3750034883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f574c063090 0x7f574c152840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:34.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.581+0000 7f574bfff700 1 -- 172.21.15.47:0/3750034883 >> 172.21.15.47:0/3750034883 conn(0x7f574c063670 msgr2=0x7f574c0b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:34.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.581+0000 7f574bfff700 1 -- 172.21.15.47:0/3750034883 shutdown_connections 2022-06-29T15:49:34.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.581+0000 7f574bfff700 1 -- 172.21.15.47:0/3750034883 wait complete. 2022-06-29T15:49:34.606 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-29T15:49:34.608 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:49:34.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.701+0000 7fa139d12700 1 Processor -- start 2022-06-29T15:49:34.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.701+0000 7fa139d12700 1 -- start start 2022-06-29T15:49:34.714 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.705+0000 7fa139d12700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa13405f0b0 0x7fa1340b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:34.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.705+0000 7fa139d12700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa1340b1260 0x7fa13414b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:34.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.705+0000 7fa139d12700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1340b0740 con 0x7fa1340b2120 2022-06-29T15:49:34.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.705+0000 7fa139d12700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa134147460 con 0x7fa13405f0b0 2022-06-29T15:49:34.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.705+0000 7fa139d12700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1340b0970 con 0x7fa1340b1260 2022-06-29T15:49:34.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.705+0000 7fa1337fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa1340b2120 0x7fa134147af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36556/0 (socket says 172.21.15.47:36556) 2022-06-29T15:49:34.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.705+0000 7fa132ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa13405f0b0 0x7fa1340b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:34.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.705+0000 7fa1337fe700 1 -- 172.21.15.47:0/2333592712 learned_addr learned my addr 172.21.15.47:0/2333592712 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:49:34.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.705+0000 7fa133fff700 1 --2- 172.21.15.47:0/2333592712 >> v2:172.21.15.64:3300/0 conn(0x7fa1340b1260 0x7fa13414b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:34.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.705+0000 7fa1327fc700 1 -- 172.21.15.47:0/2333592712 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 795565347 0 0) 0x7fa1340b0740 con 0x7fa1340b2120 2022-06-29T15:49:34.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.705+0000 7fa1327fc700 1 -- 172.21.15.47:0/2333592712 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa120003260 con 0x7fa1340b2120 2022-06-29T15:49:34.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.705+0000 7fa132ffd700 1 -- 172.21.15.47:0/2333592712 >> v2:172.21.15.64:3300/0 conn(0x7fa1340b1260 msgr2=0x7fa13414b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:34.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.705+0000 7fa132ffd700 1 --2- 172.21.15.47:0/2333592712 >> v2:172.21.15.64:3300/0 conn(0x7fa1340b1260 0x7fa13414b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:34.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.705+0000 7fa132ffd700 1 -- 172.21.15.47:0/2333592712 >> v1:172.21.15.47:6789/0 conn(0x7fa1340b2120 legacy=0x7fa134147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:34.718 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.705+0000 7fa132ffd700 1 -- 172.21.15.47:0/2333592712 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1340b0c10 con 0x7fa13405f0b0 2022-06-29T15:49:34.718 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.705+0000 7fa132ffd700 1 --2- 172.21.15.47:0/2333592712 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa13405f0b0 0x7fa1340b2500 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 rx=0x7fa134141a70 tx=0x7fa124006df0).ready entity=mon.0 client_cookie=c82987779c56acd9 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:34.718 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.705+0000 7fa1327fc700 1 -- 172.21.15.47:0/2333592712 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa12400d4a0 con 0x7fa13405f0b0 2022-06-29T15:49:34.719 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.705+0000 7fa1327fc700 1 -- 172.21.15.47:0/2333592712 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa124000e90 con 0x7fa13405f0b0 2022-06-29T15:49:34.719 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.705+0000 7fa1327fc700 1 -- 172.21.15.47:0/2333592712 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa12400d7f0 con 0x7fa13405f0b0 2022-06-29T15:49:34.719 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.705+0000 7fa139d12700 1 -- 172.21.15.47:0/2333592712 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa13405f0b0 msgr2=0x7fa1340b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:34.719 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.705+0000 7fa139d12700 1 --2- 172.21.15.47:0/2333592712 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa13405f0b0 0x7fa1340b2500 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 rx=0x7fa134141a70 tx=0x7fa124006df0).stop 2022-06-29T15:49:34.720 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.705+0000 7fa139d12700 1 -- 172.21.15.47:0/2333592712 shutdown_connections 2022-06-29T15:49:34.720 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.705+0000 7fa139d12700 1 --2- 172.21.15.47:0/2333592712 >> v2:172.21.15.64:3300/0 conn(0x7fa1340b1260 0x7fa13414b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:34.720 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.705+0000 7fa139d12700 1 --2- 172.21.15.47:0/2333592712 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa13405f0b0 0x7fa1340b2500 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:34.720 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.705+0000 7fa139d12700 1 -- 172.21.15.47:0/2333592712 >> 172.21.15.47:0/2333592712 conn(0x7fa1340626f0 msgr2=0x7fa134062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:34.721 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.705+0000 7fa139d12700 1 -- 172.21.15.47:0/2333592712 shutdown_connections 2022-06-29T15:49:34.721 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.705+0000 7fa139d12700 1 -- 172.21.15.47:0/2333592712 wait complete. 2022-06-29T15:49:34.724 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.709+0000 7fa139d12700 1 Processor -- start 2022-06-29T15:49:34.724 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.709+0000 7fa139d12700 1 -- start start 2022-06-29T15:49:34.724 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.709+0000 7fa139d12700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa1340b1260 0x7fa134149af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:34.724 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.709+0000 7fa139d12700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1340b2120 0x7fa134143b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:34.725 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.709+0000 7fa139d12700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1340af450 0x7fa134144000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:34.725 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.709+0000 7fa139d12700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa134141c30 con 0x7fa1340af450 2022-06-29T15:49:34.725 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.709+0000 7fa139d12700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa134141ed0 con 0x7fa1340b2120 2022-06-29T15:49:34.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.709+0000 7fa139d12700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa134142170 con 0x7fa1340b1260 2022-06-29T15:49:34.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.713+0000 7fa133fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1340af450 0x7fa134144000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:34.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.713+0000 7fa132ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1340b2120 0x7fa134143b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:34.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.713+0000 7fa133fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1340af450 0x7fa134144000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51010/0 (socket says 172.21.15.47:51010) 2022-06-29T15:49:34.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.713+0000 7fa133fff700 1 -- 172.21.15.47:0/1098503903 learned_addr learned my addr 172.21.15.47:0/1098503903 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:34.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.713+0000 7fa1337fe700 1 --2- 172.21.15.47:0/1098503903 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa1340b1260 0x7fa134149af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:34.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.713+0000 7fa133fff700 1 -- 172.21.15.47:0/1098503903 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa1340b1260 msgr2=0x7fa134149af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:34.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.713+0000 7fa133fff700 1 --2- 172.21.15.47:0/1098503903 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa1340b1260 0x7fa134149af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:34.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.713+0000 7fa133fff700 1 -- 172.21.15.47:0/1098503903 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1340b2120 msgr2=0x7fa134143b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:34.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.713+0000 7fa133fff700 1 --2- 172.21.15.47:0/1098503903 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1340b2120 0x7fa134143b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:34.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.713+0000 7fa133fff700 1 -- 172.21.15.47:0/1098503903 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa12400b180 con 0x7fa1340af450 2022-06-29T15:49:34.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.713+0000 7fa133fff700 1 --2- 172.21.15.47:0/1098503903 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1340af450 0x7fa134144000 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 rx=0x7fa12c00c000 tx=0x7fa12c0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:34.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.713+0000 7fa130ff9700 1 -- 172.21.15.47:0/1098503903 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa12c012600 con 0x7fa1340af450 2022-06-29T15:49:34.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.713+0000 7fa130ff9700 1 -- 172.21.15.47:0/1098503903 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa12c012760 con 0x7fa1340af450 2022-06-29T15:49:34.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.713+0000 7fa139d12700 1 -- 172.21.15.47:0/1098503903 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa134142410 con 0x7fa1340af450 2022-06-29T15:49:34.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.713+0000 7fa130ff9700 1 -- 172.21.15.47:0/1098503903 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa12c012ac0 con 0x7fa1340af450 2022-06-29T15:49:34.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.713+0000 7fa139d12700 1 -- 172.21.15.47:0/1098503903 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1340adcd0 con 0x7fa1340af450 2022-06-29T15:49:34.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.713+0000 7fa139d12700 1 -- 172.21.15.47:0/1098503903 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1340573d0 con 0x7fa1340af450 2022-06-29T15:49:34.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.713+0000 7fa130ff9700 1 -- 172.21.15.47:0/1098503903 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa12c022070 con 0x7fa1340af450 2022-06-29T15:49:34.731 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.713+0000 7fa130ff9700 1 --2- 172.21.15.47:0/1098503903 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa114053980 0x7fa114055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:34.731 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.717+0000 7fa130ff9700 1 -- 172.21.15.47:0/1098503903 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa12c06c720 con 0x7fa1340af450 2022-06-29T15:49:34.731 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.717+0000 7fa1337fe700 1 --2- 172.21.15.47:0/1098503903 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa114053980 0x7fa114055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:34.731 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.717+0000 7fa1337fe700 1 --2- 172.21.15.47:0/1098503903 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa114053980 0x7fa114055e40 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7fa12800d000 tx=0x7fa128009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:34.732 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.721+0000 7fa130ff9700 1 -- 172.21.15.47:0/1098503903 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa12c0433d0 con 0x7fa1340af450 2022-06-29T15:49:34.897 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.889+0000 7fa139d12700 1 -- 172.21.15.47:0/1098503903 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa13406a1a0 con 0x7fa1340af450 2022-06-29T15:49:34.897 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.889+0000 7fa130ff9700 1 -- 172.21.15.47:0/1098503903 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+2493 (secure 0 0 0) 0x7fa12c016020 con 0x7fa1340af450 2022-06-29T15:49:34.898 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:49:34.898 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":40,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6577,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/4286932616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":4286932616},{"type":"v1","addr":"172.21.15.64:6833","nonce":4286932616}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":38},{"gid":6657,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3528102962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3528102962},{"type":"v1","addr":"172.21.15.47:6835","nonce":3528102962}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":38}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:49:27.717245+0000","modified":"2022-06-29T15:49:28.719969+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6660},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6660":{"gid":6660,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6837/4146905199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":4146905199},{"type":"v1","addr":"172.21.15.47:6837","nonce":4146905199}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2022-06-29T15:49:34.901 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.893+0000 7fa139d12700 1 -- 172.21.15.47:0/1098503903 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa114053980 msgr2=0x7fa114055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:34.902 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.893+0000 7fa139d12700 1 --2- 172.21.15.47:0/1098503903 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa114053980 0x7fa114055e40 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7fa12800d000 tx=0x7fa128009040).stop 2022-06-29T15:49:34.902 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.893+0000 7fa139d12700 1 -- 172.21.15.47:0/1098503903 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1340af450 msgr2=0x7fa134144000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:34.902 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.893+0000 7fa139d12700 1 --2- 172.21.15.47:0/1098503903 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1340af450 0x7fa134144000 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 rx=0x7fa12c00c000 tx=0x7fa12c0091d0).stop 2022-06-29T15:49:34.903 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.893+0000 7fa139d12700 1 -- 172.21.15.47:0/1098503903 shutdown_connections 2022-06-29T15:49:34.903 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.893+0000 7fa139d12700 1 --2- 172.21.15.47:0/1098503903 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1340af450 0x7fa134144000 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:34.903 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.893+0000 7fa139d12700 1 --2- 172.21.15.47:0/1098503903 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa114053980 0x7fa114055e40 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:34.904 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.893+0000 7fa139d12700 1 --2- 172.21.15.47:0/1098503903 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1340b2120 0x7fa134143b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:34.904 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.893+0000 7fa139d12700 1 --2- 172.21.15.47:0/1098503903 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa1340b1260 0x7fa134149af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:34.904 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.893+0000 7fa139d12700 1 -- 172.21.15.47:0/1098503903 >> 172.21.15.47:0/1098503903 conn(0x7fa1340626f0 msgr2=0x7fa1340b4850 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:34.904 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.893+0000 7fa139d12700 1 -- 172.21.15.47:0/1098503903 shutdown_connections 2022-06-29T15:49:34.904 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:34.893+0000 7fa139d12700 1 -- 172.21.15.47:0/1098503903 wait complete. 2022-06-29T15:49:34.905 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 40 2022-06-29T15:49:35.215 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 40, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-29T15:49:27.717245+0000', 'modified': '2022-06-29T15:49:28.719969+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6660}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6660': {'gid': 6660, 'name': 'a', 'rank': 0, 'incarnation': 39, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.47:6837/4146905199', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6836', 'nonce': 4146905199}, {'type': 'v1', 'addr': '172.21.15.47:6837', 'nonce': 4146905199}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-29T15:49:35.216 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-29T15:49:35.216 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-06-29T15:49:35.335 INFO:teuthology.orchestra.run.smithi047.stdout:{"cluster_fsid":"9a68894a-53ce-400e-838b-14eab91b834c","whoami":0,"id":6660,"want_state":"up:active","state":"up:active","rank_uptime":7.6020226080000004,"mdsmap_epoch":40,"osdmap_epoch":90,"osdmap_epoch_barrier":89,"uptime":12.292428544} 2022-06-29T15:49:35.335 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9a68894a-53ce-400e-838b-14eab91b834c", "id": 6660, "mdsmap_epoch": 40, "osdmap_epoch": 90, "osdmap_epoch_barrier": 89, "rank_uptime": 7.602022608, "state": "up:active", "uptime": 12.292428544, "want_state": "up:active", "whoami": 0 } 2022-06-29T15:49:35.336 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-29T15:49:35.336 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2022-06-29T15:49:35.337 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi164.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2022-06-29T15:49:35.337 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:49:35.337 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:49:35.346 DEBUG:teuthology.orchestra.run.smithi164:> sudo modprobe fuse 2022-06-29T15:49:35.364 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:49:35.364 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:49:35.421 INFO:teuthology.orchestra.run.smithi164.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:49:35.423 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:49:35.424 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:49:35.424 DEBUG:teuthology.orchestra.run.smithi164:> ls /sys/fs/fuse/connections 2022-06-29T15:49:35.431 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-29T15:49:35.431 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:49:35.434 DEBUG:teuthology.orchestra.run.smithi164:> sudo modprobe fuse 2022-06-29T15:49:35.450 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:49:35.451 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:49:35.460 INFO:teuthology.orchestra.run.smithi164.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:49:35.461 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:49:35.462 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:49:35.463 DEBUG:teuthology.orchestra.run.smithi164:> ls /sys/fs/fuse/connections 2022-06-29T15:49:35.509 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi164.stderr:2022-06-29T15:49:35.499+0000 7f94461ef080 -1 init, newargv = 0x55d95768c4e0 newargc=9ceph-fuse[ 2022-06-29T15:49:35.509 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi164.stderr:12560]: starting ceph client 2022-06-29T15:49:35.520 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi164.stderr:ceph-fuse[12560]: starting fuse 2022-06-29T15:49:36.614 DEBUG:teuthology.orchestra.run.smithi164:> sudo modprobe fuse 2022-06-29T15:49:36.634 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:49:36.635 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:49:36.694 INFO:teuthology.orchestra.run.smithi164.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:49:36.695 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:49:36.696 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:49:36.696 DEBUG:teuthology.orchestra.run.smithi164:> ls /sys/fs/fuse/connections 2022-06-29T15:49:36.703 INFO:teuthology.orchestra.run.smithi164.stdout:52 2022-06-29T15:49:36.704 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-29T15:49:36.704 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:49:36.705 DEBUG:teuthology.orchestra.run.smithi164:> sudo python3 -c ' 2022-06-29T15:49:36.705 DEBUG:teuthology.orchestra.run.smithi164:> import glob 2022-06-29T15:49:36.706 DEBUG:teuthology.orchestra.run.smithi164:> import re 2022-06-29T15:49:36.706 DEBUG:teuthology.orchestra.run.smithi164:> import os 2022-06-29T15:49:36.707 DEBUG:teuthology.orchestra.run.smithi164:> import subprocess 2022-06-29T15:49:36.707 DEBUG:teuthology.orchestra.run.smithi164:> 2022-06-29T15:49:36.708 DEBUG:teuthology.orchestra.run.smithi164:> def find_socket(client_name): 2022-06-29T15:49:36.708 DEBUG:teuthology.orchestra.run.smithi164:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2022-06-29T15:49:36.709 DEBUG:teuthology.orchestra.run.smithi164:> files = glob.glob(asok_path) 2022-06-29T15:49:36.709 DEBUG:teuthology.orchestra.run.smithi164:> 2022-06-29T15:49:36.709 DEBUG:teuthology.orchestra.run.smithi164:> # Given a non-glob path, it better be there 2022-06-29T15:49:36.710 DEBUG:teuthology.orchestra.run.smithi164:> if "*" not in asok_path: 2022-06-29T15:49:36.711 DEBUG:teuthology.orchestra.run.smithi164:> assert(len(files) == 1) 2022-06-29T15:49:36.712 DEBUG:teuthology.orchestra.run.smithi164:> return files[0] 2022-06-29T15:49:36.712 DEBUG:teuthology.orchestra.run.smithi164:> 2022-06-29T15:49:36.712 DEBUG:teuthology.orchestra.run.smithi164:> for f in files: 2022-06-29T15:49:36.713 DEBUG:teuthology.orchestra.run.smithi164:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-29T15:49:36.714 DEBUG:teuthology.orchestra.run.smithi164:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-29T15:49:36.714 DEBUG:teuthology.orchestra.run.smithi164:> return f 2022-06-29T15:49:36.715 DEBUG:teuthology.orchestra.run.smithi164:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-29T15:49:36.715 DEBUG:teuthology.orchestra.run.smithi164:> 2022-06-29T15:49:36.716 DEBUG:teuthology.orchestra.run.smithi164:> print(find_socket("client.1")) 2022-06-29T15:49:36.716 DEBUG:teuthology.orchestra.run.smithi164:> ' 2022-06-29T15:49:36.758 INFO:teuthology.orchestra.run.smithi164.stdout:/var/run/ceph/ceph-client.1.12560.asok 2022-06-29T15:49:36.764 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.12560.asok 2022-06-29T15:49:36.765 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:49:36.765 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.12560.asok status 2022-06-29T15:49:36.901 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2022-06-29T15:49:36.902 INFO:teuthology.orchestra.run.smithi164.stdout: "metadata": { 2022-06-29T15:49:36.902 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_sha1": "f25ffac72e5a9e63860ff7deb6767a11f0559a16", 2022-06-29T15:49:36.902 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_version": "ceph version 15.2.16-304-gf25ffac7 (f25ffac72e5a9e63860ff7deb6767a11f0559a16) octopus (stable)", 2022-06-29T15:49:36.903 INFO:teuthology.orchestra.run.smithi164.stdout: "entity_id": "1", 2022-06-29T15:49:36.903 INFO:teuthology.orchestra.run.smithi164.stdout: "hostname": "smithi164", 2022-06-29T15:49:36.903 INFO:teuthology.orchestra.run.smithi164.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2022-06-29T15:49:36.903 INFO:teuthology.orchestra.run.smithi164.stdout: "pid": "12560", 2022-06-29T15:49:36.904 INFO:teuthology.orchestra.run.smithi164.stdout: "root": "/" 2022-06-29T15:49:36.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2022-06-29T15:49:36.904 INFO:teuthology.orchestra.run.smithi164.stdout: "dentry_count": 0, 2022-06-29T15:49:36.904 INFO:teuthology.orchestra.run.smithi164.stdout: "dentry_pinned_count": 0, 2022-06-29T15:49:36.905 INFO:teuthology.orchestra.run.smithi164.stdout: "id": 56006, 2022-06-29T15:49:36.905 INFO:teuthology.orchestra.run.smithi164.stdout: "inst": { 2022-06-29T15:49:36.905 INFO:teuthology.orchestra.run.smithi164.stdout: "name": { 2022-06-29T15:49:36.905 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "client", 2022-06-29T15:49:36.906 INFO:teuthology.orchestra.run.smithi164.stdout: "num": 56006 2022-06-29T15:49:36.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2022-06-29T15:49:36.906 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": { 2022-06-29T15:49:36.906 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "v1", 2022-06-29T15:49:36.907 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": "172.21.15.164:0", 2022-06-29T15:49:36.907 INFO:teuthology.orchestra.run.smithi164.stdout: "nonce": 532305885 2022-06-29T15:49:36.907 INFO:teuthology.orchestra.run.smithi164.stdout: } 2022-06-29T15:49:36.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2022-06-29T15:49:36.908 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": { 2022-06-29T15:49:36.908 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "v1", 2022-06-29T15:49:36.909 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": "172.21.15.164:0", 2022-06-29T15:49:36.909 INFO:teuthology.orchestra.run.smithi164.stdout: "nonce": 532305885 2022-06-29T15:49:36.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2022-06-29T15:49:36.910 INFO:teuthology.orchestra.run.smithi164.stdout: "inst_str": "client.56006 172.21.15.164:0/532305885", 2022-06-29T15:49:36.910 INFO:teuthology.orchestra.run.smithi164.stdout: "addr_str": "172.21.15.164:0/532305885", 2022-06-29T15:49:36.910 INFO:teuthology.orchestra.run.smithi164.stdout: "inode_count": 1, 2022-06-29T15:49:36.911 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_epoch": 40, 2022-06-29T15:49:36.911 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_epoch": 90, 2022-06-29T15:49:36.911 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_epoch_barrier": 0, 2022-06-29T15:49:36.912 INFO:teuthology.orchestra.run.smithi164.stdout: "blacklisted": false, 2022-06-29T15:49:36.912 INFO:teuthology.orchestra.run.smithi164.stdout: "fs_name": "cephfs" 2022-06-29T15:49:36.913 INFO:teuthology.orchestra.run.smithi164.stdout:} 2022-06-29T15:49:36.914 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:49:36.915 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:49:36.916 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:49:36.924 INFO:teuthology.orchestra.run.smithi164.stdout:fuseblk 2022-06-29T15:49:36.925 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-06-29T15:49:36.926 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:49:36.927 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:49:36.984 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:49:37.099 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.093+0000 7fcc6f881700 1 Processor -- start 2022-06-29T15:49:37.100 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.093+0000 7fcc6f881700 1 -- start start 2022-06-29T15:49:37.102 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.093+0000 7fcc6f881700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcc6805f0b0 0x7fcc680b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:37.102 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.093+0000 7fcc6f881700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcc680b1260 0x7fcc6814b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:37.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.093+0000 7fcc6f881700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc680b0740 con 0x7fcc680b2120 2022-06-29T15:49:37.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.093+0000 7fcc6f881700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc68147460 con 0x7fcc6805f0b0 2022-06-29T15:49:37.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.093+0000 7fcc6f881700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc680b0970 con 0x7fcc680b1260 2022-06-29T15:49:37.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.093+0000 7fcc6ce1c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcc6805f0b0 0x7fcc680b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:37.104 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.093+0000 7fcc6ce1c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcc6805f0b0 0x7fcc680b2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51014/0 (socket says 172.21.15.47:51014) 2022-06-29T15:49:37.104 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.093+0000 7fcc6ce1c700 1 -- 172.21.15.47:0/902674064 learned_addr learned my addr 172.21.15.47:0/902674064 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:37.104 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.093+0000 7fcc6de1e700 1 --2- 172.21.15.47:0/902674064 >> v2:172.21.15.64:3300/0 conn(0x7fcc680b1260 0x7fcc6814b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:37.104 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.093+0000 7fcc6ce1c700 1 -- 172.21.15.47:0/902674064 >> v2:172.21.15.64:3300/0 conn(0x7fcc680b1260 msgr2=0x7fcc6814b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:37.105 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.093+0000 7fcc6ce1c700 1 --2- 172.21.15.47:0/902674064 >> v2:172.21.15.64:3300/0 conn(0x7fcc680b1260 0x7fcc6814b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:37.105 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.093+0000 7fcc6ce1c700 1 -- 172.21.15.47:0/902674064 >> v1:172.21.15.47:6789/0 conn(0x7fcc680b2120 legacy=0x7fcc68147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:37.105 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.093+0000 7fcc6ce1c700 1 -- 172.21.15.47:0/902674064 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc680b0c10 con 0x7fcc6805f0b0 2022-06-29T15:49:37.105 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.093+0000 7fcc6ce1c700 1 --2- 172.21.15.47:0/902674064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc6805f0b0 0x7fcc680b2500 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 rx=0x7fcc68141a70 tx=0x7fcc64006eb0).ready entity=mon.0 client_cookie=b307fec557b17a6c server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:37.106 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.093+0000 7fcc5ffff700 1 -- 172.21.15.47:0/902674064 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc6400d5c0 con 0x7fcc6805f0b0 2022-06-29T15:49:37.106 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.093+0000 7fcc5ffff700 1 -- 172.21.15.47:0/902674064 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc64000e90 con 0x7fcc6805f0b0 2022-06-29T15:49:37.106 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.093+0000 7fcc5ffff700 1 -- 172.21.15.47:0/902674064 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc6400d950 con 0x7fcc6805f0b0 2022-06-29T15:49:37.106 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.093+0000 7fcc6f881700 1 -- 172.21.15.47:0/902674064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc6805f0b0 msgr2=0x7fcc680b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:37.107 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.093+0000 7fcc6f881700 1 --2- 172.21.15.47:0/902674064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc6805f0b0 0x7fcc680b2500 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 rx=0x7fcc68141a70 tx=0x7fcc64006eb0).stop 2022-06-29T15:49:37.107 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.093+0000 7fcc6f881700 1 -- 172.21.15.47:0/902674064 shutdown_connections 2022-06-29T15:49:37.107 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.093+0000 7fcc6f881700 1 --2- 172.21.15.47:0/902674064 >> v2:172.21.15.64:3300/0 conn(0x7fcc680b1260 0x7fcc6814b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:37.107 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.093+0000 7fcc6f881700 1 --2- 172.21.15.47:0/902674064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc6805f0b0 0x7fcc680b2500 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:37.108 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.093+0000 7fcc6f881700 1 -- 172.21.15.47:0/902674064 >> 172.21.15.47:0/902674064 conn(0x7fcc680626f0 msgr2=0x7fcc68062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:37.108 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.093+0000 7fcc6f881700 1 -- 172.21.15.47:0/902674064 shutdown_connections 2022-06-29T15:49:37.108 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.093+0000 7fcc6f881700 1 -- 172.21.15.47:0/902674064 wait complete. 2022-06-29T15:49:37.112 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.097+0000 7fcc6f881700 1 Processor -- start 2022-06-29T15:49:37.112 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.097+0000 7fcc6f881700 1 -- start start 2022-06-29T15:49:37.113 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.097+0000 7fcc6f881700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc680b1260 0x7fcc6809ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:37.113 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.097+0000 7fcc6f881700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcc680b2120 0x7fcc68159f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:37.113 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.097+0000 7fcc6f881700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc6815a420 0x7fcc681434f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:37.113 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.097+0000 7fcc6f881700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc68142320 con 0x7fcc6815a420 2022-06-29T15:49:37.114 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.097+0000 7fcc6f881700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc68144a30 con 0x7fcc680b1260 2022-06-29T15:49:37.114 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.097+0000 7fcc6f881700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6815a8b0 con 0x7fcc680b2120 2022-06-29T15:49:37.114 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.097+0000 7fcc6de1e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc6815a420 0x7fcc681434f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:37.114 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.097+0000 7fcc6de1e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc6815a420 0x7fcc681434f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51024/0 (socket says 172.21.15.47:51024) 2022-06-29T15:49:37.115 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.097+0000 7fcc6de1e700 1 -- 172.21.15.47:0/831818204 learned_addr learned my addr 172.21.15.47:0/831818204 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:37.115 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.097+0000 7fcc6d61d700 1 --2- 172.21.15.47:0/831818204 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc680b1260 0x7fcc6809ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:37.115 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.097+0000 7fcc6ce1c700 1 --2- 172.21.15.47:0/831818204 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcc680b2120 0x7fcc68159f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:37.116 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.097+0000 7fcc6de1e700 1 -- 172.21.15.47:0/831818204 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcc680b2120 msgr2=0x7fcc68159f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:37.116 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.097+0000 7fcc6de1e700 1 --2- 172.21.15.47:0/831818204 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcc680b2120 0x7fcc68159f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:37.116 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.097+0000 7fcc6de1e700 1 -- 172.21.15.47:0/831818204 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc680b1260 msgr2=0x7fcc6809ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:37.116 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.097+0000 7fcc6de1e700 1 --2- 172.21.15.47:0/831818204 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc680b1260 0x7fcc6809ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:37.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.097+0000 7fcc6de1e700 1 -- 172.21.15.47:0/831818204 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc6400b1e0 con 0x7fcc6815a420 2022-06-29T15:49:37.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.097+0000 7fcc6de1e700 1 --2- 172.21.15.47:0/831818204 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc6815a420 0x7fcc681434f0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 rx=0x7fcc681445b0 tx=0x7fcc60008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:37.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.097+0000 7fcc5e7fc700 1 -- 172.21.15.47:0/831818204 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc6000c070 con 0x7fcc6815a420 2022-06-29T15:49:37.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.097+0000 7fcc5e7fc700 1 -- 172.21.15.47:0/831818204 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc600133f0 con 0x7fcc6815a420 2022-06-29T15:49:37.118 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.097+0000 7fcc6f881700 1 -- 172.21.15.47:0/831818204 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc68142510 con 0x7fcc6815a420 2022-06-29T15:49:37.118 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.097+0000 7fcc5e7fc700 1 -- 172.21.15.47:0/831818204 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc60013750 con 0x7fcc6815a420 2022-06-29T15:49:37.118 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.097+0000 7fcc6f881700 1 -- 172.21.15.47:0/831818204 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc680af560 con 0x7fcc6815a420 2022-06-29T15:49:37.118 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.101+0000 7fcc6f881700 1 -- 172.21.15.47:0/831818204 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc681426a0 con 0x7fcc6815a420 2022-06-29T15:49:37.119 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.101+0000 7fcc5e7fc700 1 -- 172.21.15.47:0/831818204 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fcc600138b0 con 0x7fcc6815a420 2022-06-29T15:49:37.119 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.101+0000 7fcc5e7fc700 1 --2- 172.21.15.47:0/831818204 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcc500539e0 0x7fcc50055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:37.119 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.101+0000 7fcc5e7fc700 1 -- 172.21.15.47:0/831818204 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fcc6006bc70 con 0x7fcc6815a420 2022-06-29T15:49:37.119 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.101+0000 7fcc6d61d700 1 --2- 172.21.15.47:0/831818204 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcc500539e0 0x7fcc50055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:37.119 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.101+0000 7fcc6d61d700 1 --2- 172.21.15.47:0/831818204 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcc500539e0 0x7fcc50055ea0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7fcc5800d000 tx=0x7fcc58007b90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:37.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.105+0000 7fcc5e7fc700 1 -- 172.21.15.47:0/831818204 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcc60042a30 con 0x7fcc6815a420 2022-06-29T15:49:37.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.273+0000 7fcc6f881700 1 -- 172.21.15.47:0/831818204 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcc680573d0 con 0x7fcc6815a420 2022-06-29T15:49:37.282 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.273+0000 7fcc5e7fc700 1 -- 172.21.15.47:0/831818204 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+2493 (secure 0 0 0) 0x7fcc6000f040 con 0x7fcc6815a420 2022-06-29T15:49:37.282 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:49:37.283 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":40,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6577,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/4286932616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":4286932616},{"type":"v1","addr":"172.21.15.64:6833","nonce":4286932616}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":38},{"gid":6657,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3528102962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3528102962},{"type":"v1","addr":"172.21.15.47:6835","nonce":3528102962}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":38}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:49:27.717245+0000","modified":"2022-06-29T15:49:28.719969+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6660},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6660":{"gid":6660,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6837/4146905199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":4146905199},{"type":"v1","addr":"172.21.15.47:6837","nonce":4146905199}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2022-06-29T15:49:37.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.277+0000 7fcc6f881700 1 -- 172.21.15.47:0/831818204 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcc500539e0 msgr2=0x7fcc50055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:37.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.277+0000 7fcc6f881700 1 --2- 172.21.15.47:0/831818204 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcc500539e0 0x7fcc50055ea0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7fcc5800d000 tx=0x7fcc58007b90).stop 2022-06-29T15:49:37.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.277+0000 7fcc6f881700 1 -- 172.21.15.47:0/831818204 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc6815a420 msgr2=0x7fcc681434f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:37.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.277+0000 7fcc6f881700 1 --2- 172.21.15.47:0/831818204 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc6815a420 0x7fcc681434f0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 rx=0x7fcc681445b0 tx=0x7fcc60008040).stop 2022-06-29T15:49:37.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.277+0000 7fcc6f881700 1 -- 172.21.15.47:0/831818204 shutdown_connections 2022-06-29T15:49:37.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.277+0000 7fcc6f881700 1 --2- 172.21.15.47:0/831818204 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc6815a420 0x7fcc681434f0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:37.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.277+0000 7fcc6f881700 1 --2- 172.21.15.47:0/831818204 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcc680b2120 0x7fcc68159f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:37.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.277+0000 7fcc6f881700 1 --2- 172.21.15.47:0/831818204 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcc500539e0 0x7fcc50055ea0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:37.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.277+0000 7fcc6f881700 1 --2- 172.21.15.47:0/831818204 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc680b1260 0x7fcc6809ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:37.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.277+0000 7fcc6f881700 1 -- 172.21.15.47:0/831818204 >> 172.21.15.47:0/831818204 conn(0x7fcc680626f0 msgr2=0x7fcc68145bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:37.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.277+0000 7fcc6f881700 1 -- 172.21.15.47:0/831818204 shutdown_connections 2022-06-29T15:49:37.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.277+0000 7fcc6f881700 1 -- 172.21.15.47:0/831818204 wait complete. 2022-06-29T15:49:37.289 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 40 2022-06-29T15:49:37.303 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump inode 1 2022-06-29T15:49:37.409 INFO:teuthology.orchestra.run.smithi047.stdout:{"path":"/","ino":1,"rdev":0,"ctime":"2022-06-29T15:49:33.226637+0000","btime":"2022-06-29T15:49:27.722547+0000","mode":17407,"uid":0,"gid":0,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":4194304,"stripe_count":1,"object_size":4194304,"pool_id":16,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":0,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2022-06-29T15:49:27.722547+0000","atime":"0.000000","time_warp_seq":0,"change_attr":1,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"0.000000","num_files":0,"num_subdirs":0,"change_attr":0},"rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":1,"rsnaps":0,"rctime":"2022-06-29T15:49:33.226637+0000"},"accounted_rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":1,"rsnaps":0,"rctime":"2022-06-29T15:49:27.722547+0000"},"version":3,"file_data_version":0,"xattr_version":1,"backtrace_version":3,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":2,"is_frozen":false,"is_freezing":false,"pins":{"request":1,"lock":3,"dirfrag":1,"caps":1,"dirtyparent":1,"dirty":1,"authpin":1},"nref":9,"versionlock":{"gather_set":[],"state":"lock","is_leased":false,"num_rdlocks":0,"num_wrlocks":1,"num_xlocks":0,"xlock_by":{}},"authlock":{"gather_set":[],"state":"xlockdone","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":1,"xlock_by":{}},"linklock":{},"dirfragtreelock":{},"filelock":{},"xattrlock":{},"snaplock":{"gather_set":[],"state":"sync","is_leased":false,"num_rdlocks":1,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"nestlock":{},"flocklock":{},"policylock":{},"states":["auth","dirty","dirtyparent"],"client_caps":[{"client_id":6745,"pending":"pLsXsFs","issued":"pLsXsFs","wanted":"-","last_sent":2},{"client_id":56006,"pending":"pLsXsFs","issued":"pLsXsFs","wanted":"-","last_sent":2}],"loner":-1,"want_loner":-1,"mds_caps_wanted":[]} 2022-06-29T15:49:37.410 DEBUG:tasks.cephfs.filesystem:_json_asok output { "accounted_rstat": { "rbytes": 0, "rctime": "2022-06-29T15:49:27.722547+0000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "atime": "0.000000", "auth_pins": 2, "auth_state": { "replicas": {} }, "authlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 1, "state": "xlockdone", "xlock_by": {} }, "backtrace_version": 3, "btime": "2022-06-29T15:49:27.722547+0000", "change_attr": 1, "client_caps": [ { "client_id": 6745, "issued": "pLsXsFs", "last_sent": 2, "pending": "pLsXsFs", "wanted": "-" }, { "client_id": 56006, "issued": "pLsXsFs", "last_sent": 2, "pending": "pLsXsFs", "wanted": "-" } ], "client_ranges": [], "ctime": "2022-06-29T15:49:33.226637+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 0, "mtime": "0.000000", "num_files": 0, "num_subdirs": 0, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": {}, "flocklock": {}, "gid": 0, "ino": 1, "is_auth": true, "is_freezing": false, "is_frozen": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 4194304, "pool_id": 16, "pool_ns": "", "stripe_count": 1, "stripe_unit": 4194304 }, "linklock": {}, "loner": -1, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 17407, "mtime": "2022-06-29T15:49:27.722547+0000", "nestlock": {}, "nlink": 1, "nref": 9, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "path": "/", "pins": { "authpin": 1, "caps": 1, "dirfrag": 1, "dirty": 1, "dirtyparent": 1, "lock": 3, "request": 1 }, "policylock": {}, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 0, "rctime": "2022-06-29T15:49:33.226637+0000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "size": 0, "snaplock": { "gather_set": [], "is_leased": false, "num_rdlocks": 1, "num_wrlocks": 0, "num_xlocks": 0, "state": "sync", "xlock_by": {} }, "states": [ "auth", "dirty", "dirtyparent" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 0, "truncate_size": 18446744073709551615, "uid": 0, "version": 3, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 1, "num_xlocks": 0, "state": "lock", "xlock_by": {} }, "want_loner": -1, "xattr_version": 1, "xattrlock": {}, "xattrs": [] } 2022-06-29T15:49:37.411 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-06-29T15:49:37.411 DEBUG:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:49:37.430 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:49:37.454 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:49:37.475 DEBUG:teuthology.orchestra.run.smithi164:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:49:37.496 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-06-29T15:49:37.497 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:49:37.498 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:49:37.498 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:49:37.506 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2022-06-29T15:49:37.507 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-29T15:49:37.508 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi099.front.sepia.ceph.com... 2022-06-29T15:49:37.508 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-06-29T15:49:37.509 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:49:37.565 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[13011]: fuse finished with error 0 and tester_r 0 2022-06-29T15:49:37.567 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:49:37.567 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:49:37.568 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:49:37.574 INFO:teuthology.orchestra.run.smithi099.stdout:ext2/ext3 2022-06-29T15:49:37.575 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-06-29T15:49:37.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-29T15:49:37.625 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:49:37.634 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:49:37.634 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:49:37.635 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:49:37.642 INFO:teuthology.orchestra.run.smithi164.stdout:fuseblk 2022-06-29T15:49:37.643 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-06-29T15:49:37.644 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi164.front.sepia.ceph.com... 2022-06-29T15:49:37.644 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-06-29T15:49:37.645 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:49:37.700 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi164.stderr:ceph-fuse[12560]: fuse finished with error 0 and tester_r 0 2022-06-29T15:49:37.703 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:49:37.703 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:49:37.704 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:49:37.710 INFO:teuthology.orchestra.run.smithi164.stdout:ext2/ext3 2022-06-29T15:49:37.711 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-06-29T15:49:37.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-29T15:49:37.829 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:49:37.836 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-06-29T15:49:37.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.941+0000 7fb602676700 1 Processor -- start 2022-06-29T15:49:37.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.941+0000 7fb602676700 1 -- start start 2022-06-29T15:49:37.951 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.941+0000 7fb602676700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb5fc0b2a90 0x7fb5fc061050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:37.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.941+0000 7fb602676700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb5fc061600 0x7fb5fc146b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:37.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.941+0000 7fb602676700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb5fc147060 0x7fb5fc0943d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:37.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.941+0000 7fb602676700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5fc0b2170 con 0x7fb5fc061600 2022-06-29T15:49:37.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.941+0000 7fb602676700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5fc0b2e70 con 0x7fb5fc0b2a90 2022-06-29T15:49:37.953 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.941+0000 7fb602676700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5fc0606b0 con 0x7fb5fc147060 2022-06-29T15:49:37.953 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.941+0000 7fb5fb7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb5fc061600 0x7fb5fc146b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:37.953 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.941+0000 7fb5fb7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb5fc061600 0x7fb5fc146b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51030/0 (socket says 172.21.15.47:51030) 2022-06-29T15:49:37.953 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.941+0000 7fb5fb7fe700 1 -- 172.21.15.47:0/828406177 learned_addr learned my addr 172.21.15.47:0/828406177 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:37.954 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.945+0000 7fb600c13700 1 --2- 172.21.15.47:0/828406177 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb5fc147060 0x7fb5fc0943d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:37.954 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.945+0000 7fb5fbfff700 1 --2- 172.21.15.47:0/828406177 >> v2:172.21.15.64:3300/0 conn(0x7fb5fc0b2a90 0x7fb5fc061050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:37.954 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.945+0000 7fb5fb7fe700 1 -- 172.21.15.47:0/828406177 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb5fc147060 msgr2=0x7fb5fc0943d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:37.954 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.945+0000 7fb5fb7fe700 1 --2- 172.21.15.47:0/828406177 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb5fc147060 0x7fb5fc0943d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:37.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.945+0000 7fb5fb7fe700 1 -- 172.21.15.47:0/828406177 >> v2:172.21.15.64:3300/0 conn(0x7fb5fc0b2a90 msgr2=0x7fb5fc061050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:37.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.945+0000 7fb5fb7fe700 1 --2- 172.21.15.47:0/828406177 >> v2:172.21.15.64:3300/0 conn(0x7fb5fc0b2a90 0x7fb5fc061050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:37.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.945+0000 7fb5fb7fe700 1 -- 172.21.15.47:0/828406177 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5fc0a9770 con 0x7fb5fc061600 2022-06-29T15:49:37.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.945+0000 7fb5fb7fe700 1 --2- 172.21.15.47:0/828406177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5fc061600 0x7fb5fc146b70 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 rx=0x7fb5f0002f70 tx=0x7fb5f0006eb0).ready entity=mon.0 client_cookie=d581f9f43a7dc9ac server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:37.959 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.945+0000 7fb5faffd700 1 -- 172.21.15.47:0/828406177 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5f000d570 con 0x7fb5fc061600 2022-06-29T15:49:37.959 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.945+0000 7fb5faffd700 1 -- 172.21.15.47:0/828406177 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f0000e90 con 0x7fb5fc061600 2022-06-29T15:49:37.959 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.945+0000 7fb5faffd700 1 -- 172.21.15.47:0/828406177 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5f000d900 con 0x7fb5fc061600 2022-06-29T15:49:37.960 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.945+0000 7fb602676700 1 -- 172.21.15.47:0/828406177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5fc061600 msgr2=0x7fb5fc146b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:37.960 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.945+0000 7fb602676700 1 --2- 172.21.15.47:0/828406177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5fc061600 0x7fb5fc146b70 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 rx=0x7fb5f0002f70 tx=0x7fb5f0006eb0).stop 2022-06-29T15:49:37.960 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.945+0000 7fb602676700 1 -- 172.21.15.47:0/828406177 shutdown_connections 2022-06-29T15:49:37.960 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.945+0000 7fb602676700 1 --2- 172.21.15.47:0/828406177 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb5fc147060 0x7fb5fc0943d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:37.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.945+0000 7fb602676700 1 --2- 172.21.15.47:0/828406177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5fc061600 0x7fb5fc146b70 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:37.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.945+0000 7fb602676700 1 --2- 172.21.15.47:0/828406177 >> v2:172.21.15.64:3300/0 conn(0x7fb5fc0b2a90 0x7fb5fc061050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:37.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.945+0000 7fb602676700 1 -- 172.21.15.47:0/828406177 >> 172.21.15.47:0/828406177 conn(0x7fb5fc05f0b0 msgr2=0x7fb5fc05f490 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:37.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.945+0000 7fb602676700 1 -- 172.21.15.47:0/828406177 shutdown_connections 2022-06-29T15:49:37.962 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.945+0000 7fb602676700 1 -- 172.21.15.47:0/828406177 wait complete. 2022-06-29T15:49:37.962 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.949+0000 7fb602676700 1 Processor -- start 2022-06-29T15:49:37.962 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.949+0000 7fb602676700 1 -- start start 2022-06-29T15:49:37.963 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.949+0000 7fb602676700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb5fc061600 0x7fb5fc145de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:37.963 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.949+0000 7fb602676700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5fc0b2a90 0x7fb5fc096dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:37.963 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.949+0000 7fb602676700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5fc147060 0x7fb5fc097980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:37.964 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.949+0000 7fb602676700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5fc0b0030 con 0x7fb5fc147060 2022-06-29T15:49:37.964 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.949+0000 7fb602676700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5fc0972c0 con 0x7fb5fc0b2a90 2022-06-29T15:49:37.964 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.949+0000 7fb602676700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5fc0af230 con 0x7fb5fc061600 2022-06-29T15:49:37.964 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.949+0000 7fb600c13700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5fc147060 0x7fb5fc097980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:37.964 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.949+0000 7fb600c13700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5fc147060 0x7fb5fc097980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51038/0 (socket says 172.21.15.47:51038) 2022-06-29T15:49:37.965 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.949+0000 7fb600c13700 1 -- 172.21.15.47:0/3751243324 learned_addr learned my addr 172.21.15.47:0/3751243324 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:37.965 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.949+0000 7fb5fbfff700 1 --2- 172.21.15.47:0/3751243324 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb5fc061600 0x7fb5fc145de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:37.965 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.949+0000 7fb5fb7fe700 1 --2- 172.21.15.47:0/3751243324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5fc0b2a90 0x7fb5fc096dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:37.965 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.949+0000 7fb600c13700 1 -- 172.21.15.47:0/3751243324 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb5fc061600 msgr2=0x7fb5fc145de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:37.966 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.949+0000 7fb600c13700 1 --2- 172.21.15.47:0/3751243324 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb5fc061600 0x7fb5fc145de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:37.966 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.949+0000 7fb600c13700 1 -- 172.21.15.47:0/3751243324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5fc0b2a90 msgr2=0x7fb5fc096dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:37.966 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.949+0000 7fb600c13700 1 --2- 172.21.15.47:0/3751243324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5fc0b2a90 0x7fb5fc096dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:37.966 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.949+0000 7fb600c13700 1 -- 172.21.15.47:0/3751243324 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f000b1e0 con 0x7fb5fc147060 2022-06-29T15:49:37.967 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.949+0000 7fb5fbfff700 1 --2- 172.21.15.47:0/3751243324 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb5fc061600 0x7fb5fc145de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:49:37.967 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.949+0000 7fb600c13700 1 --2- 172.21.15.47:0/3751243324 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5fc147060 0x7fb5fc097980 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 rx=0x7fb5f4006e00 tx=0x7fb5f40091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:37.967 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.949+0000 7fb5f97fa700 1 -- 172.21.15.47:0/3751243324 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5f400c7f0 con 0x7fb5fc147060 2022-06-29T15:49:37.967 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.949+0000 7fb5f97fa700 1 -- 172.21.15.47:0/3751243324 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f400c950 con 0x7fb5fc147060 2022-06-29T15:49:37.968 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.949+0000 7fb602676700 1 -- 172.21.15.47:0/3751243324 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5fc0af4d0 con 0x7fb5fc147060 2022-06-29T15:49:37.968 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.949+0000 7fb5f97fa700 1 -- 172.21.15.47:0/3751243324 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5f400ccb0 con 0x7fb5fc147060 2022-06-29T15:49:37.968 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.949+0000 7fb602676700 1 -- 172.21.15.47:0/3751243324 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5fc0afc10 con 0x7fb5fc147060 2022-06-29T15:49:37.969 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.953+0000 7fb5f97fa700 1 -- 172.21.15.47:0/3751243324 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb5f40123f0 con 0x7fb5fc147060 2022-06-29T15:49:37.970 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.953+0000 7fb602676700 1 -- 172.21.15.47:0/3751243324 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5fc0573d0 con 0x7fb5fc147060 2022-06-29T15:49:37.970 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.953+0000 7fb5f97fa700 1 --2- 172.21.15.47:0/3751243324 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb5dc053980 0x7fb5dc055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:37.970 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.953+0000 7fb5f97fa700 1 -- 172.21.15.47:0/3751243324 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb5f406c3e0 con 0x7fb5fc147060 2022-06-29T15:49:37.970 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.953+0000 7fb5fbfff700 1 --2- 172.21.15.47:0/3751243324 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb5dc053980 0x7fb5dc055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:37.971 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.953+0000 7fb5fbfff700 1 --2- 172.21.15.47:0/3751243324 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb5dc053980 0x7fb5dc055e40 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7fb5ec00c000 tx=0x7fb5ec008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:37.971 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:37.957+0000 7fb5f97fa700 1 -- 172.21.15.47:0/3751243324 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb5f4043030 con 0x7fb5fc147060 2022-06-29T15:49:38.092 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.085+0000 7fb602676700 1 -- 172.21.15.47:0/3751243324 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fb5fc0b31f0 con 0x7fb5fc147060 2022-06-29T15:49:38.094 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.085+0000 7fb5f97fa700 1 -- 172.21.15.47:0/3751243324 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v90) v1 ==== 81+0+34040 (secure 0 0 0) 0x7fb5f4046650 con 0x7fb5fc147060 2022-06-29T15:49:38.095 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:49:38.096 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2022-06-29T15:49:38.096 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 90, 2022-06-29T15:49:38.096 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "9a68894a-53ce-400e-838b-14eab91b834c", 2022-06-29T15:49:38.097 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2022-06-29T15:42:14.046361+0000", 2022-06-29T15:49:38.097 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2022-06-29T15:49:28.720705+0000", 2022-06-29T15:49:38.097 INFO:teuthology.orchestra.run.smithi047.stdout: "last_up_change": "2022-06-29T15:42:20.929789+0000", 2022-06-29T15:49:38.097 INFO:teuthology.orchestra.run.smithi047.stdout: "last_in_change": "2022-06-29T15:42:20.929789+0000", 2022-06-29T15:49:38.098 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-06-29T15:49:38.098 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_num": 5799936, 2022-06-29T15:49:38.098 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_set": [ 2022-06-29T15:49:38.098 INFO:teuthology.orchestra.run.smithi047.stdout: "pglog_hardlimit", 2022-06-29T15:49:38.099 INFO:teuthology.orchestra.run.smithi047.stdout: "purged_snapdirs", 2022-06-29T15:49:38.099 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_deletes", 2022-06-29T15:49:38.099 INFO:teuthology.orchestra.run.smithi047.stdout: "sortbitwise" 2022-06-29T15:49:38.099 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:49:38.100 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_version": 5, 2022-06-29T15:49:38.100 INFO:teuthology.orchestra.run.smithi047.stdout: "full_ratio": 0.94999998807907104, 2022-06-29T15:49:38.100 INFO:teuthology.orchestra.run.smithi047.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-06-29T15:49:38.100 INFO:teuthology.orchestra.run.smithi047.stdout: "nearfull_ratio": 0.85000002384185791, 2022-06-29T15:49:38.101 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_snapshot": "", 2022-06-29T15:49:38.101 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_max": 16, 2022-06-29T15:49:38.101 INFO:teuthology.orchestra.run.smithi047.stdout: "max_osd": 8, 2022-06-29T15:49:38.101 INFO:teuthology.orchestra.run.smithi047.stdout: "require_min_compat_client": "jewel", 2022-06-29T15:49:38.101 INFO:teuthology.orchestra.run.smithi047.stdout: "min_compat_client": "jewel", 2022-06-29T15:49:38.102 INFO:teuthology.orchestra.run.smithi047.stdout: "require_osd_release": "octopus", 2022-06-29T15:49:38.102 INFO:teuthology.orchestra.run.smithi047.stdout: "pools": [ 2022-06-29T15:49:38.102 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.103 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 1, 2022-06-29T15:49:38.103 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "device_health_metrics", 2022-06-29T15:49:38.103 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:42:16.971431+0000", 2022-06-29T15:49:38.103 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:49:38.104 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:49:38.104 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:49:38.104 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:49:38.104 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:49:38.105 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:49:38.105 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:49:38.105 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:49:38.105 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 1, 2022-06-29T15:49:38.106 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 1, 2022-06-29T15:49:38.106 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 1, 2022-06-29T15:49:38.106 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 1, 2022-06-29T15:49:38.106 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 1, 2022-06-29T15:49:38.107 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:49:38.107 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:49:38.107 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:49:38.107 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:49:38.108 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:49:38.108 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:49:38.108 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:49:38.109 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.109 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "16", 2022-06-29T15:49:38.109 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:49:38.109 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:49:38.110 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:49:38.110 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:49:38.110 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:49:38.110 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:49:38.111 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:49:38.111 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:49:38.111 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:49:38.111 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:49:38.111 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:49:38.112 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:49:38.112 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:49:38.112 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:49:38.113 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:49:38.113 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:49:38.113 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:49:38.114 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:49:38.114 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:49:38.114 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:49:38.115 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:49:38.115 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:49:38.115 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:49:38.115 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:49:38.116 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:49:38.116 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:49:38.116 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.116 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:49:38.117 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:49:38.117 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:49:38.117 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:49:38.117 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:49:38.118 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:49:38.118 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:49:38.118 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:49:38.118 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:49:38.119 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:49:38.119 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:49:38.120 INFO:teuthology.orchestra.run.smithi047.stdout: "options": { 2022-06-29T15:49:38.120 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_min": 1 2022-06-29T15:49:38.120 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.121 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:49:38.121 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_devicehealth": {} 2022-06-29T15:49:38.122 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:38.122 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.123 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.123 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 2, 2022-06-29T15:49:38.124 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "rbd", 2022-06-29T15:49:38.124 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:42:25.290884+0000", 2022-06-29T15:49:38.125 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:49:38.125 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:49:38.126 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:49:38.126 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:49:38.126 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:49:38.127 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:49:38.127 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:49:38.128 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:49:38.128 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2022-06-29T15:49:38.129 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2022-06-29T15:49:38.129 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2022-06-29T15:49:38.130 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2022-06-29T15:49:38.130 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2022-06-29T15:49:38.131 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:49:38.131 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:49:38.131 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:49:38.132 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:49:38.132 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:49:38.133 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:49:38.133 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:49:38.133 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.134 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "19", 2022-06-29T15:49:38.134 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:49:38.135 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:49:38.135 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:49:38.136 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:49:38.136 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:49:38.137 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:49:38.137 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:49:38.138 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:49:38.138 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:49:38.138 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:49:38.139 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:49:38.139 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:49:38.140 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:49:38.140 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:49:38.141 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:49:38.141 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:49:38.141 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:49:38.142 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:49:38.142 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:49:38.143 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:49:38.143 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:49:38.144 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:49:38.144 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:49:38.145 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:49:38.145 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:49:38.145 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:49:38.146 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.146 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:49:38.147 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:49:38.147 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:49:38.148 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:49:38.148 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:49:38.148 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:49:38.148 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:49:38.149 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:49:38.149 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:49:38.149 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:49:38.149 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:49:38.150 INFO:teuthology.orchestra.run.smithi047.stdout: "options": {}, 2022-06-29T15:49:38.150 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:49:38.150 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd": {} 2022-06-29T15:49:38.150 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:38.151 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.151 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.151 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 15, 2022-06-29T15:49:38.151 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "cephfs_metadata", 2022-06-29T15:49:38.151 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:49:25.785974+0000", 2022-06-29T15:49:38.152 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:49:38.152 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:49:38.152 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:49:38.152 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:49:38.153 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:49:38.153 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:49:38.153 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:49:38.153 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:49:38.153 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2022-06-29T15:49:38.154 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2022-06-29T15:49:38.154 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2022-06-29T15:49:38.154 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2022-06-29T15:49:38.154 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2022-06-29T15:49:38.155 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:49:38.155 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:49:38.155 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:49:38.155 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:49:38.155 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:49:38.156 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:49:38.156 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:49:38.156 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.156 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "89", 2022-06-29T15:49:38.157 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:49:38.157 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:49:38.157 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:49:38.157 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:49:38.158 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:49:38.158 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:49:38.158 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:49:38.158 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:49:38.159 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:49:38.159 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:49:38.159 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:49:38.160 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:49:38.160 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:49:38.160 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:49:38.160 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:49:38.161 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:49:38.161 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:49:38.161 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:49:38.162 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:49:38.162 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:49:38.163 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:49:38.163 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:49:38.164 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:49:38.164 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:49:38.164 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:49:38.165 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:49:38.165 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.166 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:49:38.166 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:49:38.167 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:49:38.167 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:49:38.167 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:49:38.168 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:49:38.168 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:49:38.169 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:49:38.169 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:49:38.169 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:49:38.169 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:49:38.169 INFO:teuthology.orchestra.run.smithi047.stdout: "options": { 2022-06-29T15:49:38.170 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_bias": 4, 2022-06-29T15:49:38.170 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_min": 16, 2022-06-29T15:49:38.170 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_priority": 5 2022-06-29T15:49:38.170 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.171 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:49:38.171 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs": { 2022-06-29T15:49:38.171 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": "cephfs" 2022-06-29T15:49:38.171 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:38.172 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:38.172 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.172 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.172 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 16, 2022-06-29T15:49:38.173 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "cephfs_data", 2022-06-29T15:49:38.173 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:49:26.726660+0000", 2022-06-29T15:49:38.173 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:49:38.173 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:49:38.173 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:49:38.174 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:49:38.174 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:49:38.175 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:49:38.175 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:49:38.175 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:49:38.175 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2022-06-29T15:49:38.176 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2022-06-29T15:49:38.176 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2022-06-29T15:49:38.176 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2022-06-29T15:49:38.176 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2022-06-29T15:49:38.177 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:49:38.177 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:49:38.177 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:49:38.177 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:49:38.178 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:49:38.178 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:49:38.178 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:49:38.178 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.179 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "90", 2022-06-29T15:49:38.179 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:49:38.179 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:49:38.179 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:49:38.180 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:49:38.180 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:49:38.180 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:49:38.180 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:49:38.180 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:49:38.181 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:49:38.181 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:49:38.181 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:49:38.182 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:49:38.182 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:49:38.182 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:49:38.182 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:49:38.182 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:49:38.183 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:49:38.183 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:49:38.183 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:49:38.183 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:49:38.184 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:49:38.184 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:49:38.184 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:49:38.184 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:49:38.185 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:49:38.185 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:49:38.185 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.185 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:49:38.186 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:49:38.186 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:49:38.186 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:49:38.186 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:49:38.187 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:49:38.187 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:49:38.187 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:49:38.187 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:49:38.187 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:49:38.188 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:49:38.188 INFO:teuthology.orchestra.run.smithi047.stdout: "options": {}, 2022-06-29T15:49:38.188 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:49:38.188 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs": { 2022-06-29T15:49:38.189 INFO:teuthology.orchestra.run.smithi047.stdout: "data": "cephfs" 2022-06-29T15:49:38.189 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:38.189 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:38.189 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:38.190 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:49:38.190 INFO:teuthology.orchestra.run.smithi047.stdout: "osds": [ 2022-06-29T15:49:38.190 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.190 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2022-06-29T15:49:38.191 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "e23eb023-4706-4201-bf72-4bf19c912c5d", 2022-06-29T15:49:38.191 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:49:38.191 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:49:38.191 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:49:38.192 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:49:38.192 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:49:38.192 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:49:38.192 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:49:38.192 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 88, 2022-06-29T15:49:38.193 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:49:38.193 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:49:38.193 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:49:38.193 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:38.194 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.194 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:38.194 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6818", 2022-06-29T15:49:38.194 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:49:38.195 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.195 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.195 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:38.195 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6819", 2022-06-29T15:49:38.196 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:49:38.196 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:38.197 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:38.197 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.198 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:49:38.198 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:38.199 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.199 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:38.199 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6820", 2022-06-29T15:49:38.200 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:49:38.200 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.201 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.201 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:38.201 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6821", 2022-06-29T15:49:38.202 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:49:38.202 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:38.202 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:38.202 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.203 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:49:38.203 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:38.203 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.203 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:38.204 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6824", 2022-06-29T15:49:38.204 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:49:38.204 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.204 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.205 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:38.205 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6825", 2022-06-29T15:49:38.205 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:49:38.205 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:38.205 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:38.206 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.206 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:49:38.206 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:38.207 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.207 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:38.207 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6822", 2022-06-29T15:49:38.207 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:49:38.207 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.208 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.208 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:38.208 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6823", 2022-06-29T15:49:38.208 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:49:38.209 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:38.209 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:38.209 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.209 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6819/13096", 2022-06-29T15:49:38.210 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6821/13096", 2022-06-29T15:49:38.210 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6825/13096", 2022-06-29T15:49:38.210 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6823/13096", 2022-06-29T15:49:38.210 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:49:38.211 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:49:38.211 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:49:38.211 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:38.211 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.212 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.212 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2022-06-29T15:49:38.212 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "228bb08d-2d64-43d9-9560-bfaeff154452", 2022-06-29T15:49:38.212 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:49:38.213 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:49:38.213 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:49:38.213 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:49:38.213 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:49:38.213 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:49:38.214 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:49:38.214 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 87, 2022-06-29T15:49:38.214 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:49:38.215 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:49:38.215 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:49:38.215 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:38.215 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.215 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:38.216 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6826", 2022-06-29T15:49:38.216 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:49:38.216 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.216 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.217 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:38.217 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6827", 2022-06-29T15:49:38.217 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:49:38.217 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:38.218 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:38.218 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.218 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:49:38.218 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:38.219 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.219 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:38.219 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6828", 2022-06-29T15:49:38.220 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:49:38.220 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.220 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.220 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:38.221 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6829", 2022-06-29T15:49:38.221 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:49:38.221 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:38.221 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:38.222 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.222 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:49:38.222 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:38.222 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.223 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:38.223 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6832", 2022-06-29T15:49:38.223 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:49:38.223 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.224 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.224 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:38.224 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6833", 2022-06-29T15:49:38.225 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:49:38.225 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:38.225 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:38.226 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.226 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:49:38.226 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:38.226 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.227 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:38.227 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6830", 2022-06-29T15:49:38.227 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:49:38.227 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.228 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.228 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:38.229 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6831", 2022-06-29T15:49:38.229 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:49:38.230 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:38.230 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:38.230 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.231 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6827/13100", 2022-06-29T15:49:38.231 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6829/13100", 2022-06-29T15:49:38.232 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6833/13100", 2022-06-29T15:49:38.232 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6831/13100", 2022-06-29T15:49:38.233 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:49:38.233 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:49:38.234 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:49:38.234 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:38.235 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.235 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.235 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2022-06-29T15:49:38.235 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "797a5452-1510-4640-a111-c9c2fcf707eb", 2022-06-29T15:49:38.236 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:49:38.236 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:49:38.236 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:49:38.236 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:49:38.237 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:49:38.237 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:49:38.237 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:49:38.237 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 88, 2022-06-29T15:49:38.237 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:49:38.238 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:49:38.238 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:49:38.238 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:38.239 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.239 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:38.239 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6810", 2022-06-29T15:49:38.239 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:49:38.239 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.240 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.240 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:38.240 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6811", 2022-06-29T15:49:38.240 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:49:38.241 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:38.241 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:38.241 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.241 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:49:38.242 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:38.242 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.242 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:38.242 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6812", 2022-06-29T15:49:38.243 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:49:38.243 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.243 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.243 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:38.244 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6813", 2022-06-29T15:49:38.244 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:49:38.244 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:38.244 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:38.245 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.245 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:49:38.245 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:38.245 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.245 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:38.246 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6816", 2022-06-29T15:49:38.246 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:49:38.246 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.247 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.247 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:38.247 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6817", 2022-06-29T15:49:38.247 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:49:38.247 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:38.248 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:38.248 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.248 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:49:38.248 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:38.249 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.249 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:38.249 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6814", 2022-06-29T15:49:38.250 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:49:38.250 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.250 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.251 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:38.251 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6815", 2022-06-29T15:49:38.252 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:49:38.252 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:38.253 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:38.253 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.254 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6811/13098", 2022-06-29T15:49:38.254 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6813/13098", 2022-06-29T15:49:38.255 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6817/13098", 2022-06-29T15:49:38.255 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6815/13098", 2022-06-29T15:49:38.256 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:49:38.256 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:49:38.256 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:49:38.257 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:38.257 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.258 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.258 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2022-06-29T15:49:38.259 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "55b24fd7-bb71-4a01-b3fa-a09caa772a62", 2022-06-29T15:49:38.259 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:49:38.259 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:49:38.260 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:49:38.260 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:49:38.261 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:49:38.261 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:49:38.262 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:49:38.262 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 88, 2022-06-29T15:49:38.262 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:49:38.262 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:49:38.263 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:49:38.263 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:38.263 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.263 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:38.264 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6802", 2022-06-29T15:49:38.264 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:49:38.264 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.264 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.265 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:38.265 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6803", 2022-06-29T15:49:38.265 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:49:38.265 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:38.266 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:38.266 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.266 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:49:38.266 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:38.267 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.267 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:38.267 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6804", 2022-06-29T15:49:38.267 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:49:38.268 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.268 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.268 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:38.268 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6805", 2022-06-29T15:49:38.269 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:49:38.269 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:38.269 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:38.269 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.270 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:49:38.270 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:38.270 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.270 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:38.271 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6808", 2022-06-29T15:49:38.271 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:49:38.271 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.271 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.272 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:38.272 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6809", 2022-06-29T15:49:38.272 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:49:38.272 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:38.272 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:38.273 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.273 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:49:38.274 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:38.274 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.275 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:38.275 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6806", 2022-06-29T15:49:38.275 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:49:38.276 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.276 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.277 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:38.277 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6807", 2022-06-29T15:49:38.278 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:49:38.278 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:38.279 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:38.279 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.280 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6803/13097", 2022-06-29T15:49:38.280 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6805/13097", 2022-06-29T15:49:38.280 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6809/13097", 2022-06-29T15:49:38.281 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6807/13097", 2022-06-29T15:49:38.281 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:49:38.282 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:49:38.282 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:49:38.283 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:38.283 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.283 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.284 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2022-06-29T15:49:38.284 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3", 2022-06-29T15:49:38.285 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:49:38.285 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:49:38.285 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:49:38.286 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:49:38.286 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:49:38.286 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:49:38.286 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:49:38.287 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 88, 2022-06-29T15:49:38.287 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:49:38.287 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:49:38.287 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:49:38.288 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:38.288 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.288 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:38.288 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6816", 2022-06-29T15:49:38.289 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:49:38.289 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.289 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.289 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:38.290 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6817", 2022-06-29T15:49:38.290 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:49:38.290 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:38.290 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:38.291 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.291 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:49:38.291 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:38.291 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.292 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:38.292 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6818", 2022-06-29T15:49:38.292 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:49:38.292 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.292 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.293 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:38.293 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6819", 2022-06-29T15:49:38.293 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:49:38.293 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:38.294 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:38.294 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.294 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:49:38.294 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:38.295 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.295 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:38.295 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6822", 2022-06-29T15:49:38.295 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:49:38.296 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.296 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.296 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:38.296 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6823", 2022-06-29T15:49:38.297 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:49:38.297 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:38.297 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:38.297 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.298 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:49:38.298 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:38.298 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.298 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:38.299 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6820", 2022-06-29T15:49:38.299 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:49:38.299 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.299 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.300 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:38.300 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6821", 2022-06-29T15:49:38.300 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:49:38.300 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:38.301 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:38.301 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.301 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6817/13396", 2022-06-29T15:49:38.302 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6819/13396", 2022-06-29T15:49:38.302 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6823/13396", 2022-06-29T15:49:38.302 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6821/13396", 2022-06-29T15:49:38.302 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:49:38.303 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:49:38.303 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:49:38.303 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:38.303 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.304 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.304 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2022-06-29T15:49:38.304 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "f77beebe-6cca-4244-b2d4-677825ebf919", 2022-06-29T15:49:38.304 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:49:38.304 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:49:38.305 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:49:38.305 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:49:38.305 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:49:38.305 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:49:38.306 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:49:38.306 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 88, 2022-06-29T15:49:38.307 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:49:38.307 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:49:38.307 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:49:38.308 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:38.308 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.309 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:38.309 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6800", 2022-06-29T15:49:38.309 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:49:38.310 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.310 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.311 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:38.311 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6801", 2022-06-29T15:49:38.312 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:49:38.312 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:38.312 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:38.313 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.313 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:49:38.314 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:38.314 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.315 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:38.315 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6802", 2022-06-29T15:49:38.315 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:49:38.316 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.316 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.316 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:38.316 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6803", 2022-06-29T15:49:38.317 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:49:38.317 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:38.317 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:38.317 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.318 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:49:38.318 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:38.318 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.318 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:38.319 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6806", 2022-06-29T15:49:38.319 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:49:38.319 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.319 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.320 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:38.320 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6807", 2022-06-29T15:49:38.320 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:49:38.320 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:38.321 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:38.321 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.321 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:49:38.321 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:38.322 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.322 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:38.322 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6804", 2022-06-29T15:49:38.322 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:49:38.323 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.323 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.323 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:38.323 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6805", 2022-06-29T15:49:38.324 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:49:38.324 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:38.324 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:38.324 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.325 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6801/13393", 2022-06-29T15:49:38.325 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6803/13393", 2022-06-29T15:49:38.325 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6807/13393", 2022-06-29T15:49:38.325 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6805/13393", 2022-06-29T15:49:38.326 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:49:38.326 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:49:38.326 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:49:38.326 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:38.326 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.327 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.327 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2022-06-29T15:49:38.327 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "af0ca9b3-c02c-4910-834b-bf0b8dc6713c", 2022-06-29T15:49:38.327 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:49:38.328 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:49:38.328 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:49:38.328 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:49:38.328 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:49:38.329 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:49:38.329 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:49:38.329 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 88, 2022-06-29T15:49:38.329 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:49:38.330 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:49:38.330 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:49:38.330 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:38.330 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.331 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:38.331 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6824", 2022-06-29T15:49:38.331 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:49:38.331 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.332 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.332 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:38.332 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6825", 2022-06-29T15:49:38.332 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:49:38.333 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:38.333 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:38.333 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.333 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:49:38.334 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:38.334 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.334 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:38.334 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6826", 2022-06-29T15:49:38.335 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:49:38.335 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.335 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.335 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:38.335 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6827", 2022-06-29T15:49:38.336 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:49:38.336 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:38.336 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:38.336 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.337 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:49:38.337 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:38.337 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.337 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:38.338 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6830", 2022-06-29T15:49:38.338 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:49:38.338 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.338 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.339 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:38.339 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6831", 2022-06-29T15:49:38.339 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:49:38.339 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:38.339 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:38.340 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.340 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:49:38.340 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:38.340 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.341 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:38.341 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6828", 2022-06-29T15:49:38.341 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:49:38.341 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.342 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.342 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:38.342 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6829", 2022-06-29T15:49:38.342 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:49:38.343 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:38.343 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:38.343 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.343 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6825/13395", 2022-06-29T15:49:38.344 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6827/13395", 2022-06-29T15:49:38.344 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6831/13395", 2022-06-29T15:49:38.344 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6829/13395", 2022-06-29T15:49:38.344 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:49:38.345 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:49:38.345 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:49:38.345 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:38.345 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.345 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.346 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2022-06-29T15:49:38.346 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "f19c3ca6-df88-4bde-a150-b51a53f092e1", 2022-06-29T15:49:38.346 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:49:38.346 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:49:38.347 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:49:38.347 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:49:38.347 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:49:38.347 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:49:38.348 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:49:38.348 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 75, 2022-06-29T15:49:38.348 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:49:38.348 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:49:38.349 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:49:38.349 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:38.349 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.349 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:38.350 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6808", 2022-06-29T15:49:38.350 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:49:38.350 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.350 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.351 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:38.351 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6809", 2022-06-29T15:49:38.351 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:49:38.351 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:38.352 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:38.352 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.352 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:49:38.352 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:38.352 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.353 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:38.353 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6810", 2022-06-29T15:49:38.353 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:49:38.353 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.354 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.354 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:38.354 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6811", 2022-06-29T15:49:38.354 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:49:38.355 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:38.355 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:38.355 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.355 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:49:38.356 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:38.356 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.356 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:38.356 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6814", 2022-06-29T15:49:38.357 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:49:38.357 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.357 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.357 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:38.358 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6815", 2022-06-29T15:49:38.358 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:49:38.358 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:38.358 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:38.359 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.359 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:49:38.359 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:38.359 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.360 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:38.360 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6812", 2022-06-29T15:49:38.360 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:49:38.360 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.361 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.361 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:38.361 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6813", 2022-06-29T15:49:38.361 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:49:38.362 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:38.362 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:38.362 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.362 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6809/13394", 2022-06-29T15:49:38.363 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6811/13394", 2022-06-29T15:49:38.363 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6815/13394", 2022-06-29T15:49:38.363 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6813/13394", 2022-06-29T15:49:38.363 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:49:38.364 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:49:38.364 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:49:38.364 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:38.364 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:38.365 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:49:38.365 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_xinfo": [ 2022-06-29T15:49:38.365 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.365 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2022-06-29T15:49:38.365 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:49:38.366 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:49:38.366 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:49:38.366 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:49:38.366 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:49:38.367 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.720513+0000", 2022-06-29T15:49:38.367 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:49:38.367 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.367 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.368 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2022-06-29T15:49:38.368 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:49:38.368 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:49:38.368 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:49:38.369 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:49:38.369 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:49:38.369 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.772612+0000", 2022-06-29T15:49:38.369 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:49:38.370 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.370 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.370 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2022-06-29T15:49:38.370 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:49:38.371 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:49:38.371 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:49:38.371 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:49:38.371 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:49:38.372 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.450223+0000", 2022-06-29T15:49:38.372 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:49:38.372 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.372 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.372 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2022-06-29T15:49:38.373 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:49:38.373 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:49:38.373 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:49:38.374 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:49:38.374 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:49:38.374 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.477856+0000", 2022-06-29T15:49:38.374 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:49:38.374 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.375 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.375 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2022-06-29T15:49:38.375 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:49:38.375 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:49:38.376 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:49:38.376 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:49:38.376 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:49:38.376 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.539485+0000", 2022-06-29T15:49:38.377 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:49:38.377 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.377 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.377 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2022-06-29T15:49:38.378 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:49:38.378 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:49:38.378 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:49:38.378 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:49:38.379 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:49:38.379 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.487040+0000", 2022-06-29T15:49:38.379 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:49:38.379 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.380 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.380 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2022-06-29T15:49:38.380 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:49:38.380 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:49:38.381 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:49:38.381 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:49:38.381 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:49:38.381 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.748685+0000", 2022-06-29T15:49:38.382 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:49:38.382 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.382 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:38.382 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2022-06-29T15:49:38.383 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:49:38.383 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:49:38.383 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:49:38.383 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:49:38.384 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:49:38.384 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.459516+0000", 2022-06-29T15:49:38.384 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:49:38.384 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:38.385 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:49:38.385 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap": [], 2022-06-29T15:49:38.385 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap_items": [], 2022-06-29T15:49:38.385 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_temp": [], 2022-06-29T15:49:38.385 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_temp": [], 2022-06-29T15:49:38.386 INFO:teuthology.orchestra.run.smithi047.stdout: "blacklist": {}, 2022-06-29T15:49:38.386 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profiles": { 2022-06-29T15:49:38.386 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs_data": { 2022-06-29T15:49:38.386 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-device-class": "", 2022-06-29T15:49:38.387 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2022-06-29T15:49:38.387 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-root": "default", 2022-06-29T15:49:38.387 INFO:teuthology.orchestra.run.smithi047.stdout: "jerasure-per-chunk-alignment": "false", 2022-06-29T15:49:38.387 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2022-06-29T15:49:38.388 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "2", 2022-06-29T15:49:38.388 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2022-06-29T15:49:38.388 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2022-06-29T15:49:38.388 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van", 2022-06-29T15:49:38.389 INFO:teuthology.orchestra.run.smithi047.stdout: "w": "8" 2022-06-29T15:49:38.389 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.389 INFO:teuthology.orchestra.run.smithi047.stdout: "default": { 2022-06-29T15:49:38.389 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2022-06-29T15:49:38.390 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2022-06-29T15:49:38.390 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "1", 2022-06-29T15:49:38.390 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2022-06-29T15:49:38.390 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2022-06-29T15:49:38.391 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van" 2022-06-29T15:49:38.391 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:38.391 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:38.391 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps_queue": [], 2022-06-29T15:49:38.392 INFO:teuthology.orchestra.run.smithi047.stdout: "new_removed_snaps": [], 2022-06-29T15:49:38.392 INFO:teuthology.orchestra.run.smithi047.stdout: "new_purged_snaps": [], 2022-06-29T15:49:38.392 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_node_flags": {}, 2022-06-29T15:49:38.392 INFO:teuthology.orchestra.run.smithi047.stdout: "device_class_flags": {} 2022-06-29T15:49:38.393 INFO:teuthology.orchestra.run.smithi047.stdout:} 2022-06-29T15:49:38.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.089+0000 7fb602676700 1 -- 172.21.15.47:0/3751243324 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb5dc053980 msgr2=0x7fb5dc055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:38.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.089+0000 7fb602676700 1 --2- 172.21.15.47:0/3751243324 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb5dc053980 0x7fb5dc055e40 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7fb5ec00c000 tx=0x7fb5ec008040).stop 2022-06-29T15:49:38.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.089+0000 7fb602676700 1 -- 172.21.15.47:0/3751243324 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5fc147060 msgr2=0x7fb5fc097980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:38.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.089+0000 7fb602676700 1 --2- 172.21.15.47:0/3751243324 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5fc147060 0x7fb5fc097980 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 rx=0x7fb5f4006e00 tx=0x7fb5f40091d0).stop 2022-06-29T15:49:38.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.089+0000 7fb602676700 1 -- 172.21.15.47:0/3751243324 shutdown_connections 2022-06-29T15:49:38.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.089+0000 7fb602676700 1 --2- 172.21.15.47:0/3751243324 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5fc147060 0x7fb5fc097980 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:38.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.089+0000 7fb602676700 1 --2- 172.21.15.47:0/3751243324 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb5dc053980 0x7fb5dc055e40 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:38.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.089+0000 7fb602676700 1 --2- 172.21.15.47:0/3751243324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5fc0b2a90 0x7fb5fc096dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:38.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.089+0000 7fb602676700 1 --2- 172.21.15.47:0/3751243324 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb5fc061600 0x7fb5fc145de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:38.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.089+0000 7fb602676700 1 -- 172.21.15.47:0/3751243324 >> 172.21.15.47:0/3751243324 conn(0x7fb5fc05f0b0 msgr2=0x7fb5fc0b6450 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:38.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.089+0000 7fb602676700 1 -- 172.21.15.47:0/3751243324 shutdown_connections 2022-06-29T15:49:38.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.089+0000 7fb602676700 1 -- 172.21.15.47:0/3751243324 wait complete. 2022-06-29T15:49:38.399 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:49:38.515 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.505+0000 7fb9aa659700 1 Processor -- start 2022-06-29T15:49:38.515 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.505+0000 7fb9aa659700 1 -- start start 2022-06-29T15:49:38.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.509+0000 7fb9aa659700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb9a40b4420 0x7fb9a41401b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:38.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.509+0000 7fb9aa659700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb9a40b2490 0x7fb9a41406a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:38.520 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.509+0000 7fb9aa659700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9a40b4800 con 0x7fb9a4064820 2022-06-29T15:49:38.520 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.509+0000 7fb9aa659700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9a40b2bd0 con 0x7fb9a40b4420 2022-06-29T15:49:38.520 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.509+0000 7fb9aa659700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a4140b90 con 0x7fb9a40b2490 2022-06-29T15:49:38.520 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.509+0000 7fb9a3fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb9a40b4420 0x7fb9a41401b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:38.520 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.509+0000 7fb9a3fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb9a40b4420 0x7fb9a41401b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51044/0 (socket says 172.21.15.47:51044) 2022-06-29T15:49:38.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.509+0000 7fb9a3fff700 1 -- 172.21.15.47:0/821684808 learned_addr learned my addr 172.21.15.47:0/821684808 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:38.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.509+0000 7fb9a37fe700 1 --2- 172.21.15.47:0/821684808 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb9a40b2490 0x7fb9a41406a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:38.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.509+0000 7fb9a2ffd700 1 -- 172.21.15.47:0/821684808 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1837623419 0 0) 0x7fb9a40b4800 con 0x7fb9a4064820 2022-06-29T15:49:38.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.509+0000 7fb9a2ffd700 1 -- 172.21.15.47:0/821684808 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb984003260 con 0x7fb9a4064820 2022-06-29T15:49:38.522 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.509+0000 7fb9a3fff700 1 -- 172.21.15.47:0/821684808 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb9a40b2490 msgr2=0x7fb9a41406a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:38.522 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.509+0000 7fb9a3fff700 1 --2- 172.21.15.47:0/821684808 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb9a40b2490 0x7fb9a41406a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:38.522 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.509+0000 7fb9a3fff700 1 -- 172.21.15.47:0/821684808 >> v1:172.21.15.64:6789/0 conn(0x7fb9a4064820 legacy=0x7fb9a4064cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:38.522 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.509+0000 7fb9a3fff700 1 -- 172.21.15.47:0/821684808 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9a40b2930 con 0x7fb9a40b4420 2022-06-29T15:49:38.523 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.509+0000 7fb9a3fff700 1 --2- 172.21.15.47:0/821684808 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9a40b4420 0x7fb9a41401b0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 rx=0x7fb994002f70 tx=0x7fb994006eb0).ready entity=mon.0 client_cookie=11ba31a9e73cacfa server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:38.523 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.509+0000 7fb9a2ffd700 1 -- 172.21.15.47:0/821684808 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb99400d5c0 con 0x7fb9a40b4420 2022-06-29T15:49:38.523 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.509+0000 7fb9a2ffd700 1 -- 172.21.15.47:0/821684808 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb994000e90 con 0x7fb9a40b4420 2022-06-29T15:49:38.523 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.509+0000 7fb9a2ffd700 1 -- 172.21.15.47:0/821684808 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb99400d950 con 0x7fb9a40b4420 2022-06-29T15:49:38.524 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.509+0000 7fb9aa659700 1 -- 172.21.15.47:0/821684808 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9a40b4420 msgr2=0x7fb9a41401b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:38.524 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.509+0000 7fb9aa659700 1 --2- 172.21.15.47:0/821684808 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9a40b4420 0x7fb9a41401b0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 rx=0x7fb994002f70 tx=0x7fb994006eb0).stop 2022-06-29T15:49:38.524 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.509+0000 7fb9aa659700 1 -- 172.21.15.47:0/821684808 shutdown_connections 2022-06-29T15:49:38.524 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.509+0000 7fb9aa659700 1 --2- 172.21.15.47:0/821684808 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb9a40b2490 0x7fb9a41406a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:38.524 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.509+0000 7fb9aa659700 1 --2- 172.21.15.47:0/821684808 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9a40b4420 0x7fb9a41401b0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:38.525 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.509+0000 7fb9aa659700 1 -- 172.21.15.47:0/821684808 >> 172.21.15.47:0/821684808 conn(0x7fb9a405c510 msgr2=0x7fb9a405c920 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:38.525 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.509+0000 7fb9aa659700 1 -- 172.21.15.47:0/821684808 shutdown_connections 2022-06-29T15:49:38.526 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.509+0000 7fb9aa659700 1 -- 172.21.15.47:0/821684808 wait complete. 2022-06-29T15:49:38.529 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.513+0000 7fb9aa659700 1 Processor -- start 2022-06-29T15:49:38.530 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.513+0000 7fb9aa659700 1 -- start start 2022-06-29T15:49:38.530 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.513+0000 7fb9aa659700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9a4064820 0x7fb9a413b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:38.531 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.513+0000 7fb9aa659700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9a40b2490 0x7fb9a4136f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:38.531 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.513+0000 7fb9aa659700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb9a40b4420 0x7fb9a4137440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:38.531 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.513+0000 7fb9aa659700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a4155ce0 con 0x7fb9a4064820 2022-06-29T15:49:38.531 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.513+0000 7fb9aa659700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a4155f80 con 0x7fb9a40b2490 2022-06-29T15:49:38.532 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.513+0000 7fb9aa659700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a4156220 con 0x7fb9a40b4420 2022-06-29T15:49:38.532 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.513+0000 7fb9a3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9a4064820 0x7fb9a413b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:38.532 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.513+0000 7fb9a3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9a4064820 0x7fb9a413b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51048/0 (socket says 172.21.15.47:51048) 2022-06-29T15:49:38.533 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.513+0000 7fb9a3fff700 1 -- 172.21.15.47:0/1771218859 learned_addr learned my addr 172.21.15.47:0/1771218859 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:38.533 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.513+0000 7fb9a37fe700 1 --2- 172.21.15.47:0/1771218859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9a40b2490 0x7fb9a4136f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:38.533 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.513+0000 7fb9a8bf6700 1 --2- 172.21.15.47:0/1771218859 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb9a40b4420 0x7fb9a4137440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:38.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.513+0000 7fb9a3fff700 1 -- 172.21.15.47:0/1771218859 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb9a40b4420 msgr2=0x7fb9a4137440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:38.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.513+0000 7fb9a3fff700 1 --2- 172.21.15.47:0/1771218859 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb9a40b4420 0x7fb9a4137440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:38.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.513+0000 7fb9a3fff700 1 -- 172.21.15.47:0/1771218859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9a40b2490 msgr2=0x7fb9a4136f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:38.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.513+0000 7fb9a3fff700 1 --2- 172.21.15.47:0/1771218859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9a40b2490 0x7fb9a4136f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:38.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.513+0000 7fb9a3fff700 1 -- 172.21.15.47:0/1771218859 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb99400b1e0 con 0x7fb9a4064820 2022-06-29T15:49:38.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.513+0000 7fb9a8bf6700 1 --2- 172.21.15.47:0/1771218859 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb9a40b4420 0x7fb9a4137440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:49:38.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.513+0000 7fb9a3fff700 1 --2- 172.21.15.47:0/1771218859 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9a4064820 0x7fb9a413b750 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 rx=0x7fb994002f70 tx=0x7fb994015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:38.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.513+0000 7fb9a17fa700 1 -- 172.21.15.47:0/1771218859 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb994006b40 con 0x7fb9a4064820 2022-06-29T15:49:38.536 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.513+0000 7fb9a17fa700 1 -- 172.21.15.47:0/1771218859 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb99400de00 con 0x7fb9a4064820 2022-06-29T15:49:38.536 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.513+0000 7fb9aa659700 1 -- 172.21.15.47:0/1771218859 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9a413bc40 con 0x7fb9a4064820 2022-06-29T15:49:38.536 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.513+0000 7fb9a17fa700 1 -- 172.21.15.47:0/1771218859 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb994011e90 con 0x7fb9a4064820 2022-06-29T15:49:38.536 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.513+0000 7fb9aa659700 1 -- 172.21.15.47:0/1771218859 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9a41558b0 con 0x7fb9a4064820 2022-06-29T15:49:38.537 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.517+0000 7fb9aa659700 1 -- 172.21.15.47:0/1771218859 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9a41352d0 con 0x7fb9a4064820 2022-06-29T15:49:38.537 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.517+0000 7fb9a17fa700 1 -- 172.21.15.47:0/1771218859 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb994025030 con 0x7fb9a4064820 2022-06-29T15:49:38.537 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.517+0000 7fb9a17fa700 1 --2- 172.21.15.47:0/1771218859 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb988053af0 0x7fb988055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:38.537 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.517+0000 7fb9a17fa700 1 -- 172.21.15.47:0/1771218859 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb99406d3d0 con 0x7fb9a4064820 2022-06-29T15:49:38.538 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.517+0000 7fb9a37fe700 1 --2- 172.21.15.47:0/1771218859 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb988053af0 0x7fb988055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:38.538 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.517+0000 7fb9a37fe700 1 --2- 172.21.15.47:0/1771218859 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb988053af0 0x7fb988055fb0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7fb98c00b000 tx=0x7fb98c006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:38.539 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.525+0000 7fb9a17fa700 1 -- 172.21.15.47:0/1771218859 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb9a41352d0 con 0x7fb9a4064820 2022-06-29T15:49:38.700 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.693+0000 7fb9aa659700 1 -- 172.21.15.47:0/1771218859 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9a41352d0 con 0x7fb9a4064820 2022-06-29T15:49:38.701 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.693+0000 7fb9a17fa700 1 -- 172.21.15.47:0/1771218859 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+2493 (secure 0 0 0) 0x7fb9a41352d0 con 0x7fb9a4064820 2022-06-29T15:49:38.702 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:49:38.702 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":40,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6577,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/4286932616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":4286932616},{"type":"v1","addr":"172.21.15.64:6833","nonce":4286932616}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":38},{"gid":6657,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3528102962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3528102962},{"type":"v1","addr":"172.21.15.47:6835","nonce":3528102962}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":38}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:49:27.717245+0000","modified":"2022-06-29T15:49:28.719969+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6660},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6660":{"gid":6660,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6837/4146905199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":4146905199},{"type":"v1","addr":"172.21.15.47:6837","nonce":4146905199}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2022-06-29T15:49:38.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.697+0000 7fb9aa659700 1 -- 172.21.15.47:0/1771218859 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb988053af0 msgr2=0x7fb988055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:38.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.697+0000 7fb9aa659700 1 --2- 172.21.15.47:0/1771218859 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb988053af0 0x7fb988055fb0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7fb98c00b000 tx=0x7fb98c006df0).stop 2022-06-29T15:49:38.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.697+0000 7fb9aa659700 1 -- 172.21.15.47:0/1771218859 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9a4064820 msgr2=0x7fb9a413b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:38.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.697+0000 7fb9aa659700 1 --2- 172.21.15.47:0/1771218859 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9a4064820 0x7fb9a413b750 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 rx=0x7fb994002f70 tx=0x7fb994015000).stop 2022-06-29T15:49:38.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.697+0000 7fb9aa659700 1 -- 172.21.15.47:0/1771218859 shutdown_connections 2022-06-29T15:49:38.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.697+0000 7fb9aa659700 1 --2- 172.21.15.47:0/1771218859 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb9a40b4420 0x7fb9a4137440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:38.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.697+0000 7fb9aa659700 1 --2- 172.21.15.47:0/1771218859 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb988053af0 0x7fb988055fb0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:38.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.697+0000 7fb9aa659700 1 --2- 172.21.15.47:0/1771218859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9a40b2490 0x7fb9a4136f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:38.707 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.697+0000 7fb9aa659700 1 --2- 172.21.15.47:0/1771218859 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9a4064820 0x7fb9a413b750 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:38.707 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.697+0000 7fb9aa659700 1 -- 172.21.15.47:0/1771218859 >> 172.21.15.47:0/1771218859 conn(0x7fb9a405c510 msgr2=0x7fb9a4066850 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:38.707 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.697+0000 7fb9aa659700 1 -- 172.21.15.47:0/1771218859 shutdown_connections 2022-06-29T15:49:38.707 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.697+0000 7fb9aa659700 1 -- 172.21.15.47:0/1771218859 wait complete. 2022-06-29T15:49:38.708 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 40 2022-06-29T15:49:38.715 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-06-29T15:49:38.818 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.809+0000 7fa7a09c9700 1 Processor -- start 2022-06-29T15:49:38.818 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.809+0000 7fa7a09c9700 1 -- start start 2022-06-29T15:49:38.821 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.813+0000 7fa7a09c9700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa79c060450 0x7fa79c142820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:38.821 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.813+0000 7fa7a09c9700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa79c0b13f0 con 0x7fa79c0b0fe0 2022-06-29T15:49:38.822 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.813+0000 7fa7a09c9700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa79c098310 con 0x7fa79c097f30 2022-06-29T15:49:38.822 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.813+0000 7fa7a09c9700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa79c0b1a10 con 0x7fa79c060450 2022-06-29T15:49:38.822 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.813+0000 7fa793fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa79c060450 0x7fa79c142820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:38.823 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.813+0000 7fa79ad9d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa79c0b0fe0 0x7fa79c143de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36612/0 (socket says 172.21.15.47:36612) 2022-06-29T15:49:38.823 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.813+0000 7fa79ad9d700 1 -- 172.21.15.47:0/2911644878 learned_addr learned my addr 172.21.15.47:0/2911644878 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:49:38.823 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.813+0000 7fa793fff700 1 -- 172.21.15.47:0/2911644878 >> v1:172.21.15.64:6789/0 conn(0x7fa79c097f30 legacy=0x7fa79c0b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:38.823 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.813+0000 7fa793fff700 1 -- 172.21.15.47:0/2911644878 >> v1:172.21.15.47:6789/0 conn(0x7fa79c0b0fe0 legacy=0x7fa79c143de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:38.823 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.813+0000 7fa799d9b700 1 -- 172.21.15.47:0/2911644878 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 714537269 0 0) 0x7fa79c0b13f0 con 0x7fa79c0b0fe0 2022-06-29T15:49:38.824 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.813+0000 7fa793fff700 1 -- 172.21.15.47:0/2911644878 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa79c0b2800 con 0x7fa79c060450 2022-06-29T15:49:38.824 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.813+0000 7fa793fff700 1 --2- 172.21.15.47:0/2911644878 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa79c060450 0x7fa79c142820 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 rx=0x7fa784002f70 tx=0x7fa784006df0).ready entity=mon.0 client_cookie=eb14660b64212518 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:38.824 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.813+0000 7fa799d9b700 1 -- 172.21.15.47:0/2911644878 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa78400d610 con 0x7fa79c060450 2022-06-29T15:49:38.824 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.813+0000 7fa799d9b700 1 -- 172.21.15.47:0/2911644878 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa784000e90 con 0x7fa79c060450 2022-06-29T15:49:38.825 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.813+0000 7fa799d9b700 1 -- 172.21.15.47:0/2911644878 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa78400d960 con 0x7fa79c060450 2022-06-29T15:49:38.828 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.813+0000 7fa7a09c9700 1 -- 172.21.15.47:0/2911644878 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa79c060450 msgr2=0x7fa79c142820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:38.828 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.813+0000 7fa7a09c9700 1 --2- 172.21.15.47:0/2911644878 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa79c060450 0x7fa79c142820 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 rx=0x7fa784002f70 tx=0x7fa784006df0).stop 2022-06-29T15:49:38.829 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.813+0000 7fa7a09c9700 1 -- 172.21.15.47:0/2911644878 shutdown_connections 2022-06-29T15:49:38.829 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.813+0000 7fa7a09c9700 1 --2- 172.21.15.47:0/2911644878 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa79c060450 0x7fa79c142820 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:38.829 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.813+0000 7fa7a09c9700 1 -- 172.21.15.47:0/2911644878 >> 172.21.15.47:0/2911644878 conn(0x7fa79c05f0b0 msgr2=0x7fa79c05f490 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:38.829 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.813+0000 7fa7a09c9700 1 -- 172.21.15.47:0/2911644878 shutdown_connections 2022-06-29T15:49:38.830 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.813+0000 7fa7a09c9700 1 -- 172.21.15.47:0/2911644878 wait complete. 2022-06-29T15:49:38.830 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.817+0000 7fa7a09c9700 1 Processor -- start 2022-06-29T15:49:38.830 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.817+0000 7fa7a09c9700 1 -- start start 2022-06-29T15:49:38.830 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.817+0000 7fa7a09c9700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa79c060450 0x7fa79c099f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:38.831 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.817+0000 7fa7a09c9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa79c097f30 0x7fa79c0aef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:38.831 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.817+0000 7fa7a09c9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa79c0b0fe0 0x7fa79c0af410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:38.831 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.817+0000 7fa7a09c9700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa79c0b3610 con 0x7fa79c0b0fe0 2022-06-29T15:49:38.831 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.817+0000 7fa7a09c9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa79c151100 con 0x7fa79c097f30 2022-06-29T15:49:38.832 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.817+0000 7fa7a09c9700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa79c0aa870 con 0x7fa79c060450 2022-06-29T15:49:38.832 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.817+0000 7fa79ad9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa79c0b0fe0 0x7fa79c0af410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:38.832 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.817+0000 7fa79ad9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa79c0b0fe0 0x7fa79c0af410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51066/0 (socket says 172.21.15.47:51066) 2022-06-29T15:49:38.832 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.817+0000 7fa79ad9d700 1 -- 172.21.15.47:0/3393526044 learned_addr learned my addr 172.21.15.47:0/3393526044 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:38.833 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.817+0000 7fa793fff700 1 --2- 172.21.15.47:0/3393526044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa79c097f30 0x7fa79c0aef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:38.833 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.817+0000 7fa79a59c700 1 --2- 172.21.15.47:0/3393526044 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa79c060450 0x7fa79c099f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:38.833 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.817+0000 7fa79ad9d700 1 -- 172.21.15.47:0/3393526044 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa79c060450 msgr2=0x7fa79c099f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:38.833 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.817+0000 7fa79ad9d700 1 --2- 172.21.15.47:0/3393526044 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa79c060450 0x7fa79c099f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:38.834 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.817+0000 7fa79ad9d700 1 -- 172.21.15.47:0/3393526044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa79c097f30 msgr2=0x7fa79c0aef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:38.834 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.817+0000 7fa79ad9d700 1 --2- 172.21.15.47:0/3393526044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa79c097f30 0x7fa79c0aef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:38.834 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.817+0000 7fa79ad9d700 1 -- 172.21.15.47:0/3393526044 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa78400b180 con 0x7fa79c0b0fe0 2022-06-29T15:49:38.834 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.821+0000 7fa79ad9d700 1 --2- 172.21.15.47:0/3393526044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa79c0b0fe0 0x7fa79c0af410 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 rx=0x7fa794001250 tx=0x7fa794001360).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:38.835 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.821+0000 7fa7937fe700 1 -- 172.21.15.47:0/3393526044 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa794005ba0 con 0x7fa79c0b0fe0 2022-06-29T15:49:38.835 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.821+0000 7fa7937fe700 1 -- 172.21.15.47:0/3393526044 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa794005d00 con 0x7fa79c0b0fe0 2022-06-29T15:49:38.835 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.821+0000 7fa7a09c9700 1 -- 172.21.15.47:0/3393526044 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa79c151390 con 0x7fa79c0b0fe0 2022-06-29T15:49:38.835 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.821+0000 7fa7937fe700 1 -- 172.21.15.47:0/3393526044 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa794004a90 con 0x7fa79c0b0fe0 2022-06-29T15:49:38.836 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.821+0000 7fa7a09c9700 1 -- 172.21.15.47:0/3393526044 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa79c0aa170 con 0x7fa79c0b0fe0 2022-06-29T15:49:38.837 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.821+0000 7fa7a09c9700 1 -- 172.21.15.47:0/3393526044 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa79c0aaa70 con 0x7fa79c0b0fe0 2022-06-29T15:49:38.837 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.821+0000 7fa7937fe700 1 -- 172.21.15.47:0/3393526044 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa794004bf0 con 0x7fa79c0b0fe0 2022-06-29T15:49:38.838 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.821+0000 7fa7937fe700 1 --2- 172.21.15.47:0/3393526044 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa780053950 0x7fa780055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:38.838 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.821+0000 7fa7937fe700 1 -- 172.21.15.47:0/3393526044 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa79406bc90 con 0x7fa79c0b0fe0 2022-06-29T15:49:38.838 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.821+0000 7fa79a59c700 1 --2- 172.21.15.47:0/3393526044 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa780053950 0x7fa780055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:38.838 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.825+0000 7fa79a59c700 1 --2- 172.21.15.47:0/3393526044 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa780053950 0x7fa780055e10 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7fa78c00d000 tx=0x7fa78c009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:38.839 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.829+0000 7fa7937fe700 1 -- 172.21.15.47:0/3393526044 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa794042940 con 0x7fa79c0b0fe0 2022-06-29T15:49:39.006 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:38.997+0000 7fa7a09c9700 1 -- 172.21.15.47:0/3393526044 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa79c0573d0 con 0x7fa79c0b0fe0 2022-06-29T15:49:39.105 INFO:tasks.ceph.mon.a.smithi047.stderr:2022-06-29T15:49:39.101+0000 7ff9cf83c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-06-29T15:49:39.119 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.109+0000 7fa7937fe700 1 -- 172.21.15.47:0/3393526044 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v41) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa794045f60 con 0x7fa79c0b0fe0 2022-06-29T15:49:39.123 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.113+0000 7fa7a09c9700 1 -- 172.21.15.47:0/3393526044 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa780053950 msgr2=0x7fa780055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:39.123 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.113+0000 7fa7a09c9700 1 --2- 172.21.15.47:0/3393526044 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa780053950 0x7fa780055e10 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7fa78c00d000 tx=0x7fa78c009040).stop 2022-06-29T15:49:39.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.113+0000 7fa7a09c9700 1 -- 172.21.15.47:0/3393526044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa79c0b0fe0 msgr2=0x7fa79c0af410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:39.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.113+0000 7fa7a09c9700 1 --2- 172.21.15.47:0/3393526044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa79c0b0fe0 0x7fa79c0af410 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 rx=0x7fa794001250 tx=0x7fa794001360).stop 2022-06-29T15:49:39.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.117+0000 7fa7a09c9700 1 -- 172.21.15.47:0/3393526044 shutdown_connections 2022-06-29T15:49:39.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.117+0000 7fa7a09c9700 1 --2- 172.21.15.47:0/3393526044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa79c0b0fe0 0x7fa79c0af410 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:39.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.117+0000 7fa7a09c9700 1 --2- 172.21.15.47:0/3393526044 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa780053950 0x7fa780055e10 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:39.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.117+0000 7fa7a09c9700 1 --2- 172.21.15.47:0/3393526044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa79c097f30 0x7fa79c0aef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:39.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.117+0000 7fa7a09c9700 1 --2- 172.21.15.47:0/3393526044 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa79c060450 0x7fa79c099f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:39.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.117+0000 7fa7a09c9700 1 -- 172.21.15.47:0/3393526044 >> 172.21.15.47:0/3393526044 conn(0x7fa79c05f0b0 msgr2=0x7fa79c141600 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:39.127 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.117+0000 7fa7a09c9700 1 -- 172.21.15.47:0/3393526044 shutdown_connections 2022-06-29T15:49:39.127 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.117+0000 7fa7a09c9700 1 -- 172.21.15.47:0/3393526044 wait complete. 2022-06-29T15:49:39.127 INFO:teuthology.orchestra.run.smithi047.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-06-29T15:49:39.139 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:49:39.148 INFO:tasks.ceph.mds.a.smithi047.stdout:starting mds.a at 2022-06-29T15:49:39.239 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.233+0000 7f2466873700 1 Processor -- start 2022-06-29T15:49:39.240 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.233+0000 7f2466873700 1 -- start start 2022-06-29T15:49:39.243 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.233+0000 7f2466873700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2460067630 0x7f24600b2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:39.243 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.233+0000 7f2466873700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24600b4800 con 0x7f24600671d0 2022-06-29T15:49:39.243 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.233+0000 7f2466873700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24600b34f0 con 0x7f24600b4420 2022-06-29T15:49:39.244 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.233+0000 7f2466873700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24600b3300 con 0x7f2460067630 2022-06-29T15:49:39.244 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.233+0000 7f245f7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f24600671d0 0x7f2460144730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36638/0 (socket says 172.21.15.47:36638) 2022-06-29T15:49:39.244 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.233+0000 7f245f7fe700 1 -- 172.21.15.47:0/2701825063 learned_addr learned my addr 172.21.15.47:0/2701825063 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:49:39.245 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.233+0000 7f2464e10700 1 --2- 172.21.15.47:0/2701825063 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2460067630 0x7f24600b2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:39.245 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.233+0000 7f245effd700 1 -- 172.21.15.47:0/2701825063 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1716952553 0 0) 0x7f24600b4800 con 0x7f24600671d0 2022-06-29T15:49:39.245 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.233+0000 7f245effd700 1 -- 172.21.15.47:0/2701825063 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f244c003260 con 0x7f24600671d0 2022-06-29T15:49:39.245 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.233+0000 7f2464e10700 1 -- 172.21.15.47:0/2701825063 >> v1:172.21.15.64:6789/0 conn(0x7f24600b4420 legacy=0x7f2460066ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:39.245 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.233+0000 7f2464e10700 1 -- 172.21.15.47:0/2701825063 >> v1:172.21.15.47:6789/0 conn(0x7f24600671d0 legacy=0x7f2460144730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:39.246 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.233+0000 7f245effd700 1 -- 172.21.15.47:0/2701825063 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1661987287 0 0) 0x7f244c003260 con 0x7f24600671d0 2022-06-29T15:49:39.246 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.233+0000 7f2464e10700 1 -- 172.21.15.47:0/2701825063 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24600b3ab0 con 0x7f2460067630 2022-06-29T15:49:39.246 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.233+0000 7f2464e10700 1 --2- 172.21.15.47:0/2701825063 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2460067630 0x7f24600b2800 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f2454001800 tx=0x7f24580091d0).ready entity=mon.2 client_cookie=c7d63c5864d2188e server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:39.246 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.233+0000 7f245effd700 1 -- 172.21.15.47:0/2701825063 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2454001660 con 0x7f2460067630 2022-06-29T15:49:39.247 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.233+0000 7f245effd700 1 -- 172.21.15.47:0/2701825063 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f245800b580 con 0x7f2460067630 2022-06-29T15:49:39.247 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.233+0000 7f245effd700 1 -- 172.21.15.47:0/2701825063 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f245800b6f0 con 0x7f2460067630 2022-06-29T15:49:39.247 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.233+0000 7f2466873700 1 -- 172.21.15.47:0/2701825063 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2460067630 msgr2=0x7f24600b2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:39.247 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.233+0000 7f2466873700 1 --2- 172.21.15.47:0/2701825063 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2460067630 0x7f24600b2800 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f2454001800 tx=0x7f24580091d0).stop 2022-06-29T15:49:39.248 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.233+0000 7f2466873700 1 -- 172.21.15.47:0/2701825063 shutdown_connections 2022-06-29T15:49:39.248 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.233+0000 7f2466873700 1 --2- 172.21.15.47:0/2701825063 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2460067630 0x7f24600b2800 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:39.248 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.233+0000 7f2466873700 1 -- 172.21.15.47:0/2701825063 >> 172.21.15.47:0/2701825063 conn(0x7f246005c510 msgr2=0x7f246005c920 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:39.248 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.233+0000 7f2466873700 1 -- 172.21.15.47:0/2701825063 shutdown_connections 2022-06-29T15:49:39.248 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.233+0000 7f2466873700 1 -- 172.21.15.47:0/2701825063 wait complete. 2022-06-29T15:49:39.252 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.237+0000 7f2466873700 1 Processor -- start 2022-06-29T15:49:39.252 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.237+0000 7f2466873700 1 -- start start 2022-06-29T15:49:39.253 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.237+0000 7f2466873700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f24600671d0 0x7f24600b05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:39.253 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.237+0000 7f2466873700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2460067630 0x7f2460149280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:39.253 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.237+0000 7f2466873700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24600b4420 0x7f2460149770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:39.253 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.237+0000 7f2466873700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f246014aa70 con 0x7f2460067630 2022-06-29T15:49:39.254 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.237+0000 7f2466873700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2460060290 con 0x7f24600b4420 2022-06-29T15:49:39.254 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.237+0000 7f2466873700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2460062000 con 0x7f24600671d0 2022-06-29T15:49:39.254 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.237+0000 7f245f7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2460067630 0x7f2460149280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:39.254 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.237+0000 7f245f7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2460067630 0x7f2460149280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51090/0 (socket says 172.21.15.47:51090) 2022-06-29T15:49:39.255 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.237+0000 7f245f7fe700 1 -- 172.21.15.47:0/1570927895 learned_addr learned my addr 172.21.15.47:0/1570927895 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:39.255 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.237+0000 7f245ffff700 1 --2- 172.21.15.47:0/1570927895 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f24600671d0 0x7f24600b05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:39.255 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.237+0000 7f2464e10700 1 --2- 172.21.15.47:0/1570927895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24600b4420 0x7f2460149770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:39.255 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.237+0000 7f245f7fe700 1 -- 172.21.15.47:0/1570927895 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f24600671d0 msgr2=0x7f24600b05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:39.256 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.237+0000 7f245f7fe700 1 --2- 172.21.15.47:0/1570927895 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f24600671d0 0x7f24600b05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:39.256 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.237+0000 7f245f7fe700 1 -- 172.21.15.47:0/1570927895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24600b4420 msgr2=0x7f2460149770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:39.256 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.237+0000 7f245f7fe700 1 --2- 172.21.15.47:0/1570927895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24600b4420 0x7f2460149770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:39.256 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.237+0000 7f245f7fe700 1 -- 172.21.15.47:0/1570927895 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2458009470 con 0x7f2460067630 2022-06-29T15:49:39.256 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.237+0000 7f2464e10700 1 --2- 172.21.15.47:0/1570927895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24600b4420 0x7f2460149770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:49:39.257 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.237+0000 7f245ffff700 1 --2- 172.21.15.47:0/1570927895 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f24600671d0 0x7f24600b05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:49:39.257 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.237+0000 7f245f7fe700 1 --2- 172.21.15.47:0/1570927895 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2460067630 0x7f2460149280 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 rx=0x7f246005f750 tx=0x7f2454001a50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:39.257 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.237+0000 7f245d7fa700 1 -- 172.21.15.47:0/1570927895 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2454007170 con 0x7f2460067630 2022-06-29T15:49:39.257 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.237+0000 7f245d7fa700 1 -- 172.21.15.47:0/1570927895 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24540072d0 con 0x7f2460067630 2022-06-29T15:49:39.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.237+0000 7f2466873700 1 -- 172.21.15.47:0/1570927895 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24600604c0 con 0x7f2460067630 2022-06-29T15:49:39.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.237+0000 7f245d7fa700 1 -- 172.21.15.47:0/1570927895 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2454004920 con 0x7f2460067630 2022-06-29T15:49:39.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.237+0000 7f2466873700 1 -- 172.21.15.47:0/1570927895 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24600aeec0 con 0x7f2460067630 2022-06-29T15:49:39.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.241+0000 7f2466873700 1 -- 172.21.15.47:0/1570927895 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24600573d0 con 0x7f2460067630 2022-06-29T15:49:39.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.241+0000 7f245d7fa700 1 -- 172.21.15.47:0/1570927895 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2454004a80 con 0x7f2460067630 2022-06-29T15:49:39.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.241+0000 7f245d7fa700 1 --2- 172.21.15.47:0/1570927895 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2440053b00 0x7f2440055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:39.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.241+0000 7f245d7fa700 1 -- 172.21.15.47:0/1570927895 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f245406bd50 con 0x7f2460067630 2022-06-29T15:49:39.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.241+0000 7f245ffff700 1 --2- 172.21.15.47:0/1570927895 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2440053b00 0x7f2440055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:39.260 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.241+0000 7f245ffff700 1 --2- 172.21.15.47:0/1570927895 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2440053b00 0x7f2440055fc0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f245000b000 tx=0x7f2450009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:39.260 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.249+0000 7f245d7fa700 1 -- 172.21.15.47:0/1570927895 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2454042ac0 con 0x7f2460067630 2022-06-29T15:49:39.426 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.417+0000 7f2466873700 1 -- 172.21.15.47:0/1570927895 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f246006a0f0 con 0x7f2460067630 2022-06-29T15:49:39.427 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.421+0000 7f245d7fa700 1 -- 172.21.15.47:0/1570927895 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+2132 (secure 0 0 0) 0x7f24540460e0 con 0x7f2460067630 2022-06-29T15:49:39.428 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:49:39.428 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":41,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6577,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/4286932616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":4286932616},{"type":"v1","addr":"172.21.15.64:6833","nonce":4286932616}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":38},{"gid":6657,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3528102962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3528102962},{"type":"v1","addr":"172.21.15.47:6835","nonce":3528102962}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":38}],"filesystems":[{"mdsmap":{"epoch":41,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:49:27.717245+0000","modified":"2022-06-29T15:49:39.102062+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2022-06-29T15:49:39.429 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.421+0000 7f2466873700 1 -- 172.21.15.47:0/1570927895 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2440053b00 msgr2=0x7f2440055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:39.430 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.421+0000 7f2466873700 1 --2- 172.21.15.47:0/1570927895 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2440053b00 0x7f2440055fc0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f245000b000 tx=0x7f2450009040).stop 2022-06-29T15:49:39.430 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.421+0000 7f2466873700 1 -- 172.21.15.47:0/1570927895 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2460067630 msgr2=0x7f2460149280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:39.430 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.421+0000 7f2466873700 1 --2- 172.21.15.47:0/1570927895 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2460067630 0x7f2460149280 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 rx=0x7f246005f750 tx=0x7f2454001a50).stop 2022-06-29T15:49:39.430 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.421+0000 7f2466873700 1 -- 172.21.15.47:0/1570927895 shutdown_connections 2022-06-29T15:49:39.431 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.421+0000 7f2466873700 1 --2- 172.21.15.47:0/1570927895 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2440053b00 0x7f2440055fc0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:39.431 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.421+0000 7f2466873700 1 --2- 172.21.15.47:0/1570927895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24600b4420 0x7f2460149770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:39.431 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.421+0000 7f2466873700 1 --2- 172.21.15.47:0/1570927895 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2460067630 0x7f2460149280 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:39.431 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.421+0000 7f2466873700 1 --2- 172.21.15.47:0/1570927895 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f24600671d0 0x7f24600b05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:39.431 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.421+0000 7f2466873700 1 -- 172.21.15.47:0/1570927895 >> 172.21.15.47:0/1570927895 conn(0x7f246005c510 msgr2=0x7f246005c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:39.432 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.421+0000 7f2466873700 1 -- 172.21.15.47:0/1570927895 shutdown_connections 2022-06-29T15:49:39.432 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.421+0000 7f2466873700 1 -- 172.21.15.47:0/1570927895 wait complete. 2022-06-29T15:49:39.432 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 41 2022-06-29T15:49:39.441 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-06-29T15:49:39.551 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.541+0000 7f579de5d700 1 Processor -- start 2022-06-29T15:49:39.551 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.541+0000 7f579de5d700 1 -- start start 2022-06-29T15:49:39.554 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.545+0000 7f579de5d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5798060ee0 0x7f57980612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:39.554 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.545+0000 7f579de5d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f57980b4300 0x7f57981471a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:39.554 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.545+0000 7f579de5d700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5798067610 con 0x7f5798066f70 2022-06-29T15:49:39.555 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.545+0000 7f579de5d700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57980673a0 con 0x7f57980b4300 2022-06-29T15:49:39.555 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.545+0000 7f579de5d700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57981467c0 con 0x7f5798060ee0 2022-06-29T15:49:39.555 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.545+0000 7f578effd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f57980b4300 0x7f57981471a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:39.555 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.545+0000 7f578effd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f57980b4300 0x7f57981471a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51096/0 (socket says 172.21.15.47:51096) 2022-06-29T15:49:39.556 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.545+0000 7f578effd700 1 -- 172.21.15.47:0/1516256736 learned_addr learned my addr 172.21.15.47:0/1516256736 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:39.556 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.545+0000 7f57977fe700 1 --2- 172.21.15.47:0/1516256736 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5798060ee0 0x7f57980612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:39.556 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.545+0000 7f578effd700 1 -- 172.21.15.47:0/1516256736 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5798060ee0 msgr2=0x7f57980612c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:39.556 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.545+0000 7f578effd700 1 --2- 172.21.15.47:0/1516256736 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5798060ee0 0x7f57980612c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:39.557 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.545+0000 7f578effd700 1 -- 172.21.15.47:0/1516256736 >> v1:172.21.15.64:6789/0 conn(0x7f5798066f70 legacy=0x7f57981430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:39.557 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.545+0000 7f578effd700 1 -- 172.21.15.47:0/1516256736 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5798146a60 con 0x7f57980b4300 2022-06-29T15:49:39.557 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.545+0000 7f578effd700 1 --2- 172.21.15.47:0/1516256736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57980b4300 0x7f57981471a0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 rx=0x7f5780002f70 tx=0x7f5780006eb0).ready entity=mon.0 client_cookie=aee53e5a5d2f1640 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:39.557 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.545+0000 7f5796ffd700 1 -- 172.21.15.47:0/1516256736 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f578000d5c0 con 0x7f57980b4300 2022-06-29T15:49:39.558 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.545+0000 7f5796ffd700 1 -- 172.21.15.47:0/1516256736 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5780000e90 con 0x7f57980b4300 2022-06-29T15:49:39.558 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.545+0000 7f5796ffd700 1 -- 172.21.15.47:0/1516256736 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f578000d950 con 0x7f57980b4300 2022-06-29T15:49:39.562 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.545+0000 7f579de5d700 1 -- 172.21.15.47:0/1516256736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57980b4300 msgr2=0x7f57981471a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:39.563 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.545+0000 7f579de5d700 1 --2- 172.21.15.47:0/1516256736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57980b4300 0x7f57981471a0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 rx=0x7f5780002f70 tx=0x7f5780006eb0).stop 2022-06-29T15:49:39.563 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.545+0000 7f579de5d700 1 -- 172.21.15.47:0/1516256736 shutdown_connections 2022-06-29T15:49:39.563 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.545+0000 7f579de5d700 1 --2- 172.21.15.47:0/1516256736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57980b4300 0x7f57981471a0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:39.564 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.545+0000 7f579de5d700 1 --2- 172.21.15.47:0/1516256736 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5798060ee0 0x7f57980612c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:39.564 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.545+0000 7f579de5d700 1 -- 172.21.15.47:0/1516256736 >> 172.21.15.47:0/1516256736 conn(0x7f579805c510 msgr2=0x7f579805f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:39.564 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.545+0000 7f579de5d700 1 -- 172.21.15.47:0/1516256736 shutdown_connections 2022-06-29T15:49:39.565 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.545+0000 7f579de5d700 1 -- 172.21.15.47:0/1516256736 wait complete. 2022-06-29T15:49:39.565 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.549+0000 7f579de5d700 1 Processor -- start 2022-06-29T15:49:39.565 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.549+0000 7f579de5d700 1 -- start start 2022-06-29T15:49:39.566 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.549+0000 7f579de5d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5798060ee0 0x7f57980af880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:39.566 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.549+0000 7f579de5d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5798066f70 0x7f5798155ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:39.566 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.549+0000 7f579de5d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57980b4300 0x7f57981563b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:39.566 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.549+0000 7f579de5d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f579813b060 con 0x7f5798060ee0 2022-06-29T15:49:39.567 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.549+0000 7f579de5d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5798143f50 con 0x7f57980b4300 2022-06-29T15:49:39.567 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.549+0000 7f579de5d700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5798155a60 con 0x7f5798066f70 2022-06-29T15:49:39.567 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.549+0000 7f57977fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5798060ee0 0x7f57980af880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:39.567 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.549+0000 7f57977fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5798060ee0 0x7f57980af880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51102/0 (socket says 172.21.15.47:51102) 2022-06-29T15:49:39.568 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.549+0000 7f57977fe700 1 -- 172.21.15.47:0/2103709803 learned_addr learned my addr 172.21.15.47:0/2103709803 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:39.568 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.549+0000 7f578effd700 1 --2- 172.21.15.47:0/2103709803 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5798066f70 0x7f5798155ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:39.568 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.549+0000 7f5797fff700 1 --2- 172.21.15.47:0/2103709803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57980b4300 0x7f57981563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:39.569 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.549+0000 7f57977fe700 1 -- 172.21.15.47:0/2103709803 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5798066f70 msgr2=0x7f5798155ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:39.569 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.549+0000 7f57977fe700 1 --2- 172.21.15.47:0/2103709803 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5798066f70 0x7f5798155ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:39.569 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.549+0000 7f57977fe700 1 -- 172.21.15.47:0/2103709803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57980b4300 msgr2=0x7f57981563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:39.569 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.549+0000 7f57977fe700 1 --2- 172.21.15.47:0/2103709803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57980b4300 0x7f57981563b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:39.570 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.549+0000 7f57977fe700 1 -- 172.21.15.47:0/2103709803 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f578000b1e0 con 0x7f5798060ee0 2022-06-29T15:49:39.570 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.549+0000 7f5797fff700 1 --2- 172.21.15.47:0/2103709803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57980b4300 0x7f57981563b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:49:39.570 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.549+0000 7f57977fe700 1 --2- 172.21.15.47:0/2103709803 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5798060ee0 0x7f57980af880 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 rx=0x7f5798146300 tx=0x7f5788008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:39.571 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.549+0000 7f57957fa700 1 -- 172.21.15.47:0/2103709803 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5788011780 con 0x7f5798060ee0 2022-06-29T15:49:39.571 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.549+0000 7f57957fa700 1 -- 172.21.15.47:0/2103709803 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57880118e0 con 0x7f5798060ee0 2022-06-29T15:49:39.571 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.549+0000 7f579de5d700 1 -- 172.21.15.47:0/2103709803 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f579813adc0 con 0x7f5798060ee0 2022-06-29T15:49:39.571 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.549+0000 7f57957fa700 1 -- 172.21.15.47:0/2103709803 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5788011c40 con 0x7f5798060ee0 2022-06-29T15:49:39.571 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.549+0000 7f579de5d700 1 -- 172.21.15.47:0/2103709803 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f579813a3a0 con 0x7f5798060ee0 2022-06-29T15:49:39.572 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.553+0000 7f579de5d700 1 -- 172.21.15.47:0/2103709803 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57980573d0 con 0x7f5798060ee0 2022-06-29T15:49:39.572 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.553+0000 7f57957fa700 1 -- 172.21.15.47:0/2103709803 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5788012420 con 0x7f5798060ee0 2022-06-29T15:49:39.572 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.553+0000 7f57957fa700 1 --2- 172.21.15.47:0/2103709803 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f577c053a70 0x7f577c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:39.572 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.553+0000 7f57957fa700 1 -- 172.21.15.47:0/2103709803 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f578806c0b0 con 0x7f5798060ee0 2022-06-29T15:49:39.573 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.553+0000 7f578effd700 1 --2- 172.21.15.47:0/2103709803 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f577c053a70 0x7f577c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:39.573 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.553+0000 7f578effd700 1 --2- 172.21.15.47:0/2103709803 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f577c053a70 0x7f577c055f30 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7f578001a000 tx=0x7f5780015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:39.574 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.557+0000 7f57957fa700 1 -- 172.21.15.47:0/2103709803 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5788042cb0 con 0x7f5798060ee0 2022-06-29T15:49:39.731 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:39.721+0000 7f579de5d700 1 -- 172.21.15.47:0/2103709803 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f57980b5e50 con 0x7f5798060ee0 2022-06-29T15:49:40.122 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.113+0000 7f57957fa700 1 -- 172.21.15.47:0/2103709803 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v42) v1 ==== 104+0+0 (secure 0 0 0) 0x7f57880462d0 con 0x7f5798060ee0 2022-06-29T15:49:40.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.117+0000 7f579de5d700 1 -- 172.21.15.47:0/2103709803 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f577c053a70 msgr2=0x7f577c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:40.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.117+0000 7f579de5d700 1 --2- 172.21.15.47:0/2103709803 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f577c053a70 0x7f577c055f30 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7f578001a000 tx=0x7f5780015000).stop 2022-06-29T15:49:40.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.117+0000 7f579de5d700 1 -- 172.21.15.47:0/2103709803 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5798060ee0 msgr2=0x7f57980af880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:40.127 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.117+0000 7f579de5d700 1 --2- 172.21.15.47:0/2103709803 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5798060ee0 0x7f57980af880 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 rx=0x7f5798146300 tx=0x7f5788008040).stop 2022-06-29T15:49:40.127 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.117+0000 7f579de5d700 1 -- 172.21.15.47:0/2103709803 shutdown_connections 2022-06-29T15:49:40.127 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.117+0000 7f579de5d700 1 --2- 172.21.15.47:0/2103709803 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f577c053a70 0x7f577c055f30 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:40.127 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.117+0000 7f579de5d700 1 --2- 172.21.15.47:0/2103709803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57980b4300 0x7f57981563b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:40.127 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.117+0000 7f579de5d700 1 --2- 172.21.15.47:0/2103709803 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5798066f70 0x7f5798155ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:40.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.117+0000 7f579de5d700 1 --2- 172.21.15.47:0/2103709803 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5798060ee0 0x7f57980af880 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:40.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.117+0000 7f579de5d700 1 -- 172.21.15.47:0/2103709803 >> 172.21.15.47:0/2103709803 conn(0x7f579805c510 msgr2=0x7f57980b7b10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:40.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.117+0000 7f579de5d700 1 -- 172.21.15.47:0/2103709803 shutdown_connections 2022-06-29T15:49:40.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.117+0000 7f579de5d700 1 -- 172.21.15.47:0/2103709803 wait complete. 2022-06-29T15:49:40.140 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-06-29T15:49:40.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.249+0000 7ff61ffff700 1 Processor -- start 2022-06-29T15:49:40.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.249+0000 7ff61ffff700 1 -- start start 2022-06-29T15:49:40.261 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.253+0000 7ff61ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff6200b5d90 0x7ff620143a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:40.262 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.253+0000 7ff61ffff700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff62005cc30 con 0x7ff62005c850 2022-06-29T15:49:40.262 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.253+0000 7ff61ffff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6201430a0 con 0x7ff6200676a0 2022-06-29T15:49:40.262 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.253+0000 7ff61ffff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff620143300 con 0x7ff6200b5d90 2022-06-29T15:49:40.262 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.253+0000 7ff61effd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff62005c850 0x7ff6200b4860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36666/0 (socket says 172.21.15.47:36666) 2022-06-29T15:49:40.263 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.253+0000 7ff61effd700 1 -- 172.21.15.47:0/3214279946 learned_addr learned my addr 172.21.15.47:0/3214279946 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:49:40.263 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.253+0000 7ff61e7fc700 1 --2- 172.21.15.47:0/3214279946 >> v2:172.21.15.64:3300/0 conn(0x7ff6200b5d90 0x7ff620143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:40.263 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.253+0000 7ff61dffb700 1 -- 172.21.15.47:0/3214279946 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1891597211 0 0) 0x7ff62005cc30 con 0x7ff62005c850 2022-06-29T15:49:40.264 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.253+0000 7ff61dffb700 1 -- 172.21.15.47:0/3214279946 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff600003260 con 0x7ff62005c850 2022-06-29T15:49:40.264 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.253+0000 7ff61dffb700 1 -- 172.21.15.47:0/3214279946 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 127873701 0 0) 0x7ff6201430a0 con 0x7ff6200676a0 2022-06-29T15:49:40.264 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.253+0000 7ff61dffb700 1 -- 172.21.15.47:0/3214279946 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff62005cc30 con 0x7ff6200676a0 2022-06-29T15:49:40.264 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.253+0000 7ff61e7fc700 1 -- 172.21.15.47:0/3214279946 >> v1:172.21.15.64:6789/0 conn(0x7ff6200676a0 legacy=0x7ff620140130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:40.265 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.253+0000 7ff61e7fc700 1 -- 172.21.15.47:0/3214279946 >> v1:172.21.15.47:6789/0 conn(0x7ff62005c850 legacy=0x7ff6200b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:40.265 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.253+0000 7ff61e7fc700 1 -- 172.21.15.47:0/3214279946 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6201435a0 con 0x7ff6200b5d90 2022-06-29T15:49:40.265 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.253+0000 7ff61e7fc700 1 --2- 172.21.15.47:0/3214279946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6200b5d90 0x7ff620143a40 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7ff608002f70 tx=0x7ff608006df0).ready entity=mon.1 client_cookie=e53299efa569caf7 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:40.269 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.253+0000 7ff61dffb700 1 -- 172.21.15.47:0/3214279946 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff60800d540 con 0x7ff6200b5d90 2022-06-29T15:49:40.269 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.253+0000 7ff61dffb700 1 -- 172.21.15.47:0/3214279946 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff608000e90 con 0x7ff6200b5d90 2022-06-29T15:49:40.269 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.253+0000 7ff61dffb700 1 -- 172.21.15.47:0/3214279946 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff60800d890 con 0x7ff6200b5d90 2022-06-29T15:49:40.269 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.257+0000 7ff61ffff700 1 -- 172.21.15.47:0/3214279946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6200b5d90 msgr2=0x7ff620143a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:40.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.257+0000 7ff61ffff700 1 --2- 172.21.15.47:0/3214279946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6200b5d90 0x7ff620143a40 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7ff608002f70 tx=0x7ff608006df0).stop 2022-06-29T15:49:40.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.257+0000 7ff61ffff700 1 -- 172.21.15.47:0/3214279946 shutdown_connections 2022-06-29T15:49:40.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.257+0000 7ff61ffff700 1 --2- 172.21.15.47:0/3214279946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6200b5d90 0x7ff620143a40 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:40.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.257+0000 7ff61ffff700 1 -- 172.21.15.47:0/3214279946 >> 172.21.15.47:0/3214279946 conn(0x7ff620060e00 msgr2=0x7ff6200611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:40.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.257+0000 7ff61ffff700 1 -- 172.21.15.47:0/3214279946 shutdown_connections 2022-06-29T15:49:40.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.257+0000 7ff61ffff700 1 -- 172.21.15.47:0/3214279946 wait complete. 2022-06-29T15:49:40.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.257+0000 7ff61ffff700 1 Processor -- start 2022-06-29T15:49:40.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.257+0000 7ff61ffff700 1 -- start start 2022-06-29T15:49:40.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.261+0000 7ff61ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff62005c850 0x7ff6200af3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:40.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.261+0000 7ff61ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6200676a0 0x7ff6200b98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:40.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.261+0000 7ff61ffff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff6200b5d90 0x7ff6200b9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:40.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.261+0000 7ff61ffff700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6200b82f0 con 0x7ff6200676a0 2022-06-29T15:49:40.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.261+0000 7ff61ffff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6200b2790 con 0x7ff62005c850 2022-06-29T15:49:40.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.261+0000 7ff61ffff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6200b2a30 con 0x7ff6200b5d90 2022-06-29T15:49:40.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.261+0000 7ff61e7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6200676a0 0x7ff6200b98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:40.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.261+0000 7ff61e7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6200676a0 0x7ff6200b98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51120/0 (socket says 172.21.15.47:51120) 2022-06-29T15:49:40.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.261+0000 7ff61e7fc700 1 -- 172.21.15.47:0/284925665 learned_addr learned my addr 172.21.15.47:0/284925665 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:40.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.261+0000 7ff61effd700 1 --2- 172.21.15.47:0/284925665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff62005c850 0x7ff6200af3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:40.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.261+0000 7ff61f7fe700 1 --2- 172.21.15.47:0/284925665 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff6200b5d90 0x7ff6200b9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:40.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.261+0000 7ff61e7fc700 1 -- 172.21.15.47:0/284925665 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff6200b5d90 msgr2=0x7ff6200b9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:40.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.261+0000 7ff61e7fc700 1 --2- 172.21.15.47:0/284925665 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff6200b5d90 0x7ff6200b9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:40.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.261+0000 7ff61e7fc700 1 -- 172.21.15.47:0/284925665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff62005c850 msgr2=0x7ff6200af3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:40.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.261+0000 7ff61e7fc700 1 --2- 172.21.15.47:0/284925665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff62005c850 0x7ff6200af3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:40.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.261+0000 7ff61e7fc700 1 -- 172.21.15.47:0/284925665 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff60800b180 con 0x7ff6200676a0 2022-06-29T15:49:40.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.261+0000 7ff61effd700 1 --2- 172.21.15.47:0/284925665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff62005c850 0x7ff6200af3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:49:40.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.261+0000 7ff61e7fc700 1 --2- 172.21.15.47:0/284925665 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6200676a0 0x7ff6200b98c0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 rx=0x7ff608002f70 tx=0x7ff608014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:40.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.261+0000 7ff617fff700 1 -- 172.21.15.47:0/284925665 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff60800d1c0 con 0x7ff6200676a0 2022-06-29T15:49:40.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.261+0000 7ff617fff700 1 -- 172.21.15.47:0/284925665 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff608013ca0 con 0x7ff6200676a0 2022-06-29T15:49:40.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.261+0000 7ff61ffff700 1 -- 172.21.15.47:0/284925665 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6200b2cd0 con 0x7ff6200676a0 2022-06-29T15:49:40.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.261+0000 7ff617fff700 1 -- 172.21.15.47:0/284925665 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff60801e400 con 0x7ff6200676a0 2022-06-29T15:49:40.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.261+0000 7ff61ffff700 1 -- 172.21.15.47:0/284925665 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6200b2280 con 0x7ff6200676a0 2022-06-29T15:49:40.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.261+0000 7ff61ffff700 1 -- 172.21.15.47:0/284925665 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6200573d0 con 0x7ff6200676a0 2022-06-29T15:49:40.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.261+0000 7ff617fff700 1 -- 172.21.15.47:0/284925665 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff608025030 con 0x7ff6200676a0 2022-06-29T15:49:40.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.261+0000 7ff617fff700 1 --2- 172.21.15.47:0/284925665 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff604053a10 0x7ff604055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:40.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.265+0000 7ff617fff700 1 -- 172.21.15.47:0/284925665 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6506+0+0 (secure 0 0 0) 0x7ff60806cce0 con 0x7ff6200676a0 2022-06-29T15:49:40.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.265+0000 7ff61effd700 1 --2- 172.21.15.47:0/284925665 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff604053a10 0x7ff604055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:40.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.265+0000 7ff61effd700 1 --2- 172.21.15.47:0/284925665 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff604053a10 0x7ff604055ed0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7ff61000b000 tx=0x7ff610009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:40.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.265+0000 7ff617fff700 1 -- 172.21.15.47:0/284925665 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff608043a50 con 0x7ff6200676a0 2022-06-29T15:49:40.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:40.397+0000 7ff61ffff700 1 -- 172.21.15.47:0/284925665 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff620061b00 con 0x7ff6200676a0 2022-06-29T15:49:41.132 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.125+0000 7ff617fff700 1 -- 172.21.15.47:0/284925665 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v92) v1 ==== 185+0+0 (secure 0 0 0) 0x7ff608023070 con 0x7ff6200676a0 2022-06-29T15:49:41.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.125+0000 7ff61ffff700 1 -- 172.21.15.47:0/284925665 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff604053a10 msgr2=0x7ff604055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:41.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.125+0000 7ff61ffff700 1 --2- 172.21.15.47:0/284925665 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff604053a10 0x7ff604055ed0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7ff61000b000 tx=0x7ff610009040).stop 2022-06-29T15:49:41.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.125+0000 7ff61ffff700 1 -- 172.21.15.47:0/284925665 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6200676a0 msgr2=0x7ff6200b98c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:41.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.125+0000 7ff61ffff700 1 --2- 172.21.15.47:0/284925665 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6200676a0 0x7ff6200b98c0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 rx=0x7ff608002f70 tx=0x7ff608014000).stop 2022-06-29T15:49:41.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.129+0000 7ff61ffff700 1 -- 172.21.15.47:0/284925665 shutdown_connections 2022-06-29T15:49:41.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.129+0000 7ff61ffff700 1 --2- 172.21.15.47:0/284925665 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff6200b5d90 0x7ff6200b9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:41.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.129+0000 7ff61ffff700 1 --2- 172.21.15.47:0/284925665 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6200676a0 0x7ff6200b98c0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:41.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.129+0000 7ff61ffff700 1 --2- 172.21.15.47:0/284925665 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff604053a10 0x7ff604055ed0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:41.138 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.129+0000 7ff61ffff700 1 --2- 172.21.15.47:0/284925665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff62005c850 0x7ff6200af3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:41.138 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.129+0000 7ff61ffff700 1 -- 172.21.15.47:0/284925665 >> 172.21.15.47:0/284925665 conn(0x7ff620060e00 msgr2=0x7ff620144e80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:41.138 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.129+0000 7ff61ffff700 1 -- 172.21.15.47:0/284925665 shutdown_connections 2022-06-29T15:49:41.138 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.129+0000 7ff61ffff700 1 -- 172.21.15.47:0/284925665 wait complete. 2022-06-29T15:49:41.139 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' removed 2022-06-29T15:49:41.151 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-06-29T15:49:41.266 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.256+0000 7f729ca97700 1 Processor -- start 2022-06-29T15:49:41.266 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.256+0000 7f729ca97700 1 -- start start 2022-06-29T15:49:41.267 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.256+0000 7f729ca97700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7298060ee0 0x7f72980612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:41.268 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.256+0000 7f729ca97700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f72980b4300 0x7f72981471a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:41.268 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.260+0000 7f729ca97700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7298066f70 0x7f72981430e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:41.268 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.260+0000 7f729ca97700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7298067610 con 0x7f7298066f70 2022-06-29T15:49:41.269 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.260+0000 7f729ca97700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72980673a0 con 0x7f7298060ee0 2022-06-29T15:49:41.269 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.260+0000 7f729ca97700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72981467c0 con 0x7f72980b4300 2022-06-29T15:49:41.269 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.260+0000 7f7296d9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7298066f70 0x7f72981430e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:41.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.260+0000 7f7296d9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7298066f70 0x7f72981430e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51130/0 (socket says 172.21.15.47:51130) 2022-06-29T15:49:41.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.260+0000 7f7296d9d700 1 -- 172.21.15.47:0/2562711362 learned_addr learned my addr 172.21.15.47:0/2562711362 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:41.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.260+0000 7f7295d9b700 1 --2- 172.21.15.47:0/2562711362 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f72980b4300 0x7f72981471a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:41.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.260+0000 7f729659c700 1 --2- 172.21.15.47:0/2562711362 >> v2:172.21.15.64:3300/0 conn(0x7f7298060ee0 0x7f72980612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:41.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.260+0000 7f7295d9b700 1 -- 172.21.15.47:0/2562711362 >> v2:172.21.15.64:3300/0 conn(0x7f7298060ee0 msgr2=0x7f72980612c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:41.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.260+0000 7f7295d9b700 1 --2- 172.21.15.47:0/2562711362 >> v2:172.21.15.64:3300/0 conn(0x7f7298060ee0 0x7f72980612c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:41.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.260+0000 7f7295d9b700 1 -- 172.21.15.47:0/2562711362 >> v2:172.21.15.47:3300/0 conn(0x7f7298066f70 msgr2=0x7f72981430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:41.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.260+0000 7f7295d9b700 1 --2- 172.21.15.47:0/2562711362 >> v2:172.21.15.47:3300/0 conn(0x7f7298066f70 0x7f72981430e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:41.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.260+0000 7f7295d9b700 1 -- 172.21.15.47:0/2562711362 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7298146a60 con 0x7f72980b4300 2022-06-29T15:49:41.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.260+0000 7f7296d9d700 1 --2- 172.21.15.47:0/2562711362 >> v2:172.21.15.47:3300/0 conn(0x7f7298066f70 0x7f72981430e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:49:41.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.260+0000 7f7295d9b700 1 --2- 172.21.15.47:0/2562711362 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f72980b4300 0x7f72981471a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f7288002f70 tx=0x7f7288006df0).ready entity=mon.2 client_cookie=435d3d672f60c078 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:41.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.260+0000 7f729559a700 1 -- 172.21.15.47:0/2562711362 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f728800d4a0 con 0x7f72980b4300 2022-06-29T15:49:41.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.260+0000 7f729559a700 1 -- 172.21.15.47:0/2562711362 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7288000e90 con 0x7f72980b4300 2022-06-29T15:49:41.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.260+0000 7f729559a700 1 -- 172.21.15.47:0/2562711362 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f728800d7f0 con 0x7f72980b4300 2022-06-29T15:49:41.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.260+0000 7f729ca97700 1 -- 172.21.15.47:0/2562711362 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f72980b4300 msgr2=0x7f72981471a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:41.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.260+0000 7f729ca97700 1 --2- 172.21.15.47:0/2562711362 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f72980b4300 0x7f72981471a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f7288002f70 tx=0x7f7288006df0).stop 2022-06-29T15:49:41.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.260+0000 7f729ca97700 1 -- 172.21.15.47:0/2562711362 shutdown_connections 2022-06-29T15:49:41.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.260+0000 7f729ca97700 1 --2- 172.21.15.47:0/2562711362 >> v2:172.21.15.47:3300/0 conn(0x7f7298066f70 0x7f72981430e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:41.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.260+0000 7f729ca97700 1 --2- 172.21.15.47:0/2562711362 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f72980b4300 0x7f72981471a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:41.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.260+0000 7f729ca97700 1 --2- 172.21.15.47:0/2562711362 >> v2:172.21.15.64:3300/0 conn(0x7f7298060ee0 0x7f72980612c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:41.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.260+0000 7f729ca97700 1 -- 172.21.15.47:0/2562711362 >> 172.21.15.47:0/2562711362 conn(0x7f729805c510 msgr2=0x7f729805f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:41.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.260+0000 7f729ca97700 1 -- 172.21.15.47:0/2562711362 shutdown_connections 2022-06-29T15:49:41.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.260+0000 7f729ca97700 1 -- 172.21.15.47:0/2562711362 wait complete. 2022-06-29T15:49:41.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.264+0000 7f729ca97700 1 Processor -- start 2022-06-29T15:49:41.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.264+0000 7f729ca97700 1 -- start start 2022-06-29T15:49:41.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.264+0000 7f729ca97700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7298060ee0 0x7f729813bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:41.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.264+0000 7f729ca97700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7298066f70 0x7f7298148cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:41.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.264+0000 7f729ca97700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72980b4300 0x7f72981538d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:41.282 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.264+0000 7f729ca97700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7298153440 con 0x7f7298066f70 2022-06-29T15:49:41.282 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.264+0000 7f729ca97700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72981531a0 con 0x7f72980b4300 2022-06-29T15:49:41.282 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.264+0000 7f729ca97700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7298145760 con 0x7f7298060ee0 2022-06-29T15:49:41.282 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.264+0000 7f7295d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7298066f70 0x7f7298148cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:41.283 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.264+0000 7f7295d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7298066f70 0x7f7298148cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51132/0 (socket says 172.21.15.47:51132) 2022-06-29T15:49:41.283 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.264+0000 7f7295d9b700 1 -- 172.21.15.47:0/188216584 learned_addr learned my addr 172.21.15.47:0/188216584 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:41.283 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.264+0000 7f7296d9d700 1 --2- 172.21.15.47:0/188216584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72980b4300 0x7f72981538d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:41.283 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.264+0000 7f729659c700 1 --2- 172.21.15.47:0/188216584 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7298060ee0 0x7f729813bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:41.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.264+0000 7f7295d9b700 1 -- 172.21.15.47:0/188216584 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7298060ee0 msgr2=0x7f729813bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:41.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.264+0000 7f7295d9b700 1 --2- 172.21.15.47:0/188216584 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7298060ee0 0x7f729813bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:41.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.264+0000 7f7295d9b700 1 -- 172.21.15.47:0/188216584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72980b4300 msgr2=0x7f72981538d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:41.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.264+0000 7f7295d9b700 1 --2- 172.21.15.47:0/188216584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72980b4300 0x7f72981538d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:41.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.264+0000 7f7295d9b700 1 -- 172.21.15.47:0/188216584 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7290009530 con 0x7f7298066f70 2022-06-29T15:49:41.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.264+0000 7f7296d9d700 1 --2- 172.21.15.47:0/188216584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72980b4300 0x7f72981538d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:49:41.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.264+0000 7f7295d9b700 1 --2- 172.21.15.47:0/188216584 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7298066f70 0x7f7298148cd0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 rx=0x7f7288002f70 tx=0x7f7288014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:41.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.264+0000 7f72877fe700 1 -- 172.21.15.47:0/188216584 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7288013ca0 con 0x7f7298066f70 2022-06-29T15:49:41.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.264+0000 7f72877fe700 1 -- 172.21.15.47:0/188216584 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7288013e00 con 0x7f7298066f70 2022-06-29T15:49:41.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.264+0000 7f729ca97700 1 -- 172.21.15.47:0/188216584 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f728800b180 con 0x7f7298066f70 2022-06-29T15:49:41.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.264+0000 7f72877fe700 1 -- 172.21.15.47:0/188216584 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f728801e5f0 con 0x7f7298066f70 2022-06-29T15:49:41.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.264+0000 7f729ca97700 1 -- 172.21.15.47:0/188216584 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7298146450 con 0x7f7298066f70 2022-06-29T15:49:41.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.268+0000 7f729ca97700 1 -- 172.21.15.47:0/188216584 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f729813f260 con 0x7f7298066f70 2022-06-29T15:49:41.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.268+0000 7f72877fe700 1 -- 172.21.15.47:0/188216584 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f728801c070 con 0x7f7298066f70 2022-06-29T15:49:41.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.268+0000 7f72877fe700 1 --2- 172.21.15.47:0/188216584 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7274053a70 0x7f7274055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:41.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.268+0000 7f72877fe700 1 -- 172.21.15.47:0/188216584 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6068+0+0 (secure 0 0 0) 0x7f728806c900 con 0x7f7298066f70 2022-06-29T15:49:41.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.268+0000 7f729659c700 1 --2- 172.21.15.47:0/188216584 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7274053a70 0x7f7274055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:41.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.268+0000 7f729659c700 1 --2- 172.21.15.47:0/188216584 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7274053a70 0x7f7274055f30 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f728c00b000 tx=0x7f728c006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:41.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.276+0000 7f72877fe700 1 -- 172.21.15.47:0/188216584 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7288043780 con 0x7f7298066f70 2022-06-29T15:49:41.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:41.408+0000 7f729ca97700 1 -- 172.21.15.47:0/188216584 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f72980573d0 con 0x7f7298066f70 2022-06-29T15:49:42.144 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.136+0000 7f72877fe700 1 -- 172.21.15.47:0/188216584 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v93) v1 ==== 173+0+0 (secure 0 0 0) 0x7f7288023070 con 0x7f7298066f70 2022-06-29T15:49:42.148 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.140+0000 7f729ca97700 1 -- 172.21.15.47:0/188216584 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7274053a70 msgr2=0x7f7274055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:42.149 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.140+0000 7f729ca97700 1 --2- 172.21.15.47:0/188216584 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7274053a70 0x7f7274055f30 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f728c00b000 tx=0x7f728c006df0).stop 2022-06-29T15:49:42.149 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.140+0000 7f729ca97700 1 -- 172.21.15.47:0/188216584 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7298066f70 msgr2=0x7f7298148cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:42.149 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.140+0000 7f729ca97700 1 --2- 172.21.15.47:0/188216584 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7298066f70 0x7f7298148cd0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 rx=0x7f7288002f70 tx=0x7f7288014000).stop 2022-06-29T15:49:42.150 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.140+0000 7f729ca97700 1 -- 172.21.15.47:0/188216584 shutdown_connections 2022-06-29T15:49:42.150 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.140+0000 7f729ca97700 1 --2- 172.21.15.47:0/188216584 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7274053a70 0x7f7274055f30 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:42.150 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.140+0000 7f729ca97700 1 --2- 172.21.15.47:0/188216584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72980b4300 0x7f72981538d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:42.151 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.140+0000 7f729ca97700 1 --2- 172.21.15.47:0/188216584 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7298066f70 0x7f7298148cd0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:42.151 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.140+0000 7f729ca97700 1 --2- 172.21.15.47:0/188216584 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7298060ee0 0x7f729813bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:42.151 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.140+0000 7f729ca97700 1 -- 172.21.15.47:0/188216584 >> 172.21.15.47:0/188216584 conn(0x7f729805c510 msgr2=0x7f72981402b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:42.153 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.140+0000 7f729ca97700 1 -- 172.21.15.47:0/188216584 shutdown_connections 2022-06-29T15:49:42.153 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.140+0000 7f729ca97700 1 -- 172.21.15.47:0/188216584 wait complete. 2022-06-29T15:49:42.153 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' removed 2022-06-29T15:49:42.165 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-06-29T15:49:42.252 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:49:42.256 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:49:42.259 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:49:42.262 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:49:42.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.264+0000 7f1a9235c700 1 Processor -- start 2022-06-29T15:49:42.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.264+0000 7f1a9235c700 1 -- start start 2022-06-29T15:49:42.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.264+0000 7f1a9235c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1a8c0b6d20 0x7f1a8c0b1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:42.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.264+0000 7f1a9235c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1a8c1467a0 0x7f1a8c0b1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:42.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.268+0000 7f1a9235c700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a8c0b1050 con 0x7f1a8c140500 2022-06-29T15:49:42.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.268+0000 7f1a9235c700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a8c0b23c0 con 0x7f1a8c0b6d20 2022-06-29T15:49:42.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.268+0000 7f1a9235c700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8c05fab0 con 0x7f1a8c1467a0 2022-06-29T15:49:42.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.268+0000 7f1a9135a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1a8c0b6d20 0x7f1a8c0b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:42.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.268+0000 7f1a91b5b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1a8c140500 0x7f1a8c0b2e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36698/0 (socket says 172.21.15.47:36698) 2022-06-29T15:49:42.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.268+0000 7f1a91b5b700 1 -- 172.21.15.47:0/3288593635 learned_addr learned my addr 172.21.15.47:0/3288593635 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:49:42.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.268+0000 7f1a90b59700 1 --2- 172.21.15.47:0/3288593635 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1a8c1467a0 0x7f1a8c0b1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:42.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.268+0000 7f1a9135a700 1 -- 172.21.15.47:0/3288593635 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1a8c1467a0 msgr2=0x7f1a8c0b1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:42.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.268+0000 7f1a9135a700 1 --2- 172.21.15.47:0/3288593635 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1a8c1467a0 0x7f1a8c0b1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:42.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.268+0000 7f1a9135a700 1 -- 172.21.15.47:0/3288593635 >> v1:172.21.15.47:6789/0 conn(0x7f1a8c140500 legacy=0x7f1a8c0b2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:42.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.268+0000 7f1a9135a700 1 -- 172.21.15.47:0/3288593635 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a8c0b7100 con 0x7f1a8c0b6d20 2022-06-29T15:49:42.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.268+0000 7f1a9135a700 1 --2- 172.21.15.47:0/3288593635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a8c0b6d20 0x7f1a8c0b1480 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 rx=0x7f1a8c0b2700 tx=0x7f1a88006df0).ready entity=mon.0 client_cookie=7d92248794f1f05d server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:42.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.268+0000 7f1a7bfff700 1 -- 172.21.15.47:0/3288593635 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a8800d570 con 0x7f1a8c0b6d20 2022-06-29T15:49:42.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.268+0000 7f1a7bfff700 1 -- 172.21.15.47:0/3288593635 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a88000e90 con 0x7f1a8c0b6d20 2022-06-29T15:49:42.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.268+0000 7f1a7bfff700 1 -- 172.21.15.47:0/3288593635 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a8800d8c0 con 0x7f1a8c0b6d20 2022-06-29T15:49:42.283 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.268+0000 7f1a9235c700 1 -- 172.21.15.47:0/3288593635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a8c0b6d20 msgr2=0x7f1a8c0b1480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:42.283 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.268+0000 7f1a9235c700 1 --2- 172.21.15.47:0/3288593635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a8c0b6d20 0x7f1a8c0b1480 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 rx=0x7f1a8c0b2700 tx=0x7f1a88006df0).stop 2022-06-29T15:49:42.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.268+0000 7f1a9235c700 1 -- 172.21.15.47:0/3288593635 shutdown_connections 2022-06-29T15:49:42.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.268+0000 7f1a9235c700 1 --2- 172.21.15.47:0/3288593635 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1a8c1467a0 0x7f1a8c0b1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:42.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.268+0000 7f1a9235c700 1 --2- 172.21.15.47:0/3288593635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a8c0b6d20 0x7f1a8c0b1480 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:42.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.268+0000 7f1a9235c700 1 -- 172.21.15.47:0/3288593635 >> 172.21.15.47:0/3288593635 conn(0x7f1a8c063670 msgr2=0x7f1a8c063a50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:42.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.268+0000 7f1a9235c700 1 -- 172.21.15.47:0/3288593635 shutdown_connections 2022-06-29T15:49:42.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.268+0000 7f1a9235c700 1 -- 172.21.15.47:0/3288593635 wait complete. 2022-06-29T15:49:42.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.268+0000 7f1a9235c700 1 Processor -- start 2022-06-29T15:49:42.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.268+0000 7f1a9235c700 1 -- start start 2022-06-29T15:49:42.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.268+0000 7f1a9235c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a8c0b6d20 0x7f1a8c0b5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:42.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.268+0000 7f1a9235c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1a8c140500 0x7f1a8c152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:42.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.268+0000 7f1a9235c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a8c1467a0 0x7f1a8c152a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:42.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.268+0000 7f1a9235c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8c152ef0 con 0x7f1a8c0b6d20 2022-06-29T15:49:42.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.268+0000 7f1a9235c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8c150190 con 0x7f1a8c1467a0 2022-06-29T15:49:42.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.268+0000 7f1a9235c700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8c156800 con 0x7f1a8c140500 2022-06-29T15:49:42.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.268+0000 7f1a9135a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a8c0b6d20 0x7f1a8c0b5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:42.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.268+0000 7f1a9135a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a8c0b6d20 0x7f1a8c0b5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51146/0 (socket says 172.21.15.47:51146) 2022-06-29T15:49:42.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.268+0000 7f1a9135a700 1 -- 172.21.15.47:0/1211687561 learned_addr learned my addr 172.21.15.47:0/1211687561 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:42.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.272+0000 7f1a90b59700 1 --2- 172.21.15.47:0/1211687561 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1a8c140500 0x7f1a8c152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:42.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.272+0000 7f1a91b5b700 1 --2- 172.21.15.47:0/1211687561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a8c1467a0 0x7f1a8c152a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:42.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.272+0000 7f1a9135a700 1 -- 172.21.15.47:0/1211687561 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1a8c140500 msgr2=0x7f1a8c152510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:42.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.272+0000 7f1a9135a700 1 --2- 172.21.15.47:0/1211687561 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1a8c140500 0x7f1a8c152510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:42.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.272+0000 7f1a9135a700 1 -- 172.21.15.47:0/1211687561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a8c1467a0 msgr2=0x7f1a8c152a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:42.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.272+0000 7f1a9135a700 1 --2- 172.21.15.47:0/1211687561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a8c1467a0 0x7f1a8c152a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:42.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.272+0000 7f1a9135a700 1 -- 172.21.15.47:0/1211687561 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a8800b180 con 0x7f1a8c0b6d20 2022-06-29T15:49:42.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.272+0000 7f1a90b59700 1 --2- 172.21.15.47:0/1211687561 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1a8c140500 0x7f1a8c152510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:49:42.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.272+0000 7f1a9135a700 1 --2- 172.21.15.47:0/1211687561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a8c0b6d20 0x7f1a8c0b5730 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 rx=0x7f1a8c0b2700 tx=0x7f1a88014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:42.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.272+0000 7f1a7a7fc700 1 -- 172.21.15.47:0/1211687561 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a88016070 con 0x7f1a8c0b6d20 2022-06-29T15:49:42.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.272+0000 7f1a7a7fc700 1 -- 172.21.15.47:0/1211687561 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a88002f70 con 0x7f1a8c0b6d20 2022-06-29T15:49:42.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.272+0000 7f1a9235c700 1 -- 172.21.15.47:0/1211687561 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a8c0adc20 con 0x7f1a8c0b6d20 2022-06-29T15:49:42.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.272+0000 7f1a7a7fc700 1 -- 172.21.15.47:0/1211687561 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a8800d8f0 con 0x7f1a8c0b6d20 2022-06-29T15:49:42.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.272+0000 7f1a9235c700 1 -- 172.21.15.47:0/1211687561 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a8c145640 con 0x7f1a8c0b6d20 2022-06-29T15:49:42.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.272+0000 7f1a9235c700 1 -- 172.21.15.47:0/1211687561 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a8c0ada60 con 0x7f1a8c0b6d20 2022-06-29T15:49:42.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.272+0000 7f1a7a7fc700 1 -- 172.21.15.47:0/1211687561 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1a88025030 con 0x7f1a8c0b6d20 2022-06-29T15:49:42.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.272+0000 7f1a7a7fc700 1 --2- 172.21.15.47:0/1211687561 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1a6c0539e0 0x7f1a6c055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:42.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.272+0000 7f1a7a7fc700 1 -- 172.21.15.47:0/1211687561 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f1a8806cd10 con 0x7f1a8c0b6d20 2022-06-29T15:49:42.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.272+0000 7f1a90b59700 1 --2- 172.21.15.47:0/1211687561 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1a6c0539e0 0x7f1a6c055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:42.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.276+0000 7f1a90b59700 1 --2- 172.21.15.47:0/1211687561 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1a6c0539e0 0x7f1a6c055ea0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f1a7c00b000 tx=0x7f1a7c008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:42.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.280+0000 7f1a7a7fc700 1 -- 172.21.15.47:0/1211687561 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1a88043db0 con 0x7f1a8c0b6d20 2022-06-29T15:49:42.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.408+0000 7f1a9235c700 1 -- 172.21.15.47:0/1211687561 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f1a8c0573d0 con 0x7f1a8c0b6d20 2022-06-29T15:49:42.425 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.416+0000 7f1a7a7fc700 1 -- 172.21.15.47:0/1211687561 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1a88023020 con 0x7f1a8c0b6d20 2022-06-29T15:49:42.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.420+0000 7f1a9235c700 1 -- 172.21.15.47:0/1211687561 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1a6c0539e0 msgr2=0x7f1a6c055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:42.429 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.420+0000 7f1a9235c700 1 --2- 172.21.15.47:0/1211687561 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1a6c0539e0 0x7f1a6c055ea0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f1a7c00b000 tx=0x7f1a7c008040).stop 2022-06-29T15:49:42.429 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.420+0000 7f1a9235c700 1 -- 172.21.15.47:0/1211687561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a8c0b6d20 msgr2=0x7f1a8c0b5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:42.429 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.420+0000 7f1a9235c700 1 --2- 172.21.15.47:0/1211687561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a8c0b6d20 0x7f1a8c0b5730 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 rx=0x7f1a8c0b2700 tx=0x7f1a88014000).stop 2022-06-29T15:49:42.430 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.420+0000 7f1a9235c700 1 -- 172.21.15.47:0/1211687561 shutdown_connections 2022-06-29T15:49:42.430 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.420+0000 7f1a9235c700 1 --2- 172.21.15.47:0/1211687561 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1a6c0539e0 0x7f1a6c055ea0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:42.430 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.420+0000 7f1a9235c700 1 --2- 172.21.15.47:0/1211687561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a8c1467a0 0x7f1a8c152a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:42.431 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.420+0000 7f1a9235c700 1 --2- 172.21.15.47:0/1211687561 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1a8c140500 0x7f1a8c152510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:42.431 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.420+0000 7f1a9235c700 1 --2- 172.21.15.47:0/1211687561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a8c0b6d20 0x7f1a8c0b5730 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:42.431 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.420+0000 7f1a9235c700 1 -- 172.21.15.47:0/1211687561 >> 172.21.15.47:0/1211687561 conn(0x7f1a8c063670 msgr2=0x7f1a8c066a80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:42.431 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.420+0000 7f1a9235c700 1 -- 172.21.15.47:0/1211687561 shutdown_connections 2022-06-29T15:49:42.432 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.420+0000 7f1a9235c700 1 -- 172.21.15.47:0/1211687561 wait complete. 2022-06-29T15:49:42.441 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode' 2022-06-29T15:49:42.545 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.536+0000 7f9170935700 1 Processor -- start 2022-06-29T15:49:42.546 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.536+0000 7f9170935700 1 -- start start 2022-06-29T15:49:42.554 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.536+0000 7f9170935700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f916c0b5ac0 0x7f916c1401c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:42.554 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.536+0000 7f9170935700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f916c0b02d0 0x7f916c1406b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:42.554 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.536+0000 7f9170935700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f916c0afdc0 0x7f916c0ae0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:42.555 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.536+0000 7f9170935700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f916c0a9090 con 0x7f916c0b02d0 2022-06-29T15:49:42.555 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.536+0000 7f9170935700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f916c143d20 con 0x7f916c0b5ac0 2022-06-29T15:49:42.555 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.536+0000 7f9170935700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f916c140ba0 con 0x7f916c0afdc0 2022-06-29T15:49:42.555 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.536+0000 7f916a59c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f916c0b02d0 0x7f916c1406b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:42.556 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.536+0000 7f916a59c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f916c0b02d0 0x7f916c1406b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51154/0 (socket says 172.21.15.47:51154) 2022-06-29T15:49:42.556 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.536+0000 7f916a59c700 1 -- 172.21.15.47:0/4194858412 learned_addr learned my addr 172.21.15.47:0/4194858412 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:42.556 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.536+0000 7f916b59e700 1 --2- 172.21.15.47:0/4194858412 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f916c0afdc0 0x7f916c0ae0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:42.556 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.536+0000 7f916ad9d700 1 --2- 172.21.15.47:0/4194858412 >> v2:172.21.15.64:3300/0 conn(0x7f916c0b5ac0 0x7f916c1401c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:42.556 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.536+0000 7f916b59e700 1 -- 172.21.15.47:0/4194858412 >> v2:172.21.15.64:3300/0 conn(0x7f916c0b5ac0 msgr2=0x7f916c1401c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:42.557 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.536+0000 7f916b59e700 1 --2- 172.21.15.47:0/4194858412 >> v2:172.21.15.64:3300/0 conn(0x7f916c0b5ac0 0x7f916c1401c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:42.557 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.536+0000 7f916b59e700 1 -- 172.21.15.47:0/4194858412 >> v2:172.21.15.47:3300/0 conn(0x7f916c0b02d0 msgr2=0x7f916c1406b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:42.557 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.536+0000 7f916b59e700 1 --2- 172.21.15.47:0/4194858412 >> v2:172.21.15.47:3300/0 conn(0x7f916c0b02d0 0x7f916c1406b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:42.557 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.536+0000 7f916b59e700 1 -- 172.21.15.47:0/4194858412 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f916c0a8df0 con 0x7f916c0afdc0 2022-06-29T15:49:42.558 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.536+0000 7f916a59c700 1 --2- 172.21.15.47:0/4194858412 >> v2:172.21.15.47:3300/0 conn(0x7f916c0b02d0 0x7f916c1406b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:49:42.558 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.540+0000 7f916b59e700 1 --2- 172.21.15.47:0/4194858412 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f916c0afdc0 0x7f916c0ae0c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f91640032f0 tx=0x7f91640091d0).ready entity=mon.2 client_cookie=ed62365f603a5bbd server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:42.558 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.540+0000 7f9169d9b700 1 -- 172.21.15.47:0/4194858412 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f916400b600 con 0x7f916c0afdc0 2022-06-29T15:49:42.558 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.540+0000 7f9169d9b700 1 -- 172.21.15.47:0/4194858412 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f916400b760 con 0x7f916c0afdc0 2022-06-29T15:49:42.559 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.540+0000 7f9169d9b700 1 -- 172.21.15.47:0/4194858412 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f916400b8d0 con 0x7f916c0afdc0 2022-06-29T15:49:42.559 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.540+0000 7f9170935700 1 -- 172.21.15.47:0/4194858412 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f916c0afdc0 msgr2=0x7f916c0ae0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:42.559 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.540+0000 7f9170935700 1 --2- 172.21.15.47:0/4194858412 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f916c0afdc0 0x7f916c0ae0c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f91640032f0 tx=0x7f91640091d0).stop 2022-06-29T15:49:42.560 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.540+0000 7f9170935700 1 -- 172.21.15.47:0/4194858412 shutdown_connections 2022-06-29T15:49:42.560 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.540+0000 7f9170935700 1 --2- 172.21.15.47:0/4194858412 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f916c0afdc0 0x7f916c0ae0c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:42.560 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.540+0000 7f9170935700 1 --2- 172.21.15.47:0/4194858412 >> v2:172.21.15.47:3300/0 conn(0x7f916c0b02d0 0x7f916c1406b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:42.560 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.540+0000 7f9170935700 1 --2- 172.21.15.47:0/4194858412 >> v2:172.21.15.64:3300/0 conn(0x7f916c0b5ac0 0x7f916c1401c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:42.560 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.540+0000 7f9170935700 1 -- 172.21.15.47:0/4194858412 >> 172.21.15.47:0/4194858412 conn(0x7f916c062560 msgr2=0x7f916c062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:42.561 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.540+0000 7f9170935700 1 -- 172.21.15.47:0/4194858412 shutdown_connections 2022-06-29T15:49:42.561 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.540+0000 7f9170935700 1 -- 172.21.15.47:0/4194858412 wait complete. 2022-06-29T15:49:42.561 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.540+0000 7f9170935700 1 Processor -- start 2022-06-29T15:49:42.561 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.540+0000 7f9170935700 1 -- start start 2022-06-29T15:49:42.562 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.540+0000 7f9170935700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f916c0afdc0 0x7f916c155900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:42.562 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.540+0000 7f9170935700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f916c0b02d0 0x7f916c140fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:42.562 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.540+0000 7f9170935700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f916c0b5ac0 0x7f916c1414b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:42.562 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.540+0000 7f9170935700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916c1419a0 con 0x7f916c0b5ac0 2022-06-29T15:49:42.562 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.540+0000 7f9170935700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916c0b24c0 con 0x7f916c0b02d0 2022-06-29T15:49:42.563 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.540+0000 7f9170935700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f916c0b0cf0 con 0x7f916c0afdc0 2022-06-29T15:49:42.563 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.540+0000 7f916b59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f916c0b5ac0 0x7f916c1414b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:42.563 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.540+0000 7f916b59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f916c0b5ac0 0x7f916c1414b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51162/0 (socket says 172.21.15.47:51162) 2022-06-29T15:49:42.563 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.540+0000 7f916b59e700 1 -- 172.21.15.47:0/1444329187 learned_addr learned my addr 172.21.15.47:0/1444329187 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:42.564 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.540+0000 7f916a59c700 1 --2- 172.21.15.47:0/1444329187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f916c0b02d0 0x7f916c140fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:42.564 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.540+0000 7f916ad9d700 1 --2- 172.21.15.47:0/1444329187 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f916c0afdc0 0x7f916c155900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:42.564 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.544+0000 7f916b59e700 1 -- 172.21.15.47:0/1444329187 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f916c0afdc0 msgr2=0x7f916c155900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:42.564 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.544+0000 7f916b59e700 1 --2- 172.21.15.47:0/1444329187 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f916c0afdc0 0x7f916c155900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:42.564 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.544+0000 7f916b59e700 1 -- 172.21.15.47:0/1444329187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f916c0b02d0 msgr2=0x7f916c140fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:42.565 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.544+0000 7f916b59e700 1 --2- 172.21.15.47:0/1444329187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f916c0b02d0 0x7f916c140fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:42.565 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.544+0000 7f916b59e700 1 -- 172.21.15.47:0/1444329187 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f915400b1e0 con 0x7f916c0b5ac0 2022-06-29T15:49:42.565 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.544+0000 7f916b59e700 1 --2- 172.21.15.47:0/1444329187 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f916c0b5ac0 0x7f916c1414b0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 rx=0x7f91640032f0 tx=0x7f9164016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:42.565 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.544+0000 7f9163fff700 1 -- 172.21.15.47:0/1444329187 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f916400b600 con 0x7f916c0b5ac0 2022-06-29T15:49:42.566 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.544+0000 7f9163fff700 1 -- 172.21.15.47:0/1444329187 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9164011bf0 con 0x7f916c0b5ac0 2022-06-29T15:49:42.566 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.544+0000 7f9163fff700 1 -- 172.21.15.47:0/1444329187 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f916401f3f0 con 0x7f916c0b5ac0 2022-06-29T15:49:42.566 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.544+0000 7f9170935700 1 -- 172.21.15.47:0/1444329187 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9164009470 con 0x7f916c0b5ac0 2022-06-29T15:49:42.566 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.544+0000 7f9170935700 1 -- 172.21.15.47:0/1444329187 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f916c0b1010 con 0x7f916c0b5ac0 2022-06-29T15:49:42.566 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.544+0000 7f9170935700 1 -- 172.21.15.47:0/1444329187 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f916c0573d0 con 0x7f916c0b5ac0 2022-06-29T15:49:42.567 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.544+0000 7f9163fff700 1 -- 172.21.15.47:0/1444329187 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9164026030 con 0x7f916c0b5ac0 2022-06-29T15:49:42.568 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.544+0000 7f9163fff700 1 --2- 172.21.15.47:0/1444329187 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9150053950 0x7f9150055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:42.568 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.544+0000 7f916ad9d700 1 --2- 172.21.15.47:0/1444329187 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9150053950 0x7f9150055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:42.569 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.544+0000 7f9163fff700 1 -- 172.21.15.47:0/1444329187 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f916406c880 con 0x7f916c0b5ac0 2022-06-29T15:49:42.569 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.548+0000 7f916ad9d700 1 --2- 172.21.15.47:0/1444329187 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9150053950 0x7f9150055e10 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f915c006000 tx=0x7f915c006d90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:42.569 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.552+0000 7f9163fff700 1 -- 172.21.15.47:0/1444329187 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9164043920 con 0x7f916c0b5ac0 2022-06-29T15:49:42.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:42.720+0000 7f9170935700 1 -- 172.21.15.47:0/1444329187 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode"]} v 0) v1 -- 0x7f916c069e20 con 0x7f916c0b5ac0 2022-06-29T15:49:43.148 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.140+0000 7f9163fff700 1 -- 172.21.15.47:0/1444329187 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode"]}]=0 v316) v1 ==== 126+0+0 (secure 0 0 0) 0x7f91640240c0 con 0x7f916c0b5ac0 2022-06-29T15:49:43.152 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.144+0000 7f9170935700 1 -- 172.21.15.47:0/1444329187 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9150053950 msgr2=0x7f9150055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:43.153 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.144+0000 7f9170935700 1 --2- 172.21.15.47:0/1444329187 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9150053950 0x7f9150055e10 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f915c006000 tx=0x7f915c006d90).stop 2022-06-29T15:49:43.153 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.144+0000 7f9170935700 1 -- 172.21.15.47:0/1444329187 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f916c0b5ac0 msgr2=0x7f916c1414b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:43.153 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.144+0000 7f9170935700 1 --2- 172.21.15.47:0/1444329187 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f916c0b5ac0 0x7f916c1414b0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 rx=0x7f91640032f0 tx=0x7f9164016000).stop 2022-06-29T15:49:43.154 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.144+0000 7f9170935700 1 -- 172.21.15.47:0/1444329187 shutdown_connections 2022-06-29T15:49:43.154 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.144+0000 7f9170935700 1 --2- 172.21.15.47:0/1444329187 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f916c0b5ac0 0x7f916c1414b0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:43.154 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.144+0000 7f9170935700 1 --2- 172.21.15.47:0/1444329187 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9150053950 0x7f9150055e10 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:43.154 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.144+0000 7f9170935700 1 --2- 172.21.15.47:0/1444329187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f916c0b02d0 0x7f916c140fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:43.155 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.144+0000 7f9170935700 1 --2- 172.21.15.47:0/1444329187 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f916c0afdc0 0x7f916c155900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:43.155 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.144+0000 7f9170935700 1 -- 172.21.15.47:0/1444329187 >> 172.21.15.47:0/1444329187 conn(0x7f916c062560 msgr2=0x7f916c060230 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:43.155 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.144+0000 7f9170935700 1 -- 172.21.15.47:0/1444329187 shutdown_connections 2022-06-29T15:49:43.155 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.144+0000 7f9170935700 1 -- 172.21.15.47:0/1444329187 wait complete. 2022-06-29T15:49:43.169 INFO:tasks.cephfs_test_runner:test_dump_inode (tasks.cephfs.test_misc.TestMisc) ... ok 2022-06-29T15:49:43.169 INFO:tasks.cephfs_test_runner:Starting test: test_dump_inode_hexademical (tasks.cephfs.test_misc.TestMisc) 2022-06-29T15:49:43.170 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical' 2022-06-29T15:49:43.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.264+0000 7f55cded3700 1 Processor -- start 2022-06-29T15:49:43.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.264+0000 7f55cded3700 1 -- start start 2022-06-29T15:49:43.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.268+0000 7f55cded3700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f55c80b5c60 0x7f55c8144980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:43.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.268+0000 7f55cded3700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f55c80a72f0 0x7f55c80b2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:43.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.268+0000 7f55cded3700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55c80a9120 con 0x7f55c8060eb0 2022-06-29T15:49:43.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.268+0000 7f55cded3700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55c80b0b80 con 0x7f55c80a72f0 2022-06-29T15:49:43.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.268+0000 7f55cded3700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55c80a89e0 con 0x7f55c80b5c60 2022-06-29T15:49:43.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.268+0000 7f55cd6d2700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f55c80a72f0 0x7f55c80b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:43.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.268+0000 7f55cd6d2700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f55c80a72f0 0x7f55c80b2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51170/0 (socket says 172.21.15.47:51170) 2022-06-29T15:49:43.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.268+0000 7f55cd6d2700 1 -- 172.21.15.47:0/2327529436 learned_addr learned my addr 172.21.15.47:0/2327529436 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:43.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.268+0000 7f55cced1700 1 --2- 172.21.15.47:0/2327529436 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f55c80b5c60 0x7f55c8144980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:43.283 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.268+0000 7f55cd6d2700 1 -- 172.21.15.47:0/2327529436 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f55c80b5c60 msgr2=0x7f55c8144980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:43.283 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.268+0000 7f55cd6d2700 1 --2- 172.21.15.47:0/2327529436 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f55c80b5c60 0x7f55c8144980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:43.283 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.268+0000 7f55cd6d2700 1 -- 172.21.15.47:0/2327529436 >> v1:172.21.15.64:6789/0 conn(0x7f55c8060eb0 legacy=0x7f55c80b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:43.283 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.268+0000 7f55cd6d2700 1 -- 172.21.15.47:0/2327529436 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c80a8c80 con 0x7f55c80a72f0 2022-06-29T15:49:43.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.268+0000 7f55cced1700 1 --2- 172.21.15.47:0/2327529436 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f55c80b5c60 0x7f55c8144980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:49:43.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.268+0000 7f55cd6d2700 1 --2- 172.21.15.47:0/2327529436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55c80a72f0 0x7f55c80b2540 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 rx=0x7f55c00032f0 tx=0x7f55c0009290).ready entity=mon.0 client_cookie=5b0eb23d57766247 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:43.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.268+0000 7f55c77fe700 1 -- 172.21.15.47:0/2327529436 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55c000b8a0 con 0x7f55c80a72f0 2022-06-29T15:49:43.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.268+0000 7f55c77fe700 1 -- 172.21.15.47:0/2327529436 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55c000ba00 con 0x7f55c80a72f0 2022-06-29T15:49:43.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.268+0000 7f55c77fe700 1 -- 172.21.15.47:0/2327529436 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55c000bb70 con 0x7f55c80a72f0 2022-06-29T15:49:43.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.268+0000 7f55cded3700 1 -- 172.21.15.47:0/2327529436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55c80a72f0 msgr2=0x7f55c80b2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:43.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.268+0000 7f55cded3700 1 --2- 172.21.15.47:0/2327529436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55c80a72f0 0x7f55c80b2540 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 rx=0x7f55c00032f0 tx=0x7f55c0009290).stop 2022-06-29T15:49:43.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.268+0000 7f55cded3700 1 -- 172.21.15.47:0/2327529436 shutdown_connections 2022-06-29T15:49:43.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.268+0000 7f55cded3700 1 --2- 172.21.15.47:0/2327529436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55c80a72f0 0x7f55c80b2540 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:43.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.268+0000 7f55cded3700 1 --2- 172.21.15.47:0/2327529436 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f55c80b5c60 0x7f55c8144980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:43.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.268+0000 7f55cded3700 1 -- 172.21.15.47:0/2327529436 >> 172.21.15.47:0/2327529436 conn(0x7f55c8062560 msgr2=0x7f55c8062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:43.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.268+0000 7f55cded3700 1 -- 172.21.15.47:0/2327529436 shutdown_connections 2022-06-29T15:49:43.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.268+0000 7f55cded3700 1 -- 172.21.15.47:0/2327529436 wait complete. 2022-06-29T15:49:43.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.272+0000 7f55cded3700 1 Processor -- start 2022-06-29T15:49:43.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.272+0000 7f55cded3700 1 -- start start 2022-06-29T15:49:43.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.272+0000 7f55cded3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55c8060eb0 0x7f55c80b35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:43.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.272+0000 7f55cded3700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f55c80a72f0 0x7f55c814cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:43.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.272+0000 7f55cded3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55c80b5c60 0x7f55c80a9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:43.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.272+0000 7f55cded3700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c814ca30 con 0x7f55c80b5c60 2022-06-29T15:49:43.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.272+0000 7f55cded3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c814d170 con 0x7f55c8060eb0 2022-06-29T15:49:43.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.272+0000 7f55cded3700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55c814c4f0 con 0x7f55c80a72f0 2022-06-29T15:49:43.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.272+0000 7f55cd6d2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55c80b5c60 0x7f55c80a9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:43.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.272+0000 7f55cced1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55c8060eb0 0x7f55c80b35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:43.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.272+0000 7f55cd6d2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55c80b5c60 0x7f55c80a9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51176/0 (socket says 172.21.15.47:51176) 2022-06-29T15:49:43.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.272+0000 7f55cd6d2700 1 -- 172.21.15.47:0/894686581 learned_addr learned my addr 172.21.15.47:0/894686581 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:43.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.272+0000 7f55c7fff700 1 --2- 172.21.15.47:0/894686581 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f55c80a72f0 0x7f55c814cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:43.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.272+0000 7f55cd6d2700 1 -- 172.21.15.47:0/894686581 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f55c80a72f0 msgr2=0x7f55c814cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:43.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.272+0000 7f55cd6d2700 1 --2- 172.21.15.47:0/894686581 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f55c80a72f0 0x7f55c814cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:43.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.272+0000 7f55cd6d2700 1 -- 172.21.15.47:0/894686581 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55c8060eb0 msgr2=0x7f55c80b35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:43.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.272+0000 7f55cd6d2700 1 --2- 172.21.15.47:0/894686581 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55c8060eb0 0x7f55c80b35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:43.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.272+0000 7f55cd6d2700 1 -- 172.21.15.47:0/894686581 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c0009530 con 0x7f55c80b5c60 2022-06-29T15:49:43.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.272+0000 7f55cd6d2700 1 --2- 172.21.15.47:0/894686581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55c80b5c60 0x7f55c80a9600 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 rx=0x7f55c00032f0 tx=0x7f55c0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:43.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.272+0000 7f55c5ffb700 1 -- 172.21.15.47:0/894686581 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55c000b8a0 con 0x7f55c80b5c60 2022-06-29T15:49:43.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.272+0000 7f55c5ffb700 1 -- 172.21.15.47:0/894686581 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55c000bb70 con 0x7f55c80b5c60 2022-06-29T15:49:43.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.272+0000 7f55cded3700 1 -- 172.21.15.47:0/894686581 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55c814c790 con 0x7f55c80b5c60 2022-06-29T15:49:43.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.272+0000 7f55c5ffb700 1 -- 172.21.15.47:0/894686581 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55c0020620 con 0x7f55c80b5c60 2022-06-29T15:49:43.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.272+0000 7f55cded3700 1 -- 172.21.15.47:0/894686581 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55c80b3f90 con 0x7f55c80b5c60 2022-06-29T15:49:43.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.276+0000 7f55cded3700 1 -- 172.21.15.47:0/894686581 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55c80573d0 con 0x7f55c80b5c60 2022-06-29T15:49:43.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.276+0000 7f55c5ffb700 1 -- 172.21.15.47:0/894686581 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f55c0026020 con 0x7f55c80b5c60 2022-06-29T15:49:43.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.276+0000 7f55c5ffb700 1 --2- 172.21.15.47:0/894686581 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f55a80539e0 0x7f55a8055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:43.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.276+0000 7f55c5ffb700 1 -- 172.21.15.47:0/894686581 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f55c006cbc0 con 0x7f55c80b5c60 2022-06-29T15:49:43.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.276+0000 7f55cced1700 1 --2- 172.21.15.47:0/894686581 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f55a80539e0 0x7f55a8055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:43.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.276+0000 7f55cced1700 1 --2- 172.21.15.47:0/894686581 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f55a80539e0 0x7f55a8055ea0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7f55bc00c000 tx=0x7f55bc008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:43.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.284+0000 7f55c5ffb700 1 -- 172.21.15.47:0/894686581 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f55c0043b50 con 0x7f55c80b5c60 2022-06-29T15:49:43.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:43.452+0000 7f55cded3700 1 -- 172.21.15.47:0/894686581 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical"]} v 0) v1 -- 0x7f55c8069e20 con 0x7f55c80b5c60 2022-06-29T15:49:44.151 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.140+0000 7f55c5ffb700 1 -- 172.21.15.47:0/894686581 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical"]}]=0 v317) v1 ==== 141+0+0 (secure 0 0 0) 0x7f55c0015070 con 0x7f55c80b5c60 2022-06-29T15:49:44.154 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.144+0000 7f55cded3700 1 -- 172.21.15.47:0/894686581 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f55a80539e0 msgr2=0x7f55a8055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:44.155 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.144+0000 7f55cded3700 1 --2- 172.21.15.47:0/894686581 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f55a80539e0 0x7f55a8055ea0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7f55bc00c000 tx=0x7f55bc008040).stop 2022-06-29T15:49:44.155 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.144+0000 7f55cded3700 1 -- 172.21.15.47:0/894686581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55c80b5c60 msgr2=0x7f55c80a9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:44.155 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.144+0000 7f55cded3700 1 --2- 172.21.15.47:0/894686581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55c80b5c60 0x7f55c80a9600 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 rx=0x7f55c00032f0 tx=0x7f55c0016000).stop 2022-06-29T15:49:44.156 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.144+0000 7f55cded3700 1 -- 172.21.15.47:0/894686581 shutdown_connections 2022-06-29T15:49:44.156 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.144+0000 7f55cded3700 1 --2- 172.21.15.47:0/894686581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55c80b5c60 0x7f55c80a9600 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:44.156 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.144+0000 7f55cded3700 1 --2- 172.21.15.47:0/894686581 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f55c80a72f0 0x7f55c814cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:44.156 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.144+0000 7f55cded3700 1 --2- 172.21.15.47:0/894686581 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f55a80539e0 0x7f55a8055ea0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:44.156 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.144+0000 7f55cded3700 1 --2- 172.21.15.47:0/894686581 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55c8060eb0 0x7f55c80b35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:44.157 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.144+0000 7f55cded3700 1 -- 172.21.15.47:0/894686581 >> 172.21.15.47:0/894686581 conn(0x7f55c8062560 msgr2=0x7f55c80afae0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:44.157 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.144+0000 7f55cded3700 1 -- 172.21.15.47:0/894686581 shutdown_connections 2022-06-29T15:49:44.157 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.144+0000 7f55cded3700 1 -- 172.21.15.47:0/894686581 wait complete. 2022-06-29T15:49:44.170 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-06-29T15:49:44.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.280+0000 7ff568d52700 1 Processor -- start 2022-06-29T15:49:44.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.280+0000 7ff568d52700 1 -- start start 2022-06-29T15:49:44.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.284+0000 7ff568d52700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff56405c850 0x7ff564143a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:44.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.284+0000 7ff568d52700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff5640b5d90 0x7ff5640b4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:44.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.284+0000 7ff568d52700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff56405cc30 con 0x7ff5640676a0 2022-06-29T15:49:44.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.284+0000 7ff568d52700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5641430a0 con 0x7ff5640b5d90 2022-06-29T15:49:44.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.284+0000 7ff568d52700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff564143300 con 0x7ff56405c850 2022-06-29T15:49:44.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.284+0000 7ff55bfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff5640b5d90 0x7ff5640b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:44.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.284+0000 7ff55bfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff5640b5d90 0x7ff5640b4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51184/0 (socket says 172.21.15.47:51184) 2022-06-29T15:49:44.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.284+0000 7ff55bfff700 1 -- 172.21.15.47:0/3149647427 learned_addr learned my addr 172.21.15.47:0/3149647427 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:44.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.284+0000 7ff563fff700 1 --1- 172.21.15.47:0/3149647427 >> v1:172.21.15.64:6789/0 conn(0x7ff5640676a0 0x7ff564140130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:44300/0 (socket says 172.21.15.47:44300) 2022-06-29T15:49:44.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.284+0000 7ff5637fe700 1 --2- 172.21.15.47:0/3149647427 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff56405c850 0x7ff564143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:44.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.284+0000 7ff55bfff700 1 -- 172.21.15.47:0/3149647427 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff56405c850 msgr2=0x7ff564143a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:44.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.284+0000 7ff55bfff700 1 --2- 172.21.15.47:0/3149647427 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff56405c850 0x7ff564143a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:44.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.284+0000 7ff562ffd700 1 -- 172.21.15.47:0/3149647427 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2380177741 0 0) 0x7ff56405cc30 con 0x7ff5640676a0 2022-06-29T15:49:44.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.284+0000 7ff55bfff700 1 -- 172.21.15.47:0/3149647427 >> v1:172.21.15.64:6789/0 conn(0x7ff5640676a0 legacy=0x7ff564140130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:44.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.284+0000 7ff55bfff700 1 -- 172.21.15.47:0/3149647427 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5641435a0 con 0x7ff5640b5d90 2022-06-29T15:49:44.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.284+0000 7ff55bfff700 1 --2- 172.21.15.47:0/3149647427 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5640b5d90 0x7ff5640b4860 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 rx=0x7ff54c002f70 tx=0x7ff54c006eb0).ready entity=mon.0 client_cookie=22a0e71758c43877 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:44.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.284+0000 7ff562ffd700 1 -- 172.21.15.47:0/3149647427 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff54c00d570 con 0x7ff5640b5d90 2022-06-29T15:49:44.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.284+0000 7ff562ffd700 1 -- 172.21.15.47:0/3149647427 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff54c000e90 con 0x7ff5640b5d90 2022-06-29T15:49:44.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.284+0000 7ff562ffd700 1 -- 172.21.15.47:0/3149647427 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff54c00d8c0 con 0x7ff5640b5d90 2022-06-29T15:49:44.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.284+0000 7ff568d52700 1 -- 172.21.15.47:0/3149647427 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5640b5d90 msgr2=0x7ff5640b4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:44.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.284+0000 7ff568d52700 1 --2- 172.21.15.47:0/3149647427 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5640b5d90 0x7ff5640b4860 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 rx=0x7ff54c002f70 tx=0x7ff54c006eb0).stop 2022-06-29T15:49:44.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.284+0000 7ff568d52700 1 -- 172.21.15.47:0/3149647427 shutdown_connections 2022-06-29T15:49:44.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.284+0000 7ff568d52700 1 --2- 172.21.15.47:0/3149647427 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5640b5d90 0x7ff5640b4860 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:44.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.284+0000 7ff568d52700 1 --2- 172.21.15.47:0/3149647427 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff56405c850 0x7ff564143a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:44.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.284+0000 7ff568d52700 1 -- 172.21.15.47:0/3149647427 >> 172.21.15.47:0/3149647427 conn(0x7ff564060e00 msgr2=0x7ff5640611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:44.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.284+0000 7ff568d52700 1 -- 172.21.15.47:0/3149647427 shutdown_connections 2022-06-29T15:49:44.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.284+0000 7ff568d52700 1 -- 172.21.15.47:0/3149647427 wait complete. 2022-06-29T15:49:44.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.288+0000 7ff568d52700 1 Processor -- start 2022-06-29T15:49:44.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.288+0000 7ff568d52700 1 -- start start 2022-06-29T15:49:44.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.288+0000 7ff568d52700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff56405c850 0x7ff56412c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:44.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.288+0000 7ff568d52700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5640676a0 0x7ff564152d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:44.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.288+0000 7ff568d52700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5640b5d90 0x7ff56413fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:44.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.288+0000 7ff568d52700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5640b3390 con 0x7ff56405c850 2022-06-29T15:49:44.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.288+0000 7ff568d52700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5640b3630 con 0x7ff5640b5d90 2022-06-29T15:49:44.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.288+0000 7ff568d52700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5640b38d0 con 0x7ff5640676a0 2022-06-29T15:49:44.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.288+0000 7ff5637fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff56405c850 0x7ff56412c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:44.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.288+0000 7ff563fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5640b5d90 0x7ff56413fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:44.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.288+0000 7ff55bfff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5640676a0 0x7ff564152d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:44.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.288+0000 7ff5637fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff56405c850 0x7ff56412c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51186/0 (socket says 172.21.15.47:51186) 2022-06-29T15:49:44.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.288+0000 7ff5637fe700 1 -- 172.21.15.47:0/3366038576 learned_addr learned my addr 172.21.15.47:0/3366038576 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:44.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.288+0000 7ff563fff700 1 -- 172.21.15.47:0/3366038576 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5640676a0 msgr2=0x7ff564152d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:44.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.288+0000 7ff563fff700 1 --2- 172.21.15.47:0/3366038576 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5640676a0 0x7ff564152d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:44.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.288+0000 7ff563fff700 1 -- 172.21.15.47:0/3366038576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff56405c850 msgr2=0x7ff56412c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:44.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.288+0000 7ff563fff700 1 --2- 172.21.15.47:0/3366038576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff56405c850 0x7ff56412c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:44.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.288+0000 7ff563fff700 1 -- 172.21.15.47:0/3366038576 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff54c00b1e0 con 0x7ff5640b5d90 2022-06-29T15:49:44.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.288+0000 7ff563fff700 1 --2- 172.21.15.47:0/3366038576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5640b5d90 0x7ff56413fb80 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7ff55c001250 tx=0x7ff55c007f20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:44.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.288+0000 7ff5617fa700 1 -- 172.21.15.47:0/3366038576 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff55c00ce60 con 0x7ff5640b5d90 2022-06-29T15:49:44.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.288+0000 7ff568d52700 1 -- 172.21.15.47:0/3366038576 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5640b3b70 con 0x7ff5640b5d90 2022-06-29T15:49:44.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.288+0000 7ff5617fa700 1 -- 172.21.15.47:0/3366038576 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff55c004ab0 con 0x7ff5640b5d90 2022-06-29T15:49:44.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.288+0000 7ff5617fa700 1 -- 172.21.15.47:0/3366038576 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff55c004e10 con 0x7ff5640b5d90 2022-06-29T15:49:44.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.288+0000 7ff568d52700 1 -- 172.21.15.47:0/3366038576 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff564153210 con 0x7ff5640b5d90 2022-06-29T15:49:44.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.292+0000 7ff568d52700 1 -- 172.21.15.47:0/3366038576 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5640b3d00 con 0x7ff5640b5d90 2022-06-29T15:49:44.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.292+0000 7ff5617fa700 1 -- 172.21.15.47:0/3366038576 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff55c01b070 con 0x7ff5640b5d90 2022-06-29T15:49:44.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.292+0000 7ff5617fa700 1 --2- 172.21.15.47:0/3366038576 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff548053aa0 0x7ff548055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:44.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.292+0000 7ff5617fa700 1 -- 172.21.15.47:0/3366038576 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5686+0+0 (secure 0 0 0) 0x7ff55c06b4c0 con 0x7ff5640b5d90 2022-06-29T15:49:44.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.292+0000 7ff5637fe700 1 --2- 172.21.15.47:0/3366038576 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff548053aa0 0x7ff548055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:44.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.292+0000 7ff5637fe700 1 --2- 172.21.15.47:0/3366038576 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff548053aa0 0x7ff548055f60 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7ff554006000 tx=0x7ff554006d90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:44.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.300+0000 7ff5617fa700 1 -- 172.21.15.47:0/3366038576 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff55c042450 con 0x7ff5640b5d90 2022-06-29T15:49:44.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.432+0000 7ff568d52700 1 -- 172.21.15.47:0/3366038576 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff5640573d0 con 0x7ff5640b5d90 2022-06-29T15:49:44.450 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.440+0000 7ff5617fa700 1 -- 172.21.15.47:0/3366038576 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff55c045a70 con 0x7ff5640b5d90 2022-06-29T15:49:44.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.444+0000 7ff568d52700 1 -- 172.21.15.47:0/3366038576 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff548053aa0 msgr2=0x7ff548055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:44.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.444+0000 7ff568d52700 1 --2- 172.21.15.47:0/3366038576 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff548053aa0 0x7ff548055f60 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7ff554006000 tx=0x7ff554006d90).stop 2022-06-29T15:49:44.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.444+0000 7ff568d52700 1 -- 172.21.15.47:0/3366038576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5640b5d90 msgr2=0x7ff56413fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:44.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.444+0000 7ff568d52700 1 --2- 172.21.15.47:0/3366038576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5640b5d90 0x7ff56413fb80 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7ff55c001250 tx=0x7ff55c007f20).stop 2022-06-29T15:49:44.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.444+0000 7ff568d52700 1 -- 172.21.15.47:0/3366038576 shutdown_connections 2022-06-29T15:49:44.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.444+0000 7ff568d52700 1 --2- 172.21.15.47:0/3366038576 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff548053aa0 0x7ff548055f60 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:44.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.444+0000 7ff568d52700 1 --2- 172.21.15.47:0/3366038576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5640b5d90 0x7ff56413fb80 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:44.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.444+0000 7ff568d52700 1 --2- 172.21.15.47:0/3366038576 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5640676a0 0x7ff564152d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:44.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.444+0000 7ff568d52700 1 --2- 172.21.15.47:0/3366038576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff56405c850 0x7ff56412c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:44.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.444+0000 7ff568d52700 1 -- 172.21.15.47:0/3366038576 >> 172.21.15.47:0/3366038576 conn(0x7ff564060e00 msgr2=0x7ff5640ae9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:44.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.444+0000 7ff568d52700 1 -- 172.21.15.47:0/3366038576 shutdown_connections 2022-06-29T15:49:44.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.444+0000 7ff568d52700 1 -- 172.21.15.47:0/3366038576 wait complete. 2022-06-29T15:49:44.470 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-06-29T15:49:44.471 DEBUG:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:49:44.487 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:49:44.506 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:49:44.528 DEBUG:teuthology.orchestra.run.smithi164:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:49:44.549 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-06-29T15:49:44.550 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:49:44.550 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:49:44.550 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:49:44.576 INFO:teuthology.orchestra.run.smithi099.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-06-29T15:49:44.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T15:49:44.578 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-06-29T15:49:44.578 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:49:44.579 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:49:44.579 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:49:44.586 INFO:teuthology.orchestra.run.smithi164.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-06-29T15:49:44.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T15:49:44.588 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-06-29T15:49:44.589 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-06-29T15:49:44.702 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.692+0000 7f4fb3fff700 1 Processor -- start 2022-06-29T15:49:44.702 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.692+0000 7f4fb3fff700 1 -- start start 2022-06-29T15:49:44.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.692+0000 7f4fb3fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4fb40b5ac0 0x7f4fb41401c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:44.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.692+0000 7f4fb3fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4fb40b02d0 0x7f4fb41406b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:44.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.692+0000 7f4fb3fff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fb4143d20 con 0x7f4fb40afdc0 2022-06-29T15:49:44.707 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.692+0000 7f4fb3fff700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fb40a9090 con 0x7f4fb40b02d0 2022-06-29T15:49:44.707 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.692+0000 7f4fb3fff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fb4140ba0 con 0x7f4fb40b5ac0 2022-06-29T15:49:44.707 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.692+0000 7f4fb27fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4fb40b02d0 0x7f4fb41406b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:44.707 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.692+0000 7f4fb27fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4fb40b02d0 0x7f4fb41406b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51194/0 (socket says 172.21.15.47:51194) 2022-06-29T15:49:44.708 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.692+0000 7f4fb27fc700 1 -- 172.21.15.47:0/1678958168 learned_addr learned my addr 172.21.15.47:0/1678958168 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:44.708 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.692+0000 7f4fb2ffd700 1 --2- 172.21.15.47:0/1678958168 >> v2:172.21.15.64:3300/0 conn(0x7f4fb40b5ac0 0x7f4fb41401c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:44.708 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.696+0000 7f4fb27fc700 1 -- 172.21.15.47:0/1678958168 >> v2:172.21.15.64:3300/0 conn(0x7f4fb40b5ac0 msgr2=0x7f4fb41401c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:44.709 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.696+0000 7f4fb27fc700 1 --2- 172.21.15.47:0/1678958168 >> v2:172.21.15.64:3300/0 conn(0x7f4fb40b5ac0 0x7f4fb41401c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:44.709 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.696+0000 7f4fb27fc700 1 -- 172.21.15.47:0/1678958168 >> v1:172.21.15.64:6789/0 conn(0x7f4fb40afdc0 legacy=0x7f4fb40ae0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:44.709 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.696+0000 7f4fb27fc700 1 -- 172.21.15.47:0/1678958168 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb40a8df0 con 0x7f4fb40b02d0 2022-06-29T15:49:44.709 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.696+0000 7f4fb27fc700 1 --2- 172.21.15.47:0/1678958168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4fb40b02d0 0x7f4fb41406b0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 rx=0x7f4fa40032f0 tx=0x7f4fa4009290).ready entity=mon.0 client_cookie=c4ac712317c0f758 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:44.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.696+0000 7f4fb1ffb700 1 -- 172.21.15.47:0/1678958168 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fa400b870 con 0x7f4fb40b02d0 2022-06-29T15:49:44.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.696+0000 7f4fb1ffb700 1 -- 172.21.15.47:0/1678958168 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fa400b9d0 con 0x7f4fb40b02d0 2022-06-29T15:49:44.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.696+0000 7f4fb1ffb700 1 -- 172.21.15.47:0/1678958168 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fa400bb40 con 0x7f4fb40b02d0 2022-06-29T15:49:44.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.696+0000 7f4fb3fff700 1 -- 172.21.15.47:0/1678958168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4fb40b02d0 msgr2=0x7f4fb41406b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:44.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.696+0000 7f4fb3fff700 1 --2- 172.21.15.47:0/1678958168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4fb40b02d0 0x7f4fb41406b0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 rx=0x7f4fa40032f0 tx=0x7f4fa4009290).stop 2022-06-29T15:49:44.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.696+0000 7f4fb3fff700 1 -- 172.21.15.47:0/1678958168 shutdown_connections 2022-06-29T15:49:44.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.696+0000 7f4fb3fff700 1 --2- 172.21.15.47:0/1678958168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4fb40b02d0 0x7f4fb41406b0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:44.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.696+0000 7f4fb3fff700 1 --2- 172.21.15.47:0/1678958168 >> v2:172.21.15.64:3300/0 conn(0x7f4fb40b5ac0 0x7f4fb41401c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:44.712 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.696+0000 7f4fb3fff700 1 -- 172.21.15.47:0/1678958168 >> 172.21.15.47:0/1678958168 conn(0x7f4fb4062560 msgr2=0x7f4fb4062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:44.712 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.696+0000 7f4fb3fff700 1 -- 172.21.15.47:0/1678958168 shutdown_connections 2022-06-29T15:49:44.712 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.696+0000 7f4fb3fff700 1 -- 172.21.15.47:0/1678958168 wait complete. 2022-06-29T15:49:44.712 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.696+0000 7f4fb3fff700 1 Processor -- start 2022-06-29T15:49:44.713 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.696+0000 7f4fb3fff700 1 -- start start 2022-06-29T15:49:44.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.696+0000 7f4fb3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4fb40afdc0 0x7f4fb40b4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:44.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.700+0000 7f4fb3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4fb40b02d0 0x7f4fb4152170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:44.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.700+0000 7f4fb3fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4fb40b5ac0 0x7f4fb4152660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:44.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.700+0000 7f4fb3fff700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb40ab690 con 0x7f4fb40afdc0 2022-06-29T15:49:44.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.700+0000 7f4fb3fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb4152b50 con 0x7f4fb40b02d0 2022-06-29T15:49:44.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.700+0000 7f4fb3fff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb413e150 con 0x7f4fb40b5ac0 2022-06-29T15:49:44.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.700+0000 7f4fb2ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4fb40afdc0 0x7f4fb40b4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:44.718 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.700+0000 7f4fb2ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4fb40afdc0 0x7f4fb40b4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51200/0 (socket says 172.21.15.47:51200) 2022-06-29T15:49:44.718 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.700+0000 7f4fb2ffd700 1 -- 172.21.15.47:0/417439622 learned_addr learned my addr 172.21.15.47:0/417439622 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:44.718 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.700+0000 7f4fb27fc700 1 --2- 172.21.15.47:0/417439622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4fb40b02d0 0x7f4fb4152170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:44.718 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.700+0000 7f4fb37fe700 1 --2- 172.21.15.47:0/417439622 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4fb40b5ac0 0x7f4fb4152660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:44.719 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.700+0000 7f4fb2ffd700 1 -- 172.21.15.47:0/417439622 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4fb40b5ac0 msgr2=0x7f4fb4152660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:44.719 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.700+0000 7f4fb2ffd700 1 --2- 172.21.15.47:0/417439622 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4fb40b5ac0 0x7f4fb4152660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:44.719 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.700+0000 7f4fb2ffd700 1 -- 172.21.15.47:0/417439622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4fb40b02d0 msgr2=0x7f4fb4152170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:44.719 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.700+0000 7f4fb2ffd700 1 --2- 172.21.15.47:0/417439622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4fb40b02d0 0x7f4fb4152170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:44.720 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.700+0000 7f4fb2ffd700 1 -- 172.21.15.47:0/417439622 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa4009530 con 0x7f4fb40afdc0 2022-06-29T15:49:44.720 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.700+0000 7f4fb27fc700 1 --2- 172.21.15.47:0/417439622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4fb40b02d0 0x7f4fb4152170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:49:44.720 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.700+0000 7f4fb2ffd700 1 --2- 172.21.15.47:0/417439622 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4fb40afdc0 0x7f4fb40b4720 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 rx=0x7f4f9c00a000 tx=0x7f4f9c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:44.720 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.700+0000 7f4fabfff700 1 -- 172.21.15.47:0/417439622 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f9c011780 con 0x7f4fb40afdc0 2022-06-29T15:49:44.721 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.700+0000 7f4fabfff700 1 -- 172.21.15.47:0/417439622 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f9c0118e0 con 0x7f4fb40afdc0 2022-06-29T15:49:44.721 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.700+0000 7f4fb3fff700 1 -- 172.21.15.47:0/417439622 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fb413e3f0 con 0x7f4fb40afdc0 2022-06-29T15:49:44.721 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.700+0000 7f4fabfff700 1 -- 172.21.15.47:0/417439622 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f9c011c40 con 0x7f4fb40afdc0 2022-06-29T15:49:44.721 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.700+0000 7f4fb3fff700 1 -- 172.21.15.47:0/417439622 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fb40abb90 con 0x7f4fb40afdc0 2022-06-29T15:49:44.721 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.700+0000 7f4fb3fff700 1 -- 172.21.15.47:0/417439622 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fb40573d0 con 0x7f4fb40afdc0 2022-06-29T15:49:44.722 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.700+0000 7f4fabfff700 1 -- 172.21.15.47:0/417439622 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4f9c01c460 con 0x7f4fb40afdc0 2022-06-29T15:49:44.722 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.704+0000 7f4fabfff700 1 --2- 172.21.15.47:0/417439622 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4f980539e0 0x7f4f98055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:44.722 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.704+0000 7f4fabfff700 1 -- 172.21.15.47:0/417439622 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f4f9c06c1a0 con 0x7f4fb40afdc0 2022-06-29T15:49:44.722 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.704+0000 7f4fb27fc700 1 --2- 172.21.15.47:0/417439622 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4f980539e0 0x7f4f98055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:44.723 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.704+0000 7f4fb27fc700 1 --2- 172.21.15.47:0/417439622 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4f980539e0 0x7f4f98055ea0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f4fa4019000 tx=0x7f4fa4016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:44.723 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.708+0000 7f4fabfff700 1 -- 172.21.15.47:0/417439622 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4f9c0431e0 con 0x7f4fb40afdc0 2022-06-29T15:49:44.845 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.836+0000 7f4fb3fff700 1 -- 172.21.15.47:0/417439622 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f4fb4069e20 con 0x7f4fb40afdc0 2022-06-29T15:49:44.847 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.840+0000 7f4fabfff700 1 -- 172.21.15.47:0/417439622 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v93) v1 ==== 81+0+29220 (secure 0 0 0) 0x7f4f9c046800 con 0x7f4fb40afdc0 2022-06-29T15:49:44.848 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:49:44.848 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2022-06-29T15:49:44.849 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 93, 2022-06-29T15:49:44.849 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "9a68894a-53ce-400e-838b-14eab91b834c", 2022-06-29T15:49:44.849 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2022-06-29T15:42:14.046361+0000", 2022-06-29T15:49:44.849 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2022-06-29T15:49:42.124424+0000", 2022-06-29T15:49:44.850 INFO:teuthology.orchestra.run.smithi047.stdout: "last_up_change": "2022-06-29T15:42:20.929789+0000", 2022-06-29T15:49:44.850 INFO:teuthology.orchestra.run.smithi047.stdout: "last_in_change": "2022-06-29T15:42:20.929789+0000", 2022-06-29T15:49:44.850 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-06-29T15:49:44.850 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_num": 5799936, 2022-06-29T15:49:44.851 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_set": [ 2022-06-29T15:49:44.851 INFO:teuthology.orchestra.run.smithi047.stdout: "pglog_hardlimit", 2022-06-29T15:49:44.851 INFO:teuthology.orchestra.run.smithi047.stdout: "purged_snapdirs", 2022-06-29T15:49:44.852 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_deletes", 2022-06-29T15:49:44.852 INFO:teuthology.orchestra.run.smithi047.stdout: "sortbitwise" 2022-06-29T15:49:44.852 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:49:44.852 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_version": 5, 2022-06-29T15:49:44.853 INFO:teuthology.orchestra.run.smithi047.stdout: "full_ratio": 0.94999998807907104, 2022-06-29T15:49:44.853 INFO:teuthology.orchestra.run.smithi047.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-06-29T15:49:44.853 INFO:teuthology.orchestra.run.smithi047.stdout: "nearfull_ratio": 0.85000002384185791, 2022-06-29T15:49:44.853 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_snapshot": "", 2022-06-29T15:49:44.854 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_max": 16, 2022-06-29T15:49:44.854 INFO:teuthology.orchestra.run.smithi047.stdout: "max_osd": 8, 2022-06-29T15:49:44.854 INFO:teuthology.orchestra.run.smithi047.stdout: "require_min_compat_client": "jewel", 2022-06-29T15:49:44.855 INFO:teuthology.orchestra.run.smithi047.stdout: "min_compat_client": "jewel", 2022-06-29T15:49:44.855 INFO:teuthology.orchestra.run.smithi047.stdout: "require_osd_release": "octopus", 2022-06-29T15:49:44.855 INFO:teuthology.orchestra.run.smithi047.stdout: "pools": [ 2022-06-29T15:49:44.855 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.855 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 1, 2022-06-29T15:49:44.856 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "device_health_metrics", 2022-06-29T15:49:44.856 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:42:16.971431+0000", 2022-06-29T15:49:44.856 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:49:44.856 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:49:44.857 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:49:44.857 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:49:44.857 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:49:44.857 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:49:44.858 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:49:44.858 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:49:44.858 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 1, 2022-06-29T15:49:44.858 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 1, 2022-06-29T15:49:44.858 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 1, 2022-06-29T15:49:44.859 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 1, 2022-06-29T15:49:44.859 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 1, 2022-06-29T15:49:44.859 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:49:44.859 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:49:44.860 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:49:44.860 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:49:44.860 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:49:44.860 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:49:44.861 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:49:44.861 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.861 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "16", 2022-06-29T15:49:44.861 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:49:44.861 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:49:44.862 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:49:44.862 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:49:44.862 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:49:44.862 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:49:44.863 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:49:44.863 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:49:44.863 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:49:44.863 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:49:44.864 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:49:44.864 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:49:44.864 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:49:44.864 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:49:44.864 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:49:44.865 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:49:44.865 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:49:44.865 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:49:44.865 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:49:44.866 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:49:44.866 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:49:44.866 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:49:44.866 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:49:44.867 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:49:44.867 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:49:44.867 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:49:44.867 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.868 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:49:44.868 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:49:44.868 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:49:44.868 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:49:44.868 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:49:44.869 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:49:44.869 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:49:44.869 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:49:44.869 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:49:44.870 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:49:44.870 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:49:44.870 INFO:teuthology.orchestra.run.smithi047.stdout: "options": { 2022-06-29T15:49:44.870 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_min": 1 2022-06-29T15:49:44.871 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.871 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:49:44.871 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_devicehealth": {} 2022-06-29T15:49:44.871 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:44.871 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.872 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.872 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 2, 2022-06-29T15:49:44.872 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "rbd", 2022-06-29T15:49:44.872 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:42:25.290884+0000", 2022-06-29T15:49:44.873 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:49:44.873 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:49:44.873 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:49:44.873 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:49:44.874 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:49:44.874 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:49:44.874 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:49:44.874 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:49:44.874 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2022-06-29T15:49:44.875 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2022-06-29T15:49:44.875 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2022-06-29T15:49:44.875 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2022-06-29T15:49:44.875 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2022-06-29T15:49:44.876 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:49:44.876 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:49:44.876 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:49:44.876 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:49:44.877 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:49:44.877 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:49:44.877 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:49:44.877 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.877 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "19", 2022-06-29T15:49:44.878 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:49:44.878 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:49:44.878 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:49:44.878 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:49:44.879 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:49:44.879 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:49:44.879 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:49:44.879 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:49:44.879 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:49:44.880 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:49:44.880 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:49:44.880 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:49:44.881 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:49:44.881 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:49:44.881 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:49:44.881 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:49:44.882 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:49:44.882 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:49:44.882 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:49:44.883 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:49:44.883 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:49:44.883 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:49:44.883 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:49:44.884 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:49:44.884 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:49:44.884 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:49:44.884 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.885 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:49:44.885 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:49:44.885 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:49:44.885 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:49:44.886 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:49:44.886 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:49:44.886 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:49:44.886 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:49:44.886 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:49:44.887 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:49:44.887 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:49:44.887 INFO:teuthology.orchestra.run.smithi047.stdout: "options": {}, 2022-06-29T15:49:44.887 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:49:44.888 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd": {} 2022-06-29T15:49:44.888 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:44.888 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:44.888 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:49:44.888 INFO:teuthology.orchestra.run.smithi047.stdout: "osds": [ 2022-06-29T15:49:44.889 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.889 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2022-06-29T15:49:44.889 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "e23eb023-4706-4201-bf72-4bf19c912c5d", 2022-06-29T15:49:44.889 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:49:44.890 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:49:44.890 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:49:44.890 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:49:44.890 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:49:44.891 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:49:44.891 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:49:44.891 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 88, 2022-06-29T15:49:44.891 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:49:44.892 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:49:44.892 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:49:44.892 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:44.892 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.892 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:44.893 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6818", 2022-06-29T15:49:44.893 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:49:44.893 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.893 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.894 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:44.894 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6819", 2022-06-29T15:49:44.894 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:49:44.894 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:44.894 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:44.895 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.895 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:49:44.895 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:44.895 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.896 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:44.896 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6820", 2022-06-29T15:49:44.896 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:49:44.896 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.896 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.897 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:44.897 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6821", 2022-06-29T15:49:44.897 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:49:44.897 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:44.898 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:44.898 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.898 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:49:44.898 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:44.899 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.899 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:44.899 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6824", 2022-06-29T15:49:44.899 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:49:44.899 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.900 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.900 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:44.900 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6825", 2022-06-29T15:49:44.900 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:49:44.901 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:44.901 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:44.901 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.901 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:49:44.901 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:44.902 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.902 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:44.902 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6822", 2022-06-29T15:49:44.902 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:49:44.903 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.903 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.903 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:44.903 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6823", 2022-06-29T15:49:44.903 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:49:44.904 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:44.904 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:44.904 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.904 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6819/13096", 2022-06-29T15:49:44.905 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6821/13096", 2022-06-29T15:49:44.905 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6825/13096", 2022-06-29T15:49:44.905 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6823/13096", 2022-06-29T15:49:44.905 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:49:44.906 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:49:44.906 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:49:44.906 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:44.906 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.906 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.907 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2022-06-29T15:49:44.907 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "228bb08d-2d64-43d9-9560-bfaeff154452", 2022-06-29T15:49:44.907 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:49:44.907 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:49:44.908 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:49:44.908 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:49:44.908 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:49:44.908 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:49:44.909 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:49:44.909 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 87, 2022-06-29T15:49:44.909 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:49:44.909 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:49:44.909 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:49:44.910 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:44.910 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.910 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:44.910 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6826", 2022-06-29T15:49:44.911 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:49:44.911 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.911 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.911 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:44.911 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6827", 2022-06-29T15:49:44.912 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:49:44.912 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:44.912 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:44.912 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.913 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:49:44.913 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:44.913 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.913 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:44.914 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6828", 2022-06-29T15:49:44.914 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:49:44.914 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.914 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.915 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:44.915 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6829", 2022-06-29T15:49:44.915 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:49:44.915 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:44.916 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:44.916 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.916 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:49:44.916 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:44.917 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.917 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:44.917 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6832", 2022-06-29T15:49:44.917 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:49:44.918 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.918 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.918 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:44.918 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6833", 2022-06-29T15:49:44.918 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:49:44.919 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:44.919 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:44.919 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.920 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:49:44.920 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:44.920 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.921 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:44.921 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6830", 2022-06-29T15:49:44.921 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:49:44.921 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.922 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.922 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:44.922 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6831", 2022-06-29T15:49:44.923 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:49:44.923 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:44.923 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:44.923 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.924 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6827/13100", 2022-06-29T15:49:44.924 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6829/13100", 2022-06-29T15:49:44.924 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6833/13100", 2022-06-29T15:49:44.925 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6831/13100", 2022-06-29T15:49:44.925 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:49:44.925 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:49:44.926 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:49:44.926 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:44.926 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.926 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.927 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2022-06-29T15:49:44.927 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "797a5452-1510-4640-a111-c9c2fcf707eb", 2022-06-29T15:49:44.927 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:49:44.927 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:49:44.928 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:49:44.928 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:49:44.928 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:49:44.929 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:49:44.929 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:49:44.929 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 88, 2022-06-29T15:49:44.929 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:49:44.930 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:49:44.930 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:49:44.930 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:44.931 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.931 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:44.931 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6810", 2022-06-29T15:49:44.931 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:49:44.932 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.932 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.932 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:44.932 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6811", 2022-06-29T15:49:44.932 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:49:44.933 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:44.933 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:44.933 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.933 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:49:44.934 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:44.934 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.934 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:44.935 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6812", 2022-06-29T15:49:44.935 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:49:44.936 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.936 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.936 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:44.937 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6813", 2022-06-29T15:49:44.937 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:49:44.937 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:44.938 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:44.938 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.938 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:49:44.938 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:44.939 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.939 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:44.939 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6816", 2022-06-29T15:49:44.939 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:49:44.940 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.940 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.940 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:44.940 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6817", 2022-06-29T15:49:44.940 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:49:44.941 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:44.941 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:44.941 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.941 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:49:44.942 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:44.942 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.942 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:44.942 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6814", 2022-06-29T15:49:44.943 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:49:44.943 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.943 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.943 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:44.943 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6815", 2022-06-29T15:49:44.944 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:49:44.944 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:44.944 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:44.944 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.945 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6811/13098", 2022-06-29T15:49:44.945 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6813/13098", 2022-06-29T15:49:44.945 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6817/13098", 2022-06-29T15:49:44.945 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6815/13098", 2022-06-29T15:49:44.945 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:49:44.946 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:49:44.946 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:49:44.946 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:44.946 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.947 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.947 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2022-06-29T15:49:44.947 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "55b24fd7-bb71-4a01-b3fa-a09caa772a62", 2022-06-29T15:49:44.947 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:49:44.948 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:49:44.948 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:49:44.948 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:49:44.948 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:49:44.948 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:49:44.949 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:49:44.949 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 88, 2022-06-29T15:49:44.949 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:49:44.949 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:49:44.950 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:49:44.950 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:44.950 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.950 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:44.951 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6802", 2022-06-29T15:49:44.951 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:49:44.951 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.951 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.951 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:44.952 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6803", 2022-06-29T15:49:44.952 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:49:44.952 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:44.952 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:44.953 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.953 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:49:44.953 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:44.953 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.954 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:44.954 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6804", 2022-06-29T15:49:44.954 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:49:44.954 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.955 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.955 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:44.955 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6805", 2022-06-29T15:49:44.955 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:49:44.955 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:44.956 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:44.956 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.956 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:49:44.956 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:44.957 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.957 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:44.957 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6808", 2022-06-29T15:49:44.957 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:49:44.957 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.958 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.958 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:44.958 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6809", 2022-06-29T15:49:44.958 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:49:44.959 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:44.959 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:44.959 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.959 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:49:44.960 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:44.960 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.960 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:44.960 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6806", 2022-06-29T15:49:44.960 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:49:44.961 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.961 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.961 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:44.961 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6807", 2022-06-29T15:49:44.962 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:49:44.962 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:44.962 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:44.962 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.962 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6803/13097", 2022-06-29T15:49:44.963 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6805/13097", 2022-06-29T15:49:44.963 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6809/13097", 2022-06-29T15:49:44.963 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6807/13097", 2022-06-29T15:49:44.963 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:49:44.964 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:49:44.964 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:49:44.964 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:44.964 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.965 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.965 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2022-06-29T15:49:44.965 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3", 2022-06-29T15:49:44.965 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:49:44.965 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:49:44.966 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:49:44.966 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:49:44.966 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:49:44.966 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:49:44.967 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:49:44.967 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 88, 2022-06-29T15:49:44.967 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:49:44.967 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:49:44.967 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:49:44.968 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:44.968 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.968 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:44.968 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6816", 2022-06-29T15:49:44.969 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:49:44.969 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.969 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.969 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:44.969 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6817", 2022-06-29T15:49:44.970 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:49:44.970 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:44.970 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:44.970 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.971 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:49:44.971 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:44.971 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.971 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:44.972 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6818", 2022-06-29T15:49:44.972 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:49:44.972 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.972 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.972 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:44.973 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6819", 2022-06-29T15:49:44.973 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:49:44.973 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:44.973 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:44.974 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.974 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:49:44.974 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:44.974 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.974 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:44.975 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6822", 2022-06-29T15:49:44.975 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:49:44.975 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.975 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.976 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:44.976 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6823", 2022-06-29T15:49:44.976 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:49:44.976 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:44.976 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:44.977 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.977 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:49:44.977 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:44.977 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.978 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:44.978 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6820", 2022-06-29T15:49:44.978 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:49:44.978 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.978 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.979 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:44.979 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6821", 2022-06-29T15:49:44.979 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:49:44.979 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:44.980 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:44.980 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.980 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6817/13396", 2022-06-29T15:49:44.980 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6819/13396", 2022-06-29T15:49:44.980 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6823/13396", 2022-06-29T15:49:44.981 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6821/13396", 2022-06-29T15:49:44.981 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:49:44.981 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:49:44.981 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:49:44.982 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:44.982 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.982 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.982 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2022-06-29T15:49:44.982 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "f77beebe-6cca-4244-b2d4-677825ebf919", 2022-06-29T15:49:44.983 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:49:44.983 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:49:44.983 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:49:44.983 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:49:44.984 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:49:44.984 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:49:44.984 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:49:44.984 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 88, 2022-06-29T15:49:44.984 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:49:44.985 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:49:44.985 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:49:44.985 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:44.985 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.986 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:44.986 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6800", 2022-06-29T15:49:44.986 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:49:44.986 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.986 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.987 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:44.987 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6801", 2022-06-29T15:49:44.987 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:49:44.987 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:44.988 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:44.988 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.988 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:49:44.988 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:44.988 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.989 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:44.989 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6802", 2022-06-29T15:49:44.989 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:49:44.989 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.990 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.990 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:44.990 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6803", 2022-06-29T15:49:44.990 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:49:44.991 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:44.991 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:44.991 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.991 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:49:44.992 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:44.992 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.992 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:44.992 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6806", 2022-06-29T15:49:44.992 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:49:44.993 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.993 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.993 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:44.993 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6807", 2022-06-29T15:49:44.994 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:49:44.994 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:44.994 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:44.994 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.994 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:49:44.995 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:44.995 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.995 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:44.995 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6804", 2022-06-29T15:49:44.996 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:49:44.996 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.996 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:44.996 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:44.996 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6805", 2022-06-29T15:49:44.997 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:49:44.997 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:44.997 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:44.997 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:44.998 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6801/13393", 2022-06-29T15:49:44.998 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6803/13393", 2022-06-29T15:49:44.998 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6807/13393", 2022-06-29T15:49:44.998 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6805/13393", 2022-06-29T15:49:44.999 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:49:44.999 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:49:44.999 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:49:44.999 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:44.999 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:45.000 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:45.000 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2022-06-29T15:49:45.000 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "af0ca9b3-c02c-4910-834b-bf0b8dc6713c", 2022-06-29T15:49:45.000 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:49:45.001 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:49:45.001 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:49:45.001 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:49:45.001 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:49:45.001 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:49:45.002 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:49:45.002 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 88, 2022-06-29T15:49:45.002 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:49:45.002 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:49:45.003 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:49:45.003 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:45.003 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:45.003 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:45.003 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6824", 2022-06-29T15:49:45.004 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:49:45.004 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:45.004 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:45.004 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:45.005 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6825", 2022-06-29T15:49:45.005 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:49:45.005 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:45.005 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:45.005 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:45.006 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:49:45.006 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:45.006 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:45.006 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:45.007 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6826", 2022-06-29T15:49:45.007 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:49:45.007 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:45.007 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:45.007 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:45.008 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6827", 2022-06-29T15:49:45.008 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:49:45.008 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:45.008 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:45.009 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:45.009 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:49:45.009 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:45.009 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:45.009 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:45.010 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6830", 2022-06-29T15:49:45.010 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:49:45.010 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:45.010 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:45.011 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:45.011 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6831", 2022-06-29T15:49:45.011 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:49:45.011 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:45.012 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:45.012 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:45.012 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:49:45.012 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:45.012 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:45.013 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:45.013 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6828", 2022-06-29T15:49:45.013 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:49:45.013 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:45.014 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:45.014 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:45.014 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6829", 2022-06-29T15:49:45.014 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:49:45.014 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:45.015 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:45.015 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:45.015 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6825/13395", 2022-06-29T15:49:45.015 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6827/13395", 2022-06-29T15:49:45.016 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6831/13395", 2022-06-29T15:49:45.016 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6829/13395", 2022-06-29T15:49:45.016 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:49:45.016 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:49:45.016 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:49:45.017 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:45.017 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:45.017 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:45.017 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2022-06-29T15:49:45.018 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "f19c3ca6-df88-4bde-a150-b51a53f092e1", 2022-06-29T15:49:45.018 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:49:45.018 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:49:45.018 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:49:45.019 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:49:45.019 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:49:45.019 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:49:45.019 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:49:45.019 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 75, 2022-06-29T15:49:45.020 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:49:45.020 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:49:45.020 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:49:45.020 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:45.020 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:45.021 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:45.021 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6808", 2022-06-29T15:49:45.021 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:49:45.021 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:45.022 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:45.022 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:45.022 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6809", 2022-06-29T15:49:45.022 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:49:45.023 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:45.023 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:45.023 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:45.023 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:49:45.023 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:45.024 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:45.024 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:45.024 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6810", 2022-06-29T15:49:45.024 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:49:45.025 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:45.025 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:45.025 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:45.025 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6811", 2022-06-29T15:49:45.025 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:49:45.026 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:45.026 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:45.026 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:45.026 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:49:45.027 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:45.027 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:45.027 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:45.027 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6814", 2022-06-29T15:49:45.027 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:49:45.028 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:45.028 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:45.028 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:45.028 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6815", 2022-06-29T15:49:45.029 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:49:45.029 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:45.029 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:45.029 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:45.030 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:49:45.030 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:49:45.030 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:45.030 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:49:45.030 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6812", 2022-06-29T15:49:45.031 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:49:45.031 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:45.031 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:45.031 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:49:45.032 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6813", 2022-06-29T15:49:45.032 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:49:45.032 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:45.032 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:45.033 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:45.033 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6809/13394", 2022-06-29T15:49:45.033 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6811/13394", 2022-06-29T15:49:45.033 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6815/13394", 2022-06-29T15:49:45.033 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6813/13394", 2022-06-29T15:49:45.034 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:49:45.034 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:49:45.034 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:49:45.034 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:45.035 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:45.035 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:49:45.035 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_xinfo": [ 2022-06-29T15:49:45.035 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:45.035 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2022-06-29T15:49:45.036 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:49:45.036 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:49:45.036 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:49:45.036 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:49:45.037 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:49:45.037 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.720513+0000", 2022-06-29T15:49:45.037 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:49:45.037 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:45.037 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:45.038 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2022-06-29T15:49:45.038 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:49:45.038 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:49:45.038 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:49:45.039 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:49:45.039 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:49:45.039 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.772612+0000", 2022-06-29T15:49:45.039 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:49:45.040 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:45.040 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:45.040 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2022-06-29T15:49:45.040 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:49:45.040 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:49:45.041 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:49:45.041 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:49:45.041 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:49:45.041 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.450223+0000", 2022-06-29T15:49:45.042 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:49:45.042 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:45.042 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:45.042 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2022-06-29T15:49:45.042 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:49:45.043 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:49:45.043 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:49:45.043 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:49:45.043 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:49:45.044 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.477856+0000", 2022-06-29T15:49:45.044 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:49:45.044 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:45.044 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:45.045 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2022-06-29T15:49:45.045 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:49:45.045 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:49:45.045 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:49:45.045 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:49:45.046 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:49:45.046 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.539485+0000", 2022-06-29T15:49:45.046 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:49:45.046 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:45.047 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:45.047 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2022-06-29T15:49:45.047 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:49:45.047 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:49:45.048 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:49:45.048 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:49:45.048 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:49:45.048 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.487040+0000", 2022-06-29T15:49:45.048 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:49:45.049 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:45.049 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:45.049 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2022-06-29T15:49:45.049 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:49:45.050 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:49:45.050 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:49:45.050 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:49:45.050 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:49:45.051 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.748685+0000", 2022-06-29T15:49:45.051 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:49:45.051 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:45.051 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:45.051 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2022-06-29T15:49:45.052 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:49:45.052 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:49:45.052 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:49:45.052 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:49:45.053 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:49:45.053 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.459516+0000", 2022-06-29T15:49:45.053 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:49:45.053 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:45.054 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:49:45.054 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap": [], 2022-06-29T15:49:45.054 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap_items": [], 2022-06-29T15:49:45.054 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_temp": [], 2022-06-29T15:49:45.054 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_temp": [], 2022-06-29T15:49:45.055 INFO:teuthology.orchestra.run.smithi047.stdout: "blacklist": { 2022-06-29T15:49:45.055 INFO:teuthology.orchestra.run.smithi047.stdout: "172.21.15.47:6837/4146905199": "2022-06-30T15:49:39.004068+0000", 2022-06-29T15:49:45.055 INFO:teuthology.orchestra.run.smithi047.stdout: "172.21.15.47:6836/4146905199": "2022-06-30T15:49:39.004068+0000" 2022-06-29T15:49:45.055 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:45.056 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profiles": { 2022-06-29T15:49:45.056 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs_data": { 2022-06-29T15:49:45.056 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-device-class": "", 2022-06-29T15:49:45.056 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2022-06-29T15:49:45.056 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-root": "default", 2022-06-29T15:49:45.057 INFO:teuthology.orchestra.run.smithi047.stdout: "jerasure-per-chunk-alignment": "false", 2022-06-29T15:49:45.057 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2022-06-29T15:49:45.057 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "2", 2022-06-29T15:49:45.057 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2022-06-29T15:49:45.058 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2022-06-29T15:49:45.058 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van", 2022-06-29T15:49:45.058 INFO:teuthology.orchestra.run.smithi047.stdout: "w": "8" 2022-06-29T15:49:45.058 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:45.059 INFO:teuthology.orchestra.run.smithi047.stdout: "default": { 2022-06-29T15:49:45.059 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2022-06-29T15:49:45.059 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2022-06-29T15:49:45.059 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "1", 2022-06-29T15:49:45.059 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2022-06-29T15:49:45.060 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2022-06-29T15:49:45.060 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van" 2022-06-29T15:49:45.060 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:45.060 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:45.061 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps_queue": [], 2022-06-29T15:49:45.061 INFO:teuthology.orchestra.run.smithi047.stdout: "new_removed_snaps": [], 2022-06-29T15:49:45.061 INFO:teuthology.orchestra.run.smithi047.stdout: "new_purged_snaps": [], 2022-06-29T15:49:45.061 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_node_flags": {}, 2022-06-29T15:49:45.061 INFO:teuthology.orchestra.run.smithi047.stdout: "device_class_flags": {} 2022-06-29T15:49:45.062 INFO:teuthology.orchestra.run.smithi047.stdout:} 2022-06-29T15:49:45.064 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.840+0000 7f4fb3fff700 1 -- 172.21.15.47:0/417439622 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4f980539e0 msgr2=0x7f4f98055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:45.064 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.840+0000 7f4fb3fff700 1 --2- 172.21.15.47:0/417439622 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4f980539e0 0x7f4f98055ea0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f4fa4019000 tx=0x7f4fa4016000).stop 2022-06-29T15:49:45.064 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.840+0000 7f4fb3fff700 1 -- 172.21.15.47:0/417439622 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4fb40afdc0 msgr2=0x7f4fb40b4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:45.065 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.840+0000 7f4fb3fff700 1 --2- 172.21.15.47:0/417439622 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4fb40afdc0 0x7f4fb40b4720 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 rx=0x7f4f9c00a000 tx=0x7f4f9c008040).stop 2022-06-29T15:49:45.065 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.840+0000 7f4fb3fff700 1 -- 172.21.15.47:0/417439622 shutdown_connections 2022-06-29T15:49:45.065 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.840+0000 7f4fb3fff700 1 --2- 172.21.15.47:0/417439622 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4fb40b5ac0 0x7f4fb4152660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:45.065 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.840+0000 7f4fb3fff700 1 --2- 172.21.15.47:0/417439622 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4f980539e0 0x7f4f98055ea0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:45.065 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.840+0000 7f4fb3fff700 1 --2- 172.21.15.47:0/417439622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4fb40b02d0 0x7f4fb4152170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:45.066 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.840+0000 7f4fb3fff700 1 --2- 172.21.15.47:0/417439622 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4fb40afdc0 0x7f4fb40b4720 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:45.066 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.840+0000 7f4fb3fff700 1 -- 172.21.15.47:0/417439622 >> 172.21.15.47:0/417439622 conn(0x7f4fb4062560 msgr2=0x7f4fb4140fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:45.066 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.840+0000 7f4fb3fff700 1 -- 172.21.15.47:0/417439622 shutdown_connections 2022-06-29T15:49:45.066 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:44.840+0000 7f4fb3fff700 1 -- 172.21.15.47:0/417439622 wait complete. 2022-06-29T15:49:45.067 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:49:45.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.172+0000 7ff251190700 1 Processor -- start 2022-06-29T15:49:45.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.172+0000 7ff251190700 1 -- start start 2022-06-29T15:49:45.183 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.172+0000 7ff251190700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff24c05c510 0x7ff24c143bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:45.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.172+0000 7ff251190700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff24c142f90 con 0x7ff24c0b2c40 2022-06-29T15:49:45.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.172+0000 7ff251190700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff24c1431f0 con 0x7ff24c05c8f0 2022-06-29T15:49:45.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.172+0000 7ff251190700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff24c143490 con 0x7ff24c05c510 2022-06-29T15:49:45.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.172+0000 7ff25098f700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff24c0b2c40 0x7ff24c13cbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36766/0 (socket says 172.21.15.47:36766) 2022-06-29T15:49:45.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.172+0000 7ff25098f700 1 -- 172.21.15.47:0/4101091703 learned_addr learned my addr 172.21.15.47:0/4101091703 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:49:45.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.172+0000 7ff24bfff700 1 --2- 172.21.15.47:0/4101091703 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff24c05c510 0x7ff24c143bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:45.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.172+0000 7ff24affd700 1 -- 172.21.15.47:0/4101091703 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3044861748 0 0) 0x7ff24c142f90 con 0x7ff24c0b2c40 2022-06-29T15:49:45.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.172+0000 7ff24affd700 1 -- 172.21.15.47:0/4101091703 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff238003260 con 0x7ff24c0b2c40 2022-06-29T15:49:45.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.172+0000 7ff24affd700 1 -- 172.21.15.47:0/4101091703 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4014957853 0 0) 0x7ff24c1431f0 con 0x7ff24c05c8f0 2022-06-29T15:49:45.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.172+0000 7ff24affd700 1 -- 172.21.15.47:0/4101091703 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff24c142f90 con 0x7ff24c05c8f0 2022-06-29T15:49:45.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.172+0000 7ff24bfff700 1 -- 172.21.15.47:0/4101091703 >> v1:172.21.15.64:6789/0 conn(0x7ff24c05c8f0 legacy=0x7ff24c0b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:45.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.172+0000 7ff24bfff700 1 -- 172.21.15.47:0/4101091703 >> v1:172.21.15.47:6789/0 conn(0x7ff24c0b2c40 legacy=0x7ff24c13cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:45.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.172+0000 7ff24affd700 1 -- 172.21.15.47:0/4101091703 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 742387351 0 0) 0x7ff238003260 con 0x7ff24c0b2c40 2022-06-29T15:49:45.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.172+0000 7ff24bfff700 1 -- 172.21.15.47:0/4101091703 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff24c143730 con 0x7ff24c05c510 2022-06-29T15:49:45.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.176+0000 7ff24bfff700 1 --2- 172.21.15.47:0/4101091703 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff24c05c510 0x7ff24c143bd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7ff244001490 tx=0x7ff240006df0).ready entity=mon.2 client_cookie=486e2e71519e908e server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:45.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.176+0000 7ff24affd700 1 -- 172.21.15.47:0/4101091703 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff244001a00 con 0x7ff24c05c510 2022-06-29T15:49:45.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.176+0000 7ff24affd700 1 -- 172.21.15.47:0/4101091703 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff240000e90 con 0x7ff24c05c510 2022-06-29T15:49:45.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.176+0000 7ff24affd700 1 -- 172.21.15.47:0/4101091703 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff240012460 con 0x7ff24c05c510 2022-06-29T15:49:45.191 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.176+0000 7ff251190700 1 -- 172.21.15.47:0/4101091703 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff24c05c510 msgr2=0x7ff24c143bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:45.192 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.176+0000 7ff251190700 1 --2- 172.21.15.47:0/4101091703 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff24c05c510 0x7ff24c143bd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7ff244001490 tx=0x7ff240006df0).stop 2022-06-29T15:49:45.192 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.176+0000 7ff251190700 1 -- 172.21.15.47:0/4101091703 shutdown_connections 2022-06-29T15:49:45.192 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.176+0000 7ff251190700 1 --2- 172.21.15.47:0/4101091703 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff24c05c510 0x7ff24c143bd0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:45.192 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.176+0000 7ff251190700 1 -- 172.21.15.47:0/4101091703 >> 172.21.15.47:0/4101091703 conn(0x7ff24c060e00 msgr2=0x7ff24c0611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:45.193 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.176+0000 7ff251190700 1 -- 172.21.15.47:0/4101091703 shutdown_connections 2022-06-29T15:49:45.193 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.176+0000 7ff251190700 1 -- 172.21.15.47:0/4101091703 wait complete. 2022-06-29T15:49:45.193 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.176+0000 7ff251190700 1 Processor -- start 2022-06-29T15:49:45.193 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.176+0000 7ff251190700 1 -- start start 2022-06-29T15:49:45.194 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.180+0000 7ff251190700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff24c05c510 0x7ff24c144860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:45.194 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.180+0000 7ff251190700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff24c05c8f0 0x7ff24c13a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:45.194 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.180+0000 7ff251190700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff24c0b2c40 0x7ff24c13ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:45.194 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.180+0000 7ff251190700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff24c155ed0 con 0x7ff24c05c8f0 2022-06-29T15:49:45.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.180+0000 7ff251190700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff24c13b080 con 0x7ff24c0b2c40 2022-06-29T15:49:45.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.180+0000 7ff251190700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff24c140bf0 con 0x7ff24c05c510 2022-06-29T15:49:45.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.180+0000 7ff24b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff24c05c8f0 0x7ff24c13a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:45.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.180+0000 7ff24b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff24c05c8f0 0x7ff24c13a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51218/0 (socket says 172.21.15.47:51218) 2022-06-29T15:49:45.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.180+0000 7ff25098f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff24c0b2c40 0x7ff24c13ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:45.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.180+0000 7ff24b7fe700 1 -- 172.21.15.47:0/2855888152 learned_addr learned my addr 172.21.15.47:0/2855888152 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:45.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.180+0000 7ff24bfff700 1 --2- 172.21.15.47:0/2855888152 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff24c05c510 0x7ff24c144860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:45.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.180+0000 7ff24b7fe700 1 -- 172.21.15.47:0/2855888152 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff24c05c510 msgr2=0x7ff24c144860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:45.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.180+0000 7ff24b7fe700 1 --2- 172.21.15.47:0/2855888152 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff24c05c510 0x7ff24c144860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:45.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.180+0000 7ff24b7fe700 1 -- 172.21.15.47:0/2855888152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff24c0b2c40 msgr2=0x7ff24c13ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:45.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.180+0000 7ff24b7fe700 1 --2- 172.21.15.47:0/2855888152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff24c0b2c40 0x7ff24c13ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:45.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.180+0000 7ff24b7fe700 1 -- 172.21.15.47:0/2855888152 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff24000b180 con 0x7ff24c05c8f0 2022-06-29T15:49:45.198 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.180+0000 7ff24b7fe700 1 --2- 172.21.15.47:0/2855888152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff24c05c8f0 0x7ff24c13a6a0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 rx=0x7ff24c13b1c0 tx=0x7ff23c007ae0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:45.198 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.180+0000 7ff2497fa700 1 -- 172.21.15.47:0/2855888152 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff23c012400 con 0x7ff24c05c8f0 2022-06-29T15:49:45.198 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.180+0000 7ff2497fa700 1 -- 172.21.15.47:0/2855888152 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff23c012560 con 0x7ff24c05c8f0 2022-06-29T15:49:45.198 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.180+0000 7ff2497fa700 1 -- 172.21.15.47:0/2855888152 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff23c0128c0 con 0x7ff24c05c8f0 2022-06-29T15:49:45.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.180+0000 7ff251190700 1 -- 172.21.15.47:0/2855888152 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff24c13a240 con 0x7ff24c05c8f0 2022-06-29T15:49:45.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.180+0000 7ff251190700 1 -- 172.21.15.47:0/2855888152 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff24c1408e0 con 0x7ff24c05c8f0 2022-06-29T15:49:45.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.184+0000 7ff251190700 1 -- 172.21.15.47:0/2855888152 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff24c0573d0 con 0x7ff24c05c8f0 2022-06-29T15:49:45.200 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.184+0000 7ff2497fa700 1 -- 172.21.15.47:0/2855888152 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff23c019030 con 0x7ff24c05c8f0 2022-06-29T15:49:45.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.184+0000 7ff2497fa700 1 --2- 172.21.15.47:0/2855888152 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff22c053aa0 0x7ff22c055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:45.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.184+0000 7ff2497fa700 1 -- 172.21.15.47:0/2855888152 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5686+0+0 (secure 0 0 0) 0x7ff23c06b0e0 con 0x7ff24c05c8f0 2022-06-29T15:49:45.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.184+0000 7ff24bfff700 1 --2- 172.21.15.47:0/2855888152 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff22c053aa0 0x7ff22c055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:45.202 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.184+0000 7ff24bfff700 1 --2- 172.21.15.47:0/2855888152 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff22c053aa0 0x7ff22c055f60 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7ff24001a000 tx=0x7ff240014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:45.202 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.192+0000 7ff2497fa700 1 -- 172.21.15.47:0/2855888152 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff23c042180 con 0x7ff24c05c8f0 2022-06-29T15:49:45.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.352+0000 7ff251190700 1 -- 172.21.15.47:0/2855888152 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff24c0b0ae0 con 0x7ff24c05c8f0 2022-06-29T15:49:45.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.352+0000 7ff2497fa700 1 -- 172.21.15.47:0/2855888152 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+1562 (secure 0 0 0) 0x7ff23c0457a0 con 0x7ff24c05c8f0 2022-06-29T15:49:45.363 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:49:45.363 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":42,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6577,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/4286932616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":4286932616},{"type":"v1","addr":"172.21.15.64:6833","nonce":4286932616}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":38},{"gid":6657,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3528102962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3528102962},{"type":"v1","addr":"172.21.15.47:6835","nonce":3528102962}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":38},{"gid":6894,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/3564259639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":3564259639},{"type":"v1","addr":"172.21.15.47:6837","nonce":3564259639}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":42}],"filesystems":[]} 2022-06-29T15:49:45.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.356+0000 7ff251190700 1 -- 172.21.15.47:0/2855888152 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff22c053aa0 msgr2=0x7ff22c055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:45.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.356+0000 7ff251190700 1 --2- 172.21.15.47:0/2855888152 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff22c053aa0 0x7ff22c055f60 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7ff24001a000 tx=0x7ff240014000).stop 2022-06-29T15:49:45.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.356+0000 7ff251190700 1 -- 172.21.15.47:0/2855888152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff24c05c8f0 msgr2=0x7ff24c13a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:45.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.356+0000 7ff251190700 1 --2- 172.21.15.47:0/2855888152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff24c05c8f0 0x7ff24c13a6a0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 rx=0x7ff24c13b1c0 tx=0x7ff23c007ae0).stop 2022-06-29T15:49:45.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.356+0000 7ff251190700 1 -- 172.21.15.47:0/2855888152 shutdown_connections 2022-06-29T15:49:45.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.356+0000 7ff251190700 1 --2- 172.21.15.47:0/2855888152 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff22c053aa0 0x7ff22c055f60 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:45.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.356+0000 7ff251190700 1 --2- 172.21.15.47:0/2855888152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff24c0b2c40 0x7ff24c13ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:45.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.356+0000 7ff251190700 1 --2- 172.21.15.47:0/2855888152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff24c05c8f0 0x7ff24c13a6a0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:45.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.356+0000 7ff251190700 1 --2- 172.21.15.47:0/2855888152 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff24c05c510 0x7ff24c144860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:45.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.356+0000 7ff251190700 1 -- 172.21.15.47:0/2855888152 >> 172.21.15.47:0/2855888152 conn(0x7ff24c060e00 msgr2=0x7ff24c0b81a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:45.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.356+0000 7ff251190700 1 -- 172.21.15.47:0/2855888152 shutdown_connections 2022-06-29T15:49:45.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.356+0000 7ff251190700 1 -- 172.21.15.47:0/2855888152 wait complete. 2022-06-29T15:49:45.368 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 42 2022-06-29T15:49:45.376 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:49:45.484 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.476+0000 7fc152799700 1 Processor -- start 2022-06-29T15:49:45.485 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.476+0000 7fc152799700 1 -- start start 2022-06-29T15:49:45.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.476+0000 7fc152799700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc14c0b1fc0 0x7fc14c0b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:45.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.476+0000 7fc152799700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc14c0af440 0x7fc14c0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:45.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.476+0000 7fc152799700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc14c146510 con 0x7fc14c0b25c0 2022-06-29T15:49:45.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.476+0000 7fc152799700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc14c147430 con 0x7fc14c0af440 2022-06-29T15:49:45.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.476+0000 7fc152799700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc14c1467b0 con 0x7fc14c0b1fc0 2022-06-29T15:49:45.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.476+0000 7fc150f96700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc14c0b1fc0 0x7fc14c0b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:45.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.476+0000 7fc150f96700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc14c0b1fc0 0x7fc14c0b29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:48324/0 (socket says 172.21.15.47:48324) 2022-06-29T15:49:45.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.476+0000 7fc150f96700 1 -- 172.21.15.47:0/2923948630 learned_addr learned my addr 172.21.15.47:0/2923948630 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:45.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.476+0000 7fc151f98700 1 --2- 172.21.15.47:0/2923948630 >> v2:172.21.15.64:3300/0 conn(0x7fc14c0af440 0x7fc14c0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:45.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.476+0000 7fc151797700 1 --1- 172.21.15.47:0/2923948630 >> v1:172.21.15.64:6789/0 conn(0x7fc14c0b25c0 0x7fc14c1403b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:44340/0 (socket says 172.21.15.47:44340) 2022-06-29T15:49:45.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.476+0000 7fc150f96700 1 -- 172.21.15.47:0/2923948630 >> v2:172.21.15.64:3300/0 conn(0x7fc14c0af440 msgr2=0x7fc14c0af8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:45.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.476+0000 7fc150f96700 1 --2- 172.21.15.47:0/2923948630 >> v2:172.21.15.64:3300/0 conn(0x7fc14c0af440 0x7fc14c0af8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:45.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.476+0000 7fc150f96700 1 -- 172.21.15.47:0/2923948630 >> v1:172.21.15.64:6789/0 conn(0x7fc14c0b25c0 legacy=0x7fc14c1403b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:45.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.476+0000 7fc150f96700 1 -- 172.21.15.47:0/2923948630 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc14c146a50 con 0x7fc14c0b1fc0 2022-06-29T15:49:45.497 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.476+0000 7fc150f96700 1 --2- 172.21.15.47:0/2923948630 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc14c0b1fc0 0x7fc14c0b29a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fc140002f70 tx=0x7fc140006eb0).ready entity=mon.2 client_cookie=65601f047da9f508 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:45.497 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.476+0000 7fc13ffff700 1 -- 172.21.15.47:0/2923948630 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc14000d570 con 0x7fc14c0b1fc0 2022-06-29T15:49:45.497 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.480+0000 7fc13ffff700 1 -- 172.21.15.47:0/2923948630 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc140000e90 con 0x7fc14c0b1fc0 2022-06-29T15:49:45.497 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.480+0000 7fc13ffff700 1 -- 172.21.15.47:0/2923948630 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc14000d900 con 0x7fc14c0b1fc0 2022-06-29T15:49:45.498 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.480+0000 7fc152799700 1 -- 172.21.15.47:0/2923948630 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc14c0b1fc0 msgr2=0x7fc14c0b29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:45.498 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.480+0000 7fc152799700 1 --2- 172.21.15.47:0/2923948630 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc14c0b1fc0 0x7fc14c0b29a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fc140002f70 tx=0x7fc140006eb0).stop 2022-06-29T15:49:45.498 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.480+0000 7fc152799700 1 -- 172.21.15.47:0/2923948630 shutdown_connections 2022-06-29T15:49:45.498 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.480+0000 7fc152799700 1 --2- 172.21.15.47:0/2923948630 >> v2:172.21.15.64:3300/0 conn(0x7fc14c0af440 0x7fc14c0af8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:45.498 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.480+0000 7fc152799700 1 --2- 172.21.15.47:0/2923948630 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc14c0b1fc0 0x7fc14c0b29a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:45.499 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.480+0000 7fc152799700 1 -- 172.21.15.47:0/2923948630 >> 172.21.15.47:0/2923948630 conn(0x7fc14c062df0 msgr2=0x7fc14c0631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:45.499 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.480+0000 7fc152799700 1 -- 172.21.15.47:0/2923948630 shutdown_connections 2022-06-29T15:49:45.499 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.480+0000 7fc152799700 1 -- 172.21.15.47:0/2923948630 wait complete. 2022-06-29T15:49:45.499 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.480+0000 7fc152799700 1 Processor -- start 2022-06-29T15:49:45.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.480+0000 7fc152799700 1 -- start start 2022-06-29T15:49:45.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.480+0000 7fc152799700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc14c0af440 0x7fc14c0b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:45.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.480+0000 7fc152799700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc14c0b1fc0 0x7fc14c13f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:45.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.480+0000 7fc152799700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc14c0b25c0 0x7fc14c13fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:45.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.480+0000 7fc152799700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc14c0abe00 con 0x7fc14c0af440 2022-06-29T15:49:45.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.480+0000 7fc152799700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc14c0ac0a0 con 0x7fc14c0b25c0 2022-06-29T15:49:45.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.480+0000 7fc152799700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc14c0ac340 con 0x7fc14c0b1fc0 2022-06-29T15:49:45.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.480+0000 7fc151797700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc14c0af440 0x7fc14c0b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:45.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.480+0000 7fc151797700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc14c0af440 0x7fc14c0b4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51228/0 (socket says 172.21.15.47:51228) 2022-06-29T15:49:45.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.480+0000 7fc151797700 1 -- 172.21.15.47:0/207925333 learned_addr learned my addr 172.21.15.47:0/207925333 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:45.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.480+0000 7fc150f96700 1 --2- 172.21.15.47:0/207925333 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc14c0b1fc0 0x7fc14c13f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:45.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.480+0000 7fc151f98700 1 --2- 172.21.15.47:0/207925333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc14c0b25c0 0x7fc14c13fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:45.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.484+0000 7fc151797700 1 -- 172.21.15.47:0/207925333 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc14c0b1fc0 msgr2=0x7fc14c13f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:45.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.484+0000 7fc151797700 1 --2- 172.21.15.47:0/207925333 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc14c0b1fc0 0x7fc14c13f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:45.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.484+0000 7fc151797700 1 -- 172.21.15.47:0/207925333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc14c0b25c0 msgr2=0x7fc14c13fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:45.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.484+0000 7fc151797700 1 --2- 172.21.15.47:0/207925333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc14c0b25c0 0x7fc14c13fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:45.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.484+0000 7fc151797700 1 -- 172.21.15.47:0/207925333 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc14000b1e0 con 0x7fc14c0af440 2022-06-29T15:49:45.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.484+0000 7fc151f98700 1 --2- 172.21.15.47:0/207925333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc14c0b25c0 0x7fc14c13fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:49:45.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.484+0000 7fc151797700 1 --2- 172.21.15.47:0/207925333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc14c0af440 0x7fc14c0b4ab0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 rx=0x7fc14c05c820 tx=0x7fc148007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:45.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.484+0000 7fc13e7fc700 1 -- 172.21.15.47:0/207925333 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc1480070d0 con 0x7fc14c0af440 2022-06-29T15:49:45.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.484+0000 7fc13e7fc700 1 -- 172.21.15.47:0/207925333 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc148007230 con 0x7fc14c0af440 2022-06-29T15:49:45.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.484+0000 7fc13e7fc700 1 -- 172.21.15.47:0/207925333 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc148005550 con 0x7fc14c0af440 2022-06-29T15:49:45.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.484+0000 7fc152799700 1 -- 172.21.15.47:0/207925333 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc14c0ac5e0 con 0x7fc14c0af440 2022-06-29T15:49:45.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.484+0000 7fc152799700 1 -- 172.21.15.47:0/207925333 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc14c1400e0 con 0x7fc14c0af440 2022-06-29T15:49:45.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.484+0000 7fc152799700 1 -- 172.21.15.47:0/207925333 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc14c0573d0 con 0x7fc14c0af440 2022-06-29T15:49:45.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.484+0000 7fc13e7fc700 1 -- 172.21.15.47:0/207925333 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc148010070 con 0x7fc14c0af440 2022-06-29T15:49:45.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.484+0000 7fc13e7fc700 1 --2- 172.21.15.47:0/207925333 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc12c053950 0x7fc12c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:45.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.484+0000 7fc13e7fc700 1 -- 172.21.15.47:0/207925333 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fc14806b360 con 0x7fc14c0af440 2022-06-29T15:49:45.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.484+0000 7fc150f96700 1 --2- 172.21.15.47:0/207925333 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc12c053950 0x7fc12c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:45.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.488+0000 7fc150f96700 1 --2- 172.21.15.47:0/207925333 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc12c053950 0x7fc12c055e10 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7fc14001c000 tx=0x7fc140015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:45.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.496+0000 7fc13e7fc700 1 -- 172.21.15.47:0/207925333 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc148042400 con 0x7fc14c0af440 2022-06-29T15:49:45.667 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.656+0000 7fc152799700 1 -- 172.21.15.47:0/207925333 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc14c0b1b90 con 0x7fc14c0af440 2022-06-29T15:49:45.668 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.660+0000 7fc13e7fc700 1 -- 172.21.15.47:0/207925333 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+1562 (secure 0 0 0) 0x7fc148018020 con 0x7fc14c0af440 2022-06-29T15:49:45.668 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:49:45.668 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":42,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6577,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/4286932616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":4286932616},{"type":"v1","addr":"172.21.15.64:6833","nonce":4286932616}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":38},{"gid":6657,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3528102962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3528102962},{"type":"v1","addr":"172.21.15.47:6835","nonce":3528102962}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":38},{"gid":6894,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/3564259639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":3564259639},{"type":"v1","addr":"172.21.15.47:6837","nonce":3564259639}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":42}],"filesystems":[]} 2022-06-29T15:49:45.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.660+0000 7fc152799700 1 -- 172.21.15.47:0/207925333 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc12c053950 msgr2=0x7fc12c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:45.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.660+0000 7fc152799700 1 --2- 172.21.15.47:0/207925333 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc12c053950 0x7fc12c055e10 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7fc14001c000 tx=0x7fc140015000).stop 2022-06-29T15:49:45.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.660+0000 7fc152799700 1 -- 172.21.15.47:0/207925333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc14c0af440 msgr2=0x7fc14c0b4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:45.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.660+0000 7fc152799700 1 --2- 172.21.15.47:0/207925333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc14c0af440 0x7fc14c0b4ab0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 rx=0x7fc14c05c820 tx=0x7fc148007c40).stop 2022-06-29T15:49:45.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.660+0000 7fc152799700 1 -- 172.21.15.47:0/207925333 shutdown_connections 2022-06-29T15:49:45.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.660+0000 7fc152799700 1 --2- 172.21.15.47:0/207925333 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc12c053950 0x7fc12c055e10 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:45.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.660+0000 7fc152799700 1 --2- 172.21.15.47:0/207925333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc14c0b25c0 0x7fc14c13fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:45.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.660+0000 7fc152799700 1 --2- 172.21.15.47:0/207925333 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc14c0b1fc0 0x7fc14c13f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:45.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.660+0000 7fc152799700 1 --2- 172.21.15.47:0/207925333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc14c0af440 0x7fc14c0b4ab0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:45.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.660+0000 7fc152799700 1 -- 172.21.15.47:0/207925333 >> 172.21.15.47:0/207925333 conn(0x7fc14c062df0 msgr2=0x7fc14c143520 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:45.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.660+0000 7fc152799700 1 -- 172.21.15.47:0/207925333 shutdown_connections 2022-06-29T15:49:45.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.660+0000 7fc152799700 1 -- 172.21.15.47:0/207925333 wait complete. 2022-06-29T15:49:45.673 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 42 2022-06-29T15:49:45.681 INFO:tasks.ceph.mds.a:Restarting daemon 2022-06-29T15:49:45.681 INFO:tasks.ceph.mds.a:Stopping old one... 2022-06-29T15:49:45.682 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-06-29T15:49:45.682 INFO:teuthology.orchestra.run:waiting for 300 2022-06-29T15:49:45.682 INFO:tasks.ceph.mds.c:Restarting daemon 2022-06-29T15:49:45.683 INFO:tasks.ceph.mds.c:Stopping old one... 2022-06-29T15:49:45.683 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2022-06-29T15:49:45.683 INFO:teuthology.orchestra.run:waiting for 300 2022-06-29T15:49:45.684 INFO:tasks.ceph.mds.b:Restarting daemon 2022-06-29T15:49:45.684 INFO:tasks.ceph.mds.b:Stopping old one... 2022-06-29T15:49:45.684 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-06-29T15:49:45.684 INFO:teuthology.orchestra.run:waiting for 300 2022-06-29T15:49:45.695 INFO:tasks.ceph.mds.a:Stopped 2022-06-29T15:49:45.696 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-06-29T15:49:45.697 INFO:tasks.ceph.mds.c:Stopped 2022-06-29T15:49:45.698 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2022-06-29T15:49:45.699 INFO:tasks.ceph.mds.a:Started 2022-06-29T15:49:45.700 DEBUG:teuthology.parallel:result is None 2022-06-29T15:49:45.701 INFO:tasks.ceph.mds.c:Started 2022-06-29T15:49:45.701 DEBUG:teuthology.parallel:result is None 2022-06-29T15:49:45.702 INFO:tasks.ceph.mds.b:Stopped 2022-06-29T15:49:45.702 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-06-29T15:49:45.705 INFO:tasks.ceph.mds.b:Started 2022-06-29T15:49:45.705 DEBUG:teuthology.parallel:result is None 2022-06-29T15:49:45.706 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-06-29T15:49:45.760 INFO:tasks.ceph.mds.a.smithi047.stdout:starting mds.a at 2022-06-29T15:49:45.781 INFO:tasks.ceph.mds.b.smithi064.stdout:starting mds.b at 2022-06-29T15:49:45.787 INFO:tasks.ceph.mds.c.smithi047.stdout:starting mds.c at 2022-06-29T15:49:45.819 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.812+0000 7fa73dd16700 1 Processor -- start 2022-06-29T15:49:45.820 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.812+0000 7fa73dd16700 1 -- start start 2022-06-29T15:49:45.822 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.812+0000 7fa73dd16700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa7380b58f0 0x7fa7380af1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:45.822 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.812+0000 7fa73dd16700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa7380b10b0 0x7fa7380b1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:45.822 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.812+0000 7fa73dd16700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa738060bd0 0x7fa7381466d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:45.822 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.812+0000 7fa73dd16700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7380b5cd0 con 0x7fa7380b58f0 2022-06-29T15:49:45.823 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.812+0000 7fa73dd16700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7380ae970 con 0x7fa7380b10b0 2022-06-29T15:49:45.823 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.812+0000 7fa73dd16700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7380b1630 con 0x7fa738060bd0 2022-06-29T15:49:45.823 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.812+0000 7fa73cd14700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa7380b58f0 0x7fa7380af1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:45.823 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.812+0000 7fa73cd14700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa7380b58f0 0x7fa7380af1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51270/0 (socket says 172.21.15.47:51270) 2022-06-29T15:49:45.824 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.812+0000 7fa73cd14700 1 -- 172.21.15.47:0/4158561687 learned_addr learned my addr 172.21.15.47:0/4158561687 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:45.825 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.812+0000 7fa737fff700 1 --2- 172.21.15.47:0/4158561687 >> v2:172.21.15.64:3300/0 conn(0x7fa7380b10b0 0x7fa7380b1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:45.825 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.812+0000 7fa73d515700 1 --2- 172.21.15.47:0/4158561687 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa738060bd0 0x7fa7381466d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:45.825 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.812+0000 7fa73cd14700 1 -- 172.21.15.47:0/4158561687 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa738060bd0 msgr2=0x7fa7381466d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:45.826 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.812+0000 7fa73cd14700 1 --2- 172.21.15.47:0/4158561687 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa738060bd0 0x7fa7381466d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:45.826 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.812+0000 7fa73cd14700 1 -- 172.21.15.47:0/4158561687 >> v2:172.21.15.64:3300/0 conn(0x7fa7380b10b0 msgr2=0x7fa7380b1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:45.826 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.812+0000 7fa73cd14700 1 --2- 172.21.15.47:0/4158561687 >> v2:172.21.15.64:3300/0 conn(0x7fa7380b10b0 0x7fa7380b1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:45.826 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.812+0000 7fa73cd14700 1 -- 172.21.15.47:0/4158561687 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7380b18d0 con 0x7fa7380b58f0 2022-06-29T15:49:45.827 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.812+0000 7fa73cd14700 1 --2- 172.21.15.47:0/4158561687 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7380b58f0 0x7fa7380af1a0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 rx=0x7fa72c002f70 tx=0x7fa72c006eb0).ready entity=mon.0 client_cookie=95918f4d386a79de server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:45.827 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.812+0000 7fa7377fe700 1 -- 172.21.15.47:0/4158561687 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa72c00d570 con 0x7fa7380b58f0 2022-06-29T15:49:45.827 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.812+0000 7fa7377fe700 1 -- 172.21.15.47:0/4158561687 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa72c000e90 con 0x7fa7380b58f0 2022-06-29T15:49:45.827 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.812+0000 7fa7377fe700 1 -- 172.21.15.47:0/4158561687 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa72c00d900 con 0x7fa7380b58f0 2022-06-29T15:49:45.831 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.812+0000 7fa73dd16700 1 -- 172.21.15.47:0/4158561687 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7380b58f0 msgr2=0x7fa7380af1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:45.831 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.812+0000 7fa73dd16700 1 --2- 172.21.15.47:0/4158561687 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7380b58f0 0x7fa7380af1a0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 rx=0x7fa72c002f70 tx=0x7fa72c006eb0).stop 2022-06-29T15:49:45.832 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.812+0000 7fa73dd16700 1 -- 172.21.15.47:0/4158561687 shutdown_connections 2022-06-29T15:49:45.832 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.812+0000 7fa73dd16700 1 --2- 172.21.15.47:0/4158561687 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa738060bd0 0x7fa7381466d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:45.832 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.812+0000 7fa73dd16700 1 --2- 172.21.15.47:0/4158561687 >> v2:172.21.15.64:3300/0 conn(0x7fa7380b10b0 0x7fa7380b1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:45.833 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.812+0000 7fa73dd16700 1 --2- 172.21.15.47:0/4158561687 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7380b58f0 0x7fa7380af1a0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:45.833 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.812+0000 7fa73dd16700 1 -- 172.21.15.47:0/4158561687 >> 172.21.15.47:0/4158561687 conn(0x7fa738062560 msgr2=0x7fa738062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:45.833 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.812+0000 7fa73dd16700 1 -- 172.21.15.47:0/4158561687 shutdown_connections 2022-06-29T15:49:45.834 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.812+0000 7fa73dd16700 1 -- 172.21.15.47:0/4158561687 wait complete. 2022-06-29T15:49:45.834 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.816+0000 7fa73dd16700 1 Processor -- start 2022-06-29T15:49:45.834 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.816+0000 7fa73dd16700 1 -- start start 2022-06-29T15:49:45.835 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.816+0000 7fa73dd16700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa738060bd0 0x7fa738152110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:45.835 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.816+0000 7fa73dd16700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7380b10b0 0x7fa7380b0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:45.836 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.816+0000 7fa73dd16700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7380b58f0 0x7fa738155ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:45.836 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.816+0000 7fa73dd16700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7380b5330 con 0x7fa7380b10b0 2022-06-29T15:49:45.836 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.816+0000 7fa73dd16700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa738152600 con 0x7fa7380b58f0 2022-06-29T15:49:45.837 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.816+0000 7fa73dd16700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7381557f0 con 0x7fa738060bd0 2022-06-29T15:49:45.837 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.816+0000 7fa737fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7380b10b0 0x7fa7380b0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:45.837 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.816+0000 7fa737fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7380b10b0 0x7fa7380b0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51274/0 (socket says 172.21.15.47:51274) 2022-06-29T15:49:45.837 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.816+0000 7fa737fff700 1 -- 172.21.15.47:0/705906534 learned_addr learned my addr 172.21.15.47:0/705906534 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:45.838 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.816+0000 7fa73cd14700 1 --2- 172.21.15.47:0/705906534 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa738060bd0 0x7fa738152110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:45.838 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.816+0000 7fa73d515700 1 --2- 172.21.15.47:0/705906534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7380b58f0 0x7fa738155ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:45.839 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.816+0000 7fa737fff700 1 -- 172.21.15.47:0/705906534 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa738060bd0 msgr2=0x7fa738152110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:45.839 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.816+0000 7fa737fff700 1 --2- 172.21.15.47:0/705906534 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa738060bd0 0x7fa738152110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:45.840 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.816+0000 7fa737fff700 1 -- 172.21.15.47:0/705906534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7380b58f0 msgr2=0x7fa738155ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:45.840 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.816+0000 7fa737fff700 1 --2- 172.21.15.47:0/705906534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7380b58f0 0x7fa738155ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:45.840 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.816+0000 7fa737fff700 1 -- 172.21.15.47:0/705906534 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa72c00b1e0 con 0x7fa7380b10b0 2022-06-29T15:49:45.840 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.816+0000 7fa737fff700 1 --2- 172.21.15.47:0/705906534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7380b10b0 0x7fa7380b0450 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 rx=0x7fa72400c000 tx=0x7fa724006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:45.841 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.816+0000 7fa735ffb700 1 -- 172.21.15.47:0/705906534 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa72400b3b0 con 0x7fa7380b10b0 2022-06-29T15:49:45.841 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.816+0000 7fa735ffb700 1 -- 172.21.15.47:0/705906534 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa724010590 con 0x7fa7380b10b0 2022-06-29T15:49:45.841 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.816+0000 7fa73dd16700 1 -- 172.21.15.47:0/705906534 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7380a9650 con 0x7fa7380b10b0 2022-06-29T15:49:45.841 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.816+0000 7fa735ffb700 1 -- 172.21.15.47:0/705906534 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa7240108f0 con 0x7fa7380b10b0 2022-06-29T15:49:45.842 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.816+0000 7fa73dd16700 1 -- 172.21.15.47:0/705906534 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa738155a50 con 0x7fa7380b10b0 2022-06-29T15:49:45.842 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.820+0000 7fa735ffb700 1 -- 172.21.15.47:0/705906534 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa724010a50 con 0x7fa7380b10b0 2022-06-29T15:49:45.842 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.820+0000 7fa73dd16700 1 -- 172.21.15.47:0/705906534 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7380573d0 con 0x7fa7380b10b0 2022-06-29T15:49:45.843 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.820+0000 7fa735ffb700 1 --2- 172.21.15.47:0/705906534 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa718053a70 0x7fa718055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:45.843 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.820+0000 7fa735ffb700 1 -- 172.21.15.47:0/705906534 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fa72406be90 con 0x7fa7380b10b0 2022-06-29T15:49:45.843 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.820+0000 7fa73cd14700 1 --2- 172.21.15.47:0/705906534 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa718053a70 0x7fa718055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:45.843 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.820+0000 7fa73cd14700 1 --2- 172.21.15.47:0/705906534 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa718053a70 0x7fa718055f30 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7fa72c01c000 tx=0x7fa72c015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:45.845 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.828+0000 7fa735ffb700 1 -- 172.21.15.47:0/705906534 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa724042da0 con 0x7fa7380b10b0 2022-06-29T15:49:45.975 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:45.964+0000 7fa73dd16700 1 -- 172.21.15.47:0/705906534 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7fa738069e20 con 0x7fa7380b10b0 2022-06-29T15:49:46.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.800+0000 7fa735ffb700 1 -- 172.21.15.47:0/705906534 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v97) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa7240463c0 con 0x7fa7380b10b0 2022-06-29T15:49:46.812 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.804+0000 7fa73dd16700 1 -- 172.21.15.47:0/705906534 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa718053a70 msgr2=0x7fa718055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:46.812 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.804+0000 7fa73dd16700 1 --2- 172.21.15.47:0/705906534 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa718053a70 0x7fa718055f30 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7fa72c01c000 tx=0x7fa72c015000).stop 2022-06-29T15:49:46.812 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.804+0000 7fa73dd16700 1 -- 172.21.15.47:0/705906534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7380b10b0 msgr2=0x7fa7380b0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:46.813 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.804+0000 7fa73dd16700 1 --2- 172.21.15.47:0/705906534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7380b10b0 0x7fa7380b0450 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 rx=0x7fa72400c000 tx=0x7fa724006df0).stop 2022-06-29T15:49:46.813 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.804+0000 7fa73dd16700 1 -- 172.21.15.47:0/705906534 shutdown_connections 2022-06-29T15:49:46.813 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.804+0000 7fa73dd16700 1 --2- 172.21.15.47:0/705906534 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa718053a70 0x7fa718055f30 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:46.813 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.804+0000 7fa73dd16700 1 --2- 172.21.15.47:0/705906534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7380b58f0 0x7fa738155ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:46.813 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.804+0000 7fa73dd16700 1 --2- 172.21.15.47:0/705906534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7380b10b0 0x7fa7380b0450 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:46.814 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.804+0000 7fa73dd16700 1 --2- 172.21.15.47:0/705906534 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa738060bd0 0x7fa738152110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:46.814 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.804+0000 7fa73dd16700 1 -- 172.21.15.47:0/705906534 >> 172.21.15.47:0/705906534 conn(0x7fa738062560 msgr2=0x7fa7380b4150 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:46.814 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.804+0000 7fa73dd16700 1 -- 172.21.15.47:0/705906534 shutdown_connections 2022-06-29T15:49:46.814 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.804+0000 7fa73dd16700 1 -- 172.21.15.47:0/705906534 wait complete. 2022-06-29T15:49:46.815 INFO:teuthology.orchestra.run.smithi047.stderr: removed all blacklist entries 2022-06-29T15:49:46.828 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2022-06-29T15:49:46.828 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-06-29T15:49:46.940 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.932+0000 7f508becf700 1 Processor -- start 2022-06-29T15:49:46.941 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.932+0000 7f508becf700 1 -- start start 2022-06-29T15:49:46.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.932+0000 7f508becf700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f50840b1260 0x7f508405ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:46.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.932+0000 7f508becf700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50840b0740 con 0x7f50840b2120 2022-06-29T15:49:46.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.932+0000 7f508becf700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50840b0970 con 0x7f508405f0b0 2022-06-29T15:49:46.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.932+0000 7f508becf700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5084147460 con 0x7f50840b1260 2022-06-29T15:49:46.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.932+0000 7f5089c6b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f50840b2120 0x7f5084147af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36840/0 (socket says 172.21.15.47:36840) 2022-06-29T15:49:46.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.932+0000 7f5089c6b700 1 -- 172.21.15.47:0/363215737 learned_addr learned my addr 172.21.15.47:0/363215737 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:49:46.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.932+0000 7f508a46c700 1 --2- 172.21.15.47:0/363215737 >> v2:172.21.15.64:3300/0 conn(0x7f50840b1260 0x7f508405ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:46.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.932+0000 7f5088c69700 1 -- 172.21.15.47:0/363215737 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1453036749 0 0) 0x7f50840b0740 con 0x7f50840b2120 2022-06-29T15:49:46.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.932+0000 7f5088c69700 1 -- 172.21.15.47:0/363215737 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5070003260 con 0x7f50840b2120 2022-06-29T15:49:46.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.932+0000 7f5088c69700 1 -- 172.21.15.47:0/363215737 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2508848225 0 0) 0x7f50840b0970 con 0x7f508405f0b0 2022-06-29T15:49:46.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.932+0000 7f5088c69700 1 -- 172.21.15.47:0/363215737 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50840b0740 con 0x7f508405f0b0 2022-06-29T15:49:46.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.932+0000 7f5088c69700 1 -- 172.21.15.47:0/363215737 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1115957626 0 0) 0x7f5070003260 con 0x7f50840b2120 2022-06-29T15:49:46.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.932+0000 7f5088c69700 1 -- 172.21.15.47:0/363215737 >> v2:172.21.15.64:3300/0 conn(0x7f50840b1260 msgr2=0x7f508405ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:46.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.932+0000 7f5088c69700 1 --2- 172.21.15.47:0/363215737 >> v2:172.21.15.64:3300/0 conn(0x7f50840b1260 0x7f508405ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:46.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.932+0000 7f5088c69700 1 -- 172.21.15.47:0/363215737 >> v1:172.21.15.64:6789/0 conn(0x7f508405f0b0 legacy=0x7f508405f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:46.950 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.932+0000 7f5088c69700 1 -- 172.21.15.47:0/363215737 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50840b0c10 con 0x7f50840b2120 2022-06-29T15:49:46.950 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.932+0000 7f5088c69700 1 -- 172.21.15.47:0/363215737 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7f50800019e0 con 0x7f50840b2120 2022-06-29T15:49:46.950 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.932+0000 7f5088c69700 1 -- 172.21.15.47:0/363215737 >> v1:172.21.15.47:6789/0 conn(0x7f50840b2120 legacy=0x7f5084147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:46.950 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.932+0000 7f5088c69700 1 --2- 172.21.15.47:0/363215737 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5070003910 0x7f5070003d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:46.951 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.932+0000 7f5088c69700 1 --2- 172.21.15.47:0/363215737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5070004500 0x7f5070004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:46.951 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.932+0000 7f5088c69700 1 --2- 172.21.15.47:0/363215737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5070004e70 0x7f507000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:46.951 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.932+0000 7f5088c69700 1 -- 172.21.15.47:0/363215737 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507000bcf0 con 0x7f5070004e70 2022-06-29T15:49:46.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.932+0000 7f5088c69700 1 -- 172.21.15.47:0/363215737 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507000be60 con 0x7f5070004500 2022-06-29T15:49:46.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.932+0000 7f5088c69700 1 -- 172.21.15.47:0/363215737 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f507000c020 con 0x7f5070003910 2022-06-29T15:49:46.953 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.932+0000 7f508a46c700 1 --2- 172.21.15.47:0/363215737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5070004e70 0x7f507000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:46.953 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.932+0000 7f5089c6b700 1 --2- 172.21.15.47:0/363215737 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5070003910 0x7f5070003d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:46.954 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.932+0000 7f508946a700 1 --2- 172.21.15.47:0/363215737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5070004500 0x7f5070004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:46.954 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.932+0000 7f508a46c700 1 -- 172.21.15.47:0/363215737 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5070003910 msgr2=0x7f5070003d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:46.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.932+0000 7f508a46c700 1 --2- 172.21.15.47:0/363215737 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5070003910 0x7f5070003d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:46.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.932+0000 7f508a46c700 1 -- 172.21.15.47:0/363215737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5070004500 msgr2=0x7f5070004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:46.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.932+0000 7f508a46c700 1 --2- 172.21.15.47:0/363215737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5070004500 0x7f5070004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:46.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.932+0000 7f508a46c700 1 -- 172.21.15.47:0/363215737 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5070003260 con 0x7f5070004e70 2022-06-29T15:49:46.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.932+0000 7f508946a700 1 --2- 172.21.15.47:0/363215737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5070004500 0x7f5070004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:49:46.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.936+0000 7f508a46c700 1 -- 172.21.15.47:0/363215737 reap_dead start 2022-06-29T15:49:46.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.936+0000 7f508a46c700 1 --2- 172.21.15.47:0/363215737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5070004e70 0x7f507000b750 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 rx=0x7f507400c000 tx=0x7f5074006df0).ready entity=mon.0 client_cookie=c2a0411743a4edc6 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:46.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.936+0000 7f5088c69700 1 -- 172.21.15.47:0/363215737 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5074016070 con 0x7f5070004e70 2022-06-29T15:49:46.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.936+0000 7f5088c69700 1 -- 172.21.15.47:0/363215737 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f507400e5f0 con 0x7f5070004e70 2022-06-29T15:49:46.958 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.936+0000 7f5088c69700 1 -- 172.21.15.47:0/363215737 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f507400e950 con 0x7f5070004e70 2022-06-29T15:49:46.958 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.936+0000 7f508becf700 1 -- 172.21.15.47:0/363215737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5070004e70 msgr2=0x7f507000b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:46.959 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.936+0000 7f508becf700 1 --2- 172.21.15.47:0/363215737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5070004e70 0x7f507000b750 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 rx=0x7f507400c000 tx=0x7f5074006df0).stop 2022-06-29T15:49:46.959 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.936+0000 7f508becf700 1 -- 172.21.15.47:0/363215737 shutdown_connections 2022-06-29T15:49:46.960 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.936+0000 7f508becf700 1 --2- 172.21.15.47:0/363215737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5070004e70 0x7f507000b750 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:46.960 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.936+0000 7f508becf700 1 -- 172.21.15.47:0/363215737 >> 172.21.15.47:0/363215737 conn(0x7f50840626f0 msgr2=0x7f5084062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:46.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.936+0000 7f508becf700 1 -- 172.21.15.47:0/363215737 shutdown_connections 2022-06-29T15:49:46.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.936+0000 7f508becf700 1 -- 172.21.15.47:0/363215737 wait complete. 2022-06-29T15:49:46.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.936+0000 7f508becf700 1 Processor -- start 2022-06-29T15:49:46.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.936+0000 7f508becf700 1 -- start start 2022-06-29T15:49:46.966 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.936+0000 7f508becf700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5070004e70 0x7f50841447c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:46.966 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.936+0000 7f508becf700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f508413c210 0x7f5084143ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:46.967 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.936+0000 7f508becf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5084143fd0 0x7f50841402e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:46.967 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.936+0000 7f508becf700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50841432a0 con 0x7f508413c210 2022-06-29T15:49:46.967 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.936+0000 7f508becf700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508413ce70 con 0x7f5084143fd0 2022-06-29T15:49:46.968 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.936+0000 7f508becf700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f508413d110 con 0x7f5070004e70 2022-06-29T15:49:46.968 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.940+0000 7f508946a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f508413c210 0x7f5084143ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:46.968 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.940+0000 7f508946a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f508413c210 0x7f5084143ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51298/0 (socket says 172.21.15.47:51298) 2022-06-29T15:49:46.969 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.940+0000 7f508946a700 1 -- 172.21.15.47:0/4276090666 learned_addr learned my addr 172.21.15.47:0/4276090666 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:46.969 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.940+0000 7f508a46c700 1 --2- 172.21.15.47:0/4276090666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5084143fd0 0x7f50841402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:46.969 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.940+0000 7f5089c6b700 1 --2- 172.21.15.47:0/4276090666 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5070004e70 0x7f50841447c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:46.969 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.940+0000 7f508946a700 1 -- 172.21.15.47:0/4276090666 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5070004e70 msgr2=0x7f50841447c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:46.970 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.940+0000 7f508946a700 1 --2- 172.21.15.47:0/4276090666 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5070004e70 0x7f50841447c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:46.970 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.940+0000 7f508946a700 1 -- 172.21.15.47:0/4276090666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5084143fd0 msgr2=0x7f50841402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:46.970 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.940+0000 7f508946a700 1 --2- 172.21.15.47:0/4276090666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5084143fd0 0x7f50841402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:46.971 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.940+0000 7f508946a700 1 -- 172.21.15.47:0/4276090666 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f508413d3b0 con 0x7f508413c210 2022-06-29T15:49:46.971 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.940+0000 7f508946a700 1 --2- 172.21.15.47:0/4276090666 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f508413c210 0x7f5084143ae0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 rx=0x7f507c00b000 tx=0x7f507c009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:46.972 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.940+0000 7f507affd700 1 -- 172.21.15.47:0/4276090666 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f507c00eea0 con 0x7f508413c210 2022-06-29T15:49:46.972 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.940+0000 7f507affd700 1 -- 172.21.15.47:0/4276090666 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f507c010040 con 0x7f508413c210 2022-06-29T15:49:46.972 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.940+0000 7f508becf700 1 -- 172.21.15.47:0/4276090666 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f508413d850 con 0x7f508413c210 2022-06-29T15:49:46.973 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.940+0000 7f507affd700 1 -- 172.21.15.47:0/4276090666 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f507c0157e0 con 0x7f508413c210 2022-06-29T15:49:46.973 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.940+0000 7f508becf700 1 -- 172.21.15.47:0/4276090666 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5084142740 con 0x7f508413c210 2022-06-29T15:49:46.974 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.940+0000 7f508becf700 1 -- 172.21.15.47:0/4276090666 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f508413d540 con 0x7f508413c210 2022-06-29T15:49:46.974 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.940+0000 7f507affd700 1 -- 172.21.15.47:0/4276090666 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f507c015940 con 0x7f508413c210 2022-06-29T15:49:46.974 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.940+0000 7f507affd700 1 --2- 172.21.15.47:0/4276090666 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f50640539e0 0x7f5064055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:46.975 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.940+0000 7f507affd700 1 -- 172.21.15.47:0/4276090666 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f507c06bfb0 con 0x7f508413c210 2022-06-29T15:49:46.975 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.944+0000 7f5089c6b700 1 --2- 172.21.15.47:0/4276090666 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f50640539e0 0x7f5064055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:46.975 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.944+0000 7f5089c6b700 1 --2- 172.21.15.47:0/4276090666 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f50640539e0 0x7f5064055ea0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7f5080005000 tx=0x7f50840b0da0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:46.976 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:46.948+0000 7f507affd700 1 -- 172.21.15.47:0/4276090666 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f507c043050 con 0x7f508413c210 2022-06-29T15:49:47.123 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.116+0000 7f508becf700 1 -- 172.21.15.47:0/4276090666 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f50840573d0 con 0x7f508413c210 2022-06-29T15:49:47.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.116+0000 7f507affd700 1 -- 172.21.15.47:0/4276090666 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v25) v1 ==== 80+0+4255 (secure 0 0 0) 0x7f507c046670 con 0x7f508413c210 2022-06-29T15:49:47.124 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:49:47.125 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2022-06-29T15:49:47.125 INFO:teuthology.orchestra.run.smithi047.stdout: "auth_dump": [ 2022-06-29T15:49:47.125 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:47.125 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.a", 2022-06-29T15:49:47.126 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxiQJFFMxAA9Tv7FJD9u/BKvbkuKld6Kg==", 2022-06-29T15:49:47.126 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:49:47.126 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:49:47.126 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:49:47.127 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:49:47.127 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:49:47.127 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:47.127 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:47.127 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:47.128 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.b", 2022-06-29T15:49:47.128 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC2crxibt6FARAAEf2bwXKFhXKtqr1nN0rdXQ==", 2022-06-29T15:49:47.128 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:49:47.128 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:49:47.129 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:49:47.129 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:49:47.129 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:49:47.129 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:47.129 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:47.130 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:47.130 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.c", 2022-06-29T15:49:47.130 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxi/4ZMORAANpLbgTm3ehqi80USc3swGg==", 2022-06-29T15:49:47.130 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:49:47.131 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:49:47.131 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:49:47.131 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:49:47.131 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:49:47.131 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:47.132 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:47.132 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:47.132 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.0", 2022-06-29T15:49:47.132 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC2crxiY0YsMxAAZivb+CxL/w7vZwQv37Y4oQ==", 2022-06-29T15:49:47.133 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:49:47.133 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:49:47.133 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:49:47.133 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:49:47.133 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:47.134 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:47.134 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:47.134 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.1", 2022-06-29T15:49:47.134 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC5crxitLkKFxAA8+++LUUKLiO9SAuAfYgT5g==", 2022-06-29T15:49:47.135 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:49:47.135 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:49:47.135 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:49:47.135 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:49:47.135 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:47.136 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:47.136 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:47.136 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.2", 2022-06-29T15:49:47.136 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC7crxi1oM9NxAA2FfLdNt32YPwlyNdGQpaQA==", 2022-06-29T15:49:47.136 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:49:47.137 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:49:47.137 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:49:47.137 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:49:47.137 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:47.138 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:47.138 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:47.138 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.3", 2022-06-29T15:49:47.138 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC+crxiTsBgGhAA+gLBBDaskPbmRNWzgKwcHA==", 2022-06-29T15:49:47.139 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:49:47.139 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:49:47.139 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:49:47.139 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:49:47.139 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:47.140 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:47.140 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:47.140 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.4", 2022-06-29T15:49:47.140 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDDcrxivJebIhAAfwWJa7tK4hamkk1/1CSSfw==", 2022-06-29T15:49:47.141 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:49:47.141 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:49:47.141 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:49:47.141 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:49:47.141 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:47.142 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:47.142 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:47.142 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.5", 2022-06-29T15:49:47.142 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDGcrxijuozBhAAt+Dx3wH3WKp3DrbCbhmh8Q==", 2022-06-29T15:49:47.143 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:49:47.143 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:49:47.143 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:49:47.143 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:49:47.143 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:47.144 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:47.144 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:47.144 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.6", 2022-06-29T15:49:47.144 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDIcrxi91kXJxAA9JX7n98P394bD41Q9LUdvQ==", 2022-06-29T15:49:47.144 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:49:47.145 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:49:47.145 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:49:47.145 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:49:47.145 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:47.146 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:47.146 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:47.146 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.7", 2022-06-29T15:49:47.146 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDLcrxiHj0ZDRAASy6fAix8TmEZbAGtuFCfFg==", 2022-06-29T15:49:47.146 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:49:47.147 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:49:47.147 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:49:47.147 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:49:47.147 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:47.148 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:47.148 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:47.148 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.0", 2022-06-29T15:49:47.148 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC2crxiVDlfBxAAezkUfg4e2g0Rfakaa/Ek1Q==", 2022-06-29T15:49:47.148 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:49:47.149 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:49:47.149 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow r", 2022-06-29T15:49:47.149 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-29T15:49:47.149 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:47.150 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:47.150 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:47.150 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.1", 2022-06-29T15:49:47.150 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC2crxisXJ1CxAAhl07oF7AW66NdzfvvZTJmA==", 2022-06-29T15:49:47.150 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:49:47.151 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:49:47.151 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow r", 2022-06-29T15:49:47.151 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-29T15:49:47.151 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:47.151 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:47.152 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:47.152 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.admin", 2022-06-29T15:49:47.152 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxiW7HSFRAAhYqoGU9GFqj2HfUVGValSA==", 2022-06-29T15:49:47.152 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:49:47.153 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2022-06-29T15:49:47.153 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:49:47.153 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:49:47.153 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:49:47.153 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:47.154 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:47.154 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:47.154 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.x", 2022-06-29T15:49:47.154 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxidmuIMBAAcpNlYVu5+icCHg+yIjcKXw==", 2022-06-29T15:49:47.155 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:49:47.155 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2022-06-29T15:49:47.155 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2022-06-29T15:49:47.155 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:49:47.155 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:47.156 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:47.156 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:47.156 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.y", 2022-06-29T15:49:47.156 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxisSMFLBAAv03r4OoPjOP9nI9G/LK0iw==", 2022-06-29T15:49:47.156 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:49:47.157 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2022-06-29T15:49:47.157 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2022-06-29T15:49:47.157 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:49:47.157 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:47.158 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:47.158 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:49:47.158 INFO:teuthology.orchestra.run.smithi047.stdout:} 2022-06-29T15:49:47.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.116+0000 7f508becf700 1 -- 172.21.15.47:0/4276090666 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f50640539e0 msgr2=0x7f5064055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:47.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.116+0000 7f508becf700 1 --2- 172.21.15.47:0/4276090666 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f50640539e0 0x7f5064055ea0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7f5080005000 tx=0x7f50840b0da0).stop 2022-06-29T15:49:47.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.116+0000 7f508becf700 1 -- 172.21.15.47:0/4276090666 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f508413c210 msgr2=0x7f5084143ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:47.161 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.116+0000 7f508becf700 1 --2- 172.21.15.47:0/4276090666 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f508413c210 0x7f5084143ae0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 rx=0x7f507c00b000 tx=0x7f507c009040).stop 2022-06-29T15:49:47.161 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.120+0000 7f508becf700 1 -- 172.21.15.47:0/4276090666 shutdown_connections 2022-06-29T15:49:47.161 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.120+0000 7f508becf700 1 --2- 172.21.15.47:0/4276090666 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f50640539e0 0x7f5064055ea0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:47.161 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.120+0000 7f508becf700 1 --2- 172.21.15.47:0/4276090666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5084143fd0 0x7f50841402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:47.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.120+0000 7f508becf700 1 --2- 172.21.15.47:0/4276090666 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f508413c210 0x7f5084143ae0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:47.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.120+0000 7f508becf700 1 --2- 172.21.15.47:0/4276090666 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5070004e70 0x7f50841447c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:47.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.120+0000 7f508becf700 1 -- 172.21.15.47:0/4276090666 >> 172.21.15.47:0/4276090666 conn(0x7f50840626f0 msgr2=0x7f50840ab650 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:47.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.120+0000 7f508becf700 1 -- 172.21.15.47:0/4276090666 shutdown_connections 2022-06-29T15:49:47.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.120+0000 7f508becf700 1 -- 172.21.15.47:0/4276090666 wait complete. 2022-06-29T15:49:47.163 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:49:47.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.268+0000 7f11ff59e700 1 Processor -- start 2022-06-29T15:49:47.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.268+0000 7f11ff59e700 1 -- start start 2022-06-29T15:49:47.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.268+0000 7f11ff59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f12000b7860 0x7f12000af160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:47.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.268+0000 7f11ff59e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1200146890 0x7f12000af650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:47.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.268+0000 7f11ff59e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f12000b2d00 0x7f12000b0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:47.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.268+0000 7f11ff59e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12001477c0 con 0x7f1200146890 2022-06-29T15:49:47.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.268+0000 7f11ff59e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12000b3900 con 0x7f12000b7860 2022-06-29T15:49:47.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.268+0000 7f11ff59e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12000b7c40 con 0x7f12000b2d00 2022-06-29T15:49:47.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.268+0000 7f11fdd9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1200146890 0x7f12000af650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:47.282 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.268+0000 7f11fe59c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f12000b7860 0x7f12000af160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:47.282 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.268+0000 7f11fed9d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f12000b2d00 0x7f12000b0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:47.282 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.268+0000 7f11fe59c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f12000b7860 0x7f12000af160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:57008/0 (socket says 172.21.15.47:57008) 2022-06-29T15:49:47.282 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.268+0000 7f11fe59c700 1 -- 172.21.15.47:0/1713954021 learned_addr learned my addr 172.21.15.47:0/1713954021 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:47.283 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.268+0000 7f11fdd9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1200146890 0x7f12000af650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51306/0 (socket says 172.21.15.47:51306) 2022-06-29T15:49:47.283 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.272+0000 7f11fdd9b700 1 -- 172.21.15.47:0/1713954021 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f12000b2d00 msgr2=0x7f12000b0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:47.283 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.272+0000 7f11fdd9b700 1 --2- 172.21.15.47:0/1713954021 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f12000b2d00 0x7f12000b0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:47.283 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.272+0000 7f11fdd9b700 1 -- 172.21.15.47:0/1713954021 >> v2:172.21.15.64:3300/0 conn(0x7f12000b7860 msgr2=0x7f12000af160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:47.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.272+0000 7f11fdd9b700 1 --2- 172.21.15.47:0/1713954021 >> v2:172.21.15.64:3300/0 conn(0x7f12000b7860 0x7f12000af160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:47.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.272+0000 7f11fdd9b700 1 -- 172.21.15.47:0/1713954021 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12000b3660 con 0x7f1200146890 2022-06-29T15:49:47.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.272+0000 7f11fe59c700 1 --2- 172.21.15.47:0/1713954021 >> v2:172.21.15.64:3300/0 conn(0x7f12000b7860 0x7f12000af160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:49:47.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.272+0000 7f11fdd9b700 1 --2- 172.21.15.47:0/1713954021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1200146890 0x7f12000af650 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 rx=0x7f11f4002f70 tx=0x7f11f4006df0).ready entity=mon.0 client_cookie=9d83c182f41e65fb server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:47.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.272+0000 7f11fd59a700 1 -- 172.21.15.47:0/1713954021 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11f400d4a0 con 0x7f1200146890 2022-06-29T15:49:47.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.272+0000 7f11fd59a700 1 -- 172.21.15.47:0/1713954021 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f4000e90 con 0x7f1200146890 2022-06-29T15:49:47.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.272+0000 7f11fd59a700 1 -- 172.21.15.47:0/1713954021 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11f400d7f0 con 0x7f1200146890 2022-06-29T15:49:47.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.272+0000 7f11ff59e700 1 -- 172.21.15.47:0/1713954021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1200146890 msgr2=0x7f12000af650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:47.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.272+0000 7f11ff59e700 1 --2- 172.21.15.47:0/1713954021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1200146890 0x7f12000af650 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 rx=0x7f11f4002f70 tx=0x7f11f4006df0).stop 2022-06-29T15:49:47.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.272+0000 7f11ff59e700 1 -- 172.21.15.47:0/1713954021 shutdown_connections 2022-06-29T15:49:47.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.272+0000 7f11ff59e700 1 --2- 172.21.15.47:0/1713954021 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f12000b2d00 0x7f12000b0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:47.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.272+0000 7f11ff59e700 1 --2- 172.21.15.47:0/1713954021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1200146890 0x7f12000af650 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:47.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.272+0000 7f11ff59e700 1 --2- 172.21.15.47:0/1713954021 >> v2:172.21.15.64:3300/0 conn(0x7f12000b7860 0x7f12000af160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:47.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.272+0000 7f11ff59e700 1 -- 172.21.15.47:0/1713954021 >> 172.21.15.47:0/1713954021 conn(0x7f120005f480 msgr2=0x7f12000647d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:47.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.272+0000 7f11ff59e700 1 -- 172.21.15.47:0/1713954021 shutdown_connections 2022-06-29T15:49:47.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.272+0000 7f11ff59e700 1 -- 172.21.15.47:0/1713954021 wait complete. 2022-06-29T15:49:47.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.272+0000 7f11ff59e700 1 Processor -- start 2022-06-29T15:49:47.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.272+0000 7f11ff59e700 1 -- start start 2022-06-29T15:49:47.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.272+0000 7f11ff59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12000b2d00 0x7f12000b6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:47.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.272+0000 7f11ff59e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f12000b7860 0x7f12001557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:47.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.272+0000 7f11ff59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1200146890 0x7f1200155cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:47.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.272+0000 7f11ff59e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12000b6f80 con 0x7f1200146890 2022-06-29T15:49:47.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.272+0000 7f11ff59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1200149800 con 0x7f12000b2d00 2022-06-29T15:49:47.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.272+0000 7f11ff59e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12001561a0 con 0x7f12000b7860 2022-06-29T15:49:47.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.272+0000 7f11fed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1200146890 0x7f1200155cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:47.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.272+0000 7f11fed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1200146890 0x7f1200155cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51314/0 (socket says 172.21.15.47:51314) 2022-06-29T15:49:47.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.272+0000 7f11fed9d700 1 -- 172.21.15.47:0/4291449253 learned_addr learned my addr 172.21.15.47:0/4291449253 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:47.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.272+0000 7f11fe59c700 1 --2- 172.21.15.47:0/4291449253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12000b2d00 0x7f12000b6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:47.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.272+0000 7f11fdd9b700 1 --2- 172.21.15.47:0/4291449253 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f12000b7860 0x7f12001557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:47.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.276+0000 7f11fed9d700 1 -- 172.21.15.47:0/4291449253 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f12000b7860 msgr2=0x7f12001557c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:47.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.276+0000 7f11fed9d700 1 --2- 172.21.15.47:0/4291449253 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f12000b7860 0x7f12001557c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:47.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.276+0000 7f11fed9d700 1 -- 172.21.15.47:0/4291449253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12000b2d00 msgr2=0x7f12000b6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:47.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.276+0000 7f11fed9d700 1 --2- 172.21.15.47:0/4291449253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12000b2d00 0x7f12000b6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:47.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.276+0000 7f11fed9d700 1 -- 172.21.15.47:0/4291449253 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f000b1e0 con 0x7f1200146890 2022-06-29T15:49:47.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.276+0000 7f11fed9d700 1 --2- 172.21.15.47:0/4291449253 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1200146890 0x7f1200155cb0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 rx=0x7f11f8006e00 tx=0x7f11f80091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:47.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.276+0000 7f11ef7fe700 1 -- 172.21.15.47:0/4291449253 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11f800eeb0 con 0x7f1200146890 2022-06-29T15:49:47.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.276+0000 7f11ef7fe700 1 -- 172.21.15.47:0/4291449253 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f800c850 con 0x7f1200146890 2022-06-29T15:49:47.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.276+0000 7f11ff59e700 1 -- 172.21.15.47:0/4291449253 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11f400b180 con 0x7f1200146890 2022-06-29T15:49:47.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.276+0000 7f11ef7fe700 1 -- 172.21.15.47:0/4291449253 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11f800cbb0 con 0x7f1200146890 2022-06-29T15:49:47.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.276+0000 7f11ff59e700 1 -- 172.21.15.47:0/4291449253 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1200060c50 con 0x7f1200146890 2022-06-29T15:49:47.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.276+0000 7f11ef7fe700 1 -- 172.21.15.47:0/4291449253 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f11f8022070 con 0x7f1200146890 2022-06-29T15:49:47.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.276+0000 7f11ff59e700 1 -- 172.21.15.47:0/4291449253 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12000573d0 con 0x7f1200146890 2022-06-29T15:49:47.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.276+0000 7f11ef7fe700 1 --2- 172.21.15.47:0/4291449253 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f11dc0539e0 0x7f11dc055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:47.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.276+0000 7f11ef7fe700 1 -- 172.21.15.47:0/4291449253 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f11f806c500 con 0x7f1200146890 2022-06-29T15:49:47.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.276+0000 7f11fe59c700 1 --2- 172.21.15.47:0/4291449253 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f11dc0539e0 0x7f11dc055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:47.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.276+0000 7f11fe59c700 1 --2- 172.21.15.47:0/4291449253 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f11dc0539e0 0x7f11dc055ea0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7f11f000b000 tx=0x7f11f0006eb0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:47.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.280+0000 7f11ef7fe700 1 -- 172.21.15.47:0/4291449253 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f11f8043560 con 0x7f1200146890 2022-06-29T15:49:47.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.404+0000 7f11ff59e700 1 -- 172.21.15.47:0/4291449253 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1200068830 con 0x7f1200146890 2022-06-29T15:49:47.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.404+0000 7f11ef7fe700 1 -- 172.21.15.47:0/4291449253 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+13415 (secure 0 0 0) 0x7f11f8046b80 con 0x7f1200146890 2022-06-29T15:49:47.414 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:49:47.415 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":97,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:49:46.796242+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:49:47.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.408+0000 7f11ff59e700 1 -- 172.21.15.47:0/4291449253 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f11dc0539e0 msgr2=0x7f11dc055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:47.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.408+0000 7f11ff59e700 1 --2- 172.21.15.47:0/4291449253 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f11dc0539e0 0x7f11dc055ea0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7f11f000b000 tx=0x7f11f0006eb0).stop 2022-06-29T15:49:47.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.408+0000 7f11ff59e700 1 -- 172.21.15.47:0/4291449253 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1200146890 msgr2=0x7f1200155cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:47.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.408+0000 7f11ff59e700 1 --2- 172.21.15.47:0/4291449253 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1200146890 0x7f1200155cb0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 rx=0x7f11f8006e00 tx=0x7f11f80091d0).stop 2022-06-29T15:49:47.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.408+0000 7f11ff59e700 1 -- 172.21.15.47:0/4291449253 shutdown_connections 2022-06-29T15:49:47.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.408+0000 7f11ff59e700 1 --2- 172.21.15.47:0/4291449253 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1200146890 0x7f1200155cb0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:47.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.408+0000 7f11ff59e700 1 --2- 172.21.15.47:0/4291449253 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f12000b7860 0x7f12001557c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:47.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.408+0000 7f11ff59e700 1 --2- 172.21.15.47:0/4291449253 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f11dc0539e0 0x7f11dc055ea0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:47.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.408+0000 7f11ff59e700 1 --2- 172.21.15.47:0/4291449253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12000b2d00 0x7f12000b6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:47.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.408+0000 7f11ff59e700 1 -- 172.21.15.47:0/4291449253 >> 172.21.15.47:0/4291449253 conn(0x7f120005f480 msgr2=0x7f1200144c60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:47.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.408+0000 7f11ff59e700 1 -- 172.21.15.47:0/4291449253 shutdown_connections 2022-06-29T15:49:47.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.408+0000 7f11ff59e700 1 -- 172.21.15.47:0/4291449253 wait complete. 2022-06-29T15:49:47.428 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-29T15:42:16.971431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-29T15:42:25.290884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-06-29T15:49:47.429 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-29T15:49:47.538 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.528+0000 7f68b24c4700 1 Processor -- start 2022-06-29T15:49:47.539 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.528+0000 7f68b24c4700 1 -- start start 2022-06-29T15:49:47.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.532+0000 7f68b24c4700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f68ac061220 0x7f68ac140070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:47.541 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.532+0000 7f68b24c4700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68ac061600 con 0x7f68ac146720 2022-06-29T15:49:47.541 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.532+0000 7f68b24c4700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68ac146bc0 con 0x7f68ac0b4320 2022-06-29T15:49:47.541 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.532+0000 7f68b24c4700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac146e60 con 0x7f68ac061220 2022-06-29T15:49:47.541 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.532+0000 7f68a3fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f68ac146720 0x7f68ac140560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36872/0 (socket says 172.21.15.47:36872) 2022-06-29T15:49:47.542 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.532+0000 7f68a3fff700 1 -- 172.21.15.47:0/2712058951 learned_addr learned my addr 172.21.15.47:0/2712058951 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:49:47.542 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.532+0000 7f68abfff700 1 --2- 172.21.15.47:0/2712058951 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f68ac061220 0x7f68ac140070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:47.542 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.532+0000 7f68ab7fe700 1 -- 172.21.15.47:0/2712058951 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1884448610 0 0) 0x7f68ac061600 con 0x7f68ac146720 2022-06-29T15:49:47.542 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.532+0000 7f68ab7fe700 1 -- 172.21.15.47:0/2712058951 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f688c003260 con 0x7f68ac146720 2022-06-29T15:49:47.543 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.532+0000 7f68ab7fe700 1 -- 172.21.15.47:0/2712058951 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 768501281 0 0) 0x7f68ac146bc0 con 0x7f68ac0b4320 2022-06-29T15:49:47.543 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.532+0000 7f68ab7fe700 1 -- 172.21.15.47:0/2712058951 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68ac061600 con 0x7f68ac0b4320 2022-06-29T15:49:47.543 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.532+0000 7f68ab7fe700 1 -- 172.21.15.47:0/2712058951 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4121837822 0 0) 0x7f688c003260 con 0x7f68ac146720 2022-06-29T15:49:47.543 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.532+0000 7f68abfff700 1 -- 172.21.15.47:0/2712058951 >> v1:172.21.15.64:6789/0 conn(0x7f68ac0b4320 legacy=0x7f68ac143230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:47.544 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.532+0000 7f68abfff700 1 -- 172.21.15.47:0/2712058951 >> v1:172.21.15.47:6789/0 conn(0x7f68ac146720 legacy=0x7f68ac140560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:47.544 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.532+0000 7f68abfff700 1 -- 172.21.15.47:0/2712058951 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68ac147100 con 0x7f68ac061220 2022-06-29T15:49:47.544 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.532+0000 7f68abfff700 1 --2- 172.21.15.47:0/2712058951 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f68ac061220 0x7f68ac140070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f6894001110 tx=0x7f689c006df0).ready entity=mon.2 client_cookie=9ebd72545446b84c server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:47.546 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.532+0000 7f68ab7fe700 1 -- 172.21.15.47:0/2712058951 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6894001660 con 0x7f68ac061220 2022-06-29T15:49:47.546 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.532+0000 7f68ab7fe700 1 -- 172.21.15.47:0/2712058951 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f689c000e90 con 0x7f68ac061220 2022-06-29T15:49:47.547 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.532+0000 7f68ab7fe700 1 -- 172.21.15.47:0/2712058951 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f689c012460 con 0x7f68ac061220 2022-06-29T15:49:47.547 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.532+0000 7f68b24c4700 1 -- 172.21.15.47:0/2712058951 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f68ac061220 msgr2=0x7f68ac140070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:47.547 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.532+0000 7f68b24c4700 1 --2- 172.21.15.47:0/2712058951 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f68ac061220 0x7f68ac140070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f6894001110 tx=0x7f689c006df0).stop 2022-06-29T15:49:47.548 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.532+0000 7f68b24c4700 1 -- 172.21.15.47:0/2712058951 shutdown_connections 2022-06-29T15:49:47.548 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.532+0000 7f68b24c4700 1 --2- 172.21.15.47:0/2712058951 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f68ac061220 0x7f68ac140070 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:47.548 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.532+0000 7f68b24c4700 1 -- 172.21.15.47:0/2712058951 >> 172.21.15.47:0/2712058951 conn(0x7f68ac05c6b0 msgr2=0x7f68ac05f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:47.549 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.532+0000 7f68b24c4700 1 -- 172.21.15.47:0/2712058951 shutdown_connections 2022-06-29T15:49:47.549 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.532+0000 7f68b24c4700 1 -- 172.21.15.47:0/2712058951 wait complete. 2022-06-29T15:49:47.549 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.536+0000 7f68b24c4700 1 Processor -- start 2022-06-29T15:49:47.550 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.536+0000 7f68b24c4700 1 -- start start 2022-06-29T15:49:47.553 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.536+0000 7f68b24c4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68ac061220 0x7f68ac0b1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:47.553 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.536+0000 7f68b24c4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68ac0b4320 0x7f68ac13f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:47.553 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.536+0000 7f68b24c4700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f68ac146720 0x7f68ac13f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:47.553 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.540+0000 7f68b24c4700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac0b0b50 con 0x7f68ac0b4320 2022-06-29T15:49:47.554 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.540+0000 7f68b24c4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac0b6990 con 0x7f68ac061220 2022-06-29T15:49:47.554 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.540+0000 7f68b24c4700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac0b13c0 con 0x7f68ac146720 2022-06-29T15:49:47.554 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.540+0000 7f68a3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68ac0b4320 0x7f68ac13f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:47.554 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.540+0000 7f68abfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68ac061220 0x7f68ac0b1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:47.555 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.540+0000 7f68a3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68ac0b4320 0x7f68ac13f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51326/0 (socket says 172.21.15.47:51326) 2022-06-29T15:49:47.555 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.540+0000 7f68abfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68ac061220 0x7f68ac0b1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:57028/0 (socket says 172.21.15.47:57028) 2022-06-29T15:49:47.555 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.540+0000 7f68a3fff700 1 -- 172.21.15.47:0/318254929 learned_addr learned my addr 172.21.15.47:0/318254929 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:47.555 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.540+0000 7f68b0a61700 1 --2- 172.21.15.47:0/318254929 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f68ac146720 0x7f68ac13f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:47.556 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.540+0000 7f68a3fff700 1 -- 172.21.15.47:0/318254929 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f68ac146720 msgr2=0x7f68ac13f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:47.556 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.540+0000 7f68a3fff700 1 --2- 172.21.15.47:0/318254929 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f68ac146720 0x7f68ac13f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:47.556 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.540+0000 7f68a3fff700 1 -- 172.21.15.47:0/318254929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68ac061220 msgr2=0x7f68ac0b1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:47.556 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.540+0000 7f68a3fff700 1 --2- 172.21.15.47:0/318254929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68ac061220 0x7f68ac0b1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:47.557 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.540+0000 7f68a3fff700 1 -- 172.21.15.47:0/318254929 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f689c00b180 con 0x7f68ac0b4320 2022-06-29T15:49:47.557 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.540+0000 7f68abfff700 1 --2- 172.21.15.47:0/318254929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68ac061220 0x7f68ac0b1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:49:47.557 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.540+0000 7f68a3fff700 1 --2- 172.21.15.47:0/318254929 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68ac0b4320 0x7f68ac13f390 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 rx=0x7f6894000fe0 tx=0x7f68940018d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:47.558 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.540+0000 7f68a9ffb700 1 -- 172.21.15.47:0/318254929 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6894007170 con 0x7f68ac0b4320 2022-06-29T15:49:47.558 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.540+0000 7f68a9ffb700 1 -- 172.21.15.47:0/318254929 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68940072d0 con 0x7f68ac0b4320 2022-06-29T15:49:47.558 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.540+0000 7f68b24c4700 1 -- 172.21.15.47:0/318254929 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68ac0b06b0 con 0x7f68ac0b4320 2022-06-29T15:49:47.558 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.540+0000 7f68a9ffb700 1 -- 172.21.15.47:0/318254929 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6894004710 con 0x7f68ac0b4320 2022-06-29T15:49:47.559 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.540+0000 7f68b24c4700 1 -- 172.21.15.47:0/318254929 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68ac112b70 con 0x7f68ac0b4320 2022-06-29T15:49:47.559 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.540+0000 7f68b24c4700 1 -- 172.21.15.47:0/318254929 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68ac0b15c0 con 0x7f68ac0b4320 2022-06-29T15:49:47.559 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.540+0000 7f68a9ffb700 1 -- 172.21.15.47:0/318254929 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f689401b070 con 0x7f68ac0b4320 2022-06-29T15:49:47.559 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.540+0000 7f68a9ffb700 1 --2- 172.21.15.47:0/318254929 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6890053a70 0x7f6890055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:47.560 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.544+0000 7f68a9ffb700 1 -- 172.21.15.47:0/318254929 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f689406b2a0 con 0x7f68ac0b4320 2022-06-29T15:49:47.560 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.544+0000 7f68abfff700 1 --2- 172.21.15.47:0/318254929 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6890053a70 0x7f6890055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:47.560 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.544+0000 7f68abfff700 1 --2- 172.21.15.47:0/318254929 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6890053a70 0x7f6890055f30 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f689c01a000 tx=0x7f689c014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:47.561 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.552+0000 7f68a9ffb700 1 -- 172.21.15.47:0/318254929 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f68940423a0 con 0x7f68ac0b4320 2022-06-29T15:49:47.688 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.680+0000 7f68b24c4700 1 -- 172.21.15.47:0/318254929 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f68ac0b0840 con 0x7f68ac0b4320 2022-06-29T15:49:47.689 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.680+0000 7f68a9ffb700 1 -- 172.21.15.47:0/318254929 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v97) v1 ==== 110+0+10 (secure 0 0 0) 0x7f68940459c0 con 0x7f68ac0b4320 2022-06-29T15:49:47.689 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2022-06-29T15:49:47.691 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.680+0000 7f68b24c4700 1 -- 172.21.15.47:0/318254929 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6890053a70 msgr2=0x7f6890055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:47.691 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.680+0000 7f68b24c4700 1 --2- 172.21.15.47:0/318254929 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6890053a70 0x7f6890055f30 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f689c01a000 tx=0x7f689c014000).stop 2022-06-29T15:49:47.692 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.680+0000 7f68b24c4700 1 -- 172.21.15.47:0/318254929 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68ac0b4320 msgr2=0x7f68ac13f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:47.692 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.680+0000 7f68b24c4700 1 --2- 172.21.15.47:0/318254929 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68ac0b4320 0x7f68ac13f390 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 rx=0x7f6894000fe0 tx=0x7f68940018d0).stop 2022-06-29T15:49:47.692 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.680+0000 7f68b24c4700 1 -- 172.21.15.47:0/318254929 shutdown_connections 2022-06-29T15:49:47.692 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.680+0000 7f68b24c4700 1 --2- 172.21.15.47:0/318254929 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f68ac146720 0x7f68ac13f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:47.693 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.680+0000 7f68b24c4700 1 --2- 172.21.15.47:0/318254929 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68ac0b4320 0x7f68ac13f390 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:47.693 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.680+0000 7f68b24c4700 1 --2- 172.21.15.47:0/318254929 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6890053a70 0x7f6890055f30 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:47.693 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.680+0000 7f68b24c4700 1 --2- 172.21.15.47:0/318254929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68ac061220 0x7f68ac0b1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:47.693 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.680+0000 7f68b24c4700 1 -- 172.21.15.47:0/318254929 >> 172.21.15.47:0/318254929 conn(0x7f68ac05c6b0 msgr2=0x7f68ac05f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:47.694 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.684+0000 7f68b24c4700 1 -- 172.21.15.47:0/318254929 shutdown_connections 2022-06-29T15:49:47.694 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.684+0000 7f68b24c4700 1 -- 172.21.15.47:0/318254929 wait complete. 2022-06-29T15:49:47.702 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-29T15:49:47.811 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.800+0000 7f4cb226e700 1 Processor -- start 2022-06-29T15:49:47.811 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.800+0000 7f4cb226e700 1 -- start start 2022-06-29T15:49:47.813 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.804+0000 7f4cb226e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4cac060ee0 0x7f4cac0612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:47.813 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.804+0000 7f4cb226e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4cac0b4300 0x7f4cac1471a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:47.814 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.804+0000 7f4cb226e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cac067610 con 0x7f4cac066f70 2022-06-29T15:49:47.814 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.804+0000 7f4cb226e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cac0673a0 con 0x7f4cac0b4300 2022-06-29T15:49:47.814 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.804+0000 7f4cb226e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cac1467c0 con 0x7f4cac060ee0 2022-06-29T15:49:47.815 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.804+0000 7f4cab7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4cac0b4300 0x7f4cac1471a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:47.815 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.804+0000 7f4cab7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4cac0b4300 0x7f4cac1471a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51334/0 (socket says 172.21.15.47:51334) 2022-06-29T15:49:47.815 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.804+0000 7f4cab7fe700 1 -- 172.21.15.47:0/210955630 learned_addr learned my addr 172.21.15.47:0/210955630 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:47.815 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.804+0000 7f4cabfff700 1 --2- 172.21.15.47:0/210955630 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4cac060ee0 0x7f4cac0612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:47.815 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.804+0000 7f4cab7fe700 1 -- 172.21.15.47:0/210955630 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4cac060ee0 msgr2=0x7f4cac0612c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:47.816 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.804+0000 7f4cab7fe700 1 --2- 172.21.15.47:0/210955630 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4cac060ee0 0x7f4cac0612c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:47.816 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.804+0000 7f4cab7fe700 1 -- 172.21.15.47:0/210955630 >> v1:172.21.15.64:6789/0 conn(0x7f4cac066f70 legacy=0x7f4cac1430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:47.816 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.804+0000 7f4cab7fe700 1 -- 172.21.15.47:0/210955630 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cac146a60 con 0x7f4cac0b4300 2022-06-29T15:49:47.816 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.804+0000 7f4cab7fe700 1 --2- 172.21.15.47:0/210955630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cac0b4300 0x7f4cac1471a0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 rx=0x7f4c94002f70 tx=0x7f4c94006eb0).ready entity=mon.0 client_cookie=fd0c85610de34863 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:47.817 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.804+0000 7f4caaffd700 1 -- 172.21.15.47:0/210955630 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c9400d5c0 con 0x7f4cac0b4300 2022-06-29T15:49:47.817 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.804+0000 7f4caaffd700 1 -- 172.21.15.47:0/210955630 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c94000e90 con 0x7f4cac0b4300 2022-06-29T15:49:47.817 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.804+0000 7f4caaffd700 1 -- 172.21.15.47:0/210955630 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c9400d950 con 0x7f4cac0b4300 2022-06-29T15:49:47.822 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.804+0000 7f4cb226e700 1 -- 172.21.15.47:0/210955630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cac0b4300 msgr2=0x7f4cac1471a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:47.822 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.804+0000 7f4cb226e700 1 --2- 172.21.15.47:0/210955630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cac0b4300 0x7f4cac1471a0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 rx=0x7f4c94002f70 tx=0x7f4c94006eb0).stop 2022-06-29T15:49:47.823 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.804+0000 7f4cb226e700 1 -- 172.21.15.47:0/210955630 shutdown_connections 2022-06-29T15:49:47.823 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.804+0000 7f4cb226e700 1 --2- 172.21.15.47:0/210955630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cac0b4300 0x7f4cac1471a0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:47.824 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.804+0000 7f4cb226e700 1 --2- 172.21.15.47:0/210955630 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4cac060ee0 0x7f4cac0612c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:47.824 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.804+0000 7f4cb226e700 1 -- 172.21.15.47:0/210955630 >> 172.21.15.47:0/210955630 conn(0x7f4cac05c510 msgr2=0x7f4cac05f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:47.824 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.804+0000 7f4cb226e700 1 -- 172.21.15.47:0/210955630 shutdown_connections 2022-06-29T15:49:47.825 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.804+0000 7f4cb226e700 1 -- 172.21.15.47:0/210955630 wait complete. 2022-06-29T15:49:47.825 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.808+0000 7f4cb226e700 1 Processor -- start 2022-06-29T15:49:47.825 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.808+0000 7f4cb226e700 1 -- start start 2022-06-29T15:49:47.826 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.808+0000 7f4cb226e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4cac060ee0 0x7f4cac13a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:47.826 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.808+0000 7f4cb226e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4cac066f70 0x7f4cac1557e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:47.827 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.808+0000 7f4cb226e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cac0b4300 0x7f4cac155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:47.827 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.808+0000 7f4cb226e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cac1561c0 con 0x7f4cac0b4300 2022-06-29T15:49:47.827 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.808+0000 7f4cb226e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cac145710 con 0x7f4cac060ee0 2022-06-29T15:49:47.828 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.808+0000 7f4cb226e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cac1453b0 con 0x7f4cac066f70 2022-06-29T15:49:47.828 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.808+0000 7f4cabfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4cac060ee0 0x7f4cac13a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:47.828 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.808+0000 7f4cb080b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cac0b4300 0x7f4cac155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:47.829 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.808+0000 7f4cabfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4cac060ee0 0x7f4cac13a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:57042/0 (socket says 172.21.15.47:57042) 2022-06-29T15:49:47.829 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.808+0000 7f4cabfff700 1 -- 172.21.15.47:0/3344804364 learned_addr learned my addr 172.21.15.47:0/3344804364 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:47.829 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.808+0000 7f4cab7fe700 1 --2- 172.21.15.47:0/3344804364 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4cac066f70 0x7f4cac1557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:47.830 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.808+0000 7f4cb080b700 1 -- 172.21.15.47:0/3344804364 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4cac066f70 msgr2=0x7f4cac1557e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:47.830 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.808+0000 7f4cb080b700 1 --2- 172.21.15.47:0/3344804364 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4cac066f70 0x7f4cac1557e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:47.830 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.808+0000 7f4cb080b700 1 -- 172.21.15.47:0/3344804364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4cac060ee0 msgr2=0x7f4cac13a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:47.831 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.808+0000 7f4cb080b700 1 --2- 172.21.15.47:0/3344804364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4cac060ee0 0x7f4cac13a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:47.831 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.808+0000 7f4cb080b700 1 -- 172.21.15.47:0/3344804364 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c9400b1e0 con 0x7f4cac0b4300 2022-06-29T15:49:47.831 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.808+0000 7f4cb080b700 1 --2- 172.21.15.47:0/3344804364 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cac0b4300 0x7f4cac155cd0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 rx=0x7f4cac145540 tx=0x7f4c9c007b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:47.831 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.808+0000 7f4ca97fa700 1 -- 172.21.15.47:0/3344804364 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c9c00cd40 con 0x7f4cac0b4300 2022-06-29T15:49:47.832 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.808+0000 7f4ca97fa700 1 -- 172.21.15.47:0/3344804364 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c9c012400 con 0x7f4cac0b4300 2022-06-29T15:49:47.832 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.808+0000 7f4cb226e700 1 -- 172.21.15.47:0/3344804364 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cac144250 con 0x7f4cac0b4300 2022-06-29T15:49:47.832 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.808+0000 7f4ca97fa700 1 -- 172.21.15.47:0/3344804364 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c9c00cea0 con 0x7f4cac0b4300 2022-06-29T15:49:47.832 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.808+0000 7f4cb226e700 1 -- 172.21.15.47:0/3344804364 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cac145110 con 0x7f4cac0b4300 2022-06-29T15:49:47.833 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.812+0000 7f4cb226e700 1 -- 172.21.15.47:0/3344804364 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cac1458d0 con 0x7f4cac0b4300 2022-06-29T15:49:47.833 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.812+0000 7f4ca97fa700 1 -- 172.21.15.47:0/3344804364 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4c9c019030 con 0x7f4cac0b4300 2022-06-29T15:49:47.833 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.812+0000 7f4ca97fa700 1 --2- 172.21.15.47:0/3344804364 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4c90053aa0 0x7f4c90055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:47.833 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.812+0000 7f4ca97fa700 1 -- 172.21.15.47:0/3344804364 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f4c9c06afa0 con 0x7f4cac0b4300 2022-06-29T15:49:47.833 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.812+0000 7f4cabfff700 1 --2- 172.21.15.47:0/3344804364 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4c90053aa0 0x7f4c90055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:47.834 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.812+0000 7f4cabfff700 1 --2- 172.21.15.47:0/3344804364 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4c90053aa0 0x7f4c90055f60 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7f4ca4008000 tx=0x7f4ca4008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:47.835 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.816+0000 7f4ca97fa700 1 -- 172.21.15.47:0/3344804364 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4cac1458d0 con 0x7f4cac0b4300 2022-06-29T15:49:47.951 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.944+0000 7f4cb226e700 1 -- 172.21.15.47:0/3344804364 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4cac1458d0 con 0x7f4cac0b4300 2022-06-29T15:49:47.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.944+0000 7f4ca97fa700 1 -- 172.21.15.47:0/3344804364 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v97) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4cac1458d0 con 0x7f4cac0b4300 2022-06-29T15:49:47.953 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:49:47.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.944+0000 7f4cb226e700 1 -- 172.21.15.47:0/3344804364 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4c90053aa0 msgr2=0x7f4c90055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:47.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.944+0000 7f4cb226e700 1 --2- 172.21.15.47:0/3344804364 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4c90053aa0 0x7f4c90055f60 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7f4ca4008000 tx=0x7f4ca4008040).stop 2022-06-29T15:49:47.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.944+0000 7f4cb226e700 1 -- 172.21.15.47:0/3344804364 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cac0b4300 msgr2=0x7f4cac155cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:47.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.944+0000 7f4cb226e700 1 --2- 172.21.15.47:0/3344804364 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cac0b4300 0x7f4cac155cd0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 rx=0x7f4cac145540 tx=0x7f4c9c007b30).stop 2022-06-29T15:49:47.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.944+0000 7f4cb226e700 1 -- 172.21.15.47:0/3344804364 shutdown_connections 2022-06-29T15:49:47.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.944+0000 7f4cb226e700 1 --2- 172.21.15.47:0/3344804364 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cac0b4300 0x7f4cac155cd0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:47.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.944+0000 7f4cb226e700 1 --2- 172.21.15.47:0/3344804364 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4cac066f70 0x7f4cac1557e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:47.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.944+0000 7f4cb226e700 1 --2- 172.21.15.47:0/3344804364 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4c90053aa0 0x7f4c90055f60 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:47.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.944+0000 7f4cb226e700 1 --2- 172.21.15.47:0/3344804364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4cac060ee0 0x7f4cac13a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:47.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.944+0000 7f4cb226e700 1 -- 172.21.15.47:0/3344804364 >> 172.21.15.47:0/3344804364 conn(0x7f4cac05c510 msgr2=0x7f4cac1389a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:47.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.944+0000 7f4cb226e700 1 -- 172.21.15.47:0/3344804364 shutdown_connections 2022-06-29T15:49:47.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:47.944+0000 7f4cb226e700 1 -- 172.21.15.47:0/3344804364 wait complete. 2022-06-29T15:49:47.970 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-06-29T15:49:48.069 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.060+0000 7fc5f9ab0700 1 Processor -- start 2022-06-29T15:49:48.070 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.060+0000 7fc5f9ab0700 1 -- start start 2022-06-29T15:49:48.071 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.060+0000 7fc5f9ab0700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc5f40b0f30 0x7fc5f40606e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:48.072 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.060+0000 7fc5f9ab0700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5f4140d50 con 0x7fc5f41408e0 2022-06-29T15:49:48.072 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.060+0000 7fc5f9ab0700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5f4061d90 con 0x7fc5f40619b0 2022-06-29T15:49:48.072 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.060+0000 7fc5f9ab0700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f40b2190 con 0x7fc5f40b0f30 2022-06-29T15:49:48.072 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.060+0000 7fc5f3fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc5f41408e0 0x7fc5f4139a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36904/0 (socket says 172.21.15.47:36904) 2022-06-29T15:49:48.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.060+0000 7fc5f3fff700 1 -- 172.21.15.47:0/703844754 learned_addr learned my addr 172.21.15.47:0/703844754 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:49:48.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.064+0000 7fc5f2ffd700 1 --2- 172.21.15.47:0/703844754 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc5f40b0f30 0x7fc5f40606e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:48.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.064+0000 7fc5f27fc700 1 -- 172.21.15.47:0/703844754 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 184798576 0 0) 0x7fc5f4140d50 con 0x7fc5f41408e0 2022-06-29T15:49:48.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.064+0000 7fc5f27fc700 1 -- 172.21.15.47:0/703844754 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5e0003260 con 0x7fc5f41408e0 2022-06-29T15:49:48.077 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.064+0000 7fc5f27fc700 1 -- 172.21.15.47:0/703844754 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1018462539 0 0) 0x7fc5f4061d90 con 0x7fc5f40619b0 2022-06-29T15:49:48.077 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.064+0000 7fc5f27fc700 1 -- 172.21.15.47:0/703844754 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5f4140d50 con 0x7fc5f40619b0 2022-06-29T15:49:48.078 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.064+0000 7fc5f2ffd700 1 -- 172.21.15.47:0/703844754 >> v1:172.21.15.64:6789/0 conn(0x7fc5f40619b0 legacy=0x7fc5f40b1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:48.078 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.064+0000 7fc5f2ffd700 1 -- 172.21.15.47:0/703844754 >> v1:172.21.15.47:6789/0 conn(0x7fc5f41408e0 legacy=0x7fc5f4139a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:48.079 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.064+0000 7fc5f2ffd700 1 -- 172.21.15.47:0/703844754 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f4060d80 con 0x7fc5f40b0f30 2022-06-29T15:49:48.079 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.064+0000 7fc5f2ffd700 1 --2- 172.21.15.47:0/703844754 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc5f40b0f30 0x7fc5f40606e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7fc5e8002f70 tx=0x7fc5e8006df0).ready entity=mon.2 client_cookie=73d9a8595e123573 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:48.080 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.064+0000 7fc5f27fc700 1 -- 172.21.15.47:0/703844754 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc5e800d5c0 con 0x7fc5f40b0f30 2022-06-29T15:49:48.081 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.064+0000 7fc5f27fc700 1 -- 172.21.15.47:0/703844754 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5e8000e90 con 0x7fc5f40b0f30 2022-06-29T15:49:48.081 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.064+0000 7fc5f27fc700 1 -- 172.21.15.47:0/703844754 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc5e800d910 con 0x7fc5f40b0f30 2022-06-29T15:49:48.082 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.064+0000 7fc5f9ab0700 1 -- 172.21.15.47:0/703844754 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc5f40b0f30 msgr2=0x7fc5f40606e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:48.082 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.064+0000 7fc5f9ab0700 1 --2- 172.21.15.47:0/703844754 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc5f40b0f30 0x7fc5f40606e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7fc5e8002f70 tx=0x7fc5e8006df0).stop 2022-06-29T15:49:48.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.064+0000 7fc5f9ab0700 1 -- 172.21.15.47:0/703844754 shutdown_connections 2022-06-29T15:49:48.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.064+0000 7fc5f9ab0700 1 --2- 172.21.15.47:0/703844754 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc5f40b0f30 0x7fc5f40606e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:48.084 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.064+0000 7fc5f9ab0700 1 -- 172.21.15.47:0/703844754 >> 172.21.15.47:0/703844754 conn(0x7fc5f405c510 msgr2=0x7fc5f4063800 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:48.084 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.064+0000 7fc5f9ab0700 1 -- 172.21.15.47:0/703844754 shutdown_connections 2022-06-29T15:49:48.084 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.064+0000 7fc5f9ab0700 1 -- 172.21.15.47:0/703844754 wait complete. 2022-06-29T15:49:48.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.064+0000 7fc5f9ab0700 1 Processor -- start 2022-06-29T15:49:48.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.064+0000 7fc5f9ab0700 1 -- start start 2022-06-29T15:49:48.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.068+0000 7fc5f9ab0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5f40619b0 0x7fc5f4152790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:48.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.068+0000 7fc5f9ab0700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc5f40b0f30 0x7fc5f413a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:48.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.068+0000 7fc5f9ab0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5f41408e0 0x7fc5f4141a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:48.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.068+0000 7fc5f9ab0700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f4144bc0 con 0x7fc5f41408e0 2022-06-29T15:49:48.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.068+0000 7fc5f9ab0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f413d4a0 con 0x7fc5f40619b0 2022-06-29T15:49:48.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.068+0000 7fc5f9ab0700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f4152c80 con 0x7fc5f40b0f30 2022-06-29T15:49:48.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.068+0000 7fc5f3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5f41408e0 0x7fc5f4141a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:48.089 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.068+0000 7fc5f3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5f41408e0 0x7fc5f4141a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51354/0 (socket says 172.21.15.47:51354) 2022-06-29T15:49:48.089 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.068+0000 7fc5f3fff700 1 -- 172.21.15.47:0/3457971078 learned_addr learned my addr 172.21.15.47:0/3457971078 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:48.090 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.068+0000 7fc5f2ffd700 1 --2- 172.21.15.47:0/3457971078 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc5f40b0f30 0x7fc5f413a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:48.090 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.068+0000 7fc5f37fe700 1 --2- 172.21.15.47:0/3457971078 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5f40619b0 0x7fc5f4152790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:48.090 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.068+0000 7fc5f3fff700 1 -- 172.21.15.47:0/3457971078 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc5f40b0f30 msgr2=0x7fc5f413a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:48.091 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.068+0000 7fc5f3fff700 1 --2- 172.21.15.47:0/3457971078 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc5f40b0f30 0x7fc5f413a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:48.091 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.068+0000 7fc5f3fff700 1 -- 172.21.15.47:0/3457971078 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5f40619b0 msgr2=0x7fc5f4152790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:48.092 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.068+0000 7fc5f3fff700 1 --2- 172.21.15.47:0/3457971078 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5f40619b0 0x7fc5f4152790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:48.092 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.068+0000 7fc5f3fff700 1 -- 172.21.15.47:0/3457971078 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e800b180 con 0x7fc5f41408e0 2022-06-29T15:49:48.093 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.068+0000 7fc5f3fff700 1 --2- 172.21.15.47:0/3457971078 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5f41408e0 0x7fc5f4141a10 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 rx=0x7fc5f4143010 tx=0x7fc5ec007e90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:48.093 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.068+0000 7fc5f0ff9700 1 -- 172.21.15.47:0/3457971078 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc5ec005b80 con 0x7fc5f41408e0 2022-06-29T15:49:48.093 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.068+0000 7fc5f0ff9700 1 -- 172.21.15.47:0/3457971078 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5ec005ce0 con 0x7fc5f41408e0 2022-06-29T15:49:48.094 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.068+0000 7fc5f9ab0700 1 -- 172.21.15.47:0/3457971078 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5f4142310 con 0x7fc5f41408e0 2022-06-29T15:49:48.094 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.068+0000 7fc5f0ff9700 1 -- 172.21.15.47:0/3457971078 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc5ec00c9d0 con 0x7fc5f41408e0 2022-06-29T15:49:48.094 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.068+0000 7fc5f9ab0700 1 -- 172.21.15.47:0/3457971078 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f41438a0 con 0x7fc5f41408e0 2022-06-29T15:49:48.095 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.068+0000 7fc5f9ab0700 1 -- 172.21.15.47:0/3457971078 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5f4143a60 con 0x7fc5f41408e0 2022-06-29T15:49:48.096 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.068+0000 7fc5f0ff9700 1 -- 172.21.15.47:0/3457971078 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc5ec00cb30 con 0x7fc5f41408e0 2022-06-29T15:49:48.096 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.072+0000 7fc5f0ff9700 1 --2- 172.21.15.47:0/3457971078 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc5d4053a10 0x7fc5d4055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:48.096 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.072+0000 7fc5f0ff9700 1 -- 172.21.15.47:0/3457971078 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fc5ec06e5d0 con 0x7fc5f41408e0 2022-06-29T15:49:48.096 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.072+0000 7fc5f37fe700 1 --2- 172.21.15.47:0/3457971078 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc5d4053a10 0x7fc5d4055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:48.097 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.072+0000 7fc5f37fe700 1 --2- 172.21.15.47:0/3457971078 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc5d4053a10 0x7fc5d4055ed0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7fc5e400b000 tx=0x7fc5e4009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:48.097 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.076+0000 7fc5f0ff9700 1 -- 172.21.15.47:0/3457971078 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc5ec0456d0 con 0x7fc5f41408e0 2022-06-29T15:49:48.211 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.204+0000 7fc5f9ab0700 1 -- 172.21.15.47:0/3457971078 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc5f41420a0 con 0x7fc5f41408e0 2022-06-29T15:49:48.212 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.204+0000 7fc5f0ff9700 1 -- 172.21.15.47:0/3457971078 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v97) v1 ==== 84+0+143 (secure 0 0 0) 0x7fc5ec048cf0 con 0x7fc5f41408e0 2022-06-29T15:49:48.212 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:49:48.213 INFO:teuthology.orchestra.run.smithi047.stdout:[ 2022-06-29T15:49:48.213 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:48.213 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 1, 2022-06-29T15:49:48.213 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": "device_health_metrics" 2022-06-29T15:49:48.214 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:49:48.214 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:49:48.214 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 2, 2022-06-29T15:49:48.214 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": "rbd" 2022-06-29T15:49:48.215 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:49:48.215 INFO:teuthology.orchestra.run.smithi047.stdout:] 2022-06-29T15:49:48.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.204+0000 7fc5f9ab0700 1 -- 172.21.15.47:0/3457971078 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc5d4053a10 msgr2=0x7fc5d4055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:48.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.204+0000 7fc5f9ab0700 1 --2- 172.21.15.47:0/3457971078 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc5d4053a10 0x7fc5d4055ed0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7fc5e400b000 tx=0x7fc5e4009040).stop 2022-06-29T15:49:48.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.204+0000 7fc5f9ab0700 1 -- 172.21.15.47:0/3457971078 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5f41408e0 msgr2=0x7fc5f4141a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:48.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.204+0000 7fc5f9ab0700 1 --2- 172.21.15.47:0/3457971078 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5f41408e0 0x7fc5f4141a10 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 rx=0x7fc5f4143010 tx=0x7fc5ec007e90).stop 2022-06-29T15:49:48.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.204+0000 7fc5f9ab0700 1 -- 172.21.15.47:0/3457971078 shutdown_connections 2022-06-29T15:49:48.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.204+0000 7fc5f9ab0700 1 --2- 172.21.15.47:0/3457971078 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5f41408e0 0x7fc5f4141a10 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:48.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.204+0000 7fc5f9ab0700 1 --2- 172.21.15.47:0/3457971078 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc5f40b0f30 0x7fc5f413a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:48.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.204+0000 7fc5f9ab0700 1 --2- 172.21.15.47:0/3457971078 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc5d4053a10 0x7fc5d4055ed0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:48.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.204+0000 7fc5f9ab0700 1 --2- 172.21.15.47:0/3457971078 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5f40619b0 0x7fc5f4152790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:48.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.204+0000 7fc5f9ab0700 1 -- 172.21.15.47:0/3457971078 >> 172.21.15.47:0/3457971078 conn(0x7fc5f405c510 msgr2=0x7fc5f413dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:48.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.204+0000 7fc5f9ab0700 1 -- 172.21.15.47:0/3457971078 shutdown_connections 2022-06-29T15:49:48.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.204+0000 7fc5f9ab0700 1 -- 172.21.15.47:0/3457971078 wait complete. 2022-06-29T15:49:48.226 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-06-29T15:49:48.227 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-06-29T15:49:48.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.332+0000 7efeba5bd700 1 Processor -- start 2022-06-29T15:49:48.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.332+0000 7efeba5bd700 1 -- start start 2022-06-29T15:49:48.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.332+0000 7efeba5bd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efeb4141fd0 0x7efeb40b3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:48.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.332+0000 7efeba5bd700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeb405f7d0 con 0x7efeb40b1320 2022-06-29T15:49:48.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.332+0000 7efeba5bd700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeb40b4b90 con 0x7efeb405f3f0 2022-06-29T15:49:48.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.332+0000 7efeba5bd700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efeb40b27a0 con 0x7efeb4141fd0 2022-06-29T15:49:48.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.332+0000 7efeb8dba700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7efeb40b1320 0x7efeb40b2ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36914/0 (socket says 172.21.15.47:36914) 2022-06-29T15:49:48.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.332+0000 7efeb8dba700 1 -- 172.21.15.47:0/2148274706 learned_addr learned my addr 172.21.15.47:0/2148274706 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:49:48.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.332+0000 7efeb9dbc700 1 --2- 172.21.15.47:0/2148274706 >> v2:172.21.15.64:3300/0 conn(0x7efeb4141fd0 0x7efeb40b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:48.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.332+0000 7efea3fff700 1 -- 172.21.15.47:0/2148274706 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2033127897 0 0) 0x7efeb405f7d0 con 0x7efeb40b1320 2022-06-29T15:49:48.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.332+0000 7efea3fff700 1 -- 172.21.15.47:0/2148274706 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe9c003260 con 0x7efeb40b1320 2022-06-29T15:49:48.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.332+0000 7efea3fff700 1 -- 172.21.15.47:0/2148274706 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3995821450 0 0) 0x7efeb40b4b90 con 0x7efeb405f3f0 2022-06-29T15:49:48.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.332+0000 7efea3fff700 1 -- 172.21.15.47:0/2148274706 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efeb405f7d0 con 0x7efeb405f3f0 2022-06-29T15:49:48.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.332+0000 7efea3fff700 1 -- 172.21.15.47:0/2148274706 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3485237724 0 0) 0x7efe9c003260 con 0x7efeb40b1320 2022-06-29T15:49:48.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.332+0000 7efeb9dbc700 1 -- 172.21.15.47:0/2148274706 >> v1:172.21.15.64:6789/0 conn(0x7efeb405f3f0 legacy=0x7efeb41418c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:48.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.332+0000 7efeb9dbc700 1 -- 172.21.15.47:0/2148274706 >> v1:172.21.15.47:6789/0 conn(0x7efeb40b1320 legacy=0x7efeb40b2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:48.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.332+0000 7efeb9dbc700 1 -- 172.21.15.47:0/2148274706 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb40b2060 con 0x7efeb4141fd0 2022-06-29T15:49:48.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.332+0000 7efeb9dbc700 1 --2- 172.21.15.47:0/2148274706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efeb4141fd0 0x7efeb40b3700 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7efea4001110 tx=0x7efeac0091d0).ready entity=mon.1 client_cookie=3c1459eac7e5ffdb server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:48.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.332+0000 7efea3fff700 1 -- 172.21.15.47:0/2148274706 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efea4001660 con 0x7efeb4141fd0 2022-06-29T15:49:48.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.332+0000 7efea3fff700 1 -- 172.21.15.47:0/2148274706 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeac00f510 con 0x7efeb4141fd0 2022-06-29T15:49:48.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.332+0000 7efea3fff700 1 -- 172.21.15.47:0/2148274706 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efeac00f680 con 0x7efeb4141fd0 2022-06-29T15:49:48.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.332+0000 7efeba5bd700 1 -- 172.21.15.47:0/2148274706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efeb4141fd0 msgr2=0x7efeb40b3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:48.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.332+0000 7efeba5bd700 1 --2- 172.21.15.47:0/2148274706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efeb4141fd0 0x7efeb40b3700 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7efea4001110 tx=0x7efeac0091d0).stop 2022-06-29T15:49:48.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.332+0000 7efeba5bd700 1 -- 172.21.15.47:0/2148274706 shutdown_connections 2022-06-29T15:49:48.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.332+0000 7efeba5bd700 1 --2- 172.21.15.47:0/2148274706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efeb4141fd0 0x7efeb40b3700 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:48.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.332+0000 7efeba5bd700 1 -- 172.21.15.47:0/2148274706 >> 172.21.15.47:0/2148274706 conn(0x7efeb4062570 msgr2=0x7efeb40647d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:48.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.336+0000 7efeba5bd700 1 -- 172.21.15.47:0/2148274706 shutdown_connections 2022-06-29T15:49:48.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.336+0000 7efeba5bd700 1 -- 172.21.15.47:0/2148274706 wait complete. 2022-06-29T15:49:48.352 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.336+0000 7efeba5bd700 1 Processor -- start 2022-06-29T15:49:48.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.336+0000 7efeba5bd700 1 -- start start 2022-06-29T15:49:48.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.336+0000 7efeba5bd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efeb405f3f0 0x7efeb41482e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:48.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.336+0000 7efeba5bd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efeb40b1320 0x7efeb40acf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:48.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.340+0000 7efeba5bd700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7efeb4141fd0 0x7efeb40ad420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:48.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.340+0000 7efeba5bd700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb40b7250 con 0x7efeb405f3f0 2022-06-29T15:49:48.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.340+0000 7efeba5bd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb40b74f0 con 0x7efeb40b1320 2022-06-29T15:49:48.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.340+0000 7efeba5bd700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeb40b7790 con 0x7efeb4141fd0 2022-06-29T15:49:48.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.340+0000 7efeb95bb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efeb405f3f0 0x7efeb41482e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:48.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.340+0000 7efeb95bb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efeb405f3f0 0x7efeb41482e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51366/0 (socket says 172.21.15.47:51366) 2022-06-29T15:49:48.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.340+0000 7efeb95bb700 1 -- 172.21.15.47:0/4035894089 learned_addr learned my addr 172.21.15.47:0/4035894089 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:48.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.340+0000 7efeb8dba700 1 --2- 172.21.15.47:0/4035894089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efeb40b1320 0x7efeb40acf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:48.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.340+0000 7efeb9dbc700 1 --2- 172.21.15.47:0/4035894089 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7efeb4141fd0 0x7efeb40ad420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:48.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.340+0000 7efeb95bb700 1 -- 172.21.15.47:0/4035894089 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7efeb4141fd0 msgr2=0x7efeb40ad420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:48.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.340+0000 7efeb95bb700 1 --2- 172.21.15.47:0/4035894089 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7efeb4141fd0 0x7efeb40ad420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:48.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.340+0000 7efeb95bb700 1 -- 172.21.15.47:0/4035894089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efeb40b1320 msgr2=0x7efeb40acf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:48.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.340+0000 7efeb95bb700 1 --2- 172.21.15.47:0/4035894089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efeb40b1320 0x7efeb40acf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:48.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.340+0000 7efeb95bb700 1 -- 172.21.15.47:0/4035894089 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeac009470 con 0x7efeb405f3f0 2022-06-29T15:49:48.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.340+0000 7efeb8dba700 1 --2- 172.21.15.47:0/4035894089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efeb40b1320 0x7efeb40acf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:49:48.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.340+0000 7efeb95bb700 1 --2- 172.21.15.47:0/4035894089 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efeb405f3f0 0x7efeb41482e0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 rx=0x7efeb0001110 tx=0x7efeb0007ab0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:48.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.340+0000 7efea27fc700 1 -- 172.21.15.47:0/4035894089 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efeb000ce00 con 0x7efeb405f3f0 2022-06-29T15:49:48.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.340+0000 7efea27fc700 1 -- 172.21.15.47:0/4035894089 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeb000f470 con 0x7efeb405f3f0 2022-06-29T15:49:48.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.340+0000 7efeba5bd700 1 -- 172.21.15.47:0/4035894089 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeb4144660 con 0x7efeb405f3f0 2022-06-29T15:49:48.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.340+0000 7efea27fc700 1 -- 172.21.15.47:0/4035894089 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efeb000f7d0 con 0x7efeb405f3f0 2022-06-29T15:49:48.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.340+0000 7efeba5bd700 1 -- 172.21.15.47:0/4035894089 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeb40ada70 con 0x7efeb405f3f0 2022-06-29T15:49:48.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.340+0000 7efeba5bd700 1 -- 172.21.15.47:0/4035894089 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efeb40573d0 con 0x7efeb405f3f0 2022-06-29T15:49:48.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.340+0000 7efea27fc700 1 -- 172.21.15.47:0/4035894089 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7efeb0019030 con 0x7efeb405f3f0 2022-06-29T15:49:48.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.344+0000 7efea27fc700 1 --2- 172.21.15.47:0/4035894089 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7efe940539e0 0x7efe94055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:48.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.344+0000 7efea27fc700 1 -- 172.21.15.47:0/4035894089 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5600+0+0 (secure 0 0 0) 0x7efeb006b330 con 0x7efeb405f3f0 2022-06-29T15:49:48.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.344+0000 7efeb8dba700 1 --2- 172.21.15.47:0/4035894089 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7efe940539e0 0x7efe94055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:48.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.344+0000 7efeb8dba700 1 --2- 172.21.15.47:0/4035894089 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7efe940539e0 0x7efe94055ea0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7efea400b000 tx=0x7efea4000bc0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:48.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.348+0000 7efea27fc700 1 -- 172.21.15.47:0/4035894089 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7efeb0042320 con 0x7efeb405f3f0 2022-06-29T15:49:48.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:48.468+0000 7efeba5bd700 1 -- 172.21.15.47:0/4035894089 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7efeb4068830 con 0x7efeb405f3f0 2022-06-29T15:49:49.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.464+0000 7efea27fc700 1 -- 172.21.15.47:0/4035894089 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v98) v1 ==== 133+0+0 (secure 0 0 0) 0x7efeb0045940 con 0x7efeb405f3f0 2022-06-29T15:49:49.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.472+0000 7efeba5bd700 1 -- 172.21.15.47:0/4035894089 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7efe940539e0 msgr2=0x7efe94055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:49.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.472+0000 7efeba5bd700 1 --2- 172.21.15.47:0/4035894089 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7efe940539e0 0x7efe94055ea0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7efea400b000 tx=0x7efea4000bc0).stop 2022-06-29T15:49:49.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.472+0000 7efeba5bd700 1 -- 172.21.15.47:0/4035894089 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efeb405f3f0 msgr2=0x7efeb41482e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:49.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.472+0000 7efeba5bd700 1 --2- 172.21.15.47:0/4035894089 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efeb405f3f0 0x7efeb41482e0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 rx=0x7efeb0001110 tx=0x7efeb0007ab0).stop 2022-06-29T15:49:49.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.472+0000 7efeba5bd700 1 -- 172.21.15.47:0/4035894089 shutdown_connections 2022-06-29T15:49:49.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.472+0000 7efeba5bd700 1 --2- 172.21.15.47:0/4035894089 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7efeb4141fd0 0x7efeb40ad420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:49.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.472+0000 7efeba5bd700 1 --2- 172.21.15.47:0/4035894089 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7efe940539e0 0x7efe94055ea0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:49.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.472+0000 7efeba5bd700 1 --2- 172.21.15.47:0/4035894089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efeb40b1320 0x7efeb40acf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:49.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.472+0000 7efeba5bd700 1 --2- 172.21.15.47:0/4035894089 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efeb405f3f0 0x7efeb41482e0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:49.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.472+0000 7efeba5bd700 1 -- 172.21.15.47:0/4035894089 >> 172.21.15.47:0/4035894089 conn(0x7efeb4062570 msgr2=0x7efeb40647d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:49.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.472+0000 7efeba5bd700 1 -- 172.21.15.47:0/4035894089 shutdown_connections 2022-06-29T15:49:49.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.472+0000 7efeba5bd700 1 -- 172.21.15.47:0/4035894089 wait complete. 2022-06-29T15:49:49.483 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' created 2022-06-29T15:49:49.493 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-06-29T15:49:49.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.596+0000 7f62a197a700 1 Processor -- start 2022-06-29T15:49:49.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.596+0000 7f62a197a700 1 -- start start 2022-06-29T15:49:49.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.600+0000 7f62a197a700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f629c0614b0 0x7f629c09a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:49.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.600+0000 7f62a197a700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f629c09b5f0 con 0x7f629c143ea0 2022-06-29T15:49:49.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.600+0000 7f62a197a700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f629c0b2c70 con 0x7f629c0a9290 2022-06-29T15:49:49.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.600+0000 7f62a197a700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f629c144280 con 0x7f629c0614b0 2022-06-29T15:49:49.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.600+0000 7f62a0978700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f629c143ea0 0x7f629c060b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36928/0 (socket says 172.21.15.47:36928) 2022-06-29T15:49:49.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.600+0000 7f62a0978700 1 -- 172.21.15.47:0/288521431 learned_addr learned my addr 172.21.15.47:0/288521431 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:49:49.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.600+0000 7f629bfff700 1 --2- 172.21.15.47:0/288521431 >> v2:172.21.15.64:3300/0 conn(0x7f629c0614b0 0x7f629c09a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:49.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.600+0000 7f629b7fe700 1 -- 172.21.15.47:0/288521431 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1703242051 0 0) 0x7f629c09b5f0 con 0x7f629c143ea0 2022-06-29T15:49:49.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.600+0000 7f629b7fe700 1 -- 172.21.15.47:0/288521431 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6280003260 con 0x7f629c143ea0 2022-06-29T15:49:49.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.600+0000 7f629bfff700 1 -- 172.21.15.47:0/288521431 >> v1:172.21.15.64:6789/0 conn(0x7f629c0a9290 legacy=0x7f629c09aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:49.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.600+0000 7f629bfff700 1 -- 172.21.15.47:0/288521431 >> v1:172.21.15.47:6789/0 conn(0x7f629c143ea0 legacy=0x7f629c060b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:49.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.600+0000 7f629bfff700 1 -- 172.21.15.47:0/288521431 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f629c061270 con 0x7f629c0614b0 2022-06-29T15:49:49.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.600+0000 7f629bfff700 1 --2- 172.21.15.47:0/288521431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f629c0614b0 0x7f629c09a9f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7f6288002f70 tx=0x7f6288006df0).ready entity=mon.1 client_cookie=da9098442bba81fc server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:49.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.600+0000 7f629b7fe700 1 -- 172.21.15.47:0/288521431 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f628800d5c0 con 0x7f629c0614b0 2022-06-29T15:49:49.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.600+0000 7f629b7fe700 1 -- 172.21.15.47:0/288521431 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6288000e90 con 0x7f629c0614b0 2022-06-29T15:49:49.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.600+0000 7f629b7fe700 1 -- 172.21.15.47:0/288521431 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f628800d910 con 0x7f629c0614b0 2022-06-29T15:49:49.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.600+0000 7f62a197a700 1 -- 172.21.15.47:0/288521431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f629c0614b0 msgr2=0x7f629c09a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:49.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.600+0000 7f62a197a700 1 --2- 172.21.15.47:0/288521431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f629c0614b0 0x7f629c09a9f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7f6288002f70 tx=0x7f6288006df0).stop 2022-06-29T15:49:49.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.600+0000 7f62a197a700 1 -- 172.21.15.47:0/288521431 shutdown_connections 2022-06-29T15:49:49.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.600+0000 7f62a197a700 1 --2- 172.21.15.47:0/288521431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f629c0614b0 0x7f629c09a9f0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:49.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.600+0000 7f62a197a700 1 -- 172.21.15.47:0/288521431 >> 172.21.15.47:0/288521431 conn(0x7f629c062df0 msgr2=0x7f629c0631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:49.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.604+0000 7f62a197a700 1 -- 172.21.15.47:0/288521431 shutdown_connections 2022-06-29T15:49:49.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.604+0000 7f62a197a700 1 -- 172.21.15.47:0/288521431 wait complete. 2022-06-29T15:49:49.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.604+0000 7f62a197a700 1 Processor -- start 2022-06-29T15:49:49.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.604+0000 7f62a197a700 1 -- start start 2022-06-29T15:49:49.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.604+0000 7f62a197a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f629c0614b0 0x7f629c0b0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:49.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.608+0000 7f62a197a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f629c0a9290 0x7f629c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:49.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.608+0000 7f62a197a700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f629c143ea0 0x7f629c0b5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:49.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.608+0000 7f62a197a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629c0ade60 con 0x7f629c0614b0 2022-06-29T15:49:49.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.608+0000 7f62a197a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629c0ae480 con 0x7f629c0a9290 2022-06-29T15:49:49.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.608+0000 7f62a197a700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f629c0ae090 con 0x7f629c143ea0 2022-06-29T15:49:49.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.608+0000 7f62a0978700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f629c0614b0 0x7f629c0b0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:49.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.608+0000 7f62a0978700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f629c0614b0 0x7f629c0b0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51380/0 (socket says 172.21.15.47:51380) 2022-06-29T15:49:49.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.608+0000 7f62a0978700 1 -- 172.21.15.47:0/2566905607 learned_addr learned my addr 172.21.15.47:0/2566905607 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:49.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.608+0000 7f629bfff700 1 --2- 172.21.15.47:0/2566905607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f629c0a9290 0x7f629c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:49.622 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.608+0000 7f62a1179700 1 --2- 172.21.15.47:0/2566905607 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f629c143ea0 0x7f629c0b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:49.622 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.608+0000 7f62a0978700 1 -- 172.21.15.47:0/2566905607 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f629c143ea0 msgr2=0x7f629c0b5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:49.622 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.608+0000 7f62a0978700 1 --2- 172.21.15.47:0/2566905607 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f629c143ea0 0x7f629c0b5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:49.622 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.608+0000 7f62a0978700 1 -- 172.21.15.47:0/2566905607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f629c0a9290 msgr2=0x7f629c0b4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:49.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.608+0000 7f62a0978700 1 --2- 172.21.15.47:0/2566905607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f629c0a9290 0x7f629c0b4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:49.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.608+0000 7f62a0978700 1 -- 172.21.15.47:0/2566905607 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f628800b180 con 0x7f629c0614b0 2022-06-29T15:49:49.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.608+0000 7f62a0978700 1 --2- 172.21.15.47:0/2566905607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f629c0614b0 0x7f629c0b0890 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 rx=0x7f6290001110 tx=0x7f6290007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:49.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.608+0000 7f6299ffb700 1 -- 172.21.15.47:0/2566905607 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f62900070d0 con 0x7f629c0614b0 2022-06-29T15:49:49.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.608+0000 7f6299ffb700 1 -- 172.21.15.47:0/2566905607 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6290007230 con 0x7f629c0614b0 2022-06-29T15:49:49.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.608+0000 7f62a197a700 1 -- 172.21.15.47:0/2566905607 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f629c0b5b30 con 0x7f629c0614b0 2022-06-29T15:49:49.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.608+0000 7f6299ffb700 1 -- 172.21.15.47:0/2566905607 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6290005610 con 0x7f629c0614b0 2022-06-29T15:49:49.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.608+0000 7f62a197a700 1 -- 172.21.15.47:0/2566905607 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f629c0aeac0 con 0x7f629c0614b0 2022-06-29T15:49:49.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.608+0000 7f62a197a700 1 -- 172.21.15.47:0/2566905607 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f629c0b3950 con 0x7f629c0614b0 2022-06-29T15:49:49.626 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.612+0000 7f6299ffb700 1 -- 172.21.15.47:0/2566905607 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6290010070 con 0x7f629c0614b0 2022-06-29T15:49:49.626 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.612+0000 7f6299ffb700 1 --2- 172.21.15.47:0/2566905607 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f627c053980 0x7f627c055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:49.626 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.612+0000 7f6299ffb700 1 -- 172.21.15.47:0/2566905607 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5954+0+0 (secure 0 0 0) 0x7f629006b510 con 0x7f629c0614b0 2022-06-29T15:49:49.626 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.612+0000 7f629bfff700 1 --2- 172.21.15.47:0/2566905607 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f627c053980 0x7f627c055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:49.627 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.612+0000 7f629bfff700 1 --2- 172.21.15.47:0/2566905607 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f627c053980 0x7f627c055e40 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f628801a000 tx=0x7f6288014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:49.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.620+0000 7f6299ffb700 1 -- 172.21.15.47:0/2566905607 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6290042390 con 0x7f629c0614b0 2022-06-29T15:49:49.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:49.748+0000 7f62a197a700 1 -- 172.21.15.47:0/2566905607 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f629c0573d0 con 0x7f629c0614b0 2022-06-29T15:49:50.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.476+0000 7f6299ffb700 1 -- 172.21.15.47:0/2566905607 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v99) v1 ==== 125+0+0 (secure 0 0 0) 0x7f62900459b0 con 0x7f629c0614b0 2022-06-29T15:49:50.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.480+0000 7f62a197a700 1 -- 172.21.15.47:0/2566905607 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f627c053980 msgr2=0x7f627c055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:50.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.480+0000 7f62a197a700 1 --2- 172.21.15.47:0/2566905607 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f627c053980 0x7f627c055e40 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f628801a000 tx=0x7f6288014000).stop 2022-06-29T15:49:50.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.480+0000 7f62a197a700 1 -- 172.21.15.47:0/2566905607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f629c0614b0 msgr2=0x7f629c0b0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:50.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.480+0000 7f62a197a700 1 --2- 172.21.15.47:0/2566905607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f629c0614b0 0x7f629c0b0890 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 rx=0x7f6290001110 tx=0x7f6290007ad0).stop 2022-06-29T15:49:50.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.480+0000 7f62a197a700 1 -- 172.21.15.47:0/2566905607 shutdown_connections 2022-06-29T15:49:50.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.480+0000 7f62a197a700 1 --2- 172.21.15.47:0/2566905607 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f629c143ea0 0x7f629c0b5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:50.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.480+0000 7f62a197a700 1 --2- 172.21.15.47:0/2566905607 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f627c053980 0x7f627c055e40 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:50.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.480+0000 7f62a197a700 1 --2- 172.21.15.47:0/2566905607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f629c0a9290 0x7f629c0b4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:50.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.480+0000 7f62a197a700 1 --2- 172.21.15.47:0/2566905607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f629c0614b0 0x7f629c0b0890 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:50.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.480+0000 7f62a197a700 1 -- 172.21.15.47:0/2566905607 >> 172.21.15.47:0/2566905607 conn(0x7f629c062df0 msgr2=0x7f629c098250 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:50.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.480+0000 7f62a197a700 1 -- 172.21.15.47:0/2566905607 shutdown_connections 2022-06-29T15:49:50.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.480+0000 7f62a197a700 1 -- 172.21.15.47:0/2566905607 wait complete. 2022-06-29T15:49:50.493 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' created 2022-06-29T15:49:50.506 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-06-29T15:49:50.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.600+0000 7fa64d6ac700 1 Processor -- start 2022-06-29T15:49:50.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.600+0000 7fa64d6ac700 1 -- start start 2022-06-29T15:49:50.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.604+0000 7fa64d6ac700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa6480b2bd0 0x7fa6480b2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:50.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.604+0000 7fa64d6ac700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa64809a060 0x7fa6480600c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:50.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.604+0000 7fa64d6ac700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa648140ae0 con 0x7fa6480b1e80 2022-06-29T15:49:50.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.604+0000 7fa64d6ac700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa648146900 con 0x7fa64809a060 2022-06-29T15:49:50.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.604+0000 7fa64d6ac700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa648146ba0 con 0x7fa6480b2bd0 2022-06-29T15:49:50.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.604+0000 7fa64ceab700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa64809a060 0x7fa6480600c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:50.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.604+0000 7fa63f7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa6480b1e80 0x7fa648099950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36944/0 (socket says 172.21.15.47:36944) 2022-06-29T15:49:50.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.604+0000 7fa63f7fe700 1 -- 172.21.15.47:0/3916689616 learned_addr learned my addr 172.21.15.47:0/3916689616 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:49:50.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.604+0000 7fa647fff700 1 --2- 172.21.15.47:0/3916689616 >> v2:172.21.15.64:3300/0 conn(0x7fa6480b2bd0 0x7fa6480b2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:50.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.604+0000 7fa64ceab700 1 -- 172.21.15.47:0/3916689616 >> v2:172.21.15.64:3300/0 conn(0x7fa6480b2bd0 msgr2=0x7fa6480b2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:50.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.604+0000 7fa64ceab700 1 --2- 172.21.15.47:0/3916689616 >> v2:172.21.15.64:3300/0 conn(0x7fa6480b2bd0 0x7fa6480b2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:50.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.604+0000 7fa64ceab700 1 -- 172.21.15.47:0/3916689616 >> v1:172.21.15.47:6789/0 conn(0x7fa6480b1e80 legacy=0x7fa648099950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:50.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.604+0000 7fa64ceab700 1 -- 172.21.15.47:0/3916689616 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa648146e40 con 0x7fa64809a060 2022-06-29T15:49:50.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.604+0000 7fa6477fe700 1 -- 172.21.15.47:0/3916689616 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2153251957 0 0) 0x7fa648140ae0 con 0x7fa6480b1e80 2022-06-29T15:49:50.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.604+0000 7fa64ceab700 1 --2- 172.21.15.47:0/3916689616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa64809a060 0x7fa6480600c0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 rx=0x7fa6400032f0 tx=0x7fa6400091d0).ready entity=mon.0 client_cookie=63fa051bd6e43b3e server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:50.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.604+0000 7fa6477fe700 1 -- 172.21.15.47:0/3916689616 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa64000b680 con 0x7fa64809a060 2022-06-29T15:49:50.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.604+0000 7fa6477fe700 1 -- 172.21.15.47:0/3916689616 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa64000b7e0 con 0x7fa64809a060 2022-06-29T15:49:50.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.604+0000 7fa6477fe700 1 -- 172.21.15.47:0/3916689616 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa64000b950 con 0x7fa64809a060 2022-06-29T15:49:50.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.604+0000 7fa64d6ac700 1 -- 172.21.15.47:0/3916689616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa64809a060 msgr2=0x7fa6480600c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:50.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.604+0000 7fa64d6ac700 1 --2- 172.21.15.47:0/3916689616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa64809a060 0x7fa6480600c0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 rx=0x7fa6400032f0 tx=0x7fa6400091d0).stop 2022-06-29T15:49:50.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.604+0000 7fa64d6ac700 1 -- 172.21.15.47:0/3916689616 shutdown_connections 2022-06-29T15:49:50.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.604+0000 7fa64d6ac700 1 --2- 172.21.15.47:0/3916689616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa64809a060 0x7fa6480600c0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:50.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.604+0000 7fa64d6ac700 1 --2- 172.21.15.47:0/3916689616 >> v2:172.21.15.64:3300/0 conn(0x7fa6480b2bd0 0x7fa6480b2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:50.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.604+0000 7fa64d6ac700 1 -- 172.21.15.47:0/3916689616 >> 172.21.15.47:0/3916689616 conn(0x7fa648062df0 msgr2=0x7fa6480631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:50.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.604+0000 7fa64d6ac700 1 -- 172.21.15.47:0/3916689616 shutdown_connections 2022-06-29T15:49:50.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.604+0000 7fa64d6ac700 1 -- 172.21.15.47:0/3916689616 wait complete. 2022-06-29T15:49:50.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.604+0000 7fa64d6ac700 1 Processor -- start 2022-06-29T15:49:50.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.604+0000 7fa64d6ac700 1 -- start start 2022-06-29T15:49:50.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.608+0000 7fa64d6ac700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa64809a060 0x7fa6481446f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:50.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.608+0000 7fa64d6ac700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa6480b1e80 0x7fa648152630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:50.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.608+0000 7fa64d6ac700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6480b2bd0 0x7fa648152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:50.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.608+0000 7fa64d6ac700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64809b2d0 con 0x7fa64809a060 2022-06-29T15:49:50.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.608+0000 7fa64d6ac700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6480b3a10 con 0x7fa6480b2bd0 2022-06-29T15:49:50.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.608+0000 7fa64d6ac700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6480b3cb0 con 0x7fa6480b1e80 2022-06-29T15:49:50.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.608+0000 7fa647fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa64809a060 0x7fa6481446f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:50.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.608+0000 7fa647fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa64809a060 0x7fa6481446f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51394/0 (socket says 172.21.15.47:51394) 2022-06-29T15:49:50.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.608+0000 7fa647fff700 1 -- 172.21.15.47:0/2032343560 learned_addr learned my addr 172.21.15.47:0/2032343560 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:50.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.608+0000 7fa64ceab700 1 --2- 172.21.15.47:0/2032343560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6480b2bd0 0x7fa648152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:50.626 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.608+0000 7fa63f7fe700 1 --2- 172.21.15.47:0/2032343560 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa6480b1e80 0x7fa648152630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:50.626 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.608+0000 7fa647fff700 1 -- 172.21.15.47:0/2032343560 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa6480b1e80 msgr2=0x7fa648152630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:50.626 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.608+0000 7fa647fff700 1 --2- 172.21.15.47:0/2032343560 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa6480b1e80 0x7fa648152630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:50.627 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.608+0000 7fa647fff700 1 -- 172.21.15.47:0/2032343560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6480b2bd0 msgr2=0x7fa648152b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:50.627 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.608+0000 7fa647fff700 1 --2- 172.21.15.47:0/2032343560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6480b2bd0 0x7fa648152b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:50.627 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.608+0000 7fa647fff700 1 -- 172.21.15.47:0/2032343560 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa640009470 con 0x7fa64809a060 2022-06-29T15:49:50.627 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.608+0000 7fa647fff700 1 --2- 172.21.15.47:0/2032343560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa64809a060 0x7fa6481446f0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 rx=0x7fa63800c000 tx=0x7fa638006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:50.627 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.608+0000 7fa645ffb700 1 -- 172.21.15.47:0/2032343560 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa638011750 con 0x7fa64809a060 2022-06-29T15:49:50.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.608+0000 7fa645ffb700 1 -- 172.21.15.47:0/2032343560 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6380118b0 con 0x7fa64809a060 2022-06-29T15:49:50.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.608+0000 7fa64d6ac700 1 -- 172.21.15.47:0/2032343560 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6480b3f50 con 0x7fa64809a060 2022-06-29T15:49:50.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.608+0000 7fa645ffb700 1 -- 172.21.15.47:0/2032343560 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa638011c10 con 0x7fa64809a060 2022-06-29T15:49:50.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.608+0000 7fa64d6ac700 1 -- 172.21.15.47:0/2032343560 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa648152190 con 0x7fa64809a060 2022-06-29T15:49:50.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.608+0000 7fa64d6ac700 1 -- 172.21.15.47:0/2032343560 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6480573d0 con 0x7fa64809a060 2022-06-29T15:49:50.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.612+0000 7fa645ffb700 1 -- 172.21.15.47:0/2032343560 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa638011d70 con 0x7fa64809a060 2022-06-29T15:49:50.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.612+0000 7fa645ffb700 1 --2- 172.21.15.47:0/2032343560 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa62c053b00 0x7fa62c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:50.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.612+0000 7fa645ffb700 1 -- 172.21.15.47:0/2032343560 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6304+0+0 (secure 0 0 0) 0x7fa63806cf40 con 0x7fa64809a060 2022-06-29T15:49:50.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.612+0000 7fa63f7fe700 1 --2- 172.21.15.47:0/2032343560 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa62c053b00 0x7fa62c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:50.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.612+0000 7fa63f7fe700 1 --2- 172.21.15.47:0/2032343560 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa62c053b00 0x7fa62c055fc0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7fa63000d000 tx=0x7fa6300011a0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:50.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.616+0000 7fa645ffb700 1 -- 172.21.15.47:0/2032343560 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa638043d20 con 0x7fa64809a060 2022-06-29T15:49:50.790 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.780+0000 7fa64d6ac700 1 -- 172.21.15.47:0/2032343560 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fa64813b6b0 con 0x7fa64809a060 2022-06-29T15:49:50.798 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.788+0000 7fa645ffb700 1 -- 172.21.15.47:0/2032343560 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 17 and data pool 18 v44) v1 ==== 189+0+0 (secure 0 0 0) 0x7fa63802a0d0 con 0x7fa64809a060 2022-06-29T15:49:50.804 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.796+0000 7fa64d6ac700 1 -- 172.21.15.47:0/2032343560 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa62c053b00 msgr2=0x7fa62c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:50.805 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.796+0000 7fa64d6ac700 1 --2- 172.21.15.47:0/2032343560 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa62c053b00 0x7fa62c055fc0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7fa63000d000 tx=0x7fa6300011a0).stop 2022-06-29T15:49:50.805 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.796+0000 7fa64d6ac700 1 -- 172.21.15.47:0/2032343560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa64809a060 msgr2=0x7fa6481446f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:50.805 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.796+0000 7fa64d6ac700 1 --2- 172.21.15.47:0/2032343560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa64809a060 0x7fa6481446f0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 rx=0x7fa63800c000 tx=0x7fa638006df0).stop 2022-06-29T15:49:50.806 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.796+0000 7fa64d6ac700 1 -- 172.21.15.47:0/2032343560 shutdown_connections 2022-06-29T15:49:50.806 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.796+0000 7fa64d6ac700 1 --2- 172.21.15.47:0/2032343560 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa62c053b00 0x7fa62c055fc0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:50.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.796+0000 7fa64d6ac700 1 --2- 172.21.15.47:0/2032343560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6480b2bd0 0x7fa648152b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:50.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.796+0000 7fa64d6ac700 1 --2- 172.21.15.47:0/2032343560 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa6480b1e80 0x7fa648152630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:50.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.796+0000 7fa64d6ac700 1 --2- 172.21.15.47:0/2032343560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa64809a060 0x7fa6481446f0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:50.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.796+0000 7fa64d6ac700 1 -- 172.21.15.47:0/2032343560 >> 172.21.15.47:0/2032343560 conn(0x7fa648062df0 msgr2=0x7fa648145640 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:50.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.796+0000 7fa64d6ac700 1 -- 172.21.15.47:0/2032343560 shutdown_connections 2022-06-29T15:49:50.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.796+0000 7fa64d6ac700 1 -- 172.21.15.47:0/2032343560 wait complete. 2022-06-29T15:49:50.808 INFO:teuthology.orchestra.run.smithi047.stderr:new fs with metadata pool 17 and data pool 18 2022-06-29T15:49:50.819 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:49:50.913 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.904+0000 7ffb4359e700 1 Processor -- start 2022-06-29T15:49:50.914 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.904+0000 7ffb4359e700 1 -- start start 2022-06-29T15:49:50.916 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.908+0000 7ffb4359e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ffb440acff0 0x7ffb4405fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:50.916 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.908+0000 7ffb4359e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ffb441437d0 0x7ffb44143c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:50.917 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.908+0000 7ffb4359e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb440a97e0 con 0x7ffb440b19a0 2022-06-29T15:49:50.917 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.908+0000 7ffb4359e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb440b1d80 con 0x7ffb441437d0 2022-06-29T15:49:50.917 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.908+0000 7ffb4359e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb4405f430 con 0x7ffb440acff0 2022-06-29T15:49:50.917 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.908+0000 7ffb42d9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ffb441437d0 0x7ffb44143c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:50.918 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.908+0000 7ffb42d9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ffb441437d0 0x7ffb44143c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51406/0 (socket says 172.21.15.47:51406) 2022-06-29T15:49:50.918 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.908+0000 7ffb42d9d700 1 -- 172.21.15.47:0/359828672 learned_addr learned my addr 172.21.15.47:0/359828672 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:50.918 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.908+0000 7ffb3bfff700 1 --2- 172.21.15.47:0/359828672 >> v2:172.21.15.64:3300/0 conn(0x7ffb440acff0 0x7ffb4405fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:50.918 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.908+0000 7ffb42d9d700 1 -- 172.21.15.47:0/359828672 >> v2:172.21.15.64:3300/0 conn(0x7ffb440acff0 msgr2=0x7ffb4405fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:50.919 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.908+0000 7ffb42d9d700 1 --2- 172.21.15.47:0/359828672 >> v2:172.21.15.64:3300/0 conn(0x7ffb440acff0 0x7ffb4405fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:50.919 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.908+0000 7ffb42d9d700 1 -- 172.21.15.47:0/359828672 >> v1:172.21.15.64:6789/0 conn(0x7ffb440b19a0 legacy=0x7ffb440ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:50.919 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.908+0000 7ffb42d9d700 1 -- 172.21.15.47:0/359828672 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb440a92a0 con 0x7ffb441437d0 2022-06-29T15:49:50.919 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.908+0000 7ffb42d9d700 1 --2- 172.21.15.47:0/359828672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb441437d0 0x7ffb44143c60 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 rx=0x7ffb3c0032f0 tx=0x7ffb3c009290).ready entity=mon.0 client_cookie=ad2cc413774966f6 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:50.921 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.908+0000 7ffb41d9b700 1 -- 172.21.15.47:0/359828672 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb3c00b870 con 0x7ffb441437d0 2022-06-29T15:49:50.921 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.908+0000 7ffb41d9b700 1 -- 172.21.15.47:0/359828672 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb3c00b9d0 con 0x7ffb441437d0 2022-06-29T15:49:50.921 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.908+0000 7ffb41d9b700 1 -- 172.21.15.47:0/359828672 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb3c00bb40 con 0x7ffb441437d0 2022-06-29T15:49:50.922 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.908+0000 7ffb4359e700 1 -- 172.21.15.47:0/359828672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb441437d0 msgr2=0x7ffb44143c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:50.922 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.908+0000 7ffb4359e700 1 --2- 172.21.15.47:0/359828672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb441437d0 0x7ffb44143c60 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 rx=0x7ffb3c0032f0 tx=0x7ffb3c009290).stop 2022-06-29T15:49:50.922 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.908+0000 7ffb4359e700 1 -- 172.21.15.47:0/359828672 shutdown_connections 2022-06-29T15:49:50.923 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.908+0000 7ffb4359e700 1 --2- 172.21.15.47:0/359828672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb441437d0 0x7ffb44143c60 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:50.923 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.908+0000 7ffb4359e700 1 --2- 172.21.15.47:0/359828672 >> v2:172.21.15.64:3300/0 conn(0x7ffb440acff0 0x7ffb4405fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:50.923 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.908+0000 7ffb4359e700 1 -- 172.21.15.47:0/359828672 >> 172.21.15.47:0/359828672 conn(0x7ffb44062df0 msgr2=0x7ffb440631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:50.923 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.908+0000 7ffb4359e700 1 -- 172.21.15.47:0/359828672 shutdown_connections 2022-06-29T15:49:50.924 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.908+0000 7ffb4359e700 1 -- 172.21.15.47:0/359828672 wait complete. 2022-06-29T15:49:50.924 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.912+0000 7ffb4359e700 1 Processor -- start 2022-06-29T15:49:50.924 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.912+0000 7ffb4359e700 1 -- start start 2022-06-29T15:49:50.927 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.912+0000 7ffb4359e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffb440acff0 0x7ffb440ae980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:50.927 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.912+0000 7ffb4359e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb440b19a0 0x7ffb44152120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:50.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.912+0000 7ffb4359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb441437d0 0x7ffb44155ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:50.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.912+0000 7ffb4359e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb440aa970 con 0x7ffb440b19a0 2022-06-29T15:49:50.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.912+0000 7ffb4359e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb44152610 con 0x7ffb441437d0 2022-06-29T15:49:50.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.912+0000 7ffb4359e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb441557f0 con 0x7ffb440acff0 2022-06-29T15:49:50.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.912+0000 7ffb3bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb440b19a0 0x7ffb44152120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:50.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.912+0000 7ffb3bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb440b19a0 0x7ffb44152120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51410/0 (socket says 172.21.15.47:51410) 2022-06-29T15:49:50.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.912+0000 7ffb3bfff700 1 -- 172.21.15.47:0/138272714 learned_addr learned my addr 172.21.15.47:0/138272714 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:50.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.912+0000 7ffb4259c700 1 --2- 172.21.15.47:0/138272714 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffb440acff0 0x7ffb440ae980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:50.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.912+0000 7ffb42d9d700 1 --2- 172.21.15.47:0/138272714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb441437d0 0x7ffb44155ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:50.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.912+0000 7ffb3bfff700 1 -- 172.21.15.47:0/138272714 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffb440acff0 msgr2=0x7ffb440ae980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:50.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.912+0000 7ffb3bfff700 1 --2- 172.21.15.47:0/138272714 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffb440acff0 0x7ffb440ae980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:50.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.912+0000 7ffb3bfff700 1 -- 172.21.15.47:0/138272714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb441437d0 msgr2=0x7ffb44155ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:50.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.912+0000 7ffb3bfff700 1 --2- 172.21.15.47:0/138272714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb441437d0 0x7ffb44155ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:50.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.912+0000 7ffb3bfff700 1 -- 172.21.15.47:0/138272714 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb3c009530 con 0x7ffb440b19a0 2022-06-29T15:49:50.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.912+0000 7ffb3bfff700 1 --2- 172.21.15.47:0/138272714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb440b19a0 0x7ffb44152120 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 rx=0x7ffb2c00c000 tx=0x7ffb2c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:50.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.912+0000 7ffb3b7fe700 1 -- 172.21.15.47:0/138272714 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb2c010780 con 0x7ffb440b19a0 2022-06-29T15:49:50.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.912+0000 7ffb3b7fe700 1 -- 172.21.15.47:0/138272714 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb2c0108e0 con 0x7ffb440b19a0 2022-06-29T15:49:50.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.912+0000 7ffb4359e700 1 -- 172.21.15.47:0/138272714 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb440aa6d0 con 0x7ffb440b19a0 2022-06-29T15:49:50.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.912+0000 7ffb3b7fe700 1 -- 172.21.15.47:0/138272714 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb2c010c40 con 0x7ffb440b19a0 2022-06-29T15:49:50.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.912+0000 7ffb4359e700 1 -- 172.21.15.47:0/138272714 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb440962a0 con 0x7ffb440b19a0 2022-06-29T15:49:50.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.916+0000 7ffb4359e700 1 -- 172.21.15.47:0/138272714 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb440573d0 con 0x7ffb440b19a0 2022-06-29T15:49:50.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.916+0000 7ffb3b7fe700 1 -- 172.21.15.47:0/138272714 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ffb2c01c460 con 0x7ffb440b19a0 2022-06-29T15:49:50.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.916+0000 7ffb3b7fe700 1 --2- 172.21.15.47:0/138272714 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ffb28053aa0 0x7ffb28055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:50.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.916+0000 7ffb3b7fe700 1 -- 172.21.15.47:0/138272714 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ffb2c06c3d0 con 0x7ffb440b19a0 2022-06-29T15:49:50.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.916+0000 7ffb4259c700 1 --2- 172.21.15.47:0/138272714 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ffb28053aa0 0x7ffb28055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:50.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.916+0000 7ffb4259c700 1 --2- 172.21.15.47:0/138272714 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ffb28053aa0 0x7ffb28055f60 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7ffb3400d000 tx=0x7ffb34007b90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:50.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:50.924+0000 7ffb3b7fe700 1 -- 172.21.15.47:0/138272714 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ffb2c043130 con 0x7ffb440b19a0 2022-06-29T15:49:51.062 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.052+0000 7ffb4359e700 1 -- 172.21.15.47:0/138272714 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffb4406a0f0 con 0x7ffb440b19a0 2022-06-29T15:49:51.064 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.056+0000 7ffb3b7fe700 1 -- 172.21.15.47:0/138272714 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+16290 (secure 0 0 0) 0x7ffb2c046750 con 0x7ffb440b19a0 2022-06-29T15:49:51.065 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:49:51.065 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":100,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:49:50.788199+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:49:48.525959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2022-06-29T15:49:49.805331+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:49:51.067 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.056+0000 7ffb4359e700 1 -- 172.21.15.47:0/138272714 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ffb28053aa0 msgr2=0x7ffb28055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:51.067 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.056+0000 7ffb4359e700 1 --2- 172.21.15.47:0/138272714 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ffb28053aa0 0x7ffb28055f60 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7ffb3400d000 tx=0x7ffb34007b90).stop 2022-06-29T15:49:51.067 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.056+0000 7ffb4359e700 1 -- 172.21.15.47:0/138272714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb440b19a0 msgr2=0x7ffb44152120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:51.068 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.056+0000 7ffb4359e700 1 --2- 172.21.15.47:0/138272714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb440b19a0 0x7ffb44152120 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 rx=0x7ffb2c00c000 tx=0x7ffb2c006df0).stop 2022-06-29T15:49:51.068 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.056+0000 7ffb4359e700 1 -- 172.21.15.47:0/138272714 shutdown_connections 2022-06-29T15:49:51.068 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.056+0000 7ffb4359e700 1 --2- 172.21.15.47:0/138272714 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ffb28053aa0 0x7ffb28055f60 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:51.068 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.056+0000 7ffb4359e700 1 --2- 172.21.15.47:0/138272714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb441437d0 0x7ffb44155ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:51.069 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.056+0000 7ffb4359e700 1 --2- 172.21.15.47:0/138272714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb440b19a0 0x7ffb44152120 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:51.069 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.056+0000 7ffb4359e700 1 --2- 172.21.15.47:0/138272714 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffb440acff0 0x7ffb440ae980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:51.069 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.056+0000 7ffb4359e700 1 -- 172.21.15.47:0/138272714 >> 172.21.15.47:0/138272714 conn(0x7ffb44062df0 msgr2=0x7ffb440b2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:51.069 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.056+0000 7ffb4359e700 1 -- 172.21.15.47:0/138272714 shutdown_connections 2022-06-29T15:49:51.069 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.056+0000 7ffb4359e700 1 -- 172.21.15.47:0/138272714 wait complete. 2022-06-29T15:49:51.078 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:49:51.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.176+0000 7ff14ca12700 1 Processor -- start 2022-06-29T15:49:51.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.176+0000 7ff14ca12700 1 -- start start 2022-06-29T15:49:51.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.176+0000 7ff14ca12700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff1480b7840 0x7ff1480b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:51.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.176+0000 7ff14ca12700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff148066c00 0x7ff148141760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:51.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.176+0000 7ff14ca12700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1480b7c20 con 0x7ff148062dd0 2022-06-29T15:49:51.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.176+0000 7ff14ca12700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1480b3d10 con 0x7ff1480b7840 2022-06-29T15:49:51.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.176+0000 7ff14ca12700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff148067380 con 0x7ff148066c00 2022-06-29T15:49:51.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.176+0000 7ff13ffff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff148062dd0 0x7ff1480aea80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36970/0 (socket says 172.21.15.47:36970) 2022-06-29T15:49:51.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.176+0000 7ff13ffff700 1 -- 172.21.15.47:0/2815752942 learned_addr learned my addr 172.21.15.47:0/2815752942 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:49:51.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.176+0000 7ff1477fe700 1 --2- 172.21.15.47:0/2815752942 >> v2:172.21.15.47:3300/0 conn(0x7ff1480b7840 0x7ff1480b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:51.191 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.176+0000 7ff147fff700 1 --2- 172.21.15.47:0/2815752942 >> v2:172.21.15.64:3300/0 conn(0x7ff148066c00 0x7ff148141760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:51.191 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.176+0000 7ff146ffd700 1 -- 172.21.15.47:0/2815752942 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 673859475 0 0) 0x7ff1480b7c20 con 0x7ff148062dd0 2022-06-29T15:49:51.191 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.176+0000 7ff146ffd700 1 -- 172.21.15.47:0/2815752942 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff128003260 con 0x7ff148062dd0 2022-06-29T15:49:51.191 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.176+0000 7ff1477fe700 1 -- 172.21.15.47:0/2815752942 >> v2:172.21.15.64:3300/0 conn(0x7ff148066c00 msgr2=0x7ff148141760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:51.192 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.176+0000 7ff1477fe700 1 --2- 172.21.15.47:0/2815752942 >> v2:172.21.15.64:3300/0 conn(0x7ff148066c00 0x7ff148141760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:51.192 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.176+0000 7ff1477fe700 1 -- 172.21.15.47:0/2815752942 >> v1:172.21.15.47:6789/0 conn(0x7ff148062dd0 legacy=0x7ff1480aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:51.192 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.176+0000 7ff1477fe700 1 -- 172.21.15.47:0/2815752942 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1480b30e0 con 0x7ff1480b7840 2022-06-29T15:49:51.192 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.180+0000 7ff1477fe700 1 --2- 172.21.15.47:0/2815752942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1480b7840 0x7ff1480b3820 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 rx=0x7ff138002f70 tx=0x7ff138006df0).ready entity=mon.0 client_cookie=9803f406770644da server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:51.193 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.180+0000 7ff146ffd700 1 -- 172.21.15.47:0/2815752942 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff13800d4f0 con 0x7ff1480b7840 2022-06-29T15:49:51.193 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.180+0000 7ff146ffd700 1 -- 172.21.15.47:0/2815752942 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff138000e90 con 0x7ff1480b7840 2022-06-29T15:49:51.193 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.180+0000 7ff146ffd700 1 -- 172.21.15.47:0/2815752942 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff13800d840 con 0x7ff1480b7840 2022-06-29T15:49:51.193 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.180+0000 7ff14ca12700 1 -- 172.21.15.47:0/2815752942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1480b7840 msgr2=0x7ff1480b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:51.194 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.180+0000 7ff14ca12700 1 --2- 172.21.15.47:0/2815752942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1480b7840 0x7ff1480b3820 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 rx=0x7ff138002f70 tx=0x7ff138006df0).stop 2022-06-29T15:49:51.194 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.180+0000 7ff14ca12700 1 -- 172.21.15.47:0/2815752942 shutdown_connections 2022-06-29T15:49:51.194 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.180+0000 7ff14ca12700 1 --2- 172.21.15.47:0/2815752942 >> v2:172.21.15.64:3300/0 conn(0x7ff148066c00 0x7ff148141760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:51.194 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.180+0000 7ff14ca12700 1 --2- 172.21.15.47:0/2815752942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1480b7840 0x7ff1480b3820 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:51.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.180+0000 7ff14ca12700 1 -- 172.21.15.47:0/2815752942 >> 172.21.15.47:0/2815752942 conn(0x7ff14805f480 msgr2=0x7ff1480647d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:51.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.180+0000 7ff14ca12700 1 -- 172.21.15.47:0/2815752942 shutdown_connections 2022-06-29T15:49:51.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.180+0000 7ff14ca12700 1 -- 172.21.15.47:0/2815752942 wait complete. 2022-06-29T15:49:51.198 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.180+0000 7ff14ca12700 1 Processor -- start 2022-06-29T15:49:51.198 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.180+0000 7ff14ca12700 1 -- start start 2022-06-29T15:49:51.198 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.184+0000 7ff14ca12700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff148062dd0 0x7ff148140bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:51.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.184+0000 7ff14ca12700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff148066c00 0x7ff148153b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:51.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.184+0000 7ff14ca12700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff1480b7840 0x7ff148154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:51.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.184+0000 7ff14ca12700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff148151010 con 0x7ff148066c00 2022-06-29T15:49:51.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.184+0000 7ff14ca12700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff148145ed0 con 0x7ff148062dd0 2022-06-29T15:49:51.200 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.184+0000 7ff14ca12700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff148146100 con 0x7ff1480b7840 2022-06-29T15:49:51.200 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.184+0000 7ff13ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff148066c00 0x7ff148153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:51.200 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.184+0000 7ff13ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff148066c00 0x7ff148153b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51424/0 (socket says 172.21.15.47:51424) 2022-06-29T15:49:51.200 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.184+0000 7ff13ffff700 1 -- 172.21.15.47:0/3494906962 learned_addr learned my addr 172.21.15.47:0/3494906962 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:51.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.184+0000 7ff1477fe700 1 --2- 172.21.15.47:0/3494906962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff148062dd0 0x7ff148140bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:51.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.184+0000 7ff147fff700 1 --2- 172.21.15.47:0/3494906962 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff1480b7840 0x7ff148154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:51.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.184+0000 7ff13ffff700 1 -- 172.21.15.47:0/3494906962 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff1480b7840 msgr2=0x7ff148154060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:51.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.184+0000 7ff13ffff700 1 --2- 172.21.15.47:0/3494906962 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff1480b7840 0x7ff148154060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:51.202 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.184+0000 7ff13ffff700 1 -- 172.21.15.47:0/3494906962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff148062dd0 msgr2=0x7ff148140bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:51.202 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.184+0000 7ff13ffff700 1 --2- 172.21.15.47:0/3494906962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff148062dd0 0x7ff148140bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:51.202 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.184+0000 7ff13ffff700 1 -- 172.21.15.47:0/3494906962 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff13800b180 con 0x7ff148066c00 2022-06-29T15:49:51.202 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.184+0000 7ff13ffff700 1 --2- 172.21.15.47:0/3494906962 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff148066c00 0x7ff148153b70 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 rx=0x7ff130001110 tx=0x7ff130007aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:51.202 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.184+0000 7ff1457fa700 1 -- 172.21.15.47:0/3494906962 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff130007e00 con 0x7ff148066c00 2022-06-29T15:49:51.203 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.184+0000 7ff1457fa700 1 -- 172.21.15.47:0/3494906962 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1300072e0 con 0x7ff148066c00 2022-06-29T15:49:51.203 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.184+0000 7ff14ca12700 1 -- 172.21.15.47:0/3494906962 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1481463a0 con 0x7ff148066c00 2022-06-29T15:49:51.203 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.184+0000 7ff1457fa700 1 -- 172.21.15.47:0/3494906962 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff130005800 con 0x7ff148066c00 2022-06-29T15:49:51.203 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.184+0000 7ff14ca12700 1 -- 172.21.15.47:0/3494906962 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1480a20e0 con 0x7ff148066c00 2022-06-29T15:49:51.204 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.184+0000 7ff14ca12700 1 -- 172.21.15.47:0/3494906962 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1480573d0 con 0x7ff148066c00 2022-06-29T15:49:51.204 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.184+0000 7ff1457fa700 1 -- 172.21.15.47:0/3494906962 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff130010070 con 0x7ff148066c00 2022-06-29T15:49:51.204 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.188+0000 7ff1457fa700 1 --2- 172.21.15.47:0/3494906962 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff12c0539e0 0x7ff12c055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:51.204 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.188+0000 7ff1457fa700 1 -- 172.21.15.47:0/3494906962 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff13006b6e0 con 0x7ff148066c00 2022-06-29T15:49:51.204 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.188+0000 7ff1477fe700 1 --2- 172.21.15.47:0/3494906962 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff12c0539e0 0x7ff12c055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:51.205 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.188+0000 7ff1477fe700 1 --2- 172.21.15.47:0/3494906962 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff12c0539e0 0x7ff12c055ea0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7ff138018000 tx=0x7ff138014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:51.205 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.192+0000 7ff1457fa700 1 -- 172.21.15.47:0/3494906962 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff1300424a0 con 0x7ff148066c00 2022-06-29T15:49:51.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.320+0000 7ff14ca12700 1 -- 172.21.15.47:0/3494906962 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff148068830 con 0x7ff148066c00 2022-06-29T15:49:51.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.320+0000 7ff1457fa700 1 -- 172.21.15.47:0/3494906962 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+16290 (secure 0 0 0) 0x7ff130045ac0 con 0x7ff148066c00 2022-06-29T15:49:51.331 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:49:51.332 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":100,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:49:50.788199+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:49:48.525959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2022-06-29T15:49:49.805331+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:49:51.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.324+0000 7ff14ca12700 1 -- 172.21.15.47:0/3494906962 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff12c0539e0 msgr2=0x7ff12c055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:51.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.324+0000 7ff14ca12700 1 --2- 172.21.15.47:0/3494906962 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff12c0539e0 0x7ff12c055ea0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7ff138018000 tx=0x7ff138014000).stop 2022-06-29T15:49:51.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.324+0000 7ff14ca12700 1 -- 172.21.15.47:0/3494906962 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff148066c00 msgr2=0x7ff148153b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:51.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.324+0000 7ff14ca12700 1 --2- 172.21.15.47:0/3494906962 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff148066c00 0x7ff148153b70 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 rx=0x7ff130001110 tx=0x7ff130007aa0).stop 2022-06-29T15:49:51.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.324+0000 7ff14ca12700 1 -- 172.21.15.47:0/3494906962 shutdown_connections 2022-06-29T15:49:51.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.324+0000 7ff14ca12700 1 --2- 172.21.15.47:0/3494906962 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff1480b7840 0x7ff148154060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:51.335 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.324+0000 7ff14ca12700 1 --2- 172.21.15.47:0/3494906962 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff148066c00 0x7ff148153b70 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:51.335 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.324+0000 7ff14ca12700 1 --2- 172.21.15.47:0/3494906962 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff12c0539e0 0x7ff12c055ea0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:51.335 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.324+0000 7ff14ca12700 1 --2- 172.21.15.47:0/3494906962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff148062dd0 0x7ff148140bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:51.335 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.324+0000 7ff14ca12700 1 -- 172.21.15.47:0/3494906962 >> 172.21.15.47:0/3494906962 conn(0x7ff14805f480 msgr2=0x7ff1480647d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:51.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.324+0000 7ff14ca12700 1 -- 172.21.15.47:0/3494906962 shutdown_connections 2022-06-29T15:49:51.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.324+0000 7ff14ca12700 1 -- 172.21.15.47:0/3494906962 wait complete. 2022-06-29T15:49:51.345 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-06-29T15:49:51.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.448+0000 7f4b6cf3a700 1 Processor -- start 2022-06-29T15:49:51.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.448+0000 7f4b6cf3a700 1 -- start start 2022-06-29T15:49:51.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.448+0000 7f4b6cf3a700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4b680b01d0 0x7f4b680b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:51.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.448+0000 7f4b6cf3a700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4b6805fd60 0x7f4b680b3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:51.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.448+0000 7f4b6cf3a700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b680b1b50 con 0x7f4b68060c10 2022-06-29T15:49:51.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.448+0000 7f4b6cf3a700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b68063510 con 0x7f4b680b01d0 2022-06-29T15:49:51.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.448+0000 7f4b6cf3a700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b68060ff0 con 0x7f4b6805fd60 2022-06-29T15:49:51.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.448+0000 7f4b6659c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4b68060c10 0x7f4b680ae430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36984/0 (socket says 172.21.15.47:36984) 2022-06-29T15:49:51.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.448+0000 7f4b6659c700 1 -- 172.21.15.47:0/4133209244 learned_addr learned my addr 172.21.15.47:0/4133209244 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:49:51.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.448+0000 7f4b66d9d700 1 --2- 172.21.15.47:0/4133209244 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4b6805fd60 0x7f4b680b3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:51.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.448+0000 7f4b65d9b700 1 --2- 172.21.15.47:0/4133209244 >> v2:172.21.15.64:3300/0 conn(0x7f4b680b01d0 0x7f4b680b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:51.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.448+0000 7f4b6559a700 1 -- 172.21.15.47:0/4133209244 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4103197287 0 0) 0x7f4b58001490 con 0x7f4b68060c10 2022-06-29T15:49:51.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.448+0000 7f4b6559a700 1 -- 172.21.15.47:0/4133209244 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b48003260 con 0x7f4b68060c10 2022-06-29T15:49:51.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.448+0000 7f4b66d9d700 1 -- 172.21.15.47:0/4133209244 >> v2:172.21.15.64:3300/0 conn(0x7f4b680b01d0 msgr2=0x7f4b680b2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:51.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.448+0000 7f4b66d9d700 1 --2- 172.21.15.47:0/4133209244 >> v2:172.21.15.64:3300/0 conn(0x7f4b680b01d0 0x7f4b680b2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:51.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.448+0000 7f4b66d9d700 1 -- 172.21.15.47:0/4133209244 >> v1:172.21.15.47:6789/0 conn(0x7f4b68060c10 legacy=0x7f4b680ae430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:51.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.448+0000 7f4b66d9d700 1 -- 172.21.15.47:0/4133209244 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b6805f350 con 0x7f4b6805fd60 2022-06-29T15:49:51.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.448+0000 7f4b66d9d700 1 --2- 172.21.15.47:0/4133209244 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4b6805fd60 0x7f4b680b3230 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f4b60002f70 tx=0x7f4b60006df0).ready entity=mon.2 client_cookie=ae65aeb971d02e33 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:51.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.448+0000 7f4b6559a700 1 -- 172.21.15.47:0/4133209244 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b6000d4a0 con 0x7f4b6805fd60 2022-06-29T15:49:51.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.448+0000 7f4b6559a700 1 -- 172.21.15.47:0/4133209244 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b60000e90 con 0x7f4b6805fd60 2022-06-29T15:49:51.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.448+0000 7f4b6559a700 1 -- 172.21.15.47:0/4133209244 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b6000d7f0 con 0x7f4b6805fd60 2022-06-29T15:49:51.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.452+0000 7f4b6cf3a700 1 -- 172.21.15.47:0/4133209244 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4b6805fd60 msgr2=0x7f4b680b3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:51.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.452+0000 7f4b6cf3a700 1 --2- 172.21.15.47:0/4133209244 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4b6805fd60 0x7f4b680b3230 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f4b60002f70 tx=0x7f4b60006df0).stop 2022-06-29T15:49:51.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.452+0000 7f4b6cf3a700 1 -- 172.21.15.47:0/4133209244 shutdown_connections 2022-06-29T15:49:51.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.452+0000 7f4b6cf3a700 1 --2- 172.21.15.47:0/4133209244 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4b6805fd60 0x7f4b680b3230 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:51.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.452+0000 7f4b6cf3a700 1 --2- 172.21.15.47:0/4133209244 >> v2:172.21.15.64:3300/0 conn(0x7f4b680b01d0 0x7f4b680b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:51.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.452+0000 7f4b6cf3a700 1 -- 172.21.15.47:0/4133209244 >> 172.21.15.47:0/4133209244 conn(0x7f4b68062110 msgr2=0x7f4b680601c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:51.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.452+0000 7f4b6cf3a700 1 -- 172.21.15.47:0/4133209244 shutdown_connections 2022-06-29T15:49:51.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.452+0000 7f4b6cf3a700 1 -- 172.21.15.47:0/4133209244 wait complete. 2022-06-29T15:49:51.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.452+0000 7f4b6cf3a700 1 Processor -- start 2022-06-29T15:49:51.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.452+0000 7f4b6cf3a700 1 -- start start 2022-06-29T15:49:51.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.452+0000 7f4b6cf3a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b6805fd60 0x7f4b680a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:51.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.452+0000 7f4b6cf3a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b68060c10 0x7f4b680a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:51.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.452+0000 7f4b6cf3a700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4b680b01d0 0x7f4b680acd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:51.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.452+0000 7f4b6cf3a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b680a1370 con 0x7f4b68060c10 2022-06-29T15:49:51.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.452+0000 7f4b6cf3a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b680a3e50 con 0x7f4b6805fd60 2022-06-29T15:49:51.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.452+0000 7f4b6cf3a700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b680a40f0 con 0x7f4b680b01d0 2022-06-29T15:49:51.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.452+0000 7f4b65d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b68060c10 0x7f4b680a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:51.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.452+0000 7f4b65d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b68060c10 0x7f4b680a5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51438/0 (socket says 172.21.15.47:51438) 2022-06-29T15:49:51.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.452+0000 7f4b65d9b700 1 -- 172.21.15.47:0/3102782827 learned_addr learned my addr 172.21.15.47:0/3102782827 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:51.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.452+0000 7f4b6659c700 1 --2- 172.21.15.47:0/3102782827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b6805fd60 0x7f4b680a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:51.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.452+0000 7f4b66d9d700 1 --2- 172.21.15.47:0/3102782827 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4b680b01d0 0x7f4b680acd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:51.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.456+0000 7f4b65d9b700 1 -- 172.21.15.47:0/3102782827 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4b680b01d0 msgr2=0x7f4b680acd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:51.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.456+0000 7f4b65d9b700 1 --2- 172.21.15.47:0/3102782827 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4b680b01d0 0x7f4b680acd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:51.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.456+0000 7f4b65d9b700 1 -- 172.21.15.47:0/3102782827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b6805fd60 msgr2=0x7f4b680a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:51.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.456+0000 7f4b65d9b700 1 --2- 172.21.15.47:0/3102782827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b6805fd60 0x7f4b680a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:51.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.456+0000 7f4b65d9b700 1 -- 172.21.15.47:0/3102782827 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b6000b180 con 0x7f4b68060c10 2022-06-29T15:49:51.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.456+0000 7f4b65d9b700 1 --2- 172.21.15.47:0/3102782827 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b68060c10 0x7f4b680a5060 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 rx=0x7f4b5000c000 tx=0x7f4b50006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:51.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.456+0000 7f4b5f7fe700 1 -- 172.21.15.47:0/3102782827 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b50010780 con 0x7f4b68060c10 2022-06-29T15:49:51.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.456+0000 7f4b5f7fe700 1 -- 172.21.15.47:0/3102782827 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b500108e0 con 0x7f4b68060c10 2022-06-29T15:49:51.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.456+0000 7f4b6cf3a700 1 -- 172.21.15.47:0/3102782827 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b680a3050 con 0x7f4b68060c10 2022-06-29T15:49:51.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.456+0000 7f4b5f7fe700 1 -- 172.21.15.47:0/3102782827 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b50010c40 con 0x7f4b68060c10 2022-06-29T15:49:51.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.456+0000 7f4b6cf3a700 1 -- 172.21.15.47:0/3102782827 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b680a7690 con 0x7f4b68060c10 2022-06-29T15:49:51.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.456+0000 7f4b5f7fe700 1 -- 172.21.15.47:0/3102782827 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4b5001c460 con 0x7f4b68060c10 2022-06-29T15:49:51.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.456+0000 7f4b6cf3a700 1 -- 172.21.15.47:0/3102782827 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b680573d0 con 0x7f4b68060c10 2022-06-29T15:49:51.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.456+0000 7f4b5f7fe700 1 --2- 172.21.15.47:0/3102782827 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4b4c053aa0 0x7f4b4c055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:51.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.456+0000 7f4b6659c700 1 --2- 172.21.15.47:0/3102782827 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4b4c053aa0 0x7f4b4c055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:51.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.456+0000 7f4b5f7fe700 1 -- 172.21.15.47:0/3102782827 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f4b5006c550 con 0x7f4b68060c10 2022-06-29T15:49:51.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.456+0000 7f4b6659c700 1 --2- 172.21.15.47:0/3102782827 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4b4c053aa0 0x7f4b4c055f60 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f4b5800d000 tx=0x7f4b58000f40).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:51.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.460+0000 7f4b5f7fe700 1 -- 172.21.15.47:0/3102782827 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4b500431b0 con 0x7f4b68060c10 2022-06-29T15:49:51.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.628+0000 7f4b6cf3a700 1 -- 172.21.15.47:0/3102782827 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f4b680008d0 con 0x7f4b68060c10 2022-06-29T15:49:51.803 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.792+0000 7f4b5f7fe700 1 -- 172.21.15.47:0/3102782827 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v45) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4b500467d0 con 0x7f4b68060c10 2022-06-29T15:49:51.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.796+0000 7f4b6cf3a700 1 -- 172.21.15.47:0/3102782827 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4b4c053aa0 msgr2=0x7f4b4c055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:51.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.796+0000 7f4b6cf3a700 1 --2- 172.21.15.47:0/3102782827 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4b4c053aa0 0x7f4b4c055f60 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f4b5800d000 tx=0x7f4b58000f40).stop 2022-06-29T15:49:51.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.796+0000 7f4b6cf3a700 1 -- 172.21.15.47:0/3102782827 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b68060c10 msgr2=0x7f4b680a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:51.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.796+0000 7f4b6cf3a700 1 --2- 172.21.15.47:0/3102782827 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b68060c10 0x7f4b680a5060 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 rx=0x7f4b5000c000 tx=0x7f4b50006df0).stop 2022-06-29T15:49:51.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.796+0000 7f4b6cf3a700 1 -- 172.21.15.47:0/3102782827 shutdown_connections 2022-06-29T15:49:51.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.796+0000 7f4b6cf3a700 1 --2- 172.21.15.47:0/3102782827 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4b680b01d0 0x7f4b680acd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:51.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.796+0000 7f4b6cf3a700 1 --2- 172.21.15.47:0/3102782827 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b68060c10 0x7f4b680a5060 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:51.809 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.796+0000 7f4b6cf3a700 1 --2- 172.21.15.47:0/3102782827 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4b4c053aa0 0x7f4b4c055f60 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:51.809 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.796+0000 7f4b6cf3a700 1 --2- 172.21.15.47:0/3102782827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b6805fd60 0x7f4b680a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:51.809 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.796+0000 7f4b6cf3a700 1 -- 172.21.15.47:0/3102782827 >> 172.21.15.47:0/3102782827 conn(0x7f4b68062110 msgr2=0x7f4b680b6c20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:51.809 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.796+0000 7f4b6cf3a700 1 -- 172.21.15.47:0/3102782827 shutdown_connections 2022-06-29T15:49:51.810 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.796+0000 7f4b6cf3a700 1 -- 172.21.15.47:0/3102782827 wait complete. 2022-06-29T15:49:51.824 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:49:51.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.924+0000 7f634e880700 1 Processor -- start 2022-06-29T15:49:51.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.924+0000 7f634e880700 1 -- start start 2022-06-29T15:49:51.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.924+0000 7f634e880700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f63480b2ee0 0x7f63480b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:51.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.924+0000 7f634e880700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f63480ad970 0x7f63481407e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:51.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.924+0000 7f634e880700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63481400e0 con 0x7f63480b0e50 2022-06-29T15:49:51.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.924+0000 7f634e880700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f634813fef0 con 0x7f63480b2ee0 2022-06-29T15:49:51.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.924+0000 7f634e880700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6348143810 con 0x7f63480ad970 2022-06-29T15:49:51.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.924+0000 7f634d87e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f63480b2ee0 0x7f63480b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:51.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.924+0000 7f634e07f700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f63480b0e50 0x7f634813d290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37002/0 (socket says 172.21.15.47:37002) 2022-06-29T15:49:51.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.924+0000 7f634e07f700 1 -- 172.21.15.47:0/1282990697 learned_addr learned my addr 172.21.15.47:0/1282990697 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:49:51.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.924+0000 7f634d07d700 1 --2- 172.21.15.47:0/1282990697 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f63480ad970 0x7f63481407e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:51.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.924+0000 7f634d87e700 1 -- 172.21.15.47:0/1282990697 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f63480ad970 msgr2=0x7f63481407e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:51.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.924+0000 7f634d87e700 1 --2- 172.21.15.47:0/1282990697 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f63480ad970 0x7f63481407e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:51.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.924+0000 7f634d87e700 1 -- 172.21.15.47:0/1282990697 >> v1:172.21.15.47:6789/0 conn(0x7f63480b0e50 legacy=0x7f634813d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:51.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.924+0000 7f634d87e700 1 -- 172.21.15.47:0/1282990697 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6348143570 con 0x7f63480b2ee0 2022-06-29T15:49:51.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.924+0000 7f634d87e700 1 --2- 172.21.15.47:0/1282990697 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63480b2ee0 0x7f63480b32c0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 rx=0x7f6344002f70 tx=0x7f6344006df0).ready entity=mon.0 client_cookie=653e9cd4ce5bc305 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:51.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.924+0000 7f634c87c700 1 -- 172.21.15.47:0/1282990697 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f634400d570 con 0x7f63480b2ee0 2022-06-29T15:49:51.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.924+0000 7f634c87c700 1 -- 172.21.15.47:0/1282990697 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6344000e90 con 0x7f63480b2ee0 2022-06-29T15:49:51.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.924+0000 7f634c87c700 1 -- 172.21.15.47:0/1282990697 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f634400d8c0 con 0x7f63480b2ee0 2022-06-29T15:49:51.940 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.924+0000 7f634e880700 1 -- 172.21.15.47:0/1282990697 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63480b2ee0 msgr2=0x7f63480b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:51.941 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.924+0000 7f634e880700 1 --2- 172.21.15.47:0/1282990697 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63480b2ee0 0x7f63480b32c0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 rx=0x7f6344002f70 tx=0x7f6344006df0).stop 2022-06-29T15:49:51.941 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.924+0000 7f634e880700 1 -- 172.21.15.47:0/1282990697 shutdown_connections 2022-06-29T15:49:51.941 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.924+0000 7f634e880700 1 --2- 172.21.15.47:0/1282990697 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f63480ad970 0x7f63481407e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:51.941 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.924+0000 7f634e880700 1 --2- 172.21.15.47:0/1282990697 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63480b2ee0 0x7f63480b32c0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:51.942 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.924+0000 7f634e880700 1 -- 172.21.15.47:0/1282990697 >> 172.21.15.47:0/1282990697 conn(0x7f6348063670 msgr2=0x7f6348063a80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:51.942 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.924+0000 7f634e880700 1 -- 172.21.15.47:0/1282990697 shutdown_connections 2022-06-29T15:49:51.942 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.924+0000 7f634e880700 1 -- 172.21.15.47:0/1282990697 wait complete. 2022-06-29T15:49:51.942 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.928+0000 7f634e880700 1 Processor -- start 2022-06-29T15:49:51.942 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.928+0000 7f634e880700 1 -- start start 2022-06-29T15:49:51.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.928+0000 7f634e880700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63480ad970 0x7f6348144900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:51.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.928+0000 7f634e880700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f63480b0e50 0x7f63480aaec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:51.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.928+0000 7f634e880700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63480b2ee0 0x7f63480ab3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:51.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.928+0000 7f634e880700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6348065330 con 0x7f63480b2ee0 2022-06-29T15:49:51.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.928+0000 7f634e880700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6348066330 con 0x7f63480ad970 2022-06-29T15:49:51.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.928+0000 7f634e880700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63480665d0 con 0x7f63480b0e50 2022-06-29T15:49:51.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.928+0000 7f634e07f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63480b2ee0 0x7f63480ab3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:51.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.928+0000 7f634e07f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63480b2ee0 0x7f63480ab3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51454/0 (socket says 172.21.15.47:51454) 2022-06-29T15:49:51.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.928+0000 7f634e07f700 1 -- 172.21.15.47:0/958386126 learned_addr learned my addr 172.21.15.47:0/958386126 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:51.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.928+0000 7f634d87e700 1 --2- 172.21.15.47:0/958386126 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63480ad970 0x7f6348144900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:51.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.928+0000 7f634d07d700 1 --2- 172.21.15.47:0/958386126 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f63480b0e50 0x7f63480aaec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:51.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.932+0000 7f634e07f700 1 -- 172.21.15.47:0/958386126 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f63480b0e50 msgr2=0x7f63480aaec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:51.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.932+0000 7f634e07f700 1 --2- 172.21.15.47:0/958386126 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f63480b0e50 0x7f63480aaec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:51.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.932+0000 7f634e07f700 1 -- 172.21.15.47:0/958386126 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63480ad970 msgr2=0x7f6348144900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:51.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.932+0000 7f634e07f700 1 --2- 172.21.15.47:0/958386126 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63480ad970 0x7f6348144900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:51.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.932+0000 7f634e07f700 1 -- 172.21.15.47:0/958386126 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f634400b180 con 0x7f63480b2ee0 2022-06-29T15:49:51.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.932+0000 7f634e07f700 1 --2- 172.21.15.47:0/958386126 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63480b2ee0 0x7f63480ab3b0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 rx=0x7f63481406c0 tx=0x7f6340007f00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:51.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.932+0000 7f6336ffd700 1 -- 172.21.15.47:0/958386126 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6340005ba0 con 0x7f63480b2ee0 2022-06-29T15:49:51.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.932+0000 7f6336ffd700 1 -- 172.21.15.47:0/958386126 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6340005d00 con 0x7f63480b2ee0 2022-06-29T15:49:51.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.932+0000 7f6336ffd700 1 -- 172.21.15.47:0/958386126 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f63400072c0 con 0x7f63480b2ee0 2022-06-29T15:49:51.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.932+0000 7f634e880700 1 -- 172.21.15.47:0/958386126 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6348066870 con 0x7f63480b2ee0 2022-06-29T15:49:51.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.932+0000 7f634e880700 1 -- 172.21.15.47:0/958386126 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63480ad060 con 0x7f63480b2ee0 2022-06-29T15:49:51.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.932+0000 7f634e880700 1 -- 172.21.15.47:0/958386126 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63480573d0 con 0x7f63480b2ee0 2022-06-29T15:49:51.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.932+0000 7f6336ffd700 1 -- 172.21.15.47:0/958386126 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6340010070 con 0x7f63480b2ee0 2022-06-29T15:49:51.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.936+0000 7f6336ffd700 1 --2- 172.21.15.47:0/958386126 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6328053980 0x7f6328055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:51.950 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.936+0000 7f6336ffd700 1 -- 172.21.15.47:0/958386126 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f634006b8b0 con 0x7f63480b2ee0 2022-06-29T15:49:51.950 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.936+0000 7f634d87e700 1 --2- 172.21.15.47:0/958386126 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6328053980 0x7f6328055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:51.950 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.936+0000 7f634d87e700 1 --2- 172.21.15.47:0/958386126 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6328053980 0x7f6328055e40 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7f634401a000 tx=0x7f6344014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:51.951 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:51.940+0000 7f6336ffd700 1 -- 172.21.15.47:0/958386126 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6340042670 con 0x7f63480b2ee0 2022-06-29T15:49:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.116+0000 7f634e880700 1 -- 172.21.15.47:0/958386126 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6348069e20 con 0x7f63480b2ee0 2022-06-29T15:49:52.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.116+0000 7f6336ffd700 1 -- 172.21.15.47:0/958386126 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+2493 (secure 0 0 0) 0x7f6340018070 con 0x7f63480b2ee0 2022-06-29T15:49:52.127 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:49:52.127 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":45,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6969,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3642031880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3642031880},{"type":"v1","addr":"172.21.15.47:6835","nonce":3642031880}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":43},{"gid":6984,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/891572891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":891572891},{"type":"v1","addr":"172.21.15.47:6837","nonce":891572891}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":43}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:49:50.788599+0000","modified":"2022-06-29T15:49:51.791379+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":56102},"failed":[],"damaged":[],"stopped":[],"info":{"gid_56102":{"gid":56102,"name":"b","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6833/3819760998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3819760998},{"type":"v1","addr":"172.21.15.64:6833","nonce":3819760998}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2022-06-29T15:49:52.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.120+0000 7f634e880700 1 -- 172.21.15.47:0/958386126 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6328053980 msgr2=0x7f6328055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:52.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.120+0000 7f634e880700 1 --2- 172.21.15.47:0/958386126 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6328053980 0x7f6328055e40 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7f634401a000 tx=0x7f6344014000).stop 2022-06-29T15:49:52.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.120+0000 7f634e880700 1 -- 172.21.15.47:0/958386126 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63480b2ee0 msgr2=0x7f63480ab3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:52.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.120+0000 7f634e880700 1 --2- 172.21.15.47:0/958386126 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63480b2ee0 0x7f63480ab3b0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 rx=0x7f63481406c0 tx=0x7f6340007f00).stop 2022-06-29T15:49:52.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.120+0000 7f634e880700 1 -- 172.21.15.47:0/958386126 shutdown_connections 2022-06-29T15:49:52.130 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.120+0000 7f634e880700 1 --2- 172.21.15.47:0/958386126 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63480b2ee0 0x7f63480ab3b0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:52.130 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.120+0000 7f634e880700 1 --2- 172.21.15.47:0/958386126 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f63480b0e50 0x7f63480aaec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:52.131 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.120+0000 7f634e880700 1 --2- 172.21.15.47:0/958386126 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6328053980 0x7f6328055e40 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:52.131 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.120+0000 7f634e880700 1 --2- 172.21.15.47:0/958386126 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63480ad970 0x7f6348144900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:52.131 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.120+0000 7f634e880700 1 -- 172.21.15.47:0/958386126 >> 172.21.15.47:0/958386126 conn(0x7f6348063670 msgr2=0x7f6348141250 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:52.131 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.120+0000 7f634e880700 1 -- 172.21.15.47:0/958386126 shutdown_connections 2022-06-29T15:49:52.131 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.120+0000 7f634e880700 1 -- 172.21.15.47:0/958386126 wait complete. 2022-06-29T15:49:52.132 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 45 2022-06-29T15:49:52.141 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:49:52.247 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.240+0000 7fdf0f59e700 1 Processor -- start 2022-06-29T15:49:52.248 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.240+0000 7fdf0f59e700 1 -- start start 2022-06-29T15:49:52.252 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.240+0000 7fdf0f59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdf100b5c60 0x7fdf10144980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:52.252 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.240+0000 7fdf0f59e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdf10060eb0 0x7fdf10144e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:52.252 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.240+0000 7fdf0f59e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf100a9120 con 0x7fdf100a72f0 2022-06-29T15:49:52.253 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.240+0000 7fdf0f59e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf10145360 con 0x7fdf100b5c60 2022-06-29T15:49:52.253 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.240+0000 7fdf0f59e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf100b0b80 con 0x7fdf10060eb0 2022-06-29T15:49:52.254 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.240+0000 7fdf0dd9b700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdf10060eb0 0x7fdf10144e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:52.254 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.240+0000 7fdf0dd9b700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdf10060eb0 0x7fdf10144e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:48560/0 (socket says 172.21.15.47:48560) 2022-06-29T15:49:52.254 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.240+0000 7fdf0dd9b700 1 -- 172.21.15.47:0/690093541 learned_addr learned my addr 172.21.15.47:0/690093541 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:52.255 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.240+0000 7fdf0e59c700 1 --2- 172.21.15.47:0/690093541 >> v2:172.21.15.64:3300/0 conn(0x7fdf100b5c60 0x7fdf10144980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:52.255 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.240+0000 7fdf0dd9b700 1 -- 172.21.15.47:0/690093541 >> v2:172.21.15.64:3300/0 conn(0x7fdf100b5c60 msgr2=0x7fdf10144980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:52.255 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.240+0000 7fdf0dd9b700 1 --2- 172.21.15.47:0/690093541 >> v2:172.21.15.64:3300/0 conn(0x7fdf100b5c60 0x7fdf10144980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:52.256 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.240+0000 7fdf0dd9b700 1 -- 172.21.15.47:0/690093541 >> v1:172.21.15.64:6789/0 conn(0x7fdf100a72f0 legacy=0x7fdf100b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:52.256 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.240+0000 7fdf0dd9b700 1 -- 172.21.15.47:0/690093541 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf100a89e0 con 0x7fdf10060eb0 2022-06-29T15:49:52.257 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.240+0000 7fdf0dd9b700 1 --2- 172.21.15.47:0/690093541 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdf10060eb0 0x7fdf10144e70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7fdf00002f70 tx=0x7fdf00006eb0).ready entity=mon.2 client_cookie=a56fa1aa7201b1ac server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:52.257 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.240+0000 7fdf0d59a700 1 -- 172.21.15.47:0/690093541 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf0000d570 con 0x7fdf10060eb0 2022-06-29T15:49:52.257 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.240+0000 7fdf0d59a700 1 -- 172.21.15.47:0/690093541 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf00000e90 con 0x7fdf10060eb0 2022-06-29T15:49:52.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.240+0000 7fdf0d59a700 1 -- 172.21.15.47:0/690093541 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf0000d900 con 0x7fdf10060eb0 2022-06-29T15:49:52.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.240+0000 7fdf0f59e700 1 -- 172.21.15.47:0/690093541 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdf10060eb0 msgr2=0x7fdf10144e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:52.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.240+0000 7fdf0f59e700 1 --2- 172.21.15.47:0/690093541 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdf10060eb0 0x7fdf10144e70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7fdf00002f70 tx=0x7fdf00006eb0).stop 2022-06-29T15:49:52.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.240+0000 7fdf0f59e700 1 -- 172.21.15.47:0/690093541 shutdown_connections 2022-06-29T15:49:52.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.240+0000 7fdf0f59e700 1 --2- 172.21.15.47:0/690093541 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdf10060eb0 0x7fdf10144e70 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:52.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.240+0000 7fdf0f59e700 1 --2- 172.21.15.47:0/690093541 >> v2:172.21.15.64:3300/0 conn(0x7fdf100b5c60 0x7fdf10144980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:52.260 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.240+0000 7fdf0f59e700 1 -- 172.21.15.47:0/690093541 >> 172.21.15.47:0/690093541 conn(0x7fdf10062560 msgr2=0x7fdf10062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:52.260 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.240+0000 7fdf0f59e700 1 -- 172.21.15.47:0/690093541 shutdown_connections 2022-06-29T15:49:52.261 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.240+0000 7fdf0f59e700 1 -- 172.21.15.47:0/690093541 wait complete. 2022-06-29T15:49:52.261 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.244+0000 7fdf0f59e700 1 Processor -- start 2022-06-29T15:49:52.261 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.244+0000 7fdf0f59e700 1 -- start start 2022-06-29T15:49:52.266 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.244+0000 7fdf0f59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf10060eb0 0x7fdf10143e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:52.267 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.244+0000 7fdf0f59e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdf100a72f0 0x7fdf10156e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:52.267 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.244+0000 7fdf0f59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf100b5c60 0x7fdf10157330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:52.267 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.244+0000 7fdf0f59e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf100ad1e0 con 0x7fdf100b5c60 2022-06-29T15:49:52.268 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.244+0000 7fdf0f59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf101567b0 con 0x7fdf10060eb0 2022-06-29T15:49:52.268 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.244+0000 7fdf0f59e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf10145940 con 0x7fdf100a72f0 2022-06-29T15:49:52.269 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.244+0000 7fdf0ed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf100b5c60 0x7fdf10157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:52.269 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.244+0000 7fdf0e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf10060eb0 0x7fdf10143e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:52.269 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.244+0000 7fdf0dd9b700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdf100a72f0 0x7fdf10156e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:52.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.244+0000 7fdf0ed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf100b5c60 0x7fdf10157330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51468/0 (socket says 172.21.15.47:51468) 2022-06-29T15:49:52.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.244+0000 7fdf0dd9b700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdf100a72f0 0x7fdf10156e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:48568/0 (socket says 172.21.15.47:48568) 2022-06-29T15:49:52.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.244+0000 7fdf0ed9d700 1 -- 172.21.15.47:0/1747067855 learned_addr learned my addr 172.21.15.47:0/1747067855 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:52.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.244+0000 7fdf0e59c700 1 -- 172.21.15.47:0/1747067855 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdf100a72f0 msgr2=0x7fdf10156e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:52.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.244+0000 7fdf0e59c700 1 --2- 172.21.15.47:0/1747067855 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdf100a72f0 0x7fdf10156e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:52.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.244+0000 7fdf0e59c700 1 -- 172.21.15.47:0/1747067855 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf100b5c60 msgr2=0x7fdf10157330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:52.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.244+0000 7fdf0e59c700 1 --2- 172.21.15.47:0/1747067855 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf100b5c60 0x7fdf10157330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:52.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.244+0000 7fdf0e59c700 1 -- 172.21.15.47:0/1747067855 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf0000b1e0 con 0x7fdf10060eb0 2022-06-29T15:49:52.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.244+0000 7fdf0ed9d700 1 --2- 172.21.15.47:0/1747067855 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf100b5c60 0x7fdf10157330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:49:52.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.244+0000 7fdf0e59c700 1 --2- 172.21.15.47:0/1747067855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf10060eb0 0x7fdf10143e00 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7fdf0400c000 tx=0x7fdf04006d90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:52.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.244+0000 7fdeff7fe700 1 -- 172.21.15.47:0/1747067855 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf040104c0 con 0x7fdf10060eb0 2022-06-29T15:49:52.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.244+0000 7fdf0f59e700 1 -- 172.21.15.47:0/1747067855 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf10145be0 con 0x7fdf10060eb0 2022-06-29T15:49:52.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.244+0000 7fdeff7fe700 1 -- 172.21.15.47:0/1747067855 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf04010620 con 0x7fdf10060eb0 2022-06-29T15:49:52.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.244+0000 7fdeff7fe700 1 -- 172.21.15.47:0/1747067855 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf04010980 con 0x7fdf10060eb0 2022-06-29T15:49:52.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.244+0000 7fdf0f59e700 1 -- 172.21.15.47:0/1747067855 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf100ac050 con 0x7fdf10060eb0 2022-06-29T15:49:52.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.248+0000 7fdf0f59e700 1 -- 172.21.15.47:0/1747067855 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf100573d0 con 0x7fdf10060eb0 2022-06-29T15:49:52.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.248+0000 7fdeff7fe700 1 -- 172.21.15.47:0/1747067855 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fdf04010ae0 con 0x7fdf10060eb0 2022-06-29T15:49:52.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.248+0000 7fdeff7fe700 1 --2- 172.21.15.47:0/1747067855 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdeec0539e0 0x7fdeec055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:52.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.248+0000 7fdeff7fe700 1 -- 172.21.15.47:0/1747067855 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fdf0406bcc0 con 0x7fdf10060eb0 2022-06-29T15:49:52.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.248+0000 7fdf0dd9b700 1 --2- 172.21.15.47:0/1747067855 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdeec0539e0 0x7fdeec055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:52.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.248+0000 7fdf0dd9b700 1 --2- 172.21.15.47:0/1747067855 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdeec0539e0 0x7fdeec055ea0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7fdf0001c000 tx=0x7fdf00015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:52.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.256+0000 7fdeff7fe700 1 -- 172.21.15.47:0/1747067855 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdf04042b30 con 0x7fdf10060eb0 2022-06-29T15:49:52.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.380+0000 7fdf0f59e700 1 -- 172.21.15.47:0/1747067855 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf10145d70 con 0x7fdf10060eb0 2022-06-29T15:49:52.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.384+0000 7fdeff7fe700 1 -- 172.21.15.47:0/1747067855 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+16277 (secure 0 0 0) 0x7fdf04035020 con 0x7fdf10060eb0 2022-06-29T15:49:52.394 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:49:52.394 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":101,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:49:51.792208+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:49:48.525959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2022-06-29T15:49:49.805331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:49:52.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.384+0000 7fdf0f59e700 1 -- 172.21.15.47:0/1747067855 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdeec0539e0 msgr2=0x7fdeec055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:52.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.384+0000 7fdf0f59e700 1 --2- 172.21.15.47:0/1747067855 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdeec0539e0 0x7fdeec055ea0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7fdf0001c000 tx=0x7fdf00015000).stop 2022-06-29T15:49:52.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.384+0000 7fdf0f59e700 1 -- 172.21.15.47:0/1747067855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf10060eb0 msgr2=0x7fdf10143e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:52.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.384+0000 7fdf0f59e700 1 --2- 172.21.15.47:0/1747067855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf10060eb0 0x7fdf10143e00 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7fdf0400c000 tx=0x7fdf04006d90).stop 2022-06-29T15:49:52.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.388+0000 7fdf0f59e700 1 -- 172.21.15.47:0/1747067855 shutdown_connections 2022-06-29T15:49:52.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.388+0000 7fdf0f59e700 1 --2- 172.21.15.47:0/1747067855 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf100b5c60 0x7fdf10157330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:52.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.388+0000 7fdf0f59e700 1 --2- 172.21.15.47:0/1747067855 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdf100a72f0 0x7fdf10156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:52.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.388+0000 7fdf0f59e700 1 --2- 172.21.15.47:0/1747067855 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdeec0539e0 0x7fdeec055ea0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:52.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.388+0000 7fdf0f59e700 1 --2- 172.21.15.47:0/1747067855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf10060eb0 0x7fdf10143e00 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:52.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.388+0000 7fdf0f59e700 1 -- 172.21.15.47:0/1747067855 >> 172.21.15.47:0/1747067855 conn(0x7fdf10062560 msgr2=0x7fdf1013d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:52.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.388+0000 7fdf0f59e700 1 -- 172.21.15.47:0/1747067855 shutdown_connections 2022-06-29T15:49:52.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.388+0000 7fdf0f59e700 1 -- 172.21.15.47:0/1747067855 wait complete. 2022-06-29T15:49:52.407 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-29T15:49:52.520 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.512+0000 7f8774b3f700 1 Processor -- start 2022-06-29T15:49:52.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.512+0000 7f8774b3f700 1 -- start start 2022-06-29T15:49:52.524 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.512+0000 7f8774b3f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8770060ee0 0x7f87700612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:52.524 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.512+0000 7f8774b3f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f87700b4300 0x7f87701471a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:52.524 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.512+0000 7f8774b3f700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8770067610 con 0x7f8770066f70 2022-06-29T15:49:52.524 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.512+0000 7f8774b3f700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8770067400 con 0x7f87700b4300 2022-06-29T15:49:52.525 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.512+0000 7f8774b3f700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87701467c0 con 0x7f8770060ee0 2022-06-29T15:49:52.525 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.512+0000 7f8765d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f87700b4300 0x7f87701471a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:52.525 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.512+0000 7f8765d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f87700b4300 0x7f87701471a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51474/0 (socket says 172.21.15.47:51474) 2022-06-29T15:49:52.525 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.512+0000 7f8765d9b700 1 -- 172.21.15.47:0/2609571265 learned_addr learned my addr 172.21.15.47:0/2609571265 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:52.526 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.512+0000 7f876e59c700 1 --2- 172.21.15.47:0/2609571265 >> v2:172.21.15.64:3300/0 conn(0x7f8770060ee0 0x7f87700612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:52.526 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.512+0000 7f8765d9b700 1 -- 172.21.15.47:0/2609571265 >> v2:172.21.15.64:3300/0 conn(0x7f8770060ee0 msgr2=0x7f87700612c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:52.526 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.512+0000 7f8765d9b700 1 --2- 172.21.15.47:0/2609571265 >> v2:172.21.15.64:3300/0 conn(0x7f8770060ee0 0x7f87700612c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:52.527 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.512+0000 7f8765d9b700 1 -- 172.21.15.47:0/2609571265 >> v1:172.21.15.64:6789/0 conn(0x7f8770066f70 legacy=0x7f87701430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:52.527 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.512+0000 7f8765d9b700 1 -- 172.21.15.47:0/2609571265 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8770146a60 con 0x7f87700b4300 2022-06-29T15:49:52.527 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.512+0000 7f876e59c700 1 --2- 172.21.15.47:0/2609571265 >> v2:172.21.15.64:3300/0 conn(0x7f8770060ee0 0x7f87700612c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:49:52.527 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.512+0000 7f8765d9b700 1 --2- 172.21.15.47:0/2609571265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87700b4300 0x7f87701471a0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 rx=0x7f87700adb80 tx=0x7f8758006eb0).ready entity=mon.0 client_cookie=a0b37ea0cdb16dec server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:52.528 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.512+0000 7f876dd9b700 1 -- 172.21.15.47:0/2609571265 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f875800d5c0 con 0x7f87700b4300 2022-06-29T15:49:52.528 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.512+0000 7f876dd9b700 1 -- 172.21.15.47:0/2609571265 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8758000e90 con 0x7f87700b4300 2022-06-29T15:49:52.528 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.512+0000 7f876dd9b700 1 -- 172.21.15.47:0/2609571265 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f875800d950 con 0x7f87700b4300 2022-06-29T15:49:52.528 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.512+0000 7f8774b3f700 1 -- 172.21.15.47:0/2609571265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87700b4300 msgr2=0x7f87701471a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:52.528 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.512+0000 7f8774b3f700 1 --2- 172.21.15.47:0/2609571265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87700b4300 0x7f87701471a0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 rx=0x7f87700adb80 tx=0x7f8758006eb0).stop 2022-06-29T15:49:52.529 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.512+0000 7f8774b3f700 1 -- 172.21.15.47:0/2609571265 shutdown_connections 2022-06-29T15:49:52.530 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.512+0000 7f8774b3f700 1 --2- 172.21.15.47:0/2609571265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87700b4300 0x7f87701471a0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:52.530 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.512+0000 7f8774b3f700 1 --2- 172.21.15.47:0/2609571265 >> v2:172.21.15.64:3300/0 conn(0x7f8770060ee0 0x7f87700612c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:52.530 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.512+0000 7f8774b3f700 1 -- 172.21.15.47:0/2609571265 >> 172.21.15.47:0/2609571265 conn(0x7f877005c510 msgr2=0x7f877005f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:52.530 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.512+0000 7f8774b3f700 1 -- 172.21.15.47:0/2609571265 shutdown_connections 2022-06-29T15:49:52.531 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.512+0000 7f8774b3f700 1 -- 172.21.15.47:0/2609571265 wait complete. 2022-06-29T15:49:52.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.516+0000 7f8774b3f700 1 Processor -- start 2022-06-29T15:49:52.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.516+0000 7f8774b3f700 1 -- start start 2022-06-29T15:49:52.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.516+0000 7f8774b3f700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8770060ee0 0x7f8770137500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:52.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.516+0000 7f8774b3f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8770066f70 0x7f8770136200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:52.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.516+0000 7f8774b3f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87700b4300 0x7f87701366f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:52.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.516+0000 7f8774b3f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87700b1bf0 con 0x7f87700b4300 2022-06-29T15:49:52.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.516+0000 7f8774b3f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8770136be0 con 0x7f8770066f70 2022-06-29T15:49:52.536 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.516+0000 7f8774b3f700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87700b1e20 con 0x7f8770060ee0 2022-06-29T15:49:52.536 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.516+0000 7f876ed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87700b4300 0x7f87701366f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:52.536 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.516+0000 7f876ed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87700b4300 0x7f87701366f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51482/0 (socket says 172.21.15.47:51482) 2022-06-29T15:49:52.536 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.516+0000 7f876ed9d700 1 -- 172.21.15.47:0/1938642858 learned_addr learned my addr 172.21.15.47:0/1938642858 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:52.537 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.516+0000 7f876e59c700 1 --2- 172.21.15.47:0/1938642858 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8770060ee0 0x7f8770137500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:52.537 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.516+0000 7f8765d9b700 1 --2- 172.21.15.47:0/1938642858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8770066f70 0x7f8770136200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:52.537 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.516+0000 7f876ed9d700 1 -- 172.21.15.47:0/1938642858 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8770060ee0 msgr2=0x7f8770137500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:52.537 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.516+0000 7f876ed9d700 1 --2- 172.21.15.47:0/1938642858 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8770060ee0 0x7f8770137500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:52.537 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.516+0000 7f876ed9d700 1 -- 172.21.15.47:0/1938642858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8770066f70 msgr2=0x7f8770136200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:52.538 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.516+0000 7f876ed9d700 1 --2- 172.21.15.47:0/1938642858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8770066f70 0x7f8770136200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:52.538 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.516+0000 7f876ed9d700 1 -- 172.21.15.47:0/1938642858 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f875800b1e0 con 0x7f87700b4300 2022-06-29T15:49:52.538 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.516+0000 7f876e59c700 1 --2- 172.21.15.47:0/1938642858 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8770060ee0 0x7f8770137500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:49:52.539 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.516+0000 7f876ed9d700 1 --2- 172.21.15.47:0/1938642858 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87700b4300 0x7f87701366f0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 rx=0x7f8768002320 tx=0x7f87680023f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:52.539 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.520+0000 7f8767fff700 1 -- 172.21.15.47:0/1938642858 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8768006aa0 con 0x7f87700b4300 2022-06-29T15:49:52.539 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.520+0000 7f8767fff700 1 -- 172.21.15.47:0/1938642858 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8768006c00 con 0x7f87700b4300 2022-06-29T15:49:52.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.520+0000 7f8767fff700 1 -- 172.21.15.47:0/1938642858 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8768007030 con 0x7f87700b4300 2022-06-29T15:49:52.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.520+0000 7f8774b3f700 1 -- 172.21.15.47:0/1938642858 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87700b20c0 con 0x7f87700b4300 2022-06-29T15:49:52.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.520+0000 7f8774b3f700 1 -- 172.21.15.47:0/1938642858 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87700b2600 con 0x7f87700b4300 2022-06-29T15:49:52.541 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.520+0000 7f8767fff700 1 -- 172.21.15.47:0/1938642858 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8768010030 con 0x7f87700b4300 2022-06-29T15:49:52.541 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.520+0000 7f8774b3f700 1 -- 172.21.15.47:0/1938642858 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87700573d0 con 0x7f87700b4300 2022-06-29T15:49:52.541 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.520+0000 7f8767fff700 1 --2- 172.21.15.47:0/1938642858 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8754053a70 0x7f8754055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:52.541 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.520+0000 7f8767fff700 1 -- 172.21.15.47:0/1938642858 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f876806c160 con 0x7f87700b4300 2022-06-29T15:49:52.541 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.520+0000 7f876e59c700 1 --2- 172.21.15.47:0/1938642858 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8754053a70 0x7f8754055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:52.542 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.520+0000 7f876e59c700 1 --2- 172.21.15.47:0/1938642858 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8754053a70 0x7f8754055f30 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f876000b000 tx=0x7f8760008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:52.542 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.524+0000 7f8767fff700 1 -- 172.21.15.47:0/1938642858 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8768042e80 con 0x7f87700b4300 2022-06-29T15:49:52.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.696+0000 7f8774b3f700 1 -- 172.21.15.47:0/1938642858 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f87700b5e50 con 0x7f87700b4300 2022-06-29T15:49:52.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.704+0000 7f8767fff700 1 -- 172.21.15.47:0/1938642858 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v26) v1 ==== 184+0+0 (secure 0 0 0) 0x7f87680464a0 con 0x7f87700b4300 2022-06-29T15:49:52.714 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.704+0000 7f8774b3f700 1 -- 172.21.15.47:0/1938642858 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8754053a70 msgr2=0x7f8754055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:52.714 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.704+0000 7f8774b3f700 1 --2- 172.21.15.47:0/1938642858 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8754053a70 0x7f8754055f30 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f876000b000 tx=0x7f8760008040).stop 2022-06-29T15:49:52.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.704+0000 7f8774b3f700 1 -- 172.21.15.47:0/1938642858 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87700b4300 msgr2=0x7f87701366f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:52.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.704+0000 7f8774b3f700 1 --2- 172.21.15.47:0/1938642858 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87700b4300 0x7f87701366f0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 rx=0x7f8768002320 tx=0x7f87680023f0).stop 2022-06-29T15:49:52.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.704+0000 7f8774b3f700 1 -- 172.21.15.47:0/1938642858 shutdown_connections 2022-06-29T15:49:52.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.704+0000 7f8774b3f700 1 --2- 172.21.15.47:0/1938642858 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87700b4300 0x7f87701366f0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:52.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.704+0000 7f8774b3f700 1 --2- 172.21.15.47:0/1938642858 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8754053a70 0x7f8754055f30 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:52.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.704+0000 7f8774b3f700 1 --2- 172.21.15.47:0/1938642858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8770066f70 0x7f8770136200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:52.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.704+0000 7f8774b3f700 1 --2- 172.21.15.47:0/1938642858 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8770060ee0 0x7f8770137500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:52.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.704+0000 7f8774b3f700 1 -- 172.21.15.47:0/1938642858 >> 172.21.15.47:0/1938642858 conn(0x7f877005c510 msgr2=0x7f877013c740 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:52.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.704+0000 7f8774b3f700 1 -- 172.21.15.47:0/1938642858 shutdown_connections 2022-06-29T15:49:52.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.704+0000 7f8774b3f700 1 -- 172.21.15.47:0/1938642858 wait complete. 2022-06-29T15:49:52.717 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.0 2022-06-29T15:49:52.726 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-29T15:49:52.841 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.832+0000 7fbdc54dc700 1 Processor -- start 2022-06-29T15:49:52.842 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.832+0000 7fbdc54dc700 1 -- start start 2022-06-29T15:49:52.847 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.836+0000 7fbdc54dc700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbdc00b6d20 0x7fbdc00b1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:52.847 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.836+0000 7fbdc54dc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbdc0140c10 0x7fbdc00b1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:52.848 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.836+0000 7fbdc54dc700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdc00b1050 con 0x7fbdc01467a0 2022-06-29T15:49:52.848 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.836+0000 7fbdc54dc700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdc00b23c0 con 0x7fbdc0140c10 2022-06-29T15:49:52.848 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.836+0000 7fbdc54dc700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc005fab0 con 0x7fbdc00b6d20 2022-06-29T15:49:52.849 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.836+0000 7fbdb7fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fbdc01467a0 0x7fbdc0140500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37044/0 (socket says 172.21.15.47:37044) 2022-06-29T15:49:52.849 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.836+0000 7fbdb7fff700 1 -- 172.21.15.47:0/924626174 learned_addr learned my addr 172.21.15.47:0/924626174 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:49:52.849 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.836+0000 7fbdbffff700 1 --2- 172.21.15.47:0/924626174 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbdc00b6d20 0x7fbdc00b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:52.850 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.836+0000 7fbdc4cdb700 1 --2- 172.21.15.47:0/924626174 >> v2:172.21.15.64:3300/0 conn(0x7fbdc0140c10 0x7fbdc00b1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:52.850 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.836+0000 7fbdbf7fe700 1 -- 172.21.15.47:0/924626174 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3046969709 0 0) 0x7fbdc00b1050 con 0x7fbdc01467a0 2022-06-29T15:49:52.850 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.836+0000 7fbdbf7fe700 1 -- 172.21.15.47:0/924626174 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbda0003260 con 0x7fbdc01467a0 2022-06-29T15:49:52.850 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.836+0000 7fbdc4cdb700 1 -- 172.21.15.47:0/924626174 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbdc00b6d20 msgr2=0x7fbdc00b1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:52.851 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.836+0000 7fbdc4cdb700 1 --2- 172.21.15.47:0/924626174 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbdc00b6d20 0x7fbdc00b1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:52.851 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.836+0000 7fbdc4cdb700 1 -- 172.21.15.47:0/924626174 >> v1:172.21.15.47:6789/0 conn(0x7fbdc01467a0 legacy=0x7fbdc0140500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:52.851 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.836+0000 7fbdc4cdb700 1 -- 172.21.15.47:0/924626174 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc00b7100 con 0x7fbdc0140c10 2022-06-29T15:49:52.851 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.836+0000 7fbdc4cdb700 1 --2- 172.21.15.47:0/924626174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdc0140c10 0x7fbdc00b1970 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7fbdc00b2700 tx=0x7fbdb80091d0).ready entity=mon.1 client_cookie=50859acaa4d5b73a server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:52.852 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.836+0000 7fbdbf7fe700 1 -- 172.21.15.47:0/924626174 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbdb800b600 con 0x7fbdc0140c10 2022-06-29T15:49:52.852 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.836+0000 7fbdbf7fe700 1 -- 172.21.15.47:0/924626174 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb800b760 con 0x7fbdc0140c10 2022-06-29T15:49:52.852 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.836+0000 7fbdbf7fe700 1 -- 172.21.15.47:0/924626174 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbdb800b8d0 con 0x7fbdc0140c10 2022-06-29T15:49:52.852 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.836+0000 7fbdc54dc700 1 -- 172.21.15.47:0/924626174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdc0140c10 msgr2=0x7fbdc00b1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:52.853 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.836+0000 7fbdc54dc700 1 --2- 172.21.15.47:0/924626174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdc0140c10 0x7fbdc00b1970 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7fbdc00b2700 tx=0x7fbdb80091d0).stop 2022-06-29T15:49:52.853 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.836+0000 7fbdc54dc700 1 -- 172.21.15.47:0/924626174 shutdown_connections 2022-06-29T15:49:52.853 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.836+0000 7fbdc54dc700 1 --2- 172.21.15.47:0/924626174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdc0140c10 0x7fbdc00b1970 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:52.853 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.836+0000 7fbdc54dc700 1 --2- 172.21.15.47:0/924626174 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbdc00b6d20 0x7fbdc00b1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:52.853 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.836+0000 7fbdc54dc700 1 -- 172.21.15.47:0/924626174 >> 172.21.15.47:0/924626174 conn(0x7fbdc0063670 msgr2=0x7fbdc0063a50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:52.854 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.836+0000 7fbdc54dc700 1 -- 172.21.15.47:0/924626174 shutdown_connections 2022-06-29T15:49:52.854 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.836+0000 7fbdc54dc700 1 -- 172.21.15.47:0/924626174 wait complete. 2022-06-29T15:49:52.857 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.840+0000 7fbdc54dc700 1 Processor -- start 2022-06-29T15:49:52.857 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.840+0000 7fbdc54dc700 1 -- start start 2022-06-29T15:49:52.858 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.840+0000 7fbdc54dc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdc00b6d20 0x7fbdc01453c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:52.858 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.840+0000 7fbdc54dc700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbdc0140c10 0x7fbdc00acf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:52.858 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.840+0000 7fbdc54dc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbdc01467a0 0x7fbdc0155a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:52.858 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.840+0000 7fbdc54dc700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc00ad430 con 0x7fbdc01467a0 2022-06-29T15:49:52.858 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.840+0000 7fbdc54dc700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc0152ea0 con 0x7fbdc00b6d20 2022-06-29T15:49:52.859 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.840+0000 7fbdc54dc700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc00ac870 con 0x7fbdc0140c10 2022-06-29T15:49:52.859 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.840+0000 7fbdc4cdb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbdc01467a0 0x7fbdc0155a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:52.859 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.840+0000 7fbdc4cdb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbdc01467a0 0x7fbdc0155a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51496/0 (socket says 172.21.15.47:51496) 2022-06-29T15:49:52.859 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.840+0000 7fbdc4cdb700 1 -- 172.21.15.47:0/3644553565 learned_addr learned my addr 172.21.15.47:0/3644553565 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:52.860 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.840+0000 7fbdbffff700 1 --2- 172.21.15.47:0/3644553565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdc00b6d20 0x7fbdc01453c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:52.860 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.840+0000 7fbdb7fff700 1 --2- 172.21.15.47:0/3644553565 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbdc0140c10 0x7fbdc00acf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:52.860 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.840+0000 7fbdc4cdb700 1 -- 172.21.15.47:0/3644553565 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbdc0140c10 msgr2=0x7fbdc00acf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:52.861 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.840+0000 7fbdc4cdb700 1 --2- 172.21.15.47:0/3644553565 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbdc0140c10 0x7fbdc00acf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:52.861 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.840+0000 7fbdc4cdb700 1 -- 172.21.15.47:0/3644553565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdc00b6d20 msgr2=0x7fbdc01453c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:52.861 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.840+0000 7fbdc4cdb700 1 --2- 172.21.15.47:0/3644553565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdc00b6d20 0x7fbdc01453c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:52.862 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.840+0000 7fbdc4cdb700 1 -- 172.21.15.47:0/3644553565 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb8009470 con 0x7fbdc01467a0 2022-06-29T15:49:52.862 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.840+0000 7fbdc4cdb700 1 --2- 172.21.15.47:0/3644553565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbdc01467a0 0x7fbdc0155a80 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 rx=0x7fbdc00b2700 tx=0x7fbdb8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:52.862 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.840+0000 7fbdbdffb700 1 -- 172.21.15.47:0/3644553565 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbdb8011c10 con 0x7fbdc01467a0 2022-06-29T15:49:52.863 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.844+0000 7fbdbdffb700 1 -- 172.21.15.47:0/3644553565 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb8011d70 con 0x7fbdc01467a0 2022-06-29T15:49:52.863 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.844+0000 7fbdc54dc700 1 -- 172.21.15.47:0/3644553565 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdc01530a0 con 0x7fbdc01467a0 2022-06-29T15:49:52.864 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.844+0000 7fbdbdffb700 1 -- 172.21.15.47:0/3644553565 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbdb801f5e0 con 0x7fbdc01467a0 2022-06-29T15:49:52.864 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.844+0000 7fbdc54dc700 1 -- 172.21.15.47:0/3644553565 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdc00acaa0 con 0x7fbdc01467a0 2022-06-29T15:49:52.864 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.844+0000 7fbdc54dc700 1 -- 172.21.15.47:0/3644553565 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdc00573d0 con 0x7fbdc01467a0 2022-06-29T15:49:52.865 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.844+0000 7fbdbdffb700 1 -- 172.21.15.47:0/3644553565 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fbdb8026030 con 0x7fbdc01467a0 2022-06-29T15:49:52.865 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.844+0000 7fbdbdffb700 1 --2- 172.21.15.47:0/3644553565 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbda4053a70 0x7fbda4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:52.865 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.844+0000 7fbdbffff700 1 --2- 172.21.15.47:0/3644553565 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbda4053a70 0x7fbda4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:52.866 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.844+0000 7fbdbdffb700 1 -- 172.21.15.47:0/3644553565 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fbdb806ca90 con 0x7fbdc01467a0 2022-06-29T15:49:52.866 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.844+0000 7fbdbffff700 1 --2- 172.21.15.47:0/3644553565 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbda4053a70 0x7fbda4055f30 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7fbdb000b000 tx=0x7fbdb0006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:52.867 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:52.852+0000 7fbdbdffb700 1 -- 172.21.15.47:0/3644553565 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbdb8043850 con 0x7fbdc01467a0 2022-06-29T15:49:53.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.024+0000 7fbdc54dc700 1 -- 172.21.15.47:0/3644553565 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fbdc00acc30 con 0x7fbdc01467a0 2022-06-29T15:49:53.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.032+0000 7fbdbdffb700 1 -- 172.21.15.47:0/3644553565 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v27) v1 ==== 184+0+0 (secure 0 0 0) 0x7fbdb80150e0 con 0x7fbdc01467a0 2022-06-29T15:49:53.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.036+0000 7fbdc54dc700 1 -- 172.21.15.47:0/3644553565 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbda4053a70 msgr2=0x7fbda4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:53.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.036+0000 7fbdc54dc700 1 --2- 172.21.15.47:0/3644553565 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbda4053a70 0x7fbda4055f30 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7fbdb000b000 tx=0x7fbdb0006df0).stop 2022-06-29T15:49:53.045 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.036+0000 7fbdc54dc700 1 -- 172.21.15.47:0/3644553565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbdc01467a0 msgr2=0x7fbdc0155a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:53.045 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.036+0000 7fbdc54dc700 1 --2- 172.21.15.47:0/3644553565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbdc01467a0 0x7fbdc0155a80 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 rx=0x7fbdc00b2700 tx=0x7fbdb8016000).stop 2022-06-29T15:49:53.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.036+0000 7fbdc54dc700 1 -- 172.21.15.47:0/3644553565 shutdown_connections 2022-06-29T15:49:53.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.036+0000 7fbdc54dc700 1 --2- 172.21.15.47:0/3644553565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbdc01467a0 0x7fbdc0155a80 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:53.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.036+0000 7fbdc54dc700 1 --2- 172.21.15.47:0/3644553565 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbdc0140c10 0x7fbdc00acf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:53.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.036+0000 7fbdc54dc700 1 --2- 172.21.15.47:0/3644553565 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbda4053a70 0x7fbda4055f30 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:53.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.036+0000 7fbdc54dc700 1 --2- 172.21.15.47:0/3644553565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdc00b6d20 0x7fbdc01453c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:53.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.036+0000 7fbdc54dc700 1 -- 172.21.15.47:0/3644553565 >> 172.21.15.47:0/3644553565 conn(0x7fbdc0063670 msgr2=0x7fbdc0147880 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:53.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.036+0000 7fbdc54dc700 1 -- 172.21.15.47:0/3644553565 shutdown_connections 2022-06-29T15:49:53.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.036+0000 7fbdc54dc700 1 -- 172.21.15.47:0/3644553565 wait complete. 2022-06-29T15:49:53.049 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.1 2022-06-29T15:49:53.056 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:49:53.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.152+0000 7f9958a86700 1 Processor -- start 2022-06-29T15:49:53.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.152+0000 7f9958a86700 1 -- start start 2022-06-29T15:49:53.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.152+0000 7f9958a86700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f995413d9a0 0x7f99540b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:53.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.152+0000 7f9958a86700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99541400e0 con 0x7f99540b2ee0 2022-06-29T15:49:53.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.152+0000 7f9958a86700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9954143570 con 0x7f99540ad970 2022-06-29T15:49:53.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.152+0000 7f9958a86700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9954140ef0 con 0x7f995413d9a0 2022-06-29T15:49:53.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.152+0000 7f9953fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f995413d9a0 0x7f99540b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:53.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.152+0000 7f9953fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f995413d9a0 0x7f99540b32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51504/0 (socket says 172.21.15.47:51504) 2022-06-29T15:49:53.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.152+0000 7f9953fff700 1 -- 172.21.15.47:0/3430078753 learned_addr learned my addr 172.21.15.47:0/3430078753 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:53.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.152+0000 7f99537fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f99540b2ee0 0x7f99541407e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37054/0 (socket says 172.21.15.47:37054) 2022-06-29T15:49:53.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.156+0000 7f9953fff700 1 -- 172.21.15.47:0/3430078753 >> v1:172.21.15.64:6789/0 conn(0x7f99540ad970 legacy=0x7f995413d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:53.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.156+0000 7f9953fff700 1 -- 172.21.15.47:0/3430078753 >> v1:172.21.15.47:6789/0 conn(0x7f99540b2ee0 legacy=0x7f99541407e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:53.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.156+0000 7f9953fff700 1 -- 172.21.15.47:0/3430078753 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9954140340 con 0x7f995413d9a0 2022-06-29T15:49:53.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.156+0000 7f9953fff700 1 --2- 172.21.15.47:0/3430078753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f995413d9a0 0x7f99540b32c0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 rx=0x7f994c0032f0 tx=0x7f994c009290).ready entity=mon.0 client_cookie=921a5a97b7fc5959 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:53.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.156+0000 7f99527fc700 1 -- 172.21.15.47:0/3430078753 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f994c00b8a0 con 0x7f995413d9a0 2022-06-29T15:49:53.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.156+0000 7f99527fc700 1 -- 172.21.15.47:0/3430078753 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f994c00ba00 con 0x7f995413d9a0 2022-06-29T15:49:53.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.156+0000 7f99527fc700 1 -- 172.21.15.47:0/3430078753 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f994c00bb70 con 0x7f995413d9a0 2022-06-29T15:49:53.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.156+0000 7f9958a86700 1 -- 172.21.15.47:0/3430078753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f995413d9a0 msgr2=0x7f99540b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:53.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.156+0000 7f9958a86700 1 --2- 172.21.15.47:0/3430078753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f995413d9a0 0x7f99540b32c0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 rx=0x7f994c0032f0 tx=0x7f994c009290).stop 2022-06-29T15:49:53.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.156+0000 7f9958a86700 1 -- 172.21.15.47:0/3430078753 shutdown_connections 2022-06-29T15:49:53.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.156+0000 7f9958a86700 1 --2- 172.21.15.47:0/3430078753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f995413d9a0 0x7f99540b32c0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:53.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.156+0000 7f9958a86700 1 -- 172.21.15.47:0/3430078753 >> 172.21.15.47:0/3430078753 conn(0x7f9954063670 msgr2=0x7f9954063a80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:53.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.156+0000 7f9958a86700 1 -- 172.21.15.47:0/3430078753 shutdown_connections 2022-06-29T15:49:53.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.156+0000 7f9958a86700 1 -- 172.21.15.47:0/3430078753 wait complete. 2022-06-29T15:49:53.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.160+0000 7f9958a86700 1 Processor -- start 2022-06-29T15:49:53.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.160+0000 7f9958a86700 1 -- start start 2022-06-29T15:49:53.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.160+0000 7f9958a86700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99540ad970 0x7f9954060380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:53.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.160+0000 7f9958a86700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99540b2ee0 0x7f99540665a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:53.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.160+0000 7f9958a86700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f995413ce40 0x7f9954066a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:53.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.160+0000 7f9958a86700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99540660e0 con 0x7f99540ad970 2022-06-29T15:49:53.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.160+0000 7f9958a86700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99540ac660 con 0x7f99540b2ee0 2022-06-29T15:49:53.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.160+0000 7f9958a86700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99540ad1a0 con 0x7f995413ce40 2022-06-29T15:49:53.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.160+0000 7f99537fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99540ad970 0x7f9954060380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:53.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.160+0000 7f99537fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99540ad970 0x7f9954060380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51506/0 (socket says 172.21.15.47:51506) 2022-06-29T15:49:53.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.160+0000 7f99537fe700 1 -- 172.21.15.47:0/55116340 learned_addr learned my addr 172.21.15.47:0/55116340 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:53.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.160+0000 7f9952ffd700 1 --2- 172.21.15.47:0/55116340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99540b2ee0 0x7f99540665a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:53.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.160+0000 7f9953fff700 1 --2- 172.21.15.47:0/55116340 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f995413ce40 0x7f9954066a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:53.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.160+0000 7f99537fe700 1 -- 172.21.15.47:0/55116340 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f995413ce40 msgr2=0x7f9954066a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:53.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.160+0000 7f99537fe700 1 --2- 172.21.15.47:0/55116340 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f995413ce40 0x7f9954066a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:53.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.160+0000 7f99537fe700 1 -- 172.21.15.47:0/55116340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99540b2ee0 msgr2=0x7f99540665a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:53.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.160+0000 7f99537fe700 1 --2- 172.21.15.47:0/55116340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99540b2ee0 0x7f99540665a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:53.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.160+0000 7f99537fe700 1 -- 172.21.15.47:0/55116340 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f994c009530 con 0x7f99540ad970 2022-06-29T15:49:53.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.160+0000 7f99537fe700 1 --2- 172.21.15.47:0/55116340 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99540ad970 0x7f9954060380 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 rx=0x7f9954061bb0 tx=0x7f9948007bd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:53.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.160+0000 7f9950ff9700 1 -- 172.21.15.47:0/55116340 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9948007170 con 0x7f99540ad970 2022-06-29T15:49:53.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.160+0000 7f9950ff9700 1 -- 172.21.15.47:0/55116340 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99480072d0 con 0x7f99540ad970 2022-06-29T15:49:53.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.160+0000 7f9958a86700 1 -- 172.21.15.47:0/55116340 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9954066340 con 0x7f99540ad970 2022-06-29T15:49:53.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.160+0000 7f9950ff9700 1 -- 172.21.15.47:0/55116340 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9948004710 con 0x7f99540ad970 2022-06-29T15:49:53.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.160+0000 7f9958a86700 1 -- 172.21.15.47:0/55116340 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9954065c70 con 0x7f99540ad970 2022-06-29T15:49:53.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.164+0000 7f9958a86700 1 -- 172.21.15.47:0/55116340 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99540aa200 con 0x7f99540ad970 2022-06-29T15:49:53.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.164+0000 7f9950ff9700 1 -- 172.21.15.47:0/55116340 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9948004870 con 0x7f99540ad970 2022-06-29T15:49:53.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.164+0000 7f9950ff9700 1 --2- 172.21.15.47:0/55116340 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9934053a70 0x7f9934055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:53.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.164+0000 7f9950ff9700 1 -- 172.21.15.47:0/55116340 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f994806b870 con 0x7f99540ad970 2022-06-29T15:49:53.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.164+0000 7f9952ffd700 1 --2- 172.21.15.47:0/55116340 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9934053a70 0x7f9934055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:53.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.164+0000 7f9952ffd700 1 --2- 172.21.15.47:0/55116340 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9934053a70 0x7f9934055f30 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7f994400d000 tx=0x7f9944007b90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:53.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.168+0000 7f9950ff9700 1 -- 172.21.15.47:0/55116340 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9948042630 con 0x7f99540ad970 2022-06-29T15:49:53.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.332+0000 7f9958a86700 1 -- 172.21.15.47:0/55116340 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f99540ac7d0 con 0x7f99540ad970 2022-06-29T15:49:53.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.332+0000 7f9950ff9700 1 -- 172.21.15.47:0/55116340 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+2493 (secure 0 0 0) 0x7f9948045c50 con 0x7f99540ad970 2022-06-29T15:49:53.343 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:49:53.343 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":45,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6969,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3642031880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3642031880},{"type":"v1","addr":"172.21.15.47:6835","nonce":3642031880}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":43},{"gid":6984,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/891572891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":891572891},{"type":"v1","addr":"172.21.15.47:6837","nonce":891572891}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":43}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:49:50.788599+0000","modified":"2022-06-29T15:49:51.791379+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":56102},"failed":[],"damaged":[],"stopped":[],"info":{"gid_56102":{"gid":56102,"name":"b","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6833/3819760998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3819760998},{"type":"v1","addr":"172.21.15.64:6833","nonce":3819760998}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2022-06-29T15:49:53.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.336+0000 7f9958a86700 1 -- 172.21.15.47:0/55116340 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9934053a70 msgr2=0x7f9934055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:53.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.336+0000 7f9958a86700 1 --2- 172.21.15.47:0/55116340 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9934053a70 0x7f9934055f30 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7f994400d000 tx=0x7f9944007b90).stop 2022-06-29T15:49:53.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.336+0000 7f9958a86700 1 -- 172.21.15.47:0/55116340 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99540ad970 msgr2=0x7f9954060380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:53.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.336+0000 7f9958a86700 1 --2- 172.21.15.47:0/55116340 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99540ad970 0x7f9954060380 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 rx=0x7f9954061bb0 tx=0x7f9948007bd0).stop 2022-06-29T15:49:53.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.336+0000 7f9958a86700 1 -- 172.21.15.47:0/55116340 shutdown_connections 2022-06-29T15:49:53.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.336+0000 7f9958a86700 1 --2- 172.21.15.47:0/55116340 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f995413ce40 0x7f9954066a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:53.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.336+0000 7f9958a86700 1 --2- 172.21.15.47:0/55116340 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9934053a70 0x7f9934055f30 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:53.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.336+0000 7f9958a86700 1 --2- 172.21.15.47:0/55116340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99540b2ee0 0x7f99540665a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:53.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.336+0000 7f9958a86700 1 --2- 172.21.15.47:0/55116340 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99540ad970 0x7f9954060380 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:53.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.336+0000 7f9958a86700 1 -- 172.21.15.47:0/55116340 >> 172.21.15.47:0/55116340 conn(0x7f9954063670 msgr2=0x7f995413ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:53.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.336+0000 7f9958a86700 1 -- 172.21.15.47:0/55116340 shutdown_connections 2022-06-29T15:49:53.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.336+0000 7f9958a86700 1 -- 172.21.15.47:0/55116340 wait complete. 2022-06-29T15:49:53.351 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 45 2022-06-29T15:49:53.659 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 45, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-29T15:49:50.788599+0000', 'modified': '2022-06-29T15:49:51.791379+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 56102}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_56102': {'gid': 56102, 'name': 'b', 'rank': 0, 'incarnation': 44, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.64:6833/3819760998', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6832', 'nonce': 3819760998}, {'type': 'v1', 'addr': '172.21.15.64:6833', 'nonce': 3819760998}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-29T15:49:53.660 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-29T15:49:53.660 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-06-29T15:49:53.780 INFO:teuthology.orchestra.run.smithi064.stdout:{"cluster_fsid":"9a68894a-53ce-400e-838b-14eab91b834c","whoami":0,"id":56102,"want_state":"up:active","state":"up:active","rank_uptime":2.9735929200000002,"mdsmap_epoch":45,"osdmap_epoch":101,"osdmap_epoch_barrier":100,"uptime":7.9866405450000002} 2022-06-29T15:49:53.781 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9a68894a-53ce-400e-838b-14eab91b834c", "id": 56102, "mdsmap_epoch": 45, "osdmap_epoch": 101, "osdmap_epoch_barrier": 100, "rank_uptime": 2.97359292, "state": "up:active", "uptime": 7.986640545, "want_state": "up:active", "whoami": 0 } 2022-06-29T15:49:53.782 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:49:53.896 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.888+0000 7feca97fc700 1 Processor -- start 2022-06-29T15:49:53.897 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.888+0000 7feca97fc700 1 -- start start 2022-06-29T15:49:53.899 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.888+0000 7feca97fc700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7feca40a93c0 0x7feca4095240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:53.899 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.888+0000 7feca97fc700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feca4143be0 con 0x7feca41430f0 2022-06-29T15:49:53.899 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.888+0000 7feca97fc700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feca4060e50 con 0x7feca4095730 2022-06-29T15:49:53.900 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.888+0000 7feca97fc700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feca413d590 con 0x7feca40a93c0 2022-06-29T15:49:53.900 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.888+0000 7feca2ffd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7feca41430f0 0x7feca41434d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37068/0 (socket says 172.21.15.47:37068) 2022-06-29T15:49:53.900 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.888+0000 7feca2ffd700 1 -- 172.21.15.47:0/2579684202 learned_addr learned my addr 172.21.15.47:0/2579684202 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:49:53.900 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.888+0000 7feca27fc700 1 --2- 172.21.15.47:0/2579684202 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7feca40a93c0 0x7feca4095240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:53.901 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.888+0000 7feca1ffb700 1 -- 172.21.15.47:0/2579684202 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 125135332 0 0) 0x7feca4143be0 con 0x7feca41430f0 2022-06-29T15:49:53.901 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.888+0000 7feca1ffb700 1 -- 172.21.15.47:0/2579684202 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec88003260 con 0x7feca41430f0 2022-06-29T15:49:53.901 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.888+0000 7feca1ffb700 1 -- 172.21.15.47:0/2579684202 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1244218688 0 0) 0x7feca4060e50 con 0x7feca4095730 2022-06-29T15:49:53.901 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.888+0000 7feca1ffb700 1 -- 172.21.15.47:0/2579684202 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feca4143be0 con 0x7feca4095730 2022-06-29T15:49:53.902 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.888+0000 7feca27fc700 1 -- 172.21.15.47:0/2579684202 >> v1:172.21.15.64:6789/0 conn(0x7feca4095730 legacy=0x7feca4097690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:53.902 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.888+0000 7feca27fc700 1 -- 172.21.15.47:0/2579684202 >> v1:172.21.15.47:6789/0 conn(0x7feca41430f0 legacy=0x7feca41434d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:53.902 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.888+0000 7feca27fc700 1 -- 172.21.15.47:0/2579684202 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feca40610f0 con 0x7feca40a93c0 2022-06-29T15:49:53.902 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.888+0000 7feca27fc700 1 --2- 172.21.15.47:0/2579684202 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7feca40a93c0 0x7feca4095240 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fec940032f0 tx=0x7fec940091d0).ready entity=mon.2 client_cookie=869eb1e79eeb390c server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:53.903 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.888+0000 7feca1ffb700 1 -- 172.21.15.47:0/2579684202 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec9400b600 con 0x7feca40a93c0 2022-06-29T15:49:53.903 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.888+0000 7feca1ffb700 1 -- 172.21.15.47:0/2579684202 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec9400b760 con 0x7feca40a93c0 2022-06-29T15:49:53.903 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.888+0000 7feca1ffb700 1 -- 172.21.15.47:0/2579684202 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec9400b8d0 con 0x7feca40a93c0 2022-06-29T15:49:53.906 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.892+0000 7feca97fc700 1 -- 172.21.15.47:0/2579684202 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7feca40a93c0 msgr2=0x7feca4095240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:53.907 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.892+0000 7feca97fc700 1 --2- 172.21.15.47:0/2579684202 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7feca40a93c0 0x7feca4095240 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fec940032f0 tx=0x7fec940091d0).stop 2022-06-29T15:49:53.907 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.892+0000 7feca97fc700 1 -- 172.21.15.47:0/2579684202 shutdown_connections 2022-06-29T15:49:53.907 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.892+0000 7feca97fc700 1 --2- 172.21.15.47:0/2579684202 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7feca40a93c0 0x7feca4095240 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:53.907 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.892+0000 7feca97fc700 1 -- 172.21.15.47:0/2579684202 >> 172.21.15.47:0/2579684202 conn(0x7feca405f0b0 msgr2=0x7feca405f490 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:53.907 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.892+0000 7feca97fc700 1 -- 172.21.15.47:0/2579684202 shutdown_connections 2022-06-29T15:49:53.908 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.892+0000 7feca97fc700 1 -- 172.21.15.47:0/2579684202 wait complete. 2022-06-29T15:49:53.908 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.892+0000 7feca97fc700 1 Processor -- start 2022-06-29T15:49:53.908 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.892+0000 7feca97fc700 1 -- start start 2022-06-29T15:49:53.908 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.892+0000 7feca97fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feca4095730 0x7feca40645e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:53.909 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.892+0000 7feca97fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feca40a93c0 0x7feca4064ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:53.909 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.892+0000 7feca97fc700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7feca41430f0 0x7feca40671f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:53.909 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.892+0000 7feca97fc700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feca4140700 con 0x7feca4095730 2022-06-29T15:49:53.909 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.892+0000 7feca97fc700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feca4140110 con 0x7feca40a93c0 2022-06-29T15:49:53.910 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.892+0000 7feca97fc700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feca40953a0 con 0x7feca41430f0 2022-06-29T15:49:53.910 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.892+0000 7feca2ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feca4095730 0x7feca40645e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:53.910 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.892+0000 7feca27fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feca40a93c0 0x7feca4064ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:53.910 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.892+0000 7feca2ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feca4095730 0x7feca40645e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51524/0 (socket says 172.21.15.47:51524) 2022-06-29T15:49:53.911 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.892+0000 7feca2ffd700 1 -- 172.21.15.47:0/1071063657 learned_addr learned my addr 172.21.15.47:0/1071063657 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:53.911 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.892+0000 7feca37fe700 1 --2- 172.21.15.47:0/1071063657 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7feca41430f0 0x7feca40671f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:53.911 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.892+0000 7feca2ffd700 1 -- 172.21.15.47:0/1071063657 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7feca41430f0 msgr2=0x7feca40671f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:53.911 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.892+0000 7feca2ffd700 1 --2- 172.21.15.47:0/1071063657 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7feca41430f0 0x7feca40671f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:53.911 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.896+0000 7feca2ffd700 1 -- 172.21.15.47:0/1071063657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feca40a93c0 msgr2=0x7feca4064ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:53.912 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.896+0000 7feca2ffd700 1 --2- 172.21.15.47:0/1071063657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feca40a93c0 0x7feca4064ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:53.912 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.896+0000 7feca2ffd700 1 -- 172.21.15.47:0/1071063657 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec94009470 con 0x7feca4095730 2022-06-29T15:49:53.912 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.896+0000 7feca2ffd700 1 --2- 172.21.15.47:0/1071063657 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feca4095730 0x7feca40645e0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 rx=0x7feca4095670 tx=0x7fec8c007b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:53.912 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.896+0000 7fec9bfff700 1 -- 172.21.15.47:0/1071063657 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec8c00ce60 con 0x7feca4095730 2022-06-29T15:49:53.913 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.896+0000 7fec9bfff700 1 -- 172.21.15.47:0/1071063657 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec8c0070d0 con 0x7feca4095730 2022-06-29T15:49:53.913 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.896+0000 7feca97fc700 1 -- 172.21.15.47:0/1071063657 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feca4140340 con 0x7feca4095730 2022-06-29T15:49:53.913 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.896+0000 7fec9bfff700 1 -- 172.21.15.47:0/1071063657 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec8c007430 con 0x7feca4095730 2022-06-29T15:49:53.913 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.896+0000 7feca97fc700 1 -- 172.21.15.47:0/1071063657 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feca40afe70 con 0x7feca4095730 2022-06-29T15:49:53.913 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.896+0000 7feca97fc700 1 -- 172.21.15.47:0/1071063657 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feca40573d0 con 0x7feca4095730 2022-06-29T15:49:53.914 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.896+0000 7fec9bfff700 1 -- 172.21.15.47:0/1071063657 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fec8c01a060 con 0x7feca4095730 2022-06-29T15:49:53.915 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.896+0000 7fec9bfff700 1 --2- 172.21.15.47:0/1071063657 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fec7c053980 0x7fec7c055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:53.915 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.900+0000 7fec9bfff700 1 -- 172.21.15.47:0/1071063657 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fec8c06b7d0 con 0x7feca4095730 2022-06-29T15:49:53.915 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.900+0000 7feca27fc700 1 --2- 172.21.15.47:0/1071063657 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fec7c053980 0x7fec7c055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:53.915 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.900+0000 7feca27fc700 1 --2- 172.21.15.47:0/1071063657 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fec7c053980 0x7fec7c055e40 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7fec9401b000 tx=0x7fec94016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:53.916 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:53.904+0000 7fec9bfff700 1 -- 172.21.15.47:0/1071063657 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fec8c042480 con 0x7feca4095730 2022-06-29T15:49:54.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.032+0000 7feca97fc700 1 -- 172.21.15.47:0/1071063657 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feca4069e20 con 0x7feca4095730 2022-06-29T15:49:54.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.036+0000 7fec9bfff700 1 -- 172.21.15.47:0/1071063657 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+16277 (secure 0 0 0) 0x7fec8c018020 con 0x7feca4095730 2022-06-29T15:49:54.050 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:49:54.051 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":101,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:49:51.792208+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:49:48.525959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2022-06-29T15:49:49.805331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:49:54.055 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.036+0000 7feca97fc700 1 -- 172.21.15.47:0/1071063657 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fec7c053980 msgr2=0x7fec7c055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:54.056 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.040+0000 7feca97fc700 1 --2- 172.21.15.47:0/1071063657 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fec7c053980 0x7fec7c055e40 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7fec9401b000 tx=0x7fec94016000).stop 2022-06-29T15:49:54.056 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.040+0000 7feca97fc700 1 -- 172.21.15.47:0/1071063657 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feca4095730 msgr2=0x7feca40645e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:54.057 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.040+0000 7feca97fc700 1 --2- 172.21.15.47:0/1071063657 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feca4095730 0x7feca40645e0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 rx=0x7feca4095670 tx=0x7fec8c007b50).stop 2022-06-29T15:49:54.057 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.040+0000 7feca97fc700 1 -- 172.21.15.47:0/1071063657 shutdown_connections 2022-06-29T15:49:54.058 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.040+0000 7feca97fc700 1 --2- 172.21.15.47:0/1071063657 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7feca41430f0 0x7feca40671f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:54.058 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.040+0000 7feca97fc700 1 --2- 172.21.15.47:0/1071063657 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fec7c053980 0x7fec7c055e40 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:54.058 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.040+0000 7feca97fc700 1 --2- 172.21.15.47:0/1071063657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feca40a93c0 0x7feca4064ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:54.059 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.040+0000 7feca97fc700 1 --2- 172.21.15.47:0/1071063657 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feca4095730 0x7feca40645e0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:54.060 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.040+0000 7feca97fc700 1 -- 172.21.15.47:0/1071063657 >> 172.21.15.47:0/1071063657 conn(0x7feca405f0b0 msgr2=0x7feca4140be0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:54.060 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.040+0000 7feca97fc700 1 -- 172.21.15.47:0/1071063657 shutdown_connections 2022-06-29T15:49:54.060 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.040+0000 7feca97fc700 1 -- 172.21.15.47:0/1071063657 wait complete. 2022-06-29T15:49:54.064 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-29T15:42:16.971431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-29T15:42:25.290884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-29T15:49:48.525959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2022-06-29T15:49:49.805331+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-29T15:49:54.065 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-29T15:49:54.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.164+0000 7f05fffff700 1 Processor -- start 2022-06-29T15:49:54.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.164+0000 7f05fffff700 1 -- start start 2022-06-29T15:49:54.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.164+0000 7f05fffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f060005fc50 0x7f0600144cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:54.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.164+0000 7f05fffff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f06000b17a0 0x7f06000a7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:54.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.164+0000 7f05fffff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06000b02c0 con 0x7f060005f3f0 2022-06-29T15:49:54.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.164+0000 7f05fffff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f060005f7d0 con 0x7f060005fc50 2022-06-29T15:49:54.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.164+0000 7f05fffff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06001481e0 con 0x7f06000b17a0 2022-06-29T15:49:54.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.168+0000 7f05ff7fe700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f06000b17a0 0x7f06000a7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:54.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.168+0000 7f05fe7fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f060005fc50 0x7f0600144cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:54.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.168+0000 7f05feffd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f060005f3f0 0x7f06001445b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:44646/0 (socket says 172.21.15.47:44646) 2022-06-29T15:49:54.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.168+0000 7f05feffd700 1 -- 172.21.15.47:0/992018891 learned_addr learned my addr 172.21.15.47:0/992018891 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:49:54.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.168+0000 7f05fdffb700 1 -- 172.21.15.47:0/992018891 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1483395862 0 0) 0x7f06000b02c0 con 0x7f060005f3f0 2022-06-29T15:49:54.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.168+0000 7f05fdffb700 1 -- 172.21.15.47:0/992018891 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05e0003260 con 0x7f060005f3f0 2022-06-29T15:49:54.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.168+0000 7f05fe7fc700 1 -- 172.21.15.47:0/992018891 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f06000b17a0 msgr2=0x7f06000a7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:54.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.168+0000 7f05fe7fc700 1 --2- 172.21.15.47:0/992018891 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f06000b17a0 0x7f06000a7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:54.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.168+0000 7f05fe7fc700 1 -- 172.21.15.47:0/992018891 >> v1:172.21.15.64:6789/0 conn(0x7f060005f3f0 legacy=0x7f06001445b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:54.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.168+0000 7f05fe7fc700 1 -- 172.21.15.47:0/992018891 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06000b04f0 con 0x7f060005fc50 2022-06-29T15:49:54.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.168+0000 7f05ff7fe700 1 --2- 172.21.15.47:0/992018891 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f06000b17a0 0x7f06000a7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:49:54.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.168+0000 7f05fe7fc700 1 --2- 172.21.15.47:0/992018891 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f060005fc50 0x7f0600144cc0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7f05e8002f70 tx=0x7f05e8006df0).ready entity=mon.1 client_cookie=fd1bd90dea22bdcf server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:54.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.168+0000 7f05fdffb700 1 -- 172.21.15.47:0/992018891 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f05e800d4a0 con 0x7f060005fc50 2022-06-29T15:49:54.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.168+0000 7f05fdffb700 1 -- 172.21.15.47:0/992018891 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05e8000e90 con 0x7f060005fc50 2022-06-29T15:49:54.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.168+0000 7f05fdffb700 1 -- 172.21.15.47:0/992018891 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f05e800d7f0 con 0x7f060005fc50 2022-06-29T15:49:54.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.168+0000 7f05fffff700 1 -- 172.21.15.47:0/992018891 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f060005fc50 msgr2=0x7f0600144cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:54.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.168+0000 7f05fffff700 1 --2- 172.21.15.47:0/992018891 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f060005fc50 0x7f0600144cc0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7f05e8002f70 tx=0x7f05e8006df0).stop 2022-06-29T15:49:54.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.168+0000 7f05fffff700 1 -- 172.21.15.47:0/992018891 shutdown_connections 2022-06-29T15:49:54.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.168+0000 7f05fffff700 1 --2- 172.21.15.47:0/992018891 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f06000b17a0 0x7f06000a7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:54.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.168+0000 7f05fffff700 1 --2- 172.21.15.47:0/992018891 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f060005fc50 0x7f0600144cc0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:54.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.168+0000 7f05fffff700 1 -- 172.21.15.47:0/992018891 >> 172.21.15.47:0/992018891 conn(0x7f06000b5320 msgr2=0x7f06000b5700 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:54.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.168+0000 7f05fffff700 1 -- 172.21.15.47:0/992018891 shutdown_connections 2022-06-29T15:49:54.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.168+0000 7f05fffff700 1 -- 172.21.15.47:0/992018891 wait complete. 2022-06-29T15:49:54.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.172+0000 7f05fffff700 1 Processor -- start 2022-06-29T15:49:54.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.172+0000 7f05fffff700 1 -- start start 2022-06-29T15:49:54.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.172+0000 7f05fffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f060005f3f0 0x7f060014b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:54.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.172+0000 7f05fffff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f060005fc50 0x7f06000a8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:54.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.172+0000 7f05fffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06000b17a0 0x7f06000ab730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:54.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.172+0000 7f05fffff700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06000a9850 con 0x7f06000b17a0 2022-06-29T15:49:54.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.172+0000 7f05fffff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0600149ca0 con 0x7f060005f3f0 2022-06-29T15:49:54.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.172+0000 7f05fffff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0600149f40 con 0x7f060005fc50 2022-06-29T15:49:54.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.172+0000 7f05ff7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06000b17a0 0x7f06000ab730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:54.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.172+0000 7f05fe7fc700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f060005fc50 0x7f06000a8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:54.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.172+0000 7f05ff7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06000b17a0 0x7f06000ab730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51538/0 (socket says 172.21.15.47:51538) 2022-06-29T15:49:54.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.172+0000 7f05ff7fe700 1 -- 172.21.15.47:0/432849656 learned_addr learned my addr 172.21.15.47:0/432849656 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:54.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.172+0000 7f05feffd700 1 --2- 172.21.15.47:0/432849656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f060005f3f0 0x7f060014b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:54.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.172+0000 7f05ff7fe700 1 -- 172.21.15.47:0/432849656 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f060005fc50 msgr2=0x7f06000a8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:54.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.172+0000 7f05ff7fe700 1 --2- 172.21.15.47:0/432849656 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f060005fc50 0x7f06000a8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:54.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.172+0000 7f05ff7fe700 1 -- 172.21.15.47:0/432849656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f060005f3f0 msgr2=0x7f060014b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:54.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.172+0000 7f05ff7fe700 1 --2- 172.21.15.47:0/432849656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f060005f3f0 0x7f060014b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:54.191 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.172+0000 7f05ff7fe700 1 -- 172.21.15.47:0/432849656 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f0009470 con 0x7f06000b17a0 2022-06-29T15:49:54.191 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.172+0000 7f05fe7fc700 1 --2- 172.21.15.47:0/432849656 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f060005fc50 0x7f06000a8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:49:54.191 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.172+0000 7f05feffd700 1 --2- 172.21.15.47:0/432849656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f060005f3f0 0x7f060014b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:49:54.191 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.172+0000 7f05ff7fe700 1 --2- 172.21.15.47:0/432849656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06000b17a0 0x7f06000ab730 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 rx=0x7f05f000c000 tx=0x7f05f00091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:54.192 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.172+0000 7f05f7fff700 1 -- 172.21.15.47:0/432849656 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f05f000ee60 con 0x7f06000b17a0 2022-06-29T15:49:54.192 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.172+0000 7f05f7fff700 1 -- 172.21.15.47:0/432849656 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f000c850 con 0x7f06000b17a0 2022-06-29T15:49:54.192 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.172+0000 7f05fffff700 1 -- 172.21.15.47:0/432849656 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05e800b180 con 0x7f06000b17a0 2022-06-29T15:49:54.192 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.172+0000 7f05f7fff700 1 -- 172.21.15.47:0/432849656 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f05f000cbb0 con 0x7f06000b17a0 2022-06-29T15:49:54.193 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.172+0000 7f05fffff700 1 -- 172.21.15.47:0/432849656 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f060014a680 con 0x7f06000b17a0 2022-06-29T15:49:54.193 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.176+0000 7f05fffff700 1 -- 172.21.15.47:0/432849656 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f060014a3e0 con 0x7f06000b17a0 2022-06-29T15:49:54.193 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.176+0000 7f05f7fff700 1 -- 172.21.15.47:0/432849656 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f05f0022070 con 0x7f06000b17a0 2022-06-29T15:49:54.194 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.176+0000 7f05f7fff700 1 --2- 172.21.15.47:0/432849656 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f05e40539e0 0x7f05e4055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:54.194 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.176+0000 7f05f7fff700 1 -- 172.21.15.47:0/432849656 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f05f006c840 con 0x7f06000b17a0 2022-06-29T15:49:54.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.176+0000 7f05feffd700 1 --2- 172.21.15.47:0/432849656 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f05e40539e0 0x7f05e4055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:54.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.176+0000 7f05feffd700 1 --2- 172.21.15.47:0/432849656 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f05e40539e0 0x7f05e4055ea0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7f05f800d000 tx=0x7f05f8009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:54.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.184+0000 7f05f7fff700 1 -- 172.21.15.47:0/432849656 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f060014a3e0 con 0x7f06000b17a0 2022-06-29T15:49:54.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.308+0000 7f05fffff700 1 -- 172.21.15.47:0/432849656 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f060014a3e0 con 0x7f06000b17a0 2022-06-29T15:49:54.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.312+0000 7f05f7fff700 1 -- 172.21.15.47:0/432849656 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v101) v1 ==== 110+0+10 (secure 0 0 0) 0x7f05f0016020 con 0x7f06000b17a0 2022-06-29T15:49:54.320 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2022-06-29T15:49:54.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.312+0000 7f05fffff700 1 -- 172.21.15.47:0/432849656 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f05e40539e0 msgr2=0x7f05e4055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:54.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.312+0000 7f05fffff700 1 --2- 172.21.15.47:0/432849656 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f05e40539e0 0x7f05e4055ea0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7f05f800d000 tx=0x7f05f8009040).stop 2022-06-29T15:49:54.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.312+0000 7f05fffff700 1 -- 172.21.15.47:0/432849656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06000b17a0 msgr2=0x7f06000ab730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:54.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.312+0000 7f05fffff700 1 --2- 172.21.15.47:0/432849656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06000b17a0 0x7f06000ab730 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 rx=0x7f05f000c000 tx=0x7f05f00091d0).stop 2022-06-29T15:49:54.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.312+0000 7f05fffff700 1 -- 172.21.15.47:0/432849656 shutdown_connections 2022-06-29T15:49:54.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.312+0000 7f05fffff700 1 --2- 172.21.15.47:0/432849656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06000b17a0 0x7f06000ab730 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:54.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.312+0000 7f05fffff700 1 --2- 172.21.15.47:0/432849656 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f060005fc50 0x7f06000a8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:54.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.312+0000 7f05fffff700 1 --2- 172.21.15.47:0/432849656 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f05e40539e0 0x7f05e4055ea0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:54.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.312+0000 7f05fffff700 1 --2- 172.21.15.47:0/432849656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f060005f3f0 0x7f060014b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:54.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.312+0000 7f05fffff700 1 -- 172.21.15.47:0/432849656 >> 172.21.15.47:0/432849656 conn(0x7f06000b5320 msgr2=0x7f060014dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:54.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.312+0000 7f05fffff700 1 -- 172.21.15.47:0/432849656 shutdown_connections 2022-06-29T15:49:54.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.312+0000 7f05fffff700 1 -- 172.21.15.47:0/432849656 wait complete. 2022-06-29T15:49:54.332 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-29T15:49:54.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.428+0000 7fb355602700 1 Processor -- start 2022-06-29T15:49:54.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.432+0000 7fb355602700 1 -- start start 2022-06-29T15:49:54.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.432+0000 7fb355602700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb350060900 0x7fb3500b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:54.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.432+0000 7fb355602700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb3500b2ba0 0x7fb350060ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:54.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.432+0000 7fb355602700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3500b1d60 con 0x7fb35013fe70 2022-06-29T15:49:54.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.432+0000 7fb355602700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb350144410 con 0x7fb350060900 2022-06-29T15:49:54.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.432+0000 7fb355602700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3500a8e60 con 0x7fb3500b2ba0 2022-06-29T15:49:54.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.432+0000 7fb34effd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb350060900 0x7fb3500b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:54.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.432+0000 7fb34effd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb350060900 0x7fb3500b1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51544/0 (socket says 172.21.15.47:51544) 2022-06-29T15:49:54.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.432+0000 7fb34effd700 1 -- 172.21.15.47:0/3770760023 learned_addr learned my addr 172.21.15.47:0/3770760023 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:54.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.432+0000 7fb34e7fc700 1 --2- 172.21.15.47:0/3770760023 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb3500b2ba0 0x7fb350060ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:54.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.432+0000 7fb34effd700 1 -- 172.21.15.47:0/3770760023 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb3500b2ba0 msgr2=0x7fb350060ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:54.446 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.432+0000 7fb34effd700 1 --2- 172.21.15.47:0/3770760023 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb3500b2ba0 0x7fb350060ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:54.446 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.432+0000 7fb34effd700 1 -- 172.21.15.47:0/3770760023 >> v1:172.21.15.64:6789/0 conn(0x7fb35013fe70 legacy=0x7fb350143d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:54.446 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.432+0000 7fb34dffb700 1 -- 172.21.15.47:0/3770760023 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2005255904 0 0) 0x7fb3500b1d60 con 0x7fb35013fe70 2022-06-29T15:49:54.446 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.432+0000 7fb34effd700 1 -- 172.21.15.47:0/3770760023 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3501435c0 con 0x7fb350060900 2022-06-29T15:49:54.447 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.432+0000 7fb34e7fc700 1 --2- 172.21.15.47:0/3770760023 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb3500b2ba0 0x7fb350060ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:49:54.447 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.432+0000 7fb34effd700 1 --2- 172.21.15.47:0/3770760023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb350060900 0x7fb3500b1f60 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 rx=0x7fb344002f70 tx=0x7fb344006eb0).ready entity=mon.0 client_cookie=731712df13554fc2 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:54.447 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.432+0000 7fb34dffb700 1 -- 172.21.15.47:0/3770760023 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb34400d570 con 0x7fb350060900 2022-06-29T15:49:54.447 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.432+0000 7fb34dffb700 1 -- 172.21.15.47:0/3770760023 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb344000e90 con 0x7fb350060900 2022-06-29T15:49:54.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.432+0000 7fb34dffb700 1 -- 172.21.15.47:0/3770760023 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb34400d8c0 con 0x7fb350060900 2022-06-29T15:49:54.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.432+0000 7fb355602700 1 -- 172.21.15.47:0/3770760023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb350060900 msgr2=0x7fb3500b1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:54.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.432+0000 7fb355602700 1 --2- 172.21.15.47:0/3770760023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb350060900 0x7fb3500b1f60 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 rx=0x7fb344002f70 tx=0x7fb344006eb0).stop 2022-06-29T15:49:54.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.432+0000 7fb355602700 1 -- 172.21.15.47:0/3770760023 shutdown_connections 2022-06-29T15:49:54.449 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.432+0000 7fb355602700 1 --2- 172.21.15.47:0/3770760023 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb3500b2ba0 0x7fb350060ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:54.449 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.432+0000 7fb355602700 1 --2- 172.21.15.47:0/3770760023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb350060900 0x7fb3500b1f60 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:54.449 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.432+0000 7fb355602700 1 -- 172.21.15.47:0/3770760023 >> 172.21.15.47:0/3770760023 conn(0x7fb3500626f0 msgr2=0x7fb350062ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:54.449 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.432+0000 7fb355602700 1 -- 172.21.15.47:0/3770760023 shutdown_connections 2022-06-29T15:49:54.450 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.432+0000 7fb355602700 1 -- 172.21.15.47:0/3770760023 wait complete. 2022-06-29T15:49:54.452 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.436+0000 7fb355602700 1 Processor -- start 2022-06-29T15:49:54.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.436+0000 7fb355602700 1 -- start start 2022-06-29T15:49:54.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.436+0000 7fb355602700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb350060900 0x7fb3501425f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:54.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.436+0000 7fb355602700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb3500b2ba0 0x7fb3500aca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:54.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.436+0000 7fb355602700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb35013fe70 0x7fb3501338e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:54.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.436+0000 7fb355602700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb350142130 con 0x7fb35013fe70 2022-06-29T15:49:54.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.436+0000 7fb355602700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3500af2d0 con 0x7fb350060900 2022-06-29T15:49:54.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.436+0000 7fb355602700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3500af500 con 0x7fb3500b2ba0 2022-06-29T15:49:54.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.436+0000 7fb34f7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb35013fe70 0x7fb3501338e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:54.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.436+0000 7fb34f7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb35013fe70 0x7fb3501338e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51552/0 (socket says 172.21.15.47:51552) 2022-06-29T15:49:54.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.436+0000 7fb34f7fe700 1 -- 172.21.15.47:0/2312076704 learned_addr learned my addr 172.21.15.47:0/2312076704 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:54.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.436+0000 7fb34e7fc700 1 --2- 172.21.15.47:0/2312076704 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb3500b2ba0 0x7fb3500aca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:54.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.436+0000 7fb34effd700 1 --2- 172.21.15.47:0/2312076704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb350060900 0x7fb3501425f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:54.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.436+0000 7fb34f7fe700 1 -- 172.21.15.47:0/2312076704 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb3500b2ba0 msgr2=0x7fb3500aca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:54.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.436+0000 7fb34f7fe700 1 --2- 172.21.15.47:0/2312076704 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb3500b2ba0 0x7fb3500aca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:54.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.436+0000 7fb34f7fe700 1 -- 172.21.15.47:0/2312076704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb350060900 msgr2=0x7fb3501425f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:54.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.436+0000 7fb34f7fe700 1 --2- 172.21.15.47:0/2312076704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb350060900 0x7fb3501425f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:54.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.436+0000 7fb34f7fe700 1 -- 172.21.15.47:0/2312076704 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb34000b180 con 0x7fb35013fe70 2022-06-29T15:49:54.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.436+0000 7fb34f7fe700 1 --2- 172.21.15.47:0/2312076704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb35013fe70 0x7fb3501338e0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 rx=0x7fb350141a50 tx=0x7fb34800c800).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:54.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.436+0000 7fb33ffff700 1 -- 172.21.15.47:0/2312076704 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb348006aa0 con 0x7fb35013fe70 2022-06-29T15:49:54.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.436+0000 7fb33ffff700 1 -- 172.21.15.47:0/2312076704 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb348006c00 con 0x7fb35013fe70 2022-06-29T15:49:54.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.436+0000 7fb355602700 1 -- 172.21.15.47:0/2312076704 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb34400b1e0 con 0x7fb35013fe70 2022-06-29T15:49:54.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.436+0000 7fb33ffff700 1 -- 172.21.15.47:0/2312076704 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb348007030 con 0x7fb35013fe70 2022-06-29T15:49:54.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.436+0000 7fb355602700 1 -- 172.21.15.47:0/2312076704 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb350142360 con 0x7fb35013fe70 2022-06-29T15:49:54.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.440+0000 7fb355602700 1 -- 172.21.15.47:0/2312076704 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb350141b90 con 0x7fb35013fe70 2022-06-29T15:49:54.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.440+0000 7fb33ffff700 1 -- 172.21.15.47:0/2312076704 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb348010030 con 0x7fb35013fe70 2022-06-29T15:49:54.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.440+0000 7fb33ffff700 1 --2- 172.21.15.47:0/2312076704 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb32c053a10 0x7fb32c055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:54.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.440+0000 7fb33ffff700 1 -- 172.21.15.47:0/2312076704 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb34806beb0 con 0x7fb35013fe70 2022-06-29T15:49:54.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.440+0000 7fb34effd700 1 --2- 172.21.15.47:0/2312076704 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb32c053a10 0x7fb32c055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:54.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.440+0000 7fb34effd700 1 --2- 172.21.15.47:0/2312076704 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb32c053a10 0x7fb32c055ed0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7fb34401c000 tx=0x7fb344015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:54.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.448+0000 7fb33ffff700 1 -- 172.21.15.47:0/2312076704 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb348042b60 con 0x7fb35013fe70 2022-06-29T15:49:54.583 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.572+0000 7fb355602700 1 -- 172.21.15.47:0/2312076704 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb3500573d0 con 0x7fb35013fe70 2022-06-29T15:49:54.584 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.576+0000 7fb33ffff700 1 -- 172.21.15.47:0/2312076704 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v101) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb348046180 con 0x7fb35013fe70 2022-06-29T15:49:54.584 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:49:54.586 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.576+0000 7fb355602700 1 -- 172.21.15.47:0/2312076704 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb32c053a10 msgr2=0x7fb32c055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:54.586 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.576+0000 7fb355602700 1 --2- 172.21.15.47:0/2312076704 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb32c053a10 0x7fb32c055ed0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7fb34401c000 tx=0x7fb344015000).stop 2022-06-29T15:49:54.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.576+0000 7fb355602700 1 -- 172.21.15.47:0/2312076704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb35013fe70 msgr2=0x7fb3501338e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:54.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.576+0000 7fb355602700 1 --2- 172.21.15.47:0/2312076704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb35013fe70 0x7fb3501338e0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 rx=0x7fb350141a50 tx=0x7fb34800c800).stop 2022-06-29T15:49:54.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.576+0000 7fb355602700 1 -- 172.21.15.47:0/2312076704 shutdown_connections 2022-06-29T15:49:54.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.576+0000 7fb355602700 1 --2- 172.21.15.47:0/2312076704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb35013fe70 0x7fb3501338e0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:54.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.576+0000 7fb355602700 1 --2- 172.21.15.47:0/2312076704 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb3500b2ba0 0x7fb3500aca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:54.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.576+0000 7fb355602700 1 --2- 172.21.15.47:0/2312076704 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb32c053a10 0x7fb32c055ed0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:54.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.576+0000 7fb355602700 1 --2- 172.21.15.47:0/2312076704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb350060900 0x7fb3501425f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:54.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.576+0000 7fb355602700 1 -- 172.21.15.47:0/2312076704 >> 172.21.15.47:0/2312076704 conn(0x7fb3500626f0 msgr2=0x7fb3500b1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:54.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.576+0000 7fb355602700 1 -- 172.21.15.47:0/2312076704 shutdown_connections 2022-06-29T15:49:54.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.576+0000 7fb355602700 1 -- 172.21.15.47:0/2312076704 wait complete. 2022-06-29T15:49:54.603 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-29T15:49:54.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.696+0000 7f074359e700 1 Processor -- start 2022-06-29T15:49:54.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.696+0000 7f074359e700 1 -- start start 2022-06-29T15:49:54.721 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.700+0000 7f074359e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f07440b6d20 0x7f07440b1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:54.721 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.700+0000 7f074359e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f07441467a0 0x7f07440b1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:54.722 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.700+0000 7f074359e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07440b1050 con 0x7f0744140500 2022-06-29T15:49:54.722 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.700+0000 7f074359e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07440b23c0 con 0x7f07440b6d20 2022-06-29T15:49:54.723 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.700+0000 7f074359e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f074405fab0 con 0x7f07441467a0 2022-06-29T15:49:54.723 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.700+0000 7f0742d9d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0744140500 0x7f07440b2e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37114/0 (socket says 172.21.15.47:37114) 2022-06-29T15:49:54.724 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.700+0000 7f0742d9d700 1 -- 172.21.15.47:0/3664591335 learned_addr learned my addr 172.21.15.47:0/3664591335 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:49:54.724 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.700+0000 7f074259c700 1 --2- 172.21.15.47:0/3664591335 >> v2:172.21.15.64:3300/0 conn(0x7f07440b6d20 0x7f07440b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:54.724 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.700+0000 7f0741d9b700 1 --2- 172.21.15.47:0/3664591335 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f07441467a0 0x7f07440b1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:54.725 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.700+0000 7f074159a700 1 -- 172.21.15.47:0/3664591335 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2547742 0 0) 0x7f07440b1050 con 0x7f0744140500 2022-06-29T15:49:54.725 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.700+0000 7f074159a700 1 -- 172.21.15.47:0/3664591335 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0724003260 con 0x7f0744140500 2022-06-29T15:49:54.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.700+0000 7f074159a700 1 -- 172.21.15.47:0/3664591335 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1350377532 0 0) 0x7f0724003260 con 0x7f0744140500 2022-06-29T15:49:54.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.700+0000 7f074159a700 1 -- 172.21.15.47:0/3664591335 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f07441467a0 msgr2=0x7f07440b1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:54.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.700+0000 7f074159a700 1 --2- 172.21.15.47:0/3664591335 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f07441467a0 0x7f07440b1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:54.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.700+0000 7f074159a700 1 -- 172.21.15.47:0/3664591335 >> v2:172.21.15.64:3300/0 conn(0x7f07440b6d20 msgr2=0x7f07440b1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:54.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.700+0000 7f074159a700 1 --2- 172.21.15.47:0/3664591335 >> v2:172.21.15.64:3300/0 conn(0x7f07440b6d20 0x7f07440b1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:54.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.700+0000 7f074159a700 1 -- 172.21.15.47:0/3664591335 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07440b7100 con 0x7f0744140500 2022-06-29T15:49:54.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.700+0000 7f074159a700 1 -- 172.21.15.47:0/3664591335 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7f073c0019e0 con 0x7f0744140500 2022-06-29T15:49:54.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.700+0000 7f074159a700 1 -- 172.21.15.47:0/3664591335 >> v1:172.21.15.47:6789/0 conn(0x7f0744140500 legacy=0x7f07440b2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:54.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.700+0000 7f074159a700 1 --2- 172.21.15.47:0/3664591335 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f07441467a0 0x7f0724003750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:54.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.700+0000 7f074159a700 1 --2- 172.21.15.47:0/3664591335 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0724003c80 0x7f0724004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:54.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.700+0000 7f074159a700 1 --2- 172.21.15.47:0/3664591335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07240049f0 0x7f0724004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:54.731 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.700+0000 7f074159a700 1 -- 172.21.15.47:0/3664591335 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072400b750 con 0x7f0724003c80 2022-06-29T15:49:54.731 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.700+0000 7f074159a700 1 -- 172.21.15.47:0/3664591335 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072400b890 con 0x7f07240049f0 2022-06-29T15:49:54.732 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.700+0000 7f074159a700 1 -- 172.21.15.47:0/3664591335 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f072400b9d0 con 0x7f07441467a0 2022-06-29T15:49:54.732 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.700+0000 7f0741d9b700 1 --2- 172.21.15.47:0/3664591335 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0724003c80 0x7f0724004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:54.732 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.700+0000 7f0742d9d700 1 --2- 172.21.15.47:0/3664591335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07240049f0 0x7f0724004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:54.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.700+0000 7f074259c700 1 --2- 172.21.15.47:0/3664591335 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f07441467a0 0x7f0724003750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:54.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.700+0000 7f0741d9b700 1 -- 172.21.15.47:0/3664591335 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f07441467a0 msgr2=0x7f0724003750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:54.734 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.700+0000 7f0741d9b700 1 --2- 172.21.15.47:0/3664591335 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f07441467a0 0x7f0724003750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:54.734 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.700+0000 7f0741d9b700 1 -- 172.21.15.47:0/3664591335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07240049f0 msgr2=0x7f0724004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:54.734 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.700+0000 7f0741d9b700 1 --2- 172.21.15.47:0/3664591335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07240049f0 0x7f0724004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:54.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.700+0000 7f0741d9b700 1 -- 172.21.15.47:0/3664591335 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0724003260 con 0x7f0724003c80 2022-06-29T15:49:54.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.700+0000 7f074259c700 1 --2- 172.21.15.47:0/3664591335 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f07441467a0 0x7f0724003750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:49:54.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.700+0000 7f0741d9b700 1 --2- 172.21.15.47:0/3664591335 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0724003c80 0x7f0724004500 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 rx=0x7f07240054d0 tx=0x7f072c006df0).ready entity=mon.0 client_cookie=5c529ddaf6aa179a server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:54.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.700+0000 7f074159a700 1 -- 172.21.15.47:0/3664591335 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f072c00e830 con 0x7f0724003c80 2022-06-29T15:49:54.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.700+0000 7f074159a700 1 -- 172.21.15.47:0/3664591335 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f072c00e990 con 0x7f0724003c80 2022-06-29T15:49:54.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.700+0000 7f074159a700 1 -- 172.21.15.47:0/3664591335 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f072c00ecf0 con 0x7f0724003c80 2022-06-29T15:49:54.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.700+0000 7f074359e700 1 -- 172.21.15.47:0/3664591335 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0724003c80 msgr2=0x7f0724004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:54.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.700+0000 7f074359e700 1 --2- 172.21.15.47:0/3664591335 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0724003c80 0x7f0724004500 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 rx=0x7f07240054d0 tx=0x7f072c006df0).stop 2022-06-29T15:49:54.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.700+0000 7f0742d9d700 1 -- 172.21.15.47:0/3664591335 reap_dead start 2022-06-29T15:49:54.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.700+0000 7f074359e700 1 -- 172.21.15.47:0/3664591335 shutdown_connections 2022-06-29T15:49:54.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.700+0000 7f074359e700 1 -- 172.21.15.47:0/3664591335 >> 172.21.15.47:0/3664591335 conn(0x7f0744063670 msgr2=0x7f0744063a50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:54.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.700+0000 7f074359e700 1 -- 172.21.15.47:0/3664591335 shutdown_connections 2022-06-29T15:49:54.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.700+0000 7f074359e700 1 -- 172.21.15.47:0/3664591335 wait complete. 2022-06-29T15:49:54.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.704+0000 7f074359e700 1 Processor -- start 2022-06-29T15:49:54.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.704+0000 7f074359e700 1 -- start start 2022-06-29T15:49:54.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.704+0000 7f074359e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0724003c80 0x7f07440acec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:54.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.704+0000 7f074359e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f07440aab40 0x7f0744060160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:54.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.704+0000 7f074359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0744152290 0x7f0744152720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:54.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.704+0000 7f074359e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07441411d0 con 0x7f0724003c80 2022-06-29T15:49:54.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.704+0000 7f074359e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07440ac550 con 0x7f0744152290 2022-06-29T15:49:54.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.704+0000 7f074359e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07440aa540 con 0x7f07440aab40 2022-06-29T15:49:54.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.704+0000 7f074259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0724003c80 0x7f07440acec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:54.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.704+0000 7f074259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0724003c80 0x7f07440acec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51568/0 (socket says 172.21.15.47:51568) 2022-06-29T15:49:54.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.704+0000 7f074259c700 1 -- 172.21.15.47:0/753679503 learned_addr learned my addr 172.21.15.47:0/753679503 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:54.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.704+0000 7f0741d9b700 1 --2- 172.21.15.47:0/753679503 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f07440aab40 0x7f0744060160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:54.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.704+0000 7f0742d9d700 1 --2- 172.21.15.47:0/753679503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0744152290 0x7f0744152720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:54.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.704+0000 7f074259c700 1 -- 172.21.15.47:0/753679503 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f07440aab40 msgr2=0x7f0744060160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:54.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.704+0000 7f074259c700 1 --2- 172.21.15.47:0/753679503 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f07440aab40 0x7f0744060160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:54.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.704+0000 7f074259c700 1 -- 172.21.15.47:0/753679503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0744152290 msgr2=0x7f0744152720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:54.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.704+0000 7f074259c700 1 --2- 172.21.15.47:0/753679503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0744152290 0x7f0744152720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:54.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.704+0000 7f074259c700 1 -- 172.21.15.47:0/753679503 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0744156590 con 0x7f0724003c80 2022-06-29T15:49:54.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.704+0000 7f074259c700 1 --2- 172.21.15.47:0/753679503 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0724003c80 0x7f07440acec0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 rx=0x7f073400b000 tx=0x7f0734008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:54.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.704+0000 7f073b7fe700 1 -- 172.21.15.47:0/753679503 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0734014680 con 0x7f0724003c80 2022-06-29T15:49:54.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.704+0000 7f073b7fe700 1 -- 172.21.15.47:0/753679503 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07340147e0 con 0x7f0724003c80 2022-06-29T15:49:54.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.704+0000 7f074359e700 1 -- 172.21.15.47:0/753679503 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07440ac780 con 0x7f0724003c80 2022-06-29T15:49:54.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.704+0000 7f073b7fe700 1 -- 172.21.15.47:0/753679503 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0734014b40 con 0x7f0724003c80 2022-06-29T15:49:54.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.704+0000 7f074359e700 1 -- 172.21.15.47:0/753679503 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0744067220 con 0x7f0724003c80 2022-06-29T15:49:54.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.708+0000 7f074359e700 1 -- 172.21.15.47:0/753679503 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07440573d0 con 0x7f0724003c80 2022-06-29T15:49:54.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.708+0000 7f073b7fe700 1 -- 172.21.15.47:0/753679503 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0734024070 con 0x7f0724003c80 2022-06-29T15:49:54.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.708+0000 7f073b7fe700 1 --2- 172.21.15.47:0/753679503 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0728053aa0 0x7f0728055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:54.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.708+0000 7f073b7fe700 1 -- 172.21.15.47:0/753679503 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f073406c920 con 0x7f0724003c80 2022-06-29T15:49:54.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.708+0000 7f0741d9b700 1 --2- 172.21.15.47:0/753679503 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0728053aa0 0x7f0728055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:54.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.708+0000 7f0741d9b700 1 --2- 172.21.15.47:0/753679503 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0728053aa0 0x7f0728055f60 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7f072c01e000 tx=0x7f072c019000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:54.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.712+0000 7f073b7fe700 1 -- 172.21.15.47:0/753679503 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f07340436e0 con 0x7f0724003c80 2022-06-29T15:49:54.845 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.836+0000 7f074359e700 1 -- 172.21.15.47:0/753679503 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0744062040 con 0x7f0724003c80 2022-06-29T15:49:54.846 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.836+0000 7f073b7fe700 1 -- 172.21.15.47:0/753679503 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v101) v1 ==== 104+0+10 (secure 0 0 0) 0x7f0734046d00 con 0x7f0724003c80 2022-06-29T15:49:54.847 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:49:54.849 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.840+0000 7f074359e700 1 -- 172.21.15.47:0/753679503 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0728053aa0 msgr2=0x7f0728055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:54.850 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.840+0000 7f074359e700 1 --2- 172.21.15.47:0/753679503 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0728053aa0 0x7f0728055f60 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7f072c01e000 tx=0x7f072c019000).stop 2022-06-29T15:49:54.850 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.840+0000 7f074359e700 1 -- 172.21.15.47:0/753679503 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0724003c80 msgr2=0x7f07440acec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:54.850 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.840+0000 7f074359e700 1 --2- 172.21.15.47:0/753679503 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0724003c80 0x7f07440acec0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 rx=0x7f073400b000 tx=0x7f0734008040).stop 2022-06-29T15:49:54.850 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.840+0000 7f074359e700 1 -- 172.21.15.47:0/753679503 shutdown_connections 2022-06-29T15:49:54.851 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.840+0000 7f074359e700 1 --2- 172.21.15.47:0/753679503 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0728053aa0 0x7f0728055f60 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:54.851 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.840+0000 7f074359e700 1 --2- 172.21.15.47:0/753679503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0744152290 0x7f0744152720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:54.851 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.840+0000 7f074359e700 1 --2- 172.21.15.47:0/753679503 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f07440aab40 0x7f0744060160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:54.851 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.840+0000 7f074359e700 1 --2- 172.21.15.47:0/753679503 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0724003c80 0x7f07440acec0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:54.852 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.840+0000 7f074359e700 1 -- 172.21.15.47:0/753679503 >> 172.21.15.47:0/753679503 conn(0x7f0744063670 msgr2=0x7f074413f990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:54.852 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.840+0000 7f074359e700 1 -- 172.21.15.47:0/753679503 shutdown_connections 2022-06-29T15:49:54.852 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.840+0000 7f074359e700 1 -- 172.21.15.47:0/753679503 wait complete. 2022-06-29T15:49:54.867 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-29T15:49:54.972 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.964+0000 7fed40679700 1 Processor -- start 2022-06-29T15:49:54.973 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.964+0000 7fed40679700 1 -- start start 2022-06-29T15:49:54.974 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.964+0000 7fed40679700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fed38140e10 0x7fed38067850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:54.974 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.964+0000 7fed40679700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fed380609d0 0x7fed38143e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:54.974 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.964+0000 7fed40679700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed381434e0 con 0x7fed38067470 2022-06-29T15:49:54.975 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.964+0000 7fed40679700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed38144370 con 0x7fed38140e10 2022-06-29T15:49:54.975 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.964+0000 7fed40679700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed38143740 con 0x7fed380609d0 2022-06-29T15:49:54.975 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.964+0000 7fed3dc14700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fed38140e10 0x7fed38067850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:54.976 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.964+0000 7fed3dc14700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fed38140e10 0x7fed38067850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51580/0 (socket says 172.21.15.47:51580) 2022-06-29T15:49:54.976 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.964+0000 7fed3dc14700 1 -- 172.21.15.47:0/108628159 learned_addr learned my addr 172.21.15.47:0/108628159 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:54.976 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.964+0000 7fed3ec16700 1 --2- 172.21.15.47:0/108628159 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fed380609d0 0x7fed38143e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:54.978 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.964+0000 7fed3dc14700 1 -- 172.21.15.47:0/108628159 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fed380609d0 msgr2=0x7fed38143e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:54.978 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.964+0000 7fed3d413700 1 -- 172.21.15.47:0/108628159 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3956729474 0 0) 0x7fed381434e0 con 0x7fed38067470 2022-06-29T15:49:54.978 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.964+0000 7fed3dc14700 1 --2- 172.21.15.47:0/108628159 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fed380609d0 0x7fed38143e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:54.979 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.964+0000 7fed3dc14700 1 -- 172.21.15.47:0/108628159 >> v1:172.21.15.64:6789/0 conn(0x7fed38067470 legacy=0x7fed38140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:54.979 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.964+0000 7fed3dc14700 1 -- 172.21.15.47:0/108628159 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed381439e0 con 0x7fed38140e10 2022-06-29T15:49:54.979 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.964+0000 7fed3dc14700 1 --2- 172.21.15.47:0/108628159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed38140e10 0x7fed38067850 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 rx=0x7fed34002f70 tx=0x7fed34006eb0).ready entity=mon.0 client_cookie=7b8f5c5913b4455c server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:54.979 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.968+0000 7fed3d413700 1 -- 172.21.15.47:0/108628159 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed3400d570 con 0x7fed38140e10 2022-06-29T15:49:54.980 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.968+0000 7fed3d413700 1 -- 172.21.15.47:0/108628159 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed34000e90 con 0x7fed38140e10 2022-06-29T15:49:54.980 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.968+0000 7fed3d413700 1 -- 172.21.15.47:0/108628159 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed3400d8c0 con 0x7fed38140e10 2022-06-29T15:49:54.980 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.968+0000 7fed40679700 1 -- 172.21.15.47:0/108628159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed38140e10 msgr2=0x7fed38067850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:54.980 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.968+0000 7fed40679700 1 --2- 172.21.15.47:0/108628159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed38140e10 0x7fed38067850 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 rx=0x7fed34002f70 tx=0x7fed34006eb0).stop 2022-06-29T15:49:54.981 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.968+0000 7fed40679700 1 -- 172.21.15.47:0/108628159 shutdown_connections 2022-06-29T15:49:54.981 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.968+0000 7fed40679700 1 --2- 172.21.15.47:0/108628159 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fed380609d0 0x7fed38143e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:54.981 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.968+0000 7fed40679700 1 --2- 172.21.15.47:0/108628159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed38140e10 0x7fed38067850 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:54.981 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.968+0000 7fed40679700 1 -- 172.21.15.47:0/108628159 >> 172.21.15.47:0/108628159 conn(0x7fed3805f0b0 msgr2=0x7fed3805f490 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:54.981 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.968+0000 7fed40679700 1 -- 172.21.15.47:0/108628159 shutdown_connections 2022-06-29T15:49:54.982 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.968+0000 7fed40679700 1 -- 172.21.15.47:0/108628159 wait complete. 2022-06-29T15:49:54.985 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.968+0000 7fed40679700 1 Processor -- start 2022-06-29T15:49:54.985 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.968+0000 7fed40679700 1 -- start start 2022-06-29T15:49:54.985 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.972+0000 7fed40679700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed380609d0 0x7fed380afb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:54.986 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.972+0000 7fed40679700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed38067470 0x7fed38152310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:54.986 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.972+0000 7fed40679700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fed38140e10 0x7fed38152800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:54.986 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.972+0000 7fed40679700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed380ad600 con 0x7fed380609d0 2022-06-29T15:49:54.986 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.972+0000 7fed40679700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed380b44b0 con 0x7fed38067470 2022-06-29T15:49:54.987 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.972+0000 7fed40679700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed380ad360 con 0x7fed38140e10 2022-06-29T15:49:54.987 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.972+0000 7fed3e415700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed380609d0 0x7fed380afb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:54.987 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.972+0000 7fed3e415700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed380609d0 0x7fed380afb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51582/0 (socket says 172.21.15.47:51582) 2022-06-29T15:49:54.987 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.972+0000 7fed3e415700 1 -- 172.21.15.47:0/1078410011 learned_addr learned my addr 172.21.15.47:0/1078410011 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:54.988 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.972+0000 7fed3dc14700 1 --2- 172.21.15.47:0/1078410011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed38067470 0x7fed38152310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:54.988 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.972+0000 7fed3ec16700 1 --2- 172.21.15.47:0/1078410011 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fed38140e10 0x7fed38152800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:54.988 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.972+0000 7fed3e415700 1 -- 172.21.15.47:0/1078410011 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fed38140e10 msgr2=0x7fed38152800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:54.988 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.972+0000 7fed3e415700 1 --2- 172.21.15.47:0/1078410011 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fed38140e10 0x7fed38152800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:54.988 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.972+0000 7fed3e415700 1 -- 172.21.15.47:0/1078410011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed38067470 msgr2=0x7fed38152310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:54.989 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.972+0000 7fed3e415700 1 --2- 172.21.15.47:0/1078410011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed38067470 0x7fed38152310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:54.989 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.972+0000 7fed3e415700 1 -- 172.21.15.47:0/1078410011 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed3400b1e0 con 0x7fed380609d0 2022-06-29T15:49:54.989 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.972+0000 7fed3dc14700 1 --2- 172.21.15.47:0/1078410011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed38067470 0x7fed38152310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:49:54.989 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.972+0000 7fed3e415700 1 --2- 172.21.15.47:0/1078410011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed380609d0 0x7fed380afb50 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 rx=0x7fed30001250 tx=0x7fed30007e40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:54.990 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.972+0000 7fed2f7fe700 1 -- 172.21.15.47:0/1078410011 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed30010510 con 0x7fed380609d0 2022-06-29T15:49:54.990 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.972+0000 7fed2f7fe700 1 -- 172.21.15.47:0/1078410011 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed30010670 con 0x7fed380609d0 2022-06-29T15:49:54.990 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.972+0000 7fed40679700 1 -- 172.21.15.47:0/1078410011 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed380b4710 con 0x7fed380609d0 2022-06-29T15:49:54.990 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.972+0000 7fed2f7fe700 1 -- 172.21.15.47:0/1078410011 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed30010820 con 0x7fed380609d0 2022-06-29T15:49:54.991 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.972+0000 7fed40679700 1 -- 172.21.15.47:0/1078410011 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed380af4e0 con 0x7fed380609d0 2022-06-29T15:49:54.991 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.976+0000 7fed40679700 1 -- 172.21.15.47:0/1078410011 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed380573d0 con 0x7fed380609d0 2022-06-29T15:49:54.991 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.976+0000 7fed2f7fe700 1 -- 172.21.15.47:0/1078410011 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fed30010070 con 0x7fed380609d0 2022-06-29T15:49:54.991 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.976+0000 7fed2f7fe700 1 --2- 172.21.15.47:0/1078410011 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fed18053b00 0x7fed18055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:54.992 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.976+0000 7fed2f7fe700 1 -- 172.21.15.47:0/1078410011 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fed3006bf10 con 0x7fed380609d0 2022-06-29T15:49:54.992 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.976+0000 7fed3dc14700 1 --2- 172.21.15.47:0/1078410011 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fed18053b00 0x7fed18055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:54.993 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.976+0000 7fed3dc14700 1 --2- 172.21.15.47:0/1078410011 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fed18053b00 0x7fed18055fc0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7fed3401a000 tx=0x7fed34015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:54.993 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:54.980+0000 7fed2f7fe700 1 -- 172.21.15.47:0/1078410011 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fed30042bc0 con 0x7fed380609d0 2022-06-29T15:49:55.119 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.108+0000 7fed40679700 1 -- 172.21.15.47:0/1078410011 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fed380b48a0 con 0x7fed380609d0 2022-06-29T15:49:55.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.108+0000 7fed2f7fe700 1 -- 172.21.15.47:0/1078410011 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v101) v1 ==== 100+0+10 (secure 0 0 0) 0x7fed300461e0 con 0x7fed380609d0 2022-06-29T15:49:55.120 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:49:55.121 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.112+0000 7fed40679700 1 -- 172.21.15.47:0/1078410011 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fed18053b00 msgr2=0x7fed18055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:55.122 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.112+0000 7fed40679700 1 --2- 172.21.15.47:0/1078410011 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fed18053b00 0x7fed18055fc0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7fed3401a000 tx=0x7fed34015000).stop 2022-06-29T15:49:55.122 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.112+0000 7fed40679700 1 -- 172.21.15.47:0/1078410011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed380609d0 msgr2=0x7fed380afb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:55.122 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.112+0000 7fed40679700 1 --2- 172.21.15.47:0/1078410011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed380609d0 0x7fed380afb50 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 rx=0x7fed30001250 tx=0x7fed30007e40).stop 2022-06-29T15:49:55.122 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.112+0000 7fed40679700 1 -- 172.21.15.47:0/1078410011 shutdown_connections 2022-06-29T15:49:55.123 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.112+0000 7fed40679700 1 --2- 172.21.15.47:0/1078410011 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fed38140e10 0x7fed38152800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:55.123 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.112+0000 7fed40679700 1 --2- 172.21.15.47:0/1078410011 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fed18053b00 0x7fed18055fc0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:55.123 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.112+0000 7fed40679700 1 --2- 172.21.15.47:0/1078410011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed38067470 0x7fed38152310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:55.123 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.112+0000 7fed40679700 1 --2- 172.21.15.47:0/1078410011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed380609d0 0x7fed380afb50 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:55.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.112+0000 7fed40679700 1 -- 172.21.15.47:0/1078410011 >> 172.21.15.47:0/1078410011 conn(0x7fed3805f0b0 msgr2=0x7fed380b07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:55.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.112+0000 7fed40679700 1 -- 172.21.15.47:0/1078410011 shutdown_connections 2022-06-29T15:49:55.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.112+0000 7fed40679700 1 -- 172.21.15.47:0/1078410011 wait complete. 2022-06-29T15:49:55.133 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-29T15:49:55.133 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:49:55.242 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.228+0000 7f9f3e241700 1 Processor -- start 2022-06-29T15:49:55.243 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.228+0000 7f9f3e241700 1 -- start start 2022-06-29T15:49:55.243 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.232+0000 7f9f3e241700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9f380630d0 0x7f9f380b2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:55.243 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.232+0000 7f9f3e241700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9f380afca0 0x7f9f380b0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:55.243 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.232+0000 7f9f3e241700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f3805f5c0 con 0x7f9f38061d00 2022-06-29T15:49:55.244 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.232+0000 7f9f3e241700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f380aeac0 con 0x7f9f380afca0 2022-06-29T15:49:55.244 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.232+0000 7f9f3e241700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3805f7f0 con 0x7f9f380630d0 2022-06-29T15:49:55.244 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.232+0000 7f9f3da40700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9f380afca0 0x7f9f380b0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:55.244 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.232+0000 7f9f3ca3e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9f380630d0 0x7f9f380b2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:55.245 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.232+0000 7f9f3da40700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9f380afca0 0x7f9f380b0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51594/0 (socket says 172.21.15.47:51594) 2022-06-29T15:49:55.245 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.232+0000 7f9f3da40700 1 -- 172.21.15.47:0/3738142630 learned_addr learned my addr 172.21.15.47:0/3738142630 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:55.245 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.232+0000 7f9f3da40700 1 -- 172.21.15.47:0/3738142630 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9f380630d0 msgr2=0x7f9f380b2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:55.245 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.232+0000 7f9f3da40700 1 --2- 172.21.15.47:0/3738142630 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9f380630d0 0x7f9f380b2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:55.246 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.232+0000 7f9f3da40700 1 -- 172.21.15.47:0/3738142630 >> v1:172.21.15.64:6789/0 conn(0x7f9f38061d00 legacy=0x7f9f38143a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:55.246 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.232+0000 7f9f3da40700 1 -- 172.21.15.47:0/3738142630 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f381435c0 con 0x7f9f380afca0 2022-06-29T15:49:55.247 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.232+0000 7f9f3ca3e700 1 --2- 172.21.15.47:0/3738142630 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9f380630d0 0x7f9f380b2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:49:55.247 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.232+0000 7f9f3da40700 1 --2- 172.21.15.47:0/3738142630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f380afca0 0x7f9f380b0130 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 rx=0x7f9f300032f0 tx=0x7f9f30009290).ready entity=mon.0 client_cookie=b1bc59bb425fa634 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:55.247 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.232+0000 7f9f27fff700 1 -- 172.21.15.47:0/3738142630 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f3000b8a0 con 0x7f9f380afca0 2022-06-29T15:49:55.247 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.232+0000 7f9f27fff700 1 -- 172.21.15.47:0/3738142630 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f3000ba00 con 0x7f9f380afca0 2022-06-29T15:49:55.248 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.232+0000 7f9f27fff700 1 -- 172.21.15.47:0/3738142630 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f3000bb70 con 0x7f9f380afca0 2022-06-29T15:49:55.248 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.232+0000 7f9f3e241700 1 -- 172.21.15.47:0/3738142630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f380afca0 msgr2=0x7f9f380b0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:55.248 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.232+0000 7f9f3e241700 1 --2- 172.21.15.47:0/3738142630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f380afca0 0x7f9f380b0130 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 rx=0x7f9f300032f0 tx=0x7f9f30009290).stop 2022-06-29T15:49:55.248 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.232+0000 7f9f3e241700 1 -- 172.21.15.47:0/3738142630 shutdown_connections 2022-06-29T15:49:55.248 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.232+0000 7f9f3e241700 1 --2- 172.21.15.47:0/3738142630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f380afca0 0x7f9f380b0130 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:55.249 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.232+0000 7f9f3e241700 1 --2- 172.21.15.47:0/3738142630 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9f380630d0 0x7f9f380b2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:55.249 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.232+0000 7f9f3e241700 1 -- 172.21.15.47:0/3738142630 >> 172.21.15.47:0/3738142630 conn(0x7f9f38063670 msgr2=0x7f9f38063a80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:55.249 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.232+0000 7f9f3e241700 1 -- 172.21.15.47:0/3738142630 shutdown_connections 2022-06-29T15:49:55.249 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.232+0000 7f9f3e241700 1 -- 172.21.15.47:0/3738142630 wait complete. 2022-06-29T15:49:55.253 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.236+0000 7f9f3e241700 1 Processor -- start 2022-06-29T15:49:55.253 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.236+0000 7f9f3e241700 1 -- start start 2022-06-29T15:49:55.253 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.236+0000 7f9f3e241700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f38061d00 0x7f9f3813f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:55.254 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.236+0000 7f9f3e241700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9f380630d0 0x7f9f380aa3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:55.254 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.236+0000 7f9f3e241700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f380afca0 0x7f9f380657d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:55.254 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.236+0000 7f9f3e241700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f380661b0 con 0x7f9f38061d00 2022-06-29T15:49:55.255 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.236+0000 7f9f3e241700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f380663e0 con 0x7f9f380afca0 2022-06-29T15:49:55.255 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.236+0000 7f9f3e241700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f380ad1b0 con 0x7f9f380630d0 2022-06-29T15:49:55.255 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.236+0000 7f9f3d23f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f38061d00 0x7f9f3813f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:55.255 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.236+0000 7f9f3d23f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f38061d00 0x7f9f3813f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51596/0 (socket says 172.21.15.47:51596) 2022-06-29T15:49:55.255 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.236+0000 7f9f3d23f700 1 -- 172.21.15.47:0/38781516 learned_addr learned my addr 172.21.15.47:0/38781516 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:55.256 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.236+0000 7f9f3da40700 1 --2- 172.21.15.47:0/38781516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f380afca0 0x7f9f380657d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:55.256 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.236+0000 7f9f3ca3e700 1 --2- 172.21.15.47:0/38781516 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9f380630d0 0x7f9f380aa3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:55.256 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.236+0000 7f9f3d23f700 1 -- 172.21.15.47:0/38781516 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9f380630d0 msgr2=0x7f9f380aa3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:55.256 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.236+0000 7f9f3d23f700 1 --2- 172.21.15.47:0/38781516 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9f380630d0 0x7f9f380aa3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:55.257 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.236+0000 7f9f3d23f700 1 -- 172.21.15.47:0/38781516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f380afca0 msgr2=0x7f9f380657d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:55.257 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.236+0000 7f9f3d23f700 1 --2- 172.21.15.47:0/38781516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f380afca0 0x7f9f380657d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:55.257 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.236+0000 7f9f3d23f700 1 -- 172.21.15.47:0/38781516 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f2800b180 con 0x7f9f38061d00 2022-06-29T15:49:55.257 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.236+0000 7f9f3da40700 1 --2- 172.21.15.47:0/38781516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f380afca0 0x7f9f380657d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:49:55.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.236+0000 7f9f3d23f700 1 --2- 172.21.15.47:0/38781516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f38061d00 0x7f9f3813f020 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 rx=0x7f9f380b6c10 tx=0x7f9f34007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:55.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.236+0000 7f9f267fc700 1 -- 172.21.15.47:0/38781516 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f3400ce30 con 0x7f9f38061d00 2022-06-29T15:49:55.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.236+0000 7f9f267fc700 1 -- 172.21.15.47:0/38781516 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f3400f470 con 0x7f9f38061d00 2022-06-29T15:49:55.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.236+0000 7f9f267fc700 1 -- 172.21.15.47:0/38781516 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f3400f7d0 con 0x7f9f38061d00 2022-06-29T15:49:55.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.236+0000 7f9f3e241700 1 -- 172.21.15.47:0/38781516 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f30009530 con 0x7f9f38061d00 2022-06-29T15:49:55.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.236+0000 7f9f3e241700 1 -- 172.21.15.47:0/38781516 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f38067180 con 0x7f9f38061d00 2022-06-29T15:49:55.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.240+0000 7f9f3e241700 1 -- 172.21.15.47:0/38781516 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f380573d0 con 0x7f9f38061d00 2022-06-29T15:49:55.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.240+0000 7f9f267fc700 1 -- 172.21.15.47:0/38781516 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9f34019030 con 0x7f9f38061d00 2022-06-29T15:49:55.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.240+0000 7f9f267fc700 1 --2- 172.21.15.47:0/38781516 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9f180539e0 0x7f9f18055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:55.260 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.240+0000 7f9f267fc700 1 -- 172.21.15.47:0/38781516 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9f3406b5a0 con 0x7f9f38061d00 2022-06-29T15:49:55.260 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.240+0000 7f9f3ca3e700 1 --2- 172.21.15.47:0/38781516 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9f180539e0 0x7f9f18055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:55.260 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.240+0000 7f9f3ca3e700 1 --2- 172.21.15.47:0/38781516 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9f180539e0 0x7f9f18055ea0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7f9f2800b000 tx=0x7f9f28008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:55.261 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.248+0000 7f9f267fc700 1 -- 172.21.15.47:0/38781516 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9f340421f0 con 0x7f9f38061d00 2022-06-29T15:49:55.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.408+0000 7f9f3e241700 1 -- 172.21.15.47:0/38781516 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f380665a0 con 0x7f9f38061d00 2022-06-29T15:49:55.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.408+0000 7f9f267fc700 1 -- 172.21.15.47:0/38781516 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+2493 (secure 0 0 0) 0x7f9f34045810 con 0x7f9f38061d00 2022-06-29T15:49:55.418 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:49:55.419 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":45,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6969,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3642031880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3642031880},{"type":"v1","addr":"172.21.15.47:6835","nonce":3642031880}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":43},{"gid":6984,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/891572891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":891572891},{"type":"v1","addr":"172.21.15.47:6837","nonce":891572891}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":43}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:49:50.788599+0000","modified":"2022-06-29T15:49:51.791379+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":56102},"failed":[],"damaged":[],"stopped":[],"info":{"gid_56102":{"gid":56102,"name":"b","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6833/3819760998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3819760998},{"type":"v1","addr":"172.21.15.64:6833","nonce":3819760998}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2022-06-29T15:49:55.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.412+0000 7f9f3e241700 1 -- 172.21.15.47:0/38781516 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9f180539e0 msgr2=0x7f9f18055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:55.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.412+0000 7f9f3e241700 1 --2- 172.21.15.47:0/38781516 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9f180539e0 0x7f9f18055ea0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7f9f2800b000 tx=0x7f9f28008040).stop 2022-06-29T15:49:55.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.412+0000 7f9f3e241700 1 -- 172.21.15.47:0/38781516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f38061d00 msgr2=0x7f9f3813f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:55.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.412+0000 7f9f3e241700 1 --2- 172.21.15.47:0/38781516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f38061d00 0x7f9f3813f020 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 rx=0x7f9f380b6c10 tx=0x7f9f34007c50).stop 2022-06-29T15:49:55.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.412+0000 7f9f3e241700 1 -- 172.21.15.47:0/38781516 shutdown_connections 2022-06-29T15:49:55.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.412+0000 7f9f3e241700 1 --2- 172.21.15.47:0/38781516 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9f180539e0 0x7f9f18055ea0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:55.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.412+0000 7f9f3e241700 1 --2- 172.21.15.47:0/38781516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f380afca0 0x7f9f380657d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:55.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.412+0000 7f9f3e241700 1 --2- 172.21.15.47:0/38781516 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9f380630d0 0x7f9f380aa3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:55.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.412+0000 7f9f3e241700 1 --2- 172.21.15.47:0/38781516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f38061d00 0x7f9f3813f020 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:55.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.412+0000 7f9f3e241700 1 -- 172.21.15.47:0/38781516 >> 172.21.15.47:0/38781516 conn(0x7f9f38063670 msgr2=0x7f9f380b00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:55.424 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.412+0000 7f9f3e241700 1 -- 172.21.15.47:0/38781516 shutdown_connections 2022-06-29T15:49:55.424 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:55.412+0000 7f9f3e241700 1 -- 172.21.15.47:0/38781516 wait complete. 2022-06-29T15:49:55.425 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 45 2022-06-29T15:49:55.733 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 45, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-29T15:49:50.788599+0000', 'modified': '2022-06-29T15:49:51.791379+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 56102}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_56102': {'gid': 56102, 'name': 'b', 'rank': 0, 'incarnation': 44, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.64:6833/3819760998', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6832', 'nonce': 3819760998}, {'type': 'v1', 'addr': '172.21.15.64:6833', 'nonce': 3819760998}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-29T15:49:55.734 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-29T15:49:55.735 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-06-29T15:49:55.855 INFO:teuthology.orchestra.run.smithi064.stdout:{"cluster_fsid":"9a68894a-53ce-400e-838b-14eab91b834c","whoami":0,"id":56102,"want_state":"up:active","state":"up:active","rank_uptime":5.0487956389999997,"mdsmap_epoch":45,"osdmap_epoch":101,"osdmap_epoch_barrier":100,"uptime":10.061840987} 2022-06-29T15:49:55.855 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9a68894a-53ce-400e-838b-14eab91b834c", "id": 56102, "mdsmap_epoch": 45, "osdmap_epoch": 101, "osdmap_epoch_barrier": 100, "rank_uptime": 5.048795639, "state": "up:active", "uptime": 10.061840987, "want_state": "up:active", "whoami": 0 } 2022-06-29T15:49:55.856 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-29T15:49:55.856 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-06-29T15:49:55.856 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi099.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-06-29T15:49:55.857 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:49:55.858 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:49:55.866 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2022-06-29T15:49:55.885 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:49:55.885 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:49:55.939 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:49:55.940 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:49:55.941 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:49:55.942 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2022-06-29T15:49:55.948 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-29T15:49:55.948 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:49:55.951 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2022-06-29T15:49:55.967 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:49:55.968 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:49:56.024 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:49:56.025 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:49:56.026 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:49:56.026 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2022-06-29T15:49:56.028 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:2022-06-29T15:49:56.020+0000 7fbd2c174080 -1 init, newargv = 0x55e8b3d0d8a0 newargc=9 2022-06-29T15:49:56.028 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[13092]: starting ceph client 2022-06-29T15:49:56.038 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[13092]: starting fuse 2022-06-29T15:49:57.135 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2022-06-29T15:49:57.155 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:49:57.155 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:49:57.213 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:49:57.214 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:49:57.215 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:49:57.216 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2022-06-29T15:49:57.221 INFO:teuthology.orchestra.run.smithi099.stdout:52 2022-06-29T15:49:57.221 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-29T15:49:57.222 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:49:57.222 DEBUG:teuthology.orchestra.run.smithi099:> sudo python3 -c ' 2022-06-29T15:49:57.223 DEBUG:teuthology.orchestra.run.smithi099:> import glob 2022-06-29T15:49:57.223 DEBUG:teuthology.orchestra.run.smithi099:> import re 2022-06-29T15:49:57.224 DEBUG:teuthology.orchestra.run.smithi099:> import os 2022-06-29T15:49:57.224 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2022-06-29T15:49:57.225 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:49:57.225 DEBUG:teuthology.orchestra.run.smithi099:> def find_socket(client_name): 2022-06-29T15:49:57.225 DEBUG:teuthology.orchestra.run.smithi099:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-06-29T15:49:57.226 DEBUG:teuthology.orchestra.run.smithi099:> files = glob.glob(asok_path) 2022-06-29T15:49:57.227 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:49:57.227 DEBUG:teuthology.orchestra.run.smithi099:> # Given a non-glob path, it better be there 2022-06-29T15:49:57.228 DEBUG:teuthology.orchestra.run.smithi099:> if "*" not in asok_path: 2022-06-29T15:49:57.228 DEBUG:teuthology.orchestra.run.smithi099:> assert(len(files) == 1) 2022-06-29T15:49:57.229 DEBUG:teuthology.orchestra.run.smithi099:> return files[0] 2022-06-29T15:49:57.229 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:49:57.230 DEBUG:teuthology.orchestra.run.smithi099:> for f in files: 2022-06-29T15:49:57.230 DEBUG:teuthology.orchestra.run.smithi099:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-29T15:49:57.230 DEBUG:teuthology.orchestra.run.smithi099:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-29T15:49:57.231 DEBUG:teuthology.orchestra.run.smithi099:> return f 2022-06-29T15:49:57.231 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-29T15:49:57.232 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:49:57.232 DEBUG:teuthology.orchestra.run.smithi099:> print(find_socket("client.0")) 2022-06-29T15:49:57.232 DEBUG:teuthology.orchestra.run.smithi099:> ' 2022-06-29T15:49:57.277 INFO:teuthology.orchestra.run.smithi099.stdout:/var/run/ceph/ceph-client.0.13092.asok 2022-06-29T15:49:57.282 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.13092.asok 2022-06-29T15:49:57.283 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:49:57.283 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.13092.asok status 2022-06-29T15:49:57.379 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2022-06-29T15:49:57.379 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": { 2022-06-29T15:49:57.379 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_sha1": "f25ffac72e5a9e63860ff7deb6767a11f0559a16", 2022-06-29T15:49:57.380 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 15.2.16-304-gf25ffac7 (f25ffac72e5a9e63860ff7deb6767a11f0559a16) octopus (stable)", 2022-06-29T15:49:57.380 INFO:teuthology.orchestra.run.smithi099.stdout: "entity_id": "0", 2022-06-29T15:49:57.380 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": "smithi099", 2022-06-29T15:49:57.380 INFO:teuthology.orchestra.run.smithi099.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-06-29T15:49:57.381 INFO:teuthology.orchestra.run.smithi099.stdout: "pid": "13092", 2022-06-29T15:49:57.381 INFO:teuthology.orchestra.run.smithi099.stdout: "root": "/" 2022-06-29T15:49:57.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:49:57.381 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_count": 0, 2022-06-29T15:49:57.382 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_pinned_count": 0, 2022-06-29T15:49:57.382 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 7137, 2022-06-29T15:49:57.382 INFO:teuthology.orchestra.run.smithi099.stdout: "inst": { 2022-06-29T15:49:57.382 INFO:teuthology.orchestra.run.smithi099.stdout: "name": { 2022-06-29T15:49:57.383 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "client", 2022-06-29T15:49:57.383 INFO:teuthology.orchestra.run.smithi099.stdout: "num": 7137 2022-06-29T15:49:57.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:49:57.384 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2022-06-29T15:49:57.385 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2022-06-29T15:49:57.385 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:0", 2022-06-29T15:49:57.386 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1155596892 2022-06-29T15:49:57.386 INFO:teuthology.orchestra.run.smithi099.stdout: } 2022-06-29T15:49:57.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:49:57.387 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2022-06-29T15:49:57.387 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2022-06-29T15:49:57.388 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:0", 2022-06-29T15:49:57.388 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1155596892 2022-06-29T15:49:57.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:49:57.389 INFO:teuthology.orchestra.run.smithi099.stdout: "inst_str": "client.7137 172.21.15.99:0/1155596892", 2022-06-29T15:49:57.389 INFO:teuthology.orchestra.run.smithi099.stdout: "addr_str": "172.21.15.99:0/1155596892", 2022-06-29T15:49:57.390 INFO:teuthology.orchestra.run.smithi099.stdout: "inode_count": 1, 2022-06-29T15:49:57.390 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_epoch": 45, 2022-06-29T15:49:57.391 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch": 101, 2022-06-29T15:49:57.391 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch_barrier": 0, 2022-06-29T15:49:57.392 INFO:teuthology.orchestra.run.smithi099.stdout: "blacklisted": false, 2022-06-29T15:49:57.392 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs" 2022-06-29T15:49:57.392 INFO:teuthology.orchestra.run.smithi099.stdout:} 2022-06-29T15:49:57.394 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:49:57.395 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:49:57.395 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:49:57.403 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2022-06-29T15:49:57.405 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-29T15:49:57.405 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:49:57.406 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:49:57.426 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:49:57.541 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.532+0000 7ff332956700 1 Processor -- start 2022-06-29T15:49:57.542 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.532+0000 7ff332956700 1 -- start start 2022-06-29T15:49:57.545 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.532+0000 7ff332956700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff32c0b5ac0 0x7ff32c1401c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:57.546 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.532+0000 7ff332956700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff32c0b02d0 0x7ff32c1406b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:57.546 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.532+0000 7ff332956700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff32c0afdc0 0x7ff32c0ae0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:57.547 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.532+0000 7ff332956700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff32c0a9090 con 0x7ff32c0b5ac0 2022-06-29T15:49:57.547 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.532+0000 7ff332956700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff32c143d20 con 0x7ff32c0b02d0 2022-06-29T15:49:57.547 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.532+0000 7ff332956700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c140ba0 con 0x7ff32c0afdc0 2022-06-29T15:49:57.548 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.532+0000 7ff331954700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff32c0b5ac0 0x7ff32c1401c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:57.548 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.532+0000 7ff331153700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff32c0b02d0 0x7ff32c1406b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:57.548 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.532+0000 7ff331954700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff32c0b5ac0 0x7ff32c1401c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51606/0 (socket says 172.21.15.47:51606) 2022-06-29T15:49:57.548 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.532+0000 7ff331954700 1 -- 172.21.15.47:0/514952302 learned_addr learned my addr 172.21.15.47:0/514952302 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:57.549 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.532+0000 7ff332155700 1 --2- 172.21.15.47:0/514952302 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff32c0afdc0 0x7ff32c0ae0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:57.549 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.532+0000 7ff331954700 1 -- 172.21.15.47:0/514952302 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff32c0afdc0 msgr2=0x7ff32c0ae0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:57.549 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.532+0000 7ff331954700 1 --2- 172.21.15.47:0/514952302 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff32c0afdc0 0x7ff32c0ae0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:57.550 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.532+0000 7ff331954700 1 -- 172.21.15.47:0/514952302 >> v2:172.21.15.64:3300/0 conn(0x7ff32c0b02d0 msgr2=0x7ff32c1406b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:57.550 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.532+0000 7ff331954700 1 --2- 172.21.15.47:0/514952302 >> v2:172.21.15.64:3300/0 conn(0x7ff32c0b02d0 0x7ff32c1406b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:57.550 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.532+0000 7ff331954700 1 -- 172.21.15.47:0/514952302 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff32c0a8df0 con 0x7ff32c0b5ac0 2022-06-29T15:49:57.550 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.536+0000 7ff331954700 1 --2- 172.21.15.47:0/514952302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff32c0b5ac0 0x7ff32c1401c0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 rx=0x7ff31c002f70 tx=0x7ff31c006e50).ready entity=mon.0 client_cookie=796dfbe061b07532 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:57.551 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.536+0000 7ff330952700 1 -- 172.21.15.47:0/514952302 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff31c00d570 con 0x7ff32c0b5ac0 2022-06-29T15:49:57.551 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.536+0000 7ff330952700 1 -- 172.21.15.47:0/514952302 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff31c000e90 con 0x7ff32c0b5ac0 2022-06-29T15:49:57.551 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.536+0000 7ff330952700 1 -- 172.21.15.47:0/514952302 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff31c00d900 con 0x7ff32c0b5ac0 2022-06-29T15:49:57.551 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.536+0000 7ff332956700 1 -- 172.21.15.47:0/514952302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff32c0b5ac0 msgr2=0x7ff32c1401c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:57.552 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.536+0000 7ff332956700 1 --2- 172.21.15.47:0/514952302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff32c0b5ac0 0x7ff32c1401c0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 rx=0x7ff31c002f70 tx=0x7ff31c006e50).stop 2022-06-29T15:49:57.552 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.536+0000 7ff332956700 1 -- 172.21.15.47:0/514952302 shutdown_connections 2022-06-29T15:49:57.552 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.536+0000 7ff332956700 1 --2- 172.21.15.47:0/514952302 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff32c0afdc0 0x7ff32c0ae0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:57.553 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.536+0000 7ff332956700 1 --2- 172.21.15.47:0/514952302 >> v2:172.21.15.64:3300/0 conn(0x7ff32c0b02d0 0x7ff32c1406b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:57.553 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.536+0000 7ff332956700 1 --2- 172.21.15.47:0/514952302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff32c0b5ac0 0x7ff32c1401c0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:57.553 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.536+0000 7ff332956700 1 -- 172.21.15.47:0/514952302 >> 172.21.15.47:0/514952302 conn(0x7ff32c062560 msgr2=0x7ff32c062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:57.553 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.536+0000 7ff332956700 1 -- 172.21.15.47:0/514952302 shutdown_connections 2022-06-29T15:49:57.554 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.536+0000 7ff332956700 1 -- 172.21.15.47:0/514952302 wait complete. 2022-06-29T15:49:57.557 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.536+0000 7ff332956700 1 Processor -- start 2022-06-29T15:49:57.558 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.540+0000 7ff332956700 1 -- start start 2022-06-29T15:49:57.558 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.540+0000 7ff332956700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff32c0afdc0 0x7ff32c152d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:57.558 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.540+0000 7ff332956700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff32c0b02d0 0x7ff32c142b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:57.558 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.540+0000 7ff332956700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff32c0b5ac0 0x7ff32c13a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:57.559 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.540+0000 7ff332956700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c0a9330 con 0x7ff32c0b02d0 2022-06-29T15:49:57.559 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.540+0000 7ff332956700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c143340 con 0x7ff32c0b5ac0 2022-06-29T15:49:57.559 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.540+0000 7ff332956700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c143060 con 0x7ff32c0afdc0 2022-06-29T15:49:57.559 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.540+0000 7ff331153700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff32c0b02d0 0x7ff32c142b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:57.560 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.540+0000 7ff331153700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff32c0b02d0 0x7ff32c142b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51612/0 (socket says 172.21.15.47:51612) 2022-06-29T15:49:57.560 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.540+0000 7ff331153700 1 -- 172.21.15.47:0/525950680 learned_addr learned my addr 172.21.15.47:0/525950680 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:57.560 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.540+0000 7ff331954700 1 --2- 172.21.15.47:0/525950680 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff32c0afdc0 0x7ff32c152d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:57.560 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.540+0000 7ff332155700 1 --2- 172.21.15.47:0/525950680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff32c0b5ac0 0x7ff32c13a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:57.560 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.540+0000 7ff331153700 1 -- 172.21.15.47:0/525950680 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff32c0afdc0 msgr2=0x7ff32c152d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:57.561 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.540+0000 7ff331153700 1 --2- 172.21.15.47:0/525950680 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff32c0afdc0 0x7ff32c152d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:57.561 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.540+0000 7ff331153700 1 -- 172.21.15.47:0/525950680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff32c0b5ac0 msgr2=0x7ff32c13a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:57.561 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.540+0000 7ff331153700 1 --2- 172.21.15.47:0/525950680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff32c0b5ac0 0x7ff32c13a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:57.561 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.540+0000 7ff331153700 1 -- 172.21.15.47:0/525950680 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff31c00b1e0 con 0x7ff32c0b02d0 2022-06-29T15:49:57.562 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.540+0000 7ff331153700 1 --2- 172.21.15.47:0/525950680 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff32c0b02d0 0x7ff32c142b70 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 rx=0x7ff32800c000 tx=0x7ff328006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:57.562 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.540+0000 7ff31affd700 1 -- 172.21.15.47:0/525950680 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff328010780 con 0x7ff32c0b02d0 2022-06-29T15:49:57.562 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.540+0000 7ff31affd700 1 -- 172.21.15.47:0/525950680 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3280108e0 con 0x7ff32c0b02d0 2022-06-29T15:49:57.562 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.540+0000 7ff332956700 1 -- 172.21.15.47:0/525950680 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff32c141ad0 con 0x7ff32c0b02d0 2022-06-29T15:49:57.563 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.540+0000 7ff31affd700 1 -- 172.21.15.47:0/525950680 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff328010c40 con 0x7ff32c0b02d0 2022-06-29T15:49:57.563 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.540+0000 7ff332956700 1 -- 172.21.15.47:0/525950680 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff32c1416f0 con 0x7ff32c0b02d0 2022-06-29T15:49:57.563 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.544+0000 7ff332956700 1 -- 172.21.15.47:0/525950680 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff32c0573d0 con 0x7ff32c0b02d0 2022-06-29T15:49:57.563 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.544+0000 7ff31affd700 1 -- 172.21.15.47:0/525950680 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff32801c460 con 0x7ff32c0b02d0 2022-06-29T15:49:57.563 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.544+0000 7ff31affd700 1 --2- 172.21.15.47:0/525950680 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff30c053950 0x7ff30c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:57.564 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.544+0000 7ff31affd700 1 -- 172.21.15.47:0/525950680 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff32806c590 con 0x7ff32c0b02d0 2022-06-29T15:49:57.564 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.544+0000 7ff331954700 1 --2- 172.21.15.47:0/525950680 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff30c053950 0x7ff30c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:57.564 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.544+0000 7ff331954700 1 --2- 172.21.15.47:0/525950680 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff30c053950 0x7ff30c055e10 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7ff31c01a000 tx=0x7ff31c014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:57.565 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.552+0000 7ff31affd700 1 -- 172.21.15.47:0/525950680 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff328043160 con 0x7ff32c0b02d0 2022-06-29T15:49:57.690 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.680+0000 7ff332956700 1 -- 172.21.15.47:0/525950680 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff32c069e20 con 0x7ff32c0b02d0 2022-06-29T15:49:57.692 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.684+0000 7ff31affd700 1 -- 172.21.15.47:0/525950680 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+16277 (secure 0 0 0) 0x7ff328046780 con 0x7ff32c0b02d0 2022-06-29T15:49:57.693 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:49:57.693 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":101,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:49:51.792208+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:49:48.525959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2022-06-29T15:49:49.805331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:49:57.697 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.684+0000 7ff332956700 1 -- 172.21.15.47:0/525950680 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff30c053950 msgr2=0x7ff30c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:57.697 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.684+0000 7ff332956700 1 --2- 172.21.15.47:0/525950680 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff30c053950 0x7ff30c055e10 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7ff31c01a000 tx=0x7ff31c014000).stop 2022-06-29T15:49:57.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.684+0000 7ff332956700 1 -- 172.21.15.47:0/525950680 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff32c0b02d0 msgr2=0x7ff32c142b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:57.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.684+0000 7ff332956700 1 --2- 172.21.15.47:0/525950680 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff32c0b02d0 0x7ff32c142b70 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 rx=0x7ff32800c000 tx=0x7ff328006df0).stop 2022-06-29T15:49:57.699 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.684+0000 7ff332956700 1 -- 172.21.15.47:0/525950680 shutdown_connections 2022-06-29T15:49:57.699 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.684+0000 7ff332956700 1 --2- 172.21.15.47:0/525950680 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff30c053950 0x7ff30c055e10 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:57.700 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.684+0000 7ff332956700 1 --2- 172.21.15.47:0/525950680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff32c0b5ac0 0x7ff32c13a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:57.700 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.684+0000 7ff332956700 1 --2- 172.21.15.47:0/525950680 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff32c0b02d0 0x7ff32c142b70 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:57.701 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.684+0000 7ff332956700 1 --2- 172.21.15.47:0/525950680 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff32c0afdc0 0x7ff32c152d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:57.701 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.684+0000 7ff332956700 1 -- 172.21.15.47:0/525950680 >> 172.21.15.47:0/525950680 conn(0x7ff32c062560 msgr2=0x7ff32c13e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:57.702 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.684+0000 7ff332956700 1 -- 172.21.15.47:0/525950680 shutdown_connections 2022-06-29T15:49:57.702 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.684+0000 7ff332956700 1 -- 172.21.15.47:0/525950680 wait complete. 2022-06-29T15:49:57.706 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-29T15:42:16.971431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-29T15:42:25.290884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-29T15:49:48.525959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2022-06-29T15:49:49.805331+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-29T15:49:57.708 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-29T15:49:57.819 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.808+0000 7f8a7402e700 1 Processor -- start 2022-06-29T15:49:57.820 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.808+0000 7f8a7402e700 1 -- start start 2022-06-29T15:49:57.822 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.812+0000 7f8a7402e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8a6c061220 0x7f8a6c140070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:57.823 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.812+0000 7f8a7402e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8a6c0b4320 0x7f8a6c143230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:57.823 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.812+0000 7f8a7402e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a6c061600 con 0x7f8a6c146720 2022-06-29T15:49:57.823 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.812+0000 7f8a7402e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a6c146bc0 con 0x7f8a6c0b4320 2022-06-29T15:49:57.823 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.812+0000 7f8a7402e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6c146e60 con 0x7f8a6c061220 2022-06-29T15:49:57.824 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.812+0000 7f8a715c9700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f8a6c146720 0x7f8a6c140560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37176/0 (socket says 172.21.15.47:37176) 2022-06-29T15:49:57.824 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.812+0000 7f8a715c9700 1 -- 172.21.15.47:0/917707320 learned_addr learned my addr 172.21.15.47:0/917707320 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:49:57.824 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.812+0000 7f8a71dca700 1 --2- 172.21.15.47:0/917707320 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8a6c061220 0x7f8a6c140070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:57.824 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.812+0000 7f8a725cb700 1 --2- 172.21.15.47:0/917707320 >> v2:172.21.15.64:3300/0 conn(0x7f8a6c0b4320 0x7f8a6c143230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:57.825 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.812+0000 7f8a70dc8700 1 -- 172.21.15.47:0/917707320 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1345625273 0 0) 0x7f8a6c061600 con 0x7f8a6c146720 2022-06-29T15:49:57.825 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.812+0000 7f8a70dc8700 1 -- 172.21.15.47:0/917707320 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a54003260 con 0x7f8a6c146720 2022-06-29T15:49:57.825 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.812+0000 7f8a725cb700 1 -- 172.21.15.47:0/917707320 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8a6c061220 msgr2=0x7f8a6c140070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:57.825 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.812+0000 7f8a725cb700 1 --2- 172.21.15.47:0/917707320 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8a6c061220 0x7f8a6c140070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:57.826 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.812+0000 7f8a725cb700 1 -- 172.21.15.47:0/917707320 >> v1:172.21.15.47:6789/0 conn(0x7f8a6c146720 legacy=0x7f8a6c140560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:57.826 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.812+0000 7f8a725cb700 1 -- 172.21.15.47:0/917707320 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a6c147100 con 0x7f8a6c0b4320 2022-06-29T15:49:57.826 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.812+0000 7f8a725cb700 1 --2- 172.21.15.47:0/917707320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a6c0b4320 0x7f8a6c143230 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7f8a640032f0 tx=0x7f8a640091d0).ready entity=mon.1 client_cookie=c1a5384d05950461 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:57.826 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.812+0000 7f8a70dc8700 1 -- 172.21.15.47:0/917707320 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a6400b600 con 0x7f8a6c0b4320 2022-06-29T15:49:57.826 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.812+0000 7f8a70dc8700 1 -- 172.21.15.47:0/917707320 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a6400b760 con 0x7f8a6c0b4320 2022-06-29T15:49:57.827 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.812+0000 7f8a70dc8700 1 -- 172.21.15.47:0/917707320 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a6400b8d0 con 0x7f8a6c0b4320 2022-06-29T15:49:57.831 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.812+0000 7f8a7402e700 1 -- 172.21.15.47:0/917707320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a6c0b4320 msgr2=0x7f8a6c143230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:57.832 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.812+0000 7f8a7402e700 1 --2- 172.21.15.47:0/917707320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a6c0b4320 0x7f8a6c143230 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7f8a640032f0 tx=0x7f8a640091d0).stop 2022-06-29T15:49:57.832 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.812+0000 7f8a7402e700 1 -- 172.21.15.47:0/917707320 shutdown_connections 2022-06-29T15:49:57.833 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.812+0000 7f8a7402e700 1 --2- 172.21.15.47:0/917707320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a6c0b4320 0x7f8a6c143230 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:57.833 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.812+0000 7f8a7402e700 1 --2- 172.21.15.47:0/917707320 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8a6c061220 0x7f8a6c140070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:57.834 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.812+0000 7f8a7402e700 1 -- 172.21.15.47:0/917707320 >> 172.21.15.47:0/917707320 conn(0x7f8a6c05c6b0 msgr2=0x7f8a6c05f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:57.834 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.812+0000 7f8a7402e700 1 -- 172.21.15.47:0/917707320 shutdown_connections 2022-06-29T15:49:57.834 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.812+0000 7f8a7402e700 1 -- 172.21.15.47:0/917707320 wait complete. 2022-06-29T15:49:57.835 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.816+0000 7f8a7402e700 1 Processor -- start 2022-06-29T15:49:57.835 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.816+0000 7f8a7402e700 1 -- start start 2022-06-29T15:49:57.836 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.816+0000 7f8a7402e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a6c061220 0x7f8a6c0b6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:57.836 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.816+0000 7f8a7402e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a6c0b4320 0x7f8a6c152a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:57.836 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.816+0000 7f8a7402e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8a6c146720 0x7f8a6c155610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:57.837 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.816+0000 7f8a7402e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6c0b31f0 con 0x7f8a6c0b4320 2022-06-29T15:49:57.837 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.816+0000 7f8a7402e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6c152f50 con 0x7f8a6c061220 2022-06-29T15:49:57.837 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.816+0000 7f8a7402e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6c115db0 con 0x7f8a6c146720 2022-06-29T15:49:57.837 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.816+0000 7f8a715c9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a6c0b4320 0x7f8a6c152a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:57.838 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.816+0000 7f8a715c9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a6c0b4320 0x7f8a6c152a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51626/0 (socket says 172.21.15.47:51626) 2022-06-29T15:49:57.838 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.816+0000 7f8a715c9700 1 -- 172.21.15.47:0/4075240889 learned_addr learned my addr 172.21.15.47:0/4075240889 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:57.838 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.816+0000 7f8a725cb700 1 --2- 172.21.15.47:0/4075240889 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8a6c146720 0x7f8a6c155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:57.838 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.816+0000 7f8a71dca700 1 --2- 172.21.15.47:0/4075240889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a6c061220 0x7f8a6c0b6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:57.839 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.816+0000 7f8a715c9700 1 -- 172.21.15.47:0/4075240889 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8a6c146720 msgr2=0x7f8a6c155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:57.839 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.816+0000 7f8a715c9700 1 --2- 172.21.15.47:0/4075240889 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8a6c146720 0x7f8a6c155610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:57.839 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.816+0000 7f8a715c9700 1 -- 172.21.15.47:0/4075240889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a6c061220 msgr2=0x7f8a6c0b6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:57.839 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.816+0000 7f8a715c9700 1 --2- 172.21.15.47:0/4075240889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a6c061220 0x7f8a6c0b6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:57.839 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.816+0000 7f8a715c9700 1 -- 172.21.15.47:0/4075240889 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a64009470 con 0x7f8a6c0b4320 2022-06-29T15:49:57.840 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.816+0000 7f8a715c9700 1 --2- 172.21.15.47:0/4075240889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a6c0b4320 0x7f8a6c152a60 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 rx=0x7f8a6c1530e0 tx=0x7f8a60007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:57.840 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.816+0000 7f8a5effd700 1 -- 172.21.15.47:0/4075240889 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a600070d0 con 0x7f8a6c0b4320 2022-06-29T15:49:57.840 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.816+0000 7f8a5effd700 1 -- 172.21.15.47:0/4075240889 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a60007230 con 0x7f8a6c0b4320 2022-06-29T15:49:57.840 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.816+0000 7f8a5effd700 1 -- 172.21.15.47:0/4075240889 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a60005610 con 0x7f8a6c0b4320 2022-06-29T15:49:57.841 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.816+0000 7f8a7402e700 1 -- 172.21.15.47:0/4075240889 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a6c0b3420 con 0x7f8a6c0b4320 2022-06-29T15:49:57.841 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.816+0000 7f8a7402e700 1 -- 172.21.15.47:0/4075240889 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a6c152320 con 0x7f8a6c0b4320 2022-06-29T15:49:57.841 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.820+0000 7f8a5effd700 1 -- 172.21.15.47:0/4075240889 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8a60010070 con 0x7f8a6c0b4320 2022-06-29T15:49:57.841 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.820+0000 7f8a7402e700 1 -- 172.21.15.47:0/4075240889 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a6c152760 con 0x7f8a6c0b4320 2022-06-29T15:49:57.842 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.820+0000 7f8a5effd700 1 --2- 172.21.15.47:0/4075240889 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8a4c053980 0x7f8a4c055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:57.842 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.820+0000 7f8a5effd700 1 -- 172.21.15.47:0/4075240889 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f8a6006b690 con 0x7f8a6c0b4320 2022-06-29T15:49:57.842 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.820+0000 7f8a71dca700 1 --2- 172.21.15.47:0/4075240889 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8a4c053980 0x7f8a4c055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:57.842 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.820+0000 7f8a71dca700 1 --2- 172.21.15.47:0/4075240889 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8a4c053980 0x7f8a4c055e40 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7f8a6800b000 tx=0x7f8a68006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:57.843 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.828+0000 7f8a5effd700 1 -- 172.21.15.47:0/4075240889 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8a60042820 con 0x7f8a6c0b4320 2022-06-29T15:49:57.964 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.956+0000 7f8a7402e700 1 -- 172.21.15.47:0/4075240889 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f8a6c0573d0 con 0x7f8a6c0b4320 2022-06-29T15:49:57.965 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.956+0000 7f8a5effd700 1 -- 172.21.15.47:0/4075240889 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v101) v1 ==== 110+0+10 (secure 0 0 0) 0x7f8a60018030 con 0x7f8a6c0b4320 2022-06-29T15:49:57.965 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2022-06-29T15:49:57.967 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.956+0000 7f8a7402e700 1 -- 172.21.15.47:0/4075240889 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8a4c053980 msgr2=0x7f8a4c055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:57.967 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.956+0000 7f8a7402e700 1 --2- 172.21.15.47:0/4075240889 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8a4c053980 0x7f8a4c055e40 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7f8a6800b000 tx=0x7f8a68006df0).stop 2022-06-29T15:49:57.967 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.956+0000 7f8a7402e700 1 -- 172.21.15.47:0/4075240889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a6c0b4320 msgr2=0x7f8a6c152a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:57.968 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.956+0000 7f8a7402e700 1 --2- 172.21.15.47:0/4075240889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a6c0b4320 0x7f8a6c152a60 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 rx=0x7f8a6c1530e0 tx=0x7f8a60007ad0).stop 2022-06-29T15:49:57.968 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.956+0000 7f8a7402e700 1 -- 172.21.15.47:0/4075240889 shutdown_connections 2022-06-29T15:49:57.968 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.956+0000 7f8a7402e700 1 --2- 172.21.15.47:0/4075240889 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8a6c146720 0x7f8a6c155610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:57.968 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.956+0000 7f8a7402e700 1 --2- 172.21.15.47:0/4075240889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a6c0b4320 0x7f8a6c152a60 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:57.969 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.956+0000 7f8a7402e700 1 --2- 172.21.15.47:0/4075240889 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8a4c053980 0x7f8a4c055e40 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:57.969 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.956+0000 7f8a7402e700 1 --2- 172.21.15.47:0/4075240889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a6c061220 0x7f8a6c0b6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:57.969 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.956+0000 7f8a7402e700 1 -- 172.21.15.47:0/4075240889 >> 172.21.15.47:0/4075240889 conn(0x7f8a6c05c6b0 msgr2=0x7f8a6c145700 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:57.969 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.956+0000 7f8a7402e700 1 -- 172.21.15.47:0/4075240889 shutdown_connections 2022-06-29T15:49:57.969 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:57.956+0000 7f8a7402e700 1 -- 172.21.15.47:0/4075240889 wait complete. 2022-06-29T15:49:57.978 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-29T15:49:58.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.080+0000 7fe7563b1700 1 Processor -- start 2022-06-29T15:49:58.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.080+0000 7fe7563b1700 1 -- start start 2022-06-29T15:49:58.090 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.080+0000 7fe7563b1700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe7500b4300 0x7fe7500612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:58.090 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.080+0000 7fe7563b1700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe750067610 con 0x7fe7501471a0 2022-06-29T15:49:58.091 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.080+0000 7fe7563b1700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7501467c0 con 0x7fe750060ee0 2022-06-29T15:49:58.091 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.080+0000 7fe7563b1700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe750146a60 con 0x7fe7500b4300 2022-06-29T15:49:58.091 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.080+0000 7fe74f7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe7500b4300 0x7fe7500612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:58.091 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.080+0000 7fe75494e700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe7501471a0 0x7fe75013cdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37188/0 (socket says 172.21.15.47:37188) 2022-06-29T15:49:58.092 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.080+0000 7fe75494e700 1 -- 172.21.15.47:0/618204410 learned_addr learned my addr 172.21.15.47:0/618204410 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:49:58.092 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.080+0000 7fe74f7fe700 1 -- 172.21.15.47:0/618204410 >> v1:172.21.15.64:6789/0 conn(0x7fe750060ee0 legacy=0x7fe7501430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:58.092 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.080+0000 7fe74f7fe700 1 -- 172.21.15.47:0/618204410 >> v1:172.21.15.47:6789/0 conn(0x7fe7501471a0 legacy=0x7fe75013cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:58.092 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.080+0000 7fe74effd700 1 -- 172.21.15.47:0/618204410 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 569365524 0 0) 0x7fe750067610 con 0x7fe7501471a0 2022-06-29T15:49:58.092 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.080+0000 7fe74f7fe700 1 -- 172.21.15.47:0/618204410 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe750146d00 con 0x7fe7500b4300 2022-06-29T15:49:58.093 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.080+0000 7fe74f7fe700 1 --2- 172.21.15.47:0/618204410 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7500b4300 0x7fe7500612c0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 rx=0x7fe7500adb80 tx=0x7fe740006df0).ready entity=mon.0 client_cookie=cf192f25406af49d server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:58.093 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.080+0000 7fe74effd700 1 -- 172.21.15.47:0/618204410 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe74000d5c0 con 0x7fe7500b4300 2022-06-29T15:49:58.093 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.080+0000 7fe74effd700 1 -- 172.21.15.47:0/618204410 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe740000e90 con 0x7fe7500b4300 2022-06-29T15:49:58.093 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.080+0000 7fe74effd700 1 -- 172.21.15.47:0/618204410 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe74000d910 con 0x7fe7500b4300 2022-06-29T15:49:58.096 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.080+0000 7fe7563b1700 1 -- 172.21.15.47:0/618204410 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7500b4300 msgr2=0x7fe7500612c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:58.096 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.080+0000 7fe7563b1700 1 --2- 172.21.15.47:0/618204410 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7500b4300 0x7fe7500612c0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 rx=0x7fe7500adb80 tx=0x7fe740006df0).stop 2022-06-29T15:49:58.097 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.080+0000 7fe7563b1700 1 -- 172.21.15.47:0/618204410 shutdown_connections 2022-06-29T15:49:58.097 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.080+0000 7fe7563b1700 1 --2- 172.21.15.47:0/618204410 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7500b4300 0x7fe7500612c0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:58.097 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.080+0000 7fe7563b1700 1 -- 172.21.15.47:0/618204410 >> 172.21.15.47:0/618204410 conn(0x7fe75005c510 msgr2=0x7fe75005f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:58.097 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.084+0000 7fe7563b1700 1 -- 172.21.15.47:0/618204410 shutdown_connections 2022-06-29T15:49:58.098 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.084+0000 7fe7563b1700 1 -- 172.21.15.47:0/618204410 wait complete. 2022-06-29T15:49:58.098 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.084+0000 7fe7563b1700 1 Processor -- start 2022-06-29T15:49:58.098 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.084+0000 7fe7563b1700 1 -- start start 2022-06-29T15:49:58.098 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.084+0000 7fe7563b1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe750060ee0 0x7fe750114e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:58.099 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.084+0000 7fe7563b1700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe7500b4300 0x7fe750141f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:58.099 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.084+0000 7fe7563b1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7501471a0 0x7fe75014fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:58.099 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.084+0000 7fe7563b1700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7500b3910 con 0x7fe7501471a0 2022-06-29T15:49:58.099 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.084+0000 7fe7563b1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7500b3bb0 con 0x7fe750060ee0 2022-06-29T15:49:58.100 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.084+0000 7fe7563b1700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe750114750 con 0x7fe7500b4300 2022-06-29T15:49:58.100 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.084+0000 7fe75494e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7501471a0 0x7fe75014fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:58.100 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.084+0000 7fe75494e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7501471a0 0x7fe75014fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51640/0 (socket says 172.21.15.47:51640) 2022-06-29T15:49:58.100 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.084+0000 7fe75494e700 1 -- 172.21.15.47:0/1909606406 learned_addr learned my addr 172.21.15.47:0/1909606406 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:58.100 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.084+0000 7fe74ffff700 1 --2- 172.21.15.47:0/1909606406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe750060ee0 0x7fe750114e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:58.101 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.084+0000 7fe74f7fe700 1 --2- 172.21.15.47:0/1909606406 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe7500b4300 0x7fe750141f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:58.101 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.088+0000 7fe75494e700 1 -- 172.21.15.47:0/1909606406 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe7500b4300 msgr2=0x7fe750141f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:58.101 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.088+0000 7fe75494e700 1 --2- 172.21.15.47:0/1909606406 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe7500b4300 0x7fe750141f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:58.101 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.088+0000 7fe75494e700 1 -- 172.21.15.47:0/1909606406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe750060ee0 msgr2=0x7fe750114e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:58.102 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.088+0000 7fe75494e700 1 --2- 172.21.15.47:0/1909606406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe750060ee0 0x7fe750114e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:58.102 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.088+0000 7fe75494e700 1 -- 172.21.15.47:0/1909606406 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe74000b180 con 0x7fe7501471a0 2022-06-29T15:49:58.102 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.088+0000 7fe74ffff700 1 --2- 172.21.15.47:0/1909606406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe750060ee0 0x7fe750114e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:49:58.102 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.088+0000 7fe75494e700 1 --2- 172.21.15.47:0/1909606406 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7501471a0 0x7fe75014fa90 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 rx=0x7fe748000f40 tx=0x7fe748001360).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:58.102 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.088+0000 7fe74d7fa700 1 -- 172.21.15.47:0/1909606406 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe748005ba0 con 0x7fe7501471a0 2022-06-29T15:49:58.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.088+0000 7fe74d7fa700 1 -- 172.21.15.47:0/1909606406 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe748005d00 con 0x7fe7501471a0 2022-06-29T15:49:58.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.088+0000 7fe7563b1700 1 -- 172.21.15.47:0/1909606406 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7501149f0 con 0x7fe7501471a0 2022-06-29T15:49:58.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.088+0000 7fe74d7fa700 1 -- 172.21.15.47:0/1909606406 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe7480072c0 con 0x7fe7501471a0 2022-06-29T15:49:58.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.088+0000 7fe7563b1700 1 -- 172.21.15.47:0/1909606406 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe750142400 con 0x7fe7501471a0 2022-06-29T15:49:58.104 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.088+0000 7fe7563b1700 1 -- 172.21.15.47:0/1909606406 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe750114b80 con 0x7fe7501471a0 2022-06-29T15:49:58.105 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.088+0000 7fe74d7fa700 1 -- 172.21.15.47:0/1909606406 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe748010070 con 0x7fe7501471a0 2022-06-29T15:49:58.105 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.088+0000 7fe74d7fa700 1 --2- 172.21.15.47:0/1909606406 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe734053b00 0x7fe734055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:58.105 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.088+0000 7fe74d7fa700 1 -- 172.21.15.47:0/1909606406 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe74806baa0 con 0x7fe7501471a0 2022-06-29T15:49:58.105 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.088+0000 7fe74ffff700 1 --2- 172.21.15.47:0/1909606406 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe734053b00 0x7fe734055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:58.106 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.092+0000 7fe74ffff700 1 --2- 172.21.15.47:0/1909606406 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe734053b00 0x7fe734055fc0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7fe73800d000 tx=0x7fe738009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:58.106 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.096+0000 7fe74d7fa700 1 -- 172.21.15.47:0/1909606406 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe748042750 con 0x7fe7501471a0 2022-06-29T15:49:58.234 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.224+0000 7fe7563b1700 1 -- 172.21.15.47:0/1909606406 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe7500573d0 con 0x7fe7501471a0 2022-06-29T15:49:58.235 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.224+0000 7fe74d7fa700 1 -- 172.21.15.47:0/1909606406 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v101) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe748045d70 con 0x7fe7501471a0 2022-06-29T15:49:58.235 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:49:58.237 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.228+0000 7fe7563b1700 1 -- 172.21.15.47:0/1909606406 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe734053b00 msgr2=0x7fe734055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:58.237 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.228+0000 7fe7563b1700 1 --2- 172.21.15.47:0/1909606406 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe734053b00 0x7fe734055fc0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7fe73800d000 tx=0x7fe738009040).stop 2022-06-29T15:49:58.237 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.228+0000 7fe7563b1700 1 -- 172.21.15.47:0/1909606406 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7501471a0 msgr2=0x7fe75014fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:58.238 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.228+0000 7fe7563b1700 1 --2- 172.21.15.47:0/1909606406 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7501471a0 0x7fe75014fa90 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 rx=0x7fe748000f40 tx=0x7fe748001360).stop 2022-06-29T15:49:58.238 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.228+0000 7fe7563b1700 1 -- 172.21.15.47:0/1909606406 shutdown_connections 2022-06-29T15:49:58.238 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.228+0000 7fe7563b1700 1 --2- 172.21.15.47:0/1909606406 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7501471a0 0x7fe75014fa90 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:58.238 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.228+0000 7fe7563b1700 1 --2- 172.21.15.47:0/1909606406 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe7500b4300 0x7fe750141f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:58.239 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.228+0000 7fe7563b1700 1 --2- 172.21.15.47:0/1909606406 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe734053b00 0x7fe734055fc0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:58.239 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.228+0000 7fe7563b1700 1 --2- 172.21.15.47:0/1909606406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe750060ee0 0x7fe750114e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:58.239 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.228+0000 7fe7563b1700 1 -- 172.21.15.47:0/1909606406 >> 172.21.15.47:0/1909606406 conn(0x7fe75005c510 msgr2=0x7fe75013f710 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:58.239 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.228+0000 7fe7563b1700 1 -- 172.21.15.47:0/1909606406 shutdown_connections 2022-06-29T15:49:58.240 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.228+0000 7fe7563b1700 1 -- 172.21.15.47:0/1909606406 wait complete. 2022-06-29T15:49:58.248 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-29T15:49:58.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.348+0000 7f7b77fff700 1 Processor -- start 2022-06-29T15:49:58.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.348+0000 7f7b77fff700 1 -- start start 2022-06-29T15:49:58.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.348+0000 7f7b77fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7b780b58d0 0x7f7b780b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:58.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.348+0000 7f7b77fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7b780b1610 0x7f7b781475f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:58.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.348+0000 7f7b77fff700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b78060790 con 0x7f7b7814b4c0 2022-06-29T15:49:58.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.348+0000 7f7b77fff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b780a99e0 con 0x7f7b780b1610 2022-06-29T15:49:58.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.348+0000 7f7b77fff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7814ad80 con 0x7f7b780b58d0 2022-06-29T15:49:58.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.348+0000 7f7b777fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7b7814b4c0 0x7f7b780aa130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37202/0 (socket says 172.21.15.47:37202) 2022-06-29T15:49:58.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.348+0000 7f7b777fe700 1 -- 172.21.15.47:0/987357825 learned_addr learned my addr 172.21.15.47:0/987357825 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:49:58.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.348+0000 7f7b767fc700 1 --2- 172.21.15.47:0/987357825 >> v2:172.21.15.64:3300/0 conn(0x7f7b780b1610 0x7f7b781475f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:58.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.348+0000 7f7b76ffd700 1 --2- 172.21.15.47:0/987357825 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7b780b58d0 0x7f7b780b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:58.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.352+0000 7f7b75ffb700 1 -- 172.21.15.47:0/987357825 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1250111625 0 0) 0x7f7b78060790 con 0x7f7b7814b4c0 2022-06-29T15:49:58.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.352+0000 7f7b75ffb700 1 -- 172.21.15.47:0/987357825 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b5c003260 con 0x7f7b7814b4c0 2022-06-29T15:49:58.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.352+0000 7f7b767fc700 1 -- 172.21.15.47:0/987357825 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7b780b58d0 msgr2=0x7f7b780b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:58.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.352+0000 7f7b767fc700 1 --2- 172.21.15.47:0/987357825 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7b780b58d0 0x7f7b780b5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:58.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.352+0000 7f7b767fc700 1 -- 172.21.15.47:0/987357825 >> v1:172.21.15.47:6789/0 conn(0x7f7b7814b4c0 legacy=0x7f7b780aa130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:58.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.352+0000 7f7b767fc700 1 -- 172.21.15.47:0/987357825 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b7814b020 con 0x7f7b780b1610 2022-06-29T15:49:58.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.352+0000 7f7b76ffd700 1 --2- 172.21.15.47:0/987357825 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7b780b58d0 0x7f7b780b5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:49:58.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.352+0000 7f7b767fc700 1 --2- 172.21.15.47:0/987357825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b780b1610 0x7f7b781475f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f7b780623e0 tx=0x7f7b680091d0).ready entity=mon.1 client_cookie=6048b1fc081911da server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:58.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.352+0000 7f7b75ffb700 1 -- 172.21.15.47:0/987357825 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b6800b600 con 0x7f7b780b1610 2022-06-29T15:49:58.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.352+0000 7f7b75ffb700 1 -- 172.21.15.47:0/987357825 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b6800b760 con 0x7f7b780b1610 2022-06-29T15:49:58.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.352+0000 7f7b75ffb700 1 -- 172.21.15.47:0/987357825 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b6800b8d0 con 0x7f7b780b1610 2022-06-29T15:49:58.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.352+0000 7f7b77fff700 1 -- 172.21.15.47:0/987357825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b780b1610 msgr2=0x7f7b781475f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:58.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.352+0000 7f7b77fff700 1 --2- 172.21.15.47:0/987357825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b780b1610 0x7f7b781475f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f7b780623e0 tx=0x7f7b680091d0).stop 2022-06-29T15:49:58.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.352+0000 7f7b77fff700 1 -- 172.21.15.47:0/987357825 shutdown_connections 2022-06-29T15:49:58.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.352+0000 7f7b77fff700 1 --2- 172.21.15.47:0/987357825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b780b1610 0x7f7b781475f0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:58.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.352+0000 7f7b77fff700 1 --2- 172.21.15.47:0/987357825 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7b780b58d0 0x7f7b780b5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:58.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.352+0000 7f7b77fff700 1 -- 172.21.15.47:0/987357825 >> 172.21.15.47:0/987357825 conn(0x7f7b78062560 msgr2=0x7f7b78062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:58.370 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.352+0000 7f7b77fff700 1 -- 172.21.15.47:0/987357825 shutdown_connections 2022-06-29T15:49:58.370 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.352+0000 7f7b77fff700 1 -- 172.21.15.47:0/987357825 wait complete. 2022-06-29T15:49:58.370 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.352+0000 7f7b77fff700 1 Processor -- start 2022-06-29T15:49:58.370 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.352+0000 7f7b77fff700 1 -- start start 2022-06-29T15:49:58.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.356+0000 7f7b77fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b780b1610 0x7f7b780aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:58.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.356+0000 7f7b77fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b780b58d0 0x7f7b78066350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:58.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.356+0000 7f7b77fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7b7814b4c0 0x7f7b78066840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:58.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.356+0000 7f7b77fff700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7814d330 con 0x7f7b780b1610 2022-06-29T15:49:58.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.356+0000 7f7b77fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7814d840 con 0x7f7b780b58d0 2022-06-29T15:49:58.372 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.356+0000 7f7b77fff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7814e4e0 con 0x7f7b7814b4c0 2022-06-29T15:49:58.372 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.356+0000 7f7b76ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b780b1610 0x7f7b780aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:58.372 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.356+0000 7f7b76ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b780b1610 0x7f7b780aeaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51650/0 (socket says 172.21.15.47:51650) 2022-06-29T15:49:58.372 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.356+0000 7f7b76ffd700 1 -- 172.21.15.47:0/2559102543 learned_addr learned my addr 172.21.15.47:0/2559102543 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:58.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.356+0000 7f7b767fc700 1 --2- 172.21.15.47:0/2559102543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b780b58d0 0x7f7b78066350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:58.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.356+0000 7f7b777fe700 1 --2- 172.21.15.47:0/2559102543 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7b7814b4c0 0x7f7b78066840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:58.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.356+0000 7f7b76ffd700 1 -- 172.21.15.47:0/2559102543 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7b7814b4c0 msgr2=0x7f7b78066840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:58.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.356+0000 7f7b76ffd700 1 --2- 172.21.15.47:0/2559102543 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7b7814b4c0 0x7f7b78066840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:58.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.356+0000 7f7b76ffd700 1 -- 172.21.15.47:0/2559102543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b780b58d0 msgr2=0x7f7b78066350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:58.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.356+0000 7f7b76ffd700 1 --2- 172.21.15.47:0/2559102543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b780b58d0 0x7f7b78066350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:58.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.356+0000 7f7b76ffd700 1 -- 172.21.15.47:0/2559102543 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b6c00b180 con 0x7f7b780b1610 2022-06-29T15:49:58.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.356+0000 7f7b76ffd700 1 --2- 172.21.15.47:0/2559102543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b780b1610 0x7f7b780aeaf0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 rx=0x7f7b6c00c000 tx=0x7f7b6c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:58.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.356+0000 7f7b67fff700 1 -- 172.21.15.47:0/2559102543 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b6c00c890 con 0x7f7b780b1610 2022-06-29T15:49:58.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.356+0000 7f7b67fff700 1 -- 172.21.15.47:0/2559102543 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b6c00c9f0 con 0x7f7b780b1610 2022-06-29T15:49:58.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.356+0000 7f7b77fff700 1 -- 172.21.15.47:0/2559102543 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b68009470 con 0x7f7b780b1610 2022-06-29T15:49:58.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.356+0000 7f7b67fff700 1 -- 172.21.15.47:0/2559102543 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b6c00cd50 con 0x7f7b780b1610 2022-06-29T15:49:58.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.356+0000 7f7b77fff700 1 -- 172.21.15.47:0/2559102543 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b780af410 con 0x7f7b780b1610 2022-06-29T15:49:58.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.356+0000 7f7b77fff700 1 -- 172.21.15.47:0/2559102543 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b780573d0 con 0x7f7b780b1610 2022-06-29T15:49:58.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.356+0000 7f7b67fff700 1 -- 172.21.15.47:0/2559102543 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7b6c01c460 con 0x7f7b780b1610 2022-06-29T15:49:58.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.356+0000 7f7b67fff700 1 --2- 172.21.15.47:0/2559102543 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7b54053950 0x7f7b54055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:58.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.356+0000 7f7b767fc700 1 --2- 172.21.15.47:0/2559102543 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7b54053950 0x7f7b54055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:58.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.356+0000 7f7b67fff700 1 -- 172.21.15.47:0/2559102543 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f7b6c06c6e0 con 0x7f7b780b1610 2022-06-29T15:49:58.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.360+0000 7f7b767fc700 1 --2- 172.21.15.47:0/2559102543 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7b54053950 0x7f7b54055e10 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7f7b6801b000 tx=0x7f7b68016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:58.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.364+0000 7f7b67fff700 1 -- 172.21.15.47:0/2559102543 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7b6c0433c0 con 0x7f7b780b1610 2022-06-29T15:49:58.497 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.488+0000 7f7b77fff700 1 -- 172.21.15.47:0/2559102543 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7b78069e20 con 0x7f7b780b1610 2022-06-29T15:49:58.498 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.488+0000 7f7b67fff700 1 -- 172.21.15.47:0/2559102543 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v101) v1 ==== 104+0+10 (secure 0 0 0) 0x7f7b6c0469e0 con 0x7f7b780b1610 2022-06-29T15:49:58.499 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:49:58.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.492+0000 7f7b77fff700 1 -- 172.21.15.47:0/2559102543 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7b54053950 msgr2=0x7f7b54055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:58.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.492+0000 7f7b77fff700 1 --2- 172.21.15.47:0/2559102543 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7b54053950 0x7f7b54055e10 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7f7b6801b000 tx=0x7f7b68016000).stop 2022-06-29T15:49:58.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.492+0000 7f7b77fff700 1 -- 172.21.15.47:0/2559102543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b780b1610 msgr2=0x7f7b780aeaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:58.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.492+0000 7f7b77fff700 1 --2- 172.21.15.47:0/2559102543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b780b1610 0x7f7b780aeaf0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 rx=0x7f7b6c00c000 tx=0x7f7b6c006df0).stop 2022-06-29T15:49:58.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.492+0000 7f7b77fff700 1 -- 172.21.15.47:0/2559102543 shutdown_connections 2022-06-29T15:49:58.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.492+0000 7f7b77fff700 1 --2- 172.21.15.47:0/2559102543 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7b7814b4c0 0x7f7b78066840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:58.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.492+0000 7f7b77fff700 1 --2- 172.21.15.47:0/2559102543 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7b54053950 0x7f7b54055e10 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:58.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.492+0000 7f7b77fff700 1 --2- 172.21.15.47:0/2559102543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b780b58d0 0x7f7b78066350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:58.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.492+0000 7f7b77fff700 1 --2- 172.21.15.47:0/2559102543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b780b1610 0x7f7b780aeaf0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:58.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.492+0000 7f7b77fff700 1 -- 172.21.15.47:0/2559102543 >> 172.21.15.47:0/2559102543 conn(0x7f7b78062560 msgr2=0x7f7b780b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:58.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.492+0000 7f7b77fff700 1 -- 172.21.15.47:0/2559102543 shutdown_connections 2022-06-29T15:49:58.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.492+0000 7f7b77fff700 1 -- 172.21.15.47:0/2559102543 wait complete. 2022-06-29T15:49:58.512 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-29T15:49:58.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.612+0000 7f1012129700 1 Processor -- start 2022-06-29T15:49:58.622 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.612+0000 7f1012129700 1 -- start start 2022-06-29T15:49:58.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.612+0000 7f1012129700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f100c0b6b80 0x7f100c140f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:58.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.612+0000 7f1012129700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f100c063090 0x7f100c13e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:58.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.612+0000 7f1012129700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f100c145000 con 0x7f100c05f5c0 2022-06-29T15:49:58.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.612+0000 7f1012129700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f100c1452a0 con 0x7f100c0b6b80 2022-06-29T15:49:58.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.612+0000 7f1012129700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f100c0b2300 con 0x7f100c063090 2022-06-29T15:49:58.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.616+0000 7f1010926700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f100c063090 0x7f100c13e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:58.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.616+0000 7f1011127700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f100c0b6b80 0x7f100c140f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:58.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.616+0000 7f1011928700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f100c05f5c0 0x7f100c0b27a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:44780/0 (socket says 172.21.15.47:44780) 2022-06-29T15:49:58.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.616+0000 7f1011928700 1 -- 172.21.15.47:0/1917993964 learned_addr learned my addr 172.21.15.47:0/1917993964 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:49:58.627 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.616+0000 7f1010926700 1 -- 172.21.15.47:0/1917993964 >> v2:172.21.15.64:3300/0 conn(0x7f100c0b6b80 msgr2=0x7f100c140f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:49:58.627 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.616+0000 7f1010926700 1 --2- 172.21.15.47:0/1917993964 >> v2:172.21.15.64:3300/0 conn(0x7f100c0b6b80 0x7f100c140f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:58.627 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.616+0000 7f1010926700 1 -- 172.21.15.47:0/1917993964 >> v1:172.21.15.64:6789/0 conn(0x7f100c05f5c0 legacy=0x7f100c0b27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:58.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.616+0000 7f0ffbfff700 1 -- 172.21.15.47:0/1917993964 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2453045314 0 0) 0x7f100c145000 con 0x7f100c05f5c0 2022-06-29T15:49:58.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.616+0000 7f1010926700 1 -- 172.21.15.47:0/1917993964 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f100c05f200 con 0x7f100c063090 2022-06-29T15:49:58.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.616+0000 7f1010926700 1 --2- 172.21.15.47:0/1917993964 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f100c063090 0x7f100c13e2b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f0ffc002f70 tx=0x7f0ffc006df0).ready entity=mon.2 client_cookie=5bf4ae0dc9ccdef8 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:58.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.616+0000 7f0ffbfff700 1 -- 172.21.15.47:0/1917993964 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ffc00d570 con 0x7f100c063090 2022-06-29T15:49:58.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.616+0000 7f0ffbfff700 1 -- 172.21.15.47:0/1917993964 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ffc000e90 con 0x7f100c063090 2022-06-29T15:49:58.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.616+0000 7f0ffbfff700 1 -- 172.21.15.47:0/1917993964 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ffc00d8c0 con 0x7f100c063090 2022-06-29T15:49:58.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.616+0000 7f1012129700 1 -- 172.21.15.47:0/1917993964 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f100c063090 msgr2=0x7f100c13e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:58.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.616+0000 7f1012129700 1 --2- 172.21.15.47:0/1917993964 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f100c063090 0x7f100c13e2b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f0ffc002f70 tx=0x7f0ffc006df0).stop 2022-06-29T15:49:58.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.616+0000 7f1012129700 1 -- 172.21.15.47:0/1917993964 shutdown_connections 2022-06-29T15:49:58.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.616+0000 7f1012129700 1 --2- 172.21.15.47:0/1917993964 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f100c063090 0x7f100c13e2b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:58.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.616+0000 7f1012129700 1 --2- 172.21.15.47:0/1917993964 >> v2:172.21.15.64:3300/0 conn(0x7f100c0b6b80 0x7f100c140f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:58.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.616+0000 7f1012129700 1 -- 172.21.15.47:0/1917993964 >> 172.21.15.47:0/1917993964 conn(0x7f100c063670 msgr2=0x7f100c063a50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:58.631 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.616+0000 7f1012129700 1 -- 172.21.15.47:0/1917993964 shutdown_connections 2022-06-29T15:49:58.631 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.616+0000 7f1012129700 1 -- 172.21.15.47:0/1917993964 wait complete. 2022-06-29T15:49:58.633 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.620+0000 7f1012129700 1 Processor -- start 2022-06-29T15:49:58.634 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.620+0000 7f1012129700 1 -- start start 2022-06-29T15:49:58.634 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.620+0000 7f1012129700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f100c05f5c0 0x7f100c0acda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:58.634 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.620+0000 7f1012129700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f100c063090 0x7f100c150b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:58.635 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.620+0000 7f1012129700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f100c0b6b80 0x7f100c140bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:58.635 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.620+0000 7f1012129700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100c09ee20 con 0x7f100c063090 2022-06-29T15:49:58.635 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.620+0000 7f1012129700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100c146830 con 0x7f100c0b6b80 2022-06-29T15:49:58.635 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.620+0000 7f1012129700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f100c13ff70 con 0x7f100c05f5c0 2022-06-29T15:49:58.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.620+0000 7f1010926700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f100c063090 0x7f100c150b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:58.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.620+0000 7f1010926700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f100c063090 0x7f100c150b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51666/0 (socket says 172.21.15.47:51666) 2022-06-29T15:49:58.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.620+0000 7f1010926700 1 -- 172.21.15.47:0/3441808614 learned_addr learned my addr 172.21.15.47:0/3441808614 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:58.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.620+0000 7f1011127700 1 --2- 172.21.15.47:0/3441808614 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f100c05f5c0 0x7f100c0acda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:58.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.620+0000 7f1011928700 1 --2- 172.21.15.47:0/3441808614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f100c0b6b80 0x7f100c140bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:58.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.620+0000 7f1010926700 1 -- 172.21.15.47:0/3441808614 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f100c05f5c0 msgr2=0x7f100c0acda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:58.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.620+0000 7f1010926700 1 --2- 172.21.15.47:0/3441808614 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f100c05f5c0 0x7f100c0acda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:58.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.620+0000 7f1010926700 1 -- 172.21.15.47:0/3441808614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f100c0b6b80 msgr2=0x7f100c140bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:58.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.620+0000 7f1010926700 1 --2- 172.21.15.47:0/3441808614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f100c0b6b80 0x7f100c140bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:58.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.620+0000 7f1010926700 1 -- 172.21.15.47:0/3441808614 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ffc00b180 con 0x7f100c063090 2022-06-29T15:49:58.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.620+0000 7f1010926700 1 --2- 172.21.15.47:0/3441808614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f100c063090 0x7f100c150b70 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 rx=0x7f0ffc002f70 tx=0x7f0ffc014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:58.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.620+0000 7f0ffa7fc700 1 -- 172.21.15.47:0/3441808614 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ffc00dea0 con 0x7f100c063090 2022-06-29T15:49:58.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.624+0000 7f0ffa7fc700 1 -- 172.21.15.47:0/3441808614 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ffc013de0 con 0x7f100c063090 2022-06-29T15:49:58.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.624+0000 7f1012129700 1 -- 172.21.15.47:0/3441808614 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f100c140210 con 0x7f100c063090 2022-06-29T15:49:58.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.624+0000 7f0ffa7fc700 1 -- 172.21.15.47:0/3441808614 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ffc01e5f0 con 0x7f100c063090 2022-06-29T15:49:58.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.624+0000 7f1012129700 1 -- 172.21.15.47:0/3441808614 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f100c140750 con 0x7f100c063090 2022-06-29T15:49:58.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.624+0000 7f1012129700 1 -- 172.21.15.47:0/3441808614 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f100c13f560 con 0x7f100c063090 2022-06-29T15:49:58.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.624+0000 7f0ffa7fc700 1 -- 172.21.15.47:0/3441808614 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0ffc01c070 con 0x7f100c063090 2022-06-29T15:49:58.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.624+0000 7f0ffa7fc700 1 --2- 172.21.15.47:0/3441808614 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0fec0539e0 0x7f0fec055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:58.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.624+0000 7f0ffa7fc700 1 -- 172.21.15.47:0/3441808614 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f0ffc06ca00 con 0x7f100c063090 2022-06-29T15:49:58.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.624+0000 7f1011127700 1 --2- 172.21.15.47:0/3441808614 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0fec0539e0 0x7f0fec055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:58.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.628+0000 7f1011127700 1 --2- 172.21.15.47:0/3441808614 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0fec0539e0 0x7f0fec055ea0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7f100800b000 tx=0x7f1008006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:58.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.632+0000 7f0ffa7fc700 1 -- 172.21.15.47:0/3441808614 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f100c13f560 con 0x7f100c063090 2022-06-29T15:49:58.768 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.760+0000 7f1012129700 1 -- 172.21.15.47:0/3441808614 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f100c13f560 con 0x7f100c063090 2022-06-29T15:49:58.768 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.760+0000 7f0ffa7fc700 1 -- 172.21.15.47:0/3441808614 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v101) v1 ==== 100+0+10 (secure 0 0 0) 0x7f100c13f560 con 0x7f100c063090 2022-06-29T15:49:58.769 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:49:58.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.760+0000 7f1012129700 1 -- 172.21.15.47:0/3441808614 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0fec0539e0 msgr2=0x7f0fec055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:58.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.760+0000 7f1012129700 1 --2- 172.21.15.47:0/3441808614 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0fec0539e0 0x7f0fec055ea0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7f100800b000 tx=0x7f1008006df0).stop 2022-06-29T15:49:58.772 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.760+0000 7f1012129700 1 -- 172.21.15.47:0/3441808614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f100c063090 msgr2=0x7f100c150b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:58.772 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.760+0000 7f1012129700 1 --2- 172.21.15.47:0/3441808614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f100c063090 0x7f100c150b70 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 rx=0x7f0ffc002f70 tx=0x7f0ffc014000).stop 2022-06-29T15:49:58.772 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.760+0000 7f1012129700 1 -- 172.21.15.47:0/3441808614 shutdown_connections 2022-06-29T15:49:58.772 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.760+0000 7f1012129700 1 --2- 172.21.15.47:0/3441808614 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0fec0539e0 0x7f0fec055ea0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:58.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.760+0000 7f1012129700 1 --2- 172.21.15.47:0/3441808614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f100c0b6b80 0x7f100c140bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:58.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.760+0000 7f1012129700 1 --2- 172.21.15.47:0/3441808614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f100c063090 0x7f100c150b70 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:58.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.760+0000 7f1012129700 1 --2- 172.21.15.47:0/3441808614 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f100c05f5c0 0x7f100c0acda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:58.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.760+0000 7f1012129700 1 -- 172.21.15.47:0/3441808614 >> 172.21.15.47:0/3441808614 conn(0x7f100c063670 msgr2=0x7f100c063a50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:58.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.760+0000 7f1012129700 1 -- 172.21.15.47:0/3441808614 shutdown_connections 2022-06-29T15:49:58.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.760+0000 7f1012129700 1 -- 172.21.15.47:0/3441808614 wait complete. 2022-06-29T15:49:58.782 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-29T15:49:58.783 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:49:58.886 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.876+0000 7fb51f59e700 1 Processor -- start 2022-06-29T15:49:58.886 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.876+0000 7fb51f59e700 1 -- start start 2022-06-29T15:49:58.888 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.876+0000 7fb51f59e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb5200b2ee0 0x7fb5200b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:58.888 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.876+0000 7fb51f59e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5201400e0 con 0x7fb5200b0e50 2022-06-29T15:49:58.888 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.876+0000 7fb51f59e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb520143570 con 0x7fb5200ad970 2022-06-29T15:49:58.889 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.876+0000 7fb51f59e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb520140ef0 con 0x7fb5200b2ee0 2022-06-29T15:49:58.889 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.876+0000 7fb51e59c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb5200b2ee0 0x7fb5200b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:58.889 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.876+0000 7fb51ed9d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb5200b0e50 0x7fb52013d290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37230/0 (socket says 172.21.15.47:37230) 2022-06-29T15:49:58.889 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.876+0000 7fb51ed9d700 1 -- 172.21.15.47:0/3714768106 learned_addr learned my addr 172.21.15.47:0/3714768106 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:49:58.890 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.880+0000 7fb51d59a700 1 -- 172.21.15.47:0/3714768106 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4181953851 0 0) 0x7fb5201400e0 con 0x7fb5200b0e50 2022-06-29T15:49:58.890 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.880+0000 7fb51d59a700 1 -- 172.21.15.47:0/3714768106 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb504003260 con 0x7fb5200b0e50 2022-06-29T15:49:58.890 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.880+0000 7fb51e59c700 1 -- 172.21.15.47:0/3714768106 >> v1:172.21.15.64:6789/0 conn(0x7fb5200ad970 legacy=0x7fb5201407e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:58.890 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.880+0000 7fb51e59c700 1 -- 172.21.15.47:0/3714768106 >> v1:172.21.15.47:6789/0 conn(0x7fb5200b0e50 legacy=0x7fb52013d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:58.890 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.880+0000 7fb51e59c700 1 -- 172.21.15.47:0/3714768106 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb520140340 con 0x7fb5200b2ee0 2022-06-29T15:49:58.891 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.880+0000 7fb51e59c700 1 --2- 172.21.15.47:0/3714768106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5200b2ee0 0x7fb5200b32c0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 rx=0x7fb510002f70 tx=0x7fb510006df0).ready entity=mon.0 client_cookie=3fa7c7dec1c7a80e server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:58.891 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.880+0000 7fb51d59a700 1 -- 172.21.15.47:0/3714768106 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb51000d610 con 0x7fb5200b2ee0 2022-06-29T15:49:58.891 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.880+0000 7fb51d59a700 1 -- 172.21.15.47:0/3714768106 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb510000e90 con 0x7fb5200b2ee0 2022-06-29T15:49:58.891 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.880+0000 7fb51d59a700 1 -- 172.21.15.47:0/3714768106 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb51000d960 con 0x7fb5200b2ee0 2022-06-29T15:49:58.895 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.880+0000 7fb51f59e700 1 -- 172.21.15.47:0/3714768106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5200b2ee0 msgr2=0x7fb5200b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:58.895 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.880+0000 7fb51f59e700 1 --2- 172.21.15.47:0/3714768106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5200b2ee0 0x7fb5200b32c0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 rx=0x7fb510002f70 tx=0x7fb510006df0).stop 2022-06-29T15:49:58.895 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.880+0000 7fb51f59e700 1 -- 172.21.15.47:0/3714768106 shutdown_connections 2022-06-29T15:49:58.895 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.880+0000 7fb51f59e700 1 --2- 172.21.15.47:0/3714768106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5200b2ee0 0x7fb5200b32c0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:58.896 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.880+0000 7fb51f59e700 1 -- 172.21.15.47:0/3714768106 >> 172.21.15.47:0/3714768106 conn(0x7fb520063670 msgr2=0x7fb520063a80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:58.896 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.880+0000 7fb51f59e700 1 -- 172.21.15.47:0/3714768106 shutdown_connections 2022-06-29T15:49:58.896 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.880+0000 7fb51f59e700 1 -- 172.21.15.47:0/3714768106 wait complete. 2022-06-29T15:49:58.896 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.880+0000 7fb51f59e700 1 Processor -- start 2022-06-29T15:49:58.897 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.880+0000 7fb51f59e700 1 -- start start 2022-06-29T15:49:58.897 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.884+0000 7fb51f59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5200ad970 0x7fb5201421c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:58.897 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.884+0000 7fb51f59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5200b0e50 0x7fb5200aabd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:58.897 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.884+0000 7fb51f59e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb5200b2ee0 0x7fb5200aba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:58.897 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.884+0000 7fb51f59e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5200ab630 con 0x7fb5200b0e50 2022-06-29T15:49:58.898 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.884+0000 7fb51f59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5200696e0 con 0x7fb5200ad970 2022-06-29T15:49:58.898 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.884+0000 7fb51f59e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5200ab280 con 0x7fb5200b2ee0 2022-06-29T15:49:58.898 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.884+0000 7fb51dd9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5200b0e50 0x7fb5200aabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:58.898 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.884+0000 7fb51dd9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5200b0e50 0x7fb5200aabd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51680/0 (socket says 172.21.15.47:51680) 2022-06-29T15:49:58.899 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.884+0000 7fb51dd9b700 1 -- 172.21.15.47:0/649194624 learned_addr learned my addr 172.21.15.47:0/649194624 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:49:58.899 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.884+0000 7fb51e59c700 1 --2- 172.21.15.47:0/649194624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5200ad970 0x7fb5201421c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:58.899 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.884+0000 7fb51ed9d700 1 --2- 172.21.15.47:0/649194624 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb5200b2ee0 0x7fb5200aba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:58.899 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.884+0000 7fb51dd9b700 1 -- 172.21.15.47:0/649194624 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb5200b2ee0 msgr2=0x7fb5200aba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:58.900 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.884+0000 7fb51dd9b700 1 --2- 172.21.15.47:0/649194624 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb5200b2ee0 0x7fb5200aba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:58.900 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.884+0000 7fb51dd9b700 1 -- 172.21.15.47:0/649194624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5200ad970 msgr2=0x7fb5201421c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:58.900 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.884+0000 7fb51dd9b700 1 --2- 172.21.15.47:0/649194624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5200ad970 0x7fb5201421c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:58.900 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.884+0000 7fb51dd9b700 1 -- 172.21.15.47:0/649194624 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb51000b180 con 0x7fb5200b0e50 2022-06-29T15:49:58.900 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.884+0000 7fb51e59c700 1 --2- 172.21.15.47:0/649194624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5200ad970 0x7fb5201421c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:49:58.901 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.884+0000 7fb51dd9b700 1 --2- 172.21.15.47:0/649194624 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5200b0e50 0x7fb5200aabd0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 rx=0x7fb520062b30 tx=0x7fb514007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:58.901 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.884+0000 7fb50f7fe700 1 -- 172.21.15.47:0/649194624 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb514007e70 con 0x7fb5200b0e50 2022-06-29T15:49:58.901 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.884+0000 7fb50f7fe700 1 -- 172.21.15.47:0/649194624 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb514007030 con 0x7fb5200b0e50 2022-06-29T15:49:58.901 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.884+0000 7fb51f59e700 1 -- 172.21.15.47:0/649194624 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb520141250 con 0x7fb5200b0e50 2022-06-29T15:49:58.902 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.884+0000 7fb50f7fe700 1 -- 172.21.15.47:0/649194624 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb514007390 con 0x7fb5200b0e50 2022-06-29T15:49:58.902 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.884+0000 7fb51f59e700 1 -- 172.21.15.47:0/649194624 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5200b6040 con 0x7fb5200b0e50 2022-06-29T15:49:58.902 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.884+0000 7fb51f59e700 1 -- 172.21.15.47:0/649194624 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5200ab7a0 con 0x7fb5200b0e50 2022-06-29T15:49:58.902 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.884+0000 7fb50f7fe700 1 -- 172.21.15.47:0/649194624 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb514010030 con 0x7fb5200b0e50 2022-06-29T15:49:58.903 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.888+0000 7fb50f7fe700 1 --2- 172.21.15.47:0/649194624 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb4fc053a10 0x7fb4fc055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:49:58.904 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.888+0000 7fb50f7fe700 1 -- 172.21.15.47:0/649194624 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb51406bca0 con 0x7fb5200b0e50 2022-06-29T15:49:58.904 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.888+0000 7fb51e59c700 1 --2- 172.21.15.47:0/649194624 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb4fc053a10 0x7fb4fc055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:49:58.904 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.888+0000 7fb51e59c700 1 --2- 172.21.15.47:0/649194624 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb4fc053a10 0x7fb4fc055ed0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7fb510018000 tx=0x7fb510014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:49:58.904 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:58.892+0000 7fb50f7fe700 1 -- 172.21.15.47:0/649194624 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb514042960 con 0x7fb5200b0e50 2022-06-29T15:49:59.069 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:59.060+0000 7fb51f59e700 1 -- 172.21.15.47:0/649194624 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb5200573d0 con 0x7fb5200b0e50 2022-06-29T15:49:59.070 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:59.060+0000 7fb50f7fe700 1 -- 172.21.15.47:0/649194624 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+2493 (secure 0 0 0) 0x7fb514045f80 con 0x7fb5200b0e50 2022-06-29T15:49:59.070 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:49:59.070 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":45,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6969,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3642031880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3642031880},{"type":"v1","addr":"172.21.15.47:6835","nonce":3642031880}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":43},{"gid":6984,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/891572891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":891572891},{"type":"v1","addr":"172.21.15.47:6837","nonce":891572891}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":43}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:49:50.788599+0000","modified":"2022-06-29T15:49:51.791379+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":56102},"failed":[],"damaged":[],"stopped":[],"info":{"gid_56102":{"gid":56102,"name":"b","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6833/3819760998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3819760998},{"type":"v1","addr":"172.21.15.64:6833","nonce":3819760998}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2022-06-29T15:49:59.072 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:59.064+0000 7fb51f59e700 1 -- 172.21.15.47:0/649194624 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb4fc053a10 msgr2=0x7fb4fc055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:59.072 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:59.064+0000 7fb51f59e700 1 --2- 172.21.15.47:0/649194624 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb4fc053a10 0x7fb4fc055ed0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7fb510018000 tx=0x7fb510014000).stop 2022-06-29T15:49:59.072 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:59.064+0000 7fb51f59e700 1 -- 172.21.15.47:0/649194624 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5200b0e50 msgr2=0x7fb5200aabd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:49:59.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:59.064+0000 7fb51f59e700 1 --2- 172.21.15.47:0/649194624 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5200b0e50 0x7fb5200aabd0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 rx=0x7fb520062b30 tx=0x7fb514007c00).stop 2022-06-29T15:49:59.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:59.064+0000 7fb51f59e700 1 -- 172.21.15.47:0/649194624 shutdown_connections 2022-06-29T15:49:59.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:59.064+0000 7fb51f59e700 1 --2- 172.21.15.47:0/649194624 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb5200b2ee0 0x7fb5200aba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:59.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:59.064+0000 7fb51f59e700 1 --2- 172.21.15.47:0/649194624 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5200b0e50 0x7fb5200aabd0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:59.074 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:59.064+0000 7fb51f59e700 1 --2- 172.21.15.47:0/649194624 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb4fc053a10 0x7fb4fc055ed0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:59.074 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:59.064+0000 7fb51f59e700 1 --2- 172.21.15.47:0/649194624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5200ad970 0x7fb5201421c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:49:59.074 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:59.064+0000 7fb51f59e700 1 -- 172.21.15.47:0/649194624 >> 172.21.15.47:0/649194624 conn(0x7fb520063670 msgr2=0x7fb52013d340 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:49:59.074 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:59.064+0000 7fb51f59e700 1 -- 172.21.15.47:0/649194624 shutdown_connections 2022-06-29T15:49:59.075 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:49:59.064+0000 7fb51f59e700 1 -- 172.21.15.47:0/649194624 wait complete. 2022-06-29T15:49:59.075 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 45 2022-06-29T15:49:59.390 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 45, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-29T15:49:50.788599+0000', 'modified': '2022-06-29T15:49:51.791379+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 56102}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_56102': {'gid': 56102, 'name': 'b', 'rank': 0, 'incarnation': 44, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.64:6833/3819760998', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6832', 'nonce': 3819760998}, {'type': 'v1', 'addr': '172.21.15.64:6833', 'nonce': 3819760998}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-29T15:49:59.390 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-29T15:49:59.391 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-06-29T15:49:59.509 INFO:teuthology.orchestra.run.smithi064.stdout:{"cluster_fsid":"9a68894a-53ce-400e-838b-14eab91b834c","whoami":0,"id":56102,"want_state":"up:active","state":"up:active","rank_uptime":8.7035428780000004,"mdsmap_epoch":45,"osdmap_epoch":101,"osdmap_epoch_barrier":100,"uptime":13.716588899} 2022-06-29T15:49:59.510 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9a68894a-53ce-400e-838b-14eab91b834c", "id": 56102, "mdsmap_epoch": 45, "osdmap_epoch": 101, "osdmap_epoch_barrier": 100, "rank_uptime": 8.703542878, "state": "up:active", "uptime": 13.716588899, "want_state": "up:active", "whoami": 0 } 2022-06-29T15:49:59.510 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-29T15:49:59.511 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2022-06-29T15:49:59.512 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi164.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2022-06-29T15:49:59.513 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:49:59.513 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:49:59.522 DEBUG:teuthology.orchestra.run.smithi164:> sudo modprobe fuse 2022-06-29T15:49:59.540 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:49:59.541 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:49:59.555 INFO:teuthology.orchestra.run.smithi164.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:49:59.556 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:49:59.557 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:49:59.558 DEBUG:teuthology.orchestra.run.smithi164:> ls /sys/fs/fuse/connections 2022-06-29T15:49:59.605 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-29T15:49:59.605 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:49:59.608 DEBUG:teuthology.orchestra.run.smithi164:> sudo modprobe fuse 2022-06-29T15:49:59.625 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:49:59.625 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:49:59.682 INFO:teuthology.orchestra.run.smithi164.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:49:59.683 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi164.stderr:2022-06-29T15:49:59.674+0000 7febe8b7f080 -1 init, newargv = 0x5603106bf300 newargc=9 2022-06-29T15:49:59.683 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi164.stderr:ceph-fuse[12643]: starting ceph client 2022-06-29T15:49:59.684 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:49:59.685 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:49:59.685 DEBUG:teuthology.orchestra.run.smithi164:> ls /sys/fs/fuse/connections 2022-06-29T15:49:59.693 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi164.stderr:ceph-fuse[12643]: starting fuse 2022-06-29T15:50:00.794 DEBUG:teuthology.orchestra.run.smithi164:> sudo modprobe fuse 2022-06-29T15:50:00.813 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:50:00.813 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:50:00.870 INFO:teuthology.orchestra.run.smithi164.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:50:00.871 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:50:00.872 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:00.873 DEBUG:teuthology.orchestra.run.smithi164:> ls /sys/fs/fuse/connections 2022-06-29T15:50:00.877 INFO:teuthology.orchestra.run.smithi164.stdout:52 2022-06-29T15:50:00.878 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-29T15:50:00.878 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:00.879 DEBUG:teuthology.orchestra.run.smithi164:> sudo python3 -c ' 2022-06-29T15:50:00.879 DEBUG:teuthology.orchestra.run.smithi164:> import glob 2022-06-29T15:50:00.880 DEBUG:teuthology.orchestra.run.smithi164:> import re 2022-06-29T15:50:00.880 DEBUG:teuthology.orchestra.run.smithi164:> import os 2022-06-29T15:50:00.881 DEBUG:teuthology.orchestra.run.smithi164:> import subprocess 2022-06-29T15:50:00.881 DEBUG:teuthology.orchestra.run.smithi164:> 2022-06-29T15:50:00.881 DEBUG:teuthology.orchestra.run.smithi164:> def find_socket(client_name): 2022-06-29T15:50:00.882 DEBUG:teuthology.orchestra.run.smithi164:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2022-06-29T15:50:00.882 DEBUG:teuthology.orchestra.run.smithi164:> files = glob.glob(asok_path) 2022-06-29T15:50:00.883 DEBUG:teuthology.orchestra.run.smithi164:> 2022-06-29T15:50:00.883 DEBUG:teuthology.orchestra.run.smithi164:> # Given a non-glob path, it better be there 2022-06-29T15:50:00.884 DEBUG:teuthology.orchestra.run.smithi164:> if "*" not in asok_path: 2022-06-29T15:50:00.884 DEBUG:teuthology.orchestra.run.smithi164:> assert(len(files) == 1) 2022-06-29T15:50:00.884 DEBUG:teuthology.orchestra.run.smithi164:> return files[0] 2022-06-29T15:50:00.885 DEBUG:teuthology.orchestra.run.smithi164:> 2022-06-29T15:50:00.885 DEBUG:teuthology.orchestra.run.smithi164:> for f in files: 2022-06-29T15:50:00.886 DEBUG:teuthology.orchestra.run.smithi164:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-29T15:50:00.886 DEBUG:teuthology.orchestra.run.smithi164:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-29T15:50:00.887 DEBUG:teuthology.orchestra.run.smithi164:> return f 2022-06-29T15:50:00.887 DEBUG:teuthology.orchestra.run.smithi164:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-29T15:50:00.887 DEBUG:teuthology.orchestra.run.smithi164:> 2022-06-29T15:50:00.888 DEBUG:teuthology.orchestra.run.smithi164:> print(find_socket("client.1")) 2022-06-29T15:50:00.888 DEBUG:teuthology.orchestra.run.smithi164:> ' 2022-06-29T15:50:00.931 INFO:teuthology.orchestra.run.smithi164.stdout:/var/run/ceph/ceph-client.1.12643.asok 2022-06-29T15:50:00.935 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.12643.asok 2022-06-29T15:50:00.936 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:00.936 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.12643.asok status 2022-06-29T15:50:01.076 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2022-06-29T15:50:01.077 INFO:teuthology.orchestra.run.smithi164.stdout: "metadata": { 2022-06-29T15:50:01.077 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_sha1": "f25ffac72e5a9e63860ff7deb6767a11f0559a16", 2022-06-29T15:50:01.077 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_version": "ceph version 15.2.16-304-gf25ffac7 (f25ffac72e5a9e63860ff7deb6767a11f0559a16) octopus (stable)", 2022-06-29T15:50:01.078 INFO:teuthology.orchestra.run.smithi164.stdout: "entity_id": "1", 2022-06-29T15:50:01.078 INFO:teuthology.orchestra.run.smithi164.stdout: "hostname": "smithi164", 2022-06-29T15:50:01.078 INFO:teuthology.orchestra.run.smithi164.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2022-06-29T15:50:01.078 INFO:teuthology.orchestra.run.smithi164.stdout: "pid": "12643", 2022-06-29T15:50:01.079 INFO:teuthology.orchestra.run.smithi164.stdout: "root": "/" 2022-06-29T15:50:01.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2022-06-29T15:50:01.079 INFO:teuthology.orchestra.run.smithi164.stdout: "dentry_count": 0, 2022-06-29T15:50:01.079 INFO:teuthology.orchestra.run.smithi164.stdout: "dentry_pinned_count": 0, 2022-06-29T15:50:01.080 INFO:teuthology.orchestra.run.smithi164.stdout: "id": 56258, 2022-06-29T15:50:01.080 INFO:teuthology.orchestra.run.smithi164.stdout: "inst": { 2022-06-29T15:50:01.080 INFO:teuthology.orchestra.run.smithi164.stdout: "name": { 2022-06-29T15:50:01.080 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "client", 2022-06-29T15:50:01.080 INFO:teuthology.orchestra.run.smithi164.stdout: "num": 56258 2022-06-29T15:50:01.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2022-06-29T15:50:01.081 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": { 2022-06-29T15:50:01.081 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "v1", 2022-06-29T15:50:01.081 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": "172.21.15.164:0", 2022-06-29T15:50:01.082 INFO:teuthology.orchestra.run.smithi164.stdout: "nonce": 1812149087 2022-06-29T15:50:01.082 INFO:teuthology.orchestra.run.smithi164.stdout: } 2022-06-29T15:50:01.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2022-06-29T15:50:01.082 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": { 2022-06-29T15:50:01.083 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "v1", 2022-06-29T15:50:01.083 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": "172.21.15.164:0", 2022-06-29T15:50:01.083 INFO:teuthology.orchestra.run.smithi164.stdout: "nonce": 1812149087 2022-06-29T15:50:01.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2022-06-29T15:50:01.083 INFO:teuthology.orchestra.run.smithi164.stdout: "inst_str": "client.56258 172.21.15.164:0/1812149087", 2022-06-29T15:50:01.084 INFO:teuthology.orchestra.run.smithi164.stdout: "addr_str": "172.21.15.164:0/1812149087", 2022-06-29T15:50:01.084 INFO:teuthology.orchestra.run.smithi164.stdout: "inode_count": 1, 2022-06-29T15:50:01.084 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_epoch": 45, 2022-06-29T15:50:01.085 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_epoch": 101, 2022-06-29T15:50:01.085 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_epoch_barrier": 0, 2022-06-29T15:50:01.085 INFO:teuthology.orchestra.run.smithi164.stdout: "blacklisted": false, 2022-06-29T15:50:01.085 INFO:teuthology.orchestra.run.smithi164.stdout: "fs_name": "cephfs" 2022-06-29T15:50:01.086 INFO:teuthology.orchestra.run.smithi164.stdout:} 2022-06-29T15:50:01.087 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:01.088 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:50:01.088 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:50:01.096 INFO:teuthology.orchestra.run.smithi164.stdout:fuseblk 2022-06-29T15:50:01.097 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-06-29T15:50:01.098 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:01.098 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:50:01.117 INFO:teuthology.orchestra.run:Running command with timeout 10800 2022-06-29T15:50:01.118 DEBUG:teuthology.orchestra.run.smithi099:> cd /home/ubuntu/cephtest/mnt.0 && sudo mkdir -p foo 2022-06-29T15:50:01.150 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2022-06-29T15:50:01.151 DEBUG:teuthology.orchestra.run.smithi099:> import os 2022-06-29T15:50:01.151 DEBUG:teuthology.orchestra.run.smithi099:> import stat 2022-06-29T15:50:01.152 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:50:01.152 DEBUG:teuthology.orchestra.run.smithi099:> print(os.stat("/home/ubuntu/cephtest/mnt.0/foo").st_ino) 2022-06-29T15:50:01.152 DEBUG:teuthology.orchestra.run.smithi099:> ' 2022-06-29T15:50:01.231 INFO:teuthology.orchestra.run.smithi099.stdout:1099511627776 2022-06-29T15:50:01.421 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:50:01.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.524+0000 7fd6f72b3700 1 Processor -- start 2022-06-29T15:50:01.536 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.524+0000 7fd6f72b3700 1 -- start start 2022-06-29T15:50:01.538 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.528+0000 7fd6f72b3700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd6f00611b0 0x7fd6f0146640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:01.538 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.528+0000 7fd6f72b3700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6f00b4520 con 0x7fd6f00b06d0 2022-06-29T15:50:01.539 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.528+0000 7fd6f72b3700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6f0061590 con 0x7fd6f00b0cf0 2022-06-29T15:50:01.539 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.528+0000 7fd6f72b3700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f00b4b10 con 0x7fd6f00611b0 2022-06-29T15:50:01.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.528+0000 7fd6f5850700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd6f00b06d0 0x7fd6f013ff00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37244/0 (socket says 172.21.15.47:37244) 2022-06-29T15:50:01.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.528+0000 7fd6f5850700 1 -- 172.21.15.47:0/4021182215 learned_addr learned my addr 172.21.15.47:0/4021182215 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:01.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.528+0000 7fd6f504f700 1 --2- 172.21.15.47:0/4021182215 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd6f00611b0 0x7fd6f0146640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:01.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.528+0000 7fd6e3fff700 1 -- 172.21.15.47:0/4021182215 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1468288125 0 0) 0x7fd6f00b4520 con 0x7fd6f00b06d0 2022-06-29T15:50:01.541 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.528+0000 7fd6e3fff700 1 -- 172.21.15.47:0/4021182215 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6d4003260 con 0x7fd6f00b06d0 2022-06-29T15:50:01.541 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.528+0000 7fd6e3fff700 1 -- 172.21.15.47:0/4021182215 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2684636826 0 0) 0x7fd6f0061590 con 0x7fd6f00b0cf0 2022-06-29T15:50:01.541 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.528+0000 7fd6e3fff700 1 -- 172.21.15.47:0/4021182215 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6f00b4520 con 0x7fd6f00b0cf0 2022-06-29T15:50:01.541 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.528+0000 7fd6f504f700 1 -- 172.21.15.47:0/4021182215 >> v1:172.21.15.64:6789/0 conn(0x7fd6f00b0cf0 legacy=0x7fd6f0146b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:01.542 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.528+0000 7fd6e3fff700 1 -- 172.21.15.47:0/4021182215 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1015534344 0 0) 0x7fd6d4003260 con 0x7fd6f00b06d0 2022-06-29T15:50:01.542 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.528+0000 7fd6f504f700 1 -- 172.21.15.47:0/4021182215 >> v1:172.21.15.47:6789/0 conn(0x7fd6f00b06d0 legacy=0x7fd6f013ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:01.542 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.528+0000 7fd6f504f700 1 -- 172.21.15.47:0/4021182215 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f00b5e00 con 0x7fd6f00611b0 2022-06-29T15:50:01.542 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.528+0000 7fd6f504f700 1 --2- 172.21.15.47:0/4021182215 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd6f00611b0 0x7fd6f0146640 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7fd6e8001490 tx=0x7fd6e4006df0).ready entity=mon.2 client_cookie=eb478cb19fb14348 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:01.543 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.528+0000 7fd6e3fff700 1 -- 172.21.15.47:0/4021182215 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd6e80019e0 con 0x7fd6f00611b0 2022-06-29T15:50:01.543 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.528+0000 7fd6e3fff700 1 -- 172.21.15.47:0/4021182215 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e4000e90 con 0x7fd6f00611b0 2022-06-29T15:50:01.543 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.528+0000 7fd6e3fff700 1 -- 172.21.15.47:0/4021182215 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd6e4012460 con 0x7fd6f00611b0 2022-06-29T15:50:01.548 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.528+0000 7fd6f72b3700 1 -- 172.21.15.47:0/4021182215 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd6f00611b0 msgr2=0x7fd6f0146640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:01.549 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.528+0000 7fd6f72b3700 1 --2- 172.21.15.47:0/4021182215 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd6f00611b0 0x7fd6f0146640 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7fd6e8001490 tx=0x7fd6e4006df0).stop 2022-06-29T15:50:01.549 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.528+0000 7fd6f72b3700 1 -- 172.21.15.47:0/4021182215 shutdown_connections 2022-06-29T15:50:01.549 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.528+0000 7fd6f72b3700 1 --2- 172.21.15.47:0/4021182215 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd6f00611b0 0x7fd6f0146640 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:01.550 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.528+0000 7fd6f72b3700 1 -- 172.21.15.47:0/4021182215 >> 172.21.15.47:0/4021182215 conn(0x7fd6f00b98b0 msgr2=0x7fd6f00b9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:01.550 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.532+0000 7fd6f72b3700 1 -- 172.21.15.47:0/4021182215 shutdown_connections 2022-06-29T15:50:01.550 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.532+0000 7fd6f72b3700 1 -- 172.21.15.47:0/4021182215 wait complete. 2022-06-29T15:50:01.550 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.532+0000 7fd6f72b3700 1 Processor -- start 2022-06-29T15:50:01.551 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.532+0000 7fd6f72b3700 1 -- start start 2022-06-29T15:50:01.551 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.532+0000 7fd6f72b3700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd6f00611b0 0x7fd6f0152f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:01.551 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.532+0000 7fd6f72b3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6f00b06d0 0x7fd6f0139800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:01.551 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.532+0000 7fd6f72b3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6f00b0cf0 0x7fd6f013c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:01.552 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.532+0000 7fd6f72b3700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f00b6550 con 0x7fd6f00b06d0 2022-06-29T15:50:01.552 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.532+0000 7fd6f72b3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f01371a0 con 0x7fd6f00b0cf0 2022-06-29T15:50:01.552 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.532+0000 7fd6f72b3700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f01373d0 con 0x7fd6f00611b0 2022-06-29T15:50:01.552 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.532+0000 7fd6f484e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6f00b06d0 0x7fd6f0139800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:01.553 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.532+0000 7fd6f484e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6f00b06d0 0x7fd6f0139800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51694/0 (socket says 172.21.15.47:51694) 2022-06-29T15:50:01.553 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.532+0000 7fd6f484e700 1 -- 172.21.15.47:0/1891450171 learned_addr learned my addr 172.21.15.47:0/1891450171 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:01.553 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.532+0000 7fd6f504f700 1 --2- 172.21.15.47:0/1891450171 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd6f00611b0 0x7fd6f0152f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:01.553 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.532+0000 7fd6f5850700 1 --2- 172.21.15.47:0/1891450171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6f00b0cf0 0x7fd6f013c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:01.554 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.532+0000 7fd6f484e700 1 -- 172.21.15.47:0/1891450171 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd6f00611b0 msgr2=0x7fd6f0152f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:01.554 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.532+0000 7fd6f484e700 1 --2- 172.21.15.47:0/1891450171 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd6f00611b0 0x7fd6f0152f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:01.554 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.532+0000 7fd6f484e700 1 -- 172.21.15.47:0/1891450171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6f00b0cf0 msgr2=0x7fd6f013c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:01.555 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.532+0000 7fd6f484e700 1 --2- 172.21.15.47:0/1891450171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6f00b0cf0 0x7fd6f013c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:01.555 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.532+0000 7fd6f484e700 1 -- 172.21.15.47:0/1891450171 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e400b180 con 0x7fd6f00b06d0 2022-06-29T15:50:01.555 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.536+0000 7fd6f484e700 1 --2- 172.21.15.47:0/1891450171 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6f00b06d0 0x7fd6f0139800 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 rx=0x7fd6ec001110 tx=0x7fd6ec007ae0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:01.556 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.536+0000 7fd6e27fc700 1 -- 172.21.15.47:0/1891450171 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd6ec012400 con 0x7fd6f00b06d0 2022-06-29T15:50:01.556 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.536+0000 7fd6e27fc700 1 -- 172.21.15.47:0/1891450171 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6ec012560 con 0x7fd6f00b06d0 2022-06-29T15:50:01.556 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.536+0000 7fd6f72b3700 1 -- 172.21.15.47:0/1891450171 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6f0137670 con 0x7fd6f00b06d0 2022-06-29T15:50:01.556 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.536+0000 7fd6e27fc700 1 -- 172.21.15.47:0/1891450171 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd6ec0128c0 con 0x7fd6f00b06d0 2022-06-29T15:50:01.557 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.536+0000 7fd6f72b3700 1 -- 172.21.15.47:0/1891450171 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6f0139cf0 con 0x7fd6f00b06d0 2022-06-29T15:50:01.557 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.536+0000 7fd6f72b3700 1 -- 172.21.15.47:0/1891450171 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6f00573d0 con 0x7fd6f00b06d0 2022-06-29T15:50:01.557 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.536+0000 7fd6e27fc700 1 -- 172.21.15.47:0/1891450171 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd6ec019030 con 0x7fd6f00b06d0 2022-06-29T15:50:01.557 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.536+0000 7fd6e27fc700 1 --2- 172.21.15.47:0/1891450171 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd6d8053a10 0x7fd6d8055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:01.557 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.536+0000 7fd6e27fc700 1 -- 172.21.15.47:0/1891450171 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd6ec06b310 con 0x7fd6f00b06d0 2022-06-29T15:50:01.558 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.536+0000 7fd6f504f700 1 --2- 172.21.15.47:0/1891450171 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd6d8053a10 0x7fd6d8055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:01.558 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.540+0000 7fd6f504f700 1 --2- 172.21.15.47:0/1891450171 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd6d8053a10 0x7fd6d8055ed0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7fd6e401a000 tx=0x7fd6e4014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:01.559 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.548+0000 7fd6e27fc700 1 -- 172.21.15.47:0/1891450171 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd6ec0420d0 con 0x7fd6f00b06d0 2022-06-29T15:50:01.723 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.712+0000 7fd6f72b3700 1 -- 172.21.15.47:0/1891450171 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd6f0061750 con 0x7fd6f00b06d0 2022-06-29T15:50:01.724 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.716+0000 7fd6e27fc700 1 -- 172.21.15.47:0/1891450171 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+2493 (secure 0 0 0) 0x7fd6ec0456f0 con 0x7fd6f00b06d0 2022-06-29T15:50:01.724 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:50:01.724 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":45,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6969,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3642031880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3642031880},{"type":"v1","addr":"172.21.15.47:6835","nonce":3642031880}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":43},{"gid":6984,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/891572891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":891572891},{"type":"v1","addr":"172.21.15.47:6837","nonce":891572891}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":43}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:49:50.788599+0000","modified":"2022-06-29T15:49:51.791379+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":56102},"failed":[],"damaged":[],"stopped":[],"info":{"gid_56102":{"gid":56102,"name":"b","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6833/3819760998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3819760998},{"type":"v1","addr":"172.21.15.64:6833","nonce":3819760998}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2022-06-29T15:50:01.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.716+0000 7fd6f72b3700 1 -- 172.21.15.47:0/1891450171 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd6d8053a10 msgr2=0x7fd6d8055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:01.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.716+0000 7fd6f72b3700 1 --2- 172.21.15.47:0/1891450171 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd6d8053a10 0x7fd6d8055ed0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7fd6e401a000 tx=0x7fd6e4014000).stop 2022-06-29T15:50:01.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.716+0000 7fd6f72b3700 1 -- 172.21.15.47:0/1891450171 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6f00b06d0 msgr2=0x7fd6f0139800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:01.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.716+0000 7fd6f72b3700 1 --2- 172.21.15.47:0/1891450171 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6f00b06d0 0x7fd6f0139800 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 rx=0x7fd6ec001110 tx=0x7fd6ec007ae0).stop 2022-06-29T15:50:01.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.716+0000 7fd6f72b3700 1 -- 172.21.15.47:0/1891450171 shutdown_connections 2022-06-29T15:50:01.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.716+0000 7fd6f72b3700 1 --2- 172.21.15.47:0/1891450171 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd6d8053a10 0x7fd6d8055ed0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:01.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.716+0000 7fd6f72b3700 1 --2- 172.21.15.47:0/1891450171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6f00b0cf0 0x7fd6f013c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:01.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.716+0000 7fd6f72b3700 1 --2- 172.21.15.47:0/1891450171 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6f00b06d0 0x7fd6f0139800 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:01.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.716+0000 7fd6f72b3700 1 --2- 172.21.15.47:0/1891450171 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd6f00611b0 0x7fd6f0152f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:01.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.716+0000 7fd6f72b3700 1 -- 172.21.15.47:0/1891450171 >> 172.21.15.47:0/1891450171 conn(0x7fd6f00b98b0 msgr2=0x7fd6f00b51a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:01.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.716+0000 7fd6f72b3700 1 -- 172.21.15.47:0/1891450171 shutdown_connections 2022-06-29T15:50:01.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:01.716+0000 7fd6f72b3700 1 -- 172.21.15.47:0/1891450171 wait complete. 2022-06-29T15:50:01.729 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 45 2022-06-29T15:50:01.737 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json dump inode 0x10000000000 2022-06-29T15:50:01.852 INFO:teuthology.orchestra.run.smithi064.stdout:{"path":"/foo","ino":1099511627776,"rdev":0,"ctime":"2022-06-29T15:50:01.136743+0000","btime":"2022-06-29T15:50:01.136743+0000","mode":16877,"uid":0,"gid":0,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":0,"stripe_count":0,"object_size":0,"pool_id":-1,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2022-06-29T15:50:01.136743+0000","atime":"2022-06-29T15:50:01.136743+0000","time_warp_seq":0,"change_attr":0,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"0.000000","num_files":0,"num_subdirs":0,"change_attr":0},"rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":1,"rsnaps":0,"rctime":"0.000000"},"accounted_rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":1,"rsnaps":0,"rctime":"0.000000"},"version":2,"file_data_version":0,"xattr_version":1,"backtrace_version":2,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"dirfrag":1,"caps":1,"dirtyparent":1,"dirty":1},"nref":4,"versionlock":{"gather_set":[],"state":"lock","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{"gather_set":[],"state":"excl","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"linklock":{},"dirfragtreelock":{},"filelock":{"gather_set":[],"state":"excl","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"xattrlock":{"gather_set":[],"state":"excl","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"snaplock":{},"nestlock":{"gather_set":[],"state":"lock","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"flocklock":{},"policylock":{},"states":["auth","dirty","dirtyparent","dirtypool"],"client_caps":[{"client_id":7137,"pending":"pAsxLsXsxFsx","issued":"pAsxLsXsxFsx","wanted":"-","last_sent":1}],"loner":7137,"want_loner":7137,"mds_caps_wanted":[]} 2022-06-29T15:50:01.853 DEBUG:tasks.cephfs.filesystem:_json_asok output { "accounted_rstat": { "rbytes": 0, "rctime": "0.000000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "atime": "2022-06-29T15:50:01.136743+0000", "auth_pins": 0, "auth_state": { "replicas": {} }, "authlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "excl", "xlock_by": {} }, "backtrace_version": 2, "btime": "2022-06-29T15:50:01.136743+0000", "change_attr": 0, "client_caps": [ { "client_id": 7137, "issued": "pAsxLsXsxFsx", "last_sent": 1, "pending": "pAsxLsXsxFsx", "wanted": "-" } ], "client_ranges": [], "ctime": "2022-06-29T15:50:01.136743+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 0, "mtime": "0.000000", "num_files": 0, "num_subdirs": 0, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "excl", "xlock_by": {} }, "flocklock": {}, "gid": 0, "ino": 1099511627776, "is_auth": true, "is_freezing": false, "is_frozen": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 0, "pool_id": -1, "pool_ns": "", "stripe_count": 0, "stripe_unit": 0 }, "linklock": {}, "loner": 7137, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 16877, "mtime": "2022-06-29T15:50:01.136743+0000", "nestlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "xlock_by": {} }, "nlink": 1, "nref": 4, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "path": "/foo", "pins": { "caps": 1, "dirfrag": 1, "dirty": 1, "dirtyparent": 1, "request": 0 }, "policylock": {}, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 0, "rctime": "0.000000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "size": 0, "snaplock": {}, "states": [ "auth", "dirty", "dirtyparent", "dirtypool" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 0, "version": 2, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "xlock_by": {} }, "want_loner": 7137, "xattr_version": 1, "xattrlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "excl", "xlock_by": {} }, "xattrs": [] } 2022-06-29T15:50:01.854 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-06-29T15:50:01.854 DEBUG:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:50:01.875 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:50:01.896 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:50:01.918 DEBUG:teuthology.orchestra.run.smithi164:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:50:01.939 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-06-29T15:50:01.940 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:01.941 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:50:01.941 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:50:01.949 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2022-06-29T15:50:01.950 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-29T15:50:01.951 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi099.front.sepia.ceph.com... 2022-06-29T15:50:01.951 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-06-29T15:50:01.952 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:50:02.009 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[13092]: fuse finished with error 0 and tester_r 0 2022-06-29T15:50:02.011 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:02.011 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:50:02.012 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:50:02.017 INFO:teuthology.orchestra.run.smithi099.stdout:ext2/ext3 2022-06-29T15:50:02.018 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-06-29T15:50:02.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-29T15:50:02.136 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:50:02.185 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:02.186 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:50:02.186 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:50:02.194 INFO:teuthology.orchestra.run.smithi164.stdout:fuseblk 2022-06-29T15:50:02.196 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-06-29T15:50:02.196 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi164.front.sepia.ceph.com... 2022-06-29T15:50:02.197 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-06-29T15:50:02.197 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:50:02.213 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi164.stderr:ceph-fuse[12643]: fuse finished with error 0 and tester_r 0 2022-06-29T15:50:02.215 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:02.215 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:50:02.216 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:50:02.265 INFO:teuthology.orchestra.run.smithi164.stdout:ext2/ext3 2022-06-29T15:50:02.266 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-06-29T15:50:02.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-29T15:50:02.279 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:50:02.286 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-06-29T15:50:02.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.392+0000 7f1ad13a8700 1 Processor -- start 2022-06-29T15:50:02.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.392+0000 7f1ad13a8700 1 -- start start 2022-06-29T15:50:02.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.392+0000 7f1ad13a8700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1acc0b5c60 0x7f1acc144980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:02.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.392+0000 7f1ad13a8700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1acc060eb0 0x7f1acc144e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:02.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.392+0000 7f1ad13a8700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1acc0a72f0 0x7f1acc0b1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:02.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.392+0000 7f1ad13a8700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1acc145360 con 0x7f1acc0b5c60 2022-06-29T15:50:02.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.392+0000 7f1ad13a8700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1acc0a9120 con 0x7f1acc0a72f0 2022-06-29T15:50:02.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.392+0000 7f1ad13a8700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1acc0b0b80 con 0x7f1acc060eb0 2022-06-29T15:50:02.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.392+0000 7f1acbfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1acc0b5c60 0x7f1acc144980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:02.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.392+0000 7f1acbfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1acc0b5c60 0x7f1acc144980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51700/0 (socket says 172.21.15.47:51700) 2022-06-29T15:50:02.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.392+0000 7f1acbfff700 1 -- 172.21.15.47:0/879922791 learned_addr learned my addr 172.21.15.47:0/879922791 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:02.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.392+0000 7f1ac3fff700 1 --2- 172.21.15.47:0/879922791 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1acc060eb0 0x7f1acc144e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:02.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.392+0000 7f1ad0ba7700 1 --2- 172.21.15.47:0/879922791 >> v2:172.21.15.64:3300/0 conn(0x7f1acc0a72f0 0x7f1acc0b1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:02.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.392+0000 7f1acbfff700 1 -- 172.21.15.47:0/879922791 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1acc060eb0 msgr2=0x7f1acc144e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:02.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.392+0000 7f1acbfff700 1 --2- 172.21.15.47:0/879922791 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1acc060eb0 0x7f1acc144e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:02.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.392+0000 7f1acbfff700 1 -- 172.21.15.47:0/879922791 >> v2:172.21.15.64:3300/0 conn(0x7f1acc0a72f0 msgr2=0x7f1acc0b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:02.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.396+0000 7f1acbfff700 1 --2- 172.21.15.47:0/879922791 >> v2:172.21.15.64:3300/0 conn(0x7f1acc0a72f0 0x7f1acc0b1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:02.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.396+0000 7f1acbfff700 1 -- 172.21.15.47:0/879922791 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1acc0a89e0 con 0x7f1acc0b5c60 2022-06-29T15:50:02.409 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.396+0000 7f1acbfff700 1 --2- 172.21.15.47:0/879922791 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1acc0b5c60 0x7f1acc144980 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 rx=0x7f1abc002f70 tx=0x7f1abc006e50).ready entity=mon.0 client_cookie=390502060b7c6585 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:02.409 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.396+0000 7f1acb7fe700 1 -- 172.21.15.47:0/879922791 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1abc00d570 con 0x7f1acc0b5c60 2022-06-29T15:50:02.409 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.396+0000 7f1acb7fe700 1 -- 172.21.15.47:0/879922791 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1abc000e90 con 0x7f1acc0b5c60 2022-06-29T15:50:02.409 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.396+0000 7f1acb7fe700 1 -- 172.21.15.47:0/879922791 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1abc00d900 con 0x7f1acc0b5c60 2022-06-29T15:50:02.410 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.396+0000 7f1ad13a8700 1 -- 172.21.15.47:0/879922791 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1acc0b5c60 msgr2=0x7f1acc144980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:02.410 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.396+0000 7f1ad13a8700 1 --2- 172.21.15.47:0/879922791 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1acc0b5c60 0x7f1acc144980 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 rx=0x7f1abc002f70 tx=0x7f1abc006e50).stop 2022-06-29T15:50:02.410 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.396+0000 7f1ad13a8700 1 -- 172.21.15.47:0/879922791 shutdown_connections 2022-06-29T15:50:02.410 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.396+0000 7f1ad13a8700 1 --2- 172.21.15.47:0/879922791 >> v2:172.21.15.64:3300/0 conn(0x7f1acc0a72f0 0x7f1acc0b1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:02.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.396+0000 7f1ad13a8700 1 --2- 172.21.15.47:0/879922791 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1acc060eb0 0x7f1acc144e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:02.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.396+0000 7f1ad13a8700 1 --2- 172.21.15.47:0/879922791 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1acc0b5c60 0x7f1acc144980 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:02.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.396+0000 7f1ad13a8700 1 -- 172.21.15.47:0/879922791 >> 172.21.15.47:0/879922791 conn(0x7f1acc062560 msgr2=0x7f1acc062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:02.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.396+0000 7f1ad13a8700 1 -- 172.21.15.47:0/879922791 shutdown_connections 2022-06-29T15:50:02.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.396+0000 7f1ad13a8700 1 -- 172.21.15.47:0/879922791 wait complete. 2022-06-29T15:50:02.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.396+0000 7f1ad13a8700 1 Processor -- start 2022-06-29T15:50:02.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.396+0000 7f1ad13a8700 1 -- start start 2022-06-29T15:50:02.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.400+0000 7f1ad13a8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1acc060eb0 0x7f1acc14d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:02.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.400+0000 7f1ad13a8700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1acc0a72f0 0x7f1acc0adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:02.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.400+0000 7f1ad13a8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1acc0b5c60 0x7f1acc0ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:02.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.400+0000 7f1ad13a8700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1acc149390 con 0x7f1acc060eb0 2022-06-29T15:50:02.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.400+0000 7f1ad13a8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1acc0ae9a0 con 0x7f1acc0b5c60 2022-06-29T15:50:02.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.400+0000 7f1ad13a8700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1acc149ed0 con 0x7f1acc0a72f0 2022-06-29T15:50:02.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.400+0000 7f1acbfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1acc060eb0 0x7f1acc14d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:02.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.400+0000 7f1acbfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1acc060eb0 0x7f1acc14d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51708/0 (socket says 172.21.15.47:51708) 2022-06-29T15:50:02.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.400+0000 7f1acbfff700 1 -- 172.21.15.47:0/926727433 learned_addr learned my addr 172.21.15.47:0/926727433 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:02.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.400+0000 7f1ac3fff700 1 --2- 172.21.15.47:0/926727433 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1acc0a72f0 0x7f1acc0adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:02.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.400+0000 7f1ad0ba7700 1 --2- 172.21.15.47:0/926727433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1acc0b5c60 0x7f1acc0ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:02.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.400+0000 7f1acbfff700 1 -- 172.21.15.47:0/926727433 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1acc0a72f0 msgr2=0x7f1acc0adfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:02.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.400+0000 7f1acbfff700 1 --2- 172.21.15.47:0/926727433 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1acc0a72f0 0x7f1acc0adfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:02.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.400+0000 7f1acbfff700 1 -- 172.21.15.47:0/926727433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1acc0b5c60 msgr2=0x7f1acc0ae4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:02.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.400+0000 7f1acbfff700 1 --2- 172.21.15.47:0/926727433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1acc0b5c60 0x7f1acc0ae4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:02.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.400+0000 7f1acbfff700 1 -- 172.21.15.47:0/926727433 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1abc00b1e0 con 0x7f1acc060eb0 2022-06-29T15:50:02.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.400+0000 7f1ad0ba7700 1 --2- 172.21.15.47:0/926727433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1acc0b5c60 0x7f1acc0ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:50:02.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.400+0000 7f1acbfff700 1 --2- 172.21.15.47:0/926727433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1acc060eb0 0x7f1acc14d010 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 rx=0x7f1acc0aeae0 tx=0x7f1abc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:02.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.400+0000 7f1ac9ffb700 1 -- 172.21.15.47:0/926727433 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1abc00d8f0 con 0x7f1acc060eb0 2022-06-29T15:50:02.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.400+0000 7f1ac9ffb700 1 -- 172.21.15.47:0/926727433 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1abc011bf0 con 0x7f1acc060eb0 2022-06-29T15:50:02.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.400+0000 7f1ad13a8700 1 -- 172.21.15.47:0/926727433 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1acc0ada20 con 0x7f1acc060eb0 2022-06-29T15:50:02.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.400+0000 7f1ac9ffb700 1 -- 172.21.15.47:0/926727433 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1abc006030 con 0x7f1acc060eb0 2022-06-29T15:50:02.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.400+0000 7f1ad13a8700 1 -- 172.21.15.47:0/926727433 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1acc14a3d0 con 0x7f1acc060eb0 2022-06-29T15:50:02.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.400+0000 7f1ac9ffb700 1 -- 172.21.15.47:0/926727433 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1abc026030 con 0x7f1acc060eb0 2022-06-29T15:50:02.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.400+0000 7f1ad13a8700 1 -- 172.21.15.47:0/926727433 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1acc14a590 con 0x7f1acc060eb0 2022-06-29T15:50:02.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.404+0000 7f1ac9ffb700 1 --2- 172.21.15.47:0/926727433 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1ab0053a10 0x7f1ab0055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:02.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.404+0000 7f1ac9ffb700 1 -- 172.21.15.47:0/926727433 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f1abc06d020 con 0x7f1acc060eb0 2022-06-29T15:50:02.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.404+0000 7f1ac3fff700 1 --2- 172.21.15.47:0/926727433 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1ab0053a10 0x7f1ab0055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:02.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.404+0000 7f1ac3fff700 1 --2- 172.21.15.47:0/926727433 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1ab0053a10 0x7f1ab0055ed0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x7f1ab400c000 tx=0x7f1ab4008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:02.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.408+0000 7f1ac9ffb700 1 -- 172.21.15.47:0/926727433 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1abc043e20 con 0x7f1acc060eb0 2022-06-29T15:50:02.547 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.536+0000 7f1ad13a8700 1 -- 172.21.15.47:0/926727433 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f1acc0573d0 con 0x7f1acc060eb0 2022-06-29T15:50:02.550 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.540+0000 7f1ac9ffb700 1 -- 172.21.15.47:0/926727433 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v101) v1 ==== 81+0+34043 (secure 0 0 0) 0x7f1abc024070 con 0x7f1acc060eb0 2022-06-29T15:50:02.551 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:50:02.551 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2022-06-29T15:50:02.551 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 101, 2022-06-29T15:50:02.552 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "9a68894a-53ce-400e-838b-14eab91b834c", 2022-06-29T15:50:02.552 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2022-06-29T15:42:14.046361+0000", 2022-06-29T15:50:02.552 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2022-06-29T15:49:51.792208+0000", 2022-06-29T15:50:02.552 INFO:teuthology.orchestra.run.smithi047.stdout: "last_up_change": "2022-06-29T15:42:20.929789+0000", 2022-06-29T15:50:02.553 INFO:teuthology.orchestra.run.smithi047.stdout: "last_in_change": "2022-06-29T15:42:20.929789+0000", 2022-06-29T15:50:02.553 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-06-29T15:50:02.553 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_num": 5799936, 2022-06-29T15:50:02.553 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_set": [ 2022-06-29T15:50:02.554 INFO:teuthology.orchestra.run.smithi047.stdout: "pglog_hardlimit", 2022-06-29T15:50:02.554 INFO:teuthology.orchestra.run.smithi047.stdout: "purged_snapdirs", 2022-06-29T15:50:02.554 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_deletes", 2022-06-29T15:50:02.554 INFO:teuthology.orchestra.run.smithi047.stdout: "sortbitwise" 2022-06-29T15:50:02.555 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:50:02.555 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_version": 5, 2022-06-29T15:50:02.555 INFO:teuthology.orchestra.run.smithi047.stdout: "full_ratio": 0.94999998807907104, 2022-06-29T15:50:02.555 INFO:teuthology.orchestra.run.smithi047.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-06-29T15:50:02.556 INFO:teuthology.orchestra.run.smithi047.stdout: "nearfull_ratio": 0.85000002384185791, 2022-06-29T15:50:02.556 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_snapshot": "", 2022-06-29T15:50:02.556 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_max": 18, 2022-06-29T15:50:02.556 INFO:teuthology.orchestra.run.smithi047.stdout: "max_osd": 8, 2022-06-29T15:50:02.556 INFO:teuthology.orchestra.run.smithi047.stdout: "require_min_compat_client": "jewel", 2022-06-29T15:50:02.557 INFO:teuthology.orchestra.run.smithi047.stdout: "min_compat_client": "jewel", 2022-06-29T15:50:02.557 INFO:teuthology.orchestra.run.smithi047.stdout: "require_osd_release": "octopus", 2022-06-29T15:50:02.557 INFO:teuthology.orchestra.run.smithi047.stdout: "pools": [ 2022-06-29T15:50:02.558 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.558 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 1, 2022-06-29T15:50:02.558 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "device_health_metrics", 2022-06-29T15:50:02.558 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:42:16.971431+0000", 2022-06-29T15:50:02.558 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:50:02.559 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:50:02.559 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:50:02.559 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:50:02.559 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:50:02.560 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:50:02.560 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:50:02.561 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:50:02.561 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 1, 2022-06-29T15:50:02.561 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 1, 2022-06-29T15:50:02.561 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 1, 2022-06-29T15:50:02.562 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 1, 2022-06-29T15:50:02.562 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 1, 2022-06-29T15:50:02.562 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:50:02.562 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:50:02.563 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:50:02.563 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:50:02.563 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:50:02.563 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:50:02.564 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:50:02.564 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.564 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "16", 2022-06-29T15:50:02.564 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:50:02.565 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:50:02.565 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:50:02.565 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:50:02.565 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:50:02.565 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:50:02.566 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:50:02.566 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:50:02.566 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:50:02.567 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:50:02.567 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:50:02.567 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:50:02.567 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:50:02.568 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:50:02.568 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:50:02.568 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:50:02.568 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:50:02.568 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:50:02.569 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:50:02.569 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:50:02.569 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:50:02.569 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:50:02.570 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:50:02.570 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:50:02.570 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:50:02.570 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:50:02.571 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.571 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:50:02.571 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:50:02.572 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:50:02.572 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:50:02.572 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:50:02.572 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:50:02.573 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:50:02.573 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:50:02.573 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:50:02.573 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:50:02.574 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:50:02.574 INFO:teuthology.orchestra.run.smithi047.stdout: "options": { 2022-06-29T15:50:02.574 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_min": 1 2022-06-29T15:50:02.574 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.575 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:50:02.575 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_devicehealth": {} 2022-06-29T15:50:02.575 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:02.575 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.576 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.576 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 2, 2022-06-29T15:50:02.576 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "rbd", 2022-06-29T15:50:02.576 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:42:25.290884+0000", 2022-06-29T15:50:02.577 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:50:02.577 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:50:02.577 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:50:02.577 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:50:02.578 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:50:02.578 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:50:02.578 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:50:02.579 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:50:02.579 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2022-06-29T15:50:02.579 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2022-06-29T15:50:02.579 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2022-06-29T15:50:02.580 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2022-06-29T15:50:02.580 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2022-06-29T15:50:02.580 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:50:02.580 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:50:02.581 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:50:02.581 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:50:02.581 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:50:02.582 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:50:02.582 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:50:02.582 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.582 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "19", 2022-06-29T15:50:02.583 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:50:02.583 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:50:02.583 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:50:02.584 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:50:02.584 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:50:02.584 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:50:02.584 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:50:02.584 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:50:02.585 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:50:02.585 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:50:02.586 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:50:02.586 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:50:02.586 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:50:02.587 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:50:02.587 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:50:02.587 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:50:02.587 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:50:02.588 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:50:02.588 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:50:02.588 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:50:02.588 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:50:02.589 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:50:02.589 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:50:02.589 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:50:02.590 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:50:02.590 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:50:02.590 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.591 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:50:02.591 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:50:02.591 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:50:02.591 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:50:02.592 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:50:02.592 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:50:02.592 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:50:02.592 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:50:02.593 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:50:02.593 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:50:02.593 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:50:02.593 INFO:teuthology.orchestra.run.smithi047.stdout: "options": {}, 2022-06-29T15:50:02.593 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:50:02.594 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd": {} 2022-06-29T15:50:02.594 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:02.594 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.594 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.595 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 17, 2022-06-29T15:50:02.595 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "cephfs_metadata", 2022-06-29T15:50:02.595 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:49:48.525959+0000", 2022-06-29T15:50:02.595 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:50:02.596 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:50:02.596 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:50:02.597 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:50:02.597 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:50:02.597 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:50:02.598 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:50:02.598 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:50:02.599 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2022-06-29T15:50:02.599 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2022-06-29T15:50:02.600 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2022-06-29T15:50:02.600 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2022-06-29T15:50:02.600 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2022-06-29T15:50:02.600 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:50:02.600 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:50:02.601 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:50:02.601 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:50:02.601 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:50:02.601 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:50:02.602 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:50:02.602 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.602 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "100", 2022-06-29T15:50:02.603 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:50:02.603 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:50:02.603 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:50:02.603 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:50:02.604 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:50:02.604 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:50:02.604 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:50:02.604 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:50:02.604 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:50:02.605 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:50:02.605 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:50:02.605 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:50:02.605 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:50:02.606 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:50:02.606 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:50:02.606 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:50:02.606 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:50:02.607 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:50:02.607 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:50:02.607 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:50:02.607 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:50:02.608 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:50:02.608 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:50:02.608 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:50:02.608 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:50:02.608 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:50:02.609 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.609 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:50:02.609 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:50:02.609 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:50:02.610 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:50:02.610 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:50:02.610 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:50:02.610 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:50:02.611 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:50:02.611 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:50:02.611 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:50:02.611 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:50:02.612 INFO:teuthology.orchestra.run.smithi047.stdout: "options": { 2022-06-29T15:50:02.612 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_bias": 4, 2022-06-29T15:50:02.612 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_min": 16, 2022-06-29T15:50:02.612 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_priority": 5 2022-06-29T15:50:02.612 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.613 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:50:02.613 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs": { 2022-06-29T15:50:02.613 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": "cephfs" 2022-06-29T15:50:02.613 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:02.614 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:02.614 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.614 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.614 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 18, 2022-06-29T15:50:02.615 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "cephfs_data", 2022-06-29T15:50:02.615 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:49:49.805331+0000", 2022-06-29T15:50:02.615 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:50:02.615 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:50:02.616 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:50:02.616 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:50:02.616 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:50:02.616 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:50:02.617 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:50:02.617 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:50:02.617 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2022-06-29T15:50:02.617 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2022-06-29T15:50:02.618 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2022-06-29T15:50:02.618 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2022-06-29T15:50:02.618 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2022-06-29T15:50:02.618 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:50:02.619 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:50:02.619 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:50:02.619 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:50:02.619 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:50:02.619 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:50:02.620 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:50:02.620 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.620 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "101", 2022-06-29T15:50:02.620 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:50:02.621 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:50:02.621 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:50:02.621 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:50:02.621 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:50:02.622 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:50:02.622 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:50:02.622 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:50:02.622 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:50:02.623 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:50:02.623 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:50:02.623 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:50:02.623 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:50:02.624 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:50:02.624 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:50:02.624 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:50:02.624 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:50:02.625 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:50:02.625 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:50:02.625 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:50:02.625 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:50:02.626 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:50:02.626 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:50:02.626 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:50:02.626 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:50:02.627 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:50:02.627 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.627 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:50:02.627 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:50:02.628 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:50:02.628 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:50:02.628 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:50:02.628 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:50:02.629 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:50:02.629 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:50:02.629 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:50:02.629 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:50:02.629 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:50:02.630 INFO:teuthology.orchestra.run.smithi047.stdout: "options": {}, 2022-06-29T15:50:02.630 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:50:02.630 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs": { 2022-06-29T15:50:02.630 INFO:teuthology.orchestra.run.smithi047.stdout: "data": "cephfs" 2022-06-29T15:50:02.631 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:02.631 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:02.631 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:02.631 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:50:02.632 INFO:teuthology.orchestra.run.smithi047.stdout: "osds": [ 2022-06-29T15:50:02.632 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.632 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2022-06-29T15:50:02.632 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "e23eb023-4706-4201-bf72-4bf19c912c5d", 2022-06-29T15:50:02.633 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:50:02.633 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:50:02.633 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:50:02.633 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:50:02.634 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:50:02.634 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:50:02.634 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:50:02.634 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 99, 2022-06-29T15:50:02.635 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:50:02.635 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:50:02.635 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:50:02.635 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:02.635 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.636 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:02.636 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6818", 2022-06-29T15:50:02.636 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:50:02.636 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.637 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.637 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:02.637 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6819", 2022-06-29T15:50:02.637 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:50:02.638 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:02.638 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:02.638 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.639 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:50:02.639 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:02.639 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.639 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:02.639 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6820", 2022-06-29T15:50:02.640 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:50:02.640 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.640 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.640 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:02.641 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6821", 2022-06-29T15:50:02.641 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:50:02.641 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:02.641 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:02.641 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.642 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:50:02.642 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:02.642 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.642 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:02.643 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6824", 2022-06-29T15:50:02.643 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:50:02.643 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.643 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.644 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:02.644 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6825", 2022-06-29T15:50:02.645 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:50:02.645 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:02.646 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:02.646 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.647 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:50:02.647 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:02.647 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.648 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:02.648 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6822", 2022-06-29T15:50:02.649 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:50:02.649 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.649 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.650 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:02.650 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6823", 2022-06-29T15:50:02.651 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:50:02.651 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:02.652 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:02.652 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.652 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6819/13096", 2022-06-29T15:50:02.652 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6821/13096", 2022-06-29T15:50:02.653 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6825/13096", 2022-06-29T15:50:02.653 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6823/13096", 2022-06-29T15:50:02.653 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:50:02.653 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:50:02.653 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:50:02.654 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:02.654 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.654 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.654 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2022-06-29T15:50:02.655 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "228bb08d-2d64-43d9-9560-bfaeff154452", 2022-06-29T15:50:02.655 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:50:02.655 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:50:02.655 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:50:02.656 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:50:02.656 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:50:02.656 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:50:02.656 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:50:02.656 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 99, 2022-06-29T15:50:02.657 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:50:02.657 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:50:02.657 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:50:02.657 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:02.658 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.658 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:02.658 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6826", 2022-06-29T15:50:02.658 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:50:02.659 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.659 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.659 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:02.659 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6827", 2022-06-29T15:50:02.659 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:50:02.660 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:02.660 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:02.660 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.660 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:50:02.661 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:02.661 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.661 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:02.661 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6828", 2022-06-29T15:50:02.662 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:50:02.662 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.662 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.662 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:02.663 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6829", 2022-06-29T15:50:02.663 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:50:02.663 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:02.663 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:02.663 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.664 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:50:02.664 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:02.664 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.664 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:02.665 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6832", 2022-06-29T15:50:02.665 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:50:02.665 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.665 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.665 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:02.666 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6833", 2022-06-29T15:50:02.666 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:50:02.666 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:02.666 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:02.667 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.667 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:50:02.667 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:02.667 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.668 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:02.668 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6830", 2022-06-29T15:50:02.668 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:50:02.668 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.669 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.669 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:02.669 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6831", 2022-06-29T15:50:02.669 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:50:02.669 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:02.670 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:02.670 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.670 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6827/13100", 2022-06-29T15:50:02.670 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6829/13100", 2022-06-29T15:50:02.671 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6833/13100", 2022-06-29T15:50:02.671 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6831/13100", 2022-06-29T15:50:02.671 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:50:02.671 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:50:02.672 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:50:02.672 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:02.672 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.672 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.673 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2022-06-29T15:50:02.673 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "797a5452-1510-4640-a111-c9c2fcf707eb", 2022-06-29T15:50:02.673 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:50:02.673 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:50:02.673 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:50:02.674 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:50:02.674 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:50:02.674 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:50:02.674 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:50:02.675 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 99, 2022-06-29T15:50:02.675 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:50:02.675 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:50:02.675 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:50:02.676 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:02.676 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.676 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:02.676 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6810", 2022-06-29T15:50:02.676 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:50:02.677 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.677 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.677 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:02.677 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6811", 2022-06-29T15:50:02.678 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:50:02.678 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:02.678 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:02.678 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.679 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:50:02.679 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:02.679 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.679 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:02.680 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6812", 2022-06-29T15:50:02.680 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:50:02.680 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.680 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.680 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:02.681 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6813", 2022-06-29T15:50:02.681 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:50:02.681 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:02.681 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:02.682 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.682 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:50:02.682 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:02.682 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.683 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:02.683 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6816", 2022-06-29T15:50:02.683 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:50:02.683 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.684 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.684 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:02.684 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6817", 2022-06-29T15:50:02.684 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:50:02.684 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:02.685 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:02.685 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.685 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:50:02.685 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:02.686 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.686 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:02.686 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6814", 2022-06-29T15:50:02.686 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:50:02.686 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.687 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.687 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:02.687 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6815", 2022-06-29T15:50:02.687 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:50:02.688 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:02.688 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:02.688 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.688 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6811/13098", 2022-06-29T15:50:02.688 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6813/13098", 2022-06-29T15:50:02.689 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6817/13098", 2022-06-29T15:50:02.689 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6815/13098", 2022-06-29T15:50:02.689 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:50:02.689 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:50:02.690 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:50:02.690 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:02.690 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.691 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.691 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2022-06-29T15:50:02.691 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "55b24fd7-bb71-4a01-b3fa-a09caa772a62", 2022-06-29T15:50:02.691 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:50:02.691 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:50:02.692 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:50:02.692 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:50:02.692 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:50:02.692 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:50:02.693 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:50:02.693 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 99, 2022-06-29T15:50:02.693 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:50:02.693 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:50:02.693 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:50:02.694 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:02.694 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.694 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:02.694 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6802", 2022-06-29T15:50:02.695 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:50:02.695 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.695 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.695 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:02.696 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6803", 2022-06-29T15:50:02.696 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:50:02.696 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:02.696 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:02.696 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.697 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:50:02.697 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:02.697 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.697 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:02.698 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6804", 2022-06-29T15:50:02.698 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:50:02.698 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.698 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.698 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:02.699 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6805", 2022-06-29T15:50:02.699 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:50:02.699 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:02.699 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:02.700 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.700 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:50:02.700 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:02.700 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.700 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:02.701 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6808", 2022-06-29T15:50:02.701 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:50:02.701 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.701 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.702 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:02.702 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6809", 2022-06-29T15:50:02.702 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:50:02.702 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:02.703 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:02.703 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.703 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:50:02.703 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:02.704 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.704 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:02.704 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6806", 2022-06-29T15:50:02.704 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:50:02.704 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.705 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.705 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:02.705 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6807", 2022-06-29T15:50:02.705 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:50:02.706 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:02.706 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:02.706 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.706 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6803/13097", 2022-06-29T15:50:02.706 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6805/13097", 2022-06-29T15:50:02.707 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6809/13097", 2022-06-29T15:50:02.707 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6807/13097", 2022-06-29T15:50:02.707 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:50:02.707 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:50:02.708 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:50:02.708 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:02.708 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.708 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.709 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2022-06-29T15:50:02.709 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3", 2022-06-29T15:50:02.709 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:50:02.709 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:50:02.709 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:50:02.710 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:50:02.710 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:50:02.710 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:50:02.710 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:50:02.711 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 98, 2022-06-29T15:50:02.711 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:50:02.711 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:50:02.711 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:50:02.712 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:02.712 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.712 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:02.712 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6816", 2022-06-29T15:50:02.712 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:50:02.713 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.713 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.713 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:02.713 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6817", 2022-06-29T15:50:02.714 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:50:02.714 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:02.714 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:02.714 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.715 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:50:02.715 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:02.715 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.715 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:02.715 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6818", 2022-06-29T15:50:02.716 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:50:02.716 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.716 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.716 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:02.717 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6819", 2022-06-29T15:50:02.717 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:50:02.717 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:02.717 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:02.718 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.718 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:50:02.718 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:02.718 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.718 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:02.719 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6822", 2022-06-29T15:50:02.719 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:50:02.719 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.719 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.720 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:02.720 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6823", 2022-06-29T15:50:02.720 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:50:02.720 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:02.721 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:02.721 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.721 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:50:02.721 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:02.721 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.722 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:02.722 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6820", 2022-06-29T15:50:02.722 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:50:02.722 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.723 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.723 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:02.723 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6821", 2022-06-29T15:50:02.723 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:50:02.723 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:02.724 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:02.724 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.724 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6817/13396", 2022-06-29T15:50:02.724 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6819/13396", 2022-06-29T15:50:02.725 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6823/13396", 2022-06-29T15:50:02.725 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6821/13396", 2022-06-29T15:50:02.725 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:50:02.725 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:50:02.726 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:50:02.726 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:02.726 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.726 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.727 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2022-06-29T15:50:02.727 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "f77beebe-6cca-4244-b2d4-677825ebf919", 2022-06-29T15:50:02.727 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:50:02.727 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:50:02.728 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:50:02.728 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:50:02.728 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:50:02.728 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:50:02.729 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:50:02.729 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 88, 2022-06-29T15:50:02.729 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:50:02.729 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:50:02.729 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:50:02.730 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:02.730 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.730 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:02.730 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6800", 2022-06-29T15:50:02.731 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:50:02.731 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.731 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.731 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:02.732 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6801", 2022-06-29T15:50:02.732 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:50:02.732 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:02.732 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:02.732 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.733 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:50:02.733 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:02.733 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.733 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:02.734 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6802", 2022-06-29T15:50:02.734 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:50:02.734 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.734 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.735 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:02.735 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6803", 2022-06-29T15:50:02.735 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:50:02.735 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:02.736 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:02.736 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.736 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:50:02.736 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:02.736 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.737 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:02.737 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6806", 2022-06-29T15:50:02.737 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:50:02.737 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.738 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.738 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:02.738 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6807", 2022-06-29T15:50:02.739 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:50:02.739 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:02.739 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:02.739 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.740 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:50:02.740 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:02.740 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.740 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:02.741 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6804", 2022-06-29T15:50:02.741 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:50:02.741 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.741 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.742 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:02.742 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6805", 2022-06-29T15:50:02.742 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:50:02.742 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:02.743 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:02.743 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.743 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6801/13393", 2022-06-29T15:50:02.743 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6803/13393", 2022-06-29T15:50:02.743 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6807/13393", 2022-06-29T15:50:02.744 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6805/13393", 2022-06-29T15:50:02.744 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:50:02.744 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:50:02.744 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:50:02.745 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:02.745 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.745 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.745 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2022-06-29T15:50:02.746 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "af0ca9b3-c02c-4910-834b-bf0b8dc6713c", 2022-06-29T15:50:02.746 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:50:02.746 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:50:02.746 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:50:02.746 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:50:02.747 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:50:02.747 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:50:02.747 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:50:02.747 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 98, 2022-06-29T15:50:02.748 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:50:02.748 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:50:02.748 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:50:02.748 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:02.749 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.749 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:02.749 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6824", 2022-06-29T15:50:02.749 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:50:02.750 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.750 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.750 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:02.750 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6825", 2022-06-29T15:50:02.751 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:50:02.751 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:02.751 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:02.751 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.751 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:50:02.752 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:02.752 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.752 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:02.752 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6826", 2022-06-29T15:50:02.753 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:50:02.753 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.753 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.753 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:02.754 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6827", 2022-06-29T15:50:02.754 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:50:02.754 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:02.754 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:02.755 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.755 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:50:02.755 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:02.755 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.755 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:02.756 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6830", 2022-06-29T15:50:02.756 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:50:02.756 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.756 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.757 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:02.757 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6831", 2022-06-29T15:50:02.757 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:50:02.757 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:02.757 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:02.758 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.758 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:50:02.758 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:02.758 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.759 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:02.759 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6828", 2022-06-29T15:50:02.759 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:50:02.759 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.760 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.760 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:02.760 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6829", 2022-06-29T15:50:02.760 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:50:02.760 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:02.761 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:02.761 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.761 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6825/13395", 2022-06-29T15:50:02.761 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6827/13395", 2022-06-29T15:50:02.762 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6831/13395", 2022-06-29T15:50:02.762 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6829/13395", 2022-06-29T15:50:02.762 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:50:02.762 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:50:02.762 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:50:02.763 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:02.763 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.763 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.763 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2022-06-29T15:50:02.764 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "f19c3ca6-df88-4bde-a150-b51a53f092e1", 2022-06-29T15:50:02.764 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:50:02.764 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:50:02.764 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:50:02.764 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:50:02.765 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:50:02.765 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:50:02.765 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:50:02.765 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 99, 2022-06-29T15:50:02.766 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:50:02.766 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:50:02.766 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:50:02.766 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:02.767 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.767 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:02.767 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6808", 2022-06-29T15:50:02.767 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:50:02.767 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.768 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.768 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:02.768 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6809", 2022-06-29T15:50:02.768 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:50:02.769 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:02.769 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:02.770 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.770 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:50:02.771 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:02.771 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.772 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:02.772 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6810", 2022-06-29T15:50:02.773 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:50:02.773 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.773 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.774 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:02.774 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6811", 2022-06-29T15:50:02.775 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:50:02.775 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:02.775 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:02.776 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.776 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:50:02.777 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:02.777 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.777 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:02.777 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6814", 2022-06-29T15:50:02.778 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:50:02.778 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.778 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.778 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:02.779 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6815", 2022-06-29T15:50:02.779 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:50:02.779 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:02.779 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:02.780 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.780 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:50:02.780 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:02.780 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.781 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:02.781 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6812", 2022-06-29T15:50:02.781 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:50:02.781 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.782 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.782 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:02.782 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6813", 2022-06-29T15:50:02.782 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:50:02.783 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:02.783 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:02.783 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.783 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6809/13394", 2022-06-29T15:50:02.784 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6811/13394", 2022-06-29T15:50:02.784 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6815/13394", 2022-06-29T15:50:02.784 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6813/13394", 2022-06-29T15:50:02.784 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:50:02.784 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:50:02.785 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:50:02.785 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:02.785 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:02.785 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:50:02.786 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_xinfo": [ 2022-06-29T15:50:02.786 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.786 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2022-06-29T15:50:02.786 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:50:02.787 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:50:02.787 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:50:02.787 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:50:02.787 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:50:02.788 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.720513+0000", 2022-06-29T15:50:02.788 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:50:02.788 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.788 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.789 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2022-06-29T15:50:02.789 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:50:02.789 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:50:02.789 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:50:02.789 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:50:02.790 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:50:02.790 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.772612+0000", 2022-06-29T15:50:02.790 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:50:02.791 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.791 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.791 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2022-06-29T15:50:02.791 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:50:02.792 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:50:02.792 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:50:02.792 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:50:02.792 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:50:02.792 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.450223+0000", 2022-06-29T15:50:02.793 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:50:02.793 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.793 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.793 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2022-06-29T15:50:02.794 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:50:02.794 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:50:02.794 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:50:02.794 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:50:02.795 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:50:02.795 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.477856+0000", 2022-06-29T15:50:02.795 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:50:02.795 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.796 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.796 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2022-06-29T15:50:02.796 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:50:02.796 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:50:02.797 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:50:02.797 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:50:02.797 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:50:02.797 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.539485+0000", 2022-06-29T15:50:02.797 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:50:02.798 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.798 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.798 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2022-06-29T15:50:02.798 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:50:02.799 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:50:02.799 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:50:02.799 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:50:02.799 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:50:02.800 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.487040+0000", 2022-06-29T15:50:02.800 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:50:02.800 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.800 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.800 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2022-06-29T15:50:02.801 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:50:02.801 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:50:02.801 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:50:02.801 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:50:02.802 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:50:02.802 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.748685+0000", 2022-06-29T15:50:02.802 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:50:02.803 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.803 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:02.803 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2022-06-29T15:50:02.803 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:50:02.804 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:50:02.804 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:50:02.804 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:50:02.804 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:50:02.805 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.459516+0000", 2022-06-29T15:50:02.805 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:50:02.805 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:02.805 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:50:02.806 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap": [], 2022-06-29T15:50:02.806 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap_items": [], 2022-06-29T15:50:02.806 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_temp": [], 2022-06-29T15:50:02.806 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_temp": [], 2022-06-29T15:50:02.807 INFO:teuthology.orchestra.run.smithi047.stdout: "blacklist": {}, 2022-06-29T15:50:02.807 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profiles": { 2022-06-29T15:50:02.807 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs_data": { 2022-06-29T15:50:02.807 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-device-class": "", 2022-06-29T15:50:02.807 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2022-06-29T15:50:02.808 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-root": "default", 2022-06-29T15:50:02.808 INFO:teuthology.orchestra.run.smithi047.stdout: "jerasure-per-chunk-alignment": "false", 2022-06-29T15:50:02.808 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2022-06-29T15:50:02.809 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "2", 2022-06-29T15:50:02.809 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2022-06-29T15:50:02.810 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2022-06-29T15:50:02.810 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van", 2022-06-29T15:50:02.811 INFO:teuthology.orchestra.run.smithi047.stdout: "w": "8" 2022-06-29T15:50:02.811 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.811 INFO:teuthology.orchestra.run.smithi047.stdout: "default": { 2022-06-29T15:50:02.812 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2022-06-29T15:50:02.812 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2022-06-29T15:50:02.813 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "1", 2022-06-29T15:50:02.813 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2022-06-29T15:50:02.814 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2022-06-29T15:50:02.814 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van" 2022-06-29T15:50:02.815 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:02.815 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:02.815 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps_queue": [], 2022-06-29T15:50:02.816 INFO:teuthology.orchestra.run.smithi047.stdout: "new_removed_snaps": [], 2022-06-29T15:50:02.816 INFO:teuthology.orchestra.run.smithi047.stdout: "new_purged_snaps": [], 2022-06-29T15:50:02.817 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_node_flags": {}, 2022-06-29T15:50:02.817 INFO:teuthology.orchestra.run.smithi047.stdout: "device_class_flags": {} 2022-06-29T15:50:02.818 INFO:teuthology.orchestra.run.smithi047.stdout:} 2022-06-29T15:50:02.821 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.544+0000 7f1ad13a8700 1 -- 172.21.15.47:0/926727433 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1ab0053a10 msgr2=0x7f1ab0055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:02.821 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.544+0000 7f1ad13a8700 1 --2- 172.21.15.47:0/926727433 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1ab0053a10 0x7f1ab0055ed0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x7f1ab400c000 tx=0x7f1ab4008040).stop 2022-06-29T15:50:02.822 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.544+0000 7f1ad13a8700 1 -- 172.21.15.47:0/926727433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1acc060eb0 msgr2=0x7f1acc14d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:02.822 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.544+0000 7f1ad13a8700 1 --2- 172.21.15.47:0/926727433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1acc060eb0 0x7f1acc14d010 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 rx=0x7f1acc0aeae0 tx=0x7f1abc015000).stop 2022-06-29T15:50:02.822 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.544+0000 7f1ad13a8700 1 -- 172.21.15.47:0/926727433 shutdown_connections 2022-06-29T15:50:02.822 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.544+0000 7f1ad13a8700 1 --2- 172.21.15.47:0/926727433 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1ab0053a10 0x7f1ab0055ed0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:02.823 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.544+0000 7f1ad13a8700 1 --2- 172.21.15.47:0/926727433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1acc0b5c60 0x7f1acc0ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:02.823 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.544+0000 7f1ad13a8700 1 --2- 172.21.15.47:0/926727433 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1acc0a72f0 0x7f1acc0adfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:02.823 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.544+0000 7f1ad13a8700 1 --2- 172.21.15.47:0/926727433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1acc060eb0 0x7f1acc14d010 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:02.823 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.544+0000 7f1ad13a8700 1 -- 172.21.15.47:0/926727433 >> 172.21.15.47:0/926727433 conn(0x7f1acc062560 msgr2=0x7f1acc141360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:02.824 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.544+0000 7f1ad13a8700 1 -- 172.21.15.47:0/926727433 shutdown_connections 2022-06-29T15:50:02.824 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.544+0000 7f1ad13a8700 1 -- 172.21.15.47:0/926727433 wait complete. 2022-06-29T15:50:02.825 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:50:02.938 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.932+0000 7f9a5f59e700 1 Processor -- start 2022-06-29T15:50:02.939 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.932+0000 7f9a5f59e700 1 -- start start 2022-06-29T15:50:02.942 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.932+0000 7f9a5f59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9a6005c510 0x7f9a60143bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:02.942 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.932+0000 7f9a5f59e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9a600b2c40 0x7f9a600b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:02.942 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.932+0000 7f9a5f59e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a60142f90 con 0x7f9a6005c8f0 2022-06-29T15:50:02.942 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.932+0000 7f9a5f59e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a601431f0 con 0x7f9a6005c510 2022-06-29T15:50:02.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.932+0000 7f9a5f59e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a60143490 con 0x7f9a600b2c40 2022-06-29T15:50:02.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.932+0000 7f9a5e59c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9a6005c510 0x7f9a60143bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:02.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.932+0000 7f9a5ed9d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9a600b2c40 0x7f9a600b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:02.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.932+0000 7f9a5e59c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9a6005c510 0x7f9a60143bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:57418/0 (socket says 172.21.15.47:57418) 2022-06-29T15:50:02.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.932+0000 7f9a5e59c700 1 -- 172.21.15.47:0/808466107 learned_addr learned my addr 172.21.15.47:0/808466107 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:02.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.932+0000 7f9a5e59c700 1 -- 172.21.15.47:0/808466107 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9a600b2c40 msgr2=0x7f9a600b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:02.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.932+0000 7f9a5e59c700 1 --2- 172.21.15.47:0/808466107 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9a600b2c40 0x7f9a600b30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:02.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.932+0000 7f9a5e59c700 1 -- 172.21.15.47:0/808466107 >> v1:172.21.15.64:6789/0 conn(0x7f9a6005c8f0 legacy=0x7f9a600b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:02.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.932+0000 7f9a5d59a700 1 -- 172.21.15.47:0/808466107 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4157360135 0 0) 0x7f9a60142f90 con 0x7f9a6005c8f0 2022-06-29T15:50:02.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.932+0000 7f9a5e59c700 1 -- 172.21.15.47:0/808466107 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a60143730 con 0x7f9a6005c510 2022-06-29T15:50:02.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.932+0000 7f9a5e59c700 1 --2- 172.21.15.47:0/808466107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a6005c510 0x7f9a60143bd0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f9a500032f0 tx=0x7f9a50009290).ready entity=mon.1 client_cookie=da6a911b2b1274fb server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:02.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.932+0000 7f9a5d59a700 1 -- 172.21.15.47:0/808466107 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a5000b8a0 con 0x7f9a6005c510 2022-06-29T15:50:02.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.932+0000 7f9a5d59a700 1 -- 172.21.15.47:0/808466107 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a5000ba00 con 0x7f9a6005c510 2022-06-29T15:50:02.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.932+0000 7f9a5d59a700 1 -- 172.21.15.47:0/808466107 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a5000bb70 con 0x7f9a6005c510 2022-06-29T15:50:02.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.932+0000 7f9a5f59e700 1 -- 172.21.15.47:0/808466107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a6005c510 msgr2=0x7f9a60143bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:02.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.932+0000 7f9a5f59e700 1 --2- 172.21.15.47:0/808466107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a6005c510 0x7f9a60143bd0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f9a500032f0 tx=0x7f9a50009290).stop 2022-06-29T15:50:02.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.932+0000 7f9a5f59e700 1 -- 172.21.15.47:0/808466107 shutdown_connections 2022-06-29T15:50:02.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.932+0000 7f9a5f59e700 1 --2- 172.21.15.47:0/808466107 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9a600b2c40 0x7f9a600b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:02.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.932+0000 7f9a5f59e700 1 --2- 172.21.15.47:0/808466107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a6005c510 0x7f9a60143bd0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:02.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.932+0000 7f9a5f59e700 1 -- 172.21.15.47:0/808466107 >> 172.21.15.47:0/808466107 conn(0x7f9a60060e00 msgr2=0x7f9a600611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:02.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.932+0000 7f9a5f59e700 1 -- 172.21.15.47:0/808466107 shutdown_connections 2022-06-29T15:50:02.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.932+0000 7f9a5f59e700 1 -- 172.21.15.47:0/808466107 wait complete. 2022-06-29T15:50:02.951 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.932+0000 7f9a5f59e700 1 Processor -- start 2022-06-29T15:50:02.951 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.932+0000 7f9a5f59e700 1 -- start start 2022-06-29T15:50:02.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.936+0000 7f9a5f59e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9a6005c510 0x7f9a600b8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:02.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.936+0000 7f9a5f59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a6005c8f0 0x7f9a600af470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:02.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.936+0000 7f9a5f59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a600b2c40 0x7f9a60152cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:02.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.936+0000 7f9a5f59e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6013ffb0 con 0x7f9a600b2c40 2022-06-29T15:50:02.953 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.936+0000 7f9a5f59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a600af960 con 0x7f9a6005c8f0 2022-06-29T15:50:02.953 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.936+0000 7f9a5f59e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a600ae940 con 0x7f9a6005c510 2022-06-29T15:50:02.953 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.936+0000 7f9a5ed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a600b2c40 0x7f9a60152cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:02.953 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.936+0000 7f9a5ed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a600b2c40 0x7f9a60152cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51722/0 (socket says 172.21.15.47:51722) 2022-06-29T15:50:02.954 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.936+0000 7f9a5ed9d700 1 -- 172.21.15.47:0/86090113 learned_addr learned my addr 172.21.15.47:0/86090113 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:02.954 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.936+0000 7f9a5e59c700 1 --2- 172.21.15.47:0/86090113 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9a6005c510 0x7f9a600b8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:02.954 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.936+0000 7f9a5dd9b700 1 --2- 172.21.15.47:0/86090113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a6005c8f0 0x7f9a600af470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:02.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.936+0000 7f9a5ed9d700 1 -- 172.21.15.47:0/86090113 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9a6005c510 msgr2=0x7f9a600b8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:02.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.936+0000 7f9a5ed9d700 1 --2- 172.21.15.47:0/86090113 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9a6005c510 0x7f9a600b8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:02.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.936+0000 7f9a5ed9d700 1 -- 172.21.15.47:0/86090113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a6005c8f0 msgr2=0x7f9a600af470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:02.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.936+0000 7f9a5ed9d700 1 --2- 172.21.15.47:0/86090113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a6005c8f0 0x7f9a600af470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:02.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.936+0000 7f9a5ed9d700 1 -- 172.21.15.47:0/86090113 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a50009530 con 0x7f9a600b2c40 2022-06-29T15:50:02.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.936+0000 7f9a5ed9d700 1 --2- 172.21.15.47:0/86090113 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a600b2c40 0x7f9a60152cb0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 rx=0x7f9a5800c000 tx=0x7f9a58006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:02.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.936+0000 7f9a577fe700 1 -- 172.21.15.47:0/86090113 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a58011750 con 0x7f9a600b2c40 2022-06-29T15:50:02.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.936+0000 7f9a577fe700 1 -- 172.21.15.47:0/86090113 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a580118b0 con 0x7f9a600b2c40 2022-06-29T15:50:02.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.936+0000 7f9a5f59e700 1 -- 172.21.15.47:0/86090113 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a601401a0 con 0x7f9a600b2c40 2022-06-29T15:50:02.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.936+0000 7f9a577fe700 1 -- 172.21.15.47:0/86090113 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a58011c10 con 0x7f9a600b2c40 2022-06-29T15:50:02.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.936+0000 7f9a5f59e700 1 -- 172.21.15.47:0/86090113 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a60152570 con 0x7f9a600b2c40 2022-06-29T15:50:02.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.940+0000 7f9a5f59e700 1 -- 172.21.15.47:0/86090113 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a600573d0 con 0x7f9a600b2c40 2022-06-29T15:50:02.958 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.940+0000 7f9a577fe700 1 -- 172.21.15.47:0/86090113 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9a58011d70 con 0x7f9a600b2c40 2022-06-29T15:50:02.958 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.940+0000 7f9a577fe700 1 --2- 172.21.15.47:0/86090113 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9a38053aa0 0x7f9a38055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:02.958 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.940+0000 7f9a577fe700 1 -- 172.21.15.47:0/86090113 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9a5806ced0 con 0x7f9a600b2c40 2022-06-29T15:50:02.958 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.940+0000 7f9a5e59c700 1 --2- 172.21.15.47:0/86090113 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9a38053aa0 0x7f9a38055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:02.959 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.940+0000 7f9a5e59c700 1 --2- 172.21.15.47:0/86090113 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9a38053aa0 0x7f9a38055f60 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7f9a5001b000 tx=0x7f9a50016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:02.959 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:02.944+0000 7f9a577fe700 1 -- 172.21.15.47:0/86090113 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9a58043c90 con 0x7f9a600b2c40 2022-06-29T15:50:03.121 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.112+0000 7f9a5f59e700 1 -- 172.21.15.47:0/86090113 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a60061b00 con 0x7f9a600b2c40 2022-06-29T15:50:03.122 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.112+0000 7f9a577fe700 1 -- 172.21.15.47:0/86090113 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+2493 (secure 0 0 0) 0x7f9a580472b0 con 0x7f9a600b2c40 2022-06-29T15:50:03.123 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:50:03.123 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":45,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6969,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3642031880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3642031880},{"type":"v1","addr":"172.21.15.47:6835","nonce":3642031880}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":43},{"gid":6984,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/891572891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":891572891},{"type":"v1","addr":"172.21.15.47:6837","nonce":891572891}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":43}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:49:50.788599+0000","modified":"2022-06-29T15:49:51.791379+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":56102},"failed":[],"damaged":[],"stopped":[],"info":{"gid_56102":{"gid":56102,"name":"b","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6833/3819760998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3819760998},{"type":"v1","addr":"172.21.15.64:6833","nonce":3819760998}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2022-06-29T15:50:03.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.116+0000 7f9a5f59e700 1 -- 172.21.15.47:0/86090113 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9a38053aa0 msgr2=0x7f9a38055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:03.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.116+0000 7f9a5f59e700 1 --2- 172.21.15.47:0/86090113 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9a38053aa0 0x7f9a38055f60 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7f9a5001b000 tx=0x7f9a50016000).stop 2022-06-29T15:50:03.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.116+0000 7f9a5f59e700 1 -- 172.21.15.47:0/86090113 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a600b2c40 msgr2=0x7f9a60152cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:03.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.116+0000 7f9a5f59e700 1 --2- 172.21.15.47:0/86090113 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a600b2c40 0x7f9a60152cb0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 rx=0x7f9a5800c000 tx=0x7f9a58006df0).stop 2022-06-29T15:50:03.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.116+0000 7f9a5f59e700 1 -- 172.21.15.47:0/86090113 shutdown_connections 2022-06-29T15:50:03.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.116+0000 7f9a5f59e700 1 --2- 172.21.15.47:0/86090113 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a600b2c40 0x7f9a60152cb0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:03.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.116+0000 7f9a5f59e700 1 --2- 172.21.15.47:0/86090113 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9a38053aa0 0x7f9a38055f60 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:03.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.116+0000 7f9a5f59e700 1 --2- 172.21.15.47:0/86090113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a6005c8f0 0x7f9a600af470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:03.127 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.116+0000 7f9a5f59e700 1 --2- 172.21.15.47:0/86090113 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9a6005c510 0x7f9a600b8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:03.127 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.116+0000 7f9a5f59e700 1 -- 172.21.15.47:0/86090113 >> 172.21.15.47:0/86090113 conn(0x7f9a60060e00 msgr2=0x7f9a60141aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:03.127 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.116+0000 7f9a5f59e700 1 -- 172.21.15.47:0/86090113 shutdown_connections 2022-06-29T15:50:03.127 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.116+0000 7f9a5f59e700 1 -- 172.21.15.47:0/86090113 wait complete. 2022-06-29T15:50:03.128 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 45 2022-06-29T15:50:03.137 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-06-29T15:50:03.241 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.236+0000 7f23bb3f0700 1 Processor -- start 2022-06-29T15:50:03.241 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.236+0000 7f23bb3f0700 1 -- start start 2022-06-29T15:50:03.250 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.236+0000 7f23bb3f0700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f23b4060eb0 0x7f23b40b1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:03.251 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.236+0000 7f23bb3f0700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f23b4145090 0x7f23b40b2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:03.251 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.236+0000 7f23bb3f0700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23b40a9120 con 0x7f23b40b5c60 2022-06-29T15:50:03.251 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.236+0000 7f23bb3f0700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23b40b0b80 con 0x7f23b4145090 2022-06-29T15:50:03.251 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.236+0000 7f23bb3f0700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23b40a89e0 con 0x7f23b4060eb0 2022-06-29T15:50:03.252 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.236+0000 7f23babef700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f23b4145090 0x7f23b40b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:03.252 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.236+0000 7f23babef700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f23b4145090 0x7f23b40b2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51730/0 (socket says 172.21.15.47:51730) 2022-06-29T15:50:03.252 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.236+0000 7f23babef700 1 -- 172.21.15.47:0/2979224975 learned_addr learned my addr 172.21.15.47:0/2979224975 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:03.252 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.236+0000 7f23b9bed700 1 --2- 172.21.15.47:0/2979224975 >> v2:172.21.15.64:3300/0 conn(0x7f23b4060eb0 0x7f23b40b1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:03.253 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.236+0000 7f23babef700 1 -- 172.21.15.47:0/2979224975 >> v2:172.21.15.64:3300/0 conn(0x7f23b4060eb0 msgr2=0x7f23b40b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:03.253 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.236+0000 7f23babef700 1 --2- 172.21.15.47:0/2979224975 >> v2:172.21.15.64:3300/0 conn(0x7f23b4060eb0 0x7f23b40b1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:03.253 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.236+0000 7f23babef700 1 -- 172.21.15.47:0/2979224975 >> v1:172.21.15.64:6789/0 conn(0x7f23b40b5c60 legacy=0x7f23b4144980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:03.253 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.236+0000 7f23babef700 1 -- 172.21.15.47:0/2979224975 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b40a8c80 con 0x7f23b4145090 2022-06-29T15:50:03.254 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.236+0000 7f23babef700 1 --2- 172.21.15.47:0/2979224975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23b4145090 0x7f23b40b2320 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 rx=0x7f23ac002f70 tx=0x7f23ac006eb0).ready entity=mon.0 client_cookie=b9e0eeefd54cad3b server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:03.254 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.236+0000 7f23b93ec700 1 -- 172.21.15.47:0/2979224975 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23ac00d5c0 con 0x7f23b4145090 2022-06-29T15:50:03.254 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.236+0000 7f23b93ec700 1 -- 172.21.15.47:0/2979224975 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23ac000e90 con 0x7f23b4145090 2022-06-29T15:50:03.254 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.236+0000 7f23b93ec700 1 -- 172.21.15.47:0/2979224975 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23ac00d950 con 0x7f23b4145090 2022-06-29T15:50:03.254 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.236+0000 7f23bb3f0700 1 -- 172.21.15.47:0/2979224975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23b4145090 msgr2=0x7f23b40b2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:03.255 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.236+0000 7f23bb3f0700 1 --2- 172.21.15.47:0/2979224975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23b4145090 0x7f23b40b2320 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 rx=0x7f23ac002f70 tx=0x7f23ac006eb0).stop 2022-06-29T15:50:03.255 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.236+0000 7f23bb3f0700 1 -- 172.21.15.47:0/2979224975 shutdown_connections 2022-06-29T15:50:03.255 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.236+0000 7f23bb3f0700 1 --2- 172.21.15.47:0/2979224975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23b4145090 0x7f23b40b2320 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:03.255 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.236+0000 7f23bb3f0700 1 --2- 172.21.15.47:0/2979224975 >> v2:172.21.15.64:3300/0 conn(0x7f23b4060eb0 0x7f23b40b1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:03.256 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.236+0000 7f23bb3f0700 1 -- 172.21.15.47:0/2979224975 >> 172.21.15.47:0/2979224975 conn(0x7f23b4062560 msgr2=0x7f23b4062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:03.256 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.236+0000 7f23bb3f0700 1 -- 172.21.15.47:0/2979224975 shutdown_connections 2022-06-29T15:50:03.256 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.236+0000 7f23bb3f0700 1 -- 172.21.15.47:0/2979224975 wait complete. 2022-06-29T15:50:03.256 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.236+0000 7f23bb3f0700 1 Processor -- start 2022-06-29T15:50:03.257 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.236+0000 7f23bb3f0700 1 -- start start 2022-06-29T15:50:03.257 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.236+0000 7f23bb3f0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23b4060eb0 0x7f23b4141af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:03.257 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.236+0000 7f23bb3f0700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f23b40b5c60 0x7f23b414a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:03.257 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.236+0000 7f23bb3f0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23b4145090 0x7f23b414a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:03.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.236+0000 7f23bb3f0700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b40ac860 con 0x7f23b4145090 2022-06-29T15:50:03.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.236+0000 7f23bb3f0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b40ae200 con 0x7f23b4060eb0 2022-06-29T15:50:03.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.236+0000 7f23bb3f0700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23b40accd0 con 0x7f23b40b5c60 2022-06-29T15:50:03.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.236+0000 7f23babef700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23b4145090 0x7f23b414a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:03.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.236+0000 7f23babef700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23b4145090 0x7f23b414a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51736/0 (socket says 172.21.15.47:51736) 2022-06-29T15:50:03.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.236+0000 7f23babef700 1 -- 172.21.15.47:0/2065422296 learned_addr learned my addr 172.21.15.47:0/2065422296 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:03.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.236+0000 7f23b9bed700 1 --2- 172.21.15.47:0/2065422296 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f23b40b5c60 0x7f23b414a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:03.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.236+0000 7f23ba3ee700 1 --2- 172.21.15.47:0/2065422296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23b4060eb0 0x7f23b4141af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:03.260 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.236+0000 7f23babef700 1 -- 172.21.15.47:0/2065422296 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f23b40b5c60 msgr2=0x7f23b414a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:03.260 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.236+0000 7f23babef700 1 --2- 172.21.15.47:0/2065422296 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f23b40b5c60 0x7f23b414a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:03.260 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.236+0000 7f23babef700 1 -- 172.21.15.47:0/2065422296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23b4060eb0 msgr2=0x7f23b4141af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:03.261 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.236+0000 7f23babef700 1 --2- 172.21.15.47:0/2065422296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23b4060eb0 0x7f23b4141af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:03.261 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.236+0000 7f23babef700 1 -- 172.21.15.47:0/2065422296 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23ac00b1e0 con 0x7f23b4145090 2022-06-29T15:50:03.261 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.236+0000 7f23babef700 1 --2- 172.21.15.47:0/2065422296 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23b4145090 0x7f23b414a650 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 rx=0x7f23ac002f70 tx=0x7f23ac015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:03.262 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.236+0000 7f23a37fe700 1 -- 172.21.15.47:0/2065422296 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23ac011610 con 0x7f23b4145090 2022-06-29T15:50:03.262 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.236+0000 7f23a37fe700 1 -- 172.21.15.47:0/2065422296 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23ac011770 con 0x7f23b4145090 2022-06-29T15:50:03.263 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.236+0000 7f23bb3f0700 1 -- 172.21.15.47:0/2065422296 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23b4148b00 con 0x7f23b4145090 2022-06-29T15:50:03.263 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.236+0000 7f23a37fe700 1 -- 172.21.15.47:0/2065422296 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23ac011ad0 con 0x7f23b4145090 2022-06-29T15:50:03.263 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.236+0000 7f23bb3f0700 1 -- 172.21.15.47:0/2065422296 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23b414ab40 con 0x7f23b4145090 2022-06-29T15:50:03.264 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.240+0000 7f23bb3f0700 1 -- 172.21.15.47:0/2065422296 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23b40573d0 con 0x7f23b4145090 2022-06-29T15:50:03.264 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.240+0000 7f23a37fe700 1 -- 172.21.15.47:0/2065422296 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f23ac025030 con 0x7f23b4145090 2022-06-29T15:50:03.264 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.240+0000 7f23a37fe700 1 --2- 172.21.15.47:0/2065422296 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2394053b30 0x7f2394055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:03.265 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.240+0000 7f23a37fe700 1 -- 172.21.15.47:0/2065422296 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f23ac06cf80 con 0x7f23b4145090 2022-06-29T15:50:03.265 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.240+0000 7f23ba3ee700 1 --2- 172.21.15.47:0/2065422296 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2394053b30 0x7f2394055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:03.266 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.240+0000 7f23ba3ee700 1 --2- 172.21.15.47:0/2065422296 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2394053b30 0x7f2394055ff0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7f23a400b000 tx=0x7f23a4009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:03.267 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.244+0000 7f23a37fe700 1 -- 172.21.15.47:0/2065422296 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f23ac043d40 con 0x7f23b4145090 2022-06-29T15:50:03.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:03.408+0000 7f23bb3f0700 1 -- 172.21.15.47:0/2065422296 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f23b4148950 con 0x7f23b4145090 2022-06-29T15:50:04.144 INFO:tasks.ceph.mon.a.smithi047.stderr:2022-06-29T15:50:04.140+0000 7ff9cf83c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-06-29T15:50:04.158 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.152+0000 7f23a37fe700 1 -- 172.21.15.47:0/2065422296 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v46) v1 ==== 161+0+0 (secure 0 0 0) 0x7f23ac023020 con 0x7f23b4145090 2022-06-29T15:50:04.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.156+0000 7f23bb3f0700 1 -- 172.21.15.47:0/2065422296 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2394053b30 msgr2=0x7f2394055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:04.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.156+0000 7f23bb3f0700 1 --2- 172.21.15.47:0/2065422296 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2394053b30 0x7f2394055ff0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7f23a400b000 tx=0x7f23a4009040).stop 2022-06-29T15:50:04.161 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.156+0000 7f23bb3f0700 1 -- 172.21.15.47:0/2065422296 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23b4145090 msgr2=0x7f23b414a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:04.161 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.156+0000 7f23bb3f0700 1 --2- 172.21.15.47:0/2065422296 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23b4145090 0x7f23b414a650 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 rx=0x7f23ac002f70 tx=0x7f23ac015000).stop 2022-06-29T15:50:04.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.156+0000 7f23bb3f0700 1 -- 172.21.15.47:0/2065422296 shutdown_connections 2022-06-29T15:50:04.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.156+0000 7f23bb3f0700 1 --2- 172.21.15.47:0/2065422296 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23b4145090 0x7f23b414a650 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:04.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.156+0000 7f23bb3f0700 1 --2- 172.21.15.47:0/2065422296 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f23b40b5c60 0x7f23b414a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:04.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.156+0000 7f23bb3f0700 1 --2- 172.21.15.47:0/2065422296 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2394053b30 0x7f2394055ff0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:04.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.156+0000 7f23bb3f0700 1 --2- 172.21.15.47:0/2065422296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23b4060eb0 0x7f23b4141af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:04.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.156+0000 7f23bb3f0700 1 -- 172.21.15.47:0/2065422296 >> 172.21.15.47:0/2065422296 conn(0x7f23b4062560 msgr2=0x7f23b40b4b50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:04.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.156+0000 7f23bb3f0700 1 -- 172.21.15.47:0/2065422296 shutdown_connections 2022-06-29T15:50:04.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.156+0000 7f23bb3f0700 1 -- 172.21.15.47:0/2065422296 wait complete. 2022-06-29T15:50:04.164 INFO:teuthology.orchestra.run.smithi047.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-06-29T15:50:04.177 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:50:04.186 INFO:tasks.ceph.mds.b.smithi064.stdout:starting mds.b at 2022-06-29T15:50:04.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.280+0000 7ff03759e700 1 Processor -- start 2022-06-29T15:50:04.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.280+0000 7ff03759e700 1 -- start start 2022-06-29T15:50:04.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.284+0000 7ff03759e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff03805c8f0 0x7ff03813cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:04.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.284+0000 7ff03759e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff038142f90 con 0x7ff038143bd0 2022-06-29T15:50:04.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.284+0000 7ff03759e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0381431f0 con 0x7ff03805c510 2022-06-29T15:50:04.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.284+0000 7ff03759e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff038143490 con 0x7ff03805c8f0 2022-06-29T15:50:04.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.284+0000 7ff035d9b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff038143bd0 0x7ff0380b2c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37294/0 (socket says 172.21.15.47:37294) 2022-06-29T15:50:04.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.284+0000 7ff035d9b700 1 -- 172.21.15.47:0/3283288816 learned_addr learned my addr 172.21.15.47:0/3283288816 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:04.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.284+0000 7ff036d9d700 1 --2- 172.21.15.47:0/3283288816 >> v2:172.21.15.64:3300/0 conn(0x7ff03805c8f0 0x7ff03813cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:04.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.284+0000 7ff03559a700 1 -- 172.21.15.47:0/3283288816 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3723374322 0 0) 0x7ff038142f90 con 0x7ff038143bd0 2022-06-29T15:50:04.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.284+0000 7ff03559a700 1 -- 172.21.15.47:0/3283288816 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff01c003260 con 0x7ff038143bd0 2022-06-29T15:50:04.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.284+0000 7ff03559a700 1 -- 172.21.15.47:0/3283288816 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2957469584 0 0) 0x7ff0381431f0 con 0x7ff03805c510 2022-06-29T15:50:04.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.284+0000 7ff03559a700 1 -- 172.21.15.47:0/3283288816 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff038142f90 con 0x7ff03805c510 2022-06-29T15:50:04.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.284+0000 7ff03559a700 1 -- 172.21.15.47:0/3283288816 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3545703358 0 0) 0x7ff01c003260 con 0x7ff038143bd0 2022-06-29T15:50:04.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.284+0000 7ff03559a700 1 -- 172.21.15.47:0/3283288816 >> v2:172.21.15.64:3300/0 conn(0x7ff03805c8f0 msgr2=0x7ff03813cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:04.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.284+0000 7ff03559a700 1 --2- 172.21.15.47:0/3283288816 >> v2:172.21.15.64:3300/0 conn(0x7ff03805c8f0 0x7ff03813cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:04.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.284+0000 7ff03559a700 1 -- 172.21.15.47:0/3283288816 >> v1:172.21.15.64:6789/0 conn(0x7ff03805c510 legacy=0x7ff0380b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:04.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.284+0000 7ff03559a700 1 -- 172.21.15.47:0/3283288816 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff038143730 con 0x7ff038143bd0 2022-06-29T15:50:04.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.284+0000 7ff03559a700 1 -- 172.21.15.47:0/3283288816 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7ff02c001680 con 0x7ff038143bd0 2022-06-29T15:50:04.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.284+0000 7ff03559a700 1 -- 172.21.15.47:0/3283288816 >> v1:172.21.15.47:6789/0 conn(0x7ff038143bd0 legacy=0x7ff0380b2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:04.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.284+0000 7ff03559a700 1 --2- 172.21.15.47:0/3283288816 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff01c003930 0x7ff01c003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:04.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.284+0000 7ff03559a700 1 --2- 172.21.15.47:0/3283288816 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff01c004500 0x7ff01c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:04.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.284+0000 7ff03559a700 1 --2- 172.21.15.47:0/3283288816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff01c004e70 0x7ff01c00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:04.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.284+0000 7ff03559a700 1 -- 172.21.15.47:0/3283288816 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff038143490 con 0x7ff01c004500 2022-06-29T15:50:04.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.284+0000 7ff03559a700 1 -- 172.21.15.47:0/3283288816 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff01c00bd20 con 0x7ff01c004e70 2022-06-29T15:50:04.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.284+0000 7ff03559a700 1 -- 172.21.15.47:0/3283288816 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff01c00bee0 con 0x7ff01c003930 2022-06-29T15:50:04.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.284+0000 7ff035d9b700 1 --2- 172.21.15.47:0/3283288816 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff01c004500 0x7ff01c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:04.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.284+0000 7ff03659c700 1 --2- 172.21.15.47:0/3283288816 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff01c003930 0x7ff01c003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:04.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.284+0000 7ff035d9b700 1 -- 172.21.15.47:0/3283288816 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff01c003930 msgr2=0x7ff01c003d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:04.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.284+0000 7ff035d9b700 1 --2- 172.21.15.47:0/3283288816 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff01c003930 0x7ff01c003d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:04.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.284+0000 7ff035d9b700 1 -- 172.21.15.47:0/3283288816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff01c004e70 msgr2=0x7ff01c00b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:04.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.284+0000 7ff035d9b700 1 --2- 172.21.15.47:0/3283288816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff01c004e70 0x7ff01c00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2022-06-29T15:50:04.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.284+0000 7ff035d9b700 1 -- 172.21.15.47:0/3283288816 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff01c003260 con 0x7ff01c004500 2022-06-29T15:50:04.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.284+0000 7ff036d9d700 1 -- 172.21.15.47:0/3283288816 reap_dead start 2022-06-29T15:50:04.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.284+0000 7ff035d9b700 1 --2- 172.21.15.47:0/3283288816 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff01c004500 0x7ff01c004980 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 rx=0x7ff02c001450 tx=0x7ff038143380).ready entity=mon.0 client_cookie=5f737969ae83bf26 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:04.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.284+0000 7ff03559a700 1 -- 172.21.15.47:0/3283288816 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff02c007e90 con 0x7ff01c004500 2022-06-29T15:50:04.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.284+0000 7ff03559a700 1 -- 172.21.15.47:0/3283288816 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff02c007170 con 0x7ff01c004500 2022-06-29T15:50:04.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.284+0000 7ff03559a700 1 -- 172.21.15.47:0/3283288816 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff02c0074d0 con 0x7ff01c004500 2022-06-29T15:50:04.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.284+0000 7ff03759e700 1 -- 172.21.15.47:0/3283288816 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff01c004500 msgr2=0x7ff01c004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:04.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.284+0000 7ff03759e700 1 --2- 172.21.15.47:0/3283288816 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff01c004500 0x7ff01c004980 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 rx=0x7ff02c001450 tx=0x7ff038143380).stop 2022-06-29T15:50:04.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.284+0000 7ff03759e700 1 -- 172.21.15.47:0/3283288816 shutdown_connections 2022-06-29T15:50:04.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.284+0000 7ff03759e700 1 --2- 172.21.15.47:0/3283288816 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff01c004500 0x7ff01c004980 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:04.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.284+0000 7ff03759e700 1 -- 172.21.15.47:0/3283288816 >> 172.21.15.47:0/3283288816 conn(0x7ff038060e00 msgr2=0x7ff0380611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:04.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.288+0000 7ff03759e700 1 -- 172.21.15.47:0/3283288816 shutdown_connections 2022-06-29T15:50:04.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.288+0000 7ff03759e700 1 -- 172.21.15.47:0/3283288816 wait complete. 2022-06-29T15:50:04.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.288+0000 7ff03759e700 1 Processor -- start 2022-06-29T15:50:04.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.288+0000 7ff03759e700 1 -- start start 2022-06-29T15:50:04.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.288+0000 7ff03759e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff01c004500 0x7ff0380b0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:04.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.288+0000 7ff03759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff038146e40 0x7ff0380b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:04.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.288+0000 7ff03759e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0380adab0 0x7ff0380adf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:04.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.288+0000 7ff03759e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0380b94d0 con 0x7ff0380adab0 2022-06-29T15:50:04.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.288+0000 7ff03759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff038156770 con 0x7ff038146e40 2022-06-29T15:50:04.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.288+0000 7ff03759e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0381466b0 con 0x7ff01c004500 2022-06-29T15:50:04.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.288+0000 7ff036d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0380adab0 0x7ff0380adf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:04.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.288+0000 7ff036d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0380adab0 0x7ff0380adf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51756/0 (socket says 172.21.15.47:51756) 2022-06-29T15:50:04.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.288+0000 7ff036d9d700 1 -- 172.21.15.47:0/3448354728 learned_addr learned my addr 172.21.15.47:0/3448354728 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:04.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.288+0000 7ff03659c700 1 --2- 172.21.15.47:0/3448354728 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff01c004500 0x7ff0380b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:04.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.288+0000 7ff035d9b700 1 --2- 172.21.15.47:0/3448354728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff038146e40 0x7ff0380b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:04.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.288+0000 7ff036d9d700 1 -- 172.21.15.47:0/3448354728 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff01c004500 msgr2=0x7ff0380b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:04.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.288+0000 7ff036d9d700 1 --2- 172.21.15.47:0/3448354728 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff01c004500 0x7ff0380b0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:04.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.288+0000 7ff036d9d700 1 -- 172.21.15.47:0/3448354728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff038146e40 msgr2=0x7ff0380b0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:04.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.288+0000 7ff036d9d700 1 --2- 172.21.15.47:0/3448354728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff038146e40 0x7ff0380b0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:04.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.288+0000 7ff036d9d700 1 -- 172.21.15.47:0/3448354728 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff038146950 con 0x7ff0380adab0 2022-06-29T15:50:04.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.288+0000 7ff035d9b700 1 --2- 172.21.15.47:0/3448354728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff038146e40 0x7ff0380b0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:50:04.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.288+0000 7ff036d9d700 1 --2- 172.21.15.47:0/3448354728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0380adab0 0x7ff0380adf60 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 rx=0x7ff02c001860 tx=0x7ff0300091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:04.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.288+0000 7ff0277fe700 1 -- 172.21.15.47:0/3448354728 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff030010780 con 0x7ff0380adab0 2022-06-29T15:50:04.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.288+0000 7ff0277fe700 1 -- 172.21.15.47:0/3448354728 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0300108e0 con 0x7ff0380adab0 2022-06-29T15:50:04.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.288+0000 7ff03759e700 1 -- 172.21.15.47:0/3448354728 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0381563e0 con 0x7ff0380adab0 2022-06-29T15:50:04.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.288+0000 7ff0277fe700 1 -- 172.21.15.47:0/3448354728 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff030010c40 con 0x7ff0380adab0 2022-06-29T15:50:04.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.288+0000 7ff03759e700 1 -- 172.21.15.47:0/3448354728 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0380ae670 con 0x7ff0380adab0 2022-06-29T15:50:04.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.292+0000 7ff03759e700 1 -- 172.21.15.47:0/3448354728 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff038146ae0 con 0x7ff0380adab0 2022-06-29T15:50:04.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.292+0000 7ff0277fe700 1 -- 172.21.15.47:0/3448354728 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff03001c460 con 0x7ff0380adab0 2022-06-29T15:50:04.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.292+0000 7ff0277fe700 1 --2- 172.21.15.47:0/3448354728 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff014053b30 0x7ff014055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:04.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.292+0000 7ff0277fe700 1 -- 172.21.15.47:0/3448354728 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6506+0+0 (secure 0 0 0) 0x7ff03006c680 con 0x7ff0380adab0 2022-06-29T15:50:04.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.292+0000 7ff03659c700 1 --2- 172.21.15.47:0/3448354728 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff014053b30 0x7ff014055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:04.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.292+0000 7ff03659c700 1 --2- 172.21.15.47:0/3448354728 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff014053b30 0x7ff014055ff0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7ff028003000 tx=0x7ff028007a00).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:04.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.300+0000 7ff0277fe700 1 -- 172.21.15.47:0/3448354728 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff0300432f0 con 0x7ff0380adab0 2022-06-29T15:50:04.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.468+0000 7ff03759e700 1 -- 172.21.15.47:0/3448354728 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff0380573d0 con 0x7ff0380adab0 2022-06-29T15:50:04.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.468+0000 7ff0277fe700 1 -- 172.21.15.47:0/3448354728 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+2130 (secure 0 0 0) 0x7ff030046910 con 0x7ff0380adab0 2022-06-29T15:50:04.475 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:50:04.475 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":46,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6969,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3642031880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3642031880},{"type":"v1","addr":"172.21.15.47:6835","nonce":3642031880}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":43},{"gid":6984,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/891572891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":891572891},{"type":"v1","addr":"172.21.15.47:6837","nonce":891572891}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":43}],"filesystems":[{"mdsmap":{"epoch":46,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:49:50.788599+0000","modified":"2022-06-29T15:50:04.140221+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":102,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2022-06-29T15:50:04.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.472+0000 7ff03759e700 1 -- 172.21.15.47:0/3448354728 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff014053b30 msgr2=0x7ff014055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:04.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.472+0000 7ff03759e700 1 --2- 172.21.15.47:0/3448354728 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff014053b30 0x7ff014055ff0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7ff028003000 tx=0x7ff028007a00).stop 2022-06-29T15:50:04.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.472+0000 7ff03759e700 1 -- 172.21.15.47:0/3448354728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0380adab0 msgr2=0x7ff0380adf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:04.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.472+0000 7ff03759e700 1 --2- 172.21.15.47:0/3448354728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0380adab0 0x7ff0380adf60 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 rx=0x7ff02c001860 tx=0x7ff0300091d0).stop 2022-06-29T15:50:04.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.472+0000 7ff03759e700 1 -- 172.21.15.47:0/3448354728 shutdown_connections 2022-06-29T15:50:04.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.472+0000 7ff03759e700 1 --2- 172.21.15.47:0/3448354728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0380adab0 0x7ff0380adf60 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:04.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.472+0000 7ff03759e700 1 --2- 172.21.15.47:0/3448354728 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff014053b30 0x7ff014055ff0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:04.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.472+0000 7ff03759e700 1 --2- 172.21.15.47:0/3448354728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff038146e40 0x7ff0380b0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:04.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.472+0000 7ff03759e700 1 --2- 172.21.15.47:0/3448354728 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff01c004500 0x7ff0380b0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:04.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.472+0000 7ff03759e700 1 -- 172.21.15.47:0/3448354728 >> 172.21.15.47:0/3448354728 conn(0x7ff038060e00 msgr2=0x7ff03813fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:04.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.472+0000 7ff03759e700 1 -- 172.21.15.47:0/3448354728 shutdown_connections 2022-06-29T15:50:04.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.472+0000 7ff03759e700 1 -- 172.21.15.47:0/3448354728 wait complete. 2022-06-29T15:50:04.480 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 46 2022-06-29T15:50:04.489 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-06-29T15:50:04.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.588+0000 7f94ff59e700 1 Processor -- start 2022-06-29T15:50:04.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.588+0000 7f94ff59e700 1 -- start start 2022-06-29T15:50:04.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.592+0000 7f94ff59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f950005f3f0 0x7f95001418c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:04.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.592+0000 7f94ff59e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f95000b1320 0x7f9500141db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:04.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.592+0000 7f94ff59e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95001422a0 con 0x7f95000aaf00 2022-06-29T15:50:04.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.592+0000 7f94ff59e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95000b27a0 con 0x7f950005f3f0 2022-06-29T15:50:04.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.592+0000 7f94ff59e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f950005f7d0 con 0x7f95000b1320 2022-06-29T15:50:04.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.592+0000 7f94f5d9b700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f95000b1320 0x7f9500141db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:04.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.592+0000 7f94fe59c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f950005f3f0 0x7f95001418c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:04.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.592+0000 7f94fed9d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f95000aaf00 0x7f95000b2ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:44882/0 (socket says 172.21.15.47:44882) 2022-06-29T15:50:04.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.592+0000 7f94fed9d700 1 -- 172.21.15.47:0/1979010956 learned_addr learned my addr 172.21.15.47:0/1979010956 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:04.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.592+0000 7f94fdd9b700 1 -- 172.21.15.47:0/1979010956 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1336825933 0 0) 0x7f95001422a0 con 0x7f95000aaf00 2022-06-29T15:50:04.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.592+0000 7f94fdd9b700 1 -- 172.21.15.47:0/1979010956 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94e0003260 con 0x7f95000aaf00 2022-06-29T15:50:04.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.592+0000 7f94f5d9b700 1 -- 172.21.15.47:0/1979010956 >> v2:172.21.15.64:3300/0 conn(0x7f950005f3f0 msgr2=0x7f95001418c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:04.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.592+0000 7f94f5d9b700 1 --2- 172.21.15.47:0/1979010956 >> v2:172.21.15.64:3300/0 conn(0x7f950005f3f0 0x7f95001418c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:04.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.592+0000 7f94f5d9b700 1 -- 172.21.15.47:0/1979010956 >> v1:172.21.15.64:6789/0 conn(0x7f95000aaf00 legacy=0x7f95000b2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:04.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.592+0000 7f94f5d9b700 1 -- 172.21.15.47:0/1979010956 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95000b4b90 con 0x7f95000b1320 2022-06-29T15:50:04.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.592+0000 7f94fe59c700 1 --2- 172.21.15.47:0/1979010956 >> v2:172.21.15.64:3300/0 conn(0x7f950005f3f0 0x7f95001418c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:50:04.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.592+0000 7f94f5d9b700 1 --2- 172.21.15.47:0/1979010956 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f95000b1320 0x7f9500141db0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f94e8002f70 tx=0x7f94e8006df0).ready entity=mon.2 client_cookie=efbacc9362b2652d server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:04.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.592+0000 7f94fdd9b700 1 -- 172.21.15.47:0/1979010956 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94e800d4a0 con 0x7f95000b1320 2022-06-29T15:50:04.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.592+0000 7f94fdd9b700 1 -- 172.21.15.47:0/1979010956 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e8000e90 con 0x7f95000b1320 2022-06-29T15:50:04.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.592+0000 7f94fdd9b700 1 -- 172.21.15.47:0/1979010956 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94e800d7f0 con 0x7f95000b1320 2022-06-29T15:50:04.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.592+0000 7f94ff59e700 1 -- 172.21.15.47:0/1979010956 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f95000b1320 msgr2=0x7f9500141db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:04.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.592+0000 7f94ff59e700 1 --2- 172.21.15.47:0/1979010956 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f95000b1320 0x7f9500141db0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f94e8002f70 tx=0x7f94e8006df0).stop 2022-06-29T15:50:04.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.596+0000 7f94ff59e700 1 -- 172.21.15.47:0/1979010956 shutdown_connections 2022-06-29T15:50:04.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.596+0000 7f94ff59e700 1 --2- 172.21.15.47:0/1979010956 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f95000b1320 0x7f9500141db0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:04.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.596+0000 7f94ff59e700 1 --2- 172.21.15.47:0/1979010956 >> v2:172.21.15.64:3300/0 conn(0x7f950005f3f0 0x7f95001418c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:04.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.596+0000 7f94ff59e700 1 -- 172.21.15.47:0/1979010956 >> 172.21.15.47:0/1979010956 conn(0x7f9500062570 msgr2=0x7f95000647d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:04.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.596+0000 7f94ff59e700 1 -- 172.21.15.47:0/1979010956 shutdown_connections 2022-06-29T15:50:04.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.596+0000 7f94ff59e700 1 -- 172.21.15.47:0/1979010956 wait complete. 2022-06-29T15:50:04.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.596+0000 7f94ff59e700 1 Processor -- start 2022-06-29T15:50:04.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.596+0000 7f94ff59e700 1 -- start start 2022-06-29T15:50:04.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.596+0000 7f94ff59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f950005f3f0 0x7f9500146c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:04.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.596+0000 7f94ff59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95000aaf00 0x7f95000acd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:04.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.596+0000 7f94ff59e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f95000b1320 0x7f95000ad220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:04.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.596+0000 7f94ff59e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95000ac2f0 con 0x7f95000aaf00 2022-06-29T15:50:04.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.596+0000 7f94ff59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95000ac900 con 0x7f950005f3f0 2022-06-29T15:50:04.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.596+0000 7f94ff59e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95000ab5c0 con 0x7f95000b1320 2022-06-29T15:50:04.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.596+0000 7f94f5d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95000aaf00 0x7f95000acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:04.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.596+0000 7f94f5d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95000aaf00 0x7f95000acd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51766/0 (socket says 172.21.15.47:51766) 2022-06-29T15:50:04.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.596+0000 7f94f5d9b700 1 -- 172.21.15.47:0/230135759 learned_addr learned my addr 172.21.15.47:0/230135759 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:04.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.596+0000 7f94fed9d700 1 --2- 172.21.15.47:0/230135759 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f95000b1320 0x7f95000ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:04.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.596+0000 7f94fe59c700 1 --2- 172.21.15.47:0/230135759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f950005f3f0 0x7f9500146c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:04.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.596+0000 7f94f5d9b700 1 -- 172.21.15.47:0/230135759 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f95000b1320 msgr2=0x7f95000ad220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:04.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.596+0000 7f94f5d9b700 1 --2- 172.21.15.47:0/230135759 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f95000b1320 0x7f95000ad220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:04.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.596+0000 7f94f5d9b700 1 -- 172.21.15.47:0/230135759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f950005f3f0 msgr2=0x7f9500146c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:04.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.596+0000 7f94f5d9b700 1 --2- 172.21.15.47:0/230135759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f950005f3f0 0x7f9500146c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:04.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.596+0000 7f94f5d9b700 1 -- 172.21.15.47:0/230135759 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f000b180 con 0x7f95000aaf00 2022-06-29T15:50:04.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.596+0000 7f94fe59c700 1 --2- 172.21.15.47:0/230135759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f950005f3f0 0x7f9500146c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:50:04.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.596+0000 7f94f5d9b700 1 --2- 172.21.15.47:0/230135759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95000aaf00 0x7f95000acd30 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 rx=0x7f94e8002f70 tx=0x7f94e8014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:04.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.600+0000 7f94f7fff700 1 -- 172.21.15.47:0/230135759 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94e8013ac0 con 0x7f95000aaf00 2022-06-29T15:50:04.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.600+0000 7f94f7fff700 1 -- 172.21.15.47:0/230135759 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e8013c20 con 0x7f95000aaf00 2022-06-29T15:50:04.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.600+0000 7f94ff59e700 1 -- 172.21.15.47:0/230135759 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94e800b180 con 0x7f95000aaf00 2022-06-29T15:50:04.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.600+0000 7f94f7fff700 1 -- 172.21.15.47:0/230135759 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94e801e400 con 0x7f95000aaf00 2022-06-29T15:50:04.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.600+0000 7f94ff59e700 1 -- 172.21.15.47:0/230135759 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9500061900 con 0x7f95000aaf00 2022-06-29T15:50:04.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.600+0000 7f94ff59e700 1 -- 172.21.15.47:0/230135759 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95000573d0 con 0x7f95000aaf00 2022-06-29T15:50:04.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.600+0000 7f94f7fff700 1 -- 172.21.15.47:0/230135759 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f94e801c070 con 0x7f95000aaf00 2022-06-29T15:50:04.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.600+0000 7f94f7fff700 1 --2- 172.21.15.47:0/230135759 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f94e4053aa0 0x7f94e4055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:04.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.600+0000 7f94f7fff700 1 -- 172.21.15.47:0/230135759 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f94e806ca70 con 0x7f95000aaf00 2022-06-29T15:50:04.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.600+0000 7f94fe59c700 1 --2- 172.21.15.47:0/230135759 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f94e4053aa0 0x7f94e4055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:04.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.600+0000 7f94fe59c700 1 --2- 172.21.15.47:0/230135759 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f94e4053aa0 0x7f94e4055f60 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7f94f000b000 tx=0x7f94f0008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:04.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.608+0000 7f94f7fff700 1 -- 172.21.15.47:0/230135759 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f94e8043740 con 0x7f95000aaf00 2022-06-29T15:50:04.782 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:04.776+0000 7f94ff59e700 1 -- 172.21.15.47:0/230135759 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f9500068830 con 0x7f95000aaf00 2022-06-29T15:50:05.161 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.156+0000 7f94f7fff700 1 -- 172.21.15.47:0/230135759 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v47) v1 ==== 104+0+0 (secure 0 0 0) 0x7f94e80230a0 con 0x7f95000aaf00 2022-06-29T15:50:05.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.160+0000 7f94ff59e700 1 -- 172.21.15.47:0/230135759 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f94e4053aa0 msgr2=0x7f94e4055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:05.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.160+0000 7f94ff59e700 1 --2- 172.21.15.47:0/230135759 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f94e4053aa0 0x7f94e4055f60 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7f94f000b000 tx=0x7f94f0008040).stop 2022-06-29T15:50:05.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.160+0000 7f94ff59e700 1 -- 172.21.15.47:0/230135759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95000aaf00 msgr2=0x7f95000acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:05.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.160+0000 7f94ff59e700 1 --2- 172.21.15.47:0/230135759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95000aaf00 0x7f95000acd30 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 rx=0x7f94e8002f70 tx=0x7f94e8014000).stop 2022-06-29T15:50:05.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.160+0000 7f94ff59e700 1 -- 172.21.15.47:0/230135759 shutdown_connections 2022-06-29T15:50:05.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.160+0000 7f94ff59e700 1 --2- 172.21.15.47:0/230135759 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f95000b1320 0x7f95000ad220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:05.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.160+0000 7f94ff59e700 1 --2- 172.21.15.47:0/230135759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95000aaf00 0x7f95000acd30 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:05.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.160+0000 7f94ff59e700 1 --2- 172.21.15.47:0/230135759 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f94e4053aa0 0x7f94e4055f60 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:05.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.160+0000 7f94ff59e700 1 --2- 172.21.15.47:0/230135759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f950005f3f0 0x7f9500146c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:05.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.160+0000 7f94ff59e700 1 -- 172.21.15.47:0/230135759 >> 172.21.15.47:0/230135759 conn(0x7f9500062570 msgr2=0x7f95000b0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:05.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.160+0000 7f94ff59e700 1 -- 172.21.15.47:0/230135759 shutdown_connections 2022-06-29T15:50:05.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.160+0000 7f94ff59e700 1 -- 172.21.15.47:0/230135759 wait complete. 2022-06-29T15:50:05.182 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-06-29T15:50:05.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.292+0000 7f96f482b700 1 Processor -- start 2022-06-29T15:50:05.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.292+0000 7f96f482b700 1 -- start start 2022-06-29T15:50:05.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.292+0000 7f96f482b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f96f00b58d0 0x7f96f00b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:05.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.292+0000 7f96f482b700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f96f00b1610 0x7f96f01475f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:05.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.292+0000 7f96f482b700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96f0060790 con 0x7f96f014b4c0 2022-06-29T15:50:05.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.292+0000 7f96f482b700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96f00a99e0 con 0x7f96f00b58d0 2022-06-29T15:50:05.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.292+0000 7f96f482b700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96f014ad80 con 0x7f96f00b1610 2022-06-29T15:50:05.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.292+0000 7f96eed9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f96f00b58d0 0x7f96f00b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:05.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.292+0000 7f96ef59e700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f96f014b4c0 0x7f96f00aa130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37332/0 (socket says 172.21.15.47:37332) 2022-06-29T15:50:05.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.292+0000 7f96ef59e700 1 -- 172.21.15.47:0/4239102506 learned_addr learned my addr 172.21.15.47:0/4239102506 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:05.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.292+0000 7f96e7fff700 1 --2- 172.21.15.47:0/4239102506 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f96f00b1610 0x7f96f01475f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:05.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.292+0000 7f96eed9d700 1 -- 172.21.15.47:0/4239102506 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f96f00b1610 msgr2=0x7f96f01475f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:05.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.292+0000 7f96eed9d700 1 --2- 172.21.15.47:0/4239102506 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f96f00b1610 0x7f96f01475f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:05.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.292+0000 7f96eed9d700 1 -- 172.21.15.47:0/4239102506 >> v1:172.21.15.47:6789/0 conn(0x7f96f014b4c0 legacy=0x7f96f00aa130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:05.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.292+0000 7f96eed9d700 1 -- 172.21.15.47:0/4239102506 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f014b020 con 0x7f96f00b58d0 2022-06-29T15:50:05.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.292+0000 7f96eed9d700 1 --2- 172.21.15.47:0/4239102506 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96f00b58d0 0x7f96f00b5cb0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 rx=0x7f96f00623e0 tx=0x7f96e0006df0).ready entity=mon.0 client_cookie=383d7cc838a16872 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:05.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.292+0000 7f96ee59c700 1 -- 172.21.15.47:0/4239102506 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96e000d570 con 0x7f96f00b58d0 2022-06-29T15:50:05.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.292+0000 7f96ee59c700 1 -- 172.21.15.47:0/4239102506 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e0000e90 con 0x7f96f00b58d0 2022-06-29T15:50:05.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.292+0000 7f96ee59c700 1 -- 172.21.15.47:0/4239102506 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96e000d8c0 con 0x7f96f00b58d0 2022-06-29T15:50:05.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.296+0000 7f96f482b700 1 -- 172.21.15.47:0/4239102506 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96f00b58d0 msgr2=0x7f96f00b5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:05.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.296+0000 7f96f482b700 1 --2- 172.21.15.47:0/4239102506 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96f00b58d0 0x7f96f00b5cb0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 rx=0x7f96f00623e0 tx=0x7f96e0006df0).stop 2022-06-29T15:50:05.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.296+0000 7f96f482b700 1 -- 172.21.15.47:0/4239102506 shutdown_connections 2022-06-29T15:50:05.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.296+0000 7f96f482b700 1 --2- 172.21.15.47:0/4239102506 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f96f00b1610 0x7f96f01475f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:05.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.296+0000 7f96f482b700 1 --2- 172.21.15.47:0/4239102506 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96f00b58d0 0x7f96f00b5cb0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:05.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.296+0000 7f96f482b700 1 -- 172.21.15.47:0/4239102506 >> 172.21.15.47:0/4239102506 conn(0x7f96f0062560 msgr2=0x7f96f0062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:05.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.296+0000 7f96f482b700 1 -- 172.21.15.47:0/4239102506 shutdown_connections 2022-06-29T15:50:05.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.296+0000 7f96f482b700 1 -- 172.21.15.47:0/4239102506 wait complete. 2022-06-29T15:50:05.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.296+0000 7f96f482b700 1 Processor -- start 2022-06-29T15:50:05.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.296+0000 7f96f482b700 1 -- start start 2022-06-29T15:50:05.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.296+0000 7f96f482b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96f00b1610 0x7f96f00af180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:05.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.296+0000 7f96f482b700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f96f00b58d0 0x7f96f015a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:05.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.296+0000 7f96f482b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96f014b4c0 0x7f96f00acb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:05.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.296+0000 7f96f482b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f00ae0d0 con 0x7f96f014b4c0 2022-06-29T15:50:05.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.296+0000 7f96f482b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f015a120 con 0x7f96f00b1610 2022-06-29T15:50:05.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.296+0000 7f96f482b700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96f00b2360 con 0x7f96f00b58d0 2022-06-29T15:50:05.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.296+0000 7f96ef59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96f014b4c0 0x7f96f00acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:05.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.296+0000 7f96ef59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96f014b4c0 0x7f96f00acb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51782/0 (socket says 172.21.15.47:51782) 2022-06-29T15:50:05.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.296+0000 7f96ef59e700 1 -- 172.21.15.47:0/3429717814 learned_addr learned my addr 172.21.15.47:0/3429717814 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:05.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.296+0000 7f96e7fff700 1 --2- 172.21.15.47:0/3429717814 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f96f00b58d0 0x7f96f015a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:05.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.300+0000 7f96eed9d700 1 --2- 172.21.15.47:0/3429717814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96f00b1610 0x7f96f00af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:05.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.300+0000 7f96ef59e700 1 -- 172.21.15.47:0/3429717814 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f96f00b58d0 msgr2=0x7f96f015a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:05.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.300+0000 7f96ef59e700 1 --2- 172.21.15.47:0/3429717814 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f96f00b58d0 0x7f96f015a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:05.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.300+0000 7f96ef59e700 1 -- 172.21.15.47:0/3429717814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96f00b1610 msgr2=0x7f96f00af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:05.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.300+0000 7f96ef59e700 1 --2- 172.21.15.47:0/3429717814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96f00b1610 0x7f96f00af180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:05.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.300+0000 7f96ef59e700 1 -- 172.21.15.47:0/3429717814 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e000b180 con 0x7f96f014b4c0 2022-06-29T15:50:05.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.300+0000 7f96ef59e700 1 --2- 172.21.15.47:0/3429717814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96f014b4c0 0x7f96f00acb50 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 rx=0x7f96f00a9140 tx=0x7f96e80013a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:05.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.300+0000 7f96ecd99700 1 -- 172.21.15.47:0/3429717814 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96e8005ba0 con 0x7f96f014b4c0 2022-06-29T15:50:05.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.300+0000 7f96ecd99700 1 -- 172.21.15.47:0/3429717814 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e8005d00 con 0x7f96f014b4c0 2022-06-29T15:50:05.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.300+0000 7f96f482b700 1 -- 172.21.15.47:0/3429717814 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96f0143a90 con 0x7f96f014b4c0 2022-06-29T15:50:05.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.300+0000 7f96ecd99700 1 -- 172.21.15.47:0/3429717814 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96e80072c0 con 0x7f96f014b4c0 2022-06-29T15:50:05.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.300+0000 7f96f482b700 1 -- 172.21.15.47:0/3429717814 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96f0143780 con 0x7f96f014b4c0 2022-06-29T15:50:05.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.300+0000 7f96f482b700 1 -- 172.21.15.47:0/3429717814 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96f00573d0 con 0x7f96f014b4c0 2022-06-29T15:50:05.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.300+0000 7f96ecd99700 1 -- 172.21.15.47:0/3429717814 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f96e801a060 con 0x7f96f014b4c0 2022-06-29T15:50:05.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.300+0000 7f96ecd99700 1 --2- 172.21.15.47:0/3429717814 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f96d4053950 0x7f96d4055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:05.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.300+0000 7f96eed9d700 1 --2- 172.21.15.47:0/3429717814 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f96d4053950 0x7f96d4055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:05.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.300+0000 7f96ecd99700 1 -- 172.21.15.47:0/3429717814 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f96e806baf0 con 0x7f96f014b4c0 2022-06-29T15:50:05.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.304+0000 7f96eed9d700 1 --2- 172.21.15.47:0/3429717814 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f96d4053950 0x7f96d4055e10 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7f96e001a000 tx=0x7f96e0014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:05.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.308+0000 7f96ecd99700 1 -- 172.21.15.47:0/3429717814 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f96e8042780 con 0x7f96f014b4c0 2022-06-29T15:50:05.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:05.432+0000 7f96f482b700 1 -- 172.21.15.47:0/3429717814 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f96f0069e20 con 0x7f96f014b4c0 2022-06-29T15:50:06.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.264+0000 7f96ecd99700 1 -- 172.21.15.47:0/3429717814 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v103) v1 ==== 185+0+0 (secure 0 0 0) 0x7f96e80180e0 con 0x7f96f014b4c0 2022-06-29T15:50:06.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.268+0000 7f96f482b700 1 -- 172.21.15.47:0/3429717814 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f96d4053950 msgr2=0x7f96d4055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:06.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.268+0000 7f96f482b700 1 --2- 172.21.15.47:0/3429717814 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f96d4053950 0x7f96d4055e10 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7f96e001a000 tx=0x7f96e0014000).stop 2022-06-29T15:50:06.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.268+0000 7f96f482b700 1 -- 172.21.15.47:0/3429717814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96f014b4c0 msgr2=0x7f96f00acb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:06.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.268+0000 7f96f482b700 1 --2- 172.21.15.47:0/3429717814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96f014b4c0 0x7f96f00acb50 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 rx=0x7f96f00a9140 tx=0x7f96e80013a0).stop 2022-06-29T15:50:06.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.268+0000 7f96f482b700 1 -- 172.21.15.47:0/3429717814 shutdown_connections 2022-06-29T15:50:06.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.268+0000 7f96f482b700 1 --2- 172.21.15.47:0/3429717814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96f014b4c0 0x7f96f00acb50 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:06.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.268+0000 7f96f482b700 1 --2- 172.21.15.47:0/3429717814 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f96f00b58d0 0x7f96f015a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:06.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.268+0000 7f96f482b700 1 --2- 172.21.15.47:0/3429717814 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f96d4053950 0x7f96d4055e10 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:06.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.268+0000 7f96f482b700 1 --2- 172.21.15.47:0/3429717814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96f00b1610 0x7f96f00af180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:06.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.268+0000 7f96f482b700 1 -- 172.21.15.47:0/3429717814 >> 172.21.15.47:0/3429717814 conn(0x7f96f0062560 msgr2=0x7f96f00ab700 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:06.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.268+0000 7f96f482b700 1 -- 172.21.15.47:0/3429717814 shutdown_connections 2022-06-29T15:50:06.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.268+0000 7f96f482b700 1 -- 172.21.15.47:0/3429717814 wait complete. 2022-06-29T15:50:06.278 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' removed 2022-06-29T15:50:06.288 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-06-29T15:50:06.410 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.404+0000 7fd5469e1700 1 Processor -- start 2022-06-29T15:50:06.410 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.404+0000 7fd5469e1700 1 -- start start 2022-06-29T15:50:06.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.404+0000 7fd5469e1700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd540067470 0x7fd540067850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:06.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.404+0000 7fd5469e1700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd5400609d0 0x7fd540143e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:06.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.404+0000 7fd5469e1700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd540140700 0x7fd540140b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:06.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.404+0000 7fd5469e1700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5401434e0 con 0x7fd5400609d0 2022-06-29T15:50:06.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.404+0000 7fd5469e1700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd540144370 con 0x7fd540067470 2022-06-29T15:50:06.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.404+0000 7fd5469e1700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd540141080 con 0x7fd540140700 2022-06-29T15:50:06.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.404+0000 7fd53f7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd5400609d0 0x7fd540143e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:06.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.404+0000 7fd544f7e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd540140700 0x7fd540140b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:06.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.408+0000 7fd53ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd540067470 0x7fd540067850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:06.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.408+0000 7fd53f7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd5400609d0 0x7fd540143e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51792/0 (socket says 172.21.15.47:51792) 2022-06-29T15:50:06.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.408+0000 7fd53f7fe700 1 -- 172.21.15.47:0/2086894679 learned_addr learned my addr 172.21.15.47:0/2086894679 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:06.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.408+0000 7fd53ffff700 1 -- 172.21.15.47:0/2086894679 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd540140700 msgr2=0x7fd540140b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:06.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.408+0000 7fd53ffff700 1 --2- 172.21.15.47:0/2086894679 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd540140700 0x7fd540140b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:06.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.408+0000 7fd53ffff700 1 -- 172.21.15.47:0/2086894679 >> v2:172.21.15.47:3300/0 conn(0x7fd5400609d0 msgr2=0x7fd540143e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:06.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.408+0000 7fd53ffff700 1 --2- 172.21.15.47:0/2086894679 >> v2:172.21.15.47:3300/0 conn(0x7fd5400609d0 0x7fd540143e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:06.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.408+0000 7fd53ffff700 1 -- 172.21.15.47:0/2086894679 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd540143740 con 0x7fd540067470 2022-06-29T15:50:06.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.408+0000 7fd53ffff700 1 --2- 172.21.15.47:0/2086894679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd540067470 0x7fd540067850 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7fd5300032f0 tx=0x7fd5300091d0).ready entity=mon.1 client_cookie=431e3a015e6c92bb server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:06.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.408+0000 7fd53effd700 1 -- 172.21.15.47:0/2086894679 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd53000b600 con 0x7fd540067470 2022-06-29T15:50:06.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.408+0000 7fd53effd700 1 -- 172.21.15.47:0/2086894679 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd53000b760 con 0x7fd540067470 2022-06-29T15:50:06.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.408+0000 7fd53effd700 1 -- 172.21.15.47:0/2086894679 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd53000b8d0 con 0x7fd540067470 2022-06-29T15:50:06.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.408+0000 7fd5469e1700 1 -- 172.21.15.47:0/2086894679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd540067470 msgr2=0x7fd540067850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:06.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.408+0000 7fd5469e1700 1 --2- 172.21.15.47:0/2086894679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd540067470 0x7fd540067850 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7fd5300032f0 tx=0x7fd5300091d0).stop 2022-06-29T15:50:06.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.408+0000 7fd5469e1700 1 -- 172.21.15.47:0/2086894679 shutdown_connections 2022-06-29T15:50:06.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.408+0000 7fd5469e1700 1 --2- 172.21.15.47:0/2086894679 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd540140700 0x7fd540140b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:06.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.408+0000 7fd5469e1700 1 --2- 172.21.15.47:0/2086894679 >> v2:172.21.15.47:3300/0 conn(0x7fd5400609d0 0x7fd540143e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:06.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.408+0000 7fd5469e1700 1 --2- 172.21.15.47:0/2086894679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd540067470 0x7fd540067850 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:06.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.408+0000 7fd5469e1700 1 -- 172.21.15.47:0/2086894679 >> 172.21.15.47:0/2086894679 conn(0x7fd54005f0b0 msgr2=0x7fd54005f490 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:06.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.408+0000 7fd5469e1700 1 -- 172.21.15.47:0/2086894679 shutdown_connections 2022-06-29T15:50:06.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.408+0000 7fd5469e1700 1 -- 172.21.15.47:0/2086894679 wait complete. 2022-06-29T15:50:06.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.412+0000 7fd5469e1700 1 Processor -- start 2022-06-29T15:50:06.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.412+0000 7fd5469e1700 1 -- start start 2022-06-29T15:50:06.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.412+0000 7fd5469e1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5400609d0 0x7fd5400b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:06.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.412+0000 7fd5469e1700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd540067470 0x7fd5400ade80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:06.424 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.412+0000 7fd5469e1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd540140700 0x7fd540155d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:06.424 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.412+0000 7fd5469e1700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54009b1e0 con 0x7fd5400609d0 2022-06-29T15:50:06.424 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.412+0000 7fd5469e1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5401558d0 con 0x7fd540140700 2022-06-29T15:50:06.424 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.412+0000 7fd5469e1700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd54009abb0 con 0x7fd540067470 2022-06-29T15:50:06.425 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.412+0000 7fd53ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5400609d0 0x7fd5400b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:06.425 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.412+0000 7fd53ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5400609d0 0x7fd5400b3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51794/0 (socket says 172.21.15.47:51794) 2022-06-29T15:50:06.426 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.412+0000 7fd53ffff700 1 -- 172.21.15.47:0/1577156947 learned_addr learned my addr 172.21.15.47:0/1577156947 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:06.426 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.412+0000 7fd544f7e700 1 --2- 172.21.15.47:0/1577156947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd540140700 0x7fd540155d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:06.427 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.412+0000 7fd53f7fe700 1 --2- 172.21.15.47:0/1577156947 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd540067470 0x7fd5400ade80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:06.427 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.412+0000 7fd544f7e700 1 -- 172.21.15.47:0/1577156947 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd540067470 msgr2=0x7fd5400ade80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:06.427 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.412+0000 7fd544f7e700 1 --2- 172.21.15.47:0/1577156947 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd540067470 0x7fd5400ade80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:06.427 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.412+0000 7fd544f7e700 1 -- 172.21.15.47:0/1577156947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5400609d0 msgr2=0x7fd5400b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:06.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.412+0000 7fd544f7e700 1 --2- 172.21.15.47:0/1577156947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5400609d0 0x7fd5400b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:06.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.412+0000 7fd544f7e700 1 -- 172.21.15.47:0/1577156947 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd530009470 con 0x7fd540140700 2022-06-29T15:50:06.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.412+0000 7fd53ffff700 1 --2- 172.21.15.47:0/1577156947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5400609d0 0x7fd5400b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:50:06.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.412+0000 7fd544f7e700 1 --2- 172.21.15.47:0/1577156947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd540140700 0x7fd540155d00 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7fd53800c000 tx=0x7fd538006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:06.429 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.412+0000 7fd53d7fa700 1 -- 172.21.15.47:0/1577156947 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd538016070 con 0x7fd540140700 2022-06-29T15:50:06.429 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.412+0000 7fd53d7fa700 1 -- 172.21.15.47:0/1577156947 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd538011750 con 0x7fd540140700 2022-06-29T15:50:06.429 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.412+0000 7fd5469e1700 1 -- 172.21.15.47:0/1577156947 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd54009b410 con 0x7fd540140700 2022-06-29T15:50:06.429 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.412+0000 7fd53d7fa700 1 -- 172.21.15.47:0/1577156947 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd538011ab0 con 0x7fd540140700 2022-06-29T15:50:06.430 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.412+0000 7fd5469e1700 1 -- 172.21.15.47:0/1577156947 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd540098730 con 0x7fd540140700 2022-06-29T15:50:06.431 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.416+0000 7fd5469e1700 1 -- 172.21.15.47:0/1577156947 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd54009a970 con 0x7fd540140700 2022-06-29T15:50:06.432 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.416+0000 7fd53d7fa700 1 -- 172.21.15.47:0/1577156947 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd538011c10 con 0x7fd540140700 2022-06-29T15:50:06.432 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.416+0000 7fd53d7fa700 1 --2- 172.21.15.47:0/1577156947 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd520053a70 0x7fd520055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:06.432 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.416+0000 7fd53d7fa700 1 -- 172.21.15.47:0/1577156947 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6068+0+0 (secure 0 0 0) 0x7fd53806bfa0 con 0x7fd540140700 2022-06-29T15:50:06.432 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.416+0000 7fd53ffff700 1 --2- 172.21.15.47:0/1577156947 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd520053a70 0x7fd520055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:06.433 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.416+0000 7fd53ffff700 1 --2- 172.21.15.47:0/1577156947 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd520053a70 0x7fd520055f30 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7fd53001b000 tx=0x7fd530016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:06.433 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.424+0000 7fd53d7fa700 1 -- 172.21.15.47:0/1577156947 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd538042ec0 con 0x7fd540140700 2022-06-29T15:50:06.559 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:06.552+0000 7fd5469e1700 1 -- 172.21.15.47:0/1577156947 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd54009a970 con 0x7fd540140700 2022-06-29T15:50:07.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.280+0000 7fd53d7fa700 1 -- 172.21.15.47:0/1577156947 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v104) v1 ==== 173+0+0 (secure 0 0 0) 0x7fd54009a970 con 0x7fd540140700 2022-06-29T15:50:07.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.284+0000 7fd5469e1700 1 -- 172.21.15.47:0/1577156947 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd520053a70 msgr2=0x7fd520055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:07.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.284+0000 7fd5469e1700 1 --2- 172.21.15.47:0/1577156947 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd520053a70 0x7fd520055f30 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7fd53001b000 tx=0x7fd530016000).stop 2022-06-29T15:50:07.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.284+0000 7fd5469e1700 1 -- 172.21.15.47:0/1577156947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd540140700 msgr2=0x7fd540155d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:07.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.284+0000 7fd5469e1700 1 --2- 172.21.15.47:0/1577156947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd540140700 0x7fd540155d00 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7fd53800c000 tx=0x7fd538006df0).stop 2022-06-29T15:50:07.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.284+0000 7fd5469e1700 1 -- 172.21.15.47:0/1577156947 shutdown_connections 2022-06-29T15:50:07.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.284+0000 7fd5469e1700 1 --2- 172.21.15.47:0/1577156947 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd520053a70 0x7fd520055f30 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:07.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.284+0000 7fd5469e1700 1 --2- 172.21.15.47:0/1577156947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd540140700 0x7fd540155d00 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:07.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.284+0000 7fd5469e1700 1 --2- 172.21.15.47:0/1577156947 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd540067470 0x7fd5400ade80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:07.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.284+0000 7fd5469e1700 1 --2- 172.21.15.47:0/1577156947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5400609d0 0x7fd5400b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:07.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.284+0000 7fd5469e1700 1 -- 172.21.15.47:0/1577156947 >> 172.21.15.47:0/1577156947 conn(0x7fd54005f0b0 msgr2=0x7fd54005f490 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:07.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.284+0000 7fd5469e1700 1 -- 172.21.15.47:0/1577156947 shutdown_connections 2022-06-29T15:50:07.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.284+0000 7fd5469e1700 1 -- 172.21.15.47:0/1577156947 wait complete. 2022-06-29T15:50:07.295 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' removed 2022-06-29T15:50:07.307 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-06-29T15:50:07.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.412+0000 7f474ffff700 1 Processor -- start 2022-06-29T15:50:07.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.412+0000 7f474ffff700 1 -- start start 2022-06-29T15:50:07.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.416+0000 7f474ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f47500b58d0 0x7f47500b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:07.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.416+0000 7f474ffff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f475014b4c0 0x7f47500aa130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:07.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.416+0000 7f474ffff700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4750060790 con 0x7f47500b1610 2022-06-29T15:50:07.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.416+0000 7f474ffff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47500a99e0 con 0x7f47500b58d0 2022-06-29T15:50:07.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.416+0000 7f474ffff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f475014ad80 con 0x7f475014b4c0 2022-06-29T15:50:07.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.416+0000 7f474e7fc700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f47500b1610 0x7f47501475f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37360/0 (socket says 172.21.15.47:37360) 2022-06-29T15:50:07.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.416+0000 7f474e7fc700 1 -- 172.21.15.47:0/2915813991 learned_addr learned my addr 172.21.15.47:0/2915813991 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:07.424 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.416+0000 7f474effd700 1 --2- 172.21.15.47:0/2915813991 >> v2:172.21.15.64:3300/0 conn(0x7f47500b58d0 0x7f47500b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:07.424 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.416+0000 7f474f7fe700 1 --2- 172.21.15.47:0/2915813991 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f475014b4c0 0x7f47500aa130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:07.424 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.416+0000 7f474dffb700 1 -- 172.21.15.47:0/2915813991 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1278687355 0 0) 0x7f4750060790 con 0x7f47500b1610 2022-06-29T15:50:07.424 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.416+0000 7f474dffb700 1 -- 172.21.15.47:0/2915813991 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4734003260 con 0x7f47500b1610 2022-06-29T15:50:07.425 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.416+0000 7f474f7fe700 1 -- 172.21.15.47:0/2915813991 >> v2:172.21.15.64:3300/0 conn(0x7f47500b58d0 msgr2=0x7f47500b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:07.425 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.416+0000 7f474f7fe700 1 --2- 172.21.15.47:0/2915813991 >> v2:172.21.15.64:3300/0 conn(0x7f47500b58d0 0x7f47500b5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:07.425 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.416+0000 7f474f7fe700 1 -- 172.21.15.47:0/2915813991 >> v1:172.21.15.47:6789/0 conn(0x7f47500b1610 legacy=0x7f47501475f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:07.425 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.416+0000 7f474dffb700 1 -- 172.21.15.47:0/2915813991 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 199984553 0 0) 0x7f4734003260 con 0x7f47500b1610 2022-06-29T15:50:07.426 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.416+0000 7f474f7fe700 1 -- 172.21.15.47:0/2915813991 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f475014b020 con 0x7f475014b4c0 2022-06-29T15:50:07.426 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.416+0000 7f474f7fe700 1 --2- 172.21.15.47:0/2915813991 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f475014b4c0 0x7f47500aa130 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f4740001800 tx=0x7f47480091d0).ready entity=mon.2 client_cookie=7375ee2138f3b8dd server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:07.427 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.416+0000 7f474dffb700 1 -- 172.21.15.47:0/2915813991 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4740001660 con 0x7f475014b4c0 2022-06-29T15:50:07.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.416+0000 7f474dffb700 1 -- 172.21.15.47:0/2915813991 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f474800f510 con 0x7f475014b4c0 2022-06-29T15:50:07.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.416+0000 7f474dffb700 1 -- 172.21.15.47:0/2915813991 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f474800f680 con 0x7f475014b4c0 2022-06-29T15:50:07.435 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.416+0000 7f474ffff700 1 -- 172.21.15.47:0/2915813991 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f475014b4c0 msgr2=0x7f47500aa130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:07.436 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.416+0000 7f474ffff700 1 --2- 172.21.15.47:0/2915813991 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f475014b4c0 0x7f47500aa130 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f4740001800 tx=0x7f47480091d0).stop 2022-06-29T15:50:07.436 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.416+0000 7f474ffff700 1 -- 172.21.15.47:0/2915813991 shutdown_connections 2022-06-29T15:50:07.436 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.416+0000 7f474ffff700 1 --2- 172.21.15.47:0/2915813991 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f475014b4c0 0x7f47500aa130 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:07.437 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.416+0000 7f474ffff700 1 --2- 172.21.15.47:0/2915813991 >> v2:172.21.15.64:3300/0 conn(0x7f47500b58d0 0x7f47500b5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:07.437 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.416+0000 7f474ffff700 1 -- 172.21.15.47:0/2915813991 >> 172.21.15.47:0/2915813991 conn(0x7f4750062560 msgr2=0x7f4750062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:07.437 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.416+0000 7f474ffff700 1 -- 172.21.15.47:0/2915813991 shutdown_connections 2022-06-29T15:50:07.437 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.416+0000 7f474ffff700 1 -- 172.21.15.47:0/2915813991 wait complete. 2022-06-29T15:50:07.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.420+0000 7f474ffff700 1 Processor -- start 2022-06-29T15:50:07.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.420+0000 7f474ffff700 1 -- start start 2022-06-29T15:50:07.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.420+0000 7f474ffff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47500b1610 0x7f475014e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:07.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.420+0000 7f474ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47500b58d0 0x7f47500af000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:07.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.420+0000 7f474ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f475014b4c0 0x7f47500ac6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:07.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.420+0000 7f474ffff700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4750069750 con 0x7f475014b4c0 2022-06-29T15:50:07.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.420+0000 7f474ffff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4750067140 con 0x7f47500b58d0 2022-06-29T15:50:07.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.420+0000 7f474ffff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47500aecc0 con 0x7f47500b1610 2022-06-29T15:50:07.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.420+0000 7f474f7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f475014b4c0 0x7f47500ac6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:07.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.420+0000 7f474f7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f475014b4c0 0x7f47500ac6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51810/0 (socket says 172.21.15.47:51810) 2022-06-29T15:50:07.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.420+0000 7f474f7fe700 1 -- 172.21.15.47:0/1709957223 learned_addr learned my addr 172.21.15.47:0/1709957223 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:07.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.420+0000 7f474effd700 1 --2- 172.21.15.47:0/1709957223 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47500b1610 0x7f475014e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:07.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.420+0000 7f474e7fc700 1 --2- 172.21.15.47:0/1709957223 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47500b58d0 0x7f47500af000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:07.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.420+0000 7f474f7fe700 1 -- 172.21.15.47:0/1709957223 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47500b1610 msgr2=0x7f475014e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:07.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.420+0000 7f474f7fe700 1 --2- 172.21.15.47:0/1709957223 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47500b1610 0x7f475014e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:07.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.420+0000 7f474f7fe700 1 -- 172.21.15.47:0/1709957223 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47500b58d0 msgr2=0x7f47500af000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:07.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.420+0000 7f474f7fe700 1 --2- 172.21.15.47:0/1709957223 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47500b58d0 0x7f47500af000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:07.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.420+0000 7f474f7fe700 1 -- 172.21.15.47:0/1709957223 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4748009470 con 0x7f475014b4c0 2022-06-29T15:50:07.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.420+0000 7f474f7fe700 1 --2- 172.21.15.47:0/1709957223 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f475014b4c0 0x7f47500ac6a0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 rx=0x7f4740001800 tx=0x7f4748016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:07.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.424+0000 7f473ffff700 1 -- 172.21.15.47:0/1709957223 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4748011c30 con 0x7f475014b4c0 2022-06-29T15:50:07.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.424+0000 7f473ffff700 1 -- 172.21.15.47:0/1709957223 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4748019070 con 0x7f475014b4c0 2022-06-29T15:50:07.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.424+0000 7f474ffff700 1 -- 172.21.15.47:0/1709957223 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4750064260 con 0x7f475014b4c0 2022-06-29T15:50:07.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.424+0000 7f473ffff700 1 -- 172.21.15.47:0/1709957223 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4748022460 con 0x7f475014b4c0 2022-06-29T15:50:07.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.424+0000 7f474ffff700 1 -- 172.21.15.47:0/1709957223 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47500ae040 con 0x7f475014b4c0 2022-06-29T15:50:07.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.424+0000 7f474ffff700 1 -- 172.21.15.47:0/1709957223 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47500573d0 con 0x7f475014b4c0 2022-06-29T15:50:07.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.424+0000 7f473ffff700 1 -- 172.21.15.47:0/1709957223 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f474801e070 con 0x7f475014b4c0 2022-06-29T15:50:07.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.424+0000 7f473ffff700 1 --2- 172.21.15.47:0/1709957223 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f472c053a10 0x7f472c055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:07.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.424+0000 7f473ffff700 1 -- 172.21.15.47:0/1709957223 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f474806cd60 con 0x7f475014b4c0 2022-06-29T15:50:07.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.424+0000 7f474effd700 1 --2- 172.21.15.47:0/1709957223 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f472c053a10 0x7f472c055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:07.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.428+0000 7f474effd700 1 --2- 172.21.15.47:0/1709957223 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f472c053a10 0x7f472c055ed0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7f474400b000 tx=0x7f4744006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:07.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.432+0000 7f473ffff700 1 -- 172.21.15.47:0/1709957223 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4748043db0 con 0x7f475014b4c0 2022-06-29T15:50:07.563 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.560+0000 7f474ffff700 1 -- 172.21.15.47:0/1709957223 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f47500643f0 con 0x7f475014b4c0 2022-06-29T15:50:07.572 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.568+0000 7f473ffff700 1 -- 172.21.15.47:0/1709957223 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f47480270a0 con 0x7f475014b4c0 2022-06-29T15:50:07.574 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.568+0000 7f474ffff700 1 -- 172.21.15.47:0/1709957223 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f472c053a10 msgr2=0x7f472c055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:07.574 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.568+0000 7f474ffff700 1 --2- 172.21.15.47:0/1709957223 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f472c053a10 0x7f472c055ed0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7f474400b000 tx=0x7f4744006df0).stop 2022-06-29T15:50:07.574 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.568+0000 7f474ffff700 1 -- 172.21.15.47:0/1709957223 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f475014b4c0 msgr2=0x7f47500ac6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:07.574 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.568+0000 7f474ffff700 1 --2- 172.21.15.47:0/1709957223 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f475014b4c0 0x7f47500ac6a0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 rx=0x7f4740001800 tx=0x7f4748016000).stop 2022-06-29T15:50:07.575 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.568+0000 7f474ffff700 1 -- 172.21.15.47:0/1709957223 shutdown_connections 2022-06-29T15:50:07.575 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.568+0000 7f474ffff700 1 --2- 172.21.15.47:0/1709957223 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f475014b4c0 0x7f47500ac6a0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:07.575 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.568+0000 7f474ffff700 1 --2- 172.21.15.47:0/1709957223 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f472c053a10 0x7f472c055ed0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:07.575 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.568+0000 7f474ffff700 1 --2- 172.21.15.47:0/1709957223 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47500b58d0 0x7f47500af000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:07.576 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.568+0000 7f474ffff700 1 --2- 172.21.15.47:0/1709957223 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47500b1610 0x7f475014e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:07.576 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.568+0000 7f474ffff700 1 -- 172.21.15.47:0/1709957223 >> 172.21.15.47:0/1709957223 conn(0x7f4750062560 msgr2=0x7f47500b03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:07.576 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.568+0000 7f474ffff700 1 -- 172.21.15.47:0/1709957223 shutdown_connections 2022-06-29T15:50:07.576 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.568+0000 7f474ffff700 1 -- 172.21.15.47:0/1709957223 wait complete. 2022-06-29T15:50:07.587 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical' 2022-06-29T15:50:07.689 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.684+0000 7fb1126b9700 1 Processor -- start 2022-06-29T15:50:07.690 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.684+0000 7fb1126b9700 1 -- start start 2022-06-29T15:50:07.695 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.684+0000 7fb1126b9700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb10c0b5c60 0x7fb10c0afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:07.695 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.684+0000 7fb1126b9700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb10c0601a0 0x7fb10c0ab7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:07.695 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.684+0000 7fb1126b9700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb10c0b6040 con 0x7fb10c060a60 2022-06-29T15:50:07.696 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.684+0000 7fb1126b9700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb10c0b11a0 con 0x7fb10c0601a0 2022-06-29T15:50:07.696 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.684+0000 7fb1126b9700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c0b2900 con 0x7fb10c0b5c60 2022-06-29T15:50:07.696 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.684+0000 7fb110eb6700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb10c060a60 0x7fb10c0aa080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37370/0 (socket says 172.21.15.47:37370) 2022-06-29T15:50:07.697 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.684+0000 7fb110eb6700 1 -- 172.21.15.47:0/1099201705 learned_addr learned my addr 172.21.15.47:0/1099201705 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:07.697 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.684+0000 7fb1116b7700 1 --2- 172.21.15.47:0/1099201705 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb10c0b5c60 0x7fb10c0afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:07.697 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.684+0000 7fb103fff700 1 -- 172.21.15.47:0/1099201705 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 163977880 0 0) 0x7fb10c0b6040 con 0x7fb10c060a60 2022-06-29T15:50:07.697 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.684+0000 7fb111eb8700 1 --2- 172.21.15.47:0/1099201705 >> v2:172.21.15.64:3300/0 conn(0x7fb10c0601a0 0x7fb10c0ab7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:07.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.684+0000 7fb103fff700 1 -- 172.21.15.47:0/1099201705 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0f0003260 con 0x7fb10c060a60 2022-06-29T15:50:07.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.684+0000 7fb103fff700 1 -- 172.21.15.47:0/1099201705 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1310461783 0 0) 0x7fb0f0003260 con 0x7fb10c060a60 2022-06-29T15:50:07.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.684+0000 7fb103fff700 1 -- 172.21.15.47:0/1099201705 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb10c0b5c60 msgr2=0x7fb10c0afdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:07.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.684+0000 7fb103fff700 1 --2- 172.21.15.47:0/1099201705 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb10c0b5c60 0x7fb10c0afdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:07.699 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.688+0000 7fb103fff700 1 -- 172.21.15.47:0/1099201705 >> v2:172.21.15.64:3300/0 conn(0x7fb10c0601a0 msgr2=0x7fb10c0ab7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:07.699 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.688+0000 7fb103fff700 1 --2- 172.21.15.47:0/1099201705 >> v2:172.21.15.64:3300/0 conn(0x7fb10c0601a0 0x7fb10c0ab7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:07.699 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.688+0000 7fb103fff700 1 -- 172.21.15.47:0/1099201705 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb10c0b2ba0 con 0x7fb10c060a60 2022-06-29T15:50:07.699 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.688+0000 7fb103fff700 1 -- 172.21.15.47:0/1099201705 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7fb108001660 con 0x7fb10c060a60 2022-06-29T15:50:07.700 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.688+0000 7fb111eb8700 1 --2- 172.21.15.47:0/1099201705 >> v2:172.21.15.64:3300/0 conn(0x7fb10c0601a0 0x7fb10c0ab7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:50:07.700 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.688+0000 7fb103fff700 1 -- 172.21.15.47:0/1099201705 >> v1:172.21.15.47:6789/0 conn(0x7fb10c060a60 legacy=0x7fb10c0aa080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:07.700 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.688+0000 7fb103fff700 1 --2- 172.21.15.47:0/1099201705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb0f0003750 0x7fb0f0003b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:07.700 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.688+0000 7fb103fff700 1 --2- 172.21.15.47:0/1099201705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0f0004500 0x7fb0f0004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:07.700 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.688+0000 7fb103fff700 1 --2- 172.21.15.47:0/1099201705 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb10c0b5c60 0x7fb0f0004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:07.701 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.688+0000 7fb103fff700 1 -- 172.21.15.47:0/1099201705 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c0b11a0 con 0x7fb0f0003750 2022-06-29T15:50:07.701 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.688+0000 7fb103fff700 1 -- 172.21.15.47:0/1099201705 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c0b2900 con 0x7fb0f0004500 2022-06-29T15:50:07.701 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.688+0000 7fb103fff700 1 -- 172.21.15.47:0/1099201705 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f000b750 con 0x7fb10c0b5c60 2022-06-29T15:50:07.701 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.688+0000 7fb1116b7700 1 --2- 172.21.15.47:0/1099201705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb0f0003750 0x7fb0f0003b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:07.702 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.688+0000 7fb111eb8700 1 --2- 172.21.15.47:0/1099201705 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb10c0b5c60 0x7fb0f0004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:07.702 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.688+0000 7fb110eb6700 1 --2- 172.21.15.47:0/1099201705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0f0004500 0x7fb0f0004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:07.702 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.688+0000 7fb1116b7700 1 -- 172.21.15.47:0/1099201705 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb10c0b5c60 msgr2=0x7fb0f0004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:07.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.688+0000 7fb1116b7700 1 --2- 172.21.15.47:0/1099201705 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb10c0b5c60 0x7fb0f0004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:07.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.688+0000 7fb1116b7700 1 -- 172.21.15.47:0/1099201705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0f0004500 msgr2=0x7fb0f0004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:07.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.688+0000 7fb1116b7700 1 --2- 172.21.15.47:0/1099201705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0f0004500 0x7fb0f0004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:07.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.688+0000 7fb1116b7700 1 -- 172.21.15.47:0/1099201705 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f0003260 con 0x7fb0f0003750 2022-06-29T15:50:07.704 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.688+0000 7fb1116b7700 1 --2- 172.21.15.47:0/1099201705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb0f0003750 0x7fb0f0003b90 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 rx=0x7fb0fc00c000 tx=0x7fb0fc006d90).ready entity=mon.0 client_cookie=3321336d44ea9de6 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:07.704 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.688+0000 7fb103fff700 1 -- 172.21.15.47:0/1099201705 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb0fc013450 con 0x7fb0f0003750 2022-06-29T15:50:07.704 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.688+0000 7fb103fff700 1 -- 172.21.15.47:0/1099201705 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0fc0135b0 con 0x7fb0f0003750 2022-06-29T15:50:07.704 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.688+0000 7fb103fff700 1 -- 172.21.15.47:0/1099201705 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb0fc013910 con 0x7fb0f0003750 2022-06-29T15:50:07.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.688+0000 7fb1126b9700 1 -- 172.21.15.47:0/1099201705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb0f0003750 msgr2=0x7fb0f0003b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:07.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.688+0000 7fb1126b9700 1 --2- 172.21.15.47:0/1099201705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb0f0003750 0x7fb0f0003b90 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 rx=0x7fb0fc00c000 tx=0x7fb0fc006d90).stop 2022-06-29T15:50:07.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.688+0000 7fb111eb8700 1 -- 172.21.15.47:0/1099201705 reap_dead start 2022-06-29T15:50:07.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.688+0000 7fb1126b9700 1 -- 172.21.15.47:0/1099201705 shutdown_connections 2022-06-29T15:50:07.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.688+0000 7fb1126b9700 1 -- 172.21.15.47:0/1099201705 >> 172.21.15.47:0/1099201705 conn(0x7fb10c062560 msgr2=0x7fb10c062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:07.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.688+0000 7fb1126b9700 1 -- 172.21.15.47:0/1099201705 shutdown_connections 2022-06-29T15:50:07.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.688+0000 7fb1126b9700 1 -- 172.21.15.47:0/1099201705 wait complete. 2022-06-29T15:50:07.707 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.688+0000 7fb1126b9700 1 Processor -- start 2022-06-29T15:50:07.707 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.688+0000 7fb1126b9700 1 -- start start 2022-06-29T15:50:07.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.692+0000 7fb1126b9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb0f0003750 0x7fb10c147180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:07.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.692+0000 7fb1126b9700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb10c0651a0 0x7fb10c150a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:07.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.692+0000 7fb1126b9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb10c063240 0x7fb10c0a83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:07.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.692+0000 7fb1126b9700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c09ee60 con 0x7fb0f0003750 2022-06-29T15:50:07.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.692+0000 7fb1126b9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c150080 con 0x7fb10c063240 2022-06-29T15:50:07.712 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.692+0000 7fb1126b9700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c150320 con 0x7fb10c0651a0 2022-06-29T15:50:07.712 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.692+0000 7fb1116b7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb0f0003750 0x7fb10c147180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:07.712 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.692+0000 7fb1116b7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb0f0003750 0x7fb10c147180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51828/0 (socket says 172.21.15.47:51828) 2022-06-29T15:50:07.712 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.692+0000 7fb1116b7700 1 -- 172.21.15.47:0/4145122186 learned_addr learned my addr 172.21.15.47:0/4145122186 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:07.712 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.692+0000 7fb110eb6700 1 --2- 172.21.15.47:0/4145122186 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb10c0651a0 0x7fb10c150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:07.713 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.692+0000 7fb111eb8700 1 --2- 172.21.15.47:0/4145122186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb10c063240 0x7fb10c0a83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:07.713 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.692+0000 7fb1116b7700 1 -- 172.21.15.47:0/4145122186 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb10c0651a0 msgr2=0x7fb10c150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:07.714 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.692+0000 7fb1116b7700 1 --2- 172.21.15.47:0/4145122186 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb10c0651a0 0x7fb10c150a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:07.714 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.692+0000 7fb1116b7700 1 -- 172.21.15.47:0/4145122186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb10c063240 msgr2=0x7fb10c0a83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:07.714 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.692+0000 7fb1116b7700 1 --2- 172.21.15.47:0/4145122186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb10c063240 0x7fb10c0a83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:07.714 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.692+0000 7fb1116b7700 1 -- 172.21.15.47:0/4145122186 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb10c1505c0 con 0x7fb0f0003750 2022-06-29T15:50:07.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.692+0000 7fb110eb6700 1 --2- 172.21.15.47:0/4145122186 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb10c0651a0 0x7fb10c150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:50:07.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.692+0000 7fb1116b7700 1 --2- 172.21.15.47:0/4145122186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb0f0003750 0x7fb10c147180 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 rx=0x7fb0fc00c000 tx=0x7fb0fc017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:07.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.692+0000 7fb1027fc700 1 -- 172.21.15.47:0/4145122186 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb0fc023070 con 0x7fb0f0003750 2022-06-29T15:50:07.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.692+0000 7fb1027fc700 1 -- 172.21.15.47:0/4145122186 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0fc000e90 con 0x7fb0f0003750 2022-06-29T15:50:07.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.692+0000 7fb1126b9700 1 -- 172.21.15.47:0/4145122186 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb10c0a0db0 con 0x7fb0f0003750 2022-06-29T15:50:07.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.692+0000 7fb1027fc700 1 -- 172.21.15.47:0/4145122186 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb0fc013450 con 0x7fb0f0003750 2022-06-29T15:50:07.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.692+0000 7fb1126b9700 1 -- 172.21.15.47:0/4145122186 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb10c064da0 con 0x7fb0f0003750 2022-06-29T15:50:07.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.696+0000 7fb1027fc700 1 -- 172.21.15.47:0/4145122186 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb0fc01f450 con 0x7fb0f0003750 2022-06-29T15:50:07.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.696+0000 7fb1126b9700 1 -- 172.21.15.47:0/4145122186 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb10c150750 con 0x7fb0f0003750 2022-06-29T15:50:07.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.696+0000 7fb1027fc700 1 --2- 172.21.15.47:0/4145122186 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb0f4053a70 0x7fb0f4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:07.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.696+0000 7fb1027fc700 1 -- 172.21.15.47:0/4145122186 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fb0fc06d8b0 con 0x7fb0f0003750 2022-06-29T15:50:07.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.696+0000 7fb110eb6700 1 --2- 172.21.15.47:0/4145122186 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb0f4053a70 0x7fb0f4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:07.718 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.696+0000 7fb110eb6700 1 --2- 172.21.15.47:0/4145122186 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb0f4053a70 0x7fb0f4055f30 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7fb10800d000 tx=0x7fb108009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:07.718 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.700+0000 7fb1027fc700 1 -- 172.21.15.47:0/4145122186 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb0fc0447a0 con 0x7fb0f0003750 2022-06-29T15:50:07.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:07.872+0000 7fb1126b9700 1 -- 172.21.15.47:0/4145122186 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical"]} v 0) v1 -- 0x7fb10c0573d0 con 0x7fb0f0003750 2022-06-29T15:50:08.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.280+0000 7fb1027fc700 1 -- 172.21.15.47:0/4145122186 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical"]}]=0 v338) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb0fc047dc0 con 0x7fb0f0003750 2022-06-29T15:50:08.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.284+0000 7fb1126b9700 1 -- 172.21.15.47:0/4145122186 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb0f4053a70 msgr2=0x7fb0f4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:08.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.284+0000 7fb1126b9700 1 --2- 172.21.15.47:0/4145122186 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb0f4053a70 0x7fb0f4055f30 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7fb10800d000 tx=0x7fb108009040).stop 2022-06-29T15:50:08.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.284+0000 7fb1126b9700 1 -- 172.21.15.47:0/4145122186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb0f0003750 msgr2=0x7fb10c147180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:08.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.284+0000 7fb1126b9700 1 --2- 172.21.15.47:0/4145122186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb0f0003750 0x7fb10c147180 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 rx=0x7fb0fc00c000 tx=0x7fb0fc017000).stop 2022-06-29T15:50:08.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.284+0000 7fb1126b9700 1 -- 172.21.15.47:0/4145122186 shutdown_connections 2022-06-29T15:50:08.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.284+0000 7fb1126b9700 1 --2- 172.21.15.47:0/4145122186 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb0f4053a70 0x7fb0f4055f30 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:08.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.284+0000 7fb1126b9700 1 --2- 172.21.15.47:0/4145122186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb10c063240 0x7fb10c0a83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:08.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.284+0000 7fb1126b9700 1 --2- 172.21.15.47:0/4145122186 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb10c0651a0 0x7fb10c150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:08.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.284+0000 7fb1126b9700 1 --2- 172.21.15.47:0/4145122186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb0f0003750 0x7fb10c147180 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:08.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.284+0000 7fb1126b9700 1 -- 172.21.15.47:0/4145122186 >> 172.21.15.47:0/4145122186 conn(0x7fb10c062560 msgr2=0x7fb10c0b3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:08.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.284+0000 7fb1126b9700 1 -- 172.21.15.47:0/4145122186 shutdown_connections 2022-06-29T15:50:08.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.284+0000 7fb1126b9700 1 -- 172.21.15.47:0/4145122186 wait complete. 2022-06-29T15:50:08.307 INFO:tasks.cephfs_test_runner:test_dump_inode_hexademical (tasks.cephfs.test_misc.TestMisc) ... ok 2022-06-29T15:50:08.308 INFO:tasks.cephfs_test_runner:Starting test: test_filtered_df (tasks.cephfs.test_misc.TestMisc) 2022-06-29T15:50:08.309 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_filtered_df' 2022-06-29T15:50:08.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.412+0000 7fa69a430700 1 Processor -- start 2022-06-29T15:50:08.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.412+0000 7fa69a430700 1 -- start start 2022-06-29T15:50:08.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.412+0000 7fa69a430700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa6940b2ee0 0x7fa6940b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:08.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.412+0000 7fa69a430700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa6940b0e50 0x7fa69413d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:08.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.412+0000 7fa69a430700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6941400e0 con 0x7fa6940ad970 2022-06-29T15:50:08.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.412+0000 7fa69a430700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa694140ef0 con 0x7fa6940b0e50 2022-06-29T15:50:08.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.412+0000 7fa69a430700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa69413fef0 con 0x7fa6940b2ee0 2022-06-29T15:50:08.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.412+0000 7fa698c2d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa6940ad970 0x7fa6941407e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37392/0 (socket says 172.21.15.47:37392) 2022-06-29T15:50:08.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.412+0000 7fa698c2d700 1 -- 172.21.15.47:0/1480515117 learned_addr learned my addr 172.21.15.47:0/1480515117 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:08.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.412+0000 7fa69942e700 1 --2- 172.21.15.47:0/1480515117 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa6940b2ee0 0x7fa6940b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:08.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.412+0000 7fa699c2f700 1 --2- 172.21.15.47:0/1480515117 >> v2:172.21.15.64:3300/0 conn(0x7fa6940b0e50 0x7fa69413d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:08.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.412+0000 7fa68bfff700 1 -- 172.21.15.47:0/1480515117 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1516667065 0 0) 0x7fa6941400e0 con 0x7fa6940ad970 2022-06-29T15:50:08.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.412+0000 7fa68bfff700 1 -- 172.21.15.47:0/1480515117 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa678003260 con 0x7fa6940ad970 2022-06-29T15:50:08.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.412+0000 7fa69942e700 1 -- 172.21.15.47:0/1480515117 >> v2:172.21.15.64:3300/0 conn(0x7fa6940b0e50 msgr2=0x7fa69413d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:08.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.412+0000 7fa69942e700 1 --2- 172.21.15.47:0/1480515117 >> v2:172.21.15.64:3300/0 conn(0x7fa6940b0e50 0x7fa69413d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:08.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.412+0000 7fa69942e700 1 -- 172.21.15.47:0/1480515117 >> v1:172.21.15.47:6789/0 conn(0x7fa6940ad970 legacy=0x7fa6941407e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:08.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.412+0000 7fa69942e700 1 -- 172.21.15.47:0/1480515117 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa694143570 con 0x7fa6940b2ee0 2022-06-29T15:50:08.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.412+0000 7fa69942e700 1 --2- 172.21.15.47:0/1480515117 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa6940b2ee0 0x7fa6940b32c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7fa690002f70 tx=0x7fa690006df0).ready entity=mon.2 client_cookie=7934ae9d7e96d699 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:08.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.416+0000 7fa68bfff700 1 -- 172.21.15.47:0/1480515117 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa69000d4a0 con 0x7fa6940b2ee0 2022-06-29T15:50:08.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.416+0000 7fa68bfff700 1 -- 172.21.15.47:0/1480515117 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa690000e90 con 0x7fa6940b2ee0 2022-06-29T15:50:08.424 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.416+0000 7fa68bfff700 1 -- 172.21.15.47:0/1480515117 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa69000d7f0 con 0x7fa6940b2ee0 2022-06-29T15:50:08.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.416+0000 7fa69a430700 1 -- 172.21.15.47:0/1480515117 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa6940b2ee0 msgr2=0x7fa6940b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:08.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.416+0000 7fa69a430700 1 --2- 172.21.15.47:0/1480515117 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa6940b2ee0 0x7fa6940b32c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7fa690002f70 tx=0x7fa690006df0).stop 2022-06-29T15:50:08.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.416+0000 7fa69a430700 1 -- 172.21.15.47:0/1480515117 shutdown_connections 2022-06-29T15:50:08.429 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.416+0000 7fa69a430700 1 --2- 172.21.15.47:0/1480515117 >> v2:172.21.15.64:3300/0 conn(0x7fa6940b0e50 0x7fa69413d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:08.429 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.416+0000 7fa69a430700 1 --2- 172.21.15.47:0/1480515117 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa6940b2ee0 0x7fa6940b32c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:08.429 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.416+0000 7fa69a430700 1 -- 172.21.15.47:0/1480515117 >> 172.21.15.47:0/1480515117 conn(0x7fa694063670 msgr2=0x7fa694063a80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:08.429 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.416+0000 7fa69a430700 1 -- 172.21.15.47:0/1480515117 shutdown_connections 2022-06-29T15:50:08.429 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.416+0000 7fa69a430700 1 -- 172.21.15.47:0/1480515117 wait complete. 2022-06-29T15:50:08.430 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.416+0000 7fa69a430700 1 Processor -- start 2022-06-29T15:50:08.430 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.416+0000 7fa69a430700 1 -- start start 2022-06-29T15:50:08.430 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.420+0000 7fa69a430700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6940ad970 0x7fa69405f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:08.430 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.420+0000 7fa69a430700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6940b0e50 0x7fa6940abbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:08.431 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.420+0000 7fa69a430700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa6940b2ee0 0x7fa694153780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:08.431 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.420+0000 7fa69a430700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6940ac4d0 con 0x7fa6940ad970 2022-06-29T15:50:08.431 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.420+0000 7fa69a430700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6940aab10 con 0x7fa6940b0e50 2022-06-29T15:50:08.432 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.420+0000 7fa69a430700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6940aadb0 con 0x7fa6940b2ee0 2022-06-29T15:50:08.432 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.420+0000 7fa69942e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6940ad970 0x7fa69405f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:08.432 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.420+0000 7fa69942e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6940ad970 0x7fa69405f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51842/0 (socket says 172.21.15.47:51842) 2022-06-29T15:50:08.433 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.420+0000 7fa69942e700 1 -- 172.21.15.47:0/3567662865 learned_addr learned my addr 172.21.15.47:0/3567662865 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:08.433 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.420+0000 7fa698c2d700 1 --2- 172.21.15.47:0/3567662865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6940b0e50 0x7fa6940abbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:08.433 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.420+0000 7fa699c2f700 1 --2- 172.21.15.47:0/3567662865 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa6940b2ee0 0x7fa694153780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:08.433 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.420+0000 7fa69942e700 1 -- 172.21.15.47:0/3567662865 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa6940b2ee0 msgr2=0x7fa694153780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:08.434 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.420+0000 7fa69942e700 1 --2- 172.21.15.47:0/3567662865 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa6940b2ee0 0x7fa694153780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:08.434 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.420+0000 7fa69942e700 1 -- 172.21.15.47:0/3567662865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6940b0e50 msgr2=0x7fa6940abbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:08.434 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.420+0000 7fa69942e700 1 --2- 172.21.15.47:0/3567662865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6940b0e50 0x7fa6940abbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:08.435 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.420+0000 7fa69942e700 1 -- 172.21.15.47:0/3567662865 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa69000b180 con 0x7fa6940ad970 2022-06-29T15:50:08.435 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.420+0000 7fa69942e700 1 --2- 172.21.15.47:0/3567662865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6940ad970 0x7fa69405f3f0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 rx=0x7fa690002f70 tx=0x7fa690014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:08.435 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.420+0000 7fa68a7fc700 1 -- 172.21.15.47:0/3567662865 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa690012930 con 0x7fa6940ad970 2022-06-29T15:50:08.435 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.420+0000 7fa68a7fc700 1 -- 172.21.15.47:0/3567662865 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa690012a90 con 0x7fa6940ad970 2022-06-29T15:50:08.436 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.420+0000 7fa69a430700 1 -- 172.21.15.47:0/3567662865 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6940ab050 con 0x7fa6940ad970 2022-06-29T15:50:08.436 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.420+0000 7fa68a7fc700 1 -- 172.21.15.47:0/3567662865 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa690012df0 con 0x7fa6940ad970 2022-06-29T15:50:08.436 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.420+0000 7fa69a430700 1 -- 172.21.15.47:0/3567662865 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6940aa3c0 con 0x7fa6940ad970 2022-06-29T15:50:08.436 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.420+0000 7fa69a430700 1 -- 172.21.15.47:0/3567662865 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6940ab1e0 con 0x7fa6940ad970 2022-06-29T15:50:08.437 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.424+0000 7fa68a7fc700 1 -- 172.21.15.47:0/3567662865 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa69001c070 con 0x7fa6940ad970 2022-06-29T15:50:08.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.424+0000 7fa68a7fc700 1 --2- 172.21.15.47:0/3567662865 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa67c053a10 0x7fa67c055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:08.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.424+0000 7fa698c2d700 1 --2- 172.21.15.47:0/3567662865 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa67c053a10 0x7fa67c055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:08.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.424+0000 7fa68a7fc700 1 -- 172.21.15.47:0/3567662865 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fa69006c9a0 con 0x7fa6940ad970 2022-06-29T15:50:08.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.424+0000 7fa698c2d700 1 --2- 172.21.15.47:0/3567662865 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa67c053a10 0x7fa67c055ed0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7fa68400d000 tx=0x7fa684009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:08.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.428+0000 7fa68a7fc700 1 -- 172.21.15.47:0/3567662865 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa690043960 con 0x7fa6940ad970 2022-06-29T15:50:08.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:08.600+0000 7fa69a430700 1 -- 172.21.15.47:0/3567662865 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_filtered_df"]} v 0) v1 -- 0x7fa6940573d0 con 0x7fa6940ad970 2022-06-29T15:50:09.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.292+0000 7fa68a7fc700 1 -- 172.21.15.47:0/3567662865 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_filtered_df"]}]=0 v339) v1 ==== 130+0+0 (secure 0 0 0) 0x7fa690019070 con 0x7fa6940ad970 2022-06-29T15:50:09.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.292+0000 7fa69a430700 1 -- 172.21.15.47:0/3567662865 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa67c053a10 msgr2=0x7fa67c055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:09.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.292+0000 7fa69a430700 1 --2- 172.21.15.47:0/3567662865 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa67c053a10 0x7fa67c055ed0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7fa68400d000 tx=0x7fa684009040).stop 2022-06-29T15:50:09.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.292+0000 7fa69a430700 1 -- 172.21.15.47:0/3567662865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6940ad970 msgr2=0x7fa69405f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:09.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.292+0000 7fa69a430700 1 --2- 172.21.15.47:0/3567662865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6940ad970 0x7fa69405f3f0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 rx=0x7fa690002f70 tx=0x7fa690014000).stop 2022-06-29T15:50:09.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.296+0000 7fa69a430700 1 -- 172.21.15.47:0/3567662865 shutdown_connections 2022-06-29T15:50:09.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.296+0000 7fa69a430700 1 --2- 172.21.15.47:0/3567662865 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa6940b2ee0 0x7fa694153780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:09.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.296+0000 7fa69a430700 1 --2- 172.21.15.47:0/3567662865 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa67c053a10 0x7fa67c055ed0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:09.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.296+0000 7fa69a430700 1 --2- 172.21.15.47:0/3567662865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6940b0e50 0x7fa6940abbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:09.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.296+0000 7fa69a430700 1 --2- 172.21.15.47:0/3567662865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6940ad970 0x7fa69405f3f0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:09.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.296+0000 7fa69a430700 1 -- 172.21.15.47:0/3567662865 >> 172.21.15.47:0/3567662865 conn(0x7fa694063670 msgr2=0x7fa694063a50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:09.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.296+0000 7fa69a430700 1 -- 172.21.15.47:0/3567662865 shutdown_connections 2022-06-29T15:50:09.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.296+0000 7fa69a430700 1 -- 172.21.15.47:0/3567662865 wait complete. 2022-06-29T15:50:09.315 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-06-29T15:50:09.432 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.428+0000 7fadbd3e4700 1 Processor -- start 2022-06-29T15:50:09.433 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.428+0000 7fadbd3e4700 1 -- start start 2022-06-29T15:50:09.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.428+0000 7fadbd3e4700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fadb805f4a0 0x7fadb80b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:09.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.428+0000 7fadbd3e4700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fadb809a1d0 0x7fadb809a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:09.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.428+0000 7fadbd3e4700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadb809ab20 con 0x7fadb8142a30 2022-06-29T15:50:09.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.428+0000 7fadbd3e4700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadb805f880 con 0x7fadb809a1d0 2022-06-29T15:50:09.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.428+0000 7fadbd3e4700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadb809ae90 con 0x7fadb805f4a0 2022-06-29T15:50:09.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.428+0000 7fadbcbe3700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fadb8142a30 0x7fadb80b0550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37406/0 (socket says 172.21.15.47:37406) 2022-06-29T15:50:09.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.428+0000 7fadbcbe3700 1 -- 172.21.15.47:0/2584727516 learned_addr learned my addr 172.21.15.47:0/2584727516 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:09.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.428+0000 7fadb77fe700 1 --2- 172.21.15.47:0/2584727516 >> v2:172.21.15.64:3300/0 conn(0x7fadb809a1d0 0x7fadb809a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:09.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.428+0000 7fadb7fff700 1 --2- 172.21.15.47:0/2584727516 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fadb805f4a0 0x7fadb80b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:09.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.428+0000 7fadb6ffd700 1 -- 172.21.15.47:0/2584727516 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1021959568 0 0) 0x7fadb0001490 con 0x7fadb8142a30 2022-06-29T15:50:09.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.428+0000 7fadb6ffd700 1 -- 172.21.15.47:0/2584727516 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fada4003260 con 0x7fadb8142a30 2022-06-29T15:50:09.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.428+0000 7fadb7fff700 1 -- 172.21.15.47:0/2584727516 >> v2:172.21.15.64:3300/0 conn(0x7fadb809a1d0 msgr2=0x7fadb809a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:09.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.428+0000 7fadb7fff700 1 --2- 172.21.15.47:0/2584727516 >> v2:172.21.15.64:3300/0 conn(0x7fadb809a1d0 0x7fadb809a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:09.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.428+0000 7fadb7fff700 1 -- 172.21.15.47:0/2584727516 >> v1:172.21.15.47:6789/0 conn(0x7fadb8142a30 legacy=0x7fadb80b0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:09.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.428+0000 7fadb7fff700 1 -- 172.21.15.47:0/2584727516 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb809b130 con 0x7fadb805f4a0 2022-06-29T15:50:09.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.432+0000 7fadb7fff700 1 --2- 172.21.15.47:0/2584727516 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fadb805f4a0 0x7fadb80b2cc0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7fada8002f70 tx=0x7fada8006df0).ready entity=mon.2 client_cookie=46d87934c09a35f1 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:09.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.432+0000 7fadb6ffd700 1 -- 172.21.15.47:0/2584727516 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fada800d4a0 con 0x7fadb805f4a0 2022-06-29T15:50:09.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.432+0000 7fadb6ffd700 1 -- 172.21.15.47:0/2584727516 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fada8000e90 con 0x7fadb805f4a0 2022-06-29T15:50:09.446 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.432+0000 7fadb6ffd700 1 -- 172.21.15.47:0/2584727516 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fada800d7f0 con 0x7fadb805f4a0 2022-06-29T15:50:09.446 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.432+0000 7fadbd3e4700 1 -- 172.21.15.47:0/2584727516 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fadb805f4a0 msgr2=0x7fadb80b2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:09.446 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.432+0000 7fadbd3e4700 1 --2- 172.21.15.47:0/2584727516 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fadb805f4a0 0x7fadb80b2cc0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7fada8002f70 tx=0x7fada8006df0).stop 2022-06-29T15:50:09.446 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.432+0000 7fadbd3e4700 1 -- 172.21.15.47:0/2584727516 shutdown_connections 2022-06-29T15:50:09.447 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.432+0000 7fadbd3e4700 1 --2- 172.21.15.47:0/2584727516 >> v2:172.21.15.64:3300/0 conn(0x7fadb809a1d0 0x7fadb809a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:09.447 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.432+0000 7fadbd3e4700 1 --2- 172.21.15.47:0/2584727516 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fadb805f4a0 0x7fadb80b2cc0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:09.447 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.432+0000 7fadbd3e4700 1 -- 172.21.15.47:0/2584727516 >> 172.21.15.47:0/2584727516 conn(0x7fadb8062df0 msgr2=0x7fadb80631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:09.447 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.432+0000 7fadbd3e4700 1 -- 172.21.15.47:0/2584727516 shutdown_connections 2022-06-29T15:50:09.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.432+0000 7fadbd3e4700 1 -- 172.21.15.47:0/2584727516 wait complete. 2022-06-29T15:50:09.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.432+0000 7fadbd3e4700 1 Processor -- start 2022-06-29T15:50:09.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.432+0000 7fadbd3e4700 1 -- start start 2022-06-29T15:50:09.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.432+0000 7fadbd3e4700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fadb805f4a0 0x7fadb8145b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:09.449 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.432+0000 7fadbd3e4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fadb809a1d0 0x7fadb8144870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:09.449 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.432+0000 7fadbd3e4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadb8142a30 0x7fadb813c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:09.449 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.432+0000 7fadbd3e4700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb8143f60 con 0x7fadb809a1d0 2022-06-29T15:50:09.450 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.432+0000 7fadbd3e4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb8144470 con 0x7fadb8142a30 2022-06-29T15:50:09.450 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.432+0000 7fadbd3e4700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadb8144150 con 0x7fadb805f4a0 2022-06-29T15:50:09.450 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.432+0000 7fadb77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fadb809a1d0 0x7fadb8144870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:09.450 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.432+0000 7fadb77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fadb809a1d0 0x7fadb8144870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51856/0 (socket says 172.21.15.47:51856) 2022-06-29T15:50:09.451 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.432+0000 7fadb77fe700 1 -- 172.21.15.47:0/1054686374 learned_addr learned my addr 172.21.15.47:0/1054686374 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:09.451 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.432+0000 7fadb7fff700 1 --2- 172.21.15.47:0/1054686374 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fadb805f4a0 0x7fadb8145b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:09.451 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.436+0000 7fadbcbe3700 1 --2- 172.21.15.47:0/1054686374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadb8142a30 0x7fadb813c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:09.451 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.436+0000 7fadb77fe700 1 -- 172.21.15.47:0/1054686374 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fadb805f4a0 msgr2=0x7fadb8145b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:09.452 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.436+0000 7fadb77fe700 1 --2- 172.21.15.47:0/1054686374 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fadb805f4a0 0x7fadb8145b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:09.452 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.436+0000 7fadb77fe700 1 -- 172.21.15.47:0/1054686374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadb8142a30 msgr2=0x7fadb813c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:09.452 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.436+0000 7fadb77fe700 1 --2- 172.21.15.47:0/1054686374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadb8142a30 0x7fadb813c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:09.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.436+0000 7fadb77fe700 1 -- 172.21.15.47:0/1054686374 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fada800b180 con 0x7fadb809a1d0 2022-06-29T15:50:09.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.436+0000 7fadb77fe700 1 --2- 172.21.15.47:0/1054686374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fadb809a1d0 0x7fadb8144870 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 rx=0x7fadb81442e0 tx=0x7fadac008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:09.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.436+0000 7fadb57fa700 1 -- 172.21.15.47:0/1054686374 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fadac00aa20 con 0x7fadb809a1d0 2022-06-29T15:50:09.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.436+0000 7fadb57fa700 1 -- 172.21.15.47:0/1054686374 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadac00ab80 con 0x7fadb809a1d0 2022-06-29T15:50:09.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.436+0000 7fadbd3e4700 1 -- 172.21.15.47:0/1054686374 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadb8145470 con 0x7fadb809a1d0 2022-06-29T15:50:09.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.436+0000 7fadb57fa700 1 -- 172.21.15.47:0/1054686374 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fadac00ee90 con 0x7fadb809a1d0 2022-06-29T15:50:09.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.436+0000 7fadbd3e4700 1 -- 172.21.15.47:0/1054686374 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadb8146740 con 0x7fadb809a1d0 2022-06-29T15:50:09.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.436+0000 7fadbd3e4700 1 -- 172.21.15.47:0/1054686374 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadb8144630 con 0x7fadb809a1d0 2022-06-29T15:50:09.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.436+0000 7fadb57fa700 1 -- 172.21.15.47:0/1054686374 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fadac017030 con 0x7fadb809a1d0 2022-06-29T15:50:09.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.436+0000 7fadb57fa700 1 --2- 172.21.15.47:0/1054686374 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fad98053950 0x7fad98055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:09.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.440+0000 7fadb57fa700 1 -- 172.21.15.47:0/1054686374 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fadac06be90 con 0x7fadb809a1d0 2022-06-29T15:50:09.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.440+0000 7fadb7fff700 1 --2- 172.21.15.47:0/1054686374 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fad98053950 0x7fad98055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:09.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.440+0000 7fadb7fff700 1 --2- 172.21.15.47:0/1054686374 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fad98053950 0x7fad98055e10 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7fada801a000 tx=0x7fada8014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:09.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.444+0000 7fadb57fa700 1 -- 172.21.15.47:0/1054686374 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fadac042e90 con 0x7fadb809a1d0 2022-06-29T15:50:09.578 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.572+0000 7fadbd3e4700 1 -- 172.21.15.47:0/1054686374 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fadb80573d0 con 0x7fadb809a1d0 2022-06-29T15:50:09.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.584+0000 7fadb57fa700 1 -- 172.21.15.47:0/1054686374 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7fadac0464b0 con 0x7fadb809a1d0 2022-06-29T15:50:09.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.588+0000 7fadbd3e4700 1 -- 172.21.15.47:0/1054686374 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fad98053950 msgr2=0x7fad98055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:09.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.588+0000 7fadbd3e4700 1 --2- 172.21.15.47:0/1054686374 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fad98053950 0x7fad98055e10 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7fada801a000 tx=0x7fada8014000).stop 2022-06-29T15:50:09.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.588+0000 7fadbd3e4700 1 -- 172.21.15.47:0/1054686374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fadb809a1d0 msgr2=0x7fadb8144870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:09.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.588+0000 7fadbd3e4700 1 --2- 172.21.15.47:0/1054686374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fadb809a1d0 0x7fadb8144870 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 rx=0x7fadb81442e0 tx=0x7fadac008040).stop 2022-06-29T15:50:09.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.588+0000 7fadbd3e4700 1 -- 172.21.15.47:0/1054686374 shutdown_connections 2022-06-29T15:50:09.594 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.588+0000 7fadbd3e4700 1 --2- 172.21.15.47:0/1054686374 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fad98053950 0x7fad98055e10 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:09.594 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.588+0000 7fadbd3e4700 1 --2- 172.21.15.47:0/1054686374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadb8142a30 0x7fadb813c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:09.594 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.588+0000 7fadbd3e4700 1 --2- 172.21.15.47:0/1054686374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fadb809a1d0 0x7fadb8144870 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:09.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.588+0000 7fadbd3e4700 1 --2- 172.21.15.47:0/1054686374 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fadb805f4a0 0x7fadb8145b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:09.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.588+0000 7fadbd3e4700 1 -- 172.21.15.47:0/1054686374 >> 172.21.15.47:0/1054686374 conn(0x7fadb8062df0 msgr2=0x7fadb80631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:09.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.588+0000 7fadbd3e4700 1 -- 172.21.15.47:0/1054686374 shutdown_connections 2022-06-29T15:50:09.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.588+0000 7fadbd3e4700 1 -- 172.21.15.47:0/1054686374 wait complete. 2022-06-29T15:50:09.605 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-06-29T15:50:09.606 DEBUG:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:50:09.622 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:50:09.645 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:50:09.667 DEBUG:teuthology.orchestra.run.smithi164:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:50:09.690 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-06-29T15:50:09.690 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:09.691 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:50:09.692 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:50:09.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T15:50:09.701 INFO:teuthology.orchestra.run.smithi099.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-06-29T15:50:09.701 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-06-29T15:50:09.702 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:09.703 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:50:09.703 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:50:09.711 INFO:teuthology.orchestra.run.smithi164.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-06-29T15:50:09.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T15:50:09.713 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-06-29T15:50:09.713 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-06-29T15:50:09.826 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.820+0000 7f773f353700 1 Processor -- start 2022-06-29T15:50:09.827 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.820+0000 7f773f353700 1 -- start start 2022-06-29T15:50:09.829 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.820+0000 7f773f353700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f77380b2ee0 0x7f77380b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:09.829 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.820+0000 7f773f353700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f77380ad970 0x7f77381407e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:09.830 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.820+0000 7f773f353700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77381400e0 con 0x7f77380b0e50 2022-06-29T15:50:09.830 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.820+0000 7f773f353700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f773813fef0 con 0x7f77380b2ee0 2022-06-29T15:50:09.830 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.820+0000 7f773f353700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7738143810 con 0x7f77380ad970 2022-06-29T15:50:09.830 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.824+0000 7f773eb52700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f77380b0e50 0x7f773813d290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37418/0 (socket says 172.21.15.47:37418) 2022-06-29T15:50:09.831 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.824+0000 7f773eb52700 1 -- 172.21.15.47:0/2277369372 learned_addr learned my addr 172.21.15.47:0/2277369372 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:09.831 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.824+0000 7f773db50700 1 --2- 172.21.15.47:0/2277369372 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f77380ad970 0x7f77381407e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:09.832 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.824+0000 7f773e351700 1 --2- 172.21.15.47:0/2277369372 >> v2:172.21.15.64:3300/0 conn(0x7f77380b2ee0 0x7f77380b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:09.833 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.824+0000 7f773d34f700 1 -- 172.21.15.47:0/2277369372 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1765141322 0 0) 0x7f77381400e0 con 0x7f77380b0e50 2022-06-29T15:50:09.833 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.824+0000 7f773d34f700 1 -- 172.21.15.47:0/2277369372 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7720003260 con 0x7f77380b0e50 2022-06-29T15:50:09.833 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.824+0000 7f773db50700 1 -- 172.21.15.47:0/2277369372 >> v2:172.21.15.64:3300/0 conn(0x7f77380b2ee0 msgr2=0x7f77380b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:09.833 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.824+0000 7f773db50700 1 --2- 172.21.15.47:0/2277369372 >> v2:172.21.15.64:3300/0 conn(0x7f77380b2ee0 0x7f77380b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:09.834 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.824+0000 7f773db50700 1 -- 172.21.15.47:0/2277369372 >> v1:172.21.15.47:6789/0 conn(0x7f77380b0e50 legacy=0x7f773813d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:09.834 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.824+0000 7f773db50700 1 -- 172.21.15.47:0/2277369372 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7738143570 con 0x7f77380ad970 2022-06-29T15:50:09.835 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.824+0000 7f773db50700 1 --2- 172.21.15.47:0/2277369372 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f77380ad970 0x7f77381407e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f7734002f70 tx=0x7f7734006df0).ready entity=mon.2 client_cookie=287ba65679dd0554 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:09.835 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.824+0000 7f773d34f700 1 -- 172.21.15.47:0/2277369372 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f773400d4a0 con 0x7f77380ad970 2022-06-29T15:50:09.835 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.824+0000 7f773d34f700 1 -- 172.21.15.47:0/2277369372 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7734000e90 con 0x7f77380ad970 2022-06-29T15:50:09.835 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.824+0000 7f773d34f700 1 -- 172.21.15.47:0/2277369372 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f773400d7f0 con 0x7f77380ad970 2022-06-29T15:50:09.840 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.824+0000 7f773f353700 1 -- 172.21.15.47:0/2277369372 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f77380ad970 msgr2=0x7f77381407e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:09.841 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.824+0000 7f773f353700 1 --2- 172.21.15.47:0/2277369372 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f77380ad970 0x7f77381407e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f7734002f70 tx=0x7f7734006df0).stop 2022-06-29T15:50:09.841 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.824+0000 7f773f353700 1 -- 172.21.15.47:0/2277369372 shutdown_connections 2022-06-29T15:50:09.841 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.824+0000 7f773f353700 1 --2- 172.21.15.47:0/2277369372 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f77380ad970 0x7f77381407e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:09.842 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.824+0000 7f773f353700 1 --2- 172.21.15.47:0/2277369372 >> v2:172.21.15.64:3300/0 conn(0x7f77380b2ee0 0x7f77380b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:09.842 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.824+0000 7f773f353700 1 -- 172.21.15.47:0/2277369372 >> 172.21.15.47:0/2277369372 conn(0x7f7738063670 msgr2=0x7f7738063a80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:09.842 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.824+0000 7f773f353700 1 -- 172.21.15.47:0/2277369372 shutdown_connections 2022-06-29T15:50:09.842 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.824+0000 7f773f353700 1 -- 172.21.15.47:0/2277369372 wait complete. 2022-06-29T15:50:09.843 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.828+0000 7f773f353700 1 Processor -- start 2022-06-29T15:50:09.843 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.828+0000 7f773f353700 1 -- start start 2022-06-29T15:50:09.843 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.828+0000 7f773f353700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77380ad970 0x7f77380aa700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:09.843 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.828+0000 7f773f353700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f77380b0e50 0x7f7738155af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:09.844 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.828+0000 7f773f353700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77380b2ee0 0x7f7738155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:09.844 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.828+0000 7f773f353700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77380abb00 con 0x7f77380ad970 2022-06-29T15:50:09.844 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.828+0000 7f773f353700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77381564d0 con 0x7f77380b2ee0 2022-06-29T15:50:09.844 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.828+0000 7f773f353700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77380666b0 con 0x7f77380b0e50 2022-06-29T15:50:09.844 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.828+0000 7f773e351700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77380ad970 0x7f77380aa700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:09.845 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.828+0000 7f773e351700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77380ad970 0x7f77380aa700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51866/0 (socket says 172.21.15.47:51866) 2022-06-29T15:50:09.845 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.828+0000 7f773e351700 1 -- 172.21.15.47:0/3861010743 learned_addr learned my addr 172.21.15.47:0/3861010743 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:09.846 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.828+0000 7f773db50700 1 --2- 172.21.15.47:0/3861010743 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f77380b0e50 0x7f7738155af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:09.846 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.828+0000 7f773eb52700 1 --2- 172.21.15.47:0/3861010743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77380b2ee0 0x7f7738155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:09.846 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.832+0000 7f773e351700 1 -- 172.21.15.47:0/3861010743 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f77380b0e50 msgr2=0x7f7738155af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:09.846 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.832+0000 7f773e351700 1 --2- 172.21.15.47:0/3861010743 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f77380b0e50 0x7f7738155af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:09.847 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.832+0000 7f773e351700 1 -- 172.21.15.47:0/3861010743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77380b2ee0 msgr2=0x7f7738155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:09.847 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.832+0000 7f773e351700 1 --2- 172.21.15.47:0/3861010743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77380b2ee0 0x7f7738155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:09.847 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.832+0000 7f773e351700 1 -- 172.21.15.47:0/3861010743 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f773400b180 con 0x7f77380ad970 2022-06-29T15:50:09.847 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.832+0000 7f773e351700 1 --2- 172.21.15.47:0/3861010743 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77380ad970 0x7f77380aa700 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 rx=0x7f772c00c000 tx=0x7f772c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:09.848 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.832+0000 7f772b7fe700 1 -- 172.21.15.47:0/3861010743 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f772c011750 con 0x7f77380ad970 2022-06-29T15:50:09.848 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.832+0000 7f772b7fe700 1 -- 172.21.15.47:0/3861010743 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f772c0118b0 con 0x7f77380ad970 2022-06-29T15:50:09.848 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.832+0000 7f773f353700 1 -- 172.21.15.47:0/3861010743 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7738066950 con 0x7f77380ad970 2022-06-29T15:50:09.848 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.832+0000 7f772b7fe700 1 -- 172.21.15.47:0/3861010743 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f772c011c10 con 0x7f77380ad970 2022-06-29T15:50:09.848 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.832+0000 7f773f353700 1 -- 172.21.15.47:0/3861010743 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7738066e90 con 0x7f77380ad970 2022-06-29T15:50:09.849 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.832+0000 7f773f353700 1 -- 172.21.15.47:0/3861010743 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77380573d0 con 0x7f77380ad970 2022-06-29T15:50:09.849 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.832+0000 7f772b7fe700 1 -- 172.21.15.47:0/3861010743 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f772c011d70 con 0x7f77380ad970 2022-06-29T15:50:09.849 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.832+0000 7f772b7fe700 1 --2- 172.21.15.47:0/3861010743 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7718053a10 0x7f7718055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:09.849 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.832+0000 7f772b7fe700 1 -- 172.21.15.47:0/3861010743 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f772c06c210 con 0x7f77380ad970 2022-06-29T15:50:09.850 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.836+0000 7f773db50700 1 --2- 172.21.15.47:0/3861010743 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7718053a10 0x7f7718055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:09.850 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.836+0000 7f773db50700 1 --2- 172.21.15.47:0/3861010743 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7718053a10 0x7f7718055ed0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7f773401a000 tx=0x7f7734014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:09.851 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.840+0000 7f772b7fe700 1 -- 172.21.15.47:0/3861010743 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f772c0431b0 con 0x7f77380ad970 2022-06-29T15:50:09.974 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.968+0000 7f773f353700 1 -- 172.21.15.47:0/3861010743 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f7738069e20 con 0x7f77380ad970 2022-06-29T15:50:09.976 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.972+0000 7f772b7fe700 1 -- 172.21.15.47:0/3861010743 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v104) v1 ==== 81+0+29221 (secure 0 0 0) 0x7f772c0467d0 con 0x7f77380ad970 2022-06-29T15:50:09.977 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:50:09.977 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2022-06-29T15:50:09.978 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 104, 2022-06-29T15:50:09.978 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "9a68894a-53ce-400e-838b-14eab91b834c", 2022-06-29T15:50:09.978 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2022-06-29T15:42:14.046361+0000", 2022-06-29T15:50:09.979 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2022-06-29T15:50:07.263034+0000", 2022-06-29T15:50:09.979 INFO:teuthology.orchestra.run.smithi047.stdout: "last_up_change": "2022-06-29T15:42:20.929789+0000", 2022-06-29T15:50:09.979 INFO:teuthology.orchestra.run.smithi047.stdout: "last_in_change": "2022-06-29T15:42:20.929789+0000", 2022-06-29T15:50:09.979 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-06-29T15:50:09.980 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_num": 5799936, 2022-06-29T15:50:09.980 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_set": [ 2022-06-29T15:50:09.980 INFO:teuthology.orchestra.run.smithi047.stdout: "pglog_hardlimit", 2022-06-29T15:50:09.980 INFO:teuthology.orchestra.run.smithi047.stdout: "purged_snapdirs", 2022-06-29T15:50:09.980 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_deletes", 2022-06-29T15:50:09.981 INFO:teuthology.orchestra.run.smithi047.stdout: "sortbitwise" 2022-06-29T15:50:09.981 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:50:09.981 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_version": 5, 2022-06-29T15:50:09.982 INFO:teuthology.orchestra.run.smithi047.stdout: "full_ratio": 0.94999998807907104, 2022-06-29T15:50:09.982 INFO:teuthology.orchestra.run.smithi047.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-06-29T15:50:09.983 INFO:teuthology.orchestra.run.smithi047.stdout: "nearfull_ratio": 0.85000002384185791, 2022-06-29T15:50:09.983 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_snapshot": "", 2022-06-29T15:50:09.983 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_max": 18, 2022-06-29T15:50:09.984 INFO:teuthology.orchestra.run.smithi047.stdout: "max_osd": 8, 2022-06-29T15:50:09.984 INFO:teuthology.orchestra.run.smithi047.stdout: "require_min_compat_client": "jewel", 2022-06-29T15:50:09.984 INFO:teuthology.orchestra.run.smithi047.stdout: "min_compat_client": "jewel", 2022-06-29T15:50:09.984 INFO:teuthology.orchestra.run.smithi047.stdout: "require_osd_release": "octopus", 2022-06-29T15:50:09.985 INFO:teuthology.orchestra.run.smithi047.stdout: "pools": [ 2022-06-29T15:50:09.985 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:09.985 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 1, 2022-06-29T15:50:09.985 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "device_health_metrics", 2022-06-29T15:50:09.986 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:42:16.971431+0000", 2022-06-29T15:50:09.986 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:50:09.986 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:50:09.987 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:50:09.987 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:50:09.987 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:50:09.987 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:50:09.988 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:50:09.988 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:50:09.988 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 1, 2022-06-29T15:50:09.989 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 1, 2022-06-29T15:50:09.989 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 1, 2022-06-29T15:50:09.989 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 1, 2022-06-29T15:50:09.989 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 1, 2022-06-29T15:50:09.990 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:50:09.990 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:50:09.990 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:50:09.990 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:50:09.991 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:50:09.991 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:50:09.991 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:50:09.991 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:09.991 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "16", 2022-06-29T15:50:09.992 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:50:09.992 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:50:09.992 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:50:09.992 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:50:09.992 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:50:09.993 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:50:09.993 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:50:09.993 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:50:09.993 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:50:09.994 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:50:09.994 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:50:09.994 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:50:09.995 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:50:09.995 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:50:09.995 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:50:09.995 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:50:09.995 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:50:09.996 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:50:09.996 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:50:09.996 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:50:09.996 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:50:09.997 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:50:09.997 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:50:09.997 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:50:09.997 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:50:09.997 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:50:09.998 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:09.998 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:50:09.998 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:50:09.998 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:50:09.999 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:50:09.999 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:50:09.999 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:50:10.000 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:50:10.000 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:50:10.000 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:50:10.000 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:50:10.001 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:50:10.001 INFO:teuthology.orchestra.run.smithi047.stdout: "options": { 2022-06-29T15:50:10.001 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_min": 1 2022-06-29T15:50:10.001 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.001 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:50:10.002 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_devicehealth": {} 2022-06-29T15:50:10.002 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:10.002 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.002 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.003 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 2, 2022-06-29T15:50:10.003 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "rbd", 2022-06-29T15:50:10.003 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:42:25.290884+0000", 2022-06-29T15:50:10.003 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:50:10.003 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:50:10.004 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:50:10.004 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:50:10.004 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:50:10.005 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:50:10.005 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:50:10.005 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:50:10.005 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2022-06-29T15:50:10.006 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2022-06-29T15:50:10.006 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2022-06-29T15:50:10.006 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2022-06-29T15:50:10.007 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2022-06-29T15:50:10.007 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:50:10.007 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:50:10.007 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:50:10.008 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:50:10.008 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:50:10.008 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:50:10.008 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:50:10.009 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.009 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "19", 2022-06-29T15:50:10.009 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:50:10.009 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:50:10.009 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:50:10.010 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:50:10.010 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:50:10.010 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:50:10.011 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:50:10.011 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:50:10.011 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:50:10.011 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:50:10.012 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:50:10.012 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:50:10.012 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:50:10.012 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:50:10.013 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:50:10.013 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:50:10.013 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:50:10.013 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:50:10.013 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:50:10.014 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:50:10.014 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:50:10.014 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:50:10.014 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:50:10.015 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:50:10.015 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:50:10.015 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:50:10.016 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.016 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:50:10.016 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:50:10.016 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:50:10.017 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:50:10.017 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:50:10.017 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:50:10.017 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:50:10.017 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:50:10.018 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:50:10.018 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:50:10.018 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:50:10.018 INFO:teuthology.orchestra.run.smithi047.stdout: "options": {}, 2022-06-29T15:50:10.019 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:50:10.019 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd": {} 2022-06-29T15:50:10.019 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:10.019 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:10.020 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:50:10.020 INFO:teuthology.orchestra.run.smithi047.stdout: "osds": [ 2022-06-29T15:50:10.020 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.020 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2022-06-29T15:50:10.021 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "e23eb023-4706-4201-bf72-4bf19c912c5d", 2022-06-29T15:50:10.021 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:50:10.021 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:50:10.022 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:50:10.022 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:50:10.023 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:50:10.023 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:50:10.023 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:50:10.024 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 99, 2022-06-29T15:50:10.024 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:50:10.024 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:50:10.024 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:50:10.024 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:10.025 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.025 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:10.025 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6818", 2022-06-29T15:50:10.025 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:50:10.026 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.026 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.026 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:10.027 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6819", 2022-06-29T15:50:10.027 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:50:10.027 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:10.027 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:10.027 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.028 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:50:10.028 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:10.028 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.028 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:10.029 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6820", 2022-06-29T15:50:10.029 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:50:10.029 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.029 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.029 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:10.030 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6821", 2022-06-29T15:50:10.030 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:50:10.030 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:10.030 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:10.031 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.031 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:50:10.031 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:10.031 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.031 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:10.032 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6824", 2022-06-29T15:50:10.032 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:50:10.032 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.032 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.033 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:10.033 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6825", 2022-06-29T15:50:10.033 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:50:10.034 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:10.034 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:10.035 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.035 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:50:10.036 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:10.036 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.036 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:10.037 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6822", 2022-06-29T15:50:10.037 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:50:10.038 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.038 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.039 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:10.039 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6823", 2022-06-29T15:50:10.040 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:50:10.040 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:10.041 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:10.041 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.041 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6819/13096", 2022-06-29T15:50:10.042 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6821/13096", 2022-06-29T15:50:10.042 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6825/13096", 2022-06-29T15:50:10.043 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6823/13096", 2022-06-29T15:50:10.043 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:50:10.043 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:50:10.044 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:50:10.044 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:10.045 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.045 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.045 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2022-06-29T15:50:10.046 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "228bb08d-2d64-43d9-9560-bfaeff154452", 2022-06-29T15:50:10.046 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:50:10.047 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:50:10.047 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:50:10.048 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:50:10.048 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:50:10.048 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:50:10.049 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:50:10.049 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 99, 2022-06-29T15:50:10.050 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:50:10.050 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:50:10.050 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:50:10.051 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:10.051 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.052 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:10.052 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6826", 2022-06-29T15:50:10.052 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:50:10.053 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.053 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.053 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:10.054 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6827", 2022-06-29T15:50:10.054 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:50:10.055 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:10.055 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:10.055 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.056 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:50:10.056 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:10.057 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.057 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:10.057 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6828", 2022-06-29T15:50:10.058 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:50:10.058 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.059 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.059 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:10.060 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6829", 2022-06-29T15:50:10.060 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:50:10.061 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:10.061 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:10.061 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.062 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:50:10.062 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:10.063 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.063 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:10.063 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6832", 2022-06-29T15:50:10.064 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:50:10.064 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.064 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.065 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:10.065 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6833", 2022-06-29T15:50:10.065 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:50:10.066 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:10.066 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:10.066 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.067 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:50:10.067 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:10.068 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.068 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:10.069 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6830", 2022-06-29T15:50:10.069 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:50:10.069 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.070 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.070 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:10.071 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6831", 2022-06-29T15:50:10.071 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:50:10.071 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:10.071 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:10.072 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.072 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6827/13100", 2022-06-29T15:50:10.072 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6829/13100", 2022-06-29T15:50:10.072 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6833/13100", 2022-06-29T15:50:10.072 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6831/13100", 2022-06-29T15:50:10.073 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:50:10.073 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:50:10.073 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:50:10.073 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:10.074 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.074 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.074 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2022-06-29T15:50:10.074 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "797a5452-1510-4640-a111-c9c2fcf707eb", 2022-06-29T15:50:10.074 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:50:10.075 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:50:10.075 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:50:10.075 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:50:10.075 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:50:10.076 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:50:10.076 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:50:10.076 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 99, 2022-06-29T15:50:10.076 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:50:10.076 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:50:10.077 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:50:10.077 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:10.077 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.077 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:10.078 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6810", 2022-06-29T15:50:10.078 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:50:10.078 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.078 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.079 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:10.079 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6811", 2022-06-29T15:50:10.079 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:50:10.079 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:10.079 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:10.080 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.080 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:50:10.080 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:10.080 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.081 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:10.081 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6812", 2022-06-29T15:50:10.081 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:50:10.082 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.082 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.082 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:10.083 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6813", 2022-06-29T15:50:10.083 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:50:10.083 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:10.083 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:10.083 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.084 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:50:10.084 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:10.084 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.084 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:10.085 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6816", 2022-06-29T15:50:10.085 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:50:10.085 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.085 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.086 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:10.086 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6817", 2022-06-29T15:50:10.086 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:50:10.086 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:10.086 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:10.087 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.087 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:50:10.087 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:10.087 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.088 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:10.088 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6814", 2022-06-29T15:50:10.088 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:50:10.088 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.089 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.089 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:10.089 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6815", 2022-06-29T15:50:10.089 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:50:10.089 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:10.090 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:10.090 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.090 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6811/13098", 2022-06-29T15:50:10.090 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6813/13098", 2022-06-29T15:50:10.091 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6817/13098", 2022-06-29T15:50:10.091 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6815/13098", 2022-06-29T15:50:10.091 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:50:10.091 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:50:10.092 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:50:10.092 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:10.092 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.093 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.093 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2022-06-29T15:50:10.093 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "55b24fd7-bb71-4a01-b3fa-a09caa772a62", 2022-06-29T15:50:10.094 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:50:10.094 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:50:10.094 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:50:10.095 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:50:10.095 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:50:10.095 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:50:10.095 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:50:10.095 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 99, 2022-06-29T15:50:10.096 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:50:10.096 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:50:10.096 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:50:10.096 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:10.097 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.097 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:10.097 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6802", 2022-06-29T15:50:10.097 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:50:10.097 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.098 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.098 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:10.098 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6803", 2022-06-29T15:50:10.098 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:50:10.099 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:10.099 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:10.099 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.099 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:50:10.099 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:10.100 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.100 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:10.100 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6804", 2022-06-29T15:50:10.100 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:50:10.101 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.101 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.101 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:10.101 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6805", 2022-06-29T15:50:10.101 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:50:10.102 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:10.102 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:10.102 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.102 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:50:10.103 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:10.103 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.104 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:10.104 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6808", 2022-06-29T15:50:10.104 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:50:10.104 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.105 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.105 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:10.105 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6809", 2022-06-29T15:50:10.105 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:50:10.106 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:10.106 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:10.106 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.107 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:50:10.107 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:10.107 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.107 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:10.108 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6806", 2022-06-29T15:50:10.108 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:50:10.108 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.108 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.108 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:10.109 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6807", 2022-06-29T15:50:10.109 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:50:10.109 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:10.109 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:10.110 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.110 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6803/13097", 2022-06-29T15:50:10.110 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6805/13097", 2022-06-29T15:50:10.110 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6809/13097", 2022-06-29T15:50:10.111 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6807/13097", 2022-06-29T15:50:10.111 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:50:10.111 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:50:10.111 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:50:10.111 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:10.112 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.112 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.112 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2022-06-29T15:50:10.112 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3", 2022-06-29T15:50:10.113 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:50:10.113 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:50:10.113 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:50:10.113 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:50:10.114 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:50:10.114 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:50:10.114 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:50:10.115 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 98, 2022-06-29T15:50:10.115 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:50:10.115 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:50:10.115 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:50:10.116 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:10.116 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.116 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:10.116 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6816", 2022-06-29T15:50:10.117 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:50:10.117 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.117 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.117 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:10.117 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6817", 2022-06-29T15:50:10.118 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:50:10.118 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:10.118 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:10.118 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.119 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:50:10.119 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:10.119 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.119 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:10.119 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6818", 2022-06-29T15:50:10.120 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:50:10.120 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.120 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.120 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:10.121 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6819", 2022-06-29T15:50:10.121 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:50:10.121 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:10.121 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:10.121 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.122 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:50:10.122 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:10.122 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.122 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:10.123 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6822", 2022-06-29T15:50:10.123 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:50:10.123 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.123 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.124 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:10.124 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6823", 2022-06-29T15:50:10.124 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:50:10.124 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:10.125 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:10.125 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.126 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:50:10.126 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:10.126 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.126 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:10.127 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6820", 2022-06-29T15:50:10.127 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:50:10.127 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.127 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.127 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:10.128 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6821", 2022-06-29T15:50:10.128 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:50:10.128 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:10.128 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:10.129 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.129 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6817/13396", 2022-06-29T15:50:10.129 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6819/13396", 2022-06-29T15:50:10.129 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6823/13396", 2022-06-29T15:50:10.129 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6821/13396", 2022-06-29T15:50:10.130 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:50:10.130 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:50:10.130 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:50:10.130 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:10.131 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.131 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.131 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2022-06-29T15:50:10.131 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "f77beebe-6cca-4244-b2d4-677825ebf919", 2022-06-29T15:50:10.132 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:50:10.132 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:50:10.132 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:50:10.132 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:50:10.132 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:50:10.133 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:50:10.133 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:50:10.133 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 88, 2022-06-29T15:50:10.133 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:50:10.133 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:50:10.134 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:50:10.134 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:10.134 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.134 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:10.135 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6800", 2022-06-29T15:50:10.135 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:50:10.135 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.136 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.136 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:10.137 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6801", 2022-06-29T15:50:10.137 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:50:10.138 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:10.138 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:10.138 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.139 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:50:10.139 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:10.140 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.140 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:10.141 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6802", 2022-06-29T15:50:10.141 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:50:10.141 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.142 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.142 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:10.142 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6803", 2022-06-29T15:50:10.143 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:50:10.143 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:10.143 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:10.144 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.144 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:50:10.144 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:10.145 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.145 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:10.146 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6806", 2022-06-29T15:50:10.146 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:50:10.146 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.147 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.147 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:10.147 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6807", 2022-06-29T15:50:10.148 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:50:10.148 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:10.148 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:10.149 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.149 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:50:10.149 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:10.150 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.150 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:10.150 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6804", 2022-06-29T15:50:10.151 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:50:10.151 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.151 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.151 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:10.151 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6805", 2022-06-29T15:50:10.152 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:50:10.152 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:10.152 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:10.152 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.153 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6801/13393", 2022-06-29T15:50:10.153 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6803/13393", 2022-06-29T15:50:10.153 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6807/13393", 2022-06-29T15:50:10.153 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6805/13393", 2022-06-29T15:50:10.153 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:50:10.154 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:50:10.154 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:50:10.154 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:10.154 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.155 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.155 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2022-06-29T15:50:10.155 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "af0ca9b3-c02c-4910-834b-bf0b8dc6713c", 2022-06-29T15:50:10.155 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:50:10.155 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:50:10.156 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:50:10.156 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:50:10.156 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:50:10.156 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:50:10.157 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:50:10.157 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 98, 2022-06-29T15:50:10.157 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:50:10.157 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:50:10.157 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:50:10.158 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:10.158 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.158 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:10.158 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6824", 2022-06-29T15:50:10.159 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:50:10.159 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.159 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.159 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:10.159 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6825", 2022-06-29T15:50:10.160 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:50:10.160 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:10.160 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:10.160 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.160 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:50:10.161 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:10.161 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.161 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:10.161 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6826", 2022-06-29T15:50:10.162 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:50:10.162 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.162 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.162 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:10.163 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6827", 2022-06-29T15:50:10.163 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:50:10.163 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:10.163 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:10.163 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.164 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:50:10.164 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:10.164 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.164 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:10.164 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6830", 2022-06-29T15:50:10.165 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:50:10.165 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.165 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.165 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:10.166 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6831", 2022-06-29T15:50:10.166 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:50:10.166 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:10.166 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:10.166 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.167 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:50:10.167 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:10.167 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.167 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:10.168 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6828", 2022-06-29T15:50:10.168 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:50:10.168 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.168 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.168 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:10.169 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6829", 2022-06-29T15:50:10.169 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:50:10.169 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:10.169 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:10.169 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.170 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6825/13395", 2022-06-29T15:50:10.170 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6827/13395", 2022-06-29T15:50:10.170 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6831/13395", 2022-06-29T15:50:10.170 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6829/13395", 2022-06-29T15:50:10.171 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:50:10.171 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:50:10.171 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:50:10.171 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:10.171 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.172 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.179 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2022-06-29T15:50:10.180 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "f19c3ca6-df88-4bde-a150-b51a53f092e1", 2022-06-29T15:50:10.180 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:50:10.180 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:50:10.180 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:50:10.180 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:50:10.181 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:50:10.181 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:50:10.181 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:50:10.181 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 99, 2022-06-29T15:50:10.182 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:50:10.182 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:50:10.182 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:50:10.182 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:10.182 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.183 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:10.183 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6808", 2022-06-29T15:50:10.183 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:50:10.183 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.184 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.184 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:10.184 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6809", 2022-06-29T15:50:10.184 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:50:10.184 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:10.185 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:10.185 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.185 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:50:10.185 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:10.185 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.186 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:10.186 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6810", 2022-06-29T15:50:10.186 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:50:10.186 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.187 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.187 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:10.187 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6811", 2022-06-29T15:50:10.187 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:50:10.187 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:10.188 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:10.188 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.188 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:50:10.188 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:10.188 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.189 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:10.189 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6814", 2022-06-29T15:50:10.189 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:50:10.189 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.190 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.190 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:10.190 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6815", 2022-06-29T15:50:10.190 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:50:10.191 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:10.191 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:10.191 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.191 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:50:10.191 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:10.192 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.192 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:10.192 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6812", 2022-06-29T15:50:10.192 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:50:10.193 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.193 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.193 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:10.193 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6813", 2022-06-29T15:50:10.193 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:50:10.194 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:10.194 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:10.194 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.194 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6809/13394", 2022-06-29T15:50:10.195 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6811/13394", 2022-06-29T15:50:10.195 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6815/13394", 2022-06-29T15:50:10.195 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6813/13394", 2022-06-29T15:50:10.195 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:50:10.195 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:50:10.196 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:50:10.196 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:10.196 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:10.196 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:50:10.197 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_xinfo": [ 2022-06-29T15:50:10.197 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.197 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2022-06-29T15:50:10.197 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:50:10.197 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:50:10.198 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:50:10.198 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:50:10.198 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:50:10.198 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.720513+0000", 2022-06-29T15:50:10.199 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:50:10.199 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.199 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.199 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2022-06-29T15:50:10.199 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:50:10.200 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:50:10.200 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:50:10.200 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:50:10.200 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:50:10.201 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.772612+0000", 2022-06-29T15:50:10.201 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:50:10.201 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.202 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.202 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2022-06-29T15:50:10.202 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:50:10.203 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:50:10.203 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:50:10.203 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:50:10.204 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:50:10.204 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.450223+0000", 2022-06-29T15:50:10.205 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:50:10.205 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.205 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.206 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2022-06-29T15:50:10.206 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:50:10.206 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:50:10.207 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:50:10.207 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:50:10.207 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:50:10.208 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.477856+0000", 2022-06-29T15:50:10.208 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:50:10.208 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.208 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.208 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2022-06-29T15:50:10.209 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:50:10.209 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:50:10.209 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:50:10.209 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:50:10.210 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:50:10.210 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.539485+0000", 2022-06-29T15:50:10.210 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:50:10.210 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.210 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.211 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2022-06-29T15:50:10.211 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:50:10.211 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:50:10.211 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:50:10.212 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:50:10.212 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:50:10.212 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.487040+0000", 2022-06-29T15:50:10.212 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:50:10.213 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.213 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.213 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2022-06-29T15:50:10.213 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:50:10.213 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:50:10.214 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:50:10.214 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:50:10.214 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:50:10.215 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.748685+0000", 2022-06-29T15:50:10.215 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:50:10.216 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.216 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:10.216 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2022-06-29T15:50:10.216 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:50:10.217 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:50:10.217 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:50:10.217 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:50:10.217 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:50:10.217 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.459516+0000", 2022-06-29T15:50:10.218 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:50:10.218 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:10.218 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:50:10.219 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap": [], 2022-06-29T15:50:10.219 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap_items": [], 2022-06-29T15:50:10.219 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_temp": [], 2022-06-29T15:50:10.219 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_temp": [], 2022-06-29T15:50:10.219 INFO:teuthology.orchestra.run.smithi047.stdout: "blacklist": { 2022-06-29T15:50:10.220 INFO:teuthology.orchestra.run.smithi047.stdout: "172.21.15.64:6833/3819760998": "2022-06-30T15:50:03.413680+0000", 2022-06-29T15:50:10.220 INFO:teuthology.orchestra.run.smithi047.stdout: "172.21.15.64:6832/3819760998": "2022-06-30T15:50:03.413680+0000" 2022-06-29T15:50:10.221 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.221 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profiles": { 2022-06-29T15:50:10.221 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs_data": { 2022-06-29T15:50:10.222 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-device-class": "", 2022-06-29T15:50:10.222 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2022-06-29T15:50:10.222 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-root": "default", 2022-06-29T15:50:10.222 INFO:teuthology.orchestra.run.smithi047.stdout: "jerasure-per-chunk-alignment": "false", 2022-06-29T15:50:10.222 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2022-06-29T15:50:10.223 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "2", 2022-06-29T15:50:10.223 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2022-06-29T15:50:10.223 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2022-06-29T15:50:10.223 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van", 2022-06-29T15:50:10.224 INFO:teuthology.orchestra.run.smithi047.stdout: "w": "8" 2022-06-29T15:50:10.224 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.224 INFO:teuthology.orchestra.run.smithi047.stdout: "default": { 2022-06-29T15:50:10.224 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2022-06-29T15:50:10.224 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2022-06-29T15:50:10.225 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "1", 2022-06-29T15:50:10.225 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2022-06-29T15:50:10.225 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2022-06-29T15:50:10.226 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van" 2022-06-29T15:50:10.226 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:10.226 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:10.226 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps_queue": [], 2022-06-29T15:50:10.227 INFO:teuthology.orchestra.run.smithi047.stdout: "new_removed_snaps": [], 2022-06-29T15:50:10.227 INFO:teuthology.orchestra.run.smithi047.stdout: "new_purged_snaps": [], 2022-06-29T15:50:10.227 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_node_flags": {}, 2022-06-29T15:50:10.227 INFO:teuthology.orchestra.run.smithi047.stdout: "device_class_flags": {} 2022-06-29T15:50:10.228 INFO:teuthology.orchestra.run.smithi047.stdout:} 2022-06-29T15:50:10.230 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.972+0000 7f773f353700 1 -- 172.21.15.47:0/3861010743 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7718053a10 msgr2=0x7f7718055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:10.230 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.972+0000 7f773f353700 1 --2- 172.21.15.47:0/3861010743 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7718053a10 0x7f7718055ed0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7f773401a000 tx=0x7f7734014000).stop 2022-06-29T15:50:10.231 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.972+0000 7f773f353700 1 -- 172.21.15.47:0/3861010743 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77380ad970 msgr2=0x7f77380aa700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:10.231 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.972+0000 7f773f353700 1 --2- 172.21.15.47:0/3861010743 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77380ad970 0x7f77380aa700 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 rx=0x7f772c00c000 tx=0x7f772c006df0).stop 2022-06-29T15:50:10.231 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.972+0000 7f773f353700 1 -- 172.21.15.47:0/3861010743 shutdown_connections 2022-06-29T15:50:10.231 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.972+0000 7f773f353700 1 --2- 172.21.15.47:0/3861010743 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7718053a10 0x7f7718055ed0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:10.231 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.972+0000 7f773f353700 1 --2- 172.21.15.47:0/3861010743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77380b2ee0 0x7f7738155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:10.232 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.972+0000 7f773f353700 1 --2- 172.21.15.47:0/3861010743 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f77380b0e50 0x7f7738155af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:10.232 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.972+0000 7f773f353700 1 --2- 172.21.15.47:0/3861010743 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77380ad970 0x7f77380aa700 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:10.233 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.972+0000 7f773f353700 1 -- 172.21.15.47:0/3861010743 >> 172.21.15.47:0/3861010743 conn(0x7f7738063670 msgr2=0x7f773813e840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:10.233 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.972+0000 7f773f353700 1 -- 172.21.15.47:0/3861010743 shutdown_connections 2022-06-29T15:50:10.234 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:09.972+0000 7f773f353700 1 -- 172.21.15.47:0/3861010743 wait complete. 2022-06-29T15:50:10.235 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:50:10.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.340+0000 7f471f59e700 1 Processor -- start 2022-06-29T15:50:10.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.340+0000 7f471f59e700 1 -- start start 2022-06-29T15:50:10.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.344+0000 7f471f59e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47200b2690 0x7f47200b5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:10.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.344+0000 7f471f59e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47200b0be0 con 0x7f47200b5e10 2022-06-29T15:50:10.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.344+0000 7f471f59e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47200b0e80 con 0x7f4720140a60 2022-06-29T15:50:10.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.344+0000 7f471f59e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f472013dac0 con 0x7f47200b2690 2022-06-29T15:50:10.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.344+0000 7f4717fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f47200b5e10 0x7f4720140350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37428/0 (socket says 172.21.15.47:37428) 2022-06-29T15:50:10.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.344+0000 7f4717fff700 1 -- 172.21.15.47:0/3012085844 learned_addr learned my addr 172.21.15.47:0/3012085844 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:10.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.344+0000 7f471e59c700 1 --2- 172.21.15.47:0/3012085844 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47200b2690 0x7f47200b5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:10.352 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.344+0000 7f471dd9b700 1 -- 172.21.15.47:0/3012085844 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1876478983 0 0) 0x7f47200b0be0 con 0x7f47200b5e10 2022-06-29T15:50:10.352 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.344+0000 7f471dd9b700 1 -- 172.21.15.47:0/3012085844 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4700003260 con 0x7f47200b5e10 2022-06-29T15:50:10.352 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.344+0000 7f471dd9b700 1 -- 172.21.15.47:0/3012085844 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2053774798 0 0) 0x7f47200b0e80 con 0x7f4720140a60 2022-06-29T15:50:10.352 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.344+0000 7f471dd9b700 1 -- 172.21.15.47:0/3012085844 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47200b0be0 con 0x7f4720140a60 2022-06-29T15:50:10.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.344+0000 7f471e59c700 1 -- 172.21.15.47:0/3012085844 >> v1:172.21.15.64:6789/0 conn(0x7f4720140a60 legacy=0x7f472014a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:10.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.344+0000 7f471e59c700 1 -- 172.21.15.47:0/3012085844 >> v1:172.21.15.47:6789/0 conn(0x7f47200b5e10 legacy=0x7f4720140350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:10.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.344+0000 7f471dd9b700 1 -- 172.21.15.47:0/3012085844 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3577586509 0 0) 0x7f4700003260 con 0x7f47200b5e10 2022-06-29T15:50:10.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.344+0000 7f471e59c700 1 -- 172.21.15.47:0/3012085844 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47200b1120 con 0x7f47200b2690 2022-06-29T15:50:10.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.344+0000 7f471e59c700 1 --2- 172.21.15.47:0/3012085844 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47200b2690 0x7f47200b5920 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f4708001110 tx=0x7f4710006d90).ready entity=mon.2 client_cookie=3524a41f9c94496a server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:10.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.344+0000 7f471dd9b700 1 -- 172.21.15.47:0/3012085844 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4708001660 con 0x7f47200b2690 2022-06-29T15:50:10.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.344+0000 7f471dd9b700 1 -- 172.21.15.47:0/3012085844 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4710000e90 con 0x7f47200b2690 2022-06-29T15:50:10.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.344+0000 7f471dd9b700 1 -- 172.21.15.47:0/3012085844 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f471000bee0 con 0x7f47200b2690 2022-06-29T15:50:10.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.344+0000 7f471f59e700 1 -- 172.21.15.47:0/3012085844 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47200b2690 msgr2=0x7f47200b5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:10.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.344+0000 7f471f59e700 1 --2- 172.21.15.47:0/3012085844 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47200b2690 0x7f47200b5920 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f4708001110 tx=0x7f4710006d90).stop 2022-06-29T15:50:10.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.344+0000 7f471f59e700 1 -- 172.21.15.47:0/3012085844 shutdown_connections 2022-06-29T15:50:10.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.344+0000 7f471f59e700 1 --2- 172.21.15.47:0/3012085844 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47200b2690 0x7f47200b5920 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:10.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.344+0000 7f471f59e700 1 -- 172.21.15.47:0/3012085844 >> 172.21.15.47:0/3012085844 conn(0x7f4720062560 msgr2=0x7f4720062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:10.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.348+0000 7f471f59e700 1 -- 172.21.15.47:0/3012085844 shutdown_connections 2022-06-29T15:50:10.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.348+0000 7f471f59e700 1 -- 172.21.15.47:0/3012085844 wait complete. 2022-06-29T15:50:10.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.348+0000 7f471f59e700 1 Processor -- start 2022-06-29T15:50:10.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.348+0000 7f471f59e700 1 -- start start 2022-06-29T15:50:10.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.352+0000 7f471f59e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47200b2690 0x7f47200604e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:10.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.352+0000 7f471f59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4720140a60 0x7f4720150070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:10.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.352+0000 7f471f59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4720063220 0x7f4720150560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:10.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.352+0000 7f471f59e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47200ac7a0 con 0x7f4720140a60 2022-06-29T15:50:10.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.352+0000 7f471f59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47200ac500 con 0x7f4720063220 2022-06-29T15:50:10.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.352+0000 7f471f59e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4720064a30 con 0x7f47200b2690 2022-06-29T15:50:10.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.352+0000 7f4717fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4720140a60 0x7f4720150070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:10.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.352+0000 7f471e59c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47200b2690 0x7f47200604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:10.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.352+0000 7f4717fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4720140a60 0x7f4720150070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51882/0 (socket says 172.21.15.47:51882) 2022-06-29T15:50:10.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.352+0000 7f4717fff700 1 -- 172.21.15.47:0/68281430 learned_addr learned my addr 172.21.15.47:0/68281430 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:10.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.352+0000 7f471ed9d700 1 --2- 172.21.15.47:0/68281430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4720063220 0x7f4720150560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:10.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.352+0000 7f4717fff700 1 -- 172.21.15.47:0/68281430 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47200b2690 msgr2=0x7f47200604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:10.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.352+0000 7f4717fff700 1 --2- 172.21.15.47:0/68281430 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47200b2690 0x7f47200604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:10.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.352+0000 7f4717fff700 1 -- 172.21.15.47:0/68281430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4720063220 msgr2=0x7f4720150560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:10.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.352+0000 7f4717fff700 1 --2- 172.21.15.47:0/68281430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4720063220 0x7f4720150560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:10.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.352+0000 7f4717fff700 1 -- 172.21.15.47:0/68281430 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f471000b120 con 0x7f4720140a60 2022-06-29T15:50:10.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.352+0000 7f471e59c700 1 --2- 172.21.15.47:0/68281430 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47200b2690 0x7f47200604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:50:10.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.352+0000 7f4717fff700 1 --2- 172.21.15.47:0/68281430 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4720140a60 0x7f4720150070 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 rx=0x7f4708001840 tx=0x7f4708001720).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:10.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.352+0000 7f47177fe700 1 -- 172.21.15.47:0/68281430 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4708007170 con 0x7f4720140a60 2022-06-29T15:50:10.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.352+0000 7f47177fe700 1 -- 172.21.15.47:0/68281430 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47080072d0 con 0x7f4720140a60 2022-06-29T15:50:10.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.352+0000 7f471f59e700 1 -- 172.21.15.47:0/68281430 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4720064cd0 con 0x7f4720140a60 2022-06-29T15:50:10.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.352+0000 7f47177fe700 1 -- 172.21.15.47:0/68281430 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4708004710 con 0x7f4720140a60 2022-06-29T15:50:10.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.352+0000 7f471f59e700 1 -- 172.21.15.47:0/68281430 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4720065170 con 0x7f4720140a60 2022-06-29T15:50:10.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.352+0000 7f47177fe700 1 -- 172.21.15.47:0/68281430 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f470801b070 con 0x7f4720140a60 2022-06-29T15:50:10.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.352+0000 7f471f59e700 1 -- 172.21.15.47:0/68281430 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47200573d0 con 0x7f4720140a60 2022-06-29T15:50:10.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.356+0000 7f47177fe700 1 --2- 172.21.15.47:0/68281430 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4704053a70 0x7f4704055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:10.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.356+0000 7f47177fe700 1 -- 172.21.15.47:0/68281430 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f470806b480 con 0x7f4720140a60 2022-06-29T15:50:10.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.356+0000 7f471e59c700 1 --2- 172.21.15.47:0/68281430 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4704053a70 0x7f4704055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:10.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.356+0000 7f471e59c700 1 --2- 172.21.15.47:0/68281430 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4704053a70 0x7f4704055f30 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7f471001a000 tx=0x7f4710014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:10.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.360+0000 7f47177fe700 1 -- 172.21.15.47:0/68281430 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4708042410 con 0x7f4720140a60 2022-06-29T15:50:10.524 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.520+0000 7f471f59e700 1 -- 172.21.15.47:0/68281430 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f47200ad3d0 con 0x7f4720140a60 2022-06-29T15:50:10.525 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.520+0000 7f47177fe700 1 -- 172.21.15.47:0/68281430 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+1557 (secure 0 0 0) 0x7f4708045a30 con 0x7f4720140a60 2022-06-29T15:50:10.525 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:50:10.526 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":47,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6969,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3642031880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3642031880},{"type":"v1","addr":"172.21.15.47:6835","nonce":3642031880}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":43},{"gid":6984,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/891572891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":891572891},{"type":"v1","addr":"172.21.15.47:6837","nonce":891572891}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":43},{"gid":56285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/865149541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":865149541},{"type":"v1","addr":"172.21.15.64:6833","nonce":865149541}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":47}],"filesystems":[]} 2022-06-29T15:50:10.527 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.520+0000 7f471f59e700 1 -- 172.21.15.47:0/68281430 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4704053a70 msgr2=0x7f4704055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:10.527 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.520+0000 7f471f59e700 1 --2- 172.21.15.47:0/68281430 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4704053a70 0x7f4704055f30 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7f471001a000 tx=0x7f4710014000).stop 2022-06-29T15:50:10.528 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.520+0000 7f471f59e700 1 -- 172.21.15.47:0/68281430 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4720140a60 msgr2=0x7f4720150070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:10.528 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.520+0000 7f471f59e700 1 --2- 172.21.15.47:0/68281430 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4720140a60 0x7f4720150070 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 rx=0x7f4708001840 tx=0x7f4708001720).stop 2022-06-29T15:50:10.528 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.520+0000 7f471f59e700 1 -- 172.21.15.47:0/68281430 shutdown_connections 2022-06-29T15:50:10.528 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.520+0000 7f471f59e700 1 --2- 172.21.15.47:0/68281430 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4704053a70 0x7f4704055f30 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:10.528 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.520+0000 7f471f59e700 1 --2- 172.21.15.47:0/68281430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4720063220 0x7f4720150560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:10.529 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.520+0000 7f471f59e700 1 --2- 172.21.15.47:0/68281430 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4720140a60 0x7f4720150070 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:10.529 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.520+0000 7f471f59e700 1 --2- 172.21.15.47:0/68281430 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47200b2690 0x7f47200604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:10.529 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.520+0000 7f471f59e700 1 -- 172.21.15.47:0/68281430 >> 172.21.15.47:0/68281430 conn(0x7f4720062560 msgr2=0x7f472005bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:10.529 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.520+0000 7f471f59e700 1 -- 172.21.15.47:0/68281430 shutdown_connections 2022-06-29T15:50:10.530 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.520+0000 7f471f59e700 1 -- 172.21.15.47:0/68281430 wait complete. 2022-06-29T15:50:10.530 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 47 2022-06-29T15:50:10.538 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:50:10.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.632+0000 7f2b05927700 1 Processor -- start 2022-06-29T15:50:10.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.632+0000 7f2b05927700 1 -- start start 2022-06-29T15:50:10.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.636+0000 7f2b05927700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2b00145090 0x7f2b000b2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:10.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.636+0000 7f2b05927700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b000a9120 con 0x7f2b000b5c60 2022-06-29T15:50:10.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.636+0000 7f2b05927700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b000b0b80 con 0x7f2b00060eb0 2022-06-29T15:50:10.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.636+0000 7f2b05927700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b000a89e0 con 0x7f2b00145090 2022-06-29T15:50:10.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.636+0000 7f2b05126700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2b00145090 0x7f2b000b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:10.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.636+0000 7f2b04925700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f2b000b5c60 0x7f2b00144980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37444/0 (socket says 172.21.15.47:37444) 2022-06-29T15:50:10.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.636+0000 7f2b04925700 1 -- 172.21.15.47:0/2654973768 learned_addr learned my addr 172.21.15.47:0/2654973768 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:10.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.636+0000 7f2aff7fe700 1 -- 172.21.15.47:0/2654973768 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1417227067 0 0) 0x7f2b000b0b80 con 0x7f2b00060eb0 2022-06-29T15:50:10.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.636+0000 7f2b05126700 1 -- 172.21.15.47:0/2654973768 >> v1:172.21.15.64:6789/0 conn(0x7f2b00060eb0 legacy=0x7f2b000b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:10.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.636+0000 7f2b05126700 1 -- 172.21.15.47:0/2654973768 >> v1:172.21.15.47:6789/0 conn(0x7f2b000b5c60 legacy=0x7f2b00144980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:10.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.636+0000 7f2b05126700 1 -- 172.21.15.47:0/2654973768 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b000a8c80 con 0x7f2b00145090 2022-06-29T15:50:10.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.636+0000 7f2b05126700 1 --2- 172.21.15.47:0/2654973768 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b00145090 0x7f2b000b2540 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 rx=0x7f2af4001290 tx=0x7f2af8006df0).ready entity=mon.0 client_cookie=27860e8a8631bf60 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:10.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.636+0000 7f2aff7fe700 1 -- 172.21.15.47:0/2654973768 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2af800d5c0 con 0x7f2b00145090 2022-06-29T15:50:10.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.636+0000 7f2aff7fe700 1 -- 172.21.15.47:0/2654973768 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2af8000e90 con 0x7f2b00145090 2022-06-29T15:50:10.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.636+0000 7f2aff7fe700 1 -- 172.21.15.47:0/2654973768 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2af800d910 con 0x7f2b00145090 2022-06-29T15:50:10.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.636+0000 7f2b05927700 1 -- 172.21.15.47:0/2654973768 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b00145090 msgr2=0x7f2b000b2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:10.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.636+0000 7f2b05927700 1 --2- 172.21.15.47:0/2654973768 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b00145090 0x7f2b000b2540 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 rx=0x7f2af4001290 tx=0x7f2af8006df0).stop 2022-06-29T15:50:10.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.636+0000 7f2b05927700 1 -- 172.21.15.47:0/2654973768 shutdown_connections 2022-06-29T15:50:10.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.636+0000 7f2b05927700 1 --2- 172.21.15.47:0/2654973768 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b00145090 0x7f2b000b2540 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:10.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.636+0000 7f2b05927700 1 -- 172.21.15.47:0/2654973768 >> 172.21.15.47:0/2654973768 conn(0x7f2b00062560 msgr2=0x7f2b00062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:10.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.636+0000 7f2b05927700 1 -- 172.21.15.47:0/2654973768 shutdown_connections 2022-06-29T15:50:10.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.636+0000 7f2b05927700 1 -- 172.21.15.47:0/2654973768 wait complete. 2022-06-29T15:50:10.652 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.640+0000 7f2b05927700 1 Processor -- start 2022-06-29T15:50:10.653 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.640+0000 7f2b05927700 1 -- start start 2022-06-29T15:50:10.653 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.640+0000 7f2b05927700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2b00060eb0 0x7f2b001493b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:10.653 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.640+0000 7f2b05927700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b000b5c60 0x7f2b001432a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:10.653 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.640+0000 7f2b05927700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b00145090 0x7f2b001565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:10.654 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.640+0000 7f2b05927700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b00148a50 con 0x7f2b00145090 2022-06-29T15:50:10.654 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.640+0000 7f2b05927700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b00148cf0 con 0x7f2b000b5c60 2022-06-29T15:50:10.654 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.640+0000 7f2b05927700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b000aeb80 con 0x7f2b00060eb0 2022-06-29T15:50:10.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.640+0000 7f2b05126700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b00145090 0x7f2b001565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:10.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.640+0000 7f2b05126700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b00145090 0x7f2b001565d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51898/0 (socket says 172.21.15.47:51898) 2022-06-29T15:50:10.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.640+0000 7f2afffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b000b5c60 0x7f2b001432a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:10.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.640+0000 7f2b04925700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2b00060eb0 0x7f2b001493b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:10.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.640+0000 7f2b05126700 1 -- 172.21.15.47:0/3984375593 learned_addr learned my addr 172.21.15.47:0/3984375593 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:10.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.644+0000 7f2b05126700 1 -- 172.21.15.47:0/3984375593 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2b00060eb0 msgr2=0x7f2b001493b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:10.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.644+0000 7f2b05126700 1 --2- 172.21.15.47:0/3984375593 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2b00060eb0 0x7f2b001493b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:10.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.644+0000 7f2b05126700 1 -- 172.21.15.47:0/3984375593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b000b5c60 msgr2=0x7f2b001432a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:10.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.644+0000 7f2b05126700 1 --2- 172.21.15.47:0/3984375593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b000b5c60 0x7f2b001432a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:10.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.644+0000 7f2b05126700 1 -- 172.21.15.47:0/3984375593 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af800b180 con 0x7f2b00145090 2022-06-29T15:50:10.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.644+0000 7f2afffff700 1 --2- 172.21.15.47:0/3984375593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b000b5c60 0x7f2b001432a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:50:10.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.644+0000 7f2b05126700 1 --2- 172.21.15.47:0/3984375593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b00145090 0x7f2b001565d0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 rx=0x7f2af4001290 tx=0x7f2af8014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:10.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.644+0000 7f2afdffb700 1 -- 172.21.15.47:0/3984375593 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2af8013de0 con 0x7f2b00145090 2022-06-29T15:50:10.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.644+0000 7f2afdffb700 1 -- 172.21.15.47:0/3984375593 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2af800de90 con 0x7f2b00145090 2022-06-29T15:50:10.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.644+0000 7f2b05927700 1 -- 172.21.15.47:0/3984375593 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b00148250 con 0x7f2b00145090 2022-06-29T15:50:10.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.644+0000 7f2afdffb700 1 -- 172.21.15.47:0/3984375593 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2af801e5f0 con 0x7f2b00145090 2022-06-29T15:50:10.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.644+0000 7f2b05927700 1 -- 172.21.15.47:0/3984375593 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b0015a7a0 con 0x7f2b00145090 2022-06-29T15:50:10.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.644+0000 7f2b05927700 1 -- 172.21.15.47:0/3984375593 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b00148eb0 con 0x7f2b00145090 2022-06-29T15:50:10.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.648+0000 7f2afdffb700 1 -- 172.21.15.47:0/3984375593 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2af8025030 con 0x7f2b00145090 2022-06-29T15:50:10.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.648+0000 7f2afdffb700 1 --2- 172.21.15.47:0/3984375593 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2ae0053a70 0x7f2ae0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:10.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.648+0000 7f2afdffb700 1 -- 172.21.15.47:0/3984375593 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f2af806cb10 con 0x7f2b00145090 2022-06-29T15:50:10.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.648+0000 7f2b04925700 1 --2- 172.21.15.47:0/3984375593 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2ae0053a70 0x7f2ae0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:10.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.648+0000 7f2b04925700 1 --2- 172.21.15.47:0/3984375593 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2ae0053a70 0x7f2ae0055f30 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7f2af400d000 tx=0x7f2af40018c0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:10.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.652+0000 7f2afdffb700 1 -- 172.21.15.47:0/3984375593 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2af8043b10 con 0x7f2b00145090 2022-06-29T15:50:10.826 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.820+0000 7f2b05927700 1 -- 172.21.15.47:0/3984375593 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2b000573d0 con 0x7f2b00145090 2022-06-29T15:50:10.827 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.820+0000 7f2afdffb700 1 -- 172.21.15.47:0/3984375593 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+1557 (secure 0 0 0) 0x7f2af8023070 con 0x7f2b00145090 2022-06-29T15:50:10.827 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:50:10.828 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":47,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6969,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3642031880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3642031880},{"type":"v1","addr":"172.21.15.47:6835","nonce":3642031880}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":43},{"gid":6984,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/891572891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":891572891},{"type":"v1","addr":"172.21.15.47:6837","nonce":891572891}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":43},{"gid":56285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/865149541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":865149541},{"type":"v1","addr":"172.21.15.64:6833","nonce":865149541}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":47}],"filesystems":[]} 2022-06-29T15:50:10.829 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.824+0000 7f2b05927700 1 -- 172.21.15.47:0/3984375593 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2ae0053a70 msgr2=0x7f2ae0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:10.830 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.824+0000 7f2b05927700 1 --2- 172.21.15.47:0/3984375593 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2ae0053a70 0x7f2ae0055f30 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7f2af400d000 tx=0x7f2af40018c0).stop 2022-06-29T15:50:10.830 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.824+0000 7f2b05927700 1 -- 172.21.15.47:0/3984375593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b00145090 msgr2=0x7f2b001565d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:10.830 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.824+0000 7f2b05927700 1 --2- 172.21.15.47:0/3984375593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b00145090 0x7f2b001565d0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 rx=0x7f2af4001290 tx=0x7f2af8014000).stop 2022-06-29T15:50:10.830 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.824+0000 7f2b05927700 1 -- 172.21.15.47:0/3984375593 shutdown_connections 2022-06-29T15:50:10.831 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.824+0000 7f2b05927700 1 --2- 172.21.15.47:0/3984375593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b00145090 0x7f2b001565d0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:10.831 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.824+0000 7f2b05927700 1 --2- 172.21.15.47:0/3984375593 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2ae0053a70 0x7f2ae0055f30 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:10.831 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.824+0000 7f2b05927700 1 --2- 172.21.15.47:0/3984375593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b000b5c60 0x7f2b001432a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:10.831 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.824+0000 7f2b05927700 1 --2- 172.21.15.47:0/3984375593 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2b00060eb0 0x7f2b001493b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:10.832 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.824+0000 7f2b05927700 1 -- 172.21.15.47:0/3984375593 >> 172.21.15.47:0/3984375593 conn(0x7f2b00062560 msgr2=0x7f2b000b4220 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:10.832 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.824+0000 7f2b05927700 1 -- 172.21.15.47:0/3984375593 shutdown_connections 2022-06-29T15:50:10.832 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:10.824+0000 7f2b05927700 1 -- 172.21.15.47:0/3984375593 wait complete. 2022-06-29T15:50:10.833 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 47 2022-06-29T15:50:10.841 INFO:tasks.ceph.mds.a:Restarting daemon 2022-06-29T15:50:10.841 INFO:tasks.ceph.mds.a:Stopping old one... 2022-06-29T15:50:10.842 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-06-29T15:50:10.842 INFO:teuthology.orchestra.run:waiting for 300 2022-06-29T15:50:10.843 INFO:tasks.ceph.mds.c:Restarting daemon 2022-06-29T15:50:10.843 INFO:tasks.ceph.mds.c:Stopping old one... 2022-06-29T15:50:10.843 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2022-06-29T15:50:10.844 INFO:teuthology.orchestra.run:waiting for 300 2022-06-29T15:50:10.844 INFO:tasks.ceph.mds.b:Restarting daemon 2022-06-29T15:50:10.844 INFO:tasks.ceph.mds.b:Stopping old one... 2022-06-29T15:50:10.845 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-06-29T15:50:10.845 INFO:teuthology.orchestra.run:waiting for 300 2022-06-29T15:50:10.854 INFO:tasks.ceph.mds.c:Stopped 2022-06-29T15:50:10.855 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2022-06-29T15:50:10.857 INFO:tasks.ceph.mds.a:Stopped 2022-06-29T15:50:10.858 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-06-29T15:50:10.862 INFO:tasks.ceph.mds.b:Stopped 2022-06-29T15:50:10.862 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-06-29T15:50:10.900 INFO:tasks.ceph.mds.c:Started 2022-06-29T15:50:10.901 DEBUG:teuthology.parallel:result is None 2022-06-29T15:50:10.902 INFO:tasks.ceph.mds.a:Started 2022-06-29T15:50:10.903 DEBUG:teuthology.parallel:result is None 2022-06-29T15:50:10.905 INFO:tasks.ceph.mds.b:Started 2022-06-29T15:50:10.906 DEBUG:teuthology.parallel:result is None 2022-06-29T15:50:10.906 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-06-29T15:50:10.966 INFO:tasks.ceph.mds.a.smithi047.stdout:starting mds.a at 2022-06-29T15:50:10.978 INFO:tasks.ceph.mds.c.smithi047.stdout:starting mds.c at 2022-06-29T15:50:10.981 INFO:tasks.ceph.mds.b.smithi064.stdout:starting mds.b at 2022-06-29T15:50:11.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.000+0000 7f993a758700 1 Processor -- start 2022-06-29T15:50:11.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.000+0000 7f993a758700 1 -- start start 2022-06-29T15:50:11.012 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.004+0000 7f993a758700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f99340acff0 0x7f993405fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:11.013 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.004+0000 7f993a758700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f99341437d0 0x7f9934143c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:11.013 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.004+0000 7f993a758700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99340a97e0 con 0x7f99340b19a0 2022-06-29T15:50:11.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.004+0000 7f993a758700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99340b1d80 con 0x7f99341437d0 2022-06-29T15:50:11.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.004+0000 7f993a758700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f993405f430 con 0x7f99340acff0 2022-06-29T15:50:11.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.004+0000 7f9939f57700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f99341437d0 0x7f9934143c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:11.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.004+0000 7f9938f55700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f99340acff0 0x7f993405fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:11.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.004+0000 7f9939f57700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f99341437d0 0x7f9934143c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51936/0 (socket says 172.21.15.47:51936) 2022-06-29T15:50:11.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.004+0000 7f9939f57700 1 -- 172.21.15.47:0/4146769251 learned_addr learned my addr 172.21.15.47:0/4146769251 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:11.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.004+0000 7f9939f57700 1 -- 172.21.15.47:0/4146769251 >> v2:172.21.15.64:3300/0 conn(0x7f99340acff0 msgr2=0x7f993405fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:11.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.004+0000 7f9939f57700 1 --2- 172.21.15.47:0/4146769251 >> v2:172.21.15.64:3300/0 conn(0x7f99340acff0 0x7f993405fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:11.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.004+0000 7f9939f57700 1 -- 172.21.15.47:0/4146769251 >> v1:172.21.15.64:6789/0 conn(0x7f99340b19a0 legacy=0x7f99340ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:11.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.004+0000 7f9939f57700 1 -- 172.21.15.47:0/4146769251 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99340a92a0 con 0x7f99341437d0 2022-06-29T15:50:11.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.004+0000 7f9939f57700 1 --2- 172.21.15.47:0/4146769251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99341437d0 0x7f9934143c60 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 rx=0x7f992c0032f0 tx=0x7f992c00b530).ready entity=mon.0 client_cookie=d5fd59b180252dcc server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:11.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.004+0000 7f9923fff700 1 -- 172.21.15.47:0/4146769251 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f992c00ddf0 con 0x7f99341437d0 2022-06-29T15:50:11.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.004+0000 7f9923fff700 1 -- 172.21.15.47:0/4146769251 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992c010470 con 0x7f99341437d0 2022-06-29T15:50:11.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.004+0000 7f9923fff700 1 -- 172.21.15.47:0/4146769251 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f992c0107d0 con 0x7f99341437d0 2022-06-29T15:50:11.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.004+0000 7f993a758700 1 -- 172.21.15.47:0/4146769251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99341437d0 msgr2=0x7f9934143c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:11.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.004+0000 7f993a758700 1 --2- 172.21.15.47:0/4146769251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99341437d0 0x7f9934143c60 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 rx=0x7f992c0032f0 tx=0x7f992c00b530).stop 2022-06-29T15:50:11.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.004+0000 7f993a758700 1 -- 172.21.15.47:0/4146769251 shutdown_connections 2022-06-29T15:50:11.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.004+0000 7f993a758700 1 --2- 172.21.15.47:0/4146769251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99341437d0 0x7f9934143c60 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:11.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.004+0000 7f993a758700 1 --2- 172.21.15.47:0/4146769251 >> v2:172.21.15.64:3300/0 conn(0x7f99340acff0 0x7f993405fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:11.022 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.004+0000 7f993a758700 1 -- 172.21.15.47:0/4146769251 >> 172.21.15.47:0/4146769251 conn(0x7f9934062df0 msgr2=0x7f99340631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:11.022 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.004+0000 7f993a758700 1 -- 172.21.15.47:0/4146769251 shutdown_connections 2022-06-29T15:50:11.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.004+0000 7f993a758700 1 -- 172.21.15.47:0/4146769251 wait complete. 2022-06-29T15:50:11.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.008+0000 7f993a758700 1 Processor -- start 2022-06-29T15:50:11.024 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.008+0000 7f993a758700 1 -- start start 2022-06-29T15:50:11.029 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.008+0000 7f993a758700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99340acff0 0x7f9934066b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:11.029 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.008+0000 7f993a758700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99340b19a0 0x7f993414e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:11.030 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.008+0000 7f993a758700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f99341437d0 0x7f9934142730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:11.030 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.008+0000 7f993a758700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993409b0c0 con 0x7f99340b19a0 2022-06-29T15:50:11.031 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.008+0000 7f993a758700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993414ba70 con 0x7f99340acff0 2022-06-29T15:50:11.031 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.008+0000 7f993a758700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f993414ecf0 con 0x7f99341437d0 2022-06-29T15:50:11.031 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.008+0000 7f9939756700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99340acff0 0x7f9934066b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:11.032 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.008+0000 7f9939756700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99340acff0 0x7f9934066b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:57642/0 (socket says 172.21.15.47:57642) 2022-06-29T15:50:11.032 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.008+0000 7f9939756700 1 -- 172.21.15.47:0/1462596044 learned_addr learned my addr 172.21.15.47:0/1462596044 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:11.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.008+0000 7f9939756700 1 -- 172.21.15.47:0/1462596044 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f99341437d0 msgr2=0x7f9934142730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2022-06-29T15:50:11.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.008+0000 7f9939756700 1 --2- 172.21.15.47:0/1462596044 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f99341437d0 0x7f9934142730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2022-06-29T15:50:11.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.008+0000 7f9939756700 1 -- 172.21.15.47:0/1462596044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99340b19a0 msgr2=0x7f993414e800 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2022-06-29T15:50:11.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.008+0000 7f9939756700 1 --2- 172.21.15.47:0/1462596044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99340b19a0 0x7f993414e800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2022-06-29T15:50:11.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.008+0000 7f9939756700 1 -- 172.21.15.47:0/1462596044 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f992c00b7d0 con 0x7f99340acff0 2022-06-29T15:50:11.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.008+0000 7f9939756700 1 --2- 172.21.15.47:0/1462596044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99340acff0 0x7f9934066b20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7f993405c8d0 tx=0x7f9930007c00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:11.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.016+0000 7f99227fc700 1 -- 172.21.15.47:0/1462596044 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9930007030 con 0x7f99340acff0 2022-06-29T15:50:11.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.016+0000 7f99227fc700 1 -- 172.21.15.47:0/1462596044 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9930007190 con 0x7f99340acff0 2022-06-29T15:50:11.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.016+0000 7f993a758700 1 -- 172.21.15.47:0/1462596044 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9934067160 con 0x7f99340acff0 2022-06-29T15:50:11.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.016+0000 7f99227fc700 1 -- 172.21.15.47:0/1462596044 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9930006510 con 0x7f99340acff0 2022-06-29T15:50:11.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.016+0000 7f993a758700 1 -- 172.21.15.47:0/1462596044 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9934098310 con 0x7f99340acff0 2022-06-29T15:50:11.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.016+0000 7f993a758700 1 -- 172.21.15.47:0/1462596044 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f993409ae90 con 0x7f99340acff0 2022-06-29T15:50:11.038 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.016+0000 7f99227fc700 1 -- 172.21.15.47:0/1462596044 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9930010030 con 0x7f99340acff0 2022-06-29T15:50:11.038 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.016+0000 7f99227fc700 1 --2- 172.21.15.47:0/1462596044 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9914053b30 0x7f9914055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:11.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.016+0000 7f9938f55700 1 --2- 172.21.15.47:0/1462596044 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9914053b30 0x7f9914055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:11.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.016+0000 7f9938f55700 1 --2- 172.21.15.47:0/1462596044 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9914053b30 0x7f9914055ff0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7f992400c000 tx=0x7f9924006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:11.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.020+0000 7f99227fc700 1 -- 172.21.15.47:0/1462596044 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f9930046660 con 0x7f99340acff0 2022-06-29T15:50:11.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.024+0000 7f99227fc700 1 -- 172.21.15.47:0/1462596044 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9930042a00 con 0x7f99340acff0 2022-06-29T15:50:11.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:11.164+0000 7f993a758700 1 -- 172.21.15.47:0/1462596044 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f99340984d0 con 0x7f99340acff0 2022-06-29T15:50:12.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.032+0000 7f99227fc700 1 -- 172.21.15.47:0/1462596044 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v107) v1 ==== 97+0+0 (secure 0 0 0) 0x7f9930046020 con 0x7f99340acff0 2022-06-29T15:50:12.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.036+0000 7f993a758700 1 -- 172.21.15.47:0/1462596044 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9914053b30 msgr2=0x7f9914055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:12.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.036+0000 7f993a758700 1 --2- 172.21.15.47:0/1462596044 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9914053b30 0x7f9914055ff0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7f992400c000 tx=0x7f9924006df0).stop 2022-06-29T15:50:12.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.036+0000 7f993a758700 1 -- 172.21.15.47:0/1462596044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99340acff0 msgr2=0x7f9934066b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:12.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.036+0000 7f993a758700 1 --2- 172.21.15.47:0/1462596044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99340acff0 0x7f9934066b20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7f993405c8d0 tx=0x7f9930007c00).stop 2022-06-29T15:50:12.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.036+0000 7f993a758700 1 -- 172.21.15.47:0/1462596044 shutdown_connections 2022-06-29T15:50:12.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.036+0000 7f993a758700 1 --2- 172.21.15.47:0/1462596044 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f99341437d0 0x7f9934142730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2022-06-29T15:50:12.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.036+0000 7f993a758700 1 --2- 172.21.15.47:0/1462596044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99340b19a0 0x7f993414e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2022-06-29T15:50:12.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.036+0000 7f993a758700 1 --2- 172.21.15.47:0/1462596044 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9914053b30 0x7f9914055ff0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:12.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.036+0000 7f993a758700 1 --2- 172.21.15.47:0/1462596044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99340acff0 0x7f9934066b20 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:12.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.036+0000 7f993a758700 1 -- 172.21.15.47:0/1462596044 >> 172.21.15.47:0/1462596044 conn(0x7f9934062df0 msgr2=0x7f99340b0bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:12.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.036+0000 7f993a758700 1 -- 172.21.15.47:0/1462596044 shutdown_connections 2022-06-29T15:50:12.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.036+0000 7f993a758700 1 -- 172.21.15.47:0/1462596044 wait complete. 2022-06-29T15:50:12.044 INFO:teuthology.orchestra.run.smithi047.stderr: removed all blacklist entries 2022-06-29T15:50:12.054 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2022-06-29T15:50:12.055 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-06-29T15:50:12.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.160+0000 7f05aefda700 1 Processor -- start 2022-06-29T15:50:12.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.160+0000 7f05aefda700 1 -- start start 2022-06-29T15:50:12.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.164+0000 7f05aefda700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f05a80611b0 0x7f05a8146640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:12.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.164+0000 7f05aefda700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f05a80b0cf0 0x7f05a8146b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:12.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.164+0000 7f05aefda700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f05a80b06d0 0x7f05a80667e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:12.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.164+0000 7f05aefda700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05a80b4b10 con 0x7f05a80b0cf0 2022-06-29T15:50:12.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.164+0000 7f05aefda700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05a80621d0 con 0x7f05a80611b0 2022-06-29T15:50:12.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.164+0000 7f05aefda700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a80b4520 con 0x7f05a80b06d0 2022-06-29T15:50:12.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.164+0000 7f05a7fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f05a80b0cf0 0x7f05a8146b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:12.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.164+0000 7f05a7fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f05a80b0cf0 0x7f05a8146b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51944/0 (socket says 172.21.15.47:51944) 2022-06-29T15:50:12.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.164+0000 7f05a7fff700 1 -- 172.21.15.47:0/1545545047 learned_addr learned my addr 172.21.15.47:0/1545545047 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:12.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.164+0000 7f05ad577700 1 --2- 172.21.15.47:0/1545545047 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f05a80b06d0 0x7f05a80667e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:12.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.164+0000 7f05acd76700 1 --2- 172.21.15.47:0/1545545047 >> v2:172.21.15.64:3300/0 conn(0x7f05a80611b0 0x7f05a8146640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:12.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.164+0000 7f05a7fff700 1 -- 172.21.15.47:0/1545545047 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f05a80b06d0 msgr2=0x7f05a80667e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:12.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.164+0000 7f05a7fff700 1 --2- 172.21.15.47:0/1545545047 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f05a80b06d0 0x7f05a80667e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:12.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.164+0000 7f05a7fff700 1 -- 172.21.15.47:0/1545545047 >> v2:172.21.15.64:3300/0 conn(0x7f05a80611b0 msgr2=0x7f05a8146640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:12.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.164+0000 7f05a7fff700 1 --2- 172.21.15.47:0/1545545047 >> v2:172.21.15.64:3300/0 conn(0x7f05a80611b0 0x7f05a8146640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:12.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.164+0000 7f05a7fff700 1 -- 172.21.15.47:0/1545545047 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a8061590 con 0x7f05a80b0cf0 2022-06-29T15:50:12.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.164+0000 7f05acd76700 1 --2- 172.21.15.47:0/1545545047 >> v2:172.21.15.64:3300/0 conn(0x7f05a80611b0 0x7f05a8146640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:50:12.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.164+0000 7f05ad577700 1 --2- 172.21.15.47:0/1545545047 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f05a80b06d0 0x7f05a80667e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:50:12.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.164+0000 7f05a7fff700 1 --2- 172.21.15.47:0/1545545047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05a80b0cf0 0x7f05a8146b30 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 rx=0x7f05a80b4790 tx=0x7f0594006eb0).ready entity=mon.0 client_cookie=edb601c8d98aea1 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:12.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.164+0000 7f05a77fe700 1 -- 172.21.15.47:0/1545545047 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f059400d570 con 0x7f05a80b0cf0 2022-06-29T15:50:12.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.164+0000 7f05a77fe700 1 -- 172.21.15.47:0/1545545047 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0594000e90 con 0x7f05a80b0cf0 2022-06-29T15:50:12.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.164+0000 7f05a77fe700 1 -- 172.21.15.47:0/1545545047 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f059400d900 con 0x7f05a80b0cf0 2022-06-29T15:50:12.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.164+0000 7f05aefda700 1 -- 172.21.15.47:0/1545545047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05a80b0cf0 msgr2=0x7f05a8146b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:12.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.164+0000 7f05aefda700 1 --2- 172.21.15.47:0/1545545047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05a80b0cf0 0x7f05a8146b30 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 rx=0x7f05a80b4790 tx=0x7f0594006eb0).stop 2022-06-29T15:50:12.182 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.164+0000 7f05aefda700 1 -- 172.21.15.47:0/1545545047 shutdown_connections 2022-06-29T15:50:12.182 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.164+0000 7f05aefda700 1 --2- 172.21.15.47:0/1545545047 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f05a80b06d0 0x7f05a80667e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:12.182 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.164+0000 7f05aefda700 1 --2- 172.21.15.47:0/1545545047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05a80b0cf0 0x7f05a8146b30 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:12.183 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.164+0000 7f05aefda700 1 --2- 172.21.15.47:0/1545545047 >> v2:172.21.15.64:3300/0 conn(0x7f05a80611b0 0x7f05a8146640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:12.183 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.164+0000 7f05aefda700 1 -- 172.21.15.47:0/1545545047 >> 172.21.15.47:0/1545545047 conn(0x7f05a80b98b0 msgr2=0x7f05a80b9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:12.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.168+0000 7f05aefda700 1 -- 172.21.15.47:0/1545545047 shutdown_connections 2022-06-29T15:50:12.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.168+0000 7f05aefda700 1 -- 172.21.15.47:0/1545545047 wait complete. 2022-06-29T15:50:12.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.168+0000 7f05aefda700 1 Processor -- start 2022-06-29T15:50:12.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.168+0000 7f05aefda700 1 -- start start 2022-06-29T15:50:12.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.168+0000 7f05aefda700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f05a80611b0 0x7f05a8148ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:12.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.168+0000 7f05aefda700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05a80b06d0 0x7f05a813ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:12.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.168+0000 7f05aefda700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05a80b0cf0 0x7f05a813b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:12.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.168+0000 7f05aefda700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a80af5d0 con 0x7f05a80b0cf0 2022-06-29T15:50:12.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.168+0000 7f05aefda700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a813b540 con 0x7f05a80b06d0 2022-06-29T15:50:12.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.168+0000 7f05aefda700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a80af830 con 0x7f05a80611b0 2022-06-29T15:50:12.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.168+0000 7f05ad577700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05a80b0cf0 0x7f05a813b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:12.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.168+0000 7f05ad577700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05a80b0cf0 0x7f05a813b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51952/0 (socket says 172.21.15.47:51952) 2022-06-29T15:50:12.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.168+0000 7f05ad577700 1 -- 172.21.15.47:0/1394403131 learned_addr learned my addr 172.21.15.47:0/1394403131 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:12.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.168+0000 7f05a7fff700 1 --2- 172.21.15.47:0/1394403131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05a80b06d0 0x7f05a813ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:12.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.168+0000 7f05acd76700 1 --2- 172.21.15.47:0/1394403131 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f05a80611b0 0x7f05a8148ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:12.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.168+0000 7f05ad577700 1 -- 172.21.15.47:0/1394403131 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f05a80611b0 msgr2=0x7f05a8148ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:12.198 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.168+0000 7f05ad577700 1 --2- 172.21.15.47:0/1394403131 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f05a80611b0 0x7f05a8148ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:12.198 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.172+0000 7f05ad577700 1 -- 172.21.15.47:0/1394403131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05a80b06d0 msgr2=0x7f05a813ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:12.198 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.172+0000 7f05ad577700 1 --2- 172.21.15.47:0/1394403131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05a80b06d0 0x7f05a813ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:12.198 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.172+0000 7f05ad577700 1 -- 172.21.15.47:0/1394403131 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f059400b1e0 con 0x7f05a80b0cf0 2022-06-29T15:50:12.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.172+0000 7f05ad577700 1 --2- 172.21.15.47:0/1394403131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05a80b0cf0 0x7f05a813b050 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 rx=0x7f05a000a000 tx=0x7f05a0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:12.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.172+0000 7f05a5ffb700 1 -- 172.21.15.47:0/1394403131 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f05a000c070 con 0x7f05a80b0cf0 2022-06-29T15:50:12.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.172+0000 7f05a5ffb700 1 -- 172.21.15.47:0/1394403131 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05a00133f0 con 0x7f05a80b0cf0 2022-06-29T15:50:12.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.172+0000 7f05aefda700 1 -- 172.21.15.47:0/1394403131 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05a80afad0 con 0x7f05a80b0cf0 2022-06-29T15:50:12.200 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.172+0000 7f05a5ffb700 1 -- 172.21.15.47:0/1394403131 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f05a0013750 con 0x7f05a80b0cf0 2022-06-29T15:50:12.200 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.172+0000 7f05aefda700 1 -- 172.21.15.47:0/1394403131 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05a8149310 con 0x7f05a80b0cf0 2022-06-29T15:50:12.200 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.172+0000 7f05aefda700 1 -- 172.21.15.47:0/1394403131 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05a80573d0 con 0x7f05a80b0cf0 2022-06-29T15:50:12.200 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.172+0000 7f05a5ffb700 1 -- 172.21.15.47:0/1394403131 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f05a00138b0 con 0x7f05a80b0cf0 2022-06-29T15:50:12.200 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.172+0000 7f05a5ffb700 1 --2- 172.21.15.47:0/1394403131 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0588053a70 0x7f0588055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:12.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.176+0000 7f05a5ffb700 1 -- 172.21.15.47:0/1394403131 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f05a006ba80 con 0x7f05a80b0cf0 2022-06-29T15:50:12.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.176+0000 7f05acd76700 1 --2- 172.21.15.47:0/1394403131 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0588053a70 0x7f0588055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:12.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.176+0000 7f05acd76700 1 --2- 172.21.15.47:0/1394403131 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0588053a70 0x7f0588055f30 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7f059c00c000 tx=0x7f059c008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:12.202 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.180+0000 7f05a5ffb700 1 -- 172.21.15.47:0/1394403131 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f05a0042b20 con 0x7f05a80b0cf0 2022-06-29T15:50:12.335 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:50:12.337 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:50:12.340 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:50:12.342 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:50:12.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.344+0000 7f05aefda700 1 -- 172.21.15.47:0/1394403131 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f05a8061750 con 0x7f05a80b0cf0 2022-06-29T15:50:12.352 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.348+0000 7f05a5ffb700 1 -- 172.21.15.47:0/1394403131 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v27) v1 ==== 80+0+4255 (secure 0 0 0) 0x7f05a000f040 con 0x7f05a80b0cf0 2022-06-29T15:50:12.352 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:50:12.353 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2022-06-29T15:50:12.353 INFO:teuthology.orchestra.run.smithi047.stdout: "auth_dump": [ 2022-06-29T15:50:12.353 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:12.354 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.a", 2022-06-29T15:50:12.354 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxiQJFFMxAA9Tv7FJD9u/BKvbkuKld6Kg==", 2022-06-29T15:50:12.354 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:50:12.354 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:50:12.355 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:50:12.355 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:50:12.355 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:50:12.355 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:12.356 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:12.356 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:12.356 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.b", 2022-06-29T15:50:12.356 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC2crxibt6FARAAEf2bwXKFhXKtqr1nN0rdXQ==", 2022-06-29T15:50:12.357 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:50:12.357 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:50:12.357 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:50:12.357 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:50:12.358 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:50:12.358 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:12.358 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:12.358 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:12.359 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.c", 2022-06-29T15:50:12.359 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxi/4ZMORAANpLbgTm3ehqi80USc3swGg==", 2022-06-29T15:50:12.359 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:50:12.359 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:50:12.360 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:50:12.360 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:50:12.360 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:50:12.360 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:12.361 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:12.361 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:12.361 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.0", 2022-06-29T15:50:12.361 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC2crxiY0YsMxAAZivb+CxL/w7vZwQv37Y4oQ==", 2022-06-29T15:50:12.361 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:50:12.362 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:50:12.362 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:50:12.362 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:50:12.362 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:12.363 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:12.363 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:12.363 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.1", 2022-06-29T15:50:12.363 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC5crxitLkKFxAA8+++LUUKLiO9SAuAfYgT5g==", 2022-06-29T15:50:12.364 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:50:12.364 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:50:12.364 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:50:12.364 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:50:12.365 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:12.365 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:12.365 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:12.365 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.2", 2022-06-29T15:50:12.366 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC7crxi1oM9NxAA2FfLdNt32YPwlyNdGQpaQA==", 2022-06-29T15:50:12.366 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:50:12.366 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:50:12.366 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:50:12.367 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:50:12.367 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:12.367 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:12.367 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:12.368 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.3", 2022-06-29T15:50:12.368 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC+crxiTsBgGhAA+gLBBDaskPbmRNWzgKwcHA==", 2022-06-29T15:50:12.368 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:50:12.368 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:50:12.368 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:50:12.369 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:50:12.369 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:12.369 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:12.369 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:12.369 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.4", 2022-06-29T15:50:12.370 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDDcrxivJebIhAAfwWJa7tK4hamkk1/1CSSfw==", 2022-06-29T15:50:12.370 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:50:12.370 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:50:12.370 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:50:12.371 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:50:12.371 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:12.371 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:12.371 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:12.371 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.5", 2022-06-29T15:50:12.372 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDGcrxijuozBhAAt+Dx3wH3WKp3DrbCbhmh8Q==", 2022-06-29T15:50:12.372 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:50:12.372 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:50:12.372 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:50:12.373 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:50:12.373 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:12.373 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:12.373 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:12.373 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.6", 2022-06-29T15:50:12.374 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDIcrxi91kXJxAA9JX7n98P394bD41Q9LUdvQ==", 2022-06-29T15:50:12.374 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:50:12.374 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:50:12.374 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:50:12.375 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:50:12.375 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:12.375 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:12.375 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:12.375 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.7", 2022-06-29T15:50:12.376 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDLcrxiHj0ZDRAASy6fAix8TmEZbAGtuFCfFg==", 2022-06-29T15:50:12.376 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:50:12.376 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:50:12.376 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:50:12.376 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:50:12.377 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:12.377 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:12.377 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:12.377 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.0", 2022-06-29T15:50:12.378 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC2crxiVDlfBxAAezkUfg4e2g0Rfakaa/Ek1Q==", 2022-06-29T15:50:12.378 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:50:12.378 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:50:12.378 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow r", 2022-06-29T15:50:12.378 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-29T15:50:12.379 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:12.379 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:12.379 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:12.379 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.1", 2022-06-29T15:50:12.380 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC2crxisXJ1CxAAhl07oF7AW66NdzfvvZTJmA==", 2022-06-29T15:50:12.380 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:50:12.380 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:50:12.380 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow r", 2022-06-29T15:50:12.381 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-29T15:50:12.381 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:12.381 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:12.381 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:12.381 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.admin", 2022-06-29T15:50:12.382 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxiW7HSFRAAhYqoGU9GFqj2HfUVGValSA==", 2022-06-29T15:50:12.382 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:50:12.382 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2022-06-29T15:50:12.382 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:50:12.383 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:50:12.383 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:50:12.383 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:12.383 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:12.383 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:12.384 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.x", 2022-06-29T15:50:12.384 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxidmuIMBAAcpNlYVu5+icCHg+yIjcKXw==", 2022-06-29T15:50:12.384 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:50:12.384 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2022-06-29T15:50:12.384 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2022-06-29T15:50:12.385 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:50:12.385 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:12.385 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:12.385 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:12.386 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.y", 2022-06-29T15:50:12.386 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxisSMFLBAAv03r4OoPjOP9nI9G/LK0iw==", 2022-06-29T15:50:12.386 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:50:12.386 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2022-06-29T15:50:12.386 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2022-06-29T15:50:12.387 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:50:12.387 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:12.387 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:12.387 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:12.388 INFO:teuthology.orchestra.run.smithi047.stdout:} 2022-06-29T15:50:12.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.348+0000 7f05aefda700 1 -- 172.21.15.47:0/1394403131 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0588053a70 msgr2=0x7f0588055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:12.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.348+0000 7f05aefda700 1 --2- 172.21.15.47:0/1394403131 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0588053a70 0x7f0588055f30 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7f059c00c000 tx=0x7f059c008040).stop 2022-06-29T15:50:12.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.348+0000 7f05aefda700 1 -- 172.21.15.47:0/1394403131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05a80b0cf0 msgr2=0x7f05a813b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:12.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.348+0000 7f05aefda700 1 --2- 172.21.15.47:0/1394403131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05a80b0cf0 0x7f05a813b050 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 rx=0x7f05a000a000 tx=0x7f05a0008040).stop 2022-06-29T15:50:12.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.348+0000 7f05aefda700 1 -- 172.21.15.47:0/1394403131 shutdown_connections 2022-06-29T15:50:12.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.348+0000 7f05aefda700 1 --2- 172.21.15.47:0/1394403131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05a80b0cf0 0x7f05a813b050 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:12.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.348+0000 7f05aefda700 1 --2- 172.21.15.47:0/1394403131 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0588053a70 0x7f0588055f30 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:12.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.348+0000 7f05aefda700 1 --2- 172.21.15.47:0/1394403131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05a80b06d0 0x7f05a813ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:12.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.348+0000 7f05aefda700 1 --2- 172.21.15.47:0/1394403131 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f05a80611b0 0x7f05a8148ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:12.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.348+0000 7f05aefda700 1 -- 172.21.15.47:0/1394403131 >> 172.21.15.47:0/1394403131 conn(0x7f05a80b98b0 msgr2=0x7f05a80b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:12.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.348+0000 7f05aefda700 1 -- 172.21.15.47:0/1394403131 shutdown_connections 2022-06-29T15:50:12.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.348+0000 7f05aefda700 1 -- 172.21.15.47:0/1394403131 wait complete. 2022-06-29T15:50:12.393 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:50:12.498 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.492+0000 7f7f4b40a700 1 Processor -- start 2022-06-29T15:50:12.498 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.492+0000 7f7f4b40a700 1 -- start start 2022-06-29T15:50:12.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.492+0000 7f7f4b40a700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7f440619b0 0x7f7f440606e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:12.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.492+0000 7f7f4b40a700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7f440b0f30 0x7f7f441408e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:12.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.492+0000 7f7f4b40a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7f440b1a80 0x7f7f44139a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:12.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.492+0000 7f7f4b40a700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f44061d90 con 0x7f7f440b1a80 2022-06-29T15:50:12.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.492+0000 7f7f4b40a700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f44060d80 con 0x7f7f440619b0 2022-06-29T15:50:12.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.492+0000 7f7f4b40a700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4413ff00 con 0x7f7f440b0f30 2022-06-29T15:50:12.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.492+0000 7f7f499a7700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7f440b1a80 0x7f7f44139a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:12.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.492+0000 7f7f499a7700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7f440b1a80 0x7f7f44139a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51960/0 (socket says 172.21.15.47:51960) 2022-06-29T15:50:12.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.492+0000 7f7f499a7700 1 -- 172.21.15.47:0/957605899 learned_addr learned my addr 172.21.15.47:0/957605899 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:12.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.492+0000 7f7f489a5700 1 --2- 172.21.15.47:0/957605899 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7f440b0f30 0x7f7f441408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:12.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.492+0000 7f7f491a6700 1 --2- 172.21.15.47:0/957605899 >> v2:172.21.15.64:3300/0 conn(0x7f7f440619b0 0x7f7f440606e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:12.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.496+0000 7f7f499a7700 1 -- 172.21.15.47:0/957605899 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7f440b0f30 msgr2=0x7f7f441408e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:12.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.496+0000 7f7f499a7700 1 --2- 172.21.15.47:0/957605899 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7f440b0f30 0x7f7f441408e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:12.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.496+0000 7f7f499a7700 1 -- 172.21.15.47:0/957605899 >> v2:172.21.15.64:3300/0 conn(0x7f7f440619b0 msgr2=0x7f7f440606e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:12.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.496+0000 7f7f499a7700 1 --2- 172.21.15.47:0/957605899 >> v2:172.21.15.64:3300/0 conn(0x7f7f440619b0 0x7f7f440606e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:12.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.496+0000 7f7f499a7700 1 -- 172.21.15.47:0/957605899 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f441401a0 con 0x7f7f440b1a80 2022-06-29T15:50:12.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.496+0000 7f7f499a7700 1 --2- 172.21.15.47:0/957605899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f440b1a80 0x7f7f44139a20 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 rx=0x7f7f3c0032f0 tx=0x7f7f3c009290).ready entity=mon.0 client_cookie=4ac70738763b5322 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:12.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.496+0000 7f7f37fff700 1 -- 172.21.15.47:0/957605899 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f3c00da80 con 0x7f7f440b1a80 2022-06-29T15:50:12.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.496+0000 7f7f37fff700 1 -- 172.21.15.47:0/957605899 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f3c00dbe0 con 0x7f7f440b1a80 2022-06-29T15:50:12.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.496+0000 7f7f37fff700 1 -- 172.21.15.47:0/957605899 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f3c010470 con 0x7f7f440b1a80 2022-06-29T15:50:12.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.496+0000 7f7f4b40a700 1 -- 172.21.15.47:0/957605899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f440b1a80 msgr2=0x7f7f44139a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:12.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.496+0000 7f7f4b40a700 1 --2- 172.21.15.47:0/957605899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f440b1a80 0x7f7f44139a20 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 rx=0x7f7f3c0032f0 tx=0x7f7f3c009290).stop 2022-06-29T15:50:12.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.496+0000 7f7f4b40a700 1 -- 172.21.15.47:0/957605899 shutdown_connections 2022-06-29T15:50:12.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.496+0000 7f7f4b40a700 1 --2- 172.21.15.47:0/957605899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f440b1a80 0x7f7f44139a20 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:12.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.496+0000 7f7f4b40a700 1 --2- 172.21.15.47:0/957605899 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7f440b0f30 0x7f7f441408e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:12.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.496+0000 7f7f4b40a700 1 --2- 172.21.15.47:0/957605899 >> v2:172.21.15.64:3300/0 conn(0x7f7f440619b0 0x7f7f440606e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:12.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.496+0000 7f7f4b40a700 1 -- 172.21.15.47:0/957605899 >> 172.21.15.47:0/957605899 conn(0x7f7f4405c510 msgr2=0x7f7f44063800 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:12.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.496+0000 7f7f4b40a700 1 -- 172.21.15.47:0/957605899 shutdown_connections 2022-06-29T15:50:12.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.496+0000 7f7f4b40a700 1 -- 172.21.15.47:0/957605899 wait complete. 2022-06-29T15:50:12.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.496+0000 7f7f4b40a700 1 Processor -- start 2022-06-29T15:50:12.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.496+0000 7f7f4b40a700 1 -- start start 2022-06-29T15:50:12.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.500+0000 7f7f4b40a700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7f440619b0 0x7f7f44143500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:12.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.500+0000 7f7f4b40a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f440b0f30 0x7f7f440663a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:12.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.500+0000 7f7f4b40a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f440b1a80 0x7f7f44066890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:12.513 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.500+0000 7f7f4b40a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f44065c60 con 0x7f7f440b1a80 2022-06-29T15:50:12.513 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.500+0000 7f7f4b40a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f44065a00 con 0x7f7f440b0f30 2022-06-29T15:50:12.513 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.500+0000 7f7f4b40a700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f44064a50 con 0x7f7f440619b0 2022-06-29T15:50:12.513 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.500+0000 7f7f499a7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f440b1a80 0x7f7f44066890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:12.513 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.500+0000 7f7f499a7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f440b1a80 0x7f7f44066890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51966/0 (socket says 172.21.15.47:51966) 2022-06-29T15:50:12.514 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.500+0000 7f7f499a7700 1 -- 172.21.15.47:0/3036225468 learned_addr learned my addr 172.21.15.47:0/3036225468 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:12.514 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.500+0000 7f7f489a5700 1 --2- 172.21.15.47:0/3036225468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f440b0f30 0x7f7f440663a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:12.514 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.500+0000 7f7f491a6700 1 --2- 172.21.15.47:0/3036225468 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7f440619b0 0x7f7f44143500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:12.514 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.500+0000 7f7f499a7700 1 -- 172.21.15.47:0/3036225468 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7f440619b0 msgr2=0x7f7f44143500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:12.515 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.500+0000 7f7f499a7700 1 --2- 172.21.15.47:0/3036225468 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7f440619b0 0x7f7f44143500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:12.515 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.500+0000 7f7f499a7700 1 -- 172.21.15.47:0/3036225468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f440b0f30 msgr2=0x7f7f440663a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:12.515 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.500+0000 7f7f499a7700 1 --2- 172.21.15.47:0/3036225468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f440b0f30 0x7f7f440663a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:12.515 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.500+0000 7f7f499a7700 1 -- 172.21.15.47:0/3036225468 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f3c009530 con 0x7f7f440b1a80 2022-06-29T15:50:12.515 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.500+0000 7f7f499a7700 1 --2- 172.21.15.47:0/3036225468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f440b1a80 0x7f7f44066890 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 rx=0x7f7f3c0032f0 tx=0x7f7f3c019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:12.516 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.500+0000 7f7f367fc700 1 -- 172.21.15.47:0/3036225468 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f3c01f070 con 0x7f7f440b1a80 2022-06-29T15:50:12.516 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.500+0000 7f7f367fc700 1 -- 172.21.15.47:0/3036225468 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f3c019640 con 0x7f7f440b1a80 2022-06-29T15:50:12.516 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.500+0000 7f7f4b40a700 1 -- 172.21.15.47:0/3036225468 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f44064cf0 con 0x7f7f440b1a80 2022-06-29T15:50:12.516 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.500+0000 7f7f367fc700 1 -- 172.21.15.47:0/3036225468 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f3c0199a0 con 0x7f7f440b1a80 2022-06-29T15:50:12.517 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.500+0000 7f7f4b40a700 1 -- 172.21.15.47:0/3036225468 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f440b5f40 con 0x7f7f440b1a80 2022-06-29T15:50:12.517 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.500+0000 7f7f367fc700 1 -- 172.21.15.47:0/3036225468 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7f3c016030 con 0x7f7f440b1a80 2022-06-29T15:50:12.517 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.504+0000 7f7f4b40a700 1 -- 172.21.15.47:0/3036225468 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f44064e80 con 0x7f7f440b1a80 2022-06-29T15:50:12.517 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.504+0000 7f7f367fc700 1 --2- 172.21.15.47:0/3036225468 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7f2c053980 0x7f7f2c055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:12.517 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.504+0000 7f7f491a6700 1 --2- 172.21.15.47:0/3036225468 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7f2c053980 0x7f7f2c055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:12.518 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.504+0000 7f7f367fc700 1 -- 172.21.15.47:0/3036225468 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f7f3c06edc0 con 0x7f7f440b1a80 2022-06-29T15:50:12.518 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.504+0000 7f7f491a6700 1 --2- 172.21.15.47:0/3036225468 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7f2c053980 0x7f7f2c055e40 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7f7f4000b000 tx=0x7f7f40008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:12.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.508+0000 7f7f367fc700 1 -- 172.21.15.47:0/3036225468 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7f3c045db0 con 0x7f7f440b1a80 2022-06-29T15:50:12.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.636+0000 7f7f4b40a700 1 -- 172.21.15.47:0/3036225468 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7f440573d0 con 0x7f7f440b1a80 2022-06-29T15:50:12.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.640+0000 7f7f367fc700 1 -- 172.21.15.47:0/3036225468 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+13416 (secure 0 0 0) 0x7f7f3c0493d0 con 0x7f7f440b1a80 2022-06-29T15:50:12.644 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:50:12.645 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":107,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:50:12.017765+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:50:12.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.640+0000 7f7f4b40a700 1 -- 172.21.15.47:0/3036225468 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7f2c053980 msgr2=0x7f7f2c055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:12.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.640+0000 7f7f4b40a700 1 --2- 172.21.15.47:0/3036225468 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7f2c053980 0x7f7f2c055e40 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7f7f4000b000 tx=0x7f7f40008040).stop 2022-06-29T15:50:12.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.640+0000 7f7f4b40a700 1 -- 172.21.15.47:0/3036225468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f440b1a80 msgr2=0x7f7f44066890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:12.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.640+0000 7f7f4b40a700 1 --2- 172.21.15.47:0/3036225468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f440b1a80 0x7f7f44066890 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 rx=0x7f7f3c0032f0 tx=0x7f7f3c019000).stop 2022-06-29T15:50:12.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.640+0000 7f7f4b40a700 1 -- 172.21.15.47:0/3036225468 shutdown_connections 2022-06-29T15:50:12.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.640+0000 7f7f4b40a700 1 --2- 172.21.15.47:0/3036225468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f440b1a80 0x7f7f44066890 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:12.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.640+0000 7f7f4b40a700 1 --2- 172.21.15.47:0/3036225468 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7f2c053980 0x7f7f2c055e40 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:12.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.640+0000 7f7f4b40a700 1 --2- 172.21.15.47:0/3036225468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f440b0f30 0x7f7f440663a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:12.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.640+0000 7f7f4b40a700 1 --2- 172.21.15.47:0/3036225468 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7f440619b0 0x7f7f44143500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:12.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.640+0000 7f7f4b40a700 1 -- 172.21.15.47:0/3036225468 >> 172.21.15.47:0/3036225468 conn(0x7f7f4405c510 msgr2=0x7f7f440af300 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:12.651 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.640+0000 7f7f4b40a700 1 -- 172.21.15.47:0/3036225468 shutdown_connections 2022-06-29T15:50:12.651 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.640+0000 7f7f4b40a700 1 -- 172.21.15.47:0/3036225468 wait complete. 2022-06-29T15:50:12.658 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-29T15:42:16.971431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-29T15:42:25.290884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-06-29T15:50:12.659 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-29T15:50:12.765 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.760+0000 7f77aa632700 1 Processor -- start 2022-06-29T15:50:12.766 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.760+0000 7f77aa632700 1 -- start start 2022-06-29T15:50:12.768 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.760+0000 7f77aa632700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f77a4146720 0x7f77a4140780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:12.768 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.760+0000 7f77aa632700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77a4061600 con 0x7f77a4061220 2022-06-29T15:50:12.768 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.760+0000 7f77aa632700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77a4146bc0 con 0x7f77a40b4320 2022-06-29T15:50:12.769 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.760+0000 7f77aa632700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77a4146e60 con 0x7f77a4146720 2022-06-29T15:50:12.769 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.764+0000 7f77a3fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f77a4061220 0x7f77a4140070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37524/0 (socket says 172.21.15.47:37524) 2022-06-29T15:50:12.769 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.764+0000 7f77a3fff700 1 -- 172.21.15.47:0/398712076 learned_addr learned my addr 172.21.15.47:0/398712076 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:12.769 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.764+0000 7f77a37fe700 1 --2- 172.21.15.47:0/398712076 >> v2:172.21.15.64:3300/0 conn(0x7f77a4146720 0x7f77a4140780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:12.770 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.764+0000 7f77a2ffd700 1 -- 172.21.15.47:0/398712076 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2338991916 0 0) 0x7f77a4061600 con 0x7f77a4061220 2022-06-29T15:50:12.770 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.764+0000 7f77a2ffd700 1 -- 172.21.15.47:0/398712076 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7790003260 con 0x7f77a4061220 2022-06-29T15:50:12.770 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.764+0000 7f77a37fe700 1 -- 172.21.15.47:0/398712076 >> v1:172.21.15.64:6789/0 conn(0x7f77a40b4320 legacy=0x7f77a4143230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:12.770 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.764+0000 7f77a37fe700 1 -- 172.21.15.47:0/398712076 >> v1:172.21.15.47:6789/0 conn(0x7f77a4061220 legacy=0x7f77a4140070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:12.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.764+0000 7f77a37fe700 1 -- 172.21.15.47:0/398712076 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a4147100 con 0x7f77a4146720 2022-06-29T15:50:12.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.764+0000 7f77a37fe700 1 --2- 172.21.15.47:0/398712076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77a4146720 0x7f77a4140780 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7f7798002f70 tx=0x7f7798006df0).ready entity=mon.1 client_cookie=ab92e8bac5917297 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:12.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.764+0000 7f77a2ffd700 1 -- 172.21.15.47:0/398712076 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f779800d610 con 0x7f77a4146720 2022-06-29T15:50:12.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.764+0000 7f77a2ffd700 1 -- 172.21.15.47:0/398712076 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7798000e90 con 0x7f77a4146720 2022-06-29T15:50:12.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.764+0000 7f77a2ffd700 1 -- 172.21.15.47:0/398712076 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f779800d960 con 0x7f77a4146720 2022-06-29T15:50:12.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.764+0000 7f77aa632700 1 -- 172.21.15.47:0/398712076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77a4146720 msgr2=0x7f77a4140780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:12.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.764+0000 7f77aa632700 1 --2- 172.21.15.47:0/398712076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77a4146720 0x7f77a4140780 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7f7798002f70 tx=0x7f7798006df0).stop 2022-06-29T15:50:12.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.764+0000 7f77aa632700 1 -- 172.21.15.47:0/398712076 shutdown_connections 2022-06-29T15:50:12.776 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.764+0000 7f77aa632700 1 --2- 172.21.15.47:0/398712076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77a4146720 0x7f77a4140780 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:12.776 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.764+0000 7f77aa632700 1 -- 172.21.15.47:0/398712076 >> 172.21.15.47:0/398712076 conn(0x7f77a405c6b0 msgr2=0x7f77a405f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:12.776 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.764+0000 7f77aa632700 1 -- 172.21.15.47:0/398712076 shutdown_connections 2022-06-29T15:50:12.776 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.764+0000 7f77aa632700 1 -- 172.21.15.47:0/398712076 wait complete. 2022-06-29T15:50:12.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.764+0000 7f77aa632700 1 Processor -- start 2022-06-29T15:50:12.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.764+0000 7f77aa632700 1 -- start start 2022-06-29T15:50:12.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.764+0000 7f77aa632700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f77a4061220 0x7f77a4152ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:12.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.764+0000 7f77aa632700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77a40b4320 0x7f77a413be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:12.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.764+0000 7f77aa632700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77a4146720 0x7f77a413c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:12.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.764+0000 7f77aa632700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a4153180 con 0x7f77a4146720 2022-06-29T15:50:12.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.764+0000 7f77aa632700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a413d980 con 0x7f77a40b4320 2022-06-29T15:50:12.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.764+0000 7f77aa632700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77a413dc20 con 0x7f77a4061220 2022-06-29T15:50:12.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.768+0000 7f77a8bcf700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77a4146720 0x7f77a413c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:12.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.768+0000 7f77a3fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f77a4061220 0x7f77a4152ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:12.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.768+0000 7f77a8bcf700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77a4146720 0x7f77a413c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51980/0 (socket says 172.21.15.47:51980) 2022-06-29T15:50:12.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.768+0000 7f77a8bcf700 1 -- 172.21.15.47:0/2299001652 learned_addr learned my addr 172.21.15.47:0/2299001652 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:12.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.768+0000 7f77a37fe700 1 --2- 172.21.15.47:0/2299001652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77a40b4320 0x7f77a413be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:12.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.768+0000 7f77a8bcf700 1 -- 172.21.15.47:0/2299001652 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f77a4061220 msgr2=0x7f77a4152ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:12.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.768+0000 7f77a8bcf700 1 --2- 172.21.15.47:0/2299001652 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f77a4061220 0x7f77a4152ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:12.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.768+0000 7f77a8bcf700 1 -- 172.21.15.47:0/2299001652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77a40b4320 msgr2=0x7f77a413be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:12.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.768+0000 7f77a8bcf700 1 --2- 172.21.15.47:0/2299001652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77a40b4320 0x7f77a413be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:12.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.768+0000 7f77a8bcf700 1 -- 172.21.15.47:0/2299001652 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f779800b180 con 0x7f77a4146720 2022-06-29T15:50:12.781 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.768+0000 7f77a8bcf700 1 --2- 172.21.15.47:0/2299001652 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77a4146720 0x7f77a413c2f0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 rx=0x7f77a40ae720 tx=0x7f779c007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:12.781 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.768+0000 7f77a17fa700 1 -- 172.21.15.47:0/2299001652 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f779c00f470 con 0x7f77a4146720 2022-06-29T15:50:12.781 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.768+0000 7f77a17fa700 1 -- 172.21.15.47:0/2299001652 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f779c00f5d0 con 0x7f77a4146720 2022-06-29T15:50:12.781 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.768+0000 7f77aa632700 1 -- 172.21.15.47:0/2299001652 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77a413dec0 con 0x7f77a4146720 2022-06-29T15:50:12.782 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.768+0000 7f77a17fa700 1 -- 172.21.15.47:0/2299001652 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f779c00f930 con 0x7f77a4146720 2022-06-29T15:50:12.782 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.768+0000 7f77aa632700 1 -- 172.21.15.47:0/2299001652 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77a413c990 con 0x7f77a4146720 2022-06-29T15:50:12.782 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.768+0000 7f77aa632700 1 -- 172.21.15.47:0/2299001652 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77a40573d0 con 0x7f77a4146720 2022-06-29T15:50:12.783 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.772+0000 7f77a17fa700 1 -- 172.21.15.47:0/2299001652 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f779c019030 con 0x7f77a4146720 2022-06-29T15:50:12.784 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.772+0000 7f77a17fa700 1 --2- 172.21.15.47:0/2299001652 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7784053a70 0x7f7784055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:12.784 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.772+0000 7f77a17fa700 1 -- 172.21.15.47:0/2299001652 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f779c06b300 con 0x7f77a4146720 2022-06-29T15:50:12.784 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.772+0000 7f77a3fff700 1 --2- 172.21.15.47:0/2299001652 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7784053a70 0x7f7784055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:12.784 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.772+0000 7f77a3fff700 1 --2- 172.21.15.47:0/2299001652 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7784053a70 0x7f7784055f30 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7f779400d000 tx=0x7f7794009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:12.784 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.780+0000 7f77a17fa700 1 -- 172.21.15.47:0/2299001652 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f779c042400 con 0x7f77a4146720 2022-06-29T15:50:12.912 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.908+0000 7f77aa632700 1 -- 172.21.15.47:0/2299001652 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f77a413e050 con 0x7f77a4146720 2022-06-29T15:50:12.913 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.908+0000 7f77a17fa700 1 -- 172.21.15.47:0/2299001652 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v107) v1 ==== 110+0+10 (secure 0 0 0) 0x7f779c045a20 con 0x7f77a4146720 2022-06-29T15:50:12.913 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2022-06-29T15:50:12.915 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.908+0000 7f77aa632700 1 -- 172.21.15.47:0/2299001652 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7784053a70 msgr2=0x7f7784055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:12.915 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.908+0000 7f77aa632700 1 --2- 172.21.15.47:0/2299001652 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7784053a70 0x7f7784055f30 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7f779400d000 tx=0x7f7794009040).stop 2022-06-29T15:50:12.916 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.908+0000 7f77aa632700 1 -- 172.21.15.47:0/2299001652 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77a4146720 msgr2=0x7f77a413c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:12.916 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.908+0000 7f77aa632700 1 --2- 172.21.15.47:0/2299001652 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77a4146720 0x7f77a413c2f0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 rx=0x7f77a40ae720 tx=0x7f779c007f40).stop 2022-06-29T15:50:12.916 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.908+0000 7f77aa632700 1 -- 172.21.15.47:0/2299001652 shutdown_connections 2022-06-29T15:50:12.917 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.908+0000 7f77aa632700 1 --2- 172.21.15.47:0/2299001652 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77a4146720 0x7f77a413c2f0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:12.917 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.908+0000 7f77aa632700 1 --2- 172.21.15.47:0/2299001652 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7784053a70 0x7f7784055f30 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:12.917 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.908+0000 7f77aa632700 1 --2- 172.21.15.47:0/2299001652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77a40b4320 0x7f77a413be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:12.918 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.908+0000 7f77aa632700 1 --2- 172.21.15.47:0/2299001652 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f77a4061220 0x7f77a4152ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:12.918 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.908+0000 7f77aa632700 1 -- 172.21.15.47:0/2299001652 >> 172.21.15.47:0/2299001652 conn(0x7f77a405c6b0 msgr2=0x7f77a4140050 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:12.919 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.908+0000 7f77aa632700 1 -- 172.21.15.47:0/2299001652 shutdown_connections 2022-06-29T15:50:12.919 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:12.908+0000 7f77aa632700 1 -- 172.21.15.47:0/2299001652 wait complete. 2022-06-29T15:50:12.928 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-29T15:50:13.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.036+0000 7f591eeab700 1 Processor -- start 2022-06-29T15:50:13.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.036+0000 7f591eeab700 1 -- start start 2022-06-29T15:50:13.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.036+0000 7f591eeab700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f59181467a0 0x7f5918140500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:13.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.036+0000 7f591eeab700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f59181409f0 0x7f591814db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:13.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.036+0000 7f591eeab700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59180b1050 con 0x7f59180b6d20 2022-06-29T15:50:13.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.036+0000 7f591eeab700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59180b23c0 con 0x7f59181409f0 2022-06-29T15:50:13.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.036+0000 7f591eeab700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f591805fab0 con 0x7f59181467a0 2022-06-29T15:50:13.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.036+0000 7f591e6aa700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f59181409f0 0x7f591814db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:13.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.036+0000 7f591e6aa700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f59181409f0 0x7f591814db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51988/0 (socket says 172.21.15.47:51988) 2022-06-29T15:50:13.045 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.036+0000 7f591e6aa700 1 -- 172.21.15.47:0/697705908 learned_addr learned my addr 172.21.15.47:0/697705908 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:13.045 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.036+0000 7f591d6a8700 1 --2- 172.21.15.47:0/697705908 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f59181467a0 0x7f5918140500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:13.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.040+0000 7f591d6a8700 1 -- 172.21.15.47:0/697705908 >> v2:172.21.15.47:3300/0 conn(0x7f59181409f0 msgr2=0x7f591814db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:13.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.040+0000 7f591d6a8700 1 --2- 172.21.15.47:0/697705908 >> v2:172.21.15.47:3300/0 conn(0x7f59181409f0 0x7f591814db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:13.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.040+0000 7f591d6a8700 1 -- 172.21.15.47:0/697705908 >> v1:172.21.15.64:6789/0 conn(0x7f59180b6d20 legacy=0x7f59180b1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:13.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.040+0000 7f591cea7700 1 -- 172.21.15.47:0/697705908 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4292348621 0 0) 0x7f59180b1050 con 0x7f59180b6d20 2022-06-29T15:50:13.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.040+0000 7f591d6a8700 1 -- 172.21.15.47:0/697705908 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59180b7100 con 0x7f59181467a0 2022-06-29T15:50:13.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.040+0000 7f591e6aa700 1 --2- 172.21.15.47:0/697705908 >> v2:172.21.15.47:3300/0 conn(0x7f59181409f0 0x7f591814db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:50:13.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.040+0000 7f591d6a8700 1 --2- 172.21.15.47:0/697705908 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f59181467a0 0x7f5918140500 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f59180b2700 tx=0x7f5914006df0).ready entity=mon.2 client_cookie=e79ea7db23c55606 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:13.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.040+0000 7f591cea7700 1 -- 172.21.15.47:0/697705908 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f591400d570 con 0x7f59181467a0 2022-06-29T15:50:13.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.040+0000 7f591cea7700 1 -- 172.21.15.47:0/697705908 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5914000e90 con 0x7f59181467a0 2022-06-29T15:50:13.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.040+0000 7f591cea7700 1 -- 172.21.15.47:0/697705908 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f591400d8c0 con 0x7f59181467a0 2022-06-29T15:50:13.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.040+0000 7f591eeab700 1 -- 172.21.15.47:0/697705908 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f59181467a0 msgr2=0x7f5918140500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:13.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.040+0000 7f591eeab700 1 --2- 172.21.15.47:0/697705908 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f59181467a0 0x7f5918140500 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f59180b2700 tx=0x7f5914006df0).stop 2022-06-29T15:50:13.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.040+0000 7f591eeab700 1 -- 172.21.15.47:0/697705908 shutdown_connections 2022-06-29T15:50:13.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.040+0000 7f591eeab700 1 --2- 172.21.15.47:0/697705908 >> v2:172.21.15.47:3300/0 conn(0x7f59181409f0 0x7f591814db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:13.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.040+0000 7f591eeab700 1 --2- 172.21.15.47:0/697705908 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f59181467a0 0x7f5918140500 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:13.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.040+0000 7f591eeab700 1 -- 172.21.15.47:0/697705908 >> 172.21.15.47:0/697705908 conn(0x7f5918063670 msgr2=0x7f5918063a50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:13.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.040+0000 7f591eeab700 1 -- 172.21.15.47:0/697705908 shutdown_connections 2022-06-29T15:50:13.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.040+0000 7f591eeab700 1 -- 172.21.15.47:0/697705908 wait complete. 2022-06-29T15:50:13.056 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.044+0000 7f591eeab700 1 Processor -- start 2022-06-29T15:50:13.057 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.044+0000 7f591eeab700 1 -- start start 2022-06-29T15:50:13.057 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.044+0000 7f591eeab700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f59180b6d20 0x7f591814a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:13.058 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.044+0000 7f591eeab700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59181409f0 0x7f591813f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:13.058 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.044+0000 7f591eeab700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59181467a0 0x7f591813fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:13.058 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.044+0000 7f591eeab700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5918145260 con 0x7f59181409f0 2022-06-29T15:50:13.059 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.044+0000 7f591eeab700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5918142ef0 con 0x7f59181467a0 2022-06-29T15:50:13.059 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.044+0000 7f591eeab700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5918143190 con 0x7f59180b6d20 2022-06-29T15:50:13.060 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.044+0000 7f591d6a8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59181409f0 0x7f591813f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:13.060 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.044+0000 7f591d6a8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59181409f0 0x7f591813f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51992/0 (socket says 172.21.15.47:51992) 2022-06-29T15:50:13.060 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.044+0000 7f591d6a8700 1 -- 172.21.15.47:0/2362360148 learned_addr learned my addr 172.21.15.47:0/2362360148 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:13.061 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.044+0000 7f591e6aa700 1 --2- 172.21.15.47:0/2362360148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59181467a0 0x7f591813fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:13.061 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.044+0000 7f591dea9700 1 --2- 172.21.15.47:0/2362360148 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f59180b6d20 0x7f591814a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:13.062 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.044+0000 7f591d6a8700 1 -- 172.21.15.47:0/2362360148 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f59180b6d20 msgr2=0x7f591814a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:13.062 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.044+0000 7f591d6a8700 1 --2- 172.21.15.47:0/2362360148 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f59180b6d20 0x7f591814a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:13.063 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.044+0000 7f591d6a8700 1 -- 172.21.15.47:0/2362360148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59181467a0 msgr2=0x7f591813fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:13.063 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.044+0000 7f591d6a8700 1 --2- 172.21.15.47:0/2362360148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59181467a0 0x7f591813fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:13.063 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.044+0000 7f591d6a8700 1 -- 172.21.15.47:0/2362360148 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5910009530 con 0x7f59181409f0 2022-06-29T15:50:13.063 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.044+0000 7f591dea9700 1 --2- 172.21.15.47:0/2362360148 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f59180b6d20 0x7f591814a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:50:13.064 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.044+0000 7f591d6a8700 1 --2- 172.21.15.47:0/2362360148 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59181409f0 0x7f591813f6b0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 rx=0x7f59180b2700 tx=0x7f5914014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:13.064 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.044+0000 7f5906ffd700 1 -- 172.21.15.47:0/2362360148 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5914016070 con 0x7f59181409f0 2022-06-29T15:50:13.064 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.044+0000 7f5906ffd700 1 -- 172.21.15.47:0/2362360148 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5914013910 con 0x7f59181409f0 2022-06-29T15:50:13.064 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.048+0000 7f591eeab700 1 -- 172.21.15.47:0/2362360148 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f591400b180 con 0x7f59181409f0 2022-06-29T15:50:13.065 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.048+0000 7f5906ffd700 1 -- 172.21.15.47:0/2362360148 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5914013c70 con 0x7f59181409f0 2022-06-29T15:50:13.065 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.048+0000 7f591eeab700 1 -- 172.21.15.47:0/2362360148 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5918144f70 con 0x7f59181409f0 2022-06-29T15:50:13.065 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.048+0000 7f591eeab700 1 -- 172.21.15.47:0/2362360148 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59180b1f90 con 0x7f59181409f0 2022-06-29T15:50:13.065 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.048+0000 7f5906ffd700 1 -- 172.21.15.47:0/2362360148 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5914025030 con 0x7f59181409f0 2022-06-29T15:50:13.066 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.048+0000 7f5906ffd700 1 --2- 172.21.15.47:0/2362360148 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f58f8053950 0x7f58f8055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:13.066 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.048+0000 7f5906ffd700 1 -- 172.21.15.47:0/2362360148 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f591406cc50 con 0x7f59181409f0 2022-06-29T15:50:13.066 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.048+0000 7f591dea9700 1 --2- 172.21.15.47:0/2362360148 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f58f8053950 0x7f58f8055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:13.066 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.048+0000 7f591dea9700 1 --2- 172.21.15.47:0/2362360148 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f58f8053950 0x7f58f8055e10 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7f590800b000 tx=0x7f5908009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:13.067 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.056+0000 7f5906ffd700 1 -- 172.21.15.47:0/2362360148 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f59180b1f90 con 0x7f59181409f0 2022-06-29T15:50:13.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.184+0000 7f591eeab700 1 -- 172.21.15.47:0/2362360148 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f59180b1f90 con 0x7f59181409f0 2022-06-29T15:50:13.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.184+0000 7f5906ffd700 1 -- 172.21.15.47:0/2362360148 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v107) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5914023070 con 0x7f59181409f0 2022-06-29T15:50:13.189 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:50:13.191 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.184+0000 7f591eeab700 1 -- 172.21.15.47:0/2362360148 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f58f8053950 msgr2=0x7f58f8055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:13.192 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.184+0000 7f591eeab700 1 --2- 172.21.15.47:0/2362360148 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f58f8053950 0x7f58f8055e10 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7f590800b000 tx=0x7f5908009040).stop 2022-06-29T15:50:13.192 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.184+0000 7f591eeab700 1 -- 172.21.15.47:0/2362360148 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59181409f0 msgr2=0x7f591813f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:13.193 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.184+0000 7f591eeab700 1 --2- 172.21.15.47:0/2362360148 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59181409f0 0x7f591813f6b0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 rx=0x7f59180b2700 tx=0x7f5914014000).stop 2022-06-29T15:50:13.193 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.184+0000 7f591eeab700 1 -- 172.21.15.47:0/2362360148 shutdown_connections 2022-06-29T15:50:13.194 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.184+0000 7f591eeab700 1 --2- 172.21.15.47:0/2362360148 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f58f8053950 0x7f58f8055e10 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:13.194 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.184+0000 7f591eeab700 1 --2- 172.21.15.47:0/2362360148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59181467a0 0x7f591813fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:13.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.184+0000 7f591eeab700 1 --2- 172.21.15.47:0/2362360148 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59181409f0 0x7f591813f6b0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:13.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.184+0000 7f591eeab700 1 --2- 172.21.15.47:0/2362360148 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f59180b6d20 0x7f591814a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:13.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.184+0000 7f591eeab700 1 -- 172.21.15.47:0/2362360148 >> 172.21.15.47:0/2362360148 conn(0x7f5918063670 msgr2=0x7f5918060a80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:13.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.188+0000 7f591eeab700 1 -- 172.21.15.47:0/2362360148 shutdown_connections 2022-06-29T15:50:13.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.188+0000 7f591eeab700 1 -- 172.21.15.47:0/2362360148 wait complete. 2022-06-29T15:50:13.206 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-06-29T15:50:13.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.304+0000 7fdc7b59e700 1 Processor -- start 2022-06-29T15:50:13.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.304+0000 7fdc7b59e700 1 -- start start 2022-06-29T15:50:13.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.304+0000 7fdc7b59e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdc7c0b5c60 0x7fdc7c0afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:13.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.304+0000 7fdc7b59e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdc7c060a60 0x7fdc7c0aa080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:13.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.304+0000 7fdc7b59e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc7c0b6040 con 0x7fdc7c0601a0 2022-06-29T15:50:13.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.304+0000 7fdc7b59e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc7c0b11a0 con 0x7fdc7c0b5c60 2022-06-29T15:50:13.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.304+0000 7fdc7b59e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7c0b2900 con 0x7fdc7c060a60 2022-06-29T15:50:13.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.304+0000 7fdc7a59c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdc7c0b5c60 0x7fdc7c0afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:13.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.304+0000 7fdc79d9b700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdc7c060a60 0x7fdc7c0aa080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:13.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.304+0000 7fdc7ad9d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fdc7c0601a0 0x7fdc7c0ab7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37556/0 (socket says 172.21.15.47:37556) 2022-06-29T15:50:13.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.304+0000 7fdc7a59c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdc7c0b5c60 0x7fdc7c0afdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51998/0 (socket says 172.21.15.47:51998) 2022-06-29T15:50:13.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.304+0000 7fdc7ad9d700 1 -- 172.21.15.47:0/1337720328 learned_addr learned my addr 172.21.15.47:0/1337720328 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:13.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.304+0000 7fdc7a59c700 1 -- 172.21.15.47:0/1337720328 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdc7c060a60 msgr2=0x7fdc7c0aa080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:13.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.304+0000 7fdc7a59c700 1 --2- 172.21.15.47:0/1337720328 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdc7c060a60 0x7fdc7c0aa080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:13.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.304+0000 7fdc7a59c700 1 -- 172.21.15.47:0/1337720328 >> v1:172.21.15.47:6789/0 conn(0x7fdc7c0601a0 legacy=0x7fdc7c0ab7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:13.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.304+0000 7fdc7a59c700 1 -- 172.21.15.47:0/1337720328 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc7c0b2ba0 con 0x7fdc7c0b5c60 2022-06-29T15:50:13.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.304+0000 7fdc7a59c700 1 --2- 172.21.15.47:0/1337720328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc7c0b5c60 0x7fdc7c0afdc0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 rx=0x7fdc6c002f70 tx=0x7fdc6c006e20).ready entity=mon.0 client_cookie=5e8e3156185aac8 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:13.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.308+0000 7fdc7959a700 1 -- 172.21.15.47:0/1337720328 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc6c012070 con 0x7fdc7c0b5c60 2022-06-29T15:50:13.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.308+0000 7fdc7959a700 1 -- 172.21.15.47:0/1337720328 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc6c000e90 con 0x7fdc7c0b5c60 2022-06-29T15:50:13.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.308+0000 7fdc7959a700 1 -- 172.21.15.47:0/1337720328 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc6c00f7f0 con 0x7fdc7c0b5c60 2022-06-29T15:50:13.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.308+0000 7fdc7b59e700 1 -- 172.21.15.47:0/1337720328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc7c0b5c60 msgr2=0x7fdc7c0afdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:13.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.308+0000 7fdc7b59e700 1 --2- 172.21.15.47:0/1337720328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc7c0b5c60 0x7fdc7c0afdc0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 rx=0x7fdc6c002f70 tx=0x7fdc6c006e20).stop 2022-06-29T15:50:13.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.308+0000 7fdc7b59e700 1 -- 172.21.15.47:0/1337720328 shutdown_connections 2022-06-29T15:50:13.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.308+0000 7fdc7b59e700 1 --2- 172.21.15.47:0/1337720328 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdc7c060a60 0x7fdc7c0aa080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:13.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.308+0000 7fdc7b59e700 1 --2- 172.21.15.47:0/1337720328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc7c0b5c60 0x7fdc7c0afdc0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:13.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.308+0000 7fdc7b59e700 1 -- 172.21.15.47:0/1337720328 >> 172.21.15.47:0/1337720328 conn(0x7fdc7c062560 msgr2=0x7fdc7c062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:13.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.308+0000 7fdc7b59e700 1 -- 172.21.15.47:0/1337720328 shutdown_connections 2022-06-29T15:50:13.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.308+0000 7fdc7b59e700 1 -- 172.21.15.47:0/1337720328 wait complete. 2022-06-29T15:50:13.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.312+0000 7fdc7b59e700 1 Processor -- start 2022-06-29T15:50:13.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.312+0000 7fdc7b59e700 1 -- start start 2022-06-29T15:50:13.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.316+0000 7fdc7b59e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdc7c0601a0 0x7fdc7c13ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:13.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.316+0000 7fdc7b59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc7c060a60 0x7fdc7c13c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:13.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.316+0000 7fdc7b59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc7c0b5c60 0x7fdc7c14e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:13.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.316+0000 7fdc7b59e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7c0ac4c0 con 0x7fdc7c0b5c60 2022-06-29T15:50:13.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.316+0000 7fdc7b59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7c0aae10 con 0x7fdc7c060a60 2022-06-29T15:50:13.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.316+0000 7fdc7b59e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7c145210 con 0x7fdc7c0601a0 2022-06-29T15:50:13.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.316+0000 7fdc7a59c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdc7c0601a0 0x7fdc7c13ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:13.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.316+0000 7fdc79d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc7c060a60 0x7fdc7c13c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:13.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.316+0000 7fdc79d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc7c060a60 0x7fdc7c13c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:57710/0 (socket says 172.21.15.47:57710) 2022-06-29T15:50:13.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.316+0000 7fdc79d9b700 1 -- 172.21.15.47:0/1861462493 learned_addr learned my addr 172.21.15.47:0/1861462493 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:13.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.316+0000 7fdc7ad9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc7c0b5c60 0x7fdc7c14e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:13.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.316+0000 7fdc7ad9d700 1 -- 172.21.15.47:0/1861462493 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdc7c0601a0 msgr2=0x7fdc7c13ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:13.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.316+0000 7fdc7ad9d700 1 --2- 172.21.15.47:0/1861462493 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdc7c0601a0 0x7fdc7c13ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:13.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.316+0000 7fdc7ad9d700 1 -- 172.21.15.47:0/1861462493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc7c060a60 msgr2=0x7fdc7c13c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:13.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.316+0000 7fdc7ad9d700 1 --2- 172.21.15.47:0/1861462493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc7c060a60 0x7fdc7c13c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:13.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.316+0000 7fdc7ad9d700 1 -- 172.21.15.47:0/1861462493 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc6c00d530 con 0x7fdc7c0b5c60 2022-06-29T15:50:13.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.316+0000 7fdc7a59c700 1 --2- 172.21.15.47:0/1861462493 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdc7c0601a0 0x7fdc7c13ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:50:13.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.316+0000 7fdc7ad9d700 1 --2- 172.21.15.47:0/1861462493 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc7c0b5c60 0x7fdc7c14e020 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 rx=0x7fdc7c062280 tx=0x7fdc74007eb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:13.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.316+0000 7fdc6b7fe700 1 -- 172.21.15.47:0/1861462493 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc74005ba0 con 0x7fdc7c0b5c60 2022-06-29T15:50:13.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.316+0000 7fdc6b7fe700 1 -- 172.21.15.47:0/1861462493 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc74005d00 con 0x7fdc7c0b5c60 2022-06-29T15:50:13.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.316+0000 7fdc7b59e700 1 -- 172.21.15.47:0/1861462493 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc7c0ac1f0 con 0x7fdc7c0b5c60 2022-06-29T15:50:13.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.316+0000 7fdc6b7fe700 1 -- 172.21.15.47:0/1861462493 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc740072c0 con 0x7fdc7c0b5c60 2022-06-29T15:50:13.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.316+0000 7fdc7b59e700 1 -- 172.21.15.47:0/1861462493 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc7c0ab870 con 0x7fdc7c0b5c60 2022-06-29T15:50:13.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.320+0000 7fdc7b59e700 1 -- 172.21.15.47:0/1861462493 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc7c14de80 con 0x7fdc7c0b5c60 2022-06-29T15:50:13.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.320+0000 7fdc6b7fe700 1 -- 172.21.15.47:0/1861462493 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fdc74010070 con 0x7fdc7c0b5c60 2022-06-29T15:50:13.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.320+0000 7fdc6b7fe700 1 --2- 172.21.15.47:0/1861462493 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdc58053a70 0x7fdc58055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:13.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.320+0000 7fdc6b7fe700 1 -- 172.21.15.47:0/1861462493 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fdc7406b590 con 0x7fdc7c0b5c60 2022-06-29T15:50:13.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.320+0000 7fdc7a59c700 1 --2- 172.21.15.47:0/1861462493 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdc58053a70 0x7fdc58055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:13.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.320+0000 7fdc7a59c700 1 --2- 172.21.15.47:0/1861462493 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdc58053a70 0x7fdc58055f30 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7fdc6c01d000 tx=0x7fdc7c0b1900).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:13.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.324+0000 7fdc6b7fe700 1 -- 172.21.15.47:0/1861462493 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdc7c14de80 con 0x7fdc7c0b5c60 2022-06-29T15:50:13.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.451+0000 7fdc7b59e700 1 -- 172.21.15.47:0/1861462493 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fdc7c14de80 con 0x7fdc7c0b5c60 2022-06-29T15:50:13.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.451+0000 7fdc6b7fe700 1 -- 172.21.15.47:0/1861462493 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v107) v1 ==== 84+0+143 (secure 0 0 0) 0x7fdc7c14de80 con 0x7fdc7c0b5c60 2022-06-29T15:50:13.458 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:50:13.458 INFO:teuthology.orchestra.run.smithi047.stdout:[ 2022-06-29T15:50:13.458 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:13.459 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 1, 2022-06-29T15:50:13.459 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": "device_health_metrics" 2022-06-29T15:50:13.459 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:13.459 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:13.460 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 2, 2022-06-29T15:50:13.460 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": "rbd" 2022-06-29T15:50:13.460 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:13.460 INFO:teuthology.orchestra.run.smithi047.stdout:] 2022-06-29T15:50:13.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.455+0000 7fdc7b59e700 1 -- 172.21.15.47:0/1861462493 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdc58053a70 msgr2=0x7fdc58055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:13.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.455+0000 7fdc7b59e700 1 --2- 172.21.15.47:0/1861462493 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdc58053a70 0x7fdc58055f30 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7fdc6c01d000 tx=0x7fdc7c0b1900).stop 2022-06-29T15:50:13.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.455+0000 7fdc7b59e700 1 -- 172.21.15.47:0/1861462493 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc7c0b5c60 msgr2=0x7fdc7c14e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:13.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.455+0000 7fdc7b59e700 1 --2- 172.21.15.47:0/1861462493 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc7c0b5c60 0x7fdc7c14e020 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 rx=0x7fdc7c062280 tx=0x7fdc74007eb0).stop 2022-06-29T15:50:13.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.455+0000 7fdc7b59e700 1 -- 172.21.15.47:0/1861462493 shutdown_connections 2022-06-29T15:50:13.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.455+0000 7fdc7b59e700 1 --2- 172.21.15.47:0/1861462493 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc7c0b5c60 0x7fdc7c14e020 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:13.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.455+0000 7fdc7b59e700 1 --2- 172.21.15.47:0/1861462493 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdc58053a70 0x7fdc58055f30 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:13.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.455+0000 7fdc7b59e700 1 --2- 172.21.15.47:0/1861462493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc7c060a60 0x7fdc7c13c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:13.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.455+0000 7fdc7b59e700 1 --2- 172.21.15.47:0/1861462493 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdc7c0601a0 0x7fdc7c13ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:13.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.455+0000 7fdc7b59e700 1 -- 172.21.15.47:0/1861462493 >> 172.21.15.47:0/1861462493 conn(0x7fdc7c062560 msgr2=0x7fdc7c0adb10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:13.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.455+0000 7fdc7b59e700 1 -- 172.21.15.47:0/1861462493 shutdown_connections 2022-06-29T15:50:13.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.455+0000 7fdc7b59e700 1 -- 172.21.15.47:0/1861462493 wait complete. 2022-06-29T15:50:13.475 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-06-29T15:50:13.475 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-06-29T15:50:13.583 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.575+0000 7f01bc492700 1 Processor -- start 2022-06-29T15:50:13.584 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.575+0000 7f01bc492700 1 -- start start 2022-06-29T15:50:13.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.579+0000 7f01bc492700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f01b40612c0 0x7f01b41471a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:13.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.579+0000 7f01bc492700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01b4067610 con 0x7f01b4060ee0 2022-06-29T15:50:13.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.579+0000 7f01bc492700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01b41467c0 con 0x7f01b40b4300 2022-06-29T15:50:13.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.579+0000 7f01bc492700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01b4146a60 con 0x7f01b40612c0 2022-06-29T15:50:13.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.579+0000 7f01ba22e700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f01b4060ee0 0x7f01b41430e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37566/0 (socket says 172.21.15.47:37566) 2022-06-29T15:50:13.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.579+0000 7f01ba22e700 1 -- 172.21.15.47:0/4292373997 learned_addr learned my addr 172.21.15.47:0/4292373997 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:13.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.579+0000 7f01baa2f700 1 --2- 172.21.15.47:0/4292373997 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f01b40612c0 0x7f01b41471a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:13.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.579+0000 7f01b922c700 1 -- 172.21.15.47:0/4292373997 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2751948952 0 0) 0x7f01b4067610 con 0x7f01b4060ee0 2022-06-29T15:50:13.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.579+0000 7f01b922c700 1 -- 172.21.15.47:0/4292373997 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f019c003260 con 0x7f01b4060ee0 2022-06-29T15:50:13.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.579+0000 7f01b922c700 1 -- 172.21.15.47:0/4292373997 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1366797759 0 0) 0x7f01b41467c0 con 0x7f01b40b4300 2022-06-29T15:50:13.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.579+0000 7f01b922c700 1 -- 172.21.15.47:0/4292373997 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01b4067610 con 0x7f01b40b4300 2022-06-29T15:50:13.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.579+0000 7f01baa2f700 1 -- 172.21.15.47:0/4292373997 >> v1:172.21.15.64:6789/0 conn(0x7f01b40b4300 legacy=0x7f01b413cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:13.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.579+0000 7f01baa2f700 1 -- 172.21.15.47:0/4292373997 >> v1:172.21.15.47:6789/0 conn(0x7f01b4060ee0 legacy=0x7f01b41430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:13.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.579+0000 7f01b922c700 1 -- 172.21.15.47:0/4292373997 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3128233999 0 0) 0x7f019c003260 con 0x7f01b4060ee0 2022-06-29T15:50:13.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.579+0000 7f01baa2f700 1 -- 172.21.15.47:0/4292373997 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b4146d00 con 0x7f01b40612c0 2022-06-29T15:50:13.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.579+0000 7f01baa2f700 1 --2- 172.21.15.47:0/4292373997 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f01b40612c0 0x7f01b41471a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f01a4001110 tx=0x7f01ac0091d0).ready entity=mon.2 client_cookie=286485701d578f1b server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:13.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.579+0000 7f01b922c700 1 -- 172.21.15.47:0/4292373997 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01a4001660 con 0x7f01b40612c0 2022-06-29T15:50:13.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.579+0000 7f01b922c700 1 -- 172.21.15.47:0/4292373997 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01ac00f510 con 0x7f01b40612c0 2022-06-29T15:50:13.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.579+0000 7f01b922c700 1 -- 172.21.15.47:0/4292373997 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01ac00f680 con 0x7f01b40612c0 2022-06-29T15:50:13.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.579+0000 7f01bc492700 1 -- 172.21.15.47:0/4292373997 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f01b40612c0 msgr2=0x7f01b41471a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:13.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.579+0000 7f01bc492700 1 --2- 172.21.15.47:0/4292373997 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f01b40612c0 0x7f01b41471a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f01a4001110 tx=0x7f01ac0091d0).stop 2022-06-29T15:50:13.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.579+0000 7f01bc492700 1 -- 172.21.15.47:0/4292373997 shutdown_connections 2022-06-29T15:50:13.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.579+0000 7f01bc492700 1 --2- 172.21.15.47:0/4292373997 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f01b40612c0 0x7f01b41471a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:13.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.579+0000 7f01bc492700 1 -- 172.21.15.47:0/4292373997 >> 172.21.15.47:0/4292373997 conn(0x7f01b405c510 msgr2=0x7f01b405f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:13.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.579+0000 7f01bc492700 1 -- 172.21.15.47:0/4292373997 shutdown_connections 2022-06-29T15:50:13.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.579+0000 7f01bc492700 1 -- 172.21.15.47:0/4292373997 wait complete. 2022-06-29T15:50:13.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.583+0000 7f01bc492700 1 Processor -- start 2022-06-29T15:50:13.597 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.583+0000 7f01bc492700 1 -- start start 2022-06-29T15:50:13.597 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.583+0000 7f01bc492700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f01b4060ee0 0x7f01b4140520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:13.597 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.583+0000 7f01bc492700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01b40612c0 0x7f01b413df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:13.597 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.583+0000 7f01bc492700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01b40b4300 0x7f01b40b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:13.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.583+0000 7f01bc492700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b413f820 con 0x7f01b40b4300 2022-06-29T15:50:13.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.583+0000 7f01bc492700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b41430c0 con 0x7f01b40612c0 2022-06-29T15:50:13.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.583+0000 7f01bc492700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01b40b25f0 con 0x7f01b4060ee0 2022-06-29T15:50:13.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.583+0000 7f01baa2f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01b40b4300 0x7f01b40b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:13.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.583+0000 7f01baa2f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01b40b4300 0x7f01b40b2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52022/0 (socket says 172.21.15.47:52022) 2022-06-29T15:50:13.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.583+0000 7f01baa2f700 1 -- 172.21.15.47:0/1922174900 learned_addr learned my addr 172.21.15.47:0/1922174900 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:13.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.583+0000 7f01ba22e700 1 --2- 172.21.15.47:0/1922174900 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f01b4060ee0 0x7f01b4140520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:13.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.583+0000 7f01b9a2d700 1 --2- 172.21.15.47:0/1922174900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01b40612c0 0x7f01b413df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:13.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.583+0000 7f01baa2f700 1 -- 172.21.15.47:0/1922174900 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f01b4060ee0 msgr2=0x7f01b4140520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:13.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.583+0000 7f01baa2f700 1 --2- 172.21.15.47:0/1922174900 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f01b4060ee0 0x7f01b4140520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:13.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.583+0000 7f01baa2f700 1 -- 172.21.15.47:0/1922174900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01b40612c0 msgr2=0x7f01b413df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:13.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.583+0000 7f01baa2f700 1 --2- 172.21.15.47:0/1922174900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01b40612c0 0x7f01b413df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:13.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.583+0000 7f01baa2f700 1 -- 172.21.15.47:0/1922174900 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ac009470 con 0x7f01b40b4300 2022-06-29T15:50:13.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.583+0000 7f01ba22e700 1 --2- 172.21.15.47:0/1922174900 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f01b4060ee0 0x7f01b4140520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:50:13.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.583+0000 7f01baa2f700 1 --2- 172.21.15.47:0/1922174900 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01b40b4300 0x7f01b40b2e90 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 rx=0x7f01a4001110 tx=0x7f01ac016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:13.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.583+0000 7f01a37fe700 1 -- 172.21.15.47:0/1922174900 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01ac0032d0 con 0x7f01b40b4300 2022-06-29T15:50:13.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.583+0000 7f01a37fe700 1 -- 172.21.15.47:0/1922174900 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01ac011ca0 con 0x7f01b40b4300 2022-06-29T15:50:13.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.583+0000 7f01bc492700 1 -- 172.21.15.47:0/1922174900 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01b413fa50 con 0x7f01b40b4300 2022-06-29T15:50:13.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.583+0000 7f01a37fe700 1 -- 172.21.15.47:0/1922174900 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01ac0203f0 con 0x7f01b40b4300 2022-06-29T15:50:13.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.583+0000 7f01bc492700 1 -- 172.21.15.47:0/1922174900 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01b4140190 con 0x7f01b40b4300 2022-06-29T15:50:13.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.583+0000 7f01bc492700 1 -- 172.21.15.47:0/1922174900 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01b40b23b0 con 0x7f01b40b4300 2022-06-29T15:50:13.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.587+0000 7f01a37fe700 1 -- 172.21.15.47:0/1922174900 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f01ac01e070 con 0x7f01b40b4300 2022-06-29T15:50:13.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.587+0000 7f01a37fe700 1 --2- 172.21.15.47:0/1922174900 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0194053a70 0x7f0194055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:13.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.587+0000 7f01a37fe700 1 -- 172.21.15.47:0/1922174900 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f01ac06caf0 con 0x7f01b40b4300 2022-06-29T15:50:13.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.587+0000 7f01ba22e700 1 --2- 172.21.15.47:0/1922174900 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0194053a70 0x7f0194055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:13.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.587+0000 7f01ba22e700 1 --2- 172.21.15.47:0/1922174900 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0194053a70 0x7f0194055f30 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7f01a400b000 tx=0x7f01a4000bc0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:13.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.591+0000 7f01a37fe700 1 -- 172.21.15.47:0/1922174900 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f01b40b23b0 con 0x7f01b40b4300 2022-06-29T15:50:13.724 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:13.719+0000 7f01bc492700 1 -- 172.21.15.47:0/1922174900 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f01b40b23b0 con 0x7f01b40b4300 2022-06-29T15:50:14.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.327+0000 7f01a37fe700 1 -- 172.21.15.47:0/1922174900 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v108) v1 ==== 133+0+0 (secure 0 0 0) 0x7f01b40b23b0 con 0x7f01b40b4300 2022-06-29T15:50:14.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.331+0000 7f01bc492700 1 -- 172.21.15.47:0/1922174900 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0194053a70 msgr2=0x7f0194055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:14.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.331+0000 7f01bc492700 1 --2- 172.21.15.47:0/1922174900 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0194053a70 0x7f0194055f30 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7f01a400b000 tx=0x7f01a4000bc0).stop 2022-06-29T15:50:14.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.331+0000 7f01bc492700 1 -- 172.21.15.47:0/1922174900 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01b40b4300 msgr2=0x7f01b40b2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:14.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.331+0000 7f01bc492700 1 --2- 172.21.15.47:0/1922174900 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01b40b4300 0x7f01b40b2e90 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 rx=0x7f01a4001110 tx=0x7f01ac016000).stop 2022-06-29T15:50:14.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.331+0000 7f01bc492700 1 -- 172.21.15.47:0/1922174900 shutdown_connections 2022-06-29T15:50:14.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.331+0000 7f01bc492700 1 --2- 172.21.15.47:0/1922174900 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01b40b4300 0x7f01b40b2e90 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:14.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.331+0000 7f01bc492700 1 --2- 172.21.15.47:0/1922174900 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0194053a70 0x7f0194055f30 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:14.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.331+0000 7f01bc492700 1 --2- 172.21.15.47:0/1922174900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01b40612c0 0x7f01b413df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:14.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.331+0000 7f01bc492700 1 --2- 172.21.15.47:0/1922174900 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f01b4060ee0 0x7f01b4140520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:14.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.331+0000 7f01bc492700 1 -- 172.21.15.47:0/1922174900 >> 172.21.15.47:0/1922174900 conn(0x7f01b405c510 msgr2=0x7f01b405f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:14.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.331+0000 7f01bc492700 1 -- 172.21.15.47:0/1922174900 shutdown_connections 2022-06-29T15:50:14.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.331+0000 7f01bc492700 1 -- 172.21.15.47:0/1922174900 wait complete. 2022-06-29T15:50:14.342 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' created 2022-06-29T15:50:14.356 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-06-29T15:50:14.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.467+0000 7f8516baa700 1 Processor -- start 2022-06-29T15:50:14.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.467+0000 7f8516baa700 1 -- start start 2022-06-29T15:50:14.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.467+0000 7f8516baa700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8510098590 0x7f85100b2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:14.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.467+0000 7f8516baa700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f851005c830 0x7f85100b0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:14.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.467+0000 7f8516baa700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8510067160 con 0x7f85100b14d0 2022-06-29T15:50:14.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.467+0000 7f8516baa700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8510098970 con 0x7f851005c830 2022-06-29T15:50:14.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.467+0000 7f8516baa700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8510067390 con 0x7f8510098590 2022-06-29T15:50:14.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.467+0000 7f850ffff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f851005c830 0x7f85100b0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:14.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.467+0000 7f850ffff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f851005c830 0x7f85100b0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52028/0 (socket says 172.21.15.47:52028) 2022-06-29T15:50:14.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.467+0000 7f850ffff700 1 -- 172.21.15.47:0/2798263306 learned_addr learned my addr 172.21.15.47:0/2798263306 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:14.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.467+0000 7f8514946700 1 --2- 172.21.15.47:0/2798263306 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8510098590 0x7f85100b2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:14.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.467+0000 7f850f7fe700 1 -- 172.21.15.47:0/2798263306 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 402058148 0 0) 0x7f8510067160 con 0x7f85100b14d0 2022-06-29T15:50:14.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.471+0000 7f850f7fe700 1 -- 172.21.15.47:0/2798263306 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84f4003260 con 0x7f85100b14d0 2022-06-29T15:50:14.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.471+0000 7f8514946700 1 -- 172.21.15.47:0/2798263306 >> v2:172.21.15.47:3300/0 conn(0x7f851005c830 msgr2=0x7f85100b0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:14.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.471+0000 7f8514946700 1 --2- 172.21.15.47:0/2798263306 >> v2:172.21.15.47:3300/0 conn(0x7f851005c830 0x7f85100b0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:14.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.471+0000 7f8514946700 1 -- 172.21.15.47:0/2798263306 >> v1:172.21.15.64:6789/0 conn(0x7f85100b14d0 legacy=0x7f8510096250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:14.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.471+0000 7f8514946700 1 -- 172.21.15.47:0/2798263306 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8510067630 con 0x7f8510098590 2022-06-29T15:50:14.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.471+0000 7f850ffff700 1 --2- 172.21.15.47:0/2798263306 >> v2:172.21.15.47:3300/0 conn(0x7f851005c830 0x7f85100b0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:50:14.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.471+0000 7f8514946700 1 --2- 172.21.15.47:0/2798263306 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8510098590 0x7f85100b2d60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f8504002f70 tx=0x7f8504006df0).ready entity=mon.2 client_cookie=7c9ead6d389ef23a server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:14.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.471+0000 7f850f7fe700 1 -- 172.21.15.47:0/2798263306 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f850400d4a0 con 0x7f8510098590 2022-06-29T15:50:14.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.471+0000 7f850f7fe700 1 -- 172.21.15.47:0/2798263306 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8504000e90 con 0x7f8510098590 2022-06-29T15:50:14.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.471+0000 7f850f7fe700 1 -- 172.21.15.47:0/2798263306 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f850400d7f0 con 0x7f8510098590 2022-06-29T15:50:14.485 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.471+0000 7f8516baa700 1 -- 172.21.15.47:0/2798263306 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8510098590 msgr2=0x7f85100b2d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:14.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.471+0000 7f8516baa700 1 --2- 172.21.15.47:0/2798263306 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8510098590 0x7f85100b2d60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f8504002f70 tx=0x7f8504006df0).stop 2022-06-29T15:50:14.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.471+0000 7f8516baa700 1 -- 172.21.15.47:0/2798263306 shutdown_connections 2022-06-29T15:50:14.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.471+0000 7f8516baa700 1 --2- 172.21.15.47:0/2798263306 >> v2:172.21.15.47:3300/0 conn(0x7f851005c830 0x7f85100b0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:14.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.471+0000 7f8516baa700 1 --2- 172.21.15.47:0/2798263306 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8510098590 0x7f85100b2d60 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:14.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.471+0000 7f8516baa700 1 -- 172.21.15.47:0/2798263306 >> 172.21.15.47:0/2798263306 conn(0x7f851005f0b0 msgr2=0x7f851005f490 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:14.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.471+0000 7f8516baa700 1 -- 172.21.15.47:0/2798263306 shutdown_connections 2022-06-29T15:50:14.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.471+0000 7f8516baa700 1 -- 172.21.15.47:0/2798263306 wait complete. 2022-06-29T15:50:14.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.475+0000 7f8516baa700 1 Processor -- start 2022-06-29T15:50:14.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.475+0000 7f8516baa700 1 -- start start 2022-06-29T15:50:14.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.475+0000 7f8516baa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f851005c830 0x7f85100af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:14.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.475+0000 7f8516baa700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8510098590 0x7f85100b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:14.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.475+0000 7f8516baa700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f85100b14d0 0x7f85100b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:14.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.475+0000 7f8516baa700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8510146b50 con 0x7f8510098590 2022-06-29T15:50:14.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.475+0000 7f8516baa700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85100b44e0 con 0x7f851005c830 2022-06-29T15:50:14.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.475+0000 7f8516baa700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8510146610 con 0x7f85100b14d0 2022-06-29T15:50:14.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.475+0000 7f850ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8510098590 0x7f85100b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:14.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.475+0000 7f8514946700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f851005c830 0x7f85100af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:14.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.475+0000 7f8515147700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f85100b14d0 0x7f85100b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:14.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.475+0000 7f8514946700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f851005c830 0x7f85100af7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:57736/0 (socket says 172.21.15.47:57736) 2022-06-29T15:50:14.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.475+0000 7f8515147700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f85100b14d0 0x7f85100b3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:49138/0 (socket says 172.21.15.47:49138) 2022-06-29T15:50:14.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.475+0000 7f8514946700 1 -- 172.21.15.47:0/3924318041 learned_addr learned my addr 172.21.15.47:0/3924318041 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:14.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.475+0000 7f850ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8510098590 0x7f85100b3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52034/0 (socket says 172.21.15.47:52034) 2022-06-29T15:50:14.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.475+0000 7f850ffff700 1 -- 172.21.15.47:0/3924318041 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f85100b14d0 msgr2=0x7f85100b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:14.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.475+0000 7f850ffff700 1 --2- 172.21.15.47:0/3924318041 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f85100b14d0 0x7f85100b3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:14.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.475+0000 7f850ffff700 1 -- 172.21.15.47:0/3924318041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f851005c830 msgr2=0x7f85100af7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:14.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.475+0000 7f850ffff700 1 --2- 172.21.15.47:0/3924318041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f851005c830 0x7f85100af7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:14.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.475+0000 7f850ffff700 1 -- 172.21.15.47:0/3924318041 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84fc00b1e0 con 0x7f8510098590 2022-06-29T15:50:14.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.475+0000 7f8515147700 1 --2- 172.21.15.47:0/3924318041 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f85100b14d0 0x7f85100b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:50:14.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.479+0000 7f850ffff700 1 --2- 172.21.15.47:0/3924318041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8510098590 0x7f85100b3b00 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 rx=0x7f84fc00c000 tx=0x7f84fc006eb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:14.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.479+0000 7f850dffb700 1 -- 172.21.15.47:0/3924318041 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f84fc012600 con 0x7f8510098590 2022-06-29T15:50:14.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.479+0000 7f850dffb700 1 -- 172.21.15.47:0/3924318041 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84fc012760 con 0x7f8510098590 2022-06-29T15:50:14.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.479+0000 7f8516baa700 1 -- 172.21.15.47:0/3924318041 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f850400b180 con 0x7f8510098590 2022-06-29T15:50:14.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.479+0000 7f850dffb700 1 -- 172.21.15.47:0/3924318041 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f84fc012ac0 con 0x7f8510098590 2022-06-29T15:50:14.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.479+0000 7f8516baa700 1 -- 172.21.15.47:0/3924318041 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8510147120 con 0x7f8510098590 2022-06-29T15:50:14.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.479+0000 7f8516baa700 1 -- 172.21.15.47:0/3924318041 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85101472e0 con 0x7f8510098590 2022-06-29T15:50:14.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.479+0000 7f850dffb700 1 -- 172.21.15.47:0/3924318041 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f84fc022070 con 0x7f8510098590 2022-06-29T15:50:14.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.479+0000 7f850dffb700 1 --2- 172.21.15.47:0/3924318041 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f84f0053aa0 0x7f84f0055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:14.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.479+0000 7f850dffb700 1 -- 172.21.15.47:0/3924318041 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5954+0+0 (secure 0 0 0) 0x7f84fc06c250 con 0x7f8510098590 2022-06-29T15:50:14.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.483+0000 7f8514946700 1 --2- 172.21.15.47:0/3924318041 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f84f0053aa0 0x7f84f0055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:14.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.483+0000 7f8514946700 1 --2- 172.21.15.47:0/3924318041 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f84f0053aa0 0x7f84f0055f60 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7f850401a000 tx=0x7f8504014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:14.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.487+0000 7f850dffb700 1 -- 172.21.15.47:0/3924318041 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f84fc0430e0 con 0x7f8510098590 2022-06-29T15:50:14.626 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:14.619+0000 7f8516baa700 1 -- 172.21.15.47:0/3924318041 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f85100573d0 con 0x7f8510098590 2022-06-29T15:50:15.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.007+0000 7f850dffb700 1 -- 172.21.15.47:0/3924318041 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v109) v1 ==== 125+0+0 (secure 0 0 0) 0x7f84fc046700 con 0x7f8510098590 2022-06-29T15:50:15.025 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.019+0000 7f8516baa700 1 -- 172.21.15.47:0/3924318041 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f84f0053aa0 msgr2=0x7f84f0055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:15.025 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.019+0000 7f8516baa700 1 --2- 172.21.15.47:0/3924318041 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f84f0053aa0 0x7f84f0055f60 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7f850401a000 tx=0x7f8504014000).stop 2022-06-29T15:50:15.026 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.019+0000 7f8516baa700 1 -- 172.21.15.47:0/3924318041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8510098590 msgr2=0x7f85100b3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:15.026 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.019+0000 7f8516baa700 1 --2- 172.21.15.47:0/3924318041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8510098590 0x7f85100b3b00 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 rx=0x7f84fc00c000 tx=0x7f84fc006eb0).stop 2022-06-29T15:50:15.026 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.019+0000 7f8516baa700 1 -- 172.21.15.47:0/3924318041 shutdown_connections 2022-06-29T15:50:15.027 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.019+0000 7f8516baa700 1 --2- 172.21.15.47:0/3924318041 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f85100b14d0 0x7f85100b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:15.027 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.019+0000 7f8516baa700 1 --2- 172.21.15.47:0/3924318041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8510098590 0x7f85100b3b00 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:15.027 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.019+0000 7f8516baa700 1 --2- 172.21.15.47:0/3924318041 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f84f0053aa0 0x7f84f0055f60 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:15.028 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.019+0000 7f8516baa700 1 --2- 172.21.15.47:0/3924318041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f851005c830 0x7f85100af7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:15.028 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.019+0000 7f8516baa700 1 -- 172.21.15.47:0/3924318041 >> 172.21.15.47:0/3924318041 conn(0x7f851005f0b0 msgr2=0x7f8510098ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:15.028 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.019+0000 7f8516baa700 1 -- 172.21.15.47:0/3924318041 shutdown_connections 2022-06-29T15:50:15.029 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.019+0000 7f8516baa700 1 -- 172.21.15.47:0/3924318041 wait complete. 2022-06-29T15:50:15.029 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' created 2022-06-29T15:50:15.036 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-06-29T15:50:15.144 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.139+0000 7f7d427f5700 1 Processor -- start 2022-06-29T15:50:15.144 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.139+0000 7f7d427f5700 1 -- start start 2022-06-29T15:50:15.152 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.139+0000 7f7d427f5700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7d3c05f0b0 0x7f7d3c13e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:15.153 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.139+0000 7f7d427f5700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d3c0b71c0 con 0x7f7d3c0b6de0 2022-06-29T15:50:15.153 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.139+0000 7f7d427f5700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d3c145070 con 0x7f7d3c05f5c0 2022-06-29T15:50:15.153 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.139+0000 7f7d427f5700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d3c0aa030 con 0x7f7d3c05f0b0 2022-06-29T15:50:15.154 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.139+0000 7f7d417f3700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7d3c0b6de0 0x7f7d3c13e2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37594/0 (socket says 172.21.15.47:37594) 2022-06-29T15:50:15.154 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.139+0000 7f7d417f3700 1 -- 172.21.15.47:0/649388981 learned_addr learned my addr 172.21.15.47:0/649388981 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:15.155 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.139+0000 7f7d41ff4700 1 --2- 172.21.15.47:0/649388981 >> v2:172.21.15.64:3300/0 conn(0x7f7d3c05f0b0 0x7f7d3c13e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:15.155 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.139+0000 7f7d33fff700 1 -- 172.21.15.47:0/649388981 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1683850427 0 0) 0x7f7d3c0b71c0 con 0x7f7d3c0b6de0 2022-06-29T15:50:15.155 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.139+0000 7f7d33fff700 1 -- 172.21.15.47:0/649388981 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d20003260 con 0x7f7d3c0b6de0 2022-06-29T15:50:15.156 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.139+0000 7f7d33fff700 1 -- 172.21.15.47:0/649388981 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3313612486 0 0) 0x7f7d20003260 con 0x7f7d3c0b6de0 2022-06-29T15:50:15.156 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.139+0000 7f7d33fff700 1 -- 172.21.15.47:0/649388981 >> v2:172.21.15.64:3300/0 conn(0x7f7d3c05f0b0 msgr2=0x7f7d3c13e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:15.156 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.139+0000 7f7d33fff700 1 --2- 172.21.15.47:0/649388981 >> v2:172.21.15.64:3300/0 conn(0x7f7d3c05f0b0 0x7f7d3c13e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:15.156 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.139+0000 7f7d33fff700 1 -- 172.21.15.47:0/649388981 >> v1:172.21.15.64:6789/0 conn(0x7f7d3c05f5c0 legacy=0x7f7d3c1411c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:15.157 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.139+0000 7f7d33fff700 1 -- 172.21.15.47:0/649388981 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d3c0a9d90 con 0x7f7d3c0b6de0 2022-06-29T15:50:15.157 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.139+0000 7f7d33fff700 1 -- 172.21.15.47:0/649388981 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7f7d38001680 con 0x7f7d3c0b6de0 2022-06-29T15:50:15.157 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.139+0000 7f7d33fff700 1 -- 172.21.15.47:0/649388981 >> v1:172.21.15.47:6789/0 conn(0x7f7d3c0b6de0 legacy=0x7f7d3c13e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:15.157 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.139+0000 7f7d33fff700 1 --2- 172.21.15.47:0/649388981 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7d20003750 0x7f7d20003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:15.158 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.139+0000 7f7d33fff700 1 --2- 172.21.15.47:0/649388981 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d20004500 0x7f7d20004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:15.158 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.139+0000 7f7d33fff700 1 --2- 172.21.15.47:0/649388981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d20004e70 0x7f7d2000fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:15.158 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.139+0000 7f7d33fff700 1 -- 172.21.15.47:0/649388981 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3c0aa030 con 0x7f7d20004e70 2022-06-29T15:50:15.159 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.139+0000 7f7d33fff700 1 -- 172.21.15.47:0/649388981 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d20010130 con 0x7f7d20004500 2022-06-29T15:50:15.159 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.139+0000 7f7d33fff700 1 -- 172.21.15.47:0/649388981 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d200102f0 con 0x7f7d20003750 2022-06-29T15:50:15.159 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.139+0000 7f7d417f3700 1 --2- 172.21.15.47:0/649388981 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7d20003750 0x7f7d20003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:15.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.139+0000 7f7d41ff4700 1 --2- 172.21.15.47:0/649388981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d20004e70 0x7f7d2000fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:15.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.139+0000 7f7d40ff2700 1 --2- 172.21.15.47:0/649388981 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d20004500 0x7f7d20004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:15.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.139+0000 7f7d40ff2700 1 -- 172.21.15.47:0/649388981 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7d20003750 msgr2=0x7f7d20003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:15.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.139+0000 7f7d40ff2700 1 --2- 172.21.15.47:0/649388981 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7d20003750 0x7f7d20003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:15.161 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.139+0000 7f7d40ff2700 1 -- 172.21.15.47:0/649388981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d20004e70 msgr2=0x7f7d2000fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:15.161 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.139+0000 7f7d40ff2700 1 --2- 172.21.15.47:0/649388981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d20004e70 0x7f7d2000fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:15.161 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.139+0000 7f7d40ff2700 1 -- 172.21.15.47:0/649388981 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d20003260 con 0x7f7d20004500 2022-06-29T15:50:15.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.139+0000 7f7d417f3700 1 --2- 172.21.15.47:0/649388981 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7d20003750 0x7f7d20003b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:50:15.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.139+0000 7f7d41ff4700 1 --2- 172.21.15.47:0/649388981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d20004e70 0x7f7d2000fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:50:15.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.139+0000 7f7d41ff4700 1 -- 172.21.15.47:0/649388981 reap_dead start 2022-06-29T15:50:15.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.139+0000 7f7d40ff2700 1 --2- 172.21.15.47:0/649388981 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d20004500 0x7f7d20004980 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f7d3c0b1c00 tx=0x7f7d2c007a00).ready entity=mon.1 client_cookie=8049ed7fe9e9347d server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:15.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.143+0000 7f7d33fff700 1 -- 172.21.15.47:0/649388981 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d2c007d10 con 0x7f7d20004500 2022-06-29T15:50:15.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.143+0000 7f7d33fff700 1 -- 172.21.15.47:0/649388981 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d2c007030 con 0x7f7d20004500 2022-06-29T15:50:15.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.143+0000 7f7d33fff700 1 -- 172.21.15.47:0/649388981 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d2c007390 con 0x7f7d20004500 2022-06-29T15:50:15.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.143+0000 7f7d427f5700 1 -- 172.21.15.47:0/649388981 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d20004500 msgr2=0x7f7d20004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:15.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.143+0000 7f7d427f5700 1 --2- 172.21.15.47:0/649388981 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d20004500 0x7f7d20004980 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f7d3c0b1c00 tx=0x7f7d2c007a00).stop 2022-06-29T15:50:15.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.143+0000 7f7d427f5700 1 -- 172.21.15.47:0/649388981 shutdown_connections 2022-06-29T15:50:15.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.143+0000 7f7d427f5700 1 --2- 172.21.15.47:0/649388981 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d20004500 0x7f7d20004980 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:15.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.143+0000 7f7d427f5700 1 -- 172.21.15.47:0/649388981 >> 172.21.15.47:0/649388981 conn(0x7f7d3c063670 msgr2=0x7f7d3c063a50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:15.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.143+0000 7f7d427f5700 1 -- 172.21.15.47:0/649388981 shutdown_connections 2022-06-29T15:50:15.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.143+0000 7f7d427f5700 1 -- 172.21.15.47:0/649388981 wait complete. 2022-06-29T15:50:15.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.143+0000 7f7d427f5700 1 Processor -- start 2022-06-29T15:50:15.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.143+0000 7f7d427f5700 1 -- start start 2022-06-29T15:50:15.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.143+0000 7f7d427f5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d20004500 0x7f7d3c14e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:15.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.143+0000 7f7d427f5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d3c14e4f0 0x7f7d3c142830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:15.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.143+0000 7f7d427f5700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7d3c142d20 0x7f7d3c0aaff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:15.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.143+0000 7f7d427f5700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3c143e70 con 0x7f7d3c14e4f0 2022-06-29T15:50:15.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.143+0000 7f7d427f5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3c13b030 con 0x7f7d20004500 2022-06-29T15:50:15.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.143+0000 7f7d427f5700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3c14e970 con 0x7f7d3c142d20 2022-06-29T15:50:15.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.143+0000 7f7d40ff2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d3c14e4f0 0x7f7d3c142830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:15.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.143+0000 7f7d40ff2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d3c14e4f0 0x7f7d3c142830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52052/0 (socket says 172.21.15.47:52052) 2022-06-29T15:50:15.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.143+0000 7f7d40ff2700 1 -- 172.21.15.47:0/1890873126 learned_addr learned my addr 172.21.15.47:0/1890873126 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:15.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.143+0000 7f7d417f3700 1 --2- 172.21.15.47:0/1890873126 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d20004500 0x7f7d3c14e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:15.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.143+0000 7f7d41ff4700 1 --2- 172.21.15.47:0/1890873126 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7d3c142d20 0x7f7d3c0aaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:15.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.143+0000 7f7d40ff2700 1 -- 172.21.15.47:0/1890873126 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7d3c142d20 msgr2=0x7f7d3c0aaff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:15.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.143+0000 7f7d40ff2700 1 --2- 172.21.15.47:0/1890873126 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7d3c142d20 0x7f7d3c0aaff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:15.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.143+0000 7f7d40ff2700 1 -- 172.21.15.47:0/1890873126 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d20004500 msgr2=0x7f7d3c14e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:15.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.143+0000 7f7d40ff2700 1 --2- 172.21.15.47:0/1890873126 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d20004500 0x7f7d3c14e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:15.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.143+0000 7f7d40ff2700 1 -- 172.21.15.47:0/1890873126 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d3c143bd0 con 0x7f7d3c14e4f0 2022-06-29T15:50:15.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.147+0000 7f7d40ff2700 1 --2- 172.21.15.47:0/1890873126 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d3c14e4f0 0x7f7d3c142830 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 rx=0x7f7d3c14ecb0 tx=0x7f7d2c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:15.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.147+0000 7f7d327fc700 1 -- 172.21.15.47:0/1890873126 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d2c020450 con 0x7f7d3c14e4f0 2022-06-29T15:50:15.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.147+0000 7f7d327fc700 1 -- 172.21.15.47:0/1890873126 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d2c0205b0 con 0x7f7d3c14e4f0 2022-06-29T15:50:15.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.147+0000 7f7d427f5700 1 -- 172.21.15.47:0/1890873126 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d3c13b220 con 0x7f7d3c14e4f0 2022-06-29T15:50:15.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.147+0000 7f7d327fc700 1 -- 172.21.15.47:0/1890873126 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d2c020910 con 0x7f7d3c14e4f0 2022-06-29T15:50:15.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.147+0000 7f7d427f5700 1 -- 172.21.15.47:0/1890873126 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d3c14edd0 con 0x7f7d3c14e4f0 2022-06-29T15:50:15.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.147+0000 7f7d427f5700 1 -- 172.21.15.47:0/1890873126 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d3c13add0 con 0x7f7d3c14e4f0 2022-06-29T15:50:15.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.147+0000 7f7d327fc700 1 -- 172.21.15.47:0/1890873126 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7d2c01d070 con 0x7f7d3c14e4f0 2022-06-29T15:50:15.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.151+0000 7f7d327fc700 1 --2- 172.21.15.47:0/1890873126 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7d24053b00 0x7f7d24055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:15.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.151+0000 7f7d327fc700 1 -- 172.21.15.47:0/1890873126 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f7d2c06cff0 con 0x7f7d3c14e4f0 2022-06-29T15:50:15.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.151+0000 7f7d417f3700 1 --2- 172.21.15.47:0/1890873126 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7d24053b00 0x7f7d24055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:15.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.151+0000 7f7d417f3700 1 --2- 172.21.15.47:0/1890873126 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7d24053b00 0x7f7d24055fc0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7f7d38005000 tx=0x7f7d3c0a9f20).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:15.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.155+0000 7f7d327fc700 1 -- 172.21.15.47:0/1890873126 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7d2c043cc0 con 0x7f7d3c14e4f0 2022-06-29T15:50:15.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.323+0000 7f7d427f5700 1 -- 172.21.15.47:0/1890873126 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f7d3c0573d0 con 0x7f7d3c14e4f0 2022-06-29T15:50:15.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.343+0000 7f7d327fc700 1 -- 172.21.15.47:0/1890873126 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 19 and data pool 20 v50) v1 ==== 189+0+0 (secure 0 0 0) 0x7f7d2c0472e0 con 0x7f7d3c14e4f0 2022-06-29T15:50:15.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.347+0000 7f7d427f5700 1 -- 172.21.15.47:0/1890873126 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7d24053b00 msgr2=0x7f7d24055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:15.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.347+0000 7f7d427f5700 1 --2- 172.21.15.47:0/1890873126 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7d24053b00 0x7f7d24055fc0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7f7d38005000 tx=0x7f7d3c0a9f20).stop 2022-06-29T15:50:15.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.347+0000 7f7d427f5700 1 -- 172.21.15.47:0/1890873126 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d3c14e4f0 msgr2=0x7f7d3c142830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:15.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.347+0000 7f7d427f5700 1 --2- 172.21.15.47:0/1890873126 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d3c14e4f0 0x7f7d3c142830 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 rx=0x7f7d3c14ecb0 tx=0x7f7d2c015000).stop 2022-06-29T15:50:15.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.351+0000 7f7d427f5700 1 -- 172.21.15.47:0/1890873126 shutdown_connections 2022-06-29T15:50:15.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.351+0000 7f7d427f5700 1 --2- 172.21.15.47:0/1890873126 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7d3c142d20 0x7f7d3c0aaff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:15.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.351+0000 7f7d427f5700 1 --2- 172.21.15.47:0/1890873126 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d3c14e4f0 0x7f7d3c142830 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:15.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.351+0000 7f7d427f5700 1 --2- 172.21.15.47:0/1890873126 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7d24053b00 0x7f7d24055fc0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:15.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.351+0000 7f7d427f5700 1 --2- 172.21.15.47:0/1890873126 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d20004500 0x7f7d3c14e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:15.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.351+0000 7f7d427f5700 1 -- 172.21.15.47:0/1890873126 >> 172.21.15.47:0/1890873126 conn(0x7f7d3c063670 msgr2=0x7f7d3c13e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:15.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.351+0000 7f7d427f5700 1 -- 172.21.15.47:0/1890873126 shutdown_connections 2022-06-29T15:50:15.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.351+0000 7f7d427f5700 1 -- 172.21.15.47:0/1890873126 wait complete. 2022-06-29T15:50:15.360 INFO:teuthology.orchestra.run.smithi047.stderr:new fs with metadata pool 19 and data pool 20 2022-06-29T15:50:15.372 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:50:15.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.463+0000 7fa0e7589700 1 Processor -- start 2022-06-29T15:50:15.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.463+0000 7fa0e7589700 1 -- start start 2022-06-29T15:50:15.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.463+0000 7fa0e7589700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa0e00b5c60 0x7fa0e0144980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:15.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.463+0000 7fa0e7589700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa0e0060eb0 0x7fa0e0144e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:15.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.463+0000 7fa0e7589700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0e00a9120 con 0x7fa0e00a72f0 2022-06-29T15:50:15.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.463+0000 7fa0e7589700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0e0145360 con 0x7fa0e0060eb0 2022-06-29T15:50:15.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.463+0000 7fa0e7589700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e00b0b80 con 0x7fa0e00b5c60 2022-06-29T15:50:15.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.463+0000 7fa0e5d86700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa0e0060eb0 0x7fa0e0144e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:15.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.463+0000 7fa0e5d86700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa0e0060eb0 0x7fa0e0144e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52062/0 (socket says 172.21.15.47:52062) 2022-06-29T15:50:15.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.463+0000 7fa0e5d86700 1 -- 172.21.15.47:0/2450250638 learned_addr learned my addr 172.21.15.47:0/2450250638 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:15.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.463+0000 7fa0e6587700 1 --2- 172.21.15.47:0/2450250638 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa0e00b5c60 0x7fa0e0144980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:15.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.463+0000 7fa0e5d86700 1 -- 172.21.15.47:0/2450250638 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa0e00b5c60 msgr2=0x7fa0e0144980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:15.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.463+0000 7fa0e5d86700 1 --2- 172.21.15.47:0/2450250638 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa0e00b5c60 0x7fa0e0144980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:15.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.463+0000 7fa0e5d86700 1 -- 172.21.15.47:0/2450250638 >> v1:172.21.15.47:6789/0 conn(0x7fa0e00a72f0 legacy=0x7fa0e00b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:15.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.463+0000 7fa0e5585700 1 -- 172.21.15.47:0/2450250638 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 302910150 0 0) 0x7fa0e00a9120 con 0x7fa0e00a72f0 2022-06-29T15:50:15.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.463+0000 7fa0e5d86700 1 -- 172.21.15.47:0/2450250638 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e00a89e0 con 0x7fa0e0060eb0 2022-06-29T15:50:15.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.463+0000 7fa0e5d86700 1 --2- 172.21.15.47:0/2450250638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0e0060eb0 0x7fa0e0144e70 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 rx=0x7fa0d80032f0 tx=0x7fa0d8009290).ready entity=mon.0 client_cookie=cd5c5c439fec9926 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:15.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.463+0000 7fa0e5585700 1 -- 172.21.15.47:0/2450250638 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0d800b8a0 con 0x7fa0e0060eb0 2022-06-29T15:50:15.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.463+0000 7fa0e5585700 1 -- 172.21.15.47:0/2450250638 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0d800ba00 con 0x7fa0e0060eb0 2022-06-29T15:50:15.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.463+0000 7fa0e5585700 1 -- 172.21.15.47:0/2450250638 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0d800bb70 con 0x7fa0e0060eb0 2022-06-29T15:50:15.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.463+0000 7fa0e7589700 1 -- 172.21.15.47:0/2450250638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0e0060eb0 msgr2=0x7fa0e0144e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:15.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.463+0000 7fa0e7589700 1 --2- 172.21.15.47:0/2450250638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0e0060eb0 0x7fa0e0144e70 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 rx=0x7fa0d80032f0 tx=0x7fa0d8009290).stop 2022-06-29T15:50:15.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.463+0000 7fa0e7589700 1 -- 172.21.15.47:0/2450250638 shutdown_connections 2022-06-29T15:50:15.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.463+0000 7fa0e7589700 1 --2- 172.21.15.47:0/2450250638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0e0060eb0 0x7fa0e0144e70 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:15.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.463+0000 7fa0e7589700 1 --2- 172.21.15.47:0/2450250638 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa0e00b5c60 0x7fa0e0144980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:15.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.463+0000 7fa0e7589700 1 -- 172.21.15.47:0/2450250638 >> 172.21.15.47:0/2450250638 conn(0x7fa0e0062560 msgr2=0x7fa0e0062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:15.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.463+0000 7fa0e7589700 1 -- 172.21.15.47:0/2450250638 shutdown_connections 2022-06-29T15:50:15.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.463+0000 7fa0e7589700 1 -- 172.21.15.47:0/2450250638 wait complete. 2022-06-29T15:50:15.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.467+0000 7fa0e7589700 1 Processor -- start 2022-06-29T15:50:15.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.467+0000 7fa0e7589700 1 -- start start 2022-06-29T15:50:15.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.467+0000 7fa0e7589700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0e0060eb0 0x7fa0e00b45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:15.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.467+0000 7fa0e7589700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0e00a72f0 0x7fa0e014a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:15.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.467+0000 7fa0e7589700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa0e00b5c60 0x7fa0e014a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:15.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.467+0000 7fa0e7589700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e014b800 con 0x7fa0e00a72f0 2022-06-29T15:50:15.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.467+0000 7fa0e7589700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e0142600 con 0x7fa0e0060eb0 2022-06-29T15:50:15.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.467+0000 7fa0e7589700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e00ad330 con 0x7fa0e00b5c60 2022-06-29T15:50:15.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.467+0000 7fa0e5d86700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0e00a72f0 0x7fa0e014a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:15.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.467+0000 7fa0e6d88700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa0e00b5c60 0x7fa0e014a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:15.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.467+0000 7fa0e5d86700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0e00a72f0 0x7fa0e014a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52072/0 (socket says 172.21.15.47:52072) 2022-06-29T15:50:15.484 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.467+0000 7fa0e5d86700 1 -- 172.21.15.47:0/490748345 learned_addr learned my addr 172.21.15.47:0/490748345 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:15.484 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.467+0000 7fa0e6587700 1 --2- 172.21.15.47:0/490748345 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0e0060eb0 0x7fa0e00b45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:15.484 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.467+0000 7fa0e5d86700 1 -- 172.21.15.47:0/490748345 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa0e00b5c60 msgr2=0x7fa0e014a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:15.484 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.467+0000 7fa0e5d86700 1 --2- 172.21.15.47:0/490748345 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa0e00b5c60 0x7fa0e014a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:15.485 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.467+0000 7fa0e5d86700 1 -- 172.21.15.47:0/490748345 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0e0060eb0 msgr2=0x7fa0e00b45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:15.485 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.467+0000 7fa0e5d86700 1 --2- 172.21.15.47:0/490748345 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0e0060eb0 0x7fa0e00b45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:15.485 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.467+0000 7fa0e5d86700 1 -- 172.21.15.47:0/490748345 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0d8009530 con 0x7fa0e00a72f0 2022-06-29T15:50:15.485 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.467+0000 7fa0e6d88700 1 --2- 172.21.15.47:0/490748345 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa0e00b5c60 0x7fa0e014a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:50:15.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.467+0000 7fa0e5d86700 1 --2- 172.21.15.47:0/490748345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0e00a72f0 0x7fa0e014a160 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 rx=0x7fa0d80032f0 tx=0x7fa0d8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:15.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.467+0000 7fa0d37fe700 1 -- 172.21.15.47:0/490748345 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0d8021630 con 0x7fa0e00a72f0 2022-06-29T15:50:15.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.467+0000 7fa0d37fe700 1 -- 172.21.15.47:0/490748345 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0d8021790 con 0x7fa0e00a72f0 2022-06-29T15:50:15.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.467+0000 7fa0e7589700 1 -- 172.21.15.47:0/490748345 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0e00ae380 con 0x7fa0e00a72f0 2022-06-29T15:50:15.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.467+0000 7fa0d37fe700 1 -- 172.21.15.47:0/490748345 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0d8021af0 con 0x7fa0e00a72f0 2022-06-29T15:50:15.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.467+0000 7fa0e7589700 1 -- 172.21.15.47:0/490748345 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0e00ad590 con 0x7fa0e00a72f0 2022-06-29T15:50:15.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.471+0000 7fa0e7589700 1 -- 172.21.15.47:0/490748345 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0e00573d0 con 0x7fa0e00a72f0 2022-06-29T15:50:15.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.471+0000 7fa0d37fe700 1 -- 172.21.15.47:0/490748345 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa0d8027020 con 0x7fa0e00a72f0 2022-06-29T15:50:15.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.471+0000 7fa0d37fe700 1 --2- 172.21.15.47:0/490748345 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa0c00539e0 0x7fa0c0055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:15.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.471+0000 7fa0d37fe700 1 -- 172.21.15.47:0/490748345 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa0d806d840 con 0x7fa0e00a72f0 2022-06-29T15:50:15.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.471+0000 7fa0e6587700 1 --2- 172.21.15.47:0/490748345 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa0c00539e0 0x7fa0c0055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:15.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.471+0000 7fa0e6587700 1 --2- 172.21.15.47:0/490748345 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa0c00539e0 0x7fa0c0055ea0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7fa0d400c000 tx=0x7fa0d4008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:15.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.479+0000 7fa0d37fe700 1 -- 172.21.15.47:0/490748345 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa0d8044600 con 0x7fa0e00a72f0 2022-06-29T15:50:15.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.607+0000 7fa0e7589700 1 -- 172.21.15.47:0/490748345 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa0e0069e20 con 0x7fa0e00a72f0 2022-06-29T15:50:15.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.611+0000 7fa0d37fe700 1 -- 172.21.15.47:0/490748345 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+16298 (secure 0 0 0) 0x7fa0d8015020 con 0x7fa0e00a72f0 2022-06-29T15:50:15.616 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:50:15.616 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":110,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:50:15.337931+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:50:13.772527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2022-06-29T15:50:14.675947+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:50:15.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.611+0000 7fa0e7589700 1 -- 172.21.15.47:0/490748345 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa0c00539e0 msgr2=0x7fa0c0055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:15.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.611+0000 7fa0e7589700 1 --2- 172.21.15.47:0/490748345 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa0c00539e0 0x7fa0c0055ea0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7fa0d400c000 tx=0x7fa0d4008040).stop 2022-06-29T15:50:15.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.611+0000 7fa0e7589700 1 -- 172.21.15.47:0/490748345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0e00a72f0 msgr2=0x7fa0e014a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:15.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.611+0000 7fa0e7589700 1 --2- 172.21.15.47:0/490748345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0e00a72f0 0x7fa0e014a160 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 rx=0x7fa0d80032f0 tx=0x7fa0d8016000).stop 2022-06-29T15:50:15.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.611+0000 7fa0e7589700 1 -- 172.21.15.47:0/490748345 shutdown_connections 2022-06-29T15:50:15.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.611+0000 7fa0e7589700 1 --2- 172.21.15.47:0/490748345 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa0e00b5c60 0x7fa0e014a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:15.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.611+0000 7fa0e7589700 1 --2- 172.21.15.47:0/490748345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0e00a72f0 0x7fa0e014a160 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:15.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.611+0000 7fa0e7589700 1 --2- 172.21.15.47:0/490748345 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa0c00539e0 0x7fa0c0055ea0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:15.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.611+0000 7fa0e7589700 1 --2- 172.21.15.47:0/490748345 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0e0060eb0 0x7fa0e00b45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:15.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.611+0000 7fa0e7589700 1 -- 172.21.15.47:0/490748345 >> 172.21.15.47:0/490748345 conn(0x7fa0e0062560 msgr2=0x7fa0e00afae0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:15.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.611+0000 7fa0e7589700 1 -- 172.21.15.47:0/490748345 shutdown_connections 2022-06-29T15:50:15.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.611+0000 7fa0e7589700 1 -- 172.21.15.47:0/490748345 wait complete. 2022-06-29T15:50:15.630 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:50:15.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.731+0000 7f7afa12f700 1 Processor -- start 2022-06-29T15:50:15.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.731+0000 7f7afa12f700 1 -- start start 2022-06-29T15:50:15.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.731+0000 7f7afa12f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7af40611b0 0x7f7af4146640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:15.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.731+0000 7f7afa12f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7af40b0cf0 0x7f7af4146b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:15.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.731+0000 7f7afa12f700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7af40b06d0 0x7f7af40667e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:15.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.731+0000 7f7afa12f700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7af40b4b10 con 0x7f7af40b0cf0 2022-06-29T15:50:15.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.731+0000 7f7afa12f700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7af40621d0 con 0x7f7af40611b0 2022-06-29T15:50:15.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.731+0000 7f7afa12f700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7af40b4520 con 0x7f7af40b06d0 2022-06-29T15:50:15.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.731+0000 7f7af2ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7af40b0cf0 0x7f7af4146b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:15.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.731+0000 7f7af3fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7af40b06d0 0x7f7af40667e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:15.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.731+0000 7f7af37fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7af40611b0 0x7f7af4146640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:15.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.731+0000 7f7af2ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7af40b0cf0 0x7f7af4146b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52078/0 (socket says 172.21.15.47:52078) 2022-06-29T15:50:15.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.731+0000 7f7af2ffd700 1 -- 172.21.15.47:0/3804022644 learned_addr learned my addr 172.21.15.47:0/3804022644 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:15.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.731+0000 7f7af3fff700 1 -- 172.21.15.47:0/3804022644 >> v2:172.21.15.64:3300/0 conn(0x7f7af40611b0 msgr2=0x7f7af4146640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:15.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.731+0000 7f7af3fff700 1 --2- 172.21.15.47:0/3804022644 >> v2:172.21.15.64:3300/0 conn(0x7f7af40611b0 0x7f7af4146640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:15.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.731+0000 7f7af3fff700 1 -- 172.21.15.47:0/3804022644 >> v2:172.21.15.47:3300/0 conn(0x7f7af40b0cf0 msgr2=0x7f7af4146b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:15.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.731+0000 7f7af3fff700 1 --2- 172.21.15.47:0/3804022644 >> v2:172.21.15.47:3300/0 conn(0x7f7af40b0cf0 0x7f7af4146b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:15.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.731+0000 7f7af3fff700 1 -- 172.21.15.47:0/3804022644 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af4061590 con 0x7f7af40b06d0 2022-06-29T15:50:15.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.731+0000 7f7af3fff700 1 --2- 172.21.15.47:0/3804022644 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7af40b06d0 0x7f7af40667e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f7af40b4790 tx=0x7f7aec0091d0).ready entity=mon.2 client_cookie=4e2a5d57c6041891 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:15.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.731+0000 7f7af27fc700 1 -- 172.21.15.47:0/3804022644 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7aec00b5a0 con 0x7f7af40b06d0 2022-06-29T15:50:15.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.731+0000 7f7af27fc700 1 -- 172.21.15.47:0/3804022644 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aec00b700 con 0x7f7af40b06d0 2022-06-29T15:50:15.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.731+0000 7f7af27fc700 1 -- 172.21.15.47:0/3804022644 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7aec00b870 con 0x7f7af40b06d0 2022-06-29T15:50:15.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.735+0000 7f7afa12f700 1 -- 172.21.15.47:0/3804022644 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7af40b06d0 msgr2=0x7f7af40667e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:15.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.735+0000 7f7afa12f700 1 --2- 172.21.15.47:0/3804022644 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7af40b06d0 0x7f7af40667e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f7af40b4790 tx=0x7f7aec0091d0).stop 2022-06-29T15:50:15.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.735+0000 7f7afa12f700 1 -- 172.21.15.47:0/3804022644 shutdown_connections 2022-06-29T15:50:15.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.735+0000 7f7afa12f700 1 --2- 172.21.15.47:0/3804022644 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7af40b06d0 0x7f7af40667e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:15.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.735+0000 7f7afa12f700 1 --2- 172.21.15.47:0/3804022644 >> v2:172.21.15.47:3300/0 conn(0x7f7af40b0cf0 0x7f7af4146b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:15.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.735+0000 7f7afa12f700 1 --2- 172.21.15.47:0/3804022644 >> v2:172.21.15.64:3300/0 conn(0x7f7af40611b0 0x7f7af4146640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:15.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.735+0000 7f7afa12f700 1 -- 172.21.15.47:0/3804022644 >> 172.21.15.47:0/3804022644 conn(0x7f7af40b98b0 msgr2=0x7f7af40b9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:15.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.735+0000 7f7afa12f700 1 -- 172.21.15.47:0/3804022644 shutdown_connections 2022-06-29T15:50:15.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.735+0000 7f7afa12f700 1 -- 172.21.15.47:0/3804022644 wait complete. 2022-06-29T15:50:15.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.735+0000 7f7afa12f700 1 Processor -- start 2022-06-29T15:50:15.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.735+0000 7f7afa12f700 1 -- start start 2022-06-29T15:50:15.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.735+0000 7f7afa12f700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7af40611b0 0x7f7af4148d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:15.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.735+0000 7f7afa12f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7af40b06d0 0x7f7af4155c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:15.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.735+0000 7f7afa12f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7af40b0cf0 0x7f7af4156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:15.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.735+0000 7f7afa12f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af40b8180 con 0x7f7af40b0cf0 2022-06-29T15:50:15.752 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.735+0000 7f7afa12f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af40b8960 con 0x7f7af40b06d0 2022-06-29T15:50:15.752 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.735+0000 7f7afa12f700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7af40b9670 con 0x7f7af40611b0 2022-06-29T15:50:15.753 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.735+0000 7f7af3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7af40b0cf0 0x7f7af4156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:15.753 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.735+0000 7f7af3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7af40b0cf0 0x7f7af4156160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52086/0 (socket says 172.21.15.47:52086) 2022-06-29T15:50:15.754 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.735+0000 7f7af3fff700 1 -- 172.21.15.47:0/2625819202 learned_addr learned my addr 172.21.15.47:0/2625819202 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:15.754 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.735+0000 7f7af37fe700 1 --2- 172.21.15.47:0/2625819202 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7af40611b0 0x7f7af4148d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:15.755 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.735+0000 7f7af2ffd700 1 --2- 172.21.15.47:0/2625819202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7af40b06d0 0x7f7af4155c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:15.755 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.739+0000 7f7af3fff700 1 -- 172.21.15.47:0/2625819202 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7af40611b0 msgr2=0x7f7af4148d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:15.755 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.739+0000 7f7af3fff700 1 --2- 172.21.15.47:0/2625819202 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7af40611b0 0x7f7af4148d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:15.755 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.739+0000 7f7af3fff700 1 -- 172.21.15.47:0/2625819202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7af40b06d0 msgr2=0x7f7af4155c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:15.756 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.739+0000 7f7af3fff700 1 --2- 172.21.15.47:0/2625819202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7af40b06d0 0x7f7af4155c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:15.756 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.739+0000 7f7af3fff700 1 -- 172.21.15.47:0/2625819202 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aec009470 con 0x7f7af40b0cf0 2022-06-29T15:50:15.756 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.739+0000 7f7af3fff700 1 --2- 172.21.15.47:0/2625819202 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7af40b0cf0 0x7f7af4156160 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 rx=0x7f7af40b4790 tx=0x7f7aec016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:15.756 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.739+0000 7f7af0ff9700 1 -- 172.21.15.47:0/2625819202 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7aec00b5a0 con 0x7f7af40b0cf0 2022-06-29T15:50:15.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.739+0000 7f7af0ff9700 1 -- 172.21.15.47:0/2625819202 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aec01f5e0 con 0x7f7af40b0cf0 2022-06-29T15:50:15.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.739+0000 7f7afa12f700 1 -- 172.21.15.47:0/2625819202 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7af40b8370 con 0x7f7af40b0cf0 2022-06-29T15:50:15.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.739+0000 7f7af0ff9700 1 -- 172.21.15.47:0/2625819202 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7aec01f940 con 0x7f7af40b0cf0 2022-06-29T15:50:15.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.739+0000 7f7afa12f700 1 -- 172.21.15.47:0/2625819202 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7af40b8e30 con 0x7f7af40b0cf0 2022-06-29T15:50:15.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.739+0000 7f7afa12f700 1 -- 172.21.15.47:0/2625819202 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7af40b8ff0 con 0x7f7af40b0cf0 2022-06-29T15:50:15.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.739+0000 7f7af0ff9700 1 -- 172.21.15.47:0/2625819202 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7aec026030 con 0x7f7af40b0cf0 2022-06-29T15:50:15.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.739+0000 7f7af0ff9700 1 --2- 172.21.15.47:0/2625819202 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7ad8053a70 0x7f7ad8055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:15.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.739+0000 7f7af0ff9700 1 -- 172.21.15.47:0/2625819202 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f7aec06cb90 con 0x7f7af40b0cf0 2022-06-29T15:50:15.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.739+0000 7f7af37fe700 1 --2- 172.21.15.47:0/2625819202 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7ad8053a70 0x7f7ad8055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:15.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.743+0000 7f7af37fe700 1 --2- 172.21.15.47:0/2625819202 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7ad8053a70 0x7f7ad8055f30 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7f7adc00b000 tx=0x7f7adc006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:15.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.747+0000 7f7af0ff9700 1 -- 172.21.15.47:0/2625819202 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7aec0438f0 con 0x7f7af40b0cf0 2022-06-29T15:50:15.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.875+0000 7f7afa12f700 1 -- 172.21.15.47:0/2625819202 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7af40573d0 con 0x7f7af40b0cf0 2022-06-29T15:50:15.883 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.879+0000 7f7af0ff9700 1 -- 172.21.15.47:0/2625819202 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+16298 (secure 0 0 0) 0x7f7aec024090 con 0x7f7af40b0cf0 2022-06-29T15:50:15.884 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:50:15.884 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":110,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:50:15.337931+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:50:13.772527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2022-06-29T15:50:14.675947+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:50:15.886 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.879+0000 7f7afa12f700 1 -- 172.21.15.47:0/2625819202 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7ad8053a70 msgr2=0x7f7ad8055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:15.886 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.879+0000 7f7afa12f700 1 --2- 172.21.15.47:0/2625819202 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7ad8053a70 0x7f7ad8055f30 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7f7adc00b000 tx=0x7f7adc006df0).stop 2022-06-29T15:50:15.886 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.879+0000 7f7afa12f700 1 -- 172.21.15.47:0/2625819202 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7af40b0cf0 msgr2=0x7f7af4156160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:15.886 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.879+0000 7f7afa12f700 1 --2- 172.21.15.47:0/2625819202 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7af40b0cf0 0x7f7af4156160 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 rx=0x7f7af40b4790 tx=0x7f7aec016000).stop 2022-06-29T15:50:15.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.879+0000 7f7afa12f700 1 -- 172.21.15.47:0/2625819202 shutdown_connections 2022-06-29T15:50:15.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.879+0000 7f7afa12f700 1 --2- 172.21.15.47:0/2625819202 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7af40b0cf0 0x7f7af4156160 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:15.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.879+0000 7f7afa12f700 1 --2- 172.21.15.47:0/2625819202 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7ad8053a70 0x7f7ad8055f30 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:15.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.879+0000 7f7afa12f700 1 --2- 172.21.15.47:0/2625819202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7af40b06d0 0x7f7af4155c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:15.888 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.879+0000 7f7afa12f700 1 --2- 172.21.15.47:0/2625819202 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7af40611b0 0x7f7af4148d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:15.888 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.879+0000 7f7afa12f700 1 -- 172.21.15.47:0/2625819202 >> 172.21.15.47:0/2625819202 conn(0x7f7af40b98b0 msgr2=0x7f7af40b2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:15.888 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.879+0000 7f7afa12f700 1 -- 172.21.15.47:0/2625819202 shutdown_connections 2022-06-29T15:50:15.888 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:15.879+0000 7f7afa12f700 1 -- 172.21.15.47:0/2625819202 wait complete. 2022-06-29T15:50:15.898 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-06-29T15:50:16.011 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.003+0000 7f78be99b700 1 Processor -- start 2022-06-29T15:50:16.011 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.003+0000 7f78be99b700 1 -- start start 2022-06-29T15:50:16.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.007+0000 7f78be99b700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f78b8066fe0 0x7f78b80b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:16.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.007+0000 7f78be99b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f78b8143150 0x7f78b812bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:16.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.007+0000 7f78be99b700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78b80b68a0 con 0x7f78b80b64c0 2022-06-29T15:50:16.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.007+0000 7f78be99b700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78b80b13a0 con 0x7f78b8143150 2022-06-29T15:50:16.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.007+0000 7f78be99b700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78b81473d0 con 0x7f78b8066fe0 2022-06-29T15:50:16.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.007+0000 7f78be19a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f78b8143150 0x7f78b812bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:16.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.007+0000 7f78bd198700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f78b8066fe0 0x7f78b80b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:16.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.007+0000 7f78be19a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f78b8143150 0x7f78b812bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52094/0 (socket says 172.21.15.47:52094) 2022-06-29T15:50:16.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.007+0000 7f78be19a700 1 -- 172.21.15.47:0/1770509206 learned_addr learned my addr 172.21.15.47:0/1770509206 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:16.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.007+0000 7f78be19a700 1 -- 172.21.15.47:0/1770509206 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f78b8066fe0 msgr2=0x7f78b80b5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:16.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.007+0000 7f78be19a700 1 --2- 172.21.15.47:0/1770509206 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f78b8066fe0 0x7f78b80b5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:16.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.007+0000 7f78be19a700 1 -- 172.21.15.47:0/1770509206 >> v1:172.21.15.64:6789/0 conn(0x7f78b80b64c0 legacy=0x7f78b80b4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:16.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.007+0000 7f78be19a700 1 -- 172.21.15.47:0/1770509206 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b80b46a0 con 0x7f78b8143150 2022-06-29T15:50:16.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.007+0000 7f78be19a700 1 --2- 172.21.15.47:0/1770509206 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78b8143150 0x7f78b812bec0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 rx=0x7f78b0002f70 tx=0x7f78b0006eb0).ready entity=mon.0 client_cookie=854e1ff482f1ae75 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:16.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.007+0000 7f78bc997700 1 -- 172.21.15.47:0/1770509206 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f78b000d570 con 0x7f78b8143150 2022-06-29T15:50:16.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.007+0000 7f78bc997700 1 -- 172.21.15.47:0/1770509206 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78b0000e90 con 0x7f78b8143150 2022-06-29T15:50:16.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.007+0000 7f78bc997700 1 -- 172.21.15.47:0/1770509206 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f78b000d900 con 0x7f78b8143150 2022-06-29T15:50:16.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.007+0000 7f78be99b700 1 -- 172.21.15.47:0/1770509206 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78b8143150 msgr2=0x7f78b812bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:16.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.007+0000 7f78be99b700 1 --2- 172.21.15.47:0/1770509206 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78b8143150 0x7f78b812bec0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 rx=0x7f78b0002f70 tx=0x7f78b0006eb0).stop 2022-06-29T15:50:16.022 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.007+0000 7f78be99b700 1 -- 172.21.15.47:0/1770509206 shutdown_connections 2022-06-29T15:50:16.022 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.007+0000 7f78be99b700 1 --2- 172.21.15.47:0/1770509206 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78b8143150 0x7f78b812bec0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:16.022 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.007+0000 7f78be99b700 1 --2- 172.21.15.47:0/1770509206 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f78b8066fe0 0x7f78b80b5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:16.022 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.007+0000 7f78be99b700 1 -- 172.21.15.47:0/1770509206 >> 172.21.15.47:0/1770509206 conn(0x7f78b8060e00 msgr2=0x7f78b80611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:16.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.007+0000 7f78be99b700 1 -- 172.21.15.47:0/1770509206 shutdown_connections 2022-06-29T15:50:16.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.007+0000 7f78be99b700 1 -- 172.21.15.47:0/1770509206 wait complete. 2022-06-29T15:50:16.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.011+0000 7f78be99b700 1 Processor -- start 2022-06-29T15:50:16.024 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.011+0000 7f78be99b700 1 -- start start 2022-06-29T15:50:16.027 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.011+0000 7f78be99b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78b8066fe0 0x7f78b80b1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:16.027 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.011+0000 7f78be99b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78b80b64c0 0x7f78b81520f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:16.027 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.011+0000 7f78be99b700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f78b8143150 0x7f78b80adea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:16.028 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.011+0000 7f78be99b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b8145bf0 con 0x7f78b8066fe0 2022-06-29T15:50:16.028 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.011+0000 7f78be99b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b8145e90 con 0x7f78b80b64c0 2022-06-29T15:50:16.028 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.011+0000 7f78be99b700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78b80ad760 con 0x7f78b8143150 2022-06-29T15:50:16.028 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.011+0000 7f78bd999700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78b8066fe0 0x7f78b80b1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:16.029 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.011+0000 7f78bd999700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78b8066fe0 0x7f78b80b1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52096/0 (socket says 172.21.15.47:52096) 2022-06-29T15:50:16.029 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.011+0000 7f78bd999700 1 -- 172.21.15.47:0/1535069896 learned_addr learned my addr 172.21.15.47:0/1535069896 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:16.029 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.011+0000 7f78bd198700 1 --2- 172.21.15.47:0/1535069896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78b80b64c0 0x7f78b81520f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:16.029 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.011+0000 7f78be19a700 1 --2- 172.21.15.47:0/1535069896 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f78b8143150 0x7f78b80adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:16.029 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.011+0000 7f78bd999700 1 -- 172.21.15.47:0/1535069896 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f78b8143150 msgr2=0x7f78b80adea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:16.030 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.011+0000 7f78bd999700 1 --2- 172.21.15.47:0/1535069896 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f78b8143150 0x7f78b80adea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:16.030 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.011+0000 7f78bd999700 1 -- 172.21.15.47:0/1535069896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78b80b64c0 msgr2=0x7f78b81520f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:16.030 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.011+0000 7f78bd999700 1 --2- 172.21.15.47:0/1535069896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78b80b64c0 0x7f78b81520f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:16.030 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.011+0000 7f78bd999700 1 -- 172.21.15.47:0/1535069896 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b000b1e0 con 0x7f78b8066fe0 2022-06-29T15:50:16.031 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.011+0000 7f78bd198700 1 --2- 172.21.15.47:0/1535069896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78b80b64c0 0x7f78b81520f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:50:16.031 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.011+0000 7f78bd999700 1 --2- 172.21.15.47:0/1535069896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78b8066fe0 0x7f78b80b1e20 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 rx=0x7f78b80b3fe0 tx=0x7f78a8007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:16.031 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.015+0000 7f78aeffd700 1 -- 172.21.15.47:0/1535069896 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f78a800ce30 con 0x7f78b8066fe0 2022-06-29T15:50:16.031 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.015+0000 7f78aeffd700 1 -- 172.21.15.47:0/1535069896 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78a8007240 con 0x7f78b8066fe0 2022-06-29T15:50:16.032 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.015+0000 7f78be99b700 1 -- 172.21.15.47:0/1535069896 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78b80ada00 con 0x7f78b8066fe0 2022-06-29T15:50:16.032 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.015+0000 7f78aeffd700 1 -- 172.21.15.47:0/1535069896 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f78a8006700 con 0x7f78b8066fe0 2022-06-29T15:50:16.032 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.015+0000 7f78be99b700 1 -- 172.21.15.47:0/1535069896 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78b81525e0 con 0x7f78b8066fe0 2022-06-29T15:50:16.032 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.015+0000 7f78be99b700 1 -- 172.21.15.47:0/1535069896 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78b80adb90 con 0x7f78b8066fe0 2022-06-29T15:50:16.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.015+0000 7f78aeffd700 1 -- 172.21.15.47:0/1535069896 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f78a8010030 con 0x7f78b8066fe0 2022-06-29T15:50:16.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.019+0000 7f78aeffd700 1 --2- 172.21.15.47:0/1535069896 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7898053a10 0x7f7898055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:16.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.019+0000 7f78aeffd700 1 -- 172.21.15.47:0/1535069896 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f78a806bdb0 con 0x7f78b8066fe0 2022-06-29T15:50:16.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.019+0000 7f78bd198700 1 --2- 172.21.15.47:0/1535069896 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7898053a10 0x7f7898055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:16.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.019+0000 7f78bd198700 1 --2- 172.21.15.47:0/1535069896 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7898053a10 0x7f7898055ed0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7f78b400b000 tx=0x7f78b4008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:16.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.023+0000 7f78aeffd700 1 -- 172.21.15.47:0/1535069896 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f78a8070070 con 0x7f78b8066fe0 2022-06-29T15:50:16.192 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.187+0000 7f78be99b700 1 -- 172.21.15.47:0/1535069896 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f78b80573d0 con 0x7f78b8066fe0 2022-06-29T15:50:16.352 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.343+0000 7f78aeffd700 1 -- 172.21.15.47:0/1535069896 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v51) v1 ==== 118+0+0 (secure 0 0 0) 0x7f78a8042a60 con 0x7f78b8066fe0 2022-06-29T15:50:16.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.347+0000 7f78be99b700 1 -- 172.21.15.47:0/1535069896 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7898053a10 msgr2=0x7f7898055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:16.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.347+0000 7f78be99b700 1 --2- 172.21.15.47:0/1535069896 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7898053a10 0x7f7898055ed0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7f78b400b000 tx=0x7f78b4008040).stop 2022-06-29T15:50:16.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.347+0000 7f78be99b700 1 -- 172.21.15.47:0/1535069896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78b8066fe0 msgr2=0x7f78b80b1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:16.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.347+0000 7f78be99b700 1 --2- 172.21.15.47:0/1535069896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78b8066fe0 0x7f78b80b1e20 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 rx=0x7f78b80b3fe0 tx=0x7f78a8007c10).stop 2022-06-29T15:50:16.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.347+0000 7f78be99b700 1 -- 172.21.15.47:0/1535069896 shutdown_connections 2022-06-29T15:50:16.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.347+0000 7f78be99b700 1 --2- 172.21.15.47:0/1535069896 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f78b8143150 0x7f78b80adea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:16.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.347+0000 7f78be99b700 1 --2- 172.21.15.47:0/1535069896 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7898053a10 0x7f7898055ed0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:16.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.347+0000 7f78be99b700 1 --2- 172.21.15.47:0/1535069896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78b80b64c0 0x7f78b81520f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:16.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.347+0000 7f78be99b700 1 --2- 172.21.15.47:0/1535069896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78b8066fe0 0x7f78b80b1e20 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:16.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.347+0000 7f78be99b700 1 -- 172.21.15.47:0/1535069896 >> 172.21.15.47:0/1535069896 conn(0x7f78b8060e00 msgr2=0x7f78b805cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:16.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.347+0000 7f78be99b700 1 -- 172.21.15.47:0/1535069896 shutdown_connections 2022-06-29T15:50:16.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.347+0000 7f78be99b700 1 -- 172.21.15.47:0/1535069896 wait complete. 2022-06-29T15:50:16.371 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:50:16.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.467+0000 7f69fbfff700 1 Processor -- start 2022-06-29T15:50:16.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.467+0000 7f69fbfff700 1 -- start start 2022-06-29T15:50:16.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.471+0000 7f69fbfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f69fc05fa30 0x7f69fc0b1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:16.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.471+0000 7f69fbfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f69fc060b90 0x7f69fc13f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:16.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.471+0000 7f69fbfff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69fc060fd0 con 0x7f69fc0b2770 2022-06-29T15:50:16.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.471+0000 7f69fbfff700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69fc05cc20 con 0x7f69fc060b90 2022-06-29T15:50:16.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.471+0000 7f69fbfff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69fc05fe10 con 0x7f69fc05fa30 2022-06-29T15:50:16.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.471+0000 7f69fa7fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f69fc060b90 0x7f69fc13f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:16.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.471+0000 7f69faffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f69fc05fa30 0x7f69fc0b1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:16.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.471+0000 7f69fa7fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f69fc060b90 0x7f69fc13f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52106/0 (socket says 172.21.15.47:52106) 2022-06-29T15:50:16.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.471+0000 7f69faffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f69fc05fa30 0x7f69fc0b1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:57808/0 (socket says 172.21.15.47:57808) 2022-06-29T15:50:16.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.471+0000 7f69fa7fc700 1 -- 172.21.15.47:0/4116206492 learned_addr learned my addr 172.21.15.47:0/4116206492 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:16.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.471+0000 7f69faffd700 1 -- 172.21.15.47:0/4116206492 >> v2:172.21.15.47:3300/0 conn(0x7f69fc060b90 msgr2=0x7f69fc13f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:16.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.471+0000 7f69faffd700 1 --2- 172.21.15.47:0/4116206492 >> v2:172.21.15.47:3300/0 conn(0x7f69fc060b90 0x7f69fc13f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:16.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.471+0000 7f69faffd700 1 -- 172.21.15.47:0/4116206492 >> v1:172.21.15.64:6789/0 conn(0x7f69fc0b2770 legacy=0x7f69fc05c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:16.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.471+0000 7f69faffd700 1 -- 172.21.15.47:0/4116206492 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69fc0600e0 con 0x7f69fc05fa30 2022-06-29T15:50:16.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.471+0000 7f69faffd700 1 --2- 172.21.15.47:0/4116206492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69fc05fa30 0x7f69fc0b1ab0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f69ec002f70 tx=0x7f69ec006df0).ready entity=mon.1 client_cookie=8357f1dd783294cc server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:16.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.471+0000 7f69f9ffb700 1 -- 172.21.15.47:0/4116206492 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f69ec00d570 con 0x7f69fc05fa30 2022-06-29T15:50:16.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.471+0000 7f69f9ffb700 1 -- 172.21.15.47:0/4116206492 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69ec000e90 con 0x7f69fc05fa30 2022-06-29T15:50:16.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.471+0000 7f69f9ffb700 1 -- 172.21.15.47:0/4116206492 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f69ec00d8c0 con 0x7f69fc05fa30 2022-06-29T15:50:16.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.475+0000 7f69fbfff700 1 -- 172.21.15.47:0/4116206492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69fc05fa30 msgr2=0x7f69fc0b1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:16.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.475+0000 7f69fbfff700 1 --2- 172.21.15.47:0/4116206492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69fc05fa30 0x7f69fc0b1ab0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f69ec002f70 tx=0x7f69ec006df0).stop 2022-06-29T15:50:16.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.475+0000 7f69fbfff700 1 -- 172.21.15.47:0/4116206492 shutdown_connections 2022-06-29T15:50:16.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.475+0000 7f69fbfff700 1 --2- 172.21.15.47:0/4116206492 >> v2:172.21.15.47:3300/0 conn(0x7f69fc060b90 0x7f69fc13f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:16.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.475+0000 7f69fbfff700 1 --2- 172.21.15.47:0/4116206492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69fc05fa30 0x7f69fc0b1ab0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:16.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.475+0000 7f69fbfff700 1 -- 172.21.15.47:0/4116206492 >> 172.21.15.47:0/4116206492 conn(0x7f69fc062df0 msgr2=0x7f69fc0631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:16.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.475+0000 7f69fbfff700 1 -- 172.21.15.47:0/4116206492 shutdown_connections 2022-06-29T15:50:16.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.475+0000 7f69fbfff700 1 -- 172.21.15.47:0/4116206492 wait complete. 2022-06-29T15:50:16.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.475+0000 7f69fbfff700 1 Processor -- start 2022-06-29T15:50:16.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.475+0000 7f69fbfff700 1 -- start start 2022-06-29T15:50:16.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.475+0000 7f69fbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69fc05fa30 0x7f69fc144bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:16.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.475+0000 7f69fbfff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f69fc060b90 0x7f69fc1467b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:16.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.475+0000 7f69fbfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69fc0b2770 0x7f69fc146ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:16.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.475+0000 7f69fbfff700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69fc142e80 con 0x7f69fc0b2770 2022-06-29T15:50:16.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.475+0000 7f69fbfff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69fc1408d0 con 0x7f69fc05fa30 2022-06-29T15:50:16.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.475+0000 7f69fbfff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69fc0ac460 con 0x7f69fc060b90 2022-06-29T15:50:16.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.475+0000 7f69fb7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69fc0b2770 0x7f69fc146ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:16.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.475+0000 7f69fb7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69fc0b2770 0x7f69fc146ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52112/0 (socket says 172.21.15.47:52112) 2022-06-29T15:50:16.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.475+0000 7f69faffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69fc05fa30 0x7f69fc144bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:16.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.475+0000 7f69fb7fe700 1 -- 172.21.15.47:0/1115107322 learned_addr learned my addr 172.21.15.47:0/1115107322 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:16.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.475+0000 7f69fa7fc700 1 --2- 172.21.15.47:0/1115107322 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f69fc060b90 0x7f69fc1467b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:16.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.475+0000 7f69fb7fe700 1 -- 172.21.15.47:0/1115107322 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f69fc060b90 msgr2=0x7f69fc1467b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:16.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.475+0000 7f69fb7fe700 1 --2- 172.21.15.47:0/1115107322 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f69fc060b90 0x7f69fc1467b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:16.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.475+0000 7f69fb7fe700 1 -- 172.21.15.47:0/1115107322 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69fc05fa30 msgr2=0x7f69fc144bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:16.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.475+0000 7f69fb7fe700 1 --2- 172.21.15.47:0/1115107322 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69fc05fa30 0x7f69fc144bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:16.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.475+0000 7f69fb7fe700 1 -- 172.21.15.47:0/1115107322 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69ec00b180 con 0x7f69fc0b2770 2022-06-29T15:50:16.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.475+0000 7f69fa7fc700 1 --2- 172.21.15.47:0/1115107322 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f69fc060b90 0x7f69fc1467b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:50:16.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.479+0000 7f69fb7fe700 1 --2- 172.21.15.47:0/1115107322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69fc0b2770 0x7f69fc146ca0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 rx=0x7f69fc0a9980 tx=0x7f69f400c300).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:16.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.479+0000 7f69ebfff700 1 -- 172.21.15.47:0/1115107322 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f69f400ee30 con 0x7f69fc0b2770 2022-06-29T15:50:16.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.479+0000 7f69ebfff700 1 -- 172.21.15.47:0/1115107322 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69f4006aa0 con 0x7f69fc0b2770 2022-06-29T15:50:16.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.479+0000 7f69fbfff700 1 -- 172.21.15.47:0/1115107322 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69fc1411b0 con 0x7f69fc0b2770 2022-06-29T15:50:16.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.479+0000 7f69ebfff700 1 -- 172.21.15.47:0/1115107322 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f69f4006e00 con 0x7f69fc0b2770 2022-06-29T15:50:16.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.479+0000 7f69fbfff700 1 -- 172.21.15.47:0/1115107322 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69fc147190 con 0x7f69fc0b2770 2022-06-29T15:50:16.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.479+0000 7f69fbfff700 1 -- 172.21.15.47:0/1115107322 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69fc0573d0 con 0x7f69fc0b2770 2022-06-29T15:50:16.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.479+0000 7f69ebfff700 1 -- 172.21.15.47:0/1115107322 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f69f4007030 con 0x7f69fc0b2770 2022-06-29T15:50:16.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.479+0000 7f69ebfff700 1 --2- 172.21.15.47:0/1115107322 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f69d8053950 0x7f69d8055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:16.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.479+0000 7f69ebfff700 1 -- 172.21.15.47:0/1115107322 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f69f406e6a0 con 0x7f69fc0b2770 2022-06-29T15:50:16.497 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.479+0000 7f69faffd700 1 --2- 172.21.15.47:0/1115107322 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f69d8053950 0x7f69d8055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:16.497 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.483+0000 7f69faffd700 1 --2- 172.21.15.47:0/1115107322 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f69d8053950 0x7f69d8055e10 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7f69ec01a000 tx=0x7f69ec014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:16.497 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.487+0000 7f69ebfff700 1 -- 172.21.15.47:0/1115107322 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f69f4045460 con 0x7f69fc0b2770 2022-06-29T15:50:16.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.655+0000 7f69fbfff700 1 -- 172.21.15.47:0/1115107322 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f69fc060fd0 con 0x7f69fc0b2770 2022-06-29T15:50:16.663 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.659+0000 7f69ebfff700 1 -- 172.21.15.47:0/1115107322 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+2496 (secure 0 0 0) 0x7f69fc060fd0 con 0x7f69fc0b2770 2022-06-29T15:50:16.664 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:50:16.664 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":51,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3752554010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3752554010},{"type":"v1","addr":"172.21.15.47:6835","nonce":3752554010}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":48},{"gid":7293,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.47:6837/3263659716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":3263659716},{"type":"v1","addr":"172.21.15.47:6837","nonce":3263659716}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":49}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:50:15.338114+0000","modified":"2022-06-29T15:50:16.342008+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":56354},"failed":[],"damaged":[],"stopped":[],"info":{"gid_56354":{"gid":56354,"name":"b","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6833/2706285595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2706285595},{"type":"v1","addr":"172.21.15.64:6833","nonce":2706285595}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2022-06-29T15:50:16.666 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.659+0000 7f69fbfff700 1 -- 172.21.15.47:0/1115107322 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f69d8053950 msgr2=0x7f69d8055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:16.667 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.659+0000 7f69fbfff700 1 --2- 172.21.15.47:0/1115107322 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f69d8053950 0x7f69d8055e10 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7f69ec01a000 tx=0x7f69ec014000).stop 2022-06-29T15:50:16.667 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.659+0000 7f69fbfff700 1 -- 172.21.15.47:0/1115107322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69fc0b2770 msgr2=0x7f69fc146ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:16.667 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.659+0000 7f69fbfff700 1 --2- 172.21.15.47:0/1115107322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69fc0b2770 0x7f69fc146ca0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 rx=0x7f69fc0a9980 tx=0x7f69f400c300).stop 2022-06-29T15:50:16.668 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.659+0000 7f69fbfff700 1 -- 172.21.15.47:0/1115107322 shutdown_connections 2022-06-29T15:50:16.668 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.659+0000 7f69fbfff700 1 --2- 172.21.15.47:0/1115107322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69fc0b2770 0x7f69fc146ca0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:16.669 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.659+0000 7f69fbfff700 1 --2- 172.21.15.47:0/1115107322 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f69fc060b90 0x7f69fc1467b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:16.669 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.659+0000 7f69fbfff700 1 --2- 172.21.15.47:0/1115107322 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f69d8053950 0x7f69d8055e10 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:16.669 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.659+0000 7f69fbfff700 1 --2- 172.21.15.47:0/1115107322 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69fc05fa30 0x7f69fc144bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:16.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.659+0000 7f69fbfff700 1 -- 172.21.15.47:0/1115107322 >> 172.21.15.47:0/1115107322 conn(0x7f69fc062df0 msgr2=0x7f69fc0aded0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:16.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.659+0000 7f69fbfff700 1 -- 172.21.15.47:0/1115107322 shutdown_connections 2022-06-29T15:50:16.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.659+0000 7f69fbfff700 1 -- 172.21.15.47:0/1115107322 wait complete. 2022-06-29T15:50:16.672 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 51 2022-06-29T15:50:16.681 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:50:16.787 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.779+0000 7f8c0d996700 1 Processor -- start 2022-06-29T15:50:16.788 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.779+0000 7f8c0d996700 1 -- start start 2022-06-29T15:50:16.796 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.783+0000 7f8c0d996700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8c0805f0b0 0x7f8c080b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:16.797 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.783+0000 7f8c0d996700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8c080b1260 0x7f8c0814b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:16.797 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.783+0000 7f8c0d996700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c080b0740 con 0x7f8c080b2120 2022-06-29T15:50:16.797 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.783+0000 7f8c0d996700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c08147460 con 0x7f8c0805f0b0 2022-06-29T15:50:16.797 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.783+0000 7f8c0d996700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c080b0970 con 0x7f8c080b1260 2022-06-29T15:50:16.798 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.783+0000 7f8c067fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8c0805f0b0 0x7f8c080b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:16.798 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.783+0000 7f8c067fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8c0805f0b0 0x7f8c080b2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52118/0 (socket says 172.21.15.47:52118) 2022-06-29T15:50:16.798 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.783+0000 7f8c067fc700 1 -- 172.21.15.47:0/1791765761 learned_addr learned my addr 172.21.15.47:0/1791765761 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:16.798 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.783+0000 7f8c077fe700 1 --2- 172.21.15.47:0/1791765761 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8c080b1260 0x7f8c0814b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:16.799 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.783+0000 7f8c067fc700 1 -- 172.21.15.47:0/1791765761 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8c080b1260 msgr2=0x7f8c0814b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:16.799 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.783+0000 7f8c067fc700 1 --2- 172.21.15.47:0/1791765761 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8c080b1260 0x7f8c0814b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:16.799 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.783+0000 7f8c067fc700 1 -- 172.21.15.47:0/1791765761 >> v1:172.21.15.47:6789/0 conn(0x7f8c080b2120 legacy=0x7f8c08147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:16.799 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.783+0000 7f8c067fc700 1 -- 172.21.15.47:0/1791765761 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c080b0c10 con 0x7f8c0805f0b0 2022-06-29T15:50:16.800 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.783+0000 7f8c077fe700 1 --2- 172.21.15.47:0/1791765761 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8c080b1260 0x7f8c0814b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:50:16.800 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.783+0000 7f8c067fc700 1 --2- 172.21.15.47:0/1791765761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c0805f0b0 0x7f8c080b2500 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 rx=0x7f8c08141a70 tx=0x7f8bf8006eb0).ready entity=mon.0 client_cookie=a9fcaba56ee209e5 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:16.800 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.783+0000 7f8c05ffb700 1 -- 172.21.15.47:0/1791765761 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8bf800d5c0 con 0x7f8c0805f0b0 2022-06-29T15:50:16.800 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.783+0000 7f8c05ffb700 1 -- 172.21.15.47:0/1791765761 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf8000e90 con 0x7f8c0805f0b0 2022-06-29T15:50:16.801 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.783+0000 7f8c05ffb700 1 -- 172.21.15.47:0/1791765761 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8bf800d950 con 0x7f8c0805f0b0 2022-06-29T15:50:16.801 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.783+0000 7f8c0d996700 1 -- 172.21.15.47:0/1791765761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c0805f0b0 msgr2=0x7f8c080b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:16.801 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.783+0000 7f8c0d996700 1 --2- 172.21.15.47:0/1791765761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c0805f0b0 0x7f8c080b2500 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 rx=0x7f8c08141a70 tx=0x7f8bf8006eb0).stop 2022-06-29T15:50:16.801 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.783+0000 7f8c0d996700 1 -- 172.21.15.47:0/1791765761 shutdown_connections 2022-06-29T15:50:16.801 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.783+0000 7f8c0d996700 1 --2- 172.21.15.47:0/1791765761 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8c080b1260 0x7f8c0814b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:16.802 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.783+0000 7f8c0d996700 1 --2- 172.21.15.47:0/1791765761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c0805f0b0 0x7f8c080b2500 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:16.802 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.783+0000 7f8c0d996700 1 -- 172.21.15.47:0/1791765761 >> 172.21.15.47:0/1791765761 conn(0x7f8c080626f0 msgr2=0x7f8c08062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:16.802 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.783+0000 7f8c0d996700 1 -- 172.21.15.47:0/1791765761 shutdown_connections 2022-06-29T15:50:16.803 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.783+0000 7f8c0d996700 1 -- 172.21.15.47:0/1791765761 wait complete. 2022-06-29T15:50:16.803 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.783+0000 7f8c0d996700 1 Processor -- start 2022-06-29T15:50:16.803 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.787+0000 7f8c0d996700 1 -- start start 2022-06-29T15:50:16.803 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.787+0000 7f8c0d996700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c080b1260 0x7f8c08149d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:16.804 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.787+0000 7f8c0d996700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c080b2120 0x7f8c08140760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:16.804 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.787+0000 7f8c0d996700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8c080ae990 0x7f8c08140c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:16.804 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.787+0000 7f8c0d996700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c08143090 con 0x7f8c080b1260 2022-06-29T15:50:16.804 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.787+0000 7f8c0d996700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c080af720 con 0x7f8c080b2120 2022-06-29T15:50:16.804 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.787+0000 7f8c0d996700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c080adff0 con 0x7f8c080ae990 2022-06-29T15:50:16.805 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.787+0000 7f8c06ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c080b1260 0x7f8c08149d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:16.805 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.787+0000 7f8c06ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c080b1260 0x7f8c08149d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52124/0 (socket says 172.21.15.47:52124) 2022-06-29T15:50:16.806 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.787+0000 7f8c06ffd700 1 -- 172.21.15.47:0/1202503635 learned_addr learned my addr 172.21.15.47:0/1202503635 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:16.806 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.787+0000 7f8c077fe700 1 --2- 172.21.15.47:0/1202503635 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8c080ae990 0x7f8c08140c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:16.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.787+0000 7f8c067fc700 1 --2- 172.21.15.47:0/1202503635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c080b2120 0x7f8c08140760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:16.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.787+0000 7f8c06ffd700 1 -- 172.21.15.47:0/1202503635 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8c080ae990 msgr2=0x7f8c08140c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:16.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.787+0000 7f8c06ffd700 1 --2- 172.21.15.47:0/1202503635 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8c080ae990 0x7f8c08140c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:16.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.787+0000 7f8c06ffd700 1 -- 172.21.15.47:0/1202503635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c080b2120 msgr2=0x7f8c08140760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:16.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.787+0000 7f8c06ffd700 1 --2- 172.21.15.47:0/1202503635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c080b2120 0x7f8c08140760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:16.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.787+0000 7f8c06ffd700 1 -- 172.21.15.47:0/1202503635 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf800b1e0 con 0x7f8c080b1260 2022-06-29T15:50:16.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.787+0000 7f8c06ffd700 1 --2- 172.21.15.47:0/1202503635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c080b1260 0x7f8c08149d20 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 rx=0x7f8c080afcc0 tx=0x7f8bfc007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:16.809 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.787+0000 7f8bf7fff700 1 -- 172.21.15.47:0/1202503635 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8bfc012400 con 0x7f8c080b1260 2022-06-29T15:50:16.809 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.787+0000 7f8bf7fff700 1 -- 172.21.15.47:0/1202503635 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bfc012560 con 0x7f8c080b1260 2022-06-29T15:50:16.809 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.787+0000 7f8c0d996700 1 -- 172.21.15.47:0/1202503635 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c080ae290 con 0x7f8c080b1260 2022-06-29T15:50:16.809 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.787+0000 7f8bf7fff700 1 -- 172.21.15.47:0/1202503635 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8bfc0128c0 con 0x7f8c080b1260 2022-06-29T15:50:16.810 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.787+0000 7f8c0d996700 1 -- 172.21.15.47:0/1202503635 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c080af3a0 con 0x7f8c080b1260 2022-06-29T15:50:16.810 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.787+0000 7f8c0d996700 1 -- 172.21.15.47:0/1202503635 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c08143e90 con 0x7f8c080b1260 2022-06-29T15:50:16.810 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.791+0000 7f8bf7fff700 1 -- 172.21.15.47:0/1202503635 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8bfc019030 con 0x7f8c080b1260 2022-06-29T15:50:16.810 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.791+0000 7f8bf7fff700 1 --2- 172.21.15.47:0/1202503635 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8be4053a70 0x7f8be4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:16.811 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.791+0000 7f8bf7fff700 1 -- 172.21.15.47:0/1202503635 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f8bfc06b3a0 con 0x7f8c080b1260 2022-06-29T15:50:16.811 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.791+0000 7f8c067fc700 1 --2- 172.21.15.47:0/1202503635 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8be4053a70 0x7f8be4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:16.812 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.791+0000 7f8c067fc700 1 --2- 172.21.15.47:0/1202503635 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8be4053a70 0x7f8be4055f30 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7f8bf801b000 tx=0x7f8bf8015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:16.812 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.795+0000 7f8bf7fff700 1 -- 172.21.15.47:0/1202503635 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8bfc0420c0 con 0x7f8c080b1260 2022-06-29T15:50:16.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.923+0000 7f8c0d996700 1 -- 172.21.15.47:0/1202503635 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c080af8b0 con 0x7f8c080b1260 2022-06-29T15:50:16.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.927+0000 7f8bf7fff700 1 -- 172.21.15.47:0/1202503635 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f8bfc0456e0 con 0x7f8c080b1260 2022-06-29T15:50:16.932 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:50:16.932 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":111,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:50:16.342650+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:50:13.772527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2022-06-29T15:50:14.675947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:50:16.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.927+0000 7f8c0d996700 1 -- 172.21.15.47:0/1202503635 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8be4053a70 msgr2=0x7f8be4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:16.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.927+0000 7f8c0d996700 1 --2- 172.21.15.47:0/1202503635 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8be4053a70 0x7f8be4055f30 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7f8bf801b000 tx=0x7f8bf8015000).stop 2022-06-29T15:50:16.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.927+0000 7f8c0d996700 1 -- 172.21.15.47:0/1202503635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c080b1260 msgr2=0x7f8c08149d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:16.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.927+0000 7f8c0d996700 1 --2- 172.21.15.47:0/1202503635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c080b1260 0x7f8c08149d20 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 rx=0x7f8c080afcc0 tx=0x7f8bfc007c50).stop 2022-06-29T15:50:16.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.927+0000 7f8c0d996700 1 -- 172.21.15.47:0/1202503635 shutdown_connections 2022-06-29T15:50:16.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.927+0000 7f8c0d996700 1 --2- 172.21.15.47:0/1202503635 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8c080ae990 0x7f8c08140c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:16.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.927+0000 7f8c0d996700 1 --2- 172.21.15.47:0/1202503635 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8be4053a70 0x7f8be4055f30 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:16.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.927+0000 7f8c0d996700 1 --2- 172.21.15.47:0/1202503635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c080b2120 0x7f8c08140760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:16.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.927+0000 7f8c0d996700 1 --2- 172.21.15.47:0/1202503635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c080b1260 0x7f8c08149d20 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:16.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.927+0000 7f8c0d996700 1 -- 172.21.15.47:0/1202503635 >> 172.21.15.47:0/1202503635 conn(0x7f8c080626f0 msgr2=0x7f8c0805f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:16.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.927+0000 7f8c0d996700 1 -- 172.21.15.47:0/1202503635 shutdown_connections 2022-06-29T15:50:16.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:16.927+0000 7f8c0d996700 1 -- 172.21.15.47:0/1202503635 wait complete. 2022-06-29T15:50:16.946 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-29T15:50:17.065 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.059+0000 7f906759e700 1 Processor -- start 2022-06-29T15:50:17.066 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.059+0000 7f906759e700 1 -- start start 2022-06-29T15:50:17.067 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.059+0000 7f906759e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f906805c850 0x7f9068143a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:17.067 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.059+0000 7f906759e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f90680676a0 0x7f9068140130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:17.068 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.059+0000 7f906759e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f906805cc30 con 0x7f90680b5d90 2022-06-29T15:50:17.068 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.059+0000 7f906759e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90681430a0 con 0x7f906805c850 2022-06-29T15:50:17.068 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.059+0000 7f906759e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9068143300 con 0x7f90680676a0 2022-06-29T15:50:17.068 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.059+0000 7f905dd9b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f90680b5d90 0x7f90680b4860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37686/0 (socket says 172.21.15.47:37686) 2022-06-29T15:50:17.069 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.059+0000 7f905dd9b700 1 -- 172.21.15.47:0/425853158 learned_addr learned my addr 172.21.15.47:0/425853158 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:17.069 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.059+0000 7f906659c700 1 --2- 172.21.15.47:0/425853158 >> v2:172.21.15.47:3300/0 conn(0x7f906805c850 0x7f9068143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:17.069 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.059+0000 7f9066d9d700 1 --2- 172.21.15.47:0/425853158 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f90680676a0 0x7f9068140130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:17.070 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.063+0000 7f9065d9b700 1 -- 172.21.15.47:0/425853158 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3872146308 0 0) 0x7f906805cc30 con 0x7f90680b5d90 2022-06-29T15:50:17.070 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.063+0000 7f9065d9b700 1 -- 172.21.15.47:0/425853158 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9048003260 con 0x7f90680b5d90 2022-06-29T15:50:17.072 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.063+0000 7f9065d9b700 1 -- 172.21.15.47:0/425853158 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1867251134 0 0) 0x7f9048003260 con 0x7f90680b5d90 2022-06-29T15:50:17.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.063+0000 7f906659c700 1 -- 172.21.15.47:0/425853158 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f90680676a0 msgr2=0x7f9068140130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:17.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.063+0000 7f906659c700 1 --2- 172.21.15.47:0/425853158 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f90680676a0 0x7f9068140130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:17.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.063+0000 7f906659c700 1 -- 172.21.15.47:0/425853158 >> v1:172.21.15.47:6789/0 conn(0x7f90680b5d90 legacy=0x7f90680b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:17.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.063+0000 7f906659c700 1 -- 172.21.15.47:0/425853158 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90681435a0 con 0x7f906805c850 2022-06-29T15:50:17.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.063+0000 7f9066d9d700 1 --2- 172.21.15.47:0/425853158 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f90680676a0 0x7f9068140130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:50:17.074 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.063+0000 7f906659c700 1 --2- 172.21.15.47:0/425853158 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f906805c850 0x7f9068143a40 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 rx=0x7f9050001820 tx=0x7f9058006df0).ready entity=mon.0 client_cookie=73f892dfb904d3f4 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:17.074 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.063+0000 7f9065d9b700 1 -- 172.21.15.47:0/425853158 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9050001680 con 0x7f906805c850 2022-06-29T15:50:17.074 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.063+0000 7f9065d9b700 1 -- 172.21.15.47:0/425853158 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9058000e90 con 0x7f906805c850 2022-06-29T15:50:17.074 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.063+0000 7f9065d9b700 1 -- 172.21.15.47:0/425853158 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f905800e470 con 0x7f906805c850 2022-06-29T15:50:17.075 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.063+0000 7f906759e700 1 -- 172.21.15.47:0/425853158 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f906805c850 msgr2=0x7f9068143a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:17.075 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.063+0000 7f906759e700 1 --2- 172.21.15.47:0/425853158 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f906805c850 0x7f9068143a40 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 rx=0x7f9050001820 tx=0x7f9058006df0).stop 2022-06-29T15:50:17.075 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.063+0000 7f906759e700 1 -- 172.21.15.47:0/425853158 shutdown_connections 2022-06-29T15:50:17.075 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.063+0000 7f906759e700 1 --2- 172.21.15.47:0/425853158 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f90680676a0 0x7f9068140130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:17.076 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.063+0000 7f906759e700 1 --2- 172.21.15.47:0/425853158 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f906805c850 0x7f9068143a40 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:17.076 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.063+0000 7f906759e700 1 -- 172.21.15.47:0/425853158 >> 172.21.15.47:0/425853158 conn(0x7f9068060e00 msgr2=0x7f90680611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:17.076 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.063+0000 7f906759e700 1 -- 172.21.15.47:0/425853158 shutdown_connections 2022-06-29T15:50:17.076 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.063+0000 7f906759e700 1 -- 172.21.15.47:0/425853158 wait complete. 2022-06-29T15:50:17.077 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.067+0000 7f906759e700 1 Processor -- start 2022-06-29T15:50:17.077 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.067+0000 7f906759e700 1 -- start start 2022-06-29T15:50:17.080 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.067+0000 7f906759e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f906805c850 0x7f90680b9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:17.080 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.067+0000 7f906759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90680676a0 0x7f90680b3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:17.081 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.067+0000 7f906759e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f90680b5d90 0x7f906812c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:17.081 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.067+0000 7f906759e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906812ee40 con 0x7f906805c850 2022-06-29T15:50:17.082 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.067+0000 7f906759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90680b96e0 con 0x7f90680676a0 2022-06-29T15:50:17.082 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.067+0000 7f906759e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90680b91a0 con 0x7f90680b5d90 2022-06-29T15:50:17.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.067+0000 7f906659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f906805c850 0x7f90680b9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:17.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.067+0000 7f906659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f906805c850 0x7f90680b9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52138/0 (socket says 172.21.15.47:52138) 2022-06-29T15:50:17.084 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.067+0000 7f906659c700 1 -- 172.21.15.47:0/2367190926 learned_addr learned my addr 172.21.15.47:0/2367190926 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:17.084 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.067+0000 7f9066d9d700 1 --2- 172.21.15.47:0/2367190926 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f90680b5d90 0x7f906812c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:17.084 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.067+0000 7f905dd9b700 1 --2- 172.21.15.47:0/2367190926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90680676a0 0x7f90680b3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:17.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.067+0000 7f906659c700 1 -- 172.21.15.47:0/2367190926 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f90680b5d90 msgr2=0x7f906812c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:17.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.067+0000 7f906659c700 1 --2- 172.21.15.47:0/2367190926 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f90680b5d90 0x7f906812c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:17.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.067+0000 7f906659c700 1 -- 172.21.15.47:0/2367190926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90680676a0 msgr2=0x7f90680b3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:17.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.067+0000 7f906659c700 1 --2- 172.21.15.47:0/2367190926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90680676a0 0x7f90680b3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:17.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.067+0000 7f906659c700 1 -- 172.21.15.47:0/2367190926 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9060009470 con 0x7f906805c850 2022-06-29T15:50:17.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.067+0000 7f9066d9d700 1 --2- 172.21.15.47:0/2367190926 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f90680b5d90 0x7f906812c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:50:17.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.067+0000 7f906659c700 1 --2- 172.21.15.47:0/2367190926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f906805c850 0x7f90680b9b40 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 rx=0x7f90680b9870 tx=0x7f9058014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:17.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.067+0000 7f905ffff700 1 -- 172.21.15.47:0/2367190926 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f905800e470 con 0x7f906805c850 2022-06-29T15:50:17.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.067+0000 7f905ffff700 1 -- 172.21.15.47:0/2367190926 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f905800beb0 con 0x7f906805c850 2022-06-29T15:50:17.089 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.067+0000 7f905ffff700 1 -- 172.21.15.47:0/2367190926 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9058011e90 con 0x7f906805c850 2022-06-29T15:50:17.089 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.067+0000 7f906759e700 1 -- 172.21.15.47:0/2367190926 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f905800b120 con 0x7f906805c850 2022-06-29T15:50:17.090 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.067+0000 7f906759e700 1 -- 172.21.15.47:0/2367190926 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90680b3700 con 0x7f906805c850 2022-06-29T15:50:17.090 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.071+0000 7f906759e700 1 -- 172.21.15.47:0/2367190926 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90680573d0 con 0x7f906805c850 2022-06-29T15:50:17.091 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.071+0000 7f905ffff700 1 -- 172.21.15.47:0/2367190926 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9058023030 con 0x7f906805c850 2022-06-29T15:50:17.091 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.071+0000 7f905ffff700 1 --2- 172.21.15.47:0/2367190926 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f904c0539e0 0x7f904c055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:17.091 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.071+0000 7f905dd9b700 1 --2- 172.21.15.47:0/2367190926 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f904c0539e0 0x7f904c055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:17.092 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.071+0000 7f905ffff700 1 -- 172.21.15.47:0/2367190926 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f905806cd00 con 0x7f906805c850 2022-06-29T15:50:17.092 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.071+0000 7f905dd9b700 1 --2- 172.21.15.47:0/2367190926 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f904c0539e0 0x7f904c055ea0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7f905000b000 tx=0x7f9050001c20).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:17.093 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.075+0000 7f905ffff700 1 -- 172.21.15.47:0/2367190926 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9058043a30 con 0x7f906805c850 2022-06-29T15:50:17.252 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.247+0000 7f906759e700 1 -- 172.21.15.47:0/2367190926 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f906812be10 con 0x7f906805c850 2022-06-29T15:50:17.261 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.255+0000 7f905ffff700 1 -- 172.21.15.47:0/2367190926 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v29) v1 ==== 184+0+0 (secure 0 0 0) 0x7f9058021020 con 0x7f906805c850 2022-06-29T15:50:17.264 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.255+0000 7f906759e700 1 -- 172.21.15.47:0/2367190926 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f904c0539e0 msgr2=0x7f904c055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:17.264 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.255+0000 7f906759e700 1 --2- 172.21.15.47:0/2367190926 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f904c0539e0 0x7f904c055ea0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7f905000b000 tx=0x7f9050001c20).stop 2022-06-29T15:50:17.265 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.255+0000 7f906759e700 1 -- 172.21.15.47:0/2367190926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f906805c850 msgr2=0x7f90680b9b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:17.265 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.255+0000 7f906759e700 1 --2- 172.21.15.47:0/2367190926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f906805c850 0x7f90680b9b40 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 rx=0x7f90680b9870 tx=0x7f9058014000).stop 2022-06-29T15:50:17.265 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.255+0000 7f906759e700 1 -- 172.21.15.47:0/2367190926 shutdown_connections 2022-06-29T15:50:17.266 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.255+0000 7f906759e700 1 --2- 172.21.15.47:0/2367190926 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f90680b5d90 0x7f906812c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:17.266 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.255+0000 7f906759e700 1 --2- 172.21.15.47:0/2367190926 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f904c0539e0 0x7f904c055ea0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:17.266 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.255+0000 7f906759e700 1 --2- 172.21.15.47:0/2367190926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90680676a0 0x7f90680b3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:17.267 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.255+0000 7f906759e700 1 --2- 172.21.15.47:0/2367190926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f906805c850 0x7f90680b9b40 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:17.267 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.255+0000 7f906759e700 1 -- 172.21.15.47:0/2367190926 >> 172.21.15.47:0/2367190926 conn(0x7f9068060e00 msgr2=0x7f90680ae8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:17.267 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.259+0000 7f906759e700 1 -- 172.21.15.47:0/2367190926 shutdown_connections 2022-06-29T15:50:17.268 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.259+0000 7f906759e700 1 -- 172.21.15.47:0/2367190926 wait complete. 2022-06-29T15:50:17.269 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.0 2022-06-29T15:50:17.278 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-29T15:50:17.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.391+0000 7f869539e700 1 Processor -- start 2022-06-29T15:50:17.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.391+0000 7f869539e700 1 -- start start 2022-06-29T15:50:17.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.395+0000 7f869539e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8690062f30 0x7f8690148dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:17.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.395+0000 7f869539e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86900b3d30 con 0x7f86900612b0 2022-06-29T15:50:17.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.395+0000 7f869539e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86900b7100 con 0x7f86900b6d20 2022-06-29T15:50:17.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.395+0000 7f869539e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86900b17f0 con 0x7f8690062f30 2022-06-29T15:50:17.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.395+0000 7f8694b9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8690062f30 0x7f8690148dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:17.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.395+0000 7f86877fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f86900612b0 0x7f8690066860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37702/0 (socket says 172.21.15.47:37702) 2022-06-29T15:50:17.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.395+0000 7f86877fe700 1 -- 172.21.15.47:0/1762300226 learned_addr learned my addr 172.21.15.47:0/1762300226 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:17.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.395+0000 7f8694b9d700 1 -- 172.21.15.47:0/1762300226 >> v1:172.21.15.64:6789/0 conn(0x7f86900b6d20 legacy=0x7f86900b3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:17.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.395+0000 7f8694b9d700 1 -- 172.21.15.47:0/1762300226 >> v1:172.21.15.47:6789/0 conn(0x7f86900612b0 legacy=0x7f8690066860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:17.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.395+0000 7f8694b9d700 1 -- 172.21.15.47:0/1762300226 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86900b1e80 con 0x7f8690062f30 2022-06-29T15:50:17.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.395+0000 7f8694b9d700 1 --2- 172.21.15.47:0/1762300226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8690062f30 0x7f8690148dc0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 rx=0x7f86880032f0 tx=0x7f86880091d0).ready entity=mon.0 client_cookie=73a9c819507cb00 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:17.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.395+0000 7f868f7fe700 1 -- 172.21.15.47:0/1762300226 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8688009ea0 con 0x7f8690062f30 2022-06-29T15:50:17.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.395+0000 7f868f7fe700 1 -- 172.21.15.47:0/1762300226 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f868800b8a0 con 0x7f8690062f30 2022-06-29T15:50:17.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.395+0000 7f868f7fe700 1 -- 172.21.15.47:0/1762300226 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f868800ba10 con 0x7f8690062f30 2022-06-29T15:50:17.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.395+0000 7f869539e700 1 -- 172.21.15.47:0/1762300226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8690062f30 msgr2=0x7f8690148dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:17.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.395+0000 7f869539e700 1 --2- 172.21.15.47:0/1762300226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8690062f30 0x7f8690148dc0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 rx=0x7f86880032f0 tx=0x7f86880091d0).stop 2022-06-29T15:50:17.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.395+0000 7f869539e700 1 -- 172.21.15.47:0/1762300226 shutdown_connections 2022-06-29T15:50:17.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.395+0000 7f869539e700 1 --2- 172.21.15.47:0/1762300226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8690062f30 0x7f8690148dc0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:17.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.395+0000 7f869539e700 1 -- 172.21.15.47:0/1762300226 >> 172.21.15.47:0/1762300226 conn(0x7f8690063670 msgr2=0x7f8690063a50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:17.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.395+0000 7f869539e700 1 -- 172.21.15.47:0/1762300226 shutdown_connections 2022-06-29T15:50:17.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.395+0000 7f869539e700 1 -- 172.21.15.47:0/1762300226 wait complete. 2022-06-29T15:50:17.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.403+0000 7f869539e700 1 Processor -- start 2022-06-29T15:50:17.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.403+0000 7f869539e700 1 -- start start 2022-06-29T15:50:17.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.403+0000 7f869539e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f86900612b0 0x7f86900aca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:17.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.403+0000 7f869539e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8690062f30 0x7f8690149340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:17.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.403+0000 7f869539e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86900b6d20 0x7f86900afed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:17.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.403+0000 7f869539e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8690149cb0 con 0x7f86900b6d20 2022-06-29T15:50:17.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.403+0000 7f869539e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86900afab0 con 0x7f8690062f30 2022-06-29T15:50:17.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.403+0000 7f869539e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8690064f50 con 0x7f86900612b0 2022-06-29T15:50:17.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.403+0000 7f8694b9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86900b6d20 0x7f86900afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:17.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.403+0000 7f8694b9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86900b6d20 0x7f86900afed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52156/0 (socket says 172.21.15.47:52156) 2022-06-29T15:50:17.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.403+0000 7f8694b9d700 1 -- 172.21.15.47:0/1106075736 learned_addr learned my addr 172.21.15.47:0/1106075736 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:17.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.403+0000 7f868ffff700 1 --2- 172.21.15.47:0/1106075736 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f86900612b0 0x7f86900aca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:17.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.403+0000 7f86877fe700 1 --2- 172.21.15.47:0/1106075736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8690062f30 0x7f8690149340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:17.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.403+0000 7f8694b9d700 1 -- 172.21.15.47:0/1106075736 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f86900612b0 msgr2=0x7f86900aca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:17.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.403+0000 7f8694b9d700 1 --2- 172.21.15.47:0/1106075736 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f86900612b0 0x7f86900aca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:17.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.403+0000 7f8694b9d700 1 -- 172.21.15.47:0/1106075736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8690062f30 msgr2=0x7f8690149340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:17.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.403+0000 7f8694b9d700 1 --2- 172.21.15.47:0/1106075736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8690062f30 0x7f8690149340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:17.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.403+0000 7f8694b9d700 1 -- 172.21.15.47:0/1106075736 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8688009470 con 0x7f86900b6d20 2022-06-29T15:50:17.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.403+0000 7f86877fe700 1 --2- 172.21.15.47:0/1106075736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8690062f30 0x7f8690149340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:50:17.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.403+0000 7f868ffff700 1 --2- 172.21.15.47:0/1106075736 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f86900612b0 0x7f86900aca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:50:17.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.403+0000 7f8694b9d700 1 --2- 172.21.15.47:0/1106075736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86900b6d20 0x7f86900afed0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 rx=0x7f86880032f0 tx=0x7f8688016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:17.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.403+0000 7f868dffb700 1 -- 172.21.15.47:0/1106075736 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8688009ea0 con 0x7f86900b6d20 2022-06-29T15:50:17.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.407+0000 7f868dffb700 1 -- 172.21.15.47:0/1106075736 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f868801f5e0 con 0x7f86900b6d20 2022-06-29T15:50:17.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.407+0000 7f869539e700 1 -- 172.21.15.47:0/1106075736 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86900651f0 con 0x7f86900b6d20 2022-06-29T15:50:17.424 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.407+0000 7f868dffb700 1 -- 172.21.15.47:0/1106075736 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f868801f940 con 0x7f86900b6d20 2022-06-29T15:50:17.424 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.407+0000 7f869539e700 1 -- 172.21.15.47:0/1106075736 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86900b54d0 con 0x7f86900b6d20 2022-06-29T15:50:17.425 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.407+0000 7f869539e700 1 -- 172.21.15.47:0/1106075736 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8690149a10 con 0x7f86900b6d20 2022-06-29T15:50:17.426 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.407+0000 7f868dffb700 1 -- 172.21.15.47:0/1106075736 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8688026030 con 0x7f86900b6d20 2022-06-29T15:50:17.426 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.407+0000 7f868dffb700 1 --2- 172.21.15.47:0/1106075736 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8674053aa0 0x7f8674055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:17.426 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.407+0000 7f868ffff700 1 --2- 172.21.15.47:0/1106075736 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8674053aa0 0x7f8674055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:17.426 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.407+0000 7f868dffb700 1 -- 172.21.15.47:0/1106075736 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f868806cd80 con 0x7f86900b6d20 2022-06-29T15:50:17.427 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.411+0000 7f868ffff700 1 --2- 172.21.15.47:0/1106075736 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8674053aa0 0x7f8674055f60 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7f868000b000 tx=0x7f8680009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:17.427 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.415+0000 7f868dffb700 1 -- 172.21.15.47:0/1106075736 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8690149a10 con 0x7f86900b6d20 2022-06-29T15:50:17.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.583+0000 7f869539e700 1 -- 172.21.15.47:0/1106075736 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f8690149a10 con 0x7f86900b6d20 2022-06-29T15:50:17.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.591+0000 7f868dffb700 1 -- 172.21.15.47:0/1106075736 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v30) v1 ==== 184+0+0 (secure 0 0 0) 0x7f8690149a10 con 0x7f86900b6d20 2022-06-29T15:50:17.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.591+0000 7f869539e700 1 -- 172.21.15.47:0/1106075736 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8674053aa0 msgr2=0x7f8674055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:17.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.591+0000 7f869539e700 1 --2- 172.21.15.47:0/1106075736 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8674053aa0 0x7f8674055f60 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7f868000b000 tx=0x7f8680009040).stop 2022-06-29T15:50:17.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.591+0000 7f869539e700 1 -- 172.21.15.47:0/1106075736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86900b6d20 msgr2=0x7f86900afed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:17.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.591+0000 7f869539e700 1 --2- 172.21.15.47:0/1106075736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86900b6d20 0x7f86900afed0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 rx=0x7f86880032f0 tx=0x7f8688016000).stop 2022-06-29T15:50:17.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.591+0000 7f869539e700 1 -- 172.21.15.47:0/1106075736 shutdown_connections 2022-06-29T15:50:17.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.591+0000 7f869539e700 1 --2- 172.21.15.47:0/1106075736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86900b6d20 0x7f86900afed0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:17.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.591+0000 7f869539e700 1 --2- 172.21.15.47:0/1106075736 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8674053aa0 0x7f8674055f60 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:17.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.591+0000 7f869539e700 1 --2- 172.21.15.47:0/1106075736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8690062f30 0x7f8690149340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:17.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.591+0000 7f869539e700 1 --2- 172.21.15.47:0/1106075736 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f86900612b0 0x7f86900aca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:17.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.591+0000 7f869539e700 1 -- 172.21.15.47:0/1106075736 >> 172.21.15.47:0/1106075736 conn(0x7f8690063670 msgr2=0x7f8690064840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:17.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.591+0000 7f869539e700 1 -- 172.21.15.47:0/1106075736 shutdown_connections 2022-06-29T15:50:17.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.591+0000 7f869539e700 1 -- 172.21.15.47:0/1106075736 wait complete. 2022-06-29T15:50:17.602 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.1 2022-06-29T15:50:17.612 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:50:17.719 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.711+0000 7faa2f0f1700 1 Processor -- start 2022-06-29T15:50:17.719 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.711+0000 7faa2f0f1700 1 -- start start 2022-06-29T15:50:17.723 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.715+0000 7faa2f0f1700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7faa280b25c0 0x7faa280b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:17.723 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.715+0000 7faa2f0f1700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7faa280af440 0x7faa280af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:17.724 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.715+0000 7faa2f0f1700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa28146510 con 0x7faa280b1fc0 2022-06-29T15:50:17.724 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.715+0000 7faa2f0f1700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa28147430 con 0x7faa280b25c0 2022-06-29T15:50:17.724 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.715+0000 7faa2f0f1700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa281467b0 con 0x7faa280af440 2022-06-29T15:50:17.724 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.715+0000 7faa2e0ef700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7faa280b25c0 0x7faa280b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:17.725 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.715+0000 7faa2e0ef700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7faa280b25c0 0x7faa280b29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52162/0 (socket says 172.21.15.47:52162) 2022-06-29T15:50:17.725 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.715+0000 7faa2e0ef700 1 -- 172.21.15.47:0/2748511022 learned_addr learned my addr 172.21.15.47:0/2748511022 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:17.725 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.715+0000 7faa2e8f0700 1 --2- 172.21.15.47:0/2748511022 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7faa280af440 0x7faa280af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:17.725 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.715+0000 7faa2e0ef700 1 -- 172.21.15.47:0/2748511022 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7faa280af440 msgr2=0x7faa280af8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:17.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.715+0000 7faa2e0ef700 1 --2- 172.21.15.47:0/2748511022 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7faa280af440 0x7faa280af8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:17.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.715+0000 7faa2d0ed700 1 -- 172.21.15.47:0/2748511022 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2800594951 0 0) 0x7faa28146510 con 0x7faa280b1fc0 2022-06-29T15:50:17.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.715+0000 7faa2e0ef700 1 -- 172.21.15.47:0/2748511022 >> v1:172.21.15.64:6789/0 conn(0x7faa280b1fc0 legacy=0x7faa281403b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:17.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.715+0000 7faa2e0ef700 1 -- 172.21.15.47:0/2748511022 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa28146a50 con 0x7faa280b25c0 2022-06-29T15:50:17.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.715+0000 7faa2e0ef700 1 --2- 172.21.15.47:0/2748511022 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa280b25c0 0x7faa280b29a0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 rx=0x7faa24002f70 tx=0x7faa24006eb0).ready entity=mon.0 client_cookie=2c80a5d7b20de7bd server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:17.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.715+0000 7faa2d0ed700 1 -- 172.21.15.47:0/2748511022 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa2400d570 con 0x7faa280b25c0 2022-06-29T15:50:17.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.715+0000 7faa2d0ed700 1 -- 172.21.15.47:0/2748511022 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa24000e90 con 0x7faa280b25c0 2022-06-29T15:50:17.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.715+0000 7faa2d0ed700 1 -- 172.21.15.47:0/2748511022 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa2400d8c0 con 0x7faa280b25c0 2022-06-29T15:50:17.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.715+0000 7faa2f0f1700 1 -- 172.21.15.47:0/2748511022 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa280b25c0 msgr2=0x7faa280b29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:17.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.715+0000 7faa2f0f1700 1 --2- 172.21.15.47:0/2748511022 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa280b25c0 0x7faa280b29a0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 rx=0x7faa24002f70 tx=0x7faa24006eb0).stop 2022-06-29T15:50:17.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.715+0000 7faa2f0f1700 1 -- 172.21.15.47:0/2748511022 shutdown_connections 2022-06-29T15:50:17.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.715+0000 7faa2f0f1700 1 --2- 172.21.15.47:0/2748511022 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7faa280af440 0x7faa280af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:17.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.715+0000 7faa2f0f1700 1 --2- 172.21.15.47:0/2748511022 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa280b25c0 0x7faa280b29a0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:17.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.715+0000 7faa2f0f1700 1 -- 172.21.15.47:0/2748511022 >> 172.21.15.47:0/2748511022 conn(0x7faa28062df0 msgr2=0x7faa280631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:17.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.715+0000 7faa2f0f1700 1 -- 172.21.15.47:0/2748511022 shutdown_connections 2022-06-29T15:50:17.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.715+0000 7faa2f0f1700 1 -- 172.21.15.47:0/2748511022 wait complete. 2022-06-29T15:50:17.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.719+0000 7faa2f0f1700 1 Processor -- start 2022-06-29T15:50:17.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.719+0000 7faa2f0f1700 1 -- start start 2022-06-29T15:50:17.732 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.719+0000 7faa2f0f1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa280af440 0x7faa281438e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:17.732 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.719+0000 7faa2f0f1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa280b1fc0 0x7faa28155a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:17.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.719+0000 7faa2f0f1700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7faa280b25c0 0x7faa28155ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:17.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.719+0000 7faa2f0f1700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa280aa3d0 con 0x7faa280af440 2022-06-29T15:50:17.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.719+0000 7faa2f0f1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa280ab900 con 0x7faa280b1fc0 2022-06-29T15:50:17.734 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.719+0000 7faa2f0f1700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa280aa600 con 0x7faa280b25c0 2022-06-29T15:50:17.734 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.719+0000 7faa2e0ef700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa280af440 0x7faa281438e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:17.734 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.719+0000 7faa2e0ef700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa280af440 0x7faa281438e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52166/0 (socket says 172.21.15.47:52166) 2022-06-29T15:50:17.734 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.719+0000 7faa2e0ef700 1 -- 172.21.15.47:0/98081607 learned_addr learned my addr 172.21.15.47:0/98081607 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:17.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.719+0000 7faa2e8f0700 1 --2- 172.21.15.47:0/98081607 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7faa280b25c0 0x7faa28155ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:17.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.719+0000 7faa2d8ee700 1 --2- 172.21.15.47:0/98081607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa280b1fc0 0x7faa28155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:17.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.719+0000 7faa2e0ef700 1 -- 172.21.15.47:0/98081607 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7faa280b25c0 msgr2=0x7faa28155ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:17.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.719+0000 7faa2e0ef700 1 --2- 172.21.15.47:0/98081607 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7faa280b25c0 0x7faa28155ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:17.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.719+0000 7faa2e0ef700 1 -- 172.21.15.47:0/98081607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa280b1fc0 msgr2=0x7faa28155a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:17.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.719+0000 7faa2e0ef700 1 --2- 172.21.15.47:0/98081607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa280b1fc0 0x7faa28155a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:17.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.719+0000 7faa2e0ef700 1 -- 172.21.15.47:0/98081607 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa2400b1e0 con 0x7faa280af440 2022-06-29T15:50:17.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.719+0000 7faa2d8ee700 1 --2- 172.21.15.47:0/98081607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa280b1fc0 0x7faa28155a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:50:17.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.719+0000 7faa2e0ef700 1 --2- 172.21.15.47:0/98081607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa280af440 0x7faa281438e0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 rx=0x7faa24002f70 tx=0x7faa24015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:17.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.719+0000 7faa177fe700 1 -- 172.21.15.47:0/98081607 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa24011860 con 0x7faa280af440 2022-06-29T15:50:17.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.719+0000 7faa177fe700 1 -- 172.21.15.47:0/98081607 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa240119c0 con 0x7faa280af440 2022-06-29T15:50:17.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.719+0000 7faa177fe700 1 -- 172.21.15.47:0/98081607 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa24011d20 con 0x7faa280af440 2022-06-29T15:50:17.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.719+0000 7faa2f0f1700 1 -- 172.21.15.47:0/98081607 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa280aa8a0 con 0x7faa280af440 2022-06-29T15:50:17.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.719+0000 7faa2f0f1700 1 -- 172.21.15.47:0/98081607 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa280aade0 con 0x7faa280af440 2022-06-29T15:50:17.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.723+0000 7faa2f0f1700 1 -- 172.21.15.47:0/98081607 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa280aafa0 con 0x7faa280af440 2022-06-29T15:50:17.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.723+0000 7faa177fe700 1 -- 172.21.15.47:0/98081607 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7faa2401a040 con 0x7faa280af440 2022-06-29T15:50:17.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.723+0000 7faa177fe700 1 --2- 172.21.15.47:0/98081607 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7faa080539e0 0x7faa08055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:17.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.723+0000 7faa2d8ee700 1 --2- 172.21.15.47:0/98081607 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7faa080539e0 0x7faa08055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:17.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.723+0000 7faa177fe700 1 -- 172.21.15.47:0/98081607 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6420+0+0 (secure 0 0 0) 0x7faa2406cfc0 con 0x7faa280af440 2022-06-29T15:50:17.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.723+0000 7faa2d8ee700 1 --2- 172.21.15.47:0/98081607 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7faa080539e0 0x7faa08055ea0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7faa1800b000 tx=0x7faa18009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:17.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.731+0000 7faa177fe700 1 -- 172.21.15.47:0/98081607 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7faa24043c70 con 0x7faa280af440 2022-06-29T15:50:17.900 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.895+0000 7faa2f0f1700 1 -- 172.21.15.47:0/98081607 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa280573d0 con 0x7faa280af440 2022-06-29T15:50:17.901 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.895+0000 7faa177fe700 1 -- 172.21.15.47:0/98081607 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+2496 (secure 0 0 0) 0x7faa24023090 con 0x7faa280af440 2022-06-29T15:50:17.902 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:50:17.902 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":51,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3752554010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3752554010},{"type":"v1","addr":"172.21.15.47:6835","nonce":3752554010}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":48},{"gid":7293,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.47:6837/3263659716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":3263659716},{"type":"v1","addr":"172.21.15.47:6837","nonce":3263659716}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":49}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:50:15.338114+0000","modified":"2022-06-29T15:50:16.342008+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":56354},"failed":[],"damaged":[],"stopped":[],"info":{"gid_56354":{"gid":56354,"name":"b","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6833/2706285595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2706285595},{"type":"v1","addr":"172.21.15.64:6833","nonce":2706285595}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2022-06-29T15:50:17.903 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.895+0000 7faa2f0f1700 1 -- 172.21.15.47:0/98081607 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7faa080539e0 msgr2=0x7faa08055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:17.904 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.895+0000 7faa2f0f1700 1 --2- 172.21.15.47:0/98081607 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7faa080539e0 0x7faa08055ea0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7faa1800b000 tx=0x7faa18009040).stop 2022-06-29T15:50:17.904 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.895+0000 7faa2f0f1700 1 -- 172.21.15.47:0/98081607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa280af440 msgr2=0x7faa281438e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:17.904 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.895+0000 7faa2f0f1700 1 --2- 172.21.15.47:0/98081607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa280af440 0x7faa281438e0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 rx=0x7faa24002f70 tx=0x7faa24015000).stop 2022-06-29T15:50:17.904 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.899+0000 7faa2f0f1700 1 -- 172.21.15.47:0/98081607 shutdown_connections 2022-06-29T15:50:17.905 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.899+0000 7faa2f0f1700 1 --2- 172.21.15.47:0/98081607 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7faa280b25c0 0x7faa28155ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:17.905 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.899+0000 7faa2f0f1700 1 --2- 172.21.15.47:0/98081607 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7faa080539e0 0x7faa08055ea0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:17.905 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.899+0000 7faa2f0f1700 1 --2- 172.21.15.47:0/98081607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa280b1fc0 0x7faa28155a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:17.905 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.899+0000 7faa2f0f1700 1 --2- 172.21.15.47:0/98081607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa280af440 0x7faa281438e0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:17.906 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.899+0000 7faa2f0f1700 1 -- 172.21.15.47:0/98081607 >> 172.21.15.47:0/98081607 conn(0x7faa28062df0 msgr2=0x7faa2805fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:17.906 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.899+0000 7faa2f0f1700 1 -- 172.21.15.47:0/98081607 shutdown_connections 2022-06-29T15:50:17.906 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:17.899+0000 7faa2f0f1700 1 -- 172.21.15.47:0/98081607 wait complete. 2022-06-29T15:50:17.907 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 51 2022-06-29T15:50:18.217 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 51, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-29T15:50:15.338114+0000', 'modified': '2022-06-29T15:50:16.342008+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 56354}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_56354': {'gid': 56354, 'name': 'b', 'rank': 0, 'incarnation': 50, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.64:6833/2706285595', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6832', 'nonce': 2706285595}, {'type': 'v1', 'addr': '172.21.15.64:6833', 'nonce': 2706285595}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-29T15:50:18.218 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-29T15:50:18.218 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-06-29T15:50:18.337 INFO:teuthology.orchestra.run.smithi064.stdout:{"cluster_fsid":"9a68894a-53ce-400e-838b-14eab91b834c","whoami":0,"id":56354,"want_state":"up:active","state":"up:active","rank_uptime":2.9806466039999999,"mdsmap_epoch":51,"osdmap_epoch":111,"osdmap_epoch_barrier":110,"uptime":7.3439159390000004} 2022-06-29T15:50:18.337 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9a68894a-53ce-400e-838b-14eab91b834c", "id": 56354, "mdsmap_epoch": 51, "osdmap_epoch": 111, "osdmap_epoch_barrier": 110, "rank_uptime": 2.980646604, "state": "up:active", "uptime": 7.343915939, "want_state": "up:active", "whoami": 0 } 2022-06-29T15:50:18.339 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:50:18.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.451+0000 7f5bd1e6c700 1 Processor -- start 2022-06-29T15:50:18.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.451+0000 7f5bd1e6c700 1 -- start start 2022-06-29T15:50:18.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.451+0000 7f5bd1e6c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5bcc0b2460 0x7f5bcc0ac990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:18.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.451+0000 7f5bd1e6c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5bcc0b16e0 0x7f5bcc144dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:18.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.451+0000 7f5bd1e6c700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bcc0b2840 con 0x7f5bcc140e10 2022-06-29T15:50:18.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.451+0000 7f5bd1e6c700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bcc0af0d0 con 0x7f5bcc0b2460 2022-06-29T15:50:18.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.451+0000 7f5bd1e6c700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bcc1452b0 con 0x7f5bcc0b16e0 2022-06-29T15:50:18.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.451+0000 7f5bcb7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5bcc0b2460 0x7f5bcc0ac990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:18.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.451+0000 7f5bcaffd700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5bcc0b16e0 0x7f5bcc144dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:18.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.451+0000 7f5bcb7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5bcc0b2460 0x7f5bcc0ac990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52176/0 (socket says 172.21.15.47:52176) 2022-06-29T15:50:18.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.451+0000 7f5bcb7fe700 1 -- 172.21.15.47:0/2784220229 learned_addr learned my addr 172.21.15.47:0/2784220229 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:18.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.451+0000 7f5bca7fc700 1 -- 172.21.15.47:0/2784220229 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2883737579 0 0) 0x7f5bcc0b2840 con 0x7f5bcc140e10 2022-06-29T15:50:18.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.451+0000 7f5bca7fc700 1 -- 172.21.15.47:0/2784220229 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5bac003260 con 0x7f5bcc140e10 2022-06-29T15:50:18.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.451+0000 7f5bcaffd700 1 -- 172.21.15.47:0/2784220229 >> v2:172.21.15.47:3300/0 conn(0x7f5bcc0b2460 msgr2=0x7f5bcc0ac990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:18.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.451+0000 7f5bcaffd700 1 --2- 172.21.15.47:0/2784220229 >> v2:172.21.15.47:3300/0 conn(0x7f5bcc0b2460 0x7f5bcc0ac990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:18.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.451+0000 7f5bcaffd700 1 -- 172.21.15.47:0/2784220229 >> v1:172.21.15.64:6789/0 conn(0x7f5bcc140e10 legacy=0x7f5bcc0abd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:18.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.451+0000 7f5bcaffd700 1 -- 172.21.15.47:0/2784220229 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bcc0ad030 con 0x7f5bcc0b16e0 2022-06-29T15:50:18.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.451+0000 7f5bcb7fe700 1 --2- 172.21.15.47:0/2784220229 >> v2:172.21.15.47:3300/0 conn(0x7f5bcc0b2460 0x7f5bcc0ac990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:50:18.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.451+0000 7f5bcaffd700 1 --2- 172.21.15.47:0/2784220229 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5bcc0b16e0 0x7f5bcc144dc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f5bb4002f70 tx=0x7f5bb4006df0).ready entity=mon.2 client_cookie=2dde4c420348836d server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:18.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.451+0000 7f5bca7fc700 1 -- 172.21.15.47:0/2784220229 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5bb400d4a0 con 0x7f5bcc0b16e0 2022-06-29T15:50:18.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.451+0000 7f5bca7fc700 1 -- 172.21.15.47:0/2784220229 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb4000e90 con 0x7f5bcc0b16e0 2022-06-29T15:50:18.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.451+0000 7f5bca7fc700 1 -- 172.21.15.47:0/2784220229 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5bb400d7f0 con 0x7f5bcc0b16e0 2022-06-29T15:50:18.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.455+0000 7f5bd1e6c700 1 -- 172.21.15.47:0/2784220229 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5bcc0b16e0 msgr2=0x7f5bcc144dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:18.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.455+0000 7f5bd1e6c700 1 --2- 172.21.15.47:0/2784220229 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5bcc0b16e0 0x7f5bcc144dc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f5bb4002f70 tx=0x7f5bb4006df0).stop 2022-06-29T15:50:18.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.455+0000 7f5bd1e6c700 1 -- 172.21.15.47:0/2784220229 shutdown_connections 2022-06-29T15:50:18.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.455+0000 7f5bd1e6c700 1 --2- 172.21.15.47:0/2784220229 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5bcc0b16e0 0x7f5bcc144dc0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:18.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.455+0000 7f5bd1e6c700 1 --2- 172.21.15.47:0/2784220229 >> v2:172.21.15.47:3300/0 conn(0x7f5bcc0b2460 0x7f5bcc0ac990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:18.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.455+0000 7f5bd1e6c700 1 -- 172.21.15.47:0/2784220229 >> 172.21.15.47:0/2784220229 conn(0x7f5bcc0626f0 msgr2=0x7f5bcc062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:18.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.455+0000 7f5bd1e6c700 1 -- 172.21.15.47:0/2784220229 shutdown_connections 2022-06-29T15:50:18.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.455+0000 7f5bd1e6c700 1 -- 172.21.15.47:0/2784220229 wait complete. 2022-06-29T15:50:18.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.455+0000 7f5bd1e6c700 1 Processor -- start 2022-06-29T15:50:18.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.455+0000 7f5bd1e6c700 1 -- start start 2022-06-29T15:50:18.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.455+0000 7f5bd1e6c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5bcc0b16e0 0x7f5bcc0650f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:18.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.455+0000 7f5bd1e6c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5bcc0b2460 0x7f5bcc0655e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:18.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.455+0000 7f5bd1e6c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5bcc140e10 0x7f5bcc0600c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:18.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.455+0000 7f5bd1e6c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bcc065ad0 con 0x7f5bcc0b2460 2022-06-29T15:50:18.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.455+0000 7f5bd1e6c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bcc153040 con 0x7f5bcc0b16e0 2022-06-29T15:50:18.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.455+0000 7f5bd1e6c700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bcc13b780 con 0x7f5bcc140e10 2022-06-29T15:50:18.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.455+0000 7f5bcaffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5bcc0b2460 0x7f5bcc0655e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:18.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.455+0000 7f5bcaffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5bcc0b2460 0x7f5bcc0655e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52182/0 (socket says 172.21.15.47:52182) 2022-06-29T15:50:18.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.455+0000 7f5bcaffd700 1 -- 172.21.15.47:0/1994278360 learned_addr learned my addr 172.21.15.47:0/1994278360 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:18.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.455+0000 7f5bcb7fe700 1 --2- 172.21.15.47:0/1994278360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5bcc0b16e0 0x7f5bcc0650f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:18.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.455+0000 7f5bcbfff700 1 --2- 172.21.15.47:0/1994278360 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5bcc140e10 0x7f5bcc0600c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:18.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.459+0000 7f5bcaffd700 1 -- 172.21.15.47:0/1994278360 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5bcc140e10 msgr2=0x7f5bcc0600c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:18.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.459+0000 7f5bcaffd700 1 --2- 172.21.15.47:0/1994278360 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5bcc140e10 0x7f5bcc0600c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:18.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.459+0000 7f5bcaffd700 1 -- 172.21.15.47:0/1994278360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5bcc0b16e0 msgr2=0x7f5bcc0650f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:18.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.459+0000 7f5bcaffd700 1 --2- 172.21.15.47:0/1994278360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5bcc0b16e0 0x7f5bcc0650f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:18.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.459+0000 7f5bcaffd700 1 -- 172.21.15.47:0/1994278360 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bbc00b1e0 con 0x7f5bcc0b2460 2022-06-29T15:50:18.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.459+0000 7f5bcaffd700 1 --2- 172.21.15.47:0/1994278360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5bcc0b2460 0x7f5bcc0655e0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 rx=0x7f5bcc13b910 tx=0x7f5bb4014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:18.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.459+0000 7f5bc8ff9700 1 -- 172.21.15.47:0/1994278360 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5bb4016070 con 0x7f5bcc0b2460 2022-06-29T15:50:18.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.459+0000 7f5bc8ff9700 1 -- 172.21.15.47:0/1994278360 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb40138d0 con 0x7f5bcc0b2460 2022-06-29T15:50:18.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.459+0000 7f5bd1e6c700 1 -- 172.21.15.47:0/1994278360 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bb400b180 con 0x7f5bcc0b2460 2022-06-29T15:50:18.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.459+0000 7f5bc8ff9700 1 -- 172.21.15.47:0/1994278360 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5bb4013c30 con 0x7f5bcc0b2460 2022-06-29T15:50:18.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.459+0000 7f5bd1e6c700 1 -- 172.21.15.47:0/1994278360 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bcc1532a0 con 0x7f5bcc0b2460 2022-06-29T15:50:18.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.459+0000 7f5bd1e6c700 1 -- 172.21.15.47:0/1994278360 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bcc13e500 con 0x7f5bcc0b2460 2022-06-29T15:50:18.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.459+0000 7f5bc8ff9700 1 -- 172.21.15.47:0/1994278360 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5bb4025030 con 0x7f5bcc0b2460 2022-06-29T15:50:18.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.463+0000 7f5bc8ff9700 1 --2- 172.21.15.47:0/1994278360 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5bb0053950 0x7f5bb0055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:18.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.463+0000 7f5bc8ff9700 1 -- 172.21.15.47:0/1994278360 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f5bb406d200 con 0x7f5bcc0b2460 2022-06-29T15:50:18.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.463+0000 7f5bcb7fe700 1 --2- 172.21.15.47:0/1994278360 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5bb0053950 0x7f5bb0055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:18.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.463+0000 7f5bcb7fe700 1 --2- 172.21.15.47:0/1994278360 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5bb0053950 0x7f5bb0055e10 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7f5bbc00b000 tx=0x7f5bbc006eb0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:18.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.467+0000 7f5bc8ff9700 1 -- 172.21.15.47:0/1994278360 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5bcc13e500 con 0x7f5bcc0b2460 2022-06-29T15:50:18.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.591+0000 7f5bd1e6c700 1 -- 172.21.15.47:0/1994278360 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5bcc13e500 con 0x7f5bcc0b2460 2022-06-29T15:50:18.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.595+0000 7f5bc8ff9700 1 -- 172.21.15.47:0/1994278360 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f5bcc13e500 con 0x7f5bcc0b2460 2022-06-29T15:50:18.600 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:50:18.601 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":111,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:50:16.342650+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:50:13.772527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2022-06-29T15:50:14.675947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:50:18.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.595+0000 7f5bd1e6c700 1 -- 172.21.15.47:0/1994278360 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5bb0053950 msgr2=0x7f5bb0055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:18.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.595+0000 7f5bd1e6c700 1 --2- 172.21.15.47:0/1994278360 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5bb0053950 0x7f5bb0055e10 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7f5bbc00b000 tx=0x7f5bbc006eb0).stop 2022-06-29T15:50:18.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.595+0000 7f5bd1e6c700 1 -- 172.21.15.47:0/1994278360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5bcc0b2460 msgr2=0x7f5bcc0655e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:18.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.595+0000 7f5bd1e6c700 1 --2- 172.21.15.47:0/1994278360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5bcc0b2460 0x7f5bcc0655e0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 rx=0x7f5bcc13b910 tx=0x7f5bb4014000).stop 2022-06-29T15:50:18.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.595+0000 7f5bd1e6c700 1 -- 172.21.15.47:0/1994278360 shutdown_connections 2022-06-29T15:50:18.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.595+0000 7f5bd1e6c700 1 --2- 172.21.15.47:0/1994278360 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5bcc140e10 0x7f5bcc0600c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:18.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.595+0000 7f5bd1e6c700 1 --2- 172.21.15.47:0/1994278360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5bcc0b2460 0x7f5bcc0655e0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:18.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.595+0000 7f5bd1e6c700 1 --2- 172.21.15.47:0/1994278360 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5bb0053950 0x7f5bb0055e10 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:18.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.595+0000 7f5bd1e6c700 1 --2- 172.21.15.47:0/1994278360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5bcc0b16e0 0x7f5bcc0650f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:18.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.595+0000 7f5bd1e6c700 1 -- 172.21.15.47:0/1994278360 >> 172.21.15.47:0/1994278360 conn(0x7f5bcc0626f0 msgr2=0x7f5bcc13aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:18.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.595+0000 7f5bd1e6c700 1 -- 172.21.15.47:0/1994278360 shutdown_connections 2022-06-29T15:50:18.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.595+0000 7f5bd1e6c700 1 -- 172.21.15.47:0/1994278360 wait complete. 2022-06-29T15:50:18.613 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-29T15:42:16.971431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-29T15:42:25.290884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-29T15:50:13.772527+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2022-06-29T15:50:14.675947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-29T15:50:18.614 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-29T15:50:18.724 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.719+0000 7f854f59e700 1 Processor -- start 2022-06-29T15:50:18.724 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.719+0000 7f854f59e700 1 -- start start 2022-06-29T15:50:18.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.719+0000 7f854f59e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f855013cee0 0x7f8550060890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:18.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.719+0000 7f854f59e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85500ac990 con 0x7f8550060e40 2022-06-29T15:50:18.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.719+0000 7f854f59e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85500b0c70 con 0x7f85500b0890 2022-06-29T15:50:18.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.719+0000 7f854f59e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85500ad7a0 con 0x7f855013cee0 2022-06-29T15:50:18.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.719+0000 7f854dd9b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f8550060e40 0x7f855013f560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37742/0 (socket says 172.21.15.47:37742) 2022-06-29T15:50:18.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.719+0000 7f854dd9b700 1 -- 172.21.15.47:0/1442428825 learned_addr learned my addr 172.21.15.47:0/1442428825 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:18.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.719+0000 7f854ed9d700 1 --2- 172.21.15.47:0/1442428825 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f855013cee0 0x7f8550060890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:18.731 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.719+0000 7f854d59a700 1 -- 172.21.15.47:0/1442428825 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 997849679 0 0) 0x7f85500ac990 con 0x7f8550060e40 2022-06-29T15:50:18.731 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.719+0000 7f854d59a700 1 -- 172.21.15.47:0/1442428825 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8534003260 con 0x7f8550060e40 2022-06-29T15:50:18.731 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.719+0000 7f854d59a700 1 -- 172.21.15.47:0/1442428825 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4251979856 0 0) 0x7f85500b0c70 con 0x7f85500b0890 2022-06-29T15:50:18.731 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.719+0000 7f854d59a700 1 -- 172.21.15.47:0/1442428825 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85500ac990 con 0x7f85500b0890 2022-06-29T15:50:18.732 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.719+0000 7f854d59a700 1 -- 172.21.15.47:0/1442428825 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2479549308 0 0) 0x7f8534003260 con 0x7f8550060e40 2022-06-29T15:50:18.732 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.719+0000 7f854d59a700 1 -- 172.21.15.47:0/1442428825 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f855013cee0 msgr2=0x7f8550060890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:18.732 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.719+0000 7f854d59a700 1 --2- 172.21.15.47:0/1442428825 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f855013cee0 0x7f8550060890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:18.732 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.719+0000 7f854d59a700 1 -- 172.21.15.47:0/1442428825 >> v1:172.21.15.64:6789/0 conn(0x7f85500b0890 legacy=0x7f855013c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:18.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.719+0000 7f854d59a700 1 -- 172.21.15.47:0/1442428825 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85500aefa0 con 0x7f8550060e40 2022-06-29T15:50:18.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.719+0000 7f854d59a700 1 -- 172.21.15.47:0/1442428825 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7f8544001660 con 0x7f8550060e40 2022-06-29T15:50:18.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.719+0000 7f854d59a700 1 -- 172.21.15.47:0/1442428825 >> v1:172.21.15.47:6789/0 conn(0x7f8550060e40 legacy=0x7f855013f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:18.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.719+0000 7f854d59a700 1 --2- 172.21.15.47:0/1442428825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8534003910 0x7f8534003d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:18.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.719+0000 7f854d59a700 1 --2- 172.21.15.47:0/1442428825 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f855013cee0 0x7f8534004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:18.734 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.719+0000 7f854d59a700 1 --2- 172.21.15.47:0/1442428825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85340049f0 0x7f8534004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:18.734 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.719+0000 7f854d59a700 1 -- 172.21.15.47:0/1442428825 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85500ad7a0 con 0x7f8534003910 2022-06-29T15:50:18.734 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.719+0000 7f854d59a700 1 -- 172.21.15.47:0/1442428825 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853400b800 con 0x7f85340049f0 2022-06-29T15:50:18.734 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.719+0000 7f854d59a700 1 -- 172.21.15.47:0/1442428825 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f853400b970 con 0x7f855013cee0 2022-06-29T15:50:18.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.719+0000 7f854e59c700 1 --2- 172.21.15.47:0/1442428825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8534003910 0x7f8534003d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:18.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.719+0000 7f854ed9d700 1 --2- 172.21.15.47:0/1442428825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85340049f0 0x7f8534004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:18.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.719+0000 7f854dd9b700 1 --2- 172.21.15.47:0/1442428825 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f855013cee0 0x7f8534004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:18.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.719+0000 7f854e59c700 1 -- 172.21.15.47:0/1442428825 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f855013cee0 msgr2=0x7f8534004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:18.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.719+0000 7f854e59c700 1 --2- 172.21.15.47:0/1442428825 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f855013cee0 0x7f8534004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:18.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.719+0000 7f854e59c700 1 -- 172.21.15.47:0/1442428825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85340049f0 msgr2=0x7f8534004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:18.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.719+0000 7f854e59c700 1 --2- 172.21.15.47:0/1442428825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85340049f0 0x7f8534004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:18.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.719+0000 7f854e59c700 1 -- 172.21.15.47:0/1442428825 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8534003260 con 0x7f8534003910 2022-06-29T15:50:18.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.719+0000 7f854ed9d700 1 --2- 172.21.15.47:0/1442428825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85340049f0 0x7f8534004e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:50:18.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.719+0000 7f854e59c700 1 --2- 172.21.15.47:0/1442428825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8534003910 0x7f8534003d10 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 rx=0x7f8540001110 tx=0x7f8540007a00).ready entity=mon.0 client_cookie=cf446044e70a880a server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:18.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.723+0000 7f854d59a700 1 -- 172.21.15.47:0/1442428825 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8540007d10 con 0x7f8534003910 2022-06-29T15:50:18.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.723+0000 7f854d59a700 1 -- 172.21.15.47:0/1442428825 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8540007030 con 0x7f8534003910 2022-06-29T15:50:18.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.723+0000 7f854d59a700 1 -- 172.21.15.47:0/1442428825 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8540007390 con 0x7f8534003910 2022-06-29T15:50:18.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.723+0000 7f854f59e700 1 -- 172.21.15.47:0/1442428825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8534003910 msgr2=0x7f8534003d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:18.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.723+0000 7f854f59e700 1 --2- 172.21.15.47:0/1442428825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8534003910 0x7f8534003d10 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 rx=0x7f8540001110 tx=0x7f8540007a00).stop 2022-06-29T15:50:18.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.723+0000 7f854ed9d700 1 -- 172.21.15.47:0/1442428825 reap_dead start 2022-06-29T15:50:18.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.723+0000 7f854f59e700 1 -- 172.21.15.47:0/1442428825 shutdown_connections 2022-06-29T15:50:18.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.723+0000 7f854f59e700 1 -- 172.21.15.47:0/1442428825 >> 172.21.15.47:0/1442428825 conn(0x7f85500b5320 msgr2=0x7f85500b5700 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:18.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.723+0000 7f854f59e700 1 -- 172.21.15.47:0/1442428825 shutdown_connections 2022-06-29T15:50:18.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.723+0000 7f854f59e700 1 -- 172.21.15.47:0/1442428825 wait complete. 2022-06-29T15:50:18.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.723+0000 7f854f59e700 1 Processor -- start 2022-06-29T15:50:18.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.723+0000 7f854f59e700 1 -- start start 2022-06-29T15:50:18.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.723+0000 7f854f59e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8534003910 0x7f8550140990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:18.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.723+0000 7f854f59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85500ac040 0x7f855013f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:18.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.723+0000 7f854f59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f855013bb50 0x7f85500a9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:18.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.723+0000 7f854f59e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8550153170 con 0x7f85500ac040 2022-06-29T15:50:18.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.723+0000 7f854f59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855013fcd0 con 0x7f855013bb50 2022-06-29T15:50:18.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.723+0000 7f854f59e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f855013ff70 con 0x7f8534003910 2022-06-29T15:50:18.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.723+0000 7f854dd9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85500ac040 0x7f855013f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:18.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.723+0000 7f854dd9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85500ac040 0x7f855013f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52202/0 (socket says 172.21.15.47:52202) 2022-06-29T15:50:18.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.723+0000 7f854dd9b700 1 -- 172.21.15.47:0/3320691673 learned_addr learned my addr 172.21.15.47:0/3320691673 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:18.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.727+0000 7f854ed9d700 1 --2- 172.21.15.47:0/3320691673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f855013bb50 0x7f85500a9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:18.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.723+0000 7f854e59c700 1 --2- 172.21.15.47:0/3320691673 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8534003910 0x7f8550140990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:18.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.727+0000 7f854dd9b700 1 -- 172.21.15.47:0/3320691673 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8534003910 msgr2=0x7f8550140990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:18.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.727+0000 7f854dd9b700 1 --2- 172.21.15.47:0/3320691673 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8534003910 0x7f8550140990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:18.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.727+0000 7f854dd9b700 1 -- 172.21.15.47:0/3320691673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f855013bb50 msgr2=0x7f85500a9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:18.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.727+0000 7f854dd9b700 1 --2- 172.21.15.47:0/3320691673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f855013bb50 0x7f85500a9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:18.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.727+0000 7f854dd9b700 1 -- 172.21.15.47:0/3320691673 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8550152dc0 con 0x7f85500ac040 2022-06-29T15:50:18.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.727+0000 7f854ed9d700 1 --2- 172.21.15.47:0/3320691673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f855013bb50 0x7f85500a9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:50:18.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.727+0000 7f854dd9b700 1 --2- 172.21.15.47:0/3320691673 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85500ac040 0x7f855013f320 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 rx=0x7f854400d000 tx=0x7f85500ae8e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:18.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.727+0000 7f853f7fe700 1 -- 172.21.15.47:0/3320691673 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85440116b0 con 0x7f85500ac040 2022-06-29T15:50:18.748 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.727+0000 7f853f7fe700 1 -- 172.21.15.47:0/3320691673 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8544011810 con 0x7f85500ac040 2022-06-29T15:50:18.748 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.727+0000 7f854f59e700 1 -- 172.21.15.47:0/3320691673 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85500aa1a0 con 0x7f85500ac040 2022-06-29T15:50:18.748 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.727+0000 7f853f7fe700 1 -- 172.21.15.47:0/3320691673 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8544011b70 con 0x7f85500ac040 2022-06-29T15:50:18.748 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.727+0000 7f854f59e700 1 -- 172.21.15.47:0/3320691673 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85501430f0 con 0x7f85500ac040 2022-06-29T15:50:18.749 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.727+0000 7f854f59e700 1 -- 172.21.15.47:0/3320691673 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f855013bf30 con 0x7f85500ac040 2022-06-29T15:50:18.749 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.727+0000 7f853f7fe700 1 -- 172.21.15.47:0/3320691673 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f854401b030 con 0x7f85500ac040 2022-06-29T15:50:18.749 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.727+0000 7f853f7fe700 1 --2- 172.21.15.47:0/3320691673 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f852c053b30 0x7f852c055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:18.749 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.731+0000 7f853f7fe700 1 -- 172.21.15.47:0/3320691673 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f854406b7f0 con 0x7f85500ac040 2022-06-29T15:50:18.749 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.731+0000 7f854e59c700 1 --2- 172.21.15.47:0/3320691673 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f852c053b30 0x7f852c055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:18.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.731+0000 7f854e59c700 1 --2- 172.21.15.47:0/3320691673 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f852c053b30 0x7f852c055ff0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7f854001a000 tx=0x7f8540016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:18.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.735+0000 7f853f7fe700 1 -- 172.21.15.47:0/3320691673 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f85440425b0 con 0x7f85500ac040 2022-06-29T15:50:18.867 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.859+0000 7f854f59e700 1 -- 172.21.15.47:0/3320691673 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f85500573d0 con 0x7f85500ac040 2022-06-29T15:50:18.868 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.863+0000 7f853f7fe700 1 -- 172.21.15.47:0/3320691673 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v111) v1 ==== 110+0+10 (secure 0 0 0) 0x7f8544019030 con 0x7f85500ac040 2022-06-29T15:50:18.868 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2022-06-29T15:50:18.872 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.863+0000 7f854f59e700 1 -- 172.21.15.47:0/3320691673 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f852c053b30 msgr2=0x7f852c055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:18.872 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.863+0000 7f854f59e700 1 --2- 172.21.15.47:0/3320691673 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f852c053b30 0x7f852c055ff0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7f854001a000 tx=0x7f8540016000).stop 2022-06-29T15:50:18.872 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.863+0000 7f854f59e700 1 -- 172.21.15.47:0/3320691673 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85500ac040 msgr2=0x7f855013f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:18.873 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.863+0000 7f854f59e700 1 --2- 172.21.15.47:0/3320691673 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85500ac040 0x7f855013f320 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 rx=0x7f854400d000 tx=0x7f85500ae8e0).stop 2022-06-29T15:50:18.873 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.867+0000 7f854f59e700 1 -- 172.21.15.47:0/3320691673 shutdown_connections 2022-06-29T15:50:18.873 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.867+0000 7f854f59e700 1 --2- 172.21.15.47:0/3320691673 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f852c053b30 0x7f852c055ff0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:18.873 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.867+0000 7f854f59e700 1 --2- 172.21.15.47:0/3320691673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f855013bb50 0x7f85500a9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:18.874 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.867+0000 7f854f59e700 1 --2- 172.21.15.47:0/3320691673 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85500ac040 0x7f855013f320 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:18.874 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.867+0000 7f854f59e700 1 --2- 172.21.15.47:0/3320691673 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8534003910 0x7f8550140990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:18.874 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.867+0000 7f854f59e700 1 -- 172.21.15.47:0/3320691673 >> 172.21.15.47:0/3320691673 conn(0x7f85500b5320 msgr2=0x7f85500b5700 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:18.875 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.867+0000 7f854f59e700 1 -- 172.21.15.47:0/3320691673 shutdown_connections 2022-06-29T15:50:18.875 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:18.867+0000 7f854f59e700 1 -- 172.21.15.47:0/3320691673 wait complete. 2022-06-29T15:50:18.889 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-29T15:50:19.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.043+0000 7f4deab02700 1 Processor -- start 2022-06-29T15:50:19.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.043+0000 7f4deab02700 1 -- start start 2022-06-29T15:50:19.052 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.043+0000 7f4deab02700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4de40a9770 0x7f4de40b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:19.052 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.043+0000 7f4deab02700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4de40b1ac0 con 0x7f4de4095e30 2022-06-29T15:50:19.052 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.043+0000 7f4deab02700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4de4095c70 con 0x7f4de40b2930 2022-06-29T15:50:19.053 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.043+0000 7f4deab02700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4de40965b0 con 0x7f4de40a9770 2022-06-29T15:50:19.053 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.043+0000 7f4de909f700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4de4095e30 0x7f4de4149760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37766/0 (socket says 172.21.15.47:37766) 2022-06-29T15:50:19.053 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.043+0000 7f4de909f700 1 -- 172.21.15.47:0/2808259531 learned_addr learned my addr 172.21.15.47:0/2808259531 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:19.053 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.043+0000 7f4de3fff700 1 --2- 172.21.15.47:0/2808259531 >> v2:172.21.15.64:3300/0 conn(0x7f4de40a9770 0x7f4de40b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:19.054 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.043+0000 7f4de37fe700 1 -- 172.21.15.47:0/2808259531 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1323636395 0 0) 0x7f4de40b1ac0 con 0x7f4de4095e30 2022-06-29T15:50:19.054 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.043+0000 7f4de37fe700 1 -- 172.21.15.47:0/2808259531 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4dc4003260 con 0x7f4de4095e30 2022-06-29T15:50:19.054 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.043+0000 7f4de37fe700 1 -- 172.21.15.47:0/2808259531 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 700673088 0 0) 0x7f4dc4003260 con 0x7f4de4095e30 2022-06-29T15:50:19.054 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.043+0000 7f4de37fe700 1 -- 172.21.15.47:0/2808259531 >> v2:172.21.15.64:3300/0 conn(0x7f4de40a9770 msgr2=0x7f4de40b2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:19.055 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.043+0000 7f4de37fe700 1 --2- 172.21.15.47:0/2808259531 >> v2:172.21.15.64:3300/0 conn(0x7f4de40a9770 0x7f4de40b2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:19.055 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.043+0000 7f4de37fe700 1 -- 172.21.15.47:0/2808259531 >> v1:172.21.15.64:6789/0 conn(0x7f4de40b2930 legacy=0x7f4de4097a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:19.055 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.043+0000 7f4de37fe700 1 -- 172.21.15.47:0/2808259531 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4de40b1cf0 con 0x7f4de4095e30 2022-06-29T15:50:19.055 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.043+0000 7f4de37fe700 1 -- 172.21.15.47:0/2808259531 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7f4ddc001660 con 0x7f4de4095e30 2022-06-29T15:50:19.056 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.043+0000 7f4de37fe700 1 -- 172.21.15.47:0/2808259531 >> v1:172.21.15.47:6789/0 conn(0x7f4de4095e30 legacy=0x7f4de4149760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:19.056 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.043+0000 7f4de37fe700 1 --2- 172.21.15.47:0/2808259531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4dc4003750 0x7f4dc4003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:19.056 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.043+0000 7f4de37fe700 1 --2- 172.21.15.47:0/2808259531 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4dc4004500 0x7f4dc4004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:19.056 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.043+0000 7f4de37fe700 1 --2- 172.21.15.47:0/2808259531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4dc4004e70 0x7f4dc400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:19.057 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.043+0000 7f4de37fe700 1 -- 172.21.15.47:0/2808259531 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de40965b0 con 0x7f4dc4004e70 2022-06-29T15:50:19.057 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.043+0000 7f4de37fe700 1 -- 172.21.15.47:0/2808259531 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc400bd20 con 0x7f4dc4003750 2022-06-29T15:50:19.057 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.043+0000 7f4de37fe700 1 -- 172.21.15.47:0/2808259531 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc400bee0 con 0x7f4dc4004500 2022-06-29T15:50:19.057 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.043+0000 7f4de909f700 1 --2- 172.21.15.47:0/2808259531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4dc4004e70 0x7f4dc400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:19.058 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.043+0000 7f4de889e700 1 --2- 172.21.15.47:0/2808259531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4dc4003750 0x7f4dc4003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:19.058 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.043+0000 7f4de3fff700 1 --2- 172.21.15.47:0/2808259531 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4dc4004500 0x7f4dc4004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:19.058 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.047+0000 7f4de909f700 1 -- 172.21.15.47:0/2808259531 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4dc4004500 msgr2=0x7f4dc4004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:19.058 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.047+0000 7f4de909f700 1 --2- 172.21.15.47:0/2808259531 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4dc4004500 0x7f4dc4004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:19.059 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.047+0000 7f4de909f700 1 -- 172.21.15.47:0/2808259531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4dc4003750 msgr2=0x7f4dc4003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:19.059 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.047+0000 7f4de909f700 1 --2- 172.21.15.47:0/2808259531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4dc4003750 0x7f4dc4003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:19.059 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.047+0000 7f4de909f700 1 -- 172.21.15.47:0/2808259531 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dc4003260 con 0x7f4dc4004e70 2022-06-29T15:50:19.059 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.047+0000 7f4de909f700 1 -- 172.21.15.47:0/2808259531 reap_dead start 2022-06-29T15:50:19.060 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.047+0000 7f4de909f700 1 --2- 172.21.15.47:0/2808259531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4dc4004e70 0x7f4dc400b750 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 rx=0x7f4ddc001370 tx=0x7f4de40b1e80).ready entity=mon.0 client_cookie=72bc6185905d3f06 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:19.060 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.047+0000 7f4de37fe700 1 -- 172.21.15.47:0/2808259531 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ddc0070d0 con 0x7f4dc4004e70 2022-06-29T15:50:19.060 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.047+0000 7f4de37fe700 1 -- 172.21.15.47:0/2808259531 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ddc007230 con 0x7f4dc4004e70 2022-06-29T15:50:19.060 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.047+0000 7f4de37fe700 1 -- 172.21.15.47:0/2808259531 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ddc00c8f0 con 0x7f4dc4004e70 2022-06-29T15:50:19.064 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.047+0000 7f4deab02700 1 -- 172.21.15.47:0/2808259531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4dc4004e70 msgr2=0x7f4dc400b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:19.065 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.047+0000 7f4deab02700 1 --2- 172.21.15.47:0/2808259531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4dc4004e70 0x7f4dc400b750 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 rx=0x7f4ddc001370 tx=0x7f4de40b1e80).stop 2022-06-29T15:50:19.065 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.047+0000 7f4deab02700 1 -- 172.21.15.47:0/2808259531 shutdown_connections 2022-06-29T15:50:19.065 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.047+0000 7f4deab02700 1 --2- 172.21.15.47:0/2808259531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4dc4004e70 0x7f4dc400b750 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:19.065 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.047+0000 7f4deab02700 1 -- 172.21.15.47:0/2808259531 >> 172.21.15.47:0/2808259531 conn(0x7f4de405f0b0 msgr2=0x7f4de405f490 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:19.066 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.047+0000 7f4deab02700 1 -- 172.21.15.47:0/2808259531 shutdown_connections 2022-06-29T15:50:19.066 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.047+0000 7f4deab02700 1 -- 172.21.15.47:0/2808259531 wait complete. 2022-06-29T15:50:19.066 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.047+0000 7f4deab02700 1 Processor -- start 2022-06-29T15:50:19.066 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.047+0000 7f4deab02700 1 -- start start 2022-06-29T15:50:19.067 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.047+0000 7f4deab02700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4dc4004e70 0x7f4de4143930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:19.067 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.051+0000 7f4deab02700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4de40a9d10 0x7f4de40aa170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:19.067 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.051+0000 7f4deab02700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4de4141010 0x7f4de4141490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:19.067 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.051+0000 7f4deab02700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de4140720 con 0x7f4de40a9d10 2022-06-29T15:50:19.068 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.051+0000 7f4deab02700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de4140b80 con 0x7f4de4141010 2022-06-29T15:50:19.068 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.051+0000 7f4deab02700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4de413fd00 con 0x7f4dc4004e70 2022-06-29T15:50:19.068 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.051+0000 7f4de3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4de40a9d10 0x7f4de40aa170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:19.068 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.051+0000 7f4de3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4de40a9d10 0x7f4de40aa170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52220/0 (socket says 172.21.15.47:52220) 2022-06-29T15:50:19.069 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.051+0000 7f4de3fff700 1 -- 172.21.15.47:0/2994554430 learned_addr learned my addr 172.21.15.47:0/2994554430 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:19.069 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.051+0000 7f4de909f700 1 --2- 172.21.15.47:0/2994554430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4de4141010 0x7f4de4141490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:19.069 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.051+0000 7f4de889e700 1 --2- 172.21.15.47:0/2994554430 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4dc4004e70 0x7f4de4143930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:19.069 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.051+0000 7f4de3fff700 1 -- 172.21.15.47:0/2994554430 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4dc4004e70 msgr2=0x7f4de4143930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:19.070 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.051+0000 7f4de3fff700 1 --2- 172.21.15.47:0/2994554430 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4dc4004e70 0x7f4de4143930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:19.070 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.051+0000 7f4de3fff700 1 -- 172.21.15.47:0/2994554430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4de4141010 msgr2=0x7f4de4141490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:19.070 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.051+0000 7f4de3fff700 1 --2- 172.21.15.47:0/2994554430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4de4141010 0x7f4de4141490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:19.070 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.051+0000 7f4de3fff700 1 -- 172.21.15.47:0/2994554430 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4de4140280 con 0x7f4de40a9d10 2022-06-29T15:50:19.071 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.051+0000 7f4de889e700 1 --2- 172.21.15.47:0/2994554430 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4dc4004e70 0x7f4de4143930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:50:19.071 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.051+0000 7f4de3fff700 1 --2- 172.21.15.47:0/2994554430 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4de40a9d10 0x7f4de40aa170 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 rx=0x7f4de40b0610 tx=0x7f4dd40091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:19.071 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.051+0000 7f4de1ffb700 1 -- 172.21.15.47:0/2994554430 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4dd400f940 con 0x7f4de40a9d10 2022-06-29T15:50:19.071 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.051+0000 7f4de1ffb700 1 -- 172.21.15.47:0/2994554430 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dd400faa0 con 0x7f4de40a9d10 2022-06-29T15:50:19.072 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.051+0000 7f4de1ffb700 1 -- 172.21.15.47:0/2994554430 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4dd400fc10 con 0x7f4de40a9d10 2022-06-29T15:50:19.072 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.051+0000 7f4deab02700 1 -- 172.21.15.47:0/2994554430 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4de413f7f0 con 0x7f4de40a9d10 2022-06-29T15:50:19.072 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.051+0000 7f4deab02700 1 -- 172.21.15.47:0/2994554430 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4de40aa660 con 0x7f4de40a9d10 2022-06-29T15:50:19.072 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.051+0000 7f4deab02700 1 -- 172.21.15.47:0/2994554430 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4de40acdc0 con 0x7f4de40a9d10 2022-06-29T15:50:19.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.051+0000 7f4de1ffb700 1 -- 172.21.15.47:0/2994554430 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4dd4024070 con 0x7f4de40a9d10 2022-06-29T15:50:19.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.051+0000 7f4de1ffb700 1 --2- 172.21.15.47:0/2994554430 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4dc8053b00 0x7f4dc8055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:19.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.055+0000 7f4de1ffb700 1 -- 172.21.15.47:0/2994554430 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f4dd406ce30 con 0x7f4de40a9d10 2022-06-29T15:50:19.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.055+0000 7f4de889e700 1 --2- 172.21.15.47:0/2994554430 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4dc8053b00 0x7f4dc8055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:19.074 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.055+0000 7f4de889e700 1 --2- 172.21.15.47:0/2994554430 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4dc8053b00 0x7f4dc8055fc0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7f4dcc003000 tx=0x7f4dcc009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:19.074 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.063+0000 7f4de1ffb700 1 -- 172.21.15.47:0/2994554430 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4dd4043bf0 con 0x7f4de40a9d10 2022-06-29T15:50:19.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.187+0000 7f4deab02700 1 -- 172.21.15.47:0/2994554430 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4de413fec0 con 0x7f4de40a9d10 2022-06-29T15:50:19.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.191+0000 7f4de1ffb700 1 -- 172.21.15.47:0/2994554430 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v111) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4dd4047210 con 0x7f4de40a9d10 2022-06-29T15:50:19.196 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:50:19.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.191+0000 7f4deab02700 1 -- 172.21.15.47:0/2994554430 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4dc8053b00 msgr2=0x7f4dc8055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:19.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.191+0000 7f4deab02700 1 --2- 172.21.15.47:0/2994554430 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4dc8053b00 0x7f4dc8055fc0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7f4dcc003000 tx=0x7f4dcc009040).stop 2022-06-29T15:50:19.200 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.191+0000 7f4deab02700 1 -- 172.21.15.47:0/2994554430 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4de40a9d10 msgr2=0x7f4de40aa170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:19.200 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.191+0000 7f4deab02700 1 --2- 172.21.15.47:0/2994554430 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4de40a9d10 0x7f4de40aa170 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 rx=0x7f4de40b0610 tx=0x7f4dd40091d0).stop 2022-06-29T15:50:19.200 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.191+0000 7f4deab02700 1 -- 172.21.15.47:0/2994554430 shutdown_connections 2022-06-29T15:50:19.200 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.191+0000 7f4deab02700 1 --2- 172.21.15.47:0/2994554430 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4dc8053b00 0x7f4dc8055fc0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:19.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.191+0000 7f4deab02700 1 --2- 172.21.15.47:0/2994554430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4de4141010 0x7f4de4141490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:19.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.191+0000 7f4deab02700 1 --2- 172.21.15.47:0/2994554430 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4de40a9d10 0x7f4de40aa170 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:19.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.191+0000 7f4deab02700 1 --2- 172.21.15.47:0/2994554430 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4dc4004e70 0x7f4de4143930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:19.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.191+0000 7f4deab02700 1 -- 172.21.15.47:0/2994554430 >> 172.21.15.47:0/2994554430 conn(0x7f4de405f0b0 msgr2=0x7f4de4060370 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:19.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.191+0000 7f4deab02700 1 -- 172.21.15.47:0/2994554430 shutdown_connections 2022-06-29T15:50:19.202 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.191+0000 7f4deab02700 1 -- 172.21.15.47:0/2994554430 wait complete. 2022-06-29T15:50:19.211 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-29T15:50:19.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.315+0000 7f364d5af700 1 Processor -- start 2022-06-29T15:50:19.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.315+0000 7f364d5af700 1 -- start start 2022-06-29T15:50:19.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.319+0000 7f364d5af700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3648060ee0 0x7f36480612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:19.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.319+0000 7f364d5af700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3648067610 con 0x7f36480b4300 2022-06-29T15:50:19.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.319+0000 7f364d5af700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36481467c0 con 0x7f36481471a0 2022-06-29T15:50:19.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.319+0000 7f364d5af700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36481475d0 con 0x7f3648060ee0 2022-06-29T15:50:19.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.319+0000 7f36467fc700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f36480b4300 0x7f36481430e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37784/0 (socket says 172.21.15.47:37784) 2022-06-29T15:50:19.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.319+0000 7f36467fc700 1 -- 172.21.15.47:0/3242687013 learned_addr learned my addr 172.21.15.47:0/3242687013 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:19.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.319+0000 7f3646ffd700 1 --2- 172.21.15.47:0/3242687013 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3648060ee0 0x7f36480612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:19.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.319+0000 7f3645ffb700 1 -- 172.21.15.47:0/3242687013 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3774617705 0 0) 0x7f3648067610 con 0x7f36480b4300 2022-06-29T15:50:19.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.319+0000 7f3645ffb700 1 -- 172.21.15.47:0/3242687013 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f362c003260 con 0x7f36480b4300 2022-06-29T15:50:19.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.319+0000 7f3645ffb700 1 -- 172.21.15.47:0/3242687013 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 166404157 0 0) 0x7f36481467c0 con 0x7f36481471a0 2022-06-29T15:50:19.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.319+0000 7f3645ffb700 1 -- 172.21.15.47:0/3242687013 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3648067610 con 0x7f36481471a0 2022-06-29T15:50:19.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.319+0000 7f3646ffd700 1 -- 172.21.15.47:0/3242687013 >> v1:172.21.15.64:6789/0 conn(0x7f36481471a0 legacy=0x7f364813cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:19.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.319+0000 7f3646ffd700 1 -- 172.21.15.47:0/3242687013 >> v1:172.21.15.47:6789/0 conn(0x7f36480b4300 legacy=0x7f36481430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:19.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.319+0000 7f3646ffd700 1 -- 172.21.15.47:0/3242687013 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3648146a60 con 0x7f3648060ee0 2022-06-29T15:50:19.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.319+0000 7f3646ffd700 1 --2- 172.21.15.47:0/3242687013 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3648060ee0 0x7f36480612c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f3638002f70 tx=0x7f3638006df0).ready entity=mon.2 client_cookie=5922559055b70f4c server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:19.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.319+0000 7f3645ffb700 1 -- 172.21.15.47:0/3242687013 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f363800d5c0 con 0x7f3648060ee0 2022-06-29T15:50:19.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.319+0000 7f3645ffb700 1 -- 172.21.15.47:0/3242687013 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3638000e90 con 0x7f3648060ee0 2022-06-29T15:50:19.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.319+0000 7f3645ffb700 1 -- 172.21.15.47:0/3242687013 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f363800d910 con 0x7f3648060ee0 2022-06-29T15:50:19.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.319+0000 7f364d5af700 1 -- 172.21.15.47:0/3242687013 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3648060ee0 msgr2=0x7f36480612c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:19.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.319+0000 7f364d5af700 1 --2- 172.21.15.47:0/3242687013 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3648060ee0 0x7f36480612c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f3638002f70 tx=0x7f3638006df0).stop 2022-06-29T15:50:19.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.319+0000 7f364d5af700 1 -- 172.21.15.47:0/3242687013 shutdown_connections 2022-06-29T15:50:19.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.319+0000 7f364d5af700 1 --2- 172.21.15.47:0/3242687013 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3648060ee0 0x7f36480612c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:19.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.319+0000 7f364d5af700 1 -- 172.21.15.47:0/3242687013 >> 172.21.15.47:0/3242687013 conn(0x7f364805c510 msgr2=0x7f364805f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:19.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.319+0000 7f364d5af700 1 -- 172.21.15.47:0/3242687013 shutdown_connections 2022-06-29T15:50:19.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.319+0000 7f364d5af700 1 -- 172.21.15.47:0/3242687013 wait complete. 2022-06-29T15:50:19.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.323+0000 7f364d5af700 1 Processor -- start 2022-06-29T15:50:19.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.323+0000 7f364d5af700 1 -- start start 2022-06-29T15:50:19.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.323+0000 7f364d5af700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3648060ee0 0x7f3648144820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:19.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.323+0000 7f364d5af700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f36480b4300 0x7f36481523a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:19.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.323+0000 7f364d5af700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36481471a0 0x7f3648155ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:19.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.323+0000 7f364d5af700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3648145a10 con 0x7f36481471a0 2022-06-29T15:50:19.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.323+0000 7f364d5af700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36481558f0 con 0x7f3648060ee0 2022-06-29T15:50:19.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.323+0000 7f364d5af700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3648141600 con 0x7f36480b4300 2022-06-29T15:50:19.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.323+0000 7f36477fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36481471a0 0x7f3648155ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:19.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.323+0000 7f36467fc700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f36480b4300 0x7f36481523a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:19.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.323+0000 7f36477fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36481471a0 0x7f3648155ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52238/0 (socket says 172.21.15.47:52238) 2022-06-29T15:50:19.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.323+0000 7f3646ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3648060ee0 0x7f3648144820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:19.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.323+0000 7f36477fe700 1 -- 172.21.15.47:0/4131450076 learned_addr learned my addr 172.21.15.47:0/4131450076 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:19.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.323+0000 7f36477fe700 1 -- 172.21.15.47:0/4131450076 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f36480b4300 msgr2=0x7f36481523a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:19.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.323+0000 7f36477fe700 1 --2- 172.21.15.47:0/4131450076 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f36480b4300 0x7f36481523a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:19.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.323+0000 7f36477fe700 1 -- 172.21.15.47:0/4131450076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3648060ee0 msgr2=0x7f3648144820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:19.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.323+0000 7f36477fe700 1 --2- 172.21.15.47:0/4131450076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3648060ee0 0x7f3648144820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:19.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.323+0000 7f36477fe700 1 -- 172.21.15.47:0/4131450076 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f363800b180 con 0x7f36481471a0 2022-06-29T15:50:19.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.323+0000 7f36477fe700 1 --2- 172.21.15.47:0/4131450076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36481471a0 0x7f3648155ff0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 rx=0x7f3640001490 tx=0x7f3640007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:19.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.323+0000 7f3637fff700 1 -- 172.21.15.47:0/4131450076 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3640002310 con 0x7f36481471a0 2022-06-29T15:50:19.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.323+0000 7f3637fff700 1 -- 172.21.15.47:0/4131450076 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3640002470 con 0x7f36481471a0 2022-06-29T15:50:19.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.323+0000 7f364d5af700 1 -- 172.21.15.47:0/4131450076 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36481418a0 con 0x7f36481471a0 2022-06-29T15:50:19.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.323+0000 7f3637fff700 1 -- 172.21.15.47:0/4131450076 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f36400017a0 con 0x7f36481471a0 2022-06-29T15:50:19.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.323+0000 7f364d5af700 1 -- 172.21.15.47:0/4131450076 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f364813aef0 con 0x7f36481471a0 2022-06-29T15:50:19.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.327+0000 7f364d5af700 1 -- 172.21.15.47:0/4131450076 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36480573d0 con 0x7f36481471a0 2022-06-29T15:50:19.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.327+0000 7f3637fff700 1 -- 172.21.15.47:0/4131450076 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3640010030 con 0x7f36481471a0 2022-06-29T15:50:19.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.327+0000 7f3637fff700 1 --2- 172.21.15.47:0/4131450076 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f36240539e0 0x7f3624055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:19.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.327+0000 7f3637fff700 1 -- 172.21.15.47:0/4131450076 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f364006be80 con 0x7f36481471a0 2022-06-29T15:50:19.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.327+0000 7f3646ffd700 1 --2- 172.21.15.47:0/4131450076 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f36240539e0 0x7f3624055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:19.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.327+0000 7f3646ffd700 1 --2- 172.21.15.47:0/4131450076 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f36240539e0 0x7f3624055ea0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7f3638018000 tx=0x7f3638014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:19.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.335+0000 7f3637fff700 1 -- 172.21.15.47:0/4131450076 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3640042b60 con 0x7f36481471a0 2022-06-29T15:50:19.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.463+0000 7f364d5af700 1 -- 172.21.15.47:0/4131450076 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f36480b5e50 con 0x7f36481471a0 2022-06-29T15:50:19.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.463+0000 7f3637fff700 1 -- 172.21.15.47:0/4131450076 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v111) v1 ==== 104+0+10 (secure 0 0 0) 0x7f3640046180 con 0x7f36481471a0 2022-06-29T15:50:19.472 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:50:19.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.467+0000 7f364d5af700 1 -- 172.21.15.47:0/4131450076 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f36240539e0 msgr2=0x7f3624055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:19.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.467+0000 7f364d5af700 1 --2- 172.21.15.47:0/4131450076 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f36240539e0 0x7f3624055ea0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7f3638018000 tx=0x7f3638014000).stop 2022-06-29T15:50:19.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.467+0000 7f364d5af700 1 -- 172.21.15.47:0/4131450076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36481471a0 msgr2=0x7f3648155ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:19.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.467+0000 7f364d5af700 1 --2- 172.21.15.47:0/4131450076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36481471a0 0x7f3648155ff0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 rx=0x7f3640001490 tx=0x7f3640007dd0).stop 2022-06-29T15:50:19.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.467+0000 7f364d5af700 1 -- 172.21.15.47:0/4131450076 shutdown_connections 2022-06-29T15:50:19.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.467+0000 7f364d5af700 1 --2- 172.21.15.47:0/4131450076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36481471a0 0x7f3648155ff0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:19.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.467+0000 7f364d5af700 1 --2- 172.21.15.47:0/4131450076 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f36480b4300 0x7f36481523a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:19.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.467+0000 7f364d5af700 1 --2- 172.21.15.47:0/4131450076 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f36240539e0 0x7f3624055ea0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:19.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.467+0000 7f364d5af700 1 --2- 172.21.15.47:0/4131450076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3648060ee0 0x7f3648144820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:19.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.467+0000 7f364d5af700 1 -- 172.21.15.47:0/4131450076 >> 172.21.15.47:0/4131450076 conn(0x7f364805c510 msgr2=0x7f364805f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:19.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.467+0000 7f364d5af700 1 -- 172.21.15.47:0/4131450076 shutdown_connections 2022-06-29T15:50:19.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.467+0000 7f364d5af700 1 -- 172.21.15.47:0/4131450076 wait complete. 2022-06-29T15:50:19.489 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-29T15:50:19.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.595+0000 7f2fc2d94700 1 Processor -- start 2022-06-29T15:50:19.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.595+0000 7f2fc2d94700 1 -- start start 2022-06-29T15:50:19.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.595+0000 7f2fc2d94700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2fbc0b6dc0 0x7f2fbc0b11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:19.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.595+0000 7f2fc2d94700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2fbc0612b0 0x7f2fbc1402d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:19.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.595+0000 7f2fc2d94700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fbc0b2ce0 con 0x7f2fbc063100 2022-06-29T15:50:19.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.595+0000 7f2fc2d94700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fbc0b0e00 con 0x7f2fbc0612b0 2022-06-29T15:50:19.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.595+0000 7f2fc2d94700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fbc0b71a0 con 0x7f2fbc0b6dc0 2022-06-29T15:50:19.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.595+0000 7f2fc2593700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2fbc0612b0 0x7f2fbc1402d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:19.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.595+0000 7f2fc2593700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2fbc0612b0 0x7f2fbc1402d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52246/0 (socket says 172.21.15.47:52246) 2022-06-29T15:50:19.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.595+0000 7f2fc2593700 1 -- 172.21.15.47:0/2545424656 learned_addr learned my addr 172.21.15.47:0/2545424656 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:19.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.595+0000 7f2fc1591700 1 --1- 172.21.15.47:0/2545424656 >> v1:172.21.15.47:6789/0 conn(0x7f2fbc063100 0x7f2fbc0b16e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37798/0 (socket says 172.21.15.47:37798) 2022-06-29T15:50:19.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.595+0000 7f2fc1d92700 1 --2- 172.21.15.47:0/2545424656 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2fbc0b6dc0 0x7f2fbc0b11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:19.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.595+0000 7f2fc2593700 1 -- 172.21.15.47:0/2545424656 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2fbc0b6dc0 msgr2=0x7f2fbc0b11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:19.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.595+0000 7f2fc2593700 1 --2- 172.21.15.47:0/2545424656 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2fbc0b6dc0 0x7f2fbc0b11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:19.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.595+0000 7f2fc2593700 1 -- 172.21.15.47:0/2545424656 >> v1:172.21.15.47:6789/0 conn(0x7f2fbc063100 legacy=0x7f2fbc0b16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:19.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.595+0000 7f2fc0d90700 1 -- 172.21.15.47:0/2545424656 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3138033337 0 0) 0x7f2fbc0b2ce0 con 0x7f2fbc063100 2022-06-29T15:50:19.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.595+0000 7f2fc2593700 1 -- 172.21.15.47:0/2545424656 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fbc0620e0 con 0x7f2fbc0612b0 2022-06-29T15:50:19.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.595+0000 7f2fc2593700 1 --2- 172.21.15.47:0/2545424656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fbc0612b0 0x7f2fbc1402d0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 rx=0x7f2fac002f70 tx=0x7f2fac006eb0).ready entity=mon.0 client_cookie=15ea9ecd65156809 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:19.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.599+0000 7f2fc0d90700 1 -- 172.21.15.47:0/2545424656 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2fac00d570 con 0x7f2fbc0612b0 2022-06-29T15:50:19.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.599+0000 7f2fc0d90700 1 -- 172.21.15.47:0/2545424656 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fac000e90 con 0x7f2fbc0612b0 2022-06-29T15:50:19.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.599+0000 7f2fc0d90700 1 -- 172.21.15.47:0/2545424656 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2fac00d8c0 con 0x7f2fbc0612b0 2022-06-29T15:50:19.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.599+0000 7f2fc2d94700 1 -- 172.21.15.47:0/2545424656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fbc0612b0 msgr2=0x7f2fbc1402d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:19.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.599+0000 7f2fc2d94700 1 --2- 172.21.15.47:0/2545424656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fbc0612b0 0x7f2fbc1402d0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 rx=0x7f2fac002f70 tx=0x7f2fac006eb0).stop 2022-06-29T15:50:19.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.599+0000 7f2fc2d94700 1 -- 172.21.15.47:0/2545424656 shutdown_connections 2022-06-29T15:50:19.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.599+0000 7f2fc2d94700 1 --2- 172.21.15.47:0/2545424656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fbc0612b0 0x7f2fbc1402d0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:19.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.599+0000 7f2fc2d94700 1 --2- 172.21.15.47:0/2545424656 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2fbc0b6dc0 0x7f2fbc0b11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:19.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.599+0000 7f2fc2d94700 1 -- 172.21.15.47:0/2545424656 >> 172.21.15.47:0/2545424656 conn(0x7f2fbc063670 msgr2=0x7f2fbc063a50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:19.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.599+0000 7f2fc2d94700 1 -- 172.21.15.47:0/2545424656 shutdown_connections 2022-06-29T15:50:19.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.599+0000 7f2fc2d94700 1 -- 172.21.15.47:0/2545424656 wait complete. 2022-06-29T15:50:19.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.599+0000 7f2fc2d94700 1 Processor -- start 2022-06-29T15:50:19.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.603+0000 7f2fc2d94700 1 -- start start 2022-06-29T15:50:19.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.603+0000 7f2fc2d94700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fbc0612b0 0x7f2fbc0b5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:19.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.603+0000 7f2fc2d94700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2fbc063100 0x7f2fbc0afc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:19.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.603+0000 7f2fc2d94700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fbc0b6dc0 0x7f2fbc0655d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:19.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.603+0000 7f2fc2d94700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fbc066a60 con 0x7f2fbc0612b0 2022-06-29T15:50:19.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.603+0000 7f2fc2d94700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fbc0ae300 con 0x7f2fbc0b6dc0 2022-06-29T15:50:19.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.603+0000 7f2fc2d94700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fbc066590 con 0x7f2fbc063100 2022-06-29T15:50:19.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.603+0000 7f2fc1d92700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fbc0612b0 0x7f2fbc0b5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:19.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.603+0000 7f2fc1d92700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fbc0612b0 0x7f2fbc0b5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52248/0 (socket says 172.21.15.47:52248) 2022-06-29T15:50:19.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.603+0000 7f2fc1d92700 1 -- 172.21.15.47:0/338232019 learned_addr learned my addr 172.21.15.47:0/338232019 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:19.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.603+0000 7f2fc1591700 1 --2- 172.21.15.47:0/338232019 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2fbc063100 0x7f2fbc0afc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:19.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.603+0000 7f2fc2593700 1 --2- 172.21.15.47:0/338232019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fbc0b6dc0 0x7f2fbc0655d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:19.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.603+0000 7f2fc1d92700 1 -- 172.21.15.47:0/338232019 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2fbc063100 msgr2=0x7f2fbc0afc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:19.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.603+0000 7f2fc1d92700 1 --2- 172.21.15.47:0/338232019 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2fbc063100 0x7f2fbc0afc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:19.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.603+0000 7f2fc1d92700 1 -- 172.21.15.47:0/338232019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fbc0b6dc0 msgr2=0x7f2fbc0655d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:19.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.603+0000 7f2fc1d92700 1 --2- 172.21.15.47:0/338232019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fbc0b6dc0 0x7f2fbc0655d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:19.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.603+0000 7f2fc1d92700 1 -- 172.21.15.47:0/338232019 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fac00b1e0 con 0x7f2fbc0612b0 2022-06-29T15:50:19.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.603+0000 7f2fc2593700 1 --2- 172.21.15.47:0/338232019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fbc0b6dc0 0x7f2fbc0655d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:50:19.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.603+0000 7f2fc1d92700 1 --2- 172.21.15.47:0/338232019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fbc0612b0 0x7f2fbc0b5ce0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 rx=0x7f2fbc0ae440 tx=0x7f2fb40091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:19.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.603+0000 7f2fbaffd700 1 -- 172.21.15.47:0/338232019 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2fb400ee70 con 0x7f2fbc0612b0 2022-06-29T15:50:19.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.603+0000 7f2fbaffd700 1 -- 172.21.15.47:0/338232019 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fb400c850 con 0x7f2fbc0612b0 2022-06-29T15:50:19.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.603+0000 7f2fc2d94700 1 -- 172.21.15.47:0/338232019 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fbc0af7c0 con 0x7f2fbc0612b0 2022-06-29T15:50:19.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.603+0000 7f2fbaffd700 1 -- 172.21.15.47:0/338232019 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2fb400cbb0 con 0x7f2fbc0612b0 2022-06-29T15:50:19.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.603+0000 7f2fc2d94700 1 -- 172.21.15.47:0/338232019 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fbc0b0110 con 0x7f2fbc0612b0 2022-06-29T15:50:19.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.607+0000 7f2fc2d94700 1 -- 172.21.15.47:0/338232019 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fbc0573d0 con 0x7f2fbc0612b0 2022-06-29T15:50:19.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.607+0000 7f2fbaffd700 1 -- 172.21.15.47:0/338232019 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2fb4022070 con 0x7f2fbc0612b0 2022-06-29T15:50:19.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.607+0000 7f2fbaffd700 1 --2- 172.21.15.47:0/338232019 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2f9c053b00 0x7f2f9c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:19.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.607+0000 7f2fbaffd700 1 -- 172.21.15.47:0/338232019 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f2fb406c680 con 0x7f2fbc0612b0 2022-06-29T15:50:19.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.607+0000 7f2fc1591700 1 --2- 172.21.15.47:0/338232019 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2f9c053b00 0x7f2f9c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:19.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.607+0000 7f2fc1591700 1 --2- 172.21.15.47:0/338232019 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2f9c053b00 0x7f2f9c055fc0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7f2fb000d000 tx=0x7f2fb0007a00).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:19.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.615+0000 7f2fbaffd700 1 -- 172.21.15.47:0/338232019 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2fb4043440 con 0x7f2fbc0612b0 2022-06-29T15:50:19.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.743+0000 7f2fc2d94700 1 -- 172.21.15.47:0/338232019 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2fbc069e20 con 0x7f2fbc0612b0 2022-06-29T15:50:19.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.743+0000 7f2fbaffd700 1 -- 172.21.15.47:0/338232019 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v111) v1 ==== 100+0+10 (secure 0 0 0) 0x7f2fb4046a60 con 0x7f2fbc0612b0 2022-06-29T15:50:19.751 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:50:19.753 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.747+0000 7f2fc2d94700 1 -- 172.21.15.47:0/338232019 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2f9c053b00 msgr2=0x7f2f9c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:19.753 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.747+0000 7f2fc2d94700 1 --2- 172.21.15.47:0/338232019 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2f9c053b00 0x7f2f9c055fc0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7f2fb000d000 tx=0x7f2fb0007a00).stop 2022-06-29T15:50:19.753 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.747+0000 7f2fc2d94700 1 -- 172.21.15.47:0/338232019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fbc0612b0 msgr2=0x7f2fbc0b5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:19.754 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.747+0000 7f2fc2d94700 1 --2- 172.21.15.47:0/338232019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fbc0612b0 0x7f2fbc0b5ce0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 rx=0x7f2fbc0ae440 tx=0x7f2fb40091d0).stop 2022-06-29T15:50:19.754 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.747+0000 7f2fc2d94700 1 -- 172.21.15.47:0/338232019 shutdown_connections 2022-06-29T15:50:19.754 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.747+0000 7f2fc2d94700 1 --2- 172.21.15.47:0/338232019 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2f9c053b00 0x7f2f9c055fc0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:19.754 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.747+0000 7f2fc2d94700 1 --2- 172.21.15.47:0/338232019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fbc0b6dc0 0x7f2fbc0655d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:19.755 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.747+0000 7f2fc2d94700 1 --2- 172.21.15.47:0/338232019 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2fbc063100 0x7f2fbc0afc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:19.755 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.747+0000 7f2fc2d94700 1 --2- 172.21.15.47:0/338232019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fbc0612b0 0x7f2fbc0b5ce0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:19.755 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.747+0000 7f2fc2d94700 1 -- 172.21.15.47:0/338232019 >> 172.21.15.47:0/338232019 conn(0x7f2fbc063670 msgr2=0x7f2fbc05f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:19.755 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.747+0000 7f2fc2d94700 1 -- 172.21.15.47:0/338232019 shutdown_connections 2022-06-29T15:50:19.755 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.747+0000 7f2fc2d94700 1 -- 172.21.15.47:0/338232019 wait complete. 2022-06-29T15:50:19.765 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-29T15:50:19.765 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:50:19.874 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.867+0000 7f2c55c9f700 1 Processor -- start 2022-06-29T15:50:19.874 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.867+0000 7f2c55c9f700 1 -- start start 2022-06-29T15:50:19.877 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.867+0000 7f2c55c9f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2c500b1720 0x7f2c50140350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:19.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.867+0000 7f2c55c9f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2c500a8ce0 0x7f2c50140840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:19.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.867+0000 7f2c55c9f700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c500b6030 con 0x7f2c500b2690 2022-06-29T15:50:19.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.867+0000 7f2c55c9f700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c5013dac0 con 0x7f2c500b1720 2022-06-29T15:50:19.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.867+0000 7f2c55c9f700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c50140d30 con 0x7f2c500a8ce0 2022-06-29T15:50:19.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.867+0000 7f2c4ffff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2c500b1720 0x7f2c50140350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:19.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.867+0000 7f2c4ffff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2c500b1720 0x7f2c50140350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52256/0 (socket says 172.21.15.47:52256) 2022-06-29T15:50:19.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.867+0000 7f2c4ffff700 1 -- 172.21.15.47:0/3293355238 learned_addr learned my addr 172.21.15.47:0/3293355238 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:19.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.867+0000 7f2c5549e700 1 --2- 172.21.15.47:0/3293355238 >> v2:172.21.15.64:3300/0 conn(0x7f2c500a8ce0 0x7f2c50140840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:19.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.871+0000 7f2c4ffff700 1 -- 172.21.15.47:0/3293355238 >> v2:172.21.15.64:3300/0 conn(0x7f2c500a8ce0 msgr2=0x7f2c50140840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:19.880 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.871+0000 7f2c4ffff700 1 --2- 172.21.15.47:0/3293355238 >> v2:172.21.15.64:3300/0 conn(0x7f2c500a8ce0 0x7f2c50140840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:19.880 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.871+0000 7f2c4ffff700 1 -- 172.21.15.47:0/3293355238 >> v1:172.21.15.64:6789/0 conn(0x7f2c500b2690 legacy=0x7f2c500b5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:19.880 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.871+0000 7f2c4ffff700 1 -- 172.21.15.47:0/3293355238 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c500b0be0 con 0x7f2c500b1720 2022-06-29T15:50:19.880 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.871+0000 7f2c4ffff700 1 --2- 172.21.15.47:0/3293355238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c500b1720 0x7f2c50140350 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 rx=0x7f2c500b6370 tx=0x7f2c3c006eb0).ready entity=mon.0 client_cookie=8fab437bab0c131c server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:19.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.871+0000 7f2c4f7fe700 1 -- 172.21.15.47:0/3293355238 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c3c00d5c0 con 0x7f2c500b1720 2022-06-29T15:50:19.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.871+0000 7f2c4f7fe700 1 -- 172.21.15.47:0/3293355238 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c3c000e90 con 0x7f2c500b1720 2022-06-29T15:50:19.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.871+0000 7f2c4f7fe700 1 -- 172.21.15.47:0/3293355238 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c3c00d950 con 0x7f2c500b1720 2022-06-29T15:50:19.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.871+0000 7f2c55c9f700 1 -- 172.21.15.47:0/3293355238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c500b1720 msgr2=0x7f2c50140350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:19.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.871+0000 7f2c55c9f700 1 --2- 172.21.15.47:0/3293355238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c500b1720 0x7f2c50140350 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 rx=0x7f2c500b6370 tx=0x7f2c3c006eb0).stop 2022-06-29T15:50:19.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.871+0000 7f2c55c9f700 1 -- 172.21.15.47:0/3293355238 shutdown_connections 2022-06-29T15:50:19.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.871+0000 7f2c55c9f700 1 --2- 172.21.15.47:0/3293355238 >> v2:172.21.15.64:3300/0 conn(0x7f2c500a8ce0 0x7f2c50140840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:19.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.871+0000 7f2c55c9f700 1 --2- 172.21.15.47:0/3293355238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c500b1720 0x7f2c50140350 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:19.883 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.871+0000 7f2c55c9f700 1 -- 172.21.15.47:0/3293355238 >> 172.21.15.47:0/3293355238 conn(0x7f2c50062560 msgr2=0x7f2c50062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:19.883 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.871+0000 7f2c55c9f700 1 -- 172.21.15.47:0/3293355238 shutdown_connections 2022-06-29T15:50:19.883 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.871+0000 7f2c55c9f700 1 -- 172.21.15.47:0/3293355238 wait complete. 2022-06-29T15:50:19.883 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.871+0000 7f2c55c9f700 1 Processor -- start 2022-06-29T15:50:19.884 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.871+0000 7f2c55c9f700 1 -- start start 2022-06-29T15:50:19.886 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.871+0000 7f2c55c9f700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2c500a8ce0 0x7f2c500af120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:19.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.871+0000 7f2c55c9f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c500b1720 0x7f2c50141e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:19.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.871+0000 7f2c55c9f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c500b2690 0x7f2c501429a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:19.888 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.871+0000 7f2c55c9f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c500a80b0 con 0x7f2c500b2690 2022-06-29T15:50:19.888 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.871+0000 7f2c55c9f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c500a8310 con 0x7f2c500b1720 2022-06-29T15:50:19.889 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.871+0000 7f2c55c9f700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c500a85b0 con 0x7f2c500a8ce0 2022-06-29T15:50:19.889 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.871+0000 7f2c5549e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c500b2690 0x7f2c501429a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:19.890 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.871+0000 7f2c5549e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c500b2690 0x7f2c501429a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52266/0 (socket says 172.21.15.47:52266) 2022-06-29T15:50:19.890 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.871+0000 7f2c5549e700 1 -- 172.21.15.47:0/810912396 learned_addr learned my addr 172.21.15.47:0/810912396 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:19.891 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.871+0000 7f2c4ffff700 1 --2- 172.21.15.47:0/810912396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c500b1720 0x7f2c50141e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:19.891 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.871+0000 7f2c54c9d700 1 --2- 172.21.15.47:0/810912396 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2c500a8ce0 0x7f2c500af120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:19.892 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.875+0000 7f2c5549e700 1 -- 172.21.15.47:0/810912396 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2c500a8ce0 msgr2=0x7f2c500af120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:19.892 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.875+0000 7f2c5549e700 1 --2- 172.21.15.47:0/810912396 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2c500a8ce0 0x7f2c500af120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:19.893 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.875+0000 7f2c5549e700 1 -- 172.21.15.47:0/810912396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c500b1720 msgr2=0x7f2c50141e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:19.893 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.875+0000 7f2c5549e700 1 --2- 172.21.15.47:0/810912396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c500b1720 0x7f2c50141e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:19.894 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.875+0000 7f2c5549e700 1 -- 172.21.15.47:0/810912396 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c3c00b1e0 con 0x7f2c500b2690 2022-06-29T15:50:19.894 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.875+0000 7f2c54c9d700 1 --2- 172.21.15.47:0/810912396 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2c500a8ce0 0x7f2c500af120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:50:19.895 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.875+0000 7f2c4ffff700 1 --2- 172.21.15.47:0/810912396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c500b1720 0x7f2c50141e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:50:19.895 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.875+0000 7f2c5549e700 1 --2- 172.21.15.47:0/810912396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c500b2690 0x7f2c501429a0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 rx=0x7f2c4800a000 tx=0x7f2c48008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:19.896 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.875+0000 7f2c4dffb700 1 -- 172.21.15.47:0/810912396 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c48013620 con 0x7f2c500b2690 2022-06-29T15:50:19.896 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.875+0000 7f2c4dffb700 1 -- 172.21.15.47:0/810912396 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c48013780 con 0x7f2c500b2690 2022-06-29T15:50:19.896 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.875+0000 7f2c55c9f700 1 -- 172.21.15.47:0/810912396 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c500a8850 con 0x7f2c500b2690 2022-06-29T15:50:19.897 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.875+0000 7f2c4dffb700 1 -- 172.21.15.47:0/810912396 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c48013ae0 con 0x7f2c500b2690 2022-06-29T15:50:19.897 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.875+0000 7f2c55c9f700 1 -- 172.21.15.47:0/810912396 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c500635e0 con 0x7f2c500b2690 2022-06-29T15:50:19.898 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.875+0000 7f2c55c9f700 1 -- 172.21.15.47:0/810912396 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c500573d0 con 0x7f2c500b2690 2022-06-29T15:50:19.898 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.875+0000 7f2c4dffb700 1 -- 172.21.15.47:0/810912396 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2c48013c40 con 0x7f2c500b2690 2022-06-29T15:50:19.899 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.875+0000 7f2c4dffb700 1 --2- 172.21.15.47:0/810912396 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2c30053950 0x7f2c30055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:19.899 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.875+0000 7f2c4dffb700 1 -- 172.21.15.47:0/810912396 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f2c4806c160 con 0x7f2c500b2690 2022-06-29T15:50:19.900 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.875+0000 7f2c54c9d700 1 --2- 172.21.15.47:0/810912396 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2c30053950 0x7f2c30055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:19.900 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.879+0000 7f2c54c9d700 1 --2- 172.21.15.47:0/810912396 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2c30053950 0x7f2c30055e10 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7f2c4400c000 tx=0x7f2c44008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:19.901 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:19.883+0000 7f2c4dffb700 1 -- 172.21.15.47:0/810912396 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2c48042ec0 con 0x7f2c500b2690 2022-06-29T15:50:20.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:20.043+0000 7f2c55c9f700 1 -- 172.21.15.47:0/810912396 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2c50069e20 con 0x7f2c500b2690 2022-06-29T15:50:20.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:20.043+0000 7f2c4dffb700 1 -- 172.21.15.47:0/810912396 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+2496 (secure 0 0 0) 0x7f2c480464e0 con 0x7f2c500b2690 2022-06-29T15:50:20.052 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:50:20.052 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":51,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3752554010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3752554010},{"type":"v1","addr":"172.21.15.47:6835","nonce":3752554010}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":48},{"gid":7293,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.47:6837/3263659716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":3263659716},{"type":"v1","addr":"172.21.15.47:6837","nonce":3263659716}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":49}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:50:15.338114+0000","modified":"2022-06-29T15:50:16.342008+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":56354},"failed":[],"damaged":[],"stopped":[],"info":{"gid_56354":{"gid":56354,"name":"b","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6833/2706285595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2706285595},{"type":"v1","addr":"172.21.15.64:6833","nonce":2706285595}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2022-06-29T15:50:20.054 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:20.047+0000 7f2c55c9f700 1 -- 172.21.15.47:0/810912396 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2c30053950 msgr2=0x7f2c30055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:20.054 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:20.047+0000 7f2c55c9f700 1 --2- 172.21.15.47:0/810912396 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2c30053950 0x7f2c30055e10 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7f2c4400c000 tx=0x7f2c44008040).stop 2022-06-29T15:50:20.054 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:20.047+0000 7f2c55c9f700 1 -- 172.21.15.47:0/810912396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c500b2690 msgr2=0x7f2c501429a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:20.054 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:20.047+0000 7f2c55c9f700 1 --2- 172.21.15.47:0/810912396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c500b2690 0x7f2c501429a0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 rx=0x7f2c4800a000 tx=0x7f2c48008040).stop 2022-06-29T15:50:20.055 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:20.047+0000 7f2c55c9f700 1 -- 172.21.15.47:0/810912396 shutdown_connections 2022-06-29T15:50:20.055 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:20.047+0000 7f2c55c9f700 1 --2- 172.21.15.47:0/810912396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c500b2690 0x7f2c501429a0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:20.055 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:20.047+0000 7f2c55c9f700 1 --2- 172.21.15.47:0/810912396 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2c30053950 0x7f2c30055e10 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:20.055 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:20.047+0000 7f2c55c9f700 1 --2- 172.21.15.47:0/810912396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c500b1720 0x7f2c50141e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:20.056 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:20.047+0000 7f2c55c9f700 1 --2- 172.21.15.47:0/810912396 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2c500a8ce0 0x7f2c500af120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:20.056 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:20.047+0000 7f2c55c9f700 1 -- 172.21.15.47:0/810912396 >> 172.21.15.47:0/810912396 conn(0x7f2c50062560 msgr2=0x7f2c500ad220 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:20.056 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:20.047+0000 7f2c55c9f700 1 -- 172.21.15.47:0/810912396 shutdown_connections 2022-06-29T15:50:20.056 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:20.047+0000 7f2c55c9f700 1 -- 172.21.15.47:0/810912396 wait complete. 2022-06-29T15:50:20.057 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 51 2022-06-29T15:50:20.366 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 51, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-29T15:50:15.338114+0000', 'modified': '2022-06-29T15:50:16.342008+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 56354}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_56354': {'gid': 56354, 'name': 'b', 'rank': 0, 'incarnation': 50, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.64:6833/2706285595', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6832', 'nonce': 2706285595}, {'type': 'v1', 'addr': '172.21.15.64:6833', 'nonce': 2706285595}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-29T15:50:20.366 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-29T15:50:20.367 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-06-29T15:50:20.491 INFO:teuthology.orchestra.run.smithi064.stdout:{"cluster_fsid":"9a68894a-53ce-400e-838b-14eab91b834c","whoami":0,"id":56354,"want_state":"up:active","state":"up:active","rank_uptime":5.1273570639999999,"mdsmap_epoch":51,"osdmap_epoch":111,"osdmap_epoch_barrier":110,"uptime":9.4906178739999998} 2022-06-29T15:50:20.491 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9a68894a-53ce-400e-838b-14eab91b834c", "id": 56354, "mdsmap_epoch": 51, "osdmap_epoch": 111, "osdmap_epoch_barrier": 110, "rank_uptime": 5.127357064, "state": "up:active", "uptime": 9.490617874, "want_state": "up:active", "whoami": 0 } 2022-06-29T15:50:20.492 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-29T15:50:20.492 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-06-29T15:50:20.492 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi099.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-06-29T15:50:20.493 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:20.493 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:50:20.502 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2022-06-29T15:50:20.519 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:50:20.520 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:50:20.577 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:50:20.579 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:50:20.579 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:20.580 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2022-06-29T15:50:20.587 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-29T15:50:20.588 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:50:20.591 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2022-06-29T15:50:20.607 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:50:20.608 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:50:20.618 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:50:20.619 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:50:20.620 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:20.620 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2022-06-29T15:50:20.662 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:2022-06-29T15:50:20.651+0000 7f7a3ca8d080 -1 init, newargv = 0x55afd91a76d0 newargc=9ceph-fuse[13187 2022-06-29T15:50:20.662 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:]: starting ceph client 2022-06-29T15:50:20.671 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[13187]: starting fuse 2022-06-29T15:50:21.728 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2022-06-29T15:50:21.747 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:50:21.747 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:50:21.757 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:50:21.758 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:50:21.759 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:21.760 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2022-06-29T15:50:21.809 INFO:teuthology.orchestra.run.smithi099.stdout:52 2022-06-29T15:50:21.809 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-29T15:50:21.810 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:21.810 DEBUG:teuthology.orchestra.run.smithi099:> sudo python3 -c ' 2022-06-29T15:50:21.811 DEBUG:teuthology.orchestra.run.smithi099:> import glob 2022-06-29T15:50:21.812 DEBUG:teuthology.orchestra.run.smithi099:> import re 2022-06-29T15:50:21.812 DEBUG:teuthology.orchestra.run.smithi099:> import os 2022-06-29T15:50:21.812 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2022-06-29T15:50:21.813 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:50:21.813 DEBUG:teuthology.orchestra.run.smithi099:> def find_socket(client_name): 2022-06-29T15:50:21.814 DEBUG:teuthology.orchestra.run.smithi099:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-06-29T15:50:21.814 DEBUG:teuthology.orchestra.run.smithi099:> files = glob.glob(asok_path) 2022-06-29T15:50:21.814 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:50:21.815 DEBUG:teuthology.orchestra.run.smithi099:> # Given a non-glob path, it better be there 2022-06-29T15:50:21.815 DEBUG:teuthology.orchestra.run.smithi099:> if "*" not in asok_path: 2022-06-29T15:50:21.816 DEBUG:teuthology.orchestra.run.smithi099:> assert(len(files) == 1) 2022-06-29T15:50:21.817 DEBUG:teuthology.orchestra.run.smithi099:> return files[0] 2022-06-29T15:50:21.817 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:50:21.818 DEBUG:teuthology.orchestra.run.smithi099:> for f in files: 2022-06-29T15:50:21.819 DEBUG:teuthology.orchestra.run.smithi099:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-29T15:50:21.819 DEBUG:teuthology.orchestra.run.smithi099:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-29T15:50:21.819 DEBUG:teuthology.orchestra.run.smithi099:> return f 2022-06-29T15:50:21.820 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-29T15:50:21.820 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:50:21.821 DEBUG:teuthology.orchestra.run.smithi099:> print(find_socket("client.0")) 2022-06-29T15:50:21.821 DEBUG:teuthology.orchestra.run.smithi099:> ' 2022-06-29T15:50:21.865 INFO:teuthology.orchestra.run.smithi099.stdout:/var/run/ceph/ceph-client.0.13187.asok 2022-06-29T15:50:21.871 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.13187.asok 2022-06-29T15:50:21.871 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:21.872 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.13187.asok status 2022-06-29T15:50:21.960 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2022-06-29T15:50:21.960 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": { 2022-06-29T15:50:21.960 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_sha1": "f25ffac72e5a9e63860ff7deb6767a11f0559a16", 2022-06-29T15:50:21.961 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 15.2.16-304-gf25ffac7 (f25ffac72e5a9e63860ff7deb6767a11f0559a16) octopus (stable)", 2022-06-29T15:50:21.961 INFO:teuthology.orchestra.run.smithi099.stdout: "entity_id": "0", 2022-06-29T15:50:21.961 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": "smithi099", 2022-06-29T15:50:21.962 INFO:teuthology.orchestra.run.smithi099.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-06-29T15:50:21.962 INFO:teuthology.orchestra.run.smithi099.stdout: "pid": "13187", 2022-06-29T15:50:21.962 INFO:teuthology.orchestra.run.smithi099.stdout: "root": "/" 2022-06-29T15:50:21.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:50:21.963 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_count": 0, 2022-06-29T15:50:21.963 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_pinned_count": 0, 2022-06-29T15:50:21.963 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 56474, 2022-06-29T15:50:21.963 INFO:teuthology.orchestra.run.smithi099.stdout: "inst": { 2022-06-29T15:50:21.964 INFO:teuthology.orchestra.run.smithi099.stdout: "name": { 2022-06-29T15:50:21.964 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "client", 2022-06-29T15:50:21.964 INFO:teuthology.orchestra.run.smithi099.stdout: "num": 56474 2022-06-29T15:50:21.964 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:50:21.965 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2022-06-29T15:50:21.965 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2022-06-29T15:50:21.965 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:0", 2022-06-29T15:50:21.966 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4127554656 2022-06-29T15:50:21.966 INFO:teuthology.orchestra.run.smithi099.stdout: } 2022-06-29T15:50:21.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:50:21.967 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2022-06-29T15:50:21.967 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2022-06-29T15:50:21.967 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:0", 2022-06-29T15:50:21.967 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4127554656 2022-06-29T15:50:21.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:50:21.968 INFO:teuthology.orchestra.run.smithi099.stdout: "inst_str": "client.56474 172.21.15.99:0/4127554656", 2022-06-29T15:50:21.968 INFO:teuthology.orchestra.run.smithi099.stdout: "addr_str": "172.21.15.99:0/4127554656", 2022-06-29T15:50:21.968 INFO:teuthology.orchestra.run.smithi099.stdout: "inode_count": 1, 2022-06-29T15:50:21.968 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_epoch": 51, 2022-06-29T15:50:21.969 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch": 111, 2022-06-29T15:50:21.969 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch_barrier": 0, 2022-06-29T15:50:21.969 INFO:teuthology.orchestra.run.smithi099.stdout: "blacklisted": false, 2022-06-29T15:50:21.969 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs" 2022-06-29T15:50:21.970 INFO:teuthology.orchestra.run.smithi099.stdout:} 2022-06-29T15:50:21.971 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:21.972 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:50:21.972 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:50:21.980 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2022-06-29T15:50:21.981 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-29T15:50:21.982 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:21.982 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:50:22.000 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:50:22.115 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.107+0000 7fb622d87700 1 Processor -- start 2022-06-29T15:50:22.115 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.107+0000 7fb622d87700 1 -- start start 2022-06-29T15:50:22.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.111+0000 7fb622d87700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb61c05c510 0x7fb61c143bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:22.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.111+0000 7fb622d87700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb61c0b2c40 0x7fb61c0b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:22.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.111+0000 7fb622d87700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb61c142f90 con 0x7fb61c05c8f0 2022-06-29T15:50:22.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.111+0000 7fb622d87700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb61c1431f0 con 0x7fb61c05c510 2022-06-29T15:50:22.121 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.111+0000 7fb622d87700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c143490 con 0x7fb61c0b2c40 2022-06-29T15:50:22.121 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.111+0000 7fb621584700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb61c05c8f0 0x7fb61c0b4100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37824/0 (socket says 172.21.15.47:37824) 2022-06-29T15:50:22.121 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.111+0000 7fb621584700 1 -- 172.21.15.47:0/3130466809 learned_addr learned my addr 172.21.15.47:0/3130466809 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:22.121 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.111+0000 7fb622586700 1 --2- 172.21.15.47:0/3130466809 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb61c0b2c40 0x7fb61c0b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:22.122 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.111+0000 7fb620d83700 1 -- 172.21.15.47:0/3130466809 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3961954812 0 0) 0x7fb61c142f90 con 0x7fb61c05c8f0 2022-06-29T15:50:22.122 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.111+0000 7fb621d85700 1 --2- 172.21.15.47:0/3130466809 >> v2:172.21.15.64:3300/0 conn(0x7fb61c05c510 0x7fb61c143bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:22.122 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.111+0000 7fb620d83700 1 -- 172.21.15.47:0/3130466809 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb604003260 con 0x7fb61c05c8f0 2022-06-29T15:50:22.122 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.111+0000 7fb620d83700 1 -- 172.21.15.47:0/3130466809 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2404891297 0 0) 0x7fb604003260 con 0x7fb61c05c8f0 2022-06-29T15:50:22.123 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.111+0000 7fb620d83700 1 -- 172.21.15.47:0/3130466809 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb61c0b2c40 msgr2=0x7fb61c0b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:22.123 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.111+0000 7fb620d83700 1 --2- 172.21.15.47:0/3130466809 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb61c0b2c40 0x7fb61c0b30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:22.123 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.111+0000 7fb620d83700 1 -- 172.21.15.47:0/3130466809 >> v2:172.21.15.64:3300/0 conn(0x7fb61c05c510 msgr2=0x7fb61c143bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:22.123 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.111+0000 7fb620d83700 1 --2- 172.21.15.47:0/3130466809 >> v2:172.21.15.64:3300/0 conn(0x7fb61c05c510 0x7fb61c143bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:22.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.111+0000 7fb620d83700 1 -- 172.21.15.47:0/3130466809 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb61c143730 con 0x7fb61c05c8f0 2022-06-29T15:50:22.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.111+0000 7fb620d83700 1 -- 172.21.15.47:0/3130466809 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7fb618001a00 con 0x7fb61c05c8f0 2022-06-29T15:50:22.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.111+0000 7fb620d83700 1 -- 172.21.15.47:0/3130466809 >> v1:172.21.15.47:6789/0 conn(0x7fb61c05c8f0 legacy=0x7fb61c0b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:22.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.111+0000 7fb620d83700 1 --2- 172.21.15.47:0/3130466809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb604003750 0x7fb604003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:22.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.111+0000 7fb620d83700 1 --2- 172.21.15.47:0/3130466809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb604004500 0x7fb604004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:22.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.111+0000 7fb620d83700 1 --2- 172.21.15.47:0/3130466809 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb61c0b2c40 0x7fb604004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:22.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.111+0000 7fb620d83700 1 -- 172.21.15.47:0/3130466809 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c1431f0 con 0x7fb604003750 2022-06-29T15:50:22.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.111+0000 7fb620d83700 1 -- 172.21.15.47:0/3130466809 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c143490 con 0x7fb604004500 2022-06-29T15:50:22.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.111+0000 7fb620d83700 1 -- 172.21.15.47:0/3130466809 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb60400b750 con 0x7fb61c0b2c40 2022-06-29T15:50:22.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.111+0000 7fb621d85700 1 --2- 172.21.15.47:0/3130466809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb604003750 0x7fb604003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:22.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.111+0000 7fb621584700 1 --2- 172.21.15.47:0/3130466809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb604004500 0x7fb604004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:22.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.111+0000 7fb622586700 1 --2- 172.21.15.47:0/3130466809 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb61c0b2c40 0x7fb604004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:22.127 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.111+0000 7fb621d85700 1 -- 172.21.15.47:0/3130466809 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb61c0b2c40 msgr2=0x7fb604004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:22.127 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.111+0000 7fb621d85700 1 --2- 172.21.15.47:0/3130466809 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb61c0b2c40 0x7fb604004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:22.127 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.111+0000 7fb621d85700 1 -- 172.21.15.47:0/3130466809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb604004500 msgr2=0x7fb604004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:22.127 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.111+0000 7fb621d85700 1 --2- 172.21.15.47:0/3130466809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb604004500 0x7fb604004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:22.127 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.111+0000 7fb621d85700 1 -- 172.21.15.47:0/3130466809 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb604003260 con 0x7fb604003750 2022-06-29T15:50:22.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.111+0000 7fb621584700 1 --2- 172.21.15.47:0/3130466809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb604004500 0x7fb604004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:50:22.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.111+0000 7fb621d85700 1 --2- 172.21.15.47:0/3130466809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb604003750 0x7fb604003b70 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 rx=0x7fb60c00c000 tx=0x7fb60c006df0).ready entity=mon.0 client_cookie=ae48a6d18e61755 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:22.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.111+0000 7fb620d83700 1 -- 172.21.15.47:0/3130466809 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb60c00e830 con 0x7fb604003750 2022-06-29T15:50:22.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.111+0000 7fb620d83700 1 -- 172.21.15.47:0/3130466809 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb60c00e990 con 0x7fb604003750 2022-06-29T15:50:22.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.111+0000 7fb620d83700 1 -- 172.21.15.47:0/3130466809 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb60c00ecf0 con 0x7fb604003750 2022-06-29T15:50:22.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.115+0000 7fb622d87700 1 -- 172.21.15.47:0/3130466809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb604003750 msgr2=0x7fb604003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:22.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.115+0000 7fb622d87700 1 --2- 172.21.15.47:0/3130466809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb604003750 0x7fb604003b70 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 rx=0x7fb60c00c000 tx=0x7fb60c006df0).stop 2022-06-29T15:50:22.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.115+0000 7fb622586700 1 -- 172.21.15.47:0/3130466809 reap_dead start 2022-06-29T15:50:22.130 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.115+0000 7fb622d87700 1 -- 172.21.15.47:0/3130466809 shutdown_connections 2022-06-29T15:50:22.130 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.115+0000 7fb622d87700 1 -- 172.21.15.47:0/3130466809 >> 172.21.15.47:0/3130466809 conn(0x7fb61c060e00 msgr2=0x7fb61c0611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:22.130 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.115+0000 7fb622d87700 1 -- 172.21.15.47:0/3130466809 shutdown_connections 2022-06-29T15:50:22.130 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.115+0000 7fb622d87700 1 -- 172.21.15.47:0/3130466809 wait complete. 2022-06-29T15:50:22.133 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.115+0000 7fb622d87700 1 Processor -- start 2022-06-29T15:50:22.134 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.115+0000 7fb622d87700 1 -- start start 2022-06-29T15:50:22.134 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.119+0000 7fb622d87700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb604003750 0x7fb61c0b7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:22.134 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.119+0000 7fb622d87700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb61c155ac0 0x7fb61c152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:22.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.119+0000 7fb622d87700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb61c1465f0 0x7fb61c1530d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:22.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.119+0000 7fb622d87700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c0af780 con 0x7fb604003750 2022-06-29T15:50:22.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.119+0000 7fb622d87700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c152550 con 0x7fb61c155ac0 2022-06-29T15:50:22.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.119+0000 7fb622d87700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c146f30 con 0x7fb61c1465f0 2022-06-29T15:50:22.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.119+0000 7fb621d85700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb604003750 0x7fb61c0b7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:22.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.119+0000 7fb621d85700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb604003750 0x7fb61c0b7d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52284/0 (socket says 172.21.15.47:52284) 2022-06-29T15:50:22.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.119+0000 7fb621d85700 1 -- 172.21.15.47:0/1236702168 learned_addr learned my addr 172.21.15.47:0/1236702168 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:22.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.119+0000 7fb621584700 1 --2- 172.21.15.47:0/1236702168 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb61c155ac0 0x7fb61c152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:22.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.119+0000 7fb622586700 1 --2- 172.21.15.47:0/1236702168 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb61c1465f0 0x7fb61c1530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:22.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.119+0000 7fb621d85700 1 -- 172.21.15.47:0/1236702168 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb61c1465f0 msgr2=0x7fb61c1530d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:22.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.119+0000 7fb621d85700 1 --2- 172.21.15.47:0/1236702168 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb61c1465f0 0x7fb61c1530d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:22.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.119+0000 7fb621d85700 1 -- 172.21.15.47:0/1236702168 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb61c155ac0 msgr2=0x7fb61c152be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:22.138 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.119+0000 7fb621d85700 1 --2- 172.21.15.47:0/1236702168 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb61c155ac0 0x7fb61c152be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:22.138 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.119+0000 7fb621d85700 1 -- 172.21.15.47:0/1236702168 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb61c1471d0 con 0x7fb604003750 2022-06-29T15:50:22.138 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.119+0000 7fb621d85700 1 --2- 172.21.15.47:0/1236702168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb604003750 0x7fb61c0b7d80 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 rx=0x7fb60c00c000 tx=0x7fb60c019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:22.138 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.119+0000 7fb60affd700 1 -- 172.21.15.47:0/1236702168 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb60c01b820 con 0x7fb604003750 2022-06-29T15:50:22.138 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.119+0000 7fb60affd700 1 -- 172.21.15.47:0/1236702168 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb60c000e90 con 0x7fb604003750 2022-06-29T15:50:22.139 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.119+0000 7fb622d87700 1 -- 172.21.15.47:0/1236702168 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb61c152740 con 0x7fb604003750 2022-06-29T15:50:22.139 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.119+0000 7fb60affd700 1 -- 172.21.15.47:0/1236702168 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb60c01bb70 con 0x7fb604003750 2022-06-29T15:50:22.139 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.119+0000 7fb622d87700 1 -- 172.21.15.47:0/1236702168 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb61c1461f0 con 0x7fb604003750 2022-06-29T15:50:22.140 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.119+0000 7fb622d87700 1 -- 172.21.15.47:0/1236702168 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb61c1528d0 con 0x7fb604003750 2022-06-29T15:50:22.140 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.119+0000 7fb60affd700 1 -- 172.21.15.47:0/1236702168 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb60c01bcd0 con 0x7fb604003750 2022-06-29T15:50:22.140 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.123+0000 7fb60affd700 1 --2- 172.21.15.47:0/1236702168 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb5fc053b00 0x7fb5fc055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:22.140 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.123+0000 7fb60affd700 1 -- 172.21.15.47:0/1236702168 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb60c06d290 con 0x7fb604003750 2022-06-29T15:50:22.140 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.123+0000 7fb621584700 1 --2- 172.21.15.47:0/1236702168 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb5fc053b00 0x7fb5fc055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:22.141 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.123+0000 7fb621584700 1 --2- 172.21.15.47:0/1236702168 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb5fc053b00 0x7fb5fc055fc0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7fb618005000 tx=0x7fb61c05cd80).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:22.141 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.127+0000 7fb60affd700 1 -- 172.21.15.47:0/1236702168 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb60c043fb0 con 0x7fb604003750 2022-06-29T15:50:22.262 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.255+0000 7fb622d87700 1 -- 172.21.15.47:0/1236702168 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb61c0b6930 con 0x7fb604003750 2022-06-29T15:50:22.264 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.259+0000 7fb60affd700 1 -- 172.21.15.47:0/1236702168 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+16285 (secure 0 0 0) 0x7fb60c0475d0 con 0x7fb604003750 2022-06-29T15:50:22.265 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:50:22.265 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":111,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:50:16.342650+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:50:13.772527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2022-06-29T15:50:14.675947+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:50:22.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.259+0000 7fb622d87700 1 -- 172.21.15.47:0/1236702168 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb5fc053b00 msgr2=0x7fb5fc055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:22.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.259+0000 7fb622d87700 1 --2- 172.21.15.47:0/1236702168 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb5fc053b00 0x7fb5fc055fc0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7fb618005000 tx=0x7fb61c05cd80).stop 2022-06-29T15:50:22.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.259+0000 7fb622d87700 1 -- 172.21.15.47:0/1236702168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb604003750 msgr2=0x7fb61c0b7d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:22.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.259+0000 7fb622d87700 1 --2- 172.21.15.47:0/1236702168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb604003750 0x7fb61c0b7d80 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 rx=0x7fb60c00c000 tx=0x7fb60c019000).stop 2022-06-29T15:50:22.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.259+0000 7fb622d87700 1 -- 172.21.15.47:0/1236702168 shutdown_connections 2022-06-29T15:50:22.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.259+0000 7fb622d87700 1 --2- 172.21.15.47:0/1236702168 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb61c1465f0 0x7fb61c1530d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:22.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.259+0000 7fb622d87700 1 --2- 172.21.15.47:0/1236702168 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb5fc053b00 0x7fb5fc055fc0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:22.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.259+0000 7fb622d87700 1 --2- 172.21.15.47:0/1236702168 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb61c155ac0 0x7fb61c152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:22.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.259+0000 7fb622d87700 1 --2- 172.21.15.47:0/1236702168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb604003750 0x7fb61c0b7d80 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:22.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.259+0000 7fb622d87700 1 -- 172.21.15.47:0/1236702168 >> 172.21.15.47:0/1236702168 conn(0x7fb61c060e00 msgr2=0x7fb61c1402c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:22.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.259+0000 7fb622d87700 1 -- 172.21.15.47:0/1236702168 shutdown_connections 2022-06-29T15:50:22.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.259+0000 7fb622d87700 1 -- 172.21.15.47:0/1236702168 wait complete. 2022-06-29T15:50:22.279 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-29T15:42:16.971431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-29T15:42:25.290884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-29T15:50:13.772527+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2022-06-29T15:50:14.675947+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-29T15:50:22.280 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-29T15:50:22.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.375+0000 7f0d6279d700 1 Processor -- start 2022-06-29T15:50:22.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.375+0000 7f0d6279d700 1 -- start start 2022-06-29T15:50:22.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.379+0000 7f0d6279d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0d5c064870 0x7f0d5c066f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:22.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.379+0000 7f0d6279d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0d5c0b2580 0x7f0d5c140fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:22.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.379+0000 7f0d6279d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0d5c0b2a50 0x7f0d5c144620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:22.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.379+0000 7f0d6279d700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d5c066620 con 0x7f0d5c0b2a50 2022-06-29T15:50:22.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.379+0000 7f0d6279d700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d5c145340 con 0x7f0d5c0b2580 2022-06-29T15:50:22.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.379+0000 7f0d6279d700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5c066850 con 0x7f0d5c064870 2022-06-29T15:50:22.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.379+0000 7f0d61f9c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0d5c0b2a50 0x7f0d5c144620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:22.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.379+0000 7f0d61f9c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0d5c0b2a50 0x7f0d5c144620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52294/0 (socket says 172.21.15.47:52294) 2022-06-29T15:50:22.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.379+0000 7f0d61f9c700 1 -- 172.21.15.47:0/2648886967 learned_addr learned my addr 172.21.15.47:0/2648886967 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:22.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.379+0000 7f0d60f9a700 1 --2- 172.21.15.47:0/2648886967 >> v2:172.21.15.64:3300/0 conn(0x7f0d5c0b2580 0x7f0d5c140fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:22.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.379+0000 7f0d6179b700 1 --2- 172.21.15.47:0/2648886967 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0d5c064870 0x7f0d5c066f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:22.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.379+0000 7f0d61f9c700 1 -- 172.21.15.47:0/2648886967 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0d5c064870 msgr2=0x7f0d5c066f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:22.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.379+0000 7f0d61f9c700 1 --2- 172.21.15.47:0/2648886967 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0d5c064870 0x7f0d5c066f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:22.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.379+0000 7f0d61f9c700 1 -- 172.21.15.47:0/2648886967 >> v2:172.21.15.64:3300/0 conn(0x7f0d5c0b2580 msgr2=0x7f0d5c140fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:22.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.379+0000 7f0d61f9c700 1 --2- 172.21.15.47:0/2648886967 >> v2:172.21.15.64:3300/0 conn(0x7f0d5c0b2580 0x7f0d5c140fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:22.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.379+0000 7f0d61f9c700 1 -- 172.21.15.47:0/2648886967 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d5c066af0 con 0x7f0d5c0b2a50 2022-06-29T15:50:22.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.379+0000 7f0d61f9c700 1 --2- 172.21.15.47:0/2648886967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d5c0b2a50 0x7f0d5c144620 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 rx=0x7f0d540032f0 tx=0x7f0d54009290).ready entity=mon.0 client_cookie=8fb16172ce26632e server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:22.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.379+0000 7f0d4bfff700 1 -- 172.21.15.47:0/2648886967 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d5400b870 con 0x7f0d5c0b2a50 2022-06-29T15:50:22.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.379+0000 7f0d4bfff700 1 -- 172.21.15.47:0/2648886967 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d5400b9d0 con 0x7f0d5c0b2a50 2022-06-29T15:50:22.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.379+0000 7f0d4bfff700 1 -- 172.21.15.47:0/2648886967 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d5400bb40 con 0x7f0d5c0b2a50 2022-06-29T15:50:22.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.383+0000 7f0d6279d700 1 -- 172.21.15.47:0/2648886967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d5c0b2a50 msgr2=0x7f0d5c144620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:22.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.383+0000 7f0d6279d700 1 --2- 172.21.15.47:0/2648886967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d5c0b2a50 0x7f0d5c144620 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 rx=0x7f0d540032f0 tx=0x7f0d54009290).stop 2022-06-29T15:50:22.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.383+0000 7f0d6279d700 1 -- 172.21.15.47:0/2648886967 shutdown_connections 2022-06-29T15:50:22.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.383+0000 7f0d6279d700 1 --2- 172.21.15.47:0/2648886967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d5c0b2a50 0x7f0d5c144620 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:22.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.383+0000 7f0d6279d700 1 --2- 172.21.15.47:0/2648886967 >> v2:172.21.15.64:3300/0 conn(0x7f0d5c0b2580 0x7f0d5c140fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:22.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.383+0000 7f0d6279d700 1 --2- 172.21.15.47:0/2648886967 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0d5c064870 0x7f0d5c066f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:22.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.383+0000 7f0d6279d700 1 -- 172.21.15.47:0/2648886967 >> 172.21.15.47:0/2648886967 conn(0x7f0d5c05c510 msgr2=0x7f0d5c05c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:22.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.383+0000 7f0d6279d700 1 -- 172.21.15.47:0/2648886967 shutdown_connections 2022-06-29T15:50:22.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.383+0000 7f0d6279d700 1 -- 172.21.15.47:0/2648886967 wait complete. 2022-06-29T15:50:22.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.383+0000 7f0d6279d700 1 Processor -- start 2022-06-29T15:50:22.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.383+0000 7f0d6279d700 1 -- start start 2022-06-29T15:50:22.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.383+0000 7f0d6279d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d5c064870 0x7f0d5c0ae080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:22.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.383+0000 7f0d6279d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0d5c0b2a50 0x7f0d5c0af410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:22.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.383+0000 7f0d6279d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d5c0b5480 0x7f0d5c0b0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:22.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.383+0000 7f0d6279d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5c14ea90 con 0x7f0d5c064870 2022-06-29T15:50:22.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.383+0000 7f0d6279d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5c14ecb0 con 0x7f0d5c0b5480 2022-06-29T15:50:22.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.383+0000 7f0d6279d700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5c14ef10 con 0x7f0d5c0b2a50 2022-06-29T15:50:22.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.383+0000 7f0d6179b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d5c064870 0x7f0d5c0ae080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:22.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.383+0000 7f0d60f9a700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0d5c0b2a50 0x7f0d5c0af410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:22.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.383+0000 7f0d6179b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d5c064870 0x7f0d5c0ae080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52298/0 (socket says 172.21.15.47:52298) 2022-06-29T15:50:22.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.383+0000 7f0d6179b700 1 -- 172.21.15.47:0/4061843741 learned_addr learned my addr 172.21.15.47:0/4061843741 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:22.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.383+0000 7f0d61f9c700 1 --2- 172.21.15.47:0/4061843741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d5c0b5480 0x7f0d5c0b0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:22.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.387+0000 7f0d6179b700 1 -- 172.21.15.47:0/4061843741 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0d5c0b2a50 msgr2=0x7f0d5c0af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:22.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.387+0000 7f0d6179b700 1 --2- 172.21.15.47:0/4061843741 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0d5c0b2a50 0x7f0d5c0af410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:22.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.387+0000 7f0d6179b700 1 -- 172.21.15.47:0/4061843741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d5c0b5480 msgr2=0x7f0d5c0b0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:22.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.387+0000 7f0d6179b700 1 --2- 172.21.15.47:0/4061843741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d5c0b5480 0x7f0d5c0b0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:22.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.387+0000 7f0d6179b700 1 -- 172.21.15.47:0/4061843741 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d54009530 con 0x7f0d5c064870 2022-06-29T15:50:22.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.387+0000 7f0d6179b700 1 --2- 172.21.15.47:0/4061843741 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d5c064870 0x7f0d5c0ae080 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 rx=0x7f0d580030b0 tx=0x7f0d5800b590).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:22.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.387+0000 7f0d4a7fc700 1 -- 172.21.15.47:0/4061843741 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d5800f6a0 con 0x7f0d5c064870 2022-06-29T15:50:22.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.387+0000 7f0d4a7fc700 1 -- 172.21.15.47:0/4061843741 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d58013040 con 0x7f0d5c064870 2022-06-29T15:50:22.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.387+0000 7f0d6279d700 1 -- 172.21.15.47:0/4061843741 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d5c0ab7e0 con 0x7f0d5c064870 2022-06-29T15:50:22.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.387+0000 7f0d4a7fc700 1 -- 172.21.15.47:0/4061843741 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d5800f9f0 con 0x7f0d5c064870 2022-06-29T15:50:22.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.387+0000 7f0d6279d700 1 -- 172.21.15.47:0/4061843741 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d5c0b12d0 con 0x7f0d5c064870 2022-06-29T15:50:22.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.387+0000 7f0d6279d700 1 -- 172.21.15.47:0/4061843741 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d5c069d00 con 0x7f0d5c064870 2022-06-29T15:50:22.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.387+0000 7f0d4a7fc700 1 -- 172.21.15.47:0/4061843741 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0d5800fb50 con 0x7f0d5c064870 2022-06-29T15:50:22.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.387+0000 7f0d4a7fc700 1 --2- 172.21.15.47:0/4061843741 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0d3c053950 0x7f0d3c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:22.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.387+0000 7f0d4a7fc700 1 -- 172.21.15.47:0/4061843741 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f0d5806fdd0 con 0x7f0d5c064870 2022-06-29T15:50:22.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.387+0000 7f0d60f9a700 1 --2- 172.21.15.47:0/4061843741 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0d3c053950 0x7f0d3c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:22.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.391+0000 7f0d60f9a700 1 --2- 172.21.15.47:0/4061843741 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0d3c053950 0x7f0d3c055e10 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7f0d4c00b000 tx=0x7f0d4c006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:22.409 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.395+0000 7f0d4a7fc700 1 -- 172.21.15.47:0/4061843741 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0d5c069d00 con 0x7f0d5c064870 2022-06-29T15:50:22.529 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.523+0000 7f0d6279d700 1 -- 172.21.15.47:0/4061843741 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f0d5c069d00 con 0x7f0d5c064870 2022-06-29T15:50:22.530 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.523+0000 7f0d4a7fc700 1 -- 172.21.15.47:0/4061843741 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v111) v1 ==== 110+0+10 (secure 0 0 0) 0x7f0d5c069d00 con 0x7f0d5c064870 2022-06-29T15:50:22.531 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2022-06-29T15:50:22.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.523+0000 7f0d6279d700 1 -- 172.21.15.47:0/4061843741 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0d3c053950 msgr2=0x7f0d3c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:22.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.523+0000 7f0d6279d700 1 --2- 172.21.15.47:0/4061843741 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0d3c053950 0x7f0d3c055e10 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7f0d4c00b000 tx=0x7f0d4c006df0).stop 2022-06-29T15:50:22.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.523+0000 7f0d6279d700 1 -- 172.21.15.47:0/4061843741 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d5c064870 msgr2=0x7f0d5c0ae080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:22.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.523+0000 7f0d6279d700 1 --2- 172.21.15.47:0/4061843741 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d5c064870 0x7f0d5c0ae080 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 rx=0x7f0d580030b0 tx=0x7f0d5800b590).stop 2022-06-29T15:50:22.536 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.527+0000 7f0d6279d700 1 -- 172.21.15.47:0/4061843741 shutdown_connections 2022-06-29T15:50:22.536 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.527+0000 7f0d6279d700 1 --2- 172.21.15.47:0/4061843741 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0d3c053950 0x7f0d3c055e10 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:22.537 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.527+0000 7f0d6279d700 1 --2- 172.21.15.47:0/4061843741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d5c0b5480 0x7f0d5c0b0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:22.537 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.527+0000 7f0d6279d700 1 --2- 172.21.15.47:0/4061843741 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0d5c0b2a50 0x7f0d5c0af410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:22.538 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.527+0000 7f0d6279d700 1 --2- 172.21.15.47:0/4061843741 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d5c064870 0x7f0d5c0ae080 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:22.538 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.527+0000 7f0d6279d700 1 -- 172.21.15.47:0/4061843741 >> 172.21.15.47:0/4061843741 conn(0x7f0d5c05c510 msgr2=0x7f0d5c0b39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:22.539 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.527+0000 7f0d6279d700 1 -- 172.21.15.47:0/4061843741 shutdown_connections 2022-06-29T15:50:22.539 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.527+0000 7f0d6279d700 1 -- 172.21.15.47:0/4061843741 wait complete. 2022-06-29T15:50:22.543 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-29T15:50:22.651 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.643+0000 7ffa1cbb9700 1 Processor -- start 2022-06-29T15:50:22.651 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.643+0000 7ffa1cbb9700 1 -- start start 2022-06-29T15:50:22.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.647+0000 7ffa1cbb9700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffa1805c850 0x7ffa18143a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:22.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.647+0000 7ffa1cbb9700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ffa180676a0 0x7ffa18140130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:22.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.647+0000 7ffa1cbb9700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa1805cc30 con 0x7ffa180b5d90 2022-06-29T15:50:22.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.647+0000 7ffa1cbb9700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa181430a0 con 0x7ffa180676a0 2022-06-29T15:50:22.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.647+0000 7ffa1cbb9700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa18143300 con 0x7ffa1805c850 2022-06-29T15:50:22.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.647+0000 7ffa17fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ffa180676a0 0x7ffa18140130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:22.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.647+0000 7ffa17fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ffa180676a0 0x7ffa18140130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52308/0 (socket says 172.21.15.47:52308) 2022-06-29T15:50:22.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.647+0000 7ffa0effd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ffa180b5d90 0x7ffa180b4860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37860/0 (socket says 172.21.15.47:37860) 2022-06-29T15:50:22.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.647+0000 7ffa17fff700 1 -- 172.21.15.47:0/2118458893 learned_addr learned my addr 172.21.15.47:0/2118458893 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:22.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.647+0000 7ffa177fe700 1 --2- 172.21.15.47:0/2118458893 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffa1805c850 0x7ffa18143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:22.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.647+0000 7ffa17fff700 1 -- 172.21.15.47:0/2118458893 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffa1805c850 msgr2=0x7ffa18143a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:22.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.647+0000 7ffa17fff700 1 --2- 172.21.15.47:0/2118458893 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffa1805c850 0x7ffa18143a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:22.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.647+0000 7ffa17fff700 1 -- 172.21.15.47:0/2118458893 >> v1:172.21.15.47:6789/0 conn(0x7ffa180b5d90 legacy=0x7ffa180b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:22.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.647+0000 7ffa17fff700 1 -- 172.21.15.47:0/2118458893 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa181435a0 con 0x7ffa180676a0 2022-06-29T15:50:22.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.647+0000 7ffa16ffd700 1 -- 172.21.15.47:0/2118458893 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3014307482 0 0) 0x7ffa1805cc30 con 0x7ffa180b5d90 2022-06-29T15:50:22.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.647+0000 7ffa17fff700 1 --2- 172.21.15.47:0/2118458893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa180676a0 0x7ffa18140130 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 rx=0x7ffa100032f0 tx=0x7ffa10009290).ready entity=mon.0 client_cookie=c154296073836c8b server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:22.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.647+0000 7ffa16ffd700 1 -- 172.21.15.47:0/2118458893 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa1000b8a0 con 0x7ffa180676a0 2022-06-29T15:50:22.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.647+0000 7ffa16ffd700 1 -- 172.21.15.47:0/2118458893 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa1000ba00 con 0x7ffa180676a0 2022-06-29T15:50:22.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.647+0000 7ffa16ffd700 1 -- 172.21.15.47:0/2118458893 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa1000bb70 con 0x7ffa180676a0 2022-06-29T15:50:22.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.647+0000 7ffa1cbb9700 1 -- 172.21.15.47:0/2118458893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa180676a0 msgr2=0x7ffa18140130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:22.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.647+0000 7ffa1cbb9700 1 --2- 172.21.15.47:0/2118458893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa180676a0 0x7ffa18140130 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 rx=0x7ffa100032f0 tx=0x7ffa10009290).stop 2022-06-29T15:50:22.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.647+0000 7ffa1cbb9700 1 -- 172.21.15.47:0/2118458893 shutdown_connections 2022-06-29T15:50:22.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.647+0000 7ffa1cbb9700 1 --2- 172.21.15.47:0/2118458893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa180676a0 0x7ffa18140130 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:22.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.647+0000 7ffa1cbb9700 1 --2- 172.21.15.47:0/2118458893 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffa1805c850 0x7ffa18143a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:22.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.647+0000 7ffa1cbb9700 1 -- 172.21.15.47:0/2118458893 >> 172.21.15.47:0/2118458893 conn(0x7ffa18060e00 msgr2=0x7ffa180611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:22.663 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.647+0000 7ffa1cbb9700 1 -- 172.21.15.47:0/2118458893 shutdown_connections 2022-06-29T15:50:22.663 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.647+0000 7ffa1cbb9700 1 -- 172.21.15.47:0/2118458893 wait complete. 2022-06-29T15:50:22.666 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.651+0000 7ffa1cbb9700 1 Processor -- start 2022-06-29T15:50:22.666 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.651+0000 7ffa1cbb9700 1 -- start start 2022-06-29T15:50:22.666 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.651+0000 7ffa1cbb9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa1805c850 0x7ffa1812bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:22.667 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.651+0000 7ffa1cbb9700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffa180676a0 0x7ffa1812c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:22.667 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.651+0000 7ffa1cbb9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa180b5d90 0x7ffa1813f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:22.667 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.651+0000 7ffa1cbb9700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa180b3720 con 0x7ffa1805c850 2022-06-29T15:50:22.667 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.651+0000 7ffa1cbb9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1814ff60 con 0x7ffa180b5d90 2022-06-29T15:50:22.668 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.651+0000 7ffa1cbb9700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa18152eb0 con 0x7ffa180676a0 2022-06-29T15:50:22.668 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.651+0000 7ffa177fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa1805c850 0x7ffa1812bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:22.668 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.651+0000 7ffa177fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa1805c850 0x7ffa1812bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52310/0 (socket says 172.21.15.47:52310) 2022-06-29T15:50:22.668 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.651+0000 7ffa177fe700 1 -- 172.21.15.47:0/1768933628 learned_addr learned my addr 172.21.15.47:0/1768933628 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:22.669 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.651+0000 7ffa0effd700 1 --2- 172.21.15.47:0/1768933628 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffa180676a0 0x7ffa1812c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:22.669 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.651+0000 7ffa17fff700 1 --2- 172.21.15.47:0/1768933628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa180b5d90 0x7ffa1813f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:22.669 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.651+0000 7ffa177fe700 1 -- 172.21.15.47:0/1768933628 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffa180676a0 msgr2=0x7ffa1812c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:22.669 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.651+0000 7ffa177fe700 1 --2- 172.21.15.47:0/1768933628 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffa180676a0 0x7ffa1812c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:22.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.651+0000 7ffa177fe700 1 -- 172.21.15.47:0/1768933628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa180b5d90 msgr2=0x7ffa1813f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:22.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.651+0000 7ffa177fe700 1 --2- 172.21.15.47:0/1768933628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa180b5d90 0x7ffa1813f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:22.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.651+0000 7ffa177fe700 1 -- 172.21.15.47:0/1768933628 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa10009530 con 0x7ffa1805c850 2022-06-29T15:50:22.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.651+0000 7ffa177fe700 1 --2- 172.21.15.47:0/1768933628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa1805c850 0x7ffa1812bfd0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 rx=0x7ffa0800a000 tx=0x7ffa08008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:22.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.651+0000 7ffa157fa700 1 -- 172.21.15.47:0/1768933628 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa08011780 con 0x7ffa1805c850 2022-06-29T15:50:22.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.655+0000 7ffa157fa700 1 -- 172.21.15.47:0/1768933628 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa080118e0 con 0x7ffa1805c850 2022-06-29T15:50:22.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.655+0000 7ffa1cbb9700 1 -- 172.21.15.47:0/1768933628 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa18152c30 con 0x7ffa1805c850 2022-06-29T15:50:22.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.655+0000 7ffa157fa700 1 -- 172.21.15.47:0/1768933628 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa08011c40 con 0x7ffa1805c850 2022-06-29T15:50:22.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.655+0000 7ffa1cbb9700 1 -- 172.21.15.47:0/1768933628 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa180b3bb0 con 0x7ffa1805c850 2022-06-29T15:50:22.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.655+0000 7ffa1cbb9700 1 -- 172.21.15.47:0/1768933628 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa180b3540 con 0x7ffa1805c850 2022-06-29T15:50:22.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.655+0000 7ffa157fa700 1 -- 172.21.15.47:0/1768933628 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ffa0801c460 con 0x7ffa1805c850 2022-06-29T15:50:22.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.655+0000 7ffa157fa700 1 --2- 172.21.15.47:0/1768933628 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff9f0053b00 0x7ff9f0055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:22.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.655+0000 7ffa0effd700 1 --2- 172.21.15.47:0/1768933628 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff9f0053b00 0x7ff9f0055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:22.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.655+0000 7ffa157fa700 1 -- 172.21.15.47:0/1768933628 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ffa0806c3c0 con 0x7ffa1805c850 2022-06-29T15:50:22.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.655+0000 7ffa0effd700 1 --2- 172.21.15.47:0/1768933628 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff9f0053b00 0x7ff9f0055fc0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7ffa0000d000 tx=0x7ffa00009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:22.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.663+0000 7ffa157fa700 1 -- 172.21.15.47:0/1768933628 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ffa08043120 con 0x7ffa1805c850 2022-06-29T15:50:22.793 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.787+0000 7ffa1cbb9700 1 -- 172.21.15.47:0/1768933628 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ffa180573d0 con 0x7ffa1805c850 2022-06-29T15:50:22.794 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.787+0000 7ffa157fa700 1 -- 172.21.15.47:0/1768933628 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v111) v1 ==== 92+0+10 (secure 0 0 0) 0x7ffa08046740 con 0x7ffa1805c850 2022-06-29T15:50:22.794 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:50:22.798 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.791+0000 7ffa1cbb9700 1 -- 172.21.15.47:0/1768933628 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff9f0053b00 msgr2=0x7ff9f0055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:22.798 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.791+0000 7ffa1cbb9700 1 --2- 172.21.15.47:0/1768933628 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff9f0053b00 0x7ff9f0055fc0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7ffa0000d000 tx=0x7ffa00009040).stop 2022-06-29T15:50:22.799 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.791+0000 7ffa1cbb9700 1 -- 172.21.15.47:0/1768933628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa1805c850 msgr2=0x7ffa1812bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:22.799 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.791+0000 7ffa1cbb9700 1 --2- 172.21.15.47:0/1768933628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa1805c850 0x7ffa1812bfd0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 rx=0x7ffa0800a000 tx=0x7ffa08008040).stop 2022-06-29T15:50:22.799 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.791+0000 7ffa1cbb9700 1 -- 172.21.15.47:0/1768933628 shutdown_connections 2022-06-29T15:50:22.799 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.791+0000 7ffa1cbb9700 1 --2- 172.21.15.47:0/1768933628 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff9f0053b00 0x7ff9f0055fc0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:22.800 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.791+0000 7ffa1cbb9700 1 --2- 172.21.15.47:0/1768933628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa180b5d90 0x7ffa1813f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:22.800 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.791+0000 7ffa1cbb9700 1 --2- 172.21.15.47:0/1768933628 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffa180676a0 0x7ffa1812c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:22.800 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.791+0000 7ffa1cbb9700 1 --2- 172.21.15.47:0/1768933628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa1805c850 0x7ffa1812bfd0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:22.800 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.791+0000 7ffa1cbb9700 1 -- 172.21.15.47:0/1768933628 >> 172.21.15.47:0/1768933628 conn(0x7ffa18060e00 msgr2=0x7ffa18146c30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:22.800 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.791+0000 7ffa1cbb9700 1 -- 172.21.15.47:0/1768933628 shutdown_connections 2022-06-29T15:50:22.801 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.791+0000 7ffa1cbb9700 1 -- 172.21.15.47:0/1768933628 wait complete. 2022-06-29T15:50:22.813 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-29T15:50:22.921 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.915+0000 7fcf09cf8700 1 Processor -- start 2022-06-29T15:50:22.921 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.915+0000 7fcf09cf8700 1 -- start start 2022-06-29T15:50:22.923 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.915+0000 7fcf09cf8700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcf040abae0 0x7fcf041432b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:22.923 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.915+0000 7fcf09cf8700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcf0413f7b0 0x7fcf041437a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:22.923 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.915+0000 7fcf09cf8700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf040b0e80 con 0x7fcf040a9250 2022-06-29T15:50:22.923 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.915+0000 7fcf09cf8700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf040abec0 con 0x7fcf040abae0 2022-06-29T15:50:22.924 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.915+0000 7fcf09cf8700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf040b0940 con 0x7fcf0413f7b0 2022-06-29T15:50:22.924 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.915+0000 7fcf08cf6700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcf040abae0 0x7fcf041432b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:22.924 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.915+0000 7fcf08cf6700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcf040abae0 0x7fcf041432b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52318/0 (socket says 172.21.15.47:52318) 2022-06-29T15:50:22.924 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.915+0000 7fcf08cf6700 1 -- 172.21.15.47:0/590019118 learned_addr learned my addr 172.21.15.47:0/590019118 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:22.925 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.915+0000 7fcf03fff700 1 --2- 172.21.15.47:0/590019118 >> v2:172.21.15.64:3300/0 conn(0x7fcf0413f7b0 0x7fcf041437a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:22.927 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.915+0000 7fcf08cf6700 1 -- 172.21.15.47:0/590019118 >> v2:172.21.15.64:3300/0 conn(0x7fcf0413f7b0 msgr2=0x7fcf041437a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:22.927 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.919+0000 7fcf08cf6700 1 --2- 172.21.15.47:0/590019118 >> v2:172.21.15.64:3300/0 conn(0x7fcf0413f7b0 0x7fcf041437a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:22.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.919+0000 7fcf08cf6700 1 -- 172.21.15.47:0/590019118 >> v1:172.21.15.64:6789/0 conn(0x7fcf040a9250 legacy=0x7fcf0413fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:22.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.919+0000 7fcf037fe700 1 -- 172.21.15.47:0/590019118 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1129962784 0 0) 0x7fcf040b0e80 con 0x7fcf040a9250 2022-06-29T15:50:22.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.919+0000 7fcf08cf6700 1 -- 172.21.15.47:0/590019118 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf040b0be0 con 0x7fcf040abae0 2022-06-29T15:50:22.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.919+0000 7fcf03fff700 1 --2- 172.21.15.47:0/590019118 >> v2:172.21.15.64:3300/0 conn(0x7fcf0413f7b0 0x7fcf041437a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:50:22.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.919+0000 7fcf08cf6700 1 --2- 172.21.15.47:0/590019118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcf040abae0 0x7fcf041432b0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 rx=0x7fcef8002f70 tx=0x7fcef8006f40).ready entity=mon.0 client_cookie=3dc7106938f17daa server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:22.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.919+0000 7fcf037fe700 1 -- 172.21.15.47:0/590019118 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcef800da80 con 0x7fcf040abae0 2022-06-29T15:50:22.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.919+0000 7fcf037fe700 1 -- 172.21.15.47:0/590019118 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef8000e90 con 0x7fcf040abae0 2022-06-29T15:50:22.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.919+0000 7fcf037fe700 1 -- 172.21.15.47:0/590019118 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcef800ddd0 con 0x7fcf040abae0 2022-06-29T15:50:22.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.919+0000 7fcf09cf8700 1 -- 172.21.15.47:0/590019118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcf040abae0 msgr2=0x7fcf041432b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:22.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.919+0000 7fcf09cf8700 1 --2- 172.21.15.47:0/590019118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcf040abae0 0x7fcf041432b0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 rx=0x7fcef8002f70 tx=0x7fcef8006f40).stop 2022-06-29T15:50:22.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.919+0000 7fcf09cf8700 1 -- 172.21.15.47:0/590019118 shutdown_connections 2022-06-29T15:50:22.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.919+0000 7fcf09cf8700 1 --2- 172.21.15.47:0/590019118 >> v2:172.21.15.64:3300/0 conn(0x7fcf0413f7b0 0x7fcf041437a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:22.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.919+0000 7fcf09cf8700 1 --2- 172.21.15.47:0/590019118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcf040abae0 0x7fcf041432b0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:22.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.919+0000 7fcf09cf8700 1 -- 172.21.15.47:0/590019118 >> 172.21.15.47:0/590019118 conn(0x7fcf04062560 msgr2=0x7fcf04062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:22.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.919+0000 7fcf09cf8700 1 -- 172.21.15.47:0/590019118 shutdown_connections 2022-06-29T15:50:22.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.919+0000 7fcf09cf8700 1 -- 172.21.15.47:0/590019118 wait complete. 2022-06-29T15:50:22.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.919+0000 7fcf09cf8700 1 Processor -- start 2022-06-29T15:50:22.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.919+0000 7fcf09cf8700 1 -- start start 2022-06-29T15:50:22.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.923+0000 7fcf09cf8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcf040a9250 0x7fcf04142710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:22.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.923+0000 7fcf09cf8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf040abae0 0x7fcf04063160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:22.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.923+0000 7fcf09cf8700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcf04141f30 0x7fcf040669e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:22.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.923+0000 7fcf09cf8700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf040af800 con 0x7fcf040a9250 2022-06-29T15:50:22.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.923+0000 7fcf09cf8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf040b4820 con 0x7fcf040abae0 2022-06-29T15:50:22.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.923+0000 7fcf09cf8700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf041566e0 con 0x7fcf04141f30 2022-06-29T15:50:22.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.923+0000 7fcf08cf6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcf040a9250 0x7fcf04142710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:22.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.923+0000 7fcf08cf6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcf040a9250 0x7fcf04142710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52324/0 (socket says 172.21.15.47:52324) 2022-06-29T15:50:22.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.923+0000 7fcf08cf6700 1 -- 172.21.15.47:0/4251142997 learned_addr learned my addr 172.21.15.47:0/4251142997 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:22.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.923+0000 7fcf094f7700 1 --2- 172.21.15.47:0/4251142997 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcf04141f30 0x7fcf040669e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:22.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.923+0000 7fcf03fff700 1 --2- 172.21.15.47:0/4251142997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf040abae0 0x7fcf04063160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:22.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.923+0000 7fcf08cf6700 1 -- 172.21.15.47:0/4251142997 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcf04141f30 msgr2=0x7fcf040669e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:22.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.923+0000 7fcf08cf6700 1 --2- 172.21.15.47:0/4251142997 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcf04141f30 0x7fcf040669e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:22.938 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.923+0000 7fcf08cf6700 1 -- 172.21.15.47:0/4251142997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf040abae0 msgr2=0x7fcf04063160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:22.938 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.923+0000 7fcf08cf6700 1 --2- 172.21.15.47:0/4251142997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf040abae0 0x7fcf04063160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:22.938 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.923+0000 7fcf08cf6700 1 -- 172.21.15.47:0/4251142997 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef4009470 con 0x7fcf040a9250 2022-06-29T15:50:22.938 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.923+0000 7fcf03fff700 1 --2- 172.21.15.47:0/4251142997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf040abae0 0x7fcf04063160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:50:22.939 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.923+0000 7fcf08cf6700 1 --2- 172.21.15.47:0/4251142997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcf040a9250 0x7fcf04142710 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 rx=0x7fcef8002f70 tx=0x7fcf040aa9c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:22.939 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.923+0000 7fcf01ffb700 1 -- 172.21.15.47:0/4251142997 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcef801d070 con 0x7fcf040a9250 2022-06-29T15:50:22.939 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.923+0000 7fcf01ffb700 1 -- 172.21.15.47:0/4251142997 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef800de80 con 0x7fcf040a9250 2022-06-29T15:50:22.939 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.923+0000 7fcf09cf8700 1 -- 172.21.15.47:0/4251142997 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcef800b460 con 0x7fcf040a9250 2022-06-29T15:50:22.940 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.923+0000 7fcf01ffb700 1 -- 172.21.15.47:0/4251142997 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcef8018800 con 0x7fcf040a9250 2022-06-29T15:50:22.940 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.923+0000 7fcf09cf8700 1 -- 172.21.15.47:0/4251142997 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf040af360 con 0x7fcf040a9250 2022-06-29T15:50:22.940 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.923+0000 7fcf09cf8700 1 -- 172.21.15.47:0/4251142997 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf04142340 con 0x7fcf040a9250 2022-06-29T15:50:22.940 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.927+0000 7fcf01ffb700 1 -- 172.21.15.47:0/4251142997 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fcef8018960 con 0x7fcf040a9250 2022-06-29T15:50:22.940 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.927+0000 7fcf01ffb700 1 --2- 172.21.15.47:0/4251142997 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcee4053980 0x7fcee4055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:22.941 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.927+0000 7fcf01ffb700 1 -- 172.21.15.47:0/4251142997 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fcef806d0a0 con 0x7fcf040a9250 2022-06-29T15:50:22.941 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.927+0000 7fcf03fff700 1 --2- 172.21.15.47:0/4251142997 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcee4053980 0x7fcee4055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:22.941 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.927+0000 7fcf03fff700 1 --2- 172.21.15.47:0/4251142997 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcee4053980 0x7fcee4055e40 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7fcef400b000 tx=0x7fcef4008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:22.942 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:22.931+0000 7fcf01ffb700 1 -- 172.21.15.47:0/4251142997 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcef8043e60 con 0x7fcf040a9250 2022-06-29T15:50:23.068 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.063+0000 7fcf09cf8700 1 -- 172.21.15.47:0/4251142997 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcf040af520 con 0x7fcf040a9250 2022-06-29T15:50:23.069 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.063+0000 7fcf01ffb700 1 -- 172.21.15.47:0/4251142997 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v111) v1 ==== 104+0+10 (secure 0 0 0) 0x7fcef8047480 con 0x7fcf040a9250 2022-06-29T15:50:23.070 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:50:23.072 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.063+0000 7fcf09cf8700 1 -- 172.21.15.47:0/4251142997 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcee4053980 msgr2=0x7fcee4055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:23.072 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.063+0000 7fcf09cf8700 1 --2- 172.21.15.47:0/4251142997 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcee4053980 0x7fcee4055e40 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7fcef400b000 tx=0x7fcef4008040).stop 2022-06-29T15:50:23.072 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.063+0000 7fcf09cf8700 1 -- 172.21.15.47:0/4251142997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcf040a9250 msgr2=0x7fcf04142710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:23.072 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.063+0000 7fcf09cf8700 1 --2- 172.21.15.47:0/4251142997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcf040a9250 0x7fcf04142710 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 rx=0x7fcef8002f70 tx=0x7fcf040aa9c0).stop 2022-06-29T15:50:23.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.067+0000 7fcf09cf8700 1 -- 172.21.15.47:0/4251142997 shutdown_connections 2022-06-29T15:50:23.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.067+0000 7fcf09cf8700 1 --2- 172.21.15.47:0/4251142997 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcf04141f30 0x7fcf040669e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:23.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.067+0000 7fcf09cf8700 1 --2- 172.21.15.47:0/4251142997 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcee4053980 0x7fcee4055e40 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:23.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.067+0000 7fcf09cf8700 1 --2- 172.21.15.47:0/4251142997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf040abae0 0x7fcf04063160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:23.074 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.067+0000 7fcf09cf8700 1 --2- 172.21.15.47:0/4251142997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcf040a9250 0x7fcf04142710 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:23.074 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.067+0000 7fcf09cf8700 1 -- 172.21.15.47:0/4251142997 >> 172.21.15.47:0/4251142997 conn(0x7fcf04062560 msgr2=0x7fcf04062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:23.074 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.067+0000 7fcf09cf8700 1 -- 172.21.15.47:0/4251142997 shutdown_connections 2022-06-29T15:50:23.074 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.067+0000 7fcf09cf8700 1 -- 172.21.15.47:0/4251142997 wait complete. 2022-06-29T15:50:23.083 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-29T15:50:23.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.191+0000 7f08b358c700 1 Processor -- start 2022-06-29T15:50:23.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.191+0000 7f08b358c700 1 -- start start 2022-06-29T15:50:23.200 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.191+0000 7f08b358c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f08ac14ca80 0x7f08ac14ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:23.200 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.191+0000 7f08b358c700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08ac05f7c0 con 0x7f08ac140940 2022-06-29T15:50:23.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.191+0000 7f08b358c700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08ac0b4e50 con 0x7f08ac0b30e0 2022-06-29T15:50:23.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.191+0000 7f08b358c700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac0621e0 con 0x7f08ac14ca80 2022-06-29T15:50:23.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.191+0000 7f08b1328700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f08ac140940 0x7f08ac05f0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37888/0 (socket says 172.21.15.47:37888) 2022-06-29T15:50:23.202 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.191+0000 7f08b1328700 1 -- 172.21.15.47:0/1171637334 learned_addr learned my addr 172.21.15.47:0/1171637334 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:23.202 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.191+0000 7f08b1b29700 1 --2- 172.21.15.47:0/1171637334 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f08ac14ca80 0x7f08ac14ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:23.202 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.191+0000 7f0897fff700 1 -- 172.21.15.47:0/1171637334 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1043043273 0 0) 0x7f08ac05f7c0 con 0x7f08ac140940 2022-06-29T15:50:23.202 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.191+0000 7f0897fff700 1 -- 172.21.15.47:0/1171637334 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0890003260 con 0x7f08ac140940 2022-06-29T15:50:23.203 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.191+0000 7f0897fff700 1 -- 172.21.15.47:0/1171637334 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2045266927 0 0) 0x7f08ac0b4e50 con 0x7f08ac0b30e0 2022-06-29T15:50:23.203 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.191+0000 7f0897fff700 1 -- 172.21.15.47:0/1171637334 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08ac05f7c0 con 0x7f08ac0b30e0 2022-06-29T15:50:23.203 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.191+0000 7f08b1b29700 1 -- 172.21.15.47:0/1171637334 >> v1:172.21.15.64:6789/0 conn(0x7f08ac0b30e0 legacy=0x7f08ac0b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:23.204 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.191+0000 7f0897fff700 1 -- 172.21.15.47:0/1171637334 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2333035344 0 0) 0x7f0890003260 con 0x7f08ac140940 2022-06-29T15:50:23.204 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.191+0000 7f08b1b29700 1 -- 172.21.15.47:0/1171637334 >> v1:172.21.15.47:6789/0 conn(0x7f08ac140940 legacy=0x7f08ac05f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:23.204 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.191+0000 7f08b1b29700 1 -- 172.21.15.47:0/1171637334 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08ac143170 con 0x7f08ac14ca80 2022-06-29T15:50:23.204 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.191+0000 7f08b1b29700 1 --2- 172.21.15.47:0/1171637334 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f08ac14ca80 0x7f08ac14ee70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f08a0001110 tx=0x7f08a40091d0).ready entity=mon.2 client_cookie=89427ed5b422428d server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:23.205 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.191+0000 7f0897fff700 1 -- 172.21.15.47:0/1171637334 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f08a0001680 con 0x7f08ac14ca80 2022-06-29T15:50:23.205 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.191+0000 7f0897fff700 1 -- 172.21.15.47:0/1171637334 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a400f510 con 0x7f08ac14ca80 2022-06-29T15:50:23.205 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.191+0000 7f0897fff700 1 -- 172.21.15.47:0/1171637334 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f08a400f680 con 0x7f08ac14ca80 2022-06-29T15:50:23.205 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.195+0000 7f08b358c700 1 -- 172.21.15.47:0/1171637334 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f08ac14ca80 msgr2=0x7f08ac14ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:23.205 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.195+0000 7f08b358c700 1 --2- 172.21.15.47:0/1171637334 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f08ac14ca80 0x7f08ac14ee70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f08a0001110 tx=0x7f08a40091d0).stop 2022-06-29T15:50:23.206 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.195+0000 7f08b358c700 1 -- 172.21.15.47:0/1171637334 shutdown_connections 2022-06-29T15:50:23.206 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.195+0000 7f08b358c700 1 --2- 172.21.15.47:0/1171637334 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f08ac14ca80 0x7f08ac14ee70 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:23.206 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.195+0000 7f08b358c700 1 -- 172.21.15.47:0/1171637334 >> 172.21.15.47:0/1171637334 conn(0x7f08ac0627d0 msgr2=0x7f08ac062bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:23.206 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.195+0000 7f08b358c700 1 -- 172.21.15.47:0/1171637334 shutdown_connections 2022-06-29T15:50:23.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.195+0000 7f08b358c700 1 -- 172.21.15.47:0/1171637334 wait complete. 2022-06-29T15:50:23.210 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.195+0000 7f08b358c700 1 Processor -- start 2022-06-29T15:50:23.210 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.195+0000 7f08b358c700 1 -- start start 2022-06-29T15:50:23.211 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.195+0000 7f08b358c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f08ac0b30e0 0x7f08ac0ad990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:23.211 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.195+0000 7f08b358c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08ac140940 0x7f08ac0ade80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:23.211 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.195+0000 7f08b358c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08ac14ca80 0x7f08ac13f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:23.211 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.195+0000 7f08b358c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac0ae370 con 0x7f08ac140940 2022-06-29T15:50:23.212 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.195+0000 7f08b358c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac138880 con 0x7f08ac14ca80 2022-06-29T15:50:23.212 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.195+0000 7f08b358c700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac139ce0 con 0x7f08ac0b30e0 2022-06-29T15:50:23.212 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.195+0000 7f08b0b27700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08ac140940 0x7f08ac0ade80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:23.212 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.195+0000 7f08b0b27700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08ac140940 0x7f08ac0ade80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52340/0 (socket says 172.21.15.47:52340) 2022-06-29T15:50:23.213 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.195+0000 7f08b0b27700 1 -- 172.21.15.47:0/1496251802 learned_addr learned my addr 172.21.15.47:0/1496251802 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:23.213 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.199+0000 7f08b1328700 1 --2- 172.21.15.47:0/1496251802 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f08ac0b30e0 0x7f08ac0ad990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:23.213 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.199+0000 7f08b1b29700 1 --2- 172.21.15.47:0/1496251802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08ac14ca80 0x7f08ac13f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:23.213 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.199+0000 7f08b0b27700 1 -- 172.21.15.47:0/1496251802 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f08ac0b30e0 msgr2=0x7f08ac0ad990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:23.214 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.199+0000 7f08b0b27700 1 --2- 172.21.15.47:0/1496251802 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f08ac0b30e0 0x7f08ac0ad990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:23.214 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.199+0000 7f08b0b27700 1 -- 172.21.15.47:0/1496251802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08ac14ca80 msgr2=0x7f08ac13f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:23.214 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.199+0000 7f08b0b27700 1 --2- 172.21.15.47:0/1496251802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08ac14ca80 0x7f08ac13f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:23.214 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.199+0000 7f08b0b27700 1 -- 172.21.15.47:0/1496251802 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a4009470 con 0x7f08ac140940 2022-06-29T15:50:23.215 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.199+0000 7f08b0b27700 1 --2- 172.21.15.47:0/1496251802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08ac140940 0x7f08ac0ade80 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 rx=0x7f08a8001110 tx=0x7f08a8007ae0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:23.215 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.199+0000 7f089effd700 1 -- 172.21.15.47:0/1496251802 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f08a800f470 con 0x7f08ac140940 2022-06-29T15:50:23.215 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.199+0000 7f089effd700 1 -- 172.21.15.47:0/1496251802 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a800f5d0 con 0x7f08ac140940 2022-06-29T15:50:23.215 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.199+0000 7f08b358c700 1 -- 172.21.15.47:0/1496251802 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08ac138ab0 con 0x7f08ac140940 2022-06-29T15:50:23.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.199+0000 7f089effd700 1 -- 172.21.15.47:0/1496251802 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f08a800f930 con 0x7f08ac140940 2022-06-29T15:50:23.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.199+0000 7f08b358c700 1 -- 172.21.15.47:0/1496251802 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08ac138eb0 con 0x7f08ac140940 2022-06-29T15:50:23.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.199+0000 7f089effd700 1 -- 172.21.15.47:0/1496251802 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f08a8019030 con 0x7f08ac140940 2022-06-29T15:50:23.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.199+0000 7f08b358c700 1 -- 172.21.15.47:0/1496251802 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08ac0573d0 con 0x7f08ac140940 2022-06-29T15:50:23.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.199+0000 7f089effd700 1 --2- 172.21.15.47:0/1496251802 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0888053b00 0x7f0888055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:23.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.199+0000 7f089effd700 1 -- 172.21.15.47:0/1496251802 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f08a806b660 con 0x7f08ac140940 2022-06-29T15:50:23.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.199+0000 7f08b1328700 1 --2- 172.21.15.47:0/1496251802 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0888053b00 0x7f0888055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:23.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.203+0000 7f08b1328700 1 --2- 172.21.15.47:0/1496251802 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0888053b00 0x7f0888055fc0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7f08a000b000 tx=0x7f08a0000bc0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:23.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.207+0000 7f089effd700 1 -- 172.21.15.47:0/1496251802 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f08a8042310 con 0x7f08ac140940 2022-06-29T15:50:23.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.331+0000 7f08b358c700 1 -- 172.21.15.47:0/1496251802 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f08ac13f540 con 0x7f08ac140940 2022-06-29T15:50:23.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.335+0000 7f089effd700 1 -- 172.21.15.47:0/1496251802 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v111) v1 ==== 100+0+10 (secure 0 0 0) 0x7f08a8045930 con 0x7f08ac140940 2022-06-29T15:50:23.340 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:50:23.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.335+0000 7f08b358c700 1 -- 172.21.15.47:0/1496251802 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0888053b00 msgr2=0x7f0888055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:23.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.335+0000 7f08b358c700 1 --2- 172.21.15.47:0/1496251802 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0888053b00 0x7f0888055fc0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7f08a000b000 tx=0x7f08a0000bc0).stop 2022-06-29T15:50:23.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.335+0000 7f08b358c700 1 -- 172.21.15.47:0/1496251802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08ac140940 msgr2=0x7f08ac0ade80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:23.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.335+0000 7f08b358c700 1 --2- 172.21.15.47:0/1496251802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08ac140940 0x7f08ac0ade80 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 rx=0x7f08a8001110 tx=0x7f08a8007ae0).stop 2022-06-29T15:50:23.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.335+0000 7f08b358c700 1 -- 172.21.15.47:0/1496251802 shutdown_connections 2022-06-29T15:50:23.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.335+0000 7f08b358c700 1 --2- 172.21.15.47:0/1496251802 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0888053b00 0x7f0888055fc0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:23.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.335+0000 7f08b358c700 1 --2- 172.21.15.47:0/1496251802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08ac14ca80 0x7f08ac13f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:23.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.335+0000 7f08b358c700 1 --2- 172.21.15.47:0/1496251802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08ac140940 0x7f08ac0ade80 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:23.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.335+0000 7f08b358c700 1 --2- 172.21.15.47:0/1496251802 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f08ac0b30e0 0x7f08ac0ad990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:23.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.335+0000 7f08b358c700 1 -- 172.21.15.47:0/1496251802 >> 172.21.15.47:0/1496251802 conn(0x7f08ac0627d0 msgr2=0x7f08ac13d620 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:23.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.335+0000 7f08b358c700 1 -- 172.21.15.47:0/1496251802 shutdown_connections 2022-06-29T15:50:23.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.335+0000 7f08b358c700 1 -- 172.21.15.47:0/1496251802 wait complete. 2022-06-29T15:50:23.353 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-29T15:50:23.353 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:50:23.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.451+0000 7f642f4fe700 1 Processor -- start 2022-06-29T15:50:23.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.451+0000 7f642f4fe700 1 -- start start 2022-06-29T15:50:23.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.451+0000 7f642f4fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f642805c510 0x7f6428143bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:23.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.451+0000 7f642f4fe700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f64280b2c40 0x7f64280b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:23.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.451+0000 7f642f4fe700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6428142f90 con 0x7f642805c8f0 2022-06-29T15:50:23.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.451+0000 7f642f4fe700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64281431f0 con 0x7f642805c510 2022-06-29T15:50:23.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.451+0000 7f642f4fe700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6428143490 con 0x7f64280b2c40 2022-06-29T15:50:23.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.451+0000 7f642e4fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f642805c510 0x7f6428143bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:23.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.451+0000 7f642dcfb700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f642805c8f0 0x7f64280b4100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37900/0 (socket says 172.21.15.47:37900) 2022-06-29T15:50:23.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.451+0000 7f642dcfb700 1 -- 172.21.15.47:0/123642984 learned_addr learned my addr 172.21.15.47:0/123642984 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:23.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.451+0000 7f642ecfd700 1 --2- 172.21.15.47:0/123642984 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f64280b2c40 0x7f64280b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:23.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.455+0000 7f642d4fa700 1 -- 172.21.15.47:0/123642984 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1636320633 0 0) 0x7f6428142f90 con 0x7f642805c8f0 2022-06-29T15:50:23.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.455+0000 7f642d4fa700 1 -- 172.21.15.47:0/123642984 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6410003260 con 0x7f642805c8f0 2022-06-29T15:50:23.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.455+0000 7f642e4fc700 1 -- 172.21.15.47:0/123642984 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f64280b2c40 msgr2=0x7f64280b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:23.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.455+0000 7f642d4fa700 1 -- 172.21.15.47:0/123642984 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4044495721 0 0) 0x7f6410003260 con 0x7f642805c8f0 2022-06-29T15:50:23.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.455+0000 7f642e4fc700 1 --2- 172.21.15.47:0/123642984 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f64280b2c40 0x7f64280b30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:23.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.455+0000 7f642e4fc700 1 -- 172.21.15.47:0/123642984 >> v1:172.21.15.47:6789/0 conn(0x7f642805c8f0 legacy=0x7f64280b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:23.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.455+0000 7f642e4fc700 1 -- 172.21.15.47:0/123642984 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6428143730 con 0x7f642805c510 2022-06-29T15:50:23.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.455+0000 7f642e4fc700 1 --2- 172.21.15.47:0/123642984 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f642805c510 0x7f6428143bd0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 rx=0x7f6418001820 tx=0x7f6424006df0).ready entity=mon.0 client_cookie=3fbd5da853ae666 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:23.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.455+0000 7f642d4fa700 1 -- 172.21.15.47:0/123642984 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6418001680 con 0x7f642805c510 2022-06-29T15:50:23.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.455+0000 7f642d4fa700 1 -- 172.21.15.47:0/123642984 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6424000e90 con 0x7f642805c510 2022-06-29T15:50:23.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.455+0000 7f642d4fa700 1 -- 172.21.15.47:0/123642984 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6424012460 con 0x7f642805c510 2022-06-29T15:50:23.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.455+0000 7f642f4fe700 1 -- 172.21.15.47:0/123642984 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f642805c510 msgr2=0x7f6428143bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:23.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.455+0000 7f642f4fe700 1 --2- 172.21.15.47:0/123642984 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f642805c510 0x7f6428143bd0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 rx=0x7f6418001820 tx=0x7f6424006df0).stop 2022-06-29T15:50:23.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.455+0000 7f642f4fe700 1 -- 172.21.15.47:0/123642984 shutdown_connections 2022-06-29T15:50:23.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.455+0000 7f642f4fe700 1 --2- 172.21.15.47:0/123642984 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f64280b2c40 0x7f64280b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:23.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.455+0000 7f642f4fe700 1 --2- 172.21.15.47:0/123642984 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f642805c510 0x7f6428143bd0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:23.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.455+0000 7f642f4fe700 1 -- 172.21.15.47:0/123642984 >> 172.21.15.47:0/123642984 conn(0x7f6428060e00 msgr2=0x7f64280611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:23.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.455+0000 7f642f4fe700 1 -- 172.21.15.47:0/123642984 shutdown_connections 2022-06-29T15:50:23.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.455+0000 7f642f4fe700 1 -- 172.21.15.47:0/123642984 wait complete. 2022-06-29T15:50:23.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.459+0000 7f642f4fe700 1 Processor -- start 2022-06-29T15:50:23.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.459+0000 7f642f4fe700 1 -- start start 2022-06-29T15:50:23.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.459+0000 7f642f4fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f642805c510 0x7f64280b6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:23.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.459+0000 7f642f4fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f642805c8f0 0x7f6428155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:23.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.459+0000 7f642f4fe700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f64280b2c40 0x7f6428156280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:23.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.459+0000 7f642f4fe700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6428156770 con 0x7f642805c8f0 2022-06-29T15:50:23.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.459+0000 7f642f4fe700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6428140020 con 0x7f642805c510 2022-06-29T15:50:23.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.459+0000 7f642f4fe700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6428140280 con 0x7f64280b2c40 2022-06-29T15:50:23.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.459+0000 7f642dcfb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f642805c8f0 0x7f6428155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:23.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.459+0000 7f642dcfb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f642805c8f0 0x7f6428155d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52354/0 (socket says 172.21.15.47:52354) 2022-06-29T15:50:23.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.459+0000 7f642dcfb700 1 -- 172.21.15.47:0/2017788850 learned_addr learned my addr 172.21.15.47:0/2017788850 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:23.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.459+0000 7f642ecfd700 1 --2- 172.21.15.47:0/2017788850 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f64280b2c40 0x7f6428156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:23.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.459+0000 7f642e4fc700 1 --2- 172.21.15.47:0/2017788850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f642805c510 0x7f64280b6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:23.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.459+0000 7f642dcfb700 1 -- 172.21.15.47:0/2017788850 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f64280b2c40 msgr2=0x7f6428156280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:23.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.459+0000 7f642dcfb700 1 --2- 172.21.15.47:0/2017788850 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f64280b2c40 0x7f6428156280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:23.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.459+0000 7f642dcfb700 1 -- 172.21.15.47:0/2017788850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f642805c510 msgr2=0x7f64280b6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:23.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.459+0000 7f642dcfb700 1 --2- 172.21.15.47:0/2017788850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f642805c510 0x7f64280b6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:23.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.459+0000 7f642dcfb700 1 -- 172.21.15.47:0/2017788850 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f642400b180 con 0x7f642805c8f0 2022-06-29T15:50:23.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.459+0000 7f642dcfb700 1 --2- 172.21.15.47:0/2017788850 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f642805c8f0 0x7f6428155d90 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 rx=0x7f64180018e0 tx=0x7f6418001250).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:23.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.459+0000 7f64177fe700 1 -- 172.21.15.47:0/2017788850 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6418007170 con 0x7f642805c8f0 2022-06-29T15:50:23.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.459+0000 7f64177fe700 1 -- 172.21.15.47:0/2017788850 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64180072d0 con 0x7f642805c8f0 2022-06-29T15:50:23.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.459+0000 7f642f4fe700 1 -- 172.21.15.47:0/2017788850 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6428140520 con 0x7f642805c8f0 2022-06-29T15:50:23.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.459+0000 7f64177fe700 1 -- 172.21.15.47:0/2017788850 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6418004920 con 0x7f642805c8f0 2022-06-29T15:50:23.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.459+0000 7f642f4fe700 1 -- 172.21.15.47:0/2017788850 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64280af360 con 0x7f642805c8f0 2022-06-29T15:50:23.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.463+0000 7f642f4fe700 1 -- 172.21.15.47:0/2017788850 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64280573d0 con 0x7f642805c8f0 2022-06-29T15:50:23.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.463+0000 7f64177fe700 1 -- 172.21.15.47:0/2017788850 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6418004a80 con 0x7f642805c8f0 2022-06-29T15:50:23.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.463+0000 7f64177fe700 1 --2- 172.21.15.47:0/2017788850 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6408053a10 0x7f6408055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:23.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.463+0000 7f64177fe700 1 -- 172.21.15.47:0/2017788850 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f641806bab0 con 0x7f642805c8f0 2022-06-29T15:50:23.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.463+0000 7f642e4fc700 1 --2- 172.21.15.47:0/2017788850 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6408053a10 0x7f6408055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:23.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.463+0000 7f642e4fc700 1 --2- 172.21.15.47:0/2017788850 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6408053a10 0x7f6408055ed0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7f6424018000 tx=0x7f6424014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:23.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.467+0000 7f64177fe700 1 -- 172.21.15.47:0/2017788850 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6418042870 con 0x7f642805c8f0 2022-06-29T15:50:23.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.631+0000 7f642f4fe700 1 -- 172.21.15.47:0/2017788850 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6428061b00 con 0x7f642805c8f0 2022-06-29T15:50:23.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.631+0000 7f64177fe700 1 -- 172.21.15.47:0/2017788850 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+2496 (secure 0 0 0) 0x7f6418045e90 con 0x7f642805c8f0 2022-06-29T15:50:23.639 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:50:23.640 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":51,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3752554010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3752554010},{"type":"v1","addr":"172.21.15.47:6835","nonce":3752554010}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":48},{"gid":7293,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.47:6837/3263659716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":3263659716},{"type":"v1","addr":"172.21.15.47:6837","nonce":3263659716}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":49}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:50:15.338114+0000","modified":"2022-06-29T15:50:16.342008+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":56354},"failed":[],"damaged":[],"stopped":[],"info":{"gid_56354":{"gid":56354,"name":"b","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6833/2706285595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2706285595},{"type":"v1","addr":"172.21.15.64:6833","nonce":2706285595}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2022-06-29T15:50:23.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.635+0000 7f642f4fe700 1 -- 172.21.15.47:0/2017788850 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6408053a10 msgr2=0x7f6408055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:23.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.635+0000 7f642f4fe700 1 --2- 172.21.15.47:0/2017788850 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6408053a10 0x7f6408055ed0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7f6424018000 tx=0x7f6424014000).stop 2022-06-29T15:50:23.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.635+0000 7f642f4fe700 1 -- 172.21.15.47:0/2017788850 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f642805c8f0 msgr2=0x7f6428155d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:23.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.635+0000 7f642f4fe700 1 --2- 172.21.15.47:0/2017788850 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f642805c8f0 0x7f6428155d90 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 rx=0x7f64180018e0 tx=0x7f6418001250).stop 2022-06-29T15:50:23.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.635+0000 7f642f4fe700 1 -- 172.21.15.47:0/2017788850 shutdown_connections 2022-06-29T15:50:23.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.635+0000 7f642f4fe700 1 --2- 172.21.15.47:0/2017788850 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f64280b2c40 0x7f6428156280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:23.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.635+0000 7f642f4fe700 1 --2- 172.21.15.47:0/2017788850 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f642805c8f0 0x7f6428155d90 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:23.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.635+0000 7f642f4fe700 1 --2- 172.21.15.47:0/2017788850 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6408053a10 0x7f6408055ed0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:23.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.635+0000 7f642f4fe700 1 --2- 172.21.15.47:0/2017788850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f642805c510 0x7f64280b6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:23.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.635+0000 7f642f4fe700 1 -- 172.21.15.47:0/2017788850 >> 172.21.15.47:0/2017788850 conn(0x7f6428060e00 msgr2=0x7f64281419c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:23.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.635+0000 7f642f4fe700 1 -- 172.21.15.47:0/2017788850 shutdown_connections 2022-06-29T15:50:23.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:23.635+0000 7f642f4fe700 1 -- 172.21.15.47:0/2017788850 wait complete. 2022-06-29T15:50:23.645 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 51 2022-06-29T15:50:23.955 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 51, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-29T15:50:15.338114+0000', 'modified': '2022-06-29T15:50:16.342008+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 56354}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_56354': {'gid': 56354, 'name': 'b', 'rank': 0, 'incarnation': 50, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.64:6833/2706285595', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6832', 'nonce': 2706285595}, {'type': 'v1', 'addr': '172.21.15.64:6833', 'nonce': 2706285595}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-29T15:50:23.956 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-29T15:50:23.956 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-06-29T15:50:24.076 INFO:teuthology.orchestra.run.smithi064.stdout:{"cluster_fsid":"9a68894a-53ce-400e-838b-14eab91b834c","whoami":0,"id":56354,"want_state":"up:active","state":"up:active","rank_uptime":8.7204965879999996,"mdsmap_epoch":51,"osdmap_epoch":111,"osdmap_epoch_barrier":110,"uptime":13.083764321} 2022-06-29T15:50:24.077 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9a68894a-53ce-400e-838b-14eab91b834c", "id": 56354, "mdsmap_epoch": 51, "osdmap_epoch": 111, "osdmap_epoch_barrier": 110, "rank_uptime": 8.720496588, "state": "up:active", "uptime": 13.083764321, "want_state": "up:active", "whoami": 0 } 2022-06-29T15:50:24.078 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-29T15:50:24.078 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2022-06-29T15:50:24.079 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi164.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2022-06-29T15:50:24.079 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:24.080 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:50:24.088 DEBUG:teuthology.orchestra.run.smithi164:> sudo modprobe fuse 2022-06-29T15:50:24.148 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:50:24.148 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:50:24.163 INFO:teuthology.orchestra.run.smithi164.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:50:24.164 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:50:24.165 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:24.166 DEBUG:teuthology.orchestra.run.smithi164:> ls /sys/fs/fuse/connections 2022-06-29T15:50:24.171 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-29T15:50:24.172 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:50:24.174 DEBUG:teuthology.orchestra.run.smithi164:> sudo modprobe fuse 2022-06-29T15:50:24.193 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:50:24.193 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:50:24.246 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi164.stderr:2022-06-29T15:50:24.237+0000 7f106b03a080 -1 init, newargv = 0x55e9ffc1c580 newargc=9ceph-fuse[12728 2022-06-29T15:50:24.246 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi164.stderr:]: starting ceph client 2022-06-29T15:50:24.250 INFO:teuthology.orchestra.run.smithi164.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:50:24.251 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:50:24.252 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:24.252 DEBUG:teuthology.orchestra.run.smithi164:> ls /sys/fs/fuse/connections 2022-06-29T15:50:24.257 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi164.stderr:ceph-fuse[12728]: starting fuse 2022-06-29T15:50:24.260 INFO:teuthology.orchestra.run.smithi164.stdout:52 2022-06-29T15:50:24.261 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-29T15:50:24.261 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:24.262 DEBUG:teuthology.orchestra.run.smithi164:> sudo python3 -c ' 2022-06-29T15:50:24.262 DEBUG:teuthology.orchestra.run.smithi164:> import glob 2022-06-29T15:50:24.263 DEBUG:teuthology.orchestra.run.smithi164:> import re 2022-06-29T15:50:24.263 DEBUG:teuthology.orchestra.run.smithi164:> import os 2022-06-29T15:50:24.264 DEBUG:teuthology.orchestra.run.smithi164:> import subprocess 2022-06-29T15:50:24.264 DEBUG:teuthology.orchestra.run.smithi164:> 2022-06-29T15:50:24.265 DEBUG:teuthology.orchestra.run.smithi164:> def find_socket(client_name): 2022-06-29T15:50:24.265 DEBUG:teuthology.orchestra.run.smithi164:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2022-06-29T15:50:24.266 DEBUG:teuthology.orchestra.run.smithi164:> files = glob.glob(asok_path) 2022-06-29T15:50:24.267 DEBUG:teuthology.orchestra.run.smithi164:> 2022-06-29T15:50:24.268 DEBUG:teuthology.orchestra.run.smithi164:> # Given a non-glob path, it better be there 2022-06-29T15:50:24.268 DEBUG:teuthology.orchestra.run.smithi164:> if "*" not in asok_path: 2022-06-29T15:50:24.269 DEBUG:teuthology.orchestra.run.smithi164:> assert(len(files) == 1) 2022-06-29T15:50:24.270 DEBUG:teuthology.orchestra.run.smithi164:> return files[0] 2022-06-29T15:50:24.271 DEBUG:teuthology.orchestra.run.smithi164:> 2022-06-29T15:50:24.271 DEBUG:teuthology.orchestra.run.smithi164:> for f in files: 2022-06-29T15:50:24.272 DEBUG:teuthology.orchestra.run.smithi164:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-29T15:50:24.273 DEBUG:teuthology.orchestra.run.smithi164:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-29T15:50:24.273 DEBUG:teuthology.orchestra.run.smithi164:> return f 2022-06-29T15:50:24.273 DEBUG:teuthology.orchestra.run.smithi164:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-29T15:50:24.274 DEBUG:teuthology.orchestra.run.smithi164:> 2022-06-29T15:50:24.274 DEBUG:teuthology.orchestra.run.smithi164:> print(find_socket("client.1")) 2022-06-29T15:50:24.275 DEBUG:teuthology.orchestra.run.smithi164:> ' 2022-06-29T15:50:24.317 INFO:teuthology.orchestra.run.smithi164.stdout:/var/run/ceph/ceph-client.1.12728.asok 2022-06-29T15:50:24.322 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.12728.asok 2022-06-29T15:50:24.323 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:24.323 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.12728.asok status 2022-06-29T15:50:24.460 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2022-06-29T15:50:24.460 INFO:teuthology.orchestra.run.smithi164.stdout: "metadata": { 2022-06-29T15:50:24.460 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_sha1": "f25ffac72e5a9e63860ff7deb6767a11f0559a16", 2022-06-29T15:50:24.460 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_version": "ceph version 15.2.16-304-gf25ffac7 (f25ffac72e5a9e63860ff7deb6767a11f0559a16) octopus (stable)", 2022-06-29T15:50:24.461 INFO:teuthology.orchestra.run.smithi164.stdout: "entity_id": "1", 2022-06-29T15:50:24.461 INFO:teuthology.orchestra.run.smithi164.stdout: "hostname": "smithi164", 2022-06-29T15:50:24.461 INFO:teuthology.orchestra.run.smithi164.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2022-06-29T15:50:24.461 INFO:teuthology.orchestra.run.smithi164.stdout: "pid": "12728", 2022-06-29T15:50:24.462 INFO:teuthology.orchestra.run.smithi164.stdout: "root": "/" 2022-06-29T15:50:24.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2022-06-29T15:50:24.462 INFO:teuthology.orchestra.run.smithi164.stdout: "dentry_count": 0, 2022-06-29T15:50:24.462 INFO:teuthology.orchestra.run.smithi164.stdout: "dentry_pinned_count": 0, 2022-06-29T15:50:24.463 INFO:teuthology.orchestra.run.smithi164.stdout: "id": 7414, 2022-06-29T15:50:24.463 INFO:teuthology.orchestra.run.smithi164.stdout: "inst": { 2022-06-29T15:50:24.463 INFO:teuthology.orchestra.run.smithi164.stdout: "name": { 2022-06-29T15:50:24.463 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "client", 2022-06-29T15:50:24.463 INFO:teuthology.orchestra.run.smithi164.stdout: "num": 7414 2022-06-29T15:50:24.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2022-06-29T15:50:24.464 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": { 2022-06-29T15:50:24.464 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "v1", 2022-06-29T15:50:24.464 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": "172.21.15.164:0", 2022-06-29T15:50:24.465 INFO:teuthology.orchestra.run.smithi164.stdout: "nonce": 3716341954 2022-06-29T15:50:24.465 INFO:teuthology.orchestra.run.smithi164.stdout: } 2022-06-29T15:50:24.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2022-06-29T15:50:24.465 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": { 2022-06-29T15:50:24.465 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "v1", 2022-06-29T15:50:24.466 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": "172.21.15.164:0", 2022-06-29T15:50:24.466 INFO:teuthology.orchestra.run.smithi164.stdout: "nonce": 3716341954 2022-06-29T15:50:24.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2022-06-29T15:50:24.467 INFO:teuthology.orchestra.run.smithi164.stdout: "inst_str": "client.7414 172.21.15.164:0/3716341954", 2022-06-29T15:50:24.468 INFO:teuthology.orchestra.run.smithi164.stdout: "addr_str": "172.21.15.164:0/3716341954", 2022-06-29T15:50:24.468 INFO:teuthology.orchestra.run.smithi164.stdout: "inode_count": 1, 2022-06-29T15:50:24.468 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_epoch": 51, 2022-06-29T15:50:24.468 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_epoch": 111, 2022-06-29T15:50:24.469 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_epoch_barrier": 0, 2022-06-29T15:50:24.469 INFO:teuthology.orchestra.run.smithi164.stdout: "blacklisted": false, 2022-06-29T15:50:24.469 INFO:teuthology.orchestra.run.smithi164.stdout: "fs_name": "cephfs" 2022-06-29T15:50:24.469 INFO:teuthology.orchestra.run.smithi164.stdout:} 2022-06-29T15:50:24.471 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:24.472 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:50:24.472 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:50:24.480 INFO:teuthology.orchestra.run.smithi164.stdout:fuseblk 2022-06-29T15:50:24.481 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-06-29T15:50:24.481 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:24.482 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:50:24.498 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph df --format=json-pretty 2022-06-29T15:50:24.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.599+0000 7f92e0990700 1 Processor -- start 2022-06-29T15:50:24.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.599+0000 7f92e0990700 1 -- start start 2022-06-29T15:50:24.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.599+0000 7f92e0990700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f92dc060ee0 0x7f92dc0612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:24.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.599+0000 7f92e0990700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92dc0b4300 0x7f92dc1471a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:24.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.599+0000 7f92e0990700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92dc067610 con 0x7f92dc066f70 2022-06-29T15:50:24.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.599+0000 7f92e0990700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92dc0673a0 con 0x7f92dc060ee0 2022-06-29T15:50:24.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.599+0000 7f92e0990700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92dc1467c0 con 0x7f92dc0b4300 2022-06-29T15:50:24.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.599+0000 7f92da59c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f92dc060ee0 0x7f92dc0612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:24.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.599+0000 7f92d9d9b700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92dc0b4300 0x7f92dc1471a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:24.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.599+0000 7f92da59c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f92dc060ee0 0x7f92dc0612c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52362/0 (socket says 172.21.15.47:52362) 2022-06-29T15:50:24.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.599+0000 7f92da59c700 1 -- 172.21.15.47:0/2066606136 learned_addr learned my addr 172.21.15.47:0/2066606136 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:24.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.599+0000 7f92da59c700 1 -- 172.21.15.47:0/2066606136 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92dc0b4300 msgr2=0x7f92dc1471a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:24.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.603+0000 7f92da59c700 1 --2- 172.21.15.47:0/2066606136 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92dc0b4300 0x7f92dc1471a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:24.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.603+0000 7f92da59c700 1 -- 172.21.15.47:0/2066606136 >> v1:172.21.15.64:6789/0 conn(0x7f92dc066f70 legacy=0x7f92dc1430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:24.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.603+0000 7f92d959a700 1 -- 172.21.15.47:0/2066606136 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3283074499 0 0) 0x7f92dc067610 con 0x7f92dc066f70 2022-06-29T15:50:24.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.603+0000 7f92da59c700 1 -- 172.21.15.47:0/2066606136 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92dc146a60 con 0x7f92dc060ee0 2022-06-29T15:50:24.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.603+0000 7f92d9d9b700 1 --2- 172.21.15.47:0/2066606136 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92dc0b4300 0x7f92dc1471a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:50:24.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.603+0000 7f92da59c700 1 --2- 172.21.15.47:0/2066606136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92dc060ee0 0x7f92dc0612c0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 rx=0x7f92c4002f70 tx=0x7f92c4006eb0).ready entity=mon.0 client_cookie=5429a17a21ab7831 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:24.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.603+0000 7f92d959a700 1 -- 172.21.15.47:0/2066606136 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92c400d570 con 0x7f92dc060ee0 2022-06-29T15:50:24.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.603+0000 7f92d959a700 1 -- 172.21.15.47:0/2066606136 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92c4000e90 con 0x7f92dc060ee0 2022-06-29T15:50:24.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.603+0000 7f92d959a700 1 -- 172.21.15.47:0/2066606136 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92c400d8c0 con 0x7f92dc060ee0 2022-06-29T15:50:24.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.603+0000 7f92e0990700 1 -- 172.21.15.47:0/2066606136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92dc060ee0 msgr2=0x7f92dc0612c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:24.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.603+0000 7f92e0990700 1 --2- 172.21.15.47:0/2066606136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92dc060ee0 0x7f92dc0612c0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 rx=0x7f92c4002f70 tx=0x7f92c4006eb0).stop 2022-06-29T15:50:24.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.603+0000 7f92e0990700 1 -- 172.21.15.47:0/2066606136 shutdown_connections 2022-06-29T15:50:24.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.603+0000 7f92e0990700 1 --2- 172.21.15.47:0/2066606136 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92dc0b4300 0x7f92dc1471a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:24.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.603+0000 7f92e0990700 1 --2- 172.21.15.47:0/2066606136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92dc060ee0 0x7f92dc0612c0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:24.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.603+0000 7f92e0990700 1 -- 172.21.15.47:0/2066606136 >> 172.21.15.47:0/2066606136 conn(0x7f92dc05c510 msgr2=0x7f92dc05f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:24.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.603+0000 7f92e0990700 1 -- 172.21.15.47:0/2066606136 shutdown_connections 2022-06-29T15:50:24.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.603+0000 7f92e0990700 1 -- 172.21.15.47:0/2066606136 wait complete. 2022-06-29T15:50:24.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.603+0000 7f92e0990700 1 Processor -- start 2022-06-29T15:50:24.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.603+0000 7f92e0990700 1 -- start start 2022-06-29T15:50:24.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.607+0000 7f92e0990700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92dc060ee0 0x7f92dc13b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:24.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.607+0000 7f92e0990700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92dc066f70 0x7f92dc136150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:24.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.607+0000 7f92e0990700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92dc0b4300 0x7f92dc136640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:24.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.607+0000 7f92e0990700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92dc1496f0 con 0x7f92dc0b4300 2022-06-29T15:50:24.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.607+0000 7f92e0990700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92dc0b1b10 con 0x7f92dc066f70 2022-06-29T15:50:24.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.607+0000 7f92e0990700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92dc0b1db0 con 0x7f92dc060ee0 2022-06-29T15:50:24.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.607+0000 7f92dad9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92dc0b4300 0x7f92dc136640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:24.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.607+0000 7f92dad9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92dc0b4300 0x7f92dc136640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52370/0 (socket says 172.21.15.47:52370) 2022-06-29T15:50:24.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.607+0000 7f92dad9d700 1 -- 172.21.15.47:0/1998993989 learned_addr learned my addr 172.21.15.47:0/1998993989 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:24.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.607+0000 7f92d9d9b700 1 --2- 172.21.15.47:0/1998993989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92dc066f70 0x7f92dc136150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:24.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.607+0000 7f92da59c700 1 --2- 172.21.15.47:0/1998993989 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92dc060ee0 0x7f92dc13b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:24.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.607+0000 7f92dad9d700 1 -- 172.21.15.47:0/1998993989 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92dc060ee0 msgr2=0x7f92dc13b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:24.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.607+0000 7f92dad9d700 1 --2- 172.21.15.47:0/1998993989 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92dc060ee0 0x7f92dc13b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:24.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.607+0000 7f92dad9d700 1 -- 172.21.15.47:0/1998993989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92dc066f70 msgr2=0x7f92dc136150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:24.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.607+0000 7f92dad9d700 1 --2- 172.21.15.47:0/1998993989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92dc066f70 0x7f92dc136150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:24.622 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.607+0000 7f92dad9d700 1 -- 172.21.15.47:0/1998993989 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92cc00b180 con 0x7f92dc0b4300 2022-06-29T15:50:24.622 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.607+0000 7f92dad9d700 1 --2- 172.21.15.47:0/1998993989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92dc0b4300 0x7f92dc136640 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 rx=0x7f92d4001250 tx=0x7f92d4007dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:24.622 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.607+0000 7f92d37fe700 1 -- 172.21.15.47:0/1998993989 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92d400e4a0 con 0x7f92dc0b4300 2022-06-29T15:50:24.622 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.607+0000 7f92d37fe700 1 -- 172.21.15.47:0/1998993989 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92d400e600 con 0x7f92dc0b4300 2022-06-29T15:50:24.622 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.607+0000 7f92e0990700 1 -- 172.21.15.47:0/1998993989 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92c400b1e0 con 0x7f92dc0b4300 2022-06-29T15:50:24.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.607+0000 7f92d37fe700 1 -- 172.21.15.47:0/1998993989 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92d400e960 con 0x7f92dc0b4300 2022-06-29T15:50:24.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.607+0000 7f92e0990700 1 -- 172.21.15.47:0/1998993989 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92dc0b22f0 con 0x7f92dc0b4300 2022-06-29T15:50:24.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.607+0000 7f92e0990700 1 -- 172.21.15.47:0/1998993989 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92dc152320 con 0x7f92dc0b4300 2022-06-29T15:50:24.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.611+0000 7f92d37fe700 1 -- 172.21.15.47:0/1998993989 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f92d4010070 con 0x7f92dc0b4300 2022-06-29T15:50:24.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.611+0000 7f92d37fe700 1 --2- 172.21.15.47:0/1998993989 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f92c0053b00 0x7f92c0055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:24.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.611+0000 7f92d37fe700 1 -- 172.21.15.47:0/1998993989 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f92d406b440 con 0x7f92dc0b4300 2022-06-29T15:50:24.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.611+0000 7f92da59c700 1 --2- 172.21.15.47:0/1998993989 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f92c0053b00 0x7f92c0055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:24.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.611+0000 7f92da59c700 1 --2- 172.21.15.47:0/1998993989 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f92c0053b00 0x7f92c0055fc0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f92c401a000 tx=0x7f92c4015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:24.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.615+0000 7f92d37fe700 1 -- 172.21.15.47:0/1998993989 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f92d4042200 con 0x7f92dc0b4300 2022-06-29T15:50:24.801 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.795+0000 7f92e0990700 1 -- 172.21.15.47:0/1998993989 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "df", "format": "json-pretty"} v 0) v1 -- 0x7f92dc0573d0 con 0x7f92dc0b4300 2022-06-29T15:50:24.801 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.795+0000 7f92d37fe700 1 -- 172.21.15.47:0/1998993989 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "df", "format": "json-pretty"}]=0 v0) v1 ==== 75+0+1936 (secure 0 0 0) 0x7f92d4045820 con 0x7f92dc0b4300 2022-06-29T15:50:24.802 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:50:24.802 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2022-06-29T15:50:24.802 INFO:teuthology.orchestra.run.smithi047.stdout: "stats": { 2022-06-29T15:50:24.803 INFO:teuthology.orchestra.run.smithi047.stdout: "total_bytes": 773094113280, 2022-06-29T15:50:24.803 INFO:teuthology.orchestra.run.smithi047.stdout: "total_avail_bytes": 764492972032, 2022-06-29T15:50:24.803 INFO:teuthology.orchestra.run.smithi047.stdout: "total_used_bytes": 11206656, 2022-06-29T15:50:24.803 INFO:teuthology.orchestra.run.smithi047.stdout: "total_used_raw_bytes": 8601141248, 2022-06-29T15:50:24.804 INFO:teuthology.orchestra.run.smithi047.stdout: "total_used_raw_ratio": 0.011125607416033745, 2022-06-29T15:50:24.804 INFO:teuthology.orchestra.run.smithi047.stdout: "num_osds": 8, 2022-06-29T15:50:24.804 INFO:teuthology.orchestra.run.smithi047.stdout: "num_per_pool_osds": 8, 2022-06-29T15:50:24.804 INFO:teuthology.orchestra.run.smithi047.stdout: "num_per_pool_omap_osds": 8 2022-06-29T15:50:24.805 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:24.805 INFO:teuthology.orchestra.run.smithi047.stdout: "stats_by_class": { 2022-06-29T15:50:24.805 INFO:teuthology.orchestra.run.smithi047.stdout: "ssd": { 2022-06-29T15:50:24.805 INFO:teuthology.orchestra.run.smithi047.stdout: "total_bytes": 773094113280, 2022-06-29T15:50:24.806 INFO:teuthology.orchestra.run.smithi047.stdout: "total_avail_bytes": 764492972032, 2022-06-29T15:50:24.806 INFO:teuthology.orchestra.run.smithi047.stdout: "total_used_bytes": 11206656, 2022-06-29T15:50:24.806 INFO:teuthology.orchestra.run.smithi047.stdout: "total_used_raw_bytes": 8601141248, 2022-06-29T15:50:24.806 INFO:teuthology.orchestra.run.smithi047.stdout: "total_used_raw_ratio": 0.011125607416033745 2022-06-29T15:50:24.807 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:24.807 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:24.807 INFO:teuthology.orchestra.run.smithi047.stdout: "pools": [ 2022-06-29T15:50:24.807 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:24.808 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "device_health_metrics", 2022-06-29T15:50:24.808 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 1, 2022-06-29T15:50:24.808 INFO:teuthology.orchestra.run.smithi047.stdout: "stats": { 2022-06-29T15:50:24.808 INFO:teuthology.orchestra.run.smithi047.stdout: "stored": 0, 2022-06-29T15:50:24.808 INFO:teuthology.orchestra.run.smithi047.stdout: "objects": 2, 2022-06-29T15:50:24.809 INFO:teuthology.orchestra.run.smithi047.stdout: "kb_used": 0, 2022-06-29T15:50:24.809 INFO:teuthology.orchestra.run.smithi047.stdout: "bytes_used": 0, 2022-06-29T15:50:24.810 INFO:teuthology.orchestra.run.smithi047.stdout: "percent_used": 0, 2022-06-29T15:50:24.810 INFO:teuthology.orchestra.run.smithi047.stdout: "max_avail": 362919067648 2022-06-29T15:50:24.810 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:24.810 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:24.811 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:24.811 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd", 2022-06-29T15:50:24.811 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 2, 2022-06-29T15:50:24.811 INFO:teuthology.orchestra.run.smithi047.stdout: "stats": { 2022-06-29T15:50:24.811 INFO:teuthology.orchestra.run.smithi047.stdout: "stored": 0, 2022-06-29T15:50:24.812 INFO:teuthology.orchestra.run.smithi047.stdout: "objects": 0, 2022-06-29T15:50:24.812 INFO:teuthology.orchestra.run.smithi047.stdout: "kb_used": 0, 2022-06-29T15:50:24.812 INFO:teuthology.orchestra.run.smithi047.stdout: "bytes_used": 0, 2022-06-29T15:50:24.812 INFO:teuthology.orchestra.run.smithi047.stdout: "percent_used": 0, 2022-06-29T15:50:24.813 INFO:teuthology.orchestra.run.smithi047.stdout: "max_avail": 362919067648 2022-06-29T15:50:24.813 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:24.813 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:24.813 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:24.813 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cephfs_metadata", 2022-06-29T15:50:24.814 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 19, 2022-06-29T15:50:24.814 INFO:teuthology.orchestra.run.smithi047.stdout: "stats": { 2022-06-29T15:50:24.814 INFO:teuthology.orchestra.run.smithi047.stdout: "stored": 2742, 2022-06-29T15:50:24.814 INFO:teuthology.orchestra.run.smithi047.stdout: "objects": 22, 2022-06-29T15:50:24.815 INFO:teuthology.orchestra.run.smithi047.stdout: "kb_used": 64, 2022-06-29T15:50:24.815 INFO:teuthology.orchestra.run.smithi047.stdout: "bytes_used": 65536, 2022-06-29T15:50:24.815 INFO:teuthology.orchestra.run.smithi047.stdout: "percent_used": 9.0290093623934808e-08, 2022-06-29T15:50:24.815 INFO:teuthology.orchestra.run.smithi047.stdout: "max_avail": 362919067648 2022-06-29T15:50:24.815 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:24.816 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:24.816 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:24.816 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cephfs_data", 2022-06-29T15:50:24.816 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 20, 2022-06-29T15:50:24.817 INFO:teuthology.orchestra.run.smithi047.stdout: "stats": { 2022-06-29T15:50:24.817 INFO:teuthology.orchestra.run.smithi047.stdout: "stored": 0, 2022-06-29T15:50:24.817 INFO:teuthology.orchestra.run.smithi047.stdout: "objects": 0, 2022-06-29T15:50:24.817 INFO:teuthology.orchestra.run.smithi047.stdout: "kb_used": 0, 2022-06-29T15:50:24.817 INFO:teuthology.orchestra.run.smithi047.stdout: "bytes_used": 0, 2022-06-29T15:50:24.818 INFO:teuthology.orchestra.run.smithi047.stdout: "percent_used": 0, 2022-06-29T15:50:24.818 INFO:teuthology.orchestra.run.smithi047.stdout: "max_avail": 362919067648 2022-06-29T15:50:24.818 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:24.818 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:24.819 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:24.819 INFO:teuthology.orchestra.run.smithi047.stdout:} 2022-06-29T15:50:24.821 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.799+0000 7f92e0990700 1 -- 172.21.15.47:0/1998993989 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f92c0053b00 msgr2=0x7f92c0055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:24.822 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.799+0000 7f92e0990700 1 --2- 172.21.15.47:0/1998993989 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f92c0053b00 0x7f92c0055fc0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f92c401a000 tx=0x7f92c4015000).stop 2022-06-29T15:50:24.822 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.799+0000 7f92e0990700 1 -- 172.21.15.47:0/1998993989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92dc0b4300 msgr2=0x7f92dc136640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:24.822 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.799+0000 7f92e0990700 1 --2- 172.21.15.47:0/1998993989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92dc0b4300 0x7f92dc136640 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 rx=0x7f92d4001250 tx=0x7f92d4007dc0).stop 2022-06-29T15:50:24.822 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.799+0000 7f92e0990700 1 -- 172.21.15.47:0/1998993989 shutdown_connections 2022-06-29T15:50:24.823 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.799+0000 7f92e0990700 1 --2- 172.21.15.47:0/1998993989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92dc0b4300 0x7f92dc136640 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:24.823 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.799+0000 7f92e0990700 1 --2- 172.21.15.47:0/1998993989 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f92c0053b00 0x7f92c0055fc0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:24.823 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.799+0000 7f92e0990700 1 --2- 172.21.15.47:0/1998993989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92dc066f70 0x7f92dc136150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:24.823 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.799+0000 7f92e0990700 1 --2- 172.21.15.47:0/1998993989 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92dc060ee0 0x7f92dc13b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:24.823 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.799+0000 7f92e0990700 1 -- 172.21.15.47:0/1998993989 >> 172.21.15.47:0/1998993989 conn(0x7f92dc05c510 msgr2=0x7f92dc0b7560 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:24.824 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.799+0000 7f92e0990700 1 -- 172.21.15.47:0/1998993989 shutdown_connections 2022-06-29T15:50:24.824 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.799+0000 7f92e0990700 1 -- 172.21.15.47:0/1998993989 wait complete. 2022-06-29T15:50:24.824 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data size -f json-pretty 2022-06-29T15:50:24.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.931+0000 7f7e84c66700 1 Processor -- start 2022-06-29T15:50:24.938 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.931+0000 7f7e84c66700 1 -- start start 2022-06-29T15:50:24.940 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.931+0000 7f7e84c66700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7e800b5d90 0x7f7e80143a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:24.940 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.931+0000 7f7e84c66700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7e800676a0 0x7f7e80140130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:24.941 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.931+0000 7f7e84c66700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e8005cc30 con 0x7f7e8005c850 2022-06-29T15:50:24.941 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.931+0000 7f7e84c66700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e801430a0 con 0x7f7e800676a0 2022-06-29T15:50:24.941 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.931+0000 7f7e84c66700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e80143300 con 0x7f7e800b5d90 2022-06-29T15:50:24.941 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.931+0000 7f7e7ffff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7e800676a0 0x7f7e80140130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:24.942 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.931+0000 7f7e7f7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7e8005c850 0x7f7e800b4860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37928/0 (socket says 172.21.15.47:37928) 2022-06-29T15:50:24.942 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.931+0000 7f7e7ffff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7e800676a0 0x7f7e80140130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52378/0 (socket says 172.21.15.47:52378) 2022-06-29T15:50:24.942 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.931+0000 7f7e7f7fe700 1 -- 172.21.15.47:0/3569915367 learned_addr learned my addr 172.21.15.47:0/3569915367 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:24.942 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.931+0000 7f7e76ffd700 1 --2- 172.21.15.47:0/3569915367 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7e800b5d90 0x7f7e80143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:24.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.935+0000 7f7e7ffff700 1 -- 172.21.15.47:0/3569915367 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7e800b5d90 msgr2=0x7f7e80143a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:24.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.935+0000 7f7e7ffff700 1 --2- 172.21.15.47:0/3569915367 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7e800b5d90 0x7f7e80143a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:24.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.935+0000 7f7e7ffff700 1 -- 172.21.15.47:0/3569915367 >> v1:172.21.15.47:6789/0 conn(0x7f7e8005c850 legacy=0x7f7e800b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:24.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.935+0000 7f7e7effd700 1 -- 172.21.15.47:0/3569915367 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 386461795 0 0) 0x7f7e8005cc30 con 0x7f7e8005c850 2022-06-29T15:50:24.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.935+0000 7f7e7ffff700 1 -- 172.21.15.47:0/3569915367 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e801435a0 con 0x7f7e800676a0 2022-06-29T15:50:24.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.935+0000 7f7e7ffff700 1 --2- 172.21.15.47:0/3569915367 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e800676a0 0x7f7e80140130 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 rx=0x7f7e780032f0 tx=0x7f7e780091d0).ready entity=mon.0 client_cookie=747718101bbee455 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:24.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.935+0000 7f7e7effd700 1 -- 172.21.15.47:0/3569915367 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e7800b6c0 con 0x7f7e800676a0 2022-06-29T15:50:24.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.935+0000 7f7e7effd700 1 -- 172.21.15.47:0/3569915367 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e7800b820 con 0x7f7e800676a0 2022-06-29T15:50:24.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.935+0000 7f7e7effd700 1 -- 172.21.15.47:0/3569915367 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e7800b990 con 0x7f7e800676a0 2022-06-29T15:50:24.950 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.935+0000 7f7e84c66700 1 -- 172.21.15.47:0/3569915367 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e800676a0 msgr2=0x7f7e80140130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:24.951 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.935+0000 7f7e84c66700 1 --2- 172.21.15.47:0/3569915367 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e800676a0 0x7f7e80140130 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 rx=0x7f7e780032f0 tx=0x7f7e780091d0).stop 2022-06-29T15:50:24.951 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.935+0000 7f7e84c66700 1 -- 172.21.15.47:0/3569915367 shutdown_connections 2022-06-29T15:50:24.951 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.935+0000 7f7e84c66700 1 --2- 172.21.15.47:0/3569915367 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e800676a0 0x7f7e80140130 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:24.951 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.935+0000 7f7e84c66700 1 --2- 172.21.15.47:0/3569915367 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7e800b5d90 0x7f7e80143a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:24.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.935+0000 7f7e84c66700 1 -- 172.21.15.47:0/3569915367 >> 172.21.15.47:0/3569915367 conn(0x7f7e80060e00 msgr2=0x7f7e800611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:24.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.935+0000 7f7e84c66700 1 -- 172.21.15.47:0/3569915367 shutdown_connections 2022-06-29T15:50:24.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.935+0000 7f7e84c66700 1 -- 172.21.15.47:0/3569915367 wait complete. 2022-06-29T15:50:24.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.939+0000 7f7e84c66700 1 Processor -- start 2022-06-29T15:50:24.953 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.939+0000 7f7e84c66700 1 -- start start 2022-06-29T15:50:24.953 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.939+0000 7f7e84c66700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7e8005c850 0x7f7e800b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:24.953 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.939+0000 7f7e84c66700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e800676a0 0x7f7e800b1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:24.953 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.939+0000 7f7e84c66700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e800b5d90 0x7f7e800b9c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:24.953 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.939+0000 7f7e84c66700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e800b0ee0 con 0x7f7e800b5d90 2022-06-29T15:50:24.954 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.939+0000 7f7e84c66700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e80152fa0 con 0x7f7e800676a0 2022-06-29T15:50:24.954 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.939+0000 7f7e84c66700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e800b1110 con 0x7f7e8005c850 2022-06-29T15:50:24.954 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.939+0000 7f7e7ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e800b5d90 0x7f7e800b9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:24.954 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.939+0000 7f7e7ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e800b5d90 0x7f7e800b9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52384/0 (socket says 172.21.15.47:52384) 2022-06-29T15:50:24.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.939+0000 7f7e7ffff700 1 -- 172.21.15.47:0/2899403 learned_addr learned my addr 172.21.15.47:0/2899403 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:24.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.939+0000 7f7e76ffd700 1 --2- 172.21.15.47:0/2899403 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e800676a0 0x7f7e800b1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:24.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.939+0000 7f7e7f7fe700 1 --2- 172.21.15.47:0/2899403 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7e8005c850 0x7f7e800b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:24.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.939+0000 7f7e7ffff700 1 -- 172.21.15.47:0/2899403 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7e8005c850 msgr2=0x7f7e800b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:24.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.939+0000 7f7e7ffff700 1 --2- 172.21.15.47:0/2899403 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7e8005c850 0x7f7e800b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:24.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.939+0000 7f7e7ffff700 1 -- 172.21.15.47:0/2899403 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e800676a0 msgr2=0x7f7e800b1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:24.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.939+0000 7f7e7ffff700 1 --2- 172.21.15.47:0/2899403 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e800676a0 0x7f7e800b1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:24.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.939+0000 7f7e7ffff700 1 -- 172.21.15.47:0/2899403 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e78009470 con 0x7f7e800b5d90 2022-06-29T15:50:24.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.939+0000 7f7e7f7fe700 1 --2- 172.21.15.47:0/2899403 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7e8005c850 0x7f7e800b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:50:24.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.939+0000 7f7e7ffff700 1 --2- 172.21.15.47:0/2899403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e800b5d90 0x7f7e800b9c40 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 rx=0x7f7e780032f0 tx=0x7f7e78016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:24.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.939+0000 7f7e7d7fa700 1 -- 172.21.15.47:0/2899403 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e7800b6c0 con 0x7f7e800b5d90 2022-06-29T15:50:24.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.939+0000 7f7e7d7fa700 1 -- 172.21.15.47:0/2899403 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e780107a0 con 0x7f7e800b5d90 2022-06-29T15:50:24.958 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.939+0000 7f7e84c66700 1 -- 172.21.15.47:0/2899403 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e80153240 con 0x7f7e800b5d90 2022-06-29T15:50:24.958 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.939+0000 7f7e7d7fa700 1 -- 172.21.15.47:0/2899403 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e78010b00 con 0x7f7e800b5d90 2022-06-29T15:50:24.959 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.939+0000 7f7e84c66700 1 -- 172.21.15.47:0/2899403 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e800b7520 con 0x7f7e800b5d90 2022-06-29T15:50:24.959 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.943+0000 7f7e84c66700 1 -- 172.21.15.47:0/2899403 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e800b2640 con 0x7f7e800b5d90 2022-06-29T15:50:24.959 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.943+0000 7f7e7d7fa700 1 -- 172.21.15.47:0/2899403 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7e78027020 con 0x7f7e800b5d90 2022-06-29T15:50:24.959 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.943+0000 7f7e7d7fa700 1 --2- 172.21.15.47:0/2899403 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7e58053a70 0x7f7e58055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:24.960 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.943+0000 7f7e7d7fa700 1 -- 172.21.15.47:0/2899403 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f7e7806d5b0 con 0x7f7e800b5d90 2022-06-29T15:50:24.960 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.943+0000 7f7e7f7fe700 1 --2- 172.21.15.47:0/2899403 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7e58053a70 0x7f7e58055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:24.960 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.943+0000 7f7e7f7fe700 1 --2- 172.21.15.47:0/2899403 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7e58053a70 0x7f7e58055f30 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7f7e7000b000 tx=0x7f7e70009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:24.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:24.951+0000 7f7e7d7fa700 1 -- 172.21.15.47:0/2899403 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7e800b2640 con 0x7f7e800b5d90 2022-06-29T15:50:25.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.079+0000 7f7e84c66700 1 -- 172.21.15.47:0/2899403 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "size", "format": "json-pretty"} v 0) v1 -- 0x7f7e800b2640 con 0x7f7e800b5d90 2022-06-29T15:50:25.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.079+0000 7f7e7d7fa700 1 -- 172.21.15.47:0/2899403 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "size", "format": "json-pretty"}]=0 v111) v1 ==== 123+0+64 (secure 0 0 0) 0x7f7e800b2640 con 0x7f7e800b5d90 2022-06-29T15:50:25.087 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:50:25.087 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2022-06-29T15:50:25.087 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": "cephfs_data", 2022-06-29T15:50:25.088 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_id": 20, 2022-06-29T15:50:25.088 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2 2022-06-29T15:50:25.089 INFO:teuthology.orchestra.run.smithi047.stdout:} 2022-06-29T15:50:25.091 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.083+0000 7f7e84c66700 1 -- 172.21.15.47:0/2899403 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7e58053a70 msgr2=0x7f7e58055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:25.091 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.083+0000 7f7e84c66700 1 --2- 172.21.15.47:0/2899403 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7e58053a70 0x7f7e58055f30 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7f7e7000b000 tx=0x7f7e70009040).stop 2022-06-29T15:50:25.092 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.083+0000 7f7e84c66700 1 -- 172.21.15.47:0/2899403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e800b5d90 msgr2=0x7f7e800b9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:25.092 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.083+0000 7f7e84c66700 1 --2- 172.21.15.47:0/2899403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e800b5d90 0x7f7e800b9c40 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 rx=0x7f7e780032f0 tx=0x7f7e78016000).stop 2022-06-29T15:50:25.092 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.083+0000 7f7e84c66700 1 -- 172.21.15.47:0/2899403 shutdown_connections 2022-06-29T15:50:25.092 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.083+0000 7f7e84c66700 1 --2- 172.21.15.47:0/2899403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e800b5d90 0x7f7e800b9c40 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:25.093 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.083+0000 7f7e84c66700 1 --2- 172.21.15.47:0/2899403 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7e58053a70 0x7f7e58055f30 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:25.093 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.083+0000 7f7e84c66700 1 --2- 172.21.15.47:0/2899403 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e800676a0 0x7f7e800b1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:25.093 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.083+0000 7f7e84c66700 1 --2- 172.21.15.47:0/2899403 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7e8005c850 0x7f7e800b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:25.093 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.083+0000 7f7e84c66700 1 -- 172.21.15.47:0/2899403 >> 172.21.15.47:0/2899403 conn(0x7f7e80060e00 msgr2=0x7f7e801424b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:25.094 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.083+0000 7f7e84c66700 1 -- 172.21.15.47:0/2899403 shutdown_connections 2022-06-29T15:50:25.094 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.083+0000 7f7e84c66700 1 -- 172.21.15.47:0/2899403 wait complete. 2022-06-29T15:50:25.101 INFO:teuthology.orchestra.run:Running command with timeout 10800 2022-06-29T15:50:25.102 DEBUG:teuthology.orchestra.run.smithi099:> cd /home/ubuntu/cephtest/mnt.0 && sudo df . 2022-06-29T15:50:25.146 INFO:teuthology.orchestra.run.smithi099.stdout:Filesystem 1K-blocks Used Available Use% Mounted on 2022-06-29T15:50:25.146 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-fuse 354410496 0 354410496 0% /home/ubuntu/cephtest/mnt.0 2022-06-29T15:50:25.148 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-06-29T15:50:25.149 DEBUG:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:50:25.170 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:50:25.192 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:50:25.213 DEBUG:teuthology.orchestra.run.smithi164:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:50:25.234 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-06-29T15:50:25.234 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:25.235 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:50:25.235 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:50:25.266 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2022-06-29T15:50:25.267 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-29T15:50:25.268 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi099.front.sepia.ceph.com... 2022-06-29T15:50:25.268 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-06-29T15:50:25.268 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:50:25.283 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[13187]: fuse finished with error 0 and tester_r 0 2022-06-29T15:50:25.286 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:25.286 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:50:25.287 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:50:25.295 INFO:teuthology.orchestra.run.smithi099.stdout:ext2/ext3 2022-06-29T15:50:25.296 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-06-29T15:50:25.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-29T15:50:25.412 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:50:25.461 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:25.462 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:50:25.462 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:50:25.471 INFO:teuthology.orchestra.run.smithi164.stdout:fuseblk 2022-06-29T15:50:25.472 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-06-29T15:50:25.472 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi164.front.sepia.ceph.com... 2022-06-29T15:50:25.473 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-06-29T15:50:25.473 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:50:25.530 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi164.stderr:ceph-fuse[12728]: fuse finished with error 0 and tester_r 0 2022-06-29T15:50:25.532 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:25.532 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:50:25.533 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:50:25.539 INFO:teuthology.orchestra.run.smithi164.stdout:ext2/ext3 2022-06-29T15:50:25.540 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-06-29T15:50:25.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-29T15:50:25.641 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:50:25.648 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-06-29T15:50:25.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.755+0000 7fdf3c677700 1 Processor -- start 2022-06-29T15:50:25.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.755+0000 7fdf3c677700 1 -- start start 2022-06-29T15:50:25.763 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.755+0000 7fdf3c677700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdf3405f0b0 0x7fdf340b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:25.763 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.755+0000 7fdf3c677700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdf340b1260 0x7fdf3414b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:25.763 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.755+0000 7fdf3c677700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf340b0740 con 0x7fdf340b2120 2022-06-29T15:50:25.763 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.755+0000 7fdf3c677700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf34147460 con 0x7fdf3405f0b0 2022-06-29T15:50:25.764 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.755+0000 7fdf3c677700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf340b0970 con 0x7fdf340b1260 2022-06-29T15:50:25.764 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.755+0000 7fdf39c12700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdf3405f0b0 0x7fdf340b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:25.764 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.755+0000 7fdf39c12700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdf3405f0b0 0x7fdf340b2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52388/0 (socket says 172.21.15.47:52388) 2022-06-29T15:50:25.764 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.755+0000 7fdf39c12700 1 -- 172.21.15.47:0/3850729102 learned_addr learned my addr 172.21.15.47:0/3850729102 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:25.765 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.755+0000 7fdf3ac14700 1 --2- 172.21.15.47:0/3850729102 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdf340b1260 0x7fdf3414b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:25.765 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.755+0000 7fdf39c12700 1 -- 172.21.15.47:0/3850729102 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdf340b1260 msgr2=0x7fdf3414b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:25.765 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.755+0000 7fdf39c12700 1 --2- 172.21.15.47:0/3850729102 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdf340b1260 0x7fdf3414b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:25.765 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.755+0000 7fdf39c12700 1 -- 172.21.15.47:0/3850729102 >> v1:172.21.15.64:6789/0 conn(0x7fdf340b2120 legacy=0x7fdf34147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:25.766 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.755+0000 7fdf39c12700 1 -- 172.21.15.47:0/3850729102 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf340b0c10 con 0x7fdf3405f0b0 2022-06-29T15:50:25.766 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.755+0000 7fdf39c12700 1 --2- 172.21.15.47:0/3850729102 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf3405f0b0 0x7fdf340b2500 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 rx=0x7fdf34141a70 tx=0x7fdf24006eb0).ready entity=mon.0 client_cookie=cb31a7b82531041a server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:25.766 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.755+0000 7fdf39411700 1 -- 172.21.15.47:0/3850729102 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf2400d5c0 con 0x7fdf3405f0b0 2022-06-29T15:50:25.767 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.755+0000 7fdf39411700 1 -- 172.21.15.47:0/3850729102 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf24000e90 con 0x7fdf3405f0b0 2022-06-29T15:50:25.767 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.755+0000 7fdf39411700 1 -- 172.21.15.47:0/3850729102 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf2400d950 con 0x7fdf3405f0b0 2022-06-29T15:50:25.767 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.755+0000 7fdf3c677700 1 -- 172.21.15.47:0/3850729102 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf3405f0b0 msgr2=0x7fdf340b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:25.768 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.755+0000 7fdf3c677700 1 --2- 172.21.15.47:0/3850729102 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf3405f0b0 0x7fdf340b2500 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 rx=0x7fdf34141a70 tx=0x7fdf24006eb0).stop 2022-06-29T15:50:25.768 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.755+0000 7fdf3c677700 1 -- 172.21.15.47:0/3850729102 shutdown_connections 2022-06-29T15:50:25.768 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.755+0000 7fdf3c677700 1 --2- 172.21.15.47:0/3850729102 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdf340b1260 0x7fdf3414b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:25.768 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.755+0000 7fdf3c677700 1 --2- 172.21.15.47:0/3850729102 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf3405f0b0 0x7fdf340b2500 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:25.768 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.755+0000 7fdf3c677700 1 -- 172.21.15.47:0/3850729102 >> 172.21.15.47:0/3850729102 conn(0x7fdf340626f0 msgr2=0x7fdf34062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:25.769 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.755+0000 7fdf3c677700 1 -- 172.21.15.47:0/3850729102 shutdown_connections 2022-06-29T15:50:25.769 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.755+0000 7fdf3c677700 1 -- 172.21.15.47:0/3850729102 wait complete. 2022-06-29T15:50:25.772 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.759+0000 7fdf3c677700 1 Processor -- start 2022-06-29T15:50:25.772 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.759+0000 7fdf3c677700 1 -- start start 2022-06-29T15:50:25.772 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.759+0000 7fdf3c677700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf340b1260 0x7fdf34149d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:25.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.759+0000 7fdf3c677700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf340b2120 0x7fdf34141ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:25.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.759+0000 7fdf3c677700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdf34144f10 0x7fdf34141fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:25.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.759+0000 7fdf3c677700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf34144ab0 con 0x7fdf340b2120 2022-06-29T15:50:25.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.759+0000 7fdf3c677700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf340aec10 con 0x7fdf340b1260 2022-06-29T15:50:25.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.759+0000 7fdf3c677700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf340ae350 con 0x7fdf34144f10 2022-06-29T15:50:25.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.759+0000 7fdf39c12700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf340b2120 0x7fdf34141ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:25.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.759+0000 7fdf39c12700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf340b2120 0x7fdf34141ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52394/0 (socket says 172.21.15.47:52394) 2022-06-29T15:50:25.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.759+0000 7fdf39c12700 1 -- 172.21.15.47:0/3150949293 learned_addr learned my addr 172.21.15.47:0/3150949293 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:25.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.759+0000 7fdf3ac14700 1 --2- 172.21.15.47:0/3150949293 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdf34144f10 0x7fdf34141fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:25.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.759+0000 7fdf3a413700 1 --2- 172.21.15.47:0/3150949293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf340b1260 0x7fdf34149d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:25.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.759+0000 7fdf39c12700 1 -- 172.21.15.47:0/3150949293 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdf34144f10 msgr2=0x7fdf34141fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:25.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.759+0000 7fdf39c12700 1 --2- 172.21.15.47:0/3150949293 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdf34144f10 0x7fdf34141fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:25.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.759+0000 7fdf39c12700 1 -- 172.21.15.47:0/3150949293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf340b1260 msgr2=0x7fdf34149d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:25.776 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.759+0000 7fdf39c12700 1 --2- 172.21.15.47:0/3150949293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf340b1260 0x7fdf34149d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:25.776 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.759+0000 7fdf39c12700 1 -- 172.21.15.47:0/3150949293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf2400b1e0 con 0x7fdf340b2120 2022-06-29T15:50:25.776 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.759+0000 7fdf39c12700 1 --2- 172.21.15.47:0/3150949293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf340b2120 0x7fdf34141ab0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 rx=0x7fdf34141a70 tx=0x7fdf24015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:25.776 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.759+0000 7fdf237fe700 1 -- 172.21.15.47:0/3150949293 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf24011760 con 0x7fdf340b2120 2022-06-29T15:50:25.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.759+0000 7fdf237fe700 1 -- 172.21.15.47:0/3150949293 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf240118c0 con 0x7fdf340b2120 2022-06-29T15:50:25.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.759+0000 7fdf3c677700 1 -- 172.21.15.47:0/3150949293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf340ae5f0 con 0x7fdf340b2120 2022-06-29T15:50:25.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.759+0000 7fdf237fe700 1 -- 172.21.15.47:0/3150949293 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf24011c20 con 0x7fdf340b2120 2022-06-29T15:50:25.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.759+0000 7fdf3c677700 1 -- 172.21.15.47:0/3150949293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf34146920 con 0x7fdf340b2120 2022-06-29T15:50:25.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.763+0000 7fdf3c677700 1 -- 172.21.15.47:0/3150949293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf34144c20 con 0x7fdf340b2120 2022-06-29T15:50:25.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.763+0000 7fdf237fe700 1 -- 172.21.15.47:0/3150949293 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fdf2400f070 con 0x7fdf340b2120 2022-06-29T15:50:25.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.763+0000 7fdf237fe700 1 --2- 172.21.15.47:0/3150949293 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdf14053aa0 0x7fdf14055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:25.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.763+0000 7fdf237fe700 1 -- 172.21.15.47:0/3150949293 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fdf2406d850 con 0x7fdf340b2120 2022-06-29T15:50:25.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.763+0000 7fdf3a413700 1 --2- 172.21.15.47:0/3150949293 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdf14053aa0 0x7fdf14055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:25.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.763+0000 7fdf3a413700 1 --2- 172.21.15.47:0/3150949293 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdf14053aa0 0x7fdf14055f60 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7fdf3000d000 tx=0x7fdf30007b90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:25.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.771+0000 7fdf237fe700 1 -- 172.21.15.47:0/3150949293 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdf24044570 con 0x7fdf340b2120 2022-06-29T15:50:25.904 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.899+0000 7fdf3c677700 1 -- 172.21.15.47:0/3150949293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fdf340573d0 con 0x7fdf340b2120 2022-06-29T15:50:25.906 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.899+0000 7fdf237fe700 1 -- 172.21.15.47:0/3150949293 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v111) v1 ==== 81+0+34051 (secure 0 0 0) 0x7fdf24006a40 con 0x7fdf340b2120 2022-06-29T15:50:25.908 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:50:25.908 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2022-06-29T15:50:25.908 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 111, 2022-06-29T15:50:25.908 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "9a68894a-53ce-400e-838b-14eab91b834c", 2022-06-29T15:50:25.909 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2022-06-29T15:42:14.046361+0000", 2022-06-29T15:50:25.909 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2022-06-29T15:50:16.342650+0000", 2022-06-29T15:50:25.909 INFO:teuthology.orchestra.run.smithi047.stdout: "last_up_change": "2022-06-29T15:42:20.929789+0000", 2022-06-29T15:50:25.910 INFO:teuthology.orchestra.run.smithi047.stdout: "last_in_change": "2022-06-29T15:42:20.929789+0000", 2022-06-29T15:50:25.910 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-06-29T15:50:25.910 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_num": 5799936, 2022-06-29T15:50:25.910 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_set": [ 2022-06-29T15:50:25.910 INFO:teuthology.orchestra.run.smithi047.stdout: "pglog_hardlimit", 2022-06-29T15:50:25.911 INFO:teuthology.orchestra.run.smithi047.stdout: "purged_snapdirs", 2022-06-29T15:50:25.911 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_deletes", 2022-06-29T15:50:25.911 INFO:teuthology.orchestra.run.smithi047.stdout: "sortbitwise" 2022-06-29T15:50:25.911 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:50:25.912 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_version": 5, 2022-06-29T15:50:25.912 INFO:teuthology.orchestra.run.smithi047.stdout: "full_ratio": 0.94999998807907104, 2022-06-29T15:50:25.912 INFO:teuthology.orchestra.run.smithi047.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-06-29T15:50:25.912 INFO:teuthology.orchestra.run.smithi047.stdout: "nearfull_ratio": 0.85000002384185791, 2022-06-29T15:50:25.913 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_snapshot": "", 2022-06-29T15:50:25.913 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_max": 20, 2022-06-29T15:50:25.913 INFO:teuthology.orchestra.run.smithi047.stdout: "max_osd": 8, 2022-06-29T15:50:25.913 INFO:teuthology.orchestra.run.smithi047.stdout: "require_min_compat_client": "jewel", 2022-06-29T15:50:25.913 INFO:teuthology.orchestra.run.smithi047.stdout: "min_compat_client": "jewel", 2022-06-29T15:50:25.914 INFO:teuthology.orchestra.run.smithi047.stdout: "require_osd_release": "octopus", 2022-06-29T15:50:25.914 INFO:teuthology.orchestra.run.smithi047.stdout: "pools": [ 2022-06-29T15:50:25.914 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:25.914 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 1, 2022-06-29T15:50:25.915 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "device_health_metrics", 2022-06-29T15:50:25.915 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:42:16.971431+0000", 2022-06-29T15:50:25.915 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:50:25.915 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:50:25.916 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:50:25.916 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:50:25.916 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:50:25.916 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:50:25.917 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:50:25.917 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:50:25.917 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 1, 2022-06-29T15:50:25.917 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 1, 2022-06-29T15:50:25.917 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 1, 2022-06-29T15:50:25.918 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 1, 2022-06-29T15:50:25.918 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 1, 2022-06-29T15:50:25.918 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:50:25.918 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:50:25.919 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:50:25.919 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:50:25.919 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:50:25.919 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:50:25.919 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:50:25.920 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:25.920 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "16", 2022-06-29T15:50:25.920 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:50:25.920 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:50:25.921 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:50:25.921 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:50:25.921 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:50:25.921 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:50:25.922 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:50:25.922 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:50:25.922 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:50:25.922 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:50:25.923 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:50:25.923 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:50:25.923 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:50:25.923 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:50:25.924 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:50:25.924 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:50:25.924 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:50:25.924 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:50:25.924 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:50:25.925 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:50:25.925 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:50:25.925 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:50:25.925 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:50:25.926 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:50:25.926 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:50:25.926 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:50:25.926 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:25.927 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:50:25.927 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:50:25.927 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:50:25.927 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:50:25.927 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:50:25.928 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:50:25.928 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:50:25.928 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:50:25.928 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:50:25.929 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:50:25.929 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:50:25.929 INFO:teuthology.orchestra.run.smithi047.stdout: "options": { 2022-06-29T15:50:25.929 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_min": 1 2022-06-29T15:50:25.930 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:25.930 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:50:25.930 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_devicehealth": {} 2022-06-29T15:50:25.930 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:25.930 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:25.931 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:25.931 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 2, 2022-06-29T15:50:25.931 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "rbd", 2022-06-29T15:50:25.931 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:42:25.290884+0000", 2022-06-29T15:50:25.932 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:50:25.932 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:50:25.932 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:50:25.932 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:50:25.933 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:50:25.933 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:50:25.933 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:50:25.933 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:50:25.933 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2022-06-29T15:50:25.934 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2022-06-29T15:50:25.934 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2022-06-29T15:50:25.934 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2022-06-29T15:50:25.934 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2022-06-29T15:50:25.935 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:50:25.935 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:50:25.935 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:50:25.935 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:50:25.936 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:50:25.936 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:50:25.936 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:50:25.936 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:25.936 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "19", 2022-06-29T15:50:25.937 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:50:25.937 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:50:25.937 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:50:25.937 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:50:25.938 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:50:25.938 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:50:25.938 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:50:25.938 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:50:25.939 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:50:25.939 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:50:25.939 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:50:25.939 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:50:25.939 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:50:25.940 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:50:25.940 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:50:25.940 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:50:25.940 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:50:25.941 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:50:25.941 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:50:25.941 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:50:25.941 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:50:25.942 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:50:25.942 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:50:25.942 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:50:25.942 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:50:25.943 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:50:25.943 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:25.943 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:50:25.943 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:50:25.944 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:50:25.944 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:50:25.944 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:50:25.944 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:50:25.945 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:50:25.945 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:50:25.945 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:50:25.945 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:50:25.945 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:50:25.946 INFO:teuthology.orchestra.run.smithi047.stdout: "options": {}, 2022-06-29T15:50:25.946 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:50:25.946 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd": {} 2022-06-29T15:50:25.946 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:25.947 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:25.947 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:25.947 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 19, 2022-06-29T15:50:25.947 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "cephfs_metadata", 2022-06-29T15:50:25.947 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:50:13.772527+0000", 2022-06-29T15:50:25.948 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:50:25.948 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:50:25.948 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:50:25.948 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:50:25.949 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:50:25.949 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:50:25.949 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:50:25.949 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:50:25.950 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2022-06-29T15:50:25.950 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2022-06-29T15:50:25.950 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2022-06-29T15:50:25.950 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2022-06-29T15:50:25.950 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2022-06-29T15:50:25.951 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:50:25.951 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:50:25.951 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:50:25.951 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:50:25.952 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:50:25.952 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:50:25.952 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:50:25.952 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:25.953 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "110", 2022-06-29T15:50:25.953 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:50:25.953 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:50:25.953 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:50:25.953 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:50:25.954 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:50:25.954 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:50:25.954 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:50:25.954 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:50:25.955 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:50:25.955 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:50:25.955 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:50:25.955 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:50:25.956 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:50:25.956 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:50:25.956 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:50:25.956 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:50:25.956 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:50:25.957 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:50:25.957 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:50:25.957 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:50:25.957 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:50:25.958 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:50:25.958 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:50:25.958 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:50:25.958 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:50:25.959 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:50:25.959 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:25.959 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:50:25.959 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:50:25.960 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:50:25.960 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:50:25.960 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:50:25.960 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:50:25.961 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:50:25.961 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:50:25.961 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:50:25.961 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:50:25.962 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:50:25.962 INFO:teuthology.orchestra.run.smithi047.stdout: "options": { 2022-06-29T15:50:25.962 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_bias": 4, 2022-06-29T15:50:25.962 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_min": 16, 2022-06-29T15:50:25.962 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_priority": 5 2022-06-29T15:50:25.963 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:25.963 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:50:25.963 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs": { 2022-06-29T15:50:25.963 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": "cephfs" 2022-06-29T15:50:25.964 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:25.964 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:25.964 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:25.964 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:25.965 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 20, 2022-06-29T15:50:25.965 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "cephfs_data", 2022-06-29T15:50:25.965 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:50:14.675947+0000", 2022-06-29T15:50:25.965 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:50:25.965 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:50:25.966 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:50:25.966 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:50:25.966 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:50:25.966 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:50:25.967 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:50:25.967 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:50:25.967 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2022-06-29T15:50:25.967 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2022-06-29T15:50:25.968 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2022-06-29T15:50:25.968 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2022-06-29T15:50:25.968 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2022-06-29T15:50:25.968 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:50:25.968 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:50:25.969 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:50:25.969 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:50:25.969 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:50:25.969 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:50:25.970 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:50:25.970 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:25.970 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "111", 2022-06-29T15:50:25.970 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:50:25.970 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:50:25.971 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:50:25.971 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:50:25.971 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:50:25.971 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:50:25.972 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:50:25.972 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:50:25.972 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:50:25.972 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:50:25.973 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:50:25.973 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:50:25.973 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:50:25.973 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:50:25.973 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:50:25.974 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:50:25.974 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:50:25.974 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:50:25.974 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:50:25.975 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:50:25.975 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:50:25.975 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:50:25.975 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:50:25.976 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:50:25.976 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:50:25.976 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:50:25.976 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:25.976 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:50:25.977 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:50:25.977 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:50:25.977 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:50:25.977 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:50:25.978 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:50:25.978 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:50:25.978 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:50:25.978 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:50:25.979 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:50:25.979 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:50:25.979 INFO:teuthology.orchestra.run.smithi047.stdout: "options": {}, 2022-06-29T15:50:25.979 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:50:25.979 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs": { 2022-06-29T15:50:25.980 INFO:teuthology.orchestra.run.smithi047.stdout: "data": "cephfs" 2022-06-29T15:50:25.980 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:25.980 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:25.980 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:25.981 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:50:25.981 INFO:teuthology.orchestra.run.smithi047.stdout: "osds": [ 2022-06-29T15:50:25.981 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:25.981 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2022-06-29T15:50:25.981 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "e23eb023-4706-4201-bf72-4bf19c912c5d", 2022-06-29T15:50:25.982 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:50:25.982 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:50:25.982 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:50:25.982 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:50:25.983 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:50:25.983 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:50:25.983 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:50:25.983 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 109, 2022-06-29T15:50:25.984 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:50:25.984 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:50:25.984 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:50:25.984 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:25.985 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:25.985 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:25.985 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6818", 2022-06-29T15:50:25.985 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:50:25.986 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:25.986 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:25.986 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:25.986 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6819", 2022-06-29T15:50:25.987 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:50:25.987 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:25.987 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:25.987 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:25.988 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:50:25.988 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:25.988 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:25.988 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:25.988 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6820", 2022-06-29T15:50:25.989 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:50:25.989 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:25.989 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:25.989 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:25.990 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6821", 2022-06-29T15:50:25.990 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:50:25.990 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:25.990 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:25.991 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:25.991 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:50:25.991 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:25.991 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:25.991 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:25.992 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6824", 2022-06-29T15:50:25.992 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:50:25.992 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:25.992 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:25.993 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:25.993 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6825", 2022-06-29T15:50:25.993 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:50:25.993 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:25.993 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:25.994 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:25.994 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:50:25.994 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:25.994 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:25.995 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:25.995 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6822", 2022-06-29T15:50:25.995 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:50:25.995 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:25.995 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:25.996 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:25.996 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6823", 2022-06-29T15:50:25.996 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:50:25.996 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:25.997 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:25.997 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:25.997 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6819/13096", 2022-06-29T15:50:25.997 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6821/13096", 2022-06-29T15:50:25.998 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6825/13096", 2022-06-29T15:50:25.998 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6823/13096", 2022-06-29T15:50:25.998 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:50:25.998 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:50:25.999 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:50:25.999 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:25.999 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:25.999 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:25.999 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2022-06-29T15:50:26.000 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "228bb08d-2d64-43d9-9560-bfaeff154452", 2022-06-29T15:50:26.000 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:50:26.000 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:50:26.001 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:50:26.001 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:50:26.001 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:50:26.001 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:50:26.001 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:50:26.002 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 109, 2022-06-29T15:50:26.002 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:50:26.002 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:50:26.002 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:50:26.003 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:26.003 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.003 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:26.003 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6826", 2022-06-29T15:50:26.004 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:50:26.004 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.004 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.004 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:26.004 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6827", 2022-06-29T15:50:26.005 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:50:26.005 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:26.005 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:26.005 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.006 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:50:26.006 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:26.006 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.006 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:26.007 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6828", 2022-06-29T15:50:26.007 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:50:26.007 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.007 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.007 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:26.008 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6829", 2022-06-29T15:50:26.008 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:50:26.008 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:26.008 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:26.009 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.009 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:50:26.009 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:26.009 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.010 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:26.010 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6832", 2022-06-29T15:50:26.010 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:50:26.010 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.011 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.011 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:26.011 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6833", 2022-06-29T15:50:26.011 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:50:26.011 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:26.012 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:26.012 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.012 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:50:26.012 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:26.013 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.013 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:26.013 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6830", 2022-06-29T15:50:26.013 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:50:26.013 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.014 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.014 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:26.014 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6831", 2022-06-29T15:50:26.014 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:50:26.015 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:26.015 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:26.015 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.015 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6827/13100", 2022-06-29T15:50:26.016 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6829/13100", 2022-06-29T15:50:26.016 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6833/13100", 2022-06-29T15:50:26.016 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6831/13100", 2022-06-29T15:50:26.016 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:50:26.017 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:50:26.017 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:50:26.017 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:26.017 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.018 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.018 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2022-06-29T15:50:26.018 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "797a5452-1510-4640-a111-c9c2fcf707eb", 2022-06-29T15:50:26.018 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:50:26.018 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:50:26.019 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:50:26.019 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:50:26.019 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:50:26.019 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:50:26.020 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:50:26.020 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 108, 2022-06-29T15:50:26.020 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:50:26.020 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:50:26.020 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:50:26.021 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:26.021 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.021 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:26.021 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6810", 2022-06-29T15:50:26.022 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:50:26.022 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.022 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.022 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:26.023 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6811", 2022-06-29T15:50:26.023 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:50:26.023 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:26.023 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:26.024 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.024 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:50:26.024 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:26.024 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.025 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:26.025 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6812", 2022-06-29T15:50:26.025 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:50:26.025 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.026 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.026 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:26.026 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6813", 2022-06-29T15:50:26.026 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:50:26.027 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:26.027 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:26.027 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.027 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:50:26.028 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:26.028 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.028 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:26.028 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6816", 2022-06-29T15:50:26.029 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:50:26.029 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.029 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.029 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:26.030 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6817", 2022-06-29T15:50:26.030 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:50:26.030 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:26.030 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:26.030 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.031 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:50:26.031 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:26.031 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.031 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:26.032 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6814", 2022-06-29T15:50:26.032 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:50:26.032 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.032 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.032 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:26.033 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6815", 2022-06-29T15:50:26.033 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:50:26.033 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:26.033 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:26.034 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.034 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6811/13098", 2022-06-29T15:50:26.034 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6813/13098", 2022-06-29T15:50:26.034 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6817/13098", 2022-06-29T15:50:26.035 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6815/13098", 2022-06-29T15:50:26.035 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:50:26.035 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:50:26.035 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:50:26.035 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:26.036 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.036 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.036 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2022-06-29T15:50:26.036 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "55b24fd7-bb71-4a01-b3fa-a09caa772a62", 2022-06-29T15:50:26.037 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:50:26.037 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:50:26.037 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:50:26.037 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:50:26.037 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:50:26.038 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:50:26.038 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:50:26.038 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 109, 2022-06-29T15:50:26.038 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:50:26.039 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:50:26.039 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:50:26.039 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:26.039 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.040 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:26.040 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6802", 2022-06-29T15:50:26.040 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:50:26.040 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.041 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.041 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:26.041 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6803", 2022-06-29T15:50:26.041 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:50:26.041 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:26.042 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:26.042 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.042 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:50:26.042 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:26.043 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.043 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:26.043 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6804", 2022-06-29T15:50:26.043 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:50:26.044 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.044 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.044 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:26.044 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6805", 2022-06-29T15:50:26.044 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:50:26.045 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:26.045 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:26.045 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.045 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:50:26.046 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:26.046 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.046 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:26.046 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6808", 2022-06-29T15:50:26.046 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:50:26.047 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.047 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.047 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:26.047 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6809", 2022-06-29T15:50:26.048 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:50:26.048 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:26.048 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:26.048 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.049 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:50:26.049 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:26.049 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.049 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:26.049 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6806", 2022-06-29T15:50:26.050 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:50:26.050 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.050 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.050 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:26.051 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6807", 2022-06-29T15:50:26.051 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:50:26.051 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:26.051 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:26.052 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.052 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6803/13097", 2022-06-29T15:50:26.052 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6805/13097", 2022-06-29T15:50:26.052 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6809/13097", 2022-06-29T15:50:26.053 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6807/13097", 2022-06-29T15:50:26.053 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:50:26.053 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:50:26.053 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:50:26.053 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:26.054 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.054 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.054 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2022-06-29T15:50:26.054 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3", 2022-06-29T15:50:26.055 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:50:26.055 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:50:26.055 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:50:26.055 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:50:26.056 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:50:26.056 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:50:26.056 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:50:26.056 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 108, 2022-06-29T15:50:26.056 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:50:26.057 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:50:26.057 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:50:26.057 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:26.057 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.058 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:26.058 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6816", 2022-06-29T15:50:26.058 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:50:26.058 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.059 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.059 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:26.059 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6817", 2022-06-29T15:50:26.059 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:50:26.060 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:26.060 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:26.060 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.060 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:50:26.060 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:26.061 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.061 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:26.061 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6818", 2022-06-29T15:50:26.062 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:50:26.062 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.062 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.062 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:26.063 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6819", 2022-06-29T15:50:26.063 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:50:26.063 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:26.063 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:26.064 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.064 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:50:26.064 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:26.064 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.065 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:26.065 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6822", 2022-06-29T15:50:26.065 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:50:26.065 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.066 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.066 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:26.066 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6823", 2022-06-29T15:50:26.067 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:50:26.067 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:26.067 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:26.067 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.068 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:50:26.068 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:26.068 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.069 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:26.069 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6820", 2022-06-29T15:50:26.069 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:50:26.069 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.070 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.070 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:26.070 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6821", 2022-06-29T15:50:26.070 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:50:26.071 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:26.071 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:26.071 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.071 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6817/13396", 2022-06-29T15:50:26.072 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6819/13396", 2022-06-29T15:50:26.072 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6823/13396", 2022-06-29T15:50:26.072 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6821/13396", 2022-06-29T15:50:26.072 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:50:26.073 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:50:26.073 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:50:26.073 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:26.073 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.073 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.074 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2022-06-29T15:50:26.074 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "f77beebe-6cca-4244-b2d4-677825ebf919", 2022-06-29T15:50:26.074 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:50:26.074 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:50:26.075 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:50:26.075 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:50:26.075 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:50:26.075 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:50:26.076 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:50:26.076 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 108, 2022-06-29T15:50:26.076 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:50:26.076 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:50:26.076 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:50:26.077 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:26.077 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.077 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:26.077 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6800", 2022-06-29T15:50:26.078 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:50:26.078 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.078 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.078 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:26.079 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6801", 2022-06-29T15:50:26.079 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:50:26.079 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:26.079 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:26.079 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.080 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:50:26.080 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:26.080 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.080 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:26.081 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6802", 2022-06-29T15:50:26.081 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:50:26.081 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.081 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.081 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:26.082 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6803", 2022-06-29T15:50:26.082 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:50:26.082 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:26.082 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:26.083 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.083 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:50:26.083 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:26.083 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.084 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:26.084 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6806", 2022-06-29T15:50:26.084 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:50:26.084 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.085 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.085 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:26.085 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6807", 2022-06-29T15:50:26.085 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:50:26.086 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:26.086 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:26.086 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.086 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:50:26.086 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:26.087 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.087 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:26.087 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6804", 2022-06-29T15:50:26.087 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:50:26.088 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.088 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.088 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:26.088 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6805", 2022-06-29T15:50:26.088 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:50:26.089 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:26.089 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:26.089 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.089 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6801/13393", 2022-06-29T15:50:26.090 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6803/13393", 2022-06-29T15:50:26.090 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6807/13393", 2022-06-29T15:50:26.090 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6805/13393", 2022-06-29T15:50:26.091 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:50:26.091 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:50:26.091 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:50:26.092 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:26.092 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.092 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.093 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2022-06-29T15:50:26.093 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "af0ca9b3-c02c-4910-834b-bf0b8dc6713c", 2022-06-29T15:50:26.094 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:50:26.094 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:50:26.094 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:50:26.095 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:50:26.095 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:50:26.096 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:50:26.096 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:50:26.096 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 109, 2022-06-29T15:50:26.097 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:50:26.097 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:50:26.098 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:50:26.098 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:26.099 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.099 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:26.100 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6824", 2022-06-29T15:50:26.100 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:50:26.100 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.101 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.101 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:26.101 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6825", 2022-06-29T15:50:26.102 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:50:26.102 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:26.102 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:26.102 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.103 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:50:26.103 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:26.103 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.103 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:26.103 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6826", 2022-06-29T15:50:26.104 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:50:26.104 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.104 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.104 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:26.105 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6827", 2022-06-29T15:50:26.105 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:50:26.105 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:26.105 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:26.106 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.106 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:50:26.106 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:26.106 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.107 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:26.107 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6830", 2022-06-29T15:50:26.107 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:50:26.107 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.107 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.108 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:26.108 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6831", 2022-06-29T15:50:26.108 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:50:26.108 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:26.109 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:26.109 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.109 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:50:26.109 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:26.109 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.110 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:26.110 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6828", 2022-06-29T15:50:26.110 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:50:26.111 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.111 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.111 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:26.111 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6829", 2022-06-29T15:50:26.111 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:50:26.112 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:26.112 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:26.112 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.112 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6825/13395", 2022-06-29T15:50:26.113 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6827/13395", 2022-06-29T15:50:26.113 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6831/13395", 2022-06-29T15:50:26.113 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6829/13395", 2022-06-29T15:50:26.113 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:50:26.113 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:50:26.114 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:50:26.114 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:26.114 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.114 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.115 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2022-06-29T15:50:26.115 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "f19c3ca6-df88-4bde-a150-b51a53f092e1", 2022-06-29T15:50:26.115 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:50:26.115 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:50:26.116 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:50:26.116 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:50:26.116 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:50:26.116 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:50:26.116 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:50:26.117 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 109, 2022-06-29T15:50:26.117 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:50:26.117 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:50:26.117 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:50:26.118 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:26.118 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.118 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:26.118 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6808", 2022-06-29T15:50:26.118 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:50:26.119 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.119 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.119 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:26.119 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6809", 2022-06-29T15:50:26.120 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:50:26.120 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:26.120 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:26.120 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.120 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:50:26.121 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:26.121 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.121 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:26.121 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6810", 2022-06-29T15:50:26.122 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:50:26.122 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.122 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.122 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:26.123 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6811", 2022-06-29T15:50:26.123 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:50:26.123 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:26.123 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:26.124 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.124 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:50:26.124 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:26.124 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.124 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:26.125 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6814", 2022-06-29T15:50:26.125 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:50:26.125 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.125 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.126 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:26.126 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6815", 2022-06-29T15:50:26.126 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:50:26.126 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:26.127 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:26.127 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.127 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:50:26.127 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:26.128 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.128 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:26.128 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6812", 2022-06-29T15:50:26.128 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:50:26.129 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.129 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.129 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:26.129 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6813", 2022-06-29T15:50:26.130 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:50:26.130 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:26.130 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:26.130 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.131 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6809/13394", 2022-06-29T15:50:26.131 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6811/13394", 2022-06-29T15:50:26.131 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6815/13394", 2022-06-29T15:50:26.132 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6813/13394", 2022-06-29T15:50:26.132 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:50:26.132 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:50:26.132 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:50:26.132 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:26.133 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:26.133 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:50:26.133 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_xinfo": [ 2022-06-29T15:50:26.133 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.134 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2022-06-29T15:50:26.134 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:50:26.134 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:50:26.134 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:50:26.135 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:50:26.135 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:50:26.135 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.720513+0000", 2022-06-29T15:50:26.135 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:50:26.136 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.136 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.136 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2022-06-29T15:50:26.136 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:50:26.137 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:50:26.137 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:50:26.137 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:50:26.137 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:50:26.137 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.772612+0000", 2022-06-29T15:50:26.138 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:50:26.138 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.138 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.138 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2022-06-29T15:50:26.139 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:50:26.139 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:50:26.139 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:50:26.139 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:50:26.139 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:50:26.140 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.450223+0000", 2022-06-29T15:50:26.140 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:50:26.140 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.140 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.141 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2022-06-29T15:50:26.141 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:50:26.141 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:50:26.141 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:50:26.142 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:50:26.142 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:50:26.142 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.477856+0000", 2022-06-29T15:50:26.142 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:50:26.143 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.143 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.143 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2022-06-29T15:50:26.143 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:50:26.143 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:50:26.144 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:50:26.144 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:50:26.144 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:50:26.144 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.539485+0000", 2022-06-29T15:50:26.145 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:50:26.145 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.145 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.145 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2022-06-29T15:50:26.146 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:50:26.146 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:50:26.146 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:50:26.146 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:50:26.147 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:50:26.147 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.487040+0000", 2022-06-29T15:50:26.147 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:50:26.147 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.148 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.148 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2022-06-29T15:50:26.148 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:50:26.148 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:50:26.149 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:50:26.149 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:50:26.149 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:50:26.149 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.748685+0000", 2022-06-29T15:50:26.150 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:50:26.150 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.150 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:26.150 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2022-06-29T15:50:26.151 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:50:26.151 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:50:26.151 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:50:26.151 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:50:26.152 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:50:26.152 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.459516+0000", 2022-06-29T15:50:26.152 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:50:26.152 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:26.153 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:50:26.153 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap": [], 2022-06-29T15:50:26.153 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap_items": [], 2022-06-29T15:50:26.153 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_temp": [], 2022-06-29T15:50:26.153 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_temp": [], 2022-06-29T15:50:26.154 INFO:teuthology.orchestra.run.smithi047.stdout: "blacklist": {}, 2022-06-29T15:50:26.154 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profiles": { 2022-06-29T15:50:26.154 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs_data": { 2022-06-29T15:50:26.154 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-device-class": "", 2022-06-29T15:50:26.155 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2022-06-29T15:50:26.155 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-root": "default", 2022-06-29T15:50:26.155 INFO:teuthology.orchestra.run.smithi047.stdout: "jerasure-per-chunk-alignment": "false", 2022-06-29T15:50:26.155 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2022-06-29T15:50:26.155 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "2", 2022-06-29T15:50:26.156 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2022-06-29T15:50:26.156 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2022-06-29T15:50:26.156 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van", 2022-06-29T15:50:26.156 INFO:teuthology.orchestra.run.smithi047.stdout: "w": "8" 2022-06-29T15:50:26.157 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.157 INFO:teuthology.orchestra.run.smithi047.stdout: "default": { 2022-06-29T15:50:26.157 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2022-06-29T15:50:26.157 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2022-06-29T15:50:26.157 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "1", 2022-06-29T15:50:26.158 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2022-06-29T15:50:26.158 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2022-06-29T15:50:26.158 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van" 2022-06-29T15:50:26.158 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:26.159 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:26.159 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps_queue": [], 2022-06-29T15:50:26.159 INFO:teuthology.orchestra.run.smithi047.stdout: "new_removed_snaps": [], 2022-06-29T15:50:26.159 INFO:teuthology.orchestra.run.smithi047.stdout: "new_purged_snaps": [], 2022-06-29T15:50:26.160 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_node_flags": {}, 2022-06-29T15:50:26.160 INFO:teuthology.orchestra.run.smithi047.stdout: "device_class_flags": {} 2022-06-29T15:50:26.160 INFO:teuthology.orchestra.run.smithi047.stdout:} 2022-06-29T15:50:26.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.903+0000 7fdf3c677700 1 -- 172.21.15.47:0/3150949293 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdf14053aa0 msgr2=0x7fdf14055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:26.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.903+0000 7fdf3c677700 1 --2- 172.21.15.47:0/3150949293 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdf14053aa0 0x7fdf14055f60 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7fdf3000d000 tx=0x7fdf30007b90).stop 2022-06-29T15:50:26.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.903+0000 7fdf3c677700 1 -- 172.21.15.47:0/3150949293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf340b2120 msgr2=0x7fdf34141ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:26.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.903+0000 7fdf3c677700 1 --2- 172.21.15.47:0/3150949293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf340b2120 0x7fdf34141ab0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 rx=0x7fdf34141a70 tx=0x7fdf24015000).stop 2022-06-29T15:50:26.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.903+0000 7fdf3c677700 1 -- 172.21.15.47:0/3150949293 shutdown_connections 2022-06-29T15:50:26.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.903+0000 7fdf3c677700 1 --2- 172.21.15.47:0/3150949293 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdf34144f10 0x7fdf34141fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:26.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.903+0000 7fdf3c677700 1 --2- 172.21.15.47:0/3150949293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf340b2120 0x7fdf34141ab0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:26.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.903+0000 7fdf3c677700 1 --2- 172.21.15.47:0/3150949293 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdf14053aa0 0x7fdf14055f60 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:26.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.903+0000 7fdf3c677700 1 --2- 172.21.15.47:0/3150949293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf340b1260 0x7fdf34149d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:26.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.903+0000 7fdf3c677700 1 -- 172.21.15.47:0/3150949293 >> 172.21.15.47:0/3150949293 conn(0x7fdf340626f0 msgr2=0x7fdf340b3c70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:26.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.903+0000 7fdf3c677700 1 -- 172.21.15.47:0/3150949293 shutdown_connections 2022-06-29T15:50:26.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:25.903+0000 7fdf3c677700 1 -- 172.21.15.47:0/3150949293 wait complete. 2022-06-29T15:50:26.167 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:50:26.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.271+0000 7f7a0f59e700 1 Processor -- start 2022-06-29T15:50:26.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.271+0000 7f7a0f59e700 1 -- start start 2022-06-29T15:50:26.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.271+0000 7f7a0f59e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7a100b5ac0 0x7f7a101401c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:26.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.271+0000 7f7a0f59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7a100b02d0 0x7f7a101406b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:26.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.271+0000 7f7a0f59e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7a100afdc0 0x7f7a100ae0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:26.282 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.271+0000 7f7a0f59e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a100a9090 con 0x7f7a100afdc0 2022-06-29T15:50:26.282 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.271+0000 7f7a0f59e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a10143d20 con 0x7f7a100b02d0 2022-06-29T15:50:26.282 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.271+0000 7f7a0f59e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a10140ba0 con 0x7f7a100b5ac0 2022-06-29T15:50:26.282 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.271+0000 7f7a0ed9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7a100afdc0 0x7f7a100ae0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:26.283 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.271+0000 7f7a0ed9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7a100afdc0 0x7f7a100ae0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52402/0 (socket says 172.21.15.47:52402) 2022-06-29T15:50:26.283 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.271+0000 7f7a0ed9d700 1 -- 172.21.15.47:0/2535821950 learned_addr learned my addr 172.21.15.47:0/2535821950 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:26.283 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.271+0000 7f7a0e59c700 1 --2- 172.21.15.47:0/2535821950 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7a100b5ac0 0x7f7a101401c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:26.283 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.271+0000 7f7a0dd9b700 1 --2- 172.21.15.47:0/2535821950 >> v2:172.21.15.64:3300/0 conn(0x7f7a100b02d0 0x7f7a101406b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:26.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.275+0000 7f7a0ed9d700 1 -- 172.21.15.47:0/2535821950 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7a100b5ac0 msgr2=0x7f7a101401c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:26.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.275+0000 7f7a0ed9d700 1 --2- 172.21.15.47:0/2535821950 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7a100b5ac0 0x7f7a101401c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:26.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.275+0000 7f7a0ed9d700 1 -- 172.21.15.47:0/2535821950 >> v2:172.21.15.64:3300/0 conn(0x7f7a100b02d0 msgr2=0x7f7a101406b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:26.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.275+0000 7f7a0ed9d700 1 --2- 172.21.15.47:0/2535821950 >> v2:172.21.15.64:3300/0 conn(0x7f7a100b02d0 0x7f7a101406b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:26.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.275+0000 7f7a0ed9d700 1 -- 172.21.15.47:0/2535821950 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a100a8df0 con 0x7f7a100afdc0 2022-06-29T15:50:26.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.275+0000 7f7a0ed9d700 1 --2- 172.21.15.47:0/2535821950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a100afdc0 0x7f7a100ae0c0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 rx=0x7f7a00002f70 tx=0x7f7a00006eb0).ready entity=mon.0 client_cookie=29ba250e50c6118 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:26.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.275+0000 7f7a0d59a700 1 -- 172.21.15.47:0/2535821950 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a0000d570 con 0x7f7a100afdc0 2022-06-29T15:50:26.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.275+0000 7f7a0d59a700 1 -- 172.21.15.47:0/2535821950 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a00000e90 con 0x7f7a100afdc0 2022-06-29T15:50:26.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.275+0000 7f7a0d59a700 1 -- 172.21.15.47:0/2535821950 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a0000d900 con 0x7f7a100afdc0 2022-06-29T15:50:26.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.275+0000 7f7a0f59e700 1 -- 172.21.15.47:0/2535821950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a100afdc0 msgr2=0x7f7a100ae0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:26.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.275+0000 7f7a0f59e700 1 --2- 172.21.15.47:0/2535821950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a100afdc0 0x7f7a100ae0c0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 rx=0x7f7a00002f70 tx=0x7f7a00006eb0).stop 2022-06-29T15:50:26.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.275+0000 7f7a0f59e700 1 -- 172.21.15.47:0/2535821950 shutdown_connections 2022-06-29T15:50:26.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.275+0000 7f7a0f59e700 1 --2- 172.21.15.47:0/2535821950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a100afdc0 0x7f7a100ae0c0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:26.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.275+0000 7f7a0f59e700 1 --2- 172.21.15.47:0/2535821950 >> v2:172.21.15.64:3300/0 conn(0x7f7a100b02d0 0x7f7a101406b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:26.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.275+0000 7f7a0f59e700 1 --2- 172.21.15.47:0/2535821950 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7a100b5ac0 0x7f7a101401c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:26.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.275+0000 7f7a0f59e700 1 -- 172.21.15.47:0/2535821950 >> 172.21.15.47:0/2535821950 conn(0x7f7a10062560 msgr2=0x7f7a10062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:26.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.275+0000 7f7a0f59e700 1 -- 172.21.15.47:0/2535821950 shutdown_connections 2022-06-29T15:50:26.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.275+0000 7f7a0f59e700 1 -- 172.21.15.47:0/2535821950 wait complete. 2022-06-29T15:50:26.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.279+0000 7f7a0f59e700 1 Processor -- start 2022-06-29T15:50:26.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.279+0000 7f7a0f59e700 1 -- start start 2022-06-29T15:50:26.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.279+0000 7f7a0f59e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7a100afdc0 0x7f7a100aa380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:26.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.279+0000 7f7a0f59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a100b02d0 0x7f7a10145f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:26.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.279+0000 7f7a0f59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a100b5ac0 0x7f7a10146450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:26.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.279+0000 7f7a0f59e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a10146d00 con 0x7f7a100b5ac0 2022-06-29T15:50:26.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.279+0000 7f7a0f59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a10146940 con 0x7f7a100b02d0 2022-06-29T15:50:26.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.279+0000 7f7a0f59e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a100aab60 con 0x7f7a100afdc0 2022-06-29T15:50:26.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.279+0000 7f7a0ed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a100b5ac0 0x7f7a10146450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:26.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.279+0000 7f7a0ed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a100b5ac0 0x7f7a10146450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52410/0 (socket says 172.21.15.47:52410) 2022-06-29T15:50:26.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.279+0000 7f7a0ed9d700 1 -- 172.21.15.47:0/102599893 learned_addr learned my addr 172.21.15.47:0/102599893 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:26.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.279+0000 7f7a0e59c700 1 --2- 172.21.15.47:0/102599893 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7a100afdc0 0x7f7a100aa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:26.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.279+0000 7f7a0dd9b700 1 --2- 172.21.15.47:0/102599893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a100b02d0 0x7f7a10145f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:26.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.279+0000 7f7a0ed9d700 1 -- 172.21.15.47:0/102599893 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7a100afdc0 msgr2=0x7f7a100aa380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:26.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.279+0000 7f7a0ed9d700 1 --2- 172.21.15.47:0/102599893 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7a100afdc0 0x7f7a100aa380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:26.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.279+0000 7f7a0ed9d700 1 -- 172.21.15.47:0/102599893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a100b02d0 msgr2=0x7f7a10145f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:26.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.279+0000 7f7a0ed9d700 1 --2- 172.21.15.47:0/102599893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a100b02d0 0x7f7a10145f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:26.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.279+0000 7f7a0ed9d700 1 -- 172.21.15.47:0/102599893 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a0000b1e0 con 0x7f7a100b5ac0 2022-06-29T15:50:26.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.279+0000 7f7a0dd9b700 1 --2- 172.21.15.47:0/102599893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a100b02d0 0x7f7a10145f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:50:26.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.283+0000 7f7a0ed9d700 1 --2- 172.21.15.47:0/102599893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a100b5ac0 0x7f7a10146450 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 rx=0x7f7a00002f70 tx=0x7f7a00015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:26.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.283+0000 7f79ff7fe700 1 -- 172.21.15.47:0/102599893 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a000116e0 con 0x7f7a100b5ac0 2022-06-29T15:50:26.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.283+0000 7f79ff7fe700 1 -- 172.21.15.47:0/102599893 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a00011840 con 0x7f7a100b5ac0 2022-06-29T15:50:26.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.283+0000 7f7a0f59e700 1 -- 172.21.15.47:0/102599893 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a10146f30 con 0x7f7a100b5ac0 2022-06-29T15:50:26.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.283+0000 7f79ff7fe700 1 -- 172.21.15.47:0/102599893 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a00011ba0 con 0x7f7a100b5ac0 2022-06-29T15:50:26.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.283+0000 7f7a0f59e700 1 -- 172.21.15.47:0/102599893 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a100aae00 con 0x7f7a100b5ac0 2022-06-29T15:50:26.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.283+0000 7f7a0f59e700 1 -- 172.21.15.47:0/102599893 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a100573d0 con 0x7f7a100b5ac0 2022-06-29T15:50:26.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.283+0000 7f79ff7fe700 1 -- 172.21.15.47:0/102599893 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7a0001a040 con 0x7f7a100b5ac0 2022-06-29T15:50:26.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.283+0000 7f79ff7fe700 1 --2- 172.21.15.47:0/102599893 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f79ec053950 0x7f79ec055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:26.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.283+0000 7f79ff7fe700 1 -- 172.21.15.47:0/102599893 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f7a0006cfe0 con 0x7f7a100b5ac0 2022-06-29T15:50:26.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.283+0000 7f7a0e59c700 1 --2- 172.21.15.47:0/102599893 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f79ec053950 0x7f79ec055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:26.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.287+0000 7f7a0e59c700 1 --2- 172.21.15.47:0/102599893 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f79ec053950 0x7f79ec055e10 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7f7a04006000 tx=0x7f7a04006d90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:26.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.295+0000 7f79ff7fe700 1 -- 172.21.15.47:0/102599893 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7a00043c70 con 0x7f7a100b5ac0 2022-06-29T15:50:26.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.459+0000 7f7a0f59e700 1 -- 172.21.15.47:0/102599893 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7a100b0fe0 con 0x7f7a100b5ac0 2022-06-29T15:50:26.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.459+0000 7f79ff7fe700 1 -- 172.21.15.47:0/102599893 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+2496 (secure 0 0 0) 0x7f7a00023090 con 0x7f7a100b5ac0 2022-06-29T15:50:26.466 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:50:26.466 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":51,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3752554010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3752554010},{"type":"v1","addr":"172.21.15.47:6835","nonce":3752554010}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":48},{"gid":7293,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.47:6837/3263659716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":3263659716},{"type":"v1","addr":"172.21.15.47:6837","nonce":3263659716}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":49}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:50:15.338114+0000","modified":"2022-06-29T15:50:16.342008+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":56354},"failed":[],"damaged":[],"stopped":[],"info":{"gid_56354":{"gid":56354,"name":"b","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6833/2706285595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2706285595},{"type":"v1","addr":"172.21.15.64:6833","nonce":2706285595}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2022-06-29T15:50:26.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.463+0000 7f7a0f59e700 1 -- 172.21.15.47:0/102599893 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f79ec053950 msgr2=0x7f79ec055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:26.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.463+0000 7f7a0f59e700 1 --2- 172.21.15.47:0/102599893 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f79ec053950 0x7f79ec055e10 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7f7a04006000 tx=0x7f7a04006d90).stop 2022-06-29T15:50:26.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.463+0000 7f7a0f59e700 1 -- 172.21.15.47:0/102599893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a100b5ac0 msgr2=0x7f7a10146450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:26.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.463+0000 7f7a0f59e700 1 --2- 172.21.15.47:0/102599893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a100b5ac0 0x7f7a10146450 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 rx=0x7f7a00002f70 tx=0x7f7a00015000).stop 2022-06-29T15:50:26.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.463+0000 7f7a0f59e700 1 -- 172.21.15.47:0/102599893 shutdown_connections 2022-06-29T15:50:26.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.463+0000 7f7a0f59e700 1 --2- 172.21.15.47:0/102599893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a100b5ac0 0x7f7a10146450 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:26.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.463+0000 7f7a0f59e700 1 --2- 172.21.15.47:0/102599893 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f79ec053950 0x7f79ec055e10 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:26.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.463+0000 7f7a0f59e700 1 --2- 172.21.15.47:0/102599893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a100b02d0 0x7f7a10145f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:26.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.463+0000 7f7a0f59e700 1 --2- 172.21.15.47:0/102599893 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7a100afdc0 0x7f7a100aa380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:26.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.463+0000 7f7a0f59e700 1 -- 172.21.15.47:0/102599893 >> 172.21.15.47:0/102599893 conn(0x7f7a10062560 msgr2=0x7f7a100b0700 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:26.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.463+0000 7f7a0f59e700 1 -- 172.21.15.47:0/102599893 shutdown_connections 2022-06-29T15:50:26.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.463+0000 7f7a0f59e700 1 -- 172.21.15.47:0/102599893 wait complete. 2022-06-29T15:50:26.472 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 51 2022-06-29T15:50:26.487 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-06-29T15:50:26.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.591+0000 7f115a582700 1 Processor -- start 2022-06-29T15:50:26.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.591+0000 7f115a582700 1 -- start start 2022-06-29T15:50:26.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.591+0000 7f115a582700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f11540b5c60 0x7f1154144980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:26.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.591+0000 7f115a582700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11540a9120 con 0x7f1154060eb0 2022-06-29T15:50:26.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.591+0000 7f115a582700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11540b0b80 con 0x7f11540a72f0 2022-06-29T15:50:26.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.591+0000 7f115a582700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11540a89e0 con 0x7f11540b5c60 2022-06-29T15:50:26.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.591+0000 7f1158d7f700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1154060eb0 0x7f11540b1e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37970/0 (socket says 172.21.15.47:37970) 2022-06-29T15:50:26.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.591+0000 7f1158d7f700 1 -- 172.21.15.47:0/2877205192 learned_addr learned my addr 172.21.15.47:0/2877205192 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:26.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.591+0000 7f1159580700 1 --2- 172.21.15.47:0/2877205192 >> v2:172.21.15.64:3300/0 conn(0x7f11540b5c60 0x7f1154144980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:26.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.591+0000 7f114bfff700 1 -- 172.21.15.47:0/2877205192 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3622558762 0 0) 0x7f11540a9120 con 0x7f1154060eb0 2022-06-29T15:50:26.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.591+0000 7f114bfff700 1 -- 172.21.15.47:0/2877205192 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1138003260 con 0x7f1154060eb0 2022-06-29T15:50:26.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.591+0000 7f114bfff700 1 -- 172.21.15.47:0/2877205192 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 813300744 0 0) 0x7f11540b0b80 con 0x7f11540a72f0 2022-06-29T15:50:26.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.591+0000 7f114bfff700 1 -- 172.21.15.47:0/2877205192 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11540a9120 con 0x7f11540a72f0 2022-06-29T15:50:26.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.591+0000 7f1159580700 1 -- 172.21.15.47:0/2877205192 >> v1:172.21.15.64:6789/0 conn(0x7f11540a72f0 legacy=0x7f1154151eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:26.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.591+0000 7f1159580700 1 -- 172.21.15.47:0/2877205192 >> v1:172.21.15.47:6789/0 conn(0x7f1154060eb0 legacy=0x7f11540b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:26.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.591+0000 7f1159580700 1 -- 172.21.15.47:0/2877205192 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11540a8c80 con 0x7f11540b5c60 2022-06-29T15:50:26.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.591+0000 7f1159580700 1 --2- 172.21.15.47:0/2877205192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11540b5c60 0x7f1154144980 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x7f11500032f0 tx=0x7f11500091a0).ready entity=mon.1 client_cookie=cfb20fdb27990bd6 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:26.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.591+0000 7f114bfff700 1 -- 172.21.15.47:0/2877205192 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f115000b600 con 0x7f11540b5c60 2022-06-29T15:50:26.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.591+0000 7f114bfff700 1 -- 172.21.15.47:0/2877205192 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f115000b760 con 0x7f11540b5c60 2022-06-29T15:50:26.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.591+0000 7f114bfff700 1 -- 172.21.15.47:0/2877205192 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f115000b8d0 con 0x7f11540b5c60 2022-06-29T15:50:26.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.595+0000 7f115a582700 1 -- 172.21.15.47:0/2877205192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11540b5c60 msgr2=0x7f1154144980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:26.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.595+0000 7f115a582700 1 --2- 172.21.15.47:0/2877205192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11540b5c60 0x7f1154144980 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x7f11500032f0 tx=0x7f11500091a0).stop 2022-06-29T15:50:26.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.595+0000 7f115a582700 1 -- 172.21.15.47:0/2877205192 shutdown_connections 2022-06-29T15:50:26.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.595+0000 7f115a582700 1 --2- 172.21.15.47:0/2877205192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11540b5c60 0x7f1154144980 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:26.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.595+0000 7f115a582700 1 -- 172.21.15.47:0/2877205192 >> 172.21.15.47:0/2877205192 conn(0x7f1154062560 msgr2=0x7f1154062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:26.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.595+0000 7f115a582700 1 -- 172.21.15.47:0/2877205192 shutdown_connections 2022-06-29T15:50:26.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.595+0000 7f115a582700 1 -- 172.21.15.47:0/2877205192 wait complete. 2022-06-29T15:50:26.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.595+0000 7f115a582700 1 Processor -- start 2022-06-29T15:50:26.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.595+0000 7f115a582700 1 -- start start 2022-06-29T15:50:26.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.599+0000 7f115a582700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1154060eb0 0x7f11540b2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:26.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.599+0000 7f115a582700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11540a72f0 0x7f11540a9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:26.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.599+0000 7f115a582700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f11540b5c60 0x7f115414a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:26.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.599+0000 7f115a582700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115413d240 con 0x7f1154060eb0 2022-06-29T15:50:26.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.599+0000 7f115a582700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11540a94e0 con 0x7f11540a72f0 2022-06-29T15:50:26.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.599+0000 7f115a582700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1154148a70 con 0x7f11540b5c60 2022-06-29T15:50:26.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.599+0000 7f1159580700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1154060eb0 0x7f11540b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:26.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.599+0000 7f1159580700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1154060eb0 0x7f11540b2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52420/0 (socket says 172.21.15.47:52420) 2022-06-29T15:50:26.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.599+0000 7f1159580700 1 -- 172.21.15.47:0/1741016319 learned_addr learned my addr 172.21.15.47:0/1741016319 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:26.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.599+0000 7f1159d81700 1 --2- 172.21.15.47:0/1741016319 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f11540b5c60 0x7f115414a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:26.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.599+0000 7f1158d7f700 1 --2- 172.21.15.47:0/1741016319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11540a72f0 0x7f11540a9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:26.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.599+0000 7f1159580700 1 -- 172.21.15.47:0/1741016319 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f11540b5c60 msgr2=0x7f115414a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:26.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.599+0000 7f1159580700 1 --2- 172.21.15.47:0/1741016319 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f11540b5c60 0x7f115414a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:26.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.599+0000 7f1159580700 1 -- 172.21.15.47:0/1741016319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11540a72f0 msgr2=0x7f11540a9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:26.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.599+0000 7f1159580700 1 --2- 172.21.15.47:0/1741016319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11540a72f0 0x7f11540a9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:26.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.599+0000 7f1159580700 1 -- 172.21.15.47:0/1741016319 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1150009420 con 0x7f1154060eb0 2022-06-29T15:50:26.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.599+0000 7f1159d81700 1 --2- 172.21.15.47:0/1741016319 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f11540b5c60 0x7f115414a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:50:26.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.599+0000 7f1158d7f700 1 --2- 172.21.15.47:0/1741016319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11540a72f0 0x7f11540a9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:50:26.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.599+0000 7f1159580700 1 --2- 172.21.15.47:0/1741016319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1154060eb0 0x7f11540b2680 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 rx=0x7f11500032f0 tx=0x7f1150016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:26.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.599+0000 7f114a7fc700 1 -- 172.21.15.47:0/1741016319 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11500118d0 con 0x7f1154060eb0 2022-06-29T15:50:26.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.599+0000 7f114a7fc700 1 -- 172.21.15.47:0/1741016319 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1150011a30 con 0x7f1154060eb0 2022-06-29T15:50:26.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.599+0000 7f115a582700 1 -- 172.21.15.47:0/1741016319 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1154148d10 con 0x7f1154060eb0 2022-06-29T15:50:26.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.599+0000 7f114a7fc700 1 -- 172.21.15.47:0/1741016319 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1150011d90 con 0x7f1154060eb0 2022-06-29T15:50:26.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.599+0000 7f115a582700 1 -- 172.21.15.47:0/1741016319 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11540aabd0 con 0x7f1154060eb0 2022-06-29T15:50:26.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.603+0000 7f115a582700 1 -- 172.21.15.47:0/1741016319 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11540573d0 con 0x7f1154060eb0 2022-06-29T15:50:26.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.603+0000 7f114a7fc700 1 -- 172.21.15.47:0/1741016319 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1150027020 con 0x7f1154060eb0 2022-06-29T15:50:26.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.603+0000 7f114a7fc700 1 --2- 172.21.15.47:0/1741016319 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f113c0539e0 0x7f113c055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:26.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.603+0000 7f114a7fc700 1 -- 172.21.15.47:0/1741016319 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f115006d280 con 0x7f1154060eb0 2022-06-29T15:50:26.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.603+0000 7f1158d7f700 1 --2- 172.21.15.47:0/1741016319 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f113c0539e0 0x7f113c055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:26.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.603+0000 7f1158d7f700 1 --2- 172.21.15.47:0/1741016319 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f113c0539e0 0x7f113c055ea0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7f114400b000 tx=0x7f1144009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:26.622 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.607+0000 7f114a7fc700 1 -- 172.21.15.47:0/1741016319 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1150044040 con 0x7f1154060eb0 2022-06-29T15:50:26.781 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:26.775+0000 7f115a582700 1 -- 172.21.15.47:0/1741016319 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f11541450b0 con 0x7f1154060eb0 2022-06-29T15:50:27.413 INFO:tasks.ceph.mon.a.smithi047.stderr:2022-06-29T15:50:27.407+0000 7ff9cf83c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-06-29T15:50:27.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.415+0000 7f114a7fc700 1 -- 172.21.15.47:0/1741016319 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v52) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1150047660 con 0x7f1154060eb0 2022-06-29T15:50:27.424 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.415+0000 7f115a582700 1 -- 172.21.15.47:0/1741016319 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f113c0539e0 msgr2=0x7f113c055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:27.424 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.415+0000 7f115a582700 1 --2- 172.21.15.47:0/1741016319 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f113c0539e0 0x7f113c055ea0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7f114400b000 tx=0x7f1144009040).stop 2022-06-29T15:50:27.424 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.415+0000 7f115a582700 1 -- 172.21.15.47:0/1741016319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1154060eb0 msgr2=0x7f11540b2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:27.425 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.415+0000 7f115a582700 1 --2- 172.21.15.47:0/1741016319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1154060eb0 0x7f11540b2680 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 rx=0x7f11500032f0 tx=0x7f1150016000).stop 2022-06-29T15:50:27.425 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.419+0000 7f115a582700 1 -- 172.21.15.47:0/1741016319 shutdown_connections 2022-06-29T15:50:27.425 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.419+0000 7f115a582700 1 --2- 172.21.15.47:0/1741016319 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f11540b5c60 0x7f115414a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:27.425 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.419+0000 7f115a582700 1 --2- 172.21.15.47:0/1741016319 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f113c0539e0 0x7f113c055ea0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:27.426 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.419+0000 7f115a582700 1 --2- 172.21.15.47:0/1741016319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11540a72f0 0x7f11540a9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:27.426 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.419+0000 7f115a582700 1 --2- 172.21.15.47:0/1741016319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1154060eb0 0x7f11540b2680 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:27.426 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.419+0000 7f115a582700 1 -- 172.21.15.47:0/1741016319 >> 172.21.15.47:0/1741016319 conn(0x7f1154062560 msgr2=0x7f11540b1020 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:27.427 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.419+0000 7f115a582700 1 -- 172.21.15.47:0/1741016319 shutdown_connections 2022-06-29T15:50:27.427 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.419+0000 7f115a582700 1 -- 172.21.15.47:0/1741016319 wait complete. 2022-06-29T15:50:27.427 INFO:teuthology.orchestra.run.smithi047.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-06-29T15:50:27.440 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:50:27.447 INFO:tasks.ceph.mds.b.smithi064.stdout:starting mds.b at 2022-06-29T15:50:27.562 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.543+0000 7fcea9db9700 1 Processor -- start 2022-06-29T15:50:27.562 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.543+0000 7fcea9db9700 1 -- start start 2022-06-29T15:50:27.563 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.547+0000 7fcea9db9700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcea40b7840 0x7fcea40b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:27.563 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.547+0000 7fcea9db9700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcea4066c00 0x7fcea4141760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:27.564 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.547+0000 7fcea9db9700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcea40b7c20 con 0x7fcea4062dd0 2022-06-29T15:50:27.564 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.547+0000 7fcea9db9700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcea40b3d10 con 0x7fcea40b7840 2022-06-29T15:50:27.565 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.547+0000 7fcea9db9700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcea4067380 con 0x7fcea4066c00 2022-06-29T15:50:27.565 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.547+0000 7fcea8db7700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcea40b7840 0x7fcea40b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:27.566 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.547+0000 7fcea95b8700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcea4066c00 0x7fcea4141760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:27.566 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.547+0000 7fcea8db7700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcea40b7840 0x7fcea40b3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52430/0 (socket says 172.21.15.47:52430) 2022-06-29T15:50:27.567 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.547+0000 7fcea8db7700 1 -- 172.21.15.47:0/3247323179 learned_addr learned my addr 172.21.15.47:0/3247323179 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:27.567 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.547+0000 7fcea8db7700 1 -- 172.21.15.47:0/3247323179 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcea4066c00 msgr2=0x7fcea4141760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:27.568 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.547+0000 7fcea8db7700 1 --2- 172.21.15.47:0/3247323179 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcea4066c00 0x7fcea4141760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:27.568 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.547+0000 7fcea8db7700 1 -- 172.21.15.47:0/3247323179 >> v1:172.21.15.64:6789/0 conn(0x7fcea4062dd0 legacy=0x7fcea40aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:27.568 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.547+0000 7fcea37fe700 1 -- 172.21.15.47:0/3247323179 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3013210514 0 0) 0x7fcea40b7c20 con 0x7fcea4062dd0 2022-06-29T15:50:27.569 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.547+0000 7fcea8db7700 1 -- 172.21.15.47:0/3247323179 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea40b30e0 con 0x7fcea40b7840 2022-06-29T15:50:27.569 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.547+0000 7fcea95b8700 1 --2- 172.21.15.47:0/3247323179 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcea4066c00 0x7fcea4141760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:50:27.569 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.547+0000 7fcea8db7700 1 --2- 172.21.15.47:0/3247323179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcea40b7840 0x7fcea40b3820 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 rx=0x7fce8c002f70 tx=0x7fce8c006eb0).ready entity=mon.0 client_cookie=36a9d529352c95de server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:27.570 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.547+0000 7fcea37fe700 1 -- 172.21.15.47:0/3247323179 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce8c00d570 con 0x7fcea40b7840 2022-06-29T15:50:27.570 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.547+0000 7fcea37fe700 1 -- 172.21.15.47:0/3247323179 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce8c000e90 con 0x7fcea40b7840 2022-06-29T15:50:27.570 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.547+0000 7fcea37fe700 1 -- 172.21.15.47:0/3247323179 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce8c00d8c0 con 0x7fcea40b7840 2022-06-29T15:50:27.571 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.547+0000 7fcea9db9700 1 -- 172.21.15.47:0/3247323179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcea40b7840 msgr2=0x7fcea40b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:27.571 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.547+0000 7fcea9db9700 1 --2- 172.21.15.47:0/3247323179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcea40b7840 0x7fcea40b3820 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 rx=0x7fce8c002f70 tx=0x7fce8c006eb0).stop 2022-06-29T15:50:27.572 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.547+0000 7fcea9db9700 1 -- 172.21.15.47:0/3247323179 shutdown_connections 2022-06-29T15:50:27.572 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.547+0000 7fcea9db9700 1 --2- 172.21.15.47:0/3247323179 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcea4066c00 0x7fcea4141760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:27.573 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.547+0000 7fcea9db9700 1 --2- 172.21.15.47:0/3247323179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcea40b7840 0x7fcea40b3820 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:27.573 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.547+0000 7fcea9db9700 1 -- 172.21.15.47:0/3247323179 >> 172.21.15.47:0/3247323179 conn(0x7fcea405f480 msgr2=0x7fcea40647d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:27.573 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.547+0000 7fcea9db9700 1 -- 172.21.15.47:0/3247323179 shutdown_connections 2022-06-29T15:50:27.574 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.547+0000 7fcea9db9700 1 -- 172.21.15.47:0/3247323179 wait complete. 2022-06-29T15:50:27.574 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.547+0000 7fcea9db9700 1 Processor -- start 2022-06-29T15:50:27.575 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.551+0000 7fcea9db9700 1 -- start start 2022-06-29T15:50:27.575 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.551+0000 7fcea9db9700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcea4062dd0 0x7fcea4060d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:27.575 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.551+0000 7fcea9db9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcea4066c00 0x7fcea40ad170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:27.576 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.551+0000 7fcea9db9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcea40b7840 0x7fcea40ad660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:27.576 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.551+0000 7fcea9db9700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea41445d0 con 0x7fcea4066c00 2022-06-29T15:50:27.577 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.551+0000 7fcea9db9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea4144aa0 con 0x7fcea40b7840 2022-06-29T15:50:27.577 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.551+0000 7fcea9db9700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcea4061ce0 con 0x7fcea4062dd0 2022-06-29T15:50:27.577 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.551+0000 7fcea3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcea4066c00 0x7fcea40ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:27.577 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.551+0000 7fcea3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcea4066c00 0x7fcea40ad170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52436/0 (socket says 172.21.15.47:52436) 2022-06-29T15:50:27.578 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.551+0000 7fcea3fff700 1 -- 172.21.15.47:0/2923197793 learned_addr learned my addr 172.21.15.47:0/2923197793 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:27.578 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.551+0000 7fcea8db7700 1 --2- 172.21.15.47:0/2923197793 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcea4062dd0 0x7fcea4060d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:27.578 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.551+0000 7fcea95b8700 1 --2- 172.21.15.47:0/2923197793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcea40b7840 0x7fcea40ad660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:27.578 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.551+0000 7fcea3fff700 1 -- 172.21.15.47:0/2923197793 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcea4062dd0 msgr2=0x7fcea4060d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:27.579 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.551+0000 7fcea3fff700 1 --2- 172.21.15.47:0/2923197793 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcea4062dd0 0x7fcea4060d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:27.579 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.551+0000 7fcea3fff700 1 -- 172.21.15.47:0/2923197793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcea40b7840 msgr2=0x7fcea40ad660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:27.579 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.551+0000 7fcea3fff700 1 --2- 172.21.15.47:0/2923197793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcea40b7840 0x7fcea40ad660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:27.579 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.551+0000 7fcea3fff700 1 -- 172.21.15.47:0/2923197793 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce9c009470 con 0x7fcea4066c00 2022-06-29T15:50:27.580 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.551+0000 7fcea95b8700 1 --2- 172.21.15.47:0/2923197793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcea40b7840 0x7fcea40ad660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:50:27.580 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.551+0000 7fcea3fff700 1 --2- 172.21.15.47:0/2923197793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcea4066c00 0x7fcea40ad170 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 rx=0x7fce94000ed0 tx=0x7fce94007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:27.580 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.551+0000 7fcea1ffb700 1 -- 172.21.15.47:0/2923197793 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce94012400 con 0x7fcea4066c00 2022-06-29T15:50:27.580 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.551+0000 7fcea1ffb700 1 -- 172.21.15.47:0/2923197793 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce94012560 con 0x7fcea4066c00 2022-06-29T15:50:27.581 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.551+0000 7fcea9db9700 1 -- 172.21.15.47:0/2923197793 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce8c00b1e0 con 0x7fcea4066c00 2022-06-29T15:50:27.581 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.551+0000 7fcea1ffb700 1 -- 172.21.15.47:0/2923197793 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce940128c0 con 0x7fcea4066c00 2022-06-29T15:50:27.581 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.551+0000 7fcea9db9700 1 -- 172.21.15.47:0/2923197793 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcea41442c0 con 0x7fcea4066c00 2022-06-29T15:50:27.581 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.555+0000 7fcea9db9700 1 -- 172.21.15.47:0/2923197793 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcea40aceb0 con 0x7fcea4066c00 2022-06-29T15:50:27.581 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.555+0000 7fcea1ffb700 1 -- 172.21.15.47:0/2923197793 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fce94019030 con 0x7fcea4066c00 2022-06-29T15:50:27.582 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.555+0000 7fcea1ffb700 1 --2- 172.21.15.47:0/2923197793 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fce88053950 0x7fce88055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:27.582 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.555+0000 7fcea1ffb700 1 -- 172.21.15.47:0/2923197793 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fce9406b370 con 0x7fcea4066c00 2022-06-29T15:50:27.582 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.555+0000 7fcea8db7700 1 --2- 172.21.15.47:0/2923197793 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fce88053950 0x7fce88055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:27.583 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.555+0000 7fcea8db7700 1 --2- 172.21.15.47:0/2923197793 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fce88053950 0x7fce88055e10 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7fce8c01c000 tx=0x7fce8c015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:27.583 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.563+0000 7fcea1ffb700 1 -- 172.21.15.47:0/2923197793 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcea40aceb0 con 0x7fcea4066c00 2022-06-29T15:50:27.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.727+0000 7fcea9db9700 1 -- 172.21.15.47:0/2923197793 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcea40aceb0 con 0x7fcea4066c00 2022-06-29T15:50:27.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.731+0000 7fcea1ffb700 1 -- 172.21.15.47:0/2923197793 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+2133 (secure 0 0 0) 0x7fcea40aceb0 con 0x7fcea4066c00 2022-06-29T15:50:27.737 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:50:27.738 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":52,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3752554010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3752554010},{"type":"v1","addr":"172.21.15.47:6835","nonce":3752554010}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":48},{"gid":7293,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.47:6837/3263659716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":3263659716},{"type":"v1","addr":"172.21.15.47:6837","nonce":3263659716}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":49}],"filesystems":[{"mdsmap":{"epoch":52,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:50:15.338114+0000","modified":"2022-06-29T15:50:27.409827+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2022-06-29T15:50:27.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.731+0000 7fcea9db9700 1 -- 172.21.15.47:0/2923197793 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fce88053950 msgr2=0x7fce88055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:27.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.731+0000 7fcea9db9700 1 --2- 172.21.15.47:0/2923197793 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fce88053950 0x7fce88055e10 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7fce8c01c000 tx=0x7fce8c015000).stop 2022-06-29T15:50:27.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.731+0000 7fcea9db9700 1 -- 172.21.15.47:0/2923197793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcea4066c00 msgr2=0x7fcea40ad170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:27.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.731+0000 7fcea9db9700 1 --2- 172.21.15.47:0/2923197793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcea4066c00 0x7fcea40ad170 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 rx=0x7fce94000ed0 tx=0x7fce94007ac0).stop 2022-06-29T15:50:27.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.731+0000 7fcea9db9700 1 -- 172.21.15.47:0/2923197793 shutdown_connections 2022-06-29T15:50:27.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.731+0000 7fcea9db9700 1 --2- 172.21.15.47:0/2923197793 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fce88053950 0x7fce88055e10 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:27.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.731+0000 7fcea9db9700 1 --2- 172.21.15.47:0/2923197793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcea40b7840 0x7fcea40ad660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:27.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.731+0000 7fcea9db9700 1 --2- 172.21.15.47:0/2923197793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcea4066c00 0x7fcea40ad170 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:27.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.731+0000 7fcea9db9700 1 --2- 172.21.15.47:0/2923197793 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcea4062dd0 0x7fcea4060d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:27.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.731+0000 7fcea9db9700 1 -- 172.21.15.47:0/2923197793 >> 172.21.15.47:0/2923197793 conn(0x7fcea405f480 msgr2=0x7fcea40b01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:27.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.731+0000 7fcea9db9700 1 -- 172.21.15.47:0/2923197793 shutdown_connections 2022-06-29T15:50:27.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.731+0000 7fcea9db9700 1 -- 172.21.15.47:0/2923197793 wait complete. 2022-06-29T15:50:27.743 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 52 2022-06-29T15:50:27.750 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-06-29T15:50:27.862 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.855+0000 7f2bedd2b700 1 Processor -- start 2022-06-29T15:50:27.862 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.855+0000 7f2bedd2b700 1 -- start start 2022-06-29T15:50:27.864 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.855+0000 7f2bedd2b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2be80608d0 0x7f2be81403b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:27.864 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.855+0000 7f2bedd2b700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2be81472a0 con 0x7f2be80b2ad0 2022-06-29T15:50:27.864 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.855+0000 7f2bedd2b700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2be8147540 con 0x7f2be80a94f0 2022-06-29T15:50:27.865 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.855+0000 7f2bedd2b700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2be80b01b0 con 0x7f2be80608d0 2022-06-29T15:50:27.865 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.855+0000 7f2be7fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f2be80b2ad0 0x7f2be8143220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37996/0 (socket says 172.21.15.47:37996) 2022-06-29T15:50:27.865 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.855+0000 7f2be7fff700 1 -- 172.21.15.47:0/3486213806 learned_addr learned my addr 172.21.15.47:0/3486213806 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:27.865 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.855+0000 7f2be77fe700 1 --2- 172.21.15.47:0/3486213806 >> v2:172.21.15.64:3300/0 conn(0x7f2be80608d0 0x7f2be81403b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:27.866 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.859+0000 7f2be67fc700 1 -- 172.21.15.47:0/3486213806 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1913084059 0 0) 0x7f2be81472a0 con 0x7f2be80b2ad0 2022-06-29T15:50:27.866 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.859+0000 7f2be67fc700 1 -- 172.21.15.47:0/3486213806 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bd4003260 con 0x7f2be80b2ad0 2022-06-29T15:50:27.866 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.859+0000 7f2be67fc700 1 -- 172.21.15.47:0/3486213806 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1578858885 0 0) 0x7f2be8147540 con 0x7f2be80a94f0 2022-06-29T15:50:27.866 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.859+0000 7f2be67fc700 1 -- 172.21.15.47:0/3486213806 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2be81472a0 con 0x7f2be80a94f0 2022-06-29T15:50:27.867 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.859+0000 7f2be77fe700 1 -- 172.21.15.47:0/3486213806 >> v1:172.21.15.64:6789/0 conn(0x7f2be80a94f0 legacy=0x7f2be81408a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:27.867 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.859+0000 7f2be77fe700 1 -- 172.21.15.47:0/3486213806 >> v1:172.21.15.47:6789/0 conn(0x7f2be80b2ad0 legacy=0x7f2be8143220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:27.867 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.859+0000 7f2be77fe700 1 -- 172.21.15.47:0/3486213806 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be80a8d80 con 0x7f2be80608d0 2022-06-29T15:50:27.871 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.859+0000 7f2be77fe700 1 --2- 172.21.15.47:0/3486213806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2be80608d0 0x7f2be81403b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7f2bd8002f70 tx=0x7f2bd8006df0).ready entity=mon.1 client_cookie=3d0045f945e3a87e server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:27.871 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.859+0000 7f2be67fc700 1 -- 172.21.15.47:0/3486213806 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2bd800d540 con 0x7f2be80608d0 2022-06-29T15:50:27.872 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.859+0000 7f2be67fc700 1 -- 172.21.15.47:0/3486213806 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bd8000e90 con 0x7f2be80608d0 2022-06-29T15:50:27.872 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.859+0000 7f2be67fc700 1 -- 172.21.15.47:0/3486213806 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2bd800d890 con 0x7f2be80608d0 2022-06-29T15:50:27.872 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.859+0000 7f2bedd2b700 1 -- 172.21.15.47:0/3486213806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2be80608d0 msgr2=0x7f2be81403b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:27.872 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.859+0000 7f2bedd2b700 1 --2- 172.21.15.47:0/3486213806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2be80608d0 0x7f2be81403b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7f2bd8002f70 tx=0x7f2bd8006df0).stop 2022-06-29T15:50:27.873 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.859+0000 7f2bedd2b700 1 -- 172.21.15.47:0/3486213806 shutdown_connections 2022-06-29T15:50:27.873 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.859+0000 7f2bedd2b700 1 --2- 172.21.15.47:0/3486213806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2be80608d0 0x7f2be81403b0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:27.873 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.859+0000 7f2bedd2b700 1 -- 172.21.15.47:0/3486213806 >> 172.21.15.47:0/3486213806 conn(0x7f2be80626f0 msgr2=0x7f2be8062ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:27.873 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.859+0000 7f2bedd2b700 1 -- 172.21.15.47:0/3486213806 shutdown_connections 2022-06-29T15:50:27.874 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.859+0000 7f2bedd2b700 1 -- 172.21.15.47:0/3486213806 wait complete. 2022-06-29T15:50:27.874 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.859+0000 7f2bedd2b700 1 Processor -- start 2022-06-29T15:50:27.874 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.863+0000 7f2bedd2b700 1 -- start start 2022-06-29T15:50:27.874 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.863+0000 7f2bedd2b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2be80608d0 0x7f2be80b3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:27.875 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.863+0000 7f2bedd2b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2be80a94f0 0x7f2be80b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:27.875 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.863+0000 7f2bedd2b700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2be80b2ad0 0x7f2be80b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:27.875 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.863+0000 7f2bedd2b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be80b3a80 con 0x7f2be80608d0 2022-06-29T15:50:27.875 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.863+0000 7f2bedd2b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be8145710 con 0x7f2be80a94f0 2022-06-29T15:50:27.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.863+0000 7f2bedd2b700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2be8144760 con 0x7f2be80b2ad0 2022-06-29T15:50:27.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.863+0000 7f2be77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2be80608d0 0x7f2be80b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:27.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.863+0000 7f2be77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2be80608d0 0x7f2be80b3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52448/0 (socket says 172.21.15.47:52448) 2022-06-29T15:50:27.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.863+0000 7f2be77fe700 1 -- 172.21.15.47:0/3938841081 learned_addr learned my addr 172.21.15.47:0/3938841081 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:27.877 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.863+0000 7f2be7fff700 1 --2- 172.21.15.47:0/3938841081 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2be80b2ad0 0x7f2be80b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:27.877 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.863+0000 7f2be6ffd700 1 --2- 172.21.15.47:0/3938841081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2be80a94f0 0x7f2be80b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:27.877 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.863+0000 7f2be77fe700 1 -- 172.21.15.47:0/3938841081 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2be80b2ad0 msgr2=0x7f2be80b50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:27.877 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.863+0000 7f2be77fe700 1 --2- 172.21.15.47:0/3938841081 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2be80b2ad0 0x7f2be80b50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:27.877 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.863+0000 7f2be77fe700 1 -- 172.21.15.47:0/3938841081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2be80a94f0 msgr2=0x7f2be80b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:27.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.863+0000 7f2be77fe700 1 --2- 172.21.15.47:0/3938841081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2be80a94f0 0x7f2be80b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:27.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.863+0000 7f2be77fe700 1 -- 172.21.15.47:0/3938841081 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd800b180 con 0x7f2be80608d0 2022-06-29T15:50:27.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.863+0000 7f2be77fe700 1 --2- 172.21.15.47:0/3938841081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2be80608d0 0x7f2be80b3470 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 rx=0x7f2be8145ce0 tx=0x7f2bd8014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:27.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.863+0000 7f2be4ff9700 1 -- 172.21.15.47:0/3938841081 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2bd8016070 con 0x7f2be80608d0 2022-06-29T15:50:27.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.863+0000 7f2be4ff9700 1 -- 172.21.15.47:0/3938841081 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bd8013850 con 0x7f2be80608d0 2022-06-29T15:50:27.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.863+0000 7f2bedd2b700 1 -- 172.21.15.47:0/3938841081 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2be8145dc0 con 0x7f2be80608d0 2022-06-29T15:50:27.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.863+0000 7f2be4ff9700 1 -- 172.21.15.47:0/3938841081 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2bd8013bb0 con 0x7f2be80608d0 2022-06-29T15:50:27.880 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.863+0000 7f2bedd2b700 1 -- 172.21.15.47:0/3938841081 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2be8145070 con 0x7f2be80608d0 2022-06-29T15:50:27.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.867+0000 7f2bedd2b700 1 -- 172.21.15.47:0/3938841081 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2be81448a0 con 0x7f2be80608d0 2022-06-29T15:50:27.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.867+0000 7f2be4ff9700 1 -- 172.21.15.47:0/3938841081 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2bd8025030 con 0x7f2be80608d0 2022-06-29T15:50:27.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.867+0000 7f2be4ff9700 1 --2- 172.21.15.47:0/3938841081 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2bc80539e0 0x7f2bc8055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:27.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.867+0000 7f2be4ff9700 1 -- 172.21.15.47:0/3938841081 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f2bd806ce50 con 0x7f2be80608d0 2022-06-29T15:50:27.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.867+0000 7f2be6ffd700 1 --2- 172.21.15.47:0/3938841081 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2bc80539e0 0x7f2bc8055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:27.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.867+0000 7f2be6ffd700 1 --2- 172.21.15.47:0/3938841081 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2bc80539e0 0x7f2bc8055ea0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7f2bdc00d000 tx=0x7f2bdc007a00).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:27.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:27.871+0000 7f2be4ff9700 1 -- 172.21.15.47:0/3938841081 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2bd8043bc0 con 0x7f2be80608d0 2022-06-29T15:50:28.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.039+0000 7f2bedd2b700 1 -- 172.21.15.47:0/3938841081 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2be80573d0 con 0x7f2be80608d0 2022-06-29T15:50:28.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.431+0000 7f2be4ff9700 1 -- 172.21.15.47:0/3938841081 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v53) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2bd80230a0 con 0x7f2be80608d0 2022-06-29T15:50:28.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.435+0000 7f2bedd2b700 1 -- 172.21.15.47:0/3938841081 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2bc80539e0 msgr2=0x7f2bc8055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:28.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.435+0000 7f2bedd2b700 1 --2- 172.21.15.47:0/3938841081 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2bc80539e0 0x7f2bc8055ea0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7f2bdc00d000 tx=0x7f2bdc007a00).stop 2022-06-29T15:50:28.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.435+0000 7f2bedd2b700 1 -- 172.21.15.47:0/3938841081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2be80608d0 msgr2=0x7f2be80b3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:28.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.435+0000 7f2bedd2b700 1 --2- 172.21.15.47:0/3938841081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2be80608d0 0x7f2be80b3470 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 rx=0x7f2be8145ce0 tx=0x7f2bd8014000).stop 2022-06-29T15:50:28.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.435+0000 7f2bedd2b700 1 -- 172.21.15.47:0/3938841081 shutdown_connections 2022-06-29T15:50:28.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.435+0000 7f2bedd2b700 1 --2- 172.21.15.47:0/3938841081 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2be80b2ad0 0x7f2be80b50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:28.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.435+0000 7f2bedd2b700 1 --2- 172.21.15.47:0/3938841081 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2bc80539e0 0x7f2bc8055ea0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:28.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.435+0000 7f2bedd2b700 1 --2- 172.21.15.47:0/3938841081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2be80a94f0 0x7f2be80b4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:28.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.435+0000 7f2bedd2b700 1 --2- 172.21.15.47:0/3938841081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2be80608d0 0x7f2be80b3470 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:28.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.435+0000 7f2bedd2b700 1 -- 172.21.15.47:0/3938841081 >> 172.21.15.47:0/3938841081 conn(0x7f2be80626f0 msgr2=0x7f2be80b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:28.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.435+0000 7f2bedd2b700 1 -- 172.21.15.47:0/3938841081 shutdown_connections 2022-06-29T15:50:28.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.435+0000 7f2bedd2b700 1 -- 172.21.15.47:0/3938841081 wait complete. 2022-06-29T15:50:28.456 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-06-29T15:50:28.573 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.567+0000 7f5128b4d700 1 Processor -- start 2022-06-29T15:50:28.574 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.567+0000 7f5128b4d700 1 -- start start 2022-06-29T15:50:28.576 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.567+0000 7f5128b4d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f51240b4300 0x7f51240612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:28.577 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.567+0000 7f5128b4d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f51241471a0 0x7f5124066f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:28.577 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.567+0000 7f5128b4d700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5124067610 con 0x7f5124060ee0 2022-06-29T15:50:28.577 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.567+0000 7f5128b4d700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51241475d0 con 0x7f51241471a0 2022-06-29T15:50:28.578 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.567+0000 7f5128b4d700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51241467c0 con 0x7f51240b4300 2022-06-29T15:50:28.578 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.567+0000 7f5122d9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f51241471a0 0x7f5124066f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:28.578 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.567+0000 7f5122d9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f51241471a0 0x7f5124066f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52458/0 (socket says 172.21.15.47:52458) 2022-06-29T15:50:28.578 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.567+0000 7f5122d9d700 1 -- 172.21.15.47:0/83686390 learned_addr learned my addr 172.21.15.47:0/83686390 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:28.579 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.567+0000 7f5121d9b700 1 --2- 172.21.15.47:0/83686390 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f51240b4300 0x7f51240612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:28.579 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.571+0000 7f512159a700 1 -- 172.21.15.47:0/83686390 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 601973730 0 0) 0x7f5124067610 con 0x7f5124060ee0 2022-06-29T15:50:28.579 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.571+0000 7f512159a700 1 -- 172.21.15.47:0/83686390 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5104003260 con 0x7f5124060ee0 2022-06-29T15:50:28.579 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.571+0000 7f5122d9d700 1 -- 172.21.15.47:0/83686390 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f51240b4300 msgr2=0x7f51240612c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:28.579 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.571+0000 7f5122d9d700 1 --2- 172.21.15.47:0/83686390 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f51240b4300 0x7f51240612c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:28.580 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.571+0000 7f5122d9d700 1 -- 172.21.15.47:0/83686390 >> v1:172.21.15.64:6789/0 conn(0x7f5124060ee0 legacy=0x7f51241430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:28.580 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.571+0000 7f5122d9d700 1 -- 172.21.15.47:0/83686390 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5124146a60 con 0x7f51241471a0 2022-06-29T15:50:28.580 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.571+0000 7f5121d9b700 1 --2- 172.21.15.47:0/83686390 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f51240b4300 0x7f51240612c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:50:28.580 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.571+0000 7f5122d9d700 1 --2- 172.21.15.47:0/83686390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51241471a0 0x7f5124066f70 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 rx=0x7f511c0032f0 tx=0x7f511c009290).ready entity=mon.0 client_cookie=e1d0d27fa217d11f server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:28.581 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.571+0000 7f512159a700 1 -- 172.21.15.47:0/83686390 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f511c00b6c0 con 0x7f51241471a0 2022-06-29T15:50:28.581 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.571+0000 7f512159a700 1 -- 172.21.15.47:0/83686390 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f511c00b820 con 0x7f51241471a0 2022-06-29T15:50:28.581 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.571+0000 7f512159a700 1 -- 172.21.15.47:0/83686390 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f511c00b990 con 0x7f51241471a0 2022-06-29T15:50:28.585 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.571+0000 7f5128b4d700 1 -- 172.21.15.47:0/83686390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51241471a0 msgr2=0x7f5124066f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:28.585 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.571+0000 7f5128b4d700 1 --2- 172.21.15.47:0/83686390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51241471a0 0x7f5124066f70 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 rx=0x7f511c0032f0 tx=0x7f511c009290).stop 2022-06-29T15:50:28.585 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.571+0000 7f5128b4d700 1 -- 172.21.15.47:0/83686390 shutdown_connections 2022-06-29T15:50:28.586 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.571+0000 7f5128b4d700 1 --2- 172.21.15.47:0/83686390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51241471a0 0x7f5124066f70 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:28.586 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.571+0000 7f5128b4d700 1 --2- 172.21.15.47:0/83686390 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f51240b4300 0x7f51240612c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:28.586 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.571+0000 7f5128b4d700 1 -- 172.21.15.47:0/83686390 >> 172.21.15.47:0/83686390 conn(0x7f512405c510 msgr2=0x7f512405f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:28.586 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.571+0000 7f5128b4d700 1 -- 172.21.15.47:0/83686390 shutdown_connections 2022-06-29T15:50:28.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.571+0000 7f5128b4d700 1 -- 172.21.15.47:0/83686390 wait complete. 2022-06-29T15:50:28.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.571+0000 7f5128b4d700 1 Processor -- start 2022-06-29T15:50:28.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.571+0000 7f5128b4d700 1 -- start start 2022-06-29T15:50:28.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.571+0000 7f5128b4d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5124060ee0 0x7f512414a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:28.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.571+0000 7f5128b4d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51240b4300 0x7f51241393b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:28.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.571+0000 7f5128b4d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51241471a0 0x7f51241398a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:28.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.571+0000 7f5128b4d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51241464d0 con 0x7f51241471a0 2022-06-29T15:50:28.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.571+0000 7f5128b4d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512413b8c0 con 0x7f51240b4300 2022-06-29T15:50:28.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.571+0000 7f5128b4d700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f512413bb60 con 0x7f5124060ee0 2022-06-29T15:50:28.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.575+0000 7f5122d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51241471a0 0x7f51241398a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:28.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.575+0000 7f5122d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51241471a0 0x7f51241398a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52466/0 (socket says 172.21.15.47:52466) 2022-06-29T15:50:28.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.575+0000 7f5122d9d700 1 -- 172.21.15.47:0/4131079700 learned_addr learned my addr 172.21.15.47:0/4131079700 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:28.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.575+0000 7f512259c700 1 --2- 172.21.15.47:0/4131079700 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5124060ee0 0x7f512414a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:28.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.575+0000 7f5121d9b700 1 --2- 172.21.15.47:0/4131079700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51240b4300 0x7f51241393b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:28.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.575+0000 7f5122d9d700 1 -- 172.21.15.47:0/4131079700 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5124060ee0 msgr2=0x7f512414a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:28.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.575+0000 7f5122d9d700 1 --2- 172.21.15.47:0/4131079700 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5124060ee0 0x7f512414a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:28.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.575+0000 7f5122d9d700 1 -- 172.21.15.47:0/4131079700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51240b4300 msgr2=0x7f51241393b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:28.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.575+0000 7f5122d9d700 1 --2- 172.21.15.47:0/4131079700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51240b4300 0x7f51241393b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:28.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.575+0000 7f5122d9d700 1 -- 172.21.15.47:0/4131079700 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f510c00b180 con 0x7f51241471a0 2022-06-29T15:50:28.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.575+0000 7f5122d9d700 1 --2- 172.21.15.47:0/4131079700 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51241471a0 0x7f51241398a0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 rx=0x7f511c0032f0 tx=0x7f511c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:28.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.575+0000 7f511b7fe700 1 -- 172.21.15.47:0/4131079700 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f511c00b990 con 0x7f51241471a0 2022-06-29T15:50:28.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.575+0000 7f511b7fe700 1 -- 172.21.15.47:0/4131079700 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f511c021620 con 0x7f51241471a0 2022-06-29T15:50:28.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.575+0000 7f511b7fe700 1 -- 172.21.15.47:0/4131079700 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f511c00b6c0 con 0x7f51241471a0 2022-06-29T15:50:28.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.575+0000 7f5128b4d700 1 -- 172.21.15.47:0/4131079700 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f511c009530 con 0x7f51241471a0 2022-06-29T15:50:28.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.575+0000 7f5128b4d700 1 -- 172.21.15.47:0/4131079700 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5124148b90 con 0x7f51241471a0 2022-06-29T15:50:28.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.575+0000 7f5128b4d700 1 -- 172.21.15.47:0/4131079700 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5124139f30 con 0x7f51241471a0 2022-06-29T15:50:28.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.579+0000 7f511b7fe700 1 -- 172.21.15.47:0/4131079700 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f511c028070 con 0x7f51241471a0 2022-06-29T15:50:28.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.579+0000 7f511b7fe700 1 --2- 172.21.15.47:0/4131079700 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5108053b00 0x7f5108055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:28.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.579+0000 7f511b7fe700 1 -- 172.21.15.47:0/4131079700 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f511c06ce30 con 0x7f51241471a0 2022-06-29T15:50:28.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.579+0000 7f512259c700 1 --2- 172.21.15.47:0/4131079700 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5108053b00 0x7f5108055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:28.594 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.579+0000 7f512259c700 1 --2- 172.21.15.47:0/4131079700 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5108053b00 0x7f5108055fc0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7f511400d000 tx=0x7f5114007a00).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:28.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.587+0000 7f511b7fe700 1 -- 172.21.15.47:0/4131079700 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f511c043a90 con 0x7f51241471a0 2022-06-29T15:50:28.720 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:28.715+0000 7f5128b4d700 1 -- 172.21.15.47:0/4131079700 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f51240573d0 con 0x7f51241471a0 2022-06-29T15:50:29.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.443+0000 7f511b7fe700 1 -- 172.21.15.47:0/4131079700 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v113) v1 ==== 185+0+0 (secure 0 0 0) 0x7f511c015030 con 0x7f51241471a0 2022-06-29T15:50:29.452 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.447+0000 7f5128b4d700 1 -- 172.21.15.47:0/4131079700 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5108053b00 msgr2=0x7f5108055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:29.452 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.447+0000 7f5128b4d700 1 --2- 172.21.15.47:0/4131079700 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5108053b00 0x7f5108055fc0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7f511400d000 tx=0x7f5114007a00).stop 2022-06-29T15:50:29.452 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.447+0000 7f5128b4d700 1 -- 172.21.15.47:0/4131079700 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51241471a0 msgr2=0x7f51241398a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:29.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.447+0000 7f5128b4d700 1 --2- 172.21.15.47:0/4131079700 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51241471a0 0x7f51241398a0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 rx=0x7f511c0032f0 tx=0x7f511c016000).stop 2022-06-29T15:50:29.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.447+0000 7f5128b4d700 1 -- 172.21.15.47:0/4131079700 shutdown_connections 2022-06-29T15:50:29.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.447+0000 7f5128b4d700 1 --2- 172.21.15.47:0/4131079700 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51241471a0 0x7f51241398a0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:29.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.447+0000 7f5128b4d700 1 --2- 172.21.15.47:0/4131079700 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5108053b00 0x7f5108055fc0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:29.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.447+0000 7f5128b4d700 1 --2- 172.21.15.47:0/4131079700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51240b4300 0x7f51241393b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:29.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.447+0000 7f5128b4d700 1 --2- 172.21.15.47:0/4131079700 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5124060ee0 0x7f512414a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:29.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.447+0000 7f5128b4d700 1 -- 172.21.15.47:0/4131079700 >> 172.21.15.47:0/4131079700 conn(0x7f512405c510 msgr2=0x7f51240b7560 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:29.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.447+0000 7f5128b4d700 1 -- 172.21.15.47:0/4131079700 shutdown_connections 2022-06-29T15:50:29.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.447+0000 7f5128b4d700 1 -- 172.21.15.47:0/4131079700 wait complete. 2022-06-29T15:50:29.455 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' removed 2022-06-29T15:50:29.468 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-06-29T15:50:29.584 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.579+0000 7fc2c359e700 1 Processor -- start 2022-06-29T15:50:29.585 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.579+0000 7fc2c359e700 1 -- start start 2022-06-29T15:50:29.586 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.579+0000 7fc2c359e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc2c405f0b0 0x7fc2c405f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:29.586 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.579+0000 7fc2c359e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc2c4144670 0x7fc2c4061740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:29.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.579+0000 7fc2c359e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2c4061e50 con 0x7fc2c4147140 2022-06-29T15:50:29.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.579+0000 7fc2c359e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2c4061c30 con 0x7fc2c405f0b0 2022-06-29T15:50:29.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.579+0000 7fc2c359e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2c40620f0 con 0x7fc2c4144670 2022-06-29T15:50:29.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.579+0000 7fc2c259c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc2c405f0b0 0x7fc2c405f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:29.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.579+0000 7fc2c259c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc2c405f0b0 0x7fc2c405f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52470/0 (socket says 172.21.15.47:52470) 2022-06-29T15:50:29.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.579+0000 7fc2c259c700 1 -- 172.21.15.47:0/3964922701 learned_addr learned my addr 172.21.15.47:0/3964922701 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:29.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.579+0000 7fc2bbfff700 1 --2- 172.21.15.47:0/3964922701 >> v2:172.21.15.64:3300/0 conn(0x7fc2c4144670 0x7fc2c4061740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:29.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.579+0000 7fc2c259c700 1 -- 172.21.15.47:0/3964922701 >> v2:172.21.15.64:3300/0 conn(0x7fc2c4144670 msgr2=0x7fc2c4061740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:29.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.579+0000 7fc2c259c700 1 --2- 172.21.15.47:0/3964922701 >> v2:172.21.15.64:3300/0 conn(0x7fc2c4144670 0x7fc2c4061740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:29.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.579+0000 7fc2c259c700 1 -- 172.21.15.47:0/3964922701 >> v1:172.21.15.64:6789/0 conn(0x7fc2c4147140 legacy=0x7fc2c4143950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:29.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.579+0000 7fc2c259c700 1 -- 172.21.15.47:0/3964922701 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c41434b0 con 0x7fc2c405f0b0 2022-06-29T15:50:29.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.579+0000 7fc2c259c700 1 --2- 172.21.15.47:0/3964922701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2c405f0b0 0x7fc2c405f490 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 rx=0x7fc2b4002f70 tx=0x7fc2b4006eb0).ready entity=mon.0 client_cookie=2263eed31419a891 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:29.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.579+0000 7fc2c1d9b700 1 -- 172.21.15.47:0/3964922701 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc2b400d5c0 con 0x7fc2c405f0b0 2022-06-29T15:50:29.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.583+0000 7fc2c1d9b700 1 -- 172.21.15.47:0/3964922701 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2b4000e90 con 0x7fc2c405f0b0 2022-06-29T15:50:29.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.583+0000 7fc2c1d9b700 1 -- 172.21.15.47:0/3964922701 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc2b400d950 con 0x7fc2c405f0b0 2022-06-29T15:50:29.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.583+0000 7fc2c359e700 1 -- 172.21.15.47:0/3964922701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2c405f0b0 msgr2=0x7fc2c405f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:29.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.583+0000 7fc2c359e700 1 --2- 172.21.15.47:0/3964922701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2c405f0b0 0x7fc2c405f490 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 rx=0x7fc2b4002f70 tx=0x7fc2b4006eb0).stop 2022-06-29T15:50:29.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.583+0000 7fc2c359e700 1 -- 172.21.15.47:0/3964922701 shutdown_connections 2022-06-29T15:50:29.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.583+0000 7fc2c359e700 1 --2- 172.21.15.47:0/3964922701 >> v2:172.21.15.64:3300/0 conn(0x7fc2c4144670 0x7fc2c4061740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:29.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.583+0000 7fc2c359e700 1 --2- 172.21.15.47:0/3964922701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2c405f0b0 0x7fc2c405f490 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:29.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.583+0000 7fc2c359e700 1 -- 172.21.15.47:0/3964922701 >> 172.21.15.47:0/3964922701 conn(0x7fc2c4062570 msgr2=0x7fc2c40647d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:29.594 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.583+0000 7fc2c359e700 1 -- 172.21.15.47:0/3964922701 shutdown_connections 2022-06-29T15:50:29.594 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.583+0000 7fc2c359e700 1 -- 172.21.15.47:0/3964922701 wait complete. 2022-06-29T15:50:29.594 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.583+0000 7fc2c359e700 1 Processor -- start 2022-06-29T15:50:29.594 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.583+0000 7fc2c359e700 1 -- start start 2022-06-29T15:50:29.597 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.587+0000 7fc2c359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2c405f0b0 0x7fc2c4066bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:29.597 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.587+0000 7fc2c359e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2c4144670 0x7fc2c40a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:29.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.587+0000 7fc2c359e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc2c4147140 0x7fc2c40a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:29.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.587+0000 7fc2c359e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c40a39a0 con 0x7fc2c4144670 2022-06-29T15:50:29.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.587+0000 7fc2c359e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c40a3c90 con 0x7fc2c405f0b0 2022-06-29T15:50:29.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.587+0000 7fc2c359e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c40b6b80 con 0x7fc2c4147140 2022-06-29T15:50:29.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.587+0000 7fc2bbfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2c4144670 0x7fc2c40a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:29.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.587+0000 7fc2bbfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2c4144670 0x7fc2c40a4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52478/0 (socket says 172.21.15.47:52478) 2022-06-29T15:50:29.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.587+0000 7fc2c259c700 1 --2- 172.21.15.47:0/3655937022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2c405f0b0 0x7fc2c4066bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:29.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.587+0000 7fc2bbfff700 1 -- 172.21.15.47:0/3655937022 learned_addr learned my addr 172.21.15.47:0/3655937022 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:29.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.587+0000 7fc2c2d9d700 1 --2- 172.21.15.47:0/3655937022 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc2c4147140 0x7fc2c40a4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:29.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.587+0000 7fc2bbfff700 1 -- 172.21.15.47:0/3655937022 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc2c4147140 msgr2=0x7fc2c40a4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:29.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.587+0000 7fc2bbfff700 1 --2- 172.21.15.47:0/3655937022 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc2c4147140 0x7fc2c40a4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:29.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.587+0000 7fc2bbfff700 1 -- 172.21.15.47:0/3655937022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2c405f0b0 msgr2=0x7fc2c4066bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:29.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.587+0000 7fc2bbfff700 1 --2- 172.21.15.47:0/3655937022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2c405f0b0 0x7fc2c4066bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:29.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.587+0000 7fc2bbfff700 1 -- 172.21.15.47:0/3655937022 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b400b1e0 con 0x7fc2c4144670 2022-06-29T15:50:29.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.587+0000 7fc2bbfff700 1 --2- 172.21.15.47:0/3655937022 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2c4144670 0x7fc2c40a4770 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 rx=0x7fc2c40b6cc0 tx=0x7fc2ac006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:29.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.587+0000 7fc2bb7fe700 1 -- 172.21.15.47:0/3655937022 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc2ac010780 con 0x7fc2c4144670 2022-06-29T15:50:29.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.587+0000 7fc2bb7fe700 1 -- 172.21.15.47:0/3655937022 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2ac0108e0 con 0x7fc2c4144670 2022-06-29T15:50:29.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.587+0000 7fc2c359e700 1 -- 172.21.15.47:0/3655937022 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2c40b7620 con 0x7fc2c4144670 2022-06-29T15:50:29.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.587+0000 7fc2bb7fe700 1 -- 172.21.15.47:0/3655937022 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc2ac010c40 con 0x7fc2c4144670 2022-06-29T15:50:29.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.587+0000 7fc2c359e700 1 -- 172.21.15.47:0/3655937022 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2c40b5b60 con 0x7fc2c4144670 2022-06-29T15:50:29.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.587+0000 7fc2c359e700 1 -- 172.21.15.47:0/3655937022 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc29c003410 con 0x7fc2c4144670 2022-06-29T15:50:29.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.591+0000 7fc2bb7fe700 1 -- 172.21.15.47:0/3655937022 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc2ac01c460 con 0x7fc2c4144670 2022-06-29T15:50:29.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.591+0000 7fc2bb7fe700 1 --2- 172.21.15.47:0/3655937022 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc2a8053a70 0x7fc2a8055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:29.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.591+0000 7fc2bb7fe700 1 -- 172.21.15.47:0/3655937022 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6068+0+0 (secure 0 0 0) 0x7fc2ac06c310 con 0x7fc2c4144670 2022-06-29T15:50:29.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.591+0000 7fc2c259c700 1 --2- 172.21.15.47:0/3655937022 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc2a8053a70 0x7fc2a8055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:29.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.591+0000 7fc2c259c700 1 --2- 172.21.15.47:0/3655937022 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc2a8053a70 0x7fc2a8055f30 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7fc2b401c000 tx=0x7fc2b4015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:29.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.595+0000 7fc2bb7fe700 1 -- 172.21.15.47:0/3655937022 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc2ac043040 con 0x7fc2c4144670 2022-06-29T15:50:29.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:29.723+0000 7fc2c359e700 1 -- 172.21.15.47:0/3655937022 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc29c003c40 con 0x7fc2c4144670 2022-06-29T15:50:30.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.455+0000 7fc2bb7fe700 1 -- 172.21.15.47:0/3655937022 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v114) v1 ==== 173+0+0 (secure 0 0 0) 0x7fc2ac046660 con 0x7fc2c4144670 2022-06-29T15:50:30.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.459+0000 7fc2c359e700 1 -- 172.21.15.47:0/3655937022 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc2a8053a70 msgr2=0x7fc2a8055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:30.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.459+0000 7fc2c359e700 1 --2- 172.21.15.47:0/3655937022 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc2a8053a70 0x7fc2a8055f30 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7fc2b401c000 tx=0x7fc2b4015000).stop 2022-06-29T15:50:30.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.459+0000 7fc2c359e700 1 -- 172.21.15.47:0/3655937022 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2c4144670 msgr2=0x7fc2c40a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:30.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.459+0000 7fc2c359e700 1 --2- 172.21.15.47:0/3655937022 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2c4144670 0x7fc2c40a4770 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 rx=0x7fc2c40b6cc0 tx=0x7fc2ac006df0).stop 2022-06-29T15:50:30.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.459+0000 7fc2c359e700 1 -- 172.21.15.47:0/3655937022 shutdown_connections 2022-06-29T15:50:30.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.459+0000 7fc2c359e700 1 --2- 172.21.15.47:0/3655937022 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc2c4147140 0x7fc2c40a4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:30.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.459+0000 7fc2c359e700 1 --2- 172.21.15.47:0/3655937022 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2c4144670 0x7fc2c40a4770 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:30.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.459+0000 7fc2c359e700 1 --2- 172.21.15.47:0/3655937022 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc2a8053a70 0x7fc2a8055f30 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:30.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.459+0000 7fc2c359e700 1 --2- 172.21.15.47:0/3655937022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2c405f0b0 0x7fc2c4066bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:30.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.459+0000 7fc2c359e700 1 -- 172.21.15.47:0/3655937022 >> 172.21.15.47:0/3655937022 conn(0x7fc2c4062570 msgr2=0x7fc2c413fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:30.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.459+0000 7fc2c359e700 1 -- 172.21.15.47:0/3655937022 shutdown_connections 2022-06-29T15:50:30.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.459+0000 7fc2c359e700 1 -- 172.21.15.47:0/3655937022 wait complete. 2022-06-29T15:50:30.471 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' removed 2022-06-29T15:50:30.481 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-06-29T15:50:30.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.591+0000 7f982a6e3700 1 Processor -- start 2022-06-29T15:50:30.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.591+0000 7f982a6e3700 1 -- start start 2022-06-29T15:50:30.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.591+0000 7f982a6e3700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f982405c850 0x7f9824143a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:30.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.591+0000 7f982a6e3700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f98240676a0 0x7f9824140130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:30.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.591+0000 7f982a6e3700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f982405cc30 con 0x7f98240b5d90 2022-06-29T15:50:30.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.591+0000 7f982a6e3700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98241430a0 con 0x7f98240676a0 2022-06-29T15:50:30.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.591+0000 7f982a6e3700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9824143300 con 0x7f982405c850 2022-06-29T15:50:30.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.591+0000 7f98296e1700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f982405c850 0x7f9824143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:30.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.591+0000 7f98296e1700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f982405c850 0x7f9824143a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:49586/0 (socket says 172.21.15.47:49586) 2022-06-29T15:50:30.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.591+0000 7f98296e1700 1 -- 172.21.15.47:0/2635836415 learned_addr learned my addr 172.21.15.47:0/2635836415 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:30.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.591+0000 7f9829ee2700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f98240676a0 0x7f9824140130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:30.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.591+0000 7f98296e1700 1 -- 172.21.15.47:0/2635836415 >> v2:172.21.15.64:3300/0 conn(0x7f98240676a0 msgr2=0x7f9824140130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:30.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.591+0000 7f98296e1700 1 --2- 172.21.15.47:0/2635836415 >> v2:172.21.15.64:3300/0 conn(0x7f98240676a0 0x7f9824140130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:30.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.591+0000 7f98296e1700 1 -- 172.21.15.47:0/2635836415 >> v1:172.21.15.64:6789/0 conn(0x7f98240b5d90 legacy=0x7f98240b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:30.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.591+0000 7f98296e1700 1 -- 172.21.15.47:0/2635836415 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98241435a0 con 0x7f982405c850 2022-06-29T15:50:30.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.591+0000 7f9813fff700 1 -- 172.21.15.47:0/2635836415 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3260588796 0 0) 0x7f982405cc30 con 0x7f98240b5d90 2022-06-29T15:50:30.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.591+0000 7f98296e1700 1 --2- 172.21.15.47:0/2635836415 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f982405c850 0x7f9824143a40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f9820002f70 tx=0x7f9820006e50).ready entity=mon.2 client_cookie=1a3251eb52778d19 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:30.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.591+0000 7f9813fff700 1 -- 172.21.15.47:0/2635836415 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f982000d5c0 con 0x7f982405c850 2022-06-29T15:50:30.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.591+0000 7f9813fff700 1 -- 172.21.15.47:0/2635836415 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9820000e90 con 0x7f982405c850 2022-06-29T15:50:30.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.591+0000 7f9813fff700 1 -- 172.21.15.47:0/2635836415 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f982000d910 con 0x7f982405c850 2022-06-29T15:50:30.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.595+0000 7f982a6e3700 1 -- 172.21.15.47:0/2635836415 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f982405c850 msgr2=0x7f9824143a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:30.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.595+0000 7f982a6e3700 1 --2- 172.21.15.47:0/2635836415 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f982405c850 0x7f9824143a40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f9820002f70 tx=0x7f9820006e50).stop 2022-06-29T15:50:30.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.595+0000 7f982a6e3700 1 -- 172.21.15.47:0/2635836415 shutdown_connections 2022-06-29T15:50:30.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.595+0000 7f982a6e3700 1 --2- 172.21.15.47:0/2635836415 >> v2:172.21.15.64:3300/0 conn(0x7f98240676a0 0x7f9824140130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:30.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.595+0000 7f982a6e3700 1 --2- 172.21.15.47:0/2635836415 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f982405c850 0x7f9824143a40 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:30.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.595+0000 7f982a6e3700 1 -- 172.21.15.47:0/2635836415 >> 172.21.15.47:0/2635836415 conn(0x7f9824060e00 msgr2=0x7f98240611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:30.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.595+0000 7f982a6e3700 1 -- 172.21.15.47:0/2635836415 shutdown_connections 2022-06-29T15:50:30.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.595+0000 7f982a6e3700 1 -- 172.21.15.47:0/2635836415 wait complete. 2022-06-29T15:50:30.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.595+0000 7f982a6e3700 1 Processor -- start 2022-06-29T15:50:30.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.595+0000 7f982a6e3700 1 -- start start 2022-06-29T15:50:30.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.595+0000 7f982a6e3700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f982405c850 0x7f9824145bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:30.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.595+0000 7f982a6e3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98240676a0 0x7f9824155b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:30.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.595+0000 7f982a6e3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98240b5d90 0x7f9824156000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:30.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.595+0000 7f982a6e3700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9824152240 con 0x7f98240676a0 2022-06-29T15:50:30.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.595+0000 7f982a6e3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98240b9ea0 con 0x7f98240b5d90 2022-06-29T15:50:30.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.595+0000 7f982a6e3700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98240ba140 con 0x7f982405c850 2022-06-29T15:50:30.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.595+0000 7f98296e1700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f982405c850 0x7f9824145bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:30.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.595+0000 7f9828ee0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98240676a0 0x7f9824155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:30.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.595+0000 7f98296e1700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f982405c850 0x7f9824145bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:49592/0 (socket says 172.21.15.47:49592) 2022-06-29T15:50:30.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.595+0000 7f9828ee0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98240676a0 0x7f9824155b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52494/0 (socket says 172.21.15.47:52494) 2022-06-29T15:50:30.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.595+0000 7f9829ee2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98240b5d90 0x7f9824156000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:30.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.595+0000 7f9828ee0700 1 -- 172.21.15.47:0/162874570 learned_addr learned my addr 172.21.15.47:0/162874570 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:30.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.599+0000 7f9828ee0700 1 -- 172.21.15.47:0/162874570 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f982405c850 msgr2=0x7f9824145bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:30.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.599+0000 7f9828ee0700 1 --2- 172.21.15.47:0/162874570 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f982405c850 0x7f9824145bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:30.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.599+0000 7f9828ee0700 1 -- 172.21.15.47:0/162874570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98240b5d90 msgr2=0x7f9824156000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:30.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.599+0000 7f9828ee0700 1 --2- 172.21.15.47:0/162874570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98240b5d90 0x7f9824156000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:30.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.599+0000 7f9828ee0700 1 -- 172.21.15.47:0/162874570 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f982000b1e0 con 0x7f98240676a0 2022-06-29T15:50:30.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.599+0000 7f9828ee0700 1 --2- 172.21.15.47:0/162874570 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98240676a0 0x7f9824155b10 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 rx=0x7f9814000ed0 tx=0x7f9814007ab0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:30.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.599+0000 7f98127fc700 1 -- 172.21.15.47:0/162874570 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9814012400 con 0x7f98240676a0 2022-06-29T15:50:30.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.599+0000 7f98127fc700 1 -- 172.21.15.47:0/162874570 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9814012560 con 0x7f98240676a0 2022-06-29T15:50:30.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.599+0000 7f982a6e3700 1 -- 172.21.15.47:0/162874570 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98240ba3e0 con 0x7f98240676a0 2022-06-29T15:50:30.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.599+0000 7f98127fc700 1 -- 172.21.15.47:0/162874570 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f98140128c0 con 0x7f98240676a0 2022-06-29T15:50:30.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.599+0000 7f982a6e3700 1 -- 172.21.15.47:0/162874570 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98241564f0 con 0x7f98240676a0 2022-06-29T15:50:30.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.599+0000 7f982a6e3700 1 -- 172.21.15.47:0/162874570 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98240573d0 con 0x7f98240676a0 2022-06-29T15:50:30.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.599+0000 7f98127fc700 1 -- 172.21.15.47:0/162874570 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9814019030 con 0x7f98240676a0 2022-06-29T15:50:30.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.603+0000 7f98127fc700 1 --2- 172.21.15.47:0/162874570 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9804053a70 0x7f9804055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:30.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.603+0000 7f98127fc700 1 -- 172.21.15.47:0/162874570 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f981406b180 con 0x7f98240676a0 2022-06-29T15:50:30.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.603+0000 7f98296e1700 1 --2- 172.21.15.47:0/162874570 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9804053a70 0x7f9804055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:30.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.603+0000 7f98296e1700 1 --2- 172.21.15.47:0/162874570 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9804053a70 0x7f9804055f30 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7f982001a000 tx=0x7f9820014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:30.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.611+0000 7f98127fc700 1 -- 172.21.15.47:0/162874570 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f98140421c0 con 0x7f98240676a0 2022-06-29T15:50:30.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.739+0000 7f982a6e3700 1 -- 172.21.15.47:0/162874570 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f9824069a60 con 0x7f98240676a0 2022-06-29T15:50:30.753 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.747+0000 7f98127fc700 1 -- 172.21.15.47:0/162874570 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f98140457e0 con 0x7f98240676a0 2022-06-29T15:50:30.756 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.751+0000 7f982a6e3700 1 -- 172.21.15.47:0/162874570 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9804053a70 msgr2=0x7f9804055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:30.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.751+0000 7f982a6e3700 1 --2- 172.21.15.47:0/162874570 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9804053a70 0x7f9804055f30 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7f982001a000 tx=0x7f9820014000).stop 2022-06-29T15:50:30.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.751+0000 7f982a6e3700 1 -- 172.21.15.47:0/162874570 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98240676a0 msgr2=0x7f9824155b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:30.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.751+0000 7f982a6e3700 1 --2- 172.21.15.47:0/162874570 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98240676a0 0x7f9824155b10 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 rx=0x7f9814000ed0 tx=0x7f9814007ab0).stop 2022-06-29T15:50:30.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.751+0000 7f982a6e3700 1 -- 172.21.15.47:0/162874570 shutdown_connections 2022-06-29T15:50:30.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.751+0000 7f982a6e3700 1 --2- 172.21.15.47:0/162874570 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9804053a70 0x7f9804055f30 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:30.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.751+0000 7f982a6e3700 1 --2- 172.21.15.47:0/162874570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98240b5d90 0x7f9824156000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:30.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.751+0000 7f982a6e3700 1 --2- 172.21.15.47:0/162874570 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98240676a0 0x7f9824155b10 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:30.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.751+0000 7f982a6e3700 1 --2- 172.21.15.47:0/162874570 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f982405c850 0x7f9824145bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:30.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.751+0000 7f982a6e3700 1 -- 172.21.15.47:0/162874570 >> 172.21.15.47:0/162874570 conn(0x7f9824060e00 msgr2=0x7f9824140110 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:30.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.751+0000 7f982a6e3700 1 -- 172.21.15.47:0/162874570 shutdown_connections 2022-06-29T15:50:30.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.751+0000 7f982a6e3700 1 -- 172.21.15.47:0/162874570 wait complete. 2022-06-29T15:50:30.773 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestMisc.test_filtered_df' 2022-06-29T15:50:30.877 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.871+0000 7fa68b59e700 1 Processor -- start 2022-06-29T15:50:30.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.871+0000 7fa68b59e700 1 -- start start 2022-06-29T15:50:30.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.871+0000 7fa68b59e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa68c0b5c60 0x7fa68c144980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:30.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.871+0000 7fa68b59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa68c0a72f0 0x7fa68c0b2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:30.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.871+0000 7fa68b59e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa68c0a9120 con 0x7fa68c060eb0 2022-06-29T15:50:30.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.871+0000 7fa68b59e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa68c0b0b80 con 0x7fa68c0b5c60 2022-06-29T15:50:30.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.871+0000 7fa68b59e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa68c0a89e0 con 0x7fa68c0a72f0 2022-06-29T15:50:30.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.871+0000 7fa68a59c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa68c0b5c60 0x7fa68c144980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:30.883 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.871+0000 7fa689d9b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa68c060eb0 0x7fa68c0b1e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38052/0 (socket says 172.21.15.47:38052) 2022-06-29T15:50:30.883 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.871+0000 7fa689d9b700 1 -- 172.21.15.47:0/1115383805 learned_addr learned my addr 172.21.15.47:0/1115383805 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:30.883 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.871+0000 7fa68ad9d700 1 --2- 172.21.15.47:0/1115383805 >> v2:172.21.15.64:3300/0 conn(0x7fa68c0a72f0 0x7fa68c0b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:30.883 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.871+0000 7fa68959a700 1 -- 172.21.15.47:0/1115383805 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1705066658 0 0) 0x7fa68c0a9120 con 0x7fa68c060eb0 2022-06-29T15:50:30.884 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.871+0000 7fa68a59c700 1 -- 172.21.15.47:0/1115383805 >> v2:172.21.15.64:3300/0 conn(0x7fa68c0a72f0 msgr2=0x7fa68c0b2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:30.884 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.871+0000 7fa68a59c700 1 --2- 172.21.15.47:0/1115383805 >> v2:172.21.15.64:3300/0 conn(0x7fa68c0a72f0 0x7fa68c0b2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:30.884 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.871+0000 7fa68a59c700 1 -- 172.21.15.47:0/1115383805 >> v1:172.21.15.47:6789/0 conn(0x7fa68c060eb0 legacy=0x7fa68c0b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:30.885 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.871+0000 7fa68a59c700 1 -- 172.21.15.47:0/1115383805 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa68c0a8c80 con 0x7fa68c0b5c60 2022-06-29T15:50:30.885 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.871+0000 7fa68ad9d700 1 --2- 172.21.15.47:0/1115383805 >> v2:172.21.15.64:3300/0 conn(0x7fa68c0a72f0 0x7fa68c0b2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:50:30.885 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.871+0000 7fa68a59c700 1 --2- 172.21.15.47:0/1115383805 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa68c0b5c60 0x7fa68c144980 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 rx=0x7fa680002f70 tx=0x7fa680006d90).ready entity=mon.0 client_cookie=e81304cbd0f74582 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:30.885 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.875+0000 7fa68959a700 1 -- 172.21.15.47:0/1115383805 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa68000d4a0 con 0x7fa68c0b5c60 2022-06-29T15:50:30.886 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.875+0000 7fa68959a700 1 -- 172.21.15.47:0/1115383805 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa680000e90 con 0x7fa68c0b5c60 2022-06-29T15:50:30.886 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.875+0000 7fa68959a700 1 -- 172.21.15.47:0/1115383805 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa68000d7f0 con 0x7fa68c0b5c60 2022-06-29T15:50:30.886 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.875+0000 7fa68b59e700 1 -- 172.21.15.47:0/1115383805 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa68c0b5c60 msgr2=0x7fa68c144980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:30.886 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.875+0000 7fa68b59e700 1 --2- 172.21.15.47:0/1115383805 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa68c0b5c60 0x7fa68c144980 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 rx=0x7fa680002f70 tx=0x7fa680006d90).stop 2022-06-29T15:50:30.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.875+0000 7fa68b59e700 1 -- 172.21.15.47:0/1115383805 shutdown_connections 2022-06-29T15:50:30.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.875+0000 7fa68b59e700 1 --2- 172.21.15.47:0/1115383805 >> v2:172.21.15.64:3300/0 conn(0x7fa68c0a72f0 0x7fa68c0b2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:30.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.875+0000 7fa68b59e700 1 --2- 172.21.15.47:0/1115383805 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa68c0b5c60 0x7fa68c144980 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:30.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.875+0000 7fa68b59e700 1 -- 172.21.15.47:0/1115383805 >> 172.21.15.47:0/1115383805 conn(0x7fa68c062560 msgr2=0x7fa68c062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:30.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.875+0000 7fa68b59e700 1 -- 172.21.15.47:0/1115383805 shutdown_connections 2022-06-29T15:50:30.888 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.875+0000 7fa68b59e700 1 -- 172.21.15.47:0/1115383805 wait complete. 2022-06-29T15:50:30.894 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.875+0000 7fa68b59e700 1 Processor -- start 2022-06-29T15:50:30.894 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.875+0000 7fa68b59e700 1 -- start start 2022-06-29T15:50:30.895 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.879+0000 7fa68b59e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa68c060eb0 0x7fa68c0b35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:30.895 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.879+0000 7fa68b59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa68c0a72f0 0x7fa68c14b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:30.896 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.879+0000 7fa68b59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa68c0b5c60 0x7fa68c15a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:30.896 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.879+0000 7fa68b59e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa68c156950 con 0x7fa68c0b5c60 2022-06-29T15:50:30.897 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.879+0000 7fa68b59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa68c149480 con 0x7fa68c0a72f0 2022-06-29T15:50:30.897 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.879+0000 7fa68b59e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa68c14b000 con 0x7fa68c060eb0 2022-06-29T15:50:30.898 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.879+0000 7fa68ad9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa68c0b5c60 0x7fa68c15a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:30.898 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.879+0000 7fa68ad9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa68c0b5c60 0x7fa68c15a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52508/0 (socket says 172.21.15.47:52508) 2022-06-29T15:50:30.899 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.879+0000 7fa689d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa68c0a72f0 0x7fa68c14b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:30.899 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.879+0000 7fa68ad9d700 1 -- 172.21.15.47:0/174657316 learned_addr learned my addr 172.21.15.47:0/174657316 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:30.899 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.879+0000 7fa68a59c700 1 --2- 172.21.15.47:0/174657316 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa68c060eb0 0x7fa68c0b35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:30.900 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.879+0000 7fa68ad9d700 1 -- 172.21.15.47:0/174657316 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa68c060eb0 msgr2=0x7fa68c0b35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:30.900 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.879+0000 7fa68ad9d700 1 --2- 172.21.15.47:0/174657316 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa68c060eb0 0x7fa68c0b35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:30.901 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.879+0000 7fa68ad9d700 1 -- 172.21.15.47:0/174657316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa68c0a72f0 msgr2=0x7fa68c14b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:30.901 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.879+0000 7fa68ad9d700 1 --2- 172.21.15.47:0/174657316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa68c0a72f0 0x7fa68c14b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:30.902 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.879+0000 7fa68ad9d700 1 -- 172.21.15.47:0/174657316 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa68000b120 con 0x7fa68c0b5c60 2022-06-29T15:50:30.902 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.879+0000 7fa68ad9d700 1 --2- 172.21.15.47:0/174657316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa68c0b5c60 0x7fa68c15a230 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 rx=0x7fa684006cc0 tx=0x7fa684008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:30.902 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.879+0000 7fa67b7fe700 1 -- 172.21.15.47:0/174657316 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa684013620 con 0x7fa68c0b5c60 2022-06-29T15:50:30.903 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.879+0000 7fa67b7fe700 1 -- 172.21.15.47:0/174657316 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa684013780 con 0x7fa68c0b5c60 2022-06-29T15:50:30.903 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.879+0000 7fa68b59e700 1 -- 172.21.15.47:0/174657316 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa68c14a820 con 0x7fa68c0b5c60 2022-06-29T15:50:30.904 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.879+0000 7fa67b7fe700 1 -- 172.21.15.47:0/174657316 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa684013ae0 con 0x7fa68c0b5c60 2022-06-29T15:50:30.904 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.879+0000 7fa68b59e700 1 -- 172.21.15.47:0/174657316 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa68c14a510 con 0x7fa68c0b5c60 2022-06-29T15:50:30.905 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.883+0000 7fa68b59e700 1 -- 172.21.15.47:0/174657316 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa68c159f70 con 0x7fa68c0b5c60 2022-06-29T15:50:30.905 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.883+0000 7fa67b7fe700 1 -- 172.21.15.47:0/174657316 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa684013c40 con 0x7fa68c0b5c60 2022-06-29T15:50:30.906 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.883+0000 7fa67b7fe700 1 --2- 172.21.15.47:0/174657316 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa668053a70 0x7fa668055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:30.906 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.883+0000 7fa67b7fe700 1 -- 172.21.15.47:0/174657316 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fa68406bb70 con 0x7fa68c0b5c60 2022-06-29T15:50:30.906 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.883+0000 7fa68a59c700 1 --2- 172.21.15.47:0/174657316 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa668053a70 0x7fa668055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:30.907 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.883+0000 7fa68a59c700 1 --2- 172.21.15.47:0/174657316 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa668053a70 0x7fa668055f30 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7fa68001a000 tx=0x7fa680013000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:30.908 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:30.887+0000 7fa67b7fe700 1 -- 172.21.15.47:0/174657316 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa68c159f70 con 0x7fa68c0b5c60 2022-06-29T15:50:31.064 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.055+0000 7fa68b59e700 1 -- 172.21.15.47:0/174657316 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_filtered_df"]} v 0) v1 -- 0x7fa68c159f70 con 0x7fa68c0b5c60 2022-06-29T15:50:31.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.459+0000 7fa67b7fe700 1 -- 172.21.15.47:0/174657316 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_filtered_df"]}]=0 v361) v1 ==== 127+0+0 (secure 0 0 0) 0x7fa68c159f70 con 0x7fa68c0b5c60 2022-06-29T15:50:31.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.463+0000 7fa68b59e700 1 -- 172.21.15.47:0/174657316 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa668053a70 msgr2=0x7fa668055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:31.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.463+0000 7fa68b59e700 1 --2- 172.21.15.47:0/174657316 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa668053a70 0x7fa668055f30 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7fa68001a000 tx=0x7fa680013000).stop 2022-06-29T15:50:31.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.463+0000 7fa68b59e700 1 -- 172.21.15.47:0/174657316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa68c0b5c60 msgr2=0x7fa68c15a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:31.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.463+0000 7fa68b59e700 1 --2- 172.21.15.47:0/174657316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa68c0b5c60 0x7fa68c15a230 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 rx=0x7fa684006cc0 tx=0x7fa684008040).stop 2022-06-29T15:50:31.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.463+0000 7fa68b59e700 1 -- 172.21.15.47:0/174657316 shutdown_connections 2022-06-29T15:50:31.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.463+0000 7fa68b59e700 1 --2- 172.21.15.47:0/174657316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa68c0b5c60 0x7fa68c15a230 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:31.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.463+0000 7fa68b59e700 1 --2- 172.21.15.47:0/174657316 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa668053a70 0x7fa668055f30 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:31.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.463+0000 7fa68b59e700 1 --2- 172.21.15.47:0/174657316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa68c0a72f0 0x7fa68c14b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:31.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.463+0000 7fa68b59e700 1 --2- 172.21.15.47:0/174657316 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa68c060eb0 0x7fa68c0b35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:31.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.463+0000 7fa68b59e700 1 -- 172.21.15.47:0/174657316 >> 172.21.15.47:0/174657316 conn(0x7fa68c062560 msgr2=0x7fa68c062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:31.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.463+0000 7fa68b59e700 1 -- 172.21.15.47:0/174657316 shutdown_connections 2022-06-29T15:50:31.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.463+0000 7fa68b59e700 1 -- 172.21.15.47:0/174657316 wait complete. 2022-06-29T15:50:31.486 INFO:tasks.cephfs_test_runner:test_filtered_df (tasks.cephfs.test_misc.TestMisc) ... ok 2022-06-29T15:50:31.487 INFO:tasks.cephfs_test_runner:Starting test: test_fs_new (tasks.cephfs.test_misc.TestMisc) 2022-06-29T15:50:31.488 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_fs_new' 2022-06-29T15:50:31.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.591+0000 7fd53d909700 1 Processor -- start 2022-06-29T15:50:31.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.591+0000 7fd53d909700 1 -- start start 2022-06-29T15:50:31.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.595+0000 7fd53d909700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd5380b1950 0x7fd538099e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:31.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.595+0000 7fd53d909700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd5380acfa0 0x7fd5380af470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:31.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.595+0000 7fd53d909700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd53809b5b0 con 0x7fd5380af960 2022-06-29T15:50:31.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.595+0000 7fd53d909700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5380b1d30 con 0x7fd5380acfa0 2022-06-29T15:50:31.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.595+0000 7fd53d909700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd53809bc80 con 0x7fd5380b1950 2022-06-29T15:50:31.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.595+0000 7fd537fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd5380acfa0 0x7fd5380af470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:31.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.595+0000 7fd53d108700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd5380af960 0x7fd53805f9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38070/0 (socket says 172.21.15.47:38070) 2022-06-29T15:50:31.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.595+0000 7fd53d108700 1 -- 172.21.15.47:0/925901743 learned_addr learned my addr 172.21.15.47:0/925901743 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:31.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.595+0000 7fd53c907700 1 --2- 172.21.15.47:0/925901743 >> v2:172.21.15.64:3300/0 conn(0x7fd5380b1950 0x7fd538099e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:31.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.595+0000 7fd5377fe700 1 -- 172.21.15.47:0/925901743 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1565895407 0 0) 0x7fd53809b5b0 con 0x7fd5380af960 2022-06-29T15:50:31.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.595+0000 7fd5377fe700 1 -- 172.21.15.47:0/925901743 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd51c003260 con 0x7fd5380af960 2022-06-29T15:50:31.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.595+0000 7fd537fff700 1 -- 172.21.15.47:0/925901743 >> v2:172.21.15.64:3300/0 conn(0x7fd5380b1950 msgr2=0x7fd538099e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:31.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.595+0000 7fd537fff700 1 --2- 172.21.15.47:0/925901743 >> v2:172.21.15.64:3300/0 conn(0x7fd5380b1950 0x7fd538099e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:31.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.595+0000 7fd537fff700 1 -- 172.21.15.47:0/925901743 >> v1:172.21.15.47:6789/0 conn(0x7fd5380af960 legacy=0x7fd53805f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:31.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.595+0000 7fd537fff700 1 -- 172.21.15.47:0/925901743 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd538099200 con 0x7fd5380acfa0 2022-06-29T15:50:31.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.595+0000 7fd537fff700 1 --2- 172.21.15.47:0/925901743 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5380acfa0 0x7fd5380af470 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 rx=0x7fd524002f70 tx=0x7fd524006df0).ready entity=mon.0 client_cookie=980eaccd76ad836b server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:31.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.595+0000 7fd5377fe700 1 -- 172.21.15.47:0/925901743 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd52400d4f0 con 0x7fd5380acfa0 2022-06-29T15:50:31.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.595+0000 7fd5377fe700 1 -- 172.21.15.47:0/925901743 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd524000e90 con 0x7fd5380acfa0 2022-06-29T15:50:31.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.595+0000 7fd5377fe700 1 -- 172.21.15.47:0/925901743 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd52400d840 con 0x7fd5380acfa0 2022-06-29T15:50:31.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.599+0000 7fd53d909700 1 -- 172.21.15.47:0/925901743 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5380acfa0 msgr2=0x7fd5380af470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:31.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.599+0000 7fd53d909700 1 --2- 172.21.15.47:0/925901743 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5380acfa0 0x7fd5380af470 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 rx=0x7fd524002f70 tx=0x7fd524006df0).stop 2022-06-29T15:50:31.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.599+0000 7fd53d909700 1 -- 172.21.15.47:0/925901743 shutdown_connections 2022-06-29T15:50:31.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.599+0000 7fd53d909700 1 --2- 172.21.15.47:0/925901743 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5380acfa0 0x7fd5380af470 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:31.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.599+0000 7fd53d909700 1 --2- 172.21.15.47:0/925901743 >> v2:172.21.15.64:3300/0 conn(0x7fd5380b1950 0x7fd538099e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:31.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.599+0000 7fd53d909700 1 -- 172.21.15.47:0/925901743 >> 172.21.15.47:0/925901743 conn(0x7fd5380b5b60 msgr2=0x7fd5380b5f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:31.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.599+0000 7fd53d909700 1 -- 172.21.15.47:0/925901743 shutdown_connections 2022-06-29T15:50:31.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.599+0000 7fd53d909700 1 -- 172.21.15.47:0/925901743 wait complete. 2022-06-29T15:50:31.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.599+0000 7fd53d909700 1 Processor -- start 2022-06-29T15:50:31.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.599+0000 7fd53d909700 1 -- start start 2022-06-29T15:50:31.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.599+0000 7fd53d909700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5380acfa0 0x7fd5380ab6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:31.622 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.599+0000 7fd53d909700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd5380af960 0x7fd5381408f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:31.622 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.599+0000 7fd53d909700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5380b1950 0x7fd538140de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:31.622 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.599+0000 7fd53d909700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5380914b0 con 0x7fd5380acfa0 2022-06-29T15:50:31.622 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.599+0000 7fd53d909700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd538066ce0 con 0x7fd5380b1950 2022-06-29T15:50:31.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.599+0000 7fd53d909700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd538066f80 con 0x7fd5380af960 2022-06-29T15:50:31.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.603+0000 7fd53c907700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5380acfa0 0x7fd5380ab6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:31.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.603+0000 7fd53c907700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5380acfa0 0x7fd5380ab6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52518/0 (socket says 172.21.15.47:52518) 2022-06-29T15:50:31.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.603+0000 7fd53c907700 1 -- 172.21.15.47:0/1058997506 learned_addr learned my addr 172.21.15.47:0/1058997506 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:31.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.603+0000 7fd537fff700 1 --2- 172.21.15.47:0/1058997506 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd5380af960 0x7fd5381408f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:31.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.603+0000 7fd53d108700 1 --2- 172.21.15.47:0/1058997506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5380b1950 0x7fd538140de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:31.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.603+0000 7fd53c907700 1 -- 172.21.15.47:0/1058997506 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd5380af960 msgr2=0x7fd5381408f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:31.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.603+0000 7fd53c907700 1 --2- 172.21.15.47:0/1058997506 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd5380af960 0x7fd5381408f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:31.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.603+0000 7fd53c907700 1 -- 172.21.15.47:0/1058997506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5380b1950 msgr2=0x7fd538140de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:31.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.603+0000 7fd53c907700 1 --2- 172.21.15.47:0/1058997506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5380b1950 0x7fd538140de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:31.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.603+0000 7fd53c907700 1 -- 172.21.15.47:0/1058997506 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd52400b180 con 0x7fd5380acfa0 2022-06-29T15:50:31.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.603+0000 7fd53c907700 1 --2- 172.21.15.47:0/1058997506 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5380acfa0 0x7fd5380ab6f0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 rx=0x7fd52c00c000 tx=0x7fd52c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:31.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.603+0000 7fd535ffb700 1 -- 172.21.15.47:0/1058997506 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd52c011750 con 0x7fd5380acfa0 2022-06-29T15:50:31.626 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.603+0000 7fd535ffb700 1 -- 172.21.15.47:0/1058997506 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd52c0118b0 con 0x7fd5380acfa0 2022-06-29T15:50:31.626 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.603+0000 7fd53d909700 1 -- 172.21.15.47:0/1058997506 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd538067220 con 0x7fd5380acfa0 2022-06-29T15:50:31.626 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.603+0000 7fd535ffb700 1 -- 172.21.15.47:0/1058997506 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd52c011c10 con 0x7fd5380acfa0 2022-06-29T15:50:31.626 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.603+0000 7fd53d909700 1 -- 172.21.15.47:0/1058997506 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5381412d0 con 0x7fd5380acfa0 2022-06-29T15:50:31.627 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.603+0000 7fd535ffb700 1 -- 172.21.15.47:0/1058997506 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd52c011d70 con 0x7fd5380acfa0 2022-06-29T15:50:31.627 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.603+0000 7fd53d909700 1 -- 172.21.15.47:0/1058997506 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5380573d0 con 0x7fd5380acfa0 2022-06-29T15:50:31.627 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.603+0000 7fd535ffb700 1 --2- 172.21.15.47:0/1058997506 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd518053950 0x7fd518055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:31.627 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.607+0000 7fd535ffb700 1 -- 172.21.15.47:0/1058997506 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fd52c06c310 con 0x7fd5380acfa0 2022-06-29T15:50:31.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.607+0000 7fd537fff700 1 --2- 172.21.15.47:0/1058997506 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd518053950 0x7fd518055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:31.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.607+0000 7fd537fff700 1 --2- 172.21.15.47:0/1058997506 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd518053950 0x7fd518055e10 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7fd52401a000 tx=0x7fd524014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:31.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.611+0000 7fd535ffb700 1 -- 172.21.15.47:0/1058997506 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd52c0432a0 con 0x7fd5380acfa0 2022-06-29T15:50:31.788 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:31.783+0000 7fd53d909700 1 -- 172.21.15.47:0/1058997506 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_fs_new"]} v 0) v1 -- 0x7fd5380b5250 con 0x7fd5380acfa0 2022-06-29T15:50:32.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.463+0000 7fd535ffb700 1 -- 172.21.15.47:0/1058997506 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_fs_new"]}]=0 v362) v1 ==== 125+0+0 (secure 0 0 0) 0x7fd52c0468c0 con 0x7fd5380acfa0 2022-06-29T15:50:32.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.467+0000 7fd53d909700 1 -- 172.21.15.47:0/1058997506 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd518053950 msgr2=0x7fd518055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:32.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.467+0000 7fd53d909700 1 --2- 172.21.15.47:0/1058997506 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd518053950 0x7fd518055e10 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7fd52401a000 tx=0x7fd524014000).stop 2022-06-29T15:50:32.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.467+0000 7fd53d909700 1 -- 172.21.15.47:0/1058997506 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5380acfa0 msgr2=0x7fd5380ab6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:32.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.467+0000 7fd53d909700 1 --2- 172.21.15.47:0/1058997506 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5380acfa0 0x7fd5380ab6f0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 rx=0x7fd52c00c000 tx=0x7fd52c006df0).stop 2022-06-29T15:50:32.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.467+0000 7fd53d909700 1 -- 172.21.15.47:0/1058997506 shutdown_connections 2022-06-29T15:50:32.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.467+0000 7fd53d909700 1 --2- 172.21.15.47:0/1058997506 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd518053950 0x7fd518055e10 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:32.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.467+0000 7fd53d909700 1 --2- 172.21.15.47:0/1058997506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5380b1950 0x7fd538140de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:32.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.467+0000 7fd53d909700 1 --2- 172.21.15.47:0/1058997506 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd5380af960 0x7fd5381408f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:32.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.467+0000 7fd53d909700 1 --2- 172.21.15.47:0/1058997506 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5380acfa0 0x7fd5380ab6f0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:32.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.467+0000 7fd53d909700 1 -- 172.21.15.47:0/1058997506 >> 172.21.15.47:0/1058997506 conn(0x7fd5380b5b60 msgr2=0x7fd5380ac220 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:32.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.467+0000 7fd53d909700 1 -- 172.21.15.47:0/1058997506 shutdown_connections 2022-06-29T15:50:32.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.467+0000 7fd53d909700 1 -- 172.21.15.47:0/1058997506 wait complete. 2022-06-29T15:50:32.490 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-06-29T15:50:32.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.599+0000 7fb84d772700 1 Processor -- start 2022-06-29T15:50:32.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.599+0000 7fb84d772700 1 -- start start 2022-06-29T15:50:32.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.599+0000 7fb84d772700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb8480612c0 0x7fb8481471a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:32.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.599+0000 7fb84d772700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb848067610 con 0x7fb848060ee0 2022-06-29T15:50:32.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.599+0000 7fb84d772700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8481467c0 con 0x7fb8480b4300 2022-06-29T15:50:32.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.599+0000 7fb84d772700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb848146a60 con 0x7fb8480612c0 2022-06-29T15:50:32.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.603+0000 7fb846ffd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb848060ee0 0x7fb8481430e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38082/0 (socket says 172.21.15.47:38082) 2022-06-29T15:50:32.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.603+0000 7fb846ffd700 1 -- 172.21.15.47:0/3541563643 learned_addr learned my addr 172.21.15.47:0/3541563643 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:32.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.603+0000 7fb8477fe700 1 --2- 172.21.15.47:0/3541563643 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb8480612c0 0x7fb8481471a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:32.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.603+0000 7fb845ffb700 1 -- 172.21.15.47:0/3541563643 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1728868590 0 0) 0x7fb848067610 con 0x7fb848060ee0 2022-06-29T15:50:32.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.603+0000 7fb845ffb700 1 -- 172.21.15.47:0/3541563643 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb828003260 con 0x7fb848060ee0 2022-06-29T15:50:32.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.603+0000 7fb845ffb700 1 -- 172.21.15.47:0/3541563643 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3600276410 0 0) 0x7fb8481467c0 con 0x7fb8480b4300 2022-06-29T15:50:32.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.603+0000 7fb845ffb700 1 -- 172.21.15.47:0/3541563643 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb848067610 con 0x7fb8480b4300 2022-06-29T15:50:32.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.603+0000 7fb845ffb700 1 -- 172.21.15.47:0/3541563643 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2176023527 0 0) 0x7fb828003260 con 0x7fb848060ee0 2022-06-29T15:50:32.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.603+0000 7fb8477fe700 1 -- 172.21.15.47:0/3541563643 >> v1:172.21.15.64:6789/0 conn(0x7fb8480b4300 legacy=0x7fb84813cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:32.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.603+0000 7fb8477fe700 1 -- 172.21.15.47:0/3541563643 >> v1:172.21.15.47:6789/0 conn(0x7fb848060ee0 legacy=0x7fb8481430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:32.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.603+0000 7fb8477fe700 1 -- 172.21.15.47:0/3541563643 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb848146d00 con 0x7fb8480612c0 2022-06-29T15:50:32.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.603+0000 7fb8477fe700 1 --2- 172.21.15.47:0/3541563643 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb8480612c0 0x7fb8481471a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7fb838001110 tx=0x7fb8400091d0).ready entity=mon.2 client_cookie=8e5c8cde02d8afbe server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:32.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.603+0000 7fb845ffb700 1 -- 172.21.15.47:0/3541563643 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb838001660 con 0x7fb8480612c0 2022-06-29T15:50:32.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.603+0000 7fb845ffb700 1 -- 172.21.15.47:0/3541563643 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb84000f510 con 0x7fb8480612c0 2022-06-29T15:50:32.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.603+0000 7fb845ffb700 1 -- 172.21.15.47:0/3541563643 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb84000f680 con 0x7fb8480612c0 2022-06-29T15:50:32.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.603+0000 7fb84d772700 1 -- 172.21.15.47:0/3541563643 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb8480612c0 msgr2=0x7fb8481471a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:32.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.603+0000 7fb84d772700 1 --2- 172.21.15.47:0/3541563643 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb8480612c0 0x7fb8481471a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7fb838001110 tx=0x7fb8400091d0).stop 2022-06-29T15:50:32.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.603+0000 7fb84d772700 1 -- 172.21.15.47:0/3541563643 shutdown_connections 2022-06-29T15:50:32.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.603+0000 7fb84d772700 1 --2- 172.21.15.47:0/3541563643 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb8480612c0 0x7fb8481471a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:32.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.603+0000 7fb84d772700 1 -- 172.21.15.47:0/3541563643 >> 172.21.15.47:0/3541563643 conn(0x7fb84805c510 msgr2=0x7fb84805f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:32.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.603+0000 7fb84d772700 1 -- 172.21.15.47:0/3541563643 shutdown_connections 2022-06-29T15:50:32.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.603+0000 7fb84d772700 1 -- 172.21.15.47:0/3541563643 wait complete. 2022-06-29T15:50:32.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.607+0000 7fb84d772700 1 Processor -- start 2022-06-29T15:50:32.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.607+0000 7fb84d772700 1 -- start start 2022-06-29T15:50:32.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.607+0000 7fb84d772700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb848060ee0 0x7fb848140520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:32.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.607+0000 7fb84d772700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8480612c0 0x7fb84813df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:32.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.607+0000 7fb84d772700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8480b4300 0x7fb8480b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:32.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.607+0000 7fb84d772700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb84813f820 con 0x7fb8480612c0 2022-06-29T15:50:32.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.607+0000 7fb84d772700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8481430c0 con 0x7fb8480b4300 2022-06-29T15:50:32.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.607+0000 7fb84d772700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8480b25f0 con 0x7fb848060ee0 2022-06-29T15:50:32.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.607+0000 7fb8467fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8480612c0 0x7fb84813df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:32.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.607+0000 7fb8467fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8480612c0 0x7fb84813df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52534/0 (socket says 172.21.15.47:52534) 2022-06-29T15:50:32.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.607+0000 7fb8467fc700 1 -- 172.21.15.47:0/16654656 learned_addr learned my addr 172.21.15.47:0/16654656 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:32.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.607+0000 7fb846ffd700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb848060ee0 0x7fb848140520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:32.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.607+0000 7fb8477fe700 1 --2- 172.21.15.47:0/16654656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8480b4300 0x7fb8480b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:32.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.607+0000 7fb846ffd700 1 -- 172.21.15.47:0/16654656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8480b4300 msgr2=0x7fb8480b2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:32.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.607+0000 7fb846ffd700 1 --2- 172.21.15.47:0/16654656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8480b4300 0x7fb8480b2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:32.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.607+0000 7fb846ffd700 1 -- 172.21.15.47:0/16654656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8480612c0 msgr2=0x7fb84813df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:32.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.607+0000 7fb846ffd700 1 --2- 172.21.15.47:0/16654656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8480612c0 0x7fb84813df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:32.622 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.607+0000 7fb846ffd700 1 -- 172.21.15.47:0/16654656 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb840009470 con 0x7fb848060ee0 2022-06-29T15:50:32.622 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.607+0000 7fb8467fc700 1 --2- 172.21.15.47:0/16654656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8480612c0 0x7fb84813df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:50:32.622 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.607+0000 7fb846ffd700 1 --2- 172.21.15.47:0/16654656 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb848060ee0 0x7fb848140520 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7fb838001840 tx=0x7fb838001a00).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:32.622 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.607+0000 7fb83ffff700 1 -- 172.21.15.47:0/16654656 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb83800ce10 con 0x7fb848060ee0 2022-06-29T15:50:32.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.607+0000 7fb83ffff700 1 -- 172.21.15.47:0/16654656 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb838007170 con 0x7fb848060ee0 2022-06-29T15:50:32.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.607+0000 7fb83ffff700 1 -- 172.21.15.47:0/16654656 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb838004730 con 0x7fb848060ee0 2022-06-29T15:50:32.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.607+0000 7fb84d772700 1 -- 172.21.15.47:0/16654656 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb84813fa50 con 0x7fb848060ee0 2022-06-29T15:50:32.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.607+0000 7fb84d772700 1 -- 172.21.15.47:0/16654656 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb848140190 con 0x7fb848060ee0 2022-06-29T15:50:32.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.611+0000 7fb84d772700 1 -- 172.21.15.47:0/16654656 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8480b23b0 con 0x7fb848060ee0 2022-06-29T15:50:32.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.611+0000 7fb83ffff700 1 -- 172.21.15.47:0/16654656 <== mon.2 v2:172.21.15.64:3301/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb83801b070 con 0x7fb848060ee0 2022-06-29T15:50:32.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.611+0000 7fb83ffff700 1 --2- 172.21.15.47:0/16654656 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb82c053a70 0x7fb82c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:32.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.611+0000 7fb83ffff700 1 -- 172.21.15.47:0/16654656 <== mon.2 v2:172.21.15.64:3301/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fb83806b540 con 0x7fb848060ee0 2022-06-29T15:50:32.626 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.611+0000 7fb8467fc700 1 --2- 172.21.15.47:0/16654656 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb82c053a70 0x7fb82c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:32.626 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.615+0000 7fb8467fc700 1 --2- 172.21.15.47:0/16654656 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb82c053a70 0x7fb82c055f30 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7fb83000d000 tx=0x7fb830007ab0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:32.626 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.619+0000 7fb83ffff700 1 -- 172.21.15.47:0/16654656 <== mon.2 v2:172.21.15.64:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb8480b23b0 con 0x7fb848060ee0 2022-06-29T15:50:32.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.755+0000 7fb84d772700 1 -- 172.21.15.47:0/16654656 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb8480b23b0 con 0x7fb848060ee0 2022-06-29T15:50:32.770 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.763+0000 7fb83ffff700 1 -- 172.21.15.47:0/16654656 <== mon.2 v2:172.21.15.64:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb8480b23b0 con 0x7fb848060ee0 2022-06-29T15:50:32.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.767+0000 7fb84d772700 1 -- 172.21.15.47:0/16654656 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb82c053a70 msgr2=0x7fb82c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:32.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.767+0000 7fb84d772700 1 --2- 172.21.15.47:0/16654656 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb82c053a70 0x7fb82c055f30 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7fb83000d000 tx=0x7fb830007ab0).stop 2022-06-29T15:50:32.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.767+0000 7fb84d772700 1 -- 172.21.15.47:0/16654656 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb848060ee0 msgr2=0x7fb848140520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:32.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.767+0000 7fb84d772700 1 --2- 172.21.15.47:0/16654656 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb848060ee0 0x7fb848140520 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7fb838001840 tx=0x7fb838001a00).stop 2022-06-29T15:50:32.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.767+0000 7fb84d772700 1 -- 172.21.15.47:0/16654656 shutdown_connections 2022-06-29T15:50:32.776 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.767+0000 7fb84d772700 1 --2- 172.21.15.47:0/16654656 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb82c053a70 0x7fb82c055f30 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:32.776 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.767+0000 7fb84d772700 1 --2- 172.21.15.47:0/16654656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8480b4300 0x7fb8480b2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:32.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.767+0000 7fb84d772700 1 --2- 172.21.15.47:0/16654656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8480612c0 0x7fb84813df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:32.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.767+0000 7fb84d772700 1 --2- 172.21.15.47:0/16654656 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb848060ee0 0x7fb848140520 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:32.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.767+0000 7fb84d772700 1 -- 172.21.15.47:0/16654656 >> 172.21.15.47:0/16654656 conn(0x7fb84805c510 msgr2=0x7fb84805f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:32.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.767+0000 7fb84d772700 1 -- 172.21.15.47:0/16654656 shutdown_connections 2022-06-29T15:50:32.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:32.767+0000 7fb84d772700 1 -- 172.21.15.47:0/16654656 wait complete. 2022-06-29T15:50:32.787 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-06-29T15:50:32.788 DEBUG:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:50:32.804 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:50:32.827 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:50:32.848 DEBUG:teuthology.orchestra.run.smithi164:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:50:32.870 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-06-29T15:50:32.871 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:32.872 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:50:32.873 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:50:32.880 INFO:teuthology.orchestra.run.smithi099.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-06-29T15:50:32.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T15:50:32.883 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-06-29T15:50:32.884 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:32.885 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:50:32.885 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:50:32.917 INFO:teuthology.orchestra.run.smithi164.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-06-29T15:50:32.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T15:50:32.919 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-06-29T15:50:32.920 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-06-29T15:50:33.031 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.023+0000 7f9c4d374700 1 Processor -- start 2022-06-29T15:50:33.031 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.023+0000 7f9c4d374700 1 -- start start 2022-06-29T15:50:33.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.027+0000 7f9c4d374700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9c48147370 0x7f9c4814b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:33.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.027+0000 7f9c4d374700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c48098250 con 0x7f9c480ad730 2022-06-29T15:50:33.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.027+0000 7f9c4d374700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c4814a940 con 0x7f9c4814b530 2022-06-29T15:50:33.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.027+0000 7f9c4d374700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c48098480 con 0x7f9c48147370 2022-06-29T15:50:33.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.027+0000 7f9c47fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9c480ad730 0x7f9c480adb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38094/0 (socket says 172.21.15.47:38094) 2022-06-29T15:50:33.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.027+0000 7f9c47fff700 1 -- 172.21.15.47:0/3613581719 learned_addr learned my addr 172.21.15.47:0/3613581719 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:33.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.027+0000 7f9c477fe700 1 --2- 172.21.15.47:0/3613581719 >> v2:172.21.15.47:3300/0 conn(0x7f9c48147370 0x7f9c4814b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:33.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.027+0000 7f9c46ffd700 1 -- 172.21.15.47:0/3613581719 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3608141014 0 0) 0x7f9c48098250 con 0x7f9c480ad730 2022-06-29T15:50:33.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.027+0000 7f9c46ffd700 1 -- 172.21.15.47:0/3613581719 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c34003260 con 0x7f9c480ad730 2022-06-29T15:50:33.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.027+0000 7f9c46ffd700 1 -- 172.21.15.47:0/3613581719 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2291955059 0 0) 0x7f9c4814a940 con 0x7f9c4814b530 2022-06-29T15:50:33.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.027+0000 7f9c477fe700 1 -- 172.21.15.47:0/3613581719 >> v1:172.21.15.64:6789/0 conn(0x7f9c4814b530 legacy=0x7f9c481443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:33.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.027+0000 7f9c477fe700 1 -- 172.21.15.47:0/3613581719 >> v1:172.21.15.47:6789/0 conn(0x7f9c480ad730 legacy=0x7f9c480adb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:33.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.027+0000 7f9c477fe700 1 -- 172.21.15.47:0/3613581719 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c4814aba0 con 0x7f9c48147370 2022-06-29T15:50:33.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.027+0000 7f9c477fe700 1 --2- 172.21.15.47:0/3613581719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c48147370 0x7f9c4814b040 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 rx=0x7f9c4814af20 tx=0x7f9c3c006df0).ready entity=mon.0 client_cookie=56175366d270c6d server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:33.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.027+0000 7f9c46ffd700 1 -- 172.21.15.47:0/3613581719 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c3c00d610 con 0x7f9c48147370 2022-06-29T15:50:33.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.027+0000 7f9c46ffd700 1 -- 172.21.15.47:0/3613581719 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c3c000e90 con 0x7f9c48147370 2022-06-29T15:50:33.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.027+0000 7f9c46ffd700 1 -- 172.21.15.47:0/3613581719 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c3c00d960 con 0x7f9c48147370 2022-06-29T15:50:33.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.027+0000 7f9c4d374700 1 -- 172.21.15.47:0/3613581719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c48147370 msgr2=0x7f9c4814b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:33.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.027+0000 7f9c4d374700 1 --2- 172.21.15.47:0/3613581719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c48147370 0x7f9c4814b040 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 rx=0x7f9c4814af20 tx=0x7f9c3c006df0).stop 2022-06-29T15:50:33.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.027+0000 7f9c4d374700 1 -- 172.21.15.47:0/3613581719 shutdown_connections 2022-06-29T15:50:33.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.027+0000 7f9c4d374700 1 --2- 172.21.15.47:0/3613581719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c48147370 0x7f9c4814b040 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:33.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.027+0000 7f9c4d374700 1 -- 172.21.15.47:0/3613581719 >> 172.21.15.47:0/3613581719 conn(0x7f9c48062df0 msgr2=0x7f9c480631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:33.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.027+0000 7f9c4d374700 1 -- 172.21.15.47:0/3613581719 shutdown_connections 2022-06-29T15:50:33.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.027+0000 7f9c4d374700 1 -- 172.21.15.47:0/3613581719 wait complete. 2022-06-29T15:50:33.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.031+0000 7f9c4d374700 1 Processor -- start 2022-06-29T15:50:33.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.031+0000 7f9c4d374700 1 -- start start 2022-06-29T15:50:33.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.031+0000 7f9c4d374700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9c480ad730 0x7f9c48145550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:33.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.031+0000 7f9c4d374700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c48147370 0x7f9c4813d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:33.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.031+0000 7f9c4d374700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c4814b530 0x7f9c4813dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:33.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.031+0000 7f9c4d374700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c48145d90 con 0x7f9c4814b530 2022-06-29T15:50:33.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.031+0000 7f9c4d374700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4813edc0 con 0x7f9c48147370 2022-06-29T15:50:33.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.031+0000 7f9c4d374700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4813f060 con 0x7f9c480ad730 2022-06-29T15:50:33.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.035+0000 7f9c4cb73700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c4814b530 0x7f9c4813dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:33.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.035+0000 7f9c47fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9c480ad730 0x7f9c48145550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:33.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.035+0000 7f9c47fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9c480ad730 0x7f9c48145550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:49648/0 (socket says 172.21.15.47:49648) 2022-06-29T15:50:33.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.035+0000 7f9c4cb73700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c4814b530 0x7f9c4813dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52550/0 (socket says 172.21.15.47:52550) 2022-06-29T15:50:33.045 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.035+0000 7f9c47fff700 1 -- 172.21.15.47:0/87307524 learned_addr learned my addr 172.21.15.47:0/87307524 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:33.045 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.035+0000 7f9c477fe700 1 --2- 172.21.15.47:0/87307524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c48147370 0x7f9c4813d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:33.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.035+0000 7f9c47fff700 1 -- 172.21.15.47:0/87307524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c48147370 msgr2=0x7f9c4813d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:33.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.035+0000 7f9c47fff700 1 --2- 172.21.15.47:0/87307524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c48147370 0x7f9c4813d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:33.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.035+0000 7f9c47fff700 1 -- 172.21.15.47:0/87307524 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c4814b530 msgr2=0x7f9c4813dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:33.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.035+0000 7f9c47fff700 1 --2- 172.21.15.47:0/87307524 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c4814b530 0x7f9c4813dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:33.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.035+0000 7f9c47fff700 1 -- 172.21.15.47:0/87307524 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c3c00b180 con 0x7f9c480ad730 2022-06-29T15:50:33.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.035+0000 7f9c4cb73700 1 --2- 172.21.15.47:0/87307524 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c4814b530 0x7f9c4813dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:50:33.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.035+0000 7f9c47fff700 1 --2- 172.21.15.47:0/87307524 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9c480ad730 0x7f9c48145550 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f9c48147000 tx=0x7f9c38007aa0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:33.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.035+0000 7f9c457fa700 1 -- 172.21.15.47:0/87307524 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c3800ce70 con 0x7f9c480ad730 2022-06-29T15:50:33.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.035+0000 7f9c457fa700 1 -- 172.21.15.47:0/87307524 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c380070d0 con 0x7f9c480ad730 2022-06-29T15:50:33.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.035+0000 7f9c457fa700 1 -- 172.21.15.47:0/87307524 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c38007430 con 0x7f9c480ad730 2022-06-29T15:50:33.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.035+0000 7f9c4d374700 1 -- 172.21.15.47:0/87307524 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c4813f300 con 0x7f9c480ad730 2022-06-29T15:50:33.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.035+0000 7f9c4d374700 1 -- 172.21.15.47:0/87307524 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c481470b0 con 0x7f9c480ad730 2022-06-29T15:50:33.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.035+0000 7f9c4d374700 1 -- 172.21.15.47:0/87307524 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c480573d0 con 0x7f9c480ad730 2022-06-29T15:50:33.052 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.039+0000 7f9c457fa700 1 -- 172.21.15.47:0/87307524 <== mon.2 v2:172.21.15.64:3301/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9c38010070 con 0x7f9c480ad730 2022-06-29T15:50:33.052 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.039+0000 7f9c457fa700 1 --2- 172.21.15.47:0/87307524 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9c28053b00 0x7f9c28055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:33.052 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.039+0000 7f9c457fa700 1 -- 172.21.15.47:0/87307524 <== mon.2 v2:172.21.15.64:3301/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f9c3806b2d0 con 0x7f9c480ad730 2022-06-29T15:50:33.052 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.039+0000 7f9c477fe700 1 --2- 172.21.15.47:0/87307524 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9c28053b00 0x7f9c28055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:33.053 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.039+0000 7f9c477fe700 1 --2- 172.21.15.47:0/87307524 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9c28053b00 0x7f9c28055fc0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7f9c3c01a000 tx=0x7f9c3c014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:33.054 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.047+0000 7f9c457fa700 1 -- 172.21.15.47:0/87307524 <== mon.2 v2:172.21.15.64:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9c38042370 con 0x7f9c480ad730 2022-06-29T15:50:33.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.183+0000 7f9c4d374700 1 -- 172.21.15.47:0/87307524 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f9c4806a0f0 con 0x7f9c480ad730 2022-06-29T15:50:33.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.183+0000 7f9c457fa700 1 -- 172.21.15.47:0/87307524 <== mon.2 v2:172.21.15.64:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v114) v1 ==== 81+0+29229 (secure 0 0 0) 0x7f9c38018070 con 0x7f9c480ad730 2022-06-29T15:50:33.191 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:50:33.191 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2022-06-29T15:50:33.192 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 114, 2022-06-29T15:50:33.192 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "9a68894a-53ce-400e-838b-14eab91b834c", 2022-06-29T15:50:33.192 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2022-06-29T15:42:14.046361+0000", 2022-06-29T15:50:33.192 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2022-06-29T15:50:30.443310+0000", 2022-06-29T15:50:33.193 INFO:teuthology.orchestra.run.smithi047.stdout: "last_up_change": "2022-06-29T15:42:20.929789+0000", 2022-06-29T15:50:33.193 INFO:teuthology.orchestra.run.smithi047.stdout: "last_in_change": "2022-06-29T15:42:20.929789+0000", 2022-06-29T15:50:33.193 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-06-29T15:50:33.193 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_num": 5799936, 2022-06-29T15:50:33.193 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_set": [ 2022-06-29T15:50:33.194 INFO:teuthology.orchestra.run.smithi047.stdout: "pglog_hardlimit", 2022-06-29T15:50:33.194 INFO:teuthology.orchestra.run.smithi047.stdout: "purged_snapdirs", 2022-06-29T15:50:33.194 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_deletes", 2022-06-29T15:50:33.194 INFO:teuthology.orchestra.run.smithi047.stdout: "sortbitwise" 2022-06-29T15:50:33.195 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:50:33.195 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_version": 5, 2022-06-29T15:50:33.195 INFO:teuthology.orchestra.run.smithi047.stdout: "full_ratio": 0.94999998807907104, 2022-06-29T15:50:33.195 INFO:teuthology.orchestra.run.smithi047.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-06-29T15:50:33.196 INFO:teuthology.orchestra.run.smithi047.stdout: "nearfull_ratio": 0.85000002384185791, 2022-06-29T15:50:33.196 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_snapshot": "", 2022-06-29T15:50:33.196 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_max": 20, 2022-06-29T15:50:33.196 INFO:teuthology.orchestra.run.smithi047.stdout: "max_osd": 8, 2022-06-29T15:50:33.196 INFO:teuthology.orchestra.run.smithi047.stdout: "require_min_compat_client": "jewel", 2022-06-29T15:50:33.197 INFO:teuthology.orchestra.run.smithi047.stdout: "min_compat_client": "jewel", 2022-06-29T15:50:33.197 INFO:teuthology.orchestra.run.smithi047.stdout: "require_osd_release": "octopus", 2022-06-29T15:50:33.197 INFO:teuthology.orchestra.run.smithi047.stdout: "pools": [ 2022-06-29T15:50:33.197 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.198 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 1, 2022-06-29T15:50:33.198 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "device_health_metrics", 2022-06-29T15:50:33.198 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:42:16.971431+0000", 2022-06-29T15:50:33.198 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:50:33.199 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:50:33.199 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:50:33.199 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:50:33.199 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:50:33.200 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:50:33.200 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:50:33.200 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:50:33.200 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 1, 2022-06-29T15:50:33.200 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 1, 2022-06-29T15:50:33.201 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 1, 2022-06-29T15:50:33.201 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 1, 2022-06-29T15:50:33.201 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 1, 2022-06-29T15:50:33.201 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:50:33.202 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:50:33.202 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:50:33.202 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:50:33.202 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:50:33.202 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:50:33.203 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:50:33.203 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.203 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "16", 2022-06-29T15:50:33.203 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:50:33.204 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:50:33.204 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:50:33.204 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:50:33.204 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:50:33.205 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:50:33.205 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:50:33.205 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:50:33.205 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:50:33.205 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:50:33.206 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:50:33.206 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:50:33.206 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:50:33.206 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:50:33.207 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:50:33.207 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:50:33.207 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:50:33.207 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:50:33.208 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:50:33.208 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:50:33.208 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:50:33.208 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:50:33.209 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:50:33.209 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:50:33.209 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:50:33.209 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:50:33.209 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.210 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:50:33.210 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:50:33.210 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:50:33.211 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:50:33.211 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:50:33.211 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:50:33.212 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:50:33.212 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:50:33.212 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:50:33.212 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:50:33.213 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:50:33.213 INFO:teuthology.orchestra.run.smithi047.stdout: "options": { 2022-06-29T15:50:33.213 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_min": 1 2022-06-29T15:50:33.214 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.214 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:50:33.214 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_devicehealth": {} 2022-06-29T15:50:33.214 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:33.215 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.215 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.215 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 2, 2022-06-29T15:50:33.215 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "rbd", 2022-06-29T15:50:33.216 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:42:25.290884+0000", 2022-06-29T15:50:33.216 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:50:33.216 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:50:33.216 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:50:33.216 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:50:33.217 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:50:33.217 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:50:33.217 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:50:33.217 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:50:33.218 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2022-06-29T15:50:33.218 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2022-06-29T15:50:33.218 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2022-06-29T15:50:33.218 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2022-06-29T15:50:33.219 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2022-06-29T15:50:33.219 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:50:33.219 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:50:33.219 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:50:33.219 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:50:33.220 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:50:33.220 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:50:33.220 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:50:33.221 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.221 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "19", 2022-06-29T15:50:33.221 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:50:33.221 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:50:33.221 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:50:33.222 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:50:33.222 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:50:33.222 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:50:33.222 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:50:33.223 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:50:33.223 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:50:33.223 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:50:33.223 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:50:33.223 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:50:33.224 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:50:33.224 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:50:33.224 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:50:33.224 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:50:33.225 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:50:33.225 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:50:33.225 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:50:33.225 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:50:33.225 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:50:33.226 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:50:33.226 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:50:33.226 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:50:33.226 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:50:33.227 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:50:33.227 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.227 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:50:33.227 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:50:33.228 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:50:33.228 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:50:33.228 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:50:33.228 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:50:33.229 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:50:33.229 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:50:33.229 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:50:33.229 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:50:33.229 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:50:33.230 INFO:teuthology.orchestra.run.smithi047.stdout: "options": {}, 2022-06-29T15:50:33.230 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:50:33.230 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd": {} 2022-06-29T15:50:33.230 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:33.231 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:33.231 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:50:33.231 INFO:teuthology.orchestra.run.smithi047.stdout: "osds": [ 2022-06-29T15:50:33.231 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.232 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2022-06-29T15:50:33.232 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "e23eb023-4706-4201-bf72-4bf19c912c5d", 2022-06-29T15:50:33.232 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:50:33.233 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:50:33.233 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:50:33.233 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:50:33.233 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:50:33.234 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:50:33.234 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:50:33.234 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 109, 2022-06-29T15:50:33.234 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:50:33.235 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:50:33.235 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:50:33.235 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:33.235 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.236 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:33.236 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6818", 2022-06-29T15:50:33.236 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:50:33.237 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.237 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.237 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:33.238 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6819", 2022-06-29T15:50:33.238 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:50:33.239 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:33.239 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:33.240 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.240 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:50:33.241 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:33.241 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.241 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:33.242 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6820", 2022-06-29T15:50:33.242 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:50:33.243 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.243 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.243 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:33.244 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6821", 2022-06-29T15:50:33.244 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:50:33.244 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:33.245 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:33.245 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.245 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:50:33.246 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:33.246 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.247 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:33.247 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6824", 2022-06-29T15:50:33.247 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:50:33.247 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.248 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.248 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:33.248 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6825", 2022-06-29T15:50:33.248 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:50:33.249 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:33.249 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:33.249 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.249 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:50:33.250 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:33.250 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.250 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:33.250 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6822", 2022-06-29T15:50:33.250 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:50:33.251 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.251 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.251 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:33.251 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6823", 2022-06-29T15:50:33.252 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:50:33.252 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:33.252 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:33.252 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.252 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6819/13096", 2022-06-29T15:50:33.253 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6821/13096", 2022-06-29T15:50:33.253 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6825/13096", 2022-06-29T15:50:33.253 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6823/13096", 2022-06-29T15:50:33.253 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:50:33.254 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:50:33.254 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:50:33.254 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:33.254 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.254 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.255 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2022-06-29T15:50:33.255 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "228bb08d-2d64-43d9-9560-bfaeff154452", 2022-06-29T15:50:33.255 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:50:33.255 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:50:33.256 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:50:33.256 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:50:33.256 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:50:33.256 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:50:33.257 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:50:33.257 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 109, 2022-06-29T15:50:33.258 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:50:33.258 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:50:33.259 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:50:33.259 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:33.259 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.260 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:33.260 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6826", 2022-06-29T15:50:33.261 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:50:33.261 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.262 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.262 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:33.263 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6827", 2022-06-29T15:50:33.263 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:50:33.264 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:33.264 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:33.264 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.265 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:50:33.265 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:33.266 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.266 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:33.267 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6828", 2022-06-29T15:50:33.267 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:50:33.267 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.268 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.268 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:33.269 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6829", 2022-06-29T15:50:33.269 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:50:33.269 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:33.270 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:33.270 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.271 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:50:33.271 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:33.272 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.272 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:33.272 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6832", 2022-06-29T15:50:33.273 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:50:33.273 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.274 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.274 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:33.274 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6833", 2022-06-29T15:50:33.275 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:50:33.275 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:33.276 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:33.276 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.276 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:50:33.277 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:33.277 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.277 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:33.278 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6830", 2022-06-29T15:50:33.278 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:50:33.279 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.279 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.279 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:33.280 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6831", 2022-06-29T15:50:33.280 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:50:33.280 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:33.281 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:33.281 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.282 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6827/13100", 2022-06-29T15:50:33.282 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6829/13100", 2022-06-29T15:50:33.283 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6833/13100", 2022-06-29T15:50:33.283 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6831/13100", 2022-06-29T15:50:33.283 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:50:33.284 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:50:33.284 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:50:33.285 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:33.285 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.286 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.286 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2022-06-29T15:50:33.286 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "797a5452-1510-4640-a111-c9c2fcf707eb", 2022-06-29T15:50:33.287 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:50:33.287 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:50:33.288 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:50:33.288 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:50:33.289 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:50:33.289 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:50:33.289 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:50:33.290 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 108, 2022-06-29T15:50:33.290 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:50:33.291 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:50:33.291 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:50:33.291 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:33.292 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.292 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:33.292 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6810", 2022-06-29T15:50:33.293 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:50:33.293 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.294 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.294 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:33.295 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6811", 2022-06-29T15:50:33.295 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:50:33.296 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:33.296 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:33.296 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.297 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:50:33.297 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:33.297 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.297 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:33.298 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6812", 2022-06-29T15:50:33.298 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:50:33.298 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.298 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.299 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:33.299 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6813", 2022-06-29T15:50:33.299 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:50:33.299 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:33.300 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:33.300 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.300 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:50:33.300 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:33.300 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.301 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:33.301 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6816", 2022-06-29T15:50:33.301 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:50:33.301 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.302 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.302 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:33.302 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6817", 2022-06-29T15:50:33.302 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:50:33.302 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:33.303 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:33.303 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.303 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:50:33.303 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:33.304 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.304 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:33.304 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6814", 2022-06-29T15:50:33.304 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:50:33.304 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.305 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.305 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:33.305 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6815", 2022-06-29T15:50:33.305 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:50:33.306 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:33.306 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:33.306 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.306 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6811/13098", 2022-06-29T15:50:33.307 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6813/13098", 2022-06-29T15:50:33.307 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6817/13098", 2022-06-29T15:50:33.307 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6815/13098", 2022-06-29T15:50:33.308 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:50:33.308 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:50:33.308 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:50:33.308 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:33.309 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.309 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.309 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2022-06-29T15:50:33.309 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "55b24fd7-bb71-4a01-b3fa-a09caa772a62", 2022-06-29T15:50:33.309 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:50:33.310 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:50:33.310 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:50:33.310 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:50:33.310 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:50:33.311 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:50:33.311 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:50:33.311 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 109, 2022-06-29T15:50:33.311 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:50:33.312 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:50:33.312 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:50:33.312 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:33.312 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.312 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:33.313 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6802", 2022-06-29T15:50:33.313 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:50:33.313 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.313 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.314 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:33.314 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6803", 2022-06-29T15:50:33.314 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:50:33.314 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:33.315 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:33.315 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.315 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:50:33.315 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:33.316 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.316 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:33.316 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6804", 2022-06-29T15:50:33.316 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:50:33.316 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.317 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.317 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:33.317 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6805", 2022-06-29T15:50:33.317 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:50:33.318 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:33.318 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:33.318 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.319 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:50:33.319 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:33.319 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.320 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:33.320 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6808", 2022-06-29T15:50:33.320 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:50:33.321 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.321 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.321 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:33.321 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6809", 2022-06-29T15:50:33.322 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:50:33.322 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:33.322 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:33.322 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.323 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:50:33.323 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:33.323 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.323 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:33.324 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6806", 2022-06-29T15:50:33.324 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:50:33.324 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.324 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.325 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:33.325 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6807", 2022-06-29T15:50:33.325 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:50:33.325 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:33.326 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:33.326 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.326 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6803/13097", 2022-06-29T15:50:33.326 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6805/13097", 2022-06-29T15:50:33.327 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6809/13097", 2022-06-29T15:50:33.327 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6807/13097", 2022-06-29T15:50:33.327 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:50:33.327 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:50:33.327 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:50:33.328 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:33.328 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.328 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.329 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2022-06-29T15:50:33.329 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3", 2022-06-29T15:50:33.329 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:50:33.330 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:50:33.330 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:50:33.330 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:50:33.331 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:50:33.331 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:50:33.331 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:50:33.332 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 108, 2022-06-29T15:50:33.332 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:50:33.332 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:50:33.333 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:50:33.333 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:33.333 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.333 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:33.334 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6816", 2022-06-29T15:50:33.334 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:50:33.334 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.334 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.335 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:33.335 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6817", 2022-06-29T15:50:33.335 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:50:33.335 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:33.336 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:33.336 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.336 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:50:33.336 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:33.336 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.337 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:33.337 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6818", 2022-06-29T15:50:33.337 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:50:33.337 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.338 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.338 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:33.338 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6819", 2022-06-29T15:50:33.338 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:50:33.339 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:33.339 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:33.339 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.339 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:50:33.339 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:33.340 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.340 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:33.340 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6822", 2022-06-29T15:50:33.340 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:50:33.341 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.341 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.341 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:33.341 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6823", 2022-06-29T15:50:33.341 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:50:33.342 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:33.342 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:33.342 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.342 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:50:33.343 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:33.343 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.343 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:33.343 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6820", 2022-06-29T15:50:33.344 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:50:33.344 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.344 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.344 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:33.344 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6821", 2022-06-29T15:50:33.345 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:50:33.345 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:33.345 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:33.345 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.345 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6817/13396", 2022-06-29T15:50:33.346 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6819/13396", 2022-06-29T15:50:33.346 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6823/13396", 2022-06-29T15:50:33.346 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6821/13396", 2022-06-29T15:50:33.346 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:50:33.347 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:50:33.347 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:50:33.347 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:33.347 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.348 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.348 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2022-06-29T15:50:33.348 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "f77beebe-6cca-4244-b2d4-677825ebf919", 2022-06-29T15:50:33.348 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:50:33.348 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:50:33.349 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:50:33.349 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:50:33.349 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:50:33.349 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:50:33.350 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:50:33.350 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 108, 2022-06-29T15:50:33.350 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:50:33.350 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:50:33.350 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:50:33.351 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:33.351 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.351 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:33.351 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6800", 2022-06-29T15:50:33.352 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:50:33.352 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.352 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.352 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:33.353 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6801", 2022-06-29T15:50:33.353 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:50:33.353 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:33.353 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:33.353 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.354 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:50:33.354 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:33.354 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.354 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:33.355 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6802", 2022-06-29T15:50:33.355 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:50:33.355 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.355 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.356 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:33.356 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6803", 2022-06-29T15:50:33.356 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:50:33.356 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:33.357 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:33.357 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.357 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:50:33.357 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:33.357 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.358 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:33.358 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6806", 2022-06-29T15:50:33.358 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:50:33.358 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.359 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.359 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:33.359 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6807", 2022-06-29T15:50:33.359 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:50:33.359 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:33.360 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:33.360 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.360 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:50:33.360 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:33.361 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.361 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:33.362 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6804", 2022-06-29T15:50:33.362 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:50:33.362 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.363 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.363 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:33.363 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6805", 2022-06-29T15:50:33.363 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:50:33.364 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:33.364 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:33.364 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.364 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6801/13393", 2022-06-29T15:50:33.364 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6803/13393", 2022-06-29T15:50:33.365 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6807/13393", 2022-06-29T15:50:33.365 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6805/13393", 2022-06-29T15:50:33.365 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:50:33.365 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:50:33.366 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:50:33.366 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:33.366 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.366 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.367 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2022-06-29T15:50:33.367 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "af0ca9b3-c02c-4910-834b-bf0b8dc6713c", 2022-06-29T15:50:33.367 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:50:33.367 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:50:33.367 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:50:33.368 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:50:33.368 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:50:33.368 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:50:33.368 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:50:33.369 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 109, 2022-06-29T15:50:33.369 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:50:33.369 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:50:33.369 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:50:33.369 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:33.370 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.370 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:33.370 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6824", 2022-06-29T15:50:33.370 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:50:33.371 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.371 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.371 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:33.371 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6825", 2022-06-29T15:50:33.371 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:50:33.372 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:33.372 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:33.372 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.372 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:50:33.373 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:33.373 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.373 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:33.373 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6826", 2022-06-29T15:50:33.373 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:50:33.374 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.374 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.374 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:33.374 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6827", 2022-06-29T15:50:33.375 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:50:33.375 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:33.375 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:33.375 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.376 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:50:33.376 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:33.376 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.376 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:33.377 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6830", 2022-06-29T15:50:33.377 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:50:33.377 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.377 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.377 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:33.378 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6831", 2022-06-29T15:50:33.378 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:50:33.378 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:33.378 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:33.379 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.379 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:50:33.379 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:33.379 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.380 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:33.380 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6828", 2022-06-29T15:50:33.380 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:50:33.380 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.380 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.381 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:33.381 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6829", 2022-06-29T15:50:33.381 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:50:33.381 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:33.382 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:33.382 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.382 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6825/13395", 2022-06-29T15:50:33.382 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6827/13395", 2022-06-29T15:50:33.382 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6831/13395", 2022-06-29T15:50:33.383 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6829/13395", 2022-06-29T15:50:33.383 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:50:33.383 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:50:33.383 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:50:33.384 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:33.384 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.384 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.384 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2022-06-29T15:50:33.384 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "f19c3ca6-df88-4bde-a150-b51a53f092e1", 2022-06-29T15:50:33.385 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:50:33.385 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:50:33.385 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:50:33.385 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:50:33.386 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:50:33.386 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:50:33.386 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:50:33.386 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 109, 2022-06-29T15:50:33.387 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:50:33.387 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:50:33.388 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:50:33.388 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:33.388 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.389 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:33.389 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6808", 2022-06-29T15:50:33.390 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:50:33.390 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.391 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.391 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:33.392 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6809", 2022-06-29T15:50:33.392 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:50:33.392 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:33.393 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:33.393 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.393 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:50:33.393 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:33.394 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.394 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:33.394 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6810", 2022-06-29T15:50:33.394 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:50:33.395 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.395 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.395 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:33.395 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6811", 2022-06-29T15:50:33.395 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:50:33.396 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:33.396 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:33.396 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.396 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:50:33.397 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:33.397 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.397 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:33.397 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6814", 2022-06-29T15:50:33.397 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:50:33.398 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.398 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.398 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:33.398 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6815", 2022-06-29T15:50:33.399 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:50:33.399 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:33.399 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:33.399 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.399 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:50:33.400 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:50:33.400 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.400 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:50:33.400 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6812", 2022-06-29T15:50:33.401 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:50:33.401 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.401 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.401 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:50:33.401 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6813", 2022-06-29T15:50:33.402 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:50:33.402 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:33.402 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:33.402 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.403 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6809/13394", 2022-06-29T15:50:33.403 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6811/13394", 2022-06-29T15:50:33.403 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6815/13394", 2022-06-29T15:50:33.403 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6813/13394", 2022-06-29T15:50:33.404 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:50:33.404 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:50:33.404 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:50:33.404 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:33.405 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:33.405 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:50:33.405 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_xinfo": [ 2022-06-29T15:50:33.405 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.405 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2022-06-29T15:50:33.406 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:50:33.406 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:50:33.406 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:50:33.406 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:50:33.407 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:50:33.407 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.720513+0000", 2022-06-29T15:50:33.407 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:50:33.407 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.407 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.408 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2022-06-29T15:50:33.408 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:50:33.408 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:50:33.408 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:50:33.409 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:50:33.409 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:50:33.409 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.772612+0000", 2022-06-29T15:50:33.409 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:50:33.410 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.410 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.410 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2022-06-29T15:50:33.410 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:50:33.411 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:50:33.411 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:50:33.411 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:50:33.411 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:50:33.411 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.450223+0000", 2022-06-29T15:50:33.412 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:50:33.412 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.412 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.412 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2022-06-29T15:50:33.413 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:50:33.413 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:50:33.413 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:50:33.413 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:50:33.413 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:50:33.414 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.477856+0000", 2022-06-29T15:50:33.414 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:50:33.414 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.415 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.415 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2022-06-29T15:50:33.415 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:50:33.415 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:50:33.415 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:50:33.416 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:50:33.416 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:50:33.416 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.539485+0000", 2022-06-29T15:50:33.416 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:50:33.417 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.417 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.417 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2022-06-29T15:50:33.417 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:50:33.417 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:50:33.418 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:50:33.418 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:50:33.418 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:50:33.418 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.487040+0000", 2022-06-29T15:50:33.419 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:50:33.419 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.419 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.419 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2022-06-29T15:50:33.419 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:50:33.420 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:50:33.420 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:50:33.420 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:50:33.420 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:50:33.421 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.748685+0000", 2022-06-29T15:50:33.421 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:50:33.421 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.421 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:33.422 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2022-06-29T15:50:33.422 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:50:33.422 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:50:33.422 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:50:33.423 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:50:33.423 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:50:33.423 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.459516+0000", 2022-06-29T15:50:33.423 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:50:33.423 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:33.424 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:50:33.424 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap": [], 2022-06-29T15:50:33.424 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap_items": [], 2022-06-29T15:50:33.425 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_temp": [], 2022-06-29T15:50:33.425 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_temp": [], 2022-06-29T15:50:33.426 INFO:teuthology.orchestra.run.smithi047.stdout: "blacklist": { 2022-06-29T15:50:33.426 INFO:teuthology.orchestra.run.smithi047.stdout: "172.21.15.64:6833/2706285595": "2022-06-30T15:50:26.778120+0000", 2022-06-29T15:50:33.427 INFO:teuthology.orchestra.run.smithi047.stdout: "172.21.15.64:6832/2706285595": "2022-06-30T15:50:26.778120+0000" 2022-06-29T15:50:33.427 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.427 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profiles": { 2022-06-29T15:50:33.428 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs_data": { 2022-06-29T15:50:33.428 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-device-class": "", 2022-06-29T15:50:33.429 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2022-06-29T15:50:33.429 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-root": "default", 2022-06-29T15:50:33.429 INFO:teuthology.orchestra.run.smithi047.stdout: "jerasure-per-chunk-alignment": "false", 2022-06-29T15:50:33.430 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2022-06-29T15:50:33.430 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "2", 2022-06-29T15:50:33.430 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2022-06-29T15:50:33.431 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2022-06-29T15:50:33.431 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van", 2022-06-29T15:50:33.431 INFO:teuthology.orchestra.run.smithi047.stdout: "w": "8" 2022-06-29T15:50:33.432 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.432 INFO:teuthology.orchestra.run.smithi047.stdout: "default": { 2022-06-29T15:50:33.432 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2022-06-29T15:50:33.432 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2022-06-29T15:50:33.433 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "1", 2022-06-29T15:50:33.433 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2022-06-29T15:50:33.433 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2022-06-29T15:50:33.433 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van" 2022-06-29T15:50:33.434 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:33.434 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:33.434 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps_queue": [], 2022-06-29T15:50:33.434 INFO:teuthology.orchestra.run.smithi047.stdout: "new_removed_snaps": [], 2022-06-29T15:50:33.435 INFO:teuthology.orchestra.run.smithi047.stdout: "new_purged_snaps": [], 2022-06-29T15:50:33.435 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_node_flags": {}, 2022-06-29T15:50:33.435 INFO:teuthology.orchestra.run.smithi047.stdout: "device_class_flags": {} 2022-06-29T15:50:33.435 INFO:teuthology.orchestra.run.smithi047.stdout:} 2022-06-29T15:50:33.437 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.187+0000 7f9c4d374700 1 -- 172.21.15.47:0/87307524 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9c28053b00 msgr2=0x7f9c28055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:33.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.187+0000 7f9c4d374700 1 --2- 172.21.15.47:0/87307524 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9c28053b00 0x7f9c28055fc0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7f9c3c01a000 tx=0x7f9c3c014000).stop 2022-06-29T15:50:33.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.187+0000 7f9c4d374700 1 -- 172.21.15.47:0/87307524 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9c480ad730 msgr2=0x7f9c48145550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:33.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.187+0000 7f9c4d374700 1 --2- 172.21.15.47:0/87307524 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9c480ad730 0x7f9c48145550 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f9c48147000 tx=0x7f9c38007aa0).stop 2022-06-29T15:50:33.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.187+0000 7f9c4d374700 1 -- 172.21.15.47:0/87307524 shutdown_connections 2022-06-29T15:50:33.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.187+0000 7f9c4d374700 1 --2- 172.21.15.47:0/87307524 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c4814b530 0x7f9c4813dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:33.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.187+0000 7f9c4d374700 1 --2- 172.21.15.47:0/87307524 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9c28053b00 0x7f9c28055fc0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:33.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.187+0000 7f9c4d374700 1 --2- 172.21.15.47:0/87307524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c48147370 0x7f9c4813d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:33.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.187+0000 7f9c4d374700 1 --2- 172.21.15.47:0/87307524 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9c480ad730 0x7f9c48145550 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:33.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.187+0000 7f9c4d374700 1 -- 172.21.15.47:0/87307524 >> 172.21.15.47:0/87307524 conn(0x7f9c48062df0 msgr2=0x7f9c480ae430 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:33.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.187+0000 7f9c4d374700 1 -- 172.21.15.47:0/87307524 shutdown_connections 2022-06-29T15:50:33.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.187+0000 7f9c4d374700 1 -- 172.21.15.47:0/87307524 wait complete. 2022-06-29T15:50:33.441 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:50:33.553 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.547+0000 7fab38c34700 1 Processor -- start 2022-06-29T15:50:33.553 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.547+0000 7fab38c34700 1 -- start start 2022-06-29T15:50:33.557 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.547+0000 7fab38c34700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fab340b0cf0 0x7fab34146d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:33.557 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.547+0000 7fab38c34700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fab340b06d0 0x7fab3413ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:33.557 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.547+0000 7fab38c34700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab340b4520 con 0x7fab340611b0 2022-06-29T15:50:33.558 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.547+0000 7fab38c34700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab340b4b10 con 0x7fab340b0cf0 2022-06-29T15:50:33.558 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.547+0000 7fab38c34700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab340621d0 con 0x7fab340b06d0 2022-06-29T15:50:33.558 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.547+0000 7fab3259c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fab340611b0 0x7fab34146640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38108/0 (socket says 172.21.15.47:38108) 2022-06-29T15:50:33.558 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.547+0000 7fab3259c700 1 -- 172.21.15.47:0/2065263397 learned_addr learned my addr 172.21.15.47:0/2065263397 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:33.559 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.547+0000 7fab32d9d700 1 --2- 172.21.15.47:0/2065263397 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fab340b06d0 0x7fab3413ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:33.559 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.547+0000 7fab31d9b700 1 --2- 172.21.15.47:0/2065263397 >> v2:172.21.15.64:3300/0 conn(0x7fab340b0cf0 0x7fab34146d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:33.559 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.547+0000 7fab3159a700 1 -- 172.21.15.47:0/2065263397 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3705594606 0 0) 0x7fab340b4520 con 0x7fab340611b0 2022-06-29T15:50:33.559 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.547+0000 7fab3159a700 1 -- 172.21.15.47:0/2065263397 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab14003260 con 0x7fab340611b0 2022-06-29T15:50:33.560 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.547+0000 7fab3159a700 1 -- 172.21.15.47:0/2065263397 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1567365761 0 0) 0x7fab14003260 con 0x7fab340611b0 2022-06-29T15:50:33.560 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.547+0000 7fab3159a700 1 -- 172.21.15.47:0/2065263397 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fab340b06d0 msgr2=0x7fab3413ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:33.560 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.547+0000 7fab3159a700 1 --2- 172.21.15.47:0/2065263397 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fab340b06d0 0x7fab3413ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:33.560 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.547+0000 7fab3159a700 1 -- 172.21.15.47:0/2065263397 >> v2:172.21.15.64:3300/0 conn(0x7fab340b0cf0 msgr2=0x7fab34146d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:33.561 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.547+0000 7fab3159a700 1 --2- 172.21.15.47:0/2065263397 >> v2:172.21.15.64:3300/0 conn(0x7fab340b0cf0 0x7fab34146d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:33.561 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.547+0000 7fab3159a700 1 -- 172.21.15.47:0/2065263397 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab34061590 con 0x7fab340611b0 2022-06-29T15:50:33.561 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.547+0000 7fab3159a700 1 -- 172.21.15.47:0/2065263397 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7fab240019e0 con 0x7fab340611b0 2022-06-29T15:50:33.561 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.547+0000 7fab3159a700 1 -- 172.21.15.47:0/2065263397 >> v1:172.21.15.47:6789/0 conn(0x7fab340611b0 legacy=0x7fab34146640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:33.562 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.547+0000 7fab3159a700 1 --2- 172.21.15.47:0/2065263397 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fab340b06d0 0x7fab14003780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:33.562 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.547+0000 7fab3159a700 1 --2- 172.21.15.47:0/2065263397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab14003cb0 0x7fab14004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:33.562 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.547+0000 7fab3159a700 1 --2- 172.21.15.47:0/2065263397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab140049f0 0x7fab14004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:33.562 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.547+0000 7fab3159a700 1 -- 172.21.15.47:0/2065263397 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab340b4b10 con 0x7fab140049f0 2022-06-29T15:50:33.563 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.547+0000 7fab3159a700 1 -- 172.21.15.47:0/2065263397 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab340621d0 con 0x7fab14003cb0 2022-06-29T15:50:33.563 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.547+0000 7fab3159a700 1 -- 172.21.15.47:0/2065263397 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab1400b750 con 0x7fab340b06d0 2022-06-29T15:50:33.563 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.547+0000 7fab32d9d700 1 --2- 172.21.15.47:0/2065263397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab140049f0 0x7fab14004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:33.563 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.547+0000 7fab3259c700 1 --2- 172.21.15.47:0/2065263397 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fab340b06d0 0x7fab14003780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:33.563 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.547+0000 7fab31d9b700 1 --2- 172.21.15.47:0/2065263397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab14003cb0 0x7fab14004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:33.564 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.551+0000 7fab32d9d700 1 -- 172.21.15.47:0/2065263397 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fab340b06d0 msgr2=0x7fab14003780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:33.564 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.551+0000 7fab32d9d700 1 --2- 172.21.15.47:0/2065263397 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fab340b06d0 0x7fab14003780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:33.564 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.551+0000 7fab32d9d700 1 -- 172.21.15.47:0/2065263397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab14003cb0 msgr2=0x7fab14004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:33.564 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.551+0000 7fab32d9d700 1 --2- 172.21.15.47:0/2065263397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab14003cb0 0x7fab14004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:33.565 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.551+0000 7fab32d9d700 1 -- 172.21.15.47:0/2065263397 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab14003260 con 0x7fab140049f0 2022-06-29T15:50:33.565 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.551+0000 7fab32d9d700 1 --2- 172.21.15.47:0/2065263397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab140049f0 0x7fab14004e70 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 rx=0x7fab14004470 tx=0x7fab2c006df0).ready entity=mon.0 client_cookie=3018696164fc0f51 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:33.565 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.551+0000 7fab3159a700 1 -- 172.21.15.47:0/2065263397 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab2c00e800 con 0x7fab140049f0 2022-06-29T15:50:33.565 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.551+0000 7fab3159a700 1 -- 172.21.15.47:0/2065263397 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab2c00e960 con 0x7fab140049f0 2022-06-29T15:50:33.566 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.551+0000 7fab3159a700 1 -- 172.21.15.47:0/2065263397 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab2c00ecc0 con 0x7fab140049f0 2022-06-29T15:50:33.566 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.551+0000 7fab38c34700 1 -- 172.21.15.47:0/2065263397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab140049f0 msgr2=0x7fab14004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:33.566 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.551+0000 7fab38c34700 1 --2- 172.21.15.47:0/2065263397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab140049f0 0x7fab14004e70 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 rx=0x7fab14004470 tx=0x7fab2c006df0).stop 2022-06-29T15:50:33.566 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.551+0000 7fab32d9d700 1 -- 172.21.15.47:0/2065263397 reap_dead start 2022-06-29T15:50:33.567 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.551+0000 7fab38c34700 1 -- 172.21.15.47:0/2065263397 shutdown_connections 2022-06-29T15:50:33.567 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.551+0000 7fab38c34700 1 -- 172.21.15.47:0/2065263397 >> 172.21.15.47:0/2065263397 conn(0x7fab340b98b0 msgr2=0x7fab340b9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:33.567 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.551+0000 7fab38c34700 1 -- 172.21.15.47:0/2065263397 shutdown_connections 2022-06-29T15:50:33.567 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.551+0000 7fab38c34700 1 -- 172.21.15.47:0/2065263397 wait complete. 2022-06-29T15:50:33.571 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.551+0000 7fab38c34700 1 Processor -- start 2022-06-29T15:50:33.571 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.551+0000 7fab38c34700 1 -- start start 2022-06-29T15:50:33.571 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.551+0000 7fab38c34700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab140049f0 0x7fab340b6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:33.572 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.551+0000 7fab38c34700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab341361e0 0x7fab3413c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:33.572 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.551+0000 7fab38c34700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fab340adff0 0x7fab34135330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:33.572 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.551+0000 7fab38c34700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab34137e00 con 0x7fab140049f0 2022-06-29T15:50:33.572 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.551+0000 7fab38c34700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3413d560 con 0x7fab341361e0 2022-06-29T15:50:33.573 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.551+0000 7fab38c34700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab3413d2f0 con 0x7fab340adff0 2022-06-29T15:50:33.573 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.551+0000 7fab3259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab140049f0 0x7fab340b6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:33.573 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.551+0000 7fab3259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab140049f0 0x7fab340b6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52566/0 (socket says 172.21.15.47:52566) 2022-06-29T15:50:33.573 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.551+0000 7fab3259c700 1 -- 172.21.15.47:0/1758341212 learned_addr learned my addr 172.21.15.47:0/1758341212 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:33.574 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.555+0000 7fab32d9d700 1 --2- 172.21.15.47:0/1758341212 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fab340adff0 0x7fab34135330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:33.574 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.555+0000 7fab31d9b700 1 --2- 172.21.15.47:0/1758341212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab341361e0 0x7fab3413c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:33.574 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.555+0000 7fab3259c700 1 -- 172.21.15.47:0/1758341212 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fab340adff0 msgr2=0x7fab34135330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:33.574 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.555+0000 7fab3259c700 1 --2- 172.21.15.47:0/1758341212 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fab340adff0 0x7fab34135330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:33.575 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.555+0000 7fab3259c700 1 -- 172.21.15.47:0/1758341212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab341361e0 msgr2=0x7fab3413c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:33.575 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.555+0000 7fab3259c700 1 --2- 172.21.15.47:0/1758341212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab341361e0 0x7fab3413c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:33.575 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.555+0000 7fab3259c700 1 -- 172.21.15.47:0/1758341212 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab341371f0 con 0x7fab140049f0 2022-06-29T15:50:33.575 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.555+0000 7fab31d9b700 1 --2- 172.21.15.47:0/1758341212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab341361e0 0x7fab3413c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:50:33.575 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.555+0000 7fab3259c700 1 --2- 172.21.15.47:0/1758341212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab140049f0 0x7fab340b6b80 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 rx=0x7fab3413d6a0 tx=0x7fab24002190).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:33.576 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.555+0000 7fab2b7fe700 1 -- 172.21.15.47:0/1758341212 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab2400fa60 con 0x7fab140049f0 2022-06-29T15:50:33.576 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.555+0000 7fab2b7fe700 1 -- 172.21.15.47:0/1758341212 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab2400fbc0 con 0x7fab140049f0 2022-06-29T15:50:33.576 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.555+0000 7fab38c34700 1 -- 172.21.15.47:0/1758341212 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab34137490 con 0x7fab140049f0 2022-06-29T15:50:33.576 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.555+0000 7fab2b7fe700 1 -- 172.21.15.47:0/1758341212 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab24012350 con 0x7fab140049f0 2022-06-29T15:50:33.577 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.555+0000 7fab38c34700 1 -- 172.21.15.47:0/1758341212 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab3413b970 con 0x7fab140049f0 2022-06-29T15:50:33.577 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.555+0000 7fab38c34700 1 -- 172.21.15.47:0/1758341212 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab340573d0 con 0x7fab140049f0 2022-06-29T15:50:33.577 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.555+0000 7fab2b7fe700 1 -- 172.21.15.47:0/1758341212 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fab2401b030 con 0x7fab140049f0 2022-06-29T15:50:33.577 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.555+0000 7fab2b7fe700 1 --2- 172.21.15.47:0/1758341212 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fab18053a70 0x7fab18055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:33.578 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.555+0000 7fab2b7fe700 1 -- 172.21.15.47:0/1758341212 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fab2406bce0 con 0x7fab140049f0 2022-06-29T15:50:33.578 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.555+0000 7fab31d9b700 1 --2- 172.21.15.47:0/1758341212 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fab18053a70 0x7fab18055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:33.578 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.559+0000 7fab31d9b700 1 --2- 172.21.15.47:0/1758341212 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fab18053a70 0x7fab18055f30 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7fab1c00b000 tx=0x7fab1c008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:33.578 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.563+0000 7fab2b7fe700 1 -- 172.21.15.47:0/1758341212 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fab24042ce0 con 0x7fab140049f0 2022-06-29T15:50:33.732 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.727+0000 7fab38c34700 1 -- 172.21.15.47:0/1758341212 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab34061720 con 0x7fab140049f0 2022-06-29T15:50:33.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.727+0000 7fab2b7fe700 1 -- 172.21.15.47:0/1758341212 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+1563 (secure 0 0 0) 0x7fab24019030 con 0x7fab140049f0 2022-06-29T15:50:33.734 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:50:33.734 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":53,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3752554010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3752554010},{"type":"v1","addr":"172.21.15.47:6835","nonce":3752554010}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":48},{"gid":7293,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.47:6837/3263659716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":3263659716},{"type":"v1","addr":"172.21.15.47:6837","nonce":3263659716}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":49},{"gid":56546,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2526504380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2526504380},{"type":"v1","addr":"172.21.15.64:6833","nonce":2526504380}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":53}],"filesystems":[]} 2022-06-29T15:50:33.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.727+0000 7fab38c34700 1 -- 172.21.15.47:0/1758341212 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fab18053a70 msgr2=0x7fab18055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:33.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.727+0000 7fab38c34700 1 --2- 172.21.15.47:0/1758341212 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fab18053a70 0x7fab18055f30 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7fab1c00b000 tx=0x7fab1c008040).stop 2022-06-29T15:50:33.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.727+0000 7fab38c34700 1 -- 172.21.15.47:0/1758341212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab140049f0 msgr2=0x7fab340b6b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:33.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.727+0000 7fab38c34700 1 --2- 172.21.15.47:0/1758341212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab140049f0 0x7fab340b6b80 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 rx=0x7fab3413d6a0 tx=0x7fab24002190).stop 2022-06-29T15:50:33.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.731+0000 7fab38c34700 1 -- 172.21.15.47:0/1758341212 shutdown_connections 2022-06-29T15:50:33.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.731+0000 7fab38c34700 1 --2- 172.21.15.47:0/1758341212 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fab340adff0 0x7fab34135330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:33.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.731+0000 7fab38c34700 1 --2- 172.21.15.47:0/1758341212 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fab18053a70 0x7fab18055f30 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:33.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.731+0000 7fab38c34700 1 --2- 172.21.15.47:0/1758341212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab341361e0 0x7fab3413c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:33.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.731+0000 7fab38c34700 1 --2- 172.21.15.47:0/1758341212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab140049f0 0x7fab340b6b80 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:33.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.731+0000 7fab38c34700 1 -- 172.21.15.47:0/1758341212 >> 172.21.15.47:0/1758341212 conn(0x7fab340b98b0 msgr2=0x7fab340b7a80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:33.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.731+0000 7fab38c34700 1 -- 172.21.15.47:0/1758341212 shutdown_connections 2022-06-29T15:50:33.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.731+0000 7fab38c34700 1 -- 172.21.15.47:0/1758341212 wait complete. 2022-06-29T15:50:33.739 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 53 2022-06-29T15:50:33.753 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:50:33.856 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.851+0000 7fb55a542700 1 Processor -- start 2022-06-29T15:50:33.856 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.851+0000 7fb55a542700 1 -- start start 2022-06-29T15:50:33.864 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.851+0000 7fb55a542700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb5540b5c60 0x7fb554144980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:33.864 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.851+0000 7fb55a542700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5540a9120 con 0x7fb5540a72f0 2022-06-29T15:50:33.865 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.851+0000 7fb55a542700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5540b0b80 con 0x7fb554060eb0 2022-06-29T15:50:33.865 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.851+0000 7fb55a542700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5540a89e0 con 0x7fb5540b5c60 2022-06-29T15:50:33.865 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.851+0000 7fb559d41700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb5540a72f0 0x7fb554151eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38132/0 (socket says 172.21.15.47:38132) 2022-06-29T15:50:33.866 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.851+0000 7fb559d41700 1 -- 172.21.15.47:0/729542543 learned_addr learned my addr 172.21.15.47:0/729542543 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:33.866 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.851+0000 7fb559540700 1 --2- 172.21.15.47:0/729542543 >> v2:172.21.15.64:3300/0 conn(0x7fb5540b5c60 0x7fb554144980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:33.866 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.851+0000 7fb54bfff700 1 -- 172.21.15.47:0/729542543 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2222775594 0 0) 0x7fb5540a9120 con 0x7fb5540a72f0 2022-06-29T15:50:33.867 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.851+0000 7fb54bfff700 1 -- 172.21.15.47:0/729542543 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb538003260 con 0x7fb5540a72f0 2022-06-29T15:50:33.867 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.851+0000 7fb54bfff700 1 -- 172.21.15.47:0/729542543 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3657435571 0 0) 0x7fb538003260 con 0x7fb5540a72f0 2022-06-29T15:50:33.867 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.851+0000 7fb54bfff700 1 -- 172.21.15.47:0/729542543 >> v2:172.21.15.64:3300/0 conn(0x7fb5540b5c60 msgr2=0x7fb554144980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:33.867 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.851+0000 7fb54bfff700 1 --2- 172.21.15.47:0/729542543 >> v2:172.21.15.64:3300/0 conn(0x7fb5540b5c60 0x7fb554144980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:33.868 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.851+0000 7fb54bfff700 1 -- 172.21.15.47:0/729542543 >> v1:172.21.15.64:6789/0 conn(0x7fb554060eb0 legacy=0x7fb5540b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:33.868 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.851+0000 7fb54bfff700 1 -- 172.21.15.47:0/729542543 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5540a8c80 con 0x7fb5540a72f0 2022-06-29T15:50:33.868 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.851+0000 7fb54bfff700 1 -- 172.21.15.47:0/729542543 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7fb54c0019e0 con 0x7fb5540a72f0 2022-06-29T15:50:33.868 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.851+0000 7fb54bfff700 1 -- 172.21.15.47:0/729542543 >> v1:172.21.15.47:6789/0 conn(0x7fb5540a72f0 legacy=0x7fb554151eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:33.869 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.851+0000 7fb54bfff700 1 --2- 172.21.15.47:0/729542543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb538003750 0x7fb538003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:33.869 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.851+0000 7fb54bfff700 1 --2- 172.21.15.47:0/729542543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb538004500 0x7fb538004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:33.869 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.851+0000 7fb54bfff700 1 --2- 172.21.15.47:0/729542543 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb538004e70 0x7fb53800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:33.869 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.851+0000 7fb54bfff700 1 -- 172.21.15.47:0/729542543 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5540a89e0 con 0x7fb538004500 2022-06-29T15:50:33.869 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.851+0000 7fb54bfff700 1 -- 172.21.15.47:0/729542543 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb53800bd20 con 0x7fb538003750 2022-06-29T15:50:33.870 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.851+0000 7fb54bfff700 1 -- 172.21.15.47:0/729542543 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb53800bee0 con 0x7fb538004e70 2022-06-29T15:50:33.870 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.851+0000 7fb558d3f700 1 --2- 172.21.15.47:0/729542543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb538004500 0x7fb538004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:33.870 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.851+0000 7fb559540700 1 --2- 172.21.15.47:0/729542543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb538003750 0x7fb538003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:33.870 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.851+0000 7fb559d41700 1 --2- 172.21.15.47:0/729542543 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb538004e70 0x7fb53800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:33.871 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.851+0000 7fb558d3f700 1 -- 172.21.15.47:0/729542543 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb538004e70 msgr2=0x7fb53800b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:33.871 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.851+0000 7fb558d3f700 1 --2- 172.21.15.47:0/729542543 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb538004e70 0x7fb53800b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:33.871 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.851+0000 7fb558d3f700 1 -- 172.21.15.47:0/729542543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb538003750 msgr2=0x7fb538003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:33.871 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.851+0000 7fb558d3f700 1 --2- 172.21.15.47:0/729542543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb538003750 0x7fb538003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:33.872 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.851+0000 7fb558d3f700 1 -- 172.21.15.47:0/729542543 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb538003260 con 0x7fb538004500 2022-06-29T15:50:33.872 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.851+0000 7fb559540700 1 --2- 172.21.15.47:0/729542543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb538003750 0x7fb538003b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:50:33.872 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.851+0000 7fb559d41700 1 --2- 172.21.15.47:0/729542543 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb538004e70 0x7fb53800b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:50:33.872 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.851+0000 7fb559d41700 1 -- 172.21.15.47:0/729542543 reap_dead start 2022-06-29T15:50:33.873 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.851+0000 7fb558d3f700 1 --2- 172.21.15.47:0/729542543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb538004500 0x7fb538004980 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 rx=0x7fb55406ac40 tx=0x7fb550007a00).ready entity=mon.0 client_cookie=9241864eceada167 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:33.873 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.851+0000 7fb54bfff700 1 -- 172.21.15.47:0/729542543 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb550007d10 con 0x7fb538004500 2022-06-29T15:50:33.874 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.851+0000 7fb54bfff700 1 -- 172.21.15.47:0/729542543 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb550007030 con 0x7fb538004500 2022-06-29T15:50:33.874 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.851+0000 7fb54bfff700 1 -- 172.21.15.47:0/729542543 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb550007390 con 0x7fb538004500 2022-06-29T15:50:33.875 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.855+0000 7fb55a542700 1 -- 172.21.15.47:0/729542543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb538004500 msgr2=0x7fb538004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:33.875 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.855+0000 7fb55a542700 1 --2- 172.21.15.47:0/729542543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb538004500 0x7fb538004980 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 rx=0x7fb55406ac40 tx=0x7fb550007a00).stop 2022-06-29T15:50:33.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.855+0000 7fb55a542700 1 -- 172.21.15.47:0/729542543 shutdown_connections 2022-06-29T15:50:33.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.855+0000 7fb55a542700 1 --2- 172.21.15.47:0/729542543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb538004500 0x7fb538004980 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:33.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.855+0000 7fb55a542700 1 -- 172.21.15.47:0/729542543 >> 172.21.15.47:0/729542543 conn(0x7fb554062560 msgr2=0x7fb554062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:33.877 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.855+0000 7fb55a542700 1 -- 172.21.15.47:0/729542543 shutdown_connections 2022-06-29T15:50:33.877 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.855+0000 7fb55a542700 1 -- 172.21.15.47:0/729542543 wait complete. 2022-06-29T15:50:33.877 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.855+0000 7fb55a542700 1 Processor -- start 2022-06-29T15:50:33.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.855+0000 7fb55a542700 1 -- start start 2022-06-29T15:50:33.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.855+0000 7fb55a542700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb538004500 0x7fb55414e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:33.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.855+0000 7fb55a542700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb554142e80 0x7fb55414d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:33.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.855+0000 7fb55a542700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb554156b20 0x7fb554156fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:33.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.855+0000 7fb55a542700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb554156570 con 0x7fb554156b20 2022-06-29T15:50:33.880 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.855+0000 7fb55a542700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55414da80 con 0x7fb554142e80 2022-06-29T15:50:33.880 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.855+0000 7fb55a542700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb554145f10 con 0x7fb538004500 2022-06-29T15:50:33.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.855+0000 7fb559d41700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb554156b20 0x7fb554156fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:33.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.855+0000 7fb559d41700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb554156b20 0x7fb554156fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52588/0 (socket says 172.21.15.47:52588) 2022-06-29T15:50:33.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.855+0000 7fb559d41700 1 -- 172.21.15.47:0/3131140645 learned_addr learned my addr 172.21.15.47:0/3131140645 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:33.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.855+0000 7fb558d3f700 1 --2- 172.21.15.47:0/3131140645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb554142e80 0x7fb55414d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:33.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.855+0000 7fb559540700 1 --2- 172.21.15.47:0/3131140645 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb538004500 0x7fb55414e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:33.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.855+0000 7fb559d41700 1 -- 172.21.15.47:0/3131140645 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb538004500 msgr2=0x7fb55414e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:33.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.855+0000 7fb559d41700 1 --2- 172.21.15.47:0/3131140645 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb538004500 0x7fb55414e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:33.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.855+0000 7fb559d41700 1 -- 172.21.15.47:0/3131140645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb554142e80 msgr2=0x7fb55414d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:33.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.855+0000 7fb559d41700 1 --2- 172.21.15.47:0/3131140645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb554142e80 0x7fb55414d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:33.883 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.855+0000 7fb559d41700 1 -- 172.21.15.47:0/3131140645 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb554142520 con 0x7fb554156b20 2022-06-29T15:50:33.883 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.855+0000 7fb559d41700 1 --2- 172.21.15.47:0/3131140645 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb554156b20 0x7fb554156fa0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 rx=0x7fb54c001c40 tx=0x7fb54c009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:33.883 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.855+0000 7fb54a7fc700 1 -- 172.21.15.47:0/3131140645 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb54c004ab0 con 0x7fb554156b20 2022-06-29T15:50:33.883 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.855+0000 7fb54a7fc700 1 -- 172.21.15.47:0/3131140645 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb54c004c10 con 0x7fb554156b20 2022-06-29T15:50:33.884 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.855+0000 7fb55a542700 1 -- 172.21.15.47:0/3131140645 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5541427c0 con 0x7fb554156b20 2022-06-29T15:50:33.884 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.855+0000 7fb54a7fc700 1 -- 172.21.15.47:0/3131140645 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb54c0070d0 con 0x7fb554156b20 2022-06-29T15:50:33.884 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.855+0000 7fb55a542700 1 -- 172.21.15.47:0/3131140645 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb554157490 con 0x7fb554156b20 2022-06-29T15:50:33.886 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.859+0000 7fb55a542700 1 -- 172.21.15.47:0/3131140645 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5540573d0 con 0x7fb554156b20 2022-06-29T15:50:33.886 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.859+0000 7fb54a7fc700 1 -- 172.21.15.47:0/3131140645 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb54c007230 con 0x7fb554156b20 2022-06-29T15:50:33.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.859+0000 7fb54a7fc700 1 --2- 172.21.15.47:0/3131140645 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb53c053b30 0x7fb53c055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:33.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.859+0000 7fb54a7fc700 1 -- 172.21.15.47:0/3131140645 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fb54c06b9d0 con 0x7fb554156b20 2022-06-29T15:50:33.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.859+0000 7fb559540700 1 --2- 172.21.15.47:0/3131140645 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb53c053b30 0x7fb53c055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:33.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.859+0000 7fb559540700 1 --2- 172.21.15.47:0/3131140645 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb53c053b30 0x7fb53c055ff0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7fb544006000 tx=0x7fb544008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:33.888 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:33.867+0000 7fb54a7fc700 1 -- 172.21.15.47:0/3131140645 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb54c042a70 con 0x7fb554156b20 2022-06-29T15:50:34.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.031+0000 7fb55a542700 1 -- 172.21.15.47:0/3131140645 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb554069e20 con 0x7fb554156b20 2022-06-29T15:50:34.038 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.031+0000 7fb54a7fc700 1 -- 172.21.15.47:0/3131140645 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+1563 (secure 0 0 0) 0x7fb54c046090 con 0x7fb554156b20 2022-06-29T15:50:34.039 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:50:34.039 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":53,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3752554010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3752554010},{"type":"v1","addr":"172.21.15.47:6835","nonce":3752554010}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":48},{"gid":7293,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.47:6837/3263659716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":3263659716},{"type":"v1","addr":"172.21.15.47:6837","nonce":3263659716}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":49},{"gid":56546,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/2526504380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2526504380},{"type":"v1","addr":"172.21.15.64:6833","nonce":2526504380}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":53}],"filesystems":[]} 2022-06-29T15:50:34.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.035+0000 7fb55a542700 1 -- 172.21.15.47:0/3131140645 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb53c053b30 msgr2=0x7fb53c055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:34.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.035+0000 7fb55a542700 1 --2- 172.21.15.47:0/3131140645 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb53c053b30 0x7fb53c055ff0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7fb544006000 tx=0x7fb544008040).stop 2022-06-29T15:50:34.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.035+0000 7fb55a542700 1 -- 172.21.15.47:0/3131140645 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb554156b20 msgr2=0x7fb554156fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:34.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.035+0000 7fb55a542700 1 --2- 172.21.15.47:0/3131140645 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb554156b20 0x7fb554156fa0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 rx=0x7fb54c001c40 tx=0x7fb54c009040).stop 2022-06-29T15:50:34.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.035+0000 7fb55a542700 1 -- 172.21.15.47:0/3131140645 shutdown_connections 2022-06-29T15:50:34.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.035+0000 7fb55a542700 1 --2- 172.21.15.47:0/3131140645 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb554156b20 0x7fb554156fa0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:34.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.035+0000 7fb55a542700 1 --2- 172.21.15.47:0/3131140645 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb53c053b30 0x7fb53c055ff0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:34.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.035+0000 7fb55a542700 1 --2- 172.21.15.47:0/3131140645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb554142e80 0x7fb55414d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:34.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.035+0000 7fb55a542700 1 --2- 172.21.15.47:0/3131140645 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb538004500 0x7fb55414e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:34.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.035+0000 7fb55a542700 1 -- 172.21.15.47:0/3131140645 >> 172.21.15.47:0/3131140645 conn(0x7fb554062560 msgr2=0x7fb5540b4ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:34.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.035+0000 7fb55a542700 1 -- 172.21.15.47:0/3131140645 shutdown_connections 2022-06-29T15:50:34.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.035+0000 7fb55a542700 1 -- 172.21.15.47:0/3131140645 wait complete. 2022-06-29T15:50:34.045 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 53 2022-06-29T15:50:34.052 INFO:tasks.ceph.mds.a:Restarting daemon 2022-06-29T15:50:34.053 INFO:tasks.ceph.mds.a:Stopping old one... 2022-06-29T15:50:34.053 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-06-29T15:50:34.053 INFO:teuthology.orchestra.run:waiting for 300 2022-06-29T15:50:34.054 INFO:tasks.ceph.mds.c:Restarting daemon 2022-06-29T15:50:34.054 INFO:tasks.ceph.mds.c:Stopping old one... 2022-06-29T15:50:34.054 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2022-06-29T15:50:34.055 INFO:teuthology.orchestra.run:waiting for 300 2022-06-29T15:50:34.055 INFO:tasks.ceph.mds.b:Restarting daemon 2022-06-29T15:50:34.055 INFO:tasks.ceph.mds.b:Stopping old one... 2022-06-29T15:50:34.056 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-06-29T15:50:34.056 INFO:teuthology.orchestra.run:waiting for 300 2022-06-29T15:50:34.065 INFO:tasks.ceph.mds.c:Stopped 2022-06-29T15:50:34.065 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2022-06-29T15:50:34.067 INFO:tasks.ceph.mds.a:Stopped 2022-06-29T15:50:34.067 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-06-29T15:50:34.068 INFO:tasks.ceph.mds.c:Started 2022-06-29T15:50:34.069 DEBUG:teuthology.parallel:result is None 2022-06-29T15:50:34.070 INFO:tasks.ceph.mds.a:Started 2022-06-29T15:50:34.070 DEBUG:teuthology.parallel:result is None 2022-06-29T15:50:34.071 INFO:tasks.ceph.mds.b:Stopped 2022-06-29T15:50:34.072 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-06-29T15:50:34.118 INFO:tasks.ceph.mds.b:Started 2022-06-29T15:50:34.118 DEBUG:teuthology.parallel:result is None 2022-06-29T15:50:34.119 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-06-29T15:50:34.132 INFO:tasks.ceph.mds.c.smithi047.stdout:starting mds.c at 2022-06-29T15:50:34.141 INFO:tasks.ceph.mds.a.smithi047.stdout:starting mds.a at 2022-06-29T15:50:34.196 INFO:tasks.ceph.mds.b.smithi064.stdout:starting mds.b at 2022-06-29T15:50:34.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.215+0000 7fc9e759e700 1 Processor -- start 2022-06-29T15:50:34.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.215+0000 7fc9e759e700 1 -- start start 2022-06-29T15:50:34.224 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.215+0000 7fc9e759e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc9e80b2ee0 0x7fc9e80b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:34.224 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.215+0000 7fc9e759e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc9e80ad970 0x7fc9e81407e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:34.224 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.215+0000 7fc9e759e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc9e80b0e50 0x7fc9e813d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:34.224 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.215+0000 7fc9e759e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9e813fef0 con 0x7fc9e80ad970 2022-06-29T15:50:34.225 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.215+0000 7fc9e759e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9e8143810 con 0x7fc9e80b0e50 2022-06-29T15:50:34.225 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.215+0000 7fc9e759e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e81400e0 con 0x7fc9e80b2ee0 2022-06-29T15:50:34.225 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.215+0000 7fc9e5d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc9e80ad970 0x7fc9e81407e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:34.225 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.215+0000 7fc9e5d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc9e80ad970 0x7fc9e81407e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52618/0 (socket says 172.21.15.47:52618) 2022-06-29T15:50:34.226 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.215+0000 7fc9e5d9b700 1 -- 172.21.15.47:0/3652091424 learned_addr learned my addr 172.21.15.47:0/3652091424 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:34.226 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.215+0000 7fc9e6d9d700 1 --2- 172.21.15.47:0/3652091424 >> v2:172.21.15.64:3300/0 conn(0x7fc9e80b0e50 0x7fc9e813d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:34.226 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.215+0000 7fc9e659c700 1 --2- 172.21.15.47:0/3652091424 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc9e80b2ee0 0x7fc9e80b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:34.226 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.215+0000 7fc9e5d9b700 1 -- 172.21.15.47:0/3652091424 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc9e80b2ee0 msgr2=0x7fc9e80b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:34.227 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.215+0000 7fc9e5d9b700 1 --2- 172.21.15.47:0/3652091424 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc9e80b2ee0 0x7fc9e80b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:34.227 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.215+0000 7fc9e5d9b700 1 -- 172.21.15.47:0/3652091424 >> v2:172.21.15.64:3300/0 conn(0x7fc9e80b0e50 msgr2=0x7fc9e813d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:34.227 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.215+0000 7fc9e5d9b700 1 --2- 172.21.15.47:0/3652091424 >> v2:172.21.15.64:3300/0 conn(0x7fc9e80b0e50 0x7fc9e813d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:34.227 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.215+0000 7fc9e5d9b700 1 -- 172.21.15.47:0/3652091424 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e8143570 con 0x7fc9e80ad970 2022-06-29T15:50:34.228 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.215+0000 7fc9e6d9d700 1 --2- 172.21.15.47:0/3652091424 >> v2:172.21.15.64:3300/0 conn(0x7fc9e80b0e50 0x7fc9e813d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:50:34.228 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.215+0000 7fc9e5d9b700 1 --2- 172.21.15.47:0/3652091424 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc9e80ad970 0x7fc9e81407e0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 rx=0x7fc9e80aefc0 tx=0x7fc9d8009290).ready entity=mon.0 client_cookie=e408ba64e1eb0a97 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:34.228 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.215+0000 7fc9e559a700 1 -- 172.21.15.47:0/3652091424 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9d800b830 con 0x7fc9e80ad970 2022-06-29T15:50:34.228 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.215+0000 7fc9e559a700 1 -- 172.21.15.47:0/3652091424 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9d800b990 con 0x7fc9e80ad970 2022-06-29T15:50:34.228 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.215+0000 7fc9e559a700 1 -- 172.21.15.47:0/3652091424 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9d800bb00 con 0x7fc9e80ad970 2022-06-29T15:50:34.229 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.215+0000 7fc9e759e700 1 -- 172.21.15.47:0/3652091424 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc9e80ad970 msgr2=0x7fc9e81407e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:34.229 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.215+0000 7fc9e759e700 1 --2- 172.21.15.47:0/3652091424 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc9e80ad970 0x7fc9e81407e0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 rx=0x7fc9e80aefc0 tx=0x7fc9d8009290).stop 2022-06-29T15:50:34.229 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.215+0000 7fc9e759e700 1 -- 172.21.15.47:0/3652091424 shutdown_connections 2022-06-29T15:50:34.230 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.215+0000 7fc9e759e700 1 --2- 172.21.15.47:0/3652091424 >> v2:172.21.15.64:3300/0 conn(0x7fc9e80b0e50 0x7fc9e813d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:34.230 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.215+0000 7fc9e759e700 1 --2- 172.21.15.47:0/3652091424 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc9e80ad970 0x7fc9e81407e0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:34.230 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.215+0000 7fc9e759e700 1 --2- 172.21.15.47:0/3652091424 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc9e80b2ee0 0x7fc9e80b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:34.230 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.215+0000 7fc9e759e700 1 -- 172.21.15.47:0/3652091424 >> 172.21.15.47:0/3652091424 conn(0x7fc9e8063670 msgr2=0x7fc9e8063a80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:34.231 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.215+0000 7fc9e759e700 1 -- 172.21.15.47:0/3652091424 shutdown_connections 2022-06-29T15:50:34.231 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.215+0000 7fc9e759e700 1 -- 172.21.15.47:0/3652091424 wait complete. 2022-06-29T15:50:34.231 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.219+0000 7fc9e759e700 1 Processor -- start 2022-06-29T15:50:34.232 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.219+0000 7fc9e759e700 1 -- start start 2022-06-29T15:50:34.234 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.219+0000 7fc9e759e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc9e80ad970 0x7fc9e8141830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:34.235 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.219+0000 7fc9e759e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc9e80b0e50 0x7fc9e8155760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:34.235 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.219+0000 7fc9e759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9e80b2ee0 0x7fc9e8155c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:34.235 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.219+0000 7fc9e759e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e80ad040 con 0x7fc9e80ad970 2022-06-29T15:50:34.235 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.219+0000 7fc9e759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e80644d0 con 0x7fc9e80b2ee0 2022-06-29T15:50:34.236 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.219+0000 7fc9e759e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e8064770 con 0x7fc9e80b0e50 2022-06-29T15:50:34.236 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.219+0000 7fc9e659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc9e80ad970 0x7fc9e8141830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:34.236 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.219+0000 7fc9e659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc9e80ad970 0x7fc9e8141830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52624/0 (socket says 172.21.15.47:52624) 2022-06-29T15:50:34.236 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.219+0000 7fc9e659c700 1 -- 172.21.15.47:0/1301433211 learned_addr learned my addr 172.21.15.47:0/1301433211 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:34.236 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.219+0000 7fc9e5d9b700 1 --2- 172.21.15.47:0/1301433211 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc9e80b0e50 0x7fc9e8155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:34.237 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.219+0000 7fc9e6d9d700 1 --2- 172.21.15.47:0/1301433211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9e80b2ee0 0x7fc9e8155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:34.237 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.219+0000 7fc9e659c700 1 -- 172.21.15.47:0/1301433211 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc9e80b0e50 msgr2=0x7fc9e8155760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:34.237 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.219+0000 7fc9e659c700 1 --2- 172.21.15.47:0/1301433211 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc9e80b0e50 0x7fc9e8155760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:34.237 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.219+0000 7fc9e659c700 1 -- 172.21.15.47:0/1301433211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9e80b2ee0 msgr2=0x7fc9e8155c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:34.238 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.219+0000 7fc9e659c700 1 --2- 172.21.15.47:0/1301433211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9e80b2ee0 0x7fc9e8155c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:34.238 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.219+0000 7fc9e659c700 1 -- 172.21.15.47:0/1301433211 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d8009530 con 0x7fc9e80ad970 2022-06-29T15:50:34.238 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.219+0000 7fc9e659c700 1 --2- 172.21.15.47:0/1301433211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc9e80ad970 0x7fc9e8141830 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 rx=0x7fc9d00030b0 tx=0x7fc9d0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:34.238 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.219+0000 7fc9df7fe700 1 -- 172.21.15.47:0/1301433211 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9d0011780 con 0x7fc9e80ad970 2022-06-29T15:50:34.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.219+0000 7fc9df7fe700 1 -- 172.21.15.47:0/1301433211 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9d00118e0 con 0x7fc9e80ad970 2022-06-29T15:50:34.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.219+0000 7fc9e759e700 1 -- 172.21.15.47:0/1301433211 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9e8067210 con 0x7fc9e80ad970 2022-06-29T15:50:34.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.219+0000 7fc9df7fe700 1 -- 172.21.15.47:0/1301433211 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9d0011c40 con 0x7fc9e80ad970 2022-06-29T15:50:34.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.219+0000 7fc9e759e700 1 -- 172.21.15.47:0/1301433211 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9e8067680 con 0x7fc9e80ad970 2022-06-29T15:50:34.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.223+0000 7fc9e759e700 1 -- 172.21.15.47:0/1301433211 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9e80ad240 con 0x7fc9e80ad970 2022-06-29T15:50:34.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.223+0000 7fc9df7fe700 1 -- 172.21.15.47:0/1301433211 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc9d001c460 con 0x7fc9e80ad970 2022-06-29T15:50:34.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.223+0000 7fc9df7fe700 1 --2- 172.21.15.47:0/1301433211 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc9cc0539e0 0x7fc9cc055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:34.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.223+0000 7fc9e5d9b700 1 --2- 172.21.15.47:0/1301433211 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc9cc0539e0 0x7fc9cc055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:34.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.223+0000 7fc9df7fe700 1 -- 172.21.15.47:0/1301433211 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fc9d006c0b0 con 0x7fc9e80ad970 2022-06-29T15:50:34.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.223+0000 7fc9e5d9b700 1 --2- 172.21.15.47:0/1301433211 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc9cc0539e0 0x7fc9cc055ea0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7fc9d801a000 tx=0x7fc9d8016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:34.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.227+0000 7fc9df7fe700 1 -- 172.21.15.47:0/1301433211 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc9d0042fe0 con 0x7fc9e80ad970 2022-06-29T15:50:34.295 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2022-06-29T15:50:34.296 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2022-06-29T15:50:34.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:34.359+0000 7fc9e759e700 1 -- 172.21.15.47:0/1301433211 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7fc9e8064930 con 0x7fc9e80ad970 2022-06-29T15:50:35.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.275+0000 7fc9df7fe700 1 -- 172.21.15.47:0/1301433211 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v116) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc9d0046600 con 0x7fc9e80ad970 2022-06-29T15:50:35.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.279+0000 7fc9e759e700 1 -- 172.21.15.47:0/1301433211 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc9cc0539e0 msgr2=0x7fc9cc055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:35.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.279+0000 7fc9e759e700 1 --2- 172.21.15.47:0/1301433211 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc9cc0539e0 0x7fc9cc055ea0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7fc9d801a000 tx=0x7fc9d8016000).stop 2022-06-29T15:50:35.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.279+0000 7fc9e759e700 1 -- 172.21.15.47:0/1301433211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc9e80ad970 msgr2=0x7fc9e8141830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:35.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.279+0000 7fc9e759e700 1 --2- 172.21.15.47:0/1301433211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc9e80ad970 0x7fc9e8141830 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 rx=0x7fc9d00030b0 tx=0x7fc9d0008040).stop 2022-06-29T15:50:35.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.279+0000 7fc9e759e700 1 -- 172.21.15.47:0/1301433211 shutdown_connections 2022-06-29T15:50:35.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.279+0000 7fc9e759e700 1 --2- 172.21.15.47:0/1301433211 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc9cc0539e0 0x7fc9cc055ea0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:35.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.279+0000 7fc9e759e700 1 --2- 172.21.15.47:0/1301433211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9e80b2ee0 0x7fc9e8155c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:35.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.279+0000 7fc9e759e700 1 --2- 172.21.15.47:0/1301433211 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc9e80b0e50 0x7fc9e8155760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:35.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.279+0000 7fc9e759e700 1 --2- 172.21.15.47:0/1301433211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc9e80ad970 0x7fc9e8141830 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:35.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.279+0000 7fc9e759e700 1 -- 172.21.15.47:0/1301433211 >> 172.21.15.47:0/1301433211 conn(0x7fc9e8063670 msgr2=0x7fc9e813e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:35.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.279+0000 7fc9e759e700 1 -- 172.21.15.47:0/1301433211 shutdown_connections 2022-06-29T15:50:35.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.279+0000 7fc9e759e700 1 -- 172.21.15.47:0/1301433211 wait complete. 2022-06-29T15:50:35.288 INFO:teuthology.orchestra.run.smithi047.stderr: removed all blacklist entries 2022-06-29T15:50:35.299 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2022-06-29T15:50:35.299 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-06-29T15:50:35.452 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.447+0000 7ff1fb59e700 1 Processor -- start 2022-06-29T15:50:35.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.447+0000 7ff1fb59e700 1 -- start start 2022-06-29T15:50:35.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.447+0000 7ff1fb59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff1fc141e70 0x7ff1fc0b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:35.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.447+0000 7ff1fb59e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1fc0b7c20 con 0x7ff1fc062dd0 2022-06-29T15:50:35.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.447+0000 7ff1fb59e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1fc0b3d10 con 0x7ff1fc0b7840 2022-06-29T15:50:35.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.447+0000 7ff1fb59e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1fc1422a0 con 0x7ff1fc141e70 2022-06-29T15:50:35.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.447+0000 7ff1f9d9b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff1fc062dd0 0x7ff1fc141760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38190/0 (socket says 172.21.15.47:38190) 2022-06-29T15:50:35.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.447+0000 7ff1f9d9b700 1 -- 172.21.15.47:0/1062752214 learned_addr learned my addr 172.21.15.47:0/1062752214 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:35.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.447+0000 7ff1fad9d700 1 --2- 172.21.15.47:0/1062752214 >> v2:172.21.15.64:3300/0 conn(0x7ff1fc141e70 0x7ff1fc0b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:35.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.447+0000 7ff1f959a700 1 -- 172.21.15.47:0/1062752214 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1249874997 0 0) 0x7ff1fc0b7c20 con 0x7ff1fc062dd0 2022-06-29T15:50:35.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.447+0000 7ff1f959a700 1 -- 172.21.15.47:0/1062752214 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1dc003260 con 0x7ff1fc062dd0 2022-06-29T15:50:35.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.447+0000 7ff1f959a700 1 -- 172.21.15.47:0/1062752214 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 58308712 0 0) 0x7ff1fc0b3d10 con 0x7ff1fc0b7840 2022-06-29T15:50:35.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.447+0000 7ff1f959a700 1 -- 172.21.15.47:0/1062752214 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1fc0b7c20 con 0x7ff1fc0b7840 2022-06-29T15:50:35.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.447+0000 7ff1fad9d700 1 -- 172.21.15.47:0/1062752214 >> v1:172.21.15.64:6789/0 conn(0x7ff1fc0b7840 legacy=0x7ff1fc0aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:35.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.447+0000 7ff1fad9d700 1 -- 172.21.15.47:0/1062752214 >> v1:172.21.15.47:6789/0 conn(0x7ff1fc062dd0 legacy=0x7ff1fc141760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:35.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.447+0000 7ff1fad9d700 1 -- 172.21.15.47:0/1062752214 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1fc067380 con 0x7ff1fc141e70 2022-06-29T15:50:35.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.451+0000 7ff1fad9d700 1 --2- 172.21.15.47:0/1062752214 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1fc141e70 0x7ff1fc0b3820 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7ff1f4002f70 tx=0x7ff1f4006df0).ready entity=mon.1 client_cookie=1390b27233248252 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:35.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.451+0000 7ff1f959a700 1 -- 172.21.15.47:0/1062752214 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff1f400d540 con 0x7ff1fc141e70 2022-06-29T15:50:35.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.451+0000 7ff1f959a700 1 -- 172.21.15.47:0/1062752214 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1f4000e90 con 0x7ff1fc141e70 2022-06-29T15:50:35.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.451+0000 7ff1f959a700 1 -- 172.21.15.47:0/1062752214 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff1f400d890 con 0x7ff1fc141e70 2022-06-29T15:50:35.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.451+0000 7ff1fb59e700 1 -- 172.21.15.47:0/1062752214 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1fc141e70 msgr2=0x7ff1fc0b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:35.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.451+0000 7ff1fb59e700 1 --2- 172.21.15.47:0/1062752214 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1fc141e70 0x7ff1fc0b3820 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7ff1f4002f70 tx=0x7ff1f4006df0).stop 2022-06-29T15:50:35.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.451+0000 7ff1fb59e700 1 -- 172.21.15.47:0/1062752214 shutdown_connections 2022-06-29T15:50:35.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.451+0000 7ff1fb59e700 1 --2- 172.21.15.47:0/1062752214 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1fc141e70 0x7ff1fc0b3820 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:35.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.451+0000 7ff1fb59e700 1 -- 172.21.15.47:0/1062752214 >> 172.21.15.47:0/1062752214 conn(0x7ff1fc05f480 msgr2=0x7ff1fc0647d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:35.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.451+0000 7ff1fb59e700 1 -- 172.21.15.47:0/1062752214 shutdown_connections 2022-06-29T15:50:35.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.451+0000 7ff1fb59e700 1 -- 172.21.15.47:0/1062752214 wait complete. 2022-06-29T15:50:35.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.455+0000 7ff1fb59e700 1 Processor -- start 2022-06-29T15:50:35.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.455+0000 7ff1fb59e700 1 -- start start 2022-06-29T15:50:35.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.455+0000 7ff1fb59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1fc062dd0 0x7ff1fc09ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:35.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.455+0000 7ff1fb59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1fc0b7840 0x7ff1fc153ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:35.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.455+0000 7ff1fb59e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff1fc141e70 0x7ff1fc0b63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:35.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.455+0000 7ff1fb59e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1fc0b68b0 con 0x7ff1fc0b7840 2022-06-29T15:50:35.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.455+0000 7ff1fb59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1fc0b6ad0 con 0x7ff1fc062dd0 2022-06-29T15:50:35.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.455+0000 7ff1fb59e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1fc1534e0 con 0x7ff1fc141e70 2022-06-29T15:50:35.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.455+0000 7ff1f9d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1fc0b7840 0x7ff1fc153ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:35.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.455+0000 7ff1f9d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1fc0b7840 0x7ff1fc153ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52642/0 (socket says 172.21.15.47:52642) 2022-06-29T15:50:35.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.455+0000 7ff1f9d9b700 1 -- 172.21.15.47:0/681503035 learned_addr learned my addr 172.21.15.47:0/681503035 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:35.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.455+0000 7ff1fa59c700 1 --2- 172.21.15.47:0/681503035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1fc062dd0 0x7ff1fc09ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:35.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.455+0000 7ff1fad9d700 1 --2- 172.21.15.47:0/681503035 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff1fc141e70 0x7ff1fc0b63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:35.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.455+0000 7ff1f9d9b700 1 -- 172.21.15.47:0/681503035 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff1fc141e70 msgr2=0x7ff1fc0b63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:35.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.455+0000 7ff1f9d9b700 1 --2- 172.21.15.47:0/681503035 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff1fc141e70 0x7ff1fc0b63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:35.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.455+0000 7ff1f9d9b700 1 -- 172.21.15.47:0/681503035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1fc062dd0 msgr2=0x7ff1fc09ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:35.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.455+0000 7ff1f9d9b700 1 --2- 172.21.15.47:0/681503035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1fc062dd0 0x7ff1fc09ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:35.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.455+0000 7ff1f9d9b700 1 -- 172.21.15.47:0/681503035 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f400b180 con 0x7ff1fc0b7840 2022-06-29T15:50:35.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.455+0000 7ff1f9d9b700 1 --2- 172.21.15.47:0/681503035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1fc0b7840 0x7ff1fc153ec0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 rx=0x7ff1e4001110 tx=0x7ff1e4007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:35.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.455+0000 7ff1f37fe700 1 -- 172.21.15.47:0/681503035 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff1e40070d0 con 0x7ff1fc0b7840 2022-06-29T15:50:35.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.455+0000 7ff1fb59e700 1 -- 172.21.15.47:0/681503035 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1fc153780 con 0x7ff1fc0b7840 2022-06-29T15:50:35.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.455+0000 7ff1f37fe700 1 -- 172.21.15.47:0/681503035 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1e4007230 con 0x7ff1fc0b7840 2022-06-29T15:50:35.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.455+0000 7ff1fb59e700 1 -- 172.21.15.47:0/681503035 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1fc1452a0 con 0x7ff1fc0b7840 2022-06-29T15:50:35.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.455+0000 7ff1f37fe700 1 -- 172.21.15.47:0/681503035 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff1e40073a0 con 0x7ff1fc0b7840 2022-06-29T15:50:35.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.459+0000 7ff1fb59e700 1 -- 172.21.15.47:0/681503035 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1fc0573d0 con 0x7ff1fc0b7840 2022-06-29T15:50:35.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.459+0000 7ff1f37fe700 1 -- 172.21.15.47:0/681503035 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff1e4010070 con 0x7ff1fc0b7840 2022-06-29T15:50:35.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.459+0000 7ff1f37fe700 1 --2- 172.21.15.47:0/681503035 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff1e00539e0 0x7ff1e0055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:35.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.459+0000 7ff1f37fe700 1 -- 172.21.15.47:0/681503035 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5600+0+0 (secure 0 0 0) 0x7ff1e406b2e0 con 0x7ff1fc0b7840 2022-06-29T15:50:35.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.459+0000 7ff1fa59c700 1 --2- 172.21.15.47:0/681503035 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff1e00539e0 0x7ff1e0055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:35.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.459+0000 7ff1fa59c700 1 --2- 172.21.15.47:0/681503035 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff1e00539e0 0x7ff1e0055ea0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7ff1ec00d000 tx=0x7ff1ec007da0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:35.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.467+0000 7ff1f37fe700 1 -- 172.21.15.47:0/681503035 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff1e4042250 con 0x7ff1fc0b7840 2022-06-29T15:50:35.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.635+0000 7ff1fb59e700 1 -- 172.21.15.47:0/681503035 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff1fc068830 con 0x7ff1fc0b7840 2022-06-29T15:50:35.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.639+0000 7ff1f37fe700 1 -- 172.21.15.47:0/681503035 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v30) v1 ==== 80+0+4255 (secure 0 0 0) 0x7ff1e4045870 con 0x7ff1fc0b7840 2022-06-29T15:50:35.646 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:50:35.646 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2022-06-29T15:50:35.647 INFO:teuthology.orchestra.run.smithi047.stdout: "auth_dump": [ 2022-06-29T15:50:35.647 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:35.648 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.a", 2022-06-29T15:50:35.648 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxiQJFFMxAA9Tv7FJD9u/BKvbkuKld6Kg==", 2022-06-29T15:50:35.648 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:50:35.649 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:50:35.649 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:50:35.650 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:50:35.651 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:50:35.654 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:35.654 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:35.654 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:35.654 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.b", 2022-06-29T15:50:35.655 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC2crxibt6FARAAEf2bwXKFhXKtqr1nN0rdXQ==", 2022-06-29T15:50:35.655 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:50:35.655 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:50:35.655 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:50:35.656 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:50:35.656 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:50:35.656 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:35.657 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:35.657 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:35.657 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.c", 2022-06-29T15:50:35.658 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxi/4ZMORAANpLbgTm3ehqi80USc3swGg==", 2022-06-29T15:50:35.658 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:50:35.658 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:50:35.659 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:50:35.659 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:50:35.659 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:50:35.659 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:35.659 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:35.660 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:35.660 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.0", 2022-06-29T15:50:35.661 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC2crxiY0YsMxAAZivb+CxL/w7vZwQv37Y4oQ==", 2022-06-29T15:50:35.661 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:50:35.661 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:50:35.662 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:50:35.662 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:50:35.663 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:35.663 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:35.664 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:35.664 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.1", 2022-06-29T15:50:35.664 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC5crxitLkKFxAA8+++LUUKLiO9SAuAfYgT5g==", 2022-06-29T15:50:35.665 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:50:35.665 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:50:35.665 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:50:35.666 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:50:35.666 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:35.666 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:35.666 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:35.666 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.2", 2022-06-29T15:50:35.667 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC7crxi1oM9NxAA2FfLdNt32YPwlyNdGQpaQA==", 2022-06-29T15:50:35.667 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:50:35.667 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:50:35.667 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:50:35.668 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:50:35.668 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:35.668 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:35.669 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:35.669 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.3", 2022-06-29T15:50:35.669 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC+crxiTsBgGhAA+gLBBDaskPbmRNWzgKwcHA==", 2022-06-29T15:50:35.670 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:50:35.670 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:50:35.670 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:50:35.670 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:50:35.671 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:35.671 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:35.671 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:35.671 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.4", 2022-06-29T15:50:35.672 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDDcrxivJebIhAAfwWJa7tK4hamkk1/1CSSfw==", 2022-06-29T15:50:35.672 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:50:35.672 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:50:35.672 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:50:35.673 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:50:35.673 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:35.673 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:35.674 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:35.674 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.5", 2022-06-29T15:50:35.674 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDGcrxijuozBhAAt+Dx3wH3WKp3DrbCbhmh8Q==", 2022-06-29T15:50:35.674 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:50:35.675 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:50:35.675 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:50:35.675 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:50:35.675 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:35.676 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:35.676 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:35.676 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.6", 2022-06-29T15:50:35.676 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDIcrxi91kXJxAA9JX7n98P394bD41Q9LUdvQ==", 2022-06-29T15:50:35.677 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:50:35.677 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:50:35.678 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:50:35.678 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:50:35.679 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:35.679 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:35.679 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:35.680 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.7", 2022-06-29T15:50:35.680 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDLcrxiHj0ZDRAASy6fAix8TmEZbAGtuFCfFg==", 2022-06-29T15:50:35.681 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:50:35.681 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:50:35.681 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:50:35.681 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:50:35.681 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:35.682 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:35.682 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:35.683 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.0", 2022-06-29T15:50:35.683 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC2crxiVDlfBxAAezkUfg4e2g0Rfakaa/Ek1Q==", 2022-06-29T15:50:35.684 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:50:35.684 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:50:35.685 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow r", 2022-06-29T15:50:35.685 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-29T15:50:35.686 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:35.686 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:35.687 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:35.687 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.1", 2022-06-29T15:50:35.688 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC2crxisXJ1CxAAhl07oF7AW66NdzfvvZTJmA==", 2022-06-29T15:50:35.688 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:50:35.689 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:50:35.689 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow r", 2022-06-29T15:50:35.689 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-29T15:50:35.689 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:35.690 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:35.690 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:35.691 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.admin", 2022-06-29T15:50:35.691 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxiW7HSFRAAhYqoGU9GFqj2HfUVGValSA==", 2022-06-29T15:50:35.691 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:50:35.692 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2022-06-29T15:50:35.692 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:50:35.692 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:50:35.693 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:50:35.693 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:35.693 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:35.694 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:35.694 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.x", 2022-06-29T15:50:35.694 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxidmuIMBAAcpNlYVu5+icCHg+yIjcKXw==", 2022-06-29T15:50:35.694 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:50:35.695 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2022-06-29T15:50:35.695 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2022-06-29T15:50:35.695 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:50:35.695 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:35.695 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:35.696 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:35.696 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.y", 2022-06-29T15:50:35.696 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxisSMFLBAAv03r4OoPjOP9nI9G/LK0iw==", 2022-06-29T15:50:35.697 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:50:35.697 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2022-06-29T15:50:35.698 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2022-06-29T15:50:35.699 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:50:35.699 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:35.700 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:35.700 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:50:35.701 INFO:teuthology.orchestra.run.smithi047.stdout:} 2022-06-29T15:50:35.704 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.639+0000 7ff1fb59e700 1 -- 172.21.15.47:0/681503035 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff1e00539e0 msgr2=0x7ff1e0055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:35.704 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.639+0000 7ff1fb59e700 1 --2- 172.21.15.47:0/681503035 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff1e00539e0 0x7ff1e0055ea0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7ff1ec00d000 tx=0x7ff1ec007da0).stop 2022-06-29T15:50:35.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.639+0000 7ff1fb59e700 1 -- 172.21.15.47:0/681503035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1fc0b7840 msgr2=0x7ff1fc153ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:35.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.639+0000 7ff1fb59e700 1 --2- 172.21.15.47:0/681503035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1fc0b7840 0x7ff1fc153ec0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 rx=0x7ff1e4001110 tx=0x7ff1e4007ad0).stop 2022-06-29T15:50:35.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.639+0000 7ff1fb59e700 1 -- 172.21.15.47:0/681503035 shutdown_connections 2022-06-29T15:50:35.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.639+0000 7ff1fb59e700 1 --2- 172.21.15.47:0/681503035 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff1fc141e70 0x7ff1fc0b63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:35.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.639+0000 7ff1fb59e700 1 --2- 172.21.15.47:0/681503035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1fc0b7840 0x7ff1fc153ec0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:35.707 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.639+0000 7ff1fb59e700 1 --2- 172.21.15.47:0/681503035 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff1e00539e0 0x7ff1e0055ea0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:35.707 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.639+0000 7ff1fb59e700 1 --2- 172.21.15.47:0/681503035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1fc062dd0 0x7ff1fc09ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:35.708 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.639+0000 7ff1fb59e700 1 -- 172.21.15.47:0/681503035 >> 172.21.15.47:0/681503035 conn(0x7ff1fc05f480 msgr2=0x7ff1fc0647b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:35.708 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.639+0000 7ff1fb59e700 1 -- 172.21.15.47:0/681503035 shutdown_connections 2022-06-29T15:50:35.709 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.639+0000 7ff1fb59e700 1 -- 172.21.15.47:0/681503035 wait complete. 2022-06-29T15:50:35.710 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:50:35.822 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.815+0000 7fc536fdb700 1 Processor -- start 2022-06-29T15:50:35.822 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.815+0000 7fc536fdb700 1 -- start start 2022-06-29T15:50:35.825 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.815+0000 7fc536fdb700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc5300b1ab0 0x7fc530060b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:35.825 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.815+0000 7fc536fdb700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc5300b2770 0x7fc5300ae170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:35.826 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.815+0000 7fc536fdb700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc53005fe10 con 0x7fc53005fa30 2022-06-29T15:50:35.826 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.815+0000 7fc536fdb700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5300b1ef0 con 0x7fc5300b2770 2022-06-29T15:50:35.826 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.815+0000 7fc536fdb700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5300600e0 con 0x7fc5300b1ab0 2022-06-29T15:50:35.827 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.815+0000 7fc5367da700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc5300b2770 0x7fc5300ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:35.827 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.815+0000 7fc5367da700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc5300b2770 0x7fc5300ae170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52652/0 (socket says 172.21.15.47:52652) 2022-06-29T15:50:35.827 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.815+0000 7fc5367da700 1 -- 172.21.15.47:0/2612419158 learned_addr learned my addr 172.21.15.47:0/2612419158 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:35.827 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.815+0000 7fc5357d8700 1 --2- 172.21.15.47:0/2612419158 >> v2:172.21.15.64:3300/0 conn(0x7fc5300b1ab0 0x7fc530060b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:35.828 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.815+0000 7fc5367da700 1 -- 172.21.15.47:0/2612419158 >> v2:172.21.15.64:3300/0 conn(0x7fc5300b1ab0 msgr2=0x7fc530060b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:35.828 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.815+0000 7fc5367da700 1 --2- 172.21.15.47:0/2612419158 >> v2:172.21.15.64:3300/0 conn(0x7fc5300b1ab0 0x7fc530060b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:35.828 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.815+0000 7fc5367da700 1 -- 172.21.15.47:0/2612419158 >> v1:172.21.15.64:6789/0 conn(0x7fc53005fa30 legacy=0x7fc53013f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:35.828 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.815+0000 7fc5367da700 1 -- 172.21.15.47:0/2612419158 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc530060380 con 0x7fc5300b2770 2022-06-29T15:50:35.828 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.819+0000 7fc5367da700 1 --2- 172.21.15.47:0/2612419158 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5300b2770 0x7fc5300ae170 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 rx=0x7fc52c002f70 tx=0x7fc52c006eb0).ready entity=mon.0 client_cookie=7e7932e61416741b server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:35.829 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.819+0000 7fc534fd7700 1 -- 172.21.15.47:0/2612419158 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc52c00d610 con 0x7fc5300b2770 2022-06-29T15:50:35.829 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.819+0000 7fc534fd7700 1 -- 172.21.15.47:0/2612419158 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc52c000e90 con 0x7fc5300b2770 2022-06-29T15:50:35.829 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.819+0000 7fc534fd7700 1 -- 172.21.15.47:0/2612419158 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc52c00d9a0 con 0x7fc5300b2770 2022-06-29T15:50:35.830 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.819+0000 7fc536fdb700 1 -- 172.21.15.47:0/2612419158 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5300b2770 msgr2=0x7fc5300ae170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:35.830 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.819+0000 7fc536fdb700 1 --2- 172.21.15.47:0/2612419158 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5300b2770 0x7fc5300ae170 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 rx=0x7fc52c002f70 tx=0x7fc52c006eb0).stop 2022-06-29T15:50:35.830 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.819+0000 7fc536fdb700 1 -- 172.21.15.47:0/2612419158 shutdown_connections 2022-06-29T15:50:35.831 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.819+0000 7fc536fdb700 1 --2- 172.21.15.47:0/2612419158 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5300b2770 0x7fc5300ae170 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:35.831 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.819+0000 7fc536fdb700 1 --2- 172.21.15.47:0/2612419158 >> v2:172.21.15.64:3300/0 conn(0x7fc5300b1ab0 0x7fc530060b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:35.831 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.819+0000 7fc536fdb700 1 -- 172.21.15.47:0/2612419158 >> 172.21.15.47:0/2612419158 conn(0x7fc530062df0 msgr2=0x7fc5300631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:35.832 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.819+0000 7fc536fdb700 1 -- 172.21.15.47:0/2612419158 shutdown_connections 2022-06-29T15:50:35.832 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.819+0000 7fc536fdb700 1 -- 172.21.15.47:0/2612419158 wait complete. 2022-06-29T15:50:35.835 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.819+0000 7fc536fdb700 1 Processor -- start 2022-06-29T15:50:35.835 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.819+0000 7fc536fdb700 1 -- start start 2022-06-29T15:50:35.836 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.823+0000 7fc536fdb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc53005fa30 0x7fc53013f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:35.836 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.823+0000 7fc536fdb700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc5300b1ab0 0x7fc530099f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:35.836 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.823+0000 7fc536fdb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5300b2770 0x7fc530142460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:35.836 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.823+0000 7fc536fdb700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc530099a20 con 0x7fc5300b2770 2022-06-29T15:50:35.836 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.823+0000 7fc536fdb700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc53009a440 con 0x7fc53005fa30 2022-06-29T15:50:35.837 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.823+0000 7fc536fdb700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc530142000 con 0x7fc5300b1ab0 2022-06-29T15:50:35.837 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.823+0000 7fc5367da700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5300b2770 0x7fc530142460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:35.837 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.823+0000 7fc5367da700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5300b2770 0x7fc530142460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52658/0 (socket says 172.21.15.47:52658) 2022-06-29T15:50:35.837 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.823+0000 7fc5367da700 1 -- 172.21.15.47:0/2791364018 learned_addr learned my addr 172.21.15.47:0/2791364018 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:35.838 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.823+0000 7fc535fd9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc53005fa30 0x7fc53013f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:35.838 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.823+0000 7fc5357d8700 1 --2- 172.21.15.47:0/2791364018 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc5300b1ab0 0x7fc530099f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:35.838 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.823+0000 7fc5367da700 1 -- 172.21.15.47:0/2791364018 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc5300b1ab0 msgr2=0x7fc530099f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:35.839 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.823+0000 7fc5367da700 1 --2- 172.21.15.47:0/2791364018 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc5300b1ab0 0x7fc530099f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:35.839 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.823+0000 7fc5367da700 1 -- 172.21.15.47:0/2791364018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc53005fa30 msgr2=0x7fc53013f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:35.839 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.823+0000 7fc5367da700 1 --2- 172.21.15.47:0/2791364018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc53005fa30 0x7fc53013f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:35.839 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.823+0000 7fc5367da700 1 -- 172.21.15.47:0/2791364018 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc52c00b1e0 con 0x7fc5300b2770 2022-06-29T15:50:35.839 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.823+0000 7fc5367da700 1 --2- 172.21.15.47:0/2791364018 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5300b2770 0x7fc530142460 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 rx=0x7fc52c002f70 tx=0x7fc52c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:35.840 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.823+0000 7fc51effd700 1 -- 172.21.15.47:0/2791364018 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc52c011760 con 0x7fc5300b2770 2022-06-29T15:50:35.840 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.823+0000 7fc51effd700 1 -- 172.21.15.47:0/2791364018 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc52c0118c0 con 0x7fc5300b2770 2022-06-29T15:50:35.840 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.823+0000 7fc536fdb700 1 -- 172.21.15.47:0/2791364018 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc530099c50 con 0x7fc5300b2770 2022-06-29T15:50:35.840 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.823+0000 7fc51effd700 1 -- 172.21.15.47:0/2791364018 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc52c011c20 con 0x7fc5300b2770 2022-06-29T15:50:35.841 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.823+0000 7fc536fdb700 1 -- 172.21.15.47:0/2791364018 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc530143630 con 0x7fc5300b2770 2022-06-29T15:50:35.841 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.823+0000 7fc536fdb700 1 -- 172.21.15.47:0/2791364018 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5301421c0 con 0x7fc5300b2770 2022-06-29T15:50:35.841 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.823+0000 7fc51effd700 1 -- 172.21.15.47:0/2791364018 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc52c01a040 con 0x7fc5300b2770 2022-06-29T15:50:35.842 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.827+0000 7fc51effd700 1 --2- 172.21.15.47:0/2791364018 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc510053b00 0x7fc510055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:35.842 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.827+0000 7fc51effd700 1 -- 172.21.15.47:0/2791364018 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fc52c06cb40 con 0x7fc5300b2770 2022-06-29T15:50:35.842 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.827+0000 7fc535fd9700 1 --2- 172.21.15.47:0/2791364018 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc510053b00 0x7fc510055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:35.842 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.827+0000 7fc535fd9700 1 --2- 172.21.15.47:0/2791364018 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc510053b00 0x7fc510055fc0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7fc52800d000 tx=0x7fc528007f10).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:35.843 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.831+0000 7fc51effd700 1 -- 172.21.15.47:0/2791364018 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc5301421c0 con 0x7fc5300b2770 2022-06-29T15:50:35.969 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.963+0000 7fc536fdb700 1 -- 172.21.15.47:0/2791364018 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc5301421c0 con 0x7fc5300b2770 2022-06-29T15:50:35.971 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.963+0000 7fc51effd700 1 -- 172.21.15.47:0/2791364018 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+13424 (secure 0 0 0) 0x7fc5301421c0 con 0x7fc5300b2770 2022-06-29T15:50:35.971 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:50:35.972 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":116,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:50:35.269262+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:50:35.974 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.967+0000 7fc536fdb700 1 -- 172.21.15.47:0/2791364018 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc510053b00 msgr2=0x7fc510055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:35.975 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.967+0000 7fc536fdb700 1 --2- 172.21.15.47:0/2791364018 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc510053b00 0x7fc510055fc0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7fc52800d000 tx=0x7fc528007f10).stop 2022-06-29T15:50:35.975 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.967+0000 7fc536fdb700 1 -- 172.21.15.47:0/2791364018 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5300b2770 msgr2=0x7fc530142460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:35.975 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.967+0000 7fc536fdb700 1 --2- 172.21.15.47:0/2791364018 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5300b2770 0x7fc530142460 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 rx=0x7fc52c002f70 tx=0x7fc52c015000).stop 2022-06-29T15:50:35.975 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.967+0000 7fc536fdb700 1 -- 172.21.15.47:0/2791364018 shutdown_connections 2022-06-29T15:50:35.976 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.967+0000 7fc536fdb700 1 --2- 172.21.15.47:0/2791364018 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5300b2770 0x7fc530142460 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:35.976 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.967+0000 7fc536fdb700 1 --2- 172.21.15.47:0/2791364018 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc5300b1ab0 0x7fc530099f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:35.976 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.967+0000 7fc536fdb700 1 --2- 172.21.15.47:0/2791364018 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc510053b00 0x7fc510055fc0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:35.977 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.967+0000 7fc536fdb700 1 --2- 172.21.15.47:0/2791364018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc53005fa30 0x7fc53013f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:35.977 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.967+0000 7fc536fdb700 1 -- 172.21.15.47:0/2791364018 >> 172.21.15.47:0/2791364018 conn(0x7fc530062df0 msgr2=0x7fc530135a00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:35.977 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.967+0000 7fc536fdb700 1 -- 172.21.15.47:0/2791364018 shutdown_connections 2022-06-29T15:50:35.977 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:35.967+0000 7fc536fdb700 1 -- 172.21.15.47:0/2791364018 wait complete. 2022-06-29T15:50:35.984 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-29T15:42:16.971431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-29T15:42:25.290884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-06-29T15:50:35.985 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-29T15:50:36.101 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.095+0000 7f960e471700 1 Processor -- start 2022-06-29T15:50:36.101 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.095+0000 7f960e471700 1 -- start start 2022-06-29T15:50:36.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.095+0000 7f960e471700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f96080acfd0 0x7f96080b1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:36.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.095+0000 7f960e471700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96080611e0 con 0x7f96080b1a10 2022-06-29T15:50:36.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.095+0000 7f960e471700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96080b10c0 con 0x7f960809ada0 2022-06-29T15:50:36.104 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.095+0000 7f960e471700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96080b0e30 con 0x7f96080acfd0 2022-06-29T15:50:36.104 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.095+0000 7f960dc70700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f96080b1a10 0x7f96080af560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38218/0 (socket says 172.21.15.47:38218) 2022-06-29T15:50:36.104 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.095+0000 7f960dc70700 1 -- 172.21.15.47:0/3729111662 learned_addr learned my addr 172.21.15.47:0/3729111662 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:36.104 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.095+0000 7f960cc6e700 1 --2- 172.21.15.47:0/3729111662 >> v2:172.21.15.64:3300/0 conn(0x7f96080acfd0 0x7f96080b1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:36.105 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.095+0000 7f95fffff700 1 -- 172.21.15.47:0/3729111662 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 754377808 0 0) 0x7f96080611e0 con 0x7f96080b1a10 2022-06-29T15:50:36.105 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.095+0000 7f95fffff700 1 -- 172.21.15.47:0/3729111662 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95ec003260 con 0x7f96080b1a10 2022-06-29T15:50:36.106 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.099+0000 7f960cc6e700 1 -- 172.21.15.47:0/3729111662 >> v1:172.21.15.64:6789/0 conn(0x7f960809ada0 legacy=0x7f960809b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:36.107 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.099+0000 7f960cc6e700 1 -- 172.21.15.47:0/3729111662 >> v1:172.21.15.47:6789/0 conn(0x7f96080b1a10 legacy=0x7f96080af560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:36.107 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.099+0000 7f960cc6e700 1 -- 172.21.15.47:0/3729111662 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f960809b890 con 0x7f96080acfd0 2022-06-29T15:50:36.107 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.099+0000 7f960cc6e700 1 --2- 172.21.15.47:0/3729111662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96080acfd0 0x7f96080b1520 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7f95f8002f70 tx=0x7f95f8006d90).ready entity=mon.1 client_cookie=1dee3d98106337f0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:36.107 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.099+0000 7f95fffff700 1 -- 172.21.15.47:0/3729111662 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f95f800d4a0 con 0x7f96080acfd0 2022-06-29T15:50:36.108 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.099+0000 7f95fffff700 1 -- 172.21.15.47:0/3729111662 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f8000e90 con 0x7f96080acfd0 2022-06-29T15:50:36.108 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.099+0000 7f95fffff700 1 -- 172.21.15.47:0/3729111662 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f95f800d7f0 con 0x7f96080acfd0 2022-06-29T15:50:36.108 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.099+0000 7f960e471700 1 -- 172.21.15.47:0/3729111662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96080acfd0 msgr2=0x7f96080b1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:36.108 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.099+0000 7f960e471700 1 --2- 172.21.15.47:0/3729111662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96080acfd0 0x7f96080b1520 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7f95f8002f70 tx=0x7f95f8006d90).stop 2022-06-29T15:50:36.109 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.099+0000 7f960e471700 1 -- 172.21.15.47:0/3729111662 shutdown_connections 2022-06-29T15:50:36.109 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.099+0000 7f960e471700 1 --2- 172.21.15.47:0/3729111662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96080acfd0 0x7f96080b1520 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:36.109 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.099+0000 7f960e471700 1 -- 172.21.15.47:0/3729111662 >> 172.21.15.47:0/3729111662 conn(0x7f9608062df0 msgr2=0x7f96080631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:36.109 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.099+0000 7f960e471700 1 -- 172.21.15.47:0/3729111662 shutdown_connections 2022-06-29T15:50:36.110 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.099+0000 7f960e471700 1 -- 172.21.15.47:0/3729111662 wait complete. 2022-06-29T15:50:36.113 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.103+0000 7f960e471700 1 Processor -- start 2022-06-29T15:50:36.113 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.103+0000 7f960e471700 1 -- start start 2022-06-29T15:50:36.113 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.103+0000 7f960e471700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f960809ada0 0x7f9608063af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:36.113 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.103+0000 7f960e471700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f96080acfd0 0x7f96080b2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:36.114 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.103+0000 7f960e471700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96080b1a10 0x7f9608066e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:36.114 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.103+0000 7f960e471700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96080b3140 con 0x7f96080b1a10 2022-06-29T15:50:36.114 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.103+0000 7f960e471700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9608065d10 con 0x7f960809ada0 2022-06-29T15:50:36.114 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.103+0000 7f960e471700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9608067310 con 0x7f96080acfd0 2022-06-29T15:50:36.115 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.103+0000 7f960dc70700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96080b1a10 0x7f9608066e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:36.115 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.103+0000 7f960cc6e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f96080acfd0 0x7f96080b2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:36.115 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.103+0000 7f960dc70700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96080b1a10 0x7f9608066e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52672/0 (socket says 172.21.15.47:52672) 2022-06-29T15:50:36.115 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.103+0000 7f960d46f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f960809ada0 0x7f9608063af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:36.116 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.103+0000 7f960dc70700 1 -- 172.21.15.47:0/2766584577 learned_addr learned my addr 172.21.15.47:0/2766584577 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:36.116 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.103+0000 7f960dc70700 1 -- 172.21.15.47:0/2766584577 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f96080acfd0 msgr2=0x7f96080b2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:36.116 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.103+0000 7f960dc70700 1 --2- 172.21.15.47:0/2766584577 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f96080acfd0 0x7f96080b2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:36.116 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.103+0000 7f960dc70700 1 -- 172.21.15.47:0/2766584577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f960809ada0 msgr2=0x7f9608063af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:36.116 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.103+0000 7f960dc70700 1 --2- 172.21.15.47:0/2766584577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f960809ada0 0x7f9608063af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:36.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.103+0000 7f960dc70700 1 -- 172.21.15.47:0/2766584577 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f800b120 con 0x7f96080b1a10 2022-06-29T15:50:36.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.103+0000 7f960d46f700 1 --2- 172.21.15.47:0/2766584577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f960809ada0 0x7f9608063af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:50:36.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.103+0000 7f960dc70700 1 --2- 172.21.15.47:0/2766584577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96080b1a10 0x7f9608066e20 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 rx=0x7f9608067830 tx=0x7f9600007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:36.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.107+0000 7f95fe7fc700 1 -- 172.21.15.47:0/2766584577 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f960000cea0 con 0x7f96080b1a10 2022-06-29T15:50:36.118 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.107+0000 7f95fe7fc700 1 -- 172.21.15.47:0/2766584577 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9600005ba0 con 0x7f96080b1a10 2022-06-29T15:50:36.118 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.107+0000 7f960e471700 1 -- 172.21.15.47:0/2766584577 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9608065a70 con 0x7f96080b1a10 2022-06-29T15:50:36.118 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.107+0000 7f95fe7fc700 1 -- 172.21.15.47:0/2766584577 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96000070d0 con 0x7f96080b1a10 2022-06-29T15:50:36.118 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.107+0000 7f960e471700 1 -- 172.21.15.47:0/2766584577 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96081503a0 con 0x7f96080b1a10 2022-06-29T15:50:36.118 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.107+0000 7f960e471700 1 -- 172.21.15.47:0/2766584577 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96081505a0 con 0x7f96080b1a10 2022-06-29T15:50:36.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.107+0000 7f95fe7fc700 1 -- 172.21.15.47:0/2766584577 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9600010070 con 0x7f96080b1a10 2022-06-29T15:50:36.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.107+0000 7f95fe7fc700 1 --2- 172.21.15.47:0/2766584577 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f95f0053a70 0x7f95f0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:36.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.107+0000 7f95fe7fc700 1 -- 172.21.15.47:0/2766584577 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f960006b620 con 0x7f96080b1a10 2022-06-29T15:50:36.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.107+0000 7f960d46f700 1 --2- 172.21.15.47:0/2766584577 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f95f0053a70 0x7f95f0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:36.121 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.107+0000 7f960d46f700 1 --2- 172.21.15.47:0/2766584577 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f95f0053a70 0x7f95f0055f30 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7f960400d000 tx=0x7f9604009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:36.121 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.115+0000 7f95fe7fc700 1 -- 172.21.15.47:0/2766584577 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f96081505a0 con 0x7f96080b1a10 2022-06-29T15:50:36.245 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.239+0000 7f960e471700 1 -- 172.21.15.47:0/2766584577 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f96081505a0 con 0x7f96080b1a10 2022-06-29T15:50:36.245 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.239+0000 7f95fe7fc700 1 -- 172.21.15.47:0/2766584577 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v116) v1 ==== 110+0+10 (secure 0 0 0) 0x7f96081505a0 con 0x7f96080b1a10 2022-06-29T15:50:36.246 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2022-06-29T15:50:36.249 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.243+0000 7f960e471700 1 -- 172.21.15.47:0/2766584577 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f95f0053a70 msgr2=0x7f95f0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:36.250 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.243+0000 7f960e471700 1 --2- 172.21.15.47:0/2766584577 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f95f0053a70 0x7f95f0055f30 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7f960400d000 tx=0x7f9604009040).stop 2022-06-29T15:50:36.250 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.243+0000 7f960e471700 1 -- 172.21.15.47:0/2766584577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96080b1a10 msgr2=0x7f9608066e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:36.250 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.243+0000 7f960e471700 1 --2- 172.21.15.47:0/2766584577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96080b1a10 0x7f9608066e20 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 rx=0x7f9608067830 tx=0x7f9600007e00).stop 2022-06-29T15:50:36.250 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.243+0000 7f960e471700 1 -- 172.21.15.47:0/2766584577 shutdown_connections 2022-06-29T15:50:36.251 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.243+0000 7f960e471700 1 --2- 172.21.15.47:0/2766584577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96080b1a10 0x7f9608066e20 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:36.251 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.243+0000 7f960e471700 1 --2- 172.21.15.47:0/2766584577 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f96080acfd0 0x7f96080b2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:36.251 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.243+0000 7f960e471700 1 --2- 172.21.15.47:0/2766584577 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f95f0053a70 0x7f95f0055f30 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:36.251 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.243+0000 7f960e471700 1 --2- 172.21.15.47:0/2766584577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f960809ada0 0x7f9608063af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:36.251 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.243+0000 7f960e471700 1 -- 172.21.15.47:0/2766584577 >> 172.21.15.47:0/2766584577 conn(0x7f9608062df0 msgr2=0x7f96080ad510 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:36.252 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.243+0000 7f960e471700 1 -- 172.21.15.47:0/2766584577 shutdown_connections 2022-06-29T15:50:36.252 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.243+0000 7f960e471700 1 -- 172.21.15.47:0/2766584577 wait complete. 2022-06-29T15:50:36.267 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-29T15:50:36.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.367+0000 7f58fd517700 1 Processor -- start 2022-06-29T15:50:36.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.367+0000 7f58fd517700 1 -- start start 2022-06-29T15:50:36.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.371+0000 7f58fd517700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f58f805c850 0x7f58f8143a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:36.394 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.371+0000 7f58fd517700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f58f80676a0 0x7f58f8140130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:36.394 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.371+0000 7f58fd517700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58f805cc30 con 0x7f58f80b5d90 2022-06-29T15:50:36.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.371+0000 7f58fd517700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58f81430a0 con 0x7f58f80676a0 2022-06-29T15:50:36.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.371+0000 7f58fd517700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58f8143300 con 0x7f58f805c850 2022-06-29T15:50:36.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.371+0000 7f58fcd16700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f58f80676a0 0x7f58f8140130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:36.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.371+0000 7f58f77fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f58f80b5d90 0x7f58f80b4860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38232/0 (socket says 172.21.15.47:38232) 2022-06-29T15:50:36.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.371+0000 7f58f77fe700 1 -- 172.21.15.47:0/2951928818 learned_addr learned my addr 172.21.15.47:0/2951928818 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:36.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.371+0000 7f58f7fff700 1 --2- 172.21.15.47:0/2951928818 >> v2:172.21.15.64:3300/0 conn(0x7f58f805c850 0x7f58f8143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:36.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.371+0000 7f58fcd16700 1 -- 172.21.15.47:0/2951928818 >> v2:172.21.15.64:3300/0 conn(0x7f58f805c850 msgr2=0x7f58f8143a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:36.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.371+0000 7f58fcd16700 1 --2- 172.21.15.47:0/2951928818 >> v2:172.21.15.64:3300/0 conn(0x7f58f805c850 0x7f58f8143a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:36.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.371+0000 7f58fcd16700 1 -- 172.21.15.47:0/2951928818 >> v1:172.21.15.47:6789/0 conn(0x7f58f80b5d90 legacy=0x7f58f80b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:36.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.371+0000 7f58f6ffd700 1 -- 172.21.15.47:0/2951928818 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3152475732 0 0) 0x7f58f805cc30 con 0x7f58f80b5d90 2022-06-29T15:50:36.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.371+0000 7f58fcd16700 1 -- 172.21.15.47:0/2951928818 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f81435a0 con 0x7f58f80676a0 2022-06-29T15:50:36.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.371+0000 7f58fcd16700 1 --2- 172.21.15.47:0/2951928818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f58f80676a0 0x7f58f8140130 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 rx=0x7f58f00032f0 tx=0x7f58f00091d0).ready entity=mon.0 client_cookie=7f810e8165bc40f7 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:36.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.371+0000 7f58f6ffd700 1 -- 172.21.15.47:0/2951928818 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f58f000b680 con 0x7f58f80676a0 2022-06-29T15:50:36.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.371+0000 7f58f6ffd700 1 -- 172.21.15.47:0/2951928818 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f000b7e0 con 0x7f58f80676a0 2022-06-29T15:50:36.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.371+0000 7f58f6ffd700 1 -- 172.21.15.47:0/2951928818 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f58f000b950 con 0x7f58f80676a0 2022-06-29T15:50:36.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.371+0000 7f58fd517700 1 -- 172.21.15.47:0/2951928818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f58f80676a0 msgr2=0x7f58f8140130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:36.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.371+0000 7f58fd517700 1 --2- 172.21.15.47:0/2951928818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f58f80676a0 0x7f58f8140130 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 rx=0x7f58f00032f0 tx=0x7f58f00091d0).stop 2022-06-29T15:50:36.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.371+0000 7f58fd517700 1 -- 172.21.15.47:0/2951928818 shutdown_connections 2022-06-29T15:50:36.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.371+0000 7f58fd517700 1 --2- 172.21.15.47:0/2951928818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f58f80676a0 0x7f58f8140130 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:36.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.371+0000 7f58fd517700 1 --2- 172.21.15.47:0/2951928818 >> v2:172.21.15.64:3300/0 conn(0x7f58f805c850 0x7f58f8143a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:36.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.371+0000 7f58fd517700 1 -- 172.21.15.47:0/2951928818 >> 172.21.15.47:0/2951928818 conn(0x7f58f8060e00 msgr2=0x7f58f80611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:36.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.371+0000 7f58fd517700 1 -- 172.21.15.47:0/2951928818 shutdown_connections 2022-06-29T15:50:36.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.371+0000 7f58fd517700 1 -- 172.21.15.47:0/2951928818 wait complete. 2022-06-29T15:50:36.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.371+0000 7f58fd517700 1 Processor -- start 2022-06-29T15:50:36.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.371+0000 7f58fd517700 1 -- start start 2022-06-29T15:50:36.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.375+0000 7f58fd517700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58f805c850 0x7f58f8142100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:36.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.375+0000 7f58fd517700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f58f80676a0 0x7f58f812aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:36.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.375+0000 7f58fd517700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f58f80b5d90 0x7f58f812b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:36.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.375+0000 7f58fd517700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f80b2600 con 0x7f58f80b5d90 2022-06-29T15:50:36.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.375+0000 7f58fd517700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f80ba520 con 0x7f58f805c850 2022-06-29T15:50:36.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.375+0000 7f58fd517700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58f80b3110 con 0x7f58f80676a0 2022-06-29T15:50:36.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.375+0000 7f58fcd16700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f58f80b5d90 0x7f58f812b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:36.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.375+0000 7f58fcd16700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f58f80b5d90 0x7f58f812b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52684/0 (socket says 172.21.15.47:52684) 2022-06-29T15:50:36.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.375+0000 7f58fcd16700 1 -- 172.21.15.47:0/665795226 learned_addr learned my addr 172.21.15.47:0/665795226 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:36.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.375+0000 7f58f7fff700 1 --2- 172.21.15.47:0/665795226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58f805c850 0x7f58f8142100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:36.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.375+0000 7f58f77fe700 1 --2- 172.21.15.47:0/665795226 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f58f80676a0 0x7f58f812aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:36.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.375+0000 7f58fcd16700 1 -- 172.21.15.47:0/665795226 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f58f80676a0 msgr2=0x7f58f812aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:36.409 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.375+0000 7f58fcd16700 1 --2- 172.21.15.47:0/665795226 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f58f80676a0 0x7f58f812aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:36.410 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.375+0000 7f58fcd16700 1 -- 172.21.15.47:0/665795226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58f805c850 msgr2=0x7f58f8142100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:36.410 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.375+0000 7f58fcd16700 1 --2- 172.21.15.47:0/665795226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58f805c850 0x7f58f8142100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:36.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.375+0000 7f58fcd16700 1 -- 172.21.15.47:0/665795226 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f0009470 con 0x7f58f80b5d90 2022-06-29T15:50:36.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.375+0000 7f58f7fff700 1 --2- 172.21.15.47:0/665795226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58f805c850 0x7f58f8142100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:50:36.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.375+0000 7f58fcd16700 1 --2- 172.21.15.47:0/665795226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f58f80b5d90 0x7f58f812b3d0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 rx=0x7f58f00032f0 tx=0x7f58f0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:36.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.375+0000 7f58f57fa700 1 -- 172.21.15.47:0/665795226 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f58f000b950 con 0x7f58f80b5d90 2022-06-29T15:50:36.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.375+0000 7f58f57fa700 1 -- 172.21.15.47:0/665795226 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f00107a0 con 0x7f58f80b5d90 2022-06-29T15:50:36.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.375+0000 7f58fd517700 1 -- 172.21.15.47:0/665795226 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58f80b2830 con 0x7f58f80b5d90 2022-06-29T15:50:36.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.375+0000 7f58f57fa700 1 -- 172.21.15.47:0/665795226 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f58f000b680 con 0x7f58f80b5d90 2022-06-29T15:50:36.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.375+0000 7f58fd517700 1 -- 172.21.15.47:0/665795226 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58f812b8c0 con 0x7f58f80b5d90 2022-06-29T15:50:36.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.375+0000 7f58fd517700 1 -- 172.21.15.47:0/665795226 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58f80573d0 con 0x7f58f80b5d90 2022-06-29T15:50:36.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.375+0000 7f58f57fa700 1 -- 172.21.15.47:0/665795226 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f58f0027020 con 0x7f58f80b5d90 2022-06-29T15:50:36.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.379+0000 7f58f57fa700 1 --2- 172.21.15.47:0/665795226 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f58dc053950 0x7f58dc055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:36.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.379+0000 7f58f57fa700 1 -- 172.21.15.47:0/665795226 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f58f006d1f0 con 0x7f58f80b5d90 2022-06-29T15:50:36.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.379+0000 7f58f7fff700 1 --2- 172.21.15.47:0/665795226 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f58dc053950 0x7f58dc055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:36.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.379+0000 7f58f7fff700 1 --2- 172.21.15.47:0/665795226 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f58dc053950 0x7f58dc055e10 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7f58e8006000 tx=0x7f58e8008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:36.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.383+0000 7f58f57fa700 1 -- 172.21.15.47:0/665795226 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f58f0044250 con 0x7f58f80b5d90 2022-06-29T15:50:36.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.511+0000 7f58fd517700 1 -- 172.21.15.47:0/665795226 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f58f8069a60 con 0x7f58f80b5d90 2022-06-29T15:50:36.520 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.515+0000 7f58f57fa700 1 -- 172.21.15.47:0/665795226 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v116) v1 ==== 92+0+10 (secure 0 0 0) 0x7f58f00150a0 con 0x7f58f80b5d90 2022-06-29T15:50:36.521 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:50:36.523 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.515+0000 7f58fd517700 1 -- 172.21.15.47:0/665795226 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f58dc053950 msgr2=0x7f58dc055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:36.523 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.515+0000 7f58fd517700 1 --2- 172.21.15.47:0/665795226 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f58dc053950 0x7f58dc055e10 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7f58e8006000 tx=0x7f58e8008040).stop 2022-06-29T15:50:36.523 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.515+0000 7f58fd517700 1 -- 172.21.15.47:0/665795226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f58f80b5d90 msgr2=0x7f58f812b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:36.524 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.515+0000 7f58fd517700 1 --2- 172.21.15.47:0/665795226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f58f80b5d90 0x7f58f812b3d0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 rx=0x7f58f00032f0 tx=0x7f58f0016000).stop 2022-06-29T15:50:36.524 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.515+0000 7f58fd517700 1 -- 172.21.15.47:0/665795226 shutdown_connections 2022-06-29T15:50:36.524 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.515+0000 7f58fd517700 1 --2- 172.21.15.47:0/665795226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f58f80b5d90 0x7f58f812b3d0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:36.525 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.515+0000 7f58fd517700 1 --2- 172.21.15.47:0/665795226 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f58f80676a0 0x7f58f812aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:36.525 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.515+0000 7f58fd517700 1 --2- 172.21.15.47:0/665795226 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f58dc053950 0x7f58dc055e10 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:36.525 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.515+0000 7f58fd517700 1 --2- 172.21.15.47:0/665795226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58f805c850 0x7f58f8142100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:36.526 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.515+0000 7f58fd517700 1 -- 172.21.15.47:0/665795226 >> 172.21.15.47:0/665795226 conn(0x7f58f8060e00 msgr2=0x7f58f81403e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:36.526 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.515+0000 7f58fd517700 1 -- 172.21.15.47:0/665795226 shutdown_connections 2022-06-29T15:50:36.527 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.515+0000 7f58fd517700 1 -- 172.21.15.47:0/665795226 wait complete. 2022-06-29T15:50:36.534 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-06-29T15:50:36.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.631+0000 7f415d0c4700 1 Processor -- start 2022-06-29T15:50:36.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.631+0000 7f415d0c4700 1 -- start start 2022-06-29T15:50:36.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.631+0000 7f415d0c4700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f415805f0b0 0x7f41580b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:36.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.631+0000 7f415d0c4700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f41580b1260 0x7f415814b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:36.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.631+0000 7f415d0c4700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41580b0740 con 0x7f41580b2120 2022-06-29T15:50:36.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.631+0000 7f415d0c4700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4158147460 con 0x7f415805f0b0 2022-06-29T15:50:36.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.631+0000 7f415d0c4700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41580b0970 con 0x7f41580b1260 2022-06-29T15:50:36.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.631+0000 7f415759e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f41580b1260 0x7f415814b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:36.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.631+0000 7f415659c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f415805f0b0 0x7f41580b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:36.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.631+0000 7f415759e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f41580b1260 0x7f415814b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:49796/0 (socket says 172.21.15.47:49796) 2022-06-29T15:50:36.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.631+0000 7f415759e700 1 -- 172.21.15.47:0/1022259873 learned_addr learned my addr 172.21.15.47:0/1022259873 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:36.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.631+0000 7f415659c700 1 -- 172.21.15.47:0/1022259873 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f41580b1260 msgr2=0x7f415814b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:36.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.631+0000 7f415659c700 1 --2- 172.21.15.47:0/1022259873 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f41580b1260 0x7f415814b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:36.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.631+0000 7f415659c700 1 -- 172.21.15.47:0/1022259873 >> v1:172.21.15.64:6789/0 conn(0x7f41580b2120 legacy=0x7f4158147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:36.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.631+0000 7f415659c700 1 -- 172.21.15.47:0/1022259873 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41580b0c10 con 0x7f415805f0b0 2022-06-29T15:50:36.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.631+0000 7f415659c700 1 --2- 172.21.15.47:0/1022259873 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f415805f0b0 0x7f41580b2500 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7f4158141a70 tx=0x7f4148006df0).ready entity=mon.1 client_cookie=b5bfd6bdb45d286 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:36.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.631+0000 7f4155d9b700 1 -- 172.21.15.47:0/1022259873 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f414800d570 con 0x7f415805f0b0 2022-06-29T15:50:36.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.631+0000 7f4155d9b700 1 -- 172.21.15.47:0/1022259873 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4148000e90 con 0x7f415805f0b0 2022-06-29T15:50:36.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.631+0000 7f4155d9b700 1 -- 172.21.15.47:0/1022259873 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f414800d8c0 con 0x7f415805f0b0 2022-06-29T15:50:36.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.631+0000 7f415d0c4700 1 -- 172.21.15.47:0/1022259873 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f415805f0b0 msgr2=0x7f41580b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:36.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.631+0000 7f415d0c4700 1 --2- 172.21.15.47:0/1022259873 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f415805f0b0 0x7f41580b2500 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7f4158141a70 tx=0x7f4148006df0).stop 2022-06-29T15:50:36.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.631+0000 7f415d0c4700 1 -- 172.21.15.47:0/1022259873 shutdown_connections 2022-06-29T15:50:36.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.631+0000 7f415d0c4700 1 --2- 172.21.15.47:0/1022259873 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f41580b1260 0x7f415814b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:36.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.631+0000 7f415d0c4700 1 --2- 172.21.15.47:0/1022259873 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f415805f0b0 0x7f41580b2500 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:36.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.631+0000 7f415d0c4700 1 -- 172.21.15.47:0/1022259873 >> 172.21.15.47:0/1022259873 conn(0x7f41580626f0 msgr2=0x7f4158062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:36.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.631+0000 7f415d0c4700 1 -- 172.21.15.47:0/1022259873 shutdown_connections 2022-06-29T15:50:36.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.631+0000 7f415d0c4700 1 -- 172.21.15.47:0/1022259873 wait complete. 2022-06-29T15:50:36.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.635+0000 7f415d0c4700 1 Processor -- start 2022-06-29T15:50:36.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.635+0000 7f415d0c4700 1 -- start start 2022-06-29T15:50:36.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.635+0000 7f415d0c4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41580b1260 0x7f4158064480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:36.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.635+0000 7f415d0c4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41580b2120 0x7f41580ae9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:36.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.635+0000 7f415d0c4700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4158142230 0x7f41580aeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:36.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.635+0000 7f415d0c4700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4158142fa0 con 0x7f41580b2120 2022-06-29T15:50:36.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.635+0000 7f415d0c4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41581431d0 con 0x7f41580b1260 2022-06-29T15:50:36.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.635+0000 7f415d0c4700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41581568e0 con 0x7f4158142230 2022-06-29T15:50:36.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.635+0000 7f415659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41580b2120 0x7f41580ae9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:36.651 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.635+0000 7f415759e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4158142230 0x7f41580aeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:36.651 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.635+0000 7f4156d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41580b1260 0x7f4158064480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:36.651 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.635+0000 7f415659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41580b2120 0x7f41580ae9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52700/0 (socket says 172.21.15.47:52700) 2022-06-29T15:50:36.651 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.635+0000 7f415759e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4158142230 0x7f41580aeeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:49800/0 (socket says 172.21.15.47:49800) 2022-06-29T15:50:36.652 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.635+0000 7f415659c700 1 -- 172.21.15.47:0/274561330 learned_addr learned my addr 172.21.15.47:0/274561330 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:36.652 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.635+0000 7f4156d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41580b1260 0x7f4158064480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:58400/0 (socket says 172.21.15.47:58400) 2022-06-29T15:50:36.652 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.635+0000 7f415659c700 1 -- 172.21.15.47:0/274561330 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4158142230 msgr2=0x7f41580aeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:36.652 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.635+0000 7f415659c700 1 --2- 172.21.15.47:0/274561330 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4158142230 0x7f41580aeeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:36.653 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.635+0000 7f415659c700 1 -- 172.21.15.47:0/274561330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41580b1260 msgr2=0x7f4158064480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:36.653 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.635+0000 7f415659c700 1 --2- 172.21.15.47:0/274561330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41580b1260 0x7f4158064480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:36.653 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.635+0000 7f415659c700 1 -- 172.21.15.47:0/274561330 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f414800b180 con 0x7f41580b2120 2022-06-29T15:50:36.653 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.635+0000 7f4156d9d700 1 --2- 172.21.15.47:0/274561330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41580b1260 0x7f4158064480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:50:36.654 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.635+0000 7f415659c700 1 --2- 172.21.15.47:0/274561330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41580b2120 0x7f41580ae9c0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 rx=0x7f4158141a70 tx=0x7f4148014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:36.654 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.635+0000 7f4147fff700 1 -- 172.21.15.47:0/274561330 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f414800dea0 con 0x7f41580b2120 2022-06-29T15:50:36.654 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.639+0000 7f4147fff700 1 -- 172.21.15.47:0/274561330 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4148013de0 con 0x7f41580b2120 2022-06-29T15:50:36.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.639+0000 7f415d0c4700 1 -- 172.21.15.47:0/274561330 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4158156bf0 con 0x7f41580b2120 2022-06-29T15:50:36.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.639+0000 7f4147fff700 1 -- 172.21.15.47:0/274561330 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f414801e5f0 con 0x7f41580b2120 2022-06-29T15:50:36.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.639+0000 7f415d0c4700 1 -- 172.21.15.47:0/274561330 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4158141e80 con 0x7f41580b2120 2022-06-29T15:50:36.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.639+0000 7f415d0c4700 1 -- 172.21.15.47:0/274561330 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4158064cb0 con 0x7f41580b2120 2022-06-29T15:50:36.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.639+0000 7f4147fff700 1 -- 172.21.15.47:0/274561330 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f414801c070 con 0x7f41580b2120 2022-06-29T15:50:36.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.639+0000 7f4147fff700 1 --2- 172.21.15.47:0/274561330 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4134053950 0x7f4134055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:36.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.639+0000 7f4147fff700 1 -- 172.21.15.47:0/274561330 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f414806c870 con 0x7f41580b2120 2022-06-29T15:50:36.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.639+0000 7f4156d9d700 1 --2- 172.21.15.47:0/274561330 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4134053950 0x7f4134055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:36.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.639+0000 7f4156d9d700 1 --2- 172.21.15.47:0/274561330 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4134053950 0x7f4134055e10 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7f414c00d000 tx=0x7f414c009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:36.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.647+0000 7f4147fff700 1 -- 172.21.15.47:0/274561330 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f41480437e0 con 0x7f41580b2120 2022-06-29T15:50:36.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.771+0000 7f415d0c4700 1 -- 172.21.15.47:0/274561330 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f41580573d0 con 0x7f41580b2120 2022-06-29T15:50:36.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.775+0000 7f4147fff700 1 -- 172.21.15.47:0/274561330 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v116) v1 ==== 84+0+143 (secure 0 0 0) 0x7f4148023020 con 0x7f41580b2120 2022-06-29T15:50:36.781 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:50:36.781 INFO:teuthology.orchestra.run.smithi047.stdout:[ 2022-06-29T15:50:36.781 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:36.782 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 1, 2022-06-29T15:50:36.782 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": "device_health_metrics" 2022-06-29T15:50:36.782 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:50:36.783 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:50:36.783 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 2, 2022-06-29T15:50:36.783 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": "rbd" 2022-06-29T15:50:36.783 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:50:36.784 INFO:teuthology.orchestra.run.smithi047.stdout:] 2022-06-29T15:50:36.786 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.775+0000 7f415d0c4700 1 -- 172.21.15.47:0/274561330 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4134053950 msgr2=0x7f4134055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:36.786 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.775+0000 7f415d0c4700 1 --2- 172.21.15.47:0/274561330 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4134053950 0x7f4134055e10 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7f414c00d000 tx=0x7f414c009040).stop 2022-06-29T15:50:36.786 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.775+0000 7f415d0c4700 1 -- 172.21.15.47:0/274561330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41580b2120 msgr2=0x7f41580ae9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:36.787 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.775+0000 7f415d0c4700 1 --2- 172.21.15.47:0/274561330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41580b2120 0x7f41580ae9c0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 rx=0x7f4158141a70 tx=0x7f4148014000).stop 2022-06-29T15:50:36.787 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.775+0000 7f415d0c4700 1 -- 172.21.15.47:0/274561330 shutdown_connections 2022-06-29T15:50:36.787 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.775+0000 7f415d0c4700 1 --2- 172.21.15.47:0/274561330 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4158142230 0x7f41580aeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:36.787 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.775+0000 7f415d0c4700 1 --2- 172.21.15.47:0/274561330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41580b2120 0x7f41580ae9c0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:36.788 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.775+0000 7f415d0c4700 1 --2- 172.21.15.47:0/274561330 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4134053950 0x7f4134055e10 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:36.788 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.775+0000 7f415d0c4700 1 --2- 172.21.15.47:0/274561330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41580b1260 0x7f4158064480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:36.788 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.775+0000 7f415d0c4700 1 -- 172.21.15.47:0/274561330 >> 172.21.15.47:0/274561330 conn(0x7f41580626f0 msgr2=0x7f41581462c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:36.788 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.775+0000 7f415d0c4700 1 -- 172.21.15.47:0/274561330 shutdown_connections 2022-06-29T15:50:36.789 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.775+0000 7f415d0c4700 1 -- 172.21.15.47:0/274561330 wait complete. 2022-06-29T15:50:36.794 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-06-29T15:50:36.794 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-06-29T15:50:36.903 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.895+0000 7fe230096700 1 Processor -- start 2022-06-29T15:50:36.903 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.895+0000 7fe230096700 1 -- start start 2022-06-29T15:50:36.904 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.899+0000 7fe230096700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe2280b30e0 0x7fe2280b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:36.905 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.899+0000 7fe230096700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe22805f7c0 con 0x7fe22814ba70 2022-06-29T15:50:36.905 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.899+0000 7fe230096700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2280b4e50 con 0x7fe228140940 2022-06-29T15:50:36.905 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.899+0000 7fe230096700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2280621e0 con 0x7fe2280b30e0 2022-06-29T15:50:36.906 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.899+0000 7fe22d631700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe2280b30e0 0x7fe2280b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:36.906 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.899+0000 7fe22d631700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe2280b30e0 0x7fe2280b3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52706/0 (socket says 172.21.15.47:52706) 2022-06-29T15:50:36.906 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.899+0000 7fe22e633700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe22814ba70 0x7fe22814de60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38262/0 (socket says 172.21.15.47:38262) 2022-06-29T15:50:36.906 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.899+0000 7fe22d631700 1 -- 172.21.15.47:0/2059556087 learned_addr learned my addr 172.21.15.47:0/2059556087 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:36.906 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.899+0000 7fe22d631700 1 -- 172.21.15.47:0/2059556087 >> v1:172.21.15.64:6789/0 conn(0x7fe228140940 legacy=0x7fe22805f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:36.907 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.899+0000 7fe22d631700 1 -- 172.21.15.47:0/2059556087 >> v1:172.21.15.47:6789/0 conn(0x7fe22814ba70 legacy=0x7fe22814de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:36.907 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.899+0000 7fe22d631700 1 -- 172.21.15.47:0/2059556087 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe228143170 con 0x7fe2280b30e0 2022-06-29T15:50:36.907 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.899+0000 7fe22d631700 1 --2- 172.21.15.47:0/2059556087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2280b30e0 0x7fe2280b3560 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 rx=0x7fe21c002f70 tx=0x7fe21c006eb0).ready entity=mon.0 client_cookie=56a525b90e0bf1f3 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:36.908 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.899+0000 7fe22ce30700 1 -- 172.21.15.47:0/2059556087 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe21c00d660 con 0x7fe2280b30e0 2022-06-29T15:50:36.909 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.899+0000 7fe22ce30700 1 -- 172.21.15.47:0/2059556087 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe21c000e90 con 0x7fe2280b30e0 2022-06-29T15:50:36.909 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.899+0000 7fe22ce30700 1 -- 172.21.15.47:0/2059556087 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe21c00d9f0 con 0x7fe2280b30e0 2022-06-29T15:50:36.909 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.899+0000 7fe230096700 1 -- 172.21.15.47:0/2059556087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2280b30e0 msgr2=0x7fe2280b3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:36.909 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.899+0000 7fe230096700 1 --2- 172.21.15.47:0/2059556087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2280b30e0 0x7fe2280b3560 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 rx=0x7fe21c002f70 tx=0x7fe21c006eb0).stop 2022-06-29T15:50:36.910 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.899+0000 7fe230096700 1 -- 172.21.15.47:0/2059556087 shutdown_connections 2022-06-29T15:50:36.910 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.899+0000 7fe230096700 1 --2- 172.21.15.47:0/2059556087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2280b30e0 0x7fe2280b3560 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:36.910 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.899+0000 7fe230096700 1 -- 172.21.15.47:0/2059556087 >> 172.21.15.47:0/2059556087 conn(0x7fe2280627d0 msgr2=0x7fe228062bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:36.910 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.899+0000 7fe230096700 1 -- 172.21.15.47:0/2059556087 shutdown_connections 2022-06-29T15:50:36.911 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.899+0000 7fe230096700 1 -- 172.21.15.47:0/2059556087 wait complete. 2022-06-29T15:50:36.911 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.903+0000 7fe230096700 1 Processor -- start 2022-06-29T15:50:36.911 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.903+0000 7fe230096700 1 -- start start 2022-06-29T15:50:36.913 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.903+0000 7fe230096700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe2280b30e0 0x7fe228138e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:36.913 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.903+0000 7fe230096700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe228140940 0x7fe228136e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:36.913 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.903+0000 7fe230096700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe22814ba70 0x7fe228137320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:36.914 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.903+0000 7fe230096700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe228139360 con 0x7fe22814ba70 2022-06-29T15:50:36.914 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.903+0000 7fe230096700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2280ad780 con 0x7fe228140940 2022-06-29T15:50:36.914 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.903+0000 7fe230096700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2280ad9b0 con 0x7fe2280b30e0 2022-06-29T15:50:36.914 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.903+0000 7fe22e633700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe22814ba70 0x7fe228137320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:36.915 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.903+0000 7fe22de32700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe2280b30e0 0x7fe228138e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:36.915 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.903+0000 7fe22e633700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe22814ba70 0x7fe228137320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52714/0 (socket says 172.21.15.47:52714) 2022-06-29T15:50:36.915 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.903+0000 7fe22e633700 1 -- 172.21.15.47:0/253193433 learned_addr learned my addr 172.21.15.47:0/253193433 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:36.915 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.903+0000 7fe22d631700 1 --2- 172.21.15.47:0/253193433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe228140940 0x7fe228136e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:36.916 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.903+0000 7fe22e633700 1 -- 172.21.15.47:0/253193433 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe2280b30e0 msgr2=0x7fe228138e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:36.916 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.903+0000 7fe22e633700 1 --2- 172.21.15.47:0/253193433 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe2280b30e0 0x7fe228138e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:36.916 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.903+0000 7fe22e633700 1 -- 172.21.15.47:0/253193433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe228140940 msgr2=0x7fe228136e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:36.916 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.903+0000 7fe22e633700 1 --2- 172.21.15.47:0/253193433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe228140940 0x7fe228136e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:36.916 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.903+0000 7fe22e633700 1 -- 172.21.15.47:0/253193433 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe21c00b1e0 con 0x7fe22814ba70 2022-06-29T15:50:36.917 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.903+0000 7fe22e633700 1 --2- 172.21.15.47:0/253193433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe22814ba70 0x7fe228137320 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 rx=0x7fe2280633e0 tx=0x7fe2200013a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:36.917 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.903+0000 7fe21affd700 1 -- 172.21.15.47:0/253193433 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe220005ae0 con 0x7fe22814ba70 2022-06-29T15:50:36.917 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.903+0000 7fe21affd700 1 -- 172.21.15.47:0/253193433 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe220005c40 con 0x7fe22814ba70 2022-06-29T15:50:36.917 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.903+0000 7fe230096700 1 -- 172.21.15.47:0/253193433 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2280adc50 con 0x7fe22814ba70 2022-06-29T15:50:36.917 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.903+0000 7fe21affd700 1 -- 172.21.15.47:0/253193433 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe2200070d0 con 0x7fe22814ba70 2022-06-29T15:50:36.918 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.903+0000 7fe230096700 1 -- 172.21.15.47:0/253193433 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2280aad00 con 0x7fe22814ba70 2022-06-29T15:50:36.918 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.907+0000 7fe230096700 1 -- 172.21.15.47:0/253193433 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2280573d0 con 0x7fe22814ba70 2022-06-29T15:50:36.918 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.907+0000 7fe21affd700 1 -- 172.21.15.47:0/253193433 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe22001a060 con 0x7fe22814ba70 2022-06-29T15:50:36.919 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.907+0000 7fe21affd700 1 --2- 172.21.15.47:0/253193433 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe2080539e0 0x7fe208055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:36.919 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.907+0000 7fe22de32700 1 --2- 172.21.15.47:0/253193433 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe2080539e0 0x7fe208055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:36.920 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.907+0000 7fe21affd700 1 -- 172.21.15.47:0/253193433 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fe22006b700 con 0x7fe22814ba70 2022-06-29T15:50:36.920 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.907+0000 7fe22de32700 1 --2- 172.21.15.47:0/253193433 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe2080539e0 0x7fe208055ea0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7fe22400d000 tx=0x7fe224009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:36.921 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:36.915+0000 7fe21affd700 1 -- 172.21.15.47:0/253193433 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe2200426f0 con 0x7fe22814ba70 2022-06-29T15:50:37.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.035+0000 7fe230096700 1 -- 172.21.15.47:0/253193433 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fe22806a0f0 con 0x7fe22814ba70 2022-06-29T15:50:37.797 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.791+0000 7fe21affd700 1 -- 172.21.15.47:0/253193433 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v117) v1 ==== 133+0+0 (secure 0 0 0) 0x7fe220018030 con 0x7fe22814ba70 2022-06-29T15:50:37.803 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.795+0000 7fe230096700 1 -- 172.21.15.47:0/253193433 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe2080539e0 msgr2=0x7fe208055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:37.803 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.795+0000 7fe230096700 1 --2- 172.21.15.47:0/253193433 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe2080539e0 0x7fe208055ea0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7fe22400d000 tx=0x7fe224009040).stop 2022-06-29T15:50:37.803 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.795+0000 7fe230096700 1 -- 172.21.15.47:0/253193433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe22814ba70 msgr2=0x7fe228137320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:37.804 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.795+0000 7fe230096700 1 --2- 172.21.15.47:0/253193433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe22814ba70 0x7fe228137320 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 rx=0x7fe2280633e0 tx=0x7fe2200013a0).stop 2022-06-29T15:50:37.804 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.795+0000 7fe230096700 1 -- 172.21.15.47:0/253193433 shutdown_connections 2022-06-29T15:50:37.805 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.795+0000 7fe230096700 1 --2- 172.21.15.47:0/253193433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe22814ba70 0x7fe228137320 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:37.805 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.795+0000 7fe230096700 1 --2- 172.21.15.47:0/253193433 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe2080539e0 0x7fe208055ea0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:37.805 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.795+0000 7fe230096700 1 --2- 172.21.15.47:0/253193433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe228140940 0x7fe228136e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:37.805 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.795+0000 7fe230096700 1 --2- 172.21.15.47:0/253193433 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe2280b30e0 0x7fe228138e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:37.805 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.795+0000 7fe230096700 1 -- 172.21.15.47:0/253193433 >> 172.21.15.47:0/253193433 conn(0x7fe2280627d0 msgr2=0x7fe228143300 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:37.806 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.795+0000 7fe230096700 1 -- 172.21.15.47:0/253193433 shutdown_connections 2022-06-29T15:50:37.806 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.795+0000 7fe230096700 1 -- 172.21.15.47:0/253193433 wait complete. 2022-06-29T15:50:37.807 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' created 2022-06-29T15:50:37.817 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-06-29T15:50:37.927 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.919+0000 7fa852e4e700 1 Processor -- start 2022-06-29T15:50:37.927 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.919+0000 7fa852e4e700 1 -- start start 2022-06-29T15:50:37.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.923+0000 7fa852e4e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa84c060ee0 0x7fa84c0612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:37.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.923+0000 7fa852e4e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa84c0b4300 0x7fa84c1471a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:37.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.923+0000 7fa852e4e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa84c067610 con 0x7fa84c066f70 2022-06-29T15:50:37.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.923+0000 7fa852e4e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa84c0673a0 con 0x7fa84c0b4300 2022-06-29T15:50:37.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.923+0000 7fa852e4e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c1467c0 con 0x7fa84c060ee0 2022-06-29T15:50:37.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.923+0000 7fa84bfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa84c0b4300 0x7fa84c1471a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:37.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.923+0000 7fa84bfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa84c0b4300 0x7fa84c1471a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52720/0 (socket says 172.21.15.47:52720) 2022-06-29T15:50:37.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.923+0000 7fa84bfff700 1 -- 172.21.15.47:0/600670986 learned_addr learned my addr 172.21.15.47:0/600670986 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:37.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.923+0000 7fa850bea700 1 --2- 172.21.15.47:0/600670986 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa84c060ee0 0x7fa84c0612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:37.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.923+0000 7fa84bfff700 1 -- 172.21.15.47:0/600670986 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa84c060ee0 msgr2=0x7fa84c0612c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:37.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.923+0000 7fa84bfff700 1 --2- 172.21.15.47:0/600670986 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa84c060ee0 0x7fa84c0612c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:37.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.923+0000 7fa84bfff700 1 -- 172.21.15.47:0/600670986 >> v1:172.21.15.64:6789/0 conn(0x7fa84c066f70 legacy=0x7fa84c1430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:37.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.923+0000 7fa84bfff700 1 -- 172.21.15.47:0/600670986 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa84c146a60 con 0x7fa84c0b4300 2022-06-29T15:50:37.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.923+0000 7fa84bfff700 1 --2- 172.21.15.47:0/600670986 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa84c0b4300 0x7fa84c1471a0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 rx=0x7fa838002f70 tx=0x7fa838006eb0).ready entity=mon.0 client_cookie=128871b62da15545 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:37.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.923+0000 7fa84b7fe700 1 -- 172.21.15.47:0/600670986 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa83800d570 con 0x7fa84c0b4300 2022-06-29T15:50:37.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.923+0000 7fa84b7fe700 1 -- 172.21.15.47:0/600670986 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa838000e90 con 0x7fa84c0b4300 2022-06-29T15:50:37.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.923+0000 7fa84b7fe700 1 -- 172.21.15.47:0/600670986 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa83800d900 con 0x7fa84c0b4300 2022-06-29T15:50:37.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.927+0000 7fa852e4e700 1 -- 172.21.15.47:0/600670986 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa84c0b4300 msgr2=0x7fa84c1471a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:37.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.927+0000 7fa852e4e700 1 --2- 172.21.15.47:0/600670986 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa84c0b4300 0x7fa84c1471a0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 rx=0x7fa838002f70 tx=0x7fa838006eb0).stop 2022-06-29T15:50:37.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.927+0000 7fa852e4e700 1 -- 172.21.15.47:0/600670986 shutdown_connections 2022-06-29T15:50:37.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.927+0000 7fa852e4e700 1 --2- 172.21.15.47:0/600670986 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa84c0b4300 0x7fa84c1471a0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:37.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.927+0000 7fa852e4e700 1 --2- 172.21.15.47:0/600670986 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa84c060ee0 0x7fa84c0612c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:37.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.927+0000 7fa852e4e700 1 -- 172.21.15.47:0/600670986 >> 172.21.15.47:0/600670986 conn(0x7fa84c05c510 msgr2=0x7fa84c05f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:37.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.927+0000 7fa852e4e700 1 -- 172.21.15.47:0/600670986 shutdown_connections 2022-06-29T15:50:37.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.927+0000 7fa852e4e700 1 -- 172.21.15.47:0/600670986 wait complete. 2022-06-29T15:50:37.938 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.927+0000 7fa852e4e700 1 Processor -- start 2022-06-29T15:50:37.938 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.927+0000 7fa852e4e700 1 -- start start 2022-06-29T15:50:37.941 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.931+0000 7fa852e4e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa84c060ee0 0x7fa84c13aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:37.941 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.931+0000 7fa852e4e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa84c066f70 0x7fa84c0b1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:37.941 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.931+0000 7fa852e4e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa84c0b4300 0x7fa84c0b1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:37.942 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.931+0000 7fa852e4e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c0b1d60 con 0x7fa84c060ee0 2022-06-29T15:50:37.942 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.931+0000 7fa852e4e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c144e70 con 0x7fa84c0b4300 2022-06-29T15:50:37.942 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.931+0000 7fa852e4e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c145570 con 0x7fa84c066f70 2022-06-29T15:50:37.942 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.931+0000 7fa850bea700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa84c060ee0 0x7fa84c13aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:37.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.931+0000 7fa850bea700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa84c060ee0 0x7fa84c13aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52724/0 (socket says 172.21.15.47:52724) 2022-06-29T15:50:37.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.931+0000 7fa850bea700 1 -- 172.21.15.47:0/2907016559 learned_addr learned my addr 172.21.15.47:0/2907016559 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:37.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.931+0000 7fa84bfff700 1 --2- 172.21.15.47:0/2907016559 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa84c066f70 0x7fa84c0b1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:37.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.931+0000 7fa8513eb700 1 --2- 172.21.15.47:0/2907016559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa84c0b4300 0x7fa84c0b1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:37.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.931+0000 7fa850bea700 1 -- 172.21.15.47:0/2907016559 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa84c066f70 msgr2=0x7fa84c0b1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:37.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.931+0000 7fa850bea700 1 --2- 172.21.15.47:0/2907016559 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa84c066f70 0x7fa84c0b1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:37.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.931+0000 7fa850bea700 1 -- 172.21.15.47:0/2907016559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa84c0b4300 msgr2=0x7fa84c0b1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:37.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.931+0000 7fa850bea700 1 --2- 172.21.15.47:0/2907016559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa84c0b4300 0x7fa84c0b1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:37.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.931+0000 7fa850bea700 1 -- 172.21.15.47:0/2907016559 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa83800b1e0 con 0x7fa84c060ee0 2022-06-29T15:50:37.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.931+0000 7fa850bea700 1 --2- 172.21.15.47:0/2907016559 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa84c060ee0 0x7fa84c13aac0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 rx=0x7fa84000c000 tx=0x7fa840006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:37.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.931+0000 7fa849ffb700 1 -- 172.21.15.47:0/2907016559 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa840010780 con 0x7fa84c060ee0 2022-06-29T15:50:37.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.931+0000 7fa849ffb700 1 -- 172.21.15.47:0/2907016559 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8400108e0 con 0x7fa84c060ee0 2022-06-29T15:50:37.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.931+0000 7fa852e4e700 1 -- 172.21.15.47:0/2907016559 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa84c145990 con 0x7fa84c060ee0 2022-06-29T15:50:37.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.931+0000 7fa849ffb700 1 -- 172.21.15.47:0/2907016559 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa840010c40 con 0x7fa84c060ee0 2022-06-29T15:50:37.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.931+0000 7fa852e4e700 1 -- 172.21.15.47:0/2907016559 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa84c145ed0 con 0x7fa84c060ee0 2022-06-29T15:50:37.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.935+0000 7fa852e4e700 1 -- 172.21.15.47:0/2907016559 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa84c145270 con 0x7fa84c060ee0 2022-06-29T15:50:37.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.935+0000 7fa849ffb700 1 -- 172.21.15.47:0/2907016559 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa84001c460 con 0x7fa84c060ee0 2022-06-29T15:50:37.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.935+0000 7fa849ffb700 1 --2- 172.21.15.47:0/2907016559 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa82c053a70 0x7fa82c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:37.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.935+0000 7fa849ffb700 1 -- 172.21.15.47:0/2907016559 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5954+0+0 (secure 0 0 0) 0x7fa84006c280 con 0x7fa84c060ee0 2022-06-29T15:50:37.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.939+0000 7fa84bfff700 1 --2- 172.21.15.47:0/2907016559 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa82c053a70 0x7fa82c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:37.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.939+0000 7fa84bfff700 1 --2- 172.21.15.47:0/2907016559 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa82c053a70 0x7fa82c055f30 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7fa83801c000 tx=0x7fa838015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:37.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:37.943+0000 7fa849ffb700 1 -- 172.21.15.47:0/2907016559 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa8400431b0 con 0x7fa84c060ee0 2022-06-29T15:50:38.078 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.071+0000 7fa852e4e700 1 -- 172.21.15.47:0/2907016559 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7fa84c0573d0 con 0x7fa84c060ee0 2022-06-29T15:50:38.158 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.151+0000 7fa849ffb700 1 -- 172.21.15.47:0/2907016559 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v118) v1 ==== 125+0+0 (secure 0 0 0) 0x7fa8400467d0 con 0x7fa84c060ee0 2022-06-29T15:50:38.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.159+0000 7fa852e4e700 1 -- 172.21.15.47:0/2907016559 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa82c053a70 msgr2=0x7fa82c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:38.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.159+0000 7fa852e4e700 1 --2- 172.21.15.47:0/2907016559 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa82c053a70 0x7fa82c055f30 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7fa83801c000 tx=0x7fa838015000).stop 2022-06-29T15:50:38.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.159+0000 7fa852e4e700 1 -- 172.21.15.47:0/2907016559 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa84c060ee0 msgr2=0x7fa84c13aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:38.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.159+0000 7fa852e4e700 1 --2- 172.21.15.47:0/2907016559 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa84c060ee0 0x7fa84c13aac0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 rx=0x7fa84000c000 tx=0x7fa840006df0).stop 2022-06-29T15:50:38.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.159+0000 7fa852e4e700 1 -- 172.21.15.47:0/2907016559 shutdown_connections 2022-06-29T15:50:38.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.159+0000 7fa852e4e700 1 --2- 172.21.15.47:0/2907016559 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa82c053a70 0x7fa82c055f30 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:38.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.159+0000 7fa852e4e700 1 --2- 172.21.15.47:0/2907016559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa84c0b4300 0x7fa84c0b1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:38.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.159+0000 7fa852e4e700 1 --2- 172.21.15.47:0/2907016559 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa84c066f70 0x7fa84c0b1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:38.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.159+0000 7fa852e4e700 1 --2- 172.21.15.47:0/2907016559 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa84c060ee0 0x7fa84c13aac0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:38.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.159+0000 7fa852e4e700 1 -- 172.21.15.47:0/2907016559 >> 172.21.15.47:0/2907016559 conn(0x7fa84c05c510 msgr2=0x7fa84c13bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:38.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.159+0000 7fa852e4e700 1 -- 172.21.15.47:0/2907016559 shutdown_connections 2022-06-29T15:50:38.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.159+0000 7fa852e4e700 1 -- 172.21.15.47:0/2907016559 wait complete. 2022-06-29T15:50:38.169 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' created 2022-06-29T15:50:38.177 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-06-29T15:50:38.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.275+0000 7fd83bfff700 1 Processor -- start 2022-06-29T15:50:38.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.275+0000 7fd83bfff700 1 -- start start 2022-06-29T15:50:38.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.275+0000 7fd83bfff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd83c09a060 0x7fd83c0b2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:38.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.275+0000 7fd83bfff700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd83c140ae0 con 0x7fd83c0b2bd0 2022-06-29T15:50:38.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.275+0000 7fd83bfff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd83c146900 con 0x7fd83c0b1e80 2022-06-29T15:50:38.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.275+0000 7fd83bfff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd83c146ba0 con 0x7fd83c09a060 2022-06-29T15:50:38.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.275+0000 7fd83affd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd83c0b2bd0 0x7fd83c099950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38286/0 (socket says 172.21.15.47:38286) 2022-06-29T15:50:38.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.275+0000 7fd83affd700 1 -- 172.21.15.47:0/1580457358 learned_addr learned my addr 172.21.15.47:0/1580457358 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:38.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.275+0000 7fd83a7fc700 1 --2- 172.21.15.47:0/1580457358 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd83c09a060 0x7fd83c0b2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:38.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.275+0000 7fd839ffb700 1 -- 172.21.15.47:0/1580457358 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2851811768 0 0) 0x7fd83c140ae0 con 0x7fd83c0b2bd0 2022-06-29T15:50:38.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.275+0000 7fd839ffb700 1 -- 172.21.15.47:0/1580457358 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd820003260 con 0x7fd83c0b2bd0 2022-06-29T15:50:38.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.275+0000 7fd839ffb700 1 -- 172.21.15.47:0/1580457358 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 419357572 0 0) 0x7fd83c146900 con 0x7fd83c0b1e80 2022-06-29T15:50:38.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.275+0000 7fd839ffb700 1 -- 172.21.15.47:0/1580457358 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd83c140ae0 con 0x7fd83c0b1e80 2022-06-29T15:50:38.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.275+0000 7fd839ffb700 1 -- 172.21.15.47:0/1580457358 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1144594342 0 0) 0x7fd820003260 con 0x7fd83c0b2bd0 2022-06-29T15:50:38.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.275+0000 7fd839ffb700 1 -- 172.21.15.47:0/1580457358 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd83c09a060 msgr2=0x7fd83c0b2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:38.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.275+0000 7fd839ffb700 1 --2- 172.21.15.47:0/1580457358 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd83c09a060 0x7fd83c0b2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:38.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.275+0000 7fd839ffb700 1 -- 172.21.15.47:0/1580457358 >> v1:172.21.15.64:6789/0 conn(0x7fd83c0b1e80 legacy=0x7fd83c0600c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:38.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.275+0000 7fd839ffb700 1 -- 172.21.15.47:0/1580457358 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd83c146e40 con 0x7fd83c0b2bd0 2022-06-29T15:50:38.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.275+0000 7fd839ffb700 1 -- 172.21.15.47:0/1580457358 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7fd82c001680 con 0x7fd83c0b2bd0 2022-06-29T15:50:38.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.275+0000 7fd839ffb700 1 -- 172.21.15.47:0/1580457358 >> v1:172.21.15.47:6789/0 conn(0x7fd83c0b2bd0 legacy=0x7fd83c099950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:38.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.279+0000 7fd839ffb700 1 --2- 172.21.15.47:0/1580457358 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd83c09a060 0x7fd820003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:38.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.279+0000 7fd839ffb700 1 --2- 172.21.15.47:0/1580457358 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd820003e20 0x7fd820004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:38.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.279+0000 7fd839ffb700 1 --2- 172.21.15.47:0/1580457358 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8200049f0 0x7fd820004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:38.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.279+0000 7fd839ffb700 1 -- 172.21.15.47:0/1580457358 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83c146ba0 con 0x7fd8200049f0 2022-06-29T15:50:38.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.279+0000 7fd839ffb700 1 -- 172.21.15.47:0/1580457358 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82000b800 con 0x7fd820003e20 2022-06-29T15:50:38.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.279+0000 7fd839ffb700 1 -- 172.21.15.47:0/1580457358 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd82000b9c0 con 0x7fd83c09a060 2022-06-29T15:50:38.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.279+0000 7fd83affd700 1 --2- 172.21.15.47:0/1580457358 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd83c09a060 0x7fd820003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:38.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.279+0000 7fd83b7fe700 1 --2- 172.21.15.47:0/1580457358 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8200049f0 0x7fd820004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:38.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.279+0000 7fd83a7fc700 1 --2- 172.21.15.47:0/1580457358 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd820003e20 0x7fd820004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:38.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.279+0000 7fd83b7fe700 1 -- 172.21.15.47:0/1580457358 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd83c09a060 msgr2=0x7fd820003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:38.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.279+0000 7fd83b7fe700 1 --2- 172.21.15.47:0/1580457358 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd83c09a060 0x7fd820003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:38.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.279+0000 7fd83b7fe700 1 -- 172.21.15.47:0/1580457358 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd820003e20 msgr2=0x7fd820004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:38.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.279+0000 7fd83b7fe700 1 --2- 172.21.15.47:0/1580457358 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd820003e20 0x7fd820004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:38.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.279+0000 7fd83b7fe700 1 -- 172.21.15.47:0/1580457358 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd820003260 con 0x7fd8200049f0 2022-06-29T15:50:38.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.279+0000 7fd83b7fe700 1 --2- 172.21.15.47:0/1580457358 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8200049f0 0x7fd820004e70 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 rx=0x7fd834001490 tx=0x7fd834007d80).ready entity=mon.0 client_cookie=ab60e69420031fc3 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:38.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.279+0000 7fd839ffb700 1 -- 172.21.15.47:0/1580457358 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd834001780 con 0x7fd8200049f0 2022-06-29T15:50:38.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.279+0000 7fd839ffb700 1 -- 172.21.15.47:0/1580457358 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd834006890 con 0x7fd8200049f0 2022-06-29T15:50:38.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.279+0000 7fd839ffb700 1 -- 172.21.15.47:0/1580457358 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd834006bf0 con 0x7fd8200049f0 2022-06-29T15:50:38.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.279+0000 7fd83bfff700 1 -- 172.21.15.47:0/1580457358 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8200049f0 msgr2=0x7fd820004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:38.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.279+0000 7fd83bfff700 1 --2- 172.21.15.47:0/1580457358 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8200049f0 0x7fd820004e70 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 rx=0x7fd834001490 tx=0x7fd834007d80).stop 2022-06-29T15:50:38.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.279+0000 7fd83b7fe700 1 -- 172.21.15.47:0/1580457358 reap_dead start 2022-06-29T15:50:38.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.279+0000 7fd83bfff700 1 -- 172.21.15.47:0/1580457358 shutdown_connections 2022-06-29T15:50:38.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.279+0000 7fd83bfff700 1 -- 172.21.15.47:0/1580457358 >> 172.21.15.47:0/1580457358 conn(0x7fd83c062df0 msgr2=0x7fd83c0631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:38.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.279+0000 7fd83bfff700 1 -- 172.21.15.47:0/1580457358 shutdown_connections 2022-06-29T15:50:38.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.279+0000 7fd83bfff700 1 -- 172.21.15.47:0/1580457358 wait complete. 2022-06-29T15:50:38.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.279+0000 7fd83bfff700 1 Processor -- start 2022-06-29T15:50:38.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.279+0000 7fd83bfff700 1 -- start start 2022-06-29T15:50:38.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.279+0000 7fd83bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd83c0af370 0x7fd83c152830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:38.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.279+0000 7fd83bfff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd83c0b5eb0 0x7fd83c152d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:38.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.279+0000 7fd83bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd83c141380 0x7fd83c141800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:38.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.279+0000 7fd83bfff700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83c0b52c0 con 0x7fd83c0af370 2022-06-29T15:50:38.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.279+0000 7fd83bfff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83c0af750 con 0x7fd83c141380 2022-06-29T15:50:38.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.279+0000 7fd83bfff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd83c1523d0 con 0x7fd83c0b5eb0 2022-06-29T15:50:38.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.279+0000 7fd83affd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd83c0af370 0x7fd83c152830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:38.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.279+0000 7fd83affd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd83c0af370 0x7fd83c152830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52744/0 (socket says 172.21.15.47:52744) 2022-06-29T15:50:38.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.279+0000 7fd83affd700 1 -- 172.21.15.47:0/4018143799 learned_addr learned my addr 172.21.15.47:0/4018143799 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:38.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.283+0000 7fd83a7fc700 1 --2- 172.21.15.47:0/4018143799 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd83c0b5eb0 0x7fd83c152d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:38.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.283+0000 7fd83b7fe700 1 --2- 172.21.15.47:0/4018143799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd83c141380 0x7fd83c141800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:38.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.283+0000 7fd83affd700 1 -- 172.21.15.47:0/4018143799 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd83c0b5eb0 msgr2=0x7fd83c152d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:38.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.283+0000 7fd83affd700 1 --2- 172.21.15.47:0/4018143799 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd83c0b5eb0 0x7fd83c152d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:38.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.283+0000 7fd83affd700 1 -- 172.21.15.47:0/4018143799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd83c141380 msgr2=0x7fd83c141800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:38.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.283+0000 7fd83affd700 1 --2- 172.21.15.47:0/4018143799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd83c141380 0x7fd83c141800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:38.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.283+0000 7fd83affd700 1 -- 172.21.15.47:0/4018143799 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd83c153210 con 0x7fd83c0af370 2022-06-29T15:50:38.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.283+0000 7fd83affd700 1 --2- 172.21.15.47:0/4018143799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd83c0af370 0x7fd83c152830 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 rx=0x7fd82c00d000 tx=0x7fd83c0b16f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:38.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.283+0000 7fd82bfff700 1 -- 172.21.15.47:0/4018143799 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd82c012540 con 0x7fd83c0af370 2022-06-29T15:50:38.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.283+0000 7fd82bfff700 1 -- 172.21.15.47:0/4018143799 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd82c0126a0 con 0x7fd83c0af370 2022-06-29T15:50:38.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.283+0000 7fd83bfff700 1 -- 172.21.15.47:0/4018143799 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd83c0b5710 con 0x7fd83c0af370 2022-06-29T15:50:38.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.283+0000 7fd82bfff700 1 -- 172.21.15.47:0/4018143799 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd82c012a00 con 0x7fd83c0af370 2022-06-29T15:50:38.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.283+0000 7fd83bfff700 1 -- 172.21.15.47:0/4018143799 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd83c0b5c50 con 0x7fd83c0af370 2022-06-29T15:50:38.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.283+0000 7fd83bfff700 1 -- 172.21.15.47:0/4018143799 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd83c0573d0 con 0x7fd83c0af370 2022-06-29T15:50:38.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.283+0000 7fd82bfff700 1 -- 172.21.15.47:0/4018143799 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd82c012b60 con 0x7fd83c0af370 2022-06-29T15:50:38.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.283+0000 7fd82bfff700 1 --2- 172.21.15.47:0/4018143799 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd818053b00 0x7fd818055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:38.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.283+0000 7fd82bfff700 1 -- 172.21.15.47:0/4018143799 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6304+0+0 (secure 0 0 0) 0x7fd82c06b9b0 con 0x7fd83c0af370 2022-06-29T15:50:38.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.283+0000 7fd83a7fc700 1 --2- 172.21.15.47:0/4018143799 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd818053b00 0x7fd818055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:38.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.287+0000 7fd83a7fc700 1 --2- 172.21.15.47:0/4018143799 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd818053b00 0x7fd818055fc0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7fd83000b000 tx=0x7fd830006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:38.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.291+0000 7fd82bfff700 1 -- 172.21.15.47:0/4018143799 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd82c042800 con 0x7fd83c0af370 2022-06-29T15:50:38.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.455+0000 7fd83bfff700 1 -- 172.21.15.47:0/4018143799 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fd83c152560 con 0x7fd83c0af370 2022-06-29T15:50:38.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.463+0000 7fd82bfff700 1 -- 172.21.15.47:0/4018143799 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 21 and data pool 22 v55) v1 ==== 189+0+0 (secure 0 0 0) 0x7fd82c019020 con 0x7fd83c0af370 2022-06-29T15:50:38.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.467+0000 7fd83bfff700 1 -- 172.21.15.47:0/4018143799 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd818053b00 msgr2=0x7fd818055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:38.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.467+0000 7fd83bfff700 1 --2- 172.21.15.47:0/4018143799 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd818053b00 0x7fd818055fc0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7fd83000b000 tx=0x7fd830006df0).stop 2022-06-29T15:50:38.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.467+0000 7fd83bfff700 1 -- 172.21.15.47:0/4018143799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd83c0af370 msgr2=0x7fd83c152830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:38.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.467+0000 7fd83bfff700 1 --2- 172.21.15.47:0/4018143799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd83c0af370 0x7fd83c152830 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 rx=0x7fd82c00d000 tx=0x7fd83c0b16f0).stop 2022-06-29T15:50:38.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.467+0000 7fd83bfff700 1 -- 172.21.15.47:0/4018143799 shutdown_connections 2022-06-29T15:50:38.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.467+0000 7fd83bfff700 1 --2- 172.21.15.47:0/4018143799 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd818053b00 0x7fd818055fc0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:38.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.467+0000 7fd83bfff700 1 --2- 172.21.15.47:0/4018143799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd83c141380 0x7fd83c141800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:38.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.467+0000 7fd83bfff700 1 --2- 172.21.15.47:0/4018143799 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd83c0b5eb0 0x7fd83c152d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:38.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.467+0000 7fd83bfff700 1 --2- 172.21.15.47:0/4018143799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd83c0af370 0x7fd83c152830 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:38.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.467+0000 7fd83bfff700 1 -- 172.21.15.47:0/4018143799 >> 172.21.15.47:0/4018143799 conn(0x7fd83c062df0 msgr2=0x7fd83c05f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:38.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.467+0000 7fd83bfff700 1 -- 172.21.15.47:0/4018143799 shutdown_connections 2022-06-29T15:50:38.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.467+0000 7fd83bfff700 1 -- 172.21.15.47:0/4018143799 wait complete. 2022-06-29T15:50:38.478 INFO:teuthology.orchestra.run.smithi047.stderr:new fs with metadata pool 21 and data pool 22 2022-06-29T15:50:38.489 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:50:38.580 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.575+0000 7f6705f30700 1 Processor -- start 2022-06-29T15:50:38.581 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.575+0000 7f6705f30700 1 -- start start 2022-06-29T15:50:38.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.575+0000 7f6705f30700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f67000b0f30 0x7f67000606e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:38.594 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.575+0000 7f6705f30700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f67001408e0 0x7f6700139a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:38.594 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.575+0000 7f6705f30700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6700140d50 con 0x7f67000619b0 2022-06-29T15:50:38.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.575+0000 7f6705f30700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67000b2190 con 0x7f67000b0f30 2022-06-29T15:50:38.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.575+0000 7f6705f30700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6700061d90 con 0x7f67001408e0 2022-06-29T15:50:38.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.575+0000 7f66f7fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f67000b0f30 0x7f67000606e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:38.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.575+0000 7f66f7fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f67000b0f30 0x7f67000606e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52756/0 (socket says 172.21.15.47:52756) 2022-06-29T15:50:38.597 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.575+0000 7f66fffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f67001408e0 0x7f6700139a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:38.597 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.575+0000 7f66f7fff700 1 -- 172.21.15.47:0/420713440 learned_addr learned my addr 172.21.15.47:0/420713440 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:38.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.575+0000 7f66f7fff700 1 -- 172.21.15.47:0/420713440 >> v2:172.21.15.64:3300/0 conn(0x7f67001408e0 msgr2=0x7f6700139a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:38.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.575+0000 7f66f7fff700 1 --2- 172.21.15.47:0/420713440 >> v2:172.21.15.64:3300/0 conn(0x7f67001408e0 0x7f6700139a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:38.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.575+0000 7f66f7fff700 1 -- 172.21.15.47:0/420713440 >> v1:172.21.15.64:6789/0 conn(0x7f67000619b0 legacy=0x7f67000b1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:38.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.575+0000 7f66f7fff700 1 -- 172.21.15.47:0/420713440 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6700060d80 con 0x7f67000b0f30 2022-06-29T15:50:38.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.579+0000 7f66f7fff700 1 --2- 172.21.15.47:0/420713440 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67000b0f30 0x7f67000606e0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 rx=0x7f66e8002f70 tx=0x7f66e8006eb0).ready entity=mon.0 client_cookie=cbbe82a661f0591c server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:38.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.579+0000 7f66feffd700 1 -- 172.21.15.47:0/420713440 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66e800d5c0 con 0x7f67000b0f30 2022-06-29T15:50:38.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.579+0000 7f66feffd700 1 -- 172.21.15.47:0/420713440 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e8000e90 con 0x7f67000b0f30 2022-06-29T15:50:38.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.579+0000 7f66feffd700 1 -- 172.21.15.47:0/420713440 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66e800d950 con 0x7f67000b0f30 2022-06-29T15:50:38.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.579+0000 7f6705f30700 1 -- 172.21.15.47:0/420713440 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67000b0f30 msgr2=0x7f67000606e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:38.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.579+0000 7f6705f30700 1 --2- 172.21.15.47:0/420713440 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67000b0f30 0x7f67000606e0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 rx=0x7f66e8002f70 tx=0x7f66e8006eb0).stop 2022-06-29T15:50:38.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.579+0000 7f6705f30700 1 -- 172.21.15.47:0/420713440 shutdown_connections 2022-06-29T15:50:38.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.579+0000 7f6705f30700 1 --2- 172.21.15.47:0/420713440 >> v2:172.21.15.64:3300/0 conn(0x7f67001408e0 0x7f6700139a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:38.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.579+0000 7f6705f30700 1 --2- 172.21.15.47:0/420713440 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67000b0f30 0x7f67000606e0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:38.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.579+0000 7f6705f30700 1 -- 172.21.15.47:0/420713440 >> 172.21.15.47:0/420713440 conn(0x7f670005c510 msgr2=0x7f6700063800 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:38.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.579+0000 7f6705f30700 1 -- 172.21.15.47:0/420713440 shutdown_connections 2022-06-29T15:50:38.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.579+0000 7f6705f30700 1 -- 172.21.15.47:0/420713440 wait complete. 2022-06-29T15:50:38.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.579+0000 7f6705f30700 1 Processor -- start 2022-06-29T15:50:38.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.579+0000 7f6705f30700 1 -- start start 2022-06-29T15:50:38.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.583+0000 7f6705f30700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67000619b0 0x7f67000b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:38.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.583+0000 7f6705f30700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f67000b0f30 0x7f67000664d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:38.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.583+0000 7f6705f30700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67001408e0 0x7f67001538f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:38.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.583+0000 7f6705f30700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6700138130 con 0x7f67000619b0 2022-06-29T15:50:38.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.583+0000 7f6705f30700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67000b3070 con 0x7f67001408e0 2022-06-29T15:50:38.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.583+0000 7f6705f30700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67000b3310 con 0x7f67000b0f30 2022-06-29T15:50:38.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.583+0000 7f66ff7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67000619b0 0x7f67000b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:38.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.583+0000 7f66ff7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67000619b0 0x7f67000b3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52760/0 (socket says 172.21.15.47:52760) 2022-06-29T15:50:38.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.583+0000 7f66ff7fe700 1 -- 172.21.15.47:0/4019595760 learned_addr learned my addr 172.21.15.47:0/4019595760 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:38.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.583+0000 7f66f7fff700 1 --2- 172.21.15.47:0/4019595760 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f67000b0f30 0x7f67000664d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:38.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.583+0000 7f66fffff700 1 --2- 172.21.15.47:0/4019595760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67001408e0 0x7f67001538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:38.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.583+0000 7f66ff7fe700 1 -- 172.21.15.47:0/4019595760 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f67000b0f30 msgr2=0x7f67000664d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:38.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.583+0000 7f66ff7fe700 1 --2- 172.21.15.47:0/4019595760 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f67000b0f30 0x7f67000664d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:38.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.583+0000 7f66ff7fe700 1 -- 172.21.15.47:0/4019595760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67001408e0 msgr2=0x7f67001538f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:38.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.583+0000 7f66ff7fe700 1 --2- 172.21.15.47:0/4019595760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67001408e0 0x7f67001538f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:38.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.583+0000 7f66ff7fe700 1 -- 172.21.15.47:0/4019595760 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e800b1e0 con 0x7f67000619b0 2022-06-29T15:50:38.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.583+0000 7f66ff7fe700 1 --2- 172.21.15.47:0/4019595760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67000619b0 0x7f67000b3a90 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 rx=0x7f67000b0e20 tx=0x7f66f0007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:38.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.583+0000 7f66fd7fa700 1 -- 172.21.15.47:0/4019595760 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66f0012400 con 0x7f67000619b0 2022-06-29T15:50:38.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.583+0000 7f66fd7fa700 1 -- 172.21.15.47:0/4019595760 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f0012560 con 0x7f67000619b0 2022-06-29T15:50:38.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.583+0000 7f6705f30700 1 -- 172.21.15.47:0/4019595760 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6700065670 con 0x7f67000619b0 2022-06-29T15:50:38.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.583+0000 7f66fd7fa700 1 -- 172.21.15.47:0/4019595760 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66f00128c0 con 0x7f67000619b0 2022-06-29T15:50:38.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.583+0000 7f6705f30700 1 -- 172.21.15.47:0/4019595760 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6700065b10 con 0x7f67000619b0 2022-06-29T15:50:38.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.583+0000 7f6705f30700 1 -- 172.21.15.47:0/4019595760 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67000573d0 con 0x7f67000619b0 2022-06-29T15:50:38.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.587+0000 7f66fd7fa700 1 -- 172.21.15.47:0/4019595760 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f66f0019030 con 0x7f67000619b0 2022-06-29T15:50:38.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.587+0000 7f66fd7fa700 1 --2- 172.21.15.47:0/4019595760 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f66e4053a10 0x7f66e4055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:38.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.587+0000 7f66fd7fa700 1 -- 172.21.15.47:0/4019595760 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f66f006b460 con 0x7f67000619b0 2022-06-29T15:50:38.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.587+0000 7f66f7fff700 1 --2- 172.21.15.47:0/4019595760 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f66e4053a10 0x7f66e4055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:38.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.587+0000 7f66f7fff700 1 --2- 172.21.15.47:0/4019595760 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f66e4053a10 0x7f66e4055ed0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7f66e801c000 tx=0x7f66e8015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:38.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.591+0000 7f66fd7fa700 1 -- 172.21.15.47:0/4019595760 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f66f00420b0 con 0x7f67000619b0 2022-06-29T15:50:38.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.723+0000 7f6705f30700 1 -- 172.21.15.47:0/4019595760 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f670006a1a0 con 0x7f67000619b0 2022-06-29T15:50:38.731 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.723+0000 7f66fd7fa700 1 -- 172.21.15.47:0/4019595760 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+16298 (secure 0 0 0) 0x7f66f00456d0 con 0x7f67000619b0 2022-06-29T15:50:38.731 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:50:38.732 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":120,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:50:38.458963+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:50:37.087143+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2022-06-29T15:50:38.125088+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:50:38.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.727+0000 7f6705f30700 1 -- 172.21.15.47:0/4019595760 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f66e4053a10 msgr2=0x7f66e4055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:38.734 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.727+0000 7f6705f30700 1 --2- 172.21.15.47:0/4019595760 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f66e4053a10 0x7f66e4055ed0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7f66e801c000 tx=0x7f66e8015000).stop 2022-06-29T15:50:38.734 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.727+0000 7f6705f30700 1 -- 172.21.15.47:0/4019595760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67000619b0 msgr2=0x7f67000b3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:38.734 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.727+0000 7f6705f30700 1 --2- 172.21.15.47:0/4019595760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67000619b0 0x7f67000b3a90 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 rx=0x7f67000b0e20 tx=0x7f66f0007c50).stop 2022-06-29T15:50:38.734 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.727+0000 7f6705f30700 1 -- 172.21.15.47:0/4019595760 shutdown_connections 2022-06-29T15:50:38.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.727+0000 7f6705f30700 1 --2- 172.21.15.47:0/4019595760 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f66e4053a10 0x7f66e4055ed0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:38.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.727+0000 7f6705f30700 1 --2- 172.21.15.47:0/4019595760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67001408e0 0x7f67001538f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:38.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.727+0000 7f6705f30700 1 --2- 172.21.15.47:0/4019595760 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f67000b0f30 0x7f67000664d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:38.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.727+0000 7f6705f30700 1 --2- 172.21.15.47:0/4019595760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67000619b0 0x7f67000b3a90 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:38.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.727+0000 7f6705f30700 1 -- 172.21.15.47:0/4019595760 >> 172.21.15.47:0/4019595760 conn(0x7f670005c510 msgr2=0x7f6700143630 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:38.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.727+0000 7f6705f30700 1 -- 172.21.15.47:0/4019595760 shutdown_connections 2022-06-29T15:50:38.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.727+0000 7f6705f30700 1 -- 172.21.15.47:0/4019595760 wait complete. 2022-06-29T15:50:38.744 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:50:38.863 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.843+0000 7f7f6da8b700 1 Processor -- start 2022-06-29T15:50:38.864 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.843+0000 7f7f6da8b700 1 -- start start 2022-06-29T15:50:38.864 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.847+0000 7f7f6da8b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7f680b7860 0x7f7f680b3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:38.865 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.847+0000 7f7f6da8b700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7f6814a710 0x7f7f6814ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:38.865 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.847+0000 7f7f6da8b700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f680aac20 con 0x7f7f680b2d00 2022-06-29T15:50:38.866 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.847+0000 7f7f6da8b700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f680ab2f0 con 0x7f7f680b7860 2022-06-29T15:50:38.866 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.847+0000 7f7f6da8b700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f680af9f0 con 0x7f7f6814a710 2022-06-29T15:50:38.866 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.847+0000 7f7f67fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7f680b2d00 0x7f7f6814a000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38322/0 (socket says 172.21.15.47:38322) 2022-06-29T15:50:38.867 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.847+0000 7f7f67fff700 1 -- 172.21.15.47:0/3429753142 learned_addr learned my addr 172.21.15.47:0/3429753142 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:38.867 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.847+0000 7f7f6ca89700 1 --2- 172.21.15.47:0/3429753142 >> v2:172.21.15.64:3300/0 conn(0x7f7f680b7860 0x7f7f680b3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:38.867 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.847+0000 7f7f677fe700 1 -- 172.21.15.47:0/3429753142 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 375840377 0 0) 0x7f7f680aac20 con 0x7f7f680b2d00 2022-06-29T15:50:38.868 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.847+0000 7f7f677fe700 1 -- 172.21.15.47:0/3429753142 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f54003260 con 0x7f7f680b2d00 2022-06-29T15:50:38.868 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.847+0000 7f7f6d28a700 1 --2- 172.21.15.47:0/3429753142 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7f6814a710 0x7f7f6814ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:38.869 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.847+0000 7f7f677fe700 1 -- 172.21.15.47:0/3429753142 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2790336590 0 0) 0x7f7f54003260 con 0x7f7f680b2d00 2022-06-29T15:50:38.869 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.847+0000 7f7f677fe700 1 -- 172.21.15.47:0/3429753142 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7f6814a710 msgr2=0x7f7f6814ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:38.869 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.847+0000 7f7f677fe700 1 --2- 172.21.15.47:0/3429753142 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7f6814a710 0x7f7f6814ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:38.870 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.847+0000 7f7f677fe700 1 -- 172.21.15.47:0/3429753142 >> v2:172.21.15.64:3300/0 conn(0x7f7f680b7860 msgr2=0x7f7f680b3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:38.870 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.847+0000 7f7f677fe700 1 --2- 172.21.15.47:0/3429753142 >> v2:172.21.15.64:3300/0 conn(0x7f7f680b7860 0x7f7f680b3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:38.871 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.847+0000 7f7f677fe700 1 -- 172.21.15.47:0/3429753142 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f680b7c40 con 0x7f7f680b2d00 2022-06-29T15:50:38.871 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.847+0000 7f7f677fe700 1 -- 172.21.15.47:0/3429753142 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7f7f5c001660 con 0x7f7f680b2d00 2022-06-29T15:50:38.871 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.847+0000 7f7f677fe700 1 -- 172.21.15.47:0/3429753142 >> v1:172.21.15.47:6789/0 conn(0x7f7f680b2d00 legacy=0x7f7f6814a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:38.872 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.847+0000 7f7f677fe700 1 --2- 172.21.15.47:0/3429753142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f54003780 0x7f7f54003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:38.872 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.847+0000 7f7f677fe700 1 --2- 172.21.15.47:0/3429753142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f54004500 0x7f7f54004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:38.873 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.847+0000 7f7f677fe700 1 --2- 172.21.15.47:0/3429753142 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7f54004e70 0x7f7f5400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:38.873 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.847+0000 7f7f677fe700 1 -- 172.21.15.47:0/3429753142 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f680af9f0 con 0x7f7f54004500 2022-06-29T15:50:38.874 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.847+0000 7f7f677fe700 1 -- 172.21.15.47:0/3429753142 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5400bc40 con 0x7f7f54003780 2022-06-29T15:50:38.874 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.847+0000 7f7f677fe700 1 -- 172.21.15.47:0/3429753142 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5400bd80 con 0x7f7f54004e70 2022-06-29T15:50:38.874 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.847+0000 7f7f67fff700 1 --2- 172.21.15.47:0/3429753142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f54004500 0x7f7f54004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:38.875 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.847+0000 7f7f6ca89700 1 --2- 172.21.15.47:0/3429753142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f54003780 0x7f7f54003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:38.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.847+0000 7f7f6d28a700 1 --2- 172.21.15.47:0/3429753142 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7f54004e70 0x7f7f5400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:38.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.847+0000 7f7f67fff700 1 -- 172.21.15.47:0/3429753142 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7f54004e70 msgr2=0x7f7f5400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:38.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.847+0000 7f7f67fff700 1 --2- 172.21.15.47:0/3429753142 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7f54004e70 0x7f7f5400b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:38.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.847+0000 7f7f67fff700 1 -- 172.21.15.47:0/3429753142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f54003780 msgr2=0x7f7f54003ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:38.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.847+0000 7f7f67fff700 1 --2- 172.21.15.47:0/3429753142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f54003780 0x7f7f54003ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:38.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.847+0000 7f7f67fff700 1 -- 172.21.15.47:0/3429753142 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f54003260 con 0x7f7f54004500 2022-06-29T15:50:38.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.847+0000 7f7f67fff700 1 --2- 172.21.15.47:0/3429753142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f54004500 0x7f7f54004980 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 rx=0x7f7f54004470 tx=0x7f7f5c001e10).ready entity=mon.0 client_cookie=7541956df46e3c51 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:38.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.847+0000 7f7f677fe700 1 -- 172.21.15.47:0/3429753142 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f5c0071a0 con 0x7f7f54004500 2022-06-29T15:50:38.880 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.847+0000 7f7f677fe700 1 -- 172.21.15.47:0/3429753142 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f5c007300 con 0x7f7f54004500 2022-06-29T15:50:38.880 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.847+0000 7f7f677fe700 1 -- 172.21.15.47:0/3429753142 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f5c00c8b0 con 0x7f7f54004500 2022-06-29T15:50:38.880 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.847+0000 7f7f6da8b700 1 -- 172.21.15.47:0/3429753142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f54004500 msgr2=0x7f7f54004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:38.880 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.847+0000 7f7f6da8b700 1 --2- 172.21.15.47:0/3429753142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f54004500 0x7f7f54004980 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 rx=0x7f7f54004470 tx=0x7f7f5c001e10).stop 2022-06-29T15:50:38.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.847+0000 7f7f6d28a700 1 -- 172.21.15.47:0/3429753142 reap_dead start 2022-06-29T15:50:38.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.847+0000 7f7f6da8b700 1 -- 172.21.15.47:0/3429753142 shutdown_connections 2022-06-29T15:50:38.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.847+0000 7f7f6da8b700 1 -- 172.21.15.47:0/3429753142 >> 172.21.15.47:0/3429753142 conn(0x7f7f6805f480 msgr2=0x7f7f680647d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:38.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.847+0000 7f7f6da8b700 1 -- 172.21.15.47:0/3429753142 shutdown_connections 2022-06-29T15:50:38.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.847+0000 7f7f6da8b700 1 -- 172.21.15.47:0/3429753142 wait complete. 2022-06-29T15:50:38.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.851+0000 7f7f6da8b700 1 Processor -- start 2022-06-29T15:50:38.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.851+0000 7f7f6da8b700 1 -- start start 2022-06-29T15:50:38.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.851+0000 7f7f6da8b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f54004500 0x7f7f68061820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:38.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.851+0000 7f7f6da8b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f68144f10 0x7f7f68061d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:38.883 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.851+0000 7f7f6da8b700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7f68159fc0 0x7f7f6815a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:38.883 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.851+0000 7f7f6da8b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6805f900 con 0x7f7f54004500 2022-06-29T15:50:38.883 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.851+0000 7f7f6da8b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f680b7230 con 0x7f7f68144f10 2022-06-29T15:50:38.883 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.851+0000 7f7f6da8b700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f680b5950 con 0x7f7f68159fc0 2022-06-29T15:50:38.883 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.851+0000 7f7f6ca89700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f54004500 0x7f7f68061820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:38.884 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.851+0000 7f7f6ca89700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f54004500 0x7f7f68061820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52780/0 (socket says 172.21.15.47:52780) 2022-06-29T15:50:38.884 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.851+0000 7f7f6ca89700 1 -- 172.21.15.47:0/2956452797 learned_addr learned my addr 172.21.15.47:0/2956452797 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:38.884 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.851+0000 7f7f6d28a700 1 --2- 172.21.15.47:0/2956452797 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7f68159fc0 0x7f7f6815a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:38.884 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.851+0000 7f7f67fff700 1 --2- 172.21.15.47:0/2956452797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f68144f10 0x7f7f68061d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:38.885 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.851+0000 7f7f6ca89700 1 -- 172.21.15.47:0/2956452797 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7f68159fc0 msgr2=0x7f7f6815a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:38.885 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.851+0000 7f7f6ca89700 1 --2- 172.21.15.47:0/2956452797 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7f68159fc0 0x7f7f6815a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:38.885 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.851+0000 7f7f6ca89700 1 -- 172.21.15.47:0/2956452797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f68144f10 msgr2=0x7f7f68061d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:38.885 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.851+0000 7f7f6ca89700 1 --2- 172.21.15.47:0/2956452797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f68144f10 0x7f7f68061d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:38.885 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.851+0000 7f7f6ca89700 1 -- 172.21.15.47:0/2956452797 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f68144070 con 0x7f7f54004500 2022-06-29T15:50:38.886 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.851+0000 7f7f6ca89700 1 --2- 172.21.15.47:0/2956452797 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f54004500 0x7f7f68061820 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 rx=0x7f7f5800b000 tx=0x7f7f58006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:38.886 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.851+0000 7f7f65ffb700 1 -- 172.21.15.47:0/2956452797 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f5800f940 con 0x7f7f54004500 2022-06-29T15:50:38.886 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.851+0000 7f7f6da8b700 1 -- 172.21.15.47:0/2956452797 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f680b55a0 con 0x7f7f54004500 2022-06-29T15:50:38.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.851+0000 7f7f65ffb700 1 -- 172.21.15.47:0/2956452797 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f5800faa0 con 0x7f7f54004500 2022-06-29T15:50:38.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.851+0000 7f7f65ffb700 1 -- 172.21.15.47:0/2956452797 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f5800fe00 con 0x7f7f54004500 2022-06-29T15:50:38.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.851+0000 7f7f6da8b700 1 -- 172.21.15.47:0/2956452797 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f68062200 con 0x7f7f54004500 2022-06-29T15:50:38.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.855+0000 7f7f6da8b700 1 -- 172.21.15.47:0/2956452797 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f680573d0 con 0x7f7f54004500 2022-06-29T15:50:38.888 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.855+0000 7f7f65ffb700 1 -- 172.21.15.47:0/2956452797 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7f58014440 con 0x7f7f54004500 2022-06-29T15:50:38.888 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.855+0000 7f7f65ffb700 1 --2- 172.21.15.47:0/2956452797 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7f48053aa0 0x7f7f48055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:38.888 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.855+0000 7f7f65ffb700 1 -- 172.21.15.47:0/2956452797 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f7f5806cbf0 con 0x7f7f54004500 2022-06-29T15:50:38.888 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.855+0000 7f7f67fff700 1 --2- 172.21.15.47:0/2956452797 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7f48053aa0 0x7f7f48055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:38.888 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.855+0000 7f7f67fff700 1 --2- 172.21.15.47:0/2956452797 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7f48053aa0 0x7f7f48055f60 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7f7f5c01a000 tx=0x7f7f5c015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:38.889 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.859+0000 7f7f65ffb700 1 -- 172.21.15.47:0/2956452797 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7f580437c0 con 0x7f7f54004500 2022-06-29T15:50:38.990 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.983+0000 7f7f6da8b700 1 -- 172.21.15.47:0/2956452797 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7f68068830 con 0x7f7f54004500 2022-06-29T15:50:38.991 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.987+0000 7f7f65ffb700 1 -- 172.21.15.47:0/2956452797 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+16298 (secure 0 0 0) 0x7f7f58018070 con 0x7f7f54004500 2022-06-29T15:50:38.992 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:50:38.992 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":120,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:50:38.458963+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:50:37.087143+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2022-06-29T15:50:38.125088+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:50:38.994 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.987+0000 7f7f6da8b700 1 -- 172.21.15.47:0/2956452797 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7f48053aa0 msgr2=0x7f7f48055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:38.994 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.987+0000 7f7f6da8b700 1 --2- 172.21.15.47:0/2956452797 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7f48053aa0 0x7f7f48055f60 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7f7f5c01a000 tx=0x7f7f5c015000).stop 2022-06-29T15:50:38.994 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.987+0000 7f7f6da8b700 1 -- 172.21.15.47:0/2956452797 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f54004500 msgr2=0x7f7f68061820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:38.995 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.987+0000 7f7f6da8b700 1 --2- 172.21.15.47:0/2956452797 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f54004500 0x7f7f68061820 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 rx=0x7f7f5800b000 tx=0x7f7f58006df0).stop 2022-06-29T15:50:38.995 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.987+0000 7f7f6da8b700 1 -- 172.21.15.47:0/2956452797 shutdown_connections 2022-06-29T15:50:38.995 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.987+0000 7f7f6da8b700 1 --2- 172.21.15.47:0/2956452797 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7f68159fc0 0x7f7f6815a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:38.996 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.987+0000 7f7f6da8b700 1 --2- 172.21.15.47:0/2956452797 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7f48053aa0 0x7f7f48055f60 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:38.996 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.987+0000 7f7f6da8b700 1 --2- 172.21.15.47:0/2956452797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f68144f10 0x7f7f68061d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:38.996 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.987+0000 7f7f6da8b700 1 --2- 172.21.15.47:0/2956452797 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f54004500 0x7f7f68061820 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:38.996 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.987+0000 7f7f6da8b700 1 -- 172.21.15.47:0/2956452797 >> 172.21.15.47:0/2956452797 conn(0x7f7f6805f480 msgr2=0x7f7f6805cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:38.997 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.987+0000 7f7f6da8b700 1 -- 172.21.15.47:0/2956452797 shutdown_connections 2022-06-29T15:50:38.997 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:38.987+0000 7f7f6da8b700 1 -- 172.21.15.47:0/2956452797 wait complete. 2022-06-29T15:50:39.005 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-06-29T15:50:39.118 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.111+0000 7fe95759e700 1 Processor -- start 2022-06-29T15:50:39.119 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.111+0000 7fe95759e700 1 -- start start 2022-06-29T15:50:39.123 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.115+0000 7fe95759e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe958060a40 0x7fe958142ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:39.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.115+0000 7fe95759e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe9581409b0 0x7fe958061620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:39.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.115+0000 7fe95759e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9580af660 con 0x7fe95805fd00 2022-06-29T15:50:39.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.115+0000 7fe95759e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe958143fc0 con 0x7fe958060a40 2022-06-29T15:50:39.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.115+0000 7fe95759e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9581437e0 con 0x7fe9581409b0 2022-06-29T15:50:39.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.115+0000 7fe956d9d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe9581409b0 0x7fe958061620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:39.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.115+0000 7fe956d9d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe9581409b0 0x7fe958061620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:49892/0 (socket says 172.21.15.47:49892) 2022-06-29T15:50:39.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.115+0000 7fe95659c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe958060a40 0x7fe958142ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:39.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.115+0000 7fe956d9d700 1 -- 172.21.15.47:0/445240444 learned_addr learned my addr 172.21.15.47:0/445240444 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:39.127 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.115+0000 7fe956d9d700 1 -- 172.21.15.47:0/445240444 >> v2:172.21.15.64:3300/0 conn(0x7fe958060a40 msgr2=0x7fe958142ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:39.127 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.115+0000 7fe956d9d700 1 --2- 172.21.15.47:0/445240444 >> v2:172.21.15.64:3300/0 conn(0x7fe958060a40 0x7fe958142ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:39.127 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.115+0000 7fe956d9d700 1 -- 172.21.15.47:0/445240444 >> v1:172.21.15.64:6789/0 conn(0x7fe95805fd00 legacy=0x7fe9581402a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:39.127 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.115+0000 7fe956d9d700 1 -- 172.21.15.47:0/445240444 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe958143a80 con 0x7fe9581409b0 2022-06-29T15:50:39.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.115+0000 7fe956d9d700 1 --2- 172.21.15.47:0/445240444 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe9581409b0 0x7fe958061620 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7fe9500032f0 tx=0x7fe950009290).ready entity=mon.2 client_cookie=1391f0add658f206 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:39.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.115+0000 7fe955d9b700 1 -- 172.21.15.47:0/445240444 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe95000b8a0 con 0x7fe9581409b0 2022-06-29T15:50:39.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.115+0000 7fe955d9b700 1 -- 172.21.15.47:0/445240444 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe95000ba00 con 0x7fe9581409b0 2022-06-29T15:50:39.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.115+0000 7fe955d9b700 1 -- 172.21.15.47:0/445240444 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe95000bb70 con 0x7fe9581409b0 2022-06-29T15:50:39.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.115+0000 7fe95759e700 1 -- 172.21.15.47:0/445240444 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe9581409b0 msgr2=0x7fe958061620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:39.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.115+0000 7fe95759e700 1 --2- 172.21.15.47:0/445240444 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe9581409b0 0x7fe958061620 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7fe9500032f0 tx=0x7fe950009290).stop 2022-06-29T15:50:39.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.115+0000 7fe95759e700 1 -- 172.21.15.47:0/445240444 shutdown_connections 2022-06-29T15:50:39.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.115+0000 7fe95759e700 1 --2- 172.21.15.47:0/445240444 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe9581409b0 0x7fe958061620 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:39.130 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.115+0000 7fe95759e700 1 --2- 172.21.15.47:0/445240444 >> v2:172.21.15.64:3300/0 conn(0x7fe958060a40 0x7fe958142ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:39.130 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.115+0000 7fe95759e700 1 -- 172.21.15.47:0/445240444 >> 172.21.15.47:0/445240444 conn(0x7fe9580b6430 msgr2=0x7fe9580b6810 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:39.130 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.115+0000 7fe95759e700 1 -- 172.21.15.47:0/445240444 shutdown_connections 2022-06-29T15:50:39.130 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.115+0000 7fe95759e700 1 -- 172.21.15.47:0/445240444 wait complete. 2022-06-29T15:50:39.133 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.119+0000 7fe95759e700 1 Processor -- start 2022-06-29T15:50:39.134 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.119+0000 7fe95759e700 1 -- start start 2022-06-29T15:50:39.134 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.119+0000 7fe95759e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe95805fd00 0x7fe9580aca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:39.134 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.119+0000 7fe95759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe958060a40 0x7fe958146870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:39.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.119+0000 7fe95759e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe9581409b0 0x7fe958146d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:39.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.119+0000 7fe95759e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9580aa980 con 0x7fe95805fd00 2022-06-29T15:50:39.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.119+0000 7fe95759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9580abb30 con 0x7fe958060a40 2022-06-29T15:50:39.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.119+0000 7fe95759e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9580ab010 con 0x7fe9581409b0 2022-06-29T15:50:39.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.119+0000 7fe95659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe95805fd00 0x7fe9580aca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:39.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.119+0000 7fe95659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe95805fd00 0x7fe9580aca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52794/0 (socket says 172.21.15.47:52794) 2022-06-29T15:50:39.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.119+0000 7fe956d9d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe9581409b0 0x7fe958146d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:39.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.119+0000 7fe95659c700 1 -- 172.21.15.47:0/783515942 learned_addr learned my addr 172.21.15.47:0/783515942 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:39.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.119+0000 7fe94dd9b700 1 --2- 172.21.15.47:0/783515942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe958060a40 0x7fe958146870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:39.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.123+0000 7fe94dd9b700 1 -- 172.21.15.47:0/783515942 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe9581409b0 msgr2=0x7fe958146d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:39.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.123+0000 7fe94dd9b700 1 --2- 172.21.15.47:0/783515942 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe9581409b0 0x7fe958146d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:39.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.123+0000 7fe94dd9b700 1 -- 172.21.15.47:0/783515942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe95805fd00 msgr2=0x7fe9580aca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:39.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.123+0000 7fe94dd9b700 1 --2- 172.21.15.47:0/783515942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe95805fd00 0x7fe9580aca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:39.138 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.123+0000 7fe94dd9b700 1 -- 172.21.15.47:0/783515942 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe950009530 con 0x7fe958060a40 2022-06-29T15:50:39.138 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.123+0000 7fe95659c700 1 --2- 172.21.15.47:0/783515942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe95805fd00 0x7fe9580aca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:50:39.138 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.123+0000 7fe94dd9b700 1 --2- 172.21.15.47:0/783515942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe958060a40 0x7fe958146870 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7fe9580b26f0 tx=0x7fe940007b90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:39.138 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.123+0000 7fe94ffff700 1 -- 172.21.15.47:0/783515942 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe94000ccd0 con 0x7fe958060a40 2022-06-29T15:50:39.139 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.123+0000 7fe95759e700 1 -- 172.21.15.47:0/783515942 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe958147250 con 0x7fe958060a40 2022-06-29T15:50:39.139 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.123+0000 7fe94ffff700 1 -- 172.21.15.47:0/783515942 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe94000ce30 con 0x7fe958060a40 2022-06-29T15:50:39.139 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.123+0000 7fe95759e700 1 -- 172.21.15.47:0/783515942 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe958146580 con 0x7fe958060a40 2022-06-29T15:50:39.139 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.123+0000 7fe94ffff700 1 -- 172.21.15.47:0/783515942 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe94001c630 con 0x7fe958060a40 2022-06-29T15:50:39.140 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.123+0000 7fe95759e700 1 -- 172.21.15.47:0/783515942 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe958144570 con 0x7fe958060a40 2022-06-29T15:50:39.140 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.123+0000 7fe94ffff700 1 -- 172.21.15.47:0/783515942 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe940019030 con 0x7fe958060a40 2022-06-29T15:50:39.140 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.123+0000 7fe94ffff700 1 --2- 172.21.15.47:0/783515942 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe930053a70 0x7fe930055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:39.140 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.123+0000 7fe94ffff700 1 -- 172.21.15.47:0/783515942 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe94006b2e0 con 0x7fe958060a40 2022-06-29T15:50:39.141 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.127+0000 7fe95659c700 1 --2- 172.21.15.47:0/783515942 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe930053a70 0x7fe930055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:39.141 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.127+0000 7fe95659c700 1 --2- 172.21.15.47:0/783515942 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe930053a70 0x7fe930055f30 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7fe94800b000 tx=0x7fe948006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:39.142 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.135+0000 7fe94ffff700 1 -- 172.21.15.47:0/783515942 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe940041f60 con 0x7fe958060a40 2022-06-29T15:50:39.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:39.299+0000 7fe95759e700 1 -- 172.21.15.47:0/783515942 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe9580573d0 con 0x7fe958060a40 2022-06-29T15:50:40.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.283+0000 7fe94ffff700 1 -- 172.21.15.47:0/783515942 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v57) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe940045580 con 0x7fe958060a40 2022-06-29T15:50:40.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.287+0000 7fe95759e700 1 -- 172.21.15.47:0/783515942 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe930053a70 msgr2=0x7fe930055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:40.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.287+0000 7fe95759e700 1 --2- 172.21.15.47:0/783515942 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe930053a70 0x7fe930055f30 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7fe94800b000 tx=0x7fe948006df0).stop 2022-06-29T15:50:40.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.287+0000 7fe95759e700 1 -- 172.21.15.47:0/783515942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe958060a40 msgr2=0x7fe958146870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:40.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.287+0000 7fe95759e700 1 --2- 172.21.15.47:0/783515942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe958060a40 0x7fe958146870 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7fe9580b26f0 tx=0x7fe940007b90).stop 2022-06-29T15:50:40.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.287+0000 7fe95759e700 1 -- 172.21.15.47:0/783515942 shutdown_connections 2022-06-29T15:50:40.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.287+0000 7fe95759e700 1 --2- 172.21.15.47:0/783515942 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe9581409b0 0x7fe958146d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:40.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.287+0000 7fe95759e700 1 --2- 172.21.15.47:0/783515942 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe930053a70 0x7fe930055f30 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:40.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.287+0000 7fe95759e700 1 --2- 172.21.15.47:0/783515942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe958060a40 0x7fe958146870 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:40.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.287+0000 7fe95759e700 1 --2- 172.21.15.47:0/783515942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe95805fd00 0x7fe9580aca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:40.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.287+0000 7fe95759e700 1 -- 172.21.15.47:0/783515942 >> 172.21.15.47:0/783515942 conn(0x7fe9580b6430 msgr2=0x7fe9580b01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:40.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.287+0000 7fe95759e700 1 -- 172.21.15.47:0/783515942 shutdown_connections 2022-06-29T15:50:40.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.287+0000 7fe95759e700 1 -- 172.21.15.47:0/783515942 wait complete. 2022-06-29T15:50:40.309 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:50:40.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.415+0000 7f827a782700 1 Processor -- start 2022-06-29T15:50:40.424 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.415+0000 7f827a782700 1 -- start start 2022-06-29T15:50:40.424 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.415+0000 7f827a782700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f82740b2120 0x7f82740b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:40.424 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.415+0000 7f827a782700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f827405f0b0 0x7f827414b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:40.425 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.415+0000 7f827a782700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f82740b1260 0x7f8274147af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:40.425 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.415+0000 7f827a782700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8274147460 con 0x7f82740b2120 2022-06-29T15:50:40.425 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.415+0000 7f827a782700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82740b0740 con 0x7f82740b1260 2022-06-29T15:50:40.425 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.415+0000 7f827a782700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82740b0970 con 0x7f827405f0b0 2022-06-29T15:50:40.426 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.415+0000 7f8273fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f82740b2120 0x7f82740b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:40.426 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.415+0000 7f8273fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f82740b2120 0x7f82740b2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52806/0 (socket says 172.21.15.47:52806) 2022-06-29T15:50:40.426 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.415+0000 7f8273fff700 1 -- 172.21.15.47:0/1022169789 learned_addr learned my addr 172.21.15.47:0/1022169789 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:40.426 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.415+0000 7f8278d1f700 1 --2- 172.21.15.47:0/1022169789 >> v2:172.21.15.64:3300/0 conn(0x7f82740b1260 0x7f8274147af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:40.427 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.415+0000 7f826bfff700 1 --2- 172.21.15.47:0/1022169789 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f827405f0b0 0x7f827414b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:40.427 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.415+0000 7f8273fff700 1 -- 172.21.15.47:0/1022169789 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f827405f0b0 msgr2=0x7f827414b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:40.427 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.415+0000 7f8273fff700 1 --2- 172.21.15.47:0/1022169789 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f827405f0b0 0x7f827414b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:40.427 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.415+0000 7f8273fff700 1 -- 172.21.15.47:0/1022169789 >> v2:172.21.15.64:3300/0 conn(0x7f82740b1260 msgr2=0x7f8274147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:40.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.415+0000 7f8273fff700 1 --2- 172.21.15.47:0/1022169789 >> v2:172.21.15.64:3300/0 conn(0x7f82740b1260 0x7f8274147af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:40.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.415+0000 7f8273fff700 1 -- 172.21.15.47:0/1022169789 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82740b0c10 con 0x7f82740b2120 2022-06-29T15:50:40.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.415+0000 7f8278d1f700 1 --2- 172.21.15.47:0/1022169789 >> v2:172.21.15.64:3300/0 conn(0x7f82740b1260 0x7f8274147af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:50:40.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.415+0000 7f8273fff700 1 --2- 172.21.15.47:0/1022169789 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82740b2120 0x7f82740b2500 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 rx=0x7f8264002f70 tx=0x7f8264006eb0).ready entity=mon.0 client_cookie=8ce5016069e92376 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:40.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.415+0000 7f82737fe700 1 -- 172.21.15.47:0/1022169789 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f826400d570 con 0x7f82740b2120 2022-06-29T15:50:40.429 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.415+0000 7f82737fe700 1 -- 172.21.15.47:0/1022169789 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8264000e90 con 0x7f82740b2120 2022-06-29T15:50:40.429 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.415+0000 7f82737fe700 1 -- 172.21.15.47:0/1022169789 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f826400d900 con 0x7f82740b2120 2022-06-29T15:50:40.429 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.415+0000 7f827a782700 1 -- 172.21.15.47:0/1022169789 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82740b2120 msgr2=0x7f82740b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:40.429 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.415+0000 7f827a782700 1 --2- 172.21.15.47:0/1022169789 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82740b2120 0x7f82740b2500 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 rx=0x7f8264002f70 tx=0x7f8264006eb0).stop 2022-06-29T15:50:40.430 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.415+0000 7f827a782700 1 -- 172.21.15.47:0/1022169789 shutdown_connections 2022-06-29T15:50:40.430 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.415+0000 7f827a782700 1 --2- 172.21.15.47:0/1022169789 >> v2:172.21.15.64:3300/0 conn(0x7f82740b1260 0x7f8274147af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:40.430 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.415+0000 7f827a782700 1 --2- 172.21.15.47:0/1022169789 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f827405f0b0 0x7f827414b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:40.431 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.415+0000 7f827a782700 1 --2- 172.21.15.47:0/1022169789 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82740b2120 0x7f82740b2500 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:40.431 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.415+0000 7f827a782700 1 -- 172.21.15.47:0/1022169789 >> 172.21.15.47:0/1022169789 conn(0x7f82740626f0 msgr2=0x7f8274062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:40.431 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.415+0000 7f827a782700 1 -- 172.21.15.47:0/1022169789 shutdown_connections 2022-06-29T15:50:40.431 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.415+0000 7f827a782700 1 -- 172.21.15.47:0/1022169789 wait complete. 2022-06-29T15:50:40.435 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.419+0000 7f827a782700 1 Processor -- start 2022-06-29T15:50:40.435 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.419+0000 7f827a782700 1 -- start start 2022-06-29T15:50:40.435 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.419+0000 7f827a782700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f82740b1260 0x7f82741499a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:40.436 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.419+0000 7f827a782700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82740b2120 0x7f82741422c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:40.436 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.419+0000 7f827a782700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8274144560 0x7f82740ae4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:40.436 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.419+0000 7f827a782700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82740ae050 con 0x7f82740b2120 2022-06-29T15:50:40.436 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.419+0000 7f827a782700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8274141bc0 con 0x7f8274144560 2022-06-29T15:50:40.437 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.419+0000 7f827a782700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82740addb0 con 0x7f82740b1260 2022-06-29T15:50:40.437 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.419+0000 7f826bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82740b2120 0x7f82741422c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:40.437 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.419+0000 7f826bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82740b2120 0x7f82741422c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52810/0 (socket says 172.21.15.47:52810) 2022-06-29T15:50:40.437 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.419+0000 7f826bfff700 1 -- 172.21.15.47:0/3987457912 learned_addr learned my addr 172.21.15.47:0/3987457912 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:40.437 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.419+0000 7f8273fff700 1 --2- 172.21.15.47:0/3987457912 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f82740b1260 0x7f82741499a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:40.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.419+0000 7f8278d1f700 1 --2- 172.21.15.47:0/3987457912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8274144560 0x7f82740ae4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:40.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.419+0000 7f826bfff700 1 -- 172.21.15.47:0/3987457912 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f82740b1260 msgr2=0x7f82741499a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:40.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.419+0000 7f826bfff700 1 --2- 172.21.15.47:0/3987457912 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f82740b1260 0x7f82741499a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:40.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.419+0000 7f826bfff700 1 -- 172.21.15.47:0/3987457912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8274144560 msgr2=0x7f82740ae4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:40.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.419+0000 7f826bfff700 1 --2- 172.21.15.47:0/3987457912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8274144560 0x7f82740ae4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:40.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.419+0000 7f826bfff700 1 -- 172.21.15.47:0/3987457912 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f826c009470 con 0x7f82740b2120 2022-06-29T15:50:40.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.419+0000 7f826bfff700 1 --2- 172.21.15.47:0/3987457912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82740b2120 0x7f82741422c0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 rx=0x7f825c00c000 tx=0x7f825c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:40.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.419+0000 7f8271ffb700 1 -- 172.21.15.47:0/3987457912 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f825c010780 con 0x7f82740b2120 2022-06-29T15:50:40.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.419+0000 7f8271ffb700 1 -- 172.21.15.47:0/3987457912 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f825c0108e0 con 0x7f82740b2120 2022-06-29T15:50:40.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.419+0000 7f827a782700 1 -- 172.21.15.47:0/3987457912 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f826400b1e0 con 0x7f82740b2120 2022-06-29T15:50:40.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.419+0000 7f8271ffb700 1 -- 172.21.15.47:0/3987457912 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f825c010c40 con 0x7f82740b2120 2022-06-29T15:50:40.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.419+0000 7f827a782700 1 -- 172.21.15.47:0/3987457912 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8274144bd0 con 0x7f82740b2120 2022-06-29T15:50:40.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.423+0000 7f8271ffb700 1 -- 172.21.15.47:0/3987457912 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f825c01c460 con 0x7f82740b2120 2022-06-29T15:50:40.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.423+0000 7f827a782700 1 -- 172.21.15.47:0/3987457912 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8274144dc0 con 0x7f82740b2120 2022-06-29T15:50:40.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.423+0000 7f8271ffb700 1 --2- 172.21.15.47:0/3987457912 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8258053aa0 0x7f8258055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:40.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.423+0000 7f8271ffb700 1 -- 172.21.15.47:0/3987457912 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f825c06c5e0 con 0x7f82740b2120 2022-06-29T15:50:40.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.423+0000 7f8273fff700 1 --2- 172.21.15.47:0/3987457912 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8258053aa0 0x7f8258055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:40.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.423+0000 7f8273fff700 1 --2- 172.21.15.47:0/3987457912 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8258053aa0 0x7f8258055f60 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7f826401c000 tx=0x7f8264015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:40.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.431+0000 7f8271ffb700 1 -- 172.21.15.47:0/3987457912 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f825c0431b0 con 0x7f82740b2120 2022-06-29T15:50:40.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.595+0000 7f827a782700 1 -- 172.21.15.47:0/3987457912 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f82740573d0 con 0x7f82740b2120 2022-06-29T15:50:40.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.595+0000 7f8271ffb700 1 -- 172.21.15.47:0/3987457912 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+2498 (secure 0 0 0) 0x7f825c0467d0 con 0x7f82740b2120 2022-06-29T15:50:40.603 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:50:40.603 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":57,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7614,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/2948555969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2948555969},{"type":"v1","addr":"172.21.15.47:6835","nonce":2948555969}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":54},{"gid":7617,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.47:6837/1681295902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1681295902},{"type":"v1","addr":"172.21.15.47:6837","nonce":1681295902}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":55}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:50:38.459255+0000","modified":"2022-06-29T15:50:40.277564+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":56606},"failed":[],"damaged":[],"stopped":[],"info":{"gid_56606":{"gid":56606,"name":"b","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6833/2987151459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2987151459},{"type":"v1","addr":"172.21.15.64:6833","nonce":2987151459}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2022-06-29T15:50:40.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.599+0000 7f827a782700 1 -- 172.21.15.47:0/3987457912 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8258053aa0 msgr2=0x7f8258055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:40.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.599+0000 7f827a782700 1 --2- 172.21.15.47:0/3987457912 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8258053aa0 0x7f8258055f60 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7f826401c000 tx=0x7f8264015000).stop 2022-06-29T15:50:40.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.599+0000 7f827a782700 1 -- 172.21.15.47:0/3987457912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82740b2120 msgr2=0x7f82741422c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:40.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.599+0000 7f827a782700 1 --2- 172.21.15.47:0/3987457912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82740b2120 0x7f82741422c0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 rx=0x7f825c00c000 tx=0x7f825c006df0).stop 2022-06-29T15:50:40.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.599+0000 7f827a782700 1 -- 172.21.15.47:0/3987457912 shutdown_connections 2022-06-29T15:50:40.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.599+0000 7f827a782700 1 --2- 172.21.15.47:0/3987457912 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8258053aa0 0x7f8258055f60 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:40.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.599+0000 7f827a782700 1 --2- 172.21.15.47:0/3987457912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8274144560 0x7f82740ae4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:40.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.599+0000 7f827a782700 1 --2- 172.21.15.47:0/3987457912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82740b2120 0x7f82741422c0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:40.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.599+0000 7f827a782700 1 --2- 172.21.15.47:0/3987457912 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f82740b1260 0x7f82741499a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:40.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.599+0000 7f827a782700 1 -- 172.21.15.47:0/3987457912 >> 172.21.15.47:0/3987457912 conn(0x7f82740626f0 msgr2=0x7f82740b5450 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:40.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.599+0000 7f827a782700 1 -- 172.21.15.47:0/3987457912 shutdown_connections 2022-06-29T15:50:40.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.599+0000 7f827a782700 1 -- 172.21.15.47:0/3987457912 wait complete. 2022-06-29T15:50:40.611 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 57 2022-06-29T15:50:40.615 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:50:40.723 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.715+0000 7f5970aa6700 1 Processor -- start 2022-06-29T15:50:40.723 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.715+0000 7f5970aa6700 1 -- start start 2022-06-29T15:50:40.725 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.719+0000 7f5970aa6700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f596c0b2ee0 0x7f596c0b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:40.725 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.719+0000 7f5970aa6700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f596c0b0e50 0x7f596c13d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:40.725 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.719+0000 7f5970aa6700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f596c1400e0 con 0x7f596c0ad970 2022-06-29T15:50:40.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.719+0000 7f5970aa6700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f596c140ef0 con 0x7f596c0b2ee0 2022-06-29T15:50:40.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.719+0000 7f5970aa6700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f596c13fef0 con 0x7f596c0b0e50 2022-06-29T15:50:40.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.719+0000 7f596bfff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f596c0b0e50 0x7f596c13d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:40.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.719+0000 7f596bfff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f596c0b0e50 0x7f596c13d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:49920/0 (socket says 172.21.15.47:49920) 2022-06-29T15:50:40.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.719+0000 7f596bfff700 1 -- 172.21.15.47:0/2130115894 learned_addr learned my addr 172.21.15.47:0/2130115894 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:40.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.719+0000 7f596b7fe700 1 --2- 172.21.15.47:0/2130115894 >> v2:172.21.15.64:3300/0 conn(0x7f596c0b2ee0 0x7f596c0b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:40.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.719+0000 7f596bfff700 1 -- 172.21.15.47:0/2130115894 >> v2:172.21.15.64:3300/0 conn(0x7f596c0b2ee0 msgr2=0x7f596c0b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:40.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.719+0000 7f596bfff700 1 --2- 172.21.15.47:0/2130115894 >> v2:172.21.15.64:3300/0 conn(0x7f596c0b2ee0 0x7f596c0b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:40.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.719+0000 7f596bfff700 1 -- 172.21.15.47:0/2130115894 >> v1:172.21.15.64:6789/0 conn(0x7f596c0ad970 legacy=0x7f596c1407e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:40.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.719+0000 7f596bfff700 1 -- 172.21.15.47:0/2130115894 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f596c143570 con 0x7f596c0b0e50 2022-06-29T15:50:40.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.719+0000 7f596bfff700 1 --2- 172.21.15.47:0/2130115894 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f596c0b0e50 0x7f596c13d290 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f59640032f0 tx=0x7f596400b530).ready entity=mon.2 client_cookie=eb1649635c02e48a server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:40.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.719+0000 7f596affd700 1 -- 172.21.15.47:0/2130115894 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f596400de20 con 0x7f596c0b0e50 2022-06-29T15:50:40.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.719+0000 7f596affd700 1 -- 172.21.15.47:0/2130115894 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5964010470 con 0x7f596c0b0e50 2022-06-29T15:50:40.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.719+0000 7f596affd700 1 -- 172.21.15.47:0/2130115894 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f59640107d0 con 0x7f596c0b0e50 2022-06-29T15:50:40.731 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.719+0000 7f5970aa6700 1 -- 172.21.15.47:0/2130115894 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f596c0b0e50 msgr2=0x7f596c13d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:40.731 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.719+0000 7f5970aa6700 1 --2- 172.21.15.47:0/2130115894 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f596c0b0e50 0x7f596c13d290 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f59640032f0 tx=0x7f596400b530).stop 2022-06-29T15:50:40.731 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.719+0000 7f5970aa6700 1 -- 172.21.15.47:0/2130115894 shutdown_connections 2022-06-29T15:50:40.731 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.719+0000 7f5970aa6700 1 --2- 172.21.15.47:0/2130115894 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f596c0b0e50 0x7f596c13d290 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:40.732 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.719+0000 7f5970aa6700 1 --2- 172.21.15.47:0/2130115894 >> v2:172.21.15.64:3300/0 conn(0x7f596c0b2ee0 0x7f596c0b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:40.732 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.719+0000 7f5970aa6700 1 -- 172.21.15.47:0/2130115894 >> 172.21.15.47:0/2130115894 conn(0x7f596c063670 msgr2=0x7f596c063a80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:40.732 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.719+0000 7f5970aa6700 1 -- 172.21.15.47:0/2130115894 shutdown_connections 2022-06-29T15:50:40.732 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.719+0000 7f5970aa6700 1 -- 172.21.15.47:0/2130115894 wait complete. 2022-06-29T15:50:40.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.723+0000 7f5970aa6700 1 Processor -- start 2022-06-29T15:50:40.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.723+0000 7f5970aa6700 1 -- start start 2022-06-29T15:50:40.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.723+0000 7f5970aa6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f596c0ad970 0x7f596c142ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:40.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.723+0000 7f5970aa6700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f596c0b0e50 0x7f596c152170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:40.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.723+0000 7f5970aa6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f596c0b2ee0 0x7f596c152660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:40.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.723+0000 7f5970aa6700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596c0ac140 con 0x7f596c0b2ee0 2022-06-29T15:50:40.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.723+0000 7f5970aa6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596c155740 con 0x7f596c0ad970 2022-06-29T15:50:40.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.723+0000 7f5970aa6700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f596c1559e0 con 0x7f596c0b0e50 2022-06-29T15:50:40.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.727+0000 7f596bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f596c0b2ee0 0x7f596c152660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:40.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.727+0000 7f596bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f596c0b2ee0 0x7f596c152660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52824/0 (socket says 172.21.15.47:52824) 2022-06-29T15:50:40.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.727+0000 7f596bfff700 1 -- 172.21.15.47:0/234921818 learned_addr learned my addr 172.21.15.47:0/234921818 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:40.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.727+0000 7f596b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f596c0ad970 0x7f596c142ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:40.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.727+0000 7f5963fff700 1 --2- 172.21.15.47:0/234921818 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f596c0b0e50 0x7f596c152170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:40.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.727+0000 7f596bfff700 1 -- 172.21.15.47:0/234921818 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f596c0b0e50 msgr2=0x7f596c152170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:40.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.727+0000 7f596bfff700 1 --2- 172.21.15.47:0/234921818 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f596c0b0e50 0x7f596c152170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:40.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.727+0000 7f596bfff700 1 -- 172.21.15.47:0/234921818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f596c0ad970 msgr2=0x7f596c142ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:40.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.727+0000 7f596bfff700 1 --2- 172.21.15.47:0/234921818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f596c0ad970 0x7f596c142ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:40.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.727+0000 7f596bfff700 1 -- 172.21.15.47:0/234921818 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f596400b7d0 con 0x7f596c0b2ee0 2022-06-29T15:50:40.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.727+0000 7f596bfff700 1 --2- 172.21.15.47:0/234921818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f596c0b2ee0 0x7f596c152660 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 rx=0x7f59640032f0 tx=0x7f596c0ae510).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:40.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.727+0000 7f59697fa700 1 -- 172.21.15.47:0/234921818 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f596401d070 con 0x7f596c0b2ee0 2022-06-29T15:50:40.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.727+0000 7f59697fa700 1 -- 172.21.15.47:0/234921818 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5964018800 con 0x7f596c0b2ee0 2022-06-29T15:50:40.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.727+0000 7f59697fa700 1 -- 172.21.15.47:0/234921818 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5964018b60 con 0x7f596c0b2ee0 2022-06-29T15:50:40.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.727+0000 7f5970aa6700 1 -- 172.21.15.47:0/234921818 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f596c155c80 con 0x7f596c0b2ee0 2022-06-29T15:50:40.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.727+0000 7f5970aa6700 1 -- 172.21.15.47:0/234921818 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f596c13cd10 con 0x7f596c0b2ee0 2022-06-29T15:50:40.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.727+0000 7f5970aa6700 1 -- 172.21.15.47:0/234921818 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f596c0573d0 con 0x7f596c0b2ee0 2022-06-29T15:50:40.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.731+0000 7f59697fa700 1 -- 172.21.15.47:0/234921818 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5964018cc0 con 0x7f596c0b2ee0 2022-06-29T15:50:40.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.731+0000 7f59697fa700 1 --2- 172.21.15.47:0/234921818 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f59500539e0 0x7f5950055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:40.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.731+0000 7f59697fa700 1 -- 172.21.15.47:0/234921818 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f596406dd70 con 0x7f596c0b2ee0 2022-06-29T15:50:40.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.731+0000 7f596b7fe700 1 --2- 172.21.15.47:0/234921818 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f59500539e0 0x7f5950055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:40.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.731+0000 7f596b7fe700 1 --2- 172.21.15.47:0/234921818 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f59500539e0 0x7f5950055ea0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7f595c00b000 tx=0x7f595c006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:40.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.739+0000 7f59697fa700 1 -- 172.21.15.47:0/234921818 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5964044b30 con 0x7f596c0b2ee0 2022-06-29T15:50:40.874 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.867+0000 7f5970aa6700 1 -- 172.21.15.47:0/234921818 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f596c069e20 con 0x7f596c0b2ee0 2022-06-29T15:50:40.875 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.871+0000 7f59697fa700 1 -- 172.21.15.47:0/234921818 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f5964048150 con 0x7f596c0b2ee0 2022-06-29T15:50:40.876 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:50:40.876 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":121,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:50:39.462097+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:50:37.087143+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2022-06-29T15:50:38.125088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:50:40.877 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.871+0000 7f5970aa6700 1 -- 172.21.15.47:0/234921818 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f59500539e0 msgr2=0x7f5950055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:40.877 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.871+0000 7f5970aa6700 1 --2- 172.21.15.47:0/234921818 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f59500539e0 0x7f5950055ea0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7f595c00b000 tx=0x7f595c006df0).stop 2022-06-29T15:50:40.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.871+0000 7f5970aa6700 1 -- 172.21.15.47:0/234921818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f596c0b2ee0 msgr2=0x7f596c152660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:40.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.871+0000 7f5970aa6700 1 --2- 172.21.15.47:0/234921818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f596c0b2ee0 0x7f596c152660 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 rx=0x7f59640032f0 tx=0x7f596c0ae510).stop 2022-06-29T15:50:40.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.871+0000 7f5970aa6700 1 -- 172.21.15.47:0/234921818 shutdown_connections 2022-06-29T15:50:40.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.871+0000 7f5970aa6700 1 --2- 172.21.15.47:0/234921818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f596c0b2ee0 0x7f596c152660 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:40.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.871+0000 7f5970aa6700 1 --2- 172.21.15.47:0/234921818 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f596c0b0e50 0x7f596c152170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:40.880 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.871+0000 7f5970aa6700 1 --2- 172.21.15.47:0/234921818 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f59500539e0 0x7f5950055ea0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:40.880 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.871+0000 7f5970aa6700 1 --2- 172.21.15.47:0/234921818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f596c0ad970 0x7f596c142ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:40.880 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.871+0000 7f5970aa6700 1 -- 172.21.15.47:0/234921818 >> 172.21.15.47:0/234921818 conn(0x7f596c063670 msgr2=0x7f596c0b02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:40.880 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.871+0000 7f5970aa6700 1 -- 172.21.15.47:0/234921818 shutdown_connections 2022-06-29T15:50:40.880 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.871+0000 7f5970aa6700 1 -- 172.21.15.47:0/234921818 wait complete. 2022-06-29T15:50:40.890 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-29T15:50:40.914 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2022-06-29T15:50:40.914 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2022-06-29T15:50:41.003 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.995+0000 7f03c4e6d700 1 Processor -- start 2022-06-29T15:50:41.004 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.995+0000 7f03c4e6d700 1 -- start start 2022-06-29T15:50:41.012 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.999+0000 7f03c4e6d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f03c005f490 0x7f03c0144670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:41.013 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.999+0000 7f03c4e6d700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03c0061e50 con 0x7f03c0061740 2022-06-29T15:50:41.013 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.999+0000 7f03c4e6d700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03c00620f0 con 0x7f03c005f0b0 2022-06-29T15:50:41.013 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.999+0000 7f03c4e6d700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03c01434b0 con 0x7f03c005f490 2022-06-29T15:50:41.013 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.999+0000 7f03bffff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f03c0061740 0x7f03c0140410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38386/0 (socket says 172.21.15.47:38386) 2022-06-29T15:50:41.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.999+0000 7f03bffff700 1 -- 172.21.15.47:0/2485853158 learned_addr learned my addr 172.21.15.47:0/2485853158 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:41.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.999+0000 7f03beffd700 1 --2- 172.21.15.47:0/2485853158 >> v2:172.21.15.64:3300/0 conn(0x7f03c005f490 0x7f03c0144670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:41.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.999+0000 7f03be7fc700 1 -- 172.21.15.47:0/2485853158 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 296450659 0 0) 0x7f03c0061e50 con 0x7f03c0061740 2022-06-29T15:50:41.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.999+0000 7f03be7fc700 1 -- 172.21.15.47:0/2485853158 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03a0003260 con 0x7f03c0061740 2022-06-29T15:50:41.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.999+0000 7f03be7fc700 1 -- 172.21.15.47:0/2485853158 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2856438139 0 0) 0x7f03a0003260 con 0x7f03c0061740 2022-06-29T15:50:41.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.999+0000 7f03be7fc700 1 -- 172.21.15.47:0/2485853158 >> v2:172.21.15.64:3300/0 conn(0x7f03c005f490 msgr2=0x7f03c0144670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:41.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.999+0000 7f03be7fc700 1 --2- 172.21.15.47:0/2485853158 >> v2:172.21.15.64:3300/0 conn(0x7f03c005f490 0x7f03c0144670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:41.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.999+0000 7f03be7fc700 1 -- 172.21.15.47:0/2485853158 >> v1:172.21.15.64:6789/0 conn(0x7f03c005f0b0 legacy=0x7f03c0143950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:41.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.999+0000 7f03be7fc700 1 -- 172.21.15.47:0/2485853158 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c0144060 con 0x7f03c0061740 2022-06-29T15:50:41.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.999+0000 7f03be7fc700 1 -- 172.21.15.47:0/2485853158 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7f03b0001660 con 0x7f03c0061740 2022-06-29T15:50:41.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.999+0000 7f03be7fc700 1 -- 172.21.15.47:0/2485853158 >> v1:172.21.15.47:6789/0 conn(0x7f03c0061740 legacy=0x7f03c0140410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:41.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.999+0000 7f03be7fc700 1 --2- 172.21.15.47:0/2485853158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03a0003750 0x7f03a0003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:41.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.999+0000 7f03be7fc700 1 --2- 172.21.15.47:0/2485853158 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03a0004500 0x7f03a0004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:41.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.999+0000 7f03be7fc700 1 --2- 172.21.15.47:0/2485853158 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f03a0004e70 0x7f03a000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:41.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.999+0000 7f03be7fc700 1 -- 172.21.15.47:0/2485853158 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c01434b0 con 0x7f03a0004500 2022-06-29T15:50:41.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.999+0000 7f03be7fc700 1 -- 172.21.15.47:0/2485853158 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a000bc70 con 0x7f03a0003750 2022-06-29T15:50:41.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.999+0000 7f03be7fc700 1 -- 172.21.15.47:0/2485853158 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03a000be30 con 0x7f03a0004e70 2022-06-29T15:50:41.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.999+0000 7f03beffd700 1 --2- 172.21.15.47:0/2485853158 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03a0004500 0x7f03a0004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:41.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.999+0000 7f03bffff700 1 --2- 172.21.15.47:0/2485853158 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f03a0004e70 0x7f03a000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:41.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.999+0000 7f03bf7fe700 1 --2- 172.21.15.47:0/2485853158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03a0003750 0x7f03a0003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:41.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.999+0000 7f03beffd700 1 -- 172.21.15.47:0/2485853158 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f03a0004e70 msgr2=0x7f03a000b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:41.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.999+0000 7f03beffd700 1 --2- 172.21.15.47:0/2485853158 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f03a0004e70 0x7f03a000b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:41.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.999+0000 7f03beffd700 1 -- 172.21.15.47:0/2485853158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03a0003750 msgr2=0x7f03a0003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:41.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.999+0000 7f03beffd700 1 --2- 172.21.15.47:0/2485853158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03a0003750 0x7f03a0003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:41.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.999+0000 7f03bffff700 1 -- 172.21.15.47:0/2485853158 reap_dead start 2022-06-29T15:50:41.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.999+0000 7f03beffd700 1 -- 172.21.15.47:0/2485853158 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a0003260 con 0x7f03a0004500 2022-06-29T15:50:41.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.999+0000 7f03beffd700 1 --2- 172.21.15.47:0/2485853158 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03a0004500 0x7f03a0004980 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 rx=0x7f03a800c000 tx=0x7f03a8006df0).ready entity=mon.0 client_cookie=11793f5998776ff2 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:41.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.999+0000 7f03be7fc700 1 -- 172.21.15.47:0/2485853158 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f03a800e830 con 0x7f03a0004500 2022-06-29T15:50:41.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.999+0000 7f03be7fc700 1 -- 172.21.15.47:0/2485853158 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03a800e990 con 0x7f03a0004500 2022-06-29T15:50:41.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.999+0000 7f03be7fc700 1 -- 172.21.15.47:0/2485853158 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f03a800ecf0 con 0x7f03a0004500 2022-06-29T15:50:41.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.999+0000 7f03c4e6d700 1 -- 172.21.15.47:0/2485853158 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03a0004500 msgr2=0x7f03a0004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:41.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:40.999+0000 7f03c4e6d700 1 --2- 172.21.15.47:0/2485853158 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03a0004500 0x7f03a0004980 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 rx=0x7f03a800c000 tx=0x7f03a8006df0).stop 2022-06-29T15:50:41.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.003+0000 7f03c4e6d700 1 -- 172.21.15.47:0/2485853158 shutdown_connections 2022-06-29T15:50:41.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.003+0000 7f03c4e6d700 1 --2- 172.21.15.47:0/2485853158 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03a0004500 0x7f03a0004980 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:41.022 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.003+0000 7f03c4e6d700 1 -- 172.21.15.47:0/2485853158 >> 172.21.15.47:0/2485853158 conn(0x7f03c0062570 msgr2=0x7f03c00647d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:41.022 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.003+0000 7f03c4e6d700 1 -- 172.21.15.47:0/2485853158 shutdown_connections 2022-06-29T15:50:41.022 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.003+0000 7f03c4e6d700 1 -- 172.21.15.47:0/2485853158 wait complete. 2022-06-29T15:50:41.022 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.003+0000 7f03c4e6d700 1 Processor -- start 2022-06-29T15:50:41.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.003+0000 7f03c4e6d700 1 -- start start 2022-06-29T15:50:41.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.003+0000 7f03c4e6d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03a0004500 0x7f03c00b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:41.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.003+0000 7f03c4e6d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f03c00b6760 0x7f03c0152720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:41.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.003+0000 7f03c4e6d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03c0152c10 0x7f03c01530a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:41.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.003+0000 7f03c4e6d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c00b78c0 con 0x7f03a0004500 2022-06-29T15:50:41.024 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.003+0000 7f03c4e6d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c00b7b60 con 0x7f03c0152c10 2022-06-29T15:50:41.024 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.003+0000 7f03c4e6d700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03c00b7e00 con 0x7f03c00b6760 2022-06-29T15:50:41.024 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.003+0000 7f03bf7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03a0004500 0x7f03c00b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:41.024 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.003+0000 7f03bf7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03a0004500 0x7f03c00b2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52842/0 (socket says 172.21.15.47:52842) 2022-06-29T15:50:41.025 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.003+0000 7f03bf7fe700 1 -- 172.21.15.47:0/303300458 learned_addr learned my addr 172.21.15.47:0/303300458 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:41.025 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.003+0000 7f03beffd700 1 --2- 172.21.15.47:0/303300458 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f03c00b6760 0x7f03c0152720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:41.025 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.003+0000 7f03bffff700 1 --2- 172.21.15.47:0/303300458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03c0152c10 0x7f03c01530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:41.025 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.003+0000 7f03bf7fe700 1 -- 172.21.15.47:0/303300458 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f03c00b6760 msgr2=0x7f03c0152720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:41.026 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.003+0000 7f03bf7fe700 1 --2- 172.21.15.47:0/303300458 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f03c00b6760 0x7f03c0152720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:41.026 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.003+0000 7f03bf7fe700 1 -- 172.21.15.47:0/303300458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03c0152c10 msgr2=0x7f03c01530a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:41.026 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.003+0000 7f03bf7fe700 1 --2- 172.21.15.47:0/303300458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03c0152c10 0x7f03c01530a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:41.026 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.003+0000 7f03bf7fe700 1 -- 172.21.15.47:0/303300458 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c00abf40 con 0x7f03a0004500 2022-06-29T15:50:41.027 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.003+0000 7f03beffd700 1 --2- 172.21.15.47:0/303300458 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f03c00b6760 0x7f03c0152720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:50:41.027 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.003+0000 7f03bffff700 1 --2- 172.21.15.47:0/303300458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03c0152c10 0x7f03c01530a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:50:41.027 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.003+0000 7f03bf7fe700 1 --2- 172.21.15.47:0/303300458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03a0004500 0x7f03c00b2f20 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 rx=0x7f03b800d000 tx=0x7f03b8007d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:41.027 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.003+0000 7f03bcff9700 1 -- 172.21.15.47:0/303300458 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f03b8006d60 con 0x7f03a0004500 2022-06-29T15:50:41.028 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.007+0000 7f03bcff9700 1 -- 172.21.15.47:0/303300458 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03b8007030 con 0x7f03a0004500 2022-06-29T15:50:41.028 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.007+0000 7f03bcff9700 1 -- 172.21.15.47:0/303300458 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f03b8007390 con 0x7f03a0004500 2022-06-29T15:50:41.028 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.007+0000 7f03c4e6d700 1 -- 172.21.15.47:0/303300458 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03c00b6bc0 con 0x7f03a0004500 2022-06-29T15:50:41.028 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.007+0000 7f03c4e6d700 1 -- 172.21.15.47:0/303300458 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03c0155a70 con 0x7f03a0004500 2022-06-29T15:50:41.030 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.007+0000 7f03c4e6d700 1 -- 172.21.15.47:0/303300458 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03c00573d0 con 0x7f03a0004500 2022-06-29T15:50:41.030 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.007+0000 7f03bcff9700 1 -- 172.21.15.47:0/303300458 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f03b801b030 con 0x7f03a0004500 2022-06-29T15:50:41.030 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.007+0000 7f03bcff9700 1 --2- 172.21.15.47:0/303300458 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f03a4053a70 0x7f03a4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:41.031 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.007+0000 7f03bcff9700 1 -- 172.21.15.47:0/303300458 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f03b806bf60 con 0x7f03a0004500 2022-06-29T15:50:41.031 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.007+0000 7f03beffd700 1 --2- 172.21.15.47:0/303300458 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f03a4053a70 0x7f03a4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:41.031 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.007+0000 7f03beffd700 1 --2- 172.21.15.47:0/303300458 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f03a4053a70 0x7f03a4055f30 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7f03a801b000 tx=0x7f03a8019000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:41.031 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.019+0000 7f03bcff9700 1 -- 172.21.15.47:0/303300458 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f03b8042d20 con 0x7f03a0004500 2022-06-29T15:50:41.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.187+0000 7f03c4e6d700 1 -- 172.21.15.47:0/303300458 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f03c00b1a30 con 0x7f03a0004500 2022-06-29T15:50:41.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.195+0000 7f03bcff9700 1 -- 172.21.15.47:0/303300458 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v31) v1 ==== 184+0+0 (secure 0 0 0) 0x7f03b8046340 con 0x7f03a0004500 2022-06-29T15:50:41.204 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.195+0000 7f03c4e6d700 1 -- 172.21.15.47:0/303300458 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f03a4053a70 msgr2=0x7f03a4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:41.204 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.195+0000 7f03c4e6d700 1 --2- 172.21.15.47:0/303300458 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f03a4053a70 0x7f03a4055f30 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7f03a801b000 tx=0x7f03a8019000).stop 2022-06-29T15:50:41.205 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.195+0000 7f03c4e6d700 1 -- 172.21.15.47:0/303300458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03a0004500 msgr2=0x7f03c00b2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:41.205 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.195+0000 7f03c4e6d700 1 --2- 172.21.15.47:0/303300458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03a0004500 0x7f03c00b2f20 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 rx=0x7f03b800d000 tx=0x7f03b8007d80).stop 2022-06-29T15:50:41.206 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.195+0000 7f03c4e6d700 1 -- 172.21.15.47:0/303300458 shutdown_connections 2022-06-29T15:50:41.206 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.195+0000 7f03c4e6d700 1 --2- 172.21.15.47:0/303300458 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f03a4053a70 0x7f03a4055f30 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:41.206 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.195+0000 7f03c4e6d700 1 --2- 172.21.15.47:0/303300458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03c0152c10 0x7f03c01530a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:41.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.199+0000 7f03c4e6d700 1 --2- 172.21.15.47:0/303300458 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f03c00b6760 0x7f03c0152720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:41.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.199+0000 7f03c4e6d700 1 --2- 172.21.15.47:0/303300458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03a0004500 0x7f03c00b2f20 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:41.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.199+0000 7f03c4e6d700 1 -- 172.21.15.47:0/303300458 >> 172.21.15.47:0/303300458 conn(0x7f03c0062570 msgr2=0x7f03c0145960 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:41.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.199+0000 7f03c4e6d700 1 -- 172.21.15.47:0/303300458 shutdown_connections 2022-06-29T15:50:41.209 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.199+0000 7f03c4e6d700 1 -- 172.21.15.47:0/303300458 wait complete. 2022-06-29T15:50:41.209 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.0 2022-06-29T15:50:41.217 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-29T15:50:41.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.323+0000 7f9d6872e700 1 Processor -- start 2022-06-29T15:50:41.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.323+0000 7f9d6872e700 1 -- start start 2022-06-29T15:50:41.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.323+0000 7f9d6872e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9d60140940 0x7f9d6005f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:41.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.323+0000 7f9d6872e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9d6005f5a0 0x7f9d600b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:41.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.323+0000 7f9d6872e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d600b4e50 con 0x7f9d600b35d0 2022-06-29T15:50:41.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.323+0000 7f9d6872e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d600621e0 con 0x7f9d60140940 2022-06-29T15:50:41.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.323+0000 7f9d6872e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d60143170 con 0x7f9d6005f5a0 2022-06-29T15:50:41.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.323+0000 7f9d664ca700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9d60140940 0x7f9d6005f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:41.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.323+0000 7f9d66ccb700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9d600b35d0 0x7f9d601466e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38408/0 (socket says 172.21.15.47:38408) 2022-06-29T15:50:41.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.323+0000 7f9d66ccb700 1 -- 172.21.15.47:0/4102356608 learned_addr learned my addr 172.21.15.47:0/4102356608 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:41.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.323+0000 7f9d65cc9700 1 --2- 172.21.15.47:0/4102356608 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9d6005f5a0 0x7f9d600b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:41.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.323+0000 7f9d664ca700 1 -- 172.21.15.47:0/4102356608 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9d6005f5a0 msgr2=0x7f9d600b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:41.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.323+0000 7f9d664ca700 1 --2- 172.21.15.47:0/4102356608 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9d6005f5a0 0x7f9d600b30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:41.335 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.323+0000 7f9d664ca700 1 -- 172.21.15.47:0/4102356608 >> v1:172.21.15.47:6789/0 conn(0x7f9d600b35d0 legacy=0x7f9d601466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:41.335 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.323+0000 7f9d664ca700 1 -- 172.21.15.47:0/4102356608 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d600b4bb0 con 0x7f9d60140940 2022-06-29T15:50:41.335 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.323+0000 7f9d664ca700 1 --2- 172.21.15.47:0/4102356608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d60140940 0x7f9d6005f0b0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 rx=0x7f9d5c002f70 tx=0x7f9d5c006df0).ready entity=mon.0 client_cookie=ee921378b634112e server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:41.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.323+0000 7f9d654c8700 1 -- 172.21.15.47:0/4102356608 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d5c00d570 con 0x7f9d60140940 2022-06-29T15:50:41.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.323+0000 7f9d654c8700 1 -- 172.21.15.47:0/4102356608 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d5c000e90 con 0x7f9d60140940 2022-06-29T15:50:41.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.323+0000 7f9d654c8700 1 -- 172.21.15.47:0/4102356608 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d5c00d8c0 con 0x7f9d60140940 2022-06-29T15:50:41.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.323+0000 7f9d6872e700 1 -- 172.21.15.47:0/4102356608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d60140940 msgr2=0x7f9d6005f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:41.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.323+0000 7f9d6872e700 1 --2- 172.21.15.47:0/4102356608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d60140940 0x7f9d6005f0b0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 rx=0x7f9d5c002f70 tx=0x7f9d5c006df0).stop 2022-06-29T15:50:41.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.323+0000 7f9d6872e700 1 -- 172.21.15.47:0/4102356608 shutdown_connections 2022-06-29T15:50:41.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.323+0000 7f9d6872e700 1 --2- 172.21.15.47:0/4102356608 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9d6005f5a0 0x7f9d600b30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:41.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.323+0000 7f9d6872e700 1 --2- 172.21.15.47:0/4102356608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d60140940 0x7f9d6005f0b0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:41.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.323+0000 7f9d6872e700 1 -- 172.21.15.47:0/4102356608 >> 172.21.15.47:0/4102356608 conn(0x7f9d600627d0 msgr2=0x7f9d60062bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:41.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.323+0000 7f9d6872e700 1 -- 172.21.15.47:0/4102356608 shutdown_connections 2022-06-29T15:50:41.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.323+0000 7f9d6872e700 1 -- 172.21.15.47:0/4102356608 wait complete. 2022-06-29T15:50:41.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.327+0000 7f9d6872e700 1 Processor -- start 2022-06-29T15:50:41.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.327+0000 7f9d6872e700 1 -- start start 2022-06-29T15:50:41.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.327+0000 7f9d6872e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d600b35d0 0x7f9d60141350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:41.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.327+0000 7f9d6872e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9d60140940 0x7f9d60148330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:41.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.327+0000 7f9d6872e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d60148820 0x7f9d6013d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:41.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.327+0000 7f9d6872e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d600aeec0 con 0x7f9d60148820 2022-06-29T15:50:41.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.327+0000 7f9d6872e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d60148cd0 con 0x7f9d600b35d0 2022-06-29T15:50:41.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.327+0000 7f9d6872e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6013e780 con 0x7f9d60140940 2022-06-29T15:50:41.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.327+0000 7f9d66ccb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d60148820 0x7f9d6013d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:41.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.327+0000 7f9d66ccb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d60148820 0x7f9d6013d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52860/0 (socket says 172.21.15.47:52860) 2022-06-29T15:50:41.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.327+0000 7f9d66ccb700 1 -- 172.21.15.47:0/2652367964 learned_addr learned my addr 172.21.15.47:0/2652367964 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:41.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.327+0000 7f9d65cc9700 1 --2- 172.21.15.47:0/2652367964 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9d60140940 0x7f9d60148330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:41.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.327+0000 7f9d664ca700 1 --2- 172.21.15.47:0/2652367964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d600b35d0 0x7f9d60141350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:41.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.327+0000 7f9d66ccb700 1 -- 172.21.15.47:0/2652367964 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9d60140940 msgr2=0x7f9d60148330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:41.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.327+0000 7f9d66ccb700 1 --2- 172.21.15.47:0/2652367964 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9d60140940 0x7f9d60148330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:41.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.327+0000 7f9d66ccb700 1 -- 172.21.15.47:0/2652367964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d600b35d0 msgr2=0x7f9d60141350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:41.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.327+0000 7f9d66ccb700 1 --2- 172.21.15.47:0/2652367964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d600b35d0 0x7f9d60141350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:41.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.327+0000 7f9d66ccb700 1 -- 172.21.15.47:0/2652367964 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d5c00b180 con 0x7f9d60148820 2022-06-29T15:50:41.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.327+0000 7f9d66ccb700 1 --2- 172.21.15.47:0/2652367964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d60148820 0x7f9d6013d4c0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 rx=0x7f9d60148e10 tx=0x7f9d58007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:41.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.327+0000 7f9d4f7fe700 1 -- 172.21.15.47:0/2652367964 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d58005ba0 con 0x7f9d60148820 2022-06-29T15:50:41.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.327+0000 7f9d4f7fe700 1 -- 172.21.15.47:0/2652367964 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d58005d00 con 0x7f9d60148820 2022-06-29T15:50:41.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.327+0000 7f9d6872e700 1 -- 172.21.15.47:0/2652367964 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d60139b60 con 0x7f9d60148820 2022-06-29T15:50:41.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.327+0000 7f9d4f7fe700 1 -- 172.21.15.47:0/2652367964 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d580072c0 con 0x7f9d60148820 2022-06-29T15:50:41.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.331+0000 7f9d6872e700 1 -- 172.21.15.47:0/2652367964 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d60147920 con 0x7f9d60148820 2022-06-29T15:50:41.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.331+0000 7f9d6872e700 1 -- 172.21.15.47:0/2652367964 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d600573d0 con 0x7f9d60148820 2022-06-29T15:50:41.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.331+0000 7f9d4f7fe700 1 -- 172.21.15.47:0/2652367964 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9d58010070 con 0x7f9d60148820 2022-06-29T15:50:41.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.331+0000 7f9d4f7fe700 1 --2- 172.21.15.47:0/2652367964 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9d40053980 0x7f9d40055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:41.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.331+0000 7f9d4f7fe700 1 -- 172.21.15.47:0/2652367964 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9d5806b910 con 0x7f9d60148820 2022-06-29T15:50:41.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.331+0000 7f9d664ca700 1 --2- 172.21.15.47:0/2652367964 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9d40053980 0x7f9d40055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:41.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.331+0000 7f9d664ca700 1 --2- 172.21.15.47:0/2652367964 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9d40053980 0x7f9d40055e40 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f9d5c017000 tx=0x7f9d5c015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:41.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.339+0000 7f9d4f7fe700 1 -- 172.21.15.47:0/2652367964 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9d580426d0 con 0x7f9d60148820 2022-06-29T15:50:41.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.499+0000 7f9d6872e700 1 -- 172.21.15.47:0/2652367964 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f9d60141ec0 con 0x7f9d60148820 2022-06-29T15:50:41.513 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.507+0000 7f9d4f7fe700 1 -- 172.21.15.47:0/2652367964 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v32) v1 ==== 184+0+0 (secure 0 0 0) 0x7f9d58045cf0 con 0x7f9d60148820 2022-06-29T15:50:41.516 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.511+0000 7f9d6872e700 1 -- 172.21.15.47:0/2652367964 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9d40053980 msgr2=0x7f9d40055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:41.516 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.511+0000 7f9d6872e700 1 --2- 172.21.15.47:0/2652367964 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9d40053980 0x7f9d40055e40 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f9d5c017000 tx=0x7f9d5c015000).stop 2022-06-29T15:50:41.517 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.511+0000 7f9d6872e700 1 -- 172.21.15.47:0/2652367964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d60148820 msgr2=0x7f9d6013d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:41.517 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.511+0000 7f9d6872e700 1 --2- 172.21.15.47:0/2652367964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d60148820 0x7f9d6013d4c0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 rx=0x7f9d60148e10 tx=0x7f9d58007f40).stop 2022-06-29T15:50:41.517 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.511+0000 7f9d6872e700 1 -- 172.21.15.47:0/2652367964 shutdown_connections 2022-06-29T15:50:41.517 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.511+0000 7f9d6872e700 1 --2- 172.21.15.47:0/2652367964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d60148820 0x7f9d6013d4c0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:41.518 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.511+0000 7f9d6872e700 1 --2- 172.21.15.47:0/2652367964 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9d60140940 0x7f9d60148330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:41.518 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.511+0000 7f9d6872e700 1 --2- 172.21.15.47:0/2652367964 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9d40053980 0x7f9d40055e40 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:41.518 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.511+0000 7f9d6872e700 1 --2- 172.21.15.47:0/2652367964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d600b35d0 0x7f9d60141350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:41.518 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.511+0000 7f9d6872e700 1 -- 172.21.15.47:0/2652367964 >> 172.21.15.47:0/2652367964 conn(0x7f9d600627d0 msgr2=0x7f9d600619f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:41.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.511+0000 7f9d6872e700 1 -- 172.21.15.47:0/2652367964 shutdown_connections 2022-06-29T15:50:41.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.511+0000 7f9d6872e700 1 -- 172.21.15.47:0/2652367964 wait complete. 2022-06-29T15:50:41.520 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.1 2022-06-29T15:50:41.529 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:50:41.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.631+0000 7f8a55b40700 1 Processor -- start 2022-06-29T15:50:41.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.631+0000 7f8a55b40700 1 -- start start 2022-06-29T15:50:41.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.631+0000 7f8a55b40700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8a50145090 0x7f8a500b2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:41.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.631+0000 7f8a55b40700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a500a9120 con 0x7f8a500b5c60 2022-06-29T15:50:41.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.631+0000 7f8a55b40700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a500b0b80 con 0x7f8a50060eb0 2022-06-29T15:50:41.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.631+0000 7f8a55b40700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a500a89e0 con 0x7f8a50145090 2022-06-29T15:50:41.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.631+0000 7f8a5533f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8a50145090 0x7f8a500b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:41.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.631+0000 7f8a5533f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8a50145090 0x7f8a500b2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52868/0 (socket says 172.21.15.47:52868) 2022-06-29T15:50:41.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.631+0000 7f8a5533f700 1 -- 172.21.15.47:0/489744862 learned_addr learned my addr 172.21.15.47:0/489744862 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:41.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.631+0000 7f8a54b3e700 1 --1- 172.21.15.47:0/489744862 >> v1:172.21.15.47:6789/0 conn(0x7f8a500b5c60 0x7f8a50144980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38418/0 (socket says 172.21.15.47:38418) 2022-06-29T15:50:41.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.631+0000 7f8a5533f700 1 -- 172.21.15.47:0/489744862 >> v1:172.21.15.64:6789/0 conn(0x7f8a50060eb0 legacy=0x7f8a500b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:41.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.631+0000 7f8a5533f700 1 -- 172.21.15.47:0/489744862 >> v1:172.21.15.47:6789/0 conn(0x7f8a500b5c60 legacy=0x7f8a50144980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:41.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.631+0000 7f8a5533f700 1 -- 172.21.15.47:0/489744862 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a500a8c80 con 0x7f8a50145090 2022-06-29T15:50:41.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.631+0000 7f8a5533f700 1 --2- 172.21.15.47:0/489744862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a50145090 0x7f8a500b2540 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 rx=0x7f8a48002f70 tx=0x7f8a48006eb0).ready entity=mon.0 client_cookie=78267974eb47be4f server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:41.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.631+0000 7f8a4f7fe700 1 -- 172.21.15.47:0/489744862 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a4800d660 con 0x7f8a50145090 2022-06-29T15:50:41.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.631+0000 7f8a4f7fe700 1 -- 172.21.15.47:0/489744862 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a48000e90 con 0x7f8a50145090 2022-06-29T15:50:41.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.631+0000 7f8a4f7fe700 1 -- 172.21.15.47:0/489744862 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a4800d9f0 con 0x7f8a50145090 2022-06-29T15:50:41.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.631+0000 7f8a55b40700 1 -- 172.21.15.47:0/489744862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a50145090 msgr2=0x7f8a500b2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:41.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.631+0000 7f8a55b40700 1 --2- 172.21.15.47:0/489744862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a50145090 0x7f8a500b2540 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 rx=0x7f8a48002f70 tx=0x7f8a48006eb0).stop 2022-06-29T15:50:41.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.635+0000 7f8a55b40700 1 -- 172.21.15.47:0/489744862 shutdown_connections 2022-06-29T15:50:41.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.635+0000 7f8a55b40700 1 --2- 172.21.15.47:0/489744862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a50145090 0x7f8a500b2540 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:41.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.635+0000 7f8a55b40700 1 -- 172.21.15.47:0/489744862 >> 172.21.15.47:0/489744862 conn(0x7f8a50062560 msgr2=0x7f8a50062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:41.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.635+0000 7f8a55b40700 1 -- 172.21.15.47:0/489744862 shutdown_connections 2022-06-29T15:50:41.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.635+0000 7f8a55b40700 1 -- 172.21.15.47:0/489744862 wait complete. 2022-06-29T15:50:41.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.635+0000 7f8a55b40700 1 Processor -- start 2022-06-29T15:50:41.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.635+0000 7f8a55b40700 1 -- start start 2022-06-29T15:50:41.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.639+0000 7f8a55b40700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8a50060eb0 0x7f8a5015a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:41.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.639+0000 7f8a55b40700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a500b5c60 0x7f8a50141720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:41.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.639+0000 7f8a55b40700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a50145090 0x7f8a500b38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:41.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.639+0000 7f8a55b40700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a50149d70 con 0x7f8a50145090 2022-06-29T15:50:41.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.639+0000 7f8a55b40700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a50148980 con 0x7f8a500b5c60 2022-06-29T15:50:41.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.639+0000 7f8a55b40700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a50148c20 con 0x7f8a50060eb0 2022-06-29T15:50:41.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.639+0000 7f8a5533f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a50145090 0x7f8a500b38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:41.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.639+0000 7f8a5533f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a50145090 0x7f8a500b38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52874/0 (socket says 172.21.15.47:52874) 2022-06-29T15:50:41.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.639+0000 7f8a5533f700 1 -- 172.21.15.47:0/793793536 learned_addr learned my addr 172.21.15.47:0/793793536 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:41.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.639+0000 7f8a4ffff700 1 --2- 172.21.15.47:0/793793536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a500b5c60 0x7f8a50141720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:41.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.639+0000 7f8a54b3e700 1 --2- 172.21.15.47:0/793793536 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8a50060eb0 0x7f8a5015a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:41.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.639+0000 7f8a5533f700 1 -- 172.21.15.47:0/793793536 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8a50060eb0 msgr2=0x7f8a5015a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:41.651 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.639+0000 7f8a5533f700 1 --2- 172.21.15.47:0/793793536 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8a50060eb0 0x7f8a5015a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:41.651 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.639+0000 7f8a5533f700 1 -- 172.21.15.47:0/793793536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a500b5c60 msgr2=0x7f8a50141720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:41.651 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.639+0000 7f8a5533f700 1 --2- 172.21.15.47:0/793793536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a500b5c60 0x7f8a50141720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:41.651 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.639+0000 7f8a5533f700 1 -- 172.21.15.47:0/793793536 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a4800b1e0 con 0x7f8a50145090 2022-06-29T15:50:41.652 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.639+0000 7f8a5533f700 1 --2- 172.21.15.47:0/793793536 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a50145090 0x7f8a500b38c0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 rx=0x7f8a48002f70 tx=0x7f8a48015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:41.652 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.639+0000 7f8a4dffb700 1 -- 172.21.15.47:0/793793536 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a48011b90 con 0x7f8a50145090 2022-06-29T15:50:41.652 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.639+0000 7f8a4dffb700 1 -- 172.21.15.47:0/793793536 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a48011cf0 con 0x7f8a50145090 2022-06-29T15:50:41.652 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.639+0000 7f8a55b40700 1 -- 172.21.15.47:0/793793536 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a50148ec0 con 0x7f8a50145090 2022-06-29T15:50:41.652 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.639+0000 7f8a4dffb700 1 -- 172.21.15.47:0/793793536 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a4801e650 con 0x7f8a50145090 2022-06-29T15:50:41.653 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.639+0000 7f8a55b40700 1 -- 172.21.15.47:0/793793536 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a501492f0 con 0x7f8a50145090 2022-06-29T15:50:41.653 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.639+0000 7f8a55b40700 1 -- 172.21.15.47:0/793793536 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a500573d0 con 0x7f8a50145090 2022-06-29T15:50:41.654 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.643+0000 7f8a4dffb700 1 -- 172.21.15.47:0/793793536 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8a48025030 con 0x7f8a50145090 2022-06-29T15:50:41.654 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.643+0000 7f8a4dffb700 1 --2- 172.21.15.47:0/793793536 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8a30053a70 0x7f8a30055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:41.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.643+0000 7f8a4dffb700 1 -- 172.21.15.47:0/793793536 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f8a4806d380 con 0x7f8a50145090 2022-06-29T15:50:41.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.643+0000 7f8a54b3e700 1 --2- 172.21.15.47:0/793793536 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8a30053a70 0x7f8a30055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:41.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.643+0000 7f8a54b3e700 1 --2- 172.21.15.47:0/793793536 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8a30053a70 0x7f8a30055f30 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7f8a4400d000 tx=0x7f8a44009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:41.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.651+0000 7f8a4dffb700 1 -- 172.21.15.47:0/793793536 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8a48044070 con 0x7f8a50145090 2022-06-29T15:50:41.825 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.819+0000 7f8a55b40700 1 -- 172.21.15.47:0/793793536 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a50069e20 con 0x7f8a50145090 2022-06-29T15:50:41.826 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.819+0000 7f8a4dffb700 1 -- 172.21.15.47:0/793793536 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+2498 (secure 0 0 0) 0x7f8a48023090 con 0x7f8a50145090 2022-06-29T15:50:41.827 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:50:41.827 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":57,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7614,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/2948555969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2948555969},{"type":"v1","addr":"172.21.15.47:6835","nonce":2948555969}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":54},{"gid":7617,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.47:6837/1681295902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1681295902},{"type":"v1","addr":"172.21.15.47:6837","nonce":1681295902}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":55}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:50:38.459255+0000","modified":"2022-06-29T15:50:40.277564+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":56606},"failed":[],"damaged":[],"stopped":[],"info":{"gid_56606":{"gid":56606,"name":"b","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6833/2987151459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2987151459},{"type":"v1","addr":"172.21.15.64:6833","nonce":2987151459}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2022-06-29T15:50:41.830 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.823+0000 7f8a55b40700 1 -- 172.21.15.47:0/793793536 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8a30053a70 msgr2=0x7f8a30055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:41.830 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.823+0000 7f8a55b40700 1 --2- 172.21.15.47:0/793793536 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8a30053a70 0x7f8a30055f30 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7f8a4400d000 tx=0x7f8a44009040).stop 2022-06-29T15:50:41.830 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.823+0000 7f8a55b40700 1 -- 172.21.15.47:0/793793536 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a50145090 msgr2=0x7f8a500b38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:41.831 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.823+0000 7f8a55b40700 1 --2- 172.21.15.47:0/793793536 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a50145090 0x7f8a500b38c0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 rx=0x7f8a48002f70 tx=0x7f8a48015000).stop 2022-06-29T15:50:41.831 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.823+0000 7f8a55b40700 1 -- 172.21.15.47:0/793793536 shutdown_connections 2022-06-29T15:50:41.831 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.823+0000 7f8a55b40700 1 --2- 172.21.15.47:0/793793536 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a50145090 0x7f8a500b38c0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:41.831 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.823+0000 7f8a55b40700 1 --2- 172.21.15.47:0/793793536 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8a30053a70 0x7f8a30055f30 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:41.832 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.823+0000 7f8a55b40700 1 --2- 172.21.15.47:0/793793536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a500b5c60 0x7f8a50141720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:41.832 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.823+0000 7f8a55b40700 1 --2- 172.21.15.47:0/793793536 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8a50060eb0 0x7f8a5015a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:41.832 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.823+0000 7f8a55b40700 1 -- 172.21.15.47:0/793793536 >> 172.21.15.47:0/793793536 conn(0x7f8a50062560 msgr2=0x7f8a50144750 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:41.832 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.823+0000 7f8a55b40700 1 -- 172.21.15.47:0/793793536 shutdown_connections 2022-06-29T15:50:41.833 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:41.823+0000 7f8a55b40700 1 -- 172.21.15.47:0/793793536 wait complete. 2022-06-29T15:50:41.833 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 57 2022-06-29T15:50:42.143 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 57, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-29T15:50:38.459255+0000', 'modified': '2022-06-29T15:50:40.277564+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 56606}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_56606': {'gid': 56606, 'name': 'b', 'rank': 0, 'incarnation': 55, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.64:6833/2987151459', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6832', 'nonce': 2987151459}, {'type': 'v1', 'addr': '172.21.15.64:6833', 'nonce': 2987151459}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-29T15:50:42.144 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-29T15:50:42.144 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-06-29T15:50:42.261 INFO:teuthology.orchestra.run.smithi064.stdout:{"cluster_fsid":"9a68894a-53ce-400e-838b-14eab91b834c","whoami":0,"id":56606,"want_state":"up:active","state":"up:active","rank_uptime":3.7837611409999998,"mdsmap_epoch":57,"osdmap_epoch":121,"osdmap_epoch_barrier":121,"uptime":8.0532244130000006} 2022-06-29T15:50:42.262 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9a68894a-53ce-400e-838b-14eab91b834c", "id": 56606, "mdsmap_epoch": 57, "osdmap_epoch": 121, "osdmap_epoch_barrier": 121, "rank_uptime": 3.783761141, "state": "up:active", "uptime": 8.053224413, "want_state": "up:active", "whoami": 0 } 2022-06-29T15:50:42.263 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:50:42.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.367+0000 7ff71f59e700 1 Processor -- start 2022-06-29T15:50:42.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.367+0000 7ff71f59e700 1 -- start start 2022-06-29T15:50:42.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.371+0000 7ff71f59e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff7200b29a0 0x7ff7200afb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:42.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.371+0000 7ff71f59e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff720146510 con 0x7ff7200b25c0 2022-06-29T15:50:42.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.371+0000 7ff71f59e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7201467b0 con 0x7ff7200b1fc0 2022-06-29T15:50:42.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.371+0000 7ff71f59e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7200b0040 con 0x7ff7200b29a0 2022-06-29T15:50:42.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.371+0000 7ff71e59c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff7200b25c0 0x7ff7201403b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38432/0 (socket says 172.21.15.47:38432) 2022-06-29T15:50:42.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.371+0000 7ff71e59c700 1 -- 172.21.15.47:0/3321806287 learned_addr learned my addr 172.21.15.47:0/3321806287 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:42.379 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.371+0000 7ff71ed9d700 1 --2- 172.21.15.47:0/3321806287 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff7200b29a0 0x7ff7200afb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:42.385 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.371+0000 7ff71d59a700 1 -- 172.21.15.47:0/3321806287 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2634306715 0 0) 0x7ff720146510 con 0x7ff7200b25c0 2022-06-29T15:50:42.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.371+0000 7ff71d59a700 1 -- 172.21.15.47:0/3321806287 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff704003260 con 0x7ff7200b25c0 2022-06-29T15:50:42.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.371+0000 7ff71d59a700 1 -- 172.21.15.47:0/3321806287 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3674398237 0 0) 0x7ff7201467b0 con 0x7ff7200b1fc0 2022-06-29T15:50:42.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.371+0000 7ff71d59a700 1 -- 172.21.15.47:0/3321806287 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff720146510 con 0x7ff7200b1fc0 2022-06-29T15:50:42.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.371+0000 7ff71ed9d700 1 -- 172.21.15.47:0/3321806287 >> v1:172.21.15.64:6789/0 conn(0x7ff7200b1fc0 legacy=0x7ff7200af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:42.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.371+0000 7ff71ed9d700 1 -- 172.21.15.47:0/3321806287 >> v1:172.21.15.47:6789/0 conn(0x7ff7200b25c0 legacy=0x7ff7201403b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:42.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.371+0000 7ff71ed9d700 1 -- 172.21.15.47:0/3321806287 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff720146a50 con 0x7ff7200b29a0 2022-06-29T15:50:42.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.371+0000 7ff71ed9d700 1 --2- 172.21.15.47:0/3321806287 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff7200b29a0 0x7ff7200afb50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7ff7180032f0 tx=0x7ff7180091d0).ready entity=mon.2 client_cookie=ee8aa0c9645d5d91 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:42.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.371+0000 7ff71d59a700 1 -- 172.21.15.47:0/3321806287 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff71800b640 con 0x7ff7200b29a0 2022-06-29T15:50:42.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.371+0000 7ff71d59a700 1 -- 172.21.15.47:0/3321806287 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff71800b7a0 con 0x7ff7200b29a0 2022-06-29T15:50:42.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.371+0000 7ff71d59a700 1 -- 172.21.15.47:0/3321806287 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff71800b910 con 0x7ff7200b29a0 2022-06-29T15:50:42.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.371+0000 7ff71f59e700 1 -- 172.21.15.47:0/3321806287 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff7200b29a0 msgr2=0x7ff7200afb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:42.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.371+0000 7ff71f59e700 1 --2- 172.21.15.47:0/3321806287 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff7200b29a0 0x7ff7200afb50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7ff7180032f0 tx=0x7ff7180091d0).stop 2022-06-29T15:50:42.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.371+0000 7ff71f59e700 1 -- 172.21.15.47:0/3321806287 shutdown_connections 2022-06-29T15:50:42.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.371+0000 7ff71f59e700 1 --2- 172.21.15.47:0/3321806287 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff7200b29a0 0x7ff7200afb50 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:42.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.371+0000 7ff71f59e700 1 -- 172.21.15.47:0/3321806287 >> 172.21.15.47:0/3321806287 conn(0x7ff720062df0 msgr2=0x7ff7200631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:42.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.371+0000 7ff71f59e700 1 -- 172.21.15.47:0/3321806287 shutdown_connections 2022-06-29T15:50:42.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.371+0000 7ff71f59e700 1 -- 172.21.15.47:0/3321806287 wait complete. 2022-06-29T15:50:42.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.375+0000 7ff71f59e700 1 Processor -- start 2022-06-29T15:50:42.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.375+0000 7ff71f59e700 1 -- start start 2022-06-29T15:50:42.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.375+0000 7ff71f59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7200b1fc0 0x7ff7200ab6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:42.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.375+0000 7ff71f59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7200b25c0 0x7ff7200b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:42.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.375+0000 7ff71f59e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff7200b29a0 0x7ff72009b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:42.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.375+0000 7ff71f59e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff72009a870 con 0x7ff7200b25c0 2022-06-29T15:50:42.394 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.375+0000 7ff71f59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff72009ba10 con 0x7ff7200b1fc0 2022-06-29T15:50:42.394 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.375+0000 7ff71f59e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7200aaeb0 con 0x7ff7200b29a0 2022-06-29T15:50:42.394 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.375+0000 7ff71dd9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7200b25c0 0x7ff7200b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:42.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.375+0000 7ff71dd9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7200b25c0 0x7ff7200b4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52886/0 (socket says 172.21.15.47:52886) 2022-06-29T15:50:42.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.375+0000 7ff71dd9b700 1 -- 172.21.15.47:0/2533273734 learned_addr learned my addr 172.21.15.47:0/2533273734 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:42.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.375+0000 7ff71ed9d700 1 --2- 172.21.15.47:0/2533273734 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff7200b29a0 0x7ff72009b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:42.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.375+0000 7ff71e59c700 1 --2- 172.21.15.47:0/2533273734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7200b1fc0 0x7ff7200ab6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:42.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.375+0000 7ff71dd9b700 1 -- 172.21.15.47:0/2533273734 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff7200b29a0 msgr2=0x7ff72009b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:42.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.375+0000 7ff71dd9b700 1 --2- 172.21.15.47:0/2533273734 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff7200b29a0 0x7ff72009b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:42.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.375+0000 7ff71dd9b700 1 -- 172.21.15.47:0/2533273734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7200b1fc0 msgr2=0x7ff7200ab6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:42.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.375+0000 7ff71dd9b700 1 --2- 172.21.15.47:0/2533273734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7200b1fc0 0x7ff7200ab6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:42.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.375+0000 7ff71dd9b700 1 -- 172.21.15.47:0/2533273734 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff718009470 con 0x7ff7200b25c0 2022-06-29T15:50:42.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.375+0000 7ff71e59c700 1 --2- 172.21.15.47:0/2533273734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7200b1fc0 0x7ff7200ab6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:50:42.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.375+0000 7ff71dd9b700 1 --2- 172.21.15.47:0/2533273734 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7200b25c0 0x7ff7200b4e30 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 rx=0x7ff7200aad50 tx=0x7ff710007ae0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:42.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.375+0000 7ff70f7fe700 1 -- 172.21.15.47:0/2533273734 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff71000ce40 con 0x7ff7200b25c0 2022-06-29T15:50:42.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.375+0000 7ff70f7fe700 1 -- 172.21.15.47:0/2533273734 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff71000f470 con 0x7ff7200b25c0 2022-06-29T15:50:42.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.375+0000 7ff71f59e700 1 -- 172.21.15.47:0/2533273734 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7200ac2d0 con 0x7ff7200b25c0 2022-06-29T15:50:42.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.379+0000 7ff70f7fe700 1 -- 172.21.15.47:0/2533273734 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff71000f7d0 con 0x7ff7200b25c0 2022-06-29T15:50:42.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.379+0000 7ff71f59e700 1 -- 172.21.15.47:0/2533273734 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7200aa910 con 0x7ff7200b25c0 2022-06-29T15:50:42.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.379+0000 7ff71f59e700 1 -- 172.21.15.47:0/2533273734 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7200573d0 con 0x7ff7200b25c0 2022-06-29T15:50:42.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.379+0000 7ff70f7fe700 1 -- 172.21.15.47:0/2533273734 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff710019030 con 0x7ff7200b25c0 2022-06-29T15:50:42.399 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:50:42.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.379+0000 7ff70f7fe700 1 --2- 172.21.15.47:0/2533273734 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff6fc053950 0x7ff6fc055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:42.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.379+0000 7ff71e59c700 1 --2- 172.21.15.47:0/2533273734 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff6fc053950 0x7ff6fc055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:42.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.379+0000 7ff70f7fe700 1 -- 172.21.15.47:0/2533273734 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff71006b600 con 0x7ff7200b25c0 2022-06-29T15:50:42.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.383+0000 7ff71e59c700 1 --2- 172.21.15.47:0/2533273734 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff6fc053950 0x7ff6fc055e10 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7ff71400b000 tx=0x7ff714009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:42.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.391+0000 7ff70f7fe700 1 -- 172.21.15.47:0/2533273734 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff710042230 con 0x7ff7200b25c0 2022-06-29T15:50:42.403 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:50:42.405 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:50:42.408 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:50:42.529 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.523+0000 7ff71f59e700 1 -- 172.21.15.47:0/2533273734 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff720098730 con 0x7ff7200b25c0 2022-06-29T15:50:42.531 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.523+0000 7ff70f7fe700 1 -- 172.21.15.47:0/2533273734 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+16285 (secure 0 0 0) 0x7ff710045850 con 0x7ff7200b25c0 2022-06-29T15:50:42.531 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:50:42.532 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":121,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:50:39.462097+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:50:37.087143+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2022-06-29T15:50:38.125088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:50:42.533 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.527+0000 7ff71f59e700 1 -- 172.21.15.47:0/2533273734 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff6fc053950 msgr2=0x7ff6fc055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:42.533 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.527+0000 7ff71f59e700 1 --2- 172.21.15.47:0/2533273734 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff6fc053950 0x7ff6fc055e10 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7ff71400b000 tx=0x7ff714009040).stop 2022-06-29T15:50:42.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.527+0000 7ff71f59e700 1 -- 172.21.15.47:0/2533273734 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7200b25c0 msgr2=0x7ff7200b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:42.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.527+0000 7ff71f59e700 1 --2- 172.21.15.47:0/2533273734 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7200b25c0 0x7ff7200b4e30 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 rx=0x7ff7200aad50 tx=0x7ff710007ae0).stop 2022-06-29T15:50:42.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.527+0000 7ff71f59e700 1 -- 172.21.15.47:0/2533273734 shutdown_connections 2022-06-29T15:50:42.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.527+0000 7ff71f59e700 1 --2- 172.21.15.47:0/2533273734 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff7200b29a0 0x7ff72009b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:42.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.527+0000 7ff71f59e700 1 --2- 172.21.15.47:0/2533273734 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7200b25c0 0x7ff7200b4e30 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:42.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.527+0000 7ff71f59e700 1 --2- 172.21.15.47:0/2533273734 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff6fc053950 0x7ff6fc055e10 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:42.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.527+0000 7ff71f59e700 1 --2- 172.21.15.47:0/2533273734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7200b1fc0 0x7ff7200ab6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:42.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.527+0000 7ff71f59e700 1 -- 172.21.15.47:0/2533273734 >> 172.21.15.47:0/2533273734 conn(0x7ff720062df0 msgr2=0x7ff720143630 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:42.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.527+0000 7ff71f59e700 1 -- 172.21.15.47:0/2533273734 shutdown_connections 2022-06-29T15:50:42.536 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.527+0000 7ff71f59e700 1 -- 172.21.15.47:0/2533273734 wait complete. 2022-06-29T15:50:42.546 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-29T15:42:16.971431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-29T15:42:25.290884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-29T15:50:37.087143+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2022-06-29T15:50:38.125088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-29T15:50:42.547 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-29T15:50:42.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.651+0000 7ff5c5cbe700 1 Processor -- start 2022-06-29T15:50:42.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.651+0000 7ff5c5cbe700 1 -- start start 2022-06-29T15:50:42.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.651+0000 7ff5c5cbe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff5c00b64c0 0x7ff5c00b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:42.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.651+0000 7ff5c5cbe700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5c0066fe0 0x7ff5c00b4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:42.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.651+0000 7ff5c5cbe700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5c00b68a0 con 0x7ff5c0143150 2022-06-29T15:50:42.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.651+0000 7ff5c5cbe700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5c00b13a0 con 0x7ff5c00b64c0 2022-06-29T15:50:42.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.651+0000 7ff5c5cbe700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c01473d0 con 0x7ff5c0066fe0 2022-06-29T15:50:42.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.651+0000 7ff5bffff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5c0066fe0 0x7ff5c00b4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:42.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.651+0000 7ff5c4cbc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff5c00b64c0 0x7ff5c00b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:42.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.651+0000 7ff5bffff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5c0066fe0 0x7ff5c00b4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:49994/0 (socket says 172.21.15.47:49994) 2022-06-29T15:50:42.663 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.651+0000 7ff5bffff700 1 -- 172.21.15.47:0/880969515 learned_addr learned my addr 172.21.15.47:0/880969515 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:42.663 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.655+0000 7ff5bf7fe700 1 -- 172.21.15.47:0/880969515 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1232903301 0 0) 0x7ff5c00b68a0 con 0x7ff5c0143150 2022-06-29T15:50:42.663 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.655+0000 7ff5bf7fe700 1 -- 172.21.15.47:0/880969515 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5a0003260 con 0x7ff5c0143150 2022-06-29T15:50:42.663 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.655+0000 7ff5bffff700 1 -- 172.21.15.47:0/880969515 >> v2:172.21.15.64:3300/0 conn(0x7ff5c00b64c0 msgr2=0x7ff5c00b5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:42.664 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.655+0000 7ff5bffff700 1 --2- 172.21.15.47:0/880969515 >> v2:172.21.15.64:3300/0 conn(0x7ff5c00b64c0 0x7ff5c00b5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:42.664 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.655+0000 7ff5bffff700 1 -- 172.21.15.47:0/880969515 >> v1:172.21.15.64:6789/0 conn(0x7ff5c0143150 legacy=0x7ff5c012bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:42.664 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.655+0000 7ff5bffff700 1 -- 172.21.15.47:0/880969515 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c00b46a0 con 0x7ff5c0066fe0 2022-06-29T15:50:42.664 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.655+0000 7ff5bffff700 1 --2- 172.21.15.47:0/880969515 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5c0066fe0 0x7ff5c00b4b40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7ff5a8002f70 tx=0x7ff5a8006e50).ready entity=mon.2 client_cookie=5a9a82374150c39 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:42.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.655+0000 7ff5bf7fe700 1 -- 172.21.15.47:0/880969515 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff5a800d570 con 0x7ff5c0066fe0 2022-06-29T15:50:42.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.655+0000 7ff5bf7fe700 1 -- 172.21.15.47:0/880969515 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5a8000e90 con 0x7ff5c0066fe0 2022-06-29T15:50:42.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.655+0000 7ff5bf7fe700 1 -- 172.21.15.47:0/880969515 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff5a800d900 con 0x7ff5c0066fe0 2022-06-29T15:50:42.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.655+0000 7ff5c5cbe700 1 -- 172.21.15.47:0/880969515 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5c0066fe0 msgr2=0x7ff5c00b4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:42.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.655+0000 7ff5c5cbe700 1 --2- 172.21.15.47:0/880969515 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5c0066fe0 0x7ff5c00b4b40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7ff5a8002f70 tx=0x7ff5a8006e50).stop 2022-06-29T15:50:42.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.655+0000 7ff5c5cbe700 1 -- 172.21.15.47:0/880969515 shutdown_connections 2022-06-29T15:50:42.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.655+0000 7ff5c5cbe700 1 --2- 172.21.15.47:0/880969515 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5c0066fe0 0x7ff5c00b4b40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:42.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.655+0000 7ff5c5cbe700 1 --2- 172.21.15.47:0/880969515 >> v2:172.21.15.64:3300/0 conn(0x7ff5c00b64c0 0x7ff5c00b5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:42.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.655+0000 7ff5c5cbe700 1 -- 172.21.15.47:0/880969515 >> 172.21.15.47:0/880969515 conn(0x7ff5c0060e00 msgr2=0x7ff5c00611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:42.676 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.655+0000 7ff5c5cbe700 1 -- 172.21.15.47:0/880969515 shutdown_connections 2022-06-29T15:50:42.676 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.655+0000 7ff5c5cbe700 1 -- 172.21.15.47:0/880969515 wait complete. 2022-06-29T15:50:42.677 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.659+0000 7ff5c5cbe700 1 Processor -- start 2022-06-29T15:50:42.677 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.659+0000 7ff5c5cbe700 1 -- start start 2022-06-29T15:50:42.678 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.659+0000 7ff5c5cbe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5c0066fe0 0x7ff5c00ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:42.678 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.659+0000 7ff5c5cbe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5c00b64c0 0x7ff5c00b1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:42.679 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.659+0000 7ff5c5cbe700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5c0143150 0x7ff5c00af290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:42.679 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.659+0000 7ff5c5cbe700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c00b03a0 con 0x7ff5c0066fe0 2022-06-29T15:50:42.679 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.659+0000 7ff5c5cbe700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c00ae8b0 con 0x7ff5c00b64c0 2022-06-29T15:50:42.680 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.659+0000 7ff5c5cbe700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c00aeb50 con 0x7ff5c0143150 2022-06-29T15:50:42.680 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.659+0000 7ff5c4cbc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5c0066fe0 0x7ff5c00ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:42.681 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.659+0000 7ff5c4cbc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5c0066fe0 0x7ff5c00ad8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52896/0 (socket says 172.21.15.47:52896) 2022-06-29T15:50:42.681 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.659+0000 7ff5c4cbc700 1 -- 172.21.15.47:0/961107485 learned_addr learned my addr 172.21.15.47:0/961107485 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:42.682 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.659+0000 7ff5c54bd700 1 --2- 172.21.15.47:0/961107485 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5c0143150 0x7ff5c00af290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:42.682 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.659+0000 7ff5bffff700 1 --2- 172.21.15.47:0/961107485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5c00b64c0 0x7ff5c00b1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:42.682 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.659+0000 7ff5c4cbc700 1 -- 172.21.15.47:0/961107485 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5c0143150 msgr2=0x7ff5c00af290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:42.683 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.659+0000 7ff5c4cbc700 1 --2- 172.21.15.47:0/961107485 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5c0143150 0x7ff5c00af290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:42.683 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.659+0000 7ff5c4cbc700 1 -- 172.21.15.47:0/961107485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5c00b64c0 msgr2=0x7ff5c00b1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:42.684 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.659+0000 7ff5c4cbc700 1 --2- 172.21.15.47:0/961107485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5c00b64c0 0x7ff5c00b1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:42.684 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.659+0000 7ff5c4cbc700 1 -- 172.21.15.47:0/961107485 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a800b1e0 con 0x7ff5c0066fe0 2022-06-29T15:50:42.684 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.659+0000 7ff5c4cbc700 1 --2- 172.21.15.47:0/961107485 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5c0066fe0 0x7ff5c00ad8a0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 rx=0x7ff5b000c000 tx=0x7ff5b0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:42.685 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.659+0000 7ff5bdffb700 1 -- 172.21.15.47:0/961107485 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff5b0010780 con 0x7ff5c0066fe0 2022-06-29T15:50:42.685 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.659+0000 7ff5bdffb700 1 -- 172.21.15.47:0/961107485 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b00108e0 con 0x7ff5c0066fe0 2022-06-29T15:50:42.686 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.659+0000 7ff5c5cbe700 1 -- 172.21.15.47:0/961107485 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5c00aedf0 con 0x7ff5c0066fe0 2022-06-29T15:50:42.686 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.659+0000 7ff5bdffb700 1 -- 172.21.15.47:0/961107485 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff5b0010c40 con 0x7ff5c0066fe0 2022-06-29T15:50:42.686 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.659+0000 7ff5c5cbe700 1 -- 172.21.15.47:0/961107485 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5c012c5e0 con 0x7ff5c0066fe0 2022-06-29T15:50:42.687 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.663+0000 7ff5c5cbe700 1 -- 172.21.15.47:0/961107485 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5c00573d0 con 0x7ff5c0066fe0 2022-06-29T15:50:42.687 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.663+0000 7ff5bdffb700 1 -- 172.21.15.47:0/961107485 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff5b001c460 con 0x7ff5c0066fe0 2022-06-29T15:50:42.688 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.663+0000 7ff5bdffb700 1 --2- 172.21.15.47:0/961107485 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff5a4053a70 0x7ff5a4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:42.688 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.663+0000 7ff5bdffb700 1 -- 172.21.15.47:0/961107485 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff5b006c400 con 0x7ff5c0066fe0 2022-06-29T15:50:42.689 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.663+0000 7ff5bffff700 1 --2- 172.21.15.47:0/961107485 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff5a4053a70 0x7ff5a4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:42.689 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.663+0000 7ff5bffff700 1 --2- 172.21.15.47:0/961107485 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff5a4053a70 0x7ff5a4055f30 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7ff5a801a000 tx=0x7ff5a8014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:42.690 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.667+0000 7ff5bdffb700 1 -- 172.21.15.47:0/961107485 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff5b0043160 con 0x7ff5c0066fe0 2022-06-29T15:50:42.799 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.791+0000 7ff5c5cbe700 1 -- 172.21.15.47:0/961107485 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7ff5c00aef80 con 0x7ff5c0066fe0 2022-06-29T15:50:42.800 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.795+0000 7ff5bdffb700 1 -- 172.21.15.47:0/961107485 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v121) v1 ==== 110+0+10 (secure 0 0 0) 0x7ff5b0046780 con 0x7ff5c0066fe0 2022-06-29T15:50:42.801 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2022-06-29T15:50:42.805 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.799+0000 7ff5c5cbe700 1 -- 172.21.15.47:0/961107485 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff5a4053a70 msgr2=0x7ff5a4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:42.805 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.799+0000 7ff5c5cbe700 1 --2- 172.21.15.47:0/961107485 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff5a4053a70 0x7ff5a4055f30 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7ff5a801a000 tx=0x7ff5a8014000).stop 2022-06-29T15:50:42.805 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.799+0000 7ff5c5cbe700 1 -- 172.21.15.47:0/961107485 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5c0066fe0 msgr2=0x7ff5c00ad8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:42.806 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.799+0000 7ff5c5cbe700 1 --2- 172.21.15.47:0/961107485 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5c0066fe0 0x7ff5c00ad8a0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 rx=0x7ff5b000c000 tx=0x7ff5b0006df0).stop 2022-06-29T15:50:42.806 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.799+0000 7ff5c5cbe700 1 -- 172.21.15.47:0/961107485 shutdown_connections 2022-06-29T15:50:42.806 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.799+0000 7ff5c5cbe700 1 --2- 172.21.15.47:0/961107485 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5c0143150 0x7ff5c00af290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:42.806 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.799+0000 7ff5c5cbe700 1 --2- 172.21.15.47:0/961107485 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff5a4053a70 0x7ff5a4055f30 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:42.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.799+0000 7ff5c5cbe700 1 --2- 172.21.15.47:0/961107485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5c00b64c0 0x7ff5c00b1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:42.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.799+0000 7ff5c5cbe700 1 --2- 172.21.15.47:0/961107485 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5c0066fe0 0x7ff5c00ad8a0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:42.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.799+0000 7ff5c5cbe700 1 -- 172.21.15.47:0/961107485 >> 172.21.15.47:0/961107485 conn(0x7ff5c0060e00 msgr2=0x7ff5c005bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:42.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.799+0000 7ff5c5cbe700 1 -- 172.21.15.47:0/961107485 shutdown_connections 2022-06-29T15:50:42.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.799+0000 7ff5c5cbe700 1 -- 172.21.15.47:0/961107485 wait complete. 2022-06-29T15:50:42.818 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-29T15:50:42.927 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.919+0000 7fbdef7c0700 1 Processor -- start 2022-06-29T15:50:42.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.919+0000 7fbdef7c0700 1 -- start start 2022-06-29T15:50:42.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.923+0000 7fbdef7c0700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbde8060ee0 0x7fbde80612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:42.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.923+0000 7fbdef7c0700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbde81471a0 0x7fbde8066f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:42.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.923+0000 7fbdef7c0700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbde8067610 con 0x7fbde80b4300 2022-06-29T15:50:42.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.923+0000 7fbdef7c0700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbde81467c0 con 0x7fbde8060ee0 2022-06-29T15:50:42.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.923+0000 7fbdef7c0700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbde8146a60 con 0x7fbde81471a0 2022-06-29T15:50:42.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.923+0000 7fbdecd5b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fbde80b4300 0x7fbde81430e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38462/0 (socket says 172.21.15.47:38462) 2022-06-29T15:50:42.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.923+0000 7fbded55c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbde8060ee0 0x7fbde80612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:42.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.923+0000 7fbdecd5b700 1 -- 172.21.15.47:0/3478574656 learned_addr learned my addr 172.21.15.47:0/3478574656 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:42.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.923+0000 7fbdedd5d700 1 --2- 172.21.15.47:0/3478574656 >> v2:172.21.15.64:3300/0 conn(0x7fbde81471a0 0x7fbde8066f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:42.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.923+0000 7fbddffff700 1 -- 172.21.15.47:0/3478574656 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 831959328 0 0) 0x7fbde8067610 con 0x7fbde80b4300 2022-06-29T15:50:42.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.923+0000 7fbddffff700 1 -- 172.21.15.47:0/3478574656 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdcc003260 con 0x7fbde80b4300 2022-06-29T15:50:42.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.923+0000 7fbded55c700 1 -- 172.21.15.47:0/3478574656 >> v2:172.21.15.64:3300/0 conn(0x7fbde81471a0 msgr2=0x7fbde8066f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:42.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.923+0000 7fbded55c700 1 --2- 172.21.15.47:0/3478574656 >> v2:172.21.15.64:3300/0 conn(0x7fbde81471a0 0x7fbde8066f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:42.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.923+0000 7fbded55c700 1 -- 172.21.15.47:0/3478574656 >> v1:172.21.15.47:6789/0 conn(0x7fbde80b4300 legacy=0x7fbde81430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:42.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.923+0000 7fbded55c700 1 -- 172.21.15.47:0/3478574656 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde8146d00 con 0x7fbde8060ee0 2022-06-29T15:50:42.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.923+0000 7fbded55c700 1 --2- 172.21.15.47:0/3478574656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbde8060ee0 0x7fbde80612c0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 rx=0x7fbde80ae790 tx=0x7fbde4006df0).ready entity=mon.0 client_cookie=1bbd0c2441dc8e28 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:42.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.923+0000 7fbddffff700 1 -- 172.21.15.47:0/3478574656 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbde400d4a0 con 0x7fbde8060ee0 2022-06-29T15:50:42.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.923+0000 7fbddffff700 1 -- 172.21.15.47:0/3478574656 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbde4000e90 con 0x7fbde8060ee0 2022-06-29T15:50:42.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.923+0000 7fbddffff700 1 -- 172.21.15.47:0/3478574656 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbde400d7f0 con 0x7fbde8060ee0 2022-06-29T15:50:42.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.923+0000 7fbdef7c0700 1 -- 172.21.15.47:0/3478574656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbde8060ee0 msgr2=0x7fbde80612c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:42.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.923+0000 7fbdef7c0700 1 --2- 172.21.15.47:0/3478574656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbde8060ee0 0x7fbde80612c0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 rx=0x7fbde80ae790 tx=0x7fbde4006df0).stop 2022-06-29T15:50:42.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.923+0000 7fbdef7c0700 1 -- 172.21.15.47:0/3478574656 shutdown_connections 2022-06-29T15:50:42.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.923+0000 7fbdef7c0700 1 --2- 172.21.15.47:0/3478574656 >> v2:172.21.15.64:3300/0 conn(0x7fbde81471a0 0x7fbde8066f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:42.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.923+0000 7fbdef7c0700 1 --2- 172.21.15.47:0/3478574656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbde8060ee0 0x7fbde80612c0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:42.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.923+0000 7fbdef7c0700 1 -- 172.21.15.47:0/3478574656 >> 172.21.15.47:0/3478574656 conn(0x7fbde805c510 msgr2=0x7fbde805f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:42.938 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.923+0000 7fbdef7c0700 1 -- 172.21.15.47:0/3478574656 shutdown_connections 2022-06-29T15:50:42.938 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.923+0000 7fbdef7c0700 1 -- 172.21.15.47:0/3478574656 wait complete. 2022-06-29T15:50:42.942 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.927+0000 7fbdef7c0700 1 Processor -- start 2022-06-29T15:50:42.942 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.927+0000 7fbdef7c0700 1 -- start start 2022-06-29T15:50:42.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.927+0000 7fbdef7c0700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbde80b4300 0x7fbde8145e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:42.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.927+0000 7fbdef7c0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbde81471a0 0x7fbde80b0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:42.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.927+0000 7fbdef7c0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbde80b11c0 0x7fbde8138190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:42.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.927+0000 7fbdef7c0700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde8142a40 con 0x7fbde81471a0 2022-06-29T15:50:42.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.927+0000 7fbdef7c0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde8141c60 con 0x7fbde80b11c0 2022-06-29T15:50:42.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.927+0000 7fbdef7c0700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbde8141f00 con 0x7fbde80b4300 2022-06-29T15:50:42.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.927+0000 7fbdecd5b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbde81471a0 0x7fbde80b0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:42.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.927+0000 7fbdecd5b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbde81471a0 0x7fbde80b0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52912/0 (socket says 172.21.15.47:52912) 2022-06-29T15:50:42.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.927+0000 7fbdecd5b700 1 -- 172.21.15.47:0/2170687571 learned_addr learned my addr 172.21.15.47:0/2170687571 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:42.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.927+0000 7fbded55c700 1 --2- 172.21.15.47:0/2170687571 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbde80b4300 0x7fbde8145e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:42.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.927+0000 7fbdedd5d700 1 --2- 172.21.15.47:0/2170687571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbde80b11c0 0x7fbde8138190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:42.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.931+0000 7fbdecd5b700 1 -- 172.21.15.47:0/2170687571 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbde80b4300 msgr2=0x7fbde8145e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:42.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.931+0000 7fbdecd5b700 1 --2- 172.21.15.47:0/2170687571 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbde80b4300 0x7fbde8145e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:42.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.931+0000 7fbdecd5b700 1 -- 172.21.15.47:0/2170687571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbde80b11c0 msgr2=0x7fbde8138190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:42.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.931+0000 7fbdecd5b700 1 --2- 172.21.15.47:0/2170687571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbde80b11c0 0x7fbde8138190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:42.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.931+0000 7fbdecd5b700 1 -- 172.21.15.47:0/2170687571 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde400b180 con 0x7fbde81471a0 2022-06-29T15:50:42.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.931+0000 7fbdedd5d700 1 --2- 172.21.15.47:0/2170687571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbde80b11c0 0x7fbde8138190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:50:42.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.931+0000 7fbdecd5b700 1 --2- 172.21.15.47:0/2170687571 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbde81471a0 0x7fbde80b0cd0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 rx=0x7fbdd8001110 tx=0x7fbdd8007aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:42.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.931+0000 7fbdde7fc700 1 -- 172.21.15.47:0/2170687571 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbdd800ce00 con 0x7fbde81471a0 2022-06-29T15:50:42.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.931+0000 7fbdde7fc700 1 -- 172.21.15.47:0/2170687571 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdd80072e0 con 0x7fbde81471a0 2022-06-29T15:50:42.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.931+0000 7fbdde7fc700 1 -- 172.21.15.47:0/2170687571 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbdd8005800 con 0x7fbde81471a0 2022-06-29T15:50:42.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.931+0000 7fbdef7c0700 1 -- 172.21.15.47:0/2170687571 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbde8142c70 con 0x7fbde81471a0 2022-06-29T15:50:42.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.931+0000 7fbdef7c0700 1 -- 172.21.15.47:0/2170687571 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbde8143110 con 0x7fbde81471a0 2022-06-29T15:50:42.950 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.931+0000 7fbdef7c0700 1 -- 172.21.15.47:0/2170687571 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbde80573d0 con 0x7fbde81471a0 2022-06-29T15:50:42.950 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.931+0000 7fbdde7fc700 1 -- 172.21.15.47:0/2170687571 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fbdd8010070 con 0x7fbde81471a0 2022-06-29T15:50:42.950 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.931+0000 7fbdde7fc700 1 --2- 172.21.15.47:0/2170687571 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbdd0053b00 0x7fbdd0055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:42.950 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.935+0000 7fbded55c700 1 --2- 172.21.15.47:0/2170687571 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbdd0053b00 0x7fbdd0055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:42.951 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.935+0000 7fbdde7fc700 1 -- 172.21.15.47:0/2170687571 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fbdd806b8f0 con 0x7fbde81471a0 2022-06-29T15:50:42.951 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.935+0000 7fbded55c700 1 --2- 172.21.15.47:0/2170687571 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbdd0053b00 0x7fbdd0055fc0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7fbde4018000 tx=0x7fbde4014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:42.951 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:42.939+0000 7fbdde7fc700 1 -- 172.21.15.47:0/2170687571 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbdd80425a0 con 0x7fbde81471a0 2022-06-29T15:50:43.072 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.067+0000 7fbdef7c0700 1 -- 172.21.15.47:0/2170687571 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbde8143d00 con 0x7fbde81471a0 2022-06-29T15:50:43.073 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.067+0000 7fbdde7fc700 1 -- 172.21.15.47:0/2170687571 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v121) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbdd8045bc0 con 0x7fbde81471a0 2022-06-29T15:50:43.073 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:50:43.075 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.067+0000 7fbdef7c0700 1 -- 172.21.15.47:0/2170687571 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbdd0053b00 msgr2=0x7fbdd0055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:43.075 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.067+0000 7fbdef7c0700 1 --2- 172.21.15.47:0/2170687571 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbdd0053b00 0x7fbdd0055fc0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7fbde4018000 tx=0x7fbde4014000).stop 2022-06-29T15:50:43.075 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.067+0000 7fbdef7c0700 1 -- 172.21.15.47:0/2170687571 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbde81471a0 msgr2=0x7fbde80b0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:43.075 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.067+0000 7fbdef7c0700 1 --2- 172.21.15.47:0/2170687571 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbde81471a0 0x7fbde80b0cd0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 rx=0x7fbdd8001110 tx=0x7fbdd8007aa0).stop 2022-06-29T15:50:43.076 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.067+0000 7fbdef7c0700 1 -- 172.21.15.47:0/2170687571 shutdown_connections 2022-06-29T15:50:43.076 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.067+0000 7fbdef7c0700 1 --2- 172.21.15.47:0/2170687571 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fbdd0053b00 0x7fbdd0055fc0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:43.076 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.067+0000 7fbdef7c0700 1 --2- 172.21.15.47:0/2170687571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbde80b11c0 0x7fbde8138190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:43.076 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.067+0000 7fbdef7c0700 1 --2- 172.21.15.47:0/2170687571 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbde81471a0 0x7fbde80b0cd0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:43.077 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.067+0000 7fbdef7c0700 1 --2- 172.21.15.47:0/2170687571 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fbde80b4300 0x7fbde8145e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:43.077 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.067+0000 7fbdef7c0700 1 -- 172.21.15.47:0/2170687571 >> 172.21.15.47:0/2170687571 conn(0x7fbde805c510 msgr2=0x7fbde80b8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:43.077 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.067+0000 7fbdef7c0700 1 -- 172.21.15.47:0/2170687571 shutdown_connections 2022-06-29T15:50:43.077 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.067+0000 7fbdef7c0700 1 -- 172.21.15.47:0/2170687571 wait complete. 2022-06-29T15:50:43.086 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-29T15:50:43.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.187+0000 7f72c8a4e700 1 Processor -- start 2022-06-29T15:50:43.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.187+0000 7f72c8a4e700 1 -- start start 2022-06-29T15:50:43.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.191+0000 7f72c8a4e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f72c40b6d30 0x7f72c413d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:43.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.191+0000 7f72c8a4e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f72c4062f30 0x7f72c413d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:43.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.191+0000 7f72c8a4e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72c4143f70 con 0x7f72c41400c0 2022-06-29T15:50:43.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.191+0000 7f72c8a4e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72c40b1380 con 0x7f72c40b6d30 2022-06-29T15:50:43.200 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.191+0000 7f72c8a4e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72c40a9cf0 con 0x7f72c4062f30 2022-06-29T15:50:43.200 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.191+0000 7f72c3fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f72c41400c0 0x7f72c4140510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38476/0 (socket says 172.21.15.47:38476) 2022-06-29T15:50:43.200 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.191+0000 7f72c3fff700 1 -- 172.21.15.47:0/3986391255 learned_addr learned my addr 172.21.15.47:0/3986391255 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:43.200 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.191+0000 7f72c37fe700 1 --2- 172.21.15.47:0/3986391255 >> v2:172.21.15.47:3300/0 conn(0x7f72c40b6d30 0x7f72c413d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:43.200 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.191+0000 7f72baffd700 1 --2- 172.21.15.47:0/3986391255 >> v2:172.21.15.64:3300/0 conn(0x7f72c4062f30 0x7f72c413d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:43.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.191+0000 7f72c2ffd700 1 -- 172.21.15.47:0/3986391255 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2243081187 0 0) 0x7f72c4143f70 con 0x7f72c41400c0 2022-06-29T15:50:43.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.191+0000 7f72c2ffd700 1 -- 172.21.15.47:0/3986391255 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72a4003260 con 0x7f72c41400c0 2022-06-29T15:50:43.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.191+0000 7f72c37fe700 1 -- 172.21.15.47:0/3986391255 >> v2:172.21.15.64:3300/0 conn(0x7f72c4062f30 msgr2=0x7f72c413d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:43.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.191+0000 7f72c37fe700 1 --2- 172.21.15.47:0/3986391255 >> v2:172.21.15.64:3300/0 conn(0x7f72c4062f30 0x7f72c413d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:43.202 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.191+0000 7f72c37fe700 1 -- 172.21.15.47:0/3986391255 >> v1:172.21.15.47:6789/0 conn(0x7f72c41400c0 legacy=0x7f72c4140510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:43.202 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.191+0000 7f72c37fe700 1 -- 172.21.15.47:0/3986391255 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c413dc80 con 0x7f72c40b6d30 2022-06-29T15:50:43.202 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.191+0000 7f72c37fe700 1 --2- 172.21.15.47:0/3986391255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72c40b6d30 0x7f72c413d2a0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 rx=0x7f72b4002f70 tx=0x7f72b4006df0).ready entity=mon.0 client_cookie=85ce36d8b3aef8e0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:43.202 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.191+0000 7f72c2ffd700 1 -- 172.21.15.47:0/3986391255 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72b400d4f0 con 0x7f72c40b6d30 2022-06-29T15:50:43.203 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.191+0000 7f72c2ffd700 1 -- 172.21.15.47:0/3986391255 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72b4000e90 con 0x7f72c40b6d30 2022-06-29T15:50:43.203 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.191+0000 7f72c2ffd700 1 -- 172.21.15.47:0/3986391255 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72b400d840 con 0x7f72c40b6d30 2022-06-29T15:50:43.203 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.191+0000 7f72c8a4e700 1 -- 172.21.15.47:0/3986391255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72c40b6d30 msgr2=0x7f72c413d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:43.203 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.191+0000 7f72c8a4e700 1 --2- 172.21.15.47:0/3986391255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72c40b6d30 0x7f72c413d2a0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 rx=0x7f72b4002f70 tx=0x7f72b4006df0).stop 2022-06-29T15:50:43.204 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.191+0000 7f72c8a4e700 1 -- 172.21.15.47:0/3986391255 shutdown_connections 2022-06-29T15:50:43.204 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.191+0000 7f72c8a4e700 1 --2- 172.21.15.47:0/3986391255 >> v2:172.21.15.64:3300/0 conn(0x7f72c4062f30 0x7f72c413d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:43.204 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.191+0000 7f72c8a4e700 1 --2- 172.21.15.47:0/3986391255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72c40b6d30 0x7f72c413d2a0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:43.204 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.191+0000 7f72c8a4e700 1 -- 172.21.15.47:0/3986391255 >> 172.21.15.47:0/3986391255 conn(0x7f72c4063670 msgr2=0x7f72c4063a50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:43.204 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.191+0000 7f72c8a4e700 1 -- 172.21.15.47:0/3986391255 shutdown_connections 2022-06-29T15:50:43.205 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.191+0000 7f72c8a4e700 1 -- 172.21.15.47:0/3986391255 wait complete. 2022-06-29T15:50:43.209 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.195+0000 7f72c8a4e700 1 Processor -- start 2022-06-29T15:50:43.209 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.195+0000 7f72c8a4e700 1 -- start start 2022-06-29T15:50:43.209 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.195+0000 7f72c8a4e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72c4062f30 0x7f72c40ac820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:43.209 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.195+0000 7f72c8a4e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72c41400c0 0x7f72c413c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:43.210 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.195+0000 7f72c8a4e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f72c413cc50 0x7f72c4153540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:43.210 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.195+0000 7f72c8a4e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c41434e0 con 0x7f72c41400c0 2022-06-29T15:50:43.210 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.195+0000 7f72c8a4e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c413d0e0 con 0x7f72c4062f30 2022-06-29T15:50:43.210 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.195+0000 7f72c8a4e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72c40b0050 con 0x7f72c413cc50 2022-06-29T15:50:43.211 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.195+0000 7f72baffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72c41400c0 0x7f72c413c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:43.211 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.195+0000 7f72baffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72c41400c0 0x7f72c413c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52924/0 (socket says 172.21.15.47:52924) 2022-06-29T15:50:43.211 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.195+0000 7f72baffd700 1 -- 172.21.15.47:0/3403536719 learned_addr learned my addr 172.21.15.47:0/3403536719 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:43.211 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.195+0000 7f72c3fff700 1 --2- 172.21.15.47:0/3403536719 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f72c413cc50 0x7f72c4153540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:43.212 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.195+0000 7f72c37fe700 1 --2- 172.21.15.47:0/3403536719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72c4062f30 0x7f72c40ac820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:43.212 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.195+0000 7f72baffd700 1 -- 172.21.15.47:0/3403536719 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f72c413cc50 msgr2=0x7f72c4153540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:43.212 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.195+0000 7f72baffd700 1 --2- 172.21.15.47:0/3403536719 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f72c413cc50 0x7f72c4153540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:43.212 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.195+0000 7f72baffd700 1 -- 172.21.15.47:0/3403536719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72c4062f30 msgr2=0x7f72c40ac820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:43.213 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.195+0000 7f72baffd700 1 --2- 172.21.15.47:0/3403536719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72c4062f30 0x7f72c40ac820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:43.213 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.195+0000 7f72baffd700 1 -- 172.21.15.47:0/3403536719 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b400b180 con 0x7f72c41400c0 2022-06-29T15:50:43.213 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.195+0000 7f72c37fe700 1 --2- 172.21.15.47:0/3403536719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72c4062f30 0x7f72c40ac820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:50:43.213 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.195+0000 7f72baffd700 1 --2- 172.21.15.47:0/3403536719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72c41400c0 0x7f72c413c760 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 rx=0x7f72ac00c000 tx=0x7f72ac006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:43.213 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.195+0000 7f72c17fa700 1 -- 172.21.15.47:0/3403536719 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72ac010780 con 0x7f72c41400c0 2022-06-29T15:50:43.214 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.199+0000 7f72c17fa700 1 -- 172.21.15.47:0/3403536719 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72ac0108e0 con 0x7f72c41400c0 2022-06-29T15:50:43.214 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.199+0000 7f72c8a4e700 1 -- 172.21.15.47:0/3403536719 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72c4144970 con 0x7f72c41400c0 2022-06-29T15:50:43.214 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.199+0000 7f72c17fa700 1 -- 172.21.15.47:0/3403536719 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72ac010c40 con 0x7f72c41400c0 2022-06-29T15:50:43.214 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.199+0000 7f72c8a4e700 1 -- 172.21.15.47:0/3403536719 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72c40b0570 con 0x7f72c41400c0 2022-06-29T15:50:43.215 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.199+0000 7f72c8a4e700 1 -- 172.21.15.47:0/3403536719 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72c40573d0 con 0x7f72c41400c0 2022-06-29T15:50:43.215 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.199+0000 7f72c17fa700 1 -- 172.21.15.47:0/3403536719 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f72ac01c460 con 0x7f72c41400c0 2022-06-29T15:50:43.215 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.199+0000 7f72c17fa700 1 --2- 172.21.15.47:0/3403536719 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f72a8053a70 0x7f72a8055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:43.215 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.199+0000 7f72c17fa700 1 -- 172.21.15.47:0/3403536719 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f72ac06c630 con 0x7f72c41400c0 2022-06-29T15:50:43.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.199+0000 7f72c37fe700 1 --2- 172.21.15.47:0/3403536719 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f72a8053a70 0x7f72a8055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:43.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.203+0000 7f72c37fe700 1 --2- 172.21.15.47:0/3403536719 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f72a8053a70 0x7f72a8055f30 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7f72b4015000 tx=0x7f72b4014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:43.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.207+0000 7f72c17fa700 1 -- 172.21.15.47:0/3403536719 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f72ac043200 con 0x7f72c41400c0 2022-06-29T15:50:43.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.339+0000 7f72c8a4e700 1 -- 172.21.15.47:0/3403536719 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f72c4069e20 con 0x7f72c41400c0 2022-06-29T15:50:43.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.339+0000 7f72c17fa700 1 -- 172.21.15.47:0/3403536719 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v121) v1 ==== 104+0+10 (secure 0 0 0) 0x7f72ac046820 con 0x7f72c41400c0 2022-06-29T15:50:43.345 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:50:43.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.339+0000 7f72c8a4e700 1 -- 172.21.15.47:0/3403536719 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f72a8053a70 msgr2=0x7f72a8055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:43.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.339+0000 7f72c8a4e700 1 --2- 172.21.15.47:0/3403536719 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f72a8053a70 0x7f72a8055f30 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7f72b4015000 tx=0x7f72b4014000).stop 2022-06-29T15:50:43.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.339+0000 7f72c8a4e700 1 -- 172.21.15.47:0/3403536719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72c41400c0 msgr2=0x7f72c413c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:43.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.339+0000 7f72c8a4e700 1 --2- 172.21.15.47:0/3403536719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72c41400c0 0x7f72c413c760 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 rx=0x7f72ac00c000 tx=0x7f72ac006df0).stop 2022-06-29T15:50:43.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.339+0000 7f72c8a4e700 1 -- 172.21.15.47:0/3403536719 shutdown_connections 2022-06-29T15:50:43.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.339+0000 7f72c8a4e700 1 --2- 172.21.15.47:0/3403536719 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f72c413cc50 0x7f72c4153540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:43.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.339+0000 7f72c8a4e700 1 --2- 172.21.15.47:0/3403536719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72c41400c0 0x7f72c413c760 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:43.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.339+0000 7f72c8a4e700 1 --2- 172.21.15.47:0/3403536719 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f72a8053a70 0x7f72a8055f30 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:43.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.339+0000 7f72c8a4e700 1 --2- 172.21.15.47:0/3403536719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72c4062f30 0x7f72c40ac820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:43.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.339+0000 7f72c8a4e700 1 -- 172.21.15.47:0/3403536719 >> 172.21.15.47:0/3403536719 conn(0x7f72c4063670 msgr2=0x7f72c40b29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:43.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.343+0000 7f72c8a4e700 1 -- 172.21.15.47:0/3403536719 shutdown_connections 2022-06-29T15:50:43.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.343+0000 7f72c8a4e700 1 -- 172.21.15.47:0/3403536719 wait complete. 2022-06-29T15:50:43.359 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-29T15:50:43.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.459+0000 7f5e8b59e700 1 Processor -- start 2022-06-29T15:50:43.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.459+0000 7f5e8b59e700 1 -- start start 2022-06-29T15:50:43.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.463+0000 7f5e8b59e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5e8c13d9b0 0x7f5e8c1407d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:43.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.463+0000 7f5e8b59e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e8c143f70 con 0x7f5e8c062f30 2022-06-29T15:50:43.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.463+0000 7f5e8b59e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e8c0a9cf0 con 0x7f5e8c0b6d30 2022-06-29T15:50:43.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.463+0000 7f5e8b59e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c0b1380 con 0x7f5e8c13d9b0 2022-06-29T15:50:43.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.463+0000 7f5e83fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f5e8c062f30 0x7f5e8c1400c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38488/0 (socket says 172.21.15.47:38488) 2022-06-29T15:50:43.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.463+0000 7f5e83fff700 1 -- 172.21.15.47:0/3915036627 learned_addr learned my addr 172.21.15.47:0/3915036627 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:43.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.463+0000 7f5e8ad9d700 1 --2- 172.21.15.47:0/3915036627 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5e8c13d9b0 0x7f5e8c1407d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:43.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.463+0000 7f5e89d9b700 1 -- 172.21.15.47:0/3915036627 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3376324817 0 0) 0x7f5e8c143f70 con 0x7f5e8c062f30 2022-06-29T15:50:43.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.463+0000 7f5e89d9b700 1 -- 172.21.15.47:0/3915036627 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e6c003260 con 0x7f5e8c062f30 2022-06-29T15:50:43.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.463+0000 7f5e89d9b700 1 -- 172.21.15.47:0/3915036627 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2205582982 0 0) 0x7f5e6c003260 con 0x7f5e8c062f30 2022-06-29T15:50:43.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.463+0000 7f5e89d9b700 1 -- 172.21.15.47:0/3915036627 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5e8c13d9b0 msgr2=0x7f5e8c1407d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:43.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.463+0000 7f5e89d9b700 1 --2- 172.21.15.47:0/3915036627 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5e8c13d9b0 0x7f5e8c1407d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:43.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.463+0000 7f5e89d9b700 1 -- 172.21.15.47:0/3915036627 >> v1:172.21.15.64:6789/0 conn(0x7f5e8c0b6d30 legacy=0x7f5e8c13d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:43.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.463+0000 7f5e89d9b700 1 -- 172.21.15.47:0/3915036627 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e8c1441a0 con 0x7f5e8c062f30 2022-06-29T15:50:43.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.463+0000 7f5e89d9b700 1 -- 172.21.15.47:0/3915036627 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7f5e74001660 con 0x7f5e8c062f30 2022-06-29T15:50:43.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.463+0000 7f5e89d9b700 1 -- 172.21.15.47:0/3915036627 >> v1:172.21.15.47:6789/0 conn(0x7f5e8c062f30 legacy=0x7f5e8c1400c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:43.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.463+0000 7f5e89d9b700 1 --2- 172.21.15.47:0/3915036627 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5e8c13d9b0 0x7f5e6c003780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:43.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.463+0000 7f5e89d9b700 1 --2- 172.21.15.47:0/3915036627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e6c003c70 0x7f5e6c004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:43.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.463+0000 7f5e89d9b700 1 --2- 172.21.15.47:0/3915036627 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e6c0049f0 0x7f5e6c004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:43.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.463+0000 7f5e89d9b700 1 -- 172.21.15.47:0/3915036627 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6c00d960 con 0x7f5e6c0049f0 2022-06-29T15:50:43.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.463+0000 7f5e89d9b700 1 -- 172.21.15.47:0/3915036627 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6c00daa0 con 0x7f5e6c003c70 2022-06-29T15:50:43.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.463+0000 7f5e89d9b700 1 -- 172.21.15.47:0/3915036627 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6c00dc60 con 0x7f5e8c13d9b0 2022-06-29T15:50:43.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.463+0000 7f5e8a59c700 1 --2- 172.21.15.47:0/3915036627 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5e8c13d9b0 0x7f5e6c003780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:43.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.463+0000 7f5e83fff700 1 --2- 172.21.15.47:0/3915036627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e6c003c70 0x7f5e6c004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:43.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.463+0000 7f5e8ad9d700 1 --2- 172.21.15.47:0/3915036627 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e6c0049f0 0x7f5e6c004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:43.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.463+0000 7f5e8ad9d700 1 -- 172.21.15.47:0/3915036627 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5e8c13d9b0 msgr2=0x7f5e6c003780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:43.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.463+0000 7f5e8ad9d700 1 --2- 172.21.15.47:0/3915036627 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5e8c13d9b0 0x7f5e6c003780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:43.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.463+0000 7f5e8ad9d700 1 -- 172.21.15.47:0/3915036627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e6c003c70 msgr2=0x7f5e6c004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:43.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.463+0000 7f5e8ad9d700 1 --2- 172.21.15.47:0/3915036627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e6c003c70 0x7f5e6c004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:43.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.463+0000 7f5e8ad9d700 1 -- 172.21.15.47:0/3915036627 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e6c003260 con 0x7f5e6c0049f0 2022-06-29T15:50:43.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.463+0000 7f5e8a59c700 1 --2- 172.21.15.47:0/3915036627 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5e8c13d9b0 0x7f5e6c003780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:50:43.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.463+0000 7f5e8ad9d700 1 --2- 172.21.15.47:0/3915036627 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e6c0049f0 0x7f5e6c004e70 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 rx=0x7f5e8400c000 tx=0x7f5e840091d0).ready entity=mon.0 client_cookie=cfcf7fab4052ce14 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:43.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.463+0000 7f5e89d9b700 1 -- 172.21.15.47:0/3915036627 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e8400e7b0 con 0x7f5e6c0049f0 2022-06-29T15:50:43.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.463+0000 7f5e89d9b700 1 -- 172.21.15.47:0/3915036627 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e8400e910 con 0x7f5e6c0049f0 2022-06-29T15:50:43.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.463+0000 7f5e89d9b700 1 -- 172.21.15.47:0/3915036627 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e8400ec70 con 0x7f5e6c0049f0 2022-06-29T15:50:43.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.467+0000 7f5e8b59e700 1 -- 172.21.15.47:0/3915036627 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e6c0049f0 msgr2=0x7f5e6c004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:43.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.467+0000 7f5e8b59e700 1 --2- 172.21.15.47:0/3915036627 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e6c0049f0 0x7f5e6c004e70 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 rx=0x7f5e8400c000 tx=0x7f5e840091d0).stop 2022-06-29T15:50:43.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.467+0000 7f5e8ad9d700 1 -- 172.21.15.47:0/3915036627 reap_dead start 2022-06-29T15:50:43.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.467+0000 7f5e8b59e700 1 -- 172.21.15.47:0/3915036627 shutdown_connections 2022-06-29T15:50:43.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.467+0000 7f5e8b59e700 1 -- 172.21.15.47:0/3915036627 >> 172.21.15.47:0/3915036627 conn(0x7f5e8c063670 msgr2=0x7f5e8c063a50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:43.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.467+0000 7f5e8b59e700 1 -- 172.21.15.47:0/3915036627 shutdown_connections 2022-06-29T15:50:43.484 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.467+0000 7f5e8b59e700 1 -- 172.21.15.47:0/3915036627 wait complete. 2022-06-29T15:50:43.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.467+0000 7f5e8b59e700 1 Processor -- start 2022-06-29T15:50:43.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.467+0000 7f5e8b59e700 1 -- start start 2022-06-29T15:50:43.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.467+0000 7f5e8b59e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5e8c0b55f0 0x7f5e8c142de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:43.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.467+0000 7f5e8b59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e8c0acf70 0x7f5e8c155690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:43.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.467+0000 7f5e8b59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e8c0aa2d0 0x7f5e8c155b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:43.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.467+0000 7f5e8b59e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c0b59d0 con 0x7f5e8c0aa2d0 2022-06-29T15:50:43.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.467+0000 7f5e8b59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c152ee0 con 0x7f5e8c0acf70 2022-06-29T15:50:43.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.467+0000 7f5e8b59e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c0ad780 con 0x7f5e8c0b55f0 2022-06-29T15:50:43.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.467+0000 7f5e8ad9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e8c0aa2d0 0x7f5e8c155b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:43.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.467+0000 7f5e8a59c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5e8c0b55f0 0x7f5e8c142de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:43.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.471+0000 7f5e8ad9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e8c0aa2d0 0x7f5e8c155b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52948/0 (socket says 172.21.15.47:52948) 2022-06-29T15:50:43.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.471+0000 7f5e8ad9d700 1 -- 172.21.15.47:0/1846848757 learned_addr learned my addr 172.21.15.47:0/1846848757 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:43.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.471+0000 7f5e8a59c700 1 --2- 172.21.15.47:0/1846848757 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5e8c0b55f0 0x7f5e8c142de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:50046/0 (socket says 172.21.15.47:50046) 2022-06-29T15:50:43.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.471+0000 7f5e83fff700 1 --2- 172.21.15.47:0/1846848757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e8c0acf70 0x7f5e8c155690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:43.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.471+0000 7f5e8ad9d700 1 -- 172.21.15.47:0/1846848757 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5e8c0b55f0 msgr2=0x7f5e8c142de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:43.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.471+0000 7f5e8ad9d700 1 --2- 172.21.15.47:0/1846848757 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5e8c0b55f0 0x7f5e8c142de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:43.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.471+0000 7f5e8ad9d700 1 -- 172.21.15.47:0/1846848757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e8c0acf70 msgr2=0x7f5e8c155690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:43.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.471+0000 7f5e8ad9d700 1 --2- 172.21.15.47:0/1846848757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e8c0acf70 0x7f5e8c155690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:43.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.471+0000 7f5e8ad9d700 1 -- 172.21.15.47:0/1846848757 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e8c0ada20 con 0x7f5e8c0aa2d0 2022-06-29T15:50:43.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.471+0000 7f5e83fff700 1 --2- 172.21.15.47:0/1846848757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e8c0acf70 0x7f5e8c155690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:50:43.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.471+0000 7f5e8ad9d700 1 --2- 172.21.15.47:0/1846848757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e8c0aa2d0 0x7f5e8c155b80 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 rx=0x7f5e8c0655d0 tx=0x7f5e8401a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:43.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.471+0000 7f5e837fe700 1 -- 172.21.15.47:0/1846848757 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e8400e7b0 con 0x7f5e8c0aa2d0 2022-06-29T15:50:43.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.471+0000 7f5e837fe700 1 -- 172.21.15.47:0/1846848757 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e840127f0 con 0x7f5e8c0aa2d0 2022-06-29T15:50:43.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.471+0000 7f5e8b59e700 1 -- 172.21.15.47:0/1846848757 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e8c1432d0 con 0x7f5e8c0aa2d0 2022-06-29T15:50:43.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.471+0000 7f5e837fe700 1 -- 172.21.15.47:0/1846848757 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e84009600 con 0x7f5e8c0aa2d0 2022-06-29T15:50:43.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.471+0000 7f5e8b59e700 1 -- 172.21.15.47:0/1846848757 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e8c156070 con 0x7f5e8c0aa2d0 2022-06-29T15:50:43.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.471+0000 7f5e8b59e700 1 -- 172.21.15.47:0/1846848757 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e8c0adbb0 con 0x7f5e8c0aa2d0 2022-06-29T15:50:43.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.471+0000 7f5e837fe700 1 -- 172.21.15.47:0/1846848757 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5e8402d070 con 0x7f5e8c0aa2d0 2022-06-29T15:50:43.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.471+0000 7f5e837fe700 1 --2- 172.21.15.47:0/1846848757 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5e70053a10 0x7f5e70055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:43.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.471+0000 7f5e8a59c700 1 --2- 172.21.15.47:0/1846848757 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5e70053a10 0x7f5e70055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:43.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.471+0000 7f5e837fe700 1 -- 172.21.15.47:0/1846848757 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f5e8406de00 con 0x7f5e8c0aa2d0 2022-06-29T15:50:43.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.475+0000 7f5e8a59c700 1 --2- 172.21.15.47:0/1846848757 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5e70053a10 0x7f5e70055ed0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7f5e7c009000 tx=0x7f5e7c009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:43.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.479+0000 7f5e837fe700 1 -- 172.21.15.47:0/1846848757 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5e84044bc0 con 0x7f5e8c0aa2d0 2022-06-29T15:50:43.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.607+0000 7f5e8b59e700 1 -- 172.21.15.47:0/1846848757 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5e8c0573d0 con 0x7f5e8c0aa2d0 2022-06-29T15:50:43.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.607+0000 7f5e837fe700 1 -- 172.21.15.47:0/1846848757 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v121) v1 ==== 100+0+10 (secure 0 0 0) 0x7f5e84016070 con 0x7f5e8c0aa2d0 2022-06-29T15:50:43.613 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:50:43.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.607+0000 7f5e8b59e700 1 -- 172.21.15.47:0/1846848757 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5e70053a10 msgr2=0x7f5e70055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:43.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.607+0000 7f5e8b59e700 1 --2- 172.21.15.47:0/1846848757 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5e70053a10 0x7f5e70055ed0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7f5e7c009000 tx=0x7f5e7c009040).stop 2022-06-29T15:50:43.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.607+0000 7f5e8b59e700 1 -- 172.21.15.47:0/1846848757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e8c0aa2d0 msgr2=0x7f5e8c155b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:43.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.607+0000 7f5e8b59e700 1 --2- 172.21.15.47:0/1846848757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e8c0aa2d0 0x7f5e8c155b80 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 rx=0x7f5e8c0655d0 tx=0x7f5e8401a000).stop 2022-06-29T15:50:43.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.607+0000 7f5e8b59e700 1 -- 172.21.15.47:0/1846848757 shutdown_connections 2022-06-29T15:50:43.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.607+0000 7f5e8b59e700 1 --2- 172.21.15.47:0/1846848757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e8c0aa2d0 0x7f5e8c155b80 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:43.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.607+0000 7f5e8b59e700 1 --2- 172.21.15.47:0/1846848757 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5e70053a10 0x7f5e70055ed0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:43.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.607+0000 7f5e8b59e700 1 --2- 172.21.15.47:0/1846848757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e8c0acf70 0x7f5e8c155690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:43.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.607+0000 7f5e8b59e700 1 --2- 172.21.15.47:0/1846848757 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5e8c0b55f0 0x7f5e8c142de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:43.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.607+0000 7f5e8b59e700 1 -- 172.21.15.47:0/1846848757 >> 172.21.15.47:0/1846848757 conn(0x7f5e8c063670 msgr2=0x7f5e8c13f850 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:43.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.607+0000 7f5e8b59e700 1 -- 172.21.15.47:0/1846848757 shutdown_connections 2022-06-29T15:50:43.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.607+0000 7f5e8b59e700 1 -- 172.21.15.47:0/1846848757 wait complete. 2022-06-29T15:50:43.632 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-29T15:50:43.632 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:50:43.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.731+0000 7fea89846700 1 Processor -- start 2022-06-29T15:50:43.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.731+0000 7fea89846700 1 -- start start 2022-06-29T15:50:43.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.731+0000 7fea89846700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fea8405c510 0x7fea84143bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:43.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.731+0000 7fea89846700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fea840b2c40 0x7fea840b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:43.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.731+0000 7fea89846700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea84142f90 con 0x7fea8405c8f0 2022-06-29T15:50:43.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.731+0000 7fea89846700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea841431f0 con 0x7fea840b2c40 2022-06-29T15:50:43.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.731+0000 7fea89846700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea84143490 con 0x7fea8405c510 2022-06-29T15:50:43.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.731+0000 7fea89045700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fea840b2c40 0x7fea840b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:43.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.731+0000 7fea83fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fea8405c8f0 0x7fea840b4100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38508/0 (socket says 172.21.15.47:38508) 2022-06-29T15:50:43.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.731+0000 7fea83fff700 1 -- 172.21.15.47:0/3036703505 learned_addr learned my addr 172.21.15.47:0/3036703505 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:43.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.731+0000 7fea88844700 1 --2- 172.21.15.47:0/3036703505 >> v2:172.21.15.64:3300/0 conn(0x7fea8405c510 0x7fea84143bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:43.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.731+0000 7fea89045700 1 -- 172.21.15.47:0/3036703505 >> v2:172.21.15.64:3300/0 conn(0x7fea8405c510 msgr2=0x7fea84143bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:43.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.731+0000 7fea89045700 1 --2- 172.21.15.47:0/3036703505 >> v2:172.21.15.64:3300/0 conn(0x7fea8405c510 0x7fea84143bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:43.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.731+0000 7fea837fe700 1 -- 172.21.15.47:0/3036703505 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 694446972 0 0) 0x7fea84142f90 con 0x7fea8405c8f0 2022-06-29T15:50:43.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.731+0000 7fea89045700 1 -- 172.21.15.47:0/3036703505 >> v1:172.21.15.47:6789/0 conn(0x7fea8405c8f0 legacy=0x7fea840b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:43.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.731+0000 7fea89045700 1 -- 172.21.15.47:0/3036703505 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea84143730 con 0x7fea840b2c40 2022-06-29T15:50:43.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.731+0000 7fea88844700 1 --2- 172.21.15.47:0/3036703505 >> v2:172.21.15.64:3300/0 conn(0x7fea8405c510 0x7fea84143bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:50:43.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.731+0000 7fea89045700 1 --2- 172.21.15.47:0/3036703505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea840b2c40 0x7fea840b30d0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 rx=0x7fea7c0032f0 tx=0x7fea7c0091d0).ready entity=mon.0 client_cookie=74c31bcd97a1f1ea server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:43.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.731+0000 7fea837fe700 1 -- 172.21.15.47:0/3036703505 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea7c009ea0 con 0x7fea840b2c40 2022-06-29T15:50:43.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.731+0000 7fea837fe700 1 -- 172.21.15.47:0/3036703505 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea7c00b870 con 0x7fea840b2c40 2022-06-29T15:50:43.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.731+0000 7fea837fe700 1 -- 172.21.15.47:0/3036703505 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea7c00b9e0 con 0x7fea840b2c40 2022-06-29T15:50:43.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.735+0000 7fea89846700 1 -- 172.21.15.47:0/3036703505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea840b2c40 msgr2=0x7fea840b30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:43.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.735+0000 7fea89846700 1 --2- 172.21.15.47:0/3036703505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea840b2c40 0x7fea840b30d0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 rx=0x7fea7c0032f0 tx=0x7fea7c0091d0).stop 2022-06-29T15:50:43.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.735+0000 7fea89846700 1 -- 172.21.15.47:0/3036703505 shutdown_connections 2022-06-29T15:50:43.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.735+0000 7fea89846700 1 --2- 172.21.15.47:0/3036703505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea840b2c40 0x7fea840b30d0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:43.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.735+0000 7fea89846700 1 --2- 172.21.15.47:0/3036703505 >> v2:172.21.15.64:3300/0 conn(0x7fea8405c510 0x7fea84143bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:43.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.735+0000 7fea89846700 1 -- 172.21.15.47:0/3036703505 >> 172.21.15.47:0/3036703505 conn(0x7fea84060e00 msgr2=0x7fea840611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:43.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.735+0000 7fea89846700 1 -- 172.21.15.47:0/3036703505 shutdown_connections 2022-06-29T15:50:43.752 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.735+0000 7fea89846700 1 -- 172.21.15.47:0/3036703505 wait complete. 2022-06-29T15:50:43.752 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.735+0000 7fea89846700 1 Processor -- start 2022-06-29T15:50:43.752 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.735+0000 7fea89846700 1 -- start start 2022-06-29T15:50:43.752 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.739+0000 7fea89846700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea8405c510 0x7fea840b6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:43.753 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.739+0000 7fea89846700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea8405c8f0 0x7fea84155670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:43.753 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.739+0000 7fea89846700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fea840b2c40 0x7fea84155b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:43.753 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.739+0000 7fea89846700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea84156050 con 0x7fea8405c510 2022-06-29T15:50:43.753 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.739+0000 7fea89846700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea84149940 con 0x7fea8405c8f0 2022-06-29T15:50:43.753 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.739+0000 7fea89846700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea84149be0 con 0x7fea840b2c40 2022-06-29T15:50:43.754 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.739+0000 7fea88844700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea8405c510 0x7fea840b6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:43.754 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.739+0000 7fea88844700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea8405c510 0x7fea840b6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52958/0 (socket says 172.21.15.47:52958) 2022-06-29T15:50:43.754 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.739+0000 7fea88844700 1 -- 172.21.15.47:0/2708984315 learned_addr learned my addr 172.21.15.47:0/2708984315 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:43.754 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.739+0000 7fea89045700 1 --2- 172.21.15.47:0/2708984315 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fea840b2c40 0x7fea84155b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:43.755 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.739+0000 7fea83fff700 1 --2- 172.21.15.47:0/2708984315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea8405c8f0 0x7fea84155670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:43.755 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.739+0000 7fea88844700 1 -- 172.21.15.47:0/2708984315 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fea840b2c40 msgr2=0x7fea84155b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:43.755 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.739+0000 7fea88844700 1 --2- 172.21.15.47:0/2708984315 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fea840b2c40 0x7fea84155b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:43.755 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.739+0000 7fea88844700 1 -- 172.21.15.47:0/2708984315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea8405c8f0 msgr2=0x7fea84155670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:43.756 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.739+0000 7fea88844700 1 --2- 172.21.15.47:0/2708984315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea8405c8f0 0x7fea84155670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:43.756 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.739+0000 7fea88844700 1 -- 172.21.15.47:0/2708984315 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea7c009470 con 0x7fea8405c510 2022-06-29T15:50:43.756 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.739+0000 7fea89045700 1 --2- 172.21.15.47:0/2708984315 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fea840b2c40 0x7fea84155b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:50:43.756 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.739+0000 7fea88844700 1 --2- 172.21.15.47:0/2708984315 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea8405c510 0x7fea840b6290 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 rx=0x7fea74006940 tx=0x7fea74008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:43.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.739+0000 7fea81ffb700 1 -- 172.21.15.47:0/2708984315 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea74011780 con 0x7fea8405c510 2022-06-29T15:50:43.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.739+0000 7fea81ffb700 1 -- 172.21.15.47:0/2708984315 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea740118e0 con 0x7fea8405c510 2022-06-29T15:50:43.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.739+0000 7fea89846700 1 -- 172.21.15.47:0/2708984315 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea84149e80 con 0x7fea8405c510 2022-06-29T15:50:43.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.739+0000 7fea81ffb700 1 -- 172.21.15.47:0/2708984315 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea74011c40 con 0x7fea8405c510 2022-06-29T15:50:43.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.739+0000 7fea89846700 1 -- 172.21.15.47:0/2708984315 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea840b6ca0 con 0x7fea8405c510 2022-06-29T15:50:43.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.739+0000 7fea89846700 1 -- 172.21.15.47:0/2708984315 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea840573d0 con 0x7fea8405c510 2022-06-29T15:50:43.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.743+0000 7fea81ffb700 1 -- 172.21.15.47:0/2708984315 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fea74012420 con 0x7fea8405c510 2022-06-29T15:50:43.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.743+0000 7fea81ffb700 1 --2- 172.21.15.47:0/2708984315 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fea64053b00 0x7fea64055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:43.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.743+0000 7fea81ffb700 1 -- 172.21.15.47:0/2708984315 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fea7406be60 con 0x7fea8405c510 2022-06-29T15:50:43.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.743+0000 7fea83fff700 1 --2- 172.21.15.47:0/2708984315 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fea64053b00 0x7fea64055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:43.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.743+0000 7fea83fff700 1 --2- 172.21.15.47:0/2708984315 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fea64053b00 0x7fea64055fc0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7fea7800b000 tx=0x7fea78009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:43.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.751+0000 7fea81ffb700 1 -- 172.21.15.47:0/2708984315 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fea74042bc0 con 0x7fea8405c510 2022-06-29T15:50:43.918 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.911+0000 7fea89846700 1 -- 172.21.15.47:0/2708984315 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea84061b00 con 0x7fea8405c510 2022-06-29T15:50:43.919 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.911+0000 7fea81ffb700 1 -- 172.21.15.47:0/2708984315 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+2498 (secure 0 0 0) 0x7fea740461e0 con 0x7fea8405c510 2022-06-29T15:50:43.919 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:50:43.920 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":57,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7614,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/2948555969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2948555969},{"type":"v1","addr":"172.21.15.47:6835","nonce":2948555969}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":54},{"gid":7617,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.47:6837/1681295902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1681295902},{"type":"v1","addr":"172.21.15.47:6837","nonce":1681295902}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":55}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:50:38.459255+0000","modified":"2022-06-29T15:50:40.277564+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":56606},"failed":[],"damaged":[],"stopped":[],"info":{"gid_56606":{"gid":56606,"name":"b","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6833/2987151459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2987151459},{"type":"v1","addr":"172.21.15.64:6833","nonce":2987151459}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2022-06-29T15:50:43.921 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.915+0000 7fea89846700 1 -- 172.21.15.47:0/2708984315 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fea64053b00 msgr2=0x7fea64055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:43.922 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.915+0000 7fea89846700 1 --2- 172.21.15.47:0/2708984315 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fea64053b00 0x7fea64055fc0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7fea7800b000 tx=0x7fea78009040).stop 2022-06-29T15:50:43.922 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.915+0000 7fea89846700 1 -- 172.21.15.47:0/2708984315 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea8405c510 msgr2=0x7fea840b6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:43.922 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.915+0000 7fea89846700 1 --2- 172.21.15.47:0/2708984315 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea8405c510 0x7fea840b6290 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 rx=0x7fea74006940 tx=0x7fea74008040).stop 2022-06-29T15:50:43.923 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.915+0000 7fea89846700 1 -- 172.21.15.47:0/2708984315 shutdown_connections 2022-06-29T15:50:43.923 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.915+0000 7fea89846700 1 --2- 172.21.15.47:0/2708984315 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fea840b2c40 0x7fea84155b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:43.923 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.915+0000 7fea89846700 1 --2- 172.21.15.47:0/2708984315 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fea64053b00 0x7fea64055fc0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:43.923 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.915+0000 7fea89846700 1 --2- 172.21.15.47:0/2708984315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea8405c8f0 0x7fea84155670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:43.924 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.915+0000 7fea89846700 1 --2- 172.21.15.47:0/2708984315 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea8405c510 0x7fea840b6290 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:43.924 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.915+0000 7fea89846700 1 -- 172.21.15.47:0/2708984315 >> 172.21.15.47:0/2708984315 conn(0x7fea84060e00 msgr2=0x7fea840b3640 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:43.924 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.915+0000 7fea89846700 1 -- 172.21.15.47:0/2708984315 shutdown_connections 2022-06-29T15:50:43.924 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:43.915+0000 7fea89846700 1 -- 172.21.15.47:0/2708984315 wait complete. 2022-06-29T15:50:43.925 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 57 2022-06-29T15:50:44.236 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 57, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-29T15:50:38.459255+0000', 'modified': '2022-06-29T15:50:40.277564+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 56606}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_56606': {'gid': 56606, 'name': 'b', 'rank': 0, 'incarnation': 55, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.64:6833/2987151459', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6832', 'nonce': 2987151459}, {'type': 'v1', 'addr': '172.21.15.64:6833', 'nonce': 2987151459}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-29T15:50:44.237 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-29T15:50:44.238 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-06-29T15:50:44.358 INFO:teuthology.orchestra.run.smithi064.stdout:{"cluster_fsid":"9a68894a-53ce-400e-838b-14eab91b834c","whoami":0,"id":56606,"want_state":"up:active","state":"up:active","rank_uptime":5.8815992049999997,"mdsmap_epoch":57,"osdmap_epoch":121,"osdmap_epoch_barrier":121,"uptime":10.151069896999999} 2022-06-29T15:50:44.359 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9a68894a-53ce-400e-838b-14eab91b834c", "id": 56606, "mdsmap_epoch": 57, "osdmap_epoch": 121, "osdmap_epoch_barrier": 121, "rank_uptime": 5.881599205, "state": "up:active", "uptime": 10.151069897, "want_state": "up:active", "whoami": 0 } 2022-06-29T15:50:44.360 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-29T15:50:44.360 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-06-29T15:50:44.360 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi099.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-06-29T15:50:44.361 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:44.361 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:50:44.371 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2022-06-29T15:50:44.388 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:50:44.388 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:50:44.453 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:50:44.454 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:50:44.455 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:44.455 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2022-06-29T15:50:44.462 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-29T15:50:44.462 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:50:44.504 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2022-06-29T15:50:44.525 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:50:44.525 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:50:44.540 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:50:44.541 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:50:44.541 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:44.542 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2022-06-29T15:50:44.584 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:2022-06-29T15:50:44.575+0000 7f04650d2080 -1 init, newargv = 0x55deb9345750 newargc=9 2022-06-29T15:50:44.585 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[13288]: starting ceph client 2022-06-29T15:50:44.595 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[13288]: starting fuse 2022-06-29T15:50:45.650 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2022-06-29T15:50:45.669 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:50:45.670 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:50:45.729 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:50:45.731 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:50:45.732 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:45.732 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2022-06-29T15:50:45.739 INFO:teuthology.orchestra.run.smithi099.stdout:52 2022-06-29T15:50:45.740 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-29T15:50:45.740 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:45.741 DEBUG:teuthology.orchestra.run.smithi099:> sudo python3 -c ' 2022-06-29T15:50:45.741 DEBUG:teuthology.orchestra.run.smithi099:> import glob 2022-06-29T15:50:45.742 DEBUG:teuthology.orchestra.run.smithi099:> import re 2022-06-29T15:50:45.742 DEBUG:teuthology.orchestra.run.smithi099:> import os 2022-06-29T15:50:45.742 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2022-06-29T15:50:45.743 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:50:45.743 DEBUG:teuthology.orchestra.run.smithi099:> def find_socket(client_name): 2022-06-29T15:50:45.744 DEBUG:teuthology.orchestra.run.smithi099:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-06-29T15:50:45.744 DEBUG:teuthology.orchestra.run.smithi099:> files = glob.glob(asok_path) 2022-06-29T15:50:45.744 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:50:45.745 DEBUG:teuthology.orchestra.run.smithi099:> # Given a non-glob path, it better be there 2022-06-29T15:50:45.745 DEBUG:teuthology.orchestra.run.smithi099:> if "*" not in asok_path: 2022-06-29T15:50:45.746 DEBUG:teuthology.orchestra.run.smithi099:> assert(len(files) == 1) 2022-06-29T15:50:45.746 DEBUG:teuthology.orchestra.run.smithi099:> return files[0] 2022-06-29T15:50:45.747 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:50:45.747 DEBUG:teuthology.orchestra.run.smithi099:> for f in files: 2022-06-29T15:50:45.748 DEBUG:teuthology.orchestra.run.smithi099:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-29T15:50:45.748 DEBUG:teuthology.orchestra.run.smithi099:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-29T15:50:45.748 DEBUG:teuthology.orchestra.run.smithi099:> return f 2022-06-29T15:50:45.749 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-29T15:50:45.749 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:50:45.750 DEBUG:teuthology.orchestra.run.smithi099:> print(find_socket("client.0")) 2022-06-29T15:50:45.750 DEBUG:teuthology.orchestra.run.smithi099:> ' 2022-06-29T15:50:45.826 INFO:teuthology.orchestra.run.smithi099.stdout:/var/run/ceph/ceph-client.0.13288.asok 2022-06-29T15:50:45.831 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.13288.asok 2022-06-29T15:50:45.832 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:45.832 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.13288.asok status 2022-06-29T15:50:45.919 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2022-06-29T15:50:45.919 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": { 2022-06-29T15:50:45.920 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_sha1": "f25ffac72e5a9e63860ff7deb6767a11f0559a16", 2022-06-29T15:50:45.920 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 15.2.16-304-gf25ffac7 (f25ffac72e5a9e63860ff7deb6767a11f0559a16) octopus (stable)", 2022-06-29T15:50:45.920 INFO:teuthology.orchestra.run.smithi099.stdout: "entity_id": "0", 2022-06-29T15:50:45.920 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": "smithi099", 2022-06-29T15:50:45.920 INFO:teuthology.orchestra.run.smithi099.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-06-29T15:50:45.921 INFO:teuthology.orchestra.run.smithi099.stdout: "pid": "13288", 2022-06-29T15:50:45.921 INFO:teuthology.orchestra.run.smithi099.stdout: "root": "/" 2022-06-29T15:50:45.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:50:45.921 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_count": 0, 2022-06-29T15:50:45.922 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_pinned_count": 0, 2022-06-29T15:50:45.922 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 7705, 2022-06-29T15:50:45.922 INFO:teuthology.orchestra.run.smithi099.stdout: "inst": { 2022-06-29T15:50:45.922 INFO:teuthology.orchestra.run.smithi099.stdout: "name": { 2022-06-29T15:50:45.923 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "client", 2022-06-29T15:50:45.923 INFO:teuthology.orchestra.run.smithi099.stdout: "num": 7705 2022-06-29T15:50:45.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:50:45.923 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2022-06-29T15:50:45.923 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2022-06-29T15:50:45.924 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:0", 2022-06-29T15:50:45.924 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2992758237 2022-06-29T15:50:45.924 INFO:teuthology.orchestra.run.smithi099.stdout: } 2022-06-29T15:50:45.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:50:45.925 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2022-06-29T15:50:45.925 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2022-06-29T15:50:45.925 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:0", 2022-06-29T15:50:45.925 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2992758237 2022-06-29T15:50:45.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:50:45.926 INFO:teuthology.orchestra.run.smithi099.stdout: "inst_str": "client.7705 172.21.15.99:0/2992758237", 2022-06-29T15:50:45.926 INFO:teuthology.orchestra.run.smithi099.stdout: "addr_str": "172.21.15.99:0/2992758237", 2022-06-29T15:50:45.926 INFO:teuthology.orchestra.run.smithi099.stdout: "inode_count": 1, 2022-06-29T15:50:45.926 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_epoch": 57, 2022-06-29T15:50:45.927 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch": 121, 2022-06-29T15:50:45.927 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch_barrier": 0, 2022-06-29T15:50:45.927 INFO:teuthology.orchestra.run.smithi099.stdout: "blacklisted": false, 2022-06-29T15:50:45.927 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs" 2022-06-29T15:50:45.927 INFO:teuthology.orchestra.run.smithi099.stdout:} 2022-06-29T15:50:45.929 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:45.929 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:50:45.930 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:50:45.937 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2022-06-29T15:50:45.938 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-29T15:50:45.938 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:45.939 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:50:46.001 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:50:46.114 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.107+0000 7fad2d491700 1 Processor -- start 2022-06-29T15:50:46.114 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.107+0000 7fad2d491700 1 -- start start 2022-06-29T15:50:46.116 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.107+0000 7fad2d491700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fad280619b0 0x7fad2813aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:46.116 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.107+0000 7fad2d491700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fad28060ff0 0x7fad2813af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:46.116 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.111+0000 7fad2d491700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad28061d90 con 0x7fad280b0f30 2022-06-29T15:50:46.116 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.111+0000 7fad2d491700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad280b04b0 con 0x7fad280619b0 2022-06-29T15:50:46.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.111+0000 7fad2d491700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad280b0750 con 0x7fad28060ff0 2022-06-29T15:50:46.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.111+0000 7fad267fc700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fad280b0f30 0x7fad28144a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38522/0 (socket says 172.21.15.47:38522) 2022-06-29T15:50:46.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.111+0000 7fad267fc700 1 -- 172.21.15.47:0/2455206535 learned_addr learned my addr 172.21.15.47:0/2455206535 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:46.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.111+0000 7fad26ffd700 1 --2- 172.21.15.47:0/2455206535 >> v2:172.21.15.47:3300/0 conn(0x7fad280619b0 0x7fad2813aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:46.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.111+0000 7fad277fe700 1 --2- 172.21.15.47:0/2455206535 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fad28060ff0 0x7fad2813af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:46.119 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.111+0000 7fad26ffd700 1 -- 172.21.15.47:0/2455206535 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fad28060ff0 msgr2=0x7fad2813af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:46.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.111+0000 7fad26ffd700 1 --2- 172.21.15.47:0/2455206535 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fad28060ff0 0x7fad2813af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:46.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.111+0000 7fad26ffd700 1 -- 172.21.15.47:0/2455206535 >> v1:172.21.15.47:6789/0 conn(0x7fad280b0f30 legacy=0x7fad28144a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:46.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.111+0000 7fad26ffd700 1 -- 172.21.15.47:0/2455206535 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad28144570 con 0x7fad280619b0 2022-06-29T15:50:46.121 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.111+0000 7fad26ffd700 1 --2- 172.21.15.47:0/2455206535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad280619b0 0x7fad2813aa30 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 rx=0x7fad18002f70 tx=0x7fad18006df0).ready entity=mon.0 client_cookie=95a5f2d0ad3b8876 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:46.121 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.111+0000 7fad25ffb700 1 -- 172.21.15.47:0/2455206535 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad1800d570 con 0x7fad280619b0 2022-06-29T15:50:46.121 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.111+0000 7fad25ffb700 1 -- 172.21.15.47:0/2455206535 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad18000e90 con 0x7fad280619b0 2022-06-29T15:50:46.122 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.111+0000 7fad25ffb700 1 -- 172.21.15.47:0/2455206535 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad1800d8c0 con 0x7fad280619b0 2022-06-29T15:50:46.122 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.111+0000 7fad2d491700 1 -- 172.21.15.47:0/2455206535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad280619b0 msgr2=0x7fad2813aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:46.122 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.111+0000 7fad2d491700 1 --2- 172.21.15.47:0/2455206535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad280619b0 0x7fad2813aa30 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 rx=0x7fad18002f70 tx=0x7fad18006df0).stop 2022-06-29T15:50:46.122 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.111+0000 7fad2d491700 1 -- 172.21.15.47:0/2455206535 shutdown_connections 2022-06-29T15:50:46.123 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.111+0000 7fad2d491700 1 --2- 172.21.15.47:0/2455206535 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fad28060ff0 0x7fad2813af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:46.123 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.111+0000 7fad2d491700 1 --2- 172.21.15.47:0/2455206535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad280619b0 0x7fad2813aa30 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:46.123 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.111+0000 7fad2d491700 1 -- 172.21.15.47:0/2455206535 >> 172.21.15.47:0/2455206535 conn(0x7fad2805c510 msgr2=0x7fad28063800 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:46.123 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.111+0000 7fad2d491700 1 -- 172.21.15.47:0/2455206535 shutdown_connections 2022-06-29T15:50:46.123 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.111+0000 7fad2d491700 1 -- 172.21.15.47:0/2455206535 wait complete. 2022-06-29T15:50:46.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.115+0000 7fad2d491700 1 Processor -- start 2022-06-29T15:50:46.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.115+0000 7fad2d491700 1 -- start start 2022-06-29T15:50:46.127 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.115+0000 7fad2d491700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad28060ff0 0x7fad28150630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:46.127 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.115+0000 7fad2d491700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad280619b0 0x7fad28066da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:46.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.115+0000 7fad2d491700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fad280b0f30 0x7fad2810f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:46.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.115+0000 7fad2d491700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad28113210 con 0x7fad28060ff0 2022-06-29T15:50:46.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.115+0000 7fad2d491700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad28150c20 con 0x7fad280619b0 2022-06-29T15:50:46.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.115+0000 7fad2d491700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad28150ec0 con 0x7fad280b0f30 2022-06-29T15:50:46.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.115+0000 7fad26ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad28060ff0 0x7fad28150630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:46.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.115+0000 7fad26ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad28060ff0 0x7fad28150630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52972/0 (socket says 172.21.15.47:52972) 2022-06-29T15:50:46.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.115+0000 7fad26ffd700 1 -- 172.21.15.47:0/859839984 learned_addr learned my addr 172.21.15.47:0/859839984 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:46.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.115+0000 7fad277fe700 1 --2- 172.21.15.47:0/859839984 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fad280b0f30 0x7fad2810f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:46.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.115+0000 7fad267fc700 1 --2- 172.21.15.47:0/859839984 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad280619b0 0x7fad28066da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:46.130 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.115+0000 7fad26ffd700 1 -- 172.21.15.47:0/859839984 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fad280b0f30 msgr2=0x7fad2810f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:46.130 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.115+0000 7fad26ffd700 1 --2- 172.21.15.47:0/859839984 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fad280b0f30 0x7fad2810f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:46.130 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.115+0000 7fad26ffd700 1 -- 172.21.15.47:0/859839984 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad280619b0 msgr2=0x7fad28066da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:46.130 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.115+0000 7fad26ffd700 1 --2- 172.21.15.47:0/859839984 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad280619b0 0x7fad28066da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:46.131 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.115+0000 7fad26ffd700 1 -- 172.21.15.47:0/859839984 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad1800b180 con 0x7fad28060ff0 2022-06-29T15:50:46.131 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.115+0000 7fad26ffd700 1 --2- 172.21.15.47:0/859839984 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad28060ff0 0x7fad28150630 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 rx=0x7fad18002f70 tx=0x7fad18015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:46.131 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.115+0000 7fad17fff700 1 -- 172.21.15.47:0/859839984 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad1800d8f0 con 0x7fad28060ff0 2022-06-29T15:50:46.131 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.115+0000 7fad17fff700 1 -- 172.21.15.47:0/859839984 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad18011b80 con 0x7fad28060ff0 2022-06-29T15:50:46.132 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.115+0000 7fad17fff700 1 -- 172.21.15.47:0/859839984 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad1801e460 con 0x7fad28060ff0 2022-06-29T15:50:46.132 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.115+0000 7fad2d491700 1 -- 172.21.15.47:0/859839984 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad28151160 con 0x7fad28060ff0 2022-06-29T15:50:46.132 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.115+0000 7fad2d491700 1 -- 172.21.15.47:0/859839984 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad281110c0 con 0x7fad28060ff0 2022-06-29T15:50:46.132 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.119+0000 7fad17fff700 1 -- 172.21.15.47:0/859839984 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fad1801a040 con 0x7fad28060ff0 2022-06-29T15:50:46.133 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.119+0000 7fad2d491700 1 -- 172.21.15.47:0/859839984 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad281514e0 con 0x7fad28060ff0 2022-06-29T15:50:46.133 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.119+0000 7fad17fff700 1 --2- 172.21.15.47:0/859839984 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fad04053aa0 0x7fad04055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:46.133 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.119+0000 7fad17fff700 1 -- 172.21.15.47:0/859839984 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fad1806cea0 con 0x7fad28060ff0 2022-06-29T15:50:46.133 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.119+0000 7fad267fc700 1 --2- 172.21.15.47:0/859839984 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fad04053aa0 0x7fad04055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:46.133 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.119+0000 7fad267fc700 1 --2- 172.21.15.47:0/859839984 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fad04053aa0 0x7fad04055f60 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7fad1c00d000 tx=0x7fad1c009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:46.134 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.127+0000 7fad17fff700 1 -- 172.21.15.47:0/859839984 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fad18043c60 con 0x7fad28060ff0 2022-06-29T15:50:46.264 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.259+0000 7fad2d491700 1 -- 172.21.15.47:0/859839984 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fad280573d0 con 0x7fad28060ff0 2022-06-29T15:50:46.265 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.259+0000 7fad17fff700 1 -- 172.21.15.47:0/859839984 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+16285 (secure 0 0 0) 0x7fad18023020 con 0x7fad28060ff0 2022-06-29T15:50:46.266 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:50:46.266 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":121,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:50:39.462097+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:50:37.087143+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2022-06-29T15:50:38.125088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:50:46.268 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.263+0000 7fad2d491700 1 -- 172.21.15.47:0/859839984 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fad04053aa0 msgr2=0x7fad04055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:46.268 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.263+0000 7fad2d491700 1 --2- 172.21.15.47:0/859839984 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fad04053aa0 0x7fad04055f60 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7fad1c00d000 tx=0x7fad1c009040).stop 2022-06-29T15:50:46.268 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.263+0000 7fad2d491700 1 -- 172.21.15.47:0/859839984 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad28060ff0 msgr2=0x7fad28150630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:46.269 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.263+0000 7fad2d491700 1 --2- 172.21.15.47:0/859839984 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad28060ff0 0x7fad28150630 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 rx=0x7fad18002f70 tx=0x7fad18015000).stop 2022-06-29T15:50:46.269 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.263+0000 7fad2d491700 1 -- 172.21.15.47:0/859839984 shutdown_connections 2022-06-29T15:50:46.269 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.263+0000 7fad2d491700 1 --2- 172.21.15.47:0/859839984 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fad280b0f30 0x7fad2810f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:46.269 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.263+0000 7fad2d491700 1 --2- 172.21.15.47:0/859839984 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fad04053aa0 0x7fad04055f60 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:46.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.263+0000 7fad2d491700 1 --2- 172.21.15.47:0/859839984 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad280619b0 0x7fad28066da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:46.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.263+0000 7fad2d491700 1 --2- 172.21.15.47:0/859839984 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad28060ff0 0x7fad28150630 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:46.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.263+0000 7fad2d491700 1 -- 172.21.15.47:0/859839984 >> 172.21.15.47:0/859839984 conn(0x7fad2805c510 msgr2=0x7fad2813d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:46.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.263+0000 7fad2d491700 1 -- 172.21.15.47:0/859839984 shutdown_connections 2022-06-29T15:50:46.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.263+0000 7fad2d491700 1 -- 172.21.15.47:0/859839984 wait complete. 2022-06-29T15:50:46.280 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-29T15:42:16.971431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-29T15:42:25.290884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-29T15:50:37.087143+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2022-06-29T15:50:38.125088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-29T15:50:46.281 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-29T15:50:46.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.387+0000 7f60db522700 1 Processor -- start 2022-06-29T15:50:46.394 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.387+0000 7f60db522700 1 -- start start 2022-06-29T15:50:46.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.387+0000 7f60db522700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f60d4066dc0 0x7f60d40671a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:46.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.387+0000 7f60db522700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f60d413d1c0 0x7f60d41445d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:46.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.387+0000 7f60db522700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60d4067690 con 0x7f60d41472f0 2022-06-29T15:50:46.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.387+0000 7f60db522700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60d41434f0 con 0x7f60d4066dc0 2022-06-29T15:50:46.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.387+0000 7f60db522700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60d4143790 con 0x7f60d413d1c0 2022-06-29T15:50:46.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.387+0000 7f60d9abf700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f60d413d1c0 0x7f60d41445d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:46.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.387+0000 7f60d92be700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f60d4066dc0 0x7f60d40671a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:46.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.391+0000 7f60d8abd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f60d41472f0 0x7f60d413cab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:46100/0 (socket says 172.21.15.47:46100) 2022-06-29T15:50:46.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.391+0000 7f60d9abf700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f60d413d1c0 0x7f60d41445d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:50086/0 (socket says 172.21.15.47:50086) 2022-06-29T15:50:46.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.391+0000 7f60d8abd700 1 -- 172.21.15.47:0/2060594395 learned_addr learned my addr 172.21.15.47:0/2060594395 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:46.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.391+0000 7f60d92be700 1 -- 172.21.15.47:0/2060594395 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f60d413d1c0 msgr2=0x7f60d41445d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:46.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.391+0000 7f60d92be700 1 --2- 172.21.15.47:0/2060594395 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f60d413d1c0 0x7f60d41445d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:46.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.391+0000 7f60d92be700 1 -- 172.21.15.47:0/2060594395 >> v1:172.21.15.64:6789/0 conn(0x7f60d41472f0 legacy=0x7f60d413cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:46.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.391+0000 7f60d92be700 1 -- 172.21.15.47:0/2060594395 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60d4143a30 con 0x7f60d4066dc0 2022-06-29T15:50:46.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.391+0000 7f60c7fff700 1 -- 172.21.15.47:0/2060594395 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 919155606 0 0) 0x7f60d4067690 con 0x7f60d41472f0 2022-06-29T15:50:46.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.391+0000 7f60d9abf700 1 --2- 172.21.15.47:0/2060594395 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f60d413d1c0 0x7f60d41445d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:50:46.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.391+0000 7f60d92be700 1 --2- 172.21.15.47:0/2060594395 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60d4066dc0 0x7f60d40671a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7f60c8002f70 tx=0x7f60c8006df0).ready entity=mon.1 client_cookie=a11e44988fdc900a server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:46.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.391+0000 7f60c7fff700 1 -- 172.21.15.47:0/2060594395 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f60c800d570 con 0x7f60d4066dc0 2022-06-29T15:50:46.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.391+0000 7f60c7fff700 1 -- 172.21.15.47:0/2060594395 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60c8000e90 con 0x7f60d4066dc0 2022-06-29T15:50:46.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.391+0000 7f60c7fff700 1 -- 172.21.15.47:0/2060594395 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f60c800d8c0 con 0x7f60d4066dc0 2022-06-29T15:50:46.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.391+0000 7f60db522700 1 -- 172.21.15.47:0/2060594395 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60d4066dc0 msgr2=0x7f60d40671a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:46.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.391+0000 7f60db522700 1 --2- 172.21.15.47:0/2060594395 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60d4066dc0 0x7f60d40671a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7f60c8002f70 tx=0x7f60c8006df0).stop 2022-06-29T15:50:46.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.391+0000 7f60db522700 1 -- 172.21.15.47:0/2060594395 shutdown_connections 2022-06-29T15:50:46.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.391+0000 7f60db522700 1 --2- 172.21.15.47:0/2060594395 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f60d413d1c0 0x7f60d41445d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:46.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.391+0000 7f60db522700 1 --2- 172.21.15.47:0/2060594395 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60d4066dc0 0x7f60d40671a0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:46.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.391+0000 7f60db522700 1 -- 172.21.15.47:0/2060594395 >> 172.21.15.47:0/2060594395 conn(0x7f60d405c510 msgr2=0x7f60d405c920 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:46.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.391+0000 7f60db522700 1 -- 172.21.15.47:0/2060594395 shutdown_connections 2022-06-29T15:50:46.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.391+0000 7f60db522700 1 -- 172.21.15.47:0/2060594395 wait complete. 2022-06-29T15:50:46.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.395+0000 7f60db522700 1 Processor -- start 2022-06-29T15:50:46.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.395+0000 7f60db522700 1 -- start start 2022-06-29T15:50:46.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.395+0000 7f60db522700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f60d4066dc0 0x7f60d40b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:46.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.395+0000 7f60db522700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60d413d1c0 0x7f60d4061200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:46.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.395+0000 7f60db522700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60d41472f0 0x7f60d4060140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:46.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.395+0000 7f60db522700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d40b6840 con 0x7f60d413d1c0 2022-06-29T15:50:46.409 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.395+0000 7f60db522700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d4152e00 con 0x7f60d41472f0 2022-06-29T15:50:46.409 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.395+0000 7f60db522700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60d40b6a70 con 0x7f60d4066dc0 2022-06-29T15:50:46.409 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.395+0000 7f60d8abd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60d413d1c0 0x7f60d4061200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:46.409 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.395+0000 7f60d92be700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f60d4066dc0 0x7f60d40b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:46.409 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.395+0000 7f60d92be700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f60d4066dc0 0x7f60d40b71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:50088/0 (socket says 172.21.15.47:50088) 2022-06-29T15:50:46.410 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.395+0000 7f60d8abd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60d413d1c0 0x7f60d4061200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52990/0 (socket says 172.21.15.47:52990) 2022-06-29T15:50:46.410 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.395+0000 7f60d92be700 1 -- 172.21.15.47:0/935435334 learned_addr learned my addr 172.21.15.47:0/935435334 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:46.410 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.395+0000 7f60d9abf700 1 --2- 172.21.15.47:0/935435334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60d41472f0 0x7f60d4060140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:46.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.395+0000 7f60d8abd700 1 -- 172.21.15.47:0/935435334 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f60d4066dc0 msgr2=0x7f60d40b71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:46.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.395+0000 7f60d8abd700 1 --2- 172.21.15.47:0/935435334 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f60d4066dc0 0x7f60d40b71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:46.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.395+0000 7f60d8abd700 1 -- 172.21.15.47:0/935435334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60d41472f0 msgr2=0x7f60d4060140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:46.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.395+0000 7f60d8abd700 1 --2- 172.21.15.47:0/935435334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60d41472f0 0x7f60d4060140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:46.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.395+0000 7f60d8abd700 1 -- 172.21.15.47:0/935435334 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60cc009470 con 0x7f60d413d1c0 2022-06-29T15:50:46.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.395+0000 7f60d8abd700 1 --2- 172.21.15.47:0/935435334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60d413d1c0 0x7f60d4061200 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 rx=0x7f60d0000ed0 tx=0x7f60d0007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:46.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.395+0000 7f60c67fc700 1 -- 172.21.15.47:0/935435334 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f60d00070d0 con 0x7f60d413d1c0 2022-06-29T15:50:46.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.395+0000 7f60c67fc700 1 -- 172.21.15.47:0/935435334 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60d0007230 con 0x7f60d413d1c0 2022-06-29T15:50:46.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.395+0000 7f60db522700 1 -- 172.21.15.47:0/935435334 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60c800b180 con 0x7f60d413d1c0 2022-06-29T15:50:46.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.395+0000 7f60c67fc700 1 -- 172.21.15.47:0/935435334 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f60d0005610 con 0x7f60d413d1c0 2022-06-29T15:50:46.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.395+0000 7f60db522700 1 -- 172.21.15.47:0/935435334 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60d4060630 con 0x7f60d413d1c0 2022-06-29T15:50:46.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.399+0000 7f60db522700 1 -- 172.21.15.47:0/935435334 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60d40573d0 con 0x7f60d413d1c0 2022-06-29T15:50:46.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.399+0000 7f60c67fc700 1 -- 172.21.15.47:0/935435334 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f60d001a060 con 0x7f60d413d1c0 2022-06-29T15:50:46.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.399+0000 7f60c67fc700 1 --2- 172.21.15.47:0/935435334 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f60bc0539e0 0x7f60bc055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:46.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.399+0000 7f60c67fc700 1 -- 172.21.15.47:0/935435334 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f60d006b8b0 con 0x7f60d413d1c0 2022-06-29T15:50:46.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.399+0000 7f60d92be700 1 --2- 172.21.15.47:0/935435334 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f60bc0539e0 0x7f60bc055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:46.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.403+0000 7f60d92be700 1 --2- 172.21.15.47:0/935435334 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f60bc0539e0 0x7f60bc055ea0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7f60c801a000 tx=0x7f60c8014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:46.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.407+0000 7f60c67fc700 1 -- 172.21.15.47:0/935435334 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f60d0042560 con 0x7f60d413d1c0 2022-06-29T15:50:46.545 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.539+0000 7f60db522700 1 -- 172.21.15.47:0/935435334 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f60d40b6f10 con 0x7f60d413d1c0 2022-06-29T15:50:46.546 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.539+0000 7f60c67fc700 1 -- 172.21.15.47:0/935435334 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v121) v1 ==== 110+0+10 (secure 0 0 0) 0x7f60d0018020 con 0x7f60d413d1c0 2022-06-29T15:50:46.546 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2022-06-29T15:50:46.547 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.543+0000 7f60db522700 1 -- 172.21.15.47:0/935435334 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f60bc0539e0 msgr2=0x7f60bc055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:46.547 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.543+0000 7f60db522700 1 --2- 172.21.15.47:0/935435334 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f60bc0539e0 0x7f60bc055ea0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7f60c801a000 tx=0x7f60c8014000).stop 2022-06-29T15:50:46.548 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.543+0000 7f60db522700 1 -- 172.21.15.47:0/935435334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60d413d1c0 msgr2=0x7f60d4061200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:46.548 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.543+0000 7f60db522700 1 --2- 172.21.15.47:0/935435334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60d413d1c0 0x7f60d4061200 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 rx=0x7f60d0000ed0 tx=0x7f60d0007c20).stop 2022-06-29T15:50:46.550 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.543+0000 7f60db522700 1 -- 172.21.15.47:0/935435334 shutdown_connections 2022-06-29T15:50:46.550 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.543+0000 7f60db522700 1 --2- 172.21.15.47:0/935435334 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f60bc0539e0 0x7f60bc055ea0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:46.550 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.543+0000 7f60db522700 1 --2- 172.21.15.47:0/935435334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60d41472f0 0x7f60d4060140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:46.551 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.543+0000 7f60db522700 1 --2- 172.21.15.47:0/935435334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60d413d1c0 0x7f60d4061200 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:46.551 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.543+0000 7f60db522700 1 --2- 172.21.15.47:0/935435334 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f60d4066dc0 0x7f60d40b71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:46.551 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.543+0000 7f60db522700 1 -- 172.21.15.47:0/935435334 >> 172.21.15.47:0/935435334 conn(0x7f60d405c510 msgr2=0x7f60d413c880 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:46.551 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.543+0000 7f60db522700 1 -- 172.21.15.47:0/935435334 shutdown_connections 2022-06-29T15:50:46.552 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.543+0000 7f60db522700 1 -- 172.21.15.47:0/935435334 wait complete. 2022-06-29T15:50:46.560 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-29T15:50:46.679 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.671+0000 7fa24e384700 1 Processor -- start 2022-06-29T15:50:46.679 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.671+0000 7fa24e384700 1 -- start start 2022-06-29T15:50:46.681 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.675+0000 7fa24e384700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa2480b6d30 0x7fa24813d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:46.682 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.675+0000 7fa24e384700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa24813d790 0x7fa2481407d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:46.682 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.675+0000 7fa24e384700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa248143f70 con 0x7fa248062f30 2022-06-29T15:50:46.682 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.675+0000 7fa24e384700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2480b1380 con 0x7fa24813d790 2022-06-29T15:50:46.683 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.675+0000 7fa24e384700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2480a9cf0 con 0x7fa2480b6d30 2022-06-29T15:50:46.683 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.675+0000 7fa24db83700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa24813d790 0x7fa2481407d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:46.683 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.675+0000 7fa24cb81700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa248062f30 0x7fa2481400c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38552/0 (socket says 172.21.15.47:38552) 2022-06-29T15:50:46.683 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.675+0000 7fa24cb81700 1 -- 172.21.15.47:0/1304494074 learned_addr learned my addr 172.21.15.47:0/1304494074 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:46.684 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.675+0000 7fa24d382700 1 --2- 172.21.15.47:0/1304494074 >> v2:172.21.15.64:3300/0 conn(0x7fa2480b6d30 0x7fa24813d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:46.684 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.675+0000 7fa24db83700 1 -- 172.21.15.47:0/1304494074 >> v2:172.21.15.64:3300/0 conn(0x7fa2480b6d30 msgr2=0x7fa24813d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:46.684 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.675+0000 7fa24db83700 1 --2- 172.21.15.47:0/1304494074 >> v2:172.21.15.64:3300/0 conn(0x7fa2480b6d30 0x7fa24813d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:46.685 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.675+0000 7fa24db83700 1 -- 172.21.15.47:0/1304494074 >> v1:172.21.15.47:6789/0 conn(0x7fa248062f30 legacy=0x7fa2481400c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:46.685 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.675+0000 7fa233fff700 1 -- 172.21.15.47:0/1304494074 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3945951185 0 0) 0x7fa248143f70 con 0x7fa248062f30 2022-06-29T15:50:46.685 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.675+0000 7fa24db83700 1 -- 172.21.15.47:0/1304494074 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2481441a0 con 0x7fa24813d790 2022-06-29T15:50:46.685 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.675+0000 7fa24db83700 1 --2- 172.21.15.47:0/1304494074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa24813d790 0x7fa2481407d0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 rx=0x7fa2400032f0 tx=0x7fa2400091d0).ready entity=mon.0 client_cookie=73926e2fd6b828ab server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:46.686 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.675+0000 7fa233fff700 1 -- 172.21.15.47:0/1304494074 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa240009ea0 con 0x7fa24813d790 2022-06-29T15:50:46.687 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.675+0000 7fa233fff700 1 -- 172.21.15.47:0/1304494074 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa24000b870 con 0x7fa24813d790 2022-06-29T15:50:46.687 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.675+0000 7fa233fff700 1 -- 172.21.15.47:0/1304494074 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa24000b9e0 con 0x7fa24813d790 2022-06-29T15:50:46.690 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.675+0000 7fa24e384700 1 -- 172.21.15.47:0/1304494074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa24813d790 msgr2=0x7fa2481407d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:46.691 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.675+0000 7fa24e384700 1 --2- 172.21.15.47:0/1304494074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa24813d790 0x7fa2481407d0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 rx=0x7fa2400032f0 tx=0x7fa2400091d0).stop 2022-06-29T15:50:46.691 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.675+0000 7fa24e384700 1 -- 172.21.15.47:0/1304494074 shutdown_connections 2022-06-29T15:50:46.691 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.675+0000 7fa24e384700 1 --2- 172.21.15.47:0/1304494074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa24813d790 0x7fa2481407d0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:46.691 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.675+0000 7fa24e384700 1 --2- 172.21.15.47:0/1304494074 >> v2:172.21.15.64:3300/0 conn(0x7fa2480b6d30 0x7fa24813d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:46.692 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.675+0000 7fa24e384700 1 -- 172.21.15.47:0/1304494074 >> 172.21.15.47:0/1304494074 conn(0x7fa248063670 msgr2=0x7fa248063a50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:46.692 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.675+0000 7fa24e384700 1 -- 172.21.15.47:0/1304494074 shutdown_connections 2022-06-29T15:50:46.692 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.675+0000 7fa24e384700 1 -- 172.21.15.47:0/1304494074 wait complete. 2022-06-29T15:50:46.692 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.679+0000 7fa24e384700 1 Processor -- start 2022-06-29T15:50:46.693 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.679+0000 7fa24e384700 1 -- start start 2022-06-29T15:50:46.693 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.679+0000 7fa24e384700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa248062f30 0x7fa248141050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:46.693 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.679+0000 7fa24e384700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2480b6d30 0x7fa2480acc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:46.693 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.679+0000 7fa24e384700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa24813d790 0x7fa248138fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:46.694 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.679+0000 7fa24e384700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2480b6110 con 0x7fa248062f30 2022-06-29T15:50:46.694 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.679+0000 7fa24e384700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2481435f0 con 0x7fa2480b6d30 2022-06-29T15:50:46.694 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.679+0000 7fa24e384700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2480aa3e0 con 0x7fa24813d790 2022-06-29T15:50:46.694 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.679+0000 7fa24d382700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa248062f30 0x7fa248141050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:46.695 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.679+0000 7fa24d382700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa248062f30 0x7fa248141050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53000/0 (socket says 172.21.15.47:53000) 2022-06-29T15:50:46.695 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.679+0000 7fa24d382700 1 -- 172.21.15.47:0/613823020 learned_addr learned my addr 172.21.15.47:0/613823020 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:46.695 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.679+0000 7fa24cb81700 1 --2- 172.21.15.47:0/613823020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2480b6d30 0x7fa2480acc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:46.695 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.679+0000 7fa24db83700 1 --2- 172.21.15.47:0/613823020 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa24813d790 0x7fa248138fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:46.695 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.683+0000 7fa24d382700 1 -- 172.21.15.47:0/613823020 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa24813d790 msgr2=0x7fa248138fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:46.696 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.683+0000 7fa24d382700 1 --2- 172.21.15.47:0/613823020 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa24813d790 0x7fa248138fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:46.696 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.683+0000 7fa24d382700 1 -- 172.21.15.47:0/613823020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2480b6d30 msgr2=0x7fa2480acc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:46.696 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.683+0000 7fa24d382700 1 --2- 172.21.15.47:0/613823020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2480b6d30 0x7fa2480acc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:46.696 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.683+0000 7fa24d382700 1 -- 172.21.15.47:0/613823020 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa240009470 con 0x7fa248062f30 2022-06-29T15:50:46.697 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.683+0000 7fa24db83700 1 --2- 172.21.15.47:0/613823020 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa24813d790 0x7fa248138fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:50:46.697 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.683+0000 7fa24d382700 1 --2- 172.21.15.47:0/613823020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa248062f30 0x7fa248141050 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 rx=0x7fa2440030b0 tx=0x7fa244008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:46.697 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.683+0000 7fa23affd700 1 -- 172.21.15.47:0/613823020 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa244011780 con 0x7fa248062f30 2022-06-29T15:50:46.697 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.683+0000 7fa23affd700 1 -- 172.21.15.47:0/613823020 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2440118e0 con 0x7fa248062f30 2022-06-29T15:50:46.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.683+0000 7fa24e384700 1 -- 172.21.15.47:0/613823020 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2480aa680 con 0x7fa248062f30 2022-06-29T15:50:46.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.683+0000 7fa23affd700 1 -- 172.21.15.47:0/613823020 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa244011c40 con 0x7fa248062f30 2022-06-29T15:50:46.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.683+0000 7fa24e384700 1 -- 172.21.15.47:0/613823020 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa24813acb0 con 0x7fa248062f30 2022-06-29T15:50:46.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.683+0000 7fa24e384700 1 -- 172.21.15.47:0/613823020 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2480573d0 con 0x7fa248062f30 2022-06-29T15:50:46.699 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.683+0000 7fa23affd700 1 -- 172.21.15.47:0/613823020 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa24401c460 con 0x7fa248062f30 2022-06-29T15:50:46.700 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.683+0000 7fa23affd700 1 --2- 172.21.15.47:0/613823020 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa224053950 0x7fa224055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:46.700 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.687+0000 7fa24cb81700 1 --2- 172.21.15.47:0/613823020 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa224053950 0x7fa224055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:46.700 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.687+0000 7fa23affd700 1 -- 172.21.15.47:0/613823020 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa24406c320 con 0x7fa248062f30 2022-06-29T15:50:46.700 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.687+0000 7fa24cb81700 1 --2- 172.21.15.47:0/613823020 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa224053950 0x7fa224055e10 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7fa23c00d000 tx=0x7fa23c007a00).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:46.701 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.695+0000 7fa23affd700 1 -- 172.21.15.47:0/613823020 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa244046e60 con 0x7fa248062f30 2022-06-29T15:50:46.825 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.819+0000 7fa24e384700 1 -- 172.21.15.47:0/613823020 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa248069e20 con 0x7fa248062f30 2022-06-29T15:50:46.826 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.819+0000 7fa23affd700 1 -- 172.21.15.47:0/613823020 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v121) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa244042f70 con 0x7fa248062f30 2022-06-29T15:50:46.826 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:50:46.829 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.823+0000 7fa24e384700 1 -- 172.21.15.47:0/613823020 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa224053950 msgr2=0x7fa224055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:46.829 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.823+0000 7fa24e384700 1 --2- 172.21.15.47:0/613823020 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa224053950 0x7fa224055e10 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7fa23c00d000 tx=0x7fa23c007a00).stop 2022-06-29T15:50:46.829 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.823+0000 7fa24e384700 1 -- 172.21.15.47:0/613823020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa248062f30 msgr2=0x7fa248141050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:46.830 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.823+0000 7fa24e384700 1 --2- 172.21.15.47:0/613823020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa248062f30 0x7fa248141050 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 rx=0x7fa2440030b0 tx=0x7fa244008040).stop 2022-06-29T15:50:46.830 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.823+0000 7fa24e384700 1 -- 172.21.15.47:0/613823020 shutdown_connections 2022-06-29T15:50:46.830 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.823+0000 7fa24e384700 1 --2- 172.21.15.47:0/613823020 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa24813d790 0x7fa248138fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:46.830 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.823+0000 7fa24e384700 1 --2- 172.21.15.47:0/613823020 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa224053950 0x7fa224055e10 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:46.831 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.823+0000 7fa24e384700 1 --2- 172.21.15.47:0/613823020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2480b6d30 0x7fa2480acc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:46.831 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.823+0000 7fa24e384700 1 --2- 172.21.15.47:0/613823020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa248062f30 0x7fa248141050 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:46.831 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.823+0000 7fa24e384700 1 -- 172.21.15.47:0/613823020 >> 172.21.15.47:0/613823020 conn(0x7fa248063670 msgr2=0x7fa2480b32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:46.831 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.823+0000 7fa24e384700 1 -- 172.21.15.47:0/613823020 shutdown_connections 2022-06-29T15:50:46.832 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.823+0000 7fa24e384700 1 -- 172.21.15.47:0/613823020 wait complete. 2022-06-29T15:50:46.844 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-29T15:50:46.954 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.947+0000 7fc58d440700 1 Processor -- start 2022-06-29T15:50:46.954 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.947+0000 7fc58d440700 1 -- start start 2022-06-29T15:50:46.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.947+0000 7fc58d440700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc5880b5c10 0x7fc58813d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:46.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.947+0000 7fc58d440700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc5880a9bb0 0x7fc58813d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:46.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.947+0000 7fc58d440700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5880b04a0 con 0x7fc5880abcd0 2022-06-29T15:50:46.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.947+0000 7fc58d440700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5880b5ff0 con 0x7fc5880b5c10 2022-06-29T15:50:46.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.947+0000 7fc58d440700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5880a9360 con 0x7fc5880a9bb0 2022-06-29T15:50:46.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.951+0000 7fc587fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc5880b5c10 0x7fc58813d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:46.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.951+0000 7fc58cc3f700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc5880a9bb0 0x7fc58813d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:46.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.951+0000 7fc57ffff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc5880abcd0 0x7fc5881400f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38566/0 (socket says 172.21.15.47:38566) 2022-06-29T15:50:46.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.951+0000 7fc57ffff700 1 -- 172.21.15.47:0/3905792343 learned_addr learned my addr 172.21.15.47:0/3905792343 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:46.959 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.951+0000 7fc587fff700 1 -- 172.21.15.47:0/3905792343 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc5880a9bb0 msgr2=0x7fc58813d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:46.959 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.951+0000 7fc587fff700 1 --2- 172.21.15.47:0/3905792343 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc5880a9bb0 0x7fc58813d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:46.959 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.951+0000 7fc587fff700 1 -- 172.21.15.47:0/3905792343 >> v1:172.21.15.47:6789/0 conn(0x7fc5880abcd0 legacy=0x7fc5881400f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:46.960 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.951+0000 7fc5877fe700 1 -- 172.21.15.47:0/3905792343 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 759291661 0 0) 0x7fc5880b04a0 con 0x7fc5880abcd0 2022-06-29T15:50:46.960 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.951+0000 7fc587fff700 1 -- 172.21.15.47:0/3905792343 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5880a9600 con 0x7fc5880b5c10 2022-06-29T15:50:46.960 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.951+0000 7fc587fff700 1 --2- 172.21.15.47:0/3905792343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5880b5c10 0x7fc58813d1a0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 rx=0x7fc578002f70 tx=0x7fc578006df0).ready entity=mon.0 client_cookie=fc14ef3c147cf1b3 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:46.960 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.951+0000 7fc5877fe700 1 -- 172.21.15.47:0/3905792343 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc57800d570 con 0x7fc5880b5c10 2022-06-29T15:50:46.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.951+0000 7fc5877fe700 1 -- 172.21.15.47:0/3905792343 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc578000e90 con 0x7fc5880b5c10 2022-06-29T15:50:46.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.951+0000 7fc5877fe700 1 -- 172.21.15.47:0/3905792343 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc57800d8c0 con 0x7fc5880b5c10 2022-06-29T15:50:46.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.951+0000 7fc58d440700 1 -- 172.21.15.47:0/3905792343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5880b5c10 msgr2=0x7fc58813d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:46.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.951+0000 7fc58d440700 1 --2- 172.21.15.47:0/3905792343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5880b5c10 0x7fc58813d1a0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 rx=0x7fc578002f70 tx=0x7fc578006df0).stop 2022-06-29T15:50:46.962 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.951+0000 7fc58d440700 1 -- 172.21.15.47:0/3905792343 shutdown_connections 2022-06-29T15:50:46.962 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.951+0000 7fc58d440700 1 --2- 172.21.15.47:0/3905792343 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc5880a9bb0 0x7fc58813d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:46.962 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.951+0000 7fc58d440700 1 --2- 172.21.15.47:0/3905792343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5880b5c10 0x7fc58813d1a0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:46.962 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.951+0000 7fc58d440700 1 -- 172.21.15.47:0/3905792343 >> 172.21.15.47:0/3905792343 conn(0x7fc588062560 msgr2=0x7fc588062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:46.963 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.951+0000 7fc58d440700 1 -- 172.21.15.47:0/3905792343 shutdown_connections 2022-06-29T15:50:46.963 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.951+0000 7fc58d440700 1 -- 172.21.15.47:0/3905792343 wait complete. 2022-06-29T15:50:46.966 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.955+0000 7fc58d440700 1 Processor -- start 2022-06-29T15:50:46.966 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.955+0000 7fc58d440700 1 -- start start 2022-06-29T15:50:46.966 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.955+0000 7fc58d440700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5880a9bb0 0x7fc58813bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:46.966 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.955+0000 7fc58d440700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5880abcd0 0x7fc588140fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:46.967 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.955+0000 7fc58d440700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc5880a8ce0 0x7fc5881414b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:46.967 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.955+0000 7fc58d440700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc588145690 con 0x7fc5880abcd0 2022-06-29T15:50:46.967 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.955+0000 7fc58d440700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc588145da0 con 0x7fc5880a9bb0 2022-06-29T15:50:46.967 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.955+0000 7fc58d440700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5880ae940 con 0x7fc5880a8ce0 2022-06-29T15:50:46.968 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.955+0000 7fc57ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5880abcd0 0x7fc588140fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:46.968 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.955+0000 7fc57ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5880abcd0 0x7fc588140fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53018/0 (socket says 172.21.15.47:53018) 2022-06-29T15:50:46.968 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.955+0000 7fc57ffff700 1 -- 172.21.15.47:0/1009685557 learned_addr learned my addr 172.21.15.47:0/1009685557 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:46.968 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.955+0000 7fc58cc3f700 1 --2- 172.21.15.47:0/1009685557 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc5880a8ce0 0x7fc5881414b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:46.968 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.955+0000 7fc587fff700 1 --2- 172.21.15.47:0/1009685557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5880a9bb0 0x7fc58813bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:46.969 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.955+0000 7fc57ffff700 1 -- 172.21.15.47:0/1009685557 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc5880a8ce0 msgr2=0x7fc5881414b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:46.969 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.955+0000 7fc57ffff700 1 --2- 172.21.15.47:0/1009685557 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc5880a8ce0 0x7fc5881414b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:46.969 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.955+0000 7fc57ffff700 1 -- 172.21.15.47:0/1009685557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5880a9bb0 msgr2=0x7fc58813bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:46.969 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.955+0000 7fc57ffff700 1 --2- 172.21.15.47:0/1009685557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5880a9bb0 0x7fc58813bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:46.970 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.955+0000 7fc57ffff700 1 -- 172.21.15.47:0/1009685557 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc57800b180 con 0x7fc5880abcd0 2022-06-29T15:50:46.970 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.955+0000 7fc58cc3f700 1 --2- 172.21.15.47:0/1009685557 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc5880a8ce0 0x7fc5881414b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:50:46.970 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.955+0000 7fc587fff700 1 --2- 172.21.15.47:0/1009685557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5880a9bb0 0x7fc58813bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:50:46.970 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.955+0000 7fc57ffff700 1 --2- 172.21.15.47:0/1009685557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5880abcd0 0x7fc588140fc0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 rx=0x7fc570000ed0 tx=0x7fc570007ce0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:46.971 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.955+0000 7fc585ffb700 1 -- 172.21.15.47:0/1009685557 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc57000ce20 con 0x7fc5880abcd0 2022-06-29T15:50:46.971 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.955+0000 7fc585ffb700 1 -- 172.21.15.47:0/1009685557 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5700072e0 con 0x7fc5880abcd0 2022-06-29T15:50:46.971 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.955+0000 7fc58d440700 1 -- 172.21.15.47:0/1009685557 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5881440a0 con 0x7fc5880abcd0 2022-06-29T15:50:46.971 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.955+0000 7fc585ffb700 1 -- 172.21.15.47:0/1009685557 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc570005610 con 0x7fc5880abcd0 2022-06-29T15:50:46.972 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.955+0000 7fc58d440700 1 -- 172.21.15.47:0/1009685557 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc588146240 con 0x7fc5880abcd0 2022-06-29T15:50:46.972 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.959+0000 7fc585ffb700 1 -- 172.21.15.47:0/1009685557 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc57001a060 con 0x7fc5880abcd0 2022-06-29T15:50:46.972 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.959+0000 7fc585ffb700 1 --2- 172.21.15.47:0/1009685557 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc56c053aa0 0x7fc56c055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:46.972 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.959+0000 7fc58d440700 1 -- 172.21.15.47:0/1009685557 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc588145800 con 0x7fc5880abcd0 2022-06-29T15:50:46.973 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.959+0000 7fc585ffb700 1 -- 172.21.15.47:0/1009685557 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc57006b930 con 0x7fc5880abcd0 2022-06-29T15:50:46.973 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.959+0000 7fc587fff700 1 --2- 172.21.15.47:0/1009685557 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc56c053aa0 0x7fc56c055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:46.973 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.959+0000 7fc587fff700 1 --2- 172.21.15.47:0/1009685557 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc56c053aa0 0x7fc56c055f60 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7fc578015000 tx=0x7fc578014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:46.974 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:46.967+0000 7fc585ffb700 1 -- 172.21.15.47:0/1009685557 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc588145800 con 0x7fc5880abcd0 2022-06-29T15:50:47.097 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.091+0000 7fc58d440700 1 -- 172.21.15.47:0/1009685557 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc5881457d0 con 0x7fc5880abcd0 2022-06-29T15:50:47.098 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.091+0000 7fc585ffb700 1 -- 172.21.15.47:0/1009685557 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v121) v1 ==== 104+0+10 (secure 0 0 0) 0x7fc570018020 con 0x7fc5880abcd0 2022-06-29T15:50:47.098 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:50:47.099 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.095+0000 7fc58d440700 1 -- 172.21.15.47:0/1009685557 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc56c053aa0 msgr2=0x7fc56c055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:47.100 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.095+0000 7fc58d440700 1 --2- 172.21.15.47:0/1009685557 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc56c053aa0 0x7fc56c055f60 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7fc578015000 tx=0x7fc578014000).stop 2022-06-29T15:50:47.100 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.095+0000 7fc58d440700 1 -- 172.21.15.47:0/1009685557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5880abcd0 msgr2=0x7fc588140fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:47.100 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.095+0000 7fc58d440700 1 --2- 172.21.15.47:0/1009685557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5880abcd0 0x7fc588140fc0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 rx=0x7fc570000ed0 tx=0x7fc570007ce0).stop 2022-06-29T15:50:47.101 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.095+0000 7fc58d440700 1 -- 172.21.15.47:0/1009685557 shutdown_connections 2022-06-29T15:50:47.102 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.095+0000 7fc58d440700 1 --2- 172.21.15.47:0/1009685557 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc5880a8ce0 0x7fc5881414b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:47.102 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.095+0000 7fc58d440700 1 --2- 172.21.15.47:0/1009685557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5880abcd0 0x7fc588140fc0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:47.102 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.095+0000 7fc58d440700 1 --2- 172.21.15.47:0/1009685557 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc56c053aa0 0x7fc56c055f60 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:47.102 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.095+0000 7fc58d440700 1 --2- 172.21.15.47:0/1009685557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5880a9bb0 0x7fc58813bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:47.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.095+0000 7fc58d440700 1 -- 172.21.15.47:0/1009685557 >> 172.21.15.47:0/1009685557 conn(0x7fc588062560 msgr2=0x7fc58813caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:47.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.095+0000 7fc58d440700 1 -- 172.21.15.47:0/1009685557 shutdown_connections 2022-06-29T15:50:47.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.095+0000 7fc58d440700 1 -- 172.21.15.47:0/1009685557 wait complete. 2022-06-29T15:50:47.111 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-29T15:50:47.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.211+0000 7f3346754700 1 Processor -- start 2022-06-29T15:50:47.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.211+0000 7f3346754700 1 -- start start 2022-06-29T15:50:47.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.211+0000 7f3346754700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3340060a70 0x7f334014b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:47.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.211+0000 7f3346754700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3340144e60 0x7f3340147930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:47.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.211+0000 7f3346754700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3340060e50 con 0x7f33400a72b0 2022-06-29T15:50:47.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.211+0000 7f3346754700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33400a8120 con 0x7f3340060a70 2022-06-29T15:50:47.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.211+0000 7f3346754700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3340147540 con 0x7f3340144e60 2022-06-29T15:50:47.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.215+0000 7f333ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3340060a70 0x7f334014b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:47.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.215+0000 7f333f7fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f33400a72b0 0x7f3340144750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:46142/0 (socket says 172.21.15.47:46142) 2022-06-29T15:50:47.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.215+0000 7f3344cf1700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3340144e60 0x7f3340147930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:47.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.215+0000 7f333f7fe700 1 -- 172.21.15.47:0/2237436770 learned_addr learned my addr 172.21.15.47:0/2237436770 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:47.225 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.215+0000 7f333ffff700 1 -- 172.21.15.47:0/2237436770 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3340144e60 msgr2=0x7f3340147930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:47.226 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.215+0000 7f333ffff700 1 --2- 172.21.15.47:0/2237436770 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3340144e60 0x7f3340147930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:47.226 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.215+0000 7f333ffff700 1 -- 172.21.15.47:0/2237436770 >> v1:172.21.15.64:6789/0 conn(0x7f33400a72b0 legacy=0x7f3340144750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:47.226 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.215+0000 7f333effd700 1 -- 172.21.15.47:0/2237436770 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2220490435 0 0) 0x7f3340060e50 con 0x7f33400a72b0 2022-06-29T15:50:47.226 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.215+0000 7f333ffff700 1 -- 172.21.15.47:0/2237436770 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33400a8ac0 con 0x7f3340060a70 2022-06-29T15:50:47.227 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.215+0000 7f333ffff700 1 --2- 172.21.15.47:0/2237436770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3340060a70 0x7f334014b4e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7f3334002f70 tx=0x7f3334006df0).ready entity=mon.1 client_cookie=86a607a5e1ea3bff server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:47.227 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.215+0000 7f333effd700 1 -- 172.21.15.47:0/2237436770 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f333400d570 con 0x7f3340060a70 2022-06-29T15:50:47.227 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.215+0000 7f333effd700 1 -- 172.21.15.47:0/2237436770 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3334000e90 con 0x7f3340060a70 2022-06-29T15:50:47.227 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.215+0000 7f333effd700 1 -- 172.21.15.47:0/2237436770 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f333400d8c0 con 0x7f3340060a70 2022-06-29T15:50:47.228 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.215+0000 7f3346754700 1 -- 172.21.15.47:0/2237436770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3340060a70 msgr2=0x7f334014b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:47.228 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.215+0000 7f3346754700 1 --2- 172.21.15.47:0/2237436770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3340060a70 0x7f334014b4e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7f3334002f70 tx=0x7f3334006df0).stop 2022-06-29T15:50:47.228 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.215+0000 7f3346754700 1 -- 172.21.15.47:0/2237436770 shutdown_connections 2022-06-29T15:50:47.228 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.215+0000 7f3346754700 1 --2- 172.21.15.47:0/2237436770 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3340144e60 0x7f3340147930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:47.228 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.215+0000 7f3346754700 1 --2- 172.21.15.47:0/2237436770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3340060a70 0x7f334014b4e0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:47.229 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.215+0000 7f3346754700 1 -- 172.21.15.47:0/2237436770 >> 172.21.15.47:0/2237436770 conn(0x7f33400626f0 msgr2=0x7f3340062ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:47.229 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.215+0000 7f3346754700 1 -- 172.21.15.47:0/2237436770 shutdown_connections 2022-06-29T15:50:47.229 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.215+0000 7f3346754700 1 -- 172.21.15.47:0/2237436770 wait complete. 2022-06-29T15:50:47.229 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.215+0000 7f3346754700 1 Processor -- start 2022-06-29T15:50:47.230 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.215+0000 7f3346754700 1 -- start start 2022-06-29T15:50:47.230 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.219+0000 7f3346754700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3340060a70 0x7f3340157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:47.230 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.219+0000 7f3346754700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33400a72b0 0x7f3340065570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:47.230 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.219+0000 7f3346754700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3340144e60 0x7f3340063ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:47.231 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.219+0000 7f3346754700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33401575e0 con 0x7f3340060a70 2022-06-29T15:50:47.231 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.219+0000 7f3346754700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33400a9420 con 0x7f33400a72b0 2022-06-29T15:50:47.231 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.219+0000 7f3346754700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3340063490 con 0x7f3340144e60 2022-06-29T15:50:47.231 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.219+0000 7f333ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3340060a70 0x7f3340157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:47.232 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.219+0000 7f333ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3340060a70 0x7f3340157050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53028/0 (socket says 172.21.15.47:53028) 2022-06-29T15:50:47.232 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.219+0000 7f333ffff700 1 -- 172.21.15.47:0/2620034190 learned_addr learned my addr 172.21.15.47:0/2620034190 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:47.232 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.219+0000 7f333f7fe700 1 --2- 172.21.15.47:0/2620034190 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33400a72b0 0x7f3340065570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:47.232 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.219+0000 7f3344cf1700 1 --2- 172.21.15.47:0/2620034190 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3340144e60 0x7f3340063ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:47.233 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.219+0000 7f333ffff700 1 -- 172.21.15.47:0/2620034190 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3340144e60 msgr2=0x7f3340063ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:47.233 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.219+0000 7f333ffff700 1 --2- 172.21.15.47:0/2620034190 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3340144e60 0x7f3340063ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:47.233 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.219+0000 7f333ffff700 1 -- 172.21.15.47:0/2620034190 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33400a72b0 msgr2=0x7f3340065570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:47.233 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.219+0000 7f333ffff700 1 --2- 172.21.15.47:0/2620034190 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33400a72b0 0x7f3340065570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:47.233 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.219+0000 7f333ffff700 1 -- 172.21.15.47:0/2620034190 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f333400b180 con 0x7f3340060a70 2022-06-29T15:50:47.234 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.219+0000 7f333ffff700 1 --2- 172.21.15.47:0/2620034190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3340060a70 0x7f3340157050 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 rx=0x7f3334002f70 tx=0x7f3334014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:47.234 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.219+0000 7f333d7fa700 1 -- 172.21.15.47:0/2620034190 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f333400dea0 con 0x7f3340060a70 2022-06-29T15:50:47.234 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.219+0000 7f333d7fa700 1 -- 172.21.15.47:0/2620034190 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3334013de0 con 0x7f3340060a70 2022-06-29T15:50:47.235 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.219+0000 7f3346754700 1 -- 172.21.15.47:0/2620034190 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3340063730 con 0x7f3340060a70 2022-06-29T15:50:47.235 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.219+0000 7f333d7fa700 1 -- 172.21.15.47:0/2620034190 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f333401e5f0 con 0x7f3340060a70 2022-06-29T15:50:47.236 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.219+0000 7f3346754700 1 -- 172.21.15.47:0/2620034190 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33400a9850 con 0x7f3340060a70 2022-06-29T15:50:47.238 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.219+0000 7f3346754700 1 -- 172.21.15.47:0/2620034190 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3340064330 con 0x7f3340060a70 2022-06-29T15:50:47.239 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.219+0000 7f333d7fa700 1 -- 172.21.15.47:0/2620034190 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3334025030 con 0x7f3340060a70 2022-06-29T15:50:47.239 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.223+0000 7f333d7fa700 1 --2- 172.21.15.47:0/2620034190 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3320053a70 0x7f3320055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:47.240 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.223+0000 7f333d7fa700 1 -- 172.21.15.47:0/2620034190 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f333406cf50 con 0x7f3340060a70 2022-06-29T15:50:47.240 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.223+0000 7f333f7fe700 1 --2- 172.21.15.47:0/2620034190 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3320053a70 0x7f3320055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:47.241 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.223+0000 7f333f7fe700 1 --2- 172.21.15.47:0/2620034190 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3320053a70 0x7f3320055f30 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7f333000d000 tx=0x7f3330007a00).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:47.241 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.227+0000 7f333d7fa700 1 -- 172.21.15.47:0/2620034190 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3334043b80 con 0x7f3340060a70 2022-06-29T15:50:47.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.355+0000 7f3346754700 1 -- 172.21.15.47:0/2620034190 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3340064b80 con 0x7f3340060a70 2022-06-29T15:50:47.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.355+0000 7f333d7fa700 1 -- 172.21.15.47:0/2620034190 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v121) v1 ==== 100+0+10 (secure 0 0 0) 0x7f3340064b80 con 0x7f3340060a70 2022-06-29T15:50:47.362 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:50:47.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.359+0000 7f3346754700 1 -- 172.21.15.47:0/2620034190 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3320053a70 msgr2=0x7f3320055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:47.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.359+0000 7f3346754700 1 --2- 172.21.15.47:0/2620034190 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3320053a70 0x7f3320055f30 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7f333000d000 tx=0x7f3330007a00).stop 2022-06-29T15:50:47.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.359+0000 7f3346754700 1 -- 172.21.15.47:0/2620034190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3340060a70 msgr2=0x7f3340157050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:47.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.359+0000 7f3346754700 1 --2- 172.21.15.47:0/2620034190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3340060a70 0x7f3340157050 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 rx=0x7f3334002f70 tx=0x7f3334014000).stop 2022-06-29T15:50:47.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.359+0000 7f3346754700 1 -- 172.21.15.47:0/2620034190 shutdown_connections 2022-06-29T15:50:47.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.359+0000 7f3346754700 1 --2- 172.21.15.47:0/2620034190 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3340144e60 0x7f3340063ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:47.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.359+0000 7f3346754700 1 --2- 172.21.15.47:0/2620034190 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3320053a70 0x7f3320055f30 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:47.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.359+0000 7f3346754700 1 --2- 172.21.15.47:0/2620034190 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33400a72b0 0x7f3340065570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:47.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.359+0000 7f3346754700 1 --2- 172.21.15.47:0/2620034190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3340060a70 0x7f3340157050 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:47.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.359+0000 7f3346754700 1 -- 172.21.15.47:0/2620034190 >> 172.21.15.47:0/2620034190 conn(0x7f33400626f0 msgr2=0x7f3340062ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:47.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.359+0000 7f3346754700 1 -- 172.21.15.47:0/2620034190 shutdown_connections 2022-06-29T15:50:47.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.359+0000 7f3346754700 1 -- 172.21.15.47:0/2620034190 wait complete. 2022-06-29T15:50:47.382 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-29T15:50:47.382 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:50:47.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.479+0000 7fa39733f700 1 Processor -- start 2022-06-29T15:50:47.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.479+0000 7fa39733f700 1 -- start start 2022-06-29T15:50:47.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.483+0000 7fa39733f700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa3900b5c60 0x7fa390144980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:47.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.483+0000 7fa39733f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa390060eb0 0x7fa390144e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:47.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.483+0000 7fa39733f700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3900a9120 con 0x7fa3900a72f0 2022-06-29T15:50:47.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.483+0000 7fa39733f700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa390145360 con 0x7fa390060eb0 2022-06-29T15:50:47.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.483+0000 7fa39733f700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3900b0b80 con 0x7fa3900b5c60 2022-06-29T15:50:47.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.483+0000 7fa395b3c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa390060eb0 0x7fa390144e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:47.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.483+0000 7fa395b3c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa390060eb0 0x7fa390144e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53036/0 (socket says 172.21.15.47:53036) 2022-06-29T15:50:47.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.483+0000 7fa395b3c700 1 -- 172.21.15.47:0/1137503157 learned_addr learned my addr 172.21.15.47:0/1137503157 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:47.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.483+0000 7fa39633d700 1 --2- 172.21.15.47:0/1137503157 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa3900b5c60 0x7fa390144980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:47.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.483+0000 7fa395b3c700 1 -- 172.21.15.47:0/1137503157 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa3900b5c60 msgr2=0x7fa390144980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:47.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.483+0000 7fa395b3c700 1 --2- 172.21.15.47:0/1137503157 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa3900b5c60 0x7fa390144980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:47.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.483+0000 7fa395b3c700 1 -- 172.21.15.47:0/1137503157 >> v1:172.21.15.64:6789/0 conn(0x7fa3900a72f0 legacy=0x7fa3900b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:47.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.483+0000 7fa395b3c700 1 -- 172.21.15.47:0/1137503157 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3900a89e0 con 0x7fa390060eb0 2022-06-29T15:50:47.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.483+0000 7fa39633d700 1 --2- 172.21.15.47:0/1137503157 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa3900b5c60 0x7fa390144980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:50:47.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.483+0000 7fa395b3c700 1 --2- 172.21.15.47:0/1137503157 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa390060eb0 0x7fa390144e70 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 rx=0x7fa384002f70 tx=0x7fa384006eb0).ready entity=mon.0 client_cookie=ec8f933059d6224f server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:47.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.483+0000 7fa39533b700 1 -- 172.21.15.47:0/1137503157 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa38400d5c0 con 0x7fa390060eb0 2022-06-29T15:50:47.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.483+0000 7fa39533b700 1 -- 172.21.15.47:0/1137503157 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa384000e90 con 0x7fa390060eb0 2022-06-29T15:50:47.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.483+0000 7fa39533b700 1 -- 172.21.15.47:0/1137503157 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa38400d950 con 0x7fa390060eb0 2022-06-29T15:50:47.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.483+0000 7fa39733f700 1 -- 172.21.15.47:0/1137503157 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa390060eb0 msgr2=0x7fa390144e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:47.497 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.483+0000 7fa39733f700 1 --2- 172.21.15.47:0/1137503157 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa390060eb0 0x7fa390144e70 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 rx=0x7fa384002f70 tx=0x7fa384006eb0).stop 2022-06-29T15:50:47.497 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.483+0000 7fa39733f700 1 -- 172.21.15.47:0/1137503157 shutdown_connections 2022-06-29T15:50:47.497 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.483+0000 7fa39733f700 1 --2- 172.21.15.47:0/1137503157 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa390060eb0 0x7fa390144e70 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:47.497 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.483+0000 7fa39733f700 1 --2- 172.21.15.47:0/1137503157 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa3900b5c60 0x7fa390144980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:47.498 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.483+0000 7fa39733f700 1 -- 172.21.15.47:0/1137503157 >> 172.21.15.47:0/1137503157 conn(0x7fa390062560 msgr2=0x7fa390062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:47.499 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.483+0000 7fa39733f700 1 -- 172.21.15.47:0/1137503157 shutdown_connections 2022-06-29T15:50:47.499 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.483+0000 7fa39733f700 1 -- 172.21.15.47:0/1137503157 wait complete. 2022-06-29T15:50:47.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.487+0000 7fa39733f700 1 Processor -- start 2022-06-29T15:50:47.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.487+0000 7fa39733f700 1 -- start start 2022-06-29T15:50:47.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.487+0000 7fa39733f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa390060eb0 0x7fa3900ab720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:47.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.487+0000 7fa39733f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3900a72f0 0x7fa3901474c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:47.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.487+0000 7fa39733f700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa3900b5c60 0x7fa3901479b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:47.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.487+0000 7fa39733f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3900a9330 con 0x7fa3900a72f0 2022-06-29T15:50:47.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.487+0000 7fa39733f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa390142100 con 0x7fa390060eb0 2022-06-29T15:50:47.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.487+0000 7fa39733f700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3901575e0 con 0x7fa3900b5c60 2022-06-29T15:50:47.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.491+0000 7fa395b3c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3900a72f0 0x7fa3901474c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:47.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.491+0000 7fa39633d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa390060eb0 0x7fa3900ab720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:47.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.491+0000 7fa395b3c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3900a72f0 0x7fa3901474c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53044/0 (socket says 172.21.15.47:53044) 2022-06-29T15:50:47.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.491+0000 7fa395b3c700 1 -- 172.21.15.47:0/4100019884 learned_addr learned my addr 172.21.15.47:0/4100019884 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:47.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.491+0000 7fa396b3e700 1 --2- 172.21.15.47:0/4100019884 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa3900b5c60 0x7fa3901479b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:47.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.491+0000 7fa395b3c700 1 -- 172.21.15.47:0/4100019884 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa3900b5c60 msgr2=0x7fa3901479b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:47.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.491+0000 7fa395b3c700 1 --2- 172.21.15.47:0/4100019884 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa3900b5c60 0x7fa3901479b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:47.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.491+0000 7fa395b3c700 1 -- 172.21.15.47:0/4100019884 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa390060eb0 msgr2=0x7fa3900ab720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:47.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.491+0000 7fa395b3c700 1 --2- 172.21.15.47:0/4100019884 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa390060eb0 0x7fa3900ab720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:47.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.491+0000 7fa395b3c700 1 -- 172.21.15.47:0/4100019884 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa38400b1e0 con 0x7fa3900a72f0 2022-06-29T15:50:47.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.491+0000 7fa395b3c700 1 --2- 172.21.15.47:0/4100019884 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3900a72f0 0x7fa3901474c0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 rx=0x7fa384002f70 tx=0x7fa384015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:47.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.491+0000 7fa3837fe700 1 -- 172.21.15.47:0/4100019884 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3840067a0 con 0x7fa3900a72f0 2022-06-29T15:50:47.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.491+0000 7fa3837fe700 1 -- 172.21.15.47:0/4100019884 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa384011610 con 0x7fa3900a72f0 2022-06-29T15:50:47.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.491+0000 7fa3837fe700 1 -- 172.21.15.47:0/4100019884 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa384011970 con 0x7fa3900a72f0 2022-06-29T15:50:47.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.491+0000 7fa39733f700 1 -- 172.21.15.47:0/4100019884 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3900a9560 con 0x7fa3900a72f0 2022-06-29T15:50:47.513 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.491+0000 7fa39733f700 1 -- 172.21.15.47:0/4100019884 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa390157220 con 0x7fa3900a72f0 2022-06-29T15:50:47.513 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.491+0000 7fa39733f700 1 -- 172.21.15.47:0/4100019884 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3900573d0 con 0x7fa3900a72f0 2022-06-29T15:50:47.513 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.491+0000 7fa3837fe700 1 -- 172.21.15.47:0/4100019884 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa384025030 con 0x7fa3900a72f0 2022-06-29T15:50:47.513 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.491+0000 7fa3837fe700 1 --2- 172.21.15.47:0/4100019884 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa370053980 0x7fa370055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:47.514 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.491+0000 7fa3837fe700 1 -- 172.21.15.47:0/4100019884 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa38406d050 con 0x7fa3900a72f0 2022-06-29T15:50:47.514 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.491+0000 7fa39633d700 1 --2- 172.21.15.47:0/4100019884 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa370053980 0x7fa370055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:47.514 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.495+0000 7fa39633d700 1 --2- 172.21.15.47:0/4100019884 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa370053980 0x7fa370055e40 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7fa38c00c000 tx=0x7fa38c008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:47.515 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.499+0000 7fa3837fe700 1 -- 172.21.15.47:0/4100019884 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa384043d10 con 0x7fa3900a72f0 2022-06-29T15:50:47.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.663+0000 7fa39733f700 1 -- 172.21.15.47:0/4100019884 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa390069e20 con 0x7fa3900a72f0 2022-06-29T15:50:47.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.663+0000 7fa3837fe700 1 -- 172.21.15.47:0/4100019884 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+2498 (secure 0 0 0) 0x7fa384023070 con 0x7fa3900a72f0 2022-06-29T15:50:47.671 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:50:47.671 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":57,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7614,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/2948555969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2948555969},{"type":"v1","addr":"172.21.15.47:6835","nonce":2948555969}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":54},{"gid":7617,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.47:6837/1681295902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1681295902},{"type":"v1","addr":"172.21.15.47:6837","nonce":1681295902}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":55}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:50:38.459255+0000","modified":"2022-06-29T15:50:40.277564+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":56606},"failed":[],"damaged":[],"stopped":[],"info":{"gid_56606":{"gid":56606,"name":"b","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6833/2987151459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2987151459},{"type":"v1","addr":"172.21.15.64:6833","nonce":2987151459}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2022-06-29T15:50:47.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.667+0000 7fa39733f700 1 -- 172.21.15.47:0/4100019884 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa370053980 msgr2=0x7fa370055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:47.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.667+0000 7fa39733f700 1 --2- 172.21.15.47:0/4100019884 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa370053980 0x7fa370055e40 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7fa38c00c000 tx=0x7fa38c008040).stop 2022-06-29T15:50:47.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.667+0000 7fa39733f700 1 -- 172.21.15.47:0/4100019884 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3900a72f0 msgr2=0x7fa3901474c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:47.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.667+0000 7fa39733f700 1 --2- 172.21.15.47:0/4100019884 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3900a72f0 0x7fa3901474c0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 rx=0x7fa384002f70 tx=0x7fa384015000).stop 2022-06-29T15:50:47.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.667+0000 7fa39733f700 1 -- 172.21.15.47:0/4100019884 shutdown_connections 2022-06-29T15:50:47.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.667+0000 7fa39733f700 1 --2- 172.21.15.47:0/4100019884 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa3900b5c60 0x7fa3901479b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:47.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.667+0000 7fa39733f700 1 --2- 172.21.15.47:0/4100019884 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3900a72f0 0x7fa3901474c0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:47.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.667+0000 7fa39733f700 1 --2- 172.21.15.47:0/4100019884 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa370053980 0x7fa370055e40 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:47.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.667+0000 7fa39733f700 1 --2- 172.21.15.47:0/4100019884 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa390060eb0 0x7fa3900ab720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:47.676 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.667+0000 7fa39733f700 1 -- 172.21.15.47:0/4100019884 >> 172.21.15.47:0/4100019884 conn(0x7fa390062560 msgr2=0x7fa3900b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:47.676 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.667+0000 7fa39733f700 1 -- 172.21.15.47:0/4100019884 shutdown_connections 2022-06-29T15:50:47.676 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:47.667+0000 7fa39733f700 1 -- 172.21.15.47:0/4100019884 wait complete. 2022-06-29T15:50:47.677 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 57 2022-06-29T15:50:47.987 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 57, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-29T15:50:38.459255+0000', 'modified': '2022-06-29T15:50:40.277564+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 56606}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_56606': {'gid': 56606, 'name': 'b', 'rank': 0, 'incarnation': 55, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.64:6833/2987151459', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6832', 'nonce': 2987151459}, {'type': 'v1', 'addr': '172.21.15.64:6833', 'nonce': 2987151459}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-29T15:50:47.988 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-29T15:50:47.989 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-06-29T15:50:48.103 INFO:teuthology.orchestra.run.smithi064.stdout:{"cluster_fsid":"9a68894a-53ce-400e-838b-14eab91b834c","whoami":0,"id":56606,"want_state":"up:active","state":"up:active","rank_uptime":9.6261289439999995,"mdsmap_epoch":57,"osdmap_epoch":121,"osdmap_epoch_barrier":121,"uptime":13.895583066} 2022-06-29T15:50:48.104 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9a68894a-53ce-400e-838b-14eab91b834c", "id": 56606, "mdsmap_epoch": 57, "osdmap_epoch": 121, "osdmap_epoch_barrier": 121, "rank_uptime": 9.626128944, "state": "up:active", "uptime": 13.895583066, "want_state": "up:active", "whoami": 0 } 2022-06-29T15:50:48.105 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-29T15:50:48.105 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2022-06-29T15:50:48.106 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi164.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2022-06-29T15:50:48.107 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:48.107 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:50:48.117 DEBUG:teuthology.orchestra.run.smithi164:> sudo modprobe fuse 2022-06-29T15:50:48.177 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:50:48.178 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:50:48.192 INFO:teuthology.orchestra.run.smithi164.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:50:48.193 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:50:48.195 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:48.195 DEBUG:teuthology.orchestra.run.smithi164:> ls /sys/fs/fuse/connections 2022-06-29T15:50:48.201 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-29T15:50:48.202 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:50:48.245 DEBUG:teuthology.orchestra.run.smithi164:> sudo modprobe fuse 2022-06-29T15:50:48.262 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:50:48.263 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:50:48.273 INFO:teuthology.orchestra.run.smithi164.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:50:48.274 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:50:48.275 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:48.276 DEBUG:teuthology.orchestra.run.smithi164:> ls /sys/fs/fuse/connections 2022-06-29T15:50:48.322 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi164.stderr:2022-06-29T15:50:48.313+0000 7f3fdb804080 -1 init, newargv = 0x55c2647006a0 newargc=9 2022-06-29T15:50:48.322 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi164.stderr:ceph-fuse[12810]: starting ceph client 2022-06-29T15:50:48.334 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi164.stderr:ceph-fuse[12810]: starting fuse 2022-06-29T15:50:49.427 DEBUG:teuthology.orchestra.run.smithi164:> sudo modprobe fuse 2022-06-29T15:50:49.446 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:50:49.447 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:50:49.506 INFO:teuthology.orchestra.run.smithi164.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:50:49.507 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:50:49.508 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:49.508 DEBUG:teuthology.orchestra.run.smithi164:> ls /sys/fs/fuse/connections 2022-06-29T15:50:49.513 INFO:teuthology.orchestra.run.smithi164.stdout:52 2022-06-29T15:50:49.514 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-29T15:50:49.514 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:49.514 DEBUG:teuthology.orchestra.run.smithi164:> sudo python3 -c ' 2022-06-29T15:50:49.515 DEBUG:teuthology.orchestra.run.smithi164:> import glob 2022-06-29T15:50:49.515 DEBUG:teuthology.orchestra.run.smithi164:> import re 2022-06-29T15:50:49.516 DEBUG:teuthology.orchestra.run.smithi164:> import os 2022-06-29T15:50:49.516 DEBUG:teuthology.orchestra.run.smithi164:> import subprocess 2022-06-29T15:50:49.517 DEBUG:teuthology.orchestra.run.smithi164:> 2022-06-29T15:50:49.517 DEBUG:teuthology.orchestra.run.smithi164:> def find_socket(client_name): 2022-06-29T15:50:49.517 DEBUG:teuthology.orchestra.run.smithi164:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2022-06-29T15:50:49.518 DEBUG:teuthology.orchestra.run.smithi164:> files = glob.glob(asok_path) 2022-06-29T15:50:49.518 DEBUG:teuthology.orchestra.run.smithi164:> 2022-06-29T15:50:49.519 DEBUG:teuthology.orchestra.run.smithi164:> # Given a non-glob path, it better be there 2022-06-29T15:50:49.519 DEBUG:teuthology.orchestra.run.smithi164:> if "*" not in asok_path: 2022-06-29T15:50:49.519 DEBUG:teuthology.orchestra.run.smithi164:> assert(len(files) == 1) 2022-06-29T15:50:49.520 DEBUG:teuthology.orchestra.run.smithi164:> return files[0] 2022-06-29T15:50:49.520 DEBUG:teuthology.orchestra.run.smithi164:> 2022-06-29T15:50:49.521 DEBUG:teuthology.orchestra.run.smithi164:> for f in files: 2022-06-29T15:50:49.521 DEBUG:teuthology.orchestra.run.smithi164:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-29T15:50:49.521 DEBUG:teuthology.orchestra.run.smithi164:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-29T15:50:49.522 DEBUG:teuthology.orchestra.run.smithi164:> return f 2022-06-29T15:50:49.522 DEBUG:teuthology.orchestra.run.smithi164:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-29T15:50:49.523 DEBUG:teuthology.orchestra.run.smithi164:> 2022-06-29T15:50:49.523 DEBUG:teuthology.orchestra.run.smithi164:> print(find_socket("client.1")) 2022-06-29T15:50:49.523 DEBUG:teuthology.orchestra.run.smithi164:> ' 2022-06-29T15:50:49.565 INFO:teuthology.orchestra.run.smithi164.stdout:/var/run/ceph/ceph-client.1.12810.asok 2022-06-29T15:50:49.571 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.12810.asok 2022-06-29T15:50:49.571 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:49.572 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.12810.asok status 2022-06-29T15:50:49.708 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2022-06-29T15:50:49.708 INFO:teuthology.orchestra.run.smithi164.stdout: "metadata": { 2022-06-29T15:50:49.708 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_sha1": "f25ffac72e5a9e63860ff7deb6767a11f0559a16", 2022-06-29T15:50:49.709 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_version": "ceph version 15.2.16-304-gf25ffac7 (f25ffac72e5a9e63860ff7deb6767a11f0559a16) octopus (stable)", 2022-06-29T15:50:49.709 INFO:teuthology.orchestra.run.smithi164.stdout: "entity_id": "1", 2022-06-29T15:50:49.710 INFO:teuthology.orchestra.run.smithi164.stdout: "hostname": "smithi164", 2022-06-29T15:50:49.710 INFO:teuthology.orchestra.run.smithi164.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2022-06-29T15:50:49.710 INFO:teuthology.orchestra.run.smithi164.stdout: "pid": "12810", 2022-06-29T15:50:49.710 INFO:teuthology.orchestra.run.smithi164.stdout: "root": "/" 2022-06-29T15:50:49.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2022-06-29T15:50:49.711 INFO:teuthology.orchestra.run.smithi164.stdout: "dentry_count": 0, 2022-06-29T15:50:49.711 INFO:teuthology.orchestra.run.smithi164.stdout: "dentry_pinned_count": 0, 2022-06-29T15:50:49.711 INFO:teuthology.orchestra.run.smithi164.stdout: "id": 56756, 2022-06-29T15:50:49.712 INFO:teuthology.orchestra.run.smithi164.stdout: "inst": { 2022-06-29T15:50:49.712 INFO:teuthology.orchestra.run.smithi164.stdout: "name": { 2022-06-29T15:50:49.712 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "client", 2022-06-29T15:50:49.712 INFO:teuthology.orchestra.run.smithi164.stdout: "num": 56756 2022-06-29T15:50:49.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2022-06-29T15:50:49.713 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": { 2022-06-29T15:50:49.713 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "v1", 2022-06-29T15:50:49.713 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": "172.21.15.164:0", 2022-06-29T15:50:49.713 INFO:teuthology.orchestra.run.smithi164.stdout: "nonce": 2972384940 2022-06-29T15:50:49.714 INFO:teuthology.orchestra.run.smithi164.stdout: } 2022-06-29T15:50:49.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2022-06-29T15:50:49.714 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": { 2022-06-29T15:50:49.714 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "v1", 2022-06-29T15:50:49.715 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": "172.21.15.164:0", 2022-06-29T15:50:49.715 INFO:teuthology.orchestra.run.smithi164.stdout: "nonce": 2972384940 2022-06-29T15:50:49.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2022-06-29T15:50:49.715 INFO:teuthology.orchestra.run.smithi164.stdout: "inst_str": "client.56756 172.21.15.164:0/2972384940", 2022-06-29T15:50:49.716 INFO:teuthology.orchestra.run.smithi164.stdout: "addr_str": "172.21.15.164:0/2972384940", 2022-06-29T15:50:49.716 INFO:teuthology.orchestra.run.smithi164.stdout: "inode_count": 1, 2022-06-29T15:50:49.716 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_epoch": 57, 2022-06-29T15:50:49.716 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_epoch": 121, 2022-06-29T15:50:49.716 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_epoch_barrier": 0, 2022-06-29T15:50:49.717 INFO:teuthology.orchestra.run.smithi164.stdout: "blacklisted": false, 2022-06-29T15:50:49.717 INFO:teuthology.orchestra.run.smithi164.stdout: "fs_name": "cephfs" 2022-06-29T15:50:49.717 INFO:teuthology.orchestra.run.smithi164.stdout:} 2022-06-29T15:50:49.719 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:49.719 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:50:49.719 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:50:49.727 INFO:teuthology.orchestra.run.smithi164.stdout:fuseblk 2022-06-29T15:50:49.729 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-06-29T15:50:49.729 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:49.730 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:50:49.790 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:49.791 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:50:49.791 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:50:49.800 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2022-06-29T15:50:49.802 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-29T15:50:49.802 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:49.803 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:50:49.804 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:50:49.812 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2022-06-29T15:50:49.813 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-29T15:50:49.814 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi099.front.sepia.ceph.com... 2022-06-29T15:50:49.814 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-06-29T15:50:49.815 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:50:49.870 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[13288]: fuse finished with error 0 and tester_r 0 2022-06-29T15:50:49.872 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:49.872 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:50:49.873 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:50:49.879 INFO:teuthology.orchestra.run.smithi099.stdout:ext2/ext3 2022-06-29T15:50:49.880 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-06-29T15:50:49.881 INFO:teuthology.orchestra.run:waiting for 900 2022-06-29T15:50:55.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-29T15:50:55.983 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:50:55.991 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:55.992 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:50:55.993 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:50:56.002 INFO:teuthology.orchestra.run.smithi164.stdout:fuseblk 2022-06-29T15:50:56.004 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-06-29T15:50:56.004 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:56.005 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:50:56.006 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:50:56.054 INFO:teuthology.orchestra.run.smithi164.stdout:fuseblk 2022-06-29T15:50:56.056 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-06-29T15:50:56.056 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi164.front.sepia.ceph.com... 2022-06-29T15:50:56.056 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-06-29T15:50:56.057 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:50:56.073 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi164.stderr:ceph-fuse[12810]: fuse finished with error 0 and tester_r 0 2022-06-29T15:50:56.076 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:50:56.077 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:50:56.077 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:50:56.126 INFO:teuthology.orchestra.run.smithi164.stdout:ext2/ext3 2022-06-29T15:50:56.127 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-06-29T15:50:56.128 INFO:teuthology.orchestra.run:waiting for 900 2022-06-29T15:50:56.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-29T15:50:56.129 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:50:56.136 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-06-29T15:50:56.136 INFO:teuthology.orchestra.run:waiting for 300 2022-06-29T15:50:56.137 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2022-06-29T15:50:56.137 INFO:teuthology.orchestra.run:waiting for 300 2022-06-29T15:50:56.138 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-06-29T15:50:56.138 INFO:teuthology.orchestra.run:waiting for 300 2022-06-29T15:50:56.152 INFO:tasks.ceph.mds.a:Stopped 2022-06-29T15:50:56.153 DEBUG:teuthology.parallel:result is None 2022-06-29T15:50:56.154 INFO:tasks.ceph.mds.c:Stopped 2022-06-29T15:50:56.154 DEBUG:teuthology.parallel:result is None 2022-06-29T15:50:56.155 INFO:tasks.ceph.mds.b:Stopped 2022-06-29T15:50:56.156 DEBUG:teuthology.parallel:result is None 2022-06-29T15:50:56.156 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail a 2022-06-29T15:50:56.157 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail c 2022-06-29T15:50:56.158 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail b 2022-06-29T15:50:56.256 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.251+0000 7fd316613700 1 Processor -- start 2022-06-29T15:50:56.256 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.251+0000 7fd316613700 1 -- start start 2022-06-29T15:50:56.257 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.251+0000 7fa034fc9700 1 Processor -- start 2022-06-29T15:50:56.257 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.251+0000 7fa034fc9700 1 -- start start 2022-06-29T15:50:56.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.251+0000 7fd316613700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd3100ae7b0 0x7fd31013e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:56.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.251+0000 7fd316613700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd31013e7d0 0x7fd3100afe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:56.260 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.251+0000 7fd316613700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3100b2580 con 0x7fd3100b21a0 2022-06-29T15:50:56.260 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.251+0000 7fd316613700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3100ac6c0 con 0x7fd31013e7d0 2022-06-29T15:50:56.260 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.251+0000 7fd316613700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd310064160 con 0x7fd3100ae7b0 2022-06-29T15:50:56.260 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.251+0000 7fd30ffff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd3100b21a0 0x7fd310063770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38604/0 (socket says 172.21.15.47:38604) 2022-06-29T15:50:56.261 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.251+0000 7fd30ffff700 1 -- 172.21.15.47:0/2481262624 learned_addr learned my addr 172.21.15.47:0/2481262624 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:56.261 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.251+0000 7fd30f7fe700 1 --2- 172.21.15.47:0/2481262624 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd3100ae7b0 0x7fd31013e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:56.261 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.251+0000 7fd314bb0700 1 --2- 172.21.15.47:0/2481262624 >> v2:172.21.15.64:3300/0 conn(0x7fd31013e7d0 0x7fd3100afe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:56.262 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.251+0000 7fa034fc9700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa0300abfa0 0x7fa0300b2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:56.263 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.251+0000 7fa034fc9700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0301400a0 con 0x7fa0300b1a10 2022-06-29T15:50:56.263 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.251+0000 7fa034fc9700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa030140340 con 0x7fa0300b2610 2022-06-29T15:50:56.263 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.251+0000 7fa034fc9700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa030140d20 con 0x7fa0300abfa0 2022-06-29T15:50:56.263 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.251+0000 7fa02e59c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa0300abfa0 0x7fa0300b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:56.264 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.251+0000 7fa02e59c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa0300abfa0 0x7fa0300b2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53056/0 (socket says 172.21.15.47:53056) 2022-06-29T15:50:56.264 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.251+0000 7fa02e59c700 1 -- 172.21.15.47:0/188022693 learned_addr learned my addr 172.21.15.47:0/188022693 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:56.264 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.251+0000 7fa02ed9d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa0300b1a10 0x7fa0300afe20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38614/0 (socket says 172.21.15.47:38614) 2022-06-29T15:50:56.264 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.251+0000 7fa02dd9b700 1 --1- 172.21.15.47:0/188022693 >> v1:172.21.15.64:6789/0 conn(0x7fa0300b2610 0x7fa0300ac950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:46176/0 (socket says 172.21.15.47:46176) 2022-06-29T15:50:56.265 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.251+0000 7fd30effd700 1 -- 172.21.15.47:0/2481262624 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3830766245 0 0) 0x7fd3100b2580 con 0x7fd3100b21a0 2022-06-29T15:50:56.265 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.251+0000 7fd30effd700 1 -- 172.21.15.47:0/2481262624 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2f0003260 con 0x7fd3100b21a0 2022-06-29T15:50:56.265 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.251+0000 7fd30f7fe700 1 -- 172.21.15.47:0/2481262624 >> v2:172.21.15.64:3300/0 conn(0x7fd31013e7d0 msgr2=0x7fd3100afe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:56.265 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.251+0000 7fd30f7fe700 1 --2- 172.21.15.47:0/2481262624 >> v2:172.21.15.64:3300/0 conn(0x7fd31013e7d0 0x7fd3100afe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:56.266 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.251+0000 7fd30f7fe700 1 -- 172.21.15.47:0/2481262624 >> v1:172.21.15.47:6789/0 conn(0x7fd3100b21a0 legacy=0x7fd310063770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:56.266 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.251+0000 7fd30f7fe700 1 -- 172.21.15.47:0/2481262624 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3100addd0 con 0x7fd3100ae7b0 2022-06-29T15:50:56.266 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.251+0000 7fa02e59c700 1 -- 172.21.15.47:0/188022693 >> v1:172.21.15.64:6789/0 conn(0x7fa0300b2610 legacy=0x7fa0300ac950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:56.267 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.251+0000 7fa02e59c700 1 -- 172.21.15.47:0/188022693 >> v1:172.21.15.47:6789/0 conn(0x7fa0300b1a10 legacy=0x7fa0300afe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:56.267 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.251+0000 7fd30f7fe700 1 --2- 172.21.15.47:0/2481262624 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd3100ae7b0 0x7fd31013e2e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7fd2f8002f70 tx=0x7fd2f8006df0).ready entity=mon.2 client_cookie=9f102765bed70430 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:56.268 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.251+0000 7fa02e59c700 1 -- 172.21.15.47:0/188022693 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0301405e0 con 0x7fa0300abfa0 2022-06-29T15:50:56.268 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.251+0000 7fa02e59c700 1 --2- 172.21.15.47:0/188022693 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0300abfa0 0x7fa0300b2120 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 rx=0x7fa020002f70 tx=0x7fa020006eb0).ready entity=mon.0 client_cookie=381aa877dbead789 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:56.269 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.251+0000 7fd30effd700 1 -- 172.21.15.47:0/2481262624 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2f800d4a0 con 0x7fd3100ae7b0 2022-06-29T15:50:56.269 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.251+0000 7fd30effd700 1 -- 172.21.15.47:0/2481262624 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2f8000e90 con 0x7fd3100ae7b0 2022-06-29T15:50:56.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.251+0000 7fd30effd700 1 -- 172.21.15.47:0/2481262624 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2f800d7f0 con 0x7fd3100ae7b0 2022-06-29T15:50:56.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.251+0000 7fa02d59a700 1 -- 172.21.15.47:0/188022693 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa02000d660 con 0x7fa0300abfa0 2022-06-29T15:50:56.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.251+0000 7fa02d59a700 1 -- 172.21.15.47:0/188022693 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa020000e90 con 0x7fa0300abfa0 2022-06-29T15:50:56.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.251+0000 7fa02d59a700 1 -- 172.21.15.47:0/188022693 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa02000d9f0 con 0x7fa0300abfa0 2022-06-29T15:50:56.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fd316613700 1 -- 172.21.15.47:0/2481262624 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd3100ae7b0 msgr2=0x7fd31013e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:56.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fa034fc9700 1 -- 172.21.15.47:0/188022693 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0300abfa0 msgr2=0x7fa0300b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:56.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fa034fc9700 1 --2- 172.21.15.47:0/188022693 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0300abfa0 0x7fa0300b2120 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 rx=0x7fa020002f70 tx=0x7fa020006eb0).stop 2022-06-29T15:50:56.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fd316613700 1 --2- 172.21.15.47:0/2481262624 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd3100ae7b0 0x7fd31013e2e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7fd2f8002f70 tx=0x7fd2f8006df0).stop 2022-06-29T15:50:56.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fd316613700 1 -- 172.21.15.47:0/2481262624 shutdown_connections 2022-06-29T15:50:56.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fd316613700 1 --2- 172.21.15.47:0/2481262624 >> v2:172.21.15.64:3300/0 conn(0x7fd31013e7d0 0x7fd3100afe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:56.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fd316613700 1 --2- 172.21.15.47:0/2481262624 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd3100ae7b0 0x7fd31013e2e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:56.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fd316613700 1 -- 172.21.15.47:0/2481262624 >> 172.21.15.47:0/2481262624 conn(0x7fd31005c8b0 msgr2=0x7fd3100626f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:56.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fa034fc9700 1 -- 172.21.15.47:0/188022693 shutdown_connections 2022-06-29T15:50:56.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fa034fc9700 1 --2- 172.21.15.47:0/188022693 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0300abfa0 0x7fa0300b2120 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:56.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fa034fc9700 1 -- 172.21.15.47:0/188022693 >> 172.21.15.47:0/188022693 conn(0x7fa0300626f0 msgr2=0x7fa030062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:56.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fd316613700 1 -- 172.21.15.47:0/2481262624 shutdown_connections 2022-06-29T15:50:56.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fd316613700 1 -- 172.21.15.47:0/2481262624 wait complete. 2022-06-29T15:50:56.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fa034fc9700 1 -- 172.21.15.47:0/188022693 shutdown_connections 2022-06-29T15:50:56.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fa034fc9700 1 -- 172.21.15.47:0/188022693 wait complete. 2022-06-29T15:50:56.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fa034fc9700 1 Processor -- start 2022-06-29T15:50:56.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fa034fc9700 1 -- start start 2022-06-29T15:50:56.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fd316613700 1 Processor -- start 2022-06-29T15:50:56.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fd316613700 1 -- start start 2022-06-29T15:50:56.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fa034fc9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0300abfa0 0x7fa030136b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:56.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fa034fc9700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa0300b1a10 0x7fa030135e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:56.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fa034fc9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0300b3980 0x7fa0300acd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:56.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fa034fc9700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa030137000 con 0x7fa0300abfa0 2022-06-29T15:50:56.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fa034fc9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa030136870 con 0x7fa0300b3980 2022-06-29T15:50:56.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fa034fc9700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0301359c0 con 0x7fa0300b1a10 2022-06-29T15:50:56.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fa02e59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0300abfa0 0x7fa030136b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:56.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fa02e59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0300abfa0 0x7fa030136b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53062/0 (socket says 172.21.15.47:53062) 2022-06-29T15:50:56.282 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fa02e59c700 1 -- 172.21.15.47:0/2175470030 learned_addr learned my addr 172.21.15.47:0/2175470030 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:56.282 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fd316613700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3100ae7b0 0x7fd3100663f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:56.282 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fa02dd9b700 1 --2- 172.21.15.47:0/2175470030 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa0300b1a10 0x7fa030135e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:56.283 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fd316613700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3100b21a0 0x7fd310138ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:56.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fd316613700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd31013e7d0 0x7fd3101391d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:56.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fd316613700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd310140ef0 con 0x7fd3100b21a0 2022-06-29T15:50:56.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fd316613700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3101427a0 con 0x7fd3100ae7b0 2022-06-29T15:50:56.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fd316613700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd31013f850 con 0x7fd31013e7d0 2022-06-29T15:50:56.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fa02ed9d700 1 --2- 172.21.15.47:0/2175470030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0300b3980 0x7fa0300acd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:56.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7f129ad98700 1 Processor -- start 2022-06-29T15:50:56.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7f129ad98700 1 -- start start 2022-06-29T15:50:56.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fa02e59c700 1 -- 172.21.15.47:0/2175470030 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa0300b1a10 msgr2=0x7fa030135e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:56.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fa02e59c700 1 --2- 172.21.15.47:0/2175470030 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa0300b1a10 0x7fa030135e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:56.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fa02e59c700 1 -- 172.21.15.47:0/2175470030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0300b3980 msgr2=0x7fa0300acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:56.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fa02e59c700 1 --2- 172.21.15.47:0/2175470030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0300b3980 0x7fa0300acd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:56.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fa02e59c700 1 -- 172.21.15.47:0/2175470030 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa02000b1e0 con 0x7fa0300abfa0 2022-06-29T15:50:56.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fd30f7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3100b21a0 0x7fd310138ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:56.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fd30f7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3100b21a0 0x7fd310138ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53068/0 (socket says 172.21.15.47:53068) 2022-06-29T15:50:56.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fd30f7fe700 1 -- 172.21.15.47:0/362444418 learned_addr learned my addr 172.21.15.47:0/362444418 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:56.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fd314bb0700 1 --2- 172.21.15.47:0/362444418 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd31013e7d0 0x7fd3101391d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:56.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fa02e59c700 1 --2- 172.21.15.47:0/2175470030 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0300abfa0 0x7fa030136b10 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 rx=0x7fa020002f70 tx=0x7fa020015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:56.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fd30ffff700 1 --2- 172.21.15.47:0/362444418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3100ae7b0 0x7fd3100663f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:56.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fd30f7fe700 1 -- 172.21.15.47:0/362444418 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd31013e7d0 msgr2=0x7fd3101391d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:56.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fd30f7fe700 1 --2- 172.21.15.47:0/362444418 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd31013e7d0 0x7fd3101391d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:56.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fd30f7fe700 1 -- 172.21.15.47:0/362444418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3100ae7b0 msgr2=0x7fd3100663f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:56.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fd30f7fe700 1 --2- 172.21.15.47:0/362444418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3100ae7b0 0x7fd3100663f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:56.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fd30f7fe700 1 -- 172.21.15.47:0/362444418 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f800b180 con 0x7fd3100b21a0 2022-06-29T15:50:56.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fa01f7fe700 1 -- 172.21.15.47:0/2175470030 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa020011bf0 con 0x7fa0300abfa0 2022-06-29T15:50:56.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fa01f7fe700 1 -- 172.21.15.47:0/2175470030 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa020011d50 con 0x7fa0300abfa0 2022-06-29T15:50:56.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fa01f7fe700 1 -- 172.21.15.47:0/2175470030 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa02001e650 con 0x7fa0300abfa0 2022-06-29T15:50:56.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fd30f7fe700 1 --2- 172.21.15.47:0/362444418 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3100b21a0 0x7fd310138ce0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 rx=0x7fd2f8002f70 tx=0x7fd2f8014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:56.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fa034fc9700 1 -- 172.21.15.47:0/2175470030 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa030135520 con 0x7fa0300abfa0 2022-06-29T15:50:56.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fa034fc9700 1 -- 172.21.15.47:0/2175470030 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0300ad260 con 0x7fa0300abfa0 2022-06-29T15:50:56.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7f129ad98700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f129405fd20 0x7f129414b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:56.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7fd30d7fa700 1 -- 172.21.15.47:0/362444418 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2f8016070 con 0x7fd3100b21a0 2022-06-29T15:50:56.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.259+0000 7fd30d7fa700 1 -- 172.21.15.47:0/362444418 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2f8013a40 con 0x7fd3100b21a0 2022-06-29T15:50:56.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.259+0000 7fd30d7fa700 1 -- 172.21.15.47:0/362444418 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2f8013da0 con 0x7fd3100b21a0 2022-06-29T15:50:56.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.259+0000 7fd316613700 1 -- 172.21.15.47:0/362444418 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd31013ff50 con 0x7fd3100b21a0 2022-06-29T15:50:56.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7f129ad98700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12940acc20 con 0x7f1294144de0 2022-06-29T15:50:56.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7f129ad98700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12940acec0 con 0x7f12940b0aa0 2022-06-29T15:50:56.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.255+0000 7f129ad98700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12941451c0 con 0x7f129405fd20 2022-06-29T15:50:56.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.259+0000 7fd316613700 1 -- 172.21.15.47:0/362444418 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3101401e0 con 0x7fd3100b21a0 2022-06-29T15:50:56.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.259+0000 7f1298b34700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1294144de0 0x7f12940b1e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38632/0 (socket says 172.21.15.47:38632) 2022-06-29T15:50:56.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.259+0000 7f1298b34700 1 -- 172.21.15.47:0/2174422035 learned_addr learned my addr 172.21.15.47:0/2174422035 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:56.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.259+0000 7f1299335700 1 --2- 172.21.15.47:0/2174422035 >> v2:172.21.15.64:3300/0 conn(0x7f129405fd20 0x7f129414b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:56.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.259+0000 7f12937fe700 1 -- 172.21.15.47:0/2174422035 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2451490515 0 0) 0x7f12940acc20 con 0x7f1294144de0 2022-06-29T15:50:56.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.259+0000 7f12937fe700 1 -- 172.21.15.47:0/2174422035 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1274003260 con 0x7f1294144de0 2022-06-29T15:50:56.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.259+0000 7f12937fe700 1 -- 172.21.15.47:0/2174422035 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 272244414 0 0) 0x7f12940acec0 con 0x7f12940b0aa0 2022-06-29T15:50:56.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.259+0000 7f12937fe700 1 -- 172.21.15.47:0/2174422035 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12940acc20 con 0x7f12940b0aa0 2022-06-29T15:50:56.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.259+0000 7f1299335700 1 -- 172.21.15.47:0/2174422035 >> v1:172.21.15.64:6789/0 conn(0x7f12940b0aa0 legacy=0x7f129414aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:56.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.259+0000 7f1299335700 1 -- 172.21.15.47:0/2174422035 >> v1:172.21.15.47:6789/0 conn(0x7f1294144de0 legacy=0x7f12940b1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:56.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.259+0000 7f1299335700 1 -- 172.21.15.47:0/2174422035 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12940ad160 con 0x7f129405fd20 2022-06-29T15:50:56.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.259+0000 7fa01f7fe700 1 -- 172.21.15.47:0/2175470030 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa020025030 con 0x7fa0300abfa0 2022-06-29T15:50:56.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.259+0000 7f1299335700 1 --2- 172.21.15.47:0/2174422035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f129405fd20 0x7f129414b140 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7f1284002f70 tx=0x7f1284006df0).ready entity=mon.1 client_cookie=de9894ba6c39e23a server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:56.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.259+0000 7f12937fe700 1 -- 172.21.15.47:0/2174422035 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f128400d540 con 0x7f129405fd20 2022-06-29T15:50:56.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.259+0000 7f12937fe700 1 -- 172.21.15.47:0/2174422035 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1284000e90 con 0x7f129405fd20 2022-06-29T15:50:56.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.259+0000 7f12937fe700 1 -- 172.21.15.47:0/2174422035 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f128400d890 con 0x7f129405fd20 2022-06-29T15:50:56.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.259+0000 7fd316613700 1 -- 172.21.15.47:0/362444418 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2e8003410 con 0x7fd3100b21a0 2022-06-29T15:50:56.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.259+0000 7fa01f7fe700 1 --2- 172.21.15.47:0/2175470030 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa00c053aa0 0x7fa00c055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:56.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.259+0000 7fd30d7fa700 1 -- 172.21.15.47:0/362444418 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd2f800f030 con 0x7fd3100b21a0 2022-06-29T15:50:56.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.259+0000 7fa01f7fe700 1 -- 172.21.15.47:0/2175470030 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa02006d410 con 0x7fa0300abfa0 2022-06-29T15:50:56.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.259+0000 7fd30d7fa700 1 --2- 172.21.15.47:0/362444418 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd2f4053b00 0x7fd2f4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:56.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.259+0000 7fa034fc9700 1 -- 172.21.15.47:0/2175470030 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa030135b80 con 0x7fa0300abfa0 2022-06-29T15:50:56.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.259+0000 7fd30d7fa700 1 -- 172.21.15.47:0/362444418 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd2f806ccf0 con 0x7fd3100b21a0 2022-06-29T15:50:56.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.259+0000 7fd30ffff700 1 --2- 172.21.15.47:0/362444418 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd2f4053b00 0x7fd2f4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:56.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.259+0000 7f129ad98700 1 -- 172.21.15.47:0/2174422035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f129405fd20 msgr2=0x7f129414b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:56.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.259+0000 7f129ad98700 1 --2- 172.21.15.47:0/2174422035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f129405fd20 0x7f129414b140 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7f1284002f70 tx=0x7f1284006df0).stop 2022-06-29T15:50:56.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.259+0000 7fa02dd9b700 1 --2- 172.21.15.47:0/2175470030 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa00c053aa0 0x7fa00c055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:56.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.259+0000 7fd30ffff700 1 --2- 172.21.15.47:0/362444418 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd2f4053b00 0x7fd2f4055fc0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7fd30000b000 tx=0x7fd300009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:56.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.259+0000 7f129ad98700 1 -- 172.21.15.47:0/2174422035 shutdown_connections 2022-06-29T15:50:56.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.259+0000 7f129ad98700 1 --2- 172.21.15.47:0/2174422035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f129405fd20 0x7f129414b140 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:56.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.259+0000 7f129ad98700 1 -- 172.21.15.47:0/2174422035 >> 172.21.15.47:0/2174422035 conn(0x7f129405c8b0 msgr2=0x7f12940626f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:56.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.259+0000 7f129ad98700 1 -- 172.21.15.47:0/2174422035 shutdown_connections 2022-06-29T15:50:56.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.259+0000 7f129ad98700 1 -- 172.21.15.47:0/2174422035 wait complete. 2022-06-29T15:50:56.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.259+0000 7fa02dd9b700 1 --2- 172.21.15.47:0/2175470030 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa00c053aa0 0x7fa00c055f60 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7fa02400d000 tx=0x7fa024007b80).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:56.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.263+0000 7f129ad98700 1 Processor -- start 2022-06-29T15:50:56.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.263+0000 7f129ad98700 1 -- start start 2022-06-29T15:50:56.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.263+0000 7f129ad98700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f129405fd20 0x7f129414a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:56.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.263+0000 7f129ad98700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12940b0aa0 0x7f1294157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:56.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.263+0000 7f129ad98700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1294144de0 0x7f1294145300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:56.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.263+0000 7f129ad98700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1294060dd0 con 0x7f129405fd20 2022-06-29T15:50:56.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.263+0000 7f129ad98700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129414b3d0 con 0x7f12940b0aa0 2022-06-29T15:50:56.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.263+0000 7f129ad98700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1294156e60 con 0x7f1294144de0 2022-06-29T15:50:56.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.263+0000 7f1298b34700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f129405fd20 0x7f129414a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:56.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.263+0000 7f1298b34700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f129405fd20 0x7f129414a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53084/0 (socket says 172.21.15.47:53084) 2022-06-29T15:50:56.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.263+0000 7f1298b34700 1 -- 172.21.15.47:0/117934574 learned_addr learned my addr 172.21.15.47:0/117934574 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:56.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.263+0000 7f1299335700 1 --2- 172.21.15.47:0/117934574 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1294144de0 0x7f1294145300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:56.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.263+0000 7f1293fff700 1 --2- 172.21.15.47:0/117934574 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12940b0aa0 0x7f1294157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:56.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.263+0000 7f1298b34700 1 -- 172.21.15.47:0/117934574 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1294144de0 msgr2=0x7f1294145300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:56.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.263+0000 7f1298b34700 1 --2- 172.21.15.47:0/117934574 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1294144de0 0x7f1294145300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:56.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.263+0000 7f1298b34700 1 -- 172.21.15.47:0/117934574 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12940b0aa0 msgr2=0x7f1294157840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:56.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.263+0000 7f1298b34700 1 --2- 172.21.15.47:0/117934574 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12940b0aa0 0x7f1294157840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:56.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.263+0000 7f1298b34700 1 -- 172.21.15.47:0/117934574 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f128400b180 con 0x7f129405fd20 2022-06-29T15:50:56.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.263+0000 7f1298b34700 1 --2- 172.21.15.47:0/117934574 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f129405fd20 0x7f129414a540 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 rx=0x7f128c001550 tx=0x7f128c007ed0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:56.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.267+0000 7fd30d7fa700 1 -- 172.21.15.47:0/362444418 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd2f8043ab0 con 0x7fd3100b21a0 2022-06-29T15:50:56.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.267+0000 7f1291ffb700 1 -- 172.21.15.47:0/117934574 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f128c005ba0 con 0x7f129405fd20 2022-06-29T15:50:56.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.267+0000 7f1291ffb700 1 -- 172.21.15.47:0/117934574 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f128c005d00 con 0x7f129405fd20 2022-06-29T15:50:56.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.267+0000 7f129ad98700 1 -- 172.21.15.47:0/117934574 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1294157100 con 0x7f129405fd20 2022-06-29T15:50:56.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.267+0000 7f1291ffb700 1 -- 172.21.15.47:0/117934574 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f128c0072c0 con 0x7f129405fd20 2022-06-29T15:50:56.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.267+0000 7f129ad98700 1 -- 172.21.15.47:0/117934574 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f129414b050 con 0x7f129405fd20 2022-06-29T15:50:56.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.267+0000 7fa01f7fe700 1 -- 172.21.15.47:0/2175470030 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa020023080 con 0x7fa0300abfa0 2022-06-29T15:50:56.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.271+0000 7f129ad98700 1 -- 172.21.15.47:0/117934574 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12940573d0 con 0x7f129405fd20 2022-06-29T15:50:56.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.271+0000 7f1291ffb700 1 -- 172.21.15.47:0/117934574 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f128c010070 con 0x7f129405fd20 2022-06-29T15:50:56.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.271+0000 7f1291ffb700 1 --2- 172.21.15.47:0/117934574 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1278053b00 0x7f1278055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:56.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.271+0000 7f1291ffb700 1 -- 172.21.15.47:0/117934574 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f128c06bb20 con 0x7f129405fd20 2022-06-29T15:50:56.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.271+0000 7f1293fff700 1 --2- 172.21.15.47:0/117934574 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1278053b00 0x7f1278055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:56.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.271+0000 7f1293fff700 1 --2- 172.21.15.47:0/117934574 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1278053b00 0x7f1278055fc0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7f127c00b000 tx=0x7f127c009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:56.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.279+0000 7f1291ffb700 1 -- 172.21.15.47:0/117934574 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f128c0427d0 con 0x7f129405fd20 2022-06-29T15:50:56.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.435+0000 7fd316613700 1 -- 172.21.15.47:0/362444418 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "a"} v 0) v1 -- 0x7fd2e80036c0 con 0x7fd3100b21a0 2022-06-29T15:50:56.449 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.443+0000 7fa034fc9700 1 -- 172.21.15.47:0/2175470030 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "b"} v 0) v1 -- 0x7fa0300573d0 con 0x7fa0300abfa0 2022-06-29T15:50:56.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.451+0000 7f129ad98700 1 -- 172.21.15.47:0/117934574 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "c"} v 0) v1 -- 0x7f129405f140 con 0x7f129405fd20 2022-06-29T15:50:56.493 INFO:tasks.ceph.mon.a.smithi047.stderr:2022-06-29T15:50:56.487+0000 7ff9cf83c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-06-29T15:50:56.498 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.491+0000 7fd30d7fa700 1 -- 172.21.15.47:0/362444418 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "a"}]=0 failed mds gid 7617 v58) v1 ==== 95+0+0 (secure 0 0 0) 0x7fd2f8023020 con 0x7fd3100b21a0 2022-06-29T15:50:56.499 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.491+0000 7fa01f7fe700 1 -- 172.21.15.47:0/2175470030 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "b"}]=0 failed mds gid 56606 v58) v1 ==== 96+0+0 (secure 0 0 0) 0x7fa0200441d0 con 0x7fa0300abfa0 2022-06-29T15:50:56.499 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.491+0000 7f1291ffb700 1 -- 172.21.15.47:0/117934574 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "c"}]=0 failed mds gid 7614 v58) v1 ==== 95+0+0 (secure 0 0 0) 0x7f128c045df0 con 0x7f129405fd20 2022-06-29T15:50:56.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.495+0000 7fd316613700 1 -- 172.21.15.47:0/362444418 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd2f4053b00 msgr2=0x7fd2f4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:56.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.495+0000 7fd316613700 1 --2- 172.21.15.47:0/362444418 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd2f4053b00 0x7fd2f4055fc0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7fd30000b000 tx=0x7fd300009040).stop 2022-06-29T15:50:56.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.495+0000 7fd316613700 1 -- 172.21.15.47:0/362444418 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3100b21a0 msgr2=0x7fd310138ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:56.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.495+0000 7fd316613700 1 --2- 172.21.15.47:0/362444418 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3100b21a0 0x7fd310138ce0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 rx=0x7fd2f8002f70 tx=0x7fd2f8014000).stop 2022-06-29T15:50:56.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.495+0000 7fa034fc9700 1 -- 172.21.15.47:0/2175470030 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa00c053aa0 msgr2=0x7fa00c055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:56.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.495+0000 7fa034fc9700 1 --2- 172.21.15.47:0/2175470030 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa00c053aa0 0x7fa00c055f60 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7fa02400d000 tx=0x7fa024007b80).stop 2022-06-29T15:50:56.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.495+0000 7fa034fc9700 1 -- 172.21.15.47:0/2175470030 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0300abfa0 msgr2=0x7fa030136b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:56.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.495+0000 7fa034fc9700 1 --2- 172.21.15.47:0/2175470030 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0300abfa0 0x7fa030136b10 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 rx=0x7fa020002f70 tx=0x7fa020015000).stop 2022-06-29T15:50:56.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.495+0000 7fd316613700 1 -- 172.21.15.47:0/362444418 shutdown_connections 2022-06-29T15:50:56.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.495+0000 7fd316613700 1 --2- 172.21.15.47:0/362444418 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd31013e7d0 0x7fd3101391d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:56.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.495+0000 7fd316613700 1 --2- 172.21.15.47:0/362444418 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3100b21a0 0x7fd310138ce0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:56.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.495+0000 7fd316613700 1 --2- 172.21.15.47:0/362444418 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd2f4053b00 0x7fd2f4055fc0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:56.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.495+0000 7fd316613700 1 --2- 172.21.15.47:0/362444418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3100ae7b0 0x7fd3100663f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:56.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.495+0000 7fd316613700 1 -- 172.21.15.47:0/362444418 >> 172.21.15.47:0/362444418 conn(0x7fd31005c8b0 msgr2=0x7fd3100afd50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:56.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.495+0000 7fa034fc9700 1 -- 172.21.15.47:0/2175470030 shutdown_connections 2022-06-29T15:50:56.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.495+0000 7fa034fc9700 1 --2- 172.21.15.47:0/2175470030 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa00c053aa0 0x7fa00c055f60 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:56.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.495+0000 7fa034fc9700 1 --2- 172.21.15.47:0/2175470030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0300b3980 0x7fa0300acd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:56.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.495+0000 7fd316613700 1 -- 172.21.15.47:0/362444418 shutdown_connections 2022-06-29T15:50:56.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.495+0000 7fa034fc9700 1 --2- 172.21.15.47:0/2175470030 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa0300b1a10 0x7fa030135e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:56.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.495+0000 7fa034fc9700 1 --2- 172.21.15.47:0/2175470030 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0300abfa0 0x7fa030136b10 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:56.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.495+0000 7fd316613700 1 -- 172.21.15.47:0/362444418 wait complete. 2022-06-29T15:50:56.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.495+0000 7fa034fc9700 1 -- 172.21.15.47:0/2175470030 >> 172.21.15.47:0/2175470030 conn(0x7fa0300626f0 msgr2=0x7fa030147a70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:56.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.495+0000 7fa034fc9700 1 -- 172.21.15.47:0/2175470030 shutdown_connections 2022-06-29T15:50:56.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.495+0000 7fa034fc9700 1 -- 172.21.15.47:0/2175470030 wait complete. 2022-06-29T15:50:56.511 INFO:teuthology.orchestra.run.smithi047.stderr:failed mds gid 7617 2022-06-29T15:50:56.511 INFO:teuthology.orchestra.run.smithi047.stderr:failed mds gid 56606 2022-06-29T15:50:56.513 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.499+0000 7f129ad98700 1 -- 172.21.15.47:0/117934574 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1278053b00 msgr2=0x7f1278055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:56.513 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.499+0000 7f129ad98700 1 --2- 172.21.15.47:0/117934574 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1278053b00 0x7f1278055fc0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7f127c00b000 tx=0x7f127c009040).stop 2022-06-29T15:50:56.513 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.499+0000 7f129ad98700 1 -- 172.21.15.47:0/117934574 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f129405fd20 msgr2=0x7f129414a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:56.514 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.499+0000 7f129ad98700 1 --2- 172.21.15.47:0/117934574 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f129405fd20 0x7f129414a540 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 rx=0x7f128c001550 tx=0x7f128c007ed0).stop 2022-06-29T15:50:56.514 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.499+0000 7f129ad98700 1 -- 172.21.15.47:0/117934574 shutdown_connections 2022-06-29T15:50:56.514 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.499+0000 7f129ad98700 1 --2- 172.21.15.47:0/117934574 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1294144de0 0x7f1294145300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:56.515 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.499+0000 7f129ad98700 1 --2- 172.21.15.47:0/117934574 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1278053b00 0x7f1278055fc0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:56.515 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.499+0000 7f129ad98700 1 --2- 172.21.15.47:0/117934574 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12940b0aa0 0x7f1294157840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:56.515 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.499+0000 7f129ad98700 1 --2- 172.21.15.47:0/117934574 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f129405fd20 0x7f129414a540 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:56.515 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.499+0000 7f129ad98700 1 -- 172.21.15.47:0/117934574 >> 172.21.15.47:0/117934574 conn(0x7f129405c8b0 msgr2=0x7f12940b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:56.515 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.499+0000 7f129ad98700 1 -- 172.21.15.47:0/117934574 shutdown_connections 2022-06-29T15:50:56.516 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.499+0000 7f129ad98700 1 -- 172.21.15.47:0/117934574 wait complete. 2022-06-29T15:50:56.516 INFO:teuthology.orchestra.run.smithi047.stderr:failed mds gid 7614 2022-06-29T15:50:56.517 DEBUG:teuthology.parallel:result is '' 2022-06-29T15:50:56.520 DEBUG:teuthology.parallel:result is '' 2022-06-29T15:50:56.522 DEBUG:teuthology.parallel:result is '' 2022-06-29T15:50:56.522 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-06-29T15:50:56.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.623+0000 7fb8a9e37700 1 Processor -- start 2022-06-29T15:50:56.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.623+0000 7fb8a9e37700 1 -- start start 2022-06-29T15:50:56.632 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.623+0000 7fb8a9e37700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb8a409a1d0 0x7fb8a409a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:56.633 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.623+0000 7fb8a9e37700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb8a4142a30 0x7fb8a4142ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:56.633 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.627+0000 7fb8a9e37700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8a409ab20 con 0x7fb8a405f4a0 2022-06-29T15:50:56.633 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.627+0000 7fb8a9e37700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8a405f880 con 0x7fb8a4142a30 2022-06-29T15:50:56.633 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.627+0000 7fb8a9636700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb8a4142a30 0x7fb8a4142ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:56.634 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.627+0000 7fb8a9e37700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a409ae90 con 0x7fb8a409a1d0 2022-06-29T15:50:56.634 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.627+0000 7fb8a9636700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb8a4142a30 0x7fb8a4142ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53096/0 (socket says 172.21.15.47:53096) 2022-06-29T15:50:56.634 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.627+0000 7fb8a9636700 1 -- 172.21.15.47:0/4181930773 learned_addr learned my addr 172.21.15.47:0/4181930773 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:56.635 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.627+0000 7fb8a3fff700 1 --2- 172.21.15.47:0/4181930773 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb8a409a1d0 0x7fb8a409a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:56.635 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.627+0000 7fb8a9636700 1 -- 172.21.15.47:0/4181930773 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb8a409a1d0 msgr2=0x7fb8a409a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:56.635 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.627+0000 7fb8a9636700 1 --2- 172.21.15.47:0/4181930773 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb8a409a1d0 0x7fb8a409a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:56.635 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.627+0000 7fb8a9636700 1 -- 172.21.15.47:0/4181930773 >> v1:172.21.15.64:6789/0 conn(0x7fb8a405f4a0 legacy=0x7fb8a40b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:56.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.627+0000 7fb8a9636700 1 -- 172.21.15.47:0/4181930773 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a409b130 con 0x7fb8a4142a30 2022-06-29T15:50:56.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.627+0000 7fb8a3fff700 1 --2- 172.21.15.47:0/4181930773 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb8a409a1d0 0x7fb8a409a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:50:56.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.627+0000 7fb8a9636700 1 --2- 172.21.15.47:0/4181930773 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8a4142a30 0x7fb8a4142ea0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 rx=0x7fb89c002f70 tx=0x7fb89c006eb0).ready entity=mon.0 client_cookie=dc4febf894de1374 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:56.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.627+0000 7fb8a37fe700 1 -- 172.21.15.47:0/4181930773 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb89c014070 con 0x7fb8a4142a30 2022-06-29T15:50:56.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.627+0000 7fb8a37fe700 1 -- 172.21.15.47:0/4181930773 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb89c000e90 con 0x7fb8a4142a30 2022-06-29T15:50:56.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.627+0000 7fb8a37fe700 1 -- 172.21.15.47:0/4181930773 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb89c00e8e0 con 0x7fb8a4142a30 2022-06-29T15:50:56.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.627+0000 7fb8a9e37700 1 -- 172.21.15.47:0/4181930773 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8a4142a30 msgr2=0x7fb8a4142ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:56.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.627+0000 7fb8a9e37700 1 --2- 172.21.15.47:0/4181930773 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8a4142a30 0x7fb8a4142ea0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 rx=0x7fb89c002f70 tx=0x7fb89c006eb0).stop 2022-06-29T15:50:56.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.627+0000 7fb8a9e37700 1 -- 172.21.15.47:0/4181930773 shutdown_connections 2022-06-29T15:50:56.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.627+0000 7fb8a9e37700 1 --2- 172.21.15.47:0/4181930773 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8a4142a30 0x7fb8a4142ea0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:56.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.627+0000 7fb8a9e37700 1 --2- 172.21.15.47:0/4181930773 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb8a409a1d0 0x7fb8a409a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:56.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.627+0000 7fb8a9e37700 1 -- 172.21.15.47:0/4181930773 >> 172.21.15.47:0/4181930773 conn(0x7fb8a4062df0 msgr2=0x7fb8a40631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:56.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.627+0000 7fb8a9e37700 1 -- 172.21.15.47:0/4181930773 shutdown_connections 2022-06-29T15:50:56.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.627+0000 7fb8a9e37700 1 -- 172.21.15.47:0/4181930773 wait complete. 2022-06-29T15:50:56.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.631+0000 7fb8a9e37700 1 Processor -- start 2022-06-29T15:50:56.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.631+0000 7fb8a9e37700 1 -- start start 2022-06-29T15:50:56.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.631+0000 7fb8a9e37700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8a405f4a0 0x7fb8a4067430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:56.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.631+0000 7fb8a9e37700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb8a409a1d0 0x7fb8a4063fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:56.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.631+0000 7fb8a9e37700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8a4142a30 0x7fb8a40644a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:56.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.631+0000 7fb8a9e37700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a414eed0 con 0x7fb8a4142a30 2022-06-29T15:50:56.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.631+0000 7fb8a9e37700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a414e250 con 0x7fb8a405f4a0 2022-06-29T15:50:56.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.631+0000 7fb8a9e37700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a414e4f0 con 0x7fb8a409a1d0 2022-06-29T15:50:56.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.631+0000 7fb8a9636700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8a4142a30 0x7fb8a40644a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:56.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.631+0000 7fb8a9636700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8a4142a30 0x7fb8a40644a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53102/0 (socket says 172.21.15.47:53102) 2022-06-29T15:50:56.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.631+0000 7fb8a9636700 1 -- 172.21.15.47:0/3952985047 learned_addr learned my addr 172.21.15.47:0/3952985047 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:56.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.631+0000 7fb8a3fff700 1 --2- 172.21.15.47:0/3952985047 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb8a409a1d0 0x7fb8a4063fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:56.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.631+0000 7fb8a8e35700 1 --2- 172.21.15.47:0/3952985047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8a405f4a0 0x7fb8a4067430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:56.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.631+0000 7fb8a9636700 1 -- 172.21.15.47:0/3952985047 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb8a409a1d0 msgr2=0x7fb8a4063fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:56.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.631+0000 7fb8a9636700 1 --2- 172.21.15.47:0/3952985047 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb8a409a1d0 0x7fb8a4063fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:56.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.631+0000 7fb8a9636700 1 -- 172.21.15.47:0/3952985047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8a405f4a0 msgr2=0x7fb8a4067430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:56.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.631+0000 7fb8a9636700 1 --2- 172.21.15.47:0/3952985047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8a405f4a0 0x7fb8a4067430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:56.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.631+0000 7fb8a9636700 1 -- 172.21.15.47:0/3952985047 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb89c00c2f0 con 0x7fb8a4142a30 2022-06-29T15:50:56.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.631+0000 7fb8a9636700 1 --2- 172.21.15.47:0/3952985047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8a4142a30 0x7fb8a40644a0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 rx=0x7fb89c002f70 tx=0x7fb89c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:56.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.631+0000 7fb8a1ffb700 1 -- 172.21.15.47:0/3952985047 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb89c006be0 con 0x7fb8a4142a30 2022-06-29T15:50:56.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.631+0000 7fb8a1ffb700 1 -- 172.21.15.47:0/3952985047 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb89c01f870 con 0x7fb8a4142a30 2022-06-29T15:50:56.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.631+0000 7fb8a1ffb700 1 -- 172.21.15.47:0/3952985047 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb89c01fbd0 con 0x7fb8a4142a30 2022-06-29T15:50:56.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.631+0000 7fb8a9e37700 1 -- 172.21.15.47:0/3952985047 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8a414e790 con 0x7fb8a4142a30 2022-06-29T15:50:56.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.631+0000 7fb8a9e37700 1 -- 172.21.15.47:0/3952985047 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8a41407c0 con 0x7fb8a4142a30 2022-06-29T15:50:56.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.631+0000 7fb8a9e37700 1 -- 172.21.15.47:0/3952985047 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8a40573d0 con 0x7fb8a4142a30 2022-06-29T15:50:56.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.635+0000 7fb8a1ffb700 1 -- 172.21.15.47:0/3952985047 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb89c025020 con 0x7fb8a4142a30 2022-06-29T15:50:56.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.635+0000 7fb8a1ffb700 1 --2- 172.21.15.47:0/3952985047 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb888053b00 0x7fb888055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:56.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.635+0000 7fb8a1ffb700 1 -- 172.21.15.47:0/3952985047 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fb89c06fda0 con 0x7fb8a4142a30 2022-06-29T15:50:56.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.635+0000 7fb8a8e35700 1 --2- 172.21.15.47:0/3952985047 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb888053b00 0x7fb888055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:56.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.635+0000 7fb8a8e35700 1 --2- 172.21.15.47:0/3952985047 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb888053b00 0x7fb888055fc0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7fb89400d000 tx=0x7fb894007f10).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:56.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.639+0000 7fb8a1ffb700 1 -- 172.21.15.47:0/3952985047 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb89c046b10 con 0x7fb8a4142a30 2022-06-29T15:50:56.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:56.803+0000 7fb8a9e37700 1 -- 172.21.15.47:0/3952985047 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb8a4067b30 con 0x7fb8a4142a30 2022-06-29T15:50:57.517 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.511+0000 7fb8a1ffb700 1 -- 172.21.15.47:0/3952985047 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v59) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb89c04a130 con 0x7fb8a4142a30 2022-06-29T15:50:57.520 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.515+0000 7fb8a9e37700 1 -- 172.21.15.47:0/3952985047 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb888053b00 msgr2=0x7fb888055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:57.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.515+0000 7fb8a9e37700 1 --2- 172.21.15.47:0/3952985047 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb888053b00 0x7fb888055fc0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7fb89400d000 tx=0x7fb894007f10).stop 2022-06-29T15:50:57.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.515+0000 7fb8a9e37700 1 -- 172.21.15.47:0/3952985047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8a4142a30 msgr2=0x7fb8a40644a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:57.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.515+0000 7fb8a9e37700 1 --2- 172.21.15.47:0/3952985047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8a4142a30 0x7fb8a40644a0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 rx=0x7fb89c002f70 tx=0x7fb89c014000).stop 2022-06-29T15:50:57.522 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.515+0000 7fb8a9e37700 1 -- 172.21.15.47:0/3952985047 shutdown_connections 2022-06-29T15:50:57.522 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.515+0000 7fb8a9e37700 1 --2- 172.21.15.47:0/3952985047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8a4142a30 0x7fb8a40644a0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:57.522 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.515+0000 7fb8a9e37700 1 --2- 172.21.15.47:0/3952985047 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb8a409a1d0 0x7fb8a4063fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:57.522 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.515+0000 7fb8a9e37700 1 --2- 172.21.15.47:0/3952985047 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb888053b00 0x7fb888055fc0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:57.523 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.515+0000 7fb8a9e37700 1 --2- 172.21.15.47:0/3952985047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8a405f4a0 0x7fb8a4067430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:57.523 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.515+0000 7fb8a9e37700 1 -- 172.21.15.47:0/3952985047 >> 172.21.15.47:0/3952985047 conn(0x7fb8a4062df0 msgr2=0x7fb8a413b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:57.523 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.515+0000 7fb8a9e37700 1 -- 172.21.15.47:0/3952985047 shutdown_connections 2022-06-29T15:50:57.523 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.515+0000 7fb8a9e37700 1 -- 172.21.15.47:0/3952985047 wait complete. 2022-06-29T15:50:57.536 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-06-29T15:50:57.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.647+0000 7f6eff311700 1 Processor -- start 2022-06-29T15:50:57.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.647+0000 7f6eff311700 1 -- start start 2022-06-29T15:50:57.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.651+0000 7f6eff311700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6ef80b5d90 0x7f6ef8143a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:57.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.651+0000 7f6eff311700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ef805cc30 con 0x7f6ef80676a0 2022-06-29T15:50:57.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.651+0000 7f6eff311700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ef81430a0 con 0x7f6ef805c850 2022-06-29T15:50:57.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.651+0000 7f6eff311700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ef8143300 con 0x7f6ef80b5d90 2022-06-29T15:50:57.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.651+0000 7f6efeb10700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f6ef80676a0 0x7f6ef8140130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38664/0 (socket says 172.21.15.47:38664) 2022-06-29T15:50:57.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.651+0000 7f6efeb10700 1 -- 172.21.15.47:0/3952880306 learned_addr learned my addr 172.21.15.47:0/3952880306 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:57.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.651+0000 7f6efdb0e700 1 --2- 172.21.15.47:0/3952880306 >> v2:172.21.15.64:3300/0 conn(0x7f6ef80b5d90 0x7f6ef8143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:57.664 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.651+0000 7f6efd30d700 1 -- 172.21.15.47:0/3952880306 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2340949307 0 0) 0x7f6ef805cc30 con 0x7f6ef80676a0 2022-06-29T15:50:57.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.651+0000 7f6efd30d700 1 -- 172.21.15.47:0/3952880306 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ee0003260 con 0x7f6ef80676a0 2022-06-29T15:50:57.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.651+0000 7f6efdb0e700 1 -- 172.21.15.47:0/3952880306 >> v1:172.21.15.64:6789/0 conn(0x7f6ef805c850 legacy=0x7f6ef80b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:57.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.651+0000 7f6efdb0e700 1 -- 172.21.15.47:0/3952880306 >> v1:172.21.15.47:6789/0 conn(0x7f6ef80676a0 legacy=0x7f6ef8140130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:57.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.651+0000 7f6efdb0e700 1 -- 172.21.15.47:0/3952880306 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef81435a0 con 0x7f6ef80b5d90 2022-06-29T15:50:57.666 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.651+0000 7f6efdb0e700 1 --2- 172.21.15.47:0/3952880306 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ef80b5d90 0x7f6ef8143a40 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7f6ee8002f70 tx=0x7f6ee8006df0).ready entity=mon.1 client_cookie=e8f18aefefeeba74 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:57.666 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.651+0000 7f6efd30d700 1 -- 172.21.15.47:0/3952880306 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ee800d610 con 0x7f6ef80b5d90 2022-06-29T15:50:57.666 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.651+0000 7f6efd30d700 1 -- 172.21.15.47:0/3952880306 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ee8000e90 con 0x7f6ef80b5d90 2022-06-29T15:50:57.667 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.651+0000 7f6efd30d700 1 -- 172.21.15.47:0/3952880306 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ee800d960 con 0x7f6ef80b5d90 2022-06-29T15:50:57.667 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.655+0000 7f6eff311700 1 -- 172.21.15.47:0/3952880306 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ef80b5d90 msgr2=0x7f6ef8143a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:57.668 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.655+0000 7f6eff311700 1 --2- 172.21.15.47:0/3952880306 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ef80b5d90 0x7f6ef8143a40 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7f6ee8002f70 tx=0x7f6ee8006df0).stop 2022-06-29T15:50:57.668 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.655+0000 7f6eff311700 1 -- 172.21.15.47:0/3952880306 shutdown_connections 2022-06-29T15:50:57.668 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.655+0000 7f6eff311700 1 --2- 172.21.15.47:0/3952880306 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ef80b5d90 0x7f6ef8143a40 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:57.668 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.655+0000 7f6eff311700 1 -- 172.21.15.47:0/3952880306 >> 172.21.15.47:0/3952880306 conn(0x7f6ef8060e00 msgr2=0x7f6ef80611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:57.669 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.655+0000 7f6eff311700 1 -- 172.21.15.47:0/3952880306 shutdown_connections 2022-06-29T15:50:57.669 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.655+0000 7f6eff311700 1 -- 172.21.15.47:0/3952880306 wait complete. 2022-06-29T15:50:57.669 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.655+0000 7f6eff311700 1 Processor -- start 2022-06-29T15:50:57.669 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.655+0000 7f6eff311700 1 -- start start 2022-06-29T15:50:57.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.655+0000 7f6eff311700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6ef805c850 0x7f6ef80b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:57.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.655+0000 7f6eff311700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ef80676a0 0x7f6ef813f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:57.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.655+0000 7f6eff311700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ef80b5d90 0x7f6ef813f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:57.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.655+0000 7f6eff311700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef80b0c30 con 0x7f6ef80b5d90 2022-06-29T15:50:57.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.655+0000 7f6eff311700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef80b8ea0 con 0x7f6ef80676a0 2022-06-29T15:50:57.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.655+0000 7f6eff311700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef81531b0 con 0x7f6ef805c850 2022-06-29T15:50:57.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.655+0000 7f6efeb10700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ef80b5d90 0x7f6ef813f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:57.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.655+0000 7f6efeb10700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ef80b5d90 0x7f6ef813f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53114/0 (socket says 172.21.15.47:53114) 2022-06-29T15:50:57.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.655+0000 7f6efeb10700 1 -- 172.21.15.47:0/1984906865 learned_addr learned my addr 172.21.15.47:0/1984906865 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:57.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.659+0000 7f6efdb0e700 1 --2- 172.21.15.47:0/1984906865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ef80676a0 0x7f6ef813f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:57.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.659+0000 7f6efe30f700 1 --2- 172.21.15.47:0/1984906865 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6ef805c850 0x7f6ef80b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:57.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.659+0000 7f6efeb10700 1 -- 172.21.15.47:0/1984906865 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6ef805c850 msgr2=0x7f6ef80b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:57.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.659+0000 7f6efeb10700 1 --2- 172.21.15.47:0/1984906865 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6ef805c850 0x7f6ef80b3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:57.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.659+0000 7f6efeb10700 1 -- 172.21.15.47:0/1984906865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ef80676a0 msgr2=0x7f6ef813f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:57.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.659+0000 7f6efeb10700 1 --2- 172.21.15.47:0/1984906865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ef80676a0 0x7f6ef813f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:57.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.659+0000 7f6efeb10700 1 -- 172.21.15.47:0/1984906865 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee800b180 con 0x7f6ef80b5d90 2022-06-29T15:50:57.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.659+0000 7f6efeb10700 1 --2- 172.21.15.47:0/1984906865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ef80b5d90 0x7f6ef813f710 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 rx=0x7f6ef80b3940 tx=0x7f6ef0007dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:57.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.659+0000 7f6ee77fe700 1 -- 172.21.15.47:0/1984906865 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ef0005ba0 con 0x7f6ef80b5d90 2022-06-29T15:50:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.659+0000 7f6ee77fe700 1 -- 172.21.15.47:0/1984906865 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ef0005d00 con 0x7f6ef80b5d90 2022-06-29T15:50:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.659+0000 7f6ee77fe700 1 -- 172.21.15.47:0/1984906865 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ef00072c0 con 0x7f6ef80b5d90 2022-06-29T15:50:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.659+0000 7f6eff311700 1 -- 172.21.15.47:0/1984906865 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ef8153450 con 0x7f6ef80b5d90 2022-06-29T15:50:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.659+0000 7f6eff311700 1 -- 172.21.15.47:0/1984906865 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ef812bdc0 con 0x7f6ef80b5d90 2022-06-29T15:50:57.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.659+0000 7f6eff311700 1 -- 172.21.15.47:0/1984906865 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ef80573d0 con 0x7f6ef80b5d90 2022-06-29T15:50:57.676 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.659+0000 7f6ee77fe700 1 -- 172.21.15.47:0/1984906865 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6ef0010070 con 0x7f6ef80b5d90 2022-06-29T15:50:57.676 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.663+0000 7f6ee77fe700 1 --2- 172.21.15.47:0/1984906865 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6ed8053980 0x7f6ed8055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:57.676 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.663+0000 7f6ee77fe700 1 -- 172.21.15.47:0/1984906865 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f6ef006bb10 con 0x7f6ef80b5d90 2022-06-29T15:50:57.676 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.663+0000 7f6efe30f700 1 --2- 172.21.15.47:0/1984906865 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6ed8053980 0x7f6ed8055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:57.677 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.663+0000 7f6efe30f700 1 --2- 172.21.15.47:0/1984906865 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6ed8053980 0x7f6ed8055e40 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7f6ef400b000 tx=0x7f6ef4009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:57.677 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.667+0000 7f6ee77fe700 1 -- 172.21.15.47:0/1984906865 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6ef0042770 con 0x7f6ef80b5d90 2022-06-29T15:50:57.794 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:57.787+0000 7f6eff311700 1 -- 172.21.15.47:0/1984906865 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6ef80ba5c0 con 0x7f6ef80b5d90 2022-06-29T15:50:58.518 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.511+0000 7f6ee77fe700 1 -- 172.21.15.47:0/1984906865 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v125) v1 ==== 185+0+0 (secure 0 0 0) 0x7f6ef0045d90 con 0x7f6ef80b5d90 2022-06-29T15:50:58.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.515+0000 7f6eff311700 1 -- 172.21.15.47:0/1984906865 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6ed8053980 msgr2=0x7f6ed8055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:58.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.515+0000 7f6eff311700 1 --2- 172.21.15.47:0/1984906865 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6ed8053980 0x7f6ed8055e40 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7f6ef400b000 tx=0x7f6ef4009040).stop 2022-06-29T15:50:58.522 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.515+0000 7f6eff311700 1 -- 172.21.15.47:0/1984906865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ef80b5d90 msgr2=0x7f6ef813f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:58.522 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.515+0000 7f6eff311700 1 --2- 172.21.15.47:0/1984906865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ef80b5d90 0x7f6ef813f710 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 rx=0x7f6ef80b3940 tx=0x7f6ef0007dc0).stop 2022-06-29T15:50:58.522 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.515+0000 7f6eff311700 1 -- 172.21.15.47:0/1984906865 shutdown_connections 2022-06-29T15:50:58.522 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.515+0000 7f6eff311700 1 --2- 172.21.15.47:0/1984906865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ef80b5d90 0x7f6ef813f710 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:58.523 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.515+0000 7f6eff311700 1 --2- 172.21.15.47:0/1984906865 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6ed8053980 0x7f6ed8055e40 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:58.523 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.515+0000 7f6eff311700 1 --2- 172.21.15.47:0/1984906865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ef80676a0 0x7f6ef813f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:58.523 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.515+0000 7f6eff311700 1 --2- 172.21.15.47:0/1984906865 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6ef805c850 0x7f6ef80b3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:58.523 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.515+0000 7f6eff311700 1 -- 172.21.15.47:0/1984906865 >> 172.21.15.47:0/1984906865 conn(0x7f6ef8060e00 msgr2=0x7f6ef8144f90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:58.524 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.515+0000 7f6eff311700 1 -- 172.21.15.47:0/1984906865 shutdown_connections 2022-06-29T15:50:58.524 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.515+0000 7f6eff311700 1 -- 172.21.15.47:0/1984906865 wait complete. 2022-06-29T15:50:58.524 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' removed 2022-06-29T15:50:58.534 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-06-29T15:50:58.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.643+0000 7f285ef02700 1 Processor -- start 2022-06-29T15:50:58.651 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.643+0000 7f285ef02700 1 -- start start 2022-06-29T15:50:58.653 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.647+0000 7f285ef02700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f28580619b0 0x7f28580b1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:58.654 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.647+0000 7f285ef02700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f28580b1e20 0x7f2858146790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:58.654 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.647+0000 7f285ef02700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2858061d90 con 0x7f285805f0b0 2022-06-29T15:50:58.654 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.647+0000 7f285ef02700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f285805f6d0 con 0x7f28580b1e20 2022-06-29T15:50:58.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.647+0000 7f285ef02700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28580b3bf0 con 0x7f28580619b0 2022-06-29T15:50:58.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.647+0000 7f2857fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f28580b1e20 0x7f2858146790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:58.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.647+0000 7f285d49f700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f285805f0b0 0x7f28580b01b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38678/0 (socket says 172.21.15.47:38678) 2022-06-29T15:50:58.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.647+0000 7f285d49f700 1 -- 172.21.15.47:0/36838714 learned_addr learned my addr 172.21.15.47:0/36838714 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:50:58.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.647+0000 7f285cc9e700 1 --2- 172.21.15.47:0/36838714 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f28580619b0 0x7f28580b1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:58.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.647+0000 7f28577fe700 1 -- 172.21.15.47:0/36838714 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 304804009 0 0) 0x7f2858061d90 con 0x7f285805f0b0 2022-06-29T15:50:58.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.647+0000 7f28577fe700 1 -- 172.21.15.47:0/36838714 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2844003260 con 0x7f285805f0b0 2022-06-29T15:50:58.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.647+0000 7f2857fff700 1 -- 172.21.15.47:0/36838714 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f28580619b0 msgr2=0x7f28580b1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:58.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.647+0000 7f2857fff700 1 --2- 172.21.15.47:0/36838714 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f28580619b0 0x7f28580b1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:58.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.647+0000 7f2857fff700 1 -- 172.21.15.47:0/36838714 >> v1:172.21.15.47:6789/0 conn(0x7f285805f0b0 legacy=0x7f28580b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:58.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.647+0000 7f2857fff700 1 -- 172.21.15.47:0/36838714 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28580aea80 con 0x7f28580b1e20 2022-06-29T15:50:58.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.647+0000 7f2857fff700 1 --2- 172.21.15.47:0/36838714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28580b1e20 0x7f2858146790 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 rx=0x7f284c002f70 tx=0x7f284c006df0).ready entity=mon.0 client_cookie=d068ca19ddb9a32 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:58.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.647+0000 7f28577fe700 1 -- 172.21.15.47:0/36838714 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f284c00d4a0 con 0x7f28580b1e20 2022-06-29T15:50:58.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.647+0000 7f28577fe700 1 -- 172.21.15.47:0/36838714 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f284c000e90 con 0x7f28580b1e20 2022-06-29T15:50:58.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.647+0000 7f28577fe700 1 -- 172.21.15.47:0/36838714 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f284c00d7f0 con 0x7f28580b1e20 2022-06-29T15:50:58.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.651+0000 7f285ef02700 1 -- 172.21.15.47:0/36838714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28580b1e20 msgr2=0x7f2858146790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:58.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.651+0000 7f285ef02700 1 --2- 172.21.15.47:0/36838714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28580b1e20 0x7f2858146790 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 rx=0x7f284c002f70 tx=0x7f284c006df0).stop 2022-06-29T15:50:58.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.651+0000 7f285ef02700 1 -- 172.21.15.47:0/36838714 shutdown_connections 2022-06-29T15:50:58.663 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.651+0000 7f285ef02700 1 --2- 172.21.15.47:0/36838714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28580b1e20 0x7f2858146790 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:58.663 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.651+0000 7f285ef02700 1 --2- 172.21.15.47:0/36838714 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f28580619b0 0x7f28580b1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:58.663 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.651+0000 7f285ef02700 1 -- 172.21.15.47:0/36838714 >> 172.21.15.47:0/36838714 conn(0x7f2858063800 msgr2=0x7f2858063be0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:58.663 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.651+0000 7f285ef02700 1 -- 172.21.15.47:0/36838714 shutdown_connections 2022-06-29T15:50:58.664 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.651+0000 7f285ef02700 1 -- 172.21.15.47:0/36838714 wait complete. 2022-06-29T15:50:58.664 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.651+0000 7f285ef02700 1 Processor -- start 2022-06-29T15:50:58.664 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.651+0000 7f285ef02700 1 -- start start 2022-06-29T15:50:58.664 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.655+0000 7f285ef02700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f285805f0b0 0x7f28580b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:58.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.655+0000 7f285ef02700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28580619b0 0x7f2858141ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:58.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.655+0000 7f285ef02700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28580b1e20 0x7f2858141420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:58.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.655+0000 7f285ef02700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2858064d30 con 0x7f28580619b0 2022-06-29T15:50:58.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.655+0000 7f285ef02700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28580647f0 con 0x7f28580b1e20 2022-06-29T15:50:58.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.655+0000 7f285ef02700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2858064a90 con 0x7f285805f0b0 2022-06-29T15:50:58.666 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.655+0000 7f2857fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28580619b0 0x7f2858141ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:58.666 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.655+0000 7f2857fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28580619b0 0x7f2858141ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53128/0 (socket says 172.21.15.47:53128) 2022-06-29T15:50:58.666 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.655+0000 7f2857fff700 1 -- 172.21.15.47:0/1777233305 learned_addr learned my addr 172.21.15.47:0/1777233305 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:50:58.667 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.655+0000 7f285cc9e700 1 --2- 172.21.15.47:0/1777233305 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f285805f0b0 0x7f28580b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:58.667 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.655+0000 7f285d49f700 1 --2- 172.21.15.47:0/1777233305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28580b1e20 0x7f2858141420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:58.667 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.655+0000 7f2857fff700 1 -- 172.21.15.47:0/1777233305 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f285805f0b0 msgr2=0x7f28580b6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:58.667 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.655+0000 7f2857fff700 1 --2- 172.21.15.47:0/1777233305 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f285805f0b0 0x7f28580b6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:58.668 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.655+0000 7f2857fff700 1 -- 172.21.15.47:0/1777233305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28580b1e20 msgr2=0x7f2858141420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:58.668 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.655+0000 7f2857fff700 1 --2- 172.21.15.47:0/1777233305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28580b1e20 0x7f2858141420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:58.668 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.655+0000 7f2857fff700 1 -- 172.21.15.47:0/1777233305 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f284c00b180 con 0x7f28580619b0 2022-06-29T15:50:58.668 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.655+0000 7f2857fff700 1 --2- 172.21.15.47:0/1777233305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28580619b0 0x7f2858141ce0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 rx=0x7f284c002f70 tx=0x7f284c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:58.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.655+0000 7f2855ffb700 1 -- 172.21.15.47:0/1777233305 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f284c0139c0 con 0x7f28580619b0 2022-06-29T15:50:58.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.655+0000 7f2855ffb700 1 -- 172.21.15.47:0/1777233305 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f284c013b20 con 0x7f28580619b0 2022-06-29T15:50:58.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.655+0000 7f285ef02700 1 -- 172.21.15.47:0/1777233305 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2858140f80 con 0x7f28580619b0 2022-06-29T15:50:58.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.655+0000 7f2855ffb700 1 -- 172.21.15.47:0/1777233305 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f284c013e80 con 0x7f28580619b0 2022-06-29T15:50:58.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.655+0000 7f285ef02700 1 -- 172.21.15.47:0/1777233305 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2858066b30 con 0x7f28580619b0 2022-06-29T15:50:58.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.659+0000 7f285ef02700 1 -- 172.21.15.47:0/1777233305 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2858141140 con 0x7f28580619b0 2022-06-29T15:50:58.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.659+0000 7f2855ffb700 1 -- 172.21.15.47:0/1777233305 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f284c025030 con 0x7f28580619b0 2022-06-29T15:50:58.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.659+0000 7f2855ffb700 1 --2- 172.21.15.47:0/1777233305 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2838053aa0 0x7f2838055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:58.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.659+0000 7f2855ffb700 1 -- 172.21.15.47:0/1777233305 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6068+0+0 (secure 0 0 0) 0x7f284c06cbb0 con 0x7f28580619b0 2022-06-29T15:50:58.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.659+0000 7f285cc9e700 1 --2- 172.21.15.47:0/1777233305 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2838053aa0 0x7f2838055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:58.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.659+0000 7f285cc9e700 1 --2- 172.21.15.47:0/1777233305 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2838053aa0 0x7f2838055f60 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7f284800b000 tx=0x7f2848006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:58.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.667+0000 7f2855ffb700 1 -- 172.21.15.47:0/1777233305 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f284c073080 con 0x7f28580619b0 2022-06-29T15:50:58.803 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:58.799+0000 7f285ef02700 1 -- 172.21.15.47:0/1777233305 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f28580573d0 con 0x7f28580619b0 2022-06-29T15:50:59.532 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:59.527+0000 7f2855ffb700 1 -- 172.21.15.47:0/1777233305 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v126) v1 ==== 133+0+0 (secure 0 0 0) 0x7f284c023020 con 0x7f28580619b0 2022-06-29T15:50:59.537 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:59.531+0000 7f285ef02700 1 -- 172.21.15.47:0/1777233305 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2838053aa0 msgr2=0x7f2838055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:59.537 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:59.531+0000 7f285ef02700 1 --2- 172.21.15.47:0/1777233305 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2838053aa0 0x7f2838055f60 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7f284800b000 tx=0x7f2848006df0).stop 2022-06-29T15:50:59.537 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:59.531+0000 7f285ef02700 1 -- 172.21.15.47:0/1777233305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28580619b0 msgr2=0x7f2858141ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:59.538 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:59.531+0000 7f285ef02700 1 --2- 172.21.15.47:0/1777233305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28580619b0 0x7f2858141ce0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 rx=0x7f284c002f70 tx=0x7f284c014000).stop 2022-06-29T15:50:59.538 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:59.531+0000 7f285ef02700 1 -- 172.21.15.47:0/1777233305 shutdown_connections 2022-06-29T15:50:59.538 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:59.531+0000 7f285ef02700 1 --2- 172.21.15.47:0/1777233305 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2838053aa0 0x7f2838055f60 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:59.538 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:59.531+0000 7f285ef02700 1 --2- 172.21.15.47:0/1777233305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28580b1e20 0x7f2858141420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:59.539 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:59.531+0000 7f285ef02700 1 --2- 172.21.15.47:0/1777233305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28580619b0 0x7f2858141ce0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:59.539 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:59.531+0000 7f285ef02700 1 --2- 172.21.15.47:0/1777233305 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f285805f0b0 0x7f28580b6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:59.539 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:59.531+0000 7f285ef02700 1 -- 172.21.15.47:0/1777233305 >> 172.21.15.47:0/1777233305 conn(0x7f2858063800 msgr2=0x7f28580adf70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:59.539 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:59.531+0000 7f285ef02700 1 -- 172.21.15.47:0/1777233305 shutdown_connections 2022-06-29T15:50:59.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:50:59.531+0000 7f285ef02700 1 -- 172.21.15.47:0/1777233305 wait complete. 2022-06-29T15:50:59.540 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' created 2022-06-29T15:50:59.552 DEBUG:teuthology.orchestra.run.smithi099:> sudo rados -p cephfs_metadata put key /etc/fstab 2022-06-29T15:50:59.595 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.586+0000 7f115cb87b80 1 Processor -- start 2022-06-29T15:50:59.596 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.586+0000 7f115cb87b80 1 -- start start 2022-06-29T15:50:59.596 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.586+0000 7f115cb87b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5632ef2748b0 0x5632ef282c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:59.596 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.586+0000 7f115cb87b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5632ef275250 0x5632ef283120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:59.596 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.586+0000 7f115cb87b80 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x5632ef27aba0 0x5632ef27d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:59.597 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.586+0000 7f115cb87b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632ef167460 con 0x5632ef2748b0 2022-06-29T15:50:59.597 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.586+0000 7f115cb87b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632ef27df20 con 0x5632ef275250 2022-06-29T15:50:59.597 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.586+0000 7f115cb87b80 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x5632ef27e0e0 con 0x5632ef27aba0 2022-06-29T15:50:59.597 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.586+0000 7f11527c3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5632ef275250 0x5632ef283120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:59.598 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.586+0000 7f115afc4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5632ef2748b0 0x5632ef282c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:59.598 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.586+0000 7f115b7c5700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x5632ef27aba0 0x5632ef27d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:59.598 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.586+0000 7f11527c3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5632ef275250 0x5632ef283120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.99:37852/0 (socket says 172.21.15.99:37852) 2022-06-29T15:50:59.598 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.586+0000 7f11527c3700 1 -- 172.21.15.99:0/2518851799 learned_addr learned my addr 172.21.15.99:0/2518851799 (peer_addr_for_me v2:172.21.15.99:0/0) 2022-06-29T15:50:59.603 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.586+0000 7f115afc4700 1 -- 172.21.15.99:0/2518851799 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x5632ef27aba0 msgr2=0x5632ef27d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:59.604 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.586+0000 7f115afc4700 1 --2- 172.21.15.99:0/2518851799 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x5632ef27aba0 0x5632ef27d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:59.604 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.586+0000 7f115afc4700 1 -- 172.21.15.99:0/2518851799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5632ef275250 msgr2=0x5632ef283120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:50:59.604 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.586+0000 7f115afc4700 1 --2- 172.21.15.99:0/2518851799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5632ef275250 0x5632ef283120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:59.604 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.586+0000 7f115afc4700 1 -- 172.21.15.99:0/2518851799 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f114c005cd0 con 0x5632ef2748b0 2022-06-29T15:50:59.604 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.586+0000 7f11527c3700 1 --2- 172.21.15.99:0/2518851799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5632ef275250 0x5632ef283120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:50:59.605 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.586+0000 7f115afc4700 1 --2- 172.21.15.99:0/2518851799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5632ef2748b0 0x5632ef282c30 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 rx=0x7f114c002f70 tx=0x7f114c013000).ready entity=mon.0 client_cookie=128370541c79c8ec server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:59.605 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.586+0000 7f11597c1700 1 -- 172.21.15.99:0/2518851799 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f114c01b070 con 0x5632ef2748b0 2022-06-29T15:50:59.605 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.586+0000 7f11597c1700 1 -- 172.21.15.99:0/2518851799 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f114c008420 con 0x5632ef2748b0 2022-06-29T15:50:59.605 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.586+0000 7f11597c1700 1 -- 172.21.15.99:0/2518851799 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f114c008780 con 0x5632ef2748b0 2022-06-29T15:50:59.606 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.586+0000 7f115cb87b80 1 -- 172.21.15.99:0/2518851799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5632ef2748b0 msgr2=0x5632ef282c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:59.606 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.586+0000 7f115cb87b80 1 --2- 172.21.15.99:0/2518851799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5632ef2748b0 0x5632ef282c30 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 rx=0x7f114c002f70 tx=0x7f114c013000).stop 2022-06-29T15:50:59.606 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.586+0000 7f115cb87b80 1 -- 172.21.15.99:0/2518851799 shutdown_connections 2022-06-29T15:50:59.606 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.586+0000 7f115cb87b80 1 --2- 172.21.15.99:0/2518851799 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x5632ef27aba0 0x5632ef27d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:59.607 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.586+0000 7f115cb87b80 1 --2- 172.21.15.99:0/2518851799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5632ef275250 0x5632ef283120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:59.607 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.586+0000 7f115cb87b80 1 --2- 172.21.15.99:0/2518851799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5632ef2748b0 0x5632ef282c30 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:59.607 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.586+0000 7f115cb87b80 1 -- 172.21.15.99:0/2518851799 >> 172.21.15.99:0/2518851799 conn(0x5632ef270420 msgr2=0x5632ef277590 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:50:59.607 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.590+0000 7f115cb87b80 1 -- 172.21.15.99:0/2518851799 shutdown_connections 2022-06-29T15:50:59.607 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.590+0000 7f115cb87b80 1 -- 172.21.15.99:0/2518851799 wait complete. 2022-06-29T15:50:59.608 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.590+0000 7f115cb87b80 1 Processor -- start 2022-06-29T15:50:59.608 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.590+0000 7f115cb87b80 1 -- start start 2022-06-29T15:50:59.608 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.590+0000 7f115cb87b80 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x5632ef2748b0 0x5632ef288ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:59.608 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.590+0000 7f115cb87b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5632ef275250 0x5632ef289190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:59.609 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.590+0000 7f115cb87b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5632ef27aba0 0x5632ef283e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:59.609 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.590+0000 7f115cb87b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632ef272b90 con 0x5632ef275250 2022-06-29T15:50:59.609 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.590+0000 7f115cb87b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632ef284570 con 0x5632ef27aba0 2022-06-29T15:50:59.609 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.590+0000 7f115cb87b80 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x5632ef284730 con 0x5632ef2748b0 2022-06-29T15:50:59.609 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.590+0000 7f11527c3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5632ef275250 0x5632ef289190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:59.610 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.590+0000 7f115afc4700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x5632ef2748b0 0x5632ef288ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:59.610 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.590+0000 7f115b7c5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5632ef27aba0 0x5632ef283e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:59.610 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.590+0000 7f115afc4700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x5632ef2748b0 0x5632ef288ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.99:38042/0 (socket says 172.21.15.99:38042) 2022-06-29T15:50:59.610 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.590+0000 7f115afc4700 1 -- 172.21.15.99:0/4033227929 learned_addr learned my addr 172.21.15.99:0/4033227929 (peer_addr_for_me v2:172.21.15.99:0/0) 2022-06-29T15:50:59.611 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.590+0000 7f115b7c5700 1 -- 172.21.15.99:0/4033227929 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x5632ef2748b0 msgr2=0x5632ef288ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:59.611 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.590+0000 7f115b7c5700 1 --2- 172.21.15.99:0/4033227929 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x5632ef2748b0 0x5632ef288ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:59.611 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.590+0000 7f115b7c5700 1 -- 172.21.15.99:0/4033227929 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5632ef275250 msgr2=0x5632ef289190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:50:59.612 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.590+0000 7f115b7c5700 1 --2- 172.21.15.99:0/4033227929 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5632ef275250 0x5632ef289190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:50:59.612 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.590+0000 7f115b7c5700 1 -- 172.21.15.99:0/4033227929 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11440094f0 con 0x5632ef27aba0 2022-06-29T15:50:59.612 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.590+0000 7f115b7c5700 1 --2- 172.21.15.99:0/4033227929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5632ef27aba0 0x5632ef283e00 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7f115400c000 tx=0x7f11540069f0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:59.613 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.590+0000 7f1158fc0700 1 -- 172.21.15.99:0/4033227929 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1154002010 con 0x5632ef27aba0 2022-06-29T15:50:59.613 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.590+0000 7f1158fc0700 1 -- 172.21.15.99:0/4033227929 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1154011350 con 0x5632ef27aba0 2022-06-29T15:50:59.614 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.590+0000 7f115cb87b80 1 -- 172.21.15.99:0/4033227929 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5632ef18f7e0 con 0x5632ef27aba0 2022-06-29T15:50:59.614 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.590+0000 7f1158fc0700 1 -- 172.21.15.99:0/4033227929 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11540116b0 con 0x5632ef27aba0 2022-06-29T15:50:59.615 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.590+0000 7f115cb87b80 1 -- 172.21.15.99:0/4033227929 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5632ef18c1d0 con 0x5632ef27aba0 2022-06-29T15:50:59.617 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.594+0000 7f1158fc0700 1 -- 172.21.15.99:0/4033227929 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f115401a030 con 0x5632ef27aba0 2022-06-29T15:50:59.618 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.594+0000 7f1158fc0700 1 --2- 172.21.15.99:0/4033227929 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f113c0525f0 0x7f113c054ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:59.618 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.594+0000 7f1158fc0700 1 -- 172.21.15.99:0/4033227929 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6422+0+0 (secure 0 0 0) 0x7f115406a5f0 con 0x5632ef27aba0 2022-06-29T15:50:59.618 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.594+0000 7f115cb87b80 1 --2- 172.21.15.99:0/4033227929 >> [v2:172.21.15.47:6818/13096,v1:172.21.15.47:6819/13096] conn(0x5632ef2958e0 0x5632ef297cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:50:59.618 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.594+0000 7f115afc4700 1 --2- 172.21.15.99:0/4033227929 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f113c0525f0 0x7f113c054ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:59.619 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.594+0000 7f115cb87b80 1 -- 172.21.15.99:0/4033227929 --> [v2:172.21.15.47:6818/13096,v1:172.21.15.47:6819/13096] -- osd_op(unknown.0.0:1 23.0 23:0620e6f9:::key:head [writefull 0~617 in=617b] snapc 0=[] ondisk+write+known_if_redirected e126) v8 -- 0x5632ef298260 con 0x5632ef2958e0 2022-06-29T15:50:59.619 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.594+0000 7f11527c3700 1 --2- 172.21.15.99:0/4033227929 >> [v2:172.21.15.47:6818/13096,v1:172.21.15.47:6819/13096] conn(0x5632ef2958e0 0x5632ef297cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:50:59.619 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.598+0000 7f115afc4700 1 --2- 172.21.15.99:0/4033227929 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f113c0525f0 0x7f113c054ab0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7f114c022000 tx=0x7f114c01b000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:50:59.619 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:50:59.598+0000 7f11527c3700 1 --2- 172.21.15.99:0/4033227929 >> [v2:172.21.15.47:6818/13096,v1:172.21.15.47:6819/13096] conn(0x5632ef2958e0 0x5632ef297cd0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:00.543 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:51:00.538+0000 7f1158fc0700 1 -- 172.21.15.99:0/4033227929 <== osd.0 v2:172.21.15.47:6818/13096 1 ==== osd_map(127..127 src has 1..127) v4 ==== 320+0+0 (crc 0 0 0) 0x7f11440085d0 con 0x5632ef2958e0 2022-06-29T15:51:00.549 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:51:00.542+0000 7f11527c3700 1 -- 172.21.15.99:0/4033227929 <== osd.0 v2:172.21.15.47:6818/13096 2 ==== osd_op_reply(1 key [writefull 0~617] v127'1 uv1 ondisk = 0) v8 ==== 147+0+0 (crc 0 0 0) 0x7f1144008790 con 0x5632ef2958e0 2022-06-29T15:51:00.549 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:51:00.542+0000 7f115cb87b80 1 -- 172.21.15.99:0/4033227929 >> [v2:172.21.15.47:6818/13096,v1:172.21.15.47:6819/13096] conn(0x5632ef2958e0 msgr2=0x5632ef297cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:00.550 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:51:00.542+0000 7f115cb87b80 1 --2- 172.21.15.99:0/4033227929 >> [v2:172.21.15.47:6818/13096,v1:172.21.15.47:6819/13096] conn(0x5632ef2958e0 0x5632ef297cd0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:00.550 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:51:00.542+0000 7f115cb87b80 1 -- 172.21.15.99:0/4033227929 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f113c0525f0 msgr2=0x7f113c054ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:00.550 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:51:00.542+0000 7f115cb87b80 1 --2- 172.21.15.99:0/4033227929 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f113c0525f0 0x7f113c054ab0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7f114c022000 tx=0x7f114c01b000).stop 2022-06-29T15:51:00.551 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:51:00.542+0000 7f115cb87b80 1 -- 172.21.15.99:0/4033227929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5632ef27aba0 msgr2=0x5632ef283e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:00.551 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:51:00.542+0000 7f115cb87b80 1 --2- 172.21.15.99:0/4033227929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5632ef27aba0 0x5632ef283e00 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7f115400c000 tx=0x7f11540069f0).stop 2022-06-29T15:51:00.551 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:51:00.542+0000 7f115b7c5700 1 -- 172.21.15.99:0/4033227929 reap_dead start 2022-06-29T15:51:00.552 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:51:00.542+0000 7f115cb87b80 1 -- 172.21.15.99:0/4033227929 shutdown_connections 2022-06-29T15:51:00.552 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:51:00.542+0000 7f115cb87b80 1 -- 172.21.15.99:0/4033227929 >> 172.21.15.99:0/4033227929 conn(0x5632ef270420 msgr2=0x5632ef277590 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:00.552 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:51:00.542+0000 7f115cb87b80 1 -- 172.21.15.99:0/4033227929 shutdown_connections 2022-06-29T15:51:00.553 INFO:teuthology.orchestra.run.smithi099.stderr:2022-06-29T15:51:00.542+0000 7f115cb87b80 1 -- 172.21.15.99:0/4033227929 wait complete. 2022-06-29T15:51:00.554 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph df --format=json-pretty 2022-06-29T15:51:00.652 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.647+0000 7f2d3661e700 1 Processor -- start 2022-06-29T15:51:00.652 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.647+0000 7f2d3661e700 1 -- start start 2022-06-29T15:51:00.654 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.647+0000 7f2d3661e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2d300b1c60 0x7f2d3005f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:00.654 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.647+0000 7f2d3661e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d300609a0 con 0x7f2d300b2900 2022-06-29T15:51:00.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.647+0000 7f2d3661e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d300b0fe0 con 0x7f2d3005fc10 2022-06-29T15:51:00.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.647+0000 7f2d3661e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d300b2ce0 con 0x7f2d300b1c60 2022-06-29T15:51:00.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.647+0000 7f2d34e1b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2d300b1c60 0x7f2d3005f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:00.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.647+0000 7f2d3561c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f2d300b2900 0x7f2d30099d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38688/0 (socket says 172.21.15.47:38688) 2022-06-29T15:51:00.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.647+0000 7f2d3561c700 1 -- 172.21.15.47:0/4283760998 learned_addr learned my addr 172.21.15.47:0/4283760998 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:00.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.647+0000 7f2d34e1b700 1 -- 172.21.15.47:0/4283760998 >> v1:172.21.15.64:6789/0 conn(0x7f2d3005fc10 legacy=0x7f2d3014da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:00.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.647+0000 7f2d34e1b700 1 -- 172.21.15.47:0/4283760998 >> v1:172.21.15.47:6789/0 conn(0x7f2d300b2900 legacy=0x7f2d30099d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:00.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.647+0000 7f2d1bfff700 1 -- 172.21.15.47:0/4283760998 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1929813903 0 0) 0x7f2d300609a0 con 0x7f2d300b2900 2022-06-29T15:51:00.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.647+0000 7f2d34e1b700 1 -- 172.21.15.47:0/4283760998 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d300b0210 con 0x7f2d300b1c60 2022-06-29T15:51:00.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.647+0000 7f2d34e1b700 1 --2- 172.21.15.47:0/4283760998 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d300b1c60 0x7f2d3005f370 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 rx=0x7f2d24002f70 tx=0x7f2d24006d90).ready entity=mon.0 client_cookie=214392cd31a7fbad server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:00.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.647+0000 7f2d1bfff700 1 -- 172.21.15.47:0/4283760998 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d2400f6d0 con 0x7f2d300b1c60 2022-06-29T15:51:00.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.647+0000 7f2d1bfff700 1 -- 172.21.15.47:0/4283760998 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d24000e90 con 0x7f2d300b1c60 2022-06-29T15:51:00.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.647+0000 7f2d1bfff700 1 -- 172.21.15.47:0/4283760998 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d2400fa20 con 0x7f2d300b1c60 2022-06-29T15:51:00.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.647+0000 7f2d3661e700 1 -- 172.21.15.47:0/4283760998 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d300b1c60 msgr2=0x7f2d3005f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:00.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.647+0000 7f2d3661e700 1 --2- 172.21.15.47:0/4283760998 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d300b1c60 0x7f2d3005f370 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 rx=0x7f2d24002f70 tx=0x7f2d24006d90).stop 2022-06-29T15:51:00.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.647+0000 7f2d3661e700 1 -- 172.21.15.47:0/4283760998 shutdown_connections 2022-06-29T15:51:00.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.647+0000 7f2d3661e700 1 --2- 172.21.15.47:0/4283760998 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d300b1c60 0x7f2d3005f370 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:00.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.647+0000 7f2d3661e700 1 -- 172.21.15.47:0/4283760998 >> 172.21.15.47:0/4283760998 conn(0x7f2d30062df0 msgr2=0x7f2d300631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:00.663 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.651+0000 7f2d3661e700 1 -- 172.21.15.47:0/4283760998 shutdown_connections 2022-06-29T15:51:00.663 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.651+0000 7f2d3661e700 1 -- 172.21.15.47:0/4283760998 wait complete. 2022-06-29T15:51:00.663 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.651+0000 7f2d3661e700 1 Processor -- start 2022-06-29T15:51:00.663 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.651+0000 7f2d3661e700 1 -- start start 2022-06-29T15:51:00.664 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.651+0000 7f2d3661e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d3005fc10 0x7f2d300aefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:00.664 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.651+0000 7f2d3661e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d300b1c60 0x7f2d30156230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:00.664 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.651+0000 7f2d3661e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2d300b2900 0x7f2d30156720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:00.664 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.651+0000 7f2d3661e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d30096740 con 0x7f2d3005fc10 2022-06-29T15:51:00.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.651+0000 7f2d3661e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d30155d90 con 0x7f2d300b1c60 2022-06-29T15:51:00.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.651+0000 7f2d3661e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d30065830 con 0x7f2d300b2900 2022-06-29T15:51:00.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.651+0000 7f2d3561c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d3005fc10 0x7f2d300aefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:00.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.651+0000 7f2d3561c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d3005fc10 0x7f2d300aefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53140/0 (socket says 172.21.15.47:53140) 2022-06-29T15:51:00.666 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.651+0000 7f2d3561c700 1 -- 172.21.15.47:0/1693609853 learned_addr learned my addr 172.21.15.47:0/1693609853 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:00.666 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.651+0000 7f2d34e1b700 1 --2- 172.21.15.47:0/1693609853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d300b1c60 0x7f2d30156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:00.666 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.651+0000 7f2d35e1d700 1 --2- 172.21.15.47:0/1693609853 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2d300b2900 0x7f2d30156720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:00.666 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.651+0000 7f2d3561c700 1 -- 172.21.15.47:0/1693609853 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2d300b2900 msgr2=0x7f2d30156720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:00.666 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.651+0000 7f2d3561c700 1 --2- 172.21.15.47:0/1693609853 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2d300b2900 0x7f2d30156720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:00.667 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.651+0000 7f2d3561c700 1 -- 172.21.15.47:0/1693609853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d300b1c60 msgr2=0x7f2d30156230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:00.667 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.651+0000 7f2d3561c700 1 --2- 172.21.15.47:0/1693609853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d300b1c60 0x7f2d30156230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:00.667 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.651+0000 7f2d3561c700 1 -- 172.21.15.47:0/1693609853 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d2400b120 con 0x7f2d3005fc10 2022-06-29T15:51:00.667 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.651+0000 7f2d35e1d700 1 --2- 172.21.15.47:0/1693609853 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2d300b2900 0x7f2d30156720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:51:00.668 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.655+0000 7f2d3561c700 1 --2- 172.21.15.47:0/1693609853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d3005fc10 0x7f2d300aefd0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 rx=0x7f2d2c000f40 tx=0x7f2d2c0013f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:00.668 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.655+0000 7f2d22ffd700 1 -- 172.21.15.47:0/1693609853 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d2c005ba0 con 0x7f2d3005fc10 2022-06-29T15:51:00.668 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.655+0000 7f2d22ffd700 1 -- 172.21.15.47:0/1693609853 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d2c005d00 con 0x7f2d3005fc10 2022-06-29T15:51:00.668 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.655+0000 7f2d3661e700 1 -- 172.21.15.47:0/1693609853 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d30095e20 con 0x7f2d3005fc10 2022-06-29T15:51:00.669 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.655+0000 7f2d22ffd700 1 -- 172.21.15.47:0/1693609853 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d2c0072c0 con 0x7f2d3005fc10 2022-06-29T15:51:00.669 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.655+0000 7f2d3661e700 1 -- 172.21.15.47:0/1693609853 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d30095bc0 con 0x7f2d3005fc10 2022-06-29T15:51:00.669 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.655+0000 7f2d22ffd700 1 -- 172.21.15.47:0/1693609853 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2d2c01a060 con 0x7f2d3005fc10 2022-06-29T15:51:00.669 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.655+0000 7f2d3661e700 1 -- 172.21.15.47:0/1693609853 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d300573d0 con 0x7f2d3005fc10 2022-06-29T15:51:00.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.655+0000 7f2d22ffd700 1 --2- 172.21.15.47:0/1693609853 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2d0c053a10 0x7f2d0c055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:00.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.655+0000 7f2d22ffd700 1 -- 172.21.15.47:0/1693609853 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6422+0+0 (secure 0 0 0) 0x7f2d2c06b820 con 0x7f2d3005fc10 2022-06-29T15:51:00.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.655+0000 7f2d34e1b700 1 --2- 172.21.15.47:0/1693609853 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2d0c053a10 0x7f2d0c055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:00.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.655+0000 7f2d34e1b700 1 --2- 172.21.15.47:0/1693609853 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2d0c053a10 0x7f2d0c055ed0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7f2d24019000 tx=0x7f2d24015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:00.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.663+0000 7f2d22ffd700 1 -- 172.21.15.47:0/1693609853 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2d2c0424d0 con 0x7f2d3005fc10 2022-06-29T15:51:00.835 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.827+0000 7f2d3661e700 1 -- 172.21.15.47:0/1693609853 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "df", "format": "json-pretty"} v 0) v1 -- 0x7f2d300a9720 con 0x7f2d3005fc10 2022-06-29T15:51:00.838 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.831+0000 7f2d22ffd700 1 -- 172.21.15.47:0/1693609853 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "df", "format": "json-pretty"}]=0 v0) v1 ==== 75+0+1586 (secure 0 0 0) 0x7f2d2c018020 con 0x7f2d3005fc10 2022-06-29T15:51:00.838 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:00.838 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2022-06-29T15:51:00.839 INFO:teuthology.orchestra.run.smithi047.stdout: "stats": { 2022-06-29T15:51:00.839 INFO:teuthology.orchestra.run.smithi047.stdout: "total_bytes": 773094113280, 2022-06-29T15:51:00.840 INFO:teuthology.orchestra.run.smithi047.stdout: "total_avail_bytes": 764491886592, 2022-06-29T15:51:00.840 INFO:teuthology.orchestra.run.smithi047.stdout: "total_used_bytes": 12292096, 2022-06-29T15:51:00.840 INFO:teuthology.orchestra.run.smithi047.stdout: "total_used_raw_bytes": 8602226688, 2022-06-29T15:51:00.841 INFO:teuthology.orchestra.run.smithi047.stdout: "total_used_raw_ratio": 0.01112701091915369, 2022-06-29T15:51:00.841 INFO:teuthology.orchestra.run.smithi047.stdout: "num_osds": 8, 2022-06-29T15:51:00.841 INFO:teuthology.orchestra.run.smithi047.stdout: "num_per_pool_osds": 8, 2022-06-29T15:51:00.841 INFO:teuthology.orchestra.run.smithi047.stdout: "num_per_pool_omap_osds": 8 2022-06-29T15:51:00.842 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:00.842 INFO:teuthology.orchestra.run.smithi047.stdout: "stats_by_class": { 2022-06-29T15:51:00.842 INFO:teuthology.orchestra.run.smithi047.stdout: "ssd": { 2022-06-29T15:51:00.842 INFO:teuthology.orchestra.run.smithi047.stdout: "total_bytes": 773094113280, 2022-06-29T15:51:00.843 INFO:teuthology.orchestra.run.smithi047.stdout: "total_avail_bytes": 764491886592, 2022-06-29T15:51:00.843 INFO:teuthology.orchestra.run.smithi047.stdout: "total_used_bytes": 12292096, 2022-06-29T15:51:00.843 INFO:teuthology.orchestra.run.smithi047.stdout: "total_used_raw_bytes": 8602226688, 2022-06-29T15:51:00.843 INFO:teuthology.orchestra.run.smithi047.stdout: "total_used_raw_ratio": 0.01112701091915369 2022-06-29T15:51:00.844 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:00.844 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:00.844 INFO:teuthology.orchestra.run.smithi047.stdout: "pools": [ 2022-06-29T15:51:00.844 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:00.844 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "device_health_metrics", 2022-06-29T15:51:00.845 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 1, 2022-06-29T15:51:00.845 INFO:teuthology.orchestra.run.smithi047.stdout: "stats": { 2022-06-29T15:51:00.845 INFO:teuthology.orchestra.run.smithi047.stdout: "stored": 0, 2022-06-29T15:51:00.845 INFO:teuthology.orchestra.run.smithi047.stdout: "objects": 2, 2022-06-29T15:51:00.846 INFO:teuthology.orchestra.run.smithi047.stdout: "kb_used": 0, 2022-06-29T15:51:00.846 INFO:teuthology.orchestra.run.smithi047.stdout: "bytes_used": 0, 2022-06-29T15:51:00.846 INFO:teuthology.orchestra.run.smithi047.stdout: "percent_used": 0, 2022-06-29T15:51:00.847 INFO:teuthology.orchestra.run.smithi047.stdout: "max_avail": 362918477824 2022-06-29T15:51:00.847 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:00.847 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:00.847 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:00.848 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd", 2022-06-29T15:51:00.848 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 2, 2022-06-29T15:51:00.848 INFO:teuthology.orchestra.run.smithi047.stdout: "stats": { 2022-06-29T15:51:00.849 INFO:teuthology.orchestra.run.smithi047.stdout: "stored": 0, 2022-06-29T15:51:00.849 INFO:teuthology.orchestra.run.smithi047.stdout: "objects": 0, 2022-06-29T15:51:00.849 INFO:teuthology.orchestra.run.smithi047.stdout: "kb_used": 0, 2022-06-29T15:51:00.850 INFO:teuthology.orchestra.run.smithi047.stdout: "bytes_used": 0, 2022-06-29T15:51:00.850 INFO:teuthology.orchestra.run.smithi047.stdout: "percent_used": 0, 2022-06-29T15:51:00.850 INFO:teuthology.orchestra.run.smithi047.stdout: "max_avail": 362918477824 2022-06-29T15:51:00.851 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:00.851 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:00.851 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:00.851 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cephfs_data", 2022-06-29T15:51:00.852 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 22, 2022-06-29T15:51:00.852 INFO:teuthology.orchestra.run.smithi047.stdout: "stats": { 2022-06-29T15:51:00.852 INFO:teuthology.orchestra.run.smithi047.stdout: "stored": 0, 2022-06-29T15:51:00.853 INFO:teuthology.orchestra.run.smithi047.stdout: "objects": 0, 2022-06-29T15:51:00.853 INFO:teuthology.orchestra.run.smithi047.stdout: "kb_used": 0, 2022-06-29T15:51:00.853 INFO:teuthology.orchestra.run.smithi047.stdout: "bytes_used": 0, 2022-06-29T15:51:00.853 INFO:teuthology.orchestra.run.smithi047.stdout: "percent_used": 0, 2022-06-29T15:51:00.854 INFO:teuthology.orchestra.run.smithi047.stdout: "max_avail": 362918477824 2022-06-29T15:51:00.854 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:00.854 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:00.854 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:00.854 INFO:teuthology.orchestra.run.smithi047.stdout:} 2022-06-29T15:51:00.857 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.835+0000 7f2d3661e700 1 -- 172.21.15.47:0/1693609853 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2d0c053a10 msgr2=0x7f2d0c055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:00.857 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.835+0000 7f2d3661e700 1 --2- 172.21.15.47:0/1693609853 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2d0c053a10 0x7f2d0c055ed0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7f2d24019000 tx=0x7f2d24015000).stop 2022-06-29T15:51:00.857 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.835+0000 7f2d3661e700 1 -- 172.21.15.47:0/1693609853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d3005fc10 msgr2=0x7f2d300aefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:00.858 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.835+0000 7f2d3661e700 1 --2- 172.21.15.47:0/1693609853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d3005fc10 0x7f2d300aefd0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 rx=0x7f2d2c000f40 tx=0x7f2d2c0013f0).stop 2022-06-29T15:51:00.858 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.835+0000 7f2d3661e700 1 -- 172.21.15.47:0/1693609853 shutdown_connections 2022-06-29T15:51:00.858 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.835+0000 7f2d3661e700 1 --2- 172.21.15.47:0/1693609853 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2d300b2900 0x7f2d30156720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:00.859 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.835+0000 7f2d3661e700 1 --2- 172.21.15.47:0/1693609853 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2d0c053a10 0x7f2d0c055ed0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:00.859 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.835+0000 7f2d3661e700 1 --2- 172.21.15.47:0/1693609853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d300b1c60 0x7f2d30156230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:00.859 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.835+0000 7f2d3661e700 1 --2- 172.21.15.47:0/1693609853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d3005fc10 0x7f2d300aefd0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:00.859 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.835+0000 7f2d3661e700 1 -- 172.21.15.47:0/1693609853 >> 172.21.15.47:0/1693609853 conn(0x7f2d30062df0 msgr2=0x7f2d300b4c00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:00.860 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.835+0000 7f2d3661e700 1 -- 172.21.15.47:0/1693609853 shutdown_connections 2022-06-29T15:51:00.860 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:00.835+0000 7f2d3661e700 1 -- 172.21.15.47:0/1693609853 wait complete. 2022-06-29T15:51:00.861 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30)... 2022-06-29T15:51:05.863 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph df --format=json-pretty 2022-06-29T15:51:05.971 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.963+0000 7f41355b6700 1 Processor -- start 2022-06-29T15:51:05.972 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.963+0000 7f41355b6700 1 -- start start 2022-06-29T15:51:05.976 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.967+0000 7f41355b6700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f41300b64c0 0x7f41300b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:05.976 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.967+0000 7f41355b6700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41300b68a0 con 0x7f4130143150 2022-06-29T15:51:05.976 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.967+0000 7f41355b6700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41301473d0 con 0x7f4130066fe0 2022-06-29T15:51:05.976 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.967+0000 7f41355b6700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41300b13a0 con 0x7f41300b64c0 2022-06-29T15:51:05.977 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.967+0000 7f4134db5700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4130143150 0x7f413012bec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38706/0 (socket says 172.21.15.47:38706) 2022-06-29T15:51:05.977 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.967+0000 7f4134db5700 1 -- 172.21.15.47:0/2704935791 learned_addr learned my addr 172.21.15.47:0/2704935791 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:05.977 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.967+0000 7f412ffff700 1 --2- 172.21.15.47:0/2704935791 >> v2:172.21.15.47:3300/0 conn(0x7f41300b64c0 0x7f41300b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:05.977 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.967+0000 7f412effd700 1 -- 172.21.15.47:0/2704935791 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2765024598 0 0) 0x7f41300b68a0 con 0x7f4130143150 2022-06-29T15:51:05.978 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.967+0000 7f412effd700 1 -- 172.21.15.47:0/2704935791 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f411c003260 con 0x7f4130143150 2022-06-29T15:51:05.978 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.967+0000 7f412effd700 1 -- 172.21.15.47:0/2704935791 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 495028860 0 0) 0x7f41301473d0 con 0x7f4130066fe0 2022-06-29T15:51:05.978 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.967+0000 7f412effd700 1 -- 172.21.15.47:0/2704935791 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41300b68a0 con 0x7f4130066fe0 2022-06-29T15:51:05.979 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.967+0000 7f412ffff700 1 -- 172.21.15.47:0/2704935791 >> v1:172.21.15.64:6789/0 conn(0x7f4130066fe0 legacy=0x7f41300b4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:05.979 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.967+0000 7f412ffff700 1 -- 172.21.15.47:0/2704935791 >> v1:172.21.15.47:6789/0 conn(0x7f4130143150 legacy=0x7f413012bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:05.979 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.967+0000 7f412effd700 1 -- 172.21.15.47:0/2704935791 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3136790586 0 0) 0x7f411c003260 con 0x7f4130143150 2022-06-29T15:51:05.979 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.967+0000 7f412ffff700 1 -- 172.21.15.47:0/2704935791 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41300b46a0 con 0x7f41300b64c0 2022-06-29T15:51:05.980 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.967+0000 7f412ffff700 1 --2- 172.21.15.47:0/2704935791 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41300b64c0 0x7f41300b5a20 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 rx=0x7f4128001490 tx=0x7f4124006df0).ready entity=mon.0 client_cookie=5c065810d8e78ce8 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:05.980 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.967+0000 7f412effd700 1 -- 172.21.15.47:0/2704935791 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f41280019e0 con 0x7f41300b64c0 2022-06-29T15:51:05.980 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.967+0000 7f412effd700 1 -- 172.21.15.47:0/2704935791 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4124000e90 con 0x7f41300b64c0 2022-06-29T15:51:05.980 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.967+0000 7f412effd700 1 -- 172.21.15.47:0/2704935791 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4124012460 con 0x7f41300b64c0 2022-06-29T15:51:05.981 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.967+0000 7f41355b6700 1 -- 172.21.15.47:0/2704935791 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41300b64c0 msgr2=0x7f41300b5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:05.981 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.967+0000 7f41355b6700 1 --2- 172.21.15.47:0/2704935791 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41300b64c0 0x7f41300b5a20 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 rx=0x7f4128001490 tx=0x7f4124006df0).stop 2022-06-29T15:51:05.981 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.967+0000 7f41355b6700 1 -- 172.21.15.47:0/2704935791 shutdown_connections 2022-06-29T15:51:05.981 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.967+0000 7f41355b6700 1 --2- 172.21.15.47:0/2704935791 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41300b64c0 0x7f41300b5a20 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:05.982 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.967+0000 7f41355b6700 1 -- 172.21.15.47:0/2704935791 >> 172.21.15.47:0/2704935791 conn(0x7f4130060e00 msgr2=0x7f41300611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:05.982 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.967+0000 7f41355b6700 1 -- 172.21.15.47:0/2704935791 shutdown_connections 2022-06-29T15:51:05.982 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.967+0000 7f41355b6700 1 -- 172.21.15.47:0/2704935791 wait complete. 2022-06-29T15:51:05.986 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.971+0000 7f41355b6700 1 Processor -- start 2022-06-29T15:51:05.986 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.971+0000 7f41355b6700 1 -- start start 2022-06-29T15:51:05.986 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.971+0000 7f41355b6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4130066fe0 0x7f4130141330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:05.987 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.971+0000 7f41355b6700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f41300b64c0 0x7f4130141820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:05.987 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.971+0000 7f41355b6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4130143150 0x7f4130140260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:05.987 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.971+0000 7f41355b6700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4130149690 con 0x7f4130143150 2022-06-29T15:51:05.987 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.971+0000 7f41355b6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4130152bc0 con 0x7f4130066fe0 2022-06-29T15:51:05.988 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.971+0000 7f41355b6700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4130152e60 con 0x7f41300b64c0 2022-06-29T15:51:05.988 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.971+0000 7f4134db5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4130143150 0x7f4130140260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:05.988 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.971+0000 7f4134db5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4130143150 0x7f4130140260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53158/0 (socket says 172.21.15.47:53158) 2022-06-29T15:51:05.988 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.971+0000 7f4134db5700 1 -- 172.21.15.47:0/685860834 learned_addr learned my addr 172.21.15.47:0/685860834 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:05.989 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.971+0000 7f412f7fe700 1 --2- 172.21.15.47:0/685860834 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f41300b64c0 0x7f4130141820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:05.989 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.971+0000 7f412ffff700 1 --2- 172.21.15.47:0/685860834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4130066fe0 0x7f4130141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:05.989 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.971+0000 7f4134db5700 1 -- 172.21.15.47:0/685860834 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f41300b64c0 msgr2=0x7f4130141820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:05.989 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.971+0000 7f4134db5700 1 --2- 172.21.15.47:0/685860834 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f41300b64c0 0x7f4130141820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:05.990 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.971+0000 7f4134db5700 1 -- 172.21.15.47:0/685860834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4130066fe0 msgr2=0x7f4130141330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:05.990 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.971+0000 7f4134db5700 1 --2- 172.21.15.47:0/685860834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4130066fe0 0x7f4130141330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:05.990 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.971+0000 7f4134db5700 1 -- 172.21.15.47:0/685860834 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f412400b180 con 0x7f4130143150 2022-06-29T15:51:05.990 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.971+0000 7f412ffff700 1 --2- 172.21.15.47:0/685860834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4130066fe0 0x7f4130141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:51:05.991 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.971+0000 7f4134db5700 1 --2- 172.21.15.47:0/685860834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4130143150 0x7f4130140260 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 rx=0x7f4128001360 tx=0x7f4128001ab0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:05.991 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.971+0000 7f412d7fa700 1 -- 172.21.15.47:0/685860834 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4128004ab0 con 0x7f4130143150 2022-06-29T15:51:05.991 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.971+0000 7f412d7fa700 1 -- 172.21.15.47:0/685860834 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4128004c10 con 0x7f4130143150 2022-06-29T15:51:05.991 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.971+0000 7f41355b6700 1 -- 172.21.15.47:0/685860834 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41300b2780 con 0x7f4130143150 2022-06-29T15:51:05.992 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.971+0000 7f412d7fa700 1 -- 172.21.15.47:0/685860834 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4128007170 con 0x7f4130143150 2022-06-29T15:51:05.992 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.975+0000 7f41355b6700 1 -- 172.21.15.47:0/685860834 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41300b2150 con 0x7f4130143150 2022-06-29T15:51:05.992 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.975+0000 7f41355b6700 1 -- 172.21.15.47:0/685860834 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41300573d0 con 0x7f4130143150 2022-06-29T15:51:05.992 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.975+0000 7f412d7fa700 1 -- 172.21.15.47:0/685860834 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f41280072d0 con 0x7f4130143150 2022-06-29T15:51:05.993 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.975+0000 7f412d7fa700 1 --2- 172.21.15.47:0/685860834 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f41100539e0 0x7f4110055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:05.993 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.975+0000 7f412d7fa700 1 -- 172.21.15.47:0/685860834 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6422+0+0 (secure 0 0 0) 0x7f412806bd50 con 0x7f4130143150 2022-06-29T15:51:05.993 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.975+0000 7f412ffff700 1 --2- 172.21.15.47:0/685860834 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f41100539e0 0x7f4110055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:05.993 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.979+0000 7f412ffff700 1 --2- 172.21.15.47:0/685860834 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f41100539e0 0x7f4110055ea0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7f4124018000 tx=0x7f4124014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:05.994 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:05.983+0000 7f412d7fa700 1 -- 172.21.15.47:0/685860834 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4128042b10 con 0x7f4130143150 2022-06-29T15:51:06.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.155+0000 7f41355b6700 1 -- 172.21.15.47:0/685860834 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "df", "format": "json-pretty"} v 0) v1 -- 0x7f413006a0f0 con 0x7f4130143150 2022-06-29T15:51:06.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.155+0000 7f412d7fa700 1 -- 172.21.15.47:0/685860834 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "df", "format": "json-pretty"}]=0 v0) v1 ==== 75+0+1930 (secure 0 0 0) 0x7f4128046130 con 0x7f4130143150 2022-06-29T15:51:06.163 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:06.163 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2022-06-29T15:51:06.163 INFO:teuthology.orchestra.run.smithi047.stdout: "stats": { 2022-06-29T15:51:06.164 INFO:teuthology.orchestra.run.smithi047.stdout: "total_bytes": 773094113280, 2022-06-29T15:51:06.164 INFO:teuthology.orchestra.run.smithi047.stdout: "total_avail_bytes": 764491370496, 2022-06-29T15:51:06.164 INFO:teuthology.orchestra.run.smithi047.stdout: "total_used_bytes": 12808192, 2022-06-29T15:51:06.164 INFO:teuthology.orchestra.run.smithi047.stdout: "total_used_raw_bytes": 8602742784, 2022-06-29T15:51:06.165 INFO:teuthology.orchestra.run.smithi047.stdout: "total_used_raw_ratio": 0.01112767867743969, 2022-06-29T15:51:06.165 INFO:teuthology.orchestra.run.smithi047.stdout: "num_osds": 8, 2022-06-29T15:51:06.166 INFO:teuthology.orchestra.run.smithi047.stdout: "num_per_pool_osds": 8, 2022-06-29T15:51:06.166 INFO:teuthology.orchestra.run.smithi047.stdout: "num_per_pool_omap_osds": 8 2022-06-29T15:51:06.166 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:06.166 INFO:teuthology.orchestra.run.smithi047.stdout: "stats_by_class": { 2022-06-29T15:51:06.167 INFO:teuthology.orchestra.run.smithi047.stdout: "ssd": { 2022-06-29T15:51:06.167 INFO:teuthology.orchestra.run.smithi047.stdout: "total_bytes": 773094113280, 2022-06-29T15:51:06.167 INFO:teuthology.orchestra.run.smithi047.stdout: "total_avail_bytes": 764491370496, 2022-06-29T15:51:06.167 INFO:teuthology.orchestra.run.smithi047.stdout: "total_used_bytes": 12808192, 2022-06-29T15:51:06.168 INFO:teuthology.orchestra.run.smithi047.stdout: "total_used_raw_bytes": 8602742784, 2022-06-29T15:51:06.168 INFO:teuthology.orchestra.run.smithi047.stdout: "total_used_raw_ratio": 0.01112767867743969 2022-06-29T15:51:06.168 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:06.168 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:06.169 INFO:teuthology.orchestra.run.smithi047.stdout: "pools": [ 2022-06-29T15:51:06.169 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:06.169 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "device_health_metrics", 2022-06-29T15:51:06.169 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 1, 2022-06-29T15:51:06.170 INFO:teuthology.orchestra.run.smithi047.stdout: "stats": { 2022-06-29T15:51:06.170 INFO:teuthology.orchestra.run.smithi047.stdout: "stored": 0, 2022-06-29T15:51:06.170 INFO:teuthology.orchestra.run.smithi047.stdout: "objects": 2, 2022-06-29T15:51:06.170 INFO:teuthology.orchestra.run.smithi047.stdout: "kb_used": 0, 2022-06-29T15:51:06.171 INFO:teuthology.orchestra.run.smithi047.stdout: "bytes_used": 0, 2022-06-29T15:51:06.171 INFO:teuthology.orchestra.run.smithi047.stdout: "percent_used": 0, 2022-06-29T15:51:06.171 INFO:teuthology.orchestra.run.smithi047.stdout: "max_avail": 362918281216 2022-06-29T15:51:06.171 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:06.172 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:06.172 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:06.172 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd", 2022-06-29T15:51:06.172 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 2, 2022-06-29T15:51:06.173 INFO:teuthology.orchestra.run.smithi047.stdout: "stats": { 2022-06-29T15:51:06.173 INFO:teuthology.orchestra.run.smithi047.stdout: "stored": 0, 2022-06-29T15:51:06.173 INFO:teuthology.orchestra.run.smithi047.stdout: "objects": 0, 2022-06-29T15:51:06.173 INFO:teuthology.orchestra.run.smithi047.stdout: "kb_used": 0, 2022-06-29T15:51:06.174 INFO:teuthology.orchestra.run.smithi047.stdout: "bytes_used": 0, 2022-06-29T15:51:06.174 INFO:teuthology.orchestra.run.smithi047.stdout: "percent_used": 0, 2022-06-29T15:51:06.174 INFO:teuthology.orchestra.run.smithi047.stdout: "max_avail": 362918281216 2022-06-29T15:51:06.174 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:06.175 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:06.175 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:06.175 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cephfs_data", 2022-06-29T15:51:06.175 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 22, 2022-06-29T15:51:06.176 INFO:teuthology.orchestra.run.smithi047.stdout: "stats": { 2022-06-29T15:51:06.176 INFO:teuthology.orchestra.run.smithi047.stdout: "stored": 0, 2022-06-29T15:51:06.176 INFO:teuthology.orchestra.run.smithi047.stdout: "objects": 0, 2022-06-29T15:51:06.176 INFO:teuthology.orchestra.run.smithi047.stdout: "kb_used": 0, 2022-06-29T15:51:06.177 INFO:teuthology.orchestra.run.smithi047.stdout: "bytes_used": 0, 2022-06-29T15:51:06.177 INFO:teuthology.orchestra.run.smithi047.stdout: "percent_used": 0, 2022-06-29T15:51:06.177 INFO:teuthology.orchestra.run.smithi047.stdout: "max_avail": 362918281216 2022-06-29T15:51:06.177 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:06.178 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:06.178 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:06.178 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cephfs_metadata", 2022-06-29T15:51:06.178 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 23, 2022-06-29T15:51:06.179 INFO:teuthology.orchestra.run.smithi047.stdout: "stats": { 2022-06-29T15:51:06.179 INFO:teuthology.orchestra.run.smithi047.stdout: "stored": 617, 2022-06-29T15:51:06.179 INFO:teuthology.orchestra.run.smithi047.stdout: "objects": 1, 2022-06-29T15:51:06.179 INFO:teuthology.orchestra.run.smithi047.stdout: "kb_used": 8, 2022-06-29T15:51:06.180 INFO:teuthology.orchestra.run.smithi047.stdout: "bytes_used": 8192, 2022-06-29T15:51:06.180 INFO:teuthology.orchestra.run.smithi047.stdout: "percent_used": 1.1286287460166022e-08, 2022-06-29T15:51:06.180 INFO:teuthology.orchestra.run.smithi047.stdout: "max_avail": 362918281216 2022-06-29T15:51:06.180 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:06.181 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:06.181 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:06.181 INFO:teuthology.orchestra.run.smithi047.stdout:} 2022-06-29T15:51:06.183 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.159+0000 7f41355b6700 1 -- 172.21.15.47:0/685860834 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f41100539e0 msgr2=0x7f4110055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:06.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.159+0000 7f41355b6700 1 --2- 172.21.15.47:0/685860834 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f41100539e0 0x7f4110055ea0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7f4124018000 tx=0x7f4124014000).stop 2022-06-29T15:51:06.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.159+0000 7f41355b6700 1 -- 172.21.15.47:0/685860834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4130143150 msgr2=0x7f4130140260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:06.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.159+0000 7f41355b6700 1 --2- 172.21.15.47:0/685860834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4130143150 0x7f4130140260 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 rx=0x7f4128001360 tx=0x7f4128001ab0).stop 2022-06-29T15:51:06.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.159+0000 7f41355b6700 1 -- 172.21.15.47:0/685860834 shutdown_connections 2022-06-29T15:51:06.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.159+0000 7f41355b6700 1 --2- 172.21.15.47:0/685860834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4130143150 0x7f4130140260 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:06.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.159+0000 7f41355b6700 1 --2- 172.21.15.47:0/685860834 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f41300b64c0 0x7f4130141820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:06.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.159+0000 7f41355b6700 1 --2- 172.21.15.47:0/685860834 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f41100539e0 0x7f4110055ea0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:06.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.159+0000 7f41355b6700 1 --2- 172.21.15.47:0/685860834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4130066fe0 0x7f4130141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:06.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.159+0000 7f41355b6700 1 -- 172.21.15.47:0/685860834 >> 172.21.15.47:0/685860834 conn(0x7f4130060e00 msgr2=0x7f41300b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:06.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.159+0000 7f41355b6700 1 -- 172.21.15.47:0/685860834 shutdown_connections 2022-06-29T15:51:06.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.159+0000 7f41355b6700 1 -- 172.21.15.47:0/685860834 wait complete. 2022-06-29T15:51:06.187 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s 2022-06-29T15:51:06.187 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2022-06-29T15:51:06.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.299+0000 7f6a4ad8b700 1 Processor -- start 2022-06-29T15:51:06.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.299+0000 7f6a4ad8b700 1 -- start start 2022-06-29T15:51:06.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.299+0000 7f6a4ad8b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6a440b58d0 0x7f6a440b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:06.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.299+0000 7f6a4ad8b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6a440b1610 0x7f6a441475f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:06.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.299+0000 7f6a4ad8b700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6a4414b4c0 0x7f6a440aa130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:06.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.299+0000 7f6a4ad8b700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a440a99e0 con 0x7f6a440b58d0 2022-06-29T15:51:06.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.299+0000 7f6a4ad8b700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a44060790 con 0x7f6a440b1610 2022-06-29T15:51:06.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.299+0000 7f6a4ad8b700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4414ad80 con 0x7f6a4414b4c0 2022-06-29T15:51:06.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.299+0000 7f6a49d89700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6a440b58d0 0x7f6a440b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:06.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.299+0000 7f6a49d89700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6a440b58d0 0x7f6a440b5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53164/0 (socket says 172.21.15.47:53164) 2022-06-29T15:51:06.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.299+0000 7f6a49d89700 1 -- 172.21.15.47:0/701563457 learned_addr learned my addr 172.21.15.47:0/701563457 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:06.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.299+0000 7f6a49588700 1 --2- 172.21.15.47:0/701563457 >> v2:172.21.15.64:3300/0 conn(0x7f6a440b1610 0x7f6a441475f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:06.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.299+0000 7f6a4a58a700 1 --2- 172.21.15.47:0/701563457 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6a4414b4c0 0x7f6a440aa130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:06.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.299+0000 7f6a49d89700 1 -- 172.21.15.47:0/701563457 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6a4414b4c0 msgr2=0x7f6a440aa130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:06.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.299+0000 7f6a49d89700 1 --2- 172.21.15.47:0/701563457 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6a4414b4c0 0x7f6a440aa130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:06.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.299+0000 7f6a49d89700 1 -- 172.21.15.47:0/701563457 >> v2:172.21.15.64:3300/0 conn(0x7f6a440b1610 msgr2=0x7f6a441475f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:06.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.299+0000 7f6a49d89700 1 --2- 172.21.15.47:0/701563457 >> v2:172.21.15.64:3300/0 conn(0x7f6a440b1610 0x7f6a441475f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:06.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.299+0000 7f6a49d89700 1 -- 172.21.15.47:0/701563457 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a4414b020 con 0x7f6a440b58d0 2022-06-29T15:51:06.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.299+0000 7f6a49d89700 1 --2- 172.21.15.47:0/701563457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a440b58d0 0x7f6a440b5cb0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 rx=0x7f6a340032f0 tx=0x7f6a34009290).ready entity=mon.0 client_cookie=65e1cb6d7e5f86e7 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:06.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.299+0000 7f6a48d87700 1 -- 172.21.15.47:0/701563457 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a3400b870 con 0x7f6a440b58d0 2022-06-29T15:51:06.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.299+0000 7f6a48d87700 1 -- 172.21.15.47:0/701563457 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a3400b9d0 con 0x7f6a440b58d0 2022-06-29T15:51:06.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.299+0000 7f6a48d87700 1 -- 172.21.15.47:0/701563457 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a3400bb40 con 0x7f6a440b58d0 2022-06-29T15:51:06.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.299+0000 7f6a4ad8b700 1 -- 172.21.15.47:0/701563457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a440b58d0 msgr2=0x7f6a440b5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:06.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.299+0000 7f6a4ad8b700 1 --2- 172.21.15.47:0/701563457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a440b58d0 0x7f6a440b5cb0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 rx=0x7f6a340032f0 tx=0x7f6a34009290).stop 2022-06-29T15:51:06.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.299+0000 7f6a4ad8b700 1 -- 172.21.15.47:0/701563457 shutdown_connections 2022-06-29T15:51:06.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.299+0000 7f6a4ad8b700 1 --2- 172.21.15.47:0/701563457 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6a4414b4c0 0x7f6a440aa130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:06.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.299+0000 7f6a4ad8b700 1 --2- 172.21.15.47:0/701563457 >> v2:172.21.15.64:3300/0 conn(0x7f6a440b1610 0x7f6a441475f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:06.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.299+0000 7f6a4ad8b700 1 --2- 172.21.15.47:0/701563457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a440b58d0 0x7f6a440b5cb0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:06.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.299+0000 7f6a4ad8b700 1 -- 172.21.15.47:0/701563457 >> 172.21.15.47:0/701563457 conn(0x7f6a44062560 msgr2=0x7f6a44062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:06.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.299+0000 7f6a4ad8b700 1 -- 172.21.15.47:0/701563457 shutdown_connections 2022-06-29T15:51:06.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.299+0000 7f6a4ad8b700 1 -- 172.21.15.47:0/701563457 wait complete. 2022-06-29T15:51:06.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.303+0000 7f6a4ad8b700 1 Processor -- start 2022-06-29T15:51:06.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.303+0000 7f6a4ad8b700 1 -- start start 2022-06-29T15:51:06.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.303+0000 7f6a4ad8b700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6a440b1610 0x7f6a440a8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:06.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.303+0000 7f6a4ad8b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a440b58d0 0x7f6a44141d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:06.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.303+0000 7f6a4ad8b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a4414b4c0 0x7f6a441441f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:06.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.303+0000 7f6a4ad8b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a44064570 con 0x7f6a4414b4c0 2022-06-29T15:51:06.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.303+0000 7f6a4ad8b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a440651f0 con 0x7f6a440b58d0 2022-06-29T15:51:06.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.303+0000 7f6a4ad8b700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a44065e20 con 0x7f6a440b1610 2022-06-29T15:51:06.335 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.303+0000 7f6a49d89700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6a440b1610 0x7f6a440a8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:06.335 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.303+0000 7f6a49588700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a440b58d0 0x7f6a44141d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:06.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.303+0000 7f6a49d89700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6a440b1610 0x7f6a440a8d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:50270/0 (socket says 172.21.15.47:50270) 2022-06-29T15:51:06.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.303+0000 7f6a49d89700 1 -- 172.21.15.47:0/851433997 learned_addr learned my addr 172.21.15.47:0/851433997 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:06.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.303+0000 7f6a4a58a700 1 --2- 172.21.15.47:0/851433997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a4414b4c0 0x7f6a441441f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:06.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.303+0000 7f6a4a58a700 1 -- 172.21.15.47:0/851433997 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6a440b1610 msgr2=0x7f6a440a8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:06.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.303+0000 7f6a4a58a700 1 --2- 172.21.15.47:0/851433997 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6a440b1610 0x7f6a440a8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:06.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.303+0000 7f6a4a58a700 1 -- 172.21.15.47:0/851433997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a440b58d0 msgr2=0x7f6a44141d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:06.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.303+0000 7f6a4a58a700 1 --2- 172.21.15.47:0/851433997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a440b58d0 0x7f6a44141d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:06.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.303+0000 7f6a4a58a700 1 -- 172.21.15.47:0/851433997 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a34009530 con 0x7f6a4414b4c0 2022-06-29T15:51:06.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.303+0000 7f6a49d89700 1 --2- 172.21.15.47:0/851433997 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6a440b1610 0x7f6a440a8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:51:06.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.303+0000 7f6a4a58a700 1 --2- 172.21.15.47:0/851433997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a4414b4c0 0x7f6a441441f0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 rx=0x7f6a3c00a000 tx=0x7f6a3c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:06.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.303+0000 7f6a42ffd700 1 -- 172.21.15.47:0/851433997 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a3c0104f0 con 0x7f6a4414b4c0 2022-06-29T15:51:06.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.303+0000 7f6a4ad8b700 1 -- 172.21.15.47:0/851433997 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a44065910 con 0x7f6a4414b4c0 2022-06-29T15:51:06.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.303+0000 7f6a42ffd700 1 -- 172.21.15.47:0/851433997 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a3c010650 con 0x7f6a4414b4c0 2022-06-29T15:51:06.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.303+0000 7f6a42ffd700 1 -- 172.21.15.47:0/851433997 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a3c0109b0 con 0x7f6a4414b4c0 2022-06-29T15:51:06.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.303+0000 7f6a4ad8b700 1 -- 172.21.15.47:0/851433997 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a44065420 con 0x7f6a4414b4c0 2022-06-29T15:51:06.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.307+0000 7f6a42ffd700 1 -- 172.21.15.47:0/851433997 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6a3c01a030 con 0x7f6a4414b4c0 2022-06-29T15:51:06.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.307+0000 7f6a4ad8b700 1 -- 172.21.15.47:0/851433997 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a4405ffe0 con 0x7f6a4414b4c0 2022-06-29T15:51:06.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.307+0000 7f6a42ffd700 1 --2- 172.21.15.47:0/851433997 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6a24053950 0x7f6a24055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:06.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.307+0000 7f6a42ffd700 1 -- 172.21.15.47:0/851433997 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6422+0+0 (secure 0 0 0) 0x7f6a3c06bb10 con 0x7f6a4414b4c0 2022-06-29T15:51:06.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.307+0000 7f6a49d89700 1 --2- 172.21.15.47:0/851433997 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6a24053950 0x7f6a24055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:06.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.307+0000 7f6a49d89700 1 --2- 172.21.15.47:0/851433997 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6a24053950 0x7f6a24055e10 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7f6a3401b000 tx=0x7f6a34016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:06.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.315+0000 7f6a42ffd700 1 -- 172.21.15.47:0/851433997 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6a3c0428d0 con 0x7f6a4414b4c0 2022-06-29T15:51:06.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.479+0000 7f6a4ad8b700 1 -- 172.21.15.47:0/851433997 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f6a440573d0 con 0x7f6a4414b4c0 2022-06-29T15:51:06.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.479+0000 7f6a42ffd700 1 -- 172.21.15.47:0/851433997 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=-22 pool 'cephfs_metadata' already contains some objects. Use an empty pool instead. v59) v1 ==== 209+0+0 (secure 0 0 0) 0x7f6a3c045ef0 con 0x7f6a4414b4c0 2022-06-29T15:51:06.488 INFO:teuthology.orchestra.run.smithi047.stderr:Error EINVAL: pool 'cephfs_metadata' already contains some objects. Use an empty pool instead. 2022-06-29T15:51:06.492 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-06-29T15:51:06.493 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-06-29T15:51:06.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.599+0000 7f005ffff700 1 Processor -- start 2022-06-29T15:51:06.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.599+0000 7f005ffff700 1 -- start start 2022-06-29T15:51:06.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.599+0000 7f005ffff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f00600a9c60 0x7f006013d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:06.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.599+0000 7f005ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f00600a9250 0x7f0060140160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:06.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.599+0000 7f005ffff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00600b0190 con 0x7f00600b5b20 2022-06-29T15:51:06.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.599+0000 7f005ffff700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00600aa570 con 0x7f00600a9c60 2022-06-29T15:51:06.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.599+0000 7f005ffff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00600b03c0 con 0x7f00600a9250 2022-06-29T15:51:06.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.599+0000 7f005e7fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f00600a9c60 0x7f006013d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:06.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.599+0000 7f005e7fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f00600a9c60 0x7f006013d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53176/0 (socket says 172.21.15.47:53176) 2022-06-29T15:51:06.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.599+0000 7f005e7fc700 1 -- 172.21.15.47:0/1138143645 learned_addr learned my addr 172.21.15.47:0/1138143645 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:06.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.603+0000 7f005f7fe700 1 --2- 172.21.15.47:0/1138143645 >> v2:172.21.15.64:3300/0 conn(0x7f00600a9250 0x7f0060140160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:06.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.603+0000 7f005e7fc700 1 -- 172.21.15.47:0/1138143645 >> v2:172.21.15.64:3300/0 conn(0x7f00600a9250 msgr2=0x7f0060140160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:06.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.603+0000 7f005e7fc700 1 --2- 172.21.15.47:0/1138143645 >> v2:172.21.15.64:3300/0 conn(0x7f00600a9250 0x7f0060140160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:06.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.603+0000 7f005e7fc700 1 -- 172.21.15.47:0/1138143645 >> v1:172.21.15.64:6789/0 conn(0x7f00600b5b20 legacy=0x7f006013d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:06.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.603+0000 7f005e7fc700 1 -- 172.21.15.47:0/1138143645 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00600a9670 con 0x7f00600a9c60 2022-06-29T15:51:06.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.603+0000 7f005e7fc700 1 --2- 172.21.15.47:0/1138143645 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00600a9c60 0x7f006013d8b0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 rx=0x7f0050002f70 tx=0x7f0050006eb0).ready entity=mon.0 client_cookie=de6616ea074013e7 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:06.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.603+0000 7f005dffb700 1 -- 172.21.15.47:0/1138143645 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f005000d610 con 0x7f00600a9c60 2022-06-29T15:51:06.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.603+0000 7f005dffb700 1 -- 172.21.15.47:0/1138143645 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0050000e90 con 0x7f00600a9c60 2022-06-29T15:51:06.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.603+0000 7f005dffb700 1 -- 172.21.15.47:0/1138143645 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f005000d9a0 con 0x7f00600a9c60 2022-06-29T15:51:06.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.603+0000 7f005ffff700 1 -- 172.21.15.47:0/1138143645 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00600a9c60 msgr2=0x7f006013d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:06.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.603+0000 7f005ffff700 1 --2- 172.21.15.47:0/1138143645 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00600a9c60 0x7f006013d8b0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 rx=0x7f0050002f70 tx=0x7f0050006eb0).stop 2022-06-29T15:51:06.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.603+0000 7f005ffff700 1 -- 172.21.15.47:0/1138143645 shutdown_connections 2022-06-29T15:51:06.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.603+0000 7f005ffff700 1 --2- 172.21.15.47:0/1138143645 >> v2:172.21.15.64:3300/0 conn(0x7f00600a9250 0x7f0060140160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:06.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.603+0000 7f005ffff700 1 --2- 172.21.15.47:0/1138143645 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00600a9c60 0x7f006013d8b0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:06.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.603+0000 7f005ffff700 1 -- 172.21.15.47:0/1138143645 >> 172.21.15.47:0/1138143645 conn(0x7f0060062560 msgr2=0x7f0060062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:06.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.603+0000 7f005ffff700 1 -- 172.21.15.47:0/1138143645 shutdown_connections 2022-06-29T15:51:06.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.603+0000 7f005ffff700 1 -- 172.21.15.47:0/1138143645 wait complete. 2022-06-29T15:51:06.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.607+0000 7f005ffff700 1 Processor -- start 2022-06-29T15:51:06.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.607+0000 7f005ffff700 1 -- start start 2022-06-29T15:51:06.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.607+0000 7f005ffff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f00600a9250 0x7f0060136930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:06.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.607+0000 7f005ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00600a9c60 0x7f0060151b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:06.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.607+0000 7f005ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00600b5b20 0x7f0060152010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:06.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.607+0000 7f005ffff700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006014e230 con 0x7f00600a9c60 2022-06-29T15:51:06.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.607+0000 7f005ffff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00600b4b90 con 0x7f00600b5b20 2022-06-29T15:51:06.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.607+0000 7f005ffff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00600b3720 con 0x7f00600a9250 2022-06-29T15:51:06.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.607+0000 7f005e7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00600a9c60 0x7f0060151b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:06.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.607+0000 7f005e7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00600a9c60 0x7f0060151b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53184/0 (socket says 172.21.15.47:53184) 2022-06-29T15:51:06.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.607+0000 7f005e7fc700 1 -- 172.21.15.47:0/3782975703 learned_addr learned my addr 172.21.15.47:0/3782975703 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:06.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.607+0000 7f005effd700 1 --2- 172.21.15.47:0/3782975703 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f00600a9250 0x7f0060136930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:06.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.607+0000 7f005f7fe700 1 --2- 172.21.15.47:0/3782975703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00600b5b20 0x7f0060152010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:06.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.607+0000 7f005e7fc700 1 -- 172.21.15.47:0/3782975703 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f00600a9250 msgr2=0x7f0060136930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:06.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.607+0000 7f005e7fc700 1 --2- 172.21.15.47:0/3782975703 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f00600a9250 0x7f0060136930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:06.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.607+0000 7f005e7fc700 1 -- 172.21.15.47:0/3782975703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00600b5b20 msgr2=0x7f0060152010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:06.622 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.607+0000 7f005e7fc700 1 --2- 172.21.15.47:0/3782975703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00600b5b20 0x7f0060152010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:06.622 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.607+0000 7f005e7fc700 1 -- 172.21.15.47:0/3782975703 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f005000b1e0 con 0x7f00600a9c60 2022-06-29T15:51:06.622 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.607+0000 7f005effd700 1 --2- 172.21.15.47:0/3782975703 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f00600a9250 0x7f0060136930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:51:06.622 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.607+0000 7f005e7fc700 1 --2- 172.21.15.47:0/3782975703 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00600a9c60 0x7f0060151b20 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 rx=0x7f0050002f70 tx=0x7f0050015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:06.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.607+0000 7f004ffff700 1 -- 172.21.15.47:0/3782975703 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0050011860 con 0x7f00600a9c60 2022-06-29T15:51:06.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.611+0000 7f004ffff700 1 -- 172.21.15.47:0/3782975703 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00500119c0 con 0x7f00600a9c60 2022-06-29T15:51:06.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.611+0000 7f004ffff700 1 -- 172.21.15.47:0/3782975703 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0050011d20 con 0x7f00600a9c60 2022-06-29T15:51:06.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.611+0000 7f005ffff700 1 -- 172.21.15.47:0/3782975703 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0060152500 con 0x7f00600a9c60 2022-06-29T15:51:06.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.611+0000 7f005ffff700 1 -- 172.21.15.47:0/3782975703 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0060136fc0 con 0x7f00600a9c60 2022-06-29T15:51:06.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.611+0000 7f005ffff700 1 -- 172.21.15.47:0/3782975703 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00600573d0 con 0x7f00600a9c60 2022-06-29T15:51:06.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.611+0000 7f004ffff700 1 -- 172.21.15.47:0/3782975703 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f005001a040 con 0x7f00600a9c60 2022-06-29T15:51:06.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.611+0000 7f004ffff700 1 --2- 172.21.15.47:0/3782975703 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f003c053a10 0x7f003c055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:06.626 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.611+0000 7f004ffff700 1 -- 172.21.15.47:0/3782975703 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6422+0+0 (secure 0 0 0) 0x7f005006cf10 con 0x7f00600a9c60 2022-06-29T15:51:06.626 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.611+0000 7f005effd700 1 --2- 172.21.15.47:0/3782975703 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f003c053a10 0x7f003c055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:06.626 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.611+0000 7f005effd700 1 --2- 172.21.15.47:0/3782975703 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f003c053a10 0x7f003c055ed0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7f005400b000 tx=0x7f0054009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:06.626 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.619+0000 7f004ffff700 1 -- 172.21.15.47:0/3782975703 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0050043cd0 con 0x7f00600a9c60 2022-06-29T15:51:06.785 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.779+0000 7f005ffff700 1 -- 172.21.15.47:0/3782975703 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f00600b4d00 con 0x7f00600a9c60 2022-06-29T15:51:06.786 INFO:tasks.ceph.mon.a.smithi047.stderr:2022-06-29T15:51:06.779+0000 7ff9cd037700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-06-29T15:51:06.796 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.791+0000 7f004ffff700 1 -- 172.21.15.47:0/3782975703 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 23 and data pool 22 v60) v1 ==== 189+0+0 (secure 0 0 0) 0x7f00500472f0 con 0x7f00600a9c60 2022-06-29T15:51:06.800 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.795+0000 7f005ffff700 1 -- 172.21.15.47:0/3782975703 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f003c053a10 msgr2=0x7f003c055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:06.800 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.795+0000 7f005ffff700 1 --2- 172.21.15.47:0/3782975703 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f003c053a10 0x7f003c055ed0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7f005400b000 tx=0x7f0054009040).stop 2022-06-29T15:51:06.800 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.795+0000 7f005ffff700 1 -- 172.21.15.47:0/3782975703 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00600a9c60 msgr2=0x7f0060151b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:06.801 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.795+0000 7f005ffff700 1 --2- 172.21.15.47:0/3782975703 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00600a9c60 0x7f0060151b20 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 rx=0x7f0050002f70 tx=0x7f0050015000).stop 2022-06-29T15:51:06.801 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.795+0000 7f005ffff700 1 -- 172.21.15.47:0/3782975703 shutdown_connections 2022-06-29T15:51:06.801 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.795+0000 7f005ffff700 1 --2- 172.21.15.47:0/3782975703 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f003c053a10 0x7f003c055ed0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:06.801 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.795+0000 7f005ffff700 1 --2- 172.21.15.47:0/3782975703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00600b5b20 0x7f0060152010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:06.802 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.795+0000 7f005ffff700 1 --2- 172.21.15.47:0/3782975703 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00600a9c60 0x7f0060151b20 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:06.802 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.795+0000 7f005ffff700 1 --2- 172.21.15.47:0/3782975703 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f00600a9250 0x7f0060136930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:06.802 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.795+0000 7f005ffff700 1 -- 172.21.15.47:0/3782975703 >> 172.21.15.47:0/3782975703 conn(0x7f0060062560 msgr2=0x7f0060144630 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:06.802 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.795+0000 7f005ffff700 1 -- 172.21.15.47:0/3782975703 shutdown_connections 2022-06-29T15:51:06.802 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.795+0000 7f005ffff700 1 -- 172.21.15.47:0/3782975703 wait complete. 2022-06-29T15:51:06.803 INFO:teuthology.orchestra.run.smithi047.stderr:new fs with metadata pool 23 and data pool 22 2022-06-29T15:51:06.814 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-06-29T15:51:06.920 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.915+0000 7f8e4a18f700 1 Processor -- start 2022-06-29T15:51:06.921 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.915+0000 7f8e4a18f700 1 -- start start 2022-06-29T15:51:06.923 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.915+0000 7f8e4a18f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8e440b6d20 0x7f8e44148dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:06.923 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.915+0000 7f8e4a18f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8e440612b0 0x7f8e440b3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:06.923 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.915+0000 7f8e4a18f700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e440b7100 con 0x7f8e44062f30 2022-06-29T15:51:06.924 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.915+0000 7f8e4a18f700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e440b17f0 con 0x7f8e440612b0 2022-06-29T15:51:06.924 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.915+0000 7f8e4a18f700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e440b1e80 con 0x7f8e440b6d20 2022-06-29T15:51:06.924 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.915+0000 7f8e4898c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8e440612b0 0x7f8e440b3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:06.924 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.915+0000 7f8e4898c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8e440612b0 0x7f8e440b3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53192/0 (socket says 172.21.15.47:53192) 2022-06-29T15:51:06.925 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.915+0000 7f8e4998e700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f8e44062f30 0x7f8e44066860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38746/0 (socket says 172.21.15.47:38746) 2022-06-29T15:51:06.925 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.915+0000 7f8e4898c700 1 -- 172.21.15.47:0/3604194694 learned_addr learned my addr 172.21.15.47:0/3604194694 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:06.925 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.915+0000 7f8e4918d700 1 --2- 172.21.15.47:0/3604194694 >> v2:172.21.15.64:3300/0 conn(0x7f8e440b6d20 0x7f8e44148dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:06.925 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.915+0000 7f8e4898c700 1 -- 172.21.15.47:0/3604194694 >> v2:172.21.15.64:3300/0 conn(0x7f8e440b6d20 msgr2=0x7f8e44148dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:06.926 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.915+0000 7f8e4898c700 1 --2- 172.21.15.47:0/3604194694 >> v2:172.21.15.64:3300/0 conn(0x7f8e440b6d20 0x7f8e44148dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:06.926 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.915+0000 7f8e4898c700 1 -- 172.21.15.47:0/3604194694 >> v1:172.21.15.47:6789/0 conn(0x7f8e44062f30 legacy=0x7f8e44066860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:06.926 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.915+0000 7f8e4898c700 1 -- 172.21.15.47:0/3604194694 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e440b2ee0 con 0x7f8e440612b0 2022-06-29T15:51:06.927 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.915+0000 7f8e4898c700 1 --2- 172.21.15.47:0/3604194694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e440612b0 0x7f8e440b3620 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 rx=0x7f8e3c001110 tx=0x7f8e40006eb0).ready entity=mon.0 client_cookie=fc5efebf0fd576dd server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:06.927 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.915+0000 7f8e3bfff700 1 -- 172.21.15.47:0/3604194694 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e4000e590 con 0x7f8e440612b0 2022-06-29T15:51:06.927 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.915+0000 7f8e3bfff700 1 -- 172.21.15.47:0/3604194694 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e40000e90 con 0x7f8e440612b0 2022-06-29T15:51:06.927 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.915+0000 7f8e3bfff700 1 -- 172.21.15.47:0/3604194694 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e4000e920 con 0x7f8e440612b0 2022-06-29T15:51:06.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.919+0000 7f8e4a18f700 1 -- 172.21.15.47:0/3604194694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e440612b0 msgr2=0x7f8e440b3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:06.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.919+0000 7f8e4a18f700 1 --2- 172.21.15.47:0/3604194694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e440612b0 0x7f8e440b3620 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 rx=0x7f8e3c001110 tx=0x7f8e40006eb0).stop 2022-06-29T15:51:06.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.919+0000 7f8e4a18f700 1 -- 172.21.15.47:0/3604194694 shutdown_connections 2022-06-29T15:51:06.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.919+0000 7f8e4a18f700 1 --2- 172.21.15.47:0/3604194694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e440612b0 0x7f8e440b3620 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:06.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.919+0000 7f8e4a18f700 1 --2- 172.21.15.47:0/3604194694 >> v2:172.21.15.64:3300/0 conn(0x7f8e440b6d20 0x7f8e44148dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:06.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.919+0000 7f8e4a18f700 1 -- 172.21.15.47:0/3604194694 >> 172.21.15.47:0/3604194694 conn(0x7f8e44063670 msgr2=0x7f8e44063a50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:06.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.919+0000 7f8e4a18f700 1 -- 172.21.15.47:0/3604194694 shutdown_connections 2022-06-29T15:51:06.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.919+0000 7f8e4a18f700 1 -- 172.21.15.47:0/3604194694 wait complete. 2022-06-29T15:51:06.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.919+0000 7f8e4a18f700 1 Processor -- start 2022-06-29T15:51:06.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.919+0000 7f8e4a18f700 1 -- start start 2022-06-29T15:51:06.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.919+0000 7f8e4a18f700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8e440612b0 0x7f8e440ac450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:06.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.919+0000 7f8e4a18f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e44062f30 0x7f8e441576f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:06.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.919+0000 7f8e4a18f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e440b6d20 0x7f8e440b0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:06.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.919+0000 7f8e4a18f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e440aca60 con 0x7f8e440b6d20 2022-06-29T15:51:06.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.919+0000 7f8e4a18f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e440acc90 con 0x7f8e44062f30 2022-06-29T15:51:06.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.919+0000 7f8e4a18f700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e440acf30 con 0x7f8e440612b0 2022-06-29T15:51:06.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.923+0000 7f8e4998e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e440b6d20 0x7f8e440b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:06.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.923+0000 7f8e4998e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e440b6d20 0x7f8e440b0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53200/0 (socket says 172.21.15.47:53200) 2022-06-29T15:51:06.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.923+0000 7f8e4998e700 1 -- 172.21.15.47:0/1459041974 learned_addr learned my addr 172.21.15.47:0/1459041974 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:06.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.923+0000 7f8e4918d700 1 --2- 172.21.15.47:0/1459041974 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8e440612b0 0x7f8e440ac450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:06.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.923+0000 7f8e4898c700 1 --2- 172.21.15.47:0/1459041974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e44062f30 0x7f8e441576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:06.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.923+0000 7f8e4998e700 1 -- 172.21.15.47:0/1459041974 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8e440612b0 msgr2=0x7f8e440ac450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:06.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.923+0000 7f8e4998e700 1 --2- 172.21.15.47:0/1459041974 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8e440612b0 0x7f8e440ac450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:06.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.923+0000 7f8e4998e700 1 -- 172.21.15.47:0/1459041974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e44062f30 msgr2=0x7f8e441576f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:06.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.923+0000 7f8e4998e700 1 --2- 172.21.15.47:0/1459041974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e44062f30 0x7f8e441576f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:06.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.923+0000 7f8e4998e700 1 -- 172.21.15.47:0/1459041974 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e4000b1e0 con 0x7f8e440b6d20 2022-06-29T15:51:06.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.923+0000 7f8e4898c700 1 --2- 172.21.15.47:0/1459041974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e44062f30 0x7f8e441576f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:51:06.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.923+0000 7f8e4998e700 1 --2- 172.21.15.47:0/1459041974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e440b6d20 0x7f8e440b0480 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 rx=0x7f8e3c0011d0 tx=0x7f8e3c007b80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:06.938 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.923+0000 7f8e3a7fc700 1 -- 172.21.15.47:0/1459041974 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e3c0072e0 con 0x7f8e440b6d20 2022-06-29T15:51:06.938 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.923+0000 7f8e3a7fc700 1 -- 172.21.15.47:0/1459041974 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e3c005550 con 0x7f8e440b6d20 2022-06-29T15:51:06.938 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.923+0000 7f8e4a18f700 1 -- 172.21.15.47:0/1459041974 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e440b0970 con 0x7f8e440b6d20 2022-06-29T15:51:06.938 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.923+0000 7f8e3a7fc700 1 -- 172.21.15.47:0/1459041974 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e3c0058b0 con 0x7f8e440b6d20 2022-06-29T15:51:06.939 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.923+0000 7f8e4a18f700 1 -- 172.21.15.47:0/1459041974 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e4414d750 con 0x7f8e440b6d20 2022-06-29T15:51:06.939 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.923+0000 7f8e4a18f700 1 -- 172.21.15.47:0/1459041974 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e440ad0f0 con 0x7f8e440b6d20 2022-06-29T15:51:06.939 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.923+0000 7f8e3a7fc700 1 -- 172.21.15.47:0/1459041974 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8e3c010070 con 0x7f8e440b6d20 2022-06-29T15:51:06.940 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.927+0000 7f8e3a7fc700 1 --2- 172.21.15.47:0/1459041974 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8e24053a10 0x7f8e24055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:06.940 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.927+0000 7f8e3a7fc700 1 -- 172.21.15.47:0/1459041974 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f8e3c06bd20 con 0x7f8e440b6d20 2022-06-29T15:51:06.941 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.927+0000 7f8e4918d700 1 --2- 172.21.15.47:0/1459041974 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8e24053a10 0x7f8e24055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:06.941 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.927+0000 7f8e4918d700 1 --2- 172.21.15.47:0/1459041974 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8e24053a10 0x7f8e24055ed0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7f8e3400b000 tx=0x7f8e34008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:06.941 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:06.927+0000 7f8e3a7fc700 1 -- 172.21.15.47:0/1459041974 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8e3c042a90 con 0x7f8e440b6d20 2022-06-29T15:51:07.098 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.091+0000 7f8e4a18f700 1 -- 172.21.15.47:0/1459041974 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8e440573d0 con 0x7f8e440b6d20 2022-06-29T15:51:07.801 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.795+0000 7f8e3a7fc700 1 -- 172.21.15.47:0/1459041974 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v61) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8e3c0460b0 con 0x7f8e440b6d20 2022-06-29T15:51:07.805 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.799+0000 7f8e4a18f700 1 -- 172.21.15.47:0/1459041974 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8e24053a10 msgr2=0x7f8e24055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:07.806 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.799+0000 7f8e4a18f700 1 --2- 172.21.15.47:0/1459041974 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8e24053a10 0x7f8e24055ed0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7f8e3400b000 tx=0x7f8e34008040).stop 2022-06-29T15:51:07.806 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.799+0000 7f8e4a18f700 1 -- 172.21.15.47:0/1459041974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e440b6d20 msgr2=0x7f8e440b0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:07.806 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.799+0000 7f8e4a18f700 1 --2- 172.21.15.47:0/1459041974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e440b6d20 0x7f8e440b0480 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 rx=0x7f8e3c0011d0 tx=0x7f8e3c007b80).stop 2022-06-29T15:51:07.806 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.799+0000 7f8e4a18f700 1 -- 172.21.15.47:0/1459041974 shutdown_connections 2022-06-29T15:51:07.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.799+0000 7f8e4a18f700 1 --2- 172.21.15.47:0/1459041974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e440b6d20 0x7f8e440b0480 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:07.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.799+0000 7f8e4a18f700 1 --2- 172.21.15.47:0/1459041974 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8e24053a10 0x7f8e24055ed0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:07.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.799+0000 7f8e4a18f700 1 --2- 172.21.15.47:0/1459041974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e44062f30 0x7f8e441576f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:07.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.799+0000 7f8e4a18f700 1 --2- 172.21.15.47:0/1459041974 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8e440612b0 0x7f8e440ac450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:07.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.799+0000 7f8e4a18f700 1 -- 172.21.15.47:0/1459041974 >> 172.21.15.47:0/1459041974 conn(0x7f8e44063670 msgr2=0x7f8e4414e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:07.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.799+0000 7f8e4a18f700 1 -- 172.21.15.47:0/1459041974 shutdown_connections 2022-06-29T15:51:07.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.799+0000 7f8e4a18f700 1 -- 172.21.15.47:0/1459041974 wait complete. 2022-06-29T15:51:07.821 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-06-29T15:51:07.938 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.931+0000 7f66fbb85700 1 Processor -- start 2022-06-29T15:51:07.938 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.931+0000 7f66fbb85700 1 -- start start 2022-06-29T15:51:07.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.935+0000 7f66fbb85700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f66f4060ee0 0x7f66f40612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:07.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.935+0000 7f66fbb85700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f66f40b4300 0x7f66f41471a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:07.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.935+0000 7f66fbb85700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66f4067610 con 0x7f66f4066f70 2022-06-29T15:51:07.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.935+0000 7f66fbb85700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66f40673a0 con 0x7f66f40b4300 2022-06-29T15:51:07.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.935+0000 7f66fbb85700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f41467c0 con 0x7f66f4060ee0 2022-06-29T15:51:07.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.935+0000 7f66f9120700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f66f40b4300 0x7f66f41471a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:07.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.935+0000 7f66f9921700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f66f4060ee0 0x7f66f40612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:07.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.935+0000 7f66f9120700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f66f40b4300 0x7f66f41471a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:58910/0 (socket says 172.21.15.47:58910) 2022-06-29T15:51:07.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.935+0000 7f66f9120700 1 -- 172.21.15.47:0/831211010 learned_addr learned my addr 172.21.15.47:0/831211010 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:07.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.935+0000 7f66fa122700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f66f4066f70 0x7f66f41430e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:46326/0 (socket says 172.21.15.47:46326) 2022-06-29T15:51:07.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.935+0000 7f66f9921700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f66f4060ee0 0x7f66f40612c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:50306/0 (socket says 172.21.15.47:50306) 2022-06-29T15:51:07.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.935+0000 7f66f891f700 1 -- 172.21.15.47:0/831211010 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1408800557 0 0) 0x7f66f4067610 con 0x7f66f4066f70 2022-06-29T15:51:07.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.935+0000 7f66f891f700 1 -- 172.21.15.47:0/831211010 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66dc003260 con 0x7f66f4066f70 2022-06-29T15:51:07.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.935+0000 7f66f9921700 1 -- 172.21.15.47:0/831211010 >> v2:172.21.15.64:3300/0 conn(0x7f66f40b4300 msgr2=0x7f66f41471a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:07.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.935+0000 7f66f9921700 1 --2- 172.21.15.47:0/831211010 >> v2:172.21.15.64:3300/0 conn(0x7f66f40b4300 0x7f66f41471a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:07.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.935+0000 7f66f9921700 1 -- 172.21.15.47:0/831211010 >> v1:172.21.15.64:6789/0 conn(0x7f66f4066f70 legacy=0x7f66f41430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:07.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.935+0000 7f66f9921700 1 -- 172.21.15.47:0/831211010 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f4146a60 con 0x7f66f4060ee0 2022-06-29T15:51:07.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.935+0000 7f66f9921700 1 --2- 172.21.15.47:0/831211010 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f66f4060ee0 0x7f66f40612c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f66e4002f70 tx=0x7f66e4006df0).ready entity=mon.2 client_cookie=43d42bf02dd1cf8c server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:07.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.935+0000 7f66f891f700 1 -- 172.21.15.47:0/831211010 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66e400d4a0 con 0x7f66f4060ee0 2022-06-29T15:51:07.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.935+0000 7f66f891f700 1 -- 172.21.15.47:0/831211010 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e4000e90 con 0x7f66f4060ee0 2022-06-29T15:51:07.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.935+0000 7f66f891f700 1 -- 172.21.15.47:0/831211010 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66e400d7f0 con 0x7f66f4060ee0 2022-06-29T15:51:07.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.935+0000 7f66fbb85700 1 -- 172.21.15.47:0/831211010 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f66f4060ee0 msgr2=0x7f66f40612c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:07.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.935+0000 7f66fbb85700 1 --2- 172.21.15.47:0/831211010 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f66f4060ee0 0x7f66f40612c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f66e4002f70 tx=0x7f66e4006df0).stop 2022-06-29T15:51:07.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.935+0000 7f66fbb85700 1 -- 172.21.15.47:0/831211010 shutdown_connections 2022-06-29T15:51:07.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.935+0000 7f66fbb85700 1 --2- 172.21.15.47:0/831211010 >> v2:172.21.15.64:3300/0 conn(0x7f66f40b4300 0x7f66f41471a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:07.950 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.935+0000 7f66fbb85700 1 --2- 172.21.15.47:0/831211010 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f66f4060ee0 0x7f66f40612c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:07.950 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.935+0000 7f66fbb85700 1 -- 172.21.15.47:0/831211010 >> 172.21.15.47:0/831211010 conn(0x7f66f405c510 msgr2=0x7f66f405f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:07.950 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.935+0000 7f66fbb85700 1 -- 172.21.15.47:0/831211010 shutdown_connections 2022-06-29T15:51:07.951 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.935+0000 7f66fbb85700 1 -- 172.21.15.47:0/831211010 wait complete. 2022-06-29T15:51:07.954 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.939+0000 7f66fbb85700 1 Processor -- start 2022-06-29T15:51:07.954 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.939+0000 7f66fbb85700 1 -- start start 2022-06-29T15:51:07.954 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.939+0000 7f66fbb85700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66f4060ee0 0x7f66f40b6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:07.954 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.939+0000 7f66fbb85700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66f4066f70 0x7f66f41361c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:07.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.939+0000 7f66fbb85700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f66f40b4300 0x7f66f41366b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:07.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.939+0000 7f66fbb85700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f4144dd0 con 0x7f66f4066f70 2022-06-29T15:51:07.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.939+0000 7f66fbb85700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f413a590 con 0x7f66f4060ee0 2022-06-29T15:51:07.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.939+0000 7f66fbb85700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f4136ba0 con 0x7f66f40b4300 2022-06-29T15:51:07.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.939+0000 7f66f9120700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66f4066f70 0x7f66f41361c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:07.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.939+0000 7f66f9120700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66f4066f70 0x7f66f41361c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53212/0 (socket says 172.21.15.47:53212) 2022-06-29T15:51:07.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.939+0000 7f66f9120700 1 -- 172.21.15.47:0/3150478276 learned_addr learned my addr 172.21.15.47:0/3150478276 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:07.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.939+0000 7f66f9921700 1 --2- 172.21.15.47:0/3150478276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66f4060ee0 0x7f66f40b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:07.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.939+0000 7f66fa122700 1 --2- 172.21.15.47:0/3150478276 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f66f40b4300 0x7f66f41366b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:07.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.939+0000 7f66f9120700 1 -- 172.21.15.47:0/3150478276 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f66f40b4300 msgr2=0x7f66f41366b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:07.958 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.939+0000 7f66f9120700 1 --2- 172.21.15.47:0/3150478276 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f66f40b4300 0x7f66f41366b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:07.958 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.939+0000 7f66f9120700 1 -- 172.21.15.47:0/3150478276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66f4060ee0 msgr2=0x7f66f40b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:07.958 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.939+0000 7f66f9120700 1 --2- 172.21.15.47:0/3150478276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66f4060ee0 0x7f66f40b6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:07.959 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.939+0000 7f66f9120700 1 -- 172.21.15.47:0/3150478276 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e400b180 con 0x7f66f4066f70 2022-06-29T15:51:07.959 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.939+0000 7f66fa122700 1 --2- 172.21.15.47:0/3150478276 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f66f40b4300 0x7f66f41366b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:51:07.959 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.939+0000 7f66f9120700 1 --2- 172.21.15.47:0/3150478276 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66f4066f70 0x7f66f41361c0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 rx=0x7f66f000c000 tx=0x7f66f0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:07.960 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.939+0000 7f66e2ffd700 1 -- 172.21.15.47:0/3150478276 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66f0013610 con 0x7f66f4066f70 2022-06-29T15:51:07.960 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.939+0000 7f66e2ffd700 1 -- 172.21.15.47:0/3150478276 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f0013770 con 0x7f66f4066f70 2022-06-29T15:51:07.960 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.939+0000 7f66fbb85700 1 -- 172.21.15.47:0/3150478276 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66f413a7f0 con 0x7f66f4066f70 2022-06-29T15:51:07.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.939+0000 7f66e2ffd700 1 -- 172.21.15.47:0/3150478276 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66f0013ad0 con 0x7f66f4066f70 2022-06-29T15:51:07.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.939+0000 7f66fbb85700 1 -- 172.21.15.47:0/3150478276 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66f413a070 con 0x7f66f4066f70 2022-06-29T15:51:07.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.943+0000 7f66fbb85700 1 -- 172.21.15.47:0/3150478276 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66f40573d0 con 0x7f66f4066f70 2022-06-29T15:51:07.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.943+0000 7f66e2ffd700 1 -- 172.21.15.47:0/3150478276 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f66f0013c30 con 0x7f66f4066f70 2022-06-29T15:51:07.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.943+0000 7f66e2ffd700 1 --2- 172.21.15.47:0/3150478276 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f66d4053980 0x7f66d4055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:07.962 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.943+0000 7f66e2ffd700 1 -- 172.21.15.47:0/3150478276 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f66f006bfe0 con 0x7f66f4066f70 2022-06-29T15:51:07.962 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.943+0000 7f66f9921700 1 --2- 172.21.15.47:0/3150478276 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f66d4053980 0x7f66d4055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:07.962 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.943+0000 7f66f9921700 1 --2- 172.21.15.47:0/3150478276 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f66d4053980 0x7f66d4055e40 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7f66e401a000 tx=0x7f66e4014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:07.963 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:07.951+0000 7f66e2ffd700 1 -- 172.21.15.47:0/3150478276 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f66f0042c20 con 0x7f66f4066f70 2022-06-29T15:51:08.090 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.083+0000 7f66fbb85700 1 -- 172.21.15.47:0/3150478276 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f66f40b5e50 con 0x7f66f4066f70 2022-06-29T15:51:08.806 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.799+0000 7f66e2ffd700 1 -- 172.21.15.47:0/3150478276 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v130) v1 ==== 185+0+0 (secure 0 0 0) 0x7f66f0046240 con 0x7f66f4066f70 2022-06-29T15:51:08.811 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.803+0000 7f66fbb85700 1 -- 172.21.15.47:0/3150478276 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f66d4053980 msgr2=0x7f66d4055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:08.811 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.803+0000 7f66fbb85700 1 --2- 172.21.15.47:0/3150478276 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f66d4053980 0x7f66d4055e40 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7f66e401a000 tx=0x7f66e4014000).stop 2022-06-29T15:51:08.811 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.803+0000 7f66fbb85700 1 -- 172.21.15.47:0/3150478276 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66f4066f70 msgr2=0x7f66f41361c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:08.811 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.803+0000 7f66fbb85700 1 --2- 172.21.15.47:0/3150478276 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66f4066f70 0x7f66f41361c0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 rx=0x7f66f000c000 tx=0x7f66f0008040).stop 2022-06-29T15:51:08.812 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.803+0000 7f66fbb85700 1 -- 172.21.15.47:0/3150478276 shutdown_connections 2022-06-29T15:51:08.812 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.803+0000 7f66fbb85700 1 --2- 172.21.15.47:0/3150478276 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f66f40b4300 0x7f66f41366b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:08.812 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.803+0000 7f66fbb85700 1 --2- 172.21.15.47:0/3150478276 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66f4066f70 0x7f66f41361c0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:08.812 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.803+0000 7f66fbb85700 1 --2- 172.21.15.47:0/3150478276 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f66d4053980 0x7f66d4055e40 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:08.813 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.803+0000 7f66fbb85700 1 --2- 172.21.15.47:0/3150478276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66f4060ee0 0x7f66f40b6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:08.813 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.803+0000 7f66fbb85700 1 -- 172.21.15.47:0/3150478276 >> 172.21.15.47:0/3150478276 conn(0x7f66f405c510 msgr2=0x7f66f40b88a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:08.813 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.803+0000 7f66fbb85700 1 -- 172.21.15.47:0/3150478276 shutdown_connections 2022-06-29T15:51:08.813 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.803+0000 7f66fbb85700 1 -- 172.21.15.47:0/3150478276 wait complete. 2022-06-29T15:51:08.814 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' removed 2022-06-29T15:51:08.825 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-06-29T15:51:08.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.923+0000 7f47de056700 1 Processor -- start 2022-06-29T15:51:08.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.923+0000 7f47de056700 1 -- start start 2022-06-29T15:51:08.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.927+0000 7f47de056700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f47d805f350 0x7f47d80a9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:08.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.927+0000 7f47de056700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f47d8063100 0x7f47d813e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:08.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.927+0000 7f47de056700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47d80b7100 con 0x7f47d80b6d20 2022-06-29T15:51:08.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.927+0000 7f47de056700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47d805fdc0 con 0x7f47d805f350 2022-06-29T15:51:08.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.927+0000 7f47de056700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47d8141db0 con 0x7f47d8063100 2022-06-29T15:51:08.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.927+0000 7f47dd054700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f47d80b6d20 0x7f47d80612b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38772/0 (socket says 172.21.15.47:38772) 2022-06-29T15:51:08.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.927+0000 7f47dd054700 1 -- 172.21.15.47:0/408122033 learned_addr learned my addr 172.21.15.47:0/408122033 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:08.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.927+0000 7f47dc853700 1 --2- 172.21.15.47:0/408122033 >> v2:172.21.15.47:3300/0 conn(0x7f47d805f350 0x7f47d80a9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:08.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.927+0000 7f47dd855700 1 --2- 172.21.15.47:0/408122033 >> v2:172.21.15.64:3300/0 conn(0x7f47d8063100 0x7f47d813e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:08.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.927+0000 7f47c7fff700 1 -- 172.21.15.47:0/408122033 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3736711320 0 0) 0x7f47d80b7100 con 0x7f47d80b6d20 2022-06-29T15:51:08.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.927+0000 7f47c7fff700 1 -- 172.21.15.47:0/408122033 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47c0003260 con 0x7f47d80b6d20 2022-06-29T15:51:08.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.927+0000 7f47dc853700 1 -- 172.21.15.47:0/408122033 >> v2:172.21.15.64:3300/0 conn(0x7f47d8063100 msgr2=0x7f47d813e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:08.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.927+0000 7f47dc853700 1 --2- 172.21.15.47:0/408122033 >> v2:172.21.15.64:3300/0 conn(0x7f47d8063100 0x7f47d813e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:08.950 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.927+0000 7f47dc853700 1 -- 172.21.15.47:0/408122033 >> v1:172.21.15.47:6789/0 conn(0x7f47d80b6d20 legacy=0x7f47d80612b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:08.950 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.927+0000 7f47dc853700 1 -- 172.21.15.47:0/408122033 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d80619c0 con 0x7f47d805f350 2022-06-29T15:51:08.951 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.927+0000 7f47dc853700 1 --2- 172.21.15.47:0/408122033 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47d805f350 0x7f47d80a9d70 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 rx=0x7f47c8002f70 tx=0x7f47c8006df0).ready entity=mon.0 client_cookie=2270e933bbf96a03 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:08.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.927+0000 7f47c7fff700 1 -- 172.21.15.47:0/408122033 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47c800d4f0 con 0x7f47d805f350 2022-06-29T15:51:08.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.927+0000 7f47c7fff700 1 -- 172.21.15.47:0/408122033 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c8000e90 con 0x7f47d805f350 2022-06-29T15:51:08.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.927+0000 7f47c7fff700 1 -- 172.21.15.47:0/408122033 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47c800d840 con 0x7f47d805f350 2022-06-29T15:51:08.953 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.927+0000 7f47de056700 1 -- 172.21.15.47:0/408122033 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47d805f350 msgr2=0x7f47d80a9d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:08.953 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.927+0000 7f47de056700 1 --2- 172.21.15.47:0/408122033 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47d805f350 0x7f47d80a9d70 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 rx=0x7f47c8002f70 tx=0x7f47c8006df0).stop 2022-06-29T15:51:08.954 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.931+0000 7f47de056700 1 -- 172.21.15.47:0/408122033 shutdown_connections 2022-06-29T15:51:08.954 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.931+0000 7f47de056700 1 --2- 172.21.15.47:0/408122033 >> v2:172.21.15.64:3300/0 conn(0x7f47d8063100 0x7f47d813e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:08.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.931+0000 7f47de056700 1 --2- 172.21.15.47:0/408122033 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47d805f350 0x7f47d80a9d70 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:08.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.931+0000 7f47de056700 1 -- 172.21.15.47:0/408122033 >> 172.21.15.47:0/408122033 conn(0x7f47d8063670 msgr2=0x7f47d8063a50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:08.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.931+0000 7f47de056700 1 -- 172.21.15.47:0/408122033 shutdown_connections 2022-06-29T15:51:08.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.931+0000 7f47de056700 1 -- 172.21.15.47:0/408122033 wait complete. 2022-06-29T15:51:08.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.931+0000 7f47de056700 1 Processor -- start 2022-06-29T15:51:08.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.931+0000 7f47de056700 1 -- start start 2022-06-29T15:51:08.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.931+0000 7f47de056700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47d8063100 0x7f47d80aa610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:08.958 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.931+0000 7f47de056700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47d80b6d20 0x7f47d80aeac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:08.958 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.931+0000 7f47de056700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47d80b02d0 0x7f47d80aefb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:08.959 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.931+0000 7f47de056700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d8142690 con 0x7f47d80b6d20 2022-06-29T15:51:08.959 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.931+0000 7f47de056700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d8151320 con 0x7f47d80b02d0 2022-06-29T15:51:08.960 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.931+0000 7f47de056700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47d8150e10 con 0x7f47d8063100 2022-06-29T15:51:08.960 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.931+0000 7f47dc853700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47d80b6d20 0x7f47d80aeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:08.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.931+0000 7f47dc853700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47d80b6d20 0x7f47d80aeac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53226/0 (socket says 172.21.15.47:53226) 2022-06-29T15:51:08.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.931+0000 7f47dc853700 1 -- 172.21.15.47:0/2089589678 learned_addr learned my addr 172.21.15.47:0/2089589678 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:08.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.931+0000 7f47dd855700 1 --2- 172.21.15.47:0/2089589678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47d80b02d0 0x7f47d80aefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:08.962 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.931+0000 7f47dd054700 1 --2- 172.21.15.47:0/2089589678 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47d8063100 0x7f47d80aa610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:08.962 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.931+0000 7f47dc853700 1 -- 172.21.15.47:0/2089589678 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47d8063100 msgr2=0x7f47d80aa610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:08.963 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.931+0000 7f47dc853700 1 --2- 172.21.15.47:0/2089589678 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47d8063100 0x7f47d80aa610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:08.963 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.931+0000 7f47dc853700 1 -- 172.21.15.47:0/2089589678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47d80b02d0 msgr2=0x7f47d80aefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:08.964 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.931+0000 7f47dc853700 1 --2- 172.21.15.47:0/2089589678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47d80b02d0 0x7f47d80aefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:08.964 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.931+0000 7f47dc853700 1 -- 172.21.15.47:0/2089589678 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c800b180 con 0x7f47d80b6d20 2022-06-29T15:51:08.965 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.931+0000 7f47dc853700 1 --2- 172.21.15.47:0/2089589678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47d80b6d20 0x7f47d80aeac0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 rx=0x7f47d81427d0 tx=0x7f47c8014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:08.965 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.931+0000 7f47c67fc700 1 -- 172.21.15.47:0/2089589678 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47c8013d20 con 0x7f47d80b6d20 2022-06-29T15:51:08.965 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.931+0000 7f47c67fc700 1 -- 172.21.15.47:0/2089589678 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c8013e80 con 0x7f47d80b6d20 2022-06-29T15:51:08.966 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.931+0000 7f47de056700 1 -- 172.21.15.47:0/2089589678 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47d8142aa0 con 0x7f47d80b6d20 2022-06-29T15:51:08.966 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.931+0000 7f47c67fc700 1 -- 172.21.15.47:0/2089589678 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47c801e5f0 con 0x7f47d80b6d20 2022-06-29T15:51:08.967 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.931+0000 7f47de056700 1 -- 172.21.15.47:0/2089589678 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47d8151080 con 0x7f47d80b6d20 2022-06-29T15:51:08.967 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.935+0000 7f47c67fc700 1 -- 172.21.15.47:0/2089589678 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f47c801c070 con 0x7f47d80b6d20 2022-06-29T15:51:08.968 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.935+0000 7f47de056700 1 -- 172.21.15.47:0/2089589678 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47d80573d0 con 0x7f47d80b6d20 2022-06-29T15:51:08.968 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.935+0000 7f47c67fc700 1 --2- 172.21.15.47:0/2089589678 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f47b8053950 0x7f47b8055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:08.969 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.935+0000 7f47c67fc700 1 -- 172.21.15.47:0/2089589678 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6068+0+0 (secure 0 0 0) 0x7f47c806c720 con 0x7f47d80b6d20 2022-06-29T15:51:08.969 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.935+0000 7f47dd054700 1 --2- 172.21.15.47:0/2089589678 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f47b8053950 0x7f47b8055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:08.969 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.935+0000 7f47dd054700 1 --2- 172.21.15.47:0/2089589678 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f47b8053950 0x7f47b8055e10 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7f47d400d000 tx=0x7f47d4009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:08.971 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:08.943+0000 7f47c67fc700 1 -- 172.21.15.47:0/2089589678 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f47c80435a0 con 0x7f47d80b6d20 2022-06-29T15:51:09.079 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.071+0000 7f47de056700 1 -- 172.21.15.47:0/2089589678 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f47d80b7300 con 0x7f47d80b6d20 2022-06-29T15:51:09.815 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.807+0000 7f47c67fc700 1 -- 172.21.15.47:0/2089589678 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v131) v1 ==== 133+0+0 (secure 0 0 0) 0x7f47c80230a0 con 0x7f47d80b6d20 2022-06-29T15:51:09.819 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.811+0000 7f47de056700 1 -- 172.21.15.47:0/2089589678 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f47b8053950 msgr2=0x7f47b8055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:09.819 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.811+0000 7f47de056700 1 --2- 172.21.15.47:0/2089589678 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f47b8053950 0x7f47b8055e10 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7f47d400d000 tx=0x7f47d4009040).stop 2022-06-29T15:51:09.819 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.811+0000 7f47de056700 1 -- 172.21.15.47:0/2089589678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47d80b6d20 msgr2=0x7f47d80aeac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:09.820 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.811+0000 7f47de056700 1 --2- 172.21.15.47:0/2089589678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47d80b6d20 0x7f47d80aeac0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 rx=0x7f47d81427d0 tx=0x7f47c8014000).stop 2022-06-29T15:51:09.820 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.811+0000 7f47de056700 1 -- 172.21.15.47:0/2089589678 shutdown_connections 2022-06-29T15:51:09.820 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.811+0000 7f47de056700 1 --2- 172.21.15.47:0/2089589678 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f47b8053950 0x7f47b8055e10 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:09.820 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.811+0000 7f47de056700 1 --2- 172.21.15.47:0/2089589678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47d80b02d0 0x7f47d80aefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:09.821 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.811+0000 7f47de056700 1 --2- 172.21.15.47:0/2089589678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47d80b6d20 0x7f47d80aeac0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:09.821 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.811+0000 7f47de056700 1 --2- 172.21.15.47:0/2089589678 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47d8063100 0x7f47d80aa610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:09.821 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.811+0000 7f47de056700 1 -- 172.21.15.47:0/2089589678 >> 172.21.15.47:0/2089589678 conn(0x7f47d8063670 msgr2=0x7f47d813ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:09.821 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.811+0000 7f47de056700 1 -- 172.21.15.47:0/2089589678 shutdown_connections 2022-06-29T15:51:09.822 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.811+0000 7f47de056700 1 -- 172.21.15.47:0/2089589678 wait complete. 2022-06-29T15:51:09.822 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' created 2022-06-29T15:51:09.837 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2022-06-29T15:51:09.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.939+0000 7f35ab8fd700 1 Processor -- start 2022-06-29T15:51:09.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.939+0000 7f35ab8fd700 1 -- start start 2022-06-29T15:51:09.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.939+0000 7f35ab8fd700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f35a4140940 0x7f35a405f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:09.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.939+0000 7f35ab8fd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f35a405f5a0 0x7f35a40b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:09.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.939+0000 7f35ab8fd700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35a40b4e50 con 0x7f35a40b35d0 2022-06-29T15:51:09.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.939+0000 7f35ab8fd700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35a40621e0 con 0x7f35a405f5a0 2022-06-29T15:51:09.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.939+0000 7f35ab8fd700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35a4143170 con 0x7f35a4140940 2022-06-29T15:51:09.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.939+0000 7f35a9e9a700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f35a40b35d0 0x7f35a41466e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38790/0 (socket says 172.21.15.47:38790) 2022-06-29T15:51:09.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.939+0000 7f35a9e9a700 1 -- 172.21.15.47:0/152995081 learned_addr learned my addr 172.21.15.47:0/152995081 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:09.951 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.939+0000 7f35a8e98700 1 --2- 172.21.15.47:0/152995081 >> v2:172.21.15.64:3300/0 conn(0x7f35a405f5a0 0x7f35a40b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:09.951 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.939+0000 7f35a9699700 1 --2- 172.21.15.47:0/152995081 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f35a4140940 0x7f35a405f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:09.951 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.943+0000 7f358ffff700 1 -- 172.21.15.47:0/152995081 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1970443049 0 0) 0x7f35a40b4e50 con 0x7f35a40b35d0 2022-06-29T15:51:09.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.943+0000 7f358ffff700 1 -- 172.21.15.47:0/152995081 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3588003260 con 0x7f35a40b35d0 2022-06-29T15:51:09.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.943+0000 7f35a8e98700 1 -- 172.21.15.47:0/152995081 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f35a4140940 msgr2=0x7f35a405f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:09.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.943+0000 7f35a8e98700 1 --2- 172.21.15.47:0/152995081 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f35a4140940 0x7f35a405f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:09.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.943+0000 7f35a8e98700 1 -- 172.21.15.47:0/152995081 >> v1:172.21.15.47:6789/0 conn(0x7f35a40b35d0 legacy=0x7f35a41466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:09.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.943+0000 7f35a8e98700 1 -- 172.21.15.47:0/152995081 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a40b4bb0 con 0x7f35a405f5a0 2022-06-29T15:51:09.953 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.943+0000 7f35a8e98700 1 --2- 172.21.15.47:0/152995081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35a405f5a0 0x7f35a40b30e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7f3598002f70 tx=0x7f3598006df0).ready entity=mon.1 client_cookie=3adaa17663d30f6c server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:09.953 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.943+0000 7f358ffff700 1 -- 172.21.15.47:0/152995081 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f359800d4a0 con 0x7f35a405f5a0 2022-06-29T15:51:09.953 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.943+0000 7f358ffff700 1 -- 172.21.15.47:0/152995081 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3598000e90 con 0x7f35a405f5a0 2022-06-29T15:51:09.953 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.943+0000 7f358ffff700 1 -- 172.21.15.47:0/152995081 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f359800d7f0 con 0x7f35a405f5a0 2022-06-29T15:51:09.954 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.943+0000 7f35ab8fd700 1 -- 172.21.15.47:0/152995081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35a405f5a0 msgr2=0x7f35a40b30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:09.954 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.943+0000 7f35ab8fd700 1 --2- 172.21.15.47:0/152995081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35a405f5a0 0x7f35a40b30e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7f3598002f70 tx=0x7f3598006df0).stop 2022-06-29T15:51:09.954 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.943+0000 7f35ab8fd700 1 -- 172.21.15.47:0/152995081 shutdown_connections 2022-06-29T15:51:09.954 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.943+0000 7f35ab8fd700 1 --2- 172.21.15.47:0/152995081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35a405f5a0 0x7f35a40b30e0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:09.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.943+0000 7f35ab8fd700 1 --2- 172.21.15.47:0/152995081 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f35a4140940 0x7f35a405f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:09.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.943+0000 7f35ab8fd700 1 -- 172.21.15.47:0/152995081 >> 172.21.15.47:0/152995081 conn(0x7f35a40627d0 msgr2=0x7f35a4062bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:09.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.943+0000 7f35ab8fd700 1 -- 172.21.15.47:0/152995081 shutdown_connections 2022-06-29T15:51:09.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.943+0000 7f35ab8fd700 1 -- 172.21.15.47:0/152995081 wait complete. 2022-06-29T15:51:09.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.943+0000 7f35ab8fd700 1 Processor -- start 2022-06-29T15:51:09.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.947+0000 7f35ab8fd700 1 -- start start 2022-06-29T15:51:09.958 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.947+0000 7f35ab8fd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35a40b35d0 0x7f35a413f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:09.959 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.947+0000 7f35ab8fd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35a4140940 0x7f35a413fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:09.959 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.947+0000 7f35ab8fd700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f35a413b500 0x7f35a413c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:09.959 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.947+0000 7f35ab8fd700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a40b0a90 con 0x7f35a4140940 2022-06-29T15:51:09.959 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.947+0000 7f35ab8fd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a40b0d30 con 0x7f35a40b35d0 2022-06-29T15:51:09.960 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.947+0000 7f35ab8fd700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35a40b0fd0 con 0x7f35a413b500 2022-06-29T15:51:09.960 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.947+0000 7f35a8e98700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35a4140940 0x7f35a413fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:09.960 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.947+0000 7f35a8e98700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35a4140940 0x7f35a413fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53240/0 (socket says 172.21.15.47:53240) 2022-06-29T15:51:09.960 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.947+0000 7f35a8e98700 1 -- 172.21.15.47:0/3592337076 learned_addr learned my addr 172.21.15.47:0/3592337076 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:09.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.947+0000 7f35a9699700 1 --2- 172.21.15.47:0/3592337076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35a40b35d0 0x7f35a413f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:09.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.947+0000 7f35a9e9a700 1 --2- 172.21.15.47:0/3592337076 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f35a413b500 0x7f35a413c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:09.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.947+0000 7f35a8e98700 1 -- 172.21.15.47:0/3592337076 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f35a413b500 msgr2=0x7f35a413c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:09.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.947+0000 7f35a8e98700 1 --2- 172.21.15.47:0/3592337076 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f35a413b500 0x7f35a413c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:09.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.947+0000 7f35a8e98700 1 -- 172.21.15.47:0/3592337076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35a40b35d0 msgr2=0x7f35a413f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:09.962 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.947+0000 7f35a8e98700 1 --2- 172.21.15.47:0/3592337076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35a40b35d0 0x7f35a413f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:09.962 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.947+0000 7f35a8e98700 1 -- 172.21.15.47:0/3592337076 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f359800b180 con 0x7f35a4140940 2022-06-29T15:51:09.962 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.947+0000 7f35a8e98700 1 --2- 172.21.15.47:0/3592337076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35a4140940 0x7f35a413fb40 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 rx=0x7f3598002f70 tx=0x7f3598014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:09.962 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.947+0000 7f3596ffd700 1 -- 172.21.15.47:0/3592337076 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3598016070 con 0x7f35a4140940 2022-06-29T15:51:09.963 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.947+0000 7f3596ffd700 1 -- 172.21.15.47:0/3592337076 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35980137d0 con 0x7f35a4140940 2022-06-29T15:51:09.963 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.947+0000 7f3596ffd700 1 -- 172.21.15.47:0/3592337076 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3598013b30 con 0x7f35a4140940 2022-06-29T15:51:09.963 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.947+0000 7f35ab8fd700 1 -- 172.21.15.47:0/3592337076 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35a40b1270 con 0x7f35a4140940 2022-06-29T15:51:09.963 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.947+0000 7f35ab8fd700 1 -- 172.21.15.47:0/3592337076 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35a413c560 con 0x7f35a4140940 2022-06-29T15:51:09.964 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.951+0000 7f35ab8fd700 1 -- 172.21.15.47:0/3592337076 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35a40b1400 con 0x7f35a4140940 2022-06-29T15:51:09.964 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.951+0000 7f3596ffd700 1 -- 172.21.15.47:0/3592337076 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f359800f030 con 0x7f35a4140940 2022-06-29T15:51:09.964 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.951+0000 7f3596ffd700 1 --2- 172.21.15.47:0/3592337076 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3580053950 0x7f3580055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:09.964 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.951+0000 7f3596ffd700 1 -- 172.21.15.47:0/3592337076 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6422+0+0 (secure 0 0 0) 0x7f359806ced0 con 0x7f35a4140940 2022-06-29T15:51:09.964 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.951+0000 7f35a9699700 1 --2- 172.21.15.47:0/3592337076 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3580053950 0x7f3580055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:09.965 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.955+0000 7f35a9699700 1 --2- 172.21.15.47:0/3592337076 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3580053950 0x7f3580055e10 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7f35a000b000 tx=0x7f35a0006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:09.965 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:09.959+0000 7f3596ffd700 1 -- 172.21.15.47:0/3592337076 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3598043c90 con 0x7f35a4140940 2022-06-29T15:51:10.132 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.127+0000 7f35ab8fd700 1 -- 172.21.15.47:0/3592337076 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f35a40573d0 con 0x7f35a4140940 2022-06-29T15:51:10.175 INFO:tasks.ceph.mon.a.smithi047.stderr:2022-06-29T15:51:10.127+0000 7ff9cd037700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-06-29T15:51:10.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.135+0000 7f3596ffd700 1 -- 172.21.15.47:0/3592337076 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 24 and data pool 22 v62) v1 ==== 174+0+0 (secure 0 0 0) 0x7f3598023070 con 0x7f35a4140940 2022-06-29T15:51:10.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.139+0000 7f35ab8fd700 1 -- 172.21.15.47:0/3592337076 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3580053950 msgr2=0x7f3580055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:10.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.139+0000 7f35ab8fd700 1 --2- 172.21.15.47:0/3592337076 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3580053950 0x7f3580055e10 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7f35a000b000 tx=0x7f35a0006df0).stop 2022-06-29T15:51:10.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.139+0000 7f35ab8fd700 1 -- 172.21.15.47:0/3592337076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35a4140940 msgr2=0x7f35a413fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:10.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.139+0000 7f35ab8fd700 1 --2- 172.21.15.47:0/3592337076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35a4140940 0x7f35a413fb40 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 rx=0x7f3598002f70 tx=0x7f3598014000).stop 2022-06-29T15:51:10.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.143+0000 7f35ab8fd700 1 -- 172.21.15.47:0/3592337076 shutdown_connections 2022-06-29T15:51:10.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.143+0000 7f35ab8fd700 1 --2- 172.21.15.47:0/3592337076 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f35a413b500 0x7f35a413c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:10.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.143+0000 7f35ab8fd700 1 --2- 172.21.15.47:0/3592337076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35a4140940 0x7f35a413fb40 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:10.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.143+0000 7f35ab8fd700 1 --2- 172.21.15.47:0/3592337076 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3580053950 0x7f3580055e10 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:10.182 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.143+0000 7f35ab8fd700 1 --2- 172.21.15.47:0/3592337076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35a40b35d0 0x7f35a413f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:10.182 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.143+0000 7f35ab8fd700 1 -- 172.21.15.47:0/3592337076 >> 172.21.15.47:0/3592337076 conn(0x7f35a40627d0 msgr2=0x7f35a405ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:10.182 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.143+0000 7f35ab8fd700 1 -- 172.21.15.47:0/3592337076 shutdown_connections 2022-06-29T15:51:10.183 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.143+0000 7f35ab8fd700 1 -- 172.21.15.47:0/3592337076 wait complete. 2022-06-29T15:51:10.183 INFO:teuthology.orchestra.run.smithi047.stderr:new fs with metadata pool 24 and data pool 22 2022-06-29T15:51:10.183 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-06-29T15:51:10.184 DEBUG:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:51:10.204 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:51:10.227 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:51:10.249 DEBUG:teuthology.orchestra.run.smithi164:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:51:10.270 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-06-29T15:51:10.271 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:51:10.272 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:51:10.272 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:51:10.279 INFO:teuthology.orchestra.run.smithi099.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-06-29T15:51:10.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T15:51:10.281 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-06-29T15:51:10.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-29T15:51:10.282 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:51:10.328 INFO:teuthology.orchestra.run.smithi099.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-06-29T15:51:10.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T15:51:10.330 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:51:10.330 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:51:10.331 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:51:10.338 INFO:teuthology.orchestra.run.smithi164.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-06-29T15:51:10.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T15:51:10.341 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-06-29T15:51:10.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-29T15:51:10.342 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:51:10.348 INFO:teuthology.orchestra.run.smithi164.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-06-29T15:51:10.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T15:51:10.350 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-06-29T15:51:10.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.455+0000 7f0242225700 1 Processor -- start 2022-06-29T15:51:10.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.455+0000 7f0242225700 1 -- start start 2022-06-29T15:51:10.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.455+0000 7f0242225700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f023c05c8f0 0x7f023c13cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:10.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.455+0000 7f0242225700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f023c142f90 con 0x7f023c143bd0 2022-06-29T15:51:10.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.455+0000 7f0242225700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f023c1431f0 con 0x7f023c05c510 2022-06-29T15:51:10.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.455+0000 7f0242225700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f023c143490 con 0x7f023c05c8f0 2022-06-29T15:51:10.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.455+0000 7f0241a24700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f023c05c8f0 0x7f023c13cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:10.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.455+0000 7f0240a22700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f023c143bd0 0x7f023c0b2c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38800/0 (socket says 172.21.15.47:38800) 2022-06-29T15:51:10.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.455+0000 7f0240a22700 1 -- 172.21.15.47:0/4018511697 learned_addr learned my addr 172.21.15.47:0/4018511697 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:10.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.455+0000 7f0241a24700 1 -- 172.21.15.47:0/4018511697 >> v1:172.21.15.64:6789/0 conn(0x7f023c05c510 legacy=0x7f023c0b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:10.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.455+0000 7f0241a24700 1 -- 172.21.15.47:0/4018511697 >> v1:172.21.15.47:6789/0 conn(0x7f023c143bd0 legacy=0x7f023c0b2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:10.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.455+0000 7f0241a24700 1 -- 172.21.15.47:0/4018511697 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f023c143730 con 0x7f023c05c8f0 2022-06-29T15:51:10.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.455+0000 7f0241a24700 1 --2- 172.21.15.47:0/4018511697 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f023c05c8f0 0x7f023c13cbf0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 rx=0x7f0230002f70 tx=0x7f0230006df0).ready entity=mon.0 client_cookie=9f26083a7736bf33 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:10.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.455+0000 7f023bfff700 1 -- 172.21.15.47:0/4018511697 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f023000d610 con 0x7f023c05c8f0 2022-06-29T15:51:10.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.455+0000 7f023bfff700 1 -- 172.21.15.47:0/4018511697 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0230000e90 con 0x7f023c05c8f0 2022-06-29T15:51:10.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.455+0000 7f023bfff700 1 -- 172.21.15.47:0/4018511697 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f023000d960 con 0x7f023c05c8f0 2022-06-29T15:51:10.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.459+0000 7f0242225700 1 -- 172.21.15.47:0/4018511697 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f023c05c8f0 msgr2=0x7f023c13cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:10.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.459+0000 7f0242225700 1 --2- 172.21.15.47:0/4018511697 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f023c05c8f0 0x7f023c13cbf0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 rx=0x7f0230002f70 tx=0x7f0230006df0).stop 2022-06-29T15:51:10.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.459+0000 7f0242225700 1 -- 172.21.15.47:0/4018511697 shutdown_connections 2022-06-29T15:51:10.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.459+0000 7f0242225700 1 --2- 172.21.15.47:0/4018511697 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f023c05c8f0 0x7f023c13cbf0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:10.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.459+0000 7f0242225700 1 -- 172.21.15.47:0/4018511697 >> 172.21.15.47:0/4018511697 conn(0x7f023c060e00 msgr2=0x7f023c0611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:10.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.459+0000 7f0242225700 1 -- 172.21.15.47:0/4018511697 shutdown_connections 2022-06-29T15:51:10.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.459+0000 7f0242225700 1 -- 172.21.15.47:0/4018511697 wait complete. 2022-06-29T15:51:10.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.459+0000 7f0242225700 1 Processor -- start 2022-06-29T15:51:10.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.459+0000 7f0242225700 1 -- start start 2022-06-29T15:51:10.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.463+0000 7f0242225700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f023c05c510 0x7f023c141680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:10.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.463+0000 7f0242225700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f023c05c8f0 0x7f023c1523a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:10.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.463+0000 7f0242225700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f023c143bd0 0x7f023c152890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:10.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.463+0000 7f0242225700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023c142ad0 con 0x7f023c05c8f0 2022-06-29T15:51:10.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.463+0000 7f0242225700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023c152d80 con 0x7f023c05c510 2022-06-29T15:51:10.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.463+0000 7f0242225700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f023c147140 con 0x7f023c143bd0 2022-06-29T15:51:10.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.463+0000 7f0240a22700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f023c05c8f0 0x7f023c1523a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:10.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.463+0000 7f0240a22700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f023c05c8f0 0x7f023c1523a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53254/0 (socket says 172.21.15.47:53254) 2022-06-29T15:51:10.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.463+0000 7f0240a22700 1 -- 172.21.15.47:0/3293753409 learned_addr learned my addr 172.21.15.47:0/3293753409 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:10.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.463+0000 7f0241a24700 1 --2- 172.21.15.47:0/3293753409 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f023c143bd0 0x7f023c152890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:10.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.463+0000 7f0241223700 1 --2- 172.21.15.47:0/3293753409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f023c05c510 0x7f023c141680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:10.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.463+0000 7f0240a22700 1 -- 172.21.15.47:0/3293753409 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f023c143bd0 msgr2=0x7f023c152890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:10.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.463+0000 7f0240a22700 1 --2- 172.21.15.47:0/3293753409 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f023c143bd0 0x7f023c152890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:10.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.463+0000 7f0240a22700 1 -- 172.21.15.47:0/3293753409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f023c05c510 msgr2=0x7f023c141680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:10.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.463+0000 7f0240a22700 1 --2- 172.21.15.47:0/3293753409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f023c05c510 0x7f023c141680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:10.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.463+0000 7f0240a22700 1 -- 172.21.15.47:0/3293753409 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f023000b180 con 0x7f023c05c8f0 2022-06-29T15:51:10.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.463+0000 7f0241a24700 1 --2- 172.21.15.47:0/3293753409 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f023c143bd0 0x7f023c152890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:51:10.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.463+0000 7f0240a22700 1 --2- 172.21.15.47:0/3293753409 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f023c05c8f0 0x7f023c1523a0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 rx=0x7f023c0b7300 tx=0x7f02340014a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:10.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.463+0000 7f023a7fc700 1 -- 172.21.15.47:0/3293753409 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0234005b60 con 0x7f023c05c8f0 2022-06-29T15:51:10.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.463+0000 7f023a7fc700 1 -- 172.21.15.47:0/3293753409 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0234005cc0 con 0x7f023c05c8f0 2022-06-29T15:51:10.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.463+0000 7f0242225700 1 -- 172.21.15.47:0/3293753409 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f023c0ae010 con 0x7f023c05c8f0 2022-06-29T15:51:10.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.463+0000 7f023a7fc700 1 -- 172.21.15.47:0/3293753409 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0234007300 con 0x7f023c05c8f0 2022-06-29T15:51:10.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.463+0000 7f0242225700 1 -- 172.21.15.47:0/3293753409 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f023c0ae550 con 0x7f023c05c8f0 2022-06-29T15:51:10.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.467+0000 7f0242225700 1 -- 172.21.15.47:0/3293753409 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f023c0573d0 con 0x7f023c05c8f0 2022-06-29T15:51:10.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.467+0000 7f023a7fc700 1 -- 172.21.15.47:0/3293753409 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0234010030 con 0x7f023c05c8f0 2022-06-29T15:51:10.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.467+0000 7f023a7fc700 1 --2- 172.21.15.47:0/3293753409 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f021c053b00 0x7f021c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:10.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.467+0000 7f023a7fc700 1 -- 172.21.15.47:0/3293753409 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f023406c1b0 con 0x7f023c05c8f0 2022-06-29T15:51:10.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.467+0000 7f0241223700 1 --2- 172.21.15.47:0/3293753409 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f021c053b00 0x7f021c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:10.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.467+0000 7f0241223700 1 --2- 172.21.15.47:0/3293753409 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f021c053b00 0x7f021c055fc0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7f022c00b000 tx=0x7f022c009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:10.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.471+0000 7f023a7fc700 1 -- 172.21.15.47:0/3293753409 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0234042db0 con 0x7f023c05c8f0 2022-06-29T15:51:10.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.607+0000 7f0242225700 1 -- 172.21.15.47:0/3293753409 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f023c069a60 con 0x7f023c05c8f0 2022-06-29T15:51:10.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.611+0000 7f023a7fc700 1 -- 172.21.15.47:0/3293753409 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v132) v1 ==== 81+0+34218 (secure 0 0 0) 0x7f02340463d0 con 0x7f023c05c8f0 2022-06-29T15:51:10.617 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:10.617 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2022-06-29T15:51:10.618 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 132, 2022-06-29T15:51:10.618 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "9a68894a-53ce-400e-838b-14eab91b834c", 2022-06-29T15:51:10.618 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2022-06-29T15:42:14.046361+0000", 2022-06-29T15:51:10.618 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2022-06-29T15:51:10.129988+0000", 2022-06-29T15:51:10.619 INFO:teuthology.orchestra.run.smithi047.stdout: "last_up_change": "2022-06-29T15:42:20.929789+0000", 2022-06-29T15:51:10.619 INFO:teuthology.orchestra.run.smithi047.stdout: "last_in_change": "2022-06-29T15:42:20.929789+0000", 2022-06-29T15:51:10.619 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-06-29T15:51:10.619 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_num": 5799936, 2022-06-29T15:51:10.620 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_set": [ 2022-06-29T15:51:10.620 INFO:teuthology.orchestra.run.smithi047.stdout: "pglog_hardlimit", 2022-06-29T15:51:10.620 INFO:teuthology.orchestra.run.smithi047.stdout: "purged_snapdirs", 2022-06-29T15:51:10.620 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_deletes", 2022-06-29T15:51:10.621 INFO:teuthology.orchestra.run.smithi047.stdout: "sortbitwise" 2022-06-29T15:51:10.621 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:51:10.621 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_version": 5, 2022-06-29T15:51:10.621 INFO:teuthology.orchestra.run.smithi047.stdout: "full_ratio": 0.94999998807907104, 2022-06-29T15:51:10.621 INFO:teuthology.orchestra.run.smithi047.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-06-29T15:51:10.622 INFO:teuthology.orchestra.run.smithi047.stdout: "nearfull_ratio": 0.85000002384185791, 2022-06-29T15:51:10.622 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_snapshot": "", 2022-06-29T15:51:10.622 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_max": 24, 2022-06-29T15:51:10.622 INFO:teuthology.orchestra.run.smithi047.stdout: "max_osd": 8, 2022-06-29T15:51:10.623 INFO:teuthology.orchestra.run.smithi047.stdout: "require_min_compat_client": "jewel", 2022-06-29T15:51:10.623 INFO:teuthology.orchestra.run.smithi047.stdout: "min_compat_client": "jewel", 2022-06-29T15:51:10.623 INFO:teuthology.orchestra.run.smithi047.stdout: "require_osd_release": "octopus", 2022-06-29T15:51:10.623 INFO:teuthology.orchestra.run.smithi047.stdout: "pools": [ 2022-06-29T15:51:10.623 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.624 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 1, 2022-06-29T15:51:10.624 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "device_health_metrics", 2022-06-29T15:51:10.624 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:42:16.971431+0000", 2022-06-29T15:51:10.624 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:51:10.625 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:51:10.625 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:51:10.625 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:51:10.625 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:51:10.626 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:51:10.626 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:51:10.626 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:51:10.626 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 1, 2022-06-29T15:51:10.627 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 1, 2022-06-29T15:51:10.627 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 1, 2022-06-29T15:51:10.627 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 1, 2022-06-29T15:51:10.627 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 1, 2022-06-29T15:51:10.628 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:51:10.628 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:51:10.628 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:51:10.628 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:51:10.629 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:51:10.629 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:51:10.629 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:51:10.629 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.629 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "16", 2022-06-29T15:51:10.630 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:51:10.630 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:51:10.630 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:51:10.630 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:51:10.631 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:51:10.631 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:51:10.631 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:51:10.631 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:51:10.632 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:51:10.632 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:51:10.632 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:51:10.632 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:51:10.632 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:51:10.633 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:51:10.633 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:51:10.633 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:51:10.633 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:51:10.633 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:51:10.634 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:51:10.634 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:51:10.634 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:51:10.634 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:51:10.635 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:51:10.635 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:51:10.635 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:51:10.635 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:51:10.636 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.636 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:51:10.636 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:51:10.636 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:51:10.636 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:51:10.637 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:51:10.637 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:51:10.637 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:51:10.637 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:51:10.638 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:51:10.638 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:51:10.638 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:51:10.638 INFO:teuthology.orchestra.run.smithi047.stdout: "options": { 2022-06-29T15:51:10.638 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_min": 1 2022-06-29T15:51:10.639 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.639 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:51:10.639 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_devicehealth": {} 2022-06-29T15:51:10.639 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:10.640 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.640 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.640 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 2, 2022-06-29T15:51:10.640 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "rbd", 2022-06-29T15:51:10.640 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:42:25.290884+0000", 2022-06-29T15:51:10.641 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:51:10.641 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:51:10.641 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:51:10.641 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:51:10.642 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:51:10.642 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:51:10.642 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:51:10.642 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:51:10.643 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2022-06-29T15:51:10.643 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2022-06-29T15:51:10.643 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2022-06-29T15:51:10.643 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2022-06-29T15:51:10.643 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2022-06-29T15:51:10.644 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:51:10.644 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:51:10.644 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:51:10.644 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:51:10.645 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:51:10.645 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:51:10.645 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:51:10.645 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.645 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "19", 2022-06-29T15:51:10.646 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:51:10.646 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:51:10.646 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:51:10.646 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:51:10.647 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:51:10.647 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:51:10.647 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:51:10.647 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:51:10.647 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:51:10.648 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:51:10.648 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:51:10.648 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:51:10.648 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:51:10.649 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:51:10.649 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:51:10.649 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:51:10.649 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:51:10.649 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:51:10.650 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:51:10.650 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:51:10.650 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:51:10.650 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:51:10.651 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:51:10.651 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:51:10.651 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:51:10.651 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:51:10.651 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.652 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:51:10.652 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:51:10.652 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:51:10.652 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:51:10.653 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:51:10.653 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:51:10.653 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:51:10.653 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:51:10.653 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:51:10.654 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:51:10.654 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:51:10.654 INFO:teuthology.orchestra.run.smithi047.stdout: "options": {}, 2022-06-29T15:51:10.654 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:51:10.655 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd": {} 2022-06-29T15:51:10.655 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:10.655 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.655 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.655 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 22, 2022-06-29T15:51:10.656 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "cephfs_data", 2022-06-29T15:51:10.656 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:50:38.125088+0000", 2022-06-29T15:51:10.656 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:51:10.656 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:51:10.657 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:51:10.657 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:51:10.657 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:51:10.657 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:51:10.657 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:51:10.658 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:51:10.658 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2022-06-29T15:51:10.658 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2022-06-29T15:51:10.658 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2022-06-29T15:51:10.659 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2022-06-29T15:51:10.659 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2022-06-29T15:51:10.659 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:51:10.659 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:51:10.659 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:51:10.660 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:51:10.660 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:51:10.660 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:51:10.660 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:51:10.661 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.661 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "132", 2022-06-29T15:51:10.661 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:51:10.661 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:51:10.661 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:51:10.662 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:51:10.662 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:51:10.662 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:51:10.662 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:51:10.663 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:51:10.663 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:51:10.663 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:51:10.663 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:51:10.663 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:51:10.664 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:51:10.664 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:51:10.664 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:51:10.664 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:51:10.665 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:51:10.665 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:51:10.665 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:51:10.665 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:51:10.665 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:51:10.666 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:51:10.666 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:51:10.666 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:51:10.666 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:51:10.667 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:51:10.667 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.667 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:51:10.667 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:51:10.668 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:51:10.668 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:51:10.668 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:51:10.668 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:51:10.668 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:51:10.669 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:51:10.669 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:51:10.669 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:51:10.669 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:51:10.669 INFO:teuthology.orchestra.run.smithi047.stdout: "options": {}, 2022-06-29T15:51:10.670 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:51:10.670 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs": { 2022-06-29T15:51:10.670 INFO:teuthology.orchestra.run.smithi047.stdout: "data": "cephfs" 2022-06-29T15:51:10.670 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:10.671 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:10.671 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.671 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.671 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 24, 2022-06-29T15:51:10.672 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "cephfs_metadata", 2022-06-29T15:51:10.672 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:51:09.130693+0000", 2022-06-29T15:51:10.672 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 32769, 2022-06-29T15:51:10.672 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool,creating", 2022-06-29T15:51:10.672 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:51:10.673 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:51:10.673 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:51:10.673 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:51:10.673 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:51:10.674 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:51:10.674 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2022-06-29T15:51:10.674 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2022-06-29T15:51:10.674 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2022-06-29T15:51:10.674 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2022-06-29T15:51:10.675 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2022-06-29T15:51:10.675 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:51:10.675 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:51:10.675 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:51:10.676 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:51:10.676 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:51:10.676 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:51:10.676 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:51:10.676 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.677 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "132", 2022-06-29T15:51:10.677 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:51:10.677 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:51:10.677 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:51:10.678 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:51:10.678 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:51:10.678 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:51:10.678 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:51:10.678 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:51:10.679 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:51:10.679 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:51:10.679 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:51:10.679 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:51:10.680 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:51:10.680 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:51:10.680 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:51:10.680 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:51:10.680 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:51:10.681 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:51:10.681 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:51:10.681 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:51:10.681 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:51:10.682 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:51:10.682 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:51:10.682 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:51:10.682 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:51:10.689 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:51:10.690 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.690 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:51:10.690 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:51:10.691 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:51:10.691 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:51:10.691 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:51:10.691 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:51:10.692 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:51:10.692 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:51:10.692 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:51:10.692 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:51:10.693 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:51:10.693 INFO:teuthology.orchestra.run.smithi047.stdout: "options": { 2022-06-29T15:51:10.693 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_bias": 4, 2022-06-29T15:51:10.693 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_min": 16, 2022-06-29T15:51:10.694 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_priority": 5 2022-06-29T15:51:10.694 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.694 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:51:10.694 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs": { 2022-06-29T15:51:10.695 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": "cephfs" 2022-06-29T15:51:10.695 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:10.695 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:10.696 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:10.696 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:51:10.696 INFO:teuthology.orchestra.run.smithi047.stdout: "osds": [ 2022-06-29T15:51:10.696 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.697 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2022-06-29T15:51:10.697 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "e23eb023-4706-4201-bf72-4bf19c912c5d", 2022-06-29T15:51:10.697 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:51:10.697 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:51:10.697 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:51:10.698 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:51:10.698 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:51:10.698 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:51:10.699 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:51:10.699 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 131, 2022-06-29T15:51:10.699 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:51:10.699 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:51:10.700 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:51:10.700 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:10.700 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.700 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:10.700 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6818", 2022-06-29T15:51:10.701 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:51:10.701 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.701 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.701 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:10.701 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6819", 2022-06-29T15:51:10.702 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:51:10.702 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:10.702 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:10.702 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.703 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:51:10.703 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:10.703 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.703 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:10.704 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6820", 2022-06-29T15:51:10.704 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:51:10.704 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.704 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.704 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:10.705 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6821", 2022-06-29T15:51:10.705 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:51:10.705 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:10.705 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:10.706 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.706 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:51:10.706 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:10.706 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.707 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:10.707 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6824", 2022-06-29T15:51:10.707 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:51:10.707 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.707 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.708 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:10.708 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6825", 2022-06-29T15:51:10.708 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:51:10.708 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:10.709 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:10.709 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.709 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:51:10.709 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:10.709 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.710 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:10.710 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6822", 2022-06-29T15:51:10.710 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:51:10.710 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.711 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.711 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:10.711 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6823", 2022-06-29T15:51:10.711 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:51:10.712 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:10.712 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:10.712 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.712 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6819/13096", 2022-06-29T15:51:10.712 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6821/13096", 2022-06-29T15:51:10.713 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6825/13096", 2022-06-29T15:51:10.713 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6823/13096", 2022-06-29T15:51:10.713 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:51:10.713 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:51:10.714 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:51:10.714 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:10.714 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.714 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.715 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2022-06-29T15:51:10.715 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "228bb08d-2d64-43d9-9560-bfaeff154452", 2022-06-29T15:51:10.715 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:51:10.715 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:51:10.715 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:51:10.716 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:51:10.716 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:51:10.716 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:51:10.716 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:51:10.717 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 131, 2022-06-29T15:51:10.717 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:51:10.717 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:51:10.717 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:51:10.717 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:10.718 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.718 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:10.718 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6826", 2022-06-29T15:51:10.718 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:51:10.719 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.719 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.719 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:10.719 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6827", 2022-06-29T15:51:10.720 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:51:10.720 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:10.720 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:10.720 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.720 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:51:10.721 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:10.721 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.721 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:10.721 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6828", 2022-06-29T15:51:10.722 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:51:10.722 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.722 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.722 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:10.722 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6829", 2022-06-29T15:51:10.723 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:51:10.723 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:10.723 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:10.723 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.724 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:51:10.724 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:10.724 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.724 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:10.725 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6832", 2022-06-29T15:51:10.725 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:51:10.725 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.725 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.725 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:10.726 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6833", 2022-06-29T15:51:10.726 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:51:10.726 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:10.726 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:10.727 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.727 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:51:10.727 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:10.727 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.728 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:10.728 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6830", 2022-06-29T15:51:10.728 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:51:10.728 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.729 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.729 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:10.729 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6831", 2022-06-29T15:51:10.729 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:51:10.729 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:10.730 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:10.730 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.730 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6827/13100", 2022-06-29T15:51:10.730 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6829/13100", 2022-06-29T15:51:10.731 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6833/13100", 2022-06-29T15:51:10.731 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6831/13100", 2022-06-29T15:51:10.731 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:51:10.731 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:51:10.732 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:51:10.732 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:10.732 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.732 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.732 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2022-06-29T15:51:10.733 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "797a5452-1510-4640-a111-c9c2fcf707eb", 2022-06-29T15:51:10.733 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:51:10.733 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:51:10.733 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:51:10.734 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:51:10.734 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:51:10.734 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:51:10.734 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:51:10.734 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 126, 2022-06-29T15:51:10.735 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:51:10.735 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:51:10.735 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:51:10.735 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:10.736 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.736 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:10.736 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6810", 2022-06-29T15:51:10.736 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:51:10.737 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.737 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.737 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:10.737 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6811", 2022-06-29T15:51:10.737 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:51:10.738 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:10.738 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:10.738 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.738 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:51:10.739 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:10.739 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.739 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:10.739 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6812", 2022-06-29T15:51:10.740 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:51:10.740 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.740 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.740 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:10.741 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6813", 2022-06-29T15:51:10.741 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:51:10.741 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:10.741 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:10.741 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.742 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:51:10.742 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:10.742 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.742 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:10.743 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6816", 2022-06-29T15:51:10.743 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:51:10.743 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.743 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.743 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:10.744 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6817", 2022-06-29T15:51:10.744 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:51:10.744 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:10.744 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:10.745 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.745 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:51:10.745 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:10.745 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.746 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:10.746 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6814", 2022-06-29T15:51:10.746 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:51:10.746 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.747 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.747 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:10.747 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6815", 2022-06-29T15:51:10.747 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:51:10.747 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:10.748 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:10.748 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.748 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6811/13098", 2022-06-29T15:51:10.748 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6813/13098", 2022-06-29T15:51:10.749 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6817/13098", 2022-06-29T15:51:10.749 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6815/13098", 2022-06-29T15:51:10.749 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:51:10.749 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:51:10.749 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:51:10.750 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:10.750 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.750 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.750 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2022-06-29T15:51:10.751 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "55b24fd7-bb71-4a01-b3fa-a09caa772a62", 2022-06-29T15:51:10.751 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:51:10.751 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:51:10.751 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:51:10.752 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:51:10.752 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:51:10.752 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:51:10.752 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:51:10.753 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 131, 2022-06-29T15:51:10.753 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:51:10.753 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:51:10.753 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:51:10.754 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:10.754 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.754 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:10.754 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6802", 2022-06-29T15:51:10.755 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:51:10.755 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.755 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.755 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:10.756 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6803", 2022-06-29T15:51:10.756 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:51:10.756 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:10.756 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:10.756 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.757 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:51:10.757 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:10.757 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.757 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:10.758 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6804", 2022-06-29T15:51:10.758 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:51:10.758 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.758 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.758 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:10.759 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6805", 2022-06-29T15:51:10.759 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:51:10.759 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:10.759 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:10.760 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.760 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:51:10.760 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:10.760 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.760 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:10.761 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6808", 2022-06-29T15:51:10.761 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:51:10.761 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.761 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.761 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:10.762 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6809", 2022-06-29T15:51:10.762 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:51:10.762 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:10.762 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:10.763 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.763 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:51:10.763 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:10.763 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.764 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:10.764 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6806", 2022-06-29T15:51:10.764 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:51:10.764 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.764 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.765 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:10.765 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6807", 2022-06-29T15:51:10.765 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:51:10.765 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:10.765 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:10.766 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.766 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6803/13097", 2022-06-29T15:51:10.766 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6805/13097", 2022-06-29T15:51:10.766 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6809/13097", 2022-06-29T15:51:10.767 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6807/13097", 2022-06-29T15:51:10.767 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:51:10.767 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:51:10.767 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:51:10.767 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:10.768 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.768 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.768 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2022-06-29T15:51:10.768 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3", 2022-06-29T15:51:10.769 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:51:10.769 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:51:10.769 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:51:10.769 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:51:10.769 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:51:10.770 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:51:10.770 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:51:10.770 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 126, 2022-06-29T15:51:10.770 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:51:10.771 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:51:10.771 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:51:10.771 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:10.771 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.771 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:10.772 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6816", 2022-06-29T15:51:10.772 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:51:10.772 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.772 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.773 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:10.773 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6817", 2022-06-29T15:51:10.773 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:51:10.773 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:10.773 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:10.774 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.774 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:51:10.774 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:10.774 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.775 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:10.775 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6818", 2022-06-29T15:51:10.775 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:51:10.775 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.775 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.776 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:10.776 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6819", 2022-06-29T15:51:10.776 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:51:10.776 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:10.777 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:10.777 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.777 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:51:10.777 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:10.777 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.778 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:10.778 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6822", 2022-06-29T15:51:10.778 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:51:10.778 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.779 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.779 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:10.779 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6823", 2022-06-29T15:51:10.779 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:51:10.779 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:10.780 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:10.780 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.780 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:51:10.780 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:10.781 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.781 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:10.781 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6820", 2022-06-29T15:51:10.781 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:51:10.781 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.782 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.782 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:10.782 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6821", 2022-06-29T15:51:10.782 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:51:10.783 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:10.783 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:10.783 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.783 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6817/13396", 2022-06-29T15:51:10.783 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6819/13396", 2022-06-29T15:51:10.784 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6823/13396", 2022-06-29T15:51:10.784 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6821/13396", 2022-06-29T15:51:10.784 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:51:10.784 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:51:10.784 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:51:10.785 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:10.785 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.785 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.785 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2022-06-29T15:51:10.786 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "f77beebe-6cca-4244-b2d4-677825ebf919", 2022-06-29T15:51:10.786 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:51:10.786 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:51:10.786 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:51:10.786 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:51:10.787 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:51:10.787 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:51:10.787 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:51:10.787 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 131, 2022-06-29T15:51:10.788 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:51:10.788 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:51:10.788 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:51:10.788 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:10.788 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.789 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:10.789 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6800", 2022-06-29T15:51:10.789 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:51:10.789 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.790 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.790 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:10.790 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6801", 2022-06-29T15:51:10.790 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:51:10.791 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:10.791 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:10.791 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.791 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:51:10.791 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:10.792 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.792 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:10.792 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6802", 2022-06-29T15:51:10.792 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:51:10.792 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.793 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.793 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:10.793 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6803", 2022-06-29T15:51:10.793 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:51:10.794 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:10.794 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:10.794 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.794 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:51:10.794 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:10.795 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.795 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:10.795 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6806", 2022-06-29T15:51:10.795 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:51:10.796 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.796 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.796 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:10.796 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6807", 2022-06-29T15:51:10.797 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:51:10.797 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:10.797 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:10.797 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.797 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:51:10.798 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:10.798 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.798 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:10.798 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6804", 2022-06-29T15:51:10.799 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:51:10.799 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.799 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.799 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:10.799 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6805", 2022-06-29T15:51:10.800 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:51:10.800 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:10.800 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:10.800 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.800 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6801/13393", 2022-06-29T15:51:10.801 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6803/13393", 2022-06-29T15:51:10.801 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6807/13393", 2022-06-29T15:51:10.801 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6805/13393", 2022-06-29T15:51:10.801 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:51:10.802 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:51:10.802 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:51:10.802 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:10.802 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.802 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.803 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2022-06-29T15:51:10.803 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "af0ca9b3-c02c-4910-834b-bf0b8dc6713c", 2022-06-29T15:51:10.803 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:51:10.803 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:51:10.804 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:51:10.804 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:51:10.804 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:51:10.804 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:51:10.804 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:51:10.805 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 131, 2022-06-29T15:51:10.805 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:51:10.805 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:51:10.805 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:51:10.806 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:10.806 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.806 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:10.806 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6824", 2022-06-29T15:51:10.806 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:51:10.807 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.807 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.807 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:10.807 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6825", 2022-06-29T15:51:10.808 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:51:10.808 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:10.808 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:10.808 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.808 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:51:10.809 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:10.809 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.809 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:10.809 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6826", 2022-06-29T15:51:10.809 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:51:10.810 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.810 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.810 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:10.810 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6827", 2022-06-29T15:51:10.811 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:51:10.811 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:10.811 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:10.811 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.811 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:51:10.812 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:10.812 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.812 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:10.812 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6830", 2022-06-29T15:51:10.813 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:51:10.813 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.813 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.813 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:10.813 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6831", 2022-06-29T15:51:10.814 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:51:10.814 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:10.814 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:10.814 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.815 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:51:10.815 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:10.815 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.815 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:10.815 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6828", 2022-06-29T15:51:10.816 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:51:10.816 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.816 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.816 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:10.817 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6829", 2022-06-29T15:51:10.817 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:51:10.817 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:10.817 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:10.817 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.818 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6825/13395", 2022-06-29T15:51:10.818 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6827/13395", 2022-06-29T15:51:10.818 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6831/13395", 2022-06-29T15:51:10.818 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6829/13395", 2022-06-29T15:51:10.819 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:51:10.819 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:51:10.819 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:51:10.819 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:10.819 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.820 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.820 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2022-06-29T15:51:10.820 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "f19c3ca6-df88-4bde-a150-b51a53f092e1", 2022-06-29T15:51:10.820 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:51:10.821 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:51:10.821 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:51:10.821 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:51:10.821 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:51:10.821 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:51:10.822 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:51:10.822 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 131, 2022-06-29T15:51:10.822 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:51:10.822 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:51:10.823 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:51:10.823 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:10.823 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.823 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:10.823 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6808", 2022-06-29T15:51:10.824 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:51:10.824 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.824 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.824 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:10.824 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6809", 2022-06-29T15:51:10.825 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:51:10.825 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:10.825 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:10.825 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.826 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:51:10.826 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:10.826 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.826 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:10.826 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6810", 2022-06-29T15:51:10.827 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:51:10.827 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.827 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.827 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:10.828 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6811", 2022-06-29T15:51:10.828 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:51:10.828 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:10.828 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:10.828 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.829 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:51:10.829 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:10.829 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.829 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:10.830 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6814", 2022-06-29T15:51:10.830 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:51:10.830 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.831 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.831 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:10.831 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6815", 2022-06-29T15:51:10.831 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:51:10.831 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:10.832 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:10.832 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.832 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:51:10.832 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:10.833 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.833 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:10.833 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6812", 2022-06-29T15:51:10.833 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:51:10.833 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.834 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.834 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:10.834 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6813", 2022-06-29T15:51:10.834 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:51:10.835 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:10.835 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:10.835 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.835 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6809/13394", 2022-06-29T15:51:10.835 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6811/13394", 2022-06-29T15:51:10.836 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6815/13394", 2022-06-29T15:51:10.836 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6813/13394", 2022-06-29T15:51:10.836 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:51:10.836 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:51:10.837 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:51:10.837 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:10.837 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:10.837 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:51:10.837 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_xinfo": [ 2022-06-29T15:51:10.838 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.838 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2022-06-29T15:51:10.838 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:51:10.838 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:51:10.839 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:51:10.839 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:51:10.839 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:51:10.839 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.720513+0000", 2022-06-29T15:51:10.839 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:51:10.840 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.840 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.840 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2022-06-29T15:51:10.840 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:51:10.840 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:51:10.841 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:51:10.841 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:51:10.841 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:51:10.841 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.772612+0000", 2022-06-29T15:51:10.842 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:51:10.842 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.842 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.842 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2022-06-29T15:51:10.842 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:51:10.843 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:51:10.843 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:51:10.843 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:51:10.843 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:51:10.844 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.450223+0000", 2022-06-29T15:51:10.844 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:51:10.844 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.844 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.844 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2022-06-29T15:51:10.845 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:51:10.845 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:51:10.845 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:51:10.845 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:51:10.845 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:51:10.846 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.477856+0000", 2022-06-29T15:51:10.846 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:51:10.846 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.846 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.847 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2022-06-29T15:51:10.847 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:51:10.847 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:51:10.848 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:51:10.848 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:51:10.849 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:51:10.849 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.539485+0000", 2022-06-29T15:51:10.849 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:51:10.850 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.850 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.851 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2022-06-29T15:51:10.851 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:51:10.852 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:51:10.852 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:51:10.852 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:51:10.852 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:51:10.852 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.487040+0000", 2022-06-29T15:51:10.853 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:51:10.853 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.853 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.854 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2022-06-29T15:51:10.854 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:51:10.855 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:51:10.855 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:51:10.855 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:51:10.855 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:51:10.856 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.748685+0000", 2022-06-29T15:51:10.856 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:51:10.856 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.856 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:10.856 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2022-06-29T15:51:10.857 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:51:10.857 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:51:10.857 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:51:10.857 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:51:10.858 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:51:10.858 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.459516+0000", 2022-06-29T15:51:10.858 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:51:10.859 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:10.859 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:51:10.859 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap": [], 2022-06-29T15:51:10.859 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap_items": [], 2022-06-29T15:51:10.860 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_temp": [], 2022-06-29T15:51:10.860 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_temp": [], 2022-06-29T15:51:10.860 INFO:teuthology.orchestra.run.smithi047.stdout: "blacklist": { 2022-06-29T15:51:10.860 INFO:teuthology.orchestra.run.smithi047.stdout: "172.21.15.64:6833/2987151459": "2022-06-30T15:50:56.446691+0000", 2022-06-29T15:51:10.861 INFO:teuthology.orchestra.run.smithi047.stdout: "172.21.15.64:6832/2987151459": "2022-06-30T15:50:56.446691+0000" 2022-06-29T15:51:10.861 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.861 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profiles": { 2022-06-29T15:51:10.862 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs_data": { 2022-06-29T15:51:10.862 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-device-class": "", 2022-06-29T15:51:10.862 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2022-06-29T15:51:10.862 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-root": "default", 2022-06-29T15:51:10.863 INFO:teuthology.orchestra.run.smithi047.stdout: "jerasure-per-chunk-alignment": "false", 2022-06-29T15:51:10.863 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2022-06-29T15:51:10.863 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "2", 2022-06-29T15:51:10.863 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2022-06-29T15:51:10.864 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2022-06-29T15:51:10.864 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van", 2022-06-29T15:51:10.864 INFO:teuthology.orchestra.run.smithi047.stdout: "w": "8" 2022-06-29T15:51:10.864 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.864 INFO:teuthology.orchestra.run.smithi047.stdout: "default": { 2022-06-29T15:51:10.865 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2022-06-29T15:51:10.865 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2022-06-29T15:51:10.865 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "1", 2022-06-29T15:51:10.865 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2022-06-29T15:51:10.866 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2022-06-29T15:51:10.866 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van" 2022-06-29T15:51:10.866 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:10.866 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:10.867 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps_queue": [], 2022-06-29T15:51:10.867 INFO:teuthology.orchestra.run.smithi047.stdout: "new_removed_snaps": [], 2022-06-29T15:51:10.867 INFO:teuthology.orchestra.run.smithi047.stdout: "new_purged_snaps": [], 2022-06-29T15:51:10.867 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_node_flags": {}, 2022-06-29T15:51:10.868 INFO:teuthology.orchestra.run.smithi047.stdout: "device_class_flags": {} 2022-06-29T15:51:10.868 INFO:teuthology.orchestra.run.smithi047.stdout:} 2022-06-29T15:51:10.871 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.611+0000 7f0242225700 1 -- 172.21.15.47:0/3293753409 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f021c053b00 msgr2=0x7f021c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:10.871 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.611+0000 7f0242225700 1 --2- 172.21.15.47:0/3293753409 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f021c053b00 0x7f021c055fc0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7f022c00b000 tx=0x7f022c009040).stop 2022-06-29T15:51:10.871 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.611+0000 7f0242225700 1 -- 172.21.15.47:0/3293753409 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f023c05c8f0 msgr2=0x7f023c1523a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:10.872 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.611+0000 7f0242225700 1 --2- 172.21.15.47:0/3293753409 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f023c05c8f0 0x7f023c1523a0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 rx=0x7f023c0b7300 tx=0x7f02340014a0).stop 2022-06-29T15:51:10.872 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.611+0000 7f0242225700 1 -- 172.21.15.47:0/3293753409 shutdown_connections 2022-06-29T15:51:10.872 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.611+0000 7f0242225700 1 --2- 172.21.15.47:0/3293753409 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f023c143bd0 0x7f023c152890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:10.873 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.611+0000 7f0242225700 1 --2- 172.21.15.47:0/3293753409 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f023c05c8f0 0x7f023c1523a0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:10.873 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.611+0000 7f0242225700 1 --2- 172.21.15.47:0/3293753409 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f021c053b00 0x7f021c055fc0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:10.873 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.611+0000 7f0242225700 1 --2- 172.21.15.47:0/3293753409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f023c05c510 0x7f023c141680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:10.873 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.611+0000 7f0242225700 1 -- 172.21.15.47:0/3293753409 >> 172.21.15.47:0/3293753409 conn(0x7f023c060e00 msgr2=0x7f023c0b7590 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:10.873 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.611+0000 7f0242225700 1 -- 172.21.15.47:0/3293753409 shutdown_connections 2022-06-29T15:51:10.874 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.611+0000 7f0242225700 1 -- 172.21.15.47:0/3293753409 wait complete. 2022-06-29T15:51:10.875 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:51:10.985 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.978+0000 7fef16e49700 1 Processor -- start 2022-06-29T15:51:10.986 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.978+0000 7fef16e49700 1 -- start start 2022-06-29T15:51:10.988 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.978+0000 7fef16e49700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fef1005fa30 0x7fef100b1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:10.988 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.978+0000 7fef16e49700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fef100b2770 0x7fef100ae170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:10.988 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.978+0000 7fef16e49700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef10060fd0 con 0x7fef10060b90 2022-06-29T15:51:10.988 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.978+0000 7fef16e49700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef1005fe10 con 0x7fef1005fa30 2022-06-29T15:51:10.989 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.978+0000 7fef16e49700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef100600e0 con 0x7fef100b2770 2022-06-29T15:51:10.989 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.978+0000 7fef15e47700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fef1005fa30 0x7fef100b1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:10.989 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.978+0000 7fef16648700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fef100b2770 0x7fef100ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:10.989 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.978+0000 7fef16648700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fef100b2770 0x7fef100ae170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:50366/0 (socket says 172.21.15.47:50366) 2022-06-29T15:51:10.990 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.978+0000 7fef16648700 1 -- 172.21.15.47:0/2979747711 learned_addr learned my addr 172.21.15.47:0/2979747711 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:10.990 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.978+0000 7fef16648700 1 -- 172.21.15.47:0/2979747711 >> v2:172.21.15.64:3300/0 conn(0x7fef1005fa30 msgr2=0x7fef100b1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:10.990 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.978+0000 7fef16648700 1 --2- 172.21.15.47:0/2979747711 >> v2:172.21.15.64:3300/0 conn(0x7fef1005fa30 0x7fef100b1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:10.990 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.978+0000 7fef16648700 1 -- 172.21.15.47:0/2979747711 >> v1:172.21.15.64:6789/0 conn(0x7fef10060b90 legacy=0x7fef1013f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:10.991 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.982+0000 7fef16648700 1 -- 172.21.15.47:0/2979747711 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef10060380 con 0x7fef100b2770 2022-06-29T15:51:10.991 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.982+0000 7fef16648700 1 --2- 172.21.15.47:0/2979747711 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fef100b2770 0x7fef100ae170 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7fef08002f70 tx=0x7fef08006e40).ready entity=mon.2 client_cookie=bb1378910a55d9ff server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:10.991 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.982+0000 7fef14e45700 1 -- 172.21.15.47:0/2979747711 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef0800bc90 con 0x7fef100b2770 2022-06-29T15:51:10.991 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.982+0000 7fef14e45700 1 -- 172.21.15.47:0/2979747711 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef08000e90 con 0x7fef100b2770 2022-06-29T15:51:10.992 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.982+0000 7fef14e45700 1 -- 172.21.15.47:0/2979747711 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef080153f0 con 0x7fef100b2770 2022-06-29T15:51:10.995 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.982+0000 7fef16e49700 1 -- 172.21.15.47:0/2979747711 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fef100b2770 msgr2=0x7fef100ae170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:10.995 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.982+0000 7fef16e49700 1 --2- 172.21.15.47:0/2979747711 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fef100b2770 0x7fef100ae170 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7fef08002f70 tx=0x7fef08006e40).stop 2022-06-29T15:51:10.996 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.982+0000 7fef16e49700 1 -- 172.21.15.47:0/2979747711 shutdown_connections 2022-06-29T15:51:10.996 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.982+0000 7fef16e49700 1 --2- 172.21.15.47:0/2979747711 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fef100b2770 0x7fef100ae170 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:10.996 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.982+0000 7fef16e49700 1 --2- 172.21.15.47:0/2979747711 >> v2:172.21.15.64:3300/0 conn(0x7fef1005fa30 0x7fef100b1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:10.996 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.982+0000 7fef16e49700 1 -- 172.21.15.47:0/2979747711 >> 172.21.15.47:0/2979747711 conn(0x7fef10062df0 msgr2=0x7fef100631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:10.997 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.982+0000 7fef16e49700 1 -- 172.21.15.47:0/2979747711 shutdown_connections 2022-06-29T15:51:10.997 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.982+0000 7fef16e49700 1 -- 172.21.15.47:0/2979747711 wait complete. 2022-06-29T15:51:10.997 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.982+0000 7fef16e49700 1 Processor -- start 2022-06-29T15:51:10.997 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.982+0000 7fef16e49700 1 -- start start 2022-06-29T15:51:10.998 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.982+0000 7fef16e49700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef1005fa30 0x7fef10141e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:10.998 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.982+0000 7fef16e49700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fef10060b90 0x7fef10066520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:10.998 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.982+0000 7fef16e49700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef100b2770 0x7fef10063ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:10.998 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.982+0000 7fef16e49700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef101438b0 con 0x7fef1005fa30 2022-06-29T15:51:10.999 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.982+0000 7fef16e49700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef10066e10 con 0x7fef100b2770 2022-06-29T15:51:10.999 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.982+0000 7fef16e49700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef10142b00 con 0x7fef10060b90 2022-06-29T15:51:10.999 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.982+0000 7fef15e47700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef1005fa30 0x7fef10141e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:10.999 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.982+0000 7fef15e47700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef1005fa30 0x7fef10141e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53266/0 (socket says 172.21.15.47:53266) 2022-06-29T15:51:11.000 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.982+0000 7fef15e47700 1 -- 172.21.15.47:0/1495584547 learned_addr learned my addr 172.21.15.47:0/1495584547 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:11.000 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.982+0000 7fef16648700 1 --2- 172.21.15.47:0/1495584547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef100b2770 0x7fef10063ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:11.000 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.982+0000 7fef15646700 1 --2- 172.21.15.47:0/1495584547 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fef10060b90 0x7fef10066520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:11.000 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.982+0000 7fef15e47700 1 -- 172.21.15.47:0/1495584547 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fef10060b90 msgr2=0x7fef10066520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:11.001 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.982+0000 7fef15e47700 1 --2- 172.21.15.47:0/1495584547 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fef10060b90 0x7fef10066520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:11.001 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.982+0000 7fef15e47700 1 -- 172.21.15.47:0/1495584547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef100b2770 msgr2=0x7fef10063ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:11.001 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.982+0000 7fef15e47700 1 --2- 172.21.15.47:0/1495584547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef100b2770 0x7fef10063ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:11.001 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.982+0000 7fef15e47700 1 -- 172.21.15.47:0/1495584547 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef0800d4d0 con 0x7fef1005fa30 2022-06-29T15:51:11.002 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.982+0000 7fef16648700 1 --2- 172.21.15.47:0/1495584547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef100b2770 0x7fef10063ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:51:11.002 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.982+0000 7fef15646700 1 --2- 172.21.15.47:0/1495584547 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fef10060b90 0x7fef10066520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:51:11.002 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.982+0000 7fef15e47700 1 --2- 172.21.15.47:0/1495584547 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef1005fa30 0x7fef10141e00 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 rx=0x7fef00006e00 tx=0x7fef000091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:11.002 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.986+0000 7fef0effd700 1 -- 172.21.15.47:0/1495584547 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef0000c850 con 0x7fef1005fa30 2022-06-29T15:51:11.003 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.986+0000 7fef0effd700 1 -- 172.21.15.47:0/1495584547 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef0000c9b0 con 0x7fef1005fa30 2022-06-29T15:51:11.003 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.986+0000 7fef16e49700 1 -- 172.21.15.47:0/1495584547 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef10066a10 con 0x7fef1005fa30 2022-06-29T15:51:11.003 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.986+0000 7fef0effd700 1 -- 172.21.15.47:0/1495584547 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef0000cd10 con 0x7fef1005fa30 2022-06-29T15:51:11.003 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.986+0000 7fef16e49700 1 -- 172.21.15.47:0/1495584547 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef1013ece0 con 0x7fef1005fa30 2022-06-29T15:51:11.003 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.986+0000 7fef16e49700 1 -- 172.21.15.47:0/1495584547 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef10066fd0 con 0x7fef1005fa30 2022-06-29T15:51:11.004 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.986+0000 7fef0effd700 1 -- 172.21.15.47:0/1495584547 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fef00022070 con 0x7fef1005fa30 2022-06-29T15:51:11.004 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.986+0000 7fef0effd700 1 --2- 172.21.15.47:0/1495584547 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7feef0053a70 0x7feef0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:11.004 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.986+0000 7fef0effd700 1 -- 172.21.15.47:0/1495584547 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fef0006c8d0 con 0x7fef1005fa30 2022-06-29T15:51:11.004 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.986+0000 7fef15646700 1 --2- 172.21.15.47:0/1495584547 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7feef0053a70 0x7feef0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:11.005 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.986+0000 7fef15646700 1 --2- 172.21.15.47:0/1495584547 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7feef0053a70 0x7feef0055f30 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7fef0400b000 tx=0x7fef04009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:11.005 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:10.994+0000 7fef0effd700 1 -- 172.21.15.47:0/1495584547 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fef000435a0 con 0x7fef1005fa30 2022-06-29T15:51:11.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.162+0000 7fef16e49700 1 -- 172.21.15.47:0/1495584547 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef100573d0 con 0x7fef1005fa30 2022-06-29T15:51:11.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.162+0000 7fef0effd700 1 -- 172.21.15.47:0/1495584547 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+1426 (secure 0 0 0) 0x7fef00016020 con 0x7fef1005fa30 2022-06-29T15:51:11.172 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:11.172 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":62,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:51:10.130331+0000","modified":"2022-06-29T15:51:10.130455+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[],"up":{},"failed":[],"damaged":[],"stopped":[],"info":{},"data_pools":[22],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2022-06-29T15:51:11.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.166+0000 7fef16e49700 1 -- 172.21.15.47:0/1495584547 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7feef0053a70 msgr2=0x7feef0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:11.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.166+0000 7fef16e49700 1 --2- 172.21.15.47:0/1495584547 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7feef0053a70 0x7feef0055f30 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7fef0400b000 tx=0x7fef04009040).stop 2022-06-29T15:51:11.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.166+0000 7fef16e49700 1 -- 172.21.15.47:0/1495584547 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef1005fa30 msgr2=0x7fef10141e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:11.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.166+0000 7fef16e49700 1 --2- 172.21.15.47:0/1495584547 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef1005fa30 0x7fef10141e00 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 rx=0x7fef00006e00 tx=0x7fef000091d0).stop 2022-06-29T15:51:11.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.166+0000 7fef16e49700 1 -- 172.21.15.47:0/1495584547 shutdown_connections 2022-06-29T15:51:11.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.166+0000 7fef16e49700 1 --2- 172.21.15.47:0/1495584547 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7feef0053a70 0x7feef0055f30 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:11.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.166+0000 7fef16e49700 1 --2- 172.21.15.47:0/1495584547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef100b2770 0x7fef10063ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:11.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.166+0000 7fef16e49700 1 --2- 172.21.15.47:0/1495584547 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fef10060b90 0x7fef10066520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:11.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.166+0000 7fef16e49700 1 --2- 172.21.15.47:0/1495584547 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef1005fa30 0x7fef10141e00 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:11.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.166+0000 7fef16e49700 1 -- 172.21.15.47:0/1495584547 >> 172.21.15.47:0/1495584547 conn(0x7fef10062df0 msgr2=0x7fef1009ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:11.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.166+0000 7fef16e49700 1 -- 172.21.15.47:0/1495584547 shutdown_connections 2022-06-29T15:51:11.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.166+0000 7fef16e49700 1 -- 172.21.15.47:0/1495584547 wait complete. 2022-06-29T15:51:11.177 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 62 2022-06-29T15:51:11.184 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-06-29T15:51:11.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.278+0000 7f7ce510c700 1 Processor -- start 2022-06-29T15:51:11.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.278+0000 7f7ce510c700 1 -- start start 2022-06-29T15:51:11.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.278+0000 7f7ce510c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7ce00b2120 0x7f7ce00b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:11.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.278+0000 7f7ce510c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7ce00b1260 0x7f7ce014b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:11.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.278+0000 7f7ce510c700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ce00b0740 con 0x7f7ce005f0b0 2022-06-29T15:51:11.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.278+0000 7f7ce510c700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ce0147460 con 0x7f7ce00b2120 2022-06-29T15:51:11.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.278+0000 7f7ce510c700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce00b0970 con 0x7f7ce00b1260 2022-06-29T15:51:11.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.278+0000 7f7cded9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7ce00b2120 0x7f7ce00b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:11.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.278+0000 7f7cde59c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7ce005f0b0 0x7f7ce0147af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38828/0 (socket says 172.21.15.47:38828) 2022-06-29T15:51:11.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.278+0000 7f7cde59c700 1 -- 172.21.15.47:0/2493931587 learned_addr learned my addr 172.21.15.47:0/2493931587 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:11.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.282+0000 7f7cdf59e700 1 --2- 172.21.15.47:0/2493931587 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7ce00b1260 0x7f7ce014b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:11.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.282+0000 7f7cded9d700 1 -- 172.21.15.47:0/2493931587 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7ce00b1260 msgr2=0x7f7ce014b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:11.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.282+0000 7f7cded9d700 1 --2- 172.21.15.47:0/2493931587 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7ce00b1260 0x7f7ce014b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:11.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.282+0000 7f7cded9d700 1 -- 172.21.15.47:0/2493931587 >> v1:172.21.15.47:6789/0 conn(0x7f7ce005f0b0 legacy=0x7f7ce0147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:11.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.282+0000 7f7cded9d700 1 -- 172.21.15.47:0/2493931587 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce00b0c10 con 0x7f7ce00b2120 2022-06-29T15:51:11.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.282+0000 7f7cddd9b700 1 -- 172.21.15.47:0/2493931587 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1653227374 0 0) 0x7f7ce00b0740 con 0x7f7ce005f0b0 2022-06-29T15:51:11.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.282+0000 7f7cded9d700 1 --2- 172.21.15.47:0/2493931587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ce00b2120 0x7f7ce00b2500 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 rx=0x7f7ce0141a70 tx=0x7f7cd4006df0).ready entity=mon.0 client_cookie=a8300d9bb4dd3b28 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:11.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.282+0000 7f7cddd9b700 1 -- 172.21.15.47:0/2493931587 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7cd400d570 con 0x7f7ce00b2120 2022-06-29T15:51:11.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.282+0000 7f7cddd9b700 1 -- 172.21.15.47:0/2493931587 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cd4000e90 con 0x7f7ce00b2120 2022-06-29T15:51:11.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.282+0000 7f7cddd9b700 1 -- 172.21.15.47:0/2493931587 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7cd400d8c0 con 0x7f7ce00b2120 2022-06-29T15:51:11.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.282+0000 7f7ce510c700 1 -- 172.21.15.47:0/2493931587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ce00b2120 msgr2=0x7f7ce00b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:11.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.282+0000 7f7ce510c700 1 --2- 172.21.15.47:0/2493931587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ce00b2120 0x7f7ce00b2500 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 rx=0x7f7ce0141a70 tx=0x7f7cd4006df0).stop 2022-06-29T15:51:11.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.282+0000 7f7ce510c700 1 -- 172.21.15.47:0/2493931587 shutdown_connections 2022-06-29T15:51:11.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.282+0000 7f7ce510c700 1 --2- 172.21.15.47:0/2493931587 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7ce00b1260 0x7f7ce014b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:11.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.282+0000 7f7ce510c700 1 --2- 172.21.15.47:0/2493931587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ce00b2120 0x7f7ce00b2500 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:11.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.282+0000 7f7ce510c700 1 -- 172.21.15.47:0/2493931587 >> 172.21.15.47:0/2493931587 conn(0x7f7ce00626f0 msgr2=0x7f7ce0062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:11.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.282+0000 7f7ce510c700 1 -- 172.21.15.47:0/2493931587 shutdown_connections 2022-06-29T15:51:11.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.282+0000 7f7ce510c700 1 -- 172.21.15.47:0/2493931587 wait complete. 2022-06-29T15:51:11.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.286+0000 7f7ce510c700 1 Processor -- start 2022-06-29T15:51:11.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.286+0000 7f7ce510c700 1 -- start start 2022-06-29T15:51:11.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.286+0000 7f7ce510c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ce00b1260 0x7f7ce00ae480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:11.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.286+0000 7f7ce510c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7ce00b2120 0x7f7ce014d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:11.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.286+0000 7f7ce510c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ce0156710 0x7f7ce014dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:11.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.286+0000 7f7ce510c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce00ace00 con 0x7f7ce0156710 2022-06-29T15:51:11.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.286+0000 7f7ce510c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce00af880 con 0x7f7ce00b1260 2022-06-29T15:51:11.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.286+0000 7f7ce510c700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce00aedf0 con 0x7f7ce00b2120 2022-06-29T15:51:11.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.286+0000 7f7cdf59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ce0156710 0x7f7ce014dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:11.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.286+0000 7f7cdf59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ce0156710 0x7f7ce014dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53284/0 (socket says 172.21.15.47:53284) 2022-06-29T15:51:11.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.286+0000 7f7cdf59e700 1 -- 172.21.15.47:0/3804164895 learned_addr learned my addr 172.21.15.47:0/3804164895 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:11.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.286+0000 7f7cde59c700 1 --2- 172.21.15.47:0/3804164895 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7ce00b2120 0x7f7ce014d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:11.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.286+0000 7f7cded9d700 1 --2- 172.21.15.47:0/3804164895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ce00b1260 0x7f7ce00ae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:11.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.286+0000 7f7cdf59e700 1 -- 172.21.15.47:0/3804164895 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7ce00b2120 msgr2=0x7f7ce014d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:11.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.286+0000 7f7cdf59e700 1 --2- 172.21.15.47:0/3804164895 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7ce00b2120 0x7f7ce014d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:11.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.286+0000 7f7cdf59e700 1 -- 172.21.15.47:0/3804164895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ce00b1260 msgr2=0x7f7ce00ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:11.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.286+0000 7f7cdf59e700 1 --2- 172.21.15.47:0/3804164895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ce00b1260 0x7f7ce00ae480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:11.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.286+0000 7f7cdf59e700 1 -- 172.21.15.47:0/3804164895 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cd400b180 con 0x7f7ce0156710 2022-06-29T15:51:11.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.286+0000 7f7cde59c700 1 --2- 172.21.15.47:0/3804164895 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7ce00b2120 0x7f7ce014d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:51:11.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.290+0000 7f7cdf59e700 1 --2- 172.21.15.47:0/3804164895 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ce0156710 0x7f7ce014dd70 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 rx=0x7f7cd800c000 tx=0x7f7cd8006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:11.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.290+0000 7f7ccffff700 1 -- 172.21.15.47:0/3804164895 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7cd8011750 con 0x7f7ce0156710 2022-06-29T15:51:11.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.290+0000 7f7ccffff700 1 -- 172.21.15.47:0/3804164895 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cd80118b0 con 0x7f7ce0156710 2022-06-29T15:51:11.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.290+0000 7f7ccffff700 1 -- 172.21.15.47:0/3804164895 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7cd8011c10 con 0x7f7ce0156710 2022-06-29T15:51:11.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.290+0000 7f7ce510c700 1 -- 172.21.15.47:0/3804164895 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ce00af5e0 con 0x7f7ce0156710 2022-06-29T15:51:11.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.290+0000 7f7ce510c700 1 -- 172.21.15.47:0/3804164895 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ce014e950 con 0x7f7ce0156710 2022-06-29T15:51:11.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.290+0000 7f7ce510c700 1 -- 172.21.15.47:0/3804164895 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ce00af0c0 con 0x7f7ce0156710 2022-06-29T15:51:11.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.290+0000 7f7ccffff700 1 -- 172.21.15.47:0/3804164895 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7cd8011d70 con 0x7f7ce0156710 2022-06-29T15:51:11.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.290+0000 7f7ccffff700 1 --2- 172.21.15.47:0/3804164895 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7cbc0539e0 0x7f7cbc055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:11.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.290+0000 7f7ccffff700 1 -- 172.21.15.47:0/3804164895 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f7cd806c510 con 0x7f7ce0156710 2022-06-29T15:51:11.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.290+0000 7f7cded9d700 1 --2- 172.21.15.47:0/3804164895 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7cbc0539e0 0x7f7cbc055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:11.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.290+0000 7f7cded9d700 1 --2- 172.21.15.47:0/3804164895 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7cbc0539e0 0x7f7cbc055ea0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7f7cd401a000 tx=0x7f7cd4014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:11.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.298+0000 7f7ccffff700 1 -- 172.21.15.47:0/3804164895 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7cd8043170 con 0x7f7ce0156710 2022-06-29T15:51:11.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:11.458+0000 7f7ce510c700 1 -- 172.21.15.47:0/3804164895 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7ce00573d0 con 0x7f7ce0156710 2022-06-29T15:51:12.153 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.146+0000 7f7ccffff700 1 -- 172.21.15.47:0/3804164895 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v63) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7cd8046790 con 0x7f7ce0156710 2022-06-29T15:51:12.156 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.150+0000 7f7ce510c700 1 -- 172.21.15.47:0/3804164895 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7cbc0539e0 msgr2=0x7f7cbc055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:12.157 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.150+0000 7f7ce510c700 1 --2- 172.21.15.47:0/3804164895 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7cbc0539e0 0x7f7cbc055ea0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7f7cd401a000 tx=0x7f7cd4014000).stop 2022-06-29T15:51:12.157 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.150+0000 7f7ce510c700 1 -- 172.21.15.47:0/3804164895 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ce0156710 msgr2=0x7f7ce014dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:12.157 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.150+0000 7f7ce510c700 1 --2- 172.21.15.47:0/3804164895 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ce0156710 0x7f7ce014dd70 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 rx=0x7f7cd800c000 tx=0x7f7cd8006df0).stop 2022-06-29T15:51:12.158 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.150+0000 7f7ce510c700 1 -- 172.21.15.47:0/3804164895 shutdown_connections 2022-06-29T15:51:12.158 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.150+0000 7f7ce510c700 1 --2- 172.21.15.47:0/3804164895 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ce0156710 0x7f7ce014dd70 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:12.158 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.150+0000 7f7ce510c700 1 --2- 172.21.15.47:0/3804164895 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7ce00b2120 0x7f7ce014d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:12.158 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.150+0000 7f7ce510c700 1 --2- 172.21.15.47:0/3804164895 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7cbc0539e0 0x7f7cbc055ea0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:12.159 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.150+0000 7f7ce510c700 1 --2- 172.21.15.47:0/3804164895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ce00b1260 0x7f7ce00ae480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:12.159 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.150+0000 7f7ce510c700 1 -- 172.21.15.47:0/3804164895 >> 172.21.15.47:0/3804164895 conn(0x7f7ce00626f0 msgr2=0x7f7ce014a830 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:12.159 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.150+0000 7f7ce510c700 1 -- 172.21.15.47:0/3804164895 shutdown_connections 2022-06-29T15:51:12.159 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.150+0000 7f7ce510c700 1 -- 172.21.15.47:0/3804164895 wait complete. 2022-06-29T15:51:12.160 INFO:teuthology.orchestra.run.smithi047.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-06-29T15:51:12.173 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:51:12.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.278+0000 7fa6315d6700 1 Processor -- start 2022-06-29T15:51:12.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.278+0000 7fa6315d6700 1 -- start start 2022-06-29T15:51:12.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.282+0000 7fa6315d6700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa62c0b5c60 0x7fa62c144980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:12.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.282+0000 7fa6315d6700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa62c0a9120 con 0x7fa62c0a72f0 2022-06-29T15:51:12.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.282+0000 7fa6315d6700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa62c0b0b80 con 0x7fa62c060eb0 2022-06-29T15:51:12.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.282+0000 7fa6315d6700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa62c0a89e0 con 0x7fa62c0b5c60 2022-06-29T15:51:12.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.282+0000 7fa630dd5700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa62c0a72f0 0x7fa62c151eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38846/0 (socket says 172.21.15.47:38846) 2022-06-29T15:51:12.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.282+0000 7fa630dd5700 1 -- 172.21.15.47:0/1508741198 learned_addr learned my addr 172.21.15.47:0/1508741198 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:12.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.282+0000 7fa62bfff700 1 --2- 172.21.15.47:0/1508741198 >> v2:172.21.15.64:3300/0 conn(0x7fa62c0b5c60 0x7fa62c144980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:12.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.282+0000 7fa62b7fe700 1 -- 172.21.15.47:0/1508741198 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1151062664 0 0) 0x7fa62c0a9120 con 0x7fa62c0a72f0 2022-06-29T15:51:12.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.282+0000 7fa62b7fe700 1 -- 172.21.15.47:0/1508741198 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa60c003260 con 0x7fa62c0a72f0 2022-06-29T15:51:12.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.282+0000 7fa62b7fe700 1 -- 172.21.15.47:0/1508741198 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3094249448 0 0) 0x7fa62c0b0b80 con 0x7fa62c060eb0 2022-06-29T15:51:12.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.282+0000 7fa62b7fe700 1 -- 172.21.15.47:0/1508741198 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa62c0a9120 con 0x7fa62c060eb0 2022-06-29T15:51:12.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.282+0000 7fa62bfff700 1 -- 172.21.15.47:0/1508741198 >> v1:172.21.15.64:6789/0 conn(0x7fa62c060eb0 legacy=0x7fa62c0b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:12.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.282+0000 7fa62bfff700 1 -- 172.21.15.47:0/1508741198 >> v1:172.21.15.47:6789/0 conn(0x7fa62c0a72f0 legacy=0x7fa62c151eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:12.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.282+0000 7fa62bfff700 1 -- 172.21.15.47:0/1508741198 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa62c0a8c80 con 0x7fa62c0b5c60 2022-06-29T15:51:12.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.282+0000 7fa62bfff700 1 --2- 172.21.15.47:0/1508741198 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa62c0b5c60 0x7fa62c144980 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7fa61c002f70 tx=0x7fa61c006d90).ready entity=mon.1 client_cookie=6fedef4f1c41aa6 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:12.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.282+0000 7fa62b7fe700 1 -- 172.21.15.47:0/1508741198 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa61c00e630 con 0x7fa62c0b5c60 2022-06-29T15:51:12.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.282+0000 7fa62b7fe700 1 -- 172.21.15.47:0/1508741198 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa61c000e90 con 0x7fa62c0b5c60 2022-06-29T15:51:12.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.282+0000 7fa62b7fe700 1 -- 172.21.15.47:0/1508741198 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa61c00e980 con 0x7fa62c0b5c60 2022-06-29T15:51:12.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.282+0000 7fa6315d6700 1 -- 172.21.15.47:0/1508741198 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa62c0b5c60 msgr2=0x7fa62c144980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:12.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.282+0000 7fa6315d6700 1 --2- 172.21.15.47:0/1508741198 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa62c0b5c60 0x7fa62c144980 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7fa61c002f70 tx=0x7fa61c006d90).stop 2022-06-29T15:51:12.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.282+0000 7fa6315d6700 1 -- 172.21.15.47:0/1508741198 shutdown_connections 2022-06-29T15:51:12.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.282+0000 7fa6315d6700 1 --2- 172.21.15.47:0/1508741198 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa62c0b5c60 0x7fa62c144980 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:12.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.282+0000 7fa6315d6700 1 -- 172.21.15.47:0/1508741198 >> 172.21.15.47:0/1508741198 conn(0x7fa62c062560 msgr2=0x7fa62c062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:12.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.282+0000 7fa6315d6700 1 -- 172.21.15.47:0/1508741198 shutdown_connections 2022-06-29T15:51:12.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.282+0000 7fa6315d6700 1 -- 172.21.15.47:0/1508741198 wait complete. 2022-06-29T15:51:12.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.286+0000 7fa6315d6700 1 Processor -- start 2022-06-29T15:51:12.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.286+0000 7fa6315d6700 1 -- start start 2022-06-29T15:51:12.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.286+0000 7fa6315d6700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa62c060eb0 0x7fa62c14a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:12.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.286+0000 7fa6315d6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa62c0a72f0 0x7fa62c159d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:12.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.286+0000 7fa6315d6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa62c0b5c60 0x7fa62c15a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:12.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.286+0000 7fa6315d6700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa62c0ad7d0 con 0x7fa62c0b5c60 2022-06-29T15:51:12.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.286+0000 7fa6315d6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa62c13f8b0 con 0x7fa62c0a72f0 2022-06-29T15:51:12.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.286+0000 7fa6315d6700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa62c0ad040 con 0x7fa62c060eb0 2022-06-29T15:51:12.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.286+0000 7fa630dd5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa62c0b5c60 0x7fa62c15a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:12.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.286+0000 7fa630dd5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa62c0b5c60 0x7fa62c15a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53298/0 (socket says 172.21.15.47:53298) 2022-06-29T15:51:12.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.286+0000 7fa630dd5700 1 -- 172.21.15.47:0/1052435545 learned_addr learned my addr 172.21.15.47:0/1052435545 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:12.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.286+0000 7fa623fff700 1 --2- 172.21.15.47:0/1052435545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa62c0a72f0 0x7fa62c159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:12.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.286+0000 7fa62bfff700 1 --2- 172.21.15.47:0/1052435545 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa62c060eb0 0x7fa62c14a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:12.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.286+0000 7fa630dd5700 1 -- 172.21.15.47:0/1052435545 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa62c060eb0 msgr2=0x7fa62c14a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:12.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.286+0000 7fa630dd5700 1 --2- 172.21.15.47:0/1052435545 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa62c060eb0 0x7fa62c14a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:12.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.286+0000 7fa630dd5700 1 -- 172.21.15.47:0/1052435545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa62c0a72f0 msgr2=0x7fa62c159d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:12.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.286+0000 7fa630dd5700 1 --2- 172.21.15.47:0/1052435545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa62c0a72f0 0x7fa62c159d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:12.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.286+0000 7fa630dd5700 1 -- 172.21.15.47:0/1052435545 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa61c00b120 con 0x7fa62c0b5c60 2022-06-29T15:51:12.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.286+0000 7fa630dd5700 1 --2- 172.21.15.47:0/1052435545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa62c0b5c60 0x7fa62c15a230 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 rx=0x7fa624001490 tx=0x7fa624007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:12.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.286+0000 7fa629ffb700 1 -- 172.21.15.47:0/1052435545 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa624005ba0 con 0x7fa62c0b5c60 2022-06-29T15:51:12.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.290+0000 7fa629ffb700 1 -- 172.21.15.47:0/1052435545 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa624005d00 con 0x7fa62c0b5c60 2022-06-29T15:51:12.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.290+0000 7fa6315d6700 1 -- 172.21.15.47:0/1052435545 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa62c14a6d0 con 0x7fa62c0b5c60 2022-06-29T15:51:12.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.290+0000 7fa629ffb700 1 -- 172.21.15.47:0/1052435545 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa6240072c0 con 0x7fa62c0b5c60 2022-06-29T15:51:12.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.290+0000 7fa6315d6700 1 -- 172.21.15.47:0/1052435545 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa62c0ac8c0 con 0x7fa62c0b5c60 2022-06-29T15:51:12.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.290+0000 7fa6315d6700 1 -- 172.21.15.47:0/1052435545 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa62c0573d0 con 0x7fa62c0b5c60 2022-06-29T15:51:12.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.290+0000 7fa629ffb700 1 -- 172.21.15.47:0/1052435545 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa624010070 con 0x7fa62c0b5c60 2022-06-29T15:51:12.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.290+0000 7fa629ffb700 1 --2- 172.21.15.47:0/1052435545 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa6100539e0 0x7fa610055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:12.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.290+0000 7fa629ffb700 1 -- 172.21.15.47:0/1052435545 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fa62406bd50 con 0x7fa62c0b5c60 2022-06-29T15:51:12.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.290+0000 7fa62bfff700 1 --2- 172.21.15.47:0/1052435545 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa6100539e0 0x7fa610055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:12.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.294+0000 7fa62bfff700 1 --2- 172.21.15.47:0/1052435545 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa6100539e0 0x7fa610055ea0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7fa61c018000 tx=0x7fa61c014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:12.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.298+0000 7fa629ffb700 1 -- 172.21.15.47:0/1052435545 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa624042930 con 0x7fa62c0b5c60 2022-06-29T15:51:12.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.462+0000 7fa6315d6700 1 -- 172.21.15.47:0/1052435545 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa62c069e20 con 0x7fa62c0b5c60 2022-06-29T15:51:12.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.462+0000 7fa629ffb700 1 -- 172.21.15.47:0/1052435545 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+1426 (secure 0 0 0) 0x7fa624045f50 con 0x7fa62c0b5c60 2022-06-29T15:51:12.470 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:12.471 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":63,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":63,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:51:10.130331+0000","modified":"2022-06-29T15:51:12.136625+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[],"up":{},"failed":[],"damaged":[],"stopped":[],"info":{},"data_pools":[22],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2022-06-29T15:51:12.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.466+0000 7fa6315d6700 1 -- 172.21.15.47:0/1052435545 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa6100539e0 msgr2=0x7fa610055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:12.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.466+0000 7fa6315d6700 1 --2- 172.21.15.47:0/1052435545 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa6100539e0 0x7fa610055ea0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7fa61c018000 tx=0x7fa61c014000).stop 2022-06-29T15:51:12.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.466+0000 7fa6315d6700 1 -- 172.21.15.47:0/1052435545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa62c0b5c60 msgr2=0x7fa62c15a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:12.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.466+0000 7fa6315d6700 1 --2- 172.21.15.47:0/1052435545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa62c0b5c60 0x7fa62c15a230 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 rx=0x7fa624001490 tx=0x7fa624007e00).stop 2022-06-29T15:51:12.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.466+0000 7fa6315d6700 1 -- 172.21.15.47:0/1052435545 shutdown_connections 2022-06-29T15:51:12.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.466+0000 7fa6315d6700 1 --2- 172.21.15.47:0/1052435545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa62c0b5c60 0x7fa62c15a230 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:12.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.466+0000 7fa6315d6700 1 --2- 172.21.15.47:0/1052435545 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa6100539e0 0x7fa610055ea0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:12.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.466+0000 7fa6315d6700 1 --2- 172.21.15.47:0/1052435545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa62c0a72f0 0x7fa62c159d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:12.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.466+0000 7fa6315d6700 1 --2- 172.21.15.47:0/1052435545 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa62c060eb0 0x7fa62c14a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:12.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.466+0000 7fa6315d6700 1 -- 172.21.15.47:0/1052435545 >> 172.21.15.47:0/1052435545 conn(0x7fa62c062560 msgr2=0x7fa62c141a00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:12.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.466+0000 7fa6315d6700 1 -- 172.21.15.47:0/1052435545 shutdown_connections 2022-06-29T15:51:12.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.466+0000 7fa6315d6700 1 -- 172.21.15.47:0/1052435545 wait complete. 2022-06-29T15:51:12.476 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 63 2022-06-29T15:51:12.484 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-06-29T15:51:12.494 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:51:12.497 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:51:12.500 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:51:12.503 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:51:12.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.582+0000 7fa9c1950700 1 Processor -- start 2022-06-29T15:51:12.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.582+0000 7fa9c1950700 1 -- start start 2022-06-29T15:51:12.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.582+0000 7fa9c1950700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa9bc0b5d90 0x7fa9bc143a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:12.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.582+0000 7fa9c1950700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa9bc0676a0 0x7fa9bc140130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:12.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.582+0000 7fa9c1950700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9bc05cc30 con 0x7fa9bc05c850 2022-06-29T15:51:12.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.582+0000 7fa9c1950700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9bc1430a0 con 0x7fa9bc0676a0 2022-06-29T15:51:12.594 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.582+0000 7fa9c1950700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9bc143300 con 0x7fa9bc0b5d90 2022-06-29T15:51:12.594 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.582+0000 7fa9c114f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa9bc0676a0 0x7fa9bc140130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:12.594 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.582+0000 7fa9c114f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa9bc0676a0 0x7fa9bc140130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53306/0 (socket says 172.21.15.47:53306) 2022-06-29T15:51:12.594 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.582+0000 7fa9c114f700 1 -- 172.21.15.47:0/3791626314 learned_addr learned my addr 172.21.15.47:0/3791626314 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:12.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.582+0000 7fa9bbfff700 1 --2- 172.21.15.47:0/3791626314 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa9bc0b5d90 0x7fa9bc143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:12.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.582+0000 7fa9c114f700 1 -- 172.21.15.47:0/3791626314 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa9bc0b5d90 msgr2=0x7fa9bc143a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:12.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.582+0000 7fa9c114f700 1 --2- 172.21.15.47:0/3791626314 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa9bc0b5d90 0x7fa9bc143a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:12.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.582+0000 7fa9c114f700 1 -- 172.21.15.47:0/3791626314 >> v1:172.21.15.47:6789/0 conn(0x7fa9bc05c850 legacy=0x7fa9bc0b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:12.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.582+0000 7fa9c114f700 1 -- 172.21.15.47:0/3791626314 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9bc1435a0 con 0x7fa9bc0676a0 2022-06-29T15:51:12.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.582+0000 7fa9bbfff700 1 --2- 172.21.15.47:0/3791626314 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa9bc0b5d90 0x7fa9bc143a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:51:12.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.582+0000 7fa9c114f700 1 --2- 172.21.15.47:0/3791626314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9bc0676a0 0x7fa9bc140130 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 rx=0x7fa9b40032f0 tx=0x7fa9b4009290).ready entity=mon.0 client_cookie=e3595e6ff32c57b7 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:12.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.586+0000 7fa9bb7fe700 1 -- 172.21.15.47:0/3791626314 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9b400b8a0 con 0x7fa9bc0676a0 2022-06-29T15:51:12.597 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.586+0000 7fa9bb7fe700 1 -- 172.21.15.47:0/3791626314 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9b400ba00 con 0x7fa9bc0676a0 2022-06-29T15:51:12.597 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.586+0000 7fa9bb7fe700 1 -- 172.21.15.47:0/3791626314 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9b400bb70 con 0x7fa9bc0676a0 2022-06-29T15:51:12.597 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.586+0000 7fa9c1950700 1 -- 172.21.15.47:0/3791626314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9bc0676a0 msgr2=0x7fa9bc140130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:12.597 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.586+0000 7fa9c1950700 1 --2- 172.21.15.47:0/3791626314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9bc0676a0 0x7fa9bc140130 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 rx=0x7fa9b40032f0 tx=0x7fa9b4009290).stop 2022-06-29T15:51:12.597 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.586+0000 7fa9c1950700 1 -- 172.21.15.47:0/3791626314 shutdown_connections 2022-06-29T15:51:12.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.586+0000 7fa9c1950700 1 --2- 172.21.15.47:0/3791626314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9bc0676a0 0x7fa9bc140130 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:12.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.586+0000 7fa9c1950700 1 --2- 172.21.15.47:0/3791626314 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa9bc0b5d90 0x7fa9bc143a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:12.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.586+0000 7fa9c1950700 1 -- 172.21.15.47:0/3791626314 >> 172.21.15.47:0/3791626314 conn(0x7fa9bc060e00 msgr2=0x7fa9bc0611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:12.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.586+0000 7fa9c1950700 1 -- 172.21.15.47:0/3791626314 shutdown_connections 2022-06-29T15:51:12.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.586+0000 7fa9c1950700 1 -- 172.21.15.47:0/3791626314 wait complete. 2022-06-29T15:51:12.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.586+0000 7fa9c1950700 1 Processor -- start 2022-06-29T15:51:12.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.586+0000 7fa9c1950700 1 -- start start 2022-06-29T15:51:12.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.586+0000 7fa9c1950700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa9bc05c850 0x7fa9bc1448a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:12.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.586+0000 7fa9c1950700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9bc0676a0 0x7fa9bc1527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:12.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.586+0000 7fa9c1950700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9bc0b5d90 0x7fa9bc152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:12.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.586+0000 7fa9c1950700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9bc0b31a0 con 0x7fa9bc0676a0 2022-06-29T15:51:12.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.586+0000 7fa9c1950700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9bc0ba640 con 0x7fa9bc0b5d90 2022-06-29T15:51:12.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.586+0000 7fa9c1950700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9bc0b33d0 con 0x7fa9bc05c850 2022-06-29T15:51:12.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.586+0000 7fa9bbfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9bc0676a0 0x7fa9bc1527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:12.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.586+0000 7fa9bbfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9bc0676a0 0x7fa9bc1527f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53310/0 (socket says 172.21.15.47:53310) 2022-06-29T15:51:12.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.586+0000 7fa9bbfff700 1 -- 172.21.15.47:0/3458983999 learned_addr learned my addr 172.21.15.47:0/3458983999 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:12.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.590+0000 7fa9c094e700 1 --2- 172.21.15.47:0/3458983999 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa9bc05c850 0x7fa9bc1448a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:12.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.590+0000 7fa9c114f700 1 --2- 172.21.15.47:0/3458983999 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9bc0b5d90 0x7fa9bc152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:12.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.590+0000 7fa9bbfff700 1 -- 172.21.15.47:0/3458983999 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa9bc05c850 msgr2=0x7fa9bc1448a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:12.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.590+0000 7fa9bbfff700 1 --2- 172.21.15.47:0/3458983999 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa9bc05c850 0x7fa9bc1448a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:12.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.590+0000 7fa9bbfff700 1 -- 172.21.15.47:0/3458983999 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9bc0b5d90 msgr2=0x7fa9bc152ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:12.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.590+0000 7fa9bbfff700 1 --2- 172.21.15.47:0/3458983999 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9bc0b5d90 0x7fa9bc152ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:12.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.590+0000 7fa9bbfff700 1 -- 172.21.15.47:0/3458983999 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9b4009530 con 0x7fa9bc0676a0 2022-06-29T15:51:12.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.590+0000 7fa9bbfff700 1 --2- 172.21.15.47:0/3458983999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9bc0676a0 0x7fa9bc1527f0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 rx=0x7fa9a4006940 tx=0x7fa9a4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:12.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.590+0000 7fa9b9ffb700 1 -- 172.21.15.47:0/3458983999 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9a4011780 con 0x7fa9bc0676a0 2022-06-29T15:51:12.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.590+0000 7fa9b9ffb700 1 -- 172.21.15.47:0/3458983999 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9a40118e0 con 0x7fa9bc0676a0 2022-06-29T15:51:12.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.590+0000 7fa9c1950700 1 -- 172.21.15.47:0/3458983999 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9bc0b3670 con 0x7fa9bc0676a0 2022-06-29T15:51:12.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.590+0000 7fa9b9ffb700 1 -- 172.21.15.47:0/3458983999 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9a4011c40 con 0x7fa9bc0676a0 2022-06-29T15:51:12.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.590+0000 7fa9c1950700 1 -- 172.21.15.47:0/3458983999 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9bc13fb90 con 0x7fa9bc0676a0 2022-06-29T15:51:12.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.590+0000 7fa9c1950700 1 -- 172.21.15.47:0/3458983999 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9bc152530 con 0x7fa9bc0676a0 2022-06-29T15:51:12.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.590+0000 7fa9b9ffb700 1 -- 172.21.15.47:0/3458983999 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa9a4012420 con 0x7fa9bc0676a0 2022-06-29T15:51:12.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.594+0000 7fa9b9ffb700 1 --2- 172.21.15.47:0/3458983999 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa9a0053b00 0x7fa9a0055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:12.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.594+0000 7fa9c094e700 1 --2- 172.21.15.47:0/3458983999 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa9a0053b00 0x7fa9a0055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:12.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.594+0000 7fa9b9ffb700 1 -- 172.21.15.47:0/3458983999 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fa9a406c000 con 0x7fa9bc0676a0 2022-06-29T15:51:12.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.594+0000 7fa9c094e700 1 --2- 172.21.15.47:0/3458983999 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa9a0053b00 0x7fa9a0055fc0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7fa9ac00b000 tx=0x7fa9ac009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:12.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.598+0000 7fa9b9ffb700 1 -- 172.21.15.47:0/3458983999 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa9bc152530 con 0x7fa9bc0676a0 2022-06-29T15:51:12.769 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:12.762+0000 7fa9c1950700 1 -- 172.21.15.47:0/3458983999 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa9bc152530 con 0x7fa9bc0676a0 2022-06-29T15:51:13.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.154+0000 7fa9b9ffb700 1 -- 172.21.15.47:0/3458983999 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v64) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa9bc152530 con 0x7fa9bc0676a0 2022-06-29T15:51:13.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.158+0000 7fa9c1950700 1 -- 172.21.15.47:0/3458983999 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa9a0053b00 msgr2=0x7fa9a0055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:13.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.158+0000 7fa9c1950700 1 --2- 172.21.15.47:0/3458983999 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa9a0053b00 0x7fa9a0055fc0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7fa9ac00b000 tx=0x7fa9ac009040).stop 2022-06-29T15:51:13.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.158+0000 7fa9c1950700 1 -- 172.21.15.47:0/3458983999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9bc0676a0 msgr2=0x7fa9bc1527f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:13.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.158+0000 7fa9c1950700 1 --2- 172.21.15.47:0/3458983999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9bc0676a0 0x7fa9bc1527f0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 rx=0x7fa9a4006940 tx=0x7fa9a4008040).stop 2022-06-29T15:51:13.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.158+0000 7fa9c1950700 1 -- 172.21.15.47:0/3458983999 shutdown_connections 2022-06-29T15:51:13.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.158+0000 7fa9c1950700 1 --2- 172.21.15.47:0/3458983999 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa9a0053b00 0x7fa9a0055fc0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:13.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.158+0000 7fa9c1950700 1 --2- 172.21.15.47:0/3458983999 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9bc0b5d90 0x7fa9bc152ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:13.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.158+0000 7fa9c1950700 1 --2- 172.21.15.47:0/3458983999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9bc0676a0 0x7fa9bc1527f0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:13.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.158+0000 7fa9c1950700 1 --2- 172.21.15.47:0/3458983999 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa9bc05c850 0x7fa9bc1448a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:13.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.158+0000 7fa9c1950700 1 -- 172.21.15.47:0/3458983999 >> 172.21.15.47:0/3458983999 conn(0x7fa9bc060e00 msgr2=0x7fa9bc1455a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:13.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.158+0000 7fa9c1950700 1 -- 172.21.15.47:0/3458983999 shutdown_connections 2022-06-29T15:51:13.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.158+0000 7fa9c1950700 1 -- 172.21.15.47:0/3458983999 wait complete. 2022-06-29T15:51:13.183 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-06-29T15:51:13.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.298+0000 7f3d4759e700 1 Processor -- start 2022-06-29T15:51:13.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.298+0000 7f3d4759e700 1 -- start start 2022-06-29T15:51:13.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.302+0000 7f3d4759e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3d4805c850 0x7f3d48143a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:13.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.302+0000 7f3d4759e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3d480b5d90 0x7f3d480b4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:13.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.302+0000 7f3d4759e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d4805cc30 con 0x7f3d480676a0 2022-06-29T15:51:13.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.302+0000 7f3d4759e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d481430a0 con 0x7f3d480b5d90 2022-06-29T15:51:13.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.302+0000 7f3d4759e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d48143300 con 0x7f3d4805c850 2022-06-29T15:51:13.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.302+0000 7f3d4659c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3d4805c850 0x7f3d48143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:13.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.302+0000 7f3d45d9b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3d480b5d90 0x7f3d480b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:13.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.302+0000 7f3d46d9d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f3d480676a0 0x7f3d48140130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:46434/0 (socket says 172.21.15.47:46434) 2022-06-29T15:51:13.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.302+0000 7f3d46d9d700 1 -- 172.21.15.47:0/1628074852 learned_addr learned my addr 172.21.15.47:0/1628074852 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:13.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.302+0000 7f3d45d9b700 1 -- 172.21.15.47:0/1628074852 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3d4805c850 msgr2=0x7f3d48143a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:13.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.302+0000 7f3d45d9b700 1 --2- 172.21.15.47:0/1628074852 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3d4805c850 0x7f3d48143a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:13.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.302+0000 7f3d45d9b700 1 -- 172.21.15.47:0/1628074852 >> v1:172.21.15.64:6789/0 conn(0x7f3d480676a0 legacy=0x7f3d48140130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:13.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.302+0000 7f3d4559a700 1 -- 172.21.15.47:0/1628074852 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3204791090 0 0) 0x7f3d4805cc30 con 0x7f3d480676a0 2022-06-29T15:51:13.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.302+0000 7f3d45d9b700 1 -- 172.21.15.47:0/1628074852 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d481435a0 con 0x7f3d480b5d90 2022-06-29T15:51:13.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.302+0000 7f3d4659c700 1 --2- 172.21.15.47:0/1628074852 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3d4805c850 0x7f3d48143a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:51:13.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.302+0000 7f3d45d9b700 1 --2- 172.21.15.47:0/1628074852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d480b5d90 0x7f3d480b4860 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7f3d3c002f70 tx=0x7f3d3c006df0).ready entity=mon.1 client_cookie=1f9d0c4380ac7817 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:13.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.302+0000 7f3d4559a700 1 -- 172.21.15.47:0/1628074852 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d3c00d570 con 0x7f3d480b5d90 2022-06-29T15:51:13.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.302+0000 7f3d4559a700 1 -- 172.21.15.47:0/1628074852 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d3c000e90 con 0x7f3d480b5d90 2022-06-29T15:51:13.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.302+0000 7f3d4559a700 1 -- 172.21.15.47:0/1628074852 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d3c00d8c0 con 0x7f3d480b5d90 2022-06-29T15:51:13.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.302+0000 7f3d4759e700 1 -- 172.21.15.47:0/1628074852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d480b5d90 msgr2=0x7f3d480b4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:13.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.302+0000 7f3d4759e700 1 --2- 172.21.15.47:0/1628074852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d480b5d90 0x7f3d480b4860 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7f3d3c002f70 tx=0x7f3d3c006df0).stop 2022-06-29T15:51:13.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.302+0000 7f3d4759e700 1 -- 172.21.15.47:0/1628074852 shutdown_connections 2022-06-29T15:51:13.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.302+0000 7f3d4759e700 1 --2- 172.21.15.47:0/1628074852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d480b5d90 0x7f3d480b4860 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:13.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.302+0000 7f3d4759e700 1 --2- 172.21.15.47:0/1628074852 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3d4805c850 0x7f3d48143a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:13.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.302+0000 7f3d4759e700 1 -- 172.21.15.47:0/1628074852 >> 172.21.15.47:0/1628074852 conn(0x7f3d48060e00 msgr2=0x7f3d480611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:13.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.302+0000 7f3d4759e700 1 -- 172.21.15.47:0/1628074852 shutdown_connections 2022-06-29T15:51:13.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.302+0000 7f3d4759e700 1 -- 172.21.15.47:0/1628074852 wait complete. 2022-06-29T15:51:13.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.306+0000 7f3d4759e700 1 Processor -- start 2022-06-29T15:51:13.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.306+0000 7f3d4759e700 1 -- start start 2022-06-29T15:51:13.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.306+0000 7f3d4759e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3d4805c850 0x7f3d480b7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:13.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.306+0000 7f3d4759e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d480676a0 0x7f3d480b3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:13.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.306+0000 7f3d4759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d480b5d90 0x7f3d481521f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:13.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.306+0000 7f3d4759e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4812c300 con 0x7f3d480676a0 2022-06-29T15:51:13.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.306+0000 7f3d4759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d48156350 con 0x7f3d480b5d90 2022-06-29T15:51:13.335 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.306+0000 7f3d4759e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4812be60 con 0x7f3d4805c850 2022-06-29T15:51:13.335 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.306+0000 7f3d45d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d480676a0 0x7f3d480b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:13.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.306+0000 7f3d45d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d480676a0 0x7f3d480b3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53324/0 (socket says 172.21.15.47:53324) 2022-06-29T15:51:13.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.306+0000 7f3d45d9b700 1 -- 172.21.15.47:0/1795887040 learned_addr learned my addr 172.21.15.47:0/1795887040 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:13.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.306+0000 7f3d4659c700 1 --2- 172.21.15.47:0/1795887040 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3d4805c850 0x7f3d480b7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:13.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.306+0000 7f3d46d9d700 1 --2- 172.21.15.47:0/1795887040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d480b5d90 0x7f3d481521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:13.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.306+0000 7f3d45d9b700 1 -- 172.21.15.47:0/1795887040 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3d4805c850 msgr2=0x7f3d480b7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:13.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.306+0000 7f3d45d9b700 1 --2- 172.21.15.47:0/1795887040 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3d4805c850 0x7f3d480b7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:13.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.306+0000 7f3d45d9b700 1 -- 172.21.15.47:0/1795887040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d480b5d90 msgr2=0x7f3d481521f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:13.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.310+0000 7f3d45d9b700 1 --2- 172.21.15.47:0/1795887040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d480b5d90 0x7f3d481521f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:13.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.310+0000 7f3d45d9b700 1 -- 172.21.15.47:0/1795887040 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d3800b120 con 0x7f3d480676a0 2022-06-29T15:51:13.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.310+0000 7f3d45d9b700 1 --2- 172.21.15.47:0/1795887040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d480676a0 0x7f3d480b3490 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 rx=0x7f3d3c002f70 tx=0x7f3d3c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:13.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.310+0000 7f3d377fe700 1 -- 172.21.15.47:0/1795887040 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d3c013de0 con 0x7f3d480676a0 2022-06-29T15:51:13.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.310+0000 7f3d377fe700 1 -- 172.21.15.47:0/1795887040 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d3c00de40 con 0x7f3d480676a0 2022-06-29T15:51:13.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.310+0000 7f3d4759e700 1 -- 172.21.15.47:0/1795887040 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d3c00b180 con 0x7f3d480676a0 2022-06-29T15:51:13.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.310+0000 7f3d377fe700 1 -- 172.21.15.47:0/1795887040 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d3c01e5f0 con 0x7f3d480676a0 2022-06-29T15:51:13.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.310+0000 7f3d4759e700 1 -- 172.21.15.47:0/1795887040 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d480b7380 con 0x7f3d480676a0 2022-06-29T15:51:13.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.310+0000 7f3d4759e700 1 -- 172.21.15.47:0/1795887040 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d480573d0 con 0x7f3d480676a0 2022-06-29T15:51:13.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.310+0000 7f3d377fe700 1 -- 172.21.15.47:0/1795887040 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3d3c025030 con 0x7f3d480676a0 2022-06-29T15:51:13.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.314+0000 7f3d377fe700 1 --2- 172.21.15.47:0/1795887040 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3d24053aa0 0x7f3d24055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:13.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.314+0000 7f3d377fe700 1 -- 172.21.15.47:0/1795887040 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f3d3c06cd40 con 0x7f3d480676a0 2022-06-29T15:51:13.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.314+0000 7f3d4659c700 1 --2- 172.21.15.47:0/1795887040 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3d24053aa0 0x7f3d24055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:13.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.314+0000 7f3d4659c700 1 --2- 172.21.15.47:0/1795887040 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3d24053aa0 0x7f3d24055f60 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7f3d38008000 tx=0x7f3d38008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:13.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.318+0000 7f3d377fe700 1 -- 172.21.15.47:0/1795887040 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3d3c043aa0 con 0x7f3d480676a0 2022-06-29T15:51:13.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:13.450+0000 7f3d4759e700 1 -- 172.21.15.47:0/1795887040 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3d48069a60 con 0x7f3d480676a0 2022-06-29T15:51:14.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.166+0000 7f3d377fe700 1 -- 172.21.15.47:0/1795887040 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v134) v1 ==== 185+0+0 (secure 0 0 0) 0x7f3d3c0230e0 con 0x7f3d480676a0 2022-06-29T15:51:14.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.166+0000 7f3d4759e700 1 -- 172.21.15.47:0/1795887040 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3d24053aa0 msgr2=0x7f3d24055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:14.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.166+0000 7f3d4759e700 1 --2- 172.21.15.47:0/1795887040 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3d24053aa0 0x7f3d24055f60 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7f3d38008000 tx=0x7f3d38008040).stop 2022-06-29T15:51:14.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.166+0000 7f3d4759e700 1 -- 172.21.15.47:0/1795887040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d480676a0 msgr2=0x7f3d480b3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:14.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.166+0000 7f3d4759e700 1 --2- 172.21.15.47:0/1795887040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d480676a0 0x7f3d480b3490 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 rx=0x7f3d3c002f70 tx=0x7f3d3c014000).stop 2022-06-29T15:51:14.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.166+0000 7f3d4759e700 1 -- 172.21.15.47:0/1795887040 shutdown_connections 2022-06-29T15:51:14.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.166+0000 7f3d4759e700 1 --2- 172.21.15.47:0/1795887040 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3d24053aa0 0x7f3d24055f60 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:14.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.166+0000 7f3d4759e700 1 --2- 172.21.15.47:0/1795887040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d480b5d90 0x7f3d481521f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:14.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.166+0000 7f3d4759e700 1 --2- 172.21.15.47:0/1795887040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d480676a0 0x7f3d480b3490 secure :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 rx=0x7f3d3c002f70 tx=0x7f3d3c014000).stop 2022-06-29T15:51:14.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.166+0000 7f3d4759e700 1 --2- 172.21.15.47:0/1795887040 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3d4805c850 0x7f3d480b7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:14.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.166+0000 7f3d4759e700 1 -- 172.21.15.47:0/1795887040 >> 172.21.15.47:0/1795887040 conn(0x7f3d48060e00 msgr2=0x7f3d48146300 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:14.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.166+0000 7f3d4759e700 1 -- 172.21.15.47:0/1795887040 shutdown_connections 2022-06-29T15:51:14.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.166+0000 7f3d4759e700 1 -- 172.21.15.47:0/1795887040 wait complete. 2022-06-29T15:51:14.178 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' removed 2022-06-29T15:51:14.190 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-06-29T15:51:14.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.294+0000 7efe6359e700 1 Processor -- start 2022-06-29T15:51:14.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.294+0000 7efe6359e700 1 -- start start 2022-06-29T15:51:14.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.294+0000 7efe6359e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7efe64062f30 0x7efe641400c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:14.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.294+0000 7efe6359e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7efe6413d9b0 0x7efe641405b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:14.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.294+0000 7efe6359e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe64143f70 con 0x7efe640b6d30 2022-06-29T15:51:14.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.294+0000 7efe6359e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe640b1380 con 0x7efe64062f30 2022-06-29T15:51:14.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.294+0000 7efe6359e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe640a9cf0 con 0x7efe6413d9b0 2022-06-29T15:51:14.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.294+0000 7efe61d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7efe64062f30 0x7efe641400c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:14.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.294+0000 7efe61d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7efe64062f30 0x7efe641400c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53332/0 (socket says 172.21.15.47:53332) 2022-06-29T15:51:14.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.294+0000 7efe61d9b700 1 -- 172.21.15.47:0/3135452324 learned_addr learned my addr 172.21.15.47:0/3135452324 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:14.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.294+0000 7efe62d9d700 1 --2- 172.21.15.47:0/3135452324 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7efe6413d9b0 0x7efe641405b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:14.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.294+0000 7efe61d9b700 1 -- 172.21.15.47:0/3135452324 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7efe6413d9b0 msgr2=0x7efe641405b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:14.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.294+0000 7efe61d9b700 1 --2- 172.21.15.47:0/3135452324 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7efe6413d9b0 0x7efe641405b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:14.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.294+0000 7efe61d9b700 1 -- 172.21.15.47:0/3135452324 >> v1:172.21.15.64:6789/0 conn(0x7efe640b6d30 legacy=0x7efe6413d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:14.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.294+0000 7efe61d9b700 1 -- 172.21.15.47:0/3135452324 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe641441a0 con 0x7efe64062f30 2022-06-29T15:51:14.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.294+0000 7efe61d9b700 1 --2- 172.21.15.47:0/3135452324 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe64062f30 0x7efe641400c0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 rx=0x7efe64062a70 tx=0x7efe58006eb0).ready entity=mon.0 client_cookie=e4cc4f5aafadfc7c server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:14.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.298+0000 7efe6159a700 1 -- 172.21.15.47:0/3135452324 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe5800d570 con 0x7efe64062f30 2022-06-29T15:51:14.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.298+0000 7efe6159a700 1 -- 172.21.15.47:0/3135452324 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe58000e90 con 0x7efe64062f30 2022-06-29T15:51:14.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.298+0000 7efe6159a700 1 -- 172.21.15.47:0/3135452324 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe5800d8c0 con 0x7efe64062f30 2022-06-29T15:51:14.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.298+0000 7efe6359e700 1 -- 172.21.15.47:0/3135452324 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe64062f30 msgr2=0x7efe641400c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:14.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.298+0000 7efe6359e700 1 --2- 172.21.15.47:0/3135452324 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe64062f30 0x7efe641400c0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 rx=0x7efe64062a70 tx=0x7efe58006eb0).stop 2022-06-29T15:51:14.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.298+0000 7efe6359e700 1 -- 172.21.15.47:0/3135452324 shutdown_connections 2022-06-29T15:51:14.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.298+0000 7efe6359e700 1 --2- 172.21.15.47:0/3135452324 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7efe6413d9b0 0x7efe641405b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:14.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.298+0000 7efe6359e700 1 --2- 172.21.15.47:0/3135452324 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe64062f30 0x7efe641400c0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:14.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.298+0000 7efe6359e700 1 -- 172.21.15.47:0/3135452324 >> 172.21.15.47:0/3135452324 conn(0x7efe64063670 msgr2=0x7efe64063a50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:14.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.298+0000 7efe6359e700 1 -- 172.21.15.47:0/3135452324 shutdown_connections 2022-06-29T15:51:14.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.298+0000 7efe6359e700 1 -- 172.21.15.47:0/3135452324 wait complete. 2022-06-29T15:51:14.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.298+0000 7efe6359e700 1 Processor -- start 2022-06-29T15:51:14.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.298+0000 7efe6359e700 1 -- start start 2022-06-29T15:51:14.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.302+0000 7efe6359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe64062f30 0x7efe64133840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:14.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.302+0000 7efe6359e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe640b6d30 0x7efe64150670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:14.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.302+0000 7efe6359e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7efe6413d9b0 0x7efe64150b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:14.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.302+0000 7efe6359e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6414ffe0 con 0x7efe640b6d30 2022-06-29T15:51:14.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.302+0000 7efe6359e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe64133e50 con 0x7efe64062f30 2022-06-29T15:51:14.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.302+0000 7efe6359e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe6413bfa0 con 0x7efe6413d9b0 2022-06-29T15:51:14.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.302+0000 7efe61d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe640b6d30 0x7efe64150670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:14.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.302+0000 7efe61d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe640b6d30 0x7efe64150670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53338/0 (socket says 172.21.15.47:53338) 2022-06-29T15:51:14.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.302+0000 7efe61d9b700 1 -- 172.21.15.47:0/2510635771 learned_addr learned my addr 172.21.15.47:0/2510635771 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:14.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.302+0000 7efe62d9d700 1 --2- 172.21.15.47:0/2510635771 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7efe6413d9b0 0x7efe64150b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:14.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.302+0000 7efe6259c700 1 --2- 172.21.15.47:0/2510635771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe64062f30 0x7efe64133840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:14.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.302+0000 7efe61d9b700 1 -- 172.21.15.47:0/2510635771 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7efe6413d9b0 msgr2=0x7efe64150b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:14.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.302+0000 7efe61d9b700 1 --2- 172.21.15.47:0/2510635771 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7efe6413d9b0 0x7efe64150b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:14.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.302+0000 7efe61d9b700 1 -- 172.21.15.47:0/2510635771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe64062f30 msgr2=0x7efe64133840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:14.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.302+0000 7efe61d9b700 1 --2- 172.21.15.47:0/2510635771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe64062f30 0x7efe64133840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:14.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.302+0000 7efe61d9b700 1 -- 172.21.15.47:0/2510635771 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe5800b1e0 con 0x7efe640b6d30 2022-06-29T15:51:14.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.302+0000 7efe6259c700 1 --2- 172.21.15.47:0/2510635771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe64062f30 0x7efe64133840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:51:14.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.302+0000 7efe61d9b700 1 --2- 172.21.15.47:0/2510635771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe640b6d30 0x7efe64150670 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 rx=0x7efe64062a70 tx=0x7efe58015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:14.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.302+0000 7efe537fe700 1 -- 172.21.15.47:0/2510635771 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe58011860 con 0x7efe640b6d30 2022-06-29T15:51:14.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.302+0000 7efe537fe700 1 -- 172.21.15.47:0/2510635771 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe580119c0 con 0x7efe640b6d30 2022-06-29T15:51:14.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.302+0000 7efe6359e700 1 -- 172.21.15.47:0/2510635771 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe641501d0 con 0x7efe640b6d30 2022-06-29T15:51:14.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.302+0000 7efe537fe700 1 -- 172.21.15.47:0/2510635771 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe58011d20 con 0x7efe640b6d30 2022-06-29T15:51:14.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.302+0000 7efe6359e700 1 -- 172.21.15.47:0/2510635771 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe64136180 con 0x7efe640b6d30 2022-06-29T15:51:14.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.302+0000 7efe6359e700 1 -- 172.21.15.47:0/2510635771 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe64150360 con 0x7efe640b6d30 2022-06-29T15:51:14.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.306+0000 7efe537fe700 1 -- 172.21.15.47:0/2510635771 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7efe58025030 con 0x7efe640b6d30 2022-06-29T15:51:14.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.306+0000 7efe537fe700 1 --2- 172.21.15.47:0/2510635771 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7efe40053980 0x7efe40055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:14.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.306+0000 7efe537fe700 1 -- 172.21.15.47:0/2510635771 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6068+0+0 (secure 0 0 0) 0x7efe5806d0f0 con 0x7efe640b6d30 2022-06-29T15:51:14.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.306+0000 7efe6259c700 1 --2- 172.21.15.47:0/2510635771 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7efe40053980 0x7efe40055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:14.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.306+0000 7efe6259c700 1 --2- 172.21.15.47:0/2510635771 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7efe40053980 0x7efe40055e40 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7efe5400b000 tx=0x7efe54009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:14.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.310+0000 7efe537fe700 1 -- 172.21.15.47:0/2510635771 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7efe58044050 con 0x7efe640b6d30 2022-06-29T15:51:14.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:14.442+0000 7efe6359e700 1 -- 172.21.15.47:0/2510635771 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7efe640573d0 con 0x7efe640b6d30 2022-06-29T15:51:15.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.306+0000 7efe537fe700 1 -- 172.21.15.47:0/2510635771 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v135) v1 ==== 173+0+0 (secure 0 0 0) 0x7efe58023020 con 0x7efe640b6d30 2022-06-29T15:51:15.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.310+0000 7efe6359e700 1 -- 172.21.15.47:0/2510635771 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7efe40053980 msgr2=0x7efe40055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:15.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.310+0000 7efe6359e700 1 --2- 172.21.15.47:0/2510635771 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7efe40053980 0x7efe40055e40 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7efe5400b000 tx=0x7efe54009040).stop 2022-06-29T15:51:15.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.310+0000 7efe6359e700 1 -- 172.21.15.47:0/2510635771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe640b6d30 msgr2=0x7efe64150670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:15.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.310+0000 7efe6359e700 1 --2- 172.21.15.47:0/2510635771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe640b6d30 0x7efe64150670 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 rx=0x7efe64062a70 tx=0x7efe58015000).stop 2022-06-29T15:51:15.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.310+0000 7efe6359e700 1 -- 172.21.15.47:0/2510635771 shutdown_connections 2022-06-29T15:51:15.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.310+0000 7efe6359e700 1 --2- 172.21.15.47:0/2510635771 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7efe6413d9b0 0x7efe64150b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:15.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.310+0000 7efe6359e700 1 --2- 172.21.15.47:0/2510635771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe640b6d30 0x7efe64150670 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:15.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.310+0000 7efe6359e700 1 --2- 172.21.15.47:0/2510635771 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7efe40053980 0x7efe40055e40 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:15.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.310+0000 7efe6359e700 1 --2- 172.21.15.47:0/2510635771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe64062f30 0x7efe64133840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:15.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.310+0000 7efe6359e700 1 -- 172.21.15.47:0/2510635771 >> 172.21.15.47:0/2510635771 conn(0x7efe64063670 msgr2=0x7efe6413c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:15.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.310+0000 7efe6359e700 1 -- 172.21.15.47:0/2510635771 shutdown_connections 2022-06-29T15:51:15.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.310+0000 7efe6359e700 1 -- 172.21.15.47:0/2510635771 wait complete. 2022-06-29T15:51:15.322 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' removed 2022-06-29T15:51:15.336 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-06-29T15:51:15.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.434+0000 7f368a1d7700 1 Processor -- start 2022-06-29T15:51:15.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.434+0000 7f368a1d7700 1 -- start start 2022-06-29T15:51:15.447 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.438+0000 7f368a1d7700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f36840b4300 0x7f36840612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:15.447 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.438+0000 7f368a1d7700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3684067610 con 0x7f36841471a0 2022-06-29T15:51:15.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.438+0000 7f368a1d7700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36841467c0 con 0x7f3684060ee0 2022-06-29T15:51:15.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.438+0000 7f368a1d7700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3684146a60 con 0x7f36840b4300 2022-06-29T15:51:15.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.438+0000 7f3683fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f36841471a0 0x7f368413cdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38902/0 (socket says 172.21.15.47:38902) 2022-06-29T15:51:15.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.438+0000 7f3683fff700 1 -- 172.21.15.47:0/2523020890 learned_addr learned my addr 172.21.15.47:0/2523020890 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:15.449 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.438+0000 7f3682ffd700 1 --2- 172.21.15.47:0/2523020890 >> v2:172.21.15.64:3300/0 conn(0x7f36840b4300 0x7f36840612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:15.449 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.438+0000 7f36827fc700 1 -- 172.21.15.47:0/2523020890 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 438939618 0 0) 0x7f3684067610 con 0x7f36841471a0 2022-06-29T15:51:15.449 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.438+0000 7f36827fc700 1 -- 172.21.15.47:0/2523020890 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3668003260 con 0x7f36841471a0 2022-06-29T15:51:15.449 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.438+0000 7f36827fc700 1 -- 172.21.15.47:0/2523020890 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 374976207 0 0) 0x7f36841467c0 con 0x7f3684060ee0 2022-06-29T15:51:15.450 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.438+0000 7f3682ffd700 1 -- 172.21.15.47:0/2523020890 >> v1:172.21.15.64:6789/0 conn(0x7f3684060ee0 legacy=0x7f36841430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:15.450 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.438+0000 7f3682ffd700 1 -- 172.21.15.47:0/2523020890 >> v1:172.21.15.47:6789/0 conn(0x7f36841471a0 legacy=0x7f368413cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:15.450 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.438+0000 7f3682ffd700 1 -- 172.21.15.47:0/2523020890 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3684146d00 con 0x7f36840b4300 2022-06-29T15:51:15.451 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.438+0000 7f3682ffd700 1 --2- 172.21.15.47:0/2523020890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36840b4300 0x7f36840612c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7f36840adb80 tx=0x7f36740091d0).ready entity=mon.1 client_cookie=f943d90c45b1f4ef server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:15.451 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.438+0000 7f36827fc700 1 -- 172.21.15.47:0/2523020890 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f367400b670 con 0x7f36840b4300 2022-06-29T15:51:15.451 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.438+0000 7f36827fc700 1 -- 172.21.15.47:0/2523020890 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f367400b7d0 con 0x7f36840b4300 2022-06-29T15:51:15.451 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.438+0000 7f36827fc700 1 -- 172.21.15.47:0/2523020890 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f367400b940 con 0x7f36840b4300 2022-06-29T15:51:15.452 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.438+0000 7f368a1d7700 1 -- 172.21.15.47:0/2523020890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36840b4300 msgr2=0x7f36840612c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:15.452 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.438+0000 7f368a1d7700 1 --2- 172.21.15.47:0/2523020890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36840b4300 0x7f36840612c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7f36840adb80 tx=0x7f36740091d0).stop 2022-06-29T15:51:15.452 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.438+0000 7f368a1d7700 1 -- 172.21.15.47:0/2523020890 shutdown_connections 2022-06-29T15:51:15.452 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.438+0000 7f368a1d7700 1 --2- 172.21.15.47:0/2523020890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36840b4300 0x7f36840612c0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:15.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.438+0000 7f368a1d7700 1 -- 172.21.15.47:0/2523020890 >> 172.21.15.47:0/2523020890 conn(0x7f368405c510 msgr2=0x7f368405f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:15.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.438+0000 7f368a1d7700 1 -- 172.21.15.47:0/2523020890 shutdown_connections 2022-06-29T15:51:15.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.438+0000 7f368a1d7700 1 -- 172.21.15.47:0/2523020890 wait complete. 2022-06-29T15:51:15.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.442+0000 7f368a1d7700 1 Processor -- start 2022-06-29T15:51:15.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.442+0000 7f368a1d7700 1 -- start start 2022-06-29T15:51:15.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.442+0000 7f368a1d7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3684060ee0 0x7f3684148cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:15.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.442+0000 7f368a1d7700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f36840b4300 0x7f3684155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:15.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.442+0000 7f368a1d7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36841471a0 0x7f36841428d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:15.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.442+0000 7f368a1d7700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36840b3c60 con 0x7f36841471a0 2022-06-29T15:51:15.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.442+0000 7f368a1d7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3684152be0 con 0x7f3684060ee0 2022-06-29T15:51:15.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.442+0000 7f368a1d7700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3684152e10 con 0x7f36840b4300 2022-06-29T15:51:15.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.442+0000 7f3683fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36841471a0 0x7f36841428d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:15.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.442+0000 7f36837fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3684060ee0 0x7f3684148cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:15.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.442+0000 7f3683fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36841471a0 0x7f36841428d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53352/0 (socket says 172.21.15.47:53352) 2022-06-29T15:51:15.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.442+0000 7f3683fff700 1 -- 172.21.15.47:0/1651676546 learned_addr learned my addr 172.21.15.47:0/1651676546 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:15.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.442+0000 7f3682ffd700 1 --2- 172.21.15.47:0/1651676546 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f36840b4300 0x7f3684155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:15.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.442+0000 7f3683fff700 1 -- 172.21.15.47:0/1651676546 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f36840b4300 msgr2=0x7f3684155710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:15.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.442+0000 7f3683fff700 1 --2- 172.21.15.47:0/1651676546 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f36840b4300 0x7f3684155710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:15.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.442+0000 7f3683fff700 1 -- 172.21.15.47:0/1651676546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3684060ee0 msgr2=0x7f3684148cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:15.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.442+0000 7f3683fff700 1 --2- 172.21.15.47:0/1651676546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3684060ee0 0x7f3684148cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:15.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.442+0000 7f3683fff700 1 -- 172.21.15.47:0/1651676546 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3674009470 con 0x7f36841471a0 2022-06-29T15:51:15.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.442+0000 7f36837fe700 1 --2- 172.21.15.47:0/1651676546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3684060ee0 0x7f3684148cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:51:15.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.442+0000 7f3683fff700 1 --2- 172.21.15.47:0/1651676546 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36841471a0 0x7f36841428d0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 rx=0x7f367c001110 tx=0x7f367c007a80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:15.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.446+0000 7f3680ff9700 1 -- 172.21.15.47:0/1651676546 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f367c00cda0 con 0x7f36841471a0 2022-06-29T15:51:15.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.446+0000 7f3680ff9700 1 -- 172.21.15.47:0/1651676546 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f367c0070d0 con 0x7f36841471a0 2022-06-29T15:51:15.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.446+0000 7f368a1d7700 1 -- 172.21.15.47:0/1651676546 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3684142190 con 0x7f36841471a0 2022-06-29T15:51:15.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.446+0000 7f3680ff9700 1 -- 172.21.15.47:0/1651676546 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f367c007430 con 0x7f36841471a0 2022-06-29T15:51:15.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.446+0000 7f368a1d7700 1 -- 172.21.15.47:0/1651676546 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3684142dc0 con 0x7f36841471a0 2022-06-29T15:51:15.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.446+0000 7f368a1d7700 1 -- 172.21.15.47:0/1651676546 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36840573d0 con 0x7f36841471a0 2022-06-29T15:51:15.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.446+0000 7f3680ff9700 1 -- 172.21.15.47:0/1651676546 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f367c01a060 con 0x7f36841471a0 2022-06-29T15:51:15.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.446+0000 7f3680ff9700 1 --2- 172.21.15.47:0/1651676546 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f365c053aa0 0x7f365c055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:15.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.446+0000 7f3680ff9700 1 -- 172.21.15.47:0/1651676546 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f367c06b3c0 con 0x7f36841471a0 2022-06-29T15:51:15.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.446+0000 7f36837fe700 1 --2- 172.21.15.47:0/1651676546 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f365c053aa0 0x7f365c055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:15.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.450+0000 7f36837fe700 1 --2- 172.21.15.47:0/1651676546 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f365c053aa0 0x7f365c055f60 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7f366c00d000 tx=0x7f366c009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:15.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.454+0000 7f3680ff9700 1 -- 172.21.15.47:0/1651676546 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f367c042460 con 0x7f36841471a0 2022-06-29T15:51:15.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.582+0000 7f368a1d7700 1 -- 172.21.15.47:0/1651676546 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f36840b5e50 con 0x7f36841471a0 2022-06-29T15:51:15.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.590+0000 7f3680ff9700 1 -- 172.21.15.47:0/1651676546 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f367c018020 con 0x7f36841471a0 2022-06-29T15:51:15.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.590+0000 7f368a1d7700 1 -- 172.21.15.47:0/1651676546 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f365c053aa0 msgr2=0x7f365c055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:15.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.590+0000 7f368a1d7700 1 --2- 172.21.15.47:0/1651676546 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f365c053aa0 0x7f365c055f60 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7f366c00d000 tx=0x7f366c009040).stop 2022-06-29T15:51:15.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.590+0000 7f368a1d7700 1 -- 172.21.15.47:0/1651676546 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36841471a0 msgr2=0x7f36841428d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:15.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.590+0000 7f368a1d7700 1 --2- 172.21.15.47:0/1651676546 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36841471a0 0x7f36841428d0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 rx=0x7f367c001110 tx=0x7f367c007a80).stop 2022-06-29T15:51:15.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.590+0000 7f368a1d7700 1 -- 172.21.15.47:0/1651676546 shutdown_connections 2022-06-29T15:51:15.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.590+0000 7f368a1d7700 1 --2- 172.21.15.47:0/1651676546 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36841471a0 0x7f36841428d0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:15.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.590+0000 7f368a1d7700 1 --2- 172.21.15.47:0/1651676546 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f36840b4300 0x7f3684155710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:15.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.590+0000 7f368a1d7700 1 --2- 172.21.15.47:0/1651676546 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f365c053aa0 0x7f365c055f60 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:15.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.590+0000 7f368a1d7700 1 --2- 172.21.15.47:0/1651676546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3684060ee0 0x7f3684148cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:15.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.590+0000 7f368a1d7700 1 -- 172.21.15.47:0/1651676546 >> 172.21.15.47:0/1651676546 conn(0x7f368405c510 msgr2=0x7f36841451c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:15.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.590+0000 7f368a1d7700 1 -- 172.21.15.47:0/1651676546 shutdown_connections 2022-06-29T15:51:15.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.590+0000 7f368a1d7700 1 -- 172.21.15.47:0/1651676546 wait complete. 2022-06-29T15:51:15.610 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestMisc.test_fs_new' 2022-06-29T15:51:15.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.706+0000 7f476481d700 1 Processor -- start 2022-06-29T15:51:15.712 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.706+0000 7f476481d700 1 -- start start 2022-06-29T15:51:15.714 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.706+0000 7f476481d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47600b0ce0 0x7f47601449f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:15.714 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.706+0000 7f476481d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f47600a86f0 0x7f4760144ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:15.714 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.706+0000 7f476481d700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47600b2c00 con 0x7f47600b2820 2022-06-29T15:51:15.714 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.706+0000 7f476481d700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4760147660 con 0x7f47600a86f0 2022-06-29T15:51:15.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.706+0000 7f476481d700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47600a7920 con 0x7f47600b0ce0 2022-06-29T15:51:15.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.706+0000 7f475f59e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f47600a86f0 0x7f4760144ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:15.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.706+0000 7f475f59e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f47600a86f0 0x7f4760144ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53360/0 (socket says 172.21.15.47:53360) 2022-06-29T15:51:15.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.706+0000 7f475f59e700 1 -- 172.21.15.47:0/1501604138 learned_addr learned my addr 172.21.15.47:0/1501604138 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:15.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.706+0000 7f475e59c700 1 --2- 172.21.15.47:0/1501604138 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47600b0ce0 0x7f47601449f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:15.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.706+0000 7f475dd9b700 1 -- 172.21.15.47:0/1501604138 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 619727735 0 0) 0x7f47600b2c00 con 0x7f47600b2820 2022-06-29T15:51:15.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.706+0000 7f475dd9b700 1 -- 172.21.15.47:0/1501604138 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4744003260 con 0x7f47600b2820 2022-06-29T15:51:15.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.706+0000 7f475f59e700 1 -- 172.21.15.47:0/1501604138 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47600b0ce0 msgr2=0x7f47601449f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:15.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.706+0000 7f475f59e700 1 --2- 172.21.15.47:0/1501604138 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47600b0ce0 0x7f47601449f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:15.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.706+0000 7f475f59e700 1 -- 172.21.15.47:0/1501604138 >> v1:172.21.15.64:6789/0 conn(0x7f47600b2820 legacy=0x7f4760147d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:15.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.706+0000 7f475f59e700 1 -- 172.21.15.47:0/1501604138 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47600a7bc0 con 0x7f47600a86f0 2022-06-29T15:51:15.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.706+0000 7f475f59e700 1 --2- 172.21.15.47:0/1501604138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47600a86f0 0x7f4760144ee0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 rx=0x7f47580032f0 tx=0x7f4758009290).ready entity=mon.0 client_cookie=5f01f5687726d1c2 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:15.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.706+0000 7f475dd9b700 1 -- 172.21.15.47:0/1501604138 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f475800b6c0 con 0x7f47600a86f0 2022-06-29T15:51:15.718 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.706+0000 7f475dd9b700 1 -- 172.21.15.47:0/1501604138 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f475800b820 con 0x7f47600a86f0 2022-06-29T15:51:15.718 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.706+0000 7f475dd9b700 1 -- 172.21.15.47:0/1501604138 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f475800b990 con 0x7f47600a86f0 2022-06-29T15:51:15.721 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.706+0000 7f476481d700 1 -- 172.21.15.47:0/1501604138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47600a86f0 msgr2=0x7f4760144ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:15.721 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.706+0000 7f476481d700 1 --2- 172.21.15.47:0/1501604138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47600a86f0 0x7f4760144ee0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 rx=0x7f47580032f0 tx=0x7f4758009290).stop 2022-06-29T15:51:15.722 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.706+0000 7f476481d700 1 -- 172.21.15.47:0/1501604138 shutdown_connections 2022-06-29T15:51:15.722 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.706+0000 7f476481d700 1 --2- 172.21.15.47:0/1501604138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47600a86f0 0x7f4760144ee0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:15.722 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.706+0000 7f476481d700 1 --2- 172.21.15.47:0/1501604138 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47600b0ce0 0x7f47601449f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:15.722 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.706+0000 7f476481d700 1 -- 172.21.15.47:0/1501604138 >> 172.21.15.47:0/1501604138 conn(0x7f47600b5e00 msgr2=0x7f47600b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:15.723 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.706+0000 7f476481d700 1 -- 172.21.15.47:0/1501604138 shutdown_connections 2022-06-29T15:51:15.723 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.706+0000 7f476481d700 1 -- 172.21.15.47:0/1501604138 wait complete. 2022-06-29T15:51:15.723 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.710+0000 7f476481d700 1 Processor -- start 2022-06-29T15:51:15.723 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.710+0000 7f476481d700 1 -- start start 2022-06-29T15:51:15.724 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.710+0000 7f476481d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47600a86f0 0x7f47600aa520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:15.724 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.710+0000 7f476481d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47600b0ce0 0x7f476013d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:15.724 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.710+0000 7f476481d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47600b2820 0x7f476013dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:15.724 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.710+0000 7f476481d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47600ad2e0 con 0x7f47600b2820 2022-06-29T15:51:15.724 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.710+0000 7f476481d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47600ac900 con 0x7f47600b0ce0 2022-06-29T15:51:15.725 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.710+0000 7f476481d700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47600acba0 con 0x7f47600a86f0 2022-06-29T15:51:15.725 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.710+0000 7f475f59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47600b2820 0x7f476013dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:15.725 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.710+0000 7f475f59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47600b2820 0x7f476013dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53366/0 (socket says 172.21.15.47:53366) 2022-06-29T15:51:15.725 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.710+0000 7f475f59e700 1 -- 172.21.15.47:0/2606288986 learned_addr learned my addr 172.21.15.47:0/2606288986 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:15.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.710+0000 7f475ed9d700 1 --2- 172.21.15.47:0/2606288986 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47600a86f0 0x7f47600aa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:15.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.710+0000 7f475e59c700 1 --2- 172.21.15.47:0/2606288986 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47600b0ce0 0x7f476013d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:15.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.710+0000 7f475f59e700 1 -- 172.21.15.47:0/2606288986 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47600a86f0 msgr2=0x7f47600aa520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:15.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.710+0000 7f475f59e700 1 --2- 172.21.15.47:0/2606288986 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47600a86f0 0x7f47600aa520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:15.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.710+0000 7f475f59e700 1 -- 172.21.15.47:0/2606288986 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47600b0ce0 msgr2=0x7f476013d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:15.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.710+0000 7f475f59e700 1 --2- 172.21.15.47:0/2606288986 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47600b0ce0 0x7f476013d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:15.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.710+0000 7f475f59e700 1 -- 172.21.15.47:0/2606288986 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4758009530 con 0x7f47600b2820 2022-06-29T15:51:15.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.710+0000 7f475e59c700 1 --2- 172.21.15.47:0/2606288986 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47600b0ce0 0x7f476013d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:51:15.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.710+0000 7f475f59e700 1 --2- 172.21.15.47:0/2606288986 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47600b2820 0x7f476013dd00 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 rx=0x7f47580032f0 tx=0x7f4758016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:15.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.710+0000 7f474ffff700 1 -- 172.21.15.47:0/2606288986 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f475800b990 con 0x7f47600b2820 2022-06-29T15:51:15.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.710+0000 7f474ffff700 1 -- 172.21.15.47:0/2606288986 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4758021620 con 0x7f47600b2820 2022-06-29T15:51:15.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.710+0000 7f474ffff700 1 -- 172.21.15.47:0/2606288986 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f475800b6c0 con 0x7f47600b2820 2022-06-29T15:51:15.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.710+0000 7f476481d700 1 -- 172.21.15.47:0/2606288986 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47600ace40 con 0x7f47600b2820 2022-06-29T15:51:15.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.710+0000 7f476481d700 1 -- 172.21.15.47:0/2606288986 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f476013e560 con 0x7f47600b2820 2022-06-29T15:51:15.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.710+0000 7f476481d700 1 -- 172.21.15.47:0/2606288986 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47600a9590 con 0x7f47600b2820 2022-06-29T15:51:15.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.714+0000 7f474ffff700 1 -- 172.21.15.47:0/2606288986 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4758028070 con 0x7f47600b2820 2022-06-29T15:51:15.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.714+0000 7f474ffff700 1 --2- 172.21.15.47:0/2606288986 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f473c053a10 0x7f473c055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:15.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.714+0000 7f474ffff700 1 -- 172.21.15.47:0/2606288986 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f475806cb70 con 0x7f47600b2820 2022-06-29T15:51:15.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.714+0000 7f475ed9d700 1 --2- 172.21.15.47:0/2606288986 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f473c053a10 0x7f473c055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:15.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.714+0000 7f475ed9d700 1 --2- 172.21.15.47:0/2606288986 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f473c053a10 0x7f473c055ed0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7f475000d000 tx=0x7f47500079f0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:15.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.718+0000 7f474ffff700 1 -- 172.21.15.47:0/2606288986 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4758043b00 con 0x7f47600b2820 2022-06-29T15:51:15.892 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:15.886+0000 7f476481d700 1 -- 172.21.15.47:0/2606288986 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_fs_new"]} v 0) v1 -- 0x7f47600acfd0 con 0x7f47600b2820 2022-06-29T15:51:16.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.174+0000 7f474ffff700 1 -- 172.21.15.47:0/2606288986 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_fs_new"]}]=0 v400) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4758015030 con 0x7f47600b2820 2022-06-29T15:51:16.183 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.174+0000 7f476481d700 1 -- 172.21.15.47:0/2606288986 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f473c053a10 msgr2=0x7f473c055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:16.183 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.174+0000 7f476481d700 1 --2- 172.21.15.47:0/2606288986 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f473c053a10 0x7f473c055ed0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7f475000d000 tx=0x7f47500079f0).stop 2022-06-29T15:51:16.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.174+0000 7f476481d700 1 -- 172.21.15.47:0/2606288986 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47600b2820 msgr2=0x7f476013dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:16.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.174+0000 7f476481d700 1 --2- 172.21.15.47:0/2606288986 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47600b2820 0x7f476013dd00 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 rx=0x7f47580032f0 tx=0x7f4758016000).stop 2022-06-29T15:51:16.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.174+0000 7f476481d700 1 -- 172.21.15.47:0/2606288986 shutdown_connections 2022-06-29T15:51:16.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.174+0000 7f476481d700 1 --2- 172.21.15.47:0/2606288986 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47600b2820 0x7f476013dd00 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:16.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.174+0000 7f476481d700 1 --2- 172.21.15.47:0/2606288986 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f473c053a10 0x7f473c055ed0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:16.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.174+0000 7f476481d700 1 --2- 172.21.15.47:0/2606288986 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47600b0ce0 0x7f476013d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:16.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.174+0000 7f476481d700 1 --2- 172.21.15.47:0/2606288986 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47600a86f0 0x7f47600aa520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:16.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.174+0000 7f476481d700 1 -- 172.21.15.47:0/2606288986 >> 172.21.15.47:0/2606288986 conn(0x7f47600b5e00 msgr2=0x7f47600b1280 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:16.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.174+0000 7f476481d700 1 -- 172.21.15.47:0/2606288986 shutdown_connections 2022-06-29T15:51:16.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.178+0000 7f476481d700 1 -- 172.21.15.47:0/2606288986 wait complete. 2022-06-29T15:51:16.195 INFO:tasks.cephfs_test_runner:test_fs_new (tasks.cephfs.test_misc.TestMisc) ... ok 2022-06-29T15:51:16.195 INFO:tasks.cephfs_test_runner:Starting test: test_getattr_caps (tasks.cephfs.test_misc.TestMisc) 2022-06-29T15:51:16.196 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_getattr_caps' 2022-06-29T15:51:16.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.298+0000 7f4809a5c700 1 Processor -- start 2022-06-29T15:51:16.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.298+0000 7f4809a5c700 1 -- start start 2022-06-29T15:51:16.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.298+0000 7f4809a5c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f480405fa30 0x7f48040b1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:16.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.298+0000 7f4809a5c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f48040b2770 0x7f48040ae170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:16.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.298+0000 7f4809a5c700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4804060fd0 con 0x7f4804060b90 2022-06-29T15:51:16.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.298+0000 7f4809a5c700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f480405fe10 con 0x7f480405fa30 2022-06-29T15:51:16.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.298+0000 7f4809a5c700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48040600e0 con 0x7f48040b2770 2022-06-29T15:51:16.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.298+0000 7f4808a5a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f480405fa30 0x7f48040b1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:16.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.298+0000 7f4808a5a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f480405fa30 0x7f48040b1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53372/0 (socket says 172.21.15.47:53372) 2022-06-29T15:51:16.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.298+0000 7f4808a5a700 1 -- 172.21.15.47:0/503139810 learned_addr learned my addr 172.21.15.47:0/503139810 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:16.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.298+0000 7f480925b700 1 --2- 172.21.15.47:0/503139810 >> v2:172.21.15.64:3300/0 conn(0x7f48040b2770 0x7f48040ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:16.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.298+0000 7f4808a5a700 1 -- 172.21.15.47:0/503139810 >> v2:172.21.15.64:3300/0 conn(0x7f48040b2770 msgr2=0x7f48040ae170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:16.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.298+0000 7f4808a5a700 1 --2- 172.21.15.47:0/503139810 >> v2:172.21.15.64:3300/0 conn(0x7f48040b2770 0x7f48040ae170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:16.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.298+0000 7f4808a5a700 1 -- 172.21.15.47:0/503139810 >> v1:172.21.15.64:6789/0 conn(0x7f4804060b90 legacy=0x7f480413f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:16.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.298+0000 7f4808a5a700 1 -- 172.21.15.47:0/503139810 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4804060380 con 0x7f480405fa30 2022-06-29T15:51:16.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.298+0000 7f4808a5a700 1 --2- 172.21.15.47:0/503139810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f480405fa30 0x7f48040b1ab0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 rx=0x7f47f8002f70 tx=0x7f47f8006e40).ready entity=mon.0 client_cookie=b1dce7257425c4f9 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:16.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.298+0000 7f48037fe700 1 -- 172.21.15.47:0/503139810 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47f800bcb0 con 0x7f480405fa30 2022-06-29T15:51:16.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.298+0000 7f48037fe700 1 -- 172.21.15.47:0/503139810 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47f8000e90 con 0x7f480405fa30 2022-06-29T15:51:16.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.298+0000 7f48037fe700 1 -- 172.21.15.47:0/503139810 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47f80153f0 con 0x7f480405fa30 2022-06-29T15:51:16.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.302+0000 7f4809a5c700 1 -- 172.21.15.47:0/503139810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f480405fa30 msgr2=0x7f48040b1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:16.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.302+0000 7f4809a5c700 1 --2- 172.21.15.47:0/503139810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f480405fa30 0x7f48040b1ab0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 rx=0x7f47f8002f70 tx=0x7f47f8006e40).stop 2022-06-29T15:51:16.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.302+0000 7f4809a5c700 1 -- 172.21.15.47:0/503139810 shutdown_connections 2022-06-29T15:51:16.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.302+0000 7f4809a5c700 1 --2- 172.21.15.47:0/503139810 >> v2:172.21.15.64:3300/0 conn(0x7f48040b2770 0x7f48040ae170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:16.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.302+0000 7f4809a5c700 1 --2- 172.21.15.47:0/503139810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f480405fa30 0x7f48040b1ab0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:16.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.302+0000 7f4809a5c700 1 -- 172.21.15.47:0/503139810 >> 172.21.15.47:0/503139810 conn(0x7f4804062df0 msgr2=0x7f48040631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:16.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.302+0000 7f4809a5c700 1 -- 172.21.15.47:0/503139810 shutdown_connections 2022-06-29T15:51:16.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.302+0000 7f4809a5c700 1 -- 172.21.15.47:0/503139810 wait complete. 2022-06-29T15:51:16.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.302+0000 7f4809a5c700 1 Processor -- start 2022-06-29T15:51:16.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.302+0000 7f4809a5c700 1 -- start start 2022-06-29T15:51:16.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.306+0000 7f4809a5c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f480405fa30 0x7f4804145380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:16.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.306+0000 7f4809a5c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4804060b90 0x7f480413e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:16.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.306+0000 7f4809a5c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f48040b2770 0x7f480414e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:16.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.306+0000 7f4809a5c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f480413ef20 con 0x7f480405fa30 2022-06-29T15:51:16.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.306+0000 7f4809a5c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f480409b2c0 con 0x7f4804060b90 2022-06-29T15:51:16.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.306+0000 7f4809a5c700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f480413ec90 con 0x7f48040b2770 2022-06-29T15:51:16.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.306+0000 7f4808a5a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f480405fa30 0x7f4804145380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:16.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.306+0000 7f4808a5a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f480405fa30 0x7f4804145380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53380/0 (socket says 172.21.15.47:53380) 2022-06-29T15:51:16.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.306+0000 7f4803fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4804060b90 0x7f480413e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:16.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.306+0000 7f4808a5a700 1 -- 172.21.15.47:0/3026191629 learned_addr learned my addr 172.21.15.47:0/3026191629 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:16.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.306+0000 7f480925b700 1 --2- 172.21.15.47:0/3026191629 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f48040b2770 0x7f480414e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:16.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.306+0000 7f4808a5a700 1 -- 172.21.15.47:0/3026191629 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f48040b2770 msgr2=0x7f480414e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:16.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.306+0000 7f4808a5a700 1 --2- 172.21.15.47:0/3026191629 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f48040b2770 0x7f480414e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:16.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.306+0000 7f4808a5a700 1 -- 172.21.15.47:0/3026191629 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4804060b90 msgr2=0x7f480413e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:16.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.306+0000 7f4808a5a700 1 --2- 172.21.15.47:0/3026191629 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4804060b90 0x7f480413e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:16.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.306+0000 7f4808a5a700 1 -- 172.21.15.47:0/3026191629 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f800d4d0 con 0x7f480405fa30 2022-06-29T15:51:16.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.306+0000 7f4808a5a700 1 --2- 172.21.15.47:0/3026191629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f480405fa30 0x7f4804145380 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 rx=0x7f47f8002f70 tx=0x7f47f8018000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:16.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.306+0000 7f4801ffb700 1 -- 172.21.15.47:0/3026191629 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47f8015d40 con 0x7f480405fa30 2022-06-29T15:51:16.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.306+0000 7f4801ffb700 1 -- 172.21.15.47:0/3026191629 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47f800b740 con 0x7f480405fa30 2022-06-29T15:51:16.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.306+0000 7f4801ffb700 1 -- 172.21.15.47:0/3026191629 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47f8015ea0 con 0x7f480405fa30 2022-06-29T15:51:16.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.306+0000 7f4809a5c700 1 -- 172.21.15.47:0/3026191629 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4804098c70 con 0x7f480405fa30 2022-06-29T15:51:16.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.306+0000 7f4809a5c700 1 -- 172.21.15.47:0/3026191629 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48040991b0 con 0x7f480405fa30 2022-06-29T15:51:16.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.310+0000 7f4809a5c700 1 -- 172.21.15.47:0/3026191629 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f480413f560 con 0x7f480405fa30 2022-06-29T15:51:16.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.310+0000 7f4801ffb700 1 -- 172.21.15.47:0/3026191629 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f47f801d040 con 0x7f480405fa30 2022-06-29T15:51:16.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.310+0000 7f4801ffb700 1 --2- 172.21.15.47:0/3026191629 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f47e4053950 0x7f47e4055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:16.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.310+0000 7f4801ffb700 1 -- 172.21.15.47:0/3026191629 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f47f806f8c0 con 0x7f480405fa30 2022-06-29T15:51:16.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.310+0000 7f4803fff700 1 --2- 172.21.15.47:0/3026191629 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f47e4053950 0x7f47e4055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:16.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.310+0000 7f4803fff700 1 --2- 172.21.15.47:0/3026191629 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f47e4053950 0x7f47e4055e10 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7f47f000d000 tx=0x7f47f0009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:16.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.318+0000 7f4801ffb700 1 -- 172.21.15.47:0/3026191629 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f47f8046850 con 0x7f480405fa30 2022-06-29T15:51:16.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:16.486+0000 7f4809a5c700 1 -- 172.21.15.47:0/3026191629 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_getattr_caps"]} v 0) v1 -- 0x7f48040573d0 con 0x7f480405fa30 2022-06-29T15:51:17.183 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.174+0000 7f4801ffb700 1 -- 172.21.15.47:0/3026191629 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_getattr_caps"]}]=0 v401) v1 ==== 131+0+0 (secure 0 0 0) 0x7f47f806f020 con 0x7f480405fa30 2022-06-29T15:51:17.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.178+0000 7f4809a5c700 1 -- 172.21.15.47:0/3026191629 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f47e4053950 msgr2=0x7f47e4055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:17.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.178+0000 7f4809a5c700 1 --2- 172.21.15.47:0/3026191629 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f47e4053950 0x7f47e4055e10 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7f47f000d000 tx=0x7f47f0009040).stop 2022-06-29T15:51:17.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.178+0000 7f4809a5c700 1 -- 172.21.15.47:0/3026191629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f480405fa30 msgr2=0x7f4804145380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:17.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.178+0000 7f4809a5c700 1 --2- 172.21.15.47:0/3026191629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f480405fa30 0x7f4804145380 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 rx=0x7f47f8002f70 tx=0x7f47f8018000).stop 2022-06-29T15:51:17.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.178+0000 7f4809a5c700 1 -- 172.21.15.47:0/3026191629 shutdown_connections 2022-06-29T15:51:17.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.182+0000 7f4809a5c700 1 --2- 172.21.15.47:0/3026191629 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f48040b2770 0x7f480414e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:17.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.182+0000 7f4809a5c700 1 --2- 172.21.15.47:0/3026191629 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f47e4053950 0x7f47e4055e10 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:17.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.182+0000 7f4809a5c700 1 --2- 172.21.15.47:0/3026191629 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4804060b90 0x7f480413e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:17.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.182+0000 7f4809a5c700 1 --2- 172.21.15.47:0/3026191629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f480405fa30 0x7f4804145380 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:17.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.182+0000 7f4809a5c700 1 -- 172.21.15.47:0/3026191629 >> 172.21.15.47:0/3026191629 conn(0x7f4804062df0 msgr2=0x7f48040b53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:17.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.182+0000 7f4809a5c700 1 -- 172.21.15.47:0/3026191629 shutdown_connections 2022-06-29T15:51:17.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.182+0000 7f4809a5c700 1 -- 172.21.15.47:0/3026191629 wait complete. 2022-06-29T15:51:17.204 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-06-29T15:51:17.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.310+0000 7fa0129e1700 1 Processor -- start 2022-06-29T15:51:17.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.310+0000 7fa0129e1700 1 -- start start 2022-06-29T15:51:17.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.310+0000 7fa0129e1700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa00c060ee0 0x7fa00c0612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:17.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.310+0000 7fa0129e1700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa00c0b4300 0x7fa00c1471a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:17.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.310+0000 7fa0129e1700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa00c066f70 0x7fa00c1430e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:17.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.310+0000 7fa0129e1700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa00c067400 con 0x7fa00c060ee0 2022-06-29T15:51:17.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.314+0000 7fa0129e1700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa00c067610 con 0x7fa00c0b4300 2022-06-29T15:51:17.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.314+0000 7fa0129e1700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa00c1467c0 con 0x7fa00c066f70 2022-06-29T15:51:17.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.314+0000 7fa00bfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa00c060ee0 0x7fa00c0612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:17.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.314+0000 7fa010f7e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa00c066f70 0x7fa00c1430e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:17.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.314+0000 7fa00b7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa00c0b4300 0x7fa00c1471a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:17.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.314+0000 7fa010f7e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa00c066f70 0x7fa00c1430e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:50492/0 (socket says 172.21.15.47:50492) 2022-06-29T15:51:17.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.314+0000 7fa010f7e700 1 -- 172.21.15.47:0/1994710128 learned_addr learned my addr 172.21.15.47:0/1994710128 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:17.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.314+0000 7fa010f7e700 1 -- 172.21.15.47:0/1994710128 >> v2:172.21.15.64:3300/0 conn(0x7fa00c0b4300 msgr2=0x7fa00c1471a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:17.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.314+0000 7fa010f7e700 1 --2- 172.21.15.47:0/1994710128 >> v2:172.21.15.64:3300/0 conn(0x7fa00c0b4300 0x7fa00c1471a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:17.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.314+0000 7fa010f7e700 1 -- 172.21.15.47:0/1994710128 >> v2:172.21.15.47:3300/0 conn(0x7fa00c060ee0 msgr2=0x7fa00c0612c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:17.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.314+0000 7fa010f7e700 1 --2- 172.21.15.47:0/1994710128 >> v2:172.21.15.47:3300/0 conn(0x7fa00c060ee0 0x7fa00c0612c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:17.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.314+0000 7fa010f7e700 1 -- 172.21.15.47:0/1994710128 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa00c146a60 con 0x7fa00c066f70 2022-06-29T15:51:17.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.314+0000 7fa010f7e700 1 --2- 172.21.15.47:0/1994710128 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa00c066f70 0x7fa00c1430e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7fa00c0ae790 tx=0x7fa004009290).ready entity=mon.2 client_cookie=48d16a90b612108f server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:17.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.314+0000 7fa00affd700 1 -- 172.21.15.47:0/1994710128 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa00400b680 con 0x7fa00c066f70 2022-06-29T15:51:17.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.314+0000 7fa00affd700 1 -- 172.21.15.47:0/1994710128 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa00400b7e0 con 0x7fa00c066f70 2022-06-29T15:51:17.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.314+0000 7fa00affd700 1 -- 172.21.15.47:0/1994710128 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa00400b950 con 0x7fa00c066f70 2022-06-29T15:51:17.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.314+0000 7fa0129e1700 1 -- 172.21.15.47:0/1994710128 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa00c066f70 msgr2=0x7fa00c1430e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:17.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.314+0000 7fa0129e1700 1 --2- 172.21.15.47:0/1994710128 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa00c066f70 0x7fa00c1430e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7fa00c0ae790 tx=0x7fa004009290).stop 2022-06-29T15:51:17.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.314+0000 7fa0129e1700 1 -- 172.21.15.47:0/1994710128 shutdown_connections 2022-06-29T15:51:17.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.314+0000 7fa0129e1700 1 --2- 172.21.15.47:0/1994710128 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa00c066f70 0x7fa00c1430e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:17.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.314+0000 7fa0129e1700 1 --2- 172.21.15.47:0/1994710128 >> v2:172.21.15.64:3300/0 conn(0x7fa00c0b4300 0x7fa00c1471a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:17.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.314+0000 7fa0129e1700 1 --2- 172.21.15.47:0/1994710128 >> v2:172.21.15.47:3300/0 conn(0x7fa00c060ee0 0x7fa00c0612c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:17.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.314+0000 7fa0129e1700 1 -- 172.21.15.47:0/1994710128 >> 172.21.15.47:0/1994710128 conn(0x7fa00c05c510 msgr2=0x7fa00c05f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:17.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.314+0000 7fa0129e1700 1 -- 172.21.15.47:0/1994710128 shutdown_connections 2022-06-29T15:51:17.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.314+0000 7fa0129e1700 1 -- 172.21.15.47:0/1994710128 wait complete. 2022-06-29T15:51:17.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.318+0000 7fa0129e1700 1 Processor -- start 2022-06-29T15:51:17.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.318+0000 7fa0129e1700 1 -- start start 2022-06-29T15:51:17.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.318+0000 7fa0129e1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa00c060ee0 0x7fa00c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:17.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.318+0000 7fa0129e1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa00c066f70 0x7fa00c1496f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:17.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.318+0000 7fa0129e1700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa00c0b4300 0x7fa00c149be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:17.335 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.318+0000 7fa0129e1700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa00c144260 con 0x7fa00c066f70 2022-06-29T15:51:17.335 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.318+0000 7fa0129e1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa00c14a6e0 con 0x7fa00c060ee0 2022-06-29T15:51:17.335 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.318+0000 7fa0129e1700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa00c143fc0 con 0x7fa00c0b4300 2022-06-29T15:51:17.335 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.318+0000 7fa00b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa00c066f70 0x7fa00c1496f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:17.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.318+0000 7fa00b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa00c066f70 0x7fa00c1496f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53396/0 (socket says 172.21.15.47:53396) 2022-06-29T15:51:17.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.318+0000 7fa010f7e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa00c0b4300 0x7fa00c149be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:17.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.318+0000 7fa00b7fe700 1 -- 172.21.15.47:0/1014030706 learned_addr learned my addr 172.21.15.47:0/1014030706 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:17.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.318+0000 7fa00bfff700 1 --2- 172.21.15.47:0/1014030706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa00c060ee0 0x7fa00c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:17.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.318+0000 7fa00b7fe700 1 -- 172.21.15.47:0/1014030706 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa00c0b4300 msgr2=0x7fa00c149be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:17.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.318+0000 7fa00b7fe700 1 --2- 172.21.15.47:0/1014030706 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa00c0b4300 0x7fa00c149be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:17.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.318+0000 7fa00b7fe700 1 -- 172.21.15.47:0/1014030706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa00c060ee0 msgr2=0x7fa00c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:17.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.318+0000 7fa00b7fe700 1 --2- 172.21.15.47:0/1014030706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa00c060ee0 0x7fa00c0b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:17.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.318+0000 7fa00b7fe700 1 -- 172.21.15.47:0/1014030706 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa004009530 con 0x7fa00c066f70 2022-06-29T15:51:17.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.318+0000 7fa00b7fe700 1 --2- 172.21.15.47:0/1014030706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa00c066f70 0x7fa00c1496f0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 rx=0x7f9ffc00c000 tx=0x7f9ffc006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:17.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.318+0000 7fa0097fa700 1 -- 172.21.15.47:0/1014030706 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ffc010780 con 0x7fa00c066f70 2022-06-29T15:51:17.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.318+0000 7fa0129e1700 1 -- 172.21.15.47:0/1014030706 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa00c156510 con 0x7fa00c066f70 2022-06-29T15:51:17.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.318+0000 7fa0097fa700 1 -- 172.21.15.47:0/1014030706 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ffc0108e0 con 0x7fa00c066f70 2022-06-29T15:51:17.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.318+0000 7fa0097fa700 1 -- 172.21.15.47:0/1014030706 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ffc010c40 con 0x7fa00c066f70 2022-06-29T15:51:17.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.318+0000 7fa0129e1700 1 -- 172.21.15.47:0/1014030706 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa00c14a0d0 con 0x7fa00c066f70 2022-06-29T15:51:17.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.322+0000 7fa0097fa700 1 -- 172.21.15.47:0/1014030706 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9ffc01c460 con 0x7fa00c066f70 2022-06-29T15:51:17.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.322+0000 7fa0129e1700 1 -- 172.21.15.47:0/1014030706 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa00c156370 con 0x7fa00c066f70 2022-06-29T15:51:17.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.322+0000 7fa0097fa700 1 --2- 172.21.15.47:0/1014030706 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9fec053a70 0x7f9fec055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:17.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.322+0000 7fa0097fa700 1 -- 172.21.15.47:0/1014030706 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f9ffc06c290 con 0x7fa00c066f70 2022-06-29T15:51:17.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.322+0000 7fa00bfff700 1 --2- 172.21.15.47:0/1014030706 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9fec053a70 0x7f9fec055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:17.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.322+0000 7fa00bfff700 1 --2- 172.21.15.47:0/1014030706 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9fec053a70 0x7f9fec055f30 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7fa00000b000 tx=0x7fa000006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:17.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.326+0000 7fa0097fa700 1 -- 172.21.15.47:0/1014030706 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa00c156370 con 0x7fa00c066f70 2022-06-29T15:51:17.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.454+0000 7fa0129e1700 1 -- 172.21.15.47:0/1014030706 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa00c156370 con 0x7fa00c066f70 2022-06-29T15:51:17.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.462+0000 7fa0097fa700 1 -- 172.21.15.47:0/1014030706 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa00c156370 con 0x7fa00c066f70 2022-06-29T15:51:17.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.466+0000 7fa0129e1700 1 -- 172.21.15.47:0/1014030706 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9fec053a70 msgr2=0x7f9fec055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:17.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.466+0000 7fa0129e1700 1 --2- 172.21.15.47:0/1014030706 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9fec053a70 0x7f9fec055f30 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7fa00000b000 tx=0x7fa000006df0).stop 2022-06-29T15:51:17.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.466+0000 7fa0129e1700 1 -- 172.21.15.47:0/1014030706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa00c066f70 msgr2=0x7fa00c1496f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:17.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.466+0000 7fa0129e1700 1 --2- 172.21.15.47:0/1014030706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa00c066f70 0x7fa00c1496f0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 rx=0x7f9ffc00c000 tx=0x7f9ffc006df0).stop 2022-06-29T15:51:17.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.466+0000 7fa0129e1700 1 -- 172.21.15.47:0/1014030706 shutdown_connections 2022-06-29T15:51:17.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.466+0000 7fa0129e1700 1 --2- 172.21.15.47:0/1014030706 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa00c0b4300 0x7fa00c149be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:17.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.466+0000 7fa0129e1700 1 --2- 172.21.15.47:0/1014030706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa00c066f70 0x7fa00c1496f0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:17.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.466+0000 7fa0129e1700 1 --2- 172.21.15.47:0/1014030706 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9fec053a70 0x7f9fec055f30 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:17.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.466+0000 7fa0129e1700 1 --2- 172.21.15.47:0/1014030706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa00c060ee0 0x7fa00c0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:17.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.466+0000 7fa0129e1700 1 -- 172.21.15.47:0/1014030706 >> 172.21.15.47:0/1014030706 conn(0x7fa00c05c510 msgr2=0x7fa00c0b7990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:17.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.466+0000 7fa0129e1700 1 -- 172.21.15.47:0/1014030706 shutdown_connections 2022-06-29T15:51:17.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.466+0000 7fa0129e1700 1 -- 172.21.15.47:0/1014030706 wait complete. 2022-06-29T15:51:17.486 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-06-29T15:51:17.487 DEBUG:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:51:17.501 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:51:17.524 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:51:17.545 DEBUG:teuthology.orchestra.run.smithi164:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:51:17.567 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-06-29T15:51:17.567 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:51:17.568 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:51:17.568 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:51:17.575 INFO:teuthology.orchestra.run.smithi099.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-06-29T15:51:17.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T15:51:17.576 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-06-29T15:51:17.577 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:51:17.577 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:51:17.578 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:51:17.585 INFO:teuthology.orchestra.run.smithi164.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-06-29T15:51:17.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T15:51:17.588 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-06-29T15:51:17.588 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-06-29T15:51:17.699 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.694+0000 7f63effe3700 1 Processor -- start 2022-06-29T15:51:17.700 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.694+0000 7f63effe3700 1 -- start start 2022-06-29T15:51:17.702 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.694+0000 7f63effe3700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f63e80aca80 0x7f63e813f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:17.702 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.694+0000 7f63effe3700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63e80af710 con 0x7f63e80af190 2022-06-29T15:51:17.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.694+0000 7f63effe3700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63e8060d40 con 0x7f63e805f760 2022-06-29T15:51:17.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.694+0000 7f63effe3700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63e80b2830 con 0x7f63e80aca80 2022-06-29T15:51:17.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.694+0000 7f63edd7f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f63e80aca80 0x7f63e813f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:17.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.694+0000 7f63ed57e700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f63e80af190 0x7f63e80b2120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38954/0 (socket says 172.21.15.47:38954) 2022-06-29T15:51:17.704 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.694+0000 7f63edd7f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f63e80aca80 0x7f63e813f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53400/0 (socket says 172.21.15.47:53400) 2022-06-29T15:51:17.704 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.694+0000 7f63ed57e700 1 -- 172.21.15.47:0/1491256154 learned_addr learned my addr 172.21.15.47:0/1491256154 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:17.704 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.694+0000 7f63edd7f700 1 -- 172.21.15.47:0/1491256154 >> v1:172.21.15.64:6789/0 conn(0x7f63e805f760 legacy=0x7f63e80abb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:17.704 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.694+0000 7f63edd7f700 1 -- 172.21.15.47:0/1491256154 >> v1:172.21.15.47:6789/0 conn(0x7f63e80af190 legacy=0x7f63e80b2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:17.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.694+0000 7f63edd7f700 1 -- 172.21.15.47:0/1491256154 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e80aeec0 con 0x7f63e80aca80 2022-06-29T15:51:17.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.694+0000 7f63ecd7d700 1 -- 172.21.15.47:0/1491256154 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1657588771 0 0) 0x7f63e80af710 con 0x7f63e80af190 2022-06-29T15:51:17.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.694+0000 7f63edd7f700 1 --2- 172.21.15.47:0/1491256154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63e80aca80 0x7f63e813f5d0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 rx=0x7f63e80ae400 tx=0x7f63e4006df0).ready entity=mon.0 client_cookie=e9913bcbd5982d0a server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:17.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.694+0000 7f63ecd7d700 1 -- 172.21.15.47:0/1491256154 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f63e400d610 con 0x7f63e80aca80 2022-06-29T15:51:17.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.694+0000 7f63ecd7d700 1 -- 172.21.15.47:0/1491256154 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63e4000e90 con 0x7f63e80aca80 2022-06-29T15:51:17.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.694+0000 7f63ecd7d700 1 -- 172.21.15.47:0/1491256154 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f63e400d960 con 0x7f63e80aca80 2022-06-29T15:51:17.708 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.694+0000 7f63effe3700 1 -- 172.21.15.47:0/1491256154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63e80aca80 msgr2=0x7f63e813f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:17.708 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.694+0000 7f63effe3700 1 --2- 172.21.15.47:0/1491256154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63e80aca80 0x7f63e813f5d0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 rx=0x7f63e80ae400 tx=0x7f63e4006df0).stop 2022-06-29T15:51:17.708 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.698+0000 7f63effe3700 1 -- 172.21.15.47:0/1491256154 shutdown_connections 2022-06-29T15:51:17.709 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.698+0000 7f63effe3700 1 --2- 172.21.15.47:0/1491256154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63e80aca80 0x7f63e813f5d0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:17.709 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.698+0000 7f63effe3700 1 -- 172.21.15.47:0/1491256154 >> 172.21.15.47:0/1491256154 conn(0x7f63e80626f0 msgr2=0x7f63e8062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:17.709 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.698+0000 7f63effe3700 1 -- 172.21.15.47:0/1491256154 shutdown_connections 2022-06-29T15:51:17.709 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.698+0000 7f63effe3700 1 -- 172.21.15.47:0/1491256154 wait complete. 2022-06-29T15:51:17.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.698+0000 7f63effe3700 1 Processor -- start 2022-06-29T15:51:17.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.698+0000 7f63effe3700 1 -- start start 2022-06-29T15:51:17.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.702+0000 7f63effe3700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f63e805f760 0x7f63e8065610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:17.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.702+0000 7f63effe3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63e80aca80 0x7f63e813b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:17.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.702+0000 7f63effe3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63e80af190 0x7f63e813f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:17.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.702+0000 7f63effe3700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e8150320 con 0x7f63e80af190 2022-06-29T15:51:17.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.702+0000 7f63effe3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e8143aa0 con 0x7f63e80aca80 2022-06-29T15:51:17.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.702+0000 7f63effe3700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63e8066f00 con 0x7f63e805f760 2022-06-29T15:51:17.712 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.702+0000 7f63ee580700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63e80af190 0x7f63e813f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:17.712 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.702+0000 7f63ee580700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63e80af190 0x7f63e813f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53410/0 (socket says 172.21.15.47:53410) 2022-06-29T15:51:17.712 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.702+0000 7f63ee580700 1 -- 172.21.15.47:0/445142114 learned_addr learned my addr 172.21.15.47:0/445142114 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:17.712 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.702+0000 7f63ed57e700 1 --2- 172.21.15.47:0/445142114 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63e80aca80 0x7f63e813b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:17.713 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.702+0000 7f63edd7f700 1 --2- 172.21.15.47:0/445142114 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f63e805f760 0x7f63e8065610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:17.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.702+0000 7f63ee580700 1 -- 172.21.15.47:0/445142114 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f63e805f760 msgr2=0x7f63e8065610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:17.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.702+0000 7f63ee580700 1 --2- 172.21.15.47:0/445142114 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f63e805f760 0x7f63e8065610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:17.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.702+0000 7f63ee580700 1 -- 172.21.15.47:0/445142114 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63e80aca80 msgr2=0x7f63e813b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:17.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.702+0000 7f63ee580700 1 --2- 172.21.15.47:0/445142114 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63e80aca80 0x7f63e813b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:17.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.702+0000 7f63ee580700 1 -- 172.21.15.47:0/445142114 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e400b180 con 0x7f63e80af190 2022-06-29T15:51:17.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.702+0000 7f63ee580700 1 --2- 172.21.15.47:0/445142114 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63e80af190 0x7f63e813f150 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 rx=0x7f63e8143be0 tx=0x7f63e0007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:17.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.702+0000 7f63d6ffd700 1 -- 172.21.15.47:0/445142114 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f63e000f470 con 0x7f63e80af190 2022-06-29T15:51:17.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.702+0000 7f63d6ffd700 1 -- 172.21.15.47:0/445142114 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63e000f5d0 con 0x7f63e80af190 2022-06-29T15:51:17.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.702+0000 7f63effe3700 1 -- 172.21.15.47:0/445142114 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63e8067130 con 0x7f63e80af190 2022-06-29T15:51:17.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.702+0000 7f63d6ffd700 1 -- 172.21.15.47:0/445142114 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f63e000f930 con 0x7f63e80af190 2022-06-29T15:51:17.718 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.702+0000 7f63effe3700 1 -- 172.21.15.47:0/445142114 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63e8065ec0 con 0x7f63e80af190 2022-06-29T15:51:17.718 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.706+0000 7f63effe3700 1 -- 172.21.15.47:0/445142114 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63e8066080 con 0x7f63e80af190 2022-06-29T15:51:17.718 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.706+0000 7f63d6ffd700 1 -- 172.21.15.47:0/445142114 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f63e0019030 con 0x7f63e80af190 2022-06-29T15:51:17.718 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.706+0000 7f63d6ffd700 1 --2- 172.21.15.47:0/445142114 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f63c80539e0 0x7f63c8055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:17.719 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.706+0000 7f63d6ffd700 1 -- 172.21.15.47:0/445142114 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f63e006b4c0 con 0x7f63e80af190 2022-06-29T15:51:17.719 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.706+0000 7f63edd7f700 1 --2- 172.21.15.47:0/445142114 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f63c80539e0 0x7f63c8055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:17.719 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.706+0000 7f63edd7f700 1 --2- 172.21.15.47:0/445142114 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f63c80539e0 0x7f63c8055ea0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f63e401c000 tx=0x7f63e4015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:17.720 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.714+0000 7f63d6ffd700 1 -- 172.21.15.47:0/445142114 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f63e0042450 con 0x7f63e80af190 2022-06-29T15:51:17.847 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.842+0000 7f63effe3700 1 -- 172.21.15.47:0/445142114 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f63e80573d0 con 0x7f63e80af190 2022-06-29T15:51:17.849 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.842+0000 7f63d6ffd700 1 -- 172.21.15.47:0/445142114 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v135) v1 ==== 81+0+29229 (secure 0 0 0) 0x7f63e0045a70 con 0x7f63e80af190 2022-06-29T15:51:17.850 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:17.850 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2022-06-29T15:51:17.850 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 135, 2022-06-29T15:51:17.850 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "9a68894a-53ce-400e-838b-14eab91b834c", 2022-06-29T15:51:17.851 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2022-06-29T15:42:14.046361+0000", 2022-06-29T15:51:17.851 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2022-06-29T15:51:15.298789+0000", 2022-06-29T15:51:17.851 INFO:teuthology.orchestra.run.smithi047.stdout: "last_up_change": "2022-06-29T15:42:20.929789+0000", 2022-06-29T15:51:17.851 INFO:teuthology.orchestra.run.smithi047.stdout: "last_in_change": "2022-06-29T15:42:20.929789+0000", 2022-06-29T15:51:17.851 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-06-29T15:51:17.852 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_num": 5799936, 2022-06-29T15:51:17.852 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_set": [ 2022-06-29T15:51:17.852 INFO:teuthology.orchestra.run.smithi047.stdout: "pglog_hardlimit", 2022-06-29T15:51:17.852 INFO:teuthology.orchestra.run.smithi047.stdout: "purged_snapdirs", 2022-06-29T15:51:17.853 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_deletes", 2022-06-29T15:51:17.853 INFO:teuthology.orchestra.run.smithi047.stdout: "sortbitwise" 2022-06-29T15:51:17.853 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:51:17.853 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_version": 5, 2022-06-29T15:51:17.853 INFO:teuthology.orchestra.run.smithi047.stdout: "full_ratio": 0.94999998807907104, 2022-06-29T15:51:17.854 INFO:teuthology.orchestra.run.smithi047.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-06-29T15:51:17.854 INFO:teuthology.orchestra.run.smithi047.stdout: "nearfull_ratio": 0.85000002384185791, 2022-06-29T15:51:17.854 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_snapshot": "", 2022-06-29T15:51:17.854 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_max": 24, 2022-06-29T15:51:17.855 INFO:teuthology.orchestra.run.smithi047.stdout: "max_osd": 8, 2022-06-29T15:51:17.855 INFO:teuthology.orchestra.run.smithi047.stdout: "require_min_compat_client": "jewel", 2022-06-29T15:51:17.855 INFO:teuthology.orchestra.run.smithi047.stdout: "min_compat_client": "jewel", 2022-06-29T15:51:17.855 INFO:teuthology.orchestra.run.smithi047.stdout: "require_osd_release": "octopus", 2022-06-29T15:51:17.855 INFO:teuthology.orchestra.run.smithi047.stdout: "pools": [ 2022-06-29T15:51:17.856 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.856 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 1, 2022-06-29T15:51:17.856 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "device_health_metrics", 2022-06-29T15:51:17.856 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:42:16.971431+0000", 2022-06-29T15:51:17.856 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:51:17.857 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:51:17.857 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:51:17.857 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:51:17.857 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:51:17.858 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:51:17.858 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:51:17.858 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:51:17.858 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 1, 2022-06-29T15:51:17.859 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 1, 2022-06-29T15:51:17.859 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 1, 2022-06-29T15:51:17.859 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 1, 2022-06-29T15:51:17.859 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 1, 2022-06-29T15:51:17.860 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:51:17.860 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:51:17.860 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:51:17.860 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:51:17.860 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:51:17.861 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:51:17.861 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:51:17.861 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.861 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "16", 2022-06-29T15:51:17.862 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:51:17.862 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:51:17.862 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:51:17.862 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:51:17.862 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:51:17.863 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:51:17.863 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:51:17.863 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:51:17.863 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:51:17.863 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:51:17.864 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:51:17.864 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:51:17.864 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:51:17.864 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:51:17.865 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:51:17.865 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:51:17.865 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:51:17.865 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:51:17.865 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:51:17.866 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:51:17.866 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:51:17.866 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:51:17.866 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:51:17.867 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:51:17.867 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:51:17.867 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:51:17.867 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.867 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:51:17.868 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:51:17.868 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:51:17.868 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:51:17.868 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:51:17.869 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:51:17.869 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:51:17.869 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:51:17.869 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:51:17.869 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:51:17.870 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:51:17.870 INFO:teuthology.orchestra.run.smithi047.stdout: "options": { 2022-06-29T15:51:17.870 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_min": 1 2022-06-29T15:51:17.870 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.871 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:51:17.871 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_devicehealth": {} 2022-06-29T15:51:17.871 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:17.871 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.871 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.872 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 2, 2022-06-29T15:51:17.872 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "rbd", 2022-06-29T15:51:17.872 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:42:25.290884+0000", 2022-06-29T15:51:17.872 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:51:17.872 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:51:17.873 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:51:17.873 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:51:17.873 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:51:17.873 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:51:17.874 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:51:17.874 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:51:17.874 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2022-06-29T15:51:17.874 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2022-06-29T15:51:17.874 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2022-06-29T15:51:17.875 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2022-06-29T15:51:17.875 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2022-06-29T15:51:17.875 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:51:17.875 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:51:17.876 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:51:17.876 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:51:17.876 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:51:17.876 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:51:17.876 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:51:17.877 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.877 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "19", 2022-06-29T15:51:17.877 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:51:17.877 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:51:17.877 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:51:17.878 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:51:17.878 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:51:17.878 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:51:17.878 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:51:17.879 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:51:17.879 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:51:17.879 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:51:17.879 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:51:17.879 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:51:17.880 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:51:17.880 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:51:17.880 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:51:17.880 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:51:17.881 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:51:17.881 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:51:17.881 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:51:17.881 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:51:17.882 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:51:17.882 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:51:17.882 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:51:17.882 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:51:17.883 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:51:17.883 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:51:17.883 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.883 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:51:17.883 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:51:17.884 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:51:17.884 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:51:17.884 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:51:17.884 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:51:17.884 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:51:17.885 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:51:17.885 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:51:17.885 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:51:17.885 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:51:17.886 INFO:teuthology.orchestra.run.smithi047.stdout: "options": {}, 2022-06-29T15:51:17.886 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:51:17.886 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd": {} 2022-06-29T15:51:17.886 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:17.886 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:17.887 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:51:17.887 INFO:teuthology.orchestra.run.smithi047.stdout: "osds": [ 2022-06-29T15:51:17.887 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.887 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2022-06-29T15:51:17.887 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "e23eb023-4706-4201-bf72-4bf19c912c5d", 2022-06-29T15:51:17.888 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:51:17.888 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:51:17.888 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:51:17.888 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:51:17.889 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:51:17.889 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:51:17.889 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:51:17.889 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 131, 2022-06-29T15:51:17.889 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:51:17.890 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:51:17.890 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:51:17.890 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:17.890 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.891 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:17.891 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6818", 2022-06-29T15:51:17.891 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:51:17.891 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.891 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.892 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:17.892 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6819", 2022-06-29T15:51:17.892 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:51:17.892 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:17.892 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:17.893 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.893 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:51:17.893 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:17.893 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.894 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:17.894 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6820", 2022-06-29T15:51:17.894 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:51:17.894 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.894 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.895 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:17.895 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6821", 2022-06-29T15:51:17.895 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:51:17.895 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:17.895 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:17.896 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.896 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:51:17.896 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:17.896 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.897 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:17.897 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6824", 2022-06-29T15:51:17.897 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:51:17.897 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.897 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.898 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:17.898 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6825", 2022-06-29T15:51:17.898 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:51:17.898 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:17.899 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:17.899 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.899 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:51:17.899 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:17.899 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.900 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:17.900 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6822", 2022-06-29T15:51:17.900 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:51:17.900 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.901 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.901 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:17.901 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6823", 2022-06-29T15:51:17.901 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:51:17.902 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:17.902 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:17.902 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.902 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6819/13096", 2022-06-29T15:51:17.902 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6821/13096", 2022-06-29T15:51:17.903 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6825/13096", 2022-06-29T15:51:17.903 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6823/13096", 2022-06-29T15:51:17.903 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:51:17.903 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:51:17.903 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:51:17.904 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:17.904 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.904 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.904 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2022-06-29T15:51:17.905 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "228bb08d-2d64-43d9-9560-bfaeff154452", 2022-06-29T15:51:17.905 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:51:17.905 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:51:17.905 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:51:17.905 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:51:17.906 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:51:17.906 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:51:17.906 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:51:17.906 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 131, 2022-06-29T15:51:17.907 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:51:17.907 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:51:17.907 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:51:17.907 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:17.907 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.908 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:17.908 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6826", 2022-06-29T15:51:17.908 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:51:17.908 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.908 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.909 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:17.909 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6827", 2022-06-29T15:51:17.909 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:51:17.909 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:17.910 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:17.910 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.910 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:51:17.910 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:17.911 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.911 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:17.911 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6828", 2022-06-29T15:51:17.911 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:51:17.911 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.912 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.912 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:17.912 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6829", 2022-06-29T15:51:17.912 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:51:17.912 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:17.913 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:17.913 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.913 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:51:17.913 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:17.914 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.914 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:17.914 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6832", 2022-06-29T15:51:17.914 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:51:17.914 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.915 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.915 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:17.915 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6833", 2022-06-29T15:51:17.915 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:51:17.915 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:17.916 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:17.916 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.916 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:51:17.916 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:17.917 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.917 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:17.917 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6830", 2022-06-29T15:51:17.917 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:51:17.917 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.918 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.918 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:17.918 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6831", 2022-06-29T15:51:17.918 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:51:17.918 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:17.919 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:17.919 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.919 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6827/13100", 2022-06-29T15:51:17.919 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6829/13100", 2022-06-29T15:51:17.920 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6833/13100", 2022-06-29T15:51:17.920 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6831/13100", 2022-06-29T15:51:17.920 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:51:17.920 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:51:17.920 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:51:17.921 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:17.921 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.921 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.921 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2022-06-29T15:51:17.922 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "797a5452-1510-4640-a111-c9c2fcf707eb", 2022-06-29T15:51:17.922 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:51:17.922 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:51:17.922 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:51:17.922 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:51:17.923 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:51:17.923 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:51:17.923 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:51:17.923 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 126, 2022-06-29T15:51:17.924 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:51:17.924 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:51:17.924 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:51:17.924 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:17.924 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.925 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:17.925 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6810", 2022-06-29T15:51:17.925 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:51:17.925 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.925 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.926 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:17.926 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6811", 2022-06-29T15:51:17.926 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:51:17.926 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:17.927 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:17.927 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.927 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:51:17.927 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:17.927 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.928 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:17.928 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6812", 2022-06-29T15:51:17.928 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:51:17.928 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.929 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.929 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:17.929 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6813", 2022-06-29T15:51:17.929 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:51:17.929 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:17.930 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:17.930 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.930 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:51:17.930 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:17.931 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.931 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:17.931 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6816", 2022-06-29T15:51:17.931 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:51:17.931 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.932 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.932 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:17.932 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6817", 2022-06-29T15:51:17.932 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:51:17.933 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:17.933 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:17.933 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.933 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:51:17.933 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:17.934 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.934 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:17.934 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6814", 2022-06-29T15:51:17.934 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:51:17.935 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.935 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.935 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:17.935 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6815", 2022-06-29T15:51:17.935 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:51:17.936 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:17.936 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:17.936 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.936 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6811/13098", 2022-06-29T15:51:17.937 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6813/13098", 2022-06-29T15:51:17.937 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6817/13098", 2022-06-29T15:51:17.937 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6815/13098", 2022-06-29T15:51:17.937 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:51:17.937 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:51:17.938 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:51:17.938 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:17.938 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.938 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.938 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2022-06-29T15:51:17.939 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "55b24fd7-bb71-4a01-b3fa-a09caa772a62", 2022-06-29T15:51:17.939 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:51:17.939 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:51:17.939 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:51:17.940 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:51:17.940 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:51:17.940 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:51:17.940 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:51:17.940 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 131, 2022-06-29T15:51:17.941 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:51:17.941 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:51:17.941 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:51:17.941 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:17.942 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.942 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:17.942 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6802", 2022-06-29T15:51:17.942 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:51:17.942 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.943 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.943 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:17.943 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6803", 2022-06-29T15:51:17.943 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:51:17.944 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:17.944 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:17.944 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.944 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:51:17.944 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:17.945 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.945 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:17.945 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6804", 2022-06-29T15:51:17.945 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:51:17.946 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.946 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.946 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:17.946 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6805", 2022-06-29T15:51:17.946 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:51:17.947 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:17.947 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:17.947 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.947 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:51:17.948 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:17.948 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.948 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:17.948 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6808", 2022-06-29T15:51:17.948 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:51:17.949 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.949 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.949 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:17.949 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6809", 2022-06-29T15:51:17.950 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:51:17.950 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:17.950 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:17.950 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.950 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:51:17.951 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:17.951 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.951 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:17.951 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6806", 2022-06-29T15:51:17.951 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:51:17.952 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.952 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.952 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:17.952 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6807", 2022-06-29T15:51:17.953 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:51:17.953 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:17.953 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:17.953 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.954 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6803/13097", 2022-06-29T15:51:17.954 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6805/13097", 2022-06-29T15:51:17.954 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6809/13097", 2022-06-29T15:51:17.954 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6807/13097", 2022-06-29T15:51:17.954 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:51:17.955 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:51:17.955 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:51:17.955 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:17.955 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.956 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.956 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2022-06-29T15:51:17.956 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3", 2022-06-29T15:51:17.956 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:51:17.956 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:51:17.957 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:51:17.957 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:51:17.957 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:51:17.957 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:51:17.957 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:51:17.958 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 126, 2022-06-29T15:51:17.958 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:51:17.958 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:51:17.958 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:51:17.959 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:17.959 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.959 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:17.959 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6816", 2022-06-29T15:51:17.959 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:51:17.960 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.960 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.960 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:17.960 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6817", 2022-06-29T15:51:17.961 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:51:17.961 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:17.961 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:17.961 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.961 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:51:17.962 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:17.962 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.962 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:17.962 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6818", 2022-06-29T15:51:17.963 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:51:17.963 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.963 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.963 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:17.963 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6819", 2022-06-29T15:51:17.964 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:51:17.964 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:17.964 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:17.964 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.965 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:51:17.965 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:17.965 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.965 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:17.965 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6822", 2022-06-29T15:51:17.966 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:51:17.966 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.966 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.966 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:17.966 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6823", 2022-06-29T15:51:17.967 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:51:17.967 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:17.967 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:17.967 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.967 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:51:17.968 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:17.968 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.968 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:17.968 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6820", 2022-06-29T15:51:17.969 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:51:17.969 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.969 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.969 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:17.969 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6821", 2022-06-29T15:51:17.970 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:51:17.970 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:17.970 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:17.970 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.971 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6817/13396", 2022-06-29T15:51:17.971 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6819/13396", 2022-06-29T15:51:17.971 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6823/13396", 2022-06-29T15:51:17.971 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6821/13396", 2022-06-29T15:51:17.972 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:51:17.972 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:51:17.972 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:51:17.972 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:17.972 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.973 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.973 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2022-06-29T15:51:17.973 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "f77beebe-6cca-4244-b2d4-677825ebf919", 2022-06-29T15:51:17.973 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:51:17.974 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:51:17.974 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:51:17.974 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:51:17.974 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:51:17.975 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:51:17.975 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:51:17.975 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 131, 2022-06-29T15:51:17.975 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:51:17.975 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:51:17.976 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:51:17.976 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:17.976 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.976 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:17.977 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6800", 2022-06-29T15:51:17.977 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:51:17.977 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.977 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.977 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:17.978 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6801", 2022-06-29T15:51:17.978 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:51:17.978 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:17.978 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:17.978 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.979 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:51:17.979 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:17.979 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.979 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:17.980 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6802", 2022-06-29T15:51:17.980 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:51:17.980 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.980 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.980 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:17.981 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6803", 2022-06-29T15:51:17.981 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:51:17.981 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:17.981 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:17.982 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.982 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:51:17.982 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:17.982 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.983 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:17.983 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6806", 2022-06-29T15:51:17.983 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:51:17.983 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.984 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.984 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:17.984 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6807", 2022-06-29T15:51:17.984 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:51:17.984 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:17.985 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:17.985 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.985 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:51:17.985 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:17.985 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.986 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:17.986 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6804", 2022-06-29T15:51:17.986 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:51:17.986 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.987 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.987 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:17.987 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6805", 2022-06-29T15:51:17.987 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:51:17.987 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:17.988 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:17.988 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.988 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6801/13393", 2022-06-29T15:51:17.988 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6803/13393", 2022-06-29T15:51:17.988 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6807/13393", 2022-06-29T15:51:17.989 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6805/13393", 2022-06-29T15:51:17.989 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:51:17.989 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:51:17.989 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:51:17.990 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:17.990 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.990 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.990 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2022-06-29T15:51:17.991 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "af0ca9b3-c02c-4910-834b-bf0b8dc6713c", 2022-06-29T15:51:17.991 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:51:17.991 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:51:17.991 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:51:17.991 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:51:17.992 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:51:17.992 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:51:17.992 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:51:17.992 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 131, 2022-06-29T15:51:17.992 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:51:17.993 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:51:17.993 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:51:17.993 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:17.993 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.994 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:17.994 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6824", 2022-06-29T15:51:17.994 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:51:17.994 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.994 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.995 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:17.995 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6825", 2022-06-29T15:51:17.995 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:51:17.995 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:17.996 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:17.996 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.996 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:51:17.996 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:17.996 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.997 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:17.997 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6826", 2022-06-29T15:51:17.997 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:51:17.997 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.997 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:17.998 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:17.998 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6827", 2022-06-29T15:51:17.998 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:51:17.998 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:17.999 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:17.999 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:17.999 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:51:17.999 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:17.999 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:18.000 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:18.000 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6830", 2022-06-29T15:51:18.000 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:51:18.000 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:18.001 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:18.001 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:18.001 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6831", 2022-06-29T15:51:18.001 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:51:18.001 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:18.002 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:18.002 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:18.002 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:51:18.002 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:18.003 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:18.003 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:18.003 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6828", 2022-06-29T15:51:18.003 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:51:18.003 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:18.004 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:18.004 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:18.004 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6829", 2022-06-29T15:51:18.004 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:51:18.005 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:18.005 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:18.005 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:18.005 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6825/13395", 2022-06-29T15:51:18.005 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6827/13395", 2022-06-29T15:51:18.006 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6831/13395", 2022-06-29T15:51:18.006 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6829/13395", 2022-06-29T15:51:18.006 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:51:18.006 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:51:18.007 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:51:18.007 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:18.007 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:18.007 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:18.007 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2022-06-29T15:51:18.008 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "f19c3ca6-df88-4bde-a150-b51a53f092e1", 2022-06-29T15:51:18.008 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:51:18.008 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:51:18.008 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:51:18.008 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:51:18.009 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:51:18.009 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:51:18.009 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:51:18.009 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 131, 2022-06-29T15:51:18.010 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:51:18.010 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:51:18.010 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:51:18.010 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:18.011 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:18.011 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:18.011 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6808", 2022-06-29T15:51:18.011 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:51:18.011 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:18.012 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:18.012 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:18.012 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6809", 2022-06-29T15:51:18.012 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:51:18.013 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:18.013 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:18.013 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:18.013 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:51:18.013 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:18.014 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:18.014 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:18.014 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6810", 2022-06-29T15:51:18.014 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:51:18.015 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:18.015 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:18.015 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:18.015 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6811", 2022-06-29T15:51:18.015 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:51:18.016 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:18.016 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:18.016 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:18.016 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:51:18.016 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:18.017 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:18.017 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:18.017 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6814", 2022-06-29T15:51:18.017 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:51:18.018 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:18.018 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:18.018 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:18.018 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6815", 2022-06-29T15:51:18.018 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:51:18.019 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:18.019 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:18.019 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:18.019 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:51:18.020 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:18.020 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:18.020 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:18.020 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6812", 2022-06-29T15:51:18.020 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:51:18.021 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:18.021 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:18.021 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:18.021 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6813", 2022-06-29T15:51:18.022 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:51:18.022 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:18.022 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:18.022 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:18.022 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6809/13394", 2022-06-29T15:51:18.023 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6811/13394", 2022-06-29T15:51:18.023 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6815/13394", 2022-06-29T15:51:18.023 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6813/13394", 2022-06-29T15:51:18.023 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:51:18.023 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:51:18.024 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:51:18.024 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:18.024 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:18.024 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:51:18.025 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_xinfo": [ 2022-06-29T15:51:18.025 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:18.025 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2022-06-29T15:51:18.025 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:51:18.025 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:51:18.026 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:51:18.026 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:51:18.026 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:51:18.026 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.720513+0000", 2022-06-29T15:51:18.027 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:51:18.027 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:18.027 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:18.027 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2022-06-29T15:51:18.027 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:51:18.028 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:51:18.028 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:51:18.028 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:51:18.028 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:51:18.029 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.772612+0000", 2022-06-29T15:51:18.029 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:51:18.029 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:18.029 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:18.029 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2022-06-29T15:51:18.030 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:51:18.030 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:51:18.030 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:51:18.030 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:51:18.031 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:51:18.031 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.450223+0000", 2022-06-29T15:51:18.031 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:51:18.031 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:18.031 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:18.032 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2022-06-29T15:51:18.032 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:51:18.032 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:51:18.032 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:51:18.032 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:51:18.033 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:51:18.033 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.477856+0000", 2022-06-29T15:51:18.033 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:51:18.033 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:18.034 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:18.034 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2022-06-29T15:51:18.034 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:51:18.034 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:51:18.034 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:51:18.035 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:51:18.035 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:51:18.035 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.539485+0000", 2022-06-29T15:51:18.035 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:51:18.035 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:18.036 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:18.036 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2022-06-29T15:51:18.036 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:51:18.036 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:51:18.037 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:51:18.037 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:51:18.037 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:51:18.037 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.487040+0000", 2022-06-29T15:51:18.037 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:51:18.038 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:18.038 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:18.038 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2022-06-29T15:51:18.038 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:51:18.039 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:51:18.039 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:51:18.039 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:51:18.039 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:51:18.039 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.748685+0000", 2022-06-29T15:51:18.040 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:51:18.040 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:18.040 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:18.040 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2022-06-29T15:51:18.041 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:51:18.041 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:51:18.041 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:51:18.041 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:51:18.041 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:51:18.042 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.459516+0000", 2022-06-29T15:51:18.042 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:51:18.042 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:18.042 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:51:18.042 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap": [], 2022-06-29T15:51:18.043 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap_items": [], 2022-06-29T15:51:18.043 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_temp": [], 2022-06-29T15:51:18.043 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_temp": [], 2022-06-29T15:51:18.043 INFO:teuthology.orchestra.run.smithi047.stdout: "blacklist": { 2022-06-29T15:51:18.044 INFO:teuthology.orchestra.run.smithi047.stdout: "172.21.15.64:6833/2987151459": "2022-06-30T15:50:56.446691+0000", 2022-06-29T15:51:18.044 INFO:teuthology.orchestra.run.smithi047.stdout: "172.21.15.64:6832/2987151459": "2022-06-30T15:50:56.446691+0000" 2022-06-29T15:51:18.044 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:18.044 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profiles": { 2022-06-29T15:51:18.044 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs_data": { 2022-06-29T15:51:18.045 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-device-class": "", 2022-06-29T15:51:18.045 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2022-06-29T15:51:18.045 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-root": "default", 2022-06-29T15:51:18.045 INFO:teuthology.orchestra.run.smithi047.stdout: "jerasure-per-chunk-alignment": "false", 2022-06-29T15:51:18.045 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2022-06-29T15:51:18.046 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "2", 2022-06-29T15:51:18.046 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2022-06-29T15:51:18.046 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2022-06-29T15:51:18.046 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van", 2022-06-29T15:51:18.047 INFO:teuthology.orchestra.run.smithi047.stdout: "w": "8" 2022-06-29T15:51:18.047 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:18.047 INFO:teuthology.orchestra.run.smithi047.stdout: "default": { 2022-06-29T15:51:18.047 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2022-06-29T15:51:18.047 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2022-06-29T15:51:18.048 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "1", 2022-06-29T15:51:18.048 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2022-06-29T15:51:18.048 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2022-06-29T15:51:18.048 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van" 2022-06-29T15:51:18.048 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:18.049 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:18.049 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps_queue": [], 2022-06-29T15:51:18.049 INFO:teuthology.orchestra.run.smithi047.stdout: "new_removed_snaps": [], 2022-06-29T15:51:18.049 INFO:teuthology.orchestra.run.smithi047.stdout: "new_purged_snaps": [], 2022-06-29T15:51:18.050 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_node_flags": {}, 2022-06-29T15:51:18.050 INFO:teuthology.orchestra.run.smithi047.stdout: "device_class_flags": {} 2022-06-29T15:51:18.050 INFO:teuthology.orchestra.run.smithi047.stdout:} 2022-06-29T15:51:18.052 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.846+0000 7f63effe3700 1 -- 172.21.15.47:0/445142114 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f63c80539e0 msgr2=0x7f63c8055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:18.052 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.846+0000 7f63effe3700 1 --2- 172.21.15.47:0/445142114 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f63c80539e0 0x7f63c8055ea0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f63e401c000 tx=0x7f63e4015000).stop 2022-06-29T15:51:18.053 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.846+0000 7f63effe3700 1 -- 172.21.15.47:0/445142114 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63e80af190 msgr2=0x7f63e813f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:18.053 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.846+0000 7f63effe3700 1 --2- 172.21.15.47:0/445142114 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63e80af190 0x7f63e813f150 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 rx=0x7f63e8143be0 tx=0x7f63e0007f40).stop 2022-06-29T15:51:18.053 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.846+0000 7f63effe3700 1 -- 172.21.15.47:0/445142114 shutdown_connections 2022-06-29T15:51:18.053 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.846+0000 7f63effe3700 1 --2- 172.21.15.47:0/445142114 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63e80af190 0x7f63e813f150 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:18.054 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.846+0000 7f63effe3700 1 --2- 172.21.15.47:0/445142114 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f63c80539e0 0x7f63c8055ea0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:18.054 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.846+0000 7f63effe3700 1 --2- 172.21.15.47:0/445142114 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63e80aca80 0x7f63e813b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:18.054 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.846+0000 7f63effe3700 1 --2- 172.21.15.47:0/445142114 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f63e805f760 0x7f63e8065610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:18.054 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.846+0000 7f63effe3700 1 -- 172.21.15.47:0/445142114 >> 172.21.15.47:0/445142114 conn(0x7f63e80626f0 msgr2=0x7f63e8138430 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:18.055 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.846+0000 7f63effe3700 1 -- 172.21.15.47:0/445142114 shutdown_connections 2022-06-29T15:51:18.055 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:17.846+0000 7f63effe3700 1 -- 172.21.15.47:0/445142114 wait complete. 2022-06-29T15:51:18.056 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:51:18.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.166+0000 7f3a18851700 1 Processor -- start 2022-06-29T15:51:18.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.166+0000 7f3a18851700 1 -- start start 2022-06-29T15:51:18.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.166+0000 7f3a18851700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3a1412a140 0x7f3a1414cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:18.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.166+0000 7f3a18851700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a140b3b70 con 0x7f3a140b4190 2022-06-29T15:51:18.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.166+0000 7f3a18851700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a140ade10 con 0x7f3a140b2e00 2022-06-29T15:51:18.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.166+0000 7f3a18851700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a140b48b0 con 0x7f3a1412a140 2022-06-29T15:51:18.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.166+0000 7f3a1259c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f3a140b4190 0x7f3a140b26f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38968/0 (socket says 172.21.15.47:38968) 2022-06-29T15:51:18.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.166+0000 7f3a1259c700 1 -- 172.21.15.47:0/492494208 learned_addr learned my addr 172.21.15.47:0/492494208 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:18.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.166+0000 7f3a12d9d700 1 --2- 172.21.15.47:0/492494208 >> v2:172.21.15.47:3300/0 conn(0x7f3a1412a140 0x7f3a1414cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:18.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.166+0000 7f3a1159a700 1 -- 172.21.15.47:0/492494208 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1110475586 0 0) 0x7f3a140b3b70 con 0x7f3a140b4190 2022-06-29T15:51:18.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.166+0000 7f3a1159a700 1 -- 172.21.15.47:0/492494208 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39f8003260 con 0x7f3a140b4190 2022-06-29T15:51:18.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.166+0000 7f3a12d9d700 1 -- 172.21.15.47:0/492494208 >> v1:172.21.15.64:6789/0 conn(0x7f3a140b2e00 legacy=0x7f3a14129a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:18.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.166+0000 7f3a12d9d700 1 -- 172.21.15.47:0/492494208 >> v1:172.21.15.47:6789/0 conn(0x7f3a140b4190 legacy=0x7f3a140b26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:18.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.166+0000 7f3a12d9d700 1 -- 172.21.15.47:0/492494208 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a140ae0b0 con 0x7f3a1412a140 2022-06-29T15:51:18.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.166+0000 7f3a12d9d700 1 --2- 172.21.15.47:0/492494208 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a1412a140 0x7f3a1414cad0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 rx=0x7f3a0c002f70 tx=0x7f3a0c006df0).ready entity=mon.0 client_cookie=1d8e87347a6352a5 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:18.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.166+0000 7f3a1159a700 1 -- 172.21.15.47:0/492494208 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a0c00d610 con 0x7f3a1412a140 2022-06-29T15:51:18.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.166+0000 7f3a1159a700 1 -- 172.21.15.47:0/492494208 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a0c000e90 con 0x7f3a1412a140 2022-06-29T15:51:18.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.166+0000 7f3a1159a700 1 -- 172.21.15.47:0/492494208 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a0c00d960 con 0x7f3a1412a140 2022-06-29T15:51:18.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.166+0000 7f3a18851700 1 -- 172.21.15.47:0/492494208 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a1412a140 msgr2=0x7f3a1414cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:18.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.166+0000 7f3a18851700 1 --2- 172.21.15.47:0/492494208 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a1412a140 0x7f3a1414cad0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 rx=0x7f3a0c002f70 tx=0x7f3a0c006df0).stop 2022-06-29T15:51:18.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.166+0000 7f3a18851700 1 -- 172.21.15.47:0/492494208 shutdown_connections 2022-06-29T15:51:18.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.166+0000 7f3a18851700 1 --2- 172.21.15.47:0/492494208 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a1412a140 0x7f3a1414cad0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:18.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.166+0000 7f3a18851700 1 -- 172.21.15.47:0/492494208 >> 172.21.15.47:0/492494208 conn(0x7f3a1405c510 msgr2=0x7f3a1405c920 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:18.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.166+0000 7f3a18851700 1 -- 172.21.15.47:0/492494208 shutdown_connections 2022-06-29T15:51:18.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.166+0000 7f3a18851700 1 -- 172.21.15.47:0/492494208 wait complete. 2022-06-29T15:51:18.182 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.170+0000 7f3a18851700 1 Processor -- start 2022-06-29T15:51:18.182 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.170+0000 7f3a18851700 1 -- start start 2022-06-29T15:51:18.182 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.170+0000 7f3a18851700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3a140b2e00 0x7f3a1413c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:18.182 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.170+0000 7f3a18851700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a140b4190 0x7f3a14156150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:18.183 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.170+0000 7f3a18851700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a1412a140 0x7f3a1413db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:18.183 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.170+0000 7f3a18851700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a140b6e70 con 0x7f3a140b4190 2022-06-29T15:51:18.183 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.170+0000 7f3a18851700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a140b6690 con 0x7f3a1412a140 2022-06-29T15:51:18.183 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.170+0000 7f3a18851700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a140b6930 con 0x7f3a140b2e00 2022-06-29T15:51:18.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.170+0000 7f3a11d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a140b4190 0x7f3a14156150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:18.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.170+0000 7f3a11d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a140b4190 0x7f3a14156150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53422/0 (socket says 172.21.15.47:53422) 2022-06-29T15:51:18.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.170+0000 7f3a11d9b700 1 -- 172.21.15.47:0/841034689 learned_addr learned my addr 172.21.15.47:0/841034689 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:18.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.170+0000 7f3a12d9d700 1 --2- 172.21.15.47:0/841034689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a1412a140 0x7f3a1413db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:18.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.170+0000 7f3a1259c700 1 --2- 172.21.15.47:0/841034689 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3a140b2e00 0x7f3a1413c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:18.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.170+0000 7f3a11d9b700 1 -- 172.21.15.47:0/841034689 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3a140b2e00 msgr2=0x7f3a1413c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:18.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.170+0000 7f3a11d9b700 1 --2- 172.21.15.47:0/841034689 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3a140b2e00 0x7f3a1413c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:18.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.170+0000 7f3a11d9b700 1 -- 172.21.15.47:0/841034689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a1412a140 msgr2=0x7f3a1413db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:18.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.170+0000 7f3a11d9b700 1 --2- 172.21.15.47:0/841034689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a1412a140 0x7f3a1413db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:18.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.170+0000 7f3a11d9b700 1 -- 172.21.15.47:0/841034689 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a0c00b180 con 0x7f3a140b4190 2022-06-29T15:51:18.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.170+0000 7f3a11d9b700 1 --2- 172.21.15.47:0/841034689 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a140b4190 0x7f3a14156150 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 rx=0x7f3a140657a0 tx=0x7f3a040014a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:18.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.170+0000 7f3a0b7fe700 1 -- 172.21.15.47:0/841034689 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a0400e4a0 con 0x7f3a140b4190 2022-06-29T15:51:18.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.170+0000 7f3a0b7fe700 1 -- 172.21.15.47:0/841034689 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a0400e600 con 0x7f3a140b4190 2022-06-29T15:51:18.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.170+0000 7f3a18851700 1 -- 172.21.15.47:0/841034689 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a140b6bd0 con 0x7f3a140b4190 2022-06-29T15:51:18.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.170+0000 7f3a0b7fe700 1 -- 172.21.15.47:0/841034689 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a0400e960 con 0x7f3a140b4190 2022-06-29T15:51:18.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.170+0000 7f3a18851700 1 -- 172.21.15.47:0/841034689 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a140b6350 con 0x7f3a140b4190 2022-06-29T15:51:18.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.174+0000 7f3a18851700 1 -- 172.21.15.47:0/841034689 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a140b7050 con 0x7f3a140b4190 2022-06-29T15:51:18.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.174+0000 7f3a0b7fe700 1 -- 172.21.15.47:0/841034689 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3a04019030 con 0x7f3a140b4190 2022-06-29T15:51:18.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.174+0000 7f3a0b7fe700 1 --2- 172.21.15.47:0/841034689 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f39ec053a10 0x7f39ec055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:18.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.174+0000 7f3a0b7fe700 1 -- 172.21.15.47:0/841034689 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f3a0406b5a0 con 0x7f3a140b4190 2022-06-29T15:51:18.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.174+0000 7f3a1259c700 1 --2- 172.21.15.47:0/841034689 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f39ec053a10 0x7f39ec055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:18.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.174+0000 7f3a1259c700 1 --2- 172.21.15.47:0/841034689 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f39ec053a10 0x7f39ec055ed0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7f39fc00b000 tx=0x7f39fc009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:18.191 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.182+0000 7f3a0b7fe700 1 -- 172.21.15.47:0/841034689 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3a04042540 con 0x7f3a140b4190 2022-06-29T15:51:18.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.346+0000 7f3a18851700 1 -- 172.21.15.47:0/841034689 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3a140573d0 con 0x7f3a140b4190 2022-06-29T15:51:18.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.350+0000 7f3a0b7fe700 1 -- 172.21.15.47:0/841034689 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+504 (secure 0 0 0) 0x7f3a04017020 con 0x7f3a140b4190 2022-06-29T15:51:18.356 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:18.356 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":64,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[]} 2022-06-29T15:51:18.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.350+0000 7f3a18851700 1 -- 172.21.15.47:0/841034689 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f39ec053a10 msgr2=0x7f39ec055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:18.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.350+0000 7f3a18851700 1 --2- 172.21.15.47:0/841034689 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f39ec053a10 0x7f39ec055ed0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7f39fc00b000 tx=0x7f39fc009040).stop 2022-06-29T15:51:18.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.350+0000 7f3a18851700 1 -- 172.21.15.47:0/841034689 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a140b4190 msgr2=0x7f3a14156150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:18.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.350+0000 7f3a18851700 1 --2- 172.21.15.47:0/841034689 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a140b4190 0x7f3a14156150 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 rx=0x7f3a140657a0 tx=0x7f3a040014a0).stop 2022-06-29T15:51:18.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.350+0000 7f3a18851700 1 -- 172.21.15.47:0/841034689 shutdown_connections 2022-06-29T15:51:18.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.350+0000 7f3a18851700 1 --2- 172.21.15.47:0/841034689 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f39ec053a10 0x7f39ec055ed0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:18.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.350+0000 7f3a18851700 1 --2- 172.21.15.47:0/841034689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a1412a140 0x7f3a1413db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:18.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.350+0000 7f3a18851700 1 --2- 172.21.15.47:0/841034689 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a140b4190 0x7f3a14156150 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:18.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.350+0000 7f3a18851700 1 --2- 172.21.15.47:0/841034689 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3a140b2e00 0x7f3a1413c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:18.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.350+0000 7f3a18851700 1 -- 172.21.15.47:0/841034689 >> 172.21.15.47:0/841034689 conn(0x7f3a1405c510 msgr2=0x7f3a1405c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:18.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.350+0000 7f3a18851700 1 -- 172.21.15.47:0/841034689 shutdown_connections 2022-06-29T15:51:18.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.350+0000 7f3a18851700 1 -- 172.21.15.47:0/841034689 wait complete. 2022-06-29T15:51:18.361 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 64 2022-06-29T15:51:18.369 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:51:18.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.462+0000 7f824a350700 1 Processor -- start 2022-06-29T15:51:18.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.462+0000 7f824a350700 1 -- start start 2022-06-29T15:51:18.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.462+0000 7f824a350700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f82440b5c60 0x7f8244144980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:18.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.462+0000 7f824a350700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f82440a72f0 0x7f82440b2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:18.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.462+0000 7f824a350700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82440a9120 con 0x7f8244060eb0 2022-06-29T15:51:18.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.462+0000 7f824a350700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82440b0b80 con 0x7f82440a72f0 2022-06-29T15:51:18.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.462+0000 7f824a350700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82440a89e0 con 0x7f82440b5c60 2022-06-29T15:51:18.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.462+0000 7f8249b4f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f82440a72f0 0x7f82440b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:18.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.462+0000 7f8249b4f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f82440a72f0 0x7f82440b2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53432/0 (socket says 172.21.15.47:53432) 2022-06-29T15:51:18.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.462+0000 7f8249b4f700 1 -- 172.21.15.47:0/819820935 learned_addr learned my addr 172.21.15.47:0/819820935 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:18.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.466+0000 7f824934e700 1 --2- 172.21.15.47:0/819820935 >> v2:172.21.15.64:3300/0 conn(0x7f82440b5c60 0x7f8244144980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:18.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.466+0000 7f8249b4f700 1 -- 172.21.15.47:0/819820935 >> v2:172.21.15.64:3300/0 conn(0x7f82440b5c60 msgr2=0x7f8244144980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:18.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.466+0000 7f8249b4f700 1 --2- 172.21.15.47:0/819820935 >> v2:172.21.15.64:3300/0 conn(0x7f82440b5c60 0x7f8244144980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:18.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.466+0000 7f8249b4f700 1 -- 172.21.15.47:0/819820935 >> v1:172.21.15.64:6789/0 conn(0x7f8244060eb0 legacy=0x7f82440b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:18.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.466+0000 7f8249b4f700 1 -- 172.21.15.47:0/819820935 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82440a8c80 con 0x7f82440a72f0 2022-06-29T15:51:18.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.466+0000 7f8249b4f700 1 --2- 172.21.15.47:0/819820935 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82440a72f0 0x7f82440b2540 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 rx=0x7f823c0032f0 tx=0x7f823c009290).ready entity=mon.0 client_cookie=9dc2f3d8dd24f518 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:18.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.466+0000 7f823bfff700 1 -- 172.21.15.47:0/819820935 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f823c00b870 con 0x7f82440a72f0 2022-06-29T15:51:18.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.466+0000 7f823bfff700 1 -- 172.21.15.47:0/819820935 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f823c00b9d0 con 0x7f82440a72f0 2022-06-29T15:51:18.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.466+0000 7f823bfff700 1 -- 172.21.15.47:0/819820935 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f823c00bb40 con 0x7f82440a72f0 2022-06-29T15:51:18.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.466+0000 7f824a350700 1 -- 172.21.15.47:0/819820935 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82440a72f0 msgr2=0x7f82440b2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:18.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.466+0000 7f824a350700 1 --2- 172.21.15.47:0/819820935 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82440a72f0 0x7f82440b2540 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 rx=0x7f823c0032f0 tx=0x7f823c009290).stop 2022-06-29T15:51:18.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.466+0000 7f824a350700 1 -- 172.21.15.47:0/819820935 shutdown_connections 2022-06-29T15:51:18.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.466+0000 7f824a350700 1 --2- 172.21.15.47:0/819820935 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82440a72f0 0x7f82440b2540 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:18.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.466+0000 7f824a350700 1 --2- 172.21.15.47:0/819820935 >> v2:172.21.15.64:3300/0 conn(0x7f82440b5c60 0x7f8244144980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:18.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.466+0000 7f824a350700 1 -- 172.21.15.47:0/819820935 >> 172.21.15.47:0/819820935 conn(0x7f8244062560 msgr2=0x7f8244062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:18.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.466+0000 7f824a350700 1 -- 172.21.15.47:0/819820935 shutdown_connections 2022-06-29T15:51:18.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.466+0000 7f824a350700 1 -- 172.21.15.47:0/819820935 wait complete. 2022-06-29T15:51:18.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.470+0000 7f824a350700 1 Processor -- start 2022-06-29T15:51:18.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.470+0000 7f824a350700 1 -- start start 2022-06-29T15:51:18.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.470+0000 7f824a350700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8244060eb0 0x7f8244143200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:18.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.470+0000 7f824a350700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f82440a72f0 0x7f824415a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:18.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.470+0000 7f824a350700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82440b5c60 0x7f824415a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:18.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.470+0000 7f824a350700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8244148d30 con 0x7f82440b5c60 2022-06-29T15:51:18.484 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.470+0000 7f824a350700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82441495c0 con 0x7f8244060eb0 2022-06-29T15:51:18.484 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.470+0000 7f824a350700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8244159e00 con 0x7f82440a72f0 2022-06-29T15:51:18.484 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.470+0000 7f8249b4f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82440b5c60 0x7f824415a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:18.484 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.470+0000 7f8249b4f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82440b5c60 0x7f824415a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53438/0 (socket says 172.21.15.47:53438) 2022-06-29T15:51:18.485 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.470+0000 7f824934e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8244060eb0 0x7f8244143200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:18.485 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.470+0000 7f8249b4f700 1 -- 172.21.15.47:0/3052110815 learned_addr learned my addr 172.21.15.47:0/3052110815 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:18.485 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.470+0000 7f8248b4d700 1 --2- 172.21.15.47:0/3052110815 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f82440a72f0 0x7f824415a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:18.485 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.470+0000 7f8249b4f700 1 -- 172.21.15.47:0/3052110815 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f82440a72f0 msgr2=0x7f824415a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:18.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.470+0000 7f8249b4f700 1 --2- 172.21.15.47:0/3052110815 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f82440a72f0 0x7f824415a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:18.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.470+0000 7f8249b4f700 1 -- 172.21.15.47:0/3052110815 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8244060eb0 msgr2=0x7f8244143200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:18.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.470+0000 7f8249b4f700 1 --2- 172.21.15.47:0/3052110815 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8244060eb0 0x7f8244143200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:18.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.470+0000 7f8249b4f700 1 -- 172.21.15.47:0/3052110815 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f823c009530 con 0x7f82440b5c60 2022-06-29T15:51:18.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.470+0000 7f8249b4f700 1 --2- 172.21.15.47:0/3052110815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82440b5c60 0x7f824415a750 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 rx=0x7f82440b4870 tx=0x7f823c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:18.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.474+0000 7f823a7fc700 1 -- 172.21.15.47:0/3052110815 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f823c00b870 con 0x7f82440b5c60 2022-06-29T15:51:18.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.474+0000 7f823a7fc700 1 -- 172.21.15.47:0/3052110815 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f823c021440 con 0x7f82440b5c60 2022-06-29T15:51:18.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.474+0000 7f824a350700 1 -- 172.21.15.47:0/3052110815 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8244149320 con 0x7f82440b5c60 2022-06-29T15:51:18.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.474+0000 7f823a7fc700 1 -- 172.21.15.47:0/3052110815 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f823c0217a0 con 0x7f82440b5c60 2022-06-29T15:51:18.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.474+0000 7f824a350700 1 -- 172.21.15.47:0/3052110815 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82441466c0 con 0x7f82440b5c60 2022-06-29T15:51:18.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.474+0000 7f824a350700 1 -- 172.21.15.47:0/3052110815 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8244159fc0 con 0x7f82440b5c60 2022-06-29T15:51:18.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.474+0000 7f823a7fc700 1 -- 172.21.15.47:0/3052110815 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f823c027020 con 0x7f82440b5c60 2022-06-29T15:51:18.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.474+0000 7f823a7fc700 1 --2- 172.21.15.47:0/3052110815 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f822c053aa0 0x7f822c055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:18.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.474+0000 7f823a7fc700 1 -- 172.21.15.47:0/3052110815 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f823c06d2b0 con 0x7f82440b5c60 2022-06-29T15:51:18.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.474+0000 7f824934e700 1 --2- 172.21.15.47:0/3052110815 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f822c053aa0 0x7f822c055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:18.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.474+0000 7f824934e700 1 --2- 172.21.15.47:0/3052110815 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f822c053aa0 0x7f822c055f60 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7f823400c000 tx=0x7f8234008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:18.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.482+0000 7f823a7fc700 1 -- 172.21.15.47:0/3052110815 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8244159fc0 con 0x7f82440b5c60 2022-06-29T15:51:18.653 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.646+0000 7f824a350700 1 -- 172.21.15.47:0/3052110815 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8244159fc0 con 0x7f82440b5c60 2022-06-29T15:51:18.654 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.646+0000 7f823a7fc700 1 -- 172.21.15.47:0/3052110815 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+504 (secure 0 0 0) 0x7f8244159fc0 con 0x7f82440b5c60 2022-06-29T15:51:18.655 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:18.655 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":64,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[]} 2022-06-29T15:51:18.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.650+0000 7f824a350700 1 -- 172.21.15.47:0/3052110815 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f822c053aa0 msgr2=0x7f822c055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:18.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.650+0000 7f824a350700 1 --2- 172.21.15.47:0/3052110815 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f822c053aa0 0x7f822c055f60 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7f823400c000 tx=0x7f8234008040).stop 2022-06-29T15:51:18.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.650+0000 7f824a350700 1 -- 172.21.15.47:0/3052110815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82440b5c60 msgr2=0x7f824415a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:18.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.650+0000 7f824a350700 1 --2- 172.21.15.47:0/3052110815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82440b5c60 0x7f824415a750 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 rx=0x7f82440b4870 tx=0x7f823c016000).stop 2022-06-29T15:51:18.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.650+0000 7f824a350700 1 -- 172.21.15.47:0/3052110815 shutdown_connections 2022-06-29T15:51:18.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.650+0000 7f824a350700 1 --2- 172.21.15.47:0/3052110815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82440b5c60 0x7f824415a750 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:18.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.650+0000 7f824a350700 1 --2- 172.21.15.47:0/3052110815 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f82440a72f0 0x7f824415a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:18.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.650+0000 7f824a350700 1 --2- 172.21.15.47:0/3052110815 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f822c053aa0 0x7f822c055f60 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:18.663 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.650+0000 7f824a350700 1 --2- 172.21.15.47:0/3052110815 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8244060eb0 0x7f8244143200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:18.663 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.650+0000 7f824a350700 1 -- 172.21.15.47:0/3052110815 >> 172.21.15.47:0/3052110815 conn(0x7f8244062560 msgr2=0x7f824414d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:18.664 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.650+0000 7f824a350700 1 -- 172.21.15.47:0/3052110815 shutdown_connections 2022-06-29T15:51:18.664 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.650+0000 7f824a350700 1 -- 172.21.15.47:0/3052110815 wait complete. 2022-06-29T15:51:18.665 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 64 2022-06-29T15:51:18.668 INFO:tasks.ceph.mds.a:Restarting daemon 2022-06-29T15:51:18.668 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-06-29T15:51:18.669 INFO:tasks.ceph.mds.c:Restarting daemon 2022-06-29T15:51:18.670 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2022-06-29T15:51:18.671 INFO:tasks.ceph.mds.b:Restarting daemon 2022-06-29T15:51:18.672 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-06-29T15:51:18.675 INFO:tasks.ceph.mds.a:Started 2022-06-29T15:51:18.675 DEBUG:teuthology.parallel:result is None 2022-06-29T15:51:18.678 INFO:tasks.ceph.mds.b:Started 2022-06-29T15:51:18.678 INFO:tasks.ceph.mds.c:Started 2022-06-29T15:51:18.679 DEBUG:teuthology.parallel:result is None 2022-06-29T15:51:18.680 DEBUG:teuthology.parallel:result is None 2022-06-29T15:51:18.681 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-06-29T15:51:18.741 INFO:tasks.ceph.mds.a.smithi047.stdout:starting mds.a at 2022-06-29T15:51:18.744 INFO:tasks.ceph.mds.c.smithi047.stdout:starting mds.c at 2022-06-29T15:51:18.758 INFO:tasks.ceph.mds.b.smithi064.stdout:starting mds.b at 2022-06-29T15:51:18.786 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.778+0000 7ff35ffff700 1 Processor -- start 2022-06-29T15:51:18.786 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.778+0000 7ff35ffff700 1 -- start start 2022-06-29T15:51:18.791 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.782+0000 7ff35ffff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff3600ad970 0x7ff3600b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:18.791 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.782+0000 7ff35ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff3600b0e50 0x7ff36013d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:18.791 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.782+0000 7ff35ffff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3601400e0 con 0x7ff3600b2ee0 2022-06-29T15:51:18.792 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.782+0000 7ff35ffff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff360140ef0 con 0x7ff3600b0e50 2022-06-29T15:51:18.792 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.782+0000 7ff35ffff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff36013fef0 con 0x7ff3600ad970 2022-06-29T15:51:18.792 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.782+0000 7ff35effd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ff3600b2ee0 0x7ff3601407e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:46584/0 (socket says 172.21.15.47:46584) 2022-06-29T15:51:18.792 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.782+0000 7ff35effd700 1 -- 172.21.15.47:0/997167857 learned_addr learned my addr 172.21.15.47:0/997167857 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:18.793 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.782+0000 7ff35f7fe700 1 --2- 172.21.15.47:0/997167857 >> v2:172.21.15.64:3300/0 conn(0x7ff3600b0e50 0x7ff36013d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:18.793 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.782+0000 7ff35e7fc700 1 --2- 172.21.15.47:0/997167857 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff3600ad970 0x7ff3600b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:18.794 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.782+0000 7ff35dffb700 1 -- 172.21.15.47:0/997167857 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1789549018 0 0) 0x7ff3601400e0 con 0x7ff3600b2ee0 2022-06-29T15:51:18.794 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.782+0000 7ff35dffb700 1 -- 172.21.15.47:0/997167857 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff340003260 con 0x7ff3600b2ee0 2022-06-29T15:51:18.795 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.782+0000 7ff35f7fe700 1 -- 172.21.15.47:0/997167857 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff3600ad970 msgr2=0x7ff3600b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:18.795 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.782+0000 7ff35f7fe700 1 --2- 172.21.15.47:0/997167857 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff3600ad970 0x7ff3600b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:18.795 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.782+0000 7ff35f7fe700 1 -- 172.21.15.47:0/997167857 >> v1:172.21.15.64:6789/0 conn(0x7ff3600b2ee0 legacy=0x7ff3601407e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:18.795 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.782+0000 7ff35f7fe700 1 -- 172.21.15.47:0/997167857 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff360143570 con 0x7ff3600b0e50 2022-06-29T15:51:18.796 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.782+0000 7ff35f7fe700 1 --2- 172.21.15.47:0/997167857 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3600b0e50 0x7ff36013d290 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7ff3580032f0 tx=0x7ff3580091d0).ready entity=mon.1 client_cookie=311b15ee0e0dd677 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:18.796 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.782+0000 7ff35dffb700 1 -- 172.21.15.47:0/997167857 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff35800b600 con 0x7ff3600b0e50 2022-06-29T15:51:18.796 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.782+0000 7ff35dffb700 1 -- 172.21.15.47:0/997167857 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff35800b760 con 0x7ff3600b0e50 2022-06-29T15:51:18.796 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.782+0000 7ff35dffb700 1 -- 172.21.15.47:0/997167857 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff35800b8d0 con 0x7ff3600b0e50 2022-06-29T15:51:18.797 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.782+0000 7ff35ffff700 1 -- 172.21.15.47:0/997167857 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3600b0e50 msgr2=0x7ff36013d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:18.797 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.782+0000 7ff35ffff700 1 --2- 172.21.15.47:0/997167857 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3600b0e50 0x7ff36013d290 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7ff3580032f0 tx=0x7ff3580091d0).stop 2022-06-29T15:51:18.797 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.782+0000 7ff35ffff700 1 -- 172.21.15.47:0/997167857 shutdown_connections 2022-06-29T15:51:18.797 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.782+0000 7ff35ffff700 1 --2- 172.21.15.47:0/997167857 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3600b0e50 0x7ff36013d290 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:18.798 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.782+0000 7ff35ffff700 1 --2- 172.21.15.47:0/997167857 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff3600ad970 0x7ff3600b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:18.798 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.782+0000 7ff35ffff700 1 -- 172.21.15.47:0/997167857 >> 172.21.15.47:0/997167857 conn(0x7ff360063670 msgr2=0x7ff360063a80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:18.798 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.782+0000 7ff35ffff700 1 -- 172.21.15.47:0/997167857 shutdown_connections 2022-06-29T15:51:18.798 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.782+0000 7ff35ffff700 1 -- 172.21.15.47:0/997167857 wait complete. 2022-06-29T15:51:18.801 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.786+0000 7ff35ffff700 1 Processor -- start 2022-06-29T15:51:18.802 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.786+0000 7ff35ffff700 1 -- start start 2022-06-29T15:51:18.802 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.786+0000 7ff35ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3600ad970 0x7ff360142e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:18.802 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.786+0000 7ff35ffff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff3600b0e50 0x7ff36013ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:18.802 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.786+0000 7ff35ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3600b2ee0 0x7ff36013f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:18.803 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.786+0000 7ff35ffff700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff360144b30 con 0x7ff3600ad970 2022-06-29T15:51:18.803 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.786+0000 7ff35ffff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff360141290 con 0x7ff3600b2ee0 2022-06-29T15:51:18.803 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.786+0000 7ff35ffff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3601414c0 con 0x7ff3600b0e50 2022-06-29T15:51:18.803 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.786+0000 7ff35effd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3600ad970 0x7ff360142e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:18.804 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.786+0000 7ff35f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3600b2ee0 0x7ff36013f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:18.804 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.786+0000 7ff35effd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3600ad970 0x7ff360142e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53472/0 (socket says 172.21.15.47:53472) 2022-06-29T15:51:18.804 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.786+0000 7ff35effd700 1 -- 172.21.15.47:0/103394104 learned_addr learned my addr 172.21.15.47:0/103394104 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:18.805 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.786+0000 7ff35e7fc700 1 --2- 172.21.15.47:0/103394104 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff3600b0e50 0x7ff36013ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:18.805 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.786+0000 7ff35effd700 1 -- 172.21.15.47:0/103394104 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff3600b0e50 msgr2=0x7ff36013ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:18.805 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.786+0000 7ff35effd700 1 --2- 172.21.15.47:0/103394104 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff3600b0e50 0x7ff36013ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:18.805 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.786+0000 7ff35effd700 1 -- 172.21.15.47:0/103394104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3600b2ee0 msgr2=0x7ff36013f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:18.806 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.786+0000 7ff35effd700 1 --2- 172.21.15.47:0/103394104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3600b2ee0 0x7ff36013f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:18.806 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.786+0000 7ff35effd700 1 -- 172.21.15.47:0/103394104 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff358009470 con 0x7ff3600ad970 2022-06-29T15:51:18.806 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.786+0000 7ff35effd700 1 --2- 172.21.15.47:0/103394104 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3600ad970 0x7ff360142e20 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 rx=0x7ff360142510 tx=0x7ff350007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:18.806 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.786+0000 7ff357fff700 1 -- 172.21.15.47:0/103394104 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3500070d0 con 0x7ff3600ad970 2022-06-29T15:51:18.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.786+0000 7ff357fff700 1 -- 172.21.15.47:0/103394104 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff350007230 con 0x7ff3600ad970 2022-06-29T15:51:18.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.786+0000 7ff35ffff700 1 -- 172.21.15.47:0/103394104 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff360141760 con 0x7ff3600ad970 2022-06-29T15:51:18.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.786+0000 7ff357fff700 1 -- 172.21.15.47:0/103394104 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff350005610 con 0x7ff3600ad970 2022-06-29T15:51:18.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.786+0000 7ff35ffff700 1 -- 172.21.15.47:0/103394104 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff360141ca0 con 0x7ff3600ad970 2022-06-29T15:51:18.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.790+0000 7ff35ffff700 1 -- 172.21.15.47:0/103394104 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3600573d0 con 0x7ff3600ad970 2022-06-29T15:51:18.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.790+0000 7ff357fff700 1 -- 172.21.15.47:0/103394104 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff35001a060 con 0x7ff3600ad970 2022-06-29T15:51:18.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.790+0000 7ff357fff700 1 --2- 172.21.15.47:0/103394104 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff3440539e0 0x7ff344055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:18.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.790+0000 7ff357fff700 1 -- 172.21.15.47:0/103394104 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5686+0+0 (secure 0 0 0) 0x7ff35006b420 con 0x7ff3600ad970 2022-06-29T15:51:18.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.790+0000 7ff35e7fc700 1 --2- 172.21.15.47:0/103394104 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff3440539e0 0x7ff344055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:18.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.790+0000 7ff35e7fc700 1 --2- 172.21.15.47:0/103394104 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff3440539e0 0x7ff344055ea0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7ff34800b000 tx=0x7ff348006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:18.809 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.794+0000 7ff357fff700 1 -- 172.21.15.47:0/103394104 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff3500424c0 con 0x7ff3600ad970 2022-06-29T15:51:18.941 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:18.934+0000 7ff35ffff700 1 -- 172.21.15.47:0/103394104 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7ff360069e20 con 0x7ff3600ad970 2022-06-29T15:51:19.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.766+0000 7ff357fff700 1 -- 172.21.15.47:0/103394104 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v136) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff350018070 con 0x7ff3600ad970 2022-06-29T15:51:19.776 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.770+0000 7ff35ffff700 1 -- 172.21.15.47:0/103394104 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff3440539e0 msgr2=0x7ff344055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:19.776 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.770+0000 7ff35ffff700 1 --2- 172.21.15.47:0/103394104 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff3440539e0 0x7ff344055ea0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7ff34800b000 tx=0x7ff348006df0).stop 2022-06-29T15:51:19.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.770+0000 7ff35ffff700 1 -- 172.21.15.47:0/103394104 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3600ad970 msgr2=0x7ff360142e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:19.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.770+0000 7ff35ffff700 1 --2- 172.21.15.47:0/103394104 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3600ad970 0x7ff360142e20 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 rx=0x7ff360142510 tx=0x7ff350007ad0).stop 2022-06-29T15:51:19.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.770+0000 7ff35ffff700 1 -- 172.21.15.47:0/103394104 shutdown_connections 2022-06-29T15:51:19.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.770+0000 7ff35ffff700 1 --2- 172.21.15.47:0/103394104 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff3440539e0 0x7ff344055ea0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:19.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.770+0000 7ff35ffff700 1 --2- 172.21.15.47:0/103394104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3600b2ee0 0x7ff36013f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:19.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.770+0000 7ff35ffff700 1 --2- 172.21.15.47:0/103394104 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff3600b0e50 0x7ff36013ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:19.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.770+0000 7ff35ffff700 1 --2- 172.21.15.47:0/103394104 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3600ad970 0x7ff360142e20 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:19.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.770+0000 7ff35ffff700 1 -- 172.21.15.47:0/103394104 >> 172.21.15.47:0/103394104 conn(0x7ff360063670 msgr2=0x7ff3600af150 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:19.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.770+0000 7ff35ffff700 1 -- 172.21.15.47:0/103394104 shutdown_connections 2022-06-29T15:51:19.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.770+0000 7ff35ffff700 1 -- 172.21.15.47:0/103394104 wait complete. 2022-06-29T15:51:19.779 INFO:teuthology.orchestra.run.smithi047.stderr: removed all blacklist entries 2022-06-29T15:51:19.790 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2022-06-29T15:51:19.791 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-06-29T15:51:19.904 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.898+0000 7ff68e299700 1 Processor -- start 2022-06-29T15:51:19.904 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.898+0000 7ff68e299700 1 -- start start 2022-06-29T15:51:19.906 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.898+0000 7ff68e299700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff6880ad730 0x7ff688147370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:19.906 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.898+0000 7ff68e299700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff6880adb10 0x7ff68814b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:19.907 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.898+0000 7ff68e299700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff688098250 con 0x7ff68814b530 2022-06-29T15:51:19.907 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.898+0000 7ff68e299700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff68814a940 con 0x7ff6880ad730 2022-06-29T15:51:19.907 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.898+0000 7ff68e299700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff688098480 con 0x7ff6880adb10 2022-06-29T15:51:19.907 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.898+0000 7ff68d297700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff6880ad730 0x7ff688147370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:19.908 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.898+0000 7ff68d297700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff6880ad730 0x7ff688147370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53486/0 (socket says 172.21.15.47:53486) 2022-06-29T15:51:19.908 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.898+0000 7ff68d297700 1 -- 172.21.15.47:0/2274594113 learned_addr learned my addr 172.21.15.47:0/2274594113 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:19.908 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.898+0000 7ff68ca96700 1 --2- 172.21.15.47:0/2274594113 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff6880adb10 0x7ff68814b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:19.908 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.898+0000 7ff68d297700 1 -- 172.21.15.47:0/2274594113 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff6880adb10 msgr2=0x7ff68814b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:19.909 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.898+0000 7ff68d297700 1 --2- 172.21.15.47:0/2274594113 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff6880adb10 0x7ff68814b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:19.909 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.898+0000 7ff68d297700 1 -- 172.21.15.47:0/2274594113 >> v1:172.21.15.64:6789/0 conn(0x7ff68814b530 legacy=0x7ff6881443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:19.909 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.898+0000 7ff68d297700 1 -- 172.21.15.47:0/2274594113 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff68814aba0 con 0x7ff6880ad730 2022-06-29T15:51:19.909 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.898+0000 7ff68d297700 1 --2- 172.21.15.47:0/2274594113 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6880ad730 0x7ff688147370 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 rx=0x7ff6880ad680 tx=0x7ff684006eb0).ready entity=mon.0 client_cookie=23c0850459b82720 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:19.910 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.898+0000 7ff67ffff700 1 -- 172.21.15.47:0/2274594113 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff68400d5c0 con 0x7ff6880ad730 2022-06-29T15:51:19.910 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.898+0000 7ff67ffff700 1 -- 172.21.15.47:0/2274594113 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff684000e90 con 0x7ff6880ad730 2022-06-29T15:51:19.910 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.898+0000 7ff67ffff700 1 -- 172.21.15.47:0/2274594113 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff68400d950 con 0x7ff6880ad730 2022-06-29T15:51:19.910 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.898+0000 7ff68e299700 1 -- 172.21.15.47:0/2274594113 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6880ad730 msgr2=0x7ff688147370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:19.911 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.898+0000 7ff68e299700 1 --2- 172.21.15.47:0/2274594113 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6880ad730 0x7ff688147370 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 rx=0x7ff6880ad680 tx=0x7ff684006eb0).stop 2022-06-29T15:51:19.911 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.898+0000 7ff68e299700 1 -- 172.21.15.47:0/2274594113 shutdown_connections 2022-06-29T15:51:19.911 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.898+0000 7ff68e299700 1 --2- 172.21.15.47:0/2274594113 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff6880adb10 0x7ff68814b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:19.911 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.898+0000 7ff68e299700 1 --2- 172.21.15.47:0/2274594113 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6880ad730 0x7ff688147370 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:19.911 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.898+0000 7ff68e299700 1 -- 172.21.15.47:0/2274594113 >> 172.21.15.47:0/2274594113 conn(0x7ff688062df0 msgr2=0x7ff6880631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:19.912 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.898+0000 7ff68e299700 1 -- 172.21.15.47:0/2274594113 shutdown_connections 2022-06-29T15:51:19.912 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.898+0000 7ff68e299700 1 -- 172.21.15.47:0/2274594113 wait complete. 2022-06-29T15:51:19.915 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.902+0000 7ff68e299700 1 Processor -- start 2022-06-29T15:51:19.915 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.902+0000 7ff68e299700 1 -- start start 2022-06-29T15:51:19.915 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.902+0000 7ff68e299700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6880ad730 0x7ff688147340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:19.915 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.902+0000 7ff68e299700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff6880adb10 0x7ff6881437b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:19.916 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.902+0000 7ff68e299700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff68814b530 0x7ff688143ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:19.916 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.902+0000 7ff68e299700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff688146cb0 con 0x7ff68814b530 2022-06-29T15:51:19.916 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.902+0000 7ff68e299700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff688145640 con 0x7ff6880ad730 2022-06-29T15:51:19.916 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.902+0000 7ff68e299700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff688146a10 con 0x7ff6880adb10 2022-06-29T15:51:19.917 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.902+0000 7ff68da98700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff68814b530 0x7ff688143ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:19.917 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.902+0000 7ff68da98700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff68814b530 0x7ff688143ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53494/0 (socket says 172.21.15.47:53494) 2022-06-29T15:51:19.917 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.902+0000 7ff68ca96700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff6880adb10 0x7ff6881437b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:19.917 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.902+0000 7ff68d297700 1 --2- 172.21.15.47:0/2897975180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6880ad730 0x7ff688147340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:19.917 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.902+0000 7ff68da98700 1 -- 172.21.15.47:0/2897975180 learned_addr learned my addr 172.21.15.47:0/2897975180 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:19.918 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.902+0000 7ff68da98700 1 -- 172.21.15.47:0/2897975180 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff6880adb10 msgr2=0x7ff6881437b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:19.918 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.902+0000 7ff68da98700 1 --2- 172.21.15.47:0/2897975180 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff6880adb10 0x7ff6881437b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:19.918 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.902+0000 7ff68da98700 1 -- 172.21.15.47:0/2897975180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6880ad730 msgr2=0x7ff688147340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:19.918 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.902+0000 7ff68da98700 1 --2- 172.21.15.47:0/2897975180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6880ad730 0x7ff688147340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:19.919 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.902+0000 7ff68da98700 1 -- 172.21.15.47:0/2897975180 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff68400b1e0 con 0x7ff68814b530 2022-06-29T15:51:19.919 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.902+0000 7ff68da98700 1 --2- 172.21.15.47:0/2897975180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff68814b530 0x7ff688143ca0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 rx=0x7ff68805c820 tx=0x7ff680007f70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:19.919 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.902+0000 7ff67e7fc700 1 -- 172.21.15.47:0/2897975180 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff680006aa0 con 0x7ff68814b530 2022-06-29T15:51:19.919 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.902+0000 7ff67e7fc700 1 -- 172.21.15.47:0/2897975180 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff680006c00 con 0x7ff68814b530 2022-06-29T15:51:19.920 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.902+0000 7ff67e7fc700 1 -- 172.21.15.47:0/2897975180 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff680007030 con 0x7ff68814b530 2022-06-29T15:51:19.920 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.902+0000 7ff68e299700 1 -- 172.21.15.47:0/2897975180 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6880b5240 con 0x7ff68814b530 2022-06-29T15:51:19.920 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.902+0000 7ff68e299700 1 -- 172.21.15.47:0/2897975180 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff688098b60 con 0x7ff68814b530 2022-06-29T15:51:19.920 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.906+0000 7ff68e299700 1 -- 172.21.15.47:0/2897975180 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6880573d0 con 0x7ff68814b530 2022-06-29T15:51:19.920 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.906+0000 7ff67e7fc700 1 -- 172.21.15.47:0/2897975180 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff680010030 con 0x7ff68814b530 2022-06-29T15:51:19.921 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.906+0000 7ff67e7fc700 1 --2- 172.21.15.47:0/2897975180 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff670053950 0x7ff670055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:19.921 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.906+0000 7ff67e7fc700 1 -- 172.21.15.47:0/2897975180 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5600+0+0 (secure 0 0 0) 0x7ff68006bc60 con 0x7ff68814b530 2022-06-29T15:51:19.921 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.906+0000 7ff68d297700 1 --2- 172.21.15.47:0/2897975180 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff670053950 0x7ff670055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:19.921 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.906+0000 7ff68d297700 1 --2- 172.21.15.47:0/2897975180 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff670053950 0x7ff670055e10 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0x7ff68401c000 tx=0x7ff684015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:19.922 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:19.914+0000 7ff67e7fc700 1 -- 172.21.15.47:0/2897975180 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff680042c50 con 0x7ff68814b530 2022-06-29T15:51:20.090 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.082+0000 7ff68e299700 1 -- 172.21.15.47:0/2897975180 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff6880ac930 con 0x7ff68814b530 2022-06-29T15:51:20.091 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.086+0000 7ff67e7fc700 1 -- 172.21.15.47:0/2897975180 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v32) v1 ==== 80+0+4255 (secure 0 0 0) 0x7ff680046270 con 0x7ff68814b530 2022-06-29T15:51:20.092 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:20.092 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2022-06-29T15:51:20.092 INFO:teuthology.orchestra.run.smithi047.stdout: "auth_dump": [ 2022-06-29T15:51:20.093 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:20.093 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.a", 2022-06-29T15:51:20.093 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxiQJFFMxAA9Tv7FJD9u/BKvbkuKld6Kg==", 2022-06-29T15:51:20.093 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:51:20.094 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:51:20.094 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:51:20.094 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:51:20.094 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:51:20.095 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:20.095 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:20.095 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:20.095 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.b", 2022-06-29T15:51:20.096 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC2crxibt6FARAAEf2bwXKFhXKtqr1nN0rdXQ==", 2022-06-29T15:51:20.096 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:51:20.096 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:51:20.096 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:51:20.097 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:51:20.097 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:51:20.097 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:20.097 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:20.097 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:20.098 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.c", 2022-06-29T15:51:20.098 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxi/4ZMORAANpLbgTm3ehqi80USc3swGg==", 2022-06-29T15:51:20.098 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:51:20.098 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:51:20.099 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:51:20.099 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:51:20.099 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:51:20.099 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:20.100 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:20.100 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:20.100 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.0", 2022-06-29T15:51:20.100 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC2crxiY0YsMxAAZivb+CxL/w7vZwQv37Y4oQ==", 2022-06-29T15:51:20.100 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:51:20.101 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:51:20.101 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:51:20.101 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:51:20.101 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:20.102 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:20.102 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:20.102 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.1", 2022-06-29T15:51:20.102 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC5crxitLkKFxAA8+++LUUKLiO9SAuAfYgT5g==", 2022-06-29T15:51:20.103 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:51:20.103 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:51:20.103 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:51:20.103 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:51:20.104 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:20.104 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:20.104 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:20.104 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.2", 2022-06-29T15:51:20.104 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC7crxi1oM9NxAA2FfLdNt32YPwlyNdGQpaQA==", 2022-06-29T15:51:20.105 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:51:20.105 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:51:20.105 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:51:20.105 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:51:20.106 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:20.106 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:20.106 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:20.106 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.3", 2022-06-29T15:51:20.106 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC+crxiTsBgGhAA+gLBBDaskPbmRNWzgKwcHA==", 2022-06-29T15:51:20.107 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:51:20.107 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:51:20.107 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:51:20.107 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:51:20.108 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:20.108 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:20.108 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:20.108 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.4", 2022-06-29T15:51:20.108 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDDcrxivJebIhAAfwWJa7tK4hamkk1/1CSSfw==", 2022-06-29T15:51:20.109 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:51:20.109 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:51:20.109 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:51:20.109 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:51:20.110 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:20.110 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:20.110 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:20.110 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.5", 2022-06-29T15:51:20.110 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDGcrxijuozBhAAt+Dx3wH3WKp3DrbCbhmh8Q==", 2022-06-29T15:51:20.111 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:51:20.111 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:51:20.111 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:51:20.111 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:51:20.112 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:20.112 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:20.112 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:20.112 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.6", 2022-06-29T15:51:20.113 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDIcrxi91kXJxAA9JX7n98P394bD41Q9LUdvQ==", 2022-06-29T15:51:20.113 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:51:20.113 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:51:20.113 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:51:20.113 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:51:20.114 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:20.114 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:20.114 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:20.114 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.7", 2022-06-29T15:51:20.115 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDLcrxiHj0ZDRAASy6fAix8TmEZbAGtuFCfFg==", 2022-06-29T15:51:20.115 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:51:20.115 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:51:20.115 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:51:20.115 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:51:20.116 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:20.116 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:20.116 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:20.116 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.0", 2022-06-29T15:51:20.117 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC2crxiVDlfBxAAezkUfg4e2g0Rfakaa/Ek1Q==", 2022-06-29T15:51:20.117 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:51:20.117 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:51:20.117 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow r", 2022-06-29T15:51:20.117 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-29T15:51:20.118 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:20.118 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:20.118 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:20.118 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.1", 2022-06-29T15:51:20.119 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC2crxisXJ1CxAAhl07oF7AW66NdzfvvZTJmA==", 2022-06-29T15:51:20.119 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:51:20.119 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:51:20.119 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow r", 2022-06-29T15:51:20.119 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-29T15:51:20.120 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:20.120 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:20.120 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:20.120 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.admin", 2022-06-29T15:51:20.121 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxiW7HSFRAAhYqoGU9GFqj2HfUVGValSA==", 2022-06-29T15:51:20.121 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:51:20.121 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2022-06-29T15:51:20.121 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:51:20.121 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:51:20.122 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:51:20.122 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:20.122 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:20.122 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:20.123 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.x", 2022-06-29T15:51:20.123 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxidmuIMBAAcpNlYVu5+icCHg+yIjcKXw==", 2022-06-29T15:51:20.123 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:51:20.123 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2022-06-29T15:51:20.123 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2022-06-29T15:51:20.124 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:51:20.124 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:20.124 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:20.124 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:20.125 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.y", 2022-06-29T15:51:20.125 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxisSMFLBAAv03r4OoPjOP9nI9G/LK0iw==", 2022-06-29T15:51:20.125 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:51:20.125 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2022-06-29T15:51:20.125 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2022-06-29T15:51:20.126 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:51:20.126 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:20.126 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:20.126 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:20.127 INFO:teuthology.orchestra.run.smithi047.stdout:} 2022-06-29T15:51:20.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.086+0000 7ff68e299700 1 -- 172.21.15.47:0/2897975180 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff670053950 msgr2=0x7ff670055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:20.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.090+0000 7ff68e299700 1 --2- 172.21.15.47:0/2897975180 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff670053950 0x7ff670055e10 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0x7ff68401c000 tx=0x7ff684015000).stop 2022-06-29T15:51:20.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.090+0000 7ff68e299700 1 -- 172.21.15.47:0/2897975180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff68814b530 msgr2=0x7ff688143ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:20.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.090+0000 7ff68e299700 1 --2- 172.21.15.47:0/2897975180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff68814b530 0x7ff688143ca0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 rx=0x7ff68805c820 tx=0x7ff680007f70).stop 2022-06-29T15:51:20.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.090+0000 7ff68e299700 1 -- 172.21.15.47:0/2897975180 shutdown_connections 2022-06-29T15:51:20.130 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.090+0000 7ff68e299700 1 --2- 172.21.15.47:0/2897975180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff68814b530 0x7ff688143ca0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:20.130 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.090+0000 7ff68e299700 1 --2- 172.21.15.47:0/2897975180 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff6880adb10 0x7ff6881437b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:20.130 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.090+0000 7ff68e299700 1 --2- 172.21.15.47:0/2897975180 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff670053950 0x7ff670055e10 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:20.130 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.090+0000 7ff68e299700 1 --2- 172.21.15.47:0/2897975180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6880ad730 0x7ff688147340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:20.131 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.090+0000 7ff68e299700 1 -- 172.21.15.47:0/2897975180 >> 172.21.15.47:0/2897975180 conn(0x7ff688062df0 msgr2=0x7ff6880b0460 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:20.131 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.090+0000 7ff68e299700 1 -- 172.21.15.47:0/2897975180 shutdown_connections 2022-06-29T15:51:20.131 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.090+0000 7ff68e299700 1 -- 172.21.15.47:0/2897975180 wait complete. 2022-06-29T15:51:20.132 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:51:20.241 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.234+0000 7f683f59e700 1 Processor -- start 2022-06-29T15:51:20.241 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.234+0000 7f683f59e700 1 -- start start 2022-06-29T15:51:20.244 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.234+0000 7f683f59e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f68400b5c60 0x7f6840144980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:20.244 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.234+0000 7f683f59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f68400a72f0 0x7f68400b2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:20.244 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.234+0000 7f683f59e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68400a9120 con 0x7f6840060eb0 2022-06-29T15:51:20.244 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.234+0000 7f683f59e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68400b0b80 con 0x7f68400b5c60 2022-06-29T15:51:20.245 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.234+0000 7f683f59e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68400a89e0 con 0x7f68400a72f0 2022-06-29T15:51:20.245 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.234+0000 7f683e59c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f68400b5c60 0x7f6840144980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:20.245 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.234+0000 7f683e59c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f68400b5c60 0x7f6840144980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53500/0 (socket says 172.21.15.47:53500) 2022-06-29T15:51:20.245 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.234+0000 7f683e59c700 1 -- 172.21.15.47:0/3864639367 learned_addr learned my addr 172.21.15.47:0/3864639367 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:20.246 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.234+0000 7f683ed9d700 1 --2- 172.21.15.47:0/3864639367 >> v2:172.21.15.64:3300/0 conn(0x7f68400a72f0 0x7f68400b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:20.246 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.234+0000 7f683e59c700 1 -- 172.21.15.47:0/3864639367 >> v2:172.21.15.64:3300/0 conn(0x7f68400a72f0 msgr2=0x7f68400b2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:20.246 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.234+0000 7f683e59c700 1 --2- 172.21.15.47:0/3864639367 >> v2:172.21.15.64:3300/0 conn(0x7f68400a72f0 0x7f68400b2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:20.246 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.234+0000 7f683e59c700 1 -- 172.21.15.47:0/3864639367 >> v1:172.21.15.64:6789/0 conn(0x7f6840060eb0 legacy=0x7f68400b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:20.247 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.234+0000 7f683e59c700 1 -- 172.21.15.47:0/3864639367 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68400a8c80 con 0x7f68400b5c60 2022-06-29T15:51:20.247 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.234+0000 7f683e59c700 1 --2- 172.21.15.47:0/3864639367 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68400b5c60 0x7f6840144980 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 rx=0x7f6834002f70 tx=0x7f6834006eb0).ready entity=mon.0 client_cookie=c3681bf9426bd9e7 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:20.247 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.234+0000 7f683d59a700 1 -- 172.21.15.47:0/3864639367 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f683400d610 con 0x7f68400b5c60 2022-06-29T15:51:20.247 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.234+0000 7f683d59a700 1 -- 172.21.15.47:0/3864639367 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6834000e90 con 0x7f68400b5c60 2022-06-29T15:51:20.248 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.234+0000 7f683d59a700 1 -- 172.21.15.47:0/3864639367 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f683400d9a0 con 0x7f68400b5c60 2022-06-29T15:51:20.248 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.238+0000 7f683f59e700 1 -- 172.21.15.47:0/3864639367 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68400b5c60 msgr2=0x7f6840144980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:20.248 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.238+0000 7f683f59e700 1 --2- 172.21.15.47:0/3864639367 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68400b5c60 0x7f6840144980 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 rx=0x7f6834002f70 tx=0x7f6834006eb0).stop 2022-06-29T15:51:20.248 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.238+0000 7f683f59e700 1 -- 172.21.15.47:0/3864639367 shutdown_connections 2022-06-29T15:51:20.249 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.238+0000 7f683f59e700 1 --2- 172.21.15.47:0/3864639367 >> v2:172.21.15.64:3300/0 conn(0x7f68400a72f0 0x7f68400b2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:20.249 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.238+0000 7f683f59e700 1 --2- 172.21.15.47:0/3864639367 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68400b5c60 0x7f6840144980 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:20.249 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.238+0000 7f683f59e700 1 -- 172.21.15.47:0/3864639367 >> 172.21.15.47:0/3864639367 conn(0x7f6840062560 msgr2=0x7f6840062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:20.250 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.238+0000 7f683f59e700 1 -- 172.21.15.47:0/3864639367 shutdown_connections 2022-06-29T15:51:20.250 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.238+0000 7f683f59e700 1 -- 172.21.15.47:0/3864639367 wait complete. 2022-06-29T15:51:20.253 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.238+0000 7f683f59e700 1 Processor -- start 2022-06-29T15:51:20.253 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.238+0000 7f683f59e700 1 -- start start 2022-06-29T15:51:20.253 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.238+0000 7f683f59e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6840060eb0 0x7f68400a9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:20.254 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.238+0000 7f683f59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68400a72f0 0x7f68400b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:20.254 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.238+0000 7f683f59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68401568e0 0x7f6840156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:20.254 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.238+0000 7f683f59e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68400af880 con 0x7f68400a72f0 2022-06-29T15:51:20.255 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.238+0000 7f683f59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68400ae000 con 0x7f68401568e0 2022-06-29T15:51:20.255 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.238+0000 7f683f59e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68400adac0 con 0x7f6840060eb0 2022-06-29T15:51:20.255 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.238+0000 7f683dd9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68400a72f0 0x7f68400b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:20.255 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.238+0000 7f683dd9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68400a72f0 0x7f68400b4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53506/0 (socket says 172.21.15.47:53506) 2022-06-29T15:51:20.256 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.238+0000 7f683dd9b700 1 -- 172.21.15.47:0/3823394934 learned_addr learned my addr 172.21.15.47:0/3823394934 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:20.256 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.238+0000 7f683e59c700 1 --2- 172.21.15.47:0/3823394934 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6840060eb0 0x7f68400a9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:20.256 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.238+0000 7f683ed9d700 1 --2- 172.21.15.47:0/3823394934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68401568e0 0x7f6840156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:20.256 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.242+0000 7f683dd9b700 1 -- 172.21.15.47:0/3823394934 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6840060eb0 msgr2=0x7f68400a9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:20.257 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.242+0000 7f683dd9b700 1 --2- 172.21.15.47:0/3823394934 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6840060eb0 0x7f68400a9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:20.257 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.242+0000 7f683dd9b700 1 -- 172.21.15.47:0/3823394934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68401568e0 msgr2=0x7f6840156d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:20.257 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.242+0000 7f683dd9b700 1 --2- 172.21.15.47:0/3823394934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68401568e0 0x7f6840156d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:20.257 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.242+0000 7f683dd9b700 1 -- 172.21.15.47:0/3823394934 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f683400b1e0 con 0x7f68400a72f0 2022-06-29T15:51:20.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.242+0000 7f683dd9b700 1 --2- 172.21.15.47:0/3823394934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68400a72f0 0x7f68400b4a30 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 rx=0x7f68400ae360 tx=0x7f6830007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:20.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.242+0000 7f682f7fe700 1 -- 172.21.15.47:0/3823394934 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6830012400 con 0x7f68400a72f0 2022-06-29T15:51:20.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.242+0000 7f682f7fe700 1 -- 172.21.15.47:0/3823394934 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6830012560 con 0x7f68400a72f0 2022-06-29T15:51:20.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.242+0000 7f683f59e700 1 -- 172.21.15.47:0/3823394934 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68400add60 con 0x7f68400a72f0 2022-06-29T15:51:20.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.242+0000 7f682f7fe700 1 -- 172.21.15.47:0/3823394934 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f68300128c0 con 0x7f68400a72f0 2022-06-29T15:51:20.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.242+0000 7f683f59e700 1 -- 172.21.15.47:0/3823394934 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68400aeac0 con 0x7f68400a72f0 2022-06-29T15:51:20.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.242+0000 7f683f59e700 1 -- 172.21.15.47:0/3823394934 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68400573d0 con 0x7f68400a72f0 2022-06-29T15:51:20.260 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.242+0000 7f682f7fe700 1 -- 172.21.15.47:0/3823394934 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6830019030 con 0x7f68400a72f0 2022-06-29T15:51:20.260 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.242+0000 7f682f7fe700 1 --2- 172.21.15.47:0/3823394934 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f681c053a10 0x7f681c055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:20.260 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.242+0000 7f683e59c700 1 --2- 172.21.15.47:0/3823394934 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f681c053a10 0x7f681c055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:20.261 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.242+0000 7f682f7fe700 1 -- 172.21.15.47:0/3823394934 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f683006b0f0 con 0x7f68400a72f0 2022-06-29T15:51:20.261 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.246+0000 7f683e59c700 1 --2- 172.21.15.47:0/3823394934 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f681c053a10 0x7f681c055ed0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7f683401a000 tx=0x7f6834014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:20.261 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.250+0000 7f682f7fe700 1 -- 172.21.15.47:0/3823394934 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f68300421f0 con 0x7f68400a72f0 2022-06-29T15:51:20.385 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.378+0000 7f683f59e700 1 -- 172.21.15.47:0/3823394934 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6840069e20 con 0x7f68400a72f0 2022-06-29T15:51:20.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.378+0000 7f682f7fe700 1 -- 172.21.15.47:0/3823394934 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) v1 ==== 74+0+13424 (secure 0 0 0) 0x7f6830045810 con 0x7f68400a72f0 2022-06-29T15:51:20.387 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:20.388 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":136,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:51:19.754858+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:51:20.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.382+0000 7f683f59e700 1 -- 172.21.15.47:0/3823394934 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f681c053a10 msgr2=0x7f681c055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:20.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.382+0000 7f683f59e700 1 --2- 172.21.15.47:0/3823394934 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f681c053a10 0x7f681c055ed0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7f683401a000 tx=0x7f6834014000).stop 2022-06-29T15:51:20.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.382+0000 7f683f59e700 1 -- 172.21.15.47:0/3823394934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68400a72f0 msgr2=0x7f68400b4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:20.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.382+0000 7f683f59e700 1 --2- 172.21.15.47:0/3823394934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68400a72f0 0x7f68400b4a30 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 rx=0x7f68400ae360 tx=0x7f6830007c50).stop 2022-06-29T15:51:20.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.382+0000 7f683f59e700 1 -- 172.21.15.47:0/3823394934 shutdown_connections 2022-06-29T15:51:20.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.382+0000 7f683f59e700 1 --2- 172.21.15.47:0/3823394934 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f681c053a10 0x7f681c055ed0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:20.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.382+0000 7f683f59e700 1 --2- 172.21.15.47:0/3823394934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68401568e0 0x7f6840156d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:20.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.382+0000 7f683f59e700 1 --2- 172.21.15.47:0/3823394934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68400a72f0 0x7f68400b4a30 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:20.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.382+0000 7f683f59e700 1 --2- 172.21.15.47:0/3823394934 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6840060eb0 0x7f68400a9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:20.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.382+0000 7f683f59e700 1 -- 172.21.15.47:0/3823394934 >> 172.21.15.47:0/3823394934 conn(0x7f6840062560 msgr2=0x7f68400afae0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:20.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.382+0000 7f683f59e700 1 -- 172.21.15.47:0/3823394934 shutdown_connections 2022-06-29T15:51:20.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.382+0000 7f683f59e700 1 -- 172.21.15.47:0/3823394934 wait complete. 2022-06-29T15:51:20.401 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-29T15:42:16.971431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-29T15:42:25.290884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-06-29T15:51:20.402 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-29T15:51:20.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.498+0000 7f1a3dce6700 1 Processor -- start 2022-06-29T15:51:20.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.498+0000 7f1a3dce6700 1 -- start start 2022-06-29T15:51:20.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.502+0000 7f1a3dce6700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1a38146720 0x7f1a38140780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:20.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.502+0000 7f1a3dce6700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1a380b4320 0x7f1a38143230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:20.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.502+0000 7f1a3dce6700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a38061600 con 0x7f1a38061220 2022-06-29T15:51:20.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.502+0000 7f1a3dce6700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a38146bc0 con 0x7f1a380b4320 2022-06-29T15:51:20.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.502+0000 7f1a3dce6700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a38146e60 con 0x7f1a38146720 2022-06-29T15:51:20.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.502+0000 7f1a37fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1a380b4320 0x7f1a38143230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:20.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.502+0000 7f1a377fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1a38061220 0x7f1a38140070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39066/0 (socket says 172.21.15.47:39066) 2022-06-29T15:51:20.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.502+0000 7f1a377fe700 1 -- 172.21.15.47:0/105072394 learned_addr learned my addr 172.21.15.47:0/105072394 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:20.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.502+0000 7f1a2ffff700 1 --2- 172.21.15.47:0/105072394 >> v2:172.21.15.64:3300/0 conn(0x7f1a38146720 0x7f1a38140780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:20.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.502+0000 7f1a37fff700 1 -- 172.21.15.47:0/105072394 >> v2:172.21.15.64:3300/0 conn(0x7f1a38146720 msgr2=0x7f1a38140780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:20.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.502+0000 7f1a37fff700 1 --2- 172.21.15.47:0/105072394 >> v2:172.21.15.64:3300/0 conn(0x7f1a38146720 0x7f1a38140780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:20.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.502+0000 7f1a37fff700 1 -- 172.21.15.47:0/105072394 >> v1:172.21.15.47:6789/0 conn(0x7f1a38061220 legacy=0x7f1a38140070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:20.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.502+0000 7f1a37fff700 1 -- 172.21.15.47:0/105072394 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a38147100 con 0x7f1a380b4320 2022-06-29T15:51:20.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.502+0000 7f1a37fff700 1 --2- 172.21.15.47:0/105072394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a380b4320 0x7f1a38143230 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 rx=0x7f1a300032f0 tx=0x7f1a300091d0).ready entity=mon.0 client_cookie=92505258e55dba5a server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:20.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.502+0000 7f1a36ffd700 1 -- 172.21.15.47:0/105072394 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a30009ea0 con 0x7f1a380b4320 2022-06-29T15:51:20.513 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.502+0000 7f1a36ffd700 1 -- 172.21.15.47:0/105072394 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a3000b870 con 0x7f1a380b4320 2022-06-29T15:51:20.513 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.502+0000 7f1a36ffd700 1 -- 172.21.15.47:0/105072394 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a3000b9e0 con 0x7f1a380b4320 2022-06-29T15:51:20.516 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.502+0000 7f1a3dce6700 1 -- 172.21.15.47:0/105072394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a380b4320 msgr2=0x7f1a38143230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:20.516 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.502+0000 7f1a3dce6700 1 --2- 172.21.15.47:0/105072394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a380b4320 0x7f1a38143230 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 rx=0x7f1a300032f0 tx=0x7f1a300091d0).stop 2022-06-29T15:51:20.517 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.502+0000 7f1a3dce6700 1 -- 172.21.15.47:0/105072394 shutdown_connections 2022-06-29T15:51:20.517 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.502+0000 7f1a3dce6700 1 --2- 172.21.15.47:0/105072394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a380b4320 0x7f1a38143230 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:20.517 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.502+0000 7f1a3dce6700 1 --2- 172.21.15.47:0/105072394 >> v2:172.21.15.64:3300/0 conn(0x7f1a38146720 0x7f1a38140780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:20.517 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.502+0000 7f1a3dce6700 1 -- 172.21.15.47:0/105072394 >> 172.21.15.47:0/105072394 conn(0x7f1a3805c6b0 msgr2=0x7f1a3805f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:20.518 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.502+0000 7f1a3dce6700 1 -- 172.21.15.47:0/105072394 shutdown_connections 2022-06-29T15:51:20.518 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.502+0000 7f1a3dce6700 1 -- 172.21.15.47:0/105072394 wait complete. 2022-06-29T15:51:20.518 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.506+0000 7f1a3dce6700 1 Processor -- start 2022-06-29T15:51:20.518 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.506+0000 7f1a3dce6700 1 -- start start 2022-06-29T15:51:20.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.506+0000 7f1a3dce6700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1a38061220 0x7f1a380b1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:20.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.506+0000 7f1a3dce6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a380b4320 0x7f1a380afac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:20.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.506+0000 7f1a3dce6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a38146720 0x7f1a381557b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:20.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.506+0000 7f1a3dce6700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a380b6fb0 con 0x7f1a38146720 2022-06-29T15:51:20.520 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.506+0000 7f1a3dce6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a380b1150 con 0x7f1a380b4320 2022-06-29T15:51:20.520 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.506+0000 7f1a3dce6700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a380b13f0 con 0x7f1a38061220 2022-06-29T15:51:20.520 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.506+0000 7f1a37fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a38146720 0x7f1a381557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:20.520 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.506+0000 7f1a37fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a38146720 0x7f1a381557b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53522/0 (socket says 172.21.15.47:53522) 2022-06-29T15:51:20.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.506+0000 7f1a37fff700 1 -- 172.21.15.47:0/1263883342 learned_addr learned my addr 172.21.15.47:0/1263883342 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:20.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.506+0000 7f1a377fe700 1 --2- 172.21.15.47:0/1263883342 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1a38061220 0x7f1a380b1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:20.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.506+0000 7f1a2ffff700 1 --2- 172.21.15.47:0/1263883342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a380b4320 0x7f1a380afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:20.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.506+0000 7f1a37fff700 1 -- 172.21.15.47:0/1263883342 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1a38061220 msgr2=0x7f1a380b1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:20.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.506+0000 7f1a37fff700 1 --2- 172.21.15.47:0/1263883342 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1a38061220 0x7f1a380b1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:20.522 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.506+0000 7f1a37fff700 1 -- 172.21.15.47:0/1263883342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a380b4320 msgr2=0x7f1a380afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:20.522 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.506+0000 7f1a37fff700 1 --2- 172.21.15.47:0/1263883342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a380b4320 0x7f1a380afac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:20.522 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.506+0000 7f1a37fff700 1 -- 172.21.15.47:0/1263883342 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a30009470 con 0x7f1a38146720 2022-06-29T15:51:20.522 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.506+0000 7f1a37fff700 1 --2- 172.21.15.47:0/1263883342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a38146720 0x7f1a381557b0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 rx=0x7f1a300032f0 tx=0x7f1a30016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:20.523 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.506+0000 7f1a357fa700 1 -- 172.21.15.47:0/1263883342 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a3000b9e0 con 0x7f1a38146720 2022-06-29T15:51:20.523 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.506+0000 7f1a357fa700 1 -- 172.21.15.47:0/1263883342 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a3001f5e0 con 0x7f1a38146720 2022-06-29T15:51:20.523 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.506+0000 7f1a3dce6700 1 -- 172.21.15.47:0/1263883342 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a380b1690 con 0x7f1a38146720 2022-06-29T15:51:20.523 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.506+0000 7f1a357fa700 1 -- 172.21.15.47:0/1263883342 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a30009ea0 con 0x7f1a38146720 2022-06-29T15:51:20.524 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.506+0000 7f1a3dce6700 1 -- 172.21.15.47:0/1263883342 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a380affb0 con 0x7f1a38146720 2022-06-29T15:51:20.524 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.510+0000 7f1a3dce6700 1 -- 172.21.15.47:0/1263883342 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a380b06a0 con 0x7f1a38146720 2022-06-29T15:51:20.524 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.510+0000 7f1a357fa700 1 -- 172.21.15.47:0/1263883342 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1a30026030 con 0x7f1a38146720 2022-06-29T15:51:20.525 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.510+0000 7f1a357fa700 1 --2- 172.21.15.47:0/1263883342 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1a1c053a70 0x7f1a1c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:20.525 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.510+0000 7f1a357fa700 1 -- 172.21.15.47:0/1263883342 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f1a3006c8f0 con 0x7f1a38146720 2022-06-29T15:51:20.525 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.510+0000 7f1a377fe700 1 --2- 172.21.15.47:0/1263883342 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1a1c053a70 0x7f1a1c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:20.526 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.510+0000 7f1a377fe700 1 --2- 172.21.15.47:0/1263883342 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1a1c053a70 0x7f1a1c055f30 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7f1a2800b000 tx=0x7f1a28009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:20.526 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.518+0000 7f1a357fa700 1 -- 172.21.15.47:0/1263883342 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1a300439f0 con 0x7f1a38146720 2022-06-29T15:51:20.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.642+0000 7f1a3dce6700 1 -- 172.21.15.47:0/1263883342 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f1a380b0b40 con 0x7f1a38146720 2022-06-29T15:51:20.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.642+0000 7f1a357fa700 1 -- 172.21.15.47:0/1263883342 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v136) v1 ==== 110+0+10 (secure 0 0 0) 0x7f1a30024070 con 0x7f1a38146720 2022-06-29T15:51:20.651 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2022-06-29T15:51:20.652 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.646+0000 7f1a3dce6700 1 -- 172.21.15.47:0/1263883342 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1a1c053a70 msgr2=0x7f1a1c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:20.653 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.646+0000 7f1a3dce6700 1 --2- 172.21.15.47:0/1263883342 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1a1c053a70 0x7f1a1c055f30 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7f1a2800b000 tx=0x7f1a28009040).stop 2022-06-29T15:51:20.653 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.646+0000 7f1a3dce6700 1 -- 172.21.15.47:0/1263883342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a38146720 msgr2=0x7f1a381557b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:20.653 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.646+0000 7f1a3dce6700 1 --2- 172.21.15.47:0/1263883342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a38146720 0x7f1a381557b0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 rx=0x7f1a300032f0 tx=0x7f1a30016000).stop 2022-06-29T15:51:20.653 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.646+0000 7f1a3dce6700 1 -- 172.21.15.47:0/1263883342 shutdown_connections 2022-06-29T15:51:20.654 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.646+0000 7f1a3dce6700 1 --2- 172.21.15.47:0/1263883342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a38146720 0x7f1a381557b0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:20.654 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.646+0000 7f1a3dce6700 1 --2- 172.21.15.47:0/1263883342 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1a1c053a70 0x7f1a1c055f30 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:20.654 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.646+0000 7f1a3dce6700 1 --2- 172.21.15.47:0/1263883342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a380b4320 0x7f1a380afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:20.654 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.646+0000 7f1a3dce6700 1 --2- 172.21.15.47:0/1263883342 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1a38061220 0x7f1a380b1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:20.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.646+0000 7f1a3dce6700 1 -- 172.21.15.47:0/1263883342 >> 172.21.15.47:0/1263883342 conn(0x7f1a3805c6b0 msgr2=0x7f1a3805f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:20.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.646+0000 7f1a3dce6700 1 -- 172.21.15.47:0/1263883342 shutdown_connections 2022-06-29T15:51:20.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.646+0000 7f1a3dce6700 1 -- 172.21.15.47:0/1263883342 wait complete. 2022-06-29T15:51:20.663 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-29T15:51:20.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.766+0000 7fe147a9b700 1 Processor -- start 2022-06-29T15:51:20.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.766+0000 7fe147a9b700 1 -- start start 2022-06-29T15:51:20.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.770+0000 7fe147a9b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe140060ee0 0x7fe1400612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:20.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.770+0000 7fe147a9b700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe1401471a0 0x7fe140066f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:20.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.770+0000 7fe147a9b700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe140067610 con 0x7fe1400b4300 2022-06-29T15:51:20.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.770+0000 7fe147a9b700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1401467c0 con 0x7fe140060ee0 2022-06-29T15:51:20.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.770+0000 7fe147a9b700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe140146a60 con 0x7fe1401471a0 2022-06-29T15:51:20.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.770+0000 7fe145036700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe1400b4300 0x7fe1401430e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39084/0 (socket says 172.21.15.47:39084) 2022-06-29T15:51:20.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.770+0000 7fe145036700 1 -- 172.21.15.47:0/2156769099 learned_addr learned my addr 172.21.15.47:0/2156769099 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:20.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.770+0000 7fe146038700 1 --2- 172.21.15.47:0/2156769099 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe1401471a0 0x7fe140066f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:20.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.770+0000 7fe145837700 1 --2- 172.21.15.47:0/2156769099 >> v2:172.21.15.64:3300/0 conn(0x7fe140060ee0 0x7fe1400612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:20.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.770+0000 7fe144835700 1 -- 172.21.15.47:0/2156769099 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1003691903 0 0) 0x7fe140067610 con 0x7fe1400b4300 2022-06-29T15:51:20.781 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.770+0000 7fe144835700 1 -- 172.21.15.47:0/2156769099 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe128003260 con 0x7fe1400b4300 2022-06-29T15:51:20.781 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.770+0000 7fe146038700 1 -- 172.21.15.47:0/2156769099 >> v2:172.21.15.64:3300/0 conn(0x7fe140060ee0 msgr2=0x7fe1400612c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:20.781 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.770+0000 7fe146038700 1 --2- 172.21.15.47:0/2156769099 >> v2:172.21.15.64:3300/0 conn(0x7fe140060ee0 0x7fe1400612c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:20.782 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.770+0000 7fe144835700 1 -- 172.21.15.47:0/2156769099 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2867647965 0 0) 0x7fe128003260 con 0x7fe1400b4300 2022-06-29T15:51:20.782 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.770+0000 7fe146038700 1 -- 172.21.15.47:0/2156769099 >> v1:172.21.15.47:6789/0 conn(0x7fe1400b4300 legacy=0x7fe1401430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:20.782 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.770+0000 7fe146038700 1 -- 172.21.15.47:0/2156769099 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe140146d00 con 0x7fe1401471a0 2022-06-29T15:51:20.782 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.770+0000 7fe146038700 1 --2- 172.21.15.47:0/2156769099 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe1401471a0 0x7fe140066f70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7fe1400ae790 tx=0x7fe1380091d0).ready entity=mon.2 client_cookie=c35ca4ef613152f7 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:20.783 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.770+0000 7fe144835700 1 -- 172.21.15.47:0/2156769099 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe13c001660 con 0x7fe1401471a0 2022-06-29T15:51:20.783 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.770+0000 7fe144835700 1 -- 172.21.15.47:0/2156769099 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe13800f510 con 0x7fe1401471a0 2022-06-29T15:51:20.783 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.770+0000 7fe144835700 1 -- 172.21.15.47:0/2156769099 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe13800f680 con 0x7fe1401471a0 2022-06-29T15:51:20.787 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.770+0000 7fe147a9b700 1 -- 172.21.15.47:0/2156769099 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe1401471a0 msgr2=0x7fe140066f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:20.787 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.770+0000 7fe147a9b700 1 --2- 172.21.15.47:0/2156769099 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe1401471a0 0x7fe140066f70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7fe1400ae790 tx=0x7fe1380091d0).stop 2022-06-29T15:51:20.788 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.770+0000 7fe147a9b700 1 -- 172.21.15.47:0/2156769099 shutdown_connections 2022-06-29T15:51:20.788 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.770+0000 7fe147a9b700 1 --2- 172.21.15.47:0/2156769099 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe1401471a0 0x7fe140066f70 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:20.788 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.770+0000 7fe147a9b700 1 --2- 172.21.15.47:0/2156769099 >> v2:172.21.15.64:3300/0 conn(0x7fe140060ee0 0x7fe1400612c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:20.788 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.770+0000 7fe147a9b700 1 -- 172.21.15.47:0/2156769099 >> 172.21.15.47:0/2156769099 conn(0x7fe14005c510 msgr2=0x7fe14005f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:20.789 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.774+0000 7fe147a9b700 1 -- 172.21.15.47:0/2156769099 shutdown_connections 2022-06-29T15:51:20.789 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.774+0000 7fe147a9b700 1 -- 172.21.15.47:0/2156769099 wait complete. 2022-06-29T15:51:20.789 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.774+0000 7fe147a9b700 1 Processor -- start 2022-06-29T15:51:20.789 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.774+0000 7fe147a9b700 1 -- start start 2022-06-29T15:51:20.790 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.774+0000 7fe147a9b700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe140060ee0 0x7fe140142540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:20.790 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.774+0000 7fe147a9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1400b4300 0x7fe1400b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:20.790 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.774+0000 7fe147a9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1401471a0 0x7fe140152320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:20.790 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.774+0000 7fe147a9b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe140156250 con 0x7fe1401471a0 2022-06-29T15:51:20.791 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.774+0000 7fe147a9b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe14013f9d0 con 0x7fe1400b4300 2022-06-29T15:51:20.791 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.774+0000 7fe147a9b700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1401564b0 con 0x7fe140060ee0 2022-06-29T15:51:20.791 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.774+0000 7fe146038700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1401471a0 0x7fe140152320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:20.792 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.774+0000 7fe146038700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1401471a0 0x7fe140152320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53536/0 (socket says 172.21.15.47:53536) 2022-06-29T15:51:20.792 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.774+0000 7fe146038700 1 -- 172.21.15.47:0/1323079027 learned_addr learned my addr 172.21.15.47:0/1323079027 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:20.792 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.774+0000 7fe145837700 1 --2- 172.21.15.47:0/1323079027 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe140060ee0 0x7fe140142540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:20.792 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.774+0000 7fe145036700 1 --2- 172.21.15.47:0/1323079027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1400b4300 0x7fe1400b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:20.792 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.774+0000 7fe146038700 1 -- 172.21.15.47:0/1323079027 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe140060ee0 msgr2=0x7fe140142540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:20.793 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.774+0000 7fe146038700 1 --2- 172.21.15.47:0/1323079027 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe140060ee0 0x7fe140142540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:20.793 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.774+0000 7fe146038700 1 -- 172.21.15.47:0/1323079027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1400b4300 msgr2=0x7fe1400b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:20.793 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.774+0000 7fe146038700 1 --2- 172.21.15.47:0/1323079027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1400b4300 0x7fe1400b37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:20.793 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.774+0000 7fe146038700 1 -- 172.21.15.47:0/1323079027 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe138009470 con 0x7fe1401471a0 2022-06-29T15:51:20.794 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.774+0000 7fe145837700 1 --2- 172.21.15.47:0/1323079027 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe140060ee0 0x7fe140142540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:51:20.794 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.778+0000 7fe146038700 1 --2- 172.21.15.47:0/1323079027 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1401471a0 0x7fe140152320 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 rx=0x7fe1400ae790 tx=0x7fe138016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:20.794 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.778+0000 7fe12effd700 1 -- 172.21.15.47:0/1323079027 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe1380032d0 con 0x7fe1401471a0 2022-06-29T15:51:20.794 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.778+0000 7fe147a9b700 1 -- 172.21.15.47:0/1323079027 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe140156750 con 0x7fe1401471a0 2022-06-29T15:51:20.795 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.778+0000 7fe12effd700 1 -- 172.21.15.47:0/1323079027 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe138011c30 con 0x7fe1401471a0 2022-06-29T15:51:20.795 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.778+0000 7fe147a9b700 1 -- 172.21.15.47:0/1323079027 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe14013f450 con 0x7fe1401471a0 2022-06-29T15:51:20.795 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.778+0000 7fe12effd700 1 -- 172.21.15.47:0/1323079027 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe13802a440 con 0x7fe1401471a0 2022-06-29T15:51:20.795 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.778+0000 7fe147a9b700 1 -- 172.21.15.47:0/1323079027 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1400573d0 con 0x7fe1401471a0 2022-06-29T15:51:20.796 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.778+0000 7fe12effd700 1 -- 172.21.15.47:0/1323079027 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe138027030 con 0x7fe1401471a0 2022-06-29T15:51:20.796 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.778+0000 7fe12effd700 1 --2- 172.21.15.47:0/1323079027 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe120053b00 0x7fe120055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:20.796 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.778+0000 7fe12effd700 1 -- 172.21.15.47:0/1323079027 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fe13806cd80 con 0x7fe1401471a0 2022-06-29T15:51:20.797 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.778+0000 7fe145837700 1 --2- 172.21.15.47:0/1323079027 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe120053b00 0x7fe120055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:20.797 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.782+0000 7fe145837700 1 --2- 172.21.15.47:0/1323079027 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe120053b00 0x7fe120055fc0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7fe13000b000 tx=0x7fe130008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:20.798 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.786+0000 7fe12effd700 1 -- 172.21.15.47:0/1323079027 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe138043d80 con 0x7fe1401471a0 2022-06-29T15:51:20.926 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.918+0000 7fe147a9b700 1 -- 172.21.15.47:0/1323079027 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe1400b5e50 con 0x7fe1401471a0 2022-06-29T15:51:20.926 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.918+0000 7fe12effd700 1 -- 172.21.15.47:0/1323079027 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v136) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe13806e020 con 0x7fe1401471a0 2022-06-29T15:51:20.927 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:51:20.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.922+0000 7fe147a9b700 1 -- 172.21.15.47:0/1323079027 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe120053b00 msgr2=0x7fe120055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:20.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.922+0000 7fe147a9b700 1 --2- 172.21.15.47:0/1323079027 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe120053b00 0x7fe120055fc0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7fe13000b000 tx=0x7fe130008040).stop 2022-06-29T15:51:20.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.922+0000 7fe147a9b700 1 -- 172.21.15.47:0/1323079027 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1401471a0 msgr2=0x7fe140152320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:20.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.922+0000 7fe147a9b700 1 --2- 172.21.15.47:0/1323079027 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1401471a0 0x7fe140152320 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 rx=0x7fe1400ae790 tx=0x7fe138016000).stop 2022-06-29T15:51:20.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.922+0000 7fe147a9b700 1 -- 172.21.15.47:0/1323079027 shutdown_connections 2022-06-29T15:51:20.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.922+0000 7fe147a9b700 1 --2- 172.21.15.47:0/1323079027 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1401471a0 0x7fe140152320 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:20.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.922+0000 7fe147a9b700 1 --2- 172.21.15.47:0/1323079027 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe120053b00 0x7fe120055fc0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:20.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.922+0000 7fe147a9b700 1 --2- 172.21.15.47:0/1323079027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1400b4300 0x7fe1400b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:20.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.922+0000 7fe147a9b700 1 --2- 172.21.15.47:0/1323079027 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe140060ee0 0x7fe140142540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:20.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.922+0000 7fe147a9b700 1 -- 172.21.15.47:0/1323079027 >> 172.21.15.47:0/1323079027 conn(0x7fe14005c510 msgr2=0x7fe1401444e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:20.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.922+0000 7fe147a9b700 1 -- 172.21.15.47:0/1323079027 shutdown_connections 2022-06-29T15:51:20.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:20.922+0000 7fe147a9b700 1 -- 172.21.15.47:0/1323079027 wait complete. 2022-06-29T15:51:20.945 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-06-29T15:51:21.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.038+0000 7f484cf6b700 1 Processor -- start 2022-06-29T15:51:21.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.038+0000 7f484cf6b700 1 -- start start 2022-06-29T15:51:21.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.038+0000 7f484cf6b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f48480b5ac0 0x7f48481401c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:21.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.038+0000 7f484cf6b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f48480b02d0 0x7f48481406b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:21.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.038+0000 7f484cf6b700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f48480afdc0 0x7f48480ae0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:21.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.038+0000 7f484cf6b700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48480a9090 con 0x7f48480b02d0 2022-06-29T15:51:21.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.038+0000 7f484cf6b700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4848143d20 con 0x7f48480b5ac0 2022-06-29T15:51:21.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.038+0000 7f484cf6b700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4848140ba0 con 0x7f48480afdc0 2022-06-29T15:51:21.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.038+0000 7f4846ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f48480b02d0 0x7f48481406b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:21.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.038+0000 7f4846ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f48480b02d0 0x7f48481406b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53540/0 (socket says 172.21.15.47:53540) 2022-06-29T15:51:21.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.038+0000 7f4846ffd700 1 -- 172.21.15.47:0/1693853125 learned_addr learned my addr 172.21.15.47:0/1693853125 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:21.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.038+0000 7f4847fff700 1 --2- 172.21.15.47:0/1693853125 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f48480afdc0 0x7f48480ae0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:21.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.038+0000 7f48477fe700 1 --2- 172.21.15.47:0/1693853125 >> v2:172.21.15.64:3300/0 conn(0x7f48480b5ac0 0x7f48481401c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:21.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.038+0000 7f4846ffd700 1 -- 172.21.15.47:0/1693853125 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f48480afdc0 msgr2=0x7f48480ae0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:21.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.038+0000 7f4846ffd700 1 --2- 172.21.15.47:0/1693853125 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f48480afdc0 0x7f48480ae0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:21.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.038+0000 7f4846ffd700 1 -- 172.21.15.47:0/1693853125 >> v2:172.21.15.64:3300/0 conn(0x7f48480b5ac0 msgr2=0x7f48481401c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:21.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.038+0000 7f4846ffd700 1 --2- 172.21.15.47:0/1693853125 >> v2:172.21.15.64:3300/0 conn(0x7f48480b5ac0 0x7f48481401c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:21.052 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.038+0000 7f4846ffd700 1 -- 172.21.15.47:0/1693853125 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48480a8df0 con 0x7f48480b02d0 2022-06-29T15:51:21.052 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.038+0000 7f48477fe700 1 --2- 172.21.15.47:0/1693853125 >> v2:172.21.15.64:3300/0 conn(0x7f48480b5ac0 0x7f48481401c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:51:21.052 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.038+0000 7f4846ffd700 1 --2- 172.21.15.47:0/1693853125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48480b02d0 0x7f48481406b0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 rx=0x7f48380032f0 tx=0x7f4838009290).ready entity=mon.0 client_cookie=204a8caf7b8adc1 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:21.052 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.038+0000 7f48467fc700 1 -- 172.21.15.47:0/1693853125 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f483800b870 con 0x7f48480b02d0 2022-06-29T15:51:21.052 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.038+0000 7f48467fc700 1 -- 172.21.15.47:0/1693853125 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f483800b9d0 con 0x7f48480b02d0 2022-06-29T15:51:21.053 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.038+0000 7f48467fc700 1 -- 172.21.15.47:0/1693853125 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f483800bb40 con 0x7f48480b02d0 2022-06-29T15:51:21.053 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.042+0000 7f484cf6b700 1 -- 172.21.15.47:0/1693853125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48480b02d0 msgr2=0x7f48481406b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:21.053 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.042+0000 7f484cf6b700 1 --2- 172.21.15.47:0/1693853125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48480b02d0 0x7f48481406b0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 rx=0x7f48380032f0 tx=0x7f4838009290).stop 2022-06-29T15:51:21.056 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.042+0000 7f484cf6b700 1 -- 172.21.15.47:0/1693853125 shutdown_connections 2022-06-29T15:51:21.057 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.042+0000 7f484cf6b700 1 --2- 172.21.15.47:0/1693853125 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f48480afdc0 0x7f48480ae0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:21.057 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.042+0000 7f484cf6b700 1 --2- 172.21.15.47:0/1693853125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48480b02d0 0x7f48481406b0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:21.057 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.042+0000 7f484cf6b700 1 --2- 172.21.15.47:0/1693853125 >> v2:172.21.15.64:3300/0 conn(0x7f48480b5ac0 0x7f48481401c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:21.057 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.042+0000 7f484cf6b700 1 -- 172.21.15.47:0/1693853125 >> 172.21.15.47:0/1693853125 conn(0x7f4848062560 msgr2=0x7f4848062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:21.058 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.042+0000 7f484cf6b700 1 -- 172.21.15.47:0/1693853125 shutdown_connections 2022-06-29T15:51:21.058 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.042+0000 7f484cf6b700 1 -- 172.21.15.47:0/1693853125 wait complete. 2022-06-29T15:51:21.058 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.042+0000 7f484cf6b700 1 Processor -- start 2022-06-29T15:51:21.058 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.042+0000 7f484cf6b700 1 -- start start 2022-06-29T15:51:21.059 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.042+0000 7f484cf6b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48480afdc0 0x7f48480b1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:21.059 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.042+0000 7f484cf6b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48480b02d0 0x7f484813c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:21.059 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.042+0000 7f484cf6b700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f48480b5ac0 0x7f484813c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:21.059 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.042+0000 7f484cf6b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4848152530 con 0x7f48480afdc0 2022-06-29T15:51:21.060 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.042+0000 7f484cf6b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48481527d0 con 0x7f48480b02d0 2022-06-29T15:51:21.060 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.042+0000 7f484cf6b700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4848152a70 con 0x7f48480b5ac0 2022-06-29T15:51:21.060 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.042+0000 7f48477fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48480afdc0 0x7f48480b1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:21.060 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.042+0000 7f48477fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48480afdc0 0x7f48480b1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53546/0 (socket says 172.21.15.47:53546) 2022-06-29T15:51:21.061 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.042+0000 7f48477fe700 1 -- 172.21.15.47:0/2002762709 learned_addr learned my addr 172.21.15.47:0/2002762709 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:21.061 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.042+0000 7f4846ffd700 1 --2- 172.21.15.47:0/2002762709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48480b02d0 0x7f484813c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:21.061 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.042+0000 7f4847fff700 1 --2- 172.21.15.47:0/2002762709 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f48480b5ac0 0x7f484813c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:21.062 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.042+0000 7f48477fe700 1 -- 172.21.15.47:0/2002762709 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f48480b5ac0 msgr2=0x7f484813c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:21.062 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.042+0000 7f48477fe700 1 --2- 172.21.15.47:0/2002762709 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f48480b5ac0 0x7f484813c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:21.062 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.042+0000 7f48477fe700 1 -- 172.21.15.47:0/2002762709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48480b02d0 msgr2=0x7f484813c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:21.062 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.042+0000 7f48477fe700 1 --2- 172.21.15.47:0/2002762709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48480b02d0 0x7f484813c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:21.062 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.042+0000 7f48477fe700 1 -- 172.21.15.47:0/2002762709 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4838009530 con 0x7f48480afdc0 2022-06-29T15:51:21.063 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.042+0000 7f48477fe700 1 --2- 172.21.15.47:0/2002762709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48480afdc0 0x7f48480b1390 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 rx=0x7f483c0068e0 tx=0x7f483c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:21.063 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.046+0000 7f4844ff9700 1 -- 172.21.15.47:0/2002762709 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f483c011780 con 0x7f48480afdc0 2022-06-29T15:51:21.063 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.046+0000 7f4844ff9700 1 -- 172.21.15.47:0/2002762709 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f483c0118e0 con 0x7f48480afdc0 2022-06-29T15:51:21.063 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.046+0000 7f484cf6b700 1 -- 172.21.15.47:0/2002762709 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48480b1880 con 0x7f48480afdc0 2022-06-29T15:51:21.064 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.046+0000 7f4844ff9700 1 -- 172.21.15.47:0/2002762709 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f483c011c40 con 0x7f48480afdc0 2022-06-29T15:51:21.064 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.046+0000 7f484cf6b700 1 -- 172.21.15.47:0/2002762709 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48481446b0 con 0x7f48480afdc0 2022-06-29T15:51:21.064 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.046+0000 7f484cf6b700 1 -- 172.21.15.47:0/2002762709 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48480573d0 con 0x7f48480afdc0 2022-06-29T15:51:21.064 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.046+0000 7f4844ff9700 1 -- 172.21.15.47:0/2002762709 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f483c012420 con 0x7f48480afdc0 2022-06-29T15:51:21.065 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.046+0000 7f4844ff9700 1 --2- 172.21.15.47:0/2002762709 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4828053950 0x7f4828055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:21.065 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.046+0000 7f4846ffd700 1 --2- 172.21.15.47:0/2002762709 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4828053950 0x7f4828055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:21.065 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.046+0000 7f4844ff9700 1 -- 172.21.15.47:0/2002762709 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f483c06bc20 con 0x7f48480afdc0 2022-06-29T15:51:21.065 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.050+0000 7f4846ffd700 1 --2- 172.21.15.47:0/2002762709 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4828053950 0x7f4828055e10 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7f483801a000 tx=0x7f4838016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:21.066 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.054+0000 7f4844ff9700 1 -- 172.21.15.47:0/2002762709 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f483c042bb0 con 0x7f48480afdc0 2022-06-29T15:51:21.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.182+0000 7f484cf6b700 1 -- 172.21.15.47:0/2002762709 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f4848069e20 con 0x7f48480afdc0 2022-06-29T15:51:21.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.182+0000 7f4844ff9700 1 -- 172.21.15.47:0/2002762709 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v136) v1 ==== 84+0+143 (secure 0 0 0) 0x7f483c0461d0 con 0x7f48480afdc0 2022-06-29T15:51:21.190 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:21.190 INFO:teuthology.orchestra.run.smithi047.stdout:[ 2022-06-29T15:51:21.191 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:21.191 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 1, 2022-06-29T15:51:21.191 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": "device_health_metrics" 2022-06-29T15:51:21.191 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:21.192 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:21.192 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 2, 2022-06-29T15:51:21.192 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": "rbd" 2022-06-29T15:51:21.192 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:21.193 INFO:teuthology.orchestra.run.smithi047.stdout:] 2022-06-29T15:51:21.194 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.186+0000 7f484cf6b700 1 -- 172.21.15.47:0/2002762709 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4828053950 msgr2=0x7f4828055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:21.194 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.186+0000 7f484cf6b700 1 --2- 172.21.15.47:0/2002762709 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4828053950 0x7f4828055e10 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7f483801a000 tx=0x7f4838016000).stop 2022-06-29T15:51:21.194 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.186+0000 7f484cf6b700 1 -- 172.21.15.47:0/2002762709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48480afdc0 msgr2=0x7f48480b1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:21.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.186+0000 7f484cf6b700 1 --2- 172.21.15.47:0/2002762709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48480afdc0 0x7f48480b1390 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 rx=0x7f483c0068e0 tx=0x7f483c008040).stop 2022-06-29T15:51:21.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.186+0000 7f484cf6b700 1 -- 172.21.15.47:0/2002762709 shutdown_connections 2022-06-29T15:51:21.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.186+0000 7f484cf6b700 1 --2- 172.21.15.47:0/2002762709 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f48480b5ac0 0x7f484813c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:21.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.186+0000 7f484cf6b700 1 --2- 172.21.15.47:0/2002762709 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4828053950 0x7f4828055e10 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:21.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.186+0000 7f484cf6b700 1 --2- 172.21.15.47:0/2002762709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48480b02d0 0x7f484813c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:21.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.186+0000 7f484cf6b700 1 --2- 172.21.15.47:0/2002762709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48480afdc0 0x7f48480b1390 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:21.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.186+0000 7f484cf6b700 1 -- 172.21.15.47:0/2002762709 >> 172.21.15.47:0/2002762709 conn(0x7f4848062560 msgr2=0x7f4848062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:21.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.186+0000 7f484cf6b700 1 -- 172.21.15.47:0/2002762709 shutdown_connections 2022-06-29T15:51:21.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.186+0000 7f484cf6b700 1 -- 172.21.15.47:0/2002762709 wait complete. 2022-06-29T15:51:21.202 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-06-29T15:51:21.203 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-06-29T15:51:21.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.302+0000 7f8d0d7f2700 1 Processor -- start 2022-06-29T15:51:21.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.302+0000 7f8d0d7f2700 1 -- start start 2022-06-29T15:51:21.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.306+0000 7f8d0d7f2700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8d080b6d30 0x7f8d0813d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:21.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.306+0000 7f8d0d7f2700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8d08062f30 0x7f8d0813d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:21.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.306+0000 7f8d0d7f2700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d08143f70 con 0x7f8d081400c0 2022-06-29T15:51:21.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.306+0000 7f8d0d7f2700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d080b1380 con 0x7f8d08062f30 2022-06-29T15:51:21.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.306+0000 7f8d0d7f2700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d080a9cf0 con 0x7f8d080b6d30 2022-06-29T15:51:21.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.306+0000 7f8d077fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8d08062f30 0x7f8d0813d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:21.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.306+0000 7f8d077fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8d08062f30 0x7f8d0813d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53556/0 (socket says 172.21.15.47:53556) 2022-06-29T15:51:21.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.306+0000 7f8d077fe700 1 -- 172.21.15.47:0/3189291423 learned_addr learned my addr 172.21.15.47:0/3189291423 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:21.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.306+0000 7f8d07fff700 1 --2- 172.21.15.47:0/3189291423 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8d080b6d30 0x7f8d0813d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:21.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.306+0000 7f8d077fe700 1 -- 172.21.15.47:0/3189291423 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8d080b6d30 msgr2=0x7f8d0813d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:21.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.306+0000 7f8d077fe700 1 --2- 172.21.15.47:0/3189291423 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8d080b6d30 0x7f8d0813d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:21.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.306+0000 7f8d077fe700 1 -- 172.21.15.47:0/3189291423 >> v1:172.21.15.64:6789/0 conn(0x7f8d081400c0 legacy=0x7f8d08140550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:21.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.306+0000 7f8d077fe700 1 -- 172.21.15.47:0/3189291423 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d0813dc80 con 0x7f8d08062f30 2022-06-29T15:51:21.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.306+0000 7f8d077fe700 1 --2- 172.21.15.47:0/3189291423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d08062f30 0x7f8d0813d790 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 rx=0x7f8cfc002f70 tx=0x7f8cfc006eb0).ready entity=mon.0 client_cookie=9d75f3c6dfd9b995 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:21.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.306+0000 7f8d06ffd700 1 -- 172.21.15.47:0/3189291423 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8cfc00d570 con 0x7f8d08062f30 2022-06-29T15:51:21.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.306+0000 7f8d06ffd700 1 -- 172.21.15.47:0/3189291423 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cfc000e90 con 0x7f8d08062f30 2022-06-29T15:51:21.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.306+0000 7f8d06ffd700 1 -- 172.21.15.47:0/3189291423 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8cfc00d900 con 0x7f8d08062f30 2022-06-29T15:51:21.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.306+0000 7f8d0d7f2700 1 -- 172.21.15.47:0/3189291423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d08062f30 msgr2=0x7f8d0813d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:21.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.306+0000 7f8d0d7f2700 1 --2- 172.21.15.47:0/3189291423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d08062f30 0x7f8d0813d790 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 rx=0x7f8cfc002f70 tx=0x7f8cfc006eb0).stop 2022-06-29T15:51:21.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.306+0000 7f8d0d7f2700 1 -- 172.21.15.47:0/3189291423 shutdown_connections 2022-06-29T15:51:21.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.306+0000 7f8d0d7f2700 1 --2- 172.21.15.47:0/3189291423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d08062f30 0x7f8d0813d790 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:21.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.306+0000 7f8d0d7f2700 1 --2- 172.21.15.47:0/3189291423 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8d080b6d30 0x7f8d0813d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:21.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.306+0000 7f8d0d7f2700 1 -- 172.21.15.47:0/3189291423 >> 172.21.15.47:0/3189291423 conn(0x7f8d08063670 msgr2=0x7f8d08063a50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:21.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.306+0000 7f8d0d7f2700 1 -- 172.21.15.47:0/3189291423 shutdown_connections 2022-06-29T15:51:21.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.306+0000 7f8d0d7f2700 1 -- 172.21.15.47:0/3189291423 wait complete. 2022-06-29T15:51:21.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.310+0000 7f8d0d7f2700 1 Processor -- start 2022-06-29T15:51:21.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.310+0000 7f8d0d7f2700 1 -- start start 2022-06-29T15:51:21.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.310+0000 7f8d0d7f2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d08062f30 0x7f8d08139b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:21.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.310+0000 7f8d0d7f2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d080b6d30 0x7f8d08156040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:21.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.310+0000 7f8d0d7f2700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8d081400c0 0x7f8d08151ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:21.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.310+0000 7f8d0d7f2700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d081416d0 con 0x7f8d080b6d30 2022-06-29T15:51:21.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.310+0000 7f8d0d7f2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d08155660 con 0x7f8d08062f30 2022-06-29T15:51:21.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.310+0000 7f8d0d7f2700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d08155900 con 0x7f8d081400c0 2022-06-29T15:51:21.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.310+0000 7f8d077fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d080b6d30 0x7f8d08156040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:21.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.310+0000 7f8d077fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d080b6d30 0x7f8d08156040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53562/0 (socket says 172.21.15.47:53562) 2022-06-29T15:51:21.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.310+0000 7f8d077fe700 1 -- 172.21.15.47:0/2760629072 learned_addr learned my addr 172.21.15.47:0/2760629072 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:21.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.310+0000 7f8d0cff1700 1 --2- 172.21.15.47:0/2760629072 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8d081400c0 0x7f8d08151ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:21.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.310+0000 7f8d07fff700 1 --2- 172.21.15.47:0/2760629072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d08062f30 0x7f8d08139b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:21.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.310+0000 7f8d077fe700 1 -- 172.21.15.47:0/2760629072 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8d081400c0 msgr2=0x7f8d08151ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:21.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.310+0000 7f8d077fe700 1 --2- 172.21.15.47:0/2760629072 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8d081400c0 0x7f8d08151ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:21.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.310+0000 7f8d077fe700 1 -- 172.21.15.47:0/2760629072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d08062f30 msgr2=0x7f8d08139b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:21.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.310+0000 7f8d077fe700 1 --2- 172.21.15.47:0/2760629072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d08062f30 0x7f8d08139b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:21.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.310+0000 7f8d077fe700 1 -- 172.21.15.47:0/2760629072 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cfc00b1e0 con 0x7f8d080b6d30 2022-06-29T15:51:21.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.310+0000 7f8d077fe700 1 --2- 172.21.15.47:0/2760629072 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d080b6d30 0x7f8d08156040 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 rx=0x7f8cfc002f70 tx=0x7f8cfc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:21.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.310+0000 7f8d057fa700 1 -- 172.21.15.47:0/2760629072 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8cfc011c30 con 0x7f8d080b6d30 2022-06-29T15:51:21.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.310+0000 7f8d057fa700 1 -- 172.21.15.47:0/2760629072 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cfc011d90 con 0x7f8d080b6d30 2022-06-29T15:51:21.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.310+0000 7f8d0d7f2700 1 -- 172.21.15.47:0/2760629072 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d08155ba0 con 0x7f8d080b6d30 2022-06-29T15:51:21.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.310+0000 7f8d057fa700 1 -- 172.21.15.47:0/2760629072 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8cfc01e650 con 0x7f8d080b6d30 2022-06-29T15:51:21.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.310+0000 7f8d0d7f2700 1 -- 172.21.15.47:0/2760629072 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d0813c1a0 con 0x7f8d080b6d30 2022-06-29T15:51:21.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.314+0000 7f8d0d7f2700 1 -- 172.21.15.47:0/2760629072 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d08155d30 con 0x7f8d080b6d30 2022-06-29T15:51:21.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.314+0000 7f8d057fa700 1 -- 172.21.15.47:0/2760629072 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8cfc025030 con 0x7f8d080b6d30 2022-06-29T15:51:21.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.314+0000 7f8d057fa700 1 --2- 172.21.15.47:0/2760629072 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8ce8053aa0 0x7f8ce8055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:21.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.314+0000 7f8d057fa700 1 -- 172.21.15.47:0/2760629072 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f8cfc06d260 con 0x7f8d080b6d30 2022-06-29T15:51:21.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.314+0000 7f8d07fff700 1 --2- 172.21.15.47:0/2760629072 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8ce8053aa0 0x7f8ce8055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:21.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.314+0000 7f8d07fff700 1 --2- 172.21.15.47:0/2760629072 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8ce8053aa0 0x7f8ce8055f60 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7f8cf800b000 tx=0x7f8cf8006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:21.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.322+0000 7f8d057fa700 1 -- 172.21.15.47:0/2760629072 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8cfc044250 con 0x7f8d080b6d30 2022-06-29T15:51:21.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:21.446+0000 7f8d0d7f2700 1 -- 172.21.15.47:0/2760629072 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f8d080573d0 con 0x7f8d080b6d30 2022-06-29T15:51:22.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.214+0000 7f8d057fa700 1 -- 172.21.15.47:0/2760629072 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v137) v1 ==== 133+0+0 (secure 0 0 0) 0x7f8cfc023020 con 0x7f8d080b6d30 2022-06-29T15:51:22.225 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.218+0000 7f8d0d7f2700 1 -- 172.21.15.47:0/2760629072 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8ce8053aa0 msgr2=0x7f8ce8055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:22.226 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.218+0000 7f8d0d7f2700 1 --2- 172.21.15.47:0/2760629072 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8ce8053aa0 0x7f8ce8055f60 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7f8cf800b000 tx=0x7f8cf8006df0).stop 2022-06-29T15:51:22.226 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.218+0000 7f8d0d7f2700 1 -- 172.21.15.47:0/2760629072 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d080b6d30 msgr2=0x7f8d08156040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:22.226 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.218+0000 7f8d0d7f2700 1 --2- 172.21.15.47:0/2760629072 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d080b6d30 0x7f8d08156040 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 rx=0x7f8cfc002f70 tx=0x7f8cfc015000).stop 2022-06-29T15:51:22.226 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.218+0000 7f8d0d7f2700 1 -- 172.21.15.47:0/2760629072 shutdown_connections 2022-06-29T15:51:22.227 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.218+0000 7f8d0d7f2700 1 --2- 172.21.15.47:0/2760629072 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f8d081400c0 0x7f8d08151ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:22.227 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.218+0000 7f8d0d7f2700 1 --2- 172.21.15.47:0/2760629072 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d080b6d30 0x7f8d08156040 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:22.227 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.218+0000 7f8d0d7f2700 1 --2- 172.21.15.47:0/2760629072 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f8ce8053aa0 0x7f8ce8055f60 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:22.227 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.218+0000 7f8d0d7f2700 1 --2- 172.21.15.47:0/2760629072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d08062f30 0x7f8d08139b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:22.228 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.218+0000 7f8d0d7f2700 1 -- 172.21.15.47:0/2760629072 >> 172.21.15.47:0/2760629072 conn(0x7f8d08063670 msgr2=0x7f8d08065b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:22.228 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.218+0000 7f8d0d7f2700 1 -- 172.21.15.47:0/2760629072 shutdown_connections 2022-06-29T15:51:22.228 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.218+0000 7f8d0d7f2700 1 -- 172.21.15.47:0/2760629072 wait complete. 2022-06-29T15:51:22.229 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' created 2022-06-29T15:51:22.238 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-06-29T15:51:22.352 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.346+0000 7f2219283700 1 Processor -- start 2022-06-29T15:51:22.352 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.346+0000 7f2219283700 1 -- start start 2022-06-29T15:51:22.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.346+0000 7f2219283700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f221405c850 0x7f2214143a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:22.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.346+0000 7f2219283700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f22140b5d90 0x7f22140b4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:22.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.346+0000 7f2219283700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f221405cc30 con 0x7f22140676a0 2022-06-29T15:51:22.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.346+0000 7f2219283700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22141430a0 con 0x7f221405c850 2022-06-29T15:51:22.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.346+0000 7f2219283700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2214143300 con 0x7f22140b5d90 2022-06-29T15:51:22.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.346+0000 7f2213fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f221405c850 0x7f2214143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:22.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.346+0000 7f2218a82700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f22140676a0 0x7f2214140130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:46688/0 (socket says 172.21.15.47:46688) 2022-06-29T15:51:22.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.346+0000 7f220bfff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f22140b5d90 0x7f22140b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:22.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.346+0000 7f2218a82700 1 -- 172.21.15.47:0/2907208379 learned_addr learned my addr 172.21.15.47:0/2907208379 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:22.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.346+0000 7f2213fff700 1 -- 172.21.15.47:0/2907208379 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f22140b5d90 msgr2=0x7f22140b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:22.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.346+0000 7f2213fff700 1 --2- 172.21.15.47:0/2907208379 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f22140b5d90 0x7f22140b4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:22.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.346+0000 7f2213fff700 1 -- 172.21.15.47:0/2907208379 >> v1:172.21.15.64:6789/0 conn(0x7f22140676a0 legacy=0x7f2214140130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:22.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.346+0000 7f2213fff700 1 -- 172.21.15.47:0/2907208379 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22141435a0 con 0x7f221405c850 2022-06-29T15:51:22.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.346+0000 7f2213fff700 1 --2- 172.21.15.47:0/2907208379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f221405c850 0x7f2214143a40 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7f21fc002f70 tx=0x7f21fc006d90).ready entity=mon.1 client_cookie=ae5312cbe6a492f9 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:22.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.346+0000 7f22137fe700 1 -- 172.21.15.47:0/2907208379 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f21fc00d4a0 con 0x7f221405c850 2022-06-29T15:51:22.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.346+0000 7f22137fe700 1 -- 172.21.15.47:0/2907208379 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21fc000e90 con 0x7f221405c850 2022-06-29T15:51:22.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.346+0000 7f22137fe700 1 -- 172.21.15.47:0/2907208379 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f21fc00d7f0 con 0x7f221405c850 2022-06-29T15:51:22.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.350+0000 7f2219283700 1 -- 172.21.15.47:0/2907208379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f221405c850 msgr2=0x7f2214143a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:22.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.350+0000 7f2219283700 1 --2- 172.21.15.47:0/2907208379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f221405c850 0x7f2214143a40 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7f21fc002f70 tx=0x7f21fc006d90).stop 2022-06-29T15:51:22.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.350+0000 7f2219283700 1 -- 172.21.15.47:0/2907208379 shutdown_connections 2022-06-29T15:51:22.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.350+0000 7f2219283700 1 --2- 172.21.15.47:0/2907208379 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f22140b5d90 0x7f22140b4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:22.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.350+0000 7f2219283700 1 --2- 172.21.15.47:0/2907208379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f221405c850 0x7f2214143a40 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:22.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.350+0000 7f2219283700 1 -- 172.21.15.47:0/2907208379 >> 172.21.15.47:0/2907208379 conn(0x7f2214060e00 msgr2=0x7f22140611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:22.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.350+0000 7f2219283700 1 -- 172.21.15.47:0/2907208379 shutdown_connections 2022-06-29T15:51:22.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.350+0000 7f2219283700 1 -- 172.21.15.47:0/2907208379 wait complete. 2022-06-29T15:51:22.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.350+0000 7f2219283700 1 Processor -- start 2022-06-29T15:51:22.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.350+0000 7f2219283700 1 -- start start 2022-06-29T15:51:22.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.354+0000 7f2219283700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f221405c850 0x7f22140b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:22.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.354+0000 7f2219283700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22140676a0 0x7f22140af820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:22.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.354+0000 7f2219283700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22140b5d90 0x7f22140b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:22.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.354+0000 7f2219283700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2214152c10 con 0x7f22140676a0 2022-06-29T15:51:22.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.354+0000 7f2219283700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22140b3bb0 con 0x7f22140b5d90 2022-06-29T15:51:22.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.354+0000 7f2219283700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22140af3b0 con 0x7f221405c850 2022-06-29T15:51:22.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.354+0000 7f220bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22140676a0 0x7f22140af820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:22.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.354+0000 7f2213fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f221405c850 0x7f22140b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:22.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.354+0000 7f220bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22140676a0 0x7f22140af820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53576/0 (socket says 172.21.15.47:53576) 2022-06-29T15:51:22.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.354+0000 7f220bfff700 1 -- 172.21.15.47:0/3416081117 learned_addr learned my addr 172.21.15.47:0/3416081117 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:22.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.354+0000 7f2213fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f221405c850 0x7f22140b06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:50676/0 (socket says 172.21.15.47:50676) 2022-06-29T15:51:22.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.354+0000 7f2218a82700 1 --2- 172.21.15.47:0/3416081117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22140b5d90 0x7f22140b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:22.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.354+0000 7f220bfff700 1 -- 172.21.15.47:0/3416081117 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f221405c850 msgr2=0x7f22140b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:22.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.354+0000 7f220bfff700 1 --2- 172.21.15.47:0/3416081117 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f221405c850 0x7f22140b06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:22.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.354+0000 7f220bfff700 1 -- 172.21.15.47:0/3416081117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22140b5d90 msgr2=0x7f22140b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:22.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.354+0000 7f220bfff700 1 --2- 172.21.15.47:0/3416081117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22140b5d90 0x7f22140b2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:22.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.354+0000 7f220bfff700 1 -- 172.21.15.47:0/3416081117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21fc00b120 con 0x7f22140676a0 2022-06-29T15:51:22.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.354+0000 7f220bfff700 1 --2- 172.21.15.47:0/3416081117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22140676a0 0x7f22140af820 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 rx=0x7f220400c000 tx=0x7f2204006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:22.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.354+0000 7f2211ffb700 1 -- 172.21.15.47:0/3416081117 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2204010780 con 0x7f22140676a0 2022-06-29T15:51:22.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.354+0000 7f2211ffb700 1 -- 172.21.15.47:0/3416081117 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22040108e0 con 0x7f22140676a0 2022-06-29T15:51:22.370 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.354+0000 7f2219283700 1 -- 172.21.15.47:0/3416081117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2214152e80 con 0x7f22140676a0 2022-06-29T15:51:22.370 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.354+0000 7f2211ffb700 1 -- 172.21.15.47:0/3416081117 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2204010c40 con 0x7f22140676a0 2022-06-29T15:51:22.370 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.354+0000 7f2219283700 1 -- 172.21.15.47:0/3416081117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22140afd10 con 0x7f22140676a0 2022-06-29T15:51:22.370 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.354+0000 7f2211ffb700 1 -- 172.21.15.47:0/3416081117 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f220401c460 con 0x7f22140676a0 2022-06-29T15:51:22.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.354+0000 7f2219283700 1 -- 172.21.15.47:0/3416081117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22140af5b0 con 0x7f22140676a0 2022-06-29T15:51:22.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.354+0000 7f2211ffb700 1 --2- 172.21.15.47:0/3416081117 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f21ec053aa0 0x7f21ec055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:22.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.354+0000 7f2211ffb700 1 -- 172.21.15.47:0/3416081117 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5954+0+0 (secure 0 0 0) 0x7f220406c260 con 0x7f22140676a0 2022-06-29T15:51:22.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.354+0000 7f2213fff700 1 --2- 172.21.15.47:0/3416081117 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f21ec053aa0 0x7f21ec055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:22.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.358+0000 7f2213fff700 1 --2- 172.21.15.47:0/3416081117 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f21ec053aa0 0x7f21ec055f60 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7f21fc01a000 tx=0x7f21fc014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:22.372 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.362+0000 7f2211ffb700 1 -- 172.21.15.47:0/3416081117 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f22040430f0 con 0x7f22140676a0 2022-06-29T15:51:22.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:22.486+0000 7f2219283700 1 -- 172.21.15.47:0/3416081117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f2214153010 con 0x7f22140676a0 2022-06-29T15:51:23.229 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.218+0000 7f2211ffb700 1 -- 172.21.15.47:0/3416081117 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v138) v1 ==== 125+0+0 (secure 0 0 0) 0x7f2204046710 con 0x7f22140676a0 2022-06-29T15:51:23.237 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.230+0000 7f2219283700 1 -- 172.21.15.47:0/3416081117 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f21ec053aa0 msgr2=0x7f21ec055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:23.237 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.230+0000 7f2219283700 1 --2- 172.21.15.47:0/3416081117 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f21ec053aa0 0x7f21ec055f60 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7f21fc01a000 tx=0x7f21fc014000).stop 2022-06-29T15:51:23.237 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.230+0000 7f2219283700 1 -- 172.21.15.47:0/3416081117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22140676a0 msgr2=0x7f22140af820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:23.237 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.230+0000 7f2219283700 1 --2- 172.21.15.47:0/3416081117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22140676a0 0x7f22140af820 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 rx=0x7f220400c000 tx=0x7f2204006df0).stop 2022-06-29T15:51:23.238 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.230+0000 7f2219283700 1 -- 172.21.15.47:0/3416081117 shutdown_connections 2022-06-29T15:51:23.239 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.230+0000 7f2219283700 1 --2- 172.21.15.47:0/3416081117 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f21ec053aa0 0x7f21ec055f60 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:23.239 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.230+0000 7f2219283700 1 --2- 172.21.15.47:0/3416081117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22140b5d90 0x7f22140b2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:23.239 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.230+0000 7f2219283700 1 --2- 172.21.15.47:0/3416081117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22140676a0 0x7f22140af820 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:23.239 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.230+0000 7f2219283700 1 --2- 172.21.15.47:0/3416081117 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f221405c850 0x7f22140b06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:23.240 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.230+0000 7f2219283700 1 -- 172.21.15.47:0/3416081117 >> 172.21.15.47:0/3416081117 conn(0x7f2214060e00 msgr2=0x7f22141447e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:23.240 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.230+0000 7f2219283700 1 -- 172.21.15.47:0/3416081117 shutdown_connections 2022-06-29T15:51:23.240 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.230+0000 7f2219283700 1 -- 172.21.15.47:0/3416081117 wait complete. 2022-06-29T15:51:23.241 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' created 2022-06-29T15:51:23.253 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-06-29T15:51:23.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.354+0000 7fa514c3e700 1 Processor -- start 2022-06-29T15:51:23.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.354+0000 7fa514c3e700 1 -- start start 2022-06-29T15:51:23.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.358+0000 7fa514c3e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa510067500 0x7fa510140f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:23.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.358+0000 7fa514c3e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa51013e930 0x7fa510066ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:23.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.358+0000 7fa514c3e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa51005f7c0 con 0x7fa51013e550 2022-06-29T15:51:23.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.358+0000 7fa514c3e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5100b3a80 con 0x7fa51013e930 2022-06-29T15:51:23.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.358+0000 7fa514c3e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5100b4ca0 con 0x7fa510067500 2022-06-29T15:51:23.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.358+0000 7fa50e59c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa51013e550 0x7fa51005f0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39136/0 (socket says 172.21.15.47:39136) 2022-06-29T15:51:23.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.358+0000 7fa50e59c700 1 -- 172.21.15.47:0/529519346 learned_addr learned my addr 172.21.15.47:0/529519346 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:23.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.358+0000 7fa50dd9b700 1 --2- 172.21.15.47:0/529519346 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa510067500 0x7fa510140f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:23.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.358+0000 7fa50ed9d700 1 --2- 172.21.15.47:0/529519346 >> v2:172.21.15.64:3300/0 conn(0x7fa51013e930 0x7fa510066ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:23.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.358+0000 7fa50d59a700 1 -- 172.21.15.47:0/529519346 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2803914890 0 0) 0x7fa51005f7c0 con 0x7fa51013e550 2022-06-29T15:51:23.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.358+0000 7fa50d59a700 1 -- 172.21.15.47:0/529519346 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4f4003260 con 0x7fa51013e550 2022-06-29T15:51:23.370 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.358+0000 7fa50ed9d700 1 -- 172.21.15.47:0/529519346 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa510067500 msgr2=0x7fa510140f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:23.370 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.358+0000 7fa50ed9d700 1 --2- 172.21.15.47:0/529519346 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa510067500 0x7fa510140f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:23.370 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.358+0000 7fa50ed9d700 1 -- 172.21.15.47:0/529519346 >> v1:172.21.15.47:6789/0 conn(0x7fa51013e550 legacy=0x7fa51005f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:23.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.358+0000 7fa50ed9d700 1 -- 172.21.15.47:0/529519346 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5100b37e0 con 0x7fa51013e930 2022-06-29T15:51:23.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.358+0000 7fa50dd9b700 1 --2- 172.21.15.47:0/529519346 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa510067500 0x7fa510140f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:51:23.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.358+0000 7fa50ed9d700 1 --2- 172.21.15.47:0/529519346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa51013e930 0x7fa510066ba0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7fa5080032f0 tx=0x7fa5080091d0).ready entity=mon.1 client_cookie=6f6c7311343cb073 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:23.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.358+0000 7fa50d59a700 1 -- 172.21.15.47:0/529519346 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa50800b600 con 0x7fa51013e930 2022-06-29T15:51:23.372 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.358+0000 7fa50d59a700 1 -- 172.21.15.47:0/529519346 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa50800b760 con 0x7fa51013e930 2022-06-29T15:51:23.372 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.358+0000 7fa50d59a700 1 -- 172.21.15.47:0/529519346 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa50800b8d0 con 0x7fa51013e930 2022-06-29T15:51:23.372 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.358+0000 7fa514c3e700 1 -- 172.21.15.47:0/529519346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa51013e930 msgr2=0x7fa510066ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:23.372 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.358+0000 7fa514c3e700 1 --2- 172.21.15.47:0/529519346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa51013e930 0x7fa510066ba0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7fa5080032f0 tx=0x7fa5080091d0).stop 2022-06-29T15:51:23.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.358+0000 7fa514c3e700 1 -- 172.21.15.47:0/529519346 shutdown_connections 2022-06-29T15:51:23.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.358+0000 7fa514c3e700 1 --2- 172.21.15.47:0/529519346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa51013e930 0x7fa510066ba0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:23.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.358+0000 7fa514c3e700 1 --2- 172.21.15.47:0/529519346 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa510067500 0x7fa510140f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:23.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.358+0000 7fa514c3e700 1 -- 172.21.15.47:0/529519346 >> 172.21.15.47:0/529519346 conn(0x7fa5100627d0 msgr2=0x7fa510062bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:23.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.362+0000 7fa514c3e700 1 -- 172.21.15.47:0/529519346 shutdown_connections 2022-06-29T15:51:23.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.362+0000 7fa514c3e700 1 -- 172.21.15.47:0/529519346 wait complete. 2022-06-29T15:51:23.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.362+0000 7fa514c3e700 1 Processor -- start 2022-06-29T15:51:23.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.362+0000 7fa514c3e700 1 -- start start 2022-06-29T15:51:23.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.362+0000 7fa514c3e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa510067500 0x7fa5100b5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:23.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.362+0000 7fa514c3e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa51013e550 0x7fa510150d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:23.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.362+0000 7fa514c3e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa51013e930 0x7fa5100af9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:23.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.362+0000 7fa514c3e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5100b6ef0 con 0x7fa51013e550 2022-06-29T15:51:23.379 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.362+0000 7fa514c3e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5101503a0 con 0x7fa510067500 2022-06-29T15:51:23.379 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.362+0000 7fa514c3e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa510151200 con 0x7fa51013e930 2022-06-29T15:51:23.379 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.362+0000 7fa50dd9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa51013e550 0x7fa510150d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:23.379 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.362+0000 7fa50dd9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa51013e550 0x7fa510150d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53590/0 (socket says 172.21.15.47:53590) 2022-06-29T15:51:23.380 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.362+0000 7fa50dd9b700 1 -- 172.21.15.47:0/2764300998 learned_addr learned my addr 172.21.15.47:0/2764300998 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:23.380 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.362+0000 7fa50ed9d700 1 --2- 172.21.15.47:0/2764300998 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa51013e930 0x7fa5100af9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:23.380 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.366+0000 7fa50e59c700 1 --2- 172.21.15.47:0/2764300998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa510067500 0x7fa5100b5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:23.380 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.366+0000 7fa50dd9b700 1 -- 172.21.15.47:0/2764300998 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa51013e930 msgr2=0x7fa5100af9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:23.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.366+0000 7fa50dd9b700 1 --2- 172.21.15.47:0/2764300998 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa51013e930 0x7fa5100af9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:23.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.366+0000 7fa50dd9b700 1 -- 172.21.15.47:0/2764300998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa510067500 msgr2=0x7fa5100b5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:23.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.366+0000 7fa50dd9b700 1 --2- 172.21.15.47:0/2764300998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa510067500 0x7fa5100b5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:23.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.366+0000 7fa50dd9b700 1 -- 172.21.15.47:0/2764300998 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa50400b180 con 0x7fa51013e550 2022-06-29T15:51:23.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.366+0000 7fa50ed9d700 1 --2- 172.21.15.47:0/2764300998 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa51013e930 0x7fa5100af9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:51:23.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.366+0000 7fa50e59c700 1 --2- 172.21.15.47:0/2764300998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa510067500 0x7fa5100b5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:51:23.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.366+0000 7fa50dd9b700 1 --2- 172.21.15.47:0/2764300998 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa51013e550 0x7fa510150d10 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 rx=0x7fa510145260 tx=0x7fa504006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:23.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.366+0000 7fa4ff7fe700 1 -- 172.21.15.47:0/2764300998 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa504010780 con 0x7fa51013e550 2022-06-29T15:51:23.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.366+0000 7fa4ff7fe700 1 -- 172.21.15.47:0/2764300998 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5040108e0 con 0x7fa51013e550 2022-06-29T15:51:23.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.366+0000 7fa4ff7fe700 1 -- 172.21.15.47:0/2764300998 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa504010c40 con 0x7fa51013e550 2022-06-29T15:51:23.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.366+0000 7fa514c3e700 1 -- 172.21.15.47:0/2764300998 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa508009470 con 0x7fa51013e550 2022-06-29T15:51:23.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.366+0000 7fa514c3e700 1 -- 172.21.15.47:0/2764300998 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5101505d0 con 0x7fa51013e550 2022-06-29T15:51:23.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.366+0000 7fa514c3e700 1 -- 172.21.15.47:0/2764300998 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa510150a10 con 0x7fa51013e550 2022-06-29T15:51:23.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.370+0000 7fa4ff7fe700 1 -- 172.21.15.47:0/2764300998 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa50401c460 con 0x7fa51013e550 2022-06-29T15:51:23.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.370+0000 7fa4ff7fe700 1 --2- 172.21.15.47:0/2764300998 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa4ec053b30 0x7fa4ec055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:23.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.370+0000 7fa4ff7fe700 1 -- 172.21.15.47:0/2764300998 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6304+0+0 (secure 0 0 0) 0x7fa50406c5e0 con 0x7fa51013e550 2022-06-29T15:51:23.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.370+0000 7fa50e59c700 1 --2- 172.21.15.47:0/2764300998 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa4ec053b30 0x7fa4ec055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:23.385 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.370+0000 7fa50e59c700 1 --2- 172.21.15.47:0/2764300998 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa4ec053b30 0x7fa4ec055ff0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7fa50000d000 tx=0x7fa500009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:23.385 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.374+0000 7fa4ff7fe700 1 -- 172.21.15.47:0/2764300998 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa5040432b0 con 0x7fa51013e550 2022-06-29T15:51:23.546 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.538+0000 7fa514c3e700 1 -- 172.21.15.47:0/2764300998 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fa5100573d0 con 0x7fa51013e550 2022-06-29T15:51:23.557 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.550+0000 7fa4ff7fe700 1 -- 172.21.15.47:0/2764300998 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 25 and data pool 26 v67) v1 ==== 189+0+0 (secure 0 0 0) 0x7fa5040468d0 con 0x7fa51013e550 2022-06-29T15:51:23.561 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.554+0000 7fa514c3e700 1 -- 172.21.15.47:0/2764300998 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa4ec053b30 msgr2=0x7fa4ec055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:23.561 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.554+0000 7fa514c3e700 1 --2- 172.21.15.47:0/2764300998 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa4ec053b30 0x7fa4ec055ff0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7fa50000d000 tx=0x7fa500009040).stop 2022-06-29T15:51:23.561 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.554+0000 7fa514c3e700 1 -- 172.21.15.47:0/2764300998 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa51013e550 msgr2=0x7fa510150d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:23.561 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.554+0000 7fa514c3e700 1 --2- 172.21.15.47:0/2764300998 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa51013e550 0x7fa510150d10 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 rx=0x7fa510145260 tx=0x7fa504006df0).stop 2022-06-29T15:51:23.562 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.554+0000 7fa514c3e700 1 -- 172.21.15.47:0/2764300998 shutdown_connections 2022-06-29T15:51:23.563 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.554+0000 7fa514c3e700 1 --2- 172.21.15.47:0/2764300998 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa51013e930 0x7fa5100af9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:23.563 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.554+0000 7fa514c3e700 1 --2- 172.21.15.47:0/2764300998 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa51013e550 0x7fa510150d10 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:23.563 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.554+0000 7fa514c3e700 1 --2- 172.21.15.47:0/2764300998 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa4ec053b30 0x7fa4ec055ff0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:23.564 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.554+0000 7fa514c3e700 1 --2- 172.21.15.47:0/2764300998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa510067500 0x7fa5100b5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:23.564 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.554+0000 7fa514c3e700 1 -- 172.21.15.47:0/2764300998 >> 172.21.15.47:0/2764300998 conn(0x7fa5100627d0 msgr2=0x7fa510142340 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:23.565 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.558+0000 7fa514c3e700 1 -- 172.21.15.47:0/2764300998 shutdown_connections 2022-06-29T15:51:23.565 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.558+0000 7fa514c3e700 1 -- 172.21.15.47:0/2764300998 wait complete. 2022-06-29T15:51:23.566 INFO:teuthology.orchestra.run.smithi047.stderr:new fs with metadata pool 25 and data pool 26 2022-06-29T15:51:23.578 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:51:23.667 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.662+0000 7fc881e94700 1 Processor -- start 2022-06-29T15:51:23.668 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.662+0000 7fc881e94700 1 -- start start 2022-06-29T15:51:23.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.662+0000 7fc881e94700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc87c061670 0x7fc87c143040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:23.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.662+0000 7fc881e94700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc87c061af0 0x7fc87c143530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:23.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.662+0000 7fc881e94700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc87c0ae370 con 0x7fc87c13cfc0 2022-06-29T15:51:23.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.662+0000 7fc881e94700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc87c0b1b90 con 0x7fc87c061670 2022-06-29T15:51:23.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.662+0000 7fc881e94700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc87c05f150 con 0x7fc87c061af0 2022-06-29T15:51:23.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.662+0000 7fc872ffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc87c061670 0x7fc87c143040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:23.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.662+0000 7fc87bfff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc87c061af0 0x7fc87c143530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:23.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.662+0000 7fc872ffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc87c061670 0x7fc87c143040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:59310/0 (socket says 172.21.15.47:59310) 2022-06-29T15:51:23.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.662+0000 7fc872ffd700 1 -- 172.21.15.47:0/702482973 learned_addr learned my addr 172.21.15.47:0/702482973 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:23.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.662+0000 7fc87b7fe700 1 --1- 172.21.15.47:0/702482973 >> v1:172.21.15.64:6789/0 conn(0x7fc87c13cfc0 0x7fc87c0b1480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:46724/0 (socket says 172.21.15.47:46724) 2022-06-29T15:51:23.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.662+0000 7fc872ffd700 1 -- 172.21.15.47:0/702482973 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc87c061af0 msgr2=0x7fc87c143530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:23.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.662+0000 7fc872ffd700 1 --2- 172.21.15.47:0/702482973 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc87c061af0 0x7fc87c143530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:23.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.662+0000 7fc872ffd700 1 -- 172.21.15.47:0/702482973 >> v1:172.21.15.64:6789/0 conn(0x7fc87c13cfc0 legacy=0x7fc87c0b1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:23.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.662+0000 7fc872ffd700 1 -- 172.21.15.47:0/702482973 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc87c05f3f0 con 0x7fc87c061670 2022-06-29T15:51:23.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.662+0000 7fc872ffd700 1 --2- 172.21.15.47:0/702482973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc87c061670 0x7fc87c143040 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7fc864002f70 tx=0x7fc864006eb0).ready entity=mon.1 client_cookie=fe082bcb9cc9e3ae server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:23.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.662+0000 7fc87affd700 1 -- 172.21.15.47:0/702482973 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc86400d570 con 0x7fc87c061670 2022-06-29T15:51:23.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.662+0000 7fc87affd700 1 -- 172.21.15.47:0/702482973 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc864000e90 con 0x7fc87c061670 2022-06-29T15:51:23.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.662+0000 7fc87affd700 1 -- 172.21.15.47:0/702482973 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc86400d900 con 0x7fc87c061670 2022-06-29T15:51:23.678 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.662+0000 7fc881e94700 1 -- 172.21.15.47:0/702482973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc87c061670 msgr2=0x7fc87c143040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:23.678 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.662+0000 7fc881e94700 1 --2- 172.21.15.47:0/702482973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc87c061670 0x7fc87c143040 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7fc864002f70 tx=0x7fc864006eb0).stop 2022-06-29T15:51:23.678 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.662+0000 7fc881e94700 1 -- 172.21.15.47:0/702482973 shutdown_connections 2022-06-29T15:51:23.678 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.662+0000 7fc881e94700 1 --2- 172.21.15.47:0/702482973 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc87c061af0 0x7fc87c143530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:23.679 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.662+0000 7fc881e94700 1 --2- 172.21.15.47:0/702482973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc87c061670 0x7fc87c143040 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:23.679 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.662+0000 7fc881e94700 1 -- 172.21.15.47:0/702482973 >> 172.21.15.47:0/702482973 conn(0x7fc87c05c510 msgr2=0x7fc87c063800 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:23.679 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.662+0000 7fc881e94700 1 -- 172.21.15.47:0/702482973 shutdown_connections 2022-06-29T15:51:23.680 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.662+0000 7fc881e94700 1 -- 172.21.15.47:0/702482973 wait complete. 2022-06-29T15:51:23.680 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.666+0000 7fc881e94700 1 Processor -- start 2022-06-29T15:51:23.680 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.666+0000 7fc881e94700 1 -- start start 2022-06-29T15:51:23.680 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.666+0000 7fc881e94700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc87c061670 0x7fc87c0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:23.681 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.666+0000 7fc881e94700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc87c061af0 0x7fc87c148b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:23.681 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.666+0000 7fc881e94700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc87c13cfc0 0x7fc87c0b63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:23.681 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.666+0000 7fc881e94700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc87c0b74a0 con 0x7fc87c061670 2022-06-29T15:51:23.681 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.666+0000 7fc881e94700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc87c142d30 con 0x7fc87c13cfc0 2022-06-29T15:51:23.682 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.666+0000 7fc881e94700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc87c0b7200 con 0x7fc87c061af0 2022-06-29T15:51:23.682 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.666+0000 7fc87b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc87c061670 0x7fc87c0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:23.682 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.666+0000 7fc87b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc87c061670 0x7fc87c0b4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53610/0 (socket says 172.21.15.47:53610) 2022-06-29T15:51:23.682 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.666+0000 7fc87b7fe700 1 -- 172.21.15.47:0/690480902 learned_addr learned my addr 172.21.15.47:0/690480902 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:23.683 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.666+0000 7fc872ffd700 1 --2- 172.21.15.47:0/690480902 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc87c061af0 0x7fc87c148b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:23.683 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.666+0000 7fc87bfff700 1 --2- 172.21.15.47:0/690480902 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc87c13cfc0 0x7fc87c0b63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:23.683 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.666+0000 7fc87b7fe700 1 -- 172.21.15.47:0/690480902 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc87c061af0 msgr2=0x7fc87c148b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:23.683 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.666+0000 7fc87b7fe700 1 --2- 172.21.15.47:0/690480902 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc87c061af0 0x7fc87c148b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:23.684 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.666+0000 7fc87b7fe700 1 -- 172.21.15.47:0/690480902 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc87c13cfc0 msgr2=0x7fc87c0b63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:23.684 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.666+0000 7fc87b7fe700 1 --2- 172.21.15.47:0/690480902 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc87c13cfc0 0x7fc87c0b63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:23.684 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.666+0000 7fc87b7fe700 1 -- 172.21.15.47:0/690480902 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc86400b1e0 con 0x7fc87c061670 2022-06-29T15:51:23.684 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.666+0000 7fc87b7fe700 1 --2- 172.21.15.47:0/690480902 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc87c061670 0x7fc87c0b4cf0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 rx=0x7fc87c0b0e20 tx=0x7fc86c007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:23.684 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.666+0000 7fc8797fa700 1 -- 172.21.15.47:0/690480902 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc86c00ccd0 con 0x7fc87c061670 2022-06-29T15:51:23.685 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.666+0000 7fc8797fa700 1 -- 172.21.15.47:0/690480902 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc86c00ce30 con 0x7fc87c061670 2022-06-29T15:51:23.685 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.666+0000 7fc881e94700 1 -- 172.21.15.47:0/690480902 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc87c142a90 con 0x7fc87c061670 2022-06-29T15:51:23.685 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.666+0000 7fc8797fa700 1 -- 172.21.15.47:0/690480902 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc86c007360 con 0x7fc87c061670 2022-06-29T15:51:23.685 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.666+0000 7fc881e94700 1 -- 172.21.15.47:0/690480902 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc87c143d70 con 0x7fc87c061670 2022-06-29T15:51:23.686 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.670+0000 7fc881e94700 1 -- 172.21.15.47:0/690480902 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc87c142f90 con 0x7fc87c061670 2022-06-29T15:51:23.686 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.670+0000 7fc8797fa700 1 -- 172.21.15.47:0/690480902 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc86c01b070 con 0x7fc87c061670 2022-06-29T15:51:23.686 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.670+0000 7fc8797fa700 1 --2- 172.21.15.47:0/690480902 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc860053b30 0x7fc860055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:23.687 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.670+0000 7fc8797fa700 1 -- 172.21.15.47:0/690480902 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc86c06b330 con 0x7fc87c061670 2022-06-29T15:51:23.687 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.670+0000 7fc872ffd700 1 --2- 172.21.15.47:0/690480902 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc860053b30 0x7fc860055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:23.688 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.670+0000 7fc872ffd700 1 --2- 172.21.15.47:0/690480902 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc860053b30 0x7fc860055ff0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7fc86401c000 tx=0x7fc864015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:23.688 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.678+0000 7fc8797fa700 1 -- 172.21.15.47:0/690480902 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc86c041fe0 con 0x7fc87c061670 2022-06-29T15:51:23.809 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.802+0000 7fc881e94700 1 -- 172.21.15.47:0/690480902 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc87c0573d0 con 0x7fc87c061670 2022-06-29T15:51:23.811 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.802+0000 7fc8797fa700 1 -- 172.21.15.47:0/690480902 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) v1 ==== 74+0+16298 (secure 0 0 0) 0x7fc86c045600 con 0x7fc87c061670 2022-06-29T15:51:23.811 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:23.812 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":139,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:51:23.543715+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:51:21.506960+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2022-06-29T15:51:22.546983+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:51:23.813 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.806+0000 7fc881e94700 1 -- 172.21.15.47:0/690480902 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc860053b30 msgr2=0x7fc860055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:23.814 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.806+0000 7fc881e94700 1 --2- 172.21.15.47:0/690480902 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc860053b30 0x7fc860055ff0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7fc86401c000 tx=0x7fc864015000).stop 2022-06-29T15:51:23.814 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.806+0000 7fc881e94700 1 -- 172.21.15.47:0/690480902 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc87c061670 msgr2=0x7fc87c0b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:23.814 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.806+0000 7fc881e94700 1 --2- 172.21.15.47:0/690480902 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc87c061670 0x7fc87c0b4cf0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 rx=0x7fc87c0b0e20 tx=0x7fc86c007c30).stop 2022-06-29T15:51:23.814 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.806+0000 7fc881e94700 1 -- 172.21.15.47:0/690480902 shutdown_connections 2022-06-29T15:51:23.815 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.806+0000 7fc881e94700 1 --2- 172.21.15.47:0/690480902 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc860053b30 0x7fc860055ff0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:23.815 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.806+0000 7fc881e94700 1 --2- 172.21.15.47:0/690480902 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc87c13cfc0 0x7fc87c0b63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:23.815 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.806+0000 7fc881e94700 1 --2- 172.21.15.47:0/690480902 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc87c061af0 0x7fc87c148b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:23.815 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.806+0000 7fc881e94700 1 --2- 172.21.15.47:0/690480902 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc87c061670 0x7fc87c0b4cf0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:23.816 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.806+0000 7fc881e94700 1 -- 172.21.15.47:0/690480902 >> 172.21.15.47:0/690480902 conn(0x7fc87c05c510 msgr2=0x7fc87c1453d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:23.816 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.806+0000 7fc881e94700 1 -- 172.21.15.47:0/690480902 shutdown_connections 2022-06-29T15:51:23.816 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.806+0000 7fc881e94700 1 -- 172.21.15.47:0/690480902 wait complete. 2022-06-29T15:51:23.825 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:51:23.927 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.922+0000 7f083842b700 1 Processor -- start 2022-06-29T15:51:23.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.922+0000 7f083842b700 1 -- start start 2022-06-29T15:51:23.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.922+0000 7f083842b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f08300b2120 0x7f08300b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:23.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.922+0000 7f083842b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f08300b1260 0x7f083014b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:23.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.922+0000 7f083842b700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08300b0740 con 0x7f083005f0b0 2022-06-29T15:51:23.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.922+0000 7f083842b700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0830147460 con 0x7f08300b1260 2022-06-29T15:51:23.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.922+0000 7f083842b700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08300b0970 con 0x7f08300b2120 2022-06-29T15:51:23.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.922+0000 7f08369c8700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f08300b1260 0x7f083014b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:23.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.922+0000 7f08369c8700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f08300b1260 0x7f083014b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53622/0 (socket says 172.21.15.47:53622) 2022-06-29T15:51:23.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.922+0000 7f08359c6700 1 --1- 172.21.15.47:0/4265146510 >> v1:172.21.15.64:6789/0 conn(0x7f083005f0b0 0x7f0830147af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:46736/0 (socket says 172.21.15.47:46736) 2022-06-29T15:51:23.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.922+0000 7f08361c7700 1 --2- 172.21.15.47:0/4265146510 >> v2:172.21.15.64:3300/0 conn(0x7f08300b2120 0x7f08300b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:23.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.922+0000 7f08369c8700 1 -- 172.21.15.47:0/4265146510 learned_addr learned my addr 172.21.15.47:0/4265146510 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:23.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.922+0000 7f08369c8700 1 -- 172.21.15.47:0/4265146510 >> v2:172.21.15.64:3300/0 conn(0x7f08300b2120 msgr2=0x7f08300b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:23.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.922+0000 7f08369c8700 1 --2- 172.21.15.47:0/4265146510 >> v2:172.21.15.64:3300/0 conn(0x7f08300b2120 0x7f08300b2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:23.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.922+0000 7f08369c8700 1 -- 172.21.15.47:0/4265146510 >> v1:172.21.15.64:6789/0 conn(0x7f083005f0b0 legacy=0x7f0830147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:23.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.922+0000 7f08369c8700 1 -- 172.21.15.47:0/4265146510 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08300b0c10 con 0x7f08300b1260 2022-06-29T15:51:23.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.922+0000 7f08369c8700 1 --2- 172.21.15.47:0/4265146510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08300b1260 0x7f083014b520 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 rx=0x7f0830141a70 tx=0x7f0828006eb0).ready entity=mon.0 client_cookie=9886d614168885c7 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:23.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.922+0000 7f08351c5700 1 -- 172.21.15.47:0/4265146510 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f082800d610 con 0x7f08300b1260 2022-06-29T15:51:23.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.922+0000 7f08351c5700 1 -- 172.21.15.47:0/4265146510 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0828000e90 con 0x7f08300b1260 2022-06-29T15:51:23.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.922+0000 7f08351c5700 1 -- 172.21.15.47:0/4265146510 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f082800d960 con 0x7f08300b1260 2022-06-29T15:51:23.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.922+0000 7f083842b700 1 -- 172.21.15.47:0/4265146510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08300b1260 msgr2=0x7f083014b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:23.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.922+0000 7f083842b700 1 --2- 172.21.15.47:0/4265146510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08300b1260 0x7f083014b520 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 rx=0x7f0830141a70 tx=0x7f0828006eb0).stop 2022-06-29T15:51:23.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.922+0000 7f083842b700 1 -- 172.21.15.47:0/4265146510 shutdown_connections 2022-06-29T15:51:23.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.922+0000 7f083842b700 1 --2- 172.21.15.47:0/4265146510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08300b1260 0x7f083014b520 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:23.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.922+0000 7f083842b700 1 --2- 172.21.15.47:0/4265146510 >> v2:172.21.15.64:3300/0 conn(0x7f08300b2120 0x7f08300b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:23.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.922+0000 7f083842b700 1 -- 172.21.15.47:0/4265146510 >> 172.21.15.47:0/4265146510 conn(0x7f08300626f0 msgr2=0x7f0830062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:23.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.922+0000 7f083842b700 1 -- 172.21.15.47:0/4265146510 shutdown_connections 2022-06-29T15:51:23.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.922+0000 7f083842b700 1 -- 172.21.15.47:0/4265146510 wait complete. 2022-06-29T15:51:23.939 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.926+0000 7f083842b700 1 Processor -- start 2022-06-29T15:51:23.940 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.926+0000 7f083842b700 1 -- start start 2022-06-29T15:51:23.940 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.926+0000 7f083842b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08300b1260 0x7f0830149af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:23.940 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.926+0000 7f083842b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08300b2120 0x7f0830141ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:23.940 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.926+0000 7f083842b700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f08300ae410 0x7f0830141fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:23.941 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.926+0000 7f083842b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08300ae8b0 con 0x7f08300b1260 2022-06-29T15:51:23.941 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.926+0000 7f083842b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0830143de0 con 0x7f08300b2120 2022-06-29T15:51:23.941 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.926+0000 7f083842b700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0830144080 con 0x7f08300ae410 2022-06-29T15:51:23.941 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.926+0000 7f08361c7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08300b1260 0x7f0830149af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:23.942 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.926+0000 7f08361c7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08300b1260 0x7f0830149af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53624/0 (socket says 172.21.15.47:53624) 2022-06-29T15:51:23.942 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.926+0000 7f08361c7700 1 -- 172.21.15.47:0/3882406394 learned_addr learned my addr 172.21.15.47:0/3882406394 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:23.942 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.926+0000 7f08369c8700 1 --2- 172.21.15.47:0/3882406394 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f08300ae410 0x7f0830141fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:23.942 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.926+0000 7f08359c6700 1 --2- 172.21.15.47:0/3882406394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08300b2120 0x7f0830141ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:23.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.926+0000 7f08361c7700 1 -- 172.21.15.47:0/3882406394 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f08300ae410 msgr2=0x7f0830141fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:23.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.926+0000 7f08361c7700 1 --2- 172.21.15.47:0/3882406394 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f08300ae410 0x7f0830141fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:23.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.926+0000 7f08361c7700 1 -- 172.21.15.47:0/3882406394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08300b2120 msgr2=0x7f0830141ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:23.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.926+0000 7f08361c7700 1 --2- 172.21.15.47:0/3882406394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08300b2120 0x7f0830141ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:23.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.926+0000 7f08361c7700 1 -- 172.21.15.47:0/3882406394 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f082800b1e0 con 0x7f08300b1260 2022-06-29T15:51:23.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.930+0000 7f08361c7700 1 --2- 172.21.15.47:0/3882406394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08300b1260 0x7f0830149af0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 rx=0x7f082c00c000 tx=0x7f082c0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:23.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.930+0000 7f081f7fe700 1 -- 172.21.15.47:0/3882406394 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f082c00c6a0 con 0x7f08300b1260 2022-06-29T15:51:23.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.930+0000 7f081f7fe700 1 -- 172.21.15.47:0/3882406394 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f082c00c800 con 0x7f08300b1260 2022-06-29T15:51:23.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.930+0000 7f081f7fe700 1 -- 172.21.15.47:0/3882406394 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f082c00cb60 con 0x7f08300b1260 2022-06-29T15:51:23.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.930+0000 7f083842b700 1 -- 172.21.15.47:0/3882406394 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08300af6b0 con 0x7f08300b1260 2022-06-29T15:51:23.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.930+0000 7f083842b700 1 -- 172.21.15.47:0/3882406394 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08301446c0 con 0x7f08300b1260 2022-06-29T15:51:23.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.930+0000 7f083842b700 1 -- 172.21.15.47:0/3882406394 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08300af050 con 0x7f08300b1260 2022-06-29T15:51:23.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.930+0000 7f081f7fe700 1 -- 172.21.15.47:0/3882406394 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f082c022070 con 0x7f08300b1260 2022-06-29T15:51:23.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.930+0000 7f081f7fe700 1 --2- 172.21.15.47:0/3882406394 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0810053a10 0x7f0810055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:23.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.930+0000 7f081f7fe700 1 -- 172.21.15.47:0/3882406394 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f082c06c790 con 0x7f08300b1260 2022-06-29T15:51:23.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.930+0000 7f08359c6700 1 --2- 172.21.15.47:0/3882406394 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0810053a10 0x7f0810055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:23.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.934+0000 7f08359c6700 1 --2- 172.21.15.47:0/3882406394 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0810053a10 0x7f0810055ed0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7f082000b000 tx=0x7f0820007160).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:23.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:23.942+0000 7f081f7fe700 1 -- 172.21.15.47:0/3882406394 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f08300af050 con 0x7f08300b1260 2022-06-29T15:51:24.077 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.070+0000 7f083842b700 1 -- 172.21.15.47:0/3882406394 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f08300af050 con 0x7f08300b1260 2022-06-29T15:51:24.078 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.070+0000 7f081f7fe700 1 -- 172.21.15.47:0/3882406394 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) v1 ==== 74+0+16298 (secure 0 0 0) 0x7f08300af050 con 0x7f08300b1260 2022-06-29T15:51:24.079 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:24.079 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":139,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:51:23.543715+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:51:21.506960+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2022-06-29T15:51:22.546983+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:51:24.081 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.074+0000 7f083842b700 1 -- 172.21.15.47:0/3882406394 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0810053a10 msgr2=0x7f0810055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:24.081 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.074+0000 7f083842b700 1 --2- 172.21.15.47:0/3882406394 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0810053a10 0x7f0810055ed0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7f082000b000 tx=0x7f0820007160).stop 2022-06-29T15:51:24.082 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.074+0000 7f083842b700 1 -- 172.21.15.47:0/3882406394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08300b1260 msgr2=0x7f0830149af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:24.082 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.074+0000 7f083842b700 1 --2- 172.21.15.47:0/3882406394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08300b1260 0x7f0830149af0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 rx=0x7f082c00c000 tx=0x7f082c0091d0).stop 2022-06-29T15:51:24.082 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.074+0000 7f083842b700 1 -- 172.21.15.47:0/3882406394 shutdown_connections 2022-06-29T15:51:24.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.074+0000 7f083842b700 1 --2- 172.21.15.47:0/3882406394 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f08300ae410 0x7f0830141fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:24.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.074+0000 7f083842b700 1 --2- 172.21.15.47:0/3882406394 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0810053a10 0x7f0810055ed0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:24.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.074+0000 7f083842b700 1 --2- 172.21.15.47:0/3882406394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08300b2120 0x7f0830141ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:24.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.074+0000 7f083842b700 1 --2- 172.21.15.47:0/3882406394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08300b1260 0x7f0830149af0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:24.084 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.074+0000 7f083842b700 1 -- 172.21.15.47:0/3882406394 >> 172.21.15.47:0/3882406394 conn(0x7f08300626f0 msgr2=0x7f08300b48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:24.084 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.074+0000 7f083842b700 1 -- 172.21.15.47:0/3882406394 shutdown_connections 2022-06-29T15:51:24.084 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.074+0000 7f083842b700 1 -- 172.21.15.47:0/3882406394 wait complete. 2022-06-29T15:51:24.092 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-06-29T15:51:24.202 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.194+0000 7f9d77fff700 1 Processor -- start 2022-06-29T15:51:24.202 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.194+0000 7f9d77fff700 1 -- start start 2022-06-29T15:51:24.205 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.194+0000 7f9d77fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9d78066cd0 0x7f9d7813c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:24.205 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.194+0000 7f9d77fff700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d780670b0 con 0x7f9d780afcc0 2022-06-29T15:51:24.205 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.194+0000 7f9d77fff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d780b01d0 con 0x7f9d780b16e0 2022-06-29T15:51:24.206 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.194+0000 7f9d77fff700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d780b0470 con 0x7f9d78066cd0 2022-06-29T15:51:24.206 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.194+0000 7f9d767fc700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9d780afcc0 0x7f9d7813cdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39186/0 (socket says 172.21.15.47:39186) 2022-06-29T15:51:24.206 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.194+0000 7f9d76ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9d78066cd0 0x7f9d7813c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:24.206 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.194+0000 7f9d767fc700 1 -- 172.21.15.47:0/390239025 learned_addr learned my addr 172.21.15.47:0/390239025 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:24.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.198+0000 7f9d75ffb700 1 -- 172.21.15.47:0/390239025 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1241040863 0 0) 0x7f9d780670b0 con 0x7f9d780afcc0 2022-06-29T15:51:24.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.198+0000 7f9d75ffb700 1 -- 172.21.15.47:0/390239025 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d5c003260 con 0x7f9d780afcc0 2022-06-29T15:51:24.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.198+0000 7f9d75ffb700 1 -- 172.21.15.47:0/390239025 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2179786902 0 0) 0x7f9d780b01d0 con 0x7f9d780b16e0 2022-06-29T15:51:24.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.198+0000 7f9d76ffd700 1 -- 172.21.15.47:0/390239025 >> v1:172.21.15.64:6789/0 conn(0x7f9d780b16e0 legacy=0x7f9d780ad9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:24.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.198+0000 7f9d76ffd700 1 -- 172.21.15.47:0/390239025 >> v1:172.21.15.47:6789/0 conn(0x7f9d780afcc0 legacy=0x7f9d7813cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:24.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.198+0000 7f9d76ffd700 1 -- 172.21.15.47:0/390239025 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d780b0710 con 0x7f9d78066cd0 2022-06-29T15:51:24.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.198+0000 7f9d76ffd700 1 --2- 172.21.15.47:0/390239025 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d78066cd0 0x7f9d7813c8c0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 rx=0x7f9d68001800 tx=0x7f9d6c006df0).ready entity=mon.0 client_cookie=6b45622b834d600 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:24.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.198+0000 7f9d75ffb700 1 -- 172.21.15.47:0/390239025 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d68001660 con 0x7f9d78066cd0 2022-06-29T15:51:24.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.198+0000 7f9d75ffb700 1 -- 172.21.15.47:0/390239025 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d6c000e90 con 0x7f9d78066cd0 2022-06-29T15:51:24.209 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.198+0000 7f9d75ffb700 1 -- 172.21.15.47:0/390239025 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d6c011400 con 0x7f9d78066cd0 2022-06-29T15:51:24.209 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.198+0000 7f9d77fff700 1 -- 172.21.15.47:0/390239025 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d78066cd0 msgr2=0x7f9d7813c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:24.209 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.198+0000 7f9d77fff700 1 --2- 172.21.15.47:0/390239025 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d78066cd0 0x7f9d7813c8c0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 rx=0x7f9d68001800 tx=0x7f9d6c006df0).stop 2022-06-29T15:51:24.209 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.198+0000 7f9d77fff700 1 -- 172.21.15.47:0/390239025 shutdown_connections 2022-06-29T15:51:24.210 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.198+0000 7f9d77fff700 1 --2- 172.21.15.47:0/390239025 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d78066cd0 0x7f9d7813c8c0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:24.210 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.198+0000 7f9d77fff700 1 -- 172.21.15.47:0/390239025 >> 172.21.15.47:0/390239025 conn(0x7f9d780b5320 msgr2=0x7f9d780b5700 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:24.210 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.198+0000 7f9d77fff700 1 -- 172.21.15.47:0/390239025 shutdown_connections 2022-06-29T15:51:24.210 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.198+0000 7f9d77fff700 1 -- 172.21.15.47:0/390239025 wait complete. 2022-06-29T15:51:24.213 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.198+0000 7f9d77fff700 1 Processor -- start 2022-06-29T15:51:24.213 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.198+0000 7f9d77fff700 1 -- start start 2022-06-29T15:51:24.213 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.198+0000 7f9d77fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d78066cd0 0x7f9d780b41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:24.214 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.198+0000 7f9d77fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9d780afcc0 0x7f9d78151e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:24.214 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.198+0000 7f9d77fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d780b16e0 0x7f9d78152360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:24.214 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.198+0000 7f9d77fff700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d78141f90 con 0x7f9d780b16e0 2022-06-29T15:51:24.214 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.198+0000 7f9d77fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d78143100 con 0x7f9d78066cd0 2022-06-29T15:51:24.215 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.198+0000 7f9d77fff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d780aa9b0 con 0x7f9d780afcc0 2022-06-29T15:51:24.215 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.198+0000 7f9d777fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d780b16e0 0x7f9d78152360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:24.215 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.198+0000 7f9d777fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d780b16e0 0x7f9d78152360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53640/0 (socket says 172.21.15.47:53640) 2022-06-29T15:51:24.215 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.198+0000 7f9d777fe700 1 -- 172.21.15.47:0/2325864412 learned_addr learned my addr 172.21.15.47:0/2325864412 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:24.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.198+0000 7f9d767fc700 1 --2- 172.21.15.47:0/2325864412 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9d780afcc0 0x7f9d78151e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:24.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.202+0000 7f9d76ffd700 1 --2- 172.21.15.47:0/2325864412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d78066cd0 0x7f9d780b41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:24.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.202+0000 7f9d777fe700 1 -- 172.21.15.47:0/2325864412 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9d780afcc0 msgr2=0x7f9d78151e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:24.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.202+0000 7f9d777fe700 1 --2- 172.21.15.47:0/2325864412 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9d780afcc0 0x7f9d78151e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:24.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.202+0000 7f9d777fe700 1 -- 172.21.15.47:0/2325864412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d78066cd0 msgr2=0x7f9d780b41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:24.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.202+0000 7f9d777fe700 1 --2- 172.21.15.47:0/2325864412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d78066cd0 0x7f9d780b41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:24.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.202+0000 7f9d777fe700 1 -- 172.21.15.47:0/2325864412 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d6c00b180 con 0x7f9d780b16e0 2022-06-29T15:51:24.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.202+0000 7f9d777fe700 1 --2- 172.21.15.47:0/2325864412 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d780b16e0 0x7f9d78152360 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 rx=0x7f9d70000f40 tx=0x7f9d70007f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:24.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.202+0000 7f9d67fff700 1 -- 172.21.15.47:0/2325864412 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d70006aa0 con 0x7f9d780b16e0 2022-06-29T15:51:24.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.202+0000 7f9d67fff700 1 -- 172.21.15.47:0/2325864412 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d70006c00 con 0x7f9d780b16e0 2022-06-29T15:51:24.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.202+0000 7f9d77fff700 1 -- 172.21.15.47:0/2325864412 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d781470c0 con 0x7f9d780b16e0 2022-06-29T15:51:24.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.202+0000 7f9d67fff700 1 -- 172.21.15.47:0/2325864412 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d70007030 con 0x7f9d780b16e0 2022-06-29T15:51:24.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.202+0000 7f9d77fff700 1 -- 172.21.15.47:0/2325864412 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d780aac50 con 0x7f9d780b16e0 2022-06-29T15:51:24.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.202+0000 7f9d77fff700 1 -- 172.21.15.47:0/2325864412 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d78147250 con 0x7f9d780b16e0 2022-06-29T15:51:24.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.202+0000 7f9d67fff700 1 -- 172.21.15.47:0/2325864412 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9d70010030 con 0x7f9d780b16e0 2022-06-29T15:51:24.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.202+0000 7f9d67fff700 1 --2- 172.21.15.47:0/2325864412 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9d540539e0 0x7f9d54055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:24.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.202+0000 7f9d67fff700 1 -- 172.21.15.47:0/2325864412 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9d7006bff0 con 0x7f9d780b16e0 2022-06-29T15:51:24.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.202+0000 7f9d76ffd700 1 --2- 172.21.15.47:0/2325864412 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9d540539e0 0x7f9d54055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:24.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.202+0000 7f9d76ffd700 1 --2- 172.21.15.47:0/2325864412 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9d540539e0 0x7f9d54055ea0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7f9d6c017000 tx=0x7f9d6c006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:24.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.210+0000 7f9d67fff700 1 -- 172.21.15.47:0/2325864412 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9d70042db0 con 0x7f9d780b16e0 2022-06-29T15:51:24.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:24.374+0000 7f9d77fff700 1 -- 172.21.15.47:0/2325864412 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f9d780573d0 con 0x7f9d780b16e0 2022-06-29T15:51:25.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.314+0000 7f9d67fff700 1 -- 172.21.15.47:0/2325864412 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v69) v1 ==== 118+0+0 (secure 0 0 0) 0x7f9d700463d0 con 0x7f9d780b16e0 2022-06-29T15:51:25.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.318+0000 7f9d77fff700 1 -- 172.21.15.47:0/2325864412 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9d540539e0 msgr2=0x7f9d54055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:25.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.318+0000 7f9d77fff700 1 --2- 172.21.15.47:0/2325864412 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9d540539e0 0x7f9d54055ea0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7f9d6c017000 tx=0x7f9d6c006df0).stop 2022-06-29T15:51:25.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.318+0000 7f9d77fff700 1 -- 172.21.15.47:0/2325864412 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d780b16e0 msgr2=0x7f9d78152360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:25.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.318+0000 7f9d77fff700 1 --2- 172.21.15.47:0/2325864412 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d780b16e0 0x7f9d78152360 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 rx=0x7f9d70000f40 tx=0x7f9d70007f90).stop 2022-06-29T15:51:25.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.318+0000 7f9d77fff700 1 -- 172.21.15.47:0/2325864412 shutdown_connections 2022-06-29T15:51:25.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.318+0000 7f9d77fff700 1 --2- 172.21.15.47:0/2325864412 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d780b16e0 0x7f9d78152360 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:25.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.318+0000 7f9d77fff700 1 --2- 172.21.15.47:0/2325864412 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9d780afcc0 0x7f9d78151e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:25.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.318+0000 7f9d77fff700 1 --2- 172.21.15.47:0/2325864412 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9d540539e0 0x7f9d54055ea0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:25.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.318+0000 7f9d77fff700 1 --2- 172.21.15.47:0/2325864412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d78066cd0 0x7f9d780b41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:25.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.318+0000 7f9d77fff700 1 -- 172.21.15.47:0/2325864412 >> 172.21.15.47:0/2325864412 conn(0x7f9d780b5320 msgr2=0x7f9d7813bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:25.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.318+0000 7f9d77fff700 1 -- 172.21.15.47:0/2325864412 shutdown_connections 2022-06-29T15:51:25.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.318+0000 7f9d77fff700 1 -- 172.21.15.47:0/2325864412 wait complete. 2022-06-29T15:51:25.339 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:51:25.447 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.442+0000 7f6b3d966700 1 Processor -- start 2022-06-29T15:51:25.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.442+0000 7f6b3d966700 1 -- start start 2022-06-29T15:51:25.450 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.442+0000 7f6b3d966700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6b380b2120 0x7f6b380b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:25.451 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.442+0000 7f6b3d966700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6b380b1260 0x7f6b3814b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:25.451 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.442+0000 7f6b3d966700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b380b0740 con 0x7f6b3805f0b0 2022-06-29T15:51:25.451 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.442+0000 7f6b3d966700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b38147460 con 0x7f6b380b2120 2022-06-29T15:51:25.451 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.442+0000 7f6b3d966700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b380b0970 con 0x7f6b380b1260 2022-06-29T15:51:25.452 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.442+0000 7f6b2e7fc700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f6b3805f0b0 0x7f6b38147af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39198/0 (socket says 172.21.15.47:39198) 2022-06-29T15:51:25.452 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.442+0000 7f6b2e7fc700 1 -- 172.21.15.47:0/2656483405 learned_addr learned my addr 172.21.15.47:0/2656483405 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:25.452 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.442+0000 7f6b36ffd700 1 --2- 172.21.15.47:0/2656483405 >> v2:172.21.15.47:3300/0 conn(0x7f6b380b2120 0x7f6b380b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:25.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.442+0000 7f6b377fe700 1 --2- 172.21.15.47:0/2656483405 >> v2:172.21.15.64:3300/0 conn(0x7f6b380b1260 0x7f6b3814b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:25.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.442+0000 7f6b367fc700 1 -- 172.21.15.47:0/2656483405 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 580043687 0 0) 0x7f6b380b0740 con 0x7f6b3805f0b0 2022-06-29T15:51:25.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.442+0000 7f6b367fc700 1 -- 172.21.15.47:0/2656483405 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b18003260 con 0x7f6b3805f0b0 2022-06-29T15:51:25.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.442+0000 7f6b36ffd700 1 -- 172.21.15.47:0/2656483405 >> v2:172.21.15.64:3300/0 conn(0x7f6b380b1260 msgr2=0x7f6b3814b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:25.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.442+0000 7f6b36ffd700 1 --2- 172.21.15.47:0/2656483405 >> v2:172.21.15.64:3300/0 conn(0x7f6b380b1260 0x7f6b3814b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:25.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.442+0000 7f6b36ffd700 1 -- 172.21.15.47:0/2656483405 >> v1:172.21.15.47:6789/0 conn(0x7f6b3805f0b0 legacy=0x7f6b38147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:25.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.442+0000 7f6b36ffd700 1 -- 172.21.15.47:0/2656483405 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b380b0c10 con 0x7f6b380b2120 2022-06-29T15:51:25.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.442+0000 7f6b377fe700 1 --2- 172.21.15.47:0/2656483405 >> v2:172.21.15.64:3300/0 conn(0x7f6b380b1260 0x7f6b3814b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:51:25.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.442+0000 7f6b36ffd700 1 --2- 172.21.15.47:0/2656483405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b380b2120 0x7f6b380b2500 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 rx=0x7f6b38141a70 tx=0x7f6b28006df0).ready entity=mon.0 client_cookie=f72f7f08c1c0234a server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:25.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.442+0000 7f6b367fc700 1 -- 172.21.15.47:0/2656483405 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b2800d5c0 con 0x7f6b380b2120 2022-06-29T15:51:25.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.442+0000 7f6b367fc700 1 -- 172.21.15.47:0/2656483405 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b28000e90 con 0x7f6b380b2120 2022-06-29T15:51:25.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.442+0000 7f6b367fc700 1 -- 172.21.15.47:0/2656483405 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b2800d910 con 0x7f6b380b2120 2022-06-29T15:51:25.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.442+0000 7f6b3d966700 1 -- 172.21.15.47:0/2656483405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b380b2120 msgr2=0x7f6b380b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:25.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.442+0000 7f6b3d966700 1 --2- 172.21.15.47:0/2656483405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b380b2120 0x7f6b380b2500 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 rx=0x7f6b38141a70 tx=0x7f6b28006df0).stop 2022-06-29T15:51:25.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.442+0000 7f6b3d966700 1 -- 172.21.15.47:0/2656483405 shutdown_connections 2022-06-29T15:51:25.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.442+0000 7f6b3d966700 1 --2- 172.21.15.47:0/2656483405 >> v2:172.21.15.64:3300/0 conn(0x7f6b380b1260 0x7f6b3814b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:25.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.442+0000 7f6b3d966700 1 --2- 172.21.15.47:0/2656483405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b380b2120 0x7f6b380b2500 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:25.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.442+0000 7f6b3d966700 1 -- 172.21.15.47:0/2656483405 >> 172.21.15.47:0/2656483405 conn(0x7f6b380626f0 msgr2=0x7f6b38062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:25.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.442+0000 7f6b3d966700 1 -- 172.21.15.47:0/2656483405 shutdown_connections 2022-06-29T15:51:25.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.442+0000 7f6b3d966700 1 -- 172.21.15.47:0/2656483405 wait complete. 2022-06-29T15:51:25.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.446+0000 7f6b3d966700 1 Processor -- start 2022-06-29T15:51:25.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.446+0000 7f6b3d966700 1 -- start start 2022-06-29T15:51:25.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.446+0000 7f6b3d966700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b380b1260 0x7f6b380aec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:25.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.446+0000 7f6b3d966700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6b380b2120 0x7f6b381454b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:25.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.446+0000 7f6b3d966700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b3814e8c0 0x7f6b38159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:25.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.446+0000 7f6b3d966700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3814dd20 con 0x7f6b380b1260 2022-06-29T15:51:25.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.446+0000 7f6b3d966700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b38144d70 con 0x7f6b3814e8c0 2022-06-29T15:51:25.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.446+0000 7f6b3d966700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b38145010 con 0x7f6b380b2120 2022-06-29T15:51:25.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.446+0000 7f6b36ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b380b1260 0x7f6b380aec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:25.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.446+0000 7f6b36ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b380b1260 0x7f6b380aec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53650/0 (socket says 172.21.15.47:53650) 2022-06-29T15:51:25.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.446+0000 7f6b36ffd700 1 -- 172.21.15.47:0/2542088111 learned_addr learned my addr 172.21.15.47:0/2542088111 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:25.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.446+0000 7f6b2e7fc700 1 --2- 172.21.15.47:0/2542088111 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6b380b2120 0x7f6b381454b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:25.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.446+0000 7f6b377fe700 1 --2- 172.21.15.47:0/2542088111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b3814e8c0 0x7f6b38159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:25.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.446+0000 7f6b36ffd700 1 -- 172.21.15.47:0/2542088111 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6b380b2120 msgr2=0x7f6b381454b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:25.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.446+0000 7f6b36ffd700 1 --2- 172.21.15.47:0/2542088111 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6b380b2120 0x7f6b381454b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:25.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.446+0000 7f6b36ffd700 1 -- 172.21.15.47:0/2542088111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b3814e8c0 msgr2=0x7f6b38159b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:25.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.446+0000 7f6b36ffd700 1 --2- 172.21.15.47:0/2542088111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b3814e8c0 0x7f6b38159b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:25.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.446+0000 7f6b36ffd700 1 -- 172.21.15.47:0/2542088111 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b2800b180 con 0x7f6b380b1260 2022-06-29T15:51:25.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.446+0000 7f6b36ffd700 1 --2- 172.21.15.47:0/2542088111 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b380b1260 0x7f6b380aec20 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 rx=0x7f6b38141a70 tx=0x7f6b28014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:25.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.446+0000 7f6b34ff9700 1 -- 172.21.15.47:0/2542088111 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b28013de0 con 0x7f6b380b1260 2022-06-29T15:51:25.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.446+0000 7f6b34ff9700 1 -- 172.21.15.47:0/2542088111 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b28002f70 con 0x7f6b380b1260 2022-06-29T15:51:25.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.446+0000 7f6b34ff9700 1 -- 172.21.15.47:0/2542088111 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b2801e5f0 con 0x7f6b380b1260 2022-06-29T15:51:25.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.446+0000 7f6b3d966700 1 -- 172.21.15.47:0/2542088111 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b3814d9a0 con 0x7f6b380b1260 2022-06-29T15:51:25.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.446+0000 7f6b3d966700 1 -- 172.21.15.47:0/2542088111 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b3814d740 con 0x7f6b380b1260 2022-06-29T15:51:25.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.450+0000 7f6b3d966700 1 -- 172.21.15.47:0/2542088111 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b3814df20 con 0x7f6b380b1260 2022-06-29T15:51:25.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.450+0000 7f6b34ff9700 1 -- 172.21.15.47:0/2542088111 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6b2801c070 con 0x7f6b380b1260 2022-06-29T15:51:25.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.450+0000 7f6b34ff9700 1 --2- 172.21.15.47:0/2542088111 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6b1c053b00 0x7f6b1c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:25.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.450+0000 7f6b34ff9700 1 -- 172.21.15.47:0/2542088111 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f6b2806ca70 con 0x7f6b380b1260 2022-06-29T15:51:25.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.450+0000 7f6b2e7fc700 1 --2- 172.21.15.47:0/2542088111 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6b1c053b00 0x7f6b1c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:25.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.450+0000 7f6b2e7fc700 1 --2- 172.21.15.47:0/2542088111 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6b1c053b00 0x7f6b1c055fc0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7f6b2000d000 tx=0x7f6b20009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:25.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.454+0000 7f6b34ff9700 1 -- 172.21.15.47:0/2542088111 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6b280437d0 con 0x7f6b380b1260 2022-06-29T15:51:25.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.622+0000 7f6b3d966700 1 -- 172.21.15.47:0/2542088111 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b381451d0 con 0x7f6b380b1260 2022-06-29T15:51:25.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.622+0000 7f6b34ff9700 1 -- 172.21.15.47:0/2542088111 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+2492 (secure 0 0 0) 0x7f6b2806ce60 con 0x7f6b380b1260 2022-06-29T15:51:25.630 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:25.630 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":69,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7957,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.64:6833/298405638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":298405638},{"type":"v1","addr":"172.21.15.64:6833","nonce":298405638}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":66},{"gid":8037,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/191070564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":191070564},{"type":"v1","addr":"172.21.15.47:6835","nonce":191070564}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:51:23.543892+0000","modified":"2022-06-29T15:51:25.309120+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":56894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_56894":{"gid":56894,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6837/1370488030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1370488030},{"type":"v1","addr":"172.21.15.47:6837","nonce":1370488030}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2022-06-29T15:51:25.632 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.626+0000 7f6b3d966700 1 -- 172.21.15.47:0/2542088111 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6b1c053b00 msgr2=0x7f6b1c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:25.632 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.626+0000 7f6b3d966700 1 --2- 172.21.15.47:0/2542088111 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6b1c053b00 0x7f6b1c055fc0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7f6b2000d000 tx=0x7f6b20009040).stop 2022-06-29T15:51:25.632 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.626+0000 7f6b3d966700 1 -- 172.21.15.47:0/2542088111 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b380b1260 msgr2=0x7f6b380aec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:25.632 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.626+0000 7f6b3d966700 1 --2- 172.21.15.47:0/2542088111 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b380b1260 0x7f6b380aec20 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 rx=0x7f6b38141a70 tx=0x7f6b28014000).stop 2022-06-29T15:51:25.633 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.626+0000 7f6b3d966700 1 -- 172.21.15.47:0/2542088111 shutdown_connections 2022-06-29T15:51:25.633 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.626+0000 7f6b3d966700 1 --2- 172.21.15.47:0/2542088111 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6b1c053b00 0x7f6b1c055fc0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:25.633 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.626+0000 7f6b3d966700 1 --2- 172.21.15.47:0/2542088111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b3814e8c0 0x7f6b38159b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:25.633 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.626+0000 7f6b3d966700 1 --2- 172.21.15.47:0/2542088111 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6b380b2120 0x7f6b381454b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:25.634 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.626+0000 7f6b3d966700 1 --2- 172.21.15.47:0/2542088111 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b380b1260 0x7f6b380aec20 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:25.634 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.626+0000 7f6b3d966700 1 -- 172.21.15.47:0/2542088111 >> 172.21.15.47:0/2542088111 conn(0x7f6b380626f0 msgr2=0x7f6b3805f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:25.634 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.626+0000 7f6b3d966700 1 -- 172.21.15.47:0/2542088111 shutdown_connections 2022-06-29T15:51:25.634 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.626+0000 7f6b3d966700 1 -- 172.21.15.47:0/2542088111 wait complete. 2022-06-29T15:51:25.635 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 69 2022-06-29T15:51:25.642 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:51:25.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.734+0000 7f2014112700 1 Processor -- start 2022-06-29T15:51:25.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.734+0000 7f2014112700 1 -- start start 2022-06-29T15:51:25.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.734+0000 7f2014112700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f200c1408c0 0x7f200c064f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:25.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.734+0000 7f2014112700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f200c0b4800 con 0x7f200c0b2490 2022-06-29T15:51:25.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.734+0000 7f2014112700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f200c0b2930 con 0x7f200c0b4420 2022-06-29T15:51:25.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.734+0000 7f2014112700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f200c0b2bd0 con 0x7f200c1408c0 2022-06-29T15:51:25.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.734+0000 7f20126af700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f200c1408c0 0x7f200c064f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:25.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.734+0000 7f20116ad700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f200c0b2490 0x7f200c064820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39212/0 (socket says 172.21.15.47:39212) 2022-06-29T15:51:25.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.734+0000 7f20116ad700 1 -- 172.21.15.47:0/2493553366 learned_addr learned my addr 172.21.15.47:0/2493553366 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:25.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.734+0000 7f2010eac700 1 -- 172.21.15.47:0/2493553366 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2044233536 0 0) 0x7f200c0b4800 con 0x7f200c0b2490 2022-06-29T15:51:25.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.734+0000 7f2010eac700 1 -- 172.21.15.47:0/2493553366 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ff4003260 con 0x7f200c0b2490 2022-06-29T15:51:25.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.734+0000 7f20126af700 1 -- 172.21.15.47:0/2493553366 >> v1:172.21.15.64:6789/0 conn(0x7f200c0b4420 legacy=0x7f200c1401b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:25.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.734+0000 7f20126af700 1 -- 172.21.15.47:0/2493553366 >> v1:172.21.15.47:6789/0 conn(0x7f200c0b2490 legacy=0x7f200c064820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:25.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.734+0000 7f20126af700 1 -- 172.21.15.47:0/2493553366 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f200c0b3930 con 0x7f200c1408c0 2022-06-29T15:51:25.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.734+0000 7f20126af700 1 --2- 172.21.15.47:0/2493553366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f200c1408c0 0x7f200c064f30 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 rx=0x7f20040032f0 tx=0x7f20040091d0).ready entity=mon.0 client_cookie=33dae64e812176ae server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:25.748 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.734+0000 7f2010eac700 1 -- 172.21.15.47:0/2493553366 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f200400b6c0 con 0x7f200c1408c0 2022-06-29T15:51:25.748 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.734+0000 7f2010eac700 1 -- 172.21.15.47:0/2493553366 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f200400b820 con 0x7f200c1408c0 2022-06-29T15:51:25.748 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.734+0000 7f2010eac700 1 -- 172.21.15.47:0/2493553366 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f200400b990 con 0x7f200c1408c0 2022-06-29T15:51:25.748 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.738+0000 7f2014112700 1 -- 172.21.15.47:0/2493553366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f200c1408c0 msgr2=0x7f200c064f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:25.749 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.738+0000 7f2014112700 1 --2- 172.21.15.47:0/2493553366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f200c1408c0 0x7f200c064f30 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 rx=0x7f20040032f0 tx=0x7f20040091d0).stop 2022-06-29T15:51:25.749 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.738+0000 7f2014112700 1 -- 172.21.15.47:0/2493553366 shutdown_connections 2022-06-29T15:51:25.749 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.738+0000 7f2014112700 1 --2- 172.21.15.47:0/2493553366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f200c1408c0 0x7f200c064f30 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:25.749 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.738+0000 7f2014112700 1 -- 172.21.15.47:0/2493553366 >> 172.21.15.47:0/2493553366 conn(0x7f200c05c510 msgr2=0x7f200c05c920 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:25.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.738+0000 7f2014112700 1 -- 172.21.15.47:0/2493553366 shutdown_connections 2022-06-29T15:51:25.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.738+0000 7f2014112700 1 -- 172.21.15.47:0/2493553366 wait complete. 2022-06-29T15:51:25.753 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.738+0000 7f2014112700 1 Processor -- start 2022-06-29T15:51:25.753 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.738+0000 7f2014112700 1 -- start start 2022-06-29T15:51:25.754 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.738+0000 7f2014112700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f200c0b2490 0x7f200c0b81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:25.754 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.738+0000 7f2014112700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f200c0b4420 0x7f200c0b66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:25.754 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.738+0000 7f2014112700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f200c1408c0 0x7f200c0b0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:25.755 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.738+0000 7f2014112700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200c13f490 con 0x7f200c1408c0 2022-06-29T15:51:25.755 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.738+0000 7f2014112700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200c0b1350 con 0x7f200c0b2490 2022-06-29T15:51:25.755 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.738+0000 7f2014112700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f200c0b15f0 con 0x7f200c0b4420 2022-06-29T15:51:25.755 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.738+0000 7f20126af700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f200c1408c0 0x7f200c0b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:25.756 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.738+0000 7f20126af700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f200c1408c0 0x7f200c0b0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53668/0 (socket says 172.21.15.47:53668) 2022-06-29T15:51:25.756 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.738+0000 7f20126af700 1 -- 172.21.15.47:0/2133112000 learned_addr learned my addr 172.21.15.47:0/2133112000 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:25.756 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.738+0000 7f20116ad700 1 --2- 172.21.15.47:0/2133112000 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f200c0b4420 0x7f200c0b66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:25.756 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.738+0000 7f2011eae700 1 --2- 172.21.15.47:0/2133112000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f200c0b2490 0x7f200c0b81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:25.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.742+0000 7f20126af700 1 -- 172.21.15.47:0/2133112000 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f200c0b4420 msgr2=0x7f200c0b66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:25.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.742+0000 7f20126af700 1 --2- 172.21.15.47:0/2133112000 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f200c0b4420 0x7f200c0b66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:25.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.742+0000 7f20126af700 1 -- 172.21.15.47:0/2133112000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f200c0b2490 msgr2=0x7f200c0b81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:25.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.742+0000 7f20126af700 1 --2- 172.21.15.47:0/2133112000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f200c0b2490 0x7f200c0b81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:25.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.742+0000 7f20126af700 1 -- 172.21.15.47:0/2133112000 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2004009470 con 0x7f200c1408c0 2022-06-29T15:51:25.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.742+0000 7f20126af700 1 --2- 172.21.15.47:0/2133112000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f200c1408c0 0x7f200c0b0970 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 rx=0x7f20040032f0 tx=0x7f2004016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:25.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.742+0000 7f1ffaffd700 1 -- 172.21.15.47:0/2133112000 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f200400b6c0 con 0x7f200c1408c0 2022-06-29T15:51:25.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.742+0000 7f1ffaffd700 1 -- 172.21.15.47:0/2133112000 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2004021630 con 0x7f200c1408c0 2022-06-29T15:51:25.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.742+0000 7f2014112700 1 -- 172.21.15.47:0/2133112000 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f200c0b1890 con 0x7f200c1408c0 2022-06-29T15:51:25.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.742+0000 7f1ffaffd700 1 -- 172.21.15.47:0/2133112000 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2004021990 con 0x7f200c1408c0 2022-06-29T15:51:25.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.742+0000 7f2014112700 1 -- 172.21.15.47:0/2133112000 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f200c0b0e60 con 0x7f200c1408c0 2022-06-29T15:51:25.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.742+0000 7f2014112700 1 -- 172.21.15.47:0/2133112000 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f200c0573d0 con 0x7f200c1408c0 2022-06-29T15:51:25.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.742+0000 7f1ffaffd700 1 -- 172.21.15.47:0/2133112000 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f200400f070 con 0x7f200c1408c0 2022-06-29T15:51:25.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.742+0000 7f1ffaffd700 1 --2- 172.21.15.47:0/2133112000 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1fec053950 0x7f1fec055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:25.761 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.742+0000 7f1ffaffd700 1 -- 172.21.15.47:0/2133112000 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f200406d0c0 con 0x7f200c1408c0 2022-06-29T15:51:25.761 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.742+0000 7f2011eae700 1 --2- 172.21.15.47:0/2133112000 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1fec053950 0x7f1fec055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:25.761 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.746+0000 7f2011eae700 1 --2- 172.21.15.47:0/2133112000 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1fec053950 0x7f1fec055e10 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 rx=0x7f200800d000 tx=0x7f2008000bc0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:25.762 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.750+0000 7f1ffaffd700 1 -- 172.21.15.47:0/2133112000 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2004043d70 con 0x7f200c1408c0 2022-06-29T15:51:25.884 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.878+0000 7f2014112700 1 -- 172.21.15.47:0/2133112000 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f200c06a0f0 con 0x7f200c1408c0 2022-06-29T15:51:25.886 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.878+0000 7f1ffaffd700 1 -- 172.21.15.47:0/2133112000 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f2004015070 con 0x7f200c1408c0 2022-06-29T15:51:25.886 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:25.887 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":140,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:51:24.546533+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:51:21.506960+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2022-06-29T15:51:22.546983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:51:25.889 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.882+0000 7f2014112700 1 -- 172.21.15.47:0/2133112000 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1fec053950 msgr2=0x7f1fec055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:25.889 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.882+0000 7f2014112700 1 --2- 172.21.15.47:0/2133112000 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1fec053950 0x7f1fec055e10 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 rx=0x7f200800d000 tx=0x7f2008000bc0).stop 2022-06-29T15:51:25.889 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.882+0000 7f2014112700 1 -- 172.21.15.47:0/2133112000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f200c1408c0 msgr2=0x7f200c0b0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:25.890 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.882+0000 7f2014112700 1 --2- 172.21.15.47:0/2133112000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f200c1408c0 0x7f200c0b0970 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 rx=0x7f20040032f0 tx=0x7f2004016000).stop 2022-06-29T15:51:25.890 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.882+0000 7f2014112700 1 -- 172.21.15.47:0/2133112000 shutdown_connections 2022-06-29T15:51:25.890 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.882+0000 7f2014112700 1 --2- 172.21.15.47:0/2133112000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f200c1408c0 0x7f200c0b0970 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:25.890 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.882+0000 7f2014112700 1 --2- 172.21.15.47:0/2133112000 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f200c0b4420 0x7f200c0b66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:25.891 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.882+0000 7f2014112700 1 --2- 172.21.15.47:0/2133112000 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1fec053950 0x7f1fec055e10 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:25.891 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.882+0000 7f2014112700 1 --2- 172.21.15.47:0/2133112000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f200c0b2490 0x7f200c0b81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:25.891 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.882+0000 7f2014112700 1 -- 172.21.15.47:0/2133112000 >> 172.21.15.47:0/2133112000 conn(0x7f200c05c510 msgr2=0x7f200c13b570 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:25.892 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.882+0000 7f2014112700 1 -- 172.21.15.47:0/2133112000 shutdown_connections 2022-06-29T15:51:25.892 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:25.882+0000 7f2014112700 1 -- 172.21.15.47:0/2133112000 wait complete. 2022-06-29T15:51:25.901 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-29T15:51:26.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.014+0000 7f07cff30700 1 Processor -- start 2022-06-29T15:51:26.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.014+0000 7f07cff30700 1 -- start start 2022-06-29T15:51:26.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.014+0000 7f07cff30700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f07c8060ee0 0x7f07c80612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:26.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.014+0000 7f07cff30700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f07c81471a0 0x7f07c8066f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:26.024 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.014+0000 7f07cff30700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07c8067610 con 0x7f07c80b4300 2022-06-29T15:51:26.024 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.014+0000 7f07cff30700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07c81467c0 con 0x7f07c8060ee0 2022-06-29T15:51:26.024 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.014+0000 7f07cff30700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07c8146a60 con 0x7f07c81471a0 2022-06-29T15:51:26.025 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.014+0000 7f07cdccc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f07c8060ee0 0x7f07c80612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:26.025 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.014+0000 7f07cdccc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f07c8060ee0 0x7f07c80612c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53672/0 (socket says 172.21.15.47:53672) 2022-06-29T15:51:26.025 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.014+0000 7f07cd4cb700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f07c80b4300 0x7f07c81430e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39228/0 (socket says 172.21.15.47:39228) 2022-06-29T15:51:26.025 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.014+0000 7f07cdccc700 1 -- 172.21.15.47:0/757560470 learned_addr learned my addr 172.21.15.47:0/757560470 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:26.026 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.014+0000 7f07ce4cd700 1 --2- 172.21.15.47:0/757560470 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f07c81471a0 0x7f07c8066f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:26.026 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.014+0000 7f07cccca700 1 -- 172.21.15.47:0/757560470 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3227484552 0 0) 0x7f07c8067610 con 0x7f07c80b4300 2022-06-29T15:51:26.026 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.014+0000 7f07cccca700 1 -- 172.21.15.47:0/757560470 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07b0003260 con 0x7f07c80b4300 2022-06-29T15:51:26.026 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.014+0000 7f07cdccc700 1 -- 172.21.15.47:0/757560470 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f07c81471a0 msgr2=0x7f07c8066f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:26.027 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.014+0000 7f07cdccc700 1 --2- 172.21.15.47:0/757560470 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f07c81471a0 0x7f07c8066f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:26.027 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.014+0000 7f07cdccc700 1 -- 172.21.15.47:0/757560470 >> v1:172.21.15.47:6789/0 conn(0x7f07c80b4300 legacy=0x7f07c81430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:26.027 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.014+0000 7f07cdccc700 1 -- 172.21.15.47:0/757560470 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07c8146d00 con 0x7f07c8060ee0 2022-06-29T15:51:26.027 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.014+0000 7f07ce4cd700 1 --2- 172.21.15.47:0/757560470 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f07c81471a0 0x7f07c8066f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:51:26.028 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.014+0000 7f07cdccc700 1 --2- 172.21.15.47:0/757560470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07c8060ee0 0x7f07c80612c0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 rx=0x7f07c80ae790 tx=0x7f07c4006eb0).ready entity=mon.0 client_cookie=2ddf33f7fa1da00a server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:26.028 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.014+0000 7f07cccca700 1 -- 172.21.15.47:0/757560470 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07c400d5c0 con 0x7f07c8060ee0 2022-06-29T15:51:26.028 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.018+0000 7f07cccca700 1 -- 172.21.15.47:0/757560470 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07c4000e90 con 0x7f07c8060ee0 2022-06-29T15:51:26.028 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.018+0000 7f07cccca700 1 -- 172.21.15.47:0/757560470 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07c400d950 con 0x7f07c8060ee0 2022-06-29T15:51:26.032 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.018+0000 7f07cff30700 1 -- 172.21.15.47:0/757560470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07c8060ee0 msgr2=0x7f07c80612c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:26.032 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.018+0000 7f07cff30700 1 --2- 172.21.15.47:0/757560470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07c8060ee0 0x7f07c80612c0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 rx=0x7f07c80ae790 tx=0x7f07c4006eb0).stop 2022-06-29T15:51:26.032 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.018+0000 7f07cff30700 1 -- 172.21.15.47:0/757560470 shutdown_connections 2022-06-29T15:51:26.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.018+0000 7f07cff30700 1 --2- 172.21.15.47:0/757560470 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f07c81471a0 0x7f07c8066f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:26.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.018+0000 7f07cff30700 1 --2- 172.21.15.47:0/757560470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07c8060ee0 0x7f07c80612c0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:26.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.018+0000 7f07cff30700 1 -- 172.21.15.47:0/757560470 >> 172.21.15.47:0/757560470 conn(0x7f07c805c510 msgr2=0x7f07c805f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:26.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.018+0000 7f07cff30700 1 -- 172.21.15.47:0/757560470 shutdown_connections 2022-06-29T15:51:26.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.018+0000 7f07cff30700 1 -- 172.21.15.47:0/757560470 wait complete. 2022-06-29T15:51:26.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.018+0000 7f07cff30700 1 Processor -- start 2022-06-29T15:51:26.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.022+0000 7f07cff30700 1 -- start start 2022-06-29T15:51:26.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.022+0000 7f07cff30700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07c8060ee0 0x7f07c80b76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:26.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.022+0000 7f07cff30700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f07c80b4300 0x7f07c8155b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:26.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.022+0000 7f07cff30700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07c81471a0 0x7f07c813ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:26.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.022+0000 7f07cff30700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c813dd80 con 0x7f07c8060ee0 2022-06-29T15:51:26.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.022+0000 7f07cff30700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c813e020 con 0x7f07c81471a0 2022-06-29T15:51:26.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.022+0000 7f07cff30700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07c8140790 con 0x7f07c80b4300 2022-06-29T15:51:26.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.022+0000 7f07cdccc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07c8060ee0 0x7f07c80b76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:26.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.022+0000 7f07cdccc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07c8060ee0 0x7f07c80b76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53678/0 (socket says 172.21.15.47:53678) 2022-06-29T15:51:26.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.022+0000 7f07cdccc700 1 -- 172.21.15.47:0/940804639 learned_addr learned my addr 172.21.15.47:0/940804639 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:26.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.022+0000 7f07cd4cb700 1 --2- 172.21.15.47:0/940804639 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f07c80b4300 0x7f07c8155b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:26.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.022+0000 7f07ce4cd700 1 --2- 172.21.15.47:0/940804639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07c81471a0 0x7f07c813ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:26.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.022+0000 7f07cdccc700 1 -- 172.21.15.47:0/940804639 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f07c80b4300 msgr2=0x7f07c8155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:26.038 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.022+0000 7f07cdccc700 1 --2- 172.21.15.47:0/940804639 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f07c80b4300 0x7f07c8155b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:26.038 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.022+0000 7f07cdccc700 1 -- 172.21.15.47:0/940804639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07c81471a0 msgr2=0x7f07c813ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:26.038 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.022+0000 7f07cdccc700 1 --2- 172.21.15.47:0/940804639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07c81471a0 0x7f07c813ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:26.038 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.022+0000 7f07cdccc700 1 -- 172.21.15.47:0/940804639 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07c0009470 con 0x7f07c8060ee0 2022-06-29T15:51:26.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.022+0000 7f07ce4cd700 1 --2- 172.21.15.47:0/940804639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07c81471a0 0x7f07c813ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:51:26.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.022+0000 7f07cdccc700 1 --2- 172.21.15.47:0/940804639 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07c8060ee0 0x7f07c80b76f0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 rx=0x7f07c80ae790 tx=0x7f07c4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:26.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.022+0000 7f07b6ffd700 1 -- 172.21.15.47:0/940804639 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07c4011760 con 0x7f07c8060ee0 2022-06-29T15:51:26.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.022+0000 7f07b6ffd700 1 -- 172.21.15.47:0/940804639 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07c40118c0 con 0x7f07c8060ee0 2022-06-29T15:51:26.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.022+0000 7f07cff30700 1 -- 172.21.15.47:0/940804639 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07c400b1e0 con 0x7f07c8060ee0 2022-06-29T15:51:26.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.022+0000 7f07b6ffd700 1 -- 172.21.15.47:0/940804639 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07c4011c20 con 0x7f07c8060ee0 2022-06-29T15:51:26.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.022+0000 7f07cff30700 1 -- 172.21.15.47:0/940804639 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07c813f120 con 0x7f07c8060ee0 2022-06-29T15:51:26.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.026+0000 7f07cff30700 1 -- 172.21.15.47:0/940804639 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07c8140b10 con 0x7f07c8060ee0 2022-06-29T15:51:26.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.026+0000 7f07b6ffd700 1 -- 172.21.15.47:0/940804639 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f07c4025030 con 0x7f07c8060ee0 2022-06-29T15:51:26.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.026+0000 7f07b6ffd700 1 --2- 172.21.15.47:0/940804639 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f07a8053aa0 0x7f07a8055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:26.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.026+0000 7f07b6ffd700 1 -- 172.21.15.47:0/940804639 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f07c406d1a0 con 0x7f07c8060ee0 2022-06-29T15:51:26.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.026+0000 7f07cd4cb700 1 --2- 172.21.15.47:0/940804639 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f07a8053aa0 0x7f07a8055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:26.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.026+0000 7f07cd4cb700 1 --2- 172.21.15.47:0/940804639 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f07a8053aa0 0x7f07a8055f60 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7f07b800d000 tx=0x7f07b8007a10).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:26.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.030+0000 7f07b6ffd700 1 -- 172.21.15.47:0/940804639 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f07c4043f00 con 0x7f07c8060ee0 2022-06-29T15:51:26.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.190+0000 7f07cff30700 1 -- 172.21.15.47:0/940804639 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f07c80573d0 con 0x7f07c8060ee0 2022-06-29T15:51:26.204 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.198+0000 7f07b6ffd700 1 -- 172.21.15.47:0/940804639 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v33) v1 ==== 184+0+0 (secure 0 0 0) 0x7f07c4023020 con 0x7f07c8060ee0 2022-06-29T15:51:26.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.198+0000 7f07cff30700 1 -- 172.21.15.47:0/940804639 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f07a8053aa0 msgr2=0x7f07a8055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:26.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.198+0000 7f07cff30700 1 --2- 172.21.15.47:0/940804639 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f07a8053aa0 0x7f07a8055f60 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7f07b800d000 tx=0x7f07b8007a10).stop 2022-06-29T15:51:26.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.198+0000 7f07cff30700 1 -- 172.21.15.47:0/940804639 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07c8060ee0 msgr2=0x7f07c80b76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:26.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.198+0000 7f07cff30700 1 --2- 172.21.15.47:0/940804639 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07c8060ee0 0x7f07c80b76f0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 rx=0x7f07c80ae790 tx=0x7f07c4015000).stop 2022-06-29T15:51:26.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.198+0000 7f07cff30700 1 -- 172.21.15.47:0/940804639 shutdown_connections 2022-06-29T15:51:26.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.198+0000 7f07cff30700 1 --2- 172.21.15.47:0/940804639 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f07a8053aa0 0x7f07a8055f60 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:26.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.198+0000 7f07cff30700 1 --2- 172.21.15.47:0/940804639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07c81471a0 0x7f07c813ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:26.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.198+0000 7f07cff30700 1 --2- 172.21.15.47:0/940804639 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f07c80b4300 0x7f07c8155b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:26.209 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.198+0000 7f07cff30700 1 --2- 172.21.15.47:0/940804639 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07c8060ee0 0x7f07c80b76f0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:26.209 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.198+0000 7f07cff30700 1 -- 172.21.15.47:0/940804639 >> 172.21.15.47:0/940804639 conn(0x7f07c805c510 msgr2=0x7f07c8143440 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:26.209 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.198+0000 7f07cff30700 1 -- 172.21.15.47:0/940804639 shutdown_connections 2022-06-29T15:51:26.209 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.198+0000 7f07cff30700 1 -- 172.21.15.47:0/940804639 wait complete. 2022-06-29T15:51:26.210 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.0 2022-06-29T15:51:26.219 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-29T15:51:26.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.330+0000 7f24a7356700 1 Processor -- start 2022-06-29T15:51:26.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.330+0000 7f24a7356700 1 -- start start 2022-06-29T15:51:26.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.330+0000 7f24a7356700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f24a0141fd0 0x7f24a00b3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:26.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.330+0000 7f24a7356700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24a005f7d0 con 0x7f24a00b1320 2022-06-29T15:51:26.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.330+0000 7f24a7356700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24a00b4b90 con 0x7f24a005f3f0 2022-06-29T15:51:26.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.330+0000 7f24a7356700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24a00b27a0 con 0x7f24a0141fd0 2022-06-29T15:51:26.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.330+0000 7f24a5b53700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f24a00b1320 0x7f24a00b2ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39242/0 (socket says 172.21.15.47:39242) 2022-06-29T15:51:26.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.330+0000 7f24a5b53700 1 -- 172.21.15.47:0/3377266534 learned_addr learned my addr 172.21.15.47:0/3377266534 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:26.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.330+0000 7f24a6b55700 1 --2- 172.21.15.47:0/3377266534 >> v2:172.21.15.64:3300/0 conn(0x7f24a0141fd0 0x7f24a00b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:26.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.330+0000 7f24a5352700 1 -- 172.21.15.47:0/3377266534 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3972676624 0 0) 0x7f24a005f7d0 con 0x7f24a00b1320 2022-06-29T15:51:26.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.330+0000 7f24a5352700 1 -- 172.21.15.47:0/3377266534 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f248c003260 con 0x7f24a00b1320 2022-06-29T15:51:26.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.330+0000 7f24a5352700 1 -- 172.21.15.47:0/3377266534 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3184166531 0 0) 0x7f248c003260 con 0x7f24a00b1320 2022-06-29T15:51:26.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.330+0000 7f24a5352700 1 -- 172.21.15.47:0/3377266534 >> v2:172.21.15.64:3300/0 conn(0x7f24a0141fd0 msgr2=0x7f24a00b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:26.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.330+0000 7f24a5352700 1 --2- 172.21.15.47:0/3377266534 >> v2:172.21.15.64:3300/0 conn(0x7f24a0141fd0 0x7f24a00b3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:26.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.330+0000 7f24a5352700 1 -- 172.21.15.47:0/3377266534 >> v1:172.21.15.64:6789/0 conn(0x7f24a005f3f0 legacy=0x7f24a01418c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:26.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.330+0000 7f24a5352700 1 -- 172.21.15.47:0/3377266534 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a00b2060 con 0x7f24a00b1320 2022-06-29T15:51:26.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.330+0000 7f24a5352700 1 -- 172.21.15.47:0/3377266534 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7f2494001660 con 0x7f24a00b1320 2022-06-29T15:51:26.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.330+0000 7f24a5352700 1 -- 172.21.15.47:0/3377266534 >> v1:172.21.15.47:6789/0 conn(0x7f24a00b1320 legacy=0x7f24a00b2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:26.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.330+0000 7f24a5352700 1 --2- 172.21.15.47:0/3377266534 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f248c003750 0x7f248c003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:26.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.330+0000 7f24a5352700 1 --2- 172.21.15.47:0/3377266534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f248c004500 0x7f248c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:26.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.330+0000 7f24a5352700 1 --2- 172.21.15.47:0/3377266534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f248c004e70 0x7f248c00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:26.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.330+0000 7f24a5352700 1 -- 172.21.15.47:0/3377266534 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248c00bcf0 con 0x7f248c004500 2022-06-29T15:51:26.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.330+0000 7f24a5352700 1 -- 172.21.15.47:0/3377266534 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248c00be60 con 0x7f248c004e70 2022-06-29T15:51:26.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.330+0000 7f24a5352700 1 -- 172.21.15.47:0/3377266534 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f248c00c020 con 0x7f248c003750 2022-06-29T15:51:26.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.334+0000 7f24a5b53700 1 --2- 172.21.15.47:0/3377266534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f248c004500 0x7f248c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:26.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.334+0000 7f24a6354700 1 --2- 172.21.15.47:0/3377266534 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f248c003750 0x7f248c003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:26.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.334+0000 7f24a6b55700 1 --2- 172.21.15.47:0/3377266534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f248c004e70 0x7f248c00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:26.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.334+0000 7f24a5b53700 1 -- 172.21.15.47:0/3377266534 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f248c003750 msgr2=0x7f248c003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:26.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.334+0000 7f24a5b53700 1 --2- 172.21.15.47:0/3377266534 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f248c003750 0x7f248c003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:26.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.334+0000 7f24a5b53700 1 -- 172.21.15.47:0/3377266534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f248c004e70 msgr2=0x7f248c00b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:26.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.334+0000 7f24a5b53700 1 --2- 172.21.15.47:0/3377266534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f248c004e70 0x7f248c00b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:26.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.334+0000 7f24a6b55700 1 -- 172.21.15.47:0/3377266534 reap_dead start 2022-06-29T15:51:26.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.334+0000 7f24a5b53700 1 -- 172.21.15.47:0/3377266534 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f248c003260 con 0x7f248c004500 2022-06-29T15:51:26.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.334+0000 7f24a5b53700 1 --2- 172.21.15.47:0/3377266534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f248c004500 0x7f248c004980 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 rx=0x7f2494001370 tx=0x7f24a0066180).ready entity=mon.0 client_cookie=29b29f44f5230704 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:26.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.334+0000 7f24a5352700 1 -- 172.21.15.47:0/3377266534 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24940070d0 con 0x7f248c004500 2022-06-29T15:51:26.352 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.334+0000 7f24a5352700 1 -- 172.21.15.47:0/3377266534 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2494007230 con 0x7f248c004500 2022-06-29T15:51:26.352 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.334+0000 7f24a5352700 1 -- 172.21.15.47:0/3377266534 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f249400c870 con 0x7f248c004500 2022-06-29T15:51:26.352 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.334+0000 7f24a7356700 1 -- 172.21.15.47:0/3377266534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f248c004500 msgr2=0x7f248c004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:26.352 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.334+0000 7f24a7356700 1 --2- 172.21.15.47:0/3377266534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f248c004500 0x7f248c004980 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 rx=0x7f2494001370 tx=0x7f24a0066180).stop 2022-06-29T15:51:26.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.334+0000 7f24a7356700 1 -- 172.21.15.47:0/3377266534 shutdown_connections 2022-06-29T15:51:26.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.334+0000 7f24a7356700 1 --2- 172.21.15.47:0/3377266534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f248c004500 0x7f248c004980 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:26.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.334+0000 7f24a7356700 1 -- 172.21.15.47:0/3377266534 >> 172.21.15.47:0/3377266534 conn(0x7f24a0062570 msgr2=0x7f24a00647d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:26.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.334+0000 7f24a7356700 1 -- 172.21.15.47:0/3377266534 shutdown_connections 2022-06-29T15:51:26.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.334+0000 7f24a7356700 1 -- 172.21.15.47:0/3377266534 wait complete. 2022-06-29T15:51:26.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.338+0000 7f24a7356700 1 Processor -- start 2022-06-29T15:51:26.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.338+0000 7f24a7356700 1 -- start start 2022-06-29T15:51:26.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.338+0000 7f24a7356700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f248c004500 0x7f24a0144bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:26.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.338+0000 7f24a7356700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f24a0146380 0x7f24a0146800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:26.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.338+0000 7f24a7356700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24a0155b60 0x7f24a0155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:26.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.338+0000 7f24a7356700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a0060580 con 0x7f248c004500 2022-06-29T15:51:26.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.338+0000 7f24a7356700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a00611a0 con 0x7f24a0155b60 2022-06-29T15:51:26.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.338+0000 7f24a7356700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24a0060f40 con 0x7f24a0146380 2022-06-29T15:51:26.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.338+0000 7f24a6354700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f248c004500 0x7f24a0144bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:26.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.338+0000 7f24a6354700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f248c004500 0x7f24a0144bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53698/0 (socket says 172.21.15.47:53698) 2022-06-29T15:51:26.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.338+0000 7f24a6354700 1 -- 172.21.15.47:0/1716861802 learned_addr learned my addr 172.21.15.47:0/1716861802 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:26.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.338+0000 7f24a5b53700 1 --2- 172.21.15.47:0/1716861802 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f24a0146380 0x7f24a0146800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:26.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.338+0000 7f24a6b55700 1 --2- 172.21.15.47:0/1716861802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24a0155b60 0x7f24a0155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:26.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.338+0000 7f24a6354700 1 -- 172.21.15.47:0/1716861802 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f24a0146380 msgr2=0x7f24a0146800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:26.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.338+0000 7f24a6354700 1 --2- 172.21.15.47:0/1716861802 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f24a0146380 0x7f24a0146800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:26.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.338+0000 7f24a6354700 1 -- 172.21.15.47:0/1716861802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24a0155b60 msgr2=0x7f24a0155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:26.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.338+0000 7f24a6354700 1 --2- 172.21.15.47:0/1716861802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24a0155b60 0x7f24a0155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:26.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.338+0000 7f24a6354700 1 -- 172.21.15.47:0/1716861802 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a0061760 con 0x7f248c004500 2022-06-29T15:51:26.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.338+0000 7f24a6354700 1 --2- 172.21.15.47:0/1716861802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f248c004500 0x7f24a0144bf0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 rx=0x7f249800d000 tx=0x7f24980079e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:26.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.338+0000 7f249f7fe700 1 -- 172.21.15.47:0/1716861802 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2498017070 con 0x7f248c004500 2022-06-29T15:51:26.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.338+0000 7f249f7fe700 1 -- 172.21.15.47:0/1716861802 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2498012750 con 0x7f248c004500 2022-06-29T15:51:26.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.338+0000 7f24a7356700 1 -- 172.21.15.47:0/1716861802 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24a01450e0 con 0x7f248c004500 2022-06-29T15:51:26.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.338+0000 7f249f7fe700 1 -- 172.21.15.47:0/1716861802 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2498012ab0 con 0x7f248c004500 2022-06-29T15:51:26.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.338+0000 7f24a7356700 1 -- 172.21.15.47:0/1716861802 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24a0145380 con 0x7f248c004500 2022-06-29T15:51:26.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.342+0000 7f24a7356700 1 -- 172.21.15.47:0/1716861802 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24a00618f0 con 0x7f248c004500 2022-06-29T15:51:26.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.342+0000 7f249f7fe700 1 -- 172.21.15.47:0/1716861802 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2498012c10 con 0x7f248c004500 2022-06-29T15:51:26.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.342+0000 7f249f7fe700 1 --2- 172.21.15.47:0/1716861802 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2480053b00 0x7f2480055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:26.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.342+0000 7f249f7fe700 1 -- 172.21.15.47:0/1716861802 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f249806c240 con 0x7f248c004500 2022-06-29T15:51:26.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.342+0000 7f24a5b53700 1 --2- 172.21.15.47:0/1716861802 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2480053b00 0x7f2480055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:26.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.342+0000 7f24a5b53700 1 --2- 172.21.15.47:0/1716861802 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2480053b00 0x7f2480055fc0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7f2494018000 tx=0x7f2494013000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:26.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.350+0000 7f249f7fe700 1 -- 172.21.15.47:0/1716861802 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2498043000 con 0x7f248c004500 2022-06-29T15:51:26.523 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.514+0000 7f24a7356700 1 -- 172.21.15.47:0/1716861802 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f24a00573d0 con 0x7f248c004500 2022-06-29T15:51:26.531 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.526+0000 7f249f7fe700 1 -- 172.21.15.47:0/1716861802 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v34) v1 ==== 184+0+0 (secure 0 0 0) 0x7f2498046620 con 0x7f248c004500 2022-06-29T15:51:26.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.526+0000 7f24a7356700 1 -- 172.21.15.47:0/1716861802 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2480053b00 msgr2=0x7f2480055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:26.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.526+0000 7f24a7356700 1 --2- 172.21.15.47:0/1716861802 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2480053b00 0x7f2480055fc0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7f2494018000 tx=0x7f2494013000).stop 2022-06-29T15:51:26.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.526+0000 7f24a7356700 1 -- 172.21.15.47:0/1716861802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f248c004500 msgr2=0x7f24a0144bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:26.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.526+0000 7f24a7356700 1 --2- 172.21.15.47:0/1716861802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f248c004500 0x7f24a0144bf0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 rx=0x7f249800d000 tx=0x7f24980079e0).stop 2022-06-29T15:51:26.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.526+0000 7f24a7356700 1 -- 172.21.15.47:0/1716861802 shutdown_connections 2022-06-29T15:51:26.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.526+0000 7f24a7356700 1 --2- 172.21.15.47:0/1716861802 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2480053b00 0x7f2480055fc0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:26.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.526+0000 7f24a7356700 1 --2- 172.21.15.47:0/1716861802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24a0155b60 0x7f24a0155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:26.536 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.526+0000 7f24a7356700 1 --2- 172.21.15.47:0/1716861802 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f24a0146380 0x7f24a0146800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:26.536 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.526+0000 7f24a7356700 1 --2- 172.21.15.47:0/1716861802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f248c004500 0x7f24a0144bf0 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:26.536 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.526+0000 7f24a7356700 1 -- 172.21.15.47:0/1716861802 >> 172.21.15.47:0/1716861802 conn(0x7f24a0062570 msgr2=0x7f24a005cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:26.536 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.526+0000 7f24a7356700 1 -- 172.21.15.47:0/1716861802 shutdown_connections 2022-06-29T15:51:26.537 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.526+0000 7f24a7356700 1 -- 172.21.15.47:0/1716861802 wait complete. 2022-06-29T15:51:26.538 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.1 2022-06-29T15:51:26.546 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:51:26.654 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.646+0000 7f0a1759e700 1 Processor -- start 2022-06-29T15:51:26.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.646+0000 7f0a1759e700 1 -- start start 2022-06-29T15:51:26.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.650+0000 7f0a1759e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0a18061da0 0x7f0a180b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:26.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.650+0000 7f0a1759e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0a1805f0b0 0x7f0a180b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:26.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.650+0000 7f0a1759e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a18062180 con 0x7f0a180630f0 2022-06-29T15:51:26.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.650+0000 7f0a1759e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a1805fae0 con 0x7f0a1805f0b0 2022-06-29T15:51:26.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.650+0000 7f0a1759e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a180b2ea0 con 0x7f0a18061da0 2022-06-29T15:51:26.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.650+0000 7f0a0ffff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0a180630f0 0x7f0a180b0e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39260/0 (socket says 172.21.15.47:39260) 2022-06-29T15:51:26.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.650+0000 7f0a0ffff700 1 -- 172.21.15.47:0/3439131661 learned_addr learned my addr 172.21.15.47:0/3439131661 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:26.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.650+0000 7f0a16d9d700 1 --2- 172.21.15.47:0/3439131661 >> v2:172.21.15.47:3300/0 conn(0x7f0a1805f0b0 0x7f0a180b17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:26.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.650+0000 7f0a1659c700 1 --2- 172.21.15.47:0/3439131661 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0a18061da0 0x7f0a180b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:26.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.650+0000 7f0a15d9b700 1 -- 172.21.15.47:0/3439131661 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 539513875 0 0) 0x7f0a18062180 con 0x7f0a180630f0 2022-06-29T15:51:26.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.650+0000 7f0a15d9b700 1 -- 172.21.15.47:0/3439131661 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09f8003260 con 0x7f0a180630f0 2022-06-29T15:51:26.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.650+0000 7f0a16d9d700 1 -- 172.21.15.47:0/3439131661 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0a18061da0 msgr2=0x7f0a180b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:26.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.650+0000 7f0a16d9d700 1 --2- 172.21.15.47:0/3439131661 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0a18061da0 0x7f0a180b3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:26.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.650+0000 7f0a16d9d700 1 -- 172.21.15.47:0/3439131661 >> v1:172.21.15.47:6789/0 conn(0x7f0a180630f0 legacy=0x7f0a180b0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:26.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.650+0000 7f0a16d9d700 1 -- 172.21.15.47:0/3439131661 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a180b3140 con 0x7f0a1805f0b0 2022-06-29T15:51:26.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.650+0000 7f0a16d9d700 1 --2- 172.21.15.47:0/3439131661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a1805f0b0 0x7f0a180b17c0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 rx=0x7f0a180b23f0 tx=0x7f0a100091d0).ready entity=mon.0 client_cookie=885a85d8bcaad17d server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:26.663 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.650+0000 7f0a15d9b700 1 -- 172.21.15.47:0/3439131661 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a1000b600 con 0x7f0a1805f0b0 2022-06-29T15:51:26.663 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.650+0000 7f0a15d9b700 1 -- 172.21.15.47:0/3439131661 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a1000b760 con 0x7f0a1805f0b0 2022-06-29T15:51:26.663 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.650+0000 7f0a15d9b700 1 -- 172.21.15.47:0/3439131661 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a1000b8d0 con 0x7f0a1805f0b0 2022-06-29T15:51:26.663 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.650+0000 7f0a1759e700 1 -- 172.21.15.47:0/3439131661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a1805f0b0 msgr2=0x7f0a180b17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:26.664 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.650+0000 7f0a1759e700 1 --2- 172.21.15.47:0/3439131661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a1805f0b0 0x7f0a180b17c0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 rx=0x7f0a180b23f0 tx=0x7f0a100091d0).stop 2022-06-29T15:51:26.664 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.650+0000 7f0a1759e700 1 -- 172.21.15.47:0/3439131661 shutdown_connections 2022-06-29T15:51:26.664 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.650+0000 7f0a1759e700 1 --2- 172.21.15.47:0/3439131661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a1805f0b0 0x7f0a180b17c0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:26.664 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.650+0000 7f0a1759e700 1 --2- 172.21.15.47:0/3439131661 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0a18061da0 0x7f0a180b3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:26.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.650+0000 7f0a1759e700 1 -- 172.21.15.47:0/3439131661 >> 172.21.15.47:0/3439131661 conn(0x7f0a18063670 msgr2=0x7f0a18063a80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:26.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.650+0000 7f0a1759e700 1 -- 172.21.15.47:0/3439131661 shutdown_connections 2022-06-29T15:51:26.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.650+0000 7f0a1759e700 1 -- 172.21.15.47:0/3439131661 wait complete. 2022-06-29T15:51:26.669 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.654+0000 7f0a1759e700 1 Processor -- start 2022-06-29T15:51:26.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.654+0000 7f0a1759e700 1 -- start start 2022-06-29T15:51:26.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.654+0000 7f0a1759e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a1805f0b0 0x7f0a180a2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:26.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.654+0000 7f0a1759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a18061da0 0x7f0a1815a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:26.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.654+0000 7f0a1759e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0a180630f0 0x7f0a180ac910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:26.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.654+0000 7f0a1759e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a180a60c0 con 0x7f0a1805f0b0 2022-06-29T15:51:26.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.654+0000 7f0a1759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a180a6360 con 0x7f0a18061da0 2022-06-29T15:51:26.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.654+0000 7f0a1759e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a180a6600 con 0x7f0a180630f0 2022-06-29T15:51:26.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.654+0000 7f0a1659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a1805f0b0 0x7f0a180a2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:26.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.654+0000 7f0a1659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a1805f0b0 0x7f0a180a2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53712/0 (socket says 172.21.15.47:53712) 2022-06-29T15:51:26.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.654+0000 7f0a1659c700 1 -- 172.21.15.47:0/3251781639 learned_addr learned my addr 172.21.15.47:0/3251781639 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:26.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.654+0000 7f0a16d9d700 1 --2- 172.21.15.47:0/3251781639 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0a180630f0 0x7f0a180ac910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:26.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.654+0000 7f0a0ffff700 1 --2- 172.21.15.47:0/3251781639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a18061da0 0x7f0a1815a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:26.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.654+0000 7f0a1659c700 1 -- 172.21.15.47:0/3251781639 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0a180630f0 msgr2=0x7f0a180ac910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:26.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.654+0000 7f0a1659c700 1 --2- 172.21.15.47:0/3251781639 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0a180630f0 0x7f0a180ac910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:26.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.654+0000 7f0a1659c700 1 -- 172.21.15.47:0/3251781639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a18061da0 msgr2=0x7f0a1815a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:26.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.654+0000 7f0a1659c700 1 --2- 172.21.15.47:0/3251781639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a18061da0 0x7f0a1815a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:26.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.654+0000 7f0a1659c700 1 -- 172.21.15.47:0/3251781639 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a10009470 con 0x7f0a1805f0b0 2022-06-29T15:51:26.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.654+0000 7f0a16d9d700 1 --2- 172.21.15.47:0/3251781639 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0a180630f0 0x7f0a180ac910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:51:26.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.654+0000 7f0a1659c700 1 --2- 172.21.15.47:0/3251781639 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a1805f0b0 0x7f0a180a2200 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 rx=0x7f0a0800c000 tx=0x7f0a08006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:26.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.654+0000 7f0a0f7fe700 1 -- 172.21.15.47:0/3251781639 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a08010780 con 0x7f0a1805f0b0 2022-06-29T15:51:26.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.654+0000 7f0a0f7fe700 1 -- 172.21.15.47:0/3251781639 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a080108e0 con 0x7f0a1805f0b0 2022-06-29T15:51:26.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.654+0000 7f0a1759e700 1 -- 172.21.15.47:0/3251781639 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a1809fa60 con 0x7f0a1805f0b0 2022-06-29T15:51:26.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.654+0000 7f0a0f7fe700 1 -- 172.21.15.47:0/3251781639 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a08010c40 con 0x7f0a1805f0b0 2022-06-29T15:51:26.676 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.654+0000 7f0a1759e700 1 -- 172.21.15.47:0/3251781639 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a180a5bb0 con 0x7f0a1805f0b0 2022-06-29T15:51:26.676 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.658+0000 7f0a1759e700 1 -- 172.21.15.47:0/3251781639 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a180a5e90 con 0x7f0a1805f0b0 2022-06-29T15:51:26.676 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.658+0000 7f0a0f7fe700 1 -- 172.21.15.47:0/3251781639 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0a0801c460 con 0x7f0a1805f0b0 2022-06-29T15:51:26.676 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.658+0000 7f0a0f7fe700 1 --2- 172.21.15.47:0/3251781639 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f09fc0539e0 0x7f09fc055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:26.677 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.658+0000 7f0a0f7fe700 1 -- 172.21.15.47:0/3251781639 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f0a0806c630 con 0x7f0a1805f0b0 2022-06-29T15:51:26.677 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.658+0000 7f0a0ffff700 1 --2- 172.21.15.47:0/3251781639 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f09fc0539e0 0x7f09fc055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:26.677 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.658+0000 7f0a0ffff700 1 --2- 172.21.15.47:0/3251781639 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f09fc0539e0 0x7f09fc055ea0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7f0a0000d000 tx=0x7f0a00009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:26.678 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.666+0000 7f0a0f7fe700 1 -- 172.21.15.47:0/3251781639 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0a08043200 con 0x7f0a1805f0b0 2022-06-29T15:51:26.836 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.830+0000 7f0a1759e700 1 -- 172.21.15.47:0/3251781639 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a1809fbf0 con 0x7f0a1805f0b0 2022-06-29T15:51:26.837 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.830+0000 7f0a0f7fe700 1 -- 172.21.15.47:0/3251781639 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+2492 (secure 0 0 0) 0x7f0a08046820 con 0x7f0a1805f0b0 2022-06-29T15:51:26.838 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:26.838 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":69,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7957,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.64:6833/298405638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":298405638},{"type":"v1","addr":"172.21.15.64:6833","nonce":298405638}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":66},{"gid":8037,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/191070564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":191070564},{"type":"v1","addr":"172.21.15.47:6835","nonce":191070564}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:51:23.543892+0000","modified":"2022-06-29T15:51:25.309120+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":56894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_56894":{"gid":56894,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6837/1370488030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1370488030},{"type":"v1","addr":"172.21.15.47:6837","nonce":1370488030}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2022-06-29T15:51:26.841 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.834+0000 7f0a1759e700 1 -- 172.21.15.47:0/3251781639 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f09fc0539e0 msgr2=0x7f09fc055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:26.842 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.834+0000 7f0a1759e700 1 --2- 172.21.15.47:0/3251781639 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f09fc0539e0 0x7f09fc055ea0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7f0a0000d000 tx=0x7f0a00009040).stop 2022-06-29T15:51:26.842 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.834+0000 7f0a1759e700 1 -- 172.21.15.47:0/3251781639 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a1805f0b0 msgr2=0x7f0a180a2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:26.842 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.834+0000 7f0a1759e700 1 --2- 172.21.15.47:0/3251781639 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a1805f0b0 0x7f0a180a2200 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 rx=0x7f0a0800c000 tx=0x7f0a08006df0).stop 2022-06-29T15:51:26.843 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.834+0000 7f0a1759e700 1 -- 172.21.15.47:0/3251781639 shutdown_connections 2022-06-29T15:51:26.843 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.834+0000 7f0a1759e700 1 --2- 172.21.15.47:0/3251781639 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0a180630f0 0x7f0a180ac910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:26.843 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.834+0000 7f0a1759e700 1 --2- 172.21.15.47:0/3251781639 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f09fc0539e0 0x7f09fc055ea0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:26.843 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.834+0000 7f0a1759e700 1 --2- 172.21.15.47:0/3251781639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a18061da0 0x7f0a1815a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:26.844 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.834+0000 7f0a1759e700 1 --2- 172.21.15.47:0/3251781639 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a1805f0b0 0x7f0a180a2200 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:26.844 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.834+0000 7f0a1759e700 1 -- 172.21.15.47:0/3251781639 >> 172.21.15.47:0/3251781639 conn(0x7f0a18063670 msgr2=0x7f0a180b1590 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:26.844 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.834+0000 7f0a1759e700 1 -- 172.21.15.47:0/3251781639 shutdown_connections 2022-06-29T15:51:26.844 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:26.834+0000 7f0a1759e700 1 -- 172.21.15.47:0/3251781639 wait complete. 2022-06-29T15:51:26.845 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 69 2022-06-29T15:51:27.160 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 69, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-29T15:51:23.543892+0000', 'modified': '2022-06-29T15:51:25.309120+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 56894}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_56894': {'gid': 56894, 'name': 'c', 'rank': 0, 'incarnation': 67, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.47:6837/1370488030', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6836', 'nonce': 1370488030}, {'type': 'v1', 'addr': '172.21.15.47:6837', 'nonce': 1370488030}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-29T15:51:27.161 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-29T15:51:27.162 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2022-06-29T15:51:27.282 INFO:teuthology.orchestra.run.smithi047.stdout:{"cluster_fsid":"9a68894a-53ce-400e-838b-14eab91b834c","whoami":0,"id":56894,"want_state":"up:active","state":"up:active","rank_uptime":3.7204746850000001,"mdsmap_epoch":69,"osdmap_epoch":140,"osdmap_epoch_barrier":140,"uptime":8.5251651969999998} 2022-06-29T15:51:27.282 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9a68894a-53ce-400e-838b-14eab91b834c", "id": 56894, "mdsmap_epoch": 69, "osdmap_epoch": 140, "osdmap_epoch_barrier": 140, "rank_uptime": 3.720474685, "state": "up:active", "uptime": 8.525165197, "want_state": "up:active", "whoami": 0 } 2022-06-29T15:51:27.283 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:51:27.385 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.378+0000 7fed99c50700 1 Processor -- start 2022-06-29T15:51:27.385 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.378+0000 7fed99c50700 1 -- start start 2022-06-29T15:51:27.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.378+0000 7fed99c50700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fed940af4a0 0x7fed9405f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:27.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.378+0000 7fed99c50700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fed9405f0b0 0x7fed940b0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:27.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.378+0000 7fed99c50700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed940afe20 con 0x7fed940ac180 2022-06-29T15:51:27.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.378+0000 7fed99c50700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed940b2830 con 0x7fed9405f0b0 2022-06-29T15:51:27.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.378+0000 7fed99c50700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed94143e30 con 0x7fed940af4a0 2022-06-29T15:51:27.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.378+0000 7fed93fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fed940ac180 0x7fed940b2120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39278/0 (socket says 172.21.15.47:39278) 2022-06-29T15:51:27.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.378+0000 7fed93fff700 1 -- 172.21.15.47:0/3036791111 learned_addr learned my addr 172.21.15.47:0/3036791111 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:27.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.378+0000 7fed92ffd700 1 --2- 172.21.15.47:0/3036791111 >> v2:172.21.15.64:3300/0 conn(0x7fed9405f0b0 0x7fed940b0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:27.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.378+0000 7fed937fe700 1 --2- 172.21.15.47:0/3036791111 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fed940af4a0 0x7fed9405f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:27.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.378+0000 7fed927fc700 1 -- 172.21.15.47:0/3036791111 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2283185334 0 0) 0x7fed940afe20 con 0x7fed940ac180 2022-06-29T15:51:27.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.378+0000 7fed927fc700 1 -- 172.21.15.47:0/3036791111 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed80003260 con 0x7fed940ac180 2022-06-29T15:51:27.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.378+0000 7fed92ffd700 1 -- 172.21.15.47:0/3036791111 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fed940af4a0 msgr2=0x7fed9405f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:27.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.378+0000 7fed92ffd700 1 --2- 172.21.15.47:0/3036791111 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fed940af4a0 0x7fed9405f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:27.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.378+0000 7fed92ffd700 1 -- 172.21.15.47:0/3036791111 >> v1:172.21.15.47:6789/0 conn(0x7fed940ac180 legacy=0x7fed940b2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:27.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.378+0000 7fed92ffd700 1 -- 172.21.15.47:0/3036791111 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed940b0050 con 0x7fed9405f0b0 2022-06-29T15:51:27.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.378+0000 7fed92ffd700 1 --2- 172.21.15.47:0/3036791111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed9405f0b0 0x7fed940b0790 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7fed84002f70 tx=0x7fed84006df0).ready entity=mon.1 client_cookie=5c3de69bd6e7e24f server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:27.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.382+0000 7fed927fc700 1 -- 172.21.15.47:0/3036791111 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed8400d4a0 con 0x7fed9405f0b0 2022-06-29T15:51:27.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.382+0000 7fed927fc700 1 -- 172.21.15.47:0/3036791111 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed84000e90 con 0x7fed9405f0b0 2022-06-29T15:51:27.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.382+0000 7fed927fc700 1 -- 172.21.15.47:0/3036791111 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed8400d7f0 con 0x7fed9405f0b0 2022-06-29T15:51:27.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.382+0000 7fed99c50700 1 -- 172.21.15.47:0/3036791111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed9405f0b0 msgr2=0x7fed940b0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:27.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.382+0000 7fed99c50700 1 --2- 172.21.15.47:0/3036791111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed9405f0b0 0x7fed940b0790 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7fed84002f70 tx=0x7fed84006df0).stop 2022-06-29T15:51:27.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.382+0000 7fed99c50700 1 -- 172.21.15.47:0/3036791111 shutdown_connections 2022-06-29T15:51:27.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.382+0000 7fed99c50700 1 --2- 172.21.15.47:0/3036791111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed9405f0b0 0x7fed940b0790 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:27.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.382+0000 7fed99c50700 1 --2- 172.21.15.47:0/3036791111 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fed940af4a0 0x7fed9405f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:27.394 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.382+0000 7fed99c50700 1 -- 172.21.15.47:0/3036791111 >> 172.21.15.47:0/3036791111 conn(0x7fed940626f0 msgr2=0x7fed94062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:27.394 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.382+0000 7fed99c50700 1 -- 172.21.15.47:0/3036791111 shutdown_connections 2022-06-29T15:51:27.394 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.382+0000 7fed99c50700 1 -- 172.21.15.47:0/3036791111 wait complete. 2022-06-29T15:51:27.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.382+0000 7fed99c50700 1 Processor -- start 2022-06-29T15:51:27.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.382+0000 7fed99c50700 1 -- start start 2022-06-29T15:51:27.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.382+0000 7fed99c50700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed9405f0b0 0x7fed9413a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:27.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.382+0000 7fed99c50700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fed940ac180 0x7fed94133b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:27.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.382+0000 7fed99c50700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed940af4a0 0x7fed94134040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:27.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.382+0000 7fed99c50700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed940b2e20 con 0x7fed940af4a0 2022-06-29T15:51:27.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.382+0000 7fed99c50700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed9406b700 con 0x7fed9405f0b0 2022-06-29T15:51:27.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.382+0000 7fed99c50700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed940b3f90 con 0x7fed940ac180 2022-06-29T15:51:27.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.382+0000 7fed93fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed940af4a0 0x7fed94134040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:27.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.382+0000 7fed93fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed940af4a0 0x7fed94134040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53730/0 (socket says 172.21.15.47:53730) 2022-06-29T15:51:27.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.382+0000 7fed93fff700 1 -- 172.21.15.47:0/2575743617 learned_addr learned my addr 172.21.15.47:0/2575743617 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:27.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.382+0000 7fed92ffd700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fed940ac180 0x7fed94133b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:27.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.382+0000 7fed937fe700 1 --2- 172.21.15.47:0/2575743617 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed9405f0b0 0x7fed9413a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:27.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.382+0000 7fed93fff700 1 -- 172.21.15.47:0/2575743617 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fed940ac180 msgr2=0x7fed94133b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:27.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.382+0000 7fed93fff700 1 --2- 172.21.15.47:0/2575743617 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fed940ac180 0x7fed94133b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:27.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.382+0000 7fed93fff700 1 -- 172.21.15.47:0/2575743617 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed9405f0b0 msgr2=0x7fed9413a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:27.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.382+0000 7fed93fff700 1 --2- 172.21.15.47:0/2575743617 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed9405f0b0 0x7fed9413a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:27.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.382+0000 7fed93fff700 1 -- 172.21.15.47:0/2575743617 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed8400b180 con 0x7fed940af4a0 2022-06-29T15:51:27.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.382+0000 7fed93fff700 1 --2- 172.21.15.47:0/2575743617 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed940af4a0 0x7fed94134040 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 rx=0x7fed8c001490 tx=0x7fed8c007da0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:27.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.386+0000 7fed90ff9700 1 -- 172.21.15.47:0/2575743617 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed8c005c70 con 0x7fed940af4a0 2022-06-29T15:51:27.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.386+0000 7fed90ff9700 1 -- 172.21.15.47:0/2575743617 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed8c005dd0 con 0x7fed940af4a0 2022-06-29T15:51:27.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.386+0000 7fed99c50700 1 -- 172.21.15.47:0/2575743617 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed940b3940 con 0x7fed940af4a0 2022-06-29T15:51:27.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.386+0000 7fed90ff9700 1 -- 172.21.15.47:0/2575743617 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed8c0072c0 con 0x7fed940af4a0 2022-06-29T15:51:27.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.386+0000 7fed99c50700 1 -- 172.21.15.47:0/2575743617 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed940b36a0 con 0x7fed940af4a0 2022-06-29T15:51:27.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.386+0000 7fed99c50700 1 -- 172.21.15.47:0/2575743617 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed9405fdf0 con 0x7fed940af4a0 2022-06-29T15:51:27.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.386+0000 7fed90ff9700 1 -- 172.21.15.47:0/2575743617 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fed8c010070 con 0x7fed940af4a0 2022-06-29T15:51:27.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.386+0000 7fed90ff9700 1 --2- 172.21.15.47:0/2575743617 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fed74053a70 0x7fed74055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:27.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.386+0000 7fed90ff9700 1 -- 172.21.15.47:0/2575743617 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fed8c06b940 con 0x7fed940af4a0 2022-06-29T15:51:27.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.386+0000 7fed937fe700 1 --2- 172.21.15.47:0/2575743617 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fed74053a70 0x7fed74055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:27.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.390+0000 7fed937fe700 1 --2- 172.21.15.47:0/2575743617 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fed74053a70 0x7fed74055f30 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7fed8800b000 tx=0x7fed88006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:27.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.394+0000 7fed90ff9700 1 -- 172.21.15.47:0/2575743617 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fed8c042590 con 0x7fed940af4a0 2022-06-29T15:51:27.532 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.526+0000 7fed99c50700 1 -- 172.21.15.47:0/2575743617 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fed940573d0 con 0x7fed940af4a0 2022-06-29T15:51:27.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.526+0000 7fed90ff9700 1 -- 172.21.15.47:0/2575743617 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) v1 ==== 74+0+16285 (secure 0 0 0) 0x7fed8c045bb0 con 0x7fed940af4a0 2022-06-29T15:51:27.535 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:27.535 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":140,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:51:24.546533+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:51:21.506960+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2022-06-29T15:51:22.546983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:51:27.537 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.530+0000 7fed99c50700 1 -- 172.21.15.47:0/2575743617 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fed74053a70 msgr2=0x7fed74055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:27.537 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.530+0000 7fed99c50700 1 --2- 172.21.15.47:0/2575743617 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fed74053a70 0x7fed74055f30 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7fed8800b000 tx=0x7fed88006df0).stop 2022-06-29T15:51:27.538 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.530+0000 7fed99c50700 1 -- 172.21.15.47:0/2575743617 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed940af4a0 msgr2=0x7fed94134040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:27.538 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.530+0000 7fed99c50700 1 --2- 172.21.15.47:0/2575743617 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed940af4a0 0x7fed94134040 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 rx=0x7fed8c001490 tx=0x7fed8c007da0).stop 2022-06-29T15:51:27.539 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.530+0000 7fed99c50700 1 -- 172.21.15.47:0/2575743617 shutdown_connections 2022-06-29T15:51:27.539 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.530+0000 7fed99c50700 1 --2- 172.21.15.47:0/2575743617 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed940af4a0 0x7fed94134040 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:27.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.530+0000 7fed99c50700 1 --2- 172.21.15.47:0/2575743617 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fed940ac180 0x7fed94133b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:27.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.530+0000 7fed99c50700 1 --2- 172.21.15.47:0/2575743617 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fed74053a70 0x7fed74055f30 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:27.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.530+0000 7fed99c50700 1 --2- 172.21.15.47:0/2575743617 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed9405f0b0 0x7fed9413a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:27.541 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.530+0000 7fed99c50700 1 -- 172.21.15.47:0/2575743617 >> 172.21.15.47:0/2575743617 conn(0x7fed940626f0 msgr2=0x7fed940b20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:27.541 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.530+0000 7fed99c50700 1 -- 172.21.15.47:0/2575743617 shutdown_connections 2022-06-29T15:51:27.541 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.530+0000 7fed99c50700 1 -- 172.21.15.47:0/2575743617 wait complete. 2022-06-29T15:51:27.549 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-29T15:42:16.971431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-29T15:42:25.290884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-29T15:51:21.506960+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '139', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2022-06-29T15:51:22.546983+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '140', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-29T15:51:27.550 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-29T15:51:27.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.654+0000 7f497a0ab700 1 Processor -- start 2022-06-29T15:51:27.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.654+0000 7f497a0ab700 1 -- start start 2022-06-29T15:51:27.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.658+0000 7f497a0ab700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f49740b2580 0x7f4974140fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:27.666 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.658+0000 7f497a0ab700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f49740b2a50 0x7f4974144620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:27.666 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.658+0000 7f497a0ab700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4974066620 con 0x7f4974064870 2022-06-29T15:51:27.666 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.658+0000 7f497a0ab700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4974145340 con 0x7f49740b2a50 2022-06-29T15:51:27.666 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.658+0000 7f497a0ab700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4974066850 con 0x7f49740b2580 2022-06-29T15:51:27.667 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.658+0000 7f49798aa700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f49740b2a50 0x7f4974144620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:27.667 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.658+0000 7f49798aa700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f49740b2a50 0x7f4974144620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53738/0 (socket says 172.21.15.47:53738) 2022-06-29T15:51:27.667 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.658+0000 7f49798aa700 1 -- 172.21.15.47:0/2625990966 learned_addr learned my addr 172.21.15.47:0/2625990966 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:27.667 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.658+0000 7f49788a8700 1 --2- 172.21.15.47:0/2625990966 >> v2:172.21.15.64:3300/0 conn(0x7f49740b2580 0x7f4974140fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:27.667 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.658+0000 7f49798aa700 1 -- 172.21.15.47:0/2625990966 >> v2:172.21.15.64:3300/0 conn(0x7f49740b2580 msgr2=0x7f4974140fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:27.668 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.658+0000 7f49798aa700 1 --2- 172.21.15.47:0/2625990966 >> v2:172.21.15.64:3300/0 conn(0x7f49740b2580 0x7f4974140fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:27.668 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.658+0000 7f49798aa700 1 -- 172.21.15.47:0/2625990966 >> v1:172.21.15.64:6789/0 conn(0x7f4974064870 legacy=0x7f4974066f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:27.668 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.658+0000 7f49798aa700 1 -- 172.21.15.47:0/2625990966 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4974066af0 con 0x7f49740b2a50 2022-06-29T15:51:27.668 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.658+0000 7f49788a8700 1 --2- 172.21.15.47:0/2625990966 >> v2:172.21.15.64:3300/0 conn(0x7f49740b2580 0x7f4974140fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:51:27.669 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.658+0000 7f49798aa700 1 --2- 172.21.15.47:0/2625990966 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49740b2a50 0x7f4974144620 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 rx=0x7f496c0032f0 tx=0x7f496c009290).ready entity=mon.0 client_cookie=c7c64e2db9573f0f server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:27.669 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.658+0000 7f495ffff700 1 -- 172.21.15.47:0/2625990966 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f496c00b870 con 0x7f49740b2a50 2022-06-29T15:51:27.669 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.658+0000 7f495ffff700 1 -- 172.21.15.47:0/2625990966 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f496c00b9d0 con 0x7f49740b2a50 2022-06-29T15:51:27.669 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.658+0000 7f495ffff700 1 -- 172.21.15.47:0/2625990966 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f496c00bb40 con 0x7f49740b2a50 2022-06-29T15:51:27.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.658+0000 7f497a0ab700 1 -- 172.21.15.47:0/2625990966 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49740b2a50 msgr2=0x7f4974144620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:27.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.658+0000 7f497a0ab700 1 --2- 172.21.15.47:0/2625990966 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49740b2a50 0x7f4974144620 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 rx=0x7f496c0032f0 tx=0x7f496c009290).stop 2022-06-29T15:51:27.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.658+0000 7f497a0ab700 1 -- 172.21.15.47:0/2625990966 shutdown_connections 2022-06-29T15:51:27.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.658+0000 7f497a0ab700 1 --2- 172.21.15.47:0/2625990966 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49740b2a50 0x7f4974144620 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:27.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.658+0000 7f497a0ab700 1 --2- 172.21.15.47:0/2625990966 >> v2:172.21.15.64:3300/0 conn(0x7f49740b2580 0x7f4974140fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:27.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.658+0000 7f497a0ab700 1 -- 172.21.15.47:0/2625990966 >> 172.21.15.47:0/2625990966 conn(0x7f497405c510 msgr2=0x7f497405c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:27.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.658+0000 7f497a0ab700 1 -- 172.21.15.47:0/2625990966 shutdown_connections 2022-06-29T15:51:27.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.658+0000 7f497a0ab700 1 -- 172.21.15.47:0/2625990966 wait complete. 2022-06-29T15:51:27.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.662+0000 7f497a0ab700 1 Processor -- start 2022-06-29T15:51:27.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.662+0000 7f497a0ab700 1 -- start start 2022-06-29T15:51:27.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.662+0000 7f497a0ab700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4974064870 0x7f4974140ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:27.676 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.662+0000 7f497a0ab700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49740b2a50 0x7f49741446b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:27.676 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.662+0000 7f497a0ab700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4974154340 0x7f4974150450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:27.676 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.662+0000 7f497a0ab700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49740ad790 con 0x7f49740b2a50 2022-06-29T15:51:27.676 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.662+0000 7f497a0ab700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4974141650 con 0x7f4974154340 2022-06-29T15:51:27.677 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.662+0000 7f497a0ab700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4974154850 con 0x7f4974064870 2022-06-29T15:51:27.677 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.662+0000 7f49788a8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49740b2a50 0x7f49741446b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:27.677 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.662+0000 7f49798aa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4974154340 0x7f4974150450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:27.677 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.662+0000 7f49788a8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49740b2a50 0x7f49741446b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53742/0 (socket says 172.21.15.47:53742) 2022-06-29T15:51:27.677 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.662+0000 7f49788a8700 1 -- 172.21.15.47:0/1457841992 learned_addr learned my addr 172.21.15.47:0/1457841992 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:27.678 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.662+0000 7f49790a9700 1 --2- 172.21.15.47:0/1457841992 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4974064870 0x7f4974140ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:27.678 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.662+0000 7f49788a8700 1 -- 172.21.15.47:0/1457841992 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4974064870 msgr2=0x7f4974140ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:27.678 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.662+0000 7f49788a8700 1 --2- 172.21.15.47:0/1457841992 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4974064870 0x7f4974140ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:27.678 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.662+0000 7f49788a8700 1 -- 172.21.15.47:0/1457841992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4974154340 msgr2=0x7f4974150450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:27.679 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.662+0000 7f49788a8700 1 --2- 172.21.15.47:0/1457841992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4974154340 0x7f4974150450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:27.679 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.662+0000 7f49788a8700 1 -- 172.21.15.47:0/1457841992 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f496c009530 con 0x7f49740b2a50 2022-06-29T15:51:27.679 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.662+0000 7f49798aa700 1 --2- 172.21.15.47:0/1457841992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4974154340 0x7f4974150450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:51:27.679 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.662+0000 7f49788a8700 1 --2- 172.21.15.47:0/1457841992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49740b2a50 0x7f49741446b0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 rx=0x7f49740adf10 tx=0x7f4968008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:27.680 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.666+0000 7f4966ffd700 1 -- 172.21.15.47:0/1457841992 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4968011780 con 0x7f49740b2a50 2022-06-29T15:51:27.680 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.666+0000 7f4966ffd700 1 -- 172.21.15.47:0/1457841992 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49680118e0 con 0x7f49740b2a50 2022-06-29T15:51:27.680 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.666+0000 7f497a0ab700 1 -- 172.21.15.47:0/1457841992 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49741418b0 con 0x7f49740b2a50 2022-06-29T15:51:27.680 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.666+0000 7f4966ffd700 1 -- 172.21.15.47:0/1457841992 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4968011c40 con 0x7f49740b2a50 2022-06-29T15:51:27.680 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.666+0000 7f497a0ab700 1 -- 172.21.15.47:0/1457841992 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4974144ba0 con 0x7f49740b2a50 2022-06-29T15:51:27.681 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.666+0000 7f497a0ab700 1 -- 172.21.15.47:0/1457841992 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4974069d00 con 0x7f49740b2a50 2022-06-29T15:51:27.681 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.666+0000 7f4966ffd700 1 -- 172.21.15.47:0/1457841992 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f496801c460 con 0x7f49740b2a50 2022-06-29T15:51:27.681 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.666+0000 7f4966ffd700 1 --2- 172.21.15.47:0/1457841992 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4950053aa0 0x7f4950055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:27.681 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.666+0000 7f4966ffd700 1 -- 172.21.15.47:0/1457841992 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f496806c380 con 0x7f49740b2a50 2022-06-29T15:51:27.682 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.666+0000 7f49790a9700 1 --2- 172.21.15.47:0/1457841992 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4950053aa0 0x7f4950055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:27.682 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.670+0000 7f49790a9700 1 --2- 172.21.15.47:0/1457841992 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4950053aa0 0x7f4950055f60 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7f497000d000 tx=0x7f4970000bc0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:27.683 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.674+0000 7f4966ffd700 1 -- 172.21.15.47:0/1457841992 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4974069d00 con 0x7f49740b2a50 2022-06-29T15:51:27.809 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.802+0000 7f497a0ab700 1 -- 172.21.15.47:0/1457841992 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f4974069d00 con 0x7f49740b2a50 2022-06-29T15:51:27.809 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.802+0000 7f4966ffd700 1 -- 172.21.15.47:0/1457841992 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v140) v1 ==== 110+0+10 (secure 0 0 0) 0x7f4974069d00 con 0x7f49740b2a50 2022-06-29T15:51:27.810 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2022-06-29T15:51:27.812 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.806+0000 7f497a0ab700 1 -- 172.21.15.47:0/1457841992 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4950053aa0 msgr2=0x7f4950055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:27.812 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.806+0000 7f497a0ab700 1 --2- 172.21.15.47:0/1457841992 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4950053aa0 0x7f4950055f60 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7f497000d000 tx=0x7f4970000bc0).stop 2022-06-29T15:51:27.812 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.806+0000 7f497a0ab700 1 -- 172.21.15.47:0/1457841992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49740b2a50 msgr2=0x7f49741446b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:27.812 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.806+0000 7f497a0ab700 1 --2- 172.21.15.47:0/1457841992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49740b2a50 0x7f49741446b0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 rx=0x7f49740adf10 tx=0x7f4968008040).stop 2022-06-29T15:51:27.813 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.806+0000 7f497a0ab700 1 -- 172.21.15.47:0/1457841992 shutdown_connections 2022-06-29T15:51:27.813 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.806+0000 7f497a0ab700 1 --2- 172.21.15.47:0/1457841992 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4950053aa0 0x7f4950055f60 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:27.813 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.806+0000 7f497a0ab700 1 --2- 172.21.15.47:0/1457841992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4974154340 0x7f4974150450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:27.813 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.806+0000 7f497a0ab700 1 --2- 172.21.15.47:0/1457841992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49740b2a50 0x7f49741446b0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:27.814 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.806+0000 7f497a0ab700 1 --2- 172.21.15.47:0/1457841992 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4974064870 0x7f4974140ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:27.814 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.806+0000 7f497a0ab700 1 -- 172.21.15.47:0/1457841992 >> 172.21.15.47:0/1457841992 conn(0x7f497405c510 msgr2=0x7f4974136cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:27.814 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.806+0000 7f497a0ab700 1 -- 172.21.15.47:0/1457841992 shutdown_connections 2022-06-29T15:51:27.814 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.806+0000 7f497a0ab700 1 -- 172.21.15.47:0/1457841992 wait complete. 2022-06-29T15:51:27.822 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-29T15:51:27.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.926+0000 7f2ff6932700 1 Processor -- start 2022-06-29T15:51:27.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.926+0000 7f2ff6932700 1 -- start start 2022-06-29T15:51:27.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.926+0000 7f2ff6932700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2ff00b4300 0x7f2ff00612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:27.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.926+0000 7f2ff6932700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ff0067610 con 0x7f2ff01471a0 2022-06-29T15:51:27.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.926+0000 7f2ff6932700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ff01467c0 con 0x7f2ff0060ee0 2022-06-29T15:51:27.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.926+0000 7f2ff6932700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff01475d0 con 0x7f2ff00b4300 2022-06-29T15:51:27.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.926+0000 7f2ff4ecf700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f2ff01471a0 0x7f2ff013cdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39304/0 (socket says 172.21.15.47:39304) 2022-06-29T15:51:27.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.926+0000 7f2ff4ecf700 1 -- 172.21.15.47:0/3338848453 learned_addr learned my addr 172.21.15.47:0/3338848453 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:27.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.926+0000 7f2fef7fe700 1 --2- 172.21.15.47:0/3338848453 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2ff00b4300 0x7f2ff00612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:27.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.926+0000 7f2feeffd700 1 -- 172.21.15.47:0/3338848453 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2809565449 0 0) 0x7f2ff0067610 con 0x7f2ff01471a0 2022-06-29T15:51:27.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.926+0000 7f2feeffd700 1 -- 172.21.15.47:0/3338848453 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fd0003260 con 0x7f2ff01471a0 2022-06-29T15:51:27.938 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.926+0000 7f2feeffd700 1 -- 172.21.15.47:0/3338848453 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2146846286 0 0) 0x7f2ff01467c0 con 0x7f2ff0060ee0 2022-06-29T15:51:27.938 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.926+0000 7f2feeffd700 1 -- 172.21.15.47:0/3338848453 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ff0067610 con 0x7f2ff0060ee0 2022-06-29T15:51:27.938 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.926+0000 7f2fef7fe700 1 -- 172.21.15.47:0/3338848453 >> v1:172.21.15.64:6789/0 conn(0x7f2ff0060ee0 legacy=0x7f2ff01430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:27.938 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.926+0000 7f2fef7fe700 1 -- 172.21.15.47:0/3338848453 >> v1:172.21.15.47:6789/0 conn(0x7f2ff01471a0 legacy=0x7f2ff013cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:27.939 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.926+0000 7f2fef7fe700 1 -- 172.21.15.47:0/3338848453 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff0146a60 con 0x7f2ff00b4300 2022-06-29T15:51:27.939 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.926+0000 7f2fef7fe700 1 --2- 172.21.15.47:0/3338848453 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2ff00b4300 0x7f2ff00612c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f2fe0002f70 tx=0x7f2fe0006df0).ready entity=mon.2 client_cookie=99010dca445f0486 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:27.939 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.930+0000 7f2feeffd700 1 -- 172.21.15.47:0/3338848453 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2fe000d5c0 con 0x7f2ff00b4300 2022-06-29T15:51:27.939 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.930+0000 7f2feeffd700 1 -- 172.21.15.47:0/3338848453 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe0000e90 con 0x7f2ff00b4300 2022-06-29T15:51:27.940 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.930+0000 7f2feeffd700 1 -- 172.21.15.47:0/3338848453 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2fe000d910 con 0x7f2ff00b4300 2022-06-29T15:51:27.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.930+0000 7f2ff6932700 1 -- 172.21.15.47:0/3338848453 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2ff00b4300 msgr2=0x7f2ff00612c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:27.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.930+0000 7f2ff6932700 1 --2- 172.21.15.47:0/3338848453 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2ff00b4300 0x7f2ff00612c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f2fe0002f70 tx=0x7f2fe0006df0).stop 2022-06-29T15:51:27.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.930+0000 7f2ff6932700 1 -- 172.21.15.47:0/3338848453 shutdown_connections 2022-06-29T15:51:27.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.930+0000 7f2ff6932700 1 --2- 172.21.15.47:0/3338848453 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2ff00b4300 0x7f2ff00612c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:27.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.930+0000 7f2ff6932700 1 -- 172.21.15.47:0/3338848453 >> 172.21.15.47:0/3338848453 conn(0x7f2ff005c510 msgr2=0x7f2ff005f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:27.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.930+0000 7f2ff6932700 1 -- 172.21.15.47:0/3338848453 shutdown_connections 2022-06-29T15:51:27.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.930+0000 7f2ff6932700 1 -- 172.21.15.47:0/3338848453 wait complete. 2022-06-29T15:51:27.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.930+0000 7f2ff6932700 1 Processor -- start 2022-06-29T15:51:27.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.930+0000 7f2ff6932700 1 -- start start 2022-06-29T15:51:27.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.934+0000 7f2ff6932700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2ff0060ee0 0x7f2ff0149d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:27.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.934+0000 7f2ff6932700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ff00b4300 0x7f2ff01524e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:27.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.934+0000 7f2ff6932700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ff01471a0 0x7f2ff01529d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:27.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.934+0000 7f2ff6932700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff00b3200 con 0x7f2ff00b4300 2022-06-29T15:51:27.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.934+0000 7f2ff6932700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff00b3430 con 0x7f2ff01471a0 2022-06-29T15:51:27.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.934+0000 7f2ff6932700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff00b36d0 con 0x7f2ff0060ee0 2022-06-29T15:51:27.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.934+0000 7f2fef7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ff00b4300 0x7f2ff01524e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:27.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.934+0000 7f2fef7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ff00b4300 0x7f2ff01524e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53756/0 (socket says 172.21.15.47:53756) 2022-06-29T15:51:27.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.934+0000 7f2fef7fe700 1 -- 172.21.15.47:0/1145467850 learned_addr learned my addr 172.21.15.47:0/1145467850 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:27.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.934+0000 7f2feffff700 1 --2- 172.21.15.47:0/1145467850 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2ff0060ee0 0x7f2ff0149d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:27.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.934+0000 7f2ff4ecf700 1 --2- 172.21.15.47:0/1145467850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ff01471a0 0x7f2ff01529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:27.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.934+0000 7f2fef7fe700 1 -- 172.21.15.47:0/1145467850 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2ff0060ee0 msgr2=0x7f2ff0149d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:27.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.934+0000 7f2fef7fe700 1 --2- 172.21.15.47:0/1145467850 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2ff0060ee0 0x7f2ff0149d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:27.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.934+0000 7f2fef7fe700 1 -- 172.21.15.47:0/1145467850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ff01471a0 msgr2=0x7f2ff01529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:27.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.934+0000 7f2fef7fe700 1 --2- 172.21.15.47:0/1145467850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ff01471a0 0x7f2ff01529d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:27.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.934+0000 7f2fef7fe700 1 -- 172.21.15.47:0/1145467850 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe000b180 con 0x7f2ff00b4300 2022-06-29T15:51:27.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.934+0000 7f2fef7fe700 1 --2- 172.21.15.47:0/1145467850 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ff00b4300 0x7f2ff01524e0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 rx=0x7f2fe0002f70 tx=0x7f2fe0014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:27.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.934+0000 7f2fed7fa700 1 -- 172.21.15.47:0/1145467850 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2fe0016070 con 0x7f2ff00b4300 2022-06-29T15:51:27.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.934+0000 7f2fed7fa700 1 -- 172.21.15.47:0/1145467850 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe00137d0 con 0x7f2ff00b4300 2022-06-29T15:51:27.950 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.934+0000 7f2ff6932700 1 -- 172.21.15.47:0/1145467850 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ff00b3970 con 0x7f2ff00b4300 2022-06-29T15:51:27.950 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.934+0000 7f2fed7fa700 1 -- 172.21.15.47:0/1145467850 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2fe0013b30 con 0x7f2ff00b4300 2022-06-29T15:51:27.950 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.934+0000 7f2ff6932700 1 -- 172.21.15.47:0/1145467850 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ff013e200 con 0x7f2ff00b4300 2022-06-29T15:51:27.950 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.934+0000 7f2ff6932700 1 -- 172.21.15.47:0/1145467850 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ff00573d0 con 0x7f2ff00b4300 2022-06-29T15:51:27.951 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.934+0000 7f2fed7fa700 1 -- 172.21.15.47:0/1145467850 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2fe0025030 con 0x7f2ff00b4300 2022-06-29T15:51:27.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.938+0000 7f2fed7fa700 1 --2- 172.21.15.47:0/1145467850 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2fd4053a10 0x7f2fd4055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:27.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.938+0000 7f2fed7fa700 1 -- 172.21.15.47:0/1145467850 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f2fe006ce70 con 0x7f2ff00b4300 2022-06-29T15:51:27.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.938+0000 7f2feffff700 1 --2- 172.21.15.47:0/1145467850 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2fd4053a10 0x7f2fd4055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:27.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.938+0000 7f2feffff700 1 --2- 172.21.15.47:0/1145467850 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2fd4053a10 0x7f2fd4055ed0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7f2fd800d000 tx=0x7f2fd8007a00).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:27.953 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:27.942+0000 7f2fed7fa700 1 -- 172.21.15.47:0/1145467850 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2fe0043c30 con 0x7f2ff00b4300 2022-06-29T15:51:28.079 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.070+0000 7f2ff6932700 1 -- 172.21.15.47:0/1145467850 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2ff00b5e50 con 0x7f2ff00b4300 2022-06-29T15:51:28.080 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.074+0000 7f2fed7fa700 1 -- 172.21.15.47:0/1145467850 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v140) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2fe0023020 con 0x7f2ff00b4300 2022-06-29T15:51:28.080 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:51:28.082 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.074+0000 7f2ff6932700 1 -- 172.21.15.47:0/1145467850 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2fd4053a10 msgr2=0x7f2fd4055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:28.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.074+0000 7f2ff6932700 1 --2- 172.21.15.47:0/1145467850 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2fd4053a10 0x7f2fd4055ed0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7f2fd800d000 tx=0x7f2fd8007a00).stop 2022-06-29T15:51:28.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.074+0000 7f2ff6932700 1 -- 172.21.15.47:0/1145467850 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ff00b4300 msgr2=0x7f2ff01524e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:28.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.074+0000 7f2ff6932700 1 --2- 172.21.15.47:0/1145467850 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ff00b4300 0x7f2ff01524e0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 rx=0x7f2fe0002f70 tx=0x7f2fe0014000).stop 2022-06-29T15:51:28.084 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.074+0000 7f2ff6932700 1 -- 172.21.15.47:0/1145467850 shutdown_connections 2022-06-29T15:51:28.084 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.074+0000 7f2ff6932700 1 --2- 172.21.15.47:0/1145467850 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2fd4053a10 0x7f2fd4055ed0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:28.084 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.074+0000 7f2ff6932700 1 --2- 172.21.15.47:0/1145467850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ff01471a0 0x7f2ff01529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:28.084 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.074+0000 7f2ff6932700 1 --2- 172.21.15.47:0/1145467850 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ff00b4300 0x7f2ff01524e0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:28.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.074+0000 7f2ff6932700 1 --2- 172.21.15.47:0/1145467850 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2ff0060ee0 0x7f2ff0149d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:28.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.074+0000 7f2ff6932700 1 -- 172.21.15.47:0/1145467850 >> 172.21.15.47:0/1145467850 conn(0x7f2ff005c510 msgr2=0x7f2ff01434f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:28.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.074+0000 7f2ff6932700 1 -- 172.21.15.47:0/1145467850 shutdown_connections 2022-06-29T15:51:28.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.074+0000 7f2ff6932700 1 -- 172.21.15.47:0/1145467850 wait complete. 2022-06-29T15:51:28.094 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-29T15:51:28.205 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.198+0000 7f3a6609f700 1 Processor -- start 2022-06-29T15:51:28.205 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.198+0000 7f3a6609f700 1 -- start start 2022-06-29T15:51:28.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.198+0000 7f3a6609f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3a6005c850 0x7f3a60143a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:28.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.198+0000 7f3a6609f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3a600b5d90 0x7f3a600b4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:28.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.198+0000 7f3a6609f700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3a600676a0 0x7f3a60140130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:28.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.198+0000 7f3a6609f700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a6005cc30 con 0x7f3a6005c850 2022-06-29T15:51:28.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.198+0000 7f3a6609f700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a601430a0 con 0x7f3a600b5d90 2022-06-29T15:51:28.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.198+0000 7f3a6609f700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a60143300 con 0x7f3a600676a0 2022-06-29T15:51:28.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.198+0000 7f3a6509d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3a6005c850 0x7f3a60143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:28.209 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.198+0000 7f3a6509d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3a6005c850 0x7f3a60143a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53762/0 (socket says 172.21.15.47:53762) 2022-06-29T15:51:28.209 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.198+0000 7f3a6509d700 1 -- 172.21.15.47:0/35876402 learned_addr learned my addr 172.21.15.47:0/35876402 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:28.209 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.198+0000 7f3a6589e700 1 --2- 172.21.15.47:0/35876402 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3a600676a0 0x7f3a60140130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:28.209 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.198+0000 7f3a6489c700 1 --2- 172.21.15.47:0/35876402 >> v2:172.21.15.64:3300/0 conn(0x7f3a600b5d90 0x7f3a600b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:28.210 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.198+0000 7f3a6509d700 1 -- 172.21.15.47:0/35876402 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3a600676a0 msgr2=0x7f3a60140130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:28.210 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.198+0000 7f3a6509d700 1 --2- 172.21.15.47:0/35876402 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3a600676a0 0x7f3a60140130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:28.210 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.198+0000 7f3a6509d700 1 -- 172.21.15.47:0/35876402 >> v2:172.21.15.64:3300/0 conn(0x7f3a600b5d90 msgr2=0x7f3a600b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:28.210 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.198+0000 7f3a6509d700 1 --2- 172.21.15.47:0/35876402 >> v2:172.21.15.64:3300/0 conn(0x7f3a600b5d90 0x7f3a600b4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:28.211 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.198+0000 7f3a6509d700 1 -- 172.21.15.47:0/35876402 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a601435a0 con 0x7f3a6005c850 2022-06-29T15:51:28.211 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.198+0000 7f3a6489c700 1 --2- 172.21.15.47:0/35876402 >> v2:172.21.15.64:3300/0 conn(0x7f3a600b5d90 0x7f3a600b4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:51:28.211 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.198+0000 7f3a6509d700 1 --2- 172.21.15.47:0/35876402 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a6005c850 0x7f3a60143a40 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 rx=0x7f3a50002f70 tx=0x7f3a50006eb0).ready entity=mon.0 client_cookie=4b67da5f70c5520d server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:28.211 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.202+0000 7f3a57fff700 1 -- 172.21.15.47:0/35876402 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a5000d570 con 0x7f3a6005c850 2022-06-29T15:51:28.211 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.202+0000 7f3a57fff700 1 -- 172.21.15.47:0/35876402 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a50000e90 con 0x7f3a6005c850 2022-06-29T15:51:28.212 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.202+0000 7f3a57fff700 1 -- 172.21.15.47:0/35876402 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a5000d900 con 0x7f3a6005c850 2022-06-29T15:51:28.215 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.202+0000 7f3a6609f700 1 -- 172.21.15.47:0/35876402 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a6005c850 msgr2=0x7f3a60143a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:28.215 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.202+0000 7f3a6609f700 1 --2- 172.21.15.47:0/35876402 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a6005c850 0x7f3a60143a40 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 rx=0x7f3a50002f70 tx=0x7f3a50006eb0).stop 2022-06-29T15:51:28.215 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.202+0000 7f3a6609f700 1 -- 172.21.15.47:0/35876402 shutdown_connections 2022-06-29T15:51:28.215 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.202+0000 7f3a6609f700 1 --2- 172.21.15.47:0/35876402 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3a600676a0 0x7f3a60140130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:28.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.202+0000 7f3a6609f700 1 --2- 172.21.15.47:0/35876402 >> v2:172.21.15.64:3300/0 conn(0x7f3a600b5d90 0x7f3a600b4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:28.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.202+0000 7f3a6609f700 1 --2- 172.21.15.47:0/35876402 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a6005c850 0x7f3a60143a40 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:28.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.202+0000 7f3a6609f700 1 -- 172.21.15.47:0/35876402 >> 172.21.15.47:0/35876402 conn(0x7f3a60060e00 msgr2=0x7f3a600611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:28.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.202+0000 7f3a6609f700 1 -- 172.21.15.47:0/35876402 shutdown_connections 2022-06-29T15:51:28.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.202+0000 7f3a6609f700 1 -- 172.21.15.47:0/35876402 wait complete. 2022-06-29T15:51:28.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.202+0000 7f3a6609f700 1 Processor -- start 2022-06-29T15:51:28.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.202+0000 7f3a6609f700 1 -- start start 2022-06-29T15:51:28.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.206+0000 7f3a6609f700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3a600676a0 0x7f3a600b8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:28.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.206+0000 7f3a6609f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a600b5d90 0x7f3a6012ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:28.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.206+0000 7f3a6609f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a600b2e00 0x7f3a60152360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:28.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.206+0000 7f3a6609f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6012d6f0 con 0x7f3a600b2e00 2022-06-29T15:51:28.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.206+0000 7f3a6609f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6012c1f0 con 0x7f3a600b5d90 2022-06-29T15:51:28.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.206+0000 7f3a6609f700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6012c490 con 0x7f3a600676a0 2022-06-29T15:51:28.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.206+0000 7f3a6589e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a600b2e00 0x7f3a60152360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:28.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.206+0000 7f3a6589e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a600b2e00 0x7f3a60152360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53772/0 (socket says 172.21.15.47:53772) 2022-06-29T15:51:28.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.206+0000 7f3a6589e700 1 -- 172.21.15.47:0/1629460346 learned_addr learned my addr 172.21.15.47:0/1629460346 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:28.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.206+0000 7f3a6489c700 1 --2- 172.21.15.47:0/1629460346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a600b5d90 0x7f3a6012ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:28.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.206+0000 7f3a6509d700 1 --2- 172.21.15.47:0/1629460346 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3a600676a0 0x7f3a600b8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:28.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.206+0000 7f3a6589e700 1 -- 172.21.15.47:0/1629460346 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3a600676a0 msgr2=0x7f3a600b8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:28.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.206+0000 7f3a6589e700 1 --2- 172.21.15.47:0/1629460346 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3a600676a0 0x7f3a600b8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:28.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.206+0000 7f3a6589e700 1 -- 172.21.15.47:0/1629460346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a600b5d90 msgr2=0x7f3a6012ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:28.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.206+0000 7f3a6589e700 1 --2- 172.21.15.47:0/1629460346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a600b5d90 0x7f3a6012ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:28.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.206+0000 7f3a6589e700 1 -- 172.21.15.47:0/1629460346 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a5000b1e0 con 0x7f3a600b2e00 2022-06-29T15:51:28.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.206+0000 7f3a6509d700 1 --2- 172.21.15.47:0/1629460346 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3a600676a0 0x7f3a600b8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:51:28.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.206+0000 7f3a6589e700 1 --2- 172.21.15.47:0/1629460346 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a600b2e00 0x7f3a60152360 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 rx=0x7f3a58006e00 tx=0x7f3a580091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:28.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.206+0000 7f3a567fc700 1 -- 172.21.15.47:0/1629460346 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a5800c7f0 con 0x7f3a600b2e00 2022-06-29T15:51:28.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.206+0000 7f3a567fc700 1 -- 172.21.15.47:0/1629460346 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a5800c950 con 0x7f3a600b2e00 2022-06-29T15:51:28.222 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.206+0000 7f3a6609f700 1 -- 172.21.15.47:0/1629460346 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a6012d920 con 0x7f3a600b2e00 2022-06-29T15:51:28.222 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.206+0000 7f3a567fc700 1 -- 172.21.15.47:0/1629460346 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a5800ccb0 con 0x7f3a600b2e00 2022-06-29T15:51:28.222 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.206+0000 7f3a6609f700 1 -- 172.21.15.47:0/1629460346 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a60140ae0 con 0x7f3a600b2e00 2022-06-29T15:51:28.222 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.206+0000 7f3a567fc700 1 -- 172.21.15.47:0/1629460346 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3a580123f0 con 0x7f3a600b2e00 2022-06-29T15:51:28.223 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.206+0000 7f3a6609f700 1 -- 172.21.15.47:0/1629460346 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a6012dab0 con 0x7f3a600b2e00 2022-06-29T15:51:28.223 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.206+0000 7f3a567fc700 1 --2- 172.21.15.47:0/1629460346 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3a48053b00 0x7f3a48055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:28.224 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.210+0000 7f3a6509d700 1 --2- 172.21.15.47:0/1629460346 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3a48053b00 0x7f3a48055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:28.224 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.210+0000 7f3a567fc700 1 -- 172.21.15.47:0/1629460346 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f3a5806c4c0 con 0x7f3a600b2e00 2022-06-29T15:51:28.224 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.210+0000 7f3a6509d700 1 --2- 172.21.15.47:0/1629460346 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3a48053b00 0x7f3a48055fc0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7f3a50015000 tx=0x7f3a50014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:28.225 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.210+0000 7f3a567fc700 1 -- 172.21.15.47:0/1629460346 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3a58043110 con 0x7f3a600b2e00 2022-06-29T15:51:28.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.334+0000 7f3a6609f700 1 -- 172.21.15.47:0/1629460346 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3a60061b00 con 0x7f3a600b2e00 2022-06-29T15:51:28.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.334+0000 7f3a567fc700 1 -- 172.21.15.47:0/1629460346 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v140) v1 ==== 104+0+10 (secure 0 0 0) 0x7f3a580160a0 con 0x7f3a600b2e00 2022-06-29T15:51:28.343 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:51:28.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.338+0000 7f3a6609f700 1 -- 172.21.15.47:0/1629460346 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3a48053b00 msgr2=0x7f3a48055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:28.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.338+0000 7f3a6609f700 1 --2- 172.21.15.47:0/1629460346 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3a48053b00 0x7f3a48055fc0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7f3a50015000 tx=0x7f3a50014000).stop 2022-06-29T15:51:28.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.338+0000 7f3a6609f700 1 -- 172.21.15.47:0/1629460346 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a600b2e00 msgr2=0x7f3a60152360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:28.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.338+0000 7f3a6609f700 1 --2- 172.21.15.47:0/1629460346 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a600b2e00 0x7f3a60152360 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 rx=0x7f3a58006e00 tx=0x7f3a580091d0).stop 2022-06-29T15:51:28.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.338+0000 7f3a6609f700 1 -- 172.21.15.47:0/1629460346 shutdown_connections 2022-06-29T15:51:28.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.338+0000 7f3a6609f700 1 --2- 172.21.15.47:0/1629460346 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a600b2e00 0x7f3a60152360 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:28.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.338+0000 7f3a6609f700 1 --2- 172.21.15.47:0/1629460346 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3a48053b00 0x7f3a48055fc0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:28.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.338+0000 7f3a6609f700 1 --2- 172.21.15.47:0/1629460346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a600b5d90 0x7f3a6012ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:28.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.338+0000 7f3a6609f700 1 --2- 172.21.15.47:0/1629460346 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f3a600676a0 0x7f3a600b8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:28.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.338+0000 7f3a6609f700 1 -- 172.21.15.47:0/1629460346 >> 172.21.15.47:0/1629460346 conn(0x7f3a60060e00 msgr2=0x7f3a6012b410 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:28.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.338+0000 7f3a6609f700 1 -- 172.21.15.47:0/1629460346 shutdown_connections 2022-06-29T15:51:28.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.338+0000 7f3a6609f700 1 -- 172.21.15.47:0/1629460346 wait complete. 2022-06-29T15:51:28.361 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-29T15:51:28.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.466+0000 7fa335f8e700 1 Processor -- start 2022-06-29T15:51:28.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.466+0000 7fa335f8e700 1 -- start start 2022-06-29T15:51:28.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.470+0000 7fa335f8e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa3300b6e20 0x7fa3301447c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:28.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.470+0000 7fa335f8e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa3300630f0 0x7fa3301412e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:28.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.470+0000 7fa335f8e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3300b7200 con 0x7fa330144cb0 2022-06-29T15:51:28.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.470+0000 7fa335f8e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa33005f5c0 con 0x7fa3300b6e20 2022-06-29T15:51:28.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.470+0000 7fa335f8e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa33005f860 con 0x7fa3300630f0 2022-06-29T15:51:28.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.470+0000 7fa334f8c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa3300b6e20 0x7fa3301447c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:28.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.470+0000 7fa33578d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa330144cb0 0x7fa3301417d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39334/0 (socket says 172.21.15.47:39334) 2022-06-29T15:51:28.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.470+0000 7fa33578d700 1 -- 172.21.15.47:0/1975915015 learned_addr learned my addr 172.21.15.47:0/1975915015 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:28.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.470+0000 7fa32ffff700 1 --2- 172.21.15.47:0/1975915015 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa3300630f0 0x7fa3301412e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:28.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.470+0000 7fa32f7fe700 1 -- 172.21.15.47:0/1975915015 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 724930648 0 0) 0x7fa3300b7200 con 0x7fa330144cb0 2022-06-29T15:51:28.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.470+0000 7fa32f7fe700 1 -- 172.21.15.47:0/1975915015 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa31c003260 con 0x7fa330144cb0 2022-06-29T15:51:28.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.470+0000 7fa334f8c700 1 -- 172.21.15.47:0/1975915015 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa3300630f0 msgr2=0x7fa3301412e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:28.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.470+0000 7fa334f8c700 1 --2- 172.21.15.47:0/1975915015 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa3300630f0 0x7fa3301412e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:28.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.470+0000 7fa334f8c700 1 -- 172.21.15.47:0/1975915015 >> v1:172.21.15.47:6789/0 conn(0x7fa330144cb0 legacy=0x7fa3301417d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:28.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.470+0000 7fa334f8c700 1 -- 172.21.15.47:0/1975915015 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3300afcb0 con 0x7fa3300b6e20 2022-06-29T15:51:28.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.470+0000 7fa334f8c700 1 --2- 172.21.15.47:0/1975915015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3300b6e20 0x7fa3301447c0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 rx=0x7fa320002f70 tx=0x7fa320006df0).ready entity=mon.0 client_cookie=3d2dbe52179dfd67 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:28.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.470+0000 7fa32f7fe700 1 -- 172.21.15.47:0/1975915015 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa32000d4a0 con 0x7fa3300b6e20 2022-06-29T15:51:28.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.470+0000 7fa32f7fe700 1 -- 172.21.15.47:0/1975915015 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa320000e90 con 0x7fa3300b6e20 2022-06-29T15:51:28.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.470+0000 7fa32f7fe700 1 -- 172.21.15.47:0/1975915015 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa32000d830 con 0x7fa3300b6e20 2022-06-29T15:51:28.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.470+0000 7fa335f8e700 1 -- 172.21.15.47:0/1975915015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3300b6e20 msgr2=0x7fa3301447c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:28.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.470+0000 7fa335f8e700 1 --2- 172.21.15.47:0/1975915015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3300b6e20 0x7fa3301447c0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 rx=0x7fa320002f70 tx=0x7fa320006df0).stop 2022-06-29T15:51:28.484 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.470+0000 7fa335f8e700 1 -- 172.21.15.47:0/1975915015 shutdown_connections 2022-06-29T15:51:28.484 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.470+0000 7fa335f8e700 1 --2- 172.21.15.47:0/1975915015 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa3300630f0 0x7fa3301412e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:28.484 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.470+0000 7fa335f8e700 1 --2- 172.21.15.47:0/1975915015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3300b6e20 0x7fa3301447c0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:28.484 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.470+0000 7fa335f8e700 1 -- 172.21.15.47:0/1975915015 >> 172.21.15.47:0/1975915015 conn(0x7fa330063670 msgr2=0x7fa330063a50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:28.485 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.470+0000 7fa335f8e700 1 -- 172.21.15.47:0/1975915015 shutdown_connections 2022-06-29T15:51:28.485 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.470+0000 7fa335f8e700 1 -- 172.21.15.47:0/1975915015 wait complete. 2022-06-29T15:51:28.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.474+0000 7fa335f8e700 1 Processor -- start 2022-06-29T15:51:28.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.474+0000 7fa335f8e700 1 -- start start 2022-06-29T15:51:28.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.474+0000 7fa335f8e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa3300630f0 0x7fa3300a3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:28.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.474+0000 7fa335f8e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3300b6e20 0x7fa3300a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:28.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.474+0000 7fa335f8e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa330144cb0 0x7fa3301505b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:28.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.474+0000 7fa335f8e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3300a4ca0 con 0x7fa330144cb0 2022-06-29T15:51:28.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.474+0000 7fa335f8e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa330150040 con 0x7fa3300b6e20 2022-06-29T15:51:28.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.474+0000 7fa335f8e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3300a2620 con 0x7fa3300630f0 2022-06-29T15:51:28.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.474+0000 7fa33578d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa330144cb0 0x7fa3301505b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:28.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.474+0000 7fa33578d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa330144cb0 0x7fa3301505b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53786/0 (socket says 172.21.15.47:53786) 2022-06-29T15:51:28.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.474+0000 7fa334f8c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa3300630f0 0x7fa3300a3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:28.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.474+0000 7fa33578d700 1 -- 172.21.15.47:0/1956929578 learned_addr learned my addr 172.21.15.47:0/1956929578 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:28.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.474+0000 7fa32ffff700 1 --2- 172.21.15.47:0/1956929578 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3300b6e20 0x7fa3300a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:28.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.474+0000 7fa334f8c700 1 -- 172.21.15.47:0/1956929578 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3300b6e20 msgr2=0x7fa3300a1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:28.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.474+0000 7fa334f8c700 1 --2- 172.21.15.47:0/1956929578 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3300b6e20 0x7fa3300a1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:28.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.474+0000 7fa334f8c700 1 -- 172.21.15.47:0/1956929578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa330144cb0 msgr2=0x7fa3301505b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:28.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.474+0000 7fa334f8c700 1 --2- 172.21.15.47:0/1956929578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa330144cb0 0x7fa3301505b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:28.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.474+0000 7fa334f8c700 1 -- 172.21.15.47:0/1956929578 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa32000b180 con 0x7fa3300630f0 2022-06-29T15:51:28.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.474+0000 7fa33578d700 1 --2- 172.21.15.47:0/1956929578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa330144cb0 0x7fa3301505b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:51:28.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.474+0000 7fa334f8c700 1 --2- 172.21.15.47:0/1956929578 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa3300630f0 0x7fa3300a3070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7fa320002f70 tx=0x7fa320014000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:28.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.474+0000 7fa32dffb700 1 -- 172.21.15.47:0/1956929578 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa320013a40 con 0x7fa3300630f0 2022-06-29T15:51:28.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.474+0000 7fa32dffb700 1 -- 172.21.15.47:0/1956929578 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa320013ba0 con 0x7fa3300630f0 2022-06-29T15:51:28.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.474+0000 7fa335f8e700 1 -- 172.21.15.47:0/1956929578 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3300a28c0 con 0x7fa3300630f0 2022-06-29T15:51:28.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.474+0000 7fa32dffb700 1 -- 172.21.15.47:0/1956929578 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa32001e400 con 0x7fa3300630f0 2022-06-29T15:51:28.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.474+0000 7fa335f8e700 1 -- 172.21.15.47:0/1956929578 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3300a22a0 con 0x7fa3300630f0 2022-06-29T15:51:28.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.478+0000 7fa335f8e700 1 -- 172.21.15.47:0/1956929578 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa330143e70 con 0x7fa3300630f0 2022-06-29T15:51:28.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.478+0000 7fa32dffb700 1 -- 172.21.15.47:0/1956929578 <== mon.2 v2:172.21.15.64:3301/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa320025030 con 0x7fa3300630f0 2022-06-29T15:51:28.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.478+0000 7fa32dffb700 1 --2- 172.21.15.47:0/1956929578 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa3100539e0 0x7fa310055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:28.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.478+0000 7fa32dffb700 1 -- 172.21.15.47:0/1956929578 <== mon.2 v2:172.21.15.64:3301/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa32006cc70 con 0x7fa3300630f0 2022-06-29T15:51:28.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.478+0000 7fa32ffff700 1 --2- 172.21.15.47:0/1956929578 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa3100539e0 0x7fa310055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:28.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.478+0000 7fa32ffff700 1 --2- 172.21.15.47:0/1956929578 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa3100539e0 0x7fa310055ea0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7fa32400b000 tx=0x7fa324006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:28.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.486+0000 7fa32dffb700 1 -- 172.21.15.47:0/1956929578 <== mon.2 v2:172.21.15.64:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa320043a30 con 0x7fa3300630f0 2022-06-29T15:51:28.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.618+0000 7fa335f8e700 1 -- 172.21.15.47:0/1956929578 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa3300573d0 con 0x7fa3300630f0 2022-06-29T15:51:28.626 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.618+0000 7fa32dffb700 1 -- 172.21.15.47:0/1956929578 <== mon.2 v2:172.21.15.64:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v140) v1 ==== 100+0+10 (secure 0 0 0) 0x7fa320023070 con 0x7fa3300630f0 2022-06-29T15:51:28.626 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:51:28.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.622+0000 7fa335f8e700 1 -- 172.21.15.47:0/1956929578 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa3100539e0 msgr2=0x7fa310055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:28.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.622+0000 7fa335f8e700 1 --2- 172.21.15.47:0/1956929578 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa3100539e0 0x7fa310055ea0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7fa32400b000 tx=0x7fa324006df0).stop 2022-06-29T15:51:28.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.622+0000 7fa335f8e700 1 -- 172.21.15.47:0/1956929578 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa3300630f0 msgr2=0x7fa3300a3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:28.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.622+0000 7fa335f8e700 1 --2- 172.21.15.47:0/1956929578 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa3300630f0 0x7fa3300a3070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7fa320002f70 tx=0x7fa320014000).stop 2022-06-29T15:51:28.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.622+0000 7fa335f8e700 1 -- 172.21.15.47:0/1956929578 shutdown_connections 2022-06-29T15:51:28.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.622+0000 7fa335f8e700 1 --2- 172.21.15.47:0/1956929578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa330144cb0 0x7fa3301505b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:28.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.622+0000 7fa335f8e700 1 --2- 172.21.15.47:0/1956929578 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa3100539e0 0x7fa310055ea0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:28.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.622+0000 7fa335f8e700 1 --2- 172.21.15.47:0/1956929578 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3300b6e20 0x7fa3300a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:28.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.622+0000 7fa335f8e700 1 --2- 172.21.15.47:0/1956929578 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa3300630f0 0x7fa3300a3070 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:28.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.622+0000 7fa335f8e700 1 -- 172.21.15.47:0/1956929578 >> 172.21.15.47:0/1956929578 conn(0x7fa330063670 msgr2=0x7fa3300b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:28.631 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.622+0000 7fa335f8e700 1 -- 172.21.15.47:0/1956929578 shutdown_connections 2022-06-29T15:51:28.631 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.622+0000 7fa335f8e700 1 -- 172.21.15.47:0/1956929578 wait complete. 2022-06-29T15:51:28.640 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-29T15:51:28.641 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:51:28.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.734+0000 7ff875627700 1 Processor -- start 2022-06-29T15:51:28.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.738+0000 7ff875627700 1 -- start start 2022-06-29T15:51:28.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.738+0000 7ff875627700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff8700b19a0 0x7ff87005fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:28.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.738+0000 7ff875627700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff8701437d0 0x7ff870143c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:28.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.738+0000 7ff875627700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8700a97e0 con 0x7ff8700acff0 2022-06-29T15:51:28.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.738+0000 7ff875627700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8700b1d80 con 0x7ff8700b19a0 2022-06-29T15:51:28.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.738+0000 7ff875627700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff87005f430 con 0x7ff8701437d0 2022-06-29T15:51:28.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.738+0000 7ff86f7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff8700acff0 0x7ff8700ae840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39346/0 (socket says 172.21.15.47:39346) 2022-06-29T15:51:28.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.738+0000 7ff86ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff8700b19a0 0x7ff87005fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:28.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.738+0000 7ff86f7fe700 1 -- 172.21.15.47:0/1794199692 learned_addr learned my addr 172.21.15.47:0/1794199692 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:28.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.738+0000 7ff874e26700 1 --2- 172.21.15.47:0/1794199692 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff8701437d0 0x7ff870143c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:28.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.738+0000 7ff86effd700 1 -- 172.21.15.47:0/1794199692 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2737114247 0 0) 0x7ff8700a97e0 con 0x7ff8700acff0 2022-06-29T15:51:28.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.738+0000 7ff86effd700 1 -- 172.21.15.47:0/1794199692 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff85c003260 con 0x7ff8700acff0 2022-06-29T15:51:28.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.738+0000 7ff874e26700 1 -- 172.21.15.47:0/1794199692 >> v2:172.21.15.64:3300/0 conn(0x7ff8700b19a0 msgr2=0x7ff87005fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:28.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.738+0000 7ff874e26700 1 --2- 172.21.15.47:0/1794199692 >> v2:172.21.15.64:3300/0 conn(0x7ff8700b19a0 0x7ff87005fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:28.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.738+0000 7ff874e26700 1 -- 172.21.15.47:0/1794199692 >> v1:172.21.15.47:6789/0 conn(0x7ff8700acff0 legacy=0x7ff8700ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:28.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.738+0000 7ff874e26700 1 -- 172.21.15.47:0/1794199692 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8700a92a0 con 0x7ff8701437d0 2022-06-29T15:51:28.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.738+0000 7ff86ffff700 1 --2- 172.21.15.47:0/1794199692 >> v2:172.21.15.64:3300/0 conn(0x7ff8700b19a0 0x7ff87005fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:51:28.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.738+0000 7ff874e26700 1 --2- 172.21.15.47:0/1794199692 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff8701437d0 0x7ff870143c60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7ff8680032f0 tx=0x7ff8680091d0).ready entity=mon.2 client_cookie=57b3ad98cbc6d9c9 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:28.752 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.738+0000 7ff86effd700 1 -- 172.21.15.47:0/1794199692 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff86800b600 con 0x7ff8701437d0 2022-06-29T15:51:28.752 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.738+0000 7ff86effd700 1 -- 172.21.15.47:0/1794199692 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff86800b760 con 0x7ff8701437d0 2022-06-29T15:51:28.752 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.738+0000 7ff86effd700 1 -- 172.21.15.47:0/1794199692 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff86800b8d0 con 0x7ff8701437d0 2022-06-29T15:51:28.752 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.742+0000 7ff875627700 1 -- 172.21.15.47:0/1794199692 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff8701437d0 msgr2=0x7ff870143c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:28.753 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.742+0000 7ff875627700 1 --2- 172.21.15.47:0/1794199692 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff8701437d0 0x7ff870143c60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7ff8680032f0 tx=0x7ff8680091d0).stop 2022-06-29T15:51:28.753 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.742+0000 7ff875627700 1 -- 172.21.15.47:0/1794199692 shutdown_connections 2022-06-29T15:51:28.753 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.742+0000 7ff875627700 1 --2- 172.21.15.47:0/1794199692 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff8701437d0 0x7ff870143c60 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:28.753 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.742+0000 7ff875627700 1 --2- 172.21.15.47:0/1794199692 >> v2:172.21.15.64:3300/0 conn(0x7ff8700b19a0 0x7ff87005fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:28.753 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.742+0000 7ff875627700 1 -- 172.21.15.47:0/1794199692 >> 172.21.15.47:0/1794199692 conn(0x7ff870062df0 msgr2=0x7ff8700631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:28.754 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.742+0000 7ff875627700 1 -- 172.21.15.47:0/1794199692 shutdown_connections 2022-06-29T15:51:28.754 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.742+0000 7ff875627700 1 -- 172.21.15.47:0/1794199692 wait complete. 2022-06-29T15:51:28.754 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.742+0000 7ff875627700 1 Processor -- start 2022-06-29T15:51:28.754 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.742+0000 7ff875627700 1 -- start start 2022-06-29T15:51:28.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.746+0000 7ff875627700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8700acff0 0x7ff8700b45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:28.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.746+0000 7ff875627700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff8700b19a0 0x7ff870151e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:28.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.746+0000 7ff875627700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8701437d0 0x7ff870152370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:28.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.746+0000 7ff875627700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff870099e70 con 0x7ff8700acff0 2022-06-29T15:51:28.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.746+0000 7ff875627700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8700ac320 con 0x7ff8701437d0 2022-06-29T15:51:28.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.746+0000 7ff875627700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8700644c0 con 0x7ff8700b19a0 2022-06-29T15:51:28.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.746+0000 7ff86ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8700acff0 0x7ff8700b45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:28.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.746+0000 7ff86ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8700acff0 0x7ff8700b45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53796/0 (socket says 172.21.15.47:53796) 2022-06-29T15:51:28.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.746+0000 7ff86ffff700 1 -- 172.21.15.47:0/2310695568 learned_addr learned my addr 172.21.15.47:0/2310695568 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:28.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.746+0000 7ff86f7fe700 1 --2- 172.21.15.47:0/2310695568 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff8700b19a0 0x7ff870151e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:28.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.746+0000 7ff874e26700 1 --2- 172.21.15.47:0/2310695568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8701437d0 0x7ff870152370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:28.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.746+0000 7ff86ffff700 1 -- 172.21.15.47:0/2310695568 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff8700b19a0 msgr2=0x7ff870151e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:28.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.746+0000 7ff86ffff700 1 --2- 172.21.15.47:0/2310695568 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff8700b19a0 0x7ff870151e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:28.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.746+0000 7ff86ffff700 1 -- 172.21.15.47:0/2310695568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8701437d0 msgr2=0x7ff870152370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:28.761 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.746+0000 7ff86ffff700 1 --2- 172.21.15.47:0/2310695568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8701437d0 0x7ff870152370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:28.761 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.746+0000 7ff86ffff700 1 -- 172.21.15.47:0/2310695568 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff86000b180 con 0x7ff8700acff0 2022-06-29T15:51:28.761 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.746+0000 7ff86ffff700 1 --2- 172.21.15.47:0/2310695568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8700acff0 0x7ff8700b45b0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 rx=0x7ff86000c000 tx=0x7ff860006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:28.761 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.746+0000 7ff86d7fa700 1 -- 172.21.15.47:0/2310695568 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff86000c7e0 con 0x7ff8700acff0 2022-06-29T15:51:28.762 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.746+0000 7ff86d7fa700 1 -- 172.21.15.47:0/2310695568 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff86000c940 con 0x7ff8700acff0 2022-06-29T15:51:28.762 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.746+0000 7ff875627700 1 -- 172.21.15.47:0/2310695568 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff868009470 con 0x7ff8700acff0 2022-06-29T15:51:28.762 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.746+0000 7ff86d7fa700 1 -- 172.21.15.47:0/2310695568 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff86000cca0 con 0x7ff8700acff0 2022-06-29T15:51:28.762 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.746+0000 7ff875627700 1 -- 172.21.15.47:0/2310695568 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8700ac080 con 0x7ff8700acff0 2022-06-29T15:51:28.763 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.746+0000 7ff875627700 1 -- 172.21.15.47:0/2310695568 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8700abef0 con 0x7ff8700acff0 2022-06-29T15:51:28.763 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.746+0000 7ff86d7fa700 1 -- 172.21.15.47:0/2310695568 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff86001c460 con 0x7ff8700acff0 2022-06-29T15:51:28.763 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.750+0000 7ff86d7fa700 1 --2- 172.21.15.47:0/2310695568 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff850053980 0x7ff850055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:28.763 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.750+0000 7ff86d7fa700 1 -- 172.21.15.47:0/2310695568 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff86006c580 con 0x7ff8700acff0 2022-06-29T15:51:28.763 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.750+0000 7ff86f7fe700 1 --2- 172.21.15.47:0/2310695568 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff850053980 0x7ff850055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:28.764 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.750+0000 7ff86f7fe700 1 --2- 172.21.15.47:0/2310695568 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff850053980 0x7ff850055e40 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7ff86400d000 tx=0x7ff864009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:28.764 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.754+0000 7ff86d7fa700 1 -- 172.21.15.47:0/2310695568 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff8600432e0 con 0x7ff8700acff0 2022-06-29T15:51:28.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.922+0000 7ff875627700 1 -- 172.21.15.47:0/2310695568 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8700573d0 con 0x7ff8700acff0 2022-06-29T15:51:28.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.922+0000 7ff86d7fa700 1 -- 172.21.15.47:0/2310695568 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+2492 (secure 0 0 0) 0x7ff860046900 con 0x7ff8700acff0 2022-06-29T15:51:28.930 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:28.930 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":69,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7957,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.64:6833/298405638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":298405638},{"type":"v1","addr":"172.21.15.64:6833","nonce":298405638}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":66},{"gid":8037,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/191070564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":191070564},{"type":"v1","addr":"172.21.15.47:6835","nonce":191070564}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:51:23.543892+0000","modified":"2022-06-29T15:51:25.309120+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":56894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_56894":{"gid":56894,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6837/1370488030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1370488030},{"type":"v1","addr":"172.21.15.47:6837","nonce":1370488030}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2022-06-29T15:51:28.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.926+0000 7ff875627700 1 -- 172.21.15.47:0/2310695568 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff850053980 msgr2=0x7ff850055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:28.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.926+0000 7ff875627700 1 --2- 172.21.15.47:0/2310695568 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff850053980 0x7ff850055e40 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7ff86400d000 tx=0x7ff864009040).stop 2022-06-29T15:51:28.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.926+0000 7ff875627700 1 -- 172.21.15.47:0/2310695568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8700acff0 msgr2=0x7ff8700b45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:28.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.926+0000 7ff875627700 1 --2- 172.21.15.47:0/2310695568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8700acff0 0x7ff8700b45b0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 rx=0x7ff86000c000 tx=0x7ff860006df0).stop 2022-06-29T15:51:28.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.926+0000 7ff875627700 1 -- 172.21.15.47:0/2310695568 shutdown_connections 2022-06-29T15:51:28.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.926+0000 7ff875627700 1 --2- 172.21.15.47:0/2310695568 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff850053980 0x7ff850055e40 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:28.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.926+0000 7ff875627700 1 --2- 172.21.15.47:0/2310695568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8701437d0 0x7ff870152370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:28.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.926+0000 7ff875627700 1 --2- 172.21.15.47:0/2310695568 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff8700b19a0 0x7ff870151e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:28.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.926+0000 7ff875627700 1 --2- 172.21.15.47:0/2310695568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8700acff0 0x7ff8700b45b0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:28.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.926+0000 7ff875627700 1 -- 172.21.15.47:0/2310695568 >> 172.21.15.47:0/2310695568 conn(0x7ff870062df0 msgr2=0x7ff87009ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:28.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.926+0000 7ff875627700 1 -- 172.21.15.47:0/2310695568 shutdown_connections 2022-06-29T15:51:28.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:28.926+0000 7ff875627700 1 -- 172.21.15.47:0/2310695568 wait complete. 2022-06-29T15:51:28.936 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 69 2022-06-29T15:51:29.247 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 69, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-29T15:51:23.543892+0000', 'modified': '2022-06-29T15:51:25.309120+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 56894}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_56894': {'gid': 56894, 'name': 'c', 'rank': 0, 'incarnation': 67, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.47:6837/1370488030', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6836', 'nonce': 1370488030}, {'type': 'v1', 'addr': '172.21.15.47:6837', 'nonce': 1370488030}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-29T15:51:29.247 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-29T15:51:29.248 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2022-06-29T15:51:29.370 INFO:teuthology.orchestra.run.smithi047.stdout:{"cluster_fsid":"9a68894a-53ce-400e-838b-14eab91b834c","whoami":0,"id":56894,"want_state":"up:active","state":"up:active","rank_uptime":5.8083574320000002,"mdsmap_epoch":69,"osdmap_epoch":140,"osdmap_epoch_barrier":140,"uptime":10.613046566} 2022-06-29T15:51:29.370 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9a68894a-53ce-400e-838b-14eab91b834c", "id": 56894, "mdsmap_epoch": 69, "osdmap_epoch": 140, "osdmap_epoch_barrier": 140, "rank_uptime": 5.808357432, "state": "up:active", "uptime": 10.613046566, "want_state": "up:active", "whoami": 0 } 2022-06-29T15:51:29.371 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-29T15:51:29.371 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-06-29T15:51:29.372 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi099.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-06-29T15:51:29.372 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:51:29.373 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:51:29.381 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2022-06-29T15:51:29.395 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:51:29.396 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:51:29.408 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:51:29.409 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:51:29.410 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:51:29.410 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2022-06-29T15:51:29.415 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-29T15:51:29.416 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:51:29.460 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2022-06-29T15:51:29.478 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:51:29.479 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:51:29.490 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:51:29.491 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:51:29.491 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:51:29.492 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2022-06-29T15:51:29.541 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:2022-06-29T15:51:29.534+0000 7f52490b8080 -1 init, newargv = 0x5596cd8e7840 newargc=9ceph-fuse[13405 2022-06-29T15:51:29.541 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:]: starting ceph client 2022-06-29T15:51:29.555 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[13405]: starting fuse 2022-06-29T15:51:30.642 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2022-06-29T15:51:30.661 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:51:30.662 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:51:30.718 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:51:30.719 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:51:30.721 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:51:30.722 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2022-06-29T15:51:30.728 INFO:teuthology.orchestra.run.smithi099.stdout:52 2022-06-29T15:51:30.729 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-29T15:51:30.729 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:51:30.730 DEBUG:teuthology.orchestra.run.smithi099:> sudo python3 -c ' 2022-06-29T15:51:30.730 DEBUG:teuthology.orchestra.run.smithi099:> import glob 2022-06-29T15:51:30.731 DEBUG:teuthology.orchestra.run.smithi099:> import re 2022-06-29T15:51:30.731 DEBUG:teuthology.orchestra.run.smithi099:> import os 2022-06-29T15:51:30.732 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2022-06-29T15:51:30.732 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:51:30.732 DEBUG:teuthology.orchestra.run.smithi099:> def find_socket(client_name): 2022-06-29T15:51:30.733 DEBUG:teuthology.orchestra.run.smithi099:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-06-29T15:51:30.733 DEBUG:teuthology.orchestra.run.smithi099:> files = glob.glob(asok_path) 2022-06-29T15:51:30.734 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:51:30.734 DEBUG:teuthology.orchestra.run.smithi099:> # Given a non-glob path, it better be there 2022-06-29T15:51:30.735 DEBUG:teuthology.orchestra.run.smithi099:> if "*" not in asok_path: 2022-06-29T15:51:30.735 DEBUG:teuthology.orchestra.run.smithi099:> assert(len(files) == 1) 2022-06-29T15:51:30.735 DEBUG:teuthology.orchestra.run.smithi099:> return files[0] 2022-06-29T15:51:30.736 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:51:30.736 DEBUG:teuthology.orchestra.run.smithi099:> for f in files: 2022-06-29T15:51:30.737 DEBUG:teuthology.orchestra.run.smithi099:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-29T15:51:30.737 DEBUG:teuthology.orchestra.run.smithi099:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-29T15:51:30.737 DEBUG:teuthology.orchestra.run.smithi099:> return f 2022-06-29T15:51:30.738 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-29T15:51:30.738 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:51:30.739 DEBUG:teuthology.orchestra.run.smithi099:> print(find_socket("client.0")) 2022-06-29T15:51:30.739 DEBUG:teuthology.orchestra.run.smithi099:> ' 2022-06-29T15:51:30.816 INFO:teuthology.orchestra.run.smithi099.stdout:/var/run/ceph/ceph-client.0.13405.asok 2022-06-29T15:51:30.821 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.13405.asok 2022-06-29T15:51:30.821 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:51:30.822 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.13405.asok status 2022-06-29T15:51:30.908 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2022-06-29T15:51:30.909 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": { 2022-06-29T15:51:30.909 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_sha1": "f25ffac72e5a9e63860ff7deb6767a11f0559a16", 2022-06-29T15:51:30.909 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 15.2.16-304-gf25ffac7 (f25ffac72e5a9e63860ff7deb6767a11f0559a16) octopus (stable)", 2022-06-29T15:51:30.910 INFO:teuthology.orchestra.run.smithi099.stdout: "entity_id": "0", 2022-06-29T15:51:30.910 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": "smithi099", 2022-06-29T15:51:30.910 INFO:teuthology.orchestra.run.smithi099.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-06-29T15:51:30.910 INFO:teuthology.orchestra.run.smithi099.stdout: "pid": "13405", 2022-06-29T15:51:30.911 INFO:teuthology.orchestra.run.smithi099.stdout: "root": "/" 2022-06-29T15:51:30.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:51:30.911 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_count": 0, 2022-06-29T15:51:30.912 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_pinned_count": 0, 2022-06-29T15:51:30.912 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 8110, 2022-06-29T15:51:30.912 INFO:teuthology.orchestra.run.smithi099.stdout: "inst": { 2022-06-29T15:51:30.913 INFO:teuthology.orchestra.run.smithi099.stdout: "name": { 2022-06-29T15:51:30.913 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "client", 2022-06-29T15:51:30.913 INFO:teuthology.orchestra.run.smithi099.stdout: "num": 8110 2022-06-29T15:51:30.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:51:30.914 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2022-06-29T15:51:30.914 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2022-06-29T15:51:30.914 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:0", 2022-06-29T15:51:30.915 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2780862953 2022-06-29T15:51:30.915 INFO:teuthology.orchestra.run.smithi099.stdout: } 2022-06-29T15:51:30.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:51:30.916 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2022-06-29T15:51:30.916 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2022-06-29T15:51:30.916 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:0", 2022-06-29T15:51:30.917 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2780862953 2022-06-29T15:51:30.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:51:30.918 INFO:teuthology.orchestra.run.smithi099.stdout: "inst_str": "client.8110 172.21.15.99:0/2780862953", 2022-06-29T15:51:30.918 INFO:teuthology.orchestra.run.smithi099.stdout: "addr_str": "172.21.15.99:0/2780862953", 2022-06-29T15:51:30.919 INFO:teuthology.orchestra.run.smithi099.stdout: "inode_count": 1, 2022-06-29T15:51:30.919 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_epoch": 69, 2022-06-29T15:51:30.920 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch": 140, 2022-06-29T15:51:30.920 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch_barrier": 0, 2022-06-29T15:51:30.920 INFO:teuthology.orchestra.run.smithi099.stdout: "blacklisted": false, 2022-06-29T15:51:30.921 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs" 2022-06-29T15:51:30.921 INFO:teuthology.orchestra.run.smithi099.stdout:} 2022-06-29T15:51:30.923 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:51:30.924 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:51:30.925 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:51:30.933 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2022-06-29T15:51:30.934 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-29T15:51:30.935 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:51:30.936 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:51:30.997 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:51:31.111 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.102+0000 7f5361d86700 1 Processor -- start 2022-06-29T15:51:31.112 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.102+0000 7f5361d86700 1 -- start start 2022-06-29T15:51:31.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.106+0000 7f5361d86700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f535c0b5ac0 0x7f535c1401c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:31.118 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.106+0000 7f5361d86700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f535c0b02d0 0x7f535c1406b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:31.118 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.106+0000 7f5361d86700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f535c0afdc0 0x7f535c0ae0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:31.118 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.106+0000 7f5361d86700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f535c0a9090 con 0x7f535c0afdc0 2022-06-29T15:51:31.119 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.106+0000 7f5361d86700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f535c143d20 con 0x7f535c0b5ac0 2022-06-29T15:51:31.119 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.106+0000 7f5361d86700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f535c140ba0 con 0x7f535c0b02d0 2022-06-29T15:51:31.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.106+0000 7f5361585700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f535c0afdc0 0x7f535c0ae0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:31.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.106+0000 7f5361585700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f535c0afdc0 0x7f535c0ae0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53808/0 (socket says 172.21.15.47:53808) 2022-06-29T15:51:31.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.106+0000 7f5353fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f535c0b02d0 0x7f535c1406b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:31.121 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.106+0000 7f5361585700 1 -- 172.21.15.47:0/326201854 learned_addr learned my addr 172.21.15.47:0/326201854 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:31.121 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.106+0000 7f5360d84700 1 --2- 172.21.15.47:0/326201854 >> v2:172.21.15.64:3300/0 conn(0x7f535c0b5ac0 0x7f535c1401c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:31.122 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.106+0000 7f5361585700 1 -- 172.21.15.47:0/326201854 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f535c0b02d0 msgr2=0x7f535c1406b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:31.122 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.106+0000 7f5361585700 1 --2- 172.21.15.47:0/326201854 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f535c0b02d0 0x7f535c1406b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:31.122 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.106+0000 7f5361585700 1 -- 172.21.15.47:0/326201854 >> v2:172.21.15.64:3300/0 conn(0x7f535c0b5ac0 msgr2=0x7f535c1401c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:31.123 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.106+0000 7f5361585700 1 --2- 172.21.15.47:0/326201854 >> v2:172.21.15.64:3300/0 conn(0x7f535c0b5ac0 0x7f535c1401c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:31.123 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.106+0000 7f5361585700 1 -- 172.21.15.47:0/326201854 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f535c0a8df0 con 0x7f535c0afdc0 2022-06-29T15:51:31.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.106+0000 7f5361585700 1 --2- 172.21.15.47:0/326201854 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f535c0afdc0 0x7f535c0ae0c0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 rx=0x7f53540032f0 tx=0x7f5354009290).ready entity=mon.0 client_cookie=e6f5d2bf0dea6741 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:31.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.106+0000 7f53537fe700 1 -- 172.21.15.47:0/326201854 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f535400b680 con 0x7f535c0afdc0 2022-06-29T15:51:31.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.106+0000 7f53537fe700 1 -- 172.21.15.47:0/326201854 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f535400b7e0 con 0x7f535c0afdc0 2022-06-29T15:51:31.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.106+0000 7f53537fe700 1 -- 172.21.15.47:0/326201854 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f535400b950 con 0x7f535c0afdc0 2022-06-29T15:51:31.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.106+0000 7f5361d86700 1 -- 172.21.15.47:0/326201854 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f535c0afdc0 msgr2=0x7f535c0ae0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:31.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.106+0000 7f5361d86700 1 --2- 172.21.15.47:0/326201854 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f535c0afdc0 0x7f535c0ae0c0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 rx=0x7f53540032f0 tx=0x7f5354009290).stop 2022-06-29T15:51:31.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.106+0000 7f5361d86700 1 -- 172.21.15.47:0/326201854 shutdown_connections 2022-06-29T15:51:31.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.106+0000 7f5361d86700 1 --2- 172.21.15.47:0/326201854 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f535c0afdc0 0x7f535c0ae0c0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:31.127 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.106+0000 7f5361d86700 1 --2- 172.21.15.47:0/326201854 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f535c0b02d0 0x7f535c1406b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:31.127 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.106+0000 7f5361d86700 1 --2- 172.21.15.47:0/326201854 >> v2:172.21.15.64:3300/0 conn(0x7f535c0b5ac0 0x7f535c1401c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:31.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.106+0000 7f5361d86700 1 -- 172.21.15.47:0/326201854 >> 172.21.15.47:0/326201854 conn(0x7f535c062560 msgr2=0x7f535c062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:31.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.106+0000 7f5361d86700 1 -- 172.21.15.47:0/326201854 shutdown_connections 2022-06-29T15:51:31.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.106+0000 7f5361d86700 1 -- 172.21.15.47:0/326201854 wait complete. 2022-06-29T15:51:31.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.110+0000 7f5361d86700 1 Processor -- start 2022-06-29T15:51:31.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.110+0000 7f5361d86700 1 -- start start 2022-06-29T15:51:31.134 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.110+0000 7f5361d86700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f535c0afdc0 0x7f535c13ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:31.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.110+0000 7f5361d86700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f535c0b02d0 0x7f535c152830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:31.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.110+0000 7f5361d86700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f535c0b5ac0 0x7f535c152d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:31.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.110+0000 7f5361d86700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535c1521a0 con 0x7f535c0b5ac0 2022-06-29T15:51:31.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.110+0000 7f5361d86700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535c1454d0 con 0x7f535c0afdc0 2022-06-29T15:51:31.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.110+0000 7f5361d86700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f535c156100 con 0x7f535c0b02d0 2022-06-29T15:51:31.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.110+0000 7f5361585700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f535c0b5ac0 0x7f535c152d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:31.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.110+0000 7f5360d84700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f535c0afdc0 0x7f535c13ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:31.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.110+0000 7f5361585700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f535c0b5ac0 0x7f535c152d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53814/0 (socket says 172.21.15.47:53814) 2022-06-29T15:51:31.138 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.110+0000 7f5353fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f535c0b02d0 0x7f535c152830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:31.138 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.110+0000 7f5361585700 1 -- 172.21.15.47:0/3036139620 learned_addr learned my addr 172.21.15.47:0/3036139620 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:31.139 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.110+0000 7f5360d84700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f535c0afdc0 0x7f535c13ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:59514/0 (socket says 172.21.15.47:59514) 2022-06-29T15:51:31.139 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.114+0000 7f5361585700 1 -- 172.21.15.47:0/3036139620 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f535c0b02d0 msgr2=0x7f535c152830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:31.139 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.114+0000 7f5361585700 1 --2- 172.21.15.47:0/3036139620 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f535c0b02d0 0x7f535c152830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:31.140 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.114+0000 7f5361585700 1 -- 172.21.15.47:0/3036139620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f535c0afdc0 msgr2=0x7f535c13ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:31.140 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.114+0000 7f5361585700 1 --2- 172.21.15.47:0/3036139620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f535c0afdc0 0x7f535c13ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:31.140 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.114+0000 7f5361585700 1 -- 172.21.15.47:0/3036139620 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5354009530 con 0x7f535c0b5ac0 2022-06-29T15:51:31.141 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.114+0000 7f5361585700 1 --2- 172.21.15.47:0/3036139620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f535c0b5ac0 0x7f535c152d20 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 rx=0x7f53540032f0 tx=0x7f5354017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:31.141 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.114+0000 7f5351ffb700 1 -- 172.21.15.47:0/3036139620 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f535400b680 con 0x7f535c0b5ac0 2022-06-29T15:51:31.142 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.114+0000 7f5351ffb700 1 -- 172.21.15.47:0/3036139620 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f535400bea0 con 0x7f535c0b5ac0 2022-06-29T15:51:31.142 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.114+0000 7f5361d86700 1 -- 172.21.15.47:0/3036139620 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f535c1563a0 con 0x7f535c0b5ac0 2022-06-29T15:51:31.142 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.114+0000 7f5351ffb700 1 -- 172.21.15.47:0/3036139620 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f535401b600 con 0x7f535c0b5ac0 2022-06-29T15:51:31.143 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.114+0000 7f5361d86700 1 -- 172.21.15.47:0/3036139620 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f535c0aa260 con 0x7f535c0b5ac0 2022-06-29T15:51:31.143 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.114+0000 7f5361d86700 1 -- 172.21.15.47:0/3036139620 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f535c1565a0 con 0x7f535c0b5ac0 2022-06-29T15:51:31.144 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.114+0000 7f5351ffb700 1 -- 172.21.15.47:0/3036139620 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f535400f070 con 0x7f535c0b5ac0 2022-06-29T15:51:31.144 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.114+0000 7f5351ffb700 1 --2- 172.21.15.47:0/3036139620 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f533c053980 0x7f533c055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:31.144 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.118+0000 7f5351ffb700 1 -- 172.21.15.47:0/3036139620 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f535406d3d0 con 0x7f535c0b5ac0 2022-06-29T15:51:31.145 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.118+0000 7f5360d84700 1 --2- 172.21.15.47:0/3036139620 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f533c053980 0x7f533c055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:31.145 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.118+0000 7f5360d84700 1 --2- 172.21.15.47:0/3036139620 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f533c053980 0x7f533c055e40 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f5358006000 tx=0x7f5358006d90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:31.146 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.122+0000 7f5351ffb700 1 -- 172.21.15.47:0/3036139620 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f53540441d0 con 0x7f535c0b5ac0 2022-06-29T15:51:31.260 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.254+0000 7f5361d86700 1 -- 172.21.15.47:0/3036139620 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f535c0573d0 con 0x7f535c0b5ac0 2022-06-29T15:51:31.262 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.254+0000 7f5351ffb700 1 -- 172.21.15.47:0/3036139620 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f535401f070 con 0x7f535c0b5ac0 2022-06-29T15:51:31.263 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:31.263 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":140,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:51:24.546533+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:51:21.506960+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2022-06-29T15:51:22.546983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:51:31.265 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.258+0000 7f5361d86700 1 -- 172.21.15.47:0/3036139620 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f533c053980 msgr2=0x7f533c055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:31.265 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.258+0000 7f5361d86700 1 --2- 172.21.15.47:0/3036139620 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f533c053980 0x7f533c055e40 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f5358006000 tx=0x7f5358006d90).stop 2022-06-29T15:51:31.266 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.258+0000 7f5361d86700 1 -- 172.21.15.47:0/3036139620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f535c0b5ac0 msgr2=0x7f535c152d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:31.266 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.258+0000 7f5361d86700 1 --2- 172.21.15.47:0/3036139620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f535c0b5ac0 0x7f535c152d20 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 rx=0x7f53540032f0 tx=0x7f5354017000).stop 2022-06-29T15:51:31.266 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.258+0000 7f5361d86700 1 -- 172.21.15.47:0/3036139620 shutdown_connections 2022-06-29T15:51:31.266 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.258+0000 7f5361d86700 1 --2- 172.21.15.47:0/3036139620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f535c0b5ac0 0x7f535c152d20 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:31.267 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.258+0000 7f5361d86700 1 --2- 172.21.15.47:0/3036139620 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f535c0b02d0 0x7f535c152830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:31.267 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.258+0000 7f5361d86700 1 --2- 172.21.15.47:0/3036139620 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f533c053980 0x7f533c055e40 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:31.267 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.258+0000 7f5361d86700 1 --2- 172.21.15.47:0/3036139620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f535c0afdc0 0x7f535c13ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:31.267 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.258+0000 7f5361d86700 1 -- 172.21.15.47:0/3036139620 >> 172.21.15.47:0/3036139620 conn(0x7f535c062560 msgr2=0x7f535c13d860 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:31.267 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.258+0000 7f5361d86700 1 -- 172.21.15.47:0/3036139620 shutdown_connections 2022-06-29T15:51:31.268 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.258+0000 7f5361d86700 1 -- 172.21.15.47:0/3036139620 wait complete. 2022-06-29T15:51:31.277 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-29T15:42:16.971431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-29T15:42:25.290884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-29T15:51:21.506960+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '139', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2022-06-29T15:51:22.546983+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '140', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-29T15:51:31.278 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-29T15:51:31.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.386+0000 7f6c5a43c700 1 Processor -- start 2022-06-29T15:51:31.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.386+0000 7f6c5a43c700 1 -- start start 2022-06-29T15:51:31.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.390+0000 7f6c5a43c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6c5405fc50 0x7f6c54144cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:31.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.390+0000 7f6c5a43c700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c540b02c0 con 0x7f6c5405f3f0 2022-06-29T15:51:31.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.390+0000 7f6c5a43c700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c5405f7d0 con 0x7f6c540b17a0 2022-06-29T15:51:31.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.390+0000 7f6c5a43c700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c541481e0 con 0x7f6c5405fc50 2022-06-29T15:51:31.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.390+0000 7f6c5943a700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f6c5405f3f0 0x7f6c541445b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39372/0 (socket says 172.21.15.47:39372) 2022-06-29T15:51:31.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.390+0000 7f6c5943a700 1 -- 172.21.15.47:0/2084186822 learned_addr learned my addr 172.21.15.47:0/2084186822 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:31.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.390+0000 7f6c58c39700 1 --2- 172.21.15.47:0/2084186822 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6c5405fc50 0x7f6c54144cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:31.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.390+0000 7f6c43fff700 1 -- 172.21.15.47:0/2084186822 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1611203173 0 0) 0x7f6c540b02c0 con 0x7f6c5405f3f0 2022-06-29T15:51:31.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.390+0000 7f6c43fff700 1 -- 172.21.15.47:0/2084186822 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c3c003260 con 0x7f6c5405f3f0 2022-06-29T15:51:31.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.390+0000 7f6c43fff700 1 -- 172.21.15.47:0/2084186822 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1930478495 0 0) 0x7f6c5405f7d0 con 0x7f6c540b17a0 2022-06-29T15:51:31.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.390+0000 7f6c43fff700 1 -- 172.21.15.47:0/2084186822 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c540b02c0 con 0x7f6c540b17a0 2022-06-29T15:51:31.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.390+0000 7f6c58c39700 1 -- 172.21.15.47:0/2084186822 >> v1:172.21.15.64:6789/0 conn(0x7f6c540b17a0 legacy=0x7f6c540a7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:31.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.390+0000 7f6c58c39700 1 -- 172.21.15.47:0/2084186822 >> v1:172.21.15.47:6789/0 conn(0x7f6c5405f3f0 legacy=0x7f6c541445b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:31.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.390+0000 7f6c43fff700 1 -- 172.21.15.47:0/2084186822 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1951884394 0 0) 0x7f6c3c003260 con 0x7f6c5405f3f0 2022-06-29T15:51:31.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.390+0000 7f6c58c39700 1 -- 172.21.15.47:0/2084186822 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c540b04f0 con 0x7f6c5405fc50 2022-06-29T15:51:31.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.390+0000 7f6c58c39700 1 --2- 172.21.15.47:0/2084186822 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6c5405fc50 0x7f6c54144cc0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f6c50001800 tx=0x7f6c44006df0).ready entity=mon.2 client_cookie=673c2c64a21df4b0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:31.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.390+0000 7f6c43fff700 1 -- 172.21.15.47:0/2084186822 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c50001660 con 0x7f6c5405fc50 2022-06-29T15:51:31.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.390+0000 7f6c43fff700 1 -- 172.21.15.47:0/2084186822 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c44000e90 con 0x7f6c5405fc50 2022-06-29T15:51:31.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.390+0000 7f6c43fff700 1 -- 172.21.15.47:0/2084186822 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c44012460 con 0x7f6c5405fc50 2022-06-29T15:51:31.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.390+0000 7f6c5a43c700 1 -- 172.21.15.47:0/2084186822 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6c5405fc50 msgr2=0x7f6c54144cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:31.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.390+0000 7f6c5a43c700 1 --2- 172.21.15.47:0/2084186822 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6c5405fc50 0x7f6c54144cc0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f6c50001800 tx=0x7f6c44006df0).stop 2022-06-29T15:51:31.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.390+0000 7f6c5a43c700 1 -- 172.21.15.47:0/2084186822 shutdown_connections 2022-06-29T15:51:31.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.390+0000 7f6c5a43c700 1 --2- 172.21.15.47:0/2084186822 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6c5405fc50 0x7f6c54144cc0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:31.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.390+0000 7f6c5a43c700 1 -- 172.21.15.47:0/2084186822 >> 172.21.15.47:0/2084186822 conn(0x7f6c540b5320 msgr2=0x7f6c540b5700 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:31.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.390+0000 7f6c5a43c700 1 -- 172.21.15.47:0/2084186822 shutdown_connections 2022-06-29T15:51:31.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.390+0000 7f6c5a43c700 1 -- 172.21.15.47:0/2084186822 wait complete. 2022-06-29T15:51:31.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.394+0000 7f6c5a43c700 1 Processor -- start 2022-06-29T15:51:31.409 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.394+0000 7f6c5a43c700 1 -- start start 2022-06-29T15:51:31.409 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.394+0000 7f6c5a43c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6c5405f3f0 0x7f6c5414b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:31.409 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.394+0000 7f6c5a43c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c5405fc50 0x7f6c540ab4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:31.409 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.394+0000 7f6c5a43c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c540b17a0 0x7f6c540ab9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:31.410 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.394+0000 7f6c5a43c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c54148c30 con 0x7f6c5405fc50 2022-06-29T15:51:31.410 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.394+0000 7f6c5a43c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5414a1f0 con 0x7f6c540b17a0 2022-06-29T15:51:31.410 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.394+0000 7f6c5a43c700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c540abe90 con 0x7f6c5405f3f0 2022-06-29T15:51:31.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.394+0000 7f6c58c39700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c5405fc50 0x7f6c540ab4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:31.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.394+0000 7f6c5943a700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6c5405f3f0 0x7f6c5414b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:31.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.394+0000 7f6c58c39700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c5405fc50 0x7f6c540ab4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53828/0 (socket says 172.21.15.47:53828) 2022-06-29T15:51:31.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.394+0000 7f6c59c3b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c540b17a0 0x7f6c540ab9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:31.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.394+0000 7f6c58c39700 1 -- 172.21.15.47:0/2319531672 learned_addr learned my addr 172.21.15.47:0/2319531672 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:31.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.394+0000 7f6c58c39700 1 -- 172.21.15.47:0/2319531672 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6c5405f3f0 msgr2=0x7f6c5414b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:31.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.394+0000 7f6c58c39700 1 --2- 172.21.15.47:0/2319531672 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6c5405f3f0 0x7f6c5414b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:31.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.394+0000 7f6c58c39700 1 -- 172.21.15.47:0/2319531672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c540b17a0 msgr2=0x7f6c540ab9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:31.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.394+0000 7f6c58c39700 1 --2- 172.21.15.47:0/2319531672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c540b17a0 0x7f6c540ab9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:31.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.394+0000 7f6c58c39700 1 -- 172.21.15.47:0/2319531672 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c4400b180 con 0x7f6c5405fc50 2022-06-29T15:51:31.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.394+0000 7f6c59c3b700 1 --2- 172.21.15.47:0/2319531672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c540b17a0 0x7f6c540ab9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:51:31.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.394+0000 7f6c5943a700 1 --2- 172.21.15.47:0/2319531672 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6c5405f3f0 0x7f6c5414b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:51:31.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.394+0000 7f6c58c39700 1 --2- 172.21.15.47:0/2319531672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c5405fc50 0x7f6c540ab4b0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 rx=0x7f6c50001800 tx=0x7f6c44014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:31.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.394+0000 7f6c427fc700 1 -- 172.21.15.47:0/2319531672 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c44016030 con 0x7f6c5405fc50 2022-06-29T15:51:31.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.394+0000 7f6c427fc700 1 -- 172.21.15.47:0/2319531672 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c4400db50 con 0x7f6c5405fc50 2022-06-29T15:51:31.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.394+0000 7f6c427fc700 1 -- 172.21.15.47:0/2319531672 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c44012460 con 0x7f6c5405fc50 2022-06-29T15:51:31.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.394+0000 7f6c5a43c700 1 -- 172.21.15.47:0/2319531672 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c5414a450 con 0x7f6c5405fc50 2022-06-29T15:51:31.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.394+0000 7f6c5a43c700 1 -- 172.21.15.47:0/2319531672 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c540aaa50 con 0x7f6c5405fc50 2022-06-29T15:51:31.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.398+0000 7f6c5a43c700 1 -- 172.21.15.47:0/2319531672 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c540aae90 con 0x7f6c5405fc50 2022-06-29T15:51:31.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.398+0000 7f6c427fc700 1 -- 172.21.15.47:0/2319531672 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6c44024020 con 0x7f6c5405fc50 2022-06-29T15:51:31.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.398+0000 7f6c427fc700 1 --2- 172.21.15.47:0/2319531672 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6c34053b30 0x7f6c34055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:31.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.398+0000 7f6c427fc700 1 -- 172.21.15.47:0/2319531672 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f6c44017070 con 0x7f6c5405fc50 2022-06-29T15:51:31.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.398+0000 7f6c5943a700 1 --2- 172.21.15.47:0/2319531672 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6c34053b30 0x7f6c34055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:31.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.398+0000 7f6c5943a700 1 --2- 172.21.15.47:0/2319531672 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6c34053b30 0x7f6c34055ff0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7f6c5000d000 tx=0x7f6c50001740).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:31.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.402+0000 7f6c427fc700 1 -- 172.21.15.47:0/2319531672 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6c44043c60 con 0x7f6c5405fc50 2022-06-29T15:51:31.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.526+0000 7f6c5a43c700 1 -- 172.21.15.47:0/2319531672 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f6c540573d0 con 0x7f6c5405fc50 2022-06-29T15:51:31.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.526+0000 7f6c427fc700 1 -- 172.21.15.47:0/2319531672 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v140) v1 ==== 110+0+10 (secure 0 0 0) 0x7f6c44047280 con 0x7f6c5405fc50 2022-06-29T15:51:31.535 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2022-06-29T15:51:31.538 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.530+0000 7f6c5a43c700 1 -- 172.21.15.47:0/2319531672 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6c34053b30 msgr2=0x7f6c34055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:31.539 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.530+0000 7f6c5a43c700 1 --2- 172.21.15.47:0/2319531672 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6c34053b30 0x7f6c34055ff0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7f6c5000d000 tx=0x7f6c50001740).stop 2022-06-29T15:51:31.539 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.530+0000 7f6c5a43c700 1 -- 172.21.15.47:0/2319531672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c5405fc50 msgr2=0x7f6c540ab4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:31.539 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.530+0000 7f6c5a43c700 1 --2- 172.21.15.47:0/2319531672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c5405fc50 0x7f6c540ab4b0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 rx=0x7f6c50001800 tx=0x7f6c44014000).stop 2022-06-29T15:51:31.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.530+0000 7f6c5a43c700 1 -- 172.21.15.47:0/2319531672 shutdown_connections 2022-06-29T15:51:31.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.530+0000 7f6c5a43c700 1 --2- 172.21.15.47:0/2319531672 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6c34053b30 0x7f6c34055ff0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:31.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.530+0000 7f6c5a43c700 1 --2- 172.21.15.47:0/2319531672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c540b17a0 0x7f6c540ab9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:31.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.530+0000 7f6c5a43c700 1 --2- 172.21.15.47:0/2319531672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c5405fc50 0x7f6c540ab4b0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:31.541 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.530+0000 7f6c5a43c700 1 --2- 172.21.15.47:0/2319531672 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6c5405f3f0 0x7f6c5414b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:31.541 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.530+0000 7f6c5a43c700 1 -- 172.21.15.47:0/2319531672 >> 172.21.15.47:0/2319531672 conn(0x7f6c540b5320 msgr2=0x7f6c540a7240 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:31.541 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.530+0000 7f6c5a43c700 1 -- 172.21.15.47:0/2319531672 shutdown_connections 2022-06-29T15:51:31.541 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.530+0000 7f6c5a43c700 1 -- 172.21.15.47:0/2319531672 wait complete. 2022-06-29T15:51:31.552 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-29T15:51:31.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.654+0000 7f7a9e83a700 1 Processor -- start 2022-06-29T15:51:31.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.654+0000 7f7a9e83a700 1 -- start start 2022-06-29T15:51:31.664 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.654+0000 7f7a9e83a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7a98060ee0 0x7f7a980612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:31.664 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.654+0000 7f7a9e83a700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7a980b4300 0x7f7a981471a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:31.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.654+0000 7f7a9e83a700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a98067610 con 0x7f7a98066f70 2022-06-29T15:51:31.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.654+0000 7f7a9e83a700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a98067400 con 0x7f7a98060ee0 2022-06-29T15:51:31.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.654+0000 7f7a9e83a700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a981467c0 con 0x7f7a980b4300 2022-06-29T15:51:31.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.654+0000 7f7a9cdd7700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7a98066f70 0x7f7a981430e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39390/0 (socket says 172.21.15.47:39390) 2022-06-29T15:51:31.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.654+0000 7f7a9cdd7700 1 -- 172.21.15.47:0/3278932644 learned_addr learned my addr 172.21.15.47:0/3278932644 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:31.666 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.654+0000 7f7a97fff700 1 --2- 172.21.15.47:0/3278932644 >> v2:172.21.15.47:3300/0 conn(0x7f7a98060ee0 0x7f7a980612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:31.666 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.654+0000 7f7a977fe700 1 --2- 172.21.15.47:0/3278932644 >> v2:172.21.15.64:3300/0 conn(0x7f7a980b4300 0x7f7a981471a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:31.666 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.654+0000 7f7a96ffd700 1 -- 172.21.15.47:0/3278932644 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 373995838 0 0) 0x7f7a98067610 con 0x7f7a98066f70 2022-06-29T15:51:31.666 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.654+0000 7f7a96ffd700 1 -- 172.21.15.47:0/3278932644 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a84003260 con 0x7f7a98066f70 2022-06-29T15:51:31.667 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.654+0000 7f7a96ffd700 1 -- 172.21.15.47:0/3278932644 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 679434392 0 0) 0x7f7a84003260 con 0x7f7a98066f70 2022-06-29T15:51:31.667 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.654+0000 7f7a96ffd700 1 -- 172.21.15.47:0/3278932644 >> v2:172.21.15.64:3300/0 conn(0x7f7a980b4300 msgr2=0x7f7a981471a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:31.667 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.654+0000 7f7a96ffd700 1 --2- 172.21.15.47:0/3278932644 >> v2:172.21.15.64:3300/0 conn(0x7f7a980b4300 0x7f7a981471a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:31.667 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.654+0000 7f7a96ffd700 1 -- 172.21.15.47:0/3278932644 >> v2:172.21.15.47:3300/0 conn(0x7f7a98060ee0 msgr2=0x7f7a980612c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:31.668 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.654+0000 7f7a96ffd700 1 --2- 172.21.15.47:0/3278932644 >> v2:172.21.15.47:3300/0 conn(0x7f7a98060ee0 0x7f7a980612c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:31.668 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.654+0000 7f7a96ffd700 1 -- 172.21.15.47:0/3278932644 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a98146a60 con 0x7f7a98066f70 2022-06-29T15:51:31.668 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.654+0000 7f7a96ffd700 1 -- 172.21.15.47:0/3278932644 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7f7a900019e0 con 0x7f7a98066f70 2022-06-29T15:51:31.668 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.654+0000 7f7a97fff700 1 --2- 172.21.15.47:0/3278932644 >> v2:172.21.15.47:3300/0 conn(0x7f7a98060ee0 0x7f7a980612c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:51:31.669 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.654+0000 7f7a977fe700 1 --2- 172.21.15.47:0/3278932644 >> v2:172.21.15.64:3300/0 conn(0x7f7a980b4300 0x7f7a981471a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:51:31.669 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.654+0000 7f7a96ffd700 1 -- 172.21.15.47:0/3278932644 >> v1:172.21.15.47:6789/0 conn(0x7f7a98066f70 legacy=0x7f7a981430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:31.669 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.654+0000 7f7a96ffd700 1 --2- 172.21.15.47:0/3278932644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a84003750 0x7f7a84003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:31.669 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.654+0000 7f7a96ffd700 1 --2- 172.21.15.47:0/3278932644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a84004500 0x7f7a84004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:31.669 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.654+0000 7f7a96ffd700 1 --2- 172.21.15.47:0/3278932644 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7a84004e70 0x7f7a8400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:31.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.654+0000 7f7a96ffd700 1 -- 172.21.15.47:0/3278932644 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a98067400 con 0x7f7a84003750 2022-06-29T15:51:31.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.654+0000 7f7a96ffd700 1 -- 172.21.15.47:0/3278932644 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a981467c0 con 0x7f7a84004500 2022-06-29T15:51:31.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.654+0000 7f7a96ffd700 1 -- 172.21.15.47:0/3278932644 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8400bc40 con 0x7f7a84004e70 2022-06-29T15:51:31.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.654+0000 7f7a97fff700 1 --2- 172.21.15.47:0/3278932644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a84003750 0x7f7a84003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:31.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.654+0000 7f7a977fe700 1 --2- 172.21.15.47:0/3278932644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a84004500 0x7f7a84004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:31.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.654+0000 7f7a9cdd7700 1 --2- 172.21.15.47:0/3278932644 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7a84004e70 0x7f7a8400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:31.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.654+0000 7f7a97fff700 1 -- 172.21.15.47:0/3278932644 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7a84004e70 msgr2=0x7f7a8400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:31.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.654+0000 7f7a97fff700 1 --2- 172.21.15.47:0/3278932644 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7a84004e70 0x7f7a8400b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:31.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.654+0000 7f7a97fff700 1 -- 172.21.15.47:0/3278932644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a84004500 msgr2=0x7f7a84004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:31.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.654+0000 7f7a97fff700 1 --2- 172.21.15.47:0/3278932644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a84004500 0x7f7a84004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:31.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.654+0000 7f7a97fff700 1 -- 172.21.15.47:0/3278932644 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a84003260 con 0x7f7a84003750 2022-06-29T15:51:31.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.654+0000 7f7a9cdd7700 1 -- 172.21.15.47:0/3278932644 reap_dead start 2022-06-29T15:51:31.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.654+0000 7f7a97fff700 1 --2- 172.21.15.47:0/3278932644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a84003750 0x7f7a84003b70 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 rx=0x7f7a8800c000 tx=0x7f7a88006df0).ready entity=mon.0 client_cookie=962b41290adb366e server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:31.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.658+0000 7f7a96ffd700 1 -- 172.21.15.47:0/3278932644 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a8800e7d0 con 0x7f7a84003750 2022-06-29T15:51:31.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.658+0000 7f7a96ffd700 1 -- 172.21.15.47:0/3278932644 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a8800e930 con 0x7f7a84003750 2022-06-29T15:51:31.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.658+0000 7f7a96ffd700 1 -- 172.21.15.47:0/3278932644 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a8800ec90 con 0x7f7a84003750 2022-06-29T15:51:31.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.658+0000 7f7a9e83a700 1 -- 172.21.15.47:0/3278932644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a84003750 msgr2=0x7f7a84003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:31.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.658+0000 7f7a9e83a700 1 --2- 172.21.15.47:0/3278932644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a84003750 0x7f7a84003b70 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 rx=0x7f7a8800c000 tx=0x7f7a88006df0).stop 2022-06-29T15:51:31.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.658+0000 7f7a9e83a700 1 -- 172.21.15.47:0/3278932644 shutdown_connections 2022-06-29T15:51:31.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.658+0000 7f7a9e83a700 1 --2- 172.21.15.47:0/3278932644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a84003750 0x7f7a84003b70 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:31.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.658+0000 7f7a9e83a700 1 -- 172.21.15.47:0/3278932644 >> 172.21.15.47:0/3278932644 conn(0x7f7a9805c510 msgr2=0x7f7a9805f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:31.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.658+0000 7f7a9e83a700 1 -- 172.21.15.47:0/3278932644 shutdown_connections 2022-06-29T15:51:31.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.658+0000 7f7a9e83a700 1 -- 172.21.15.47:0/3278932644 wait complete. 2022-06-29T15:51:31.678 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.658+0000 7f7a9e83a700 1 Processor -- start 2022-06-29T15:51:31.679 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.658+0000 7f7a9e83a700 1 -- start start 2022-06-29T15:51:31.679 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.658+0000 7f7a9e83a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a84003750 0x7f7a981421c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:31.679 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.658+0000 7f7a9e83a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a981426b0 0x7f7a9813a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:31.679 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.658+0000 7f7a9e83a700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7a9813ad90 0x7f7a98145470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:31.680 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.658+0000 7f7a9e83a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a981413c0 con 0x7f7a981426b0 2022-06-29T15:51:31.680 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.658+0000 7f7a9e83a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a98149070 con 0x7f7a84003750 2022-06-29T15:51:31.680 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.658+0000 7f7a9e83a700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a98144b60 con 0x7f7a9813ad90 2022-06-29T15:51:31.680 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.658+0000 7f7a977fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a981426b0 0x7f7a9813a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:31.681 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.658+0000 7f7a977fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a981426b0 0x7f7a9813a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53846/0 (socket says 172.21.15.47:53846) 2022-06-29T15:51:31.681 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.658+0000 7f7a977fe700 1 -- 172.21.15.47:0/3417403163 learned_addr learned my addr 172.21.15.47:0/3417403163 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:31.681 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.658+0000 7f7a97fff700 1 --2- 172.21.15.47:0/3417403163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a84003750 0x7f7a981421c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:31.681 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.658+0000 7f7a977fe700 1 -- 172.21.15.47:0/3417403163 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7a9813ad90 msgr2=0x7f7a98145470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:31.682 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.658+0000 7f7a977fe700 1 --2- 172.21.15.47:0/3417403163 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7a9813ad90 0x7f7a98145470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2022-06-29T15:51:31.682 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.658+0000 7f7a977fe700 1 -- 172.21.15.47:0/3417403163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a84003750 msgr2=0x7f7a981421c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:31.682 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.658+0000 7f7a977fe700 1 --2- 172.21.15.47:0/3417403163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a84003750 0x7f7a981421c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:31.682 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.658+0000 7f7a977fe700 1 -- 172.21.15.47:0/3417403163 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a9814a240 con 0x7f7a981426b0 2022-06-29T15:51:31.683 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.658+0000 7f7a97fff700 1 --2- 172.21.15.47:0/3417403163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a84003750 0x7f7a981421c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:51:31.683 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.658+0000 7f7a977fe700 1 --2- 172.21.15.47:0/3417403163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a981426b0 0x7f7a9813a8a0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 rx=0x7f7a98145030 tx=0x7f7a8c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:31.683 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.658+0000 7f7a957fa700 1 -- 172.21.15.47:0/3417403163 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a8c00fa70 con 0x7f7a981426b0 2022-06-29T15:51:31.683 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.658+0000 7f7a957fa700 1 -- 172.21.15.47:0/3417403163 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a8c00fbd0 con 0x7f7a981426b0 2022-06-29T15:51:31.683 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.658+0000 7f7a9e83a700 1 -- 172.21.15.47:0/3417403163 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a98144d90 con 0x7f7a981426b0 2022-06-29T15:51:31.684 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.658+0000 7f7a957fa700 1 -- 172.21.15.47:0/3417403163 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a8c00fd40 con 0x7f7a981426b0 2022-06-29T15:51:31.684 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.658+0000 7f7a9e83a700 1 -- 172.21.15.47:0/3417403163 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a9813b210 con 0x7f7a981426b0 2022-06-29T15:51:31.684 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.662+0000 7f7a9e83a700 1 -- 172.21.15.47:0/3417403163 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a980573d0 con 0x7f7a981426b0 2022-06-29T15:51:31.684 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.662+0000 7f7a957fa700 1 -- 172.21.15.47:0/3417403163 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7a8c014440 con 0x7f7a981426b0 2022-06-29T15:51:31.685 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.662+0000 7f7a957fa700 1 --2- 172.21.15.47:0/3417403163 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7a78053b00 0x7f7a78055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:31.685 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.662+0000 7f7a97fff700 1 --2- 172.21.15.47:0/3417403163 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7a78053b00 0x7f7a78055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:31.685 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.662+0000 7f7a957fa700 1 -- 172.21.15.47:0/3417403163 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f7a8c06cb10 con 0x7f7a981426b0 2022-06-29T15:51:31.685 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.662+0000 7f7a97fff700 1 --2- 172.21.15.47:0/3417403163 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7a78053b00 0x7f7a78055fc0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7f7a8801a000 tx=0x7f7a88019000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:31.685 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.670+0000 7f7a957fa700 1 -- 172.21.15.47:0/3417403163 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7a8c043760 con 0x7f7a981426b0 2022-06-29T15:51:31.801 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.794+0000 7f7a9e83a700 1 -- 172.21.15.47:0/3417403163 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7a980b5e50 con 0x7f7a981426b0 2022-06-29T15:51:31.802 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.794+0000 7f7a957fa700 1 -- 172.21.15.47:0/3417403163 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v140) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7a8c00fea0 con 0x7f7a981426b0 2022-06-29T15:51:31.802 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:51:31.806 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.798+0000 7f7a9e83a700 1 -- 172.21.15.47:0/3417403163 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7a78053b00 msgr2=0x7f7a78055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:31.806 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.798+0000 7f7a9e83a700 1 --2- 172.21.15.47:0/3417403163 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7a78053b00 0x7f7a78055fc0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7f7a8801a000 tx=0x7f7a88019000).stop 2022-06-29T15:51:31.806 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.798+0000 7f7a9e83a700 1 -- 172.21.15.47:0/3417403163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a981426b0 msgr2=0x7f7a9813a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:31.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.798+0000 7f7a9e83a700 1 --2- 172.21.15.47:0/3417403163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a981426b0 0x7f7a9813a8a0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 rx=0x7f7a98145030 tx=0x7f7a8c006df0).stop 2022-06-29T15:51:31.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.798+0000 7f7a9e83a700 1 -- 172.21.15.47:0/3417403163 shutdown_connections 2022-06-29T15:51:31.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.798+0000 7f7a9e83a700 1 --2- 172.21.15.47:0/3417403163 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7a9813ad90 0x7f7a98145470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2022-06-29T15:51:31.807 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.798+0000 7f7a9e83a700 1 --2- 172.21.15.47:0/3417403163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a981426b0 0x7f7a9813a8a0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:31.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.798+0000 7f7a9e83a700 1 --2- 172.21.15.47:0/3417403163 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7a78053b00 0x7f7a78055fc0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:31.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.798+0000 7f7a9e83a700 1 --2- 172.21.15.47:0/3417403163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a84003750 0x7f7a981421c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:31.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.798+0000 7f7a9e83a700 1 -- 172.21.15.47:0/3417403163 >> 172.21.15.47:0/3417403163 conn(0x7f7a9805c510 msgr2=0x7f7a98140170 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:31.808 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.798+0000 7f7a9e83a700 1 -- 172.21.15.47:0/3417403163 shutdown_connections 2022-06-29T15:51:31.809 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.798+0000 7f7a9e83a700 1 -- 172.21.15.47:0/3417403163 wait complete. 2022-06-29T15:51:31.819 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-29T15:51:31.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.922+0000 7fd38632d700 1 Processor -- start 2022-06-29T15:51:31.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.922+0000 7fd38632d700 1 -- start start 2022-06-29T15:51:31.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.922+0000 7fd38632d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd3800b58d0 0x7fd3800b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:31.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.922+0000 7fd38632d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd3800b1610 0x7fd3801475f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:31.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.922+0000 7fd38632d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd38014b4c0 0x7fd3800aa130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:31.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.922+0000 7fd38632d700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3800a99e0 con 0x7fd3800b58d0 2022-06-29T15:51:31.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.922+0000 7fd38632d700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd380060790 con 0x7fd3800b1610 2022-06-29T15:51:31.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.922+0000 7fd38632d700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd38014ad80 con 0x7fd38014b4c0 2022-06-29T15:51:31.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.922+0000 7fd38532b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd3800b58d0 0x7fd3800b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:31.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.922+0000 7fd38532b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd3800b58d0 0x7fd3800b5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53852/0 (socket says 172.21.15.47:53852) 2022-06-29T15:51:31.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.922+0000 7fd38532b700 1 -- 172.21.15.47:0/1771937500 learned_addr learned my addr 172.21.15.47:0/1771937500 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:31.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.922+0000 7fd385b2c700 1 --2- 172.21.15.47:0/1771937500 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd38014b4c0 0x7fd3800aa130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:31.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.922+0000 7fd384b2a700 1 --2- 172.21.15.47:0/1771937500 >> v2:172.21.15.64:3300/0 conn(0x7fd3800b1610 0x7fd3801475f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:31.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.922+0000 7fd38532b700 1 -- 172.21.15.47:0/1771937500 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd38014b4c0 msgr2=0x7fd3800aa130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:31.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.922+0000 7fd38532b700 1 --2- 172.21.15.47:0/1771937500 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd38014b4c0 0x7fd3800aa130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:31.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.922+0000 7fd38532b700 1 -- 172.21.15.47:0/1771937500 >> v2:172.21.15.64:3300/0 conn(0x7fd3800b1610 msgr2=0x7fd3801475f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:31.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.922+0000 7fd38532b700 1 --2- 172.21.15.47:0/1771937500 >> v2:172.21.15.64:3300/0 conn(0x7fd3800b1610 0x7fd3801475f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:31.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.922+0000 7fd38532b700 1 -- 172.21.15.47:0/1771937500 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd38014b020 con 0x7fd3800b58d0 2022-06-29T15:51:31.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.922+0000 7fd385b2c700 1 --2- 172.21.15.47:0/1771937500 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd38014b4c0 0x7fd3800aa130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:51:31.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.926+0000 7fd38532b700 1 --2- 172.21.15.47:0/1771937500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3800b58d0 0x7fd3800b5cb0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 rx=0x7fd3700032f0 tx=0x7fd370009290).ready entity=mon.0 client_cookie=4d0cf0d264faa9fb server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:31.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.926+0000 7fd37ffff700 1 -- 172.21.15.47:0/1771937500 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd37000b870 con 0x7fd3800b58d0 2022-06-29T15:51:31.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.926+0000 7fd37ffff700 1 -- 172.21.15.47:0/1771937500 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37000b9d0 con 0x7fd3800b58d0 2022-06-29T15:51:31.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.926+0000 7fd37ffff700 1 -- 172.21.15.47:0/1771937500 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd37000bb40 con 0x7fd3800b58d0 2022-06-29T15:51:31.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.926+0000 7fd38632d700 1 -- 172.21.15.47:0/1771937500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3800b58d0 msgr2=0x7fd3800b5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:31.938 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.926+0000 7fd38632d700 1 --2- 172.21.15.47:0/1771937500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3800b58d0 0x7fd3800b5cb0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 rx=0x7fd3700032f0 tx=0x7fd370009290).stop 2022-06-29T15:51:31.938 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.926+0000 7fd38632d700 1 -- 172.21.15.47:0/1771937500 shutdown_connections 2022-06-29T15:51:31.938 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.926+0000 7fd38632d700 1 --2- 172.21.15.47:0/1771937500 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd38014b4c0 0x7fd3800aa130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:31.938 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.926+0000 7fd38632d700 1 --2- 172.21.15.47:0/1771937500 >> v2:172.21.15.64:3300/0 conn(0x7fd3800b1610 0x7fd3801475f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:31.939 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.926+0000 7fd38632d700 1 --2- 172.21.15.47:0/1771937500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3800b58d0 0x7fd3800b5cb0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:31.939 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.926+0000 7fd38632d700 1 -- 172.21.15.47:0/1771937500 >> 172.21.15.47:0/1771937500 conn(0x7fd380062560 msgr2=0x7fd380062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:31.939 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.926+0000 7fd38632d700 1 -- 172.21.15.47:0/1771937500 shutdown_connections 2022-06-29T15:51:31.939 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.926+0000 7fd38632d700 1 -- 172.21.15.47:0/1771937500 wait complete. 2022-06-29T15:51:31.942 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.926+0000 7fd38632d700 1 Processor -- start 2022-06-29T15:51:31.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.926+0000 7fd38632d700 1 -- start start 2022-06-29T15:51:31.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.930+0000 7fd38632d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3800b1610 0x7fd380142040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:31.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.930+0000 7fd38632d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd3800b58d0 0x7fd380140b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:31.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.930+0000 7fd38632d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd38014b4c0 0x7fd380141080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:31.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.930+0000 7fd38632d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd380143360 con 0x7fd38014b4c0 2022-06-29T15:51:31.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.930+0000 7fd38632d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd380066360 con 0x7fd3800b1610 2022-06-29T15:51:31.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.930+0000 7fd38632d700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd380066600 con 0x7fd3800b58d0 2022-06-29T15:51:31.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.930+0000 7fd385b2c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd38014b4c0 0x7fd380141080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:31.944 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.930+0000 7fd384b2a700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd3800b58d0 0x7fd380140b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:31.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.930+0000 7fd385b2c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd38014b4c0 0x7fd380141080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53862/0 (socket says 172.21.15.47:53862) 2022-06-29T15:51:31.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.930+0000 7fd385b2c700 1 -- 172.21.15.47:0/2808316842 learned_addr learned my addr 172.21.15.47:0/2808316842 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:31.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.930+0000 7fd38532b700 1 --2- 172.21.15.47:0/2808316842 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3800b1610 0x7fd380142040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:31.945 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.930+0000 7fd385b2c700 1 -- 172.21.15.47:0/2808316842 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd3800b58d0 msgr2=0x7fd380140b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:31.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.930+0000 7fd385b2c700 1 --2- 172.21.15.47:0/2808316842 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd3800b58d0 0x7fd380140b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:31.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.930+0000 7fd385b2c700 1 -- 172.21.15.47:0/2808316842 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3800b1610 msgr2=0x7fd380142040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:31.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.930+0000 7fd385b2c700 1 --2- 172.21.15.47:0/2808316842 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3800b1610 0x7fd380142040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:31.946 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.930+0000 7fd385b2c700 1 -- 172.21.15.47:0/2808316842 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd370009530 con 0x7fd38014b4c0 2022-06-29T15:51:31.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.930+0000 7fd385b2c700 1 --2- 172.21.15.47:0/2808316842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd38014b4c0 0x7fd380141080 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 rx=0x7fd378006940 tx=0x7fd378008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:31.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.930+0000 7fd37e7fc700 1 -- 172.21.15.47:0/2808316842 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd378011590 con 0x7fd38014b4c0 2022-06-29T15:51:31.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.930+0000 7fd37e7fc700 1 -- 172.21.15.47:0/2808316842 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3780116f0 con 0x7fd38014b4c0 2022-06-29T15:51:31.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.930+0000 7fd38632d700 1 -- 172.21.15.47:0/2808316842 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd380065170 con 0x7fd38014b4c0 2022-06-29T15:51:31.947 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.930+0000 7fd37e7fc700 1 -- 172.21.15.47:0/2808316842 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd378011a50 con 0x7fd38014b4c0 2022-06-29T15:51:31.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.930+0000 7fd38632d700 1 -- 172.21.15.47:0/2808316842 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd380141570 con 0x7fd38014b4c0 2022-06-29T15:51:31.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.930+0000 7fd37e7fc700 1 -- 172.21.15.47:0/2808316842 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd378011bb0 con 0x7fd38014b4c0 2022-06-29T15:51:31.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.930+0000 7fd38632d700 1 -- 172.21.15.47:0/2808316842 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd38005ffe0 con 0x7fd38014b4c0 2022-06-29T15:51:31.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.934+0000 7fd37e7fc700 1 --2- 172.21.15.47:0/2808316842 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd360053950 0x7fd360055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:31.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.934+0000 7fd37e7fc700 1 -- 172.21.15.47:0/2808316842 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd37806c3a0 con 0x7fd38014b4c0 2022-06-29T15:51:31.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.934+0000 7fd38532b700 1 --2- 172.21.15.47:0/2808316842 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd360053950 0x7fd360055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:31.949 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.934+0000 7fd38532b700 1 --2- 172.21.15.47:0/2808316842 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd360053950 0x7fd360055e10 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7fd37001b000 tx=0x7fd370016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:31.950 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:31.938+0000 7fd37e7fc700 1 -- 172.21.15.47:0/2808316842 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd3780430c0 con 0x7fd38014b4c0 2022-06-29T15:51:32.072 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.066+0000 7fd38632d700 1 -- 172.21.15.47:0/2808316842 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd3800573d0 con 0x7fd38014b4c0 2022-06-29T15:51:32.072 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.066+0000 7fd37e7fc700 1 -- 172.21.15.47:0/2808316842 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v140) v1 ==== 104+0+10 (secure 0 0 0) 0x7fd3780466e0 con 0x7fd38014b4c0 2022-06-29T15:51:32.073 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:51:32.074 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.066+0000 7fd38632d700 1 -- 172.21.15.47:0/2808316842 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd360053950 msgr2=0x7fd360055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:32.075 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.066+0000 7fd38632d700 1 --2- 172.21.15.47:0/2808316842 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd360053950 0x7fd360055e10 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7fd37001b000 tx=0x7fd370016000).stop 2022-06-29T15:51:32.075 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.066+0000 7fd38632d700 1 -- 172.21.15.47:0/2808316842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd38014b4c0 msgr2=0x7fd380141080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:32.075 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.066+0000 7fd38632d700 1 --2- 172.21.15.47:0/2808316842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd38014b4c0 0x7fd380141080 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 rx=0x7fd378006940 tx=0x7fd378008040).stop 2022-06-29T15:51:32.075 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.066+0000 7fd38632d700 1 -- 172.21.15.47:0/2808316842 shutdown_connections 2022-06-29T15:51:32.076 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.066+0000 7fd38632d700 1 --2- 172.21.15.47:0/2808316842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd38014b4c0 0x7fd380141080 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:32.076 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.066+0000 7fd38632d700 1 --2- 172.21.15.47:0/2808316842 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd3800b58d0 0x7fd380140b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:32.076 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.066+0000 7fd38632d700 1 --2- 172.21.15.47:0/2808316842 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd360053950 0x7fd360055e10 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:32.076 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.066+0000 7fd38632d700 1 --2- 172.21.15.47:0/2808316842 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3800b1610 0x7fd380142040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:32.077 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.066+0000 7fd38632d700 1 -- 172.21.15.47:0/2808316842 >> 172.21.15.47:0/2808316842 conn(0x7fd380062560 msgr2=0x7fd3800b4880 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:32.077 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.066+0000 7fd38632d700 1 -- 172.21.15.47:0/2808316842 shutdown_connections 2022-06-29T15:51:32.077 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.066+0000 7fd38632d700 1 -- 172.21.15.47:0/2808316842 wait complete. 2022-06-29T15:51:32.091 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-29T15:51:32.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.202+0000 7f40abe42700 1 Processor -- start 2022-06-29T15:51:32.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.202+0000 7f40abe42700 1 -- start start 2022-06-29T15:51:32.212 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.202+0000 7f40abe42700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f40a40b0fe0 0x7f40a4061a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:32.212 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.202+0000 7f40abe42700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40a413db90 con 0x7f40a4061670 2022-06-29T15:51:32.212 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.202+0000 7f40abe42700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40a40af0a0 con 0x7f40a40b0700 2022-06-29T15:51:32.213 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.202+0000 7f40abe42700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40a413fed0 con 0x7f40a40b0fe0 2022-06-29T15:51:32.213 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.202+0000 7f40a9bde700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f40a4061670 0x7f40a413d480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39420/0 (socket says 172.21.15.47:39420) 2022-06-29T15:51:32.213 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.202+0000 7f40a9bde700 1 -- 172.21.15.47:0/1560511597 learned_addr learned my addr 172.21.15.47:0/1560511597 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:32.213 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.202+0000 7f40a93dd700 1 --2- 172.21.15.47:0/1560511597 >> v2:172.21.15.64:3300/0 conn(0x7f40a40b0fe0 0x7f40a4061a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:32.214 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.202+0000 7f40a8bdc700 1 -- 172.21.15.47:0/1560511597 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1671615659 0 0) 0x7f40a413db90 con 0x7f40a4061670 2022-06-29T15:51:32.214 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.202+0000 7f40a8bdc700 1 -- 172.21.15.47:0/1560511597 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f408c003260 con 0x7f40a4061670 2022-06-29T15:51:32.214 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.202+0000 7f40a8bdc700 1 -- 172.21.15.47:0/1560511597 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1889056819 0 0) 0x7f40a40af0a0 con 0x7f40a40b0700 2022-06-29T15:51:32.214 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.202+0000 7f40a8bdc700 1 -- 172.21.15.47:0/1560511597 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40a413db90 con 0x7f40a40b0700 2022-06-29T15:51:32.215 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.202+0000 7f40a8bdc700 1 -- 172.21.15.47:0/1560511597 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 481348155 0 0) 0x7f408c003260 con 0x7f40a4061670 2022-06-29T15:51:32.215 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.202+0000 7f40a8bdc700 1 -- 172.21.15.47:0/1560511597 >> v2:172.21.15.64:3300/0 conn(0x7f40a40b0fe0 msgr2=0x7f40a4061a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:32.215 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.202+0000 7f40a8bdc700 1 --2- 172.21.15.47:0/1560511597 >> v2:172.21.15.64:3300/0 conn(0x7f40a40b0fe0 0x7f40a4061a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:32.215 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.202+0000 7f40a8bdc700 1 -- 172.21.15.47:0/1560511597 >> v1:172.21.15.64:6789/0 conn(0x7f40a40b0700 legacy=0x7f40a41436c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:32.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.202+0000 7f40a8bdc700 1 -- 172.21.15.47:0/1560511597 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a40af340 con 0x7f40a4061670 2022-06-29T15:51:32.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.202+0000 7f40a8bdc700 1 -- 172.21.15.47:0/1560511597 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7f4094001660 con 0x7f40a4061670 2022-06-29T15:51:32.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.202+0000 7f40a8bdc700 1 -- 172.21.15.47:0/1560511597 >> v1:172.21.15.47:6789/0 conn(0x7f40a4061670 legacy=0x7f40a413d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:32.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.202+0000 7f40a8bdc700 1 --2- 172.21.15.47:0/1560511597 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f408c0038e0 0x7f408c003ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:32.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.202+0000 7f40a8bdc700 1 --2- 172.21.15.47:0/1560511597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f408c004500 0x7f408c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:32.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.202+0000 7f40a8bdc700 1 --2- 172.21.15.47:0/1560511597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f408c004e70 0x7f408c00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:32.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.202+0000 7f40a8bdc700 1 -- 172.21.15.47:0/1560511597 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408c00bcf0 con 0x7f408c004500 2022-06-29T15:51:32.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.202+0000 7f40a8bdc700 1 -- 172.21.15.47:0/1560511597 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408c00be30 con 0x7f408c004e70 2022-06-29T15:51:32.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.202+0000 7f40a8bdc700 1 -- 172.21.15.47:0/1560511597 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f408c00bff0 con 0x7f408c0038e0 2022-06-29T15:51:32.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.202+0000 7f40a93dd700 1 --2- 172.21.15.47:0/1560511597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f408c004500 0x7f408c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:32.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.202+0000 7f40a9bde700 1 --2- 172.21.15.47:0/1560511597 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f408c0038e0 0x7f408c003ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:32.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.202+0000 7f40aa3df700 1 --2- 172.21.15.47:0/1560511597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f408c004e70 0x7f408c00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:32.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.202+0000 7f40a93dd700 1 -- 172.21.15.47:0/1560511597 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f408c0038e0 msgr2=0x7f408c003ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:32.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.202+0000 7f40a93dd700 1 --2- 172.21.15.47:0/1560511597 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f408c0038e0 0x7f408c003ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:32.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.202+0000 7f40a93dd700 1 -- 172.21.15.47:0/1560511597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f408c004e70 msgr2=0x7f408c00b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:32.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.202+0000 7f40a93dd700 1 --2- 172.21.15.47:0/1560511597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f408c004e70 0x7f408c00b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:32.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.202+0000 7f40a93dd700 1 -- 172.21.15.47:0/1560511597 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f408c003260 con 0x7f408c004500 2022-06-29T15:51:32.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.206+0000 7f40aa3df700 1 --2- 172.21.15.47:0/1560511597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f408c004e70 0x7f408c00b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:51:32.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.206+0000 7f40aa3df700 1 -- 172.21.15.47:0/1560511597 reap_dead start 2022-06-29T15:51:32.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.206+0000 7f40a93dd700 1 --2- 172.21.15.47:0/1560511597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f408c004500 0x7f408c004980 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 rx=0x7f40a000c000 tx=0x7f40a0006df0).ready entity=mon.0 client_cookie=4374ffa85ea62a5a server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:32.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.206+0000 7f40a8bdc700 1 -- 172.21.15.47:0/1560511597 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40a000e800 con 0x7f408c004500 2022-06-29T15:51:32.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.206+0000 7f40a8bdc700 1 -- 172.21.15.47:0/1560511597 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40a000e960 con 0x7f408c004500 2022-06-29T15:51:32.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.206+0000 7f40a8bdc700 1 -- 172.21.15.47:0/1560511597 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40a000ecc0 con 0x7f408c004500 2022-06-29T15:51:32.225 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.206+0000 7f40abe42700 1 -- 172.21.15.47:0/1560511597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f408c004500 msgr2=0x7f408c004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:32.225 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.206+0000 7f40abe42700 1 --2- 172.21.15.47:0/1560511597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f408c004500 0x7f408c004980 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 rx=0x7f40a000c000 tx=0x7f40a0006df0).stop 2022-06-29T15:51:32.225 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.206+0000 7f40abe42700 1 -- 172.21.15.47:0/1560511597 shutdown_connections 2022-06-29T15:51:32.226 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.206+0000 7f40abe42700 1 --2- 172.21.15.47:0/1560511597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f408c004500 0x7f408c004980 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:32.226 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.206+0000 7f40abe42700 1 -- 172.21.15.47:0/1560511597 >> 172.21.15.47:0/1560511597 conn(0x7f40a4063800 msgr2=0x7f40a4063be0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:32.226 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.206+0000 7f40abe42700 1 -- 172.21.15.47:0/1560511597 shutdown_connections 2022-06-29T15:51:32.226 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.206+0000 7f40abe42700 1 -- 172.21.15.47:0/1560511597 wait complete. 2022-06-29T15:51:32.227 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.210+0000 7f40abe42700 1 Processor -- start 2022-06-29T15:51:32.227 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.210+0000 7f40abe42700 1 -- start start 2022-06-29T15:51:32.227 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.210+0000 7f40abe42700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f408c004500 0x7f40a4067370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:32.227 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.210+0000 7f40abe42700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40a405f610 0x7f40a413a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:32.228 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.210+0000 7f40abe42700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40a413aca0 0x7f40a4151a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:32.228 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.210+0000 7f40abe42700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a40aba80 con 0x7f40a413aca0 2022-06-29T15:51:32.228 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.210+0000 7f40abe42700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a40abcb0 con 0x7f40a405f610 2022-06-29T15:51:32.228 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.210+0000 7f40abe42700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40a40abf50 con 0x7f408c004500 2022-06-29T15:51:32.229 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.210+0000 7f40aa3df700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40a413aca0 0x7f40a4151a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:32.229 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.210+0000 7f40aa3df700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40a413aca0 0x7f40a4151a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53882/0 (socket says 172.21.15.47:53882) 2022-06-29T15:51:32.229 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.210+0000 7f40aa3df700 1 -- 172.21.15.47:0/1136148148 learned_addr learned my addr 172.21.15.47:0/1136148148 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:32.229 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.210+0000 7f40a93dd700 1 --2- 172.21.15.47:0/1136148148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40a405f610 0x7f40a413a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:32.230 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.210+0000 7f40a9bde700 1 --2- 172.21.15.47:0/1136148148 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f408c004500 0x7f40a4067370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:32.230 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.210+0000 7f40aa3df700 1 -- 172.21.15.47:0/1136148148 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f408c004500 msgr2=0x7f40a4067370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:32.230 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.210+0000 7f40aa3df700 1 --2- 172.21.15.47:0/1136148148 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f408c004500 0x7f40a4067370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:32.230 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.210+0000 7f40aa3df700 1 -- 172.21.15.47:0/1136148148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40a405f610 msgr2=0x7f40a413a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:32.231 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.210+0000 7f40aa3df700 1 --2- 172.21.15.47:0/1136148148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40a405f610 0x7f40a413a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:32.231 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.210+0000 7f40aa3df700 1 -- 172.21.15.47:0/1136148148 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a40ac1f0 con 0x7f40a413aca0 2022-06-29T15:51:32.231 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.210+0000 7f40aa3df700 1 --2- 172.21.15.47:0/1136148148 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40a413aca0 0x7f40a4151a30 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 rx=0x7f409c001550 tx=0x7f409c009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:32.231 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.210+0000 7f4092ffd700 1 -- 172.21.15.47:0/1136148148 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f409c0032c0 con 0x7f40a413aca0 2022-06-29T15:51:32.232 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.210+0000 7f4092ffd700 1 -- 172.21.15.47:0/1136148148 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f409c003420 con 0x7f40a413aca0 2022-06-29T15:51:32.232 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.210+0000 7f4092ffd700 1 -- 172.21.15.47:0/1136148148 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f409c006a80 con 0x7f40a413aca0 2022-06-29T15:51:32.232 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.210+0000 7f40abe42700 1 -- 172.21.15.47:0/1136148148 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40a40ac490 con 0x7f40a413aca0 2022-06-29T15:51:32.232 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.210+0000 7f40abe42700 1 -- 172.21.15.47:0/1136148148 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40a413b150 con 0x7f40a413aca0 2022-06-29T15:51:32.233 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.214+0000 7f40abe42700 1 -- 172.21.15.47:0/1136148148 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40a405f9f0 con 0x7f40a413aca0 2022-06-29T15:51:32.233 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.214+0000 7f4092ffd700 1 -- 172.21.15.47:0/1136148148 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f409c01e070 con 0x7f40a413aca0 2022-06-29T15:51:32.233 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.214+0000 7f4092ffd700 1 --2- 172.21.15.47:0/1136148148 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4084053aa0 0x7f4084055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:32.234 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.214+0000 7f4092ffd700 1 -- 172.21.15.47:0/1136148148 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f409c06b550 con 0x7f40a413aca0 2022-06-29T15:51:32.234 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.214+0000 7f40a9bde700 1 --2- 172.21.15.47:0/1136148148 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4084053aa0 0x7f4084055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:32.234 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.214+0000 7f40a9bde700 1 --2- 172.21.15.47:0/1136148148 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4084053aa0 0x7f4084055f60 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7f409400d000 tx=0x7f4094009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:32.235 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.222+0000 7f4092ffd700 1 -- 172.21.15.47:0/1136148148 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f409c0422b0 con 0x7f40a413aca0 2022-06-29T15:51:32.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.350+0000 7f40abe42700 1 -- 172.21.15.47:0/1136148148 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f40a4151830 con 0x7f40a413aca0 2022-06-29T15:51:32.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.350+0000 7f4092ffd700 1 -- 172.21.15.47:0/1136148148 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v140) v1 ==== 100+0+10 (secure 0 0 0) 0x7f409c0458d0 con 0x7f40a413aca0 2022-06-29T15:51:32.359 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:51:32.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.354+0000 7f40abe42700 1 -- 172.21.15.47:0/1136148148 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4084053aa0 msgr2=0x7f4084055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:32.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.354+0000 7f40abe42700 1 --2- 172.21.15.47:0/1136148148 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4084053aa0 0x7f4084055f60 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7f409400d000 tx=0x7f4094009040).stop 2022-06-29T15:51:32.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.354+0000 7f40abe42700 1 -- 172.21.15.47:0/1136148148 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40a413aca0 msgr2=0x7f40a4151a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:32.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.354+0000 7f40abe42700 1 --2- 172.21.15.47:0/1136148148 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40a413aca0 0x7f40a4151a30 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 rx=0x7f409c001550 tx=0x7f409c009040).stop 2022-06-29T15:51:32.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.354+0000 7f40abe42700 1 -- 172.21.15.47:0/1136148148 shutdown_connections 2022-06-29T15:51:32.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.354+0000 7f40abe42700 1 --2- 172.21.15.47:0/1136148148 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40a413aca0 0x7f40a4151a30 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:32.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.354+0000 7f40abe42700 1 --2- 172.21.15.47:0/1136148148 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4084053aa0 0x7f4084055f60 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:32.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.354+0000 7f40abe42700 1 --2- 172.21.15.47:0/1136148148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40a405f610 0x7f40a413a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:32.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.354+0000 7f40abe42700 1 --2- 172.21.15.47:0/1136148148 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f408c004500 0x7f40a4067370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:32.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.354+0000 7f40abe42700 1 -- 172.21.15.47:0/1136148148 >> 172.21.15.47:0/1136148148 conn(0x7f40a4063800 msgr2=0x7f40a40b4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:32.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.354+0000 7f40abe42700 1 -- 172.21.15.47:0/1136148148 shutdown_connections 2022-06-29T15:51:32.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.354+0000 7f40abe42700 1 -- 172.21.15.47:0/1136148148 wait complete. 2022-06-29T15:51:32.372 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-29T15:51:32.373 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:51:32.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.470+0000 7ff31ffff700 1 Processor -- start 2022-06-29T15:51:32.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.470+0000 7ff31ffff700 1 -- start start 2022-06-29T15:51:32.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.470+0000 7ff31ffff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff3200b5c60 0x7ff320144980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:32.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.470+0000 7ff31ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff3200a72f0 0x7ff3200b2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:32.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.470+0000 7ff31ffff700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3200a9120 con 0x7ff320060eb0 2022-06-29T15:51:32.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.470+0000 7ff31ffff700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3200b0b80 con 0x7ff3200b5c60 2022-06-29T15:51:32.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.470+0000 7ff31ffff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3200a89e0 con 0x7ff3200a72f0 2022-06-29T15:51:32.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.470+0000 7ff31e7fc700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff320060eb0 0x7ff3200b1e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39440/0 (socket says 172.21.15.47:39440) 2022-06-29T15:51:32.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.470+0000 7ff31e7fc700 1 -- 172.21.15.47:0/3160029937 learned_addr learned my addr 172.21.15.47:0/3160029937 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:32.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.470+0000 7ff31effd700 1 --2- 172.21.15.47:0/3160029937 >> v2:172.21.15.47:3300/0 conn(0x7ff3200b5c60 0x7ff320144980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:32.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.470+0000 7ff31f7fe700 1 --2- 172.21.15.47:0/3160029937 >> v2:172.21.15.64:3300/0 conn(0x7ff3200a72f0 0x7ff3200b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:32.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.470+0000 7ff31dffb700 1 -- 172.21.15.47:0/3160029937 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 701555561 0 0) 0x7ff3200a9120 con 0x7ff320060eb0 2022-06-29T15:51:32.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.470+0000 7ff31dffb700 1 -- 172.21.15.47:0/3160029937 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff304003260 con 0x7ff320060eb0 2022-06-29T15:51:32.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.470+0000 7ff31effd700 1 -- 172.21.15.47:0/3160029937 >> v2:172.21.15.64:3300/0 conn(0x7ff3200a72f0 msgr2=0x7ff3200b2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:32.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.470+0000 7ff31effd700 1 --2- 172.21.15.47:0/3160029937 >> v2:172.21.15.64:3300/0 conn(0x7ff3200a72f0 0x7ff3200b2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:32.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.470+0000 7ff31dffb700 1 -- 172.21.15.47:0/3160029937 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 937325416 0 0) 0x7ff304003260 con 0x7ff320060eb0 2022-06-29T15:51:32.484 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.470+0000 7ff31effd700 1 -- 172.21.15.47:0/3160029937 >> v1:172.21.15.47:6789/0 conn(0x7ff320060eb0 legacy=0x7ff3200b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:32.484 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.470+0000 7ff31effd700 1 -- 172.21.15.47:0/3160029937 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3200a8c80 con 0x7ff3200b5c60 2022-06-29T15:51:32.484 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.470+0000 7ff31effd700 1 --2- 172.21.15.47:0/3160029937 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3200b5c60 0x7ff320144980 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 rx=0x7ff310001b80 tx=0x7ff314006df0).ready entity=mon.0 client_cookie=ccb564e9ff164c1c server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:32.484 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.470+0000 7ff31dffb700 1 -- 172.21.15.47:0/3160029937 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3100019e0 con 0x7ff3200b5c60 2022-06-29T15:51:32.485 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.470+0000 7ff31dffb700 1 -- 172.21.15.47:0/3160029937 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff314000e90 con 0x7ff3200b5c60 2022-06-29T15:51:32.485 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.470+0000 7ff31dffb700 1 -- 172.21.15.47:0/3160029937 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff31400be80 con 0x7ff3200b5c60 2022-06-29T15:51:32.485 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.474+0000 7ff31ffff700 1 -- 172.21.15.47:0/3160029937 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3200b5c60 msgr2=0x7ff320144980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:32.485 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.474+0000 7ff31ffff700 1 --2- 172.21.15.47:0/3160029937 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3200b5c60 0x7ff320144980 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 rx=0x7ff310001b80 tx=0x7ff314006df0).stop 2022-06-29T15:51:32.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.474+0000 7ff31ffff700 1 -- 172.21.15.47:0/3160029937 shutdown_connections 2022-06-29T15:51:32.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.474+0000 7ff31ffff700 1 --2- 172.21.15.47:0/3160029937 >> v2:172.21.15.64:3300/0 conn(0x7ff3200a72f0 0x7ff3200b2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:32.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.474+0000 7ff31ffff700 1 --2- 172.21.15.47:0/3160029937 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3200b5c60 0x7ff320144980 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:32.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.474+0000 7ff31ffff700 1 -- 172.21.15.47:0/3160029937 >> 172.21.15.47:0/3160029937 conn(0x7ff320062560 msgr2=0x7ff320062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:32.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.474+0000 7ff31ffff700 1 -- 172.21.15.47:0/3160029937 shutdown_connections 2022-06-29T15:51:32.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.474+0000 7ff31ffff700 1 -- 172.21.15.47:0/3160029937 wait complete. 2022-06-29T15:51:32.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.474+0000 7ff31ffff700 1 Processor -- start 2022-06-29T15:51:32.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.474+0000 7ff31ffff700 1 -- start start 2022-06-29T15:51:32.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.474+0000 7ff31ffff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff320060eb0 0x7ff320156480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:32.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.474+0000 7ff31ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3200a72f0 0x7ff3200af1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:32.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.474+0000 7ff31ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3200b5c60 0x7ff320149dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:32.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.474+0000 7ff31ffff700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff320149140 con 0x7ff3200a72f0 2022-06-29T15:51:32.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.474+0000 7ff31ffff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3201493e0 con 0x7ff3200b5c60 2022-06-29T15:51:32.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.474+0000 7ff31ffff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff320149680 con 0x7ff320060eb0 2022-06-29T15:51:32.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.474+0000 7ff31e7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3200a72f0 0x7ff3200af1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:32.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.474+0000 7ff31e7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3200a72f0 0x7ff3200af1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53894/0 (socket says 172.21.15.47:53894) 2022-06-29T15:51:32.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.474+0000 7ff31e7fc700 1 -- 172.21.15.47:0/293737820 learned_addr learned my addr 172.21.15.47:0/293737820 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:32.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.474+0000 7ff31effd700 1 --2- 172.21.15.47:0/293737820 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff320060eb0 0x7ff320156480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:32.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.474+0000 7ff31f7fe700 1 --2- 172.21.15.47:0/293737820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3200b5c60 0x7ff320149dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:32.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.478+0000 7ff31e7fc700 1 -- 172.21.15.47:0/293737820 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff320060eb0 msgr2=0x7ff320156480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:32.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.478+0000 7ff31e7fc700 1 --2- 172.21.15.47:0/293737820 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff320060eb0 0x7ff320156480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:32.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.478+0000 7ff31e7fc700 1 -- 172.21.15.47:0/293737820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3200b5c60 msgr2=0x7ff320149dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:32.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.478+0000 7ff31e7fc700 1 --2- 172.21.15.47:0/293737820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3200b5c60 0x7ff320149dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:32.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.478+0000 7ff31e7fc700 1 -- 172.21.15.47:0/293737820 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff31400b120 con 0x7ff3200a72f0 2022-06-29T15:51:32.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.478+0000 7ff31e7fc700 1 --2- 172.21.15.47:0/293737820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3200a72f0 0x7ff3200af1f0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 rx=0x7ff310001c40 tx=0x7ff3100015b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:32.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.478+0000 7ff30ffff700 1 -- 172.21.15.47:0/293737820 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff310004ab0 con 0x7ff3200a72f0 2022-06-29T15:51:32.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.478+0000 7ff30ffff700 1 -- 172.21.15.47:0/293737820 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff310004c10 con 0x7ff3200a72f0 2022-06-29T15:51:32.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.478+0000 7ff31ffff700 1 -- 172.21.15.47:0/293737820 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff320149920 con 0x7ff3200a72f0 2022-06-29T15:51:32.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.478+0000 7ff30ffff700 1 -- 172.21.15.47:0/293737820 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff310007170 con 0x7ff3200a72f0 2022-06-29T15:51:32.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.478+0000 7ff31ffff700 1 -- 172.21.15.47:0/293737820 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff32014a5c0 con 0x7ff3200a72f0 2022-06-29T15:51:32.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.478+0000 7ff30ffff700 1 -- 172.21.15.47:0/293737820 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff31001b060 con 0x7ff3200a72f0 2022-06-29T15:51:32.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.478+0000 7ff31ffff700 1 -- 172.21.15.47:0/293737820 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff320149ab0 con 0x7ff3200a72f0 2022-06-29T15:51:32.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.478+0000 7ff30ffff700 1 --2- 172.21.15.47:0/293737820 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff2fc053950 0x7ff2fc055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:32.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.478+0000 7ff30ffff700 1 -- 172.21.15.47:0/293737820 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff31006be60 con 0x7ff3200a72f0 2022-06-29T15:51:32.497 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.478+0000 7ff31effd700 1 --2- 172.21.15.47:0/293737820 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff2fc053950 0x7ff2fc055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:32.497 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.482+0000 7ff31effd700 1 --2- 172.21.15.47:0/293737820 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff2fc053950 0x7ff2fc055e10 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7ff31401a000 tx=0x7ff314014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:32.497 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.486+0000 7ff30ffff700 1 -- 172.21.15.47:0/293737820 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff310042b10 con 0x7ff3200a72f0 2022-06-29T15:51:32.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.650+0000 7ff31ffff700 1 -- 172.21.15.47:0/293737820 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff3200573d0 con 0x7ff3200a72f0 2022-06-29T15:51:32.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.650+0000 7ff30ffff700 1 -- 172.21.15.47:0/293737820 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+2492 (secure 0 0 0) 0x7ff310019030 con 0x7ff3200a72f0 2022-06-29T15:51:32.656 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:32.657 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":69,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7957,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.64:6833/298405638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":298405638},{"type":"v1","addr":"172.21.15.64:6833","nonce":298405638}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":66},{"gid":8037,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/191070564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":191070564},{"type":"v1","addr":"172.21.15.47:6835","nonce":191070564}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:51:23.543892+0000","modified":"2022-06-29T15:51:25.309120+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":56894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_56894":{"gid":56894,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6837/1370488030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1370488030},{"type":"v1","addr":"172.21.15.47:6837","nonce":1370488030}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2022-06-29T15:51:32.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.650+0000 7ff31ffff700 1 -- 172.21.15.47:0/293737820 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff2fc053950 msgr2=0x7ff2fc055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:32.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.650+0000 7ff31ffff700 1 --2- 172.21.15.47:0/293737820 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff2fc053950 0x7ff2fc055e10 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7ff31401a000 tx=0x7ff314014000).stop 2022-06-29T15:51:32.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.650+0000 7ff31ffff700 1 -- 172.21.15.47:0/293737820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3200a72f0 msgr2=0x7ff3200af1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:32.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.650+0000 7ff31ffff700 1 --2- 172.21.15.47:0/293737820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3200a72f0 0x7ff3200af1f0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 rx=0x7ff310001c40 tx=0x7ff3100015b0).stop 2022-06-29T15:51:32.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.650+0000 7ff31ffff700 1 -- 172.21.15.47:0/293737820 shutdown_connections 2022-06-29T15:51:32.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.650+0000 7ff31ffff700 1 --2- 172.21.15.47:0/293737820 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff2fc053950 0x7ff2fc055e10 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:32.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.650+0000 7ff31ffff700 1 --2- 172.21.15.47:0/293737820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3200b5c60 0x7ff320149dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:32.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.650+0000 7ff31ffff700 1 --2- 172.21.15.47:0/293737820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3200a72f0 0x7ff3200af1f0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:32.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.650+0000 7ff31ffff700 1 --2- 172.21.15.47:0/293737820 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff320060eb0 0x7ff320156480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:32.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.650+0000 7ff31ffff700 1 -- 172.21.15.47:0/293737820 >> 172.21.15.47:0/293737820 conn(0x7ff320062560 msgr2=0x7ff3201406d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:32.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.650+0000 7ff31ffff700 1 -- 172.21.15.47:0/293737820 shutdown_connections 2022-06-29T15:51:32.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:32.650+0000 7ff31ffff700 1 -- 172.21.15.47:0/293737820 wait complete. 2022-06-29T15:51:32.662 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 69 2022-06-29T15:51:32.974 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 69, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-29T15:51:23.543892+0000', 'modified': '2022-06-29T15:51:25.309120+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 56894}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_56894': {'gid': 56894, 'name': 'c', 'rank': 0, 'incarnation': 67, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.47:6837/1370488030', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6836', 'nonce': 1370488030}, {'type': 'v1', 'addr': '172.21.15.47:6837', 'nonce': 1370488030}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-29T15:51:32.974 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-29T15:51:32.975 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2022-06-29T15:51:33.092 INFO:teuthology.orchestra.run.smithi047.stdout:{"cluster_fsid":"9a68894a-53ce-400e-838b-14eab91b834c","whoami":0,"id":56894,"want_state":"up:active","state":"up:active","rank_uptime":9.5311075580000004,"mdsmap_epoch":69,"osdmap_epoch":140,"osdmap_epoch_barrier":140,"uptime":14.33579522} 2022-06-29T15:51:33.092 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9a68894a-53ce-400e-838b-14eab91b834c", "id": 56894, "mdsmap_epoch": 69, "osdmap_epoch": 140, "osdmap_epoch_barrier": 140, "rank_uptime": 9.531107558, "state": "up:active", "uptime": 14.33579522, "want_state": "up:active", "whoami": 0 } 2022-06-29T15:51:33.093 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-29T15:51:33.093 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2022-06-29T15:51:33.094 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi164.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2022-06-29T15:51:33.094 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:51:33.095 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:51:33.103 DEBUG:teuthology.orchestra.run.smithi164:> sudo modprobe fuse 2022-06-29T15:51:33.164 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:51:33.164 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:51:33.179 INFO:teuthology.orchestra.run.smithi164.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:51:33.180 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:51:33.182 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:51:33.182 DEBUG:teuthology.orchestra.run.smithi164:> ls /sys/fs/fuse/connections 2022-06-29T15:51:33.188 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-29T15:51:33.189 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:51:33.191 DEBUG:teuthology.orchestra.run.smithi164:> sudo modprobe fuse 2022-06-29T15:51:33.211 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:51:33.211 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:51:33.265 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi164.stderr:2022-06-29T15:51:33.256+0000 7fb149afe080 -1 init, newargv = 0x556583be03c0 newargc=9 2022-06-29T15:51:33.266 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi164.stderr:ceph-fuse[12914]: starting ceph client 2022-06-29T15:51:33.269 INFO:teuthology.orchestra.run.smithi164.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:51:33.270 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:51:33.271 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:51:33.272 DEBUG:teuthology.orchestra.run.smithi164:> ls /sys/fs/fuse/connections 2022-06-29T15:51:33.275 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi164.stderr:ceph-fuse[12914]: starting fuse 2022-06-29T15:51:33.277 INFO:teuthology.orchestra.run.smithi164.stdout:52 2022-06-29T15:51:33.278 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-29T15:51:33.278 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:51:33.279 DEBUG:teuthology.orchestra.run.smithi164:> sudo python3 -c ' 2022-06-29T15:51:33.279 DEBUG:teuthology.orchestra.run.smithi164:> import glob 2022-06-29T15:51:33.280 DEBUG:teuthology.orchestra.run.smithi164:> import re 2022-06-29T15:51:33.280 DEBUG:teuthology.orchestra.run.smithi164:> import os 2022-06-29T15:51:33.281 DEBUG:teuthology.orchestra.run.smithi164:> import subprocess 2022-06-29T15:51:33.281 DEBUG:teuthology.orchestra.run.smithi164:> 2022-06-29T15:51:33.282 DEBUG:teuthology.orchestra.run.smithi164:> def find_socket(client_name): 2022-06-29T15:51:33.282 DEBUG:teuthology.orchestra.run.smithi164:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2022-06-29T15:51:33.283 DEBUG:teuthology.orchestra.run.smithi164:> files = glob.glob(asok_path) 2022-06-29T15:51:33.283 DEBUG:teuthology.orchestra.run.smithi164:> 2022-06-29T15:51:33.283 DEBUG:teuthology.orchestra.run.smithi164:> # Given a non-glob path, it better be there 2022-06-29T15:51:33.284 DEBUG:teuthology.orchestra.run.smithi164:> if "*" not in asok_path: 2022-06-29T15:51:33.284 DEBUG:teuthology.orchestra.run.smithi164:> assert(len(files) == 1) 2022-06-29T15:51:33.285 DEBUG:teuthology.orchestra.run.smithi164:> return files[0] 2022-06-29T15:51:33.285 DEBUG:teuthology.orchestra.run.smithi164:> 2022-06-29T15:51:33.286 DEBUG:teuthology.orchestra.run.smithi164:> for f in files: 2022-06-29T15:51:33.286 DEBUG:teuthology.orchestra.run.smithi164:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-29T15:51:33.287 DEBUG:teuthology.orchestra.run.smithi164:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-29T15:51:33.287 DEBUG:teuthology.orchestra.run.smithi164:> return f 2022-06-29T15:51:33.288 DEBUG:teuthology.orchestra.run.smithi164:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-29T15:51:33.288 DEBUG:teuthology.orchestra.run.smithi164:> 2022-06-29T15:51:33.289 DEBUG:teuthology.orchestra.run.smithi164:> print(find_socket("client.1")) 2022-06-29T15:51:33.289 DEBUG:teuthology.orchestra.run.smithi164:> ' 2022-06-29T15:51:33.363 INFO:teuthology.orchestra.run.smithi164.stdout:/var/run/ceph/ceph-client.1.12914.asok 2022-06-29T15:51:33.368 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.12914.asok 2022-06-29T15:51:33.368 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:51:33.369 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.12914.asok status 2022-06-29T15:51:33.455 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2022-06-29T15:51:33.455 INFO:teuthology.orchestra.run.smithi164.stdout: "metadata": { 2022-06-29T15:51:33.455 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_sha1": "f25ffac72e5a9e63860ff7deb6767a11f0559a16", 2022-06-29T15:51:33.456 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_version": "ceph version 15.2.16-304-gf25ffac7 (f25ffac72e5a9e63860ff7deb6767a11f0559a16) octopus (stable)", 2022-06-29T15:51:33.456 INFO:teuthology.orchestra.run.smithi164.stdout: "entity_id": "1", 2022-06-29T15:51:33.456 INFO:teuthology.orchestra.run.smithi164.stdout: "hostname": "smithi164", 2022-06-29T15:51:33.456 INFO:teuthology.orchestra.run.smithi164.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2022-06-29T15:51:33.457 INFO:teuthology.orchestra.run.smithi164.stdout: "pid": "12914", 2022-06-29T15:51:33.457 INFO:teuthology.orchestra.run.smithi164.stdout: "root": "/" 2022-06-29T15:51:33.457 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2022-06-29T15:51:33.457 INFO:teuthology.orchestra.run.smithi164.stdout: "dentry_count": 0, 2022-06-29T15:51:33.458 INFO:teuthology.orchestra.run.smithi164.stdout: "dentry_pinned_count": 0, 2022-06-29T15:51:33.458 INFO:teuthology.orchestra.run.smithi164.stdout: "id": 8250, 2022-06-29T15:51:33.458 INFO:teuthology.orchestra.run.smithi164.stdout: "inst": { 2022-06-29T15:51:33.458 INFO:teuthology.orchestra.run.smithi164.stdout: "name": { 2022-06-29T15:51:33.459 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "client", 2022-06-29T15:51:33.459 INFO:teuthology.orchestra.run.smithi164.stdout: "num": 8250 2022-06-29T15:51:33.459 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2022-06-29T15:51:33.459 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": { 2022-06-29T15:51:33.460 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "v1", 2022-06-29T15:51:33.460 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": "172.21.15.164:0", 2022-06-29T15:51:33.460 INFO:teuthology.orchestra.run.smithi164.stdout: "nonce": 328319020 2022-06-29T15:51:33.460 INFO:teuthology.orchestra.run.smithi164.stdout: } 2022-06-29T15:51:33.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2022-06-29T15:51:33.461 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": { 2022-06-29T15:51:33.461 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "v1", 2022-06-29T15:51:33.461 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": "172.21.15.164:0", 2022-06-29T15:51:33.462 INFO:teuthology.orchestra.run.smithi164.stdout: "nonce": 328319020 2022-06-29T15:51:33.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2022-06-29T15:51:33.462 INFO:teuthology.orchestra.run.smithi164.stdout: "inst_str": "client.8250 172.21.15.164:0/328319020", 2022-06-29T15:51:33.462 INFO:teuthology.orchestra.run.smithi164.stdout: "addr_str": "172.21.15.164:0/328319020", 2022-06-29T15:51:33.463 INFO:teuthology.orchestra.run.smithi164.stdout: "inode_count": 1, 2022-06-29T15:51:33.463 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_epoch": 69, 2022-06-29T15:51:33.463 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_epoch": 140, 2022-06-29T15:51:33.463 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_epoch_barrier": 0, 2022-06-29T15:51:33.464 INFO:teuthology.orchestra.run.smithi164.stdout: "blacklisted": false, 2022-06-29T15:51:33.464 INFO:teuthology.orchestra.run.smithi164.stdout: "fs_name": "cephfs" 2022-06-29T15:51:33.464 INFO:teuthology.orchestra.run.smithi164.stdout:} 2022-06-29T15:51:33.466 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:51:33.466 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:51:33.467 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:51:33.473 INFO:teuthology.orchestra.run.smithi164.stdout:fuseblk 2022-06-29T15:51:33.475 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-06-29T15:51:33.475 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:51:33.476 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:51:33.537 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:51:33.538 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:51:33.538 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:51:33.546 INFO:teuthology.orchestra.run.smithi164.stdout:fuseblk 2022-06-29T15:51:33.547 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-06-29T15:51:33.547 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:51:33.548 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:51:33.548 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:51:33.556 INFO:teuthology.orchestra.run.smithi164.stdout:fuseblk 2022-06-29T15:51:33.557 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-06-29T15:51:33.557 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi164.front.sepia.ceph.com... 2022-06-29T15:51:33.558 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-06-29T15:51:33.558 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:51:33.617 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi164.stderr:ceph-fuse[12914]: fuse finished with error 0 and tester_r 0 2022-06-29T15:51:33.619 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:51:33.620 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:51:33.620 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:51:33.626 INFO:teuthology.orchestra.run.smithi164.stdout:ext2/ext3 2022-06-29T15:51:33.628 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-06-29T15:51:33.628 INFO:teuthology.orchestra.run:waiting for 900 2022-06-29T15:51:33.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-29T15:51:33.657 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:51:33.666 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-06-29T15:51:33.669 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-06-29T15:51:33.672 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-06-29T15:51:33.675 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-06-29T15:51:33.702 INFO:teuthology.orchestra.run.smithi047.stdout:[global] 2022-06-29T15:51:33.702 INFO:teuthology.orchestra.run.smithi047.stdout: chdir = "" 2022-06-29T15:51:33.702 INFO:teuthology.orchestra.run.smithi047.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-06-29T15:51:33.703 INFO:teuthology.orchestra.run.smithi047.stdout: auth supported = cephx 2022-06-29T15:51:33.703 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:33.703 INFO:teuthology.orchestra.run.smithi047.stdout: filestore xattr use omap = true 2022-06-29T15:51:33.703 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:33.704 INFO:teuthology.orchestra.run.smithi047.stdout: mon clock drift allowed = 1.000 2022-06-29T15:51:33.704 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:33.704 INFO:teuthology.orchestra.run.smithi047.stdout: osd crush chooseleaf type = 0 2022-06-29T15:51:33.704 INFO:teuthology.orchestra.run.smithi047.stdout: auth debug = true 2022-06-29T15:51:33.705 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:33.705 INFO:teuthology.orchestra.run.smithi047.stdout: ms die on old message = true 2022-06-29T15:51:33.705 INFO:teuthology.orchestra.run.smithi047.stdout: ms die on bug = true 2022-06-29T15:51:33.705 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:33.706 INFO:teuthology.orchestra.run.smithi047.stdout: mon max pg per osd = 10000 # >= luminous 2022-06-29T15:51:33.706 INFO:teuthology.orchestra.run.smithi047.stdout: mon pg warn max object skew = 0 2022-06-29T15:51:33.706 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:33.706 INFO:teuthology.orchestra.run.smithi047.stdout: # disable pg_autoscaler by default for new pools 2022-06-29T15:51:33.707 INFO:teuthology.orchestra.run.smithi047.stdout: osd_pool_default_pg_autoscale_mode = off 2022-06-29T15:51:33.707 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:33.707 INFO:teuthology.orchestra.run.smithi047.stdout: osd pool default size = 2 2022-06-29T15:51:33.707 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:33.708 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd allow primary affinity = true 2022-06-29T15:51:33.708 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd allow pg remap = true 2022-06-29T15:51:33.708 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on legacy crush tunables = false 2022-06-29T15:51:33.708 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on crush straw calc version zero = false 2022-06-29T15:51:33.708 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on no sortbitwise = false 2022-06-29T15:51:33.709 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on osd down out interval zero = false 2022-06-29T15:51:33.709 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on too few osds = false 2022-06-29T15:51:33.709 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-06-29T15:51:33.709 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_pool_no_redundancy = false 2022-06-29T15:51:33.710 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:33.710 INFO:teuthology.orchestra.run.smithi047.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-06-29T15:51:33.710 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:33.710 INFO:teuthology.orchestra.run.smithi047.stdout: osd default data pool replay window = 5 2022-06-29T15:51:33.710 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:33.711 INFO:teuthology.orchestra.run.smithi047.stdout: mon allow pool delete = true 2022-06-29T15:51:33.711 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:33.711 INFO:teuthology.orchestra.run.smithi047.stdout: mon cluster log file level = debug 2022-06-29T15:51:33.711 INFO:teuthology.orchestra.run.smithi047.stdout: debug asserts on shutdown = true 2022-06-29T15:51:33.712 INFO:teuthology.orchestra.run.smithi047.stdout: mon health detail to clog = false 2022-06-29T15:51:33.712 INFO:teuthology.orchestra.run.smithi047.stdout: mon host = "172.21.15.47,172.21.15.64,[v2:172.21.15.64:3301,v1:172.21.15.64:6790]" 2022-06-29T15:51:33.712 INFO:teuthology.orchestra.run.smithi047.stdout: fsid = 9a68894a-53ce-400e-838b-14eab91b834c 2022-06-29T15:51:33.712 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:33.712 INFO:teuthology.orchestra.run.smithi047.stdout:[osd] 2022-06-29T15:51:33.713 INFO:teuthology.orchestra.run.smithi047.stdout: osd journal size = 100 2022-06-29T15:51:33.713 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:33.713 INFO:teuthology.orchestra.run.smithi047.stdout: osd scrub load threshold = 5.0 2022-06-29T15:51:33.713 INFO:teuthology.orchestra.run.smithi047.stdout: osd scrub max interval = 600 2022-06-29T15:51:33.714 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:33.714 INFO:teuthology.orchestra.run.smithi047.stdout: osd recover clone overlap = true 2022-06-29T15:51:33.714 INFO:teuthology.orchestra.run.smithi047.stdout: osd recovery max chunk = 1048576 2022-06-29T15:51:33.714 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:33.715 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug shutdown = true 2022-06-29T15:51:33.715 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug op order = true 2022-06-29T15:51:33.715 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug verify stray on activate = true 2022-06-29T15:51:33.715 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:33.715 INFO:teuthology.orchestra.run.smithi047.stdout: osd open classes on start = true 2022-06-29T15:51:33.716 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug pg log writeout = true 2022-06-29T15:51:33.716 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:33.716 INFO:teuthology.orchestra.run.smithi047.stdout: osd deep scrub update digest min age = 30 2022-06-29T15:51:33.716 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:33.717 INFO:teuthology.orchestra.run.smithi047.stdout: osd map max advance = 10 2022-06-29T15:51:33.717 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:33.717 INFO:teuthology.orchestra.run.smithi047.stdout: journal zero on create = true 2022-06-29T15:51:33.717 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:33.717 INFO:teuthology.orchestra.run.smithi047.stdout: filestore ondisk finisher threads = 3 2022-06-29T15:51:33.718 INFO:teuthology.orchestra.run.smithi047.stdout: filestore apply finisher threads = 3 2022-06-29T15:51:33.718 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:33.718 INFO:teuthology.orchestra.run.smithi047.stdout: bdev debug aio = true 2022-06-29T15:51:33.718 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug misdirected ops = true 2022-06-29T15:51:33.719 INFO:teuthology.orchestra.run.smithi047.stdout: bluestore block size = 96636764160 2022-06-29T15:51:33.719 INFO:teuthology.orchestra.run.smithi047.stdout: bluestore compression mode = aggressive 2022-06-29T15:51:33.719 INFO:teuthology.orchestra.run.smithi047.stdout: bluestore fsck on mount = True 2022-06-29T15:51:33.719 INFO:teuthology.orchestra.run.smithi047.stdout: debug bluefs = 20 2022-06-29T15:51:33.719 INFO:teuthology.orchestra.run.smithi047.stdout: debug bluestore = 20 2022-06-29T15:51:33.720 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2022-06-29T15:51:33.720 INFO:teuthology.orchestra.run.smithi047.stdout: debug osd = 20 2022-06-29T15:51:33.720 INFO:teuthology.orchestra.run.smithi047.stdout: debug rocksdb = 10 2022-06-29T15:51:33.720 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd backfillfull_ratio = 0.85 2022-06-29T15:51:33.720 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd full ratio = 0.9 2022-06-29T15:51:33.721 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd nearfull ratio = 0.8 2022-06-29T15:51:33.721 INFO:teuthology.orchestra.run.smithi047.stdout: osd failsafe full ratio = 0.95 2022-06-29T15:51:33.721 INFO:teuthology.orchestra.run.smithi047.stdout: osd objectstore = bluestore 2022-06-29T15:51:33.721 INFO:teuthology.orchestra.run.smithi047.stdout: osd op complaint time = 180 2022-06-29T15:51:33.722 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:33.722 INFO:teuthology.orchestra.run.smithi047.stdout:[mgr] 2022-06-29T15:51:33.722 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2022-06-29T15:51:33.722 INFO:teuthology.orchestra.run.smithi047.stdout: debug mgr = 20 2022-06-29T15:51:33.722 INFO:teuthology.orchestra.run.smithi047.stdout: debug mon = 20 2022-06-29T15:51:33.723 INFO:teuthology.orchestra.run.smithi047.stdout: debug auth = 20 2022-06-29T15:51:33.723 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min pgs per osd = 4 2022-06-29T15:51:33.723 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min bytes per osd = 10 2022-06-29T15:51:33.723 INFO:teuthology.orchestra.run.smithi047.stdout: mgr/telemetry/nag = false 2022-06-29T15:51:33.724 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:33.724 INFO:teuthology.orchestra.run.smithi047.stdout:[mon] 2022-06-29T15:51:33.724 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2022-06-29T15:51:33.724 INFO:teuthology.orchestra.run.smithi047.stdout: debug mon = 20 2022-06-29T15:51:33.724 INFO:teuthology.orchestra.run.smithi047.stdout: debug paxos = 20 2022-06-29T15:51:33.725 INFO:teuthology.orchestra.run.smithi047.stdout: debug auth = 20 2022-06-29T15:51:33.725 INFO:teuthology.orchestra.run.smithi047.stdout: mon data avail warn = 5 2022-06-29T15:51:33.725 INFO:teuthology.orchestra.run.smithi047.stdout: mon mgr mkfs grace = 240 2022-06-29T15:51:33.725 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min pgs per osd = 4 2022-06-29T15:51:33.726 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd reporter subtree level = osd 2022-06-29T15:51:33.726 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd prime pg temp = true 2022-06-29T15:51:33.726 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min bytes per osd = 10 2022-06-29T15:51:33.726 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:33.726 INFO:teuthology.orchestra.run.smithi047.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-06-29T15:51:33.727 INFO:teuthology.orchestra.run.smithi047.stdout: auth mon ticket ttl = 660 # 11m 2022-06-29T15:51:33.727 INFO:teuthology.orchestra.run.smithi047.stdout: auth service ticket ttl = 240 # 4m 2022-06-29T15:51:33.727 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:33.727 INFO:teuthology.orchestra.run.smithi047.stdout: # don't complain about insecure global_id in the test suite 2022-06-29T15:51:33.728 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-06-29T15:51:33.728 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-06-29T15:51:33.728 INFO:teuthology.orchestra.run.smithi047.stdout: mon op complaint time = 120 2022-06-29T15:51:33.728 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:33.728 INFO:teuthology.orchestra.run.smithi047.stdout:[client] 2022-06-29T15:51:33.729 INFO:teuthology.orchestra.run.smithi047.stdout: rgw cache enabled = true 2022-06-29T15:51:33.729 INFO:teuthology.orchestra.run.smithi047.stdout: rgw enable ops log = true 2022-06-29T15:51:33.729 INFO:teuthology.orchestra.run.smithi047.stdout: rgw enable usage log = true 2022-06-29T15:51:33.729 INFO:teuthology.orchestra.run.smithi047.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-06-29T15:51:33.730 INFO:teuthology.orchestra.run.smithi047.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-06-29T15:51:33.730 INFO:teuthology.orchestra.run.smithi047.stdout: client mount timeout = 600 2022-06-29T15:51:33.730 INFO:teuthology.orchestra.run.smithi047.stdout: debug client = 20 2022-06-29T15:51:33.730 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2022-06-29T15:51:33.730 INFO:teuthology.orchestra.run.smithi047.stdout: rados mon op timeout = 15m 2022-06-29T15:51:33.731 INFO:teuthology.orchestra.run.smithi047.stdout: rados osd op timeout = 15m 2022-06-29T15:51:33.731 INFO:teuthology.orchestra.run.smithi047.stdout: client debug getattr caps = true 2022-06-29T15:51:33.731 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.a] 2022-06-29T15:51:33.731 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.b] 2022-06-29T15:51:33.732 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.c] 2022-06-29T15:51:33.732 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.a] 2022-06-29T15:51:33.732 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.c] 2022-06-29T15:51:33.732 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.b] 2022-06-29T15:51:33.732 INFO:teuthology.orchestra.run.smithi047.stdout:[mds] 2022-06-29T15:51:33.733 INFO:teuthology.orchestra.run.smithi047.stdout: debug mds = 20 2022-06-29T15:51:33.733 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2022-06-29T15:51:33.733 INFO:teuthology.orchestra.run.smithi047.stdout: mds bal frag = True 2022-06-29T15:51:33.733 INFO:teuthology.orchestra.run.smithi047.stdout: mds bal fragment size max = 10000 2022-06-29T15:51:33.734 INFO:teuthology.orchestra.run.smithi047.stdout: mds bal merge size = 5 2022-06-29T15:51:33.734 INFO:teuthology.orchestra.run.smithi047.stdout: mds bal split bits = 3 2022-06-29T15:51:33.734 INFO:teuthology.orchestra.run.smithi047.stdout: mds bal split size = 100 2022-06-29T15:51:33.734 INFO:teuthology.orchestra.run.smithi047.stdout: mds debug frag = True 2022-06-29T15:51:33.734 INFO:teuthology.orchestra.run.smithi047.stdout: mds debug scatterstat = True 2022-06-29T15:51:33.735 INFO:teuthology.orchestra.run.smithi047.stdout: mds op complaint time = 180 2022-06-29T15:51:33.735 INFO:teuthology.orchestra.run.smithi047.stdout: mds verify scatter = True 2022-06-29T15:51:33.735 INFO:teuthology.orchestra.run.smithi047.stdout: osd op complaint time = 180 2022-06-29T15:51:33.735 INFO:teuthology.orchestra.run.smithi047.stdout: rados mon op timeout = 15m 2022-06-29T15:51:33.736 INFO:teuthology.orchestra.run.smithi047.stdout: rados osd op timeout = 15m 2022-06-29T15:51:33.739 INFO:teuthology.orchestra.run.smithi099.stdout:[global] 2022-06-29T15:51:33.739 INFO:teuthology.orchestra.run.smithi099.stdout: chdir = "" 2022-06-29T15:51:33.739 INFO:teuthology.orchestra.run.smithi099.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-06-29T15:51:33.739 INFO:teuthology.orchestra.run.smithi099.stdout: auth supported = cephx 2022-06-29T15:51:33.740 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:33.740 INFO:teuthology.orchestra.run.smithi099.stdout: filestore xattr use omap = true 2022-06-29T15:51:33.740 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:33.740 INFO:teuthology.orchestra.run.smithi099.stdout: mon clock drift allowed = 1.000 2022-06-29T15:51:33.741 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:33.741 INFO:teuthology.orchestra.run.smithi099.stdout: osd crush chooseleaf type = 0 2022-06-29T15:51:33.741 INFO:teuthology.orchestra.run.smithi099.stdout: auth debug = true 2022-06-29T15:51:33.741 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:33.741 INFO:teuthology.orchestra.run.smithi099.stdout: ms die on old message = true 2022-06-29T15:51:33.742 INFO:teuthology.orchestra.run.smithi099.stdout: ms die on bug = true 2022-06-29T15:51:33.742 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:33.742 INFO:teuthology.orchestra.run.smithi099.stdout: mon max pg per osd = 10000 # >= luminous 2022-06-29T15:51:33.742 INFO:teuthology.orchestra.run.smithi099.stdout: mon pg warn max object skew = 0 2022-06-29T15:51:33.743 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:33.743 INFO:teuthology.orchestra.run.smithi099.stdout: # disable pg_autoscaler by default for new pools 2022-06-29T15:51:33.743 INFO:teuthology.orchestra.run.smithi099.stdout: osd_pool_default_pg_autoscale_mode = off 2022-06-29T15:51:33.743 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:33.743 INFO:teuthology.orchestra.run.smithi099.stdout: osd pool default size = 2 2022-06-29T15:51:33.744 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:33.744 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd allow primary affinity = true 2022-06-29T15:51:33.744 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd allow pg remap = true 2022-06-29T15:51:33.744 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on legacy crush tunables = false 2022-06-29T15:51:33.745 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on crush straw calc version zero = false 2022-06-29T15:51:33.745 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on no sortbitwise = false 2022-06-29T15:51:33.745 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on osd down out interval zero = false 2022-06-29T15:51:33.745 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on too few osds = false 2022-06-29T15:51:33.746 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-06-29T15:51:33.746 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_pool_no_redundancy = false 2022-06-29T15:51:33.746 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:33.746 INFO:teuthology.orchestra.run.smithi099.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-06-29T15:51:33.747 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:33.747 INFO:teuthology.orchestra.run.smithi099.stdout: osd default data pool replay window = 5 2022-06-29T15:51:33.747 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:33.747 INFO:teuthology.orchestra.run.smithi099.stdout: mon allow pool delete = true 2022-06-29T15:51:33.747 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:33.748 INFO:teuthology.orchestra.run.smithi099.stdout: mon cluster log file level = debug 2022-06-29T15:51:33.748 INFO:teuthology.orchestra.run.smithi099.stdout: debug asserts on shutdown = true 2022-06-29T15:51:33.748 INFO:teuthology.orchestra.run.smithi099.stdout: mon health detail to clog = false 2022-06-29T15:51:33.748 INFO:teuthology.orchestra.run.smithi099.stdout: mon host = "172.21.15.47,172.21.15.64,[v2:172.21.15.64:3301,v1:172.21.15.64:6790]" 2022-06-29T15:51:33.749 INFO:teuthology.orchestra.run.smithi099.stdout: fsid = 9a68894a-53ce-400e-838b-14eab91b834c 2022-06-29T15:51:33.749 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:33.749 INFO:teuthology.orchestra.run.smithi099.stdout:[osd] 2022-06-29T15:51:33.749 INFO:teuthology.orchestra.run.smithi099.stdout: osd journal size = 100 2022-06-29T15:51:33.749 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:33.750 INFO:teuthology.orchestra.run.smithi099.stdout: osd scrub load threshold = 5.0 2022-06-29T15:51:33.750 INFO:teuthology.orchestra.run.smithi099.stdout: osd scrub max interval = 600 2022-06-29T15:51:33.750 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:33.750 INFO:teuthology.orchestra.run.smithi099.stdout: osd recover clone overlap = true 2022-06-29T15:51:33.751 INFO:teuthology.orchestra.run.smithi099.stdout: osd recovery max chunk = 1048576 2022-06-29T15:51:33.751 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:33.751 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug shutdown = true 2022-06-29T15:51:33.751 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug op order = true 2022-06-29T15:51:33.752 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug verify stray on activate = true 2022-06-29T15:51:33.752 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:33.752 INFO:teuthology.orchestra.run.smithi099.stdout: osd open classes on start = true 2022-06-29T15:51:33.752 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug pg log writeout = true 2022-06-29T15:51:33.752 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:33.753 INFO:teuthology.orchestra.run.smithi099.stdout: osd deep scrub update digest min age = 30 2022-06-29T15:51:33.753 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:33.753 INFO:teuthology.orchestra.run.smithi099.stdout: osd map max advance = 10 2022-06-29T15:51:33.753 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:33.754 INFO:teuthology.orchestra.run.smithi099.stdout: journal zero on create = true 2022-06-29T15:51:33.754 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:33.754 INFO:teuthology.orchestra.run.smithi099.stdout: filestore ondisk finisher threads = 3 2022-06-29T15:51:33.754 INFO:teuthology.orchestra.run.smithi099.stdout: filestore apply finisher threads = 3 2022-06-29T15:51:33.755 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:33.755 INFO:teuthology.orchestra.run.smithi099.stdout: bdev debug aio = true 2022-06-29T15:51:33.755 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug misdirected ops = true 2022-06-29T15:51:33.755 INFO:teuthology.orchestra.run.smithi099.stdout: bluestore block size = 96636764160 2022-06-29T15:51:33.755 INFO:teuthology.orchestra.run.smithi099.stdout: bluestore compression mode = aggressive 2022-06-29T15:51:33.756 INFO:teuthology.orchestra.run.smithi099.stdout: bluestore fsck on mount = True 2022-06-29T15:51:33.756 INFO:teuthology.orchestra.run.smithi099.stdout: debug bluefs = 20 2022-06-29T15:51:33.756 INFO:teuthology.orchestra.run.smithi099.stdout: debug bluestore = 20 2022-06-29T15:51:33.756 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2022-06-29T15:51:33.757 INFO:teuthology.orchestra.run.smithi099.stdout: debug osd = 20 2022-06-29T15:51:33.757 INFO:teuthology.orchestra.run.smithi099.stdout: debug rocksdb = 10 2022-06-29T15:51:33.757 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd backfillfull_ratio = 0.85 2022-06-29T15:51:33.757 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd full ratio = 0.9 2022-06-29T15:51:33.757 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd nearfull ratio = 0.8 2022-06-29T15:51:33.758 INFO:teuthology.orchestra.run.smithi099.stdout: osd failsafe full ratio = 0.95 2022-06-29T15:51:33.758 INFO:teuthology.orchestra.run.smithi099.stdout: osd objectstore = bluestore 2022-06-29T15:51:33.758 INFO:teuthology.orchestra.run.smithi099.stdout: osd op complaint time = 180 2022-06-29T15:51:33.758 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:33.759 INFO:teuthology.orchestra.run.smithi099.stdout:[mgr] 2022-06-29T15:51:33.759 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2022-06-29T15:51:33.759 INFO:teuthology.orchestra.run.smithi099.stdout: debug mgr = 20 2022-06-29T15:51:33.759 INFO:teuthology.orchestra.run.smithi099.stdout: debug mon = 20 2022-06-29T15:51:33.760 INFO:teuthology.orchestra.run.smithi099.stdout: debug auth = 20 2022-06-29T15:51:33.760 INFO:teuthology.orchestra.run.smithi099.stdout: mon reweight min pgs per osd = 4 2022-06-29T15:51:33.760 INFO:teuthology.orchestra.run.smithi099.stdout: mon reweight min bytes per osd = 10 2022-06-29T15:51:33.760 INFO:teuthology.orchestra.run.smithi099.stdout: mgr/telemetry/nag = false 2022-06-29T15:51:33.760 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:33.761 INFO:teuthology.orchestra.run.smithi099.stdout:[mon] 2022-06-29T15:51:33.761 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2022-06-29T15:51:33.761 INFO:teuthology.orchestra.run.smithi099.stdout: debug mon = 20 2022-06-29T15:51:33.761 INFO:teuthology.orchestra.run.smithi099.stdout: debug paxos = 20 2022-06-29T15:51:33.762 INFO:teuthology.orchestra.run.smithi099.stdout: debug auth = 20 2022-06-29T15:51:33.762 INFO:teuthology.orchestra.run.smithi099.stdout: mon data avail warn = 5 2022-06-29T15:51:33.762 INFO:teuthology.orchestra.run.smithi099.stdout: mon mgr mkfs grace = 240 2022-06-29T15:51:33.762 INFO:teuthology.orchestra.run.smithi099.stdout: mon reweight min pgs per osd = 4 2022-06-29T15:51:33.762 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd reporter subtree level = osd 2022-06-29T15:51:33.763 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd prime pg temp = true 2022-06-29T15:51:33.763 INFO:teuthology.orchestra.run.smithi099.stdout: mon reweight min bytes per osd = 10 2022-06-29T15:51:33.763 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:33.763 INFO:teuthology.orchestra.run.smithi099.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-06-29T15:51:33.764 INFO:teuthology.orchestra.run.smithi099.stdout: auth mon ticket ttl = 660 # 11m 2022-06-29T15:51:33.764 INFO:teuthology.orchestra.run.smithi099.stdout: auth service ticket ttl = 240 # 4m 2022-06-29T15:51:33.764 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:33.764 INFO:teuthology.orchestra.run.smithi099.stdout: # don't complain about insecure global_id in the test suite 2022-06-29T15:51:33.764 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-06-29T15:51:33.765 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-06-29T15:51:33.765 INFO:teuthology.orchestra.run.smithi099.stdout: mon op complaint time = 120 2022-06-29T15:51:33.765 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:33.765 INFO:teuthology.orchestra.run.smithi099.stdout:[client] 2022-06-29T15:51:33.766 INFO:teuthology.orchestra.run.smithi099.stdout: rgw cache enabled = true 2022-06-29T15:51:33.766 INFO:teuthology.orchestra.run.smithi099.stdout: rgw enable ops log = true 2022-06-29T15:51:33.766 INFO:teuthology.orchestra.run.smithi099.stdout: rgw enable usage log = true 2022-06-29T15:51:33.766 INFO:teuthology.orchestra.run.smithi099.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-06-29T15:51:33.766 INFO:teuthology.orchestra.run.smithi099.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-06-29T15:51:33.767 INFO:teuthology.orchestra.run.smithi099.stdout: client mount timeout = 600 2022-06-29T15:51:33.767 INFO:teuthology.orchestra.run.smithi099.stdout: debug client = 20 2022-06-29T15:51:33.767 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2022-06-29T15:51:33.767 INFO:teuthology.orchestra.run.smithi099.stdout: rados mon op timeout = 15m 2022-06-29T15:51:33.768 INFO:teuthology.orchestra.run.smithi099.stdout: rados osd op timeout = 15m 2022-06-29T15:51:33.768 INFO:teuthology.orchestra.run.smithi099.stdout: client debug getattr caps = true 2022-06-29T15:51:33.768 INFO:teuthology.orchestra.run.smithi099.stdout:[mon.a] 2022-06-29T15:51:33.768 INFO:teuthology.orchestra.run.smithi099.stdout:[mon.b] 2022-06-29T15:51:33.768 INFO:teuthology.orchestra.run.smithi099.stdout:[mon.c] 2022-06-29T15:51:33.769 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.a] 2022-06-29T15:51:33.769 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.c] 2022-06-29T15:51:33.769 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.b] 2022-06-29T15:51:33.769 INFO:teuthology.orchestra.run.smithi099.stdout:[mds] 2022-06-29T15:51:33.770 INFO:teuthology.orchestra.run.smithi099.stdout: debug mds = 20 2022-06-29T15:51:33.770 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2022-06-29T15:51:33.770 INFO:teuthology.orchestra.run.smithi099.stdout: mds bal frag = True 2022-06-29T15:51:33.770 INFO:teuthology.orchestra.run.smithi099.stdout: mds bal fragment size max = 10000 2022-06-29T15:51:33.770 INFO:teuthology.orchestra.run.smithi099.stdout: mds bal merge size = 5 2022-06-29T15:51:33.771 INFO:teuthology.orchestra.run.smithi099.stdout: mds bal split bits = 3 2022-06-29T15:51:33.771 INFO:teuthology.orchestra.run.smithi099.stdout: mds bal split size = 100 2022-06-29T15:51:33.771 INFO:teuthology.orchestra.run.smithi099.stdout: mds debug frag = True 2022-06-29T15:51:33.771 INFO:teuthology.orchestra.run.smithi099.stdout: mds debug scatterstat = True 2022-06-29T15:51:33.772 INFO:teuthology.orchestra.run.smithi099.stdout: mds op complaint time = 180 2022-06-29T15:51:33.772 INFO:teuthology.orchestra.run.smithi099.stdout: mds verify scatter = True 2022-06-29T15:51:33.772 INFO:teuthology.orchestra.run.smithi099.stdout: osd op complaint time = 180 2022-06-29T15:51:33.772 INFO:teuthology.orchestra.run.smithi099.stdout: rados mon op timeout = 15m 2022-06-29T15:51:33.772 INFO:teuthology.orchestra.run.smithi099.stdout: rados osd op timeout = 15m 2022-06-29T15:51:33.773 INFO:teuthology.orchestra.run.smithi164.stdout:[global] 2022-06-29T15:51:33.773 INFO:teuthology.orchestra.run.smithi164.stdout: chdir = "" 2022-06-29T15:51:33.773 INFO:teuthology.orchestra.run.smithi164.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-06-29T15:51:33.773 INFO:teuthology.orchestra.run.smithi164.stdout: auth supported = cephx 2022-06-29T15:51:33.774 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:33.774 INFO:teuthology.orchestra.run.smithi164.stdout: filestore xattr use omap = true 2022-06-29T15:51:33.774 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:33.774 INFO:teuthology.orchestra.run.smithi164.stdout: mon clock drift allowed = 1.000 2022-06-29T15:51:33.775 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:33.775 INFO:teuthology.orchestra.run.smithi164.stdout: osd crush chooseleaf type = 0 2022-06-29T15:51:33.775 INFO:teuthology.orchestra.run.smithi164.stdout: auth debug = true 2022-06-29T15:51:33.775 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:33.776 INFO:teuthology.orchestra.run.smithi164.stdout: ms die on old message = true 2022-06-29T15:51:33.776 INFO:teuthology.orchestra.run.smithi164.stdout: ms die on bug = true 2022-06-29T15:51:33.776 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:33.776 INFO:teuthology.orchestra.run.smithi164.stdout: mon max pg per osd = 10000 # >= luminous 2022-06-29T15:51:33.776 INFO:teuthology.orchestra.run.smithi164.stdout: mon pg warn max object skew = 0 2022-06-29T15:51:33.777 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:33.777 INFO:teuthology.orchestra.run.smithi164.stdout: # disable pg_autoscaler by default for new pools 2022-06-29T15:51:33.777 INFO:teuthology.orchestra.run.smithi164.stdout: osd_pool_default_pg_autoscale_mode = off 2022-06-29T15:51:33.777 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:33.778 INFO:teuthology.orchestra.run.smithi164.stdout: osd pool default size = 2 2022-06-29T15:51:33.778 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:33.778 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd allow primary affinity = true 2022-06-29T15:51:33.778 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd allow pg remap = true 2022-06-29T15:51:33.778 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on legacy crush tunables = false 2022-06-29T15:51:33.779 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on crush straw calc version zero = false 2022-06-29T15:51:33.779 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on no sortbitwise = false 2022-06-29T15:51:33.779 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on osd down out interval zero = false 2022-06-29T15:51:33.779 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on too few osds = false 2022-06-29T15:51:33.780 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-06-29T15:51:33.780 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_pool_no_redundancy = false 2022-06-29T15:51:33.780 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:33.780 INFO:teuthology.orchestra.run.smithi164.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-06-29T15:51:33.781 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:33.781 INFO:teuthology.orchestra.run.smithi164.stdout: osd default data pool replay window = 5 2022-06-29T15:51:33.781 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:33.781 INFO:teuthology.orchestra.run.smithi164.stdout: mon allow pool delete = true 2022-06-29T15:51:33.781 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:33.782 INFO:teuthology.orchestra.run.smithi164.stdout: mon cluster log file level = debug 2022-06-29T15:51:33.782 INFO:teuthology.orchestra.run.smithi164.stdout: debug asserts on shutdown = true 2022-06-29T15:51:33.782 INFO:teuthology.orchestra.run.smithi164.stdout: mon health detail to clog = false 2022-06-29T15:51:33.782 INFO:teuthology.orchestra.run.smithi164.stdout: mon host = "172.21.15.47,172.21.15.64,[v2:172.21.15.64:3301,v1:172.21.15.64:6790]" 2022-06-29T15:51:33.783 INFO:teuthology.orchestra.run.smithi164.stdout: fsid = 9a68894a-53ce-400e-838b-14eab91b834c 2022-06-29T15:51:33.783 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:33.783 INFO:teuthology.orchestra.run.smithi164.stdout:[osd] 2022-06-29T15:51:33.783 INFO:teuthology.orchestra.run.smithi164.stdout: osd journal size = 100 2022-06-29T15:51:33.784 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:33.784 INFO:teuthology.orchestra.run.smithi164.stdout: osd scrub load threshold = 5.0 2022-06-29T15:51:33.784 INFO:teuthology.orchestra.run.smithi164.stdout: osd scrub max interval = 600 2022-06-29T15:51:33.784 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:33.784 INFO:teuthology.orchestra.run.smithi164.stdout: osd recover clone overlap = true 2022-06-29T15:51:33.785 INFO:teuthology.orchestra.run.smithi164.stdout: osd recovery max chunk = 1048576 2022-06-29T15:51:33.785 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:33.785 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug shutdown = true 2022-06-29T15:51:33.785 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug op order = true 2022-06-29T15:51:33.786 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug verify stray on activate = true 2022-06-29T15:51:33.786 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:33.786 INFO:teuthology.orchestra.run.smithi164.stdout: osd open classes on start = true 2022-06-29T15:51:33.786 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug pg log writeout = true 2022-06-29T15:51:33.786 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:33.787 INFO:teuthology.orchestra.run.smithi164.stdout: osd deep scrub update digest min age = 30 2022-06-29T15:51:33.787 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:33.787 INFO:teuthology.orchestra.run.smithi164.stdout: osd map max advance = 10 2022-06-29T15:51:33.787 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:33.788 INFO:teuthology.orchestra.run.smithi164.stdout: journal zero on create = true 2022-06-29T15:51:33.788 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:33.788 INFO:teuthology.orchestra.run.smithi164.stdout: filestore ondisk finisher threads = 3 2022-06-29T15:51:33.788 INFO:teuthology.orchestra.run.smithi164.stdout: filestore apply finisher threads = 3 2022-06-29T15:51:33.788 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:33.789 INFO:teuthology.orchestra.run.smithi164.stdout: bdev debug aio = true 2022-06-29T15:51:33.789 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug misdirected ops = true 2022-06-29T15:51:33.789 INFO:teuthology.orchestra.run.smithi164.stdout: bluestore block size = 96636764160 2022-06-29T15:51:33.789 INFO:teuthology.orchestra.run.smithi164.stdout: bluestore compression mode = aggressive 2022-06-29T15:51:33.790 INFO:teuthology.orchestra.run.smithi164.stdout: bluestore fsck on mount = True 2022-06-29T15:51:33.790 INFO:teuthology.orchestra.run.smithi164.stdout: debug bluefs = 20 2022-06-29T15:51:33.790 INFO:teuthology.orchestra.run.smithi164.stdout: debug bluestore = 20 2022-06-29T15:51:33.790 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2022-06-29T15:51:33.791 INFO:teuthology.orchestra.run.smithi164.stdout: debug osd = 20 2022-06-29T15:51:33.791 INFO:teuthology.orchestra.run.smithi164.stdout: debug rocksdb = 10 2022-06-29T15:51:33.791 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd backfillfull_ratio = 0.85 2022-06-29T15:51:33.791 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd full ratio = 0.9 2022-06-29T15:51:33.791 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd nearfull ratio = 0.8 2022-06-29T15:51:33.792 INFO:teuthology.orchestra.run.smithi164.stdout: osd failsafe full ratio = 0.95 2022-06-29T15:51:33.792 INFO:teuthology.orchestra.run.smithi164.stdout: osd objectstore = bluestore 2022-06-29T15:51:33.792 INFO:teuthology.orchestra.run.smithi164.stdout: osd op complaint time = 180 2022-06-29T15:51:33.792 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:33.793 INFO:teuthology.orchestra.run.smithi164.stdout:[mgr] 2022-06-29T15:51:33.793 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2022-06-29T15:51:33.793 INFO:teuthology.orchestra.run.smithi164.stdout: debug mgr = 20 2022-06-29T15:51:33.793 INFO:teuthology.orchestra.run.smithi164.stdout: debug mon = 20 2022-06-29T15:51:33.793 INFO:teuthology.orchestra.run.smithi164.stdout: debug auth = 20 2022-06-29T15:51:33.794 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min pgs per osd = 4 2022-06-29T15:51:33.794 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min bytes per osd = 10 2022-06-29T15:51:33.794 INFO:teuthology.orchestra.run.smithi164.stdout: mgr/telemetry/nag = false 2022-06-29T15:51:33.794 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:33.795 INFO:teuthology.orchestra.run.smithi164.stdout:[mon] 2022-06-29T15:51:33.795 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2022-06-29T15:51:33.795 INFO:teuthology.orchestra.run.smithi164.stdout: debug mon = 20 2022-06-29T15:51:33.795 INFO:teuthology.orchestra.run.smithi164.stdout: debug paxos = 20 2022-06-29T15:51:33.796 INFO:teuthology.orchestra.run.smithi164.stdout: debug auth = 20 2022-06-29T15:51:33.796 INFO:teuthology.orchestra.run.smithi164.stdout: mon data avail warn = 5 2022-06-29T15:51:33.796 INFO:teuthology.orchestra.run.smithi164.stdout: mon mgr mkfs grace = 240 2022-06-29T15:51:33.796 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min pgs per osd = 4 2022-06-29T15:51:33.796 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd reporter subtree level = osd 2022-06-29T15:51:33.797 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd prime pg temp = true 2022-06-29T15:51:33.797 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min bytes per osd = 10 2022-06-29T15:51:33.797 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:33.797 INFO:teuthology.orchestra.run.smithi164.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-06-29T15:51:33.798 INFO:teuthology.orchestra.run.smithi164.stdout: auth mon ticket ttl = 660 # 11m 2022-06-29T15:51:33.798 INFO:teuthology.orchestra.run.smithi164.stdout: auth service ticket ttl = 240 # 4m 2022-06-29T15:51:33.798 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:33.798 INFO:teuthology.orchestra.run.smithi164.stdout: # don't complain about insecure global_id in the test suite 2022-06-29T15:51:33.798 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-06-29T15:51:33.799 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-06-29T15:51:33.799 INFO:teuthology.orchestra.run.smithi164.stdout: mon op complaint time = 120 2022-06-29T15:51:33.799 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:33.799 INFO:teuthology.orchestra.run.smithi164.stdout:[client] 2022-06-29T15:51:33.800 INFO:teuthology.orchestra.run.smithi164.stdout: rgw cache enabled = true 2022-06-29T15:51:33.800 INFO:teuthology.orchestra.run.smithi164.stdout: rgw enable ops log = true 2022-06-29T15:51:33.800 INFO:teuthology.orchestra.run.smithi164.stdout: rgw enable usage log = true 2022-06-29T15:51:33.800 INFO:teuthology.orchestra.run.smithi164.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-06-29T15:51:33.800 INFO:teuthology.orchestra.run.smithi164.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-06-29T15:51:33.801 INFO:teuthology.orchestra.run.smithi164.stdout: client mount timeout = 600 2022-06-29T15:51:33.801 INFO:teuthology.orchestra.run.smithi164.stdout: debug client = 20 2022-06-29T15:51:33.801 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2022-06-29T15:51:33.801 INFO:teuthology.orchestra.run.smithi164.stdout: rados mon op timeout = 15m 2022-06-29T15:51:33.802 INFO:teuthology.orchestra.run.smithi164.stdout: rados osd op timeout = 15m 2022-06-29T15:51:33.802 INFO:teuthology.orchestra.run.smithi164.stdout: client debug getattr caps = true 2022-06-29T15:51:33.802 INFO:teuthology.orchestra.run.smithi164.stdout:[mon.a] 2022-06-29T15:51:33.803 INFO:teuthology.orchestra.run.smithi164.stdout:[mon.b] 2022-06-29T15:51:33.803 INFO:teuthology.orchestra.run.smithi164.stdout:[mon.c] 2022-06-29T15:51:33.803 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.a] 2022-06-29T15:51:33.803 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.c] 2022-06-29T15:51:33.803 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.b] 2022-06-29T15:51:33.804 INFO:teuthology.orchestra.run.smithi164.stdout:[mds] 2022-06-29T15:51:33.804 INFO:teuthology.orchestra.run.smithi164.stdout: debug mds = 20 2022-06-29T15:51:33.804 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2022-06-29T15:51:33.804 INFO:teuthology.orchestra.run.smithi164.stdout: mds bal frag = True 2022-06-29T15:51:33.805 INFO:teuthology.orchestra.run.smithi164.stdout: mds bal fragment size max = 10000 2022-06-29T15:51:33.805 INFO:teuthology.orchestra.run.smithi164.stdout: mds bal merge size = 5 2022-06-29T15:51:33.805 INFO:teuthology.orchestra.run.smithi164.stdout: mds bal split bits = 3 2022-06-29T15:51:33.805 INFO:teuthology.orchestra.run.smithi164.stdout: mds bal split size = 100 2022-06-29T15:51:33.805 INFO:teuthology.orchestra.run.smithi164.stdout: mds debug frag = True 2022-06-29T15:51:33.806 INFO:teuthology.orchestra.run.smithi164.stdout: mds debug scatterstat = True 2022-06-29T15:51:33.806 INFO:teuthology.orchestra.run.smithi164.stdout: mds op complaint time = 180 2022-06-29T15:51:33.806 INFO:teuthology.orchestra.run.smithi164.stdout: mds verify scatter = True 2022-06-29T15:51:33.806 INFO:teuthology.orchestra.run.smithi164.stdout: osd op complaint time = 180 2022-06-29T15:51:33.807 INFO:teuthology.orchestra.run.smithi164.stdout: rados mon op timeout = 15m 2022-06-29T15:51:33.807 INFO:teuthology.orchestra.run.smithi164.stdout: rados osd op timeout = 15m 2022-06-29T15:51:33.807 INFO:teuthology.orchestra.run.smithi064.stdout:[global] 2022-06-29T15:51:33.807 INFO:teuthology.orchestra.run.smithi064.stdout: chdir = "" 2022-06-29T15:51:33.808 INFO:teuthology.orchestra.run.smithi064.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-06-29T15:51:33.808 INFO:teuthology.orchestra.run.smithi064.stdout: auth supported = cephx 2022-06-29T15:51:33.808 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:33.808 INFO:teuthology.orchestra.run.smithi064.stdout: filestore xattr use omap = true 2022-06-29T15:51:33.808 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:33.809 INFO:teuthology.orchestra.run.smithi064.stdout: mon clock drift allowed = 1.000 2022-06-29T15:51:33.809 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:33.809 INFO:teuthology.orchestra.run.smithi064.stdout: osd crush chooseleaf type = 0 2022-06-29T15:51:33.809 INFO:teuthology.orchestra.run.smithi064.stdout: auth debug = true 2022-06-29T15:51:33.810 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:33.810 INFO:teuthology.orchestra.run.smithi064.stdout: ms die on old message = true 2022-06-29T15:51:33.810 INFO:teuthology.orchestra.run.smithi064.stdout: ms die on bug = true 2022-06-29T15:51:33.810 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:33.811 INFO:teuthology.orchestra.run.smithi064.stdout: mon max pg per osd = 10000 # >= luminous 2022-06-29T15:51:33.811 INFO:teuthology.orchestra.run.smithi064.stdout: mon pg warn max object skew = 0 2022-06-29T15:51:33.811 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:33.811 INFO:teuthology.orchestra.run.smithi064.stdout: # disable pg_autoscaler by default for new pools 2022-06-29T15:51:33.812 INFO:teuthology.orchestra.run.smithi064.stdout: osd_pool_default_pg_autoscale_mode = off 2022-06-29T15:51:33.812 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:33.812 INFO:teuthology.orchestra.run.smithi064.stdout: osd pool default size = 2 2022-06-29T15:51:33.813 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:33.813 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd allow primary affinity = true 2022-06-29T15:51:33.813 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd allow pg remap = true 2022-06-29T15:51:33.813 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on legacy crush tunables = false 2022-06-29T15:51:33.814 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on crush straw calc version zero = false 2022-06-29T15:51:33.814 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on no sortbitwise = false 2022-06-29T15:51:33.814 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on osd down out interval zero = false 2022-06-29T15:51:33.814 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on too few osds = false 2022-06-29T15:51:33.815 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-06-29T15:51:33.815 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_pool_no_redundancy = false 2022-06-29T15:51:33.815 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:33.815 INFO:teuthology.orchestra.run.smithi064.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-06-29T15:51:33.815 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:33.816 INFO:teuthology.orchestra.run.smithi064.stdout: osd default data pool replay window = 5 2022-06-29T15:51:33.816 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:33.816 INFO:teuthology.orchestra.run.smithi064.stdout: mon allow pool delete = true 2022-06-29T15:51:33.816 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:33.817 INFO:teuthology.orchestra.run.smithi064.stdout: mon cluster log file level = debug 2022-06-29T15:51:33.817 INFO:teuthology.orchestra.run.smithi064.stdout: debug asserts on shutdown = true 2022-06-29T15:51:33.817 INFO:teuthology.orchestra.run.smithi064.stdout: mon health detail to clog = false 2022-06-29T15:51:33.817 INFO:teuthology.orchestra.run.smithi064.stdout: mon host = "172.21.15.47,172.21.15.64,[v2:172.21.15.64:3301,v1:172.21.15.64:6790]" 2022-06-29T15:51:33.818 INFO:teuthology.orchestra.run.smithi064.stdout: fsid = 9a68894a-53ce-400e-838b-14eab91b834c 2022-06-29T15:51:33.818 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:33.818 INFO:teuthology.orchestra.run.smithi064.stdout:[osd] 2022-06-29T15:51:33.818 INFO:teuthology.orchestra.run.smithi064.stdout: osd journal size = 100 2022-06-29T15:51:33.818 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:33.819 INFO:teuthology.orchestra.run.smithi064.stdout: osd scrub load threshold = 5.0 2022-06-29T15:51:33.819 INFO:teuthology.orchestra.run.smithi064.stdout: osd scrub max interval = 600 2022-06-29T15:51:33.819 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:33.819 INFO:teuthology.orchestra.run.smithi064.stdout: osd recover clone overlap = true 2022-06-29T15:51:33.820 INFO:teuthology.orchestra.run.smithi064.stdout: osd recovery max chunk = 1048576 2022-06-29T15:51:33.820 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:33.820 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug shutdown = true 2022-06-29T15:51:33.820 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug op order = true 2022-06-29T15:51:33.820 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug verify stray on activate = true 2022-06-29T15:51:33.821 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:33.821 INFO:teuthology.orchestra.run.smithi064.stdout: osd open classes on start = true 2022-06-29T15:51:33.821 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug pg log writeout = true 2022-06-29T15:51:33.821 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:33.822 INFO:teuthology.orchestra.run.smithi064.stdout: osd deep scrub update digest min age = 30 2022-06-29T15:51:33.822 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:33.822 INFO:teuthology.orchestra.run.smithi064.stdout: osd map max advance = 10 2022-06-29T15:51:33.822 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:33.823 INFO:teuthology.orchestra.run.smithi064.stdout: journal zero on create = true 2022-06-29T15:51:33.823 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:33.823 INFO:teuthology.orchestra.run.smithi064.stdout: filestore ondisk finisher threads = 3 2022-06-29T15:51:33.824 INFO:teuthology.orchestra.run.smithi064.stdout: filestore apply finisher threads = 3 2022-06-29T15:51:33.824 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:33.824 INFO:teuthology.orchestra.run.smithi064.stdout: bdev debug aio = true 2022-06-29T15:51:33.824 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug misdirected ops = true 2022-06-29T15:51:33.825 INFO:teuthology.orchestra.run.smithi064.stdout: bluestore block size = 96636764160 2022-06-29T15:51:33.825 INFO:teuthology.orchestra.run.smithi064.stdout: bluestore compression mode = aggressive 2022-06-29T15:51:33.825 INFO:teuthology.orchestra.run.smithi064.stdout: bluestore fsck on mount = True 2022-06-29T15:51:33.825 INFO:teuthology.orchestra.run.smithi064.stdout: debug bluefs = 20 2022-06-29T15:51:33.825 INFO:teuthology.orchestra.run.smithi064.stdout: debug bluestore = 20 2022-06-29T15:51:33.826 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2022-06-29T15:51:33.826 INFO:teuthology.orchestra.run.smithi064.stdout: debug osd = 20 2022-06-29T15:51:33.826 INFO:teuthology.orchestra.run.smithi064.stdout: debug rocksdb = 10 2022-06-29T15:51:33.826 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd backfillfull_ratio = 0.85 2022-06-29T15:51:33.827 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd full ratio = 0.9 2022-06-29T15:51:33.827 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd nearfull ratio = 0.8 2022-06-29T15:51:33.827 INFO:teuthology.orchestra.run.smithi064.stdout: osd failsafe full ratio = 0.95 2022-06-29T15:51:33.827 INFO:teuthology.orchestra.run.smithi064.stdout: osd objectstore = bluestore 2022-06-29T15:51:33.827 INFO:teuthology.orchestra.run.smithi064.stdout: osd op complaint time = 180 2022-06-29T15:51:33.828 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:33.828 INFO:teuthology.orchestra.run.smithi064.stdout:[mgr] 2022-06-29T15:51:33.828 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2022-06-29T15:51:33.828 INFO:teuthology.orchestra.run.smithi064.stdout: debug mgr = 20 2022-06-29T15:51:33.829 INFO:teuthology.orchestra.run.smithi064.stdout: debug mon = 20 2022-06-29T15:51:33.829 INFO:teuthology.orchestra.run.smithi064.stdout: debug auth = 20 2022-06-29T15:51:33.829 INFO:teuthology.orchestra.run.smithi064.stdout: mon reweight min pgs per osd = 4 2022-06-29T15:51:33.829 INFO:teuthology.orchestra.run.smithi064.stdout: mon reweight min bytes per osd = 10 2022-06-29T15:51:33.829 INFO:teuthology.orchestra.run.smithi064.stdout: mgr/telemetry/nag = false 2022-06-29T15:51:33.830 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:33.830 INFO:teuthology.orchestra.run.smithi064.stdout:[mon] 2022-06-29T15:51:33.830 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2022-06-29T15:51:33.830 INFO:teuthology.orchestra.run.smithi064.stdout: debug mon = 20 2022-06-29T15:51:33.831 INFO:teuthology.orchestra.run.smithi064.stdout: debug paxos = 20 2022-06-29T15:51:33.831 INFO:teuthology.orchestra.run.smithi064.stdout: debug auth = 20 2022-06-29T15:51:33.831 INFO:teuthology.orchestra.run.smithi064.stdout: mon data avail warn = 5 2022-06-29T15:51:33.831 INFO:teuthology.orchestra.run.smithi064.stdout: mon mgr mkfs grace = 240 2022-06-29T15:51:33.832 INFO:teuthology.orchestra.run.smithi064.stdout: mon reweight min pgs per osd = 4 2022-06-29T15:51:33.832 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd reporter subtree level = osd 2022-06-29T15:51:33.832 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd prime pg temp = true 2022-06-29T15:51:33.832 INFO:teuthology.orchestra.run.smithi064.stdout: mon reweight min bytes per osd = 10 2022-06-29T15:51:33.832 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:33.833 INFO:teuthology.orchestra.run.smithi064.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-06-29T15:51:33.833 INFO:teuthology.orchestra.run.smithi064.stdout: auth mon ticket ttl = 660 # 11m 2022-06-29T15:51:33.833 INFO:teuthology.orchestra.run.smithi064.stdout: auth service ticket ttl = 240 # 4m 2022-06-29T15:51:33.833 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:33.834 INFO:teuthology.orchestra.run.smithi064.stdout: # don't complain about insecure global_id in the test suite 2022-06-29T15:51:33.834 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-06-29T15:51:33.834 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-06-29T15:51:33.834 INFO:teuthology.orchestra.run.smithi064.stdout: mon op complaint time = 120 2022-06-29T15:51:33.834 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:33.835 INFO:teuthology.orchestra.run.smithi064.stdout:[client] 2022-06-29T15:51:33.835 INFO:teuthology.orchestra.run.smithi064.stdout: rgw cache enabled = true 2022-06-29T15:51:33.835 INFO:teuthology.orchestra.run.smithi064.stdout: rgw enable ops log = true 2022-06-29T15:51:33.835 INFO:teuthology.orchestra.run.smithi064.stdout: rgw enable usage log = true 2022-06-29T15:51:33.836 INFO:teuthology.orchestra.run.smithi064.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-06-29T15:51:33.836 INFO:teuthology.orchestra.run.smithi064.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-06-29T15:51:33.836 INFO:teuthology.orchestra.run.smithi064.stdout: client mount timeout = 600 2022-06-29T15:51:33.836 INFO:teuthology.orchestra.run.smithi064.stdout: debug client = 20 2022-06-29T15:51:33.836 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2022-06-29T15:51:33.837 INFO:teuthology.orchestra.run.smithi064.stdout: rados mon op timeout = 15m 2022-06-29T15:51:33.837 INFO:teuthology.orchestra.run.smithi064.stdout: rados osd op timeout = 15m 2022-06-29T15:51:33.837 INFO:teuthology.orchestra.run.smithi064.stdout: client debug getattr caps = true 2022-06-29T15:51:33.837 INFO:teuthology.orchestra.run.smithi064.stdout:[mon.a] 2022-06-29T15:51:33.838 INFO:teuthology.orchestra.run.smithi064.stdout:[mon.b] 2022-06-29T15:51:33.838 INFO:teuthology.orchestra.run.smithi064.stdout:[mon.c] 2022-06-29T15:51:33.838 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.a] 2022-06-29T15:51:33.838 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.c] 2022-06-29T15:51:33.839 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.b] 2022-06-29T15:51:33.839 INFO:teuthology.orchestra.run.smithi064.stdout:[mds] 2022-06-29T15:51:33.839 INFO:teuthology.orchestra.run.smithi064.stdout: debug mds = 20 2022-06-29T15:51:33.839 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2022-06-29T15:51:33.839 INFO:teuthology.orchestra.run.smithi064.stdout: mds bal frag = True 2022-06-29T15:51:33.840 INFO:teuthology.orchestra.run.smithi064.stdout: mds bal fragment size max = 10000 2022-06-29T15:51:33.840 INFO:teuthology.orchestra.run.smithi064.stdout: mds bal merge size = 5 2022-06-29T15:51:33.840 INFO:teuthology.orchestra.run.smithi064.stdout: mds bal split bits = 3 2022-06-29T15:51:33.840 INFO:teuthology.orchestra.run.smithi064.stdout: mds bal split size = 100 2022-06-29T15:51:33.840 INFO:teuthology.orchestra.run.smithi064.stdout: mds debug frag = True 2022-06-29T15:51:33.841 INFO:teuthology.orchestra.run.smithi064.stdout: mds debug scatterstat = True 2022-06-29T15:51:33.841 INFO:teuthology.orchestra.run.smithi064.stdout: mds op complaint time = 180 2022-06-29T15:51:33.841 INFO:teuthology.orchestra.run.smithi064.stdout: mds verify scatter = True 2022-06-29T15:51:33.841 INFO:teuthology.orchestra.run.smithi064.stdout: osd op complaint time = 180 2022-06-29T15:51:33.842 INFO:teuthology.orchestra.run.smithi064.stdout: rados mon op timeout = 15m 2022-06-29T15:51:33.842 INFO:teuthology.orchestra.run.smithi064.stdout: rados osd op timeout = 15m 2022-06-29T15:51:33.843 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:51:33.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.946+0000 7fad0312f700 1 Processor -- start 2022-06-29T15:51:33.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.946+0000 7fad0312f700 1 -- start start 2022-06-29T15:51:33.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.946+0000 7fad0312f700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7facfc148060 0x7facfc0a7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:33.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.946+0000 7fad0312f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7facfc0a8ab0 0x7facfc1444c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:33.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.946+0000 7fad0312f700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facfc060700 con 0x7facfc060320 2022-06-29T15:51:33.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.946+0000 7fad0312f700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facfc1485a0 con 0x7facfc0a8ab0 2022-06-29T15:51:33.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.946+0000 7fad0312f700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facfc0b0040 con 0x7facfc148060 2022-06-29T15:51:33.958 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.946+0000 7fad00ecb700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7facfc060320 0x7facfc0b0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39454/0 (socket says 172.21.15.47:39454) 2022-06-29T15:51:33.958 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.946+0000 7fad00ecb700 1 -- 172.21.15.47:0/361243550 learned_addr learned my addr 172.21.15.47:0/361243550 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:33.958 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.946+0000 7facfbfff700 1 --2- 172.21.15.47:0/361243550 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7facfc148060 0x7facfc0a7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:33.958 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.946+0000 7fad016cc700 1 --2- 172.21.15.47:0/361243550 >> v2:172.21.15.64:3300/0 conn(0x7facfc0a8ab0 0x7facfc1444c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:33.959 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.946+0000 7facfb7fe700 1 -- 172.21.15.47:0/361243550 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2260391148 0 0) 0x7facfc060700 con 0x7facfc060320 2022-06-29T15:51:33.959 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.946+0000 7facfb7fe700 1 -- 172.21.15.47:0/361243550 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facdc003260 con 0x7facfc060320 2022-06-29T15:51:33.959 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.946+0000 7facfb7fe700 1 -- 172.21.15.47:0/361243550 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1203574535 0 0) 0x7facdc003260 con 0x7facfc060320 2022-06-29T15:51:33.959 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.946+0000 7facfb7fe700 1 -- 172.21.15.47:0/361243550 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7facfc148060 msgr2=0x7facfc0a7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:33.960 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.946+0000 7facfb7fe700 1 --2- 172.21.15.47:0/361243550 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7facfc148060 0x7facfc0a7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:33.960 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.946+0000 7facfb7fe700 1 -- 172.21.15.47:0/361243550 >> v2:172.21.15.64:3300/0 conn(0x7facfc0a8ab0 msgr2=0x7facfc1444c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:33.960 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.946+0000 7facfb7fe700 1 --2- 172.21.15.47:0/361243550 >> v2:172.21.15.64:3300/0 conn(0x7facfc0a8ab0 0x7facfc1444c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:33.960 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.946+0000 7facfb7fe700 1 -- 172.21.15.47:0/361243550 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facfc0b02e0 con 0x7facfc060320 2022-06-29T15:51:33.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.946+0000 7facfb7fe700 1 -- 172.21.15.47:0/361243550 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7face4001680 con 0x7facfc060320 2022-06-29T15:51:33.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.946+0000 7facfb7fe700 1 -- 172.21.15.47:0/361243550 >> v1:172.21.15.47:6789/0 conn(0x7facfc060320 legacy=0x7facfc0b0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:33.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.946+0000 7facfb7fe700 1 --2- 172.21.15.47:0/361243550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facdc003750 0x7facdc003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:33.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.946+0000 7facfb7fe700 1 --2- 172.21.15.47:0/361243550 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7facfc148060 0x7facdc004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:33.962 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.946+0000 7facfb7fe700 1 --2- 172.21.15.47:0/361243550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7facdc0049f0 0x7facdc004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:33.962 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.946+0000 7facfb7fe700 1 -- 172.21.15.47:0/361243550 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facfc1485a0 con 0x7facdc0049f0 2022-06-29T15:51:33.963 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.946+0000 7facfb7fe700 1 -- 172.21.15.47:0/361243550 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facfc0b0040 con 0x7facdc003750 2022-06-29T15:51:33.964 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.946+0000 7facfb7fe700 1 -- 172.21.15.47:0/361243550 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facdc00b750 con 0x7facfc148060 2022-06-29T15:51:33.964 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.946+0000 7fad016cc700 1 --2- 172.21.15.47:0/361243550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7facdc0049f0 0x7facdc004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:33.965 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.946+0000 7facfbfff700 1 --2- 172.21.15.47:0/361243550 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7facfc148060 0x7facdc004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:33.965 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.946+0000 7fad016cc700 1 -- 172.21.15.47:0/361243550 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7facfc148060 msgr2=0x7facdc004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:33.965 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.946+0000 7fad016cc700 1 --2- 172.21.15.47:0/361243550 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7facfc148060 0x7facdc004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:33.966 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.946+0000 7fad016cc700 1 -- 172.21.15.47:0/361243550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facdc003750 msgr2=0x7facdc003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:33.966 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.946+0000 7fad016cc700 1 --2- 172.21.15.47:0/361243550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facdc003750 0x7facdc003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2022-06-29T15:51:33.966 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.946+0000 7fad016cc700 1 -- 172.21.15.47:0/361243550 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facdc003260 con 0x7facdc0049f0 2022-06-29T15:51:33.966 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.946+0000 7fad016cc700 1 --2- 172.21.15.47:0/361243550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7facdc0049f0 0x7facdc004e70 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 rx=0x7facf400c000 tx=0x7facf40091d0).ready entity=mon.0 client_cookie=3efc2b3cbf057a8a server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:33.967 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.946+0000 7facfb7fe700 1 -- 172.21.15.47:0/361243550 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7facf400e7e0 con 0x7facdc0049f0 2022-06-29T15:51:33.967 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.946+0000 7facfb7fe700 1 -- 172.21.15.47:0/361243550 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facf400e940 con 0x7facdc0049f0 2022-06-29T15:51:33.967 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.946+0000 7facfb7fe700 1 -- 172.21.15.47:0/361243550 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7facf400eca0 con 0x7facdc0049f0 2022-06-29T15:51:33.968 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.950+0000 7fad0312f700 1 -- 172.21.15.47:0/361243550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7facdc0049f0 msgr2=0x7facdc004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:33.968 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.950+0000 7fad0312f700 1 --2- 172.21.15.47:0/361243550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7facdc0049f0 0x7facdc004e70 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 rx=0x7facf400c000 tx=0x7facf40091d0).stop 2022-06-29T15:51:33.968 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.950+0000 7fad016cc700 1 -- 172.21.15.47:0/361243550 reap_dead start 2022-06-29T15:51:33.968 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.950+0000 7fad0312f700 1 -- 172.21.15.47:0/361243550 shutdown_connections 2022-06-29T15:51:33.969 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.950+0000 7fad0312f700 1 -- 172.21.15.47:0/361243550 >> 172.21.15.47:0/361243550 conn(0x7facfc05f840 msgr2=0x7facfc0b5600 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:33.969 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.950+0000 7fad0312f700 1 -- 172.21.15.47:0/361243550 shutdown_connections 2022-06-29T15:51:33.969 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.950+0000 7fad0312f700 1 -- 172.21.15.47:0/361243550 wait complete. 2022-06-29T15:51:33.973 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.950+0000 7fad0312f700 1 Processor -- start 2022-06-29T15:51:33.973 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.950+0000 7fad0312f700 1 -- start start 2022-06-29T15:51:33.973 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.954+0000 7fad0312f700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7facfc0a9730 0x7facfc0af320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:33.973 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.954+0000 7fad0312f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facfc15a230 0x7facfc15a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:33.974 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.954+0000 7fad0312f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7facfc0abaa0 0x7facfc0abf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:33.974 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.954+0000 7fad0312f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facfc15aba0 con 0x7facfc0abaa0 2022-06-29T15:51:33.974 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.954+0000 7fad0312f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facfc0afcd0 con 0x7facfc15a230 2022-06-29T15:51:33.974 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.954+0000 7fad0312f700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facfc159c40 con 0x7facfc0a9730 2022-06-29T15:51:33.975 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.954+0000 7fad00ecb700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7facfc0a9730 0x7facfc0af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:33.975 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.954+0000 7fad016cc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7facfc0abaa0 0x7facfc0abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:33.975 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.954+0000 7fad00ecb700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7facfc0a9730 0x7facfc0af320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:51014/0 (socket says 172.21.15.47:51014) 2022-06-29T15:51:33.975 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.954+0000 7fad00ecb700 1 -- 172.21.15.47:0/516449926 learned_addr learned my addr 172.21.15.47:0/516449926 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:33.976 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.954+0000 7facfbfff700 1 --2- 172.21.15.47:0/516449926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facfc15a230 0x7facfc15a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:33.976 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.954+0000 7fad00ecb700 1 -- 172.21.15.47:0/516449926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facfc15a230 msgr2=0x7facfc15a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:33.976 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.954+0000 7fad00ecb700 1 --2- 172.21.15.47:0/516449926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facfc15a230 0x7facfc15a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:33.976 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.954+0000 7fad00ecb700 1 -- 172.21.15.47:0/516449926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7facfc0abaa0 msgr2=0x7facfc0abf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:33.977 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.954+0000 7fad00ecb700 1 --2- 172.21.15.47:0/516449926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7facfc0abaa0 0x7facfc0abf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:33.977 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.954+0000 7fad00ecb700 1 -- 172.21.15.47:0/516449926 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facfc0a9b10 con 0x7facfc0a9730 2022-06-29T15:51:33.977 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.954+0000 7fad00ecb700 1 --2- 172.21.15.47:0/516449926 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7facfc0a9730 0x7facfc0af320 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7facfc14b460 tx=0x7facfc0acd90).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:33.977 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.954+0000 7facf9ffb700 1 -- 172.21.15.47:0/516449926 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7face4007170 con 0x7facfc0a9730 2022-06-29T15:51:33.978 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.954+0000 7facf9ffb700 1 -- 172.21.15.47:0/516449926 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face40072d0 con 0x7facfc0a9730 2022-06-29T15:51:33.978 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.954+0000 7fad0312f700 1 -- 172.21.15.47:0/516449926 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facfc0aadc0 con 0x7facfc0a9730 2022-06-29T15:51:33.978 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.954+0000 7facf9ffb700 1 -- 172.21.15.47:0/516449926 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7face40049e0 con 0x7facfc0a9730 2022-06-29T15:51:33.978 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.954+0000 7fad0312f700 1 -- 172.21.15.47:0/516449926 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facfc0ab300 con 0x7facfc0a9730 2022-06-29T15:51:33.979 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.958+0000 7facf9ffb700 1 -- 172.21.15.47:0/516449926 <== mon.2 v2:172.21.15.64:3301/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7face4004b40 con 0x7facfc0a9730 2022-06-29T15:51:33.979 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.958+0000 7fad0312f700 1 -- 172.21.15.47:0/516449926 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facfc0ab740 con 0x7facfc0a9730 2022-06-29T15:51:33.979 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.958+0000 7facf9ffb700 1 --2- 172.21.15.47:0/516449926 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7face0053aa0 0x7face0055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:33.979 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.958+0000 7facf9ffb700 1 -- 172.21.15.47:0/516449926 <== mon.2 v2:172.21.15.64:3301/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6420+0+0 (secure 0 0 0) 0x7face406bc70 con 0x7facfc0a9730 2022-06-29T15:51:33.979 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.958+0000 7facfbfff700 1 --2- 172.21.15.47:0/516449926 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7face0053aa0 0x7face0055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:33.980 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.958+0000 7facfbfff700 1 --2- 172.21.15.47:0/516449926 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7face0053aa0 0x7face0055f60 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7facec006000 tx=0x7facec008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:33.980 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:33.966+0000 7facf9ffb700 1 -- 172.21.15.47:0/516449926 <== mon.2 v2:172.21.15.64:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7face40428a0 con 0x7facfc0a9730 2022-06-29T15:51:34.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.098+0000 7fad0312f700 1 -- 172.21.15.47:0/516449926 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7facfc0573d0 con 0x7facfc0a9730 2022-06-29T15:51:34.105 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.098+0000 7facf9ffb700 1 -- 172.21.15.47:0/516449926 <== mon.2 v2:172.21.15.64:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) v1 ==== 74+0+16285 (secure 0 0 0) 0x7face4045ec0 con 0x7facfc0a9730 2022-06-29T15:51:34.106 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:34.106 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":140,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:51:24.546533+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:51:21.506960+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2022-06-29T15:51:22.546983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:51:34.108 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.102+0000 7fad0312f700 1 -- 172.21.15.47:0/516449926 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7face0053aa0 msgr2=0x7face0055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:34.108 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.102+0000 7fad0312f700 1 --2- 172.21.15.47:0/516449926 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7face0053aa0 0x7face0055f60 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7facec006000 tx=0x7facec008040).stop 2022-06-29T15:51:34.109 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.102+0000 7fad0312f700 1 -- 172.21.15.47:0/516449926 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7facfc0a9730 msgr2=0x7facfc0af320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:34.109 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.102+0000 7fad0312f700 1 --2- 172.21.15.47:0/516449926 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7facfc0a9730 0x7facfc0af320 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7facfc14b460 tx=0x7facfc0acd90).stop 2022-06-29T15:51:34.109 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.102+0000 7fad0312f700 1 -- 172.21.15.47:0/516449926 shutdown_connections 2022-06-29T15:51:34.109 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.102+0000 7fad0312f700 1 --2- 172.21.15.47:0/516449926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7facfc0abaa0 0x7facfc0abf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:34.110 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.102+0000 7fad0312f700 1 --2- 172.21.15.47:0/516449926 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7face0053aa0 0x7face0055f60 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:34.110 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.102+0000 7fad0312f700 1 --2- 172.21.15.47:0/516449926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facfc15a230 0x7facfc15a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:34.110 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.102+0000 7fad0312f700 1 --2- 172.21.15.47:0/516449926 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7facfc0a9730 0x7facfc0af320 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:34.110 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.102+0000 7fad0312f700 1 -- 172.21.15.47:0/516449926 >> 172.21.15.47:0/516449926 conn(0x7facfc05f840 msgr2=0x7facfc0b55e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:34.111 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.102+0000 7fad0312f700 1 -- 172.21.15.47:0/516449926 shutdown_connections 2022-06-29T15:51:34.111 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.102+0000 7fad0312f700 1 -- 172.21.15.47:0/516449926 wait complete. 2022-06-29T15:51:34.119 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-29T15:42:16.971431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-29T15:42:25.290884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-29T15:51:21.506960+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '139', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2022-06-29T15:51:22.546983+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '140', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-29T15:51:34.120 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-29T15:51:34.229 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.222+0000 7f1f073f2700 1 Processor -- start 2022-06-29T15:51:34.229 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.222+0000 7f1f073f2700 1 -- start start 2022-06-29T15:51:34.231 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.222+0000 7f1f073f2700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1f000b6420 0x7f1f00062290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:34.232 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.222+0000 7f1f073f2700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1f000b5c40 0x7f1f00062780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:34.232 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.222+0000 7f1f073f2700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f001477b0 con 0x7f1f00146870 2022-06-29T15:51:34.232 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.222+0000 7f1f073f2700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f001432a0 con 0x7f1f000b6420 2022-06-29T15:51:34.232 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.222+0000 7f1f073f2700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f00143490 con 0x7f1f000b5c40 2022-06-29T15:51:34.232 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.222+0000 7f1f06bf1700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1f00146870 0x7f1f00061430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39474/0 (socket says 172.21.15.47:39474) 2022-06-29T15:51:34.233 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.222+0000 7f1f06bf1700 1 -- 172.21.15.47:0/734998582 learned_addr learned my addr 172.21.15.47:0/734998582 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:34.233 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.222+0000 7f1f063f0700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1f000b6420 0x7f1f00062290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:34.233 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.222+0000 7f1f05bef700 1 --2- 172.21.15.47:0/734998582 >> v2:172.21.15.64:3300/0 conn(0x7f1f000b5c40 0x7f1f00062780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:34.233 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.222+0000 7f1f053ee700 1 -- 172.21.15.47:0/734998582 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2186996894 0 0) 0x7f1f001477b0 con 0x7f1f00146870 2022-06-29T15:51:34.234 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.222+0000 7f1f053ee700 1 -- 172.21.15.47:0/734998582 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ee8003260 con 0x7f1f00146870 2022-06-29T15:51:34.234 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.222+0000 7f1f053ee700 1 -- 172.21.15.47:0/734998582 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 941927367 0 0) 0x7f1ee8003260 con 0x7f1f00146870 2022-06-29T15:51:34.234 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.222+0000 7f1f063f0700 1 -- 172.21.15.47:0/734998582 >> v2:172.21.15.64:3300/0 conn(0x7f1f000b5c40 msgr2=0x7f1f00062780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:34.234 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.222+0000 7f1f063f0700 1 --2- 172.21.15.47:0/734998582 >> v2:172.21.15.64:3300/0 conn(0x7f1f000b5c40 0x7f1f00062780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:34.235 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.222+0000 7f1f063f0700 1 -- 172.21.15.47:0/734998582 >> v1:172.21.15.47:6789/0 conn(0x7f1f00146870 legacy=0x7f1f00061430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:34.235 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.222+0000 7f1f063f0700 1 -- 172.21.15.47:0/734998582 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f00143730 con 0x7f1f000b6420 2022-06-29T15:51:34.235 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.226+0000 7f1f063f0700 1 --2- 172.21.15.47:0/734998582 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f000b6420 0x7f1f00062290 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 rx=0x7f1ef8001800 tx=0x7f1ef0006df0).ready entity=mon.0 client_cookie=86e3f27ca2fc2365 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:34.235 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.226+0000 7f1f053ee700 1 -- 172.21.15.47:0/734998582 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ef8001660 con 0x7f1f000b6420 2022-06-29T15:51:34.235 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.226+0000 7f1f053ee700 1 -- 172.21.15.47:0/734998582 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef0000e90 con 0x7f1f000b6420 2022-06-29T15:51:34.236 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.226+0000 7f1f053ee700 1 -- 172.21.15.47:0/734998582 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ef0012460 con 0x7f1f000b6420 2022-06-29T15:51:34.239 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.226+0000 7f1f073f2700 1 -- 172.21.15.47:0/734998582 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f000b6420 msgr2=0x7f1f00062290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:34.239 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.226+0000 7f1f073f2700 1 --2- 172.21.15.47:0/734998582 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f000b6420 0x7f1f00062290 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 rx=0x7f1ef8001800 tx=0x7f1ef0006df0).stop 2022-06-29T15:51:34.239 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.226+0000 7f1f073f2700 1 -- 172.21.15.47:0/734998582 shutdown_connections 2022-06-29T15:51:34.239 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.226+0000 7f1f073f2700 1 --2- 172.21.15.47:0/734998582 >> v2:172.21.15.64:3300/0 conn(0x7f1f000b5c40 0x7f1f00062780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:34.239 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.226+0000 7f1f073f2700 1 --2- 172.21.15.47:0/734998582 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f000b6420 0x7f1f00062290 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:34.240 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.226+0000 7f1f073f2700 1 -- 172.21.15.47:0/734998582 >> 172.21.15.47:0/734998582 conn(0x7f1f0005c5b0 msgr2=0x7f1f0005fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:34.240 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.226+0000 7f1f073f2700 1 -- 172.21.15.47:0/734998582 shutdown_connections 2022-06-29T15:51:34.240 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.226+0000 7f1f073f2700 1 -- 172.21.15.47:0/734998582 wait complete. 2022-06-29T15:51:34.240 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.226+0000 7f1f073f2700 1 Processor -- start 2022-06-29T15:51:34.241 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.226+0000 7f1f073f2700 1 -- start start 2022-06-29T15:51:34.241 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.230+0000 7f1f073f2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f000b5c40 0x7f1f000b9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:34.241 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.230+0000 7f1f073f2700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1f000b6420 0x7f1f000ae020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:34.241 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.230+0000 7f1f073f2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f00146870 0x7f1f000b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:34.241 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.230+0000 7f1f073f2700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f000b8730 con 0x7f1f000b5c40 2022-06-29T15:51:34.242 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.230+0000 7f1f073f2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f000b8070 con 0x7f1f00146870 2022-06-29T15:51:34.242 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.230+0000 7f1f073f2700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f00155b70 con 0x7f1f000b6420 2022-06-29T15:51:34.242 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.230+0000 7f1f063f0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f000b5c40 0x7f1f000b9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:34.242 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.230+0000 7f1f063f0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f000b5c40 0x7f1f000b9330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53924/0 (socket says 172.21.15.47:53924) 2022-06-29T15:51:34.243 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.230+0000 7f1f063f0700 1 -- 172.21.15.47:0/3456154238 learned_addr learned my addr 172.21.15.47:0/3456154238 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:34.243 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.230+0000 7f1f06bf1700 1 --2- 172.21.15.47:0/3456154238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f00146870 0x7f1f000b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:34.243 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.230+0000 7f1f05bef700 1 --2- 172.21.15.47:0/3456154238 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1f000b6420 0x7f1f000ae020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:34.243 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.230+0000 7f1f063f0700 1 -- 172.21.15.47:0/3456154238 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1f000b6420 msgr2=0x7f1f000ae020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:34.243 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.230+0000 7f1f063f0700 1 --2- 172.21.15.47:0/3456154238 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1f000b6420 0x7f1f000ae020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:34.244 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.230+0000 7f1f063f0700 1 -- 172.21.15.47:0/3456154238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f00146870 msgr2=0x7f1f000b8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:34.244 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.230+0000 7f1f063f0700 1 --2- 172.21.15.47:0/3456154238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f00146870 0x7f1f000b8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:34.244 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.230+0000 7f1f063f0700 1 -- 172.21.15.47:0/3456154238 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef000b180 con 0x7f1f000b5c40 2022-06-29T15:51:34.244 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.230+0000 7f1f063f0700 1 --2- 172.21.15.47:0/3456154238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f000b5c40 0x7f1f000b9330 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 rx=0x7f1ef8001800 tx=0x7f1ef0014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:34.245 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.230+0000 7f1eef7fe700 1 -- 172.21.15.47:0/3456154238 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ef0016070 con 0x7f1f000b5c40 2022-06-29T15:51:34.245 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.230+0000 7f1eef7fe700 1 -- 172.21.15.47:0/3456154238 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef0002f50 con 0x7f1f000b5c40 2022-06-29T15:51:34.245 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.230+0000 7f1eef7fe700 1 -- 172.21.15.47:0/3456154238 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ef0012460 con 0x7f1f000b5c40 2022-06-29T15:51:34.245 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.230+0000 7f1f073f2700 1 -- 172.21.15.47:0/3456154238 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f00155e10 con 0x7f1f000b5c40 2022-06-29T15:51:34.245 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.230+0000 7f1f073f2700 1 -- 172.21.15.47:0/3456154238 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f000b82d0 con 0x7f1f000b5c40 2022-06-29T15:51:34.246 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.230+0000 7f1f073f2700 1 -- 172.21.15.47:0/3456154238 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f00155fa0 con 0x7f1f000b5c40 2022-06-29T15:51:34.247 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.234+0000 7f1eef7fe700 1 -- 172.21.15.47:0/3456154238 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1ef000f030 con 0x7f1f000b5c40 2022-06-29T15:51:34.247 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.234+0000 7f1eef7fe700 1 --2- 172.21.15.47:0/3456154238 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1ee0053950 0x7f1ee0055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:34.248 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.234+0000 7f1eef7fe700 1 -- 172.21.15.47:0/3456154238 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f1ef006cb80 con 0x7f1f000b5c40 2022-06-29T15:51:34.248 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.234+0000 7f1f05bef700 1 --2- 172.21.15.47:0/3456154238 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1ee0053950 0x7f1ee0055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:34.248 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.234+0000 7f1f05bef700 1 --2- 172.21.15.47:0/3456154238 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1ee0053950 0x7f1ee0055e10 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7f1efc00b000 tx=0x7f1efc0091d0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:34.248 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.238+0000 7f1eef7fe700 1 -- 172.21.15.47:0/3456154238 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1ef0043940 con 0x7f1f000b5c40 2022-06-29T15:51:34.371 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.362+0000 7f1f073f2700 1 -- 172.21.15.47:0/3456154238 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f1f00067ae0 con 0x7f1f000b5c40 2022-06-29T15:51:34.372 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.366+0000 7f1eef7fe700 1 -- 172.21.15.47:0/3456154238 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v140) v1 ==== 110+0+10 (secure 0 0 0) 0x7f1ef0023020 con 0x7f1f000b5c40 2022-06-29T15:51:34.372 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2022-06-29T15:51:34.374 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.366+0000 7f1f073f2700 1 -- 172.21.15.47:0/3456154238 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1ee0053950 msgr2=0x7f1ee0055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:34.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.366+0000 7f1f073f2700 1 --2- 172.21.15.47:0/3456154238 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1ee0053950 0x7f1ee0055e10 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7f1efc00b000 tx=0x7f1efc0091d0).stop 2022-06-29T15:51:34.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.366+0000 7f1f073f2700 1 -- 172.21.15.47:0/3456154238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f000b5c40 msgr2=0x7f1f000b9330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:34.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.366+0000 7f1f073f2700 1 --2- 172.21.15.47:0/3456154238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f000b5c40 0x7f1f000b9330 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 rx=0x7f1ef8001800 tx=0x7f1ef0014000).stop 2022-06-29T15:51:34.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.366+0000 7f1f073f2700 1 -- 172.21.15.47:0/3456154238 shutdown_connections 2022-06-29T15:51:34.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.366+0000 7f1f073f2700 1 --2- 172.21.15.47:0/3456154238 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1ee0053950 0x7f1ee0055e10 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:34.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.366+0000 7f1f073f2700 1 --2- 172.21.15.47:0/3456154238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f00146870 0x7f1f000b8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:34.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.366+0000 7f1f073f2700 1 --2- 172.21.15.47:0/3456154238 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1f000b6420 0x7f1f000ae020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:34.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.366+0000 7f1f073f2700 1 --2- 172.21.15.47:0/3456154238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f000b5c40 0x7f1f000b9330 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:34.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.366+0000 7f1f073f2700 1 -- 172.21.15.47:0/3456154238 >> 172.21.15.47:0/3456154238 conn(0x7f1f0005c5b0 msgr2=0x7f1f00061430 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:34.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.366+0000 7f1f073f2700 1 -- 172.21.15.47:0/3456154238 shutdown_connections 2022-06-29T15:51:34.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.366+0000 7f1f073f2700 1 -- 172.21.15.47:0/3456154238 wait complete. 2022-06-29T15:51:34.385 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-29T15:51:34.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.482+0000 7f11aa7fd700 1 Processor -- start 2022-06-29T15:51:34.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.482+0000 7f11aa7fd700 1 -- start start 2022-06-29T15:51:34.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.482+0000 7f11aa7fd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f11a4060320 0x7f11a40b4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:34.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.482+0000 7f11aa7fd700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f11a40b7370 0x7f11a4144030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:34.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.482+0000 7f11aa7fd700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11a40b5050 con 0x7f11a40b6c60 2022-06-29T15:51:34.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.482+0000 7f11aa7fd700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11a4143700 con 0x7f11a4060320 2022-06-29T15:51:34.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.482+0000 7f11aa7fd700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11a4060700 con 0x7f11a40b7370 2022-06-29T15:51:34.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.482+0000 7f11a8d9a700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f11a40b6c60 0x7f11a4140640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39490/0 (socket says 172.21.15.47:39490) 2022-06-29T15:51:34.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.482+0000 7f11a8d9a700 1 -- 172.21.15.47:0/4092858497 learned_addr learned my addr 172.21.15.47:0/4092858497 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:34.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.486+0000 7f11a37fe700 1 --2- 172.21.15.47:0/4092858497 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f11a40b7370 0x7f11a4144030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:34.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.486+0000 7f11a3fff700 1 --2- 172.21.15.47:0/4092858497 >> v2:172.21.15.64:3300/0 conn(0x7f11a4060320 0x7f11a40b4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:34.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.486+0000 7f11a2ffd700 1 -- 172.21.15.47:0/4092858497 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 334029252 0 0) 0x7f119c001490 con 0x7f11a40b6c60 2022-06-29T15:51:34.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.486+0000 7f11a2ffd700 1 -- 172.21.15.47:0/4092858497 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1190003260 con 0x7f11a40b6c60 2022-06-29T15:51:34.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.486+0000 7f11a37fe700 1 -- 172.21.15.47:0/4092858497 >> v2:172.21.15.64:3300/0 conn(0x7f11a4060320 msgr2=0x7f11a40b4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:34.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.486+0000 7f11a37fe700 1 --2- 172.21.15.47:0/4092858497 >> v2:172.21.15.64:3300/0 conn(0x7f11a4060320 0x7f11a40b4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:34.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.486+0000 7f11a37fe700 1 -- 172.21.15.47:0/4092858497 >> v1:172.21.15.47:6789/0 conn(0x7f11a40b6c60 legacy=0x7f11a4140640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:34.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.486+0000 7f11a37fe700 1 -- 172.21.15.47:0/4092858497 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a41479f0 con 0x7f11a40b7370 2022-06-29T15:51:34.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.486+0000 7f11a37fe700 1 --2- 172.21.15.47:0/4092858497 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f11a40b7370 0x7f11a4144030 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f1198002f70 tx=0x7f1198006df0).ready entity=mon.2 client_cookie=21bad74c5c2ec5dd server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:34.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.486+0000 7f11a2ffd700 1 -- 172.21.15.47:0/4092858497 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f119800d4a0 con 0x7f11a40b7370 2022-06-29T15:51:34.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.486+0000 7f11a2ffd700 1 -- 172.21.15.47:0/4092858497 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1198000e90 con 0x7f11a40b7370 2022-06-29T15:51:34.497 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.486+0000 7f11a2ffd700 1 -- 172.21.15.47:0/4092858497 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f119800d7f0 con 0x7f11a40b7370 2022-06-29T15:51:34.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.486+0000 7f11aa7fd700 1 -- 172.21.15.47:0/4092858497 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f11a40b7370 msgr2=0x7f11a4144030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:34.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.486+0000 7f11aa7fd700 1 --2- 172.21.15.47:0/4092858497 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f11a40b7370 0x7f11a4144030 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f1198002f70 tx=0x7f1198006df0).stop 2022-06-29T15:51:34.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.486+0000 7f11aa7fd700 1 -- 172.21.15.47:0/4092858497 shutdown_connections 2022-06-29T15:51:34.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.486+0000 7f11aa7fd700 1 --2- 172.21.15.47:0/4092858497 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f11a40b7370 0x7f11a4144030 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:34.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.486+0000 7f11aa7fd700 1 --2- 172.21.15.47:0/4092858497 >> v2:172.21.15.64:3300/0 conn(0x7f11a4060320 0x7f11a40b4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:34.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.486+0000 7f11aa7fd700 1 -- 172.21.15.47:0/4092858497 >> 172.21.15.47:0/4092858497 conn(0x7f11a405d030 msgr2=0x7f11a4062cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:34.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.486+0000 7f11aa7fd700 1 -- 172.21.15.47:0/4092858497 shutdown_connections 2022-06-29T15:51:34.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.486+0000 7f11aa7fd700 1 -- 172.21.15.47:0/4092858497 wait complete. 2022-06-29T15:51:34.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.486+0000 7f11aa7fd700 1 Processor -- start 2022-06-29T15:51:34.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.486+0000 7f11aa7fd700 1 -- start start 2022-06-29T15:51:34.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.490+0000 7f11aa7fd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11a4060320 0x7f11a41535f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:34.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.490+0000 7f11aa7fd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11a40b6c60 0x7f11a40ae690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:34.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.490+0000 7f11aa7fd700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f11a40b7370 0x7f11a40adab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:34.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.490+0000 7f11aa7fd700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a4152a20 con 0x7f11a4060320 2022-06-29T15:51:34.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.490+0000 7f11aa7fd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a40adfa0 con 0x7f11a40b6c60 2022-06-29T15:51:34.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.490+0000 7f11aa7fd700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11a4152c10 con 0x7f11a40b7370 2022-06-29T15:51:34.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.490+0000 7f11a3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11a4060320 0x7f11a41535f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:34.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.490+0000 7f11a3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11a4060320 0x7f11a41535f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53938/0 (socket says 172.21.15.47:53938) 2022-06-29T15:51:34.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.490+0000 7f11a3fff700 1 -- 172.21.15.47:0/2222918702 learned_addr learned my addr 172.21.15.47:0/2222918702 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:34.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.490+0000 7f11a8d9a700 1 --2- 172.21.15.47:0/2222918702 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f11a40b7370 0x7f11a40adab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:34.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.490+0000 7f11a37fe700 1 --2- 172.21.15.47:0/2222918702 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11a40b6c60 0x7f11a40ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:34.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.490+0000 7f11a3fff700 1 -- 172.21.15.47:0/2222918702 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f11a40b7370 msgr2=0x7f11a40adab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:34.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.490+0000 7f11a3fff700 1 --2- 172.21.15.47:0/2222918702 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f11a40b7370 0x7f11a40adab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:34.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.490+0000 7f11a3fff700 1 -- 172.21.15.47:0/2222918702 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11a40b6c60 msgr2=0x7f11a40ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:34.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.490+0000 7f11a3fff700 1 --2- 172.21.15.47:0/2222918702 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11a40b6c60 0x7f11a40ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:34.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.490+0000 7f11a3fff700 1 -- 172.21.15.47:0/2222918702 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f119800b180 con 0x7f11a4060320 2022-06-29T15:51:34.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.490+0000 7f11a3fff700 1 --2- 172.21.15.47:0/2222918702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11a4060320 0x7f11a41535f0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 rx=0x7f119400c000 tx=0x7f1194006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:34.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.490+0000 7f11a17fa700 1 -- 172.21.15.47:0/2222918702 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1194010780 con 0x7f11a4060320 2022-06-29T15:51:34.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.490+0000 7f11a17fa700 1 -- 172.21.15.47:0/2222918702 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11940108e0 con 0x7f11a4060320 2022-06-29T15:51:34.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.490+0000 7f11a17fa700 1 -- 172.21.15.47:0/2222918702 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1194010c40 con 0x7f11a4060320 2022-06-29T15:51:34.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.490+0000 7f11aa7fd700 1 -- 172.21.15.47:0/2222918702 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11a4152eb0 con 0x7f11a4060320 2022-06-29T15:51:34.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.490+0000 7f11aa7fd700 1 -- 172.21.15.47:0/2222918702 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11a412e710 con 0x7f11a4060320 2022-06-29T15:51:34.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.490+0000 7f11a17fa700 1 -- 172.21.15.47:0/2222918702 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f119401c460 con 0x7f11a4060320 2022-06-29T15:51:34.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.490+0000 7f11aa7fd700 1 -- 172.21.15.47:0/2222918702 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11a40573d0 con 0x7f11a4060320 2022-06-29T15:51:34.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.494+0000 7f11a17fa700 1 --2- 172.21.15.47:0/2222918702 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1184053b00 0x7f1184055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:34.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.494+0000 7f11a17fa700 1 -- 172.21.15.47:0/2222918702 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f119406c4f0 con 0x7f11a4060320 2022-06-29T15:51:34.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.494+0000 7f11a37fe700 1 --2- 172.21.15.47:0/2222918702 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1184053b00 0x7f1184055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:34.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.494+0000 7f11a37fe700 1 --2- 172.21.15.47:0/2222918702 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1184053b00 0x7f1184055fc0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7f1198018000 tx=0x7f1198014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:34.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.498+0000 7f11a17fa700 1 -- 172.21.15.47:0/2222918702 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1194043250 con 0x7f11a4060320 2022-06-29T15:51:34.634 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.626+0000 7f11aa7fd700 1 -- 172.21.15.47:0/2222918702 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f11a40b6330 con 0x7f11a4060320 2022-06-29T15:51:34.635 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.626+0000 7f11a17fa700 1 -- 172.21.15.47:0/2222918702 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v140) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1194046870 con 0x7f11a4060320 2022-06-29T15:51:34.635 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:51:34.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.630+0000 7f11aa7fd700 1 -- 172.21.15.47:0/2222918702 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1184053b00 msgr2=0x7f1184055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:34.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.630+0000 7f11aa7fd700 1 --2- 172.21.15.47:0/2222918702 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1184053b00 0x7f1184055fc0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7f1198018000 tx=0x7f1198014000).stop 2022-06-29T15:51:34.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.630+0000 7f11aa7fd700 1 -- 172.21.15.47:0/2222918702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11a4060320 msgr2=0x7f11a41535f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:34.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.630+0000 7f11aa7fd700 1 --2- 172.21.15.47:0/2222918702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11a4060320 0x7f11a41535f0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 rx=0x7f119400c000 tx=0x7f1194006df0).stop 2022-06-29T15:51:34.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.630+0000 7f11aa7fd700 1 -- 172.21.15.47:0/2222918702 shutdown_connections 2022-06-29T15:51:34.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.630+0000 7f11aa7fd700 1 --2- 172.21.15.47:0/2222918702 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f11a40b7370 0x7f11a40adab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:34.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.630+0000 7f11aa7fd700 1 --2- 172.21.15.47:0/2222918702 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1184053b00 0x7f1184055fc0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:34.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.630+0000 7f11aa7fd700 1 --2- 172.21.15.47:0/2222918702 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11a40b6c60 0x7f11a40ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:34.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.630+0000 7f11aa7fd700 1 --2- 172.21.15.47:0/2222918702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11a4060320 0x7f11a41535f0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:34.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.630+0000 7f11aa7fd700 1 -- 172.21.15.47:0/2222918702 >> 172.21.15.47:0/2222918702 conn(0x7f11a405d030 msgr2=0x7f11a412b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:34.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.630+0000 7f11aa7fd700 1 -- 172.21.15.47:0/2222918702 shutdown_connections 2022-06-29T15:51:34.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.630+0000 7f11aa7fd700 1 -- 172.21.15.47:0/2222918702 wait complete. 2022-06-29T15:51:34.653 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-29T15:51:34.766 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.758+0000 7ff518bdf700 1 Processor -- start 2022-06-29T15:51:34.766 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.758+0000 7ff518bdf700 1 -- start start 2022-06-29T15:51:34.769 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.758+0000 7ff518bdf700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff5140a8e20 0x7ff5140afe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:34.769 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.758+0000 7ff518bdf700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff51413e2f0 0x7ff51413e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:34.769 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.758+0000 7ff518bdf700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5140b18a0 con 0x7ff5140aca70 2022-06-29T15:51:34.769 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.758+0000 7ff518bdf700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff51413ec50 con 0x7ff51413e2f0 2022-06-29T15:51:34.770 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.758+0000 7ff518bdf700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5140ab910 con 0x7ff5140a8e20 2022-06-29T15:51:34.770 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.758+0000 7ff513fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff51413e2f0 0x7ff51413e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:34.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.758+0000 7ff5137fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff5140aca70 0x7ff5141417d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39500/0 (socket says 172.21.15.47:39500) 2022-06-29T15:51:34.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.758+0000 7ff5137fe700 1 -- 172.21.15.47:0/1368263963 learned_addr learned my addr 172.21.15.47:0/1368263963 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:34.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.758+0000 7ff50bfff700 1 --2- 172.21.15.47:0/1368263963 >> v2:172.21.15.64:3300/0 conn(0x7ff5140a8e20 0x7ff5140afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:34.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.762+0000 7ff513fff700 1 -- 172.21.15.47:0/1368263963 >> v2:172.21.15.64:3300/0 conn(0x7ff5140a8e20 msgr2=0x7ff5140afe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:34.772 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.762+0000 7ff513fff700 1 --2- 172.21.15.47:0/1368263963 >> v2:172.21.15.64:3300/0 conn(0x7ff5140a8e20 0x7ff5140afe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:34.772 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.762+0000 7ff513fff700 1 -- 172.21.15.47:0/1368263963 >> v1:172.21.15.47:6789/0 conn(0x7ff5140aca70 legacy=0x7ff5141417d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:34.772 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.762+0000 7ff513fff700 1 -- 172.21.15.47:0/1368263963 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff514141ee0 con 0x7ff51413e2f0 2022-06-29T15:51:34.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.762+0000 7ff513fff700 1 --2- 172.21.15.47:0/1368263963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff51413e2f0 0x7ff51413e760 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 rx=0x7ff50c0032f0 tx=0x7ff50c0091d0).ready entity=mon.0 client_cookie=34b9052deb22c2ad server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:34.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.762+0000 7ff512ffd700 1 -- 172.21.15.47:0/1368263963 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff50c00b680 con 0x7ff51413e2f0 2022-06-29T15:51:34.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.762+0000 7ff512ffd700 1 -- 172.21.15.47:0/1368263963 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff50c00b7e0 con 0x7ff51413e2f0 2022-06-29T15:51:34.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.762+0000 7ff512ffd700 1 -- 172.21.15.47:0/1368263963 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff50c00b950 con 0x7ff51413e2f0 2022-06-29T15:51:34.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.762+0000 7ff518bdf700 1 -- 172.21.15.47:0/1368263963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff51413e2f0 msgr2=0x7ff51413e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:34.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.762+0000 7ff518bdf700 1 --2- 172.21.15.47:0/1368263963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff51413e2f0 0x7ff51413e760 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 rx=0x7ff50c0032f0 tx=0x7ff50c0091d0).stop 2022-06-29T15:51:34.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.762+0000 7ff518bdf700 1 -- 172.21.15.47:0/1368263963 shutdown_connections 2022-06-29T15:51:34.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.762+0000 7ff518bdf700 1 --2- 172.21.15.47:0/1368263963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff51413e2f0 0x7ff51413e760 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:34.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.762+0000 7ff518bdf700 1 --2- 172.21.15.47:0/1368263963 >> v2:172.21.15.64:3300/0 conn(0x7ff5140a8e20 0x7ff5140afe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:34.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.762+0000 7ff518bdf700 1 -- 172.21.15.47:0/1368263963 >> 172.21.15.47:0/1368263963 conn(0x7ff514062580 msgr2=0x7ff514062990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:34.776 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.762+0000 7ff518bdf700 1 -- 172.21.15.47:0/1368263963 shutdown_connections 2022-06-29T15:51:34.776 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.762+0000 7ff518bdf700 1 -- 172.21.15.47:0/1368263963 wait complete. 2022-06-29T15:51:34.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.762+0000 7ff518bdf700 1 Processor -- start 2022-06-29T15:51:34.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.762+0000 7ff518bdf700 1 -- start start 2022-06-29T15:51:34.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.766+0000 7ff518bdf700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5140a8e20 0x7ff5140a96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:34.781 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.766+0000 7ff518bdf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5140aca70 0x7ff514154220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:34.781 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.766+0000 7ff518bdf700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff51413e2f0 0x7ff514137510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:34.781 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.766+0000 7ff518bdf700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff514136dd0 con 0x7ff5140a8e20 2022-06-29T15:51:34.782 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.766+0000 7ff518bdf700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5141367e0 con 0x7ff5140aca70 2022-06-29T15:51:34.782 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.766+0000 7ff518bdf700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff514153b50 con 0x7ff51413e2f0 2022-06-29T15:51:34.782 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.766+0000 7ff5137fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5140a8e20 0x7ff5140a96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:34.782 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.766+0000 7ff50bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5140aca70 0x7ff514154220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:34.783 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.766+0000 7ff50bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5140aca70 0x7ff514154220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:59656/0 (socket says 172.21.15.47:59656) 2022-06-29T15:51:34.783 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.766+0000 7ff50bfff700 1 -- 172.21.15.47:0/623767013 learned_addr learned my addr 172.21.15.47:0/623767013 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:34.783 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.766+0000 7ff513fff700 1 --2- 172.21.15.47:0/623767013 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff51413e2f0 0x7ff514137510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:34.784 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.766+0000 7ff5137fe700 1 -- 172.21.15.47:0/623767013 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff51413e2f0 msgr2=0x7ff514137510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:34.784 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.766+0000 7ff5137fe700 1 --2- 172.21.15.47:0/623767013 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff51413e2f0 0x7ff514137510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:34.784 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.766+0000 7ff5137fe700 1 -- 172.21.15.47:0/623767013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5140aca70 msgr2=0x7ff514154220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:34.785 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.766+0000 7ff5137fe700 1 --2- 172.21.15.47:0/623767013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5140aca70 0x7ff514154220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:34.785 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.766+0000 7ff5137fe700 1 -- 172.21.15.47:0/623767013 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff50c009470 con 0x7ff5140a8e20 2022-06-29T15:51:34.785 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.766+0000 7ff50bfff700 1 --2- 172.21.15.47:0/623767013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5140aca70 0x7ff514154220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:51:34.785 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.766+0000 7ff513fff700 1 --2- 172.21.15.47:0/623767013 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff51413e2f0 0x7ff514137510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:51:34.786 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.766+0000 7ff5137fe700 1 --2- 172.21.15.47:0/623767013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5140a8e20 0x7ff5140a96e0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 rx=0x7ff5141450d0 tx=0x7ff504007b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:34.786 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.766+0000 7ff5117fa700 1 -- 172.21.15.47:0/623767013 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff50400ce90 con 0x7ff5140a8e20 2022-06-29T15:51:34.786 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.766+0000 7ff5117fa700 1 -- 172.21.15.47:0/623767013 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5040070d0 con 0x7ff5140a8e20 2022-06-29T15:51:34.787 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.766+0000 7ff518bdf700 1 -- 172.21.15.47:0/623767013 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5141425b0 con 0x7ff5140a8e20 2022-06-29T15:51:34.787 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.766+0000 7ff5117fa700 1 -- 172.21.15.47:0/623767013 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff504007430 con 0x7ff5140a8e20 2022-06-29T15:51:34.787 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.766+0000 7ff518bdf700 1 -- 172.21.15.47:0/623767013 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff51413bfa0 con 0x7ff5140a8e20 2022-06-29T15:51:34.787 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.770+0000 7ff518bdf700 1 -- 172.21.15.47:0/623767013 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5140573d0 con 0x7ff5140a8e20 2022-06-29T15:51:34.788 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.770+0000 7ff5117fa700 1 -- 172.21.15.47:0/623767013 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff504010070 con 0x7ff5140a8e20 2022-06-29T15:51:34.788 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.770+0000 7ff5117fa700 1 --2- 172.21.15.47:0/623767013 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff4f80539e0 0x7ff4f8055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:34.788 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.770+0000 7ff5117fa700 1 -- 172.21.15.47:0/623767013 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff50406b5b0 con 0x7ff5140a8e20 2022-06-29T15:51:34.789 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.770+0000 7ff50bfff700 1 --2- 172.21.15.47:0/623767013 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff4f80539e0 0x7ff4f8055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:34.789 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.770+0000 7ff50bfff700 1 --2- 172.21.15.47:0/623767013 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff4f80539e0 0x7ff4f8055ea0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7ff4fc00b000 tx=0x7ff4fc006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:34.790 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.774+0000 7ff5117fa700 1 -- 172.21.15.47:0/623767013 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff5040421e0 con 0x7ff5140a8e20 2022-06-29T15:51:34.909 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.902+0000 7ff518bdf700 1 -- 172.21.15.47:0/623767013 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff5141423d0 con 0x7ff5140a8e20 2022-06-29T15:51:34.910 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.902+0000 7ff5117fa700 1 -- 172.21.15.47:0/623767013 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v140) v1 ==== 104+0+10 (secure 0 0 0) 0x7ff504018020 con 0x7ff5140a8e20 2022-06-29T15:51:34.911 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:51:34.912 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.906+0000 7ff518bdf700 1 -- 172.21.15.47:0/623767013 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff4f80539e0 msgr2=0x7ff4f8055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:34.913 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.906+0000 7ff518bdf700 1 --2- 172.21.15.47:0/623767013 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff4f80539e0 0x7ff4f8055ea0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7ff4fc00b000 tx=0x7ff4fc006df0).stop 2022-06-29T15:51:34.913 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.906+0000 7ff518bdf700 1 -- 172.21.15.47:0/623767013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5140a8e20 msgr2=0x7ff5140a96e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:34.913 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.906+0000 7ff518bdf700 1 --2- 172.21.15.47:0/623767013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5140a8e20 0x7ff5140a96e0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 rx=0x7ff5141450d0 tx=0x7ff504007b50).stop 2022-06-29T15:51:34.913 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.906+0000 7ff518bdf700 1 -- 172.21.15.47:0/623767013 shutdown_connections 2022-06-29T15:51:34.914 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.906+0000 7ff518bdf700 1 --2- 172.21.15.47:0/623767013 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff51413e2f0 0x7ff514137510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:34.914 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.906+0000 7ff518bdf700 1 --2- 172.21.15.47:0/623767013 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff4f80539e0 0x7ff4f8055ea0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:34.914 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.906+0000 7ff518bdf700 1 --2- 172.21.15.47:0/623767013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5140aca70 0x7ff514154220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:34.914 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.906+0000 7ff518bdf700 1 --2- 172.21.15.47:0/623767013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5140a8e20 0x7ff5140a96e0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:34.915 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.906+0000 7ff518bdf700 1 -- 172.21.15.47:0/623767013 >> 172.21.15.47:0/623767013 conn(0x7ff514062580 msgr2=0x7ff514140800 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:34.915 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.906+0000 7ff518bdf700 1 -- 172.21.15.47:0/623767013 shutdown_connections 2022-06-29T15:51:34.915 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:34.906+0000 7ff518bdf700 1 -- 172.21.15.47:0/623767013 wait complete. 2022-06-29T15:51:34.924 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-29T15:51:35.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.026+0000 7f1fbdd48700 1 Processor -- start 2022-06-29T15:51:35.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.030+0000 7f1fbdd48700 1 -- start start 2022-06-29T15:51:35.038 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.030+0000 7f1fbdd48700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1fb8062910 0x7f1fb80b62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:35.038 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.030+0000 7f1fbdd48700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1fb8140700 0x7f1fb8140b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:35.038 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.030+0000 7f1fbdd48700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fb80b5240 con 0x7f1fb80b5670 2022-06-29T15:51:35.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.030+0000 7f1fbdd48700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fb8062cf0 con 0x7f1fb8062910 2022-06-29T15:51:35.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.030+0000 7f1fbdd48700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb8141010 con 0x7f1fb8140700 2022-06-29T15:51:35.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.030+0000 7f1fb7fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1fb8140700 0x7f1fb8140b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:35.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.030+0000 7f1fb7fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1fb8140700 0x7f1fb8140b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:51066/0 (socket says 172.21.15.47:51066) 2022-06-29T15:51:35.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.030+0000 7f1fb7fff700 1 -- 172.21.15.47:0/4171676069 learned_addr learned my addr 172.21.15.47:0/4171676069 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:35.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.030+0000 7f1fbcd46700 1 --2- 172.21.15.47:0/4171676069 >> v2:172.21.15.64:3300/0 conn(0x7f1fb8062910 0x7f1fb80b62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:35.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.030+0000 7f1fb7fff700 1 -- 172.21.15.47:0/4171676069 >> v2:172.21.15.64:3300/0 conn(0x7f1fb8062910 msgr2=0x7f1fb80b62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:35.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.030+0000 7f1fb7fff700 1 --2- 172.21.15.47:0/4171676069 >> v2:172.21.15.64:3300/0 conn(0x7f1fb8062910 0x7f1fb80b62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:35.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.030+0000 7f1fb7fff700 1 -- 172.21.15.47:0/4171676069 >> v1:172.21.15.64:6789/0 conn(0x7f1fb80b5670 legacy=0x7f1fb80b5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:35.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.030+0000 7f1fb7fff700 1 -- 172.21.15.47:0/4171676069 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fb80b2390 con 0x7f1fb8140700 2022-06-29T15:51:35.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.030+0000 7f1fb7fff700 1 --2- 172.21.15.47:0/4171676069 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1fb8140700 0x7f1fb8140b20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f1fac002f70 tx=0x7f1fac006eb0).ready entity=mon.2 client_cookie=c24cc7e7f4809735 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:35.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.030+0000 7f1fb77fe700 1 -- 172.21.15.47:0/4171676069 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1fac00d570 con 0x7f1fb8140700 2022-06-29T15:51:35.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.030+0000 7f1fb77fe700 1 -- 172.21.15.47:0/4171676069 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fac000e90 con 0x7f1fb8140700 2022-06-29T15:51:35.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.030+0000 7f1fb77fe700 1 -- 172.21.15.47:0/4171676069 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1fac00d900 con 0x7f1fb8140700 2022-06-29T15:51:35.045 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.030+0000 7f1fbdd48700 1 -- 172.21.15.47:0/4171676069 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1fb8140700 msgr2=0x7f1fb8140b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:35.045 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.030+0000 7f1fbdd48700 1 --2- 172.21.15.47:0/4171676069 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1fb8140700 0x7f1fb8140b20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f1fac002f70 tx=0x7f1fac006eb0).stop 2022-06-29T15:51:35.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.030+0000 7f1fbdd48700 1 -- 172.21.15.47:0/4171676069 shutdown_connections 2022-06-29T15:51:35.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.030+0000 7f1fbdd48700 1 --2- 172.21.15.47:0/4171676069 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1fb8140700 0x7f1fb8140b20 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:35.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.030+0000 7f1fbdd48700 1 --2- 172.21.15.47:0/4171676069 >> v2:172.21.15.64:3300/0 conn(0x7f1fb8062910 0x7f1fb80b62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:35.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.034+0000 7f1fbdd48700 1 -- 172.21.15.47:0/4171676069 >> 172.21.15.47:0/4171676069 conn(0x7f1fb805fce0 msgr2=0x7f1fb80600c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:35.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.034+0000 7f1fbdd48700 1 -- 172.21.15.47:0/4171676069 shutdown_connections 2022-06-29T15:51:35.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.034+0000 7f1fbdd48700 1 -- 172.21.15.47:0/4171676069 wait complete. 2022-06-29T15:51:35.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.034+0000 7f1fbdd48700 1 Processor -- start 2022-06-29T15:51:35.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.034+0000 7f1fbdd48700 1 -- start start 2022-06-29T15:51:35.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.034+0000 7f1fbdd48700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1fb8062910 0x7f1fb8141540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:35.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.034+0000 7f1fbdd48700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1fb80b5670 0x7f1fb80afe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:35.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.034+0000 7f1fbdd48700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fb8140700 0x7f1fb813db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:35.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.034+0000 7f1fbdd48700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb80ace00 con 0x7f1fb8062910 2022-06-29T15:51:35.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.034+0000 7f1fbdd48700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb80ab9a0 con 0x7f1fb8140700 2022-06-29T15:51:35.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.034+0000 7f1fbdd48700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb80ac960 con 0x7f1fb80b5670 2022-06-29T15:51:35.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.034+0000 7f1fbcd46700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1fb8062910 0x7f1fb8141540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:35.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.034+0000 7f1fbcd46700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1fb8062910 0x7f1fb8141540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53966/0 (socket says 172.21.15.47:53966) 2022-06-29T15:51:35.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.034+0000 7f1fbcd46700 1 -- 172.21.15.47:0/2454124566 learned_addr learned my addr 172.21.15.47:0/2454124566 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:35.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.038+0000 7f1fb7fff700 1 --2- 172.21.15.47:0/2454124566 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1fb80b5670 0x7f1fb80afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:35.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.038+0000 7f1fbd547700 1 --2- 172.21.15.47:0/2454124566 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fb8140700 0x7f1fb813db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:35.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.038+0000 7f1fb7fff700 1 -- 172.21.15.47:0/2454124566 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fb8140700 msgr2=0x7f1fb813db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:35.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.038+0000 7f1fb7fff700 1 --2- 172.21.15.47:0/2454124566 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fb8140700 0x7f1fb813db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:35.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.038+0000 7f1fb7fff700 1 -- 172.21.15.47:0/2454124566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1fb8062910 msgr2=0x7f1fb8141540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:35.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.038+0000 7f1fb7fff700 1 --2- 172.21.15.47:0/2454124566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1fb8062910 0x7f1fb8141540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:35.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.038+0000 7f1fb7fff700 1 -- 172.21.15.47:0/2454124566 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fac00b1e0 con 0x7f1fb80b5670 2022-06-29T15:51:35.052 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.038+0000 7f1fb7fff700 1 --2- 172.21.15.47:0/2454124566 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1fb80b5670 0x7f1fb80afe30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f1fac002f70 tx=0x7f1fac015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:35.052 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.038+0000 7f1fb5ffb700 1 -- 172.21.15.47:0/2454124566 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1fac006b40 con 0x7f1fb80b5670 2022-06-29T15:51:35.052 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.038+0000 7f1fbdd48700 1 -- 172.21.15.47:0/2454124566 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fb81500f0 con 0x7f1fb80b5670 2022-06-29T15:51:35.052 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.038+0000 7f1fb5ffb700 1 -- 172.21.15.47:0/2454124566 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fac00d8f0 con 0x7f1fb80b5670 2022-06-29T15:51:35.053 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.038+0000 7f1fb5ffb700 1 -- 172.21.15.47:0/2454124566 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1fac011d70 con 0x7f1fb80b5670 2022-06-29T15:51:35.053 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.038+0000 7f1fbdd48700 1 -- 172.21.15.47:0/2454124566 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fb80ab280 con 0x7f1fb80b5670 2022-06-29T15:51:35.054 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.038+0000 7f1fbdd48700 1 -- 172.21.15.47:0/2454124566 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fb80573d0 con 0x7f1fb80b5670 2022-06-29T15:51:35.055 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.038+0000 7f1fb5ffb700 1 -- 172.21.15.47:0/2454124566 <== mon.2 v2:172.21.15.64:3301/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1fac01a040 con 0x7f1fb80b5670 2022-06-29T15:51:35.055 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.042+0000 7f1fb5ffb700 1 --2- 172.21.15.47:0/2454124566 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1f98053980 0x7f1f98055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:35.055 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.042+0000 7f1fb5ffb700 1 -- 172.21.15.47:0/2454124566 <== mon.2 v2:172.21.15.64:3301/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f1fac06cee0 con 0x7f1fb80b5670 2022-06-29T15:51:35.055 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.042+0000 7f1fbcd46700 1 --2- 172.21.15.47:0/2454124566 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1f98053980 0x7f1f98055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:35.055 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.042+0000 7f1fbcd46700 1 --2- 172.21.15.47:0/2454124566 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1f98053980 0x7f1f98055e40 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7f1fa800b000 tx=0x7f1fa8006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:35.057 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.050+0000 7f1fb5ffb700 1 -- 172.21.15.47:0/2454124566 <== mon.2 v2:172.21.15.64:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1fac043b10 con 0x7f1fb80b5670 2022-06-29T15:51:35.183 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.174+0000 7f1fbdd48700 1 -- 172.21.15.47:0/2454124566 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1fb8067ae0 con 0x7f1fb80b5670 2022-06-29T15:51:35.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.178+0000 7f1fb5ffb700 1 -- 172.21.15.47:0/2454124566 <== mon.2 v2:172.21.15.64:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v140) v1 ==== 100+0+10 (secure 0 0 0) 0x7f1fac0230a0 con 0x7f1fb80b5670 2022-06-29T15:51:35.184 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:51:35.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.178+0000 7f1fbdd48700 1 -- 172.21.15.47:0/2454124566 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1f98053980 msgr2=0x7f1f98055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:35.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.178+0000 7f1fbdd48700 1 --2- 172.21.15.47:0/2454124566 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1f98053980 0x7f1f98055e40 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7f1fa800b000 tx=0x7f1fa8006df0).stop 2022-06-29T15:51:35.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.178+0000 7f1fbdd48700 1 -- 172.21.15.47:0/2454124566 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1fb80b5670 msgr2=0x7f1fb80afe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:35.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.178+0000 7f1fbdd48700 1 --2- 172.21.15.47:0/2454124566 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1fb80b5670 0x7f1fb80afe30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f1fac002f70 tx=0x7f1fac015000).stop 2022-06-29T15:51:35.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.178+0000 7f1fbdd48700 1 -- 172.21.15.47:0/2454124566 shutdown_connections 2022-06-29T15:51:35.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.178+0000 7f1fbdd48700 1 --2- 172.21.15.47:0/2454124566 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1f98053980 0x7f1f98055e40 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:35.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.178+0000 7f1fbdd48700 1 --2- 172.21.15.47:0/2454124566 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fb8140700 0x7f1fb813db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:35.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.178+0000 7f1fbdd48700 1 --2- 172.21.15.47:0/2454124566 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1fb80b5670 0x7f1fb80afe30 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:35.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.178+0000 7f1fbdd48700 1 --2- 172.21.15.47:0/2454124566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1fb8062910 0x7f1fb8141540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:35.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.178+0000 7f1fbdd48700 1 -- 172.21.15.47:0/2454124566 >> 172.21.15.47:0/2454124566 conn(0x7f1fb805fce0 msgr2=0x7f1fb8141de0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:35.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.178+0000 7f1fbdd48700 1 -- 172.21.15.47:0/2454124566 shutdown_connections 2022-06-29T15:51:35.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.178+0000 7f1fbdd48700 1 -- 172.21.15.47:0/2454124566 wait complete. 2022-06-29T15:51:35.197 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-29T15:51:35.198 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:51:35.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.294+0000 7f7a2359e700 1 Processor -- start 2022-06-29T15:51:35.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.294+0000 7f7a2359e700 1 -- start start 2022-06-29T15:51:35.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.298+0000 7f7a2359e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7a240b7000 0x7f7a240b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:35.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.298+0000 7f7a2359e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7a24063000 0x7f7a240b29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:35.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.298+0000 7f7a2359e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a240b1b90 con 0x7f7a240b0f50 2022-06-29T15:51:35.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.298+0000 7f7a2359e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a240b2ee0 con 0x7f7a240b7000 2022-06-29T15:51:35.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.298+0000 7f7a2359e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a240b73e0 con 0x7f7a24063000 2022-06-29T15:51:35.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.298+0000 7f7a2259c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7a240b7000 0x7f7a240b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:35.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.298+0000 7f7a21d9b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7a240b0f50 0x7f7a2405fed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39530/0 (socket says 172.21.15.47:39530) 2022-06-29T15:51:35.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.298+0000 7f7a2259c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7a240b7000 0x7f7a240b2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53974/0 (socket says 172.21.15.47:53974) 2022-06-29T15:51:35.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.298+0000 7f7a21d9b700 1 -- 172.21.15.47:0/3158909019 learned_addr learned my addr 172.21.15.47:0/3158909019 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:35.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.298+0000 7f7a22d9d700 1 --2- 172.21.15.47:0/3158909019 >> v2:172.21.15.64:3300/0 conn(0x7f7a24063000 0x7f7a240b29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:35.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.298+0000 7f7a2259c700 1 -- 172.21.15.47:0/3158909019 >> v2:172.21.15.64:3300/0 conn(0x7f7a24063000 msgr2=0x7f7a240b29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:35.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.298+0000 7f7a2159a700 1 -- 172.21.15.47:0/3158909019 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2594528531 0 0) 0x7f7a240b1b90 con 0x7f7a240b0f50 2022-06-29T15:51:35.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.298+0000 7f7a2259c700 1 --2- 172.21.15.47:0/3158909019 >> v2:172.21.15.64:3300/0 conn(0x7f7a24063000 0x7f7a240b29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:35.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.298+0000 7f7a2259c700 1 -- 172.21.15.47:0/3158909019 >> v1:172.21.15.47:6789/0 conn(0x7f7a240b0f50 legacy=0x7f7a2405fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:35.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.298+0000 7f7a2259c700 1 -- 172.21.15.47:0/3158909019 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a24144390 con 0x7f7a240b7000 2022-06-29T15:51:35.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.298+0000 7f7a22d9d700 1 --2- 172.21.15.47:0/3158909019 >> v2:172.21.15.64:3300/0 conn(0x7f7a24063000 0x7f7a240b29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:51:35.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.298+0000 7f7a2259c700 1 --2- 172.21.15.47:0/3158909019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a240b7000 0x7f7a240b2500 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 rx=0x7f7a0c002f70 tx=0x7f7a0c006df0).ready entity=mon.0 client_cookie=a114c14c2eec767 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:35.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.298+0000 7f7a2159a700 1 -- 172.21.15.47:0/3158909019 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a0c00d5c0 con 0x7f7a240b7000 2022-06-29T15:51:35.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.298+0000 7f7a2159a700 1 -- 172.21.15.47:0/3158909019 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a0c000e90 con 0x7f7a240b7000 2022-06-29T15:51:35.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.298+0000 7f7a2159a700 1 -- 172.21.15.47:0/3158909019 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a0c00d910 con 0x7f7a240b7000 2022-06-29T15:51:35.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.298+0000 7f7a2359e700 1 -- 172.21.15.47:0/3158909019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a240b7000 msgr2=0x7f7a240b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:35.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.298+0000 7f7a2359e700 1 --2- 172.21.15.47:0/3158909019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a240b7000 0x7f7a240b2500 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 rx=0x7f7a0c002f70 tx=0x7f7a0c006df0).stop 2022-06-29T15:51:35.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.298+0000 7f7a2359e700 1 -- 172.21.15.47:0/3158909019 shutdown_connections 2022-06-29T15:51:35.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.298+0000 7f7a2359e700 1 --2- 172.21.15.47:0/3158909019 >> v2:172.21.15.64:3300/0 conn(0x7f7a24063000 0x7f7a240b29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:35.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.298+0000 7f7a2359e700 1 --2- 172.21.15.47:0/3158909019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a240b7000 0x7f7a240b2500 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:35.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.298+0000 7f7a2359e700 1 -- 172.21.15.47:0/3158909019 >> 172.21.15.47:0/3158909019 conn(0x7f7a24061f00 msgr2=0x7f7a240b65b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:35.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.298+0000 7f7a2359e700 1 -- 172.21.15.47:0/3158909019 shutdown_connections 2022-06-29T15:51:35.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.298+0000 7f7a2359e700 1 -- 172.21.15.47:0/3158909019 wait complete. 2022-06-29T15:51:35.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.298+0000 7f7a2359e700 1 Processor -- start 2022-06-29T15:51:35.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.298+0000 7f7a2359e700 1 -- start start 2022-06-29T15:51:35.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.302+0000 7f7a2359e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7a24063000 0x7f7a2414f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:35.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.302+0000 7f7a2359e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a240b0f50 0x7f7a2413e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:35.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.302+0000 7f7a2359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a240b7000 0x7f7a24152de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:35.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.302+0000 7f7a2359e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a24140780 con 0x7f7a240b0f50 2022-06-29T15:51:35.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.302+0000 7f7a2359e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a241526a0 con 0x7f7a240b7000 2022-06-29T15:51:35.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.302+0000 7f7a2359e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a24152940 con 0x7f7a24063000 2022-06-29T15:51:35.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.302+0000 7f7a21d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a240b0f50 0x7f7a2413e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:35.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.302+0000 7f7a21d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a240b0f50 0x7f7a2413e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53980/0 (socket says 172.21.15.47:53980) 2022-06-29T15:51:35.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.302+0000 7f7a21d9b700 1 -- 172.21.15.47:0/1759809974 learned_addr learned my addr 172.21.15.47:0/1759809974 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:35.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.302+0000 7f7a22d9d700 1 --2- 172.21.15.47:0/1759809974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a240b7000 0x7f7a24152de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:35.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.302+0000 7f7a2259c700 1 --2- 172.21.15.47:0/1759809974 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7a24063000 0x7f7a2414f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:35.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.302+0000 7f7a21d9b700 1 -- 172.21.15.47:0/1759809974 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7a24063000 msgr2=0x7f7a2414f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:35.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.302+0000 7f7a21d9b700 1 --2- 172.21.15.47:0/1759809974 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7a24063000 0x7f7a2414f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:35.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.302+0000 7f7a21d9b700 1 -- 172.21.15.47:0/1759809974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a240b7000 msgr2=0x7f7a24152de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:35.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.302+0000 7f7a21d9b700 1 --2- 172.21.15.47:0/1759809974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a240b7000 0x7f7a24152de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:35.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.302+0000 7f7a21d9b700 1 -- 172.21.15.47:0/1759809974 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a0c00b180 con 0x7f7a240b0f50 2022-06-29T15:51:35.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.302+0000 7f7a2259c700 1 --2- 172.21.15.47:0/1759809974 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7a24063000 0x7f7a2414f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:51:35.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.302+0000 7f7a22d9d700 1 --2- 172.21.15.47:0/1759809974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a240b7000 0x7f7a24152de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:51:35.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.302+0000 7f7a21d9b700 1 --2- 172.21.15.47:0/1759809974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a240b0f50 0x7f7a2413e2e0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 rx=0x7f7a1c001250 tx=0x7f7a1c007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:35.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.302+0000 7f7a1b7fe700 1 -- 172.21.15.47:0/1759809974 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a1c005c70 con 0x7f7a240b0f50 2022-06-29T15:51:35.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.302+0000 7f7a1b7fe700 1 -- 172.21.15.47:0/1759809974 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a1c005dd0 con 0x7f7a240b0f50 2022-06-29T15:51:35.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.302+0000 7f7a2359e700 1 -- 172.21.15.47:0/1759809974 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a2413e7d0 con 0x7f7a240b0f50 2022-06-29T15:51:35.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.302+0000 7f7a1b7fe700 1 -- 172.21.15.47:0/1759809974 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a1c007300 con 0x7f7a240b0f50 2022-06-29T15:51:35.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.302+0000 7f7a2359e700 1 -- 172.21.15.47:0/1759809974 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a2413f2b0 con 0x7f7a240b0f50 2022-06-29T15:51:35.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.302+0000 7f7a2359e700 1 -- 172.21.15.47:0/1759809974 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a24152b00 con 0x7f7a240b0f50 2022-06-29T15:51:35.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.302+0000 7f7a1b7fe700 1 -- 172.21.15.47:0/1759809974 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7a1c010030 con 0x7f7a240b0f50 2022-06-29T15:51:35.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.306+0000 7f7a1b7fe700 1 --2- 172.21.15.47:0/1759809974 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f79fc053980 0x7f79fc055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:35.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.306+0000 7f7a1b7fe700 1 -- 172.21.15.47:0/1759809974 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f7a1c06c840 con 0x7f7a240b0f50 2022-06-29T15:51:35.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.306+0000 7f7a2259c700 1 --2- 172.21.15.47:0/1759809974 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f79fc053980 0x7f79fc055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:35.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.306+0000 7f7a2259c700 1 --2- 172.21.15.47:0/1759809974 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f79fc053980 0x7f79fc055e40 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7f7a0c015000 tx=0x7f7a0c014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:35.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.314+0000 7f7a1b7fe700 1 -- 172.21.15.47:0/1759809974 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7a1c0434f0 con 0x7f7a240b0f50 2022-06-29T15:51:35.484 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.478+0000 7f7a2359e700 1 -- 172.21.15.47:0/1759809974 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7a240573d0 con 0x7f7a240b0f50 2022-06-29T15:51:35.485 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.478+0000 7f7a1b7fe700 1 -- 172.21.15.47:0/1759809974 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+2492 (secure 0 0 0) 0x7f7a1c046b10 con 0x7f7a240b0f50 2022-06-29T15:51:35.485 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:35.486 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":69,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7957,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.64:6833/298405638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":298405638},{"type":"v1","addr":"172.21.15.64:6833","nonce":298405638}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":66},{"gid":8037,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/191070564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":191070564},{"type":"v1","addr":"172.21.15.47:6835","nonce":191070564}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:51:23.543892+0000","modified":"2022-06-29T15:51:25.309120+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":56894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_56894":{"gid":56894,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6837/1370488030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1370488030},{"type":"v1","addr":"172.21.15.47:6837","nonce":1370488030}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2022-06-29T15:51:35.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.478+0000 7f7a2359e700 1 -- 172.21.15.47:0/1759809974 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f79fc053980 msgr2=0x7f79fc055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:35.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.478+0000 7f7a2359e700 1 --2- 172.21.15.47:0/1759809974 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f79fc053980 0x7f79fc055e40 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7f7a0c015000 tx=0x7f7a0c014000).stop 2022-06-29T15:51:35.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.478+0000 7f7a2359e700 1 -- 172.21.15.47:0/1759809974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a240b0f50 msgr2=0x7f7a2413e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:35.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.478+0000 7f7a2359e700 1 --2- 172.21.15.47:0/1759809974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a240b0f50 0x7f7a2413e2e0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 rx=0x7f7a1c001250 tx=0x7f7a1c007e00).stop 2022-06-29T15:51:35.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.478+0000 7f7a2359e700 1 -- 172.21.15.47:0/1759809974 shutdown_connections 2022-06-29T15:51:35.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.478+0000 7f7a2359e700 1 --2- 172.21.15.47:0/1759809974 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f79fc053980 0x7f79fc055e40 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:35.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.478+0000 7f7a2359e700 1 --2- 172.21.15.47:0/1759809974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a240b7000 0x7f7a24152de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:35.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.478+0000 7f7a2359e700 1 --2- 172.21.15.47:0/1759809974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a240b0f50 0x7f7a2413e2e0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:35.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.478+0000 7f7a2359e700 1 --2- 172.21.15.47:0/1759809974 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7a24063000 0x7f7a2414f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:35.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.478+0000 7f7a2359e700 1 -- 172.21.15.47:0/1759809974 >> 172.21.15.47:0/1759809974 conn(0x7f7a24061f00 msgr2=0x7f7a240b5420 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:35.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.482+0000 7f7a2359e700 1 -- 172.21.15.47:0/1759809974 shutdown_connections 2022-06-29T15:51:35.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:35.482+0000 7f7a2359e700 1 -- 172.21.15.47:0/1759809974 wait complete. 2022-06-29T15:51:35.490 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 69 2022-06-29T15:51:35.800 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 69, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-29T15:51:23.543892+0000', 'modified': '2022-06-29T15:51:25.309120+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 56894}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_56894': {'gid': 56894, 'name': 'c', 'rank': 0, 'incarnation': 67, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.47:6837/1370488030', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6836', 'nonce': 1370488030}, {'type': 'v1', 'addr': '172.21.15.47:6837', 'nonce': 1370488030}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-29T15:51:35.801 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-29T15:51:35.802 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2022-06-29T15:51:35.921 INFO:teuthology.orchestra.run.smithi047.stdout:{"cluster_fsid":"9a68894a-53ce-400e-838b-14eab91b834c","whoami":0,"id":56894,"want_state":"up:active","state":"up:active","rank_uptime":12.359872717,"mdsmap_epoch":69,"osdmap_epoch":140,"osdmap_epoch_barrier":140,"uptime":17.164563824999998} 2022-06-29T15:51:35.921 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9a68894a-53ce-400e-838b-14eab91b834c", "id": 56894, "mdsmap_epoch": 69, "osdmap_epoch": 140, "osdmap_epoch_barrier": 140, "rank_uptime": 12.359872717, "state": "up:active", "uptime": 17.164563825, "want_state": "up:active", "whoami": 0 } 2022-06-29T15:51:35.922 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-29T15:51:35.922 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2022-06-29T15:51:35.923 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi164.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2022-06-29T15:51:35.923 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:51:35.924 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:51:35.932 DEBUG:teuthology.orchestra.run.smithi164:> sudo modprobe fuse 2022-06-29T15:51:35.950 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:51:35.951 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:51:35.965 INFO:teuthology.orchestra.run.smithi164.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:51:35.966 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:51:35.967 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:51:35.967 DEBUG:teuthology.orchestra.run.smithi164:> ls /sys/fs/fuse/connections 2022-06-29T15:51:36.021 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-29T15:51:36.022 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:51:36.024 DEBUG:teuthology.orchestra.run.smithi164:> sudo modprobe fuse 2022-06-29T15:51:36.041 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:51:36.042 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:51:36.051 INFO:teuthology.orchestra.run.smithi164.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:51:36.052 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:51:36.053 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:51:36.054 DEBUG:teuthology.orchestra.run.smithi164:> ls /sys/fs/fuse/connections 2022-06-29T15:51:36.101 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi164.stderr:2022-06-29T15:51:36.092+0000 7f6b965fe080 -1 init, newargv = 0x55e802ee88e0 newargc=9ceph-fuse[12993]: starting ceph client 2022-06-29T15:51:36.101 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi164.stderr: 2022-06-29T15:51:36.111 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi164.stderr:ceph-fuse[12993]: starting fuse 2022-06-29T15:51:37.208 DEBUG:teuthology.orchestra.run.smithi164:> sudo modprobe fuse 2022-06-29T15:51:37.227 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:51:37.227 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:51:37.237 INFO:teuthology.orchestra.run.smithi164.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:51:37.238 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:51:37.239 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:51:37.240 DEBUG:teuthology.orchestra.run.smithi164:> ls /sys/fs/fuse/connections 2022-06-29T15:51:37.289 INFO:teuthology.orchestra.run.smithi164.stdout:52 2022-06-29T15:51:37.290 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-29T15:51:37.290 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:51:37.291 DEBUG:teuthology.orchestra.run.smithi164:> sudo python3 -c ' 2022-06-29T15:51:37.291 DEBUG:teuthology.orchestra.run.smithi164:> import glob 2022-06-29T15:51:37.291 DEBUG:teuthology.orchestra.run.smithi164:> import re 2022-06-29T15:51:37.292 DEBUG:teuthology.orchestra.run.smithi164:> import os 2022-06-29T15:51:37.292 DEBUG:teuthology.orchestra.run.smithi164:> import subprocess 2022-06-29T15:51:37.293 DEBUG:teuthology.orchestra.run.smithi164:> 2022-06-29T15:51:37.293 DEBUG:teuthology.orchestra.run.smithi164:> def find_socket(client_name): 2022-06-29T15:51:37.294 DEBUG:teuthology.orchestra.run.smithi164:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2022-06-29T15:51:37.294 DEBUG:teuthology.orchestra.run.smithi164:> files = glob.glob(asok_path) 2022-06-29T15:51:37.294 DEBUG:teuthology.orchestra.run.smithi164:> 2022-06-29T15:51:37.295 DEBUG:teuthology.orchestra.run.smithi164:> # Given a non-glob path, it better be there 2022-06-29T15:51:37.295 DEBUG:teuthology.orchestra.run.smithi164:> if "*" not in asok_path: 2022-06-29T15:51:37.296 DEBUG:teuthology.orchestra.run.smithi164:> assert(len(files) == 1) 2022-06-29T15:51:37.296 DEBUG:teuthology.orchestra.run.smithi164:> return files[0] 2022-06-29T15:51:37.297 DEBUG:teuthology.orchestra.run.smithi164:> 2022-06-29T15:51:37.297 DEBUG:teuthology.orchestra.run.smithi164:> for f in files: 2022-06-29T15:51:37.297 DEBUG:teuthology.orchestra.run.smithi164:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-29T15:51:37.298 DEBUG:teuthology.orchestra.run.smithi164:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-29T15:51:37.298 DEBUG:teuthology.orchestra.run.smithi164:> return f 2022-06-29T15:51:37.299 DEBUG:teuthology.orchestra.run.smithi164:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-29T15:51:37.299 DEBUG:teuthology.orchestra.run.smithi164:> 2022-06-29T15:51:37.299 DEBUG:teuthology.orchestra.run.smithi164:> print(find_socket("client.1")) 2022-06-29T15:51:37.300 DEBUG:teuthology.orchestra.run.smithi164:> ' 2022-06-29T15:51:37.345 INFO:teuthology.orchestra.run.smithi164.stdout:/var/run/ceph/ceph-client.1.12993.asok 2022-06-29T15:51:37.350 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.12993.asok 2022-06-29T15:51:37.350 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:51:37.351 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.12993.asok status 2022-06-29T15:51:37.444 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2022-06-29T15:51:37.444 INFO:teuthology.orchestra.run.smithi164.stdout: "metadata": { 2022-06-29T15:51:37.444 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_sha1": "f25ffac72e5a9e63860ff7deb6767a11f0559a16", 2022-06-29T15:51:37.445 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_version": "ceph version 15.2.16-304-gf25ffac7 (f25ffac72e5a9e63860ff7deb6767a11f0559a16) octopus (stable)", 2022-06-29T15:51:37.445 INFO:teuthology.orchestra.run.smithi164.stdout: "entity_id": "1", 2022-06-29T15:51:37.445 INFO:teuthology.orchestra.run.smithi164.stdout: "hostname": "smithi164", 2022-06-29T15:51:37.445 INFO:teuthology.orchestra.run.smithi164.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2022-06-29T15:51:37.445 INFO:teuthology.orchestra.run.smithi164.stdout: "pid": "12993", 2022-06-29T15:51:37.446 INFO:teuthology.orchestra.run.smithi164.stdout: "root": "/" 2022-06-29T15:51:37.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2022-06-29T15:51:37.446 INFO:teuthology.orchestra.run.smithi164.stdout: "dentry_count": 0, 2022-06-29T15:51:37.446 INFO:teuthology.orchestra.run.smithi164.stdout: "dentry_pinned_count": 0, 2022-06-29T15:51:37.447 INFO:teuthology.orchestra.run.smithi164.stdout: "id": 57074, 2022-06-29T15:51:37.447 INFO:teuthology.orchestra.run.smithi164.stdout: "inst": { 2022-06-29T15:51:37.447 INFO:teuthology.orchestra.run.smithi164.stdout: "name": { 2022-06-29T15:51:37.447 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "client", 2022-06-29T15:51:37.448 INFO:teuthology.orchestra.run.smithi164.stdout: "num": 57074 2022-06-29T15:51:37.448 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2022-06-29T15:51:37.448 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": { 2022-06-29T15:51:37.448 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "v1", 2022-06-29T15:51:37.449 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": "172.21.15.164:0", 2022-06-29T15:51:37.449 INFO:teuthology.orchestra.run.smithi164.stdout: "nonce": 3438188789 2022-06-29T15:51:37.449 INFO:teuthology.orchestra.run.smithi164.stdout: } 2022-06-29T15:51:37.449 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2022-06-29T15:51:37.449 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": { 2022-06-29T15:51:37.450 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "v1", 2022-06-29T15:51:37.450 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": "172.21.15.164:0", 2022-06-29T15:51:37.450 INFO:teuthology.orchestra.run.smithi164.stdout: "nonce": 3438188789 2022-06-29T15:51:37.450 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2022-06-29T15:51:37.451 INFO:teuthology.orchestra.run.smithi164.stdout: "inst_str": "client.57074 172.21.15.164:0/3438188789", 2022-06-29T15:51:37.451 INFO:teuthology.orchestra.run.smithi164.stdout: "addr_str": "172.21.15.164:0/3438188789", 2022-06-29T15:51:37.451 INFO:teuthology.orchestra.run.smithi164.stdout: "inode_count": 1, 2022-06-29T15:51:37.451 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_epoch": 69, 2022-06-29T15:51:37.452 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_epoch": 140, 2022-06-29T15:51:37.452 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_epoch_barrier": 0, 2022-06-29T15:51:37.452 INFO:teuthology.orchestra.run.smithi164.stdout: "blacklisted": false, 2022-06-29T15:51:37.452 INFO:teuthology.orchestra.run.smithi164.stdout: "fs_name": "cephfs" 2022-06-29T15:51:37.452 INFO:teuthology.orchestra.run.smithi164.stdout:} 2022-06-29T15:51:37.454 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:51:37.454 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:51:37.455 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:51:37.462 INFO:teuthology.orchestra.run.smithi164.stdout:fuseblk 2022-06-29T15:51:37.463 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-06-29T15:51:37.464 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:51:37.464 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:51:37.479 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:51:37.479 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:51:37.480 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:51:37.488 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2022-06-29T15:51:37.489 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-29T15:51:37.490 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2022-06-29T15:51:37.490 DEBUG:teuthology.orchestra.run.smithi099:> import time 2022-06-29T15:51:37.491 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:51:37.491 DEBUG:teuthology.orchestra.run.smithi099:> with open("/home/ubuntu/cephtest/mnt.0/testfile", '"'"'w'"'"') as f: 2022-06-29T15:51:37.491 DEBUG:teuthology.orchestra.run.smithi099:> f.write('"'"'content'"'"') 2022-06-29T15:51:37.492 DEBUG:teuthology.orchestra.run.smithi099:> f.flush() 2022-06-29T15:51:37.492 DEBUG:teuthology.orchestra.run.smithi099:> f.write('"'"'content2'"'"') 2022-06-29T15:51:37.493 DEBUG:teuthology.orchestra.run.smithi099:> while True: 2022-06-29T15:51:37.493 DEBUG:teuthology.orchestra.run.smithi099:> time.sleep(1) 2022-06-29T15:51:37.493 DEBUG:teuthology.orchestra.run.smithi099:> ' 2022-06-29T15:51:37.496 DEBUG:teuthology.orchestra.run.smithi099:> sudo ls /home/ubuntu/cephtest/mnt.0/testfile 2022-06-29T15:51:37.518 INFO:teuthology.orchestra.run.smithi099.stderr:ls: cannot access '/home/ubuntu/cephtest/mnt.0/testfile': No such file or directory 2022-06-29T15:51:37.519 DEBUG:teuthology.orchestra.run:got remote process result: 2 2022-06-29T15:51:38.521 DEBUG:teuthology.orchestra.run.smithi099:> sudo ls /home/ubuntu/cephtest/mnt.0/testfile 2022-06-29T15:51:38.538 INFO:teuthology.orchestra.run.smithi099.stdout:/home/ubuntu/cephtest/mnt.0/testfile 2022-06-29T15:51:38.539 DEBUG:tasks.cephfs.mount:File testfile became visible from 0 after 1s 2022-06-29T15:51:38.540 DEBUG:teuthology.orchestra.run.smithi164:> sudo ls /home/ubuntu/cephtest/mnt.1/testfile 2022-06-29T15:51:38.579 INFO:teuthology.orchestra.run.smithi164.stdout:/home/ubuntu/cephtest/mnt.1/testfile 2022-06-29T15:51:38.580 DEBUG:tasks.cephfs.mount:File testfile became visible from 1 after 0s 2022-06-29T15:51:38.581 INFO:teuthology.orchestra.run:Running command with timeout 10800 2022-06-29T15:51:38.581 DEBUG:teuthology.orchestra.run.smithi164:> cd /home/ubuntu/cephtest/mnt.1 && sudo cat testfile 2022-06-29T15:51:38.623 INFO:teuthology.orchestra.run.smithi164.stdout:content 2022-06-29T15:51:38.639 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-06-29T15:51:38.639 DEBUG:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:51:38.661 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:51:38.682 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:51:38.703 DEBUG:teuthology.orchestra.run.smithi164:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:51:38.723 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-06-29T15:51:38.724 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:51:38.724 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:51:38.724 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:51:38.732 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2022-06-29T15:51:38.733 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-29T15:51:38.734 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi099.front.sepia.ceph.com... 2022-06-29T15:51:38.734 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-06-29T15:51:38.735 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:51:38.788 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[13405]: fuse finished with error 0 and tester_r 0 2022-06-29T15:51:38.791 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:51:38.792 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:51:38.792 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:51:38.799 INFO:teuthology.orchestra.run.smithi099.stdout:ext2/ext3 2022-06-29T15:51:38.800 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-06-29T15:51:38.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-29T15:51:38.917 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:51:38.925 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:51:38.925 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:51:38.926 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:51:38.934 INFO:teuthology.orchestra.run.smithi164.stdout:fuseblk 2022-06-29T15:51:38.935 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-06-29T15:51:38.935 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi164.front.sepia.ceph.com... 2022-06-29T15:51:38.936 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-06-29T15:51:38.936 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:51:38.951 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi164.stderr:ceph-fuse[12993]: fuse finished with error 0 and tester_r 0 2022-06-29T15:51:38.954 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:51:38.954 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:51:38.955 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:51:39.001 INFO:teuthology.orchestra.run.smithi164.stdout:ext2/ext3 2022-06-29T15:51:39.002 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-06-29T15:51:39.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-29T15:51:39.099 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:51:39.106 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-06-29T15:51:39.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.210+0000 7ff773fff700 1 Processor -- start 2022-06-29T15:51:39.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.210+0000 7ff773fff700 1 -- start start 2022-06-29T15:51:39.223 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.214+0000 7ff773fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff774062910 0x7ff7740b62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:39.223 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.214+0000 7ff773fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff774140700 0x7ff774140b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:39.224 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.214+0000 7ff773fff700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7740b5240 con 0x7ff7740b5670 2022-06-29T15:51:39.224 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.214+0000 7ff773fff700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff774062cf0 con 0x7ff774140700 2022-06-29T15:51:39.224 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.214+0000 7ff773fff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff774141010 con 0x7ff774062910 2022-06-29T15:51:39.224 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.214+0000 7ff7727fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff774140700 0x7ff774140b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:39.225 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.214+0000 7ff7737fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff7740b5670 0x7ff7740b5b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39546/0 (socket says 172.21.15.47:39546) 2022-06-29T15:51:39.225 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.214+0000 7ff7737fe700 1 -- 172.21.15.47:0/605328118 learned_addr learned my addr 172.21.15.47:0/605328118 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:39.225 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.214+0000 7ff772ffd700 1 --2- 172.21.15.47:0/605328118 >> v2:172.21.15.64:3300/0 conn(0x7ff774062910 0x7ff7740b62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:39.226 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.214+0000 7ff7727fc700 1 -- 172.21.15.47:0/605328118 >> v2:172.21.15.64:3300/0 conn(0x7ff774062910 msgr2=0x7ff7740b62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:39.226 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.214+0000 7ff7727fc700 1 --2- 172.21.15.47:0/605328118 >> v2:172.21.15.64:3300/0 conn(0x7ff774062910 0x7ff7740b62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:39.226 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.214+0000 7ff7727fc700 1 -- 172.21.15.47:0/605328118 >> v1:172.21.15.47:6789/0 conn(0x7ff7740b5670 legacy=0x7ff7740b5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:39.226 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.214+0000 7ff771ffb700 1 -- 172.21.15.47:0/605328118 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3531348820 0 0) 0x7ff7740b5240 con 0x7ff7740b5670 2022-06-29T15:51:39.227 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.214+0000 7ff7727fc700 1 -- 172.21.15.47:0/605328118 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7740b2390 con 0x7ff774140700 2022-06-29T15:51:39.227 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.214+0000 7ff772ffd700 1 --2- 172.21.15.47:0/605328118 >> v2:172.21.15.64:3300/0 conn(0x7ff774062910 0x7ff7740b62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:51:39.227 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.214+0000 7ff7727fc700 1 --2- 172.21.15.47:0/605328118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff774140700 0x7ff774140b20 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 rx=0x7ff75c002f70 tx=0x7ff75c006df0).ready entity=mon.0 client_cookie=d5c47f46d7f18b49 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:39.227 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.214+0000 7ff771ffb700 1 -- 172.21.15.47:0/605328118 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff75c00d5c0 con 0x7ff774140700 2022-06-29T15:51:39.228 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.214+0000 7ff771ffb700 1 -- 172.21.15.47:0/605328118 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff75c000e90 con 0x7ff774140700 2022-06-29T15:51:39.228 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.214+0000 7ff771ffb700 1 -- 172.21.15.47:0/605328118 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff75c00d910 con 0x7ff774140700 2022-06-29T15:51:39.228 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.214+0000 7ff773fff700 1 -- 172.21.15.47:0/605328118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff774140700 msgr2=0x7ff774140b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:39.228 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.214+0000 7ff773fff700 1 --2- 172.21.15.47:0/605328118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff774140700 0x7ff774140b20 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 rx=0x7ff75c002f70 tx=0x7ff75c006df0).stop 2022-06-29T15:51:39.228 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.214+0000 7ff773fff700 1 -- 172.21.15.47:0/605328118 shutdown_connections 2022-06-29T15:51:39.229 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.214+0000 7ff773fff700 1 --2- 172.21.15.47:0/605328118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff774140700 0x7ff774140b20 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:39.229 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.214+0000 7ff773fff700 1 --2- 172.21.15.47:0/605328118 >> v2:172.21.15.64:3300/0 conn(0x7ff774062910 0x7ff7740b62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:39.229 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.214+0000 7ff773fff700 1 -- 172.21.15.47:0/605328118 >> 172.21.15.47:0/605328118 conn(0x7ff77405fce0 msgr2=0x7ff7740600c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:39.230 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.214+0000 7ff773fff700 1 -- 172.21.15.47:0/605328118 shutdown_connections 2022-06-29T15:51:39.230 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.214+0000 7ff773fff700 1 -- 172.21.15.47:0/605328118 wait complete. 2022-06-29T15:51:39.230 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.218+0000 7ff773fff700 1 Processor -- start 2022-06-29T15:51:39.230 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.218+0000 7ff773fff700 1 -- start start 2022-06-29T15:51:39.233 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.218+0000 7ff773fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff774062910 0x7ff7740b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:39.233 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.218+0000 7ff773fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff7740b5670 0x7ff77413fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:39.234 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.218+0000 7ff773fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff774140700 0x7ff77413ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:39.234 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.218+0000 7ff773fff700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff774153030 con 0x7ff774062910 2022-06-29T15:51:39.234 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.218+0000 7ff773fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7740b9520 con 0x7ff774140700 2022-06-29T15:51:39.234 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.218+0000 7ff773fff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7740b97c0 con 0x7ff7740b5670 2022-06-29T15:51:39.235 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.218+0000 7ff772ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff774062910 0x7ff7740b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:39.235 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.218+0000 7ff772ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff774062910 0x7ff7740b03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53994/0 (socket says 172.21.15.47:53994) 2022-06-29T15:51:39.235 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.218+0000 7ff772ffd700 1 -- 172.21.15.47:0/1888707090 learned_addr learned my addr 172.21.15.47:0/1888707090 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:39.235 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.218+0000 7ff7727fc700 1 --2- 172.21.15.47:0/1888707090 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff7740b5670 0x7ff77413fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:39.236 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.218+0000 7ff7737fe700 1 --2- 172.21.15.47:0/1888707090 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff774140700 0x7ff77413ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:39.236 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.218+0000 7ff772ffd700 1 -- 172.21.15.47:0/1888707090 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff7740b5670 msgr2=0x7ff77413fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:39.236 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.218+0000 7ff772ffd700 1 --2- 172.21.15.47:0/1888707090 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff7740b5670 0x7ff77413fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:39.236 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.218+0000 7ff772ffd700 1 -- 172.21.15.47:0/1888707090 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff774140700 msgr2=0x7ff77413ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:39.237 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.218+0000 7ff772ffd700 1 --2- 172.21.15.47:0/1888707090 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff774140700 0x7ff77413ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:39.237 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.218+0000 7ff772ffd700 1 -- 172.21.15.47:0/1888707090 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff75c00b180 con 0x7ff774062910 2022-06-29T15:51:39.237 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.218+0000 7ff772ffd700 1 --2- 172.21.15.47:0/1888707090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff774062910 0x7ff7740b03d0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 rx=0x7ff764007000 tx=0x7ff764008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:39.237 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.218+0000 7ff76bfff700 1 -- 172.21.15.47:0/1888707090 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff764010650 con 0x7ff774062910 2022-06-29T15:51:39.237 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.218+0000 7ff76bfff700 1 -- 172.21.15.47:0/1888707090 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7640107b0 con 0x7ff774062910 2022-06-29T15:51:39.238 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.218+0000 7ff773fff700 1 -- 172.21.15.47:0/1888707090 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7740b9a60 con 0x7ff774062910 2022-06-29T15:51:39.238 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.218+0000 7ff76bfff700 1 -- 172.21.15.47:0/1888707090 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff764010b10 con 0x7ff774062910 2022-06-29T15:51:39.238 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.218+0000 7ff773fff700 1 -- 172.21.15.47:0/1888707090 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7740ba470 con 0x7ff774062910 2022-06-29T15:51:39.238 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.222+0000 7ff773fff700 1 -- 172.21.15.47:0/1888707090 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7740573d0 con 0x7ff774062910 2022-06-29T15:51:39.239 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.222+0000 7ff76bfff700 1 -- 172.21.15.47:0/1888707090 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff764010c70 con 0x7ff774062910 2022-06-29T15:51:39.239 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.222+0000 7ff76bfff700 1 --2- 172.21.15.47:0/1888707090 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff7580539e0 0x7ff758055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:39.239 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.222+0000 7ff76bfff700 1 -- 172.21.15.47:0/1888707090 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff76406c290 con 0x7ff774062910 2022-06-29T15:51:39.239 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.222+0000 7ff7727fc700 1 --2- 172.21.15.47:0/1888707090 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff7580539e0 0x7ff758055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:39.240 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.222+0000 7ff7727fc700 1 --2- 172.21.15.47:0/1888707090 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff7580539e0 0x7ff758055ea0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7ff75c01a000 tx=0x7ff75c014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:39.240 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.230+0000 7ff76bfff700 1 -- 172.21.15.47:0/1888707090 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff764043050 con 0x7ff774062910 2022-06-29T15:51:39.364 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.358+0000 7ff773fff700 1 -- 172.21.15.47:0/1888707090 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7ff774067ae0 con 0x7ff774062910 2022-06-29T15:51:39.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.358+0000 7ff76bfff700 1 -- 172.21.15.47:0/1888707090 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v140) v1 ==== 81+0+34051 (secure 0 0 0) 0x7ff764046670 con 0x7ff774062910 2022-06-29T15:51:39.367 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:39.367 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2022-06-29T15:51:39.368 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 140, 2022-06-29T15:51:39.368 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "9a68894a-53ce-400e-838b-14eab91b834c", 2022-06-29T15:51:39.368 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2022-06-29T15:42:14.046361+0000", 2022-06-29T15:51:39.368 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2022-06-29T15:51:24.546533+0000", 2022-06-29T15:51:39.368 INFO:teuthology.orchestra.run.smithi047.stdout: "last_up_change": "2022-06-29T15:42:20.929789+0000", 2022-06-29T15:51:39.369 INFO:teuthology.orchestra.run.smithi047.stdout: "last_in_change": "2022-06-29T15:42:20.929789+0000", 2022-06-29T15:51:39.369 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-06-29T15:51:39.369 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_num": 5799936, 2022-06-29T15:51:39.369 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_set": [ 2022-06-29T15:51:39.370 INFO:teuthology.orchestra.run.smithi047.stdout: "pglog_hardlimit", 2022-06-29T15:51:39.370 INFO:teuthology.orchestra.run.smithi047.stdout: "purged_snapdirs", 2022-06-29T15:51:39.370 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_deletes", 2022-06-29T15:51:39.370 INFO:teuthology.orchestra.run.smithi047.stdout: "sortbitwise" 2022-06-29T15:51:39.371 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:51:39.371 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_version": 5, 2022-06-29T15:51:39.371 INFO:teuthology.orchestra.run.smithi047.stdout: "full_ratio": 0.94999998807907104, 2022-06-29T15:51:39.371 INFO:teuthology.orchestra.run.smithi047.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-06-29T15:51:39.372 INFO:teuthology.orchestra.run.smithi047.stdout: "nearfull_ratio": 0.85000002384185791, 2022-06-29T15:51:39.372 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_snapshot": "", 2022-06-29T15:51:39.372 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_max": 26, 2022-06-29T15:51:39.372 INFO:teuthology.orchestra.run.smithi047.stdout: "max_osd": 8, 2022-06-29T15:51:39.373 INFO:teuthology.orchestra.run.smithi047.stdout: "require_min_compat_client": "jewel", 2022-06-29T15:51:39.373 INFO:teuthology.orchestra.run.smithi047.stdout: "min_compat_client": "jewel", 2022-06-29T15:51:39.373 INFO:teuthology.orchestra.run.smithi047.stdout: "require_osd_release": "octopus", 2022-06-29T15:51:39.373 INFO:teuthology.orchestra.run.smithi047.stdout: "pools": [ 2022-06-29T15:51:39.374 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.374 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 1, 2022-06-29T15:51:39.374 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "device_health_metrics", 2022-06-29T15:51:39.374 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:42:16.971431+0000", 2022-06-29T15:51:39.374 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:51:39.375 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:51:39.375 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:51:39.375 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:51:39.375 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:51:39.376 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:51:39.376 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:51:39.376 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:51:39.376 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 1, 2022-06-29T15:51:39.377 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 1, 2022-06-29T15:51:39.377 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 1, 2022-06-29T15:51:39.377 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 1, 2022-06-29T15:51:39.377 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 1, 2022-06-29T15:51:39.378 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:51:39.378 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:51:39.378 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:51:39.378 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:51:39.378 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:51:39.379 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:51:39.379 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:51:39.379 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.379 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "16", 2022-06-29T15:51:39.380 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:51:39.380 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:51:39.380 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:51:39.380 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:51:39.381 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:51:39.381 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:51:39.381 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:51:39.381 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:51:39.381 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:51:39.382 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:51:39.382 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:51:39.382 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:51:39.382 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:51:39.383 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:51:39.383 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:51:39.383 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:51:39.383 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:51:39.383 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:51:39.384 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:51:39.384 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:51:39.384 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:51:39.384 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:51:39.384 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:51:39.385 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:51:39.385 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:51:39.385 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:51:39.385 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.386 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:51:39.386 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:51:39.386 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:51:39.386 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:51:39.386 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:51:39.387 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:51:39.387 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:51:39.387 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:51:39.387 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:51:39.388 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:51:39.388 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:51:39.388 INFO:teuthology.orchestra.run.smithi047.stdout: "options": { 2022-06-29T15:51:39.388 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_min": 1 2022-06-29T15:51:39.388 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.389 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:51:39.389 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_devicehealth": {} 2022-06-29T15:51:39.389 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:39.389 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.390 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.390 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 2, 2022-06-29T15:51:39.390 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "rbd", 2022-06-29T15:51:39.390 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:42:25.290884+0000", 2022-06-29T15:51:39.391 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:51:39.391 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:51:39.391 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:51:39.391 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:51:39.392 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:51:39.392 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:51:39.392 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:51:39.392 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:51:39.393 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2022-06-29T15:51:39.393 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2022-06-29T15:51:39.393 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2022-06-29T15:51:39.393 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2022-06-29T15:51:39.393 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2022-06-29T15:51:39.394 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:51:39.394 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:51:39.394 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:51:39.394 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:51:39.395 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:51:39.395 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:51:39.395 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:51:39.395 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.395 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "19", 2022-06-29T15:51:39.396 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:51:39.396 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:51:39.396 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:51:39.396 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:51:39.397 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:51:39.397 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:51:39.397 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:51:39.397 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:51:39.397 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:51:39.398 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:51:39.398 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:51:39.398 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:51:39.398 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:51:39.399 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:51:39.399 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:51:39.399 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:51:39.399 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:51:39.400 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:51:39.400 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:51:39.400 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:51:39.400 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:51:39.400 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:51:39.401 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:51:39.401 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:51:39.401 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:51:39.401 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:51:39.402 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.402 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:51:39.402 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:51:39.402 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:51:39.403 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:51:39.403 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:51:39.403 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:51:39.403 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:51:39.403 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:51:39.404 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:51:39.404 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:51:39.404 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:51:39.404 INFO:teuthology.orchestra.run.smithi047.stdout: "options": {}, 2022-06-29T15:51:39.405 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:51:39.405 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd": {} 2022-06-29T15:51:39.405 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:39.405 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.405 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.406 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 25, 2022-06-29T15:51:39.406 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "cephfs_metadata", 2022-06-29T15:51:39.406 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:51:21.506960+0000", 2022-06-29T15:51:39.406 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:51:39.407 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:51:39.407 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:51:39.407 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:51:39.407 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:51:39.408 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:51:39.408 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:51:39.408 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:51:39.408 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2022-06-29T15:51:39.408 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2022-06-29T15:51:39.409 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2022-06-29T15:51:39.409 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2022-06-29T15:51:39.409 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2022-06-29T15:51:39.409 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:51:39.410 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:51:39.410 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:51:39.410 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:51:39.410 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:51:39.410 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:51:39.411 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:51:39.411 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.411 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "139", 2022-06-29T15:51:39.411 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:51:39.412 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:51:39.412 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:51:39.412 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:51:39.412 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:51:39.412 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:51:39.413 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:51:39.413 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:51:39.413 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:51:39.413 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:51:39.414 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:51:39.414 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:51:39.414 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:51:39.414 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:51:39.414 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:51:39.415 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:51:39.415 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:51:39.415 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:51:39.415 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:51:39.416 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:51:39.416 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:51:39.416 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:51:39.416 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:51:39.417 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:51:39.417 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:51:39.417 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:51:39.418 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.418 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:51:39.419 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:51:39.419 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:51:39.419 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:51:39.419 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:51:39.419 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:51:39.420 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:51:39.420 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:51:39.420 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:51:39.420 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:51:39.421 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:51:39.421 INFO:teuthology.orchestra.run.smithi047.stdout: "options": { 2022-06-29T15:51:39.421 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_bias": 4, 2022-06-29T15:51:39.421 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_min": 16, 2022-06-29T15:51:39.421 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_priority": 5 2022-06-29T15:51:39.422 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.422 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:51:39.422 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs": { 2022-06-29T15:51:39.422 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": "cephfs" 2022-06-29T15:51:39.423 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:39.423 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:39.423 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.423 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.424 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 26, 2022-06-29T15:51:39.424 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "cephfs_data", 2022-06-29T15:51:39.424 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:51:22.546983+0000", 2022-06-29T15:51:39.424 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:51:39.424 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:51:39.425 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:51:39.425 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:51:39.425 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:51:39.425 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:51:39.426 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:51:39.426 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:51:39.426 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2022-06-29T15:51:39.426 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2022-06-29T15:51:39.427 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2022-06-29T15:51:39.427 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2022-06-29T15:51:39.427 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2022-06-29T15:51:39.427 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:51:39.428 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:51:39.428 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:51:39.428 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:51:39.428 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:51:39.428 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:51:39.429 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:51:39.429 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.429 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "140", 2022-06-29T15:51:39.429 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:51:39.429 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:51:39.430 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:51:39.430 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:51:39.430 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:51:39.430 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:51:39.431 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:51:39.431 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:51:39.431 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:51:39.431 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:51:39.432 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:51:39.432 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:51:39.432 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:51:39.432 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:51:39.432 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:51:39.433 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:51:39.433 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:51:39.433 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:51:39.433 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:51:39.434 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:51:39.434 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:51:39.434 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:51:39.434 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:51:39.435 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:51:39.435 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:51:39.435 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:51:39.435 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.436 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:51:39.436 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:51:39.436 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:51:39.436 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:51:39.436 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:51:39.437 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:51:39.437 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:51:39.437 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:51:39.437 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:51:39.438 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:51:39.438 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:51:39.438 INFO:teuthology.orchestra.run.smithi047.stdout: "options": {}, 2022-06-29T15:51:39.438 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:51:39.439 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs": { 2022-06-29T15:51:39.439 INFO:teuthology.orchestra.run.smithi047.stdout: "data": "cephfs" 2022-06-29T15:51:39.439 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:39.440 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:39.440 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:39.441 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:51:39.441 INFO:teuthology.orchestra.run.smithi047.stdout: "osds": [ 2022-06-29T15:51:39.441 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.442 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2022-06-29T15:51:39.442 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "e23eb023-4706-4201-bf72-4bf19c912c5d", 2022-06-29T15:51:39.443 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:51:39.443 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:51:39.443 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:51:39.443 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:51:39.444 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:51:39.444 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:51:39.444 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:51:39.444 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 137, 2022-06-29T15:51:39.444 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:51:39.445 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:51:39.445 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:51:39.445 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:39.445 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.446 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:39.446 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6818", 2022-06-29T15:51:39.446 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:51:39.446 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.446 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.447 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:39.447 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6819", 2022-06-29T15:51:39.447 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:51:39.447 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:39.448 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:39.448 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.448 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:51:39.448 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:39.448 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.449 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:39.449 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6820", 2022-06-29T15:51:39.449 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:51:39.449 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.450 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.450 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:39.450 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6821", 2022-06-29T15:51:39.450 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:51:39.451 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:39.451 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:39.451 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.451 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:51:39.452 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:39.452 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.452 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:39.452 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6824", 2022-06-29T15:51:39.453 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:51:39.453 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.453 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.453 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:39.454 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6825", 2022-06-29T15:51:39.454 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:51:39.454 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:39.454 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:39.454 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.455 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:51:39.455 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:39.455 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.455 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:39.456 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6822", 2022-06-29T15:51:39.456 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:51:39.456 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.456 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.456 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:39.457 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6823", 2022-06-29T15:51:39.457 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:51:39.457 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:39.457 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:39.458 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.458 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6819/13096", 2022-06-29T15:51:39.458 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6821/13096", 2022-06-29T15:51:39.458 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6825/13096", 2022-06-29T15:51:39.459 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6823/13096", 2022-06-29T15:51:39.459 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:51:39.459 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:51:39.459 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:51:39.459 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:39.460 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.460 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.460 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2022-06-29T15:51:39.460 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "228bb08d-2d64-43d9-9560-bfaeff154452", 2022-06-29T15:51:39.460 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:51:39.461 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:51:39.461 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:51:39.461 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:51:39.461 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:51:39.462 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:51:39.462 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:51:39.462 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 138, 2022-06-29T15:51:39.462 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:51:39.463 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:51:39.463 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:51:39.463 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:39.463 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.463 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:39.464 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6826", 2022-06-29T15:51:39.464 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:51:39.464 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.464 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.464 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:39.465 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6827", 2022-06-29T15:51:39.465 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:51:39.465 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:39.465 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:39.466 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.466 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:51:39.466 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:39.466 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.467 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:39.467 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6828", 2022-06-29T15:51:39.467 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:51:39.467 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.467 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.468 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:39.468 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6829", 2022-06-29T15:51:39.468 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:51:39.468 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:39.468 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:39.469 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.469 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:51:39.469 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:39.469 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.470 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:39.470 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6832", 2022-06-29T15:51:39.470 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:51:39.470 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.471 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.471 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:39.471 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6833", 2022-06-29T15:51:39.472 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:51:39.472 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:39.472 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:39.473 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.473 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:51:39.473 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:39.473 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.474 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:39.474 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6830", 2022-06-29T15:51:39.474 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:51:39.475 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.475 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.475 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:39.476 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6831", 2022-06-29T15:51:39.476 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:51:39.476 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:39.477 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:39.477 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.477 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6827/13100", 2022-06-29T15:51:39.477 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6829/13100", 2022-06-29T15:51:39.478 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6833/13100", 2022-06-29T15:51:39.478 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6831/13100", 2022-06-29T15:51:39.478 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:51:39.478 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:51:39.479 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:51:39.479 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:39.479 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.479 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.479 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2022-06-29T15:51:39.480 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "797a5452-1510-4640-a111-c9c2fcf707eb", 2022-06-29T15:51:39.480 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:51:39.480 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:51:39.480 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:51:39.481 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:51:39.481 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:51:39.481 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:51:39.481 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:51:39.481 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 138, 2022-06-29T15:51:39.482 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:51:39.482 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:51:39.482 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:51:39.482 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:39.483 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.483 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:39.483 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6810", 2022-06-29T15:51:39.483 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:51:39.484 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.484 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.484 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:39.484 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6811", 2022-06-29T15:51:39.484 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:51:39.485 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:39.485 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:39.485 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.485 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:51:39.486 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:39.486 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.486 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:39.486 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6812", 2022-06-29T15:51:39.487 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:51:39.487 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.487 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.487 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:39.487 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6813", 2022-06-29T15:51:39.488 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:51:39.488 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:39.488 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:39.488 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.488 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:51:39.489 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:39.489 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.489 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:39.489 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6816", 2022-06-29T15:51:39.490 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:51:39.490 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.490 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.490 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:39.490 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6817", 2022-06-29T15:51:39.491 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:51:39.491 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:39.491 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:39.491 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.492 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:51:39.492 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:39.492 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.492 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:39.493 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6814", 2022-06-29T15:51:39.493 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:51:39.493 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.493 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.493 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:39.494 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6815", 2022-06-29T15:51:39.494 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:51:39.494 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:39.494 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:39.495 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.495 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6811/13098", 2022-06-29T15:51:39.495 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6813/13098", 2022-06-29T15:51:39.495 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6817/13098", 2022-06-29T15:51:39.496 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6815/13098", 2022-06-29T15:51:39.496 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:51:39.496 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:51:39.496 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:51:39.497 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:39.497 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.497 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.497 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2022-06-29T15:51:39.497 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "55b24fd7-bb71-4a01-b3fa-a09caa772a62", 2022-06-29T15:51:39.498 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:51:39.498 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:51:39.498 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:51:39.498 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:51:39.499 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:51:39.499 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:51:39.499 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:51:39.499 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 137, 2022-06-29T15:51:39.499 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:51:39.500 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:51:39.500 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:51:39.500 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:39.500 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.501 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:39.501 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6802", 2022-06-29T15:51:39.501 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:51:39.501 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.501 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.502 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:39.502 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6803", 2022-06-29T15:51:39.502 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:51:39.502 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:39.503 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:39.503 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.503 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:51:39.503 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:39.503 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.504 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:39.504 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6804", 2022-06-29T15:51:39.504 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:51:39.504 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.505 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.505 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:39.505 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6805", 2022-06-29T15:51:39.505 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:51:39.505 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:39.506 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:39.506 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.506 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:51:39.506 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:39.507 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.507 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:39.507 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6808", 2022-06-29T15:51:39.507 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:51:39.507 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.508 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.508 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:39.508 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6809", 2022-06-29T15:51:39.508 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:51:39.509 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:39.509 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:39.509 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.509 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:51:39.509 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:39.510 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.510 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:39.510 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6806", 2022-06-29T15:51:39.510 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:51:39.510 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.511 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.511 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:39.511 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6807", 2022-06-29T15:51:39.511 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:51:39.512 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:39.512 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:39.512 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.512 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6803/13097", 2022-06-29T15:51:39.512 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6805/13097", 2022-06-29T15:51:39.513 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6809/13097", 2022-06-29T15:51:39.513 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6807/13097", 2022-06-29T15:51:39.513 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:51:39.513 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:51:39.513 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:51:39.514 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:39.514 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.514 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.514 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2022-06-29T15:51:39.515 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3", 2022-06-29T15:51:39.515 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:51:39.515 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:51:39.515 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:51:39.516 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:51:39.516 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:51:39.516 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:51:39.516 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:51:39.517 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 138, 2022-06-29T15:51:39.517 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:51:39.517 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:51:39.517 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:51:39.517 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:39.518 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.518 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:39.518 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6816", 2022-06-29T15:51:39.518 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:51:39.519 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.519 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.519 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:39.519 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6817", 2022-06-29T15:51:39.519 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:51:39.520 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:39.520 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:39.520 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.520 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:51:39.521 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:39.521 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.521 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:39.521 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6818", 2022-06-29T15:51:39.522 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:51:39.522 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.522 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.522 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:39.522 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6819", 2022-06-29T15:51:39.523 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:51:39.523 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:39.523 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:39.523 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.524 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:51:39.524 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:39.524 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.524 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:39.524 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6822", 2022-06-29T15:51:39.525 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:51:39.525 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.525 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.525 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:39.526 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6823", 2022-06-29T15:51:39.526 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:51:39.526 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:39.526 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:39.527 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.527 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:51:39.527 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:39.527 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.528 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:39.528 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6820", 2022-06-29T15:51:39.528 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:51:39.528 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.528 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.529 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:39.529 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6821", 2022-06-29T15:51:39.529 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:51:39.529 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:39.530 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:39.530 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.530 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6817/13396", 2022-06-29T15:51:39.530 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6819/13396", 2022-06-29T15:51:39.531 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6823/13396", 2022-06-29T15:51:39.531 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6821/13396", 2022-06-29T15:51:39.531 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:51:39.531 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:51:39.532 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:51:39.532 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:39.532 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.533 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.533 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2022-06-29T15:51:39.533 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "f77beebe-6cca-4244-b2d4-677825ebf919", 2022-06-29T15:51:39.533 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:51:39.534 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:51:39.534 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:51:39.534 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:51:39.535 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:51:39.535 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:51:39.535 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:51:39.536 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 131, 2022-06-29T15:51:39.536 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:51:39.536 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:51:39.537 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:51:39.537 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:39.537 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.537 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:39.538 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6800", 2022-06-29T15:51:39.538 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:51:39.538 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.539 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.539 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:39.539 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6801", 2022-06-29T15:51:39.539 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:51:39.540 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:39.540 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:39.540 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.541 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:51:39.541 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:39.541 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.542 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:39.542 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6802", 2022-06-29T15:51:39.542 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:51:39.542 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.542 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.543 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:39.543 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6803", 2022-06-29T15:51:39.543 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:51:39.543 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:39.544 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:39.544 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.544 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:51:39.544 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:39.544 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.545 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:39.545 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6806", 2022-06-29T15:51:39.545 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:51:39.545 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.546 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.546 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:39.546 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6807", 2022-06-29T15:51:39.547 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:51:39.547 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:39.547 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:39.547 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.548 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:51:39.548 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:39.548 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.549 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:39.549 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6804", 2022-06-29T15:51:39.549 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:51:39.549 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.550 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.550 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:39.550 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6805", 2022-06-29T15:51:39.551 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:51:39.551 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:39.551 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:39.552 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.552 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6801/13393", 2022-06-29T15:51:39.552 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6803/13393", 2022-06-29T15:51:39.553 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6807/13393", 2022-06-29T15:51:39.553 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6805/13393", 2022-06-29T15:51:39.554 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:51:39.554 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:51:39.554 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:51:39.555 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:39.555 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.555 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.556 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2022-06-29T15:51:39.556 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "af0ca9b3-c02c-4910-834b-bf0b8dc6713c", 2022-06-29T15:51:39.557 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:51:39.557 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:51:39.557 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:51:39.558 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:51:39.558 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:51:39.558 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:51:39.559 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:51:39.559 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 138, 2022-06-29T15:51:39.559 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:51:39.560 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:51:39.560 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:51:39.561 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:39.561 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.561 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:39.562 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6824", 2022-06-29T15:51:39.562 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:51:39.562 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.562 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.563 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:39.563 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6825", 2022-06-29T15:51:39.563 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:51:39.563 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:39.563 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:39.564 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.564 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:51:39.564 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:39.564 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.565 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:39.565 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6826", 2022-06-29T15:51:39.565 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:51:39.565 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.565 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.566 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:39.566 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6827", 2022-06-29T15:51:39.566 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:51:39.566 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:39.566 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:39.567 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.567 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:51:39.567 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:39.567 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.568 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:39.568 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6830", 2022-06-29T15:51:39.568 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:51:39.568 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.568 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.569 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:39.569 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6831", 2022-06-29T15:51:39.569 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:51:39.569 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:39.570 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:39.570 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.570 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:51:39.570 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:39.570 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.571 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:39.571 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6828", 2022-06-29T15:51:39.571 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:51:39.571 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.572 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.572 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:39.572 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6829", 2022-06-29T15:51:39.572 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:51:39.572 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:39.573 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:39.573 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.573 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6825/13395", 2022-06-29T15:51:39.573 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6827/13395", 2022-06-29T15:51:39.574 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6831/13395", 2022-06-29T15:51:39.574 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6829/13395", 2022-06-29T15:51:39.574 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:51:39.574 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:51:39.574 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:51:39.575 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:39.575 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.575 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.575 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2022-06-29T15:51:39.576 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "f19c3ca6-df88-4bde-a150-b51a53f092e1", 2022-06-29T15:51:39.576 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:51:39.576 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:51:39.576 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:51:39.576 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:51:39.577 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:51:39.577 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:51:39.577 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:51:39.577 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 138, 2022-06-29T15:51:39.578 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:51:39.578 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:51:39.578 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:51:39.578 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:39.578 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.579 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:39.579 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6808", 2022-06-29T15:51:39.579 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:51:39.579 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.580 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.580 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:39.580 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6809", 2022-06-29T15:51:39.580 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:51:39.581 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:39.581 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:39.581 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.581 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:51:39.582 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:39.582 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.582 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:39.582 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6810", 2022-06-29T15:51:39.583 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:51:39.583 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.583 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.583 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:39.584 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6811", 2022-06-29T15:51:39.584 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:51:39.584 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:39.584 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:39.585 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.585 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:51:39.585 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:39.585 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.586 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:39.586 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6814", 2022-06-29T15:51:39.586 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:51:39.586 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.587 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.587 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:39.587 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6815", 2022-06-29T15:51:39.588 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:51:39.588 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:39.588 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:39.588 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.589 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:51:39.589 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:39.589 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.589 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:39.590 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6812", 2022-06-29T15:51:39.590 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:51:39.590 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.591 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.591 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:39.591 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6813", 2022-06-29T15:51:39.591 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:51:39.592 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:39.592 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:39.592 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.592 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6809/13394", 2022-06-29T15:51:39.593 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6811/13394", 2022-06-29T15:51:39.593 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6815/13394", 2022-06-29T15:51:39.593 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6813/13394", 2022-06-29T15:51:39.593 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:51:39.593 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:51:39.594 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:51:39.594 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:39.594 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:39.594 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:51:39.595 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_xinfo": [ 2022-06-29T15:51:39.595 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.595 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2022-06-29T15:51:39.595 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:51:39.595 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:51:39.596 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:51:39.596 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:51:39.596 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:51:39.596 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.720513+0000", 2022-06-29T15:51:39.596 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:51:39.597 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.597 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.597 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2022-06-29T15:51:39.597 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:51:39.598 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:51:39.598 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:51:39.598 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:51:39.598 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:51:39.598 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.772612+0000", 2022-06-29T15:51:39.599 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:51:39.599 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.599 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.599 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2022-06-29T15:51:39.600 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:51:39.600 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:51:39.600 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:51:39.600 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:51:39.600 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:51:39.601 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.450223+0000", 2022-06-29T15:51:39.601 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:51:39.601 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.601 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.601 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2022-06-29T15:51:39.602 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:51:39.602 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:51:39.602 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:51:39.602 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:51:39.603 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:51:39.603 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.477856+0000", 2022-06-29T15:51:39.603 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:51:39.603 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.603 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.604 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2022-06-29T15:51:39.604 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:51:39.604 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:51:39.604 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:51:39.604 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:51:39.605 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:51:39.605 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.539485+0000", 2022-06-29T15:51:39.605 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:51:39.605 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.606 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.606 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2022-06-29T15:51:39.606 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:51:39.606 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:51:39.606 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:51:39.607 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:51:39.607 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:51:39.607 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.487040+0000", 2022-06-29T15:51:39.607 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:51:39.607 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.608 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.608 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2022-06-29T15:51:39.608 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:51:39.608 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:51:39.609 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:51:39.609 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:51:39.609 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:51:39.609 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.748685+0000", 2022-06-29T15:51:39.610 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:51:39.610 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.610 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:39.610 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2022-06-29T15:51:39.610 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:51:39.611 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:51:39.611 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:51:39.611 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:51:39.611 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:51:39.612 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.459516+0000", 2022-06-29T15:51:39.612 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:51:39.612 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:39.612 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:51:39.612 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap": [], 2022-06-29T15:51:39.613 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap_items": [], 2022-06-29T15:51:39.613 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_temp": [], 2022-06-29T15:51:39.613 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_temp": [], 2022-06-29T15:51:39.613 INFO:teuthology.orchestra.run.smithi047.stdout: "blacklist": {}, 2022-06-29T15:51:39.614 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profiles": { 2022-06-29T15:51:39.614 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs_data": { 2022-06-29T15:51:39.614 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-device-class": "", 2022-06-29T15:51:39.614 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2022-06-29T15:51:39.614 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-root": "default", 2022-06-29T15:51:39.615 INFO:teuthology.orchestra.run.smithi047.stdout: "jerasure-per-chunk-alignment": "false", 2022-06-29T15:51:39.615 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2022-06-29T15:51:39.615 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "2", 2022-06-29T15:51:39.615 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2022-06-29T15:51:39.616 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2022-06-29T15:51:39.616 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van", 2022-06-29T15:51:39.616 INFO:teuthology.orchestra.run.smithi047.stdout: "w": "8" 2022-06-29T15:51:39.616 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.616 INFO:teuthology.orchestra.run.smithi047.stdout: "default": { 2022-06-29T15:51:39.617 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2022-06-29T15:51:39.617 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2022-06-29T15:51:39.617 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "1", 2022-06-29T15:51:39.617 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2022-06-29T15:51:39.617 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2022-06-29T15:51:39.618 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van" 2022-06-29T15:51:39.618 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:39.618 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:39.618 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps_queue": [], 2022-06-29T15:51:39.619 INFO:teuthology.orchestra.run.smithi047.stdout: "new_removed_snaps": [], 2022-06-29T15:51:39.619 INFO:teuthology.orchestra.run.smithi047.stdout: "new_purged_snaps": [], 2022-06-29T15:51:39.619 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_node_flags": {}, 2022-06-29T15:51:39.619 INFO:teuthology.orchestra.run.smithi047.stdout: "device_class_flags": {} 2022-06-29T15:51:39.619 INFO:teuthology.orchestra.run.smithi047.stdout:} 2022-06-29T15:51:39.622 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.362+0000 7ff773fff700 1 -- 172.21.15.47:0/1888707090 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff7580539e0 msgr2=0x7ff758055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:39.622 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.362+0000 7ff773fff700 1 --2- 172.21.15.47:0/1888707090 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff7580539e0 0x7ff758055ea0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7ff75c01a000 tx=0x7ff75c014000).stop 2022-06-29T15:51:39.622 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.362+0000 7ff773fff700 1 -- 172.21.15.47:0/1888707090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff774062910 msgr2=0x7ff7740b03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:39.622 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.362+0000 7ff773fff700 1 --2- 172.21.15.47:0/1888707090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff774062910 0x7ff7740b03d0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 rx=0x7ff764007000 tx=0x7ff764008040).stop 2022-06-29T15:51:39.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.362+0000 7ff773fff700 1 -- 172.21.15.47:0/1888707090 shutdown_connections 2022-06-29T15:51:39.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.362+0000 7ff773fff700 1 --2- 172.21.15.47:0/1888707090 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff7580539e0 0x7ff758055ea0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:39.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.362+0000 7ff773fff700 1 --2- 172.21.15.47:0/1888707090 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff774140700 0x7ff77413ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:39.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.362+0000 7ff773fff700 1 --2- 172.21.15.47:0/1888707090 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff7740b5670 0x7ff77413fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:39.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.362+0000 7ff773fff700 1 --2- 172.21.15.47:0/1888707090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff774062910 0x7ff7740b03d0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:39.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.362+0000 7ff773fff700 1 -- 172.21.15.47:0/1888707090 >> 172.21.15.47:0/1888707090 conn(0x7ff77405fce0 msgr2=0x7ff7741466f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:39.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.362+0000 7ff773fff700 1 -- 172.21.15.47:0/1888707090 shutdown_connections 2022-06-29T15:51:39.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.362+0000 7ff773fff700 1 -- 172.21.15.47:0/1888707090 wait complete. 2022-06-29T15:51:39.625 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:51:39.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.730+0000 7f5b67fff700 1 Processor -- start 2022-06-29T15:51:39.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.734+0000 7f5b67fff700 1 -- start start 2022-06-29T15:51:39.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.734+0000 7f5b67fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5b680b7000 0x7f5b680b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:39.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.734+0000 7f5b67fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5b68063000 0x7f5b680b29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:39.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.734+0000 7f5b67fff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b680b1b90 con 0x7f5b680b0f50 2022-06-29T15:51:39.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.734+0000 7f5b67fff700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b680b2ee0 con 0x7f5b680b7000 2022-06-29T15:51:39.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.734+0000 7f5b67fff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b680b73e0 con 0x7f5b68063000 2022-06-29T15:51:39.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.734+0000 7f5b66ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5b680b7000 0x7f5b680b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:39.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.734+0000 7f5b677fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5b68063000 0x7f5b680b29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:39.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.734+0000 7f5b66ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5b680b7000 0x7f5b680b2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54002/0 (socket says 172.21.15.47:54002) 2022-06-29T15:51:39.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.734+0000 7f5b66ffd700 1 -- 172.21.15.47:0/3988672803 learned_addr learned my addr 172.21.15.47:0/3988672803 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:39.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.734+0000 7f5b66ffd700 1 -- 172.21.15.47:0/3988672803 >> v2:172.21.15.64:3300/0 conn(0x7f5b68063000 msgr2=0x7f5b680b29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:39.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.734+0000 7f5b66ffd700 1 --2- 172.21.15.47:0/3988672803 >> v2:172.21.15.64:3300/0 conn(0x7f5b68063000 0x7f5b680b29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:39.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.734+0000 7f5b66ffd700 1 -- 172.21.15.47:0/3988672803 >> v1:172.21.15.64:6789/0 conn(0x7f5b680b0f50 legacy=0x7f5b6805fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:39.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.734+0000 7f5b66ffd700 1 -- 172.21.15.47:0/3988672803 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b68144390 con 0x7f5b680b7000 2022-06-29T15:51:39.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.734+0000 7f5b66ffd700 1 --2- 172.21.15.47:0/3988672803 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b680b7000 0x7f5b680b2500 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 rx=0x7f5b58002f70 tx=0x7f5b58006eb0).ready entity=mon.0 client_cookie=a1d05a2c646147e3 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:39.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.734+0000 7f5b65ffb700 1 -- 172.21.15.47:0/3988672803 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b5800d5c0 con 0x7f5b680b7000 2022-06-29T15:51:39.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.738+0000 7f5b65ffb700 1 -- 172.21.15.47:0/3988672803 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b58000e90 con 0x7f5b680b7000 2022-06-29T15:51:39.748 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.738+0000 7f5b65ffb700 1 -- 172.21.15.47:0/3988672803 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b5800d950 con 0x7f5b680b7000 2022-06-29T15:51:39.748 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.738+0000 7f5b67fff700 1 -- 172.21.15.47:0/3988672803 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b680b7000 msgr2=0x7f5b680b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:39.749 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.738+0000 7f5b67fff700 1 --2- 172.21.15.47:0/3988672803 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b680b7000 0x7f5b680b2500 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 rx=0x7f5b58002f70 tx=0x7f5b58006eb0).stop 2022-06-29T15:51:39.749 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.738+0000 7f5b67fff700 1 -- 172.21.15.47:0/3988672803 shutdown_connections 2022-06-29T15:51:39.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.738+0000 7f5b67fff700 1 --2- 172.21.15.47:0/3988672803 >> v2:172.21.15.64:3300/0 conn(0x7f5b68063000 0x7f5b680b29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:39.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.738+0000 7f5b67fff700 1 --2- 172.21.15.47:0/3988672803 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b680b7000 0x7f5b680b2500 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:39.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.738+0000 7f5b67fff700 1 -- 172.21.15.47:0/3988672803 >> 172.21.15.47:0/3988672803 conn(0x7f5b68061f00 msgr2=0x7f5b680b65b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:39.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.738+0000 7f5b67fff700 1 -- 172.21.15.47:0/3988672803 shutdown_connections 2022-06-29T15:51:39.752 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.738+0000 7f5b67fff700 1 -- 172.21.15.47:0/3988672803 wait complete. 2022-06-29T15:51:39.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.738+0000 7f5b67fff700 1 Processor -- start 2022-06-29T15:51:39.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.738+0000 7f5b67fff700 1 -- start start 2022-06-29T15:51:39.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.738+0000 7f5b67fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b68063000 0x7f5b6813c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:39.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.738+0000 7f5b67fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b680b0f50 0x7f5b68150290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:39.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.738+0000 7f5b67fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5b680b7000 0x7f5b680acc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:39.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.738+0000 7f5b67fff700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b680ae4e0 con 0x7f5b680b0f50 2022-06-29T15:51:39.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.738+0000 7f5b67fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6813cc80 con 0x7f5b68063000 2022-06-29T15:51:39.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.738+0000 7f5b67fff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b680ae750 con 0x7f5b680b7000 2022-06-29T15:51:39.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.738+0000 7f5b667fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b680b0f50 0x7f5b68150290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:39.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.742+0000 7f5b667fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b680b0f50 0x7f5b68150290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54010/0 (socket says 172.21.15.47:54010) 2022-06-29T15:51:39.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.742+0000 7f5b667fc700 1 -- 172.21.15.47:0/437775092 learned_addr learned my addr 172.21.15.47:0/437775092 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:39.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.742+0000 7f5b66ffd700 1 --2- 172.21.15.47:0/437775092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b68063000 0x7f5b6813c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:39.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.742+0000 7f5b677fe700 1 --2- 172.21.15.47:0/437775092 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5b680b7000 0x7f5b680acc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:39.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.742+0000 7f5b667fc700 1 -- 172.21.15.47:0/437775092 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5b680b7000 msgr2=0x7f5b680acc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:39.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.742+0000 7f5b667fc700 1 --2- 172.21.15.47:0/437775092 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5b680b7000 0x7f5b680acc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:39.761 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.742+0000 7f5b667fc700 1 -- 172.21.15.47:0/437775092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b68063000 msgr2=0x7f5b6813c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:39.761 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.742+0000 7f5b667fc700 1 --2- 172.21.15.47:0/437775092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b68063000 0x7f5b6813c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:39.761 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.742+0000 7f5b667fc700 1 -- 172.21.15.47:0/437775092 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b5800b1e0 con 0x7f5b680b0f50 2022-06-29T15:51:39.761 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.742+0000 7f5b667fc700 1 --2- 172.21.15.47:0/437775092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b680b0f50 0x7f5b68150290 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 rx=0x7f5b68068a10 tx=0x7f5b5c007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:39.762 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.742+0000 7f5b57fff700 1 -- 172.21.15.47:0/437775092 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b5c012400 con 0x7f5b680b0f50 2022-06-29T15:51:39.762 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.742+0000 7f5b57fff700 1 -- 172.21.15.47:0/437775092 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b5c012560 con 0x7f5b680b0f50 2022-06-29T15:51:39.762 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.742+0000 7f5b67fff700 1 -- 172.21.15.47:0/437775092 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b680ae9f0 con 0x7f5b680b0f50 2022-06-29T15:51:39.762 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.742+0000 7f5b57fff700 1 -- 172.21.15.47:0/437775092 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b5c0128c0 con 0x7f5b680b0f50 2022-06-29T15:51:39.763 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.742+0000 7f5b67fff700 1 -- 172.21.15.47:0/437775092 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b68150780 con 0x7f5b680b0f50 2022-06-29T15:51:39.763 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.742+0000 7f5b67fff700 1 -- 172.21.15.47:0/437775092 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b6813ce80 con 0x7f5b680b0f50 2022-06-29T15:51:39.763 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.742+0000 7f5b57fff700 1 -- 172.21.15.47:0/437775092 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5b5c019030 con 0x7f5b680b0f50 2022-06-29T15:51:39.764 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.746+0000 7f5b57fff700 1 --2- 172.21.15.47:0/437775092 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5b440539e0 0x7f5b44055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:39.764 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.746+0000 7f5b66ffd700 1 --2- 172.21.15.47:0/437775092 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5b440539e0 0x7f5b44055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:39.764 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.746+0000 7f5b57fff700 1 -- 172.21.15.47:0/437775092 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f5b5c06b5a0 con 0x7f5b680b0f50 2022-06-29T15:51:39.764 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.746+0000 7f5b66ffd700 1 --2- 172.21.15.47:0/437775092 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5b440539e0 0x7f5b44055ea0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7f5b5801c000 tx=0x7f5b58015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:39.765 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.750+0000 7f5b57fff700 1 -- 172.21.15.47:0/437775092 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5b5c0421d0 con 0x7f5b680b0f50 2022-06-29T15:51:39.919 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.914+0000 7f5b67fff700 1 -- 172.21.15.47:0/437775092 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5b680573d0 con 0x7f5b680b0f50 2022-06-29T15:51:39.920 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.914+0000 7f5b57fff700 1 -- 172.21.15.47:0/437775092 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+2492 (secure 0 0 0) 0x7f5b5c0457f0 con 0x7f5b680b0f50 2022-06-29T15:51:39.921 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:39.921 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":69,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7957,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.64:6833/298405638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":298405638},{"type":"v1","addr":"172.21.15.64:6833","nonce":298405638}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":66},{"gid":8037,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/191070564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":191070564},{"type":"v1","addr":"172.21.15.47:6835","nonce":191070564}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:51:23.543892+0000","modified":"2022-06-29T15:51:25.309120+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":56894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_56894":{"gid":56894,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6837/1370488030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1370488030},{"type":"v1","addr":"172.21.15.47:6837","nonce":1370488030}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2022-06-29T15:51:39.923 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.914+0000 7f5b67fff700 1 -- 172.21.15.47:0/437775092 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5b440539e0 msgr2=0x7f5b44055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:39.923 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.914+0000 7f5b67fff700 1 --2- 172.21.15.47:0/437775092 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5b440539e0 0x7f5b44055ea0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7f5b5801c000 tx=0x7f5b58015000).stop 2022-06-29T15:51:39.924 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.914+0000 7f5b67fff700 1 -- 172.21.15.47:0/437775092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b680b0f50 msgr2=0x7f5b68150290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:39.924 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.914+0000 7f5b67fff700 1 --2- 172.21.15.47:0/437775092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b680b0f50 0x7f5b68150290 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 rx=0x7f5b68068a10 tx=0x7f5b5c007c20).stop 2022-06-29T15:51:39.924 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.914+0000 7f5b67fff700 1 -- 172.21.15.47:0/437775092 shutdown_connections 2022-06-29T15:51:39.924 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.914+0000 7f5b67fff700 1 --2- 172.21.15.47:0/437775092 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5b680b7000 0x7f5b680acc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:39.925 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.914+0000 7f5b67fff700 1 --2- 172.21.15.47:0/437775092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b680b0f50 0x7f5b68150290 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:39.925 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.914+0000 7f5b67fff700 1 --2- 172.21.15.47:0/437775092 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5b440539e0 0x7f5b44055ea0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:39.925 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.914+0000 7f5b67fff700 1 --2- 172.21.15.47:0/437775092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b68063000 0x7f5b6813c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:39.926 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.914+0000 7f5b67fff700 1 -- 172.21.15.47:0/437775092 >> 172.21.15.47:0/437775092 conn(0x7f5b68061f00 msgr2=0x7f5b680b5be0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:39.926 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.914+0000 7f5b67fff700 1 -- 172.21.15.47:0/437775092 shutdown_connections 2022-06-29T15:51:39.926 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:39.914+0000 7f5b67fff700 1 -- 172.21.15.47:0/437775092 wait complete. 2022-06-29T15:51:39.927 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 69 2022-06-29T15:51:39.934 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-06-29T15:51:40.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.026+0000 7fc8d15e4700 1 Processor -- start 2022-06-29T15:51:40.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.026+0000 7fc8d15e4700 1 -- start start 2022-06-29T15:51:40.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.026+0000 7fc8d15e4700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc8cc0b7000 0x7fc8cc0b21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:40.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.026+0000 7fc8d15e4700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc8cc146290 0x7fc8cc0665f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:40.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.026+0000 7fc8d15e4700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8cc146700 con 0x7fc8cc0b2b30 2022-06-29T15:51:40.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.026+0000 7fc8d15e4700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8cc0b36f0 con 0x7fc8cc0b7000 2022-06-29T15:51:40.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.026+0000 7fc8d15e4700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8cc0b73e0 con 0x7fc8cc146290 2022-06-29T15:51:40.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.030+0000 7fc8cbfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc8cc0b7000 0x7fc8cc0b21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:40.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.030+0000 7fc8cbfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc8cc0b7000 0x7fc8cc0b21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54016/0 (socket says 172.21.15.47:54016) 2022-06-29T15:51:40.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.030+0000 7fc8cbfff700 1 -- 172.21.15.47:0/3027053917 learned_addr learned my addr 172.21.15.47:0/3027053917 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:40.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.030+0000 7fc8d0de3700 1 --2- 172.21.15.47:0/3027053917 >> v2:172.21.15.64:3300/0 conn(0x7fc8cc146290 0x7fc8cc0665f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:40.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.030+0000 7fc8cbfff700 1 -- 172.21.15.47:0/3027053917 >> v2:172.21.15.64:3300/0 conn(0x7fc8cc146290 msgr2=0x7fc8cc0665f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:40.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.030+0000 7fc8cbfff700 1 --2- 172.21.15.47:0/3027053917 >> v2:172.21.15.64:3300/0 conn(0x7fc8cc146290 0x7fc8cc0665f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:40.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.030+0000 7fc8cbfff700 1 -- 172.21.15.47:0/3027053917 >> v1:172.21.15.64:6789/0 conn(0x7fc8cc0b2b30 legacy=0x7fc8cc0b2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:40.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.030+0000 7fc8cb7fe700 1 -- 172.21.15.47:0/3027053917 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1135152794 0 0) 0x7fc8cc146700 con 0x7fc8cc0b2b30 2022-06-29T15:51:40.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.030+0000 7fc8cbfff700 1 -- 172.21.15.47:0/3027053917 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8cc0653a0 con 0x7fc8cc0b7000 2022-06-29T15:51:40.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.030+0000 7fc8cbfff700 1 --2- 172.21.15.47:0/3027053917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8cc0b7000 0x7fc8cc0b21c0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 rx=0x7fc8bc002f70 tx=0x7fc8bc006eb0).ready entity=mon.0 client_cookie=8c2c53b3b3a767b6 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:40.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.030+0000 7fc8cb7fe700 1 -- 172.21.15.47:0/3027053917 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc8bc00d5c0 con 0x7fc8cc0b7000 2022-06-29T15:51:40.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.030+0000 7fc8cb7fe700 1 -- 172.21.15.47:0/3027053917 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8bc000e90 con 0x7fc8cc0b7000 2022-06-29T15:51:40.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.030+0000 7fc8cb7fe700 1 -- 172.21.15.47:0/3027053917 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc8bc00d910 con 0x7fc8cc0b7000 2022-06-29T15:51:40.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.030+0000 7fc8d15e4700 1 -- 172.21.15.47:0/3027053917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8cc0b7000 msgr2=0x7fc8cc0b21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:40.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.030+0000 7fc8d15e4700 1 --2- 172.21.15.47:0/3027053917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8cc0b7000 0x7fc8cc0b21c0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 rx=0x7fc8bc002f70 tx=0x7fc8bc006eb0).stop 2022-06-29T15:51:40.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.030+0000 7fc8d15e4700 1 -- 172.21.15.47:0/3027053917 shutdown_connections 2022-06-29T15:51:40.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.030+0000 7fc8d15e4700 1 --2- 172.21.15.47:0/3027053917 >> v2:172.21.15.64:3300/0 conn(0x7fc8cc146290 0x7fc8cc0665f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:40.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.030+0000 7fc8d15e4700 1 --2- 172.21.15.47:0/3027053917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8cc0b7000 0x7fc8cc0b21c0 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:40.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.030+0000 7fc8d15e4700 1 -- 172.21.15.47:0/3027053917 >> 172.21.15.47:0/3027053917 conn(0x7fc8cc061f00 msgr2=0x7fc8cc0b65b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:40.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.030+0000 7fc8d15e4700 1 -- 172.21.15.47:0/3027053917 shutdown_connections 2022-06-29T15:51:40.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.030+0000 7fc8d15e4700 1 -- 172.21.15.47:0/3027053917 wait complete. 2022-06-29T15:51:40.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.034+0000 7fc8d15e4700 1 Processor -- start 2022-06-29T15:51:40.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.034+0000 7fc8d15e4700 1 -- start start 2022-06-29T15:51:40.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.034+0000 7fc8d15e4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8cc0b2b30 0x7fc8cc14a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:40.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.034+0000 7fc8d15e4700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc8cc0b7000 0x7fc8cc0a6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:40.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.034+0000 7fc8d15e4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8cc146290 0x7fc8cc1558c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:40.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.034+0000 7fc8d15e4700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8cc0a5fa0 con 0x7fc8cc0b2b30 2022-06-29T15:51:40.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.034+0000 7fc8d15e4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8cc0a51d0 con 0x7fc8cc146290 2022-06-29T15:51:40.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.034+0000 7fc8d15e4700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8cc0a6180 con 0x7fc8cc0b7000 2022-06-29T15:51:40.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.034+0000 7fc8cbfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8cc0b2b30 0x7fc8cc14a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:40.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.034+0000 7fc8cbfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8cc0b2b30 0x7fc8cc14a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54024/0 (socket says 172.21.15.47:54024) 2022-06-29T15:51:40.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.034+0000 7fc8cbfff700 1 -- 172.21.15.47:0/2420637799 learned_addr learned my addr 172.21.15.47:0/2420637799 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:40.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.034+0000 7fc8c3fff700 1 --2- 172.21.15.47:0/2420637799 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc8cc0b7000 0x7fc8cc0a6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:40.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.034+0000 7fc8d0de3700 1 --2- 172.21.15.47:0/2420637799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8cc146290 0x7fc8cc1558c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:40.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.034+0000 7fc8cbfff700 1 -- 172.21.15.47:0/2420637799 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc8cc0b7000 msgr2=0x7fc8cc0a6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:40.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.034+0000 7fc8cbfff700 1 --2- 172.21.15.47:0/2420637799 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc8cc0b7000 0x7fc8cc0a6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:40.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.034+0000 7fc8cbfff700 1 -- 172.21.15.47:0/2420637799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8cc146290 msgr2=0x7fc8cc1558c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:40.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.034+0000 7fc8cbfff700 1 --2- 172.21.15.47:0/2420637799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8cc146290 0x7fc8cc1558c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:40.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.034+0000 7fc8cbfff700 1 -- 172.21.15.47:0/2420637799 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8bc00b1e0 con 0x7fc8cc0b2b30 2022-06-29T15:51:40.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.034+0000 7fc8d0de3700 1 --2- 172.21.15.47:0/2420637799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8cc146290 0x7fc8cc1558c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:51:40.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.034+0000 7fc8cbfff700 1 --2- 172.21.15.47:0/2420637799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8cc0b2b30 0x7fc8cc14a6b0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 rx=0x7fc8bc002f70 tx=0x7fc8bc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:40.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.034+0000 7fc8c9ffb700 1 -- 172.21.15.47:0/2420637799 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc8bc011760 con 0x7fc8cc0b2b30 2022-06-29T15:51:40.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.034+0000 7fc8c9ffb700 1 -- 172.21.15.47:0/2420637799 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8bc0118c0 con 0x7fc8cc0b2b30 2022-06-29T15:51:40.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.034+0000 7fc8d15e4700 1 -- 172.21.15.47:0/2420637799 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8cc0a53c0 con 0x7fc8cc0b2b30 2022-06-29T15:51:40.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.034+0000 7fc8c9ffb700 1 -- 172.21.15.47:0/2420637799 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc8bc011c20 con 0x7fc8cc0b2b30 2022-06-29T15:51:40.052 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.034+0000 7fc8d15e4700 1 -- 172.21.15.47:0/2420637799 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8cc148f40 con 0x7fc8cc0b2b30 2022-06-29T15:51:40.052 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.038+0000 7fc8c9ffb700 1 -- 172.21.15.47:0/2420637799 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc8bc025030 con 0x7fc8cc0b2b30 2022-06-29T15:51:40.052 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.038+0000 7fc8d15e4700 1 -- 172.21.15.47:0/2420637799 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8cc0a5580 con 0x7fc8cc0b2b30 2022-06-29T15:51:40.052 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.038+0000 7fc8c9ffb700 1 --2- 172.21.15.47:0/2420637799 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc8b00539e0 0x7fc8b0055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:40.053 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.038+0000 7fc8c9ffb700 1 -- 172.21.15.47:0/2420637799 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc8bc06d250 con 0x7fc8cc0b2b30 2022-06-29T15:51:40.053 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.038+0000 7fc8c3fff700 1 --2- 172.21.15.47:0/2420637799 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc8b00539e0 0x7fc8b0055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:40.053 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.038+0000 7fc8c3fff700 1 --2- 172.21.15.47:0/2420637799 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc8b00539e0 0x7fc8b0055ea0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7fc8b400d000 tx=0x7fc8b4007a00).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:40.053 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.046+0000 7fc8c9ffb700 1 -- 172.21.15.47:0/2420637799 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc8bc043f00 con 0x7fc8cc0b2b30 2022-06-29T15:51:40.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:40.214+0000 7fc8d15e4700 1 -- 172.21.15.47:0/2420637799 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc8cc0573d0 con 0x7fc8cc0b2b30 2022-06-29T15:51:41.141 INFO:tasks.ceph.mon.a.smithi047.stderr:2022-06-29T15:51:41.134+0000 7ff9cf83c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-06-29T15:51:41.158 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.150+0000 7fc8c9ffb700 1 -- 172.21.15.47:0/2420637799 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v70) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc8bc023090 con 0x7fc8cc0b2b30 2022-06-29T15:51:41.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.154+0000 7fc8d15e4700 1 -- 172.21.15.47:0/2420637799 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc8b00539e0 msgr2=0x7fc8b0055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:41.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.154+0000 7fc8d15e4700 1 --2- 172.21.15.47:0/2420637799 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc8b00539e0 0x7fc8b0055ea0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7fc8b400d000 tx=0x7fc8b4007a00).stop 2022-06-29T15:51:41.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.154+0000 7fc8d15e4700 1 -- 172.21.15.47:0/2420637799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8cc0b2b30 msgr2=0x7fc8cc14a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:41.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.154+0000 7fc8d15e4700 1 --2- 172.21.15.47:0/2420637799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8cc0b2b30 0x7fc8cc14a6b0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 rx=0x7fc8bc002f70 tx=0x7fc8bc015000).stop 2022-06-29T15:51:41.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.154+0000 7fc8d15e4700 1 -- 172.21.15.47:0/2420637799 shutdown_connections 2022-06-29T15:51:41.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.154+0000 7fc8d15e4700 1 --2- 172.21.15.47:0/2420637799 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc8b00539e0 0x7fc8b0055ea0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:41.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.154+0000 7fc8d15e4700 1 --2- 172.21.15.47:0/2420637799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8cc146290 0x7fc8cc1558c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:41.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.154+0000 7fc8d15e4700 1 --2- 172.21.15.47:0/2420637799 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc8cc0b7000 0x7fc8cc0a6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:41.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.154+0000 7fc8d15e4700 1 --2- 172.21.15.47:0/2420637799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8cc0b2b30 0x7fc8cc14a6b0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:41.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.154+0000 7fc8d15e4700 1 -- 172.21.15.47:0/2420637799 >> 172.21.15.47:0/2420637799 conn(0x7fc8cc061f00 msgr2=0x7fc8cc0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:41.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.154+0000 7fc8d15e4700 1 -- 172.21.15.47:0/2420637799 shutdown_connections 2022-06-29T15:51:41.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.154+0000 7fc8d15e4700 1 -- 172.21.15.47:0/2420637799 wait complete. 2022-06-29T15:51:41.165 INFO:teuthology.orchestra.run.smithi047.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-06-29T15:51:41.179 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:51:41.189 INFO:tasks.ceph.mds.c.smithi047.stdout:starting mds.c at 2022-06-29T15:51:41.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.274+0000 7f9287fff700 1 Processor -- start 2022-06-29T15:51:41.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.274+0000 7f9287fff700 1 -- start start 2022-06-29T15:51:41.284 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.274+0000 7f9287fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f92880b2870 0x7f9288140a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:41.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.274+0000 7f9287fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92880ad890 0x7f92880a8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:41.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.274+0000 7f9287fff700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92880af880 con 0x7f92880b2110 2022-06-29T15:51:41.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.274+0000 7f9287fff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92880add20 con 0x7f92880b2870 2022-06-29T15:51:41.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.274+0000 7f9287fff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92880b0490 con 0x7f92880ad890 2022-06-29T15:51:41.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.274+0000 7f92867fc700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f92880b2110 0x7f92880ad180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39596/0 (socket says 172.21.15.47:39596) 2022-06-29T15:51:41.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.274+0000 7f92867fc700 1 -- 172.21.15.47:0/1428000014 learned_addr learned my addr 172.21.15.47:0/1428000014 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:41.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.274+0000 7f92877fe700 1 --2- 172.21.15.47:0/1428000014 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92880ad890 0x7f92880a8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:41.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.274+0000 7f9286ffd700 1 --2- 172.21.15.47:0/1428000014 >> v2:172.21.15.64:3300/0 conn(0x7f92880b2870 0x7f9288140a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:41.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.274+0000 7f9285ffb700 1 -- 172.21.15.47:0/1428000014 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3453453684 0 0) 0x7f92880af880 con 0x7f92880b2110 2022-06-29T15:51:41.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.274+0000 7f9285ffb700 1 -- 172.21.15.47:0/1428000014 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f926c003260 con 0x7f92880b2110 2022-06-29T15:51:41.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.274+0000 7f9285ffb700 1 -- 172.21.15.47:0/1428000014 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3128559669 0 0) 0x7f926c003260 con 0x7f92880b2110 2022-06-29T15:51:41.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.274+0000 7f9285ffb700 1 -- 172.21.15.47:0/1428000014 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92880ad890 msgr2=0x7f92880a8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:41.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.274+0000 7f9285ffb700 1 --2- 172.21.15.47:0/1428000014 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92880ad890 0x7f92880a8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:41.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.274+0000 7f9285ffb700 1 -- 172.21.15.47:0/1428000014 >> v2:172.21.15.64:3300/0 conn(0x7f92880b2870 msgr2=0x7f9288140a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:41.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.274+0000 7f9285ffb700 1 --2- 172.21.15.47:0/1428000014 >> v2:172.21.15.64:3300/0 conn(0x7f92880b2870 0x7f9288140a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:41.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.274+0000 7f9285ffb700 1 -- 172.21.15.47:0/1428000014 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92880b2c50 con 0x7f92880b2110 2022-06-29T15:51:41.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.274+0000 7f9285ffb700 1 -- 172.21.15.47:0/1428000014 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7f927c001660 con 0x7f92880b2110 2022-06-29T15:51:41.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.274+0000 7f9285ffb700 1 -- 172.21.15.47:0/1428000014 >> v1:172.21.15.47:6789/0 conn(0x7f92880b2110 legacy=0x7f92880ad180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:41.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.274+0000 7f9285ffb700 1 --2- 172.21.15.47:0/1428000014 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92880ad890 0x7f926c003780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:41.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.274+0000 7f9285ffb700 1 --2- 172.21.15.47:0/1428000014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f926c003cb0 0x7f926c004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:41.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.274+0000 7f9285ffb700 1 --2- 172.21.15.47:0/1428000014 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f926c0049f0 0x7f926c004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:41.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.274+0000 7f9285ffb700 1 -- 172.21.15.47:0/1428000014 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92880b0490 con 0x7f926c0049f0 2022-06-29T15:51:41.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.274+0000 7f9285ffb700 1 -- 172.21.15.47:0/1428000014 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926c00b750 con 0x7f926c003cb0 2022-06-29T15:51:41.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.274+0000 7f9285ffb700 1 -- 172.21.15.47:0/1428000014 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f926c00b890 con 0x7f92880ad890 2022-06-29T15:51:41.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.274+0000 7f9286ffd700 1 --2- 172.21.15.47:0/1428000014 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92880ad890 0x7f926c003780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:41.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.274+0000 7f92877fe700 1 --2- 172.21.15.47:0/1428000014 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f926c0049f0 0x7f926c004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:41.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.274+0000 7f92867fc700 1 --2- 172.21.15.47:0/1428000014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f926c003cb0 0x7f926c004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:41.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.278+0000 7f92877fe700 1 -- 172.21.15.47:0/1428000014 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92880ad890 msgr2=0x7f926c003780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:41.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.278+0000 7f92877fe700 1 --2- 172.21.15.47:0/1428000014 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92880ad890 0x7f926c003780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:41.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.278+0000 7f92877fe700 1 -- 172.21.15.47:0/1428000014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f926c003cb0 msgr2=0x7f926c004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:41.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.278+0000 7f92877fe700 1 --2- 172.21.15.47:0/1428000014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f926c003cb0 0x7f926c004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:41.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.278+0000 7f92877fe700 1 -- 172.21.15.47:0/1428000014 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f926c003260 con 0x7f926c0049f0 2022-06-29T15:51:41.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.278+0000 7f92877fe700 1 --2- 172.21.15.47:0/1428000014 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f926c0049f0 0x7f926c004e70 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 rx=0x7f926c004470 tx=0x7f92800091d0).ready entity=mon.0 client_cookie=bcf60e5e887d51d5 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:41.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.278+0000 7f9285ffb700 1 -- 172.21.15.47:0/1428000014 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f928000e7e0 con 0x7f926c0049f0 2022-06-29T15:51:41.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.278+0000 7f9285ffb700 1 -- 172.21.15.47:0/1428000014 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f928000e940 con 0x7f926c0049f0 2022-06-29T15:51:41.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.278+0000 7f9285ffb700 1 -- 172.21.15.47:0/1428000014 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f928000eca0 con 0x7f926c0049f0 2022-06-29T15:51:41.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.278+0000 7f9287fff700 1 -- 172.21.15.47:0/1428000014 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f926c0049f0 msgr2=0x7f926c004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:41.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.278+0000 7f9287fff700 1 --2- 172.21.15.47:0/1428000014 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f926c0049f0 0x7f926c004e70 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 rx=0x7f926c004470 tx=0x7f92800091d0).stop 2022-06-29T15:51:41.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.278+0000 7f92877fe700 1 -- 172.21.15.47:0/1428000014 reap_dead start 2022-06-29T15:51:41.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.278+0000 7f9287fff700 1 -- 172.21.15.47:0/1428000014 shutdown_connections 2022-06-29T15:51:41.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.278+0000 7f9287fff700 1 -- 172.21.15.47:0/1428000014 >> 172.21.15.47:0/1428000014 conn(0x7f9288062d90 msgr2=0x7f9288063170 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:41.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.278+0000 7f9287fff700 1 -- 172.21.15.47:0/1428000014 shutdown_connections 2022-06-29T15:51:41.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.278+0000 7f9287fff700 1 -- 172.21.15.47:0/1428000014 wait complete. 2022-06-29T15:51:41.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.278+0000 7f9287fff700 1 Processor -- start 2022-06-29T15:51:41.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.278+0000 7f9287fff700 1 -- start start 2022-06-29T15:51:41.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.278+0000 7f9287fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f926c0049f0 0x7f92880641e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:41.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.278+0000 7f9287fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92880ac7a0 0x7f928813d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:41.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.278+0000 7f9287fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92880b3940 0x7f928813d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:41.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.278+0000 7f9287fff700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9288064a20 con 0x7f92880b3940 2022-06-29T15:51:41.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.278+0000 7f9287fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92880b4a50 con 0x7f92880ac7a0 2022-06-29T15:51:41.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.278+0000 7f9287fff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92880b4380 con 0x7f926c0049f0 2022-06-29T15:51:41.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.278+0000 7f92877fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92880b3940 0x7f928813d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:41.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.278+0000 7f92877fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92880b3940 0x7f928813d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54058/0 (socket says 172.21.15.47:54058) 2022-06-29T15:51:41.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.278+0000 7f92877fe700 1 -- 172.21.15.47:0/727786779 learned_addr learned my addr 172.21.15.47:0/727786779 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:41.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.278+0000 7f92867fc700 1 --2- 172.21.15.47:0/727786779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92880ac7a0 0x7f928813d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:41.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.278+0000 7f9286ffd700 1 --2- 172.21.15.47:0/727786779 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f926c0049f0 0x7f92880641e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:41.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.282+0000 7f92877fe700 1 -- 172.21.15.47:0/727786779 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f926c0049f0 msgr2=0x7f92880641e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:41.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.282+0000 7f92877fe700 1 --2- 172.21.15.47:0/727786779 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f926c0049f0 0x7f92880641e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:41.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.282+0000 7f92877fe700 1 -- 172.21.15.47:0/727786779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92880ac7a0 msgr2=0x7f928813d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:41.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.282+0000 7f92877fe700 1 --2- 172.21.15.47:0/727786779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92880ac7a0 0x7f928813d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:41.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.282+0000 7f92877fe700 1 -- 172.21.15.47:0/727786779 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f928813da20 con 0x7f92880b3940 2022-06-29T15:51:41.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.282+0000 7f92877fe700 1 --2- 172.21.15.47:0/727786779 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92880b3940 0x7f928813d530 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 rx=0x7f92880acc00 tx=0x7f928001a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:41.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.282+0000 7f9277fff700 1 -- 172.21.15.47:0/727786779 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f928001dcf0 con 0x7f92880b3940 2022-06-29T15:51:41.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.282+0000 7f9277fff700 1 -- 172.21.15.47:0/727786779 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f928001de50 con 0x7f92880b3940 2022-06-29T15:51:41.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.282+0000 7f9287fff700 1 -- 172.21.15.47:0/727786779 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f928814e710 con 0x7f92880b3940 2022-06-29T15:51:41.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.282+0000 7f9277fff700 1 -- 172.21.15.47:0/727786779 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f928001b640 con 0x7f92880b3940 2022-06-29T15:51:41.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.282+0000 7f9287fff700 1 -- 172.21.15.47:0/727786779 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92880b45b0 con 0x7f92880b3940 2022-06-29T15:51:41.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.282+0000 7f9287fff700 1 -- 172.21.15.47:0/727786779 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92880573d0 con 0x7f92880b3940 2022-06-29T15:51:41.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.282+0000 7f9277fff700 1 -- 172.21.15.47:0/727786779 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f928002d070 con 0x7f92880b3940 2022-06-29T15:51:41.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.282+0000 7f9277fff700 1 --2- 172.21.15.47:0/727786779 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9264053980 0x7f9264055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:41.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.282+0000 7f9277fff700 1 -- 172.21.15.47:0/727786779 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f928006de60 con 0x7f92880b3940 2022-06-29T15:51:41.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.282+0000 7f9286ffd700 1 --2- 172.21.15.47:0/727786779 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9264053980 0x7f9264055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:41.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.282+0000 7f9286ffd700 1 --2- 172.21.15.47:0/727786779 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9264053980 0x7f9264055e40 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7f927800b000 tx=0x7f9278006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:41.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.290+0000 7f9277fff700 1 -- 172.21.15.47:0/727786779 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9280044c90 con 0x7f92880b3940 2022-06-29T15:51:41.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.450+0000 7f9287fff700 1 -- 172.21.15.47:0/727786779 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f928806a210 con 0x7f92880b3940 2022-06-29T15:51:41.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.454+0000 7f9277fff700 1 -- 172.21.15.47:0/727786779 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+2129 (secure 0 0 0) 0x7f9280016070 con 0x7f92880b3940 2022-06-29T15:51:41.460 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:41.460 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":70,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7957,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.64:6833/298405638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":298405638},{"type":"v1","addr":"172.21.15.64:6833","nonce":298405638}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":66},{"gid":8037,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/191070564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":191070564},{"type":"v1","addr":"172.21.15.47:6835","nonce":191070564}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65}],"filesystems":[{"mdsmap":{"epoch":70,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:51:23.543892+0000","modified":"2022-06-29T15:51:41.137822+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":141,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2022-06-29T15:51:41.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.454+0000 7f9287fff700 1 -- 172.21.15.47:0/727786779 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9264053980 msgr2=0x7f9264055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:41.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.454+0000 7f9287fff700 1 --2- 172.21.15.47:0/727786779 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9264053980 0x7f9264055e40 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7f927800b000 tx=0x7f9278006df0).stop 2022-06-29T15:51:41.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.454+0000 7f9287fff700 1 -- 172.21.15.47:0/727786779 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92880b3940 msgr2=0x7f928813d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:41.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.454+0000 7f9287fff700 1 --2- 172.21.15.47:0/727786779 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92880b3940 0x7f928813d530 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 rx=0x7f92880acc00 tx=0x7f928001a000).stop 2022-06-29T15:51:41.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.454+0000 7f9287fff700 1 -- 172.21.15.47:0/727786779 shutdown_connections 2022-06-29T15:51:41.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.454+0000 7f9287fff700 1 --2- 172.21.15.47:0/727786779 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92880b3940 0x7f928813d530 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:41.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.454+0000 7f9287fff700 1 --2- 172.21.15.47:0/727786779 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9264053980 0x7f9264055e40 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:41.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.454+0000 7f9287fff700 1 --2- 172.21.15.47:0/727786779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92880ac7a0 0x7f928813d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:41.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.454+0000 7f9287fff700 1 --2- 172.21.15.47:0/727786779 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f926c0049f0 0x7f92880641e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:41.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.454+0000 7f9287fff700 1 -- 172.21.15.47:0/727786779 >> 172.21.15.47:0/727786779 conn(0x7f9288062d90 msgr2=0x7f9288144640 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:41.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.454+0000 7f9287fff700 1 -- 172.21.15.47:0/727786779 shutdown_connections 2022-06-29T15:51:41.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.454+0000 7f9287fff700 1 -- 172.21.15.47:0/727786779 wait complete. 2022-06-29T15:51:41.465 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 70 2022-06-29T15:51:41.473 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-06-29T15:51:41.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.618+0000 7f28e4b50700 1 Processor -- start 2022-06-29T15:51:41.625 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.618+0000 7f28e4b50700 1 -- start start 2022-06-29T15:51:41.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.618+0000 7f28e4b50700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f28e0062910 0x7f28e00b62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:41.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.618+0000 7f28e4b50700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f28e00b5d80 0x7f28e0140b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:41.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.618+0000 7f28e4b50700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28e00b5240 con 0x7f28e0140700 2022-06-29T15:51:41.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.618+0000 7f28e4b50700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28e0062cf0 con 0x7f28e0062910 2022-06-29T15:51:41.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.618+0000 7f28e4b50700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28e0141010 con 0x7f28e00b5d80 2022-06-29T15:51:41.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.618+0000 7f28deffd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f28e0140700 0x7f28e00b5670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39616/0 (socket says 172.21.15.47:39616) 2022-06-29T15:51:41.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.618+0000 7f28deffd700 1 -- 172.21.15.47:0/3165313769 learned_addr learned my addr 172.21.15.47:0/3165313769 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:41.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.618+0000 7f28dffff700 1 --2- 172.21.15.47:0/3165313769 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f28e00b5d80 0x7f28e0140b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:41.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.618+0000 7f28df7fe700 1 --2- 172.21.15.47:0/3165313769 >> v2:172.21.15.64:3300/0 conn(0x7f28e0062910 0x7f28e00b62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:41.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.618+0000 7f28de7fc700 1 -- 172.21.15.47:0/3165313769 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 874490189 0 0) 0x7f28e00b5240 con 0x7f28e0140700 2022-06-29T15:51:41.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.618+0000 7f28de7fc700 1 -- 172.21.15.47:0/3165313769 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28c0003260 con 0x7f28e0140700 2022-06-29T15:51:41.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.622+0000 7f28dffff700 1 -- 172.21.15.47:0/3165313769 >> v2:172.21.15.64:3300/0 conn(0x7f28e0062910 msgr2=0x7f28e00b62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:41.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.622+0000 7f28dffff700 1 --2- 172.21.15.47:0/3165313769 >> v2:172.21.15.64:3300/0 conn(0x7f28e0062910 0x7f28e00b62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:41.631 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.622+0000 7f28dffff700 1 -- 172.21.15.47:0/3165313769 >> v1:172.21.15.47:6789/0 conn(0x7f28e0140700 legacy=0x7f28e00b5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:41.631 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.622+0000 7f28de7fc700 1 -- 172.21.15.47:0/3165313769 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1534917874 0 0) 0x7f28c0003260 con 0x7f28e0140700 2022-06-29T15:51:41.631 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.622+0000 7f28dffff700 1 -- 172.21.15.47:0/3165313769 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e00b2390 con 0x7f28e00b5d80 2022-06-29T15:51:41.631 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.622+0000 7f28dffff700 1 --2- 172.21.15.47:0/3165313769 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f28e00b5d80 0x7f28e0140b20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f28c8001800 tx=0x7f28d80091d0).ready entity=mon.2 client_cookie=acaea8364a92ffed server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:41.632 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.622+0000 7f28de7fc700 1 -- 172.21.15.47:0/3165313769 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28c8001660 con 0x7f28e00b5d80 2022-06-29T15:51:41.632 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.622+0000 7f28de7fc700 1 -- 172.21.15.47:0/3165313769 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28d800f510 con 0x7f28e00b5d80 2022-06-29T15:51:41.632 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.622+0000 7f28de7fc700 1 -- 172.21.15.47:0/3165313769 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28d800f680 con 0x7f28e00b5d80 2022-06-29T15:51:41.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.622+0000 7f28e4b50700 1 -- 172.21.15.47:0/3165313769 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f28e00b5d80 msgr2=0x7f28e0140b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:41.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.622+0000 7f28e4b50700 1 --2- 172.21.15.47:0/3165313769 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f28e00b5d80 0x7f28e0140b20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f28c8001800 tx=0x7f28d80091d0).stop 2022-06-29T15:51:41.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.622+0000 7f28e4b50700 1 -- 172.21.15.47:0/3165313769 shutdown_connections 2022-06-29T15:51:41.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.622+0000 7f28e4b50700 1 --2- 172.21.15.47:0/3165313769 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f28e00b5d80 0x7f28e0140b20 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:41.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.622+0000 7f28e4b50700 1 --2- 172.21.15.47:0/3165313769 >> v2:172.21.15.64:3300/0 conn(0x7f28e0062910 0x7f28e00b62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:41.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.622+0000 7f28e4b50700 1 -- 172.21.15.47:0/3165313769 >> 172.21.15.47:0/3165313769 conn(0x7f28e005fce0 msgr2=0x7f28e00600c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:41.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.622+0000 7f28e4b50700 1 -- 172.21.15.47:0/3165313769 shutdown_connections 2022-06-29T15:51:41.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.622+0000 7f28e4b50700 1 -- 172.21.15.47:0/3165313769 wait complete. 2022-06-29T15:51:41.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.622+0000 7f28e4b50700 1 Processor -- start 2022-06-29T15:51:41.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.622+0000 7f28e4b50700 1 -- start start 2022-06-29T15:51:41.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.626+0000 7f28e4b50700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28e0062910 0x7f28e00afb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:41.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.626+0000 7f28e4b50700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28e00b5d80 0x7f28e0152460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:41.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.626+0000 7f28e4b50700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f28e0140700 0x7f28e0152950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:41.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.626+0000 7f28e4b50700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e00ac0b0 con 0x7f28e00b5d80 2022-06-29T15:51:41.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.626+0000 7f28e4b50700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e0145da0 con 0x7f28e0062910 2022-06-29T15:51:41.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.626+0000 7f28e4b50700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28e0146040 con 0x7f28e0140700 2022-06-29T15:51:41.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.626+0000 7f28deffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28e00b5d80 0x7f28e0152460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:41.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.626+0000 7f28deffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28e00b5d80 0x7f28e0152460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54070/0 (socket says 172.21.15.47:54070) 2022-06-29T15:51:41.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.626+0000 7f28deffd700 1 -- 172.21.15.47:0/1637261589 learned_addr learned my addr 172.21.15.47:0/1637261589 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:41.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.626+0000 7f28dffff700 1 --2- 172.21.15.47:0/1637261589 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f28e0140700 0x7f28e0152950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:41.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.626+0000 7f28df7fe700 1 --2- 172.21.15.47:0/1637261589 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28e0062910 0x7f28e00afb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:41.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.626+0000 7f28deffd700 1 -- 172.21.15.47:0/1637261589 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f28e0140700 msgr2=0x7f28e0152950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:41.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.626+0000 7f28deffd700 1 --2- 172.21.15.47:0/1637261589 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f28e0140700 0x7f28e0152950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:41.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.626+0000 7f28deffd700 1 -- 172.21.15.47:0/1637261589 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28e0062910 msgr2=0x7f28e00afb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:41.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.626+0000 7f28deffd700 1 --2- 172.21.15.47:0/1637261589 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28e0062910 0x7f28e00afb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:41.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.626+0000 7f28deffd700 1 -- 172.21.15.47:0/1637261589 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d8009470 con 0x7f28e00b5d80 2022-06-29T15:51:41.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.626+0000 7f28dffff700 1 --2- 172.21.15.47:0/1637261589 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f28e0140700 0x7f28e0152950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:51:41.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.626+0000 7f28deffd700 1 --2- 172.21.15.47:0/1637261589 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28e00b5d80 0x7f28e0152460 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 rx=0x7f28c80018c0 tx=0x7f28c80011a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:41.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.626+0000 7f28dcff9700 1 -- 172.21.15.47:0/1637261589 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28c8007170 con 0x7f28e00b5d80 2022-06-29T15:51:41.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.626+0000 7f28dcff9700 1 -- 172.21.15.47:0/1637261589 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c80072d0 con 0x7f28e00b5d80 2022-06-29T15:51:41.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.626+0000 7f28e4b50700 1 -- 172.21.15.47:0/1637261589 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28e01462e0 con 0x7f28e00b5d80 2022-06-29T15:51:41.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.626+0000 7f28dcff9700 1 -- 172.21.15.47:0/1637261589 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28c8004920 con 0x7f28e00b5d80 2022-06-29T15:51:41.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.626+0000 7f28e4b50700 1 -- 172.21.15.47:0/1637261589 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28e0146780 con 0x7f28e00b5d80 2022-06-29T15:51:41.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.630+0000 7f28dcff9700 1 -- 172.21.15.47:0/1637261589 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f28c8004a80 con 0x7f28e00b5d80 2022-06-29T15:51:41.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.630+0000 7f28e4b50700 1 -- 172.21.15.47:0/1637261589 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28e0146470 con 0x7f28e00b5d80 2022-06-29T15:51:41.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.630+0000 7f28dcff9700 1 --2- 172.21.15.47:0/1637261589 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f28c40539e0 0x7f28c4055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:41.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.630+0000 7f28dcff9700 1 -- 172.21.15.47:0/1637261589 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f28c806be10 con 0x7f28e00b5d80 2022-06-29T15:51:41.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.630+0000 7f28df7fe700 1 --2- 172.21.15.47:0/1637261589 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f28c40539e0 0x7f28c4055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:41.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.630+0000 7f28df7fe700 1 --2- 172.21.15.47:0/1637261589 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f28c40539e0 0x7f28c4055ea0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7f28d000b000 tx=0x7f28d0006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:41.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.634+0000 7f28dcff9700 1 -- 172.21.15.47:0/1637261589 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f28c806f030 con 0x7f28e00b5d80 2022-06-29T15:51:41.803 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:41.798+0000 7f28e4b50700 1 -- 172.21.15.47:0/1637261589 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f28e00b0fe0 con 0x7f28e00b5d80 2022-06-29T15:51:42.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.154+0000 7f28dcff9700 1 -- 172.21.15.47:0/1637261589 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v71) v1 ==== 104+0+0 (secure 0 0 0) 0x7f28c8042a70 con 0x7f28e00b5d80 2022-06-29T15:51:42.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.158+0000 7f28e4b50700 1 -- 172.21.15.47:0/1637261589 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f28c40539e0 msgr2=0x7f28c4055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:42.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.158+0000 7f28e4b50700 1 --2- 172.21.15.47:0/1637261589 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f28c40539e0 0x7f28c4055ea0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7f28d000b000 tx=0x7f28d0006df0).stop 2022-06-29T15:51:42.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.158+0000 7f28e4b50700 1 -- 172.21.15.47:0/1637261589 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28e00b5d80 msgr2=0x7f28e0152460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:42.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.158+0000 7f28e4b50700 1 --2- 172.21.15.47:0/1637261589 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28e00b5d80 0x7f28e0152460 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 rx=0x7f28c80018c0 tx=0x7f28c80011a0).stop 2022-06-29T15:51:42.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.158+0000 7f28e4b50700 1 -- 172.21.15.47:0/1637261589 shutdown_connections 2022-06-29T15:51:42.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.158+0000 7f28e4b50700 1 --2- 172.21.15.47:0/1637261589 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f28e0140700 0x7f28e0152950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:42.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.158+0000 7f28e4b50700 1 --2- 172.21.15.47:0/1637261589 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28e00b5d80 0x7f28e0152460 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:42.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.158+0000 7f28e4b50700 1 --2- 172.21.15.47:0/1637261589 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f28c40539e0 0x7f28c4055ea0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:42.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.158+0000 7f28e4b50700 1 --2- 172.21.15.47:0/1637261589 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28e0062910 0x7f28e00afb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:42.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.158+0000 7f28e4b50700 1 -- 172.21.15.47:0/1637261589 >> 172.21.15.47:0/1637261589 conn(0x7f28e005fce0 msgr2=0x7f28e0141200 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:42.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.158+0000 7f28e4b50700 1 -- 172.21.15.47:0/1637261589 shutdown_connections 2022-06-29T15:51:42.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.158+0000 7f28e4b50700 1 -- 172.21.15.47:0/1637261589 wait complete. 2022-06-29T15:51:42.183 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-06-29T15:51:42.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.290+0000 7f98fe192700 1 Processor -- start 2022-06-29T15:51:42.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.290+0000 7f98fe192700 1 -- start start 2022-06-29T15:51:42.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.294+0000 7f98fe192700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f98f80b59c0 0x7f98f80b4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:42.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.294+0000 7f98fe192700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98f80b9f60 con 0x7f98f80b9b80 2022-06-29T15:51:42.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.294+0000 7f98fe192700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98f80b7070 con 0x7f98f8140570 2022-06-29T15:51:42.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.294+0000 7f98fe192700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98f81433d0 con 0x7f98f80b59c0 2022-06-29T15:51:42.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.294+0000 7f98fd190700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f98f80b9b80 0x7f98f8143b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39632/0 (socket says 172.21.15.47:39632) 2022-06-29T15:51:42.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.294+0000 7f98fd190700 1 -- 172.21.15.47:0/1473335279 learned_addr learned my addr 172.21.15.47:0/1473335279 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:42.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.294+0000 7f98fc98f700 1 --2- 172.21.15.47:0/1473335279 >> v2:172.21.15.64:3300/0 conn(0x7f98f80b59c0 0x7f98f80b4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:42.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.294+0000 7f98ebfff700 1 -- 172.21.15.47:0/1473335279 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2359793231 0 0) 0x7f98f80b9f60 con 0x7f98f80b9b80 2022-06-29T15:51:42.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.294+0000 7f98ebfff700 1 -- 172.21.15.47:0/1473335279 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98dc003260 con 0x7f98f80b9b80 2022-06-29T15:51:42.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.294+0000 7f98fc98f700 1 -- 172.21.15.47:0/1473335279 >> v1:172.21.15.64:6789/0 conn(0x7f98f8140570 legacy=0x7f98f814da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:42.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.294+0000 7f98ebfff700 1 -- 172.21.15.47:0/1473335279 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1602121150 0 0) 0x7f98f80b7070 con 0x7f98f8140570 2022-06-29T15:51:42.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.294+0000 7f98fc98f700 1 -- 172.21.15.47:0/1473335279 >> v1:172.21.15.47:6789/0 conn(0x7f98f80b9b80 legacy=0x7f98f8143b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:42.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.294+0000 7f98fc98f700 1 -- 172.21.15.47:0/1473335279 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f8143670 con 0x7f98f80b59c0 2022-06-29T15:51:42.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.294+0000 7f98fc98f700 1 --2- 172.21.15.47:0/1473335279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98f80b59c0 0x7f98f80b4500 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f98ec002f70 tx=0x7f98ec006df0).ready entity=mon.1 client_cookie=6dbf03cef82bbfa7 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:42.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.294+0000 7f98ebfff700 1 -- 172.21.15.47:0/1473335279 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f98ec00d610 con 0x7f98f80b59c0 2022-06-29T15:51:42.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.294+0000 7f98ebfff700 1 -- 172.21.15.47:0/1473335279 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98ec000e90 con 0x7f98f80b59c0 2022-06-29T15:51:42.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.294+0000 7f98ebfff700 1 -- 172.21.15.47:0/1473335279 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f98ec00d960 con 0x7f98f80b59c0 2022-06-29T15:51:42.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.294+0000 7f98fe192700 1 -- 172.21.15.47:0/1473335279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98f80b59c0 msgr2=0x7f98f80b4500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:42.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.294+0000 7f98fe192700 1 --2- 172.21.15.47:0/1473335279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98f80b59c0 0x7f98f80b4500 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f98ec002f70 tx=0x7f98ec006df0).stop 2022-06-29T15:51:42.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.294+0000 7f98fe192700 1 -- 172.21.15.47:0/1473335279 shutdown_connections 2022-06-29T15:51:42.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.294+0000 7f98fe192700 1 --2- 172.21.15.47:0/1473335279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98f80b59c0 0x7f98f80b4500 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:42.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.294+0000 7f98fe192700 1 -- 172.21.15.47:0/1473335279 >> 172.21.15.47:0/1473335279 conn(0x7f98f8062650 msgr2=0x7f98f8062a60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:42.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.294+0000 7f98fe192700 1 -- 172.21.15.47:0/1473335279 shutdown_connections 2022-06-29T15:51:42.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.298+0000 7f98fe192700 1 -- 172.21.15.47:0/1473335279 wait complete. 2022-06-29T15:51:42.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.298+0000 7f98fe192700 1 Processor -- start 2022-06-29T15:51:42.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.298+0000 7f98fe192700 1 -- start start 2022-06-29T15:51:42.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.298+0000 7f98fe192700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98f80b59c0 0x7f98f8139170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:42.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.298+0000 7f98fe192700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f98f80b9b80 0x7f98f80ae420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:42.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.298+0000 7f98fe192700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98f8140570 0x7f98f80ae910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:42.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.298+0000 7f98fe192700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f80aee00 con 0x7f98f8140570 2022-06-29T15:51:42.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.298+0000 7f98fe192700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f813f830 con 0x7f98f80b59c0 2022-06-29T15:51:42.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.298+0000 7f98fe192700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98f8140b20 con 0x7f98f80b9b80 2022-06-29T15:51:42.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.298+0000 7f98fd991700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98f8140570 0x7f98f80ae910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:42.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.298+0000 7f98fd991700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98f8140570 0x7f98f80ae910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54088/0 (socket says 172.21.15.47:54088) 2022-06-29T15:51:42.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.298+0000 7f98fd991700 1 -- 172.21.15.47:0/1409763407 learned_addr learned my addr 172.21.15.47:0/1409763407 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:42.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.298+0000 7f98fc98f700 1 --2- 172.21.15.47:0/1409763407 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f98f80b9b80 0x7f98f80ae420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:42.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.298+0000 7f98fd190700 1 --2- 172.21.15.47:0/1409763407 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98f80b59c0 0x7f98f8139170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:42.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.298+0000 7f98fd991700 1 -- 172.21.15.47:0/1409763407 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f98f80b9b80 msgr2=0x7f98f80ae420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:42.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.298+0000 7f98fd991700 1 --2- 172.21.15.47:0/1409763407 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f98f80b9b80 0x7f98f80ae420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:42.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.298+0000 7f98fd991700 1 -- 172.21.15.47:0/1409763407 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98f80b59c0 msgr2=0x7f98f8139170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:42.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.298+0000 7f98fd991700 1 --2- 172.21.15.47:0/1409763407 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98f80b59c0 0x7f98f8139170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:42.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.298+0000 7f98fd991700 1 -- 172.21.15.47:0/1409763407 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98ec00b180 con 0x7f98f8140570 2022-06-29T15:51:42.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.298+0000 7f98fd991700 1 --2- 172.21.15.47:0/1409763407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98f8140570 0x7f98f80ae910 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 rx=0x7f98f8144790 tx=0x7f98f000c800).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:42.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.302+0000 7f98ea7fc700 1 -- 172.21.15.47:0/1409763407 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f98f0006aa0 con 0x7f98f8140570 2022-06-29T15:51:42.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.302+0000 7f98ea7fc700 1 -- 172.21.15.47:0/1409763407 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f0006c00 con 0x7f98f8140570 2022-06-29T15:51:42.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.302+0000 7f98fe192700 1 -- 172.21.15.47:0/1409763407 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98f813f000 con 0x7f98f8140570 2022-06-29T15:51:42.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.302+0000 7f98ea7fc700 1 -- 172.21.15.47:0/1409763407 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f98f0007030 con 0x7f98f8140570 2022-06-29T15:51:42.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.302+0000 7f98fe192700 1 -- 172.21.15.47:0/1409763407 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98f8145250 con 0x7f98f8140570 2022-06-29T15:51:42.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.302+0000 7f98fe192700 1 -- 172.21.15.47:0/1409763407 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98f80573d0 con 0x7f98f8140570 2022-06-29T15:51:42.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.302+0000 7f98ea7fc700 1 -- 172.21.15.47:0/1409763407 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f98f0010030 con 0x7f98f8140570 2022-06-29T15:51:42.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.302+0000 7f98ea7fc700 1 --2- 172.21.15.47:0/1409763407 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f98e0053a70 0x7f98e0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:42.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.302+0000 7f98ea7fc700 1 -- 172.21.15.47:0/1409763407 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f98f006bf40 con 0x7f98f8140570 2022-06-29T15:51:42.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.302+0000 7f98fd190700 1 --2- 172.21.15.47:0/1409763407 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f98e0053a70 0x7f98e0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:42.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.302+0000 7f98fd190700 1 --2- 172.21.15.47:0/1409763407 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f98e0053a70 0x7f98e0055f30 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7f98f400d000 tx=0x7f98f4009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:42.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.314+0000 7f98ea7fc700 1 -- 172.21.15.47:0/1409763407 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f98f0042c10 con 0x7f98f8140570 2022-06-29T15:51:42.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:42.442+0000 7f98fe192700 1 -- 172.21.15.47:0/1409763407 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f98f81474c0 con 0x7f98f8140570 2022-06-29T15:51:42.575 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:51:42.578 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:51:42.581 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:51:42.583 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:51:43.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.162+0000 7f98ea7fc700 1 -- 172.21.15.47:0/1409763407 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v142) v1 ==== 185+0+0 (secure 0 0 0) 0x7f98f0046230 con 0x7f98f8140570 2022-06-29T15:51:43.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.166+0000 7f98fe192700 1 -- 172.21.15.47:0/1409763407 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f98e0053a70 msgr2=0x7f98e0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:43.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.166+0000 7f98fe192700 1 --2- 172.21.15.47:0/1409763407 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f98e0053a70 0x7f98e0055f30 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7f98f400d000 tx=0x7f98f4009040).stop 2022-06-29T15:51:43.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.166+0000 7f98fe192700 1 -- 172.21.15.47:0/1409763407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98f8140570 msgr2=0x7f98f80ae910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:43.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.166+0000 7f98fe192700 1 --2- 172.21.15.47:0/1409763407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98f8140570 0x7f98f80ae910 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 rx=0x7f98f8144790 tx=0x7f98f000c800).stop 2022-06-29T15:51:43.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.166+0000 7f98fe192700 1 -- 172.21.15.47:0/1409763407 shutdown_connections 2022-06-29T15:51:43.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.166+0000 7f98fe192700 1 --2- 172.21.15.47:0/1409763407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98f8140570 0x7f98f80ae910 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:43.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.166+0000 7f98fe192700 1 --2- 172.21.15.47:0/1409763407 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f98f80b9b80 0x7f98f80ae420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:43.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.166+0000 7f98fe192700 1 --2- 172.21.15.47:0/1409763407 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f98e0053a70 0x7f98e0055f30 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:43.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.166+0000 7f98fe192700 1 --2- 172.21.15.47:0/1409763407 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98f80b59c0 0x7f98f8139170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:43.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.166+0000 7f98fe192700 1 -- 172.21.15.47:0/1409763407 >> 172.21.15.47:0/1409763407 conn(0x7f98f8062650 msgr2=0x7f98f80b6b60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:43.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.166+0000 7f98fe192700 1 -- 172.21.15.47:0/1409763407 shutdown_connections 2022-06-29T15:51:43.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.166+0000 7f98fe192700 1 -- 172.21.15.47:0/1409763407 wait complete. 2022-06-29T15:51:43.175 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' removed 2022-06-29T15:51:43.185 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-06-29T15:51:43.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.294+0000 7f59d359e700 1 Processor -- start 2022-06-29T15:51:43.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.294+0000 7f59d359e700 1 -- start start 2022-06-29T15:51:43.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.294+0000 7f59d359e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f59d40b0e50 0x7f59d40b0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:43.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.294+0000 7f59d359e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f59d4143200 0x7f59d4143630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:43.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.294+0000 7f59d359e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59d40b2ec0 con 0x7f59d40ad590 2022-06-29T15:51:43.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.294+0000 7f59d359e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59d4143b20 con 0x7f59d40b0e50 2022-06-29T15:51:43.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.294+0000 7f59d359e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59d40b1c50 con 0x7f59d4143200 2022-06-29T15:51:43.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.294+0000 7f59d259c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f59d40ad590 0x7f59d4140080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39646/0 (socket says 172.21.15.47:39646) 2022-06-29T15:51:43.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.294+0000 7f59d259c700 1 -- 172.21.15.47:0/312350997 learned_addr learned my addr 172.21.15.47:0/312350997 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:43.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.294+0000 7f59c9d9b700 1 --2- 172.21.15.47:0/312350997 >> v2:172.21.15.47:3300/0 conn(0x7f59d40b0e50 0x7f59d40b0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:43.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.294+0000 7f59d2d9d700 1 --2- 172.21.15.47:0/312350997 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f59d4143200 0x7f59d4143630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:43.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.294+0000 7f59d1d9b700 1 -- 172.21.15.47:0/312350997 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3219939874 0 0) 0x7f59d40b2ec0 con 0x7f59d40ad590 2022-06-29T15:51:43.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.294+0000 7f59d1d9b700 1 -- 172.21.15.47:0/312350997 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59b4003260 con 0x7f59d40ad590 2022-06-29T15:51:43.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.294+0000 7f59d1d9b700 1 -- 172.21.15.47:0/312350997 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2509229124 0 0) 0x7f59b4003260 con 0x7f59d40ad590 2022-06-29T15:51:43.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.294+0000 7f59c9d9b700 1 -- 172.21.15.47:0/312350997 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f59d4143200 msgr2=0x7f59d4143630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:43.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.294+0000 7f59c9d9b700 1 --2- 172.21.15.47:0/312350997 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f59d4143200 0x7f59d4143630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:43.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.294+0000 7f59c9d9b700 1 -- 172.21.15.47:0/312350997 >> v1:172.21.15.47:6789/0 conn(0x7f59d40ad590 legacy=0x7f59d4140080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:43.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.294+0000 7f59c9d9b700 1 -- 172.21.15.47:0/312350997 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d40b1ef0 con 0x7f59d40b0e50 2022-06-29T15:51:43.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.294+0000 7f59c9d9b700 1 --2- 172.21.15.47:0/312350997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59d40b0e50 0x7f59d40b0680 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 rx=0x7f59c4001820 tx=0x7f59bc006e80).ready entity=mon.0 client_cookie=25cf1c52e95d5313 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:43.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.294+0000 7f59d1d9b700 1 -- 172.21.15.47:0/312350997 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f59bc014070 con 0x7f59d40b0e50 2022-06-29T15:51:43.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.294+0000 7f59d1d9b700 1 -- 172.21.15.47:0/312350997 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59bc000e90 con 0x7f59d40b0e50 2022-06-29T15:51:43.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.298+0000 7f59d1d9b700 1 -- 172.21.15.47:0/312350997 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f59c4001680 con 0x7f59d40b0e50 2022-06-29T15:51:43.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.298+0000 7f59d359e700 1 -- 172.21.15.47:0/312350997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59d40b0e50 msgr2=0x7f59d40b0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:43.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.298+0000 7f59d359e700 1 --2- 172.21.15.47:0/312350997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59d40b0e50 0x7f59d40b0680 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 rx=0x7f59c4001820 tx=0x7f59bc006e80).stop 2022-06-29T15:51:43.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.298+0000 7f59d359e700 1 -- 172.21.15.47:0/312350997 shutdown_connections 2022-06-29T15:51:43.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.298+0000 7f59d359e700 1 --2- 172.21.15.47:0/312350997 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f59d4143200 0x7f59d4143630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:43.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.298+0000 7f59d359e700 1 --2- 172.21.15.47:0/312350997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59d40b0e50 0x7f59d40b0680 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:43.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.298+0000 7f59d359e700 1 -- 172.21.15.47:0/312350997 >> 172.21.15.47:0/312350997 conn(0x7f59d4060550 msgr2=0x7f59d4062d90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:43.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.298+0000 7f59d359e700 1 -- 172.21.15.47:0/312350997 shutdown_connections 2022-06-29T15:51:43.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.298+0000 7f59d359e700 1 -- 172.21.15.47:0/312350997 wait complete. 2022-06-29T15:51:43.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.298+0000 7f59d359e700 1 Processor -- start 2022-06-29T15:51:43.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.298+0000 7f59d359e700 1 -- start start 2022-06-29T15:51:43.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.302+0000 7f59d359e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f59d40ad590 0x7f59d4139230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:43.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.302+0000 7f59d359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59d40b0e50 0x7f59d4155d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:43.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.302+0000 7f59d359e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59d4143200 0x7f59d4156240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:43.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.302+0000 7f59d359e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d4152c10 con 0x7f59d4143200 2022-06-29T15:51:43.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.302+0000 7f59d359e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d4152ea0 con 0x7f59d40b0e50 2022-06-29T15:51:43.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.302+0000 7f59d359e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59d41558b0 con 0x7f59d40ad590 2022-06-29T15:51:43.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.302+0000 7f59d2d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59d4143200 0x7f59d4156240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:43.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.302+0000 7f59d259c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f59d40ad590 0x7f59d4139230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:43.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.302+0000 7f59d2d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59d4143200 0x7f59d4156240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54102/0 (socket says 172.21.15.47:54102) 2022-06-29T15:51:43.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.302+0000 7f59c9d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59d40b0e50 0x7f59d4155d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:43.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.302+0000 7f59d259c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f59d40ad590 0x7f59d4139230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:51200/0 (socket says 172.21.15.47:51200) 2022-06-29T15:51:43.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.302+0000 7f59d2d9d700 1 -- 172.21.15.47:0/3969951392 learned_addr learned my addr 172.21.15.47:0/3969951392 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:43.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.302+0000 7f59d2d9d700 1 -- 172.21.15.47:0/3969951392 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f59d40ad590 msgr2=0x7f59d4139230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:43.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.302+0000 7f59d2d9d700 1 --2- 172.21.15.47:0/3969951392 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f59d40ad590 0x7f59d4139230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:43.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.302+0000 7f59d2d9d700 1 -- 172.21.15.47:0/3969951392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59d40b0e50 msgr2=0x7f59d4155d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:43.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.302+0000 7f59d2d9d700 1 --2- 172.21.15.47:0/3969951392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59d40b0e50 0x7f59d4155d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:43.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.302+0000 7f59d2d9d700 1 -- 172.21.15.47:0/3969951392 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59bc00c310 con 0x7f59d4143200 2022-06-29T15:51:43.335 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.302+0000 7f59c9d9b700 1 --2- 172.21.15.47:0/3969951392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59d40b0e50 0x7f59d4155d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:51:43.335 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.302+0000 7f59d259c700 1 --2- 172.21.15.47:0/3969951392 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f59d40ad590 0x7f59d4139230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:51:43.335 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.302+0000 7f59d2d9d700 1 --2- 172.21.15.47:0/3969951392 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59d4143200 0x7f59d4156240 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 rx=0x7f59cc006e00 tx=0x7f59cc0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:43.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.302+0000 7f59cbfff700 1 -- 172.21.15.47:0/3969951392 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f59cc012600 con 0x7f59d4143200 2022-06-29T15:51:43.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.302+0000 7f59cbfff700 1 -- 172.21.15.47:0/3969951392 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59cc012760 con 0x7f59d4143200 2022-06-29T15:51:43.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.302+0000 7f59d359e700 1 -- 172.21.15.47:0/3969951392 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59d413f950 con 0x7f59d4143200 2022-06-29T15:51:43.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.302+0000 7f59cbfff700 1 -- 172.21.15.47:0/3969951392 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f59cc012ac0 con 0x7f59d4143200 2022-06-29T15:51:43.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.302+0000 7f59d359e700 1 -- 172.21.15.47:0/3969951392 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59d40b3d90 con 0x7f59d4143200 2022-06-29T15:51:43.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.302+0000 7f59d359e700 1 -- 172.21.15.47:0/3969951392 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59d41530a0 con 0x7f59d4143200 2022-06-29T15:51:43.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.302+0000 7f59cbfff700 1 -- 172.21.15.47:0/3969951392 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f59cc022070 con 0x7f59d4143200 2022-06-29T15:51:43.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.302+0000 7f59cbfff700 1 --2- 172.21.15.47:0/3969951392 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f59b8053950 0x7f59b8055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:43.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.306+0000 7f59cbfff700 1 -- 172.21.15.47:0/3969951392 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6068+0+0 (secure 0 0 0) 0x7f59cc06c4e0 con 0x7f59d4143200 2022-06-29T15:51:43.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.306+0000 7f59d259c700 1 --2- 172.21.15.47:0/3969951392 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f59b8053950 0x7f59b8055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:43.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.306+0000 7f59d259c700 1 --2- 172.21.15.47:0/3969951392 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f59b8053950 0x7f59b8055e10 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f59c400d000 tx=0x7f59c4001e40).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:43.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.314+0000 7f59cbfff700 1 -- 172.21.15.47:0/3969951392 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f59cc043410 con 0x7f59d4143200 2022-06-29T15:51:43.449 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:43.442+0000 7f59d359e700 1 -- 172.21.15.47:0/3969951392 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f59d40573d0 con 0x7f59d4143200 2022-06-29T15:51:44.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.166+0000 7f59cbfff700 1 -- 172.21.15.47:0/3969951392 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v143) v1 ==== 173+0+0 (secure 0 0 0) 0x7f59cc016020 con 0x7f59d4143200 2022-06-29T15:51:44.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.170+0000 7f59d359e700 1 -- 172.21.15.47:0/3969951392 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f59b8053950 msgr2=0x7f59b8055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:44.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.170+0000 7f59d359e700 1 --2- 172.21.15.47:0/3969951392 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f59b8053950 0x7f59b8055e10 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f59c400d000 tx=0x7f59c4001e40).stop 2022-06-29T15:51:44.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.170+0000 7f59d359e700 1 -- 172.21.15.47:0/3969951392 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59d4143200 msgr2=0x7f59d4156240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:44.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.170+0000 7f59d359e700 1 --2- 172.21.15.47:0/3969951392 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59d4143200 0x7f59d4156240 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 rx=0x7f59cc006e00 tx=0x7f59cc0091d0).stop 2022-06-29T15:51:44.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.170+0000 7f59d359e700 1 -- 172.21.15.47:0/3969951392 shutdown_connections 2022-06-29T15:51:44.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.170+0000 7f59d359e700 1 --2- 172.21.15.47:0/3969951392 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59d4143200 0x7f59d4156240 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:44.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.170+0000 7f59d359e700 1 --2- 172.21.15.47:0/3969951392 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f59b8053950 0x7f59b8055e10 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:44.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.170+0000 7f59d359e700 1 --2- 172.21.15.47:0/3969951392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59d40b0e50 0x7f59d4155d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:44.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.170+0000 7f59d359e700 1 --2- 172.21.15.47:0/3969951392 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f59d40ad590 0x7f59d4139230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:44.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.170+0000 7f59d359e700 1 -- 172.21.15.47:0/3969951392 >> 172.21.15.47:0/3969951392 conn(0x7f59d4060550 msgr2=0x7f59d40aea80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:44.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.170+0000 7f59d359e700 1 -- 172.21.15.47:0/3969951392 shutdown_connections 2022-06-29T15:51:44.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.170+0000 7f59d359e700 1 -- 172.21.15.47:0/3969951392 wait complete. 2022-06-29T15:51:44.182 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' removed 2022-06-29T15:51:44.195 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-06-29T15:51:44.198 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-06-29T15:51:44.200 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-06-29T15:51:44.203 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-06-29T15:51:44.225 INFO:teuthology.orchestra.run.smithi064.stdout:[global] 2022-06-29T15:51:44.226 INFO:teuthology.orchestra.run.smithi064.stdout: chdir = "" 2022-06-29T15:51:44.226 INFO:teuthology.orchestra.run.smithi064.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-06-29T15:51:44.226 INFO:teuthology.orchestra.run.smithi064.stdout: auth supported = cephx 2022-06-29T15:51:44.226 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:44.227 INFO:teuthology.orchestra.run.smithi064.stdout: filestore xattr use omap = true 2022-06-29T15:51:44.227 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:44.227 INFO:teuthology.orchestra.run.smithi064.stdout: mon clock drift allowed = 1.000 2022-06-29T15:51:44.227 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:44.228 INFO:teuthology.orchestra.run.smithi064.stdout: osd crush chooseleaf type = 0 2022-06-29T15:51:44.228 INFO:teuthology.orchestra.run.smithi064.stdout: auth debug = true 2022-06-29T15:51:44.228 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:44.228 INFO:teuthology.orchestra.run.smithi064.stdout: ms die on old message = true 2022-06-29T15:51:44.229 INFO:teuthology.orchestra.run.smithi064.stdout: ms die on bug = true 2022-06-29T15:51:44.229 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:44.229 INFO:teuthology.orchestra.run.smithi064.stdout: mon max pg per osd = 10000 # >= luminous 2022-06-29T15:51:44.229 INFO:teuthology.orchestra.run.smithi064.stdout: mon pg warn max object skew = 0 2022-06-29T15:51:44.230 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:44.230 INFO:teuthology.orchestra.run.smithi064.stdout: # disable pg_autoscaler by default for new pools 2022-06-29T15:51:44.230 INFO:teuthology.orchestra.run.smithi064.stdout: osd_pool_default_pg_autoscale_mode = off 2022-06-29T15:51:44.230 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:44.231 INFO:teuthology.orchestra.run.smithi064.stdout: osd pool default size = 2 2022-06-29T15:51:44.231 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:44.231 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd allow primary affinity = true 2022-06-29T15:51:44.231 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd allow pg remap = true 2022-06-29T15:51:44.232 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on legacy crush tunables = false 2022-06-29T15:51:44.232 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on crush straw calc version zero = false 2022-06-29T15:51:44.232 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on no sortbitwise = false 2022-06-29T15:51:44.232 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on osd down out interval zero = false 2022-06-29T15:51:44.232 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on too few osds = false 2022-06-29T15:51:44.233 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-06-29T15:51:44.233 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_pool_no_redundancy = false 2022-06-29T15:51:44.233 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:44.233 INFO:teuthology.orchestra.run.smithi064.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-06-29T15:51:44.234 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:44.234 INFO:teuthology.orchestra.run.smithi064.stdout: osd default data pool replay window = 5 2022-06-29T15:51:44.234 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:44.234 INFO:teuthology.orchestra.run.smithi064.stdout: mon allow pool delete = true 2022-06-29T15:51:44.235 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:44.235 INFO:teuthology.orchestra.run.smithi064.stdout: mon cluster log file level = debug 2022-06-29T15:51:44.235 INFO:teuthology.orchestra.run.smithi064.stdout: debug asserts on shutdown = true 2022-06-29T15:51:44.235 INFO:teuthology.orchestra.run.smithi064.stdout: mon health detail to clog = false 2022-06-29T15:51:44.236 INFO:teuthology.orchestra.run.smithi064.stdout: mon host = "172.21.15.47,172.21.15.64,[v2:172.21.15.64:3301,v1:172.21.15.64:6790]" 2022-06-29T15:51:44.236 INFO:teuthology.orchestra.run.smithi064.stdout: fsid = 9a68894a-53ce-400e-838b-14eab91b834c 2022-06-29T15:51:44.236 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:44.236 INFO:teuthology.orchestra.run.smithi064.stdout:[osd] 2022-06-29T15:51:44.237 INFO:teuthology.orchestra.run.smithi064.stdout: osd journal size = 100 2022-06-29T15:51:44.237 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:44.237 INFO:teuthology.orchestra.run.smithi064.stdout: osd scrub load threshold = 5.0 2022-06-29T15:51:44.237 INFO:teuthology.orchestra.run.smithi064.stdout: osd scrub max interval = 600 2022-06-29T15:51:44.238 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:44.238 INFO:teuthology.orchestra.run.smithi064.stdout: osd recover clone overlap = true 2022-06-29T15:51:44.238 INFO:teuthology.orchestra.run.smithi064.stdout: osd recovery max chunk = 1048576 2022-06-29T15:51:44.238 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:44.238 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug shutdown = true 2022-06-29T15:51:44.239 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug op order = true 2022-06-29T15:51:44.239 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug verify stray on activate = true 2022-06-29T15:51:44.239 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:44.240 INFO:teuthology.orchestra.run.smithi064.stdout: osd open classes on start = true 2022-06-29T15:51:44.240 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug pg log writeout = true 2022-06-29T15:51:44.240 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:44.240 INFO:teuthology.orchestra.run.smithi064.stdout: osd deep scrub update digest min age = 30 2022-06-29T15:51:44.240 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:44.241 INFO:teuthology.orchestra.run.smithi064.stdout: osd map max advance = 10 2022-06-29T15:51:44.241 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:44.241 INFO:teuthology.orchestra.run.smithi064.stdout: journal zero on create = true 2022-06-29T15:51:44.241 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:44.242 INFO:teuthology.orchestra.run.smithi064.stdout: filestore ondisk finisher threads = 3 2022-06-29T15:51:44.242 INFO:teuthology.orchestra.run.smithi064.stdout: filestore apply finisher threads = 3 2022-06-29T15:51:44.242 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:44.242 INFO:teuthology.orchestra.run.smithi064.stdout: bdev debug aio = true 2022-06-29T15:51:44.243 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug misdirected ops = true 2022-06-29T15:51:44.243 INFO:teuthology.orchestra.run.smithi064.stdout: bluestore block size = 96636764160 2022-06-29T15:51:44.243 INFO:teuthology.orchestra.run.smithi064.stdout: bluestore compression mode = aggressive 2022-06-29T15:51:44.243 INFO:teuthology.orchestra.run.smithi064.stdout: bluestore fsck on mount = True 2022-06-29T15:51:44.244 INFO:teuthology.orchestra.run.smithi064.stdout: debug bluefs = 20 2022-06-29T15:51:44.244 INFO:teuthology.orchestra.run.smithi064.stdout: debug bluestore = 20 2022-06-29T15:51:44.244 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2022-06-29T15:51:44.244 INFO:teuthology.orchestra.run.smithi064.stdout: debug osd = 20 2022-06-29T15:51:44.245 INFO:teuthology.orchestra.run.smithi064.stdout: debug rocksdb = 10 2022-06-29T15:51:44.245 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd backfillfull_ratio = 0.85 2022-06-29T15:51:44.245 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd full ratio = 0.9 2022-06-29T15:51:44.245 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd nearfull ratio = 0.8 2022-06-29T15:51:44.246 INFO:teuthology.orchestra.run.smithi064.stdout: osd failsafe full ratio = 0.95 2022-06-29T15:51:44.246 INFO:teuthology.orchestra.run.smithi064.stdout: osd objectstore = bluestore 2022-06-29T15:51:44.246 INFO:teuthology.orchestra.run.smithi064.stdout: osd op complaint time = 180 2022-06-29T15:51:44.246 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:44.247 INFO:teuthology.orchestra.run.smithi064.stdout:[mgr] 2022-06-29T15:51:44.247 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2022-06-29T15:51:44.247 INFO:teuthology.orchestra.run.smithi064.stdout: debug mgr = 20 2022-06-29T15:51:44.247 INFO:teuthology.orchestra.run.smithi064.stdout: debug mon = 20 2022-06-29T15:51:44.248 INFO:teuthology.orchestra.run.smithi064.stdout: debug auth = 20 2022-06-29T15:51:44.248 INFO:teuthology.orchestra.run.smithi064.stdout: mon reweight min pgs per osd = 4 2022-06-29T15:51:44.248 INFO:teuthology.orchestra.run.smithi064.stdout: mon reweight min bytes per osd = 10 2022-06-29T15:51:44.248 INFO:teuthology.orchestra.run.smithi064.stdout: mgr/telemetry/nag = false 2022-06-29T15:51:44.248 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:44.249 INFO:teuthology.orchestra.run.smithi064.stdout:[mon] 2022-06-29T15:51:44.249 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2022-06-29T15:51:44.249 INFO:teuthology.orchestra.run.smithi064.stdout: debug mon = 20 2022-06-29T15:51:44.249 INFO:teuthology.orchestra.run.smithi064.stdout: debug paxos = 20 2022-06-29T15:51:44.250 INFO:teuthology.orchestra.run.smithi064.stdout: debug auth = 20 2022-06-29T15:51:44.250 INFO:teuthology.orchestra.run.smithi064.stdout: mon data avail warn = 5 2022-06-29T15:51:44.250 INFO:teuthology.orchestra.run.smithi064.stdout: mon mgr mkfs grace = 240 2022-06-29T15:51:44.250 INFO:teuthology.orchestra.run.smithi064.stdout: mon reweight min pgs per osd = 4 2022-06-29T15:51:44.251 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd reporter subtree level = osd 2022-06-29T15:51:44.251 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd prime pg temp = true 2022-06-29T15:51:44.251 INFO:teuthology.orchestra.run.smithi064.stdout: mon reweight min bytes per osd = 10 2022-06-29T15:51:44.251 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:44.252 INFO:teuthology.orchestra.run.smithi064.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-06-29T15:51:44.252 INFO:teuthology.orchestra.run.smithi064.stdout: auth mon ticket ttl = 660 # 11m 2022-06-29T15:51:44.252 INFO:teuthology.orchestra.run.smithi064.stdout: auth service ticket ttl = 240 # 4m 2022-06-29T15:51:44.252 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:44.252 INFO:teuthology.orchestra.run.smithi064.stdout: # don't complain about insecure global_id in the test suite 2022-06-29T15:51:44.253 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-06-29T15:51:44.253 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-06-29T15:51:44.253 INFO:teuthology.orchestra.run.smithi064.stdout: mon op complaint time = 120 2022-06-29T15:51:44.253 INFO:teuthology.orchestra.run.smithi064.stdout: 2022-06-29T15:51:44.254 INFO:teuthology.orchestra.run.smithi064.stdout:[client] 2022-06-29T15:51:44.254 INFO:teuthology.orchestra.run.smithi064.stdout: rgw cache enabled = true 2022-06-29T15:51:44.254 INFO:teuthology.orchestra.run.smithi064.stdout: rgw enable ops log = true 2022-06-29T15:51:44.254 INFO:teuthology.orchestra.run.smithi064.stdout: rgw enable usage log = true 2022-06-29T15:51:44.255 INFO:teuthology.orchestra.run.smithi064.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-06-29T15:51:44.255 INFO:teuthology.orchestra.run.smithi064.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-06-29T15:51:44.255 INFO:teuthology.orchestra.run.smithi064.stdout: client mount timeout = 600 2022-06-29T15:51:44.255 INFO:teuthology.orchestra.run.smithi064.stdout: debug client = 20 2022-06-29T15:51:44.256 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2022-06-29T15:51:44.256 INFO:teuthology.orchestra.run.smithi064.stdout: rados mon op timeout = 15m 2022-06-29T15:51:44.256 INFO:teuthology.orchestra.run.smithi064.stdout: rados osd op timeout = 15m 2022-06-29T15:51:44.256 INFO:teuthology.orchestra.run.smithi064.stdout:[mon.a] 2022-06-29T15:51:44.256 INFO:teuthology.orchestra.run.smithi064.stdout:[mon.b] 2022-06-29T15:51:44.257 INFO:teuthology.orchestra.run.smithi064.stdout:[mon.c] 2022-06-29T15:51:44.257 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.a] 2022-06-29T15:51:44.257 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.c] 2022-06-29T15:51:44.257 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.b] 2022-06-29T15:51:44.258 INFO:teuthology.orchestra.run.smithi064.stdout:[mds] 2022-06-29T15:51:44.258 INFO:teuthology.orchestra.run.smithi064.stdout: debug mds = 20 2022-06-29T15:51:44.258 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2022-06-29T15:51:44.258 INFO:teuthology.orchestra.run.smithi064.stdout: mds bal frag = True 2022-06-29T15:51:44.259 INFO:teuthology.orchestra.run.smithi064.stdout: mds bal fragment size max = 10000 2022-06-29T15:51:44.259 INFO:teuthology.orchestra.run.smithi064.stdout: mds bal merge size = 5 2022-06-29T15:51:44.259 INFO:teuthology.orchestra.run.smithi064.stdout: mds bal split bits = 3 2022-06-29T15:51:44.259 INFO:teuthology.orchestra.run.smithi064.stdout: mds bal split size = 100 2022-06-29T15:51:44.260 INFO:teuthology.orchestra.run.smithi064.stdout: mds debug frag = True 2022-06-29T15:51:44.260 INFO:teuthology.orchestra.run.smithi064.stdout: mds debug scatterstat = True 2022-06-29T15:51:44.260 INFO:teuthology.orchestra.run.smithi064.stdout: mds op complaint time = 180 2022-06-29T15:51:44.260 INFO:teuthology.orchestra.run.smithi064.stdout: mds verify scatter = True 2022-06-29T15:51:44.261 INFO:teuthology.orchestra.run.smithi064.stdout: osd op complaint time = 180 2022-06-29T15:51:44.261 INFO:teuthology.orchestra.run.smithi064.stdout: rados mon op timeout = 15m 2022-06-29T15:51:44.261 INFO:teuthology.orchestra.run.smithi064.stdout: rados osd op timeout = 15m 2022-06-29T15:51:44.264 INFO:teuthology.orchestra.run.smithi164.stdout:[global] 2022-06-29T15:51:44.265 INFO:teuthology.orchestra.run.smithi164.stdout: chdir = "" 2022-06-29T15:51:44.265 INFO:teuthology.orchestra.run.smithi164.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-06-29T15:51:44.265 INFO:teuthology.orchestra.run.smithi164.stdout: auth supported = cephx 2022-06-29T15:51:44.265 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:44.266 INFO:teuthology.orchestra.run.smithi164.stdout: filestore xattr use omap = true 2022-06-29T15:51:44.266 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:44.266 INFO:teuthology.orchestra.run.smithi164.stdout: mon clock drift allowed = 1.000 2022-06-29T15:51:44.266 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:44.267 INFO:teuthology.orchestra.run.smithi164.stdout: osd crush chooseleaf type = 0 2022-06-29T15:51:44.267 INFO:teuthology.orchestra.run.smithi164.stdout: auth debug = true 2022-06-29T15:51:44.267 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:44.267 INFO:teuthology.orchestra.run.smithi164.stdout: ms die on old message = true 2022-06-29T15:51:44.268 INFO:teuthology.orchestra.run.smithi164.stdout: ms die on bug = true 2022-06-29T15:51:44.268 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:44.268 INFO:teuthology.orchestra.run.smithi164.stdout: mon max pg per osd = 10000 # >= luminous 2022-06-29T15:51:44.268 INFO:teuthology.orchestra.run.smithi164.stdout: mon pg warn max object skew = 0 2022-06-29T15:51:44.269 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:44.269 INFO:teuthology.orchestra.run.smithi164.stdout: # disable pg_autoscaler by default for new pools 2022-06-29T15:51:44.269 INFO:teuthology.orchestra.run.smithi164.stdout: osd_pool_default_pg_autoscale_mode = off 2022-06-29T15:51:44.269 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:44.270 INFO:teuthology.orchestra.run.smithi164.stdout: osd pool default size = 2 2022-06-29T15:51:44.270 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:44.270 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd allow primary affinity = true 2022-06-29T15:51:44.270 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd allow pg remap = true 2022-06-29T15:51:44.271 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on legacy crush tunables = false 2022-06-29T15:51:44.271 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on crush straw calc version zero = false 2022-06-29T15:51:44.271 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on no sortbitwise = false 2022-06-29T15:51:44.271 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on osd down out interval zero = false 2022-06-29T15:51:44.272 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on too few osds = false 2022-06-29T15:51:44.272 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-06-29T15:51:44.272 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_pool_no_redundancy = false 2022-06-29T15:51:44.272 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:44.273 INFO:teuthology.orchestra.run.smithi164.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-06-29T15:51:44.273 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:44.273 INFO:teuthology.orchestra.run.smithi164.stdout: osd default data pool replay window = 5 2022-06-29T15:51:44.273 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:44.273 INFO:teuthology.orchestra.run.smithi164.stdout: mon allow pool delete = true 2022-06-29T15:51:44.274 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:44.274 INFO:teuthology.orchestra.run.smithi164.stdout: mon cluster log file level = debug 2022-06-29T15:51:44.274 INFO:teuthology.orchestra.run.smithi164.stdout: debug asserts on shutdown = true 2022-06-29T15:51:44.274 INFO:teuthology.orchestra.run.smithi164.stdout: mon health detail to clog = false 2022-06-29T15:51:44.275 INFO:teuthology.orchestra.run.smithi164.stdout: mon host = "172.21.15.47,172.21.15.64,[v2:172.21.15.64:3301,v1:172.21.15.64:6790]" 2022-06-29T15:51:44.275 INFO:teuthology.orchestra.run.smithi164.stdout: fsid = 9a68894a-53ce-400e-838b-14eab91b834c 2022-06-29T15:51:44.275 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:44.275 INFO:teuthology.orchestra.run.smithi164.stdout:[osd] 2022-06-29T15:51:44.276 INFO:teuthology.orchestra.run.smithi164.stdout: osd journal size = 100 2022-06-29T15:51:44.276 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:44.276 INFO:teuthology.orchestra.run.smithi164.stdout: osd scrub load threshold = 5.0 2022-06-29T15:51:44.276 INFO:teuthology.orchestra.run.smithi164.stdout: osd scrub max interval = 600 2022-06-29T15:51:44.277 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:44.277 INFO:teuthology.orchestra.run.smithi164.stdout: osd recover clone overlap = true 2022-06-29T15:51:44.277 INFO:teuthology.orchestra.run.smithi164.stdout: osd recovery max chunk = 1048576 2022-06-29T15:51:44.277 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:44.278 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug shutdown = true 2022-06-29T15:51:44.278 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug op order = true 2022-06-29T15:51:44.278 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug verify stray on activate = true 2022-06-29T15:51:44.278 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:44.279 INFO:teuthology.orchestra.run.smithi164.stdout: osd open classes on start = true 2022-06-29T15:51:44.279 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug pg log writeout = true 2022-06-29T15:51:44.279 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:44.279 INFO:teuthology.orchestra.run.smithi164.stdout: osd deep scrub update digest min age = 30 2022-06-29T15:51:44.279 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:44.280 INFO:teuthology.orchestra.run.smithi164.stdout: osd map max advance = 10 2022-06-29T15:51:44.280 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:44.280 INFO:teuthology.orchestra.run.smithi164.stdout: journal zero on create = true 2022-06-29T15:51:44.280 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:44.281 INFO:teuthology.orchestra.run.smithi164.stdout: filestore ondisk finisher threads = 3 2022-06-29T15:51:44.281 INFO:teuthology.orchestra.run.smithi164.stdout: filestore apply finisher threads = 3 2022-06-29T15:51:44.281 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:44.281 INFO:teuthology.orchestra.run.smithi164.stdout: bdev debug aio = true 2022-06-29T15:51:44.282 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug misdirected ops = true 2022-06-29T15:51:44.282 INFO:teuthology.orchestra.run.smithi164.stdout: bluestore block size = 96636764160 2022-06-29T15:51:44.282 INFO:teuthology.orchestra.run.smithi164.stdout: bluestore compression mode = aggressive 2022-06-29T15:51:44.282 INFO:teuthology.orchestra.run.smithi164.stdout: bluestore fsck on mount = True 2022-06-29T15:51:44.283 INFO:teuthology.orchestra.run.smithi164.stdout: debug bluefs = 20 2022-06-29T15:51:44.283 INFO:teuthology.orchestra.run.smithi164.stdout: debug bluestore = 20 2022-06-29T15:51:44.283 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2022-06-29T15:51:44.283 INFO:teuthology.orchestra.run.smithi164.stdout: debug osd = 20 2022-06-29T15:51:44.284 INFO:teuthology.orchestra.run.smithi164.stdout: debug rocksdb = 10 2022-06-29T15:51:44.284 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd backfillfull_ratio = 0.85 2022-06-29T15:51:44.284 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd full ratio = 0.9 2022-06-29T15:51:44.284 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd nearfull ratio = 0.8 2022-06-29T15:51:44.285 INFO:teuthology.orchestra.run.smithi164.stdout: osd failsafe full ratio = 0.95 2022-06-29T15:51:44.285 INFO:teuthology.orchestra.run.smithi164.stdout: osd objectstore = bluestore 2022-06-29T15:51:44.285 INFO:teuthology.orchestra.run.smithi164.stdout: osd op complaint time = 180 2022-06-29T15:51:44.285 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:44.286 INFO:teuthology.orchestra.run.smithi164.stdout:[mgr] 2022-06-29T15:51:44.286 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2022-06-29T15:51:44.286 INFO:teuthology.orchestra.run.smithi164.stdout: debug mgr = 20 2022-06-29T15:51:44.286 INFO:teuthology.orchestra.run.smithi164.stdout: debug mon = 20 2022-06-29T15:51:44.287 INFO:teuthology.orchestra.run.smithi164.stdout: debug auth = 20 2022-06-29T15:51:44.287 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min pgs per osd = 4 2022-06-29T15:51:44.287 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min bytes per osd = 10 2022-06-29T15:51:44.287 INFO:teuthology.orchestra.run.smithi164.stdout: mgr/telemetry/nag = false 2022-06-29T15:51:44.288 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:44.288 INFO:teuthology.orchestra.run.smithi164.stdout:[mon] 2022-06-29T15:51:44.288 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2022-06-29T15:51:44.288 INFO:teuthology.orchestra.run.smithi164.stdout: debug mon = 20 2022-06-29T15:51:44.289 INFO:teuthology.orchestra.run.smithi164.stdout: debug paxos = 20 2022-06-29T15:51:44.289 INFO:teuthology.orchestra.run.smithi164.stdout: debug auth = 20 2022-06-29T15:51:44.289 INFO:teuthology.orchestra.run.smithi164.stdout: mon data avail warn = 5 2022-06-29T15:51:44.289 INFO:teuthology.orchestra.run.smithi164.stdout: mon mgr mkfs grace = 240 2022-06-29T15:51:44.289 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min pgs per osd = 4 2022-06-29T15:51:44.290 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd reporter subtree level = osd 2022-06-29T15:51:44.290 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd prime pg temp = true 2022-06-29T15:51:44.290 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min bytes per osd = 10 2022-06-29T15:51:44.290 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:44.291 INFO:teuthology.orchestra.run.smithi164.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-06-29T15:51:44.291 INFO:teuthology.orchestra.run.smithi164.stdout: auth mon ticket ttl = 660 # 11m 2022-06-29T15:51:44.291 INFO:teuthology.orchestra.run.smithi164.stdout: auth service ticket ttl = 240 # 4m 2022-06-29T15:51:44.291 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:44.292 INFO:teuthology.orchestra.run.smithi164.stdout: # don't complain about insecure global_id in the test suite 2022-06-29T15:51:44.292 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-06-29T15:51:44.292 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-06-29T15:51:44.292 INFO:teuthology.orchestra.run.smithi164.stdout: mon op complaint time = 120 2022-06-29T15:51:44.293 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-06-29T15:51:44.293 INFO:teuthology.orchestra.run.smithi164.stdout:[client] 2022-06-29T15:51:44.293 INFO:teuthology.orchestra.run.smithi164.stdout: rgw cache enabled = true 2022-06-29T15:51:44.293 INFO:teuthology.orchestra.run.smithi164.stdout: rgw enable ops log = true 2022-06-29T15:51:44.294 INFO:teuthology.orchestra.run.smithi164.stdout: rgw enable usage log = true 2022-06-29T15:51:44.294 INFO:teuthology.orchestra.run.smithi164.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-06-29T15:51:44.294 INFO:teuthology.orchestra.run.smithi164.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-06-29T15:51:44.294 INFO:teuthology.orchestra.run.smithi164.stdout: client mount timeout = 600 2022-06-29T15:51:44.294 INFO:teuthology.orchestra.run.smithi164.stdout: debug client = 20 2022-06-29T15:51:44.295 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2022-06-29T15:51:44.295 INFO:teuthology.orchestra.run.smithi164.stdout: rados mon op timeout = 15m 2022-06-29T15:51:44.295 INFO:teuthology.orchestra.run.smithi164.stdout: rados osd op timeout = 15m 2022-06-29T15:51:44.295 INFO:teuthology.orchestra.run.smithi164.stdout:[mon.a] 2022-06-29T15:51:44.296 INFO:teuthology.orchestra.run.smithi164.stdout:[mon.b] 2022-06-29T15:51:44.296 INFO:teuthology.orchestra.run.smithi164.stdout:[mon.c] 2022-06-29T15:51:44.296 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.a] 2022-06-29T15:51:44.296 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.c] 2022-06-29T15:51:44.297 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.b] 2022-06-29T15:51:44.297 INFO:teuthology.orchestra.run.smithi164.stdout:[mds] 2022-06-29T15:51:44.297 INFO:teuthology.orchestra.run.smithi164.stdout: debug mds = 20 2022-06-29T15:51:44.297 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2022-06-29T15:51:44.298 INFO:teuthology.orchestra.run.smithi164.stdout: mds bal frag = True 2022-06-29T15:51:44.298 INFO:teuthology.orchestra.run.smithi164.stdout: mds bal fragment size max = 10000 2022-06-29T15:51:44.298 INFO:teuthology.orchestra.run.smithi164.stdout: mds bal merge size = 5 2022-06-29T15:51:44.298 INFO:teuthology.orchestra.run.smithi164.stdout: mds bal split bits = 3 2022-06-29T15:51:44.299 INFO:teuthology.orchestra.run.smithi164.stdout: mds bal split size = 100 2022-06-29T15:51:44.299 INFO:teuthology.orchestra.run.smithi164.stdout: mds debug frag = True 2022-06-29T15:51:44.299 INFO:teuthology.orchestra.run.smithi164.stdout: mds debug scatterstat = True 2022-06-29T15:51:44.299 INFO:teuthology.orchestra.run.smithi164.stdout: mds op complaint time = 180 2022-06-29T15:51:44.299 INFO:teuthology.orchestra.run.smithi164.stdout: mds verify scatter = True 2022-06-29T15:51:44.300 INFO:teuthology.orchestra.run.smithi164.stdout: osd op complaint time = 180 2022-06-29T15:51:44.300 INFO:teuthology.orchestra.run.smithi164.stdout: rados mon op timeout = 15m 2022-06-29T15:51:44.300 INFO:teuthology.orchestra.run.smithi164.stdout: rados osd op timeout = 15m 2022-06-29T15:51:44.300 INFO:teuthology.orchestra.run.smithi099.stdout:[global] 2022-06-29T15:51:44.301 INFO:teuthology.orchestra.run.smithi099.stdout: chdir = "" 2022-06-29T15:51:44.301 INFO:teuthology.orchestra.run.smithi099.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-06-29T15:51:44.301 INFO:teuthology.orchestra.run.smithi099.stdout: auth supported = cephx 2022-06-29T15:51:44.302 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:44.302 INFO:teuthology.orchestra.run.smithi099.stdout: filestore xattr use omap = true 2022-06-29T15:51:44.302 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:44.302 INFO:teuthology.orchestra.run.smithi099.stdout: mon clock drift allowed = 1.000 2022-06-29T15:51:44.303 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:44.303 INFO:teuthology.orchestra.run.smithi099.stdout: osd crush chooseleaf type = 0 2022-06-29T15:51:44.303 INFO:teuthology.orchestra.run.smithi099.stdout: auth debug = true 2022-06-29T15:51:44.303 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:44.304 INFO:teuthology.orchestra.run.smithi099.stdout: ms die on old message = true 2022-06-29T15:51:44.304 INFO:teuthology.orchestra.run.smithi099.stdout: ms die on bug = true 2022-06-29T15:51:44.304 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:44.304 INFO:teuthology.orchestra.run.smithi099.stdout: mon max pg per osd = 10000 # >= luminous 2022-06-29T15:51:44.305 INFO:teuthology.orchestra.run.smithi099.stdout: mon pg warn max object skew = 0 2022-06-29T15:51:44.305 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:44.305 INFO:teuthology.orchestra.run.smithi099.stdout: # disable pg_autoscaler by default for new pools 2022-06-29T15:51:44.305 INFO:teuthology.orchestra.run.smithi099.stdout: osd_pool_default_pg_autoscale_mode = off 2022-06-29T15:51:44.306 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:44.306 INFO:teuthology.orchestra.run.smithi099.stdout: osd pool default size = 2 2022-06-29T15:51:44.306 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:44.306 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd allow primary affinity = true 2022-06-29T15:51:44.307 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd allow pg remap = true 2022-06-29T15:51:44.307 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on legacy crush tunables = false 2022-06-29T15:51:44.307 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on crush straw calc version zero = false 2022-06-29T15:51:44.307 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on no sortbitwise = false 2022-06-29T15:51:44.307 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on osd down out interval zero = false 2022-06-29T15:51:44.308 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on too few osds = false 2022-06-29T15:51:44.308 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-06-29T15:51:44.308 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_pool_no_redundancy = false 2022-06-29T15:51:44.308 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:44.309 INFO:teuthology.orchestra.run.smithi099.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-06-29T15:51:44.309 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:44.309 INFO:teuthology.orchestra.run.smithi099.stdout: osd default data pool replay window = 5 2022-06-29T15:51:44.309 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:44.310 INFO:teuthology.orchestra.run.smithi099.stdout: mon allow pool delete = true 2022-06-29T15:51:44.310 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:44.310 INFO:teuthology.orchestra.run.smithi099.stdout: mon cluster log file level = debug 2022-06-29T15:51:44.310 INFO:teuthology.orchestra.run.smithi099.stdout: debug asserts on shutdown = true 2022-06-29T15:51:44.311 INFO:teuthology.orchestra.run.smithi099.stdout: mon health detail to clog = false 2022-06-29T15:51:44.311 INFO:teuthology.orchestra.run.smithi099.stdout: mon host = "172.21.15.47,172.21.15.64,[v2:172.21.15.64:3301,v1:172.21.15.64:6790]" 2022-06-29T15:51:44.311 INFO:teuthology.orchestra.run.smithi099.stdout: fsid = 9a68894a-53ce-400e-838b-14eab91b834c 2022-06-29T15:51:44.311 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:44.312 INFO:teuthology.orchestra.run.smithi099.stdout:[osd] 2022-06-29T15:51:44.312 INFO:teuthology.orchestra.run.smithi099.stdout: osd journal size = 100 2022-06-29T15:51:44.312 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:44.312 INFO:teuthology.orchestra.run.smithi099.stdout: osd scrub load threshold = 5.0 2022-06-29T15:51:44.312 INFO:teuthology.orchestra.run.smithi099.stdout: osd scrub max interval = 600 2022-06-29T15:51:44.313 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:44.313 INFO:teuthology.orchestra.run.smithi099.stdout: osd recover clone overlap = true 2022-06-29T15:51:44.313 INFO:teuthology.orchestra.run.smithi099.stdout: osd recovery max chunk = 1048576 2022-06-29T15:51:44.313 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:44.314 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug shutdown = true 2022-06-29T15:51:44.314 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug op order = true 2022-06-29T15:51:44.314 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug verify stray on activate = true 2022-06-29T15:51:44.314 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:44.315 INFO:teuthology.orchestra.run.smithi099.stdout: osd open classes on start = true 2022-06-29T15:51:44.315 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug pg log writeout = true 2022-06-29T15:51:44.315 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:44.315 INFO:teuthology.orchestra.run.smithi099.stdout: osd deep scrub update digest min age = 30 2022-06-29T15:51:44.315 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:44.316 INFO:teuthology.orchestra.run.smithi099.stdout: osd map max advance = 10 2022-06-29T15:51:44.316 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:44.316 INFO:teuthology.orchestra.run.smithi099.stdout: journal zero on create = true 2022-06-29T15:51:44.316 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:44.317 INFO:teuthology.orchestra.run.smithi099.stdout: filestore ondisk finisher threads = 3 2022-06-29T15:51:44.317 INFO:teuthology.orchestra.run.smithi099.stdout: filestore apply finisher threads = 3 2022-06-29T15:51:44.317 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:44.317 INFO:teuthology.orchestra.run.smithi099.stdout: bdev debug aio = true 2022-06-29T15:51:44.318 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug misdirected ops = true 2022-06-29T15:51:44.318 INFO:teuthology.orchestra.run.smithi099.stdout: bluestore block size = 96636764160 2022-06-29T15:51:44.318 INFO:teuthology.orchestra.run.smithi099.stdout: bluestore compression mode = aggressive 2022-06-29T15:51:44.318 INFO:teuthology.orchestra.run.smithi099.stdout: bluestore fsck on mount = True 2022-06-29T15:51:44.319 INFO:teuthology.orchestra.run.smithi099.stdout: debug bluefs = 20 2022-06-29T15:51:44.319 INFO:teuthology.orchestra.run.smithi099.stdout: debug bluestore = 20 2022-06-29T15:51:44.319 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2022-06-29T15:51:44.319 INFO:teuthology.orchestra.run.smithi099.stdout: debug osd = 20 2022-06-29T15:51:44.320 INFO:teuthology.orchestra.run.smithi099.stdout: debug rocksdb = 10 2022-06-29T15:51:44.320 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd backfillfull_ratio = 0.85 2022-06-29T15:51:44.320 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd full ratio = 0.9 2022-06-29T15:51:44.320 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd nearfull ratio = 0.8 2022-06-29T15:51:44.320 INFO:teuthology.orchestra.run.smithi099.stdout: osd failsafe full ratio = 0.95 2022-06-29T15:51:44.321 INFO:teuthology.orchestra.run.smithi099.stdout: osd objectstore = bluestore 2022-06-29T15:51:44.321 INFO:teuthology.orchestra.run.smithi099.stdout: osd op complaint time = 180 2022-06-29T15:51:44.321 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:44.321 INFO:teuthology.orchestra.run.smithi099.stdout:[mgr] 2022-06-29T15:51:44.322 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2022-06-29T15:51:44.322 INFO:teuthology.orchestra.run.smithi099.stdout: debug mgr = 20 2022-06-29T15:51:44.322 INFO:teuthology.orchestra.run.smithi099.stdout: debug mon = 20 2022-06-29T15:51:44.322 INFO:teuthology.orchestra.run.smithi099.stdout: debug auth = 20 2022-06-29T15:51:44.323 INFO:teuthology.orchestra.run.smithi099.stdout: mon reweight min pgs per osd = 4 2022-06-29T15:51:44.323 INFO:teuthology.orchestra.run.smithi099.stdout: mon reweight min bytes per osd = 10 2022-06-29T15:51:44.323 INFO:teuthology.orchestra.run.smithi099.stdout: mgr/telemetry/nag = false 2022-06-29T15:51:44.323 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:44.324 INFO:teuthology.orchestra.run.smithi099.stdout:[mon] 2022-06-29T15:51:44.324 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2022-06-29T15:51:44.324 INFO:teuthology.orchestra.run.smithi099.stdout: debug mon = 20 2022-06-29T15:51:44.324 INFO:teuthology.orchestra.run.smithi099.stdout: debug paxos = 20 2022-06-29T15:51:44.325 INFO:teuthology.orchestra.run.smithi099.stdout: debug auth = 20 2022-06-29T15:51:44.325 INFO:teuthology.orchestra.run.smithi099.stdout: mon data avail warn = 5 2022-06-29T15:51:44.325 INFO:teuthology.orchestra.run.smithi099.stdout: mon mgr mkfs grace = 240 2022-06-29T15:51:44.325 INFO:teuthology.orchestra.run.smithi099.stdout: mon reweight min pgs per osd = 4 2022-06-29T15:51:44.326 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd reporter subtree level = osd 2022-06-29T15:51:44.326 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd prime pg temp = true 2022-06-29T15:51:44.326 INFO:teuthology.orchestra.run.smithi099.stdout: mon reweight min bytes per osd = 10 2022-06-29T15:51:44.326 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:44.327 INFO:teuthology.orchestra.run.smithi099.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-06-29T15:51:44.327 INFO:teuthology.orchestra.run.smithi099.stdout: auth mon ticket ttl = 660 # 11m 2022-06-29T15:51:44.327 INFO:teuthology.orchestra.run.smithi099.stdout: auth service ticket ttl = 240 # 4m 2022-06-29T15:51:44.327 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:44.328 INFO:teuthology.orchestra.run.smithi099.stdout: # don't complain about insecure global_id in the test suite 2022-06-29T15:51:44.328 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-06-29T15:51:44.328 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-06-29T15:51:44.328 INFO:teuthology.orchestra.run.smithi099.stdout: mon op complaint time = 120 2022-06-29T15:51:44.329 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-06-29T15:51:44.329 INFO:teuthology.orchestra.run.smithi099.stdout:[client] 2022-06-29T15:51:44.329 INFO:teuthology.orchestra.run.smithi099.stdout: rgw cache enabled = true 2022-06-29T15:51:44.329 INFO:teuthology.orchestra.run.smithi099.stdout: rgw enable ops log = true 2022-06-29T15:51:44.330 INFO:teuthology.orchestra.run.smithi099.stdout: rgw enable usage log = true 2022-06-29T15:51:44.330 INFO:teuthology.orchestra.run.smithi099.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-06-29T15:51:44.330 INFO:teuthology.orchestra.run.smithi099.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-06-29T15:51:44.330 INFO:teuthology.orchestra.run.smithi099.stdout: client mount timeout = 600 2022-06-29T15:51:44.331 INFO:teuthology.orchestra.run.smithi099.stdout: debug client = 20 2022-06-29T15:51:44.331 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2022-06-29T15:51:44.331 INFO:teuthology.orchestra.run.smithi099.stdout: rados mon op timeout = 15m 2022-06-29T15:51:44.331 INFO:teuthology.orchestra.run.smithi099.stdout: rados osd op timeout = 15m 2022-06-29T15:51:44.332 INFO:teuthology.orchestra.run.smithi099.stdout:[mon.a] 2022-06-29T15:51:44.332 INFO:teuthology.orchestra.run.smithi099.stdout:[mon.b] 2022-06-29T15:51:44.332 INFO:teuthology.orchestra.run.smithi099.stdout:[mon.c] 2022-06-29T15:51:44.332 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.a] 2022-06-29T15:51:44.333 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.c] 2022-06-29T15:51:44.333 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.b] 2022-06-29T15:51:44.333 INFO:teuthology.orchestra.run.smithi099.stdout:[mds] 2022-06-29T15:51:44.333 INFO:teuthology.orchestra.run.smithi099.stdout: debug mds = 20 2022-06-29T15:51:44.333 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2022-06-29T15:51:44.334 INFO:teuthology.orchestra.run.smithi099.stdout: mds bal frag = True 2022-06-29T15:51:44.334 INFO:teuthology.orchestra.run.smithi099.stdout: mds bal fragment size max = 10000 2022-06-29T15:51:44.334 INFO:teuthology.orchestra.run.smithi099.stdout: mds bal merge size = 5 2022-06-29T15:51:44.335 INFO:teuthology.orchestra.run.smithi099.stdout: mds bal split bits = 3 2022-06-29T15:51:44.335 INFO:teuthology.orchestra.run.smithi099.stdout: mds bal split size = 100 2022-06-29T15:51:44.335 INFO:teuthology.orchestra.run.smithi099.stdout: mds debug frag = True 2022-06-29T15:51:44.335 INFO:teuthology.orchestra.run.smithi099.stdout: mds debug scatterstat = True 2022-06-29T15:51:44.336 INFO:teuthology.orchestra.run.smithi099.stdout: mds op complaint time = 180 2022-06-29T15:51:44.336 INFO:teuthology.orchestra.run.smithi099.stdout: mds verify scatter = True 2022-06-29T15:51:44.336 INFO:teuthology.orchestra.run.smithi099.stdout: osd op complaint time = 180 2022-06-29T15:51:44.336 INFO:teuthology.orchestra.run.smithi099.stdout: rados mon op timeout = 15m 2022-06-29T15:51:44.337 INFO:teuthology.orchestra.run.smithi099.stdout: rados osd op timeout = 15m 2022-06-29T15:51:44.337 INFO:teuthology.orchestra.run.smithi047.stdout:[global] 2022-06-29T15:51:44.337 INFO:teuthology.orchestra.run.smithi047.stdout: chdir = "" 2022-06-29T15:51:44.338 INFO:teuthology.orchestra.run.smithi047.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-06-29T15:51:44.338 INFO:teuthology.orchestra.run.smithi047.stdout: auth supported = cephx 2022-06-29T15:51:44.338 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:44.338 INFO:teuthology.orchestra.run.smithi047.stdout: filestore xattr use omap = true 2022-06-29T15:51:44.339 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:44.339 INFO:teuthology.orchestra.run.smithi047.stdout: mon clock drift allowed = 1.000 2022-06-29T15:51:44.339 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:44.339 INFO:teuthology.orchestra.run.smithi047.stdout: osd crush chooseleaf type = 0 2022-06-29T15:51:44.339 INFO:teuthology.orchestra.run.smithi047.stdout: auth debug = true 2022-06-29T15:51:44.340 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:44.340 INFO:teuthology.orchestra.run.smithi047.stdout: ms die on old message = true 2022-06-29T15:51:44.340 INFO:teuthology.orchestra.run.smithi047.stdout: ms die on bug = true 2022-06-29T15:51:44.340 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:44.341 INFO:teuthology.orchestra.run.smithi047.stdout: mon max pg per osd = 10000 # >= luminous 2022-06-29T15:51:44.341 INFO:teuthology.orchestra.run.smithi047.stdout: mon pg warn max object skew = 0 2022-06-29T15:51:44.341 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:44.341 INFO:teuthology.orchestra.run.smithi047.stdout: # disable pg_autoscaler by default for new pools 2022-06-29T15:51:44.342 INFO:teuthology.orchestra.run.smithi047.stdout: osd_pool_default_pg_autoscale_mode = off 2022-06-29T15:51:44.342 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:44.342 INFO:teuthology.orchestra.run.smithi047.stdout: osd pool default size = 2 2022-06-29T15:51:44.342 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:44.343 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd allow primary affinity = true 2022-06-29T15:51:44.343 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd allow pg remap = true 2022-06-29T15:51:44.343 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on legacy crush tunables = false 2022-06-29T15:51:44.343 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on crush straw calc version zero = false 2022-06-29T15:51:44.344 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on no sortbitwise = false 2022-06-29T15:51:44.344 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on osd down out interval zero = false 2022-06-29T15:51:44.344 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on too few osds = false 2022-06-29T15:51:44.344 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-06-29T15:51:44.345 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_pool_no_redundancy = false 2022-06-29T15:51:44.345 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:44.345 INFO:teuthology.orchestra.run.smithi047.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-06-29T15:51:44.345 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:44.345 INFO:teuthology.orchestra.run.smithi047.stdout: osd default data pool replay window = 5 2022-06-29T15:51:44.346 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:44.346 INFO:teuthology.orchestra.run.smithi047.stdout: mon allow pool delete = true 2022-06-29T15:51:44.346 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:44.346 INFO:teuthology.orchestra.run.smithi047.stdout: mon cluster log file level = debug 2022-06-29T15:51:44.347 INFO:teuthology.orchestra.run.smithi047.stdout: debug asserts on shutdown = true 2022-06-29T15:51:44.347 INFO:teuthology.orchestra.run.smithi047.stdout: mon health detail to clog = false 2022-06-29T15:51:44.347 INFO:teuthology.orchestra.run.smithi047.stdout: mon host = "172.21.15.47,172.21.15.64,[v2:172.21.15.64:3301,v1:172.21.15.64:6790]" 2022-06-29T15:51:44.348 INFO:teuthology.orchestra.run.smithi047.stdout: fsid = 9a68894a-53ce-400e-838b-14eab91b834c 2022-06-29T15:51:44.348 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:44.348 INFO:teuthology.orchestra.run.smithi047.stdout:[osd] 2022-06-29T15:51:44.348 INFO:teuthology.orchestra.run.smithi047.stdout: osd journal size = 100 2022-06-29T15:51:44.349 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:44.349 INFO:teuthology.orchestra.run.smithi047.stdout: osd scrub load threshold = 5.0 2022-06-29T15:51:44.349 INFO:teuthology.orchestra.run.smithi047.stdout: osd scrub max interval = 600 2022-06-29T15:51:44.349 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:44.350 INFO:teuthology.orchestra.run.smithi047.stdout: osd recover clone overlap = true 2022-06-29T15:51:44.350 INFO:teuthology.orchestra.run.smithi047.stdout: osd recovery max chunk = 1048576 2022-06-29T15:51:44.350 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:44.350 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug shutdown = true 2022-06-29T15:51:44.351 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug op order = true 2022-06-29T15:51:44.351 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug verify stray on activate = true 2022-06-29T15:51:44.351 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:44.351 INFO:teuthology.orchestra.run.smithi047.stdout: osd open classes on start = true 2022-06-29T15:51:44.352 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug pg log writeout = true 2022-06-29T15:51:44.352 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:44.352 INFO:teuthology.orchestra.run.smithi047.stdout: osd deep scrub update digest min age = 30 2022-06-29T15:51:44.353 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:44.353 INFO:teuthology.orchestra.run.smithi047.stdout: osd map max advance = 10 2022-06-29T15:51:44.353 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:44.353 INFO:teuthology.orchestra.run.smithi047.stdout: journal zero on create = true 2022-06-29T15:51:44.354 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:44.354 INFO:teuthology.orchestra.run.smithi047.stdout: filestore ondisk finisher threads = 3 2022-06-29T15:51:44.354 INFO:teuthology.orchestra.run.smithi047.stdout: filestore apply finisher threads = 3 2022-06-29T15:51:44.354 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:44.355 INFO:teuthology.orchestra.run.smithi047.stdout: bdev debug aio = true 2022-06-29T15:51:44.355 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug misdirected ops = true 2022-06-29T15:51:44.355 INFO:teuthology.orchestra.run.smithi047.stdout: bluestore block size = 96636764160 2022-06-29T15:51:44.355 INFO:teuthology.orchestra.run.smithi047.stdout: bluestore compression mode = aggressive 2022-06-29T15:51:44.355 INFO:teuthology.orchestra.run.smithi047.stdout: bluestore fsck on mount = True 2022-06-29T15:51:44.356 INFO:teuthology.orchestra.run.smithi047.stdout: debug bluefs = 20 2022-06-29T15:51:44.356 INFO:teuthology.orchestra.run.smithi047.stdout: debug bluestore = 20 2022-06-29T15:51:44.356 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2022-06-29T15:51:44.356 INFO:teuthology.orchestra.run.smithi047.stdout: debug osd = 20 2022-06-29T15:51:44.357 INFO:teuthology.orchestra.run.smithi047.stdout: debug rocksdb = 10 2022-06-29T15:51:44.357 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd backfillfull_ratio = 0.85 2022-06-29T15:51:44.357 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd full ratio = 0.9 2022-06-29T15:51:44.357 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd nearfull ratio = 0.8 2022-06-29T15:51:44.358 INFO:teuthology.orchestra.run.smithi047.stdout: osd failsafe full ratio = 0.95 2022-06-29T15:51:44.358 INFO:teuthology.orchestra.run.smithi047.stdout: osd objectstore = bluestore 2022-06-29T15:51:44.358 INFO:teuthology.orchestra.run.smithi047.stdout: osd op complaint time = 180 2022-06-29T15:51:44.358 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:44.359 INFO:teuthology.orchestra.run.smithi047.stdout:[mgr] 2022-06-29T15:51:44.359 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2022-06-29T15:51:44.359 INFO:teuthology.orchestra.run.smithi047.stdout: debug mgr = 20 2022-06-29T15:51:44.359 INFO:teuthology.orchestra.run.smithi047.stdout: debug mon = 20 2022-06-29T15:51:44.360 INFO:teuthology.orchestra.run.smithi047.stdout: debug auth = 20 2022-06-29T15:51:44.360 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min pgs per osd = 4 2022-06-29T15:51:44.360 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min bytes per osd = 10 2022-06-29T15:51:44.360 INFO:teuthology.orchestra.run.smithi047.stdout: mgr/telemetry/nag = false 2022-06-29T15:51:44.361 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:44.361 INFO:teuthology.orchestra.run.smithi047.stdout:[mon] 2022-06-29T15:51:44.361 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2022-06-29T15:51:44.361 INFO:teuthology.orchestra.run.smithi047.stdout: debug mon = 20 2022-06-29T15:51:44.362 INFO:teuthology.orchestra.run.smithi047.stdout: debug paxos = 20 2022-06-29T15:51:44.362 INFO:teuthology.orchestra.run.smithi047.stdout: debug auth = 20 2022-06-29T15:51:44.362 INFO:teuthology.orchestra.run.smithi047.stdout: mon data avail warn = 5 2022-06-29T15:51:44.362 INFO:teuthology.orchestra.run.smithi047.stdout: mon mgr mkfs grace = 240 2022-06-29T15:51:44.362 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min pgs per osd = 4 2022-06-29T15:51:44.363 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd reporter subtree level = osd 2022-06-29T15:51:44.363 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd prime pg temp = true 2022-06-29T15:51:44.363 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min bytes per osd = 10 2022-06-29T15:51:44.363 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:44.364 INFO:teuthology.orchestra.run.smithi047.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-06-29T15:51:44.364 INFO:teuthology.orchestra.run.smithi047.stdout: auth mon ticket ttl = 660 # 11m 2022-06-29T15:51:44.364 INFO:teuthology.orchestra.run.smithi047.stdout: auth service ticket ttl = 240 # 4m 2022-06-29T15:51:44.364 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:44.365 INFO:teuthology.orchestra.run.smithi047.stdout: # don't complain about insecure global_id in the test suite 2022-06-29T15:51:44.365 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-06-29T15:51:44.365 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-06-29T15:51:44.365 INFO:teuthology.orchestra.run.smithi047.stdout: mon op complaint time = 120 2022-06-29T15:51:44.365 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:44.366 INFO:teuthology.orchestra.run.smithi047.stdout:[client] 2022-06-29T15:51:44.366 INFO:teuthology.orchestra.run.smithi047.stdout: rgw cache enabled = true 2022-06-29T15:51:44.366 INFO:teuthology.orchestra.run.smithi047.stdout: rgw enable ops log = true 2022-06-29T15:51:44.366 INFO:teuthology.orchestra.run.smithi047.stdout: rgw enable usage log = true 2022-06-29T15:51:44.367 INFO:teuthology.orchestra.run.smithi047.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-06-29T15:51:44.367 INFO:teuthology.orchestra.run.smithi047.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-06-29T15:51:44.367 INFO:teuthology.orchestra.run.smithi047.stdout: client mount timeout = 600 2022-06-29T15:51:44.367 INFO:teuthology.orchestra.run.smithi047.stdout: debug client = 20 2022-06-29T15:51:44.368 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2022-06-29T15:51:44.368 INFO:teuthology.orchestra.run.smithi047.stdout: rados mon op timeout = 15m 2022-06-29T15:51:44.368 INFO:teuthology.orchestra.run.smithi047.stdout: rados osd op timeout = 15m 2022-06-29T15:51:44.368 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.a] 2022-06-29T15:51:44.369 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.b] 2022-06-29T15:51:44.369 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.c] 2022-06-29T15:51:44.369 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.a] 2022-06-29T15:51:44.369 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.c] 2022-06-29T15:51:44.369 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.b] 2022-06-29T15:51:44.370 INFO:teuthology.orchestra.run.smithi047.stdout:[mds] 2022-06-29T15:51:44.370 INFO:teuthology.orchestra.run.smithi047.stdout: debug mds = 20 2022-06-29T15:51:44.370 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2022-06-29T15:51:44.370 INFO:teuthology.orchestra.run.smithi047.stdout: mds bal frag = True 2022-06-29T15:51:44.371 INFO:teuthology.orchestra.run.smithi047.stdout: mds bal fragment size max = 10000 2022-06-29T15:51:44.371 INFO:teuthology.orchestra.run.smithi047.stdout: mds bal merge size = 5 2022-06-29T15:51:44.371 INFO:teuthology.orchestra.run.smithi047.stdout: mds bal split bits = 3 2022-06-29T15:51:44.371 INFO:teuthology.orchestra.run.smithi047.stdout: mds bal split size = 100 2022-06-29T15:51:44.372 INFO:teuthology.orchestra.run.smithi047.stdout: mds debug frag = True 2022-06-29T15:51:44.372 INFO:teuthology.orchestra.run.smithi047.stdout: mds debug scatterstat = True 2022-06-29T15:51:44.372 INFO:teuthology.orchestra.run.smithi047.stdout: mds op complaint time = 180 2022-06-29T15:51:44.372 INFO:teuthology.orchestra.run.smithi047.stdout: mds verify scatter = True 2022-06-29T15:51:44.373 INFO:teuthology.orchestra.run.smithi047.stdout: osd op complaint time = 180 2022-06-29T15:51:44.373 INFO:teuthology.orchestra.run.smithi047.stdout: rados mon op timeout = 15m 2022-06-29T15:51:44.373 INFO:teuthology.orchestra.run.smithi047.stdout: rados osd op timeout = 15m 2022-06-29T15:51:44.374 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-06-29T15:51:44.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.482+0000 7f5bf3152700 1 Processor -- start 2022-06-29T15:51:44.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.482+0000 7f5bf3152700 1 -- start start 2022-06-29T15:51:44.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.482+0000 7f5bf3152700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5bec0b6d20 0x7f5bec0b1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:44.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.482+0000 7f5bf3152700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5bec1467a0 0x7f5bec0b1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:44.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.482+0000 7f5bf3152700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bec0b1050 con 0x7f5bec140500 2022-06-29T15:51:44.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.482+0000 7f5bf3152700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bec0b23c0 con 0x7f5bec0b6d20 2022-06-29T15:51:44.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.482+0000 7f5bf3152700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bec05fab0 con 0x7f5bec1467a0 2022-06-29T15:51:44.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.482+0000 7f5bf2150700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5bec0b6d20 0x7f5bec0b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:44.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.482+0000 7f5bf2951700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f5bec140500 0x7f5bec0b2e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39664/0 (socket says 172.21.15.47:39664) 2022-06-29T15:51:44.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.482+0000 7f5bf2951700 1 -- 172.21.15.47:0/1452022030 learned_addr learned my addr 172.21.15.47:0/1452022030 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:44.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.482+0000 7f5bf194f700 1 --2- 172.21.15.47:0/1452022030 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5bec1467a0 0x7f5bec0b1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:44.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.482+0000 7f5bf2150700 1 -- 172.21.15.47:0/1452022030 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5bec1467a0 msgr2=0x7f5bec0b1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:44.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.482+0000 7f5bf2150700 1 --2- 172.21.15.47:0/1452022030 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5bec1467a0 0x7f5bec0b1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:44.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.482+0000 7f5bf2150700 1 -- 172.21.15.47:0/1452022030 >> v1:172.21.15.47:6789/0 conn(0x7f5bec140500 legacy=0x7f5bec0b2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:44.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.482+0000 7f5bf114e700 1 -- 172.21.15.47:0/1452022030 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 923992144 0 0) 0x7f5bec0b1050 con 0x7f5bec140500 2022-06-29T15:51:44.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.482+0000 7f5bf2150700 1 -- 172.21.15.47:0/1452022030 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bec0b7100 con 0x7f5bec0b6d20 2022-06-29T15:51:44.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.486+0000 7f5bf2150700 1 --2- 172.21.15.47:0/1452022030 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5bec0b6d20 0x7f5bec0b1480 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 rx=0x7f5bec0b2700 tx=0x7f5be8006df0).ready entity=mon.0 client_cookie=6430f00a80fc718d server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:44.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.486+0000 7f5bf114e700 1 -- 172.21.15.47:0/1452022030 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5be800d570 con 0x7f5bec0b6d20 2022-06-29T15:51:44.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.486+0000 7f5bf114e700 1 -- 172.21.15.47:0/1452022030 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5be8000e90 con 0x7f5bec0b6d20 2022-06-29T15:51:44.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.486+0000 7f5bf114e700 1 -- 172.21.15.47:0/1452022030 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5be800d8c0 con 0x7f5bec0b6d20 2022-06-29T15:51:44.498 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.486+0000 7f5bf3152700 1 -- 172.21.15.47:0/1452022030 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5bec0b6d20 msgr2=0x7f5bec0b1480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:44.499 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.486+0000 7f5bf3152700 1 --2- 172.21.15.47:0/1452022030 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5bec0b6d20 0x7f5bec0b1480 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 rx=0x7f5bec0b2700 tx=0x7f5be8006df0).stop 2022-06-29T15:51:44.499 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.486+0000 7f5bf3152700 1 -- 172.21.15.47:0/1452022030 shutdown_connections 2022-06-29T15:51:44.499 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.486+0000 7f5bf3152700 1 --2- 172.21.15.47:0/1452022030 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5bec1467a0 0x7f5bec0b1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:44.499 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.486+0000 7f5bf3152700 1 --2- 172.21.15.47:0/1452022030 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5bec0b6d20 0x7f5bec0b1480 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:44.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.486+0000 7f5bf3152700 1 -- 172.21.15.47:0/1452022030 >> 172.21.15.47:0/1452022030 conn(0x7f5bec063670 msgr2=0x7f5bec063a50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:44.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.486+0000 7f5bf3152700 1 -- 172.21.15.47:0/1452022030 shutdown_connections 2022-06-29T15:51:44.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.486+0000 7f5bf3152700 1 -- 172.21.15.47:0/1452022030 wait complete. 2022-06-29T15:51:44.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.486+0000 7f5bf3152700 1 Processor -- start 2022-06-29T15:51:44.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.486+0000 7f5bf3152700 1 -- start start 2022-06-29T15:51:44.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.490+0000 7f5bf3152700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5bec0b6d20 0x7f5bec1456b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:44.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.490+0000 7f5bf3152700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5bec140500 0x7f5bec0ada30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:44.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.490+0000 7f5bf3152700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5bec1467a0 0x7f5bec0adf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:44.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.490+0000 7f5bf3152700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bec0af3e0 con 0x7f5bec140500 2022-06-29T15:51:44.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.490+0000 7f5bf3152700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bec0ad600 con 0x7f5bec0b6d20 2022-06-29T15:51:44.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.490+0000 7f5bf3152700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bec0ac400 con 0x7f5bec1467a0 2022-06-29T15:51:44.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.490+0000 7f5bf194f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5bec140500 0x7f5bec0ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:44.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.490+0000 7f5bf194f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5bec140500 0x7f5bec0ada30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54114/0 (socket says 172.21.15.47:54114) 2022-06-29T15:51:44.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.490+0000 7f5bf194f700 1 -- 172.21.15.47:0/502428913 learned_addr learned my addr 172.21.15.47:0/502428913 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:44.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.490+0000 7f5bf2150700 1 --2- 172.21.15.47:0/502428913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5bec0b6d20 0x7f5bec1456b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:44.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.490+0000 7f5bf2951700 1 --2- 172.21.15.47:0/502428913 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5bec1467a0 0x7f5bec0adf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:44.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.490+0000 7f5bf194f700 1 -- 172.21.15.47:0/502428913 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5bec1467a0 msgr2=0x7f5bec0adf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:44.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.490+0000 7f5bf194f700 1 --2- 172.21.15.47:0/502428913 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5bec1467a0 0x7f5bec0adf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:44.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.490+0000 7f5bf194f700 1 -- 172.21.15.47:0/502428913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5bec0b6d20 msgr2=0x7f5bec1456b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:44.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.490+0000 7f5bf194f700 1 --2- 172.21.15.47:0/502428913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5bec0b6d20 0x7f5bec1456b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:44.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.490+0000 7f5bf194f700 1 -- 172.21.15.47:0/502428913 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be800b180 con 0x7f5bec140500 2022-06-29T15:51:44.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.490+0000 7f5bf194f700 1 --2- 172.21.15.47:0/502428913 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5bec140500 0x7f5bec0ada30 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 rx=0x7f5bdc00c000 tx=0x7f5bdc006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:44.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.490+0000 7f5bdb7fe700 1 -- 172.21.15.47:0/502428913 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5bdc011750 con 0x7f5bec140500 2022-06-29T15:51:44.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.490+0000 7f5bdb7fe700 1 -- 172.21.15.47:0/502428913 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bdc0118b0 con 0x7f5bec140500 2022-06-29T15:51:44.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.490+0000 7f5bf3152700 1 -- 172.21.15.47:0/502428913 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bec0ac6a0 con 0x7f5bec140500 2022-06-29T15:51:44.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.490+0000 7f5bdb7fe700 1 -- 172.21.15.47:0/502428913 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5bdc011c10 con 0x7f5bec140500 2022-06-29T15:51:44.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.490+0000 7f5bf3152700 1 -- 172.21.15.47:0/502428913 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bec0ac170 con 0x7f5bec140500 2022-06-29T15:51:44.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.494+0000 7f5bf3152700 1 -- 172.21.15.47:0/502428913 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bec0610f0 con 0x7f5bec140500 2022-06-29T15:51:44.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.494+0000 7f5bdb7fe700 1 -- 172.21.15.47:0/502428913 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5bdc011d70 con 0x7f5bec140500 2022-06-29T15:51:44.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.494+0000 7f5bdb7fe700 1 --2- 172.21.15.47:0/502428913 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5bcc053a70 0x7f5bcc055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:44.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.494+0000 7f5bdb7fe700 1 -- 172.21.15.47:0/502428913 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f5bdc06c0a0 con 0x7f5bec140500 2022-06-29T15:51:44.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.494+0000 7f5bf2150700 1 --2- 172.21.15.47:0/502428913 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5bcc053a70 0x7f5bcc055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:44.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.494+0000 7f5bf2150700 1 --2- 172.21.15.47:0/502428913 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5bcc053a70 0x7f5bcc055f30 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7f5be801a000 tx=0x7f5be8014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:44.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.498+0000 7f5bdb7fe700 1 -- 172.21.15.47:0/502428913 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5bdc043140 con 0x7f5bec140500 2022-06-29T15:51:44.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.630+0000 7f5bf3152700 1 -- 172.21.15.47:0/502428913 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f5bec0573d0 con 0x7f5bec140500 2022-06-29T15:51:44.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.638+0000 7f5bdb7fe700 1 -- 172.21.15.47:0/502428913 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5bdc046760 con 0x7f5bec140500 2022-06-29T15:51:44.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.638+0000 7f5bf3152700 1 -- 172.21.15.47:0/502428913 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5bcc053a70 msgr2=0x7f5bcc055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:44.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.638+0000 7f5bf3152700 1 --2- 172.21.15.47:0/502428913 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5bcc053a70 0x7f5bcc055f30 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7f5be801a000 tx=0x7f5be8014000).stop 2022-06-29T15:51:44.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.638+0000 7f5bf3152700 1 -- 172.21.15.47:0/502428913 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5bec140500 msgr2=0x7f5bec0ada30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:44.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.638+0000 7f5bf3152700 1 --2- 172.21.15.47:0/502428913 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5bec140500 0x7f5bec0ada30 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 rx=0x7f5bdc00c000 tx=0x7f5bdc006df0).stop 2022-06-29T15:51:44.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.638+0000 7f5bf3152700 1 -- 172.21.15.47:0/502428913 shutdown_connections 2022-06-29T15:51:44.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.638+0000 7f5bf3152700 1 --2- 172.21.15.47:0/502428913 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f5bec1467a0 0x7f5bec0adf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:44.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.638+0000 7f5bf3152700 1 --2- 172.21.15.47:0/502428913 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5bec140500 0x7f5bec0ada30 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:44.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.638+0000 7f5bf3152700 1 --2- 172.21.15.47:0/502428913 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5bcc053a70 0x7f5bcc055f30 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:44.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.638+0000 7f5bf3152700 1 --2- 172.21.15.47:0/502428913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5bec0b6d20 0x7f5bec1456b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:44.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.638+0000 7f5bf3152700 1 -- 172.21.15.47:0/502428913 >> 172.21.15.47:0/502428913 conn(0x7f5bec063670 msgr2=0x7f5bec1490d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:44.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.638+0000 7f5bf3152700 1 -- 172.21.15.47:0/502428913 shutdown_connections 2022-06-29T15:51:44.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.638+0000 7f5bf3152700 1 -- 172.21.15.47:0/502428913 wait complete. 2022-06-29T15:51:44.659 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestMisc.test_getattr_caps' 2022-06-29T15:51:44.761 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.754+0000 7f0f3653b700 1 Processor -- start 2022-06-29T15:51:44.762 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.754+0000 7f0f3653b700 1 -- start start 2022-06-29T15:51:44.764 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.754+0000 7f0f3653b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0f300b2120 0x7f0f300b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:44.764 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.754+0000 7f0f3653b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0f3005f0b0 0x7f0f3014b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:44.764 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.754+0000 7f0f3653b700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0f300b1260 0x7f0f30147af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:44.765 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.754+0000 7f0f3653b700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f30147460 con 0x7f0f300b2120 2022-06-29T15:51:44.765 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.754+0000 7f0f3653b700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f300b0740 con 0x7f0f3005f0b0 2022-06-29T15:51:44.765 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.754+0000 7f0f3653b700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f300b0970 con 0x7f0f300b1260 2022-06-29T15:51:44.765 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.754+0000 7f0f2ffff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0f300b2120 0x7f0f300b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:44.766 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.754+0000 7f0f2ffff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0f300b2120 0x7f0f300b2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54120/0 (socket says 172.21.15.47:54120) 2022-06-29T15:51:44.766 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.754+0000 7f0f2ffff700 1 -- 172.21.15.47:0/3400710662 learned_addr learned my addr 172.21.15.47:0/3400710662 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:44.766 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.754+0000 7f0f34ad8700 1 --2- 172.21.15.47:0/3400710662 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0f300b1260 0x7f0f30147af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:44.767 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.754+0000 7f0f27fff700 1 --2- 172.21.15.47:0/3400710662 >> v2:172.21.15.64:3300/0 conn(0x7f0f3005f0b0 0x7f0f3014b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:44.767 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.758+0000 7f0f2ffff700 1 -- 172.21.15.47:0/3400710662 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0f300b1260 msgr2=0x7f0f30147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:44.767 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.758+0000 7f0f2ffff700 1 --2- 172.21.15.47:0/3400710662 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0f300b1260 0x7f0f30147af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:44.767 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.758+0000 7f0f2ffff700 1 -- 172.21.15.47:0/3400710662 >> v2:172.21.15.64:3300/0 conn(0x7f0f3005f0b0 msgr2=0x7f0f3014b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:44.768 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.758+0000 7f0f2ffff700 1 --2- 172.21.15.47:0/3400710662 >> v2:172.21.15.64:3300/0 conn(0x7f0f3005f0b0 0x7f0f3014b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:44.768 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.758+0000 7f0f2ffff700 1 -- 172.21.15.47:0/3400710662 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f300b0c10 con 0x7f0f300b2120 2022-06-29T15:51:44.768 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.758+0000 7f0f2ffff700 1 --2- 172.21.15.47:0/3400710662 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f300b2120 0x7f0f300b2500 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 rx=0x7f0f20002f70 tx=0x7f0f20006eb0).ready entity=mon.0 client_cookie=510bc926f5587401 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:44.768 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.758+0000 7f0f2f7fe700 1 -- 172.21.15.47:0/3400710662 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f2000d570 con 0x7f0f300b2120 2022-06-29T15:51:44.769 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.758+0000 7f0f2f7fe700 1 -- 172.21.15.47:0/3400710662 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f20000e90 con 0x7f0f300b2120 2022-06-29T15:51:44.769 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.758+0000 7f0f2f7fe700 1 -- 172.21.15.47:0/3400710662 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f2000d900 con 0x7f0f300b2120 2022-06-29T15:51:44.772 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.758+0000 7f0f3653b700 1 -- 172.21.15.47:0/3400710662 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f300b2120 msgr2=0x7f0f300b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:44.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.758+0000 7f0f3653b700 1 --2- 172.21.15.47:0/3400710662 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f300b2120 0x7f0f300b2500 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 rx=0x7f0f20002f70 tx=0x7f0f20006eb0).stop 2022-06-29T15:51:44.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.758+0000 7f0f3653b700 1 -- 172.21.15.47:0/3400710662 shutdown_connections 2022-06-29T15:51:44.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.758+0000 7f0f3653b700 1 --2- 172.21.15.47:0/3400710662 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0f300b1260 0x7f0f30147af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:44.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.758+0000 7f0f3653b700 1 --2- 172.21.15.47:0/3400710662 >> v2:172.21.15.64:3300/0 conn(0x7f0f3005f0b0 0x7f0f3014b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:44.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.758+0000 7f0f3653b700 1 --2- 172.21.15.47:0/3400710662 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f300b2120 0x7f0f300b2500 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:44.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.758+0000 7f0f3653b700 1 -- 172.21.15.47:0/3400710662 >> 172.21.15.47:0/3400710662 conn(0x7f0f300626f0 msgr2=0x7f0f30062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:44.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.758+0000 7f0f3653b700 1 -- 172.21.15.47:0/3400710662 shutdown_connections 2022-06-29T15:51:44.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.758+0000 7f0f3653b700 1 -- 172.21.15.47:0/3400710662 wait complete. 2022-06-29T15:51:44.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.758+0000 7f0f3653b700 1 Processor -- start 2022-06-29T15:51:44.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.758+0000 7f0f3653b700 1 -- start start 2022-06-29T15:51:44.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.762+0000 7f0f3653b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f300b1260 0x7f0f3014e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:44.776 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.762+0000 7f0f3653b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f300b2120 0x7f0f30157340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:44.776 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.762+0000 7f0f3653b700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0f30140740 0x7f0f30140bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:44.776 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.762+0000 7f0f3653b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f300adcd0 con 0x7f0f300b1260 2022-06-29T15:51:44.776 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.762+0000 7f0f3653b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f30157830 con 0x7f0f300b2120 2022-06-29T15:51:44.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.762+0000 7f0f3653b700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f30144010 con 0x7f0f30140740 2022-06-29T15:51:44.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.762+0000 7f0f2ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f300b1260 0x7f0f3014e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:44.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.762+0000 7f0f2ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f300b1260 0x7f0f3014e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54128/0 (socket says 172.21.15.47:54128) 2022-06-29T15:51:44.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.762+0000 7f0f2ffff700 1 -- 172.21.15.47:0/254508836 learned_addr learned my addr 172.21.15.47:0/254508836 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:44.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.762+0000 7f0f34ad8700 1 --2- 172.21.15.47:0/254508836 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0f30140740 0x7f0f30140bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:44.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.762+0000 7f0f27fff700 1 --2- 172.21.15.47:0/254508836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f300b2120 0x7f0f30157340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:44.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.762+0000 7f0f2ffff700 1 -- 172.21.15.47:0/254508836 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0f30140740 msgr2=0x7f0f30140bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:44.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.762+0000 7f0f2ffff700 1 --2- 172.21.15.47:0/254508836 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0f30140740 0x7f0f30140bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:44.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.762+0000 7f0f2ffff700 1 -- 172.21.15.47:0/254508836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f300b2120 msgr2=0x7f0f30157340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:44.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.762+0000 7f0f2ffff700 1 --2- 172.21.15.47:0/254508836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f300b2120 0x7f0f30157340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:44.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.762+0000 7f0f2ffff700 1 -- 172.21.15.47:0/254508836 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f2000b1e0 con 0x7f0f300b1260 2022-06-29T15:51:44.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.762+0000 7f0f2ffff700 1 --2- 172.21.15.47:0/254508836 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f300b1260 0x7f0f3014e090 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 rx=0x7f0f301568f0 tx=0x7f0f20015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:44.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.762+0000 7f0f2dffb700 1 -- 172.21.15.47:0/254508836 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f200117e0 con 0x7f0f300b1260 2022-06-29T15:51:44.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.762+0000 7f0f2dffb700 1 -- 172.21.15.47:0/254508836 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f20011940 con 0x7f0f300b1260 2022-06-29T15:51:44.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.762+0000 7f0f3653b700 1 -- 172.21.15.47:0/254508836 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f301442b0 con 0x7f0f300b1260 2022-06-29T15:51:44.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.762+0000 7f0f2dffb700 1 -- 172.21.15.47:0/254508836 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f20011ca0 con 0x7f0f300b1260 2022-06-29T15:51:44.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.762+0000 7f0f3653b700 1 -- 172.21.15.47:0/254508836 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f301447f0 con 0x7f0f300b1260 2022-06-29T15:51:44.781 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.762+0000 7f0f3653b700 1 -- 172.21.15.47:0/254508836 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f300573d0 con 0x7f0f300b1260 2022-06-29T15:51:44.781 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.766+0000 7f0f2dffb700 1 -- 172.21.15.47:0/254508836 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0f20025030 con 0x7f0f300b1260 2022-06-29T15:51:44.781 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.766+0000 7f0f2dffb700 1 --2- 172.21.15.47:0/254508836 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0f14053980 0x7f0f14055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:44.781 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.766+0000 7f0f2dffb700 1 -- 172.21.15.47:0/254508836 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f0f2006cff0 con 0x7f0f300b1260 2022-06-29T15:51:44.782 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.766+0000 7f0f27fff700 1 --2- 172.21.15.47:0/254508836 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0f14053980 0x7f0f14055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:44.783 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.766+0000 7f0f27fff700 1 --2- 172.21.15.47:0/254508836 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0f14053980 0x7f0f14055e40 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7f0f1800b000 tx=0x7f0f18006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:44.783 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.774+0000 7f0f2dffb700 1 -- 172.21.15.47:0/254508836 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0f20043fe0 con 0x7f0f300b1260 2022-06-29T15:51:44.948 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:44.942+0000 7f0f3653b700 1 -- 172.21.15.47:0/254508836 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_getattr_caps"]} v 0) v1 -- 0x7f0f30146a10 con 0x7f0f300b1260 2022-06-29T15:51:45.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.182+0000 7f0f2dffb700 1 -- 172.21.15.47:0/254508836 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_getattr_caps"]}]=0 v425) v1 ==== 128+0+0 (secure 0 0 0) 0x7f0f20023020 con 0x7f0f300b1260 2022-06-29T15:51:45.192 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.186+0000 7f0f3653b700 1 -- 172.21.15.47:0/254508836 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0f14053980 msgr2=0x7f0f14055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:45.192 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.186+0000 7f0f3653b700 1 --2- 172.21.15.47:0/254508836 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0f14053980 0x7f0f14055e40 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7f0f1800b000 tx=0x7f0f18006df0).stop 2022-06-29T15:51:45.192 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.186+0000 7f0f3653b700 1 -- 172.21.15.47:0/254508836 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f300b1260 msgr2=0x7f0f3014e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:45.192 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.186+0000 7f0f3653b700 1 --2- 172.21.15.47:0/254508836 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f300b1260 0x7f0f3014e090 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 rx=0x7f0f301568f0 tx=0x7f0f20015000).stop 2022-06-29T15:51:45.193 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.186+0000 7f0f3653b700 1 -- 172.21.15.47:0/254508836 shutdown_connections 2022-06-29T15:51:45.193 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.186+0000 7f0f3653b700 1 --2- 172.21.15.47:0/254508836 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0f30140740 0x7f0f30140bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:45.193 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.186+0000 7f0f3653b700 1 --2- 172.21.15.47:0/254508836 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0f14053980 0x7f0f14055e40 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:45.193 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.186+0000 7f0f3653b700 1 --2- 172.21.15.47:0/254508836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f300b2120 0x7f0f30157340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:45.194 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.186+0000 7f0f3653b700 1 --2- 172.21.15.47:0/254508836 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f300b1260 0x7f0f3014e090 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:45.194 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.186+0000 7f0f3653b700 1 -- 172.21.15.47:0/254508836 >> 172.21.15.47:0/254508836 conn(0x7f0f300626f0 msgr2=0x7f0f300b4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:45.194 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.186+0000 7f0f3653b700 1 -- 172.21.15.47:0/254508836 shutdown_connections 2022-06-29T15:51:45.194 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.186+0000 7f0f3653b700 1 -- 172.21.15.47:0/254508836 wait complete. 2022-06-29T15:51:45.206 INFO:tasks.cephfs_test_runner:test_getattr_caps (tasks.cephfs.test_misc.TestMisc) ... ok 2022-06-29T15:51:45.207 INFO:tasks.cephfs_test_runner:Starting test: test_root_rctime (tasks.cephfs.test_misc.TestMisc) 2022-06-29T15:51:45.208 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_root_rctime' 2022-06-29T15:51:45.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.314+0000 7feec4921700 1 Processor -- start 2022-06-29T15:51:45.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.314+0000 7feec4921700 1 -- start start 2022-06-29T15:51:45.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.314+0000 7feec4921700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7feec005c510 0x7feec0143bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:45.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.314+0000 7feec4921700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7feec005c8f0 0x7feec00b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:45.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.314+0000 7feec4921700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7feec00b2c40 0x7feec00b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:45.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.314+0000 7feec4921700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feec0142f90 con 0x7feec005c8f0 2022-06-29T15:51:45.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.314+0000 7feec4921700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feec01431f0 con 0x7feec005c510 2022-06-29T15:51:45.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.314+0000 7feec4921700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feec0143490 con 0x7feec00b2c40 2022-06-29T15:51:45.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.314+0000 7feebeffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7feec005c8f0 0x7feec00b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:45.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.314+0000 7feebf7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7feec005c510 0x7feec0143bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:45.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.314+0000 7feebffff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7feec00b2c40 0x7feec00b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:45.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.314+0000 7feebeffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7feec005c8f0 0x7feec00b4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54138/0 (socket says 172.21.15.47:54138) 2022-06-29T15:51:45.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.314+0000 7feebeffd700 1 -- 172.21.15.47:0/3756372317 learned_addr learned my addr 172.21.15.47:0/3756372317 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:45.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.318+0000 7feebf7fe700 1 -- 172.21.15.47:0/3756372317 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7feec00b2c40 msgr2=0x7feec00b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:45.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.318+0000 7feebf7fe700 1 --2- 172.21.15.47:0/3756372317 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7feec00b2c40 0x7feec00b30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:45.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.318+0000 7feebf7fe700 1 -- 172.21.15.47:0/3756372317 >> v2:172.21.15.47:3300/0 conn(0x7feec005c8f0 msgr2=0x7feec00b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:45.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.318+0000 7feebf7fe700 1 --2- 172.21.15.47:0/3756372317 >> v2:172.21.15.47:3300/0 conn(0x7feec005c8f0 0x7feec00b4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:45.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.318+0000 7feebf7fe700 1 -- 172.21.15.47:0/3756372317 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec0143730 con 0x7feec005c510 2022-06-29T15:51:45.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.318+0000 7feebf7fe700 1 --2- 172.21.15.47:0/3756372317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feec005c510 0x7feec0143bd0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7feeb0002f70 tx=0x7feeb0006df0).ready entity=mon.1 client_cookie=741b716d9afb917e server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:45.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.318+0000 7feebe7fc700 1 -- 172.21.15.47:0/3756372317 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feeb000d4a0 con 0x7feec005c510 2022-06-29T15:51:45.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.318+0000 7feebe7fc700 1 -- 172.21.15.47:0/3756372317 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeb0000e90 con 0x7feec005c510 2022-06-29T15:51:45.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.318+0000 7feebe7fc700 1 -- 172.21.15.47:0/3756372317 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feeb000d7f0 con 0x7feec005c510 2022-06-29T15:51:45.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.318+0000 7feec4921700 1 -- 172.21.15.47:0/3756372317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feec005c510 msgr2=0x7feec0143bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:45.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.318+0000 7feec4921700 1 --2- 172.21.15.47:0/3756372317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feec005c510 0x7feec0143bd0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7feeb0002f70 tx=0x7feeb0006df0).stop 2022-06-29T15:51:45.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.318+0000 7feec4921700 1 -- 172.21.15.47:0/3756372317 shutdown_connections 2022-06-29T15:51:45.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.318+0000 7feec4921700 1 --2- 172.21.15.47:0/3756372317 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7feec00b2c40 0x7feec00b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:45.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.318+0000 7feec4921700 1 --2- 172.21.15.47:0/3756372317 >> v2:172.21.15.47:3300/0 conn(0x7feec005c8f0 0x7feec00b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:45.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.318+0000 7feec4921700 1 --2- 172.21.15.47:0/3756372317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feec005c510 0x7feec0143bd0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:45.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.318+0000 7feec4921700 1 -- 172.21.15.47:0/3756372317 >> 172.21.15.47:0/3756372317 conn(0x7feec0060e00 msgr2=0x7feec00611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:45.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.318+0000 7feec4921700 1 -- 172.21.15.47:0/3756372317 shutdown_connections 2022-06-29T15:51:45.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.318+0000 7feec4921700 1 -- 172.21.15.47:0/3756372317 wait complete. 2022-06-29T15:51:45.335 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.318+0000 7feec4921700 1 Processor -- start 2022-06-29T15:51:45.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.318+0000 7feec4921700 1 -- start start 2022-06-29T15:51:45.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.318+0000 7feec4921700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7feec005c510 0x7feec00afec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:45.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.318+0000 7feec4921700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feec005c8f0 0x7feec0149cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:45.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.318+0000 7feec4921700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feec00b2c40 0x7feec01400f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:45.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.318+0000 7feec4921700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec01449d0 con 0x7feec005c8f0 2022-06-29T15:51:45.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.318+0000 7feec4921700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec0148f20 con 0x7feec00b2c40 2022-06-29T15:51:45.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.318+0000 7feec4921700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feec0148380 con 0x7feec005c510 2022-06-29T15:51:45.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.322+0000 7feebeffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feec005c8f0 0x7feec0149cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:45.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.322+0000 7feebf7fe700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7feec005c510 0x7feec00afec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:45.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.322+0000 7feebeffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feec005c8f0 0x7feec0149cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54142/0 (socket says 172.21.15.47:54142) 2022-06-29T15:51:45.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.322+0000 7feebffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feec00b2c40 0x7feec01400f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:45.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.322+0000 7feebeffd700 1 -- 172.21.15.47:0/3301130464 learned_addr learned my addr 172.21.15.47:0/3301130464 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:45.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.322+0000 7feebeffd700 1 -- 172.21.15.47:0/3301130464 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7feec005c510 msgr2=0x7feec00afec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:45.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.322+0000 7feebeffd700 1 --2- 172.21.15.47:0/3301130464 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7feec005c510 0x7feec00afec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:45.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.322+0000 7feebeffd700 1 -- 172.21.15.47:0/3301130464 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feec00b2c40 msgr2=0x7feec01400f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:45.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.322+0000 7feebeffd700 1 --2- 172.21.15.47:0/3301130464 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feec00b2c40 0x7feec01400f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:45.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.322+0000 7feebeffd700 1 -- 172.21.15.47:0/3301130464 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb000b180 con 0x7feec005c8f0 2022-06-29T15:51:45.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.322+0000 7feebf7fe700 1 --2- 172.21.15.47:0/3301130464 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7feec005c510 0x7feec00afec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:51:45.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.322+0000 7feebeffd700 1 --2- 172.21.15.47:0/3301130464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feec005c8f0 0x7feec0149cd0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 rx=0x7feeb400c000 tx=0x7feeb4006eb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:45.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.322+0000 7feebcff9700 1 -- 172.21.15.47:0/3301130464 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feeb4012600 con 0x7feec005c8f0 2022-06-29T15:51:45.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.322+0000 7feec4921700 1 -- 172.21.15.47:0/3301130464 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feec0148620 con 0x7feec005c8f0 2022-06-29T15:51:45.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.322+0000 7feebcff9700 1 -- 172.21.15.47:0/3301130464 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeb4012760 con 0x7feec005c8f0 2022-06-29T15:51:45.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.322+0000 7feebcff9700 1 -- 172.21.15.47:0/3301130464 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feeb4012ac0 con 0x7feec005c8f0 2022-06-29T15:51:45.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.322+0000 7feec4921700 1 -- 172.21.15.47:0/3301130464 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feec01446c0 con 0x7feec005c8f0 2022-06-29T15:51:45.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.322+0000 7feec4921700 1 -- 172.21.15.47:0/3301130464 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feec00573d0 con 0x7feec005c8f0 2022-06-29T15:51:45.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.322+0000 7feebcff9700 1 -- 172.21.15.47:0/3301130464 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7feeb4022070 con 0x7feec005c8f0 2022-06-29T15:51:45.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.326+0000 7feebcff9700 1 --2- 172.21.15.47:0/3301130464 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7feea0053980 0x7feea0055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:45.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.326+0000 7feebcff9700 1 -- 172.21.15.47:0/3301130464 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5686+0+0 (secure 0 0 0) 0x7feeb406c110 con 0x7feec005c8f0 2022-06-29T15:51:45.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.326+0000 7feebf7fe700 1 --2- 172.21.15.47:0/3301130464 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7feea0053980 0x7feea0055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:45.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.326+0000 7feebf7fe700 1 --2- 172.21.15.47:0/3301130464 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7feea0053980 0x7feea0055e40 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7feeb001a000 tx=0x7feeb0014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:45.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.330+0000 7feebcff9700 1 -- 172.21.15.47:0/3301130464 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7feeb4043110 con 0x7feec005c8f0 2022-06-29T15:51:45.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:45.498+0000 7feec4921700 1 -- 172.21.15.47:0/3301130464 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_root_rctime"]} v 0) v1 -- 0x7feec0061b00 con 0x7feec005c8f0 2022-06-29T15:51:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.182+0000 7feebcff9700 1 -- 172.21.15.47:0/3301130464 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_root_rctime"]}]=0 v426) v1 ==== 130+0+0 (secure 0 0 0) 0x7feeb4046730 con 0x7feec005c8f0 2022-06-29T15:51:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.186+0000 7feec4921700 1 -- 172.21.15.47:0/3301130464 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7feea0053980 msgr2=0x7feea0055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.186+0000 7feec4921700 1 --2- 172.21.15.47:0/3301130464 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7feea0053980 0x7feea0055e40 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7feeb001a000 tx=0x7feeb0014000).stop 2022-06-29T15:51:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.186+0000 7feec4921700 1 -- 172.21.15.47:0/3301130464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feec005c8f0 msgr2=0x7feec0149cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.186+0000 7feec4921700 1 --2- 172.21.15.47:0/3301130464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feec005c8f0 0x7feec0149cd0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 rx=0x7feeb400c000 tx=0x7feeb4006eb0).stop 2022-06-29T15:51:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.186+0000 7feec4921700 1 -- 172.21.15.47:0/3301130464 shutdown_connections 2022-06-29T15:51:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.186+0000 7feec4921700 1 --2- 172.21.15.47:0/3301130464 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7feea0053980 0x7feea0055e40 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.186+0000 7feec4921700 1 --2- 172.21.15.47:0/3301130464 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feec00b2c40 0x7feec01400f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.186+0000 7feec4921700 1 --2- 172.21.15.47:0/3301130464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feec005c8f0 0x7feec0149cd0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.186+0000 7feec4921700 1 --2- 172.21.15.47:0/3301130464 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7feec005c510 0x7feec00afec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.186+0000 7feec4921700 1 -- 172.21.15.47:0/3301130464 >> 172.21.15.47:0/3301130464 conn(0x7feec0060e00 msgr2=0x7feec00b7600 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.186+0000 7feec4921700 1 -- 172.21.15.47:0/3301130464 shutdown_connections 2022-06-29T15:51:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.186+0000 7feec4921700 1 -- 172.21.15.47:0/3301130464 wait complete. 2022-06-29T15:51:46.211 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-06-29T15:51:46.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.322+0000 7f0de4715700 1 Processor -- start 2022-06-29T15:51:46.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.322+0000 7f0de4715700 1 -- start start 2022-06-29T15:51:46.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.322+0000 7f0de4715700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0ddc0612c0 0x7f0ddc1471a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:46.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.322+0000 7f0de4715700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ddc067610 con 0x7f0ddc0b4300 2022-06-29T15:51:46.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.322+0000 7f0de4715700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ddc1467c0 con 0x7f0ddc060ee0 2022-06-29T15:51:46.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.322+0000 7f0de4715700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ddc146a60 con 0x7f0ddc0612c0 2022-06-29T15:51:46.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.322+0000 7f0de1cb0700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0ddc0b4300 0x7f0ddc13cdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39704/0 (socket says 172.21.15.47:39704) 2022-06-29T15:51:46.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.322+0000 7f0de1cb0700 1 -- 172.21.15.47:0/811597609 learned_addr learned my addr 172.21.15.47:0/811597609 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:46.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.322+0000 7f0de2cb2700 1 --2- 172.21.15.47:0/811597609 >> v2:172.21.15.64:3300/0 conn(0x7f0ddc0612c0 0x7f0ddc1471a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:46.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.322+0000 7f0de14af700 1 -- 172.21.15.47:0/811597609 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 197972902 0 0) 0x7f0ddc067610 con 0x7f0ddc0b4300 2022-06-29T15:51:46.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.322+0000 7f0de14af700 1 -- 172.21.15.47:0/811597609 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0dc4003260 con 0x7f0ddc0b4300 2022-06-29T15:51:46.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.322+0000 7f0de2cb2700 1 -- 172.21.15.47:0/811597609 >> v1:172.21.15.64:6789/0 conn(0x7f0ddc060ee0 legacy=0x7f0ddc1430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:46.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.322+0000 7f0de2cb2700 1 -- 172.21.15.47:0/811597609 >> v1:172.21.15.47:6789/0 conn(0x7f0ddc0b4300 legacy=0x7f0ddc13cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:46.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.322+0000 7f0de2cb2700 1 -- 172.21.15.47:0/811597609 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ddc146d00 con 0x7f0ddc0612c0 2022-06-29T15:51:46.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.322+0000 7f0de2cb2700 1 --2- 172.21.15.47:0/811597609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ddc0612c0 0x7f0ddc1471a0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7f0ddc0ae790 tx=0x7f0dd40091d0).ready entity=mon.1 client_cookie=9a9272fc1f2edc96 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:46.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.322+0000 7f0de14af700 1 -- 172.21.15.47:0/811597609 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0dd400b680 con 0x7f0ddc0612c0 2022-06-29T15:51:46.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.322+0000 7f0de14af700 1 -- 172.21.15.47:0/811597609 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dd400b7e0 con 0x7f0ddc0612c0 2022-06-29T15:51:46.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.322+0000 7f0de14af700 1 -- 172.21.15.47:0/811597609 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0dd400b950 con 0x7f0ddc0612c0 2022-06-29T15:51:46.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.326+0000 7f0de4715700 1 -- 172.21.15.47:0/811597609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ddc0612c0 msgr2=0x7f0ddc1471a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:46.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.326+0000 7f0de4715700 1 --2- 172.21.15.47:0/811597609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ddc0612c0 0x7f0ddc1471a0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7f0ddc0ae790 tx=0x7f0dd40091d0).stop 2022-06-29T15:51:46.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.326+0000 7f0de4715700 1 -- 172.21.15.47:0/811597609 shutdown_connections 2022-06-29T15:51:46.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.326+0000 7f0de4715700 1 --2- 172.21.15.47:0/811597609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ddc0612c0 0x7f0ddc1471a0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:46.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.326+0000 7f0de4715700 1 -- 172.21.15.47:0/811597609 >> 172.21.15.47:0/811597609 conn(0x7f0ddc05c510 msgr2=0x7f0ddc05f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:46.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.326+0000 7f0de4715700 1 -- 172.21.15.47:0/811597609 shutdown_connections 2022-06-29T15:51:46.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.326+0000 7f0de4715700 1 -- 172.21.15.47:0/811597609 wait complete. 2022-06-29T15:51:46.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.326+0000 7f0de4715700 1 Processor -- start 2022-06-29T15:51:46.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.326+0000 7f0de4715700 1 -- start start 2022-06-29T15:51:46.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.330+0000 7f0de4715700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ddc060ee0 0x7f0ddc0b3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:46.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.330+0000 7f0de4715700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ddc0612c0 0x7f0ddc0b0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:46.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.330+0000 7f0de4715700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0ddc0b4300 0x7f0ddc0b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:46.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.330+0000 7f0de4715700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ddc1530a0 con 0x7f0ddc0612c0 2022-06-29T15:51:46.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.330+0000 7f0de4715700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ddc141a70 con 0x7f0ddc060ee0 2022-06-29T15:51:46.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.330+0000 7f0de4715700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ddc0b16b0 con 0x7f0ddc0b4300 2022-06-29T15:51:46.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.330+0000 7f0de1cb0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ddc0612c0 0x7f0ddc0b0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:46.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.330+0000 7f0de1cb0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ddc0612c0 0x7f0ddc0b0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54156/0 (socket says 172.21.15.47:54156) 2022-06-29T15:51:46.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.330+0000 7f0de1cb0700 1 -- 172.21.15.47:0/3671294062 learned_addr learned my addr 172.21.15.47:0/3671294062 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:46.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.330+0000 7f0de24b1700 1 --2- 172.21.15.47:0/3671294062 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ddc060ee0 0x7f0ddc0b3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:46.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.330+0000 7f0de2cb2700 1 --2- 172.21.15.47:0/3671294062 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0ddc0b4300 0x7f0ddc0b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:46.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.330+0000 7f0de1cb0700 1 -- 172.21.15.47:0/3671294062 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0ddc0b4300 msgr2=0x7f0ddc0b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:46.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.330+0000 7f0de1cb0700 1 --2- 172.21.15.47:0/3671294062 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0ddc0b4300 0x7f0ddc0b11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:46.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.330+0000 7f0de1cb0700 1 -- 172.21.15.47:0/3671294062 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ddc060ee0 msgr2=0x7f0ddc0b3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:46.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.330+0000 7f0de1cb0700 1 --2- 172.21.15.47:0/3671294062 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ddc060ee0 0x7f0ddc0b3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:46.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.330+0000 7f0de1cb0700 1 -- 172.21.15.47:0/3671294062 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd4009470 con 0x7f0ddc0612c0 2022-06-29T15:51:46.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.330+0000 7f0de1cb0700 1 --2- 172.21.15.47:0/3671294062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ddc0612c0 0x7f0ddc0b0cd0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 rx=0x7f0dcc001110 tx=0x7f0dcc007aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:46.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.330+0000 7f0dcb7fe700 1 -- 172.21.15.47:0/3671294062 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0dcc00ce00 con 0x7f0ddc0612c0 2022-06-29T15:51:46.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.330+0000 7f0dcb7fe700 1 -- 172.21.15.47:0/3671294062 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dcc0070d0 con 0x7f0ddc0612c0 2022-06-29T15:51:46.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.330+0000 7f0dcb7fe700 1 -- 172.21.15.47:0/3671294062 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0dcc007430 con 0x7f0ddc0612c0 2022-06-29T15:51:46.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.330+0000 7f0de4715700 1 -- 172.21.15.47:0/3671294062 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ddc152dc0 con 0x7f0ddc0612c0 2022-06-29T15:51:46.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.330+0000 7f0de4715700 1 -- 172.21.15.47:0/3671294062 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ddc140140 con 0x7f0ddc0612c0 2022-06-29T15:51:46.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.334+0000 7f0de4715700 1 -- 172.21.15.47:0/3671294062 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ddc1532a0 con 0x7f0ddc0612c0 2022-06-29T15:51:46.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.334+0000 7f0dcb7fe700 1 -- 172.21.15.47:0/3671294062 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0dcc010070 con 0x7f0ddc0612c0 2022-06-29T15:51:46.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.334+0000 7f0dcb7fe700 1 --2- 172.21.15.47:0/3671294062 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0dbc053b30 0x7f0dbc055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:46.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.334+0000 7f0de24b1700 1 --2- 172.21.15.47:0/3671294062 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0dbc053b30 0x7f0dbc055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:46.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.334+0000 7f0dcb7fe700 1 -- 172.21.15.47:0/3671294062 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f0dcc06b3c0 con 0x7f0ddc0612c0 2022-06-29T15:51:46.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.334+0000 7f0de24b1700 1 --2- 172.21.15.47:0/3671294062 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0dbc053b30 0x7f0dbc055ff0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7f0dd800d000 tx=0x7f0dd8007b90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:46.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.342+0000 7f0dcb7fe700 1 -- 172.21.15.47:0/3671294062 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0dcc042350 con 0x7f0ddc0612c0 2022-06-29T15:51:46.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.470+0000 7f0de4715700 1 -- 172.21.15.47:0/3671294062 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0ddc0573d0 con 0x7f0ddc0612c0 2022-06-29T15:51:46.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.478+0000 7f0dcb7fe700 1 -- 172.21.15.47:0/3671294062 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0dcc018030 con 0x7f0ddc0612c0 2022-06-29T15:51:46.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.482+0000 7f0de4715700 1 -- 172.21.15.47:0/3671294062 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0dbc053b30 msgr2=0x7f0dbc055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:46.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.482+0000 7f0de4715700 1 --2- 172.21.15.47:0/3671294062 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0dbc053b30 0x7f0dbc055ff0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7f0dd800d000 tx=0x7f0dd8007b90).stop 2022-06-29T15:51:46.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.482+0000 7f0de4715700 1 -- 172.21.15.47:0/3671294062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ddc0612c0 msgr2=0x7f0ddc0b0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:46.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.482+0000 7f0de4715700 1 --2- 172.21.15.47:0/3671294062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ddc0612c0 0x7f0ddc0b0cd0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 rx=0x7f0dcc001110 tx=0x7f0dcc007aa0).stop 2022-06-29T15:51:46.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.482+0000 7f0de4715700 1 -- 172.21.15.47:0/3671294062 shutdown_connections 2022-06-29T15:51:46.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.482+0000 7f0de4715700 1 --2- 172.21.15.47:0/3671294062 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0ddc0b4300 0x7f0ddc0b11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:46.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.482+0000 7f0de4715700 1 --2- 172.21.15.47:0/3671294062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ddc0612c0 0x7f0ddc0b0cd0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:46.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.482+0000 7f0de4715700 1 --2- 172.21.15.47:0/3671294062 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0dbc053b30 0x7f0dbc055ff0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:46.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.482+0000 7f0de4715700 1 --2- 172.21.15.47:0/3671294062 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ddc060ee0 0x7f0ddc0b3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:46.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.482+0000 7f0de4715700 1 -- 172.21.15.47:0/3671294062 >> 172.21.15.47:0/3671294062 conn(0x7f0ddc05c510 msgr2=0x7f0ddc1404f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:46.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.482+0000 7f0de4715700 1 -- 172.21.15.47:0/3671294062 shutdown_connections 2022-06-29T15:51:46.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.482+0000 7f0de4715700 1 -- 172.21.15.47:0/3671294062 wait complete. 2022-06-29T15:51:46.506 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-06-29T15:51:46.506 DEBUG:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:51:46.521 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:51:46.543 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:51:46.565 DEBUG:teuthology.orchestra.run.smithi164:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:51:46.586 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-06-29T15:51:46.587 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:51:46.588 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:51:46.588 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:51:46.595 INFO:teuthology.orchestra.run.smithi099.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-06-29T15:51:46.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T15:51:46.597 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-06-29T15:51:46.598 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:51:46.598 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:51:46.598 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:51:46.606 INFO:teuthology.orchestra.run.smithi164.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-06-29T15:51:46.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T15:51:46.607 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-06-29T15:51:46.608 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-06-29T15:51:46.723 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.718+0000 7effc255b700 1 Processor -- start 2022-06-29T15:51:46.724 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.718+0000 7effc255b700 1 -- start start 2022-06-29T15:51:46.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.718+0000 7effc255b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7effbc0af590 0x7effbc0ad0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:46.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.718+0000 7effc255b700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effbc0b1850 con 0x7effbc0b1470 2022-06-29T15:51:46.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.718+0000 7effc255b700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effbc142d50 con 0x7effbc0b04e0 2022-06-29T15:51:46.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.718+0000 7effc255b700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effbc0b27c0 con 0x7effbc0af590 2022-06-29T15:51:46.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.718+0000 7effc1559700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7effbc0b1470 0x7effbc0ac9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39716/0 (socket says 172.21.15.47:39716) 2022-06-29T15:51:46.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.718+0000 7effc1559700 1 -- 172.21.15.47:0/2063974523 learned_addr learned my addr 172.21.15.47:0/2063974523 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:46.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.718+0000 7effc1d5a700 1 --2- 172.21.15.47:0/2063974523 >> v2:172.21.15.64:3300/0 conn(0x7effbc0af590 0x7effbc0ad0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:46.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.718+0000 7effabfff700 1 -- 172.21.15.47:0/2063974523 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2937986203 0 0) 0x7effbc0b1850 con 0x7effbc0b1470 2022-06-29T15:51:46.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.718+0000 7effabfff700 1 -- 172.21.15.47:0/2063974523 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effa4003260 con 0x7effbc0b1470 2022-06-29T15:51:46.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.718+0000 7effabfff700 1 -- 172.21.15.47:0/2063974523 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1996490107 0 0) 0x7effbc142d50 con 0x7effbc0b04e0 2022-06-29T15:51:46.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.718+0000 7effabfff700 1 -- 172.21.15.47:0/2063974523 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effbc0b1850 con 0x7effbc0b04e0 2022-06-29T15:51:46.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.718+0000 7effc1d5a700 1 -- 172.21.15.47:0/2063974523 >> v1:172.21.15.64:6789/0 conn(0x7effbc0b04e0 legacy=0x7effbc143090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:46.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.718+0000 7effabfff700 1 -- 172.21.15.47:0/2063974523 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1455411932 0 0) 0x7effa4003260 con 0x7effbc0b1470 2022-06-29T15:51:46.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.718+0000 7effc1d5a700 1 -- 172.21.15.47:0/2063974523 >> v1:172.21.15.47:6789/0 conn(0x7effbc0b1470 legacy=0x7effbc0ac9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:46.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.718+0000 7effc1d5a700 1 -- 172.21.15.47:0/2063974523 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effbc0ae9a0 con 0x7effbc0af590 2022-06-29T15:51:46.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.718+0000 7effc1d5a700 1 --2- 172.21.15.47:0/2063974523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effbc0af590 0x7effbc0ad0d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7effb8001290 tx=0x7effb40091d0).ready entity=mon.1 client_cookie=3594eb9bb19da95b server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:46.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.718+0000 7effabfff700 1 -- 172.21.15.47:0/2063974523 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effb80017e0 con 0x7effbc0af590 2022-06-29T15:51:46.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.718+0000 7effabfff700 1 -- 172.21.15.47:0/2063974523 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb400f510 con 0x7effbc0af590 2022-06-29T15:51:46.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.718+0000 7effabfff700 1 -- 172.21.15.47:0/2063974523 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effb400f680 con 0x7effbc0af590 2022-06-29T15:51:46.734 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.718+0000 7effc255b700 1 -- 172.21.15.47:0/2063974523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effbc0af590 msgr2=0x7effbc0ad0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:46.734 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.718+0000 7effc255b700 1 --2- 172.21.15.47:0/2063974523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effbc0af590 0x7effbc0ad0d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7effb8001290 tx=0x7effb40091d0).stop 2022-06-29T15:51:46.734 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.718+0000 7effc255b700 1 -- 172.21.15.47:0/2063974523 shutdown_connections 2022-06-29T15:51:46.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.718+0000 7effc255b700 1 --2- 172.21.15.47:0/2063974523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effbc0af590 0x7effbc0ad0d0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:46.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.718+0000 7effc255b700 1 -- 172.21.15.47:0/2063974523 >> 172.21.15.47:0/2063974523 conn(0x7effbc062560 msgr2=0x7effbc062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:46.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.722+0000 7effc255b700 1 -- 172.21.15.47:0/2063974523 shutdown_connections 2022-06-29T15:51:46.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.722+0000 7effc255b700 1 -- 172.21.15.47:0/2063974523 wait complete. 2022-06-29T15:51:46.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.722+0000 7effc255b700 1 Processor -- start 2022-06-29T15:51:46.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.722+0000 7effc255b700 1 -- start start 2022-06-29T15:51:46.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.722+0000 7effc255b700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7effbc0af590 0x7effbc066b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:46.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.722+0000 7effc255b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effbc0b04e0 0x7effbc064940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:46.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.722+0000 7effc255b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effbc0b1470 0x7effbc064e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:46.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.722+0000 7effc255b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effbc065fb0 con 0x7effbc0b1470 2022-06-29T15:51:46.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.722+0000 7effc255b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effbc0a93d0 con 0x7effbc0b04e0 2022-06-29T15:51:46.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.722+0000 7effc255b700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effbc152550 con 0x7effbc0af590 2022-06-29T15:51:46.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.722+0000 7effc1d5a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effbc0b1470 0x7effbc064e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:46.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.722+0000 7effc1559700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7effbc0af590 0x7effbc066b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:46.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.722+0000 7effc1d5a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effbc0b1470 0x7effbc064e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54172/0 (socket says 172.21.15.47:54172) 2022-06-29T15:51:46.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.722+0000 7effc1d5a700 1 -- 172.21.15.47:0/353299207 learned_addr learned my addr 172.21.15.47:0/353299207 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:46.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.722+0000 7effc0d58700 1 --2- 172.21.15.47:0/353299207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effbc0b04e0 0x7effbc064940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:46.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.722+0000 7effc1d5a700 1 -- 172.21.15.47:0/353299207 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7effbc0af590 msgr2=0x7effbc066b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:46.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.722+0000 7effc1d5a700 1 --2- 172.21.15.47:0/353299207 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7effbc0af590 0x7effbc066b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:46.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.722+0000 7effc1d5a700 1 -- 172.21.15.47:0/353299207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effbc0b04e0 msgr2=0x7effbc064940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:46.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.722+0000 7effc1d5a700 1 --2- 172.21.15.47:0/353299207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effbc0b04e0 0x7effbc064940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:46.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.722+0000 7effc1d5a700 1 -- 172.21.15.47:0/353299207 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb4009470 con 0x7effbc0b1470 2022-06-29T15:51:46.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.722+0000 7effc1d5a700 1 --2- 172.21.15.47:0/353299207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effbc0b1470 0x7effbc064e30 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 rx=0x7effbc0ab490 tx=0x7effb4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:46.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.722+0000 7effaa7fc700 1 -- 172.21.15.47:0/353299207 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effb40032d0 con 0x7effbc0b1470 2022-06-29T15:51:46.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.722+0000 7effaa7fc700 1 -- 172.21.15.47:0/353299207 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb4011b70 con 0x7effbc0b1470 2022-06-29T15:51:46.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.722+0000 7effc255b700 1 -- 172.21.15.47:0/353299207 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effbc1522b0 con 0x7effbc0b1470 2022-06-29T15:51:46.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.722+0000 7effaa7fc700 1 -- 172.21.15.47:0/353299207 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effb4021440 con 0x7effbc0b1470 2022-06-29T15:51:46.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.722+0000 7effc255b700 1 -- 172.21.15.47:0/353299207 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effbc0ab670 con 0x7effbc0b1470 2022-06-29T15:51:46.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.726+0000 7effc255b700 1 -- 172.21.15.47:0/353299207 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effbc0573d0 con 0x7effbc0b1470 2022-06-29T15:51:46.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.726+0000 7effaa7fc700 1 -- 172.21.15.47:0/353299207 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7effb4027020 con 0x7effbc0b1470 2022-06-29T15:51:46.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.726+0000 7effaa7fc700 1 --2- 172.21.15.47:0/353299207 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7eff9c053950 0x7eff9c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:46.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.726+0000 7effaa7fc700 1 -- 172.21.15.47:0/353299207 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5686+0+0 (secure 0 0 0) 0x7effb406ce60 con 0x7effbc0b1470 2022-06-29T15:51:46.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.726+0000 7effc1559700 1 --2- 172.21.15.47:0/353299207 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7eff9c053950 0x7eff9c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:46.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.726+0000 7effc1559700 1 --2- 172.21.15.47:0/353299207 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7eff9c053950 0x7eff9c055e10 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7effb800d000 tx=0x7effb80018d0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:46.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.730+0000 7effaa7fc700 1 -- 172.21.15.47:0/353299207 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7effb4043e70 con 0x7effbc0b1470 2022-06-29T15:51:46.866 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.858+0000 7effc255b700 1 -- 172.21.15.47:0/353299207 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7effbc069e20 con 0x7effbc0b1470 2022-06-29T15:51:46.868 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.862+0000 7effaa7fc700 1 -- 172.21.15.47:0/353299207 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v143) v1 ==== 81+0+29229 (secure 0 0 0) 0x7effb4015090 con 0x7effbc0b1470 2022-06-29T15:51:46.869 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:46.870 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2022-06-29T15:51:46.870 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 143, 2022-06-29T15:51:46.870 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "9a68894a-53ce-400e-838b-14eab91b834c", 2022-06-29T15:51:46.870 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2022-06-29T15:42:14.046361+0000", 2022-06-29T15:51:46.871 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2022-06-29T15:51:44.163141+0000", 2022-06-29T15:51:46.871 INFO:teuthology.orchestra.run.smithi047.stdout: "last_up_change": "2022-06-29T15:42:20.929789+0000", 2022-06-29T15:51:46.871 INFO:teuthology.orchestra.run.smithi047.stdout: "last_in_change": "2022-06-29T15:42:20.929789+0000", 2022-06-29T15:51:46.871 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-06-29T15:51:46.872 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_num": 5799936, 2022-06-29T15:51:46.872 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_set": [ 2022-06-29T15:51:46.872 INFO:teuthology.orchestra.run.smithi047.stdout: "pglog_hardlimit", 2022-06-29T15:51:46.872 INFO:teuthology.orchestra.run.smithi047.stdout: "purged_snapdirs", 2022-06-29T15:51:46.873 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_deletes", 2022-06-29T15:51:46.873 INFO:teuthology.orchestra.run.smithi047.stdout: "sortbitwise" 2022-06-29T15:51:46.873 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:51:46.873 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_version": 5, 2022-06-29T15:51:46.874 INFO:teuthology.orchestra.run.smithi047.stdout: "full_ratio": 0.94999998807907104, 2022-06-29T15:51:46.874 INFO:teuthology.orchestra.run.smithi047.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-06-29T15:51:46.874 INFO:teuthology.orchestra.run.smithi047.stdout: "nearfull_ratio": 0.85000002384185791, 2022-06-29T15:51:46.874 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_snapshot": "", 2022-06-29T15:51:46.875 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_max": 26, 2022-06-29T15:51:46.875 INFO:teuthology.orchestra.run.smithi047.stdout: "max_osd": 8, 2022-06-29T15:51:46.875 INFO:teuthology.orchestra.run.smithi047.stdout: "require_min_compat_client": "jewel", 2022-06-29T15:51:46.875 INFO:teuthology.orchestra.run.smithi047.stdout: "min_compat_client": "jewel", 2022-06-29T15:51:46.875 INFO:teuthology.orchestra.run.smithi047.stdout: "require_osd_release": "octopus", 2022-06-29T15:51:46.876 INFO:teuthology.orchestra.run.smithi047.stdout: "pools": [ 2022-06-29T15:51:46.876 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:46.876 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 1, 2022-06-29T15:51:46.876 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "device_health_metrics", 2022-06-29T15:51:46.877 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:42:16.971431+0000", 2022-06-29T15:51:46.877 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:51:46.877 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:51:46.877 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:51:46.878 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:51:46.878 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:51:46.878 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:51:46.878 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:51:46.879 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:51:46.879 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 1, 2022-06-29T15:51:46.879 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 1, 2022-06-29T15:51:46.879 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 1, 2022-06-29T15:51:46.879 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 1, 2022-06-29T15:51:46.880 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 1, 2022-06-29T15:51:46.880 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:51:46.880 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:51:46.880 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:51:46.881 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:51:46.881 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:51:46.881 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:51:46.881 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:51:46.881 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:46.882 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "16", 2022-06-29T15:51:46.882 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:51:46.882 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:51:46.882 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:51:46.883 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:51:46.883 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:51:46.883 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:51:46.883 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:51:46.884 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:51:46.884 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:51:46.884 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:51:46.884 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:51:46.884 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:51:46.885 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:51:46.885 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:51:46.885 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:51:46.885 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:51:46.886 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:51:46.886 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:51:46.886 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:51:46.886 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:51:46.887 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:51:46.887 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:51:46.887 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:51:46.887 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:51:46.888 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:51:46.888 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:51:46.888 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:46.888 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:51:46.888 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:51:46.889 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:51:46.889 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:51:46.889 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:51:46.889 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:51:46.890 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:51:46.890 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:51:46.890 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:51:46.890 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:51:46.891 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:51:46.891 INFO:teuthology.orchestra.run.smithi047.stdout: "options": { 2022-06-29T15:51:46.891 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_min": 1 2022-06-29T15:51:46.891 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:46.891 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:51:46.892 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_devicehealth": {} 2022-06-29T15:51:46.892 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:46.892 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:46.892 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:46.893 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 2, 2022-06-29T15:51:46.893 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "rbd", 2022-06-29T15:51:46.893 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:42:25.290884+0000", 2022-06-29T15:51:46.893 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:51:46.893 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:51:46.894 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:51:46.894 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:51:46.894 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:51:46.894 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:51:46.895 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:51:46.895 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:51:46.895 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2022-06-29T15:51:46.895 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2022-06-29T15:51:46.896 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2022-06-29T15:51:46.896 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2022-06-29T15:51:46.896 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2022-06-29T15:51:46.896 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:51:46.896 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:51:46.897 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:51:46.897 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:51:46.897 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:51:46.897 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:51:46.898 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:51:46.898 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:46.898 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "19", 2022-06-29T15:51:46.898 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:51:46.899 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:51:46.899 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:51:46.899 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:51:46.899 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:51:46.899 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:51:46.900 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:51:46.900 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:51:46.900 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:51:46.900 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:51:46.901 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:51:46.901 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:51:46.901 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:51:46.901 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:51:46.902 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:51:46.902 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:51:46.902 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:51:46.902 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:51:46.903 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:51:46.903 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:51:46.903 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:51:46.903 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:51:46.903 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:51:46.904 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:51:46.904 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:51:46.904 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:51:46.904 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:46.905 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:51:46.905 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:51:46.905 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:51:46.905 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:51:46.905 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:51:46.906 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:51:46.906 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:51:46.906 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:51:46.907 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:51:46.907 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:51:46.907 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:51:46.907 INFO:teuthology.orchestra.run.smithi047.stdout: "options": {}, 2022-06-29T15:51:46.907 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:51:46.908 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd": {} 2022-06-29T15:51:46.908 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:46.908 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:46.908 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:51:46.908 INFO:teuthology.orchestra.run.smithi047.stdout: "osds": [ 2022-06-29T15:51:46.909 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:46.909 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2022-06-29T15:51:46.909 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "e23eb023-4706-4201-bf72-4bf19c912c5d", 2022-06-29T15:51:46.909 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:51:46.910 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:51:46.910 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:51:46.910 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:51:46.910 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:51:46.911 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:51:46.911 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:51:46.911 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 137, 2022-06-29T15:51:46.911 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:51:46.911 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:51:46.912 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:51:46.912 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:46.912 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:46.912 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:46.913 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6818", 2022-06-29T15:51:46.913 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:51:46.913 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:46.913 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:46.913 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:46.914 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6819", 2022-06-29T15:51:46.914 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:51:46.914 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:46.914 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:46.915 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:46.915 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:51:46.915 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:46.915 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:46.916 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:46.916 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6820", 2022-06-29T15:51:46.916 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:51:46.916 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:46.916 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:46.917 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:46.917 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6821", 2022-06-29T15:51:46.917 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:51:46.917 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:46.918 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:46.918 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:46.918 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:51:46.918 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:46.919 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:46.919 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:46.919 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6824", 2022-06-29T15:51:46.919 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:51:46.919 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:46.920 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:46.920 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:46.920 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6825", 2022-06-29T15:51:46.920 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:51:46.921 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:46.921 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:46.921 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:46.921 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:51:46.922 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:46.922 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:46.922 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:46.922 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6822", 2022-06-29T15:51:46.922 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:51:46.923 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:46.923 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:46.923 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:46.923 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6823", 2022-06-29T15:51:46.924 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:51:46.924 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:46.924 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:46.925 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:46.925 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6819/13096", 2022-06-29T15:51:46.925 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6821/13096", 2022-06-29T15:51:46.925 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6825/13096", 2022-06-29T15:51:46.925 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6823/13096", 2022-06-29T15:51:46.926 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:51:46.926 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:51:46.926 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:51:46.926 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:46.927 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:46.927 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:46.927 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2022-06-29T15:51:46.927 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "228bb08d-2d64-43d9-9560-bfaeff154452", 2022-06-29T15:51:46.927 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:51:46.928 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:51:46.928 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:51:46.928 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:51:46.928 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:51:46.929 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:51:46.929 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:51:46.929 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 138, 2022-06-29T15:51:46.930 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:51:46.930 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:51:46.930 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:51:46.930 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:46.931 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:46.931 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:46.931 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6826", 2022-06-29T15:51:46.931 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:51:46.931 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:46.932 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:46.932 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:46.932 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6827", 2022-06-29T15:51:46.932 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:51:46.933 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:46.933 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:46.933 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:46.933 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:51:46.933 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:46.934 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:46.934 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:46.934 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6828", 2022-06-29T15:51:46.934 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:51:46.935 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:46.935 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:46.935 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:46.935 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6829", 2022-06-29T15:51:46.936 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:51:46.936 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:46.936 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:46.936 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:46.936 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:51:46.937 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:46.937 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:46.937 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:46.937 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6832", 2022-06-29T15:51:46.938 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:51:46.938 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:46.938 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:46.938 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:46.938 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6833", 2022-06-29T15:51:46.939 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:51:46.939 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:46.939 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:46.939 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:46.940 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:51:46.940 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:46.940 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:46.940 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:46.940 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6830", 2022-06-29T15:51:46.941 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:51:46.941 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:46.941 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:46.941 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:46.942 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6831", 2022-06-29T15:51:46.942 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:51:46.942 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:46.942 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:46.943 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:46.943 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6827/13100", 2022-06-29T15:51:46.943 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6829/13100", 2022-06-29T15:51:46.943 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6833/13100", 2022-06-29T15:51:46.944 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6831/13100", 2022-06-29T15:51:46.944 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:51:46.944 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:51:46.944 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:51:46.944 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:46.945 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:46.945 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:46.945 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2022-06-29T15:51:46.945 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "797a5452-1510-4640-a111-c9c2fcf707eb", 2022-06-29T15:51:46.946 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:51:46.946 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:51:46.946 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:51:46.946 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:51:46.947 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:51:46.947 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:51:46.947 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:51:46.947 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 138, 2022-06-29T15:51:46.947 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:51:46.948 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:51:46.948 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:51:46.948 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:46.948 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:46.949 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:46.949 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6810", 2022-06-29T15:51:46.949 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:51:46.949 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:46.949 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:46.950 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:46.950 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6811", 2022-06-29T15:51:46.950 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:51:46.950 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:46.951 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:46.951 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:46.951 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:51:46.951 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:46.951 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:46.952 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:46.952 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6812", 2022-06-29T15:51:46.952 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:51:46.952 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:46.953 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:46.953 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:46.953 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6813", 2022-06-29T15:51:46.953 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:51:46.954 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:46.954 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:46.954 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:46.954 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:51:46.954 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:46.955 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:46.955 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:46.955 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6816", 2022-06-29T15:51:46.955 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:51:46.956 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:46.956 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:46.956 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:46.956 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6817", 2022-06-29T15:51:46.956 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:51:46.957 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:46.957 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:46.957 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:46.957 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:51:46.958 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:46.958 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:46.958 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:46.958 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6814", 2022-06-29T15:51:46.958 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:51:46.959 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:46.959 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:46.959 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:46.959 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6815", 2022-06-29T15:51:46.960 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:51:46.960 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:46.960 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:46.960 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:46.961 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6811/13098", 2022-06-29T15:51:46.961 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6813/13098", 2022-06-29T15:51:46.961 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6817/13098", 2022-06-29T15:51:46.961 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6815/13098", 2022-06-29T15:51:46.962 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:51:46.962 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:51:46.962 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:51:46.962 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:46.963 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:46.963 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:46.963 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2022-06-29T15:51:46.963 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "55b24fd7-bb71-4a01-b3fa-a09caa772a62", 2022-06-29T15:51:46.964 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:51:46.964 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:51:46.964 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:51:46.964 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:51:46.964 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:51:46.965 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:51:46.965 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:51:46.965 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 137, 2022-06-29T15:51:46.965 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:51:46.966 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:51:46.966 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:51:46.966 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:46.966 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:46.967 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:46.967 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6802", 2022-06-29T15:51:46.967 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:51:46.967 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:46.967 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:46.968 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:46.968 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6803", 2022-06-29T15:51:46.968 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:51:46.968 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:46.969 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:46.969 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:46.969 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:51:46.969 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:46.970 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:46.970 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:46.970 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6804", 2022-06-29T15:51:46.976 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:51:46.976 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:46.976 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:46.977 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:46.977 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6805", 2022-06-29T15:51:46.977 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:51:46.978 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:46.978 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:46.978 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:46.978 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:51:46.978 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:46.979 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:46.979 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:46.979 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6808", 2022-06-29T15:51:46.979 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:51:46.980 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:46.980 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:46.980 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:46.980 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6809", 2022-06-29T15:51:46.981 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:51:46.981 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:46.981 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:46.981 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:46.981 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:51:46.982 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:46.982 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:46.982 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:46.982 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6806", 2022-06-29T15:51:46.983 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:51:46.983 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:46.983 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:46.983 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:46.984 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6807", 2022-06-29T15:51:46.984 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:51:46.984 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:46.984 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:46.984 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:46.985 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6803/13097", 2022-06-29T15:51:46.985 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6805/13097", 2022-06-29T15:51:46.985 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6809/13097", 2022-06-29T15:51:46.985 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6807/13097", 2022-06-29T15:51:46.986 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:51:46.986 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:51:46.986 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:51:46.986 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:46.987 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:46.987 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:46.987 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2022-06-29T15:51:46.987 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3", 2022-06-29T15:51:46.987 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:51:46.988 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:51:46.988 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:51:46.988 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:51:46.988 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:51:46.989 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:51:46.989 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:51:46.989 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 138, 2022-06-29T15:51:46.989 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:51:46.989 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:51:46.990 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:51:46.990 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:46.990 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:46.990 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:46.991 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6816", 2022-06-29T15:51:46.991 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:51:46.991 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:46.991 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:46.992 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:46.992 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6817", 2022-06-29T15:51:46.992 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:51:46.992 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:46.992 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:46.993 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:46.993 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:51:46.993 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:46.993 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:46.994 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:46.994 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6818", 2022-06-29T15:51:46.994 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:51:46.994 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:46.995 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:46.995 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:46.995 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6819", 2022-06-29T15:51:46.995 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:51:46.996 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:46.996 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:46.996 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:46.996 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:51:46.996 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:46.997 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:46.997 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:46.997 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6822", 2022-06-29T15:51:46.997 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:51:46.998 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:46.998 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:46.998 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:46.998 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6823", 2022-06-29T15:51:46.999 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:51:46.999 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:46.999 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:46.999 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:47.000 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:51:47.000 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:47.000 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:47.000 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:47.000 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6820", 2022-06-29T15:51:47.001 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:51:47.001 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:47.001 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:47.001 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:47.002 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6821", 2022-06-29T15:51:47.002 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:51:47.002 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:47.002 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:47.002 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:47.003 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6817/13396", 2022-06-29T15:51:47.003 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6819/13396", 2022-06-29T15:51:47.003 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6823/13396", 2022-06-29T15:51:47.003 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6821/13396", 2022-06-29T15:51:47.004 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:51:47.004 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:51:47.004 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:51:47.004 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:47.004 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:47.005 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:47.005 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2022-06-29T15:51:47.005 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "f77beebe-6cca-4244-b2d4-677825ebf919", 2022-06-29T15:51:47.005 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:51:47.006 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:51:47.006 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:51:47.006 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:51:47.006 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:51:47.006 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:51:47.007 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:51:47.007 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 131, 2022-06-29T15:51:47.007 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:51:47.007 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:51:47.008 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:51:47.008 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:47.008 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:47.008 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:47.008 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6800", 2022-06-29T15:51:47.009 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:51:47.009 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:47.009 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:47.009 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:47.010 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6801", 2022-06-29T15:51:47.010 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:51:47.010 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:47.010 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:47.011 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:47.011 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:51:47.011 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:47.011 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:47.012 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:47.012 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6802", 2022-06-29T15:51:47.012 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:51:47.012 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:47.013 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:47.013 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:47.013 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6803", 2022-06-29T15:51:47.013 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:51:47.013 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:47.014 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:47.014 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:47.014 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:51:47.014 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:47.015 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:47.015 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:47.015 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6806", 2022-06-29T15:51:47.015 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:51:47.015 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:47.016 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:47.016 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:47.016 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6807", 2022-06-29T15:51:47.016 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:51:47.017 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:47.017 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:47.017 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:47.017 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:51:47.017 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:47.018 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:47.018 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:47.018 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6804", 2022-06-29T15:51:47.018 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:51:47.019 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:47.019 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:47.019 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:47.019 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6805", 2022-06-29T15:51:47.020 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:51:47.020 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:47.020 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:47.020 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:47.020 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6801/13393", 2022-06-29T15:51:47.021 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6803/13393", 2022-06-29T15:51:47.021 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6807/13393", 2022-06-29T15:51:47.021 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6805/13393", 2022-06-29T15:51:47.021 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:51:47.022 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:51:47.022 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:51:47.022 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:47.022 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:47.023 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:47.023 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2022-06-29T15:51:47.023 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "af0ca9b3-c02c-4910-834b-bf0b8dc6713c", 2022-06-29T15:51:47.023 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:51:47.023 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:51:47.024 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:51:47.024 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:51:47.024 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:51:47.024 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:51:47.024 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:51:47.025 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 138, 2022-06-29T15:51:47.025 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:51:47.025 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:51:47.025 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:51:47.025 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:47.026 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:47.026 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:47.026 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6824", 2022-06-29T15:51:47.026 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:51:47.027 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:47.027 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:47.027 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:47.027 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6825", 2022-06-29T15:51:47.028 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:51:47.028 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:47.028 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:47.028 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:47.028 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:51:47.029 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:47.029 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:47.029 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:47.029 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6826", 2022-06-29T15:51:47.030 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:51:47.030 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:47.030 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:47.030 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:47.031 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6827", 2022-06-29T15:51:47.031 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:51:47.031 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:47.031 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:47.031 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:47.032 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:51:47.032 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:47.032 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:47.032 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:47.033 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6830", 2022-06-29T15:51:47.033 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:51:47.033 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:47.033 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:47.033 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:47.034 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6831", 2022-06-29T15:51:47.034 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:51:47.034 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:47.034 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:47.035 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:47.035 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:51:47.035 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:47.035 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:47.036 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:47.036 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6828", 2022-06-29T15:51:47.036 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:51:47.036 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:47.036 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:47.037 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:47.037 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6829", 2022-06-29T15:51:47.037 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:51:47.037 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:47.037 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:47.038 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:47.038 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6825/13395", 2022-06-29T15:51:47.038 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6827/13395", 2022-06-29T15:51:47.038 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6831/13395", 2022-06-29T15:51:47.039 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6829/13395", 2022-06-29T15:51:47.039 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:51:47.039 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:51:47.039 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:51:47.040 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:47.040 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:47.040 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:47.040 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2022-06-29T15:51:47.041 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "f19c3ca6-df88-4bde-a150-b51a53f092e1", 2022-06-29T15:51:47.041 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:51:47.041 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:51:47.041 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:51:47.041 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:51:47.042 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:51:47.042 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:51:47.042 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:51:47.042 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 138, 2022-06-29T15:51:47.043 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:51:47.043 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:51:47.043 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:51:47.043 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:47.043 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:47.044 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:47.044 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6808", 2022-06-29T15:51:47.044 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:51:47.044 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:47.045 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:47.045 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:47.045 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6809", 2022-06-29T15:51:47.045 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:51:47.045 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:47.046 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:47.046 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:47.046 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:51:47.046 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:47.047 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:47.047 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:47.047 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6810", 2022-06-29T15:51:47.047 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:51:47.048 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:47.048 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:47.048 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:47.048 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6811", 2022-06-29T15:51:47.048 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:51:47.049 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:47.049 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:47.049 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:47.049 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:51:47.050 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:47.050 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:47.050 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:47.050 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6814", 2022-06-29T15:51:47.051 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:51:47.051 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:47.051 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:47.051 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:47.051 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6815", 2022-06-29T15:51:47.052 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:51:47.052 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:47.052 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:47.052 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:47.053 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:51:47.053 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:51:47.053 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:47.053 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:51:47.054 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6812", 2022-06-29T15:51:47.054 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:51:47.054 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:47.054 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:47.055 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:51:47.055 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6813", 2022-06-29T15:51:47.055 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:51:47.055 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:47.055 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:47.056 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:47.056 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6809/13394", 2022-06-29T15:51:47.056 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6811/13394", 2022-06-29T15:51:47.056 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6815/13394", 2022-06-29T15:51:47.057 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6813/13394", 2022-06-29T15:51:47.057 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:51:47.057 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:51:47.057 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:51:47.058 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:47.058 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:47.058 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:51:47.058 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_xinfo": [ 2022-06-29T15:51:47.059 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:47.059 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2022-06-29T15:51:47.059 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:51:47.059 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:51:47.059 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:51:47.060 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:51:47.060 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:51:47.060 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.720513+0000", 2022-06-29T15:51:47.060 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:51:47.061 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:47.061 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:47.061 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2022-06-29T15:51:47.061 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:51:47.062 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:51:47.062 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:51:47.062 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:51:47.062 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:51:47.062 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.772612+0000", 2022-06-29T15:51:47.063 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:51:47.063 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:47.063 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:47.063 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2022-06-29T15:51:47.064 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:51:47.064 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:51:47.064 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:51:47.064 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:51:47.065 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:51:47.065 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.450223+0000", 2022-06-29T15:51:47.065 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:51:47.065 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:47.065 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:47.066 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2022-06-29T15:51:47.066 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:51:47.066 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:51:47.066 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:51:47.067 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:51:47.067 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:51:47.067 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.477856+0000", 2022-06-29T15:51:47.067 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:51:47.068 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:47.068 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:47.068 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2022-06-29T15:51:47.068 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:51:47.068 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:51:47.069 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:51:47.069 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:51:47.069 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:51:47.069 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.539485+0000", 2022-06-29T15:51:47.070 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:51:47.070 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:47.070 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:47.070 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2022-06-29T15:51:47.071 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:51:47.071 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:51:47.071 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:51:47.071 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:51:47.072 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:51:47.072 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.487040+0000", 2022-06-29T15:51:47.072 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:51:47.072 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:47.072 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:47.073 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2022-06-29T15:51:47.073 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:51:47.073 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:51:47.073 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:51:47.074 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:51:47.074 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:51:47.074 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.748685+0000", 2022-06-29T15:51:47.074 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:51:47.074 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:47.075 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:47.075 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2022-06-29T15:51:47.075 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:51:47.075 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:51:47.076 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:51:47.076 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:51:47.076 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:51:47.076 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.459516+0000", 2022-06-29T15:51:47.076 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:51:47.077 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:47.077 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:51:47.077 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap": [], 2022-06-29T15:51:47.077 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap_items": [], 2022-06-29T15:51:47.078 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_temp": [], 2022-06-29T15:51:47.078 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_temp": [], 2022-06-29T15:51:47.078 INFO:teuthology.orchestra.run.smithi047.stdout: "blacklist": { 2022-06-29T15:51:47.078 INFO:teuthology.orchestra.run.smithi047.stdout: "172.21.15.47:6837/1370488030": "2022-06-30T15:51:40.217041+0000", 2022-06-29T15:51:47.079 INFO:teuthology.orchestra.run.smithi047.stdout: "172.21.15.47:6836/1370488030": "2022-06-30T15:51:40.217041+0000" 2022-06-29T15:51:47.079 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:47.079 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profiles": { 2022-06-29T15:51:47.079 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs_data": { 2022-06-29T15:51:47.079 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-device-class": "", 2022-06-29T15:51:47.080 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2022-06-29T15:51:47.080 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-root": "default", 2022-06-29T15:51:47.080 INFO:teuthology.orchestra.run.smithi047.stdout: "jerasure-per-chunk-alignment": "false", 2022-06-29T15:51:47.080 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2022-06-29T15:51:47.081 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "2", 2022-06-29T15:51:47.081 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2022-06-29T15:51:47.081 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2022-06-29T15:51:47.081 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van", 2022-06-29T15:51:47.082 INFO:teuthology.orchestra.run.smithi047.stdout: "w": "8" 2022-06-29T15:51:47.082 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:47.082 INFO:teuthology.orchestra.run.smithi047.stdout: "default": { 2022-06-29T15:51:47.082 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2022-06-29T15:51:47.082 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2022-06-29T15:51:47.083 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "1", 2022-06-29T15:51:47.083 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2022-06-29T15:51:47.083 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2022-06-29T15:51:47.083 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van" 2022-06-29T15:51:47.084 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:47.084 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:47.084 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps_queue": [], 2022-06-29T15:51:47.084 INFO:teuthology.orchestra.run.smithi047.stdout: "new_removed_snaps": [], 2022-06-29T15:51:47.084 INFO:teuthology.orchestra.run.smithi047.stdout: "new_purged_snaps": [], 2022-06-29T15:51:47.085 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_node_flags": {}, 2022-06-29T15:51:47.085 INFO:teuthology.orchestra.run.smithi047.stdout: "device_class_flags": {} 2022-06-29T15:51:47.085 INFO:teuthology.orchestra.run.smithi047.stdout:} 2022-06-29T15:51:47.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.862+0000 7effc255b700 1 -- 172.21.15.47:0/353299207 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7eff9c053950 msgr2=0x7eff9c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:47.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.862+0000 7effc255b700 1 --2- 172.21.15.47:0/353299207 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7eff9c053950 0x7eff9c055e10 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7effb800d000 tx=0x7effb80018d0).stop 2022-06-29T15:51:47.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.862+0000 7effc255b700 1 -- 172.21.15.47:0/353299207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effbc0b1470 msgr2=0x7effbc064e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:47.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.862+0000 7effc255b700 1 --2- 172.21.15.47:0/353299207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effbc0b1470 0x7effbc064e30 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 rx=0x7effbc0ab490 tx=0x7effb4016000).stop 2022-06-29T15:51:47.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.862+0000 7effc255b700 1 -- 172.21.15.47:0/353299207 shutdown_connections 2022-06-29T15:51:47.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.862+0000 7effc255b700 1 --2- 172.21.15.47:0/353299207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effbc0b1470 0x7effbc064e30 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:47.089 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.862+0000 7effc255b700 1 --2- 172.21.15.47:0/353299207 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7eff9c053950 0x7eff9c055e10 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:47.089 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.862+0000 7effc255b700 1 --2- 172.21.15.47:0/353299207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effbc0b04e0 0x7effbc064940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:47.089 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.866+0000 7effc255b700 1 --2- 172.21.15.47:0/353299207 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7effbc0af590 0x7effbc066b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:47.089 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.866+0000 7effc255b700 1 -- 172.21.15.47:0/353299207 >> 172.21.15.47:0/353299207 conn(0x7effbc062560 msgr2=0x7effbc0acda0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:47.089 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.866+0000 7effc255b700 1 -- 172.21.15.47:0/353299207 shutdown_connections 2022-06-29T15:51:47.090 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:46.866+0000 7effc255b700 1 -- 172.21.15.47:0/353299207 wait complete. 2022-06-29T15:51:47.091 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:51:47.202 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.194+0000 7f7ea3fbb700 1 Processor -- start 2022-06-29T15:51:47.202 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.194+0000 7f7ea3fbb700 1 -- start start 2022-06-29T15:51:47.205 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.194+0000 7f7ea3fbb700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7e9c0b2120 0x7f7e9c0b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:47.205 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.194+0000 7f7ea3fbb700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7e9c05f0b0 0x7f7e9c14b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:47.206 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.194+0000 7f7ea3fbb700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e9c0b0740 con 0x7f7e9c0b1260 2022-06-29T15:51:47.206 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.194+0000 7f7ea3fbb700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e9c147460 con 0x7f7e9c05f0b0 2022-06-29T15:51:47.206 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.194+0000 7f7ea3fbb700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c0b0970 con 0x7f7e9c0b2120 2022-06-29T15:51:47.206 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.198+0000 7f7ea2558700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7e9c0b1260 0x7f7e9c147af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39734/0 (socket says 172.21.15.47:39734) 2022-06-29T15:51:47.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.198+0000 7f7ea2558700 1 -- 172.21.15.47:0/46296461 learned_addr learned my addr 172.21.15.47:0/46296461 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:47.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.198+0000 7f7ea1556700 1 --2- 172.21.15.47:0/46296461 >> v2:172.21.15.64:3300/0 conn(0x7f7e9c05f0b0 0x7f7e9c14b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:47.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.198+0000 7f7ea1d57700 1 --2- 172.21.15.47:0/46296461 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7e9c0b2120 0x7f7e9c0b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:47.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.198+0000 7f7ea0d55700 1 -- 172.21.15.47:0/46296461 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 175759818 0 0) 0x7f7e9c0b0740 con 0x7f7e9c0b1260 2022-06-29T15:51:47.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.198+0000 7f7ea0d55700 1 -- 172.21.15.47:0/46296461 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e84003260 con 0x7f7e9c0b1260 2022-06-29T15:51:47.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.198+0000 7f7ea1556700 1 -- 172.21.15.47:0/46296461 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7e9c0b2120 msgr2=0x7f7e9c0b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:47.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.198+0000 7f7ea1556700 1 --2- 172.21.15.47:0/46296461 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7e9c0b2120 0x7f7e9c0b2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:47.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.198+0000 7f7ea1556700 1 -- 172.21.15.47:0/46296461 >> v1:172.21.15.47:6789/0 conn(0x7f7e9c0b1260 legacy=0x7f7e9c147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:47.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.198+0000 7f7ea1556700 1 -- 172.21.15.47:0/46296461 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e9c0b0c10 con 0x7f7e9c05f0b0 2022-06-29T15:51:47.209 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.198+0000 7f7ea1d57700 1 --2- 172.21.15.47:0/46296461 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7e9c0b2120 0x7f7e9c0b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:51:47.209 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.198+0000 7f7ea1556700 1 --2- 172.21.15.47:0/46296461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e9c05f0b0 0x7f7e9c14b520 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7f7e90002f70 tx=0x7f7e90006df0).ready entity=mon.1 client_cookie=2d7dab29f4eaaa1a server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:47.209 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.198+0000 7f7ea0d55700 1 -- 172.21.15.47:0/46296461 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e9000d570 con 0x7f7e9c05f0b0 2022-06-29T15:51:47.209 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.198+0000 7f7ea0d55700 1 -- 172.21.15.47:0/46296461 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e90000e90 con 0x7f7e9c05f0b0 2022-06-29T15:51:47.210 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.198+0000 7f7ea0d55700 1 -- 172.21.15.47:0/46296461 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e9000d8c0 con 0x7f7e9c05f0b0 2022-06-29T15:51:47.210 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.198+0000 7f7ea3fbb700 1 -- 172.21.15.47:0/46296461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e9c05f0b0 msgr2=0x7f7e9c14b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:47.210 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.198+0000 7f7ea3fbb700 1 --2- 172.21.15.47:0/46296461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e9c05f0b0 0x7f7e9c14b520 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7f7e90002f70 tx=0x7f7e90006df0).stop 2022-06-29T15:51:47.210 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.198+0000 7f7ea3fbb700 1 -- 172.21.15.47:0/46296461 shutdown_connections 2022-06-29T15:51:47.211 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.198+0000 7f7ea3fbb700 1 --2- 172.21.15.47:0/46296461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e9c05f0b0 0x7f7e9c14b520 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:47.211 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.198+0000 7f7ea3fbb700 1 --2- 172.21.15.47:0/46296461 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7e9c0b2120 0x7f7e9c0b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:47.211 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.198+0000 7f7ea3fbb700 1 -- 172.21.15.47:0/46296461 >> 172.21.15.47:0/46296461 conn(0x7f7e9c0626f0 msgr2=0x7f7e9c062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:47.211 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.198+0000 7f7ea3fbb700 1 -- 172.21.15.47:0/46296461 shutdown_connections 2022-06-29T15:51:47.212 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.198+0000 7f7ea3fbb700 1 -- 172.21.15.47:0/46296461 wait complete. 2022-06-29T15:51:47.215 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.198+0000 7f7ea3fbb700 1 Processor -- start 2022-06-29T15:51:47.215 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.198+0000 7f7ea3fbb700 1 -- start start 2022-06-29T15:51:47.215 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.198+0000 7f7ea3fbb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e9c0b1260 0x7f7e9c149e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:47.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.198+0000 7f7ea3fbb700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7e9c0b2120 0x7f7e9c141a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:47.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.198+0000 7f7ea3fbb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e9c0af470 0x7f7e9c141f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:47.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.198+0000 7f7ea3fbb700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c0af910 con 0x7f7e9c0af470 2022-06-29T15:51:47.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.198+0000 7f7ea3fbb700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c0ae0a0 con 0x7f7e9c0b1260 2022-06-29T15:51:47.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.198+0000 7f7ea3fbb700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c0ae340 con 0x7f7e9c0b2120 2022-06-29T15:51:47.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.202+0000 7f7ea2558700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e9c0af470 0x7f7e9c141f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:47.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.202+0000 7f7ea1d57700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e9c0b1260 0x7f7e9c149e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:47.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.202+0000 7f7ea1d57700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e9c0b1260 0x7f7e9c149e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:59886/0 (socket says 172.21.15.47:59886) 2022-06-29T15:51:47.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.202+0000 7f7ea1556700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7e9c0b2120 0x7f7e9c141a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:47.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.202+0000 7f7ea1d57700 1 -- 172.21.15.47:0/1759184568 learned_addr learned my addr 172.21.15.47:0/1759184568 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:47.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.202+0000 7f7ea1d57700 1 -- 172.21.15.47:0/1759184568 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7e9c0b2120 msgr2=0x7f7e9c141a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:47.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.202+0000 7f7ea1d57700 1 --2- 172.21.15.47:0/1759184568 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7e9c0b2120 0x7f7e9c141a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:47.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.202+0000 7f7ea1d57700 1 -- 172.21.15.47:0/1759184568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e9c0af470 msgr2=0x7f7e9c141f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:47.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.202+0000 7f7ea1d57700 1 --2- 172.21.15.47:0/1759184568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e9c0af470 0x7f7e9c141f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:47.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.202+0000 7f7ea1d57700 1 -- 172.21.15.47:0/1759184568 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e9000b180 con 0x7f7e9c0b1260 2022-06-29T15:51:47.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.202+0000 7f7ea2558700 1 --2- 172.21.15.47:0/1759184568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e9c0af470 0x7f7e9c141f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:51:47.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.202+0000 7f7ea1d57700 1 --2- 172.21.15.47:0/1759184568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e9c0b1260 0x7f7e9c149e90 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7f7e98006940 tx=0x7f7e98008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:47.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.202+0000 7f7e8effd700 1 -- 172.21.15.47:0/1759184568 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e98011780 con 0x7f7e9c0b1260 2022-06-29T15:51:47.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.202+0000 7f7e8effd700 1 -- 172.21.15.47:0/1759184568 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e980118e0 con 0x7f7e9c0b1260 2022-06-29T15:51:47.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.202+0000 7f7ea3fbb700 1 -- 172.21.15.47:0/1759184568 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e9c0ae5e0 con 0x7f7e9c0b1260 2022-06-29T15:51:47.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.202+0000 7f7e8effd700 1 -- 172.21.15.47:0/1759184568 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e98011c40 con 0x7f7e9c0b1260 2022-06-29T15:51:47.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.202+0000 7f7ea3fbb700 1 -- 172.21.15.47:0/1759184568 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e9c0aea80 con 0x7f7e9c0b1260 2022-06-29T15:51:47.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.202+0000 7f7ea3fbb700 1 -- 172.21.15.47:0/1759184568 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e9c0af1e0 con 0x7f7e9c0b1260 2022-06-29T15:51:47.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.202+0000 7f7e8effd700 1 -- 172.21.15.47:0/1759184568 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7e98012420 con 0x7f7e9c0b1260 2022-06-29T15:51:47.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.202+0000 7f7e8effd700 1 --2- 172.21.15.47:0/1759184568 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7e7c053b30 0x7f7e7c055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:47.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.202+0000 7f7e8effd700 1 -- 172.21.15.47:0/1759184568 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f7e9806bba0 con 0x7f7e9c0b1260 2022-06-29T15:51:47.222 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.202+0000 7f7ea1556700 1 --2- 172.21.15.47:0/1759184568 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7e7c053b30 0x7f7e7c055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:47.222 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.206+0000 7f7ea1556700 1 --2- 172.21.15.47:0/1759184568 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7e7c053b30 0x7f7e7c055ff0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7f7e9001a000 tx=0x7f7e90014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:47.222 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.214+0000 7f7e8effd700 1 -- 172.21.15.47:0/1759184568 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7e9c0af1e0 con 0x7f7e9c0b1260 2022-06-29T15:51:47.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.382+0000 7f7ea3fbb700 1 -- 172.21.15.47:0/1759184568 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e9c0af1e0 con 0x7f7e9c0b1260 2022-06-29T15:51:47.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.382+0000 7f7e8effd700 1 -- 172.21.15.47:0/1759184568 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+1554 (secure 0 0 0) 0x7f7e9c0af1e0 con 0x7f7e9c0b1260 2022-06-29T15:51:47.389 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:47.390 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":71,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7957,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.64:6833/298405638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":298405638},{"type":"v1","addr":"172.21.15.64:6833","nonce":298405638}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":66},{"gid":8037,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/191070564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":191070564},{"type":"v1","addr":"172.21.15.47:6835","nonce":191070564}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65},{"gid":57089,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/166422124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":166422124},{"type":"v1","addr":"172.21.15.47:6837","nonce":166422124}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":71}],"filesystems":[]} 2022-06-29T15:51:47.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.386+0000 7f7ea3fbb700 1 -- 172.21.15.47:0/1759184568 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7e7c053b30 msgr2=0x7f7e7c055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:47.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.386+0000 7f7ea3fbb700 1 --2- 172.21.15.47:0/1759184568 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7e7c053b30 0x7f7e7c055ff0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7f7e9001a000 tx=0x7f7e90014000).stop 2022-06-29T15:51:47.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.386+0000 7f7ea3fbb700 1 -- 172.21.15.47:0/1759184568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e9c0b1260 msgr2=0x7f7e9c149e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:47.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.386+0000 7f7ea3fbb700 1 --2- 172.21.15.47:0/1759184568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e9c0b1260 0x7f7e9c149e90 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7f7e98006940 tx=0x7f7e98008040).stop 2022-06-29T15:51:47.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.386+0000 7f7ea3fbb700 1 -- 172.21.15.47:0/1759184568 shutdown_connections 2022-06-29T15:51:47.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.386+0000 7f7ea3fbb700 1 --2- 172.21.15.47:0/1759184568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e9c0af470 0x7f7e9c141f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:47.394 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.386+0000 7f7ea3fbb700 1 --2- 172.21.15.47:0/1759184568 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7e9c0b2120 0x7f7e9c141a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:47.394 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.386+0000 7f7ea3fbb700 1 --2- 172.21.15.47:0/1759184568 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7e7c053b30 0x7f7e7c055ff0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:47.394 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.386+0000 7f7ea3fbb700 1 --2- 172.21.15.47:0/1759184568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e9c0b1260 0x7f7e9c149e90 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:47.394 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.386+0000 7f7ea3fbb700 1 -- 172.21.15.47:0/1759184568 >> 172.21.15.47:0/1759184568 conn(0x7f7e9c0626f0 msgr2=0x7f7e9c05f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:47.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.386+0000 7f7ea3fbb700 1 -- 172.21.15.47:0/1759184568 shutdown_connections 2022-06-29T15:51:47.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.386+0000 7f7ea3fbb700 1 -- 172.21.15.47:0/1759184568 wait complete. 2022-06-29T15:51:47.395 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 71 2022-06-29T15:51:47.407 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:51:47.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.502+0000 7f943e866700 1 Processor -- start 2022-06-29T15:51:47.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.502+0000 7f943e866700 1 -- start start 2022-06-29T15:51:47.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.506+0000 7f943e866700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9438141e70 0x7f94380b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:47.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.506+0000 7f943e866700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94380b7c20 con 0x7f9438062dd0 2022-06-29T15:51:47.513 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.506+0000 7f943e866700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94380b3d10 con 0x7f94380b7840 2022-06-29T15:51:47.513 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.506+0000 7f943e866700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94381422a0 con 0x7f9438141e70 2022-06-29T15:51:47.513 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.506+0000 7f943d063700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9438062dd0 0x7f9438141760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39744/0 (socket says 172.21.15.47:39744) 2022-06-29T15:51:47.513 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.506+0000 7f943d063700 1 -- 172.21.15.47:0/1123045531 learned_addr learned my addr 172.21.15.47:0/1123045531 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:47.514 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.506+0000 7f943e065700 1 --2- 172.21.15.47:0/1123045531 >> v2:172.21.15.64:3300/0 conn(0x7f9438141e70 0x7f94380b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:47.514 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.506+0000 7f943c862700 1 -- 172.21.15.47:0/1123045531 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 494237844 0 0) 0x7f94380b7c20 con 0x7f9438062dd0 2022-06-29T15:51:47.514 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.506+0000 7f943c862700 1 -- 172.21.15.47:0/1123045531 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9420003260 con 0x7f9438062dd0 2022-06-29T15:51:47.514 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.506+0000 7f943e065700 1 -- 172.21.15.47:0/1123045531 >> v1:172.21.15.64:6789/0 conn(0x7f94380b7840 legacy=0x7f94380aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:47.515 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.506+0000 7f943e065700 1 -- 172.21.15.47:0/1123045531 >> v1:172.21.15.47:6789/0 conn(0x7f9438062dd0 legacy=0x7f9438141760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:47.515 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.506+0000 7f943e065700 1 -- 172.21.15.47:0/1123045531 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9438067380 con 0x7f9438141e70 2022-06-29T15:51:47.515 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.506+0000 7f943e065700 1 --2- 172.21.15.47:0/1123045531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9438141e70 0x7f94380b3820 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7f94300032f0 tx=0x7f94300091d0).ready entity=mon.1 client_cookie=871900e2d8b07f17 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:47.517 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.506+0000 7f943c862700 1 -- 172.21.15.47:0/1123045531 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f943000b6c0 con 0x7f9438141e70 2022-06-29T15:51:47.518 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.506+0000 7f943c862700 1 -- 172.21.15.47:0/1123045531 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f943000b820 con 0x7f9438141e70 2022-06-29T15:51:47.518 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.506+0000 7f943c862700 1 -- 172.21.15.47:0/1123045531 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f943000b990 con 0x7f9438141e70 2022-06-29T15:51:47.518 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.506+0000 7f943e866700 1 -- 172.21.15.47:0/1123045531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9438141e70 msgr2=0x7f94380b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:47.518 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.506+0000 7f943e866700 1 --2- 172.21.15.47:0/1123045531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9438141e70 0x7f94380b3820 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7f94300032f0 tx=0x7f94300091d0).stop 2022-06-29T15:51:47.518 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.506+0000 7f943e866700 1 -- 172.21.15.47:0/1123045531 shutdown_connections 2022-06-29T15:51:47.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.506+0000 7f943e866700 1 --2- 172.21.15.47:0/1123045531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9438141e70 0x7f94380b3820 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:47.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.506+0000 7f943e866700 1 -- 172.21.15.47:0/1123045531 >> 172.21.15.47:0/1123045531 conn(0x7f943805f480 msgr2=0x7f94380647d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:47.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.506+0000 7f943e866700 1 -- 172.21.15.47:0/1123045531 shutdown_connections 2022-06-29T15:51:47.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.506+0000 7f943e866700 1 -- 172.21.15.47:0/1123045531 wait complete. 2022-06-29T15:51:47.520 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.510+0000 7f943e866700 1 Processor -- start 2022-06-29T15:51:47.520 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.510+0000 7f943e866700 1 -- start start 2022-06-29T15:51:47.520 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.510+0000 7f943e866700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9438062dd0 0x7f94380a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:47.520 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.510+0000 7f943e866700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f94380b7840 0x7f9438153ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:47.520 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.510+0000 7f943e866700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9438141e70 0x7f94381433a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:47.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.510+0000 7f943e866700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94380a1a80 con 0x7f9438141e70 2022-06-29T15:51:47.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.510+0000 7f943e866700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9438154190 con 0x7f9438062dd0 2022-06-29T15:51:47.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.510+0000 7f943e866700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9438143890 con 0x7f94380b7840 2022-06-29T15:51:47.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.510+0000 7f943e065700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9438141e70 0x7f94381433a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:47.522 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.510+0000 7f943e065700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9438141e70 0x7f94381433a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54200/0 (socket says 172.21.15.47:54200) 2022-06-29T15:51:47.522 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.510+0000 7f943e065700 1 -- 172.21.15.47:0/2103006329 learned_addr learned my addr 172.21.15.47:0/2103006329 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:47.522 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.510+0000 7f943d063700 1 --2- 172.21.15.47:0/2103006329 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f94380b7840 0x7f9438153ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:47.522 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.510+0000 7f943d864700 1 --2- 172.21.15.47:0/2103006329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9438062dd0 0x7f94380a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:47.524 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.510+0000 7f943e065700 1 -- 172.21.15.47:0/2103006329 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f94380b7840 msgr2=0x7f9438153ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:47.524 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.510+0000 7f943e065700 1 --2- 172.21.15.47:0/2103006329 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f94380b7840 0x7f9438153ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:47.525 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.510+0000 7f943e065700 1 -- 172.21.15.47:0/2103006329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9438062dd0 msgr2=0x7f94380a0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:47.525 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.510+0000 7f943e065700 1 --2- 172.21.15.47:0/2103006329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9438062dd0 0x7f94380a0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:47.525 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.510+0000 7f943e065700 1 -- 172.21.15.47:0/2103006329 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9430009470 con 0x7f9438141e70 2022-06-29T15:51:47.525 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.510+0000 7f943d063700 1 --2- 172.21.15.47:0/2103006329 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f94380b7840 0x7f9438153ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:51:47.526 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.510+0000 7f943e065700 1 --2- 172.21.15.47:0/2103006329 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9438141e70 0x7f94381433a0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 rx=0x7f94380b6930 tx=0x7f9430016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:47.526 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.510+0000 7f9426ffd700 1 -- 172.21.15.47:0/2103006329 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f943000b6c0 con 0x7f9438141e70 2022-06-29T15:51:47.526 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.510+0000 7f9426ffd700 1 -- 172.21.15.47:0/2103006329 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9430021670 con 0x7f9438141e70 2022-06-29T15:51:47.526 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.510+0000 7f943e866700 1 -- 172.21.15.47:0/2103006329 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94381543c0 con 0x7f9438141e70 2022-06-29T15:51:47.527 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.510+0000 7f9426ffd700 1 -- 172.21.15.47:0/2103006329 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94300219d0 con 0x7f9438141e70 2022-06-29T15:51:47.527 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.510+0000 7f943e866700 1 -- 172.21.15.47:0/2103006329 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94380b6540 con 0x7f9438141e70 2022-06-29T15:51:47.527 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.514+0000 7f943e866700 1 -- 172.21.15.47:0/2103006329 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94380573d0 con 0x7f9438141e70 2022-06-29T15:51:47.527 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.514+0000 7f9426ffd700 1 -- 172.21.15.47:0/2103006329 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f943000f070 con 0x7f9438141e70 2022-06-29T15:51:47.528 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.514+0000 7f9426ffd700 1 --2- 172.21.15.47:0/2103006329 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9418053980 0x7f9418055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:47.528 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.514+0000 7f9426ffd700 1 -- 172.21.15.47:0/2103006329 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f943006d210 con 0x7f9438141e70 2022-06-29T15:51:47.528 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.514+0000 7f943d864700 1 --2- 172.21.15.47:0/2103006329 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9418053980 0x7f9418055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:47.528 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.514+0000 7f943d864700 1 --2- 172.21.15.47:0/2103006329 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9418053980 0x7f9418055e40 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7f943400d000 tx=0x7f9434000bc0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:47.529 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.522+0000 7f9426ffd700 1 -- 172.21.15.47:0/2103006329 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f94300441a0 con 0x7f9438141e70 2022-06-29T15:51:47.698 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.690+0000 7f943e866700 1 -- 172.21.15.47:0/2103006329 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9438065590 con 0x7f9438141e70 2022-06-29T15:51:47.699 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.694+0000 7f9426ffd700 1 -- 172.21.15.47:0/2103006329 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+1554 (secure 0 0 0) 0x7f9430015020 con 0x7f9438141e70 2022-06-29T15:51:47.699 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:47.700 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":71,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7957,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.64:6833/298405638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":298405638},{"type":"v1","addr":"172.21.15.64:6833","nonce":298405638}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":66},{"gid":8037,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/191070564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":191070564},{"type":"v1","addr":"172.21.15.47:6835","nonce":191070564}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65},{"gid":57089,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/166422124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":166422124},{"type":"v1","addr":"172.21.15.47:6837","nonce":166422124}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":71}],"filesystems":[]} 2022-06-29T15:51:47.701 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.694+0000 7f943e866700 1 -- 172.21.15.47:0/2103006329 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9418053980 msgr2=0x7f9418055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:47.701 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.694+0000 7f943e866700 1 --2- 172.21.15.47:0/2103006329 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9418053980 0x7f9418055e40 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7f943400d000 tx=0x7f9434000bc0).stop 2022-06-29T15:51:47.702 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.694+0000 7f943e866700 1 -- 172.21.15.47:0/2103006329 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9438141e70 msgr2=0x7f94381433a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:47.702 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.694+0000 7f943e866700 1 --2- 172.21.15.47:0/2103006329 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9438141e70 0x7f94381433a0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 rx=0x7f94380b6930 tx=0x7f9430016000).stop 2022-06-29T15:51:47.702 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.694+0000 7f943e866700 1 -- 172.21.15.47:0/2103006329 shutdown_connections 2022-06-29T15:51:47.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.694+0000 7f943e866700 1 --2- 172.21.15.47:0/2103006329 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9438141e70 0x7f94381433a0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:47.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.694+0000 7f943e866700 1 --2- 172.21.15.47:0/2103006329 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f94380b7840 0x7f9438153ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:47.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.694+0000 7f943e866700 1 --2- 172.21.15.47:0/2103006329 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9418053980 0x7f9418055e40 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:47.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.694+0000 7f943e866700 1 --2- 172.21.15.47:0/2103006329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9438062dd0 0x7f94380a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:47.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.694+0000 7f943e866700 1 -- 172.21.15.47:0/2103006329 >> 172.21.15.47:0/2103006329 conn(0x7f943805f480 msgr2=0x7f94380b69b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:47.704 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.694+0000 7f943e866700 1 -- 172.21.15.47:0/2103006329 shutdown_connections 2022-06-29T15:51:47.704 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.694+0000 7f943e866700 1 -- 172.21.15.47:0/2103006329 wait complete. 2022-06-29T15:51:47.704 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 71 2022-06-29T15:51:47.713 INFO:tasks.ceph.mds.a:Restarting daemon 2022-06-29T15:51:47.713 INFO:tasks.ceph.mds.a:Stopping old one... 2022-06-29T15:51:47.713 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-06-29T15:51:47.714 INFO:teuthology.orchestra.run:waiting for 300 2022-06-29T15:51:47.714 INFO:tasks.ceph.mds.c:Restarting daemon 2022-06-29T15:51:47.714 INFO:tasks.ceph.mds.c:Stopping old one... 2022-06-29T15:51:47.715 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2022-06-29T15:51:47.715 INFO:teuthology.orchestra.run:waiting for 300 2022-06-29T15:51:47.715 INFO:tasks.ceph.mds.b:Restarting daemon 2022-06-29T15:51:47.716 INFO:tasks.ceph.mds.b:Stopping old one... 2022-06-29T15:51:47.716 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-06-29T15:51:47.716 INFO:teuthology.orchestra.run:waiting for 300 2022-06-29T15:51:47.726 INFO:tasks.ceph.mds.c:Stopped 2022-06-29T15:51:47.727 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2022-06-29T15:51:47.729 INFO:tasks.ceph.mds.a:Stopped 2022-06-29T15:51:47.729 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-06-29T15:51:47.730 INFO:tasks.ceph.mds.c:Started 2022-06-29T15:51:47.731 DEBUG:teuthology.parallel:result is None 2022-06-29T15:51:47.732 INFO:tasks.ceph.mds.b:Stopped 2022-06-29T15:51:47.732 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-06-29T15:51:47.733 INFO:tasks.ceph.mds.a:Started 2022-06-29T15:51:47.734 DEBUG:teuthology.parallel:result is None 2022-06-29T15:51:47.736 INFO:tasks.ceph.mds.b:Started 2022-06-29T15:51:47.736 DEBUG:teuthology.parallel:result is None 2022-06-29T15:51:47.737 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-06-29T15:51:47.810 INFO:tasks.ceph.mds.b.smithi064.stdout:starting mds.b at 2022-06-29T15:51:47.810 INFO:tasks.ceph.mds.a.smithi047.stdout:starting mds.a at 2022-06-29T15:51:47.812 INFO:tasks.ceph.mds.c.smithi047.stdout:starting mds.c at 2022-06-29T15:51:47.830 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.822+0000 7f02262a2700 1 Processor -- start 2022-06-29T15:51:47.830 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.822+0000 7f02262a2700 1 -- start start 2022-06-29T15:51:47.833 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.822+0000 7f02262a2700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f02200b2a90 0x7f0220061050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:47.833 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.822+0000 7f02262a2700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02200b2170 con 0x7f02200601a0 2022-06-29T15:51:47.834 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.822+0000 7f02262a2700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02200b2e70 con 0x7f0220061600 2022-06-29T15:51:47.834 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.822+0000 7f02262a2700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02200606b0 con 0x7f02200b2a90 2022-06-29T15:51:47.834 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.822+0000 7f022483f700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f02200601a0 0x7f0220140520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39792/0 (socket says 172.21.15.47:39792) 2022-06-29T15:51:47.834 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.822+0000 7f022483f700 1 -- 172.21.15.47:0/1756461159 learned_addr learned my addr 172.21.15.47:0/1756461159 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:47.835 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.822+0000 7f021ffff700 1 --2- 172.21.15.47:0/1756461159 >> v2:172.21.15.64:3300/0 conn(0x7f02200b2a90 0x7f0220061050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:47.835 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.826+0000 7f021effd700 1 -- 172.21.15.47:0/1756461159 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4087717864 0 0) 0x7f02200b2170 con 0x7f02200601a0 2022-06-29T15:51:47.835 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.826+0000 7f021effd700 1 -- 172.21.15.47:0/1756461159 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0200003260 con 0x7f02200601a0 2022-06-29T15:51:47.835 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.826+0000 7f021effd700 1 -- 172.21.15.47:0/1756461159 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2886613276 0 0) 0x7f02200b2e70 con 0x7f0220061600 2022-06-29T15:51:47.835 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.826+0000 7f021effd700 1 -- 172.21.15.47:0/1756461159 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02200b2170 con 0x7f0220061600 2022-06-29T15:51:47.836 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.826+0000 7f021ffff700 1 -- 172.21.15.47:0/1756461159 >> v1:172.21.15.64:6789/0 conn(0x7f0220061600 legacy=0x7f0220146b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:47.836 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.826+0000 7f021ffff700 1 -- 172.21.15.47:0/1756461159 >> v1:172.21.15.47:6789/0 conn(0x7f02200601a0 legacy=0x7f0220140520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:47.836 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.826+0000 7f021ffff700 1 -- 172.21.15.47:0/1756461159 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02200a9770 con 0x7f02200b2a90 2022-06-29T15:51:47.836 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.826+0000 7f021ffff700 1 --2- 172.21.15.47:0/1756461159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02200b2a90 0x7f0220061050 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7f0210002f70 tx=0x7f0210006df0).ready entity=mon.1 client_cookie=891111f1db201c72 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:47.837 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.826+0000 7f021effd700 1 -- 172.21.15.47:0/1756461159 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f021000d540 con 0x7f02200b2a90 2022-06-29T15:51:47.837 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.826+0000 7f021effd700 1 -- 172.21.15.47:0/1756461159 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0210000e90 con 0x7f02200b2a90 2022-06-29T15:51:47.837 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.826+0000 7f021effd700 1 -- 172.21.15.47:0/1756461159 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f021000d890 con 0x7f02200b2a90 2022-06-29T15:51:47.837 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.826+0000 7f02262a2700 1 -- 172.21.15.47:0/1756461159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02200b2a90 msgr2=0x7f0220061050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:47.838 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.826+0000 7f02262a2700 1 --2- 172.21.15.47:0/1756461159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02200b2a90 0x7f0220061050 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7f0210002f70 tx=0x7f0210006df0).stop 2022-06-29T15:51:47.838 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.826+0000 7f02262a2700 1 -- 172.21.15.47:0/1756461159 shutdown_connections 2022-06-29T15:51:47.838 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.826+0000 7f02262a2700 1 --2- 172.21.15.47:0/1756461159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02200b2a90 0x7f0220061050 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:47.838 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.826+0000 7f02262a2700 1 -- 172.21.15.47:0/1756461159 >> 172.21.15.47:0/1756461159 conn(0x7f022005f0b0 msgr2=0x7f022005f490 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:47.839 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.826+0000 7f02262a2700 1 -- 172.21.15.47:0/1756461159 shutdown_connections 2022-06-29T15:51:47.839 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.826+0000 7f02262a2700 1 -- 172.21.15.47:0/1756461159 wait complete. 2022-06-29T15:51:47.841 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.826+0000 7f02262a2700 1 Processor -- start 2022-06-29T15:51:47.841 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.826+0000 7f02262a2700 1 -- start start 2022-06-29T15:51:47.842 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.830+0000 7f02262a2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02200601a0 0x7f02200ac7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:47.842 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.830+0000 7f02262a2700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0220061600 0x7f02200a9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:47.842 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.830+0000 7f02262a2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02200b2a90 0x7f02201525f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:47.842 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.830+0000 7f02262a2700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0220147060 con 0x7f02200601a0 2022-06-29T15:51:47.843 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.830+0000 7f02262a2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0220146b20 con 0x7f02200b2a90 2022-06-29T15:51:47.843 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.830+0000 7f02262a2700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0220146dc0 con 0x7f0220061600 2022-06-29T15:51:47.843 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.830+0000 7f021ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02200601a0 0x7f02200ac7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:47.843 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.830+0000 7f021ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02200601a0 0x7f02200ac7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54240/0 (socket says 172.21.15.47:54240) 2022-06-29T15:51:47.843 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.830+0000 7f021ffff700 1 -- 172.21.15.47:0/3655963192 learned_addr learned my addr 172.21.15.47:0/3655963192 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:47.844 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.830+0000 7f022483f700 1 --2- 172.21.15.47:0/3655963192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02200b2a90 0x7f02201525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:47.844 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.830+0000 7f021f7fe700 1 --2- 172.21.15.47:0/3655963192 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0220061600 0x7f02200a9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:47.844 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.830+0000 7f021ffff700 1 -- 172.21.15.47:0/3655963192 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0220061600 msgr2=0x7f02200a9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:47.844 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.830+0000 7f021ffff700 1 --2- 172.21.15.47:0/3655963192 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0220061600 0x7f02200a9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:47.845 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.830+0000 7f021ffff700 1 -- 172.21.15.47:0/3655963192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02200b2a90 msgr2=0x7f02201525f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:47.845 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.830+0000 7f021ffff700 1 --2- 172.21.15.47:0/3655963192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02200b2a90 0x7f02201525f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:47.845 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.830+0000 7f021ffff700 1 -- 172.21.15.47:0/3655963192 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f021000b180 con 0x7f02200601a0 2022-06-29T15:51:47.845 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.830+0000 7f021ffff700 1 --2- 172.21.15.47:0/3655963192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02200601a0 0x7f02200ac7c0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 rx=0x7f0210002f70 tx=0x7f0210014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:47.845 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.834+0000 7f021d7fa700 1 -- 172.21.15.47:0/3655963192 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0210016070 con 0x7f02200601a0 2022-06-29T15:51:47.846 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.834+0000 7f02262a2700 1 -- 172.21.15.47:0/3655963192 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0220145a00 con 0x7f02200601a0 2022-06-29T15:51:47.846 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.834+0000 7f021d7fa700 1 -- 172.21.15.47:0/3655963192 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0210013850 con 0x7f02200601a0 2022-06-29T15:51:47.846 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.834+0000 7f021d7fa700 1 -- 172.21.15.47:0/3655963192 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0210013bb0 con 0x7f02200601a0 2022-06-29T15:51:47.846 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.834+0000 7f02262a2700 1 -- 172.21.15.47:0/3655963192 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02200aa1c0 con 0x7f02200601a0 2022-06-29T15:51:47.847 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.834+0000 7f02262a2700 1 -- 172.21.15.47:0/3655963192 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02200573d0 con 0x7f02200601a0 2022-06-29T15:51:47.848 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.834+0000 7f021d7fa700 1 -- 172.21.15.47:0/3655963192 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f021000f030 con 0x7f02200601a0 2022-06-29T15:51:47.848 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.834+0000 7f021d7fa700 1 --2- 172.21.15.47:0/3655963192 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0204053950 0x7f0204055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:47.848 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.834+0000 7f021f7fe700 1 --2- 172.21.15.47:0/3655963192 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0204053950 0x7f0204055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:47.848 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.838+0000 7f021f7fe700 1 --2- 172.21.15.47:0/3655963192 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0204053950 0x7f0204055e10 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7f020800d000 tx=0x7f0208007a00).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:47.849 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.838+0000 7f021d7fa700 1 -- 172.21.15.47:0/3655963192 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f0210047610 con 0x7f02200601a0 2022-06-29T15:51:47.849 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.842+0000 7f021d7fa700 1 -- 172.21.15.47:0/3655963192 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f02100439b0 con 0x7f02200601a0 2022-06-29T15:51:47.985 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:47.978+0000 7f02262a2700 1 -- 172.21.15.47:0/3655963192 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f0220069e20 con 0x7f02200601a0 2022-06-29T15:51:48.858 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.850+0000 7f021d7fa700 1 -- 172.21.15.47:0/3655963192 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v147) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0210023030 con 0x7f02200601a0 2022-06-29T15:51:48.862 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.854+0000 7f02262a2700 1 -- 172.21.15.47:0/3655963192 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0204053950 msgr2=0x7f0204055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:48.862 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.854+0000 7f02262a2700 1 --2- 172.21.15.47:0/3655963192 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0204053950 0x7f0204055e10 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7f020800d000 tx=0x7f0208007a00).stop 2022-06-29T15:51:48.863 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.854+0000 7f02262a2700 1 -- 172.21.15.47:0/3655963192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02200601a0 msgr2=0x7f02200ac7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:48.863 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.854+0000 7f02262a2700 1 --2- 172.21.15.47:0/3655963192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02200601a0 0x7f02200ac7c0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 rx=0x7f0210002f70 tx=0x7f0210014000).stop 2022-06-29T15:51:48.863 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.854+0000 7f02262a2700 1 -- 172.21.15.47:0/3655963192 shutdown_connections 2022-06-29T15:51:48.863 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.854+0000 7f02262a2700 1 --2- 172.21.15.47:0/3655963192 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0204053950 0x7f0204055e10 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:48.863 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.854+0000 7f02262a2700 1 --2- 172.21.15.47:0/3655963192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02200b2a90 0x7f02201525f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:48.864 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.854+0000 7f02262a2700 1 --2- 172.21.15.47:0/3655963192 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0220061600 0x7f02200a9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:48.864 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.854+0000 7f02262a2700 1 --2- 172.21.15.47:0/3655963192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02200601a0 0x7f02200ac7c0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:48.864 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.854+0000 7f02262a2700 1 -- 172.21.15.47:0/3655963192 >> 172.21.15.47:0/3655963192 conn(0x7f022005f0b0 msgr2=0x7f02200aef70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:48.864 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.854+0000 7f02262a2700 1 -- 172.21.15.47:0/3655963192 shutdown_connections 2022-06-29T15:51:48.865 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.854+0000 7f02262a2700 1 -- 172.21.15.47:0/3655963192 wait complete. 2022-06-29T15:51:48.865 INFO:teuthology.orchestra.run.smithi047.stderr: removed all blacklist entries 2022-06-29T15:51:48.880 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2022-06-29T15:51:48.880 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-06-29T15:51:48.985 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.978+0000 7fd261a2f700 1 Processor -- start 2022-06-29T15:51:48.986 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.978+0000 7fd261a2f700 1 -- start start 2022-06-29T15:51:49.001 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.982+0000 7fd261a2f700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd25c0619b0 0x7fd25c13aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:49.001 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.982+0000 7fd261a2f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd25c0b0f30 0x7fd25c13af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:49.001 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.982+0000 7fd261a2f700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd25c061d90 con 0x7fd25c060ff0 2022-06-29T15:51:49.002 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.982+0000 7fd261a2f700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd25c0b04b0 con 0x7fd25c0b0f30 2022-06-29T15:51:49.002 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.982+0000 7fd261a2f700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c0b0750 con 0x7fd25c0619b0 2022-06-29T15:51:49.002 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.982+0000 7fd25affd700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd25c0619b0 0x7fd25c13aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:49.002 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.982+0000 7fd25a7fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd25c0b0f30 0x7fd25c13af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:49.002 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.982+0000 7fd25b7fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd25c060ff0 0x7fd25c144a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:47374/0 (socket says 172.21.15.47:47374) 2022-06-29T15:51:49.003 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.982+0000 7fd25b7fe700 1 -- 172.21.15.47:0/1205454659 learned_addr learned my addr 172.21.15.47:0/1205454659 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:49.003 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.982+0000 7fd25affd700 1 --2- 172.21.15.47:0/1205454659 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd25c0619b0 0x7fd25c13aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:51354/0 (socket says 172.21.15.47:51354) 2022-06-29T15:51:49.003 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.982+0000 7fd25affd700 1 -- 172.21.15.47:0/1205454659 >> v2:172.21.15.64:3300/0 conn(0x7fd25c0b0f30 msgr2=0x7fd25c13af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:49.003 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.982+0000 7fd25affd700 1 --2- 172.21.15.47:0/1205454659 >> v2:172.21.15.64:3300/0 conn(0x7fd25c0b0f30 0x7fd25c13af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:49.004 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.982+0000 7fd25affd700 1 -- 172.21.15.47:0/1205454659 >> v1:172.21.15.64:6789/0 conn(0x7fd25c060ff0 legacy=0x7fd25c144a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:49.004 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.982+0000 7fd259ffb700 1 -- 172.21.15.47:0/1205454659 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2801344231 0 0) 0x7fd25c061d90 con 0x7fd25c060ff0 2022-06-29T15:51:49.004 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.982+0000 7fd25affd700 1 -- 172.21.15.47:0/1205454659 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd25c144570 con 0x7fd25c0619b0 2022-06-29T15:51:49.004 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.982+0000 7fd25affd700 1 --2- 172.21.15.47:0/1205454659 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd25c0619b0 0x7fd25c13aa30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7fd25c060260 tx=0x7fd24c006df0).ready entity=mon.2 client_cookie=892d09cdb30b5627 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:49.005 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.982+0000 7fd259ffb700 1 -- 172.21.15.47:0/1205454659 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd24c00d570 con 0x7fd25c0619b0 2022-06-29T15:51:49.005 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.982+0000 7fd259ffb700 1 -- 172.21.15.47:0/1205454659 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd24c000e90 con 0x7fd25c0619b0 2022-06-29T15:51:49.005 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.982+0000 7fd259ffb700 1 -- 172.21.15.47:0/1205454659 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd24c00d8c0 con 0x7fd25c0619b0 2022-06-29T15:51:49.005 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.982+0000 7fd261a2f700 1 -- 172.21.15.47:0/1205454659 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd25c0619b0 msgr2=0x7fd25c13aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:49.005 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.982+0000 7fd261a2f700 1 --2- 172.21.15.47:0/1205454659 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd25c0619b0 0x7fd25c13aa30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7fd25c060260 tx=0x7fd24c006df0).stop 2022-06-29T15:51:49.006 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.982+0000 7fd261a2f700 1 -- 172.21.15.47:0/1205454659 shutdown_connections 2022-06-29T15:51:49.006 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.982+0000 7fd261a2f700 1 --2- 172.21.15.47:0/1205454659 >> v2:172.21.15.64:3300/0 conn(0x7fd25c0b0f30 0x7fd25c13af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:49.006 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.982+0000 7fd261a2f700 1 --2- 172.21.15.47:0/1205454659 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd25c0619b0 0x7fd25c13aa30 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:49.006 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.982+0000 7fd261a2f700 1 -- 172.21.15.47:0/1205454659 >> 172.21.15.47:0/1205454659 conn(0x7fd25c05c510 msgr2=0x7fd25c063800 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:49.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.982+0000 7fd261a2f700 1 -- 172.21.15.47:0/1205454659 shutdown_connections 2022-06-29T15:51:49.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.982+0000 7fd261a2f700 1 -- 172.21.15.47:0/1205454659 wait complete. 2022-06-29T15:51:49.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.986+0000 7fd261a2f700 1 Processor -- start 2022-06-29T15:51:49.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.986+0000 7fd261a2f700 1 -- start start 2022-06-29T15:51:49.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.986+0000 7fd261a2f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd25c060ff0 0x7fd25c0b52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:49.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.986+0000 7fd261a2f700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd25c0619b0 0x7fd25c140320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:49.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.986+0000 7fd261a2f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd25c0b0f30 0x7fd25c066ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:49.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.986+0000 7fd261a2f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c0af940 con 0x7fd25c060ff0 2022-06-29T15:51:49.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.986+0000 7fd261a2f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c0b48b0 con 0x7fd25c0b0f30 2022-06-29T15:51:49.009 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.986+0000 7fd261a2f700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c0b4170 con 0x7fd25c0619b0 2022-06-29T15:51:49.009 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.986+0000 7fd25affd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd25c060ff0 0x7fd25c0b52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:49.009 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.986+0000 7fd25affd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd25c060ff0 0x7fd25c0b52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54258/0 (socket says 172.21.15.47:54258) 2022-06-29T15:51:49.009 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.986+0000 7fd25affd700 1 -- 172.21.15.47:0/3160166031 learned_addr learned my addr 172.21.15.47:0/3160166031 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:49.010 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.986+0000 7fd25a7fc700 1 --2- 172.21.15.47:0/3160166031 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd25c0619b0 0x7fd25c140320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:49.010 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.986+0000 7fd25b7fe700 1 --2- 172.21.15.47:0/3160166031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd25c0b0f30 0x7fd25c066ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:49.010 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.986+0000 7fd25affd700 1 -- 172.21.15.47:0/3160166031 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd25c0619b0 msgr2=0x7fd25c140320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:49.010 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.986+0000 7fd25affd700 1 --2- 172.21.15.47:0/3160166031 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd25c0619b0 0x7fd25c140320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:49.011 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.986+0000 7fd25affd700 1 -- 172.21.15.47:0/3160166031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd25c0b0f30 msgr2=0x7fd25c066ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:49.011 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.986+0000 7fd25affd700 1 --2- 172.21.15.47:0/3160166031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd25c0b0f30 0x7fd25c066ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:49.011 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.986+0000 7fd25affd700 1 -- 172.21.15.47:0/3160166031 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd24c00b180 con 0x7fd25c060ff0 2022-06-29T15:51:49.011 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.986+0000 7fd25a7fc700 1 --2- 172.21.15.47:0/3160166031 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd25c0619b0 0x7fd25c140320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:51:49.011 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.986+0000 7fd25affd700 1 --2- 172.21.15.47:0/3160166031 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd25c060ff0 0x7fd25c0b52d0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 rx=0x7fd25c060260 tx=0x7fd24c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:49.012 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.990+0000 7fd24bfff700 1 -- 172.21.15.47:0/3160166031 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd24c013de0 con 0x7fd25c060ff0 2022-06-29T15:51:49.012 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.990+0000 7fd24bfff700 1 -- 172.21.15.47:0/3160166031 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd24c002f70 con 0x7fd25c060ff0 2022-06-29T15:51:49.012 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.990+0000 7fd261a2f700 1 -- 172.21.15.47:0/3160166031 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd25c0b4410 con 0x7fd25c060ff0 2022-06-29T15:51:49.012 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.990+0000 7fd24bfff700 1 -- 172.21.15.47:0/3160166031 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd24c01e5f0 con 0x7fd25c060ff0 2022-06-29T15:51:49.013 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.990+0000 7fd261a2f700 1 -- 172.21.15.47:0/3160166031 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd25c0664d0 con 0x7fd25c060ff0 2022-06-29T15:51:49.013 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.990+0000 7fd24bfff700 1 -- 172.21.15.47:0/3160166031 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd24c025030 con 0x7fd25c060ff0 2022-06-29T15:51:49.013 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.990+0000 7fd261a2f700 1 -- 172.21.15.47:0/3160166031 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd25c0573d0 con 0x7fd25c060ff0 2022-06-29T15:51:49.013 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.990+0000 7fd24bfff700 1 --2- 172.21.15.47:0/3160166031 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd238053b00 0x7fd238055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:49.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.990+0000 7fd24bfff700 1 -- 172.21.15.47:0/3160166031 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fd24c06c8e0 con 0x7fd25c060ff0 2022-06-29T15:51:49.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.990+0000 7fd25a7fc700 1 --2- 172.21.15.47:0/3160166031 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd238053b00 0x7fd238055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:49.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.990+0000 7fd25a7fc700 1 --2- 172.21.15.47:0/3160166031 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd238053b00 0x7fd238055fc0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7fd25000b000 tx=0x7fd250006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:49.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:48.998+0000 7fd24bfff700 1 -- 172.21.15.47:0/3160166031 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd24c0439e0 con 0x7fd25c060ff0 2022-06-29T15:51:49.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.170+0000 7fd261a2f700 1 -- 172.21.15.47:0/3160166031 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fd25c066a00 con 0x7fd25c060ff0 2022-06-29T15:51:49.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.170+0000 7fd24bfff700 1 -- 172.21.15.47:0/3160166031 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v34) v1 ==== 80+0+4255 (secure 0 0 0) 0x7fd24c023070 con 0x7fd25c060ff0 2022-06-29T15:51:49.177 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:49.178 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2022-06-29T15:51:49.178 INFO:teuthology.orchestra.run.smithi047.stdout: "auth_dump": [ 2022-06-29T15:51:49.178 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:49.178 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.a", 2022-06-29T15:51:49.179 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxiQJFFMxAA9Tv7FJD9u/BKvbkuKld6Kg==", 2022-06-29T15:51:49.179 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:51:49.179 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:51:49.179 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:51:49.180 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:51:49.180 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:51:49.180 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:49.180 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:49.180 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:49.181 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.b", 2022-06-29T15:51:49.181 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC2crxibt6FARAAEf2bwXKFhXKtqr1nN0rdXQ==", 2022-06-29T15:51:49.181 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:51:49.181 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:51:49.182 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:51:49.182 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:51:49.182 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:51:49.182 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:49.183 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:49.183 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:49.183 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.c", 2022-06-29T15:51:49.183 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxi/4ZMORAANpLbgTm3ehqi80USc3swGg==", 2022-06-29T15:51:49.183 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:51:49.184 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:51:49.184 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:51:49.184 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:51:49.184 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:51:49.185 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:49.185 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:49.185 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:49.185 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.0", 2022-06-29T15:51:49.185 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC2crxiY0YsMxAAZivb+CxL/w7vZwQv37Y4oQ==", 2022-06-29T15:51:49.186 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:51:49.186 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:51:49.186 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:51:49.186 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:51:49.187 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:49.187 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:49.187 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:49.187 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.1", 2022-06-29T15:51:49.188 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC5crxitLkKFxAA8+++LUUKLiO9SAuAfYgT5g==", 2022-06-29T15:51:49.188 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:51:49.188 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:51:49.188 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:51:49.189 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:51:49.189 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:49.189 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:49.189 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:49.189 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.2", 2022-06-29T15:51:49.190 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC7crxi1oM9NxAA2FfLdNt32YPwlyNdGQpaQA==", 2022-06-29T15:51:49.190 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:51:49.190 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:51:49.190 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:51:49.191 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:51:49.191 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:49.191 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:49.191 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:49.192 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.3", 2022-06-29T15:51:49.192 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC+crxiTsBgGhAA+gLBBDaskPbmRNWzgKwcHA==", 2022-06-29T15:51:49.192 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:51:49.192 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:51:49.192 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:51:49.193 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:51:49.193 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:49.193 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:49.193 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:49.194 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.4", 2022-06-29T15:51:49.194 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDDcrxivJebIhAAfwWJa7tK4hamkk1/1CSSfw==", 2022-06-29T15:51:49.194 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:51:49.194 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:51:49.195 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:51:49.195 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:51:49.195 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:49.195 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:49.195 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:49.196 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.5", 2022-06-29T15:51:49.196 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDGcrxijuozBhAAt+Dx3wH3WKp3DrbCbhmh8Q==", 2022-06-29T15:51:49.196 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:51:49.196 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:51:49.197 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:51:49.197 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:51:49.197 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:49.197 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:49.197 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:49.198 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.6", 2022-06-29T15:51:49.198 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDIcrxi91kXJxAA9JX7n98P394bD41Q9LUdvQ==", 2022-06-29T15:51:49.198 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:51:49.198 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:51:49.199 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:51:49.199 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:51:49.199 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:49.199 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:49.199 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:49.200 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.7", 2022-06-29T15:51:49.200 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDLcrxiHj0ZDRAASy6fAix8TmEZbAGtuFCfFg==", 2022-06-29T15:51:49.200 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:51:49.200 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:51:49.201 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:51:49.201 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:51:49.201 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:49.201 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:49.202 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:49.202 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.0", 2022-06-29T15:51:49.202 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC2crxiVDlfBxAAezkUfg4e2g0Rfakaa/Ek1Q==", 2022-06-29T15:51:49.202 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:51:49.202 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:51:49.203 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow r", 2022-06-29T15:51:49.203 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-29T15:51:49.203 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:49.203 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:49.204 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:49.204 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.1", 2022-06-29T15:51:49.204 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC2crxisXJ1CxAAhl07oF7AW66NdzfvvZTJmA==", 2022-06-29T15:51:49.204 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:51:49.205 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:51:49.205 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow r", 2022-06-29T15:51:49.205 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-29T15:51:49.205 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:49.206 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:49.206 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:49.206 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.admin", 2022-06-29T15:51:49.206 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxiW7HSFRAAhYqoGU9GFqj2HfUVGValSA==", 2022-06-29T15:51:49.206 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:51:49.207 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2022-06-29T15:51:49.207 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:51:49.207 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:51:49.207 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:51:49.208 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:49.208 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:49.208 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:49.208 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.x", 2022-06-29T15:51:49.209 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxidmuIMBAAcpNlYVu5+icCHg+yIjcKXw==", 2022-06-29T15:51:49.209 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:51:49.209 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2022-06-29T15:51:49.209 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2022-06-29T15:51:49.209 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:51:49.210 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:49.210 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:49.210 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:49.210 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.y", 2022-06-29T15:51:49.211 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxisSMFLBAAv03r4OoPjOP9nI9G/LK0iw==", 2022-06-29T15:51:49.211 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:51:49.211 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2022-06-29T15:51:49.211 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2022-06-29T15:51:49.211 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:51:49.212 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:49.212 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:49.212 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:51:49.212 INFO:teuthology.orchestra.run.smithi047.stdout:} 2022-06-29T15:51:49.215 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.174+0000 7fd261a2f700 1 -- 172.21.15.47:0/3160166031 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd238053b00 msgr2=0x7fd238055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:49.215 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.174+0000 7fd261a2f700 1 --2- 172.21.15.47:0/3160166031 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd238053b00 0x7fd238055fc0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7fd25000b000 tx=0x7fd250006df0).stop 2022-06-29T15:51:49.215 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.174+0000 7fd261a2f700 1 -- 172.21.15.47:0/3160166031 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd25c060ff0 msgr2=0x7fd25c0b52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:49.215 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.174+0000 7fd261a2f700 1 --2- 172.21.15.47:0/3160166031 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd25c060ff0 0x7fd25c0b52d0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 rx=0x7fd25c060260 tx=0x7fd24c014000).stop 2022-06-29T15:51:49.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.174+0000 7fd261a2f700 1 -- 172.21.15.47:0/3160166031 shutdown_connections 2022-06-29T15:51:49.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.174+0000 7fd261a2f700 1 --2- 172.21.15.47:0/3160166031 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd238053b00 0x7fd238055fc0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:49.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.174+0000 7fd261a2f700 1 --2- 172.21.15.47:0/3160166031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd25c0b0f30 0x7fd25c066ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:49.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.174+0000 7fd261a2f700 1 --2- 172.21.15.47:0/3160166031 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd25c0619b0 0x7fd25c140320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:49.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.174+0000 7fd261a2f700 1 --2- 172.21.15.47:0/3160166031 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd25c060ff0 0x7fd25c0b52d0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:49.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.174+0000 7fd261a2f700 1 -- 172.21.15.47:0/3160166031 >> 172.21.15.47:0/3160166031 conn(0x7fd25c05c510 msgr2=0x7fd25c144930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:49.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.174+0000 7fd261a2f700 1 -- 172.21.15.47:0/3160166031 shutdown_connections 2022-06-29T15:51:49.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.174+0000 7fd261a2f700 1 -- 172.21.15.47:0/3160166031 wait complete. 2022-06-29T15:51:49.218 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:51:49.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.314+0000 7f57e32f4700 1 Processor -- start 2022-06-29T15:51:49.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.314+0000 7f57e32f4700 1 -- start start 2022-06-29T15:51:49.335 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.318+0000 7f57e32f4700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f57dc0611b0 0x7f57dc146640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:49.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.318+0000 7f57e32f4700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f57dc0b0cf0 0x7f57dc146b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:49.336 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.318+0000 7f57e32f4700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f57dc0b06d0 0x7f57dc0667e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:49.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.318+0000 7f57e32f4700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57dc0b4b10 con 0x7f57dc0b06d0 2022-06-29T15:51:49.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.318+0000 7f57e32f4700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57dc0621d0 con 0x7f57dc0b0cf0 2022-06-29T15:51:49.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.318+0000 7f57e32f4700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc0b4520 con 0x7f57dc0611b0 2022-06-29T15:51:49.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.318+0000 7f57e1891700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f57dc0b06d0 0x7f57dc0667e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:49.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.318+0000 7f57e1891700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f57dc0b06d0 0x7f57dc0667e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54270/0 (socket says 172.21.15.47:54270) 2022-06-29T15:51:49.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.318+0000 7f57e1891700 1 -- 172.21.15.47:0/2648375676 learned_addr learned my addr 172.21.15.47:0/2648375676 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:49.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.318+0000 7f57e088f700 1 --2- 172.21.15.47:0/2648375676 >> v2:172.21.15.64:3300/0 conn(0x7f57dc0b0cf0 0x7f57dc146b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:49.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.318+0000 7f57e1090700 1 --2- 172.21.15.47:0/2648375676 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f57dc0611b0 0x7f57dc146640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:49.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.318+0000 7f57e1891700 1 -- 172.21.15.47:0/2648375676 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f57dc0611b0 msgr2=0x7f57dc146640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:49.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.318+0000 7f57e1891700 1 --2- 172.21.15.47:0/2648375676 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f57dc0611b0 0x7f57dc146640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:49.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.318+0000 7f57e1891700 1 -- 172.21.15.47:0/2648375676 >> v2:172.21.15.64:3300/0 conn(0x7f57dc0b0cf0 msgr2=0x7f57dc146b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:49.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.318+0000 7f57e1891700 1 --2- 172.21.15.47:0/2648375676 >> v2:172.21.15.64:3300/0 conn(0x7f57dc0b0cf0 0x7f57dc146b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:49.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.318+0000 7f57e1891700 1 -- 172.21.15.47:0/2648375676 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57dc061590 con 0x7f57dc0b06d0 2022-06-29T15:51:49.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.318+0000 7f57e1891700 1 --2- 172.21.15.47:0/2648375676 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57dc0b06d0 0x7f57dc0667e0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 rx=0x7f57dc062310 tx=0x7f57d4009290).ready entity=mon.0 client_cookie=e39fbcf289e07138 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:49.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.318+0000 7f57dbfff700 1 -- 172.21.15.47:0/2648375676 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f57d400b830 con 0x7f57dc0b06d0 2022-06-29T15:51:49.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.318+0000 7f57dbfff700 1 -- 172.21.15.47:0/2648375676 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57d400b990 con 0x7f57dc0b06d0 2022-06-29T15:51:49.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.318+0000 7f57dbfff700 1 -- 172.21.15.47:0/2648375676 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f57d400bb00 con 0x7f57dc0b06d0 2022-06-29T15:51:49.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.318+0000 7f57e32f4700 1 -- 172.21.15.47:0/2648375676 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57dc0b06d0 msgr2=0x7f57dc0667e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:49.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.318+0000 7f57e32f4700 1 --2- 172.21.15.47:0/2648375676 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57dc0b06d0 0x7f57dc0667e0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 rx=0x7f57dc062310 tx=0x7f57d4009290).stop 2022-06-29T15:51:49.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.318+0000 7f57e32f4700 1 -- 172.21.15.47:0/2648375676 shutdown_connections 2022-06-29T15:51:49.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.318+0000 7f57e32f4700 1 --2- 172.21.15.47:0/2648375676 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57dc0b06d0 0x7f57dc0667e0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:49.346 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.318+0000 7f57e32f4700 1 --2- 172.21.15.47:0/2648375676 >> v2:172.21.15.64:3300/0 conn(0x7f57dc0b0cf0 0x7f57dc146b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:49.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.318+0000 7f57e32f4700 1 --2- 172.21.15.47:0/2648375676 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f57dc0611b0 0x7f57dc146640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:49.347 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.318+0000 7f57e32f4700 1 -- 172.21.15.47:0/2648375676 >> 172.21.15.47:0/2648375676 conn(0x7f57dc0b98b0 msgr2=0x7f57dc0b9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:49.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.318+0000 7f57e32f4700 1 -- 172.21.15.47:0/2648375676 shutdown_connections 2022-06-29T15:51:49.348 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.318+0000 7f57e32f4700 1 -- 172.21.15.47:0/2648375676 wait complete. 2022-06-29T15:51:49.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.318+0000 7f57e32f4700 1 Processor -- start 2022-06-29T15:51:49.349 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.318+0000 7f57e32f4700 1 -- start start 2022-06-29T15:51:49.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.322+0000 7f57e32f4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57dc0611b0 0x7f57dc0aff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:49.350 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.322+0000 7f57e32f4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57dc0b06d0 0x7f57dc149070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:49.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.322+0000 7f57e32f4700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f57dc0b0cf0 0x7f57dc0aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:49.351 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.322+0000 7f57e32f4700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc0b78f0 con 0x7f57dc0611b0 2022-06-29T15:51:49.352 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.322+0000 7f57e32f4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc0b7b90 con 0x7f57dc0b06d0 2022-06-29T15:51:49.352 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.322+0000 7f57e32f4700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc0b8e60 con 0x7f57dc0b0cf0 2022-06-29T15:51:49.352 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.322+0000 7f57e1090700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57dc0611b0 0x7f57dc0aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:49.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.322+0000 7f57e1090700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57dc0611b0 0x7f57dc0aff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54272/0 (socket says 172.21.15.47:54272) 2022-06-29T15:51:49.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.322+0000 7f57e1090700 1 -- 172.21.15.47:0/1386867238 learned_addr learned my addr 172.21.15.47:0/1386867238 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:49.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.322+0000 7f57e1891700 1 --2- 172.21.15.47:0/1386867238 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f57dc0b0cf0 0x7f57dc0aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:49.354 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.322+0000 7f57e088f700 1 --2- 172.21.15.47:0/1386867238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57dc0b06d0 0x7f57dc149070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:49.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.322+0000 7f57e1090700 1 -- 172.21.15.47:0/1386867238 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f57dc0b0cf0 msgr2=0x7f57dc0aeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:49.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.322+0000 7f57e1090700 1 --2- 172.21.15.47:0/1386867238 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f57dc0b0cf0 0x7f57dc0aeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:49.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.322+0000 7f57e1090700 1 -- 172.21.15.47:0/1386867238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57dc0b06d0 msgr2=0x7f57dc149070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:49.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.322+0000 7f57e1090700 1 --2- 172.21.15.47:0/1386867238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57dc0b06d0 0x7f57dc149070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:49.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.322+0000 7f57e1090700 1 -- 172.21.15.47:0/1386867238 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d4009530 con 0x7f57dc0611b0 2022-06-29T15:51:49.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.322+0000 7f57e088f700 1 --2- 172.21.15.47:0/1386867238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57dc0b06d0 0x7f57dc149070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:51:49.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.322+0000 7f57e1090700 1 --2- 172.21.15.47:0/1386867238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57dc0611b0 0x7f57dc0aff10 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 rx=0x7f57cc0030b0 tx=0x7f57cc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:49.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.322+0000 7f57da7fc700 1 -- 172.21.15.47:0/1386867238 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f57cc011780 con 0x7f57dc0611b0 2022-06-29T15:51:49.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.322+0000 7f57da7fc700 1 -- 172.21.15.47:0/1386867238 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57cc0118e0 con 0x7f57dc0611b0 2022-06-29T15:51:49.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.322+0000 7f57e32f4700 1 -- 172.21.15.47:0/1386867238 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57dc0b9100 con 0x7f57dc0611b0 2022-06-29T15:51:49.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.322+0000 7f57da7fc700 1 -- 172.21.15.47:0/1386867238 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f57cc011c40 con 0x7f57dc0611b0 2022-06-29T15:51:49.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.322+0000 7f57e32f4700 1 -- 172.21.15.47:0/1386867238 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57dc13dc70 con 0x7f57dc0611b0 2022-06-29T15:51:49.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.322+0000 7f57e32f4700 1 -- 172.21.15.47:0/1386867238 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57dc149560 con 0x7f57dc0611b0 2022-06-29T15:51:49.358 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.326+0000 7f57da7fc700 1 -- 172.21.15.47:0/1386867238 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f57cc012420 con 0x7f57dc0611b0 2022-06-29T15:51:49.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.326+0000 7f57da7fc700 1 --2- 172.21.15.47:0/1386867238 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f57bc053b00 0x7f57bc055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:49.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.326+0000 7f57da7fc700 1 -- 172.21.15.47:0/1386867238 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f57cc06bd10 con 0x7f57dc0611b0 2022-06-29T15:51:49.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.326+0000 7f57e088f700 1 --2- 172.21.15.47:0/1386867238 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f57bc053b00 0x7f57bc055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:49.359 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.326+0000 7f57e088f700 1 --2- 172.21.15.47:0/1386867238 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f57bc053b00 0x7f57bc055fc0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7f57d0008000 tx=0x7f57d0008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:49.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.330+0000 7f57da7fc700 1 -- 172.21.15.47:0/1386867238 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f57cc071020 con 0x7f57dc0611b0 2022-06-29T15:51:49.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.458+0000 7f57e32f4700 1 -- 172.21.15.47:0/1386867238 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f57dc0573d0 con 0x7f57dc0611b0 2022-06-29T15:51:49.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.458+0000 7f57da7fc700 1 -- 172.21.15.47:0/1386867238 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v147) v1 ==== 74+0+13424 (secure 0 0 0) 0x7f57cc042ca0 con 0x7f57dc0611b0 2022-06-29T15:51:49.467 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:49.467 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":147,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:51:48.846114+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:51:49.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.462+0000 7f57e32f4700 1 -- 172.21.15.47:0/1386867238 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f57bc053b00 msgr2=0x7f57bc055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:49.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.462+0000 7f57e32f4700 1 --2- 172.21.15.47:0/1386867238 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f57bc053b00 0x7f57bc055fc0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7f57d0008000 tx=0x7f57d0008040).stop 2022-06-29T15:51:49.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.462+0000 7f57e32f4700 1 -- 172.21.15.47:0/1386867238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57dc0611b0 msgr2=0x7f57dc0aff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:49.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.462+0000 7f57e32f4700 1 --2- 172.21.15.47:0/1386867238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57dc0611b0 0x7f57dc0aff10 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 rx=0x7f57cc0030b0 tx=0x7f57cc008040).stop 2022-06-29T15:51:49.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.462+0000 7f57e32f4700 1 -- 172.21.15.47:0/1386867238 shutdown_connections 2022-06-29T15:51:49.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.462+0000 7f57e32f4700 1 --2- 172.21.15.47:0/1386867238 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f57dc0b0cf0 0x7f57dc0aeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:49.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.462+0000 7f57e32f4700 1 --2- 172.21.15.47:0/1386867238 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f57bc053b00 0x7f57bc055fc0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:49.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.462+0000 7f57e32f4700 1 --2- 172.21.15.47:0/1386867238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57dc0b06d0 0x7f57dc149070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:49.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.462+0000 7f57e32f4700 1 --2- 172.21.15.47:0/1386867238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57dc0611b0 0x7f57dc0aff10 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:49.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.462+0000 7f57e32f4700 1 -- 172.21.15.47:0/1386867238 >> 172.21.15.47:0/1386867238 conn(0x7f57dc0b98b0 msgr2=0x7f57dc1430f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:49.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.462+0000 7f57e32f4700 1 -- 172.21.15.47:0/1386867238 shutdown_connections 2022-06-29T15:51:49.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.462+0000 7f57e32f4700 1 -- 172.21.15.47:0/1386867238 wait complete. 2022-06-29T15:51:49.481 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-29T15:42:16.971431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-29T15:42:25.290884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-06-29T15:51:49.482 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-29T15:51:49.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.582+0000 7f92bbfff700 1 Processor -- start 2022-06-29T15:51:49.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.582+0000 7f92bbfff700 1 -- start start 2022-06-29T15:51:49.594 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.586+0000 7f92bbfff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92bc099d00 0x7f92bc09a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:49.594 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.586+0000 7f92bbfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f92bc14b0c0 0x7f92bc150d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:49.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.586+0000 7f92bbfff700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92bc0b1c00 con 0x7f92bc060920 2022-06-29T15:51:49.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.586+0000 7f92bbfff700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92bc0b1ea0 con 0x7f92bc14b0c0 2022-06-29T15:51:49.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.586+0000 7f92bbfff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92bc098210 con 0x7f92bc099d00 2022-06-29T15:51:49.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.586+0000 7f92b27fc700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f92bc060920 0x7f92bc14a9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39834/0 (socket says 172.21.15.47:39834) 2022-06-29T15:51:49.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.586+0000 7f92b27fc700 1 -- 172.21.15.47:0/1594182464 learned_addr learned my addr 172.21.15.47:0/1594182464 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:49.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.586+0000 7f92bb7fe700 1 --2- 172.21.15.47:0/1594182464 >> v2:172.21.15.47:3300/0 conn(0x7f92bc14b0c0 0x7f92bc150d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:49.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.586+0000 7f92baffd700 1 --2- 172.21.15.47:0/1594182464 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92bc099d00 0x7f92bc09a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:49.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.586+0000 7f92ba7fc700 1 -- 172.21.15.47:0/1594182464 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1515234091 0 0) 0x7f92bc0b1c00 con 0x7f92bc060920 2022-06-29T15:51:49.597 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.586+0000 7f92ba7fc700 1 -- 172.21.15.47:0/1594182464 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f929c003260 con 0x7f92bc060920 2022-06-29T15:51:49.597 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.586+0000 7f92bb7fe700 1 -- 172.21.15.47:0/1594182464 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92bc099d00 msgr2=0x7f92bc09a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:49.597 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.586+0000 7f92bb7fe700 1 --2- 172.21.15.47:0/1594182464 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92bc099d00 0x7f92bc09a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:49.597 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.586+0000 7f92bb7fe700 1 -- 172.21.15.47:0/1594182464 >> v1:172.21.15.47:6789/0 conn(0x7f92bc060920 legacy=0x7f92bc14a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:49.597 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.586+0000 7f92bb7fe700 1 -- 172.21.15.47:0/1594182464 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92bc0984b0 con 0x7f92bc14b0c0 2022-06-29T15:51:49.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.586+0000 7f92bb7fe700 1 --2- 172.21.15.47:0/1594182464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92bc14b0c0 0x7f92bc150d70 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 rx=0x7f92b40032f0 tx=0x7f92b40091d0).ready entity=mon.0 client_cookie=76dec147594d3758 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:49.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.586+0000 7f92ba7fc700 1 -- 172.21.15.47:0/1594182464 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92b400b600 con 0x7f92bc14b0c0 2022-06-29T15:51:49.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.586+0000 7f92ba7fc700 1 -- 172.21.15.47:0/1594182464 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b400b760 con 0x7f92bc14b0c0 2022-06-29T15:51:49.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.586+0000 7f92ba7fc700 1 -- 172.21.15.47:0/1594182464 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92b400b8d0 con 0x7f92bc14b0c0 2022-06-29T15:51:49.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.586+0000 7f92bbfff700 1 -- 172.21.15.47:0/1594182464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92bc14b0c0 msgr2=0x7f92bc150d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:49.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.586+0000 7f92bbfff700 1 --2- 172.21.15.47:0/1594182464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92bc14b0c0 0x7f92bc150d70 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 rx=0x7f92b40032f0 tx=0x7f92b40091d0).stop 2022-06-29T15:51:49.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.586+0000 7f92bbfff700 1 -- 172.21.15.47:0/1594182464 shutdown_connections 2022-06-29T15:51:49.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.586+0000 7f92bbfff700 1 --2- 172.21.15.47:0/1594182464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92bc14b0c0 0x7f92bc150d70 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:49.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.586+0000 7f92bbfff700 1 --2- 172.21.15.47:0/1594182464 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92bc099d00 0x7f92bc09a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:49.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.586+0000 7f92bbfff700 1 -- 172.21.15.47:0/1594182464 >> 172.21.15.47:0/1594182464 conn(0x7f92bc062df0 msgr2=0x7f92bc0631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:49.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.586+0000 7f92bbfff700 1 -- 172.21.15.47:0/1594182464 shutdown_connections 2022-06-29T15:51:49.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.586+0000 7f92bbfff700 1 -- 172.21.15.47:0/1594182464 wait complete. 2022-06-29T15:51:49.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.590+0000 7f92bbfff700 1 Processor -- start 2022-06-29T15:51:49.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.590+0000 7f92bbfff700 1 -- start start 2022-06-29T15:51:49.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.590+0000 7f92bbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92bc060920 0x7f92bc09aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:49.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.590+0000 7f92bbfff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92bc099d00 0x7f92bc0aeca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:49.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.590+0000 7f92bbfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92bc14b0c0 0x7f92bc0af190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:49.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.590+0000 7f92bbfff700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92bc159960 con 0x7f92bc14b0c0 2022-06-29T15:51:49.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.590+0000 7f92bbfff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92bc159b90 con 0x7f92bc060920 2022-06-29T15:51:49.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.590+0000 7f92bbfff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92bc159e30 con 0x7f92bc099d00 2022-06-29T15:51:49.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.590+0000 7f92bb7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92bc14b0c0 0x7f92bc0af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:49.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.590+0000 7f92bb7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92bc14b0c0 0x7f92bc0af190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54288/0 (socket says 172.21.15.47:54288) 2022-06-29T15:51:49.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.590+0000 7f92bb7fe700 1 -- 172.21.15.47:0/127033772 learned_addr learned my addr 172.21.15.47:0/127033772 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:49.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.590+0000 7f92baffd700 1 --2- 172.21.15.47:0/127033772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92bc060920 0x7f92bc09aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:49.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.590+0000 7f92b27fc700 1 --2- 172.21.15.47:0/127033772 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92bc099d00 0x7f92bc0aeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:49.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.590+0000 7f92bb7fe700 1 -- 172.21.15.47:0/127033772 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92bc099d00 msgr2=0x7f92bc0aeca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:49.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.590+0000 7f92bb7fe700 1 --2- 172.21.15.47:0/127033772 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92bc099d00 0x7f92bc0aeca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:49.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.590+0000 7f92bb7fe700 1 -- 172.21.15.47:0/127033772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92bc060920 msgr2=0x7f92bc09aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:49.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.590+0000 7f92bb7fe700 1 --2- 172.21.15.47:0/127033772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92bc060920 0x7f92bc09aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:49.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.590+0000 7f92bb7fe700 1 -- 172.21.15.47:0/127033772 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b4009470 con 0x7f92bc14b0c0 2022-06-29T15:51:49.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.590+0000 7f92bb7fe700 1 --2- 172.21.15.47:0/127033772 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92bc14b0c0 0x7f92bc0af190 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 rx=0x7f92b40032f0 tx=0x7f92b4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:49.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.590+0000 7f92b8ff9700 1 -- 172.21.15.47:0/127033772 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92b400b600 con 0x7f92bc14b0c0 2022-06-29T15:51:49.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.590+0000 7f92b8ff9700 1 -- 172.21.15.47:0/127033772 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b4011e20 con 0x7f92bc14b0c0 2022-06-29T15:51:49.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.590+0000 7f92bbfff700 1 -- 172.21.15.47:0/127033772 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92bc15a0d0 con 0x7f92bc14b0c0 2022-06-29T15:51:49.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.590+0000 7f92b8ff9700 1 -- 172.21.15.47:0/127033772 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92b40205f0 con 0x7f92bc14b0c0 2022-06-29T15:51:49.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.590+0000 7f92bbfff700 1 -- 172.21.15.47:0/127033772 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92bc147be0 con 0x7f92bc14b0c0 2022-06-29T15:51:49.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.594+0000 7f92bbfff700 1 -- 172.21.15.47:0/127033772 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92bc15a260 con 0x7f92bc14b0c0 2022-06-29T15:51:49.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.594+0000 7f92b8ff9700 1 -- 172.21.15.47:0/127033772 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f92b4027030 con 0x7f92bc14b0c0 2022-06-29T15:51:49.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.594+0000 7f92b8ff9700 1 --2- 172.21.15.47:0/127033772 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f92a0053a10 0x7f92a0055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:49.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.594+0000 7f92b8ff9700 1 -- 172.21.15.47:0/127033772 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f92b406cce0 con 0x7f92bc14b0c0 2022-06-29T15:51:49.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.594+0000 7f92baffd700 1 --2- 172.21.15.47:0/127033772 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f92a0053a10 0x7f92a0055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:49.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.594+0000 7f92baffd700 1 --2- 172.21.15.47:0/127033772 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f92a0053a10 0x7f92a0055ed0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7f92ac00b000 tx=0x7f92ac006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:49.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.602+0000 7f92b8ff9700 1 -- 172.21.15.47:0/127033772 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f92b4043cd0 con 0x7f92bc14b0c0 2022-06-29T15:51:49.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.722+0000 7f92bbfff700 1 -- 172.21.15.47:0/127033772 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f92bc0573d0 con 0x7f92bc14b0c0 2022-06-29T15:51:49.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.722+0000 7f92b8ff9700 1 -- 172.21.15.47:0/127033772 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v147) v1 ==== 110+0+10 (secure 0 0 0) 0x7f92b40250b0 con 0x7f92bc14b0c0 2022-06-29T15:51:49.730 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2022-06-29T15:51:49.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.726+0000 7f92bbfff700 1 -- 172.21.15.47:0/127033772 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f92a0053a10 msgr2=0x7f92a0055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:49.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.726+0000 7f92bbfff700 1 --2- 172.21.15.47:0/127033772 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f92a0053a10 0x7f92a0055ed0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7f92ac00b000 tx=0x7f92ac006df0).stop 2022-06-29T15:51:49.734 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.726+0000 7f92bbfff700 1 -- 172.21.15.47:0/127033772 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92bc14b0c0 msgr2=0x7f92bc0af190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:49.734 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.726+0000 7f92bbfff700 1 --2- 172.21.15.47:0/127033772 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92bc14b0c0 0x7f92bc0af190 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 rx=0x7f92b40032f0 tx=0x7f92b4016000).stop 2022-06-29T15:51:49.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.726+0000 7f92bbfff700 1 -- 172.21.15.47:0/127033772 shutdown_connections 2022-06-29T15:51:49.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.726+0000 7f92bbfff700 1 --2- 172.21.15.47:0/127033772 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92bc14b0c0 0x7f92bc0af190 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:49.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.726+0000 7f92bbfff700 1 --2- 172.21.15.47:0/127033772 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92bc099d00 0x7f92bc0aeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:49.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.726+0000 7f92bbfff700 1 --2- 172.21.15.47:0/127033772 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f92a0053a10 0x7f92a0055ed0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:49.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.726+0000 7f92bbfff700 1 --2- 172.21.15.47:0/127033772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92bc060920 0x7f92bc09aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:49.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.726+0000 7f92bbfff700 1 -- 172.21.15.47:0/127033772 >> 172.21.15.47:0/127033772 conn(0x7f92bc062df0 msgr2=0x7f92bc0631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:49.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.726+0000 7f92bbfff700 1 -- 172.21.15.47:0/127033772 shutdown_connections 2022-06-29T15:51:49.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.726+0000 7f92bbfff700 1 -- 172.21.15.47:0/127033772 wait complete. 2022-06-29T15:51:49.747 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-29T15:51:49.859 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.850+0000 7fa59898f700 1 Processor -- start 2022-06-29T15:51:49.860 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.854+0000 7fa59898f700 1 -- start start 2022-06-29T15:51:49.862 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.854+0000 7fa59898f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa5940b2bd0 0x7fa5940b2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:49.863 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.854+0000 7fa59898f700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa5941443a0 0x7fa5941447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:49.863 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.854+0000 7fa59898f700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa594144cd0 con 0x7fa59409a2c0 2022-06-29T15:51:49.863 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.854+0000 7fa59898f700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5940b0020 con 0x7fa5940b2bd0 2022-06-29T15:51:49.864 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.854+0000 7fa59898f700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa594144f30 con 0x7fa5941443a0 2022-06-29T15:51:49.864 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.854+0000 7fa592ffd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa59409a2c0 0x7fa594098040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39850/0 (socket says 172.21.15.47:39850) 2022-06-29T15:51:49.864 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.854+0000 7fa592ffd700 1 -- 172.21.15.47:0/2038919293 learned_addr learned my addr 172.21.15.47:0/2038919293 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:49.864 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.854+0000 7fa593fff700 1 --2- 172.21.15.47:0/2038919293 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa5941443a0 0x7fa5941447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:49.864 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.854+0000 7fa5937fe700 1 --2- 172.21.15.47:0/2038919293 >> v2:172.21.15.64:3300/0 conn(0x7fa5940b2bd0 0x7fa5940b2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:49.865 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.854+0000 7fa5927fc700 1 -- 172.21.15.47:0/2038919293 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3622970589 0 0) 0x7fa594144cd0 con 0x7fa59409a2c0 2022-06-29T15:51:49.865 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.854+0000 7fa5927fc700 1 -- 172.21.15.47:0/2038919293 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa580003260 con 0x7fa59409a2c0 2022-06-29T15:51:49.865 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.854+0000 7fa5927fc700 1 -- 172.21.15.47:0/2038919293 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 305998347 0 0) 0x7fa580003260 con 0x7fa59409a2c0 2022-06-29T15:51:49.866 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.854+0000 7fa593fff700 1 -- 172.21.15.47:0/2038919293 >> v2:172.21.15.64:3300/0 conn(0x7fa5940b2bd0 msgr2=0x7fa5940b2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:49.866 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.854+0000 7fa593fff700 1 --2- 172.21.15.47:0/2038919293 >> v2:172.21.15.64:3300/0 conn(0x7fa5940b2bd0 0x7fa5940b2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:49.866 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.854+0000 7fa593fff700 1 -- 172.21.15.47:0/2038919293 >> v1:172.21.15.47:6789/0 conn(0x7fa59409a2c0 legacy=0x7fa594098040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:49.866 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.854+0000 7fa593fff700 1 -- 172.21.15.47:0/2038919293 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5941451d0 con 0x7fa5941443a0 2022-06-29T15:51:49.867 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.854+0000 7fa5937fe700 1 --2- 172.21.15.47:0/2038919293 >> v2:172.21.15.64:3300/0 conn(0x7fa5940b2bd0 0x7fa5940b2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:51:49.867 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.854+0000 7fa593fff700 1 --2- 172.21.15.47:0/2038919293 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa5941443a0 0x7fa5941447e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7fa588001820 tx=0x7fa58c0091d0).ready entity=mon.2 client_cookie=62274f067cced84e server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:49.867 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.854+0000 7fa5927fc700 1 -- 172.21.15.47:0/2038919293 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa588001680 con 0x7fa5941443a0 2022-06-29T15:51:49.867 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.854+0000 7fa5927fc700 1 -- 172.21.15.47:0/2038919293 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa58c00f510 con 0x7fa5941443a0 2022-06-29T15:51:49.867 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.854+0000 7fa5927fc700 1 -- 172.21.15.47:0/2038919293 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa58c00f680 con 0x7fa5941443a0 2022-06-29T15:51:49.871 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.858+0000 7fa59898f700 1 -- 172.21.15.47:0/2038919293 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa5941443a0 msgr2=0x7fa5941447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:49.871 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.858+0000 7fa59898f700 1 --2- 172.21.15.47:0/2038919293 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa5941443a0 0x7fa5941447e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7fa588001820 tx=0x7fa58c0091d0).stop 2022-06-29T15:51:49.872 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.858+0000 7fa59898f700 1 -- 172.21.15.47:0/2038919293 shutdown_connections 2022-06-29T15:51:49.872 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.858+0000 7fa59898f700 1 --2- 172.21.15.47:0/2038919293 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa5941443a0 0x7fa5941447e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:49.872 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.858+0000 7fa59898f700 1 --2- 172.21.15.47:0/2038919293 >> v2:172.21.15.64:3300/0 conn(0x7fa5940b2bd0 0x7fa5940b2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:49.872 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.858+0000 7fa59898f700 1 -- 172.21.15.47:0/2038919293 >> 172.21.15.47:0/2038919293 conn(0x7fa594062df0 msgr2=0x7fa5940631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:49.872 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.858+0000 7fa59898f700 1 -- 172.21.15.47:0/2038919293 shutdown_connections 2022-06-29T15:51:49.873 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.858+0000 7fa59898f700 1 -- 172.21.15.47:0/2038919293 wait complete. 2022-06-29T15:51:49.873 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.858+0000 7fa59898f700 1 Processor -- start 2022-06-29T15:51:49.873 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.858+0000 7fa59898f700 1 -- start start 2022-06-29T15:51:49.874 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.862+0000 7fa59898f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa59409a2c0 0x7fa594140150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:49.874 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.862+0000 7fa59898f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5940b2bd0 0x7fa5940a9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:49.874 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.862+0000 7fa59898f700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa5941443a0 0x7fa59413f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:49.874 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.862+0000 7fa59898f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5940b3820 con 0x7fa5940b2bd0 2022-06-29T15:51:49.875 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.862+0000 7fa59898f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59413fbb0 con 0x7fa59409a2c0 2022-06-29T15:51:49.875 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.862+0000 7fa59898f700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa59413fde0 con 0x7fa5941443a0 2022-06-29T15:51:49.875 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.862+0000 7fa592ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5940b2bd0 0x7fa5940a9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:49.875 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.862+0000 7fa592ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5940b2bd0 0x7fa5940a9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54302/0 (socket says 172.21.15.47:54302) 2022-06-29T15:51:49.875 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.862+0000 7fa592ffd700 1 -- 172.21.15.47:0/3781463445 learned_addr learned my addr 172.21.15.47:0/3781463445 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:49.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.862+0000 7fa593fff700 1 --2- 172.21.15.47:0/3781463445 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa5941443a0 0x7fa59413f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:49.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.862+0000 7fa5937fe700 1 --2- 172.21.15.47:0/3781463445 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa59409a2c0 0x7fa594140150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:49.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.862+0000 7fa592ffd700 1 -- 172.21.15.47:0/3781463445 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa5941443a0 msgr2=0x7fa59413f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:49.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.862+0000 7fa592ffd700 1 --2- 172.21.15.47:0/3781463445 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa5941443a0 0x7fa59413f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:49.877 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.862+0000 7fa592ffd700 1 -- 172.21.15.47:0/3781463445 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa59409a2c0 msgr2=0x7fa594140150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:49.877 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.862+0000 7fa592ffd700 1 --2- 172.21.15.47:0/3781463445 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa59409a2c0 0x7fa594140150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:49.877 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.862+0000 7fa592ffd700 1 -- 172.21.15.47:0/3781463445 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa58400b180 con 0x7fa5940b2bd0 2022-06-29T15:51:49.877 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.862+0000 7fa592ffd700 1 --2- 172.21.15.47:0/3781463445 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5940b2bd0 0x7fa5940a9a00 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 rx=0x7fa5880018e0 tx=0x7fa588001420).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:49.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.862+0000 7fa590ff9700 1 -- 172.21.15.47:0/3781463445 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa588007170 con 0x7fa5940b2bd0 2022-06-29T15:51:49.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.862+0000 7fa590ff9700 1 -- 172.21.15.47:0/3781463445 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5880072d0 con 0x7fa5940b2bd0 2022-06-29T15:51:49.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.862+0000 7fa59898f700 1 -- 172.21.15.47:0/3781463445 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa58c009470 con 0x7fa5940b2bd0 2022-06-29T15:51:49.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.862+0000 7fa590ff9700 1 -- 172.21.15.47:0/3781463445 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa588004920 con 0x7fa5940b2bd0 2022-06-29T15:51:49.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.862+0000 7fa59898f700 1 -- 172.21.15.47:0/3781463445 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5940a9ef0 con 0x7fa5940b2bd0 2022-06-29T15:51:49.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.862+0000 7fa59898f700 1 -- 172.21.15.47:0/3781463445 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5940ad340 con 0x7fa5940b2bd0 2022-06-29T15:51:49.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.862+0000 7fa590ff9700 1 -- 172.21.15.47:0/3781463445 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa588004a80 con 0x7fa5940b2bd0 2022-06-29T15:51:49.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.862+0000 7fa590ff9700 1 --2- 172.21.15.47:0/3781463445 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa574053b00 0x7fa574055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:49.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.866+0000 7fa590ff9700 1 -- 172.21.15.47:0/3781463445 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fa58806bbb0 con 0x7fa5940b2bd0 2022-06-29T15:51:49.880 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.866+0000 7fa5937fe700 1 --2- 172.21.15.47:0/3781463445 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa574053b00 0x7fa574055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:49.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.866+0000 7fa5937fe700 1 --2- 172.21.15.47:0/3781463445 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa574053b00 0x7fa574055fc0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7fa58400b000 tx=0x7fa584008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:49.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.866+0000 7fa590ff9700 1 -- 172.21.15.47:0/3781463445 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa588042ba0 con 0x7fa5940b2bd0 2022-06-29T15:51:50.001 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.994+0000 7fa59898f700 1 -- 172.21.15.47:0/3781463445 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa59413e630 con 0x7fa5940b2bd0 2022-06-29T15:51:50.002 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.994+0000 7fa590ff9700 1 -- 172.21.15.47:0/3781463445 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v147) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa5880461c0 con 0x7fa5940b2bd0 2022-06-29T15:51:50.003 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:51:50.005 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.998+0000 7fa59898f700 1 -- 172.21.15.47:0/3781463445 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa574053b00 msgr2=0x7fa574055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:50.005 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.998+0000 7fa59898f700 1 --2- 172.21.15.47:0/3781463445 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa574053b00 0x7fa574055fc0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7fa58400b000 tx=0x7fa584008040).stop 2022-06-29T15:51:50.005 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.998+0000 7fa59898f700 1 -- 172.21.15.47:0/3781463445 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5940b2bd0 msgr2=0x7fa5940a9a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:50.005 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.998+0000 7fa59898f700 1 --2- 172.21.15.47:0/3781463445 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5940b2bd0 0x7fa5940a9a00 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 rx=0x7fa5880018e0 tx=0x7fa588001420).stop 2022-06-29T15:51:50.006 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.998+0000 7fa59898f700 1 -- 172.21.15.47:0/3781463445 shutdown_connections 2022-06-29T15:51:50.006 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.998+0000 7fa59898f700 1 --2- 172.21.15.47:0/3781463445 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa5941443a0 0x7fa59413f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:50.006 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.998+0000 7fa59898f700 1 --2- 172.21.15.47:0/3781463445 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5940b2bd0 0x7fa5940a9a00 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:50.006 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.998+0000 7fa59898f700 1 --2- 172.21.15.47:0/3781463445 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa574053b00 0x7fa574055fc0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:50.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.998+0000 7fa59898f700 1 --2- 172.21.15.47:0/3781463445 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa59409a2c0 0x7fa594140150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:50.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.998+0000 7fa59898f700 1 -- 172.21.15.47:0/3781463445 >> 172.21.15.47:0/3781463445 conn(0x7fa594062df0 msgr2=0x7fa594143ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:50.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.998+0000 7fa59898f700 1 -- 172.21.15.47:0/3781463445 shutdown_connections 2022-06-29T15:51:50.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:49.998+0000 7fa59898f700 1 -- 172.21.15.47:0/3781463445 wait complete. 2022-06-29T15:51:50.015 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-06-29T15:51:50.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.110+0000 7ffb82693700 1 Processor -- start 2022-06-29T15:51:50.118 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.110+0000 7ffb82693700 1 -- start start 2022-06-29T15:51:50.119 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.110+0000 7ffb82693700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ffb7c143bd0 0x7ffb7c05c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:50.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.110+0000 7ffb82693700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb7c142f90 con 0x7ffb7c0b2c40 2022-06-29T15:51:50.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.110+0000 7ffb82693700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb7c1431f0 con 0x7ffb7c05c510 2022-06-29T15:51:50.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.110+0000 7ffb82693700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb7c143490 con 0x7ffb7c143bd0 2022-06-29T15:51:50.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.110+0000 7ffb81e92700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ffb7c0b2c40 0x7ffb7c13cbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39866/0 (socket says 172.21.15.47:39866) 2022-06-29T15:51:50.121 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.110+0000 7ffb81e92700 1 -- 172.21.15.47:0/3973850515 learned_addr learned my addr 172.21.15.47:0/3973850515 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:50.121 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.110+0000 7ffb80e90700 1 --2- 172.21.15.47:0/3973850515 >> v2:172.21.15.64:3300/0 conn(0x7ffb7c143bd0 0x7ffb7c05c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:50.121 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.114+0000 7ffb6bfff700 1 -- 172.21.15.47:0/3973850515 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1924363025 0 0) 0x7ffb7c142f90 con 0x7ffb7c0b2c40 2022-06-29T15:51:50.121 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.114+0000 7ffb6bfff700 1 -- 172.21.15.47:0/3973850515 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb64003260 con 0x7ffb7c0b2c40 2022-06-29T15:51:50.122 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.114+0000 7ffb80e90700 1 -- 172.21.15.47:0/3973850515 >> v1:172.21.15.64:6789/0 conn(0x7ffb7c05c510 legacy=0x7ffb7c0b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:50.122 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.114+0000 7ffb6bfff700 1 -- 172.21.15.47:0/3973850515 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4130906804 0 0) 0x7ffb64003260 con 0x7ffb7c0b2c40 2022-06-29T15:51:50.122 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.114+0000 7ffb80e90700 1 -- 172.21.15.47:0/3973850515 >> v1:172.21.15.47:6789/0 conn(0x7ffb7c0b2c40 legacy=0x7ffb7c13cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:50.122 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.114+0000 7ffb80e90700 1 -- 172.21.15.47:0/3973850515 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb7c143730 con 0x7ffb7c143bd0 2022-06-29T15:51:50.122 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.114+0000 7ffb80e90700 1 --2- 172.21.15.47:0/3973850515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb7c143bd0 0x7ffb7c05c8f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7ffb74001ba0 tx=0x7ffb6c006df0).ready entity=mon.1 client_cookie=81b649bac66e1001 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:50.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.114+0000 7ffb6bfff700 1 -- 172.21.15.47:0/3973850515 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb74001a00 con 0x7ffb7c143bd0 2022-06-29T15:51:50.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.114+0000 7ffb6bfff700 1 -- 172.21.15.47:0/3973850515 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb6c000e90 con 0x7ffb7c143bd0 2022-06-29T15:51:50.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.114+0000 7ffb6bfff700 1 -- 172.21.15.47:0/3973850515 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb6c011400 con 0x7ffb7c143bd0 2022-06-29T15:51:50.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.114+0000 7ffb82693700 1 -- 172.21.15.47:0/3973850515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb7c143bd0 msgr2=0x7ffb7c05c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:50.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.114+0000 7ffb82693700 1 --2- 172.21.15.47:0/3973850515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb7c143bd0 0x7ffb7c05c8f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7ffb74001ba0 tx=0x7ffb6c006df0).stop 2022-06-29T15:51:50.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.114+0000 7ffb82693700 1 -- 172.21.15.47:0/3973850515 shutdown_connections 2022-06-29T15:51:50.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.114+0000 7ffb82693700 1 --2- 172.21.15.47:0/3973850515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb7c143bd0 0x7ffb7c05c8f0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:50.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.114+0000 7ffb82693700 1 -- 172.21.15.47:0/3973850515 >> 172.21.15.47:0/3973850515 conn(0x7ffb7c060e00 msgr2=0x7ffb7c0611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:50.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.114+0000 7ffb82693700 1 -- 172.21.15.47:0/3973850515 shutdown_connections 2022-06-29T15:51:50.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.114+0000 7ffb82693700 1 -- 172.21.15.47:0/3973850515 wait complete. 2022-06-29T15:51:50.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.118+0000 7ffb82693700 1 Processor -- start 2022-06-29T15:51:50.127 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.118+0000 7ffb82693700 1 -- start start 2022-06-29T15:51:50.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.118+0000 7ffb82693700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb7c05c510 0x7ffb7c0ad670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:50.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.118+0000 7ffb82693700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffb7c0b2c40 0x7ffb7c13e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:50.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.118+0000 7ffb82693700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb7c143bd0 0x7ffb7c0b0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:50.130 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.118+0000 7ffb82693700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7c140820 con 0x7ffb7c05c510 2022-06-29T15:51:50.130 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.118+0000 7ffb82693700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7c146c40 con 0x7ffb7c143bd0 2022-06-29T15:51:50.130 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.118+0000 7ffb82693700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7c1469a0 con 0x7ffb7c0b2c40 2022-06-29T15:51:50.130 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.118+0000 7ffb81691700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb7c05c510 0x7ffb7c0ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:50.131 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.118+0000 7ffb81691700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb7c05c510 0x7ffb7c0ad670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54314/0 (socket says 172.21.15.47:54314) 2022-06-29T15:51:50.131 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.118+0000 7ffb81691700 1 -- 172.21.15.47:0/3022355361 learned_addr learned my addr 172.21.15.47:0/3022355361 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:50.131 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.118+0000 7ffb80e90700 1 --2- 172.21.15.47:0/3022355361 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffb7c0b2c40 0x7ffb7c13e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:50.131 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.118+0000 7ffb81e92700 1 --2- 172.21.15.47:0/3022355361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb7c143bd0 0x7ffb7c0b0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:50.131 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.118+0000 7ffb81691700 1 -- 172.21.15.47:0/3022355361 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffb7c0b2c40 msgr2=0x7ffb7c13e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:50.132 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.118+0000 7ffb81691700 1 --2- 172.21.15.47:0/3022355361 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffb7c0b2c40 0x7ffb7c13e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:50.132 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.118+0000 7ffb81691700 1 -- 172.21.15.47:0/3022355361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb7c143bd0 msgr2=0x7ffb7c0b0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:50.132 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.118+0000 7ffb81691700 1 --2- 172.21.15.47:0/3022355361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb7c143bd0 0x7ffb7c0b0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:50.132 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.118+0000 7ffb81691700 1 -- 172.21.15.47:0/3022355361 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb6c00b180 con 0x7ffb7c05c510 2022-06-29T15:51:50.133 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.118+0000 7ffb81e92700 1 --2- 172.21.15.47:0/3022355361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb7c143bd0 0x7ffb7c0b0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:51:50.133 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.122+0000 7ffb81691700 1 --2- 172.21.15.47:0/3022355361 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb7c05c510 0x7ffb7c0ad670 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 rx=0x7ffb7c145dd0 tx=0x7ffb78007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:50.133 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.122+0000 7ffb6a7fc700 1 -- 172.21.15.47:0/3022355361 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb78007240 con 0x7ffb7c05c510 2022-06-29T15:51:50.133 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.122+0000 7ffb6a7fc700 1 -- 172.21.15.47:0/3022355361 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb780073a0 con 0x7ffb7c05c510 2022-06-29T15:51:50.133 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.122+0000 7ffb82693700 1 -- 172.21.15.47:0/3022355361 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb7c140580 con 0x7ffb7c05c510 2022-06-29T15:51:50.134 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.122+0000 7ffb6a7fc700 1 -- 172.21.15.47:0/3022355361 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb78006700 con 0x7ffb7c05c510 2022-06-29T15:51:50.134 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.122+0000 7ffb82693700 1 -- 172.21.15.47:0/3022355361 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb7c13d6a0 con 0x7ffb7c05c510 2022-06-29T15:51:50.134 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.122+0000 7ffb82693700 1 -- 172.21.15.47:0/3022355361 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb7c146740 con 0x7ffb7c05c510 2022-06-29T15:51:50.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.122+0000 7ffb6a7fc700 1 -- 172.21.15.47:0/3022355361 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ffb78010030 con 0x7ffb7c05c510 2022-06-29T15:51:50.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.122+0000 7ffb6a7fc700 1 --2- 172.21.15.47:0/3022355361 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ffb5c053b30 0x7ffb5c055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:50.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.126+0000 7ffb6a7fc700 1 -- 172.21.15.47:0/3022355361 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5600+0+0 (secure 0 0 0) 0x7ffb7806bb70 con 0x7ffb7c05c510 2022-06-29T15:51:50.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.126+0000 7ffb80e90700 1 --2- 172.21.15.47:0/3022355361 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ffb5c053b30 0x7ffb5c055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:50.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.126+0000 7ffb80e90700 1 --2- 172.21.15.47:0/3022355361 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ffb5c053b30 0x7ffb5c055ff0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7ffb6c017000 tx=0x7ffb7c143380).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:50.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.130+0000 7ffb6a7fc700 1 -- 172.21.15.47:0/3022355361 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ffb78042c70 con 0x7ffb7c05c510 2022-06-29T15:51:50.264 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.258+0000 7ffb82693700 1 -- 172.21.15.47:0/3022355361 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ffb7c061b00 con 0x7ffb7c05c510 2022-06-29T15:51:50.265 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.258+0000 7ffb6a7fc700 1 -- 172.21.15.47:0/3022355361 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v147) v1 ==== 84+0+143 (secure 0 0 0) 0x7ffb78046290 con 0x7ffb7c05c510 2022-06-29T15:51:50.265 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:50.266 INFO:teuthology.orchestra.run.smithi047.stdout:[ 2022-06-29T15:51:50.266 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:50.266 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 1, 2022-06-29T15:51:50.266 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": "device_health_metrics" 2022-06-29T15:51:50.267 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:51:50.267 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:51:50.267 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 2, 2022-06-29T15:51:50.267 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": "rbd" 2022-06-29T15:51:50.268 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:51:50.268 INFO:teuthology.orchestra.run.smithi047.stdout:] 2022-06-29T15:51:50.269 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.258+0000 7ffb82693700 1 -- 172.21.15.47:0/3022355361 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ffb5c053b30 msgr2=0x7ffb5c055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:50.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.258+0000 7ffb82693700 1 --2- 172.21.15.47:0/3022355361 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ffb5c053b30 0x7ffb5c055ff0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7ffb6c017000 tx=0x7ffb7c143380).stop 2022-06-29T15:51:50.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.258+0000 7ffb82693700 1 -- 172.21.15.47:0/3022355361 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb7c05c510 msgr2=0x7ffb7c0ad670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:50.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.258+0000 7ffb82693700 1 --2- 172.21.15.47:0/3022355361 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb7c05c510 0x7ffb7c0ad670 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 rx=0x7ffb7c145dd0 tx=0x7ffb78007c00).stop 2022-06-29T15:51:50.270 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.262+0000 7ffb82693700 1 -- 172.21.15.47:0/3022355361 shutdown_connections 2022-06-29T15:51:50.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.262+0000 7ffb82693700 1 --2- 172.21.15.47:0/3022355361 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ffb5c053b30 0x7ffb5c055ff0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:50.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.262+0000 7ffb82693700 1 --2- 172.21.15.47:0/3022355361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb7c143bd0 0x7ffb7c0b0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:50.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.262+0000 7ffb82693700 1 --2- 172.21.15.47:0/3022355361 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffb7c0b2c40 0x7ffb7c13e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:50.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.262+0000 7ffb82693700 1 --2- 172.21.15.47:0/3022355361 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb7c05c510 0x7ffb7c0ad670 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:50.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.262+0000 7ffb82693700 1 -- 172.21.15.47:0/3022355361 >> 172.21.15.47:0/3022355361 conn(0x7ffb7c060e00 msgr2=0x7ffb7c142830 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:50.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.262+0000 7ffb82693700 1 -- 172.21.15.47:0/3022355361 shutdown_connections 2022-06-29T15:51:50.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.262+0000 7ffb82693700 1 -- 172.21.15.47:0/3022355361 wait complete. 2022-06-29T15:51:50.278 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-06-29T15:51:50.279 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-06-29T15:51:50.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.382+0000 7f8822192700 1 Processor -- start 2022-06-29T15:51:50.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.382+0000 7f8822192700 1 -- start start 2022-06-29T15:51:50.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.382+0000 7f8822192700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f881c0b6d20 0x7f881c0a9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:50.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.382+0000 7f8822192700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f881c063100 0x7f881c13e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:50.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.382+0000 7f8822192700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f881c0b7100 con 0x7f881c05f350 2022-06-29T15:51:50.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.382+0000 7f8822192700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f881c05fdc0 con 0x7f881c063100 2022-06-29T15:51:50.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.382+0000 7f8822192700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f881c141db0 con 0x7f881c0b6d20 2022-06-29T15:51:50.394 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.382+0000 7f8821991700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f881c063100 0x7f881c13e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:50.394 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.382+0000 7f8821190700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f881c0b6d20 0x7f881c0a9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:50.394 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.382+0000 7f8821991700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f881c063100 0x7f881c13e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54326/0 (socket says 172.21.15.47:54326) 2022-06-29T15:51:50.394 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.382+0000 7f8821991700 1 -- 172.21.15.47:0/1675674974 learned_addr learned my addr 172.21.15.47:0/1675674974 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:50.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.382+0000 7f8821991700 1 -- 172.21.15.47:0/1675674974 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f881c0b6d20 msgr2=0x7f881c0a9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:50.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.382+0000 7f8821991700 1 --2- 172.21.15.47:0/1675674974 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f881c0b6d20 0x7f881c0a9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:50.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.382+0000 7f881bfff700 1 -- 172.21.15.47:0/1675674974 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 80080168 0 0) 0x7f881c0b7100 con 0x7f881c05f350 2022-06-29T15:51:50.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.382+0000 7f8821991700 1 -- 172.21.15.47:0/1675674974 >> v1:172.21.15.64:6789/0 conn(0x7f881c05f350 legacy=0x7f881c0612b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:50.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.382+0000 7f8821991700 1 -- 172.21.15.47:0/1675674974 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f881c0619c0 con 0x7f881c063100 2022-06-29T15:51:50.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.382+0000 7f8821190700 1 --2- 172.21.15.47:0/1675674974 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f881c0b6d20 0x7f881c0a9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:51:50.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.382+0000 7f8821991700 1 --2- 172.21.15.47:0/1675674974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f881c063100 0x7f881c13e2b0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 rx=0x7f8814002f70 tx=0x7f8814006eb0).ready entity=mon.0 client_cookie=2edfe3709e1b0366 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:50.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.382+0000 7f881bfff700 1 -- 172.21.15.47:0/1675674974 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f881400d570 con 0x7f881c063100 2022-06-29T15:51:50.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.382+0000 7f881bfff700 1 -- 172.21.15.47:0/1675674974 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8814000e90 con 0x7f881c063100 2022-06-29T15:51:50.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.382+0000 7f881bfff700 1 -- 172.21.15.47:0/1675674974 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f881400d8c0 con 0x7f881c063100 2022-06-29T15:51:50.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.386+0000 7f8822192700 1 -- 172.21.15.47:0/1675674974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f881c063100 msgr2=0x7f881c13e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:50.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.386+0000 7f8822192700 1 --2- 172.21.15.47:0/1675674974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f881c063100 0x7f881c13e2b0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 rx=0x7f8814002f70 tx=0x7f8814006eb0).stop 2022-06-29T15:51:50.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.386+0000 7f8822192700 1 -- 172.21.15.47:0/1675674974 shutdown_connections 2022-06-29T15:51:50.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.386+0000 7f8822192700 1 --2- 172.21.15.47:0/1675674974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f881c063100 0x7f881c13e2b0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:50.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.386+0000 7f8822192700 1 --2- 172.21.15.47:0/1675674974 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f881c0b6d20 0x7f881c0a9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:50.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.386+0000 7f8822192700 1 -- 172.21.15.47:0/1675674974 >> 172.21.15.47:0/1675674974 conn(0x7f881c063670 msgr2=0x7f881c063a50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:50.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.386+0000 7f8822192700 1 -- 172.21.15.47:0/1675674974 shutdown_connections 2022-06-29T15:51:50.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.386+0000 7f8822192700 1 -- 172.21.15.47:0/1675674974 wait complete. 2022-06-29T15:51:50.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.386+0000 7f8822192700 1 Processor -- start 2022-06-29T15:51:50.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.386+0000 7f8822192700 1 -- start start 2022-06-29T15:51:50.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.386+0000 7f8822192700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f881c05f350 0x7f881c13cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:50.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.386+0000 7f8822192700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f881c063100 0x7f881c150110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:50.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.386+0000 7f8822192700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f881c0b6d20 0x7f881c150600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:50.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.386+0000 7f8822192700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f881c150af0 con 0x7f881c0b6d20 2022-06-29T15:51:50.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.386+0000 7f8822192700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f881c153780 con 0x7f881c063100 2022-06-29T15:51:50.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.386+0000 7f8822192700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f881c153a20 con 0x7f881c05f350 2022-06-29T15:51:50.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.386+0000 7f8821991700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f881c0b6d20 0x7f881c150600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:50.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.386+0000 7f8821991700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f881c0b6d20 0x7f881c150600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54332/0 (socket says 172.21.15.47:54332) 2022-06-29T15:51:50.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.386+0000 7f8821991700 1 -- 172.21.15.47:0/144677487 learned_addr learned my addr 172.21.15.47:0/144677487 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:50.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.386+0000 7f8821190700 1 --2- 172.21.15.47:0/144677487 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f881c05f350 0x7f881c13cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:50.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.386+0000 7f882098f700 1 --2- 172.21.15.47:0/144677487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f881c063100 0x7f881c150110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:50.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.386+0000 7f8821991700 1 -- 172.21.15.47:0/144677487 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f881c05f350 msgr2=0x7f881c13cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:50.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.386+0000 7f8821991700 1 --2- 172.21.15.47:0/144677487 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f881c05f350 0x7f881c13cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:50.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.386+0000 7f8821991700 1 -- 172.21.15.47:0/144677487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f881c063100 msgr2=0x7f881c150110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:50.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.386+0000 7f8821991700 1 --2- 172.21.15.47:0/144677487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f881c063100 0x7f881c150110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:50.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.386+0000 7f8821991700 1 -- 172.21.15.47:0/144677487 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8804009470 con 0x7f881c0b6d20 2022-06-29T15:51:50.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.386+0000 7f8821991700 1 --2- 172.21.15.47:0/144677487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f881c0b6d20 0x7f881c150600 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 rx=0x7f8814002f70 tx=0x7f8814015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:50.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.390+0000 7f881a7fc700 1 -- 172.21.15.47:0/144677487 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8814006b40 con 0x7f881c0b6d20 2022-06-29T15:51:50.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.390+0000 7f881a7fc700 1 -- 172.21.15.47:0/144677487 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88140068b0 con 0x7f881c0b6d20 2022-06-29T15:51:50.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.390+0000 7f8822192700 1 -- 172.21.15.47:0/144677487 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f881400b1e0 con 0x7f881c0b6d20 2022-06-29T15:51:50.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.390+0000 7f881a7fc700 1 -- 172.21.15.47:0/144677487 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8814011bf0 con 0x7f881c0b6d20 2022-06-29T15:51:50.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.390+0000 7f8822192700 1 -- 172.21.15.47:0/144677487 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f881c153f60 con 0x7f881c0b6d20 2022-06-29T15:51:50.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.390+0000 7f8822192700 1 -- 172.21.15.47:0/144677487 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f881c1540f0 con 0x7f881c0b6d20 2022-06-29T15:51:50.409 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.390+0000 7f881a7fc700 1 -- 172.21.15.47:0/144677487 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8814025030 con 0x7f881c0b6d20 2022-06-29T15:51:50.409 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.390+0000 7f881a7fc700 1 --2- 172.21.15.47:0/144677487 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f88000539a0 0x7f8800055e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:50.409 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.394+0000 7f8821190700 1 --2- 172.21.15.47:0/144677487 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f88000539a0 0x7f8800055e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:50.410 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.394+0000 7f881a7fc700 1 -- 172.21.15.47:0/144677487 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f881406d340 con 0x7f881c0b6d20 2022-06-29T15:51:50.410 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.394+0000 7f8821190700 1 --2- 172.21.15.47:0/144677487 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f88000539a0 0x7f8800055e60 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7f880400b000 tx=0x7f88040091d0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:50.410 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.394+0000 7f881a7fc700 1 -- 172.21.15.47:0/144677487 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8814044330 con 0x7f881c0b6d20 2022-06-29T15:51:50.531 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:50.522+0000 7f8822192700 1 -- 172.21.15.47:0/144677487 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f881c1545a0 con 0x7f881c0b6d20 2022-06-29T15:51:51.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.506+0000 7f881a7fc700 1 -- 172.21.15.47:0/144677487 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v148) v1 ==== 133+0+0 (secure 0 0 0) 0x7f88140230a0 con 0x7f881c0b6d20 2022-06-29T15:51:51.516 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.510+0000 7f8822192700 1 -- 172.21.15.47:0/144677487 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f88000539a0 msgr2=0x7f8800055e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:51.517 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.510+0000 7f8822192700 1 --2- 172.21.15.47:0/144677487 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f88000539a0 0x7f8800055e60 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7f880400b000 tx=0x7f88040091d0).stop 2022-06-29T15:51:51.517 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.510+0000 7f8822192700 1 -- 172.21.15.47:0/144677487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f881c0b6d20 msgr2=0x7f881c150600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:51.518 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.510+0000 7f8822192700 1 --2- 172.21.15.47:0/144677487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f881c0b6d20 0x7f881c150600 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 rx=0x7f8814002f70 tx=0x7f8814015000).stop 2022-06-29T15:51:51.518 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.510+0000 7f8822192700 1 -- 172.21.15.47:0/144677487 shutdown_connections 2022-06-29T15:51:51.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.510+0000 7f8822192700 1 --2- 172.21.15.47:0/144677487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f881c0b6d20 0x7f881c150600 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:51.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.510+0000 7f8822192700 1 --2- 172.21.15.47:0/144677487 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f88000539a0 0x7f8800055e60 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:51.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.510+0000 7f8822192700 1 --2- 172.21.15.47:0/144677487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f881c063100 0x7f881c150110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:51.520 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.510+0000 7f8822192700 1 --2- 172.21.15.47:0/144677487 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f881c05f350 0x7f881c13cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:51.520 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.510+0000 7f8822192700 1 -- 172.21.15.47:0/144677487 >> 172.21.15.47:0/144677487 conn(0x7f881c063670 msgr2=0x7f881c1400f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:51.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.510+0000 7f8822192700 1 -- 172.21.15.47:0/144677487 shutdown_connections 2022-06-29T15:51:51.522 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.510+0000 7f8822192700 1 -- 172.21.15.47:0/144677487 wait complete. 2022-06-29T15:51:51.522 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' created 2022-06-29T15:51:51.532 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-06-29T15:51:51.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.634+0000 7fb80bfff700 1 Processor -- start 2022-06-29T15:51:51.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.634+0000 7fb80bfff700 1 -- start start 2022-06-29T15:51:51.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.638+0000 7fb80bfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb80c144670 0x7fb80c061740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:51.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.638+0000 7fb80bfff700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb80c061e50 con 0x7fb80c05f490 2022-06-29T15:51:51.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.638+0000 7fb80bfff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb80c0620f0 con 0x7fb80c05f0b0 2022-06-29T15:51:51.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.638+0000 7fb80bfff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb80c061c30 con 0x7fb80c144670 2022-06-29T15:51:51.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.638+0000 7fb80a7fc700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb80c05f490 0x7fb80c140410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39892/0 (socket says 172.21.15.47:39892) 2022-06-29T15:51:51.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.638+0000 7fb80a7fc700 1 -- 172.21.15.47:0/2264978545 learned_addr learned my addr 172.21.15.47:0/2264978545 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:51.651 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.638+0000 7fb80b7fe700 1 --2- 172.21.15.47:0/2264978545 >> v2:172.21.15.64:3300/0 conn(0x7fb80c144670 0x7fb80c061740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:51.651 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.638+0000 7fb809ffb700 1 -- 172.21.15.47:0/2264978545 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2961772742 0 0) 0x7fb80c061e50 con 0x7fb80c05f490 2022-06-29T15:51:51.651 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.638+0000 7fb809ffb700 1 -- 172.21.15.47:0/2264978545 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7ec003260 con 0x7fb80c05f490 2022-06-29T15:51:51.652 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.638+0000 7fb809ffb700 1 -- 172.21.15.47:0/2264978545 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1799959890 0 0) 0x7fb7ec003260 con 0x7fb80c05f490 2022-06-29T15:51:51.652 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.638+0000 7fb809ffb700 1 -- 172.21.15.47:0/2264978545 >> v2:172.21.15.64:3300/0 conn(0x7fb80c144670 msgr2=0x7fb80c061740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:51.653 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.638+0000 7fb809ffb700 1 --2- 172.21.15.47:0/2264978545 >> v2:172.21.15.64:3300/0 conn(0x7fb80c144670 0x7fb80c061740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:51.653 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.638+0000 7fb809ffb700 1 -- 172.21.15.47:0/2264978545 >> v1:172.21.15.64:6789/0 conn(0x7fb80c05f0b0 legacy=0x7fb80c143950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:51.653 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.638+0000 7fb809ffb700 1 -- 172.21.15.47:0/2264978545 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb80c1434b0 con 0x7fb80c05f490 2022-06-29T15:51:51.654 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.638+0000 7fb809ffb700 1 -- 172.21.15.47:0/2264978545 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7fb7fc001660 con 0x7fb80c05f490 2022-06-29T15:51:51.654 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.638+0000 7fb809ffb700 1 -- 172.21.15.47:0/2264978545 >> v1:172.21.15.47:6789/0 conn(0x7fb80c05f490 legacy=0x7fb80c140410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:51.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.638+0000 7fb809ffb700 1 --2- 172.21.15.47:0/2264978545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7ec003750 0x7fb7ec003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:51.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.638+0000 7fb809ffb700 1 --2- 172.21.15.47:0/2264978545 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb7ec004500 0x7fb7ec004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:51.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.638+0000 7fb809ffb700 1 --2- 172.21.15.47:0/2264978545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7ec004e70 0x7fb7ec00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:51.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.638+0000 7fb809ffb700 1 -- 172.21.15.47:0/2264978545 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80c061c30 con 0x7fb7ec004e70 2022-06-29T15:51:51.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.638+0000 7fb809ffb700 1 -- 172.21.15.47:0/2264978545 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ec00bd20 con 0x7fb7ec003750 2022-06-29T15:51:51.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.638+0000 7fb809ffb700 1 -- 172.21.15.47:0/2264978545 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ec00bee0 con 0x7fb7ec004500 2022-06-29T15:51:51.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.638+0000 7fb80b7fe700 1 --2- 172.21.15.47:0/2264978545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7ec004e70 0x7fb7ec00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:51.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.638+0000 7fb80a7fc700 1 --2- 172.21.15.47:0/2264978545 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb7ec004500 0x7fb7ec004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:51.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.638+0000 7fb80affd700 1 --2- 172.21.15.47:0/2264978545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7ec003750 0x7fb7ec003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:51.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.638+0000 7fb80b7fe700 1 -- 172.21.15.47:0/2264978545 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb7ec004500 msgr2=0x7fb7ec004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:51.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.638+0000 7fb80b7fe700 1 --2- 172.21.15.47:0/2264978545 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb7ec004500 0x7fb7ec004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:51.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.638+0000 7fb80b7fe700 1 -- 172.21.15.47:0/2264978545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7ec003750 msgr2=0x7fb7ec003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:51.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.638+0000 7fb80b7fe700 1 --2- 172.21.15.47:0/2264978545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7ec003750 0x7fb7ec003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:51.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.638+0000 7fb80b7fe700 1 -- 172.21.15.47:0/2264978545 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7ec003260 con 0x7fb7ec004e70 2022-06-29T15:51:51.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.638+0000 7fb80b7fe700 1 -- 172.21.15.47:0/2264978545 reap_dead start 2022-06-29T15:51:51.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.638+0000 7fb80b7fe700 1 --2- 172.21.15.47:0/2264978545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7ec004e70 0x7fb7ec00b750 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 rx=0x7fb80400c000 tx=0x7fb8040091d0).ready entity=mon.0 client_cookie=7566c147049cf431 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:51.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.638+0000 7fb809ffb700 1 -- 172.21.15.47:0/2264978545 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb80400e7e0 con 0x7fb7ec004e70 2022-06-29T15:51:51.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.638+0000 7fb809ffb700 1 -- 172.21.15.47:0/2264978545 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb80400e940 con 0x7fb7ec004e70 2022-06-29T15:51:51.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.638+0000 7fb809ffb700 1 -- 172.21.15.47:0/2264978545 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb80400eca0 con 0x7fb7ec004e70 2022-06-29T15:51:51.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.638+0000 7fb80bfff700 1 -- 172.21.15.47:0/2264978545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7ec004e70 msgr2=0x7fb7ec00b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:51.663 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.638+0000 7fb80bfff700 1 --2- 172.21.15.47:0/2264978545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7ec004e70 0x7fb7ec00b750 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 rx=0x7fb80400c000 tx=0x7fb8040091d0).stop 2022-06-29T15:51:51.663 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.638+0000 7fb80bfff700 1 -- 172.21.15.47:0/2264978545 shutdown_connections 2022-06-29T15:51:51.663 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.638+0000 7fb80bfff700 1 --2- 172.21.15.47:0/2264978545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7ec004e70 0x7fb7ec00b750 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:51.664 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.638+0000 7fb80bfff700 1 -- 172.21.15.47:0/2264978545 >> 172.21.15.47:0/2264978545 conn(0x7fb80c062570 msgr2=0x7fb80c0647d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:51.664 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.638+0000 7fb80bfff700 1 -- 172.21.15.47:0/2264978545 shutdown_connections 2022-06-29T15:51:51.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.638+0000 7fb80bfff700 1 -- 172.21.15.47:0/2264978545 wait complete. 2022-06-29T15:51:51.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.642+0000 7fb80bfff700 1 Processor -- start 2022-06-29T15:51:51.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.642+0000 7fb80bfff700 1 -- start start 2022-06-29T15:51:51.669 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.642+0000 7fb80bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7ec004e70 0x7fb80c0b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:51.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.642+0000 7fb80bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb80c0b5df0 0x7fb80c0b74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:51.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.642+0000 7fb80bfff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb80c0b79e0 0x7fb80c13b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:51.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.642+0000 7fb80bfff700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80c0b35c0 con 0x7fb80c0b5df0 2022-06-29T15:51:51.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.642+0000 7fb80bfff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80c0b2fd0 con 0x7fb7ec004e70 2022-06-29T15:51:51.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.642+0000 7fb80bfff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb80c13ee50 con 0x7fb80c0b79e0 2022-06-29T15:51:51.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.642+0000 7fb80a7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb80c0b5df0 0x7fb80c0b74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:51.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.642+0000 7fb80a7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb80c0b5df0 0x7fb80c0b74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54350/0 (socket says 172.21.15.47:54350) 2022-06-29T15:51:51.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.642+0000 7fb80a7fc700 1 -- 172.21.15.47:0/2083757742 learned_addr learned my addr 172.21.15.47:0/2083757742 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:51.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.642+0000 7fb80affd700 1 --2- 172.21.15.47:0/2083757742 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7ec004e70 0x7fb80c0b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:51.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.642+0000 7fb80b7fe700 1 --2- 172.21.15.47:0/2083757742 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb80c0b79e0 0x7fb80c13b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:51.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.642+0000 7fb80a7fc700 1 -- 172.21.15.47:0/2083757742 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb80c0b79e0 msgr2=0x7fb80c13b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:51.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.642+0000 7fb80a7fc700 1 --2- 172.21.15.47:0/2083757742 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb80c0b79e0 0x7fb80c13b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:51.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.642+0000 7fb80a7fc700 1 -- 172.21.15.47:0/2083757742 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7ec004e70 msgr2=0x7fb80c0b3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:51.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.642+0000 7fb80a7fc700 1 --2- 172.21.15.47:0/2083757742 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7ec004e70 0x7fb80c0b3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:51.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.642+0000 7fb80a7fc700 1 -- 172.21.15.47:0/2083757742 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb80c0b2bf0 con 0x7fb80c0b5df0 2022-06-29T15:51:51.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.642+0000 7fb80affd700 1 --2- 172.21.15.47:0/2083757742 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7ec004e70 0x7fb80c0b3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:51:51.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.646+0000 7fb80a7fc700 1 --2- 172.21.15.47:0/2083757742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb80c0b5df0 0x7fb80c0b74f0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 rx=0x7fb7fc00d000 tx=0x7fb80c062280).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:51.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.646+0000 7fb803fff700 1 -- 172.21.15.47:0/2083757742 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb7fc011780 con 0x7fb80c0b5df0 2022-06-29T15:51:51.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.646+0000 7fb803fff700 1 -- 172.21.15.47:0/2083757742 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7fc0118e0 con 0x7fb80c0b5df0 2022-06-29T15:51:51.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.646+0000 7fb80bfff700 1 -- 172.21.15.47:0/2083757742 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb80c0b3200 con 0x7fb80c0b5df0 2022-06-29T15:51:51.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.646+0000 7fb803fff700 1 -- 172.21.15.47:0/2083757742 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb7fc011c40 con 0x7fb80c0b5df0 2022-06-29T15:51:51.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.646+0000 7fb80bfff700 1 -- 172.21.15.47:0/2083757742 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb80c0b26c0 con 0x7fb80c0b5df0 2022-06-29T15:51:51.676 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.646+0000 7fb80bfff700 1 -- 172.21.15.47:0/2083757742 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb80c0b2d80 con 0x7fb80c0b5df0 2022-06-29T15:51:51.676 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.646+0000 7fb803fff700 1 -- 172.21.15.47:0/2083757742 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb7fc01b030 con 0x7fb80c0b5df0 2022-06-29T15:51:51.676 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.650+0000 7fb803fff700 1 --2- 172.21.15.47:0/2083757742 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb7f0053b00 0x7fb7f0055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:51.677 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.650+0000 7fb803fff700 1 -- 172.21.15.47:0/2083757742 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5954+0+0 (secure 0 0 0) 0x7fb7fc06b850 con 0x7fb80c0b5df0 2022-06-29T15:51:51.677 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.650+0000 7fb80affd700 1 --2- 172.21.15.47:0/2083757742 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb7f0053b00 0x7fb7f0055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:51.677 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.650+0000 7fb80affd700 1 --2- 172.21.15.47:0/2083757742 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb7f0053b00 0x7fb7f0055fc0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7fb7f400d000 tx=0x7fb7f4009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:51.678 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.654+0000 7fb803fff700 1 -- 172.21.15.47:0/2083757742 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb7fc0427e0 con 0x7fb80c0b5df0 2022-06-29T15:51:51.783 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:51.774+0000 7fb80bfff700 1 -- 172.21.15.47:0/2083757742 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7fb80c13efe0 con 0x7fb80c0b5df0 2022-06-29T15:51:52.518 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.510+0000 7fb803fff700 1 -- 172.21.15.47:0/2083757742 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v149) v1 ==== 125+0+0 (secure 0 0 0) 0x7fb7fc019030 con 0x7fb80c0b5df0 2022-06-29T15:51:52.522 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.514+0000 7fb80bfff700 1 -- 172.21.15.47:0/2083757742 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb7f0053b00 msgr2=0x7fb7f0055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:52.522 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.514+0000 7fb80bfff700 1 --2- 172.21.15.47:0/2083757742 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb7f0053b00 0x7fb7f0055fc0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7fb7f400d000 tx=0x7fb7f4009040).stop 2022-06-29T15:51:52.522 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.514+0000 7fb80bfff700 1 -- 172.21.15.47:0/2083757742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb80c0b5df0 msgr2=0x7fb80c0b74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:52.523 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.514+0000 7fb80bfff700 1 --2- 172.21.15.47:0/2083757742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb80c0b5df0 0x7fb80c0b74f0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 rx=0x7fb7fc00d000 tx=0x7fb80c062280).stop 2022-06-29T15:51:52.523 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.514+0000 7fb80bfff700 1 -- 172.21.15.47:0/2083757742 shutdown_connections 2022-06-29T15:51:52.524 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.514+0000 7fb80bfff700 1 --2- 172.21.15.47:0/2083757742 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb80c0b79e0 0x7fb80c13b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:52.524 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.514+0000 7fb80bfff700 1 --2- 172.21.15.47:0/2083757742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb80c0b5df0 0x7fb80c0b74f0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:52.525 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.514+0000 7fb80bfff700 1 --2- 172.21.15.47:0/2083757742 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb7f0053b00 0x7fb7f0055fc0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:52.525 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.514+0000 7fb80bfff700 1 --2- 172.21.15.47:0/2083757742 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7ec004e70 0x7fb80c0b3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:52.525 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.514+0000 7fb80bfff700 1 -- 172.21.15.47:0/2083757742 >> 172.21.15.47:0/2083757742 conn(0x7fb80c062570 msgr2=0x7fb80c1478b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:52.525 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.518+0000 7fb80bfff700 1 -- 172.21.15.47:0/2083757742 shutdown_connections 2022-06-29T15:51:52.526 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.518+0000 7fb80bfff700 1 -- 172.21.15.47:0/2083757742 wait complete. 2022-06-29T15:51:52.526 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' created 2022-06-29T15:51:52.538 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-06-29T15:51:52.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.638+0000 7f4094f3e700 1 Processor -- start 2022-06-29T15:51:52.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.638+0000 7f4094f3e700 1 -- start start 2022-06-29T15:51:52.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.638+0000 7f4094f3e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f409005f490 0x7f4090144670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:52.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.638+0000 7f4094f3e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4090061e50 con 0x7f409005f0b0 2022-06-29T15:51:52.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.638+0000 7f4094f3e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40900620f0 con 0x7f4090061740 2022-06-29T15:51:52.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.638+0000 7f4094f3e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40901434b0 con 0x7f409005f490 2022-06-29T15:51:52.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.638+0000 7f408f7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f409005f0b0 0x7f4090143950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39910/0 (socket says 172.21.15.47:39910) 2022-06-29T15:51:52.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.638+0000 7f408f7fe700 1 -- 172.21.15.47:0/2703008497 learned_addr learned my addr 172.21.15.47:0/2703008497 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:52.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.638+0000 7f408effd700 1 --2- 172.21.15.47:0/2703008497 >> v2:172.21.15.64:3300/0 conn(0x7f409005f490 0x7f4090144670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:52.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.638+0000 7f408e7fc700 1 -- 172.21.15.47:0/2703008497 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3217915647 0 0) 0x7f4090061e50 con 0x7f409005f0b0 2022-06-29T15:51:52.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.642+0000 7f408e7fc700 1 -- 172.21.15.47:0/2703008497 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f407c003260 con 0x7f409005f0b0 2022-06-29T15:51:52.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.642+0000 7f408e7fc700 1 -- 172.21.15.47:0/2703008497 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 545812103 0 0) 0x7f40900620f0 con 0x7f4090061740 2022-06-29T15:51:52.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.642+0000 7f408e7fc700 1 -- 172.21.15.47:0/2703008497 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4090061e50 con 0x7f4090061740 2022-06-29T15:51:52.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.642+0000 7f408e7fc700 1 -- 172.21.15.47:0/2703008497 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2361788260 0 0) 0x7f407c003260 con 0x7f409005f0b0 2022-06-29T15:51:52.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.642+0000 7f408e7fc700 1 -- 172.21.15.47:0/2703008497 >> v2:172.21.15.64:3300/0 conn(0x7f409005f490 msgr2=0x7f4090144670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:52.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.642+0000 7f408e7fc700 1 --2- 172.21.15.47:0/2703008497 >> v2:172.21.15.64:3300/0 conn(0x7f409005f490 0x7f4090144670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:52.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.642+0000 7f408e7fc700 1 -- 172.21.15.47:0/2703008497 >> v1:172.21.15.64:6789/0 conn(0x7f4090061740 legacy=0x7f4090140410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:52.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.642+0000 7f408e7fc700 1 -- 172.21.15.47:0/2703008497 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4090144060 con 0x7f409005f0b0 2022-06-29T15:51:52.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.642+0000 7f408e7fc700 1 -- 172.21.15.47:0/2703008497 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7f4080001660 con 0x7f409005f0b0 2022-06-29T15:51:52.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.642+0000 7f408e7fc700 1 -- 172.21.15.47:0/2703008497 >> v1:172.21.15.47:6789/0 conn(0x7f409005f0b0 legacy=0x7f4090143950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:52.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.642+0000 7f408e7fc700 1 --2- 172.21.15.47:0/2703008497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f407c003930 0x7f407c003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:52.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.642+0000 7f408e7fc700 1 --2- 172.21.15.47:0/2703008497 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f407c004500 0x7f407c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:52.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.642+0000 7f408e7fc700 1 --2- 172.21.15.47:0/2703008497 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f407c004e70 0x7f407c00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:52.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.642+0000 7f408e7fc700 1 -- 172.21.15.47:0/2703008497 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40901434b0 con 0x7f407c004e70 2022-06-29T15:51:52.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.642+0000 7f408e7fc700 1 -- 172.21.15.47:0/2703008497 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f407c00bd20 con 0x7f407c003930 2022-06-29T15:51:52.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.642+0000 7f408e7fc700 1 -- 172.21.15.47:0/2703008497 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f407c00bee0 con 0x7f407c004500 2022-06-29T15:51:52.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.642+0000 7f408ffff700 1 --2- 172.21.15.47:0/2703008497 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f407c004e70 0x7f407c00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:52.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.642+0000 7f408f7fe700 1 --2- 172.21.15.47:0/2703008497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f407c003930 0x7f407c003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:52.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.642+0000 7f408effd700 1 --2- 172.21.15.47:0/2703008497 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f407c004500 0x7f407c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:52.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.642+0000 7f408ffff700 1 -- 172.21.15.47:0/2703008497 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f407c004500 msgr2=0x7f407c004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:52.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.642+0000 7f408ffff700 1 --2- 172.21.15.47:0/2703008497 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f407c004500 0x7f407c004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:52.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.642+0000 7f408ffff700 1 -- 172.21.15.47:0/2703008497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f407c003930 msgr2=0x7f407c003d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:52.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.642+0000 7f408ffff700 1 --2- 172.21.15.47:0/2703008497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f407c003930 0x7f407c003d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:52.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.642+0000 7f408ffff700 1 -- 172.21.15.47:0/2703008497 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f407c003260 con 0x7f407c004e70 2022-06-29T15:51:52.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.642+0000 7f408ffff700 1 -- 172.21.15.47:0/2703008497 reap_dead start 2022-06-29T15:51:52.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.642+0000 7f408ffff700 1 --2- 172.21.15.47:0/2703008497 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f407c004e70 0x7f407c00b750 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 rx=0x7f4088001490 tx=0x7f4088007d80).ready entity=mon.0 client_cookie=5f7ac31a6b545db2 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:52.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.642+0000 7f408e7fc700 1 -- 172.21.15.47:0/2703008497 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4088001780 con 0x7f407c004e70 2022-06-29T15:51:52.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.642+0000 7f408e7fc700 1 -- 172.21.15.47:0/2703008497 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4088006890 con 0x7f407c004e70 2022-06-29T15:51:52.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.642+0000 7f408e7fc700 1 -- 172.21.15.47:0/2703008497 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4088006bf0 con 0x7f407c004e70 2022-06-29T15:51:52.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.642+0000 7f4094f3e700 1 -- 172.21.15.47:0/2703008497 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f407c004e70 msgr2=0x7f407c00b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:52.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.642+0000 7f4094f3e700 1 --2- 172.21.15.47:0/2703008497 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f407c004e70 0x7f407c00b750 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 rx=0x7f4088001490 tx=0x7f4088007d80).stop 2022-06-29T15:51:52.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.642+0000 7f4094f3e700 1 -- 172.21.15.47:0/2703008497 shutdown_connections 2022-06-29T15:51:52.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.642+0000 7f4094f3e700 1 --2- 172.21.15.47:0/2703008497 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f407c004e70 0x7f407c00b750 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:52.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.642+0000 7f4094f3e700 1 -- 172.21.15.47:0/2703008497 >> 172.21.15.47:0/2703008497 conn(0x7f4090062570 msgr2=0x7f40900647d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:52.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.642+0000 7f4094f3e700 1 -- 172.21.15.47:0/2703008497 shutdown_connections 2022-06-29T15:51:52.663 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.642+0000 7f4094f3e700 1 -- 172.21.15.47:0/2703008497 wait complete. 2022-06-29T15:51:52.663 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.646+0000 7f4094f3e700 1 Processor -- start 2022-06-29T15:51:52.663 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.646+0000 7f4094f3e700 1 -- start start 2022-06-29T15:51:52.663 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.646+0000 7f4094f3e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f407c004e70 0x7f4090144e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:52.663 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.646+0000 7f4094f3e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f409013df50 0x7f40900a3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:52.664 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.646+0000 7f4094f3e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40900b7110 0x7f40900a3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:52.664 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.646+0000 7f4094f3e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40900b53c0 con 0x7f409013df50 2022-06-29T15:51:52.664 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.646+0000 7f4094f3e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40900b55f0 con 0x7f40900b7110 2022-06-29T15:51:52.664 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.646+0000 7f4094f3e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40900b5890 con 0x7f407c004e70 2022-06-29T15:51:52.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.646+0000 7f408effd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f409013df50 0x7f40900a3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:52.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.646+0000 7f408effd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f409013df50 0x7f40900a3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54370/0 (socket says 172.21.15.47:54370) 2022-06-29T15:51:52.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.646+0000 7f408effd700 1 -- 172.21.15.47:0/2157751935 learned_addr learned my addr 172.21.15.47:0/2157751935 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:52.665 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.646+0000 7f408ffff700 1 --2- 172.21.15.47:0/2157751935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40900b7110 0x7f40900a3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:52.666 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.646+0000 7f408f7fe700 1 --2- 172.21.15.47:0/2157751935 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f407c004e70 0x7f4090144e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:52.666 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.646+0000 7f408effd700 1 -- 172.21.15.47:0/2157751935 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f407c004e70 msgr2=0x7f4090144e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:52.666 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.646+0000 7f408effd700 1 --2- 172.21.15.47:0/2157751935 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f407c004e70 0x7f4090144e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:52.667 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.646+0000 7f408effd700 1 -- 172.21.15.47:0/2157751935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40900b7110 msgr2=0x7f40900a3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:52.667 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.646+0000 7f408effd700 1 --2- 172.21.15.47:0/2157751935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40900b7110 0x7f40900a3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:52.667 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.646+0000 7f408effd700 1 -- 172.21.15.47:0/2157751935 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40900b5b30 con 0x7f409013df50 2022-06-29T15:51:52.667 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.646+0000 7f408ffff700 1 --2- 172.21.15.47:0/2157751935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40900b7110 0x7f40900a3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:51:52.667 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.646+0000 7f408effd700 1 --2- 172.21.15.47:0/2157751935 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f409013df50 0x7f40900a3270 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 rx=0x7f408400b000 tx=0x7f4084006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:52.668 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.646+0000 7f408cff9700 1 -- 172.21.15.47:0/2157751935 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f408400f730 con 0x7f409013df50 2022-06-29T15:51:52.668 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.646+0000 7f408cff9700 1 -- 172.21.15.47:0/2157751935 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f408400f890 con 0x7f409013df50 2022-06-29T15:51:52.668 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.646+0000 7f4094f3e700 1 -- 172.21.15.47:0/2157751935 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40900a3c50 con 0x7f409013df50 2022-06-29T15:51:52.668 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.646+0000 7f408cff9700 1 -- 172.21.15.47:0/2157751935 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f408400fbf0 con 0x7f409013df50 2022-06-29T15:51:52.668 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.646+0000 7f4094f3e700 1 -- 172.21.15.47:0/2157751935 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f409013c4a0 con 0x7f409013df50 2022-06-29T15:51:52.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.650+0000 7f4094f3e700 1 -- 172.21.15.47:0/2157751935 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40900b5cc0 con 0x7f409013df50 2022-06-29T15:51:52.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.650+0000 7f408cff9700 1 -- 172.21.15.47:0/2157751935 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f408400fd50 con 0x7f409013df50 2022-06-29T15:51:52.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.650+0000 7f408cff9700 1 --2- 172.21.15.47:0/2157751935 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4070053a70 0x7f4070055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:52.670 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.650+0000 7f408cff9700 1 -- 172.21.15.47:0/2157751935 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f408406c950 con 0x7f409013df50 2022-06-29T15:51:52.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.650+0000 7f408f7fe700 1 --2- 172.21.15.47:0/2157751935 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4070053a70 0x7f4070055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:52.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.650+0000 7f408f7fe700 1 --2- 172.21.15.47:0/2157751935 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4070053a70 0x7f4070055f30 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7f4080005000 tx=0x7f4080009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:52.671 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.658+0000 7f408cff9700 1 -- 172.21.15.47:0/2157751935 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4084043730 con 0x7f409013df50 2022-06-29T15:51:52.830 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.822+0000 7f4094f3e700 1 -- 172.21.15.47:0/2157751935 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f40900573d0 con 0x7f409013df50 2022-06-29T15:51:52.841 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.834+0000 7f408cff9700 1 -- 172.21.15.47:0/2157751935 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 27 and data pool 28 v73) v1 ==== 189+0+0 (secure 0 0 0) 0x7f4084018020 con 0x7f409013df50 2022-06-29T15:51:52.846 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.838+0000 7f4094f3e700 1 -- 172.21.15.47:0/2157751935 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4070053a70 msgr2=0x7f4070055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:52.847 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.838+0000 7f4094f3e700 1 --2- 172.21.15.47:0/2157751935 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4070053a70 0x7f4070055f30 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7f4080005000 tx=0x7f4080009040).stop 2022-06-29T15:51:52.847 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.838+0000 7f4094f3e700 1 -- 172.21.15.47:0/2157751935 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f409013df50 msgr2=0x7f40900a3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:52.847 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.838+0000 7f4094f3e700 1 --2- 172.21.15.47:0/2157751935 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f409013df50 0x7f40900a3270 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 rx=0x7f408400b000 tx=0x7f4084006df0).stop 2022-06-29T15:51:52.847 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.838+0000 7f4094f3e700 1 -- 172.21.15.47:0/2157751935 shutdown_connections 2022-06-29T15:51:52.848 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.838+0000 7f4094f3e700 1 --2- 172.21.15.47:0/2157751935 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4070053a70 0x7f4070055f30 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:52.848 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.838+0000 7f4094f3e700 1 --2- 172.21.15.47:0/2157751935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40900b7110 0x7f40900a3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:52.848 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.838+0000 7f4094f3e700 1 --2- 172.21.15.47:0/2157751935 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f409013df50 0x7f40900a3270 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:52.848 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.838+0000 7f4094f3e700 1 --2- 172.21.15.47:0/2157751935 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f407c004e70 0x7f4090144e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:52.849 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.838+0000 7f4094f3e700 1 -- 172.21.15.47:0/2157751935 >> 172.21.15.47:0/2157751935 conn(0x7f4090062570 msgr2=0x7f4090142ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:52.849 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.842+0000 7f4094f3e700 1 -- 172.21.15.47:0/2157751935 shutdown_connections 2022-06-29T15:51:52.850 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.842+0000 7f4094f3e700 1 -- 172.21.15.47:0/2157751935 wait complete. 2022-06-29T15:51:52.850 INFO:teuthology.orchestra.run.smithi047.stderr:new fs with metadata pool 27 and data pool 28 2022-06-29T15:51:52.861 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:51:52.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.946+0000 7fccde73c700 1 Processor -- start 2022-06-29T15:51:52.952 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.946+0000 7fccde73c700 1 -- start start 2022-06-29T15:51:52.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.946+0000 7fccde73c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fccd805f0b0 0x7fccd80b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:52.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.946+0000 7fccde73c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fccd80b1260 0x7fccd814b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:52.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.946+0000 7fccde73c700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccd80b0740 con 0x7fccd80b2120 2022-06-29T15:51:52.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.946+0000 7fccde73c700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccd8147460 con 0x7fccd805f0b0 2022-06-29T15:51:52.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.946+0000 7fccde73c700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccd80b0970 con 0x7fccd80b1260 2022-06-29T15:51:52.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.946+0000 7fccd7fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fccd80b2120 0x7fccd8147af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39940/0 (socket says 172.21.15.47:39940) 2022-06-29T15:51:52.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.946+0000 7fccd7fff700 1 -- 172.21.15.47:0/1995530891 learned_addr learned my addr 172.21.15.47:0/1995530891 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:52.958 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.946+0000 7fccd6ffd700 1 -- 172.21.15.47:0/1995530891 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2276703665 0 0) 0x7fccd80b0740 con 0x7fccd80b2120 2022-06-29T15:51:52.958 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.946+0000 7fccd6ffd700 1 -- 172.21.15.47:0/1995530891 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccb8003260 con 0x7fccd80b2120 2022-06-29T15:51:52.958 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.946+0000 7fccdccd9700 1 --2- 172.21.15.47:0/1995530891 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fccd80b1260 0x7fccd814b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:52.958 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.946+0000 7fccd77fe700 1 --2- 172.21.15.47:0/1995530891 >> v2:172.21.15.64:3300/0 conn(0x7fccd805f0b0 0x7fccd80b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:52.959 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.946+0000 7fccd6ffd700 1 -- 172.21.15.47:0/1995530891 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3685124616 0 0) 0x7fccb8003260 con 0x7fccd80b2120 2022-06-29T15:51:52.959 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.946+0000 7fccd6ffd700 1 -- 172.21.15.47:0/1995530891 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fccd80b1260 msgr2=0x7fccd814b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:52.959 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.946+0000 7fccd6ffd700 1 --2- 172.21.15.47:0/1995530891 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fccd80b1260 0x7fccd814b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:52.959 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.946+0000 7fccd6ffd700 1 -- 172.21.15.47:0/1995530891 >> v2:172.21.15.64:3300/0 conn(0x7fccd805f0b0 msgr2=0x7fccd80b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:52.960 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.946+0000 7fccd6ffd700 1 --2- 172.21.15.47:0/1995530891 >> v2:172.21.15.64:3300/0 conn(0x7fccd805f0b0 0x7fccd80b2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:52.960 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.946+0000 7fccd6ffd700 1 -- 172.21.15.47:0/1995530891 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd80b0c10 con 0x7fccd80b2120 2022-06-29T15:51:52.960 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.946+0000 7fccd6ffd700 1 -- 172.21.15.47:0/1995530891 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7fccc80019e0 con 0x7fccd80b2120 2022-06-29T15:51:52.960 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.946+0000 7fccd6ffd700 1 -- 172.21.15.47:0/1995530891 >> v1:172.21.15.47:6789/0 conn(0x7fccd80b2120 legacy=0x7fccd8147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:52.960 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.946+0000 7fccd6ffd700 1 --2- 172.21.15.47:0/1995530891 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccb8003750 0x7fccb8003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:52.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.946+0000 7fccd6ffd700 1 --2- 172.21.15.47:0/1995530891 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fccb8004500 0x7fccb8004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:52.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.946+0000 7fccd6ffd700 1 --2- 172.21.15.47:0/1995530891 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fccd80b1260 0x7fccb8004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:52.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.946+0000 7fccd6ffd700 1 -- 172.21.15.47:0/1995530891 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd80b0970 con 0x7fccb8004500 2022-06-29T15:51:52.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.946+0000 7fccd6ffd700 1 -- 172.21.15.47:0/1995530891 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb800b750 con 0x7fccb8003750 2022-06-29T15:51:52.962 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.946+0000 7fccd6ffd700 1 -- 172.21.15.47:0/1995530891 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccb800b8e0 con 0x7fccd80b1260 2022-06-29T15:51:52.962 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.946+0000 7fccd77fe700 1 --2- 172.21.15.47:0/1995530891 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fccb8004500 0x7fccb8004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:52.962 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.946+0000 7fccd7fff700 1 --2- 172.21.15.47:0/1995530891 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccb8003750 0x7fccb8003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:52.962 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.946+0000 7fccdccd9700 1 --2- 172.21.15.47:0/1995530891 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fccd80b1260 0x7fccb8004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:52.963 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.946+0000 7fccd77fe700 1 -- 172.21.15.47:0/1995530891 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fccd80b1260 msgr2=0x7fccb8004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:52.963 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.946+0000 7fccd77fe700 1 --2- 172.21.15.47:0/1995530891 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fccd80b1260 0x7fccb8004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:52.963 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.946+0000 7fccd77fe700 1 -- 172.21.15.47:0/1995530891 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccb8003750 msgr2=0x7fccb8003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:52.963 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.946+0000 7fccd77fe700 1 --2- 172.21.15.47:0/1995530891 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccb8003750 0x7fccb8003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:52.964 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.946+0000 7fccd77fe700 1 -- 172.21.15.47:0/1995530891 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb8003260 con 0x7fccb8004500 2022-06-29T15:51:52.964 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.946+0000 7fccd77fe700 1 --2- 172.21.15.47:0/1995530891 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fccb8004500 0x7fccb8004980 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 rx=0x7fccc000a000 tx=0x7fccc0008040).ready entity=mon.0 client_cookie=ca391bbdadb75d32 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:52.964 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.950+0000 7fccd6ffd700 1 -- 172.21.15.47:0/1995530891 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fccc0013670 con 0x7fccb8004500 2022-06-29T15:51:52.964 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.950+0000 7fccd6ffd700 1 -- 172.21.15.47:0/1995530891 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccc00137d0 con 0x7fccb8004500 2022-06-29T15:51:52.964 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.950+0000 7fccd6ffd700 1 -- 172.21.15.47:0/1995530891 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fccc0013b30 con 0x7fccb8004500 2022-06-29T15:51:52.965 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.950+0000 7fccde73c700 1 -- 172.21.15.47:0/1995530891 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fccb8004500 msgr2=0x7fccb8004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:52.965 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.950+0000 7fccde73c700 1 --2- 172.21.15.47:0/1995530891 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fccb8004500 0x7fccb8004980 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 rx=0x7fccc000a000 tx=0x7fccc0008040).stop 2022-06-29T15:51:52.965 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.950+0000 7fccdccd9700 1 -- 172.21.15.47:0/1995530891 reap_dead start 2022-06-29T15:51:52.965 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.950+0000 7fccde73c700 1 -- 172.21.15.47:0/1995530891 shutdown_connections 2022-06-29T15:51:52.966 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.950+0000 7fccde73c700 1 -- 172.21.15.47:0/1995530891 >> 172.21.15.47:0/1995530891 conn(0x7fccd80626f0 msgr2=0x7fccd8062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:52.966 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.950+0000 7fccde73c700 1 -- 172.21.15.47:0/1995530891 shutdown_connections 2022-06-29T15:51:52.966 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.950+0000 7fccde73c700 1 -- 172.21.15.47:0/1995530891 wait complete. 2022-06-29T15:51:52.969 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.950+0000 7fccde73c700 1 Processor -- start 2022-06-29T15:51:52.969 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.950+0000 7fccde73c700 1 -- start start 2022-06-29T15:51:52.969 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.954+0000 7fccde73c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fccb8004500 0x7fccd8145f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:52.970 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.954+0000 7fccde73c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccd8141e80 0x7fccd8142300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:52.970 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.954+0000 7fccde73c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fccd8156af0 0x7fccd8156f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:52.970 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.954+0000 7fccde73c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd8149fc0 con 0x7fccd8156af0 2022-06-29T15:51:52.971 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.954+0000 7fccde73c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd814a260 con 0x7fccd8141e80 2022-06-29T15:51:52.971 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.954+0000 7fccde73c700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccd814a500 con 0x7fccb8004500 2022-06-29T15:51:52.971 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.954+0000 7fccdccd9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fccd8156af0 0x7fccd8156f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:52.971 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.954+0000 7fccdccd9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fccd8156af0 0x7fccd8156f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54402/0 (socket says 172.21.15.47:54402) 2022-06-29T15:51:52.971 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.954+0000 7fccdccd9700 1 -- 172.21.15.47:0/1095467708 learned_addr learned my addr 172.21.15.47:0/1095467708 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:52.972 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.954+0000 7fccd7fff700 1 --2- 172.21.15.47:0/1095467708 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fccb8004500 0x7fccd8145f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:52.972 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.954+0000 7fccd77fe700 1 --2- 172.21.15.47:0/1095467708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccd8141e80 0x7fccd8142300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:52.972 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.954+0000 7fccdccd9700 1 -- 172.21.15.47:0/1095467708 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fccb8004500 msgr2=0x7fccd8145f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:52.972 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.954+0000 7fccdccd9700 1 --2- 172.21.15.47:0/1095467708 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fccb8004500 0x7fccd8145f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:52.973 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.954+0000 7fccdccd9700 1 -- 172.21.15.47:0/1095467708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccd8141e80 msgr2=0x7fccd8142300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:52.973 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.954+0000 7fccdccd9700 1 --2- 172.21.15.47:0/1095467708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccd8141e80 0x7fccd8142300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:52.973 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.954+0000 7fccdccd9700 1 -- 172.21.15.47:0/1095467708 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd814a7a0 con 0x7fccd8156af0 2022-06-29T15:51:52.973 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.954+0000 7fccd7fff700 1 --2- 172.21.15.47:0/1095467708 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fccb8004500 0x7fccd8145f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:51:52.974 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.954+0000 7fccdccd9700 1 --2- 172.21.15.47:0/1095467708 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fccd8156af0 0x7fccd8156f80 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 rx=0x7fccd000c000 tx=0x7fccd0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:52.974 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.954+0000 7fccd57fa700 1 -- 172.21.15.47:0/1095467708 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fccd00137d0 con 0x7fccd8156af0 2022-06-29T15:51:52.974 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.954+0000 7fccd57fa700 1 -- 172.21.15.47:0/1095467708 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccd0013930 con 0x7fccd8156af0 2022-06-29T15:51:52.974 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.954+0000 7fccde73c700 1 -- 172.21.15.47:0/1095467708 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccd8149cb0 con 0x7fccd8156af0 2022-06-29T15:51:52.974 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.954+0000 7fccd57fa700 1 -- 172.21.15.47:0/1095467708 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fccd0013c90 con 0x7fccd8156af0 2022-06-29T15:51:52.975 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.954+0000 7fccde73c700 1 -- 172.21.15.47:0/1095467708 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccd8157690 con 0x7fccd8156af0 2022-06-29T15:51:52.975 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.954+0000 7fccde73c700 1 -- 172.21.15.47:0/1095467708 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccd80573d0 con 0x7fccd8156af0 2022-06-29T15:51:52.975 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.954+0000 7fccd57fa700 1 -- 172.21.15.47:0/1095467708 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fccd0014470 con 0x7fccd8156af0 2022-06-29T15:51:52.975 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.958+0000 7fccd57fa700 1 --2- 172.21.15.47:0/1095467708 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fccbc053a10 0x7fccbc055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:52.976 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.958+0000 7fccd57fa700 1 -- 172.21.15.47:0/1095467708 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fccd006c1f0 con 0x7fccd8156af0 2022-06-29T15:51:52.976 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.958+0000 7fccd7fff700 1 --2- 172.21.15.47:0/1095467708 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fccbc053a10 0x7fccbc055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:52.976 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.958+0000 7fccd7fff700 1 --2- 172.21.15.47:0/1095467708 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fccbc053a10 0x7fccbc055ed0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7fccc8005000 tx=0x7fccc8009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:52.976 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:52.962+0000 7fccd57fa700 1 -- 172.21.15.47:0/1095467708 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fccd0042f50 con 0x7fccd8156af0 2022-06-29T15:51:53.095 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.086+0000 7fccde73c700 1 -- 172.21.15.47:0/1095467708 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fccd806a1a0 con 0x7fccd8156af0 2022-06-29T15:51:53.097 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.090+0000 7fccd57fa700 1 -- 172.21.15.47:0/1095467708 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) v1 ==== 74+0+16298 (secure 0 0 0) 0x7fccd0046570 con 0x7fccd8156af0 2022-06-29T15:51:53.097 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:53.098 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":150,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:51:52.827745+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:51:50.583017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2022-06-29T15:51:51.834830+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:51:53.100 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.094+0000 7fccde73c700 1 -- 172.21.15.47:0/1095467708 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fccbc053a10 msgr2=0x7fccbc055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:53.100 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.094+0000 7fccde73c700 1 --2- 172.21.15.47:0/1095467708 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fccbc053a10 0x7fccbc055ed0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7fccc8005000 tx=0x7fccc8009040).stop 2022-06-29T15:51:53.101 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.094+0000 7fccde73c700 1 -- 172.21.15.47:0/1095467708 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fccd8156af0 msgr2=0x7fccd8156f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:53.101 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.094+0000 7fccde73c700 1 --2- 172.21.15.47:0/1095467708 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fccd8156af0 0x7fccd8156f80 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 rx=0x7fccd000c000 tx=0x7fccd0008040).stop 2022-06-29T15:51:53.101 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.094+0000 7fccde73c700 1 -- 172.21.15.47:0/1095467708 shutdown_connections 2022-06-29T15:51:53.101 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.094+0000 7fccde73c700 1 --2- 172.21.15.47:0/1095467708 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fccd8156af0 0x7fccd8156f80 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:53.102 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.094+0000 7fccde73c700 1 --2- 172.21.15.47:0/1095467708 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fccbc053a10 0x7fccbc055ed0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:53.102 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.094+0000 7fccde73c700 1 --2- 172.21.15.47:0/1095467708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccd8141e80 0x7fccd8142300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:53.102 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.094+0000 7fccde73c700 1 --2- 172.21.15.47:0/1095467708 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fccb8004500 0x7fccd8145f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:53.102 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.094+0000 7fccde73c700 1 -- 172.21.15.47:0/1095467708 >> 172.21.15.47:0/1095467708 conn(0x7fccd80626f0 msgr2=0x7fccd80b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:53.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.094+0000 7fccde73c700 1 -- 172.21.15.47:0/1095467708 shutdown_connections 2022-06-29T15:51:53.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.094+0000 7fccde73c700 1 -- 172.21.15.47:0/1095467708 wait complete. 2022-06-29T15:51:53.115 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:51:53.213 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.206+0000 7f4021b54700 1 Processor -- start 2022-06-29T15:51:53.213 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.206+0000 7f4021b54700 1 -- start start 2022-06-29T15:51:53.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.206+0000 7f4021b54700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f401c05c510 0x7f401c143bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:53.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.206+0000 7f4021b54700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f401c0b2c40 0x7f401c0b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:53.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.206+0000 7f4021b54700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f401c142f90 con 0x7f401c05c8f0 2022-06-29T15:51:53.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.206+0000 7f4021b54700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f401c1431f0 con 0x7f401c05c510 2022-06-29T15:51:53.217 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.206+0000 7f4021b54700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f401c143490 con 0x7f401c0b2c40 2022-06-29T15:51:53.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.206+0000 7f4021353700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f401c0b2c40 0x7f401c0b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:53.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.206+0000 7f4020b52700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f401c05c510 0x7f401c143bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:53.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.206+0000 7f401bfff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f401c05c8f0 0x7f401c0b4100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:47524/0 (socket says 172.21.15.47:47524) 2022-06-29T15:51:53.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.206+0000 7f4021353700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f401c0b2c40 0x7f401c0b30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:51512/0 (socket says 172.21.15.47:51512) 2022-06-29T15:51:53.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.206+0000 7f401bfff700 1 -- 172.21.15.47:0/138631725 learned_addr learned my addr 172.21.15.47:0/138631725 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:53.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.206+0000 7f401b7fe700 1 -- 172.21.15.47:0/138631725 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1157898393 0 0) 0x7f401c142f90 con 0x7f401c05c8f0 2022-06-29T15:51:53.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.206+0000 7f401b7fe700 1 -- 172.21.15.47:0/138631725 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4008003260 con 0x7f401c05c8f0 2022-06-29T15:51:53.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.210+0000 7f4021353700 1 -- 172.21.15.47:0/138631725 >> v2:172.21.15.64:3300/0 conn(0x7f401c05c510 msgr2=0x7f401c143bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:53.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.210+0000 7f4021353700 1 --2- 172.21.15.47:0/138631725 >> v2:172.21.15.64:3300/0 conn(0x7f401c05c510 0x7f401c143bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:53.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.210+0000 7f4021353700 1 -- 172.21.15.47:0/138631725 >> v1:172.21.15.64:6789/0 conn(0x7f401c05c8f0 legacy=0x7f401c0b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:53.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.210+0000 7f4021353700 1 -- 172.21.15.47:0/138631725 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f401c143730 con 0x7f401c0b2c40 2022-06-29T15:51:53.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.210+0000 7f4021353700 1 --2- 172.21.15.47:0/138631725 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f401c0b2c40 0x7f401c0b30d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f40140032f0 tx=0x7f40140091d0).ready entity=mon.2 client_cookie=59651cc114f27515 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:53.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.210+0000 7f401b7fe700 1 -- 172.21.15.47:0/138631725 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f401400b600 con 0x7f401c0b2c40 2022-06-29T15:51:53.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.210+0000 7f401b7fe700 1 -- 172.21.15.47:0/138631725 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f401400b760 con 0x7f401c0b2c40 2022-06-29T15:51:53.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.210+0000 7f401b7fe700 1 -- 172.21.15.47:0/138631725 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f401400b8d0 con 0x7f401c0b2c40 2022-06-29T15:51:53.222 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.210+0000 7f4021b54700 1 -- 172.21.15.47:0/138631725 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f401c0b2c40 msgr2=0x7f401c0b30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:53.222 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.210+0000 7f4021b54700 1 --2- 172.21.15.47:0/138631725 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f401c0b2c40 0x7f401c0b30d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f40140032f0 tx=0x7f40140091d0).stop 2022-06-29T15:51:53.223 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.210+0000 7f4021b54700 1 -- 172.21.15.47:0/138631725 shutdown_connections 2022-06-29T15:51:53.223 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.210+0000 7f4021b54700 1 --2- 172.21.15.47:0/138631725 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f401c0b2c40 0x7f401c0b30d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:53.223 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.210+0000 7f4021b54700 1 --2- 172.21.15.47:0/138631725 >> v2:172.21.15.64:3300/0 conn(0x7f401c05c510 0x7f401c143bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:53.223 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.210+0000 7f4021b54700 1 -- 172.21.15.47:0/138631725 >> 172.21.15.47:0/138631725 conn(0x7f401c060e00 msgr2=0x7f401c0611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:53.223 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.210+0000 7f4021b54700 1 -- 172.21.15.47:0/138631725 shutdown_connections 2022-06-29T15:51:53.224 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.210+0000 7f4021b54700 1 -- 172.21.15.47:0/138631725 wait complete. 2022-06-29T15:51:53.227 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.210+0000 7f4021b54700 1 Processor -- start 2022-06-29T15:51:53.227 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.210+0000 7f4021b54700 1 -- start start 2022-06-29T15:51:53.227 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.214+0000 7f4021b54700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f401c05c510 0x7f401c1520b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:53.227 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.214+0000 7f4021b54700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f401c05c8f0 0x7f401c13b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:53.228 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.214+0000 7f4021b54700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f401c0b2c40 0x7f401c0b66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:53.228 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.214+0000 7f4021b54700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401c0b4ed0 con 0x7f401c05c510 2022-06-29T15:51:53.228 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.214+0000 7f4021b54700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401c13d6a0 con 0x7f401c05c8f0 2022-06-29T15:51:53.228 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.214+0000 7f4021b54700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f401c13d940 con 0x7f401c0b2c40 2022-06-29T15:51:53.229 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.214+0000 7f4020b52700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f401c05c510 0x7f401c1520b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:53.229 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.214+0000 7f4020b52700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f401c05c510 0x7f401c1520b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54412/0 (socket says 172.21.15.47:54412) 2022-06-29T15:51:53.229 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.214+0000 7f4020b52700 1 -- 172.21.15.47:0/3623217740 learned_addr learned my addr 172.21.15.47:0/3623217740 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:53.229 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.214+0000 7f4021353700 1 --2- 172.21.15.47:0/3623217740 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f401c0b2c40 0x7f401c0b66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:53.230 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.214+0000 7f401bfff700 1 --2- 172.21.15.47:0/3623217740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f401c05c8f0 0x7f401c13b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:53.230 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.214+0000 7f4020b52700 1 -- 172.21.15.47:0/3623217740 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f401c0b2c40 msgr2=0x7f401c0b66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:53.230 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.214+0000 7f4020b52700 1 --2- 172.21.15.47:0/3623217740 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f401c0b2c40 0x7f401c0b66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:53.230 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.214+0000 7f4020b52700 1 -- 172.21.15.47:0/3623217740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f401c05c8f0 msgr2=0x7f401c13b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:53.231 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.214+0000 7f4020b52700 1 --2- 172.21.15.47:0/3623217740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f401c05c8f0 0x7f401c13b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:53.231 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.214+0000 7f4020b52700 1 -- 172.21.15.47:0/3623217740 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4014009470 con 0x7f401c05c510 2022-06-29T15:51:53.231 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.214+0000 7f4020b52700 1 --2- 172.21.15.47:0/3623217740 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f401c05c510 0x7f401c1520b0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 rx=0x7f401000c000 tx=0x7f4010006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:53.231 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.214+0000 7f4019ffb700 1 -- 172.21.15.47:0/3623217740 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4010010780 con 0x7f401c05c510 2022-06-29T15:51:53.232 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.214+0000 7f4019ffb700 1 -- 172.21.15.47:0/3623217740 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40100108e0 con 0x7f401c05c510 2022-06-29T15:51:53.232 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.214+0000 7f4021b54700 1 -- 172.21.15.47:0/3623217740 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f401c0b6250 con 0x7f401c05c510 2022-06-29T15:51:53.232 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.214+0000 7f4019ffb700 1 -- 172.21.15.47:0/3623217740 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4010010c40 con 0x7f401c05c510 2022-06-29T15:51:53.232 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.214+0000 7f4021b54700 1 -- 172.21.15.47:0/3623217740 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f401c0b5b40 con 0x7f401c05c510 2022-06-29T15:51:53.233 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.214+0000 7f4021b54700 1 -- 172.21.15.47:0/3623217740 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f401c13db00 con 0x7f401c05c510 2022-06-29T15:51:53.233 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.218+0000 7f4019ffb700 1 -- 172.21.15.47:0/3623217740 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f401001c460 con 0x7f401c05c510 2022-06-29T15:51:53.233 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.218+0000 7f4019ffb700 1 --2- 172.21.15.47:0/3623217740 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3ffc053a70 0x7f3ffc055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:53.233 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.218+0000 7f4019ffb700 1 -- 172.21.15.47:0/3623217740 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f401006c3a0 con 0x7f401c05c510 2022-06-29T15:51:53.233 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.218+0000 7f401bfff700 1 --2- 172.21.15.47:0/3623217740 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3ffc053a70 0x7f3ffc055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:53.234 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.218+0000 7f401bfff700 1 --2- 172.21.15.47:0/3623217740 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3ffc053a70 0x7f3ffc055f30 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7f400c00d000 tx=0x7f400c009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:53.235 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.226+0000 7f4019ffb700 1 -- 172.21.15.47:0/3623217740 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4010043100 con 0x7f401c05c510 2022-06-29T15:51:53.361 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.354+0000 7f4021b54700 1 -- 172.21.15.47:0/3623217740 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f401c0573d0 con 0x7f401c05c510 2022-06-29T15:51:53.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.358+0000 7f4019ffb700 1 -- 172.21.15.47:0/3623217740 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) v1 ==== 74+0+16298 (secure 0 0 0) 0x7f4010046720 con 0x7f401c05c510 2022-06-29T15:51:53.364 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:53.364 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":150,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:51:52.827745+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:51:50.583017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2022-06-29T15:51:51.834830+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:51:53.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.358+0000 7f4021b54700 1 -- 172.21.15.47:0/3623217740 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3ffc053a70 msgr2=0x7f3ffc055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:53.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.358+0000 7f4021b54700 1 --2- 172.21.15.47:0/3623217740 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3ffc053a70 0x7f3ffc055f30 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7f400c00d000 tx=0x7f400c009040).stop 2022-06-29T15:51:53.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.358+0000 7f4021b54700 1 -- 172.21.15.47:0/3623217740 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f401c05c510 msgr2=0x7f401c1520b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:53.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.358+0000 7f4021b54700 1 --2- 172.21.15.47:0/3623217740 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f401c05c510 0x7f401c1520b0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 rx=0x7f401000c000 tx=0x7f4010006df0).stop 2022-06-29T15:51:53.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.358+0000 7f4021b54700 1 -- 172.21.15.47:0/3623217740 shutdown_connections 2022-06-29T15:51:53.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.358+0000 7f4021b54700 1 --2- 172.21.15.47:0/3623217740 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f401c0b2c40 0x7f401c0b66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:53.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.358+0000 7f4021b54700 1 --2- 172.21.15.47:0/3623217740 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3ffc053a70 0x7f3ffc055f30 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:53.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.358+0000 7f4021b54700 1 --2- 172.21.15.47:0/3623217740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f401c05c8f0 0x7f401c13b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:53.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.358+0000 7f4021b54700 1 --2- 172.21.15.47:0/3623217740 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f401c05c510 0x7f401c1520b0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:53.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.358+0000 7f4021b54700 1 -- 172.21.15.47:0/3623217740 >> 172.21.15.47:0/3623217740 conn(0x7f401c060e00 msgr2=0x7f401c1419f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:53.368 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.358+0000 7f4021b54700 1 -- 172.21.15.47:0/3623217740 shutdown_connections 2022-06-29T15:51:53.369 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.358+0000 7f4021b54700 1 -- 172.21.15.47:0/3623217740 wait complete. 2022-06-29T15:51:53.378 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-06-29T15:51:53.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.482+0000 7f476e5eb700 1 Processor -- start 2022-06-29T15:51:53.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.482+0000 7f476e5eb700 1 -- start start 2022-06-29T15:51:53.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.482+0000 7f476e5eb700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f47680b2900 0x7f476805f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:53.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.482+0000 7f476e5eb700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f47680b1c60 0x7f4768099d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:53.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.482+0000 7f476e5eb700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f476809a230 0x7f476805fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:53.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.482+0000 7f476e5eb700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47680609a0 con 0x7f47680b2900 2022-06-29T15:51:53.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.482+0000 7f476e5eb700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47680b0fe0 con 0x7f47680b1c60 2022-06-29T15:51:53.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.482+0000 7f476e5eb700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47680b2ce0 con 0x7f476809a230 2022-06-29T15:51:53.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.482+0000 7f476d5e9700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f47680b2900 0x7f476805f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:53.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.482+0000 7f476d5e9700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f47680b2900 0x7f476805f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54420/0 (socket says 172.21.15.47:54420) 2022-06-29T15:51:53.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.482+0000 7f476d5e9700 1 -- 172.21.15.47:0/2406701746 learned_addr learned my addr 172.21.15.47:0/2406701746 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:53.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.482+0000 7f476cde8700 1 --2- 172.21.15.47:0/2406701746 >> v2:172.21.15.64:3300/0 conn(0x7f47680b1c60 0x7f4768099d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:53.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.482+0000 7f476ddea700 1 --2- 172.21.15.47:0/2406701746 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f476809a230 0x7f476805fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:53.494 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.482+0000 7f476d5e9700 1 -- 172.21.15.47:0/2406701746 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f476809a230 msgr2=0x7f476805fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:53.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.482+0000 7f476d5e9700 1 --2- 172.21.15.47:0/2406701746 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f476809a230 0x7f476805fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:53.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.482+0000 7f476d5e9700 1 -- 172.21.15.47:0/2406701746 >> v2:172.21.15.64:3300/0 conn(0x7f47680b1c60 msgr2=0x7f4768099d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:53.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.482+0000 7f476d5e9700 1 --2- 172.21.15.47:0/2406701746 >> v2:172.21.15.64:3300/0 conn(0x7f47680b1c60 0x7f4768099d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:53.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.482+0000 7f476d5e9700 1 -- 172.21.15.47:0/2406701746 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47680b0210 con 0x7f47680b2900 2022-06-29T15:51:53.495 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.486+0000 7f476d5e9700 1 --2- 172.21.15.47:0/2406701746 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47680b2900 0x7f476805f370 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 rx=0x7f4758002f70 tx=0x7f4758006eb0).ready entity=mon.0 client_cookie=ff43ea550a8f5531 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:53.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.486+0000 7f4767fff700 1 -- 172.21.15.47:0/2406701746 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f475800d570 con 0x7f47680b2900 2022-06-29T15:51:53.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.486+0000 7f4767fff700 1 -- 172.21.15.47:0/2406701746 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4758000e90 con 0x7f47680b2900 2022-06-29T15:51:53.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.486+0000 7f4767fff700 1 -- 172.21.15.47:0/2406701746 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f475800d900 con 0x7f47680b2900 2022-06-29T15:51:53.499 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.486+0000 7f476e5eb700 1 -- 172.21.15.47:0/2406701746 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47680b2900 msgr2=0x7f476805f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:53.499 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.486+0000 7f476e5eb700 1 --2- 172.21.15.47:0/2406701746 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47680b2900 0x7f476805f370 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 rx=0x7f4758002f70 tx=0x7f4758006eb0).stop 2022-06-29T15:51:53.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.486+0000 7f476e5eb700 1 -- 172.21.15.47:0/2406701746 shutdown_connections 2022-06-29T15:51:53.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.486+0000 7f476e5eb700 1 --2- 172.21.15.47:0/2406701746 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f476809a230 0x7f476805fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:53.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.486+0000 7f476e5eb700 1 --2- 172.21.15.47:0/2406701746 >> v2:172.21.15.64:3300/0 conn(0x7f47680b1c60 0x7f4768099d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:53.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.486+0000 7f476e5eb700 1 --2- 172.21.15.47:0/2406701746 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47680b2900 0x7f476805f370 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:53.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.486+0000 7f476e5eb700 1 -- 172.21.15.47:0/2406701746 >> 172.21.15.47:0/2406701746 conn(0x7f4768062df0 msgr2=0x7f47680631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:53.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.486+0000 7f476e5eb700 1 -- 172.21.15.47:0/2406701746 shutdown_connections 2022-06-29T15:51:53.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.486+0000 7f476e5eb700 1 -- 172.21.15.47:0/2406701746 wait complete. 2022-06-29T15:51:53.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.490+0000 7f476e5eb700 1 Processor -- start 2022-06-29T15:51:53.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.490+0000 7f476e5eb700 1 -- start start 2022-06-29T15:51:53.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.490+0000 7f476e5eb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f476809a230 0x7f476814f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:53.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.490+0000 7f476e5eb700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47680b1c60 0x7f476814fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:53.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.490+0000 7f476e5eb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47680b2900 0x7f4768095c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:53.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.490+0000 7f476e5eb700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4768094780 con 0x7f47680b2900 2022-06-29T15:51:53.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.490+0000 7f476e5eb700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47681501a0 con 0x7f476809a230 2022-06-29T15:51:53.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.490+0000 7f476e5eb700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47681530b0 con 0x7f47680b1c60 2022-06-29T15:51:53.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.490+0000 7f476ddea700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47680b2900 0x7f4768095c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:53.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.490+0000 7f476ddea700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47680b2900 0x7f4768095c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54430/0 (socket says 172.21.15.47:54430) 2022-06-29T15:51:53.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.490+0000 7f476ddea700 1 -- 172.21.15.47:0/1985223587 learned_addr learned my addr 172.21.15.47:0/1985223587 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:53.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.490+0000 7f476cde8700 1 --2- 172.21.15.47:0/1985223587 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47680b1c60 0x7f476814fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:53.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.490+0000 7f476d5e9700 1 --2- 172.21.15.47:0/1985223587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f476809a230 0x7f476814f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:53.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.490+0000 7f476ddea700 1 -- 172.21.15.47:0/1985223587 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47680b1c60 msgr2=0x7f476814fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:53.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.490+0000 7f476ddea700 1 --2- 172.21.15.47:0/1985223587 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47680b1c60 0x7f476814fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:53.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.490+0000 7f476ddea700 1 -- 172.21.15.47:0/1985223587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f476809a230 msgr2=0x7f476814f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:53.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.490+0000 7f476ddea700 1 --2- 172.21.15.47:0/1985223587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f476809a230 0x7f476814f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:53.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.490+0000 7f476ddea700 1 -- 172.21.15.47:0/1985223587 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f475800b1e0 con 0x7f47680b2900 2022-06-29T15:51:53.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.490+0000 7f476ddea700 1 --2- 172.21.15.47:0/1985223587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47680b2900 0x7f4768095c20 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 rx=0x7f476000a000 tx=0x7f4760008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:53.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.490+0000 7f47667fc700 1 -- 172.21.15.47:0/1985223587 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f476000c070 con 0x7f47680b2900 2022-06-29T15:51:53.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.494+0000 7f47667fc700 1 -- 172.21.15.47:0/1985223587 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47600133f0 con 0x7f47680b2900 2022-06-29T15:51:53.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.494+0000 7f476e5eb700 1 -- 172.21.15.47:0/1985223587 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47680944e0 con 0x7f47680b2900 2022-06-29T15:51:53.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.494+0000 7f47667fc700 1 -- 172.21.15.47:0/1985223587 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4760013750 con 0x7f47680b2900 2022-06-29T15:51:53.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.494+0000 7f476e5eb700 1 -- 172.21.15.47:0/1985223587 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4768065770 con 0x7f47680b2900 2022-06-29T15:51:53.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.494+0000 7f476e5eb700 1 -- 172.21.15.47:0/1985223587 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47680573d0 con 0x7f47680b2900 2022-06-29T15:51:53.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.494+0000 7f47667fc700 1 -- 172.21.15.47:0/1985223587 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f47600138b0 con 0x7f47680b2900 2022-06-29T15:51:53.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.494+0000 7f47667fc700 1 --2- 172.21.15.47:0/1985223587 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4748053980 0x7f4748055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:53.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.494+0000 7f47667fc700 1 -- 172.21.15.47:0/1985223587 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f476006bbc0 con 0x7f47680b2900 2022-06-29T15:51:53.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.494+0000 7f476d5e9700 1 --2- 172.21.15.47:0/1985223587 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4748053980 0x7f4748055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:53.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.498+0000 7f476d5e9700 1 --2- 172.21.15.47:0/1985223587 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4748053980 0x7f4748055e40 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7f475801c000 tx=0x7f4758015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:53.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.502+0000 7f47667fc700 1 -- 172.21.15.47:0/1985223587 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4760042980 con 0x7f47680b2900 2022-06-29T15:51:53.676 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.670+0000 7f476e5eb700 1 -- 172.21.15.47:0/1985223587 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f4768146260 con 0x7f47680b2900 2022-06-29T15:51:53.842 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.834+0000 7f47667fc700 1 -- 172.21.15.47:0/1985223587 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v74) v1 ==== 118+0+0 (secure 0 0 0) 0x7f476000f040 con 0x7f47680b2900 2022-06-29T15:51:53.846 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.838+0000 7f476e5eb700 1 -- 172.21.15.47:0/1985223587 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4748053980 msgr2=0x7f4748055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:53.846 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.838+0000 7f476e5eb700 1 --2- 172.21.15.47:0/1985223587 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4748053980 0x7f4748055e40 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7f475801c000 tx=0x7f4758015000).stop 2022-06-29T15:51:53.846 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.838+0000 7f476e5eb700 1 -- 172.21.15.47:0/1985223587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47680b2900 msgr2=0x7f4768095c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:53.847 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.838+0000 7f476e5eb700 1 --2- 172.21.15.47:0/1985223587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47680b2900 0x7f4768095c20 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 rx=0x7f476000a000 tx=0x7f4760008040).stop 2022-06-29T15:51:53.847 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.838+0000 7f476e5eb700 1 -- 172.21.15.47:0/1985223587 shutdown_connections 2022-06-29T15:51:53.848 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.838+0000 7f476e5eb700 1 --2- 172.21.15.47:0/1985223587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47680b2900 0x7f4768095c20 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:53.848 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.838+0000 7f476e5eb700 1 --2- 172.21.15.47:0/1985223587 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47680b1c60 0x7f476814fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:53.848 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.838+0000 7f476e5eb700 1 --2- 172.21.15.47:0/1985223587 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4748053980 0x7f4748055e40 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:53.848 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.838+0000 7f476e5eb700 1 --2- 172.21.15.47:0/1985223587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f476809a230 0x7f476814f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:53.849 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.838+0000 7f476e5eb700 1 -- 172.21.15.47:0/1985223587 >> 172.21.15.47:0/1985223587 conn(0x7f4768062df0 msgr2=0x7f476809b480 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:53.849 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.838+0000 7f476e5eb700 1 -- 172.21.15.47:0/1985223587 shutdown_connections 2022-06-29T15:51:53.849 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.838+0000 7f476e5eb700 1 -- 172.21.15.47:0/1985223587 wait complete. 2022-06-29T15:51:53.861 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:51:53.976 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.970+0000 7f62e2c11700 1 Processor -- start 2022-06-29T15:51:53.976 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.970+0000 7f62e2c11700 1 -- start start 2022-06-29T15:51:53.978 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.970+0000 7f62e2c11700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f62dc05fa30 0x7f62dc0b1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:53.978 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.970+0000 7f62e2c11700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f62dc060b90 0x7f62dc13f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:53.979 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.970+0000 7f62e2c11700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f62dc0b2770 0x7f62dc0ae170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:53.979 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.970+0000 7f62e2c11700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62dc060fd0 con 0x7f62dc05fa30 2022-06-29T15:51:53.979 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.970+0000 7f62e2c11700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62dc05fe10 con 0x7f62dc0b2770 2022-06-29T15:51:53.979 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.970+0000 7f62e2c11700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62dc0600e0 con 0x7f62dc060b90 2022-06-29T15:51:53.980 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.970+0000 7f62e1c0f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f62dc05fa30 0x7f62dc0b1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:53.980 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.970+0000 7f62e1c0f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f62dc05fa30 0x7f62dc0b1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54434/0 (socket says 172.21.15.47:54434) 2022-06-29T15:51:53.980 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.970+0000 7f62e1c0f700 1 -- 172.21.15.47:0/2426482385 learned_addr learned my addr 172.21.15.47:0/2426482385 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:53.980 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.970+0000 7f62e140e700 1 --2- 172.21.15.47:0/2426482385 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f62dc060b90 0x7f62dc13f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:53.981 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.970+0000 7f62e2410700 1 --2- 172.21.15.47:0/2426482385 >> v2:172.21.15.64:3300/0 conn(0x7f62dc0b2770 0x7f62dc0ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:53.981 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.970+0000 7f62e1c0f700 1 -- 172.21.15.47:0/2426482385 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f62dc060b90 msgr2=0x7f62dc13f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:53.981 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.970+0000 7f62e1c0f700 1 --2- 172.21.15.47:0/2426482385 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f62dc060b90 0x7f62dc13f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:53.982 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.970+0000 7f62e1c0f700 1 -- 172.21.15.47:0/2426482385 >> v2:172.21.15.64:3300/0 conn(0x7f62dc0b2770 msgr2=0x7f62dc0ae170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:53.982 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.970+0000 7f62e1c0f700 1 --2- 172.21.15.47:0/2426482385 >> v2:172.21.15.64:3300/0 conn(0x7f62dc0b2770 0x7f62dc0ae170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:53.982 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.970+0000 7f62e1c0f700 1 -- 172.21.15.47:0/2426482385 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62dc060380 con 0x7f62dc05fa30 2022-06-29T15:51:53.982 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.970+0000 7f62e140e700 1 --2- 172.21.15.47:0/2426482385 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f62dc060b90 0x7f62dc13f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:51:53.983 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.970+0000 7f62e1c0f700 1 --2- 172.21.15.47:0/2426482385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62dc05fa30 0x7f62dc0b1ab0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 rx=0x7f62dc0b2650 tx=0x7f62c8006eb0).ready entity=mon.0 client_cookie=6b60e2899df19be server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:53.983 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.970+0000 7f62e0c0d700 1 -- 172.21.15.47:0/2426482385 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f62c800d570 con 0x7f62dc05fa30 2022-06-29T15:51:53.983 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.970+0000 7f62e0c0d700 1 -- 172.21.15.47:0/2426482385 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c8000e90 con 0x7f62dc05fa30 2022-06-29T15:51:53.983 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.970+0000 7f62e0c0d700 1 -- 172.21.15.47:0/2426482385 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f62c800d8c0 con 0x7f62dc05fa30 2022-06-29T15:51:53.987 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.970+0000 7f62e2c11700 1 -- 172.21.15.47:0/2426482385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62dc05fa30 msgr2=0x7f62dc0b1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:53.987 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.970+0000 7f62e2c11700 1 --2- 172.21.15.47:0/2426482385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62dc05fa30 0x7f62dc0b1ab0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 rx=0x7f62dc0b2650 tx=0x7f62c8006eb0).stop 2022-06-29T15:51:53.988 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.970+0000 7f62e2c11700 1 -- 172.21.15.47:0/2426482385 shutdown_connections 2022-06-29T15:51:53.988 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.970+0000 7f62e2c11700 1 --2- 172.21.15.47:0/2426482385 >> v2:172.21.15.64:3300/0 conn(0x7f62dc0b2770 0x7f62dc0ae170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:53.988 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.970+0000 7f62e2c11700 1 --2- 172.21.15.47:0/2426482385 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f62dc060b90 0x7f62dc13f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:53.988 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.970+0000 7f62e2c11700 1 --2- 172.21.15.47:0/2426482385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62dc05fa30 0x7f62dc0b1ab0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:53.988 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.970+0000 7f62e2c11700 1 -- 172.21.15.47:0/2426482385 >> 172.21.15.47:0/2426482385 conn(0x7f62dc062df0 msgr2=0x7f62dc0631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:53.989 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.974+0000 7f62e2c11700 1 -- 172.21.15.47:0/2426482385 shutdown_connections 2022-06-29T15:51:53.989 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.974+0000 7f62e2c11700 1 -- 172.21.15.47:0/2426482385 wait complete. 2022-06-29T15:51:53.989 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.974+0000 7f62e2c11700 1 Processor -- start 2022-06-29T15:51:53.989 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.974+0000 7f62e2c11700 1 -- start start 2022-06-29T15:51:53.990 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.974+0000 7f62e2c11700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62dc05fa30 0x7f62dc09a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:53.990 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.974+0000 7f62e2c11700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f62dc060b90 0x7f62dc1428a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:53.990 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.974+0000 7f62e2c11700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62dc0b2770 0x7f62dc142d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:53.990 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.974+0000 7f62e2c11700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62dc099730 con 0x7f62dc05fa30 2022-06-29T15:51:53.991 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.974+0000 7f62e2c11700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62dc098da0 con 0x7f62dc0b2770 2022-06-29T15:51:53.991 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.974+0000 7f62e2c11700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62dc099980 con 0x7f62dc060b90 2022-06-29T15:51:53.991 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.974+0000 7f62e1c0f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62dc05fa30 0x7f62dc09a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:53.991 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.974+0000 7f62e1c0f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62dc05fa30 0x7f62dc09a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54440/0 (socket says 172.21.15.47:54440) 2022-06-29T15:51:53.992 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.974+0000 7f62e1c0f700 1 -- 172.21.15.47:0/14686892 learned_addr learned my addr 172.21.15.47:0/14686892 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:53.992 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.974+0000 7f62e2410700 1 --2- 172.21.15.47:0/14686892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62dc0b2770 0x7f62dc142d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:53.992 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.974+0000 7f62e140e700 1 --2- 172.21.15.47:0/14686892 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f62dc060b90 0x7f62dc1428a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:53.992 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.978+0000 7f62e1c0f700 1 -- 172.21.15.47:0/14686892 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f62dc060b90 msgr2=0x7f62dc1428a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:53.993 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.978+0000 7f62e1c0f700 1 --2- 172.21.15.47:0/14686892 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f62dc060b90 0x7f62dc1428a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:53.993 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.978+0000 7f62e1c0f700 1 -- 172.21.15.47:0/14686892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62dc0b2770 msgr2=0x7f62dc142d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:53.993 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.978+0000 7f62e1c0f700 1 --2- 172.21.15.47:0/14686892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62dc0b2770 0x7f62dc142d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:53.993 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.978+0000 7f62e1c0f700 1 -- 172.21.15.47:0/14686892 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62c800b1e0 con 0x7f62dc05fa30 2022-06-29T15:51:53.994 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.978+0000 7f62e1c0f700 1 --2- 172.21.15.47:0/14686892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62dc05fa30 0x7f62dc09a180 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 rx=0x7f62dc0b2650 tx=0x7f62c8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:53.994 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.978+0000 7f62daffd700 1 -- 172.21.15.47:0/14686892 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f62c800bec0 con 0x7f62dc05fa30 2022-06-29T15:51:53.994 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.978+0000 7f62daffd700 1 -- 172.21.15.47:0/14686892 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c800d8f0 con 0x7f62dc05fa30 2022-06-29T15:51:53.994 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.978+0000 7f62e2c11700 1 -- 172.21.15.47:0/14686892 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62dc099000 con 0x7f62dc05fa30 2022-06-29T15:51:53.995 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.978+0000 7f62daffd700 1 -- 172.21.15.47:0/14686892 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f62c8011c30 con 0x7f62dc05fa30 2022-06-29T15:51:53.995 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.978+0000 7f62e2c11700 1 -- 172.21.15.47:0/14686892 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62dc13f520 con 0x7f62dc05fa30 2022-06-29T15:51:53.995 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.978+0000 7f62e2c11700 1 -- 172.21.15.47:0/14686892 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62dc0573d0 con 0x7f62dc05fa30 2022-06-29T15:51:53.995 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.978+0000 7f62daffd700 1 -- 172.21.15.47:0/14686892 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f62c8025030 con 0x7f62dc05fa30 2022-06-29T15:51:53.996 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.978+0000 7f62daffd700 1 --2- 172.21.15.47:0/14686892 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f62b8053950 0x7f62b8055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:53.996 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.978+0000 7f62daffd700 1 -- 172.21.15.47:0/14686892 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f62c806d160 con 0x7f62dc05fa30 2022-06-29T15:51:53.996 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.978+0000 7f62e140e700 1 --2- 172.21.15.47:0/14686892 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f62b8053950 0x7f62b8055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:53.996 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.982+0000 7f62e140e700 1 --2- 172.21.15.47:0/14686892 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f62b8053950 0x7f62b8055e10 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7f62d400d000 tx=0x7f62d4008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:53.997 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:53.986+0000 7f62daffd700 1 -- 172.21.15.47:0/14686892 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f62c8043f20 con 0x7f62dc05fa30 2022-06-29T15:51:54.155 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.150+0000 7f62e2c11700 1 -- 172.21.15.47:0/14686892 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f62dc0b23c0 con 0x7f62dc05fa30 2022-06-29T15:51:54.156 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.150+0000 7f62daffd700 1 -- 172.21.15.47:0/14686892 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+2495 (secure 0 0 0) 0x7f62c8023020 con 0x7f62dc05fa30 2022-06-29T15:51:54.156 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:54.157 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":74,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8296,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/3321939421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3321939421},{"type":"v1","addr":"172.21.15.64:6833","nonce":3321939421}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":72},{"gid":8415,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1886881850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1886881850},{"type":"v1","addr":"172.21.15.47:6835","nonce":1886881850}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":72}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:51:52.828119+0000","modified":"2022-06-29T15:51:53.831305+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8421":{"gid":8421,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6837/2466961065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2466961065},{"type":"v1","addr":"172.21.15.47:6837","nonce":2466961065}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2022-06-29T15:51:54.157 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.150+0000 7f62e2c11700 1 -- 172.21.15.47:0/14686892 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f62b8053950 msgr2=0x7f62b8055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:54.158 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.150+0000 7f62e2c11700 1 --2- 172.21.15.47:0/14686892 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f62b8053950 0x7f62b8055e10 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7f62d400d000 tx=0x7f62d4008040).stop 2022-06-29T15:51:54.158 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.150+0000 7f62e2c11700 1 -- 172.21.15.47:0/14686892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62dc05fa30 msgr2=0x7f62dc09a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:54.158 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.150+0000 7f62e2c11700 1 --2- 172.21.15.47:0/14686892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62dc05fa30 0x7f62dc09a180 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 rx=0x7f62dc0b2650 tx=0x7f62c8015000).stop 2022-06-29T15:51:54.159 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.150+0000 7f62e2c11700 1 -- 172.21.15.47:0/14686892 shutdown_connections 2022-06-29T15:51:54.159 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.150+0000 7f62e2c11700 1 --2- 172.21.15.47:0/14686892 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f62b8053950 0x7f62b8055e10 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:54.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.150+0000 7f62e2c11700 1 --2- 172.21.15.47:0/14686892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62dc0b2770 0x7f62dc142d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:54.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.150+0000 7f62e2c11700 1 --2- 172.21.15.47:0/14686892 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f62dc060b90 0x7f62dc1428a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:54.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.150+0000 7f62e2c11700 1 --2- 172.21.15.47:0/14686892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62dc05fa30 0x7f62dc09a180 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:54.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.150+0000 7f62e2c11700 1 -- 172.21.15.47:0/14686892 >> 172.21.15.47:0/14686892 conn(0x7f62dc062df0 msgr2=0x7f62dc0b4a80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:54.161 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.150+0000 7f62e2c11700 1 -- 172.21.15.47:0/14686892 shutdown_connections 2022-06-29T15:51:54.161 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.150+0000 7f62e2c11700 1 -- 172.21.15.47:0/14686892 wait complete. 2022-06-29T15:51:54.162 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 74 2022-06-29T15:51:54.170 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:51:54.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.266+0000 7f33b9174700 1 Processor -- start 2022-06-29T15:51:54.275 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.266+0000 7f33b9174700 1 -- start start 2022-06-29T15:51:54.277 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.270+0000 7f33b9174700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f33b405c510 0x7f33b4143bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:54.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.270+0000 7f33b9174700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f33b405c8f0 0x7f33b40b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:54.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.270+0000 7f33b9174700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33b4142f90 con 0x7f33b40b2c40 2022-06-29T15:51:54.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.270+0000 7f33b9174700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33b41431f0 con 0x7f33b405c510 2022-06-29T15:51:54.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.270+0000 7f33b9174700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33b4143490 con 0x7f33b405c8f0 2022-06-29T15:51:54.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.270+0000 7f33b3fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f33b405c510 0x7f33b4143bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:54.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.270+0000 7f33b8973700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f33b40b2c40 0x7f33b413cbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40006/0 (socket says 172.21.15.47:40006) 2022-06-29T15:51:54.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.270+0000 7f33b8973700 1 -- 172.21.15.47:0/1027034485 learned_addr learned my addr 172.21.15.47:0/1027034485 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:54.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.270+0000 7f33b37fe700 1 --2- 172.21.15.47:0/1027034485 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f33b405c8f0 0x7f33b40b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:54.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.270+0000 7f33b2ffd700 1 -- 172.21.15.47:0/1027034485 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3965530525 0 0) 0x7f33b4142f90 con 0x7f33b40b2c40 2022-06-29T15:51:54.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.270+0000 7f33b2ffd700 1 -- 172.21.15.47:0/1027034485 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3394003260 con 0x7f33b40b2c40 2022-06-29T15:51:54.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.270+0000 7f33b3fff700 1 -- 172.21.15.47:0/1027034485 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f33b405c8f0 msgr2=0x7f33b40b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:54.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.270+0000 7f33b3fff700 1 --2- 172.21.15.47:0/1027034485 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f33b405c8f0 0x7f33b40b4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:54.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.270+0000 7f33b3fff700 1 -- 172.21.15.47:0/1027034485 >> v1:172.21.15.47:6789/0 conn(0x7f33b40b2c40 legacy=0x7f33b413cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:54.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.270+0000 7f33b3fff700 1 -- 172.21.15.47:0/1027034485 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b4143730 con 0x7f33b405c510 2022-06-29T15:51:54.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.270+0000 7f33b3fff700 1 --2- 172.21.15.47:0/1027034485 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33b405c510 0x7f33b4143bd0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 rx=0x7f33a4002f70 tx=0x7f33a4006df0).ready entity=mon.0 client_cookie=b9ef199b193b071 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:54.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.270+0000 7f33b2ffd700 1 -- 172.21.15.47:0/1027034485 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33a400d4a0 con 0x7f33b405c510 2022-06-29T15:51:54.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.270+0000 7f33b2ffd700 1 -- 172.21.15.47:0/1027034485 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33a4000e90 con 0x7f33b405c510 2022-06-29T15:51:54.282 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.270+0000 7f33b2ffd700 1 -- 172.21.15.47:0/1027034485 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33a400d7f0 con 0x7f33b405c510 2022-06-29T15:51:54.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.270+0000 7f33b9174700 1 -- 172.21.15.47:0/1027034485 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33b405c510 msgr2=0x7f33b4143bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:54.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.270+0000 7f33b9174700 1 --2- 172.21.15.47:0/1027034485 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33b405c510 0x7f33b4143bd0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 rx=0x7f33a4002f70 tx=0x7f33a4006df0).stop 2022-06-29T15:51:54.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.270+0000 7f33b9174700 1 -- 172.21.15.47:0/1027034485 shutdown_connections 2022-06-29T15:51:54.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.270+0000 7f33b9174700 1 --2- 172.21.15.47:0/1027034485 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f33b405c8f0 0x7f33b40b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:54.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.270+0000 7f33b9174700 1 --2- 172.21.15.47:0/1027034485 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33b405c510 0x7f33b4143bd0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:54.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.270+0000 7f33b9174700 1 -- 172.21.15.47:0/1027034485 >> 172.21.15.47:0/1027034485 conn(0x7f33b4060e00 msgr2=0x7f33b40611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:54.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.270+0000 7f33b9174700 1 -- 172.21.15.47:0/1027034485 shutdown_connections 2022-06-29T15:51:54.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.270+0000 7f33b9174700 1 -- 172.21.15.47:0/1027034485 wait complete. 2022-06-29T15:51:54.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.274+0000 7f33b9174700 1 Processor -- start 2022-06-29T15:51:54.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.274+0000 7f33b9174700 1 -- start start 2022-06-29T15:51:54.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.274+0000 7f33b9174700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33b405c510 0x7f33b4146aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:54.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.274+0000 7f33b9174700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f33b405c8f0 0x7f33b40ad790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:54.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.274+0000 7f33b9174700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33b40b2c40 0x7f33b41453a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:54.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.274+0000 7f33b9174700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b40afc80 con 0x7f33b40b2c40 2022-06-29T15:51:54.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.274+0000 7f33b9174700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b40b07d0 con 0x7f33b405c510 2022-06-29T15:51:54.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.274+0000 7f33b9174700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33b41561e0 con 0x7f33b405c8f0 2022-06-29T15:51:54.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.274+0000 7f33b8973700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33b40b2c40 0x7f33b41453a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:54.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.274+0000 7f33b8973700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33b40b2c40 0x7f33b41453a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54456/0 (socket says 172.21.15.47:54456) 2022-06-29T15:51:54.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.274+0000 7f33b8973700 1 -- 172.21.15.47:0/3601371008 learned_addr learned my addr 172.21.15.47:0/3601371008 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:54.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.274+0000 7f33b3fff700 1 --2- 172.21.15.47:0/3601371008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33b405c510 0x7f33b4146aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:54.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.274+0000 7f33b37fe700 1 --2- 172.21.15.47:0/3601371008 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f33b405c8f0 0x7f33b40ad790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:54.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.274+0000 7f33b8973700 1 -- 172.21.15.47:0/3601371008 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f33b405c8f0 msgr2=0x7f33b40ad790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:54.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.274+0000 7f33b8973700 1 --2- 172.21.15.47:0/3601371008 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f33b405c8f0 0x7f33b40ad790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:54.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.274+0000 7f33b8973700 1 -- 172.21.15.47:0/3601371008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33b405c510 msgr2=0x7f33b4146aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:54.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.274+0000 7f33b8973700 1 --2- 172.21.15.47:0/3601371008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33b405c510 0x7f33b4146aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:54.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.274+0000 7f33b8973700 1 -- 172.21.15.47:0/3601371008 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a400b180 con 0x7f33b40b2c40 2022-06-29T15:51:54.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.274+0000 7f33b3fff700 1 --2- 172.21.15.47:0/3601371008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33b405c510 0x7f33b4146aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:51:54.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.274+0000 7f33b37fe700 1 --2- 172.21.15.47:0/3601371008 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f33b405c8f0 0x7f33b40ad790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:51:54.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.274+0000 7f33b8973700 1 --2- 172.21.15.47:0/3601371008 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33b40b2c40 0x7f33b41453a0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 rx=0x7f33ac001490 tx=0x7f33ac007dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:54.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.274+0000 7f33b17fa700 1 -- 172.21.15.47:0/3601371008 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33ac005ba0 con 0x7f33b40b2c40 2022-06-29T15:51:54.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.274+0000 7f33b17fa700 1 -- 172.21.15.47:0/3601371008 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33ac005d00 con 0x7f33b40b2c40 2022-06-29T15:51:54.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.274+0000 7f33b9174700 1 -- 172.21.15.47:0/3601371008 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33b40b04c0 con 0x7f33b40b2c40 2022-06-29T15:51:54.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.274+0000 7f33b17fa700 1 -- 172.21.15.47:0/3601371008 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33ac0072c0 con 0x7f33b40b2c40 2022-06-29T15:51:54.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.278+0000 7f33b9174700 1 -- 172.21.15.47:0/3601371008 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33b4140760 con 0x7f33b40b2c40 2022-06-29T15:51:54.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.278+0000 7f33b9174700 1 -- 172.21.15.47:0/3601371008 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33b4156020 con 0x7f33b40b2c40 2022-06-29T15:51:54.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.278+0000 7f33b17fa700 1 -- 172.21.15.47:0/3601371008 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f33ac010070 con 0x7f33b40b2c40 2022-06-29T15:51:54.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.278+0000 7f33b17fa700 1 --2- 172.21.15.47:0/3601371008 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3398053aa0 0x7f3398055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:54.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.278+0000 7f33b17fa700 1 -- 172.21.15.47:0/3601371008 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f33ac06b9e0 con 0x7f33b40b2c40 2022-06-29T15:51:54.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.278+0000 7f33b3fff700 1 --2- 172.21.15.47:0/3601371008 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3398053aa0 0x7f3398055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:54.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.278+0000 7f33b3fff700 1 --2- 172.21.15.47:0/3601371008 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3398053aa0 0x7f3398055f60 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7f33a4018000 tx=0x7f33a4014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:54.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.286+0000 7f33b17fa700 1 -- 172.21.15.47:0/3601371008 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f33ac0427a0 con 0x7f33b40b2c40 2022-06-29T15:51:54.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.414+0000 7f33b9174700 1 -- 172.21.15.47:0/3601371008 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f33b40b0980 con 0x7f33b40b2c40 2022-06-29T15:51:54.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.414+0000 7f33b17fa700 1 -- 172.21.15.47:0/3601371008 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f33ac045dc0 con 0x7f33b40b2c40 2022-06-29T15:51:54.422 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:54.423 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":151,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:51:53.832186+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:51:50.583017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2022-06-29T15:51:51.834830+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:51:54.424 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.418+0000 7f33b9174700 1 -- 172.21.15.47:0/3601371008 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3398053aa0 msgr2=0x7f3398055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:54.424 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.418+0000 7f33b9174700 1 --2- 172.21.15.47:0/3601371008 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3398053aa0 0x7f3398055f60 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7f33a4018000 tx=0x7f33a4014000).stop 2022-06-29T15:51:54.425 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.418+0000 7f33b9174700 1 -- 172.21.15.47:0/3601371008 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33b40b2c40 msgr2=0x7f33b41453a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:54.425 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.418+0000 7f33b9174700 1 --2- 172.21.15.47:0/3601371008 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33b40b2c40 0x7f33b41453a0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 rx=0x7f33ac001490 tx=0x7f33ac007dc0).stop 2022-06-29T15:51:54.425 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.418+0000 7f33b9174700 1 -- 172.21.15.47:0/3601371008 shutdown_connections 2022-06-29T15:51:54.425 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.418+0000 7f33b9174700 1 --2- 172.21.15.47:0/3601371008 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33b40b2c40 0x7f33b41453a0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:54.426 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.418+0000 7f33b9174700 1 --2- 172.21.15.47:0/3601371008 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f33b405c8f0 0x7f33b40ad790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:54.426 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.418+0000 7f33b9174700 1 --2- 172.21.15.47:0/3601371008 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3398053aa0 0x7f3398055f60 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:54.426 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.418+0000 7f33b9174700 1 --2- 172.21.15.47:0/3601371008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33b405c510 0x7f33b4146aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:54.426 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.418+0000 7f33b9174700 1 -- 172.21.15.47:0/3601371008 >> 172.21.15.47:0/3601371008 conn(0x7f33b4060e00 msgr2=0x7f33b40b65c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:54.427 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.418+0000 7f33b9174700 1 -- 172.21.15.47:0/3601371008 shutdown_connections 2022-06-29T15:51:54.427 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.418+0000 7f33b9174700 1 -- 172.21.15.47:0/3601371008 wait complete. 2022-06-29T15:51:54.436 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-29T15:51:54.555 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.546+0000 7f118aae7700 1 Processor -- start 2022-06-29T15:51:54.555 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.550+0000 7f118aae7700 1 -- start start 2022-06-29T15:51:54.558 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.550+0000 7f118aae7700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f11840afcd0 0x7f11840acff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:54.558 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.550+0000 7f118aae7700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1184099760 con 0x7f1184098d10 2022-06-29T15:51:54.558 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.550+0000 7f118aae7700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f118409b5a0 con 0x7f11840b2d70 2022-06-29T15:51:54.558 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.550+0000 7f118aae7700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1184099220 con 0x7f11840afcd0 2022-06-29T15:51:54.559 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.550+0000 7f11892e4700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1184098d10 0x7f11840af5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40020/0 (socket says 172.21.15.47:40020) 2022-06-29T15:51:54.559 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.550+0000 7f11892e4700 1 -- 172.21.15.47:0/3625909694 learned_addr learned my addr 172.21.15.47:0/3625909694 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:54.559 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.550+0000 7f118a2e6700 1 --2- 172.21.15.47:0/3625909694 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f11840afcd0 0x7f11840acff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:54.559 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.550+0000 7f1188ae3700 1 -- 172.21.15.47:0/3625909694 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2759121599 0 0) 0x7f1184099760 con 0x7f1184098d10 2022-06-29T15:51:54.560 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.550+0000 7f1188ae3700 1 -- 172.21.15.47:0/3625909694 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f116c003260 con 0x7f1184098d10 2022-06-29T15:51:54.560 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.550+0000 7f1188ae3700 1 -- 172.21.15.47:0/3625909694 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3777238931 0 0) 0x7f118409b5a0 con 0x7f11840b2d70 2022-06-29T15:51:54.560 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.550+0000 7f1188ae3700 1 -- 172.21.15.47:0/3625909694 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1184099760 con 0x7f11840b2d70 2022-06-29T15:51:54.560 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.550+0000 7f118a2e6700 1 -- 172.21.15.47:0/3625909694 >> v1:172.21.15.64:6789/0 conn(0x7f11840b2d70 legacy=0x7f118409ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:54.561 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.550+0000 7f118a2e6700 1 -- 172.21.15.47:0/3625909694 >> v1:172.21.15.47:6789/0 conn(0x7f1184098d10 legacy=0x7f11840af5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:54.561 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.550+0000 7f118a2e6700 1 -- 172.21.15.47:0/3625909694 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11840994c0 con 0x7f11840afcd0 2022-06-29T15:51:54.561 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.550+0000 7f118a2e6700 1 --2- 172.21.15.47:0/3625909694 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f11840afcd0 0x7f11840acff0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f117c0032f0 tx=0x7f117c0091d0).ready entity=mon.2 client_cookie=835e13e19d6075fa server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:54.561 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.550+0000 7f1188ae3700 1 -- 172.21.15.47:0/3625909694 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f117c00b640 con 0x7f11840afcd0 2022-06-29T15:51:54.562 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.550+0000 7f1188ae3700 1 -- 172.21.15.47:0/3625909694 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f117c00b7a0 con 0x7f11840afcd0 2022-06-29T15:51:54.562 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.550+0000 7f1188ae3700 1 -- 172.21.15.47:0/3625909694 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f117c00b910 con 0x7f11840afcd0 2022-06-29T15:51:54.565 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.550+0000 7f118aae7700 1 -- 172.21.15.47:0/3625909694 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f11840afcd0 msgr2=0x7f11840acff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:54.565 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.550+0000 7f118aae7700 1 --2- 172.21.15.47:0/3625909694 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f11840afcd0 0x7f11840acff0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f117c0032f0 tx=0x7f117c0091d0).stop 2022-06-29T15:51:54.565 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.550+0000 7f118aae7700 1 -- 172.21.15.47:0/3625909694 shutdown_connections 2022-06-29T15:51:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.550+0000 7f118aae7700 1 --2- 172.21.15.47:0/3625909694 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f11840afcd0 0x7f11840acff0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.550+0000 7f118aae7700 1 -- 172.21.15.47:0/3625909694 >> 172.21.15.47:0/3625909694 conn(0x7f1184062df0 msgr2=0x7f11840631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.550+0000 7f118aae7700 1 -- 172.21.15.47:0/3625909694 shutdown_connections 2022-06-29T15:51:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.550+0000 7f118aae7700 1 -- 172.21.15.47:0/3625909694 wait complete. 2022-06-29T15:51:54.567 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.554+0000 7f118aae7700 1 Processor -- start 2022-06-29T15:51:54.567 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.554+0000 7f118aae7700 1 -- start start 2022-06-29T15:51:54.567 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.554+0000 7f118aae7700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1184098d10 0x7f11840b3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:54.567 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.554+0000 7f118aae7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11840afcd0 0x7f11840aaa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:54.568 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.554+0000 7f118aae7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11840b2d70 0x7f11840aaf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:54.568 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.554+0000 7f118aae7700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1184064410 con 0x7f11840b2d70 2022-06-29T15:51:54.568 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.554+0000 7f118aae7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11840ae490 con 0x7f11840afcd0 2022-06-29T15:51:54.568 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.554+0000 7f118aae7700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11840ad570 con 0x7f1184098d10 2022-06-29T15:51:54.569 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.554+0000 7f118a2e6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11840b2d70 0x7f11840aaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:54.569 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.554+0000 7f118a2e6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11840b2d70 0x7f11840aaf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54472/0 (socket says 172.21.15.47:54472) 2022-06-29T15:51:54.569 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.554+0000 7f118a2e6700 1 -- 172.21.15.47:0/2305868510 learned_addr learned my addr 172.21.15.47:0/2305868510 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:54.569 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.554+0000 7f11892e4700 1 --2- 172.21.15.47:0/2305868510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11840afcd0 0x7f11840aaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:54.570 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.554+0000 7f1189ae5700 1 --2- 172.21.15.47:0/2305868510 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1184098d10 0x7f11840b3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:54.570 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.554+0000 7f118a2e6700 1 -- 172.21.15.47:0/2305868510 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1184098d10 msgr2=0x7f11840b3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:54.570 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.554+0000 7f118a2e6700 1 --2- 172.21.15.47:0/2305868510 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1184098d10 0x7f11840b3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:54.570 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.554+0000 7f118a2e6700 1 -- 172.21.15.47:0/2305868510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11840afcd0 msgr2=0x7f11840aaa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:54.571 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.554+0000 7f118a2e6700 1 --2- 172.21.15.47:0/2305868510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11840afcd0 0x7f11840aaa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:54.571 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.554+0000 7f118a2e6700 1 -- 172.21.15.47:0/2305868510 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f117c009470 con 0x7f11840b2d70 2022-06-29T15:51:54.571 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.554+0000 7f118a2e6700 1 --2- 172.21.15.47:0/2305868510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11840b2d70 0x7f11840aaf00 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 rx=0x7f11840ae5d0 tx=0x7f117c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:54.571 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.558+0000 7f1172ffd700 1 -- 172.21.15.47:0/2305868510 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f117c00b910 con 0x7f11840b2d70 2022-06-29T15:51:54.572 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.558+0000 7f1172ffd700 1 -- 172.21.15.47:0/2305868510 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f117c011da0 con 0x7f11840b2d70 2022-06-29T15:51:54.572 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.558+0000 7f118aae7700 1 -- 172.21.15.47:0/2305868510 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11840ad810 con 0x7f11840b2d70 2022-06-29T15:51:54.572 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.558+0000 7f1172ffd700 1 -- 172.21.15.47:0/2305868510 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f117c00b640 con 0x7f11840b2d70 2022-06-29T15:51:54.572 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.558+0000 7f118aae7700 1 -- 172.21.15.47:0/2305868510 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11840adff0 con 0x7f11840b2d70 2022-06-29T15:51:54.574 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.558+0000 7f118aae7700 1 -- 172.21.15.47:0/2305868510 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11840ae1b0 con 0x7f11840b2d70 2022-06-29T15:51:54.574 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.558+0000 7f1172ffd700 1 -- 172.21.15.47:0/2305868510 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f117c00f070 con 0x7f11840b2d70 2022-06-29T15:51:54.574 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.562+0000 7f1172ffd700 1 --2- 172.21.15.47:0/2305868510 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1164053a70 0x7f1164055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:54.574 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.562+0000 7f1172ffd700 1 -- 172.21.15.47:0/2305868510 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f117c06cb10 con 0x7f11840b2d70 2022-06-29T15:51:54.575 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.562+0000 7f1189ae5700 1 --2- 172.21.15.47:0/2305868510 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1164053a70 0x7f1164055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:54.575 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.562+0000 7f1189ae5700 1 --2- 172.21.15.47:0/2305868510 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1164053a70 0x7f1164055f30 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7f118000d000 tx=0x7f1180007a00).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:54.575 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.570+0000 7f1172ffd700 1 -- 172.21.15.47:0/2305868510 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f117c043870 con 0x7f11840b2d70 2022-06-29T15:51:54.748 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.742+0000 7f118aae7700 1 -- 172.21.15.47:0/2305868510 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f11840573d0 con 0x7f11840b2d70 2022-06-29T15:51:54.755 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.750+0000 7f1172ffd700 1 -- 172.21.15.47:0/2305868510 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v35) v1 ==== 184+0+0 (secure 0 0 0) 0x7f117c015070 con 0x7f11840b2d70 2022-06-29T15:51:54.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.750+0000 7f118aae7700 1 -- 172.21.15.47:0/2305868510 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1164053a70 msgr2=0x7f1164055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:54.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.750+0000 7f118aae7700 1 --2- 172.21.15.47:0/2305868510 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1164053a70 0x7f1164055f30 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7f118000d000 tx=0x7f1180007a00).stop 2022-06-29T15:51:54.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.750+0000 7f118aae7700 1 -- 172.21.15.47:0/2305868510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11840b2d70 msgr2=0x7f11840aaf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:54.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.750+0000 7f118aae7700 1 --2- 172.21.15.47:0/2305868510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11840b2d70 0x7f11840aaf00 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 rx=0x7f11840ae5d0 tx=0x7f117c016000).stop 2022-06-29T15:51:54.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.750+0000 7f118aae7700 1 -- 172.21.15.47:0/2305868510 shutdown_connections 2022-06-29T15:51:54.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.750+0000 7f118aae7700 1 --2- 172.21.15.47:0/2305868510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11840b2d70 0x7f11840aaf00 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:54.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.750+0000 7f118aae7700 1 --2- 172.21.15.47:0/2305868510 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1164053a70 0x7f1164055f30 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:54.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.750+0000 7f118aae7700 1 --2- 172.21.15.47:0/2305868510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11840afcd0 0x7f11840aaa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:54.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.750+0000 7f118aae7700 1 --2- 172.21.15.47:0/2305868510 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1184098d10 0x7f11840b3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:54.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.750+0000 7f118aae7700 1 -- 172.21.15.47:0/2305868510 >> 172.21.15.47:0/2305868510 conn(0x7f1184062df0 msgr2=0x7f11840b4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:54.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.750+0000 7f118aae7700 1 -- 172.21.15.47:0/2305868510 shutdown_connections 2022-06-29T15:51:54.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.750+0000 7f118aae7700 1 -- 172.21.15.47:0/2305868510 wait complete. 2022-06-29T15:51:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.0 2022-06-29T15:51:54.770 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-29T15:51:54.880 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.874+0000 7f2da2d7f700 1 Processor -- start 2022-06-29T15:51:54.880 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.874+0000 7f2da2d7f700 1 -- start start 2022-06-29T15:51:54.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.874+0000 7f2da2d7f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2d9c0b4300 0x7f2d9c0612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:54.883 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.874+0000 7f2da2d7f700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d9c067610 con 0x7f2d9c1471a0 2022-06-29T15:51:54.883 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.874+0000 7f2da2d7f700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d9c1467c0 con 0x7f2d9c060ee0 2022-06-29T15:51:54.883 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.874+0000 7f2da2d7f700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d9c146a60 con 0x7f2d9c0b4300 2022-06-29T15:51:54.883 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.874+0000 7f2da131c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f2d9c1471a0 0x7f2d9c13cdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40032/0 (socket says 172.21.15.47:40032) 2022-06-29T15:51:54.884 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.874+0000 7f2da131c700 1 -- 172.21.15.47:0/2308706096 learned_addr learned my addr 172.21.15.47:0/2308706096 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:54.884 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.874+0000 7f2d93fff700 1 --2- 172.21.15.47:0/2308706096 >> v2:172.21.15.64:3300/0 conn(0x7f2d9c0b4300 0x7f2d9c0612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:54.884 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.874+0000 7f2d937fe700 1 -- 172.21.15.47:0/2308706096 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1878013044 0 0) 0x7f2d9c067610 con 0x7f2d9c1471a0 2022-06-29T15:51:54.884 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.874+0000 7f2d937fe700 1 -- 172.21.15.47:0/2308706096 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d88003260 con 0x7f2d9c1471a0 2022-06-29T15:51:54.885 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.874+0000 7f2d93fff700 1 -- 172.21.15.47:0/2308706096 >> v1:172.21.15.64:6789/0 conn(0x7f2d9c060ee0 legacy=0x7f2d9c1430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:54.885 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.874+0000 7f2d93fff700 1 -- 172.21.15.47:0/2308706096 >> v1:172.21.15.47:6789/0 conn(0x7f2d9c1471a0 legacy=0x7f2d9c13cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:54.885 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.874+0000 7f2d93fff700 1 -- 172.21.15.47:0/2308706096 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d9c146d00 con 0x7f2d9c0b4300 2022-06-29T15:51:54.885 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.874+0000 7f2d93fff700 1 --2- 172.21.15.47:0/2308706096 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d9c0b4300 0x7f2d9c0612c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7f2d9c0adb80 tx=0x7f2d8c006df0).ready entity=mon.1 client_cookie=2c34c7c2497ab4b8 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:54.886 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.874+0000 7f2d937fe700 1 -- 172.21.15.47:0/2308706096 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d8c00d610 con 0x7f2d9c0b4300 2022-06-29T15:51:54.886 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.874+0000 7f2d937fe700 1 -- 172.21.15.47:0/2308706096 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d8c000e90 con 0x7f2d9c0b4300 2022-06-29T15:51:54.886 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.874+0000 7f2d937fe700 1 -- 172.21.15.47:0/2308706096 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d8c00d960 con 0x7f2d9c0b4300 2022-06-29T15:51:54.890 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.878+0000 7f2da2d7f700 1 -- 172.21.15.47:0/2308706096 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d9c0b4300 msgr2=0x7f2d9c0612c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:54.891 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.878+0000 7f2da2d7f700 1 --2- 172.21.15.47:0/2308706096 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d9c0b4300 0x7f2d9c0612c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7f2d9c0adb80 tx=0x7f2d8c006df0).stop 2022-06-29T15:51:54.891 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.878+0000 7f2da2d7f700 1 -- 172.21.15.47:0/2308706096 shutdown_connections 2022-06-29T15:51:54.891 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.878+0000 7f2da2d7f700 1 --2- 172.21.15.47:0/2308706096 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d9c0b4300 0x7f2d9c0612c0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:54.891 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.878+0000 7f2da2d7f700 1 -- 172.21.15.47:0/2308706096 >> 172.21.15.47:0/2308706096 conn(0x7f2d9c05c510 msgr2=0x7f2d9c05f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:54.892 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.878+0000 7f2da2d7f700 1 -- 172.21.15.47:0/2308706096 shutdown_connections 2022-06-29T15:51:54.892 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.878+0000 7f2da2d7f700 1 -- 172.21.15.47:0/2308706096 wait complete. 2022-06-29T15:51:54.892 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.878+0000 7f2da2d7f700 1 Processor -- start 2022-06-29T15:51:54.893 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.878+0000 7f2da2d7f700 1 -- start start 2022-06-29T15:51:54.893 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.882+0000 7f2da2d7f700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2d9c060ee0 0x7f2d9c0b86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:54.893 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.882+0000 7f2da2d7f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d9c0b4300 0x7f2d9c13fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:54.893 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.882+0000 7f2da2d7f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d9c1471a0 0x7f2d9c13ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:54.894 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.882+0000 7f2da2d7f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9c0b0cd0 con 0x7f2d9c0b4300 2022-06-29T15:51:54.894 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.882+0000 7f2da2d7f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9c1532b0 con 0x7f2d9c1471a0 2022-06-29T15:51:54.894 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.882+0000 7f2da2d7f700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9c152da0 con 0x7f2d9c060ee0 2022-06-29T15:51:54.894 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.882+0000 7f2d93fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d9c0b4300 0x7f2d9c13fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:54.895 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.882+0000 7f2d93fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d9c0b4300 0x7f2d9c13fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54484/0 (socket says 172.21.15.47:54484) 2022-06-29T15:51:54.895 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.882+0000 7f2d93fff700 1 -- 172.21.15.47:0/4269380848 learned_addr learned my addr 172.21.15.47:0/4269380848 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:54.895 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.882+0000 7f2da131c700 1 --2- 172.21.15.47:0/4269380848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d9c1471a0 0x7f2d9c13ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:54.895 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.882+0000 7f2da0b1b700 1 --2- 172.21.15.47:0/4269380848 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2d9c060ee0 0x7f2d9c0b86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:54.895 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.882+0000 7f2d93fff700 1 -- 172.21.15.47:0/4269380848 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2d9c060ee0 msgr2=0x7f2d9c0b86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:54.896 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.882+0000 7f2d93fff700 1 --2- 172.21.15.47:0/4269380848 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2d9c060ee0 0x7f2d9c0b86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:54.896 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.882+0000 7f2d93fff700 1 -- 172.21.15.47:0/4269380848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d9c1471a0 msgr2=0x7f2d9c13ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:54.896 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.882+0000 7f2d93fff700 1 --2- 172.21.15.47:0/4269380848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d9c1471a0 0x7f2d9c13ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:54.896 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.882+0000 7f2d93fff700 1 -- 172.21.15.47:0/4269380848 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d8c00b180 con 0x7f2d9c0b4300 2022-06-29T15:51:54.897 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.882+0000 7f2d93fff700 1 --2- 172.21.15.47:0/4269380848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d9c0b4300 0x7f2d9c13fae0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 rx=0x7f2d9c152f30 tx=0x7f2d8c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:54.897 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.882+0000 7f2d91ffb700 1 -- 172.21.15.47:0/4269380848 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d8c016070 con 0x7f2d9c0b4300 2022-06-29T15:51:54.897 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.882+0000 7f2d91ffb700 1 -- 172.21.15.47:0/4269380848 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d8c00de90 con 0x7f2d9c0b4300 2022-06-29T15:51:54.898 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.882+0000 7f2d91ffb700 1 -- 172.21.15.47:0/4269380848 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d8c013e10 con 0x7f2d9c0b4300 2022-06-29T15:51:54.898 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.882+0000 7f2da2d7f700 1 -- 172.21.15.47:0/4269380848 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d9c1404c0 con 0x7f2d9c0b4300 2022-06-29T15:51:54.898 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.882+0000 7f2da2d7f700 1 -- 172.21.15.47:0/4269380848 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d9c141090 con 0x7f2d9c0b4300 2022-06-29T15:51:54.900 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.886+0000 7f2da2d7f700 1 -- 172.21.15.47:0/4269380848 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d9c142c80 con 0x7f2d9c0b4300 2022-06-29T15:51:54.900 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.886+0000 7f2d91ffb700 1 -- 172.21.15.47:0/4269380848 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2d8c00f030 con 0x7f2d9c0b4300 2022-06-29T15:51:54.900 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.886+0000 7f2d91ffb700 1 --2- 172.21.15.47:0/4269380848 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2d7c053b30 0x7f2d7c055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:54.900 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.886+0000 7f2d91ffb700 1 -- 172.21.15.47:0/4269380848 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f2d8c06cef0 con 0x7f2d9c0b4300 2022-06-29T15:51:54.901 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.886+0000 7f2da0b1b700 1 --2- 172.21.15.47:0/4269380848 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2d7c053b30 0x7f2d7c055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:54.901 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.886+0000 7f2da0b1b700 1 --2- 172.21.15.47:0/4269380848 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2d7c053b30 0x7f2d7c055ff0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7f2d9800b000 tx=0x7f2d98009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:54.901 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:54.890+0000 7f2d91ffb700 1 -- 172.21.15.47:0/4269380848 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2d8c043b20 con 0x7f2d9c0b4300 2022-06-29T15:51:55.071 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.062+0000 7f2da2d7f700 1 -- 172.21.15.47:0/4269380848 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f2d9c0573d0 con 0x7f2d9c0b4300 2022-06-29T15:51:55.079 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.074+0000 7f2d91ffb700 1 -- 172.21.15.47:0/4269380848 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v36) v1 ==== 184+0+0 (secure 0 0 0) 0x7f2d8c023020 con 0x7f2d9c0b4300 2022-06-29T15:51:55.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.074+0000 7f2da2d7f700 1 -- 172.21.15.47:0/4269380848 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2d7c053b30 msgr2=0x7f2d7c055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:55.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.074+0000 7f2da2d7f700 1 --2- 172.21.15.47:0/4269380848 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2d7c053b30 0x7f2d7c055ff0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7f2d9800b000 tx=0x7f2d98009040).stop 2022-06-29T15:51:55.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.074+0000 7f2da2d7f700 1 -- 172.21.15.47:0/4269380848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d9c0b4300 msgr2=0x7f2d9c13fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:55.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.074+0000 7f2da2d7f700 1 --2- 172.21.15.47:0/4269380848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d9c0b4300 0x7f2d9c13fae0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 rx=0x7f2d9c152f30 tx=0x7f2d8c014000).stop 2022-06-29T15:51:55.084 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.074+0000 7f2da2d7f700 1 -- 172.21.15.47:0/4269380848 shutdown_connections 2022-06-29T15:51:55.084 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.074+0000 7f2da2d7f700 1 --2- 172.21.15.47:0/4269380848 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2d7c053b30 0x7f2d7c055ff0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:55.084 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.074+0000 7f2da2d7f700 1 --2- 172.21.15.47:0/4269380848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d9c1471a0 0x7f2d9c13ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:55.084 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.074+0000 7f2da2d7f700 1 --2- 172.21.15.47:0/4269380848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d9c0b4300 0x7f2d9c13fae0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:55.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.074+0000 7f2da2d7f700 1 --2- 172.21.15.47:0/4269380848 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2d9c060ee0 0x7f2d9c0b86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:55.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.074+0000 7f2da2d7f700 1 -- 172.21.15.47:0/4269380848 >> 172.21.15.47:0/4269380848 conn(0x7f2d9c05c510 msgr2=0x7f2d9c066d70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:55.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.074+0000 7f2da2d7f700 1 -- 172.21.15.47:0/4269380848 shutdown_connections 2022-06-29T15:51:55.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.074+0000 7f2da2d7f700 1 -- 172.21.15.47:0/4269380848 wait complete. 2022-06-29T15:51:55.086 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.1 2022-06-29T15:51:55.096 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:51:55.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.190+0000 7fa29a5a1700 1 Processor -- start 2022-06-29T15:51:55.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.190+0000 7fa29a5a1700 1 -- start start 2022-06-29T15:51:55.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.190+0000 7fa29a5a1700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa2940b2120 0x7fa2940b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:55.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.190+0000 7fa29a5a1700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2940b0740 con 0x7fa29405f0b0 2022-06-29T15:51:55.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.190+0000 7fa29a5a1700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2940b0970 con 0x7fa2940b1260 2022-06-29T15:51:55.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.190+0000 7fa29a5a1700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa294147460 con 0x7fa2940b2120 2022-06-29T15:51:55.200 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.190+0000 7fa2937fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa29405f0b0 0x7fa294147af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40044/0 (socket says 172.21.15.47:40044) 2022-06-29T15:51:55.200 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.190+0000 7fa2937fe700 1 -- 172.21.15.47:0/2964677900 learned_addr learned my addr 172.21.15.47:0/2964677900 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:55.200 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.190+0000 7fa293fff700 1 --2- 172.21.15.47:0/2964677900 >> v2:172.21.15.47:3300/0 conn(0x7fa2940b2120 0x7fa2940b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:55.200 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.190+0000 7fa292ffd700 1 -- 172.21.15.47:0/2964677900 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4178976194 0 0) 0x7fa2940b0740 con 0x7fa29405f0b0 2022-06-29T15:51:55.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.190+0000 7fa292ffd700 1 -- 172.21.15.47:0/2964677900 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa274003260 con 0x7fa29405f0b0 2022-06-29T15:51:55.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.190+0000 7fa293fff700 1 -- 172.21.15.47:0/2964677900 >> v1:172.21.15.64:6789/0 conn(0x7fa2940b1260 legacy=0x7fa29405f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:55.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.190+0000 7fa293fff700 1 -- 172.21.15.47:0/2964677900 >> v1:172.21.15.47:6789/0 conn(0x7fa29405f0b0 legacy=0x7fa294147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:55.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.190+0000 7fa292ffd700 1 -- 172.21.15.47:0/2964677900 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3574497197 0 0) 0x7fa274003260 con 0x7fa29405f0b0 2022-06-29T15:51:55.202 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.190+0000 7fa293fff700 1 -- 172.21.15.47:0/2964677900 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2940b0c10 con 0x7fa2940b2120 2022-06-29T15:51:55.202 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.190+0000 7fa293fff700 1 --2- 172.21.15.47:0/2964677900 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2940b2120 0x7fa2940b2500 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 rx=0x7fa294141a70 tx=0x7fa284006df0).ready entity=mon.0 client_cookie=192f216165b4e62b server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:55.202 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.190+0000 7fa292ffd700 1 -- 172.21.15.47:0/2964677900 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa27c001660 con 0x7fa2940b2120 2022-06-29T15:51:55.202 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.190+0000 7fa292ffd700 1 -- 172.21.15.47:0/2964677900 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa284000e90 con 0x7fa2940b2120 2022-06-29T15:51:55.203 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.190+0000 7fa292ffd700 1 -- 172.21.15.47:0/2964677900 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa28400d4f0 con 0x7fa2940b2120 2022-06-29T15:51:55.203 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.190+0000 7fa29a5a1700 1 -- 172.21.15.47:0/2964677900 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2940b2120 msgr2=0x7fa2940b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:55.203 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.190+0000 7fa29a5a1700 1 --2- 172.21.15.47:0/2964677900 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2940b2120 0x7fa2940b2500 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 rx=0x7fa294141a70 tx=0x7fa284006df0).stop 2022-06-29T15:51:55.203 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.190+0000 7fa29a5a1700 1 -- 172.21.15.47:0/2964677900 shutdown_connections 2022-06-29T15:51:55.203 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.190+0000 7fa29a5a1700 1 --2- 172.21.15.47:0/2964677900 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2940b2120 0x7fa2940b2500 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:55.204 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.190+0000 7fa29a5a1700 1 -- 172.21.15.47:0/2964677900 >> 172.21.15.47:0/2964677900 conn(0x7fa2940626f0 msgr2=0x7fa294062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:55.204 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.190+0000 7fa29a5a1700 1 -- 172.21.15.47:0/2964677900 shutdown_connections 2022-06-29T15:51:55.204 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.190+0000 7fa29a5a1700 1 -- 172.21.15.47:0/2964677900 wait complete. 2022-06-29T15:51:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.194+0000 7fa29a5a1700 1 Processor -- start 2022-06-29T15:51:55.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.194+0000 7fa29a5a1700 1 -- start start 2022-06-29T15:51:55.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.194+0000 7fa29a5a1700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa2940b1260 0x7fa294159cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:55.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.194+0000 7fa29a5a1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2940b2120 0x7fa29415a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:55.209 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.194+0000 7fa29a5a1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2940668e0 0x7fa294066d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:55.209 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.194+0000 7fa29a5a1700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa294065c90 con 0x7fa2940b2120 2022-06-29T15:51:55.209 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.194+0000 7fa29a5a1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa294067260 con 0x7fa2940668e0 2022-06-29T15:51:55.209 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.194+0000 7fa29a5a1700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2940af6d0 con 0x7fa2940b1260 2022-06-29T15:51:55.210 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.194+0000 7fa2937fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2940b2120 0x7fa29415a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:55.210 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.194+0000 7fa2937fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2940b2120 0x7fa29415a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54498/0 (socket says 172.21.15.47:54498) 2022-06-29T15:51:55.210 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.194+0000 7fa2937fe700 1 -- 172.21.15.47:0/1121707934 learned_addr learned my addr 172.21.15.47:0/1121707934 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:55.210 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.194+0000 7fa293fff700 1 --2- 172.21.15.47:0/1121707934 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa2940b1260 0x7fa294159cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:55.211 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.194+0000 7fa298b3e700 1 --2- 172.21.15.47:0/1121707934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2940668e0 0x7fa294066d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:55.211 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.198+0000 7fa2937fe700 1 -- 172.21.15.47:0/1121707934 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa2940b1260 msgr2=0x7fa294159cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:55.211 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.198+0000 7fa2937fe700 1 --2- 172.21.15.47:0/1121707934 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa2940b1260 0x7fa294159cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:55.211 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.198+0000 7fa2937fe700 1 -- 172.21.15.47:0/1121707934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2940668e0 msgr2=0x7fa294066d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:55.212 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.198+0000 7fa2937fe700 1 --2- 172.21.15.47:0/1121707934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2940668e0 0x7fa294066d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:55.212 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.198+0000 7fa2937fe700 1 -- 172.21.15.47:0/1121707934 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa28400b180 con 0x7fa2940b2120 2022-06-29T15:51:55.212 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.198+0000 7fa2937fe700 1 --2- 172.21.15.47:0/1121707934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2940b2120 0x7fa29415a1b0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 rx=0x7fa27c0018c0 tx=0x7fa27c001d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:55.212 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.198+0000 7fa2917fa700 1 -- 172.21.15.47:0/1121707934 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa27c0070d0 con 0x7fa2940b2120 2022-06-29T15:51:55.213 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.198+0000 7fa2917fa700 1 -- 172.21.15.47:0/1121707934 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa27c007230 con 0x7fa2940b2120 2022-06-29T15:51:55.213 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.198+0000 7fa29a5a1700 1 -- 172.21.15.47:0/1121707934 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa29415a850 con 0x7fa2940b2120 2022-06-29T15:51:55.213 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.198+0000 7fa2917fa700 1 -- 172.21.15.47:0/1121707934 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa27c005610 con 0x7fa2940b2120 2022-06-29T15:51:55.213 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.198+0000 7fa29a5a1700 1 -- 172.21.15.47:0/1121707934 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2940ae5b0 con 0x7fa2940b2120 2022-06-29T15:51:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.198+0000 7fa2917fa700 1 -- 172.21.15.47:0/1121707934 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa27c010070 con 0x7fa2940b2120 2022-06-29T15:51:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.198+0000 7fa29a5a1700 1 -- 172.21.15.47:0/1121707934 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa29415a6a0 con 0x7fa2940b2120 2022-06-29T15:51:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.198+0000 7fa2917fa700 1 --2- 172.21.15.47:0/1121707934 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa278053a10 0x7fa278055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.198+0000 7fa2917fa700 1 -- 172.21.15.47:0/1121707934 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa27c06b5d0 con 0x7fa2940b2120 2022-06-29T15:51:55.215 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.198+0000 7fa293fff700 1 --2- 172.21.15.47:0/1121707934 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa278053a10 0x7fa278055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:55.215 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.198+0000 7fa293fff700 1 --2- 172.21.15.47:0/1121707934 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa278053a10 0x7fa278055ed0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7fa28401a000 tx=0x7fa284014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:55.215 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.206+0000 7fa2917fa700 1 -- 172.21.15.47:0/1121707934 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa27c042390 con 0x7fa2940b2120 2022-06-29T15:51:55.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.370+0000 7fa29a5a1700 1 -- 172.21.15.47:0/1121707934 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa294147650 con 0x7fa2940b2120 2022-06-29T15:51:55.376 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.370+0000 7fa2917fa700 1 -- 172.21.15.47:0/1121707934 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+2495 (secure 0 0 0) 0x7fa27c0459b0 con 0x7fa2940b2120 2022-06-29T15:51:55.377 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:55.377 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":74,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8296,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/3321939421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3321939421},{"type":"v1","addr":"172.21.15.64:6833","nonce":3321939421}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":72},{"gid":8415,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1886881850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1886881850},{"type":"v1","addr":"172.21.15.47:6835","nonce":1886881850}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":72}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:51:52.828119+0000","modified":"2022-06-29T15:51:53.831305+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8421":{"gid":8421,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6837/2466961065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2466961065},{"type":"v1","addr":"172.21.15.47:6837","nonce":2466961065}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2022-06-29T15:51:55.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.374+0000 7fa29a5a1700 1 -- 172.21.15.47:0/1121707934 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa278053a10 msgr2=0x7fa278055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:55.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.374+0000 7fa29a5a1700 1 --2- 172.21.15.47:0/1121707934 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa278053a10 0x7fa278055ed0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7fa28401a000 tx=0x7fa284014000).stop 2022-06-29T15:51:55.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.374+0000 7fa29a5a1700 1 -- 172.21.15.47:0/1121707934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2940b2120 msgr2=0x7fa29415a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:55.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.374+0000 7fa29a5a1700 1 --2- 172.21.15.47:0/1121707934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2940b2120 0x7fa29415a1b0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 rx=0x7fa27c0018c0 tx=0x7fa27c001d80).stop 2022-06-29T15:51:55.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.374+0000 7fa29a5a1700 1 -- 172.21.15.47:0/1121707934 shutdown_connections 2022-06-29T15:51:55.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.374+0000 7fa29a5a1700 1 --2- 172.21.15.47:0/1121707934 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa278053a10 0x7fa278055ed0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:55.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.374+0000 7fa29a5a1700 1 --2- 172.21.15.47:0/1121707934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2940668e0 0x7fa294066d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:55.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.374+0000 7fa29a5a1700 1 --2- 172.21.15.47:0/1121707934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2940b2120 0x7fa29415a1b0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:55.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.374+0000 7fa29a5a1700 1 --2- 172.21.15.47:0/1121707934 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa2940b1260 0x7fa294159cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:55.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.374+0000 7fa29a5a1700 1 -- 172.21.15.47:0/1121707934 >> 172.21.15.47:0/1121707934 conn(0x7fa2940626f0 msgr2=0x7fa294146530 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:55.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.374+0000 7fa29a5a1700 1 -- 172.21.15.47:0/1121707934 shutdown_connections 2022-06-29T15:51:55.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.374+0000 7fa29a5a1700 1 -- 172.21.15.47:0/1121707934 wait complete. 2022-06-29T15:51:55.384 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 74 2022-06-29T15:51:55.698 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-29T15:51:52.828119+0000', 'modified': '2022-06-29T15:51:53.831305+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8421}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8421': {'gid': 8421, 'name': 'c', 'rank': 0, 'incarnation': 73, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.47:6837/2466961065', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6836', 'nonce': 2466961065}, {'type': 'v1', 'addr': '172.21.15.47:6837', 'nonce': 2466961065}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-29T15:51:55.699 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-29T15:51:55.700 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2022-06-29T15:51:55.817 INFO:teuthology.orchestra.run.smithi047.stdout:{"cluster_fsid":"9a68894a-53ce-400e-838b-14eab91b834c","whoami":0,"id":8421,"want_state":"up:active","state":"up:active","rank_uptime":2.9728174470000002,"mdsmap_epoch":74,"osdmap_epoch":151,"osdmap_epoch_barrier":150,"uptime":7.993636425} 2022-06-29T15:51:55.818 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9a68894a-53ce-400e-838b-14eab91b834c", "id": 8421, "mdsmap_epoch": 74, "osdmap_epoch": 151, "osdmap_epoch_barrier": 150, "rank_uptime": 2.972817447, "state": "up:active", "uptime": 7.993636425, "want_state": "up:active", "whoami": 0 } 2022-06-29T15:51:55.818 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:51:55.918 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.910+0000 7fcc45302700 1 Processor -- start 2022-06-29T15:51:55.919 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.910+0000 7fcc45302700 1 -- start start 2022-06-29T15:51:55.920 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.914+0000 7fcc45302700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcc400b1260 0x7fcc4005ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:55.921 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.914+0000 7fcc45302700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc400b0740 con 0x7fcc400b2120 2022-06-29T15:51:55.921 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.914+0000 7fcc45302700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc400b0970 con 0x7fcc4005f0b0 2022-06-29T15:51:55.921 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.914+0000 7fcc45302700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc40147460 con 0x7fcc400b1260 2022-06-29T15:51:55.921 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.914+0000 7fcc3effd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fcc400b2120 0x7fcc40147af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40058/0 (socket says 172.21.15.47:40058) 2022-06-29T15:51:55.922 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.914+0000 7fcc3f7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcc400b1260 0x7fcc4005ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:55.922 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.914+0000 7fcc3effd700 1 -- 172.21.15.47:0/2588924910 learned_addr learned my addr 172.21.15.47:0/2588924910 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:55.922 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.914+0000 7fcc3f7fe700 1 -- 172.21.15.47:0/2588924910 >> v1:172.21.15.64:6789/0 conn(0x7fcc4005f0b0 legacy=0x7fcc4005f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:55.922 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.914+0000 7fcc3f7fe700 1 -- 172.21.15.47:0/2588924910 >> v1:172.21.15.47:6789/0 conn(0x7fcc400b2120 legacy=0x7fcc40147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:55.923 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.914+0000 7fcc3dffb700 1 -- 172.21.15.47:0/2588924910 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3766080288 0 0) 0x7fcc400b0740 con 0x7fcc400b2120 2022-06-29T15:51:55.923 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.914+0000 7fcc3f7fe700 1 -- 172.21.15.47:0/2588924910 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc400b0c10 con 0x7fcc400b1260 2022-06-29T15:51:55.923 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.914+0000 7fcc3f7fe700 1 --2- 172.21.15.47:0/2588924910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc400b1260 0x7fcc4005ffc0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 rx=0x7fcc40141a70 tx=0x7fcc380091d0).ready entity=mon.0 client_cookie=ab5b8f5062d05bbb server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:55.923 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.914+0000 7fcc3dffb700 1 -- 172.21.15.47:0/2588924910 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc3800b820 con 0x7fcc400b1260 2022-06-29T15:51:55.924 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.914+0000 7fcc3dffb700 1 -- 172.21.15.47:0/2588924910 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc3800b980 con 0x7fcc400b1260 2022-06-29T15:51:55.924 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.914+0000 7fcc3dffb700 1 -- 172.21.15.47:0/2588924910 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc3800baf0 con 0x7fcc400b1260 2022-06-29T15:51:55.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.914+0000 7fcc45302700 1 -- 172.21.15.47:0/2588924910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc400b1260 msgr2=0x7fcc4005ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:55.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.914+0000 7fcc45302700 1 --2- 172.21.15.47:0/2588924910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc400b1260 0x7fcc4005ffc0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 rx=0x7fcc40141a70 tx=0x7fcc380091d0).stop 2022-06-29T15:51:55.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.914+0000 7fcc45302700 1 -- 172.21.15.47:0/2588924910 shutdown_connections 2022-06-29T15:51:55.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.914+0000 7fcc45302700 1 --2- 172.21.15.47:0/2588924910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc400b1260 0x7fcc4005ffc0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:55.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.914+0000 7fcc45302700 1 -- 172.21.15.47:0/2588924910 >> 172.21.15.47:0/2588924910 conn(0x7fcc400626f0 msgr2=0x7fcc40062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:55.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.914+0000 7fcc45302700 1 -- 172.21.15.47:0/2588924910 shutdown_connections 2022-06-29T15:51:55.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.914+0000 7fcc45302700 1 -- 172.21.15.47:0/2588924910 wait complete. 2022-06-29T15:51:55.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.914+0000 7fcc45302700 1 Processor -- start 2022-06-29T15:51:55.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.914+0000 7fcc45302700 1 -- start start 2022-06-29T15:51:55.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.918+0000 7fcc45302700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcc400b1260 0x7fcc4014a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:55.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.918+0000 7fcc45302700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc400b2120 0x7fcc400633b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:55.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.918+0000 7fcc45302700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc4009ecf0 0x7fcc401567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:55.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.918+0000 7fcc45302700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc400a1e40 con 0x7fcc400b2120 2022-06-29T15:51:55.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.918+0000 7fcc45302700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4009eb80 con 0x7fcc4009ecf0 2022-06-29T15:51:55.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.918+0000 7fcc45302700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4015a740 con 0x7fcc400b1260 2022-06-29T15:51:55.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.918+0000 7fcc3e7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc400b2120 0x7fcc400633b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:55.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.918+0000 7fcc3e7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc400b2120 0x7fcc400633b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54512/0 (socket says 172.21.15.47:54512) 2022-06-29T15:51:55.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.918+0000 7fcc3e7fc700 1 -- 172.21.15.47:0/388314207 learned_addr learned my addr 172.21.15.47:0/388314207 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:55.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.918+0000 7fcc3f7fe700 1 --2- 172.21.15.47:0/388314207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc4009ecf0 0x7fcc401567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:55.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.918+0000 7fcc3effd700 1 --2- 172.21.15.47:0/388314207 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcc400b1260 0x7fcc4014a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:55.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.918+0000 7fcc3e7fc700 1 -- 172.21.15.47:0/388314207 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcc400b1260 msgr2=0x7fcc4014a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:55.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.918+0000 7fcc3e7fc700 1 --2- 172.21.15.47:0/388314207 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcc400b1260 0x7fcc4014a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:55.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.918+0000 7fcc3e7fc700 1 -- 172.21.15.47:0/388314207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc4009ecf0 msgr2=0x7fcc401567e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:55.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.918+0000 7fcc3e7fc700 1 --2- 172.21.15.47:0/388314207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc4009ecf0 0x7fcc401567e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:55.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.918+0000 7fcc3e7fc700 1 -- 172.21.15.47:0/388314207 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc38009470 con 0x7fcc400b2120 2022-06-29T15:51:55.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.918+0000 7fcc3f7fe700 1 --2- 172.21.15.47:0/388314207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc4009ecf0 0x7fcc401567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:51:55.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.918+0000 7fcc3e7fc700 1 --2- 172.21.15.47:0/388314207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc400b2120 0x7fcc400633b0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 rx=0x7fcc400a25b0 tx=0x7fcc30007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:55.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.918+0000 7fcc2ffff700 1 -- 172.21.15.47:0/388314207 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc3000ce20 con 0x7fcc400b2120 2022-06-29T15:51:55.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.918+0000 7fcc2ffff700 1 -- 172.21.15.47:0/388314207 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc3000f470 con 0x7fcc400b2120 2022-06-29T15:51:55.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.918+0000 7fcc45302700 1 -- 172.21.15.47:0/388314207 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc400a2970 con 0x7fcc400b2120 2022-06-29T15:51:55.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.918+0000 7fcc2ffff700 1 -- 172.21.15.47:0/388314207 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc3000f7d0 con 0x7fcc400b2120 2022-06-29T15:51:55.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.918+0000 7fcc45302700 1 -- 172.21.15.47:0/388314207 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc4009f770 con 0x7fcc400b2120 2022-06-29T15:51:55.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.918+0000 7fcc45302700 1 -- 172.21.15.47:0/388314207 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc40147650 con 0x7fcc400b2120 2022-06-29T15:51:55.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.918+0000 7fcc2ffff700 1 -- 172.21.15.47:0/388314207 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fcc30019030 con 0x7fcc400b2120 2022-06-29T15:51:55.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.922+0000 7fcc2ffff700 1 --2- 172.21.15.47:0/388314207 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcc1c053a10 0x7fcc1c055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:55.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.922+0000 7fcc2ffff700 1 -- 172.21.15.47:0/388314207 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fcc3006b490 con 0x7fcc400b2120 2022-06-29T15:51:55.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.922+0000 7fcc3effd700 1 --2- 172.21.15.47:0/388314207 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcc1c053a10 0x7fcc1c055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:55.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.922+0000 7fcc3effd700 1 --2- 172.21.15.47:0/388314207 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcc1c053a10 0x7fcc1c055ed0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7fcc3400b000 tx=0x7fcc34009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:55.938 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:55.926+0000 7fcc2ffff700 1 -- 172.21.15.47:0/388314207 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcc30042250 con 0x7fcc400b2120 2022-06-29T15:51:56.060 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.054+0000 7fcc45302700 1 -- 172.21.15.47:0/388314207 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc4015a880 con 0x7fcc400b2120 2022-06-29T15:51:56.062 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.054+0000 7fcc2ffff700 1 -- 172.21.15.47:0/388314207 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) v1 ==== 74+0+16285 (secure 0 0 0) 0x7fcc30045870 con 0x7fcc400b2120 2022-06-29T15:51:56.063 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:56.063 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":151,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:51:53.832186+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:51:50.583017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2022-06-29T15:51:51.834830+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:51:56.066 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.058+0000 7fcc45302700 1 -- 172.21.15.47:0/388314207 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcc1c053a10 msgr2=0x7fcc1c055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:56.066 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.058+0000 7fcc45302700 1 --2- 172.21.15.47:0/388314207 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcc1c053a10 0x7fcc1c055ed0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7fcc3400b000 tx=0x7fcc34009040).stop 2022-06-29T15:51:56.066 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.058+0000 7fcc45302700 1 -- 172.21.15.47:0/388314207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc400b2120 msgr2=0x7fcc400633b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.058+0000 7fcc45302700 1 --2- 172.21.15.47:0/388314207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc400b2120 0x7fcc400633b0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 rx=0x7fcc400a25b0 tx=0x7fcc30007c20).stop 2022-06-29T15:51:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.058+0000 7fcc45302700 1 -- 172.21.15.47:0/388314207 shutdown_connections 2022-06-29T15:51:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.058+0000 7fcc45302700 1 --2- 172.21.15.47:0/388314207 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcc1c053a10 0x7fcc1c055ed0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.058+0000 7fcc45302700 1 --2- 172.21.15.47:0/388314207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc4009ecf0 0x7fcc401567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:56.068 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.058+0000 7fcc45302700 1 --2- 172.21.15.47:0/388314207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc400b2120 0x7fcc400633b0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:56.068 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.058+0000 7fcc45302700 1 --2- 172.21.15.47:0/388314207 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcc400b1260 0x7fcc4014a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:56.068 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.058+0000 7fcc45302700 1 -- 172.21.15.47:0/388314207 >> 172.21.15.47:0/388314207 conn(0x7fcc400626f0 msgr2=0x7fcc400b5480 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:56.068 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.058+0000 7fcc45302700 1 -- 172.21.15.47:0/388314207 shutdown_connections 2022-06-29T15:51:56.069 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.058+0000 7fcc45302700 1 -- 172.21.15.47:0/388314207 wait complete. 2022-06-29T15:51:56.081 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-29T15:42:16.971431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-29T15:42:25.290884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 27, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-29T15:51:50.583017+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '150', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 28, 'pool_name': 'cephfs_data', 'create_time': '2022-06-29T15:51:51.834830+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '151', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-29T15:51:56.082 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-29T15:51:56.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.186+0000 7fa3c6e6a700 1 Processor -- start 2022-06-29T15:51:56.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.186+0000 7fa3c6e6a700 1 -- start start 2022-06-29T15:51:56.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.190+0000 7fa3c6e6a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa3c00afcc0 0x7fa3c013cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:56.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.190+0000 7fa3c6e6a700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa3c00b16e0 0x7fa3c00ad9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:56.198 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.190+0000 7fa3c6e6a700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3c00670b0 con 0x7fa3c0066cd0 2022-06-29T15:51:56.198 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.190+0000 7fa3c6e6a700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3c00b01d0 con 0x7fa3c00afcc0 2022-06-29T15:51:56.198 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.190+0000 7fa3c6e6a700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c00b0470 con 0x7fa3c00b16e0 2022-06-29T15:51:56.198 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.190+0000 7fa3c5667700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa3c00afcc0 0x7fa3c013cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:56.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.190+0000 7fa3c5667700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa3c00afcc0 0x7fa3c013cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54518/0 (socket says 172.21.15.47:54518) 2022-06-29T15:51:56.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.190+0000 7fa3c5e68700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa3c0066cd0 0x7fa3c013c8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40072/0 (socket says 172.21.15.47:40072) 2022-06-29T15:51:56.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.190+0000 7fa3c5667700 1 -- 172.21.15.47:0/2685624031 learned_addr learned my addr 172.21.15.47:0/2685624031 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:56.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.190+0000 7fa3c6669700 1 --2- 172.21.15.47:0/2685624031 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa3c00b16e0 0x7fa3c00ad9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:56.200 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.190+0000 7fa3c5667700 1 -- 172.21.15.47:0/2685624031 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa3c00b16e0 msgr2=0x7fa3c00ad9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:56.200 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.190+0000 7fa3c5667700 1 --2- 172.21.15.47:0/2685624031 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa3c00b16e0 0x7fa3c00ad9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:56.200 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.190+0000 7fa3c5667700 1 -- 172.21.15.47:0/2685624031 >> v1:172.21.15.47:6789/0 conn(0x7fa3c0066cd0 legacy=0x7fa3c013c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:56.200 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.190+0000 7fa3c4e66700 1 -- 172.21.15.47:0/2685624031 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1663232228 0 0) 0x7fa3c00670b0 con 0x7fa3c0066cd0 2022-06-29T15:51:56.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.190+0000 7fa3c5667700 1 -- 172.21.15.47:0/2685624031 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c00b0710 con 0x7fa3c00afcc0 2022-06-29T15:51:56.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.190+0000 7fa3c5667700 1 --2- 172.21.15.47:0/2685624031 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3c00afcc0 0x7fa3c013cfd0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 rx=0x7fa3b0002f70 tx=0x7fa3b0006eb0).ready entity=mon.0 client_cookie=bad3b454d67c6c66 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:56.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.190+0000 7fa3c4e66700 1 -- 172.21.15.47:0/2685624031 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3b000d5c0 con 0x7fa3c00afcc0 2022-06-29T15:51:56.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.190+0000 7fa3c4e66700 1 -- 172.21.15.47:0/2685624031 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b0000e90 con 0x7fa3c00afcc0 2022-06-29T15:51:56.202 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.190+0000 7fa3c4e66700 1 -- 172.21.15.47:0/2685624031 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3b000d910 con 0x7fa3c00afcc0 2022-06-29T15:51:56.204 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.190+0000 7fa3c6e6a700 1 -- 172.21.15.47:0/2685624031 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3c00afcc0 msgr2=0x7fa3c013cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:56.205 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.190+0000 7fa3c6e6a700 1 --2- 172.21.15.47:0/2685624031 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3c00afcc0 0x7fa3c013cfd0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 rx=0x7fa3b0002f70 tx=0x7fa3b0006eb0).stop 2022-06-29T15:51:56.205 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.190+0000 7fa3c6e6a700 1 -- 172.21.15.47:0/2685624031 shutdown_connections 2022-06-29T15:51:56.205 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.190+0000 7fa3c6e6a700 1 --2- 172.21.15.47:0/2685624031 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa3c00b16e0 0x7fa3c00ad9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:56.205 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.190+0000 7fa3c6e6a700 1 --2- 172.21.15.47:0/2685624031 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3c00afcc0 0x7fa3c013cfd0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:56.206 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.190+0000 7fa3c6e6a700 1 -- 172.21.15.47:0/2685624031 >> 172.21.15.47:0/2685624031 conn(0x7fa3c00b5320 msgr2=0x7fa3c00b5700 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:56.206 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.190+0000 7fa3c6e6a700 1 -- 172.21.15.47:0/2685624031 shutdown_connections 2022-06-29T15:51:56.206 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.190+0000 7fa3c6e6a700 1 -- 172.21.15.47:0/2685624031 wait complete. 2022-06-29T15:51:56.206 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.194+0000 7fa3c6e6a700 1 Processor -- start 2022-06-29T15:51:56.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.194+0000 7fa3c6e6a700 1 -- start start 2022-06-29T15:51:56.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.194+0000 7fa3c6e6a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3c0066cd0 0x7fa3c0155c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:56.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.194+0000 7fa3c6e6a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3c00afcc0 0x7fa3c0144820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:56.207 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.194+0000 7fa3c6e6a700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa3c00b16e0 0x7fa3c013f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:56.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.194+0000 7fa3c6e6a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c00ac0d0 con 0x7fa3c00afcc0 2022-06-29T15:51:56.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.194+0000 7fa3c6e6a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c0156ba0 con 0x7fa3c0066cd0 2022-06-29T15:51:56.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.194+0000 7fa3c6e6a700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c0146460 con 0x7fa3c00b16e0 2022-06-29T15:51:56.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.194+0000 7fa3c5667700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3c00afcc0 0x7fa3c0144820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:56.208 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.194+0000 7fa3c5667700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3c00afcc0 0x7fa3c0144820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54526/0 (socket says 172.21.15.47:54526) 2022-06-29T15:51:56.209 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.194+0000 7fa3c5667700 1 -- 172.21.15.47:0/4185964572 learned_addr learned my addr 172.21.15.47:0/4185964572 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:56.209 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.194+0000 7fa3c6669700 1 --2- 172.21.15.47:0/4185964572 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa3c00b16e0 0x7fa3c013f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:56.209 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.194+0000 7fa3c5e68700 1 --2- 172.21.15.47:0/4185964572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3c0066cd0 0x7fa3c0155c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:56.209 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.198+0000 7fa3c5667700 1 -- 172.21.15.47:0/4185964572 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa3c00b16e0 msgr2=0x7fa3c013f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:56.210 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.198+0000 7fa3c5667700 1 --2- 172.21.15.47:0/4185964572 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa3c00b16e0 0x7fa3c013f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:56.210 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.198+0000 7fa3c5667700 1 -- 172.21.15.47:0/4185964572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3c0066cd0 msgr2=0x7fa3c0155c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:56.210 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.198+0000 7fa3c5667700 1 --2- 172.21.15.47:0/4185964572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3c0066cd0 0x7fa3c0155c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:56.210 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.198+0000 7fa3c5667700 1 -- 172.21.15.47:0/4185964572 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b000b1e0 con 0x7fa3c00afcc0 2022-06-29T15:51:56.211 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.198+0000 7fa3c5667700 1 --2- 172.21.15.47:0/4185964572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3c00afcc0 0x7fa3c0144820 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 rx=0x7fa3b0002f70 tx=0x7fa3b0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:56.211 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.198+0000 7fa3beffd700 1 -- 172.21.15.47:0/4185964572 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3b0006b40 con 0x7fa3c00afcc0 2022-06-29T15:51:56.211 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.198+0000 7fa3beffd700 1 -- 172.21.15.47:0/4185964572 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b000de40 con 0x7fa3c00afcc0 2022-06-29T15:51:56.211 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.198+0000 7fa3c6e6a700 1 -- 172.21.15.47:0/4185964572 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3c0144380 con 0x7fa3c00afcc0 2022-06-29T15:51:56.211 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.198+0000 7fa3beffd700 1 -- 172.21.15.47:0/4185964572 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3b0011d70 con 0x7fa3c00afcc0 2022-06-29T15:51:56.212 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.198+0000 7fa3c6e6a700 1 -- 172.21.15.47:0/4185964572 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3c0140570 con 0x7fa3c00afcc0 2022-06-29T15:51:56.213 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.198+0000 7fa3c6e6a700 1 -- 172.21.15.47:0/4185964572 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3c01462b0 con 0x7fa3c00afcc0 2022-06-29T15:51:56.213 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.198+0000 7fa3beffd700 1 -- 172.21.15.47:0/4185964572 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa3b0025030 con 0x7fa3c00afcc0 2022-06-29T15:51:56.213 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.202+0000 7fa3beffd700 1 --2- 172.21.15.47:0/4185964572 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa3a0053a70 0x7fa3a0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:56.213 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.202+0000 7fa3c5e68700 1 --2- 172.21.15.47:0/4185964572 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa3a0053a70 0x7fa3a0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:56.214 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.202+0000 7fa3beffd700 1 -- 172.21.15.47:0/4185964572 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa3b006d250 con 0x7fa3c00afcc0 2022-06-29T15:51:56.214 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.202+0000 7fa3c5e68700 1 --2- 172.21.15.47:0/4185964572 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa3a0053a70 0x7fa3a0055f30 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x7fa3b400d000 tx=0x7fa3b4007a00).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:56.215 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.210+0000 7fa3beffd700 1 -- 172.21.15.47:0/4185964572 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa3b0043f80 con 0x7fa3c00afcc0 2022-06-29T15:51:56.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.330+0000 7fa3c6e6a700 1 -- 172.21.15.47:0/4185964572 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fa3c00a8d70 con 0x7fa3c00afcc0 2022-06-29T15:51:56.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.334+0000 7fa3beffd700 1 -- 172.21.15.47:0/4185964572 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v151) v1 ==== 110+0+10 (secure 0 0 0) 0x7fa3b0047520 con 0x7fa3c00afcc0 2022-06-29T15:51:56.340 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2022-06-29T15:51:56.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.334+0000 7fa3c6e6a700 1 -- 172.21.15.47:0/4185964572 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa3a0053a70 msgr2=0x7fa3a0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:56.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.334+0000 7fa3c6e6a700 1 --2- 172.21.15.47:0/4185964572 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa3a0053a70 0x7fa3a0055f30 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x7fa3b400d000 tx=0x7fa3b4007a00).stop 2022-06-29T15:51:56.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.334+0000 7fa3c6e6a700 1 -- 172.21.15.47:0/4185964572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3c00afcc0 msgr2=0x7fa3c0144820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:56.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.334+0000 7fa3c6e6a700 1 --2- 172.21.15.47:0/4185964572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3c00afcc0 0x7fa3c0144820 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 rx=0x7fa3b0002f70 tx=0x7fa3b0015000).stop 2022-06-29T15:51:56.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.334+0000 7fa3c6e6a700 1 -- 172.21.15.47:0/4185964572 shutdown_connections 2022-06-29T15:51:56.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.334+0000 7fa3c6e6a700 1 --2- 172.21.15.47:0/4185964572 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa3c00b16e0 0x7fa3c013f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:56.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.334+0000 7fa3c6e6a700 1 --2- 172.21.15.47:0/4185964572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3c00afcc0 0x7fa3c0144820 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:56.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.334+0000 7fa3c6e6a700 1 --2- 172.21.15.47:0/4185964572 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa3a0053a70 0x7fa3a0055f30 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:56.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.334+0000 7fa3c6e6a700 1 --2- 172.21.15.47:0/4185964572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3c0066cd0 0x7fa3c0155c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:56.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.334+0000 7fa3c6e6a700 1 -- 172.21.15.47:0/4185964572 >> 172.21.15.47:0/4185964572 conn(0x7fa3c00b5320 msgr2=0x7fa3c013a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:56.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.334+0000 7fa3c6e6a700 1 -- 172.21.15.47:0/4185964572 shutdown_connections 2022-06-29T15:51:56.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.334+0000 7fa3c6e6a700 1 -- 172.21.15.47:0/4185964572 wait complete. 2022-06-29T15:51:56.352 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-29T15:51:56.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.458+0000 7f52aafef700 1 Processor -- start 2022-06-29T15:51:56.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.458+0000 7f52aafef700 1 -- start start 2022-06-29T15:51:56.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.458+0000 7f52aafef700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f52a413e550 0x7f52a4140f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:56.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.458+0000 7f52aafef700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f52a4067500 0x7f52a405f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:56.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.458+0000 7f52aafef700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52a40b4ca0 con 0x7f52a405f5a0 2022-06-29T15:51:56.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.458+0000 7f52aafef700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52a40b3a80 con 0x7f52a413e550 2022-06-29T15:51:56.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.458+0000 7f52aafef700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52a40b37e0 con 0x7f52a4067500 2022-06-29T15:51:56.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.458+0000 7f52a8d8b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f52a413e550 0x7f52a4140f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:56.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.458+0000 7f52a8d8b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f52a413e550 0x7f52a4140f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54532/0 (socket says 172.21.15.47:54532) 2022-06-29T15:51:56.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.458+0000 7f52a8d8b700 1 -- 172.21.15.47:0/1376492812 learned_addr learned my addr 172.21.15.47:0/1376492812 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:56.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.458+0000 7f52a3fff700 1 --2- 172.21.15.47:0/1376492812 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f52a4067500 0x7f52a405f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:56.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.458+0000 7f52a8d8b700 1 -- 172.21.15.47:0/1376492812 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f52a4067500 msgr2=0x7f52a405f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:56.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.458+0000 7f52a8d8b700 1 --2- 172.21.15.47:0/1376492812 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f52a4067500 0x7f52a405f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:56.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.458+0000 7f52a8d8b700 1 -- 172.21.15.47:0/1376492812 >> v1:172.21.15.64:6789/0 conn(0x7f52a405f5a0 legacy=0x7f52a4066ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:56.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.458+0000 7f52a8d8b700 1 -- 172.21.15.47:0/1376492812 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52a40672b0 con 0x7f52a413e550 2022-06-29T15:51:56.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.458+0000 7f52a8d8b700 1 --2- 172.21.15.47:0/1376492812 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52a413e550 0x7f52a4140f40 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 rx=0x7f5298002f70 tx=0x7f5298006eb0).ready entity=mon.0 client_cookie=fad9c46672770c93 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:56.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.458+0000 7f52a37fe700 1 -- 172.21.15.47:0/1376492812 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f529800d570 con 0x7f52a413e550 2022-06-29T15:51:56.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.458+0000 7f52a37fe700 1 -- 172.21.15.47:0/1376492812 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5298000e90 con 0x7f52a413e550 2022-06-29T15:51:56.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.458+0000 7f52a37fe700 1 -- 172.21.15.47:0/1376492812 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f529800d900 con 0x7f52a413e550 2022-06-29T15:51:56.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.462+0000 7f52aafef700 1 -- 172.21.15.47:0/1376492812 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52a413e550 msgr2=0x7f52a4140f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:56.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.462+0000 7f52aafef700 1 --2- 172.21.15.47:0/1376492812 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52a413e550 0x7f52a4140f40 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 rx=0x7f5298002f70 tx=0x7f5298006eb0).stop 2022-06-29T15:51:56.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.462+0000 7f52aafef700 1 -- 172.21.15.47:0/1376492812 shutdown_connections 2022-06-29T15:51:56.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.462+0000 7f52aafef700 1 --2- 172.21.15.47:0/1376492812 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f52a4067500 0x7f52a405f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:56.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.462+0000 7f52aafef700 1 --2- 172.21.15.47:0/1376492812 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52a413e550 0x7f52a4140f40 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:56.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.462+0000 7f52aafef700 1 -- 172.21.15.47:0/1376492812 >> 172.21.15.47:0/1376492812 conn(0x7f52a40627d0 msgr2=0x7f52a4062bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:56.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.462+0000 7f52aafef700 1 -- 172.21.15.47:0/1376492812 shutdown_connections 2022-06-29T15:51:56.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.462+0000 7f52aafef700 1 -- 172.21.15.47:0/1376492812 wait complete. 2022-06-29T15:51:56.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.462+0000 7f52aafef700 1 Processor -- start 2022-06-29T15:51:56.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.462+0000 7f52aafef700 1 -- start start 2022-06-29T15:51:56.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.462+0000 7f52aafef700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f52a4067500 0x7f52a40b5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:56.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.462+0000 7f52aafef700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52a413e550 0x7f52a4144d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:56.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.462+0000 7f52aafef700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52a40b6f20 0x7f52a40b73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:56.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.462+0000 7f52aafef700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a40abf80 con 0x7f52a40b6f20 2022-06-29T15:51:56.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.462+0000 7f52aafef700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a40ac1b0 con 0x7f52a413e550 2022-06-29T15:51:56.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.462+0000 7f52aafef700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52a40ac450 con 0x7f52a4067500 2022-06-29T15:51:56.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.462+0000 7f52a958c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52a40b6f20 0x7f52a40b73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:56.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.462+0000 7f52a958c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52a40b6f20 0x7f52a40b73d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54542/0 (socket says 172.21.15.47:54542) 2022-06-29T15:51:56.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.462+0000 7f52a958c700 1 -- 172.21.15.47:0/4101606593 learned_addr learned my addr 172.21.15.47:0/4101606593 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:56.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.462+0000 7f52a8d8b700 1 --2- 172.21.15.47:0/4101606593 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f52a4067500 0x7f52a40b5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:56.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.466+0000 7f52a3fff700 1 --2- 172.21.15.47:0/4101606593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52a413e550 0x7f52a4144d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:56.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.466+0000 7f52a958c700 1 -- 172.21.15.47:0/4101606593 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f52a4067500 msgr2=0x7f52a40b5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:56.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.466+0000 7f52a958c700 1 --2- 172.21.15.47:0/4101606593 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f52a4067500 0x7f52a40b5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:56.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.466+0000 7f52a958c700 1 -- 172.21.15.47:0/4101606593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52a413e550 msgr2=0x7f52a4144d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:56.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.466+0000 7f52a958c700 1 --2- 172.21.15.47:0/4101606593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52a413e550 0x7f52a4144d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:56.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.466+0000 7f52a958c700 1 -- 172.21.15.47:0/4101606593 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f529800b1e0 con 0x7f52a40b6f20 2022-06-29T15:51:56.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.466+0000 7f52a3fff700 1 --2- 172.21.15.47:0/4101606593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52a413e550 0x7f52a4144d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:51:56.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.466+0000 7f52a958c700 1 --2- 172.21.15.47:0/4101606593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52a40b6f20 0x7f52a40b73d0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 rx=0x7f52a4066a80 tx=0x7f529c007f10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:56.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.466+0000 7f52a1ffb700 1 -- 172.21.15.47:0/4101606593 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f529c00f470 con 0x7f52a40b6f20 2022-06-29T15:51:56.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.466+0000 7f52a1ffb700 1 -- 172.21.15.47:0/4101606593 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f529c00f5d0 con 0x7f52a40b6f20 2022-06-29T15:51:56.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.466+0000 7f52a1ffb700 1 -- 172.21.15.47:0/4101606593 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f529c00f930 con 0x7f52a40b6f20 2022-06-29T15:51:56.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.466+0000 7f52aafef700 1 -- 172.21.15.47:0/4101606593 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52a40ac6f0 con 0x7f52a40b6f20 2022-06-29T15:51:56.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.466+0000 7f52aafef700 1 -- 172.21.15.47:0/4101606593 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52a4061c10 con 0x7f52a40b6f20 2022-06-29T15:51:56.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.466+0000 7f52aafef700 1 -- 172.21.15.47:0/4101606593 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52a40b4e80 con 0x7f52a40b6f20 2022-06-29T15:51:56.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.466+0000 7f52a1ffb700 1 -- 172.21.15.47:0/4101606593 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f529c019030 con 0x7f52a40b6f20 2022-06-29T15:51:56.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.466+0000 7f52a1ffb700 1 --2- 172.21.15.47:0/4101606593 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5284053b00 0x7f5284055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:56.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.466+0000 7f52a1ffb700 1 -- 172.21.15.47:0/4101606593 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f529c06b880 con 0x7f52a40b6f20 2022-06-29T15:51:56.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.466+0000 7f52a8d8b700 1 --2- 172.21.15.47:0/4101606593 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5284053b00 0x7f5284055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:56.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.466+0000 7f52a8d8b700 1 --2- 172.21.15.47:0/4101606593 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5284053b00 0x7f5284055fc0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7f529801c000 tx=0x7f5298015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:56.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.474+0000 7f52a1ffb700 1 -- 172.21.15.47:0/4101606593 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f52a40b4e80 con 0x7f52a40b6f20 2022-06-29T15:51:56.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.598+0000 7f52aafef700 1 -- 172.21.15.47:0/4101606593 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f52a40b4e80 con 0x7f52a40b6f20 2022-06-29T15:51:56.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.598+0000 7f52a1ffb700 1 -- 172.21.15.47:0/4101606593 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v151) v1 ==== 92+0+10 (secure 0 0 0) 0x7f52a40b4e80 con 0x7f52a40b6f20 2022-06-29T15:51:56.605 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:51:56.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.598+0000 7f52aafef700 1 -- 172.21.15.47:0/4101606593 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5284053b00 msgr2=0x7f5284055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:56.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.598+0000 7f52aafef700 1 --2- 172.21.15.47:0/4101606593 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5284053b00 0x7f5284055fc0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7f529801c000 tx=0x7f5298015000).stop 2022-06-29T15:51:56.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.598+0000 7f52aafef700 1 -- 172.21.15.47:0/4101606593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52a40b6f20 msgr2=0x7f52a40b73d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:56.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.598+0000 7f52aafef700 1 --2- 172.21.15.47:0/4101606593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52a40b6f20 0x7f52a40b73d0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 rx=0x7f52a4066a80 tx=0x7f529c007f10).stop 2022-06-29T15:51:56.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.602+0000 7f52aafef700 1 -- 172.21.15.47:0/4101606593 shutdown_connections 2022-06-29T15:51:56.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.602+0000 7f52aafef700 1 --2- 172.21.15.47:0/4101606593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52a40b6f20 0x7f52a40b73d0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:56.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.602+0000 7f52aafef700 1 --2- 172.21.15.47:0/4101606593 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f5284053b00 0x7f5284055fc0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:56.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.602+0000 7f52aafef700 1 --2- 172.21.15.47:0/4101606593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52a413e550 0x7f52a4144d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:56.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.602+0000 7f52aafef700 1 --2- 172.21.15.47:0/4101606593 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f52a4067500 0x7f52a40b5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:56.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.602+0000 7f52aafef700 1 -- 172.21.15.47:0/4101606593 >> 172.21.15.47:0/4101606593 conn(0x7f52a40627d0 msgr2=0x7f52a40b2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:56.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.602+0000 7f52aafef700 1 -- 172.21.15.47:0/4101606593 shutdown_connections 2022-06-29T15:51:56.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.602+0000 7f52aafef700 1 -- 172.21.15.47:0/4101606593 wait complete. 2022-06-29T15:51:56.619 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-29T15:51:56.725 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.718+0000 7f2450248700 1 Processor -- start 2022-06-29T15:51:56.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.718+0000 7f2450248700 1 -- start start 2022-06-29T15:51:56.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.718+0000 7f2450248700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2448060ee0 0x7f24480612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:56.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.718+0000 7f2450248700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f24480b4300 0x7f24481471a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:56.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.718+0000 7f2450248700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2448067610 con 0x7f2448066f70 2022-06-29T15:51:56.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.718+0000 7f2450248700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24480673a0 con 0x7f24480b4300 2022-06-29T15:51:56.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.718+0000 7f2450248700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24481467c0 con 0x7f2448060ee0 2022-06-29T15:51:56.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.722+0000 7f244d7e3700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f24480b4300 0x7f24481471a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:56.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.722+0000 7f244d7e3700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f24480b4300 0x7f24481471a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54546/0 (socket says 172.21.15.47:54546) 2022-06-29T15:51:56.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.722+0000 7f244d7e3700 1 -- 172.21.15.47:0/3491957887 learned_addr learned my addr 172.21.15.47:0/3491957887 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:56.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.722+0000 7f244dfe4700 1 --2- 172.21.15.47:0/3491957887 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2448060ee0 0x7f24480612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:56.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.722+0000 7f244d7e3700 1 -- 172.21.15.47:0/3491957887 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2448060ee0 msgr2=0x7f24480612c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:56.731 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.722+0000 7f244cfe2700 1 -- 172.21.15.47:0/3491957887 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1759345348 0 0) 0x7f2448067610 con 0x7f2448066f70 2022-06-29T15:51:56.731 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.722+0000 7f244d7e3700 1 --2- 172.21.15.47:0/3491957887 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2448060ee0 0x7f24480612c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:56.731 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.722+0000 7f244d7e3700 1 -- 172.21.15.47:0/3491957887 >> v1:172.21.15.64:6789/0 conn(0x7f2448066f70 legacy=0x7f24481430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:56.731 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.722+0000 7f244d7e3700 1 -- 172.21.15.47:0/3491957887 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2448146a60 con 0x7f24480b4300 2022-06-29T15:51:56.732 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.722+0000 7f244d7e3700 1 --2- 172.21.15.47:0/3491957887 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24480b4300 0x7f24481471a0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 rx=0x7f2444002f70 tx=0x7f2444006eb0).ready entity=mon.0 client_cookie=562f98b044e5d386 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:56.732 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.722+0000 7f244cfe2700 1 -- 172.21.15.47:0/3491957887 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f244400d570 con 0x7f24480b4300 2022-06-29T15:51:56.732 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.722+0000 7f244cfe2700 1 -- 172.21.15.47:0/3491957887 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2444000e90 con 0x7f24480b4300 2022-06-29T15:51:56.732 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.722+0000 7f244cfe2700 1 -- 172.21.15.47:0/3491957887 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f244400d8c0 con 0x7f24480b4300 2022-06-29T15:51:56.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.722+0000 7f2450248700 1 -- 172.21.15.47:0/3491957887 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24480b4300 msgr2=0x7f24481471a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:56.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.722+0000 7f2450248700 1 --2- 172.21.15.47:0/3491957887 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24480b4300 0x7f24481471a0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 rx=0x7f2444002f70 tx=0x7f2444006eb0).stop 2022-06-29T15:51:56.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.722+0000 7f2450248700 1 -- 172.21.15.47:0/3491957887 shutdown_connections 2022-06-29T15:51:56.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.722+0000 7f2450248700 1 --2- 172.21.15.47:0/3491957887 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24480b4300 0x7f24481471a0 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:56.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.722+0000 7f2450248700 1 --2- 172.21.15.47:0/3491957887 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2448060ee0 0x7f24480612c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:56.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.722+0000 7f2450248700 1 -- 172.21.15.47:0/3491957887 >> 172.21.15.47:0/3491957887 conn(0x7f244805c510 msgr2=0x7f244805f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:56.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.722+0000 7f2450248700 1 -- 172.21.15.47:0/3491957887 shutdown_connections 2022-06-29T15:51:56.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.722+0000 7f2450248700 1 -- 172.21.15.47:0/3491957887 wait complete. 2022-06-29T15:51:56.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.722+0000 7f2450248700 1 Processor -- start 2022-06-29T15:51:56.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.722+0000 7f2450248700 1 -- start start 2022-06-29T15:51:56.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.726+0000 7f2450248700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2448060ee0 0x7f2448144820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:56.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.726+0000 7f2450248700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2448066f70 0x7f24481360e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:56.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.726+0000 7f2450248700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24480b4300 0x7f24481365d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:56.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.726+0000 7f2450248700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244813c260 con 0x7f2448066f70 2022-06-29T15:51:56.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.726+0000 7f2450248700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2448149250 con 0x7f24480b4300 2022-06-29T15:51:56.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.726+0000 7f2450248700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2448148ce0 con 0x7f2448060ee0 2022-06-29T15:51:56.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.726+0000 7f244d7e3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2448066f70 0x7f24481360e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:56.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.726+0000 7f244d7e3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2448066f70 0x7f24481360e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54554/0 (socket says 172.21.15.47:54554) 2022-06-29T15:51:56.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.726+0000 7f244dfe4700 1 --2- 172.21.15.47:0/447615649 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2448060ee0 0x7f2448144820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:56.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.726+0000 7f244d7e3700 1 -- 172.21.15.47:0/447615649 learned_addr learned my addr 172.21.15.47:0/447615649 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:56.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.726+0000 7f244e7e5700 1 --2- 172.21.15.47:0/447615649 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24480b4300 0x7f24481365d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:56.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.726+0000 7f244dfe4700 1 -- 172.21.15.47:0/447615649 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24480b4300 msgr2=0x7f24481365d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:56.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.726+0000 7f244dfe4700 1 --2- 172.21.15.47:0/447615649 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24480b4300 0x7f24481365d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:56.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.726+0000 7f244dfe4700 1 -- 172.21.15.47:0/447615649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2448066f70 msgr2=0x7f24481360e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:56.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.726+0000 7f244dfe4700 1 --2- 172.21.15.47:0/447615649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2448066f70 0x7f24481360e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:56.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.726+0000 7f244dfe4700 1 -- 172.21.15.47:0/447615649 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f244400b1e0 con 0x7f2448060ee0 2022-06-29T15:51:56.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.726+0000 7f244d7e3700 1 --2- 172.21.15.47:0/447615649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2448066f70 0x7f24481360e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:51:56.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.726+0000 7f244dfe4700 1 --2- 172.21.15.47:0/447615649 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2448060ee0 0x7f2448144820 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f243c00c000 tx=0x7f243c006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:56.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.726+0000 7f243affd700 1 -- 172.21.15.47:0/447615649 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f243c010780 con 0x7f2448060ee0 2022-06-29T15:51:56.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.726+0000 7f243affd700 1 -- 172.21.15.47:0/447615649 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f243c0108e0 con 0x7f2448060ee0 2022-06-29T15:51:56.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.726+0000 7f2450248700 1 -- 172.21.15.47:0/447615649 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2448147f50 con 0x7f2448060ee0 2022-06-29T15:51:56.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.726+0000 7f243affd700 1 -- 172.21.15.47:0/447615649 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f243c010c40 con 0x7f2448060ee0 2022-06-29T15:51:56.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.726+0000 7f2450248700 1 -- 172.21.15.47:0/447615649 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2448136ac0 con 0x7f2448060ee0 2022-06-29T15:51:56.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.726+0000 7f2450248700 1 -- 172.21.15.47:0/447615649 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24480573d0 con 0x7f2448060ee0 2022-06-29T15:51:56.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.726+0000 7f243affd700 1 -- 172.21.15.47:0/447615649 <== mon.2 v2:172.21.15.64:3301/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f243c01c460 con 0x7f2448060ee0 2022-06-29T15:51:56.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.730+0000 7f243affd700 1 --2- 172.21.15.47:0/447615649 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2428053b00 0x7f2428055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:56.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.730+0000 7f243affd700 1 -- 172.21.15.47:0/447615649 <== mon.2 v2:172.21.15.64:3301/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f243c06bda0 con 0x7f2448060ee0 2022-06-29T15:51:56.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.730+0000 7f244d7e3700 1 --2- 172.21.15.47:0/447615649 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2428053b00 0x7f2428055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:56.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.730+0000 7f244d7e3700 1 --2- 172.21.15.47:0/447615649 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2428053b00 0x7f2428055fc0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7f244401c000 tx=0x7f2444015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:56.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.738+0000 7f243affd700 1 -- 172.21.15.47:0/447615649 <== mon.2 v2:172.21.15.64:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f243c042a60 con 0x7f2448060ee0 2022-06-29T15:51:56.873 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.866+0000 7f2450248700 1 -- 172.21.15.47:0/447615649 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f24480b5e50 con 0x7f2448060ee0 2022-06-29T15:51:56.874 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.866+0000 7f243affd700 1 -- 172.21.15.47:0/447615649 <== mon.2 v2:172.21.15.64:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v151) v1 ==== 104+0+10 (secure 0 0 0) 0x7f243c046080 con 0x7f2448060ee0 2022-06-29T15:51:56.874 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:51:56.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.870+0000 7f2450248700 1 -- 172.21.15.47:0/447615649 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2428053b00 msgr2=0x7f2428055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:56.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.870+0000 7f2450248700 1 --2- 172.21.15.47:0/447615649 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2428053b00 0x7f2428055fc0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7f244401c000 tx=0x7f2444015000).stop 2022-06-29T15:51:56.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.870+0000 7f2450248700 1 -- 172.21.15.47:0/447615649 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2448060ee0 msgr2=0x7f2448144820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:56.877 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.870+0000 7f2450248700 1 --2- 172.21.15.47:0/447615649 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2448060ee0 0x7f2448144820 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f243c00c000 tx=0x7f243c006df0).stop 2022-06-29T15:51:56.877 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.870+0000 7f2450248700 1 -- 172.21.15.47:0/447615649 shutdown_connections 2022-06-29T15:51:56.877 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.870+0000 7f2450248700 1 --2- 172.21.15.47:0/447615649 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2428053b00 0x7f2428055fc0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:56.877 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.870+0000 7f2450248700 1 --2- 172.21.15.47:0/447615649 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24480b4300 0x7f24481365d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:56.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.870+0000 7f2450248700 1 --2- 172.21.15.47:0/447615649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2448066f70 0x7f24481360e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:56.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.870+0000 7f2450248700 1 --2- 172.21.15.47:0/447615649 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f2448060ee0 0x7f2448144820 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:56.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.870+0000 7f2450248700 1 -- 172.21.15.47:0/447615649 >> 172.21.15.47:0/447615649 conn(0x7f244805c510 msgr2=0x7f24480b6110 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:56.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.870+0000 7f2450248700 1 -- 172.21.15.47:0/447615649 shutdown_connections 2022-06-29T15:51:56.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.870+0000 7f2450248700 1 -- 172.21.15.47:0/447615649 wait complete. 2022-06-29T15:51:56.888 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-29T15:51:57.003 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.998+0000 7fcaf5eb2700 1 Processor -- start 2022-06-29T15:51:57.004 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.998+0000 7fcaf5eb2700 1 -- start start 2022-06-29T15:51:57.006 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.998+0000 7fcaf5eb2700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcaf01439c0 0x7fcaf013fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:57.006 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.998+0000 7fcaf5eb2700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcaf013f7b0 0x7fcaf01401d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:57.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.998+0000 7fcaf5eb2700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaf00b0e80 con 0x7fcaf00abae0 2022-06-29T15:51:57.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.998+0000 7fcaf5eb2700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaf00abec0 con 0x7fcaf01439c0 2022-06-29T15:51:57.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.998+0000 7fcaf5eb2700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaf00b0940 con 0x7fcaf013f7b0 2022-06-29T15:51:57.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.998+0000 7fcaeffff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcaf01439c0 0x7fcaf013fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:57.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.998+0000 7fcaeffff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcaf01439c0 0x7fcaf013fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54564/0 (socket says 172.21.15.47:54564) 2022-06-29T15:51:57.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.998+0000 7fcaeffff700 1 -- 172.21.15.47:0/715591516 learned_addr learned my addr 172.21.15.47:0/715591516 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:57.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.998+0000 7fcaf56b1700 1 --2- 172.21.15.47:0/715591516 >> v2:172.21.15.64:3300/0 conn(0x7fcaf013f7b0 0x7fcaf01401d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:57.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.998+0000 7fcaeffff700 1 -- 172.21.15.47:0/715591516 >> v2:172.21.15.64:3300/0 conn(0x7fcaf013f7b0 msgr2=0x7fcaf01401d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:57.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.998+0000 7fcaeffff700 1 --2- 172.21.15.47:0/715591516 >> v2:172.21.15.64:3300/0 conn(0x7fcaf013f7b0 0x7fcaf01401d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:57.009 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.998+0000 7fcaeffff700 1 -- 172.21.15.47:0/715591516 >> v1:172.21.15.64:6789/0 conn(0x7fcaf00abae0 legacy=0x7fcaf01432b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:57.009 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.998+0000 7fcaeffff700 1 -- 172.21.15.47:0/715591516 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf00b0be0 con 0x7fcaf01439c0 2022-06-29T15:51:57.009 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.998+0000 7fcaef7fe700 1 -- 172.21.15.47:0/715591516 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4281545412 0 0) 0x7fcaf00b0e80 con 0x7fcaf00abae0 2022-06-29T15:51:57.009 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.998+0000 7fcaeffff700 1 --2- 172.21.15.47:0/715591516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcaf01439c0 0x7fcaf013fce0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 rx=0x7fcadc002f70 tx=0x7fcadc006f40).ready entity=mon.0 client_cookie=fb777a5fd42c53d4 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:57.010 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.998+0000 7fcaef7fe700 1 -- 172.21.15.47:0/715591516 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcadc00c880 con 0x7fcaf01439c0 2022-06-29T15:51:57.010 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.998+0000 7fcaef7fe700 1 -- 172.21.15.47:0/715591516 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcadc000e90 con 0x7fcaf01439c0 2022-06-29T15:51:57.010 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:56.998+0000 7fcaef7fe700 1 -- 172.21.15.47:0/715591516 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcadc00cbd0 con 0x7fcaf01439c0 2022-06-29T15:51:57.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.002+0000 7fcaf5eb2700 1 -- 172.21.15.47:0/715591516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcaf01439c0 msgr2=0x7fcaf013fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:57.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.002+0000 7fcaf5eb2700 1 --2- 172.21.15.47:0/715591516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcaf01439c0 0x7fcaf013fce0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 rx=0x7fcadc002f70 tx=0x7fcadc006f40).stop 2022-06-29T15:51:57.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.002+0000 7fcaf5eb2700 1 -- 172.21.15.47:0/715591516 shutdown_connections 2022-06-29T15:51:57.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.002+0000 7fcaf5eb2700 1 --2- 172.21.15.47:0/715591516 >> v2:172.21.15.64:3300/0 conn(0x7fcaf013f7b0 0x7fcaf01401d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:57.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.002+0000 7fcaf5eb2700 1 --2- 172.21.15.47:0/715591516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcaf01439c0 0x7fcaf013fce0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:57.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.002+0000 7fcaf5eb2700 1 -- 172.21.15.47:0/715591516 >> 172.21.15.47:0/715591516 conn(0x7fcaf0062560 msgr2=0x7fcaf0062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:57.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.002+0000 7fcaf5eb2700 1 -- 172.21.15.47:0/715591516 shutdown_connections 2022-06-29T15:51:57.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.002+0000 7fcaf5eb2700 1 -- 172.21.15.47:0/715591516 wait complete. 2022-06-29T15:51:57.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.002+0000 7fcaf5eb2700 1 Processor -- start 2022-06-29T15:51:57.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.002+0000 7fcaf5eb2700 1 -- start start 2022-06-29T15:51:57.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.006+0000 7fcaf5eb2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcaf00abae0 0x7fcaf013a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:57.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.006+0000 7fcaf5eb2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaf01439c0 0x7fcaf0066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:57.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.006+0000 7fcaf5eb2700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcaf0142310 0x7fcaf0066a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:57.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.006+0000 7fcaf5eb2700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf0064860 con 0x7fcaf00abae0 2022-06-29T15:51:57.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.006+0000 7fcaf5eb2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf0154480 con 0x7fcaf01439c0 2022-06-29T15:51:57.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.006+0000 7fcaf5eb2700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf0150520 con 0x7fcaf0142310 2022-06-29T15:51:57.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.006+0000 7fcaf4eb0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcaf00abae0 0x7fcaf013a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:57.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.006+0000 7fcaf4eb0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcaf00abae0 0x7fcaf013a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54566/0 (socket says 172.21.15.47:54566) 2022-06-29T15:51:57.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.006+0000 7fcaf4eb0700 1 -- 172.21.15.47:0/222044510 learned_addr learned my addr 172.21.15.47:0/222044510 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:57.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.006+0000 7fcaeffff700 1 --2- 172.21.15.47:0/222044510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaf01439c0 0x7fcaf0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:57.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.006+0000 7fcaf56b1700 1 --2- 172.21.15.47:0/222044510 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcaf0142310 0x7fcaf0066a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:57.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.006+0000 7fcaf4eb0700 1 -- 172.21.15.47:0/222044510 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcaf0142310 msgr2=0x7fcaf0066a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:57.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.006+0000 7fcaf4eb0700 1 --2- 172.21.15.47:0/222044510 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcaf0142310 0x7fcaf0066a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:57.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.006+0000 7fcaf4eb0700 1 -- 172.21.15.47:0/222044510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaf01439c0 msgr2=0x7fcaf0066570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:57.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.006+0000 7fcaf4eb0700 1 --2- 172.21.15.47:0/222044510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaf01439c0 0x7fcaf0066570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:57.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.006+0000 7fcaf4eb0700 1 -- 172.21.15.47:0/222044510 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcadc00a360 con 0x7fcaf00abae0 2022-06-29T15:51:57.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.006+0000 7fcaf56b1700 1 --2- 172.21.15.47:0/222044510 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcaf0142310 0x7fcaf0066a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:51:57.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.006+0000 7fcaf4eb0700 1 --2- 172.21.15.47:0/222044510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcaf00abae0 0x7fcaf013a9e0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 rx=0x7fcae4000ed0 tx=0x7fcae4007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:57.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.006+0000 7fcaedffb700 1 -- 172.21.15.47:0/222044510 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcae4012400 con 0x7fcaf00abae0 2022-06-29T15:51:57.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.006+0000 7fcaedffb700 1 -- 172.21.15.47:0/222044510 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcae4012560 con 0x7fcaf00abae0 2022-06-29T15:51:57.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.006+0000 7fcaf5eb2700 1 -- 172.21.15.47:0/222044510 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaf0064a90 con 0x7fcaf00abae0 2022-06-29T15:51:57.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.006+0000 7fcaedffb700 1 -- 172.21.15.47:0/222044510 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcae40128c0 con 0x7fcaf00abae0 2022-06-29T15:51:57.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.006+0000 7fcaf5eb2700 1 -- 172.21.15.47:0/222044510 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaf013b540 con 0x7fcaf00abae0 2022-06-29T15:51:57.022 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.010+0000 7fcaedffb700 1 -- 172.21.15.47:0/222044510 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fcae4019030 con 0x7fcaf00abae0 2022-06-29T15:51:57.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.010+0000 7fcaf5eb2700 1 -- 172.21.15.47:0/222044510 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaf01548e0 con 0x7fcaf00abae0 2022-06-29T15:51:57.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.010+0000 7fcaedffb700 1 --2- 172.21.15.47:0/222044510 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcad0053aa0 0x7fcad0055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:57.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.010+0000 7fcaedffb700 1 -- 172.21.15.47:0/222044510 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fcae406b330 con 0x7fcaf00abae0 2022-06-29T15:51:57.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.010+0000 7fcaeffff700 1 --2- 172.21.15.47:0/222044510 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcad0053aa0 0x7fcad0055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:57.024 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.010+0000 7fcaeffff700 1 --2- 172.21.15.47:0/222044510 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcad0053aa0 0x7fcad0055f60 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7fcadc01b000 tx=0x7fcaf0142d00).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:57.024 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.018+0000 7fcaedffb700 1 -- 172.21.15.47:0/222044510 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcae40421a0 con 0x7fcaf00abae0 2022-06-29T15:51:57.157 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.150+0000 7fcaf5eb2700 1 -- 172.21.15.47:0/222044510 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcaf013b070 con 0x7fcaf00abae0 2022-06-29T15:51:57.157 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.150+0000 7fcaedffb700 1 -- 172.21.15.47:0/222044510 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v151) v1 ==== 100+0+10 (secure 0 0 0) 0x7fcae40457c0 con 0x7fcaf00abae0 2022-06-29T15:51:57.158 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:51:57.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.154+0000 7fcaf5eb2700 1 -- 172.21.15.47:0/222044510 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcad0053aa0 msgr2=0x7fcad0055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:57.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.154+0000 7fcaf5eb2700 1 --2- 172.21.15.47:0/222044510 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcad0053aa0 0x7fcad0055f60 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7fcadc01b000 tx=0x7fcaf0142d00).stop 2022-06-29T15:51:57.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.154+0000 7fcaf5eb2700 1 -- 172.21.15.47:0/222044510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcaf00abae0 msgr2=0x7fcaf013a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:57.161 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.154+0000 7fcaf5eb2700 1 --2- 172.21.15.47:0/222044510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcaf00abae0 0x7fcaf013a9e0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 rx=0x7fcae4000ed0 tx=0x7fcae4007ac0).stop 2022-06-29T15:51:57.161 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.154+0000 7fcaf5eb2700 1 -- 172.21.15.47:0/222044510 shutdown_connections 2022-06-29T15:51:57.161 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.154+0000 7fcaf5eb2700 1 --2- 172.21.15.47:0/222044510 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fcaf0142310 0x7fcaf0066a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:57.161 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.154+0000 7fcaf5eb2700 1 --2- 172.21.15.47:0/222044510 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcad0053aa0 0x7fcad0055f60 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:57.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.154+0000 7fcaf5eb2700 1 --2- 172.21.15.47:0/222044510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaf01439c0 0x7fcaf0066570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:57.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.154+0000 7fcaf5eb2700 1 --2- 172.21.15.47:0/222044510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcaf00abae0 0x7fcaf013a9e0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:57.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.154+0000 7fcaf5eb2700 1 -- 172.21.15.47:0/222044510 >> 172.21.15.47:0/222044510 conn(0x7fcaf0062560 msgr2=0x7fcaf00ae8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:57.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.154+0000 7fcaf5eb2700 1 -- 172.21.15.47:0/222044510 shutdown_connections 2022-06-29T15:51:57.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.154+0000 7fcaf5eb2700 1 -- 172.21.15.47:0/222044510 wait complete. 2022-06-29T15:51:57.171 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-29T15:51:57.172 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:51:57.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.270+0000 7f6fb273f700 1 Processor -- start 2022-06-29T15:51:57.276 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.270+0000 7f6fb273f700 1 -- start start 2022-06-29T15:51:57.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.270+0000 7f6fb273f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6fac0b5c60 0x7f6fac144980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:57.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.270+0000 7f6fb273f700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6fac0a72f0 0x7f6fac0b2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:57.278 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.270+0000 7f6fb273f700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fac0a9120 con 0x7f6fac060eb0 2022-06-29T15:51:57.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.270+0000 7f6fb273f700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fac0b0b80 con 0x7f6fac0b5c60 2022-06-29T15:51:57.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.270+0000 7f6fb273f700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fac0a89e0 con 0x7f6fac0a72f0 2022-06-29T15:51:57.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.270+0000 7f6fb1f3e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6fac0a72f0 0x7f6fac0b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:57.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.270+0000 7f6fb1f3e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6fac0a72f0 0x7f6fac0b2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:51680/0 (socket says 172.21.15.47:51680) 2022-06-29T15:51:57.279 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.270+0000 7f6fb1f3e700 1 -- 172.21.15.47:0/1417729016 learned_addr learned my addr 172.21.15.47:0/1417729016 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:57.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.270+0000 7f6fb173d700 1 --2- 172.21.15.47:0/1417729016 >> v2:172.21.15.64:3300/0 conn(0x7f6fac0b5c60 0x7f6fac144980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:57.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.270+0000 7f6fb1f3e700 1 -- 172.21.15.47:0/1417729016 >> v2:172.21.15.64:3300/0 conn(0x7f6fac0b5c60 msgr2=0x7f6fac144980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:57.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.270+0000 7f6fb1f3e700 1 --2- 172.21.15.47:0/1417729016 >> v2:172.21.15.64:3300/0 conn(0x7f6fac0b5c60 0x7f6fac144980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:57.280 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.270+0000 7f6fb1f3e700 1 -- 172.21.15.47:0/1417729016 >> v1:172.21.15.64:6789/0 conn(0x7f6fac060eb0 legacy=0x7f6fac0b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:57.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.270+0000 7f6fb1f3e700 1 -- 172.21.15.47:0/1417729016 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fac0a8c80 con 0x7f6fac0a72f0 2022-06-29T15:51:57.281 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.270+0000 7f6fb1f3e700 1 --2- 172.21.15.47:0/1417729016 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6fac0a72f0 0x7f6fac0b2540 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f6fa40032f0 tx=0x7f6fa4009290).ready entity=mon.2 client_cookie=32fba9c584a0a03 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:57.285 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.270+0000 7f6fa3fff700 1 -- 172.21.15.47:0/1417729016 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6fa400b8a0 con 0x7f6fac0a72f0 2022-06-29T15:51:57.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.270+0000 7f6fa3fff700 1 -- 172.21.15.47:0/1417729016 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa400ba00 con 0x7f6fac0a72f0 2022-06-29T15:51:57.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.270+0000 7f6fa3fff700 1 -- 172.21.15.47:0/1417729016 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6fa400bb70 con 0x7f6fac0a72f0 2022-06-29T15:51:57.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.274+0000 7f6fb273f700 1 -- 172.21.15.47:0/1417729016 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6fac0a72f0 msgr2=0x7f6fac0b2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:57.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.274+0000 7f6fb273f700 1 --2- 172.21.15.47:0/1417729016 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6fac0a72f0 0x7f6fac0b2540 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f6fa40032f0 tx=0x7f6fa4009290).stop 2022-06-29T15:51:57.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.274+0000 7f6fb273f700 1 -- 172.21.15.47:0/1417729016 shutdown_connections 2022-06-29T15:51:57.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.274+0000 7f6fb273f700 1 --2- 172.21.15.47:0/1417729016 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6fac0a72f0 0x7f6fac0b2540 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:57.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.274+0000 7f6fb273f700 1 --2- 172.21.15.47:0/1417729016 >> v2:172.21.15.64:3300/0 conn(0x7f6fac0b5c60 0x7f6fac144980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:57.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.274+0000 7f6fb273f700 1 -- 172.21.15.47:0/1417729016 >> 172.21.15.47:0/1417729016 conn(0x7f6fac062560 msgr2=0x7f6fac062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:57.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.274+0000 7f6fb273f700 1 -- 172.21.15.47:0/1417729016 shutdown_connections 2022-06-29T15:51:57.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.274+0000 7f6fb273f700 1 -- 172.21.15.47:0/1417729016 wait complete. 2022-06-29T15:51:57.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.274+0000 7f6fb273f700 1 Processor -- start 2022-06-29T15:51:57.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.274+0000 7f6fb273f700 1 -- start start 2022-06-29T15:51:57.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.274+0000 7f6fb273f700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6fac060eb0 0x7f6fac1457b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:57.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.274+0000 7f6fb273f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fac0a72f0 0x7f6fac14bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:57.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.274+0000 7f6fb273f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fac0b5c60 0x7f6fac156bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:57.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.274+0000 7f6fb273f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fac15a640 con 0x7f6fac0b5c60 2022-06-29T15:51:57.289 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.274+0000 7f6fb273f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fac15a8b0 con 0x7f6fac0a72f0 2022-06-29T15:51:57.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.274+0000 7f6fb273f700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fac148a90 con 0x7f6fac060eb0 2022-06-29T15:51:57.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.274+0000 7f6fb1f3e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fac0b5c60 0x7f6fac156bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:57.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.274+0000 7f6fb1f3e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fac0b5c60 0x7f6fac156bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54582/0 (socket says 172.21.15.47:54582) 2022-06-29T15:51:57.290 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.274+0000 7f6fb1f3e700 1 -- 172.21.15.47:0/781413897 learned_addr learned my addr 172.21.15.47:0/781413897 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:57.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.274+0000 7f6fb173d700 1 --2- 172.21.15.47:0/781413897 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6fac060eb0 0x7f6fac1457b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:57.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.274+0000 7f6fb0f3c700 1 --2- 172.21.15.47:0/781413897 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fac0a72f0 0x7f6fac14bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:57.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.278+0000 7f6fb1f3e700 1 -- 172.21.15.47:0/781413897 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6fac060eb0 msgr2=0x7f6fac1457b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:57.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.278+0000 7f6fb1f3e700 1 --2- 172.21.15.47:0/781413897 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6fac060eb0 0x7f6fac1457b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:57.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.278+0000 7f6fb1f3e700 1 -- 172.21.15.47:0/781413897 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fac0a72f0 msgr2=0x7f6fac14bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:57.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.278+0000 7f6fb1f3e700 1 --2- 172.21.15.47:0/781413897 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fac0a72f0 0x7f6fac14bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:57.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.278+0000 7f6fb1f3e700 1 -- 172.21.15.47:0/781413897 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa4009530 con 0x7f6fac0b5c60 2022-06-29T15:51:57.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.278+0000 7f6fb1f3e700 1 --2- 172.21.15.47:0/781413897 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fac0b5c60 0x7f6fac156bf0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 rx=0x7f6fa40032f0 tx=0x7f6fa4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:57.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.278+0000 7f6fa27fc700 1 -- 172.21.15.47:0/781413897 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6fa400b8a0 con 0x7f6fac0b5c60 2022-06-29T15:51:57.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.278+0000 7f6fa27fc700 1 -- 172.21.15.47:0/781413897 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa4021630 con 0x7f6fac0b5c60 2022-06-29T15:51:57.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.278+0000 7f6fb273f700 1 -- 172.21.15.47:0/781413897 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fac156720 con 0x7f6fac0b5c60 2022-06-29T15:51:57.293 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.278+0000 7f6fa27fc700 1 -- 172.21.15.47:0/781413897 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6fa4021990 con 0x7f6fac0b5c60 2022-06-29T15:51:57.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.278+0000 7f6fb273f700 1 -- 172.21.15.47:0/781413897 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fac143000 con 0x7f6fac0b5c60 2022-06-29T15:51:57.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.278+0000 7f6fb273f700 1 -- 172.21.15.47:0/781413897 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fac148c50 con 0x7f6fac0b5c60 2022-06-29T15:51:57.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.278+0000 7f6fa27fc700 1 -- 172.21.15.47:0/781413897 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6fa4027020 con 0x7f6fac0b5c60 2022-06-29T15:51:57.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.282+0000 7f6fa27fc700 1 --2- 172.21.15.47:0/781413897 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6f94053950 0x7f6f94055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:57.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.282+0000 7f6fa27fc700 1 -- 172.21.15.47:0/781413897 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f6fa406d260 con 0x7f6fac0b5c60 2022-06-29T15:51:57.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.282+0000 7f6fb173d700 1 --2- 172.21.15.47:0/781413897 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6f94053950 0x7f6f94055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:57.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.282+0000 7f6fb173d700 1 --2- 172.21.15.47:0/781413897 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6f94053950 0x7f6f94055e10 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7f6f9c006000 tx=0x7f6f9c006d90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:57.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.286+0000 7f6fa27fc700 1 -- 172.21.15.47:0/781413897 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6fa40440d0 con 0x7f6fac0b5c60 2022-06-29T15:51:57.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.446+0000 7f6fb273f700 1 -- 172.21.15.47:0/781413897 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6fac0573d0 con 0x7f6fac0b5c60 2022-06-29T15:51:57.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.446+0000 7f6fa27fc700 1 -- 172.21.15.47:0/781413897 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+2495 (secure 0 0 0) 0x7f6fa4015020 con 0x7f6fac0b5c60 2022-06-29T15:51:57.454 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:57.455 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":74,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8296,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/3321939421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3321939421},{"type":"v1","addr":"172.21.15.64:6833","nonce":3321939421}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":72},{"gid":8415,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1886881850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1886881850},{"type":"v1","addr":"172.21.15.47:6835","nonce":1886881850}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":72}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:51:52.828119+0000","modified":"2022-06-29T15:51:53.831305+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8421":{"gid":8421,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6837/2466961065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2466961065},{"type":"v1","addr":"172.21.15.47:6837","nonce":2466961065}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2022-06-29T15:51:57.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.450+0000 7f6fb273f700 1 -- 172.21.15.47:0/781413897 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6f94053950 msgr2=0x7f6f94055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:57.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.450+0000 7f6fb273f700 1 --2- 172.21.15.47:0/781413897 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6f94053950 0x7f6f94055e10 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7f6f9c006000 tx=0x7f6f9c006d90).stop 2022-06-29T15:51:57.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.450+0000 7f6fb273f700 1 -- 172.21.15.47:0/781413897 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fac0b5c60 msgr2=0x7f6fac156bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:57.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.450+0000 7f6fb273f700 1 --2- 172.21.15.47:0/781413897 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fac0b5c60 0x7f6fac156bf0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 rx=0x7f6fa40032f0 tx=0x7f6fa4016000).stop 2022-06-29T15:51:57.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.450+0000 7f6fb273f700 1 -- 172.21.15.47:0/781413897 shutdown_connections 2022-06-29T15:51:57.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.450+0000 7f6fb273f700 1 --2- 172.21.15.47:0/781413897 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fac0b5c60 0x7f6fac156bf0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:57.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.450+0000 7f6fb273f700 1 --2- 172.21.15.47:0/781413897 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6f94053950 0x7f6f94055e10 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:57.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.450+0000 7f6fb273f700 1 --2- 172.21.15.47:0/781413897 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fac0a72f0 0x7f6fac14bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:57.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.450+0000 7f6fb273f700 1 --2- 172.21.15.47:0/781413897 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6fac060eb0 0x7f6fac1457b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:57.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.450+0000 7f6fb273f700 1 -- 172.21.15.47:0/781413897 >> 172.21.15.47:0/781413897 conn(0x7f6fac062560 msgr2=0x7f6fac144750 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:57.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.450+0000 7f6fb273f700 1 -- 172.21.15.47:0/781413897 shutdown_connections 2022-06-29T15:51:57.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:57.450+0000 7f6fb273f700 1 -- 172.21.15.47:0/781413897 wait complete. 2022-06-29T15:51:57.459 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 74 2022-06-29T15:51:57.770 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-29T15:51:52.828119+0000', 'modified': '2022-06-29T15:51:53.831305+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8421}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8421': {'gid': 8421, 'name': 'c', 'rank': 0, 'incarnation': 73, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.47:6837/2466961065', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6836', 'nonce': 2466961065}, {'type': 'v1', 'addr': '172.21.15.47:6837', 'nonce': 2466961065}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-29T15:51:57.771 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-29T15:51:57.771 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2022-06-29T15:51:57.893 INFO:teuthology.orchestra.run.smithi047.stdout:{"cluster_fsid":"9a68894a-53ce-400e-838b-14eab91b834c","whoami":0,"id":8421,"want_state":"up:active","state":"up:active","rank_uptime":5.0489603369999996,"mdsmap_epoch":74,"osdmap_epoch":151,"osdmap_epoch_barrier":150,"uptime":10.069778557999999} 2022-06-29T15:51:57.894 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9a68894a-53ce-400e-838b-14eab91b834c", "id": 8421, "mdsmap_epoch": 74, "osdmap_epoch": 151, "osdmap_epoch_barrier": 150, "rank_uptime": 5.048960337, "state": "up:active", "uptime": 10.069778558, "want_state": "up:active", "whoami": 0 } 2022-06-29T15:51:57.895 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-29T15:51:57.895 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-06-29T15:51:57.895 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi099.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-06-29T15:51:57.896 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:51:57.896 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:51:57.905 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2022-06-29T15:51:57.963 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:51:57.964 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:51:57.978 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:51:57.979 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:51:57.980 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:51:57.980 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2022-06-29T15:51:57.986 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-29T15:51:57.987 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:51:57.989 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2022-06-29T15:51:58.005 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:51:58.006 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:51:58.062 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:51:58.063 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:51:58.064 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:51:58.064 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2022-06-29T15:51:58.067 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[13519]: starting ceph client2022-06-29T15:51:58.061+0000 7efcd802e080 -1 init, newargv = 0x562043b4e460 newargc=9 2022-06-29T15:51:58.067 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr: 2022-06-29T15:51:58.078 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[13519]: starting fuse 2022-06-29T15:51:59.174 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2022-06-29T15:51:59.193 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:51:59.193 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:51:59.249 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:51:59.249 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:51:59.251 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:51:59.251 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2022-06-29T15:51:59.256 INFO:teuthology.orchestra.run.smithi099.stdout:52 2022-06-29T15:51:59.257 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-29T15:51:59.257 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:51:59.258 DEBUG:teuthology.orchestra.run.smithi099:> sudo python3 -c ' 2022-06-29T15:51:59.258 DEBUG:teuthology.orchestra.run.smithi099:> import glob 2022-06-29T15:51:59.259 DEBUG:teuthology.orchestra.run.smithi099:> import re 2022-06-29T15:51:59.259 DEBUG:teuthology.orchestra.run.smithi099:> import os 2022-06-29T15:51:59.259 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2022-06-29T15:51:59.260 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:51:59.260 DEBUG:teuthology.orchestra.run.smithi099:> def find_socket(client_name): 2022-06-29T15:51:59.261 DEBUG:teuthology.orchestra.run.smithi099:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-06-29T15:51:59.261 DEBUG:teuthology.orchestra.run.smithi099:> files = glob.glob(asok_path) 2022-06-29T15:51:59.262 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:51:59.262 DEBUG:teuthology.orchestra.run.smithi099:> # Given a non-glob path, it better be there 2022-06-29T15:51:59.262 DEBUG:teuthology.orchestra.run.smithi099:> if "*" not in asok_path: 2022-06-29T15:51:59.263 DEBUG:teuthology.orchestra.run.smithi099:> assert(len(files) == 1) 2022-06-29T15:51:59.263 DEBUG:teuthology.orchestra.run.smithi099:> return files[0] 2022-06-29T15:51:59.264 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:51:59.264 DEBUG:teuthology.orchestra.run.smithi099:> for f in files: 2022-06-29T15:51:59.265 DEBUG:teuthology.orchestra.run.smithi099:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-29T15:51:59.265 DEBUG:teuthology.orchestra.run.smithi099:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-29T15:51:59.265 DEBUG:teuthology.orchestra.run.smithi099:> return f 2022-06-29T15:51:59.266 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-29T15:51:59.266 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:51:59.267 DEBUG:teuthology.orchestra.run.smithi099:> print(find_socket("client.0")) 2022-06-29T15:51:59.267 DEBUG:teuthology.orchestra.run.smithi099:> ' 2022-06-29T15:51:59.341 INFO:teuthology.orchestra.run.smithi099.stdout:/var/run/ceph/ceph-client.0.13519.asok 2022-06-29T15:51:59.346 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.13519.asok 2022-06-29T15:51:59.347 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:51:59.347 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.13519.asok status 2022-06-29T15:51:59.441 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2022-06-29T15:51:59.441 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": { 2022-06-29T15:51:59.442 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_sha1": "f25ffac72e5a9e63860ff7deb6767a11f0559a16", 2022-06-29T15:51:59.442 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 15.2.16-304-gf25ffac7 (f25ffac72e5a9e63860ff7deb6767a11f0559a16) octopus (stable)", 2022-06-29T15:51:59.442 INFO:teuthology.orchestra.run.smithi099.stdout: "entity_id": "0", 2022-06-29T15:51:59.442 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": "smithi099", 2022-06-29T15:51:59.443 INFO:teuthology.orchestra.run.smithi099.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-06-29T15:51:59.443 INFO:teuthology.orchestra.run.smithi099.stdout: "pid": "13519", 2022-06-29T15:51:59.443 INFO:teuthology.orchestra.run.smithi099.stdout: "root": "/" 2022-06-29T15:51:59.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:51:59.443 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_count": 0, 2022-06-29T15:51:59.444 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_pinned_count": 0, 2022-06-29T15:51:59.444 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 8571, 2022-06-29T15:51:59.444 INFO:teuthology.orchestra.run.smithi099.stdout: "inst": { 2022-06-29T15:51:59.444 INFO:teuthology.orchestra.run.smithi099.stdout: "name": { 2022-06-29T15:51:59.445 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "client", 2022-06-29T15:51:59.445 INFO:teuthology.orchestra.run.smithi099.stdout: "num": 8571 2022-06-29T15:51:59.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:51:59.446 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2022-06-29T15:51:59.446 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2022-06-29T15:51:59.447 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:0", 2022-06-29T15:51:59.447 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2162058054 2022-06-29T15:51:59.447 INFO:teuthology.orchestra.run.smithi099.stdout: } 2022-06-29T15:51:59.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:51:59.448 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2022-06-29T15:51:59.448 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2022-06-29T15:51:59.448 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:0", 2022-06-29T15:51:59.448 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2162058054 2022-06-29T15:51:59.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:51:59.449 INFO:teuthology.orchestra.run.smithi099.stdout: "inst_str": "client.8571 172.21.15.99:0/2162058054", 2022-06-29T15:51:59.449 INFO:teuthology.orchestra.run.smithi099.stdout: "addr_str": "172.21.15.99:0/2162058054", 2022-06-29T15:51:59.449 INFO:teuthology.orchestra.run.smithi099.stdout: "inode_count": 1, 2022-06-29T15:51:59.449 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_epoch": 74, 2022-06-29T15:51:59.450 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch": 151, 2022-06-29T15:51:59.450 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch_barrier": 0, 2022-06-29T15:51:59.450 INFO:teuthology.orchestra.run.smithi099.stdout: "blacklisted": false, 2022-06-29T15:51:59.451 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs" 2022-06-29T15:51:59.451 INFO:teuthology.orchestra.run.smithi099.stdout:} 2022-06-29T15:51:59.453 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:51:59.453 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:51:59.454 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:51:59.462 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2022-06-29T15:51:59.463 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-29T15:51:59.463 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:51:59.464 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:51:59.524 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:51:59.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.630+0000 7f0df1645700 1 Processor -- start 2022-06-29T15:51:59.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.630+0000 7f0df1645700 1 -- start start 2022-06-29T15:51:59.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.634+0000 7f0df1645700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0dec0b2ee0 0x7f0dec0b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:59.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.634+0000 7f0df1645700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0dec0ad970 0x7f0dec1407e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:59.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.634+0000 7f0df1645700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dec1400e0 con 0x7f0dec0b0e50 2022-06-29T15:51:59.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.634+0000 7f0df1645700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dec13fef0 con 0x7f0dec0b2ee0 2022-06-29T15:51:59.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.634+0000 7f0df1645700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dec143810 con 0x7f0dec0ad970 2022-06-29T15:51:59.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.634+0000 7f0debfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0dec0b2ee0 0x7f0dec0b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:59.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.634+0000 7f0df0e44700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0dec0b0e50 0x7f0dec13d290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40146/0 (socket says 172.21.15.47:40146) 2022-06-29T15:51:59.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.634+0000 7f0df0e44700 1 -- 172.21.15.47:0/3605324968 learned_addr learned my addr 172.21.15.47:0/3605324968 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:51:59.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.634+0000 7f0de3fff700 1 --2- 172.21.15.47:0/3605324968 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0dec0ad970 0x7f0dec1407e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:59.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.634+0000 7f0debfff700 1 -- 172.21.15.47:0/3605324968 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0dec0ad970 msgr2=0x7f0dec1407e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:59.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.634+0000 7f0debfff700 1 --2- 172.21.15.47:0/3605324968 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0dec0ad970 0x7f0dec1407e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:59.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.634+0000 7f0debfff700 1 -- 172.21.15.47:0/3605324968 >> v1:172.21.15.47:6789/0 conn(0x7f0dec0b0e50 legacy=0x7f0dec13d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:59.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.634+0000 7f0debfff700 1 -- 172.21.15.47:0/3605324968 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dec143570 con 0x7f0dec0b2ee0 2022-06-29T15:51:59.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.634+0000 7f0deb7fe700 1 -- 172.21.15.47:0/3605324968 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4118286320 0 0) 0x7f0dec1400e0 con 0x7f0dec0b0e50 2022-06-29T15:51:59.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.634+0000 7f0debfff700 1 --2- 172.21.15.47:0/3605324968 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0dec0b2ee0 0x7f0dec0b32c0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 rx=0x7f0ddc002f70 tx=0x7f0ddc006df0).ready entity=mon.0 client_cookie=272dae00333ef77c server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:59.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.634+0000 7f0deb7fe700 1 -- 172.21.15.47:0/3605324968 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ddc00d570 con 0x7f0dec0b2ee0 2022-06-29T15:51:59.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.634+0000 7f0deb7fe700 1 -- 172.21.15.47:0/3605324968 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ddc000e90 con 0x7f0dec0b2ee0 2022-06-29T15:51:59.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.634+0000 7f0deb7fe700 1 -- 172.21.15.47:0/3605324968 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ddc00d8c0 con 0x7f0dec0b2ee0 2022-06-29T15:51:59.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.638+0000 7f0df1645700 1 -- 172.21.15.47:0/3605324968 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0dec0b2ee0 msgr2=0x7f0dec0b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:59.648 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.638+0000 7f0df1645700 1 --2- 172.21.15.47:0/3605324968 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0dec0b2ee0 0x7f0dec0b32c0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 rx=0x7f0ddc002f70 tx=0x7f0ddc006df0).stop 2022-06-29T15:51:59.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.638+0000 7f0df1645700 1 -- 172.21.15.47:0/3605324968 shutdown_connections 2022-06-29T15:51:59.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.638+0000 7f0df1645700 1 --2- 172.21.15.47:0/3605324968 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0dec0ad970 0x7f0dec1407e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:59.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.638+0000 7f0df1645700 1 --2- 172.21.15.47:0/3605324968 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0dec0b2ee0 0x7f0dec0b32c0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:59.649 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.638+0000 7f0df1645700 1 -- 172.21.15.47:0/3605324968 >> 172.21.15.47:0/3605324968 conn(0x7f0dec063670 msgr2=0x7f0dec063a80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:59.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.638+0000 7f0df1645700 1 -- 172.21.15.47:0/3605324968 shutdown_connections 2022-06-29T15:51:59.650 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.638+0000 7f0df1645700 1 -- 172.21.15.47:0/3605324968 wait complete. 2022-06-29T15:51:59.652 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.638+0000 7f0df1645700 1 Processor -- start 2022-06-29T15:51:59.653 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.638+0000 7f0df1645700 1 -- start start 2022-06-29T15:51:59.653 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.642+0000 7f0df1645700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0dec0ad970 0x7f0dec05f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:59.653 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.642+0000 7f0df1645700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0dec0b0e50 0x7f0dec13c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:59.654 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.642+0000 7f0df1645700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dec0b2ee0 0x7f0dec13cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:59.654 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.642+0000 7f0df1645700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dec1470a0 con 0x7f0dec0b0e50 2022-06-29T15:51:59.654 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.642+0000 7f0df1645700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dec146700 con 0x7f0dec0b2ee0 2022-06-29T15:51:59.654 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.642+0000 7f0df1645700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dec1469a0 con 0x7f0dec0ad970 2022-06-29T15:51:59.654 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.642+0000 7f0de3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0dec0b0e50 0x7f0dec13c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:59.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.642+0000 7f0de3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0dec0b0e50 0x7f0dec13c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54596/0 (socket says 172.21.15.47:54596) 2022-06-29T15:51:59.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.642+0000 7f0de3fff700 1 -- 172.21.15.47:0/2235566069 learned_addr learned my addr 172.21.15.47:0/2235566069 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:59.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.642+0000 7f0df0e44700 1 --2- 172.21.15.47:0/2235566069 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dec0b2ee0 0x7f0dec13cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:59.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.642+0000 7f0debfff700 1 --2- 172.21.15.47:0/2235566069 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0dec0ad970 0x7f0dec05f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:59.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.642+0000 7f0de3fff700 1 -- 172.21.15.47:0/2235566069 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0dec0ad970 msgr2=0x7f0dec05f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:59.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.642+0000 7f0de3fff700 1 --2- 172.21.15.47:0/2235566069 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0dec0ad970 0x7f0dec05f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:59.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.642+0000 7f0de3fff700 1 -- 172.21.15.47:0/2235566069 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dec0b2ee0 msgr2=0x7f0dec13cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:59.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.642+0000 7f0de3fff700 1 --2- 172.21.15.47:0/2235566069 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dec0b2ee0 0x7f0dec13cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:59.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.642+0000 7f0de3fff700 1 -- 172.21.15.47:0/2235566069 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ddc00b180 con 0x7f0dec0b0e50 2022-06-29T15:51:59.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.642+0000 7f0de3fff700 1 --2- 172.21.15.47:0/2235566069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0dec0b0e50 0x7f0dec13c6c0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 rx=0x7f0dd400c000 tx=0x7f0dd4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:59.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.642+0000 7f0de9ffb700 1 -- 172.21.15.47:0/2235566069 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0dd4010780 con 0x7f0dec0b0e50 2022-06-29T15:51:59.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.642+0000 7f0de9ffb700 1 -- 172.21.15.47:0/2235566069 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dd40108e0 con 0x7f0dec0b0e50 2022-06-29T15:51:59.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.642+0000 7f0df1645700 1 -- 172.21.15.47:0/2235566069 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dec13d0a0 con 0x7f0dec0b0e50 2022-06-29T15:51:59.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.642+0000 7f0de9ffb700 1 -- 172.21.15.47:0/2235566069 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0dd4010c40 con 0x7f0dec0b0e50 2022-06-29T15:51:59.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.642+0000 7f0df1645700 1 -- 172.21.15.47:0/2235566069 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dec145ce0 con 0x7f0dec0b0e50 2022-06-29T15:51:59.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.642+0000 7f0de9ffb700 1 -- 172.21.15.47:0/2235566069 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0dd401c460 con 0x7f0dec0b0e50 2022-06-29T15:51:59.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.642+0000 7f0df1645700 1 -- 172.21.15.47:0/2235566069 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dec1472a0 con 0x7f0dec0b0e50 2022-06-29T15:51:59.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.646+0000 7f0de9ffb700 1 --2- 172.21.15.47:0/2235566069 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0dd00539e0 0x7f0dd0055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:59.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.646+0000 7f0de9ffb700 1 -- 172.21.15.47:0/2235566069 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f0dd406c460 con 0x7f0dec0b0e50 2022-06-29T15:51:59.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.646+0000 7f0debfff700 1 --2- 172.21.15.47:0/2235566069 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0dd00539e0 0x7f0dd0055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:59.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.646+0000 7f0debfff700 1 --2- 172.21.15.47:0/2235566069 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0dd00539e0 0x7f0dd0055ea0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7f0ddc01a000 tx=0x7f0ddc014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:59.660 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.650+0000 7f0de9ffb700 1 -- 172.21.15.47:0/2235566069 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0dd40431c0 con 0x7f0dec0b0e50 2022-06-29T15:51:59.783 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.774+0000 7f0df1645700 1 -- 172.21.15.47:0/2235566069 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0dec0573d0 con 0x7f0dec0b0e50 2022-06-29T15:51:59.784 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.778+0000 7f0de9ffb700 1 -- 172.21.15.47:0/2235566069 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f0dd40467e0 con 0x7f0dec0b0e50 2022-06-29T15:51:59.784 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:51:59.785 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":151,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:51:53.832186+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:51:50.583017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2022-06-29T15:51:51.834830+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:51:59.786 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.778+0000 7f0df1645700 1 -- 172.21.15.47:0/2235566069 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0dd00539e0 msgr2=0x7f0dd0055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:59.787 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.778+0000 7f0df1645700 1 --2- 172.21.15.47:0/2235566069 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0dd00539e0 0x7f0dd0055ea0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7f0ddc01a000 tx=0x7f0ddc014000).stop 2022-06-29T15:51:59.787 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.778+0000 7f0df1645700 1 -- 172.21.15.47:0/2235566069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0dec0b0e50 msgr2=0x7f0dec13c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:59.787 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.778+0000 7f0df1645700 1 --2- 172.21.15.47:0/2235566069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0dec0b0e50 0x7f0dec13c6c0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 rx=0x7f0dd400c000 tx=0x7f0dd4006df0).stop 2022-06-29T15:51:59.787 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.778+0000 7f0df1645700 1 -- 172.21.15.47:0/2235566069 shutdown_connections 2022-06-29T15:51:59.788 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.778+0000 7f0df1645700 1 --2- 172.21.15.47:0/2235566069 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0dd00539e0 0x7f0dd0055ea0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:59.788 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.778+0000 7f0df1645700 1 --2- 172.21.15.47:0/2235566069 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dec0b2ee0 0x7f0dec13cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:59.788 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.778+0000 7f0df1645700 1 --2- 172.21.15.47:0/2235566069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0dec0b0e50 0x7f0dec13c6c0 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:59.788 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.778+0000 7f0df1645700 1 --2- 172.21.15.47:0/2235566069 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0dec0ad970 0x7f0dec05f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:59.789 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.778+0000 7f0df1645700 1 -- 172.21.15.47:0/2235566069 >> 172.21.15.47:0/2235566069 conn(0x7f0dec063670 msgr2=0x7f0dec0b0150 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:59.789 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.778+0000 7f0df1645700 1 -- 172.21.15.47:0/2235566069 shutdown_connections 2022-06-29T15:51:59.789 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.778+0000 7f0df1645700 1 -- 172.21.15.47:0/2235566069 wait complete. 2022-06-29T15:51:59.798 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-29T15:42:16.971431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-29T15:42:25.290884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 27, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-29T15:51:50.583017+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '150', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 28, 'pool_name': 'cephfs_data', 'create_time': '2022-06-29T15:51:51.834830+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '151', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-29T15:51:59.799 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-29T15:51:59.911 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.902+0000 7f27bf45c700 1 Processor -- start 2022-06-29T15:51:59.911 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.902+0000 7f27bf45c700 1 -- start start 2022-06-29T15:51:59.914 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.906+0000 7f27bf45c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f27b80b1c60 0x7f27b805f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:59.915 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.906+0000 7f27bf45c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f27b805fc10 0x7f27b80600a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:59.915 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.906+0000 7f27bf45c700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27b80609a0 con 0x7f27b80b2900 2022-06-29T15:51:59.915 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.906+0000 7f27bf45c700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27b80b2ce0 con 0x7f27b80b1c60 2022-06-29T15:51:59.915 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.906+0000 7f27bf45c700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27b80b0210 con 0x7f27b805fc10 2022-06-29T15:51:59.916 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.906+0000 7f27bdc59700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f27b80b1c60 0x7f27b805f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:59.916 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.906+0000 7f27bdc59700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f27b80b1c60 0x7f27b805f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54604/0 (socket says 172.21.15.47:54604) 2022-06-29T15:51:59.916 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.906+0000 7f27bdc59700 1 -- 172.21.15.47:0/3704477881 learned_addr learned my addr 172.21.15.47:0/3704477881 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:59.916 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.906+0000 7f27be45a700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f27b80b2900 0x7f27b8099d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40156/0 (socket says 172.21.15.47:40156) 2022-06-29T15:51:59.917 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.906+0000 7f27bec5b700 1 --2- 172.21.15.47:0/3704477881 >> v2:172.21.15.64:3300/0 conn(0x7f27b805fc10 0x7f27b80600a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:59.917 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.906+0000 7f27bdc59700 1 -- 172.21.15.47:0/3704477881 >> v2:172.21.15.64:3300/0 conn(0x7f27b805fc10 msgr2=0x7f27b80600a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:51:59.917 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.906+0000 7f27bdc59700 1 --2- 172.21.15.47:0/3704477881 >> v2:172.21.15.64:3300/0 conn(0x7f27b805fc10 0x7f27b80600a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:59.917 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.906+0000 7f27bdc59700 1 -- 172.21.15.47:0/3704477881 >> v1:172.21.15.47:6789/0 conn(0x7f27b80b2900 legacy=0x7f27b8099d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:59.918 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.906+0000 7f27bd458700 1 -- 172.21.15.47:0/3704477881 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3605513405 0 0) 0x7f27b80609a0 con 0x7f27b80b2900 2022-06-29T15:51:59.918 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.906+0000 7f27bdc59700 1 -- 172.21.15.47:0/3704477881 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b80b0fe0 con 0x7f27b80b1c60 2022-06-29T15:51:59.918 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.906+0000 7f27bec5b700 1 --2- 172.21.15.47:0/3704477881 >> v2:172.21.15.64:3300/0 conn(0x7f27b805fc10 0x7f27b80600a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:51:59.918 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.906+0000 7f27bdc59700 1 --2- 172.21.15.47:0/3704477881 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27b80b1c60 0x7f27b805f370 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 rx=0x7f27ac002f70 tx=0x7f27ac006e50).ready entity=mon.0 client_cookie=d2ce7706969a241a server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:59.918 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.906+0000 7f27bd458700 1 -- 172.21.15.47:0/3704477881 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f27ac00d610 con 0x7f27b80b1c60 2022-06-29T15:51:59.919 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.906+0000 7f27bd458700 1 -- 172.21.15.47:0/3704477881 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27ac000e90 con 0x7f27b80b1c60 2022-06-29T15:51:59.919 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.906+0000 7f27bd458700 1 -- 172.21.15.47:0/3704477881 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f27ac00d960 con 0x7f27b80b1c60 2022-06-29T15:51:59.919 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.906+0000 7f27bf45c700 1 -- 172.21.15.47:0/3704477881 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27b80b1c60 msgr2=0x7f27b805f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:59.919 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.906+0000 7f27bf45c700 1 --2- 172.21.15.47:0/3704477881 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27b80b1c60 0x7f27b805f370 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 rx=0x7f27ac002f70 tx=0x7f27ac006e50).stop 2022-06-29T15:51:59.920 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.906+0000 7f27bf45c700 1 -- 172.21.15.47:0/3704477881 shutdown_connections 2022-06-29T15:51:59.920 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.906+0000 7f27bf45c700 1 --2- 172.21.15.47:0/3704477881 >> v2:172.21.15.64:3300/0 conn(0x7f27b805fc10 0x7f27b80600a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:59.920 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.906+0000 7f27bf45c700 1 --2- 172.21.15.47:0/3704477881 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27b80b1c60 0x7f27b805f370 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:59.920 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.906+0000 7f27bf45c700 1 -- 172.21.15.47:0/3704477881 >> 172.21.15.47:0/3704477881 conn(0x7f27b8062df0 msgr2=0x7f27b80631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:51:59.920 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.906+0000 7f27bf45c700 1 -- 172.21.15.47:0/3704477881 shutdown_connections 2022-06-29T15:51:59.921 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.906+0000 7f27bf45c700 1 -- 172.21.15.47:0/3704477881 wait complete. 2022-06-29T15:51:59.924 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.910+0000 7f27bf45c700 1 Processor -- start 2022-06-29T15:51:59.924 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.910+0000 7f27bf45c700 1 -- start start 2022-06-29T15:51:59.924 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.910+0000 7f27bf45c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f27b805fc10 0x7f27b80ae720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:59.925 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.910+0000 7f27bf45c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27b80b1c60 0x7f27b8066840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:59.925 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.910+0000 7f27bf45c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27b80b2900 0x7f27b8066d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:59.925 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.910+0000 7f27bf45c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b8064e90 con 0x7f27b80b1c60 2022-06-29T15:51:59.925 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.910+0000 7f27bf45c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b8156770 con 0x7f27b80b2900 2022-06-29T15:51:59.926 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.910+0000 7f27bf45c700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27b80655a0 con 0x7f27b805fc10 2022-06-29T15:51:59.926 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.910+0000 7f27bdc59700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27b80b1c60 0x7f27b8066840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:59.926 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.910+0000 7f27bdc59700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27b80b1c60 0x7f27b8066840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54610/0 (socket says 172.21.15.47:54610) 2022-06-29T15:51:59.926 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.910+0000 7f27bdc59700 1 -- 172.21.15.47:0/3982330079 learned_addr learned my addr 172.21.15.47:0/3982330079 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:51:59.927 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.910+0000 7f27bec5b700 1 --2- 172.21.15.47:0/3982330079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27b80b2900 0x7f27b8066d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:59.927 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.910+0000 7f27be45a700 1 --2- 172.21.15.47:0/3982330079 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f27b805fc10 0x7f27b80ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:59.927 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.910+0000 7f27bdc59700 1 -- 172.21.15.47:0/3982330079 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f27b805fc10 msgr2=0x7f27b80ae720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:59.927 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.910+0000 7f27bdc59700 1 --2- 172.21.15.47:0/3982330079 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f27b805fc10 0x7f27b80ae720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:59.927 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.910+0000 7f27bdc59700 1 -- 172.21.15.47:0/3982330079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27b80b2900 msgr2=0x7f27b8066d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:51:59.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.910+0000 7f27bdc59700 1 --2- 172.21.15.47:0/3982330079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27b80b2900 0x7f27b8066d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:51:59.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.910+0000 7f27bdc59700 1 -- 172.21.15.47:0/3982330079 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27ac00b1e0 con 0x7f27b80b1c60 2022-06-29T15:51:59.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.910+0000 7f27bec5b700 1 --2- 172.21.15.47:0/3982330079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27b80b2900 0x7f27b8066d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:51:59.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.910+0000 7f27bdc59700 1 --2- 172.21.15.47:0/3982330079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27b80b1c60 0x7f27b8066840 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 rx=0x7f27ac002f70 tx=0x7f27ac015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:59.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.910+0000 7f27ab7fe700 1 -- 172.21.15.47:0/3982330079 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f27ac00de90 con 0x7f27b80b1c60 2022-06-29T15:51:59.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.914+0000 7f27ab7fe700 1 -- 172.21.15.47:0/3982330079 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27ac006ae0 con 0x7f27b80b1c60 2022-06-29T15:51:59.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.914+0000 7f27bf45c700 1 -- 172.21.15.47:0/3982330079 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27b8067220 con 0x7f27b80b1c60 2022-06-29T15:51:59.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.914+0000 7f27ab7fe700 1 -- 172.21.15.47:0/3982330079 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f27ac01e460 con 0x7f27b80b1c60 2022-06-29T15:51:59.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.914+0000 7f27bf45c700 1 -- 172.21.15.47:0/3982330079 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27b8065a40 con 0x7f27b80b1c60 2022-06-29T15:51:59.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.914+0000 7f27bf45c700 1 -- 172.21.15.47:0/3982330079 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27b8156940 con 0x7f27b80b1c60 2022-06-29T15:51:59.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.914+0000 7f27ab7fe700 1 -- 172.21.15.47:0/3982330079 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f27ac025030 con 0x7f27b80b1c60 2022-06-29T15:51:59.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.914+0000 7f27ab7fe700 1 --2- 172.21.15.47:0/3982330079 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2798053980 0x7f2798055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:51:59.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.914+0000 7f27ab7fe700 1 -- 172.21.15.47:0/3982330079 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f27ac06d280 con 0x7f27b80b1c60 2022-06-29T15:51:59.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.914+0000 7f27be45a700 1 --2- 172.21.15.47:0/3982330079 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2798053980 0x7f2798055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:51:59.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.918+0000 7f27be45a700 1 --2- 172.21.15.47:0/3982330079 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2798053980 0x7f2798055e40 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7f27b400b000 tx=0x7f27b4007160).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:51:59.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:51:59.922+0000 7f27ab7fe700 1 -- 172.21.15.47:0/3982330079 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f27ac044080 con 0x7f27b80b1c60 2022-06-29T15:52:00.057 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.050+0000 7f27bf45c700 1 -- 172.21.15.47:0/3982330079 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f27b8065760 con 0x7f27b80b1c60 2022-06-29T15:52:00.057 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.050+0000 7f27ab7fe700 1 -- 172.21.15.47:0/3982330079 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v151) v1 ==== 110+0+10 (secure 0 0 0) 0x7f27ac047660 con 0x7f27b80b1c60 2022-06-29T15:52:00.058 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2022-06-29T15:52:00.061 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.054+0000 7f27bf45c700 1 -- 172.21.15.47:0/3982330079 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2798053980 msgr2=0x7f2798055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:00.061 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.054+0000 7f27bf45c700 1 --2- 172.21.15.47:0/3982330079 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2798053980 0x7f2798055e40 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7f27b400b000 tx=0x7f27b4007160).stop 2022-06-29T15:52:00.062 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.054+0000 7f27bf45c700 1 -- 172.21.15.47:0/3982330079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27b80b1c60 msgr2=0x7f27b8066840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:00.062 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.054+0000 7f27bf45c700 1 --2- 172.21.15.47:0/3982330079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27b80b1c60 0x7f27b8066840 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 rx=0x7f27ac002f70 tx=0x7f27ac015000).stop 2022-06-29T15:52:00.062 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.054+0000 7f27bf45c700 1 -- 172.21.15.47:0/3982330079 shutdown_connections 2022-06-29T15:52:00.062 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.054+0000 7f27bf45c700 1 --2- 172.21.15.47:0/3982330079 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2798053980 0x7f2798055e40 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:00.063 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.054+0000 7f27bf45c700 1 --2- 172.21.15.47:0/3982330079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27b80b2900 0x7f27b8066d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:00.063 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.054+0000 7f27bf45c700 1 --2- 172.21.15.47:0/3982330079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27b80b1c60 0x7f27b8066840 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:00.063 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.054+0000 7f27bf45c700 1 --2- 172.21.15.47:0/3982330079 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f27b805fc10 0x7f27b80ae720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:00.063 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.054+0000 7f27bf45c700 1 -- 172.21.15.47:0/3982330079 >> 172.21.15.47:0/3982330079 conn(0x7f27b8062df0 msgr2=0x7f27b8146c60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:00.064 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.054+0000 7f27bf45c700 1 -- 172.21.15.47:0/3982330079 shutdown_connections 2022-06-29T15:52:00.064 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.054+0000 7f27bf45c700 1 -- 172.21.15.47:0/3982330079 wait complete. 2022-06-29T15:52:00.075 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-29T15:52:00.183 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.178+0000 7fc30ba3b700 1 Processor -- start 2022-06-29T15:52:00.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.178+0000 7fc30ba3b700 1 -- start start 2022-06-29T15:52:00.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.178+0000 7fc30ba3b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc304060a70 0x7fc30414b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:00.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.178+0000 7fc30ba3b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc3040a72b0 0x7fc304144750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:00.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.178+0000 7fc30ba3b700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc304060e50 con 0x7fc304144c40 2022-06-29T15:52:00.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.178+0000 7fc30ba3b700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3040a8120 con 0x7fc304060a70 2022-06-29T15:52:00.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.178+0000 7fc30ba3b700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc304147540 con 0x7fc3040a72b0 2022-06-29T15:52:00.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.178+0000 7fc3097d7700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc304060a70 0x7fc30414b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:00.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.178+0000 7fc3097d7700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc304060a70 0x7fc30414b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54616/0 (socket says 172.21.15.47:54616) 2022-06-29T15:52:00.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.178+0000 7fc3097d7700 1 -- 172.21.15.47:0/2238958908 learned_addr learned my addr 172.21.15.47:0/2238958908 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:00.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.178+0000 7fc308fd6700 1 --2- 172.21.15.47:0/2238958908 >> v2:172.21.15.64:3300/0 conn(0x7fc3040a72b0 0x7fc304144750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:00.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.178+0000 7fc3097d7700 1 -- 172.21.15.47:0/2238958908 >> v2:172.21.15.64:3300/0 conn(0x7fc3040a72b0 msgr2=0x7fc304144750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:00.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.178+0000 7fc3097d7700 1 --2- 172.21.15.47:0/2238958908 >> v2:172.21.15.64:3300/0 conn(0x7fc3040a72b0 0x7fc304144750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:00.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.178+0000 7fc3097d7700 1 -- 172.21.15.47:0/2238958908 >> v1:172.21.15.64:6789/0 conn(0x7fc304144c40 legacy=0x7fc304147930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:00.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.178+0000 7fc3097d7700 1 -- 172.21.15.47:0/2238958908 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3040a8ac0 con 0x7fc304060a70 2022-06-29T15:52:00.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.178+0000 7fc3097d7700 1 --2- 172.21.15.47:0/2238958908 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc304060a70 0x7fc30414b4e0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 rx=0x7fc3040a7f20 tx=0x7fc2fc006eb0).ready entity=mon.0 client_cookie=bae7093cc085af8a server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:00.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.178+0000 7fc2f7fff700 1 -- 172.21.15.47:0/2238958908 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc2fc00d610 con 0x7fc304060a70 2022-06-29T15:52:00.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.178+0000 7fc2f7fff700 1 -- 172.21.15.47:0/2238958908 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2fc000e90 con 0x7fc304060a70 2022-06-29T15:52:00.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.178+0000 7fc2f7fff700 1 -- 172.21.15.47:0/2238958908 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc2fc00d960 con 0x7fc304060a70 2022-06-29T15:52:00.193 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.178+0000 7fc30ba3b700 1 -- 172.21.15.47:0/2238958908 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc304060a70 msgr2=0x7fc30414b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:00.193 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.178+0000 7fc30ba3b700 1 --2- 172.21.15.47:0/2238958908 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc304060a70 0x7fc30414b4e0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 rx=0x7fc3040a7f20 tx=0x7fc2fc006eb0).stop 2022-06-29T15:52:00.194 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.178+0000 7fc30ba3b700 1 -- 172.21.15.47:0/2238958908 shutdown_connections 2022-06-29T15:52:00.194 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.178+0000 7fc30ba3b700 1 --2- 172.21.15.47:0/2238958908 >> v2:172.21.15.64:3300/0 conn(0x7fc3040a72b0 0x7fc304144750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:00.194 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.178+0000 7fc30ba3b700 1 --2- 172.21.15.47:0/2238958908 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc304060a70 0x7fc30414b4e0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:00.194 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.178+0000 7fc30ba3b700 1 -- 172.21.15.47:0/2238958908 >> 172.21.15.47:0/2238958908 conn(0x7fc3040626f0 msgr2=0x7fc304062ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:00.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.182+0000 7fc30ba3b700 1 -- 172.21.15.47:0/2238958908 shutdown_connections 2022-06-29T15:52:00.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.182+0000 7fc30ba3b700 1 -- 172.21.15.47:0/2238958908 wait complete. 2022-06-29T15:52:00.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.182+0000 7fc30ba3b700 1 Processor -- start 2022-06-29T15:52:00.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.182+0000 7fc30ba3b700 1 -- start start 2022-06-29T15:52:00.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.182+0000 7fc30ba3b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc304060a70 0x7fc3040a9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:00.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.182+0000 7fc30ba3b700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc3040a72b0 0x7fc3040aef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:00.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.182+0000 7fc30ba3b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc304144c40 0x7fc304065290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:00.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.182+0000 7fc30ba3b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc304065bb0 con 0x7fc304060a70 2022-06-29T15:52:00.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.182+0000 7fc30ba3b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3040a9910 con 0x7fc304144c40 2022-06-29T15:52:00.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.182+0000 7fc30ba3b700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3040af450 con 0x7fc3040a72b0 2022-06-29T15:52:00.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.182+0000 7fc3097d7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc304060a70 0x7fc3040a9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:00.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.182+0000 7fc3097d7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc304060a70 0x7fc3040a9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54622/0 (socket says 172.21.15.47:54622) 2022-06-29T15:52:00.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.182+0000 7fc3097d7700 1 -- 172.21.15.47:0/1560576207 learned_addr learned my addr 172.21.15.47:0/1560576207 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:00.198 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.182+0000 7fc308fd6700 1 --2- 172.21.15.47:0/1560576207 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc3040a72b0 0x7fc3040aef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:00.198 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.182+0000 7fc309fd8700 1 --2- 172.21.15.47:0/1560576207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc304144c40 0x7fc304065290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:00.198 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.186+0000 7fc3097d7700 1 -- 172.21.15.47:0/1560576207 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc3040a72b0 msgr2=0x7fc3040aef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:00.198 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.186+0000 7fc3097d7700 1 --2- 172.21.15.47:0/1560576207 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc3040a72b0 0x7fc3040aef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:00.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.186+0000 7fc3097d7700 1 -- 172.21.15.47:0/1560576207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc304144c40 msgr2=0x7fc304065290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:00.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.186+0000 7fc3097d7700 1 --2- 172.21.15.47:0/1560576207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc304144c40 0x7fc304065290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:00.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.186+0000 7fc3097d7700 1 -- 172.21.15.47:0/1560576207 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2fc00b1e0 con 0x7fc304060a70 2022-06-29T15:52:00.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.186+0000 7fc309fd8700 1 --2- 172.21.15.47:0/1560576207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc304144c40 0x7fc304065290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:52:00.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.186+0000 7fc308fd6700 1 --2- 172.21.15.47:0/1560576207 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc3040a72b0 0x7fc3040aef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:52:00.200 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.186+0000 7fc3097d7700 1 --2- 172.21.15.47:0/1560576207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc304060a70 0x7fc3040a9420 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 rx=0x7fc3040a9aa0 tx=0x7fc2fc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:00.200 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.186+0000 7fc2f6ffd700 1 -- 172.21.15.47:0/1560576207 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc2fc00bec0 con 0x7fc304060a70 2022-06-29T15:52:00.200 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.186+0000 7fc2f6ffd700 1 -- 172.21.15.47:0/1560576207 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2fc00d970 con 0x7fc304060a70 2022-06-29T15:52:00.200 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.186+0000 7fc30ba3b700 1 -- 172.21.15.47:0/1560576207 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc304064700 con 0x7fc304060a70 2022-06-29T15:52:00.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.186+0000 7fc2f6ffd700 1 -- 172.21.15.47:0/1560576207 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc2fc011e50 con 0x7fc304060a70 2022-06-29T15:52:00.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.186+0000 7fc30ba3b700 1 -- 172.21.15.47:0/1560576207 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3040b5c20 con 0x7fc304060a70 2022-06-29T15:52:00.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.186+0000 7fc30ba3b700 1 -- 172.21.15.47:0/1560576207 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc304066ef0 con 0x7fc304060a70 2022-06-29T15:52:00.201 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.186+0000 7fc2f6ffd700 1 -- 172.21.15.47:0/1560576207 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc2fc025030 con 0x7fc304060a70 2022-06-29T15:52:00.202 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.186+0000 7fc2f6ffd700 1 --2- 172.21.15.47:0/1560576207 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc2e0053980 0x7fc2e0055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:00.202 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.186+0000 7fc308fd6700 1 --2- 172.21.15.47:0/1560576207 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc2e0053980 0x7fc2e0055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:00.203 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.186+0000 7fc2f6ffd700 1 -- 172.21.15.47:0/1560576207 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc2fc06d2b0 con 0x7fc304060a70 2022-06-29T15:52:00.203 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.190+0000 7fc308fd6700 1 --2- 172.21.15.47:0/1560576207 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc2e0053980 0x7fc2e0055e40 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7fc2f800d000 tx=0x7fc2f8008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:00.203 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.194+0000 7fc2f6ffd700 1 -- 172.21.15.47:0/1560576207 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc2fc044070 con 0x7fc304060a70 2022-06-29T15:52:00.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.318+0000 7fc30ba3b700 1 -- 172.21.15.47:0/1560576207 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc304064890 con 0x7fc304060a70 2022-06-29T15:52:00.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.318+0000 7fc2f6ffd700 1 -- 172.21.15.47:0/1560576207 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v151) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc2fc0230a0 con 0x7fc304060a70 2022-06-29T15:52:00.327 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:52:00.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.322+0000 7fc30ba3b700 1 -- 172.21.15.47:0/1560576207 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc2e0053980 msgr2=0x7fc2e0055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:00.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.322+0000 7fc30ba3b700 1 --2- 172.21.15.47:0/1560576207 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc2e0053980 0x7fc2e0055e40 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7fc2f800d000 tx=0x7fc2f8008040).stop 2022-06-29T15:52:00.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.322+0000 7fc30ba3b700 1 -- 172.21.15.47:0/1560576207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc304060a70 msgr2=0x7fc3040a9420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:00.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.322+0000 7fc30ba3b700 1 --2- 172.21.15.47:0/1560576207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc304060a70 0x7fc3040a9420 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 rx=0x7fc3040a9aa0 tx=0x7fc2fc015000).stop 2022-06-29T15:52:00.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.322+0000 7fc30ba3b700 1 -- 172.21.15.47:0/1560576207 shutdown_connections 2022-06-29T15:52:00.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.322+0000 7fc30ba3b700 1 --2- 172.21.15.47:0/1560576207 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc2e0053980 0x7fc2e0055e40 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:00.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.322+0000 7fc30ba3b700 1 --2- 172.21.15.47:0/1560576207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc304144c40 0x7fc304065290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:00.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.322+0000 7fc30ba3b700 1 --2- 172.21.15.47:0/1560576207 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc3040a72b0 0x7fc3040aef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:00.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.322+0000 7fc30ba3b700 1 --2- 172.21.15.47:0/1560576207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc304060a70 0x7fc3040a9420 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:00.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.322+0000 7fc30ba3b700 1 -- 172.21.15.47:0/1560576207 >> 172.21.15.47:0/1560576207 conn(0x7fc3040626f0 msgr2=0x7fc30405f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:00.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.322+0000 7fc30ba3b700 1 -- 172.21.15.47:0/1560576207 shutdown_connections 2022-06-29T15:52:00.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.322+0000 7fc30ba3b700 1 -- 172.21.15.47:0/1560576207 wait complete. 2022-06-29T15:52:00.340 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-29T15:52:00.449 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.442+0000 7fd5e59d0700 1 Processor -- start 2022-06-29T15:52:00.450 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.442+0000 7fd5e59d0700 1 -- start start 2022-06-29T15:52:00.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.446+0000 7fd5e59d0700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd5e00b6d30 0x7fd5e013d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:00.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.446+0000 7fd5e59d0700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd5e0062f30 0x7fd5e013d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:00.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.446+0000 7fd5e59d0700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5e0143f70 con 0x7fd5e01400c0 2022-06-29T15:52:00.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.446+0000 7fd5e59d0700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5e00b1380 con 0x7fd5e0062f30 2022-06-29T15:52:00.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.446+0000 7fd5e59d0700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5e00a9cf0 con 0x7fd5e00b6d30 2022-06-29T15:52:00.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.446+0000 7fd5d7fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd5e0062f30 0x7fd5e013d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:00.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.446+0000 7fd5d7fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd5e0062f30 0x7fd5e013d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54630/0 (socket says 172.21.15.47:54630) 2022-06-29T15:52:00.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.446+0000 7fd5d7fff700 1 -- 172.21.15.47:0/2573824233 learned_addr learned my addr 172.21.15.47:0/2573824233 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:00.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.446+0000 7fd5e49ce700 1 --2- 172.21.15.47:0/2573824233 >> v2:172.21.15.64:3300/0 conn(0x7fd5e00b6d30 0x7fd5e013d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:00.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.446+0000 7fd5d7fff700 1 -- 172.21.15.47:0/2573824233 >> v2:172.21.15.64:3300/0 conn(0x7fd5e00b6d30 msgr2=0x7fd5e013d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:00.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.446+0000 7fd5d7fff700 1 --2- 172.21.15.47:0/2573824233 >> v2:172.21.15.64:3300/0 conn(0x7fd5e00b6d30 0x7fd5e013d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:00.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.446+0000 7fd5d7fff700 1 -- 172.21.15.47:0/2573824233 >> v1:172.21.15.64:6789/0 conn(0x7fd5e01400c0 legacy=0x7fd5e0140510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:00.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.446+0000 7fd5d7fff700 1 -- 172.21.15.47:0/2573824233 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e013dc80 con 0x7fd5e0062f30 2022-06-29T15:52:00.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.446+0000 7fd5e49ce700 1 --2- 172.21.15.47:0/2573824233 >> v2:172.21.15.64:3300/0 conn(0x7fd5e00b6d30 0x7fd5e013d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:52:00.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.446+0000 7fd5d7fff700 1 --2- 172.21.15.47:0/2573824233 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5e0062f30 0x7fd5e013d790 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 rx=0x7fd5e0062a70 tx=0x7fd5d0006f40).ready entity=mon.0 client_cookie=2fccaf140ec28c1c server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:00.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.446+0000 7fd5d77fe700 1 -- 172.21.15.47:0/2573824233 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd5d000dc30 con 0x7fd5e0062f30 2022-06-29T15:52:00.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.446+0000 7fd5d77fe700 1 -- 172.21.15.47:0/2573824233 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d0000e90 con 0x7fd5e0062f30 2022-06-29T15:52:00.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.446+0000 7fd5d77fe700 1 -- 172.21.15.47:0/2573824233 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd5d000be90 con 0x7fd5e0062f30 2022-06-29T15:52:00.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.446+0000 7fd5e59d0700 1 -- 172.21.15.47:0/2573824233 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5e0062f30 msgr2=0x7fd5e013d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:00.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.446+0000 7fd5e59d0700 1 --2- 172.21.15.47:0/2573824233 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5e0062f30 0x7fd5e013d790 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 rx=0x7fd5e0062a70 tx=0x7fd5d0006f40).stop 2022-06-29T15:52:00.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.446+0000 7fd5e59d0700 1 -- 172.21.15.47:0/2573824233 shutdown_connections 2022-06-29T15:52:00.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.446+0000 7fd5e59d0700 1 --2- 172.21.15.47:0/2573824233 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5e0062f30 0x7fd5e013d790 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:00.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.446+0000 7fd5e59d0700 1 --2- 172.21.15.47:0/2573824233 >> v2:172.21.15.64:3300/0 conn(0x7fd5e00b6d30 0x7fd5e013d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:00.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.446+0000 7fd5e59d0700 1 -- 172.21.15.47:0/2573824233 >> 172.21.15.47:0/2573824233 conn(0x7fd5e0063670 msgr2=0x7fd5e0063a50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:00.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.446+0000 7fd5e59d0700 1 -- 172.21.15.47:0/2573824233 shutdown_connections 2022-06-29T15:52:00.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.446+0000 7fd5e59d0700 1 -- 172.21.15.47:0/2573824233 wait complete. 2022-06-29T15:52:00.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.446+0000 7fd5e59d0700 1 Processor -- start 2022-06-29T15:52:00.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.446+0000 7fd5e59d0700 1 -- start start 2022-06-29T15:52:00.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.446+0000 7fd5e59d0700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd5e0062f30 0x7fd5e0060940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:00.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.446+0000 7fd5e59d0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5e00b6d30 0x7fd5e013e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:00.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.446+0000 7fd5e59d0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5e01400c0 0x7fd5e0151400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:00.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.446+0000 7fd5e59d0700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e0143990 con 0x7fd5e01400c0 2022-06-29T15:52:00.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.446+0000 7fd5e59d0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e0142320 con 0x7fd5e00b6d30 2022-06-29T15:52:00.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.446+0000 7fd5e59d0700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e013c280 con 0x7fd5e0062f30 2022-06-29T15:52:00.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.450+0000 7fd5e51cf700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5e01400c0 0x7fd5e0151400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:00.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.450+0000 7fd5e51cf700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5e01400c0 0x7fd5e0151400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54638/0 (socket says 172.21.15.47:54638) 2022-06-29T15:52:00.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.450+0000 7fd5e51cf700 1 -- 172.21.15.47:0/1154682053 learned_addr learned my addr 172.21.15.47:0/1154682053 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:00.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.450+0000 7fd5e49ce700 1 --2- 172.21.15.47:0/1154682053 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd5e0062f30 0x7fd5e0060940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:00.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.450+0000 7fd5d7fff700 1 --2- 172.21.15.47:0/1154682053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5e00b6d30 0x7fd5e013e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:00.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.450+0000 7fd5e51cf700 1 -- 172.21.15.47:0/1154682053 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd5e0062f30 msgr2=0x7fd5e0060940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:00.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.450+0000 7fd5e51cf700 1 --2- 172.21.15.47:0/1154682053 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd5e0062f30 0x7fd5e0060940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:00.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.450+0000 7fd5e51cf700 1 -- 172.21.15.47:0/1154682053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5e00b6d30 msgr2=0x7fd5e013e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:00.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.450+0000 7fd5e51cf700 1 --2- 172.21.15.47:0/1154682053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5e00b6d30 0x7fd5e013e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:00.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.450+0000 7fd5e51cf700 1 -- 172.21.15.47:0/1154682053 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d000b460 con 0x7fd5e01400c0 2022-06-29T15:52:00.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.450+0000 7fd5e51cf700 1 --2- 172.21.15.47:0/1154682053 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5e01400c0 0x7fd5e0151400 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 rx=0x7fd5e01417f0 tx=0x7fd5d80090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:00.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.450+0000 7fd5d5ffb700 1 -- 172.21.15.47:0/1154682053 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd5d800e470 con 0x7fd5e01400c0 2022-06-29T15:52:00.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.450+0000 7fd5d5ffb700 1 -- 172.21.15.47:0/1154682053 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d800e5d0 con 0x7fd5e01400c0 2022-06-29T15:52:00.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.450+0000 7fd5e59d0700 1 -- 172.21.15.47:0/1154682053 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5e0142580 con 0x7fd5e01400c0 2022-06-29T15:52:00.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.450+0000 7fd5d5ffb700 1 -- 172.21.15.47:0/1154682053 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd5d800e930 con 0x7fd5e01400c0 2022-06-29T15:52:00.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.450+0000 7fd5e59d0700 1 -- 172.21.15.47:0/1154682053 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5e0142cc0 con 0x7fd5e01400c0 2022-06-29T15:52:00.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.450+0000 7fd5e59d0700 1 -- 172.21.15.47:0/1154682053 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5e00573d0 con 0x7fd5e01400c0 2022-06-29T15:52:00.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.450+0000 7fd5d5ffb700 1 -- 172.21.15.47:0/1154682053 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd5d8018030 con 0x7fd5e01400c0 2022-06-29T15:52:00.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.454+0000 7fd5d5ffb700 1 --2- 172.21.15.47:0/1154682053 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd5c0053950 0x7fd5c0055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:00.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.454+0000 7fd5d5ffb700 1 -- 172.21.15.47:0/1154682053 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd5d806b910 con 0x7fd5e01400c0 2022-06-29T15:52:00.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.454+0000 7fd5e49ce700 1 --2- 172.21.15.47:0/1154682053 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd5c0053950 0x7fd5c0055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:00.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.454+0000 7fd5e49ce700 1 --2- 172.21.15.47:0/1154682053 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd5c0053950 0x7fd5c0055e10 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7fd5dc00d000 tx=0x7fd5dc008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:00.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.458+0000 7fd5d5ffb700 1 -- 172.21.15.47:0/1154682053 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd5d8041e90 con 0x7fd5e01400c0 2022-06-29T15:52:00.594 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.586+0000 7fd5e59d0700 1 -- 172.21.15.47:0/1154682053 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd5e0066780 con 0x7fd5e01400c0 2022-06-29T15:52:00.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.590+0000 7fd5d5ffb700 1 -- 172.21.15.47:0/1154682053 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v151) v1 ==== 104+0+10 (secure 0 0 0) 0x7fd5d8042670 con 0x7fd5e01400c0 2022-06-29T15:52:00.596 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:52:00.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.590+0000 7fd5e59d0700 1 -- 172.21.15.47:0/1154682053 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd5c0053950 msgr2=0x7fd5c0055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:00.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.590+0000 7fd5e59d0700 1 --2- 172.21.15.47:0/1154682053 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd5c0053950 0x7fd5c0055e10 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7fd5dc00d000 tx=0x7fd5dc008040).stop 2022-06-29T15:52:00.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.590+0000 7fd5e59d0700 1 -- 172.21.15.47:0/1154682053 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5e01400c0 msgr2=0x7fd5e0151400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:00.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.590+0000 7fd5e59d0700 1 --2- 172.21.15.47:0/1154682053 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5e01400c0 0x7fd5e0151400 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 rx=0x7fd5e01417f0 tx=0x7fd5d80090f0).stop 2022-06-29T15:52:00.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.590+0000 7fd5e59d0700 1 -- 172.21.15.47:0/1154682053 shutdown_connections 2022-06-29T15:52:00.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.590+0000 7fd5e59d0700 1 --2- 172.21.15.47:0/1154682053 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5e01400c0 0x7fd5e0151400 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:00.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.590+0000 7fd5e59d0700 1 --2- 172.21.15.47:0/1154682053 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd5c0053950 0x7fd5c0055e10 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:00.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.590+0000 7fd5e59d0700 1 --2- 172.21.15.47:0/1154682053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5e00b6d30 0x7fd5e013e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:00.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.590+0000 7fd5e59d0700 1 --2- 172.21.15.47:0/1154682053 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd5e0062f30 0x7fd5e0060940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:00.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.590+0000 7fd5e59d0700 1 -- 172.21.15.47:0/1154682053 >> 172.21.15.47:0/1154682053 conn(0x7fd5e0063670 msgr2=0x7fd5e01404d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:00.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.594+0000 7fd5e59d0700 1 -- 172.21.15.47:0/1154682053 shutdown_connections 2022-06-29T15:52:00.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.594+0000 7fd5e59d0700 1 -- 172.21.15.47:0/1154682053 wait complete. 2022-06-29T15:52:00.614 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-29T15:52:00.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.722+0000 7fb1dbdd0700 1 Processor -- start 2022-06-29T15:52:00.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.722+0000 7fb1dbdd0700 1 -- start start 2022-06-29T15:52:00.732 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.726+0000 7fb1dbdd0700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb1d4060ee0 0x7fb1d40612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:00.732 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.726+0000 7fb1dbdd0700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1d4067610 con 0x7fb1d40b4300 2022-06-29T15:52:00.732 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.726+0000 7fb1dbdd0700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1d41467c0 con 0x7fb1d41471a0 2022-06-29T15:52:00.732 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.726+0000 7fb1dbdd0700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1d4146a60 con 0x7fb1d4060ee0 2022-06-29T15:52:00.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.726+0000 7fb1d9b6c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb1d4060ee0 0x7fb1d40612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:00.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.726+0000 7fb1d936b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb1d40b4300 0x7fb1d41430e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40198/0 (socket says 172.21.15.47:40198) 2022-06-29T15:52:00.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.726+0000 7fb1d936b700 1 -- 172.21.15.47:0/2618447082 learned_addr learned my addr 172.21.15.47:0/2618447082 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:52:00.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.726+0000 7fb1d8b6a700 1 -- 172.21.15.47:0/2618447082 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 697718607 0 0) 0x7fb1d4067610 con 0x7fb1d40b4300 2022-06-29T15:52:00.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.726+0000 7fb1d8b6a700 1 -- 172.21.15.47:0/2618447082 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1bc003260 con 0x7fb1d40b4300 2022-06-29T15:52:00.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.726+0000 7fb1d9b6c700 1 -- 172.21.15.47:0/2618447082 >> v1:172.21.15.64:6789/0 conn(0x7fb1d41471a0 legacy=0x7fb1d413cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:00.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.726+0000 7fb1d8b6a700 1 -- 172.21.15.47:0/2618447082 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3673649984 0 0) 0x7fb1d41467c0 con 0x7fb1d41471a0 2022-06-29T15:52:00.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.726+0000 7fb1d9b6c700 1 -- 172.21.15.47:0/2618447082 >> v1:172.21.15.47:6789/0 conn(0x7fb1d40b4300 legacy=0x7fb1d41430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:00.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.726+0000 7fb1d9b6c700 1 -- 172.21.15.47:0/2618447082 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d4146d00 con 0x7fb1d4060ee0 2022-06-29T15:52:00.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.726+0000 7fb1d9b6c700 1 --2- 172.21.15.47:0/2618447082 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1d4060ee0 0x7fb1d40612c0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 rx=0x7fb1d40ae790 tx=0x7fb1d0006df0).ready entity=mon.0 client_cookie=461c068c1b74d6fb server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:00.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.726+0000 7fb1d8b6a700 1 -- 172.21.15.47:0/2618447082 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1d000d610 con 0x7fb1d4060ee0 2022-06-29T15:52:00.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.726+0000 7fb1d8b6a700 1 -- 172.21.15.47:0/2618447082 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1d0000e90 con 0x7fb1d4060ee0 2022-06-29T15:52:00.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.726+0000 7fb1d8b6a700 1 -- 172.21.15.47:0/2618447082 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1d000d960 con 0x7fb1d4060ee0 2022-06-29T15:52:00.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.726+0000 7fb1dbdd0700 1 -- 172.21.15.47:0/2618447082 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1d4060ee0 msgr2=0x7fb1d40612c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:00.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.726+0000 7fb1dbdd0700 1 --2- 172.21.15.47:0/2618447082 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1d4060ee0 0x7fb1d40612c0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 rx=0x7fb1d40ae790 tx=0x7fb1d0006df0).stop 2022-06-29T15:52:00.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.726+0000 7fb1dbdd0700 1 -- 172.21.15.47:0/2618447082 shutdown_connections 2022-06-29T15:52:00.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.726+0000 7fb1dbdd0700 1 --2- 172.21.15.47:0/2618447082 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1d4060ee0 0x7fb1d40612c0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:00.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.726+0000 7fb1dbdd0700 1 -- 172.21.15.47:0/2618447082 >> 172.21.15.47:0/2618447082 conn(0x7fb1d405c510 msgr2=0x7fb1d405f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:00.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.726+0000 7fb1dbdd0700 1 -- 172.21.15.47:0/2618447082 shutdown_connections 2022-06-29T15:52:00.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.726+0000 7fb1dbdd0700 1 -- 172.21.15.47:0/2618447082 wait complete. 2022-06-29T15:52:00.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.730+0000 7fb1dbdd0700 1 Processor -- start 2022-06-29T15:52:00.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.730+0000 7fb1dbdd0700 1 -- start start 2022-06-29T15:52:00.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.730+0000 7fb1dbdd0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1d4060ee0 0x7fb1d40b1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:00.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.730+0000 7fb1dbdd0700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb1d40b4300 0x7fb1d4137370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:00.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.730+0000 7fb1dbdd0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1d41471a0 0x7fb1d4155750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:00.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.730+0000 7fb1dbdd0700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d4137a10 con 0x7fb1d4060ee0 2022-06-29T15:52:00.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.730+0000 7fb1dbdd0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d414a8c0 con 0x7fb1d41471a0 2022-06-29T15:52:00.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.730+0000 7fb1dbdd0700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d4140f60 con 0x7fb1d40b4300 2022-06-29T15:52:00.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.730+0000 7fb1d9b6c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1d4060ee0 0x7fb1d40b1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:00.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.730+0000 7fb1d9b6c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1d4060ee0 0x7fb1d40b1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54650/0 (socket says 172.21.15.47:54650) 2022-06-29T15:52:00.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.734+0000 7fb1d9b6c700 1 -- 172.21.15.47:0/3130440747 learned_addr learned my addr 172.21.15.47:0/3130440747 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:00.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.734+0000 7fb1d936b700 1 --2- 172.21.15.47:0/3130440747 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb1d40b4300 0x7fb1d4137370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:00.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.734+0000 7fb1da36d700 1 --2- 172.21.15.47:0/3130440747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1d41471a0 0x7fb1d4155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:00.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.734+0000 7fb1d9b6c700 1 -- 172.21.15.47:0/3130440747 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb1d40b4300 msgr2=0x7fb1d4137370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:00.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.734+0000 7fb1d9b6c700 1 --2- 172.21.15.47:0/3130440747 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb1d40b4300 0x7fb1d4137370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:00.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.734+0000 7fb1d9b6c700 1 -- 172.21.15.47:0/3130440747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1d41471a0 msgr2=0x7fb1d4155750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:00.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.734+0000 7fb1d9b6c700 1 --2- 172.21.15.47:0/3130440747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1d41471a0 0x7fb1d4155750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:00.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.734+0000 7fb1d9b6c700 1 -- 172.21.15.47:0/3130440747 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d000b180 con 0x7fb1d4060ee0 2022-06-29T15:52:00.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.734+0000 7fb1da36d700 1 --2- 172.21.15.47:0/3130440747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1d41471a0 0x7fb1d4155750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:52:00.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.734+0000 7fb1d9b6c700 1 --2- 172.21.15.47:0/3130440747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1d4060ee0 0x7fb1d40b1010 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 rx=0x7fb1d40ae790 tx=0x7fb1d0014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:00.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.734+0000 7fb1c2ffd700 1 -- 172.21.15.47:0/3130440747 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1d0013e50 con 0x7fb1d4060ee0 2022-06-29T15:52:00.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.734+0000 7fb1c2ffd700 1 -- 172.21.15.47:0/3130440747 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1d0002f70 con 0x7fb1d4060ee0 2022-06-29T15:52:00.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.734+0000 7fb1dbdd0700 1 -- 172.21.15.47:0/3130440747 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1d4141200 con 0x7fb1d4060ee0 2022-06-29T15:52:00.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.734+0000 7fb1c2ffd700 1 -- 172.21.15.47:0/3130440747 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1d001e5f0 con 0x7fb1d4060ee0 2022-06-29T15:52:00.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.734+0000 7fb1dbdd0700 1 -- 172.21.15.47:0/3130440747 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1d4141940 con 0x7fb1d4060ee0 2022-06-29T15:52:00.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.734+0000 7fb1dbdd0700 1 -- 172.21.15.47:0/3130440747 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1d40573d0 con 0x7fb1d4060ee0 2022-06-29T15:52:00.748 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.734+0000 7fb1c2ffd700 1 -- 172.21.15.47:0/3130440747 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb1d0025030 con 0x7fb1d4060ee0 2022-06-29T15:52:00.749 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.738+0000 7fb1c2ffd700 1 --2- 172.21.15.47:0/3130440747 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb1b4053a70 0x7fb1b4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:00.749 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.738+0000 7fb1c2ffd700 1 -- 172.21.15.47:0/3130440747 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb1d006ceb0 con 0x7fb1d4060ee0 2022-06-29T15:52:00.749 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.738+0000 7fb1d936b700 1 --2- 172.21.15.47:0/3130440747 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb1b4053a70 0x7fb1b4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:00.749 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.738+0000 7fb1d936b700 1 --2- 172.21.15.47:0/3130440747 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb1b4053a70 0x7fb1b4055f30 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7fb1c400d000 tx=0x7fb1c4009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:00.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.742+0000 7fb1c2ffd700 1 -- 172.21.15.47:0/3130440747 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb1d0043bf0 con 0x7fb1d4060ee0 2022-06-29T15:52:00.883 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.878+0000 7fb1dbdd0700 1 -- 172.21.15.47:0/3130440747 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb1d40b5e50 con 0x7fb1d4060ee0 2022-06-29T15:52:00.884 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.878+0000 7fb1c2ffd700 1 -- 172.21.15.47:0/3130440747 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v151) v1 ==== 100+0+10 (secure 0 0 0) 0x7fb1d0023070 con 0x7fb1d4060ee0 2022-06-29T15:52:00.884 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:52:00.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.878+0000 7fb1dbdd0700 1 -- 172.21.15.47:0/3130440747 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb1b4053a70 msgr2=0x7fb1b4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:00.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.878+0000 7fb1dbdd0700 1 --2- 172.21.15.47:0/3130440747 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb1b4053a70 0x7fb1b4055f30 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7fb1c400d000 tx=0x7fb1c4009040).stop 2022-06-29T15:52:00.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.878+0000 7fb1dbdd0700 1 -- 172.21.15.47:0/3130440747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1d4060ee0 msgr2=0x7fb1d40b1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:00.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.878+0000 7fb1dbdd0700 1 --2- 172.21.15.47:0/3130440747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1d4060ee0 0x7fb1d40b1010 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 rx=0x7fb1d40ae790 tx=0x7fb1d0014000).stop 2022-06-29T15:52:00.888 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.878+0000 7fb1dbdd0700 1 -- 172.21.15.47:0/3130440747 shutdown_connections 2022-06-29T15:52:00.888 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.878+0000 7fb1dbdd0700 1 --2- 172.21.15.47:0/3130440747 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb1b4053a70 0x7fb1b4055f30 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:00.888 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.878+0000 7fb1dbdd0700 1 --2- 172.21.15.47:0/3130440747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1d41471a0 0x7fb1d4155750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:00.888 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.878+0000 7fb1dbdd0700 1 --2- 172.21.15.47:0/3130440747 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb1d40b4300 0x7fb1d4137370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:00.888 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.878+0000 7fb1dbdd0700 1 --2- 172.21.15.47:0/3130440747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1d4060ee0 0x7fb1d40b1010 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:00.889 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.878+0000 7fb1dbdd0700 1 -- 172.21.15.47:0/3130440747 >> 172.21.15.47:0/3130440747 conn(0x7fb1d405c510 msgr2=0x7fb1d41444e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:00.889 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.878+0000 7fb1dbdd0700 1 -- 172.21.15.47:0/3130440747 shutdown_connections 2022-06-29T15:52:00.889 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.878+0000 7fb1dbdd0700 1 -- 172.21.15.47:0/3130440747 wait complete. 2022-06-29T15:52:00.898 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-29T15:52:00.899 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:52:01.003 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.994+0000 7f5373179700 1 Processor -- start 2022-06-29T15:52:01.003 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.998+0000 7f5373179700 1 -- start start 2022-06-29T15:52:01.005 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.998+0000 7f5373179700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f536c0aca80 0x7f536c13f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:01.006 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.998+0000 7f5373179700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f536c0af190 0x7f536c13fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:01.006 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.998+0000 7f5373179700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f536c0af710 con 0x7f536c05f760 2022-06-29T15:52:01.006 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.998+0000 7f5373179700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f536c0b2830 con 0x7f536c0af190 2022-06-29T15:52:01.006 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.998+0000 7f5373179700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f536c060d40 con 0x7f536c0aca80 2022-06-29T15:52:01.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.998+0000 7f536bfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f536c0af190 0x7f536c13fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:01.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.998+0000 7f536bfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f536c0af190 0x7f536c13fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54658/0 (socket says 172.21.15.47:54658) 2022-06-29T15:52:01.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.998+0000 7f536bfff700 1 -- 172.21.15.47:0/1178051564 learned_addr learned my addr 172.21.15.47:0/1178051564 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:01.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.998+0000 7f5370f15700 1 --2- 172.21.15.47:0/1178051564 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f536c0aca80 0x7f536c13f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:01.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.998+0000 7f536bfff700 1 -- 172.21.15.47:0/1178051564 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f536c0aca80 msgr2=0x7f536c13f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:01.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.998+0000 7f536bfff700 1 --2- 172.21.15.47:0/1178051564 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f536c0aca80 0x7f536c13f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:01.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.998+0000 7f536bfff700 1 -- 172.21.15.47:0/1178051564 >> v1:172.21.15.47:6789/0 conn(0x7f536c05f760 legacy=0x7f536c0b2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:01.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.998+0000 7f536bfff700 1 -- 172.21.15.47:0/1178051564 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f536c0aeec0 con 0x7f536c0af190 2022-06-29T15:52:01.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.998+0000 7f5370f15700 1 --2- 172.21.15.47:0/1178051564 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f536c0aca80 0x7f536c13f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:52:01.009 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.998+0000 7f536bfff700 1 --2- 172.21.15.47:0/1178051564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f536c0af190 0x7f536c13fac0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 rx=0x7f5358002f70 tx=0x7f5358006eb0).ready entity=mon.0 client_cookie=c9023cf93fdc69a6 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:01.009 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.998+0000 7f536b7fe700 1 -- 172.21.15.47:0/1178051564 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f535800d5c0 con 0x7f536c0af190 2022-06-29T15:52:01.009 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.998+0000 7f536b7fe700 1 -- 172.21.15.47:0/1178051564 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5358000e90 con 0x7f536c0af190 2022-06-29T15:52:01.009 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.998+0000 7f536b7fe700 1 -- 172.21.15.47:0/1178051564 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f535800d950 con 0x7f536c0af190 2022-06-29T15:52:01.012 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.998+0000 7f5373179700 1 -- 172.21.15.47:0/1178051564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f536c0af190 msgr2=0x7f536c13fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:01.013 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.998+0000 7f5373179700 1 --2- 172.21.15.47:0/1178051564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f536c0af190 0x7f536c13fac0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 rx=0x7f5358002f70 tx=0x7f5358006eb0).stop 2022-06-29T15:52:01.013 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.998+0000 7f5373179700 1 -- 172.21.15.47:0/1178051564 shutdown_connections 2022-06-29T15:52:01.013 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.998+0000 7f5373179700 1 --2- 172.21.15.47:0/1178051564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f536c0af190 0x7f536c13fac0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:01.013 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.998+0000 7f5373179700 1 --2- 172.21.15.47:0/1178051564 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f536c0aca80 0x7f536c13f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:01.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.998+0000 7f5373179700 1 -- 172.21.15.47:0/1178051564 >> 172.21.15.47:0/1178051564 conn(0x7f536c0626f0 msgr2=0x7f536c062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:01.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.998+0000 7f5373179700 1 -- 172.21.15.47:0/1178051564 shutdown_connections 2022-06-29T15:52:01.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:00.998+0000 7f5373179700 1 -- 172.21.15.47:0/1178051564 wait complete. 2022-06-29T15:52:01.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:01.002+0000 7f5373179700 1 Processor -- start 2022-06-29T15:52:01.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:01.002+0000 7f5373179700 1 -- start start 2022-06-29T15:52:01.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:01.002+0000 7f5373179700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f536c05f760 0x7f536c134a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:01.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:01.002+0000 7f5373179700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f536c0aca80 0x7f536c1561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:01.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:01.002+0000 7f5373179700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f536c0af190 0x7f536c1566c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:01.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:01.002+0000 7f5373179700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536c156bb0 con 0x7f536c05f760 2022-06-29T15:52:01.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:01.002+0000 7f5373179700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536c140290 con 0x7f536c0af190 2022-06-29T15:52:01.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:01.002+0000 7f5373179700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f536c140530 con 0x7f536c0aca80 2022-06-29T15:52:01.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:01.002+0000 7f5370f15700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f536c05f760 0x7f536c134a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:01.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:01.002+0000 7f5370f15700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f536c05f760 0x7f536c134a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54666/0 (socket says 172.21.15.47:54666) 2022-06-29T15:52:01.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:01.002+0000 7f536bfff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f536c0aca80 0x7f536c1561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:01.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:01.002+0000 7f5370f15700 1 -- 172.21.15.47:0/2417966983 learned_addr learned my addr 172.21.15.47:0/2417966983 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:01.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:01.002+0000 7f5371716700 1 --2- 172.21.15.47:0/2417966983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f536c0af190 0x7f536c1566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:01.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:01.002+0000 7f5370f15700 1 -- 172.21.15.47:0/2417966983 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f536c0aca80 msgr2=0x7f536c1561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:01.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:01.002+0000 7f5370f15700 1 --2- 172.21.15.47:0/2417966983 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f536c0aca80 0x7f536c1561d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:01.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:01.002+0000 7f5370f15700 1 -- 172.21.15.47:0/2417966983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f536c0af190 msgr2=0x7f536c1566c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:01.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:01.002+0000 7f5370f15700 1 --2- 172.21.15.47:0/2417966983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f536c0af190 0x7f536c1566c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:01.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:01.002+0000 7f5370f15700 1 -- 172.21.15.47:0/2417966983 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f535800b1e0 con 0x7f536c05f760 2022-06-29T15:52:01.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:01.002+0000 7f5370f15700 1 --2- 172.21.15.47:0/2417966983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f536c05f760 0x7f536c134a90 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 rx=0x7f5360007000 tx=0x7f5360008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:01.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:01.002+0000 7f5369ffb700 1 -- 172.21.15.47:0/2417966983 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5360010580 con 0x7f536c05f760 2022-06-29T15:52:01.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:01.002+0000 7f5369ffb700 1 -- 172.21.15.47:0/2417966983 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53600106e0 con 0x7f536c05f760 2022-06-29T15:52:01.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:01.002+0000 7f5373179700 1 -- 172.21.15.47:0/2417966983 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f536c155d30 con 0x7f536c05f760 2022-06-29T15:52:01.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:01.002+0000 7f5369ffb700 1 -- 172.21.15.47:0/2417966983 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5360010a40 con 0x7f536c05f760 2022-06-29T15:52:01.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:01.002+0000 7f5373179700 1 -- 172.21.15.47:0/2417966983 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f536c14b850 con 0x7f536c05f760 2022-06-29T15:52:01.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:01.006+0000 7f5373179700 1 -- 172.21.15.47:0/2417966983 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f536c155ec0 con 0x7f536c05f760 2022-06-29T15:52:01.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:01.006+0000 7f5369ffb700 1 -- 172.21.15.47:0/2417966983 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5360010ba0 con 0x7f536c05f760 2022-06-29T15:52:01.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:01.006+0000 7f5369ffb700 1 --2- 172.21.15.47:0/2417966983 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f534c053950 0x7f534c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:01.022 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:01.006+0000 7f5369ffb700 1 -- 172.21.15.47:0/2417966983 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f536006c200 con 0x7f536c05f760 2022-06-29T15:52:01.022 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:01.006+0000 7f536bfff700 1 --2- 172.21.15.47:0/2417966983 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f534c053950 0x7f534c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:01.022 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:01.006+0000 7f536bfff700 1 --2- 172.21.15.47:0/2417966983 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f534c053950 0x7f534c055e10 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7f535801c000 tx=0x7f5358015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:01.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:01.014+0000 7f5369ffb700 1 -- 172.21.15.47:0/2417966983 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5360042e50 con 0x7f536c05f760 2022-06-29T15:52:01.191 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:01.186+0000 7f5373179700 1 -- 172.21.15.47:0/2417966983 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f536c0573d0 con 0x7f536c05f760 2022-06-29T15:52:01.192 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:01.186+0000 7f5369ffb700 1 -- 172.21.15.47:0/2417966983 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+2495 (secure 0 0 0) 0x7f5360046470 con 0x7f536c05f760 2022-06-29T15:52:01.193 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:52:01.193 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":74,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8296,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/3321939421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3321939421},{"type":"v1","addr":"172.21.15.64:6833","nonce":3321939421}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":72},{"gid":8415,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1886881850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1886881850},{"type":"v1","addr":"172.21.15.47:6835","nonce":1886881850}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":72}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:51:52.828119+0000","modified":"2022-06-29T15:51:53.831305+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8421":{"gid":8421,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6837/2466961065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2466961065},{"type":"v1","addr":"172.21.15.47:6837","nonce":2466961065}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2022-06-29T15:52:01.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:01.186+0000 7f5373179700 1 -- 172.21.15.47:0/2417966983 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f534c053950 msgr2=0x7f534c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:01.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:01.186+0000 7f5373179700 1 --2- 172.21.15.47:0/2417966983 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f534c053950 0x7f534c055e10 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7f535801c000 tx=0x7f5358015000).stop 2022-06-29T15:52:01.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:01.190+0000 7f5373179700 1 -- 172.21.15.47:0/2417966983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f536c05f760 msgr2=0x7f536c134a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:01.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:01.190+0000 7f5373179700 1 --2- 172.21.15.47:0/2417966983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f536c05f760 0x7f536c134a90 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 rx=0x7f5360007000 tx=0x7f5360008040).stop 2022-06-29T15:52:01.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:01.190+0000 7f5373179700 1 -- 172.21.15.47:0/2417966983 shutdown_connections 2022-06-29T15:52:01.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:01.190+0000 7f5373179700 1 --2- 172.21.15.47:0/2417966983 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f534c053950 0x7f534c055e10 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:01.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:01.190+0000 7f5373179700 1 --2- 172.21.15.47:0/2417966983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f536c0af190 0x7f536c1566c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:01.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:01.190+0000 7f5373179700 1 --2- 172.21.15.47:0/2417966983 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f536c0aca80 0x7f536c1561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:01.198 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:01.190+0000 7f5373179700 1 --2- 172.21.15.47:0/2417966983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f536c05f760 0x7f536c134a90 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:01.198 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:01.190+0000 7f5373179700 1 -- 172.21.15.47:0/2417966983 >> 172.21.15.47:0/2417966983 conn(0x7f536c0626f0 msgr2=0x7f536c0b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:01.198 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:01.190+0000 7f5373179700 1 -- 172.21.15.47:0/2417966983 shutdown_connections 2022-06-29T15:52:01.198 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:01.190+0000 7f5373179700 1 -- 172.21.15.47:0/2417966983 wait complete. 2022-06-29T15:52:01.199 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 74 2022-06-29T15:52:01.513 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-29T15:51:52.828119+0000', 'modified': '2022-06-29T15:51:53.831305+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8421}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8421': {'gid': 8421, 'name': 'c', 'rank': 0, 'incarnation': 73, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.47:6837/2466961065', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6836', 'nonce': 2466961065}, {'type': 'v1', 'addr': '172.21.15.47:6837', 'nonce': 2466961065}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-29T15:52:01.514 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-29T15:52:01.515 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2022-06-29T15:52:01.631 INFO:teuthology.orchestra.run.smithi047.stdout:{"cluster_fsid":"9a68894a-53ce-400e-838b-14eab91b834c","whoami":0,"id":8421,"want_state":"up:active","state":"up:active","rank_uptime":8.7869769830000006,"mdsmap_epoch":74,"osdmap_epoch":151,"osdmap_epoch_barrier":150,"uptime":13.807796332000001} 2022-06-29T15:52:01.632 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9a68894a-53ce-400e-838b-14eab91b834c", "id": 8421, "mdsmap_epoch": 74, "osdmap_epoch": 151, "osdmap_epoch_barrier": 150, "rank_uptime": 8.786976983, "state": "up:active", "uptime": 13.807796332, "want_state": "up:active", "whoami": 0 } 2022-06-29T15:52:01.632 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-29T15:52:01.633 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2022-06-29T15:52:01.633 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi164.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2022-06-29T15:52:01.633 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:52:01.634 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:52:01.642 DEBUG:teuthology.orchestra.run.smithi164:> sudo modprobe fuse 2022-06-29T15:52:01.655 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:52:01.656 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:52:01.666 INFO:teuthology.orchestra.run.smithi164.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:52:01.667 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:52:01.668 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:52:01.669 DEBUG:teuthology.orchestra.run.smithi164:> ls /sys/fs/fuse/connections 2022-06-29T15:52:01.717 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-29T15:52:01.718 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:52:01.721 DEBUG:teuthology.orchestra.run.smithi164:> sudo modprobe fuse 2022-06-29T15:52:01.738 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:52:01.739 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:52:01.793 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi164.stderr:2022-06-29T15:52:01.784+0000 7f66931d3080 -1 init, newargv = 0x5649399fb870 newargc=9 2022-06-29T15:52:01.793 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi164.stderr:ceph-fuse[13089]: starting ceph client 2022-06-29T15:52:01.795 INFO:teuthology.orchestra.run.smithi164.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:52:01.796 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:52:01.797 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:52:01.797 DEBUG:teuthology.orchestra.run.smithi164:> ls /sys/fs/fuse/connections 2022-06-29T15:52:01.804 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi164.stderr:ceph-fuse[13089]: starting fuse 2022-06-29T15:52:02.903 DEBUG:teuthology.orchestra.run.smithi164:> sudo modprobe fuse 2022-06-29T15:52:02.922 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:52:02.923 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:52:02.978 INFO:teuthology.orchestra.run.smithi164.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:52:02.979 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:52:02.980 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:52:02.980 DEBUG:teuthology.orchestra.run.smithi164:> ls /sys/fs/fuse/connections 2022-06-29T15:52:02.985 INFO:teuthology.orchestra.run.smithi164.stdout:52 2022-06-29T15:52:02.986 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-29T15:52:02.986 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:52:02.987 DEBUG:teuthology.orchestra.run.smithi164:> sudo python3 -c ' 2022-06-29T15:52:02.987 DEBUG:teuthology.orchestra.run.smithi164:> import glob 2022-06-29T15:52:02.988 DEBUG:teuthology.orchestra.run.smithi164:> import re 2022-06-29T15:52:02.988 DEBUG:teuthology.orchestra.run.smithi164:> import os 2022-06-29T15:52:02.988 DEBUG:teuthology.orchestra.run.smithi164:> import subprocess 2022-06-29T15:52:02.989 DEBUG:teuthology.orchestra.run.smithi164:> 2022-06-29T15:52:02.989 DEBUG:teuthology.orchestra.run.smithi164:> def find_socket(client_name): 2022-06-29T15:52:02.990 DEBUG:teuthology.orchestra.run.smithi164:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2022-06-29T15:52:02.990 DEBUG:teuthology.orchestra.run.smithi164:> files = glob.glob(asok_path) 2022-06-29T15:52:02.991 DEBUG:teuthology.orchestra.run.smithi164:> 2022-06-29T15:52:02.991 DEBUG:teuthology.orchestra.run.smithi164:> # Given a non-glob path, it better be there 2022-06-29T15:52:02.991 DEBUG:teuthology.orchestra.run.smithi164:> if "*" not in asok_path: 2022-06-29T15:52:02.992 DEBUG:teuthology.orchestra.run.smithi164:> assert(len(files) == 1) 2022-06-29T15:52:02.992 DEBUG:teuthology.orchestra.run.smithi164:> return files[0] 2022-06-29T15:52:02.993 DEBUG:teuthology.orchestra.run.smithi164:> 2022-06-29T15:52:02.993 DEBUG:teuthology.orchestra.run.smithi164:> for f in files: 2022-06-29T15:52:02.993 DEBUG:teuthology.orchestra.run.smithi164:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-29T15:52:02.994 DEBUG:teuthology.orchestra.run.smithi164:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-29T15:52:02.994 DEBUG:teuthology.orchestra.run.smithi164:> return f 2022-06-29T15:52:02.995 DEBUG:teuthology.orchestra.run.smithi164:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-29T15:52:02.995 DEBUG:teuthology.orchestra.run.smithi164:> 2022-06-29T15:52:02.996 DEBUG:teuthology.orchestra.run.smithi164:> print(find_socket("client.1")) 2022-06-29T15:52:02.996 DEBUG:teuthology.orchestra.run.smithi164:> ' 2022-06-29T15:52:03.036 INFO:teuthology.orchestra.run.smithi164.stdout:/var/run/ceph/ceph-client.1.13089.asok 2022-06-29T15:52:03.042 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.13089.asok 2022-06-29T15:52:03.042 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:52:03.043 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.13089.asok status 2022-06-29T15:52:03.175 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2022-06-29T15:52:03.176 INFO:teuthology.orchestra.run.smithi164.stdout: "metadata": { 2022-06-29T15:52:03.176 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_sha1": "f25ffac72e5a9e63860ff7deb6767a11f0559a16", 2022-06-29T15:52:03.176 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_version": "ceph version 15.2.16-304-gf25ffac7 (f25ffac72e5a9e63860ff7deb6767a11f0559a16) octopus (stable)", 2022-06-29T15:52:03.177 INFO:teuthology.orchestra.run.smithi164.stdout: "entity_id": "1", 2022-06-29T15:52:03.177 INFO:teuthology.orchestra.run.smithi164.stdout: "hostname": "smithi164", 2022-06-29T15:52:03.177 INFO:teuthology.orchestra.run.smithi164.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2022-06-29T15:52:03.177 INFO:teuthology.orchestra.run.smithi164.stdout: "pid": "13089", 2022-06-29T15:52:03.178 INFO:teuthology.orchestra.run.smithi164.stdout: "root": "/" 2022-06-29T15:52:03.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2022-06-29T15:52:03.178 INFO:teuthology.orchestra.run.smithi164.stdout: "dentry_count": 0, 2022-06-29T15:52:03.178 INFO:teuthology.orchestra.run.smithi164.stdout: "dentry_pinned_count": 0, 2022-06-29T15:52:03.178 INFO:teuthology.orchestra.run.smithi164.stdout: "id": 8628, 2022-06-29T15:52:03.179 INFO:teuthology.orchestra.run.smithi164.stdout: "inst": { 2022-06-29T15:52:03.179 INFO:teuthology.orchestra.run.smithi164.stdout: "name": { 2022-06-29T15:52:03.179 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "client", 2022-06-29T15:52:03.179 INFO:teuthology.orchestra.run.smithi164.stdout: "num": 8628 2022-06-29T15:52:03.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2022-06-29T15:52:03.180 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": { 2022-06-29T15:52:03.180 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "v1", 2022-06-29T15:52:03.180 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": "172.21.15.164:0", 2022-06-29T15:52:03.180 INFO:teuthology.orchestra.run.smithi164.stdout: "nonce": 1030362232 2022-06-29T15:52:03.181 INFO:teuthology.orchestra.run.smithi164.stdout: } 2022-06-29T15:52:03.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2022-06-29T15:52:03.181 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": { 2022-06-29T15:52:03.181 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "v1", 2022-06-29T15:52:03.182 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": "172.21.15.164:0", 2022-06-29T15:52:03.182 INFO:teuthology.orchestra.run.smithi164.stdout: "nonce": 1030362232 2022-06-29T15:52:03.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2022-06-29T15:52:03.182 INFO:teuthology.orchestra.run.smithi164.stdout: "inst_str": "client.8628 172.21.15.164:0/1030362232", 2022-06-29T15:52:03.183 INFO:teuthology.orchestra.run.smithi164.stdout: "addr_str": "172.21.15.164:0/1030362232", 2022-06-29T15:52:03.183 INFO:teuthology.orchestra.run.smithi164.stdout: "inode_count": 1, 2022-06-29T15:52:03.183 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_epoch": 74, 2022-06-29T15:52:03.183 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_epoch": 151, 2022-06-29T15:52:03.184 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_epoch_barrier": 0, 2022-06-29T15:52:03.184 INFO:teuthology.orchestra.run.smithi164.stdout: "blacklisted": false, 2022-06-29T15:52:03.184 INFO:teuthology.orchestra.run.smithi164.stdout: "fs_name": "cephfs" 2022-06-29T15:52:03.184 INFO:teuthology.orchestra.run.smithi164.stdout:} 2022-06-29T15:52:03.185 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:52:03.186 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:52:03.186 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:52:03.194 INFO:teuthology.orchestra.run.smithi164.stdout:fuseblk 2022-06-29T15:52:03.195 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-06-29T15:52:03.196 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:52:03.197 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:52:03.216 INFO:teuthology.orchestra.run:Running command with timeout 10800 2022-06-29T15:52:03.216 DEBUG:teuthology.orchestra.run.smithi099:> cd /home/ubuntu/cephtest/mnt.0 && sudo getfattr --only-values -n ceph.dir.rctime . 2022-06-29T15:52:03.251 INFO:teuthology.orchestra.run.smithi099.stdout:1656517923.208984471 2022-06-29T15:52:03.252 INFO:tasks.cephfs.test_misc:rctime = 1656517923.208984471 2022-06-29T15:52:03.252 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-06-29T15:52:03.253 DEBUG:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:52:03.275 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:52:03.295 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:52:03.316 DEBUG:teuthology.orchestra.run.smithi164:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:52:03.337 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-06-29T15:52:03.337 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:52:03.338 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:52:03.338 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:52:03.346 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2022-06-29T15:52:03.347 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-29T15:52:03.348 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi099.front.sepia.ceph.com... 2022-06-29T15:52:03.348 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-06-29T15:52:03.348 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:52:03.404 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[13519]: fuse finished with error 0 and tester_r 0 2022-06-29T15:52:03.406 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:52:03.406 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:52:03.407 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:52:03.414 INFO:teuthology.orchestra.run.smithi099.stdout:ext2/ext3 2022-06-29T15:52:03.415 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-06-29T15:52:03.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-29T15:52:03.465 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:52:03.473 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:52:03.473 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:52:03.474 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:52:03.482 INFO:teuthology.orchestra.run.smithi164.stdout:fuseblk 2022-06-29T15:52:03.483 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-06-29T15:52:03.484 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi164.front.sepia.ceph.com... 2022-06-29T15:52:03.484 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-06-29T15:52:03.484 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:52:03.542 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi164.stderr:ceph-fuse[13089]: fuse finished with error 0 and tester_r 0 2022-06-29T15:52:03.544 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:52:03.545 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:52:03.545 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:52:03.551 INFO:teuthology.orchestra.run.smithi164.stdout:ext2/ext3 2022-06-29T15:52:03.552 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-06-29T15:52:03.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-29T15:52:03.589 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:52:03.598 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-06-29T15:52:03.709 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.702+0000 7f6aa4c35700 1 Processor -- start 2022-06-29T15:52:03.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.702+0000 7f6aa4c35700 1 -- start start 2022-06-29T15:52:03.712 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.702+0000 7f6aa4c35700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6aa0060eb0 0x7f6aa00b1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:03.712 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.702+0000 7f6aa4c35700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6aa0145090 0x7f6aa00b2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:03.712 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.702+0000 7f6aa4c35700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aa00a9120 con 0x7f6aa00b5c60 2022-06-29T15:52:03.712 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.702+0000 7f6aa4c35700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6aa00b0b80 con 0x7f6aa0145090 2022-06-29T15:52:03.713 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.702+0000 7f6aa4c35700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6aa00a89e0 con 0x7f6aa0060eb0 2022-06-29T15:52:03.713 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.702+0000 7f6a9f7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f6aa00b5c60 0x7f6aa0144980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40226/0 (socket says 172.21.15.47:40226) 2022-06-29T15:52:03.713 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.702+0000 7f6a9f7fe700 1 -- 172.21.15.47:0/4187365763 learned_addr learned my addr 172.21.15.47:0/4187365763 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:52:03.713 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.702+0000 7f6a9ffff700 1 --2- 172.21.15.47:0/4187365763 >> v2:172.21.15.47:3300/0 conn(0x7f6aa0145090 0x7f6aa00b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:03.714 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.702+0000 7f6a9effd700 1 --2- 172.21.15.47:0/4187365763 >> v2:172.21.15.64:3300/0 conn(0x7f6aa0060eb0 0x7f6aa00b1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:03.714 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.702+0000 7f6a9e7fc700 1 -- 172.21.15.47:0/4187365763 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3903498677 0 0) 0x7f6aa00a9120 con 0x7f6aa00b5c60 2022-06-29T15:52:03.714 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.706+0000 7f6a9e7fc700 1 -- 172.21.15.47:0/4187365763 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a8c003260 con 0x7f6aa00b5c60 2022-06-29T15:52:03.714 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.706+0000 7f6a9ffff700 1 -- 172.21.15.47:0/4187365763 >> v2:172.21.15.64:3300/0 conn(0x7f6aa0060eb0 msgr2=0x7f6aa00b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:03.714 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.706+0000 7f6a9ffff700 1 --2- 172.21.15.47:0/4187365763 >> v2:172.21.15.64:3300/0 conn(0x7f6aa0060eb0 0x7f6aa00b1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:03.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.706+0000 7f6a9ffff700 1 -- 172.21.15.47:0/4187365763 >> v1:172.21.15.47:6789/0 conn(0x7f6aa00b5c60 legacy=0x7f6aa0144980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:03.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.706+0000 7f6a9ffff700 1 -- 172.21.15.47:0/4187365763 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa00a8c80 con 0x7f6aa0145090 2022-06-29T15:52:03.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.706+0000 7f6a9ffff700 1 --2- 172.21.15.47:0/4187365763 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6aa0145090 0x7f6aa00b2320 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 rx=0x7f6a98002f70 tx=0x7f6a98006df0).ready entity=mon.0 client_cookie=624f4769fc778fba server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:03.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.706+0000 7f6a9e7fc700 1 -- 172.21.15.47:0/4187365763 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a9800d4f0 con 0x7f6aa0145090 2022-06-29T15:52:03.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.706+0000 7f6a9e7fc700 1 -- 172.21.15.47:0/4187365763 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a98000e90 con 0x7f6aa0145090 2022-06-29T15:52:03.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.706+0000 7f6a9e7fc700 1 -- 172.21.15.47:0/4187365763 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a9800d840 con 0x7f6aa0145090 2022-06-29T15:52:03.719 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.706+0000 7f6aa4c35700 1 -- 172.21.15.47:0/4187365763 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6aa0145090 msgr2=0x7f6aa00b2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:03.720 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.706+0000 7f6aa4c35700 1 --2- 172.21.15.47:0/4187365763 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6aa0145090 0x7f6aa00b2320 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 rx=0x7f6a98002f70 tx=0x7f6a98006df0).stop 2022-06-29T15:52:03.720 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.706+0000 7f6aa4c35700 1 -- 172.21.15.47:0/4187365763 shutdown_connections 2022-06-29T15:52:03.720 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.706+0000 7f6aa4c35700 1 --2- 172.21.15.47:0/4187365763 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6aa0145090 0x7f6aa00b2320 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:03.720 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.706+0000 7f6aa4c35700 1 --2- 172.21.15.47:0/4187365763 >> v2:172.21.15.64:3300/0 conn(0x7f6aa0060eb0 0x7f6aa00b1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:03.721 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.706+0000 7f6aa4c35700 1 -- 172.21.15.47:0/4187365763 >> 172.21.15.47:0/4187365763 conn(0x7f6aa0062560 msgr2=0x7f6aa0062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:03.721 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.706+0000 7f6aa4c35700 1 -- 172.21.15.47:0/4187365763 shutdown_connections 2022-06-29T15:52:03.721 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.706+0000 7f6aa4c35700 1 -- 172.21.15.47:0/4187365763 wait complete. 2022-06-29T15:52:03.721 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.706+0000 7f6aa4c35700 1 Processor -- start 2022-06-29T15:52:03.722 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.706+0000 7f6aa4c35700 1 -- start start 2022-06-29T15:52:03.722 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.706+0000 7f6aa4c35700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6aa0060eb0 0x7f6aa00b35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:03.722 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.710+0000 7f6aa4c35700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6aa00b5c60 0x7f6aa0157200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:03.722 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.710+0000 7f6aa4c35700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6aa0145090 0x7f6aa014c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:03.723 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.710+0000 7f6aa4c35700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa01576f0 con 0x7f6aa00b5c60 2022-06-29T15:52:03.723 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.710+0000 7f6aa4c35700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa013b8d0 con 0x7f6aa0145090 2022-06-29T15:52:03.723 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.710+0000 7f6aa4c35700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa013bb70 con 0x7f6aa0060eb0 2022-06-29T15:52:03.723 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.710+0000 7f6a9effd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6aa00b5c60 0x7f6aa0157200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:03.724 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.710+0000 7f6a9effd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6aa00b5c60 0x7f6aa0157200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54680/0 (socket says 172.21.15.47:54680) 2022-06-29T15:52:03.724 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.710+0000 7f6a9effd700 1 -- 172.21.15.47:0/2648384604 learned_addr learned my addr 172.21.15.47:0/2648384604 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:03.724 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.710+0000 7f6a9f7fe700 1 --2- 172.21.15.47:0/2648384604 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6aa0060eb0 0x7f6aa00b35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:03.724 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.710+0000 7f6a9ffff700 1 --2- 172.21.15.47:0/2648384604 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6aa0145090 0x7f6aa014c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:03.724 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.710+0000 7f6a9effd700 1 -- 172.21.15.47:0/2648384604 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6aa0060eb0 msgr2=0x7f6aa00b35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:03.725 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.710+0000 7f6a9effd700 1 --2- 172.21.15.47:0/2648384604 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6aa0060eb0 0x7f6aa00b35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:03.725 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.710+0000 7f6a9effd700 1 -- 172.21.15.47:0/2648384604 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6aa0145090 msgr2=0x7f6aa014c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:03.725 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.710+0000 7f6a9effd700 1 --2- 172.21.15.47:0/2648384604 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6aa0145090 0x7f6aa014c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:03.725 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.710+0000 7f6a9effd700 1 -- 172.21.15.47:0/2648384604 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a9800b180 con 0x7f6aa00b5c60 2022-06-29T15:52:03.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.710+0000 7f6a9f7fe700 1 --2- 172.21.15.47:0/2648384604 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6aa0060eb0 0x7f6aa00b35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:52:03.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.710+0000 7f6a9effd700 1 --2- 172.21.15.47:0/2648384604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6aa00b5c60 0x7f6aa0157200 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 rx=0x7f6a9000c000 tx=0x7f6a900091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:03.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.710+0000 7f6a9cff9700 1 -- 172.21.15.47:0/2648384604 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a9000c850 con 0x7f6aa00b5c60 2022-06-29T15:52:03.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.710+0000 7f6a9cff9700 1 -- 172.21.15.47:0/2648384604 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a9000c9b0 con 0x7f6aa00b5c60 2022-06-29T15:52:03.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.710+0000 7f6aa4c35700 1 -- 172.21.15.47:0/2648384604 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aa013be10 con 0x7f6aa00b5c60 2022-06-29T15:52:03.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.710+0000 7f6a9cff9700 1 -- 172.21.15.47:0/2648384604 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a9000cd10 con 0x7f6aa00b5c60 2022-06-29T15:52:03.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.710+0000 7f6aa4c35700 1 -- 172.21.15.47:0/2648384604 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aa013c370 con 0x7f6aa00b5c60 2022-06-29T15:52:03.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.710+0000 7f6aa4c35700 1 -- 172.21.15.47:0/2648384604 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aa013bfa0 con 0x7f6aa00b5c60 2022-06-29T15:52:03.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.714+0000 7f6a9cff9700 1 -- 172.21.15.47:0/2648384604 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6a90022070 con 0x7f6aa00b5c60 2022-06-29T15:52:03.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.714+0000 7f6a9cff9700 1 --2- 172.21.15.47:0/2648384604 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6a80053b00 0x7f6a80055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:03.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.714+0000 7f6a9cff9700 1 -- 172.21.15.47:0/2648384604 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f6a9006cce0 con 0x7f6aa00b5c60 2022-06-29T15:52:03.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.714+0000 7f6a9f7fe700 1 --2- 172.21.15.47:0/2648384604 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6a80053b00 0x7f6a80055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:03.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.714+0000 7f6a9f7fe700 1 --2- 172.21.15.47:0/2648384604 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6a80053b00 0x7f6a80055fc0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7f6a9400b000 tx=0x7f6a94009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:03.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.718+0000 7f6a9cff9700 1 -- 172.21.15.47:0/2648384604 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6a90043aa0 con 0x7f6aa00b5c60 2022-06-29T15:52:03.850 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.842+0000 7f6aa4c35700 1 -- 172.21.15.47:0/2648384604 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f6aa00573d0 con 0x7f6aa00b5c60 2022-06-29T15:52:03.852 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.846+0000 7f6a9cff9700 1 -- 172.21.15.47:0/2648384604 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v151) v1 ==== 81+0+34051 (secure 0 0 0) 0x7f6a90029090 con 0x7f6aa00b5c60 2022-06-29T15:52:03.854 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:52:03.854 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2022-06-29T15:52:03.855 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 151, 2022-06-29T15:52:03.855 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "9a68894a-53ce-400e-838b-14eab91b834c", 2022-06-29T15:52:03.855 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2022-06-29T15:42:14.046361+0000", 2022-06-29T15:52:03.856 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2022-06-29T15:51:53.832186+0000", 2022-06-29T15:52:03.856 INFO:teuthology.orchestra.run.smithi047.stdout: "last_up_change": "2022-06-29T15:42:20.929789+0000", 2022-06-29T15:52:03.856 INFO:teuthology.orchestra.run.smithi047.stdout: "last_in_change": "2022-06-29T15:42:20.929789+0000", 2022-06-29T15:52:03.856 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-06-29T15:52:03.857 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_num": 5799936, 2022-06-29T15:52:03.857 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_set": [ 2022-06-29T15:52:03.857 INFO:teuthology.orchestra.run.smithi047.stdout: "pglog_hardlimit", 2022-06-29T15:52:03.857 INFO:teuthology.orchestra.run.smithi047.stdout: "purged_snapdirs", 2022-06-29T15:52:03.858 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_deletes", 2022-06-29T15:52:03.858 INFO:teuthology.orchestra.run.smithi047.stdout: "sortbitwise" 2022-06-29T15:52:03.858 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:52:03.858 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_version": 5, 2022-06-29T15:52:03.859 INFO:teuthology.orchestra.run.smithi047.stdout: "full_ratio": 0.94999998807907104, 2022-06-29T15:52:03.859 INFO:teuthology.orchestra.run.smithi047.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-06-29T15:52:03.859 INFO:teuthology.orchestra.run.smithi047.stdout: "nearfull_ratio": 0.85000002384185791, 2022-06-29T15:52:03.859 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_snapshot": "", 2022-06-29T15:52:03.859 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_max": 28, 2022-06-29T15:52:03.860 INFO:teuthology.orchestra.run.smithi047.stdout: "max_osd": 8, 2022-06-29T15:52:03.860 INFO:teuthology.orchestra.run.smithi047.stdout: "require_min_compat_client": "jewel", 2022-06-29T15:52:03.860 INFO:teuthology.orchestra.run.smithi047.stdout: "min_compat_client": "jewel", 2022-06-29T15:52:03.860 INFO:teuthology.orchestra.run.smithi047.stdout: "require_osd_release": "octopus", 2022-06-29T15:52:03.860 INFO:teuthology.orchestra.run.smithi047.stdout: "pools": [ 2022-06-29T15:52:03.861 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:03.861 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 1, 2022-06-29T15:52:03.861 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "device_health_metrics", 2022-06-29T15:52:03.861 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:42:16.971431+0000", 2022-06-29T15:52:03.862 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:52:03.862 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:52:03.862 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:52:03.862 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:52:03.863 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:52:03.863 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:52:03.863 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:52:03.863 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:52:03.864 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 1, 2022-06-29T15:52:03.864 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 1, 2022-06-29T15:52:03.864 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 1, 2022-06-29T15:52:03.864 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 1, 2022-06-29T15:52:03.865 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 1, 2022-06-29T15:52:03.865 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:52:03.865 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:52:03.865 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:52:03.866 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:52:03.866 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:52:03.866 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:52:03.866 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:52:03.867 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:03.867 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "16", 2022-06-29T15:52:03.867 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:52:03.867 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:52:03.868 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:52:03.868 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:52:03.868 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:52:03.868 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:52:03.869 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:52:03.869 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:52:03.869 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:52:03.869 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:52:03.870 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:52:03.870 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:52:03.870 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:52:03.870 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:52:03.870 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:52:03.871 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:52:03.871 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:52:03.871 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:52:03.871 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:52:03.872 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:52:03.872 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:52:03.872 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:52:03.872 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:52:03.873 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:52:03.873 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:52:03.873 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:52:03.874 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:03.874 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:52:03.874 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:52:03.874 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:52:03.875 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:52:03.875 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:52:03.875 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:52:03.875 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:52:03.876 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:52:03.876 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:52:03.876 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:52:03.876 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:52:03.877 INFO:teuthology.orchestra.run.smithi047.stdout: "options": { 2022-06-29T15:52:03.877 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_min": 1 2022-06-29T15:52:03.877 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:03.878 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:52:03.878 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_devicehealth": {} 2022-06-29T15:52:03.879 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:03.879 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:03.879 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:03.879 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 2, 2022-06-29T15:52:03.880 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "rbd", 2022-06-29T15:52:03.880 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:42:25.290884+0000", 2022-06-29T15:52:03.880 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:52:03.881 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:52:03.881 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:52:03.881 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:52:03.881 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:52:03.881 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:52:03.882 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:52:03.882 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:52:03.882 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2022-06-29T15:52:03.883 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2022-06-29T15:52:03.883 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2022-06-29T15:52:03.883 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2022-06-29T15:52:03.883 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2022-06-29T15:52:03.883 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:52:03.884 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:52:03.884 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:52:03.884 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:52:03.884 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:52:03.885 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:52:03.885 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:52:03.885 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:03.885 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "19", 2022-06-29T15:52:03.885 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:52:03.886 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:52:03.886 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:52:03.886 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:52:03.886 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:52:03.887 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:52:03.887 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:52:03.887 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:52:03.887 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:52:03.887 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:52:03.888 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:52:03.888 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:52:03.888 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:52:03.888 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:52:03.889 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:52:03.889 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:52:03.889 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:52:03.889 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:52:03.890 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:52:03.890 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:52:03.890 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:52:03.890 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:52:03.890 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:52:03.891 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:52:03.891 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:52:03.891 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:52:03.891 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:03.892 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:52:03.892 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:52:03.892 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:52:03.892 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:52:03.892 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:52:03.893 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:52:03.893 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:52:03.893 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:52:03.893 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:52:03.893 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:52:03.894 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:52:03.894 INFO:teuthology.orchestra.run.smithi047.stdout: "options": {}, 2022-06-29T15:52:03.894 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:52:03.894 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd": {} 2022-06-29T15:52:03.895 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:03.895 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:03.895 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:03.895 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 27, 2022-06-29T15:52:03.895 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "cephfs_metadata", 2022-06-29T15:52:03.896 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:51:50.583017+0000", 2022-06-29T15:52:03.896 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:52:03.896 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:52:03.896 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:52:03.897 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:52:03.897 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:52:03.897 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:52:03.897 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:52:03.898 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:52:03.898 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2022-06-29T15:52:03.898 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2022-06-29T15:52:03.898 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2022-06-29T15:52:03.899 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2022-06-29T15:52:03.899 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2022-06-29T15:52:03.899 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:52:03.899 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:52:03.900 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:52:03.900 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:52:03.900 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:52:03.900 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:52:03.901 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:52:03.901 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:03.901 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "150", 2022-06-29T15:52:03.901 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:52:03.902 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:52:03.902 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:52:03.902 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:52:03.902 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:52:03.903 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:52:03.903 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:52:03.903 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:52:03.903 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:52:03.903 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:52:03.904 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:52:03.904 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:52:03.904 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:52:03.904 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:52:03.904 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:52:03.905 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:52:03.905 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:52:03.905 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:52:03.905 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:52:03.906 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:52:03.906 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:52:03.906 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:52:03.906 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:52:03.907 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:52:03.907 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:52:03.907 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:52:03.907 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:03.907 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:52:03.908 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:52:03.908 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:52:03.908 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:52:03.908 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:52:03.909 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:52:03.909 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:52:03.909 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:52:03.909 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:52:03.910 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:52:03.910 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:52:03.910 INFO:teuthology.orchestra.run.smithi047.stdout: "options": { 2022-06-29T15:52:03.910 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_bias": 4, 2022-06-29T15:52:03.911 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_min": 16, 2022-06-29T15:52:03.911 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_priority": 5 2022-06-29T15:52:03.911 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:03.911 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:52:03.911 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs": { 2022-06-29T15:52:03.912 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": "cephfs" 2022-06-29T15:52:03.912 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:03.912 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:03.912 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:03.913 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:03.913 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 28, 2022-06-29T15:52:03.913 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "cephfs_data", 2022-06-29T15:52:03.913 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:51:51.834830+0000", 2022-06-29T15:52:03.913 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:52:03.914 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:52:03.914 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:52:03.914 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:52:03.914 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:52:03.915 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:52:03.915 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:52:03.915 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:52:03.915 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2022-06-29T15:52:03.916 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2022-06-29T15:52:03.916 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2022-06-29T15:52:03.916 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2022-06-29T15:52:03.916 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2022-06-29T15:52:03.917 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:52:03.917 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:52:03.917 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:52:03.917 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:52:03.917 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:52:03.918 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:52:03.918 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:52:03.918 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:03.918 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "151", 2022-06-29T15:52:03.919 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:52:03.919 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:52:03.919 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:52:03.919 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:52:03.920 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:52:03.920 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:52:03.920 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:52:03.920 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:52:03.920 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:52:03.921 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:52:03.921 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:52:03.921 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:52:03.921 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:52:03.922 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:52:03.922 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:52:03.922 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:52:03.922 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:52:03.923 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:52:03.923 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:52:03.923 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:52:03.923 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:52:03.923 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:52:03.924 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:52:03.924 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:52:03.924 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:52:03.924 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:52:03.925 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:03.925 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:52:03.925 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:52:03.925 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:52:03.925 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:52:03.926 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:52:03.926 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:52:03.926 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:52:03.926 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:52:03.927 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:52:03.927 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:52:03.927 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:52:03.927 INFO:teuthology.orchestra.run.smithi047.stdout: "options": {}, 2022-06-29T15:52:03.927 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:52:03.928 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs": { 2022-06-29T15:52:03.928 INFO:teuthology.orchestra.run.smithi047.stdout: "data": "cephfs" 2022-06-29T15:52:03.928 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:03.928 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:03.929 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:03.929 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:52:03.929 INFO:teuthology.orchestra.run.smithi047.stdout: "osds": [ 2022-06-29T15:52:03.929 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:03.929 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2022-06-29T15:52:03.930 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "e23eb023-4706-4201-bf72-4bf19c912c5d", 2022-06-29T15:52:03.930 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:52:03.930 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:52:03.930 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:52:03.931 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:52:03.931 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:52:03.931 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:52:03.931 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:52:03.932 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 149, 2022-06-29T15:52:03.932 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:52:03.932 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:52:03.932 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:52:03.932 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:03.933 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:03.933 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:03.933 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6818", 2022-06-29T15:52:03.933 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:52:03.933 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:03.934 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:03.934 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:03.934 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6819", 2022-06-29T15:52:03.934 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:52:03.935 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:03.935 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:03.935 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:03.935 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:52:03.935 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:03.936 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:03.936 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:03.936 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6820", 2022-06-29T15:52:03.936 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:52:03.937 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:03.937 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:03.937 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:03.937 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6821", 2022-06-29T15:52:03.937 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:52:03.938 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:03.938 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:03.938 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:03.938 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:52:03.939 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:03.939 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:03.939 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:03.939 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6824", 2022-06-29T15:52:03.939 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:52:03.940 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:03.940 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:03.940 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:03.940 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6825", 2022-06-29T15:52:03.941 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:52:03.941 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:03.941 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:03.941 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:03.941 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:52:03.942 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:03.942 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:03.942 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:03.942 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6822", 2022-06-29T15:52:03.943 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:52:03.943 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:03.943 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:03.944 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:03.944 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6823", 2022-06-29T15:52:03.944 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:52:03.944 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:03.945 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:03.945 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:03.945 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6819/13096", 2022-06-29T15:52:03.945 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6821/13096", 2022-06-29T15:52:03.945 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6825/13096", 2022-06-29T15:52:03.946 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6823/13096", 2022-06-29T15:52:03.946 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:52:03.946 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:52:03.946 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:52:03.947 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:03.947 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:03.947 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:03.947 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2022-06-29T15:52:03.948 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "228bb08d-2d64-43d9-9560-bfaeff154452", 2022-06-29T15:52:03.948 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:52:03.948 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:52:03.948 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:52:03.948 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:52:03.949 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:52:03.949 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:52:03.949 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:52:03.949 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 149, 2022-06-29T15:52:03.950 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:52:03.950 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:52:03.950 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:52:03.950 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:03.951 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:03.951 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:03.951 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6826", 2022-06-29T15:52:03.951 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:52:03.951 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:03.952 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:03.952 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:03.952 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6827", 2022-06-29T15:52:03.952 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:52:03.953 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:03.953 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:03.953 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:03.953 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:52:03.953 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:03.954 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:03.954 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:03.954 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6828", 2022-06-29T15:52:03.954 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:52:03.955 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:03.955 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:03.955 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:03.955 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6829", 2022-06-29T15:52:03.956 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:52:03.956 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:03.956 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:03.956 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:03.956 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:52:03.957 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:03.957 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:03.957 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:03.957 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6832", 2022-06-29T15:52:03.958 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:52:03.958 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:03.958 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:03.958 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:03.958 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6833", 2022-06-29T15:52:03.959 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:52:03.959 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:03.959 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:03.959 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:03.960 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:52:03.960 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:03.960 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:03.960 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:03.961 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6830", 2022-06-29T15:52:03.961 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:52:03.961 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:03.961 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:03.961 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:03.962 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6831", 2022-06-29T15:52:03.962 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:52:03.962 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:03.962 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:03.963 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:03.963 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6827/13100", 2022-06-29T15:52:03.963 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6829/13100", 2022-06-29T15:52:03.963 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6833/13100", 2022-06-29T15:52:03.963 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6831/13100", 2022-06-29T15:52:03.964 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:52:03.964 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:52:03.964 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:52:03.964 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:03.965 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:03.965 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:03.965 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2022-06-29T15:52:03.965 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "797a5452-1510-4640-a111-c9c2fcf707eb", 2022-06-29T15:52:03.965 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:52:03.966 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:52:03.966 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:52:03.966 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:52:03.966 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:52:03.967 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:52:03.967 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:52:03.967 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 148, 2022-06-29T15:52:03.967 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:52:03.968 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:52:03.968 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:52:03.968 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:03.968 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:03.968 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:03.969 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6810", 2022-06-29T15:52:03.969 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:52:03.969 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:03.969 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:03.970 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:03.970 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6811", 2022-06-29T15:52:03.970 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:52:03.970 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:03.971 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:03.971 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:03.971 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:52:03.971 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:03.971 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:03.972 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:03.972 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6812", 2022-06-29T15:52:03.972 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:52:03.972 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:03.973 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:03.973 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:03.973 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6813", 2022-06-29T15:52:03.973 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:52:03.973 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:03.974 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:03.974 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:03.974 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:52:03.974 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:03.975 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:03.975 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:03.975 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6816", 2022-06-29T15:52:03.975 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:52:03.975 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:03.976 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:03.976 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:03.976 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6817", 2022-06-29T15:52:03.976 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:52:03.977 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:03.977 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:03.977 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:03.977 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:52:03.978 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:03.978 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:03.978 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:03.978 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6814", 2022-06-29T15:52:03.978 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:52:03.979 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:03.979 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:03.979 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:03.979 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6815", 2022-06-29T15:52:03.980 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:52:03.980 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:03.980 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:03.980 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:03.980 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6811/13098", 2022-06-29T15:52:03.981 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6813/13098", 2022-06-29T15:52:03.981 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6817/13098", 2022-06-29T15:52:03.981 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6815/13098", 2022-06-29T15:52:03.981 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:52:03.981 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:52:03.982 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:52:03.982 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:03.982 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:03.982 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:03.983 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2022-06-29T15:52:03.983 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "55b24fd7-bb71-4a01-b3fa-a09caa772a62", 2022-06-29T15:52:03.983 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:52:03.983 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:52:03.984 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:52:03.984 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:52:03.984 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:52:03.984 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:52:03.984 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:52:03.985 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 137, 2022-06-29T15:52:03.985 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:52:03.985 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:52:03.985 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:52:03.985 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:03.986 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:03.986 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:03.986 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6802", 2022-06-29T15:52:03.986 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:52:03.987 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:03.987 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:03.987 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:03.987 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6803", 2022-06-29T15:52:03.988 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:52:03.988 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:03.988 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:03.988 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:03.988 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:52:03.989 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:03.989 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:03.989 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:03.989 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6804", 2022-06-29T15:52:03.990 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:52:03.990 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:03.990 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:03.990 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:03.990 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6805", 2022-06-29T15:52:03.991 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:52:03.991 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:03.991 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:03.991 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:03.992 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:52:03.992 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:03.992 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:03.992 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:03.993 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6808", 2022-06-29T15:52:03.993 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:52:03.993 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:03.993 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:03.993 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:03.994 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6809", 2022-06-29T15:52:03.994 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:52:03.994 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:03.994 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:03.995 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:03.995 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:52:03.995 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:03.995 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:03.996 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:03.996 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6806", 2022-06-29T15:52:03.996 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:52:03.996 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:03.996 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:03.997 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:03.997 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6807", 2022-06-29T15:52:03.997 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:52:03.997 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:03.998 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:03.998 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:03.998 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6803/13097", 2022-06-29T15:52:03.998 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6805/13097", 2022-06-29T15:52:03.998 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6809/13097", 2022-06-29T15:52:03.999 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6807/13097", 2022-06-29T15:52:03.999 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:52:03.999 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:52:03.999 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:52:04.000 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:04.000 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:04.000 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:04.000 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2022-06-29T15:52:04.001 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3", 2022-06-29T15:52:04.001 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:52:04.001 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:52:04.001 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:52:04.002 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:52:04.002 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:52:04.002 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:52:04.002 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:52:04.002 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 148, 2022-06-29T15:52:04.003 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:52:04.003 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:52:04.003 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:52:04.003 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:04.004 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:04.004 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:04.004 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6816", 2022-06-29T15:52:04.004 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:52:04.004 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:04.005 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:04.005 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:04.005 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6817", 2022-06-29T15:52:04.005 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:52:04.006 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:04.006 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:04.006 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:04.006 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:52:04.007 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:04.007 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:04.007 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:04.007 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6818", 2022-06-29T15:52:04.007 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:52:04.008 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:04.008 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:04.008 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:04.008 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6819", 2022-06-29T15:52:04.008 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:52:04.009 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:04.009 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:04.009 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:04.009 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:52:04.010 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:04.010 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:04.010 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:04.010 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6822", 2022-06-29T15:52:04.010 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:52:04.011 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:04.011 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:04.011 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:04.011 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6823", 2022-06-29T15:52:04.012 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:52:04.012 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:04.012 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:04.012 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:04.012 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:52:04.013 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:04.013 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:04.013 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:04.013 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6820", 2022-06-29T15:52:04.014 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:52:04.014 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:04.014 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:04.014 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:04.015 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6821", 2022-06-29T15:52:04.015 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:52:04.015 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:04.015 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:04.015 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:04.016 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6817/13396", 2022-06-29T15:52:04.016 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6819/13396", 2022-06-29T15:52:04.016 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6823/13396", 2022-06-29T15:52:04.016 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6821/13396", 2022-06-29T15:52:04.017 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:52:04.017 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:52:04.017 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:52:04.017 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:04.017 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:04.018 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:04.018 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2022-06-29T15:52:04.018 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "f77beebe-6cca-4244-b2d4-677825ebf919", 2022-06-29T15:52:04.018 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:52:04.019 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:52:04.019 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:52:04.019 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:52:04.019 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:52:04.019 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:52:04.020 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:52:04.020 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 149, 2022-06-29T15:52:04.020 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:52:04.020 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:52:04.021 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:52:04.021 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:04.021 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:04.021 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:04.022 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6800", 2022-06-29T15:52:04.022 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:52:04.022 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:04.022 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:04.022 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:04.023 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6801", 2022-06-29T15:52:04.023 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:52:04.023 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:04.023 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:04.024 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:04.024 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:52:04.024 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:04.024 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:04.024 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:04.025 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6802", 2022-06-29T15:52:04.025 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:52:04.025 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:04.025 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:04.025 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:04.026 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6803", 2022-06-29T15:52:04.026 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:52:04.026 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:04.026 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:04.027 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:04.027 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:52:04.027 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:04.027 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:04.027 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:04.028 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6806", 2022-06-29T15:52:04.028 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:52:04.028 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:04.028 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:04.029 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:04.029 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6807", 2022-06-29T15:52:04.029 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:52:04.029 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:04.029 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:04.030 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:04.030 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:52:04.030 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:04.030 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:04.031 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:04.031 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6804", 2022-06-29T15:52:04.031 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:52:04.031 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:04.031 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:04.032 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:04.032 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6805", 2022-06-29T15:52:04.032 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:52:04.032 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:04.033 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:04.033 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:04.033 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6801/13393", 2022-06-29T15:52:04.033 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6803/13393", 2022-06-29T15:52:04.033 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6807/13393", 2022-06-29T15:52:04.034 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6805/13393", 2022-06-29T15:52:04.034 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:52:04.034 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:52:04.034 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:52:04.035 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:04.035 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:04.035 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:04.035 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2022-06-29T15:52:04.036 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "af0ca9b3-c02c-4910-834b-bf0b8dc6713c", 2022-06-29T15:52:04.036 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:52:04.036 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:52:04.036 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:52:04.036 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:52:04.037 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:52:04.037 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:52:04.037 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:52:04.037 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 149, 2022-06-29T15:52:04.037 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:52:04.038 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:52:04.038 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:52:04.038 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:04.038 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:04.039 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:04.039 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6824", 2022-06-29T15:52:04.039 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:52:04.039 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:04.040 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:04.040 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:04.040 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6825", 2022-06-29T15:52:04.040 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:52:04.040 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:04.041 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:04.041 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:04.041 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:52:04.041 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:04.041 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:04.042 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:04.042 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6826", 2022-06-29T15:52:04.042 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:52:04.042 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:04.043 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:04.043 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:04.043 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6827", 2022-06-29T15:52:04.044 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:52:04.044 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:04.044 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:04.044 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:04.045 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:52:04.045 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:04.045 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:04.045 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:04.046 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6830", 2022-06-29T15:52:04.046 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:52:04.046 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:04.046 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:04.046 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:04.047 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6831", 2022-06-29T15:52:04.047 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:52:04.047 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:04.047 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:04.048 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:04.048 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:52:04.048 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:04.048 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:04.049 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:04.049 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6828", 2022-06-29T15:52:04.049 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:52:04.049 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:04.049 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:04.050 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:04.050 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6829", 2022-06-29T15:52:04.050 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:52:04.050 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:04.051 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:04.051 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:04.051 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6825/13395", 2022-06-29T15:52:04.051 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6827/13395", 2022-06-29T15:52:04.051 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6831/13395", 2022-06-29T15:52:04.052 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6829/13395", 2022-06-29T15:52:04.052 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:52:04.052 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:52:04.052 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:52:04.052 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:04.053 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:04.053 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:04.053 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2022-06-29T15:52:04.053 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "f19c3ca6-df88-4bde-a150-b51a53f092e1", 2022-06-29T15:52:04.053 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:52:04.054 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:52:04.054 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:52:04.054 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:52:04.054 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:52:04.055 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:52:04.055 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:52:04.055 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 149, 2022-06-29T15:52:04.055 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:52:04.055 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:52:04.056 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:52:04.056 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:04.056 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:04.056 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:04.056 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6808", 2022-06-29T15:52:04.057 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:52:04.057 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:04.057 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:04.057 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:04.058 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6809", 2022-06-29T15:52:04.058 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:52:04.058 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:04.058 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:04.058 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:04.059 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:52:04.059 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:04.059 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:04.059 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:04.059 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6810", 2022-06-29T15:52:04.060 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:52:04.060 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:04.060 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:04.060 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:04.061 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6811", 2022-06-29T15:52:04.061 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:52:04.061 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:04.061 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:04.062 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:04.062 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:52:04.062 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:04.062 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:04.062 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:04.063 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6814", 2022-06-29T15:52:04.063 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:52:04.063 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:04.063 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:04.064 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:04.064 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6815", 2022-06-29T15:52:04.064 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:52:04.064 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:04.064 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:04.065 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:04.065 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:52:04.065 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:04.065 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:04.066 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:04.066 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6812", 2022-06-29T15:52:04.066 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:52:04.066 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:04.066 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:04.067 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:04.067 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6813", 2022-06-29T15:52:04.067 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:52:04.067 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:04.068 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:04.068 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:04.068 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6809/13394", 2022-06-29T15:52:04.068 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6811/13394", 2022-06-29T15:52:04.069 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6815/13394", 2022-06-29T15:52:04.069 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6813/13394", 2022-06-29T15:52:04.069 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:52:04.069 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:52:04.069 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:52:04.070 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:04.070 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:04.070 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:52:04.070 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_xinfo": [ 2022-06-29T15:52:04.071 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:04.071 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2022-06-29T15:52:04.071 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:52:04.071 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:52:04.071 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:52:04.072 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:52:04.072 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:52:04.072 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.720513+0000", 2022-06-29T15:52:04.072 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:52:04.073 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:04.073 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:04.073 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2022-06-29T15:52:04.073 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:52:04.073 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:52:04.074 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:52:04.074 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:52:04.074 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:52:04.074 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.772612+0000", 2022-06-29T15:52:04.075 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:52:04.075 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:04.075 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:04.075 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2022-06-29T15:52:04.075 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:52:04.076 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:52:04.076 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:52:04.076 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:52:04.076 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:52:04.076 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.450223+0000", 2022-06-29T15:52:04.077 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:52:04.077 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:04.077 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:04.077 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2022-06-29T15:52:04.077 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:52:04.078 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:52:04.078 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:52:04.078 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:52:04.078 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:52:04.078 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.477856+0000", 2022-06-29T15:52:04.079 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:52:04.079 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:04.079 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:04.079 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2022-06-29T15:52:04.079 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:52:04.080 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:52:04.080 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:52:04.080 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:52:04.080 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:52:04.081 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.539485+0000", 2022-06-29T15:52:04.081 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:52:04.081 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:04.081 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:04.081 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2022-06-29T15:52:04.082 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:52:04.082 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:52:04.082 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:52:04.082 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:52:04.083 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:52:04.083 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.487040+0000", 2022-06-29T15:52:04.083 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:52:04.083 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:04.084 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:04.084 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2022-06-29T15:52:04.084 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:52:04.084 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:52:04.084 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:52:04.085 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:52:04.085 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:52:04.085 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.748685+0000", 2022-06-29T15:52:04.085 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:52:04.086 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:04.086 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:04.086 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2022-06-29T15:52:04.086 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:52:04.087 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:52:04.087 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:52:04.087 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:52:04.087 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:52:04.087 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.459516+0000", 2022-06-29T15:52:04.088 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:52:04.088 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:04.088 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:52:04.088 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap": [], 2022-06-29T15:52:04.089 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap_items": [], 2022-06-29T15:52:04.089 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_temp": [], 2022-06-29T15:52:04.089 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_temp": [], 2022-06-29T15:52:04.089 INFO:teuthology.orchestra.run.smithi047.stdout: "blacklist": {}, 2022-06-29T15:52:04.089 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profiles": { 2022-06-29T15:52:04.090 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs_data": { 2022-06-29T15:52:04.090 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-device-class": "", 2022-06-29T15:52:04.090 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2022-06-29T15:52:04.090 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-root": "default", 2022-06-29T15:52:04.090 INFO:teuthology.orchestra.run.smithi047.stdout: "jerasure-per-chunk-alignment": "false", 2022-06-29T15:52:04.091 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2022-06-29T15:52:04.091 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "2", 2022-06-29T15:52:04.091 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2022-06-29T15:52:04.091 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2022-06-29T15:52:04.092 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van", 2022-06-29T15:52:04.092 INFO:teuthology.orchestra.run.smithi047.stdout: "w": "8" 2022-06-29T15:52:04.092 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:04.092 INFO:teuthology.orchestra.run.smithi047.stdout: "default": { 2022-06-29T15:52:04.092 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2022-06-29T15:52:04.093 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2022-06-29T15:52:04.093 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "1", 2022-06-29T15:52:04.093 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2022-06-29T15:52:04.093 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2022-06-29T15:52:04.094 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van" 2022-06-29T15:52:04.094 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:04.094 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:04.094 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps_queue": [], 2022-06-29T15:52:04.094 INFO:teuthology.orchestra.run.smithi047.stdout: "new_removed_snaps": [], 2022-06-29T15:52:04.095 INFO:teuthology.orchestra.run.smithi047.stdout: "new_purged_snaps": [], 2022-06-29T15:52:04.095 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_node_flags": {}, 2022-06-29T15:52:04.095 INFO:teuthology.orchestra.run.smithi047.stdout: "device_class_flags": {} 2022-06-29T15:52:04.095 INFO:teuthology.orchestra.run.smithi047.stdout:} 2022-06-29T15:52:04.097 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.846+0000 7f6aa4c35700 1 -- 172.21.15.47:0/2648384604 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6a80053b00 msgr2=0x7f6a80055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:04.097 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.846+0000 7f6aa4c35700 1 --2- 172.21.15.47:0/2648384604 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6a80053b00 0x7f6a80055fc0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7f6a9400b000 tx=0x7f6a94009040).stop 2022-06-29T15:52:04.098 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.846+0000 7f6aa4c35700 1 -- 172.21.15.47:0/2648384604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6aa00b5c60 msgr2=0x7f6aa0157200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:04.098 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.846+0000 7f6aa4c35700 1 --2- 172.21.15.47:0/2648384604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6aa00b5c60 0x7f6aa0157200 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 rx=0x7f6a9000c000 tx=0x7f6a900091d0).stop 2022-06-29T15:52:04.098 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.846+0000 7f6aa4c35700 1 -- 172.21.15.47:0/2648384604 shutdown_connections 2022-06-29T15:52:04.098 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.846+0000 7f6aa4c35700 1 --2- 172.21.15.47:0/2648384604 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6a80053b00 0x7f6a80055fc0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:04.099 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.846+0000 7f6aa4c35700 1 --2- 172.21.15.47:0/2648384604 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6aa0145090 0x7f6aa014c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:04.099 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.846+0000 7f6aa4c35700 1 --2- 172.21.15.47:0/2648384604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6aa00b5c60 0x7f6aa0157200 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:04.099 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.846+0000 7f6aa4c35700 1 --2- 172.21.15.47:0/2648384604 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6aa0060eb0 0x7f6aa00b35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:04.099 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.846+0000 7f6aa4c35700 1 -- 172.21.15.47:0/2648384604 >> 172.21.15.47:0/2648384604 conn(0x7f6aa0062560 msgr2=0x7f6aa00b1020 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:04.100 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.846+0000 7f6aa4c35700 1 -- 172.21.15.47:0/2648384604 shutdown_connections 2022-06-29T15:52:04.100 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:03.846+0000 7f6aa4c35700 1 -- 172.21.15.47:0/2648384604 wait complete. 2022-06-29T15:52:04.101 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:52:04.215 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.210+0000 7f24d0733700 1 Processor -- start 2022-06-29T15:52:04.216 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.210+0000 7f24d0733700 1 -- start start 2022-06-29T15:52:04.218 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.210+0000 7f24d0733700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f24c80b0cf0 0x7f24c8146d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:04.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.210+0000 7f24d0733700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f24c80b06d0 0x7f24c813ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:04.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.210+0000 7f24d0733700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24c80b4520 con 0x7f24c80611b0 2022-06-29T15:52:04.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.210+0000 7f24d0733700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24c80b4b10 con 0x7f24c80b0cf0 2022-06-29T15:52:04.219 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.210+0000 7f24d0733700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24c80621d0 con 0x7f24c80b06d0 2022-06-29T15:52:04.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.210+0000 7f24cdcce700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f24c80b0cf0 0x7f24c8146d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:04.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.210+0000 7f24cdcce700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f24c80b0cf0 0x7f24c8146d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54688/0 (socket says 172.21.15.47:54688) 2022-06-29T15:52:04.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.210+0000 7f24cdcce700 1 -- 172.21.15.47:0/2947602051 learned_addr learned my addr 172.21.15.47:0/2947602051 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:04.220 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.210+0000 7f24cecd0700 1 --2- 172.21.15.47:0/2947602051 >> v2:172.21.15.64:3300/0 conn(0x7f24c80b06d0 0x7f24c813ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:04.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.210+0000 7f24cdcce700 1 -- 172.21.15.47:0/2947602051 >> v2:172.21.15.64:3300/0 conn(0x7f24c80b06d0 msgr2=0x7f24c813ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:04.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.210+0000 7f24cdcce700 1 --2- 172.21.15.47:0/2947602051 >> v2:172.21.15.64:3300/0 conn(0x7f24c80b06d0 0x7f24c813ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:04.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.210+0000 7f24cdcce700 1 -- 172.21.15.47:0/2947602051 >> v1:172.21.15.64:6789/0 conn(0x7f24c80611b0 legacy=0x7f24c8146640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:04.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.210+0000 7f24cdcce700 1 -- 172.21.15.47:0/2947602051 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c8061590 con 0x7f24c80b0cf0 2022-06-29T15:52:04.221 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.210+0000 7f24cecd0700 1 --2- 172.21.15.47:0/2947602051 >> v2:172.21.15.64:3300/0 conn(0x7f24c80b06d0 0x7f24c813ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:52:04.222 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.210+0000 7f24cdcce700 1 --2- 172.21.15.47:0/2947602051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24c80b0cf0 0x7f24c8146d50 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 rx=0x7f24c80b1c00 tx=0x7f24b8006eb0).ready entity=mon.0 client_cookie=bc464dea66345399 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:04.222 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.210+0000 7f24cd4cd700 1 -- 172.21.15.47:0/2947602051 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24b800d5c0 con 0x7f24c80b0cf0 2022-06-29T15:52:04.222 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.210+0000 7f24cd4cd700 1 -- 172.21.15.47:0/2947602051 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b8000e90 con 0x7f24c80b0cf0 2022-06-29T15:52:04.222 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.210+0000 7f24cd4cd700 1 -- 172.21.15.47:0/2947602051 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24b800d950 con 0x7f24c80b0cf0 2022-06-29T15:52:04.223 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.210+0000 7f24d0733700 1 -- 172.21.15.47:0/2947602051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24c80b0cf0 msgr2=0x7f24c8146d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:04.223 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.210+0000 7f24d0733700 1 --2- 172.21.15.47:0/2947602051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24c80b0cf0 0x7f24c8146d50 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 rx=0x7f24c80b1c00 tx=0x7f24b8006eb0).stop 2022-06-29T15:52:04.223 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.210+0000 7f24d0733700 1 -- 172.21.15.47:0/2947602051 shutdown_connections 2022-06-29T15:52:04.223 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.210+0000 7f24d0733700 1 --2- 172.21.15.47:0/2947602051 >> v2:172.21.15.64:3300/0 conn(0x7f24c80b06d0 0x7f24c813ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:04.223 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.210+0000 7f24d0733700 1 --2- 172.21.15.47:0/2947602051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24c80b0cf0 0x7f24c8146d50 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:04.224 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.210+0000 7f24d0733700 1 -- 172.21.15.47:0/2947602051 >> 172.21.15.47:0/2947602051 conn(0x7f24c80b98b0 msgr2=0x7f24c80b9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:04.224 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.210+0000 7f24d0733700 1 -- 172.21.15.47:0/2947602051 shutdown_connections 2022-06-29T15:52:04.224 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.210+0000 7f24d0733700 1 -- 172.21.15.47:0/2947602051 wait complete. 2022-06-29T15:52:04.227 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.214+0000 7f24d0733700 1 Processor -- start 2022-06-29T15:52:04.227 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.214+0000 7f24d0733700 1 -- start start 2022-06-29T15:52:04.227 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.214+0000 7f24d0733700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f24c80611b0 0x7f24c8153250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:04.228 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.214+0000 7f24d0733700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24c80b06d0 0x7f24c8144730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:04.228 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.214+0000 7f24d0733700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24c80b0cf0 0x7f24c814fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:04.228 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.214+0000 7f24d0733700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c8114320 con 0x7f24c80b06d0 2022-06-29T15:52:04.228 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.214+0000 7f24d0733700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c8153740 con 0x7f24c80b0cf0 2022-06-29T15:52:04.229 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.214+0000 7f24d0733700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24c8113b40 con 0x7f24c80611b0 2022-06-29T15:52:04.229 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.214+0000 7f24cdcce700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24c80b06d0 0x7f24c8144730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:04.229 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.214+0000 7f24cdcce700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24c80b06d0 0x7f24c8144730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54694/0 (socket says 172.21.15.47:54694) 2022-06-29T15:52:04.229 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.214+0000 7f24cdcce700 1 -- 172.21.15.47:0/342219802 learned_addr learned my addr 172.21.15.47:0/342219802 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:04.230 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.214+0000 7f24cecd0700 1 --2- 172.21.15.47:0/342219802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24c80b0cf0 0x7f24c814fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:04.230 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.214+0000 7f24ce4cf700 1 --2- 172.21.15.47:0/342219802 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f24c80611b0 0x7f24c8153250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:04.230 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.214+0000 7f24cdcce700 1 -- 172.21.15.47:0/342219802 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f24c80611b0 msgr2=0x7f24c8153250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:04.230 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.214+0000 7f24cdcce700 1 --2- 172.21.15.47:0/342219802 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f24c80611b0 0x7f24c8153250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:04.231 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.214+0000 7f24cdcce700 1 -- 172.21.15.47:0/342219802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24c80b0cf0 msgr2=0x7f24c814fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:04.231 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.214+0000 7f24cdcce700 1 --2- 172.21.15.47:0/342219802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24c80b0cf0 0x7f24c814fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:04.231 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.214+0000 7f24cdcce700 1 -- 172.21.15.47:0/342219802 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b800b1e0 con 0x7f24c80b06d0 2022-06-29T15:52:04.231 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.214+0000 7f24cecd0700 1 --2- 172.21.15.47:0/342219802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24c80b0cf0 0x7f24c814fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:52:04.231 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.214+0000 7f24cdcce700 1 --2- 172.21.15.47:0/342219802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24c80b06d0 0x7f24c8144730 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 rx=0x7f24c80b1c00 tx=0x7f24b8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:04.232 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.214+0000 7f24b77fe700 1 -- 172.21.15.47:0/342219802 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24b8021070 con 0x7f24c80b06d0 2022-06-29T15:52:04.232 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.214+0000 7f24b77fe700 1 -- 172.21.15.47:0/342219802 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b8011710 con 0x7f24c80b06d0 2022-06-29T15:52:04.232 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.214+0000 7f24d0733700 1 -- 172.21.15.47:0/342219802 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24c8113de0 con 0x7f24c80b06d0 2022-06-29T15:52:04.232 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.214+0000 7f24b77fe700 1 -- 172.21.15.47:0/342219802 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24b8011a70 con 0x7f24c80b06d0 2022-06-29T15:52:04.233 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.214+0000 7f24d0733700 1 -- 172.21.15.47:0/342219802 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24c8144c20 con 0x7f24c80b06d0 2022-06-29T15:52:04.233 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.218+0000 7f24d0733700 1 -- 172.21.15.47:0/342219802 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24c80b5e00 con 0x7f24c80b06d0 2022-06-29T15:52:04.233 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.218+0000 7f24b77fe700 1 -- 172.21.15.47:0/342219802 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f24b800f070 con 0x7f24c80b06d0 2022-06-29T15:52:04.233 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.218+0000 7f24b77fe700 1 --2- 172.21.15.47:0/342219802 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f24a8053b30 0x7f24a8055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:04.233 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.218+0000 7f24ce4cf700 1 --2- 172.21.15.47:0/342219802 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f24a8053b30 0x7f24a8055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:04.234 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.218+0000 7f24b77fe700 1 -- 172.21.15.47:0/342219802 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f24b806e190 con 0x7f24c80b06d0 2022-06-29T15:52:04.234 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.218+0000 7f24ce4cf700 1 --2- 172.21.15.47:0/342219802 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f24a8053b30 0x7f24a8055ff0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7f24c400b000 tx=0x7f24c4009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:04.234 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.226+0000 7f24b77fe700 1 -- 172.21.15.47:0/342219802 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f24b8044e40 con 0x7f24c80b06d0 2022-06-29T15:52:04.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.390+0000 7f24d0733700 1 -- 172.21.15.47:0/342219802 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f24c8145960 con 0x7f24c80b06d0 2022-06-29T15:52:04.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.394+0000 7f24b77fe700 1 -- 172.21.15.47:0/342219802 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+2495 (secure 0 0 0) 0x7f24b8048460 con 0x7f24c80b06d0 2022-06-29T15:52:04.400 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:52:04.400 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":74,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8296,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/3321939421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3321939421},{"type":"v1","addr":"172.21.15.64:6833","nonce":3321939421}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":72},{"gid":8415,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1886881850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1886881850},{"type":"v1","addr":"172.21.15.47:6835","nonce":1886881850}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":72}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:51:52.828119+0000","modified":"2022-06-29T15:51:53.831305+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8421":{"gid":8421,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6837/2466961065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2466961065},{"type":"v1","addr":"172.21.15.47:6837","nonce":2466961065}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2022-06-29T15:52:04.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.394+0000 7f24d0733700 1 -- 172.21.15.47:0/342219802 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f24a8053b30 msgr2=0x7f24a8055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:04.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.394+0000 7f24d0733700 1 --2- 172.21.15.47:0/342219802 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f24a8053b30 0x7f24a8055ff0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7f24c400b000 tx=0x7f24c4009040).stop 2022-06-29T15:52:04.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.394+0000 7f24d0733700 1 -- 172.21.15.47:0/342219802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24c80b06d0 msgr2=0x7f24c8144730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:04.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.394+0000 7f24d0733700 1 --2- 172.21.15.47:0/342219802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24c80b06d0 0x7f24c8144730 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 rx=0x7f24c80b1c00 tx=0x7f24b8015000).stop 2022-06-29T15:52:04.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.394+0000 7f24d0733700 1 -- 172.21.15.47:0/342219802 shutdown_connections 2022-06-29T15:52:04.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.394+0000 7f24d0733700 1 --2- 172.21.15.47:0/342219802 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f24a8053b30 0x7f24a8055ff0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:04.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.394+0000 7f24d0733700 1 --2- 172.21.15.47:0/342219802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24c80b0cf0 0x7f24c814fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:04.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.394+0000 7f24d0733700 1 --2- 172.21.15.47:0/342219802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24c80b06d0 0x7f24c8144730 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:04.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.394+0000 7f24d0733700 1 --2- 172.21.15.47:0/342219802 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f24c80611b0 0x7f24c8153250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:04.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.394+0000 7f24d0733700 1 -- 172.21.15.47:0/342219802 >> 172.21.15.47:0/342219802 conn(0x7f24c80b98b0 msgr2=0x7f24c813e650 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:04.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.394+0000 7f24d0733700 1 -- 172.21.15.47:0/342219802 shutdown_connections 2022-06-29T15:52:04.405 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.394+0000 7f24d0733700 1 -- 172.21.15.47:0/342219802 wait complete. 2022-06-29T15:52:04.406 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 74 2022-06-29T15:52:04.414 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-06-29T15:52:04.514 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.506+0000 7fd825d8a700 1 Processor -- start 2022-06-29T15:52:04.514 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.506+0000 7fd825d8a700 1 -- start start 2022-06-29T15:52:04.516 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.506+0000 7fd825d8a700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd8200b2120 0x7fd8200b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:04.516 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.506+0000 7fd825d8a700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd82005f0b0 0x7fd82014b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:04.516 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.506+0000 7fd825d8a700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8200b0740 con 0x7fd8200b1260 2022-06-29T15:52:04.517 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.506+0000 7fd825d8a700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd820147460 con 0x7fd8200b2120 2022-06-29T15:52:04.517 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.506+0000 7fd825d8a700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8200b0970 con 0x7fd82005f0b0 2022-06-29T15:52:04.517 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.506+0000 7fd81f7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd8200b2120 0x7fd8200b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:04.517 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.506+0000 7fd81ffff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd8200b1260 0x7fd820147af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:47822/0 (socket says 172.21.15.47:47822) 2022-06-29T15:52:04.518 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.506+0000 7fd81ffff700 1 -- 172.21.15.47:0/438950422 learned_addr learned my addr 172.21.15.47:0/438950422 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:52:04.518 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.506+0000 7fd81effd700 1 --2- 172.21.15.47:0/438950422 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd82005f0b0 0x7fd82014b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:04.518 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.510+0000 7fd81e7fc700 1 -- 172.21.15.47:0/438950422 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1047402023 0 0) 0x7fd8200b0740 con 0x7fd8200b1260 2022-06-29T15:52:04.518 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.510+0000 7fd81e7fc700 1 -- 172.21.15.47:0/438950422 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd80c003260 con 0x7fd8200b1260 2022-06-29T15:52:04.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.510+0000 7fd81f7fe700 1 -- 172.21.15.47:0/438950422 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd82005f0b0 msgr2=0x7fd82014b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:04.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.510+0000 7fd81f7fe700 1 --2- 172.21.15.47:0/438950422 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd82005f0b0 0x7fd82014b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:04.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.510+0000 7fd81f7fe700 1 -- 172.21.15.47:0/438950422 >> v1:172.21.15.64:6789/0 conn(0x7fd8200b1260 legacy=0x7fd820147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:04.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.510+0000 7fd81f7fe700 1 -- 172.21.15.47:0/438950422 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8200b0c10 con 0x7fd8200b2120 2022-06-29T15:52:04.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.510+0000 7fd81f7fe700 1 --2- 172.21.15.47:0/438950422 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd8200b2120 0x7fd8200b2500 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7fd8180032f0 tx=0x7fd8180091d0).ready entity=mon.1 client_cookie=f5b7cb55143113c0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:04.523 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.510+0000 7fd81e7fc700 1 -- 172.21.15.47:0/438950422 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd81800b600 con 0x7fd8200b2120 2022-06-29T15:52:04.523 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.510+0000 7fd81e7fc700 1 -- 172.21.15.47:0/438950422 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd81800b760 con 0x7fd8200b2120 2022-06-29T15:52:04.524 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.510+0000 7fd81e7fc700 1 -- 172.21.15.47:0/438950422 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd81800b8d0 con 0x7fd8200b2120 2022-06-29T15:52:04.524 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.510+0000 7fd825d8a700 1 -- 172.21.15.47:0/438950422 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd8200b2120 msgr2=0x7fd8200b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:04.524 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.510+0000 7fd825d8a700 1 --2- 172.21.15.47:0/438950422 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd8200b2120 0x7fd8200b2500 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7fd8180032f0 tx=0x7fd8180091d0).stop 2022-06-29T15:52:04.524 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.510+0000 7fd825d8a700 1 -- 172.21.15.47:0/438950422 shutdown_connections 2022-06-29T15:52:04.525 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.510+0000 7fd825d8a700 1 --2- 172.21.15.47:0/438950422 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd82005f0b0 0x7fd82014b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:04.525 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.510+0000 7fd825d8a700 1 --2- 172.21.15.47:0/438950422 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd8200b2120 0x7fd8200b2500 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:04.525 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.510+0000 7fd825d8a700 1 -- 172.21.15.47:0/438950422 >> 172.21.15.47:0/438950422 conn(0x7fd8200626f0 msgr2=0x7fd820062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:04.525 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.510+0000 7fd825d8a700 1 -- 172.21.15.47:0/438950422 shutdown_connections 2022-06-29T15:52:04.526 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.510+0000 7fd825d8a700 1 -- 172.21.15.47:0/438950422 wait complete. 2022-06-29T15:52:04.526 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.510+0000 7fd825d8a700 1 Processor -- start 2022-06-29T15:52:04.526 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.510+0000 7fd825d8a700 1 -- start start 2022-06-29T15:52:04.526 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.514+0000 7fd825d8a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd8200b1260 0x7fd820146550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:04.527 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.514+0000 7fd825d8a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8200b2120 0x7fd82013e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:04.527 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.514+0000 7fd825d8a700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd8200af410 0x7fd82013ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:04.527 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.514+0000 7fd825d8a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8200ad7e0 con 0x7fd8200b2120 2022-06-29T15:52:04.527 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.514+0000 7fd825d8a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8200ad130 con 0x7fd8200b1260 2022-06-29T15:52:04.528 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.514+0000 7fd825d8a700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8200ada10 con 0x7fd8200af410 2022-06-29T15:52:04.528 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.514+0000 7fd81effd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8200b2120 0x7fd82013e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:04.528 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.514+0000 7fd81effd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8200b2120 0x7fd82013e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54708/0 (socket says 172.21.15.47:54708) 2022-06-29T15:52:04.528 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.514+0000 7fd81effd700 1 -- 172.21.15.47:0/3241248332 learned_addr learned my addr 172.21.15.47:0/3241248332 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:04.528 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.514+0000 7fd81ffff700 1 --2- 172.21.15.47:0/3241248332 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd8200af410 0x7fd82013ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:04.529 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.514+0000 7fd81f7fe700 1 --2- 172.21.15.47:0/3241248332 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd8200b1260 0x7fd820146550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:04.529 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.514+0000 7fd81effd700 1 -- 172.21.15.47:0/3241248332 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd8200af410 msgr2=0x7fd82013ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:04.529 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.514+0000 7fd81effd700 1 --2- 172.21.15.47:0/3241248332 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd8200af410 0x7fd82013ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:04.529 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.514+0000 7fd81effd700 1 -- 172.21.15.47:0/3241248332 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd8200b1260 msgr2=0x7fd820146550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:04.530 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.514+0000 7fd81effd700 1 --2- 172.21.15.47:0/3241248332 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd8200b1260 0x7fd820146550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:04.530 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.514+0000 7fd81effd700 1 -- 172.21.15.47:0/3241248332 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd818009470 con 0x7fd8200b2120 2022-06-29T15:52:04.530 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.514+0000 7fd81effd700 1 --2- 172.21.15.47:0/3241248332 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8200b2120 0x7fd82013e810 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 rx=0x7fd81400c000 tx=0x7fd814006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:04.530 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.514+0000 7fd81cff9700 1 -- 172.21.15.47:0/3241248332 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd814010780 con 0x7fd8200b2120 2022-06-29T15:52:04.531 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.514+0000 7fd81cff9700 1 -- 172.21.15.47:0/3241248332 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8140108e0 con 0x7fd8200b2120 2022-06-29T15:52:04.531 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.514+0000 7fd81cff9700 1 -- 172.21.15.47:0/3241248332 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd814010c40 con 0x7fd8200b2120 2022-06-29T15:52:04.531 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.514+0000 7fd825d8a700 1 -- 172.21.15.47:0/3241248332 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8200adcb0 con 0x7fd8200b2120 2022-06-29T15:52:04.531 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.514+0000 7fd825d8a700 1 -- 172.21.15.47:0/3241248332 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8200ae150 con 0x7fd8200b2120 2022-06-29T15:52:04.532 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.518+0000 7fd81cff9700 1 -- 172.21.15.47:0/3241248332 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd81401c460 con 0x7fd8200b2120 2022-06-29T15:52:04.532 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.518+0000 7fd825d8a700 1 -- 172.21.15.47:0/3241248332 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8200ade40 con 0x7fd8200b2120 2022-06-29T15:52:04.532 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.518+0000 7fd81cff9700 1 --2- 172.21.15.47:0/3241248332 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd800053950 0x7fd800055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:04.532 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.518+0000 7fd81cff9700 1 -- 172.21.15.47:0/3241248332 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd81406c400 con 0x7fd8200b2120 2022-06-29T15:52:04.532 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.518+0000 7fd81f7fe700 1 --2- 172.21.15.47:0/3241248332 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd800053950 0x7fd800055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:04.533 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.518+0000 7fd81f7fe700 1 --2- 172.21.15.47:0/3241248332 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd800053950 0x7fd800055e10 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7fd81801b000 tx=0x7fd818016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:04.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.526+0000 7fd81cff9700 1 -- 172.21.15.47:0/3241248332 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd814043160 con 0x7fd8200b2120 2022-06-29T15:52:04.695 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:04.690+0000 7fd825d8a700 1 -- 172.21.15.47:0/3241248332 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd8200573d0 con 0x7fd8200b2120 2022-06-29T15:52:05.318 INFO:tasks.ceph.mon.a.smithi047.stderr:2022-06-29T15:52:05.314+0000 7ff9cf83c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-06-29T15:52:05.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.318+0000 7fd81cff9700 1 -- 172.21.15.47:0/3241248332 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v75) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd814046780 con 0x7fd8200b2120 2022-06-29T15:52:05.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.322+0000 7fd825d8a700 1 -- 172.21.15.47:0/3241248332 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd800053950 msgr2=0x7fd800055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:05.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.322+0000 7fd825d8a700 1 --2- 172.21.15.47:0/3241248332 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd800053950 0x7fd800055e10 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7fd81801b000 tx=0x7fd818016000).stop 2022-06-29T15:52:05.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.322+0000 7fd825d8a700 1 -- 172.21.15.47:0/3241248332 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8200b2120 msgr2=0x7fd82013e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:05.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.322+0000 7fd825d8a700 1 --2- 172.21.15.47:0/3241248332 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8200b2120 0x7fd82013e810 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 rx=0x7fd81400c000 tx=0x7fd814006df0).stop 2022-06-29T15:52:05.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.322+0000 7fd825d8a700 1 -- 172.21.15.47:0/3241248332 shutdown_connections 2022-06-29T15:52:05.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.322+0000 7fd825d8a700 1 --2- 172.21.15.47:0/3241248332 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd8200af410 0x7fd82013ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:05.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.322+0000 7fd825d8a700 1 --2- 172.21.15.47:0/3241248332 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8200b2120 0x7fd82013e810 secure :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 rx=0x7fd81400c000 tx=0x7fd814006df0).stop 2022-06-29T15:52:05.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.322+0000 7fd825d8a700 1 --2- 172.21.15.47:0/3241248332 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd800053950 0x7fd800055e10 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:05.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.322+0000 7fd825d8a700 1 --2- 172.21.15.47:0/3241248332 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd8200b1260 0x7fd820146550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:05.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.322+0000 7fd825d8a700 1 -- 172.21.15.47:0/3241248332 >> 172.21.15.47:0/3241248332 conn(0x7fd8200626f0 msgr2=0x7fd8200b5450 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:05.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.326+0000 7fd825d8a700 1 -- 172.21.15.47:0/3241248332 shutdown_connections 2022-06-29T15:52:05.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.326+0000 7fd825d8a700 1 -- 172.21.15.47:0/3241248332 wait complete. 2022-06-29T15:52:05.334 INFO:teuthology.orchestra.run.smithi047.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-06-29T15:52:05.349 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:52:05.353 INFO:tasks.ceph.mds.c.smithi047.stdout:starting mds.c at 2022-06-29T15:52:05.450 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.442+0000 7fea52999700 1 Processor -- start 2022-06-29T15:52:05.451 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.442+0000 7fea52999700 1 -- start start 2022-06-29T15:52:05.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.446+0000 7fea52999700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fea4c1430f0 0x7fea4c1434d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:05.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.446+0000 7fea52999700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fea4c0a93c0 0x7fea4c095240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:05.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.446+0000 7fea52999700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea4c060e50 con 0x7fea4c095730 2022-06-29T15:52:05.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.446+0000 7fea52999700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea4c13d590 con 0x7fea4c0a93c0 2022-06-29T15:52:05.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.446+0000 7fea52999700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c0610f0 con 0x7fea4c1430f0 2022-06-29T15:52:05.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.446+0000 7fea4b7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fea4c0a93c0 0x7fea4c095240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:05.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.446+0000 7fea4b7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fea4c0a93c0 0x7fea4c095240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54726/0 (socket says 172.21.15.47:54726) 2022-06-29T15:52:05.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.446+0000 7fea4b7fe700 1 -- 172.21.15.47:0/628689507 learned_addr learned my addr 172.21.15.47:0/628689507 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:05.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.446+0000 7fea4bfff700 1 --2- 172.21.15.47:0/628689507 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fea4c1430f0 0x7fea4c1434d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:05.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.446+0000 7fea4b7fe700 1 -- 172.21.15.47:0/628689507 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fea4c1430f0 msgr2=0x7fea4c1434d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:05.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.446+0000 7fea4b7fe700 1 --2- 172.21.15.47:0/628689507 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fea4c1430f0 0x7fea4c1434d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:05.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.446+0000 7fea4b7fe700 1 -- 172.21.15.47:0/628689507 >> v1:172.21.15.64:6789/0 conn(0x7fea4c095730 legacy=0x7fea4c097690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:05.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.446+0000 7fea4b7fe700 1 -- 172.21.15.47:0/628689507 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea4c0971f0 con 0x7fea4c0a93c0 2022-06-29T15:52:05.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.446+0000 7fea4b7fe700 1 --2- 172.21.15.47:0/628689507 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea4c0a93c0 0x7fea4c095240 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 rx=0x7fea3c002f70 tx=0x7fea3c006eb0).ready entity=mon.0 client_cookie=1e338145960a90de server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:05.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.446+0000 7fea4affd700 1 -- 172.21.15.47:0/628689507 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea3c00d5c0 con 0x7fea4c0a93c0 2022-06-29T15:52:05.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.446+0000 7fea4affd700 1 -- 172.21.15.47:0/628689507 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea3c000e90 con 0x7fea4c0a93c0 2022-06-29T15:52:05.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.446+0000 7fea4affd700 1 -- 172.21.15.47:0/628689507 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea3c00d950 con 0x7fea4c0a93c0 2022-06-29T15:52:05.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.446+0000 7fea52999700 1 -- 172.21.15.47:0/628689507 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea4c0a93c0 msgr2=0x7fea4c095240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:05.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.446+0000 7fea52999700 1 --2- 172.21.15.47:0/628689507 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea4c0a93c0 0x7fea4c095240 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 rx=0x7fea3c002f70 tx=0x7fea3c006eb0).stop 2022-06-29T15:52:05.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.446+0000 7fea52999700 1 -- 172.21.15.47:0/628689507 shutdown_connections 2022-06-29T15:52:05.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.446+0000 7fea52999700 1 --2- 172.21.15.47:0/628689507 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea4c0a93c0 0x7fea4c095240 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:05.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.446+0000 7fea52999700 1 --2- 172.21.15.47:0/628689507 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fea4c1430f0 0x7fea4c1434d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:05.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.446+0000 7fea52999700 1 -- 172.21.15.47:0/628689507 >> 172.21.15.47:0/628689507 conn(0x7fea4c05f0b0 msgr2=0x7fea4c05f490 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:05.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.446+0000 7fea52999700 1 -- 172.21.15.47:0/628689507 shutdown_connections 2022-06-29T15:52:05.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.446+0000 7fea52999700 1 -- 172.21.15.47:0/628689507 wait complete. 2022-06-29T15:52:05.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.450+0000 7fea52999700 1 Processor -- start 2022-06-29T15:52:05.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.450+0000 7fea52999700 1 -- start start 2022-06-29T15:52:05.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.450+0000 7fea52999700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea4c095730 0x7fea4c153c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:05.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.450+0000 7fea52999700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fea4c0a93c0 0x7fea4c064220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:05.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.450+0000 7fea52999700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea4c1430f0 0x7fea4c155c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:05.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.450+0000 7fea52999700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c156120 con 0x7fea4c095730 2022-06-29T15:52:05.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.450+0000 7fea52999700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c099100 con 0x7fea4c1430f0 2022-06-29T15:52:05.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.450+0000 7fea52999700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c0993a0 con 0x7fea4c0a93c0 2022-06-29T15:52:05.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.450+0000 7fea4bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea4c095730 0x7fea4c153c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:05.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.450+0000 7fea4bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea4c095730 0x7fea4c153c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54732/0 (socket says 172.21.15.47:54732) 2022-06-29T15:52:05.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.450+0000 7fea4bfff700 1 -- 172.21.15.47:0/1651144665 learned_addr learned my addr 172.21.15.47:0/1651144665 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:05.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.450+0000 7fea4b7fe700 1 --2- 172.21.15.47:0/1651144665 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fea4c0a93c0 0x7fea4c064220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:05.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.450+0000 7fea50f36700 1 --2- 172.21.15.47:0/1651144665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea4c1430f0 0x7fea4c155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:05.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.454+0000 7fea4bfff700 1 -- 172.21.15.47:0/1651144665 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fea4c0a93c0 msgr2=0x7fea4c064220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:05.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.454+0000 7fea4bfff700 1 --2- 172.21.15.47:0/1651144665 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fea4c0a93c0 0x7fea4c064220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:05.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.454+0000 7fea4bfff700 1 -- 172.21.15.47:0/1651144665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea4c1430f0 msgr2=0x7fea4c155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:05.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.454+0000 7fea4bfff700 1 --2- 172.21.15.47:0/1651144665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea4c1430f0 0x7fea4c155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:05.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.454+0000 7fea4bfff700 1 -- 172.21.15.47:0/1651144665 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea3c00b1e0 con 0x7fea4c095730 2022-06-29T15:52:05.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.454+0000 7fea4bfff700 1 --2- 172.21.15.47:0/1651144665 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea4c095730 0x7fea4c153c50 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 rx=0x7fea400030b0 tx=0x7fea40008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:05.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.454+0000 7fea497fa700 1 -- 172.21.15.47:0/1651144665 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea40011780 con 0x7fea4c095730 2022-06-29T15:52:05.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.454+0000 7fea497fa700 1 -- 172.21.15.47:0/1651144665 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea400118e0 con 0x7fea4c095730 2022-06-29T15:52:05.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.454+0000 7fea52999700 1 -- 172.21.15.47:0/1651144665 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea4c099640 con 0x7fea4c095730 2022-06-29T15:52:05.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.454+0000 7fea497fa700 1 -- 172.21.15.47:0/1651144665 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea40011c40 con 0x7fea4c095730 2022-06-29T15:52:05.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.454+0000 7fea52999700 1 -- 172.21.15.47:0/1651144665 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea4c098d80 con 0x7fea4c095730 2022-06-29T15:52:05.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.454+0000 7fea52999700 1 -- 172.21.15.47:0/1651144665 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea4c0573d0 con 0x7fea4c095730 2022-06-29T15:52:05.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.454+0000 7fea497fa700 1 -- 172.21.15.47:0/1651144665 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fea40012420 con 0x7fea4c095730 2022-06-29T15:52:05.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.458+0000 7fea497fa700 1 --2- 172.21.15.47:0/1651144665 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fea2c053980 0x7fea2c055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:05.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.458+0000 7fea497fa700 1 -- 172.21.15.47:0/1651144665 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fea4006bdb0 con 0x7fea4c095730 2022-06-29T15:52:05.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.458+0000 7fea4b7fe700 1 --2- 172.21.15.47:0/1651144665 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fea2c053980 0x7fea2c055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:05.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.458+0000 7fea4b7fe700 1 --2- 172.21.15.47:0/1651144665 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fea2c053980 0x7fea2c055e40 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7fea3c01c000 tx=0x7fea3c015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:05.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.462+0000 7fea497fa700 1 -- 172.21.15.47:0/1651144665 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fea40042ac0 con 0x7fea4c095730 2022-06-29T15:52:05.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.626+0000 7fea52999700 1 -- 172.21.15.47:0/1651144665 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea4c067cb0 con 0x7fea4c095730 2022-06-29T15:52:05.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.630+0000 7fea497fa700 1 -- 172.21.15.47:0/1651144665 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+2135 (secure 0 0 0) 0x7fea400460e0 con 0x7fea4c095730 2022-06-29T15:52:05.638 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:52:05.639 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":75,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8296,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/3321939421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3321939421},{"type":"v1","addr":"172.21.15.64:6833","nonce":3321939421}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":72},{"gid":8415,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1886881850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1886881850},{"type":"v1","addr":"172.21.15.47:6835","nonce":1886881850}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":72}],"filesystems":[{"mdsmap":{"epoch":75,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:51:52.828119+0000","modified":"2022-06-29T15:52:05.315026+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":152,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2022-06-29T15:52:05.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.630+0000 7fea52999700 1 -- 172.21.15.47:0/1651144665 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fea2c053980 msgr2=0x7fea2c055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:05.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.630+0000 7fea52999700 1 --2- 172.21.15.47:0/1651144665 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fea2c053980 0x7fea2c055e40 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7fea3c01c000 tx=0x7fea3c015000).stop 2022-06-29T15:52:05.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.630+0000 7fea52999700 1 -- 172.21.15.47:0/1651144665 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea4c095730 msgr2=0x7fea4c153c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:05.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.630+0000 7fea52999700 1 --2- 172.21.15.47:0/1651144665 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea4c095730 0x7fea4c153c50 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 rx=0x7fea400030b0 tx=0x7fea40008040).stop 2022-06-29T15:52:05.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.630+0000 7fea52999700 1 -- 172.21.15.47:0/1651144665 shutdown_connections 2022-06-29T15:52:05.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.630+0000 7fea52999700 1 --2- 172.21.15.47:0/1651144665 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fea2c053980 0x7fea2c055e40 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:05.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.630+0000 7fea52999700 1 --2- 172.21.15.47:0/1651144665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea4c1430f0 0x7fea4c155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:05.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.630+0000 7fea52999700 1 --2- 172.21.15.47:0/1651144665 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fea4c0a93c0 0x7fea4c064220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:05.645 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.630+0000 7fea52999700 1 --2- 172.21.15.47:0/1651144665 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea4c095730 0x7fea4c153c50 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:05.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.630+0000 7fea52999700 1 -- 172.21.15.47:0/1651144665 >> 172.21.15.47:0/1651144665 conn(0x7fea4c05f0b0 msgr2=0x7fea4c0b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:05.646 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.630+0000 7fea52999700 1 -- 172.21.15.47:0/1651144665 shutdown_connections 2022-06-29T15:52:05.647 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.630+0000 7fea52999700 1 -- 172.21.15.47:0/1651144665 wait complete. 2022-06-29T15:52:05.647 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 75 2022-06-29T15:52:05.650 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-06-29T15:52:05.802 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.794+0000 7fb7b5968700 1 Processor -- start 2022-06-29T15:52:05.803 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.794+0000 7fb7b5968700 1 -- start start 2022-06-29T15:52:05.813 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.798+0000 7fb7b5968700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb7b00b5d90 0x7fb7b0143a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:05.814 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.798+0000 7fb7b5968700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb7b00676a0 0x7fb7b0140130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:05.814 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.798+0000 7fb7b5968700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7b005cc30 con 0x7fb7b005c850 2022-06-29T15:52:05.815 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.798+0000 7fb7b5968700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7b01430a0 con 0x7fb7b00b5d90 2022-06-29T15:52:05.815 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.798+0000 7fb7b5968700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b0143300 con 0x7fb7b00676a0 2022-06-29T15:52:05.816 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.798+0000 7fb7affff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb7b00b5d90 0x7fb7b0143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:05.816 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.798+0000 7fb7b4966700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb7b005c850 0x7fb7b00b4860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40294/0 (socket says 172.21.15.47:40294) 2022-06-29T15:52:05.816 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.798+0000 7fb7b4966700 1 -- 172.21.15.47:0/589739314 learned_addr learned my addr 172.21.15.47:0/589739314 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:52:05.817 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.798+0000 7fb7b5167700 1 --2- 172.21.15.47:0/589739314 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb7b00676a0 0x7fb7b0140130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:05.817 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.798+0000 7fb7affff700 1 -- 172.21.15.47:0/589739314 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb7b00676a0 msgr2=0x7fb7b0140130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:05.818 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.798+0000 7fb7affff700 1 --2- 172.21.15.47:0/589739314 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb7b00676a0 0x7fb7b0140130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:05.818 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.798+0000 7fb7affff700 1 -- 172.21.15.47:0/589739314 >> v1:172.21.15.47:6789/0 conn(0x7fb7b005c850 legacy=0x7fb7b00b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:05.819 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.798+0000 7fb7affff700 1 -- 172.21.15.47:0/589739314 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b01435a0 con 0x7fb7b00b5d90 2022-06-29T15:52:05.819 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.798+0000 7fb7af7fe700 1 -- 172.21.15.47:0/589739314 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2763888753 0 0) 0x7fb7b005cc30 con 0x7fb7b005c850 2022-06-29T15:52:05.820 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.798+0000 7fb7affff700 1 --2- 172.21.15.47:0/589739314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7b00b5d90 0x7fb7b0143a40 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 rx=0x7fb798002f70 tx=0x7fb798006df0).ready entity=mon.0 client_cookie=a3ee11bbb5e6c80a server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:05.820 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.798+0000 7fb7af7fe700 1 -- 172.21.15.47:0/589739314 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb79800d570 con 0x7fb7b00b5d90 2022-06-29T15:52:05.821 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.798+0000 7fb7af7fe700 1 -- 172.21.15.47:0/589739314 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb798000e90 con 0x7fb7b00b5d90 2022-06-29T15:52:05.821 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.798+0000 7fb7af7fe700 1 -- 172.21.15.47:0/589739314 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb79800d8c0 con 0x7fb7b00b5d90 2022-06-29T15:52:05.822 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.798+0000 7fb7b5968700 1 -- 172.21.15.47:0/589739314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7b00b5d90 msgr2=0x7fb7b0143a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:05.822 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.798+0000 7fb7b5968700 1 --2- 172.21.15.47:0/589739314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7b00b5d90 0x7fb7b0143a40 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 rx=0x7fb798002f70 tx=0x7fb798006df0).stop 2022-06-29T15:52:05.823 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.798+0000 7fb7b5968700 1 -- 172.21.15.47:0/589739314 shutdown_connections 2022-06-29T15:52:05.823 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.798+0000 7fb7b5968700 1 --2- 172.21.15.47:0/589739314 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb7b00676a0 0x7fb7b0140130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:05.823 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.798+0000 7fb7b5968700 1 --2- 172.21.15.47:0/589739314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7b00b5d90 0x7fb7b0143a40 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:05.824 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.798+0000 7fb7b5968700 1 -- 172.21.15.47:0/589739314 >> 172.21.15.47:0/589739314 conn(0x7fb7b0060e00 msgr2=0x7fb7b00611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:05.824 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.798+0000 7fb7b5968700 1 -- 172.21.15.47:0/589739314 shutdown_connections 2022-06-29T15:52:05.825 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.798+0000 7fb7b5968700 1 -- 172.21.15.47:0/589739314 wait complete. 2022-06-29T15:52:05.825 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.798+0000 7fb7b5968700 1 Processor -- start 2022-06-29T15:52:05.826 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.798+0000 7fb7b5968700 1 -- start start 2022-06-29T15:52:05.826 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.802+0000 7fb7b5968700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb7b005c850 0x7fb7b0141280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:05.827 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.802+0000 7fb7b5968700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7b00676a0 0x7fb7b00b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:05.827 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.802+0000 7fb7b5968700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7b00b5d90 0x7fb7b0152170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:05.828 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.802+0000 7fb7b5968700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b00b35f0 con 0x7fb7b00b5d90 2022-06-29T15:52:05.828 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.802+0000 7fb7b5968700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b0141f30 con 0x7fb7b00676a0 2022-06-29T15:52:05.829 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.802+0000 7fb7b5968700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b0141770 con 0x7fb7b005c850 2022-06-29T15:52:05.829 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.802+0000 7fb7b5167700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7b00b5d90 0x7fb7b0152170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:05.830 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.802+0000 7fb7b5167700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7b00b5d90 0x7fb7b0152170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54750/0 (socket says 172.21.15.47:54750) 2022-06-29T15:52:05.830 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.802+0000 7fb7b5167700 1 -- 172.21.15.47:0/3458399037 learned_addr learned my addr 172.21.15.47:0/3458399037 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:05.831 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.802+0000 7fb7b4966700 1 --2- 172.21.15.47:0/3458399037 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb7b005c850 0x7fb7b0141280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:05.831 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.802+0000 7fb7affff700 1 --2- 172.21.15.47:0/3458399037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7b00676a0 0x7fb7b00b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:05.832 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.802+0000 7fb7b5167700 1 -- 172.21.15.47:0/3458399037 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb7b005c850 msgr2=0x7fb7b0141280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:05.832 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.802+0000 7fb7b5167700 1 --2- 172.21.15.47:0/3458399037 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb7b005c850 0x7fb7b0141280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:05.833 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.802+0000 7fb7b5167700 1 -- 172.21.15.47:0/3458399037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7b00676a0 msgr2=0x7fb7b00b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:05.833 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.802+0000 7fb7b5167700 1 --2- 172.21.15.47:0/3458399037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7b00676a0 0x7fb7b00b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:05.834 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.802+0000 7fb7b5167700 1 -- 172.21.15.47:0/3458399037 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb79800b180 con 0x7fb7b00b5d90 2022-06-29T15:52:05.834 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.802+0000 7fb7b5167700 1 --2- 172.21.15.47:0/3458399037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7b00b5d90 0x7fb7b0152170 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 rx=0x7fb7a800c000 tx=0x7fb7a8006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:05.835 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.802+0000 7fb7adffb700 1 -- 172.21.15.47:0/3458399037 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb7a8011750 con 0x7fb7b00b5d90 2022-06-29T15:52:05.835 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.802+0000 7fb7adffb700 1 -- 172.21.15.47:0/3458399037 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7a80118b0 con 0x7fb7b00b5d90 2022-06-29T15:52:05.836 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.802+0000 7fb7b5968700 1 -- 172.21.15.47:0/3458399037 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7b0141c90 con 0x7fb7b00b5d90 2022-06-29T15:52:05.836 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.802+0000 7fb7adffb700 1 -- 172.21.15.47:0/3458399037 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb7a8011c10 con 0x7fb7b00b5d90 2022-06-29T15:52:05.836 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.802+0000 7fb7b5968700 1 -- 172.21.15.47:0/3458399037 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7b00b11e0 con 0x7fb7b00b5d90 2022-06-29T15:52:05.837 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.802+0000 7fb7b5968700 1 -- 172.21.15.47:0/3458399037 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7b00573d0 con 0x7fb7b00b5d90 2022-06-29T15:52:05.837 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.802+0000 7fb7adffb700 1 -- 172.21.15.47:0/3458399037 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb7a8011d70 con 0x7fb7b00b5d90 2022-06-29T15:52:05.838 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.806+0000 7fb7adffb700 1 --2- 172.21.15.47:0/3458399037 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb7940539e0 0x7fb794055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:05.838 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.806+0000 7fb7b4966700 1 --2- 172.21.15.47:0/3458399037 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb7940539e0 0x7fb794055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:05.838 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.806+0000 7fb7adffb700 1 -- 172.21.15.47:0/3458399037 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fb7a806c470 con 0x7fb7b00b5d90 2022-06-29T15:52:05.839 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.806+0000 7fb7b4966700 1 --2- 172.21.15.47:0/3458399037 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb7940539e0 0x7fb794055ea0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7fb7a000d000 tx=0x7fb7a0009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:05.840 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.810+0000 7fb7adffb700 1 -- 172.21.15.47:0/3458399037 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb7a8043160 con 0x7fb7b00b5d90 2022-06-29T15:52:05.985 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:05.978+0000 7fb7b5968700 1 -- 172.21.15.47:0/3458399037 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb7b0061b00 con 0x7fb7b00b5d90 2022-06-29T15:52:06.337 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.330+0000 7fb7adffb700 1 -- 172.21.15.47:0/3458399037 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v76) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb7a8046780 con 0x7fb7b00b5d90 2022-06-29T15:52:06.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.334+0000 7fb7b5968700 1 -- 172.21.15.47:0/3458399037 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb7940539e0 msgr2=0x7fb794055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:06.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.334+0000 7fb7b5968700 1 --2- 172.21.15.47:0/3458399037 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb7940539e0 0x7fb794055ea0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7fb7a000d000 tx=0x7fb7a0009040).stop 2022-06-29T15:52:06.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.334+0000 7fb7b5968700 1 -- 172.21.15.47:0/3458399037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7b00b5d90 msgr2=0x7fb7b0152170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:06.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.334+0000 7fb7b5968700 1 --2- 172.21.15.47:0/3458399037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7b00b5d90 0x7fb7b0152170 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 rx=0x7fb7a800c000 tx=0x7fb7a8006df0).stop 2022-06-29T15:52:06.342 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.334+0000 7fb7b5968700 1 -- 172.21.15.47:0/3458399037 shutdown_connections 2022-06-29T15:52:06.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.334+0000 7fb7b5968700 1 --2- 172.21.15.47:0/3458399037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7b00b5d90 0x7fb7b0152170 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:06.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.334+0000 7fb7b5968700 1 --2- 172.21.15.47:0/3458399037 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb7940539e0 0x7fb794055ea0 secure :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 rx=0x7fb7a000d000 tx=0x7fb7a0009040).stop 2022-06-29T15:52:06.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.334+0000 7fb7b5968700 1 --2- 172.21.15.47:0/3458399037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7b00676a0 0x7fb7b00b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:06.343 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.334+0000 7fb7b5968700 1 --2- 172.21.15.47:0/3458399037 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb7b005c850 0x7fb7b0141280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:06.344 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.334+0000 7fb7b5968700 1 -- 172.21.15.47:0/3458399037 >> 172.21.15.47:0/3458399037 conn(0x7fb7b0060e00 msgr2=0x7fb7b01447e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:06.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.334+0000 7fb7b5968700 1 -- 172.21.15.47:0/3458399037 shutdown_connections 2022-06-29T15:52:06.345 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.334+0000 7fb7b5968700 1 -- 172.21.15.47:0/3458399037 wait complete. 2022-06-29T15:52:06.356 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-06-29T15:52:06.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.458+0000 7f4bbd2c2700 1 Processor -- start 2022-06-29T15:52:06.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.458+0000 7f4bbd2c2700 1 -- start start 2022-06-29T15:52:06.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.458+0000 7f4bbd2c2700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4bb80b6d20 0x7f4bb8145cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:06.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.458+0000 7f4bbd2c2700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bb80b7100 con 0x7f4bb8062f30 2022-06-29T15:52:06.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.458+0000 7f4bbd2c2700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bb81448f0 con 0x7f4bb80b1c80 2022-06-29T15:52:06.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.458+0000 7f4bbd2c2700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bb81444d0 con 0x7f4bb80b6d20 2022-06-29T15:52:06.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.458+0000 7f4bb7fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4bb80b6d20 0x7f4bb8145cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:06.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.458+0000 7f4bb77fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4bb8062f30 0x7f4bb81461e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40312/0 (socket says 172.21.15.47:40312) 2022-06-29T15:52:06.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.458+0000 7f4bb77fe700 1 -- 172.21.15.47:0/3335124061 learned_addr learned my addr 172.21.15.47:0/3335124061 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:52:06.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.458+0000 7f4bb7fff700 1 -- 172.21.15.47:0/3335124061 >> v1:172.21.15.64:6789/0 conn(0x7f4bb80b1c80 legacy=0x7f4bb80b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:06.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.458+0000 7f4bb7fff700 1 -- 172.21.15.47:0/3335124061 >> v1:172.21.15.47:6789/0 conn(0x7f4bb8062f30 legacy=0x7f4bb81461e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:06.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.458+0000 7f4bb7fff700 1 -- 172.21.15.47:0/3335124061 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb80b1810 con 0x7f4bb80b6d20 2022-06-29T15:52:06.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.458+0000 7f4bb6ffd700 1 -- 172.21.15.47:0/3335124061 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3703642293 0 0) 0x7f4bb80b7100 con 0x7f4bb8062f30 2022-06-29T15:52:06.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.458+0000 7f4bb7fff700 1 --2- 172.21.15.47:0/3335124061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bb80b6d20 0x7f4bb8145cf0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 rx=0x7f4ba8002f70 tx=0x7f4ba8006df0).ready entity=mon.0 client_cookie=97b95389b1542b11 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:06.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.458+0000 7f4bb6ffd700 1 -- 172.21.15.47:0/3335124061 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ba800d610 con 0x7f4bb80b6d20 2022-06-29T15:52:06.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.458+0000 7f4bb6ffd700 1 -- 172.21.15.47:0/3335124061 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ba8000e90 con 0x7f4bb80b6d20 2022-06-29T15:52:06.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.458+0000 7f4bb6ffd700 1 -- 172.21.15.47:0/3335124061 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ba800d960 con 0x7f4bb80b6d20 2022-06-29T15:52:06.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.462+0000 7f4bbd2c2700 1 -- 172.21.15.47:0/3335124061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bb80b6d20 msgr2=0x7f4bb8145cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:06.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.462+0000 7f4bbd2c2700 1 --2- 172.21.15.47:0/3335124061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bb80b6d20 0x7f4bb8145cf0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 rx=0x7f4ba8002f70 tx=0x7f4ba8006df0).stop 2022-06-29T15:52:06.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.462+0000 7f4bbd2c2700 1 -- 172.21.15.47:0/3335124061 shutdown_connections 2022-06-29T15:52:06.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.462+0000 7f4bbd2c2700 1 --2- 172.21.15.47:0/3335124061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bb80b6d20 0x7f4bb8145cf0 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:06.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.462+0000 7f4bbd2c2700 1 -- 172.21.15.47:0/3335124061 >> 172.21.15.47:0/3335124061 conn(0x7f4bb8063670 msgr2=0x7f4bb8063a50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:06.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.462+0000 7f4bbd2c2700 1 -- 172.21.15.47:0/3335124061 shutdown_connections 2022-06-29T15:52:06.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.462+0000 7f4bbd2c2700 1 -- 172.21.15.47:0/3335124061 wait complete. 2022-06-29T15:52:06.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.462+0000 7f4bbd2c2700 1 Processor -- start 2022-06-29T15:52:06.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.462+0000 7f4bbd2c2700 1 -- start start 2022-06-29T15:52:06.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.466+0000 7f4bbd2c2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bb8062f30 0x7f4bb80b02d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:06.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.466+0000 7f4bbd2c2700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4bb80b1c80 0x7f4bb81559c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:06.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.466+0000 7f4bbd2c2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bb80b6d20 0x7f4bb8155eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:06.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.466+0000 7f4bbd2c2700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb80ac1a0 con 0x7f4bb8062f30 2022-06-29T15:52:06.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.466+0000 7f4bbd2c2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb80ac870 con 0x7f4bb80b6d20 2022-06-29T15:52:06.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.466+0000 7f4bbd2c2700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb80ae760 con 0x7f4bb80b1c80 2022-06-29T15:52:06.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.466+0000 7f4bb7fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bb8062f30 0x7f4bb80b02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:06.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.466+0000 7f4bb7fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bb8062f30 0x7f4bb80b02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54762/0 (socket says 172.21.15.47:54762) 2022-06-29T15:52:06.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.466+0000 7f4bb7fff700 1 -- 172.21.15.47:0/2921982454 learned_addr learned my addr 172.21.15.47:0/2921982454 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:06.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.466+0000 7f4bb77fe700 1 --2- 172.21.15.47:0/2921982454 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4bb80b1c80 0x7f4bb81559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:06.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.466+0000 7f4bbcac1700 1 --2- 172.21.15.47:0/2921982454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bb80b6d20 0x7f4bb8155eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:06.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.466+0000 7f4bb7fff700 1 -- 172.21.15.47:0/2921982454 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4bb80b1c80 msgr2=0x7f4bb81559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:06.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.466+0000 7f4bb7fff700 1 --2- 172.21.15.47:0/2921982454 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4bb80b1c80 0x7f4bb81559c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:06.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.466+0000 7f4bb7fff700 1 -- 172.21.15.47:0/2921982454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bb80b6d20 msgr2=0x7f4bb8155eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:06.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.466+0000 7f4bb7fff700 1 --2- 172.21.15.47:0/2921982454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bb80b6d20 0x7f4bb8155eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:06.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.466+0000 7f4bb7fff700 1 -- 172.21.15.47:0/2921982454 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba800b180 con 0x7f4bb8062f30 2022-06-29T15:52:06.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.466+0000 7f4bb77fe700 1 --2- 172.21.15.47:0/2921982454 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4bb80b1c80 0x7f4bb81559c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:52:06.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.466+0000 7f4bb7fff700 1 --2- 172.21.15.47:0/2921982454 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bb8062f30 0x7f4bb80b02d0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 rx=0x7f4ba8002f70 tx=0x7f4ba8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:06.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.466+0000 7f4bb57fa700 1 -- 172.21.15.47:0/2921982454 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ba800d990 con 0x7f4bb8062f30 2022-06-29T15:52:06.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.466+0000 7f4bb57fa700 1 -- 172.21.15.47:0/2921982454 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ba8011c30 con 0x7f4bb8062f30 2022-06-29T15:52:06.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.466+0000 7f4bbd2c2700 1 -- 172.21.15.47:0/2921982454 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bb80aea00 con 0x7f4bb8062f30 2022-06-29T15:52:06.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.466+0000 7f4bb57fa700 1 -- 172.21.15.47:0/2921982454 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ba801e460 con 0x7f4bb8062f30 2022-06-29T15:52:06.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.466+0000 7f4bbd2c2700 1 -- 172.21.15.47:0/2921982454 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bb81563a0 con 0x7f4bb8062f30 2022-06-29T15:52:06.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.470+0000 7f4bbd2c2700 1 -- 172.21.15.47:0/2921982454 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bb80573d0 con 0x7f4bb8062f30 2022-06-29T15:52:06.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.470+0000 7f4bb57fa700 1 -- 172.21.15.47:0/2921982454 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4ba8025030 con 0x7f4bb8062f30 2022-06-29T15:52:06.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.470+0000 7f4bb57fa700 1 --2- 172.21.15.47:0/2921982454 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4b98053b30 0x7f4b98055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:06.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.470+0000 7f4bb77fe700 1 --2- 172.21.15.47:0/2921982454 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4b98053b30 0x7f4b98055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:06.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.470+0000 7f4bb57fa700 1 -- 172.21.15.47:0/2921982454 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f4ba806d380 con 0x7f4bb8062f30 2022-06-29T15:52:06.483 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.470+0000 7f4bb77fe700 1 --2- 172.21.15.47:0/2921982454 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4b98053b30 0x7f4b98055ff0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7f4bac00b000 tx=0x7f4bac009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:06.484 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.478+0000 7f4bb57fa700 1 -- 172.21.15.47:0/2921982454 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4ba80440f0 con 0x7f4bb8062f30 2022-06-29T15:52:06.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:06.610+0000 7f4bbd2c2700 1 -- 172.21.15.47:0/2921982454 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4bb80b7300 con 0x7f4bb8062f30 2022-06-29T15:52:07.360 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.354+0000 7f4bb57fa700 1 -- 172.21.15.47:0/2921982454 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v153) v1 ==== 185+0+0 (secure 0 0 0) 0x7f4ba80230e0 con 0x7f4bb8062f30 2022-06-29T15:52:07.362 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.354+0000 7f4bbd2c2700 1 -- 172.21.15.47:0/2921982454 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4b98053b30 msgr2=0x7f4b98055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:07.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.354+0000 7f4bbd2c2700 1 --2- 172.21.15.47:0/2921982454 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4b98053b30 0x7f4b98055ff0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7f4bac00b000 tx=0x7f4bac009040).stop 2022-06-29T15:52:07.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.354+0000 7f4bbd2c2700 1 -- 172.21.15.47:0/2921982454 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bb8062f30 msgr2=0x7f4bb80b02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:07.363 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.354+0000 7f4bbd2c2700 1 --2- 172.21.15.47:0/2921982454 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bb8062f30 0x7f4bb80b02d0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 rx=0x7f4ba8002f70 tx=0x7f4ba8015000).stop 2022-06-29T15:52:07.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.358+0000 7f4bbd2c2700 1 -- 172.21.15.47:0/2921982454 shutdown_connections 2022-06-29T15:52:07.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.358+0000 7f4bbd2c2700 1 --2- 172.21.15.47:0/2921982454 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4b98053b30 0x7f4b98055ff0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:07.365 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.358+0000 7f4bbd2c2700 1 --2- 172.21.15.47:0/2921982454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bb80b6d20 0x7f4bb8155eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:07.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.358+0000 7f4bbd2c2700 1 --2- 172.21.15.47:0/2921982454 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4bb80b1c80 0x7f4bb81559c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:07.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.358+0000 7f4bbd2c2700 1 --2- 172.21.15.47:0/2921982454 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4bb8062f30 0x7f4bb80b02d0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:07.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.358+0000 7f4bbd2c2700 1 -- 172.21.15.47:0/2921982454 >> 172.21.15.47:0/2921982454 conn(0x7f4bb8063670 msgr2=0x7f4bb80b42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:07.366 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.358+0000 7f4bbd2c2700 1 -- 172.21.15.47:0/2921982454 shutdown_connections 2022-06-29T15:52:07.367 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.358+0000 7f4bbd2c2700 1 -- 172.21.15.47:0/2921982454 wait complete. 2022-06-29T15:52:07.367 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' removed 2022-06-29T15:52:07.376 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-06-29T15:52:07.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.482+0000 7f9f548b1700 1 Processor -- start 2022-06-29T15:52:07.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.482+0000 7f9f548b1700 1 -- start start 2022-06-29T15:52:07.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.482+0000 7f9f548b1700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9f500b6e20 0x7f9f501447c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:07.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.482+0000 7f9f548b1700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9f50144cb0 0x7f9f501419f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:07.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.482+0000 7f9f548b1700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f500b7200 con 0x7f9f500630f0 2022-06-29T15:52:07.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.482+0000 7f9f548b1700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f5005f5c0 con 0x7f9f50144cb0 2022-06-29T15:52:07.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.482+0000 7f9f548b1700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5005f860 con 0x7f9f500b6e20 2022-06-29T15:52:07.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.482+0000 7f9f4f59e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9f50144cb0 0x7f9f501419f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:07.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.482+0000 7f9f4f59e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9f50144cb0 0x7f9f501419f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54772/0 (socket says 172.21.15.47:54772) 2022-06-29T15:52:07.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.482+0000 7f9f4f59e700 1 -- 172.21.15.47:0/1895442952 learned_addr learned my addr 172.21.15.47:0/1895442952 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:07.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.482+0000 7f9f4ed9d700 1 --2- 172.21.15.47:0/1895442952 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9f500b6e20 0x7f9f501447c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:07.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.482+0000 7f9f4f59e700 1 -- 172.21.15.47:0/1895442952 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9f500b6e20 msgr2=0x7f9f501447c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:07.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.482+0000 7f9f4f59e700 1 --2- 172.21.15.47:0/1895442952 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9f500b6e20 0x7f9f501447c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:07.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.482+0000 7f9f4f59e700 1 -- 172.21.15.47:0/1895442952 >> v1:172.21.15.64:6789/0 conn(0x7f9f500630f0 legacy=0x7f9f501412e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:07.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.482+0000 7f9f4f59e700 1 -- 172.21.15.47:0/1895442952 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f500afcb0 con 0x7f9f50144cb0 2022-06-29T15:52:07.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.482+0000 7f9f4f59e700 1 --2- 172.21.15.47:0/1895442952 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f50144cb0 0x7f9f501419f0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 rx=0x7f9f480032f0 tx=0x7f9f48009290).ready entity=mon.0 client_cookie=8c557f4e50de53af server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:07.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.482+0000 7f9f4dd9b700 1 -- 172.21.15.47:0/1895442952 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f4800b870 con 0x7f9f50144cb0 2022-06-29T15:52:07.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.482+0000 7f9f4dd9b700 1 -- 172.21.15.47:0/1895442952 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f4800b9d0 con 0x7f9f50144cb0 2022-06-29T15:52:07.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.482+0000 7f9f4dd9b700 1 -- 172.21.15.47:0/1895442952 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f4800bb80 con 0x7f9f50144cb0 2022-06-29T15:52:07.496 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.482+0000 7f9f548b1700 1 -- 172.21.15.47:0/1895442952 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f50144cb0 msgr2=0x7f9f501419f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:07.497 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.482+0000 7f9f548b1700 1 --2- 172.21.15.47:0/1895442952 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f50144cb0 0x7f9f501419f0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 rx=0x7f9f480032f0 tx=0x7f9f48009290).stop 2022-06-29T15:52:07.497 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.482+0000 7f9f548b1700 1 -- 172.21.15.47:0/1895442952 shutdown_connections 2022-06-29T15:52:07.497 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.482+0000 7f9f548b1700 1 --2- 172.21.15.47:0/1895442952 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f50144cb0 0x7f9f501419f0 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:07.497 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.482+0000 7f9f548b1700 1 --2- 172.21.15.47:0/1895442952 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9f500b6e20 0x7f9f501447c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:07.497 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.482+0000 7f9f548b1700 1 -- 172.21.15.47:0/1895442952 >> 172.21.15.47:0/1895442952 conn(0x7f9f50063670 msgr2=0x7f9f50063a50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:07.498 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.482+0000 7f9f548b1700 1 -- 172.21.15.47:0/1895442952 shutdown_connections 2022-06-29T15:52:07.498 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.482+0000 7f9f548b1700 1 -- 172.21.15.47:0/1895442952 wait complete. 2022-06-29T15:52:07.498 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.486+0000 7f9f548b1700 1 Processor -- start 2022-06-29T15:52:07.498 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.486+0000 7f9f548b1700 1 -- start start 2022-06-29T15:52:07.499 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.486+0000 7f9f548b1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f500630f0 0x7f9f50150c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:07.499 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.486+0000 7f9f548b1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f500b6e20 0x7f9f50143340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:07.499 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.486+0000 7f9f548b1700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9f50144cb0 0x7f9f500acb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:07.499 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.486+0000 7f9f548b1700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f500aa320 con 0x7f9f500630f0 2022-06-29T15:52:07.499 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.486+0000 7f9f548b1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f500ac1d0 con 0x7f9f500b6e20 2022-06-29T15:52:07.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.486+0000 7f9f548b1700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f501510f0 con 0x7f9f50144cb0 2022-06-29T15:52:07.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.486+0000 7f9f4ed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f500630f0 0x7f9f50150c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:07.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.486+0000 7f9f4ed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f500630f0 0x7f9f50150c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54776/0 (socket says 172.21.15.47:54776) 2022-06-29T15:52:07.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.486+0000 7f9f4ed9d700 1 -- 172.21.15.47:0/1030869904 learned_addr learned my addr 172.21.15.47:0/1030869904 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:07.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.486+0000 7f9f4e59c700 1 --2- 172.21.15.47:0/1030869904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f500b6e20 0x7f9f50143340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:07.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.486+0000 7f9f4f59e700 1 --2- 172.21.15.47:0/1030869904 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9f50144cb0 0x7f9f500acb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:07.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.486+0000 7f9f4ed9d700 1 -- 172.21.15.47:0/1030869904 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9f50144cb0 msgr2=0x7f9f500acb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:07.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.486+0000 7f9f4ed9d700 1 --2- 172.21.15.47:0/1030869904 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9f50144cb0 0x7f9f500acb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:07.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.486+0000 7f9f4ed9d700 1 -- 172.21.15.47:0/1030869904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f500b6e20 msgr2=0x7f9f50143340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:07.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.486+0000 7f9f4ed9d700 1 --2- 172.21.15.47:0/1030869904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f500b6e20 0x7f9f50143340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:07.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.486+0000 7f9f4ed9d700 1 -- 172.21.15.47:0/1030869904 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f48009530 con 0x7f9f500630f0 2022-06-29T15:52:07.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.486+0000 7f9f4f59e700 1 --2- 172.21.15.47:0/1030869904 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9f50144cb0 0x7f9f500acb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:52:07.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.486+0000 7f9f4ed9d700 1 --2- 172.21.15.47:0/1030869904 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f500630f0 0x7f9f50150c00 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 rx=0x7f9f500ac660 tx=0x7f9f40008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:07.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.490+0000 7f9f3ffff700 1 -- 172.21.15.47:0/1030869904 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f40011780 con 0x7f9f500630f0 2022-06-29T15:52:07.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.490+0000 7f9f3ffff700 1 -- 172.21.15.47:0/1030869904 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f400118e0 con 0x7f9f500630f0 2022-06-29T15:52:07.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.490+0000 7f9f548b1700 1 -- 172.21.15.47:0/1030869904 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f500ab6d0 con 0x7f9f500630f0 2022-06-29T15:52:07.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.490+0000 7f9f3ffff700 1 -- 172.21.15.47:0/1030869904 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f40011c40 con 0x7f9f500630f0 2022-06-29T15:52:07.503 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.490+0000 7f9f548b1700 1 -- 172.21.15.47:0/1030869904 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f50143830 con 0x7f9f500630f0 2022-06-29T15:52:07.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.490+0000 7f9f548b1700 1 -- 172.21.15.47:0/1030869904 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f500ac360 con 0x7f9f500630f0 2022-06-29T15:52:07.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.490+0000 7f9f3ffff700 1 -- 172.21.15.47:0/1030869904 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9f40012420 con 0x7f9f500630f0 2022-06-29T15:52:07.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.494+0000 7f9f3ffff700 1 --2- 172.21.15.47:0/1030869904 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9f2c053a70 0x7f9f2c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:07.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.494+0000 7f9f3ffff700 1 -- 172.21.15.47:0/1030869904 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6068+0+0 (secure 0 0 0) 0x7f9f4006be50 con 0x7f9f500630f0 2022-06-29T15:52:07.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.494+0000 7f9f4e59c700 1 --2- 172.21.15.47:0/1030869904 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9f2c053a70 0x7f9f2c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:07.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.494+0000 7f9f4e59c700 1 --2- 172.21.15.47:0/1030869904 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9f2c053a70 0x7f9f2c055f30 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7f9f4400d000 tx=0x7f9f44000bc0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:07.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.498+0000 7f9f3ffff700 1 -- 172.21.15.47:0/1030869904 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9f500ac360 con 0x7f9f500630f0 2022-06-29T15:52:07.631 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:07.626+0000 7f9f548b1700 1 -- 172.21.15.47:0/1030869904 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9f500ac360 con 0x7f9f500630f0 2022-06-29T15:52:08.373 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.366+0000 7f9f3ffff700 1 -- 172.21.15.47:0/1030869904 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v154) v1 ==== 173+0+0 (secure 0 0 0) 0x7f9f500ac360 con 0x7f9f500630f0 2022-06-29T15:52:08.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.370+0000 7f9f548b1700 1 -- 172.21.15.47:0/1030869904 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9f2c053a70 msgr2=0x7f9f2c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:08.377 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.370+0000 7f9f548b1700 1 --2- 172.21.15.47:0/1030869904 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9f2c053a70 0x7f9f2c055f30 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7f9f4400d000 tx=0x7f9f44000bc0).stop 2022-06-29T15:52:08.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.370+0000 7f9f548b1700 1 -- 172.21.15.47:0/1030869904 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f500630f0 msgr2=0x7f9f50150c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:08.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.370+0000 7f9f548b1700 1 --2- 172.21.15.47:0/1030869904 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f500630f0 0x7f9f50150c00 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 rx=0x7f9f500ac660 tx=0x7f9f40008040).stop 2022-06-29T15:52:08.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.370+0000 7f9f548b1700 1 -- 172.21.15.47:0/1030869904 shutdown_connections 2022-06-29T15:52:08.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.370+0000 7f9f548b1700 1 --2- 172.21.15.47:0/1030869904 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9f50144cb0 0x7f9f500acb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:08.379 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.370+0000 7f9f548b1700 1 --2- 172.21.15.47:0/1030869904 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9f2c053a70 0x7f9f2c055f30 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:08.379 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.370+0000 7f9f548b1700 1 --2- 172.21.15.47:0/1030869904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f500b6e20 0x7f9f50143340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:08.379 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.370+0000 7f9f548b1700 1 --2- 172.21.15.47:0/1030869904 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f500630f0 0x7f9f50150c00 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:08.379 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.370+0000 7f9f548b1700 1 -- 172.21.15.47:0/1030869904 >> 172.21.15.47:0/1030869904 conn(0x7f9f50063670 msgr2=0x7f9f500612b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:08.380 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.370+0000 7f9f548b1700 1 -- 172.21.15.47:0/1030869904 shutdown_connections 2022-06-29T15:52:08.380 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.370+0000 7f9f548b1700 1 -- 172.21.15.47:0/1030869904 wait complete. 2022-06-29T15:52:08.380 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' removed 2022-06-29T15:52:08.392 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-06-29T15:52:08.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.498+0000 7f73c2b5e700 1 Processor -- start 2022-06-29T15:52:08.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.498+0000 7f73c2b5e700 1 -- start start 2022-06-29T15:52:08.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.498+0000 7f73c2b5e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f73bc05f0b0 0x7f73bc05f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:08.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.498+0000 7f73c2b5e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73bc061e50 con 0x7f73bc061740 2022-06-29T15:52:08.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.498+0000 7f73c2b5e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73bc0620f0 con 0x7f73bc144670 2022-06-29T15:52:08.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.498+0000 7f73c2b5e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73bc1434b0 con 0x7f73bc05f0b0 2022-06-29T15:52:08.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.502+0000 7f73c235d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f73bc061740 0x7f73bc140410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40340/0 (socket says 172.21.15.47:40340) 2022-06-29T15:52:08.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.502+0000 7f73c235d700 1 -- 172.21.15.47:0/3710688150 learned_addr learned my addr 172.21.15.47:0/3710688150 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:52:08.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.502+0000 7f73c1b5c700 1 --2- 172.21.15.47:0/3710688150 >> v2:172.21.15.64:3300/0 conn(0x7f73bc05f0b0 0x7f73bc05f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:08.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.502+0000 7f73c0b5a700 1 -- 172.21.15.47:0/3710688150 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3392182389 0 0) 0x7f73bc061e50 con 0x7f73bc061740 2022-06-29T15:52:08.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.502+0000 7f73c0b5a700 1 -- 172.21.15.47:0/3710688150 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73a4003260 con 0x7f73bc061740 2022-06-29T15:52:08.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.502+0000 7f73c0b5a700 1 -- 172.21.15.47:0/3710688150 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 826483414 0 0) 0x7f73bc0620f0 con 0x7f73bc144670 2022-06-29T15:52:08.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.502+0000 7f73c0b5a700 1 -- 172.21.15.47:0/3710688150 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73bc061e50 con 0x7f73bc144670 2022-06-29T15:52:08.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.502+0000 7f73c1b5c700 1 -- 172.21.15.47:0/3710688150 >> v1:172.21.15.64:6789/0 conn(0x7f73bc144670 legacy=0x7f73bc143950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:08.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.502+0000 7f73c1b5c700 1 -- 172.21.15.47:0/3710688150 >> v1:172.21.15.47:6789/0 conn(0x7f73bc061740 legacy=0x7f73bc140410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:08.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.502+0000 7f73c1b5c700 1 -- 172.21.15.47:0/3710688150 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73bc061bb0 con 0x7f73bc05f0b0 2022-06-29T15:52:08.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.502+0000 7f73c1b5c700 1 --2- 172.21.15.47:0/3710688150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73bc05f0b0 0x7f73bc05f490 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 rx=0x7f73b8002f70 tx=0x7f73b8006df0).ready entity=mon.1 client_cookie=3178f3673184c6d1 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:08.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.502+0000 7f73c0b5a700 1 -- 172.21.15.47:0/3710688150 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f73b800d540 con 0x7f73bc05f0b0 2022-06-29T15:52:08.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.502+0000 7f73c0b5a700 1 -- 172.21.15.47:0/3710688150 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73b8000e90 con 0x7f73bc05f0b0 2022-06-29T15:52:08.513 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.502+0000 7f73c0b5a700 1 -- 172.21.15.47:0/3710688150 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f73b800d890 con 0x7f73bc05f0b0 2022-06-29T15:52:08.513 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.502+0000 7f73c2b5e700 1 -- 172.21.15.47:0/3710688150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73bc05f0b0 msgr2=0x7f73bc05f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:08.513 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.502+0000 7f73c2b5e700 1 --2- 172.21.15.47:0/3710688150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73bc05f0b0 0x7f73bc05f490 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 rx=0x7f73b8002f70 tx=0x7f73b8006df0).stop 2022-06-29T15:52:08.513 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.502+0000 7f73c2b5e700 1 -- 172.21.15.47:0/3710688150 shutdown_connections 2022-06-29T15:52:08.514 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.502+0000 7f73c2b5e700 1 --2- 172.21.15.47:0/3710688150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73bc05f0b0 0x7f73bc05f490 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:08.514 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.502+0000 7f73c2b5e700 1 -- 172.21.15.47:0/3710688150 >> 172.21.15.47:0/3710688150 conn(0x7f73bc062570 msgr2=0x7f73bc0647d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:08.514 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.502+0000 7f73c2b5e700 1 -- 172.21.15.47:0/3710688150 shutdown_connections 2022-06-29T15:52:08.514 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.502+0000 7f73c2b5e700 1 -- 172.21.15.47:0/3710688150 wait complete. 2022-06-29T15:52:08.517 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.506+0000 7f73c2b5e700 1 Processor -- start 2022-06-29T15:52:08.518 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.506+0000 7f73c2b5e700 1 -- start start 2022-06-29T15:52:08.518 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.506+0000 7f73c2b5e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f73bc05f0b0 0x7f73bc0aa710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:08.518 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.506+0000 7f73c2b5e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73bc061740 0x7f73bc1427c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:08.518 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.506+0000 7f73c2b5e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73bc144670 0x7f73bc0b5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:08.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.506+0000 7f73c2b5e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73bc0b6340 con 0x7f73bc144670 2022-06-29T15:52:08.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.506+0000 7f73c2b5e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73bc0aad40 con 0x7f73bc061740 2022-06-29T15:52:08.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.506+0000 7f73c2b5e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73bc0b68b0 con 0x7f73bc05f0b0 2022-06-29T15:52:08.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.506+0000 7f73c235d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73bc144670 0x7f73bc0b5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:08.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.506+0000 7f73c1b5c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f73bc05f0b0 0x7f73bc0aa710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:08.520 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.506+0000 7f73c1b5c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f73bc05f0b0 0x7f73bc0aa710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:51890/0 (socket says 172.21.15.47:51890) 2022-06-29T15:52:08.520 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.506+0000 7f73c235d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73bc144670 0x7f73bc0b5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54792/0 (socket says 172.21.15.47:54792) 2022-06-29T15:52:08.520 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.506+0000 7f73c1b5c700 1 -- 172.21.15.47:0/2412956651 learned_addr learned my addr 172.21.15.47:0/2412956651 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:08.520 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.506+0000 7f73c135b700 1 --2- 172.21.15.47:0/2412956651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73bc061740 0x7f73bc1427c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:08.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.506+0000 7f73c235d700 1 -- 172.21.15.47:0/2412956651 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f73bc05f0b0 msgr2=0x7f73bc0aa710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:08.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.506+0000 7f73c235d700 1 --2- 172.21.15.47:0/2412956651 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f73bc05f0b0 0x7f73bc0aa710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:08.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.506+0000 7f73c235d700 1 -- 172.21.15.47:0/2412956651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73bc061740 msgr2=0x7f73bc1427c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:08.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.506+0000 7f73c235d700 1 --2- 172.21.15.47:0/2412956651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73bc061740 0x7f73bc1427c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:08.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.506+0000 7f73c235d700 1 -- 172.21.15.47:0/2412956651 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73b800b180 con 0x7f73bc144670 2022-06-29T15:52:08.522 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.506+0000 7f73c235d700 1 --2- 172.21.15.47:0/2412956651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73bc144670 0x7f73bc0b5e50 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 rx=0x7f73b4001490 tx=0x7f73b4007d60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:08.522 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.506+0000 7f73aaffd700 1 -- 172.21.15.47:0/2412956651 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f73b4005ba0 con 0x7f73bc144670 2022-06-29T15:52:08.522 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.506+0000 7f73aaffd700 1 -- 172.21.15.47:0/2412956651 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73b4005d00 con 0x7f73bc144670 2022-06-29T15:52:08.522 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.510+0000 7f73c2b5e700 1 -- 172.21.15.47:0/2412956651 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73bc0b6b50 con 0x7f73bc144670 2022-06-29T15:52:08.523 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.510+0000 7f73aaffd700 1 -- 172.21.15.47:0/2412956651 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f73b40072c0 con 0x7f73bc144670 2022-06-29T15:52:08.523 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.510+0000 7f73c2b5e700 1 -- 172.21.15.47:0/2412956651 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73bc143030 con 0x7f73bc144670 2022-06-29T15:52:08.523 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.510+0000 7f73c2b5e700 1 -- 172.21.15.47:0/2412956651 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73bc0573d0 con 0x7f73bc144670 2022-06-29T15:52:08.523 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.510+0000 7f73aaffd700 1 -- 172.21.15.47:0/2412956651 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f73b4010070 con 0x7f73bc144670 2022-06-29T15:52:08.523 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.510+0000 7f73aaffd700 1 --2- 172.21.15.47:0/2412956651 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f739c053a70 0x7f739c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:08.524 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.510+0000 7f73aaffd700 1 -- 172.21.15.47:0/2412956651 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f73b406b600 con 0x7f73bc144670 2022-06-29T15:52:08.524 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.510+0000 7f73c1b5c700 1 --2- 172.21.15.47:0/2412956651 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f739c053a70 0x7f739c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:08.525 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.510+0000 7f73c1b5c700 1 --2- 172.21.15.47:0/2412956651 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f739c053a70 0x7f739c055f30 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7f73b8018000 tx=0x7f73b8014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:08.525 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.518+0000 7f73aaffd700 1 -- 172.21.15.47:0/2412956651 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f73b4042590 con 0x7f73bc144670 2022-06-29T15:52:08.654 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.646+0000 7f73c2b5e700 1 -- 172.21.15.47:0/2412956651 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f73bc068830 con 0x7f73bc144670 2022-06-29T15:52:08.659 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.654+0000 7f73aaffd700 1 -- 172.21.15.47:0/2412956651 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7f73b4045bb0 con 0x7f73bc144670 2022-06-29T15:52:08.661 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.654+0000 7f73c2b5e700 1 -- 172.21.15.47:0/2412956651 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f739c053a70 msgr2=0x7f739c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:08.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.654+0000 7f73c2b5e700 1 --2- 172.21.15.47:0/2412956651 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f739c053a70 0x7f739c055f30 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7f73b8018000 tx=0x7f73b8014000).stop 2022-06-29T15:52:08.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.654+0000 7f73c2b5e700 1 -- 172.21.15.47:0/2412956651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73bc144670 msgr2=0x7f73bc0b5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:08.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.654+0000 7f73c2b5e700 1 --2- 172.21.15.47:0/2412956651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73bc144670 0x7f73bc0b5e50 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 rx=0x7f73b4001490 tx=0x7f73b4007d60).stop 2022-06-29T15:52:08.662 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.654+0000 7f73c2b5e700 1 -- 172.21.15.47:0/2412956651 shutdown_connections 2022-06-29T15:52:08.663 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.654+0000 7f73c2b5e700 1 --2- 172.21.15.47:0/2412956651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73bc144670 0x7f73bc0b5e50 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:08.663 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.654+0000 7f73c2b5e700 1 --2- 172.21.15.47:0/2412956651 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f739c053a70 0x7f739c055f30 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:08.663 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.654+0000 7f73c2b5e700 1 --2- 172.21.15.47:0/2412956651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73bc061740 0x7f73bc1427c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:08.663 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.654+0000 7f73c2b5e700 1 --2- 172.21.15.47:0/2412956651 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f73bc05f0b0 0x7f73bc0aa710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:08.664 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.654+0000 7f73c2b5e700 1 -- 172.21.15.47:0/2412956651 >> 172.21.15.47:0/2412956651 conn(0x7f73bc062570 msgr2=0x7f73bc065f50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:08.664 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.654+0000 7f73c2b5e700 1 -- 172.21.15.47:0/2412956651 shutdown_connections 2022-06-29T15:52:08.664 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.654+0000 7f73c2b5e700 1 -- 172.21.15.47:0/2412956651 wait complete. 2022-06-29T15:52:08.673 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestMisc.test_root_rctime' 2022-06-29T15:52:08.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.766+0000 7f7c1e8c6700 1 Processor -- start 2022-06-29T15:52:08.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.766+0000 7f7c1e8c6700 1 -- start start 2022-06-29T15:52:08.776 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.766+0000 7f7c1e8c6700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7c18061da0 0x7f7c180b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:08.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.766+0000 7f7c1e8c6700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7c180630f0 0x7f7c180b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:08.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.766+0000 7f7c1e8c6700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7c1805f0b0 0x7f7c180b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:08.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.766+0000 7f7c1e8c6700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c18062180 con 0x7f7c18061da0 2022-06-29T15:52:08.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.766+0000 7f7c1e8c6700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c1805fae0 con 0x7f7c1805f0b0 2022-06-29T15:52:08.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.766+0000 7f7c1e8c6700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c180b2ea0 con 0x7f7c180630f0 2022-06-29T15:52:08.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.766+0000 7f7c1d8c4700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7c18061da0 0x7f7c180b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:08.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.766+0000 7f7c1d8c4700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7c18061da0 0x7f7c180b3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54796/0 (socket says 172.21.15.47:54796) 2022-06-29T15:52:08.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.766+0000 7f7c1d8c4700 1 -- 172.21.15.47:0/171008703 learned_addr learned my addr 172.21.15.47:0/171008703 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:08.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.766+0000 7f7c1e0c5700 1 --2- 172.21.15.47:0/171008703 >> v2:172.21.15.64:3300/0 conn(0x7f7c1805f0b0 0x7f7c180b17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:08.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.766+0000 7f7c1d0c3700 1 --2- 172.21.15.47:0/171008703 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7c180630f0 0x7f7c180b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:08.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.770+0000 7f7c1d8c4700 1 -- 172.21.15.47:0/171008703 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7c180630f0 msgr2=0x7f7c180b0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:08.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.770+0000 7f7c1d8c4700 1 --2- 172.21.15.47:0/171008703 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7c180630f0 0x7f7c180b0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:08.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.770+0000 7f7c1d8c4700 1 -- 172.21.15.47:0/171008703 >> v2:172.21.15.64:3300/0 conn(0x7f7c1805f0b0 msgr2=0x7f7c180b17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:08.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.770+0000 7f7c1d8c4700 1 --2- 172.21.15.47:0/171008703 >> v2:172.21.15.64:3300/0 conn(0x7f7c1805f0b0 0x7f7c180b17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:08.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.770+0000 7f7c1d8c4700 1 -- 172.21.15.47:0/171008703 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c180b3140 con 0x7f7c18061da0 2022-06-29T15:52:08.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.770+0000 7f7c1d8c4700 1 --2- 172.21.15.47:0/171008703 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c18061da0 0x7f7c180b3880 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 rx=0x7f7c100032f0 tx=0x7f7c10009290).ready entity=mon.0 client_cookie=ea0ad95e3461b160 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:08.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.770+0000 7f7c1c8c2700 1 -- 172.21.15.47:0/171008703 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c1000b870 con 0x7f7c18061da0 2022-06-29T15:52:08.781 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.770+0000 7f7c1c8c2700 1 -- 172.21.15.47:0/171008703 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c1000b9d0 con 0x7f7c18061da0 2022-06-29T15:52:08.781 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.770+0000 7f7c1c8c2700 1 -- 172.21.15.47:0/171008703 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c1000bb40 con 0x7f7c18061da0 2022-06-29T15:52:08.781 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.770+0000 7f7c1e8c6700 1 -- 172.21.15.47:0/171008703 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c18061da0 msgr2=0x7f7c180b3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:08.781 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.770+0000 7f7c1e8c6700 1 --2- 172.21.15.47:0/171008703 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c18061da0 0x7f7c180b3880 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 rx=0x7f7c100032f0 tx=0x7f7c10009290).stop 2022-06-29T15:52:08.782 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.770+0000 7f7c1e8c6700 1 -- 172.21.15.47:0/171008703 shutdown_connections 2022-06-29T15:52:08.782 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.770+0000 7f7c1e8c6700 1 --2- 172.21.15.47:0/171008703 >> v2:172.21.15.64:3300/0 conn(0x7f7c1805f0b0 0x7f7c180b17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:08.782 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.770+0000 7f7c1e8c6700 1 --2- 172.21.15.47:0/171008703 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7c180630f0 0x7f7c180b0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:08.782 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.770+0000 7f7c1e8c6700 1 --2- 172.21.15.47:0/171008703 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c18061da0 0x7f7c180b3880 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:08.782 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.770+0000 7f7c1e8c6700 1 -- 172.21.15.47:0/171008703 >> 172.21.15.47:0/171008703 conn(0x7f7c18063670 msgr2=0x7f7c18063a80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:08.783 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.770+0000 7f7c1e8c6700 1 -- 172.21.15.47:0/171008703 shutdown_connections 2022-06-29T15:52:08.783 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.770+0000 7f7c1e8c6700 1 -- 172.21.15.47:0/171008703 wait complete. 2022-06-29T15:52:08.786 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.774+0000 7f7c1e8c6700 1 Processor -- start 2022-06-29T15:52:08.787 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.774+0000 7f7c1e8c6700 1 -- start start 2022-06-29T15:52:08.787 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.774+0000 7f7c1e8c6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c1805f0b0 0x7f7c180a41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:08.787 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.774+0000 7f7c1e8c6700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7c18061da0 0x7f7c180a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:08.787 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.774+0000 7f7c1e8c6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c180630f0 0x7f7c180acf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:08.788 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.774+0000 7f7c1e8c6700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1814bba0 con 0x7f7c180630f0 2022-06-29T15:52:08.788 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.774+0000 7f7c1e8c6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c180a2380 con 0x7f7c1805f0b0 2022-06-29T15:52:08.788 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.774+0000 7f7c1e8c6700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c180a2620 con 0x7f7c18061da0 2022-06-29T15:52:08.788 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.774+0000 7f7c1e0c5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c180630f0 0x7f7c180acf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:08.789 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.774+0000 7f7c1d8c4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c1805f0b0 0x7f7c180a41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:08.789 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.774+0000 7f7c1e0c5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c180630f0 0x7f7c180acf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54806/0 (socket says 172.21.15.47:54806) 2022-06-29T15:52:08.789 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.774+0000 7f7c1e0c5700 1 -- 172.21.15.47:0/2900054562 learned_addr learned my addr 172.21.15.47:0/2900054562 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:08.789 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.774+0000 7f7c1d8c4700 1 --2- 172.21.15.47:0/2900054562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c1805f0b0 0x7f7c180a41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:60506/0 (socket says 172.21.15.47:60506) 2022-06-29T15:52:08.789 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.774+0000 7f7c1d0c3700 1 --2- 172.21.15.47:0/2900054562 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7c18061da0 0x7f7c180a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:08.790 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.774+0000 7f7c1e0c5700 1 -- 172.21.15.47:0/2900054562 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7c18061da0 msgr2=0x7f7c180a6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:08.790 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.774+0000 7f7c1e0c5700 1 --2- 172.21.15.47:0/2900054562 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7c18061da0 0x7f7c180a6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:08.790 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.774+0000 7f7c1e0c5700 1 -- 172.21.15.47:0/2900054562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c1805f0b0 msgr2=0x7f7c180a41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:08.790 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.774+0000 7f7c1e0c5700 1 --2- 172.21.15.47:0/2900054562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c1805f0b0 0x7f7c180a41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:08.791 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.774+0000 7f7c1e0c5700 1 -- 172.21.15.47:0/2900054562 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c10009530 con 0x7f7c180630f0 2022-06-29T15:52:08.791 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.774+0000 7f7c1e0c5700 1 --2- 172.21.15.47:0/2900054562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c180630f0 0x7f7c180acf80 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 rx=0x7f7c0800a000 tx=0x7f7c08008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:08.791 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.774+0000 7f7c16ffd700 1 -- 172.21.15.47:0/2900054562 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c08011780 con 0x7f7c180630f0 2022-06-29T15:52:08.791 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.774+0000 7f7c16ffd700 1 -- 172.21.15.47:0/2900054562 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c080118e0 con 0x7f7c180630f0 2022-06-29T15:52:08.792 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.774+0000 7f7c1e8c6700 1 -- 172.21.15.47:0/2900054562 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c180a28c0 con 0x7f7c180630f0 2022-06-29T15:52:08.792 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.774+0000 7f7c16ffd700 1 -- 172.21.15.47:0/2900054562 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c08011c40 con 0x7f7c180630f0 2022-06-29T15:52:08.792 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.774+0000 7f7c1e8c6700 1 -- 172.21.15.47:0/2900054562 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c180a2de0 con 0x7f7c180630f0 2022-06-29T15:52:08.792 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.778+0000 7f7c1e8c6700 1 -- 172.21.15.47:0/2900054562 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c180a2a50 con 0x7f7c180630f0 2022-06-29T15:52:08.793 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.778+0000 7f7c16ffd700 1 -- 172.21.15.47:0/2900054562 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7c08012420 con 0x7f7c180630f0 2022-06-29T15:52:08.793 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.778+0000 7f7c16ffd700 1 --2- 172.21.15.47:0/2900054562 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7bf8053980 0x7f7bf8055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:08.793 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.778+0000 7f7c16ffd700 1 -- 172.21.15.47:0/2900054562 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f7c0806b9f0 con 0x7f7c180630f0 2022-06-29T15:52:08.793 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.778+0000 7f7c1d8c4700 1 --2- 172.21.15.47:0/2900054562 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7bf8053980 0x7f7bf8055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:08.793 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.778+0000 7f7c1d8c4700 1 --2- 172.21.15.47:0/2900054562 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7bf8053980 0x7f7bf8055e40 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7f7c1001b000 tx=0x7f7c10016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:08.794 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.786+0000 7f7c16ffd700 1 -- 172.21.15.47:0/2900054562 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7c0806de60 con 0x7f7c180630f0 2022-06-29T15:52:08.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:08.954+0000 7f7c1e8c6700 1 -- 172.21.15.47:0/2900054562 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_root_rctime"]} v 0) v1 -- 0x7f7c180573d0 con 0x7f7c180630f0 2022-06-29T15:52:09.375 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.370+0000 7f7c16ffd700 1 -- 172.21.15.47:0/2900054562 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_root_rctime"]}]=0 v446) v1 ==== 127+0+0 (secure 0 0 0) 0x7f7c08046460 con 0x7f7c180630f0 2022-06-29T15:52:09.380 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.374+0000 7f7c1e8c6700 1 -- 172.21.15.47:0/2900054562 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7bf8053980 msgr2=0x7f7bf8055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:09.380 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.374+0000 7f7c1e8c6700 1 --2- 172.21.15.47:0/2900054562 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7bf8053980 0x7f7bf8055e40 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7f7c1001b000 tx=0x7f7c10016000).stop 2022-06-29T15:52:09.380 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.374+0000 7f7c1e8c6700 1 -- 172.21.15.47:0/2900054562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c180630f0 msgr2=0x7f7c180acf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:09.380 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.374+0000 7f7c1e8c6700 1 --2- 172.21.15.47:0/2900054562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c180630f0 0x7f7c180acf80 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 rx=0x7f7c0800a000 tx=0x7f7c08008040).stop 2022-06-29T15:52:09.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.374+0000 7f7c1e8c6700 1 -- 172.21.15.47:0/2900054562 shutdown_connections 2022-06-29T15:52:09.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.374+0000 7f7c1e8c6700 1 --2- 172.21.15.47:0/2900054562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c180630f0 0x7f7c180acf80 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:09.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.374+0000 7f7c1e8c6700 1 --2- 172.21.15.47:0/2900054562 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7c18061da0 0x7f7c180a6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:09.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.374+0000 7f7c1e8c6700 1 --2- 172.21.15.47:0/2900054562 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7bf8053980 0x7f7bf8055e40 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:09.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.374+0000 7f7c1e8c6700 1 --2- 172.21.15.47:0/2900054562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c1805f0b0 0x7f7c180a41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:09.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.374+0000 7f7c1e8c6700 1 -- 172.21.15.47:0/2900054562 >> 172.21.15.47:0/2900054562 conn(0x7f7c18063670 msgr2=0x7f7c180b5580 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:09.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.374+0000 7f7c1e8c6700 1 -- 172.21.15.47:0/2900054562 shutdown_connections 2022-06-29T15:52:09.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.374+0000 7f7c1e8c6700 1 -- 172.21.15.47:0/2900054562 wait complete. 2022-06-29T15:52:09.394 INFO:tasks.cephfs_test_runner:test_root_rctime (tasks.cephfs.test_misc.TestMisc) ... ok 2022-06-29T15:52:09.395 INFO:tasks.cephfs_test_runner:Starting test: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2022-06-29T15:52:09.395 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_statfs_on_deleted_fs' 2022-06-29T15:52:09.500 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.494+0000 7f45363b0700 1 Processor -- start 2022-06-29T15:52:09.501 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.494+0000 7f45363b0700 1 -- start start 2022-06-29T15:52:09.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.494+0000 7f45363b0700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f45300b7840 0x7f45300b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:09.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.494+0000 7f45363b0700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4530062dd0 0x7f4530066c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:09.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.494+0000 7f45363b0700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45300b7c20 con 0x7f45300aea80 2022-06-29T15:52:09.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.494+0000 7f45363b0700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45300b3d10 con 0x7f45300b7840 2022-06-29T15:52:09.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.494+0000 7f45363b0700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45300670f0 con 0x7f4530062dd0 2022-06-29T15:52:09.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.494+0000 7f45353ae700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f45300b7840 0x7f45300b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:09.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.494+0000 7f45353ae700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f45300b7840 0x7f45300b3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54812/0 (socket says 172.21.15.47:54812) 2022-06-29T15:52:09.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.494+0000 7f45353ae700 1 -- 172.21.15.47:0/1540219707 learned_addr learned my addr 172.21.15.47:0/1540219707 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:09.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.494+0000 7f4534bad700 1 --2- 172.21.15.47:0/1540219707 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4530062dd0 0x7f4530066c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:09.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.494+0000 7f45353ae700 1 -- 172.21.15.47:0/1540219707 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4530062dd0 msgr2=0x7f4530066c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:09.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.494+0000 7f45353ae700 1 --2- 172.21.15.47:0/1540219707 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4530062dd0 0x7f4530066c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:09.507 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.494+0000 7f45353ae700 1 -- 172.21.15.47:0/1540219707 >> v1:172.21.15.64:6789/0 conn(0x7f45300aea80 legacy=0x7f4530141760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:09.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.494+0000 7f45353ae700 1 -- 172.21.15.47:0/1540219707 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4530067380 con 0x7f45300b7840 2022-06-29T15:52:09.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.494+0000 7f45353ae700 1 --2- 172.21.15.47:0/1540219707 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45300b7840 0x7f45300b3820 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 rx=0x7f4520002f70 tx=0x7f4520006eb0).ready entity=mon.0 client_cookie=4b8ef4107e38525f server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:09.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.494+0000 7f4527fff700 1 -- 172.21.15.47:0/1540219707 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f452000d570 con 0x7f45300b7840 2022-06-29T15:52:09.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.494+0000 7f4527fff700 1 -- 172.21.15.47:0/1540219707 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4520000e90 con 0x7f45300b7840 2022-06-29T15:52:09.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.494+0000 7f4527fff700 1 -- 172.21.15.47:0/1540219707 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f452000d900 con 0x7f45300b7840 2022-06-29T15:52:09.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.498+0000 7f45363b0700 1 -- 172.21.15.47:0/1540219707 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45300b7840 msgr2=0x7f45300b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:09.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.498+0000 7f45363b0700 1 --2- 172.21.15.47:0/1540219707 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45300b7840 0x7f45300b3820 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 rx=0x7f4520002f70 tx=0x7f4520006eb0).stop 2022-06-29T15:52:09.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.498+0000 7f45363b0700 1 -- 172.21.15.47:0/1540219707 shutdown_connections 2022-06-29T15:52:09.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.498+0000 7f45363b0700 1 --2- 172.21.15.47:0/1540219707 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4530062dd0 0x7f4530066c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:09.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.498+0000 7f45363b0700 1 --2- 172.21.15.47:0/1540219707 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45300b7840 0x7f45300b3820 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:09.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.498+0000 7f45363b0700 1 -- 172.21.15.47:0/1540219707 >> 172.21.15.47:0/1540219707 conn(0x7f453005f480 msgr2=0x7f45300647d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:09.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.498+0000 7f45363b0700 1 -- 172.21.15.47:0/1540219707 shutdown_connections 2022-06-29T15:52:09.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.498+0000 7f45363b0700 1 -- 172.21.15.47:0/1540219707 wait complete. 2022-06-29T15:52:09.514 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.498+0000 7f45363b0700 1 Processor -- start 2022-06-29T15:52:09.514 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.498+0000 7f45363b0700 1 -- start start 2022-06-29T15:52:09.514 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.498+0000 7f45363b0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4530062dd0 0x7f4530144e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:09.514 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.498+0000 7f45363b0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45300aea80 0x7f45300aa040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:09.515 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.498+0000 7f45363b0700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f45300b7840 0x7f45300aa530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:09.515 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.498+0000 7f45363b0700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45301435a0 con 0x7f45300aea80 2022-06-29T15:52:09.515 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.498+0000 7f45363b0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4530141ae0 con 0x7f4530062dd0 2022-06-29T15:52:09.515 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.498+0000 7f45363b0700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4530150d30 con 0x7f45300b7840 2022-06-29T15:52:09.516 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.498+0000 7f4534bad700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45300aea80 0x7f45300aa040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:09.516 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.498+0000 7f4534bad700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45300aea80 0x7f45300aa040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54816/0 (socket says 172.21.15.47:54816) 2022-06-29T15:52:09.516 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.498+0000 7f4534bad700 1 -- 172.21.15.47:0/388368259 learned_addr learned my addr 172.21.15.47:0/388368259 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:09.516 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.502+0000 7f45353ae700 1 --2- 172.21.15.47:0/388368259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4530062dd0 0x7f4530144e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:09.517 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.502+0000 7f4535baf700 1 --2- 172.21.15.47:0/388368259 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f45300b7840 0x7f45300aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:09.517 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.502+0000 7f4534bad700 1 -- 172.21.15.47:0/388368259 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f45300b7840 msgr2=0x7f45300aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:09.517 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.502+0000 7f4534bad700 1 --2- 172.21.15.47:0/388368259 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f45300b7840 0x7f45300aa530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:09.517 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.502+0000 7f4534bad700 1 -- 172.21.15.47:0/388368259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4530062dd0 msgr2=0x7f4530144e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:09.518 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.502+0000 7f4534bad700 1 --2- 172.21.15.47:0/388368259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4530062dd0 0x7f4530144e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:09.518 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.502+0000 7f4534bad700 1 -- 172.21.15.47:0/388368259 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f452000b1e0 con 0x7f45300aea80 2022-06-29T15:52:09.518 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.502+0000 7f4535baf700 1 --2- 172.21.15.47:0/388368259 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f45300b7840 0x7f45300aa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:52:09.518 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.502+0000 7f4534bad700 1 --2- 172.21.15.47:0/388368259 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45300aea80 0x7f45300aa040 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 rx=0x7f452c00c000 tx=0x7f452c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:09.518 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.502+0000 7f45267fc700 1 -- 172.21.15.47:0/388368259 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f452c010780 con 0x7f45300aea80 2022-06-29T15:52:09.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.502+0000 7f45267fc700 1 -- 172.21.15.47:0/388368259 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f452c0108e0 con 0x7f45300aea80 2022-06-29T15:52:09.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.502+0000 7f45363b0700 1 -- 172.21.15.47:0/388368259 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45300aaa20 con 0x7f45300aea80 2022-06-29T15:52:09.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.502+0000 7f45267fc700 1 -- 172.21.15.47:0/388368259 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f452c010c40 con 0x7f45300aea80 2022-06-29T15:52:09.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.502+0000 7f45363b0700 1 -- 172.21.15.47:0/388368259 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45301511f0 con 0x7f45300aea80 2022-06-29T15:52:09.520 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.502+0000 7f45363b0700 1 -- 172.21.15.47:0/388368259 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45300573d0 con 0x7f45300aea80 2022-06-29T15:52:09.520 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.502+0000 7f45267fc700 1 -- 172.21.15.47:0/388368259 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f452c01c460 con 0x7f45300aea80 2022-06-29T15:52:09.520 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.502+0000 7f45267fc700 1 --2- 172.21.15.47:0/388368259 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4518053980 0x7f4518055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:09.520 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.502+0000 7f45267fc700 1 -- 172.21.15.47:0/388368259 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f452c06c2f0 con 0x7f45300aea80 2022-06-29T15:52:09.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.502+0000 7f45353ae700 1 --2- 172.21.15.47:0/388368259 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4518053980 0x7f4518055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:09.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.506+0000 7f45353ae700 1 --2- 172.21.15.47:0/388368259 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4518053980 0x7f4518055e40 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7f452001a000 tx=0x7f4520015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:09.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.510+0000 7f45267fc700 1 -- 172.21.15.47:0/388368259 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f452c043290 con 0x7f45300aea80 2022-06-29T15:52:09.687 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:09.678+0000 7f45363b0700 1 -- 172.21.15.47:0/388368259 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_statfs_on_deleted_fs"]} v 0) v1 -- 0x7f4530068830 con 0x7f45300aea80 2022-06-29T15:52:10.378 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.370+0000 7f45267fc700 1 -- 172.21.15.47:0/388368259 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_statfs_on_deleted_fs"]}]=0 v447) v1 ==== 139+0+0 (secure 0 0 0) 0x7f452c0468b0 con 0x7f45300aea80 2022-06-29T15:52:10.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.374+0000 7f45363b0700 1 -- 172.21.15.47:0/388368259 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4518053980 msgr2=0x7f4518055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:10.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.374+0000 7f45363b0700 1 --2- 172.21.15.47:0/388368259 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4518053980 0x7f4518055e40 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7f452001a000 tx=0x7f4520015000).stop 2022-06-29T15:52:10.383 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.374+0000 7f45363b0700 1 -- 172.21.15.47:0/388368259 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45300aea80 msgr2=0x7f45300aa040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:10.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.374+0000 7f45363b0700 1 --2- 172.21.15.47:0/388368259 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45300aea80 0x7f45300aa040 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 rx=0x7f452c00c000 tx=0x7f452c006df0).stop 2022-06-29T15:52:10.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.374+0000 7f45363b0700 1 -- 172.21.15.47:0/388368259 shutdown_connections 2022-06-29T15:52:10.385 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.374+0000 7f45363b0700 1 --2- 172.21.15.47:0/388368259 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f45300b7840 0x7f45300aa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:10.385 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.374+0000 7f45363b0700 1 --2- 172.21.15.47:0/388368259 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45300aea80 0x7f45300aa040 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:10.385 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.374+0000 7f45363b0700 1 --2- 172.21.15.47:0/388368259 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4518053980 0x7f4518055e40 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:10.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.374+0000 7f45363b0700 1 --2- 172.21.15.47:0/388368259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4530062dd0 0x7f4530144e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:10.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.374+0000 7f45363b0700 1 -- 172.21.15.47:0/388368259 >> 172.21.15.47:0/388368259 conn(0x7f453005f480 msgr2=0x7f45300aac20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:10.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.374+0000 7f45363b0700 1 -- 172.21.15.47:0/388368259 shutdown_connections 2022-06-29T15:52:10.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.374+0000 7f45363b0700 1 -- 172.21.15.47:0/388368259 wait complete. 2022-06-29T15:52:10.398 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-06-29T15:52:10.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.514+0000 7f60790bc700 1 Processor -- start 2022-06-29T15:52:10.520 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.514+0000 7f60790bc700 1 -- start start 2022-06-29T15:52:10.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.514+0000 7f60790bc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f60740b1e80 0x7f60740b2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:10.522 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.514+0000 7f60790bc700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6074140ae0 con 0x7f60740b2bd0 2022-06-29T15:52:10.522 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.514+0000 7f60790bc700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6074146900 con 0x7f607409a060 2022-06-29T15:52:10.522 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.514+0000 7f60790bc700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6074146ba0 con 0x7f60740b1e80 2022-06-29T15:52:10.522 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.514+0000 7f60788bb700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f60740b1e80 0x7f60740b2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:10.523 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.514+0000 7f6073fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f60740b2bd0 0x7f6074099950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40378/0 (socket says 172.21.15.47:40378) 2022-06-29T15:52:10.523 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.514+0000 7f6073fff700 1 -- 172.21.15.47:0/27243242 learned_addr learned my addr 172.21.15.47:0/27243242 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:52:10.528 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.514+0000 7f60788bb700 1 -- 172.21.15.47:0/27243242 >> v1:172.21.15.64:6789/0 conn(0x7f607409a060 legacy=0x7f60740600c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:10.528 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.514+0000 7f60788bb700 1 -- 172.21.15.47:0/27243242 >> v1:172.21.15.47:6789/0 conn(0x7f60740b2bd0 legacy=0x7f6074099950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:10.529 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.514+0000 7f60788bb700 1 -- 172.21.15.47:0/27243242 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6074146e40 con 0x7f60740b1e80 2022-06-29T15:52:10.529 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.514+0000 7f60788bb700 1 --2- 172.21.15.47:0/27243242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60740b1e80 0x7f60740b2fb0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 rx=0x7f606c0032f0 tx=0x7f606c0091d0).ready entity=mon.0 client_cookie=6777e418c2622af5 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:10.529 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.514+0000 7f6072ffd700 1 -- 172.21.15.47:0/27243242 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f606c009ea0 con 0x7f60740b1e80 2022-06-29T15:52:10.529 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.514+0000 7f6072ffd700 1 -- 172.21.15.47:0/27243242 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f606c00b8a0 con 0x7f60740b1e80 2022-06-29T15:52:10.529 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.514+0000 7f6072ffd700 1 -- 172.21.15.47:0/27243242 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f606c00ba10 con 0x7f60740b1e80 2022-06-29T15:52:10.530 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.518+0000 7f60790bc700 1 -- 172.21.15.47:0/27243242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60740b1e80 msgr2=0x7f60740b2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:10.530 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.518+0000 7f60790bc700 1 --2- 172.21.15.47:0/27243242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60740b1e80 0x7f60740b2fb0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 rx=0x7f606c0032f0 tx=0x7f606c0091d0).stop 2022-06-29T15:52:10.530 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.518+0000 7f60790bc700 1 -- 172.21.15.47:0/27243242 shutdown_connections 2022-06-29T15:52:10.530 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.518+0000 7f60790bc700 1 --2- 172.21.15.47:0/27243242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60740b1e80 0x7f60740b2fb0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:10.531 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.518+0000 7f60790bc700 1 -- 172.21.15.47:0/27243242 >> 172.21.15.47:0/27243242 conn(0x7f6074062df0 msgr2=0x7f60740631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:10.531 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.518+0000 7f60790bc700 1 -- 172.21.15.47:0/27243242 shutdown_connections 2022-06-29T15:52:10.531 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.518+0000 7f60790bc700 1 -- 172.21.15.47:0/27243242 wait complete. 2022-06-29T15:52:10.531 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.518+0000 7f60790bc700 1 Processor -- start 2022-06-29T15:52:10.532 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.518+0000 7f60790bc700 1 -- start start 2022-06-29T15:52:10.532 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.518+0000 7f60790bc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f607409a060 0x7f60741439f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:10.532 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.518+0000 7f60790bc700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f60740b1e80 0x7f607409b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:10.532 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.518+0000 7f60790bc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60740b2bd0 0x7f60741429b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:10.532 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.518+0000 7f60790bc700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60741421a0 con 0x7f60740b2bd0 2022-06-29T15:52:10.533 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.518+0000 7f60790bc700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6074149020 con 0x7f607409a060 2022-06-29T15:52:10.533 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.518+0000 7f60790bc700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6074149210 con 0x7f60740b1e80 2022-06-29T15:52:10.533 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.518+0000 7f60788bb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60740b2bd0 0x7f60741429b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:10.533 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.518+0000 7f60788bb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60740b2bd0 0x7f60741429b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54834/0 (socket says 172.21.15.47:54834) 2022-06-29T15:52:10.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.518+0000 7f60788bb700 1 -- 172.21.15.47:0/2675171810 learned_addr learned my addr 172.21.15.47:0/2675171810 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:10.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.518+0000 7f6073fff700 1 --2- 172.21.15.47:0/2675171810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f607409a060 0x7f60741439f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:10.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.518+0000 7f60737fe700 1 --2- 172.21.15.47:0/2675171810 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f60740b1e80 0x7f607409b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:10.534 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.518+0000 7f60788bb700 1 -- 172.21.15.47:0/2675171810 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f60740b1e80 msgr2=0x7f607409b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:10.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.518+0000 7f60788bb700 1 --2- 172.21.15.47:0/2675171810 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f60740b1e80 0x7f607409b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:10.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.518+0000 7f60788bb700 1 -- 172.21.15.47:0/2675171810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f607409a060 msgr2=0x7f60741439f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:10.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.518+0000 7f60788bb700 1 --2- 172.21.15.47:0/2675171810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f607409a060 0x7f60741439f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:10.535 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.518+0000 7f60788bb700 1 -- 172.21.15.47:0/2675171810 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f606c009470 con 0x7f60740b2bd0 2022-06-29T15:52:10.536 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.518+0000 7f60788bb700 1 --2- 172.21.15.47:0/2675171810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60740b2bd0 0x7f60741429b0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 rx=0x7f606c0032f0 tx=0x7f606c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:10.536 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.518+0000 7f60717fa700 1 -- 172.21.15.47:0/2675171810 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f606c009ea0 con 0x7f60740b2bd0 2022-06-29T15:52:10.537 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.518+0000 7f60717fa700 1 -- 172.21.15.47:0/2675171810 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f606c01f3f0 con 0x7f60740b2bd0 2022-06-29T15:52:10.537 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.518+0000 7f60790bc700 1 -- 172.21.15.47:0/2675171810 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60741494b0 con 0x7f60740b2bd0 2022-06-29T15:52:10.538 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.518+0000 7f60717fa700 1 -- 172.21.15.47:0/2675171810 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f606c01f750 con 0x7f60740b2bd0 2022-06-29T15:52:10.538 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.518+0000 7f60790bc700 1 -- 172.21.15.47:0/2675171810 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6074148d20 con 0x7f60740b2bd0 2022-06-29T15:52:10.538 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.522+0000 7f60597fa700 1 -- 172.21.15.47:0/2675171810 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6050003410 con 0x7f60740b2bd0 2022-06-29T15:52:10.539 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.522+0000 7f60717fa700 1 -- 172.21.15.47:0/2675171810 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f606c026030 con 0x7f60740b2bd0 2022-06-29T15:52:10.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.522+0000 7f60717fa700 1 --2- 172.21.15.47:0/2675171810 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f60540539e0 0x7f6054055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:10.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.522+0000 7f60717fa700 1 -- 172.21.15.47:0/2675171810 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f606c06cb00 con 0x7f60740b2bd0 2022-06-29T15:52:10.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.522+0000 7f6073fff700 1 --2- 172.21.15.47:0/2675171810 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f60540539e0 0x7f6054055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:10.540 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.522+0000 7f6073fff700 1 --2- 172.21.15.47:0/2675171810 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f60540539e0 0x7f6054055ea0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7f606400d000 tx=0x7f6064007be0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:10.541 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.530+0000 7f60717fa700 1 -- 172.21.15.47:0/2675171810 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f606c047530 con 0x7f60740b2bd0 2022-06-29T15:52:10.664 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.658+0000 7f60597fa700 1 -- 172.21.15.47:0/2675171810 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f6050003c40 con 0x7f60740b2bd0 2022-06-29T15:52:10.669 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.662+0000 7f60717fa700 1 -- 172.21.15.47:0/2675171810 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7f606c024020 con 0x7f60740b2bd0 2022-06-29T15:52:10.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.666+0000 7f60597fa700 1 -- 172.21.15.47:0/2675171810 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f60540539e0 msgr2=0x7f6054055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:10.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.666+0000 7f60597fa700 1 --2- 172.21.15.47:0/2675171810 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f60540539e0 0x7f6054055ea0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7f606400d000 tx=0x7f6064007be0).stop 2022-06-29T15:52:10.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.666+0000 7f60597fa700 1 -- 172.21.15.47:0/2675171810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60740b2bd0 msgr2=0x7f60741429b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:10.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.666+0000 7f60597fa700 1 --2- 172.21.15.47:0/2675171810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60740b2bd0 0x7f60741429b0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 rx=0x7f606c0032f0 tx=0x7f606c016000).stop 2022-06-29T15:52:10.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.666+0000 7f60597fa700 1 -- 172.21.15.47:0/2675171810 shutdown_connections 2022-06-29T15:52:10.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.666+0000 7f60597fa700 1 --2- 172.21.15.47:0/2675171810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60740b2bd0 0x7f60741429b0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:10.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.666+0000 7f60597fa700 1 --2- 172.21.15.47:0/2675171810 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f60740b1e80 0x7f607409b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:10.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.666+0000 7f60597fa700 1 --2- 172.21.15.47:0/2675171810 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f60540539e0 0x7f6054055ea0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:10.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.666+0000 7f60597fa700 1 --2- 172.21.15.47:0/2675171810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f607409a060 0x7f60741439f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:10.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.666+0000 7f60597fa700 1 -- 172.21.15.47:0/2675171810 >> 172.21.15.47:0/2675171810 conn(0x7f6074062df0 msgr2=0x7f6074144800 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:10.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.666+0000 7f60597fa700 1 -- 172.21.15.47:0/2675171810 shutdown_connections 2022-06-29T15:52:10.676 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.666+0000 7f60597fa700 1 -- 172.21.15.47:0/2675171810 wait complete. 2022-06-29T15:52:10.687 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-06-29T15:52:10.688 DEBUG:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:52:10.704 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:52:10.726 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:52:10.748 DEBUG:teuthology.orchestra.run.smithi164:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T15:52:10.769 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-06-29T15:52:10.770 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:52:10.770 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:52:10.771 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:52:10.778 INFO:teuthology.orchestra.run.smithi099.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-06-29T15:52:10.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T15:52:10.780 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-06-29T15:52:10.780 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:52:10.781 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:52:10.781 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:52:10.788 INFO:teuthology.orchestra.run.smithi164.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-06-29T15:52:10.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T15:52:10.789 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-06-29T15:52:10.790 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-06-29T15:52:10.903 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.894+0000 7fdedc8de700 1 Processor -- start 2022-06-29T15:52:10.903 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.894+0000 7fdedc8de700 1 -- start start 2022-06-29T15:52:10.908 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.898+0000 7fdedc8de700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fded80b2120 0x7fded80b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:10.908 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.898+0000 7fdedc8de700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fded805f0b0 0x7fded814b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:10.908 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.898+0000 7fdedc8de700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fded80b0740 con 0x7fded80b1260 2022-06-29T15:52:10.909 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.898+0000 7fdedc8de700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fded8147460 con 0x7fded80b2120 2022-06-29T15:52:10.909 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.898+0000 7fdedc8de700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fded80b0970 con 0x7fded805f0b0 2022-06-29T15:52:10.909 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.898+0000 7fded659c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fded80b2120 0x7fded80b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:10.909 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.898+0000 7fded6d9d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fded80b1260 0x7fded8147af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40396/0 (socket says 172.21.15.47:40396) 2022-06-29T15:52:10.910 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.898+0000 7fded6d9d700 1 -- 172.21.15.47:0/2704104832 learned_addr learned my addr 172.21.15.47:0/2704104832 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:52:10.910 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.898+0000 7fded5d9b700 1 --2- 172.21.15.47:0/2704104832 >> v2:172.21.15.64:3300/0 conn(0x7fded805f0b0 0x7fded814b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:10.910 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.898+0000 7fded659c700 1 -- 172.21.15.47:0/2704104832 >> v2:172.21.15.64:3300/0 conn(0x7fded805f0b0 msgr2=0x7fded814b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:10.910 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.898+0000 7fded659c700 1 --2- 172.21.15.47:0/2704104832 >> v2:172.21.15.64:3300/0 conn(0x7fded805f0b0 0x7fded814b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:10.911 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.898+0000 7fded659c700 1 -- 172.21.15.47:0/2704104832 >> v1:172.21.15.47:6789/0 conn(0x7fded80b1260 legacy=0x7fded8147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:10.911 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.898+0000 7fded559a700 1 -- 172.21.15.47:0/2704104832 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 767109306 0 0) 0x7fded80b0740 con 0x7fded80b1260 2022-06-29T15:52:10.911 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.898+0000 7fded659c700 1 -- 172.21.15.47:0/2704104832 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded80b0c10 con 0x7fded80b2120 2022-06-29T15:52:10.911 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.898+0000 7fded659c700 1 --2- 172.21.15.47:0/2704104832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fded80b2120 0x7fded80b2500 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 rx=0x7fded8060d70 tx=0x7fdecc006df0).ready entity=mon.0 client_cookie=8268222efb9a3fe5 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:10.912 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.898+0000 7fded559a700 1 -- 172.21.15.47:0/2704104832 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdecc00d5c0 con 0x7fded80b2120 2022-06-29T15:52:10.912 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.898+0000 7fded559a700 1 -- 172.21.15.47:0/2704104832 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdecc000e90 con 0x7fded80b2120 2022-06-29T15:52:10.912 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.898+0000 7fded559a700 1 -- 172.21.15.47:0/2704104832 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdecc00d910 con 0x7fded80b2120 2022-06-29T15:52:10.912 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.898+0000 7fdedc8de700 1 -- 172.21.15.47:0/2704104832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fded80b2120 msgr2=0x7fded80b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:10.913 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.898+0000 7fdedc8de700 1 --2- 172.21.15.47:0/2704104832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fded80b2120 0x7fded80b2500 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 rx=0x7fded8060d70 tx=0x7fdecc006df0).stop 2022-06-29T15:52:10.913 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.898+0000 7fdedc8de700 1 -- 172.21.15.47:0/2704104832 shutdown_connections 2022-06-29T15:52:10.913 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.898+0000 7fdedc8de700 1 --2- 172.21.15.47:0/2704104832 >> v2:172.21.15.64:3300/0 conn(0x7fded805f0b0 0x7fded814b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:10.913 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.898+0000 7fdedc8de700 1 --2- 172.21.15.47:0/2704104832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fded80b2120 0x7fded80b2500 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:10.914 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.898+0000 7fdedc8de700 1 -- 172.21.15.47:0/2704104832 >> 172.21.15.47:0/2704104832 conn(0x7fded80626f0 msgr2=0x7fded8062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:10.914 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.898+0000 7fdedc8de700 1 -- 172.21.15.47:0/2704104832 shutdown_connections 2022-06-29T15:52:10.914 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.898+0000 7fdedc8de700 1 -- 172.21.15.47:0/2704104832 wait complete. 2022-06-29T15:52:10.914 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.902+0000 7fdedc8de700 1 Processor -- start 2022-06-29T15:52:10.915 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.902+0000 7fdedc8de700 1 -- start start 2022-06-29T15:52:10.920 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.902+0000 7fdedc8de700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fded80b1260 0x7fded814ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:10.920 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.902+0000 7fdedc8de700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fded80b2120 0x7fded8157100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:10.921 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.902+0000 7fdedc8de700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fded81406f0 0x7fded8140ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:10.921 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.902+0000 7fdedc8de700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded81575f0 con 0x7fded80b1260 2022-06-29T15:52:10.921 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.902+0000 7fdedc8de700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded8144cd0 con 0x7fded81406f0 2022-06-29T15:52:10.922 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.902+0000 7fdedc8de700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fded8144270 con 0x7fded80b2120 2022-06-29T15:52:10.922 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.902+0000 7fded659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fded80b1260 0x7fded814ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:10.923 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.902+0000 7fded659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fded80b1260 0x7fded814ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54844/0 (socket says 172.21.15.47:54844) 2022-06-29T15:52:10.923 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.902+0000 7fded659c700 1 -- 172.21.15.47:0/2741161898 learned_addr learned my addr 172.21.15.47:0/2741161898 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:10.923 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.902+0000 7fded6d9d700 1 --2- 172.21.15.47:0/2741161898 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fded81406f0 0x7fded8140ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:10.924 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.902+0000 7fded5d9b700 1 --2- 172.21.15.47:0/2741161898 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fded80b2120 0x7fded8157100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:10.924 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.902+0000 7fded659c700 1 -- 172.21.15.47:0/2741161898 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fded80b2120 msgr2=0x7fded8157100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:10.925 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.902+0000 7fded659c700 1 --2- 172.21.15.47:0/2741161898 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fded80b2120 0x7fded8157100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:10.925 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.902+0000 7fded659c700 1 -- 172.21.15.47:0/2741161898 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fded81406f0 msgr2=0x7fded8140ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:10.925 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.902+0000 7fded659c700 1 --2- 172.21.15.47:0/2741161898 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fded81406f0 0x7fded8140ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:10.926 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.902+0000 7fded659c700 1 -- 172.21.15.47:0/2741161898 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdecc00b180 con 0x7fded80b1260 2022-06-29T15:52:10.926 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.902+0000 7fded6d9d700 1 --2- 172.21.15.47:0/2741161898 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fded81406f0 0x7fded8140ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:52:10.927 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.902+0000 7fded5d9b700 1 --2- 172.21.15.47:0/2741161898 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fded80b2120 0x7fded8157100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:52:10.927 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.902+0000 7fded659c700 1 --2- 172.21.15.47:0/2741161898 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fded80b1260 0x7fded814ded0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 rx=0x7fded8060d70 tx=0x7fdecc014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:10.927 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.902+0000 7fdec77fe700 1 -- 172.21.15.47:0/2741161898 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdecc016070 con 0x7fded80b1260 2022-06-29T15:52:10.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.902+0000 7fdec77fe700 1 -- 172.21.15.47:0/2741161898 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdecc00de40 con 0x7fded80b1260 2022-06-29T15:52:10.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.902+0000 7fdedc8de700 1 -- 172.21.15.47:0/2741161898 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fded8144510 con 0x7fded80b1260 2022-06-29T15:52:10.928 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.902+0000 7fdec77fe700 1 -- 172.21.15.47:0/2741161898 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdecc01e450 con 0x7fded80b1260 2022-06-29T15:52:10.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.902+0000 7fdedc8de700 1 -- 172.21.15.47:0/2741161898 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fded8143ef0 con 0x7fded80b1260 2022-06-29T15:52:10.929 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.906+0000 7fdedc8de700 1 -- 172.21.15.47:0/2741161898 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fded81449b0 con 0x7fded80b1260 2022-06-29T15:52:10.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.906+0000 7fdec77fe700 1 -- 172.21.15.47:0/2741161898 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fdecc025030 con 0x7fded80b1260 2022-06-29T15:52:10.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.906+0000 7fdec77fe700 1 --2- 172.21.15.47:0/2741161898 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdeb4053980 0x7fdeb4055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:10.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.906+0000 7fdec77fe700 1 -- 172.21.15.47:0/2741161898 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fdecc06cc10 con 0x7fded80b1260 2022-06-29T15:52:10.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.906+0000 7fded5d9b700 1 --2- 172.21.15.47:0/2741161898 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdeb4053980 0x7fdeb4055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:10.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.906+0000 7fded5d9b700 1 --2- 172.21.15.47:0/2741161898 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdeb4053980 0x7fdeb4055e40 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7fdec800b000 tx=0x7fdec8008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:10.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:10.914+0000 7fdec77fe700 1 -- 172.21.15.47:0/2741161898 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdecc073080 con 0x7fded80b1260 2022-06-29T15:52:11.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.038+0000 7fdedc8de700 1 -- 172.21.15.47:0/2741161898 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fded80573d0 con 0x7fded80b1260 2022-06-29T15:52:11.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.042+0000 7fdec77fe700 1 -- 172.21.15.47:0/2741161898 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v154) v1 ==== 81+0+29229 (secure 0 0 0) 0x7fdecc023020 con 0x7fded80b1260 2022-06-29T15:52:11.049 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:52:11.049 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2022-06-29T15:52:11.050 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 154, 2022-06-29T15:52:11.050 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "9a68894a-53ce-400e-838b-14eab91b834c", 2022-06-29T15:52:11.050 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2022-06-29T15:42:14.046361+0000", 2022-06-29T15:52:11.051 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2022-06-29T15:52:08.351387+0000", 2022-06-29T15:52:11.051 INFO:teuthology.orchestra.run.smithi047.stdout: "last_up_change": "2022-06-29T15:42:20.929789+0000", 2022-06-29T15:52:11.051 INFO:teuthology.orchestra.run.smithi047.stdout: "last_in_change": "2022-06-29T15:42:20.929789+0000", 2022-06-29T15:52:11.051 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-06-29T15:52:11.051 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_num": 5799936, 2022-06-29T15:52:11.052 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_set": [ 2022-06-29T15:52:11.052 INFO:teuthology.orchestra.run.smithi047.stdout: "pglog_hardlimit", 2022-06-29T15:52:11.052 INFO:teuthology.orchestra.run.smithi047.stdout: "purged_snapdirs", 2022-06-29T15:52:11.052 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_deletes", 2022-06-29T15:52:11.053 INFO:teuthology.orchestra.run.smithi047.stdout: "sortbitwise" 2022-06-29T15:52:11.053 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:52:11.053 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_version": 5, 2022-06-29T15:52:11.053 INFO:teuthology.orchestra.run.smithi047.stdout: "full_ratio": 0.94999998807907104, 2022-06-29T15:52:11.054 INFO:teuthology.orchestra.run.smithi047.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-06-29T15:52:11.054 INFO:teuthology.orchestra.run.smithi047.stdout: "nearfull_ratio": 0.85000002384185791, 2022-06-29T15:52:11.054 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_snapshot": "", 2022-06-29T15:52:11.054 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_max": 28, 2022-06-29T15:52:11.055 INFO:teuthology.orchestra.run.smithi047.stdout: "max_osd": 8, 2022-06-29T15:52:11.055 INFO:teuthology.orchestra.run.smithi047.stdout: "require_min_compat_client": "jewel", 2022-06-29T15:52:11.055 INFO:teuthology.orchestra.run.smithi047.stdout: "min_compat_client": "jewel", 2022-06-29T15:52:11.055 INFO:teuthology.orchestra.run.smithi047.stdout: "require_osd_release": "octopus", 2022-06-29T15:52:11.056 INFO:teuthology.orchestra.run.smithi047.stdout: "pools": [ 2022-06-29T15:52:11.056 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.056 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 1, 2022-06-29T15:52:11.056 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "device_health_metrics", 2022-06-29T15:52:11.057 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:42:16.971431+0000", 2022-06-29T15:52:11.057 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:52:11.057 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:52:11.057 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:52:11.058 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:52:11.058 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:52:11.058 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:52:11.059 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:52:11.059 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:52:11.059 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 1, 2022-06-29T15:52:11.059 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 1, 2022-06-29T15:52:11.060 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 1, 2022-06-29T15:52:11.060 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 1, 2022-06-29T15:52:11.060 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 1, 2022-06-29T15:52:11.060 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:52:11.060 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:52:11.061 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:52:11.061 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:52:11.061 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:52:11.061 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:52:11.062 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:52:11.062 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.062 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "16", 2022-06-29T15:52:11.062 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:52:11.063 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:52:11.063 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:52:11.063 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:52:11.063 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:52:11.064 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:52:11.064 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:52:11.064 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:52:11.064 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:52:11.064 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:52:11.065 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:52:11.065 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:52:11.065 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:52:11.065 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:52:11.066 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:52:11.066 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:52:11.066 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:52:11.066 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:52:11.067 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:52:11.067 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:52:11.067 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:52:11.067 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:52:11.067 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:52:11.068 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:52:11.068 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:52:11.068 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:52:11.068 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.069 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:52:11.069 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:52:11.069 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:52:11.069 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:52:11.070 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:52:11.070 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:52:11.070 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:52:11.071 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:52:11.071 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:52:11.071 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:52:11.071 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:52:11.072 INFO:teuthology.orchestra.run.smithi047.stdout: "options": { 2022-06-29T15:52:11.072 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_min": 1 2022-06-29T15:52:11.072 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.072 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:52:11.073 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_devicehealth": {} 2022-06-29T15:52:11.073 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:11.073 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.073 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.074 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 2, 2022-06-29T15:52:11.074 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "rbd", 2022-06-29T15:52:11.074 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:42:25.290884+0000", 2022-06-29T15:52:11.074 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:52:11.074 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:52:11.075 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:52:11.075 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:52:11.075 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:52:11.075 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:52:11.076 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:52:11.076 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:52:11.076 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2022-06-29T15:52:11.076 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2022-06-29T15:52:11.076 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2022-06-29T15:52:11.077 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2022-06-29T15:52:11.077 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2022-06-29T15:52:11.077 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:52:11.077 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:52:11.078 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:52:11.078 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:52:11.078 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:52:11.078 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:52:11.078 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:52:11.079 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.079 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "19", 2022-06-29T15:52:11.079 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:52:11.079 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:52:11.080 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:52:11.080 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:52:11.080 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:52:11.080 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:52:11.081 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:52:11.081 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:52:11.081 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:52:11.081 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:52:11.082 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:52:11.082 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:52:11.082 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:52:11.082 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:52:11.083 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:52:11.083 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:52:11.083 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:52:11.083 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:52:11.084 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:52:11.084 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:52:11.084 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:52:11.084 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:52:11.085 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:52:11.085 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:52:11.085 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:52:11.085 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:52:11.085 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.086 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:52:11.086 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:52:11.086 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:52:11.087 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:52:11.087 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:52:11.087 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:52:11.087 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:52:11.088 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:52:11.088 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:52:11.088 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:52:11.088 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:52:11.088 INFO:teuthology.orchestra.run.smithi047.stdout: "options": {}, 2022-06-29T15:52:11.089 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:52:11.089 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd": {} 2022-06-29T15:52:11.089 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:11.089 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:11.090 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:52:11.090 INFO:teuthology.orchestra.run.smithi047.stdout: "osds": [ 2022-06-29T15:52:11.090 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.090 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2022-06-29T15:52:11.090 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "e23eb023-4706-4201-bf72-4bf19c912c5d", 2022-06-29T15:52:11.091 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:52:11.091 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:52:11.091 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:52:11.091 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:52:11.092 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:52:11.092 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:52:11.092 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:52:11.092 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 149, 2022-06-29T15:52:11.092 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:52:11.093 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:52:11.093 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:52:11.093 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:11.093 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.094 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:11.094 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6818", 2022-06-29T15:52:11.094 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:52:11.094 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.095 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.095 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:11.095 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6819", 2022-06-29T15:52:11.095 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:52:11.095 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:11.096 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:11.096 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.096 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:52:11.096 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:11.097 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.097 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:11.097 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6820", 2022-06-29T15:52:11.097 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:52:11.097 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.098 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.098 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:11.098 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6821", 2022-06-29T15:52:11.099 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:52:11.099 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:11.099 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:11.099 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.100 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:52:11.100 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:11.100 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.100 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:11.101 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6824", 2022-06-29T15:52:11.101 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:52:11.101 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.101 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.102 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:11.102 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6825", 2022-06-29T15:52:11.103 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:52:11.103 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:11.104 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:11.104 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.104 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:52:11.105 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:11.105 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.105 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:11.106 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6822", 2022-06-29T15:52:11.106 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:52:11.106 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.106 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.107 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:11.107 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6823", 2022-06-29T15:52:11.107 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:52:11.107 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:11.108 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:11.108 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.108 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6819/13096", 2022-06-29T15:52:11.108 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6821/13096", 2022-06-29T15:52:11.109 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6825/13096", 2022-06-29T15:52:11.109 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6823/13096", 2022-06-29T15:52:11.109 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:52:11.109 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:52:11.109 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:52:11.110 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:11.110 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.110 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.110 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2022-06-29T15:52:11.111 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "228bb08d-2d64-43d9-9560-bfaeff154452", 2022-06-29T15:52:11.111 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:52:11.111 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:52:11.111 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:52:11.112 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:52:11.112 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:52:11.112 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:52:11.112 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:52:11.112 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 149, 2022-06-29T15:52:11.113 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:52:11.113 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:52:11.113 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:52:11.113 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:11.114 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.114 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:11.114 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6826", 2022-06-29T15:52:11.114 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:52:11.114 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.115 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.115 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:11.115 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6827", 2022-06-29T15:52:11.115 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:52:11.116 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:11.116 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:11.116 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.117 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:52:11.117 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:11.117 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.118 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:11.118 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6828", 2022-06-29T15:52:11.119 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:52:11.119 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.120 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.120 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:11.120 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6829", 2022-06-29T15:52:11.121 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:52:11.121 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:11.122 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:11.122 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.123 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:52:11.123 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:11.124 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.124 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:11.125 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6832", 2022-06-29T15:52:11.125 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:52:11.125 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.126 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.126 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:11.127 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6833", 2022-06-29T15:52:11.127 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:52:11.127 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:11.128 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:11.128 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.129 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:52:11.129 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:11.129 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.129 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:11.130 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6830", 2022-06-29T15:52:11.130 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:52:11.130 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.131 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.131 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:11.131 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6831", 2022-06-29T15:52:11.131 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:52:11.132 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:11.132 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:11.132 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.132 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6827/13100", 2022-06-29T15:52:11.132 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6829/13100", 2022-06-29T15:52:11.133 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6833/13100", 2022-06-29T15:52:11.133 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6831/13100", 2022-06-29T15:52:11.133 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:52:11.133 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:52:11.134 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:52:11.134 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:11.134 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.134 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.135 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2022-06-29T15:52:11.135 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "797a5452-1510-4640-a111-c9c2fcf707eb", 2022-06-29T15:52:11.135 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:52:11.135 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:52:11.135 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:52:11.136 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:52:11.136 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:52:11.136 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:52:11.136 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:52:11.137 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 148, 2022-06-29T15:52:11.137 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:52:11.137 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:52:11.137 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:52:11.137 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:11.138 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.138 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:11.138 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6810", 2022-06-29T15:52:11.138 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:52:11.139 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.139 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.139 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:11.139 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6811", 2022-06-29T15:52:11.139 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:52:11.140 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:11.140 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:11.140 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.140 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:52:11.141 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:11.141 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.141 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:11.141 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6812", 2022-06-29T15:52:11.142 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:52:11.142 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.142 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.142 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:11.142 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6813", 2022-06-29T15:52:11.143 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:52:11.143 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:11.143 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:11.143 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.144 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:52:11.144 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:11.144 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.144 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:11.145 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6816", 2022-06-29T15:52:11.145 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:52:11.145 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.145 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.145 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:11.146 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6817", 2022-06-29T15:52:11.146 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:52:11.146 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:11.147 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:11.147 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.147 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:52:11.148 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:11.148 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.149 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:11.149 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6814", 2022-06-29T15:52:11.150 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:52:11.150 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.150 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.151 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:11.151 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6815", 2022-06-29T15:52:11.152 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:52:11.152 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:11.152 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:11.153 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.153 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6811/13098", 2022-06-29T15:52:11.153 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6813/13098", 2022-06-29T15:52:11.154 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6817/13098", 2022-06-29T15:52:11.154 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6815/13098", 2022-06-29T15:52:11.154 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:52:11.155 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:52:11.155 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:52:11.155 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:11.156 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.156 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.156 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2022-06-29T15:52:11.157 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "55b24fd7-bb71-4a01-b3fa-a09caa772a62", 2022-06-29T15:52:11.157 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:52:11.157 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:52:11.157 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:52:11.158 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:52:11.158 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:52:11.158 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:52:11.159 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:52:11.159 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 137, 2022-06-29T15:52:11.159 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:52:11.159 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:52:11.160 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:52:11.160 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:11.160 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.160 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:11.160 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6802", 2022-06-29T15:52:11.161 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:52:11.161 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.161 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.161 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:11.162 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6803", 2022-06-29T15:52:11.162 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:52:11.162 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:11.162 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:11.162 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.163 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:52:11.167 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:11.168 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.168 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:11.168 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6804", 2022-06-29T15:52:11.168 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:52:11.169 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.169 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.169 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:11.169 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6805", 2022-06-29T15:52:11.170 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:52:11.170 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:11.170 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:11.170 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.171 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:52:11.171 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:11.172 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.172 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:11.172 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6808", 2022-06-29T15:52:11.172 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:52:11.173 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.173 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.173 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:11.173 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6809", 2022-06-29T15:52:11.173 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:52:11.174 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:11.174 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:11.174 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.175 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:52:11.175 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:11.175 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.175 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:11.175 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6806", 2022-06-29T15:52:11.176 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:52:11.176 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.176 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.176 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:11.177 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6807", 2022-06-29T15:52:11.177 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:52:11.177 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:11.177 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:11.177 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.178 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6803/13097", 2022-06-29T15:52:11.178 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6805/13097", 2022-06-29T15:52:11.178 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6809/13097", 2022-06-29T15:52:11.178 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6807/13097", 2022-06-29T15:52:11.179 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:52:11.179 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:52:11.179 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:52:11.179 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:11.180 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.180 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.180 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2022-06-29T15:52:11.180 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3", 2022-06-29T15:52:11.181 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:52:11.181 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:52:11.181 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:52:11.182 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:52:11.182 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:52:11.182 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:52:11.182 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:52:11.183 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 148, 2022-06-29T15:52:11.183 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:52:11.183 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:52:11.183 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:52:11.183 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:11.184 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.184 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:11.184 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6816", 2022-06-29T15:52:11.184 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:52:11.185 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.185 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.185 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:11.185 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6817", 2022-06-29T15:52:11.186 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:52:11.186 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:11.186 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:11.187 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.187 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:52:11.187 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:11.187 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.187 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:11.188 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6818", 2022-06-29T15:52:11.188 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:52:11.188 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.188 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.189 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:11.189 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6819", 2022-06-29T15:52:11.189 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:52:11.189 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:11.190 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:11.190 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.190 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:52:11.190 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:11.190 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.191 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:11.191 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6822", 2022-06-29T15:52:11.191 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:52:11.191 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.192 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.192 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:11.192 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6823", 2022-06-29T15:52:11.192 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:52:11.193 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:11.193 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:11.193 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.193 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:52:11.193 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:11.194 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.194 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:11.194 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6820", 2022-06-29T15:52:11.194 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:52:11.195 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.195 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.195 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:11.195 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6821", 2022-06-29T15:52:11.196 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:52:11.196 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:11.196 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:11.196 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.196 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6817/13396", 2022-06-29T15:52:11.197 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6819/13396", 2022-06-29T15:52:11.197 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6823/13396", 2022-06-29T15:52:11.197 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6821/13396", 2022-06-29T15:52:11.197 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:52:11.198 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:52:11.198 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:52:11.198 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:11.198 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.199 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.199 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2022-06-29T15:52:11.199 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "f77beebe-6cca-4244-b2d4-677825ebf919", 2022-06-29T15:52:11.199 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:52:11.199 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:52:11.200 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:52:11.200 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:52:11.200 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:52:11.200 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:52:11.201 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:52:11.201 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 149, 2022-06-29T15:52:11.201 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:52:11.201 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:52:11.201 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:52:11.202 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:11.202 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.202 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:11.202 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6800", 2022-06-29T15:52:11.203 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:52:11.203 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.203 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.203 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:11.203 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6801", 2022-06-29T15:52:11.204 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:52:11.204 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:11.204 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:11.204 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.205 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:52:11.205 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:11.205 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.205 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:11.206 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6802", 2022-06-29T15:52:11.206 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:52:11.206 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.206 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.207 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:11.207 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6803", 2022-06-29T15:52:11.207 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:52:11.207 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:11.208 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:11.208 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.208 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:52:11.208 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:11.209 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.209 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:11.209 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6806", 2022-06-29T15:52:11.209 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:52:11.209 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.210 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.210 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:11.210 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6807", 2022-06-29T15:52:11.210 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:52:11.211 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:11.211 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:11.211 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.211 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:52:11.212 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:11.212 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.212 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:11.212 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6804", 2022-06-29T15:52:11.213 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:52:11.213 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.213 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.213 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:11.213 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6805", 2022-06-29T15:52:11.214 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:52:11.214 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:11.214 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:11.214 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.215 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6801/13393", 2022-06-29T15:52:11.215 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6803/13393", 2022-06-29T15:52:11.215 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6807/13393", 2022-06-29T15:52:11.215 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6805/13393", 2022-06-29T15:52:11.216 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:52:11.216 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:52:11.216 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:52:11.216 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:11.217 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.217 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.217 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2022-06-29T15:52:11.217 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "af0ca9b3-c02c-4910-834b-bf0b8dc6713c", 2022-06-29T15:52:11.217 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:52:11.218 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:52:11.218 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:52:11.218 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:52:11.218 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:52:11.219 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:52:11.219 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:52:11.219 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 149, 2022-06-29T15:52:11.220 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:52:11.220 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:52:11.220 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:52:11.220 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:11.221 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.221 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:11.221 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6824", 2022-06-29T15:52:11.221 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:52:11.221 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.222 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.222 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:11.222 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6825", 2022-06-29T15:52:11.222 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:52:11.223 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:11.223 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:11.223 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.223 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:52:11.224 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:11.224 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.224 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:11.224 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6826", 2022-06-29T15:52:11.224 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:52:11.225 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.225 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.225 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:11.225 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6827", 2022-06-29T15:52:11.226 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:52:11.226 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:11.226 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:11.226 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.227 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:52:11.227 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:11.227 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.227 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:11.227 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6830", 2022-06-29T15:52:11.228 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:52:11.228 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.228 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.228 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:11.229 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6831", 2022-06-29T15:52:11.229 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:52:11.229 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:11.229 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:11.230 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.230 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:52:11.230 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:11.230 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.230 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:11.231 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6828", 2022-06-29T15:52:11.231 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:52:11.231 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.232 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.232 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:11.232 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6829", 2022-06-29T15:52:11.232 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:52:11.233 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:11.233 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:11.233 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.233 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6825/13395", 2022-06-29T15:52:11.233 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6827/13395", 2022-06-29T15:52:11.234 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6831/13395", 2022-06-29T15:52:11.234 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6829/13395", 2022-06-29T15:52:11.234 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:52:11.234 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:52:11.235 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:52:11.235 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:11.235 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.235 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.236 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2022-06-29T15:52:11.236 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "f19c3ca6-df88-4bde-a150-b51a53f092e1", 2022-06-29T15:52:11.236 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:52:11.236 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:52:11.237 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:52:11.237 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:52:11.237 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:52:11.237 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:52:11.237 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:52:11.238 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 149, 2022-06-29T15:52:11.238 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:52:11.238 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:52:11.238 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:52:11.239 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:11.239 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.239 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:11.239 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6808", 2022-06-29T15:52:11.239 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:52:11.240 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.240 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.240 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:11.240 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6809", 2022-06-29T15:52:11.241 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:52:11.241 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:11.241 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:11.241 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.242 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:52:11.242 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:11.242 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.242 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:11.243 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6810", 2022-06-29T15:52:11.243 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:52:11.243 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.243 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.243 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:11.244 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6811", 2022-06-29T15:52:11.244 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:52:11.244 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:11.244 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:11.245 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.245 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:52:11.245 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:11.245 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.246 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:11.246 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6814", 2022-06-29T15:52:11.246 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:52:11.246 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.247 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.247 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:11.247 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6815", 2022-06-29T15:52:11.248 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:52:11.248 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:11.248 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:11.248 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.249 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:52:11.249 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:11.249 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.249 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:11.249 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6812", 2022-06-29T15:52:11.250 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:52:11.250 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.250 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.250 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:11.251 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6813", 2022-06-29T15:52:11.251 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:52:11.251 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:11.251 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:11.252 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.252 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6809/13394", 2022-06-29T15:52:11.252 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6811/13394", 2022-06-29T15:52:11.252 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6815/13394", 2022-06-29T15:52:11.253 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6813/13394", 2022-06-29T15:52:11.253 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:52:11.253 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:52:11.253 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:52:11.253 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:11.254 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:11.254 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:52:11.254 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_xinfo": [ 2022-06-29T15:52:11.255 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.255 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2022-06-29T15:52:11.255 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:52:11.255 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:52:11.255 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:52:11.256 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:52:11.256 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:52:11.256 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.720513+0000", 2022-06-29T15:52:11.256 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:52:11.257 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.257 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.257 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2022-06-29T15:52:11.257 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:52:11.258 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:52:11.258 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:52:11.258 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:52:11.258 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:52:11.259 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.772612+0000", 2022-06-29T15:52:11.259 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:52:11.260 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.260 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.260 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2022-06-29T15:52:11.261 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:52:11.261 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:52:11.261 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:52:11.261 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:52:11.262 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:52:11.262 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.450223+0000", 2022-06-29T15:52:11.262 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:52:11.262 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.262 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.263 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2022-06-29T15:52:11.263 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:52:11.263 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:52:11.263 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:52:11.264 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:52:11.264 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:52:11.264 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.477856+0000", 2022-06-29T15:52:11.264 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:52:11.264 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.265 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.265 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2022-06-29T15:52:11.265 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:52:11.265 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:52:11.266 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:52:11.266 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:52:11.266 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:52:11.266 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.539485+0000", 2022-06-29T15:52:11.267 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:52:11.267 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.267 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.267 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2022-06-29T15:52:11.267 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:52:11.268 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:52:11.268 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:52:11.268 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:52:11.269 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:52:11.269 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.487040+0000", 2022-06-29T15:52:11.269 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:52:11.269 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.269 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.270 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2022-06-29T15:52:11.270 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:52:11.270 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:52:11.270 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:52:11.271 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:52:11.271 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:52:11.271 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.748685+0000", 2022-06-29T15:52:11.271 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:52:11.272 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.272 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:11.272 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2022-06-29T15:52:11.273 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:52:11.273 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:52:11.273 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:52:11.273 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:52:11.274 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:52:11.274 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.459516+0000", 2022-06-29T15:52:11.274 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:52:11.274 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:11.275 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:52:11.275 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap": [], 2022-06-29T15:52:11.275 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap_items": [], 2022-06-29T15:52:11.275 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_temp": [], 2022-06-29T15:52:11.275 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_temp": [], 2022-06-29T15:52:11.276 INFO:teuthology.orchestra.run.smithi047.stdout: "blacklist": { 2022-06-29T15:52:11.276 INFO:teuthology.orchestra.run.smithi047.stdout: "172.21.15.47:6837/2466961065": "2022-06-30T15:52:04.693426+0000", 2022-06-29T15:52:11.276 INFO:teuthology.orchestra.run.smithi047.stdout: "172.21.15.47:6836/2466961065": "2022-06-30T15:52:04.693426+0000" 2022-06-29T15:52:11.277 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.277 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profiles": { 2022-06-29T15:52:11.277 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs_data": { 2022-06-29T15:52:11.277 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-device-class": "", 2022-06-29T15:52:11.277 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2022-06-29T15:52:11.278 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-root": "default", 2022-06-29T15:52:11.278 INFO:teuthology.orchestra.run.smithi047.stdout: "jerasure-per-chunk-alignment": "false", 2022-06-29T15:52:11.278 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2022-06-29T15:52:11.278 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "2", 2022-06-29T15:52:11.279 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2022-06-29T15:52:11.279 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2022-06-29T15:52:11.279 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van", 2022-06-29T15:52:11.279 INFO:teuthology.orchestra.run.smithi047.stdout: "w": "8" 2022-06-29T15:52:11.280 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.280 INFO:teuthology.orchestra.run.smithi047.stdout: "default": { 2022-06-29T15:52:11.280 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2022-06-29T15:52:11.280 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2022-06-29T15:52:11.281 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "1", 2022-06-29T15:52:11.281 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2022-06-29T15:52:11.281 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2022-06-29T15:52:11.281 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van" 2022-06-29T15:52:11.282 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:11.282 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:11.282 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps_queue": [], 2022-06-29T15:52:11.283 INFO:teuthology.orchestra.run.smithi047.stdout: "new_removed_snaps": [], 2022-06-29T15:52:11.283 INFO:teuthology.orchestra.run.smithi047.stdout: "new_purged_snaps": [], 2022-06-29T15:52:11.283 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_node_flags": {}, 2022-06-29T15:52:11.283 INFO:teuthology.orchestra.run.smithi047.stdout: "device_class_flags": {} 2022-06-29T15:52:11.284 INFO:teuthology.orchestra.run.smithi047.stdout:} 2022-06-29T15:52:11.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.042+0000 7fdedc8de700 1 -- 172.21.15.47:0/2741161898 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdeb4053980 msgr2=0x7fdeb4055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:11.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.042+0000 7fdedc8de700 1 --2- 172.21.15.47:0/2741161898 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdeb4053980 0x7fdeb4055e40 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7fdec800b000 tx=0x7fdec8008040).stop 2022-06-29T15:52:11.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.042+0000 7fdedc8de700 1 -- 172.21.15.47:0/2741161898 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fded80b1260 msgr2=0x7fded814ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:11.286 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.042+0000 7fdedc8de700 1 --2- 172.21.15.47:0/2741161898 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fded80b1260 0x7fded814ded0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 rx=0x7fded8060d70 tx=0x7fdecc014000).stop 2022-06-29T15:52:11.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.042+0000 7fdedc8de700 1 -- 172.21.15.47:0/2741161898 shutdown_connections 2022-06-29T15:52:11.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.042+0000 7fdedc8de700 1 --2- 172.21.15.47:0/2741161898 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdeb4053980 0x7fdeb4055e40 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:11.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.042+0000 7fdedc8de700 1 --2- 172.21.15.47:0/2741161898 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fded81406f0 0x7fded8140ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:11.287 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.042+0000 7fdedc8de700 1 --2- 172.21.15.47:0/2741161898 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fded80b2120 0x7fded8157100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:11.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.042+0000 7fdedc8de700 1 --2- 172.21.15.47:0/2741161898 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fded80b1260 0x7fded814ded0 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:11.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.042+0000 7fdedc8de700 1 -- 172.21.15.47:0/2741161898 >> 172.21.15.47:0/2741161898 conn(0x7fded80626f0 msgr2=0x7fded80b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:11.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.042+0000 7fdedc8de700 1 -- 172.21.15.47:0/2741161898 shutdown_connections 2022-06-29T15:52:11.288 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.042+0000 7fdedc8de700 1 -- 172.21.15.47:0/2741161898 wait complete. 2022-06-29T15:52:11.289 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:52:11.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.398+0000 7f33d594e700 1 Processor -- start 2022-06-29T15:52:11.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.398+0000 7f33d594e700 1 -- start start 2022-06-29T15:52:11.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.398+0000 7f33d594e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f33d00b4190 0x7f33d00b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:11.406 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.398+0000 7f33d594e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33d00b3b70 con 0x7f33d00b3070 2022-06-29T15:52:11.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.398+0000 7f33d594e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33d005fed0 con 0x7f33d00b2be0 2022-06-29T15:52:11.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.398+0000 7f33d594e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33d00b48b0 con 0x7f33d00b4190 2022-06-29T15:52:11.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.398+0000 7f33cf7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f33d00b3070 0x7f33d005f7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40410/0 (socket says 172.21.15.47:40410) 2022-06-29T15:52:11.407 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.398+0000 7f33cf7fe700 1 -- 172.21.15.47:0/3128859452 learned_addr learned my addr 172.21.15.47:0/3128859452 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:52:11.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.398+0000 7f33ceffd700 1 --2- 172.21.15.47:0/3128859452 >> v2:172.21.15.64:3300/0 conn(0x7f33d00b4190 0x7f33d00b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:11.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.398+0000 7f33cdffb700 1 -- 172.21.15.47:0/3128859452 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3728653211 0 0) 0x7f33d005fed0 con 0x7f33d00b2be0 2022-06-29T15:52:11.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.398+0000 7f33cdffb700 1 -- 172.21.15.47:0/3128859452 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33b0003260 con 0x7f33d00b2be0 2022-06-29T15:52:11.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.398+0000 7f33cdffb700 1 -- 172.21.15.47:0/3128859452 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3469665049 0 0) 0x7f33d00b3b70 con 0x7f33d00b3070 2022-06-29T15:52:11.409 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.398+0000 7f33ceffd700 1 -- 172.21.15.47:0/3128859452 >> v1:172.21.15.64:6789/0 conn(0x7f33d00b2be0 legacy=0x7f33d005f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:11.409 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.398+0000 7f33ceffd700 1 -- 172.21.15.47:0/3128859452 >> v1:172.21.15.47:6789/0 conn(0x7f33d00b3070 legacy=0x7f33d005f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:11.409 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.398+0000 7f33ceffd700 1 -- 172.21.15.47:0/3128859452 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d00ade10 con 0x7f33d00b4190 2022-06-29T15:52:11.409 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.398+0000 7f33ceffd700 1 --2- 172.21.15.47:0/3128859452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33d00b4190 0x7f33d00b26f0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7f33c8002f70 tx=0x7f33c8006df0).ready entity=mon.1 client_cookie=623fe447984a4031 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:11.410 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.398+0000 7f33cdffb700 1 -- 172.21.15.47:0/3128859452 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33c800d610 con 0x7f33d00b4190 2022-06-29T15:52:11.410 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.398+0000 7f33cdffb700 1 -- 172.21.15.47:0/3128859452 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33c8000e90 con 0x7f33d00b4190 2022-06-29T15:52:11.410 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.398+0000 7f33cdffb700 1 -- 172.21.15.47:0/3128859452 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33c800d960 con 0x7f33d00b4190 2022-06-29T15:52:11.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.398+0000 7f33d594e700 1 -- 172.21.15.47:0/3128859452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33d00b4190 msgr2=0x7f33d00b26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:11.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.398+0000 7f33d594e700 1 --2- 172.21.15.47:0/3128859452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33d00b4190 0x7f33d00b26f0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7f33c8002f70 tx=0x7f33c8006df0).stop 2022-06-29T15:52:11.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.402+0000 7f33d594e700 1 -- 172.21.15.47:0/3128859452 shutdown_connections 2022-06-29T15:52:11.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.402+0000 7f33d594e700 1 --2- 172.21.15.47:0/3128859452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33d00b4190 0x7f33d00b26f0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:11.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.402+0000 7f33d594e700 1 -- 172.21.15.47:0/3128859452 >> 172.21.15.47:0/3128859452 conn(0x7f33d005c510 msgr2=0x7f33d005c920 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:11.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.402+0000 7f33d594e700 1 -- 172.21.15.47:0/3128859452 shutdown_connections 2022-06-29T15:52:11.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.402+0000 7f33d594e700 1 -- 172.21.15.47:0/3128859452 wait complete. 2022-06-29T15:52:11.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.402+0000 7f33d594e700 1 Processor -- start 2022-06-29T15:52:11.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.402+0000 7f33d594e700 1 -- start start 2022-06-29T15:52:11.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.402+0000 7f33d594e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33d00b2be0 0x7f33d0123270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:11.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.402+0000 7f33d594e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33d00b3070 0x7f33d0128fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:11.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.402+0000 7f33d594e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f33d00b4190 0x7f33d0142760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:11.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.402+0000 7f33d594e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d0124380 con 0x7f33d00b2be0 2022-06-29T15:52:11.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.402+0000 7f33d594e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d01287f0 con 0x7f33d00b3070 2022-06-29T15:52:11.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.402+0000 7f33d594e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33d0152eb0 con 0x7f33d00b4190 2022-06-29T15:52:11.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.402+0000 7f33ceffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33d00b2be0 0x7f33d0123270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:11.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.402+0000 7f33ceffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33d00b2be0 0x7f33d0123270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54858/0 (socket says 172.21.15.47:54858) 2022-06-29T15:52:11.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.402+0000 7f33ceffd700 1 -- 172.21.15.47:0/3764454215 learned_addr learned my addr 172.21.15.47:0/3764454215 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:11.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.402+0000 7f33cf7fe700 1 --2- 172.21.15.47:0/3764454215 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f33d00b4190 0x7f33d0142760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:11.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.402+0000 7f33ce7fc700 1 --2- 172.21.15.47:0/3764454215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33d00b3070 0x7f33d0128fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:11.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.402+0000 7f33ceffd700 1 -- 172.21.15.47:0/3764454215 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f33d00b4190 msgr2=0x7f33d0142760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:11.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.402+0000 7f33ceffd700 1 --2- 172.21.15.47:0/3764454215 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f33d00b4190 0x7f33d0142760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:11.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.406+0000 7f33ceffd700 1 -- 172.21.15.47:0/3764454215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33d00b3070 msgr2=0x7f33d0128fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:11.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.406+0000 7f33ceffd700 1 --2- 172.21.15.47:0/3764454215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33d00b3070 0x7f33d0128fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:11.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.406+0000 7f33ceffd700 1 -- 172.21.15.47:0/3764454215 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c800b180 con 0x7f33d00b2be0 2022-06-29T15:52:11.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.406+0000 7f33ceffd700 1 --2- 172.21.15.47:0/3764454215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33d00b2be0 0x7f33d0123270 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 rx=0x7f33c8002f70 tx=0x7f33c8014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:11.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.406+0000 7f33c7fff700 1 -- 172.21.15.47:0/3764454215 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33c8016070 con 0x7f33d00b2be0 2022-06-29T15:52:11.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.406+0000 7f33c7fff700 1 -- 172.21.15.47:0/3764454215 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33c800de90 con 0x7f33d00b2be0 2022-06-29T15:52:11.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.406+0000 7f33d594e700 1 -- 172.21.15.47:0/3764454215 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33d01530e0 con 0x7f33d00b2be0 2022-06-29T15:52:11.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.406+0000 7f33c7fff700 1 -- 172.21.15.47:0/3764454215 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33c8013d60 con 0x7f33d00b2be0 2022-06-29T15:52:11.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.406+0000 7f33d594e700 1 -- 172.21.15.47:0/3764454215 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33d0122dd0 con 0x7f33d00b2be0 2022-06-29T15:52:11.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.406+0000 7f33d594e700 1 -- 172.21.15.47:0/3764454215 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33d00573d0 con 0x7f33d00b2be0 2022-06-29T15:52:11.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.406+0000 7f33c7fff700 1 -- 172.21.15.47:0/3764454215 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f33c8025030 con 0x7f33d00b2be0 2022-06-29T15:52:11.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.406+0000 7f33c7fff700 1 --2- 172.21.15.47:0/3764454215 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f33b40539e0 0x7f33b4055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:11.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.406+0000 7f33c7fff700 1 -- 172.21.15.47:0/3764454215 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f33c806cdc0 con 0x7f33d00b2be0 2022-06-29T15:52:11.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.406+0000 7f33ce7fc700 1 --2- 172.21.15.47:0/3764454215 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f33b40539e0 0x7f33b4055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:11.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.406+0000 7f33ce7fc700 1 --2- 172.21.15.47:0/3764454215 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f33b40539e0 0x7f33b4055ea0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7f33b800d000 tx=0x7f33b8007da0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:11.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.414+0000 7f33c7fff700 1 -- 172.21.15.47:0/3764454215 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f33c8043d50 con 0x7f33d00b2be0 2022-06-29T15:52:11.580 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.574+0000 7f33d594e700 1 -- 172.21.15.47:0/3764454215 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33d00b4fc0 con 0x7f33d00b2be0 2022-06-29T15:52:11.581 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.574+0000 7f33c7fff700 1 -- 172.21.15.47:0/3764454215 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+1562 (secure 0 0 0) 0x7f33c8023070 con 0x7f33d00b2be0 2022-06-29T15:52:11.582 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:52:11.582 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":76,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8296,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/3321939421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3321939421},{"type":"v1","addr":"172.21.15.64:6833","nonce":3321939421}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":72},{"gid":8415,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1886881850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1886881850},{"type":"v1","addr":"172.21.15.47:6835","nonce":1886881850}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":72},{"gid":8652,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/1613373360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1613373360},{"type":"v1","addr":"172.21.15.47:6837","nonce":1613373360}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":76}],"filesystems":[]} 2022-06-29T15:52:11.584 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.578+0000 7f33d594e700 1 -- 172.21.15.47:0/3764454215 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f33b40539e0 msgr2=0x7f33b4055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:11.584 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.578+0000 7f33d594e700 1 --2- 172.21.15.47:0/3764454215 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f33b40539e0 0x7f33b4055ea0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7f33b800d000 tx=0x7f33b8007da0).stop 2022-06-29T15:52:11.584 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.578+0000 7f33d594e700 1 -- 172.21.15.47:0/3764454215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33d00b2be0 msgr2=0x7f33d0123270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:11.584 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.578+0000 7f33d594e700 1 --2- 172.21.15.47:0/3764454215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33d00b2be0 0x7f33d0123270 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 rx=0x7f33c8002f70 tx=0x7f33c8014000).stop 2022-06-29T15:52:11.585 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.578+0000 7f33d594e700 1 -- 172.21.15.47:0/3764454215 shutdown_connections 2022-06-29T15:52:11.585 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.578+0000 7f33d594e700 1 --2- 172.21.15.47:0/3764454215 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f33d00b4190 0x7f33d0142760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:11.585 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.578+0000 7f33d594e700 1 --2- 172.21.15.47:0/3764454215 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f33b40539e0 0x7f33b4055ea0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:11.585 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.578+0000 7f33d594e700 1 --2- 172.21.15.47:0/3764454215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33d00b3070 0x7f33d0128fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:11.586 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.578+0000 7f33d594e700 1 --2- 172.21.15.47:0/3764454215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33d00b2be0 0x7f33d0123270 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:11.586 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.578+0000 7f33d594e700 1 -- 172.21.15.47:0/3764454215 >> 172.21.15.47:0/3764454215 conn(0x7f33d005c510 msgr2=0x7f33d005c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:11.586 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.578+0000 7f33d594e700 1 -- 172.21.15.47:0/3764454215 shutdown_connections 2022-06-29T15:52:11.586 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.578+0000 7f33d594e700 1 -- 172.21.15.47:0/3764454215 wait complete. 2022-06-29T15:52:11.587 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 76 2022-06-29T15:52:11.595 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:52:11.696 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.690+0000 7f92c9931700 1 Processor -- start 2022-06-29T15:52:11.697 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.690+0000 7f92c9931700 1 -- start start 2022-06-29T15:52:11.699 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.690+0000 7f92c9931700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f92c40b2120 0x7f92c40b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:11.699 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.690+0000 7f92c9931700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f92c40b1260 0x7f92c414b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:11.699 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.690+0000 7f92c9931700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92c40b0740 con 0x7f92c405f0b0 2022-06-29T15:52:11.700 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.690+0000 7f92c9931700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92c4147460 con 0x7f92c40b2120 2022-06-29T15:52:11.700 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.690+0000 7f92c9931700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92c40b0970 con 0x7f92c40b1260 2022-06-29T15:52:11.700 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.690+0000 7f92c2ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f92c40b2120 0x7f92c40b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:11.701 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.690+0000 7f92c27fc700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f92c405f0b0 0x7f92c4147af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40420/0 (socket says 172.21.15.47:40420) 2022-06-29T15:52:11.701 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.690+0000 7f92c27fc700 1 -- 172.21.15.47:0/1013018824 learned_addr learned my addr 172.21.15.47:0/1013018824 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:52:11.701 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.690+0000 7f92c37fe700 1 --2- 172.21.15.47:0/1013018824 >> v2:172.21.15.64:3300/0 conn(0x7f92c40b1260 0x7f92c414b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:11.701 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.690+0000 7f92c1ffb700 1 -- 172.21.15.47:0/1013018824 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 783687505 0 0) 0x7f92c40b0740 con 0x7f92c405f0b0 2022-06-29T15:52:11.701 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.690+0000 7f92c1ffb700 1 -- 172.21.15.47:0/1013018824 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92a8003260 con 0x7f92c405f0b0 2022-06-29T15:52:11.702 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.690+0000 7f92c2ffd700 1 -- 172.21.15.47:0/1013018824 >> v2:172.21.15.64:3300/0 conn(0x7f92c40b1260 msgr2=0x7f92c414b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:11.702 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.690+0000 7f92c2ffd700 1 --2- 172.21.15.47:0/1013018824 >> v2:172.21.15.64:3300/0 conn(0x7f92c40b1260 0x7f92c414b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:11.702 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.690+0000 7f92c2ffd700 1 -- 172.21.15.47:0/1013018824 >> v1:172.21.15.47:6789/0 conn(0x7f92c405f0b0 legacy=0x7f92c4147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:11.702 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.690+0000 7f92c1ffb700 1 -- 172.21.15.47:0/1013018824 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1609704669 0 0) 0x7f92a8003260 con 0x7f92c405f0b0 2022-06-29T15:52:11.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.690+0000 7f92c2ffd700 1 -- 172.21.15.47:0/1013018824 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c40b0c10 con 0x7f92c40b2120 2022-06-29T15:52:11.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.694+0000 7f92c2ffd700 1 --2- 172.21.15.47:0/1013018824 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92c40b2120 0x7f92c40b2500 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 rx=0x7f92c4141a70 tx=0x7f92b8006df0).ready entity=mon.0 client_cookie=2451fa33f2bb9796 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:11.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.694+0000 7f92c1ffb700 1 -- 172.21.15.47:0/1013018824 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92b4001660 con 0x7f92c40b2120 2022-06-29T15:52:11.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.694+0000 7f92c1ffb700 1 -- 172.21.15.47:0/1013018824 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b8000e90 con 0x7f92c40b2120 2022-06-29T15:52:11.704 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.694+0000 7f92c1ffb700 1 -- 172.21.15.47:0/1013018824 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92b8012460 con 0x7f92c40b2120 2022-06-29T15:52:11.707 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.694+0000 7f92c9931700 1 -- 172.21.15.47:0/1013018824 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92c40b2120 msgr2=0x7f92c40b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:11.707 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.694+0000 7f92c9931700 1 --2- 172.21.15.47:0/1013018824 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92c40b2120 0x7f92c40b2500 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 rx=0x7f92c4141a70 tx=0x7f92b8006df0).stop 2022-06-29T15:52:11.708 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.694+0000 7f92c9931700 1 -- 172.21.15.47:0/1013018824 shutdown_connections 2022-06-29T15:52:11.708 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.694+0000 7f92c9931700 1 --2- 172.21.15.47:0/1013018824 >> v2:172.21.15.64:3300/0 conn(0x7f92c40b1260 0x7f92c414b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:11.708 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.694+0000 7f92c9931700 1 --2- 172.21.15.47:0/1013018824 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92c40b2120 0x7f92c40b2500 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:11.708 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.694+0000 7f92c9931700 1 -- 172.21.15.47:0/1013018824 >> 172.21.15.47:0/1013018824 conn(0x7f92c40626f0 msgr2=0x7f92c4062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:11.708 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.694+0000 7f92c9931700 1 -- 172.21.15.47:0/1013018824 shutdown_connections 2022-06-29T15:52:11.709 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.694+0000 7f92c9931700 1 -- 172.21.15.47:0/1013018824 wait complete. 2022-06-29T15:52:11.709 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.694+0000 7f92c9931700 1 Processor -- start 2022-06-29T15:52:11.709 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.694+0000 7f92c9931700 1 -- start start 2022-06-29T15:52:11.709 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.694+0000 7f92c9931700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92c40b1260 0x7f92c40aec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:11.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.694+0000 7f92c9931700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92c4067160 0x7f92c4063490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:11.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.694+0000 7f92c9931700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92c4065720 0x7f92c4063980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:11.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.694+0000 7f92c9931700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c4065280 con 0x7f92c4067160 2022-06-29T15:52:11.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.694+0000 7f92c9931700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c4064660 con 0x7f92c4065720 2022-06-29T15:52:11.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.694+0000 7f92c9931700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92c4064fe0 con 0x7f92c40b1260 2022-06-29T15:52:11.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.694+0000 7f92c27fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92c4067160 0x7f92c4063490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:11.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.694+0000 7f92c27fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92c4067160 0x7f92c4063490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54872/0 (socket says 172.21.15.47:54872) 2022-06-29T15:52:11.712 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.694+0000 7f92c27fc700 1 -- 172.21.15.47:0/2833989395 learned_addr learned my addr 172.21.15.47:0/2833989395 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:11.712 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.698+0000 7f92c37fe700 1 --2- 172.21.15.47:0/2833989395 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92c4065720 0x7f92c4063980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:11.712 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.698+0000 7f92c2ffd700 1 --2- 172.21.15.47:0/2833989395 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92c40b1260 0x7f92c40aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:11.712 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.698+0000 7f92c27fc700 1 -- 172.21.15.47:0/2833989395 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92c40b1260 msgr2=0x7f92c40aec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:11.713 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.698+0000 7f92c27fc700 1 --2- 172.21.15.47:0/2833989395 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92c40b1260 0x7f92c40aec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:11.713 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.698+0000 7f92c27fc700 1 -- 172.21.15.47:0/2833989395 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92c4065720 msgr2=0x7f92c4063980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:11.713 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.698+0000 7f92c27fc700 1 --2- 172.21.15.47:0/2833989395 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92c4065720 0x7f92c4063980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:11.713 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.698+0000 7f92c27fc700 1 -- 172.21.15.47:0/2833989395 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b800b180 con 0x7f92c4067160 2022-06-29T15:52:11.714 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.698+0000 7f92c37fe700 1 --2- 172.21.15.47:0/2833989395 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92c4065720 0x7f92c4063980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:52:11.714 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.698+0000 7f92c27fc700 1 --2- 172.21.15.47:0/2833989395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92c4067160 0x7f92c4063490 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 rx=0x7f92b40018c0 tx=0x7f92b4001e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:11.714 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.698+0000 7f92b3fff700 1 -- 172.21.15.47:0/2833989395 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92b40072e0 con 0x7f92c4067160 2022-06-29T15:52:11.714 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.698+0000 7f92b3fff700 1 -- 172.21.15.47:0/2833989395 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b4007440 con 0x7f92c4067160 2022-06-29T15:52:11.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.698+0000 7f92c9931700 1 -- 172.21.15.47:0/2833989395 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92c40648c0 con 0x7f92c4067160 2022-06-29T15:52:11.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.698+0000 7f92b3fff700 1 -- 172.21.15.47:0/2833989395 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92b4005800 con 0x7f92c4067160 2022-06-29T15:52:11.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.698+0000 7f92c9931700 1 -- 172.21.15.47:0/2833989395 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92c4063e70 con 0x7f92c4067160 2022-06-29T15:52:11.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.698+0000 7f92c9931700 1 -- 172.21.15.47:0/2833989395 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92c4064a50 con 0x7f92c4067160 2022-06-29T15:52:11.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.698+0000 7f92b3fff700 1 -- 172.21.15.47:0/2833989395 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f92b4010070 con 0x7f92c4067160 2022-06-29T15:52:11.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.698+0000 7f92b3fff700 1 --2- 172.21.15.47:0/2833989395 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f92a0053980 0x7f92a0055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:11.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.698+0000 7f92b3fff700 1 -- 172.21.15.47:0/2833989395 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f92b406b520 con 0x7f92c4067160 2022-06-29T15:52:11.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.698+0000 7f92c2ffd700 1 --2- 172.21.15.47:0/2833989395 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f92a0053980 0x7f92a0055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:11.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.702+0000 7f92c2ffd700 1 --2- 172.21.15.47:0/2833989395 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f92a0053980 0x7f92a0055e40 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 rx=0x7f92b8018000 tx=0x7f92b8014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:11.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.706+0000 7f92b3fff700 1 -- 172.21.15.47:0/2833989395 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f92b4042560 con 0x7f92c4067160 2022-06-29T15:52:11.875 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.870+0000 7f92c9931700 1 -- 172.21.15.47:0/2833989395 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f92c40573d0 con 0x7f92c4067160 2022-06-29T15:52:11.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.870+0000 7f92b3fff700 1 -- 172.21.15.47:0/2833989395 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+1562 (secure 0 0 0) 0x7f92b4045b80 con 0x7f92c4067160 2022-06-29T15:52:11.876 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:52:11.876 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":76,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8296,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/3321939421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3321939421},{"type":"v1","addr":"172.21.15.64:6833","nonce":3321939421}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":72},{"gid":8415,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1886881850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1886881850},{"type":"v1","addr":"172.21.15.47:6835","nonce":1886881850}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":72},{"gid":8652,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/1613373360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1613373360},{"type":"v1","addr":"172.21.15.47:6837","nonce":1613373360}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":76}],"filesystems":[]} 2022-06-29T15:52:11.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.870+0000 7f92c9931700 1 -- 172.21.15.47:0/2833989395 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f92a0053980 msgr2=0x7f92a0055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:11.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.870+0000 7f92c9931700 1 --2- 172.21.15.47:0/2833989395 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f92a0053980 0x7f92a0055e40 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 rx=0x7f92b8018000 tx=0x7f92b8014000).stop 2022-06-29T15:52:11.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.870+0000 7f92c9931700 1 -- 172.21.15.47:0/2833989395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92c4067160 msgr2=0x7f92c4063490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:11.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.870+0000 7f92c9931700 1 --2- 172.21.15.47:0/2833989395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92c4067160 0x7f92c4063490 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 rx=0x7f92b40018c0 tx=0x7f92b4001e00).stop 2022-06-29T15:52:11.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.870+0000 7f92c9931700 1 -- 172.21.15.47:0/2833989395 shutdown_connections 2022-06-29T15:52:11.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.870+0000 7f92c9931700 1 --2- 172.21.15.47:0/2833989395 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f92a0053980 0x7f92a0055e40 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:11.880 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.870+0000 7f92c9931700 1 --2- 172.21.15.47:0/2833989395 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92c4065720 0x7f92c4063980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:11.880 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.870+0000 7f92c9931700 1 --2- 172.21.15.47:0/2833989395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92c4067160 0x7f92c4063490 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:11.880 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.870+0000 7f92c9931700 1 --2- 172.21.15.47:0/2833989395 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f92c40b1260 0x7f92c40aec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:11.880 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.870+0000 7f92c9931700 1 -- 172.21.15.47:0/2833989395 >> 172.21.15.47:0/2833989395 conn(0x7f92c40626f0 msgr2=0x7f92c405f730 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:11.880 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.870+0000 7f92c9931700 1 -- 172.21.15.47:0/2833989395 shutdown_connections 2022-06-29T15:52:11.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:11.870+0000 7f92c9931700 1 -- 172.21.15.47:0/2833989395 wait complete. 2022-06-29T15:52:11.881 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 76 2022-06-29T15:52:11.889 INFO:tasks.ceph.mds.a:Restarting daemon 2022-06-29T15:52:11.889 INFO:tasks.ceph.mds.a:Stopping old one... 2022-06-29T15:52:11.890 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-06-29T15:52:11.890 INFO:teuthology.orchestra.run:waiting for 300 2022-06-29T15:52:11.890 INFO:tasks.ceph.mds.c:Restarting daemon 2022-06-29T15:52:11.890 INFO:tasks.ceph.mds.c:Stopping old one... 2022-06-29T15:52:11.891 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2022-06-29T15:52:11.891 INFO:teuthology.orchestra.run:waiting for 300 2022-06-29T15:52:11.892 INFO:tasks.ceph.mds.b:Restarting daemon 2022-06-29T15:52:11.892 INFO:tasks.ceph.mds.b:Stopping old one... 2022-06-29T15:52:11.892 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-06-29T15:52:11.892 INFO:teuthology.orchestra.run:waiting for 300 2022-06-29T15:52:11.900 INFO:tasks.ceph.mds.a:Stopped 2022-06-29T15:52:11.900 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-06-29T15:52:11.902 INFO:tasks.ceph.mds.a:Started 2022-06-29T15:52:11.903 DEBUG:teuthology.parallel:result is None 2022-06-29T15:52:11.904 INFO:tasks.ceph.mds.c:Stopped 2022-06-29T15:52:11.904 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2022-06-29T15:52:11.906 INFO:tasks.ceph.mds.c:Started 2022-06-29T15:52:11.906 DEBUG:teuthology.parallel:result is None 2022-06-29T15:52:11.907 INFO:tasks.ceph.mds.b:Stopped 2022-06-29T15:52:11.908 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-06-29T15:52:11.910 INFO:tasks.ceph.mds.b:Started 2022-06-29T15:52:11.911 DEBUG:teuthology.parallel:result is None 2022-06-29T15:52:11.911 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-06-29T15:52:11.972 INFO:tasks.ceph.mds.a.smithi047.stdout:starting mds.a at 2022-06-29T15:52:11.973 INFO:tasks.ceph.mds.c.smithi047.stdout:starting mds.c at 2022-06-29T15:52:11.989 INFO:tasks.ceph.mds.b.smithi064.stdout:starting mds.b at 2022-06-29T15:52:12.025 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.018+0000 7f633a094700 1 Processor -- start 2022-06-29T15:52:12.026 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.018+0000 7f633a094700 1 -- start start 2022-06-29T15:52:12.029 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.018+0000 7f633a094700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f63340b0cf0 0x7f6334146d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:12.029 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.018+0000 7f633a094700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f63340b06d0 0x7f633413ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:12.029 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.018+0000 7f633a094700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63340b4520 con 0x7f63340611b0 2022-06-29T15:52:12.029 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.018+0000 7f633a094700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63340b4b10 con 0x7f63340b0cf0 2022-06-29T15:52:12.030 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.018+0000 7f633a094700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63340621d0 con 0x7f63340b06d0 2022-06-29T15:52:12.030 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.018+0000 7f632bfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f63340b0cf0 0x7f6334146d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:12.030 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.022+0000 7f6333fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f63340b06d0 0x7f633413ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:12.031 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.022+0000 7f63337fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f63340611b0 0x7f6334146640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:48028/0 (socket says 172.21.15.47:48028) 2022-06-29T15:52:12.031 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.022+0000 7f63337fe700 1 -- 172.21.15.47:0/931619885 learned_addr learned my addr 172.21.15.47:0/931619885 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:52:12.031 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.022+0000 7f6333fff700 1 -- 172.21.15.47:0/931619885 >> v2:172.21.15.47:3300/0 conn(0x7f63340b0cf0 msgr2=0x7f6334146d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:12.031 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.022+0000 7f6333fff700 1 --2- 172.21.15.47:0/931619885 >> v2:172.21.15.47:3300/0 conn(0x7f63340b0cf0 0x7f6334146d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:12.032 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.022+0000 7f6333fff700 1 -- 172.21.15.47:0/931619885 >> v1:172.21.15.64:6789/0 conn(0x7f63340611b0 legacy=0x7f6334146640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:12.032 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.022+0000 7f6333fff700 1 -- 172.21.15.47:0/931619885 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6334061590 con 0x7f63340b06d0 2022-06-29T15:52:12.032 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.022+0000 7f6332ffd700 1 -- 172.21.15.47:0/931619885 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3488102560 0 0) 0x7f63340b4520 con 0x7f63340611b0 2022-06-29T15:52:12.032 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.022+0000 7f6333fff700 1 --2- 172.21.15.47:0/931619885 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f63340b06d0 0x7f633413ff00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f632c0032f0 tx=0x7f632c0091d0).ready entity=mon.2 client_cookie=d7bbae95669a657e server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:12.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.022+0000 7f6332ffd700 1 -- 172.21.15.47:0/931619885 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f632c00b6c0 con 0x7f63340b06d0 2022-06-29T15:52:12.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.022+0000 7f6332ffd700 1 -- 172.21.15.47:0/931619885 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f632c00b820 con 0x7f63340b06d0 2022-06-29T15:52:12.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.022+0000 7f6332ffd700 1 -- 172.21.15.47:0/931619885 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f632c00b990 con 0x7f63340b06d0 2022-06-29T15:52:12.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.022+0000 7f633a094700 1 -- 172.21.15.47:0/931619885 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f63340b06d0 msgr2=0x7f633413ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:12.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.022+0000 7f633a094700 1 --2- 172.21.15.47:0/931619885 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f63340b06d0 0x7f633413ff00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f632c0032f0 tx=0x7f632c0091d0).stop 2022-06-29T15:52:12.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.022+0000 7f633a094700 1 -- 172.21.15.47:0/931619885 shutdown_connections 2022-06-29T15:52:12.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.022+0000 7f633a094700 1 --2- 172.21.15.47:0/931619885 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f63340b06d0 0x7f633413ff00 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:12.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.022+0000 7f633a094700 1 --2- 172.21.15.47:0/931619885 >> v2:172.21.15.47:3300/0 conn(0x7f63340b0cf0 0x7f6334146d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:12.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.022+0000 7f633a094700 1 -- 172.21.15.47:0/931619885 >> 172.21.15.47:0/931619885 conn(0x7f63340b98b0 msgr2=0x7f63340b9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:12.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.022+0000 7f633a094700 1 -- 172.21.15.47:0/931619885 shutdown_connections 2022-06-29T15:52:12.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.022+0000 7f633a094700 1 -- 172.21.15.47:0/931619885 wait complete. 2022-06-29T15:52:12.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.022+0000 7f633a094700 1 Processor -- start 2022-06-29T15:52:12.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.022+0000 7f633a094700 1 -- start start 2022-06-29T15:52:12.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.022+0000 7f633a094700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63340611b0 0x7f633413daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:12.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.022+0000 7f633a094700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63340b06d0 0x7f63340b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:12.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.022+0000 7f633a094700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f63340b0cf0 0x7f6334152480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:12.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.022+0000 7f633a094700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6334156730 con 0x7f63340611b0 2022-06-29T15:52:12.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.022+0000 7f633a094700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6334143430 con 0x7f63340b06d0 2022-06-29T15:52:12.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.022+0000 7f633a094700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63341436d0 con 0x7f63340b0cf0 2022-06-29T15:52:12.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.022+0000 7f63337fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63340611b0 0x7f633413daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:12.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.022+0000 7f63337fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63340611b0 0x7f633413daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54916/0 (socket says 172.21.15.47:54916) 2022-06-29T15:52:12.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.022+0000 7f63337fe700 1 -- 172.21.15.47:0/2409520502 learned_addr learned my addr 172.21.15.47:0/2409520502 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:12.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.022+0000 7f6333fff700 1 --2- 172.21.15.47:0/2409520502 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f63340b0cf0 0x7f6334152480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:12.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.022+0000 7f632bfff700 1 --2- 172.21.15.47:0/2409520502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63340b06d0 0x7f63340b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:12.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.026+0000 7f63337fe700 1 -- 172.21.15.47:0/2409520502 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f63340b0cf0 msgr2=0x7f6334152480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:12.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.026+0000 7f63337fe700 1 --2- 172.21.15.47:0/2409520502 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f63340b0cf0 0x7f6334152480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:12.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.026+0000 7f63337fe700 1 -- 172.21.15.47:0/2409520502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63340b06d0 msgr2=0x7f63340b8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:12.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.026+0000 7f63337fe700 1 --2- 172.21.15.47:0/2409520502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63340b06d0 0x7f63340b8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:12.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.026+0000 7f63337fe700 1 -- 172.21.15.47:0/2409520502 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f632c009470 con 0x7f63340611b0 2022-06-29T15:52:12.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.026+0000 7f63337fe700 1 --2- 172.21.15.47:0/2409520502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63340611b0 0x7f633413daf0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 rx=0x7f6324000bc0 tx=0x7f6324007a60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:12.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.026+0000 7f63317fa700 1 -- 172.21.15.47:0/2409520502 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f63240070d0 con 0x7f63340611b0 2022-06-29T15:52:12.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.026+0000 7f63317fa700 1 -- 172.21.15.47:0/2409520502 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6324007230 con 0x7f63340611b0 2022-06-29T15:52:12.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.026+0000 7f633a094700 1 -- 172.21.15.47:0/2409520502 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63341563f0 con 0x7f63340611b0 2022-06-29T15:52:12.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.026+0000 7f63317fa700 1 -- 172.21.15.47:0/2409520502 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6324005610 con 0x7f63340611b0 2022-06-29T15:52:12.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.026+0000 7f633a094700 1 -- 172.21.15.47:0/2409520502 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63340b8ff0 con 0x7f63340611b0 2022-06-29T15:52:12.045 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.030+0000 7f633a094700 1 -- 172.21.15.47:0/2409520502 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6334156930 con 0x7f63340611b0 2022-06-29T15:52:12.045 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.030+0000 7f63317fa700 1 -- 172.21.15.47:0/2409520502 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f632401a060 con 0x7f63340611b0 2022-06-29T15:52:12.045 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.030+0000 7f63317fa700 1 --2- 172.21.15.47:0/2409520502 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6318053b80 0x7f6318056040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:12.045 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.030+0000 7f63317fa700 1 -- 172.21.15.47:0/2409520502 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f632406b0a0 con 0x7f63340611b0 2022-06-29T15:52:12.045 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.030+0000 7f632bfff700 1 --2- 172.21.15.47:0/2409520502 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6318053b80 0x7f6318056040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:12.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.030+0000 7f632bfff700 1 --2- 172.21.15.47:0/2409520502 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6318053b80 0x7f6318056040 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7f631c00b000 tx=0x7f631c016040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:12.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.034+0000 7f63317fa700 1 -- 172.21.15.47:0/2409520502 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6324042140 con 0x7f63340611b0 2022-06-29T15:52:12.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:12.170+0000 7f633a094700 1 -- 172.21.15.47:0/2409520502 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f63340573d0 con 0x7f63340611b0 2022-06-29T15:52:12.634 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:52:12.637 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:52:12.640 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:52:12.643 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:52:13.030 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.021+0000 7f63317fa700 1 -- 172.21.15.47:0/2409520502 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v158) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6324018070 con 0x7f63340611b0 2022-06-29T15:52:13.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.025+0000 7f633a094700 1 -- 172.21.15.47:0/2409520502 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6318053b80 msgr2=0x7f6318056040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:13.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.025+0000 7f633a094700 1 --2- 172.21.15.47:0/2409520502 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6318053b80 0x7f6318056040 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7f631c00b000 tx=0x7f631c016040).stop 2022-06-29T15:52:13.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.025+0000 7f633a094700 1 -- 172.21.15.47:0/2409520502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63340611b0 msgr2=0x7f633413daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:13.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.025+0000 7f633a094700 1 --2- 172.21.15.47:0/2409520502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63340611b0 0x7f633413daf0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 rx=0x7f6324000bc0 tx=0x7f6324007a60).stop 2022-06-29T15:52:13.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.025+0000 7f633a094700 1 -- 172.21.15.47:0/2409520502 shutdown_connections 2022-06-29T15:52:13.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.025+0000 7f633a094700 1 --2- 172.21.15.47:0/2409520502 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f63340b0cf0 0x7f6334152480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:13.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.025+0000 7f633a094700 1 --2- 172.21.15.47:0/2409520502 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6318053b80 0x7f6318056040 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:13.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.025+0000 7f633a094700 1 --2- 172.21.15.47:0/2409520502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63340b06d0 0x7f63340b8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:13.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.025+0000 7f633a094700 1 --2- 172.21.15.47:0/2409520502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63340611b0 0x7f633413daf0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:13.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.025+0000 7f633a094700 1 -- 172.21.15.47:0/2409520502 >> 172.21.15.47:0/2409520502 conn(0x7f63340b98b0 msgr2=0x7f6334147920 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:13.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.025+0000 7f633a094700 1 -- 172.21.15.47:0/2409520502 shutdown_connections 2022-06-29T15:52:13.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.025+0000 7f633a094700 1 -- 172.21.15.47:0/2409520502 wait complete. 2022-06-29T15:52:13.038 INFO:teuthology.orchestra.run.smithi047.stderr: removed all blacklist entries 2022-06-29T15:52:13.050 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2022-06-29T15:52:13.051 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-06-29T15:52:13.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.157+0000 7fc86eddc700 1 Processor -- start 2022-06-29T15:52:13.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.157+0000 7fc86eddc700 1 -- start start 2022-06-29T15:52:13.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.157+0000 7fc86eddc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc8680b2120 0x7fc8680b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:13.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.157+0000 7fc86eddc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc86805f0b0 0x7fc86814b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:13.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.157+0000 7fc86eddc700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8680b0740 con 0x7fc8680b1260 2022-06-29T15:52:13.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.157+0000 7fc86eddc700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc868147460 con 0x7fc86805f0b0 2022-06-29T15:52:13.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.157+0000 7fc86eddc700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8680b0970 con 0x7fc8680b2120 2022-06-29T15:52:13.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.157+0000 7fc867fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc86805f0b0 0x7fc86814b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:13.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.157+0000 7fc867fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc86805f0b0 0x7fc86814b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54930/0 (socket says 172.21.15.47:54930) 2022-06-29T15:52:13.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.157+0000 7fc86d379700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc8680b1260 0x7fc868147af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40486/0 (socket says 172.21.15.47:40486) 2022-06-29T15:52:13.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.157+0000 7fc867fff700 1 -- 172.21.15.47:0/4281275276 learned_addr learned my addr 172.21.15.47:0/4281275276 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:13.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.157+0000 7fc86cb78700 1 --2- 172.21.15.47:0/4281275276 >> v2:172.21.15.64:3300/0 conn(0x7fc8680b2120 0x7fc8680b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:13.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.157+0000 7fc867fff700 1 -- 172.21.15.47:0/4281275276 >> v2:172.21.15.64:3300/0 conn(0x7fc8680b2120 msgr2=0x7fc8680b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:13.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.157+0000 7fc867fff700 1 --2- 172.21.15.47:0/4281275276 >> v2:172.21.15.64:3300/0 conn(0x7fc8680b2120 0x7fc8680b2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:13.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.157+0000 7fc867fff700 1 -- 172.21.15.47:0/4281275276 >> v1:172.21.15.47:6789/0 conn(0x7fc8680b1260 legacy=0x7fc868147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:13.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.157+0000 7fc8677fe700 1 -- 172.21.15.47:0/4281275276 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 208418503 0 0) 0x7fc8680b0740 con 0x7fc8680b1260 2022-06-29T15:52:13.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.157+0000 7fc867fff700 1 -- 172.21.15.47:0/4281275276 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8680b0c10 con 0x7fc86805f0b0 2022-06-29T15:52:13.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.157+0000 7fc867fff700 1 --2- 172.21.15.47:0/4281275276 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc86805f0b0 0x7fc86814b520 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 rx=0x7fc868060d70 tx=0x7fc858009290).ready entity=mon.0 client_cookie=32319d5465da6eb6 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:13.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.157+0000 7fc8677fe700 1 -- 172.21.15.47:0/4281275276 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc85800b860 con 0x7fc86805f0b0 2022-06-29T15:52:13.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.157+0000 7fc8677fe700 1 -- 172.21.15.47:0/4281275276 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc85800b9c0 con 0x7fc86805f0b0 2022-06-29T15:52:13.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.157+0000 7fc8677fe700 1 -- 172.21.15.47:0/4281275276 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc85800bb30 con 0x7fc86805f0b0 2022-06-29T15:52:13.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.157+0000 7fc86eddc700 1 -- 172.21.15.47:0/4281275276 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc86805f0b0 msgr2=0x7fc86814b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:13.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.157+0000 7fc86eddc700 1 --2- 172.21.15.47:0/4281275276 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc86805f0b0 0x7fc86814b520 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 rx=0x7fc868060d70 tx=0x7fc858009290).stop 2022-06-29T15:52:13.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.157+0000 7fc86eddc700 1 -- 172.21.15.47:0/4281275276 shutdown_connections 2022-06-29T15:52:13.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.157+0000 7fc86eddc700 1 --2- 172.21.15.47:0/4281275276 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc86805f0b0 0x7fc86814b520 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:13.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.157+0000 7fc86eddc700 1 --2- 172.21.15.47:0/4281275276 >> v2:172.21.15.64:3300/0 conn(0x7fc8680b2120 0x7fc8680b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:13.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.157+0000 7fc86eddc700 1 -- 172.21.15.47:0/4281275276 >> 172.21.15.47:0/4281275276 conn(0x7fc8680626f0 msgr2=0x7fc868062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:13.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.161+0000 7fc86eddc700 1 -- 172.21.15.47:0/4281275276 shutdown_connections 2022-06-29T15:52:13.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.161+0000 7fc86eddc700 1 -- 172.21.15.47:0/4281275276 wait complete. 2022-06-29T15:52:13.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.161+0000 7fc86eddc700 1 Processor -- start 2022-06-29T15:52:13.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.161+0000 7fc86eddc700 1 -- start start 2022-06-29T15:52:13.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.161+0000 7fc86eddc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8680b1260 0x7fc868064930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:13.183 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.161+0000 7fc86eddc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8680b2120 0x7fc86815a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:13.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.161+0000 7fc86eddc700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc8680652b0 0x7fc86815a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:13.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.161+0000 7fc86eddc700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8680aeb10 con 0x7fc8680b2120 2022-06-29T15:52:13.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.161+0000 7fc86eddc700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8680a05d0 con 0x7fc8680b1260 2022-06-29T15:52:13.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.161+0000 7fc86eddc700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8680ae330 con 0x7fc8680652b0 2022-06-29T15:52:13.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.161+0000 7fc867fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8680b2120 0x7fc86815a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:13.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.161+0000 7fc867fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8680b2120 0x7fc86815a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54936/0 (socket says 172.21.15.47:54936) 2022-06-29T15:52:13.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.161+0000 7fc867fff700 1 -- 172.21.15.47:0/2802870998 learned_addr learned my addr 172.21.15.47:0/2802870998 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:13.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.161+0000 7fc86cb78700 1 --2- 172.21.15.47:0/2802870998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8680b1260 0x7fc868064930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:13.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.161+0000 7fc86d379700 1 --2- 172.21.15.47:0/2802870998 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc8680652b0 0x7fc86815a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:13.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.161+0000 7fc867fff700 1 -- 172.21.15.47:0/2802870998 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc8680652b0 msgr2=0x7fc86815a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:13.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.161+0000 7fc867fff700 1 --2- 172.21.15.47:0/2802870998 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc8680652b0 0x7fc86815a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:13.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.161+0000 7fc867fff700 1 -- 172.21.15.47:0/2802870998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8680b1260 msgr2=0x7fc868064930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:13.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.161+0000 7fc867fff700 1 --2- 172.21.15.47:0/2802870998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8680b1260 0x7fc868064930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:13.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.161+0000 7fc867fff700 1 -- 172.21.15.47:0/2802870998 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc858009530 con 0x7fc8680b2120 2022-06-29T15:52:13.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.161+0000 7fc86cb78700 1 --2- 172.21.15.47:0/2802870998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8680b1260 0x7fc868064930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:52:13.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.161+0000 7fc867fff700 1 --2- 172.21.15.47:0/2802870998 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8680b2120 0x7fc86815a280 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 rx=0x7fc868060d70 tx=0x7fc858016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:13.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.161+0000 7fc865ffb700 1 -- 172.21.15.47:0/2802870998 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc858021630 con 0x7fc8680b2120 2022-06-29T15:52:13.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.161+0000 7fc865ffb700 1 -- 172.21.15.47:0/2802870998 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc858021790 con 0x7fc8680b2120 2022-06-29T15:52:13.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.161+0000 7fc86eddc700 1 -- 172.21.15.47:0/2802870998 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8680ae5d0 con 0x7fc8680b2120 2022-06-29T15:52:13.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.161+0000 7fc865ffb700 1 -- 172.21.15.47:0/2802870998 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc858021af0 con 0x7fc8680b2120 2022-06-29T15:52:13.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.161+0000 7fc86eddc700 1 -- 172.21.15.47:0/2802870998 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc868159de0 con 0x7fc8680b2120 2022-06-29T15:52:13.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.165+0000 7fc86eddc700 1 -- 172.21.15.47:0/2802870998 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc868159fa0 con 0x7fc8680b2120 2022-06-29T15:52:13.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.165+0000 7fc865ffb700 1 -- 172.21.15.47:0/2802870998 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc85800f070 con 0x7fc8680b2120 2022-06-29T15:52:13.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.165+0000 7fc865ffb700 1 --2- 172.21.15.47:0/2802870998 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc84c053b00 0x7fc84c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:13.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.165+0000 7fc865ffb700 1 -- 172.21.15.47:0/2802870998 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fc85806d4c0 con 0x7fc8680b2120 2022-06-29T15:52:13.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.165+0000 7fc86cb78700 1 --2- 172.21.15.47:0/2802870998 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc84c053b00 0x7fc84c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:13.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.165+0000 7fc86cb78700 1 --2- 172.21.15.47:0/2802870998 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc84c053b00 0x7fc84c055fc0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 rx=0x7fc85000b000 tx=0x7fc850008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:13.191 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.173+0000 7fc865ffb700 1 -- 172.21.15.47:0/2802870998 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc8580444b0 con 0x7fc8680b2120 2022-06-29T15:52:13.352 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.345+0000 7fc86eddc700 1 -- 172.21.15.47:0/2802870998 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fc8680573d0 con 0x7fc8680b2120 2022-06-29T15:52:13.353 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.345+0000 7fc865ffb700 1 -- 172.21.15.47:0/2802870998 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v36) v1 ==== 80+0+4255 (secure 0 0 0) 0x7fc858015070 con 0x7fc8680b2120 2022-06-29T15:52:13.353 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:52:13.353 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2022-06-29T15:52:13.353 INFO:teuthology.orchestra.run.smithi047.stdout: "auth_dump": [ 2022-06-29T15:52:13.354 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:13.354 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.a", 2022-06-29T15:52:13.354 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxiQJFFMxAA9Tv7FJD9u/BKvbkuKld6Kg==", 2022-06-29T15:52:13.355 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:52:13.355 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:52:13.355 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:52:13.355 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:52:13.356 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:52:13.356 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:13.356 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:13.356 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:13.357 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.b", 2022-06-29T15:52:13.357 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC2crxibt6FARAAEf2bwXKFhXKtqr1nN0rdXQ==", 2022-06-29T15:52:13.357 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:52:13.357 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:52:13.358 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:52:13.358 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:52:13.358 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:52:13.358 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:13.359 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:13.359 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:13.359 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.c", 2022-06-29T15:52:13.359 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxi/4ZMORAANpLbgTm3ehqi80USc3swGg==", 2022-06-29T15:52:13.360 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:52:13.360 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:52:13.360 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:52:13.360 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:52:13.361 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:52:13.361 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:13.361 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:13.361 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:13.361 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.0", 2022-06-29T15:52:13.362 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC2crxiY0YsMxAAZivb+CxL/w7vZwQv37Y4oQ==", 2022-06-29T15:52:13.362 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:52:13.362 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:52:13.362 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:52:13.363 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:52:13.363 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:13.363 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:13.363 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:13.364 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.1", 2022-06-29T15:52:13.364 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC5crxitLkKFxAA8+++LUUKLiO9SAuAfYgT5g==", 2022-06-29T15:52:13.364 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:52:13.364 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:52:13.365 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:52:13.365 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:52:13.365 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:13.365 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:13.366 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:13.366 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.2", 2022-06-29T15:52:13.366 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC7crxi1oM9NxAA2FfLdNt32YPwlyNdGQpaQA==", 2022-06-29T15:52:13.366 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:52:13.367 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:52:13.367 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:52:13.367 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:52:13.367 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:13.367 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:13.368 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:13.368 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.3", 2022-06-29T15:52:13.368 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC+crxiTsBgGhAA+gLBBDaskPbmRNWzgKwcHA==", 2022-06-29T15:52:13.368 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:52:13.369 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:52:13.369 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:52:13.369 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:52:13.369 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:13.370 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:13.370 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:13.370 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.4", 2022-06-29T15:52:13.370 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDDcrxivJebIhAAfwWJa7tK4hamkk1/1CSSfw==", 2022-06-29T15:52:13.371 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:52:13.371 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:52:13.371 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:52:13.371 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:52:13.371 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:13.372 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:13.372 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:13.372 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.5", 2022-06-29T15:52:13.372 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDGcrxijuozBhAAt+Dx3wH3WKp3DrbCbhmh8Q==", 2022-06-29T15:52:13.373 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:52:13.373 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:52:13.373 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:52:13.373 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:52:13.374 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:13.374 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:13.374 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:13.374 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.6", 2022-06-29T15:52:13.374 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDIcrxi91kXJxAA9JX7n98P394bD41Q9LUdvQ==", 2022-06-29T15:52:13.375 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:52:13.375 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:52:13.375 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:52:13.375 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:52:13.376 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:13.376 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:13.376 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:13.376 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.7", 2022-06-29T15:52:13.377 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQDLcrxiHj0ZDRAASy6fAix8TmEZbAGtuFCfFg==", 2022-06-29T15:52:13.377 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:52:13.377 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:52:13.377 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:52:13.378 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:52:13.378 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:13.378 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:13.378 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:13.378 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.0", 2022-06-29T15:52:13.379 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC2crxiVDlfBxAAezkUfg4e2g0Rfakaa/Ek1Q==", 2022-06-29T15:52:13.379 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:52:13.379 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:52:13.379 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow r", 2022-06-29T15:52:13.380 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-29T15:52:13.380 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:13.380 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:13.380 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:13.381 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.1", 2022-06-29T15:52:13.381 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC2crxisXJ1CxAAhl07oF7AW66NdzfvvZTJmA==", 2022-06-29T15:52:13.381 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:52:13.381 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2022-06-29T15:52:13.382 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow r", 2022-06-29T15:52:13.382 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rw pool=cephfs_data" 2022-06-29T15:52:13.382 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:13.382 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:13.383 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:13.383 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.admin", 2022-06-29T15:52:13.383 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxiW7HSFRAAhYqoGU9GFqj2HfUVGValSA==", 2022-06-29T15:52:13.383 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:52:13.383 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2022-06-29T15:52:13.384 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2022-06-29T15:52:13.384 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2022-06-29T15:52:13.384 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:52:13.384 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:13.385 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:13.385 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:13.385 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.x", 2022-06-29T15:52:13.385 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxidmuIMBAAcpNlYVu5+icCHg+yIjcKXw==", 2022-06-29T15:52:13.386 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:52:13.386 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2022-06-29T15:52:13.386 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2022-06-29T15:52:13.386 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:52:13.387 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:13.387 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:13.387 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:13.387 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.y", 2022-06-29T15:52:13.387 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQC1crxisSMFLBAAv03r4OoPjOP9nI9G/LK0iw==", 2022-06-29T15:52:13.388 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2022-06-29T15:52:13.388 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2022-06-29T15:52:13.388 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2022-06-29T15:52:13.388 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2022-06-29T15:52:13.389 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:13.389 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:13.389 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:13.389 INFO:teuthology.orchestra.run.smithi047.stdout:} 2022-06-29T15:52:13.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.345+0000 7fc86eddc700 1 -- 172.21.15.47:0/2802870998 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc84c053b00 msgr2=0x7fc84c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:13.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.345+0000 7fc86eddc700 1 --2- 172.21.15.47:0/2802870998 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc84c053b00 0x7fc84c055fc0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 rx=0x7fc85000b000 tx=0x7fc850008040).stop 2022-06-29T15:52:13.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.345+0000 7fc86eddc700 1 -- 172.21.15.47:0/2802870998 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8680b2120 msgr2=0x7fc86815a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:13.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.345+0000 7fc86eddc700 1 --2- 172.21.15.47:0/2802870998 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8680b2120 0x7fc86815a280 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 rx=0x7fc868060d70 tx=0x7fc858016000).stop 2022-06-29T15:52:13.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.345+0000 7fc86eddc700 1 -- 172.21.15.47:0/2802870998 shutdown_connections 2022-06-29T15:52:13.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.345+0000 7fc86eddc700 1 --2- 172.21.15.47:0/2802870998 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc8680652b0 0x7fc86815a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:13.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.345+0000 7fc86eddc700 1 --2- 172.21.15.47:0/2802870998 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8680b2120 0x7fc86815a280 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:13.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.345+0000 7fc86eddc700 1 --2- 172.21.15.47:0/2802870998 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc84c053b00 0x7fc84c055fc0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:13.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.345+0000 7fc86eddc700 1 --2- 172.21.15.47:0/2802870998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8680b1260 0x7fc868064930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:13.394 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.345+0000 7fc86eddc700 1 -- 172.21.15.47:0/2802870998 >> 172.21.15.47:0/2802870998 conn(0x7fc8680626f0 msgr2=0x7fc8680b5480 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:13.394 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.345+0000 7fc86eddc700 1 -- 172.21.15.47:0/2802870998 shutdown_connections 2022-06-29T15:52:13.394 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.349+0000 7fc86eddc700 1 -- 172.21.15.47:0/2802870998 wait complete. 2022-06-29T15:52:13.395 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:52:13.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.493+0000 7f036359e700 1 Processor -- start 2022-06-29T15:52:13.502 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.493+0000 7f036359e700 1 -- start start 2022-06-29T15:52:13.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.497+0000 7f036359e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f03640aa9b0 0x7f03640ac180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:13.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.497+0000 7f036359e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f03640ac670 0x7f036414b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:13.504 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.497+0000 7f036359e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03640ae3e0 con 0x7f03640b2740 2022-06-29T15:52:13.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.497+0000 7f036359e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03640af110 con 0x7f03640aa9b0 2022-06-29T15:52:13.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.497+0000 7f036359e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03640a9e80 con 0x7f03640ac670 2022-06-29T15:52:13.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.497+0000 7f036259c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f03640b2740 0x7f03640af990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40496/0 (socket says 172.21.15.47:40496) 2022-06-29T15:52:13.505 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.497+0000 7f036259c700 1 -- 172.21.15.47:0/3181778508 learned_addr learned my addr 172.21.15.47:0/3181778508 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:52:13.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.497+0000 7f0362d9d700 1 --2- 172.21.15.47:0/3181778508 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f03640ac670 0x7f036414b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:13.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.497+0000 7f0361d9b700 1 --2- 172.21.15.47:0/3181778508 >> v2:172.21.15.64:3300/0 conn(0x7f03640aa9b0 0x7f03640ac180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:13.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.497+0000 7f036159a700 1 -- 172.21.15.47:0/3181778508 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1763619139 0 0) 0x7f03640ae3e0 con 0x7f03640b2740 2022-06-29T15:52:13.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.497+0000 7f036159a700 1 -- 172.21.15.47:0/3181778508 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0348003260 con 0x7f03640b2740 2022-06-29T15:52:13.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.497+0000 7f0362d9d700 1 -- 172.21.15.47:0/3181778508 >> v2:172.21.15.64:3300/0 conn(0x7f03640aa9b0 msgr2=0x7f03640ac180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:13.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.497+0000 7f0362d9d700 1 --2- 172.21.15.47:0/3181778508 >> v2:172.21.15.64:3300/0 conn(0x7f03640aa9b0 0x7f03640ac180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:13.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.497+0000 7f0362d9d700 1 -- 172.21.15.47:0/3181778508 >> v1:172.21.15.47:6789/0 conn(0x7f03640b2740 legacy=0x7f03640af990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:13.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.497+0000 7f036159a700 1 -- 172.21.15.47:0/3181778508 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1167258626 0 0) 0x7f0348003260 con 0x7f03640b2740 2022-06-29T15:52:13.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.497+0000 7f0362d9d700 1 -- 172.21.15.47:0/3181778508 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03640b1750 con 0x7f03640ac670 2022-06-29T15:52:13.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.497+0000 7f0362d9d700 1 --2- 172.21.15.47:0/3181778508 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f03640ac670 0x7f036414b6c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f0354001800 tx=0x7f035c0091d0).ready entity=mon.2 client_cookie=6b57ae54e58fdba server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:13.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.497+0000 7f036159a700 1 -- 172.21.15.47:0/3181778508 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0354001660 con 0x7f03640ac670 2022-06-29T15:52:13.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.497+0000 7f036159a700 1 -- 172.21.15.47:0/3181778508 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f035c00f510 con 0x7f03640ac670 2022-06-29T15:52:13.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.497+0000 7f036159a700 1 -- 172.21.15.47:0/3181778508 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f035c00f680 con 0x7f03640ac670 2022-06-29T15:52:13.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.497+0000 7f036359e700 1 -- 172.21.15.47:0/3181778508 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f03640ac670 msgr2=0x7f036414b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:13.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.497+0000 7f036359e700 1 --2- 172.21.15.47:0/3181778508 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f03640ac670 0x7f036414b6c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f0354001800 tx=0x7f035c0091d0).stop 2022-06-29T15:52:13.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.497+0000 7f036359e700 1 -- 172.21.15.47:0/3181778508 shutdown_connections 2022-06-29T15:52:13.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.497+0000 7f036359e700 1 --2- 172.21.15.47:0/3181778508 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f03640ac670 0x7f036414b6c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:13.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.497+0000 7f036359e700 1 --2- 172.21.15.47:0/3181778508 >> v2:172.21.15.64:3300/0 conn(0x7f03640aa9b0 0x7f03640ac180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:13.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.497+0000 7f036359e700 1 -- 172.21.15.47:0/3181778508 >> 172.21.15.47:0/3181778508 conn(0x7f0364062df0 msgr2=0x7f03640631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:13.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.497+0000 7f036359e700 1 -- 172.21.15.47:0/3181778508 shutdown_connections 2022-06-29T15:52:13.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.497+0000 7f036359e700 1 -- 172.21.15.47:0/3181778508 wait complete. 2022-06-29T15:52:13.515 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.501+0000 7f036359e700 1 Processor -- start 2022-06-29T15:52:13.516 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.501+0000 7f036359e700 1 -- start start 2022-06-29T15:52:13.516 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.505+0000 7f036359e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03640aa9b0 0x7f0364146940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:13.516 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.505+0000 7f036359e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f03640ac670 0x7f03640b4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:13.516 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.505+0000 7f036359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03640b2740 0x7f03640b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:13.517 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.505+0000 7f036359e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03640b4510 con 0x7f03640aa9b0 2022-06-29T15:52:13.517 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.505+0000 7f036359e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03640b3bc0 con 0x7f03640b2740 2022-06-29T15:52:13.517 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.505+0000 7f036359e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f036405f770 con 0x7f03640ac670 2022-06-29T15:52:13.517 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.505+0000 7f036259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03640aa9b0 0x7f0364146940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:13.518 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.505+0000 7f036259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03640aa9b0 0x7f0364146940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54948/0 (socket says 172.21.15.47:54948) 2022-06-29T15:52:13.518 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.505+0000 7f036259c700 1 -- 172.21.15.47:0/3057493043 learned_addr learned my addr 172.21.15.47:0/3057493043 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:13.518 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.505+0000 7f0362d9d700 1 --2- 172.21.15.47:0/3057493043 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03640b2740 0x7f03640b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:13.518 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.505+0000 7f0361d9b700 1 --2- 172.21.15.47:0/3057493043 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f03640ac670 0x7f03640b4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:13.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.505+0000 7f036259c700 1 -- 172.21.15.47:0/3057493043 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f03640ac670 msgr2=0x7f03640b4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:13.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.505+0000 7f036259c700 1 --2- 172.21.15.47:0/3057493043 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f03640ac670 0x7f03640b4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:13.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.505+0000 7f036259c700 1 -- 172.21.15.47:0/3057493043 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03640b2740 msgr2=0x7f03640b55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:13.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.505+0000 7f036259c700 1 --2- 172.21.15.47:0/3057493043 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03640b2740 0x7f03640b55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:13.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.505+0000 7f036259c700 1 -- 172.21.15.47:0/3057493043 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f035c009470 con 0x7f03640aa9b0 2022-06-29T15:52:13.520 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.505+0000 7f036259c700 1 --2- 172.21.15.47:0/3057493043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03640aa9b0 0x7f0364146940 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 rx=0x7f03540018c0 tx=0x7f03540011a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:13.520 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.505+0000 7f03537fe700 1 -- 172.21.15.47:0/3057493043 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0354007170 con 0x7f03640aa9b0 2022-06-29T15:52:13.520 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.505+0000 7f03537fe700 1 -- 172.21.15.47:0/3057493043 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03540072d0 con 0x7f03640aa9b0 2022-06-29T15:52:13.520 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.505+0000 7f036359e700 1 -- 172.21.15.47:0/3057493043 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03640ac2d0 con 0x7f03640aa9b0 2022-06-29T15:52:13.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.505+0000 7f03537fe700 1 -- 172.21.15.47:0/3057493043 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0354004920 con 0x7f03640aa9b0 2022-06-29T15:52:13.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.505+0000 7f036359e700 1 -- 172.21.15.47:0/3057493043 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f036405f4d0 con 0x7f03640aa9b0 2022-06-29T15:52:13.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.509+0000 7f03537fe700 1 -- 172.21.15.47:0/3057493043 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0354004a80 con 0x7f03640aa9b0 2022-06-29T15:52:13.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.509+0000 7f036359e700 1 -- 172.21.15.47:0/3057493043 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03640573d0 con 0x7f03640aa9b0 2022-06-29T15:52:13.522 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.509+0000 7f03537fe700 1 --2- 172.21.15.47:0/3057493043 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0340053aa0 0x7f0340055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:13.523 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.509+0000 7f03537fe700 1 -- 172.21.15.47:0/3057493043 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f035406b990 con 0x7f03640aa9b0 2022-06-29T15:52:13.523 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.509+0000 7f0361d9b700 1 --2- 172.21.15.47:0/3057493043 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0340053aa0 0x7f0340055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:13.523 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.509+0000 7f0361d9b700 1 --2- 172.21.15.47:0/3057493043 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0340053aa0 0x7f0340055f60 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7f035800b000 tx=0x7f0358006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:13.524 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.517+0000 7f03537fe700 1 -- 172.21.15.47:0/3057493043 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f03540429f0 con 0x7f03640aa9b0 2022-06-29T15:52:13.652 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.645+0000 7f036359e700 1 -- 172.21.15.47:0/3057493043 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03640986c0 con 0x7f03640aa9b0 2022-06-29T15:52:13.653 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.645+0000 7f03537fe700 1 -- 172.21.15.47:0/3057493043 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) v1 ==== 74+0+13424 (secure 0 0 0) 0x7f0354046010 con 0x7f03640aa9b0 2022-06-29T15:52:13.654 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:52:13.654 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":158,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:52:13.018442+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:52:13.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.649+0000 7f036359e700 1 -- 172.21.15.47:0/3057493043 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0340053aa0 msgr2=0x7f0340055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:13.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.649+0000 7f036359e700 1 --2- 172.21.15.47:0/3057493043 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0340053aa0 0x7f0340055f60 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7f035800b000 tx=0x7f0358006df0).stop 2022-06-29T15:52:13.656 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.649+0000 7f036359e700 1 -- 172.21.15.47:0/3057493043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03640aa9b0 msgr2=0x7f0364146940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:13.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.649+0000 7f036359e700 1 --2- 172.21.15.47:0/3057493043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03640aa9b0 0x7f0364146940 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 rx=0x7f03540018c0 tx=0x7f03540011a0).stop 2022-06-29T15:52:13.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.649+0000 7f036359e700 1 -- 172.21.15.47:0/3057493043 shutdown_connections 2022-06-29T15:52:13.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.649+0000 7f036359e700 1 --2- 172.21.15.47:0/3057493043 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0340053aa0 0x7f0340055f60 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:13.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.649+0000 7f036359e700 1 --2- 172.21.15.47:0/3057493043 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03640b2740 0x7f03640b55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:13.657 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.649+0000 7f036359e700 1 --2- 172.21.15.47:0/3057493043 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f03640ac670 0x7f03640b4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:13.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.649+0000 7f036359e700 1 --2- 172.21.15.47:0/3057493043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03640aa9b0 0x7f0364146940 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:13.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.649+0000 7f036359e700 1 -- 172.21.15.47:0/3057493043 >> 172.21.15.47:0/3057493043 conn(0x7f0364062df0 msgr2=0x7f0364099ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:13.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.649+0000 7f036359e700 1 -- 172.21.15.47:0/3057493043 shutdown_connections 2022-06-29T15:52:13.658 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.649+0000 7f036359e700 1 -- 172.21.15.47:0/3057493043 wait complete. 2022-06-29T15:52:13.667 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-29T15:42:16.971431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-29T15:42:25.290884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-06-29T15:52:13.668 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-29T15:52:13.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.773+0000 7f9bbc80f700 1 Processor -- start 2022-06-29T15:52:13.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.773+0000 7f9bbc80f700 1 -- start start 2022-06-29T15:52:13.782 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.773+0000 7f9bbc80f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9bb8146720 0x7f9bb8140780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:13.782 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.773+0000 7f9bbc80f700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9bb80b4320 0x7f9bb8143230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:13.782 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.773+0000 7f9bbc80f700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bb8061600 con 0x7f9bb8061220 2022-06-29T15:52:13.782 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.773+0000 7f9bbc80f700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bb8146bc0 con 0x7f9bb8146720 2022-06-29T15:52:13.783 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.773+0000 7f9bbc80f700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb8146e60 con 0x7f9bb80b4320 2022-06-29T15:52:13.783 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.773+0000 7f9bb5d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9bb8146720 0x7f9bb8140780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:13.783 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.773+0000 7f9bb5d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9bb8146720 0x7f9bb8140780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54958/0 (socket says 172.21.15.47:54958) 2022-06-29T15:52:13.783 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.773+0000 7f9bb5d9b700 1 -- 172.21.15.47:0/3071841379 learned_addr learned my addr 172.21.15.47:0/3071841379 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:13.783 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.773+0000 7f9bb6d9d700 1 --2- 172.21.15.47:0/3071841379 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9bb80b4320 0x7f9bb8143230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:13.784 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.773+0000 7f9bb5d9b700 1 -- 172.21.15.47:0/3071841379 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9bb80b4320 msgr2=0x7f9bb8143230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:13.784 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.773+0000 7f9bb5d9b700 1 --2- 172.21.15.47:0/3071841379 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9bb80b4320 0x7f9bb8143230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:13.784 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.773+0000 7f9bb5d9b700 1 -- 172.21.15.47:0/3071841379 >> v1:172.21.15.64:6789/0 conn(0x7f9bb8061220 legacy=0x7f9bb8140070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:13.784 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.773+0000 7f9bb5d9b700 1 -- 172.21.15.47:0/3071841379 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb8147100 con 0x7f9bb8146720 2022-06-29T15:52:13.784 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.773+0000 7f9bb5d9b700 1 --2- 172.21.15.47:0/3071841379 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bb8146720 0x7f9bb8140780 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 rx=0x7f9ba8002f70 tx=0x7f9ba8006eb0).ready entity=mon.0 client_cookie=a7e25d3add269bce server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:13.785 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.773+0000 7f9bb559a700 1 -- 172.21.15.47:0/3071841379 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ba800d5c0 con 0x7f9bb8146720 2022-06-29T15:52:13.785 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.773+0000 7f9bb559a700 1 -- 172.21.15.47:0/3071841379 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ba8000e90 con 0x7f9bb8146720 2022-06-29T15:52:13.785 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.773+0000 7f9bb559a700 1 -- 172.21.15.47:0/3071841379 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ba800d950 con 0x7f9bb8146720 2022-06-29T15:52:13.788 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.773+0000 7f9bbc80f700 1 -- 172.21.15.47:0/3071841379 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bb8146720 msgr2=0x7f9bb8140780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:13.788 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.773+0000 7f9bbc80f700 1 --2- 172.21.15.47:0/3071841379 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bb8146720 0x7f9bb8140780 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 rx=0x7f9ba8002f70 tx=0x7f9ba8006eb0).stop 2022-06-29T15:52:13.788 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.773+0000 7f9bbc80f700 1 -- 172.21.15.47:0/3071841379 shutdown_connections 2022-06-29T15:52:13.788 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.773+0000 7f9bbc80f700 1 --2- 172.21.15.47:0/3071841379 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9bb80b4320 0x7f9bb8143230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:13.789 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.773+0000 7f9bbc80f700 1 --2- 172.21.15.47:0/3071841379 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bb8146720 0x7f9bb8140780 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:13.789 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.773+0000 7f9bbc80f700 1 -- 172.21.15.47:0/3071841379 >> 172.21.15.47:0/3071841379 conn(0x7f9bb805c6b0 msgr2=0x7f9bb805f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:13.789 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.773+0000 7f9bbc80f700 1 -- 172.21.15.47:0/3071841379 shutdown_connections 2022-06-29T15:52:13.789 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.777+0000 7f9bbc80f700 1 -- 172.21.15.47:0/3071841379 wait complete. 2022-06-29T15:52:13.790 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.777+0000 7f9bbc80f700 1 Processor -- start 2022-06-29T15:52:13.790 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.777+0000 7f9bbc80f700 1 -- start start 2022-06-29T15:52:13.790 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.777+0000 7f9bbc80f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bb8061220 0x7f9bb80b3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:13.790 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.777+0000 7f9bbc80f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bb80b4320 0x7f9bb8113740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:13.791 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.777+0000 7f9bbc80f700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9bb8146720 0x7f9bb8113c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:13.791 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.777+0000 7f9bbc80f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb8113310 con 0x7f9bb80b4320 2022-06-29T15:52:13.791 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.777+0000 7f9bbc80f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb8140270 con 0x7f9bb8061220 2022-06-29T15:52:13.791 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.777+0000 7f9bbc80f700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb8140510 con 0x7f9bb8146720 2022-06-29T15:52:13.792 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.777+0000 7f9bb5d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bb80b4320 0x7f9bb8113740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:13.792 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.777+0000 7f9bb6d9d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9bb8146720 0x7f9bb8113c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:13.792 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.777+0000 7f9bb5d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bb80b4320 0x7f9bb8113740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54966/0 (socket says 172.21.15.47:54966) 2022-06-29T15:52:13.792 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.777+0000 7f9bb5d9b700 1 -- 172.21.15.47:0/762951457 learned_addr learned my addr 172.21.15.47:0/762951457 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:13.793 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.777+0000 7f9bb659c700 1 --2- 172.21.15.47:0/762951457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bb8061220 0x7f9bb80b3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:13.793 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.781+0000 7f9bb5d9b700 1 -- 172.21.15.47:0/762951457 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9bb8146720 msgr2=0x7f9bb8113c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:13.793 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.781+0000 7f9bb5d9b700 1 --2- 172.21.15.47:0/762951457 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9bb8146720 0x7f9bb8113c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:13.793 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.781+0000 7f9bb5d9b700 1 -- 172.21.15.47:0/762951457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bb8061220 msgr2=0x7f9bb80b3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:13.794 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.781+0000 7f9bb5d9b700 1 --2- 172.21.15.47:0/762951457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bb8061220 0x7f9bb80b3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:13.794 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.781+0000 7f9bb5d9b700 1 -- 172.21.15.47:0/762951457 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ba800b1e0 con 0x7f9bb80b4320 2022-06-29T15:52:13.794 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.781+0000 7f9bb5d9b700 1 --2- 172.21.15.47:0/762951457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bb80b4320 0x7f9bb8113740 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 rx=0x7f9ba8002f70 tx=0x7f9ba8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:13.794 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.781+0000 7f9baf7fe700 1 -- 172.21.15.47:0/762951457 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ba80116e0 con 0x7f9bb80b4320 2022-06-29T15:52:13.795 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.781+0000 7f9baf7fe700 1 -- 172.21.15.47:0/762951457 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ba8011840 con 0x7f9bb80b4320 2022-06-29T15:52:13.795 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.781+0000 7f9bbc80f700 1 -- 172.21.15.47:0/762951457 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bb80b3710 con 0x7f9bb80b4320 2022-06-29T15:52:13.795 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.781+0000 7f9baf7fe700 1 -- 172.21.15.47:0/762951457 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ba8011ba0 con 0x7f9bb80b4320 2022-06-29T15:52:13.795 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.781+0000 7f9bbc80f700 1 -- 172.21.15.47:0/762951457 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bb81407b0 con 0x7f9bb80b4320 2022-06-29T15:52:13.796 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.781+0000 7f9bbc80f700 1 -- 172.21.15.47:0/762951457 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bb8113480 con 0x7f9bb80b4320 2022-06-29T15:52:13.797 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.781+0000 7f9baf7fe700 1 -- 172.21.15.47:0/762951457 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9ba8025030 con 0x7f9bb80b4320 2022-06-29T15:52:13.797 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.781+0000 7f9baf7fe700 1 --2- 172.21.15.47:0/762951457 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9b9c053a70 0x7f9b9c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:13.797 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.781+0000 7f9baf7fe700 1 -- 172.21.15.47:0/762951457 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f9ba806cf20 con 0x7f9bb80b4320 2022-06-29T15:52:13.797 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.785+0000 7f9bb659c700 1 --2- 172.21.15.47:0/762951457 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9b9c053a70 0x7f9b9c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:13.798 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.785+0000 7f9bb659c700 1 --2- 172.21.15.47:0/762951457 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9b9c053a70 0x7f9b9c055f30 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7f9ba000d000 tx=0x7f9ba0007b90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:13.798 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.789+0000 7f9baf7fe700 1 -- 172.21.15.47:0/762951457 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9bb8113480 con 0x7f9bb80b4320 2022-06-29T15:52:13.919 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.913+0000 7f9bbc80f700 1 -- 172.21.15.47:0/762951457 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f9bb8113450 con 0x7f9bb80b4320 2022-06-29T15:52:13.920 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.913+0000 7f9baf7fe700 1 -- 172.21.15.47:0/762951457 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v158) v1 ==== 110+0+10 (secure 0 0 0) 0x7f9ba8023020 con 0x7f9bb80b4320 2022-06-29T15:52:13.921 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2022-06-29T15:52:13.923 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.913+0000 7f9bbc80f700 1 -- 172.21.15.47:0/762951457 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9b9c053a70 msgr2=0x7f9b9c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:13.923 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.913+0000 7f9bbc80f700 1 --2- 172.21.15.47:0/762951457 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9b9c053a70 0x7f9b9c055f30 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7f9ba000d000 tx=0x7f9ba0007b90).stop 2022-06-29T15:52:13.923 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.913+0000 7f9bbc80f700 1 -- 172.21.15.47:0/762951457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bb80b4320 msgr2=0x7f9bb8113740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:13.923 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.913+0000 7f9bbc80f700 1 --2- 172.21.15.47:0/762951457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bb80b4320 0x7f9bb8113740 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 rx=0x7f9ba8002f70 tx=0x7f9ba8015000).stop 2022-06-29T15:52:13.924 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.913+0000 7f9bbc80f700 1 -- 172.21.15.47:0/762951457 shutdown_connections 2022-06-29T15:52:13.924 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.913+0000 7f9bbc80f700 1 --2- 172.21.15.47:0/762951457 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9bb8146720 0x7f9bb8113c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:13.924 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.913+0000 7f9bbc80f700 1 --2- 172.21.15.47:0/762951457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bb80b4320 0x7f9bb8113740 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:13.925 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.913+0000 7f9bbc80f700 1 --2- 172.21.15.47:0/762951457 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9b9c053a70 0x7f9b9c055f30 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:13.925 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.913+0000 7f9bbc80f700 1 --2- 172.21.15.47:0/762951457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bb8061220 0x7f9bb80b3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:13.925 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.913+0000 7f9bbc80f700 1 -- 172.21.15.47:0/762951457 >> 172.21.15.47:0/762951457 conn(0x7f9bb805c6b0 msgr2=0x7f9bb8144830 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:13.925 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.913+0000 7f9bbc80f700 1 -- 172.21.15.47:0/762951457 shutdown_connections 2022-06-29T15:52:13.925 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:13.913+0000 7f9bbc80f700 1 -- 172.21.15.47:0/762951457 wait complete. 2022-06-29T15:52:13.934 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-29T15:52:14.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.029+0000 7f4d2ad17700 1 Processor -- start 2022-06-29T15:52:14.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.029+0000 7f4d2ad17700 1 -- start start 2022-06-29T15:52:14.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.029+0000 7f4d2ad17700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4d2413e550 0x7f4d24140f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:14.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.029+0000 7f4d2ad17700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4d24067500 0x7f4d2405f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:14.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.029+0000 7f4d2ad17700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d240b4ca0 con 0x7f4d2405f5a0 2022-06-29T15:52:14.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.029+0000 7f4d2ad17700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d240b3a80 con 0x7f4d24067500 2022-06-29T15:52:14.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.029+0000 7f4d2ad17700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d240b37e0 con 0x7f4d2413e550 2022-06-29T15:52:14.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.029+0000 7f4d292b4700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4d2405f5a0 0x7f4d24066ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40526/0 (socket says 172.21.15.47:40526) 2022-06-29T15:52:14.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.029+0000 7f4d23fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4d24067500 0x7f4d2405f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:14.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.029+0000 7f4d292b4700 1 -- 172.21.15.47:0/3676749405 learned_addr learned my addr 172.21.15.47:0/3676749405 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:52:14.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.029+0000 7f4d28ab3700 1 --2- 172.21.15.47:0/3676749405 >> v2:172.21.15.64:3300/0 conn(0x7f4d2413e550 0x7f4d24140f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:14.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.029+0000 7f4d237fe700 1 -- 172.21.15.47:0/3676749405 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2583121900 0 0) 0x7f4d240b4ca0 con 0x7f4d2405f5a0 2022-06-29T15:52:14.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.029+0000 7f4d237fe700 1 -- 172.21.15.47:0/3676749405 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d04003260 con 0x7f4d2405f5a0 2022-06-29T15:52:14.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.029+0000 7f4d23fff700 1 -- 172.21.15.47:0/3676749405 >> v2:172.21.15.64:3300/0 conn(0x7f4d2413e550 msgr2=0x7f4d24140f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:14.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.029+0000 7f4d23fff700 1 --2- 172.21.15.47:0/3676749405 >> v2:172.21.15.64:3300/0 conn(0x7f4d2413e550 0x7f4d24140f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:14.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.029+0000 7f4d23fff700 1 -- 172.21.15.47:0/3676749405 >> v1:172.21.15.47:6789/0 conn(0x7f4d2405f5a0 legacy=0x7f4d24066ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:14.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.029+0000 7f4d23fff700 1 -- 172.21.15.47:0/3676749405 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d240672b0 con 0x7f4d24067500 2022-06-29T15:52:14.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.029+0000 7f4d23fff700 1 --2- 172.21.15.47:0/3676749405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d24067500 0x7f4d2405f0b0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 rx=0x7f4d0c002f70 tx=0x7f4d0c006df0).ready entity=mon.0 client_cookie=b919f139c7f333d3 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:14.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.033+0000 7f4d237fe700 1 -- 172.21.15.47:0/3676749405 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d0c00d4f0 con 0x7f4d24067500 2022-06-29T15:52:14.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.033+0000 7f4d237fe700 1 -- 172.21.15.47:0/3676749405 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d0c000e90 con 0x7f4d24067500 2022-06-29T15:52:14.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.033+0000 7f4d237fe700 1 -- 172.21.15.47:0/3676749405 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d0c00d840 con 0x7f4d24067500 2022-06-29T15:52:14.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.033+0000 7f4d2ad17700 1 -- 172.21.15.47:0/3676749405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d24067500 msgr2=0x7f4d2405f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:14.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.033+0000 7f4d2ad17700 1 --2- 172.21.15.47:0/3676749405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d24067500 0x7f4d2405f0b0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 rx=0x7f4d0c002f70 tx=0x7f4d0c006df0).stop 2022-06-29T15:52:14.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.033+0000 7f4d2ad17700 1 -- 172.21.15.47:0/3676749405 shutdown_connections 2022-06-29T15:52:14.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.033+0000 7f4d2ad17700 1 --2- 172.21.15.47:0/3676749405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d24067500 0x7f4d2405f0b0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:14.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.033+0000 7f4d2ad17700 1 --2- 172.21.15.47:0/3676749405 >> v2:172.21.15.64:3300/0 conn(0x7f4d2413e550 0x7f4d24140f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:14.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.033+0000 7f4d2ad17700 1 -- 172.21.15.47:0/3676749405 >> 172.21.15.47:0/3676749405 conn(0x7f4d240627d0 msgr2=0x7f4d24062bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:14.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.033+0000 7f4d2ad17700 1 -- 172.21.15.47:0/3676749405 shutdown_connections 2022-06-29T15:52:14.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.033+0000 7f4d2ad17700 1 -- 172.21.15.47:0/3676749405 wait complete. 2022-06-29T15:52:14.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.033+0000 7f4d2ad17700 1 Processor -- start 2022-06-29T15:52:14.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.033+0000 7f4d2ad17700 1 -- start start 2022-06-29T15:52:14.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.037+0000 7f4d2ad17700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4d24067500 0x7f4d241400b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:14.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.037+0000 7f4d2ad17700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d2413e550 0x7f4d24153ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:14.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.037+0000 7f4d2ad17700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d240b1330 0x7f4d24153fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:14.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.037+0000 7f4d2ad17700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d240b7320 con 0x7f4d240b1330 2022-06-29T15:52:14.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.037+0000 7f4d2ad17700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d240b01e0 con 0x7f4d2413e550 2022-06-29T15:52:14.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.037+0000 7f4d2ad17700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d240b0700 con 0x7f4d24067500 2022-06-29T15:52:14.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.037+0000 7f4d292b4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d240b1330 0x7f4d24153fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:14.052 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.037+0000 7f4d292b4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d240b1330 0x7f4d24153fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54980/0 (socket says 172.21.15.47:54980) 2022-06-29T15:52:14.052 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.037+0000 7f4d292b4700 1 -- 172.21.15.47:0/3282655856 learned_addr learned my addr 172.21.15.47:0/3282655856 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:14.052 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.037+0000 7f4d28ab3700 1 --2- 172.21.15.47:0/3282655856 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4d24067500 0x7f4d241400b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:14.052 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.037+0000 7f4d23fff700 1 --2- 172.21.15.47:0/3282655856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d2413e550 0x7f4d24153ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:14.053 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.037+0000 7f4d292b4700 1 -- 172.21.15.47:0/3282655856 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4d24067500 msgr2=0x7f4d241400b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:14.053 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.037+0000 7f4d292b4700 1 --2- 172.21.15.47:0/3282655856 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4d24067500 0x7f4d241400b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:14.053 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.037+0000 7f4d292b4700 1 -- 172.21.15.47:0/3282655856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d2413e550 msgr2=0x7f4d24153ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:14.053 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.037+0000 7f4d292b4700 1 --2- 172.21.15.47:0/3282655856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d2413e550 0x7f4d24153ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:14.053 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.037+0000 7f4d292b4700 1 -- 172.21.15.47:0/3282655856 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d0c00b180 con 0x7f4d240b1330 2022-06-29T15:52:14.054 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.037+0000 7f4d292b4700 1 --2- 172.21.15.47:0/3282655856 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d240b1330 0x7f4d24153fd0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 rx=0x7f4d1c001110 tx=0x7f4d1c007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:14.054 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.037+0000 7f4d21ffb700 1 -- 172.21.15.47:0/3282655856 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d1c0070d0 con 0x7f4d240b1330 2022-06-29T15:52:14.054 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.037+0000 7f4d21ffb700 1 -- 172.21.15.47:0/3282655856 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d1c007230 con 0x7f4d240b1330 2022-06-29T15:52:14.054 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.037+0000 7f4d2ad17700 1 -- 172.21.15.47:0/3282655856 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d24150190 con 0x7f4d240b1330 2022-06-29T15:52:14.055 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.037+0000 7f4d21ffb700 1 -- 172.21.15.47:0/3282655856 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d1c005610 con 0x7f4d240b1330 2022-06-29T15:52:14.055 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.037+0000 7f4d2ad17700 1 -- 172.21.15.47:0/3282655856 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d240b0bd0 con 0x7f4d240b1330 2022-06-29T15:52:14.055 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.041+0000 7f4d2ad17700 1 -- 172.21.15.47:0/3282655856 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d240573d0 con 0x7f4d240b1330 2022-06-29T15:52:14.056 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.041+0000 7f4d21ffb700 1 -- 172.21.15.47:0/3282655856 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4d1c01a060 con 0x7f4d240b1330 2022-06-29T15:52:14.057 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.041+0000 7f4d21ffb700 1 --2- 172.21.15.47:0/3282655856 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4d08053980 0x7f4d08055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:14.057 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.041+0000 7f4d21ffb700 1 -- 172.21.15.47:0/3282655856 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f4d1c06b170 con 0x7f4d240b1330 2022-06-29T15:52:14.057 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.041+0000 7f4d28ab3700 1 --2- 172.21.15.47:0/3282655856 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4d08053980 0x7f4d08055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:14.057 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.041+0000 7f4d28ab3700 1 --2- 172.21.15.47:0/3282655856 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4d08053980 0x7f4d08055e40 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7f4d1400b000 tx=0x7f4d140091d0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:14.058 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.049+0000 7f4d21ffb700 1 -- 172.21.15.47:0/3282655856 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4d1c042160 con 0x7f4d240b1330 2022-06-29T15:52:14.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.173+0000 7f4d2ad17700 1 -- 172.21.15.47:0/3282655856 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4d24153790 con 0x7f4d240b1330 2022-06-29T15:52:14.182 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.173+0000 7f4d21ffb700 1 -- 172.21.15.47:0/3282655856 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v158) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4d1c018020 con 0x7f4d240b1330 2022-06-29T15:52:14.183 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:52:14.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.177+0000 7f4d2ad17700 1 -- 172.21.15.47:0/3282655856 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4d08053980 msgr2=0x7f4d08055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:14.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.177+0000 7f4d2ad17700 1 --2- 172.21.15.47:0/3282655856 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4d08053980 0x7f4d08055e40 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7f4d1400b000 tx=0x7f4d140091d0).stop 2022-06-29T15:52:14.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.177+0000 7f4d2ad17700 1 -- 172.21.15.47:0/3282655856 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d240b1330 msgr2=0x7f4d24153fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:14.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.177+0000 7f4d2ad17700 1 --2- 172.21.15.47:0/3282655856 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d240b1330 0x7f4d24153fd0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 rx=0x7f4d1c001110 tx=0x7f4d1c007a40).stop 2022-06-29T15:52:14.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.177+0000 7f4d2ad17700 1 -- 172.21.15.47:0/3282655856 shutdown_connections 2022-06-29T15:52:14.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.177+0000 7f4d2ad17700 1 --2- 172.21.15.47:0/3282655856 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d240b1330 0x7f4d24153fd0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:14.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.177+0000 7f4d2ad17700 1 --2- 172.21.15.47:0/3282655856 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4d08053980 0x7f4d08055e40 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:14.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.177+0000 7f4d2ad17700 1 --2- 172.21.15.47:0/3282655856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d2413e550 0x7f4d24153ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:14.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.177+0000 7f4d2ad17700 1 --2- 172.21.15.47:0/3282655856 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4d24067500 0x7f4d241400b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:14.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.177+0000 7f4d2ad17700 1 -- 172.21.15.47:0/3282655856 >> 172.21.15.47:0/3282655856 conn(0x7f4d240627d0 msgr2=0x7f4d2413cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:14.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.177+0000 7f4d2ad17700 1 -- 172.21.15.47:0/3282655856 shutdown_connections 2022-06-29T15:52:14.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.177+0000 7f4d2ad17700 1 -- 172.21.15.47:0/3282655856 wait complete. 2022-06-29T15:52:14.201 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-06-29T15:52:14.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.301+0000 7f79b18e5700 1 Processor -- start 2022-06-29T15:52:14.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.301+0000 7f79b18e5700 1 -- start start 2022-06-29T15:52:14.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.305+0000 7f79b18e5700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f79ac143bd0 0x7f79ac05c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:14.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.305+0000 7f79b18e5700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79ac142f90 con 0x7f79ac05c510 2022-06-29T15:52:14.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.305+0000 7f79b18e5700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79ac1431f0 con 0x7f79ac0b2c40 2022-06-29T15:52:14.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.305+0000 7f79b18e5700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac143490 con 0x7f79ac143bd0 2022-06-29T15:52:14.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.305+0000 7f79b08e3700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f79ac05c510 0x7f79ac0b4100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40538/0 (socket says 172.21.15.47:40538) 2022-06-29T15:52:14.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.305+0000 7f79b08e3700 1 -- 172.21.15.47:0/2009311403 learned_addr learned my addr 172.21.15.47:0/2009311403 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:52:14.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.305+0000 7f79abfff700 1 --2- 172.21.15.47:0/2009311403 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f79ac143bd0 0x7f79ac05c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:14.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.305+0000 7f79ab7fe700 1 -- 172.21.15.47:0/2009311403 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1084804702 0 0) 0x7f79ac142f90 con 0x7f79ac05c510 2022-06-29T15:52:14.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.305+0000 7f79ab7fe700 1 -- 172.21.15.47:0/2009311403 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f798c003260 con 0x7f79ac05c510 2022-06-29T15:52:14.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.305+0000 7f79ab7fe700 1 -- 172.21.15.47:0/2009311403 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2016685601 0 0) 0x7f79ac1431f0 con 0x7f79ac0b2c40 2022-06-29T15:52:14.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.305+0000 7f79ab7fe700 1 -- 172.21.15.47:0/2009311403 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79ac142f90 con 0x7f79ac0b2c40 2022-06-29T15:52:14.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.305+0000 7f79abfff700 1 -- 172.21.15.47:0/2009311403 >> v1:172.21.15.64:6789/0 conn(0x7f79ac0b2c40 legacy=0x7f79ac13cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:14.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.305+0000 7f79abfff700 1 -- 172.21.15.47:0/2009311403 >> v1:172.21.15.47:6789/0 conn(0x7f79ac05c510 legacy=0x7f79ac0b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:14.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.305+0000 7f79abfff700 1 -- 172.21.15.47:0/2009311403 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79ac143730 con 0x7f79ac143bd0 2022-06-29T15:52:14.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.305+0000 7f79abfff700 1 --2- 172.21.15.47:0/2009311403 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f79ac143bd0 0x7f79ac05c8f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f7994002f70 tx=0x7f7994006df0).ready entity=mon.2 client_cookie=97e5d97e86bf016b server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:14.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.305+0000 7f79ab7fe700 1 -- 172.21.15.47:0/2009311403 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f799400d5c0 con 0x7f79ac143bd0 2022-06-29T15:52:14.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.305+0000 7f79ab7fe700 1 -- 172.21.15.47:0/2009311403 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7994000e90 con 0x7f79ac143bd0 2022-06-29T15:52:14.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.305+0000 7f79ab7fe700 1 -- 172.21.15.47:0/2009311403 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f799400d910 con 0x7f79ac143bd0 2022-06-29T15:52:14.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.305+0000 7f79b18e5700 1 -- 172.21.15.47:0/2009311403 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f79ac143bd0 msgr2=0x7f79ac05c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:14.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.305+0000 7f79b18e5700 1 --2- 172.21.15.47:0/2009311403 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f79ac143bd0 0x7f79ac05c8f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f7994002f70 tx=0x7f7994006df0).stop 2022-06-29T15:52:14.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.305+0000 7f79b18e5700 1 -- 172.21.15.47:0/2009311403 shutdown_connections 2022-06-29T15:52:14.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.305+0000 7f79b18e5700 1 --2- 172.21.15.47:0/2009311403 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f79ac143bd0 0x7f79ac05c8f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:14.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.305+0000 7f79b18e5700 1 -- 172.21.15.47:0/2009311403 >> 172.21.15.47:0/2009311403 conn(0x7f79ac060e00 msgr2=0x7f79ac0611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:14.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.305+0000 7f79b18e5700 1 -- 172.21.15.47:0/2009311403 shutdown_connections 2022-06-29T15:52:14.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.305+0000 7f79b18e5700 1 -- 172.21.15.47:0/2009311403 wait complete. 2022-06-29T15:52:14.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.309+0000 7f79b18e5700 1 Processor -- start 2022-06-29T15:52:14.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.309+0000 7f79b18e5700 1 -- start start 2022-06-29T15:52:14.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.309+0000 7f79b18e5700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f79ac05c510 0x7f79ac0af460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:14.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.309+0000 7f79b18e5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79ac0b2c40 0x7f79ac144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:14.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.309+0000 7f79b18e5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79ac143bd0 0x7f79ac13ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:14.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.309+0000 7f79b18e5700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac156520 con 0x7f79ac0b2c40 2022-06-29T15:52:14.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.309+0000 7f79b18e5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac147430 con 0x7f79ac143bd0 2022-06-29T15:52:14.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.309+0000 7f79b18e5700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac0b58d0 con 0x7f79ac05c510 2022-06-29T15:52:14.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.309+0000 7f79abfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79ac0b2c40 0x7f79ac144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:14.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.309+0000 7f79abfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79ac0b2c40 0x7f79ac144cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54992/0 (socket says 172.21.15.47:54992) 2022-06-29T15:52:14.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.309+0000 7f79abfff700 1 -- 172.21.15.47:0/1690398825 learned_addr learned my addr 172.21.15.47:0/1690398825 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:14.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.309+0000 7f79b08e3700 1 --2- 172.21.15.47:0/1690398825 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f79ac05c510 0x7f79ac0af460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:14.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.309+0000 7f79b10e4700 1 --2- 172.21.15.47:0/1690398825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79ac143bd0 0x7f79ac13ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:14.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.309+0000 7f79abfff700 1 -- 172.21.15.47:0/1690398825 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f79ac05c510 msgr2=0x7f79ac0af460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:14.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.309+0000 7f79abfff700 1 --2- 172.21.15.47:0/1690398825 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f79ac05c510 0x7f79ac0af460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:14.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.309+0000 7f79abfff700 1 -- 172.21.15.47:0/1690398825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79ac143bd0 msgr2=0x7f79ac13ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:14.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.309+0000 7f79abfff700 1 --2- 172.21.15.47:0/1690398825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79ac143bd0 0x7f79ac13ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:14.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.309+0000 7f79abfff700 1 -- 172.21.15.47:0/1690398825 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f799400b180 con 0x7f79ac0b2c40 2022-06-29T15:52:14.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.309+0000 7f79b10e4700 1 --2- 172.21.15.47:0/1690398825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79ac143bd0 0x7f79ac13ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:52:14.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.309+0000 7f79abfff700 1 --2- 172.21.15.47:0/1690398825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79ac0b2c40 0x7f79ac144cd0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 rx=0x7f79ac145f60 tx=0x7f7994014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:14.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.309+0000 7f79a9ffb700 1 -- 172.21.15.47:0/1690398825 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7994016070 con 0x7f79ac0b2c40 2022-06-29T15:52:14.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.313+0000 7f79a9ffb700 1 -- 172.21.15.47:0/1690398825 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7994013850 con 0x7f79ac0b2c40 2022-06-29T15:52:14.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.313+0000 7f79b18e5700 1 -- 172.21.15.47:0/1690398825 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79ac0b50f0 con 0x7f79ac0b2c40 2022-06-29T15:52:14.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.313+0000 7f79a9ffb700 1 -- 172.21.15.47:0/1690398825 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7994013bb0 con 0x7f79ac0b2c40 2022-06-29T15:52:14.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.313+0000 7f79b18e5700 1 -- 172.21.15.47:0/1690398825 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79ac0b5630 con 0x7f79ac0b2c40 2022-06-29T15:52:14.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.313+0000 7f79b18e5700 1 -- 172.21.15.47:0/1690398825 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79ac144530 con 0x7f79ac0b2c40 2022-06-29T15:52:14.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.313+0000 7f79a9ffb700 1 -- 172.21.15.47:0/1690398825 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7994025030 con 0x7f79ac0b2c40 2022-06-29T15:52:14.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.313+0000 7f79a9ffb700 1 --2- 172.21.15.47:0/1690398825 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7990053b00 0x7f7990055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:14.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.313+0000 7f79a9ffb700 1 -- 172.21.15.47:0/1690398825 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f799406cb60 con 0x7f79ac0b2c40 2022-06-29T15:52:14.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.313+0000 7f79b08e3700 1 --2- 172.21.15.47:0/1690398825 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7990053b00 0x7f7990055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:14.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.313+0000 7f79b08e3700 1 --2- 172.21.15.47:0/1690398825 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7990053b00 0x7f7990055fc0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7f799c00d000 tx=0x7f799c009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:14.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.321+0000 7f79a9ffb700 1 -- 172.21.15.47:0/1690398825 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7994043bc0 con 0x7f79ac0b2c40 2022-06-29T15:52:14.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.449+0000 7f79b18e5700 1 -- 172.21.15.47:0/1690398825 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f79ac0573d0 con 0x7f79ac0b2c40 2022-06-29T15:52:14.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.453+0000 7f79a9ffb700 1 -- 172.21.15.47:0/1690398825 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v158) v1 ==== 84+0+143 (secure 0 0 0) 0x7f7994023070 con 0x7f79ac0b2c40 2022-06-29T15:52:14.460 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:52:14.460 INFO:teuthology.orchestra.run.smithi047.stdout:[ 2022-06-29T15:52:14.460 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:14.460 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 1, 2022-06-29T15:52:14.461 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": "device_health_metrics" 2022-06-29T15:52:14.461 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:14.461 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:14.461 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 2, 2022-06-29T15:52:14.462 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": "rbd" 2022-06-29T15:52:14.462 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:14.462 INFO:teuthology.orchestra.run.smithi047.stdout:] 2022-06-29T15:52:14.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.453+0000 7f79b18e5700 1 -- 172.21.15.47:0/1690398825 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7990053b00 msgr2=0x7f7990055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:14.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.453+0000 7f79b18e5700 1 --2- 172.21.15.47:0/1690398825 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7990053b00 0x7f7990055fc0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7f799c00d000 tx=0x7f799c009040).stop 2022-06-29T15:52:14.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.453+0000 7f79b18e5700 1 -- 172.21.15.47:0/1690398825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79ac0b2c40 msgr2=0x7f79ac144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:14.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.453+0000 7f79b18e5700 1 --2- 172.21.15.47:0/1690398825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79ac0b2c40 0x7f79ac144cd0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 rx=0x7f79ac145f60 tx=0x7f7994014000).stop 2022-06-29T15:52:14.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.453+0000 7f79b18e5700 1 -- 172.21.15.47:0/1690398825 shutdown_connections 2022-06-29T15:52:14.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.453+0000 7f79b18e5700 1 --2- 172.21.15.47:0/1690398825 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7990053b00 0x7f7990055fc0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:14.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.453+0000 7f79b18e5700 1 --2- 172.21.15.47:0/1690398825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79ac143bd0 0x7f79ac13ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:14.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.453+0000 7f79b18e5700 1 --2- 172.21.15.47:0/1690398825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79ac0b2c40 0x7f79ac144cd0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:14.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.453+0000 7f79b18e5700 1 --2- 172.21.15.47:0/1690398825 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f79ac05c510 0x7f79ac0af460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:14.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.453+0000 7f79b18e5700 1 -- 172.21.15.47:0/1690398825 >> 172.21.15.47:0/1690398825 conn(0x7f79ac060e00 msgr2=0x7f79ac0b7a30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:14.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.453+0000 7f79b18e5700 1 -- 172.21.15.47:0/1690398825 shutdown_connections 2022-06-29T15:52:14.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.453+0000 7f79b18e5700 1 -- 172.21.15.47:0/1690398825 wait complete. 2022-06-29T15:52:14.473 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-06-29T15:52:14.473 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-06-29T15:52:14.585 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.577+0000 7f359b59e700 1 Processor -- start 2022-06-29T15:52:14.586 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.577+0000 7f359b59e700 1 -- start start 2022-06-29T15:52:14.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.577+0000 7f359b59e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f359c05c850 0x7f359c143a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:14.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.577+0000 7f359b59e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f359c0676a0 0x7f359c140130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:14.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.577+0000 7f359b59e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f359c05cc30 con 0x7f359c0b5d90 2022-06-29T15:52:14.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.577+0000 7f359b59e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f359c1430a0 con 0x7f359c0676a0 2022-06-29T15:52:14.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.577+0000 7f359b59e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f359c143300 con 0x7f359c05c850 2022-06-29T15:52:14.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.577+0000 7f3599d9b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f359c0b5d90 0x7f359c0b4860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40552/0 (socket says 172.21.15.47:40552) 2022-06-29T15:52:14.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.577+0000 7f359ad9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f359c0676a0 0x7f359c140130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:14.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.577+0000 7f3599d9b700 1 -- 172.21.15.47:0/1209986614 learned_addr learned my addr 172.21.15.47:0/1209986614 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:52:14.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.581+0000 7f359a59c700 1 --2- 172.21.15.47:0/1209986614 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f359c05c850 0x7f359c143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:14.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.581+0000 7f359959a700 1 -- 172.21.15.47:0/1209986614 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 842312238 0 0) 0x7f359c05cc30 con 0x7f359c0b5d90 2022-06-29T15:52:14.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.581+0000 7f359959a700 1 -- 172.21.15.47:0/1209986614 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f357c003260 con 0x7f359c0b5d90 2022-06-29T15:52:14.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.581+0000 7f359ad9d700 1 -- 172.21.15.47:0/1209986614 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f359c05c850 msgr2=0x7f359c143a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:14.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.581+0000 7f359ad9d700 1 --2- 172.21.15.47:0/1209986614 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f359c05c850 0x7f359c143a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:14.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.581+0000 7f359ad9d700 1 -- 172.21.15.47:0/1209986614 >> v1:172.21.15.47:6789/0 conn(0x7f359c0b5d90 legacy=0x7f359c0b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:14.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.581+0000 7f359ad9d700 1 -- 172.21.15.47:0/1209986614 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f359c1435a0 con 0x7f359c0676a0 2022-06-29T15:52:14.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.581+0000 7f359a59c700 1 --2- 172.21.15.47:0/1209986614 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f359c05c850 0x7f359c143a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:52:14.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.581+0000 7f359ad9d700 1 --2- 172.21.15.47:0/1209986614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f359c0676a0 0x7f359c140130 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 rx=0x7f35940032f0 tx=0x7f35940091d0).ready entity=mon.0 client_cookie=24c9fd56ff07692f server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:14.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.581+0000 7f359959a700 1 -- 172.21.15.47:0/1209986614 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f359400b6c0 con 0x7f359c0676a0 2022-06-29T15:52:14.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.581+0000 7f359959a700 1 -- 172.21.15.47:0/1209986614 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f359400b820 con 0x7f359c0676a0 2022-06-29T15:52:14.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.581+0000 7f359959a700 1 -- 172.21.15.47:0/1209986614 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f359400b990 con 0x7f359c0676a0 2022-06-29T15:52:14.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.581+0000 7f359b59e700 1 -- 172.21.15.47:0/1209986614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f359c0676a0 msgr2=0x7f359c140130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:14.597 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.581+0000 7f359b59e700 1 --2- 172.21.15.47:0/1209986614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f359c0676a0 0x7f359c140130 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 rx=0x7f35940032f0 tx=0x7f35940091d0).stop 2022-06-29T15:52:14.597 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.581+0000 7f359b59e700 1 -- 172.21.15.47:0/1209986614 shutdown_connections 2022-06-29T15:52:14.597 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.581+0000 7f359b59e700 1 --2- 172.21.15.47:0/1209986614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f359c0676a0 0x7f359c140130 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:14.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.581+0000 7f359b59e700 1 --2- 172.21.15.47:0/1209986614 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f359c05c850 0x7f359c143a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:14.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.581+0000 7f359b59e700 1 -- 172.21.15.47:0/1209986614 >> 172.21.15.47:0/1209986614 conn(0x7f359c060e00 msgr2=0x7f359c0611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:14.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.581+0000 7f359b59e700 1 -- 172.21.15.47:0/1209986614 shutdown_connections 2022-06-29T15:52:14.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.581+0000 7f359b59e700 1 -- 172.21.15.47:0/1209986614 wait complete. 2022-06-29T15:52:14.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.585+0000 7f359b59e700 1 Processor -- start 2022-06-29T15:52:14.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.585+0000 7f359b59e700 1 -- start start 2022-06-29T15:52:14.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.585+0000 7f359b59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f359c05c850 0x7f359c0b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:14.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.585+0000 7f359b59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f359c0676a0 0x7f359c12aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:14.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.585+0000 7f359b59e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f359c0b5d90 0x7f359c12b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:14.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.585+0000 7f359b59e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359c0b0fe0 con 0x7f359c0676a0 2022-06-29T15:52:14.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.585+0000 7f359b59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359c0b1210 con 0x7f359c05c850 2022-06-29T15:52:14.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.585+0000 7f359b59e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f359c152c70 con 0x7f359c0b5d90 2022-06-29T15:52:14.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.585+0000 7f3599d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f359c0676a0 0x7f359c12aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:14.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.585+0000 7f3599d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f359c0676a0 0x7f359c12aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55004/0 (socket says 172.21.15.47:55004) 2022-06-29T15:52:14.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.585+0000 7f3599d9b700 1 -- 172.21.15.47:0/2268454272 learned_addr learned my addr 172.21.15.47:0/2268454272 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:14.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.585+0000 7f359a59c700 1 --2- 172.21.15.47:0/2268454272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f359c05c850 0x7f359c0b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:14.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.585+0000 7f359ad9d700 1 --2- 172.21.15.47:0/2268454272 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f359c0b5d90 0x7f359c12b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:14.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.585+0000 7f3599d9b700 1 -- 172.21.15.47:0/2268454272 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f359c0b5d90 msgr2=0x7f359c12b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:14.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.585+0000 7f3599d9b700 1 --2- 172.21.15.47:0/2268454272 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f359c0b5d90 0x7f359c12b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:14.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.585+0000 7f3599d9b700 1 -- 172.21.15.47:0/2268454272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f359c05c850 msgr2=0x7f359c0b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:14.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.585+0000 7f3599d9b700 1 --2- 172.21.15.47:0/2268454272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f359c05c850 0x7f359c0b3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:14.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.585+0000 7f3599d9b700 1 -- 172.21.15.47:0/2268454272 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3594009470 con 0x7f359c0676a0 2022-06-29T15:52:14.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.585+0000 7f3599d9b700 1 --2- 172.21.15.47:0/2268454272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f359c0676a0 0x7f359c12aee0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 rx=0x7f3584001110 tx=0x7f3584007aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:14.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.589+0000 7f35937fe700 1 -- 172.21.15.47:0/2268454272 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35840070d0 con 0x7f359c0676a0 2022-06-29T15:52:14.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.589+0000 7f35937fe700 1 -- 172.21.15.47:0/2268454272 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3584007230 con 0x7f359c0676a0 2022-06-29T15:52:14.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.589+0000 7f35937fe700 1 -- 172.21.15.47:0/2268454272 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3584005610 con 0x7f359c0676a0 2022-06-29T15:52:14.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.589+0000 7f359b59e700 1 -- 172.21.15.47:0/2268454272 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f359c152f10 con 0x7f359c0676a0 2022-06-29T15:52:14.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.589+0000 7f359b59e700 1 -- 172.21.15.47:0/2268454272 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f359c12b8c0 con 0x7f359c0676a0 2022-06-29T15:52:14.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.589+0000 7f35937fe700 1 -- 172.21.15.47:0/2268454272 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3584010070 con 0x7f359c0676a0 2022-06-29T15:52:14.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.589+0000 7f359b59e700 1 -- 172.21.15.47:0/2268454272 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f359c1530a0 con 0x7f359c0676a0 2022-06-29T15:52:14.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.589+0000 7f35937fe700 1 --2- 172.21.15.47:0/2268454272 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3580053b30 0x7f3580055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:14.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.589+0000 7f35937fe700 1 -- 172.21.15.47:0/2268454272 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f358406b310 con 0x7f359c0676a0 2022-06-29T15:52:14.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.589+0000 7f359a59c700 1 --2- 172.21.15.47:0/2268454272 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3580053b30 0x7f3580055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:14.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.589+0000 7f359a59c700 1 --2- 172.21.15.47:0/2268454272 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3580053b30 0x7f3580055ff0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7f358c00c000 tx=0x7f358c008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:14.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.597+0000 7f35937fe700 1 -- 172.21.15.47:0/2268454272 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3584042410 con 0x7f359c0676a0 2022-06-29T15:52:14.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:14.725+0000 7f359b59e700 1 -- 172.21.15.47:0/2268454272 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f359c0573d0 con 0x7f359c0676a0 2022-06-29T15:52:15.696 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.689+0000 7f35937fe700 1 -- 172.21.15.47:0/2268454272 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v159) v1 ==== 133+0+0 (secure 0 0 0) 0x7f3584045a30 con 0x7f359c0676a0 2022-06-29T15:52:15.700 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.693+0000 7f359b59e700 1 -- 172.21.15.47:0/2268454272 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3580053b30 msgr2=0x7f3580055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:15.701 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.693+0000 7f359b59e700 1 --2- 172.21.15.47:0/2268454272 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3580053b30 0x7f3580055ff0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7f358c00c000 tx=0x7f358c008040).stop 2022-06-29T15:52:15.701 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.693+0000 7f359b59e700 1 -- 172.21.15.47:0/2268454272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f359c0676a0 msgr2=0x7f359c12aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:15.702 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.693+0000 7f359b59e700 1 --2- 172.21.15.47:0/2268454272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f359c0676a0 0x7f359c12aee0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 rx=0x7f3584001110 tx=0x7f3584007aa0).stop 2022-06-29T15:52:15.702 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.693+0000 7f359b59e700 1 -- 172.21.15.47:0/2268454272 shutdown_connections 2022-06-29T15:52:15.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.693+0000 7f359b59e700 1 --2- 172.21.15.47:0/2268454272 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f359c0b5d90 0x7f359c12b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:15.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.693+0000 7f359b59e700 1 --2- 172.21.15.47:0/2268454272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f359c0676a0 0x7f359c12aee0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:15.704 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.693+0000 7f359b59e700 1 --2- 172.21.15.47:0/2268454272 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3580053b30 0x7f3580055ff0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:15.704 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.693+0000 7f359b59e700 1 --2- 172.21.15.47:0/2268454272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f359c05c850 0x7f359c0b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:15.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.693+0000 7f359b59e700 1 -- 172.21.15.47:0/2268454272 >> 172.21.15.47:0/2268454272 conn(0x7f359c060e00 msgr2=0x7f359c1423d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:15.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.693+0000 7f359b59e700 1 -- 172.21.15.47:0/2268454272 shutdown_connections 2022-06-29T15:52:15.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.693+0000 7f359b59e700 1 -- 172.21.15.47:0/2268454272 wait complete. 2022-06-29T15:52:15.707 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' created 2022-06-29T15:52:15.715 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-06-29T15:52:15.831 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.821+0000 7f4d16dc0700 1 Processor -- start 2022-06-29T15:52:15.831 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.825+0000 7f4d16dc0700 1 -- start start 2022-06-29T15:52:15.833 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.825+0000 7f4d16dc0700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4d100b2ad0 0x7f4d10140ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:15.834 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.825+0000 7f4d16dc0700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d101472a0 con 0x7f4d100a94f0 2022-06-29T15:52:15.834 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.825+0000 7f4d16dc0700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d10147540 con 0x7f4d100608d0 2022-06-29T15:52:15.834 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.825+0000 7f4d16dc0700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d100b01b0 con 0x7f4d100b2ad0 2022-06-29T15:52:15.835 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.825+0000 7f4d1535d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4d100b2ad0 0x7f4d10140ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:15.835 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.825+0000 7f4d0ffff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4d100a94f0 0x7f4d10143220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40566/0 (socket says 172.21.15.47:40566) 2022-06-29T15:52:15.835 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.825+0000 7f4d0ffff700 1 -- 172.21.15.47:0/2024723936 learned_addr learned my addr 172.21.15.47:0/2024723936 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:52:15.835 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.825+0000 7f4d1535d700 1 -- 172.21.15.47:0/2024723936 >> v1:172.21.15.64:6789/0 conn(0x7f4d100608d0 legacy=0x7f4d101403b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:15.836 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.825+0000 7f4d1535d700 1 -- 172.21.15.47:0/2024723936 >> v1:172.21.15.47:6789/0 conn(0x7f4d100a94f0 legacy=0x7f4d10143220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:15.836 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.825+0000 7f4d0f7fe700 1 -- 172.21.15.47:0/2024723936 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 118742606 0 0) 0x7f4d101472a0 con 0x7f4d100a94f0 2022-06-29T15:52:15.836 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.825+0000 7f4d1535d700 1 -- 172.21.15.47:0/2024723936 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d100a8d80 con 0x7f4d100b2ad0 2022-06-29T15:52:15.836 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.825+0000 7f4d1535d700 1 --2- 172.21.15.47:0/2024723936 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d100b2ad0 0x7f4d10140ac0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 rx=0x7f4d100b2f70 tx=0x7f4d00006df0).ready entity=mon.0 client_cookie=8243e884e79f54f6 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:15.837 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.825+0000 7f4d0f7fe700 1 -- 172.21.15.47:0/2024723936 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d0000d610 con 0x7f4d100b2ad0 2022-06-29T15:52:15.837 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.825+0000 7f4d0f7fe700 1 -- 172.21.15.47:0/2024723936 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d00000e90 con 0x7f4d100b2ad0 2022-06-29T15:52:15.837 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.825+0000 7f4d0f7fe700 1 -- 172.21.15.47:0/2024723936 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d0000d960 con 0x7f4d100b2ad0 2022-06-29T15:52:15.841 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.825+0000 7f4d16dc0700 1 -- 172.21.15.47:0/2024723936 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d100b2ad0 msgr2=0x7f4d10140ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:15.841 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.825+0000 7f4d16dc0700 1 --2- 172.21.15.47:0/2024723936 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d100b2ad0 0x7f4d10140ac0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 rx=0x7f4d100b2f70 tx=0x7f4d00006df0).stop 2022-06-29T15:52:15.841 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.825+0000 7f4d16dc0700 1 -- 172.21.15.47:0/2024723936 shutdown_connections 2022-06-29T15:52:15.842 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.825+0000 7f4d16dc0700 1 --2- 172.21.15.47:0/2024723936 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d100b2ad0 0x7f4d10140ac0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:15.842 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.825+0000 7f4d16dc0700 1 -- 172.21.15.47:0/2024723936 >> 172.21.15.47:0/2024723936 conn(0x7f4d100626f0 msgr2=0x7f4d10062ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:15.842 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.825+0000 7f4d16dc0700 1 -- 172.21.15.47:0/2024723936 shutdown_connections 2022-06-29T15:52:15.842 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.825+0000 7f4d16dc0700 1 -- 172.21.15.47:0/2024723936 wait complete. 2022-06-29T15:52:15.843 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.829+0000 7f4d16dc0700 1 Processor -- start 2022-06-29T15:52:15.843 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.829+0000 7f4d16dc0700 1 -- start start 2022-06-29T15:52:15.843 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.829+0000 7f4d16dc0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d100608d0 0x7f4d1013da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:15.843 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.829+0000 7f4d16dc0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d100a94f0 0x7f4d1013b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:15.844 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.829+0000 7f4d16dc0700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4d100b2ad0 0x7f4d1013bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:15.844 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.829+0000 7f4d16dc0700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d100b5520 con 0x7f4d100608d0 2022-06-29T15:52:15.844 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.829+0000 7f4d16dc0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d10143e00 con 0x7f4d100a94f0 2022-06-29T15:52:15.844 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.829+0000 7f4d16dc0700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d10140fe0 con 0x7f4d100b2ad0 2022-06-29T15:52:15.844 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.829+0000 7f4d14b5c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d100608d0 0x7f4d1013da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:15.845 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.829+0000 7f4d14b5c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d100608d0 0x7f4d1013da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55016/0 (socket says 172.21.15.47:55016) 2022-06-29T15:52:15.845 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.829+0000 7f4d14b5c700 1 -- 172.21.15.47:0/3186101021 learned_addr learned my addr 172.21.15.47:0/3186101021 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:15.845 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.829+0000 7f4d0ffff700 1 --2- 172.21.15.47:0/3186101021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d100a94f0 0x7f4d1013b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:15.845 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.829+0000 7f4d1535d700 1 --2- 172.21.15.47:0/3186101021 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4d100b2ad0 0x7f4d1013bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:15.846 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.829+0000 7f4d14b5c700 1 -- 172.21.15.47:0/3186101021 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4d100b2ad0 msgr2=0x7f4d1013bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:15.846 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.829+0000 7f4d14b5c700 1 --2- 172.21.15.47:0/3186101021 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4d100b2ad0 0x7f4d1013bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:15.846 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.829+0000 7f4d14b5c700 1 -- 172.21.15.47:0/3186101021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d100a94f0 msgr2=0x7f4d1013b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:15.846 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.829+0000 7f4d14b5c700 1 --2- 172.21.15.47:0/3186101021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d100a94f0 0x7f4d1013b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:15.847 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.829+0000 7f4d14b5c700 1 -- 172.21.15.47:0/3186101021 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d0000b180 con 0x7f4d100608d0 2022-06-29T15:52:15.847 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.829+0000 7f4d14b5c700 1 --2- 172.21.15.47:0/3186101021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d100608d0 0x7f4d1013da70 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 rx=0x7f4d10151fe0 tx=0x7f4d080014a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:15.847 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.829+0000 7f4d0dffb700 1 -- 172.21.15.47:0/3186101021 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d08010510 con 0x7f4d100608d0 2022-06-29T15:52:15.847 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.829+0000 7f4d0dffb700 1 -- 172.21.15.47:0/3186101021 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d08010670 con 0x7f4d100608d0 2022-06-29T15:52:15.848 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.829+0000 7f4d0dffb700 1 -- 172.21.15.47:0/3186101021 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d080107e0 con 0x7f4d100608d0 2022-06-29T15:52:15.848 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.829+0000 7f4d16dc0700 1 -- 172.21.15.47:0/3186101021 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d100b31d0 con 0x7f4d100608d0 2022-06-29T15:52:15.848 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.833+0000 7f4d16dc0700 1 -- 172.21.15.47:0/3186101021 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d10152bb0 con 0x7f4d100608d0 2022-06-29T15:52:15.848 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.833+0000 7f4d16dc0700 1 -- 172.21.15.47:0/3186101021 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d100b35a0 con 0x7f4d100608d0 2022-06-29T15:52:15.848 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.833+0000 7f4d0dffb700 1 -- 172.21.15.47:0/3186101021 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4d08010070 con 0x7f4d100608d0 2022-06-29T15:52:15.849 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.833+0000 7f4d0dffb700 1 --2- 172.21.15.47:0/3186101021 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4cf4053950 0x7f4cf4055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:15.850 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.833+0000 7f4d0dffb700 1 -- 172.21.15.47:0/3186101021 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5954+0+0 (secure 0 0 0) 0x7f4d0806bba0 con 0x7f4d100608d0 2022-06-29T15:52:15.850 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.833+0000 7f4d0ffff700 1 --2- 172.21.15.47:0/3186101021 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4cf4053950 0x7f4cf4055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:15.850 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.837+0000 7f4d0ffff700 1 --2- 172.21.15.47:0/3186101021 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4cf4053950 0x7f4cf4055e10 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 rx=0x7f4cf800b000 tx=0x7f4cf8009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:15.850 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.841+0000 7f4d0dffb700 1 -- 172.21.15.47:0/3186101021 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4d08042a90 con 0x7f4d100608d0 2022-06-29T15:52:15.974 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:15.965+0000 7f4d16dc0700 1 -- 172.21.15.47:0/3186101021 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f4d100573d0 con 0x7f4d100608d0 2022-06-29T15:52:16.708 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.701+0000 7f4d0dffb700 1 -- 172.21.15.47:0/3186101021 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v160) v1 ==== 125+0+0 (secure 0 0 0) 0x7f4d080460b0 con 0x7f4d100608d0 2022-06-29T15:52:16.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.705+0000 7f4d16dc0700 1 -- 172.21.15.47:0/3186101021 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4cf4053950 msgr2=0x7f4cf4055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:16.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.705+0000 7f4d16dc0700 1 --2- 172.21.15.47:0/3186101021 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4cf4053950 0x7f4cf4055e10 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 rx=0x7f4cf800b000 tx=0x7f4cf8009040).stop 2022-06-29T15:52:16.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.705+0000 7f4d16dc0700 1 -- 172.21.15.47:0/3186101021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d100608d0 msgr2=0x7f4d1013da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:16.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.705+0000 7f4d16dc0700 1 --2- 172.21.15.47:0/3186101021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d100608d0 0x7f4d1013da70 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 rx=0x7f4d10151fe0 tx=0x7f4d080014a0).stop 2022-06-29T15:52:16.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.709+0000 7f4d16dc0700 1 -- 172.21.15.47:0/3186101021 shutdown_connections 2022-06-29T15:52:16.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.709+0000 7f4d16dc0700 1 --2- 172.21.15.47:0/3186101021 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4d100b2ad0 0x7f4d1013bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:16.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.709+0000 7f4d16dc0700 1 --2- 172.21.15.47:0/3186101021 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4cf4053950 0x7f4cf4055e10 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:16.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.709+0000 7f4d16dc0700 1 --2- 172.21.15.47:0/3186101021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d100a94f0 0x7f4d1013b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:16.718 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.709+0000 7f4d16dc0700 1 --2- 172.21.15.47:0/3186101021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d100608d0 0x7f4d1013da70 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:16.718 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.709+0000 7f4d16dc0700 1 -- 172.21.15.47:0/3186101021 >> 172.21.15.47:0/3186101021 conn(0x7f4d100626f0 msgr2=0x7f4d101404f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:16.718 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.709+0000 7f4d16dc0700 1 -- 172.21.15.47:0/3186101021 shutdown_connections 2022-06-29T15:52:16.718 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.709+0000 7f4d16dc0700 1 -- 172.21.15.47:0/3186101021 wait complete. 2022-06-29T15:52:16.719 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' created 2022-06-29T15:52:16.729 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-06-29T15:52:16.845 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.837+0000 7f81b7fff700 1 Processor -- start 2022-06-29T15:52:16.845 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.837+0000 7f81b7fff700 1 -- start start 2022-06-29T15:52:16.846 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.837+0000 7f81b7fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f81b805f0b0 0x7f81b805f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:16.847 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.837+0000 7f81b7fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f81b8061740 0x7f81b8147140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:16.847 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.837+0000 7f81b7fff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81b8061e50 con 0x7f81b8144670 2022-06-29T15:52:16.847 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.837+0000 7f81b7fff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81b8147630 con 0x7f81b805f0b0 2022-06-29T15:52:16.847 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.837+0000 7f81b7fff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81b80620f0 con 0x7f81b8061740 2022-06-29T15:52:16.848 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.837+0000 7f81b77fe700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f81b8061740 0x7f81b8147140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:16.848 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.837+0000 7f81b77fe700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f81b8061740 0x7f81b8147140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:52130/0 (socket says 172.21.15.47:52130) 2022-06-29T15:52:16.848 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.837+0000 7f81b77fe700 1 -- 172.21.15.47:0/2689218273 learned_addr learned my addr 172.21.15.47:0/2689218273 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:16.849 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.837+0000 7f81b6ffd700 1 --2- 172.21.15.47:0/2689218273 >> v2:172.21.15.64:3300/0 conn(0x7f81b805f0b0 0x7f81b805f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:16.849 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.837+0000 7f81b67fc700 1 --1- 172.21.15.47:0/2689218273 >> v1:172.21.15.64:6789/0 conn(0x7f81b8144670 0x7f81b8143950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:48144/0 (socket says 172.21.15.47:48144) 2022-06-29T15:52:16.851 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.841+0000 7f81b77fe700 1 -- 172.21.15.47:0/2689218273 >> v2:172.21.15.64:3300/0 conn(0x7f81b805f0b0 msgr2=0x7f81b805f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:16.851 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.841+0000 7f81b77fe700 1 --2- 172.21.15.47:0/2689218273 >> v2:172.21.15.64:3300/0 conn(0x7f81b805f0b0 0x7f81b805f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:16.851 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.841+0000 7f81b77fe700 1 -- 172.21.15.47:0/2689218273 >> v1:172.21.15.64:6789/0 conn(0x7f81b8144670 legacy=0x7f81b8143950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:16.851 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.841+0000 7f81b77fe700 1 -- 172.21.15.47:0/2689218273 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b81434b0 con 0x7f81b8061740 2022-06-29T15:52:16.852 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.841+0000 7f81b77fe700 1 --2- 172.21.15.47:0/2689218273 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f81b8061740 0x7f81b8147140 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f81b00032f0 tx=0x7f81b0009290).ready entity=mon.2 client_cookie=48f2641a1f4b8ef0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:16.852 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.841+0000 7f81b5ffb700 1 -- 172.21.15.47:0/2689218273 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81b000b8a0 con 0x7f81b8061740 2022-06-29T15:52:16.852 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.841+0000 7f81b5ffb700 1 -- 172.21.15.47:0/2689218273 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81b000ba00 con 0x7f81b8061740 2022-06-29T15:52:16.852 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.841+0000 7f81b5ffb700 1 -- 172.21.15.47:0/2689218273 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81b000bb70 con 0x7f81b8061740 2022-06-29T15:52:16.853 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.841+0000 7f81b7fff700 1 -- 172.21.15.47:0/2689218273 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f81b8061740 msgr2=0x7f81b8147140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:16.853 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.841+0000 7f81b7fff700 1 --2- 172.21.15.47:0/2689218273 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f81b8061740 0x7f81b8147140 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f81b00032f0 tx=0x7f81b0009290).stop 2022-06-29T15:52:16.853 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.841+0000 7f81b7fff700 1 -- 172.21.15.47:0/2689218273 shutdown_connections 2022-06-29T15:52:16.853 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.841+0000 7f81b7fff700 1 --2- 172.21.15.47:0/2689218273 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f81b8061740 0x7f81b8147140 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:16.853 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.841+0000 7f81b7fff700 1 --2- 172.21.15.47:0/2689218273 >> v2:172.21.15.64:3300/0 conn(0x7f81b805f0b0 0x7f81b805f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:16.854 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.841+0000 7f81b7fff700 1 -- 172.21.15.47:0/2689218273 >> 172.21.15.47:0/2689218273 conn(0x7f81b8062570 msgr2=0x7f81b80647d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:16.854 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.841+0000 7f81b7fff700 1 -- 172.21.15.47:0/2689218273 shutdown_connections 2022-06-29T15:52:16.854 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.841+0000 7f81b7fff700 1 -- 172.21.15.47:0/2689218273 wait complete. 2022-06-29T15:52:16.854 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.841+0000 7f81b7fff700 1 Processor -- start 2022-06-29T15:52:16.855 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.841+0000 7f81b7fff700 1 -- start start 2022-06-29T15:52:16.857 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.845+0000 7f81b7fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f81b805f0b0 0x7f81b8066ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:16.858 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.845+0000 7f81b7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81b8061740 0x7f81b8152a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:16.858 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.845+0000 7f81b7fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81b8144670 0x7f81b8152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:16.858 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.845+0000 7f81b7fff700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b81523e0 con 0x7f81b8144670 2022-06-29T15:52:16.859 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.845+0000 7f81b7fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b80a3700 con 0x7f81b8061740 2022-06-29T15:52:16.859 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.845+0000 7f81b7fff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81b80a3460 con 0x7f81b805f0b0 2022-06-29T15:52:16.859 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.845+0000 7f81b77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81b8144670 0x7f81b8152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:16.859 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.845+0000 7f81b77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81b8144670 0x7f81b8152f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55034/0 (socket says 172.21.15.47:55034) 2022-06-29T15:52:16.859 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.845+0000 7f81b6ffd700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f81b805f0b0 0x7f81b8066ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:16.860 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.845+0000 7f81b77fe700 1 -- 172.21.15.47:0/95479949 learned_addr learned my addr 172.21.15.47:0/95479949 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:16.860 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.845+0000 7f81b67fc700 1 --2- 172.21.15.47:0/95479949 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81b8061740 0x7f81b8152a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:16.860 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.845+0000 7f81b77fe700 1 -- 172.21.15.47:0/95479949 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f81b805f0b0 msgr2=0x7f81b8066ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:16.860 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.845+0000 7f81b77fe700 1 --2- 172.21.15.47:0/95479949 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f81b805f0b0 0x7f81b8066ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:16.861 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.845+0000 7f81b77fe700 1 -- 172.21.15.47:0/95479949 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81b8061740 msgr2=0x7f81b8152a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:16.861 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.845+0000 7f81b77fe700 1 --2- 172.21.15.47:0/95479949 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81b8061740 0x7f81b8152a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:16.861 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.845+0000 7f81b77fe700 1 -- 172.21.15.47:0/95479949 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b0009530 con 0x7f81b8144670 2022-06-29T15:52:16.861 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.845+0000 7f81b77fe700 1 --2- 172.21.15.47:0/95479949 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81b8144670 0x7f81b8152f20 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 rx=0x7f81b00032f0 tx=0x7f81b0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:16.862 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.845+0000 7f81affff700 1 -- 172.21.15.47:0/95479949 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81b000b8a0 con 0x7f81b8144670 2022-06-29T15:52:16.862 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.845+0000 7f81b7fff700 1 -- 172.21.15.47:0/95479949 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81b80a5050 con 0x7f81b8144670 2022-06-29T15:52:16.862 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.845+0000 7f81affff700 1 -- 172.21.15.47:0/95479949 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81b0021440 con 0x7f81b8144670 2022-06-29T15:52:16.862 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.845+0000 7f81affff700 1 -- 172.21.15.47:0/95479949 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81b00217a0 con 0x7f81b8144670 2022-06-29T15:52:16.863 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.845+0000 7f81b7fff700 1 -- 172.21.15.47:0/95479949 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81b80b7d60 con 0x7f81b8144670 2022-06-29T15:52:16.863 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.849+0000 7f81b7fff700 1 -- 172.21.15.47:0/95479949 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81b80573d0 con 0x7f81b8144670 2022-06-29T15:52:16.863 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.849+0000 7f81affff700 1 -- 172.21.15.47:0/95479949 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f81b0029070 con 0x7f81b8144670 2022-06-29T15:52:16.863 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.849+0000 7f81affff700 1 --2- 172.21.15.47:0/95479949 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f819c053950 0x7f819c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:16.863 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.849+0000 7f81affff700 1 -- 172.21.15.47:0/95479949 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f81b006d160 con 0x7f81b8144670 2022-06-29T15:52:16.864 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.849+0000 7f81b6ffd700 1 --2- 172.21.15.47:0/95479949 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f819c053950 0x7f819c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:16.864 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.849+0000 7f81b6ffd700 1 --2- 172.21.15.47:0/95479949 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f819c053950 0x7f819c055e10 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7f81a000b000 tx=0x7f81a0006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:16.865 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:16.857+0000 7f81affff700 1 -- 172.21.15.47:0/95479949 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f81b0043f00 con 0x7f81b8144670 2022-06-29T15:52:17.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.025+0000 7f81b7fff700 1 -- 172.21.15.47:0/95479949 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f81b8068830 con 0x7f81b8144670 2022-06-29T15:52:17.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.037+0000 7f81affff700 1 -- 172.21.15.47:0/95479949 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 29 and data pool 30 v78) v1 ==== 189+0+0 (secure 0 0 0) 0x7f81b00150d0 con 0x7f81b8144670 2022-06-29T15:52:17.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.041+0000 7f81b7fff700 1 -- 172.21.15.47:0/95479949 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f819c053950 msgr2=0x7f819c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:17.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.041+0000 7f81b7fff700 1 --2- 172.21.15.47:0/95479949 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f819c053950 0x7f819c055e10 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7f81a000b000 tx=0x7f81a0006df0).stop 2022-06-29T15:52:17.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.041+0000 7f81b7fff700 1 -- 172.21.15.47:0/95479949 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81b8144670 msgr2=0x7f81b8152f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:17.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.041+0000 7f81b7fff700 1 --2- 172.21.15.47:0/95479949 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81b8144670 0x7f81b8152f20 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 rx=0x7f81b00032f0 tx=0x7f81b0016000).stop 2022-06-29T15:52:17.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.041+0000 7f81b7fff700 1 -- 172.21.15.47:0/95479949 shutdown_connections 2022-06-29T15:52:17.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.041+0000 7f81b7fff700 1 --2- 172.21.15.47:0/95479949 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81b8144670 0x7f81b8152f20 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:17.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.041+0000 7f81b7fff700 1 --2- 172.21.15.47:0/95479949 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f819c053950 0x7f819c055e10 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:17.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.041+0000 7f81b7fff700 1 --2- 172.21.15.47:0/95479949 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81b8061740 0x7f81b8152a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:17.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.041+0000 7f81b7fff700 1 --2- 172.21.15.47:0/95479949 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f81b805f0b0 0x7f81b8066ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:17.052 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.041+0000 7f81b7fff700 1 -- 172.21.15.47:0/95479949 >> 172.21.15.47:0/95479949 conn(0x7f81b8062570 msgr2=0x7f81b813faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:17.052 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.041+0000 7f81b7fff700 1 -- 172.21.15.47:0/95479949 shutdown_connections 2022-06-29T15:52:17.053 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.041+0000 7f81b7fff700 1 -- 172.21.15.47:0/95479949 wait complete. 2022-06-29T15:52:17.055 INFO:teuthology.orchestra.run.smithi047.stderr:new fs with metadata pool 29 and data pool 30 2022-06-29T15:52:17.066 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:52:17.159 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.149+0000 7f4ec1915700 1 Processor -- start 2022-06-29T15:52:17.159 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.149+0000 7f4ec1915700 1 -- start start 2022-06-29T15:52:17.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.153+0000 7f4ec1915700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4ebc05c8f0 0x7f4ebc13cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:17.161 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.153+0000 7f4ec1915700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ebc142f90 con 0x7f4ebc143bd0 2022-06-29T15:52:17.161 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.153+0000 7f4ec1915700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ebc1431f0 con 0x7f4ebc05c510 2022-06-29T15:52:17.161 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.153+0000 7f4ec1915700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ebc143490 con 0x7f4ebc05c8f0 2022-06-29T15:52:17.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.153+0000 7f4ec1114700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4ebc05c8f0 0x7f4ebc13cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:17.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.153+0000 7f4ec1114700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4ebc05c8f0 0x7f4ebc13cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55042/0 (socket says 172.21.15.47:55042) 2022-06-29T15:52:17.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.153+0000 7f4ec1114700 1 -- 172.21.15.47:0/1039302992 learned_addr learned my addr 172.21.15.47:0/1039302992 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:17.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.153+0000 7f4ec1114700 1 -- 172.21.15.47:0/1039302992 >> v1:172.21.15.64:6789/0 conn(0x7f4ebc05c510 legacy=0x7f4ebc0b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:17.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.153+0000 7f4ec1114700 1 -- 172.21.15.47:0/1039302992 >> v1:172.21.15.47:6789/0 conn(0x7f4ebc143bd0 legacy=0x7f4ebc0b2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:17.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.153+0000 7f4ec1114700 1 -- 172.21.15.47:0/1039302992 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ebc143730 con 0x7f4ebc05c8f0 2022-06-29T15:52:17.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.153+0000 7f4ebb7fe700 1 -- 172.21.15.47:0/1039302992 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 319030934 0 0) 0x7f4ebc142f90 con 0x7f4ebc143bd0 2022-06-29T15:52:17.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.153+0000 7f4ec1114700 1 --2- 172.21.15.47:0/1039302992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ebc05c8f0 0x7f4ebc13cbf0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 rx=0x7f4eb40032f0 tx=0x7f4eb4009290).ready entity=mon.0 client_cookie=36586acef43787f server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:17.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.153+0000 7f4ebb7fe700 1 -- 172.21.15.47:0/1039302992 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4eb400b8a0 con 0x7f4ebc05c8f0 2022-06-29T15:52:17.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.153+0000 7f4ebb7fe700 1 -- 172.21.15.47:0/1039302992 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eb400ba00 con 0x7f4ebc05c8f0 2022-06-29T15:52:17.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.153+0000 7f4ebb7fe700 1 -- 172.21.15.47:0/1039302992 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4eb400bb70 con 0x7f4ebc05c8f0 2022-06-29T15:52:17.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.153+0000 7f4ec1915700 1 -- 172.21.15.47:0/1039302992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ebc05c8f0 msgr2=0x7f4ebc13cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:17.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.153+0000 7f4ec1915700 1 --2- 172.21.15.47:0/1039302992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ebc05c8f0 0x7f4ebc13cbf0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 rx=0x7f4eb40032f0 tx=0x7f4eb4009290).stop 2022-06-29T15:52:17.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.153+0000 7f4ec1915700 1 -- 172.21.15.47:0/1039302992 shutdown_connections 2022-06-29T15:52:17.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.153+0000 7f4ec1915700 1 --2- 172.21.15.47:0/1039302992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ebc05c8f0 0x7f4ebc13cbf0 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:17.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.153+0000 7f4ec1915700 1 -- 172.21.15.47:0/1039302992 >> 172.21.15.47:0/1039302992 conn(0x7f4ebc060e00 msgr2=0x7f4ebc0611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:17.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.153+0000 7f4ec1915700 1 -- 172.21.15.47:0/1039302992 shutdown_connections 2022-06-29T15:52:17.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.153+0000 7f4ec1915700 1 -- 172.21.15.47:0/1039302992 wait complete. 2022-06-29T15:52:17.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.157+0000 7f4ec1915700 1 Processor -- start 2022-06-29T15:52:17.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.157+0000 7f4ec1915700 1 -- start start 2022-06-29T15:52:17.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.157+0000 7f4ec1915700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4ebc05c510 0x7f4ebc0ad670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:17.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.157+0000 7f4ec1915700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ebc05c8f0 0x7f4ebc0b5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:17.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.157+0000 7f4ec1915700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ebc143bd0 0x7f4ebc1529a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:17.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.157+0000 7f4ec1915700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ebc1473d0 con 0x7f4ebc05c8f0 2022-06-29T15:52:17.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.157+0000 7f4ec1915700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ebc1522a0 con 0x7f4ebc143bd0 2022-06-29T15:52:17.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.157+0000 7f4ec1915700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ebc0b0220 con 0x7f4ebc05c510 2022-06-29T15:52:17.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.157+0000 7f4ebbfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ebc05c8f0 0x7f4ebc0b5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:17.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.157+0000 7f4ebbfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ebc05c8f0 0x7f4ebc0b5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55046/0 (socket says 172.21.15.47:55046) 2022-06-29T15:52:17.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.157+0000 7f4ebbfff700 1 -- 172.21.15.47:0/656169759 learned_addr learned my addr 172.21.15.47:0/656169759 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:17.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.157+0000 7f4ec1114700 1 --2- 172.21.15.47:0/656169759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ebc143bd0 0x7f4ebc1529a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:17.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.157+0000 7f4ec0913700 1 --2- 172.21.15.47:0/656169759 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4ebc05c510 0x7f4ebc0ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:17.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.157+0000 7f4ebbfff700 1 -- 172.21.15.47:0/656169759 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4ebc05c510 msgr2=0x7f4ebc0ad670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:17.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.157+0000 7f4ebbfff700 1 --2- 172.21.15.47:0/656169759 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4ebc05c510 0x7f4ebc0ad670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:17.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.157+0000 7f4ebbfff700 1 -- 172.21.15.47:0/656169759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ebc143bd0 msgr2=0x7f4ebc1529a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:17.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.157+0000 7f4ebbfff700 1 --2- 172.21.15.47:0/656169759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ebc143bd0 0x7f4ebc1529a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:17.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.157+0000 7f4ebbfff700 1 -- 172.21.15.47:0/656169759 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb4009530 con 0x7f4ebc05c8f0 2022-06-29T15:52:17.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.157+0000 7f4ebbfff700 1 --2- 172.21.15.47:0/656169759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ebc05c8f0 0x7f4ebc0b5fd0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 rx=0x7f4ebc0b03b0 tx=0x7f4ea4007ab0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:17.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.161+0000 7f4eb9ffb700 1 -- 172.21.15.47:0/656169759 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ea4012400 con 0x7f4ebc05c8f0 2022-06-29T15:52:17.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.161+0000 7f4eb9ffb700 1 -- 172.21.15.47:0/656169759 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ea4012560 con 0x7f4ebc05c8f0 2022-06-29T15:52:17.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.161+0000 7f4ec1915700 1 -- 172.21.15.47:0/656169759 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ebc0aff80 con 0x7f4ebc05c8f0 2022-06-29T15:52:17.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.161+0000 7f4eb9ffb700 1 -- 172.21.15.47:0/656169759 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ea40128c0 con 0x7f4ebc05c8f0 2022-06-29T15:52:17.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.161+0000 7f4ec1915700 1 -- 172.21.15.47:0/656169759 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ebc0af040 con 0x7f4ebc05c8f0 2022-06-29T15:52:17.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.161+0000 7f4ec1915700 1 -- 172.21.15.47:0/656169759 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ebc0573d0 con 0x7f4ebc05c8f0 2022-06-29T15:52:17.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.161+0000 7f4eb9ffb700 1 -- 172.21.15.47:0/656169759 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4ea4019030 con 0x7f4ebc05c8f0 2022-06-29T15:52:17.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.161+0000 7f4eb9ffb700 1 --2- 172.21.15.47:0/656169759 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4ea0053b00 0x7f4ea0055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:17.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.161+0000 7f4eb9ffb700 1 -- 172.21.15.47:0/656169759 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f4ea406b450 con 0x7f4ebc05c8f0 2022-06-29T15:52:17.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.161+0000 7f4ec0913700 1 --2- 172.21.15.47:0/656169759 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4ea0053b00 0x7f4ea0055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:17.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.165+0000 7f4ec0913700 1 --2- 172.21.15.47:0/656169759 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4ea0053b00 0x7f4ea0055fc0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 rx=0x7f4eac00b000 tx=0x7f4eac009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:17.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.165+0000 7f4eb9ffb700 1 -- 172.21.15.47:0/656169759 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4ea4042220 con 0x7f4ebc05c8f0 2022-06-29T15:52:17.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.293+0000 7f4ec1915700 1 -- 172.21.15.47:0/656169759 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ebc061b00 con 0x7f4ebc05c8f0 2022-06-29T15:52:17.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.297+0000 7f4eb9ffb700 1 -- 172.21.15.47:0/656169759 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v161) v1 ==== 74+0+16298 (secure 0 0 0) 0x7f4ea4024fc0 con 0x7f4ebc05c8f0 2022-06-29T15:52:17.304 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:52:17.304 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":161,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:52:17.031214+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:52:14.780743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2022-06-29T15:52:16.023272+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:52:17.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.297+0000 7f4ec1915700 1 -- 172.21.15.47:0/656169759 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4ea0053b00 msgr2=0x7f4ea0055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:17.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.297+0000 7f4ec1915700 1 --2- 172.21.15.47:0/656169759 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4ea0053b00 0x7f4ea0055fc0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 rx=0x7f4eac00b000 tx=0x7f4eac009040).stop 2022-06-29T15:52:17.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.297+0000 7f4ec1915700 1 -- 172.21.15.47:0/656169759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ebc05c8f0 msgr2=0x7f4ebc0b5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:17.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.297+0000 7f4ec1915700 1 --2- 172.21.15.47:0/656169759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ebc05c8f0 0x7f4ebc0b5fd0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 rx=0x7f4ebc0b03b0 tx=0x7f4ea4007ab0).stop 2022-06-29T15:52:17.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.297+0000 7f4ec1915700 1 -- 172.21.15.47:0/656169759 shutdown_connections 2022-06-29T15:52:17.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.297+0000 7f4ec1915700 1 --2- 172.21.15.47:0/656169759 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4ea0053b00 0x7f4ea0055fc0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:17.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.297+0000 7f4ec1915700 1 --2- 172.21.15.47:0/656169759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ebc143bd0 0x7f4ebc1529a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:17.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.297+0000 7f4ec1915700 1 --2- 172.21.15.47:0/656169759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ebc05c8f0 0x7f4ebc0b5fd0 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:17.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.297+0000 7f4ec1915700 1 --2- 172.21.15.47:0/656169759 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f4ebc05c510 0x7f4ebc0ad670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:17.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.297+0000 7f4ec1915700 1 -- 172.21.15.47:0/656169759 >> 172.21.15.47:0/656169759 conn(0x7f4ebc060e00 msgr2=0x7f4ebc141aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:17.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.297+0000 7f4ec1915700 1 -- 172.21.15.47:0/656169759 shutdown_connections 2022-06-29T15:52:17.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.297+0000 7f4ec1915700 1 -- 172.21.15.47:0/656169759 wait complete. 2022-06-29T15:52:17.317 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:52:17.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.413+0000 7f47d9dd5700 1 Processor -- start 2022-06-29T15:52:17.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.413+0000 7f47d9dd5700 1 -- start start 2022-06-29T15:52:17.426 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.417+0000 7f47d9dd5700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f47d40b2120 0x7f47d40b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:17.427 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.417+0000 7f47d9dd5700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47d405f0b0 0x7f47d414b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:17.427 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.417+0000 7f47d9dd5700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f47d40b1260 0x7f47d4147af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:17.427 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.417+0000 7f47d9dd5700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47d4147460 con 0x7f47d40b2120 2022-06-29T15:52:17.427 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.417+0000 7f47d9dd5700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47d40b0740 con 0x7f47d40b1260 2022-06-29T15:52:17.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.417+0000 7f47d9dd5700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47d40b0970 con 0x7f47d405f0b0 2022-06-29T15:52:17.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.417+0000 7f47d37fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f47d40b2120 0x7f47d40b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:17.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.417+0000 7f47d37fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f47d40b2120 0x7f47d40b2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55054/0 (socket says 172.21.15.47:55054) 2022-06-29T15:52:17.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.417+0000 7f47d2ffd700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47d405f0b0 0x7f47d414b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:17.429 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.417+0000 7f47d37fe700 1 -- 172.21.15.47:0/3139521412 learned_addr learned my addr 172.21.15.47:0/3139521412 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:17.429 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.417+0000 7f47d3fff700 1 --2- 172.21.15.47:0/3139521412 >> v2:172.21.15.64:3300/0 conn(0x7f47d40b1260 0x7f47d4147af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:17.429 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.417+0000 7f47d37fe700 1 -- 172.21.15.47:0/3139521412 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47d405f0b0 msgr2=0x7f47d414b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:17.429 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.417+0000 7f47d37fe700 1 --2- 172.21.15.47:0/3139521412 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47d405f0b0 0x7f47d414b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:17.430 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.417+0000 7f47d37fe700 1 -- 172.21.15.47:0/3139521412 >> v2:172.21.15.64:3300/0 conn(0x7f47d40b1260 msgr2=0x7f47d4147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:17.430 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.417+0000 7f47d37fe700 1 --2- 172.21.15.47:0/3139521412 >> v2:172.21.15.64:3300/0 conn(0x7f47d40b1260 0x7f47d4147af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:17.430 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.417+0000 7f47d37fe700 1 -- 172.21.15.47:0/3139521412 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d40b0c10 con 0x7f47d40b2120 2022-06-29T15:52:17.430 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.417+0000 7f47d2ffd700 1 --2- 172.21.15.47:0/3139521412 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47d405f0b0 0x7f47d414b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:52:17.431 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.417+0000 7f47d3fff700 1 --2- 172.21.15.47:0/3139521412 >> v2:172.21.15.64:3300/0 conn(0x7f47d40b1260 0x7f47d4147af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:52:17.431 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.417+0000 7f47d37fe700 1 --2- 172.21.15.47:0/3139521412 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47d40b2120 0x7f47d40b2500 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 rx=0x7f47c8002f70 tx=0x7f47c8006eb0).ready entity=mon.0 client_cookie=441930e6934f6294 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:17.431 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.417+0000 7f47d27fc700 1 -- 172.21.15.47:0/3139521412 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47c800d570 con 0x7f47d40b2120 2022-06-29T15:52:17.431 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.417+0000 7f47d27fc700 1 -- 172.21.15.47:0/3139521412 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c8000e90 con 0x7f47d40b2120 2022-06-29T15:52:17.432 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.417+0000 7f47d27fc700 1 -- 172.21.15.47:0/3139521412 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47c800d900 con 0x7f47d40b2120 2022-06-29T15:52:17.432 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.417+0000 7f47d9dd5700 1 -- 172.21.15.47:0/3139521412 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47d40b2120 msgr2=0x7f47d40b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:17.432 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.417+0000 7f47d9dd5700 1 --2- 172.21.15.47:0/3139521412 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47d40b2120 0x7f47d40b2500 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 rx=0x7f47c8002f70 tx=0x7f47c8006eb0).stop 2022-06-29T15:52:17.432 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.417+0000 7f47d9dd5700 1 -- 172.21.15.47:0/3139521412 shutdown_connections 2022-06-29T15:52:17.432 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.417+0000 7f47d9dd5700 1 --2- 172.21.15.47:0/3139521412 >> v2:172.21.15.64:3300/0 conn(0x7f47d40b1260 0x7f47d4147af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:17.433 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.417+0000 7f47d9dd5700 1 --2- 172.21.15.47:0/3139521412 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47d405f0b0 0x7f47d414b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:17.433 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.417+0000 7f47d9dd5700 1 --2- 172.21.15.47:0/3139521412 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47d40b2120 0x7f47d40b2500 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:17.433 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.417+0000 7f47d9dd5700 1 -- 172.21.15.47:0/3139521412 >> 172.21.15.47:0/3139521412 conn(0x7f47d40626f0 msgr2=0x7f47d4062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:17.433 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.417+0000 7f47d9dd5700 1 -- 172.21.15.47:0/3139521412 shutdown_connections 2022-06-29T15:52:17.434 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.417+0000 7f47d9dd5700 1 -- 172.21.15.47:0/3139521412 wait complete. 2022-06-29T15:52:17.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.421+0000 7f47d9dd5700 1 Processor -- start 2022-06-29T15:52:17.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.421+0000 7f47d9dd5700 1 -- start start 2022-06-29T15:52:17.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.421+0000 7f47d9dd5700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47d40b1260 0x7f47d40a25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:17.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.421+0000 7f47d9dd5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47d40b2120 0x7f47d4159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:17.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.421+0000 7f47d9dd5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47d40af580 0x7f47d4141a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:17.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.421+0000 7f47d9dd5700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d41434e0 con 0x7f47d40b2120 2022-06-29T15:52:17.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.421+0000 7f47d9dd5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d4143240 con 0x7f47d40af580 2022-06-29T15:52:17.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.421+0000 7f47d9dd5700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47d4144a00 con 0x7f47d40b1260 2022-06-29T15:52:17.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.421+0000 7f47d2ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47d40b2120 0x7f47d4159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:17.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.421+0000 7f47d2ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47d40b2120 0x7f47d4159cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55060/0 (socket says 172.21.15.47:55060) 2022-06-29T15:52:17.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.421+0000 7f47d2ffd700 1 -- 172.21.15.47:0/1639020526 learned_addr learned my addr 172.21.15.47:0/1639020526 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:17.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.421+0000 7f47d3fff700 1 --2- 172.21.15.47:0/1639020526 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47d40af580 0x7f47d4141a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:17.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.421+0000 7f47d37fe700 1 --2- 172.21.15.47:0/1639020526 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47d40b1260 0x7f47d40a25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:17.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.425+0000 7f47d2ffd700 1 -- 172.21.15.47:0/1639020526 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47d40b1260 msgr2=0x7f47d40a25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:17.441 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.425+0000 7f47d2ffd700 1 --2- 172.21.15.47:0/1639020526 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47d40b1260 0x7f47d40a25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:17.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.425+0000 7f47d2ffd700 1 -- 172.21.15.47:0/1639020526 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47d40af580 msgr2=0x7f47d4141a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:17.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.425+0000 7f47d2ffd700 1 --2- 172.21.15.47:0/1639020526 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47d40af580 0x7f47d4141a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:17.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.425+0000 7f47d2ffd700 1 -- 172.21.15.47:0/1639020526 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c400b180 con 0x7f47d40b2120 2022-06-29T15:52:17.442 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.425+0000 7f47d2ffd700 1 --2- 172.21.15.47:0/1639020526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47d40b2120 0x7f47d4159cd0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 rx=0x7f47c400c000 tx=0x7f47c4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:17.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.425+0000 7f47d0ff9700 1 -- 172.21.15.47:0/1639020526 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47c400c890 con 0x7f47d40b2120 2022-06-29T15:52:17.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.425+0000 7f47d0ff9700 1 -- 172.21.15.47:0/1639020526 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c400c9f0 con 0x7f47d40b2120 2022-06-29T15:52:17.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.425+0000 7f47d9dd5700 1 -- 172.21.15.47:0/1639020526 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47cc009470 con 0x7f47d40b2120 2022-06-29T15:52:17.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.425+0000 7f47d0ff9700 1 -- 172.21.15.47:0/1639020526 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47c400cd50 con 0x7f47d40b2120 2022-06-29T15:52:17.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.425+0000 7f47d9dd5700 1 -- 172.21.15.47:0/1639020526 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c800b1e0 con 0x7f47d40b2120 2022-06-29T15:52:17.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.425+0000 7f47d0ff9700 1 -- 172.21.15.47:0/1639020526 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f47c401c460 con 0x7f47d40b2120 2022-06-29T15:52:17.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.425+0000 7f47d9dd5700 1 -- 172.21.15.47:0/1639020526 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47d40a2ad0 con 0x7f47d40b2120 2022-06-29T15:52:17.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.425+0000 7f47d0ff9700 1 --2- 172.21.15.47:0/1639020526 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f47b4053a70 0x7f47b4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:17.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.425+0000 7f47d0ff9700 1 -- 172.21.15.47:0/1639020526 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f47c406c680 con 0x7f47d40b2120 2022-06-29T15:52:17.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.425+0000 7f47d37fe700 1 --2- 172.21.15.47:0/1639020526 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f47b4053a70 0x7f47b4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:17.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.425+0000 7f47d37fe700 1 --2- 172.21.15.47:0/1639020526 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f47b4053a70 0x7f47b4055f30 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 rx=0x7f47c801c000 tx=0x7f47c8015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:17.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.433+0000 7f47d0ff9700 1 -- 172.21.15.47:0/1639020526 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f47c4043360 con 0x7f47d40b2120 2022-06-29T15:52:17.571 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.565+0000 7f47d9dd5700 1 -- 172.21.15.47:0/1639020526 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f47d40573d0 con 0x7f47d40b2120 2022-06-29T15:52:17.573 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.565+0000 7f47d0ff9700 1 -- 172.21.15.47:0/1639020526 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v161) v1 ==== 74+0+16298 (secure 0 0 0) 0x7f47c4046980 con 0x7f47d40b2120 2022-06-29T15:52:17.573 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:52:17.574 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":161,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:52:17.031214+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:52:14.780743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2022-06-29T15:52:16.023272+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:52:17.576 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.565+0000 7f47d9dd5700 1 -- 172.21.15.47:0/1639020526 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f47b4053a70 msgr2=0x7f47b4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:17.577 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.565+0000 7f47d9dd5700 1 --2- 172.21.15.47:0/1639020526 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f47b4053a70 0x7f47b4055f30 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 rx=0x7f47c801c000 tx=0x7f47c8015000).stop 2022-06-29T15:52:17.577 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.565+0000 7f47d9dd5700 1 -- 172.21.15.47:0/1639020526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47d40b2120 msgr2=0x7f47d4159cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:17.577 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.565+0000 7f47d9dd5700 1 --2- 172.21.15.47:0/1639020526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47d40b2120 0x7f47d4159cd0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 rx=0x7f47c400c000 tx=0x7f47c4006df0).stop 2022-06-29T15:52:17.577 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.569+0000 7f47d9dd5700 1 -- 172.21.15.47:0/1639020526 shutdown_connections 2022-06-29T15:52:17.577 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.569+0000 7f47d9dd5700 1 --2- 172.21.15.47:0/1639020526 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f47b4053a70 0x7f47b4055f30 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:17.578 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.569+0000 7f47d9dd5700 1 --2- 172.21.15.47:0/1639020526 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47d40af580 0x7f47d4141a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:17.578 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.569+0000 7f47d9dd5700 1 --2- 172.21.15.47:0/1639020526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47d40b2120 0x7f47d4159cd0 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:17.578 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.569+0000 7f47d9dd5700 1 --2- 172.21.15.47:0/1639020526 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f47d40b1260 0x7f47d40a25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:17.579 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.569+0000 7f47d9dd5700 1 -- 172.21.15.47:0/1639020526 >> 172.21.15.47:0/1639020526 conn(0x7f47d40626f0 msgr2=0x7f47d414cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:17.579 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.569+0000 7f47d9dd5700 1 -- 172.21.15.47:0/1639020526 shutdown_connections 2022-06-29T15:52:17.579 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.569+0000 7f47d9dd5700 1 -- 172.21.15.47:0/1639020526 wait complete. 2022-06-29T15:52:17.587 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-06-29T15:52:17.700 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.693+0000 7f6d44959700 1 Processor -- start 2022-06-29T15:52:17.700 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.693+0000 7f6d44959700 1 -- start start 2022-06-29T15:52:17.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.693+0000 7f6d44959700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6d400b6050 0x7f6d400aeb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:17.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.693+0000 7f6d44959700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6d400b0720 0x7f6d4013c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:17.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.693+0000 7f6d44959700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d400afda0 con 0x7f6d400af890 2022-06-29T15:52:17.703 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.693+0000 7f6d44959700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d4013f680 con 0x7f6d400b6050 2022-06-29T15:52:17.704 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.693+0000 7f6d44959700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d400b6430 con 0x7f6d400b0720 2022-06-29T15:52:17.704 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.693+0000 7f6d3e59c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6d400b6050 0x7f6d400aeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:17.704 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.693+0000 7f6d3dd9b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f6d400af890 0x7f6d400af080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40622/0 (socket says 172.21.15.47:40622) 2022-06-29T15:52:17.704 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.693+0000 7f6d3dd9b700 1 -- 172.21.15.47:0/4165685060 learned_addr learned my addr 172.21.15.47:0/4165685060 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:52:17.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.693+0000 7f6d3ed9d700 1 --2- 172.21.15.47:0/4165685060 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6d400b0720 0x7f6d4013c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:17.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.693+0000 7f6d3d59a700 1 -- 172.21.15.47:0/4165685060 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1735772513 0 0) 0x7f6d400afda0 con 0x7f6d400af890 2022-06-29T15:52:17.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.693+0000 7f6d3d59a700 1 -- 172.21.15.47:0/4165685060 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d24003260 con 0x7f6d400af890 2022-06-29T15:52:17.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.693+0000 7f6d3e59c700 1 -- 172.21.15.47:0/4165685060 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6d400b0720 msgr2=0x7f6d4013c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:17.705 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.693+0000 7f6d3e59c700 1 --2- 172.21.15.47:0/4165685060 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6d400b0720 0x7f6d4013c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:17.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.693+0000 7f6d3e59c700 1 -- 172.21.15.47:0/4165685060 >> v1:172.21.15.47:6789/0 conn(0x7f6d400af890 legacy=0x7f6d400af080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:17.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.693+0000 7f6d3e59c700 1 -- 172.21.15.47:0/4165685060 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d400b2370 con 0x7f6d400b6050 2022-06-29T15:52:17.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.693+0000 7f6d3ed9d700 1 --2- 172.21.15.47:0/4165685060 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6d400b0720 0x7f6d4013c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:52:17.706 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.693+0000 7f6d3e59c700 1 --2- 172.21.15.47:0/4165685060 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d400b6050 0x7f6d400aeb90 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 rx=0x7f6d30002f70 tx=0x7f6d30006df0).ready entity=mon.0 client_cookie=891a4f3f982abf08 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:17.709 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.697+0000 7f6d3d59a700 1 -- 172.21.15.47:0/4165685060 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d3000d570 con 0x7f6d400b6050 2022-06-29T15:52:17.709 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.697+0000 7f6d3d59a700 1 -- 172.21.15.47:0/4165685060 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d30000e90 con 0x7f6d400b6050 2022-06-29T15:52:17.709 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.697+0000 7f6d3d59a700 1 -- 172.21.15.47:0/4165685060 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d3000d8c0 con 0x7f6d400b6050 2022-06-29T15:52:17.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.697+0000 7f6d44959700 1 -- 172.21.15.47:0/4165685060 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d400b6050 msgr2=0x7f6d400aeb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:17.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.697+0000 7f6d44959700 1 --2- 172.21.15.47:0/4165685060 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d400b6050 0x7f6d400aeb90 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 rx=0x7f6d30002f70 tx=0x7f6d30006df0).stop 2022-06-29T15:52:17.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.697+0000 7f6d44959700 1 -- 172.21.15.47:0/4165685060 shutdown_connections 2022-06-29T15:52:17.710 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.697+0000 7f6d44959700 1 --2- 172.21.15.47:0/4165685060 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6d400b0720 0x7f6d4013c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:17.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.697+0000 7f6d44959700 1 --2- 172.21.15.47:0/4165685060 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d400b6050 0x7f6d400aeb90 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:17.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.697+0000 7f6d44959700 1 -- 172.21.15.47:0/4165685060 >> 172.21.15.47:0/4165685060 conn(0x7f6d400613a0 msgr2=0x7f6d40062f60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:17.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.697+0000 7f6d44959700 1 -- 172.21.15.47:0/4165685060 shutdown_connections 2022-06-29T15:52:17.711 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.697+0000 7f6d44959700 1 -- 172.21.15.47:0/4165685060 wait complete. 2022-06-29T15:52:17.712 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.697+0000 7f6d44959700 1 Processor -- start 2022-06-29T15:52:17.712 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.697+0000 7f6d44959700 1 -- start start 2022-06-29T15:52:17.712 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.701+0000 7f6d44959700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6d400af890 0x7f6d4012ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:17.712 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.701+0000 7f6d44959700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d400b0720 0x7f6d4013c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:17.713 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.701+0000 7f6d44959700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d400b6050 0x7f6d4013ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:17.713 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.701+0000 7f6d44959700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d400b3d30 con 0x7f6d400b6050 2022-06-29T15:52:17.713 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.701+0000 7f6d44959700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d400b5230 con 0x7f6d400b0720 2022-06-29T15:52:17.713 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.701+0000 7f6d44959700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d400b3a90 con 0x7f6d400af890 2022-06-29T15:52:17.714 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.701+0000 7f6d3ed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d400b6050 0x7f6d4013ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:17.714 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.701+0000 7f6d3ed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d400b6050 0x7f6d4013ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55076/0 (socket says 172.21.15.47:55076) 2022-06-29T15:52:17.714 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.701+0000 7f6d3ed9d700 1 -- 172.21.15.47:0/4084331437 learned_addr learned my addr 172.21.15.47:0/4084331437 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:17.714 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.701+0000 7f6d3dd9b700 1 --2- 172.21.15.47:0/4084331437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d400b0720 0x7f6d4013c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:17.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.701+0000 7f6d3e59c700 1 --2- 172.21.15.47:0/4084331437 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6d400af890 0x7f6d4012ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:17.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.701+0000 7f6d3ed9d700 1 -- 172.21.15.47:0/4084331437 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6d400af890 msgr2=0x7f6d4012ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:17.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.701+0000 7f6d3ed9d700 1 --2- 172.21.15.47:0/4084331437 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6d400af890 0x7f6d4012ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:17.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.701+0000 7f6d3ed9d700 1 -- 172.21.15.47:0/4084331437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d400b0720 msgr2=0x7f6d4013c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:17.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.701+0000 7f6d3ed9d700 1 --2- 172.21.15.47:0/4084331437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d400b0720 0x7f6d4013c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:17.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.701+0000 7f6d3ed9d700 1 -- 172.21.15.47:0/4084331437 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d3000b180 con 0x7f6d400b6050 2022-06-29T15:52:17.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.701+0000 7f6d3ed9d700 1 --2- 172.21.15.47:0/4084331437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d400b6050 0x7f6d4013ccc0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 rx=0x7f6d3800a000 tx=0x7f6d38008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:17.718 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.705+0000 7f6d2f7fe700 1 -- 172.21.15.47:0/4084331437 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d3800f710 con 0x7f6d400b6050 2022-06-29T15:52:17.718 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.705+0000 7f6d44959700 1 -- 172.21.15.47:0/4084331437 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d4013c4e0 con 0x7f6d400b6050 2022-06-29T15:52:17.718 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.705+0000 7f6d44959700 1 -- 172.21.15.47:0/4084331437 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d400b4e90 con 0x7f6d400b6050 2022-06-29T15:52:17.718 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.705+0000 7f6d2f7fe700 1 -- 172.21.15.47:0/4084331437 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d38005a70 con 0x7f6d400b6050 2022-06-29T15:52:17.719 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.705+0000 7f6d2f7fe700 1 -- 172.21.15.47:0/4084331437 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d38005dd0 con 0x7f6d400b6050 2022-06-29T15:52:17.719 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.705+0000 7f6d2f7fe700 1 -- 172.21.15.47:0/4084331437 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6d3800f870 con 0x7f6d400b6050 2022-06-29T15:52:17.719 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.705+0000 7f6d44959700 1 -- 172.21.15.47:0/4084331437 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d400b5430 con 0x7f6d400b6050 2022-06-29T15:52:17.719 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.705+0000 7f6d2f7fe700 1 --2- 172.21.15.47:0/4084331437 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6d1c053b00 0x7f6d1c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:17.719 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.705+0000 7f6d2f7fe700 1 -- 172.21.15.47:0/4084331437 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f6d3806be90 con 0x7f6d400b6050 2022-06-29T15:52:17.720 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.705+0000 7f6d3e59c700 1 --2- 172.21.15.47:0/4084331437 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6d1c053b00 0x7f6d1c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:17.720 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.709+0000 7f6d3e59c700 1 --2- 172.21.15.47:0/4084331437 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6d1c053b00 0x7f6d1c055fc0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7f6d3001a000 tx=0x7f6d30014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:17.720 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.713+0000 7f6d2f7fe700 1 -- 172.21.15.47:0/4084331437 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6d38042bf0 con 0x7f6d400b6050 2022-06-29T15:52:17.885 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:17.877+0000 7f6d44959700 1 -- 172.21.15.47:0/4084331437 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f6d40069a60 con 0x7f6d400b6050 2022-06-29T15:52:18.045 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.037+0000 7f6d2f7fe700 1 -- 172.21.15.47:0/4084331437 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v79) v1 ==== 118+0+0 (secure 0 0 0) 0x7f6d38046210 con 0x7f6d400b6050 2022-06-29T15:52:18.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.041+0000 7f6d44959700 1 -- 172.21.15.47:0/4084331437 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6d1c053b00 msgr2=0x7f6d1c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:18.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.041+0000 7f6d44959700 1 --2- 172.21.15.47:0/4084331437 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6d1c053b00 0x7f6d1c055fc0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7f6d3001a000 tx=0x7f6d30014000).stop 2022-06-29T15:52:18.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.041+0000 7f6d44959700 1 -- 172.21.15.47:0/4084331437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d400b6050 msgr2=0x7f6d4013ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:18.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.041+0000 7f6d44959700 1 --2- 172.21.15.47:0/4084331437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d400b6050 0x7f6d4013ccc0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 rx=0x7f6d3800a000 tx=0x7f6d38008040).stop 2022-06-29T15:52:18.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.041+0000 7f6d44959700 1 -- 172.21.15.47:0/4084331437 shutdown_connections 2022-06-29T15:52:18.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.041+0000 7f6d44959700 1 --2- 172.21.15.47:0/4084331437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d400b6050 0x7f6d4013ccc0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:18.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.041+0000 7f6d44959700 1 --2- 172.21.15.47:0/4084331437 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6d1c053b00 0x7f6d1c055fc0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:18.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.041+0000 7f6d44959700 1 --2- 172.21.15.47:0/4084331437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d400b0720 0x7f6d4013c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:18.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.041+0000 7f6d44959700 1 --2- 172.21.15.47:0/4084331437 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6d400af890 0x7f6d4012ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:18.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.041+0000 7f6d44959700 1 -- 172.21.15.47:0/4084331437 >> 172.21.15.47:0/4084331437 conn(0x7f6d400613a0 msgr2=0x7f6d40131390 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:18.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.041+0000 7f6d44959700 1 -- 172.21.15.47:0/4084331437 shutdown_connections 2022-06-29T15:52:18.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.041+0000 7f6d44959700 1 -- 172.21.15.47:0/4084331437 wait complete. 2022-06-29T15:52:18.065 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:52:18.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.161+0000 7fc73b129700 1 Processor -- start 2022-06-29T15:52:18.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.161+0000 7fc73b129700 1 -- start start 2022-06-29T15:52:18.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.165+0000 7fc73b129700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc7340af4a0 0x7fc73405f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:18.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.165+0000 7fc73b129700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc73405f0b0 0x7fc7340b0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:18.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.165+0000 7fc73b129700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7340afe20 con 0x7fc7340ac180 2022-06-29T15:52:18.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.165+0000 7fc73b129700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7340b2830 con 0x7fc7340af4a0 2022-06-29T15:52:18.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.165+0000 7fc73b129700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc734143e30 con 0x7fc73405f0b0 2022-06-29T15:52:18.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.165+0000 7fc738ec5700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc7340af4a0 0x7fc73405f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:18.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.165+0000 7fc738ec5700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc7340af4a0 0x7fc73405f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55080/0 (socket says 172.21.15.47:55080) 2022-06-29T15:52:18.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.165+0000 7fc738ec5700 1 -- 172.21.15.47:0/1616689385 learned_addr learned my addr 172.21.15.47:0/1616689385 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:18.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.165+0000 7fc733fff700 1 --2- 172.21.15.47:0/1616689385 >> v2:172.21.15.64:3300/0 conn(0x7fc73405f0b0 0x7fc7340b0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:18.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.165+0000 7fc738ec5700 1 -- 172.21.15.47:0/1616689385 >> v2:172.21.15.64:3300/0 conn(0x7fc73405f0b0 msgr2=0x7fc7340b0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:18.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.165+0000 7fc738ec5700 1 --2- 172.21.15.47:0/1616689385 >> v2:172.21.15.64:3300/0 conn(0x7fc73405f0b0 0x7fc7340b0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:18.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.165+0000 7fc738ec5700 1 -- 172.21.15.47:0/1616689385 >> v1:172.21.15.64:6789/0 conn(0x7fc7340ac180 legacy=0x7fc7340b2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:18.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.165+0000 7fc738ec5700 1 -- 172.21.15.47:0/1616689385 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7340b0050 con 0x7fc7340af4a0 2022-06-29T15:52:18.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.165+0000 7fc738ec5700 1 --2- 172.21.15.47:0/1616689385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7340af4a0 0x7fc73405f9e0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 rx=0x7fc724002f70 tx=0x7fc724006eb0).ready entity=mon.0 client_cookie=906615fbfac4ae81 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:18.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.165+0000 7fc7337fe700 1 -- 172.21.15.47:0/1616689385 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc72400d5c0 con 0x7fc7340af4a0 2022-06-29T15:52:18.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.165+0000 7fc7337fe700 1 -- 172.21.15.47:0/1616689385 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc724000e90 con 0x7fc7340af4a0 2022-06-29T15:52:18.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.165+0000 7fc7337fe700 1 -- 172.21.15.47:0/1616689385 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc72400d950 con 0x7fc7340af4a0 2022-06-29T15:52:18.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.165+0000 7fc73b129700 1 -- 172.21.15.47:0/1616689385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7340af4a0 msgr2=0x7fc73405f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:18.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.165+0000 7fc73b129700 1 --2- 172.21.15.47:0/1616689385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7340af4a0 0x7fc73405f9e0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 rx=0x7fc724002f70 tx=0x7fc724006eb0).stop 2022-06-29T15:52:18.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.165+0000 7fc73b129700 1 -- 172.21.15.47:0/1616689385 shutdown_connections 2022-06-29T15:52:18.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.165+0000 7fc73b129700 1 --2- 172.21.15.47:0/1616689385 >> v2:172.21.15.64:3300/0 conn(0x7fc73405f0b0 0x7fc7340b0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:18.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.165+0000 7fc73b129700 1 --2- 172.21.15.47:0/1616689385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7340af4a0 0x7fc73405f9e0 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:18.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.165+0000 7fc73b129700 1 -- 172.21.15.47:0/1616689385 >> 172.21.15.47:0/1616689385 conn(0x7fc7340626f0 msgr2=0x7fc734062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:18.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.165+0000 7fc73b129700 1 -- 172.21.15.47:0/1616689385 shutdown_connections 2022-06-29T15:52:18.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.165+0000 7fc73b129700 1 -- 172.21.15.47:0/1616689385 wait complete. 2022-06-29T15:52:18.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.169+0000 7fc73b129700 1 Processor -- start 2022-06-29T15:52:18.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.169+0000 7fc73b129700 1 -- start start 2022-06-29T15:52:18.182 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.173+0000 7fc73b129700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc73405f0b0 0x7fc73413d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:18.182 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.173+0000 7fc73b129700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc7340ac180 0x7fc7341384d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:18.182 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.173+0000 7fc73b129700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7340af4a0 0x7fc7341389c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:18.182 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.173+0000 7fc73b129700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc734145540 con 0x7fc7340af4a0 2022-06-29T15:52:18.182 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.173+0000 7fc73b129700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7341457e0 con 0x7fc73405f0b0 2022-06-29T15:52:18.183 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.173+0000 7fc73b129700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc734145a80 con 0x7fc7340ac180 2022-06-29T15:52:18.183 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.173+0000 7fc7396c6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7340af4a0 0x7fc7341389c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:18.183 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.173+0000 7fc7396c6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7340af4a0 0x7fc7341389c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55090/0 (socket says 172.21.15.47:55090) 2022-06-29T15:52:18.183 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.173+0000 7fc7396c6700 1 -- 172.21.15.47:0/2021399577 learned_addr learned my addr 172.21.15.47:0/2021399577 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:18.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.173+0000 7fc738ec5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc73405f0b0 0x7fc73413d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:18.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.173+0000 7fc733fff700 1 --2- 172.21.15.47:0/2021399577 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc7340ac180 0x7fc7341384d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:18.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.173+0000 7fc7396c6700 1 -- 172.21.15.47:0/2021399577 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc7340ac180 msgr2=0x7fc7341384d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:18.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.173+0000 7fc7396c6700 1 --2- 172.21.15.47:0/2021399577 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc7340ac180 0x7fc7341384d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:18.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.173+0000 7fc7396c6700 1 -- 172.21.15.47:0/2021399577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc73405f0b0 msgr2=0x7fc73413d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:18.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.173+0000 7fc7396c6700 1 --2- 172.21.15.47:0/2021399577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc73405f0b0 0x7fc73413d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:18.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.173+0000 7fc7396c6700 1 -- 172.21.15.47:0/2021399577 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc72400b1e0 con 0x7fc7340af4a0 2022-06-29T15:52:18.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.173+0000 7fc7396c6700 1 --2- 172.21.15.47:0/2021399577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7340af4a0 0x7fc7341389c0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 rx=0x7fc7340b2ac0 tx=0x7fc72c007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:18.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.173+0000 7fc731ffb700 1 -- 172.21.15.47:0/2021399577 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc72c00f470 con 0x7fc7340af4a0 2022-06-29T15:52:18.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.173+0000 7fc731ffb700 1 -- 172.21.15.47:0/2021399577 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc72c00f5d0 con 0x7fc7340af4a0 2022-06-29T15:52:18.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.173+0000 7fc73b129700 1 -- 172.21.15.47:0/2021399577 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc734145d20 con 0x7fc7340af4a0 2022-06-29T15:52:18.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.173+0000 7fc731ffb700 1 -- 172.21.15.47:0/2021399577 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc72c00f930 con 0x7fc7340af4a0 2022-06-29T15:52:18.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.173+0000 7fc73b129700 1 -- 172.21.15.47:0/2021399577 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc734142da0 con 0x7fc7340af4a0 2022-06-29T15:52:18.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.173+0000 7fc731ffb700 1 -- 172.21.15.47:0/2021399577 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc72c019030 con 0x7fc7340af4a0 2022-06-29T15:52:18.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.173+0000 7fc73b129700 1 -- 172.21.15.47:0/2021399577 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7340573d0 con 0x7fc7340af4a0 2022-06-29T15:52:18.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.173+0000 7fc731ffb700 1 --2- 172.21.15.47:0/2021399577 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc714053a70 0x7fc714055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:18.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.173+0000 7fc731ffb700 1 -- 172.21.15.47:0/2021399577 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc72c06b670 con 0x7fc7340af4a0 2022-06-29T15:52:18.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.173+0000 7fc738ec5700 1 --2- 172.21.15.47:0/2021399577 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc714053a70 0x7fc714055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:18.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.177+0000 7fc738ec5700 1 --2- 172.21.15.47:0/2021399577 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc714053a70 0x7fc714055f30 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 rx=0x7fc72401c000 tx=0x7fc724015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:18.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.181+0000 7fc731ffb700 1 -- 172.21.15.47:0/2021399577 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc72c042430 con 0x7fc7340af4a0 2022-06-29T15:52:18.352 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.345+0000 7fc73b129700 1 -- 172.21.15.47:0/2021399577 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc7340b1c10 con 0x7fc7340af4a0 2022-06-29T15:52:18.352 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.345+0000 7fc731ffb700 1 -- 172.21.15.47:0/2021399577 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+2495 (secure 0 0 0) 0x7fc72c045a50 con 0x7fc7340af4a0 2022-06-29T15:52:18.353 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:52:18.353 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":79,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8596,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6836/1456033193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1456033193},{"type":"v1","addr":"172.21.15.47:6836","nonce":1456033193}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":77},{"gid":8745,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/1940204812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6835","nonce":1940204812},{"type":"v1","addr":"172.21.15.47:6837","nonce":1940204812}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":77}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:52:17.031576+0000","modified":"2022-06-29T15:52:18.034584+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":57377},"failed":[],"damaged":[],"stopped":[],"info":{"gid_57377":{"gid":57377,"name":"b","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6833/642573447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":642573447},{"type":"v1","addr":"172.21.15.64:6833","nonce":642573447}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2022-06-29T15:52:18.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.345+0000 7fc73b129700 1 -- 172.21.15.47:0/2021399577 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc714053a70 msgr2=0x7fc714055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:18.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.345+0000 7fc73b129700 1 --2- 172.21.15.47:0/2021399577 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc714053a70 0x7fc714055f30 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 rx=0x7fc72401c000 tx=0x7fc724015000).stop 2022-06-29T15:52:18.355 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.345+0000 7fc73b129700 1 -- 172.21.15.47:0/2021399577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7340af4a0 msgr2=0x7fc7341389c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:18.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.345+0000 7fc73b129700 1 --2- 172.21.15.47:0/2021399577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7340af4a0 0x7fc7341389c0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 rx=0x7fc7340b2ac0 tx=0x7fc72c007f40).stop 2022-06-29T15:52:18.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.345+0000 7fc73b129700 1 -- 172.21.15.47:0/2021399577 shutdown_connections 2022-06-29T15:52:18.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.345+0000 7fc73b129700 1 --2- 172.21.15.47:0/2021399577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7340af4a0 0x7fc7341389c0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:18.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.345+0000 7fc73b129700 1 --2- 172.21.15.47:0/2021399577 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc7340ac180 0x7fc7341384d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:18.356 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.345+0000 7fc73b129700 1 --2- 172.21.15.47:0/2021399577 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc714053a70 0x7fc714055f30 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:18.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.345+0000 7fc73b129700 1 --2- 172.21.15.47:0/2021399577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc73405f0b0 0x7fc73413d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:18.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.345+0000 7fc73b129700 1 -- 172.21.15.47:0/2021399577 >> 172.21.15.47:0/2021399577 conn(0x7fc7340626f0 msgr2=0x7fc73413c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:18.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.349+0000 7fc73b129700 1 -- 172.21.15.47:0/2021399577 shutdown_connections 2022-06-29T15:52:18.357 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.349+0000 7fc73b129700 1 -- 172.21.15.47:0/2021399577 wait complete. 2022-06-29T15:52:18.358 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 79 2022-06-29T15:52:18.366 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:52:18.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.461+0000 7f1fe281f700 1 Processor -- start 2022-06-29T15:52:18.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.461+0000 7f1fe281f700 1 -- start start 2022-06-29T15:52:18.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.461+0000 7f1fe281f700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1fdc0b2120 0x7f1fdc0b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:18.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.461+0000 7f1fe281f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1fdc0b1260 0x7f1fdc14b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:18.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.465+0000 7f1fe281f700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fdc0b0740 con 0x7f1fdc05f0b0 2022-06-29T15:52:18.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.465+0000 7f1fe281f700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fdc147460 con 0x7f1fdc0b1260 2022-06-29T15:52:18.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.465+0000 7f1fe281f700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fdc0b0970 con 0x7f1fdc0b2120 2022-06-29T15:52:18.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.465+0000 7f1fdb7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1fdc05f0b0 0x7f1fdc147af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40648/0 (socket says 172.21.15.47:40648) 2022-06-29T15:52:18.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.465+0000 7f1fdb7fe700 1 -- 172.21.15.47:0/1844456396 learned_addr learned my addr 172.21.15.47:0/1844456396 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:52:18.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.465+0000 7f1fdbfff700 1 --2- 172.21.15.47:0/1844456396 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1fdc0b2120 0x7f1fdc0b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:18.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.465+0000 7f1fe0dbc700 1 --2- 172.21.15.47:0/1844456396 >> v2:172.21.15.64:3300/0 conn(0x7f1fdc0b1260 0x7f1fdc14b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:18.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.465+0000 7f1fdaffd700 1 -- 172.21.15.47:0/1844456396 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1139068964 0 0) 0x7f1fdc0b0740 con 0x7f1fdc05f0b0 2022-06-29T15:52:18.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.465+0000 7f1fdaffd700 1 -- 172.21.15.47:0/1844456396 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fc8003260 con 0x7f1fdc05f0b0 2022-06-29T15:52:18.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.465+0000 7f1fdbfff700 1 -- 172.21.15.47:0/1844456396 >> v2:172.21.15.64:3300/0 conn(0x7f1fdc0b1260 msgr2=0x7f1fdc14b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:18.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.465+0000 7f1fdbfff700 1 --2- 172.21.15.47:0/1844456396 >> v2:172.21.15.64:3300/0 conn(0x7f1fdc0b1260 0x7f1fdc14b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:18.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.465+0000 7f1fdbfff700 1 -- 172.21.15.47:0/1844456396 >> v1:172.21.15.47:6789/0 conn(0x7f1fdc05f0b0 legacy=0x7f1fdc147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:18.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.465+0000 7f1fdbfff700 1 -- 172.21.15.47:0/1844456396 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fdc0b0c10 con 0x7f1fdc0b2120 2022-06-29T15:52:18.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.465+0000 7f1fe0dbc700 1 --2- 172.21.15.47:0/1844456396 >> v2:172.21.15.64:3300/0 conn(0x7f1fdc0b1260 0x7f1fdc14b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:52:18.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.465+0000 7f1fdbfff700 1 --2- 172.21.15.47:0/1844456396 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1fdc0b2120 0x7f1fdc0b2500 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f1fd0002f70 tx=0x7f1fd0006df0).ready entity=mon.2 client_cookie=8789af99a2aaa571 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:18.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.465+0000 7f1fdaffd700 1 -- 172.21.15.47:0/1844456396 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1fd000d4a0 con 0x7f1fdc0b2120 2022-06-29T15:52:18.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.465+0000 7f1fdaffd700 1 -- 172.21.15.47:0/1844456396 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fd0000e90 con 0x7f1fdc0b2120 2022-06-29T15:52:18.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.465+0000 7f1fdaffd700 1 -- 172.21.15.47:0/1844456396 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1fd000d7f0 con 0x7f1fdc0b2120 2022-06-29T15:52:18.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.465+0000 7f1fe281f700 1 -- 172.21.15.47:0/1844456396 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1fdc0b2120 msgr2=0x7f1fdc0b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:18.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.465+0000 7f1fe281f700 1 --2- 172.21.15.47:0/1844456396 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1fdc0b2120 0x7f1fdc0b2500 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f1fd0002f70 tx=0x7f1fd0006df0).stop 2022-06-29T15:52:18.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.465+0000 7f1fe281f700 1 -- 172.21.15.47:0/1844456396 shutdown_connections 2022-06-29T15:52:18.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.465+0000 7f1fe281f700 1 --2- 172.21.15.47:0/1844456396 >> v2:172.21.15.64:3300/0 conn(0x7f1fdc0b1260 0x7f1fdc14b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:18.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.465+0000 7f1fe281f700 1 --2- 172.21.15.47:0/1844456396 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1fdc0b2120 0x7f1fdc0b2500 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:18.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.465+0000 7f1fe281f700 1 -- 172.21.15.47:0/1844456396 >> 172.21.15.47:0/1844456396 conn(0x7f1fdc0626f0 msgr2=0x7f1fdc062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:18.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.465+0000 7f1fe281f700 1 -- 172.21.15.47:0/1844456396 shutdown_connections 2022-06-29T15:52:18.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.465+0000 7f1fe281f700 1 -- 172.21.15.47:0/1844456396 wait complete. 2022-06-29T15:52:18.485 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.469+0000 7f1fe281f700 1 Processor -- start 2022-06-29T15:52:18.485 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.469+0000 7f1fe281f700 1 -- start start 2022-06-29T15:52:18.485 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.469+0000 7f1fe281f700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1fdc0b1260 0x7f1fdc0ad0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:18.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.469+0000 7f1fe281f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1fdc0b2120 0x7f1fdc144ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:18.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.469+0000 7f1fe281f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fdc063840 0x7f1fdc144fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:18.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.469+0000 7f1fe281f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fdc0aeb00 con 0x7f1fdc0b2120 2022-06-29T15:52:18.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.469+0000 7f1fe281f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fdc1454c0 con 0x7f1fdc063840 2022-06-29T15:52:18.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.469+0000 7f1fe281f700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fdc0642f0 con 0x7f1fdc0b1260 2022-06-29T15:52:18.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.469+0000 7f1fdb7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1fdc0b2120 0x7f1fdc144ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:18.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.469+0000 7f1fdbfff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1fdc0b1260 0x7f1fdc0ad0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:18.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.469+0000 7f1fdb7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1fdc0b2120 0x7f1fdc144ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55102/0 (socket says 172.21.15.47:55102) 2022-06-29T15:52:18.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.469+0000 7f1fdb7fe700 1 -- 172.21.15.47:0/2330089647 learned_addr learned my addr 172.21.15.47:0/2330089647 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:18.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.469+0000 7f1fdbfff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1fdc0b1260 0x7f1fdc0ad0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:52202/0 (socket says 172.21.15.47:52202) 2022-06-29T15:52:18.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.469+0000 7f1fe0dbc700 1 --2- 172.21.15.47:0/2330089647 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fdc063840 0x7f1fdc144fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:18.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.473+0000 7f1fdb7fe700 1 -- 172.21.15.47:0/2330089647 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1fdc0b1260 msgr2=0x7f1fdc0ad0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:18.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.473+0000 7f1fdb7fe700 1 --2- 172.21.15.47:0/2330089647 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1fdc0b1260 0x7f1fdc0ad0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:18.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.473+0000 7f1fdb7fe700 1 -- 172.21.15.47:0/2330089647 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fdc063840 msgr2=0x7f1fdc144fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:18.489 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.473+0000 7f1fdb7fe700 1 --2- 172.21.15.47:0/2330089647 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fdc063840 0x7f1fdc144fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:18.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.473+0000 7f1fdb7fe700 1 -- 172.21.15.47:0/2330089647 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd4009470 con 0x7f1fdc0b2120 2022-06-29T15:52:18.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.473+0000 7f1fdb7fe700 1 --2- 172.21.15.47:0/2330089647 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1fdc0b2120 0x7f1fdc144ae0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 rx=0x7f1fcc001110 tx=0x7f1fcc007aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:18.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.473+0000 7f1fd97fa700 1 -- 172.21.15.47:0/2330089647 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1fcc0070d0 con 0x7f1fdc0b2120 2022-06-29T15:52:18.490 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.473+0000 7f1fd97fa700 1 -- 172.21.15.47:0/2330089647 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fcc007230 con 0x7f1fdc0b2120 2022-06-29T15:52:18.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.473+0000 7f1fe281f700 1 -- 172.21.15.47:0/2330089647 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fd000b180 con 0x7f1fdc0b2120 2022-06-29T15:52:18.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.473+0000 7f1fd97fa700 1 -- 172.21.15.47:0/2330089647 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1fcc005610 con 0x7f1fdc0b2120 2022-06-29T15:52:18.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.473+0000 7f1fe281f700 1 -- 172.21.15.47:0/2330089647 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fdc0aed10 con 0x7f1fdc0b2120 2022-06-29T15:52:18.491 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.473+0000 7f1fe281f700 1 -- 172.21.15.47:0/2330089647 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fdc0672e0 con 0x7f1fdc0b2120 2022-06-29T15:52:18.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.473+0000 7f1fd97fa700 1 -- 172.21.15.47:0/2330089647 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1fcc010070 con 0x7f1fdc0b2120 2022-06-29T15:52:18.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.473+0000 7f1fd97fa700 1 --2- 172.21.15.47:0/2330089647 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1fbc053ac0 0x7f1fbc055f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:18.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.477+0000 7f1fd97fa700 1 -- 172.21.15.47:0/2330089647 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f1fcc06b6e0 con 0x7f1fdc0b2120 2022-06-29T15:52:18.492 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.477+0000 7f1fdbfff700 1 --2- 172.21.15.47:0/2330089647 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1fbc053ac0 0x7f1fbc055f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:18.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.477+0000 7f1fdbfff700 1 --2- 172.21.15.47:0/2330089647 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1fbc053ac0 0x7f1fbc055f80 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 rx=0x7f1fd001a000 tx=0x7f1fd0014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:18.493 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.485+0000 7f1fd97fa700 1 -- 172.21.15.47:0/2330089647 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1fcc0424a0 con 0x7f1fdc0b2120 2022-06-29T15:52:18.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.617+0000 7f1fe281f700 1 -- 172.21.15.47:0/2330089647 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1fdc0573d0 con 0x7f1fdc0b2120 2022-06-29T15:52:18.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.617+0000 7f1fd97fa700 1 -- 172.21.15.47:0/2330089647 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v162) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f1fcc045ac0 con 0x7f1fdc0b2120 2022-06-29T15:52:18.625 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:52:18.625 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":162,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:52:18.035479+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:52:14.780743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2022-06-29T15:52:16.023272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:52:18.626 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.617+0000 7f1fe281f700 1 -- 172.21.15.47:0/2330089647 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1fbc053ac0 msgr2=0x7f1fbc055f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:18.627 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.617+0000 7f1fe281f700 1 --2- 172.21.15.47:0/2330089647 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1fbc053ac0 0x7f1fbc055f80 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 rx=0x7f1fd001a000 tx=0x7f1fd0014000).stop 2022-06-29T15:52:18.627 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.617+0000 7f1fe281f700 1 -- 172.21.15.47:0/2330089647 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1fdc0b2120 msgr2=0x7f1fdc144ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:18.627 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.617+0000 7f1fe281f700 1 --2- 172.21.15.47:0/2330089647 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1fdc0b2120 0x7f1fdc144ae0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 rx=0x7f1fcc001110 tx=0x7f1fcc007aa0).stop 2022-06-29T15:52:18.627 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.617+0000 7f1fe281f700 1 -- 172.21.15.47:0/2330089647 shutdown_connections 2022-06-29T15:52:18.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.617+0000 7f1fe281f700 1 --2- 172.21.15.47:0/2330089647 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1fbc053ac0 0x7f1fbc055f80 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:18.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.617+0000 7f1fe281f700 1 --2- 172.21.15.47:0/2330089647 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fdc063840 0x7f1fdc144fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:18.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.617+0000 7f1fe281f700 1 --2- 172.21.15.47:0/2330089647 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1fdc0b2120 0x7f1fdc144ae0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:18.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.617+0000 7f1fe281f700 1 --2- 172.21.15.47:0/2330089647 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1fdc0b1260 0x7f1fdc0ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:18.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.617+0000 7f1fe281f700 1 -- 172.21.15.47:0/2330089647 >> 172.21.15.47:0/2330089647 conn(0x7f1fdc0626f0 msgr2=0x7f1fdc0b4b60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:18.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.617+0000 7f1fe281f700 1 -- 172.21.15.47:0/2330089647 shutdown_connections 2022-06-29T15:52:18.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.617+0000 7f1fe281f700 1 -- 172.21.15.47:0/2330089647 wait complete. 2022-06-29T15:52:18.638 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-29T15:52:18.754 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.745+0000 7fce05225700 1 Processor -- start 2022-06-29T15:52:18.754 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.745+0000 7fce05225700 1 -- start start 2022-06-29T15:52:18.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.749+0000 7fce05225700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fce000b5d90 0x7fce00143a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:18.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.749+0000 7fce05225700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fce000676a0 0x7fce00140130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:18.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.749+0000 7fce05225700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce0005cc30 con 0x7fce0005c850 2022-06-29T15:52:18.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.749+0000 7fce05225700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce001430a0 con 0x7fce000b5d90 2022-06-29T15:52:18.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.749+0000 7fce05225700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce00143300 con 0x7fce000676a0 2022-06-29T15:52:18.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.749+0000 7fcdff7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fce000b5d90 0x7fce00143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:18.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.749+0000 7fcdff7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fce000b5d90 0x7fce00143a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55112/0 (socket says 172.21.15.47:55112) 2022-06-29T15:52:18.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.749+0000 7fcdff7fe700 1 -- 172.21.15.47:0/3232259168 learned_addr learned my addr 172.21.15.47:0/3232259168 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:18.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.749+0000 7fce04a24700 1 --2- 172.21.15.47:0/3232259168 >> v2:172.21.15.64:3300/0 conn(0x7fce000676a0 0x7fce00140130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:18.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.749+0000 7fcdff7fe700 1 -- 172.21.15.47:0/3232259168 >> v2:172.21.15.64:3300/0 conn(0x7fce000676a0 msgr2=0x7fce00140130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:18.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.749+0000 7fcdff7fe700 1 --2- 172.21.15.47:0/3232259168 >> v2:172.21.15.64:3300/0 conn(0x7fce000676a0 0x7fce00140130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:18.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.749+0000 7fcdff7fe700 1 -- 172.21.15.47:0/3232259168 >> v1:172.21.15.64:6789/0 conn(0x7fce0005c850 legacy=0x7fce000b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:18.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.749+0000 7fcdff7fe700 1 -- 172.21.15.47:0/3232259168 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce001435a0 con 0x7fce000b5d90 2022-06-29T15:52:18.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.749+0000 7fce04a24700 1 --2- 172.21.15.47:0/3232259168 >> v2:172.21.15.64:3300/0 conn(0x7fce000676a0 0x7fce00140130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:52:18.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.749+0000 7fcdff7fe700 1 --2- 172.21.15.47:0/3232259168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce000b5d90 0x7fce00143a40 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 rx=0x7fcdf0002f70 tx=0x7fcdf0006eb0).ready entity=mon.0 client_cookie=4f4855f96ee8de8b server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:18.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.749+0000 7fcdfeffd700 1 -- 172.21.15.47:0/3232259168 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcdf000d5c0 con 0x7fce000b5d90 2022-06-29T15:52:18.761 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.749+0000 7fcdfeffd700 1 -- 172.21.15.47:0/3232259168 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdf0000e90 con 0x7fce000b5d90 2022-06-29T15:52:18.761 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.749+0000 7fcdfeffd700 1 -- 172.21.15.47:0/3232259168 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcdf000d950 con 0x7fce000b5d90 2022-06-29T15:52:18.764 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.749+0000 7fce05225700 1 -- 172.21.15.47:0/3232259168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce000b5d90 msgr2=0x7fce00143a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:18.764 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.749+0000 7fce05225700 1 --2- 172.21.15.47:0/3232259168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce000b5d90 0x7fce00143a40 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 rx=0x7fcdf0002f70 tx=0x7fcdf0006eb0).stop 2022-06-29T15:52:18.764 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.749+0000 7fce05225700 1 -- 172.21.15.47:0/3232259168 shutdown_connections 2022-06-29T15:52:18.765 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.749+0000 7fce05225700 1 --2- 172.21.15.47:0/3232259168 >> v2:172.21.15.64:3300/0 conn(0x7fce000676a0 0x7fce00140130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:18.765 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.749+0000 7fce05225700 1 --2- 172.21.15.47:0/3232259168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce000b5d90 0x7fce00143a40 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:18.765 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.749+0000 7fce05225700 1 -- 172.21.15.47:0/3232259168 >> 172.21.15.47:0/3232259168 conn(0x7fce00060e00 msgr2=0x7fce000611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:18.765 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.749+0000 7fce05225700 1 -- 172.21.15.47:0/3232259168 shutdown_connections 2022-06-29T15:52:18.766 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.749+0000 7fce05225700 1 -- 172.21.15.47:0/3232259168 wait complete. 2022-06-29T15:52:18.766 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.753+0000 7fce05225700 1 Processor -- start 2022-06-29T15:52:18.766 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.753+0000 7fce05225700 1 -- start start 2022-06-29T15:52:18.766 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.753+0000 7fce05225700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce0005c850 0x7fce000b27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:18.767 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.753+0000 7fce05225700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce000676a0 0x7fce000b7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:18.767 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.753+0000 7fce05225700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fce000b5d90 0x7fce000b7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:18.767 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.753+0000 7fce05225700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce000ba390 con 0x7fce000676a0 2022-06-29T15:52:18.767 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.753+0000 7fce05225700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce00146280 con 0x7fce0005c850 2022-06-29T15:52:18.767 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.753+0000 7fce05225700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce00146520 con 0x7fce000b5d90 2022-06-29T15:52:18.768 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.753+0000 7fcdff7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce000676a0 0x7fce000b7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:18.768 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.753+0000 7fcdff7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce000676a0 0x7fce000b7460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55116/0 (socket says 172.21.15.47:55116) 2022-06-29T15:52:18.768 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.753+0000 7fcdff7fe700 1 -- 172.21.15.47:0/1547069134 learned_addr learned my addr 172.21.15.47:0/1547069134 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:18.768 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.753+0000 7fcdfffff700 1 --2- 172.21.15.47:0/1547069134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce0005c850 0x7fce000b27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:18.769 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.753+0000 7fce04a24700 1 --2- 172.21.15.47:0/1547069134 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fce000b5d90 0x7fce000b7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:18.769 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.753+0000 7fcdff7fe700 1 -- 172.21.15.47:0/1547069134 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fce000b5d90 msgr2=0x7fce000b7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:18.769 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.753+0000 7fcdff7fe700 1 --2- 172.21.15.47:0/1547069134 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fce000b5d90 0x7fce000b7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:18.769 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.753+0000 7fcdff7fe700 1 -- 172.21.15.47:0/1547069134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce0005c850 msgr2=0x7fce000b27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:18.769 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.753+0000 7fcdff7fe700 1 --2- 172.21.15.47:0/1547069134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce0005c850 0x7fce000b27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:18.770 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.753+0000 7fcdff7fe700 1 -- 172.21.15.47:0/1547069134 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf8009470 con 0x7fce000676a0 2022-06-29T15:52:18.770 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.753+0000 7fcdff7fe700 1 --2- 172.21.15.47:0/1547069134 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce000676a0 0x7fce000b7460 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 rx=0x7fcdf0002f70 tx=0x7fcdf0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:18.770 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.753+0000 7fcdfd7fa700 1 -- 172.21.15.47:0/1547069134 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcdf000d940 con 0x7fce000676a0 2022-06-29T15:52:18.770 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.753+0000 7fcdfd7fa700 1 -- 172.21.15.47:0/1547069134 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdf0011c30 con 0x7fce000676a0 2022-06-29T15:52:18.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.753+0000 7fce05225700 1 -- 172.21.15.47:0/1547069134 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdf000b1e0 con 0x7fce000676a0 2022-06-29T15:52:18.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.753+0000 7fcdfd7fa700 1 -- 172.21.15.47:0/1547069134 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcdf001e460 con 0x7fce000676a0 2022-06-29T15:52:18.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.753+0000 7fce05225700 1 -- 172.21.15.47:0/1547069134 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce001522e0 con 0x7fce000676a0 2022-06-29T15:52:18.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.757+0000 7fce05225700 1 -- 172.21.15.47:0/1547069134 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce001469c0 con 0x7fce000676a0 2022-06-29T15:52:18.772 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.757+0000 7fcdfd7fa700 1 -- 172.21.15.47:0/1547069134 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fcdf001a040 con 0x7fce000676a0 2022-06-29T15:52:18.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.757+0000 7fcdfd7fa700 1 --2- 172.21.15.47:0/1547069134 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcde0053a70 0x7fcde0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:18.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.757+0000 7fcdfd7fa700 1 -- 172.21.15.47:0/1547069134 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fcdf006cfa0 con 0x7fce000676a0 2022-06-29T15:52:18.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.757+0000 7fcdfffff700 1 --2- 172.21.15.47:0/1547069134 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcde0053a70 0x7fcde0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:18.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.757+0000 7fcdfffff700 1 --2- 172.21.15.47:0/1547069134 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcde0053a70 0x7fcde0055f30 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 rx=0x7fcdf400d000 tx=0x7fcdf4007bc0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:18.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.765+0000 7fcdfd7fa700 1 -- 172.21.15.47:0/1547069134 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcdf0043d10 con 0x7fce000676a0 2022-06-29T15:52:18.943 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.937+0000 7fce05225700 1 -- 172.21.15.47:0/1547069134 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fce00061b00 con 0x7fce000676a0 2022-06-29T15:52:18.951 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.945+0000 7fcdfd7fa700 1 -- 172.21.15.47:0/1547069134 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v37) v1 ==== 184+0+0 (secure 0 0 0) 0x7fcdf00230e0 con 0x7fce000676a0 2022-06-29T15:52:18.954 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.945+0000 7fce05225700 1 -- 172.21.15.47:0/1547069134 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcde0053a70 msgr2=0x7fcde0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:18.954 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.945+0000 7fce05225700 1 --2- 172.21.15.47:0/1547069134 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcde0053a70 0x7fcde0055f30 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 rx=0x7fcdf400d000 tx=0x7fcdf4007bc0).stop 2022-06-29T15:52:18.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.945+0000 7fce05225700 1 -- 172.21.15.47:0/1547069134 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce000676a0 msgr2=0x7fce000b7460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:18.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.945+0000 7fce05225700 1 --2- 172.21.15.47:0/1547069134 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce000676a0 0x7fce000b7460 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 rx=0x7fcdf0002f70 tx=0x7fcdf0015000).stop 2022-06-29T15:52:18.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.945+0000 7fce05225700 1 -- 172.21.15.47:0/1547069134 shutdown_connections 2022-06-29T15:52:18.955 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.945+0000 7fce05225700 1 --2- 172.21.15.47:0/1547069134 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fce000b5d90 0x7fce000b7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:18.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.945+0000 7fce05225700 1 --2- 172.21.15.47:0/1547069134 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce000676a0 0x7fce000b7460 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:18.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.945+0000 7fce05225700 1 --2- 172.21.15.47:0/1547069134 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fcde0053a70 0x7fcde0055f30 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:18.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.945+0000 7fce05225700 1 --2- 172.21.15.47:0/1547069134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce0005c850 0x7fce000b27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:18.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.945+0000 7fce05225700 1 -- 172.21.15.47:0/1547069134 >> 172.21.15.47:0/1547069134 conn(0x7fce00060e00 msgr2=0x7fce000b4860 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:18.956 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.945+0000 7fce05225700 1 -- 172.21.15.47:0/1547069134 shutdown_connections 2022-06-29T15:52:18.957 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:18.945+0000 7fce05225700 1 -- 172.21.15.47:0/1547069134 wait complete. 2022-06-29T15:52:18.957 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.0 2022-06-29T15:52:18.967 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-06-29T15:52:19.081 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.073+0000 7fa1d4c60700 1 Processor -- start 2022-06-29T15:52:19.081 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.073+0000 7fa1d4c60700 1 -- start start 2022-06-29T15:52:19.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.073+0000 7fa1d4c60700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa1d00b4300 0x7fa1d00612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:19.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.073+0000 7fa1d4c60700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1d0067610 con 0x7fa1d01471a0 2022-06-29T15:52:19.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.073+0000 7fa1d4c60700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1d01467c0 con 0x7fa1d0060ee0 2022-06-29T15:52:19.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.073+0000 7fa1d4c60700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d01475d0 con 0x7fa1d00b4300 2022-06-29T15:52:19.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.073+0000 7fa1ced9d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa1d01471a0 0x7fa1d013cdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40680/0 (socket says 172.21.15.47:40680) 2022-06-29T15:52:19.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.073+0000 7fa1ced9d700 1 -- 172.21.15.47:0/4143083568 learned_addr learned my addr 172.21.15.47:0/4143083568 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:52:19.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.073+0000 7fa1cdd9b700 1 --2- 172.21.15.47:0/4143083568 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa1d00b4300 0x7fa1d00612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:19.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.077+0000 7fa1cd59a700 1 -- 172.21.15.47:0/4143083568 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1896076972 0 0) 0x7fa1d0067610 con 0x7fa1d01471a0 2022-06-29T15:52:19.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.077+0000 7fa1cd59a700 1 -- 172.21.15.47:0/4143083568 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1b0003260 con 0x7fa1d01471a0 2022-06-29T15:52:19.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.077+0000 7fa1cd59a700 1 -- 172.21.15.47:0/4143083568 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2365968119 0 0) 0x7fa1b0003260 con 0x7fa1d01471a0 2022-06-29T15:52:19.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.077+0000 7fa1cd59a700 1 -- 172.21.15.47:0/4143083568 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa1d00b4300 msgr2=0x7fa1d00612c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:19.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.077+0000 7fa1cd59a700 1 --2- 172.21.15.47:0/4143083568 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa1d00b4300 0x7fa1d00612c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:19.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.077+0000 7fa1cd59a700 1 -- 172.21.15.47:0/4143083568 >> v1:172.21.15.64:6789/0 conn(0x7fa1d0060ee0 legacy=0x7fa1d01430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:19.088 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.077+0000 7fa1cd59a700 1 -- 172.21.15.47:0/4143083568 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d0146a60 con 0x7fa1d01471a0 2022-06-29T15:52:19.089 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.077+0000 7fa1cd59a700 1 -- 172.21.15.47:0/4143083568 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7fa1c80019e0 con 0x7fa1d01471a0 2022-06-29T15:52:19.089 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.077+0000 7fa1cd59a700 1 -- 172.21.15.47:0/4143083568 >> v1:172.21.15.47:6789/0 conn(0x7fa1d01471a0 legacy=0x7fa1d013cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:19.089 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.077+0000 7fa1cd59a700 1 --2- 172.21.15.47:0/4143083568 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa1d00b4300 0x7fa1b0003780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:19.089 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.077+0000 7fa1cd59a700 1 --2- 172.21.15.47:0/4143083568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1b0003c70 0x7fa1b0004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:19.090 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.077+0000 7fa1cd59a700 1 --2- 172.21.15.47:0/4143083568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1b00049f0 0x7fa1b0004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:19.090 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.077+0000 7fa1cd59a700 1 -- 172.21.15.47:0/4143083568 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d01475d0 con 0x7fa1b0003c70 2022-06-29T15:52:19.090 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.077+0000 7fa1cd59a700 1 -- 172.21.15.47:0/4143083568 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b000b7b0 con 0x7fa1b00049f0 2022-06-29T15:52:19.090 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.077+0000 7fa1cd59a700 1 -- 172.21.15.47:0/4143083568 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b000b970 con 0x7fa1d00b4300 2022-06-29T15:52:19.091 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.077+0000 7fa1cdd9b700 1 --2- 172.21.15.47:0/4143083568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1b0003c70 0x7fa1b0004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:19.091 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.077+0000 7fa1ce59c700 1 --2- 172.21.15.47:0/4143083568 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa1d00b4300 0x7fa1b0003780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:19.091 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.077+0000 7fa1ced9d700 1 --2- 172.21.15.47:0/4143083568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1b00049f0 0x7fa1b0004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:19.091 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.077+0000 7fa1cdd9b700 1 -- 172.21.15.47:0/4143083568 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa1d00b4300 msgr2=0x7fa1b0003780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:19.091 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.077+0000 7fa1cdd9b700 1 --2- 172.21.15.47:0/4143083568 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa1d00b4300 0x7fa1b0003780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:19.092 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.077+0000 7fa1cdd9b700 1 -- 172.21.15.47:0/4143083568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1b00049f0 msgr2=0x7fa1b0004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:19.092 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.077+0000 7fa1cdd9b700 1 --2- 172.21.15.47:0/4143083568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1b00049f0 0x7fa1b0004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:19.092 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.077+0000 7fa1cdd9b700 1 -- 172.21.15.47:0/4143083568 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b0003260 con 0x7fa1b0003c70 2022-06-29T15:52:19.093 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.077+0000 7fa1cdd9b700 1 --2- 172.21.15.47:0/4143083568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1b0003c70 0x7fa1b0004500 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 rx=0x7fa1b800c000 tx=0x7fa1b8006df0).ready entity=mon.0 client_cookie=6d7883d1d04c493d server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:19.093 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.077+0000 7fa1cd59a700 1 -- 172.21.15.47:0/4143083568 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1b800e830 con 0x7fa1b0003c70 2022-06-29T15:52:19.093 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.077+0000 7fa1cd59a700 1 -- 172.21.15.47:0/4143083568 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1b800e990 con 0x7fa1b0003c70 2022-06-29T15:52:19.093 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.077+0000 7fa1cd59a700 1 -- 172.21.15.47:0/4143083568 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1b800ecf0 con 0x7fa1b0003c70 2022-06-29T15:52:19.094 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.077+0000 7fa1d4c60700 1 -- 172.21.15.47:0/4143083568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1b0003c70 msgr2=0x7fa1b0004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:19.094 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.077+0000 7fa1d4c60700 1 --2- 172.21.15.47:0/4143083568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1b0003c70 0x7fa1b0004500 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 rx=0x7fa1b800c000 tx=0x7fa1b8006df0).stop 2022-06-29T15:52:19.094 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.077+0000 7fa1ced9d700 1 -- 172.21.15.47:0/4143083568 reap_dead start 2022-06-29T15:52:19.094 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.077+0000 7fa1d4c60700 1 -- 172.21.15.47:0/4143083568 shutdown_connections 2022-06-29T15:52:19.095 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.077+0000 7fa1d4c60700 1 -- 172.21.15.47:0/4143083568 >> 172.21.15.47:0/4143083568 conn(0x7fa1d005c510 msgr2=0x7fa1d005f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:19.095 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.077+0000 7fa1d4c60700 1 -- 172.21.15.47:0/4143083568 shutdown_connections 2022-06-29T15:52:19.095 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.077+0000 7fa1d4c60700 1 -- 172.21.15.47:0/4143083568 wait complete. 2022-06-29T15:52:19.098 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.081+0000 7fa1d4c60700 1 Processor -- start 2022-06-29T15:52:19.098 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.081+0000 7fa1d4c60700 1 -- start start 2022-06-29T15:52:19.099 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.081+0000 7fa1d4c60700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa1b0003c70 0x7fa1d013d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:19.099 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.081+0000 7fa1d4c60700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1d00b2420 0x7fa1d00b1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:19.099 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.081+0000 7fa1d4c60700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1d00ba520 0x7fa1d0142380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:19.099 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.081+0000 7fa1d4c60700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d0140ae0 con 0x7fa1d00b2420 2022-06-29T15:52:19.100 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.081+0000 7fa1d4c60700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d015a8b0 con 0x7fa1d00ba520 2022-06-29T15:52:19.100 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.081+0000 7fa1d4c60700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d013dce0 con 0x7fa1b0003c70 2022-06-29T15:52:19.100 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.081+0000 7fa1cdd9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1d00b2420 0x7fa1d00b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:19.100 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.081+0000 7fa1cdd9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1d00b2420 0x7fa1d00b1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55134/0 (socket says 172.21.15.47:55134) 2022-06-29T15:52:19.101 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.081+0000 7fa1cdd9b700 1 -- 172.21.15.47:0/407812944 learned_addr learned my addr 172.21.15.47:0/407812944 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:19.101 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.081+0000 7fa1ced9d700 1 --2- 172.21.15.47:0/407812944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1d00ba520 0x7fa1d0142380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:19.101 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.081+0000 7fa1ce59c700 1 --2- 172.21.15.47:0/407812944 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa1b0003c70 0x7fa1d013d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:19.101 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.081+0000 7fa1cdd9b700 1 -- 172.21.15.47:0/407812944 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa1b0003c70 msgr2=0x7fa1d013d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:19.101 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.081+0000 7fa1cdd9b700 1 --2- 172.21.15.47:0/407812944 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa1b0003c70 0x7fa1d013d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:19.102 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.081+0000 7fa1cdd9b700 1 -- 172.21.15.47:0/407812944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1d00ba520 msgr2=0x7fa1d0142380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:19.102 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.081+0000 7fa1cdd9b700 1 --2- 172.21.15.47:0/407812944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1d00ba520 0x7fa1d0142380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:19.102 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.081+0000 7fa1cdd9b700 1 -- 172.21.15.47:0/407812944 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d00b9d00 con 0x7fa1d00b2420 2022-06-29T15:52:19.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.081+0000 7fa1cdd9b700 1 --2- 172.21.15.47:0/407812944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1d00b2420 0x7fa1d00b1710 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 rx=0x7fa1d00b3620 tx=0x7fa1b8019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:19.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.081+0000 7fa1c77fe700 1 -- 172.21.15.47:0/407812944 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1b801b820 con 0x7fa1d00b2420 2022-06-29T15:52:19.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.081+0000 7fa1d4c60700 1 -- 172.21.15.47:0/407812944 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1d00b9fa0 con 0x7fa1d00b2420 2022-06-29T15:52:19.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.081+0000 7fa1c77fe700 1 -- 172.21.15.47:0/407812944 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1b8000e90 con 0x7fa1d00b2420 2022-06-29T15:52:19.103 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.081+0000 7fa1c77fe700 1 -- 172.21.15.47:0/407812944 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1b801bb70 con 0x7fa1d00b2420 2022-06-29T15:52:19.104 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.081+0000 7fa1d4c60700 1 -- 172.21.15.47:0/407812944 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1d00b1c00 con 0x7fa1d00b2420 2022-06-29T15:52:19.104 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.085+0000 7fa1d4c60700 1 -- 172.21.15.47:0/407812944 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1d00573d0 con 0x7fa1d00b2420 2022-06-29T15:52:19.104 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.085+0000 7fa1c77fe700 1 -- 172.21.15.47:0/407812944 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa1b801bcd0 con 0x7fa1d00b2420 2022-06-29T15:52:19.104 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.085+0000 7fa1c77fe700 1 --2- 172.21.15.47:0/407812944 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa1b4053aa0 0x7fa1b4055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:19.105 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.085+0000 7fa1c77fe700 1 -- 172.21.15.47:0/407812944 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa1b806d5f0 con 0x7fa1d00b2420 2022-06-29T15:52:19.105 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.085+0000 7fa1ce59c700 1 --2- 172.21.15.47:0/407812944 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa1b4053aa0 0x7fa1b4055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:19.105 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.085+0000 7fa1ce59c700 1 --2- 172.21.15.47:0/407812944 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa1b4053aa0 0x7fa1b4055f60 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 rx=0x7fa1c0003000 tx=0x7fa1c00079e0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:19.105 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.093+0000 7fa1c77fe700 1 -- 172.21.15.47:0/407812944 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa1b8047d40 con 0x7fa1d00b2420 2022-06-29T15:52:19.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.253+0000 7fa1d4c60700 1 -- 172.21.15.47:0/407812944 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fa1d006a210 con 0x7fa1d00b2420 2022-06-29T15:52:19.268 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.261+0000 7fa1c77fe700 1 -- 172.21.15.47:0/407812944 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v38) v1 ==== 184+0+0 (secure 0 0 0) 0x7fa1b802a070 con 0x7fa1d00b2420 2022-06-29T15:52:19.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.261+0000 7fa1d4c60700 1 -- 172.21.15.47:0/407812944 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa1b4053aa0 msgr2=0x7fa1b4055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:19.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.261+0000 7fa1d4c60700 1 --2- 172.21.15.47:0/407812944 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa1b4053aa0 0x7fa1b4055f60 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 rx=0x7fa1c0003000 tx=0x7fa1c00079e0).stop 2022-06-29T15:52:19.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.261+0000 7fa1d4c60700 1 -- 172.21.15.47:0/407812944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1d00b2420 msgr2=0x7fa1d00b1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:19.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.261+0000 7fa1d4c60700 1 --2- 172.21.15.47:0/407812944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1d00b2420 0x7fa1d00b1710 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 rx=0x7fa1d00b3620 tx=0x7fa1b8019000).stop 2022-06-29T15:52:19.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.261+0000 7fa1d4c60700 1 -- 172.21.15.47:0/407812944 shutdown_connections 2022-06-29T15:52:19.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.261+0000 7fa1d4c60700 1 --2- 172.21.15.47:0/407812944 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa1b4053aa0 0x7fa1b4055f60 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:19.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.261+0000 7fa1d4c60700 1 --2- 172.21.15.47:0/407812944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1d00ba520 0x7fa1d0142380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:19.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.261+0000 7fa1d4c60700 1 --2- 172.21.15.47:0/407812944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1d00b2420 0x7fa1d00b1710 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:19.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.261+0000 7fa1d4c60700 1 --2- 172.21.15.47:0/407812944 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa1b0003c70 0x7fa1d013d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:19.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.261+0000 7fa1d4c60700 1 -- 172.21.15.47:0/407812944 >> 172.21.15.47:0/407812944 conn(0x7fa1d005c510 msgr2=0x7fa1d00b6070 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:19.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.265+0000 7fa1d4c60700 1 -- 172.21.15.47:0/407812944 shutdown_connections 2022-06-29T15:52:19.274 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.265+0000 7fa1d4c60700 1 -- 172.21.15.47:0/407812944 wait complete. 2022-06-29T15:52:19.274 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.1 2022-06-29T15:52:19.284 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:52:19.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.377+0000 7fd427576700 1 Processor -- start 2022-06-29T15:52:19.385 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.377+0000 7fd427576700 1 -- start start 2022-06-29T15:52:19.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.377+0000 7fd427576700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd4200b7840 0x7fd4200b44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:19.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.377+0000 7fd427576700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd4200b3240 0x7fd4200b49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:19.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.377+0000 7fd427576700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4200b7c20 con 0x7fd420146890 2022-06-29T15:52:19.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.377+0000 7fd427576700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4200b4ec0 con 0x7fd4200b7840 2022-06-29T15:52:19.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.377+0000 7fd427576700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd420066840 con 0x7fd4200b3240 2022-06-29T15:52:19.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.377+0000 7fd426574700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd4200b7840 0x7fd4200b44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:19.389 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.377+0000 7fd426574700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd4200b7840 0x7fd4200b44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55144/0 (socket says 172.21.15.47:55144) 2022-06-29T15:52:19.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.377+0000 7fd426574700 1 -- 172.21.15.47:0/2249152856 learned_addr learned my addr 172.21.15.47:0/2249152856 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:19.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.377+0000 7fd425d73700 1 --2- 172.21.15.47:0/2249152856 >> v2:172.21.15.64:3300/0 conn(0x7fd4200b3240 0x7fd4200b49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:19.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.377+0000 7fd426574700 1 -- 172.21.15.47:0/2249152856 >> v2:172.21.15.64:3300/0 conn(0x7fd4200b3240 msgr2=0x7fd4200b49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:19.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.377+0000 7fd426574700 1 --2- 172.21.15.47:0/2249152856 >> v2:172.21.15.64:3300/0 conn(0x7fd4200b3240 0x7fd4200b49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:19.390 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.377+0000 7fd426574700 1 -- 172.21.15.47:0/2249152856 >> v1:172.21.15.64:6789/0 conn(0x7fd420146890 legacy=0x7fd420146d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:19.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.377+0000 7fd426574700 1 -- 172.21.15.47:0/2249152856 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4200b2ff0 con 0x7fd4200b7840 2022-06-29T15:52:19.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.377+0000 7fd426574700 1 --2- 172.21.15.47:0/2249152856 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4200b7840 0x7fd4200b44e0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 rx=0x7fd410002f70 tx=0x7fd410006eb0).ready entity=mon.0 client_cookie=6d8b1e5ddac2583 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:19.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.381+0000 7fd425572700 1 -- 172.21.15.47:0/2249152856 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd41000d5c0 con 0x7fd4200b7840 2022-06-29T15:52:19.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.381+0000 7fd425572700 1 -- 172.21.15.47:0/2249152856 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd410000e90 con 0x7fd4200b7840 2022-06-29T15:52:19.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.381+0000 7fd425572700 1 -- 172.21.15.47:0/2249152856 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd41000d950 con 0x7fd4200b7840 2022-06-29T15:52:19.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.381+0000 7fd427576700 1 -- 172.21.15.47:0/2249152856 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4200b7840 msgr2=0x7fd4200b44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:19.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.381+0000 7fd427576700 1 --2- 172.21.15.47:0/2249152856 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4200b7840 0x7fd4200b44e0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 rx=0x7fd410002f70 tx=0x7fd410006eb0).stop 2022-06-29T15:52:19.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.381+0000 7fd427576700 1 -- 172.21.15.47:0/2249152856 shutdown_connections 2022-06-29T15:52:19.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.381+0000 7fd427576700 1 --2- 172.21.15.47:0/2249152856 >> v2:172.21.15.64:3300/0 conn(0x7fd4200b3240 0x7fd4200b49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:19.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.381+0000 7fd427576700 1 --2- 172.21.15.47:0/2249152856 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4200b7840 0x7fd4200b44e0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:19.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.381+0000 7fd427576700 1 -- 172.21.15.47:0/2249152856 >> 172.21.15.47:0/2249152856 conn(0x7fd42005f480 msgr2=0x7fd4200647d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:19.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.381+0000 7fd427576700 1 -- 172.21.15.47:0/2249152856 shutdown_connections 2022-06-29T15:52:19.394 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.381+0000 7fd427576700 1 -- 172.21.15.47:0/2249152856 wait complete. 2022-06-29T15:52:19.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.381+0000 7fd427576700 1 Processor -- start 2022-06-29T15:52:19.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.381+0000 7fd427576700 1 -- start start 2022-06-29T15:52:19.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.381+0000 7fd427576700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4200b3240 0x7fd4200b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:19.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.381+0000 7fd427576700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd420146890 0x7fd4200aca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:19.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.381+0000 7fd427576700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd4200ab320 0x7fd4200acf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:19.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.381+0000 7fd427576700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd420061390 con 0x7fd4200b3240 2022-06-29T15:52:19.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.381+0000 7fd427576700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4200ad470 con 0x7fd420146890 2022-06-29T15:52:19.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.381+0000 7fd427576700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd420061780 con 0x7fd4200ab320 2022-06-29T15:52:19.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.381+0000 7fd426574700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4200b3240 0x7fd4200b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:19.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.381+0000 7fd426574700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4200b3240 0x7fd4200b05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55148/0 (socket says 172.21.15.47:55148) 2022-06-29T15:52:19.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.381+0000 7fd426574700 1 -- 172.21.15.47:0/3962718645 learned_addr learned my addr 172.21.15.47:0/3962718645 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:19.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.381+0000 7fd425d73700 1 --2- 172.21.15.47:0/3962718645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd420146890 0x7fd4200aca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:19.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.381+0000 7fd426d75700 1 --2- 172.21.15.47:0/3962718645 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd4200ab320 0x7fd4200acf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:19.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.385+0000 7fd426574700 1 -- 172.21.15.47:0/3962718645 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd4200ab320 msgr2=0x7fd4200acf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:19.400 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.385+0000 7fd426574700 1 --2- 172.21.15.47:0/3962718645 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd4200ab320 0x7fd4200acf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:19.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.385+0000 7fd426574700 1 -- 172.21.15.47:0/3962718645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd420146890 msgr2=0x7fd4200aca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:19.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.385+0000 7fd426574700 1 --2- 172.21.15.47:0/3962718645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd420146890 0x7fd4200aca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:19.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.385+0000 7fd426574700 1 -- 172.21.15.47:0/3962718645 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd41000b1e0 con 0x7fd4200b3240 2022-06-29T15:52:19.401 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.385+0000 7fd426574700 1 --2- 172.21.15.47:0/3962718645 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4200b3240 0x7fd4200b05b0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 rx=0x7fd410002f70 tx=0x7fd410015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:19.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.385+0000 7fd40f7fe700 1 -- 172.21.15.47:0/3962718645 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd410011790 con 0x7fd4200b3240 2022-06-29T15:52:19.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.385+0000 7fd40f7fe700 1 -- 172.21.15.47:0/3962718645 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4100118f0 con 0x7fd4200b3240 2022-06-29T15:52:19.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.385+0000 7fd40f7fe700 1 -- 172.21.15.47:0/3962718645 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd410011c50 con 0x7fd4200b3240 2022-06-29T15:52:19.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.385+0000 7fd427576700 1 -- 172.21.15.47:0/3962718645 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4200610f0 con 0x7fd4200b3240 2022-06-29T15:52:19.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.385+0000 7fd427576700 1 -- 172.21.15.47:0/3962718645 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4200ad690 con 0x7fd4200b3240 2022-06-29T15:52:19.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.385+0000 7fd427576700 1 -- 172.21.15.47:0/3962718645 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4200573d0 con 0x7fd4200b3240 2022-06-29T15:52:19.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.385+0000 7fd40f7fe700 1 -- 172.21.15.47:0/3962718645 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd41001a040 con 0x7fd4200b3240 2022-06-29T15:52:19.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.385+0000 7fd40f7fe700 1 --2- 172.21.15.47:0/3962718645 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd400053950 0x7fd400055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:19.403 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.385+0000 7fd40f7fe700 1 -- 172.21.15.47:0/3962718645 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd41006cf20 con 0x7fd4200b3240 2022-06-29T15:52:19.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.385+0000 7fd425d73700 1 --2- 172.21.15.47:0/3962718645 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd400053950 0x7fd400055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:19.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.385+0000 7fd425d73700 1 --2- 172.21.15.47:0/3962718645 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd400053950 0x7fd400055e10 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 rx=0x7fd41c00b000 tx=0x7fd41c006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:19.404 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.393+0000 7fd40f7fe700 1 -- 172.21.15.47:0/3962718645 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd410043ce0 con 0x7fd4200b3240 2022-06-29T15:52:19.565 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.557+0000 7fd427576700 1 -- 172.21.15.47:0/3962718645 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd420068830 con 0x7fd4200b3240 2022-06-29T15:52:19.566 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.557+0000 7fd40f7fe700 1 -- 172.21.15.47:0/3962718645 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+2495 (secure 0 0 0) 0x7fd410023020 con 0x7fd4200b3240 2022-06-29T15:52:19.566 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:52:19.567 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":79,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8596,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6836/1456033193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1456033193},{"type":"v1","addr":"172.21.15.47:6836","nonce":1456033193}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":77},{"gid":8745,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/1940204812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6835","nonce":1940204812},{"type":"v1","addr":"172.21.15.47:6837","nonce":1940204812}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":77}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:52:17.031576+0000","modified":"2022-06-29T15:52:18.034584+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":57377},"failed":[],"damaged":[],"stopped":[],"info":{"gid_57377":{"gid":57377,"name":"b","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6833/642573447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":642573447},{"type":"v1","addr":"172.21.15.64:6833","nonce":642573447}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2022-06-29T15:52:19.568 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.561+0000 7fd427576700 1 -- 172.21.15.47:0/3962718645 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd400053950 msgr2=0x7fd400055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:19.569 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.561+0000 7fd427576700 1 --2- 172.21.15.47:0/3962718645 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd400053950 0x7fd400055e10 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 rx=0x7fd41c00b000 tx=0x7fd41c006df0).stop 2022-06-29T15:52:19.569 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.561+0000 7fd427576700 1 -- 172.21.15.47:0/3962718645 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4200b3240 msgr2=0x7fd4200b05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:19.569 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.561+0000 7fd427576700 1 --2- 172.21.15.47:0/3962718645 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4200b3240 0x7fd4200b05b0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 rx=0x7fd410002f70 tx=0x7fd410015000).stop 2022-06-29T15:52:19.569 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.561+0000 7fd427576700 1 -- 172.21.15.47:0/3962718645 shutdown_connections 2022-06-29T15:52:19.570 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.561+0000 7fd427576700 1 --2- 172.21.15.47:0/3962718645 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd4200ab320 0x7fd4200acf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:19.570 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.561+0000 7fd427576700 1 --2- 172.21.15.47:0/3962718645 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd400053950 0x7fd400055e10 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:19.570 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.561+0000 7fd427576700 1 --2- 172.21.15.47:0/3962718645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd420146890 0x7fd4200aca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:19.570 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.561+0000 7fd427576700 1 --2- 172.21.15.47:0/3962718645 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4200b3240 0x7fd4200b05b0 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:19.571 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.561+0000 7fd427576700 1 -- 172.21.15.47:0/3962718645 >> 172.21.15.47:0/3962718645 conn(0x7fd42005f480 msgr2=0x7fd42005cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:19.571 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.561+0000 7fd427576700 1 -- 172.21.15.47:0/3962718645 shutdown_connections 2022-06-29T15:52:19.571 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:19.561+0000 7fd427576700 1 -- 172.21.15.47:0/3962718645 wait complete. 2022-06-29T15:52:19.572 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 79 2022-06-29T15:52:19.882 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 79, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-29T15:52:17.031576+0000', 'modified': '2022-06-29T15:52:18.034584+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 57377}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_57377': {'gid': 57377, 'name': 'b', 'rank': 0, 'incarnation': 78, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.64:6833/642573447', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6832', 'nonce': 642573447}, {'type': 'v1', 'addr': '172.21.15.64:6833', 'nonce': 642573447}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-29T15:52:19.883 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-29T15:52:19.883 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-06-29T15:52:20.000 INFO:teuthology.orchestra.run.smithi064.stdout:{"cluster_fsid":"9a68894a-53ce-400e-838b-14eab91b834c","whoami":0,"id":57377,"want_state":"up:active","state":"up:active","rank_uptime":2.9516506769999999,"mdsmap_epoch":79,"osdmap_epoch":162,"osdmap_epoch_barrier":161,"uptime":8.0005920960000001} 2022-06-29T15:52:20.001 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9a68894a-53ce-400e-838b-14eab91b834c", "id": 57377, "mdsmap_epoch": 79, "osdmap_epoch": 162, "osdmap_epoch_barrier": 161, "rank_uptime": 2.951650677, "state": "up:active", "uptime": 8.000592096, "want_state": "up:active", "whoami": 0 } 2022-06-29T15:52:20.001 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:52:20.114 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.105+0000 7f41a8d14700 1 Processor -- start 2022-06-29T15:52:20.114 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.105+0000 7f41a8d14700 1 -- start start 2022-06-29T15:52:20.116 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.109+0000 7f41a8d14700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f41a40b5c60 0x7f41a4144980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:20.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.109+0000 7f41a8d14700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f41a4060eb0 0x7f41a4144e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:20.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.109+0000 7f41a8d14700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f41a40a72f0 0x7f41a40b1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:20.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.109+0000 7f41a8d14700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41a4145360 con 0x7f41a40a72f0 2022-06-29T15:52:20.117 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.109+0000 7f41a8d14700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41a40a9120 con 0x7f41a4060eb0 2022-06-29T15:52:20.118 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.109+0000 7f41a8d14700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41a40b0b80 con 0x7f41a40b5c60 2022-06-29T15:52:20.118 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.109+0000 7f41a3fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f41a40a72f0 0x7f41a40b1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:20.118 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.109+0000 7f41a3fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f41a40a72f0 0x7f41a40b1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55160/0 (socket says 172.21.15.47:55160) 2022-06-29T15:52:20.118 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.109+0000 7f41a3fff700 1 -- 172.21.15.47:0/3420749612 learned_addr learned my addr 172.21.15.47:0/3420749612 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:20.119 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.109+0000 7f41a2ffd700 1 --2- 172.21.15.47:0/3420749612 >> v2:172.21.15.64:3300/0 conn(0x7f41a4060eb0 0x7f41a4144e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:20.119 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.109+0000 7f41a37fe700 1 --2- 172.21.15.47:0/3420749612 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f41a40b5c60 0x7f41a4144980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:20.119 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.109+0000 7f41a3fff700 1 -- 172.21.15.47:0/3420749612 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f41a40b5c60 msgr2=0x7f41a4144980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:20.119 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.109+0000 7f41a3fff700 1 --2- 172.21.15.47:0/3420749612 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f41a40b5c60 0x7f41a4144980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:20.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.109+0000 7f41a3fff700 1 -- 172.21.15.47:0/3420749612 >> v2:172.21.15.64:3300/0 conn(0x7f41a4060eb0 msgr2=0x7f41a4144e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:20.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.109+0000 7f41a3fff700 1 --2- 172.21.15.47:0/3420749612 >> v2:172.21.15.64:3300/0 conn(0x7f41a4060eb0 0x7f41a4144e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:20.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.109+0000 7f41a3fff700 1 -- 172.21.15.47:0/3420749612 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a40a89e0 con 0x7f41a40a72f0 2022-06-29T15:52:20.120 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.109+0000 7f41a3fff700 1 --2- 172.21.15.47:0/3420749612 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41a40a72f0 0x7f41a40b1e30 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 rx=0x7f419c0032f0 tx=0x7f419c009290).ready entity=mon.0 client_cookie=e92c5cad1a431252 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:20.121 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.109+0000 7f41a27fc700 1 -- 172.21.15.47:0/3420749612 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f419c00b680 con 0x7f41a40a72f0 2022-06-29T15:52:20.121 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.109+0000 7f41a27fc700 1 -- 172.21.15.47:0/3420749612 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f419c00b7e0 con 0x7f41a40a72f0 2022-06-29T15:52:20.121 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.109+0000 7f41a27fc700 1 -- 172.21.15.47:0/3420749612 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f419c00b950 con 0x7f41a40a72f0 2022-06-29T15:52:20.124 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.109+0000 7f41a8d14700 1 -- 172.21.15.47:0/3420749612 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41a40a72f0 msgr2=0x7f41a40b1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:20.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.109+0000 7f41a8d14700 1 --2- 172.21.15.47:0/3420749612 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41a40a72f0 0x7f41a40b1e30 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 rx=0x7f419c0032f0 tx=0x7f419c009290).stop 2022-06-29T15:52:20.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.109+0000 7f41a8d14700 1 -- 172.21.15.47:0/3420749612 shutdown_connections 2022-06-29T15:52:20.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.109+0000 7f41a8d14700 1 --2- 172.21.15.47:0/3420749612 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41a40a72f0 0x7f41a40b1e30 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:20.125 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.109+0000 7f41a8d14700 1 --2- 172.21.15.47:0/3420749612 >> v2:172.21.15.64:3300/0 conn(0x7f41a4060eb0 0x7f41a4144e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:20.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.109+0000 7f41a8d14700 1 --2- 172.21.15.47:0/3420749612 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f41a40b5c60 0x7f41a4144980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:20.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.109+0000 7f41a8d14700 1 -- 172.21.15.47:0/3420749612 >> 172.21.15.47:0/3420749612 conn(0x7f41a4062560 msgr2=0x7f41a4062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:20.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.109+0000 7f41a8d14700 1 -- 172.21.15.47:0/3420749612 shutdown_connections 2022-06-29T15:52:20.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.109+0000 7f41a8d14700 1 -- 172.21.15.47:0/3420749612 wait complete. 2022-06-29T15:52:20.126 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.113+0000 7f41a8d14700 1 Processor -- start 2022-06-29T15:52:20.127 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.113+0000 7f41a8d14700 1 -- start start 2022-06-29T15:52:20.127 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.113+0000 7f41a8d14700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41a4060eb0 0x7f41a414b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:20.127 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.113+0000 7f41a8d14700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f41a40a72f0 0x7f41a41409c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:20.127 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.113+0000 7f41a8d14700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41a40b5c60 0x7f41a40ae100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:20.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.113+0000 7f41a8d14700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a4149df0 con 0x7f41a40b5c60 2022-06-29T15:52:20.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.113+0000 7f41a8d14700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a4149120 con 0x7f41a4060eb0 2022-06-29T15:52:20.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.113+0000 7f41a8d14700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41a40ac9a0 con 0x7f41a40a72f0 2022-06-29T15:52:20.128 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.113+0000 7f41a3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41a40b5c60 0x7f41a40ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:20.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.113+0000 7f41a37fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41a4060eb0 0x7f41a414b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:20.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.113+0000 7f41a2ffd700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f41a40a72f0 0x7f41a41409c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:20.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.113+0000 7f41a3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41a40b5c60 0x7f41a40ae100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55166/0 (socket says 172.21.15.47:55166) 2022-06-29T15:52:20.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.113+0000 7f41a37fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41a4060eb0 0x7f41a414b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:60866/0 (socket says 172.21.15.47:60866) 2022-06-29T15:52:20.129 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.113+0000 7f41a3fff700 1 -- 172.21.15.47:0/3128838449 learned_addr learned my addr 172.21.15.47:0/3128838449 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:20.130 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.113+0000 7f41a37fe700 1 -- 172.21.15.47:0/3128838449 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f41a40a72f0 msgr2=0x7f41a41409c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:20.130 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.113+0000 7f41a37fe700 1 --2- 172.21.15.47:0/3128838449 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f41a40a72f0 0x7f41a41409c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:20.130 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.113+0000 7f41a37fe700 1 -- 172.21.15.47:0/3128838449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41a40b5c60 msgr2=0x7f41a40ae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:20.130 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.113+0000 7f41a37fe700 1 --2- 172.21.15.47:0/3128838449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41a40b5c60 0x7f41a40ae100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:20.131 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.113+0000 7f41a37fe700 1 -- 172.21.15.47:0/3128838449 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f419c009530 con 0x7f41a4060eb0 2022-06-29T15:52:20.131 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.113+0000 7f41a3fff700 1 --2- 172.21.15.47:0/3128838449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41a40b5c60 0x7f41a40ae100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:52:20.131 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.113+0000 7f41a37fe700 1 --2- 172.21.15.47:0/3128838449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41a4060eb0 0x7f41a414b5b0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7f419800c000 tx=0x7f4198006d90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:20.131 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.113+0000 7f41a0ff9700 1 -- 172.21.15.47:0/3128838449 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f41980104c0 con 0x7f41a4060eb0 2022-06-29T15:52:20.132 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.113+0000 7f41a0ff9700 1 -- 172.21.15.47:0/3128838449 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4198010620 con 0x7f41a4060eb0 2022-06-29T15:52:20.132 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.113+0000 7f41a8d14700 1 -- 172.21.15.47:0/3128838449 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41a40acc40 con 0x7f41a4060eb0 2022-06-29T15:52:20.132 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.113+0000 7f41a0ff9700 1 -- 172.21.15.47:0/3128838449 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4198010980 con 0x7f41a4060eb0 2022-06-29T15:52:20.132 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.113+0000 7f41a8d14700 1 -- 172.21.15.47:0/3128838449 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41a4149720 con 0x7f41a4060eb0 2022-06-29T15:52:20.132 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.117+0000 7f41a8d14700 1 -- 172.21.15.47:0/3128838449 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41a40573d0 con 0x7f41a4060eb0 2022-06-29T15:52:20.133 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.117+0000 7f41a0ff9700 1 -- 172.21.15.47:0/3128838449 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4198010ae0 con 0x7f41a4060eb0 2022-06-29T15:52:20.133 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.117+0000 7f41a0ff9700 1 --2- 172.21.15.47:0/3128838449 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4184053aa0 0x7f4184055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:20.133 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.117+0000 7f41a0ff9700 1 -- 172.21.15.47:0/3128838449 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f419806ba80 con 0x7f41a4060eb0 2022-06-29T15:52:20.133 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.117+0000 7f41a2ffd700 1 --2- 172.21.15.47:0/3128838449 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4184053aa0 0x7f4184055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:20.134 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.117+0000 7f41a2ffd700 1 --2- 172.21.15.47:0/3128838449 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4184053aa0 0x7f4184055f60 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7f419400b000 tx=0x7f4194006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:20.134 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.125+0000 7f41a0ff9700 1 -- 172.21.15.47:0/3128838449 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f41980428f0 con 0x7f41a4060eb0 2022-06-29T15:52:20.264 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.257+0000 7f41a8d14700 1 -- 172.21.15.47:0/3128838449 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f41a4069e20 con 0x7f41a4060eb0 2022-06-29T15:52:20.266 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.257+0000 7f41a0ff9700 1 -- 172.21.15.47:0/3128838449 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v162) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f4198024fb0 con 0x7f41a4060eb0 2022-06-29T15:52:20.267 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:52:20.267 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":162,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:52:18.035479+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:52:14.780743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2022-06-29T15:52:16.023272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:52:20.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.261+0000 7f41a8d14700 1 -- 172.21.15.47:0/3128838449 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4184053aa0 msgr2=0x7f4184055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:20.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.261+0000 7f41a8d14700 1 --2- 172.21.15.47:0/3128838449 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4184053aa0 0x7f4184055f60 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7f419400b000 tx=0x7f4194006df0).stop 2022-06-29T15:52:20.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.261+0000 7f41a8d14700 1 -- 172.21.15.47:0/3128838449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41a4060eb0 msgr2=0x7f41a414b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:20.271 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.261+0000 7f41a8d14700 1 --2- 172.21.15.47:0/3128838449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41a4060eb0 0x7f41a414b5b0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7f419800c000 tx=0x7f4198006d90).stop 2022-06-29T15:52:20.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.261+0000 7f41a8d14700 1 -- 172.21.15.47:0/3128838449 shutdown_connections 2022-06-29T15:52:20.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.261+0000 7f41a8d14700 1 --2- 172.21.15.47:0/3128838449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41a40b5c60 0x7f41a40ae100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:20.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.261+0000 7f41a8d14700 1 --2- 172.21.15.47:0/3128838449 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f41a40a72f0 0x7f41a41409c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:20.272 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.261+0000 7f41a8d14700 1 --2- 172.21.15.47:0/3128838449 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f4184053aa0 0x7f4184055f60 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:20.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.261+0000 7f41a8d14700 1 --2- 172.21.15.47:0/3128838449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41a4060eb0 0x7f41a414b5b0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:20.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.261+0000 7f41a8d14700 1 -- 172.21.15.47:0/3128838449 >> 172.21.15.47:0/3128838449 conn(0x7f41a4062560 msgr2=0x7f41a40b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:20.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.261+0000 7f41a8d14700 1 -- 172.21.15.47:0/3128838449 shutdown_connections 2022-06-29T15:52:20.273 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.261+0000 7f41a8d14700 1 -- 172.21.15.47:0/3128838449 wait complete. 2022-06-29T15:52:20.289 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-29T15:42:16.971431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-29T15:42:25.290884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 29, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-29T15:52:14.780743+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '161', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 30, 'pool_name': 'cephfs_data', 'create_time': '2022-06-29T15:52:16.023272+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '162', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-29T15:52:20.290 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-29T15:52:20.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.393+0000 7f0aa759e700 1 Processor -- start 2022-06-29T15:52:20.402 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.393+0000 7f0aa759e700 1 -- start start 2022-06-29T15:52:20.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.397+0000 7f0aa759e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0aa80acfd0 0x7f0aa80b1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:20.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.397+0000 7f0aa759e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0aa80b1a10 0x7f0aa80af560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:20.408 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.397+0000 7f0aa759e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aa80611e0 con 0x7f0aa809ada0 2022-06-29T15:52:20.409 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.397+0000 7f0aa759e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0aa80b10c0 con 0x7f0aa80b1a10 2022-06-29T15:52:20.409 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.397+0000 7f0aa759e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0aa80b0e30 con 0x7f0aa80acfd0 2022-06-29T15:52:20.409 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.397+0000 7f0aa659c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0aa809ada0 0x7f0aa809b180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40724/0 (socket says 172.21.15.47:40724) 2022-06-29T15:52:20.410 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.397+0000 7f0aa659c700 1 -- 172.21.15.47:0/2204685948 learned_addr learned my addr 172.21.15.47:0/2204685948 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:52:20.410 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.397+0000 7f0aa6d9d700 1 --2- 172.21.15.47:0/2204685948 >> v2:172.21.15.47:3300/0 conn(0x7f0aa80b1a10 0x7f0aa80af560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:20.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.397+0000 7f0a9ffff700 1 --2- 172.21.15.47:0/2204685948 >> v2:172.21.15.64:3300/0 conn(0x7f0aa80acfd0 0x7f0aa80b1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:20.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.397+0000 7f0aa5d9b700 1 -- 172.21.15.47:0/2204685948 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 334941672 0 0) 0x7f0aa80611e0 con 0x7f0aa809ada0 2022-06-29T15:52:20.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.397+0000 7f0aa5d9b700 1 -- 172.21.15.47:0/2204685948 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a88003260 con 0x7f0aa809ada0 2022-06-29T15:52:20.411 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.397+0000 7f0aa5d9b700 1 -- 172.21.15.47:0/2204685948 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2065938170 0 0) 0x7f0a88003260 con 0x7f0aa809ada0 2022-06-29T15:52:20.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.397+0000 7f0aa5d9b700 1 -- 172.21.15.47:0/2204685948 >> v2:172.21.15.64:3300/0 conn(0x7f0aa80acfd0 msgr2=0x7f0aa80b1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:20.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.397+0000 7f0aa5d9b700 1 --2- 172.21.15.47:0/2204685948 >> v2:172.21.15.64:3300/0 conn(0x7f0aa80acfd0 0x7f0aa80b1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:20.412 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.397+0000 7f0aa5d9b700 1 -- 172.21.15.47:0/2204685948 >> v2:172.21.15.47:3300/0 conn(0x7f0aa80b1a10 msgr2=0x7f0aa80af560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:20.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.397+0000 7f0aa5d9b700 1 --2- 172.21.15.47:0/2204685948 >> v2:172.21.15.47:3300/0 conn(0x7f0aa80b1a10 0x7f0aa80af560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:20.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.397+0000 7f0aa5d9b700 1 -- 172.21.15.47:0/2204685948 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa809b890 con 0x7f0aa809ada0 2022-06-29T15:52:20.413 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.397+0000 7f0aa5d9b700 1 -- 172.21.15.47:0/2204685948 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7f0a98001660 con 0x7f0aa809ada0 2022-06-29T15:52:20.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.397+0000 7f0aa5d9b700 1 -- 172.21.15.47:0/2204685948 >> v1:172.21.15.47:6789/0 conn(0x7f0aa809ada0 legacy=0x7f0aa809b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:20.414 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.397+0000 7f0aa5d9b700 1 --2- 172.21.15.47:0/2204685948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a88003750 0x7f0a88003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:20.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.397+0000 7f0aa5d9b700 1 --2- 172.21.15.47:0/2204685948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a88004500 0x7f0a88004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:20.415 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.397+0000 7f0aa5d9b700 1 --2- 172.21.15.47:0/2204685948 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0a88004e70 0x7f0a8800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:20.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.397+0000 7f0aa5d9b700 1 -- 172.21.15.47:0/2204685948 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa80b10c0 con 0x7f0a88004500 2022-06-29T15:52:20.416 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.397+0000 7f0aa5d9b700 1 -- 172.21.15.47:0/2204685948 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa80b0e30 con 0x7f0a88003750 2022-06-29T15:52:20.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.397+0000 7f0aa5d9b700 1 -- 172.21.15.47:0/2204685948 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8800bc40 con 0x7f0a88004e70 2022-06-29T15:52:20.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.397+0000 7f0aa6d9d700 1 --2- 172.21.15.47:0/2204685948 >> v2:172.21.15.47:3300/0 conn(0x7f0aa80b1a10 0x7f0aa80af560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:52:20.417 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.397+0000 7f0a9ffff700 1 --2- 172.21.15.47:0/2204685948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a88004500 0x7f0a88004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:20.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.397+0000 7f0aa659c700 1 --2- 172.21.15.47:0/2204685948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a88003750 0x7f0a88003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:20.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.397+0000 7f0aa6d9d700 1 --2- 172.21.15.47:0/2204685948 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0a88004e70 0x7f0a8800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:20.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.397+0000 7f0a9ffff700 1 -- 172.21.15.47:0/2204685948 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0a88004e70 msgr2=0x7f0a8800b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:20.418 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.397+0000 7f0a9ffff700 1 --2- 172.21.15.47:0/2204685948 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0a88004e70 0x7f0a8800b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:20.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.397+0000 7f0a9ffff700 1 -- 172.21.15.47:0/2204685948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a88003750 msgr2=0x7f0a88003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:20.419 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.397+0000 7f0a9ffff700 1 --2- 172.21.15.47:0/2204685948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a88003750 0x7f0a88003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:20.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.397+0000 7f0a9ffff700 1 -- 172.21.15.47:0/2204685948 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a88003260 con 0x7f0a88004500 2022-06-29T15:52:20.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.397+0000 7f0aa6d9d700 1 --2- 172.21.15.47:0/2204685948 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0a88004e70 0x7f0a8800b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:52:20.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.397+0000 7f0aa6d9d700 1 -- 172.21.15.47:0/2204685948 reap_dead start 2022-06-29T15:52:20.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.397+0000 7f0a9ffff700 1 --2- 172.21.15.47:0/2204685948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a88004500 0x7f0a88004980 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 rx=0x7f0a9000c000 tx=0x7f0a90006d90).ready entity=mon.0 client_cookie=e6345302d19e047d server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:20.420 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.397+0000 7f0aa5d9b700 1 -- 172.21.15.47:0/2204685948 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a90013450 con 0x7f0a88004500 2022-06-29T15:52:20.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.397+0000 7f0aa5d9b700 1 -- 172.21.15.47:0/2204685948 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a900135b0 con 0x7f0a88004500 2022-06-29T15:52:20.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.397+0000 7f0aa5d9b700 1 -- 172.21.15.47:0/2204685948 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a90013910 con 0x7f0a88004500 2022-06-29T15:52:20.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.397+0000 7f0aa759e700 1 -- 172.21.15.47:0/2204685948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a88004500 msgr2=0x7f0a88004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:20.421 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.397+0000 7f0aa759e700 1 --2- 172.21.15.47:0/2204685948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a88004500 0x7f0a88004980 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 rx=0x7f0a9000c000 tx=0x7f0a90006d90).stop 2022-06-29T15:52:20.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.397+0000 7f0aa759e700 1 -- 172.21.15.47:0/2204685948 shutdown_connections 2022-06-29T15:52:20.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.397+0000 7f0aa759e700 1 --2- 172.21.15.47:0/2204685948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a88004500 0x7f0a88004980 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:20.422 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.397+0000 7f0aa759e700 1 -- 172.21.15.47:0/2204685948 >> 172.21.15.47:0/2204685948 conn(0x7f0aa8062df0 msgr2=0x7f0aa80631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:20.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.397+0000 7f0aa759e700 1 -- 172.21.15.47:0/2204685948 shutdown_connections 2022-06-29T15:52:20.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.397+0000 7f0aa759e700 1 -- 172.21.15.47:0/2204685948 wait complete. 2022-06-29T15:52:20.423 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.401+0000 7f0aa759e700 1 Processor -- start 2022-06-29T15:52:20.424 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.401+0000 7f0aa759e700 1 -- start start 2022-06-29T15:52:20.427 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.401+0000 7f0aa759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a88004500 0x7f0aa8145900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:20.427 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.401+0000 7f0aa759e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0aa80943d0 0x7f0aa81438f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:20.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.401+0000 7f0aa759e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0aa8143de0 0x7f0aa81555a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:20.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.401+0000 7f0aa759e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa8066220 con 0x7f0aa8143de0 2022-06-29T15:52:20.428 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.401+0000 7f0aa759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa8065a00 con 0x7f0a88004500 2022-06-29T15:52:20.429 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.401+0000 7f0aa759e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa8066570 con 0x7f0aa80943d0 2022-06-29T15:52:20.429 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.401+0000 7f0aa6d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0aa8143de0 0x7f0aa81555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:20.430 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.401+0000 7f0aa6d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0aa8143de0 0x7f0aa81555a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55186/0 (socket says 172.21.15.47:55186) 2022-06-29T15:52:20.430 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.401+0000 7f0aa6d9d700 1 -- 172.21.15.47:0/3840055133 learned_addr learned my addr 172.21.15.47:0/3840055133 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:20.431 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.401+0000 7f0aa659c700 1 --2- 172.21.15.47:0/3840055133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a88004500 0x7f0aa8145900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:20.431 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.401+0000 7f0a9ffff700 1 --2- 172.21.15.47:0/3840055133 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0aa80943d0 0x7f0aa81438f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:20.431 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.401+0000 7f0aa6d9d700 1 -- 172.21.15.47:0/3840055133 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0aa80943d0 msgr2=0x7f0aa81438f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:20.431 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.401+0000 7f0aa6d9d700 1 --2- 172.21.15.47:0/3840055133 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0aa80943d0 0x7f0aa81438f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:20.432 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.401+0000 7f0aa6d9d700 1 -- 172.21.15.47:0/3840055133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a88004500 msgr2=0x7f0aa8145900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:20.432 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.401+0000 7f0aa6d9d700 1 --2- 172.21.15.47:0/3840055133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a88004500 0x7f0aa8145900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:20.432 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.401+0000 7f0aa6d9d700 1 -- 172.21.15.47:0/3840055133 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa8067b90 con 0x7f0aa8143de0 2022-06-29T15:52:20.433 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.401+0000 7f0aa6d9d700 1 --2- 172.21.15.47:0/3840055133 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0aa8143de0 0x7f0aa81555a0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 rx=0x7f0aa8066100 tx=0x7f0aa0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:20.433 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.401+0000 7f0a9f7fe700 1 -- 172.21.15.47:0/3840055133 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0aa0019070 con 0x7f0aa8143de0 2022-06-29T15:52:20.433 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.401+0000 7f0a9f7fe700 1 -- 172.21.15.47:0/3840055133 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa0014810 con 0x7f0aa8143de0 2022-06-29T15:52:20.434 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.405+0000 7f0aa759e700 1 -- 172.21.15.47:0/3840055133 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0aa8144290 con 0x7f0aa8143de0 2022-06-29T15:52:20.434 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.405+0000 7f0a9f7fe700 1 -- 172.21.15.47:0/3840055133 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0aa0014b70 con 0x7f0aa8143de0 2022-06-29T15:52:20.434 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.405+0000 7f0aa759e700 1 -- 172.21.15.47:0/3840055133 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aa80678f0 con 0x7f0aa8143de0 2022-06-29T15:52:20.434 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.405+0000 7f0aa759e700 1 -- 172.21.15.47:0/3840055133 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0aa80657a0 con 0x7f0aa8143de0 2022-06-29T15:52:20.435 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.405+0000 7f0a9f7fe700 1 -- 172.21.15.47:0/3840055133 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0aa0012040 con 0x7f0aa8143de0 2022-06-29T15:52:20.435 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.405+0000 7f0a9f7fe700 1 --2- 172.21.15.47:0/3840055133 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0a8c053b30 0x7f0a8c055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:20.436 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.405+0000 7f0a9f7fe700 1 -- 172.21.15.47:0/3840055133 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f0aa006c0e0 con 0x7f0aa8143de0 2022-06-29T15:52:20.436 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.405+0000 7f0aa659c700 1 --2- 172.21.15.47:0/3840055133 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0a8c053b30 0x7f0a8c055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:20.437 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.405+0000 7f0aa659c700 1 --2- 172.21.15.47:0/3840055133 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0a8c053b30 0x7f0a8c055ff0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0x7f0a98005000 tx=0x7f0aa809a980).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:20.437 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.413+0000 7f0a9f7fe700 1 -- 172.21.15.47:0/3840055133 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0aa0042ea0 con 0x7f0aa8143de0 2022-06-29T15:52:20.543 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.537+0000 7f0aa759e700 1 -- 172.21.15.47:0/3840055133 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f0aa8065f10 con 0x7f0aa8143de0 2022-06-29T15:52:20.544 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.537+0000 7f0a9f7fe700 1 -- 172.21.15.47:0/3840055133 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v162) v1 ==== 110+0+10 (secure 0 0 0) 0x7f0aa00464c0 con 0x7f0aa8143de0 2022-06-29T15:52:20.544 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2022-06-29T15:52:20.546 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.537+0000 7f0aa759e700 1 -- 172.21.15.47:0/3840055133 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0a8c053b30 msgr2=0x7f0a8c055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:20.547 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.537+0000 7f0aa759e700 1 --2- 172.21.15.47:0/3840055133 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0a8c053b30 0x7f0a8c055ff0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0x7f0a98005000 tx=0x7f0aa809a980).stop 2022-06-29T15:52:20.547 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.537+0000 7f0aa759e700 1 -- 172.21.15.47:0/3840055133 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0aa8143de0 msgr2=0x7f0aa81555a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:20.547 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.537+0000 7f0aa759e700 1 --2- 172.21.15.47:0/3840055133 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0aa8143de0 0x7f0aa81555a0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 rx=0x7f0aa8066100 tx=0x7f0aa0008040).stop 2022-06-29T15:52:20.547 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.537+0000 7f0aa759e700 1 -- 172.21.15.47:0/3840055133 shutdown_connections 2022-06-29T15:52:20.548 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.537+0000 7f0aa759e700 1 --2- 172.21.15.47:0/3840055133 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0aa8143de0 0x7f0aa81555a0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:20.548 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.537+0000 7f0aa759e700 1 --2- 172.21.15.47:0/3840055133 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0aa80943d0 0x7f0aa81438f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:20.548 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.537+0000 7f0aa759e700 1 --2- 172.21.15.47:0/3840055133 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0a8c053b30 0x7f0a8c055ff0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:20.548 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.537+0000 7f0aa759e700 1 --2- 172.21.15.47:0/3840055133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a88004500 0x7f0aa8145900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:20.549 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.537+0000 7f0aa759e700 1 -- 172.21.15.47:0/3840055133 >> 172.21.15.47:0/3840055133 conn(0x7f0aa8062df0 msgr2=0x7f0aa80adbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:20.549 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.537+0000 7f0aa759e700 1 -- 172.21.15.47:0/3840055133 shutdown_connections 2022-06-29T15:52:20.549 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.537+0000 7f0aa759e700 1 -- 172.21.15.47:0/3840055133 wait complete. 2022-06-29T15:52:20.557 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-29T15:52:20.672 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.653+0000 7fe594015700 1 Processor -- start 2022-06-29T15:52:20.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.653+0000 7fe594015700 1 -- start start 2022-06-29T15:52:20.673 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.653+0000 7fe594015700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe58c060a70 0x7fe58c0afed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:20.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.653+0000 7fe594015700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe58c0a8ce0 0x7fe58c0b03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:20.674 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.653+0000 7fe594015700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe58c0b5e40 con 0x7fe58c0b08b0 2022-06-29T15:52:20.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.653+0000 7fe594015700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe58c0ac710 con 0x7fe58c0a8ce0 2022-06-29T15:52:20.675 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.653+0000 7fe594015700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe58c060e50 con 0x7fe58c060a70 2022-06-29T15:52:20.676 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.653+0000 7fe5925b2700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe58c0b08b0 0x7fe58c144670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40748/0 (socket says 172.21.15.47:40748) 2022-06-29T15:52:20.676 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.653+0000 7fe5925b2700 1 -- 172.21.15.47:0/145321889 learned_addr learned my addr 172.21.15.47:0/145321889 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:52:20.677 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.653+0000 7fe5915b0700 1 --2- 172.21.15.47:0/145321889 >> v2:172.21.15.64:3300/0 conn(0x7fe58c0a8ce0 0x7fe58c0b03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:20.677 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.653+0000 7fe591db1700 1 --2- 172.21.15.47:0/145321889 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe58c060a70 0x7fe58c0afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:20.677 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.653+0000 7fe590daf700 1 -- 172.21.15.47:0/145321889 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 999518314 0 0) 0x7fe584001490 con 0x7fe58c0b08b0 2022-06-29T15:52:20.678 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.653+0000 7fe590daf700 1 -- 172.21.15.47:0/145321889 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe574003260 con 0x7fe58c0b08b0 2022-06-29T15:52:20.678 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.653+0000 7fe5915b0700 1 -- 172.21.15.47:0/145321889 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe58c060a70 msgr2=0x7fe58c0afed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:20.678 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.653+0000 7fe5915b0700 1 --2- 172.21.15.47:0/145321889 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe58c060a70 0x7fe58c0afed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:20.678 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.653+0000 7fe5915b0700 1 -- 172.21.15.47:0/145321889 >> v1:172.21.15.47:6789/0 conn(0x7fe58c0b08b0 legacy=0x7fe58c144670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:20.679 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.653+0000 7fe5915b0700 1 -- 172.21.15.47:0/145321889 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe58c0ac2e0 con 0x7fe58c0a8ce0 2022-06-29T15:52:20.679 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.653+0000 7fe5915b0700 1 --2- 172.21.15.47:0/145321889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe58c0a8ce0 0x7fe58c0b03c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7fe588002f70 tx=0x7fe588006df0).ready entity=mon.1 client_cookie=16a8cce86da68d9a server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:20.679 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.653+0000 7fe590daf700 1 -- 172.21.15.47:0/145321889 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe58800d4a0 con 0x7fe58c0a8ce0 2022-06-29T15:52:20.680 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.653+0000 7fe590daf700 1 -- 172.21.15.47:0/145321889 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe588000e90 con 0x7fe58c0a8ce0 2022-06-29T15:52:20.680 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.653+0000 7fe590daf700 1 -- 172.21.15.47:0/145321889 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe58800d7f0 con 0x7fe58c0a8ce0 2022-06-29T15:52:20.680 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.653+0000 7fe594015700 1 -- 172.21.15.47:0/145321889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe58c0a8ce0 msgr2=0x7fe58c0b03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:20.680 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.653+0000 7fe594015700 1 --2- 172.21.15.47:0/145321889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe58c0a8ce0 0x7fe58c0b03c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7fe588002f70 tx=0x7fe588006df0).stop 2022-06-29T15:52:20.681 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.657+0000 7fe594015700 1 -- 172.21.15.47:0/145321889 shutdown_connections 2022-06-29T15:52:20.681 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.657+0000 7fe594015700 1 --2- 172.21.15.47:0/145321889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe58c0a8ce0 0x7fe58c0b03c0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:20.681 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.657+0000 7fe594015700 1 --2- 172.21.15.47:0/145321889 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe58c060a70 0x7fe58c0afed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:20.681 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.657+0000 7fe594015700 1 -- 172.21.15.47:0/145321889 >> 172.21.15.47:0/145321889 conn(0x7fe58c0626f0 msgr2=0x7fe58c062ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:20.682 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.657+0000 7fe594015700 1 -- 172.21.15.47:0/145321889 shutdown_connections 2022-06-29T15:52:20.682 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.657+0000 7fe594015700 1 -- 172.21.15.47:0/145321889 wait complete. 2022-06-29T15:52:20.682 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.657+0000 7fe594015700 1 Processor -- start 2022-06-29T15:52:20.682 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.657+0000 7fe594015700 1 -- start start 2022-06-29T15:52:20.683 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.657+0000 7fe594015700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe58c060a70 0x7fe58c064f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:20.683 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.657+0000 7fe594015700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe58c0a8ce0 0x7fe58c150af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:20.683 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.657+0000 7fe594015700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe58c0b08b0 0x7fe58c063380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:20.683 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.657+0000 7fe594015700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58c065740 con 0x7fe58c060a70 2022-06-29T15:52:20.684 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.657+0000 7fe594015700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58c0659e0 con 0x7fe58c0a8ce0 2022-06-29T15:52:20.684 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.657+0000 7fe594015700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe58c1503b0 con 0x7fe58c0b08b0 2022-06-29T15:52:20.684 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.657+0000 7fe591db1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe58c060a70 0x7fe58c064f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:20.684 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.657+0000 7fe591db1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe58c060a70 0x7fe58c064f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55196/0 (socket says 172.21.15.47:55196) 2022-06-29T15:52:20.684 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.657+0000 7fe591db1700 1 -- 172.21.15.47:0/4255902946 learned_addr learned my addr 172.21.15.47:0/4255902946 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:20.685 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.657+0000 7fe5915b0700 1 --2- 172.21.15.47:0/4255902946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe58c0a8ce0 0x7fe58c150af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:20.685 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.657+0000 7fe5925b2700 1 --2- 172.21.15.47:0/4255902946 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe58c0b08b0 0x7fe58c063380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:20.685 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.657+0000 7fe591db1700 1 -- 172.21.15.47:0/4255902946 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe58c0b08b0 msgr2=0x7fe58c063380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:20.685 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.657+0000 7fe591db1700 1 --2- 172.21.15.47:0/4255902946 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe58c0b08b0 0x7fe58c063380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:20.686 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.657+0000 7fe591db1700 1 -- 172.21.15.47:0/4255902946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe58c0a8ce0 msgr2=0x7fe58c150af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:20.686 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.657+0000 7fe591db1700 1 --2- 172.21.15.47:0/4255902946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe58c0a8ce0 0x7fe58c150af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:20.686 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.657+0000 7fe591db1700 1 -- 172.21.15.47:0/4255902946 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe58800b180 con 0x7fe58c060a70 2022-06-29T15:52:20.686 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.657+0000 7fe591db1700 1 --2- 172.21.15.47:0/4255902946 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe58c060a70 0x7fe58c064f50 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 rx=0x7fe58000c000 tx=0x7fe580006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:20.687 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.657+0000 7fe57effd700 1 -- 172.21.15.47:0/4255902946 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe580010780 con 0x7fe58c060a70 2022-06-29T15:52:20.687 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.657+0000 7fe594015700 1 -- 172.21.15.47:0/4255902946 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe58c150650 con 0x7fe58c060a70 2022-06-29T15:52:20.687 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.657+0000 7fe57effd700 1 -- 172.21.15.47:0/4255902946 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5800108e0 con 0x7fe58c060a70 2022-06-29T15:52:20.687 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.657+0000 7fe57effd700 1 -- 172.21.15.47:0/4255902946 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe580010c40 con 0x7fe58c060a70 2022-06-29T15:52:20.688 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.661+0000 7fe594015700 1 -- 172.21.15.47:0/4255902946 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe58c150fe0 con 0x7fe58c060a70 2022-06-29T15:52:20.688 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.661+0000 7fe594015700 1 -- 172.21.15.47:0/4255902946 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe58c0573d0 con 0x7fe58c060a70 2022-06-29T15:52:20.688 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.661+0000 7fe57effd700 1 -- 172.21.15.47:0/4255902946 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe58001c460 con 0x7fe58c060a70 2022-06-29T15:52:20.688 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.661+0000 7fe57effd700 1 --2- 172.21.15.47:0/4255902946 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe56c053b00 0x7fe56c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:20.688 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.661+0000 7fe57effd700 1 -- 172.21.15.47:0/4255902946 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe58006c6f0 con 0x7fe58c060a70 2022-06-29T15:52:20.689 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.661+0000 7fe5915b0700 1 --2- 172.21.15.47:0/4255902946 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe56c053b00 0x7fe56c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:20.689 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.661+0000 7fe5915b0700 1 --2- 172.21.15.47:0/4255902946 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe56c053b00 0x7fe56c055fc0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 rx=0x7fe588018000 tx=0x7fe588014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:20.689 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.665+0000 7fe57effd700 1 -- 172.21.15.47:0/4255902946 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe5800432c0 con 0x7fe58c060a70 2022-06-29T15:52:20.798 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.789+0000 7fe594015700 1 -- 172.21.15.47:0/4255902946 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe58c06a210 con 0x7fe58c060a70 2022-06-29T15:52:20.799 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.793+0000 7fe57effd700 1 -- 172.21.15.47:0/4255902946 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v162) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe5800468e0 con 0x7fe58c060a70 2022-06-29T15:52:20.799 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:52:20.801 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.793+0000 7fe594015700 1 -- 172.21.15.47:0/4255902946 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe56c053b00 msgr2=0x7fe56c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:20.802 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.793+0000 7fe594015700 1 --2- 172.21.15.47:0/4255902946 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe56c053b00 0x7fe56c055fc0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 rx=0x7fe588018000 tx=0x7fe588014000).stop 2022-06-29T15:52:20.802 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.793+0000 7fe594015700 1 -- 172.21.15.47:0/4255902946 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe58c060a70 msgr2=0x7fe58c064f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:20.802 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.793+0000 7fe594015700 1 --2- 172.21.15.47:0/4255902946 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe58c060a70 0x7fe58c064f50 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 rx=0x7fe58000c000 tx=0x7fe580006df0).stop 2022-06-29T15:52:20.802 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.793+0000 7fe594015700 1 -- 172.21.15.47:0/4255902946 shutdown_connections 2022-06-29T15:52:20.803 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.793+0000 7fe594015700 1 --2- 172.21.15.47:0/4255902946 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe58c0b08b0 0x7fe58c063380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:20.803 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.793+0000 7fe594015700 1 --2- 172.21.15.47:0/4255902946 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe56c053b00 0x7fe56c055fc0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:20.803 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.793+0000 7fe594015700 1 --2- 172.21.15.47:0/4255902946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe58c0a8ce0 0x7fe58c150af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:20.803 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.793+0000 7fe594015700 1 --2- 172.21.15.47:0/4255902946 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe58c060a70 0x7fe58c064f50 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:20.803 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.793+0000 7fe594015700 1 -- 172.21.15.47:0/4255902946 >> 172.21.15.47:0/4255902946 conn(0x7fe58c0626f0 msgr2=0x7fe58c0b56b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:20.804 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.793+0000 7fe594015700 1 -- 172.21.15.47:0/4255902946 shutdown_connections 2022-06-29T15:52:20.804 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.793+0000 7fe594015700 1 -- 172.21.15.47:0/4255902946 wait complete. 2022-06-29T15:52:20.813 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-29T15:52:20.917 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.909+0000 7f2089acf700 1 Processor -- start 2022-06-29T15:52:20.917 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.909+0000 7f2089acf700 1 -- start start 2022-06-29T15:52:20.919 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.909+0000 7f2089acf700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f20840b5d90 0x7f2084143a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:20.919 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.909+0000 7f2089acf700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f20840676a0 0x7f2084140130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:20.919 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.909+0000 7f2089acf700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f208405cc30 con 0x7f208405c850 2022-06-29T15:52:20.920 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.909+0000 7f2089acf700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20841430a0 con 0x7f20840676a0 2022-06-29T15:52:20.920 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.909+0000 7f2089acf700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2084143300 con 0x7f20840b5d90 2022-06-29T15:52:20.920 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.909+0000 7f20892ce700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f20840676a0 0x7f2084140130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:20.920 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.913+0000 7f20892ce700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f20840676a0 0x7f2084140130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55208/0 (socket says 172.21.15.47:55208) 2022-06-29T15:52:20.921 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.913+0000 7f20892ce700 1 -- 172.21.15.47:0/4160378421 learned_addr learned my addr 172.21.15.47:0/4160378421 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:20.921 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.913+0000 7f2083fff700 1 --2- 172.21.15.47:0/4160378421 >> v2:172.21.15.64:3300/0 conn(0x7f20840b5d90 0x7f2084143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:20.923 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.913+0000 7f20892ce700 1 -- 172.21.15.47:0/4160378421 >> v2:172.21.15.64:3300/0 conn(0x7f20840b5d90 msgr2=0x7f2084143a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:20.923 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.913+0000 7f20892ce700 1 --2- 172.21.15.47:0/4160378421 >> v2:172.21.15.64:3300/0 conn(0x7f20840b5d90 0x7f2084143a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:20.923 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.913+0000 7f20892ce700 1 -- 172.21.15.47:0/4160378421 >> v1:172.21.15.64:6789/0 conn(0x7f208405c850 legacy=0x7f20840b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:20.924 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.913+0000 7f20892ce700 1 -- 172.21.15.47:0/4160378421 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20841435a0 con 0x7f20840676a0 2022-06-29T15:52:20.924 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.913+0000 7f20892ce700 1 --2- 172.21.15.47:0/4160378421 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20840676a0 0x7f2084140130 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 rx=0x7f207c0032f0 tx=0x7f207c009290).ready entity=mon.0 client_cookie=33fd4a552441b432 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:20.924 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.913+0000 7f20837fe700 1 -- 172.21.15.47:0/4160378421 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f207c00b870 con 0x7f20840676a0 2022-06-29T15:52:20.924 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.913+0000 7f20837fe700 1 -- 172.21.15.47:0/4160378421 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f207c00b9d0 con 0x7f20840676a0 2022-06-29T15:52:20.925 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.913+0000 7f20837fe700 1 -- 172.21.15.47:0/4160378421 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f207c00bb40 con 0x7f20840676a0 2022-06-29T15:52:20.925 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.913+0000 7f2089acf700 1 -- 172.21.15.47:0/4160378421 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20840676a0 msgr2=0x7f2084140130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:20.925 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.913+0000 7f2089acf700 1 --2- 172.21.15.47:0/4160378421 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20840676a0 0x7f2084140130 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 rx=0x7f207c0032f0 tx=0x7f207c009290).stop 2022-06-29T15:52:20.926 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.913+0000 7f2089acf700 1 -- 172.21.15.47:0/4160378421 shutdown_connections 2022-06-29T15:52:20.926 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.913+0000 7f2089acf700 1 --2- 172.21.15.47:0/4160378421 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20840676a0 0x7f2084140130 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:20.926 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.913+0000 7f2089acf700 1 --2- 172.21.15.47:0/4160378421 >> v2:172.21.15.64:3300/0 conn(0x7f20840b5d90 0x7f2084143a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:20.926 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.913+0000 7f2089acf700 1 -- 172.21.15.47:0/4160378421 >> 172.21.15.47:0/4160378421 conn(0x7f2084060e00 msgr2=0x7f20840611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:20.927 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.913+0000 7f2089acf700 1 -- 172.21.15.47:0/4160378421 shutdown_connections 2022-06-29T15:52:20.927 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.913+0000 7f2089acf700 1 -- 172.21.15.47:0/4160378421 wait complete. 2022-06-29T15:52:20.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.917+0000 7f2089acf700 1 Processor -- start 2022-06-29T15:52:20.930 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.917+0000 7f2089acf700 1 -- start start 2022-06-29T15:52:20.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.917+0000 7f2089acf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f208405c850 0x7f2084152680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:20.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.917+0000 7f2089acf700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f20840676a0 0x7f20841299f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:20.931 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.917+0000 7f2089acf700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20840b5d90 0x7f20840b0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:20.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.917+0000 7f2089acf700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20840b0b50 con 0x7f20840b5d90 2022-06-29T15:52:20.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.917+0000 7f2089acf700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2084141750 con 0x7f208405c850 2022-06-29T15:52:20.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.917+0000 7f2089acf700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20841419f0 con 0x7f20840676a0 2022-06-29T15:52:20.932 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.917+0000 7f20892ce700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20840b5d90 0x7f20840b0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:20.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.917+0000 7f20892ce700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20840b5d90 0x7f20840b0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55214/0 (socket says 172.21.15.47:55214) 2022-06-29T15:52:20.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.917+0000 7f20892ce700 1 -- 172.21.15.47:0/2021447940 learned_addr learned my addr 172.21.15.47:0/2021447940 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:20.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.917+0000 7f2088acd700 1 --2- 172.21.15.47:0/2021447940 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f208405c850 0x7f2084152680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:20.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.917+0000 7f2083fff700 1 --2- 172.21.15.47:0/2021447940 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f20840676a0 0x7f20841299f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:20.933 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.917+0000 7f20892ce700 1 -- 172.21.15.47:0/2021447940 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f20840676a0 msgr2=0x7f20841299f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:20.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.917+0000 7f20892ce700 1 --2- 172.21.15.47:0/2021447940 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f20840676a0 0x7f20841299f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:20.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.917+0000 7f20892ce700 1 -- 172.21.15.47:0/2021447940 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f208405c850 msgr2=0x7f2084152680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:20.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.917+0000 7f20892ce700 1 --2- 172.21.15.47:0/2021447940 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f208405c850 0x7f2084152680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:20.934 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.917+0000 7f20892ce700 1 -- 172.21.15.47:0/2021447940 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f207c009530 con 0x7f20840b5d90 2022-06-29T15:52:20.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.917+0000 7f2083fff700 1 --2- 172.21.15.47:0/2021447940 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f20840676a0 0x7f20841299f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:52:20.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.921+0000 7f20892ce700 1 --2- 172.21.15.47:0/2021447940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20840b5d90 0x7f20840b0f40 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 rx=0x7f207c0032f0 tx=0x7f207c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:20.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.921+0000 7f2081ffb700 1 -- 172.21.15.47:0/2021447940 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f207c00b870 con 0x7f20840b5d90 2022-06-29T15:52:20.935 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.921+0000 7f2081ffb700 1 -- 172.21.15.47:0/2021447940 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f207c021440 con 0x7f20840b5d90 2022-06-29T15:52:20.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.921+0000 7f2081ffb700 1 -- 172.21.15.47:0/2021447940 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f207c0217a0 con 0x7f20840b5d90 2022-06-29T15:52:20.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.921+0000 7f2089acf700 1 -- 172.21.15.47:0/2021447940 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2084141c90 con 0x7f20840b5d90 2022-06-29T15:52:20.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.921+0000 7f2089acf700 1 -- 172.21.15.47:0/2021447940 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f208412a2c0 con 0x7f20840b5d90 2022-06-29T15:52:20.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.921+0000 7f2089acf700 1 -- 172.21.15.47:0/2021447940 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20840573d0 con 0x7f20840b5d90 2022-06-29T15:52:20.936 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.921+0000 7f2081ffb700 1 -- 172.21.15.47:0/2021447940 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f207c027020 con 0x7f20840b5d90 2022-06-29T15:52:20.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.921+0000 7f2081ffb700 1 --2- 172.21.15.47:0/2021447940 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2064053b30 0x7f2064055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:20.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.921+0000 7f2081ffb700 1 -- 172.21.15.47:0/2021447940 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f207c06d4e0 con 0x7f20840b5d90 2022-06-29T15:52:20.937 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.921+0000 7f2088acd700 1 --2- 172.21.15.47:0/2021447940 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2064053b30 0x7f2064055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:20.938 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.925+0000 7f2088acd700 1 --2- 172.21.15.47:0/2021447940 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2064053b30 0x7f2064055ff0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7f207800d000 tx=0x7f2078007bc0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:20.938 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:20.925+0000 7f2081ffb700 1 -- 172.21.15.47:0/2021447940 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f207c0442a0 con 0x7f20840b5d90 2022-06-29T15:52:21.060 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.053+0000 7f2089acf700 1 -- 172.21.15.47:0/2021447940 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2084144170 con 0x7f20840b5d90 2022-06-29T15:52:21.061 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.053+0000 7f2081ffb700 1 -- 172.21.15.47:0/2021447940 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v162) v1 ==== 104+0+10 (secure 0 0 0) 0x7f207c0150a0 con 0x7f20840b5d90 2022-06-29T15:52:21.062 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:52:21.064 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.057+0000 7f2089acf700 1 -- 172.21.15.47:0/2021447940 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2064053b30 msgr2=0x7f2064055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:21.064 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.057+0000 7f2089acf700 1 --2- 172.21.15.47:0/2021447940 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2064053b30 0x7f2064055ff0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7f207800d000 tx=0x7f2078007bc0).stop 2022-06-29T15:52:21.064 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.057+0000 7f2089acf700 1 -- 172.21.15.47:0/2021447940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20840b5d90 msgr2=0x7f20840b0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:21.064 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.057+0000 7f2089acf700 1 --2- 172.21.15.47:0/2021447940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20840b5d90 0x7f20840b0f40 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 rx=0x7f207c0032f0 tx=0x7f207c016000).stop 2022-06-29T15:52:21.065 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.057+0000 7f2089acf700 1 -- 172.21.15.47:0/2021447940 shutdown_connections 2022-06-29T15:52:21.065 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.057+0000 7f2089acf700 1 --2- 172.21.15.47:0/2021447940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20840b5d90 0x7f20840b0f40 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:21.065 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.057+0000 7f2089acf700 1 --2- 172.21.15.47:0/2021447940 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f20840676a0 0x7f20841299f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:21.065 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.057+0000 7f2089acf700 1 --2- 172.21.15.47:0/2021447940 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2064053b30 0x7f2064055ff0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:21.066 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.057+0000 7f2089acf700 1 --2- 172.21.15.47:0/2021447940 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f208405c850 0x7f2084152680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:21.066 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.057+0000 7f2089acf700 1 -- 172.21.15.47:0/2021447940 >> 172.21.15.47:0/2021447940 conn(0x7f2084060e00 msgr2=0x7f20840b3c00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:21.066 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.057+0000 7f2089acf700 1 -- 172.21.15.47:0/2021447940 shutdown_connections 2022-06-29T15:52:21.066 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.057+0000 7f2089acf700 1 -- 172.21.15.47:0/2021447940 wait complete. 2022-06-29T15:52:21.074 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-29T15:52:21.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.169+0000 7fdc31e1c700 1 Processor -- start 2022-06-29T15:52:21.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.173+0000 7fdc31e1c700 1 -- start start 2022-06-29T15:52:21.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.173+0000 7fdc31e1c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdc2c0b5d90 0x7fdc2c143a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:21.182 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.173+0000 7fdc31e1c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdc2c0676a0 0x7fdc2c140130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:21.182 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.173+0000 7fdc31e1c700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc2c05cc30 con 0x7fdc2c05c850 2022-06-29T15:52:21.182 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.173+0000 7fdc31e1c700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc2c1430a0 con 0x7fdc2c0676a0 2022-06-29T15:52:21.182 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.173+0000 7fdc31e1c700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c143300 con 0x7fdc2c0b5d90 2022-06-29T15:52:21.183 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.173+0000 7fdc3161b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdc2c0676a0 0x7fdc2c140130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:21.183 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.173+0000 7fdc3161b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdc2c0676a0 0x7fdc2c140130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55220/0 (socket says 172.21.15.47:55220) 2022-06-29T15:52:21.183 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.173+0000 7fdc3161b700 1 -- 172.21.15.47:0/1993637021 learned_addr learned my addr 172.21.15.47:0/1993637021 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:21.183 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.173+0000 7fdc2bfff700 1 --2- 172.21.15.47:0/1993637021 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdc2c0b5d90 0x7fdc2c143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:21.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.173+0000 7fdc3161b700 1 -- 172.21.15.47:0/1993637021 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdc2c0b5d90 msgr2=0x7fdc2c143a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:21.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.173+0000 7fdc3161b700 1 --2- 172.21.15.47:0/1993637021 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdc2c0b5d90 0x7fdc2c143a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:21.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.173+0000 7fdc3161b700 1 -- 172.21.15.47:0/1993637021 >> v1:172.21.15.64:6789/0 conn(0x7fdc2c05c850 legacy=0x7fdc2c0b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:21.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.173+0000 7fdc3161b700 1 -- 172.21.15.47:0/1993637021 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc2c1435a0 con 0x7fdc2c0676a0 2022-06-29T15:52:21.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.173+0000 7fdc3161b700 1 --2- 172.21.15.47:0/1993637021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc2c0676a0 0x7fdc2c140130 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 rx=0x7fdc240032f0 tx=0x7fdc24009290).ready entity=mon.0 client_cookie=f7de88eb8667e6f6 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:21.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.173+0000 7fdc2b7fe700 1 -- 172.21.15.47:0/1993637021 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc2400b8a0 con 0x7fdc2c0676a0 2022-06-29T15:52:21.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.173+0000 7fdc2b7fe700 1 -- 172.21.15.47:0/1993637021 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc2400ba00 con 0x7fdc2c0676a0 2022-06-29T15:52:21.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.173+0000 7fdc2b7fe700 1 -- 172.21.15.47:0/1993637021 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc2400bb70 con 0x7fdc2c0676a0 2022-06-29T15:52:21.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.173+0000 7fdc31e1c700 1 -- 172.21.15.47:0/1993637021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc2c0676a0 msgr2=0x7fdc2c140130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:21.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.173+0000 7fdc31e1c700 1 --2- 172.21.15.47:0/1993637021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc2c0676a0 0x7fdc2c140130 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 rx=0x7fdc240032f0 tx=0x7fdc24009290).stop 2022-06-29T15:52:21.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.173+0000 7fdc31e1c700 1 -- 172.21.15.47:0/1993637021 shutdown_connections 2022-06-29T15:52:21.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.173+0000 7fdc31e1c700 1 --2- 172.21.15.47:0/1993637021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc2c0676a0 0x7fdc2c140130 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:21.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.173+0000 7fdc31e1c700 1 --2- 172.21.15.47:0/1993637021 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdc2c0b5d90 0x7fdc2c143a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:21.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.173+0000 7fdc31e1c700 1 -- 172.21.15.47:0/1993637021 >> 172.21.15.47:0/1993637021 conn(0x7fdc2c060e00 msgr2=0x7fdc2c0611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:21.191 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.173+0000 7fdc31e1c700 1 -- 172.21.15.47:0/1993637021 shutdown_connections 2022-06-29T15:52:21.191 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.173+0000 7fdc31e1c700 1 -- 172.21.15.47:0/1993637021 wait complete. 2022-06-29T15:52:21.191 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.177+0000 7fdc31e1c700 1 Processor -- start 2022-06-29T15:52:21.191 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.177+0000 7fdc31e1c700 1 -- start start 2022-06-29T15:52:21.191 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.177+0000 7fdc31e1c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdc2c05c850 0x7fdc2c144be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:21.192 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.177+0000 7fdc31e1c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc2c0676a0 0x7fdc2c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:21.192 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.177+0000 7fdc31e1c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc2c0b5d90 0x7fdc2c152b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:21.192 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.177+0000 7fdc31e1c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c152210 con 0x7fdc2c0676a0 2022-06-29T15:52:21.192 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.177+0000 7fdc31e1c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c0b7c90 con 0x7fdc2c0b5d90 2022-06-29T15:52:21.193 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.177+0000 7fdc31e1c700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c14a600 con 0x7fdc2c05c850 2022-06-29T15:52:21.193 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.177+0000 7fdc2bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc2c0676a0 0x7fdc2c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:21.193 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.177+0000 7fdc2bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc2c0676a0 0x7fdc2c0b77a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55224/0 (socket says 172.21.15.47:55224) 2022-06-29T15:52:21.193 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.177+0000 7fdc2bfff700 1 -- 172.21.15.47:0/754681886 learned_addr learned my addr 172.21.15.47:0/754681886 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:21.194 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.177+0000 7fdc3161b700 1 --2- 172.21.15.47:0/754681886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc2c0b5d90 0x7fdc2c152b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:21.194 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.177+0000 7fdc30e1a700 1 --2- 172.21.15.47:0/754681886 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdc2c05c850 0x7fdc2c144be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:21.194 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.177+0000 7fdc2bfff700 1 -- 172.21.15.47:0/754681886 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdc2c05c850 msgr2=0x7fdc2c144be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:21.194 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.177+0000 7fdc2bfff700 1 --2- 172.21.15.47:0/754681886 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdc2c05c850 0x7fdc2c144be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:21.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.177+0000 7fdc2bfff700 1 -- 172.21.15.47:0/754681886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc2c0b5d90 msgr2=0x7fdc2c152b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:21.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.177+0000 7fdc2bfff700 1 --2- 172.21.15.47:0/754681886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc2c0b5d90 0x7fdc2c152b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:21.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.177+0000 7fdc2bfff700 1 -- 172.21.15.47:0/754681886 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc24009530 con 0x7fdc2c0676a0 2022-06-29T15:52:21.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.177+0000 7fdc3161b700 1 --2- 172.21.15.47:0/754681886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc2c0b5d90 0x7fdc2c152b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:52:21.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.177+0000 7fdc2bfff700 1 --2- 172.21.15.47:0/754681886 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc2c0676a0 0x7fdc2c0b77a0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 rx=0x7fdc2c148f50 tx=0x7fdc14006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:21.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.177+0000 7fdc29ffb700 1 -- 172.21.15.47:0/754681886 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc1400c830 con 0x7fdc2c0676a0 2022-06-29T15:52:21.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.177+0000 7fdc29ffb700 1 -- 172.21.15.47:0/754681886 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc1400c990 con 0x7fdc2c0676a0 2022-06-29T15:52:21.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.181+0000 7fdc31e1c700 1 -- 172.21.15.47:0/754681886 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc2c152400 con 0x7fdc2c0676a0 2022-06-29T15:52:21.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.181+0000 7fdc29ffb700 1 -- 172.21.15.47:0/754681886 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc1400ccf0 con 0x7fdc2c0676a0 2022-06-29T15:52:21.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.181+0000 7fdc31e1c700 1 -- 172.21.15.47:0/754681886 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc2c0b74a0 con 0x7fdc2c0676a0 2022-06-29T15:52:21.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.181+0000 7fdc31e1c700 1 -- 172.21.15.47:0/754681886 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc2c14a800 con 0x7fdc2c0676a0 2022-06-29T15:52:21.198 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.181+0000 7fdc29ffb700 1 -- 172.21.15.47:0/754681886 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fdc1401c460 con 0x7fdc2c0676a0 2022-06-29T15:52:21.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.181+0000 7fdc29ffb700 1 --2- 172.21.15.47:0/754681886 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdc10053b30 0x7fdc10055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:21.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.181+0000 7fdc29ffb700 1 -- 172.21.15.47:0/754681886 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fdc1406c550 con 0x7fdc2c0676a0 2022-06-29T15:52:21.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.181+0000 7fdc30e1a700 1 --2- 172.21.15.47:0/754681886 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdc10053b30 0x7fdc10055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:21.200 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.181+0000 7fdc30e1a700 1 --2- 172.21.15.47:0/754681886 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdc10053b30 0x7fdc10055ff0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0x7fdc1c00b000 tx=0x7fdc1c009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:21.200 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.189+0000 7fdc29ffb700 1 -- 172.21.15.47:0/754681886 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdc140432b0 con 0x7fdc2c0676a0 2022-06-29T15:52:21.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.317+0000 7fdc31e1c700 1 -- 172.21.15.47:0/754681886 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdc2c0573d0 con 0x7fdc2c0676a0 2022-06-29T15:52:21.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.317+0000 7fdc29ffb700 1 -- 172.21.15.47:0/754681886 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v162) v1 ==== 100+0+10 (secure 0 0 0) 0x7fdc140468d0 con 0x7fdc2c0676a0 2022-06-29T15:52:21.327 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:52:21.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.321+0000 7fdc31e1c700 1 -- 172.21.15.47:0/754681886 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdc10053b30 msgr2=0x7fdc10055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:21.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.321+0000 7fdc31e1c700 1 --2- 172.21.15.47:0/754681886 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdc10053b30 0x7fdc10055ff0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0x7fdc1c00b000 tx=0x7fdc1c009040).stop 2022-06-29T15:52:21.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.321+0000 7fdc31e1c700 1 -- 172.21.15.47:0/754681886 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc2c0676a0 msgr2=0x7fdc2c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:21.329 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.321+0000 7fdc31e1c700 1 --2- 172.21.15.47:0/754681886 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc2c0676a0 0x7fdc2c0b77a0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 rx=0x7fdc2c148f50 tx=0x7fdc14006df0).stop 2022-06-29T15:52:21.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.321+0000 7fdc31e1c700 1 -- 172.21.15.47:0/754681886 shutdown_connections 2022-06-29T15:52:21.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.321+0000 7fdc31e1c700 1 --2- 172.21.15.47:0/754681886 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdc10053b30 0x7fdc10055ff0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:21.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.321+0000 7fdc31e1c700 1 --2- 172.21.15.47:0/754681886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc2c0b5d90 0x7fdc2c152b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:21.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.321+0000 7fdc31e1c700 1 --2- 172.21.15.47:0/754681886 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc2c0676a0 0x7fdc2c0b77a0 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:21.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.321+0000 7fdc31e1c700 1 --2- 172.21.15.47:0/754681886 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdc2c05c850 0x7fdc2c144be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:21.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.321+0000 7fdc31e1c700 1 -- 172.21.15.47:0/754681886 >> 172.21.15.47:0/754681886 conn(0x7fdc2c060e00 msgr2=0x7fdc2c141950 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:21.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.321+0000 7fdc31e1c700 1 -- 172.21.15.47:0/754681886 shutdown_connections 2022-06-29T15:52:21.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.321+0000 7fdc31e1c700 1 -- 172.21.15.47:0/754681886 wait complete. 2022-06-29T15:52:21.344 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-29T15:52:21.345 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:52:21.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.441+0000 7f6233b5b700 1 Processor -- start 2022-06-29T15:52:21.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.441+0000 7f6233b5b700 1 -- start start 2022-06-29T15:52:21.450 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.441+0000 7f6233b5b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f622c0a93c0 0x7f622c095240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:21.451 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.441+0000 7f6233b5b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f622c095730 0x7f622c097690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:21.451 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.441+0000 7f6233b5b700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f622c143be0 con 0x7f622c1430f0 2022-06-29T15:52:21.451 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.441+0000 7f6233b5b700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f622c13d590 con 0x7f622c095730 2022-06-29T15:52:21.451 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.441+0000 7f6233b5b700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f622c060e50 con 0x7f622c0a93c0 2022-06-29T15:52:21.452 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.441+0000 7f62320f8700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f622c095730 0x7f622c097690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:21.452 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.441+0000 7f62310f6700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f622c0a93c0 0x7f622c095240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:21.452 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.441+0000 7f62320f8700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f622c095730 0x7f622c097690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55236/0 (socket says 172.21.15.47:55236) 2022-06-29T15:52:21.452 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.441+0000 7f62310f6700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f622c0a93c0 0x7f622c095240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:60938/0 (socket says 172.21.15.47:60938) 2022-06-29T15:52:21.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.441+0000 7f62318f7700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f622c1430f0 0x7f622c1434d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:48350/0 (socket says 172.21.15.47:48350) 2022-06-29T15:52:21.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.441+0000 7f62320f8700 1 -- 172.21.15.47:0/1807802518 learned_addr learned my addr 172.21.15.47:0/1807802518 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:21.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.441+0000 7f62320f8700 1 -- 172.21.15.47:0/1807802518 >> v2:172.21.15.64:3300/0 conn(0x7f622c0a93c0 msgr2=0x7f622c095240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:21.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.441+0000 7f62320f8700 1 --2- 172.21.15.47:0/1807802518 >> v2:172.21.15.64:3300/0 conn(0x7f622c0a93c0 0x7f622c095240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:21.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.441+0000 7f62320f8700 1 -- 172.21.15.47:0/1807802518 >> v1:172.21.15.64:6789/0 conn(0x7f622c1430f0 legacy=0x7f622c1434d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:21.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.441+0000 7f62320f8700 1 -- 172.21.15.47:0/1807802518 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f622c0610f0 con 0x7f622c095730 2022-06-29T15:52:21.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.441+0000 7f62310f6700 1 --2- 172.21.15.47:0/1807802518 >> v2:172.21.15.64:3300/0 conn(0x7f622c0a93c0 0x7f622c095240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:52:21.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.441+0000 7f62320f8700 1 --2- 172.21.15.47:0/1807802518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f622c095730 0x7f622c097690 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 rx=0x7f6228002f70 tx=0x7f6228006eb0).ready entity=mon.0 client_cookie=a899b554643b620f server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:21.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.441+0000 7f62308f5700 1 -- 172.21.15.47:0/1807802518 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f622800d5c0 con 0x7f622c095730 2022-06-29T15:52:21.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.441+0000 7f62308f5700 1 -- 172.21.15.47:0/1807802518 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6228000e90 con 0x7f622c095730 2022-06-29T15:52:21.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.441+0000 7f62308f5700 1 -- 172.21.15.47:0/1807802518 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f622800d950 con 0x7f622c095730 2022-06-29T15:52:21.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.445+0000 7f6233b5b700 1 -- 172.21.15.47:0/1807802518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f622c095730 msgr2=0x7f622c097690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:21.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.445+0000 7f6233b5b700 1 --2- 172.21.15.47:0/1807802518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f622c095730 0x7f622c097690 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 rx=0x7f6228002f70 tx=0x7f6228006eb0).stop 2022-06-29T15:52:21.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.445+0000 7f6233b5b700 1 -- 172.21.15.47:0/1807802518 shutdown_connections 2022-06-29T15:52:21.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.445+0000 7f6233b5b700 1 --2- 172.21.15.47:0/1807802518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f622c095730 0x7f622c097690 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:21.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.445+0000 7f6233b5b700 1 --2- 172.21.15.47:0/1807802518 >> v2:172.21.15.64:3300/0 conn(0x7f622c0a93c0 0x7f622c095240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:21.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.445+0000 7f6233b5b700 1 -- 172.21.15.47:0/1807802518 >> 172.21.15.47:0/1807802518 conn(0x7f622c05f0b0 msgr2=0x7f622c05f490 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:21.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.445+0000 7f6233b5b700 1 -- 172.21.15.47:0/1807802518 shutdown_connections 2022-06-29T15:52:21.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.445+0000 7f6233b5b700 1 -- 172.21.15.47:0/1807802518 wait complete. 2022-06-29T15:52:21.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.445+0000 7f6233b5b700 1 Processor -- start 2022-06-29T15:52:21.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.445+0000 7f6233b5b700 1 -- start start 2022-06-29T15:52:21.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.449+0000 7f6233b5b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f622c095730 0x7f622c0b5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:21.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.449+0000 7f6233b5b700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f622c0a93c0 0x7f622c135470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:21.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.449+0000 7f6233b5b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f622c1430f0 0x7f622c099d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:21.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.449+0000 7f6233b5b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622c0a98a0 con 0x7f622c1430f0 2022-06-29T15:52:21.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.449+0000 7f6233b5b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622c135960 con 0x7f622c095730 2022-06-29T15:52:21.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.449+0000 7f6233b5b700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f622c13f6e0 con 0x7f622c0a93c0 2022-06-29T15:52:21.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.449+0000 7f62320f8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f622c1430f0 0x7f622c099d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:21.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.449+0000 7f62318f7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f622c095730 0x7f622c0b5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:21.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.449+0000 7f62320f8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f622c1430f0 0x7f622c099d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55242/0 (socket says 172.21.15.47:55242) 2022-06-29T15:52:21.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.449+0000 7f62320f8700 1 -- 172.21.15.47:0/1694676211 learned_addr learned my addr 172.21.15.47:0/1694676211 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:21.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.449+0000 7f62310f6700 1 --2- 172.21.15.47:0/1694676211 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f622c0a93c0 0x7f622c135470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:21.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.449+0000 7f62320f8700 1 -- 172.21.15.47:0/1694676211 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f622c0a93c0 msgr2=0x7f622c135470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:21.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.449+0000 7f62320f8700 1 --2- 172.21.15.47:0/1694676211 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f622c0a93c0 0x7f622c135470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:21.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.449+0000 7f62320f8700 1 -- 172.21.15.47:0/1694676211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f622c095730 msgr2=0x7f622c0b5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:21.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.449+0000 7f62320f8700 1 --2- 172.21.15.47:0/1694676211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f622c095730 0x7f622c0b5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:21.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.449+0000 7f62320f8700 1 -- 172.21.15.47:0/1694676211 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6224009470 con 0x7f622c1430f0 2022-06-29T15:52:21.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.449+0000 7f62320f8700 1 --2- 172.21.15.47:0/1694676211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f622c1430f0 0x7f622c099d10 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 rx=0x7f6228002f70 tx=0x7f6228015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:21.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.449+0000 7f6222ffd700 1 -- 172.21.15.47:0/1694676211 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f62280116e0 con 0x7f622c1430f0 2022-06-29T15:52:21.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.449+0000 7f6222ffd700 1 -- 172.21.15.47:0/1694676211 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6228011840 con 0x7f622c1430f0 2022-06-29T15:52:21.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.449+0000 7f6233b5b700 1 -- 172.21.15.47:0/1694676211 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f622800b1e0 con 0x7f622c1430f0 2022-06-29T15:52:21.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.449+0000 7f6222ffd700 1 -- 172.21.15.47:0/1694676211 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6228011ba0 con 0x7f622c1430f0 2022-06-29T15:52:21.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.449+0000 7f6233b5b700 1 -- 172.21.15.47:0/1694676211 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f622c09af50 con 0x7f622c1430f0 2022-06-29T15:52:21.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.453+0000 7f6233b5b700 1 -- 172.21.15.47:0/1694676211 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f622c13d940 con 0x7f622c1430f0 2022-06-29T15:52:21.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.453+0000 7f6222ffd700 1 -- 172.21.15.47:0/1694676211 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6228025030 con 0x7f622c1430f0 2022-06-29T15:52:21.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.453+0000 7f6222ffd700 1 --2- 172.21.15.47:0/1694676211 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6214053a70 0x7f6214055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:21.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.453+0000 7f6222ffd700 1 -- 172.21.15.47:0/1694676211 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f622806d1f0 con 0x7f622c1430f0 2022-06-29T15:52:21.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.453+0000 7f62318f7700 1 --2- 172.21.15.47:0/1694676211 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6214053a70 0x7f6214055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:21.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.453+0000 7f62318f7700 1 --2- 172.21.15.47:0/1694676211 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6214053a70 0x7f6214055f30 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7f621c00d000 tx=0x7f621c007b80).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:21.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.461+0000 7f6222ffd700 1 -- 172.21.15.47:0/1694676211 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6228043ea0 con 0x7f622c1430f0 2022-06-29T15:52:21.634 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.625+0000 7f6233b5b700 1 -- 172.21.15.47:0/1694676211 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f622c0573d0 con 0x7f622c1430f0 2022-06-29T15:52:21.635 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.629+0000 7f6222ffd700 1 -- 172.21.15.47:0/1694676211 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+2495 (secure 0 0 0) 0x7f6228023090 con 0x7f622c1430f0 2022-06-29T15:52:21.635 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:52:21.636 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":79,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8596,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6836/1456033193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1456033193},{"type":"v1","addr":"172.21.15.47:6836","nonce":1456033193}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":77},{"gid":8745,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/1940204812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6835","nonce":1940204812},{"type":"v1","addr":"172.21.15.47:6837","nonce":1940204812}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":77}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:52:17.031576+0000","modified":"2022-06-29T15:52:18.034584+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":57377},"failed":[],"damaged":[],"stopped":[],"info":{"gid_57377":{"gid":57377,"name":"b","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6833/642573447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":642573447},{"type":"v1","addr":"172.21.15.64:6833","nonce":642573447}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2022-06-29T15:52:21.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.629+0000 7f6233b5b700 1 -- 172.21.15.47:0/1694676211 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6214053a70 msgr2=0x7f6214055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:21.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.629+0000 7f6233b5b700 1 --2- 172.21.15.47:0/1694676211 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6214053a70 0x7f6214055f30 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7f621c00d000 tx=0x7f621c007b80).stop 2022-06-29T15:52:21.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.629+0000 7f6233b5b700 1 -- 172.21.15.47:0/1694676211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f622c1430f0 msgr2=0x7f622c099d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:21.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.629+0000 7f6233b5b700 1 --2- 172.21.15.47:0/1694676211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f622c1430f0 0x7f622c099d10 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 rx=0x7f6228002f70 tx=0x7f6228015000).stop 2022-06-29T15:52:21.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.629+0000 7f6233b5b700 1 -- 172.21.15.47:0/1694676211 shutdown_connections 2022-06-29T15:52:21.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.629+0000 7f6233b5b700 1 --2- 172.21.15.47:0/1694676211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f622c1430f0 0x7f622c099d10 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:21.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.629+0000 7f6233b5b700 1 --2- 172.21.15.47:0/1694676211 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f622c0a93c0 0x7f622c135470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:21.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.629+0000 7f6233b5b700 1 --2- 172.21.15.47:0/1694676211 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6214053a70 0x7f6214055f30 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:21.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.629+0000 7f6233b5b700 1 --2- 172.21.15.47:0/1694676211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f622c095730 0x7f622c0b5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:21.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.629+0000 7f6233b5b700 1 -- 172.21.15.47:0/1694676211 >> 172.21.15.47:0/1694676211 conn(0x7f622c05f0b0 msgr2=0x7f622c0b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:21.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.629+0000 7f6233b5b700 1 -- 172.21.15.47:0/1694676211 shutdown_connections 2022-06-29T15:52:21.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:21.629+0000 7f6233b5b700 1 -- 172.21.15.47:0/1694676211 wait complete. 2022-06-29T15:52:21.641 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 79 2022-06-29T15:52:21.951 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 79, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-29T15:52:17.031576+0000', 'modified': '2022-06-29T15:52:18.034584+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 57377}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_57377': {'gid': 57377, 'name': 'b', 'rank': 0, 'incarnation': 78, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.64:6833/642573447', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6832', 'nonce': 642573447}, {'type': 'v1', 'addr': '172.21.15.64:6833', 'nonce': 642573447}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-29T15:52:21.951 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-29T15:52:21.952 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-06-29T15:52:22.066 INFO:teuthology.orchestra.run.smithi064.stdout:{"cluster_fsid":"9a68894a-53ce-400e-838b-14eab91b834c","whoami":0,"id":57377,"want_state":"up:active","state":"up:active","rank_uptime":5.0176739980000002,"mdsmap_epoch":79,"osdmap_epoch":162,"osdmap_epoch_barrier":161,"uptime":10.066596706} 2022-06-29T15:52:22.067 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9a68894a-53ce-400e-838b-14eab91b834c", "id": 57377, "mdsmap_epoch": 79, "osdmap_epoch": 162, "osdmap_epoch_barrier": 161, "rank_uptime": 5.017673998, "state": "up:active", "uptime": 10.066596706, "want_state": "up:active", "whoami": 0 } 2022-06-29T15:52:22.067 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-29T15:52:22.068 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-06-29T15:52:22.068 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi099.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-06-29T15:52:22.068 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:52:22.069 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:52:22.078 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2022-06-29T15:52:22.135 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:52:22.136 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:52:22.150 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:52:22.151 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:52:22.152 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:52:22.152 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2022-06-29T15:52:22.159 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-29T15:52:22.159 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:52:22.162 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2022-06-29T15:52:22.178 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:52:22.179 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:52:22.236 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:2022-06-29T15:52:22.229+0000 7fc2af6ed080 -1 init, newargv = 0x55d8e8f3d7e0 newargc=9 2022-06-29T15:52:22.237 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[13607]: starting ceph client 2022-06-29T15:52:22.240 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:52:22.241 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:52:22.242 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:52:22.243 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2022-06-29T15:52:22.247 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[13607]: starting fuse 2022-06-29T15:52:22.248 INFO:teuthology.orchestra.run.smithi099.stdout:52 2022-06-29T15:52:22.249 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-29T15:52:22.249 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:52:22.249 DEBUG:teuthology.orchestra.run.smithi099:> sudo python3 -c ' 2022-06-29T15:52:22.250 DEBUG:teuthology.orchestra.run.smithi099:> import glob 2022-06-29T15:52:22.250 DEBUG:teuthology.orchestra.run.smithi099:> import re 2022-06-29T15:52:22.251 DEBUG:teuthology.orchestra.run.smithi099:> import os 2022-06-29T15:52:22.251 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2022-06-29T15:52:22.252 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:52:22.252 DEBUG:teuthology.orchestra.run.smithi099:> def find_socket(client_name): 2022-06-29T15:52:22.252 DEBUG:teuthology.orchestra.run.smithi099:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-06-29T15:52:22.253 DEBUG:teuthology.orchestra.run.smithi099:> files = glob.glob(asok_path) 2022-06-29T15:52:22.253 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:52:22.254 DEBUG:teuthology.orchestra.run.smithi099:> # Given a non-glob path, it better be there 2022-06-29T15:52:22.254 DEBUG:teuthology.orchestra.run.smithi099:> if "*" not in asok_path: 2022-06-29T15:52:22.254 DEBUG:teuthology.orchestra.run.smithi099:> assert(len(files) == 1) 2022-06-29T15:52:22.255 DEBUG:teuthology.orchestra.run.smithi099:> return files[0] 2022-06-29T15:52:22.255 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:52:22.256 DEBUG:teuthology.orchestra.run.smithi099:> for f in files: 2022-06-29T15:52:22.256 DEBUG:teuthology.orchestra.run.smithi099:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-29T15:52:22.257 DEBUG:teuthology.orchestra.run.smithi099:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-29T15:52:22.257 DEBUG:teuthology.orchestra.run.smithi099:> return f 2022-06-29T15:52:22.257 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-29T15:52:22.258 DEBUG:teuthology.orchestra.run.smithi099:> 2022-06-29T15:52:22.258 DEBUG:teuthology.orchestra.run.smithi099:> print(find_socket("client.0")) 2022-06-29T15:52:22.259 DEBUG:teuthology.orchestra.run.smithi099:> ' 2022-06-29T15:52:22.333 INFO:teuthology.orchestra.run.smithi099.stdout:/var/run/ceph/ceph-client.0.13607.asok 2022-06-29T15:52:22.338 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.13607.asok 2022-06-29T15:52:22.339 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:52:22.339 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.13607.asok status 2022-06-29T15:52:22.426 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2022-06-29T15:52:22.427 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": { 2022-06-29T15:52:22.427 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_sha1": "f25ffac72e5a9e63860ff7deb6767a11f0559a16", 2022-06-29T15:52:22.427 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 15.2.16-304-gf25ffac7 (f25ffac72e5a9e63860ff7deb6767a11f0559a16) octopus (stable)", 2022-06-29T15:52:22.427 INFO:teuthology.orchestra.run.smithi099.stdout: "entity_id": "0", 2022-06-29T15:52:22.428 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": "smithi099", 2022-06-29T15:52:22.428 INFO:teuthology.orchestra.run.smithi099.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-06-29T15:52:22.428 INFO:teuthology.orchestra.run.smithi099.stdout: "pid": "13607", 2022-06-29T15:52:22.428 INFO:teuthology.orchestra.run.smithi099.stdout: "root": "/" 2022-06-29T15:52:22.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:52:22.429 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_count": 0, 2022-06-29T15:52:22.429 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_pinned_count": 0, 2022-06-29T15:52:22.429 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 8764, 2022-06-29T15:52:22.429 INFO:teuthology.orchestra.run.smithi099.stdout: "inst": { 2022-06-29T15:52:22.430 INFO:teuthology.orchestra.run.smithi099.stdout: "name": { 2022-06-29T15:52:22.430 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "client", 2022-06-29T15:52:22.430 INFO:teuthology.orchestra.run.smithi099.stdout: "num": 8764 2022-06-29T15:52:22.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:52:22.431 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2022-06-29T15:52:22.431 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2022-06-29T15:52:22.431 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:0", 2022-06-29T15:52:22.431 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3176295219 2022-06-29T15:52:22.432 INFO:teuthology.orchestra.run.smithi099.stdout: } 2022-06-29T15:52:22.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:52:22.432 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2022-06-29T15:52:22.432 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2022-06-29T15:52:22.432 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:0", 2022-06-29T15:52:22.433 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3176295219 2022-06-29T15:52:22.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-06-29T15:52:22.433 INFO:teuthology.orchestra.run.smithi099.stdout: "inst_str": "client.8764 172.21.15.99:0/3176295219", 2022-06-29T15:52:22.433 INFO:teuthology.orchestra.run.smithi099.stdout: "addr_str": "172.21.15.99:0/3176295219", 2022-06-29T15:52:22.433 INFO:teuthology.orchestra.run.smithi099.stdout: "inode_count": 1, 2022-06-29T15:52:22.434 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_epoch": 79, 2022-06-29T15:52:22.434 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch": 162, 2022-06-29T15:52:22.434 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch_barrier": 0, 2022-06-29T15:52:22.434 INFO:teuthology.orchestra.run.smithi099.stdout: "blacklisted": false, 2022-06-29T15:52:22.435 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs" 2022-06-29T15:52:22.435 INFO:teuthology.orchestra.run.smithi099.stdout:} 2022-06-29T15:52:22.436 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:52:22.436 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:52:22.437 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:52:22.445 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2022-06-29T15:52:22.446 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-06-29T15:52:22.446 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:52:22.447 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:52:22.509 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T15:52:22.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.617+0000 7f228e86c700 1 Processor -- start 2022-06-29T15:52:22.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.617+0000 7f228e86c700 1 -- start start 2022-06-29T15:52:22.626 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.617+0000 7f228e86c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f228805c510 0x7f2288143bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:22.627 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.617+0000 7f228e86c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f228805c8f0 0x7f22880b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:22.627 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.617+0000 7f228e86c700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2288142f90 con 0x7f22880b2c40 2022-06-29T15:52:22.627 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.617+0000 7f228e86c700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22881431f0 con 0x7f228805c8f0 2022-06-29T15:52:22.627 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.617+0000 7f228e86c700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2288143490 con 0x7f228805c510 2022-06-29T15:52:22.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.617+0000 7f228d069700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f228805c8f0 0x7f22880b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:22.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.617+0000 7f228d069700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f228805c8f0 0x7f22880b4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55248/0 (socket says 172.21.15.47:55248) 2022-06-29T15:52:22.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.617+0000 7f228d069700 1 -- 172.21.15.47:0/1567894933 learned_addr learned my addr 172.21.15.47:0/1567894933 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:22.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.617+0000 7f228d86a700 1 --2- 172.21.15.47:0/1567894933 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f228805c510 0x7f2288143bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:22.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.617+0000 7f228c868700 1 -- 172.21.15.47:0/1567894933 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2940499548 0 0) 0x7f2288142f90 con 0x7f22880b2c40 2022-06-29T15:52:22.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.617+0000 7f228c868700 1 -- 172.21.15.47:0/1567894933 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2270003260 con 0x7f22880b2c40 2022-06-29T15:52:22.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.617+0000 7f228d069700 1 -- 172.21.15.47:0/1567894933 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f228805c510 msgr2=0x7f2288143bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:22.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.617+0000 7f228d069700 1 --2- 172.21.15.47:0/1567894933 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f228805c510 0x7f2288143bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:22.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.617+0000 7f228d069700 1 -- 172.21.15.47:0/1567894933 >> v1:172.21.15.64:6789/0 conn(0x7f22880b2c40 legacy=0x7f228813cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:22.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.617+0000 7f228d069700 1 -- 172.21.15.47:0/1567894933 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2288143730 con 0x7f228805c8f0 2022-06-29T15:52:22.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.617+0000 7f228d86a700 1 --2- 172.21.15.47:0/1567894933 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f228805c510 0x7f2288143bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:52:22.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.617+0000 7f228d069700 1 --2- 172.21.15.47:0/1567894933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f228805c8f0 0x7f22880b4100 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 rx=0x7f227c002f70 tx=0x7f227c006eb0).ready entity=mon.0 client_cookie=2a8b2fb285a4d009 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:22.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.617+0000 7f228c868700 1 -- 172.21.15.47:0/1567894933 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f227c00d5c0 con 0x7f228805c8f0 2022-06-29T15:52:22.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.617+0000 7f228c868700 1 -- 172.21.15.47:0/1567894933 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f227c000e90 con 0x7f228805c8f0 2022-06-29T15:52:22.631 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.617+0000 7f228c868700 1 -- 172.21.15.47:0/1567894933 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f227c00d950 con 0x7f228805c8f0 2022-06-29T15:52:22.635 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.621+0000 7f228e86c700 1 -- 172.21.15.47:0/1567894933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f228805c8f0 msgr2=0x7f22880b4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:22.635 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.621+0000 7f228e86c700 1 --2- 172.21.15.47:0/1567894933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f228805c8f0 0x7f22880b4100 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 rx=0x7f227c002f70 tx=0x7f227c006eb0).stop 2022-06-29T15:52:22.635 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.621+0000 7f228e86c700 1 -- 172.21.15.47:0/1567894933 shutdown_connections 2022-06-29T15:52:22.635 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.621+0000 7f228e86c700 1 --2- 172.21.15.47:0/1567894933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f228805c8f0 0x7f22880b4100 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:22.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.621+0000 7f228e86c700 1 --2- 172.21.15.47:0/1567894933 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f228805c510 0x7f2288143bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:22.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.621+0000 7f228e86c700 1 -- 172.21.15.47:0/1567894933 >> 172.21.15.47:0/1567894933 conn(0x7f2288060e00 msgr2=0x7f22880611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:22.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.621+0000 7f228e86c700 1 -- 172.21.15.47:0/1567894933 shutdown_connections 2022-06-29T15:52:22.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.621+0000 7f228e86c700 1 -- 172.21.15.47:0/1567894933 wait complete. 2022-06-29T15:52:22.636 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.621+0000 7f228e86c700 1 Processor -- start 2022-06-29T15:52:22.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.621+0000 7f228e86c700 1 -- start start 2022-06-29T15:52:22.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.621+0000 7f228e86c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f228805c510 0x7f22881446d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:22.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.621+0000 7f228e86c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f228805c8f0 0x7f2288138c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:22.637 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.621+0000 7f228e86c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f22880b2c40 0x7f2288155da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:22.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.621+0000 7f228e86c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22881556a0 con 0x7f228805c510 2022-06-29T15:52:22.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.621+0000 7f228e86c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2288152460 con 0x7f228805c8f0 2022-06-29T15:52:22.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.621+0000 7f228e86c700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22880b5090 con 0x7f22880b2c40 2022-06-29T15:52:22.638 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.621+0000 7f228d86a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f228805c510 0x7f22881446d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:22.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.621+0000 7f228d86a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f228805c510 0x7f22881446d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55252/0 (socket says 172.21.15.47:55252) 2022-06-29T15:52:22.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.621+0000 7f228d86a700 1 -- 172.21.15.47:0/1456953070 learned_addr learned my addr 172.21.15.47:0/1456953070 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:22.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.621+0000 7f228e06b700 1 --2- 172.21.15.47:0/1456953070 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f22880b2c40 0x7f2288155da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:22.639 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.621+0000 7f228d069700 1 --2- 172.21.15.47:0/1456953070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f228805c8f0 0x7f2288138c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:22.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.621+0000 7f228d86a700 1 -- 172.21.15.47:0/1456953070 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f22880b2c40 msgr2=0x7f2288155da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:22.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.621+0000 7f228d86a700 1 --2- 172.21.15.47:0/1456953070 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f22880b2c40 0x7f2288155da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:22.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.621+0000 7f228d86a700 1 -- 172.21.15.47:0/1456953070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f228805c8f0 msgr2=0x7f2288138c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:22.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.621+0000 7f228d86a700 1 --2- 172.21.15.47:0/1456953070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f228805c8f0 0x7f2288138c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:22.640 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.621+0000 7f228d86a700 1 -- 172.21.15.47:0/1456953070 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f228400b180 con 0x7f228805c510 2022-06-29T15:52:22.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.621+0000 7f228e06b700 1 --2- 172.21.15.47:0/1456953070 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f22880b2c40 0x7f2288155da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:52:22.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.621+0000 7f228d069700 1 --2- 172.21.15.47:0/1456953070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f228805c8f0 0x7f2288138c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:52:22.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.625+0000 7f228d86a700 1 --2- 172.21.15.47:0/1456953070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f228805c510 0x7f22881446d0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 rx=0x7f228400c000 tx=0x7f2284006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:22.641 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.625+0000 7f227affd700 1 -- 172.21.15.47:0/1456953070 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f228400ca40 con 0x7f228805c510 2022-06-29T15:52:22.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.625+0000 7f227affd700 1 -- 172.21.15.47:0/1456953070 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f228400cba0 con 0x7f228805c510 2022-06-29T15:52:22.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.625+0000 7f228e86c700 1 -- 172.21.15.47:0/1456953070 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f227c00b1e0 con 0x7f228805c510 2022-06-29T15:52:22.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.625+0000 7f227affd700 1 -- 172.21.15.47:0/1456953070 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f22840123f0 con 0x7f228805c510 2022-06-29T15:52:22.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.625+0000 7f228e86c700 1 -- 172.21.15.47:0/1456953070 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2288155900 con 0x7f228805c510 2022-06-29T15:52:22.642 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.625+0000 7f228e86c700 1 -- 172.21.15.47:0/1456953070 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2288155ac0 con 0x7f228805c510 2022-06-29T15:52:22.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.625+0000 7f227affd700 1 -- 172.21.15.47:0/1456953070 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2284012550 con 0x7f228805c510 2022-06-29T15:52:22.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.625+0000 7f227affd700 1 --2- 172.21.15.47:0/1456953070 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2268053a10 0x7f2268055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:22.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.625+0000 7f227affd700 1 -- 172.21.15.47:0/1456953070 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f228406d210 con 0x7f228805c510 2022-06-29T15:52:22.643 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.625+0000 7f228d069700 1 --2- 172.21.15.47:0/1456953070 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2268053a10 0x7f2268055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:22.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.629+0000 7f228d069700 1 --2- 172.21.15.47:0/1456953070 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2268053a10 0x7f2268055ed0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0x7f227c017000 tx=0x7f227c015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:22.644 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.633+0000 7f227affd700 1 -- 172.21.15.47:0/1456953070 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2284047e60 con 0x7f228805c510 2022-06-29T15:52:22.765 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.757+0000 7f228e86c700 1 -- 172.21.15.47:0/1456953070 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f22880573d0 con 0x7f228805c510 2022-06-29T15:52:22.766 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.757+0000 7f227affd700 1 -- 172.21.15.47:0/1456953070 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v162) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f2284043f70 con 0x7f228805c510 2022-06-29T15:52:22.767 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:52:22.767 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":162,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:52:18.035479+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2022-06-29T15:52:14.780743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2022-06-29T15:52:16.023272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T15:52:22.769 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.761+0000 7f228e86c700 1 -- 172.21.15.47:0/1456953070 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2268053a10 msgr2=0x7f2268055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:22.769 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.761+0000 7f228e86c700 1 --2- 172.21.15.47:0/1456953070 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2268053a10 0x7f2268055ed0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0x7f227c017000 tx=0x7f227c015000).stop 2022-06-29T15:52:22.769 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.761+0000 7f228e86c700 1 -- 172.21.15.47:0/1456953070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f228805c510 msgr2=0x7f22881446d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:22.770 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.761+0000 7f228e86c700 1 --2- 172.21.15.47:0/1456953070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f228805c510 0x7f22881446d0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 rx=0x7f228400c000 tx=0x7f2284006df0).stop 2022-06-29T15:52:22.770 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.761+0000 7f228e86c700 1 -- 172.21.15.47:0/1456953070 shutdown_connections 2022-06-29T15:52:22.770 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.761+0000 7f228e86c700 1 --2- 172.21.15.47:0/1456953070 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f22880b2c40 0x7f2288155da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:22.770 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.761+0000 7f228e86c700 1 --2- 172.21.15.47:0/1456953070 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2268053a10 0x7f2268055ed0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:22.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.761+0000 7f228e86c700 1 --2- 172.21.15.47:0/1456953070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f228805c8f0 0x7f2288138c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:22.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.761+0000 7f228e86c700 1 --2- 172.21.15.47:0/1456953070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f228805c510 0x7f22881446d0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:22.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.761+0000 7f228e86c700 1 -- 172.21.15.47:0/1456953070 >> 172.21.15.47:0/1456953070 conn(0x7f2288060e00 msgr2=0x7f228813c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:22.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.761+0000 7f228e86c700 1 -- 172.21.15.47:0/1456953070 shutdown_connections 2022-06-29T15:52:22.772 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.761+0000 7f228e86c700 1 -- 172.21.15.47:0/1456953070 wait complete. 2022-06-29T15:52:22.781 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-06-29T15:42:16.971431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-06-29T15:42:25.290884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 29, 'pool_name': 'cephfs_metadata', 'create_time': '2022-06-29T15:52:14.780743+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '161', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 30, 'pool_name': 'cephfs_data', 'create_time': '2022-06-29T15:52:16.023272+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '162', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-06-29T15:52:22.781 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-06-29T15:52:22.893 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.885+0000 7f29b46b9700 1 Processor -- start 2022-06-29T15:52:22.894 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.885+0000 7f29b46b9700 1 -- start start 2022-06-29T15:52:22.896 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.885+0000 7f29b46b9700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f29ac060c10 0x7f29ac0b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:22.896 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.885+0000 7f29b46b9700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f29ac0aa0c0 0x7f29ac0b1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:22.897 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.885+0000 7f29b46b9700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29ac060ff0 con 0x7f29ac05f0b0 2022-06-29T15:52:22.897 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.885+0000 7f29b46b9700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29ac05ffe0 con 0x7f29ac0aa0c0 2022-06-29T15:52:22.897 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.885+0000 7f29b46b9700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29ac05faa0 con 0x7f29ac060c10 2022-06-29T15:52:22.898 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.889+0000 7f29b1c54700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f29ac0aa0c0 0x7f29ac0b1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:22.898 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.889+0000 7f29b1c54700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f29ac0aa0c0 0x7f29ac0b1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55260/0 (socket says 172.21.15.47:55260) 2022-06-29T15:52:22.898 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.889+0000 7f29b1c54700 1 -- 172.21.15.47:0/68570186 learned_addr learned my addr 172.21.15.47:0/68570186 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:22.898 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.889+0000 7f29b2455700 1 --2- 172.21.15.47:0/68570186 >> v2:172.21.15.64:3300/0 conn(0x7f29ac060c10 0x7f29ac0b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:22.898 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.889+0000 7f29b1c54700 1 -- 172.21.15.47:0/68570186 >> v2:172.21.15.64:3300/0 conn(0x7f29ac060c10 msgr2=0x7f29ac0b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:22.899 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.889+0000 7f29b1c54700 1 --2- 172.21.15.47:0/68570186 >> v2:172.21.15.64:3300/0 conn(0x7f29ac060c10 0x7f29ac0b1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:22.899 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.889+0000 7f29b1c54700 1 -- 172.21.15.47:0/68570186 >> v1:172.21.15.64:6789/0 conn(0x7f29ac05f0b0 legacy=0x7f29ac146970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:22.899 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.889+0000 7f29b1453700 1 -- 172.21.15.47:0/68570186 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1825082148 0 0) 0x7f29ac060ff0 con 0x7f29ac05f0b0 2022-06-29T15:52:22.899 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.889+0000 7f29b1c54700 1 -- 172.21.15.47:0/68570186 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29ac05fd40 con 0x7f29ac0aa0c0 2022-06-29T15:52:22.900 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.889+0000 7f29b1c54700 1 --2- 172.21.15.47:0/68570186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29ac0aa0c0 0x7f29ac0b1d10 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 rx=0x7f29ac0adee0 tx=0x7f29a0006eb0).ready entity=mon.0 client_cookie=4f550f89f58b73dd server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:22.900 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.889+0000 7f29b1453700 1 -- 172.21.15.47:0/68570186 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29a000d610 con 0x7f29ac0aa0c0 2022-06-29T15:52:22.900 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.889+0000 7f29b1453700 1 -- 172.21.15.47:0/68570186 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29a0000e90 con 0x7f29ac0aa0c0 2022-06-29T15:52:22.900 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.889+0000 7f29b1453700 1 -- 172.21.15.47:0/68570186 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29a000d960 con 0x7f29ac0aa0c0 2022-06-29T15:52:22.904 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.889+0000 7f29b46b9700 1 -- 172.21.15.47:0/68570186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29ac0aa0c0 msgr2=0x7f29ac0b1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:22.904 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.889+0000 7f29b46b9700 1 --2- 172.21.15.47:0/68570186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29ac0aa0c0 0x7f29ac0b1d10 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 rx=0x7f29ac0adee0 tx=0x7f29a0006eb0).stop 2022-06-29T15:52:22.905 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.889+0000 7f29b46b9700 1 -- 172.21.15.47:0/68570186 shutdown_connections 2022-06-29T15:52:22.905 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.889+0000 7f29b46b9700 1 --2- 172.21.15.47:0/68570186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29ac0aa0c0 0x7f29ac0b1d10 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:22.905 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.889+0000 7f29b46b9700 1 --2- 172.21.15.47:0/68570186 >> v2:172.21.15.64:3300/0 conn(0x7f29ac060c10 0x7f29ac0b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:22.905 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.889+0000 7f29b46b9700 1 -- 172.21.15.47:0/68570186 >> 172.21.15.47:0/68570186 conn(0x7f29ac062110 msgr2=0x7f29ac0601c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:22.906 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.889+0000 7f29b46b9700 1 -- 172.21.15.47:0/68570186 shutdown_connections 2022-06-29T15:52:22.906 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.889+0000 7f29b46b9700 1 -- 172.21.15.47:0/68570186 wait complete. 2022-06-29T15:52:22.906 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.889+0000 7f29b46b9700 1 Processor -- start 2022-06-29T15:52:22.906 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.889+0000 7f29b46b9700 1 -- start start 2022-06-29T15:52:22.907 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.893+0000 7f29b46b9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29ac05f0b0 0x7f29ac0abde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:22.907 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.893+0000 7f29b46b9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29ac060c10 0x7f29ac0aa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:22.907 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.893+0000 7f29b46b9700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f29ac0aa0c0 0x7f29ac152000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:22.907 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.893+0000 7f29b46b9700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ac0ab370 con 0x7f29ac060c10 2022-06-29T15:52:22.908 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.893+0000 7f29b46b9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ac0ac780 con 0x7f29ac05f0b0 2022-06-29T15:52:22.908 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.893+0000 7f29b46b9700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29ac0ab750 con 0x7f29ac0aa0c0 2022-06-29T15:52:22.908 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.893+0000 7f29b1c54700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29ac060c10 0x7f29ac0aa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:22.908 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.893+0000 7f29b1c54700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29ac060c10 0x7f29ac0aa6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55268/0 (socket says 172.21.15.47:55268) 2022-06-29T15:52:22.909 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.893+0000 7f29b1c54700 1 -- 172.21.15.47:0/4239761318 learned_addr learned my addr 172.21.15.47:0/4239761318 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:22.909 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.893+0000 7f29b2455700 1 --2- 172.21.15.47:0/4239761318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29ac05f0b0 0x7f29ac0abde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:22.909 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.893+0000 7f29b2c56700 1 --2- 172.21.15.47:0/4239761318 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f29ac0aa0c0 0x7f29ac152000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:22.909 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.893+0000 7f29b1c54700 1 -- 172.21.15.47:0/4239761318 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f29ac0aa0c0 msgr2=0x7f29ac152000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:22.909 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.893+0000 7f29b1c54700 1 --2- 172.21.15.47:0/4239761318 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f29ac0aa0c0 0x7f29ac152000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:22.910 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.893+0000 7f29b1c54700 1 -- 172.21.15.47:0/4239761318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29ac05f0b0 msgr2=0x7f29ac0abde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:22.910 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.893+0000 7f29b1c54700 1 --2- 172.21.15.47:0/4239761318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29ac05f0b0 0x7f29ac0abde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:22.910 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.893+0000 7f29b1c54700 1 -- 172.21.15.47:0/4239761318 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a000b1e0 con 0x7f29ac060c10 2022-06-29T15:52:22.910 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.893+0000 7f29b1c54700 1 --2- 172.21.15.47:0/4239761318 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29ac060c10 0x7f29ac0aa6b0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 rx=0x7f29ac0ad0e0 tx=0x7f29a0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:22.911 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.893+0000 7f299f7fe700 1 -- 172.21.15.47:0/4239761318 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29a0002f50 con 0x7f29ac060c10 2022-06-29T15:52:22.911 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.893+0000 7f299f7fe700 1 -- 172.21.15.47:0/4239761318 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29a000d990 con 0x7f29ac060c10 2022-06-29T15:52:22.911 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.893+0000 7f29b46b9700 1 -- 172.21.15.47:0/4239761318 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29ac0ab940 con 0x7f29ac060c10 2022-06-29T15:52:22.911 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.893+0000 7f299f7fe700 1 -- 172.21.15.47:0/4239761318 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29a0011e50 con 0x7f29ac060c10 2022-06-29T15:52:22.912 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.893+0000 7f29b46b9700 1 -- 172.21.15.47:0/4239761318 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29ac0aaba0 con 0x7f29ac060c10 2022-06-29T15:52:22.912 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.893+0000 7f29b46b9700 1 -- 172.21.15.47:0/4239761318 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29ac0573d0 con 0x7f29ac060c10 2022-06-29T15:52:22.912 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.893+0000 7f299f7fe700 1 -- 172.21.15.47:0/4239761318 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f29a0025030 con 0x7f29ac060c10 2022-06-29T15:52:22.912 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.893+0000 7f299f7fe700 1 --2- 172.21.15.47:0/4239761318 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f298c0539e0 0x7f298c055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:22.912 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.893+0000 7f299f7fe700 1 -- 172.21.15.47:0/4239761318 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f29a006d2b0 con 0x7f29ac060c10 2022-06-29T15:52:22.913 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.893+0000 7f29b2455700 1 --2- 172.21.15.47:0/4239761318 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f298c0539e0 0x7f298c055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:22.913 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.897+0000 7f29b2455700 1 --2- 172.21.15.47:0/4239761318 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f298c0539e0 0x7f298c055ea0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 rx=0x7f29a800b000 tx=0x7f29a8006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:22.914 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:22.901+0000 7f299f7fe700 1 -- 172.21.15.47:0/4239761318 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f29a0044070 con 0x7f29ac060c10 2022-06-29T15:52:23.029 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.021+0000 7f29b46b9700 1 -- 172.21.15.47:0/4239761318 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f29ac06a210 con 0x7f29ac060c10 2022-06-29T15:52:23.030 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.021+0000 7f299f7fe700 1 -- 172.21.15.47:0/4239761318 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v162) v1 ==== 110+0+10 (secure 0 0 0) 0x7f29a0023020 con 0x7f29ac060c10 2022-06-29T15:52:23.030 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2022-06-29T15:52:23.032 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.025+0000 7f29b46b9700 1 -- 172.21.15.47:0/4239761318 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f298c0539e0 msgr2=0x7f298c055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:23.032 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.025+0000 7f29b46b9700 1 --2- 172.21.15.47:0/4239761318 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f298c0539e0 0x7f298c055ea0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 rx=0x7f29a800b000 tx=0x7f29a8006df0).stop 2022-06-29T15:52:23.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.025+0000 7f29b46b9700 1 -- 172.21.15.47:0/4239761318 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29ac060c10 msgr2=0x7f29ac0aa6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:23.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.025+0000 7f29b46b9700 1 --2- 172.21.15.47:0/4239761318 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29ac060c10 0x7f29ac0aa6b0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 rx=0x7f29ac0ad0e0 tx=0x7f29a0015000).stop 2022-06-29T15:52:23.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.025+0000 7f29b46b9700 1 -- 172.21.15.47:0/4239761318 shutdown_connections 2022-06-29T15:52:23.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.025+0000 7f29b46b9700 1 --2- 172.21.15.47:0/4239761318 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f29ac0aa0c0 0x7f29ac152000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:23.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.025+0000 7f29b46b9700 1 --2- 172.21.15.47:0/4239761318 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29ac060c10 0x7f29ac0aa6b0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:23.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.025+0000 7f29b46b9700 1 --2- 172.21.15.47:0/4239761318 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f298c0539e0 0x7f298c055ea0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:23.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.025+0000 7f29b46b9700 1 --2- 172.21.15.47:0/4239761318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29ac05f0b0 0x7f29ac0abde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:23.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.025+0000 7f29b46b9700 1 -- 172.21.15.47:0/4239761318 >> 172.21.15.47:0/4239761318 conn(0x7f29ac062110 msgr2=0x7f29ac141cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:23.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.025+0000 7f29b46b9700 1 -- 172.21.15.47:0/4239761318 shutdown_connections 2022-06-29T15:52:23.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.025+0000 7f29b46b9700 1 -- 172.21.15.47:0/4239761318 wait complete. 2022-06-29T15:52:23.043 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-06-29T15:52:23.158 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.149+0000 7fdedffff700 1 Processor -- start 2022-06-29T15:52:23.158 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.149+0000 7fdedffff700 1 -- start start 2022-06-29T15:52:23.161 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.153+0000 7fdedffff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdee0062f30 0x7fdee01400c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:23.161 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.153+0000 7fdedffff700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdee0143f70 con 0x7fdee00b6d30 2022-06-29T15:52:23.161 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.153+0000 7fdedffff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdee00a9cf0 con 0x7fdee013d9b0 2022-06-29T15:52:23.161 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.153+0000 7fdedffff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee00b1380 con 0x7fdee0062f30 2022-06-29T15:52:23.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.153+0000 7fdedeffd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fdee00b6d30 0x7fdee013d2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40828/0 (socket says 172.21.15.47:40828) 2022-06-29T15:52:23.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.153+0000 7fdedeffd700 1 -- 172.21.15.47:0/559798302 learned_addr learned my addr 172.21.15.47:0/559798302 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:52:23.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.153+0000 7fded67fc700 1 --2- 172.21.15.47:0/559798302 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdee0062f30 0x7fdee01400c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:23.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.153+0000 7fdede7fc700 1 -- 172.21.15.47:0/559798302 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1045616484 0 0) 0x7fdee0143f70 con 0x7fdee00b6d30 2022-06-29T15:52:23.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.153+0000 7fdede7fc700 1 -- 172.21.15.47:0/559798302 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdec0003260 con 0x7fdee00b6d30 2022-06-29T15:52:23.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.153+0000 7fdede7fc700 1 -- 172.21.15.47:0/559798302 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3300391164 0 0) 0x7fdee00a9cf0 con 0x7fdee013d9b0 2022-06-29T15:52:23.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.153+0000 7fdede7fc700 1 -- 172.21.15.47:0/559798302 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdee0143f70 con 0x7fdee013d9b0 2022-06-29T15:52:23.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.153+0000 7fdede7fc700 1 -- 172.21.15.47:0/559798302 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3908327269 0 0) 0x7fdec0003260 con 0x7fdee00b6d30 2022-06-29T15:52:23.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.153+0000 7fded67fc700 1 -- 172.21.15.47:0/559798302 >> v1:172.21.15.64:6789/0 conn(0x7fdee013d9b0 legacy=0x7fdee01405b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:23.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.153+0000 7fded67fc700 1 -- 172.21.15.47:0/559798302 >> v1:172.21.15.47:6789/0 conn(0x7fdee00b6d30 legacy=0x7fdee013d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:23.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.153+0000 7fded67fc700 1 -- 172.21.15.47:0/559798302 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee01441a0 con 0x7fdee0062f30 2022-06-29T15:52:23.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.153+0000 7fded67fc700 1 --2- 172.21.15.47:0/559798302 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdee0062f30 0x7fdee01400c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7fded0001110 tx=0x7fdec8006df0).ready entity=mon.2 client_cookie=b550737a145b9670 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:23.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.153+0000 7fdede7fc700 1 -- 172.21.15.47:0/559798302 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fded0001660 con 0x7fdee0062f30 2022-06-29T15:52:23.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.153+0000 7fdede7fc700 1 -- 172.21.15.47:0/559798302 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec8000e90 con 0x7fdee0062f30 2022-06-29T15:52:23.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.153+0000 7fdede7fc700 1 -- 172.21.15.47:0/559798302 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdec8012460 con 0x7fdee0062f30 2022-06-29T15:52:23.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.153+0000 7fdedffff700 1 -- 172.21.15.47:0/559798302 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdee0062f30 msgr2=0x7fdee01400c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:23.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.153+0000 7fdedffff700 1 --2- 172.21.15.47:0/559798302 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdee0062f30 0x7fdee01400c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7fded0001110 tx=0x7fdec8006df0).stop 2022-06-29T15:52:23.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.157+0000 7fdedffff700 1 -- 172.21.15.47:0/559798302 shutdown_connections 2022-06-29T15:52:23.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.157+0000 7fdedffff700 1 --2- 172.21.15.47:0/559798302 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdee0062f30 0x7fdee01400c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:23.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.157+0000 7fdedffff700 1 -- 172.21.15.47:0/559798302 >> 172.21.15.47:0/559798302 conn(0x7fdee0063670 msgr2=0x7fdee0063a50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:23.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.157+0000 7fdedffff700 1 -- 172.21.15.47:0/559798302 shutdown_connections 2022-06-29T15:52:23.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.157+0000 7fdedffff700 1 -- 172.21.15.47:0/559798302 wait complete. 2022-06-29T15:52:23.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.157+0000 7fdedffff700 1 Processor -- start 2022-06-29T15:52:23.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.157+0000 7fdedffff700 1 -- start start 2022-06-29T15:52:23.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.157+0000 7fdedffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdee0062f30 0x7fdee00acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:23.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.157+0000 7fdedffff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdee00b6d30 0x7fdee00aec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:23.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.157+0000 7fdedffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdee013d9b0 0x7fdee005f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:23.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.157+0000 7fdedffff700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee00ae120 con 0x7fdee0062f30 2022-06-29T15:52:23.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.157+0000 7fdedffff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee00ab6a0 con 0x7fdee013d9b0 2022-06-29T15:52:23.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.157+0000 7fdedffff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee00ad2e0 con 0x7fdee00b6d30 2022-06-29T15:52:23.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.157+0000 7fdedeffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdee0062f30 0x7fdee00acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:23.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.157+0000 7fdedeffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdee0062f30 0x7fdee00acdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55280/0 (socket says 172.21.15.47:55280) 2022-06-29T15:52:23.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.157+0000 7fdedeffd700 1 -- 172.21.15.47:0/1221737608 learned_addr learned my addr 172.21.15.47:0/1221737608 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:23.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.161+0000 7fded67fc700 1 --2- 172.21.15.47:0/1221737608 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdee00b6d30 0x7fdee00aec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:23.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.161+0000 7fdedf7fe700 1 --2- 172.21.15.47:0/1221737608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdee013d9b0 0x7fdee005f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:23.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.161+0000 7fdedeffd700 1 -- 172.21.15.47:0/1221737608 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdee00b6d30 msgr2=0x7fdee00aec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:23.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.161+0000 7fdedeffd700 1 --2- 172.21.15.47:0/1221737608 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdee00b6d30 0x7fdee00aec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:23.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.161+0000 7fdedeffd700 1 -- 172.21.15.47:0/1221737608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdee013d9b0 msgr2=0x7fdee005f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:23.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.161+0000 7fdedeffd700 1 --2- 172.21.15.47:0/1221737608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdee013d9b0 0x7fdee005f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:23.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.161+0000 7fdedeffd700 1 -- 172.21.15.47:0/1221737608 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec800b180 con 0x7fdee0062f30 2022-06-29T15:52:23.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.161+0000 7fded67fc700 1 --2- 172.21.15.47:0/1221737608 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdee00b6d30 0x7fdee00aec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:52:23.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.161+0000 7fdedf7fe700 1 --2- 172.21.15.47:0/1221737608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdee013d9b0 0x7fdee005f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:52:23.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.161+0000 7fdedeffd700 1 --2- 172.21.15.47:0/1221737608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdee0062f30 0x7fdee00acdf0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 rx=0x7fded0000fe0 tx=0x7fded0001970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:23.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.161+0000 7fdedcff9700 1 -- 172.21.15.47:0/1221737608 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fded0007170 con 0x7fdee0062f30 2022-06-29T15:52:23.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.161+0000 7fdedcff9700 1 -- 172.21.15.47:0/1221737608 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded00072d0 con 0x7fdee0062f30 2022-06-29T15:52:23.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.161+0000 7fdedffff700 1 -- 172.21.15.47:0/1221737608 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdee0153720 con 0x7fdee0062f30 2022-06-29T15:52:23.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.161+0000 7fdedcff9700 1 -- 172.21.15.47:0/1221737608 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fded0004920 con 0x7fdee0062f30 2022-06-29T15:52:23.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.161+0000 7fdedffff700 1 -- 172.21.15.47:0/1221737608 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdee00aca70 con 0x7fdee0062f30 2022-06-29T15:52:23.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.161+0000 7fdedcff9700 1 -- 172.21.15.47:0/1221737608 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fded001b070 con 0x7fdee0062f30 2022-06-29T15:52:23.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.161+0000 7fdedffff700 1 -- 172.21.15.47:0/1221737608 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdee00af4b0 con 0x7fdee0062f30 2022-06-29T15:52:23.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.161+0000 7fdedcff9700 1 --2- 172.21.15.47:0/1221737608 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdec4053980 0x7fdec4055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:23.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.161+0000 7fded67fc700 1 --2- 172.21.15.47:0/1221737608 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdec4053980 0x7fdec4055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:23.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.161+0000 7fdedcff9700 1 -- 172.21.15.47:0/1221737608 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fded006b8f0 con 0x7fdee0062f30 2022-06-29T15:52:23.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.161+0000 7fded67fc700 1 --2- 172.21.15.47:0/1221737608 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdec4053980 0x7fdec4055e40 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 rx=0x7fdec8015000 tx=0x7fdec8014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:23.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.165+0000 7fdedcff9700 1 -- 172.21.15.47:0/1221737608 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fded00425a0 con 0x7fdee0062f30 2022-06-29T15:52:23.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.293+0000 7fdedffff700 1 -- 172.21.15.47:0/1221737608 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fdee00abb00 con 0x7fdee0062f30 2022-06-29T15:52:23.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.293+0000 7fdedcff9700 1 -- 172.21.15.47:0/1221737608 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v162) v1 ==== 92+0+10 (secure 0 0 0) 0x7fded0045bc0 con 0x7fdee0062f30 2022-06-29T15:52:23.302 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:52:23.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.297+0000 7fdedffff700 1 -- 172.21.15.47:0/1221737608 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdec4053980 msgr2=0x7fdec4055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:23.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.297+0000 7fdedffff700 1 --2- 172.21.15.47:0/1221737608 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdec4053980 0x7fdec4055e40 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 rx=0x7fdec8015000 tx=0x7fdec8014000).stop 2022-06-29T15:52:23.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.297+0000 7fdedffff700 1 -- 172.21.15.47:0/1221737608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdee0062f30 msgr2=0x7fdee00acdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:23.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.297+0000 7fdedffff700 1 --2- 172.21.15.47:0/1221737608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdee0062f30 0x7fdee00acdf0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 rx=0x7fded0000fe0 tx=0x7fded0001970).stop 2022-06-29T15:52:23.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.297+0000 7fdedffff700 1 -- 172.21.15.47:0/1221737608 shutdown_connections 2022-06-29T15:52:23.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.297+0000 7fdedffff700 1 --2- 172.21.15.47:0/1221737608 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdec4053980 0x7fdec4055e40 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:23.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.297+0000 7fdedffff700 1 --2- 172.21.15.47:0/1221737608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdee013d9b0 0x7fdee005f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:23.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.297+0000 7fdedffff700 1 --2- 172.21.15.47:0/1221737608 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdee00b6d30 0x7fdee00aec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:23.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.297+0000 7fdedffff700 1 --2- 172.21.15.47:0/1221737608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdee0062f30 0x7fdee00acdf0 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:23.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.297+0000 7fdedffff700 1 -- 172.21.15.47:0/1221737608 >> 172.21.15.47:0/1221737608 conn(0x7fdee0063670 msgr2=0x7fdee013f300 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:23.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.297+0000 7fdedffff700 1 -- 172.21.15.47:0/1221737608 shutdown_connections 2022-06-29T15:52:23.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.297+0000 7fdedffff700 1 -- 172.21.15.47:0/1221737608 wait complete. 2022-06-29T15:52:23.320 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-06-29T15:52:23.435 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.429+0000 7fd3412dd700 1 Processor -- start 2022-06-29T15:52:23.436 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.429+0000 7fd3412dd700 1 -- start start 2022-06-29T15:52:23.437 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.429+0000 7fd3412dd700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd33c0b4300 0x7fd33c0612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:23.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.429+0000 7fd3412dd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd33c1471a0 0x7fd33c066f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:23.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.429+0000 7fd3412dd700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd33c067610 con 0x7fd33c060ee0 2022-06-29T15:52:23.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.429+0000 7fd3412dd700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd33c1475d0 con 0x7fd33c1471a0 2022-06-29T15:52:23.438 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.429+0000 7fd3412dd700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd33c1467c0 con 0x7fd33c0b4300 2022-06-29T15:52:23.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.429+0000 7fd33a7fc700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd33c0b4300 0x7fd33c0612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:23.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.429+0000 7fd33b7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd33c1471a0 0x7fd33c066f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:23.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.429+0000 7fd33a7fc700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd33c0b4300 0x7fd33c0612c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:52392/0 (socket says 172.21.15.47:52392) 2022-06-29T15:52:23.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.429+0000 7fd33a7fc700 1 -- 172.21.15.47:0/3192789728 learned_addr learned my addr 172.21.15.47:0/3192789728 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:23.439 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.429+0000 7fd33a7fc700 1 -- 172.21.15.47:0/3192789728 >> v2:172.21.15.64:3300/0 conn(0x7fd33c1471a0 msgr2=0x7fd33c066f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:23.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.429+0000 7fd33a7fc700 1 --2- 172.21.15.47:0/3192789728 >> v2:172.21.15.64:3300/0 conn(0x7fd33c1471a0 0x7fd33c066f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:23.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.429+0000 7fd33a7fc700 1 -- 172.21.15.47:0/3192789728 >> v1:172.21.15.64:6789/0 conn(0x7fd33c060ee0 legacy=0x7fd33c1430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:23.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.429+0000 7fd33a7fc700 1 -- 172.21.15.47:0/3192789728 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd33c146a60 con 0x7fd33c0b4300 2022-06-29T15:52:23.440 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.429+0000 7fd33a7fc700 1 --2- 172.21.15.47:0/3192789728 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd33c0b4300 0x7fd33c0612c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7fd330002f70 tx=0x7fd330006eb0).ready entity=mon.2 client_cookie=bae9804584257181 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:23.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.429+0000 7fd339ffb700 1 -- 172.21.15.47:0/3192789728 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd33000d570 con 0x7fd33c0b4300 2022-06-29T15:52:23.444 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.429+0000 7fd339ffb700 1 -- 172.21.15.47:0/3192789728 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd330000e90 con 0x7fd33c0b4300 2022-06-29T15:52:23.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.429+0000 7fd339ffb700 1 -- 172.21.15.47:0/3192789728 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd33000d900 con 0x7fd33c0b4300 2022-06-29T15:52:23.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.429+0000 7fd3412dd700 1 -- 172.21.15.47:0/3192789728 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd33c0b4300 msgr2=0x7fd33c0612c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:23.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.429+0000 7fd3412dd700 1 --2- 172.21.15.47:0/3192789728 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd33c0b4300 0x7fd33c0612c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7fd330002f70 tx=0x7fd330006eb0).stop 2022-06-29T15:52:23.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.429+0000 7fd3412dd700 1 -- 172.21.15.47:0/3192789728 shutdown_connections 2022-06-29T15:52:23.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.429+0000 7fd3412dd700 1 --2- 172.21.15.47:0/3192789728 >> v2:172.21.15.64:3300/0 conn(0x7fd33c1471a0 0x7fd33c066f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:23.446 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.429+0000 7fd3412dd700 1 --2- 172.21.15.47:0/3192789728 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd33c0b4300 0x7fd33c0612c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:23.446 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.429+0000 7fd3412dd700 1 -- 172.21.15.47:0/3192789728 >> 172.21.15.47:0/3192789728 conn(0x7fd33c05c510 msgr2=0x7fd33c05f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:23.446 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.433+0000 7fd3412dd700 1 -- 172.21.15.47:0/3192789728 shutdown_connections 2022-06-29T15:52:23.446 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.433+0000 7fd3412dd700 1 -- 172.21.15.47:0/3192789728 wait complete. 2022-06-29T15:52:23.447 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.433+0000 7fd3412dd700 1 Processor -- start 2022-06-29T15:52:23.447 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.433+0000 7fd3412dd700 1 -- start start 2022-06-29T15:52:23.447 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.433+0000 7fd3412dd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd33c060ee0 0x7fd33c0b3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:23.447 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.433+0000 7fd3412dd700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd33c0b4300 0x7fd33c0b1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:23.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.433+0000 7fd3412dd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd33c1471a0 0x7fd33c0b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:23.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.433+0000 7fd3412dd700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33c136200 con 0x7fd33c1471a0 2022-06-29T15:52:23.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.433+0000 7fd3412dd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33c137240 con 0x7fd33c060ee0 2022-06-29T15:52:23.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.433+0000 7fd3412dd700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd33c0b2060 con 0x7fd33c0b4300 2022-06-29T15:52:23.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.433+0000 7fd33a7fc700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd33c0b4300 0x7fd33c0b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:23.449 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.433+0000 7fd33a7fc700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd33c0b4300 0x7fd33c0b1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:52398/0 (socket says 172.21.15.47:52398) 2022-06-29T15:52:23.449 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.433+0000 7fd33a7fc700 1 -- 172.21.15.47:0/2370846038 learned_addr learned my addr 172.21.15.47:0/2370846038 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:23.449 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.433+0000 7fd33b7fe700 1 --2- 172.21.15.47:0/2370846038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd33c1471a0 0x7fd33c0b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:23.449 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.433+0000 7fd33affd700 1 --2- 172.21.15.47:0/2370846038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd33c060ee0 0x7fd33c0b3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:23.450 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.433+0000 7fd33a7fc700 1 -- 172.21.15.47:0/2370846038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd33c060ee0 msgr2=0x7fd33c0b3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:23.450 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.433+0000 7fd33a7fc700 1 --2- 172.21.15.47:0/2370846038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd33c060ee0 0x7fd33c0b3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:23.450 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.433+0000 7fd33a7fc700 1 -- 172.21.15.47:0/2370846038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd33c1471a0 msgr2=0x7fd33c0b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:23.450 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.433+0000 7fd33a7fc700 1 --2- 172.21.15.47:0/2370846038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd33c1471a0 0x7fd33c0b1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:23.451 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.433+0000 7fd33a7fc700 1 -- 172.21.15.47:0/2370846038 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd33000b1e0 con 0x7fd33c0b4300 2022-06-29T15:52:23.451 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.433+0000 7fd33a7fc700 1 --2- 172.21.15.47:0/2370846038 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd33c0b4300 0x7fd33c0b1680 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7fd33c136a50 tx=0x7fd330015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:23.451 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.433+0000 7fd32bfff700 1 -- 172.21.15.47:0/2370846038 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd330011760 con 0x7fd33c0b4300 2022-06-29T15:52:23.451 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.433+0000 7fd32bfff700 1 -- 172.21.15.47:0/2370846038 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3300118c0 con 0x7fd33c0b4300 2022-06-29T15:52:23.451 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.433+0000 7fd3412dd700 1 -- 172.21.15.47:0/2370846038 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd33c145ee0 con 0x7fd33c0b4300 2022-06-29T15:52:23.452 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.433+0000 7fd32bfff700 1 -- 172.21.15.47:0/2370846038 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd330011c20 con 0x7fd33c0b4300 2022-06-29T15:52:23.452 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.437+0000 7fd3412dd700 1 -- 172.21.15.47:0/2370846038 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd33c1366d0 con 0x7fd33c0b4300 2022-06-29T15:52:23.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.437+0000 7fd32bfff700 1 -- 172.21.15.47:0/2370846038 <== mon.2 v2:172.21.15.64:3301/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd330025030 con 0x7fd33c0b4300 2022-06-29T15:52:23.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.437+0000 7fd3412dd700 1 -- 172.21.15.47:0/2370846038 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd33c136860 con 0x7fd33c0b4300 2022-06-29T15:52:23.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.437+0000 7fd32bfff700 1 --2- 172.21.15.47:0/2370846038 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd318053980 0x7fd318055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:23.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.437+0000 7fd32bfff700 1 -- 172.21.15.47:0/2370846038 <== mon.2 v2:172.21.15.64:3301/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd33006d390 con 0x7fd33c0b4300 2022-06-29T15:52:23.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.437+0000 7fd33affd700 1 --2- 172.21.15.47:0/2370846038 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd318053980 0x7fd318055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:23.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.437+0000 7fd33affd700 1 --2- 172.21.15.47:0/2370846038 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd318053980 0x7fd318055e40 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x7fd32c00d000 tx=0x7fd32c007b90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:23.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.449+0000 7fd32bfff700 1 -- 172.21.15.47:0/2370846038 <== mon.2 v2:172.21.15.64:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd330043fc0 con 0x7fd33c0b4300 2022-06-29T15:52:23.583 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.573+0000 7fd3412dd700 1 -- 172.21.15.47:0/2370846038 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd33c0573d0 con 0x7fd33c0b4300 2022-06-29T15:52:23.584 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.577+0000 7fd32bfff700 1 -- 172.21.15.47:0/2370846038 <== mon.2 v2:172.21.15.64:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v162) v1 ==== 104+0+10 (secure 0 0 0) 0x7fd330023020 con 0x7fd33c0b4300 2022-06-29T15:52:23.584 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:52:23.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.577+0000 7fd3412dd700 1 -- 172.21.15.47:0/2370846038 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd318053980 msgr2=0x7fd318055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:23.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.577+0000 7fd3412dd700 1 --2- 172.21.15.47:0/2370846038 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd318053980 0x7fd318055e40 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x7fd32c00d000 tx=0x7fd32c007b90).stop 2022-06-29T15:52:23.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.577+0000 7fd3412dd700 1 -- 172.21.15.47:0/2370846038 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd33c0b4300 msgr2=0x7fd33c0b1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:23.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.581+0000 7fd3412dd700 1 --2- 172.21.15.47:0/2370846038 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd33c0b4300 0x7fd33c0b1680 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7fd33c136a50 tx=0x7fd330015000).stop 2022-06-29T15:52:23.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.581+0000 7fd3412dd700 1 -- 172.21.15.47:0/2370846038 shutdown_connections 2022-06-29T15:52:23.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.581+0000 7fd3412dd700 1 --2- 172.21.15.47:0/2370846038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd33c1471a0 0x7fd33c0b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:23.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.581+0000 7fd3412dd700 1 --2- 172.21.15.47:0/2370846038 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd33c0b4300 0x7fd33c0b1680 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:23.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.581+0000 7fd3412dd700 1 --2- 172.21.15.47:0/2370846038 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd318053980 0x7fd318055e40 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:23.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.581+0000 7fd3412dd700 1 --2- 172.21.15.47:0/2370846038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd33c060ee0 0x7fd33c0b3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:23.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.581+0000 7fd3412dd700 1 -- 172.21.15.47:0/2370846038 >> 172.21.15.47:0/2370846038 conn(0x7fd33c05c510 msgr2=0x7fd33c0b7bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:23.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.581+0000 7fd3412dd700 1 -- 172.21.15.47:0/2370846038 shutdown_connections 2022-06-29T15:52:23.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.581+0000 7fd3412dd700 1 -- 172.21.15.47:0/2370846038 wait complete. 2022-06-29T15:52:23.602 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-06-29T15:52:23.712 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.705+0000 7fb97a128700 1 Processor -- start 2022-06-29T15:52:23.712 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.705+0000 7fb97a128700 1 -- start start 2022-06-29T15:52:23.714 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.705+0000 7fb97a128700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb9740b2d10 0x7fb974095e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:23.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.705+0000 7fb97a128700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9740b1ac0 con 0x7fb9740b2930 2022-06-29T15:52:23.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.705+0000 7fb97a128700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb974095c70 con 0x7fb9740a9770 2022-06-29T15:52:23.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.705+0000 7fb97a128700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9740965b0 con 0x7fb9740b2d10 2022-06-29T15:52:23.715 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.705+0000 7fb9737fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb9740b2930 0x7fb974097a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40856/0 (socket says 172.21.15.47:40856) 2022-06-29T15:52:23.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.705+0000 7fb9737fe700 1 -- 172.21.15.47:0/2832765594 learned_addr learned my addr 172.21.15.47:0/2832765594 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:52:23.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.705+0000 7fb973fff700 1 --2- 172.21.15.47:0/2832765594 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb9740b2d10 0x7fb974095e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:23.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.705+0000 7fb9727fc700 1 -- 172.21.15.47:0/2832765594 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3650804025 0 0) 0x7fb9740b1ac0 con 0x7fb9740b2930 2022-06-29T15:52:23.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.705+0000 7fb9727fc700 1 -- 172.21.15.47:0/2832765594 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb954003260 con 0x7fb9740b2930 2022-06-29T15:52:23.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.705+0000 7fb973fff700 1 -- 172.21.15.47:0/2832765594 >> v1:172.21.15.64:6789/0 conn(0x7fb9740a9770 legacy=0x7fb974147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:23.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.705+0000 7fb973fff700 1 -- 172.21.15.47:0/2832765594 >> v1:172.21.15.47:6789/0 conn(0x7fb9740b2930 legacy=0x7fb974097a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:23.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.705+0000 7fb973fff700 1 -- 172.21.15.47:0/2832765594 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9740b1cf0 con 0x7fb9740b2d10 2022-06-29T15:52:23.717 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.705+0000 7fb973fff700 1 --2- 172.21.15.47:0/2832765594 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb9740b2d10 0x7fb974095e30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7fb96c0032f0 tx=0x7fb96c00b4b0).ready entity=mon.2 client_cookie=7e7e8eaf539deb1 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:23.718 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.705+0000 7fb9727fc700 1 -- 172.21.15.47:0/2832765594 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb96c00de70 con 0x7fb9740b2d10 2022-06-29T15:52:23.718 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.705+0000 7fb9727fc700 1 -- 172.21.15.47:0/2832765594 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb96c0153f0 con 0x7fb9740b2d10 2022-06-29T15:52:23.718 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.705+0000 7fb9727fc700 1 -- 172.21.15.47:0/2832765594 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb96c015750 con 0x7fb9740b2d10 2022-06-29T15:52:23.721 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.709+0000 7fb97a128700 1 -- 172.21.15.47:0/2832765594 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb9740b2d10 msgr2=0x7fb974095e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:23.722 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.709+0000 7fb97a128700 1 --2- 172.21.15.47:0/2832765594 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb9740b2d10 0x7fb974095e30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7fb96c0032f0 tx=0x7fb96c00b4b0).stop 2022-06-29T15:52:23.722 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.709+0000 7fb97a128700 1 -- 172.21.15.47:0/2832765594 shutdown_connections 2022-06-29T15:52:23.722 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.709+0000 7fb97a128700 1 --2- 172.21.15.47:0/2832765594 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb9740b2d10 0x7fb974095e30 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:23.723 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.709+0000 7fb97a128700 1 -- 172.21.15.47:0/2832765594 >> 172.21.15.47:0/2832765594 conn(0x7fb97405f0b0 msgr2=0x7fb97405f490 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:23.723 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.709+0000 7fb97a128700 1 -- 172.21.15.47:0/2832765594 shutdown_connections 2022-06-29T15:52:23.723 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.709+0000 7fb97a128700 1 -- 172.21.15.47:0/2832765594 wait complete. 2022-06-29T15:52:23.723 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.709+0000 7fb97a128700 1 Processor -- start 2022-06-29T15:52:23.723 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.709+0000 7fb97a128700 1 -- start start 2022-06-29T15:52:23.724 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.709+0000 7fb97a128700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9740a9770 0x7fb974094710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:23.724 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.709+0000 7fb97a128700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9740b2930 0x7fb974152880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:23.724 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.709+0000 7fb97a128700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb9740b2d10 0x7fb974156620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:23.724 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.709+0000 7fb97a128700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb974143bd0 con 0x7fb9740a9770 2022-06-29T15:52:23.725 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.709+0000 7fb97a128700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9741440a0 con 0x7fb9740b2930 2022-06-29T15:52:23.725 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.709+0000 7fb97a128700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb974152d70 con 0x7fb9740b2d10 2022-06-29T15:52:23.725 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.709+0000 7fb9737fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9740a9770 0x7fb974094710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:23.725 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.709+0000 7fb973fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb9740b2d10 0x7fb974156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:23.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.709+0000 7fb973fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb9740b2d10 0x7fb974156620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:52414/0 (socket says 172.21.15.47:52414) 2022-06-29T15:52:23.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.709+0000 7fb9737fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9740a9770 0x7fb974094710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55310/0 (socket says 172.21.15.47:55310) 2022-06-29T15:52:23.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.709+0000 7fb973fff700 1 -- 172.21.15.47:0/3964607099 learned_addr learned my addr 172.21.15.47:0/3964607099 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:23.726 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.709+0000 7fb972ffd700 1 --2- 172.21.15.47:0/3964607099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9740b2930 0x7fb974152880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:23.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.713+0000 7fb9737fe700 1 -- 172.21.15.47:0/3964607099 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb9740b2d10 msgr2=0x7fb974156620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:23.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.713+0000 7fb9737fe700 1 --2- 172.21.15.47:0/3964607099 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb9740b2d10 0x7fb974156620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:23.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.713+0000 7fb9737fe700 1 -- 172.21.15.47:0/3964607099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9740b2930 msgr2=0x7fb974152880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:23.727 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.713+0000 7fb9737fe700 1 --2- 172.21.15.47:0/3964607099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9740b2930 0x7fb974152880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:23.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.713+0000 7fb9737fe700 1 -- 172.21.15.47:0/3964607099 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb96c00b710 con 0x7fb9740a9770 2022-06-29T15:52:23.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.713+0000 7fb973fff700 1 --2- 172.21.15.47:0/3964607099 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb9740b2d10 0x7fb974156620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:52:23.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.713+0000 7fb9737fe700 1 --2- 172.21.15.47:0/3964607099 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9740a9770 0x7fb974094710 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 rx=0x7fb974143b20 tx=0x7fb964007a20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:23.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.713+0000 7fb970ff9700 1 -- 172.21.15.47:0/3964607099 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb96400ce10 con 0x7fb9740a9770 2022-06-29T15:52:23.728 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.713+0000 7fb970ff9700 1 -- 172.21.15.47:0/3964607099 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9640070d0 con 0x7fb9740a9770 2022-06-29T15:52:23.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.713+0000 7fb97a128700 1 -- 172.21.15.47:0/3964607099 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb974143e00 con 0x7fb9740a9770 2022-06-29T15:52:23.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.713+0000 7fb970ff9700 1 -- 172.21.15.47:0/3964607099 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb964007430 con 0x7fb9740a9770 2022-06-29T15:52:23.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.713+0000 7fb97a128700 1 -- 172.21.15.47:0/3964607099 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb974156180 con 0x7fb9740a9770 2022-06-29T15:52:23.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.713+0000 7fb97a128700 1 -- 172.21.15.47:0/3964607099 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9740573d0 con 0x7fb9740a9770 2022-06-29T15:52:23.731 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.713+0000 7fb970ff9700 1 -- 172.21.15.47:0/3964607099 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb964010070 con 0x7fb9740a9770 2022-06-29T15:52:23.731 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.713+0000 7fb970ff9700 1 --2- 172.21.15.47:0/3964607099 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb958053a70 0x7fb958055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:23.731 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.717+0000 7fb970ff9700 1 -- 172.21.15.47:0/3964607099 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb96406b780 con 0x7fb9740a9770 2022-06-29T15:52:23.731 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.717+0000 7fb972ffd700 1 --2- 172.21.15.47:0/3964607099 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb958053a70 0x7fb958055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:23.732 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.717+0000 7fb972ffd700 1 --2- 172.21.15.47:0/3964607099 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb958053a70 0x7fb958055f30 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 rx=0x7fb95c00d000 tx=0x7fb95c007b90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:23.732 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.721+0000 7fb970ff9700 1 -- 172.21.15.47:0/3964607099 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb9640423b0 con 0x7fb9740a9770 2022-06-29T15:52:23.857 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.849+0000 7fb97a128700 1 -- 172.21.15.47:0/3964607099 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb9740b4500 con 0x7fb9740a9770 2022-06-29T15:52:23.857 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.849+0000 7fb970ff9700 1 -- 172.21.15.47:0/3964607099 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v162) v1 ==== 100+0+10 (secure 0 0 0) 0x7fb9640459d0 con 0x7fb9740a9770 2022-06-29T15:52:23.858 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2022-06-29T15:52:23.860 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.853+0000 7fb97a128700 1 -- 172.21.15.47:0/3964607099 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb958053a70 msgr2=0x7fb958055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:23.860 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.853+0000 7fb97a128700 1 --2- 172.21.15.47:0/3964607099 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb958053a70 0x7fb958055f30 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 rx=0x7fb95c00d000 tx=0x7fb95c007b90).stop 2022-06-29T15:52:23.860 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.853+0000 7fb97a128700 1 -- 172.21.15.47:0/3964607099 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9740a9770 msgr2=0x7fb974094710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:23.861 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.853+0000 7fb97a128700 1 --2- 172.21.15.47:0/3964607099 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9740a9770 0x7fb974094710 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 rx=0x7fb974143b20 tx=0x7fb964007a20).stop 2022-06-29T15:52:23.861 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.853+0000 7fb97a128700 1 -- 172.21.15.47:0/3964607099 shutdown_connections 2022-06-29T15:52:23.861 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.853+0000 7fb97a128700 1 --2- 172.21.15.47:0/3964607099 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb9740b2d10 0x7fb974156620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:23.861 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.853+0000 7fb97a128700 1 --2- 172.21.15.47:0/3964607099 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb958053a70 0x7fb958055f30 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:23.862 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.853+0000 7fb97a128700 1 --2- 172.21.15.47:0/3964607099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9740b2930 0x7fb974152880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:23.862 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.853+0000 7fb97a128700 1 --2- 172.21.15.47:0/3964607099 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9740a9770 0x7fb974094710 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:23.862 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.853+0000 7fb97a128700 1 -- 172.21.15.47:0/3964607099 >> 172.21.15.47:0/3964607099 conn(0x7fb97405f0b0 msgr2=0x7fb9740b6720 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:23.862 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.853+0000 7fb97a128700 1 -- 172.21.15.47:0/3964607099 shutdown_connections 2022-06-29T15:52:23.863 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.853+0000 7fb97a128700 1 -- 172.21.15.47:0/3964607099 wait complete. 2022-06-29T15:52:23.871 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-06-29T15:52:23.871 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:52:23.978 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.969+0000 7fa6fbfff700 1 Processor -- start 2022-06-29T15:52:23.978 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.969+0000 7fa6fbfff700 1 -- start start 2022-06-29T15:52:23.980 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.973+0000 7fa6fbfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa6fc0b2190 0x7fa6fc13caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:23.980 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.973+0000 7fa6fbfff700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6fc0b0440 con 0x7fa6fc0acff0 2022-06-29T15:52:23.980 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.973+0000 7fa6fbfff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6fc0ad420 con 0x7fa6fc0b1940 2022-06-29T15:52:23.981 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.973+0000 7fa6fbfff700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6fc0b1110 con 0x7fa6fc0b2190 2022-06-29T15:52:23.981 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.973+0000 7fa6fb7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa6fc0acff0 0x7fa6fc14a020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40874/0 (socket says 172.21.15.47:40874) 2022-06-29T15:52:23.981 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.973+0000 7fa6faffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa6fc0b2190 0x7fa6fc13caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:23.981 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.973+0000 7fa6fb7fe700 1 -- 172.21.15.47:0/2397780682 learned_addr learned my addr 172.21.15.47:0/2397780682 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:52:23.983 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.973+0000 7fa6f9ffb700 1 -- 172.21.15.47:0/2397780682 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2135237697 0 0) 0x7fa6fc0b0440 con 0x7fa6fc0acff0 2022-06-29T15:52:23.983 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.973+0000 7fa6f9ffb700 1 -- 172.21.15.47:0/2397780682 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6e0003260 con 0x7fa6fc0acff0 2022-06-29T15:52:23.984 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.973+0000 7fa6faffd700 1 -- 172.21.15.47:0/2397780682 >> v1:172.21.15.64:6789/0 conn(0x7fa6fc0b1940 legacy=0x7fa6fc0ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:23.984 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.973+0000 7fa6faffd700 1 -- 172.21.15.47:0/2397780682 >> v1:172.21.15.47:6789/0 conn(0x7fa6fc0acff0 legacy=0x7fa6fc14a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:23.984 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.973+0000 7fa6faffd700 1 -- 172.21.15.47:0/2397780682 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6fc0b2570 con 0x7fa6fc0b2190 2022-06-29T15:52:23.984 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.973+0000 7fa6faffd700 1 --2- 172.21.15.47:0/2397780682 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6fc0b2190 0x7fa6fc13caa0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 rx=0x7fa6f4002f70 tx=0x7fa6f4006df0).ready entity=mon.0 client_cookie=8699ed8aad0edf25 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:23.985 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.973+0000 7fa6f9ffb700 1 -- 172.21.15.47:0/2397780682 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa6f400d4a0 con 0x7fa6fc0b2190 2022-06-29T15:52:23.985 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.973+0000 7fa6f9ffb700 1 -- 172.21.15.47:0/2397780682 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f4000e90 con 0x7fa6fc0b2190 2022-06-29T15:52:23.985 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.973+0000 7fa6f9ffb700 1 -- 172.21.15.47:0/2397780682 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa6f400d7f0 con 0x7fa6fc0b2190 2022-06-29T15:52:23.985 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.973+0000 7fa6fbfff700 1 -- 172.21.15.47:0/2397780682 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6fc0b2190 msgr2=0x7fa6fc13caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:23.985 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.973+0000 7fa6fbfff700 1 --2- 172.21.15.47:0/2397780682 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6fc0b2190 0x7fa6fc13caa0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 rx=0x7fa6f4002f70 tx=0x7fa6f4006df0).stop 2022-06-29T15:52:23.986 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.973+0000 7fa6fbfff700 1 -- 172.21.15.47:0/2397780682 shutdown_connections 2022-06-29T15:52:23.986 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.973+0000 7fa6fbfff700 1 --2- 172.21.15.47:0/2397780682 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6fc0b2190 0x7fa6fc13caa0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:23.986 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.973+0000 7fa6fbfff700 1 -- 172.21.15.47:0/2397780682 >> 172.21.15.47:0/2397780682 conn(0x7fa6fc062560 msgr2=0x7fa6fc062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:23.986 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.973+0000 7fa6fbfff700 1 -- 172.21.15.47:0/2397780682 shutdown_connections 2022-06-29T15:52:23.987 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.973+0000 7fa6fbfff700 1 -- 172.21.15.47:0/2397780682 wait complete. 2022-06-29T15:52:23.990 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.977+0000 7fa6fbfff700 1 Processor -- start 2022-06-29T15:52:23.990 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.977+0000 7fa6fbfff700 1 -- start start 2022-06-29T15:52:23.991 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.977+0000 7fa6fbfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6fc0acff0 0x7fa6fc140970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:23.991 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.977+0000 7fa6fbfff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa6fc0b1940 0x7fa6fc1528a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:23.991 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.977+0000 7fa6fbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6fc0b2190 0x7fa6fc152d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:23.991 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.977+0000 7fa6fbfff700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc140e60 con 0x7fa6fc0acff0 2022-06-29T15:52:23.991 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.977+0000 7fa6fbfff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc064010 con 0x7fa6fc0b2190 2022-06-29T15:52:23.992 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.977+0000 7fa6fbfff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc067060 con 0x7fa6fc0b1940 2022-06-29T15:52:23.992 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.977+0000 7fa6faffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6fc0acff0 0x7fa6fc140970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:23.992 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.977+0000 7fa6faffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6fc0acff0 0x7fa6fc140970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55322/0 (socket says 172.21.15.47:55322) 2022-06-29T15:52:23.992 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.977+0000 7fa6faffd700 1 -- 172.21.15.47:0/471551523 learned_addr learned my addr 172.21.15.47:0/471551523 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:23.993 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.977+0000 7fa6fb7fe700 1 --2- 172.21.15.47:0/471551523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6fc0b2190 0x7fa6fc152d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:23.993 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.977+0000 7fa6fa7fc700 1 --2- 172.21.15.47:0/471551523 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa6fc0b1940 0x7fa6fc1528a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:23.993 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.977+0000 7fa6faffd700 1 -- 172.21.15.47:0/471551523 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa6fc0b1940 msgr2=0x7fa6fc1528a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:23.993 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.977+0000 7fa6faffd700 1 --2- 172.21.15.47:0/471551523 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa6fc0b1940 0x7fa6fc1528a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:23.993 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.977+0000 7fa6faffd700 1 -- 172.21.15.47:0/471551523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6fc0b2190 msgr2=0x7fa6fc152d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:23.994 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.977+0000 7fa6faffd700 1 --2- 172.21.15.47:0/471551523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6fc0b2190 0x7fa6fc152d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:23.994 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.977+0000 7fa6faffd700 1 -- 172.21.15.47:0/471551523 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f400b120 con 0x7fa6fc0acff0 2022-06-29T15:52:23.994 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.977+0000 7fa6fb7fe700 1 --2- 172.21.15.47:0/471551523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6fc0b2190 0x7fa6fc152d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:52:23.994 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.977+0000 7fa6fa7fc700 1 --2- 172.21.15.47:0/471551523 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa6fc0b1940 0x7fa6fc1528a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:52:23.995 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.977+0000 7fa6faffd700 1 --2- 172.21.15.47:0/471551523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6fc0acff0 0x7fa6fc140970 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 rx=0x7fa6f4002f70 tx=0x7fa6f4014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:23.995 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.977+0000 7fa6ebfff700 1 -- 172.21.15.47:0/471551523 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa6f401e070 con 0x7fa6fc0acff0 2022-06-29T15:52:23.995 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.977+0000 7fa6ebfff700 1 -- 172.21.15.47:0/471551523 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f40198f0 con 0x7fa6fc0acff0 2022-06-29T15:52:23.995 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.977+0000 7fa6ebfff700 1 -- 172.21.15.47:0/471551523 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa6f4019c50 con 0x7fa6fc0acff0 2022-06-29T15:52:23.995 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.977+0000 7fa6fbfff700 1 -- 172.21.15.47:0/471551523 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6fc153280 con 0x7fa6fc0acff0 2022-06-29T15:52:23.996 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.981+0000 7fa6fbfff700 1 -- 172.21.15.47:0/471551523 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6fc063450 con 0x7fa6fc0acff0 2022-06-29T15:52:23.996 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.981+0000 7fa6fbfff700 1 -- 172.21.15.47:0/471551523 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6fc067220 con 0x7fa6fc0acff0 2022-06-29T15:52:23.996 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.981+0000 7fa6ebfff700 1 -- 172.21.15.47:0/471551523 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa6f402a070 con 0x7fa6fc0acff0 2022-06-29T15:52:23.996 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.981+0000 7fa6ebfff700 1 --2- 172.21.15.47:0/471551523 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa6d80539e0 0x7fa6d8055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:23.997 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.981+0000 7fa6ebfff700 1 -- 172.21.15.47:0/471551523 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa6f406e850 con 0x7fa6fc0acff0 2022-06-29T15:52:23.997 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.981+0000 7fa6fa7fc700 1 --2- 172.21.15.47:0/471551523 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa6d80539e0 0x7fa6d8055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:23.997 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.981+0000 7fa6fa7fc700 1 --2- 172.21.15.47:0/471551523 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa6d80539e0 0x7fa6d8055ea0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 rx=0x7fa6f0008000 tx=0x7fa6f0009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:23.998 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:23.989+0000 7fa6ebfff700 1 -- 172.21.15.47:0/471551523 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa6f4045540 con 0x7fa6fc0acff0 2022-06-29T15:52:24.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:24.153+0000 7fa6fbfff700 1 -- 172.21.15.47:0/471551523 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa6fc0573d0 con 0x7fa6fc0acff0 2022-06-29T15:52:24.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:24.157+0000 7fa6ebfff700 1 -- 172.21.15.47:0/471551523 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+2495 (secure 0 0 0) 0x7fa6f4015040 con 0x7fa6fc0acff0 2022-06-29T15:52:24.163 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:52:24.164 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":79,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8596,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6836/1456033193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1456033193},{"type":"v1","addr":"172.21.15.47:6836","nonce":1456033193}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":77},{"gid":8745,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/1940204812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6835","nonce":1940204812},{"type":"v1","addr":"172.21.15.47:6837","nonce":1940204812}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":77}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:52:17.031576+0000","modified":"2022-06-29T15:52:18.034584+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":57377},"failed":[],"damaged":[],"stopped":[],"info":{"gid_57377":{"gid":57377,"name":"b","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6833/642573447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":642573447},{"type":"v1","addr":"172.21.15.64:6833","nonce":642573447}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2022-06-29T15:52:24.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:24.157+0000 7fa6fbfff700 1 -- 172.21.15.47:0/471551523 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa6d80539e0 msgr2=0x7fa6d8055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:24.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:24.157+0000 7fa6fbfff700 1 --2- 172.21.15.47:0/471551523 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa6d80539e0 0x7fa6d8055ea0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 rx=0x7fa6f0008000 tx=0x7fa6f0009040).stop 2022-06-29T15:52:24.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:24.157+0000 7fa6fbfff700 1 -- 172.21.15.47:0/471551523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6fc0acff0 msgr2=0x7fa6fc140970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:24.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:24.157+0000 7fa6fbfff700 1 --2- 172.21.15.47:0/471551523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6fc0acff0 0x7fa6fc140970 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 rx=0x7fa6f4002f70 tx=0x7fa6f4014000).stop 2022-06-29T15:52:24.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:24.157+0000 7fa6fbfff700 1 -- 172.21.15.47:0/471551523 shutdown_connections 2022-06-29T15:52:24.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:24.157+0000 7fa6fbfff700 1 --2- 172.21.15.47:0/471551523 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa6d80539e0 0x7fa6d8055ea0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:24.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:24.157+0000 7fa6fbfff700 1 --2- 172.21.15.47:0/471551523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6fc0b2190 0x7fa6fc152d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:24.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:24.157+0000 7fa6fbfff700 1 --2- 172.21.15.47:0/471551523 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa6fc0b1940 0x7fa6fc1528a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:24.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:24.157+0000 7fa6fbfff700 1 --2- 172.21.15.47:0/471551523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6fc0acff0 0x7fa6fc140970 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:24.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:24.157+0000 7fa6fbfff700 1 -- 172.21.15.47:0/471551523 >> 172.21.15.47:0/471551523 conn(0x7fa6fc062560 msgr2=0x7fa6fc0b4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:24.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:24.157+0000 7fa6fbfff700 1 -- 172.21.15.47:0/471551523 shutdown_connections 2022-06-29T15:52:24.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:24.157+0000 7fa6fbfff700 1 -- 172.21.15.47:0/471551523 wait complete. 2022-06-29T15:52:24.169 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 79 2022-06-29T15:52:24.479 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 79, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-06-29T15:52:17.031576+0000', 'modified': '2022-06-29T15:52:18.034584+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 57377}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_57377': {'gid': 57377, 'name': 'b', 'rank': 0, 'incarnation': 78, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.64:6833/642573447', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6832', 'nonce': 642573447}, {'type': 'v1', 'addr': '172.21.15.64:6833', 'nonce': 642573447}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-06-29T15:52:24.479 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-06-29T15:52:24.480 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-06-29T15:52:24.597 INFO:teuthology.orchestra.run.smithi064.stdout:{"cluster_fsid":"9a68894a-53ce-400e-838b-14eab91b834c","whoami":0,"id":57377,"want_state":"up:active","state":"up:active","rank_uptime":7.5486609299999996,"mdsmap_epoch":79,"osdmap_epoch":162,"osdmap_epoch_barrier":161,"uptime":12.597598823} 2022-06-29T15:52:24.598 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9a68894a-53ce-400e-838b-14eab91b834c", "id": 57377, "mdsmap_epoch": 79, "osdmap_epoch": 162, "osdmap_epoch_barrier": 161, "rank_uptime": 7.54866093, "state": "up:active", "uptime": 12.597598823, "want_state": "up:active", "whoami": 0 } 2022-06-29T15:52:24.598 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-06-29T15:52:24.599 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2022-06-29T15:52:24.599 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi164.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2022-06-29T15:52:24.600 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:52:24.600 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:52:24.609 DEBUG:teuthology.orchestra.run.smithi164:> sudo modprobe fuse 2022-06-29T15:52:24.628 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:52:24.628 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:52:24.686 INFO:teuthology.orchestra.run.smithi164.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:52:24.687 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:52:24.688 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:52:24.689 DEBUG:teuthology.orchestra.run.smithi164:> ls /sys/fs/fuse/connections 2022-06-29T15:52:24.693 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-06-29T15:52:24.694 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:52:24.696 DEBUG:teuthology.orchestra.run.smithi164:> sudo modprobe fuse 2022-06-29T15:52:24.711 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:52:24.711 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:52:24.724 INFO:teuthology.orchestra.run.smithi164.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:52:24.725 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:52:24.726 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:52:24.726 DEBUG:teuthology.orchestra.run.smithi164:> ls /sys/fs/fuse/connections 2022-06-29T15:52:24.766 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi164.stderr:2022-06-29T15:52:24.755+0000 7fc09fe28080 -1 init, newargv = 0x55c6dad833c0 newargc=9ceph-fuse[13176]: starting ceph client 2022-06-29T15:52:24.766 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi164.stderr: 2022-06-29T15:52:24.775 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi164.stderr:ceph-fuse[13176]: starting fuse 2022-06-29T15:52:25.872 DEBUG:teuthology.orchestra.run.smithi164:> sudo modprobe fuse 2022-06-29T15:52:25.891 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-06-29T15:52:25.892 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-06-29T15:52:25.950 INFO:teuthology.orchestra.run.smithi164.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-06-29T15:52:25.950 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-06-29T15:52:25.951 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:52:25.952 DEBUG:teuthology.orchestra.run.smithi164:> ls /sys/fs/fuse/connections 2022-06-29T15:52:25.957 INFO:teuthology.orchestra.run.smithi164.stdout:52 2022-06-29T15:52:25.958 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-06-29T15:52:25.959 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:52:25.959 DEBUG:teuthology.orchestra.run.smithi164:> sudo python3 -c ' 2022-06-29T15:52:25.960 DEBUG:teuthology.orchestra.run.smithi164:> import glob 2022-06-29T15:52:25.960 DEBUG:teuthology.orchestra.run.smithi164:> import re 2022-06-29T15:52:25.961 DEBUG:teuthology.orchestra.run.smithi164:> import os 2022-06-29T15:52:25.961 DEBUG:teuthology.orchestra.run.smithi164:> import subprocess 2022-06-29T15:52:25.961 DEBUG:teuthology.orchestra.run.smithi164:> 2022-06-29T15:52:25.962 DEBUG:teuthology.orchestra.run.smithi164:> def find_socket(client_name): 2022-06-29T15:52:25.962 DEBUG:teuthology.orchestra.run.smithi164:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2022-06-29T15:52:25.963 DEBUG:teuthology.orchestra.run.smithi164:> files = glob.glob(asok_path) 2022-06-29T15:52:25.963 DEBUG:teuthology.orchestra.run.smithi164:> 2022-06-29T15:52:25.964 DEBUG:teuthology.orchestra.run.smithi164:> # Given a non-glob path, it better be there 2022-06-29T15:52:25.964 DEBUG:teuthology.orchestra.run.smithi164:> if "*" not in asok_path: 2022-06-29T15:52:25.965 DEBUG:teuthology.orchestra.run.smithi164:> assert(len(files) == 1) 2022-06-29T15:52:25.965 DEBUG:teuthology.orchestra.run.smithi164:> return files[0] 2022-06-29T15:52:25.966 DEBUG:teuthology.orchestra.run.smithi164:> 2022-06-29T15:52:25.966 DEBUG:teuthology.orchestra.run.smithi164:> for f in files: 2022-06-29T15:52:25.966 DEBUG:teuthology.orchestra.run.smithi164:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-06-29T15:52:25.967 DEBUG:teuthology.orchestra.run.smithi164:> if os.path.exists("/proc/{0}".format(pid)): 2022-06-29T15:52:25.967 DEBUG:teuthology.orchestra.run.smithi164:> return f 2022-06-29T15:52:25.968 DEBUG:teuthology.orchestra.run.smithi164:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-06-29T15:52:25.968 DEBUG:teuthology.orchestra.run.smithi164:> 2022-06-29T15:52:25.968 DEBUG:teuthology.orchestra.run.smithi164:> print(find_socket("client.1")) 2022-06-29T15:52:25.969 DEBUG:teuthology.orchestra.run.smithi164:> ' 2022-06-29T15:52:26.045 INFO:teuthology.orchestra.run.smithi164.stdout:/var/run/ceph/ceph-client.1.13176.asok 2022-06-29T15:52:26.050 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.13176.asok 2022-06-29T15:52:26.051 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:52:26.052 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.13176.asok status 2022-06-29T15:52:26.141 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2022-06-29T15:52:26.142 INFO:teuthology.orchestra.run.smithi164.stdout: "metadata": { 2022-06-29T15:52:26.142 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_sha1": "f25ffac72e5a9e63860ff7deb6767a11f0559a16", 2022-06-29T15:52:26.142 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_version": "ceph version 15.2.16-304-gf25ffac7 (f25ffac72e5a9e63860ff7deb6767a11f0559a16) octopus (stable)", 2022-06-29T15:52:26.142 INFO:teuthology.orchestra.run.smithi164.stdout: "entity_id": "1", 2022-06-29T15:52:26.143 INFO:teuthology.orchestra.run.smithi164.stdout: "hostname": "smithi164", 2022-06-29T15:52:26.143 INFO:teuthology.orchestra.run.smithi164.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2022-06-29T15:52:26.143 INFO:teuthology.orchestra.run.smithi164.stdout: "pid": "13176", 2022-06-29T15:52:26.143 INFO:teuthology.orchestra.run.smithi164.stdout: "root": "/" 2022-06-29T15:52:26.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2022-06-29T15:52:26.144 INFO:teuthology.orchestra.run.smithi164.stdout: "dentry_count": 0, 2022-06-29T15:52:26.144 INFO:teuthology.orchestra.run.smithi164.stdout: "dentry_pinned_count": 0, 2022-06-29T15:52:26.144 INFO:teuthology.orchestra.run.smithi164.stdout: "id": 8955, 2022-06-29T15:52:26.144 INFO:teuthology.orchestra.run.smithi164.stdout: "inst": { 2022-06-29T15:52:26.145 INFO:teuthology.orchestra.run.smithi164.stdout: "name": { 2022-06-29T15:52:26.145 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "client", 2022-06-29T15:52:26.145 INFO:teuthology.orchestra.run.smithi164.stdout: "num": 8955 2022-06-29T15:52:26.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2022-06-29T15:52:26.145 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": { 2022-06-29T15:52:26.146 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "v1", 2022-06-29T15:52:26.146 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": "172.21.15.164:0", 2022-06-29T15:52:26.146 INFO:teuthology.orchestra.run.smithi164.stdout: "nonce": 1664264716 2022-06-29T15:52:26.146 INFO:teuthology.orchestra.run.smithi164.stdout: } 2022-06-29T15:52:26.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2022-06-29T15:52:26.147 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": { 2022-06-29T15:52:26.147 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "v1", 2022-06-29T15:52:26.147 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": "172.21.15.164:0", 2022-06-29T15:52:26.147 INFO:teuthology.orchestra.run.smithi164.stdout: "nonce": 1664264716 2022-06-29T15:52:26.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2022-06-29T15:52:26.148 INFO:teuthology.orchestra.run.smithi164.stdout: "inst_str": "client.8955 172.21.15.164:0/1664264716", 2022-06-29T15:52:26.148 INFO:teuthology.orchestra.run.smithi164.stdout: "addr_str": "172.21.15.164:0/1664264716", 2022-06-29T15:52:26.148 INFO:teuthology.orchestra.run.smithi164.stdout: "inode_count": 1, 2022-06-29T15:52:26.149 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_epoch": 79, 2022-06-29T15:52:26.149 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_epoch": 162, 2022-06-29T15:52:26.149 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_epoch_barrier": 0, 2022-06-29T15:52:26.149 INFO:teuthology.orchestra.run.smithi164.stdout: "blacklisted": false, 2022-06-29T15:52:26.149 INFO:teuthology.orchestra.run.smithi164.stdout: "fs_name": "cephfs" 2022-06-29T15:52:26.150 INFO:teuthology.orchestra.run.smithi164.stdout:} 2022-06-29T15:52:26.151 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:52:26.152 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:52:26.152 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:52:26.159 INFO:teuthology.orchestra.run.smithi164.stdout:fuseblk 2022-06-29T15:52:26.160 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-06-29T15:52:26.161 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:52:26.161 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:52:26.221 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:52:26.222 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:52:26.222 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:52:26.230 INFO:teuthology.orchestra.run.smithi164.stdout:fuseblk 2022-06-29T15:52:26.231 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-06-29T15:52:26.232 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:52:26.233 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:52:26.233 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:52:26.240 INFO:teuthology.orchestra.run.smithi164.stdout:fuseblk 2022-06-29T15:52:26.241 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-06-29T15:52:26.242 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi164.front.sepia.ceph.com... 2022-06-29T15:52:26.242 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-06-29T15:52:26.243 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:52:26.297 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi164.stderr:ceph-fuse[13176]: fuse finished with error 0 and tester_r 0 2022-06-29T15:52:26.300 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:52:26.300 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:52:26.301 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:52:26.308 INFO:teuthology.orchestra.run.smithi164.stdout:ext2/ext3 2022-06-29T15:52:26.309 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-06-29T15:52:26.309 INFO:teuthology.orchestra.run:waiting for 900 2022-06-29T15:52:26.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-06-29T15:52:26.361 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-06-29T15:52:26.368 INFO:teuthology.orchestra.run:Running command with timeout 10800 2022-06-29T15:52:26.369 DEBUG:teuthology.orchestra.run.smithi099:> cd /home/ubuntu/cephtest/mnt.0 && sudo bash -c 'stat -f .' 2022-06-29T15:52:26.388 INFO:teuthology.orchestra.run.smithi099.stdout: File: "." 2022-06-29T15:52:26.389 INFO:teuthology.orchestra.run.smithi099.stdout: ID: 0 Namelen: 255 Type: fuseblk 2022-06-29T15:52:26.389 INFO:teuthology.orchestra.run.smithi099.stdout:Block size: 4194304 Fundamental block size: 4194304 2022-06-29T15:52:26.389 INFO:teuthology.orchestra.run.smithi099.stdout:Blocks: Total: 86526 Free: 86526 Available: 86526 2022-06-29T15:52:26.390 INFO:teuthology.orchestra.run.smithi099.stdout:Inodes: Total: 1 Free: 0 2022-06-29T15:52:26.391 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-06-29T15:52:26.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.497+0000 7effd5e9c700 1 Processor -- start 2022-06-29T15:52:26.506 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.497+0000 7effd5e9c700 1 -- start start 2022-06-29T15:52:26.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.497+0000 7effd5e9c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7effd005c510 0x7effd0143bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:26.508 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.497+0000 7effd5e9c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7effd00b2c40 0x7effd00b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:26.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.497+0000 7effd5e9c700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effd0142f90 con 0x7effd005c8f0 2022-06-29T15:52:26.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.497+0000 7effd5e9c700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effd01431f0 con 0x7effd005c510 2022-06-29T15:52:26.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.497+0000 7effd5e9c700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effd0143490 con 0x7effd00b2c40 2022-06-29T15:52:26.509 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.501+0000 7effd4e9a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7effd005c510 0x7effd0143bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:26.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.501+0000 7effd4e9a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7effd005c510 0x7effd0143bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55332/0 (socket says 172.21.15.47:55332) 2022-06-29T15:52:26.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.501+0000 7effd4e9a700 1 -- 172.21.15.47:0/2808017699 learned_addr learned my addr 172.21.15.47:0/2808017699 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:26.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.501+0000 7effd569b700 1 --2- 172.21.15.47:0/2808017699 >> v2:172.21.15.64:3300/0 conn(0x7effd00b2c40 0x7effd00b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:26.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.501+0000 7effd4e9a700 1 -- 172.21.15.47:0/2808017699 >> v2:172.21.15.64:3300/0 conn(0x7effd00b2c40 msgr2=0x7effd00b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:26.510 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.501+0000 7effd4e9a700 1 --2- 172.21.15.47:0/2808017699 >> v2:172.21.15.64:3300/0 conn(0x7effd00b2c40 0x7effd00b30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:26.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.501+0000 7effd4e9a700 1 -- 172.21.15.47:0/2808017699 >> v1:172.21.15.64:6789/0 conn(0x7effd005c8f0 legacy=0x7effd00b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:26.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.501+0000 7effd4e9a700 1 -- 172.21.15.47:0/2808017699 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd0143730 con 0x7effd005c510 2022-06-29T15:52:26.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.501+0000 7effd4e9a700 1 --2- 172.21.15.47:0/2808017699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effd005c510 0x7effd0143bd0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 rx=0x7effc0002f70 tx=0x7effc0006eb0).ready entity=mon.0 client_cookie=959e4f4b4258f357 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:26.511 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.501+0000 7effcf7fe700 1 -- 172.21.15.47:0/2808017699 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effc000d5c0 con 0x7effd005c510 2022-06-29T15:52:26.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.501+0000 7effcf7fe700 1 -- 172.21.15.47:0/2808017699 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc0000e90 con 0x7effd005c510 2022-06-29T15:52:26.512 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.501+0000 7effcf7fe700 1 -- 172.21.15.47:0/2808017699 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effc000d950 con 0x7effd005c510 2022-06-29T15:52:26.515 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.501+0000 7effd5e9c700 1 -- 172.21.15.47:0/2808017699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effd005c510 msgr2=0x7effd0143bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:26.515 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.501+0000 7effd5e9c700 1 --2- 172.21.15.47:0/2808017699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effd005c510 0x7effd0143bd0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 rx=0x7effc0002f70 tx=0x7effc0006eb0).stop 2022-06-29T15:52:26.516 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.501+0000 7effd5e9c700 1 -- 172.21.15.47:0/2808017699 shutdown_connections 2022-06-29T15:52:26.516 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.501+0000 7effd5e9c700 1 --2- 172.21.15.47:0/2808017699 >> v2:172.21.15.64:3300/0 conn(0x7effd00b2c40 0x7effd00b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:26.516 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.501+0000 7effd5e9c700 1 --2- 172.21.15.47:0/2808017699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effd005c510 0x7effd0143bd0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:26.516 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.501+0000 7effd5e9c700 1 -- 172.21.15.47:0/2808017699 >> 172.21.15.47:0/2808017699 conn(0x7effd0060e00 msgr2=0x7effd00611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:26.517 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.501+0000 7effd5e9c700 1 -- 172.21.15.47:0/2808017699 shutdown_connections 2022-06-29T15:52:26.517 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.501+0000 7effd5e9c700 1 -- 172.21.15.47:0/2808017699 wait complete. 2022-06-29T15:52:26.517 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.501+0000 7effd5e9c700 1 Processor -- start 2022-06-29T15:52:26.517 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.501+0000 7effd5e9c700 1 -- start start 2022-06-29T15:52:26.518 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.505+0000 7effd5e9c700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7effd005c8f0 0x7effd0136920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:26.518 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.505+0000 7effd5e9c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effd00b2c40 0x7effd013b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:26.518 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.505+0000 7effd5e9c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effd013b910 0x7effd0155f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:26.518 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.505+0000 7effd5e9c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd00ae0f0 con 0x7effd013b910 2022-06-29T15:52:26.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.505+0000 7effd5e9c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd0137a90 con 0x7effd00b2c40 2022-06-29T15:52:26.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.505+0000 7effd5e9c700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effd0142ae0 con 0x7effd005c8f0 2022-06-29T15:52:26.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.505+0000 7effd569b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effd013b910 0x7effd0155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:26.519 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.505+0000 7effd4e9a700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7effd005c8f0 0x7effd0136920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:26.520 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.505+0000 7effd569b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effd013b910 0x7effd0155f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55340/0 (socket says 172.21.15.47:55340) 2022-06-29T15:52:26.520 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.505+0000 7effd569b700 1 -- 172.21.15.47:0/1441047835 learned_addr learned my addr 172.21.15.47:0/1441047835 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:26.520 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.505+0000 7effcffff700 1 --2- 172.21.15.47:0/1441047835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effd00b2c40 0x7effd013b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:26.520 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.505+0000 7effd569b700 1 -- 172.21.15.47:0/1441047835 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7effd005c8f0 msgr2=0x7effd0136920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:26.520 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.505+0000 7effd569b700 1 --2- 172.21.15.47:0/1441047835 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7effd005c8f0 0x7effd0136920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:26.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.505+0000 7effd569b700 1 -- 172.21.15.47:0/1441047835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effd00b2c40 msgr2=0x7effd013b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:26.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.505+0000 7effd569b700 1 --2- 172.21.15.47:0/1441047835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effd00b2c40 0x7effd013b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:26.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.505+0000 7effd569b700 1 -- 172.21.15.47:0/1441047835 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc000b1e0 con 0x7effd013b910 2022-06-29T15:52:26.521 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.505+0000 7effd569b700 1 --2- 172.21.15.47:0/1441047835 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effd013b910 0x7effd0155f20 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 rx=0x7effc8006e00 tx=0x7effc80091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:26.522 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.505+0000 7effcdffb700 1 -- 172.21.15.47:0/1441047835 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effc8012600 con 0x7effd013b910 2022-06-29T15:52:26.522 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.505+0000 7effcdffb700 1 -- 172.21.15.47:0/1441047835 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc8012760 con 0x7effd013b910 2022-06-29T15:52:26.522 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.505+0000 7effd5e9c700 1 -- 172.21.15.47:0/1441047835 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effd00adbb0 con 0x7effd013b910 2022-06-29T15:52:26.522 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.505+0000 7effcdffb700 1 -- 172.21.15.47:0/1441047835 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effc8012ac0 con 0x7effd013b910 2022-06-29T15:52:26.523 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.505+0000 7effd5e9c700 1 -- 172.21.15.47:0/1441047835 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effd01427d0 con 0x7effd013b910 2022-06-29T15:52:26.523 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.505+0000 7effd5e9c700 1 -- 172.21.15.47:0/1441047835 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effd0155c10 con 0x7effd013b910 2022-06-29T15:52:26.523 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.509+0000 7effcdffb700 1 -- 172.21.15.47:0/1441047835 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7effc8022070 con 0x7effd013b910 2022-06-29T15:52:26.523 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.509+0000 7effcdffb700 1 --2- 172.21.15.47:0/1441047835 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7effa8053b00 0x7effa8055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:26.523 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.509+0000 7effcdffb700 1 -- 172.21.15.47:0/1441047835 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6420+0+0 (secure 0 0 0) 0x7effc806c670 con 0x7effd013b910 2022-06-29T15:52:26.524 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.509+0000 7effd4e9a700 1 --2- 172.21.15.47:0/1441047835 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7effa8053b00 0x7effa8055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:26.524 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.509+0000 7effd4e9a700 1 --2- 172.21.15.47:0/1441047835 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7effa8053b00 0x7effa8055fc0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7effc001c000 tx=0x7effc0015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:26.525 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.513+0000 7effcdffb700 1 -- 172.21.15.47:0/1441047835 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7effc8043390 con 0x7effd013b910 2022-06-29T15:52:26.653 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.645+0000 7effd5e9c700 1 -- 172.21.15.47:0/1441047835 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7effd00573d0 con 0x7effd013b910 2022-06-29T15:52:26.655 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.649+0000 7effcdffb700 1 -- 172.21.15.47:0/1441047835 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v162) v1 ==== 81+0+34051 (secure 0 0 0) 0x7effc8016020 con 0x7effd013b910 2022-06-29T15:52:26.656 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:52:26.656 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2022-06-29T15:52:26.656 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 162, 2022-06-29T15:52:26.657 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "9a68894a-53ce-400e-838b-14eab91b834c", 2022-06-29T15:52:26.657 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2022-06-29T15:42:14.046361+0000", 2022-06-29T15:52:26.657 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2022-06-29T15:52:18.035479+0000", 2022-06-29T15:52:26.657 INFO:teuthology.orchestra.run.smithi047.stdout: "last_up_change": "2022-06-29T15:42:20.929789+0000", 2022-06-29T15:52:26.658 INFO:teuthology.orchestra.run.smithi047.stdout: "last_in_change": "2022-06-29T15:42:20.929789+0000", 2022-06-29T15:52:26.658 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-06-29T15:52:26.658 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_num": 5799936, 2022-06-29T15:52:26.658 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_set": [ 2022-06-29T15:52:26.659 INFO:teuthology.orchestra.run.smithi047.stdout: "pglog_hardlimit", 2022-06-29T15:52:26.659 INFO:teuthology.orchestra.run.smithi047.stdout: "purged_snapdirs", 2022-06-29T15:52:26.659 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_deletes", 2022-06-29T15:52:26.659 INFO:teuthology.orchestra.run.smithi047.stdout: "sortbitwise" 2022-06-29T15:52:26.659 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:52:26.660 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_version": 5, 2022-06-29T15:52:26.660 INFO:teuthology.orchestra.run.smithi047.stdout: "full_ratio": 0.94999998807907104, 2022-06-29T15:52:26.660 INFO:teuthology.orchestra.run.smithi047.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-06-29T15:52:26.660 INFO:teuthology.orchestra.run.smithi047.stdout: "nearfull_ratio": 0.85000002384185791, 2022-06-29T15:52:26.661 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_snapshot": "", 2022-06-29T15:52:26.661 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_max": 30, 2022-06-29T15:52:26.661 INFO:teuthology.orchestra.run.smithi047.stdout: "max_osd": 8, 2022-06-29T15:52:26.661 INFO:teuthology.orchestra.run.smithi047.stdout: "require_min_compat_client": "jewel", 2022-06-29T15:52:26.661 INFO:teuthology.orchestra.run.smithi047.stdout: "min_compat_client": "jewel", 2022-06-29T15:52:26.662 INFO:teuthology.orchestra.run.smithi047.stdout: "require_osd_release": "octopus", 2022-06-29T15:52:26.662 INFO:teuthology.orchestra.run.smithi047.stdout: "pools": [ 2022-06-29T15:52:26.662 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.662 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 1, 2022-06-29T15:52:26.663 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "device_health_metrics", 2022-06-29T15:52:26.663 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:42:16.971431+0000", 2022-06-29T15:52:26.663 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:52:26.663 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:52:26.663 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:52:26.664 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:52:26.664 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:52:26.664 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:52:26.664 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:52:26.665 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:52:26.665 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 1, 2022-06-29T15:52:26.665 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 1, 2022-06-29T15:52:26.665 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 1, 2022-06-29T15:52:26.666 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 1, 2022-06-29T15:52:26.666 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 1, 2022-06-29T15:52:26.666 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:52:26.667 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:52:26.667 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:52:26.667 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:52:26.667 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:52:26.668 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:52:26.668 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:52:26.668 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.668 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "16", 2022-06-29T15:52:26.669 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:52:26.669 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:52:26.669 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:52:26.670 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:52:26.670 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:52:26.670 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:52:26.670 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:52:26.671 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:52:26.671 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:52:26.671 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:52:26.672 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:52:26.672 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:52:26.672 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:52:26.672 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:52:26.673 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:52:26.673 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:52:26.673 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:52:26.673 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:52:26.673 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:52:26.674 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:52:26.674 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:52:26.674 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:52:26.674 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:52:26.675 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:52:26.675 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:52:26.675 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:52:26.675 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.676 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:52:26.676 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:52:26.676 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:52:26.676 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:52:26.676 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:52:26.677 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:52:26.677 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:52:26.677 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:52:26.677 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:52:26.678 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:52:26.678 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:52:26.678 INFO:teuthology.orchestra.run.smithi047.stdout: "options": { 2022-06-29T15:52:26.678 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_min": 1 2022-06-29T15:52:26.679 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.679 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:52:26.679 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_devicehealth": {} 2022-06-29T15:52:26.679 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:26.679 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.680 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.680 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 2, 2022-06-29T15:52:26.680 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "rbd", 2022-06-29T15:52:26.680 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:42:25.290884+0000", 2022-06-29T15:52:26.681 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:52:26.681 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:52:26.681 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:52:26.681 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:52:26.681 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:52:26.682 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:52:26.682 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:52:26.682 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:52:26.682 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2022-06-29T15:52:26.683 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2022-06-29T15:52:26.683 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2022-06-29T15:52:26.683 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2022-06-29T15:52:26.683 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2022-06-29T15:52:26.683 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:52:26.684 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:52:26.684 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:52:26.684 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:52:26.684 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:52:26.685 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:52:26.685 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:52:26.685 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.685 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "19", 2022-06-29T15:52:26.685 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:52:26.686 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:52:26.686 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:52:26.686 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:52:26.686 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:52:26.687 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:52:26.687 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:52:26.687 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:52:26.687 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:52:26.687 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:52:26.688 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:52:26.688 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:52:26.688 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:52:26.688 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:52:26.689 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:52:26.689 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:52:26.689 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:52:26.689 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:52:26.689 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:52:26.690 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:52:26.690 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:52:26.690 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:52:26.691 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:52:26.691 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:52:26.691 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:52:26.691 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:52:26.692 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.692 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:52:26.692 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:52:26.692 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:52:26.692 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:52:26.693 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:52:26.693 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:52:26.693 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:52:26.693 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:52:26.694 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:52:26.694 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:52:26.694 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:52:26.694 INFO:teuthology.orchestra.run.smithi047.stdout: "options": {}, 2022-06-29T15:52:26.695 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:52:26.695 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd": {} 2022-06-29T15:52:26.695 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:26.695 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.695 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.696 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 29, 2022-06-29T15:52:26.696 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "cephfs_metadata", 2022-06-29T15:52:26.696 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:52:14.780743+0000", 2022-06-29T15:52:26.696 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:52:26.697 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:52:26.697 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:52:26.697 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:52:26.697 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:52:26.697 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:52:26.698 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:52:26.698 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:52:26.698 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2022-06-29T15:52:26.698 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2022-06-29T15:52:26.699 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2022-06-29T15:52:26.699 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2022-06-29T15:52:26.699 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2022-06-29T15:52:26.699 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:52:26.699 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:52:26.700 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:52:26.700 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:52:26.700 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:52:26.700 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:52:26.701 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:52:26.701 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.701 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "161", 2022-06-29T15:52:26.701 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:52:26.702 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:52:26.702 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:52:26.702 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:52:26.702 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:52:26.702 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:52:26.703 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:52:26.703 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:52:26.703 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:52:26.703 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:52:26.704 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:52:26.704 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:52:26.705 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:52:26.705 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:52:26.705 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:52:26.706 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:52:26.706 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:52:26.707 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:52:26.707 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:52:26.708 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:52:26.708 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:52:26.708 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:52:26.709 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:52:26.709 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:52:26.710 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:52:26.710 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:52:26.710 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.711 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:52:26.711 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:52:26.712 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:52:26.712 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:52:26.712 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:52:26.713 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:52:26.713 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:52:26.713 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:52:26.714 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:52:26.714 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:52:26.715 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:52:26.715 INFO:teuthology.orchestra.run.smithi047.stdout: "options": { 2022-06-29T15:52:26.716 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_bias": 4, 2022-06-29T15:52:26.716 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_min": 16, 2022-06-29T15:52:26.716 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_priority": 5 2022-06-29T15:52:26.717 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.717 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:52:26.718 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs": { 2022-06-29T15:52:26.718 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": "cephfs" 2022-06-29T15:52:26.718 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:26.719 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:26.719 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.720 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.720 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 30, 2022-06-29T15:52:26.720 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "cephfs_data", 2022-06-29T15:52:26.721 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2022-06-29T15:52:16.023272+0000", 2022-06-29T15:52:26.721 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2022-06-29T15:52:26.722 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2022-06-29T15:52:26.722 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2022-06-29T15:52:26.723 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2022-06-29T15:52:26.723 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2022-06-29T15:52:26.723 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2022-06-29T15:52:26.723 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2022-06-29T15:52:26.724 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2022-06-29T15:52:26.724 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2022-06-29T15:52:26.724 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2022-06-29T15:52:26.725 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2022-06-29T15:52:26.725 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2022-06-29T15:52:26.726 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2022-06-29T15:52:26.726 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2022-06-29T15:52:26.727 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2022-06-29T15:52:26.727 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2022-06-29T15:52:26.728 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2022-06-29T15:52:26.728 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2022-06-29T15:52:26.729 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2022-06-29T15:52:26.729 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2022-06-29T15:52:26.729 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.730 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "162", 2022-06-29T15:52:26.730 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2022-06-29T15:52:26.731 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2022-06-29T15:52:26.731 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2022-06-29T15:52:26.732 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2022-06-29T15:52:26.732 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2022-06-29T15:52:26.733 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2022-06-29T15:52:26.733 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2022-06-29T15:52:26.733 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2022-06-29T15:52:26.734 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2022-06-29T15:52:26.734 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2022-06-29T15:52:26.735 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2022-06-29T15:52:26.735 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2022-06-29T15:52:26.736 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2022-06-29T15:52:26.736 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2022-06-29T15:52:26.737 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2022-06-29T15:52:26.737 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2022-06-29T15:52:26.737 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2022-06-29T15:52:26.738 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2022-06-29T15:52:26.738 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-06-29T15:52:26.739 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-06-29T15:52:26.739 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2022-06-29T15:52:26.740 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2022-06-29T15:52:26.740 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2022-06-29T15:52:26.740 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2022-06-29T15:52:26.741 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2022-06-29T15:52:26.741 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2022-06-29T15:52:26.741 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.742 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2022-06-29T15:52:26.742 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2022-06-29T15:52:26.742 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2022-06-29T15:52:26.743 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2022-06-29T15:52:26.743 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2022-06-29T15:52:26.743 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2022-06-29T15:52:26.744 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2022-06-29T15:52:26.744 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2022-06-29T15:52:26.744 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2022-06-29T15:52:26.745 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2022-06-29T15:52:26.745 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2022-06-29T15:52:26.745 INFO:teuthology.orchestra.run.smithi047.stdout: "options": {}, 2022-06-29T15:52:26.746 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2022-06-29T15:52:26.746 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs": { 2022-06-29T15:52:26.746 INFO:teuthology.orchestra.run.smithi047.stdout: "data": "cephfs" 2022-06-29T15:52:26.747 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:26.747 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:26.747 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:26.747 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:52:26.748 INFO:teuthology.orchestra.run.smithi047.stdout: "osds": [ 2022-06-29T15:52:26.748 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.748 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2022-06-29T15:52:26.749 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "e23eb023-4706-4201-bf72-4bf19c912c5d", 2022-06-29T15:52:26.749 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:52:26.749 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:52:26.749 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:52:26.750 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:52:26.750 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:52:26.750 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:52:26.751 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:52:26.751 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 160, 2022-06-29T15:52:26.751 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:52:26.752 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:52:26.752 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:52:26.752 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:26.752 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.753 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:26.753 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6818", 2022-06-29T15:52:26.753 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:52:26.753 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.754 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.754 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:26.754 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6819", 2022-06-29T15:52:26.755 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:52:26.755 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:26.755 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:26.756 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.756 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:52:26.756 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:26.756 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.757 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:26.757 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6820", 2022-06-29T15:52:26.757 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:52:26.757 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.758 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.758 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:26.758 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6821", 2022-06-29T15:52:26.759 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:52:26.759 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:26.759 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:26.760 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.760 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:52:26.760 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:26.760 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.761 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:26.761 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6824", 2022-06-29T15:52:26.761 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:52:26.762 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.762 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.762 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:26.763 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6825", 2022-06-29T15:52:26.763 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:52:26.763 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:26.764 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:26.764 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.764 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:52:26.765 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:26.765 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.765 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:26.766 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6822", 2022-06-29T15:52:26.766 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:52:26.766 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.766 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.767 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:26.767 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6823", 2022-06-29T15:52:26.767 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13096 2022-06-29T15:52:26.768 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:26.768 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:26.768 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.768 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6819/13096", 2022-06-29T15:52:26.769 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6821/13096", 2022-06-29T15:52:26.769 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6825/13096", 2022-06-29T15:52:26.769 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6823/13096", 2022-06-29T15:52:26.769 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:52:26.770 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:52:26.770 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:52:26.770 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:26.771 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.771 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.771 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2022-06-29T15:52:26.772 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "228bb08d-2d64-43d9-9560-bfaeff154452", 2022-06-29T15:52:26.772 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:52:26.772 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:52:26.773 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:52:26.773 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:52:26.773 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:52:26.773 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:52:26.774 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:52:26.774 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 159, 2022-06-29T15:52:26.774 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:52:26.775 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:52:26.775 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:52:26.775 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:26.775 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.776 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:26.776 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6826", 2022-06-29T15:52:26.776 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:52:26.777 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.777 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.777 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:26.777 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6827", 2022-06-29T15:52:26.778 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:52:26.778 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:26.778 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:26.778 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.779 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:52:26.779 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:26.779 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.780 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:26.780 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6828", 2022-06-29T15:52:26.780 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:52:26.781 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.781 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.781 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:26.781 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6829", 2022-06-29T15:52:26.782 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:52:26.782 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:26.782 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:26.782 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.783 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:52:26.783 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:26.783 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.783 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:26.784 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6832", 2022-06-29T15:52:26.784 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:52:26.784 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.785 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.785 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:26.785 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6833", 2022-06-29T15:52:26.786 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:52:26.786 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:26.786 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:26.787 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.787 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:52:26.787 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:26.787 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.788 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:26.788 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6830", 2022-06-29T15:52:26.788 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:52:26.788 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.789 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.789 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:26.789 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6831", 2022-06-29T15:52:26.790 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13100 2022-06-29T15:52:26.790 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:26.790 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:26.790 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.791 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6827/13100", 2022-06-29T15:52:26.791 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6829/13100", 2022-06-29T15:52:26.791 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6833/13100", 2022-06-29T15:52:26.792 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6831/13100", 2022-06-29T15:52:26.792 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:52:26.792 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:52:26.793 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:52:26.793 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:26.794 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.794 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.795 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2022-06-29T15:52:26.795 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "797a5452-1510-4640-a111-c9c2fcf707eb", 2022-06-29T15:52:26.796 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:52:26.796 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:52:26.797 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:52:26.797 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:52:26.798 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:52:26.798 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:52:26.799 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:52:26.799 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 160, 2022-06-29T15:52:26.800 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:52:26.800 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:52:26.801 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:52:26.801 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:26.802 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.802 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:26.803 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6810", 2022-06-29T15:52:26.803 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:52:26.804 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.804 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.805 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:26.805 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6811", 2022-06-29T15:52:26.806 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:52:26.806 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:26.806 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:26.807 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.807 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:52:26.807 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:26.807 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.808 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:26.808 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6812", 2022-06-29T15:52:26.808 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:52:26.809 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.809 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.809 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:26.809 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6813", 2022-06-29T15:52:26.810 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:52:26.810 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:26.810 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:26.811 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.811 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:52:26.811 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:26.811 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.812 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:26.812 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6816", 2022-06-29T15:52:26.812 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:52:26.812 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.813 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.813 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:26.813 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6817", 2022-06-29T15:52:26.813 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:52:26.814 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:26.814 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:26.814 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.814 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:52:26.815 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:26.815 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.815 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:26.816 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6814", 2022-06-29T15:52:26.816 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:52:26.816 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.817 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.817 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:26.817 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6815", 2022-06-29T15:52:26.818 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13098 2022-06-29T15:52:26.818 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:26.818 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:26.819 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.819 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6811/13098", 2022-06-29T15:52:26.819 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6813/13098", 2022-06-29T15:52:26.819 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6817/13098", 2022-06-29T15:52:26.820 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6815/13098", 2022-06-29T15:52:26.820 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:52:26.820 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:52:26.821 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:52:26.821 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:26.821 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.821 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.822 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2022-06-29T15:52:26.822 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "55b24fd7-bb71-4a01-b3fa-a09caa772a62", 2022-06-29T15:52:26.822 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:52:26.823 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:52:26.823 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:52:26.823 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:52:26.824 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:52:26.824 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:52:26.824 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:52:26.824 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 160, 2022-06-29T15:52:26.825 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:52:26.825 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:52:26.825 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:52:26.826 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:26.826 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.826 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:26.826 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6802", 2022-06-29T15:52:26.827 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:52:26.827 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.827 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.828 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:26.828 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6803", 2022-06-29T15:52:26.828 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:52:26.828 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:26.829 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:26.829 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.829 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:52:26.830 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:26.830 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.831 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:26.831 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6804", 2022-06-29T15:52:26.831 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:52:26.831 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.832 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.832 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:26.832 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6805", 2022-06-29T15:52:26.832 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:52:26.833 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:26.833 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:26.833 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.834 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:52:26.834 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:26.834 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.834 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:26.835 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6808", 2022-06-29T15:52:26.835 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:52:26.835 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.835 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.835 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:26.836 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6809", 2022-06-29T15:52:26.836 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:52:26.836 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:26.836 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:26.837 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.837 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:52:26.837 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:26.838 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.838 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:26.838 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6806", 2022-06-29T15:52:26.838 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:52:26.839 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.839 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.839 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:26.839 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6807", 2022-06-29T15:52:26.840 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13097 2022-06-29T15:52:26.840 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:26.841 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:26.841 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.841 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6803/13097", 2022-06-29T15:52:26.842 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6805/13097", 2022-06-29T15:52:26.842 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6809/13097", 2022-06-29T15:52:26.842 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6807/13097", 2022-06-29T15:52:26.843 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:52:26.843 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:52:26.843 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:52:26.844 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:26.844 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.844 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.845 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2022-06-29T15:52:26.845 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3", 2022-06-29T15:52:26.845 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:52:26.846 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:52:26.846 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:52:26.846 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:52:26.847 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:52:26.847 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:52:26.847 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:52:26.848 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 159, 2022-06-29T15:52:26.848 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:52:26.848 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:52:26.848 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:52:26.849 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:26.849 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.849 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:26.849 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6816", 2022-06-29T15:52:26.850 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:52:26.850 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.850 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.850 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:26.851 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6817", 2022-06-29T15:52:26.851 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:52:26.851 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:26.851 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:26.851 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.852 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:52:26.852 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:26.852 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.852 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:26.853 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6818", 2022-06-29T15:52:26.853 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:52:26.853 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.853 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.854 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:26.854 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6819", 2022-06-29T15:52:26.854 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:52:26.854 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:26.855 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:26.855 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.855 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:52:26.855 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:26.856 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.856 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:26.856 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6822", 2022-06-29T15:52:26.856 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:52:26.857 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.857 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.857 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:26.857 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6823", 2022-06-29T15:52:26.858 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:52:26.858 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:26.858 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:26.858 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.859 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:52:26.859 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:26.859 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.859 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:26.859 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6820", 2022-06-29T15:52:26.860 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:52:26.860 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.860 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.860 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:26.861 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6821", 2022-06-29T15:52:26.861 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13396 2022-06-29T15:52:26.861 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:26.861 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:26.862 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.862 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6817/13396", 2022-06-29T15:52:26.862 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6819/13396", 2022-06-29T15:52:26.862 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6823/13396", 2022-06-29T15:52:26.863 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6821/13396", 2022-06-29T15:52:26.863 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:52:26.863 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:52:26.863 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:52:26.864 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:26.864 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.864 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.864 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2022-06-29T15:52:26.864 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "f77beebe-6cca-4244-b2d4-677825ebf919", 2022-06-29T15:52:26.865 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:52:26.865 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:52:26.865 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:52:26.865 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:52:26.866 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:52:26.866 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:52:26.866 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:52:26.866 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 160, 2022-06-29T15:52:26.867 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:52:26.867 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:52:26.867 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:52:26.867 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:26.868 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.868 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:26.868 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6800", 2022-06-29T15:52:26.868 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:52:26.869 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.869 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.869 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:26.869 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6801", 2022-06-29T15:52:26.870 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:52:26.870 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:26.870 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:26.870 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.871 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:52:26.871 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:26.871 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.871 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:26.871 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6802", 2022-06-29T15:52:26.872 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:52:26.872 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.872 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.872 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:26.873 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6803", 2022-06-29T15:52:26.873 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:52:26.873 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:26.873 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:26.874 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.874 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:52:26.874 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:26.874 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.875 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:26.875 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6806", 2022-06-29T15:52:26.875 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:52:26.876 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.876 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.876 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:26.876 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6807", 2022-06-29T15:52:26.877 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:52:26.877 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:26.877 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:26.878 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.878 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:52:26.879 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:26.879 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.879 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:26.879 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6804", 2022-06-29T15:52:26.880 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:52:26.880 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.880 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.880 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:26.881 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6805", 2022-06-29T15:52:26.881 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13393 2022-06-29T15:52:26.881 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:26.881 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:26.882 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.882 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6801/13393", 2022-06-29T15:52:26.882 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6803/13393", 2022-06-29T15:52:26.883 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6807/13393", 2022-06-29T15:52:26.883 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6805/13393", 2022-06-29T15:52:26.883 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:52:26.883 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:52:26.884 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:52:26.884 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:26.884 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.884 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.884 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2022-06-29T15:52:26.885 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "af0ca9b3-c02c-4910-834b-bf0b8dc6713c", 2022-06-29T15:52:26.885 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:52:26.885 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:52:26.886 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:52:26.886 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:52:26.886 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:52:26.886 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:52:26.887 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:52:26.887 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 160, 2022-06-29T15:52:26.887 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:52:26.888 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:52:26.888 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:52:26.888 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:26.888 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.889 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:26.889 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6824", 2022-06-29T15:52:26.889 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:52:26.889 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.890 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.890 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:26.890 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6825", 2022-06-29T15:52:26.890 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:52:26.891 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:26.891 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:26.891 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.891 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:52:26.892 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:26.892 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.892 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:26.892 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6826", 2022-06-29T15:52:26.893 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:52:26.893 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.893 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.894 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:26.894 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6827", 2022-06-29T15:52:26.894 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:52:26.895 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:26.895 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:26.895 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.895 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:52:26.896 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:26.896 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.897 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:26.897 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6830", 2022-06-29T15:52:26.897 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:52:26.897 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.898 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.898 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:26.899 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6831", 2022-06-29T15:52:26.899 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:52:26.899 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:26.900 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:26.900 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.900 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:52:26.901 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:26.901 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.902 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:26.902 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6828", 2022-06-29T15:52:26.902 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:52:26.903 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.903 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.904 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:26.904 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6829", 2022-06-29T15:52:26.904 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13395 2022-06-29T15:52:26.905 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:26.905 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:26.906 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.906 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6825/13395", 2022-06-29T15:52:26.906 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6827/13395", 2022-06-29T15:52:26.907 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6831/13395", 2022-06-29T15:52:26.907 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6829/13395", 2022-06-29T15:52:26.907 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:52:26.907 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:52:26.908 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:52:26.908 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:26.909 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.909 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.909 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2022-06-29T15:52:26.910 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "f19c3ca6-df88-4bde-a150-b51a53f092e1", 2022-06-29T15:52:26.910 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2022-06-29T15:52:26.910 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2022-06-29T15:52:26.911 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2022-06-29T15:52:26.911 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2022-06-29T15:52:26.912 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 0, 2022-06-29T15:52:26.912 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 0, 2022-06-29T15:52:26.912 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 14, 2022-06-29T15:52:26.913 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 160, 2022-06-29T15:52:26.913 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 0, 2022-06-29T15:52:26.914 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2022-06-29T15:52:26.914 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2022-06-29T15:52:26.915 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:26.915 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.916 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:26.916 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6808", 2022-06-29T15:52:26.917 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:52:26.917 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.917 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.918 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:26.918 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6809", 2022-06-29T15:52:26.919 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:52:26.919 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:26.919 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:26.920 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.920 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2022-06-29T15:52:26.920 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:26.920 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.921 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:26.921 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6810", 2022-06-29T15:52:26.921 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:52:26.921 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.922 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.922 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:26.922 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6811", 2022-06-29T15:52:26.922 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:52:26.923 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:26.923 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:26.923 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.923 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2022-06-29T15:52:26.923 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:26.924 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.924 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:26.924 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6814", 2022-06-29T15:52:26.924 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:52:26.925 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.925 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.925 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:26.925 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6815", 2022-06-29T15:52:26.925 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:52:26.926 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:26.926 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:26.926 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.926 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2022-06-29T15:52:26.927 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2022-06-29T15:52:26.927 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.927 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2022-06-29T15:52:26.927 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6812", 2022-06-29T15:52:26.927 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:52:26.928 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.928 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.928 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2022-06-29T15:52:26.928 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.64:6813", 2022-06-29T15:52:26.929 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 13394 2022-06-29T15:52:26.929 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:26.929 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:26.929 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.929 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.64:6809/13394", 2022-06-29T15:52:26.930 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.64:6811/13394", 2022-06-29T15:52:26.930 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.64:6815/13394", 2022-06-29T15:52:26.930 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.64:6813/13394", 2022-06-29T15:52:26.930 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2022-06-29T15:52:26.931 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2022-06-29T15:52:26.931 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2022-06-29T15:52:26.931 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2022-06-29T15:52:26.931 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:26.931 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:52:26.932 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_xinfo": [ 2022-06-29T15:52:26.932 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.932 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2022-06-29T15:52:26.932 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:52:26.933 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:52:26.933 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:52:26.933 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:52:26.933 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:52:26.933 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.720513+0000", 2022-06-29T15:52:26.934 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:52:26.934 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.934 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.934 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2022-06-29T15:52:26.935 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:52:26.935 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:52:26.935 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:52:26.935 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:52:26.935 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:52:26.936 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.772612+0000", 2022-06-29T15:52:26.936 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:52:26.936 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.937 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.937 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2022-06-29T15:52:26.937 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:52:26.937 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:52:26.938 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:52:26.938 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:52:26.938 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:52:26.938 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.450223+0000", 2022-06-29T15:52:26.939 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:52:26.939 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.939 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.939 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2022-06-29T15:52:26.940 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:52:26.940 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:52:26.940 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:52:26.940 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:52:26.940 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:52:26.941 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.477856+0000", 2022-06-29T15:52:26.941 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:52:26.941 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.941 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.942 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 4, 2022-06-29T15:52:26.942 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:52:26.942 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:52:26.942 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:52:26.943 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:52:26.943 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:52:26.943 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.539485+0000", 2022-06-29T15:52:26.943 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:52:26.943 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.944 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.944 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 5, 2022-06-29T15:52:26.944 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:52:26.944 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:52:26.945 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:52:26.945 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:52:26.945 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:52:26.945 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.487040+0000", 2022-06-29T15:52:26.946 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:52:26.946 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.946 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.946 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 6, 2022-06-29T15:52:26.947 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:52:26.947 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:52:26.947 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:52:26.947 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:52:26.947 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:52:26.948 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.748685+0000", 2022-06-29T15:52:26.948 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:52:26.948 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.948 INFO:teuthology.orchestra.run.smithi047.stdout: { 2022-06-29T15:52:26.949 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 7, 2022-06-29T15:52:26.949 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2022-06-29T15:52:26.949 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2022-06-29T15:52:26.949 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2022-06-29T15:52:26.950 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540138292840890367, 2022-06-29T15:52:26.950 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2022-06-29T15:52:26.950 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2022-06-29T15:42:19.459516+0000", 2022-06-29T15:52:26.950 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2022-06-29T15:52:26.950 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:26.951 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2022-06-29T15:52:26.951 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap": [], 2022-06-29T15:52:26.951 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap_items": [], 2022-06-29T15:52:26.951 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_temp": [], 2022-06-29T15:52:26.952 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_temp": [], 2022-06-29T15:52:26.952 INFO:teuthology.orchestra.run.smithi047.stdout: "blacklist": {}, 2022-06-29T15:52:26.952 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profiles": { 2022-06-29T15:52:26.952 INFO:teuthology.orchestra.run.smithi047.stdout: "cephfs_data": { 2022-06-29T15:52:26.952 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-device-class": "", 2022-06-29T15:52:26.953 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2022-06-29T15:52:26.953 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-root": "default", 2022-06-29T15:52:26.953 INFO:teuthology.orchestra.run.smithi047.stdout: "jerasure-per-chunk-alignment": "false", 2022-06-29T15:52:26.953 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2022-06-29T15:52:26.954 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "2", 2022-06-29T15:52:26.954 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2022-06-29T15:52:26.954 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2022-06-29T15:52:26.954 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van", 2022-06-29T15:52:26.955 INFO:teuthology.orchestra.run.smithi047.stdout: "w": "8" 2022-06-29T15:52:26.955 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.955 INFO:teuthology.orchestra.run.smithi047.stdout: "default": { 2022-06-29T15:52:26.955 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2022-06-29T15:52:26.955 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2022-06-29T15:52:26.956 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "1", 2022-06-29T15:52:26.956 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2022-06-29T15:52:26.956 INFO:teuthology.orchestra.run.smithi047.stdout: "ruleset-failure-domain": "osd", 2022-06-29T15:52:26.956 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van" 2022-06-29T15:52:26.957 INFO:teuthology.orchestra.run.smithi047.stdout: } 2022-06-29T15:52:26.957 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2022-06-29T15:52:26.957 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps_queue": [], 2022-06-29T15:52:26.957 INFO:teuthology.orchestra.run.smithi047.stdout: "new_removed_snaps": [], 2022-06-29T15:52:26.958 INFO:teuthology.orchestra.run.smithi047.stdout: "new_purged_snaps": [], 2022-06-29T15:52:26.958 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_node_flags": {}, 2022-06-29T15:52:26.958 INFO:teuthology.orchestra.run.smithi047.stdout: "device_class_flags": {} 2022-06-29T15:52:26.958 INFO:teuthology.orchestra.run.smithi047.stdout:} 2022-06-29T15:52:26.960 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.649+0000 7effd5e9c700 1 -- 172.21.15.47:0/1441047835 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7effa8053b00 msgr2=0x7effa8055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:26.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.649+0000 7effd5e9c700 1 --2- 172.21.15.47:0/1441047835 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7effa8053b00 0x7effa8055fc0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7effc001c000 tx=0x7effc0015000).stop 2022-06-29T15:52:26.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.649+0000 7effd5e9c700 1 -- 172.21.15.47:0/1441047835 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effd013b910 msgr2=0x7effd0155f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:26.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.649+0000 7effd5e9c700 1 --2- 172.21.15.47:0/1441047835 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effd013b910 0x7effd0155f20 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 rx=0x7effc8006e00 tx=0x7effc80091d0).stop 2022-06-29T15:52:26.961 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.649+0000 7effd5e9c700 1 -- 172.21.15.47:0/1441047835 shutdown_connections 2022-06-29T15:52:26.962 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.649+0000 7effd5e9c700 1 --2- 172.21.15.47:0/1441047835 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effd013b910 0x7effd0155f20 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:26.962 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.649+0000 7effd5e9c700 1 --2- 172.21.15.47:0/1441047835 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7effa8053b00 0x7effa8055fc0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:26.962 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.649+0000 7effd5e9c700 1 --2- 172.21.15.47:0/1441047835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effd00b2c40 0x7effd013b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:26.962 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.649+0000 7effd5e9c700 1 --2- 172.21.15.47:0/1441047835 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7effd005c8f0 0x7effd0136920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:26.963 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.649+0000 7effd5e9c700 1 -- 172.21.15.47:0/1441047835 >> 172.21.15.47:0/1441047835 conn(0x7effd0060e00 msgr2=0x7effd00b4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:26.963 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.649+0000 7effd5e9c700 1 -- 172.21.15.47:0/1441047835 shutdown_connections 2022-06-29T15:52:26.963 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:26.649+0000 7effd5e9c700 1 -- 172.21.15.47:0/1441047835 wait complete. 2022-06-29T15:52:26.964 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:52:27.076 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.069+0000 7fb7d1b90700 1 Processor -- start 2022-06-29T15:52:27.076 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.069+0000 7fb7d1b90700 1 -- start start 2022-06-29T15:52:27.079 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.069+0000 7fb7d1b90700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb7cc0b5c60 0x7fb7cc144980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:27.080 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.069+0000 7fb7d1b90700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb7cc060eb0 0x7fb7cc144e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:27.080 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.069+0000 7fb7d1b90700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7cc0a9120 con 0x7fb7cc0a72f0 2022-06-29T15:52:27.080 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.069+0000 7fb7d1b90700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7cc145360 con 0x7fb7cc060eb0 2022-06-29T15:52:27.081 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.069+0000 7fb7d1b90700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7cc0b0b80 con 0x7fb7cc0b5c60 2022-06-29T15:52:27.081 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.069+0000 7fb7cbfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb7cc060eb0 0x7fb7cc144e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:27.081 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.069+0000 7fb7d138f700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb7cc0a72f0 0x7fb7cc0b1e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40902/0 (socket says 172.21.15.47:40902) 2022-06-29T15:52:27.081 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.069+0000 7fb7d138f700 1 -- 172.21.15.47:0/247385655 learned_addr learned my addr 172.21.15.47:0/247385655 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:52:27.082 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.069+0000 7fb7d0b8e700 1 --2- 172.21.15.47:0/247385655 >> v2:172.21.15.64:3300/0 conn(0x7fb7cc0b5c60 0x7fb7cc144980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:27.082 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.069+0000 7fb7cbfff700 1 -- 172.21.15.47:0/247385655 >> v2:172.21.15.64:3300/0 conn(0x7fb7cc0b5c60 msgr2=0x7fb7cc144980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:27.082 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.069+0000 7fb7cbfff700 1 --2- 172.21.15.47:0/247385655 >> v2:172.21.15.64:3300/0 conn(0x7fb7cc0b5c60 0x7fb7cc144980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:27.082 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.069+0000 7fb7cb7fe700 1 -- 172.21.15.47:0/247385655 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 164228925 0 0) 0x7fb7cc0a9120 con 0x7fb7cc0a72f0 2022-06-29T15:52:27.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.069+0000 7fb7cbfff700 1 -- 172.21.15.47:0/247385655 >> v1:172.21.15.47:6789/0 conn(0x7fb7cc0a72f0 legacy=0x7fb7cc0b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:27.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.069+0000 7fb7cbfff700 1 -- 172.21.15.47:0/247385655 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7cc0a89e0 con 0x7fb7cc060eb0 2022-06-29T15:52:27.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.069+0000 7fb7d0b8e700 1 --2- 172.21.15.47:0/247385655 >> v2:172.21.15.64:3300/0 conn(0x7fb7cc0b5c60 0x7fb7cc144980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T15:52:27.083 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.069+0000 7fb7cbfff700 1 --2- 172.21.15.47:0/247385655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7cc060eb0 0x7fb7cc144e70 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 rx=0x7fb7b4002f70 tx=0x7fb7b4006df0).ready entity=mon.0 client_cookie=7cc27b2e15422f29 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:27.084 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.069+0000 7fb7cb7fe700 1 -- 172.21.15.47:0/247385655 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb7b400d570 con 0x7fb7cc060eb0 2022-06-29T15:52:27.084 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.069+0000 7fb7cb7fe700 1 -- 172.21.15.47:0/247385655 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b4000e90 con 0x7fb7cc060eb0 2022-06-29T15:52:27.084 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.069+0000 7fb7cb7fe700 1 -- 172.21.15.47:0/247385655 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb7b400d8c0 con 0x7fb7cc060eb0 2022-06-29T15:52:27.084 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.069+0000 7fb7d1b90700 1 -- 172.21.15.47:0/247385655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7cc060eb0 msgr2=0x7fb7cc144e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:27.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.069+0000 7fb7d1b90700 1 --2- 172.21.15.47:0/247385655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7cc060eb0 0x7fb7cc144e70 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 rx=0x7fb7b4002f70 tx=0x7fb7b4006df0).stop 2022-06-29T15:52:27.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.069+0000 7fb7d1b90700 1 -- 172.21.15.47:0/247385655 shutdown_connections 2022-06-29T15:52:27.085 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.069+0000 7fb7d1b90700 1 --2- 172.21.15.47:0/247385655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7cc060eb0 0x7fb7cc144e70 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:27.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.069+0000 7fb7d1b90700 1 --2- 172.21.15.47:0/247385655 >> v2:172.21.15.64:3300/0 conn(0x7fb7cc0b5c60 0x7fb7cc144980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:27.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.069+0000 7fb7d1b90700 1 -- 172.21.15.47:0/247385655 >> 172.21.15.47:0/247385655 conn(0x7fb7cc062560 msgr2=0x7fb7cc062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:27.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.069+0000 7fb7d1b90700 1 -- 172.21.15.47:0/247385655 shutdown_connections 2022-06-29T15:52:27.086 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.073+0000 7fb7d1b90700 1 -- 172.21.15.47:0/247385655 wait complete. 2022-06-29T15:52:27.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.073+0000 7fb7d1b90700 1 Processor -- start 2022-06-29T15:52:27.087 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.073+0000 7fb7d1b90700 1 -- start start 2022-06-29T15:52:27.089 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.073+0000 7fb7d1b90700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb7cc060eb0 0x7fb7cc1403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:27.090 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.073+0000 7fb7d1b90700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7cc0a72f0 0x7fb7cc159d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:27.090 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.073+0000 7fb7d1b90700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7cc0b5c60 0x7fb7cc15a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:27.090 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.073+0000 7fb7d1b90700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7cc156800 con 0x7fb7cc0b5c60 2022-06-29T15:52:27.091 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.073+0000 7fb7d1b90700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7cc15a770 con 0x7fb7cc0a72f0 2022-06-29T15:52:27.091 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.073+0000 7fb7d1b90700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7cc1569f0 con 0x7fb7cc060eb0 2022-06-29T15:52:27.091 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.073+0000 7fb7d138f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7cc0b5c60 0x7fb7cc15a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:27.091 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.073+0000 7fb7d138f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7cc0b5c60 0x7fb7cc15a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55354/0 (socket says 172.21.15.47:55354) 2022-06-29T15:52:27.092 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.073+0000 7fb7d138f700 1 -- 172.21.15.47:0/2188441315 learned_addr learned my addr 172.21.15.47:0/2188441315 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:27.092 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.073+0000 7fb7cbfff700 1 --2- 172.21.15.47:0/2188441315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7cc0a72f0 0x7fb7cc159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:27.092 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.073+0000 7fb7d0b8e700 1 --2- 172.21.15.47:0/2188441315 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb7cc060eb0 0x7fb7cc1403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:27.092 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.073+0000 7fb7d138f700 1 -- 172.21.15.47:0/2188441315 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb7cc060eb0 msgr2=0x7fb7cc1403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:27.092 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.073+0000 7fb7d138f700 1 --2- 172.21.15.47:0/2188441315 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb7cc060eb0 0x7fb7cc1403c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:27.093 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.073+0000 7fb7d138f700 1 -- 172.21.15.47:0/2188441315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7cc0a72f0 msgr2=0x7fb7cc159d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:27.093 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.073+0000 7fb7d138f700 1 --2- 172.21.15.47:0/2188441315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7cc0a72f0 0x7fb7cc159d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:27.093 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.073+0000 7fb7d138f700 1 -- 172.21.15.47:0/2188441315 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c4009420 con 0x7fb7cc0b5c60 2022-06-29T15:52:27.093 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.073+0000 7fb7d138f700 1 --2- 172.21.15.47:0/2188441315 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7cc0b5c60 0x7fb7cc15a280 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 rx=0x7fb7bc000bc0 tx=0x7fb7bc007a00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:27.094 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.073+0000 7fb7c9ffb700 1 -- 172.21.15.47:0/2188441315 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb7bc0070d0 con 0x7fb7cc0b5c60 2022-06-29T15:52:27.094 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.073+0000 7fb7c9ffb700 1 -- 172.21.15.47:0/2188441315 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7bc007230 con 0x7fb7cc0b5c60 2022-06-29T15:52:27.094 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.073+0000 7fb7d1b90700 1 -- 172.21.15.47:0/2188441315 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7b400b180 con 0x7fb7cc0b5c60 2022-06-29T15:52:27.094 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.073+0000 7fb7c9ffb700 1 -- 172.21.15.47:0/2188441315 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb7bc005610 con 0x7fb7cc0b5c60 2022-06-29T15:52:27.095 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.073+0000 7fb7d1b90700 1 -- 172.21.15.47:0/2188441315 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7cc1571d0 con 0x7fb7cc0b5c60 2022-06-29T15:52:27.095 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.077+0000 7fb7d1b90700 1 -- 172.21.15.47:0/2188441315 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7cc156da0 con 0x7fb7cc0b5c60 2022-06-29T15:52:27.095 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.077+0000 7fb7c9ffb700 1 -- 172.21.15.47:0/2188441315 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb7bc01a060 con 0x7fb7cc0b5c60 2022-06-29T15:52:27.095 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.077+0000 7fb7c9ffb700 1 --2- 172.21.15.47:0/2188441315 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb7b0053aa0 0x7fb7b0055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:27.096 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.077+0000 7fb7d0b8e700 1 --2- 172.21.15.47:0/2188441315 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb7b0053aa0 0x7fb7b0055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:27.096 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.077+0000 7fb7c9ffb700 1 -- 172.21.15.47:0/2188441315 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb7bc06b680 con 0x7fb7cc0b5c60 2022-06-29T15:52:27.096 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.077+0000 7fb7d0b8e700 1 --2- 172.21.15.47:0/2188441315 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb7b0053aa0 0x7fb7b0055f60 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7fb7c400b000 tx=0x7fb7c40091a0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:27.097 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.085+0000 7fb7c9ffb700 1 -- 172.21.15.47:0/2188441315 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb7bc042440 con 0x7fb7cc0b5c60 2022-06-29T15:52:27.254 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.245+0000 7fb7d1b90700 1 -- 172.21.15.47:0/2188441315 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb7cc156b30 con 0x7fb7cc0b5c60 2022-06-29T15:52:27.255 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.249+0000 7fb7c9ffb700 1 -- 172.21.15.47:0/2188441315 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+2495 (secure 0 0 0) 0x7fb7bc018070 con 0x7fb7cc0b5c60 2022-06-29T15:52:27.256 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:52:27.256 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":79,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8596,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6836/1456033193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1456033193},{"type":"v1","addr":"172.21.15.47:6836","nonce":1456033193}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":77},{"gid":8745,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/1940204812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6835","nonce":1940204812},{"type":"v1","addr":"172.21.15.47:6837","nonce":1940204812}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":77}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:52:17.031576+0000","modified":"2022-06-29T15:52:18.034584+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":57377},"failed":[],"damaged":[],"stopped":[],"info":{"gid_57377":{"gid":57377,"name":"b","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6833/642573447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":642573447},{"type":"v1","addr":"172.21.15.64:6833","nonce":642573447}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2022-06-29T15:52:27.258 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.249+0000 7fb7d1b90700 1 -- 172.21.15.47:0/2188441315 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb7b0053aa0 msgr2=0x7fb7b0055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:27.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.249+0000 7fb7d1b90700 1 --2- 172.21.15.47:0/2188441315 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb7b0053aa0 0x7fb7b0055f60 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7fb7c400b000 tx=0x7fb7c40091a0).stop 2022-06-29T15:52:27.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.249+0000 7fb7d1b90700 1 -- 172.21.15.47:0/2188441315 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7cc0b5c60 msgr2=0x7fb7cc15a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:27.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.249+0000 7fb7d1b90700 1 --2- 172.21.15.47:0/2188441315 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7cc0b5c60 0x7fb7cc15a280 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 rx=0x7fb7bc000bc0 tx=0x7fb7bc007a00).stop 2022-06-29T15:52:27.259 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.249+0000 7fb7d1b90700 1 -- 172.21.15.47:0/2188441315 shutdown_connections 2022-06-29T15:52:27.260 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.249+0000 7fb7d1b90700 1 --2- 172.21.15.47:0/2188441315 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7cc0b5c60 0x7fb7cc15a280 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:27.260 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.249+0000 7fb7d1b90700 1 --2- 172.21.15.47:0/2188441315 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb7b0053aa0 0x7fb7b0055f60 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:27.260 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.249+0000 7fb7d1b90700 1 --2- 172.21.15.47:0/2188441315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7cc0a72f0 0x7fb7cc159d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:27.260 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.249+0000 7fb7d1b90700 1 --2- 172.21.15.47:0/2188441315 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb7cc060eb0 0x7fb7cc1403c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:27.261 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.249+0000 7fb7d1b90700 1 -- 172.21.15.47:0/2188441315 >> 172.21.15.47:0/2188441315 conn(0x7fb7cc062560 msgr2=0x7fb7cc0b4910 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:27.261 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.249+0000 7fb7d1b90700 1 -- 172.21.15.47:0/2188441315 shutdown_connections 2022-06-29T15:52:27.261 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.249+0000 7fb7d1b90700 1 -- 172.21.15.47:0/2188441315 wait complete. 2022-06-29T15:52:27.262 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 79 2022-06-29T15:52:27.271 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-06-29T15:52:27.380 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.373+0000 7f78912a5700 1 Processor -- start 2022-06-29T15:52:27.380 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.373+0000 7f78912a5700 1 -- start start 2022-06-29T15:52:27.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.373+0000 7f78912a5700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f788c0b5c60 0x7f788c144980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:27.381 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.373+0000 7f78912a5700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f788c0a72f0 0x7f788c0b2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:27.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.373+0000 7f78912a5700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f788c0a9120 con 0x7f788c060eb0 2022-06-29T15:52:27.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.373+0000 7f78912a5700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f788c0b0b80 con 0x7f788c0a72f0 2022-06-29T15:52:27.382 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.373+0000 7f78912a5700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f788c0a89e0 con 0x7f788c0b5c60 2022-06-29T15:52:27.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.373+0000 7f788bfff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f788c0b5c60 0x7f788c144980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:27.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.373+0000 7f788bfff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f788c0b5c60 0x7f788c144980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:52460/0 (socket says 172.21.15.47:52460) 2022-06-29T15:52:27.384 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.373+0000 7f788bfff700 1 -- 172.21.15.47:0/4280169908 learned_addr learned my addr 172.21.15.47:0/4280169908 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:27.385 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.373+0000 7f7890aa4700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f788c0a72f0 0x7f788c0b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:27.385 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.373+0000 7f788bfff700 1 -- 172.21.15.47:0/4280169908 >> v2:172.21.15.64:3300/0 conn(0x7f788c0a72f0 msgr2=0x7f788c0b2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:27.385 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.373+0000 7f788bfff700 1 --2- 172.21.15.47:0/4280169908 >> v2:172.21.15.64:3300/0 conn(0x7f788c0a72f0 0x7f788c0b2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:27.385 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.373+0000 7f788bfff700 1 -- 172.21.15.47:0/4280169908 >> v1:172.21.15.64:6789/0 conn(0x7f788c060eb0 legacy=0x7f788c0b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:27.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.373+0000 7f788bfff700 1 -- 172.21.15.47:0/4280169908 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f788c0a8c80 con 0x7f788c0b5c60 2022-06-29T15:52:27.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.373+0000 7f788bfff700 1 --2- 172.21.15.47:0/4280169908 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f788c0b5c60 0x7f788c144980 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f7874002f70 tx=0x7f7874006e50).ready entity=mon.2 client_cookie=d1778be8b929d20e server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:27.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.377+0000 7f788affd700 1 -- 172.21.15.47:0/4280169908 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f787400d5c0 con 0x7f788c0b5c60 2022-06-29T15:52:27.386 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.377+0000 7f788affd700 1 -- 172.21.15.47:0/4280169908 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7874000e90 con 0x7f788c0b5c60 2022-06-29T15:52:27.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.377+0000 7f788affd700 1 -- 172.21.15.47:0/4280169908 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f787400d950 con 0x7f788c0b5c60 2022-06-29T15:52:27.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.377+0000 7f78912a5700 1 -- 172.21.15.47:0/4280169908 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f788c0b5c60 msgr2=0x7f788c144980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:27.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.377+0000 7f78912a5700 1 --2- 172.21.15.47:0/4280169908 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f788c0b5c60 0x7f788c144980 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f7874002f70 tx=0x7f7874006e50).stop 2022-06-29T15:52:27.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.377+0000 7f78912a5700 1 -- 172.21.15.47:0/4280169908 shutdown_connections 2022-06-29T15:52:27.387 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.377+0000 7f78912a5700 1 --2- 172.21.15.47:0/4280169908 >> v2:172.21.15.64:3300/0 conn(0x7f788c0a72f0 0x7f788c0b2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:27.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.377+0000 7f78912a5700 1 --2- 172.21.15.47:0/4280169908 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f788c0b5c60 0x7f788c144980 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:27.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.377+0000 7f78912a5700 1 -- 172.21.15.47:0/4280169908 >> 172.21.15.47:0/4280169908 conn(0x7f788c062560 msgr2=0x7f788c062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:27.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.377+0000 7f78912a5700 1 -- 172.21.15.47:0/4280169908 shutdown_connections 2022-06-29T15:52:27.388 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.377+0000 7f78912a5700 1 -- 172.21.15.47:0/4280169908 wait complete. 2022-06-29T15:52:27.391 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.377+0000 7f78912a5700 1 Processor -- start 2022-06-29T15:52:27.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.377+0000 7f78912a5700 1 -- start start 2022-06-29T15:52:27.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.381+0000 7f78912a5700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f788c060eb0 0x7f788c0b35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:27.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.381+0000 7f78912a5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f788c0a72f0 0x7f788c0ad190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:27.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.381+0000 7f78912a5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f788c0b5c60 0x7f788c159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:27.392 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.381+0000 7f78912a5700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788c0ac990 con 0x7f788c0b5c60 2022-06-29T15:52:27.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.381+0000 7f78912a5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788c148170 con 0x7f788c0a72f0 2022-06-29T15:52:27.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.381+0000 7f78912a5700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f788c148410 con 0x7f788c060eb0 2022-06-29T15:52:27.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.381+0000 7f7890aa4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f788c0b5c60 0x7f788c159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:27.393 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.381+0000 7f7890aa4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f788c0b5c60 0x7f788c159ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55368/0 (socket says 172.21.15.47:55368) 2022-06-29T15:52:27.394 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.381+0000 7f7890aa4700 1 -- 172.21.15.47:0/154977849 learned_addr learned my addr 172.21.15.47:0/154977849 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:27.394 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.381+0000 7f788bfff700 1 --2- 172.21.15.47:0/154977849 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f788c060eb0 0x7f788c0b35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:27.394 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.381+0000 7f788b7fe700 1 --2- 172.21.15.47:0/154977849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f788c0a72f0 0x7f788c0ad190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:27.394 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.381+0000 7f7890aa4700 1 -- 172.21.15.47:0/154977849 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f788c060eb0 msgr2=0x7f788c0b35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:27.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.381+0000 7f7890aa4700 1 --2- 172.21.15.47:0/154977849 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f788c060eb0 0x7f788c0b35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:27.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.381+0000 7f7890aa4700 1 -- 172.21.15.47:0/154977849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f788c0a72f0 msgr2=0x7f788c0ad190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:27.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.381+0000 7f7890aa4700 1 --2- 172.21.15.47:0/154977849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f788c0a72f0 0x7f788c0ad190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:27.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.381+0000 7f7890aa4700 1 -- 172.21.15.47:0/154977849 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f787400b1e0 con 0x7f788c0b5c60 2022-06-29T15:52:27.395 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.381+0000 7f788b7fe700 1 --2- 172.21.15.47:0/154977849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f788c0a72f0 0x7f788c0ad190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T15:52:27.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.381+0000 7f7890aa4700 1 --2- 172.21.15.47:0/154977849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f788c0b5c60 0x7f788c159ce0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 rx=0x7f787c00c000 tx=0x7f787c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:27.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.381+0000 7f78897fa700 1 -- 172.21.15.47:0/154977849 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f787c011750 con 0x7f788c0b5c60 2022-06-29T15:52:27.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.381+0000 7f78897fa700 1 -- 172.21.15.47:0/154977849 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f787c0118b0 con 0x7f788c0b5c60 2022-06-29T15:52:27.396 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.381+0000 7f78912a5700 1 -- 172.21.15.47:0/154977849 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f788c143fa0 con 0x7f788c0b5c60 2022-06-29T15:52:27.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.381+0000 7f78897fa700 1 -- 172.21.15.47:0/154977849 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f787c011c10 con 0x7f788c0b5c60 2022-06-29T15:52:27.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.381+0000 7f78912a5700 1 -- 172.21.15.47:0/154977849 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f788c0aea50 con 0x7f788c0b5c60 2022-06-29T15:52:27.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.381+0000 7f78912a5700 1 -- 172.21.15.47:0/154977849 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f788c147fd0 con 0x7f788c0b5c60 2022-06-29T15:52:27.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.381+0000 7f78897fa700 1 -- 172.21.15.47:0/154977849 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f787c011d70 con 0x7f788c0b5c60 2022-06-29T15:52:27.397 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.385+0000 7f78897fa700 1 --2- 172.21.15.47:0/154977849 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7864053a70 0x7f7864055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:27.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.385+0000 7f788bfff700 1 --2- 172.21.15.47:0/154977849 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7864053a70 0x7f7864055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:27.398 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.385+0000 7f78897fa700 1 -- 172.21.15.47:0/154977849 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f787c06c530 con 0x7f788c0b5c60 2022-06-29T15:52:27.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.385+0000 7f788bfff700 1 --2- 172.21.15.47:0/154977849 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7864053a70 0x7f7864055f30 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7f787401a000 tx=0x7f7874014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:27.399 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.389+0000 7f78897fa700 1 -- 172.21.15.47:0/154977849 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f788c147fd0 con 0x7f788c0b5c60 2022-06-29T15:52:27.557 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:27.549+0000 7f78912a5700 1 -- 172.21.15.47:0/154977849 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f788c147fd0 con 0x7f788c0b5c60 2022-06-29T15:52:27.994 INFO:tasks.ceph.mon.a.smithi047.stderr:2022-06-29T15:52:27.985+0000 7ff9cf83c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-06-29T15:52:28.008 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.001+0000 7f78897fa700 1 -- 172.21.15.47:0/154977849 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v80) v1 ==== 161+0+0 (secure 0 0 0) 0x7f788c147fd0 con 0x7f788c0b5c60 2022-06-29T15:52:28.013 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.005+0000 7f78912a5700 1 -- 172.21.15.47:0/154977849 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7864053a70 msgr2=0x7f7864055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:28.013 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.005+0000 7f78912a5700 1 --2- 172.21.15.47:0/154977849 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7864053a70 0x7f7864055f30 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7f787401a000 tx=0x7f7874014000).stop 2022-06-29T15:52:28.013 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.005+0000 7f78912a5700 1 -- 172.21.15.47:0/154977849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f788c0b5c60 msgr2=0x7f788c159ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:28.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.005+0000 7f78912a5700 1 --2- 172.21.15.47:0/154977849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f788c0b5c60 0x7f788c159ce0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 rx=0x7f787c00c000 tx=0x7f787c006df0).stop 2022-06-29T15:52:28.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.005+0000 7f78912a5700 1 -- 172.21.15.47:0/154977849 shutdown_connections 2022-06-29T15:52:28.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.005+0000 7f78912a5700 1 --2- 172.21.15.47:0/154977849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f788c0b5c60 0x7f788c159ce0 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:28.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.005+0000 7f78912a5700 1 --2- 172.21.15.47:0/154977849 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7864053a70 0x7f7864055f30 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:28.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.005+0000 7f78912a5700 1 --2- 172.21.15.47:0/154977849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f788c0a72f0 0x7f788c0ad190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:28.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.005+0000 7f78912a5700 1 --2- 172.21.15.47:0/154977849 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f788c060eb0 0x7f788c0b35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:28.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.005+0000 7f78912a5700 1 -- 172.21.15.47:0/154977849 >> 172.21.15.47:0/154977849 conn(0x7f788c062560 msgr2=0x7f788c14d680 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:28.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.005+0000 7f78912a5700 1 -- 172.21.15.47:0/154977849 shutdown_connections 2022-06-29T15:52:28.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.005+0000 7f78912a5700 1 -- 172.21.15.47:0/154977849 wait complete. 2022-06-29T15:52:28.016 INFO:teuthology.orchestra.run.smithi047.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-06-29T15:52:28.028 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-06-29T15:52:28.034 INFO:tasks.ceph.mds.b.smithi064.stdout:starting mds.b at 2022-06-29T15:52:28.135 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.129+0000 7f30e5353700 1 Processor -- start 2022-06-29T15:52:28.136 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.129+0000 7f30e5353700 1 -- start start 2022-06-29T15:52:28.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.129+0000 7f30e5353700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f30e005c510 0x7f30e0143bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:28.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.129+0000 7f30e5353700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f30e005c8f0 0x7f30e00b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:28.137 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.129+0000 7f30e5353700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f30e00b2c40 0x7f30e00b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:28.138 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.129+0000 7f30e5353700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30e0142f90 con 0x7f30e00b2c40 2022-06-29T15:52:28.138 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.129+0000 7f30e5353700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30e01431f0 con 0x7f30e005c8f0 2022-06-29T15:52:28.138 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.129+0000 7f30e5353700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30e0143490 con 0x7f30e005c510 2022-06-29T15:52:28.138 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.129+0000 7f30e4b52700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f30e00b2c40 0x7f30e00b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:28.139 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.129+0000 7f30e4b52700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f30e00b2c40 0x7f30e00b30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55376/0 (socket says 172.21.15.47:55376) 2022-06-29T15:52:28.139 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.129+0000 7f30df7fe700 1 --2- 172.21.15.47:0/1617677872 >> v2:172.21.15.64:3300/0 conn(0x7f30e005c8f0 0x7f30e00b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:28.139 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.129+0000 7f30e4b52700 1 -- 172.21.15.47:0/1617677872 learned_addr learned my addr 172.21.15.47:0/1617677872 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:28.139 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.129+0000 7f30dffff700 1 --2- 172.21.15.47:0/1617677872 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f30e005c510 0x7f30e0143bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:28.141 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.129+0000 7f30e4b52700 1 -- 172.21.15.47:0/1617677872 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f30e005c510 msgr2=0x7f30e0143bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:28.141 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.129+0000 7f30e4b52700 1 --2- 172.21.15.47:0/1617677872 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f30e005c510 0x7f30e0143bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:28.142 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.129+0000 7f30e4b52700 1 -- 172.21.15.47:0/1617677872 >> v2:172.21.15.64:3300/0 conn(0x7f30e005c8f0 msgr2=0x7f30e00b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:28.142 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.129+0000 7f30e4b52700 1 --2- 172.21.15.47:0/1617677872 >> v2:172.21.15.64:3300/0 conn(0x7f30e005c8f0 0x7f30e00b4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:28.142 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.129+0000 7f30e4b52700 1 -- 172.21.15.47:0/1617677872 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30e0143730 con 0x7f30e00b2c40 2022-06-29T15:52:28.143 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.129+0000 7f30e4b52700 1 --2- 172.21.15.47:0/1617677872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30e00b2c40 0x7f30e00b30d0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 rx=0x7f30d8002f70 tx=0x7f30d8006eb0).ready entity=mon.0 client_cookie=26953c3084f0e265 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:28.143 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.129+0000 7f30deffd700 1 -- 172.21.15.47:0/1617677872 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f30d800d570 con 0x7f30e00b2c40 2022-06-29T15:52:28.143 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.129+0000 7f30deffd700 1 -- 172.21.15.47:0/1617677872 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30d8000e90 con 0x7f30e00b2c40 2022-06-29T15:52:28.144 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.129+0000 7f30deffd700 1 -- 172.21.15.47:0/1617677872 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f30d800d900 con 0x7f30e00b2c40 2022-06-29T15:52:28.144 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.133+0000 7f30e5353700 1 -- 172.21.15.47:0/1617677872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30e00b2c40 msgr2=0x7f30e00b30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:28.144 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.133+0000 7f30e5353700 1 --2- 172.21.15.47:0/1617677872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30e00b2c40 0x7f30e00b30d0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 rx=0x7f30d8002f70 tx=0x7f30d8006eb0).stop 2022-06-29T15:52:28.144 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.133+0000 7f30e5353700 1 -- 172.21.15.47:0/1617677872 shutdown_connections 2022-06-29T15:52:28.145 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.133+0000 7f30e5353700 1 --2- 172.21.15.47:0/1617677872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30e00b2c40 0x7f30e00b30d0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:28.145 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.133+0000 7f30e5353700 1 --2- 172.21.15.47:0/1617677872 >> v2:172.21.15.64:3300/0 conn(0x7f30e005c8f0 0x7f30e00b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:28.145 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.133+0000 7f30e5353700 1 --2- 172.21.15.47:0/1617677872 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f30e005c510 0x7f30e0143bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:28.145 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.133+0000 7f30e5353700 1 -- 172.21.15.47:0/1617677872 >> 172.21.15.47:0/1617677872 conn(0x7f30e0060e00 msgr2=0x7f30e00611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:28.145 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.133+0000 7f30e5353700 1 -- 172.21.15.47:0/1617677872 shutdown_connections 2022-06-29T15:52:28.146 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.133+0000 7f30e5353700 1 -- 172.21.15.47:0/1617677872 wait complete. 2022-06-29T15:52:28.148 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.133+0000 7f30e5353700 1 Processor -- start 2022-06-29T15:52:28.148 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.133+0000 7f30e5353700 1 -- start start 2022-06-29T15:52:28.149 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.137+0000 7f30e5353700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f30e005c510 0x7f30e00b5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:28.149 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.137+0000 7f30e5353700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30e005c8f0 0x7f30e00afb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:28.149 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.137+0000 7f30e5353700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30e00b2c40 0x7f30e0155a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:28.149 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.137+0000 7f30e5353700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e01555f0 con 0x7f30e00b2c40 2022-06-29T15:52:28.150 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.137+0000 7f30e5353700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e00aef20 con 0x7f30e005c8f0 2022-06-29T15:52:28.150 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.137+0000 7f30e5353700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30e00af1c0 con 0x7f30e005c510 2022-06-29T15:52:28.150 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.137+0000 7f30e4b52700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30e00b2c40 0x7f30e0155a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:28.150 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.137+0000 7f30e4b52700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30e00b2c40 0x7f30e0155a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55382/0 (socket says 172.21.15.47:55382) 2022-06-29T15:52:28.151 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.137+0000 7f30e4b52700 1 -- 172.21.15.47:0/59693777 learned_addr learned my addr 172.21.15.47:0/59693777 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:28.151 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.137+0000 7f30dffff700 1 --2- 172.21.15.47:0/59693777 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f30e005c510 0x7f30e00b5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:28.151 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.137+0000 7f30df7fe700 1 --2- 172.21.15.47:0/59693777 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30e005c8f0 0x7f30e00afb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:28.151 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.137+0000 7f30e4b52700 1 -- 172.21.15.47:0/59693777 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f30e005c510 msgr2=0x7f30e00b5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:28.151 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.137+0000 7f30e4b52700 1 --2- 172.21.15.47:0/59693777 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f30e005c510 0x7f30e00b5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:28.152 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.137+0000 7f30e4b52700 1 -- 172.21.15.47:0/59693777 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30e005c8f0 msgr2=0x7f30e00afb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:28.152 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.137+0000 7f30e4b52700 1 --2- 172.21.15.47:0/59693777 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30e005c8f0 0x7f30e00afb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:28.152 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.137+0000 7f30e4b52700 1 -- 172.21.15.47:0/59693777 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d800b1e0 con 0x7f30e00b2c40 2022-06-29T15:52:28.152 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.137+0000 7f30e4b52700 1 --2- 172.21.15.47:0/59693777 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30e00b2c40 0x7f30e0155a50 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 rx=0x7f30d8002f70 tx=0x7f30d8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:28.153 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.137+0000 7f30dd7fa700 1 -- 172.21.15.47:0/59693777 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f30d8011760 con 0x7f30e00b2c40 2022-06-29T15:52:28.153 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.137+0000 7f30dd7fa700 1 -- 172.21.15.47:0/59693777 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30d80118c0 con 0x7f30e00b2c40 2022-06-29T15:52:28.153 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.137+0000 7f30e5353700 1 -- 172.21.15.47:0/59693777 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30e00af460 con 0x7f30e00b2c40 2022-06-29T15:52:28.153 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.137+0000 7f30dd7fa700 1 -- 172.21.15.47:0/59693777 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f30d8011c20 con 0x7f30e00b2c40 2022-06-29T15:52:28.154 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.137+0000 7f30e5353700 1 -- 172.21.15.47:0/59693777 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30e00b0000 con 0x7f30e00b2c40 2022-06-29T15:52:28.154 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.141+0000 7f30e5353700 1 -- 172.21.15.47:0/59693777 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30e0155760 con 0x7f30e00b2c40 2022-06-29T15:52:28.154 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.141+0000 7f30dd7fa700 1 -- 172.21.15.47:0/59693777 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f30d8025030 con 0x7f30e00b2c40 2022-06-29T15:52:28.155 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.141+0000 7f30dd7fa700 1 --2- 172.21.15.47:0/59693777 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f30c0053950 0x7f30c0055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:28.155 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.141+0000 7f30dd7fa700 1 -- 172.21.15.47:0/59693777 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f30d806d380 con 0x7f30e00b2c40 2022-06-29T15:52:28.156 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.141+0000 7f30dffff700 1 --2- 172.21.15.47:0/59693777 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f30c0053950 0x7f30c0055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:28.156 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.141+0000 7f30dffff700 1 --2- 172.21.15.47:0/59693777 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f30c0053950 0x7f30c0055e10 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7f30d000b000 tx=0x7f30d00091d0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:28.156 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.149+0000 7f30dd7fa700 1 -- 172.21.15.47:0/59693777 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f30d8043fe0 con 0x7f30e00b2c40 2022-06-29T15:52:28.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.313+0000 7f30e5353700 1 -- 172.21.15.47:0/59693777 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f30e0061b00 con 0x7f30e00b2c40 2022-06-29T15:52:28.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.313+0000 7f30dd7fa700 1 -- 172.21.15.47:0/59693777 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+2135 (secure 0 0 0) 0x7f30d8023090 con 0x7f30e00b2c40 2022-06-29T15:52:28.323 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T15:52:28.323 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":80,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8596,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6836/1456033193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1456033193},{"type":"v1","addr":"172.21.15.47:6836","nonce":1456033193}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":77},{"gid":8745,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/1940204812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6835","nonce":1940204812},{"type":"v1","addr":"172.21.15.47:6837","nonce":1940204812}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":77}],"filesystems":[{"mdsmap":{"epoch":80,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-06-29T15:52:17.031576+0000","modified":"2022-06-29T15:52:27.991719+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":163,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2022-06-29T15:52:28.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.317+0000 7f30e5353700 1 -- 172.21.15.47:0/59693777 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f30c0053950 msgr2=0x7f30c0055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:28.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.317+0000 7f30e5353700 1 --2- 172.21.15.47:0/59693777 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f30c0053950 0x7f30c0055e10 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7f30d000b000 tx=0x7f30d00091d0).stop 2022-06-29T15:52:28.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.317+0000 7f30e5353700 1 -- 172.21.15.47:0/59693777 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30e00b2c40 msgr2=0x7f30e0155a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:28.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.317+0000 7f30e5353700 1 --2- 172.21.15.47:0/59693777 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30e00b2c40 0x7f30e0155a50 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 rx=0x7f30d8002f70 tx=0x7f30d8015000).stop 2022-06-29T15:52:28.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.317+0000 7f30e5353700 1 -- 172.21.15.47:0/59693777 shutdown_connections 2022-06-29T15:52:28.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.317+0000 7f30e5353700 1 --2- 172.21.15.47:0/59693777 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30e00b2c40 0x7f30e0155a50 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:28.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.317+0000 7f30e5353700 1 --2- 172.21.15.47:0/59693777 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f30c0053950 0x7f30c0055e10 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:28.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.317+0000 7f30e5353700 1 --2- 172.21.15.47:0/59693777 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30e005c8f0 0x7f30e00afb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:28.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.317+0000 7f30e5353700 1 --2- 172.21.15.47:0/59693777 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f30e005c510 0x7f30e00b5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:28.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.317+0000 7f30e5353700 1 -- 172.21.15.47:0/59693777 >> 172.21.15.47:0/59693777 conn(0x7f30e0060e00 msgr2=0x7f30e0140660 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:28.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.317+0000 7f30e5353700 1 -- 172.21.15.47:0/59693777 shutdown_connections 2022-06-29T15:52:28.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.317+0000 7f30e5353700 1 -- 172.21.15.47:0/59693777 wait complete. 2022-06-29T15:52:28.329 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 80 2022-06-29T15:52:28.336 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-06-29T15:52:28.447 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.437+0000 7f75d231e700 1 Processor -- start 2022-06-29T15:52:28.447 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.437+0000 7f75d231e700 1 -- start start 2022-06-29T15:52:28.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.441+0000 7f75d231e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f75cc0b58d0 0x7f75cc0b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:28.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.441+0000 7f75d231e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f75cc147160 0x7f75cc0aa130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:28.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.441+0000 7f75d231e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75cc0b1380 con 0x7f75cc0609e0 2022-06-29T15:52:28.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.441+0000 7f75d231e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75cc1467c0 con 0x7f75cc0b58d0 2022-06-29T15:52:28.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.441+0000 7f75d231e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc0b15b0 con 0x7f75cc147160 2022-06-29T15:52:28.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.441+0000 7f75d0b1b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f75cc0609e0 0x7f75cc1431f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40940/0 (socket says 172.21.15.47:40940) 2022-06-29T15:52:28.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.441+0000 7f75d0b1b700 1 -- 172.21.15.47:0/2309070504 learned_addr learned my addr 172.21.15.47:0/2309070504 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:52:28.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.441+0000 7f75d1b1d700 1 --2- 172.21.15.47:0/2309070504 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f75cc147160 0x7f75cc0aa130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:28.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.441+0000 7f75c3fff700 1 -- 172.21.15.47:0/2309070504 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1480638037 0 0) 0x7f75cc0b1380 con 0x7f75cc0609e0 2022-06-29T15:52:28.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.441+0000 7f75d131c700 1 --2- 172.21.15.47:0/2309070504 >> v2:172.21.15.64:3300/0 conn(0x7f75cc0b58d0 0x7f75cc0b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:28.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.441+0000 7f75c3fff700 1 -- 172.21.15.47:0/2309070504 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75b0003260 con 0x7f75cc0609e0 2022-06-29T15:52:28.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.441+0000 7f75c3fff700 1 -- 172.21.15.47:0/2309070504 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1787929081 0 0) 0x7f75b0003260 con 0x7f75cc0609e0 2022-06-29T15:52:28.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.441+0000 7f75c3fff700 1 -- 172.21.15.47:0/2309070504 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f75cc147160 msgr2=0x7f75cc0aa130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:28.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.441+0000 7f75c3fff700 1 --2- 172.21.15.47:0/2309070504 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f75cc147160 0x7f75cc0aa130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:28.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.441+0000 7f75c3fff700 1 -- 172.21.15.47:0/2309070504 >> v2:172.21.15.64:3300/0 conn(0x7f75cc0b58d0 msgr2=0x7f75cc0b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:28.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.441+0000 7f75c3fff700 1 --2- 172.21.15.47:0/2309070504 >> v2:172.21.15.64:3300/0 conn(0x7f75cc0b58d0 0x7f75cc0b5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:28.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.441+0000 7f75c3fff700 1 -- 172.21.15.47:0/2309070504 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75cc146a20 con 0x7f75cc0609e0 2022-06-29T15:52:28.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.441+0000 7f75c3fff700 1 -- 172.21.15.47:0/2309070504 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 989631641 0 0) 0x7f75c8001660 con 0x7f75cc0609e0 2022-06-29T15:52:28.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.441+0000 7f75c3fff700 1 -- 172.21.15.47:0/2309070504 >> v1:172.21.15.47:6789/0 conn(0x7f75cc0609e0 legacy=0x7f75cc1431f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:28.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.441+0000 7f75c3fff700 1 --2- 172.21.15.47:0/2309070504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75b0003750 0x7f75b0003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:28.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.441+0000 7f75c3fff700 1 --2- 172.21.15.47:0/2309070504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75b0004500 0x7f75b0004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:28.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.441+0000 7f75c3fff700 1 --2- 172.21.15.47:0/2309070504 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f75cc147160 0x7f75b0004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:28.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.441+0000 7f75c3fff700 1 -- 172.21.15.47:0/2309070504 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc1467c0 con 0x7f75b0004500 2022-06-29T15:52:28.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.441+0000 7f75c3fff700 1 -- 172.21.15.47:0/2309070504 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc0b15b0 con 0x7f75b0003750 2022-06-29T15:52:28.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.441+0000 7f75c3fff700 1 -- 172.21.15.47:0/2309070504 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75b000b750 con 0x7f75cc147160 2022-06-29T15:52:28.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.441+0000 7f75d0b1b700 1 --2- 172.21.15.47:0/2309070504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75b0004500 0x7f75b0004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:28.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.441+0000 7f75d131c700 1 --2- 172.21.15.47:0/2309070504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75b0003750 0x7f75b0003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:28.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.441+0000 7f75d1b1d700 1 --2- 172.21.15.47:0/2309070504 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f75cc147160 0x7f75b0004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:28.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.441+0000 7f75d0b1b700 1 -- 172.21.15.47:0/2309070504 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f75cc147160 msgr2=0x7f75b0004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:28.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.441+0000 7f75d0b1b700 1 --2- 172.21.15.47:0/2309070504 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f75cc147160 0x7f75b0004e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:28.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.441+0000 7f75d0b1b700 1 -- 172.21.15.47:0/2309070504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75b0003750 msgr2=0x7f75b0003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:28.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.441+0000 7f75d0b1b700 1 --2- 172.21.15.47:0/2309070504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75b0003750 0x7f75b0003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:28.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.441+0000 7f75d0b1b700 1 -- 172.21.15.47:0/2309070504 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b0003260 con 0x7f75b0004500 2022-06-29T15:52:28.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.441+0000 7f75d131c700 1 --2- 172.21.15.47:0/2309070504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75b0003750 0x7f75b0003b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T15:52:28.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.441+0000 7f75d0b1b700 1 --2- 172.21.15.47:0/2309070504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75b0004500 0x7f75b0004980 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 rx=0x7f75cc0b1850 tx=0x7f75cc067a10).ready entity=mon.0 client_cookie=47c510939673955a server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:28.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.441+0000 7f75c3fff700 1 -- 172.21.15.47:0/2309070504 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f75c80070d0 con 0x7f75b0004500 2022-06-29T15:52:28.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.441+0000 7f75c3fff700 1 -- 172.21.15.47:0/2309070504 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75c8007230 con 0x7f75b0004500 2022-06-29T15:52:28.465 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.441+0000 7f75c3fff700 1 -- 172.21.15.47:0/2309070504 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f75c800c8f0 con 0x7f75b0004500 2022-06-29T15:52:28.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.441+0000 7f75d231e700 1 -- 172.21.15.47:0/2309070504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75b0004500 msgr2=0x7f75b0004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:28.466 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.441+0000 7f75d231e700 1 --2- 172.21.15.47:0/2309070504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75b0004500 0x7f75b0004980 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 rx=0x7f75cc0b1850 tx=0x7f75cc067a10).stop 2022-06-29T15:52:28.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.441+0000 7f75d1b1d700 1 -- 172.21.15.47:0/2309070504 reap_dead start 2022-06-29T15:52:28.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.441+0000 7f75d231e700 1 -- 172.21.15.47:0/2309070504 shutdown_connections 2022-06-29T15:52:28.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.441+0000 7f75d231e700 1 -- 172.21.15.47:0/2309070504 >> 172.21.15.47:0/2309070504 conn(0x7f75cc062560 msgr2=0x7f75cc062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:28.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.441+0000 7f75d231e700 1 -- 172.21.15.47:0/2309070504 shutdown_connections 2022-06-29T15:52:28.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.441+0000 7f75d231e700 1 -- 172.21.15.47:0/2309070504 wait complete. 2022-06-29T15:52:28.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.445+0000 7f75d231e700 1 Processor -- start 2022-06-29T15:52:28.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.445+0000 7f75d231e700 1 -- start start 2022-06-29T15:52:28.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.449+0000 7f75d231e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75b0004500 0x7f75cc152890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:28.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.449+0000 7f75d231e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75cc0ac6a0 0x7f75cc0b0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:28.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.449+0000 7f75d231e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f75cc142880 0x7f75cc1565a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:28.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.449+0000 7f75d231e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc0b0d60 con 0x7f75cc0ac6a0 2022-06-29T15:52:28.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.449+0000 7f75d231e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc140350 con 0x7f75b0004500 2022-06-29T15:52:28.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.449+0000 7f75d231e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc1405f0 con 0x7f75cc142880 2022-06-29T15:52:28.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.449+0000 7f75d0b1b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75cc0ac6a0 0x7f75cc0b0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:28.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.449+0000 7f75d0b1b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75cc0ac6a0 0x7f75cc0b0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55400/0 (socket says 172.21.15.47:55400) 2022-06-29T15:52:28.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.449+0000 7f75d0b1b700 1 -- 172.21.15.47:0/1314981177 learned_addr learned my addr 172.21.15.47:0/1314981177 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:28.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.449+0000 7f75d131c700 1 --2- 172.21.15.47:0/1314981177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75b0004500 0x7f75cc152890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:28.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.449+0000 7f75d1b1d700 1 --2- 172.21.15.47:0/1314981177 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f75cc142880 0x7f75cc1565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:28.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.449+0000 7f75d0b1b700 1 -- 172.21.15.47:0/1314981177 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f75cc142880 msgr2=0x7f75cc1565a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:28.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.449+0000 7f75d0b1b700 1 --2- 172.21.15.47:0/1314981177 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f75cc142880 0x7f75cc1565a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:28.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.449+0000 7f75d0b1b700 1 -- 172.21.15.47:0/1314981177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75b0004500 msgr2=0x7f75cc152890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:28.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.449+0000 7f75d0b1b700 1 --2- 172.21.15.47:0/1314981177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75b0004500 0x7f75cc152890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:28.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.449+0000 7f75d0b1b700 1 -- 172.21.15.47:0/1314981177 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75cc155e60 con 0x7f75cc0ac6a0 2022-06-29T15:52:28.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.449+0000 7f75d0b1b700 1 --2- 172.21.15.47:0/1314981177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75cc0ac6a0 0x7f75cc0b0070 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 rx=0x7f75cc0b1850 tx=0x7f75c8013000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:28.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.449+0000 7f75c27fc700 1 -- 172.21.15.47:0/1314981177 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f75c801a070 con 0x7f75cc0ac6a0 2022-06-29T15:52:28.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.449+0000 7f75c27fc700 1 -- 172.21.15.47:0/1314981177 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75c801b040 con 0x7f75cc0ac6a0 2022-06-29T15:52:28.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.449+0000 7f75d231e700 1 -- 172.21.15.47:0/1314981177 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75cc156100 con 0x7f75cc0ac6a0 2022-06-29T15:52:28.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.449+0000 7f75c27fc700 1 -- 172.21.15.47:0/1314981177 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f75c80070d0 con 0x7f75cc0ac6a0 2022-06-29T15:52:28.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.449+0000 7f75d231e700 1 -- 172.21.15.47:0/1314981177 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75cc156a90 con 0x7f75cc0ac6a0 2022-06-29T15:52:28.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.449+0000 7f75d231e700 1 -- 172.21.15.47:0/1314981177 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75cc156290 con 0x7f75cc0ac6a0 2022-06-29T15:52:28.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.449+0000 7f75c27fc700 1 -- 172.21.15.47:0/1314981177 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f75c8010070 con 0x7f75cc0ac6a0 2022-06-29T15:52:28.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.453+0000 7f75c27fc700 1 --2- 172.21.15.47:0/1314981177 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f75b4053aa0 0x7f75b4055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:28.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.453+0000 7f75d131c700 1 --2- 172.21.15.47:0/1314981177 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f75b4053aa0 0x7f75b4055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:28.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.453+0000 7f75c27fc700 1 -- 172.21.15.47:0/1314981177 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f75c806cc90 con 0x7f75cc0ac6a0 2022-06-29T15:52:28.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.453+0000 7f75d131c700 1 --2- 172.21.15.47:0/1314981177 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f75b4053aa0 0x7f75b4055f60 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 rx=0x7f75bc00b000 tx=0x7f75bc008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:28.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.461+0000 7f75c27fc700 1 -- 172.21.15.47:0/1314981177 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f75c8043930 con 0x7f75cc0ac6a0 2022-06-29T15:52:28.634 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:28.625+0000 7f75d231e700 1 -- 172.21.15.47:0/1314981177 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f75cc0573d0 con 0x7f75cc0ac6a0 2022-06-29T15:52:29.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.013+0000 7f75c27fc700 1 -- 172.21.15.47:0/1314981177 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v81) v1 ==== 104+0+0 (secure 0 0 0) 0x7f75c8046f10 con 0x7f75cc0ac6a0 2022-06-29T15:52:29.025 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.017+0000 7f75d231e700 1 -- 172.21.15.47:0/1314981177 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f75b4053aa0 msgr2=0x7f75b4055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:29.026 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.017+0000 7f75d231e700 1 --2- 172.21.15.47:0/1314981177 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f75b4053aa0 0x7f75b4055f60 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 rx=0x7f75bc00b000 tx=0x7f75bc008040).stop 2022-06-29T15:52:29.026 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.017+0000 7f75d231e700 1 -- 172.21.15.47:0/1314981177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75cc0ac6a0 msgr2=0x7f75cc0b0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:29.026 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.017+0000 7f75d231e700 1 --2- 172.21.15.47:0/1314981177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75cc0ac6a0 0x7f75cc0b0070 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 rx=0x7f75cc0b1850 tx=0x7f75c8013000).stop 2022-06-29T15:52:29.027 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.017+0000 7f75d231e700 1 -- 172.21.15.47:0/1314981177 shutdown_connections 2022-06-29T15:52:29.027 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.017+0000 7f75d231e700 1 --2- 172.21.15.47:0/1314981177 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f75cc142880 0x7f75cc1565a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:29.027 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.017+0000 7f75d231e700 1 --2- 172.21.15.47:0/1314981177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75cc0ac6a0 0x7f75cc0b0070 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:29.028 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.017+0000 7f75d231e700 1 --2- 172.21.15.47:0/1314981177 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f75b4053aa0 0x7f75b4055f60 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:29.028 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.017+0000 7f75d231e700 1 --2- 172.21.15.47:0/1314981177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75b0004500 0x7f75cc152890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:29.028 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.017+0000 7f75d231e700 1 -- 172.21.15.47:0/1314981177 >> 172.21.15.47:0/1314981177 conn(0x7f75cc062560 msgr2=0x7f75cc0b3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:29.029 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.017+0000 7f75d231e700 1 -- 172.21.15.47:0/1314981177 shutdown_connections 2022-06-29T15:52:29.029 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.017+0000 7f75d231e700 1 -- 172.21.15.47:0/1314981177 wait complete. 2022-06-29T15:52:29.042 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-06-29T15:52:29.158 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.149+0000 7f21583f6700 1 Processor -- start 2022-06-29T15:52:29.158 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.149+0000 7f21583f6700 1 -- start start 2022-06-29T15:52:29.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.153+0000 7f21583f6700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f21500b4300 0x7f21500612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:29.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.153+0000 7f21583f6700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2150067610 con 0x7f2150060ee0 2022-06-29T15:52:29.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.153+0000 7f21583f6700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21501467c0 con 0x7f21501471a0 2022-06-29T15:52:29.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.153+0000 7f21583f6700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21501475d0 con 0x7f21500b4300 2022-06-29T15:52:29.161 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.153+0000 7f2156192700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f2150060ee0 0x7f21501430e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40960/0 (socket says 172.21.15.47:40960) 2022-06-29T15:52:29.161 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.153+0000 7f2156192700 1 -- 172.21.15.47:0/1244487192 learned_addr learned my addr 172.21.15.47:0/1244487192 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:52:29.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.153+0000 7f2155991700 1 --2- 172.21.15.47:0/1244487192 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f21500b4300 0x7f21500612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:29.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.153+0000 7f2155190700 1 -- 172.21.15.47:0/1244487192 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 70502132 0 0) 0x7f2150067610 con 0x7f2150060ee0 2022-06-29T15:52:29.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.153+0000 7f2155190700 1 -- 172.21.15.47:0/1244487192 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2138003260 con 0x7f2150060ee0 2022-06-29T15:52:29.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.153+0000 7f2155190700 1 -- 172.21.15.47:0/1244487192 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3770990974 0 0) 0x7f21501467c0 con 0x7f21501471a0 2022-06-29T15:52:29.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.153+0000 7f2155190700 1 -- 172.21.15.47:0/1244487192 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2150067610 con 0x7f21501471a0 2022-06-29T15:52:29.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.153+0000 7f2155991700 1 -- 172.21.15.47:0/1244487192 >> v1:172.21.15.64:6789/0 conn(0x7f21501471a0 legacy=0x7f215013cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:29.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.153+0000 7f2155991700 1 -- 172.21.15.47:0/1244487192 >> v1:172.21.15.47:6789/0 conn(0x7f2150060ee0 legacy=0x7f21501430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:29.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.153+0000 7f2155991700 1 -- 172.21.15.47:0/1244487192 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2150146a60 con 0x7f21500b4300 2022-06-29T15:52:29.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.153+0000 7f2155991700 1 --2- 172.21.15.47:0/1244487192 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f21500b4300 0x7f21500612c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f2140002f70 tx=0x7f2140006df0).ready entity=mon.2 client_cookie=2f5f2bfbe5055ebc server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:29.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.153+0000 7f2155190700 1 -- 172.21.15.47:0/1244487192 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f214000d5c0 con 0x7f21500b4300 2022-06-29T15:52:29.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.153+0000 7f2155190700 1 -- 172.21.15.47:0/1244487192 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2140000e90 con 0x7f21500b4300 2022-06-29T15:52:29.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.153+0000 7f2155190700 1 -- 172.21.15.47:0/1244487192 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f214000d910 con 0x7f21500b4300 2022-06-29T15:52:29.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.153+0000 7f21583f6700 1 -- 172.21.15.47:0/1244487192 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f21500b4300 msgr2=0x7f21500612c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:29.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.153+0000 7f21583f6700 1 --2- 172.21.15.47:0/1244487192 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f21500b4300 0x7f21500612c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f2140002f70 tx=0x7f2140006df0).stop 2022-06-29T15:52:29.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.153+0000 7f21583f6700 1 -- 172.21.15.47:0/1244487192 shutdown_connections 2022-06-29T15:52:29.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.153+0000 7f21583f6700 1 --2- 172.21.15.47:0/1244487192 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f21500b4300 0x7f21500612c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:29.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.153+0000 7f21583f6700 1 -- 172.21.15.47:0/1244487192 >> 172.21.15.47:0/1244487192 conn(0x7f215005c510 msgr2=0x7f215005f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:29.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.153+0000 7f21583f6700 1 -- 172.21.15.47:0/1244487192 shutdown_connections 2022-06-29T15:52:29.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.153+0000 7f21583f6700 1 -- 172.21.15.47:0/1244487192 wait complete. 2022-06-29T15:52:29.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.157+0000 7f21583f6700 1 Processor -- start 2022-06-29T15:52:29.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.157+0000 7f21583f6700 1 -- start start 2022-06-29T15:52:29.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.157+0000 7f21583f6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2150060ee0 0x7f215014a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:29.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.157+0000 7f21583f6700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f21500b4300 0x7f215013d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:29.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.157+0000 7f21583f6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21501471a0 0x7f215013e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:29.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.157+0000 7f21583f6700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21500b2750 con 0x7f2150060ee0 2022-06-29T15:52:29.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.157+0000 7f21583f6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2150140730 con 0x7f21501471a0 2022-06-29T15:52:29.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.157+0000 7f21583f6700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21500b3270 con 0x7f21500b4300 2022-06-29T15:52:29.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.157+0000 7f2156192700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2150060ee0 0x7f215014a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:29.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.157+0000 7f2156192700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2150060ee0 0x7f215014a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55412/0 (socket says 172.21.15.47:55412) 2022-06-29T15:52:29.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.157+0000 7f2156192700 1 -- 172.21.15.47:0/836476927 learned_addr learned my addr 172.21.15.47:0/836476927 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:29.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.157+0000 7f2155991700 1 --2- 172.21.15.47:0/836476927 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f21500b4300 0x7f215013d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:29.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.157+0000 7f2156993700 1 --2- 172.21.15.47:0/836476927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21501471a0 0x7f215013e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:29.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.157+0000 7f2156192700 1 -- 172.21.15.47:0/836476927 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f21500b4300 msgr2=0x7f215013d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:29.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.157+0000 7f2156192700 1 --2- 172.21.15.47:0/836476927 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f21500b4300 0x7f215013d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:29.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.157+0000 7f2156192700 1 -- 172.21.15.47:0/836476927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21501471a0 msgr2=0x7f215013e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:29.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.157+0000 7f2156192700 1 --2- 172.21.15.47:0/836476927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21501471a0 0x7f215013e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:29.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.157+0000 7f2156192700 1 -- 172.21.15.47:0/836476927 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f214000b180 con 0x7f2150060ee0 2022-06-29T15:52:29.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.157+0000 7f2156192700 1 --2- 172.21.15.47:0/836476927 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2150060ee0 0x7f215014a1c0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 rx=0x7f214c0011d0 tx=0x7f214c007b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:29.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.157+0000 7f213f7fe700 1 -- 172.21.15.47:0/836476927 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f214c0070d0 con 0x7f2150060ee0 2022-06-29T15:52:29.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.157+0000 7f213f7fe700 1 -- 172.21.15.47:0/836476927 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f214c007230 con 0x7f2150060ee0 2022-06-29T15:52:29.182 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.157+0000 7f21583f6700 1 -- 172.21.15.47:0/836476927 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21500b3510 con 0x7f2150060ee0 2022-06-29T15:52:29.182 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.157+0000 7f213f7fe700 1 -- 172.21.15.47:0/836476927 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f214c005610 con 0x7f2150060ee0 2022-06-29T15:52:29.182 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.157+0000 7f21583f6700 1 -- 172.21.15.47:0/836476927 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21500b29b0 con 0x7f2150060ee0 2022-06-29T15:52:29.182 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.161+0000 7f21583f6700 1 -- 172.21.15.47:0/836476927 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21500573d0 con 0x7f2150060ee0 2022-06-29T15:52:29.183 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.161+0000 7f213f7fe700 1 -- 172.21.15.47:0/836476927 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f214c01a060 con 0x7f2150060ee0 2022-06-29T15:52:29.183 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.161+0000 7f213f7fe700 1 --2- 172.21.15.47:0/836476927 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2130053a10 0x7f2130055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:29.183 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.161+0000 7f213f7fe700 1 -- 172.21.15.47:0/836476927 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f214c06b810 con 0x7f2150060ee0 2022-06-29T15:52:29.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.161+0000 7f2155991700 1 --2- 172.21.15.47:0/836476927 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2130053a10 0x7f2130055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:29.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.161+0000 7f2155991700 1 --2- 172.21.15.47:0/836476927 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2130053a10 0x7f2130055ed0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7f214001a000 tx=0x7f2140014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:29.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.165+0000 7f213f7fe700 1 -- 172.21.15.47:0/836476927 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f214c042500 con 0x7f2150060ee0 2022-06-29T15:52:29.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:29.297+0000 7f21583f6700 1 -- 172.21.15.47:0/836476927 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f21500b5e50 con 0x7f2150060ee0 2022-06-29T15:52:30.028 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.021+0000 7f213f7fe700 1 -- 172.21.15.47:0/836476927 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v164) v1 ==== 185+0+0 (secure 0 0 0) 0x7f214c018020 con 0x7f2150060ee0 2022-06-29T15:52:30.032 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.025+0000 7f21583f6700 1 -- 172.21.15.47:0/836476927 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2130053a10 msgr2=0x7f2130055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:30.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.025+0000 7f21583f6700 1 --2- 172.21.15.47:0/836476927 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2130053a10 0x7f2130055ed0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7f214001a000 tx=0x7f2140014000).stop 2022-06-29T15:52:30.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.025+0000 7f21583f6700 1 -- 172.21.15.47:0/836476927 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2150060ee0 msgr2=0x7f215014a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:30.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.025+0000 7f21583f6700 1 --2- 172.21.15.47:0/836476927 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2150060ee0 0x7f215014a1c0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 rx=0x7f214c0011d0 tx=0x7f214c007b50).stop 2022-06-29T15:52:30.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.025+0000 7f21583f6700 1 -- 172.21.15.47:0/836476927 shutdown_connections 2022-06-29T15:52:30.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.025+0000 7f21583f6700 1 --2- 172.21.15.47:0/836476927 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2130053a10 0x7f2130055ed0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:30.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.025+0000 7f21583f6700 1 --2- 172.21.15.47:0/836476927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21501471a0 0x7f215013e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:30.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.025+0000 7f21583f6700 1 --2- 172.21.15.47:0/836476927 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f21500b4300 0x7f215013d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:30.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.025+0000 7f21583f6700 1 --2- 172.21.15.47:0/836476927 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2150060ee0 0x7f215014a1c0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:30.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.025+0000 7f21583f6700 1 -- 172.21.15.47:0/836476927 >> 172.21.15.47:0/836476927 conn(0x7f215005c510 msgr2=0x7f21500b6930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:30.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.025+0000 7f21583f6700 1 -- 172.21.15.47:0/836476927 shutdown_connections 2022-06-29T15:52:30.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.025+0000 7f21583f6700 1 -- 172.21.15.47:0/836476927 wait complete. 2022-06-29T15:52:30.036 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' removed 2022-06-29T15:52:30.049 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-06-29T15:52:30.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.157+0000 7fdefb59e700 1 Processor -- start 2022-06-29T15:52:30.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.157+0000 7fdefb59e700 1 -- start start 2022-06-29T15:52:30.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.157+0000 7fdefb59e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdefc0b5700 0x7fdefc147270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:30.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.157+0000 7fdefb59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdefc0b1310 0x7fdefc060610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:30.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.157+0000 7fdefb59e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdefc13d6a0 con 0x7fdefc143140 2022-06-29T15:52:30.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.157+0000 7fdefb59e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdefc146890 con 0x7fdefc0b1310 2022-06-29T15:52:30.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.157+0000 7fdefb59e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc146b30 con 0x7fdefc0b5700 2022-06-29T15:52:30.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.157+0000 7fdefad9d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fdefc143140 0x7fdefc0aa330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40978/0 (socket says 172.21.15.47:40978) 2022-06-29T15:52:30.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.157+0000 7fdefad9d700 1 -- 172.21.15.47:0/1195490108 learned_addr learned my addr 172.21.15.47:0/1195490108 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T15:52:30.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.157+0000 7fdef9d9b700 1 --2- 172.21.15.47:0/1195490108 >> v2:172.21.15.64:3300/0 conn(0x7fdefc0b1310 0x7fdefc060610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:30.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.157+0000 7fdefa59c700 1 --2- 172.21.15.47:0/1195490108 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdefc0b5700 0x7fdefc147270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:30.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.157+0000 7fdef959a700 1 -- 172.21.15.47:0/1195490108 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2140745405 0 0) 0x7fdefc13d6a0 con 0x7fdefc143140 2022-06-29T15:52:30.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.157+0000 7fdef959a700 1 -- 172.21.15.47:0/1195490108 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdedc003260 con 0x7fdefc143140 2022-06-29T15:52:30.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.157+0000 7fdefa59c700 1 -- 172.21.15.47:0/1195490108 >> v2:172.21.15.64:3300/0 conn(0x7fdefc0b1310 msgr2=0x7fdefc060610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T15:52:30.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.157+0000 7fdefa59c700 1 --2- 172.21.15.47:0/1195490108 >> v2:172.21.15.64:3300/0 conn(0x7fdefc0b1310 0x7fdefc060610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:30.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.157+0000 7fdefa59c700 1 -- 172.21.15.47:0/1195490108 >> v1:172.21.15.47:6789/0 conn(0x7fdefc143140 legacy=0x7fdefc0aa330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:30.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.157+0000 7fdefa59c700 1 -- 172.21.15.47:0/1195490108 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdefc146dd0 con 0x7fdefc0b5700 2022-06-29T15:52:30.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.161+0000 7fdefa59c700 1 --2- 172.21.15.47:0/1195490108 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdefc0b5700 0x7fdefc147270 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7fdeec0032f0 tx=0x7fdeec0091d0).ready entity=mon.2 client_cookie=e064c4f46ded82d6 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:30.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.161+0000 7fdef959a700 1 -- 172.21.15.47:0/1195490108 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdeec00b600 con 0x7fdefc0b5700 2022-06-29T15:52:30.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.161+0000 7fdef959a700 1 -- 172.21.15.47:0/1195490108 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeec00b760 con 0x7fdefc0b5700 2022-06-29T15:52:30.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.161+0000 7fdef959a700 1 -- 172.21.15.47:0/1195490108 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdeec00b8d0 con 0x7fdefc0b5700 2022-06-29T15:52:30.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.161+0000 7fdefb59e700 1 -- 172.21.15.47:0/1195490108 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdefc0b5700 msgr2=0x7fdefc147270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:30.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.161+0000 7fdefb59e700 1 --2- 172.21.15.47:0/1195490108 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdefc0b5700 0x7fdefc147270 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7fdeec0032f0 tx=0x7fdeec0091d0).stop 2022-06-29T15:52:30.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.161+0000 7fdefb59e700 1 -- 172.21.15.47:0/1195490108 shutdown_connections 2022-06-29T15:52:30.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.161+0000 7fdefb59e700 1 --2- 172.21.15.47:0/1195490108 >> v2:172.21.15.64:3300/0 conn(0x7fdefc0b1310 0x7fdefc060610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:30.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.161+0000 7fdefb59e700 1 --2- 172.21.15.47:0/1195490108 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdefc0b5700 0x7fdefc147270 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:30.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.161+0000 7fdefb59e700 1 -- 172.21.15.47:0/1195490108 >> 172.21.15.47:0/1195490108 conn(0x7fdefc062560 msgr2=0x7fdefc062970 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:30.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.161+0000 7fdefb59e700 1 -- 172.21.15.47:0/1195490108 shutdown_connections 2022-06-29T15:52:30.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.161+0000 7fdefb59e700 1 -- 172.21.15.47:0/1195490108 wait complete. 2022-06-29T15:52:30.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.161+0000 7fdefb59e700 1 Processor -- start 2022-06-29T15:52:30.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.161+0000 7fdefb59e700 1 -- start start 2022-06-29T15:52:30.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.161+0000 7fdefb59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdefc0b1310 0x7fdefc1419b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:30.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.161+0000 7fdefb59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdefc0b5700 0x7fdefc138db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:30.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.161+0000 7fdefb59e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdefc143140 0x7fdefc139560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:30.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.161+0000 7fdefb59e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc141ea0 con 0x7fdefc0b1310 2022-06-29T15:52:30.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.161+0000 7fdefb59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc0b0c10 con 0x7fdefc0b5700 2022-06-29T15:52:30.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.161+0000 7fdefb59e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc136950 con 0x7fdefc143140 2022-06-29T15:52:30.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.161+0000 7fdefa59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdefc0b1310 0x7fdefc1419b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:30.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.161+0000 7fdefa59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdefc0b1310 0x7fdefc1419b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55426/0 (socket says 172.21.15.47:55426) 2022-06-29T15:52:30.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.161+0000 7fdefa59c700 1 -- 172.21.15.47:0/3424021188 learned_addr learned my addr 172.21.15.47:0/3424021188 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T15:52:30.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.161+0000 7fdefad9d700 1 --2- 172.21.15.47:0/3424021188 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdefc143140 0x7fdefc139560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:30.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.161+0000 7fdef9d9b700 1 --2- 172.21.15.47:0/3424021188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdefc0b5700 0x7fdefc138db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:30.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.161+0000 7fdefa59c700 1 -- 172.21.15.47:0/3424021188 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdefc143140 msgr2=0x7fdefc139560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:30.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.161+0000 7fdefa59c700 1 --2- 172.21.15.47:0/3424021188 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdefc143140 0x7fdefc139560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:30.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.161+0000 7fdefa59c700 1 -- 172.21.15.47:0/3424021188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdefc0b5700 msgr2=0x7fdefc138db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:30.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.161+0000 7fdefa59c700 1 --2- 172.21.15.47:0/3424021188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdefc0b5700 0x7fdefc138db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:30.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.165+0000 7fdefa59c700 1 -- 172.21.15.47:0/3424021188 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeec009470 con 0x7fdefc0b1310 2022-06-29T15:52:30.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.165+0000 7fdefa59c700 1 --2- 172.21.15.47:0/3424021188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdefc0b1310 0x7fdefc1419b0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 rx=0x7fdefc139ec0 tx=0x7fdeec016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:30.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.165+0000 7fdef37fe700 1 -- 172.21.15.47:0/3424021188 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdeec011c00 con 0x7fdefc0b1310 2022-06-29T15:52:30.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.165+0000 7fdef37fe700 1 -- 172.21.15.47:0/3424021188 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeec011d60 con 0x7fdefc0b1310 2022-06-29T15:52:30.182 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.165+0000 7fdefb59e700 1 -- 172.21.15.47:0/3424021188 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdefc13a020 con 0x7fdefc0b1310 2022-06-29T15:52:30.182 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.165+0000 7fdef37fe700 1 -- 172.21.15.47:0/3424021188 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdeec01f5e0 con 0x7fdefc0b1310 2022-06-29T15:52:30.182 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.165+0000 7fdefb59e700 1 -- 172.21.15.47:0/3424021188 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdefc136b80 con 0x7fdefc0b1310 2022-06-29T15:52:30.182 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.165+0000 7fdef37fe700 1 -- 172.21.15.47:0/3424021188 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fdeec026030 con 0x7fdefc0b1310 2022-06-29T15:52:30.183 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.165+0000 7fdef37fe700 1 --2- 172.21.15.47:0/3424021188 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdee0053950 0x7fdee0055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T15:52:30.183 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.165+0000 7fdef37fe700 1 -- 172.21.15.47:0/3424021188 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6068+0+0 (secure 0 0 0) 0x7fdeec06c930 con 0x7fdefc0b1310 2022-06-29T15:52:30.183 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.165+0000 7fdef9d9b700 1 --2- 172.21.15.47:0/3424021188 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdee0053950 0x7fdee0055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T15:52:30.183 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.165+0000 7fdefb59e700 1 -- 172.21.15.47:0/3424021188 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdefc13a860 con 0x7fdefc0b1310 2022-06-29T15:52:30.183 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.169+0000 7fdef9d9b700 1 --2- 172.21.15.47:0/3424021188 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdee0053950 0x7fdee0055e10 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7fdee400b000 tx=0x7fdee4006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T15:52:30.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.173+0000 7fdef37fe700 1 -- 172.21.15.47:0/3424021188 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdeec043850 con 0x7fdefc0b1310 2022-06-29T15:52:30.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:30.305+0000 7fdefb59e700 1 -- 172.21.15.47:0/3424021188 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fdefc0573d0 con 0x7fdefc0b1310 2022-06-29T15:52:31.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:31.033+0000 7fdef37fe700 1 -- 172.21.15.47:0/3424021188 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v165) v1 ==== 173+0+0 (secure 0 0 0) 0x7fdeec024030 con 0x7fdefc0b1310 2022-06-29T15:52:31.045 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:31.037+0000 7fdefb59e700 1 -- 172.21.15.47:0/3424021188 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdee0053950 msgr2=0x7fdee0055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:31.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:31.037+0000 7fdefb59e700 1 --2- 172.21.15.47:0/3424021188 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdee0053950 0x7fdee0055e10 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7fdee400b000 tx=0x7fdee4006df0).stop 2022-06-29T15:52:31.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:31.037+0000 7fdefb59e700 1 -- 172.21.15.47:0/3424021188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdefc0b1310 msgr2=0x7fdefc1419b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T15:52:31.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:31.037+0000 7fdefb59e700 1 --2- 172.21.15.47:0/3424021188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdefc0b1310 0x7fdefc1419b0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 rx=0x7fdefc139ec0 tx=0x7fdeec016000).stop 2022-06-29T15:52:31.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:31.037+0000 7fdefb59e700 1 -- 172.21.15.47:0/3424021188 shutdown_connections 2022-06-29T15:52:31.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:31.037+0000 7fdefb59e700 1 --2- 172.21.15.47:0/3424021188 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fdefc143140 0x7fdefc139560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:31.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:31.037+0000 7fdefb59e700 1 --2- 172.21.15.47:0/3424021188 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fdee0053950 0x7fdee0055e10 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:31.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:31.037+0000 7fdefb59e700 1 --2- 172.21.15.47:0/3424021188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdefc0b5700 0x7fdefc138db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:31.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:31.037+0000 7fdefb59e700 1 --2- 172.21.15.47:0/3424021188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdefc0b1310 0x7fdefc1419b0 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T15:52:31.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:31.037+0000 7fdefb59e700 1 -- 172.21.15.47:0/3424021188 >> 172.21.15.47:0/3424021188 conn(0x7fdefc062560 msgr2=0x7fdefc0ab040 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T15:52:31.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:31.037+0000 7fdefb59e700 1 -- 172.21.15.47:0/3424021188 shutdown_connections 2022-06-29T15:52:31.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T15:52:31.037+0000 7fdefb59e700 1 -- 172.21.15.47:0/3424021188 wait complete. 2022-06-29T15:52:31.049 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' removed 2022-06-29T15:52:31.060 INFO:teuthology.orchestra.run:Running command with timeout 60 2022-06-29T15:52:31.060 DEBUG:teuthology.orchestra.run.smithi099:> cd /home/ubuntu/cephtest/mnt.0 && sudo bash -c 'stat -f .' 2022-06-29T15:52:42.690 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:52:42.693 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:52:42.696 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:52:42.698 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:53:01.165 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T15:53:01.166 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T15:53:01.167 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T15:53:12.755 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:53:12.759 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:53:12.762 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:53:12.765 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:53:31.067 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2022-06-29T15:53:31.071 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 699, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1361, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 701, in recv raise socket.timeout() socket.timeout 2022-06-29T15:53:42.816 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:53:42.820 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:53:42.823 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:53:42.825 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:54:12.873 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:54:12.877 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:54:12.879 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:54:12.882 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:54:42.956 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:54:42.959 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:54:42.962 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:54:42.964 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:55:13.036 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:55:13.040 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:55:13.043 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:55:13.046 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:55:43.109 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:55:43.113 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:55:43.116 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:55:43.119 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:56:13.257 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:56:13.265 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:56:13.267 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:56:13.270 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:56:43.350 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:56:43.354 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:56:43.357 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:56:43.359 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:57:13.412 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:57:13.416 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:57:13.419 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:57:13.422 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:57:43.484 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:57:43.488 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:57:43.491 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:57:43.493 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:58:13.606 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:58:13.610 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:58:13.613 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:58:13.616 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:58:43.691 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:58:43.695 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:58:43.698 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:58:43.700 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:59:13.765 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:59:13.770 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:59:13.773 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:59:13.776 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:59:43.860 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:59:43.864 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:59:43.867 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T15:59:43.869 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:00:13.924 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:00:13.928 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:00:13.930 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:00:13.933 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:00:44.006 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:00:44.011 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:00:44.015 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:00:44.018 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:01:14.090 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:01:14.095 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:01:14.098 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:01:14.100 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:01:44.175 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:01:44.179 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:01:44.182 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:01:44.185 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:02:14.241 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:02:14.244 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:02:14.247 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:02:14.250 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:02:44.321 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:02:44.324 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:02:44.327 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:02:44.330 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:03:14.412 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:03:14.416 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:03:14.419 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:03:14.421 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:03:44.479 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:03:44.483 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:03:44.486 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:03:44.489 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:04:14.577 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:04:14.581 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:04:14.583 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:04:14.586 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:04:44.656 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:04:44.660 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:04:44.663 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:04:44.665 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:05:14.717 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:05:14.721 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:05:14.724 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:05:14.726 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:05:44.797 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:05:44.801 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:05:44.804 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:05:44.806 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:06:14.854 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:06:14.858 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:06:14.861 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:06:14.863 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:06:44.932 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:06:44.935 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:06:44.938 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:06:44.940 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:07:14.996 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:07:15.000 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:07:15.002 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:07:15.005 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:07:45.081 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:07:45.084 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:07:45.087 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:07:45.090 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:08:01.171 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2022-06-29T16:08:01.173 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 699, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1361, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 701, in recv raise socket.timeout() socket.timeout 2022-06-29T16:08:01.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T16:08:01.177 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-06-29T16:08:01.177 DEBUG:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T16:08:01.201 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T16:08:01.224 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T16:08:01.244 DEBUG:teuthology.orchestra.run.smithi164:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-06-29T16:08:01.267 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-06-29T16:08:01.268 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T16:08:01.269 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T16:08:01.270 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T16:08:15.143 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:08:15.146 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:08:15.149 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:08:15.152 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:08:45.221 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:08:45.225 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:08:45.228 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:08:45.230 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:09:15.306 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:09:15.310 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:09:15.313 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:09:15.316 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:09:45.379 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:09:45.383 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:09:45.386 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:09:45.389 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:10:15.439 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:10:15.442 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:10:15.445 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:10:15.447 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:10:45.508 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:10:45.511 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:10:45.514 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:10:45.516 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:11:15.636 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:11:15.640 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:11:15.644 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:11:15.647 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:11:45.719 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:11:45.723 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:11:45.725 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:11:45.728 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:12:15.779 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:12:15.782 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:12:15.785 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:12:15.787 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:12:45.869 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:12:45.872 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:12:45.875 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:12:45.877 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:13:15.936 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:13:15.939 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:13:15.942 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:13:15.945 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:13:46.012 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:13:46.015 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:13:46.018 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:13:46.020 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:14:16.069 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:14:16.073 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:14:16.076 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:14:16.078 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:14:46.137 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:14:46.141 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:14:46.144 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:14:46.147 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:15:16.193 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:15:16.197 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:15:16.199 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:15:16.202 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:15:46.264 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:15:46.267 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:15:46.270 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:15:46.272 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:16:16.327 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:16:16.331 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:16:16.334 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:16:16.336 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:16:46.415 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:16:46.419 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:16:46.423 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:16:46.426 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:17:16.486 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:17:16.490 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:17:16.493 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:17:16.495 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:17:46.553 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:17:46.557 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:17:46.559 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:17:46.562 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:18:16.612 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:18:16.615 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:18:16.618 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:18:16.620 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:18:46.692 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:18:46.695 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:18:46.698 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:18:46.700 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:19:16.756 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:19:16.761 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:19:16.764 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:19:16.767 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:19:46.825 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:19:46.829 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:19:46.832 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:19:46.834 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:20:16.896 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:20:16.899 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:20:16.902 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:20:16.905 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:20:46.957 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:20:46.961 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:20:46.964 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:20:46.966 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:21:17.022 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:21:17.026 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:21:17.028 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:21:17.031 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:21:47.152 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:21:47.156 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:21:47.159 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:21:47.161 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:22:17.215 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:22:17.219 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:22:17.222 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:22:17.224 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:22:47.269 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:22:47.273 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:22:47.275 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:22:47.278 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:23:01.276 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2022-06-29T16:23:01.278 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 699, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1361, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 701, in recv raise socket.timeout() socket.timeout 2022-06-29T16:23:01.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T16:23:01.287 INFO:tasks.cephfs_test_runner:test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) ... ERROR 2022-06-29T16:23:01.288 INFO:tasks.cephfs_test_runner:ERROR 2022-06-29T16:23:01.289 INFO:tasks.cephfs_test_runner: 2022-06-29T16:23:01.289 INFO:tasks.cephfs_test_runner:====================================================================== 2022-06-29T16:23:01.290 INFO:tasks.cephfs_test_runner:ERROR: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2022-06-29T16:23:01.290 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2022-06-29T16:23:01.290 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2022-06-29T16:23:01.291 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr 2022-06-29T16:23:01.291 INFO:tasks.cephfs_test_runner: out = self.in_stderr_buffer.read(nbytes, self.timeout) 2022-06-29T16:23:01.291 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read 2022-06-29T16:23:01.292 INFO:tasks.cephfs_test_runner: raise PipeTimeout() 2022-06-29T16:23:01.292 INFO:tasks.cephfs_test_runner:paramiko.buffered_pipe.PipeTimeout 2022-06-29T16:23:01.292 INFO:tasks.cephfs_test_runner: 2022-06-29T16:23:01.293 INFO:tasks.cephfs_test_runner:During handling of the above exception, another exception occurred: 2022-06-29T16:23:01.293 INFO:tasks.cephfs_test_runner: 2022-06-29T16:23:01.293 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2022-06-29T16:23:01.294 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_f25ffac72e5a9e63860ff7deb6767a11f0559a16/qa/tasks/cephfs/test_misc.py", line 28, in test_statfs_on_deleted_fs 2022-06-29T16:23:01.294 INFO:tasks.cephfs_test_runner: self.mount_a.umount_wait(force=True) 2022-06-29T16:23:01.294 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_f25ffac72e5a9e63860ff7deb6767a11f0559a16/qa/tasks/cephfs/fuse_mount.py", line 326, in umount_wait 2022-06-29T16:23:01.295 INFO:tasks.cephfs_test_runner: if not (self.is_mounted() and self.fuse_daemon): 2022-06-29T16:23:01.295 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_f25ffac72e5a9e63860ff7deb6767a11f0559a16/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted 2022-06-29T16:23:01.295 INFO:tasks.cephfs_test_runner: proc.wait() 2022-06-29T16:23:01.296 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/orchestra/run.py", line 148, in wait 2022-06-29T16:23:01.296 INFO:tasks.cephfs_test_runner: greenlet.get(block=True,timeout=60) 2022-06-29T16:23:01.296 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get 2022-06-29T16:23:01.297 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception 2022-06-29T16:23:01.297 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise 2022-06-29T16:23:01.297 INFO:tasks.cephfs_test_runner: raise value.with_traceback(tb) 2022-06-29T16:23:01.298 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run 2022-06-29T16:23:01.298 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/orchestra/run.py", line 323, in copy_file_to 2022-06-29T16:23:01.298 INFO:tasks.cephfs_test_runner: copy_to_log(src, logger, capture=stream, quiet=quiet) 2022-06-29T16:23:01.299 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/orchestra/run.py", line 276, in copy_to_log 2022-06-29T16:23:01.299 INFO:tasks.cephfs_test_runner: for line in f: 2022-06-29T16:23:01.299 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ 2022-06-29T16:23:01.299 INFO:tasks.cephfs_test_runner: line = self.readline() 2022-06-29T16:23:01.300 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline 2022-06-29T16:23:01.300 INFO:tasks.cephfs_test_runner: new_data = self._read(n) 2022-06-29T16:23:01.300 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read 2022-06-29T16:23:01.301 INFO:tasks.cephfs_test_runner: return self.channel.recv_stderr(size) 2022-06-29T16:23:01.301 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr 2022-06-29T16:23:01.301 INFO:tasks.cephfs_test_runner: raise socket.timeout() 2022-06-29T16:23:01.302 INFO:tasks.cephfs_test_runner:socket.timeout 2022-06-29T16:23:01.302 INFO:tasks.cephfs_test_runner: 2022-06-29T16:23:01.302 INFO:tasks.cephfs_test_runner:====================================================================== 2022-06-29T16:23:01.303 INFO:tasks.cephfs_test_runner:ERROR: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2022-06-29T16:23:01.303 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2022-06-29T16:23:01.303 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2022-06-29T16:23:01.304 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr 2022-06-29T16:23:01.304 INFO:tasks.cephfs_test_runner: out = self.in_stderr_buffer.read(nbytes, self.timeout) 2022-06-29T16:23:01.304 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read 2022-06-29T16:23:01.305 INFO:tasks.cephfs_test_runner: raise PipeTimeout() 2022-06-29T16:23:01.305 INFO:tasks.cephfs_test_runner:paramiko.buffered_pipe.PipeTimeout 2022-06-29T16:23:01.305 INFO:tasks.cephfs_test_runner: 2022-06-29T16:23:01.306 INFO:tasks.cephfs_test_runner:During handling of the above exception, another exception occurred: 2022-06-29T16:23:01.306 INFO:tasks.cephfs_test_runner: 2022-06-29T16:23:01.306 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2022-06-29T16:23:01.307 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_f25ffac72e5a9e63860ff7deb6767a11f0559a16/qa/tasks/cephfs/cephfs_test_case.py", line 175, in tearDown 2022-06-29T16:23:01.307 INFO:tasks.cephfs_test_runner: m.teardown() 2022-06-29T16:23:01.307 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_f25ffac72e5a9e63860ff7deb6767a11f0559a16/qa/tasks/cephfs/fuse_mount.py", line 408, in teardown 2022-06-29T16:23:01.308 INFO:tasks.cephfs_test_runner: self.umount() 2022-06-29T16:23:01.308 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_f25ffac72e5a9e63860ff7deb6767a11f0559a16/qa/tasks/cephfs/fuse_mount.py", line 261, in umount 2022-06-29T16:23:01.308 INFO:tasks.cephfs_test_runner: if not self.is_mounted(): 2022-06-29T16:23:01.309 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_f25ffac72e5a9e63860ff7deb6767a11f0559a16/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted 2022-06-29T16:23:01.309 INFO:tasks.cephfs_test_runner: proc.wait() 2022-06-29T16:23:01.309 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/orchestra/run.py", line 148, in wait 2022-06-29T16:23:01.310 INFO:tasks.cephfs_test_runner: greenlet.get(block=True,timeout=60) 2022-06-29T16:23:01.310 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get 2022-06-29T16:23:01.310 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception 2022-06-29T16:23:01.311 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise 2022-06-29T16:23:01.311 INFO:tasks.cephfs_test_runner: raise value.with_traceback(tb) 2022-06-29T16:23:01.311 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run 2022-06-29T16:23:01.312 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/orchestra/run.py", line 323, in copy_file_to 2022-06-29T16:23:01.312 INFO:tasks.cephfs_test_runner: copy_to_log(src, logger, capture=stream, quiet=quiet) 2022-06-29T16:23:01.312 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/orchestra/run.py", line 276, in copy_to_log 2022-06-29T16:23:01.312 INFO:tasks.cephfs_test_runner: for line in f: 2022-06-29T16:23:01.313 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ 2022-06-29T16:23:01.313 INFO:tasks.cephfs_test_runner: line = self.readline() 2022-06-29T16:23:01.314 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline 2022-06-29T16:23:01.314 INFO:tasks.cephfs_test_runner: new_data = self._read(n) 2022-06-29T16:23:01.314 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read 2022-06-29T16:23:01.315 INFO:tasks.cephfs_test_runner: return self.channel.recv_stderr(size) 2022-06-29T16:23:01.315 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr 2022-06-29T16:23:01.315 INFO:tasks.cephfs_test_runner: raise socket.timeout() 2022-06-29T16:23:01.316 INFO:tasks.cephfs_test_runner:socket.timeout 2022-06-29T16:23:01.316 INFO:tasks.cephfs_test_runner: 2022-06-29T16:23:01.316 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2022-06-29T16:23:01.317 INFO:tasks.cephfs_test_runner:Ran 12 tests in 2402.907s 2022-06-29T16:23:01.317 INFO:tasks.cephfs_test_runner: 2022-06-29T16:23:01.317 INFO:tasks.cephfs_test_runner:FAILED (errors=2) 2022-06-29T16:23:01.318 INFO:tasks.cephfs_test_runner: 2022-06-29T16:23:01.318 INFO:tasks.cephfs_test_runner:====================================================================== 2022-06-29T16:23:01.318 INFO:tasks.cephfs_test_runner:ERROR: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2022-06-29T16:23:01.319 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2022-06-29T16:23:01.319 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2022-06-29T16:23:01.319 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr 2022-06-29T16:23:01.320 INFO:tasks.cephfs_test_runner: out = self.in_stderr_buffer.read(nbytes, self.timeout) 2022-06-29T16:23:01.320 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read 2022-06-29T16:23:01.320 INFO:tasks.cephfs_test_runner: raise PipeTimeout() 2022-06-29T16:23:01.321 INFO:tasks.cephfs_test_runner:paramiko.buffered_pipe.PipeTimeout 2022-06-29T16:23:01.321 INFO:tasks.cephfs_test_runner: 2022-06-29T16:23:01.321 INFO:tasks.cephfs_test_runner:During handling of the above exception, another exception occurred: 2022-06-29T16:23:01.322 INFO:tasks.cephfs_test_runner: 2022-06-29T16:23:01.322 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2022-06-29T16:23:01.322 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_f25ffac72e5a9e63860ff7deb6767a11f0559a16/qa/tasks/cephfs/test_misc.py", line 28, in test_statfs_on_deleted_fs 2022-06-29T16:23:01.323 INFO:tasks.cephfs_test_runner: self.mount_a.umount_wait(force=True) 2022-06-29T16:23:01.323 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_f25ffac72e5a9e63860ff7deb6767a11f0559a16/qa/tasks/cephfs/fuse_mount.py", line 326, in umount_wait 2022-06-29T16:23:01.323 INFO:tasks.cephfs_test_runner: if not (self.is_mounted() and self.fuse_daemon): 2022-06-29T16:23:01.324 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_f25ffac72e5a9e63860ff7deb6767a11f0559a16/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted 2022-06-29T16:23:01.324 INFO:tasks.cephfs_test_runner: proc.wait() 2022-06-29T16:23:01.324 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/orchestra/run.py", line 148, in wait 2022-06-29T16:23:01.325 INFO:tasks.cephfs_test_runner: greenlet.get(block=True,timeout=60) 2022-06-29T16:23:01.325 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get 2022-06-29T16:23:01.325 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception 2022-06-29T16:23:01.326 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise 2022-06-29T16:23:01.326 INFO:tasks.cephfs_test_runner: raise value.with_traceback(tb) 2022-06-29T16:23:01.326 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run 2022-06-29T16:23:01.327 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/orchestra/run.py", line 323, in copy_file_to 2022-06-29T16:23:01.327 INFO:tasks.cephfs_test_runner: copy_to_log(src, logger, capture=stream, quiet=quiet) 2022-06-29T16:23:01.327 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/orchestra/run.py", line 276, in copy_to_log 2022-06-29T16:23:01.328 INFO:tasks.cephfs_test_runner: for line in f: 2022-06-29T16:23:01.328 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ 2022-06-29T16:23:01.328 INFO:tasks.cephfs_test_runner: line = self.readline() 2022-06-29T16:23:01.329 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline 2022-06-29T16:23:01.329 INFO:tasks.cephfs_test_runner: new_data = self._read(n) 2022-06-29T16:23:01.329 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read 2022-06-29T16:23:01.330 INFO:tasks.cephfs_test_runner: return self.channel.recv_stderr(size) 2022-06-29T16:23:01.330 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr 2022-06-29T16:23:01.330 INFO:tasks.cephfs_test_runner: raise socket.timeout() 2022-06-29T16:23:01.331 INFO:tasks.cephfs_test_runner:socket.timeout 2022-06-29T16:23:01.331 INFO:tasks.cephfs_test_runner: 2022-06-29T16:23:01.331 INFO:tasks.cephfs_test_runner:====================================================================== 2022-06-29T16:23:01.332 INFO:tasks.cephfs_test_runner:ERROR: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2022-06-29T16:23:01.332 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2022-06-29T16:23:01.332 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2022-06-29T16:23:01.333 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr 2022-06-29T16:23:01.333 INFO:tasks.cephfs_test_runner: out = self.in_stderr_buffer.read(nbytes, self.timeout) 2022-06-29T16:23:01.333 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read 2022-06-29T16:23:01.334 INFO:tasks.cephfs_test_runner: raise PipeTimeout() 2022-06-29T16:23:01.334 INFO:tasks.cephfs_test_runner:paramiko.buffered_pipe.PipeTimeout 2022-06-29T16:23:01.334 INFO:tasks.cephfs_test_runner: 2022-06-29T16:23:01.335 INFO:tasks.cephfs_test_runner:During handling of the above exception, another exception occurred: 2022-06-29T16:23:01.335 INFO:tasks.cephfs_test_runner: 2022-06-29T16:23:01.335 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2022-06-29T16:23:01.336 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_f25ffac72e5a9e63860ff7deb6767a11f0559a16/qa/tasks/cephfs/cephfs_test_case.py", line 175, in tearDown 2022-06-29T16:23:01.336 INFO:tasks.cephfs_test_runner: m.teardown() 2022-06-29T16:23:01.336 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_f25ffac72e5a9e63860ff7deb6767a11f0559a16/qa/tasks/cephfs/fuse_mount.py", line 408, in teardown 2022-06-29T16:23:01.337 INFO:tasks.cephfs_test_runner: self.umount() 2022-06-29T16:23:01.337 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_f25ffac72e5a9e63860ff7deb6767a11f0559a16/qa/tasks/cephfs/fuse_mount.py", line 261, in umount 2022-06-29T16:23:01.337 INFO:tasks.cephfs_test_runner: if not self.is_mounted(): 2022-06-29T16:23:01.338 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_f25ffac72e5a9e63860ff7deb6767a11f0559a16/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted 2022-06-29T16:23:01.338 INFO:tasks.cephfs_test_runner: proc.wait() 2022-06-29T16:23:01.339 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/orchestra/run.py", line 148, in wait 2022-06-29T16:23:01.339 INFO:tasks.cephfs_test_runner: greenlet.get(block=True,timeout=60) 2022-06-29T16:23:01.339 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get 2022-06-29T16:23:01.340 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception 2022-06-29T16:23:01.340 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise 2022-06-29T16:23:01.340 INFO:tasks.cephfs_test_runner: raise value.with_traceback(tb) 2022-06-29T16:23:01.340 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run 2022-06-29T16:23:01.341 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/orchestra/run.py", line 323, in copy_file_to 2022-06-29T16:23:01.341 INFO:tasks.cephfs_test_runner: copy_to_log(src, logger, capture=stream, quiet=quiet) 2022-06-29T16:23:01.341 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/orchestra/run.py", line 276, in copy_to_log 2022-06-29T16:23:01.342 INFO:tasks.cephfs_test_runner: for line in f: 2022-06-29T16:23:01.342 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ 2022-06-29T16:23:01.342 INFO:tasks.cephfs_test_runner: line = self.readline() 2022-06-29T16:23:01.343 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline 2022-06-29T16:23:01.343 INFO:tasks.cephfs_test_runner: new_data = self._read(n) 2022-06-29T16:23:01.343 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read 2022-06-29T16:23:01.343 INFO:tasks.cephfs_test_runner: return self.channel.recv_stderr(size) 2022-06-29T16:23:01.344 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr 2022-06-29T16:23:01.344 INFO:tasks.cephfs_test_runner: raise socket.timeout() 2022-06-29T16:23:01.344 INFO:tasks.cephfs_test_runner:socket.timeout 2022-06-29T16:23:01.345 INFO:tasks.cephfs_test_runner: 2022-06-29T16:23:01.345 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/run_tasks.py", line 106, in run_tasks manager.__enter__() File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_f25ffac72e5a9e63860ff7deb6767a11f0559a16/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc), test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2022-06-29T16:23:01.473 ERROR:teuthology.run_tasks: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=6a0b9bf7749c4f699a15b58241c8ed15 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/run_tasks.py", line 106, in run_tasks manager.__enter__() File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_f25ffac72e5a9e63860ff7deb6767a11f0559a16/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc), test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2022-06-29T16:23:01.475 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2022-06-29T16:23:01.488 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2022-06-29T16:23:01.521 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2022-06-29T16:23:01.522 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T16:23:01.522 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T16:23:01.523 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T16:23:17.348 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:23:17.352 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:23:17.355 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:23:17.358 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:23:47.429 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:23:47.433 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:23:47.436 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:23:47.439 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:24:17.525 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:24:17.529 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:24:17.532 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:24:17.534 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:24:47.595 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:24:47.599 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:24:47.602 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:24:47.604 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:25:17.658 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:25:17.661 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:25:17.664 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:25:17.667 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:25:47.763 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:25:47.767 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:25:47.769 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:25:47.772 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:26:17.834 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:26:17.838 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:26:17.840 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:26:17.843 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:26:47.937 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:26:47.941 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:26:47.944 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:26:47.946 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:27:18.002 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:27:18.006 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:27:18.008 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:27:18.011 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:27:48.071 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:27:48.074 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:27:48.077 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:27:48.080 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:28:18.131 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:28:18.135 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:28:18.139 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:28:18.142 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:28:48.201 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:28:48.205 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:28:48.208 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:28:48.211 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:29:18.278 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:29:18.281 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:29:18.284 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:29:18.287 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:29:48.341 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:29:48.344 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:29:48.347 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:29:48.350 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:30:18.420 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:30:18.424 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:30:18.427 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:30:18.429 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:30:48.543 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:30:48.547 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:30:48.550 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:30:48.552 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:31:18.630 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:31:18.634 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:31:18.637 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:31:18.639 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:31:48.750 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:31:48.754 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:31:48.757 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:31:48.760 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:32:18.813 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:32:18.817 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:32:18.819 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:32:18.822 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:32:48.873 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:32:48.877 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:32:48.880 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:32:48.882 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:33:18.954 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:33:18.957 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:33:18.960 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:33:18.962 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:33:49.015 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:33:49.019 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:33:49.021 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:33:49.024 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:34:19.074 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:34:19.077 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:34:19.080 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:34:19.083 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:34:49.144 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:34:49.147 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:34:49.150 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:34:49.152 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:35:19.221 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:35:19.224 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:35:19.227 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:35:19.229 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:35:49.278 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:35:49.282 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:35:49.285 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:35:49.287 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:36:19.343 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:36:19.347 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:36:19.350 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:36:19.352 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:36:49.421 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:36:49.425 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:36:49.427 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:36:49.430 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:37:19.507 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:37:19.511 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:37:19.514 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:37:19.516 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:37:49.571 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:37:49.575 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:37:49.578 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:37:49.580 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:38:01.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T16:38:01.536 INFO:teuthology.orchestra.run.smithi099.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': Connection timed out 2022-06-29T16:38:01.538 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.0 2022-06-29T16:38:01.539 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-06-29T16:38:01.539 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-06-29T16:38:01.540 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-06-29T16:38:19.634 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:38:19.638 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:38:19.641 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:38:19.644 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:38:49.735 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:38:49.740 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:38:49.743 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:38:49.746 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:39:19.802 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:39:19.805 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:39:19.808 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:39:19.811 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:39:49.864 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:39:49.869 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:39:49.872 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:39:49.874 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:40:19.928 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:40:19.932 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:40:19.934 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:40:19.937 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:40:49.986 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:40:49.989 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:40:49.992 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:40:49.995 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:41:20.096 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:41:20.099 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:41:20.102 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:41:20.105 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:41:50.184 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:41:50.188 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:41:50.190 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:41:50.193 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:42:20.256 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:42:20.260 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:42:20.264 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:42:20.268 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:42:50.416 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:42:50.420 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:42:50.423 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:42:50.426 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:43:20.493 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:43:20.496 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:43:20.499 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:43:20.502 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:43:50.560 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:43:50.564 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:43:50.567 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:43:50.570 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:44:20.627 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:44:20.631 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:44:20.634 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:44:20.637 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:44:50.723 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:44:50.727 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:44:50.730 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:44:50.733 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:45:20.803 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:45:20.807 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:45:20.809 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:45:20.812 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:45:50.871 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:45:50.875 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:45:50.878 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:45:50.880 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:46:20.930 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:46:20.934 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:46:20.937 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:46:20.939 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:46:51.007 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:46:51.012 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:46:51.015 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:46:51.017 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:47:21.072 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:47:21.076 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:47:21.078 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:47:21.081 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:47:51.131 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:47:51.135 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:47:51.138 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:47:51.140 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:48:21.194 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:48:21.198 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:48:21.202 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:48:21.205 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:48:51.259 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:48:51.263 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:48:51.265 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:48:51.268 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:49:21.330 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:49:21.334 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:49:21.336 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:49:21.339 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:49:51.385 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:49:51.389 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:49:51.392 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:49:51.394 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:50:21.447 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:50:21.450 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:50:21.453 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:50:21.456 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:50:51.510 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:50:51.513 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:50:51.523 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:50:51.525 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:51:21.589 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:51:21.593 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:51:21.596 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:51:21.598 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:51:51.677 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:51:51.681 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:51:51.683 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:51:51.686 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:52:21.741 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:52:21.745 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:52:21.747 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:52:21.750 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:52:51.832 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:52:51.838 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:52:51.842 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:52:51.846 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:53:01.545 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2022-06-29T16:53:01.563 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 699, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1361, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 701, in recv raise socket.timeout() socket.timeout 2022-06-29T16:53:01.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T16:53:01.594 ERROR:teuthology.run_tasks:Manager failed: ceph-fuse Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/run_tasks.py", line 188, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_f25ffac72e5a9e63860ff7deb6767a11f0559a16/qa/tasks/ceph_fuse.py", line 160, in task mount.umount_wait() File "/home/teuthworker/src/github.com_ceph_ceph-c_f25ffac72e5a9e63860ff7deb6767a11f0559a16/qa/tasks/cephfs/fuse_mount.py", line 326, in umount_wait if not (self.is_mounted() and self.fuse_daemon): File "/home/teuthworker/src/github.com_ceph_ceph-c_f25ffac72e5a9e63860ff7deb6767a11f0559a16/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted proc.wait() File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/orchestra/run.py", line 148, in wait greenlet.get(block=True,timeout=60) File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise raise value.with_traceback(tb) File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2022-06-29T16:53:01.606 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2022-06-29T16:53:01.622 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2022-06-29T16:53:01.624 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T16:53:01.753 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.748+0000 7f109dc79700 1 Processor -- start 2022-06-29T16:53:01.754 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.748+0000 7f109dc79700 1 -- start start 2022-06-29T16:53:01.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.748+0000 7f109dc79700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f10980671d0 0x7f10980b2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:01.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.748+0000 7f109dc79700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1098067630 0x7f1098144730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:01.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.748+0000 7f109dc79700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10980b4800 con 0x7f10980b4420 2022-06-29T16:53:01.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.748+0000 7f109dc79700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10980b3300 con 0x7f10980671d0 2022-06-29T16:53:01.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.748+0000 7f109dc79700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10980b34f0 con 0x7f1098067630 2022-06-29T16:53:01.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.748+0000 7f10977fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f10980b4420 0x7f1098066ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41076/0 (socket says 172.21.15.47:41076) 2022-06-29T16:53:01.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.748+0000 7f10977fe700 1 -- 172.21.15.47:0/2841810064 learned_addr learned my addr 172.21.15.47:0/2841810064 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T16:53:01.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.748+0000 7f1096ffd700 1 --2- 172.21.15.47:0/2841810064 >> v2:172.21.15.64:3300/0 conn(0x7f10980671d0 0x7f10980b2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:01.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.748+0000 7f1097fff700 1 --2- 172.21.15.47:0/2841810064 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1098067630 0x7f1098144730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:01.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.748+0000 7f10967fc700 1 -- 172.21.15.47:0/2841810064 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2287636270 0 0) 0x7f10980b4800 con 0x7f10980b4420 2022-06-29T16:53:01.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.748+0000 7f10967fc700 1 -- 172.21.15.47:0/2841810064 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f107c003260 con 0x7f10980b4420 2022-06-29T16:53:01.761 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.748+0000 7f1096ffd700 1 -- 172.21.15.47:0/2841810064 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1098067630 msgr2=0x7f1098144730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T16:53:01.761 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.748+0000 7f1096ffd700 1 --2- 172.21.15.47:0/2841810064 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1098067630 0x7f1098144730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:01.761 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.748+0000 7f1096ffd700 1 -- 172.21.15.47:0/2841810064 >> v1:172.21.15.47:6789/0 conn(0x7f10980b4420 legacy=0x7f1098066ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:01.762 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.748+0000 7f10967fc700 1 -- 172.21.15.47:0/2841810064 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2435123077 0 0) 0x7f107c003260 con 0x7f10980b4420 2022-06-29T16:53:01.762 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.748+0000 7f1096ffd700 1 -- 172.21.15.47:0/2841810064 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10980b3ab0 con 0x7f10980671d0 2022-06-29T16:53:01.763 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.748+0000 7f1096ffd700 1 --2- 172.21.15.47:0/2841810064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10980671d0 0x7f10980b2800 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7f1080001800 tx=0x7f10880091d0).ready entity=mon.1 client_cookie=44a1129a43157b82 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:01.763 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.748+0000 7f10967fc700 1 -- 172.21.15.47:0/2841810064 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1080001660 con 0x7f10980671d0 2022-06-29T16:53:01.763 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.748+0000 7f10967fc700 1 -- 172.21.15.47:0/2841810064 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f108800f510 con 0x7f10980671d0 2022-06-29T16:53:01.764 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.748+0000 7f10967fc700 1 -- 172.21.15.47:0/2841810064 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f108800f680 con 0x7f10980671d0 2022-06-29T16:53:01.764 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.748+0000 7f109dc79700 1 -- 172.21.15.47:0/2841810064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10980671d0 msgr2=0x7f10980b2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:01.764 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.748+0000 7f109dc79700 1 --2- 172.21.15.47:0/2841810064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10980671d0 0x7f10980b2800 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7f1080001800 tx=0x7f10880091d0).stop 2022-06-29T16:53:01.765 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.748+0000 7f109dc79700 1 -- 172.21.15.47:0/2841810064 shutdown_connections 2022-06-29T16:53:01.765 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.748+0000 7f109dc79700 1 --2- 172.21.15.47:0/2841810064 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1098067630 0x7f1098144730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:01.766 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.748+0000 7f109dc79700 1 --2- 172.21.15.47:0/2841810064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10980671d0 0x7f10980b2800 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:01.766 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.748+0000 7f109dc79700 1 -- 172.21.15.47:0/2841810064 >> 172.21.15.47:0/2841810064 conn(0x7f109805c510 msgr2=0x7f109805c920 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T16:53:01.766 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.748+0000 7f109dc79700 1 -- 172.21.15.47:0/2841810064 shutdown_connections 2022-06-29T16:53:01.767 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.748+0000 7f109dc79700 1 -- 172.21.15.47:0/2841810064 wait complete. 2022-06-29T16:53:01.770 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.752+0000 7f109dc79700 1 Processor -- start 2022-06-29T16:53:01.770 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.752+0000 7f109dc79700 1 -- start start 2022-06-29T16:53:01.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.752+0000 7f109dc79700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10980671d0 0x7f10980608f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:01.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.752+0000 7f109dc79700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1098067630 0x7f10980b0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:01.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.752+0000 7f109dc79700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10980b4420 0x7f109814a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:01.772 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.752+0000 7f109dc79700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1098148b90 con 0x7f10980671d0 2022-06-29T16:53:01.772 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.752+0000 7f109dc79700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109814db70 con 0x7f10980b4420 2022-06-29T16:53:01.772 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.752+0000 7f109dc79700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1098060f30 con 0x7f1098067630 2022-06-29T16:53:01.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.752+0000 7f10977fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10980671d0 0x7f10980608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:01.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.752+0000 7f10977fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10980671d0 0x7f10980608f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55530/0 (socket says 172.21.15.47:55530) 2022-06-29T16:53:01.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.752+0000 7f10977fe700 1 -- 172.21.15.47:0/3777406629 learned_addr learned my addr 172.21.15.47:0/3777406629 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T16:53:01.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.752+0000 7f1096ffd700 1 --2- 172.21.15.47:0/3777406629 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1098067630 0x7f10980b0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:01.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.752+0000 7f1097fff700 1 --2- 172.21.15.47:0/3777406629 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10980b4420 0x7f109814a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:01.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.752+0000 7f10977fe700 1 -- 172.21.15.47:0/3777406629 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1098067630 msgr2=0x7f10980b0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:01.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.752+0000 7f10977fe700 1 --2- 172.21.15.47:0/3777406629 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1098067630 0x7f10980b0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:01.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.752+0000 7f10977fe700 1 -- 172.21.15.47:0/3777406629 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10980b4420 msgr2=0x7f109814a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:01.776 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.752+0000 7f10977fe700 1 --2- 172.21.15.47:0/3777406629 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10980b4420 0x7f109814a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:01.776 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.752+0000 7f10977fe700 1 -- 172.21.15.47:0/3777406629 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1088009470 con 0x7f10980671d0 2022-06-29T16:53:01.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.752+0000 7f10977fe700 1 --2- 172.21.15.47:0/3777406629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10980671d0 0x7f10980608f0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 rx=0x7f1080001430 tx=0x7f1080001a50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:01.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.752+0000 7f1094ff9700 1 -- 172.21.15.47:0/3777406629 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1080007170 con 0x7f10980671d0 2022-06-29T16:53:01.777 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.752+0000 7f1094ff9700 1 -- 172.21.15.47:0/3777406629 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10800072d0 con 0x7f10980671d0 2022-06-29T16:53:01.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.756+0000 7f109dc79700 1 -- 172.21.15.47:0/3777406629 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10980b1320 con 0x7f10980671d0 2022-06-29T16:53:01.778 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.756+0000 7f1094ff9700 1 -- 172.21.15.47:0/3777406629 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1080004920 con 0x7f10980671d0 2022-06-29T16:53:01.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.756+0000 7f109dc79700 1 -- 172.21.15.47:0/3777406629 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10980af5a0 con 0x7f10980671d0 2022-06-29T16:53:01.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.756+0000 7f109dc79700 1 -- 172.21.15.47:0/3777406629 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1098061290 con 0x7f10980671d0 2022-06-29T16:53:01.779 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.756+0000 7f1094ff9700 1 -- 172.21.15.47:0/3777406629 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f108001b070 con 0x7f10980671d0 2022-06-29T16:53:01.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.756+0000 7f1094ff9700 1 --2- 172.21.15.47:0/3777406629 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1070053a70 0x7f1070055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:01.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.756+0000 7f1094ff9700 1 -- 172.21.15.47:0/3777406629 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f108006b670 con 0x7f10980671d0 2022-06-29T16:53:01.780 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.756+0000 7f1096ffd700 1 --2- 172.21.15.47:0/3777406629 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1070053a70 0x7f1070055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:01.781 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.760+0000 7f1096ffd700 1 --2- 172.21.15.47:0/3777406629 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1070053a70 0x7f1070055f30 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7f108801b000 tx=0x7f1088016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:01.781 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.764+0000 7f1094ff9700 1 -- 172.21.15.47:0/3777406629 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1098061290 con 0x7f10980671d0 2022-06-29T16:53:01.902 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.896+0000 7f109dc79700 1 -- 172.21.15.47:0/3777406629 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1098061290 con 0x7f10980671d0 2022-06-29T16:53:01.903 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.896+0000 7f1094ff9700 1 -- 172.21.15.47:0/3777406629 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v165) v1 ==== 74+0+13551 (secure 0 0 0) 0x7f1080042600 con 0x7f10980671d0 2022-06-29T16:53:01.904 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T16:53:01.904 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":165,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:52:31.023695+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{"172.21.15.64:6833/642573447":"2022-06-30T15:52:27.555481+0000","172.21.15.64:6832/642573447":"2022-06-30T15:52:27.555481+0000"},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T16:53:01.906 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.900+0000 7f109dc79700 1 -- 172.21.15.47:0/3777406629 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1070053a70 msgr2=0x7f1070055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:01.907 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.900+0000 7f109dc79700 1 --2- 172.21.15.47:0/3777406629 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1070053a70 0x7f1070055f30 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7f108801b000 tx=0x7f1088016000).stop 2022-06-29T16:53:01.907 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.900+0000 7f109dc79700 1 -- 172.21.15.47:0/3777406629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10980671d0 msgr2=0x7f10980608f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:01.907 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.900+0000 7f109dc79700 1 --2- 172.21.15.47:0/3777406629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10980671d0 0x7f10980608f0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 rx=0x7f1080001430 tx=0x7f1080001a50).stop 2022-06-29T16:53:01.908 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.900+0000 7f109dc79700 1 -- 172.21.15.47:0/3777406629 shutdown_connections 2022-06-29T16:53:01.908 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.900+0000 7f109dc79700 1 --2- 172.21.15.47:0/3777406629 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f1070053a70 0x7f1070055f30 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:01.908 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.900+0000 7f109dc79700 1 --2- 172.21.15.47:0/3777406629 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10980b4420 0x7f109814a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:01.909 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.900+0000 7f109dc79700 1 --2- 172.21.15.47:0/3777406629 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1098067630 0x7f10980b0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:01.909 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.900+0000 7f109dc79700 1 --2- 172.21.15.47:0/3777406629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10980671d0 0x7f10980608f0 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:01.909 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.900+0000 7f109dc79700 1 -- 172.21.15.47:0/3777406629 >> 172.21.15.47:0/3777406629 conn(0x7f109805c510 msgr2=0x7f10980647b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T16:53:01.910 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.900+0000 7f109dc79700 1 -- 172.21.15.47:0/3777406629 shutdown_connections 2022-06-29T16:53:01.910 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:01.900+0000 7f109dc79700 1 -- 172.21.15.47:0/3777406629 wait complete. 2022-06-29T16:53:01.918 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2022-06-29T16:53:02.024 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.016+0000 7f7764a27700 1 Processor -- start 2022-06-29T16:53:02.024 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.016+0000 7f7764a27700 1 -- start start 2022-06-29T16:53:02.028 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.020+0000 7f7764a27700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f77600b7840 0x7f77600b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:02.028 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.020+0000 7f7764a27700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7760062dd0 0x7f7760066c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:02.029 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.020+0000 7f7764a27700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77600b7c20 con 0x7f77600aea80 2022-06-29T16:53:02.029 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.020+0000 7f7764a27700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77600b3d10 con 0x7f77600b7840 2022-06-29T16:53:02.029 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.020+0000 7f7764a27700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77600670f0 con 0x7f7760062dd0 2022-06-29T16:53:02.030 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.020+0000 7f775ffff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f77600aea80 0x7f7760141760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41094/0 (socket says 172.21.15.47:41094) 2022-06-29T16:53:02.030 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.020+0000 7f775f7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f77600b7840 0x7f77600b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:02.031 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.020+0000 7f775ffff700 1 -- 172.21.15.47:0/1677031125 learned_addr learned my addr 172.21.15.47:0/1677031125 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T16:53:02.032 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.020+0000 7f775effd700 1 --2- 172.21.15.47:0/1677031125 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7760062dd0 0x7f7760066c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:02.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.020+0000 7f775e7fc700 1 -- 172.21.15.47:0/1677031125 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2584005261 0 0) 0x7f77600b7c20 con 0x7f77600aea80 2022-06-29T16:53:02.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.020+0000 7f775e7fc700 1 -- 172.21.15.47:0/1677031125 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7740003260 con 0x7f77600aea80 2022-06-29T16:53:02.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.020+0000 7f775effd700 1 -- 172.21.15.47:0/1677031125 >> v2:172.21.15.47:3300/0 conn(0x7f77600b7840 msgr2=0x7f77600b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T16:53:02.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.020+0000 7f775effd700 1 --2- 172.21.15.47:0/1677031125 >> v2:172.21.15.47:3300/0 conn(0x7f77600b7840 0x7f77600b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:02.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.020+0000 7f775effd700 1 -- 172.21.15.47:0/1677031125 >> v1:172.21.15.47:6789/0 conn(0x7f77600aea80 legacy=0x7f7760141760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:02.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.020+0000 7f775effd700 1 -- 172.21.15.47:0/1677031125 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7760067380 con 0x7f7760062dd0 2022-06-29T16:53:02.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.020+0000 7f775effd700 1 --2- 172.21.15.47:0/1677031125 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7760062dd0 0x7f7760066c00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f7750002f70 tx=0x7f7750006e80).ready entity=mon.2 client_cookie=c4466daac67d4c9a server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:02.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.020+0000 7f775e7fc700 1 -- 172.21.15.47:0/1677031125 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7750012070 con 0x7f7760062dd0 2022-06-29T16:53:02.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.020+0000 7f775e7fc700 1 -- 172.21.15.47:0/1677031125 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7750000e90 con 0x7f7760062dd0 2022-06-29T16:53:02.038 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.020+0000 7f775e7fc700 1 -- 172.21.15.47:0/1677031125 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7750017850 con 0x7f7760062dd0 2022-06-29T16:53:02.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.020+0000 7f7764a27700 1 -- 172.21.15.47:0/1677031125 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7760062dd0 msgr2=0x7f7760066c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:02.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.020+0000 7f7764a27700 1 --2- 172.21.15.47:0/1677031125 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7760062dd0 0x7f7760066c00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f7750002f70 tx=0x7f7750006e80).stop 2022-06-29T16:53:02.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.020+0000 7f7764a27700 1 -- 172.21.15.47:0/1677031125 shutdown_connections 2022-06-29T16:53:02.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.020+0000 7f7764a27700 1 --2- 172.21.15.47:0/1677031125 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7760062dd0 0x7f7760066c00 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:02.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.020+0000 7f7764a27700 1 --2- 172.21.15.47:0/1677031125 >> v2:172.21.15.47:3300/0 conn(0x7f77600b7840 0x7f77600b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:02.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.020+0000 7f7764a27700 1 -- 172.21.15.47:0/1677031125 >> 172.21.15.47:0/1677031125 conn(0x7f776005f480 msgr2=0x7f77600647d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T16:53:02.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.020+0000 7f7764a27700 1 -- 172.21.15.47:0/1677031125 shutdown_connections 2022-06-29T16:53:02.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.020+0000 7f7764a27700 1 -- 172.21.15.47:0/1677031125 wait complete. 2022-06-29T16:53:02.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.024+0000 7f7764a27700 1 Processor -- start 2022-06-29T16:53:02.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.024+0000 7f7764a27700 1 -- start start 2022-06-29T16:53:02.052 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.024+0000 7f7764a27700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7760062dd0 0x7f7760143f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:02.053 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.024+0000 7f7764a27700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f77600aea80 0x7f77601422b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:02.053 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.024+0000 7f7764a27700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77600b7840 0x7f7760140920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:02.054 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.024+0000 7f7764a27700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f776009f790 con 0x7f77600b7840 2022-06-29T16:53:02.055 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.024+0000 7f7764a27700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f776009edb0 con 0x7f7760062dd0 2022-06-29T16:53:02.056 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.024+0000 7f7764a27700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f776009f050 con 0x7f77600aea80 2022-06-29T16:53:02.057 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.024+0000 7f775ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77600b7840 0x7f7760140920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:02.057 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.024+0000 7f775ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77600b7840 0x7f7760140920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55544/0 (socket says 172.21.15.47:55544) 2022-06-29T16:53:02.058 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.024+0000 7f775ffff700 1 -- 172.21.15.47:0/2057825205 learned_addr learned my addr 172.21.15.47:0/2057825205 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T16:53:02.059 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.024+0000 7f775f7fe700 1 --2- 172.21.15.47:0/2057825205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7760062dd0 0x7f7760143f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:02.059 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.024+0000 7f775effd700 1 --2- 172.21.15.47:0/2057825205 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f77600aea80 0x7f77601422b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:02.060 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.024+0000 7f775f7fe700 1 -- 172.21.15.47:0/2057825205 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f77600aea80 msgr2=0x7f77601422b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:02.061 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.024+0000 7f775f7fe700 1 --2- 172.21.15.47:0/2057825205 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f77600aea80 0x7f77601422b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:02.061 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.024+0000 7f775f7fe700 1 -- 172.21.15.47:0/2057825205 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77600b7840 msgr2=0x7f7760140920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:02.062 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.024+0000 7f775f7fe700 1 --2- 172.21.15.47:0/2057825205 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77600b7840 0x7f7760140920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:02.062 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.024+0000 7f775f7fe700 1 -- 172.21.15.47:0/2057825205 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f775000d590 con 0x7f7760062dd0 2022-06-29T16:53:02.063 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.024+0000 7f775ffff700 1 --2- 172.21.15.47:0/2057825205 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77600b7840 0x7f7760140920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T16:53:02.064 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.024+0000 7f775f7fe700 1 --2- 172.21.15.47:0/2057825205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7760062dd0 0x7f7760143f00 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7f774800c000 tx=0x7f7748006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:02.064 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.024+0000 7f775cff9700 1 -- 172.21.15.47:0/2057825205 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7748011750 con 0x7f7760062dd0 2022-06-29T16:53:02.065 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.024+0000 7f775cff9700 1 -- 172.21.15.47:0/2057825205 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77480118b0 con 0x7f7760062dd0 2022-06-29T16:53:02.066 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.024+0000 7f775cff9700 1 -- 172.21.15.47:0/2057825205 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7748011c10 con 0x7f7760062dd0 2022-06-29T16:53:02.066 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.024+0000 7f7764a27700 1 -- 172.21.15.47:0/2057825205 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f776009f2f0 con 0x7f7760062dd0 2022-06-29T16:53:02.067 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.024+0000 7f7764a27700 1 -- 172.21.15.47:0/2057825205 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7760143020 con 0x7f7760062dd0 2022-06-29T16:53:02.067 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.028+0000 7f7764a27700 1 -- 172.21.15.47:0/2057825205 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f776009f480 con 0x7f7760062dd0 2022-06-29T16:53:02.068 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.028+0000 7f775cff9700 1 -- 172.21.15.47:0/2057825205 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7748011d70 con 0x7f7760062dd0 2022-06-29T16:53:02.069 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.028+0000 7f775cff9700 1 --2- 172.21.15.47:0/2057825205 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7744053950 0x7f7744055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:02.069 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.028+0000 7f775cff9700 1 -- 172.21.15.47:0/2057825205 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f774806c100 con 0x7f7760062dd0 2022-06-29T16:53:02.070 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.028+0000 7f775effd700 1 --2- 172.21.15.47:0/2057825205 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7744053950 0x7f7744055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:02.070 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.028+0000 7f775effd700 1 --2- 172.21.15.47:0/2057825205 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7744053950 0x7f7744055e10 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 rx=0x7f775001d000 tx=0x7f77600b18c0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:02.071 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.040+0000 7f775cff9700 1 -- 172.21.15.47:0/2057825205 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f77480430a0 con 0x7f7760062dd0 2022-06-29T16:53:02.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.164+0000 7f7764a27700 1 -- 172.21.15.47:0/2057825205 --> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f776009ec30 con 0x7f7744053950 2022-06-29T16:53:02.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.172+0000 7f775cff9700 1 -- 172.21.15.47:0/2057825205 <== mgr.4102 v2:172.21.15.47:6800/12928 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+67513 (secure 0 0 0) 0x7f7750006940 con 0x7f7744053950 2022-06-29T16:53:02.182 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T16:53:02.183 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.176+0000 7f7764a27700 1 -- 172.21.15.47:0/2057825205 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7744053950 msgr2=0x7f7744055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:02.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.176+0000 7f7764a27700 1 --2- 172.21.15.47:0/2057825205 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7744053950 0x7f7744055e10 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 rx=0x7f775001d000 tx=0x7f77600b18c0).stop 2022-06-29T16:53:02.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.176+0000 7f7764a27700 1 -- 172.21.15.47:0/2057825205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7760062dd0 msgr2=0x7f7760143f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:02.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.176+0000 7f7764a27700 1 --2- 172.21.15.47:0/2057825205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7760062dd0 0x7f7760143f00 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7f774800c000 tx=0x7f7748006df0).stop 2022-06-29T16:53:02.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.176+0000 7f7764a27700 1 -- 172.21.15.47:0/2057825205 shutdown_connections 2022-06-29T16:53:02.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.176+0000 7f7764a27700 1 --2- 172.21.15.47:0/2057825205 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77600b7840 0x7f7760140920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:02.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.176+0000 7f7764a27700 1 --2- 172.21.15.47:0/2057825205 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f77600aea80 0x7f77601422b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:02.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.176+0000 7f7764a27700 1 --2- 172.21.15.47:0/2057825205 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7744053950 0x7f7744055e10 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:02.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.176+0000 7f7764a27700 1 --2- 172.21.15.47:0/2057825205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7760062dd0 0x7f7760143f00 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:02.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.176+0000 7f7764a27700 1 -- 172.21.15.47:0/2057825205 >> 172.21.15.47:0/2057825205 conn(0x7f776005f480 msgr2=0x7f77600aad00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T16:53:02.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.176+0000 7f7764a27700 1 -- 172.21.15.47:0/2057825205 shutdown_connections 2022-06-29T16:53:02.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.176+0000 7f7764a27700 1 -- 172.21.15.47:0/2057825205 wait complete. 2022-06-29T16:53:02.187 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2022-06-29T16:53:02.193 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":2278,"stamp":"2022-06-29T16:53:02.120959+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":8405056,"kb_used_data":16384,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746569664,"statfs":{"total":773094113280,"available":764487335936,"internally_reserved":8589934592,"allocated":16777216,"data_stored":8168800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003301"},"pg_stats":[{"pgid":"1.0","version":"16'2","reported_seq":143,"reported_epoch":164,"state":"active+clean","last_fresh":"2022-06-29T15:52:30.026079+0000","last_change":"2022-06-29T15:42:22.794564+0000","last_active":"2022-06-29T15:52:30.026079+0000","last_peered":"2022-06-29T15:52:30.026079+0000","last_clean":"2022-06-29T15:52:30.026079+0000","last_became_active":"2022-06-29T15:42:22.793734+0000","last_became_peered":"2022-06-29T15:42:22.793734+0000","last_unstale":"2022-06-29T15:52:30.026079+0000","last_undegraded":"2022-06-29T15:52:30.026079+0000","last_fullsized":"2022-06-29T15:52:30.026079+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:17.908903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:17.908903+0000","last_clean_scrub_stamp":"2022-06-29T15:42:17.908903+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":137,"reported_epoch":164,"state":"active+clean","last_fresh":"2022-06-29T15:52:30.437350+0000","last_change":"2022-06-29T15:42:27.002337+0000","last_active":"2022-06-29T15:52:30.437350+0000","last_peered":"2022-06-29T15:52:30.437350+0000","last_clean":"2022-06-29T15:52:30.437350+0000","last_became_active":"2022-06-29T15:42:27.001981+0000","last_became_peered":"2022-06-29T15:42:27.001981+0000","last_unstale":"2022-06-29T15:52:30.437350+0000","last_undegraded":"2022-06-29T15:52:30.437350+0000","last_fullsized":"2022-06-29T15:52:30.437350+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_clean_scrub_stamp":"2022-06-29T15:42:25.967805+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":138,"reported_epoch":164,"state":"active+clean","last_fresh":"2022-06-29T15:52:30.025975+0000","last_change":"2022-06-29T15:42:27.000351+0000","last_active":"2022-06-29T15:52:30.025975+0000","last_peered":"2022-06-29T15:52:30.025975+0000","last_clean":"2022-06-29T15:52:30.025975+0000","last_became_active":"2022-06-29T15:42:26.999820+0000","last_became_peered":"2022-06-29T15:42:26.999820+0000","last_unstale":"2022-06-29T15:52:30.025975+0000","last_undegraded":"2022-06-29T15:52:30.025975+0000","last_fullsized":"2022-06-29T15:52:30.025975+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_clean_scrub_stamp":"2022-06-29T15:42:25.967805+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":130,"reported_epoch":164,"state":"active+clean","last_fresh":"2022-06-29T15:52:30.438342+0000","last_change":"2022-06-29T15:42:27.554899+0000","last_active":"2022-06-29T15:52:30.438342+0000","last_peered":"2022-06-29T15:52:30.438342+0000","last_clean":"2022-06-29T15:52:30.438342+0000","last_became_active":"2022-06-29T15:42:27.554328+0000","last_became_peered":"2022-06-29T15:42:27.554328+0000","last_unstale":"2022-06-29T15:52:30.438342+0000","last_undegraded":"2022-06-29T15:52:30.438342+0000","last_fullsized":"2022-06-29T15:52:30.438342+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_clean_scrub_stamp":"2022-06-29T15:42:25.967805+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":137,"reported_epoch":164,"state":"active+clean","last_fresh":"2022-06-29T15:52:30.437346+0000","last_change":"2022-06-29T15:42:27.001536+0000","last_active":"2022-06-29T15:52:30.437346+0000","last_peered":"2022-06-29T15:52:30.437346+0000","last_clean":"2022-06-29T15:52:30.437346+0000","last_became_active":"2022-06-29T15:42:27.001089+0000","last_became_peered":"2022-06-29T15:42:27.001089+0000","last_unstale":"2022-06-29T15:52:30.437346+0000","last_undegraded":"2022-06-29T15:52:30.437346+0000","last_fullsized":"2022-06-29T15:52:30.437346+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_clean_scrub_stamp":"2022-06-29T15:42:25.967805+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":137,"reported_epoch":164,"state":"active+clean","last_fresh":"2022-06-29T15:52:30.439563+0000","last_change":"2022-06-29T15:42:27.721658+0000","last_active":"2022-06-29T15:52:30.439563+0000","last_peered":"2022-06-29T15:52:30.439563+0000","last_clean":"2022-06-29T15:52:30.439563+0000","last_became_active":"2022-06-29T15:42:27.720770+0000","last_became_peered":"2022-06-29T15:42:27.720770+0000","last_unstale":"2022-06-29T15:52:30.439563+0000","last_undegraded":"2022-06-29T15:52:30.439563+0000","last_fullsized":"2022-06-29T15:52:30.439563+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_clean_scrub_stamp":"2022-06-29T15:42:25.967805+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":138,"reported_epoch":164,"state":"active+clean","last_fresh":"2022-06-29T15:52:30.026620+0000","last_change":"2022-06-29T15:42:27.723852+0000","last_active":"2022-06-29T15:52:30.026620+0000","last_peered":"2022-06-29T15:52:30.026620+0000","last_clean":"2022-06-29T15:52:30.026620+0000","last_became_active":"2022-06-29T15:42:27.722958+0000","last_became_peered":"2022-06-29T15:42:27.722958+0000","last_unstale":"2022-06-29T15:52:30.026620+0000","last_undegraded":"2022-06-29T15:52:30.026620+0000","last_fullsized":"2022-06-29T15:52:30.026620+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_clean_scrub_stamp":"2022-06-29T15:42:25.967805+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":137,"reported_epoch":164,"state":"active+clean","last_fresh":"2022-06-29T15:52:30.439978+0000","last_change":"2022-06-29T15:42:26.998189+0000","last_active":"2022-06-29T15:52:30.439978+0000","last_peered":"2022-06-29T15:52:30.439978+0000","last_clean":"2022-06-29T15:52:30.439978+0000","last_became_active":"2022-06-29T15:42:26.997677+0000","last_became_peered":"2022-06-29T15:42:26.997677+0000","last_unstale":"2022-06-29T15:52:30.439978+0000","last_undegraded":"2022-06-29T15:52:30.439978+0000","last_fullsized":"2022-06-29T15:52:30.439978+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_clean_scrub_stamp":"2022-06-29T15:42:25.967805+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":133,"reported_epoch":164,"state":"active+clean","last_fresh":"2022-06-29T15:52:30.437874+0000","last_change":"2022-06-29T15:42:26.999647+0000","last_active":"2022-06-29T15:52:30.437874+0000","last_peered":"2022-06-29T15:52:30.437874+0000","last_clean":"2022-06-29T15:52:30.437874+0000","last_became_active":"2022-06-29T15:42:26.999119+0000","last_became_peered":"2022-06-29T15:42:26.999119+0000","last_unstale":"2022-06-29T15:52:30.437874+0000","last_undegraded":"2022-06-29T15:52:30.437874+0000","last_fullsized":"2022-06-29T15:52:30.437874+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_clean_scrub_stamp":"2022-06-29T15:42:25.967805+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":14,"seq":60129542995,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050632,"kb_used_data":2048,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321208,"statfs":{"total":96636764160,"available":95560916992,"internally_reserved":1073741824,"allocated":2097152,"data_stored":1021100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.84099999999999997,"15min":0.84699999999999998},"min":{"1min":0.58899999999999997,"5min":0.58099999999999996,"15min":0.54900000000000004},"max":{"1min":1.125,"5min":1.165,"15min":1.335},"last":1.091},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.68700000000000006,"5min":0.57199999999999995,"15min":0.49399999999999999},"max":{"1min":1.1479999999999999,"5min":1.151,"15min":1.282},"last":0.77900000000000003}]},{"osd":1,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.83199999999999996,"15min":0.83099999999999996},"min":{"1min":0.626,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0580000000000001,"5min":1.224,"15min":1.224},"last":1.1020000000000001},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.871,"15min":0.879},"min":{"1min":0.64500000000000002,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.403},"last":0.91800000000000004}]},{"osd":2,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.89400000000000002,"15min":0.89300000000000002},"min":{"1min":0.69699999999999995,"5min":0.60599999999999998,"15min":0.56799999999999995},"max":{"1min":1.107,"5min":1.194,"15min":1.2290000000000001},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.86599999999999999,"15min":0.88500000000000001},"min":{"1min":0.65400000000000003,"5min":0.65200000000000002,"15min":0.63800000000000001},"max":{"1min":1.071,"5min":1.2130000000000001,"15min":1.276},"last":0.72099999999999997}]},{"osd":3,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.873,"15min":0.876},"min":{"1min":0.626,"5min":0.60799999999999998,"15min":0.58699999999999997},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2929999999999999},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.042,"5min":1.0269999999999999,"15min":1.0209999999999999},"min":{"1min":0.86399999999999999,"5min":0.753,"15min":0.71999999999999997},"max":{"1min":1.212,"5min":1.25,"15min":1.3089999999999999},"last":1.0089999999999999}]},{"osd":4,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.94499999999999995,"15min":0.89800000000000002},"min":{"1min":0.65500000000000003,"5min":0.58599999999999997,"15min":0.53100000000000003},"max":{"1min":1.1930000000000001,"5min":1.337,"15min":1.337},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.92900000000000005,"15min":0.89600000000000002},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.501},"max":{"1min":1.1879999999999999,"5min":1.369,"15min":1.369},"last":0.93200000000000005}]},{"osd":6,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.95799999999999996,"15min":0.92900000000000005},"min":{"1min":0.63400000000000001,"5min":0.61199999999999999,"15min":0.59999999999999998},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.86899999999999999},{"interface":"front","average":{"1min":1.002,"5min":1.044,"15min":1.02},"min":{"1min":0.71599999999999997,"5min":0.623,"15min":0.54200000000000004},"max":{"1min":1.3080000000000001,"5min":1.4610000000000001,"15min":1.4770000000000001},"last":0.97799999999999998}]},{"osd":7,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":1.018,"15min":0.98399999999999999},"min":{"1min":0.75800000000000001,"5min":0.63,"15min":0.63},"max":{"1min":1.2969999999999999,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.1080000000000001,"15min":1.079},"min":{"1min":0.79600000000000004,"5min":0.76600000000000001,"15min":0.69999999999999996},"max":{"1min":1.2869999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":1.1859999999999999}]}]},{"osd":7,"up_from":14,"seq":60129542994,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050632,"kb_used_data":2048,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321208,"statfs":{"total":96636764160,"available":95560916992,"internally_reserved":1073741824,"allocated":2097152,"data_stored":1021100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 29 16:52:42 2022","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.877,"15min":0.88},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.48699999999999999},"max":{"1min":1.1679999999999999,"5min":1.21,"15min":1.2410000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.58199999999999996},"max":{"1min":1.1319999999999999,"5min":1.171,"15min":1.2569999999999999},"last":1.083}]},{"osd":1,"last update":"Wed Jun 29 16:52:42 2022","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.79600000000000004,"15min":0.79900000000000004},"min":{"1min":0.59599999999999997,"5min":0.57899999999999996,"15min":0.49299999999999999},"max":{"1min":0.997,"5min":1.1850000000000001,"15min":1.202},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.82199999999999995,"15min":0.82799999999999996},"min":{"1min":0.60399999999999998,"5min":0.59499999999999997,"15min":0.58199999999999996},"max":{"1min":1.0780000000000001,"5min":1.161,"15min":1.226},"last":0.97799999999999998}]},{"osd":2,"last update":"Wed Jun 29 16:52:42 2022","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.88400000000000001,"15min":0.88800000000000001},"min":{"1min":0.63900000000000001,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.04,"5min":1.1679999999999999,"15min":1.333},"last":1.0169999999999999},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.94899999999999995,"15min":0.94599999999999995},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.55900000000000005},"max":{"1min":1.1639999999999999,"5min":1.2250000000000001,"15min":1.2569999999999999},"last":0.82299999999999995}]},{"osd":3,"last update":"Wed Jun 29 16:52:42 2022","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.90700000000000003,"15min":0.90000000000000002},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.095,"5min":1.224,"15min":1.276},"last":0.749},{"interface":"front","average":{"1min":1.032,"5min":0.97099999999999997,"15min":0.97799999999999998},"min":{"1min":0.87,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.2050000000000001,"5min":1.347,"15min":1.347},"last":0.88400000000000001}]},{"osd":4,"last update":"Wed Jun 29 16:52:42 2022","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.80100000000000005,"15min":0.82799999999999996},"min":{"1min":0.61499999999999999,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.016,"5min":1.3049999999999999,"15min":1.349},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.66200000000000003,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.125,"5min":1.226,"15min":1.246},"last":0.91500000000000004}]},{"osd":5,"last update":"Wed Jun 29 16:52:42 2022","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.94799999999999995,"15min":0.94099999999999995},"min":{"1min":0.67900000000000005,"5min":0.56699999999999995,"15min":0.56599999999999995},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.306},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.93799999999999994,"5min":1.0009999999999999,"15min":0.98499999999999999},"min":{"1min":0.621,"5min":0.621,"15min":0.59299999999999997},"max":{"1min":1.151,"5min":1.2849999999999999,"15min":1.3200000000000001},"last":0.71299999999999997}]},{"osd":6,"last update":"Wed Jun 29 16:52:42 2022","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.0629999999999999,"15min":1.044},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68400000000000005},"max":{"1min":1.244,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":1.145},{"interface":"front","average":{"1min":0.99199999999999999,"5min":1.028,"15min":1.0169999999999999},"min":{"1min":0.74399999999999999,"5min":0.70499999999999996,"15min":0.65700000000000003},"max":{"1min":1.23,"5min":1.52,"15min":1.52},"last":1.0660000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542995,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050632,"kb_used_data":2048,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321208,"statfs":{"total":96636764160,"available":95560916992,"internally_reserved":1073741824,"allocated":2097152,"data_stored":1021100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 29 16:52:51 2022","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.83499999999999996,"15min":0.85799999999999998},"min":{"1min":0.57399999999999995,"5min":0.54500000000000004,"15min":0.53700000000000003},"max":{"1min":1.0369999999999999,"5min":1.1850000000000001,"15min":1.522},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.90800000000000003,"15min":0.93700000000000006},"min":{"1min":0.68300000000000005,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.135,"5min":1.349,"15min":1.5469999999999999},"last":0.90000000000000002}]},{"osd":1,"last update":"Wed Jun 29 16:52:51 2022","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.78800000000000003,"15min":0.79000000000000004},"min":{"1min":0.58799999999999997,"5min":0.54700000000000004,"15min":0.53200000000000003},"max":{"1min":1.0149999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79500000000000004,"15min":0.81100000000000005},"min":{"1min":0.629,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.038,"5min":1.1080000000000001,"15min":1.359},"last":0.74399999999999999}]},{"osd":2,"last update":"Wed Jun 29 16:52:51 2022","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.88600000000000001,"15min":0.88300000000000001},"min":{"1min":0.67300000000000004,"5min":0.59899999999999998,"15min":0.58799999999999997},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":1.0009999999999999},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93000000000000005,"15min":0.91800000000000004},"min":{"1min":0.72299999999999998,"5min":0.60299999999999998,"15min":0.58199999999999996},"max":{"1min":1.1379999999999999,"5min":1.2270000000000001,"15min":1.3680000000000001},"last":0.93000000000000005}]},{"osd":3,"last update":"Wed Jun 29 16:52:51 2022","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.879,"15min":0.875},"min":{"1min":0.629,"5min":0.621,"15min":0.60899999999999999},"max":{"1min":1.127,"5min":1.23,"15min":1.232},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.97799999999999998,"15min":0.97599999999999998},"min":{"1min":0.80100000000000005,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.1859999999999999,"5min":1.22,"15min":1.278},"last":1.1020000000000001}]},{"osd":4,"last update":"Wed Jun 29 16:52:51 2022","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.83899999999999997,"15min":0.84199999999999997},"min":{"1min":0.60899999999999999,"5min":0.58799999999999997,"15min":0.54700000000000004},"max":{"1min":0.98699999999999999,"5min":1.27,"15min":1.27},"last":0.77400000000000002},{"interface":"front","average":{"1min":1.034,"5min":0.97199999999999998,"15min":0.93200000000000005},"min":{"1min":0.79000000000000004,"5min":0.64500000000000002,"15min":0.57199999999999995},"max":{"1min":1.1839999999999999,"5min":1.298,"15min":1.3420000000000001},"last":0.97399999999999998}]},{"osd":5,"last update":"Wed Jun 29 16:52:51 2022","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93100000000000005,"15min":0.92000000000000004},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.58499999999999996},"max":{"1min":1.198,"5min":1.216,"15min":1.385},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.999,"15min":0.98399999999999999},"min":{"1min":0.83199999999999996,"5min":0.67100000000000004,"15min":0.63500000000000001},"max":{"1min":1.2989999999999999,"5min":1.3320000000000001,"15min":1.423},"last":1.0189999999999999}]},{"osd":7,"last update":"Wed Jun 29 16:52:51 2022","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.0449999999999999,"15min":1.034},"min":{"1min":0.72699999999999998,"5min":0.71099999999999997,"15min":0.63300000000000001},"max":{"1min":1.157,"5min":1.391,"15min":1.4019999999999999},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.107,"15min":1.097},"min":{"1min":0.97599999999999998,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.581},"last":1.2470000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542994,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1050632,"kb_used_data":2048,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321208,"statfs":{"total":96636764160,"available":95560916992,"internally_reserved":1073741824,"allocated":2097152,"data_stored":1021100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 29 16:52:23 2022","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.86199999999999999,"15min":0.86099999999999999},"min":{"1min":0.51200000000000001,"5min":0.505,"15min":0.505},"max":{"1min":1.0229999999999999,"5min":1.135,"15min":1.228},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.95099999999999996,"15min":0.93799999999999994},"min":{"1min":0.72499999999999998,"5min":0.65100000000000002,"15min":0.63600000000000001},"max":{"1min":1.089,"5min":1.181,"15min":1.2909999999999999},"last":1.028}]},{"osd":1,"last update":"Wed Jun 29 16:52:23 2022","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.81999999999999995,"15min":0.82799999999999996},"min":{"1min":0.59099999999999997,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.202},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.83499999999999996,"15min":0.83299999999999996},"min":{"1min":0.67400000000000004,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.145,"5min":1.1699999999999999,"15min":1.21},"last":0.86899999999999999}]},{"osd":2,"last update":"Wed Jun 29 16:52:23 2022","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.86499999999999999,"15min":0.871},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.85499999999999998,"15min":0.84599999999999997},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.232},"last":0.91800000000000004}]},{"osd":3,"last update":"Wed Jun 29 16:52:23 2022","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.91200000000000003,"15min":0.91800000000000004},"min":{"1min":0.68799999999999994,"5min":0.61799999999999999,"15min":0.60099999999999998},"max":{"1min":1.111,"5min":1.157,"15min":1.2470000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.014,"5min":1.004,"15min":1.004},"min":{"1min":0.80500000000000005,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.222,"5min":1.2430000000000001,"15min":1.52},"last":1.0629999999999999}]},{"osd":5,"last update":"Wed Jun 29 16:52:23 2022","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.80500000000000005,"15min":0.80800000000000005},"min":{"1min":0.56799999999999995,"5min":0.56000000000000005,"15min":0.52000000000000002},"max":{"1min":1.077,"5min":1.179,"15min":1.179},"last":1.002},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.89600000000000002,"15min":0.91400000000000003},"min":{"1min":0.60899999999999999,"5min":0.58399999999999996,"15min":0.54400000000000004},"max":{"1min":1.2090000000000001,"5min":1.2190000000000001,"15min":1.304},"last":0.94699999999999995}]},{"osd":6,"last update":"Wed Jun 29 16:52:23 2022","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.95099999999999996,"15min":0.94999999999999996},"min":{"1min":0.66000000000000003,"5min":0.63,"15min":0.58299999999999996},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.4390000000000001},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.028,"5min":1.004,"15min":1.0109999999999999},"min":{"1min":0.73599999999999999,"5min":0.72299999999999998,"15min":0.61799999999999999},"max":{"1min":1.1839999999999999,"5min":1.264,"15min":1.3009999999999999},"last":1.115}]},{"osd":7,"last update":"Wed Jun 29 16:52:23 2022","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.0029999999999999,"15min":1.0109999999999999},"min":{"1min":0.81000000000000005,"5min":0.66000000000000003,"15min":0.61599999999999999},"max":{"1min":1.2490000000000001,"5min":1.425,"15min":1.4930000000000001},"last":1.155},{"interface":"front","average":{"1min":1.054,"5min":1.04,"15min":1.042},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.496},"max":{"1min":1.663,"5min":1.663,"15min":1.663},"last":1.131}]}]},{"osd":3,"up_from":14,"seq":60129542994,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050632,"kb_used_data":2048,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321208,"statfs":{"total":96636764160,"available":95560916992,"internally_reserved":1073741824,"allocated":2097152,"data_stored":1021100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63200000000000001,"15min":0.64100000000000001},"min":{"1min":0.5,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.80400000000000005,"5min":1.075,"15min":1.103},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.755,"5min":0.78300000000000003,"15min":0.78000000000000003},"min":{"1min":0.55600000000000005,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.92600000000000005,"5min":1.194,"15min":1.232},"last":0.72599999999999998}]},{"osd":1,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.65300000000000002,"15min":0.64800000000000002},"min":{"1min":0.51200000000000001,"5min":0.48099999999999998,"15min":0.46899999999999997},"max":{"1min":0.96799999999999997,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77600000000000002,"15min":0.77400000000000002},"min":{"1min":0.53900000000000003,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.1220000000000001,"5min":1.288,"15min":1.288},"last":0.56200000000000006}]},{"osd":2,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.70599999999999996,"15min":0.68600000000000005},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.97899999999999998,"5min":1.365,"15min":1.365},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.85699999999999998,"15min":0.84099999999999997},"min":{"1min":0.68999999999999995,"5min":0.60299999999999998,"15min":0.57399999999999995},"max":{"1min":1.046,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.76200000000000001}]},{"osd":4,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.94199999999999995,"15min":0.93999999999999995},"min":{"1min":0.71599999999999997,"5min":0.71399999999999997,"15min":0.67600000000000005},"max":{"1min":1.1240000000000001,"5min":1.556,"15min":1.556},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.91800000000000004,"15min":0.91100000000000003},"min":{"1min":0.71299999999999997,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.0569999999999999,"5min":1.3169999999999999,"15min":1.3420000000000001},"last":1.3360000000000001}]},{"osd":5,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.96399999999999997,"15min":0.93999999999999995},"min":{"1min":0.72199999999999998,"5min":0.69999999999999996,"15min":0.68500000000000005},"max":{"1min":1.2390000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.024,"15min":1.016},"min":{"1min":0.82399999999999995,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.1970000000000001,"5min":1.3640000000000001,"15min":1.454},"last":0.86799999999999999}]},{"osd":6,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":1.0329999999999999,"15min":1.0229999999999999},"min":{"1min":0.745,"5min":0.73499999999999999,"15min":0.67200000000000004},"max":{"1min":1.337,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.88400000000000001},{"interface":"front","average":{"1min":1.028,"5min":1.0289999999999999,"15min":1.0249999999999999},"min":{"1min":0.77000000000000002,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.2849999999999999,"5min":1.496,"15min":1.508},"last":1.236}]},{"osd":7,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.82699999999999996,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4970000000000001},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.1379999999999999,"15min":1.1240000000000001},"min":{"1min":0.84199999999999997,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.3999999999999999,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":1.024}]}]},{"osd":2,"up_from":14,"seq":60129542994,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050632,"kb_used_data":2048,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321208,"statfs":{"total":96636764160,"available":95560916992,"internally_reserved":1073741824,"allocated":2097152,"data_stored":1021100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.628,"15min":0.624},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.42099999999999999},"max":{"1min":1.03,"5min":1.137,"15min":1.256},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.75,"5min":0.72699999999999998,"15min":0.70399999999999996},"min":{"1min":0.53600000000000003,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.099,"5min":1.151,"15min":1.151},"last":0.66500000000000004}]},{"osd":1,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.48399999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.91600000000000004,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.751,"15min":0.76500000000000001},"min":{"1min":0.54000000000000004,"5min":0.502,"15min":0.502},"max":{"1min":0.92100000000000004,"5min":1.131,"15min":1.131},"last":0.80100000000000005}]},{"osd":3,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.78500000000000003,"15min":0.80800000000000005},"min":{"1min":0.58499999999999996,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0900000000000001,"5min":1.133,"15min":1.1930000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.84399999999999997,"15min":0.83399999999999996},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.55600000000000005},"max":{"1min":1.1040000000000001,"5min":1.155,"15min":1.1990000000000001},"last":0.86699999999999999}]},{"osd":4,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.877,"15min":0.88400000000000001},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.2390000000000001,"5min":1.2509999999999999,"15min":1.323},"last":1.109},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.92600000000000005,"15min":0.93100000000000005},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.84499999999999997}]},{"osd":5,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":1.0209999999999999,"15min":1.0129999999999999},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.478,"5min":1.478,"15min":1.5169999999999999},"last":1.3340000000000001},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.98099999999999998,"15min":0.97699999999999998},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.64500000000000002},"max":{"1min":1.2130000000000001,"5min":1.3540000000000001,"15min":1.5029999999999999},"last":1.194}]},{"osd":6,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.032,"15min":1.0289999999999999},"min":{"1min":0.81000000000000005,"5min":0.752,"15min":0.69999999999999996},"max":{"1min":1.3260000000000001,"5min":1.464,"15min":1.53},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0980000000000001,"15min":1.093},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.75900000000000001},"max":{"1min":1.413,"5min":1.516,"15min":1.5669999999999999},"last":1.2789999999999999}]},{"osd":7,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.0700000000000001,"15min":1.0669999999999999},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":1.252},{"interface":"front","average":{"1min":1.123,"5min":1.109,"15min":1.109},"min":{"1min":0.94399999999999995,"5min":0.86299999999999999,"15min":0.85099999999999998},"max":{"1min":1.405,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":1.095}]}]},{"osd":1,"up_from":14,"seq":60129542994,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050632,"kb_used_data":2048,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321208,"statfs":{"total":96636764160,"available":95560916992,"internally_reserved":1073741824,"allocated":2097152,"data_stored":1021100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 29 16:52:38 2022","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.622,"15min":0.63600000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.47199999999999998},"max":{"1min":0.75600000000000001,"5min":1.024,"15min":1.1000000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70899999999999996,"15min":0.70199999999999996},"min":{"1min":0.54100000000000004,"5min":0.47999999999999998,"15min":0.47299999999999998},"max":{"1min":0.93000000000000005,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.752}]},{"osd":2,"last update":"Wed Jun 29 16:52:38 2022","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.65100000000000002,"15min":0.64800000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.46700000000000003},"max":{"1min":0.80000000000000004,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76300000000000001,"15min":0.76800000000000002},"min":{"1min":0.61699999999999999,"5min":0.54600000000000004,"15min":0.51700000000000002},"max":{"1min":0.98699999999999999,"5min":1.22,"15min":1.22},"last":0.96299999999999997}]},{"osd":3,"last update":"Wed Jun 29 16:52:38 2022","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.82399999999999995,"15min":0.84699999999999998},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.56100000000000005},"max":{"1min":1.01,"5min":1.109,"15min":1.157},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78900000000000003,"15min":0.77700000000000002},"min":{"1min":0.60599999999999998,"5min":0.52900000000000003,"15min":0.47699999999999998},"max":{"1min":0.95699999999999996,"5min":1.788,"15min":1.788},"last":0.65700000000000003}]},{"osd":4,"last update":"Wed Jun 29 16:52:38 2022","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.877,"15min":0.873},"min":{"1min":0.69899999999999995,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.248,"5min":1.288,"15min":1.4610000000000001},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.995,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.81000000000000005,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.24,"5min":1.405,"15min":1.4330000000000001},"last":1.0609999999999999}]},{"osd":5,"last update":"Wed Jun 29 16:52:38 2022","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.93899999999999995,"15min":0.96899999999999997},"min":{"1min":0.78100000000000003,"5min":0.71299999999999997,"15min":0.66700000000000004},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.4279999999999999},"last":1.278},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.95899999999999996,"15min":0.96599999999999997},"min":{"1min":0.75,"5min":0.72799999999999998,"15min":0.67300000000000004},"max":{"1min":1.167,"5min":1.357,"15min":1.3959999999999999},"last":0.99199999999999999}]},{"osd":6,"last update":"Wed Jun 29 16:52:38 2022","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.0209999999999999,"15min":1.0509999999999999},"min":{"1min":0.77300000000000002,"5min":0.76900000000000002,"15min":0.70499999999999996},"max":{"1min":1.375,"5min":1.375,"15min":1.516},"last":1.171},{"interface":"front","average":{"1min":1.014,"5min":1.0389999999999999,"15min":1.054},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.343,"5min":1.444,"15min":1.5649999999999999},"last":1.0369999999999999}]},{"osd":7,"last update":"Wed Jun 29 16:52:38 2022","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.0880000000000001,"15min":1.1020000000000001},"min":{"1min":0.77200000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.3149999999999999,"5min":1.4550000000000001,"15min":1.6100000000000001},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.0920000000000001,"15min":1.1020000000000001},"min":{"1min":0.92900000000000005,"5min":0.82099999999999995,"15min":0.72699999999999998},"max":{"1min":1.413,"5min":1.413,"15min":1.603},"last":1.238}]}]},{"osd":0,"up_from":14,"seq":60129542994,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050632,"kb_used_data":2048,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321208,"statfs":{"total":96636764160,"available":95560916992,"internally_reserved":1073741824,"allocated":2097152,"data_stored":1021100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jun 29 16:52:42 2022","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.64900000000000002,"15min":0.63500000000000001},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.45200000000000001},"max":{"1min":0.79100000000000004,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.62,"5min":0.64900000000000002,"15min":0.67100000000000004},"min":{"1min":0.52300000000000002,"5min":0.46800000000000003,"15min":0.45300000000000001},"max":{"1min":0.84499999999999997,"5min":0.98299999999999998,"15min":1.1259999999999999},"last":0.53100000000000003}]},{"osd":2,"last update":"Wed Jun 29 16:52:42 2022","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.71999999999999997,"15min":0.73499999999999999},"min":{"1min":0.51300000000000001,"5min":0.499,"15min":0.46800000000000003},"max":{"1min":1.0429999999999999,"5min":1.0549999999999999,"15min":1.2190000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.80700000000000005,"15min":0.82599999999999996},"min":{"1min":0.67500000000000004,"5min":0.60499999999999998,"15min":0.59699999999999998},"max":{"1min":1.0329999999999999,"5min":1.085,"15min":1.143},"last":0.67400000000000004}]},{"osd":3,"last update":"Wed Jun 29 16:52:42 2022","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.71299999999999997,"15min":0.70099999999999996},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.48899999999999999},"max":{"1min":0.97999999999999998,"5min":1.117,"15min":1.147},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.80600000000000005,"15min":0.82599999999999996},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.028,"5min":1.145,"15min":1.2090000000000001},"last":0.74199999999999999}]},{"osd":4,"last update":"Wed Jun 29 16:52:42 2022","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.91400000000000003,"15min":0.92400000000000004},"min":{"1min":0.73999999999999999,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.077,"5min":1.2769999999999999,"15min":1.3759999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.88600000000000001,"15min":0.91100000000000003},"min":{"1min":0.68600000000000005,"5min":0.67700000000000005,"15min":0.66600000000000004},"max":{"1min":1.1699999999999999,"5min":1.381,"15min":1.5},"last":1.1479999999999999}]},{"osd":5,"last update":"Wed Jun 29 16:52:42 2022","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.91400000000000003,"15min":0.93100000000000005},"min":{"1min":0.70099999999999996,"5min":0.66600000000000004,"15min":0.66100000000000003},"max":{"1min":1.266,"5min":1.4419999999999999,"15min":1.6040000000000001},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.032,"5min":1.0149999999999999,"15min":1.024},"min":{"1min":0.80300000000000005,"5min":0.752,"15min":0.67900000000000005},"max":{"1min":1.2210000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":1.1830000000000001}]},{"osd":6,"last update":"Wed Jun 29 16:52:42 2022","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.0309999999999999,"15min":1.052},"min":{"1min":0.76000000000000001,"5min":0.749,"15min":0.73599999999999999},"max":{"1min":1.2949999999999999,"5min":1.458,"15min":1.524},"last":1.2529999999999999},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.024,"15min":1.0509999999999999},"min":{"1min":0.82499999999999996,"5min":0.78700000000000003,"15min":0.70799999999999996},"max":{"1min":1.163,"5min":1.502,"15min":1.552},"last":0.94099999999999995}]},{"osd":7,"last update":"Wed Jun 29 16:52:42 2022","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.0680000000000001,"15min":1.0840000000000001},"min":{"1min":0.85999999999999999,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.2270000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.069,"15min":1.077},"min":{"1min":0.876,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.466,"5min":1.486,"15min":1.5960000000000001},"last":1.21}]}]}]}} 2022-06-29T16:53:02.196 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-06-29T16:53:02.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.292+0000 7f9f2e1c3700 1 Processor -- start 2022-06-29T16:53:02.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.292+0000 7f9f2e1c3700 1 -- start start 2022-06-29T16:53:02.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.296+0000 7f9f2e1c3700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9f2805c510 0x7f9f28143bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:02.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.296+0000 7f9f2e1c3700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f28142f90 con 0x7f9f2805c8f0 2022-06-29T16:53:02.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.296+0000 7f9f2e1c3700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f281431f0 con 0x7f9f280b2c40 2022-06-29T16:53:02.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.296+0000 7f9f2e1c3700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f28143490 con 0x7f9f2805c510 2022-06-29T16:53:02.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.296+0000 7f9f2c9c0700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9f2805c8f0 0x7f9f280b4100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41104/0 (socket says 172.21.15.47:41104) 2022-06-29T16:53:02.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.296+0000 7f9f2c9c0700 1 -- 172.21.15.47:0/3092475548 learned_addr learned my addr 172.21.15.47:0/3092475548 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T16:53:02.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.296+0000 7f9f2d1c1700 1 --2- 172.21.15.47:0/3092475548 >> v2:172.21.15.64:3300/0 conn(0x7f9f2805c510 0x7f9f28143bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:02.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.296+0000 7f9f17fff700 1 -- 172.21.15.47:0/3092475548 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2048262633 0 0) 0x7f9f28142f90 con 0x7f9f2805c8f0 2022-06-29T16:53:02.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.296+0000 7f9f17fff700 1 -- 172.21.15.47:0/3092475548 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f10003260 con 0x7f9f2805c8f0 2022-06-29T16:53:02.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.296+0000 7f9f2d1c1700 1 -- 172.21.15.47:0/3092475548 >> v1:172.21.15.64:6789/0 conn(0x7f9f280b2c40 legacy=0x7f9f2813cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:02.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.296+0000 7f9f2d1c1700 1 -- 172.21.15.47:0/3092475548 >> v1:172.21.15.47:6789/0 conn(0x7f9f2805c8f0 legacy=0x7f9f280b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:02.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.296+0000 7f9f2d1c1700 1 -- 172.21.15.47:0/3092475548 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f28143730 con 0x7f9f2805c510 2022-06-29T16:53:02.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.296+0000 7f9f2d1c1700 1 --2- 172.21.15.47:0/3092475548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f2805c510 0x7f9f28143bd0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7f9f24002f70 tx=0x7f9f24006df0).ready entity=mon.1 client_cookie=5d7e0b8b0658ddad server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:02.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.296+0000 7f9f17fff700 1 -- 172.21.15.47:0/3092475548 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f2400d610 con 0x7f9f2805c510 2022-06-29T16:53:02.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.296+0000 7f9f17fff700 1 -- 172.21.15.47:0/3092475548 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f24000e90 con 0x7f9f2805c510 2022-06-29T16:53:02.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.296+0000 7f9f17fff700 1 -- 172.21.15.47:0/3092475548 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f2400d960 con 0x7f9f2805c510 2022-06-29T16:53:02.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.296+0000 7f9f2e1c3700 1 -- 172.21.15.47:0/3092475548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f2805c510 msgr2=0x7f9f28143bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:02.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.296+0000 7f9f2e1c3700 1 --2- 172.21.15.47:0/3092475548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f2805c510 0x7f9f28143bd0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7f9f24002f70 tx=0x7f9f24006df0).stop 2022-06-29T16:53:02.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.296+0000 7f9f2e1c3700 1 -- 172.21.15.47:0/3092475548 shutdown_connections 2022-06-29T16:53:02.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.296+0000 7f9f2e1c3700 1 --2- 172.21.15.47:0/3092475548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f2805c510 0x7f9f28143bd0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:02.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.296+0000 7f9f2e1c3700 1 -- 172.21.15.47:0/3092475548 >> 172.21.15.47:0/3092475548 conn(0x7f9f28060e00 msgr2=0x7f9f280611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T16:53:02.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.296+0000 7f9f2e1c3700 1 -- 172.21.15.47:0/3092475548 shutdown_connections 2022-06-29T16:53:02.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.296+0000 7f9f2e1c3700 1 -- 172.21.15.47:0/3092475548 wait complete. 2022-06-29T16:53:02.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.300+0000 7f9f2e1c3700 1 Processor -- start 2022-06-29T16:53:02.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.300+0000 7f9f2e1c3700 1 -- start start 2022-06-29T16:53:02.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.300+0000 7f9f2e1c3700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9f2805c510 0x7f9f28146e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:02.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.300+0000 7f9f2e1c3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f2805c8f0 0x7f9f2813e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:02.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.300+0000 7f9f2e1c3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f280b2c40 0x7f9f2813e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:02.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.300+0000 7f9f2e1c3700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2813ee00 con 0x7f9f2805c8f0 2022-06-29T16:53:02.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.300+0000 7f9f2e1c3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2813f310 con 0x7f9f280b2c40 2022-06-29T16:53:02.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.300+0000 7f9f2e1c3700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2813f070 con 0x7f9f2805c510 2022-06-29T16:53:02.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.300+0000 7f9f2c9c0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f2805c8f0 0x7f9f2813e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:02.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.300+0000 7f9f2d9c2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f280b2c40 0x7f9f2813e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:02.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.300+0000 7f9f2d9c2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f280b2c40 0x7f9f2813e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:33030/0 (socket says 172.21.15.47:33030) 2022-06-29T16:53:02.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.300+0000 7f9f2c9c0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f2805c8f0 0x7f9f2813e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55560/0 (socket says 172.21.15.47:55560) 2022-06-29T16:53:02.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.300+0000 7f9f2d9c2700 1 -- 172.21.15.47:0/2777061461 learned_addr learned my addr 172.21.15.47:0/2777061461 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T16:53:02.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.300+0000 7f9f2d1c1700 1 --2- 172.21.15.47:0/2777061461 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9f2805c510 0x7f9f28146e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:02.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.300+0000 7f9f2d9c2700 1 -- 172.21.15.47:0/2777061461 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9f2805c510 msgr2=0x7f9f28146e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:02.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.300+0000 7f9f2d9c2700 1 --2- 172.21.15.47:0/2777061461 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9f2805c510 0x7f9f28146e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:02.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.300+0000 7f9f2d9c2700 1 -- 172.21.15.47:0/2777061461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f2805c8f0 msgr2=0x7f9f2813e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:02.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.300+0000 7f9f2d9c2700 1 --2- 172.21.15.47:0/2777061461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f2805c8f0 0x7f9f2813e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:02.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.300+0000 7f9f2d9c2700 1 -- 172.21.15.47:0/2777061461 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f2400b180 con 0x7f9f280b2c40 2022-06-29T16:53:02.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.300+0000 7f9f2c9c0700 1 --2- 172.21.15.47:0/2777061461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f2805c8f0 0x7f9f2813e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T16:53:02.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.300+0000 7f9f2d9c2700 1 --2- 172.21.15.47:0/2777061461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f280b2c40 0x7f9f2813e910 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7f9f2813ef40 tx=0x7f9f20007f40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:02.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.300+0000 7f9f167fc700 1 -- 172.21.15.47:0/2777061461 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f2000f470 con 0x7f9f280b2c40 2022-06-29T16:53:02.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.300+0000 7f9f2e1c3700 1 -- 172.21.15.47:0/2777061461 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f28152ec0 con 0x7f9f280b2c40 2022-06-29T16:53:02.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.300+0000 7f9f167fc700 1 -- 172.21.15.47:0/2777061461 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f2000f5d0 con 0x7f9f280b2c40 2022-06-29T16:53:02.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.300+0000 7f9f167fc700 1 -- 172.21.15.47:0/2777061461 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f2000f930 con 0x7f9f280b2c40 2022-06-29T16:53:02.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.300+0000 7f9f2e1c3700 1 -- 172.21.15.47:0/2777061461 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f28145ce0 con 0x7f9f280b2c40 2022-06-29T16:53:02.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.304+0000 7f9f2e1c3700 1 -- 172.21.15.47:0/2777061461 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f2813e220 con 0x7f9f280b2c40 2022-06-29T16:53:02.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.304+0000 7f9f167fc700 1 -- 172.21.15.47:0/2777061461 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9f20019030 con 0x7f9f280b2c40 2022-06-29T16:53:02.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.304+0000 7f9f167fc700 1 --2- 172.21.15.47:0/2777061461 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9f08053b00 0x7f9f08055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:02.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.304+0000 7f9f167fc700 1 -- 172.21.15.47:0/2777061461 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f9f2006b520 con 0x7f9f280b2c40 2022-06-29T16:53:02.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.304+0000 7f9f2d1c1700 1 --2- 172.21.15.47:0/2777061461 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9f08053b00 0x7f9f08055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:02.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.304+0000 7f9f2d1c1700 1 --2- 172.21.15.47:0/2777061461 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9f08053b00 0x7f9f08055fc0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 rx=0x7f9f2401a000 tx=0x7f9f24014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:02.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.312+0000 7f9f167fc700 1 -- 172.21.15.47:0/2777061461 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9f20042430 con 0x7f9f280b2c40 2022-06-29T16:53:02.445 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.440+0000 7f9f2e1c3700 1 -- 172.21.15.47:0/2777061461 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9f280573d0 con 0x7f9f280b2c40 2022-06-29T16:53:02.447 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.440+0000 7f9f167fc700 1 -- 172.21.15.47:0/2777061461 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v165) v1 ==== 74+0+13551 (secure 0 0 0) 0x7f9f20045a50 con 0x7f9f280b2c40 2022-06-29T16:53:02.447 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T16:53:02.448 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":165,"fsid":"9a68894a-53ce-400e-838b-14eab91b834c","created":"2022-06-29T15:42:14.046361+0000","modified":"2022-06-29T15:52:31.023695+0000","last_up_change":"2022-06-29T15:42:20.929789+0000","last_in_change":"2022-06-29T15:42:20.929789+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-06-29T15:42:16.971431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-06-29T15:42:25.290884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"e23eb023-4706-4201-bf72-4bf19c912c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":13096},{"type":"v1","addr":"172.21.15.47:6819","nonce":13096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":13096},{"type":"v1","addr":"172.21.15.47:6821","nonce":13096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":13096},{"type":"v1","addr":"172.21.15.47:6825","nonce":13096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":13096},{"type":"v1","addr":"172.21.15.47:6823","nonce":13096}]},"public_addr":"172.21.15.47:6819/13096","cluster_addr":"172.21.15.47:6821/13096","heartbeat_back_addr":"172.21.15.47:6825/13096","heartbeat_front_addr":"172.21.15.47:6823/13096","state":["exists","up"]},{"osd":1,"uuid":"228bb08d-2d64-43d9-9560-bfaeff154452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":13100},{"type":"v1","addr":"172.21.15.47:6827","nonce":13100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":13100},{"type":"v1","addr":"172.21.15.47:6829","nonce":13100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":13100},{"type":"v1","addr":"172.21.15.47:6833","nonce":13100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":13100},{"type":"v1","addr":"172.21.15.47:6831","nonce":13100}]},"public_addr":"172.21.15.47:6827/13100","cluster_addr":"172.21.15.47:6829/13100","heartbeat_back_addr":"172.21.15.47:6833/13100","heartbeat_front_addr":"172.21.15.47:6831/13100","state":["exists","up"]},{"osd":2,"uuid":"797a5452-1510-4640-a111-c9c2fcf707eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":13098},{"type":"v1","addr":"172.21.15.47:6811","nonce":13098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":13098},{"type":"v1","addr":"172.21.15.47:6813","nonce":13098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":13098},{"type":"v1","addr":"172.21.15.47:6817","nonce":13098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":13098},{"type":"v1","addr":"172.21.15.47:6815","nonce":13098}]},"public_addr":"172.21.15.47:6811/13098","cluster_addr":"172.21.15.47:6813/13098","heartbeat_back_addr":"172.21.15.47:6817/13098","heartbeat_front_addr":"172.21.15.47:6815/13098","state":["exists","up"]},{"osd":3,"uuid":"55b24fd7-bb71-4a01-b3fa-a09caa772a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":13097},{"type":"v1","addr":"172.21.15.47:6803","nonce":13097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":13097},{"type":"v1","addr":"172.21.15.47:6805","nonce":13097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":13097},{"type":"v1","addr":"172.21.15.47:6809","nonce":13097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":13097},{"type":"v1","addr":"172.21.15.47:6807","nonce":13097}]},"public_addr":"172.21.15.47:6803/13097","cluster_addr":"172.21.15.47:6805/13097","heartbeat_back_addr":"172.21.15.47:6809/13097","heartbeat_front_addr":"172.21.15.47:6807/13097","state":["exists","up"]},{"osd":4,"uuid":"a59fcc1b-c409-4bd3-9b6a-8d5e3859f7d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":13396},{"type":"v1","addr":"172.21.15.64:6817","nonce":13396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":13396},{"type":"v1","addr":"172.21.15.64:6819","nonce":13396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":13396},{"type":"v1","addr":"172.21.15.64:6823","nonce":13396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":13396},{"type":"v1","addr":"172.21.15.64:6821","nonce":13396}]},"public_addr":"172.21.15.64:6817/13396","cluster_addr":"172.21.15.64:6819/13396","heartbeat_back_addr":"172.21.15.64:6823/13396","heartbeat_front_addr":"172.21.15.64:6821/13396","state":["exists","up"]},{"osd":5,"uuid":"f77beebe-6cca-4244-b2d4-677825ebf919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":13393},{"type":"v1","addr":"172.21.15.64:6801","nonce":13393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":13393},{"type":"v1","addr":"172.21.15.64:6803","nonce":13393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":13393},{"type":"v1","addr":"172.21.15.64:6807","nonce":13393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":13393},{"type":"v1","addr":"172.21.15.64:6805","nonce":13393}]},"public_addr":"172.21.15.64:6801/13393","cluster_addr":"172.21.15.64:6803/13393","heartbeat_back_addr":"172.21.15.64:6807/13393","heartbeat_front_addr":"172.21.15.64:6805/13393","state":["exists","up"]},{"osd":6,"uuid":"af0ca9b3-c02c-4910-834b-bf0b8dc6713c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":13395},{"type":"v1","addr":"172.21.15.64:6825","nonce":13395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":13395},{"type":"v1","addr":"172.21.15.64:6827","nonce":13395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":13395},{"type":"v1","addr":"172.21.15.64:6831","nonce":13395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":13395},{"type":"v1","addr":"172.21.15.64:6829","nonce":13395}]},"public_addr":"172.21.15.64:6825/13395","cluster_addr":"172.21.15.64:6827/13395","heartbeat_back_addr":"172.21.15.64:6831/13395","heartbeat_front_addr":"172.21.15.64:6829/13395","state":["exists","up"]},{"osd":7,"uuid":"f19c3ca6-df88-4bde-a150-b51a53f092e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":13394},{"type":"v1","addr":"172.21.15.64:6809","nonce":13394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":13394},{"type":"v1","addr":"172.21.15.64:6811","nonce":13394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":13394},{"type":"v1","addr":"172.21.15.64:6815","nonce":13394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":13394},{"type":"v1","addr":"172.21.15.64:6813","nonce":13394}]},"public_addr":"172.21.15.64:6809/13394","cluster_addr":"172.21.15.64:6811/13394","heartbeat_back_addr":"172.21.15.64:6815/13394","heartbeat_front_addr":"172.21.15.64:6813/13394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.720513+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.772612+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.450223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.477856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.539485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.487040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.748685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-06-29T15:42:19.459516+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{"172.21.15.64:6833/642573447":"2022-06-30T15:52:27.555481+0000","172.21.15.64:6832/642573447":"2022-06-30T15:52:27.555481+0000"},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-06-29T16:53:02.451 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.444+0000 7f9f2e1c3700 1 -- 172.21.15.47:0/2777061461 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9f08053b00 msgr2=0x7f9f08055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:02.452 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.444+0000 7f9f2e1c3700 1 --2- 172.21.15.47:0/2777061461 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9f08053b00 0x7f9f08055fc0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 rx=0x7f9f2401a000 tx=0x7f9f24014000).stop 2022-06-29T16:53:02.452 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.444+0000 7f9f2e1c3700 1 -- 172.21.15.47:0/2777061461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f280b2c40 msgr2=0x7f9f2813e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:02.452 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.444+0000 7f9f2e1c3700 1 --2- 172.21.15.47:0/2777061461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f280b2c40 0x7f9f2813e910 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7f9f2813ef40 tx=0x7f9f20007f40).stop 2022-06-29T16:53:02.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.444+0000 7f9f2e1c3700 1 -- 172.21.15.47:0/2777061461 shutdown_connections 2022-06-29T16:53:02.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.444+0000 7f9f2e1c3700 1 --2- 172.21.15.47:0/2777061461 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9f08053b00 0x7f9f08055fc0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:02.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.444+0000 7f9f2e1c3700 1 --2- 172.21.15.47:0/2777061461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f280b2c40 0x7f9f2813e910 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:02.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.444+0000 7f9f2e1c3700 1 --2- 172.21.15.47:0/2777061461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f2805c8f0 0x7f9f2813e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:02.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.444+0000 7f9f2e1c3700 1 --2- 172.21.15.47:0/2777061461 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9f2805c510 0x7f9f28146e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:02.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.444+0000 7f9f2e1c3700 1 -- 172.21.15.47:0/2777061461 >> 172.21.15.47:0/2777061461 conn(0x7f9f28060e00 msgr2=0x7f9f280b6560 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T16:53:02.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.444+0000 7f9f2e1c3700 1 -- 172.21.15.47:0/2777061461 shutdown_connections 2022-06-29T16:53:02.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:02.444+0000 7f9f2e1c3700 1 -- 172.21.15.47:0/2777061461 wait complete. 2022-06-29T16:53:03.467 INFO:tasks.ceph:Scrubbing osd.0 2022-06-29T16:53:03.468 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2022-06-29T16:53:03.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.584+0000 7f986bfa1700 1 Processor -- start 2022-06-29T16:53:03.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.584+0000 7f986bfa1700 1 -- start start 2022-06-29T16:53:03.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.584+0000 7f986bfa1700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9864140940 0x7f986405f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:03.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.584+0000 7f986bfa1700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f98640b37f0 0x7f986414da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:03.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.584+0000 7f986bfa1700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98640b4e50 con 0x7f986405f5a0 2022-06-29T16:53:03.597 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.584+0000 7f986bfa1700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98640621e0 con 0x7f98640b37f0 2022-06-29T16:53:03.597 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.584+0000 7f986bfa1700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98640b3c60 con 0x7f9864140940 2022-06-29T16:53:03.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.584+0000 7f986953c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f986405f5a0 0x7f98640b30e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41118/0 (socket says 172.21.15.47:41118) 2022-06-29T16:53:03.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.588+0000 7f986953c700 1 -- 172.21.15.47:0/4283302216 learned_addr learned my addr 172.21.15.47:0/4283302216 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T16:53:03.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.588+0000 7f986a53e700 1 --2- 172.21.15.47:0/4283302216 >> v2:172.21.15.64:3300/0 conn(0x7f98640b37f0 0x7f986414da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:03.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.588+0000 7f9869d3d700 1 --2- 172.21.15.47:0/4283302216 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9864140940 0x7f986405f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:03.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.588+0000 7f9868d3b700 1 -- 172.21.15.47:0/4283302216 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 72823148 0 0) 0x7f98640b4e50 con 0x7f986405f5a0 2022-06-29T16:53:03.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.588+0000 7f9868d3b700 1 -- 172.21.15.47:0/4283302216 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f984c003260 con 0x7f986405f5a0 2022-06-29T16:53:03.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.588+0000 7f9868d3b700 1 -- 172.21.15.47:0/4283302216 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2654051636 0 0) 0x7f984c003260 con 0x7f986405f5a0 2022-06-29T16:53:03.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.588+0000 7f986a53e700 1 -- 172.21.15.47:0/4283302216 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9864140940 msgr2=0x7f986405f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T16:53:03.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.588+0000 7f986a53e700 1 --2- 172.21.15.47:0/4283302216 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9864140940 0x7f986405f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:03.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.588+0000 7f986a53e700 1 -- 172.21.15.47:0/4283302216 >> v1:172.21.15.47:6789/0 conn(0x7f986405f5a0 legacy=0x7f98640b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:03.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.588+0000 7f986a53e700 1 -- 172.21.15.47:0/4283302216 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9864143170 con 0x7f98640b37f0 2022-06-29T16:53:03.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.588+0000 7f986a53e700 1 --2- 172.21.15.47:0/4283302216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98640b37f0 0x7f986414da80 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7f9860001800 tx=0x7f985c0091d0).ready entity=mon.1 client_cookie=7ad95c87bd301d92 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:03.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.588+0000 7f9868d3b700 1 -- 172.21.15.47:0/4283302216 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9860001660 con 0x7f98640b37f0 2022-06-29T16:53:03.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.588+0000 7f9868d3b700 1 -- 172.21.15.47:0/4283302216 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f985c00f510 con 0x7f98640b37f0 2022-06-29T16:53:03.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.588+0000 7f9868d3b700 1 -- 172.21.15.47:0/4283302216 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f985c00f680 con 0x7f98640b37f0 2022-06-29T16:53:03.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.588+0000 7f986bfa1700 1 -- 172.21.15.47:0/4283302216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98640b37f0 msgr2=0x7f986414da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:03.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.588+0000 7f986bfa1700 1 --2- 172.21.15.47:0/4283302216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98640b37f0 0x7f986414da80 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7f9860001800 tx=0x7f985c0091d0).stop 2022-06-29T16:53:03.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.588+0000 7f986bfa1700 1 -- 172.21.15.47:0/4283302216 shutdown_connections 2022-06-29T16:53:03.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.588+0000 7f986bfa1700 1 --2- 172.21.15.47:0/4283302216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98640b37f0 0x7f986414da80 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:03.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.588+0000 7f986bfa1700 1 --2- 172.21.15.47:0/4283302216 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9864140940 0x7f986405f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:03.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.588+0000 7f986bfa1700 1 -- 172.21.15.47:0/4283302216 >> 172.21.15.47:0/4283302216 conn(0x7f98640627d0 msgr2=0x7f9864062bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T16:53:03.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.588+0000 7f986bfa1700 1 -- 172.21.15.47:0/4283302216 shutdown_connections 2022-06-29T16:53:03.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.588+0000 7f986bfa1700 1 -- 172.21.15.47:0/4283302216 wait complete. 2022-06-29T16:53:03.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.592+0000 7f986bfa1700 1 Processor -- start 2022-06-29T16:53:03.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.592+0000 7f986bfa1700 1 -- start start 2022-06-29T16:53:03.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.592+0000 7f986bfa1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98640b37f0 0x7f9864066ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:03.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.592+0000 7f986bfa1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9864140940 0x7f9864152740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:03.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.592+0000 7f986bfa1700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9864144510 0x7f9864152c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:03.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.592+0000 7f986bfa1700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98641520b0 con 0x7f98640b37f0 2022-06-29T16:53:03.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.592+0000 7f986bfa1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98640b81a0 con 0x7f9864140940 2022-06-29T16:53:03.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.592+0000 7f986bfa1700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9864144a90 con 0x7f9864144510 2022-06-29T16:53:03.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.592+0000 7f9869d3d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98640b37f0 0x7f9864066ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:03.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.592+0000 7f9869d3d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98640b37f0 0x7f9864066ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55570/0 (socket says 172.21.15.47:55570) 2022-06-29T16:53:03.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.592+0000 7f9869d3d700 1 -- 172.21.15.47:0/2931072894 learned_addr learned my addr 172.21.15.47:0/2931072894 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T16:53:03.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.592+0000 7f986a53e700 1 --2- 172.21.15.47:0/2931072894 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9864144510 0x7f9864152c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:03.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.592+0000 7f986953c700 1 --2- 172.21.15.47:0/2931072894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9864140940 0x7f9864152740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:03.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.592+0000 7f9869d3d700 1 -- 172.21.15.47:0/2931072894 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9864144510 msgr2=0x7f9864152c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:03.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.592+0000 7f9869d3d700 1 --2- 172.21.15.47:0/2931072894 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9864144510 0x7f9864152c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:03.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.592+0000 7f9869d3d700 1 -- 172.21.15.47:0/2931072894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9864140940 msgr2=0x7f9864152740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:03.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.592+0000 7f9869d3d700 1 --2- 172.21.15.47:0/2931072894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9864140940 0x7f9864152740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:03.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.592+0000 7f9869d3d700 1 -- 172.21.15.47:0/2931072894 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f985c009470 con 0x7f98640b37f0 2022-06-29T16:53:03.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.592+0000 7f9869d3d700 1 --2- 172.21.15.47:0/2931072894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98640b37f0 0x7f9864066ba0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 rx=0x7f985400c000 tx=0x7f9854006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:03.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.592+0000 7f9852ffd700 1 -- 172.21.15.47:0/2931072894 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9854011750 con 0x7f98640b37f0 2022-06-29T16:53:03.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.592+0000 7f9852ffd700 1 -- 172.21.15.47:0/2931072894 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98540118b0 con 0x7f98640b37f0 2022-06-29T16:53:03.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.592+0000 7f986bfa1700 1 -- 172.21.15.47:0/2931072894 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9864144d30 con 0x7f98640b37f0 2022-06-29T16:53:03.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.592+0000 7f9852ffd700 1 -- 172.21.15.47:0/2931072894 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9854011c10 con 0x7f98640b37f0 2022-06-29T16:53:03.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.596+0000 7f986bfa1700 1 -- 172.21.15.47:0/2931072894 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98640b55d0 con 0x7f98640b37f0 2022-06-29T16:53:03.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.596+0000 7f986bfa1700 1 -- 172.21.15.47:0/2931072894 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f98641524a0 con 0x7f98640b37f0 2022-06-29T16:53:03.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.596+0000 7f9852ffd700 1 -- 172.21.15.47:0/2931072894 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9854011d70 con 0x7f98640b37f0 2022-06-29T16:53:03.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.596+0000 7f9852ffd700 1 --2- 172.21.15.47:0/2931072894 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9844053950 0x7f9844055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:03.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.596+0000 7f9852ffd700 1 -- 172.21.15.47:0/2931072894 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f985406c200 con 0x7f98640b37f0 2022-06-29T16:53:03.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.596+0000 7f986953c700 1 --2- 172.21.15.47:0/2931072894 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9844053950 0x7f9844055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:03.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.596+0000 7f9852ffd700 1 --2- 172.21.15.47:0/2931072894 >> [v2:172.21.15.47:6818/13096,v1:172.21.15.47:6819/13096] conn(0x7f984405a680 0x7f984405cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:03.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.596+0000 7f9852ffd700 1 -- 172.21.15.47:0/2931072894 --> [v2:172.21.15.47:6818/13096,v1:172.21.15.47:6819/13096] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f984405d1a0 con 0x7f984405a680 2022-06-29T16:53:03.622 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.596+0000 7f9852ffd700 1 -- 172.21.15.47:0/2931072894 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=165) v2 ==== 24+0+0 (secure 0 0 0) 0x7f985406e870 con 0x7f98640b37f0 2022-06-29T16:53:03.622 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.596+0000 7f986a53e700 1 --2- 172.21.15.47:0/2931072894 >> [v2:172.21.15.47:6818/13096,v1:172.21.15.47:6819/13096] conn(0x7f984405a680 0x7f984405cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:03.622 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.596+0000 7f986953c700 1 --2- 172.21.15.47:0/2931072894 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9844053950 0x7f9844055e10 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 rx=0x7f986000d000 tx=0x7f9860001400).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:03.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.600+0000 7f986a53e700 1 --2- 172.21.15.47:0/2931072894 >> [v2:172.21.15.47:6818/13096,v1:172.21.15.47:6819/13096] conn(0x7f984405a680 0x7f984405cb20 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:03.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.600+0000 7f9852ffd700 1 -- 172.21.15.47:0/2931072894 <== osd.0 v2:172.21.15.47:6818/13096 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f984405d1a0 con 0x7f984405a680 2022-06-29T16:53:03.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.616+0000 7f986bfa1700 1 -- 172.21.15.47:0/2931072894 --> [v2:172.21.15.47:6818/13096,v1:172.21.15.47:6819/13096] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f98640b6f50 con 0x7f984405a680 2022-06-29T16:53:03.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.616+0000 7f9852ffd700 1 -- 172.21.15.47:0/2931072894 <== osd.0 v2:172.21.15.47:6818/13096 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f98640b6f50 con 0x7f984405a680 2022-06-29T16:53:03.625 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2022-06-29T16:53:03.626 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2022-06-29T16:53:03.627 INFO:teuthology.orchestra.run.smithi047.stdout:} 2022-06-29T16:53:03.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.616+0000 7f986bfa1700 1 -- 172.21.15.47:0/2931072894 >> [v2:172.21.15.47:6818/13096,v1:172.21.15.47:6819/13096] conn(0x7f984405a680 msgr2=0x7f984405cb20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:03.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.616+0000 7f986bfa1700 1 --2- 172.21.15.47:0/2931072894 >> [v2:172.21.15.47:6818/13096,v1:172.21.15.47:6819/13096] conn(0x7f984405a680 0x7f984405cb20 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:03.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.616+0000 7f986bfa1700 1 -- 172.21.15.47:0/2931072894 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9844053950 msgr2=0x7f9844055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:03.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.616+0000 7f986bfa1700 1 --2- 172.21.15.47:0/2931072894 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9844053950 0x7f9844055e10 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 rx=0x7f986000d000 tx=0x7f9860001400).stop 2022-06-29T16:53:03.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.616+0000 7f986bfa1700 1 -- 172.21.15.47:0/2931072894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98640b37f0 msgr2=0x7f9864066ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:03.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.616+0000 7f986bfa1700 1 --2- 172.21.15.47:0/2931072894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98640b37f0 0x7f9864066ba0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 rx=0x7f985400c000 tx=0x7f9854006df0).stop 2022-06-29T16:53:03.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.616+0000 7f986a53e700 1 -- 172.21.15.47:0/2931072894 reap_dead start 2022-06-29T16:53:03.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.616+0000 7f986bfa1700 1 -- 172.21.15.47:0/2931072894 shutdown_connections 2022-06-29T16:53:03.631 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.616+0000 7f986bfa1700 1 -- 172.21.15.47:0/2931072894 >> 172.21.15.47:0/2931072894 conn(0x7f98640627d0 msgr2=0x7f9864142470 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T16:53:03.631 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.616+0000 7f986bfa1700 1 -- 172.21.15.47:0/2931072894 shutdown_connections 2022-06-29T16:53:03.631 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.616+0000 7f986bfa1700 1 -- 172.21.15.47:0/2931072894 wait complete. 2022-06-29T16:53:03.636 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2022-06-29T16:53:03.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.736+0000 7ffb91c0b700 1 Processor -- start 2022-06-29T16:53:03.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.736+0000 7ffb91c0b700 1 -- start start 2022-06-29T16:53:03.749 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.736+0000 7ffb91c0b700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffb8c0b1fe0 0x7ffb8c14dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:03.749 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.736+0000 7ffb91c0b700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb8c061d90 con 0x7ffb8c0619b0 2022-06-29T16:53:03.749 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.736+0000 7ffb91c0b700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb8c0b1b80 con 0x7ffb8c05fac0 2022-06-29T16:53:03.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.736+0000 7ffb91c0b700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c05ffd0 con 0x7ffb8c0b1fe0 2022-06-29T16:53:03.750 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.736+0000 7ffb8b7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ffb8c0619b0 0x7ffb8c143380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41136/0 (socket says 172.21.15.47:41136) 2022-06-29T16:53:03.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.736+0000 7ffb8b7fe700 1 -- 172.21.15.47:0/2045789431 learned_addr learned my addr 172.21.15.47:0/2045789431 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T16:53:03.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.736+0000 7ffb8bfff700 1 --2- 172.21.15.47:0/2045789431 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffb8c0b1fe0 0x7ffb8c14dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:03.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.736+0000 7ffb8a7fc700 1 -- 172.21.15.47:0/2045789431 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4235287306 0 0) 0x7ffb8c061d90 con 0x7ffb8c0619b0 2022-06-29T16:53:03.752 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.736+0000 7ffb8a7fc700 1 -- 172.21.15.47:0/2045789431 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb6c003260 con 0x7ffb8c0619b0 2022-06-29T16:53:03.752 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.736+0000 7ffb8a7fc700 1 -- 172.21.15.47:0/2045789431 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1107213536 0 0) 0x7ffb8c0b1b80 con 0x7ffb8c05fac0 2022-06-29T16:53:03.752 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.736+0000 7ffb8a7fc700 1 -- 172.21.15.47:0/2045789431 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb8c061d90 con 0x7ffb8c05fac0 2022-06-29T16:53:03.753 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.736+0000 7ffb8bfff700 1 -- 172.21.15.47:0/2045789431 >> v1:172.21.15.64:6789/0 conn(0x7ffb8c05fac0 legacy=0x7ffb8c143a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:03.753 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.736+0000 7ffb8bfff700 1 -- 172.21.15.47:0/2045789431 >> v1:172.21.15.47:6789/0 conn(0x7ffb8c0619b0 legacy=0x7ffb8c143380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:03.753 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.736+0000 7ffb8bfff700 1 -- 172.21.15.47:0/2045789431 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb8c0b1640 con 0x7ffb8c0b1fe0 2022-06-29T16:53:03.754 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.740+0000 7ffb8bfff700 1 --2- 172.21.15.47:0/2045789431 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffb8c0b1fe0 0x7ffb8c14dbc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7ffb8c05fef0 tx=0x7ffb840091d0).ready entity=mon.2 client_cookie=8eba42f2921cd63 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:03.754 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.740+0000 7ffb8a7fc700 1 -- 172.21.15.47:0/2045789431 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb8400b600 con 0x7ffb8c0b1fe0 2022-06-29T16:53:03.754 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.740+0000 7ffb8a7fc700 1 -- 172.21.15.47:0/2045789431 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb8400b760 con 0x7ffb8c0b1fe0 2022-06-29T16:53:03.755 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.740+0000 7ffb8a7fc700 1 -- 172.21.15.47:0/2045789431 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb8400b8d0 con 0x7ffb8c0b1fe0 2022-06-29T16:53:03.755 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.740+0000 7ffb91c0b700 1 -- 172.21.15.47:0/2045789431 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffb8c0b1fe0 msgr2=0x7ffb8c14dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:03.755 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.740+0000 7ffb91c0b700 1 --2- 172.21.15.47:0/2045789431 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffb8c0b1fe0 0x7ffb8c14dbc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7ffb8c05fef0 tx=0x7ffb840091d0).stop 2022-06-29T16:53:03.756 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.740+0000 7ffb91c0b700 1 -- 172.21.15.47:0/2045789431 shutdown_connections 2022-06-29T16:53:03.756 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.740+0000 7ffb91c0b700 1 --2- 172.21.15.47:0/2045789431 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffb8c0b1fe0 0x7ffb8c14dbc0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:03.756 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.740+0000 7ffb91c0b700 1 -- 172.21.15.47:0/2045789431 >> 172.21.15.47:0/2045789431 conn(0x7ffb8c05c510 msgr2=0x7ffb8c063800 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T16:53:03.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.740+0000 7ffb91c0b700 1 -- 172.21.15.47:0/2045789431 shutdown_connections 2022-06-29T16:53:03.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.740+0000 7ffb91c0b700 1 -- 172.21.15.47:0/2045789431 wait complete. 2022-06-29T16:53:03.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.740+0000 7ffb91c0b700 1 Processor -- start 2022-06-29T16:53:03.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.740+0000 7ffb91c0b700 1 -- start start 2022-06-29T16:53:03.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.740+0000 7ffb91c0b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb8c05fac0 0x7ffb8c0b0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:03.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.740+0000 7ffb91c0b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb8c0619b0 0x7ffb8c1527d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:03.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.740+0000 7ffb91c0b700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffb8c0b1fe0 0x7ffb8c152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:03.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.740+0000 7ffb91c0b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c0af9b0 con 0x7ffb8c05fac0 2022-06-29T16:53:03.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.740+0000 7ffb91c0b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c0afbe0 con 0x7ffb8c0619b0 2022-06-29T16:53:03.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.740+0000 7ffb91c0b700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c0afe80 con 0x7ffb8c0b1fe0 2022-06-29T16:53:03.761 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.744+0000 7ffb8b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb8c05fac0 0x7ffb8c0b0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:03.761 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.744+0000 7ffb8b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb8c05fac0 0x7ffb8c0b0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55586/0 (socket says 172.21.15.47:55586) 2022-06-29T16:53:03.761 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.744+0000 7ffb8b7fe700 1 -- 172.21.15.47:0/1283970760 learned_addr learned my addr 172.21.15.47:0/1283970760 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T16:53:03.762 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.744+0000 7ffb8bfff700 1 --2- 172.21.15.47:0/1283970760 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffb8c0b1fe0 0x7ffb8c152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:03.762 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.744+0000 7ffb8affd700 1 --2- 172.21.15.47:0/1283970760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb8c0619b0 0x7ffb8c1527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:03.765 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.744+0000 7ffb8b7fe700 1 -- 172.21.15.47:0/1283970760 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffb8c0b1fe0 msgr2=0x7ffb8c152cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:03.765 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.744+0000 7ffb8b7fe700 1 --2- 172.21.15.47:0/1283970760 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffb8c0b1fe0 0x7ffb8c152cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:03.766 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.744+0000 7ffb8b7fe700 1 -- 172.21.15.47:0/1283970760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb8c0619b0 msgr2=0x7ffb8c1527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:03.766 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.744+0000 7ffb8b7fe700 1 --2- 172.21.15.47:0/1283970760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb8c0619b0 0x7ffb8c1527d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:03.766 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.744+0000 7ffb8b7fe700 1 -- 172.21.15.47:0/1283970760 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb84009470 con 0x7ffb8c05fac0 2022-06-29T16:53:03.767 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.744+0000 7ffb8affd700 1 --2- 172.21.15.47:0/1283970760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb8c0619b0 0x7ffb8c1527d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T16:53:03.767 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.744+0000 7ffb8b7fe700 1 --2- 172.21.15.47:0/1283970760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb8c05fac0 0x7ffb8c0b0320 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 rx=0x7ffb7c0010d0 tx=0x7ffb7c007b00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:03.767 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.744+0000 7ffb88ff9700 1 -- 172.21.15.47:0/1283970760 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb7c0070d0 con 0x7ffb8c05fac0 2022-06-29T16:53:03.768 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.744+0000 7ffb88ff9700 1 -- 172.21.15.47:0/1283970760 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb7c007230 con 0x7ffb8c05fac0 2022-06-29T16:53:03.768 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.744+0000 7ffb91c0b700 1 -- 172.21.15.47:0/1283970760 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb8c0b61a0 con 0x7ffb8c05fac0 2022-06-29T16:53:03.769 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.744+0000 7ffb88ff9700 1 -- 172.21.15.47:0/1283970760 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb7c005610 con 0x7ffb8c05fac0 2022-06-29T16:53:03.769 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.744+0000 7ffb91c0b700 1 -- 172.21.15.47:0/1283970760 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb8c0b4b70 con 0x7ffb8c05fac0 2022-06-29T16:53:03.770 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.744+0000 7ffb91c0b700 1 -- 172.21.15.47:0/1283970760 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb8c0b0040 con 0x7ffb8c05fac0 2022-06-29T16:53:03.770 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.744+0000 7ffb88ff9700 1 -- 172.21.15.47:0/1283970760 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ffb7c010070 con 0x7ffb8c05fac0 2022-06-29T16:53:03.770 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.748+0000 7ffb88ff9700 1 --2- 172.21.15.47:0/1283970760 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ffb70053a70 0x7ffb70055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:03.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.748+0000 7ffb88ff9700 1 -- 172.21.15.47:0/1283970760 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5686+0+0 (secure 0 0 0) 0x7ffb7c06b110 con 0x7ffb8c05fac0 2022-06-29T16:53:03.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.748+0000 7ffb8affd700 1 --2- 172.21.15.47:0/1283970760 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ffb70053a70 0x7ffb70055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:03.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.748+0000 7ffb8affd700 1 --2- 172.21.15.47:0/1283970760 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ffb70053a70 0x7ffb70055f30 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 rx=0x7ffb7400d000 tx=0x7ffb74007a20).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:03.772 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.752+0000 7ffb88ff9700 1 -- 172.21.15.47:0/1283970760 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ffb7c0421b0 con 0x7ffb8c05fac0 2022-06-29T16:53:03.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.880+0000 7ffb91c0b700 1 -- 172.21.15.47:0/1283970760 --> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7ffb8c0573d0 con 0x7ffb70053a70 2022-06-29T16:53:03.888 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.880+0000 7ffb88ff9700 1 -- 172.21.15.47:0/1283970760 <== mgr.4102 v2:172.21.15.47:6800/12928 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ffb8c0573d0 con 0x7ffb70053a70 2022-06-29T16:53:03.891 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.884+0000 7ffb91c0b700 1 -- 172.21.15.47:0/1283970760 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ffb70053a70 msgr2=0x7ffb70055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:03.891 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.884+0000 7ffb91c0b700 1 --2- 172.21.15.47:0/1283970760 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ffb70053a70 0x7ffb70055f30 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 rx=0x7ffb7400d000 tx=0x7ffb74007a20).stop 2022-06-29T16:53:03.891 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.884+0000 7ffb91c0b700 1 -- 172.21.15.47:0/1283970760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb8c05fac0 msgr2=0x7ffb8c0b0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:03.892 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.884+0000 7ffb91c0b700 1 --2- 172.21.15.47:0/1283970760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb8c05fac0 0x7ffb8c0b0320 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 rx=0x7ffb7c0010d0 tx=0x7ffb7c007b00).stop 2022-06-29T16:53:03.892 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.884+0000 7ffb91c0b700 1 -- 172.21.15.47:0/1283970760 shutdown_connections 2022-06-29T16:53:03.892 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.884+0000 7ffb91c0b700 1 --2- 172.21.15.47:0/1283970760 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ffb8c0b1fe0 0x7ffb8c152cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:03.893 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.884+0000 7ffb91c0b700 1 --2- 172.21.15.47:0/1283970760 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ffb70053a70 0x7ffb70055f30 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:03.893 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.884+0000 7ffb91c0b700 1 --2- 172.21.15.47:0/1283970760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb8c0619b0 0x7ffb8c1527d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:03.893 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.884+0000 7ffb91c0b700 1 --2- 172.21.15.47:0/1283970760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb8c05fac0 0x7ffb8c0b0320 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:03.894 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.884+0000 7ffb91c0b700 1 -- 172.21.15.47:0/1283970760 >> 172.21.15.47:0/1283970760 conn(0x7ffb8c05c510 msgr2=0x7ffb8c0b36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T16:53:03.894 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.884+0000 7ffb91c0b700 1 -- 172.21.15.47:0/1283970760 shutdown_connections 2022-06-29T16:53:03.895 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:03.884+0000 7ffb91c0b700 1 -- 172.21.15.47:0/1283970760 wait complete. 2022-06-29T16:53:03.895 INFO:teuthology.orchestra.run.smithi047.stderr:instructed osd(s) 0 to deep-scrub 2022-06-29T16:53:03.906 INFO:tasks.ceph:Scrubbing osd.1 2022-06-29T16:53:03.907 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2022-06-29T16:53:04.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.008+0000 7f6b38f46700 1 Processor -- start 2022-06-29T16:53:04.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.008+0000 7f6b38f46700 1 -- start start 2022-06-29T16:53:04.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.008+0000 7f6b38f46700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6b340b0fe0 0x7f6b34061a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:04.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.008+0000 7f6b38f46700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6b340b0700 0x7f6b341436c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:04.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.008+0000 7f6b38f46700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b3413db90 con 0x7f6b34061670 2022-06-29T16:53:04.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.008+0000 7f6b38f46700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b3413fed0 con 0x7f6b340b0fe0 2022-06-29T16:53:04.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.008+0000 7f6b38f46700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b340af0a0 con 0x7f6b340b0700 2022-06-29T16:53:04.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.008+0000 7f6b31d9b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6b340b0fe0 0x7f6b34061a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:04.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.008+0000 7f6b3259c700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f6b34061670 0x7f6b3413d480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:48712/0 (socket says 172.21.15.47:48712) 2022-06-29T16:53:04.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.008+0000 7f6b32d9d700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6b340b0700 0x7f6b341436c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:04.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.008+0000 7f6b3259c700 1 -- 172.21.15.47:0/3407887963 learned_addr learned my addr 172.21.15.47:0/3407887963 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T16:53:04.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.008+0000 7f6b31d9b700 1 -- 172.21.15.47:0/3407887963 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6b340b0700 msgr2=0x7f6b341436c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T16:53:04.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.008+0000 7f6b31d9b700 1 --2- 172.21.15.47:0/3407887963 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6b340b0700 0x7f6b341436c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:04.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.008+0000 7f6b31d9b700 1 -- 172.21.15.47:0/3407887963 >> v1:172.21.15.64:6789/0 conn(0x7f6b34061670 legacy=0x7f6b3413d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:04.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.008+0000 7f6b3159a700 1 -- 172.21.15.47:0/3407887963 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1236132941 0 0) 0x7f6b3413db90 con 0x7f6b34061670 2022-06-29T16:53:04.022 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.008+0000 7f6b31d9b700 1 -- 172.21.15.47:0/3407887963 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b340af340 con 0x7f6b340b0fe0 2022-06-29T16:53:04.022 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.012+0000 7f6b32d9d700 1 --2- 172.21.15.47:0/3407887963 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6b340b0700 0x7f6b341436c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T16:53:04.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.012+0000 7f6b31d9b700 1 --2- 172.21.15.47:0/3407887963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b340b0fe0 0x7f6b34061a50 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7f6b1c002f70 tx=0x7f6b1c006df0).ready entity=mon.1 client_cookie=285984c0f8cfcf7a server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:04.024 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.012+0000 7f6b3159a700 1 -- 172.21.15.47:0/3407887963 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b1c00d570 con 0x7f6b340b0fe0 2022-06-29T16:53:04.024 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.012+0000 7f6b3159a700 1 -- 172.21.15.47:0/3407887963 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b1c000e90 con 0x7f6b340b0fe0 2022-06-29T16:53:04.025 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.012+0000 7f6b3159a700 1 -- 172.21.15.47:0/3407887963 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b1c00d8c0 con 0x7f6b340b0fe0 2022-06-29T16:53:04.030 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.012+0000 7f6b38f46700 1 -- 172.21.15.47:0/3407887963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b340b0fe0 msgr2=0x7f6b34061a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:04.030 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.012+0000 7f6b38f46700 1 --2- 172.21.15.47:0/3407887963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b340b0fe0 0x7f6b34061a50 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7f6b1c002f70 tx=0x7f6b1c006df0).stop 2022-06-29T16:53:04.030 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.012+0000 7f6b38f46700 1 -- 172.21.15.47:0/3407887963 shutdown_connections 2022-06-29T16:53:04.031 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.012+0000 7f6b38f46700 1 --2- 172.21.15.47:0/3407887963 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6b340b0700 0x7f6b341436c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:04.031 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.012+0000 7f6b38f46700 1 --2- 172.21.15.47:0/3407887963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b340b0fe0 0x7f6b34061a50 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:04.032 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.012+0000 7f6b38f46700 1 -- 172.21.15.47:0/3407887963 >> 172.21.15.47:0/3407887963 conn(0x7f6b34063800 msgr2=0x7f6b34063be0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T16:53:04.032 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.012+0000 7f6b38f46700 1 -- 172.21.15.47:0/3407887963 shutdown_connections 2022-06-29T16:53:04.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.012+0000 7f6b38f46700 1 -- 172.21.15.47:0/3407887963 wait complete. 2022-06-29T16:53:04.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.016+0000 7f6b38f46700 1 Processor -- start 2022-06-29T16:53:04.033 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.016+0000 7f6b38f46700 1 -- start start 2022-06-29T16:53:04.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.016+0000 7f6b38f46700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b34061670 0x7f6b3413f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:04.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.016+0000 7f6b38f46700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6b340b0700 0x7f6b340aa510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:04.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.016+0000 7f6b38f46700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b340b0fe0 0x7f6b340ad190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:04.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.016+0000 7f6b38f46700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b34153300 con 0x7f6b34061670 2022-06-29T16:53:04.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.016+0000 7f6b38f46700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b341535a0 con 0x7f6b340b0fe0 2022-06-29T16:53:04.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.016+0000 7f6b38f46700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b34142e10 con 0x7f6b340b0700 2022-06-29T16:53:04.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.016+0000 7f6b3259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b34061670 0x7f6b3413f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:04.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.016+0000 7f6b3259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b34061670 0x7f6b3413f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55600/0 (socket says 172.21.15.47:55600) 2022-06-29T16:53:04.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.016+0000 7f6b3259c700 1 -- 172.21.15.47:0/2716071151 learned_addr learned my addr 172.21.15.47:0/2716071151 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T16:53:04.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.016+0000 7f6b31d9b700 1 --2- 172.21.15.47:0/2716071151 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6b340b0700 0x7f6b340aa510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:04.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.016+0000 7f6b32d9d700 1 --2- 172.21.15.47:0/2716071151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b340b0fe0 0x7f6b340ad190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:04.038 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.016+0000 7f6b3259c700 1 -- 172.21.15.47:0/2716071151 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6b340b0700 msgr2=0x7f6b340aa510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:04.038 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.016+0000 7f6b3259c700 1 --2- 172.21.15.47:0/2716071151 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f6b340b0700 0x7f6b340aa510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:04.038 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.016+0000 7f6b3259c700 1 -- 172.21.15.47:0/2716071151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b340b0fe0 msgr2=0x7f6b340ad190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:04.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.016+0000 7f6b3259c700 1 --2- 172.21.15.47:0/2716071151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b340b0fe0 0x7f6b340ad190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:04.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.016+0000 7f6b3259c700 1 -- 172.21.15.47:0/2716071151 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b2c00b180 con 0x7f6b34061670 2022-06-29T16:53:04.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.016+0000 7f6b3259c700 1 --2- 172.21.15.47:0/2716071151 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b34061670 0x7f6b3413f040 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 rx=0x7f6b24001250 tx=0x7f6b24007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:04.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.016+0000 7f6b2b7fe700 1 -- 172.21.15.47:0/2716071151 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b24005bf0 con 0x7f6b34061670 2022-06-29T16:53:04.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.016+0000 7f6b2b7fe700 1 -- 172.21.15.47:0/2716071151 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b24005d50 con 0x7f6b34061670 2022-06-29T16:53:04.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.016+0000 7f6b38f46700 1 -- 172.21.15.47:0/2716071151 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b1c00b180 con 0x7f6b34061670 2022-06-29T16:53:04.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.016+0000 7f6b2b7fe700 1 -- 172.21.15.47:0/2716071151 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b240072c0 con 0x7f6b34061670 2022-06-29T16:53:04.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.016+0000 7f6b38f46700 1 -- 172.21.15.47:0/2716071151 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b34067080 con 0x7f6b34061670 2022-06-29T16:53:04.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.020+0000 7f6b38f46700 1 -- 172.21.15.47:0/2716071151 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6b34153180 con 0x7f6b34061670 2022-06-29T16:53:04.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.020+0000 7f6b2b7fe700 1 -- 172.21.15.47:0/2716071151 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6b24010070 con 0x7f6b34061670 2022-06-29T16:53:04.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.020+0000 7f6b2b7fe700 1 --2- 172.21.15.47:0/2716071151 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6b180539e0 0x7f6b18055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:04.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.020+0000 7f6b2b7fe700 1 -- 172.21.15.47:0/2716071151 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f6b2406b860 con 0x7f6b34061670 2022-06-29T16:53:04.043 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.020+0000 7f6b31d9b700 1 --2- 172.21.15.47:0/2716071151 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6b180539e0 0x7f6b18055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:04.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.020+0000 7f6b2b7fe700 1 --2- 172.21.15.47:0/2716071151 >> [v2:172.21.15.47:6826/13100,v1:172.21.15.47:6827/13100] conn(0x7f6b1805a670 0x7f6b1805cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:04.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.020+0000 7f6b2b7fe700 1 -- 172.21.15.47:0/2716071151 --> [v2:172.21.15.47:6826/13100,v1:172.21.15.47:6827/13100] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6b1805d190 con 0x7f6b1805a670 2022-06-29T16:53:04.044 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.020+0000 7f6b2b7fe700 1 -- 172.21.15.47:0/2716071151 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=165) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b24070220 con 0x7f6b34061670 2022-06-29T16:53:04.045 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.020+0000 7f6b32d9d700 1 --2- 172.21.15.47:0/2716071151 >> [v2:172.21.15.47:6826/13100,v1:172.21.15.47:6827/13100] conn(0x7f6b1805a670 0x7f6b1805cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:04.045 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.020+0000 7f6b31d9b700 1 --2- 172.21.15.47:0/2716071151 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6b180539e0 0x7f6b18055ea0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 rx=0x7f6b1c01c000 tx=0x7f6b1c014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:04.045 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.020+0000 7f6b32d9d700 1 --2- 172.21.15.47:0/2716071151 >> [v2:172.21.15.47:6826/13100,v1:172.21.15.47:6827/13100] conn(0x7f6b1805a670 0x7f6b1805cb10 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:04.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.024+0000 7f6b2b7fe700 1 -- 172.21.15.47:0/2716071151 <== osd.1 v2:172.21.15.47:6826/13100 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f6b1805d190 con 0x7f6b1805a670 2022-06-29T16:53:04.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.036+0000 7f6b38f46700 1 -- 172.21.15.47:0/2716071151 --> [v2:172.21.15.47:6826/13100,v1:172.21.15.47:6827/13100] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f6b34142fd0 con 0x7f6b1805a670 2022-06-29T16:53:04.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.040+0000 7f6b2b7fe700 1 -- 172.21.15.47:0/2716071151 <== osd.1 v2:172.21.15.47:6826/13100 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f6b34142fd0 con 0x7f6b1805a670 2022-06-29T16:53:04.047 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2022-06-29T16:53:04.048 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2022-06-29T16:53:04.048 INFO:teuthology.orchestra.run.smithi047.stdout:} 2022-06-29T16:53:04.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.040+0000 7f6b38f46700 1 -- 172.21.15.47:0/2716071151 >> [v2:172.21.15.47:6826/13100,v1:172.21.15.47:6827/13100] conn(0x7f6b1805a670 msgr2=0x7f6b1805cb10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:04.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.040+0000 7f6b38f46700 1 --2- 172.21.15.47:0/2716071151 >> [v2:172.21.15.47:6826/13100,v1:172.21.15.47:6827/13100] conn(0x7f6b1805a670 0x7f6b1805cb10 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:04.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.040+0000 7f6b38f46700 1 -- 172.21.15.47:0/2716071151 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6b180539e0 msgr2=0x7f6b18055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:04.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.040+0000 7f6b38f46700 1 --2- 172.21.15.47:0/2716071151 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f6b180539e0 0x7f6b18055ea0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 rx=0x7f6b1c01c000 tx=0x7f6b1c014000).stop 2022-06-29T16:53:04.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.040+0000 7f6b38f46700 1 -- 172.21.15.47:0/2716071151 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b34061670 msgr2=0x7f6b3413f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:04.052 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.040+0000 7f6b38f46700 1 --2- 172.21.15.47:0/2716071151 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b34061670 0x7f6b3413f040 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 rx=0x7f6b24001250 tx=0x7f6b24007e20).stop 2022-06-29T16:53:04.052 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.040+0000 7f6b32d9d700 1 -- 172.21.15.47:0/2716071151 reap_dead start 2022-06-29T16:53:04.052 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.040+0000 7f6b38f46700 1 -- 172.21.15.47:0/2716071151 shutdown_connections 2022-06-29T16:53:04.053 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.040+0000 7f6b38f46700 1 -- 172.21.15.47:0/2716071151 >> 172.21.15.47:0/2716071151 conn(0x7f6b34063800 msgr2=0x7f6b34140810 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T16:53:04.053 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.040+0000 7f6b38f46700 1 -- 172.21.15.47:0/2716071151 shutdown_connections 2022-06-29T16:53:04.053 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.040+0000 7f6b38f46700 1 -- 172.21.15.47:0/2716071151 wait complete. 2022-06-29T16:53:04.061 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2022-06-29T16:53:04.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.164+0000 7f54cb59e700 1 Processor -- start 2022-06-29T16:53:04.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.164+0000 7f54cb59e700 1 -- start start 2022-06-29T16:53:04.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.164+0000 7f54cb59e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f54cc0b7860 0x7f54cc0af160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:04.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.164+0000 7f54cb59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f54cc0b2d00 0x7f54cc0af650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:04.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.164+0000 7f54cb59e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54cc0b3900 con 0x7f54cc146890 2022-06-29T16:53:04.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.164+0000 7f54cb59e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54cc1477c0 con 0x7f54cc0b7860 2022-06-29T16:53:04.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.164+0000 7f54cb59e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54cc0b7c40 con 0x7f54cc0b2d00 2022-06-29T16:53:04.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.168+0000 7f54c3fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f54cc146890 0x7f54cc0b0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41164/0 (socket says 172.21.15.47:41164) 2022-06-29T16:53:04.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.168+0000 7f54c3fff700 1 -- 172.21.15.47:0/3082097209 learned_addr learned my addr 172.21.15.47:0/3082097209 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T16:53:04.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.168+0000 7f54ca59c700 1 --2- 172.21.15.47:0/3082097209 >> v2:172.21.15.47:3300/0 conn(0x7f54cc0b7860 0x7f54cc0af160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:04.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.168+0000 7f54cad9d700 1 --2- 172.21.15.47:0/3082097209 >> v2:172.21.15.64:3300/0 conn(0x7f54cc0b2d00 0x7f54cc0af650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:04.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.168+0000 7f54c9d9b700 1 -- 172.21.15.47:0/3082097209 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 9859197 0 0) 0x7f54cc0b3900 con 0x7f54cc146890 2022-06-29T16:53:04.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.168+0000 7f54c9d9b700 1 -- 172.21.15.47:0/3082097209 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54ac003260 con 0x7f54cc146890 2022-06-29T16:53:04.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.168+0000 7f54ca59c700 1 -- 172.21.15.47:0/3082097209 >> v2:172.21.15.64:3300/0 conn(0x7f54cc0b2d00 msgr2=0x7f54cc0af650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T16:53:04.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.168+0000 7f54ca59c700 1 --2- 172.21.15.47:0/3082097209 >> v2:172.21.15.64:3300/0 conn(0x7f54cc0b2d00 0x7f54cc0af650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:04.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.168+0000 7f54ca59c700 1 -- 172.21.15.47:0/3082097209 >> v1:172.21.15.47:6789/0 conn(0x7f54cc146890 legacy=0x7f54cc0b0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:04.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.168+0000 7f54ca59c700 1 -- 172.21.15.47:0/3082097209 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54cc0b3660 con 0x7f54cc0b7860 2022-06-29T16:53:04.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.168+0000 7f54ca59c700 1 --2- 172.21.15.47:0/3082097209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54cc0b7860 0x7f54cc0af160 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 rx=0x7f54bc002f70 tx=0x7f54bc006df0).ready entity=mon.0 client_cookie=134d7e913c2b6003 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:04.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.168+0000 7f54c9d9b700 1 -- 172.21.15.47:0/3082097209 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54bc00d4f0 con 0x7f54cc0b7860 2022-06-29T16:53:04.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.168+0000 7f54c9d9b700 1 -- 172.21.15.47:0/3082097209 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54bc000e90 con 0x7f54cc0b7860 2022-06-29T16:53:04.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.168+0000 7f54c9d9b700 1 -- 172.21.15.47:0/3082097209 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54bc00d840 con 0x7f54cc0b7860 2022-06-29T16:53:04.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.168+0000 7f54cb59e700 1 -- 172.21.15.47:0/3082097209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54cc0b7860 msgr2=0x7f54cc0af160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:04.185 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.168+0000 7f54cb59e700 1 --2- 172.21.15.47:0/3082097209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54cc0b7860 0x7f54cc0af160 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 rx=0x7f54bc002f70 tx=0x7f54bc006df0).stop 2022-06-29T16:53:04.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.168+0000 7f54cb59e700 1 -- 172.21.15.47:0/3082097209 shutdown_connections 2022-06-29T16:53:04.186 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.168+0000 7f54cb59e700 1 --2- 172.21.15.47:0/3082097209 >> v2:172.21.15.64:3300/0 conn(0x7f54cc0b2d00 0x7f54cc0af650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:04.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.168+0000 7f54cb59e700 1 --2- 172.21.15.47:0/3082097209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54cc0b7860 0x7f54cc0af160 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:04.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.168+0000 7f54cb59e700 1 -- 172.21.15.47:0/3082097209 >> 172.21.15.47:0/3082097209 conn(0x7f54cc05f480 msgr2=0x7f54cc0647d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T16:53:04.187 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.168+0000 7f54cb59e700 1 -- 172.21.15.47:0/3082097209 shutdown_connections 2022-06-29T16:53:04.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.168+0000 7f54cb59e700 1 -- 172.21.15.47:0/3082097209 wait complete. 2022-06-29T16:53:04.188 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.172+0000 7f54cb59e700 1 Processor -- start 2022-06-29T16:53:04.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.172+0000 7f54cb59e700 1 -- start start 2022-06-29T16:53:04.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.172+0000 7f54cb59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54cc0b2d00 0x7f54cc1420b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:04.189 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.172+0000 7f54cb59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54cc0b7860 0x7f54cc149b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:04.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.172+0000 7f54cb59e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f54cc146890 0x7f54cc14a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:04.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.172+0000 7f54cb59e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54cc14a930 con 0x7f54cc0b7860 2022-06-29T16:53:04.190 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.172+0000 7f54cb59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54cc062610 con 0x7f54cc0b2d00 2022-06-29T16:53:04.191 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.172+0000 7f54cb59e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54cc062c00 con 0x7f54cc146890 2022-06-29T16:53:04.191 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.172+0000 7f54c3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54cc0b7860 0x7f54cc149b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:04.191 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.172+0000 7f54c3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54cc0b7860 0x7f54cc149b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55616/0 (socket says 172.21.15.47:55616) 2022-06-29T16:53:04.192 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.172+0000 7f54c3fff700 1 -- 172.21.15.47:0/737914449 learned_addr learned my addr 172.21.15.47:0/737914449 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T16:53:04.192 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.172+0000 7f54cad9d700 1 --2- 172.21.15.47:0/737914449 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f54cc146890 0x7f54cc14a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:04.192 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.172+0000 7f54ca59c700 1 --2- 172.21.15.47:0/737914449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54cc0b2d00 0x7f54cc1420b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:04.193 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.172+0000 7f54c3fff700 1 -- 172.21.15.47:0/737914449 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f54cc146890 msgr2=0x7f54cc14a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:04.193 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.172+0000 7f54c3fff700 1 --2- 172.21.15.47:0/737914449 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f54cc146890 0x7f54cc14a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:04.193 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.172+0000 7f54c3fff700 1 -- 172.21.15.47:0/737914449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54cc0b2d00 msgr2=0x7f54cc1420b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:04.194 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.172+0000 7f54c3fff700 1 --2- 172.21.15.47:0/737914449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54cc0b2d00 0x7f54cc1420b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:04.194 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.172+0000 7f54c3fff700 1 -- 172.21.15.47:0/737914449 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54bc00b180 con 0x7f54cc0b7860 2022-06-29T16:53:04.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.172+0000 7f54ca59c700 1 --2- 172.21.15.47:0/737914449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54cc0b2d00 0x7f54cc1420b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T16:53:04.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.172+0000 7f54c3fff700 1 --2- 172.21.15.47:0/737914449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54cc0b7860 0x7f54cc149b10 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 rx=0x7f54b4001110 tx=0x7f54b4007aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:04.195 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.172+0000 7f54c37fe700 1 -- 172.21.15.47:0/737914449 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54b40070d0 con 0x7f54cc0b7860 2022-06-29T16:53:04.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.172+0000 7f54c37fe700 1 -- 172.21.15.47:0/737914449 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54b4007230 con 0x7f54cc0b7860 2022-06-29T16:53:04.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.172+0000 7f54cb59e700 1 -- 172.21.15.47:0/737914449 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54cc062ea0 con 0x7f54cc0b7860 2022-06-29T16:53:04.196 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.172+0000 7f54c37fe700 1 -- 172.21.15.47:0/737914449 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54b4005610 con 0x7f54cc0b7860 2022-06-29T16:53:04.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.172+0000 7f54cb59e700 1 -- 172.21.15.47:0/737914449 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54cc142780 con 0x7f54cc0b7860 2022-06-29T16:53:04.197 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.176+0000 7f54cb59e700 1 -- 172.21.15.47:0/737914449 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54cc0573d0 con 0x7f54cc0b7860 2022-06-29T16:53:04.198 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.176+0000 7f54c37fe700 1 -- 172.21.15.47:0/737914449 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f54b401a070 con 0x7f54cc0b7860 2022-06-29T16:53:04.198 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.176+0000 7f54c37fe700 1 --2- 172.21.15.47:0/737914449 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f54b0053a10 0x7f54b0055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:04.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.176+0000 7f54ca59c700 1 --2- 172.21.15.47:0/737914449 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f54b0053a10 0x7f54b0055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:04.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.176+0000 7f54c37fe700 1 -- 172.21.15.47:0/737914449 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f54b406ad00 con 0x7f54cc0b7860 2022-06-29T16:53:04.199 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.176+0000 7f54ca59c700 1 --2- 172.21.15.47:0/737914449 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f54b0053a10 0x7f54b0055ed0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 rx=0x7f54bc015000 tx=0x7f54bc014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:04.200 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.184+0000 7f54c37fe700 1 -- 172.21.15.47:0/737914449 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f54b406e030 con 0x7f54cc0b7860 2022-06-29T16:53:04.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.312+0000 7f54cb59e700 1 -- 172.21.15.47:0/737914449 --> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f54cc0618d0 con 0x7f54b0053a10 2022-06-29T16:53:04.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.312+0000 7f54c37fe700 1 -- 172.21.15.47:0/737914449 <== mgr.4102 v2:172.21.15.47:6800/12928 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f54bc00d6d0 con 0x7f54b0053a10 2022-06-29T16:53:04.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.316+0000 7f54cb59e700 1 -- 172.21.15.47:0/737914449 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f54b0053a10 msgr2=0x7f54b0055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:04.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.316+0000 7f54cb59e700 1 --2- 172.21.15.47:0/737914449 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f54b0053a10 0x7f54b0055ed0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 rx=0x7f54bc015000 tx=0x7f54bc014000).stop 2022-06-29T16:53:04.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.316+0000 7f54cb59e700 1 -- 172.21.15.47:0/737914449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54cc0b7860 msgr2=0x7f54cc149b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:04.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.316+0000 7f54cb59e700 1 --2- 172.21.15.47:0/737914449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54cc0b7860 0x7f54cc149b10 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 rx=0x7f54b4001110 tx=0x7f54b4007aa0).stop 2022-06-29T16:53:04.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.316+0000 7f54cb59e700 1 -- 172.21.15.47:0/737914449 shutdown_connections 2022-06-29T16:53:04.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.316+0000 7f54cb59e700 1 --2- 172.21.15.47:0/737914449 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f54cc146890 0x7f54cc14a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:04.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.316+0000 7f54cb59e700 1 --2- 172.21.15.47:0/737914449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54cc0b7860 0x7f54cc149b10 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:04.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.316+0000 7f54cb59e700 1 --2- 172.21.15.47:0/737914449 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f54b0053a10 0x7f54b0055ed0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:04.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.316+0000 7f54cb59e700 1 --2- 172.21.15.47:0/737914449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54cc0b2d00 0x7f54cc1420b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:04.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.316+0000 7f54cb59e700 1 -- 172.21.15.47:0/737914449 >> 172.21.15.47:0/737914449 conn(0x7f54cc05f480 msgr2=0x7f54cc142ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T16:53:04.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.316+0000 7f54cb59e700 1 -- 172.21.15.47:0/737914449 shutdown_connections 2022-06-29T16:53:04.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.316+0000 7f54cb59e700 1 -- 172.21.15.47:0/737914449 wait complete. 2022-06-29T16:53:04.325 INFO:teuthology.orchestra.run.smithi047.stderr:instructed osd(s) 1 to deep-scrub 2022-06-29T16:53:04.335 INFO:tasks.ceph:Scrubbing osd.2 2022-06-29T16:53:04.337 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2022-06-29T16:53:04.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.444+0000 7f30819a8700 1 Processor -- start 2022-06-29T16:53:04.449 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.444+0000 7f30819a8700 1 -- start start 2022-06-29T16:53:04.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.444+0000 7f30819a8700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f307c0b58d0 0x7f307c0b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:04.453 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.444+0000 7f30819a8700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f307c0b1610 0x7f307c1475f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:04.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.444+0000 7f30819a8700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f307c14b4c0 0x7f307c0aa130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:04.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.444+0000 7f30819a8700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f307c0a99e0 con 0x7f307c14b4c0 2022-06-29T16:53:04.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.444+0000 7f30819a8700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f307c060790 con 0x7f307c0b1610 2022-06-29T16:53:04.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.444+0000 7f30819a8700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f307c14ad80 con 0x7f307c0b58d0 2022-06-29T16:53:04.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.444+0000 7f30811a7700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f307c14b4c0 0x7f307c0aa130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:04.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.444+0000 7f30811a7700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f307c14b4c0 0x7f307c0aa130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55628/0 (socket says 172.21.15.47:55628) 2022-06-29T16:53:04.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.444+0000 7f30811a7700 1 -- 172.21.15.47:0/3430262194 learned_addr learned my addr 172.21.15.47:0/3430262194 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T16:53:04.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.444+0000 7f307bfff700 1 --2- 172.21.15.47:0/3430262194 >> v2:172.21.15.64:3300/0 conn(0x7f307c0b1610 0x7f307c1475f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:04.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.444+0000 7f30809a6700 1 --2- 172.21.15.47:0/3430262194 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f307c0b58d0 0x7f307c0b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:04.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.444+0000 7f30811a7700 1 -- 172.21.15.47:0/3430262194 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f307c0b58d0 msgr2=0x7f307c0b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T16:53:04.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.444+0000 7f30811a7700 1 --2- 172.21.15.47:0/3430262194 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f307c0b58d0 0x7f307c0b5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:04.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.444+0000 7f30811a7700 1 -- 172.21.15.47:0/3430262194 >> v2:172.21.15.64:3300/0 conn(0x7f307c0b1610 msgr2=0x7f307c1475f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T16:53:04.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.444+0000 7f30811a7700 1 --2- 172.21.15.47:0/3430262194 >> v2:172.21.15.64:3300/0 conn(0x7f307c0b1610 0x7f307c1475f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:04.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.444+0000 7f30811a7700 1 -- 172.21.15.47:0/3430262194 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f307c14b020 con 0x7f307c14b4c0 2022-06-29T16:53:04.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.444+0000 7f30809a6700 1 --2- 172.21.15.47:0/3430262194 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f307c0b58d0 0x7f307c0b5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T16:53:04.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.444+0000 7f30811a7700 1 --2- 172.21.15.47:0/3430262194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f307c14b4c0 0x7f307c0aa130 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 rx=0x7f307c0623e0 tx=0x7f3074009290).ready entity=mon.0 client_cookie=cde3f40c29ca1326 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:04.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.444+0000 7f307b7fe700 1 -- 172.21.15.47:0/3430262194 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f307400b860 con 0x7f307c14b4c0 2022-06-29T16:53:04.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.444+0000 7f307b7fe700 1 -- 172.21.15.47:0/3430262194 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f307400b9c0 con 0x7f307c14b4c0 2022-06-29T16:53:04.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.444+0000 7f307b7fe700 1 -- 172.21.15.47:0/3430262194 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f307400bb30 con 0x7f307c14b4c0 2022-06-29T16:53:04.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.444+0000 7f30819a8700 1 -- 172.21.15.47:0/3430262194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f307c14b4c0 msgr2=0x7f307c0aa130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:04.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.444+0000 7f30819a8700 1 --2- 172.21.15.47:0/3430262194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f307c14b4c0 0x7f307c0aa130 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 rx=0x7f307c0623e0 tx=0x7f3074009290).stop 2022-06-29T16:53:04.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.444+0000 7f30819a8700 1 -- 172.21.15.47:0/3430262194 shutdown_connections 2022-06-29T16:53:04.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.444+0000 7f30819a8700 1 --2- 172.21.15.47:0/3430262194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f307c14b4c0 0x7f307c0aa130 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:04.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.444+0000 7f30819a8700 1 --2- 172.21.15.47:0/3430262194 >> v2:172.21.15.64:3300/0 conn(0x7f307c0b1610 0x7f307c1475f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:04.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.444+0000 7f30819a8700 1 --2- 172.21.15.47:0/3430262194 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f307c0b58d0 0x7f307c0b5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:04.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.444+0000 7f30819a8700 1 -- 172.21.15.47:0/3430262194 >> 172.21.15.47:0/3430262194 conn(0x7f307c062560 msgr2=0x7f307c062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T16:53:04.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.444+0000 7f30819a8700 1 -- 172.21.15.47:0/3430262194 shutdown_connections 2022-06-29T16:53:04.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.444+0000 7f30819a8700 1 -- 172.21.15.47:0/3430262194 wait complete. 2022-06-29T16:53:04.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.448+0000 7f30819a8700 1 Processor -- start 2022-06-29T16:53:04.467 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.448+0000 7f30819a8700 1 -- start start 2022-06-29T16:53:04.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.448+0000 7f30819a8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f307c0b1610 0x7f307c0ac6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:04.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.448+0000 7f30819a8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f307c0b58d0 0x7f307c14e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:04.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.448+0000 7f30819a8700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f307c14b4c0 0x7f307c0aea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:04.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.448+0000 7f30819a8700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307c0adcb0 con 0x7f307c0b1610 2022-06-29T16:53:04.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.448+0000 7f30819a8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307c065e20 con 0x7f307c0b58d0 2022-06-29T16:53:04.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.448+0000 7f30819a8700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f307c0660c0 con 0x7f307c14b4c0 2022-06-29T16:53:04.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.448+0000 7f30809a6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f307c0b1610 0x7f307c0ac6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:04.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.448+0000 7f307bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f307c0b58d0 0x7f307c14e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:04.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.448+0000 7f307bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f307c0b58d0 0x7f307c14e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:33102/0 (socket says 172.21.15.47:33102) 2022-06-29T16:53:04.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.448+0000 7f307bfff700 1 -- 172.21.15.47:0/1717129930 learned_addr learned my addr 172.21.15.47:0/1717129930 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T16:53:04.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.448+0000 7f30811a7700 1 --2- 172.21.15.47:0/1717129930 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f307c14b4c0 0x7f307c0aea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:04.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.448+0000 7f307bfff700 1 -- 172.21.15.47:0/1717129930 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f307c14b4c0 msgr2=0x7f307c0aea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:04.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.448+0000 7f307bfff700 1 --2- 172.21.15.47:0/1717129930 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f307c14b4c0 0x7f307c0aea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:04.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.448+0000 7f307bfff700 1 -- 172.21.15.47:0/1717129930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f307c0b1610 msgr2=0x7f307c0ac6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:04.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.448+0000 7f307bfff700 1 --2- 172.21.15.47:0/1717129930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f307c0b1610 0x7f307c0ac6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:04.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.448+0000 7f307bfff700 1 -- 172.21.15.47:0/1717129930 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3074009530 con 0x7f307c0b58d0 2022-06-29T16:53:04.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.452+0000 7f307bfff700 1 --2- 172.21.15.47:0/1717129930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f307c0b58d0 0x7f307c14e1c0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7f307c0addf0 tx=0x7f306c008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:04.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.452+0000 7f3079ffb700 1 -- 172.21.15.47:0/1717129930 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f306c011780 con 0x7f307c0b58d0 2022-06-29T16:53:04.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.452+0000 7f3079ffb700 1 -- 172.21.15.47:0/1717129930 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f306c00de90 con 0x7f307c0b58d0 2022-06-29T16:53:04.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.452+0000 7f30819a8700 1 -- 172.21.15.47:0/1717129930 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f307c066360 con 0x7f307c0b58d0 2022-06-29T16:53:04.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.452+0000 7f3079ffb700 1 -- 172.21.15.47:0/1717129930 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f306c011ad0 con 0x7f307c0b58d0 2022-06-29T16:53:04.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.452+0000 7f30819a8700 1 -- 172.21.15.47:0/1717129930 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f307c14e6b0 con 0x7f307c0b58d0 2022-06-29T16:53:04.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.452+0000 7f30819a8700 1 -- 172.21.15.47:0/1717129930 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f307c0ae7a0 con 0x7f307c0b58d0 2022-06-29T16:53:04.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.452+0000 7f3079ffb700 1 -- 172.21.15.47:0/1717129930 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f306c011c30 con 0x7f307c0b58d0 2022-06-29T16:53:04.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.452+0000 7f3079ffb700 1 --2- 172.21.15.47:0/1717129930 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3060053980 0x7f3060055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:04.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.456+0000 7f3079ffb700 1 -- 172.21.15.47:0/1717129930 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f306c06ca80 con 0x7f307c0b58d0 2022-06-29T16:53:04.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.456+0000 7f30809a6700 1 --2- 172.21.15.47:0/1717129930 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3060053980 0x7f3060055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:04.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.456+0000 7f3079ffb700 1 --2- 172.21.15.47:0/1717129930 >> [v2:172.21.15.47:6810/13098,v1:172.21.15.47:6811/13098] conn(0x7f306005a6b0 0x7f306005cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:04.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.456+0000 7f3079ffb700 1 -- 172.21.15.47:0/1717129930 --> [v2:172.21.15.47:6810/13098,v1:172.21.15.47:6811/13098] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f306005d1d0 con 0x7f306005a6b0 2022-06-29T16:53:04.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.456+0000 7f3079ffb700 1 -- 172.21.15.47:0/1717129930 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=165) v2 ==== 24+0+0 (secure 0 0 0) 0x7f306c06ce60 con 0x7f307c0b58d0 2022-06-29T16:53:04.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.456+0000 7f30811a7700 1 --2- 172.21.15.47:0/1717129930 >> [v2:172.21.15.47:6810/13098,v1:172.21.15.47:6811/13098] conn(0x7f306005a6b0 0x7f306005cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:04.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.456+0000 7f30809a6700 1 --2- 172.21.15.47:0/1717129930 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3060053980 0x7f3060055e40 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 rx=0x7f306400d000 tx=0x7f3064008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:04.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.456+0000 7f30811a7700 1 --2- 172.21.15.47:0/1717129930 >> [v2:172.21.15.47:6810/13098,v1:172.21.15.47:6811/13098] conn(0x7f306005a6b0 0x7f306005cb50 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:04.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.456+0000 7f3079ffb700 1 -- 172.21.15.47:0/1717129930 <== osd.2 v2:172.21.15.47:6810/13098 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f306005d1d0 con 0x7f306005a6b0 2022-06-29T16:53:04.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.472+0000 7f30819a8700 1 -- 172.21.15.47:0/1717129930 --> [v2:172.21.15.47:6810/13098,v1:172.21.15.47:6811/13098] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f307c0a93f0 con 0x7f306005a6b0 2022-06-29T16:53:04.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.472+0000 7f3079ffb700 1 -- 172.21.15.47:0/1717129930 <== osd.2 v2:172.21.15.47:6810/13098 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f307c0a93f0 con 0x7f306005a6b0 2022-06-29T16:53:04.482 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2022-06-29T16:53:04.483 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2022-06-29T16:53:04.483 INFO:teuthology.orchestra.run.smithi047.stdout:} 2022-06-29T16:53:04.485 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.472+0000 7f30819a8700 1 -- 172.21.15.47:0/1717129930 >> [v2:172.21.15.47:6810/13098,v1:172.21.15.47:6811/13098] conn(0x7f306005a6b0 msgr2=0x7f306005cb50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:04.485 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.472+0000 7f30819a8700 1 --2- 172.21.15.47:0/1717129930 >> [v2:172.21.15.47:6810/13098,v1:172.21.15.47:6811/13098] conn(0x7f306005a6b0 0x7f306005cb50 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:04.485 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.472+0000 7f30819a8700 1 -- 172.21.15.47:0/1717129930 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3060053980 msgr2=0x7f3060055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:04.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.472+0000 7f30819a8700 1 --2- 172.21.15.47:0/1717129930 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f3060053980 0x7f3060055e40 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 rx=0x7f306400d000 tx=0x7f3064008040).stop 2022-06-29T16:53:04.486 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.472+0000 7f30819a8700 1 -- 172.21.15.47:0/1717129930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f307c0b58d0 msgr2=0x7f307c14e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:04.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.472+0000 7f30819a8700 1 --2- 172.21.15.47:0/1717129930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f307c0b58d0 0x7f307c14e1c0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7f307c0addf0 tx=0x7f306c008040).stop 2022-06-29T16:53:04.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.472+0000 7f30811a7700 1 -- 172.21.15.47:0/1717129930 reap_dead start 2022-06-29T16:53:04.487 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.472+0000 7f30819a8700 1 -- 172.21.15.47:0/1717129930 shutdown_connections 2022-06-29T16:53:04.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.472+0000 7f30819a8700 1 -- 172.21.15.47:0/1717129930 >> 172.21.15.47:0/1717129930 conn(0x7f307c062560 msgr2=0x7f307c14a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T16:53:04.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.472+0000 7f30819a8700 1 -- 172.21.15.47:0/1717129930 shutdown_connections 2022-06-29T16:53:04.488 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.472+0000 7f30819a8700 1 -- 172.21.15.47:0/1717129930 wait complete. 2022-06-29T16:53:04.491 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2022-06-29T16:53:04.594 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.588+0000 7f9c0ce4e700 1 Processor -- start 2022-06-29T16:53:04.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.588+0000 7f9c0ce4e700 1 -- start start 2022-06-29T16:53:04.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.588+0000 7f9c0ce4e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9c080b7840 0x7f9c080b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:04.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.588+0000 7f9c0ce4e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9c08066c00 0x7f9c08141760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:04.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.588+0000 7f9c0ce4e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c080b7c20 con 0x7f9c08062dd0 2022-06-29T16:53:04.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.588+0000 7f9c0ce4e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c080b3d10 con 0x7f9c080b7840 2022-06-29T16:53:04.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.588+0000 7f9c0ce4e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c08067380 con 0x7f9c08066c00 2022-06-29T16:53:04.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.588+0000 7f9c077fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9c080b7840 0x7f9c080b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:04.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.588+0000 7f9c077fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9c080b7840 0x7f9c080b3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55640/0 (socket says 172.21.15.47:55640) 2022-06-29T16:53:04.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.588+0000 7f9c077fe700 1 -- 172.21.15.47:0/146292076 learned_addr learned my addr 172.21.15.47:0/146292076 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T16:53:04.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.588+0000 7f9c07fff700 1 --2- 172.21.15.47:0/146292076 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9c08066c00 0x7f9c08141760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:04.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.588+0000 7f9c067fc700 1 -- 172.21.15.47:0/146292076 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2458439574 0 0) 0x7f9c080b7c20 con 0x7f9c08062dd0 2022-06-29T16:53:04.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.588+0000 7f9c077fe700 1 -- 172.21.15.47:0/146292076 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9c08066c00 msgr2=0x7f9c08141760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T16:53:04.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.588+0000 7f9c077fe700 1 --2- 172.21.15.47:0/146292076 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9c08066c00 0x7f9c08141760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:04.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.588+0000 7f9c077fe700 1 -- 172.21.15.47:0/146292076 >> v1:172.21.15.64:6789/0 conn(0x7f9c08062dd0 legacy=0x7f9c080aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:04.603 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.588+0000 7f9c077fe700 1 -- 172.21.15.47:0/146292076 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c080b30e0 con 0x7f9c080b7840 2022-06-29T16:53:04.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.588+0000 7f9c077fe700 1 --2- 172.21.15.47:0/146292076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c080b7840 0x7f9c080b3820 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 rx=0x7f9bf0002f70 tx=0x7f9bf0006eb0).ready entity=mon.0 client_cookie=45a691b05eb78ec server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:04.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.588+0000 7f9c067fc700 1 -- 172.21.15.47:0/146292076 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9bf000d570 con 0x7f9c080b7840 2022-06-29T16:53:04.604 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.588+0000 7f9c067fc700 1 -- 172.21.15.47:0/146292076 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf0000e90 con 0x7f9c080b7840 2022-06-29T16:53:04.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.588+0000 7f9c067fc700 1 -- 172.21.15.47:0/146292076 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9bf000d8c0 con 0x7f9c080b7840 2022-06-29T16:53:04.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.592+0000 7f9c0ce4e700 1 -- 172.21.15.47:0/146292076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c080b7840 msgr2=0x7f9c080b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:04.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.592+0000 7f9c0ce4e700 1 --2- 172.21.15.47:0/146292076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c080b7840 0x7f9c080b3820 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 rx=0x7f9bf0002f70 tx=0x7f9bf0006eb0).stop 2022-06-29T16:53:04.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.592+0000 7f9c0ce4e700 1 -- 172.21.15.47:0/146292076 shutdown_connections 2022-06-29T16:53:04.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.592+0000 7f9c0ce4e700 1 --2- 172.21.15.47:0/146292076 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9c08066c00 0x7f9c08141760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:04.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.592+0000 7f9c0ce4e700 1 --2- 172.21.15.47:0/146292076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c080b7840 0x7f9c080b3820 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:04.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.592+0000 7f9c0ce4e700 1 -- 172.21.15.47:0/146292076 >> 172.21.15.47:0/146292076 conn(0x7f9c0805f480 msgr2=0x7f9c080647d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T16:53:04.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.592+0000 7f9c0ce4e700 1 -- 172.21.15.47:0/146292076 shutdown_connections 2022-06-29T16:53:04.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.592+0000 7f9c0ce4e700 1 -- 172.21.15.47:0/146292076 wait complete. 2022-06-29T16:53:04.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.592+0000 7f9c0ce4e700 1 Processor -- start 2022-06-29T16:53:04.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.592+0000 7f9c0ce4e700 1 -- start start 2022-06-29T16:53:04.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.592+0000 7f9c0ce4e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9c08062dd0 0x7f9c080aec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:04.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.592+0000 7f9c0ce4e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c08066c00 0x7f9c081453d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:04.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.592+0000 7f9c0ce4e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c080b7840 0x7f9c08152060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:04.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.592+0000 7f9c0ce4e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c08145dc0 con 0x7f9c080b7840 2022-06-29T16:53:04.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.592+0000 7f9c0ce4e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c08146370 con 0x7f9c08066c00 2022-06-29T16:53:04.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.592+0000 7f9c0ce4e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c08146610 con 0x7f9c08062dd0 2022-06-29T16:53:04.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.592+0000 7f9c07fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c080b7840 0x7f9c08152060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:04.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.592+0000 7f9c07fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c080b7840 0x7f9c08152060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55650/0 (socket says 172.21.15.47:55650) 2022-06-29T16:53:04.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.592+0000 7f9c07fff700 1 -- 172.21.15.47:0/3622651061 learned_addr learned my addr 172.21.15.47:0/3622651061 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T16:53:04.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.592+0000 7f9c077fe700 1 --2- 172.21.15.47:0/3622651061 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9c08062dd0 0x7f9c080aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:04.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.592+0000 7f9c06ffd700 1 --2- 172.21.15.47:0/3622651061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c08066c00 0x7f9c081453d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:04.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.592+0000 7f9c07fff700 1 -- 172.21.15.47:0/3622651061 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9c08062dd0 msgr2=0x7f9c080aec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:04.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.592+0000 7f9c07fff700 1 --2- 172.21.15.47:0/3622651061 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9c08062dd0 0x7f9c080aec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:04.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.592+0000 7f9c07fff700 1 -- 172.21.15.47:0/3622651061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c08066c00 msgr2=0x7f9c081453d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:04.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.592+0000 7f9c07fff700 1 --2- 172.21.15.47:0/3622651061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c08066c00 0x7f9c081453d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:04.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.592+0000 7f9c07fff700 1 -- 172.21.15.47:0/3622651061 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf000b1e0 con 0x7f9c080b7840 2022-06-29T16:53:04.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.592+0000 7f9c06ffd700 1 --2- 172.21.15.47:0/3622651061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c08066c00 0x7f9c081453d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T16:53:04.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.592+0000 7f9c07fff700 1 --2- 172.21.15.47:0/3622651061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c080b7840 0x7f9c08152060 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 rx=0x7f9c00006e00 tx=0x7f9c000091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:04.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.596+0000 7f9c04ff9700 1 -- 172.21.15.47:0/3622651061 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c0000eeb0 con 0x7f9c080b7840 2022-06-29T16:53:04.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.596+0000 7f9c04ff9700 1 -- 172.21.15.47:0/3622651061 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c0000c850 con 0x7f9c080b7840 2022-06-29T16:53:04.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.596+0000 7f9c0ce4e700 1 -- 172.21.15.47:0/3622651061 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c081460b0 con 0x7f9c080b7840 2022-06-29T16:53:04.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.596+0000 7f9c04ff9700 1 -- 172.21.15.47:0/3622651061 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c0000cbb0 con 0x7f9c080b7840 2022-06-29T16:53:04.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.596+0000 7f9c0ce4e700 1 -- 172.21.15.47:0/3622651061 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c081458c0 con 0x7f9c080b7840 2022-06-29T16:53:04.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.596+0000 7f9c0ce4e700 1 -- 172.21.15.47:0/3622651061 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c081467d0 con 0x7f9c080b7840 2022-06-29T16:53:04.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.596+0000 7f9c04ff9700 1 -- 172.21.15.47:0/3622651061 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9c0000cd10 con 0x7f9c080b7840 2022-06-29T16:53:04.622 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.596+0000 7f9c04ff9700 1 --2- 172.21.15.47:0/3622651061 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9bec053a70 0x7f9bec055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:04.622 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.596+0000 7f9c04ff9700 1 -- 172.21.15.47:0/3622651061 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f9c0006c040 con 0x7f9c080b7840 2022-06-29T16:53:04.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.596+0000 7f9c077fe700 1 --2- 172.21.15.47:0/3622651061 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9bec053a70 0x7f9bec055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:04.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.600+0000 7f9c077fe700 1 --2- 172.21.15.47:0/3622651061 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9bec053a70 0x7f9bec055f30 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 rx=0x7f9bf001c000 tx=0x7f9bf0015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:04.623 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.604+0000 7f9c04ff9700 1 -- 172.21.15.47:0/3622651061 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9c000430e0 con 0x7f9c080b7840 2022-06-29T16:53:04.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.736+0000 7f9c0ce4e700 1 -- 172.21.15.47:0/3622651061 --> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c080573d0 con 0x7f9bec053a70 2022-06-29T16:53:04.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.736+0000 7f9c04ff9700 1 -- 172.21.15.47:0/3622651061 <== mgr.4102 v2:172.21.15.47:6800/12928 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f9c080573d0 con 0x7f9bec053a70 2022-06-29T16:53:04.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.736+0000 7f9c0ce4e700 1 -- 172.21.15.47:0/3622651061 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9bec053a70 msgr2=0x7f9bec055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:04.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.736+0000 7f9c0ce4e700 1 --2- 172.21.15.47:0/3622651061 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9bec053a70 0x7f9bec055f30 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 rx=0x7f9bf001c000 tx=0x7f9bf0015000).stop 2022-06-29T16:53:04.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.736+0000 7f9c0ce4e700 1 -- 172.21.15.47:0/3622651061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c080b7840 msgr2=0x7f9c08152060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:04.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.736+0000 7f9c0ce4e700 1 --2- 172.21.15.47:0/3622651061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c080b7840 0x7f9c08152060 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 rx=0x7f9c00006e00 tx=0x7f9c000091d0).stop 2022-06-29T16:53:04.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.736+0000 7f9c0ce4e700 1 -- 172.21.15.47:0/3622651061 shutdown_connections 2022-06-29T16:53:04.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.736+0000 7f9c0ce4e700 1 --2- 172.21.15.47:0/3622651061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c080b7840 0x7f9c08152060 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:04.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.736+0000 7f9c0ce4e700 1 --2- 172.21.15.47:0/3622651061 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f9bec053a70 0x7f9bec055f30 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:04.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.736+0000 7f9c0ce4e700 1 --2- 172.21.15.47:0/3622651061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c08066c00 0x7f9c081453d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:04.748 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.736+0000 7f9c0ce4e700 1 --2- 172.21.15.47:0/3622651061 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f9c08062dd0 0x7f9c080aec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:04.748 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.736+0000 7f9c0ce4e700 1 -- 172.21.15.47:0/3622651061 >> 172.21.15.47:0/3622651061 conn(0x7f9c0805f480 msgr2=0x7f9c080b0250 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T16:53:04.748 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.736+0000 7f9c0ce4e700 1 -- 172.21.15.47:0/3622651061 shutdown_connections 2022-06-29T16:53:04.749 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.736+0000 7f9c0ce4e700 1 -- 172.21.15.47:0/3622651061 wait complete. 2022-06-29T16:53:04.749 INFO:teuthology.orchestra.run.smithi047.stderr:instructed osd(s) 2 to deep-scrub 2022-06-29T16:53:04.755 INFO:tasks.ceph:Scrubbing osd.3 2022-06-29T16:53:04.756 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2022-06-29T16:53:04.874 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.868+0000 7fb03c044700 1 Processor -- start 2022-06-29T16:53:04.875 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.868+0000 7fb03c044700 1 -- start start 2022-06-29T16:53:04.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.868+0000 7fb03c044700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb034067500 0x7fb034140f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:04.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.868+0000 7fb03c044700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb03413e930 0x7fb034066ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:04.880 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.868+0000 7fb03c044700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb03405f7c0 con 0x7fb03413e550 2022-06-29T16:53:04.880 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.868+0000 7fb03c044700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0340b3a80 con 0x7fb03413e930 2022-06-29T16:53:04.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.868+0000 7fb03c044700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0340b4ca0 con 0x7fb034067500 2022-06-29T16:53:04.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.868+0000 7fb039de0700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb03413e550 0x7fb03405f0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41208/0 (socket says 172.21.15.47:41208) 2022-06-29T16:53:04.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.868+0000 7fb039de0700 1 -- 172.21.15.47:0/3782911029 learned_addr learned my addr 172.21.15.47:0/3782911029 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T16:53:04.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.868+0000 7fb03a5e1700 1 --2- 172.21.15.47:0/3782911029 >> v2:172.21.15.47:3300/0 conn(0x7fb03413e930 0x7fb034066ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:04.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.868+0000 7fb0395df700 1 --2- 172.21.15.47:0/3782911029 >> v2:172.21.15.64:3300/0 conn(0x7fb034067500 0x7fb034140f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:04.883 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.868+0000 7fb038dde700 1 -- 172.21.15.47:0/3782911029 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1509812301 0 0) 0x7fb03405f7c0 con 0x7fb03413e550 2022-06-29T16:53:04.883 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.868+0000 7fb038dde700 1 -- 172.21.15.47:0/3782911029 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb020003260 con 0x7fb03413e550 2022-06-29T16:53:04.883 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.868+0000 7fb03a5e1700 1 -- 172.21.15.47:0/3782911029 >> v2:172.21.15.64:3300/0 conn(0x7fb034067500 msgr2=0x7fb034140f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T16:53:04.884 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.868+0000 7fb03a5e1700 1 --2- 172.21.15.47:0/3782911029 >> v2:172.21.15.64:3300/0 conn(0x7fb034067500 0x7fb034140f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:04.884 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.868+0000 7fb03a5e1700 1 -- 172.21.15.47:0/3782911029 >> v1:172.21.15.47:6789/0 conn(0x7fb03413e550 legacy=0x7fb03405f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:04.885 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.868+0000 7fb03a5e1700 1 -- 172.21.15.47:0/3782911029 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0340b37e0 con 0x7fb03413e930 2022-06-29T16:53:04.885 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.872+0000 7fb03a5e1700 1 --2- 172.21.15.47:0/3782911029 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb03413e930 0x7fb034066ba0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 rx=0x7fb02c0032f0 tx=0x7fb02c0091d0).ready entity=mon.0 client_cookie=2b01b8264fbb5c65 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:04.886 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.872+0000 7fb038dde700 1 -- 172.21.15.47:0/3782911029 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb02c00b600 con 0x7fb03413e930 2022-06-29T16:53:04.886 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.872+0000 7fb038dde700 1 -- 172.21.15.47:0/3782911029 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb02c00b760 con 0x7fb03413e930 2022-06-29T16:53:04.886 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.872+0000 7fb038dde700 1 -- 172.21.15.47:0/3782911029 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb02c00b8d0 con 0x7fb03413e930 2022-06-29T16:53:04.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.872+0000 7fb03c044700 1 -- 172.21.15.47:0/3782911029 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb03413e930 msgr2=0x7fb034066ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:04.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.872+0000 7fb03c044700 1 --2- 172.21.15.47:0/3782911029 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb03413e930 0x7fb034066ba0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 rx=0x7fb02c0032f0 tx=0x7fb02c0091d0).stop 2022-06-29T16:53:04.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.872+0000 7fb03c044700 1 -- 172.21.15.47:0/3782911029 shutdown_connections 2022-06-29T16:53:04.888 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.872+0000 7fb03c044700 1 --2- 172.21.15.47:0/3782911029 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb03413e930 0x7fb034066ba0 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:04.888 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.872+0000 7fb03c044700 1 --2- 172.21.15.47:0/3782911029 >> v2:172.21.15.64:3300/0 conn(0x7fb034067500 0x7fb034140f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:04.888 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.872+0000 7fb03c044700 1 -- 172.21.15.47:0/3782911029 >> 172.21.15.47:0/3782911029 conn(0x7fb0340627d0 msgr2=0x7fb034062bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T16:53:04.889 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.872+0000 7fb03c044700 1 -- 172.21.15.47:0/3782911029 shutdown_connections 2022-06-29T16:53:04.889 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.872+0000 7fb03c044700 1 -- 172.21.15.47:0/3782911029 wait complete. 2022-06-29T16:53:04.889 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.872+0000 7fb03c044700 1 Processor -- start 2022-06-29T16:53:04.890 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.872+0000 7fb03c044700 1 -- start start 2022-06-29T16:53:04.895 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.872+0000 7fb03c044700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb034067500 0x7fb03413d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:04.895 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.872+0000 7fb03c044700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb03413e550 0x7fb034144380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:04.896 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.872+0000 7fb03c044700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb03413e930 0x7fb034144870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:04.896 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.872+0000 7fb03c044700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb034143720 con 0x7fb03413e930 2022-06-29T16:53:04.896 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.872+0000 7fb03c044700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0341439c0 con 0x7fb034067500 2022-06-29T16:53:04.897 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.872+0000 7fb03c044700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb034143c60 con 0x7fb03413e550 2022-06-29T16:53:04.897 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.872+0000 7fb03a5e1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb03413e930 0x7fb034144870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:04.897 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.872+0000 7fb03a5e1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb03413e930 0x7fb034144870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55662/0 (socket says 172.21.15.47:55662) 2022-06-29T16:53:04.898 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.872+0000 7fb03a5e1700 1 -- 172.21.15.47:0/3805514044 learned_addr learned my addr 172.21.15.47:0/3805514044 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T16:53:04.898 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.872+0000 7fb039de0700 1 --2- 172.21.15.47:0/3805514044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb034067500 0x7fb03413d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:04.898 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.872+0000 7fb0395df700 1 --2- 172.21.15.47:0/3805514044 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb03413e550 0x7fb034144380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:04.899 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.872+0000 7fb03a5e1700 1 -- 172.21.15.47:0/3805514044 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb03413e550 msgr2=0x7fb034144380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:04.899 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.872+0000 7fb03a5e1700 1 --2- 172.21.15.47:0/3805514044 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb03413e550 0x7fb034144380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:04.899 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.872+0000 7fb03a5e1700 1 -- 172.21.15.47:0/3805514044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb034067500 msgr2=0x7fb03413d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:04.900 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.872+0000 7fb03a5e1700 1 --2- 172.21.15.47:0/3805514044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb034067500 0x7fb03413d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:04.900 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.872+0000 7fb03a5e1700 1 -- 172.21.15.47:0/3805514044 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb02c009470 con 0x7fb03413e930 2022-06-29T16:53:04.900 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.872+0000 7fb039de0700 1 --2- 172.21.15.47:0/3805514044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb034067500 0x7fb03413d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T16:53:04.901 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.872+0000 7fb0395df700 1 --2- 172.21.15.47:0/3805514044 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fb03413e550 0x7fb034144380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-06-29T16:53:04.901 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.872+0000 7fb03a5e1700 1 --2- 172.21.15.47:0/3805514044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb03413e930 0x7fb034144870 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 rx=0x7fb02c0032f0 tx=0x7fb02c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:04.901 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.872+0000 7fb032ffd700 1 -- 172.21.15.47:0/3805514044 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb02c00b8d0 con 0x7fb03413e930 2022-06-29T16:53:04.902 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.876+0000 7fb032ffd700 1 -- 172.21.15.47:0/3805514044 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb02c01f5e0 con 0x7fb03413e930 2022-06-29T16:53:04.902 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.876+0000 7fb03c044700 1 -- 172.21.15.47:0/3805514044 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb034143f00 con 0x7fb03413e930 2022-06-29T16:53:04.902 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.876+0000 7fb032ffd700 1 -- 172.21.15.47:0/3805514044 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb02c00b600 con 0x7fb03413e930 2022-06-29T16:53:04.903 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.876+0000 7fb03c044700 1 -- 172.21.15.47:0/3805514044 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0340adf00 con 0x7fb03413e930 2022-06-29T16:53:04.903 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.876+0000 7fb03c044700 1 -- 172.21.15.47:0/3805514044 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb034144090 con 0x7fb03413e930 2022-06-29T16:53:04.904 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.876+0000 7fb032ffd700 1 -- 172.21.15.47:0/3805514044 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb02c026030 con 0x7fb03413e930 2022-06-29T16:53:04.904 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.876+0000 7fb032ffd700 1 --2- 172.21.15.47:0/3805514044 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb0140539d0 0x7fb014055e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:04.904 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.876+0000 7fb032ffd700 1 -- 172.21.15.47:0/3805514044 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fb02c06c710 con 0x7fb03413e930 2022-06-29T16:53:04.905 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.876+0000 7fb032ffd700 1 --2- 172.21.15.47:0/3805514044 >> [v2:172.21.15.47:6802/13097,v1:172.21.15.47:6803/13097] conn(0x7fb01405a780 0x7fb01405cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:04.905 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.876+0000 7fb032ffd700 1 -- 172.21.15.47:0/3805514044 --> [v2:172.21.15.47:6802/13097,v1:172.21.15.47:6803/13097] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb01405d2a0 con 0x7fb01405a780 2022-06-29T16:53:04.905 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.876+0000 7fb032ffd700 1 -- 172.21.15.47:0/3805514044 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=165) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb02c06ed20 con 0x7fb03413e930 2022-06-29T16:53:04.906 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.876+0000 7fb039de0700 1 --2- 172.21.15.47:0/3805514044 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb0140539d0 0x7fb014055e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:04.906 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.876+0000 7fb0395df700 1 --2- 172.21.15.47:0/3805514044 >> [v2:172.21.15.47:6802/13097,v1:172.21.15.47:6803/13097] conn(0x7fb01405a780 0x7fb01405cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:04.907 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.876+0000 7fb039de0700 1 --2- 172.21.15.47:0/3805514044 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb0140539d0 0x7fb014055e90 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 rx=0x7fb024008000 tx=0x7fb024009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:04.907 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.876+0000 7fb0395df700 1 --2- 172.21.15.47:0/3805514044 >> [v2:172.21.15.47:6802/13097,v1:172.21.15.47:6803/13097] conn(0x7fb01405a780 0x7fb01405cc20 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:04.907 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.880+0000 7fb032ffd700 1 -- 172.21.15.47:0/3805514044 <== osd.3 v2:172.21.15.47:6802/13097 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7fb01405d2a0 con 0x7fb01405a780 2022-06-29T16:53:04.908 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.896+0000 7fb03c044700 1 -- 172.21.15.47:0/3805514044 --> [v2:172.21.15.47:6802/13097,v1:172.21.15.47:6803/13097] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb034145020 con 0x7fb01405a780 2022-06-29T16:53:04.909 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.896+0000 7fb032ffd700 1 -- 172.21.15.47:0/3805514044 <== osd.3 v2:172.21.15.47:6802/13097 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fb034145020 con 0x7fb01405a780 2022-06-29T16:53:04.909 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2022-06-29T16:53:04.910 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2022-06-29T16:53:04.910 INFO:teuthology.orchestra.run.smithi047.stdout:} 2022-06-29T16:53:04.911 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.896+0000 7fb03c044700 1 -- 172.21.15.47:0/3805514044 >> [v2:172.21.15.47:6802/13097,v1:172.21.15.47:6803/13097] conn(0x7fb01405a780 msgr2=0x7fb01405cc20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:04.912 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.896+0000 7fb03c044700 1 --2- 172.21.15.47:0/3805514044 >> [v2:172.21.15.47:6802/13097,v1:172.21.15.47:6803/13097] conn(0x7fb01405a780 0x7fb01405cc20 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:04.912 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.896+0000 7fb03c044700 1 -- 172.21.15.47:0/3805514044 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb0140539d0 msgr2=0x7fb014055e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:04.912 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.896+0000 7fb03c044700 1 --2- 172.21.15.47:0/3805514044 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fb0140539d0 0x7fb014055e90 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 rx=0x7fb024008000 tx=0x7fb024009040).stop 2022-06-29T16:53:04.913 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.896+0000 7fb03c044700 1 -- 172.21.15.47:0/3805514044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb03413e930 msgr2=0x7fb034144870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:04.913 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.896+0000 7fb03c044700 1 --2- 172.21.15.47:0/3805514044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb03413e930 0x7fb034144870 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 rx=0x7fb02c0032f0 tx=0x7fb02c016000).stop 2022-06-29T16:53:04.913 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.896+0000 7fb03a5e1700 1 -- 172.21.15.47:0/3805514044 reap_dead start 2022-06-29T16:53:04.914 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.896+0000 7fb03c044700 1 -- 172.21.15.47:0/3805514044 shutdown_connections 2022-06-29T16:53:04.914 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.896+0000 7fb03c044700 1 -- 172.21.15.47:0/3805514044 >> 172.21.15.47:0/3805514044 conn(0x7fb0340627d0 msgr2=0x7fb03405f410 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T16:53:04.914 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.900+0000 7fb03c044700 1 -- 172.21.15.47:0/3805514044 shutdown_connections 2022-06-29T16:53:04.915 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:04.900+0000 7fb03c044700 1 -- 172.21.15.47:0/3805514044 wait complete. 2022-06-29T16:53:04.918 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2022-06-29T16:53:05.030 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.024+0000 7fd92b59e700 1 Processor -- start 2022-06-29T16:53:05.031 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.024+0000 7fd92b59e700 1 -- start start 2022-06-29T16:53:05.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.024+0000 7fd92b59e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd92c0ad730 0x7fd92c147370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:05.034 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.024+0000 7fd92b59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd92c1443f0 0x7fd92c144880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:05.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.024+0000 7fd92b59e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd92c098250 con 0x7fd92c0adb10 2022-06-29T16:53:05.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.024+0000 7fd92b59e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd92c14a940 con 0x7fd92c0ad730 2022-06-29T16:53:05.035 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.024+0000 7fd92b59e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd92c098480 con 0x7fd92c1443f0 2022-06-29T16:53:05.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.024+0000 7fd92a59c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd92c0ad730 0x7fd92c147370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:05.036 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.024+0000 7fd92a59c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd92c0ad730 0x7fd92c147370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55672/0 (socket says 172.21.15.47:55672) 2022-06-29T16:53:05.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.024+0000 7fd92a59c700 1 -- 172.21.15.47:0/1614763699 learned_addr learned my addr 172.21.15.47:0/1614763699 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T16:53:05.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.024+0000 7fd92ad9d700 1 --2- 172.21.15.47:0/1614763699 >> v2:172.21.15.64:3300/0 conn(0x7fd92c1443f0 0x7fd92c144880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:05.037 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.024+0000 7fd92a59c700 1 -- 172.21.15.47:0/1614763699 >> v2:172.21.15.64:3300/0 conn(0x7fd92c1443f0 msgr2=0x7fd92c144880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T16:53:05.038 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.024+0000 7fd92a59c700 1 --2- 172.21.15.47:0/1614763699 >> v2:172.21.15.64:3300/0 conn(0x7fd92c1443f0 0x7fd92c144880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:05.038 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.024+0000 7fd92a59c700 1 -- 172.21.15.47:0/1614763699 >> v1:172.21.15.64:6789/0 conn(0x7fd92c0adb10 legacy=0x7fd92c14b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:05.038 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.028+0000 7fd92a59c700 1 -- 172.21.15.47:0/1614763699 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd92c14aba0 con 0x7fd92c0ad730 2022-06-29T16:53:05.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.028+0000 7fd92a59c700 1 --2- 172.21.15.47:0/1614763699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd92c0ad730 0x7fd92c147370 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 rx=0x7fd92c14af20 tx=0x7fd920006eb0).ready entity=mon.0 client_cookie=3d4b582945e57aab server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:05.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.028+0000 7fd92959a700 1 -- 172.21.15.47:0/1614763699 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd92000d610 con 0x7fd92c0ad730 2022-06-29T16:53:05.039 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.028+0000 7fd92959a700 1 -- 172.21.15.47:0/1614763699 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd920000e90 con 0x7fd92c0ad730 2022-06-29T16:53:05.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.028+0000 7fd92959a700 1 -- 172.21.15.47:0/1614763699 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd92000d9a0 con 0x7fd92c0ad730 2022-06-29T16:53:05.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.028+0000 7fd92b59e700 1 -- 172.21.15.47:0/1614763699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd92c0ad730 msgr2=0x7fd92c147370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:05.040 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.028+0000 7fd92b59e700 1 --2- 172.21.15.47:0/1614763699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd92c0ad730 0x7fd92c147370 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 rx=0x7fd92c14af20 tx=0x7fd920006eb0).stop 2022-06-29T16:53:05.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.028+0000 7fd92b59e700 1 -- 172.21.15.47:0/1614763699 shutdown_connections 2022-06-29T16:53:05.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.028+0000 7fd92b59e700 1 --2- 172.21.15.47:0/1614763699 >> v2:172.21.15.64:3300/0 conn(0x7fd92c1443f0 0x7fd92c144880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:05.041 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.028+0000 7fd92b59e700 1 --2- 172.21.15.47:0/1614763699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd92c0ad730 0x7fd92c147370 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:05.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.028+0000 7fd92b59e700 1 -- 172.21.15.47:0/1614763699 >> 172.21.15.47:0/1614763699 conn(0x7fd92c062df0 msgr2=0x7fd92c0631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T16:53:05.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.028+0000 7fd92b59e700 1 -- 172.21.15.47:0/1614763699 shutdown_connections 2022-06-29T16:53:05.042 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.028+0000 7fd92b59e700 1 -- 172.21.15.47:0/1614763699 wait complete. 2022-06-29T16:53:05.045 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.028+0000 7fd92b59e700 1 Processor -- start 2022-06-29T16:53:05.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.028+0000 7fd92b59e700 1 -- start start 2022-06-29T16:53:05.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.028+0000 7fd92b59e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd92c0ad730 0x7fd92c145d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:05.046 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.028+0000 7fd92b59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd92c0adb10 0x7fd92c09a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:05.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.028+0000 7fd92b59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd92c1443f0 0x7fd92c156da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:05.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.028+0000 7fd92b59e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92c145830 con 0x7fd92c1443f0 2022-06-29T16:53:05.047 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.028+0000 7fd92b59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92c145590 con 0x7fd92c0adb10 2022-06-29T16:53:05.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.028+0000 7fd92b59e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd92c156660 con 0x7fd92c0ad730 2022-06-29T16:53:05.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.028+0000 7fd92ad9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd92c1443f0 0x7fd92c156da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:05.048 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.028+0000 7fd92ad9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd92c1443f0 0x7fd92c156da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55680/0 (socket says 172.21.15.47:55680) 2022-06-29T16:53:05.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.028+0000 7fd92ad9d700 1 -- 172.21.15.47:0/3308258722 learned_addr learned my addr 172.21.15.47:0/3308258722 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T16:53:05.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.028+0000 7fd92a59c700 1 --2- 172.21.15.47:0/3308258722 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd92c0ad730 0x7fd92c145d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:05.049 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.032+0000 7fd929d9b700 1 --2- 172.21.15.47:0/3308258722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd92c0adb10 0x7fd92c09a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:05.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.032+0000 7fd92ad9d700 1 -- 172.21.15.47:0/3308258722 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd92c0ad730 msgr2=0x7fd92c145d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:05.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.032+0000 7fd92ad9d700 1 --2- 172.21.15.47:0/3308258722 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd92c0ad730 0x7fd92c145d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:05.050 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.032+0000 7fd92ad9d700 1 -- 172.21.15.47:0/3308258722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd92c0adb10 msgr2=0x7fd92c09a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:05.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.032+0000 7fd92ad9d700 1 --2- 172.21.15.47:0/3308258722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd92c0adb10 0x7fd92c09a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:05.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.032+0000 7fd92ad9d700 1 -- 172.21.15.47:0/3308258722 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd92000b1e0 con 0x7fd92c1443f0 2022-06-29T16:53:05.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.032+0000 7fd92ad9d700 1 --2- 172.21.15.47:0/3308258722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd92c1443f0 0x7fd92c156da0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 rx=0x7fd924006e00 tx=0x7fd9240091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:05.051 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.032+0000 7fd91b7fe700 1 -- 172.21.15.47:0/3308258722 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd924012600 con 0x7fd92c1443f0 2022-06-29T16:53:05.052 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.032+0000 7fd91b7fe700 1 -- 172.21.15.47:0/3308258722 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd924012760 con 0x7fd92c1443f0 2022-06-29T16:53:05.052 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.032+0000 7fd92b59e700 1 -- 172.21.15.47:0/3308258722 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd92c156900 con 0x7fd92c1443f0 2022-06-29T16:53:05.053 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.032+0000 7fd91b7fe700 1 -- 172.21.15.47:0/3308258722 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd924012ac0 con 0x7fd92c1443f0 2022-06-29T16:53:05.053 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.032+0000 7fd92b59e700 1 -- 172.21.15.47:0/3308258722 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd92c157290 con 0x7fd92c1443f0 2022-06-29T16:53:05.053 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.032+0000 7fd92b59e700 1 -- 172.21.15.47:0/3308258722 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd92c156a90 con 0x7fd92c1443f0 2022-06-29T16:53:05.054 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.032+0000 7fd91b7fe700 1 -- 172.21.15.47:0/3308258722 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd924022070 con 0x7fd92c1443f0 2022-06-29T16:53:05.054 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.032+0000 7fd91b7fe700 1 --2- 172.21.15.47:0/3308258722 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd908053b00 0x7fd908055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:05.054 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.032+0000 7fd91b7fe700 1 -- 172.21.15.47:0/3308258722 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fd92406c280 con 0x7fd92c1443f0 2022-06-29T16:53:05.055 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.032+0000 7fd92a59c700 1 --2- 172.21.15.47:0/3308258722 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd908053b00 0x7fd908055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:05.055 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.036+0000 7fd92a59c700 1 --2- 172.21.15.47:0/3308258722 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd908053b00 0x7fd908055fc0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 rx=0x7fd92001c000 tx=0x7fd920015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:05.056 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.040+0000 7fd91b7fe700 1 -- 172.21.15.47:0/3308258722 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd924043330 con 0x7fd92c1443f0 2022-06-29T16:53:05.175 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.168+0000 7fd92b59e700 1 -- 172.21.15.47:0/3308258722 --> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fd92c1564e0 con 0x7fd908053b00 2022-06-29T16:53:05.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.168+0000 7fd91b7fe700 1 -- 172.21.15.47:0/3308258722 <== mgr.4102 v2:172.21.15.47:6800/12928 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd92000db90 con 0x7fd908053b00 2022-06-29T16:53:05.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.172+0000 7fd92b59e700 1 -- 172.21.15.47:0/3308258722 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd908053b00 msgr2=0x7fd908055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:05.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.172+0000 7fd92b59e700 1 --2- 172.21.15.47:0/3308258722 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd908053b00 0x7fd908055fc0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 rx=0x7fd92001c000 tx=0x7fd920015000).stop 2022-06-29T16:53:05.178 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.172+0000 7fd92b59e700 1 -- 172.21.15.47:0/3308258722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd92c1443f0 msgr2=0x7fd92c156da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:05.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.172+0000 7fd92b59e700 1 --2- 172.21.15.47:0/3308258722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd92c1443f0 0x7fd92c156da0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 rx=0x7fd924006e00 tx=0x7fd9240091d0).stop 2022-06-29T16:53:05.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.172+0000 7fd92b59e700 1 -- 172.21.15.47:0/3308258722 shutdown_connections 2022-06-29T16:53:05.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.172+0000 7fd92b59e700 1 --2- 172.21.15.47:0/3308258722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd92c1443f0 0x7fd92c156da0 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:05.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.172+0000 7fd92b59e700 1 --2- 172.21.15.47:0/3308258722 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fd908053b00 0x7fd908055fc0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:05.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.172+0000 7fd92b59e700 1 --2- 172.21.15.47:0/3308258722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd92c0adb10 0x7fd92c09a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:05.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.172+0000 7fd92b59e700 1 --2- 172.21.15.47:0/3308258722 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fd92c0ad730 0x7fd92c145d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:05.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.172+0000 7fd92b59e700 1 -- 172.21.15.47:0/3308258722 >> 172.21.15.47:0/3308258722 conn(0x7fd92c062df0 msgr2=0x7fd92c144880 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T16:53:05.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.172+0000 7fd92b59e700 1 -- 172.21.15.47:0/3308258722 shutdown_connections 2022-06-29T16:53:05.182 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.172+0000 7fd92b59e700 1 -- 172.21.15.47:0/3308258722 wait complete. 2022-06-29T16:53:05.183 INFO:teuthology.orchestra.run.smithi047.stderr:instructed osd(s) 3 to deep-scrub 2022-06-29T16:53:05.189 INFO:tasks.ceph:Scrubbing osd.4 2022-06-29T16:53:05.190 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2022-06-29T16:53:05.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.292+0000 7f7de3fff700 1 Processor -- start 2022-06-29T16:53:05.301 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.292+0000 7f7de3fff700 1 -- start start 2022-06-29T16:53:05.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.296+0000 7f7de3fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7de405f4a0 0x7f7de40b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:05.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.296+0000 7f7de3fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7de409a1d0 0x7f7de409a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:05.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.296+0000 7f7de3fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7de4142a30 0x7f7de4142ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:05.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.296+0000 7f7de3fff700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7de409ab20 con 0x7f7de4142a30 2022-06-29T16:53:05.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.296+0000 7f7de3fff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7de405f880 con 0x7f7de409a1d0 2022-06-29T16:53:05.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.296+0000 7f7de3fff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7de409ae90 con 0x7f7de405f4a0 2022-06-29T16:53:05.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.296+0000 7f7de37fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7de4142a30 0x7f7de4142ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:05.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.296+0000 7f7de37fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7de4142a30 0x7f7de4142ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55688/0 (socket says 172.21.15.47:55688) 2022-06-29T16:53:05.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.296+0000 7f7de37fe700 1 -- 172.21.15.47:0/2122667569 learned_addr learned my addr 172.21.15.47:0/2122667569 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T16:53:05.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.296+0000 7f7dda7fc700 1 --2- 172.21.15.47:0/2122667569 >> v2:172.21.15.64:3300/0 conn(0x7f7de409a1d0 0x7f7de409a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:05.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.296+0000 7f7de2ffd700 1 --2- 172.21.15.47:0/2122667569 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7de405f4a0 0x7f7de40b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:05.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.296+0000 7f7de37fe700 1 -- 172.21.15.47:0/2122667569 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7de405f4a0 msgr2=0x7f7de40b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T16:53:05.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.296+0000 7f7de37fe700 1 --2- 172.21.15.47:0/2122667569 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7de405f4a0 0x7f7de40b2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:05.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.296+0000 7f7de37fe700 1 -- 172.21.15.47:0/2122667569 >> v2:172.21.15.64:3300/0 conn(0x7f7de409a1d0 msgr2=0x7f7de409a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T16:53:05.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.296+0000 7f7de37fe700 1 --2- 172.21.15.47:0/2122667569 >> v2:172.21.15.64:3300/0 conn(0x7f7de409a1d0 0x7f7de409a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:05.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.296+0000 7f7de37fe700 1 -- 172.21.15.47:0/2122667569 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7de409b130 con 0x7f7de4142a30 2022-06-29T16:53:05.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.296+0000 7f7de37fe700 1 --2- 172.21.15.47:0/2122667569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7de4142a30 0x7f7de4142ea0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 rx=0x7f7ddc0032f0 tx=0x7f7ddc009290).ready entity=mon.0 client_cookie=55a2151cb1a94fd2 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:05.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.296+0000 7f7de27fc700 1 -- 172.21.15.47:0/2122667569 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ddc00cb70 con 0x7f7de4142a30 2022-06-29T16:53:05.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.296+0000 7f7de27fc700 1 -- 172.21.15.47:0/2122667569 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ddc00ccd0 con 0x7f7de4142a30 2022-06-29T16:53:05.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.296+0000 7f7de27fc700 1 -- 172.21.15.47:0/2122667569 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ddc010600 con 0x7f7de4142a30 2022-06-29T16:53:05.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.296+0000 7f7de3fff700 1 -- 172.21.15.47:0/2122667569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7de4142a30 msgr2=0x7f7de4142ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:05.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.296+0000 7f7de3fff700 1 --2- 172.21.15.47:0/2122667569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7de4142a30 0x7f7de4142ea0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 rx=0x7f7ddc0032f0 tx=0x7f7ddc009290).stop 2022-06-29T16:53:05.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.296+0000 7f7de3fff700 1 -- 172.21.15.47:0/2122667569 shutdown_connections 2022-06-29T16:53:05.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.296+0000 7f7de3fff700 1 --2- 172.21.15.47:0/2122667569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7de4142a30 0x7f7de4142ea0 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:05.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.296+0000 7f7de3fff700 1 --2- 172.21.15.47:0/2122667569 >> v2:172.21.15.64:3300/0 conn(0x7f7de409a1d0 0x7f7de409a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:05.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.296+0000 7f7de3fff700 1 --2- 172.21.15.47:0/2122667569 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7de405f4a0 0x7f7de40b2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:05.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.296+0000 7f7de3fff700 1 -- 172.21.15.47:0/2122667569 >> 172.21.15.47:0/2122667569 conn(0x7f7de4062df0 msgr2=0x7f7de40631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T16:53:05.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.296+0000 7f7de3fff700 1 -- 172.21.15.47:0/2122667569 shutdown_connections 2022-06-29T16:53:05.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.296+0000 7f7de3fff700 1 -- 172.21.15.47:0/2122667569 wait complete. 2022-06-29T16:53:05.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.296+0000 7f7de3fff700 1 Processor -- start 2022-06-29T16:53:05.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.300+0000 7f7de3fff700 1 -- start start 2022-06-29T16:53:05.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.300+0000 7f7de3fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7de405f4a0 0x7f7de4095610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:05.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.300+0000 7f7de3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7de409a1d0 0x7f7de40af2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:05.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.300+0000 7f7de3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7de4142a30 0x7f7de40943d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:05.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.300+0000 7f7de3fff700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de4152d80 con 0x7f7de409a1d0 2022-06-29T16:53:05.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.300+0000 7f7de3fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de4153020 con 0x7f7de4142a30 2022-06-29T16:53:05.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.300+0000 7f7de3fff700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7de40ac6a0 con 0x7f7de405f4a0 2022-06-29T16:53:05.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.300+0000 7f7dda7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7de409a1d0 0x7f7de40af2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:05.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.300+0000 7f7dda7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7de409a1d0 0x7f7de40af2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55694/0 (socket says 172.21.15.47:55694) 2022-06-29T16:53:05.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.300+0000 7f7dda7fc700 1 -- 172.21.15.47:0/210358210 learned_addr learned my addr 172.21.15.47:0/210358210 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T16:53:05.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.300+0000 7f7de37fe700 1 --2- 172.21.15.47:0/210358210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7de4142a30 0x7f7de40943d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:05.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.300+0000 7f7de2ffd700 1 --2- 172.21.15.47:0/210358210 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7de405f4a0 0x7f7de4095610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:05.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.300+0000 7f7dda7fc700 1 -- 172.21.15.47:0/210358210 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7de405f4a0 msgr2=0x7f7de4095610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:05.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.300+0000 7f7dda7fc700 1 --2- 172.21.15.47:0/210358210 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f7de405f4a0 0x7f7de4095610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:05.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.300+0000 7f7dda7fc700 1 -- 172.21.15.47:0/210358210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7de4142a30 msgr2=0x7f7de40943d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:05.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.300+0000 7f7dda7fc700 1 --2- 172.21.15.47:0/210358210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7de4142a30 0x7f7de40943d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:05.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.300+0000 7f7dda7fc700 1 -- 172.21.15.47:0/210358210 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ddc009530 con 0x7f7de409a1d0 2022-06-29T16:53:05.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.300+0000 7f7dda7fc700 1 --2- 172.21.15.47:0/210358210 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7de409a1d0 0x7f7de40af2a0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 rx=0x7f7dcc00c000 tx=0x7f7dcc006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:05.326 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.300+0000 7f7de0ff9700 1 -- 172.21.15.47:0/210358210 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7dcc010780 con 0x7f7de409a1d0 2022-06-29T16:53:05.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.300+0000 7f7de0ff9700 1 -- 172.21.15.47:0/210358210 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dcc0108e0 con 0x7f7de409a1d0 2022-06-29T16:53:05.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.300+0000 7f7de3fff700 1 -- 172.21.15.47:0/210358210 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7de40acb40 con 0x7f7de409a1d0 2022-06-29T16:53:05.327 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.300+0000 7f7de0ff9700 1 -- 172.21.15.47:0/210358210 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7dcc010c40 con 0x7f7de409a1d0 2022-06-29T16:53:05.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.300+0000 7f7de3fff700 1 -- 172.21.15.47:0/210358210 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7de40a9a00 con 0x7f7de409a1d0 2022-06-29T16:53:05.328 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.300+0000 7f7de3fff700 1 -- 172.21.15.47:0/210358210 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7de4153220 con 0x7f7de409a1d0 2022-06-29T16:53:05.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.304+0000 7f7de0ff9700 1 -- 172.21.15.47:0/210358210 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7dcc026430 con 0x7f7de409a1d0 2022-06-29T16:53:05.330 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.304+0000 7f7de0ff9700 1 --2- 172.21.15.47:0/210358210 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7dc8053950 0x7f7dc8055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:05.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.304+0000 7f7de0ff9700 1 -- 172.21.15.47:0/210358210 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f7dcc06cc00 con 0x7f7de409a1d0 2022-06-29T16:53:05.331 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.304+0000 7f7de2ffd700 1 --2- 172.21.15.47:0/210358210 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7dc8053950 0x7f7dc8055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:05.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.304+0000 7f7de0ff9700 1 --2- 172.21.15.47:0/210358210 >> [v2:172.21.15.64:6816/13396,v1:172.21.15.64:6817/13396] conn(0x7f7dc805a680 0x7f7dc805cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:05.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.304+0000 7f7de0ff9700 1 -- 172.21.15.47:0/210358210 --> [v2:172.21.15.64:6816/13396,v1:172.21.15.64:6817/13396] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7dc805d1a0 con 0x7f7dc805a680 2022-06-29T16:53:05.332 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.304+0000 7f7de0ff9700 1 -- 172.21.15.47:0/210358210 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=165) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7dcc06cfe0 con 0x7f7de409a1d0 2022-06-29T16:53:05.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.304+0000 7f7de37fe700 1 --2- 172.21.15.47:0/210358210 >> [v2:172.21.15.64:6816/13396,v1:172.21.15.64:6817/13396] conn(0x7f7dc805a680 0x7f7dc805cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:05.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.304+0000 7f7de2ffd700 1 --2- 172.21.15.47:0/210358210 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7dc8053950 0x7f7dc8055e10 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 rx=0x7f7dd400c000 tx=0x7f7dd4008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:05.333 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.304+0000 7f7de37fe700 1 --2- 172.21.15.47:0/210358210 >> [v2:172.21.15.64:6816/13396,v1:172.21.15.64:6817/13396] conn(0x7f7dc805a680 0x7f7dc805cb20 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:05.334 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.308+0000 7f7de0ff9700 1 -- 172.21.15.47:0/210358210 <== osd.4 v2:172.21.15.64:6816/13396 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f7dc805d1a0 con 0x7f7dc805a680 2022-06-29T16:53:05.335 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.324+0000 7f7de3fff700 1 -- 172.21.15.47:0/210358210 --> [v2:172.21.15.64:6816/13396,v1:172.21.15.64:6817/13396] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f7de4094d80 con 0x7f7dc805a680 2022-06-29T16:53:05.335 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.324+0000 7f7de0ff9700 1 -- 172.21.15.47:0/210358210 <== osd.4 v2:172.21.15.64:6816/13396 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f7de4094d80 con 0x7f7dc805a680 2022-06-29T16:53:05.336 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2022-06-29T16:53:05.336 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2022-06-29T16:53:05.337 INFO:teuthology.orchestra.run.smithi047.stdout:} 2022-06-29T16:53:05.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.328+0000 7f7de3fff700 1 -- 172.21.15.47:0/210358210 >> [v2:172.21.15.64:6816/13396,v1:172.21.15.64:6817/13396] conn(0x7f7dc805a680 msgr2=0x7f7dc805cb20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:05.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.328+0000 7f7de3fff700 1 --2- 172.21.15.47:0/210358210 >> [v2:172.21.15.64:6816/13396,v1:172.21.15.64:6817/13396] conn(0x7f7dc805a680 0x7f7dc805cb20 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:05.338 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.328+0000 7f7de3fff700 1 -- 172.21.15.47:0/210358210 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7dc8053950 msgr2=0x7f7dc8055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:05.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.328+0000 7f7de3fff700 1 --2- 172.21.15.47:0/210358210 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f7dc8053950 0x7f7dc8055e10 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 rx=0x7f7dd400c000 tx=0x7f7dd4008040).stop 2022-06-29T16:53:05.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.328+0000 7f7de3fff700 1 -- 172.21.15.47:0/210358210 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7de409a1d0 msgr2=0x7f7de40af2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:05.339 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.328+0000 7f7de3fff700 1 --2- 172.21.15.47:0/210358210 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7de409a1d0 0x7f7de40af2a0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 rx=0x7f7dcc00c000 tx=0x7f7dcc006df0).stop 2022-06-29T16:53:05.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.328+0000 7f7de37fe700 1 -- 172.21.15.47:0/210358210 reap_dead start 2022-06-29T16:53:05.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.328+0000 7f7de3fff700 1 -- 172.21.15.47:0/210358210 shutdown_connections 2022-06-29T16:53:05.340 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.328+0000 7f7de3fff700 1 -- 172.21.15.47:0/210358210 >> 172.21.15.47:0/210358210 conn(0x7f7de4062df0 msgr2=0x7f7de40631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T16:53:05.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.328+0000 7f7de3fff700 1 -- 172.21.15.47:0/210358210 shutdown_connections 2022-06-29T16:53:05.341 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.328+0000 7f7de3fff700 1 -- 172.21.15.47:0/210358210 wait complete. 2022-06-29T16:53:05.344 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2022-06-29T16:53:05.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.448+0000 7fe62e23b700 1 Processor -- start 2022-06-29T16:53:05.454 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.448+0000 7fe62e23b700 1 -- start start 2022-06-29T16:53:05.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.448+0000 7fe62e23b700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe6280b5c60 0x7fe628144980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:05.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.448+0000 7fe62e23b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe628060eb0 0x7fe628144e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:05.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.448+0000 7fe62e23b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe6280a72f0 0x7fe6280b1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:05.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.448+0000 7fe62e23b700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe628145360 con 0x7fe628060eb0 2022-06-29T16:53:05.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.448+0000 7fe62e23b700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6280a9120 con 0x7fe6280a72f0 2022-06-29T16:53:05.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.448+0000 7fe62e23b700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6280b0b80 con 0x7fe6280b5c60 2022-06-29T16:53:05.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.448+0000 7fe62ca38700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe628060eb0 0x7fe628144e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:05.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.448+0000 7fe62ca38700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe628060eb0 0x7fe628144e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55702/0 (socket says 172.21.15.47:55702) 2022-06-29T16:53:05.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.448+0000 7fe62ca38700 1 -- 172.21.15.47:0/1803427857 learned_addr learned my addr 172.21.15.47:0/1803427857 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T16:53:05.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.448+0000 7fe62d239700 1 --2- 172.21.15.47:0/1803427857 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe6280b5c60 0x7fe628144980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:05.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.448+0000 7fe62da3a700 1 --2- 172.21.15.47:0/1803427857 >> v2:172.21.15.64:3300/0 conn(0x7fe6280a72f0 0x7fe6280b1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:05.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.448+0000 7fe62d239700 1 -- 172.21.15.47:0/1803427857 >> v2:172.21.15.64:3300/0 conn(0x7fe6280a72f0 msgr2=0x7fe6280b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T16:53:05.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.448+0000 7fe62d239700 1 --2- 172.21.15.47:0/1803427857 >> v2:172.21.15.64:3300/0 conn(0x7fe6280a72f0 0x7fe6280b1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:05.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.448+0000 7fe62d239700 1 -- 172.21.15.47:0/1803427857 >> v2:172.21.15.47:3300/0 conn(0x7fe628060eb0 msgr2=0x7fe628144e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T16:53:05.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.448+0000 7fe62d239700 1 --2- 172.21.15.47:0/1803427857 >> v2:172.21.15.47:3300/0 conn(0x7fe628060eb0 0x7fe628144e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:05.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.448+0000 7fe62d239700 1 -- 172.21.15.47:0/1803427857 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6280a89e0 con 0x7fe6280b5c60 2022-06-29T16:53:05.462 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.448+0000 7fe62ca38700 1 --2- 172.21.15.47:0/1803427857 >> v2:172.21.15.47:3300/0 conn(0x7fe628060eb0 0x7fe628144e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T16:53:05.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.448+0000 7fe62d239700 1 --2- 172.21.15.47:0/1803427857 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe6280b5c60 0x7fe628144980 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7fe61c002f70 tx=0x7fe61c006d90).ready entity=mon.2 client_cookie=424b2d68a1f8e44e server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:05.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.448+0000 7fe627fff700 1 -- 172.21.15.47:0/1803427857 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe61c00e470 con 0x7fe6280b5c60 2022-06-29T16:53:05.463 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.448+0000 7fe627fff700 1 -- 172.21.15.47:0/1803427857 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe61c000e90 con 0x7fe6280b5c60 2022-06-29T16:53:05.464 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.448+0000 7fe627fff700 1 -- 172.21.15.47:0/1803427857 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe61c00e7c0 con 0x7fe6280b5c60 2022-06-29T16:53:05.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.452+0000 7fe62e23b700 1 -- 172.21.15.47:0/1803427857 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe6280b5c60 msgr2=0x7fe628144980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:05.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.452+0000 7fe62e23b700 1 --2- 172.21.15.47:0/1803427857 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe6280b5c60 0x7fe628144980 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7fe61c002f70 tx=0x7fe61c006d90).stop 2022-06-29T16:53:05.468 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.452+0000 7fe62e23b700 1 -- 172.21.15.47:0/1803427857 shutdown_connections 2022-06-29T16:53:05.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.452+0000 7fe62e23b700 1 --2- 172.21.15.47:0/1803427857 >> v2:172.21.15.64:3300/0 conn(0x7fe6280a72f0 0x7fe6280b1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:05.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.452+0000 7fe62e23b700 1 --2- 172.21.15.47:0/1803427857 >> v2:172.21.15.47:3300/0 conn(0x7fe628060eb0 0x7fe628144e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:05.469 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.452+0000 7fe62e23b700 1 --2- 172.21.15.47:0/1803427857 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe6280b5c60 0x7fe628144980 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:05.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.452+0000 7fe62e23b700 1 -- 172.21.15.47:0/1803427857 >> 172.21.15.47:0/1803427857 conn(0x7fe628062560 msgr2=0x7fe628062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T16:53:05.470 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.452+0000 7fe62e23b700 1 -- 172.21.15.47:0/1803427857 shutdown_connections 2022-06-29T16:53:05.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.452+0000 7fe62e23b700 1 -- 172.21.15.47:0/1803427857 wait complete. 2022-06-29T16:53:05.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.452+0000 7fe62e23b700 1 Processor -- start 2022-06-29T16:53:05.471 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.452+0000 7fe62e23b700 1 -- start start 2022-06-29T16:53:05.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.456+0000 7fe62e23b700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe628060eb0 0x7fe6281454a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:05.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.456+0000 7fe62e23b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6280a72f0 0x7fe6281564f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:05.472 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.456+0000 7fe62e23b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6280b5c60 0x7fe6280b48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:05.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.456+0000 7fe62e23b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe628143700 con 0x7fe6280a72f0 2022-06-29T16:53:05.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.456+0000 7fe62e23b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe628145ad0 con 0x7fe6280b5c60 2022-06-29T16:53:05.473 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.456+0000 7fe62e23b700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe628145d70 con 0x7fe628060eb0 2022-06-29T16:53:05.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.456+0000 7fe62ca38700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6280a72f0 0x7fe6281564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:05.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.456+0000 7fe62ca38700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6280a72f0 0x7fe6281564f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55708/0 (socket says 172.21.15.47:55708) 2022-06-29T16:53:05.474 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.456+0000 7fe62ca38700 1 -- 172.21.15.47:0/2932407825 learned_addr learned my addr 172.21.15.47:0/2932407825 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T16:53:05.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.456+0000 7fe62d239700 1 --2- 172.21.15.47:0/2932407825 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe628060eb0 0x7fe6281454a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:05.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.456+0000 7fe62da3a700 1 --2- 172.21.15.47:0/2932407825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6280b5c60 0x7fe6280b48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:05.475 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.456+0000 7fe62ca38700 1 -- 172.21.15.47:0/2932407825 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe628060eb0 msgr2=0x7fe6281454a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:05.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.456+0000 7fe62ca38700 1 --2- 172.21.15.47:0/2932407825 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe628060eb0 0x7fe6281454a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:05.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.456+0000 7fe62ca38700 1 -- 172.21.15.47:0/2932407825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6280b5c60 msgr2=0x7fe6280b48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:05.476 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.456+0000 7fe62ca38700 1 --2- 172.21.15.47:0/2932407825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6280b5c60 0x7fe6280b48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:05.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.456+0000 7fe62ca38700 1 -- 172.21.15.47:0/2932407825 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe61800b1e0 con 0x7fe6280a72f0 2022-06-29T16:53:05.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.456+0000 7fe62ca38700 1 --2- 172.21.15.47:0/2932407825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6280a72f0 0x7fe6281564f0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 rx=0x7fe61800c000 tx=0x7fe618006eb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:05.477 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.456+0000 7fe6267fc700 1 -- 172.21.15.47:0/2932407825 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe618012600 con 0x7fe6280a72f0 2022-06-29T16:53:05.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.456+0000 7fe6267fc700 1 -- 172.21.15.47:0/2932407825 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe618012760 con 0x7fe6280a72f0 2022-06-29T16:53:05.478 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.456+0000 7fe62e23b700 1 -- 172.21.15.47:0/2932407825 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe61c00b120 con 0x7fe6280a72f0 2022-06-29T16:53:05.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.456+0000 7fe6267fc700 1 -- 172.21.15.47:0/2932407825 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe618012ac0 con 0x7fe6280a72f0 2022-06-29T16:53:05.479 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.456+0000 7fe62e23b700 1 -- 172.21.15.47:0/2932407825 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe628159e00 con 0x7fe6280a72f0 2022-06-29T16:53:05.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.456+0000 7fe62e23b700 1 -- 172.21.15.47:0/2932407825 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6280af7b0 con 0x7fe6280a72f0 2022-06-29T16:53:05.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.460+0000 7fe6267fc700 1 -- 172.21.15.47:0/2932407825 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe618022070 con 0x7fe6280a72f0 2022-06-29T16:53:05.480 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.460+0000 7fe6267fc700 1 --2- 172.21.15.47:0/2932407825 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe6080539e0 0x7fe608055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:05.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.460+0000 7fe6267fc700 1 -- 172.21.15.47:0/2932407825 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fe61806c220 con 0x7fe6280a72f0 2022-06-29T16:53:05.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.460+0000 7fe62d239700 1 --2- 172.21.15.47:0/2932407825 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe6080539e0 0x7fe608055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:05.481 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.460+0000 7fe62d239700 1 --2- 172.21.15.47:0/2932407825 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe6080539e0 0x7fe608055ea0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 rx=0x7fe61c01a000 tx=0x7fe61c014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:05.482 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.468+0000 7fe6267fc700 1 -- 172.21.15.47:0/2932407825 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe6280af7b0 con 0x7fe6280a72f0 2022-06-29T16:53:05.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.588+0000 7fe62e23b700 1 -- 172.21.15.47:0/2932407825 --> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fe6280af7b0 con 0x7fe6080539e0 2022-06-29T16:53:05.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.588+0000 7fe6267fc700 1 -- 172.21.15.47:0/2932407825 <== mgr.4102 v2:172.21.15.47:6800/12928 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe6280af7b0 con 0x7fe6080539e0 2022-06-29T16:53:05.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.592+0000 7fe62e23b700 1 -- 172.21.15.47:0/2932407825 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe6080539e0 msgr2=0x7fe608055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:05.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.592+0000 7fe62e23b700 1 --2- 172.21.15.47:0/2932407825 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe6080539e0 0x7fe608055ea0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 rx=0x7fe61c01a000 tx=0x7fe61c014000).stop 2022-06-29T16:53:05.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.592+0000 7fe62e23b700 1 -- 172.21.15.47:0/2932407825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6280a72f0 msgr2=0x7fe6281564f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:05.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.592+0000 7fe62e23b700 1 --2- 172.21.15.47:0/2932407825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6280a72f0 0x7fe6281564f0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 rx=0x7fe61800c000 tx=0x7fe618006eb0).stop 2022-06-29T16:53:05.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.592+0000 7fe62e23b700 1 -- 172.21.15.47:0/2932407825 shutdown_connections 2022-06-29T16:53:05.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.592+0000 7fe62e23b700 1 --2- 172.21.15.47:0/2932407825 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fe6080539e0 0x7fe608055ea0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:05.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.592+0000 7fe62e23b700 1 --2- 172.21.15.47:0/2932407825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6280b5c60 0x7fe6280b48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:05.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.592+0000 7fe62e23b700 1 --2- 172.21.15.47:0/2932407825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6280a72f0 0x7fe6281564f0 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:05.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.592+0000 7fe62e23b700 1 --2- 172.21.15.47:0/2932407825 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fe628060eb0 0x7fe6281454a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:05.601 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.592+0000 7fe62e23b700 1 -- 172.21.15.47:0/2932407825 >> 172.21.15.47:0/2932407825 conn(0x7fe628062560 msgr2=0x7fe6281419c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T16:53:05.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.592+0000 7fe62e23b700 1 -- 172.21.15.47:0/2932407825 shutdown_connections 2022-06-29T16:53:05.602 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.592+0000 7fe62e23b700 1 -- 172.21.15.47:0/2932407825 wait complete. 2022-06-29T16:53:05.603 INFO:teuthology.orchestra.run.smithi047.stderr:instructed osd(s) 4 to deep-scrub 2022-06-29T16:53:05.609 INFO:tasks.ceph:Scrubbing osd.5 2022-06-29T16:53:05.610 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2022-06-29T16:53:05.724 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.716+0000 7fc4c4533700 1 Processor -- start 2022-06-29T16:53:05.725 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.716+0000 7fc4c4533700 1 -- start start 2022-06-29T16:53:05.731 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.720+0000 7fc4c4533700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc4bc060ee0 0x7fc4bc0612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:05.731 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.720+0000 7fc4c4533700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc4bc1471a0 0x7fc4bc066f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:05.732 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.720+0000 7fc4c4533700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4bc067610 con 0x7fc4bc0b4300 2022-06-29T16:53:05.732 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.720+0000 7fc4c4533700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4bc1475d0 con 0x7fc4bc1471a0 2022-06-29T16:53:05.732 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.720+0000 7fc4c4533700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4bc1467c0 con 0x7fc4bc060ee0 2022-06-29T16:53:05.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.720+0000 7fc4c1ace700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc4bc0b4300 0x7fc4bc1430e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41270/0 (socket says 172.21.15.47:41270) 2022-06-29T16:53:05.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.720+0000 7fc4c1ace700 1 -- 172.21.15.47:0/1148434044 learned_addr learned my addr 172.21.15.47:0/1148434044 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T16:53:05.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.720+0000 7fc4c2ad0700 1 --2- 172.21.15.47:0/1148434044 >> v2:172.21.15.47:3300/0 conn(0x7fc4bc1471a0 0x7fc4bc066f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:05.734 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.720+0000 7fc4c22cf700 1 --2- 172.21.15.47:0/1148434044 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc4bc060ee0 0x7fc4bc0612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:05.734 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.720+0000 7fc4c12cd700 1 -- 172.21.15.47:0/1148434044 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4174226156 0 0) 0x7fc4bc067610 con 0x7fc4bc0b4300 2022-06-29T16:53:05.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.720+0000 7fc4c12cd700 1 -- 172.21.15.47:0/1148434044 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4a4003260 con 0x7fc4bc0b4300 2022-06-29T16:53:05.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.720+0000 7fc4c22cf700 1 -- 172.21.15.47:0/1148434044 >> v2:172.21.15.47:3300/0 conn(0x7fc4bc1471a0 msgr2=0x7fc4bc066f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T16:53:05.735 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.720+0000 7fc4c22cf700 1 --2- 172.21.15.47:0/1148434044 >> v2:172.21.15.47:3300/0 conn(0x7fc4bc1471a0 0x7fc4bc066f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:05.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.720+0000 7fc4c22cf700 1 -- 172.21.15.47:0/1148434044 >> v1:172.21.15.47:6789/0 conn(0x7fc4bc0b4300 legacy=0x7fc4bc1430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:05.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.720+0000 7fc4c22cf700 1 -- 172.21.15.47:0/1148434044 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4bc146a60 con 0x7fc4bc060ee0 2022-06-29T16:53:05.736 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.720+0000 7fc4c22cf700 1 --2- 172.21.15.47:0/1148434044 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc4bc060ee0 0x7fc4bc0612c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7fc4b0002f70 tx=0x7fc4b0006df0).ready entity=mon.2 client_cookie=2aca2d1c53870d48 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:05.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.720+0000 7fc4c2ad0700 1 --2- 172.21.15.47:0/1148434044 >> v2:172.21.15.47:3300/0 conn(0x7fc4bc1471a0 0x7fc4bc066f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T16:53:05.737 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.720+0000 7fc4c12cd700 1 -- 172.21.15.47:0/1148434044 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc4b000d4a0 con 0x7fc4bc060ee0 2022-06-29T16:53:05.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.720+0000 7fc4c12cd700 1 -- 172.21.15.47:0/1148434044 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b0000e90 con 0x7fc4bc060ee0 2022-06-29T16:53:05.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.720+0000 7fc4c12cd700 1 -- 172.21.15.47:0/1148434044 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc4b000d830 con 0x7fc4bc060ee0 2022-06-29T16:53:05.738 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.720+0000 7fc4c4533700 1 -- 172.21.15.47:0/1148434044 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc4bc060ee0 msgr2=0x7fc4bc0612c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:05.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.720+0000 7fc4c4533700 1 --2- 172.21.15.47:0/1148434044 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc4bc060ee0 0x7fc4bc0612c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7fc4b0002f70 tx=0x7fc4b0006df0).stop 2022-06-29T16:53:05.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.720+0000 7fc4c4533700 1 -- 172.21.15.47:0/1148434044 shutdown_connections 2022-06-29T16:53:05.739 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.720+0000 7fc4c4533700 1 --2- 172.21.15.47:0/1148434044 >> v2:172.21.15.47:3300/0 conn(0x7fc4bc1471a0 0x7fc4bc066f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:05.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.720+0000 7fc4c4533700 1 --2- 172.21.15.47:0/1148434044 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc4bc060ee0 0x7fc4bc0612c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:05.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.720+0000 7fc4c4533700 1 -- 172.21.15.47:0/1148434044 >> 172.21.15.47:0/1148434044 conn(0x7fc4bc05c510 msgr2=0x7fc4bc05f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T16:53:05.740 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.720+0000 7fc4c4533700 1 -- 172.21.15.47:0/1148434044 shutdown_connections 2022-06-29T16:53:05.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.720+0000 7fc4c4533700 1 -- 172.21.15.47:0/1148434044 wait complete. 2022-06-29T16:53:05.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.720+0000 7fc4c4533700 1 Processor -- start 2022-06-29T16:53:05.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.720+0000 7fc4c4533700 1 -- start start 2022-06-29T16:53:05.741 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.724+0000 7fc4c4533700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4bc060ee0 0x7fc4bc13ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:05.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.724+0000 7fc4c4533700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4bc0b4300 0x7fc4bc1446c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:05.742 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.724+0000 7fc4c4533700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc4bc1471a0 0x7fc4bc137100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:05.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.724+0000 7fc4c4533700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4bc144bb0 con 0x7fc4bc060ee0 2022-06-29T16:53:05.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.724+0000 7fc4c4533700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4bc1384d0 con 0x7fc4bc0b4300 2022-06-29T16:53:05.743 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.724+0000 7fc4c4533700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4bc138730 con 0x7fc4bc1471a0 2022-06-29T16:53:05.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.724+0000 7fc4c22cf700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4bc060ee0 0x7fc4bc13ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:05.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.724+0000 7fc4c22cf700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4bc060ee0 0x7fc4bc13ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55720/0 (socket says 172.21.15.47:55720) 2022-06-29T16:53:05.744 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.724+0000 7fc4c22cf700 1 -- 172.21.15.47:0/1174947813 learned_addr learned my addr 172.21.15.47:0/1174947813 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T16:53:05.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.724+0000 7fc4c2ad0700 1 --2- 172.21.15.47:0/1174947813 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc4bc1471a0 0x7fc4bc137100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:05.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.724+0000 7fc4c1ace700 1 --2- 172.21.15.47:0/1174947813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4bc0b4300 0x7fc4bc1446c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:05.745 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.724+0000 7fc4c22cf700 1 -- 172.21.15.47:0/1174947813 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc4bc1471a0 msgr2=0x7fc4bc137100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:05.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.724+0000 7fc4c22cf700 1 --2- 172.21.15.47:0/1174947813 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fc4bc1471a0 0x7fc4bc137100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:05.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.724+0000 7fc4c22cf700 1 -- 172.21.15.47:0/1174947813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4bc0b4300 msgr2=0x7fc4bc1446c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:05.746 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.724+0000 7fc4c22cf700 1 --2- 172.21.15.47:0/1174947813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4bc0b4300 0x7fc4bc1446c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:05.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.724+0000 7fc4c22cf700 1 -- 172.21.15.47:0/1174947813 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b4009470 con 0x7fc4bc060ee0 2022-06-29T16:53:05.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.724+0000 7fc4c22cf700 1 --2- 172.21.15.47:0/1174947813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4bc060ee0 0x7fc4bc13ba30 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 rx=0x7fc4b0002f70 tx=0x7fc4b0014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:05.747 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.724+0000 7fc4af7fe700 1 -- 172.21.15.47:0/1174947813 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc4b0016070 con 0x7fc4bc060ee0 2022-06-29T16:53:05.748 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.724+0000 7fc4af7fe700 1 -- 172.21.15.47:0/1174947813 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b00138d0 con 0x7fc4bc060ee0 2022-06-29T16:53:05.748 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.724+0000 7fc4c4533700 1 -- 172.21.15.47:0/1174947813 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4b000b180 con 0x7fc4bc060ee0 2022-06-29T16:53:05.748 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.724+0000 7fc4af7fe700 1 -- 172.21.15.47:0/1174947813 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc4b0013c30 con 0x7fc4bc060ee0 2022-06-29T16:53:05.749 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.724+0000 7fc4c4533700 1 -- 172.21.15.47:0/1174947813 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4bc144220 con 0x7fc4bc060ee0 2022-06-29T16:53:05.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.724+0000 7fc4c4533700 1 -- 172.21.15.47:0/1174947813 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc4bc144420 con 0x7fc4bc060ee0 2022-06-29T16:53:05.751 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.724+0000 7fc4af7fe700 1 -- 172.21.15.47:0/1174947813 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc4b0024030 con 0x7fc4bc060ee0 2022-06-29T16:53:05.752 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.728+0000 7fc4af7fe700 1 --2- 172.21.15.47:0/1174947813 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc49c053a70 0x7fc49c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:05.752 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.728+0000 7fc4af7fe700 1 -- 172.21.15.47:0/1174947813 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fc4b006cb60 con 0x7fc4bc060ee0 2022-06-29T16:53:05.752 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.728+0000 7fc4c1ace700 1 --2- 172.21.15.47:0/1174947813 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc49c053a70 0x7fc49c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:05.752 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.728+0000 7fc4af7fe700 1 --2- 172.21.15.47:0/1174947813 >> [v2:172.21.15.64:6800/13393,v1:172.21.15.64:6801/13393] conn(0x7fc49c05a700 0x7fc49c05cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:05.753 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.728+0000 7fc4af7fe700 1 -- 172.21.15.47:0/1174947813 --> [v2:172.21.15.64:6800/13393,v1:172.21.15.64:6801/13393] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc49c05d180 con 0x7fc49c05a700 2022-06-29T16:53:05.753 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.728+0000 7fc4af7fe700 1 -- 172.21.15.47:0/1174947813 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=165) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4b0071240 con 0x7fc4bc060ee0 2022-06-29T16:53:05.753 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.728+0000 7fc4c1ace700 1 --2- 172.21.15.47:0/1174947813 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc49c053a70 0x7fc49c055f30 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 rx=0x7fc4b800d000 tx=0x7fc4b8009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:05.754 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.728+0000 7fc4c2ad0700 1 --2- 172.21.15.47:0/1174947813 >> [v2:172.21.15.64:6800/13393,v1:172.21.15.64:6801/13393] conn(0x7fc49c05a700 0x7fc49c05cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:05.754 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.728+0000 7fc4c2ad0700 1 --2- 172.21.15.47:0/1174947813 >> [v2:172.21.15.64:6800/13393,v1:172.21.15.64:6801/13393] conn(0x7fc49c05a700 0x7fc49c05cba0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:05.755 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.728+0000 7fc4af7fe700 1 -- 172.21.15.47:0/1174947813 <== osd.5 v2:172.21.15.64:6800/13393 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7fc49c05d180 con 0x7fc49c05a700 2022-06-29T16:53:05.756 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.748+0000 7fc4c4533700 1 -- 172.21.15.47:0/1174947813 --> [v2:172.21.15.64:6800/13393,v1:172.21.15.64:6801/13393] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc4bc0b4e00 con 0x7fc49c05a700 2022-06-29T16:53:05.756 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.748+0000 7fc4af7fe700 1 -- 172.21.15.47:0/1174947813 <== osd.5 v2:172.21.15.64:6800/13393 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fc4bc0b4e00 con 0x7fc49c05a700 2022-06-29T16:53:05.757 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2022-06-29T16:53:05.757 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2022-06-29T16:53:05.757 INFO:teuthology.orchestra.run.smithi047.stdout:} 2022-06-29T16:53:05.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.748+0000 7fc4c4533700 1 -- 172.21.15.47:0/1174947813 >> [v2:172.21.15.64:6800/13393,v1:172.21.15.64:6801/13393] conn(0x7fc49c05a700 msgr2=0x7fc49c05cba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:05.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.748+0000 7fc4c4533700 1 --2- 172.21.15.47:0/1174947813 >> [v2:172.21.15.64:6800/13393,v1:172.21.15.64:6801/13393] conn(0x7fc49c05a700 0x7fc49c05cba0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:05.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.748+0000 7fc4c4533700 1 -- 172.21.15.47:0/1174947813 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc49c053a70 msgr2=0x7fc49c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:05.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.748+0000 7fc4c4533700 1 --2- 172.21.15.47:0/1174947813 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fc49c053a70 0x7fc49c055f30 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 rx=0x7fc4b800d000 tx=0x7fc4b8009040).stop 2022-06-29T16:53:05.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.748+0000 7fc4c4533700 1 -- 172.21.15.47:0/1174947813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4bc060ee0 msgr2=0x7fc4bc13ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:05.761 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.748+0000 7fc4c4533700 1 --2- 172.21.15.47:0/1174947813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4bc060ee0 0x7fc4bc13ba30 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 rx=0x7fc4b0002f70 tx=0x7fc4b0014000).stop 2022-06-29T16:53:05.761 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.748+0000 7fc4c2ad0700 1 -- 172.21.15.47:0/1174947813 reap_dead start 2022-06-29T16:53:05.761 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.748+0000 7fc4c4533700 1 -- 172.21.15.47:0/1174947813 shutdown_connections 2022-06-29T16:53:05.762 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.748+0000 7fc4c4533700 1 -- 172.21.15.47:0/1174947813 >> 172.21.15.47:0/1174947813 conn(0x7fc4bc05c510 msgr2=0x7fc4bc0b7850 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T16:53:05.762 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.748+0000 7fc4c4533700 1 -- 172.21.15.47:0/1174947813 shutdown_connections 2022-06-29T16:53:05.762 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.748+0000 7fc4c4533700 1 -- 172.21.15.47:0/1174947813 wait complete. 2022-06-29T16:53:05.767 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2022-06-29T16:53:05.873 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.868+0000 7fa0d284b700 1 Processor -- start 2022-06-29T16:53:05.874 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.868+0000 7fa0d284b700 1 -- start start 2022-06-29T16:53:05.875 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.868+0000 7fa0d284b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa0cc0b4420 0x7fa0cc1401b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:05.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.868+0000 7fa0d284b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa0cc0b2490 0x7fa0cc1406a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:05.876 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.868+0000 7fa0d284b700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa0cc064820 0x7fa0cc064cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:05.877 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.868+0000 7fa0d284b700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0cc0b2bd0 con 0x7fa0cc0b4420 2022-06-29T16:53:05.877 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.868+0000 7fa0d284b700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0cc140b90 con 0x7fa0cc0b2490 2022-06-29T16:53:05.877 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.868+0000 7fa0d284b700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0cc0b4800 con 0x7fa0cc064820 2022-06-29T16:53:05.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.868+0000 7fa0cbfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa0cc0b4420 0x7fa0cc1401b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:05.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.868+0000 7fa0cbfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa0cc0b4420 0x7fa0cc1401b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55730/0 (socket says 172.21.15.47:55730) 2022-06-29T16:53:05.878 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.868+0000 7fa0cbfff700 1 -- 172.21.15.47:0/542388194 learned_addr learned my addr 172.21.15.47:0/542388194 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T16:53:05.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.868+0000 7fa0cb7fe700 1 --2- 172.21.15.47:0/542388194 >> v2:172.21.15.64:3300/0 conn(0x7fa0cc0b2490 0x7fa0cc1406a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:05.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.868+0000 7fa0d0de8700 1 --2- 172.21.15.47:0/542388194 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa0cc064820 0x7fa0cc064cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:05.879 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.868+0000 7fa0cbfff700 1 -- 172.21.15.47:0/542388194 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa0cc064820 msgr2=0x7fa0cc064cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T16:53:05.880 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.868+0000 7fa0cbfff700 1 --2- 172.21.15.47:0/542388194 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa0cc064820 0x7fa0cc064cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:05.880 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.868+0000 7fa0cbfff700 1 -- 172.21.15.47:0/542388194 >> v2:172.21.15.64:3300/0 conn(0x7fa0cc0b2490 msgr2=0x7fa0cc1406a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T16:53:05.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.868+0000 7fa0cbfff700 1 --2- 172.21.15.47:0/542388194 >> v2:172.21.15.64:3300/0 conn(0x7fa0cc0b2490 0x7fa0cc1406a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:05.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.868+0000 7fa0cbfff700 1 -- 172.21.15.47:0/542388194 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0cc0b2930 con 0x7fa0cc0b4420 2022-06-29T16:53:05.881 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.868+0000 7fa0cbfff700 1 --2- 172.21.15.47:0/542388194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0cc0b4420 0x7fa0cc1401b0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 rx=0x7fa0cc140cd0 tx=0x7fa0bc006eb0).ready entity=mon.0 client_cookie=4a1a1cf1485d0345 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:05.885 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.868+0000 7fa0caffd700 1 -- 172.21.15.47:0/542388194 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0bc00e610 con 0x7fa0cc0b4420 2022-06-29T16:53:05.886 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.868+0000 7fa0caffd700 1 -- 172.21.15.47:0/542388194 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0bc000e90 con 0x7fa0cc0b4420 2022-06-29T16:53:05.886 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.868+0000 7fa0caffd700 1 -- 172.21.15.47:0/542388194 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0bc00e9a0 con 0x7fa0cc0b4420 2022-06-29T16:53:05.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.872+0000 7fa0d284b700 1 -- 172.21.15.47:0/542388194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0cc0b4420 msgr2=0x7fa0cc1401b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:05.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.872+0000 7fa0d284b700 1 --2- 172.21.15.47:0/542388194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0cc0b4420 0x7fa0cc1401b0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 rx=0x7fa0cc140cd0 tx=0x7fa0bc006eb0).stop 2022-06-29T16:53:05.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.872+0000 7fa0d284b700 1 -- 172.21.15.47:0/542388194 shutdown_connections 2022-06-29T16:53:05.888 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.872+0000 7fa0d284b700 1 --2- 172.21.15.47:0/542388194 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa0cc064820 0x7fa0cc064cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:05.888 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.872+0000 7fa0d284b700 1 --2- 172.21.15.47:0/542388194 >> v2:172.21.15.64:3300/0 conn(0x7fa0cc0b2490 0x7fa0cc1406a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:05.888 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.872+0000 7fa0d284b700 1 --2- 172.21.15.47:0/542388194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0cc0b4420 0x7fa0cc1401b0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:05.889 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.872+0000 7fa0d284b700 1 -- 172.21.15.47:0/542388194 >> 172.21.15.47:0/542388194 conn(0x7fa0cc05c510 msgr2=0x7fa0cc05c920 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T16:53:05.889 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.872+0000 7fa0d284b700 1 -- 172.21.15.47:0/542388194 shutdown_connections 2022-06-29T16:53:05.889 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.872+0000 7fa0d284b700 1 -- 172.21.15.47:0/542388194 wait complete. 2022-06-29T16:53:05.890 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.872+0000 7fa0d284b700 1 Processor -- start 2022-06-29T16:53:05.890 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.872+0000 7fa0d284b700 1 -- start start 2022-06-29T16:53:05.891 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.872+0000 7fa0d284b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0cc064820 0x7fa0cc152d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:05.891 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.872+0000 7fa0d284b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0cc0b2490 0x7fa0cc05f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:05.891 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.872+0000 7fa0d284b700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa0cc0b4420 0x7fa0cc1558e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:05.892 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.872+0000 7fa0d284b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0cc149070 con 0x7fa0cc0b2490 2022-06-29T16:53:05.892 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.872+0000 7fa0d284b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0cc0b1da0 con 0x7fa0cc064820 2022-06-29T16:53:05.892 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.872+0000 7fa0d284b700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0cc148500 con 0x7fa0cc0b4420 2022-06-29T16:53:05.893 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.876+0000 7fa0cb7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0cc0b2490 0x7fa0cc05f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:05.893 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.876+0000 7fa0cb7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0cc0b2490 0x7fa0cc05f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55738/0 (socket says 172.21.15.47:55738) 2022-06-29T16:53:05.893 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.876+0000 7fa0cb7fe700 1 -- 172.21.15.47:0/308659013 learned_addr learned my addr 172.21.15.47:0/308659013 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T16:53:05.894 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.876+0000 7fa0cbfff700 1 --2- 172.21.15.47:0/308659013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0cc064820 0x7fa0cc152d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:05.894 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.876+0000 7fa0d0de8700 1 --2- 172.21.15.47:0/308659013 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa0cc0b4420 0x7fa0cc1558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:05.894 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.876+0000 7fa0cb7fe700 1 -- 172.21.15.47:0/308659013 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa0cc0b4420 msgr2=0x7fa0cc1558e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:05.895 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.876+0000 7fa0cb7fe700 1 --2- 172.21.15.47:0/308659013 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa0cc0b4420 0x7fa0cc1558e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:05.895 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.876+0000 7fa0cb7fe700 1 -- 172.21.15.47:0/308659013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0cc064820 msgr2=0x7fa0cc152d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:05.895 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.876+0000 7fa0cb7fe700 1 --2- 172.21.15.47:0/308659013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0cc064820 0x7fa0cc152d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:05.896 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.876+0000 7fa0cb7fe700 1 -- 172.21.15.47:0/308659013 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0bc00b1e0 con 0x7fa0cc0b2490 2022-06-29T16:53:05.896 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.876+0000 7fa0cb7fe700 1 --2- 172.21.15.47:0/308659013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0cc0b2490 0x7fa0cc05f690 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 rx=0x7fa0b400c000 tx=0x7fa0b4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:05.896 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.876+0000 7fa0c97fa700 1 -- 172.21.15.47:0/308659013 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0b4010780 con 0x7fa0cc0b2490 2022-06-29T16:53:05.897 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.876+0000 7fa0c97fa700 1 -- 172.21.15.47:0/308659013 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b40108e0 con 0x7fa0cc0b2490 2022-06-29T16:53:05.897 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.876+0000 7fa0d284b700 1 -- 172.21.15.47:0/308659013 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0cc153280 con 0x7fa0cc0b2490 2022-06-29T16:53:05.897 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.876+0000 7fa0c97fa700 1 -- 172.21.15.47:0/308659013 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0b4010c40 con 0x7fa0cc0b2490 2022-06-29T16:53:05.898 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.876+0000 7fa0d284b700 1 -- 172.21.15.47:0/308659013 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0cc1493d0 con 0x7fa0cc0b2490 2022-06-29T16:53:05.898 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.876+0000 7fa0d284b700 1 -- 172.21.15.47:0/308659013 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0cc0b1f30 con 0x7fa0cc0b2490 2022-06-29T16:53:05.898 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.876+0000 7fa0c97fa700 1 -- 172.21.15.47:0/308659013 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa0b401c460 con 0x7fa0cc0b2490 2022-06-29T16:53:05.899 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.876+0000 7fa0c97fa700 1 --2- 172.21.15.47:0/308659013 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa0b00539e0 0x7fa0b0055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:05.899 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.876+0000 7fa0c97fa700 1 -- 172.21.15.47:0/308659013 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fa0b406c170 con 0x7fa0cc0b2490 2022-06-29T16:53:05.899 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.876+0000 7fa0cbfff700 1 --2- 172.21.15.47:0/308659013 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa0b00539e0 0x7fa0b0055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:05.900 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.880+0000 7fa0cbfff700 1 --2- 172.21.15.47:0/308659013 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa0b00539e0 0x7fa0b0055ea0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 rx=0x7fa0bc01b000 tx=0x7fa0bc015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:05.901 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:05.880+0000 7fa0c97fa700 1 -- 172.21.15.47:0/308659013 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa0b40431b0 con 0x7fa0cc0b2490 2022-06-29T16:53:06.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.008+0000 7fa0d284b700 1 -- 172.21.15.47:0/308659013 --> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fa0cc05fec0 con 0x7fa0b00539e0 2022-06-29T16:53:06.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.012+0000 7fa0c97fa700 1 -- 172.21.15.47:0/308659013 <== mgr.4102 v2:172.21.15.47:6800/12928 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa0cc05fec0 con 0x7fa0b00539e0 2022-06-29T16:53:06.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.012+0000 7fa0d284b700 1 -- 172.21.15.47:0/308659013 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa0b00539e0 msgr2=0x7fa0b0055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:06.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.012+0000 7fa0d284b700 1 --2- 172.21.15.47:0/308659013 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa0b00539e0 0x7fa0b0055ea0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 rx=0x7fa0bc01b000 tx=0x7fa0bc015000).stop 2022-06-29T16:53:06.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.012+0000 7fa0d284b700 1 -- 172.21.15.47:0/308659013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0cc0b2490 msgr2=0x7fa0cc05f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:06.022 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.012+0000 7fa0d284b700 1 --2- 172.21.15.47:0/308659013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0cc0b2490 0x7fa0cc05f690 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 rx=0x7fa0b400c000 tx=0x7fa0b4006df0).stop 2022-06-29T16:53:06.022 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.012+0000 7fa0d284b700 1 -- 172.21.15.47:0/308659013 shutdown_connections 2022-06-29T16:53:06.022 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.012+0000 7fa0d284b700 1 --2- 172.21.15.47:0/308659013 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fa0cc0b4420 0x7fa0cc1558e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:06.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.012+0000 7fa0d284b700 1 --2- 172.21.15.47:0/308659013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0cc0b2490 0x7fa0cc05f690 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:06.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.012+0000 7fa0d284b700 1 --2- 172.21.15.47:0/308659013 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fa0b00539e0 0x7fa0b0055ea0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:06.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.012+0000 7fa0d284b700 1 --2- 172.21.15.47:0/308659013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0cc064820 0x7fa0cc152d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:06.024 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.012+0000 7fa0d284b700 1 -- 172.21.15.47:0/308659013 >> 172.21.15.47:0/308659013 conn(0x7fa0cc05c510 msgr2=0x7fa0cc067360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T16:53:06.024 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.012+0000 7fa0d284b700 1 -- 172.21.15.47:0/308659013 shutdown_connections 2022-06-29T16:53:06.024 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.016+0000 7fa0d284b700 1 -- 172.21.15.47:0/308659013 wait complete. 2022-06-29T16:53:06.025 INFO:teuthology.orchestra.run.smithi047.stderr:instructed osd(s) 5 to deep-scrub 2022-06-29T16:53:06.031 INFO:tasks.ceph:Scrubbing osd.6 2022-06-29T16:53:06.032 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2022-06-29T16:53:06.140 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.132+0000 7f0eb1da9700 1 Processor -- start 2022-06-29T16:53:06.141 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.132+0000 7f0eb1da9700 1 -- start start 2022-06-29T16:53:06.143 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.136+0000 7f0eb1da9700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0eac061670 0x7f0eac061a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:06.143 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.136+0000 7f0eb1da9700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0eac0b0700 0x7f0eac1436c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:06.144 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.136+0000 7f0eb1da9700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0eac13db90 con 0x7f0eac0b0fe0 2022-06-29T16:53:06.144 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.136+0000 7f0eb1da9700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0eac13fed0 con 0x7f0eac061670 2022-06-29T16:53:06.144 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.136+0000 7f0eb1da9700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eac0af0a0 con 0x7f0eac0b0700 2022-06-29T16:53:06.145 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.136+0000 7f0eabfff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0eac0b0700 0x7f0eac1436c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:06.145 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.136+0000 7f0eabfff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0eac0b0700 0x7f0eac1436c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:52850/0 (socket says 172.21.15.47:52850) 2022-06-29T16:53:06.146 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.136+0000 7f0eabfff700 1 -- 172.21.15.47:0/1285856803 learned_addr learned my addr 172.21.15.47:0/1285856803 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T16:53:06.146 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.136+0000 7f0eab7fe700 1 --2- 172.21.15.47:0/1285856803 >> v2:172.21.15.64:3300/0 conn(0x7f0eac061670 0x7f0eac061a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:06.146 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.136+0000 7f0eabfff700 1 -- 172.21.15.47:0/1285856803 >> v2:172.21.15.64:3300/0 conn(0x7f0eac061670 msgr2=0x7f0eac061a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T16:53:06.147 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.136+0000 7f0eabfff700 1 --2- 172.21.15.47:0/1285856803 >> v2:172.21.15.64:3300/0 conn(0x7f0eac061670 0x7f0eac061a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:06.148 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.136+0000 7f0eabfff700 1 -- 172.21.15.47:0/1285856803 >> v1:172.21.15.64:6789/0 conn(0x7f0eac0b0fe0 legacy=0x7f0eac13d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:06.148 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.136+0000 7f0eaa7fc700 1 -- 172.21.15.47:0/1285856803 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4248615563 0 0) 0x7f0eac13db90 con 0x7f0eac0b0fe0 2022-06-29T16:53:06.148 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.136+0000 7f0eabfff700 1 -- 172.21.15.47:0/1285856803 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eac0af340 con 0x7f0eac0b0700 2022-06-29T16:53:06.149 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.136+0000 7f0eabfff700 1 --2- 172.21.15.47:0/1285856803 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0eac0b0700 0x7f0eac1436c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f0ea40032f0 tx=0x7f0ea4009290).ready entity=mon.2 client_cookie=6d0bd4ee13630ee5 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:06.149 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.136+0000 7f0eaa7fc700 1 -- 172.21.15.47:0/1285856803 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ea400b8a0 con 0x7f0eac0b0700 2022-06-29T16:53:06.149 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.136+0000 7f0eaa7fc700 1 -- 172.21.15.47:0/1285856803 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea400ba00 con 0x7f0eac0b0700 2022-06-29T16:53:06.150 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.136+0000 7f0eaa7fc700 1 -- 172.21.15.47:0/1285856803 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ea400bb70 con 0x7f0eac0b0700 2022-06-29T16:53:06.155 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.136+0000 7f0eb1da9700 1 -- 172.21.15.47:0/1285856803 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0eac0b0700 msgr2=0x7f0eac1436c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:06.156 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.136+0000 7f0eb1da9700 1 --2- 172.21.15.47:0/1285856803 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0eac0b0700 0x7f0eac1436c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f0ea40032f0 tx=0x7f0ea4009290).stop 2022-06-29T16:53:06.157 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.140+0000 7f0eb1da9700 1 -- 172.21.15.47:0/1285856803 shutdown_connections 2022-06-29T16:53:06.157 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.140+0000 7f0eb1da9700 1 --2- 172.21.15.47:0/1285856803 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0eac0b0700 0x7f0eac1436c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:06.157 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.140+0000 7f0eb1da9700 1 --2- 172.21.15.47:0/1285856803 >> v2:172.21.15.64:3300/0 conn(0x7f0eac061670 0x7f0eac061a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:06.158 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.140+0000 7f0eb1da9700 1 -- 172.21.15.47:0/1285856803 >> 172.21.15.47:0/1285856803 conn(0x7f0eac063800 msgr2=0x7f0eac063be0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T16:53:06.158 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.140+0000 7f0eb1da9700 1 -- 172.21.15.47:0/1285856803 shutdown_connections 2022-06-29T16:53:06.159 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.140+0000 7f0eb1da9700 1 -- 172.21.15.47:0/1285856803 wait complete. 2022-06-29T16:53:06.159 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.140+0000 7f0eb1da9700 1 Processor -- start 2022-06-29T16:53:06.159 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.140+0000 7f0eb1da9700 1 -- start start 2022-06-29T16:53:06.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.144+0000 7f0eb1da9700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0eac061670 0x7f0eac13d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:06.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.144+0000 7f0eb1da9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0eac0b0700 0x7f0eac142bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:06.160 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.144+0000 7f0eb1da9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0eac0b0fe0 0x7f0eac1408c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:06.161 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.144+0000 7f0eb1da9700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eac1405e0 con 0x7f0eac0b0700 2022-06-29T16:53:06.161 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.144+0000 7f0eb1da9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eac0acdc0 con 0x7f0eac0b0fe0 2022-06-29T16:53:06.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.144+0000 7f0eb1da9700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eac0ac120 con 0x7f0eac061670 2022-06-29T16:53:06.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.144+0000 7f0eaaffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0eac0b0700 0x7f0eac142bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:06.162 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.144+0000 7f0eaaffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0eac0b0700 0x7f0eac142bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55752/0 (socket says 172.21.15.47:55752) 2022-06-29T16:53:06.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.144+0000 7f0eaaffd700 1 -- 172.21.15.47:0/2945138860 learned_addr learned my addr 172.21.15.47:0/2945138860 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T16:53:06.163 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.144+0000 7f0eab7fe700 1 --2- 172.21.15.47:0/2945138860 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0eac061670 0x7f0eac13d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:06.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.144+0000 7f0eabfff700 1 --2- 172.21.15.47:0/2945138860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0eac0b0fe0 0x7f0eac1408c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:06.164 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.144+0000 7f0eaaffd700 1 -- 172.21.15.47:0/2945138860 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0eac061670 msgr2=0x7f0eac13d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:06.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.144+0000 7f0eaaffd700 1 --2- 172.21.15.47:0/2945138860 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0eac061670 0x7f0eac13d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:06.165 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.144+0000 7f0eaaffd700 1 -- 172.21.15.47:0/2945138860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0eac0b0fe0 msgr2=0x7f0eac1408c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:06.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.144+0000 7f0eaaffd700 1 --2- 172.21.15.47:0/2945138860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0eac0b0fe0 0x7f0eac1408c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:06.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.144+0000 7f0eaaffd700 1 -- 172.21.15.47:0/2945138860 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea4009530 con 0x7f0eac0b0700 2022-06-29T16:53:06.166 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.144+0000 7f0eabfff700 1 --2- 172.21.15.47:0/2945138860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0eac0b0fe0 0x7f0eac1408c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T16:53:06.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.144+0000 7f0eaaffd700 1 --2- 172.21.15.47:0/2945138860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0eac0b0700 0x7f0eac142bf0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 rx=0x7f0ea0000ed0 tx=0x7f0ea0007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:06.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.144+0000 7f0ea8ff9700 1 -- 172.21.15.47:0/2945138860 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ea000ce80 con 0x7f0eac0b0700 2022-06-29T16:53:06.167 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.144+0000 7f0ea8ff9700 1 -- 172.21.15.47:0/2945138860 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea0013450 con 0x7f0eac0b0700 2022-06-29T16:53:06.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.144+0000 7f0eb1da9700 1 -- 172.21.15.47:0/2945138860 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eac0acb20 con 0x7f0eac0b0700 2022-06-29T16:53:06.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.144+0000 7f0ea8ff9700 1 -- 172.21.15.47:0/2945138860 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ea00137b0 con 0x7f0eac0b0700 2022-06-29T16:53:06.168 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.144+0000 7f0eb1da9700 1 -- 172.21.15.47:0/2945138860 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eac0b2c90 con 0x7f0eac0b0700 2022-06-29T16:53:06.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.148+0000 7f0eb1da9700 1 -- 172.21.15.47:0/2945138860 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0eac0ac8c0 con 0x7f0eac0b0700 2022-06-29T16:53:06.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.148+0000 7f0ea8ff9700 1 -- 172.21.15.47:0/2945138860 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0ea001a060 con 0x7f0eac0b0700 2022-06-29T16:53:06.169 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.148+0000 7f0ea8ff9700 1 --2- 172.21.15.47:0/2945138860 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0e8c053a10 0x7f0e8c055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:06.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.148+0000 7f0ea8ff9700 1 -- 172.21.15.47:0/2945138860 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f0ea006b6e0 con 0x7f0eac0b0700 2022-06-29T16:53:06.170 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.148+0000 7f0eab7fe700 1 --2- 172.21.15.47:0/2945138860 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0e8c053a10 0x7f0e8c055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:06.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.148+0000 7f0ea8ff9700 1 --2- 172.21.15.47:0/2945138860 >> [v2:172.21.15.64:6824/13395,v1:172.21.15.64:6825/13395] conn(0x7f0e8c05a6a0 0x7f0e8c05cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:06.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.148+0000 7f0ea8ff9700 1 -- 172.21.15.47:0/2945138860 --> [v2:172.21.15.64:6824/13395,v1:172.21.15.64:6825/13395] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0e8c05d1c0 con 0x7f0e8c05a6a0 2022-06-29T16:53:06.171 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.148+0000 7f0ea8ff9700 1 -- 172.21.15.47:0/2945138860 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=165) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ea006dc60 con 0x7f0eac0b0700 2022-06-29T16:53:06.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.148+0000 7f0eabfff700 1 --2- 172.21.15.47:0/2945138860 >> [v2:172.21.15.64:6824/13395,v1:172.21.15.64:6825/13395] conn(0x7f0e8c05a6a0 0x7f0e8c05cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:06.172 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.148+0000 7f0eab7fe700 1 --2- 172.21.15.47:0/2945138860 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0e8c053a10 0x7f0e8c055ed0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 rx=0x7f0e9c00b000 tx=0x7f0e9c006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:06.173 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.148+0000 7f0eabfff700 1 --2- 172.21.15.47:0/2945138860 >> [v2:172.21.15.64:6824/13395,v1:172.21.15.64:6825/13395] conn(0x7f0e8c05a6a0 0x7f0e8c05cb40 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:06.174 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.152+0000 7f0ea8ff9700 1 -- 172.21.15.47:0/2945138860 <== osd.6 v2:172.21.15.64:6824/13395 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f0e8c05d1c0 con 0x7f0e8c05a6a0 2022-06-29T16:53:06.176 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.168+0000 7f0eb1da9700 1 -- 172.21.15.47:0/2945138860 --> [v2:172.21.15.64:6824/13395,v1:172.21.15.64:6825/13395] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f0eac0573d0 con 0x7f0e8c05a6a0 2022-06-29T16:53:06.177 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.172+0000 7f0ea8ff9700 1 -- 172.21.15.47:0/2945138860 <== osd.6 v2:172.21.15.64:6824/13395 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f0eac0573d0 con 0x7f0e8c05a6a0 2022-06-29T16:53:06.177 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2022-06-29T16:53:06.178 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2022-06-29T16:53:06.178 INFO:teuthology.orchestra.run.smithi047.stdout:} 2022-06-29T16:53:06.179 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.172+0000 7f0eb1da9700 1 -- 172.21.15.47:0/2945138860 >> [v2:172.21.15.64:6824/13395,v1:172.21.15.64:6825/13395] conn(0x7f0e8c05a6a0 msgr2=0x7f0e8c05cb40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:06.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.172+0000 7f0eb1da9700 1 --2- 172.21.15.47:0/2945138860 >> [v2:172.21.15.64:6824/13395,v1:172.21.15.64:6825/13395] conn(0x7f0e8c05a6a0 0x7f0e8c05cb40 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:06.180 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.172+0000 7f0eb1da9700 1 -- 172.21.15.47:0/2945138860 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0e8c053a10 msgr2=0x7f0e8c055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:06.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.172+0000 7f0eb1da9700 1 --2- 172.21.15.47:0/2945138860 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0e8c053a10 0x7f0e8c055ed0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 rx=0x7f0e9c00b000 tx=0x7f0e9c006df0).stop 2022-06-29T16:53:06.181 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.172+0000 7f0eb1da9700 1 -- 172.21.15.47:0/2945138860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0eac0b0700 msgr2=0x7f0eac142bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:06.182 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.172+0000 7f0eb1da9700 1 --2- 172.21.15.47:0/2945138860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0eac0b0700 0x7f0eac142bf0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 rx=0x7f0ea0000ed0 tx=0x7f0ea0007c20).stop 2022-06-29T16:53:06.182 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.172+0000 7f0eabfff700 1 -- 172.21.15.47:0/2945138860 reap_dead start 2022-06-29T16:53:06.183 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.172+0000 7f0eb1da9700 1 -- 172.21.15.47:0/2945138860 shutdown_connections 2022-06-29T16:53:06.183 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.172+0000 7f0eb1da9700 1 -- 172.21.15.47:0/2945138860 >> 172.21.15.47:0/2945138860 conn(0x7f0eac063800 msgr2=0x7f0eac0b14f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T16:53:06.183 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.172+0000 7f0eb1da9700 1 -- 172.21.15.47:0/2945138860 shutdown_connections 2022-06-29T16:53:06.184 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.172+0000 7f0eb1da9700 1 -- 172.21.15.47:0/2945138860 wait complete. 2022-06-29T16:53:06.189 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2022-06-29T16:53:06.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.292+0000 7f0b8ad49700 1 Processor -- start 2022-06-29T16:53:06.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.292+0000 7f0b8ad49700 1 -- start start 2022-06-29T16:53:06.302 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.292+0000 7f0b8ad49700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0b840b0cf0 0x7f0b84146d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:06.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.292+0000 7f0b8ad49700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b840b4520 con 0x7f0b840b06d0 2022-06-29T16:53:06.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.292+0000 7f0b8ad49700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b84061590 con 0x7f0b840611b0 2022-06-29T16:53:06.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.292+0000 7f0b8ad49700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b840b4b10 con 0x7f0b840b0cf0 2022-06-29T16:53:06.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.292+0000 7f0b83fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0b840b0cf0 0x7f0b84146d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:06.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.292+0000 7f0b892e6700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0b840b06d0 0x7f0b8413ff00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41318/0 (socket says 172.21.15.47:41318) 2022-06-29T16:53:06.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.292+0000 7f0b892e6700 1 -- 172.21.15.47:0/2577016476 learned_addr learned my addr 172.21.15.47:0/2577016476 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T16:53:06.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.296+0000 7f0b837fe700 1 -- 172.21.15.47:0/2577016476 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2469903405 0 0) 0x7f0b84061590 con 0x7f0b840611b0 2022-06-29T16:53:06.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.296+0000 7f0b83fff700 1 -- 172.21.15.47:0/2577016476 >> v1:172.21.15.64:6789/0 conn(0x7f0b840611b0 legacy=0x7f0b84146640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:06.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.296+0000 7f0b83fff700 1 -- 172.21.15.47:0/2577016476 >> v1:172.21.15.47:6789/0 conn(0x7f0b840b06d0 legacy=0x7f0b8413ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:06.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.296+0000 7f0b83fff700 1 -- 172.21.15.47:0/2577016476 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b840b5e00 con 0x7f0b840b0cf0 2022-06-29T16:53:06.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.296+0000 7f0b83fff700 1 --2- 172.21.15.47:0/2577016476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b840b0cf0 0x7f0b84146d50 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 rx=0x7f0b840b1c00 tx=0x7f0b6c0091d0).ready entity=mon.0 client_cookie=a28e9a3176229abe server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:06.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.296+0000 7f0b837fe700 1 -- 172.21.15.47:0/2577016476 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b6c00b860 con 0x7f0b840b0cf0 2022-06-29T16:53:06.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.296+0000 7f0b837fe700 1 -- 172.21.15.47:0/2577016476 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b6c00b9c0 con 0x7f0b840b0cf0 2022-06-29T16:53:06.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.296+0000 7f0b837fe700 1 -- 172.21.15.47:0/2577016476 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b6c00bb30 con 0x7f0b840b0cf0 2022-06-29T16:53:06.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.296+0000 7f0b8ad49700 1 -- 172.21.15.47:0/2577016476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b840b0cf0 msgr2=0x7f0b84146d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:06.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.296+0000 7f0b8ad49700 1 --2- 172.21.15.47:0/2577016476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b840b0cf0 0x7f0b84146d50 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 rx=0x7f0b840b1c00 tx=0x7f0b6c0091d0).stop 2022-06-29T16:53:06.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.296+0000 7f0b8ad49700 1 -- 172.21.15.47:0/2577016476 shutdown_connections 2022-06-29T16:53:06.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.296+0000 7f0b8ad49700 1 --2- 172.21.15.47:0/2577016476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b840b0cf0 0x7f0b84146d50 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:06.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.296+0000 7f0b8ad49700 1 -- 172.21.15.47:0/2577016476 >> 172.21.15.47:0/2577016476 conn(0x7f0b840b98b0 msgr2=0x7f0b840b9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T16:53:06.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.296+0000 7f0b8ad49700 1 -- 172.21.15.47:0/2577016476 shutdown_connections 2022-06-29T16:53:06.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.296+0000 7f0b8ad49700 1 -- 172.21.15.47:0/2577016476 wait complete. 2022-06-29T16:53:06.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.296+0000 7f0b8ad49700 1 Processor -- start 2022-06-29T16:53:06.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.296+0000 7f0b8ad49700 1 -- start start 2022-06-29T16:53:06.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.300+0000 7f0b8ad49700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0b840611b0 0x7f0b840b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:06.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.300+0000 7f0b8ad49700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b840b06d0 0x7f0b84148d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:06.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.300+0000 7f0b8ad49700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b840b0cf0 0x7f0b84156020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:06.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.300+0000 7f0b8ad49700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b840b8100 con 0x7f0b840b0cf0 2022-06-29T16:53:06.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.300+0000 7f0b8ad49700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b840b7690 con 0x7f0b840b06d0 2022-06-29T16:53:06.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.300+0000 7f0b8ad49700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b840b8330 con 0x7f0b840611b0 2022-06-29T16:53:06.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.300+0000 7f0b892e6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b840b0cf0 0x7f0b84156020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:06.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.300+0000 7f0b83fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b840b06d0 0x7f0b84148d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:06.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.300+0000 7f0b88ae5700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0b840611b0 0x7f0b840b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:06.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.300+0000 7f0b892e6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b840b0cf0 0x7f0b84156020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55770/0 (socket says 172.21.15.47:55770) 2022-06-29T16:53:06.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.300+0000 7f0b892e6700 1 -- 172.21.15.47:0/2331870838 learned_addr learned my addr 172.21.15.47:0/2331870838 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T16:53:06.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.300+0000 7f0b88ae5700 1 -- 172.21.15.47:0/2331870838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b840b06d0 msgr2=0x7f0b84148d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:06.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.300+0000 7f0b88ae5700 1 --2- 172.21.15.47:0/2331870838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b840b06d0 0x7f0b84148d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:06.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.300+0000 7f0b88ae5700 1 -- 172.21.15.47:0/2331870838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b840b0cf0 msgr2=0x7f0b84156020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:06.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.300+0000 7f0b88ae5700 1 --2- 172.21.15.47:0/2331870838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b840b0cf0 0x7f0b84156020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:06.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.300+0000 7f0b88ae5700 1 -- 172.21.15.47:0/2331870838 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b6c009470 con 0x7f0b840611b0 2022-06-29T16:53:06.319 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.300+0000 7f0b892e6700 1 --2- 172.21.15.47:0/2331870838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b840b0cf0 0x7f0b84156020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T16:53:06.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.300+0000 7f0b88ae5700 1 --2- 172.21.15.47:0/2331870838 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0b840611b0 0x7f0b840b8c20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f0b74000ed0 tx=0x7f0b74007a70).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:06.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.300+0000 7f0b81ffb700 1 -- 172.21.15.47:0/2331870838 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b7400cde0 con 0x7f0b840611b0 2022-06-29T16:53:06.320 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.300+0000 7f0b81ffb700 1 -- 172.21.15.47:0/2331870838 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b74012400 con 0x7f0b840611b0 2022-06-29T16:53:06.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.300+0000 7f0b81ffb700 1 -- 172.21.15.47:0/2331870838 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b74012760 con 0x7f0b840611b0 2022-06-29T16:53:06.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.300+0000 7f0b8ad49700 1 -- 172.21.15.47:0/2331870838 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b840b85d0 con 0x7f0b840611b0 2022-06-29T16:53:06.321 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.300+0000 7f0b8ad49700 1 -- 172.21.15.47:0/2331870838 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b840b9510 con 0x7f0b840611b0 2022-06-29T16:53:06.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.300+0000 7f0b8ad49700 1 -- 172.21.15.47:0/2331870838 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b84155d80 con 0x7f0b840611b0 2022-06-29T16:53:06.322 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.304+0000 7f0b81ffb700 1 -- 172.21.15.47:0/2331870838 <== mon.2 v2:172.21.15.64:3301/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0b74019030 con 0x7f0b840611b0 2022-06-29T16:53:06.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.304+0000 7f0b81ffb700 1 --2- 172.21.15.47:0/2331870838 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0b68053b00 0x7f0b68055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:06.323 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.304+0000 7f0b81ffb700 1 -- 172.21.15.47:0/2331870838 <== mon.2 v2:172.21.15.64:3301/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f0b7406b070 con 0x7f0b840611b0 2022-06-29T16:53:06.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.304+0000 7f0b83fff700 1 --2- 172.21.15.47:0/2331870838 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0b68053b00 0x7f0b68055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:06.324 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.304+0000 7f0b83fff700 1 --2- 172.21.15.47:0/2331870838 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0b68053b00 0x7f0b68055fc0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 rx=0x7f0b6c01b000 tx=0x7f0b6c016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:06.325 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.312+0000 7f0b81ffb700 1 -- 172.21.15.47:0/2331870838 <== mon.2 v2:172.21.15.64:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0b84155d80 con 0x7f0b840611b0 2022-06-29T16:53:06.451 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.444+0000 7f0b8ad49700 1 -- 172.21.15.47:0/2331870838 --> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f0b84155d80 con 0x7f0b68053b00 2022-06-29T16:53:06.451 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.444+0000 7f0b81ffb700 1 -- 172.21.15.47:0/2331870838 <== mgr.4102 v2:172.21.15.47:6800/12928 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f0b84155d80 con 0x7f0b68053b00 2022-06-29T16:53:06.455 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.448+0000 7f0b8ad49700 1 -- 172.21.15.47:0/2331870838 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0b68053b00 msgr2=0x7f0b68055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:06.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.448+0000 7f0b8ad49700 1 --2- 172.21.15.47:0/2331870838 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0b68053b00 0x7f0b68055fc0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 rx=0x7f0b6c01b000 tx=0x7f0b6c016000).stop 2022-06-29T16:53:06.456 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.448+0000 7f0b8ad49700 1 -- 172.21.15.47:0/2331870838 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0b840611b0 msgr2=0x7f0b840b8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:06.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.448+0000 7f0b8ad49700 1 --2- 172.21.15.47:0/2331870838 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0b840611b0 0x7f0b840b8c20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f0b74000ed0 tx=0x7f0b74007a70).stop 2022-06-29T16:53:06.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.448+0000 7f0b8ad49700 1 -- 172.21.15.47:0/2331870838 shutdown_connections 2022-06-29T16:53:06.457 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.448+0000 7f0b8ad49700 1 --2- 172.21.15.47:0/2331870838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b840b0cf0 0x7f0b84156020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:06.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.448+0000 7f0b8ad49700 1 --2- 172.21.15.47:0/2331870838 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0b68053b00 0x7f0b68055fc0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:06.458 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.448+0000 7f0b8ad49700 1 --2- 172.21.15.47:0/2331870838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b840b06d0 0x7f0b84148d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:06.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.448+0000 7f0b8ad49700 1 --2- 172.21.15.47:0/2331870838 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0b840611b0 0x7f0b840b8c20 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:06.459 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.448+0000 7f0b8ad49700 1 -- 172.21.15.47:0/2331870838 >> 172.21.15.47:0/2331870838 conn(0x7f0b840b98b0 msgr2=0x7f0b841481e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T16:53:06.460 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.448+0000 7f0b8ad49700 1 -- 172.21.15.47:0/2331870838 shutdown_connections 2022-06-29T16:53:06.461 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.448+0000 7f0b8ad49700 1 -- 172.21.15.47:0/2331870838 wait complete. 2022-06-29T16:53:06.463 INFO:teuthology.orchestra.run.smithi047.stderr:instructed osd(s) 6 to deep-scrub 2022-06-29T16:53:06.468 INFO:tasks.ceph:Scrubbing osd.7 2022-06-29T16:53:06.469 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2022-06-29T16:53:06.583 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.576+0000 7ff5ed18a700 1 Processor -- start 2022-06-29T16:53:06.583 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.576+0000 7ff5ed18a700 1 -- start start 2022-06-29T16:53:06.586 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.576+0000 7ff5ed18a700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5e8060ee0 0x7ff5e80612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:06.586 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.576+0000 7ff5ed18a700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5e8067610 con 0x7ff5e81471a0 2022-06-29T16:53:06.586 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.576+0000 7ff5ed18a700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5e81467c0 con 0x7ff5e80b4300 2022-06-29T16:53:06.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.576+0000 7ff5ed18a700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e81475d0 con 0x7ff5e8060ee0 2022-06-29T16:53:06.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.576+0000 7ff5e759e700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff5e81471a0 0x7ff5e813cdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41332/0 (socket says 172.21.15.47:41332) 2022-06-29T16:53:06.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.580+0000 7ff5e759e700 1 -- 172.21.15.47:0/4047940485 learned_addr learned my addr 172.21.15.47:0/4047940485 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T16:53:06.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.580+0000 7ff5e6d9d700 1 --2- 172.21.15.47:0/4047940485 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5e8060ee0 0x7ff5e80612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:06.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.580+0000 7ff5e659c700 1 -- 172.21.15.47:0/4047940485 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1970154474 0 0) 0x7ff5e8067610 con 0x7ff5e81471a0 2022-06-29T16:53:06.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.580+0000 7ff5e659c700 1 -- 172.21.15.47:0/4047940485 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5cc003260 con 0x7ff5e81471a0 2022-06-29T16:53:06.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.580+0000 7ff5e6d9d700 1 -- 172.21.15.47:0/4047940485 >> v1:172.21.15.64:6789/0 conn(0x7ff5e80b4300 legacy=0x7ff5e81430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:06.597 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.580+0000 7ff5e659c700 1 -- 172.21.15.47:0/4047940485 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 739041874 0 0) 0x7ff5e81467c0 con 0x7ff5e80b4300 2022-06-29T16:53:06.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.580+0000 7ff5e6d9d700 1 -- 172.21.15.47:0/4047940485 >> v1:172.21.15.47:6789/0 conn(0x7ff5e81471a0 legacy=0x7ff5e813cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:06.598 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.580+0000 7ff5e6d9d700 1 -- 172.21.15.47:0/4047940485 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e8146a60 con 0x7ff5e8060ee0 2022-06-29T16:53:06.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.580+0000 7ff5e6d9d700 1 --2- 172.21.15.47:0/4047940485 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5e8060ee0 0x7ff5e80612c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7ff5d8002f70 tx=0x7ff5d8006df0).ready entity=mon.2 client_cookie=f57cec5ee314be68 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:06.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.580+0000 7ff5e659c700 1 -- 172.21.15.47:0/4047940485 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff5d800d570 con 0x7ff5e8060ee0 2022-06-29T16:53:06.599 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.580+0000 7ff5e659c700 1 -- 172.21.15.47:0/4047940485 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5d8000e90 con 0x7ff5e8060ee0 2022-06-29T16:53:06.600 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.580+0000 7ff5e659c700 1 -- 172.21.15.47:0/4047940485 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff5d800d8c0 con 0x7ff5e8060ee0 2022-06-29T16:53:06.605 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.580+0000 7ff5ed18a700 1 -- 172.21.15.47:0/4047940485 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5e8060ee0 msgr2=0x7ff5e80612c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:06.606 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.580+0000 7ff5ed18a700 1 --2- 172.21.15.47:0/4047940485 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5e8060ee0 0x7ff5e80612c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7ff5d8002f70 tx=0x7ff5d8006df0).stop 2022-06-29T16:53:06.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.580+0000 7ff5ed18a700 1 -- 172.21.15.47:0/4047940485 shutdown_connections 2022-06-29T16:53:06.607 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.580+0000 7ff5ed18a700 1 --2- 172.21.15.47:0/4047940485 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5e8060ee0 0x7ff5e80612c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:06.608 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.580+0000 7ff5ed18a700 1 -- 172.21.15.47:0/4047940485 >> 172.21.15.47:0/4047940485 conn(0x7ff5e805c510 msgr2=0x7ff5e805f360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T16:53:06.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.580+0000 7ff5ed18a700 1 -- 172.21.15.47:0/4047940485 shutdown_connections 2022-06-29T16:53:06.609 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.580+0000 7ff5ed18a700 1 -- 172.21.15.47:0/4047940485 wait complete. 2022-06-29T16:53:06.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.584+0000 7ff5ed18a700 1 Processor -- start 2022-06-29T16:53:06.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.584+0000 7ff5ed18a700 1 -- start start 2022-06-29T16:53:06.610 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.584+0000 7ff5ed18a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5e8060ee0 0x7ff5e8149c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:06.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.584+0000 7ff5ed18a700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5e80b4300 0x7ff5e813c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:06.611 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.584+0000 7ff5ed18a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5e81471a0 0x7ff5e813c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:06.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.584+0000 7ff5ed18a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e814a560 con 0x7ff5e81471a0 2022-06-29T16:53:06.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.584+0000 7ff5ed18a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e80b3db0 con 0x7ff5e8060ee0 2022-06-29T16:53:06.612 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.584+0000 7ff5ed18a700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e81530c0 con 0x7ff5e80b4300 2022-06-29T16:53:06.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.584+0000 7ff5e759e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5e81471a0 0x7ff5e813c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:06.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.584+0000 7ff5e759e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5e81471a0 0x7ff5e813c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55784/0 (socket says 172.21.15.47:55784) 2022-06-29T16:53:06.613 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.584+0000 7ff5e759e700 1 -- 172.21.15.47:0/2850742024 learned_addr learned my addr 172.21.15.47:0/2850742024 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T16:53:06.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.584+0000 7ff5e6d9d700 1 --2- 172.21.15.47:0/2850742024 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5e8060ee0 0x7ff5e8149c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:06.614 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.584+0000 7ff5de59c700 1 --2- 172.21.15.47:0/2850742024 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5e80b4300 0x7ff5e813c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:06.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.584+0000 7ff5e759e700 1 -- 172.21.15.47:0/2850742024 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5e80b4300 msgr2=0x7ff5e813c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:06.615 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.584+0000 7ff5e759e700 1 --2- 172.21.15.47:0/2850742024 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7ff5e80b4300 0x7ff5e813c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:06.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.584+0000 7ff5e759e700 1 -- 172.21.15.47:0/2850742024 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5e8060ee0 msgr2=0x7ff5e8149c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:06.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.584+0000 7ff5e759e700 1 --2- 172.21.15.47:0/2850742024 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5e8060ee0 0x7ff5e8149c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:06.616 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.584+0000 7ff5e759e700 1 -- 172.21.15.47:0/2850742024 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5d800b180 con 0x7ff5e81471a0 2022-06-29T16:53:06.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.584+0000 7ff5e759e700 1 --2- 172.21.15.47:0/2850742024 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5e81471a0 0x7ff5e813c6c0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 rx=0x7ff5e0001490 tx=0x7ff5e0007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:06.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.584+0000 7ff5e4d99700 1 -- 172.21.15.47:0/2850742024 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff5e0005ba0 con 0x7ff5e81471a0 2022-06-29T16:53:06.617 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.584+0000 7ff5ed18a700 1 -- 172.21.15.47:0/2850742024 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5e8153360 con 0x7ff5e81471a0 2022-06-29T16:53:06.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.584+0000 7ff5e4d99700 1 -- 172.21.15.47:0/2850742024 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e0005d00 con 0x7ff5e81471a0 2022-06-29T16:53:06.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.584+0000 7ff5ed18a700 1 -- 172.21.15.47:0/2850742024 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5e80b3aa0 con 0x7ff5e81471a0 2022-06-29T16:53:06.618 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.584+0000 7ff5e4d99700 1 -- 172.21.15.47:0/2850742024 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff5e0005e70 con 0x7ff5e81471a0 2022-06-29T16:53:06.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.588+0000 7ff5e4d99700 1 -- 172.21.15.47:0/2850742024 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff5e0010070 con 0x7ff5e81471a0 2022-06-29T16:53:06.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.588+0000 7ff5ed18a700 1 -- 172.21.15.47:0/2850742024 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff5e8153560 con 0x7ff5e81471a0 2022-06-29T16:53:06.619 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.588+0000 7ff5e4d99700 1 --2- 172.21.15.47:0/2850742024 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff5c0053a70 0x7ff5c0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:06.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.588+0000 7ff5e4d99700 1 -- 172.21.15.47:0/2850742024 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5686+0+0 (secure 0 0 0) 0x7ff5e006b610 con 0x7ff5e81471a0 2022-06-29T16:53:06.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.588+0000 7ff5e6d9d700 1 --2- 172.21.15.47:0/2850742024 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff5c0053a70 0x7ff5c0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:06.620 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.588+0000 7ff5e4d99700 1 --2- 172.21.15.47:0/2850742024 >> [v2:172.21.15.64:6808/13394,v1:172.21.15.64:6809/13394] conn(0x7ff5c005a700 0x7ff5c005cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:06.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.588+0000 7ff5e4d99700 1 -- 172.21.15.47:0/2850742024 --> [v2:172.21.15.64:6808/13394,v1:172.21.15.64:6809/13394] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff5c005d180 con 0x7ff5c005a700 2022-06-29T16:53:06.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.588+0000 7ff5e4d99700 1 -- 172.21.15.47:0/2850742024 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=165) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5e0042520 con 0x7ff5e81471a0 2022-06-29T16:53:06.621 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.588+0000 7ff5de59c700 1 --2- 172.21.15.47:0/2850742024 >> [v2:172.21.15.64:6808/13394,v1:172.21.15.64:6809/13394] conn(0x7ff5c005a700 0x7ff5c005cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:06.622 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.588+0000 7ff5e6d9d700 1 --2- 172.21.15.47:0/2850742024 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff5c0053a70 0x7ff5c0055f30 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 rx=0x7ff5d8018000 tx=0x7ff5d8014000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:06.622 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.588+0000 7ff5de59c700 1 --2- 172.21.15.47:0/2850742024 >> [v2:172.21.15.64:6808/13394,v1:172.21.15.64:6809/13394] conn(0x7ff5c005a700 0x7ff5c005cba0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:06.622 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.592+0000 7ff5e4d99700 1 -- 172.21.15.47:0/2850742024 <== osd.7 v2:172.21.15.64:6808/13394 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7ff5c005d180 con 0x7ff5c005a700 2022-06-29T16:53:06.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.608+0000 7ff5ed18a700 1 -- 172.21.15.47:0/2850742024 --> [v2:172.21.15.64:6808/13394,v1:172.21.15.64:6809/13394] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff5e80573d0 con 0x7ff5c005a700 2022-06-29T16:53:06.624 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.608+0000 7ff5e4d99700 1 -- 172.21.15.47:0/2850742024 <== osd.7 v2:172.21.15.64:6808/13394 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7ff5e80573d0 con 0x7ff5c005a700 2022-06-29T16:53:06.625 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2022-06-29T16:53:06.626 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2022-06-29T16:53:06.626 INFO:teuthology.orchestra.run.smithi047.stdout:} 2022-06-29T16:53:06.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.608+0000 7ff5ed18a700 1 -- 172.21.15.47:0/2850742024 >> [v2:172.21.15.64:6808/13394,v1:172.21.15.64:6809/13394] conn(0x7ff5c005a700 msgr2=0x7ff5c005cba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:06.628 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.608+0000 7ff5ed18a700 1 --2- 172.21.15.47:0/2850742024 >> [v2:172.21.15.64:6808/13394,v1:172.21.15.64:6809/13394] conn(0x7ff5c005a700 0x7ff5c005cba0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:06.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.608+0000 7ff5ed18a700 1 -- 172.21.15.47:0/2850742024 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff5c0053a70 msgr2=0x7ff5c0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:06.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.608+0000 7ff5ed18a700 1 --2- 172.21.15.47:0/2850742024 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7ff5c0053a70 0x7ff5c0055f30 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 rx=0x7ff5d8018000 tx=0x7ff5d8014000).stop 2022-06-29T16:53:06.629 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.608+0000 7ff5ed18a700 1 -- 172.21.15.47:0/2850742024 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5e81471a0 msgr2=0x7ff5e813c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:06.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.608+0000 7ff5ed18a700 1 --2- 172.21.15.47:0/2850742024 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5e81471a0 0x7ff5e813c6c0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 rx=0x7ff5e0001490 tx=0x7ff5e0007e00).stop 2022-06-29T16:53:06.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.608+0000 7ff5e759e700 1 -- 172.21.15.47:0/2850742024 reap_dead start 2022-06-29T16:53:06.630 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.608+0000 7ff5ed18a700 1 -- 172.21.15.47:0/2850742024 shutdown_connections 2022-06-29T16:53:06.631 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.608+0000 7ff5ed18a700 1 -- 172.21.15.47:0/2850742024 >> 172.21.15.47:0/2850742024 conn(0x7ff5e805c510 msgr2=0x7ff5e80b7990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T16:53:06.631 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.608+0000 7ff5ed18a700 1 -- 172.21.15.47:0/2850742024 shutdown_connections 2022-06-29T16:53:06.632 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.608+0000 7ff5ed18a700 1 -- 172.21.15.47:0/2850742024 wait complete. 2022-06-29T16:53:06.633 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2022-06-29T16:53:06.752 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.736+0000 7f147cd4e700 1 Processor -- start 2022-06-29T16:53:06.753 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.736+0000 7f147cd4e700 1 -- start start 2022-06-29T16:53:06.753 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.736+0000 7f147cd4e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f14780b5c60 0x7f1478144980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:06.753 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.736+0000 7f147cd4e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1478060eb0 0x7f1478144e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:06.754 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.736+0000 7f147cd4e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f14780a72f0 0x7f14780b1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:06.754 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.736+0000 7f147cd4e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1478145360 con 0x7f14780a72f0 2022-06-29T16:53:06.754 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.736+0000 7f147cd4e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14780a9120 con 0x7f14780b5c60 2022-06-29T16:53:06.755 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.736+0000 7f147cd4e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14780b0b80 con 0x7f1478060eb0 2022-06-29T16:53:06.755 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.736+0000 7f1477fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f14780a72f0 0x7f14780b1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:06.756 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.736+0000 7f1476ffd700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1478060eb0 0x7f1478144e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:06.756 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.736+0000 7f14777fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f14780b5c60 0x7f1478144980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:06.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.736+0000 7f1476ffd700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1478060eb0 0x7f1478144e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3301/0 says I am v2:172.21.15.47:52894/0 (socket says 172.21.15.47:52894) 2022-06-29T16:53:06.757 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.736+0000 7f1476ffd700 1 -- 172.21.15.47:0/1860054086 learned_addr learned my addr 172.21.15.47:0/1860054086 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T16:53:06.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.736+0000 7f1476ffd700 1 -- 172.21.15.47:0/1860054086 >> v2:172.21.15.64:3300/0 conn(0x7f14780b5c60 msgr2=0x7f1478144980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T16:53:06.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.736+0000 7f1476ffd700 1 --2- 172.21.15.47:0/1860054086 >> v2:172.21.15.64:3300/0 conn(0x7f14780b5c60 0x7f1478144980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:06.758 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.736+0000 7f1476ffd700 1 -- 172.21.15.47:0/1860054086 >> v2:172.21.15.47:3300/0 conn(0x7f14780a72f0 msgr2=0x7f14780b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T16:53:06.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.736+0000 7f1476ffd700 1 --2- 172.21.15.47:0/1860054086 >> v2:172.21.15.47:3300/0 conn(0x7f14780a72f0 0x7f14780b1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:06.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.736+0000 7f1476ffd700 1 -- 172.21.15.47:0/1860054086 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14780a89e0 con 0x7f1478060eb0 2022-06-29T16:53:06.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.736+0000 7f1477fff700 1 --2- 172.21.15.47:0/1860054086 >> v2:172.21.15.47:3300/0 conn(0x7f14780a72f0 0x7f14780b1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-06-29T16:53:06.759 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.736+0000 7f1476ffd700 1 --2- 172.21.15.47:0/1860054086 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1478060eb0 0x7f1478144e70 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f14700032f0 tx=0x7f1470009290).ready entity=mon.2 client_cookie=4e9b3628a595b86f server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:06.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.736+0000 7f14767fc700 1 -- 172.21.15.47:0/1860054086 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f147000b680 con 0x7f1478060eb0 2022-06-29T16:53:06.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.736+0000 7f14767fc700 1 -- 172.21.15.47:0/1860054086 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f147000b7e0 con 0x7f1478060eb0 2022-06-29T16:53:06.760 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.736+0000 7f14767fc700 1 -- 172.21.15.47:0/1860054086 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f147000b950 con 0x7f1478060eb0 2022-06-29T16:53:06.761 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.736+0000 7f147cd4e700 1 -- 172.21.15.47:0/1860054086 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1478060eb0 msgr2=0x7f1478144e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:06.761 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.736+0000 7f147cd4e700 1 --2- 172.21.15.47:0/1860054086 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1478060eb0 0x7f1478144e70 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f14700032f0 tx=0x7f1470009290).stop 2022-06-29T16:53:06.761 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.736+0000 7f147cd4e700 1 -- 172.21.15.47:0/1860054086 shutdown_connections 2022-06-29T16:53:06.762 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.736+0000 7f147cd4e700 1 --2- 172.21.15.47:0/1860054086 >> v2:172.21.15.47:3300/0 conn(0x7f14780a72f0 0x7f14780b1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:06.762 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.736+0000 7f147cd4e700 1 --2- 172.21.15.47:0/1860054086 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f1478060eb0 0x7f1478144e70 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:06.762 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.736+0000 7f147cd4e700 1 --2- 172.21.15.47:0/1860054086 >> v2:172.21.15.64:3300/0 conn(0x7f14780b5c60 0x7f1478144980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:06.763 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.736+0000 7f147cd4e700 1 -- 172.21.15.47:0/1860054086 >> 172.21.15.47:0/1860054086 conn(0x7f1478062560 msgr2=0x7f1478062940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T16:53:06.763 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.736+0000 7f147cd4e700 1 -- 172.21.15.47:0/1860054086 shutdown_connections 2022-06-29T16:53:06.763 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.736+0000 7f147cd4e700 1 -- 172.21.15.47:0/1860054086 wait complete. 2022-06-29T16:53:06.764 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.740+0000 7f147cd4e700 1 Processor -- start 2022-06-29T16:53:06.764 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.740+0000 7f147cd4e700 1 -- start start 2022-06-29T16:53:06.764 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.740+0000 7f147cd4e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1478060eb0 0x7f14780b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:06.765 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.740+0000 7f147cd4e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f14780a72f0 0x7f147814ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:06.765 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.740+0000 7f147cd4e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14780b5c60 0x7f147814cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:06.766 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.740+0000 7f147cd4e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f147814bef0 con 0x7f14780b5c60 2022-06-29T16:53:06.766 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.740+0000 7f147cd4e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1478069750 con 0x7f1478060eb0 2022-06-29T16:53:06.767 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.740+0000 7f147cd4e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f147813adb0 con 0x7f14780a72f0 2022-06-29T16:53:06.767 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.740+0000 7f1477fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14780b5c60 0x7f147814cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:06.767 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.740+0000 7f1476ffd700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f14780a72f0 0x7f147814ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:06.768 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.740+0000 7f1477fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14780b5c60 0x7f147814cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55800/0 (socket says 172.21.15.47:55800) 2022-06-29T16:53:06.768 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.740+0000 7f14777fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1478060eb0 0x7f14780b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:06.768 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.740+0000 7f1477fff700 1 -- 172.21.15.47:0/850757476 learned_addr learned my addr 172.21.15.47:0/850757476 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T16:53:06.769 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.740+0000 7f1477fff700 1 -- 172.21.15.47:0/850757476 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f14780a72f0 msgr2=0x7f147814ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:06.769 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.740+0000 7f1477fff700 1 --2- 172.21.15.47:0/850757476 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f14780a72f0 0x7f147814ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:06.769 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.740+0000 7f1477fff700 1 -- 172.21.15.47:0/850757476 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1478060eb0 msgr2=0x7f14780b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:06.770 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.740+0000 7f1477fff700 1 --2- 172.21.15.47:0/850757476 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1478060eb0 0x7f14780b20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:06.770 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.740+0000 7f1477fff700 1 -- 172.21.15.47:0/850757476 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f146800b180 con 0x7f14780b5c60 2022-06-29T16:53:06.770 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.740+0000 7f1477fff700 1 --2- 172.21.15.47:0/850757476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14780b5c60 0x7f147814cf10 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 rx=0x7f146800c000 tx=0x7f1468006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:06.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.740+0000 7f1474ff9700 1 -- 172.21.15.47:0/850757476 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1468010780 con 0x7f14780b5c60 2022-06-29T16:53:06.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.740+0000 7f1474ff9700 1 -- 172.21.15.47:0/850757476 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14680108e0 con 0x7f14780b5c60 2022-06-29T16:53:06.771 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.740+0000 7f147cd4e700 1 -- 172.21.15.47:0/850757476 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1470009530 con 0x7f14780b5c60 2022-06-29T16:53:06.772 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.744+0000 7f1474ff9700 1 -- 172.21.15.47:0/850757476 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1468010c40 con 0x7f14780b5c60 2022-06-29T16:53:06.772 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.744+0000 7f147cd4e700 1 -- 172.21.15.47:0/850757476 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f147814bc50 con 0x7f14780b5c60 2022-06-29T16:53:06.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.744+0000 7f147cd4e700 1 -- 172.21.15.47:0/850757476 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f147814a210 con 0x7f14780b5c60 2022-06-29T16:53:06.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.744+0000 7f1474ff9700 1 -- 172.21.15.47:0/850757476 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f146801c460 con 0x7f14780b5c60 2022-06-29T16:53:06.773 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.744+0000 7f1474ff9700 1 --2- 172.21.15.47:0/850757476 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f145c053950 0x7f145c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:06.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.744+0000 7f1474ff9700 1 -- 172.21.15.47:0/850757476 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f146806c1c0 con 0x7f14780b5c60 2022-06-29T16:53:06.774 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.744+0000 7f14777fe700 1 --2- 172.21.15.47:0/850757476 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f145c053950 0x7f145c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:06.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.744+0000 7f14777fe700 1 --2- 172.21.15.47:0/850757476 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f145c053950 0x7f145c055e10 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 rx=0x7f1460006000 tx=0x7f1460006d90).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:06.775 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.748+0000 7f1474ff9700 1 -- 172.21.15.47:0/850757476 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1468043160 con 0x7f14780b5c60 2022-06-29T16:53:06.882 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.876+0000 7f147cd4e700 1 -- 172.21.15.47:0/850757476 --> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f14780573d0 con 0x7f145c053950 2022-06-29T16:53:06.883 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.876+0000 7f1474ff9700 1 -- 172.21.15.47:0/850757476 <== mgr.4102 v2:172.21.15.47:6800/12928 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f14780573d0 con 0x7f145c053950 2022-06-29T16:53:06.886 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.880+0000 7f147cd4e700 1 -- 172.21.15.47:0/850757476 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f145c053950 msgr2=0x7f145c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:06.886 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.880+0000 7f147cd4e700 1 --2- 172.21.15.47:0/850757476 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f145c053950 0x7f145c055e10 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 rx=0x7f1460006000 tx=0x7f1460006d90).stop 2022-06-29T16:53:06.886 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.880+0000 7f147cd4e700 1 -- 172.21.15.47:0/850757476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14780b5c60 msgr2=0x7f147814cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:06.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.880+0000 7f147cd4e700 1 --2- 172.21.15.47:0/850757476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14780b5c60 0x7f147814cf10 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 rx=0x7f146800c000 tx=0x7f1468006df0).stop 2022-06-29T16:53:06.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.880+0000 7f147cd4e700 1 -- 172.21.15.47:0/850757476 shutdown_connections 2022-06-29T16:53:06.887 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.880+0000 7f147cd4e700 1 --2- 172.21.15.47:0/850757476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14780b5c60 0x7f147814cf10 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:06.888 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.880+0000 7f147cd4e700 1 --2- 172.21.15.47:0/850757476 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f14780a72f0 0x7f147814ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:06.888 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.880+0000 7f147cd4e700 1 --2- 172.21.15.47:0/850757476 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f145c053950 0x7f145c055e10 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:06.889 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.880+0000 7f147cd4e700 1 --2- 172.21.15.47:0/850757476 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1478060eb0 0x7f14780b20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:06.889 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.880+0000 7f147cd4e700 1 -- 172.21.15.47:0/850757476 >> 172.21.15.47:0/850757476 conn(0x7f1478062560 msgr2=0x7f1478143aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T16:53:06.889 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.880+0000 7f147cd4e700 1 -- 172.21.15.47:0/850757476 shutdown_connections 2022-06-29T16:53:06.890 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.880+0000 7f147cd4e700 1 -- 172.21.15.47:0/850757476 wait complete. 2022-06-29T16:53:06.890 INFO:teuthology.orchestra.run.smithi047.stderr:instructed osd(s) 7 to deep-scrub 2022-06-29T16:53:06.897 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2022-06-29T16:53:06.999 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.992+0000 7f0fba5c8700 1 Processor -- start 2022-06-29T16:53:07.000 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.992+0000 7f0fba5c8700 1 -- start start 2022-06-29T16:53:07.000 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.992+0000 7f0fba5c8700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0fb40b1260 0x7f0fb405ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:07.001 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.992+0000 7f0fba5c8700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fb40b0740 con 0x7f0fb405f0b0 2022-06-29T16:53:07.001 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.992+0000 7f0fba5c8700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fb40b0970 con 0x7f0fb40b2120 2022-06-29T16:53:07.001 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.992+0000 7f0fba5c8700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb4147460 con 0x7f0fb40b1260 2022-06-29T16:53:07.002 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.992+0000 7f0fb37fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0fb405f0b0 0x7f0fb405f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41358/0 (socket says 172.21.15.47:41358) 2022-06-29T16:53:07.002 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.992+0000 7f0fb37fe700 1 -- 172.21.15.47:0/928197869 learned_addr learned my addr 172.21.15.47:0/928197869 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T16:53:07.002 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.992+0000 7f0fb8b65700 1 --2- 172.21.15.47:0/928197869 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0fb40b1260 0x7f0fb405ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:07.003 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.992+0000 7f0fb2ffd700 1 -- 172.21.15.47:0/928197869 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 760134904 0 0) 0x7f0fb40b0740 con 0x7f0fb405f0b0 2022-06-29T16:53:07.003 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.992+0000 7f0fb2ffd700 1 -- 172.21.15.47:0/928197869 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fa0003260 con 0x7f0fb405f0b0 2022-06-29T16:53:07.003 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.992+0000 7f0fb2ffd700 1 -- 172.21.15.47:0/928197869 <== mon.1 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3465346351 0 0) 0x7f0fb40b0970 con 0x7f0fb40b2120 2022-06-29T16:53:07.004 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.992+0000 7f0fb2ffd700 1 -- 172.21.15.47:0/928197869 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fb40b0740 con 0x7f0fb40b2120 2022-06-29T16:53:07.004 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.992+0000 7f0fb8b65700 1 -- 172.21.15.47:0/928197869 >> v1:172.21.15.64:6789/0 conn(0x7f0fb40b2120 legacy=0x7f0fb4147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:07.004 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.992+0000 7f0fb2ffd700 1 -- 172.21.15.47:0/928197869 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4070035518 0 0) 0x7f0fa0003260 con 0x7f0fb405f0b0 2022-06-29T16:53:07.005 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.992+0000 7f0fb8b65700 1 -- 172.21.15.47:0/928197869 >> v1:172.21.15.47:6789/0 conn(0x7f0fb405f0b0 legacy=0x7f0fb405f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:07.005 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.992+0000 7f0fb8b65700 1 -- 172.21.15.47:0/928197869 --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb40b0c10 con 0x7f0fb40b1260 2022-06-29T16:53:07.005 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.992+0000 7f0fb8b65700 1 --2- 172.21.15.47:0/928197869 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0fb40b1260 0x7f0fb405ffc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f0fb4141a70 tx=0x7f0fac0091d0).ready entity=mon.2 client_cookie=76d90507e8504e77 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:07.006 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.992+0000 7f0fb2ffd700 1 -- 172.21.15.47:0/928197869 <== mon.2 v2:172.21.15.64:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0fa4001660 con 0x7f0fb40b1260 2022-06-29T16:53:07.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.992+0000 7f0fb2ffd700 1 -- 172.21.15.47:0/928197869 <== mon.2 v2:172.21.15.64:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fac00f510 con 0x7f0fb40b1260 2022-06-29T16:53:07.007 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.992+0000 7f0fb2ffd700 1 -- 172.21.15.47:0/928197869 <== mon.2 v2:172.21.15.64:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0fac00f680 con 0x7f0fb40b1260 2022-06-29T16:53:07.011 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.992+0000 7f0fba5c8700 1 -- 172.21.15.47:0/928197869 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0fb40b1260 msgr2=0x7f0fb405ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:07.012 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.992+0000 7f0fba5c8700 1 --2- 172.21.15.47:0/928197869 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0fb40b1260 0x7f0fb405ffc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f0fb4141a70 tx=0x7f0fac0091d0).stop 2022-06-29T16:53:07.012 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.992+0000 7f0fba5c8700 1 -- 172.21.15.47:0/928197869 shutdown_connections 2022-06-29T16:53:07.012 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.996+0000 7f0fba5c8700 1 --2- 172.21.15.47:0/928197869 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0fb40b1260 0x7f0fb405ffc0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:07.013 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.996+0000 7f0fba5c8700 1 -- 172.21.15.47:0/928197869 >> 172.21.15.47:0/928197869 conn(0x7f0fb40626f0 msgr2=0x7f0fb4062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T16:53:07.013 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.996+0000 7f0fba5c8700 1 -- 172.21.15.47:0/928197869 shutdown_connections 2022-06-29T16:53:07.013 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.996+0000 7f0fba5c8700 1 -- 172.21.15.47:0/928197869 wait complete. 2022-06-29T16:53:07.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.996+0000 7f0fba5c8700 1 Processor -- start 2022-06-29T16:53:07.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:06.996+0000 7f0fba5c8700 1 -- start start 2022-06-29T16:53:07.014 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:07.000+0000 7f0fba5c8700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0fb40b1260 0x7f0fb4144230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:07.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:07.000+0000 7f0fba5c8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fb40b2120 0x7f0fb413c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:07.015 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:07.000+0000 7f0fba5c8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fb413cda0 0x7f0fb4141ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:07.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:07.000+0000 7f0fba5c8700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb414d1e0 con 0x7f0fb413cda0 2022-06-29T16:53:07.016 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:07.000+0000 7f0fba5c8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb414c600 con 0x7f0fb40b2120 2022-06-29T16:53:07.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:07.000+0000 7f0fba5c8700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb414cd90 con 0x7f0fb40b1260 2022-06-29T16:53:07.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:07.000+0000 7f0fb8b65700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fb413cda0 0x7f0fb4141ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:07.017 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:07.000+0000 7f0fb8b65700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fb413cda0 0x7f0fb4141ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55814/0 (socket says 172.21.15.47:55814) 2022-06-29T16:53:07.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:07.000+0000 7f0fb8b65700 1 -- 172.21.15.47:0/3993426766 learned_addr learned my addr 172.21.15.47:0/3993426766 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T16:53:07.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:07.000+0000 7f0fb3fff700 1 --2- 172.21.15.47:0/3993426766 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0fb40b1260 0x7f0fb4144230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:07.018 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:07.000+0000 7f0fb37fe700 1 --2- 172.21.15.47:0/3993426766 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fb40b2120 0x7f0fb413c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:07.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:07.000+0000 7f0fb8b65700 1 -- 172.21.15.47:0/3993426766 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0fb40b1260 msgr2=0x7f0fb4144230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:07.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:07.000+0000 7f0fb8b65700 1 --2- 172.21.15.47:0/3993426766 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0fb40b1260 0x7f0fb4144230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:07.019 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:07.000+0000 7f0fb8b65700 1 -- 172.21.15.47:0/3993426766 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fb40b2120 msgr2=0x7f0fb413c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:07.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:07.000+0000 7f0fb8b65700 1 --2- 172.21.15.47:0/3993426766 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fb40b2120 0x7f0fb413c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:07.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:07.000+0000 7f0fb8b65700 1 -- 172.21.15.47:0/3993426766 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fac009470 con 0x7f0fb413cda0 2022-06-29T16:53:07.020 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:07.000+0000 7f0fb8b65700 1 --2- 172.21.15.47:0/3993426766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fb413cda0 0x7f0fb4141ab0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 rx=0x7f0fb4141a70 tx=0x7f0fac016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:07.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:07.000+0000 7f0fb17fa700 1 -- 172.21.15.47:0/3993426766 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0fac011b40 con 0x7f0fb413cda0 2022-06-29T16:53:07.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:07.000+0000 7f0fb17fa700 1 -- 172.21.15.47:0/3993426766 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fac019070 con 0x7f0fb413cda0 2022-06-29T16:53:07.021 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:07.000+0000 7f0fba5c8700 1 -- 172.21.15.47:0/3993426766 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fb414c300 con 0x7f0fb413cda0 2022-06-29T16:53:07.022 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:07.000+0000 7f0fb17fa700 1 -- 172.21.15.47:0/3993426766 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0fac011e90 con 0x7f0fb413cda0 2022-06-29T16:53:07.022 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:07.000+0000 7f0fba5c8700 1 -- 172.21.15.47:0/3993426766 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fb414bd00 con 0x7f0fb413cda0 2022-06-29T16:53:07.022 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:07.004+0000 7f0fba5c8700 1 -- 172.21.15.47:0/3993426766 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fb40573d0 con 0x7f0fb413cda0 2022-06-29T16:53:07.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:07.004+0000 7f0fb17fa700 1 -- 172.21.15.47:0/3993426766 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0fac01e070 con 0x7f0fb413cda0 2022-06-29T16:53:07.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:07.004+0000 7f0fb17fa700 1 --2- 172.21.15.47:0/3993426766 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0f94053aa0 0x7f0f94055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:07.023 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:07.004+0000 7f0fb3fff700 1 --2- 172.21.15.47:0/3993426766 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0f94053aa0 0x7f0f94055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:07.024 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:07.004+0000 7f0fb17fa700 1 -- 172.21.15.47:0/3993426766 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f0fac06cf10 con 0x7f0fb413cda0 2022-06-29T16:53:07.024 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:07.004+0000 7f0fb3fff700 1 --2- 172.21.15.47:0/3993426766 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0f94053aa0 0x7f0f94055f60 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 rx=0x7f0fa800d000 tx=0x7f0fa8007a20).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:07.025 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:07.012+0000 7f0fb17fa700 1 -- 172.21.15.47:0/3993426766 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0fac043fb0 con 0x7f0fb413cda0 2022-06-29T16:53:07.144 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:07.136+0000 7f0fba5c8700 1 -- 172.21.15.47:0/3993426766 --> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0fb41448c0 con 0x7f0f94053aa0 2022-06-29T16:53:07.150 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:07.144+0000 7f0fb17fa700 1 -- 172.21.15.47:0/3993426766 <== mgr.4102 v2:172.21.15.47:6800/12928 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+67459 (secure 0 0 0) 0x7f0fa8015410 con 0x7f0f94053aa0 2022-06-29T16:53:07.152 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T16:53:07.154 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:07.144+0000 7f0fba5c8700 1 -- 172.21.15.47:0/3993426766 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0f94053aa0 msgr2=0x7f0f94055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:07.155 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:07.144+0000 7f0fba5c8700 1 --2- 172.21.15.47:0/3993426766 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0f94053aa0 0x7f0f94055f60 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 rx=0x7f0fa800d000 tx=0x7f0fa8007a20).stop 2022-06-29T16:53:07.155 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:07.144+0000 7f0fba5c8700 1 -- 172.21.15.47:0/3993426766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fb413cda0 msgr2=0x7f0fb4141ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:07.156 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:07.144+0000 7f0fba5c8700 1 --2- 172.21.15.47:0/3993426766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fb413cda0 0x7f0fb4141ab0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 rx=0x7f0fb4141a70 tx=0x7f0fac016000).stop 2022-06-29T16:53:07.157 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:07.144+0000 7f0fba5c8700 1 -- 172.21.15.47:0/3993426766 shutdown_connections 2022-06-29T16:53:07.157 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:07.144+0000 7f0fba5c8700 1 --2- 172.21.15.47:0/3993426766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fb413cda0 0x7f0fb4141ab0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:07.157 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:07.144+0000 7f0fba5c8700 1 --2- 172.21.15.47:0/3993426766 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f0f94053aa0 0x7f0f94055f60 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:07.158 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:07.144+0000 7f0fba5c8700 1 --2- 172.21.15.47:0/3993426766 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fb40b2120 0x7f0fb413c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:07.158 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:07.144+0000 7f0fba5c8700 1 --2- 172.21.15.47:0/3993426766 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f0fb40b1260 0x7f0fb4144230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:07.159 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:07.144+0000 7f0fba5c8700 1 -- 172.21.15.47:0/3993426766 >> 172.21.15.47:0/3993426766 conn(0x7f0fb40626f0 msgr2=0x7f0fb40ab650 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T16:53:07.159 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:07.148+0000 7f0fba5c8700 1 -- 172.21.15.47:0/3993426766 shutdown_connections 2022-06-29T16:53:07.159 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:07.148+0000 7f0fba5c8700 1 -- 172.21.15.47:0/3993426766 wait complete. 2022-06-29T16:53:07.160 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2022-06-29T16:53:07.165 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":2280,"stamp":"2022-06-29T16:53:06.121949+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":8405056,"kb_used_data":16384,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746569664,"statfs":{"total":773094113280,"available":764487335936,"internally_reserved":8589934592,"allocated":16777216,"data_stored":8168800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003105"},"pg_stats":[{"pgid":"1.0","version":"16'2","reported_seq":143,"reported_epoch":164,"state":"active+clean","last_fresh":"2022-06-29T15:52:30.026079+0000","last_change":"2022-06-29T15:42:22.794564+0000","last_active":"2022-06-29T15:52:30.026079+0000","last_peered":"2022-06-29T15:52:30.026079+0000","last_clean":"2022-06-29T15:52:30.026079+0000","last_became_active":"2022-06-29T15:42:22.793734+0000","last_became_peered":"2022-06-29T15:42:22.793734+0000","last_unstale":"2022-06-29T15:52:30.026079+0000","last_undegraded":"2022-06-29T15:52:30.026079+0000","last_fullsized":"2022-06-29T15:52:30.026079+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:17.908903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:17.908903+0000","last_clean_scrub_stamp":"2022-06-29T15:42:17.908903+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":137,"reported_epoch":164,"state":"active+clean","last_fresh":"2022-06-29T15:52:30.437350+0000","last_change":"2022-06-29T15:42:27.002337+0000","last_active":"2022-06-29T15:52:30.437350+0000","last_peered":"2022-06-29T15:52:30.437350+0000","last_clean":"2022-06-29T15:52:30.437350+0000","last_became_active":"2022-06-29T15:42:27.001981+0000","last_became_peered":"2022-06-29T15:42:27.001981+0000","last_unstale":"2022-06-29T15:52:30.437350+0000","last_undegraded":"2022-06-29T15:52:30.437350+0000","last_fullsized":"2022-06-29T15:52:30.437350+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_clean_scrub_stamp":"2022-06-29T15:42:25.967805+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":138,"reported_epoch":164,"state":"active+clean","last_fresh":"2022-06-29T15:52:30.025975+0000","last_change":"2022-06-29T15:42:27.000351+0000","last_active":"2022-06-29T15:52:30.025975+0000","last_peered":"2022-06-29T15:52:30.025975+0000","last_clean":"2022-06-29T15:52:30.025975+0000","last_became_active":"2022-06-29T15:42:26.999820+0000","last_became_peered":"2022-06-29T15:42:26.999820+0000","last_unstale":"2022-06-29T15:52:30.025975+0000","last_undegraded":"2022-06-29T15:52:30.025975+0000","last_fullsized":"2022-06-29T15:52:30.025975+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_clean_scrub_stamp":"2022-06-29T15:42:25.967805+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":133,"reported_epoch":165,"state":"active+clean","last_fresh":"2022-06-29T16:53:04.956047+0000","last_change":"2022-06-29T16:53:04.956192+0000","last_active":"2022-06-29T16:53:04.956047+0000","last_peered":"2022-06-29T16:53:04.956047+0000","last_clean":"2022-06-29T16:53:04.956047+0000","last_became_active":"2022-06-29T15:42:27.554328+0000","last_became_peered":"2022-06-29T15:42:27.554328+0000","last_unstale":"2022-06-29T16:53:04.956047+0000","last_undegraded":"2022-06-29T16:53:04.956047+0000","last_fullsized":"2022-06-29T16:53:04.956047+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T16:53:04.956047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T16:53:04.956047+0000","last_clean_scrub_stamp":"2022-06-29T16:53:04.956047+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":137,"reported_epoch":164,"state":"active+clean","last_fresh":"2022-06-29T15:52:30.437346+0000","last_change":"2022-06-29T15:42:27.001536+0000","last_active":"2022-06-29T15:52:30.437346+0000","last_peered":"2022-06-29T15:52:30.437346+0000","last_clean":"2022-06-29T15:52:30.437346+0000","last_became_active":"2022-06-29T15:42:27.001089+0000","last_became_peered":"2022-06-29T15:42:27.001089+0000","last_unstale":"2022-06-29T15:52:30.437346+0000","last_undegraded":"2022-06-29T15:52:30.437346+0000","last_fullsized":"2022-06-29T15:52:30.437346+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_clean_scrub_stamp":"2022-06-29T15:42:25.967805+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":137,"reported_epoch":164,"state":"active+clean","last_fresh":"2022-06-29T15:52:30.439563+0000","last_change":"2022-06-29T15:42:27.721658+0000","last_active":"2022-06-29T15:52:30.439563+0000","last_peered":"2022-06-29T15:52:30.439563+0000","last_clean":"2022-06-29T15:52:30.439563+0000","last_became_active":"2022-06-29T15:42:27.720770+0000","last_became_peered":"2022-06-29T15:42:27.720770+0000","last_unstale":"2022-06-29T15:52:30.439563+0000","last_undegraded":"2022-06-29T15:52:30.439563+0000","last_fullsized":"2022-06-29T15:52:30.439563+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_clean_scrub_stamp":"2022-06-29T15:42:25.967805+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":138,"reported_epoch":164,"state":"active+clean","last_fresh":"2022-06-29T15:52:30.026620+0000","last_change":"2022-06-29T15:42:27.723852+0000","last_active":"2022-06-29T15:52:30.026620+0000","last_peered":"2022-06-29T15:52:30.026620+0000","last_clean":"2022-06-29T15:52:30.026620+0000","last_became_active":"2022-06-29T15:42:27.722958+0000","last_became_peered":"2022-06-29T15:42:27.722958+0000","last_unstale":"2022-06-29T15:52:30.026620+0000","last_undegraded":"2022-06-29T15:52:30.026620+0000","last_fullsized":"2022-06-29T15:52:30.026620+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_clean_scrub_stamp":"2022-06-29T15:42:25.967805+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":137,"reported_epoch":164,"state":"active+clean","last_fresh":"2022-06-29T15:52:30.439978+0000","last_change":"2022-06-29T15:42:26.998189+0000","last_active":"2022-06-29T15:52:30.439978+0000","last_peered":"2022-06-29T15:52:30.439978+0000","last_clean":"2022-06-29T15:52:30.439978+0000","last_became_active":"2022-06-29T15:42:26.997677+0000","last_became_peered":"2022-06-29T15:42:26.997677+0000","last_unstale":"2022-06-29T15:52:30.439978+0000","last_undegraded":"2022-06-29T15:52:30.439978+0000","last_fullsized":"2022-06-29T15:52:30.439978+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_clean_scrub_stamp":"2022-06-29T15:42:25.967805+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":133,"reported_epoch":164,"state":"active+clean","last_fresh":"2022-06-29T15:52:30.437874+0000","last_change":"2022-06-29T15:42:26.999647+0000","last_active":"2022-06-29T15:52:30.437874+0000","last_peered":"2022-06-29T15:52:30.437874+0000","last_clean":"2022-06-29T15:52:30.437874+0000","last_became_active":"2022-06-29T15:42:26.999119+0000","last_became_peered":"2022-06-29T15:42:26.999119+0000","last_unstale":"2022-06-29T15:52:30.437874+0000","last_undegraded":"2022-06-29T15:52:30.437874+0000","last_fullsized":"2022-06-29T15:52:30.437874+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T15:42:25.967805+0000","last_clean_scrub_stamp":"2022-06-29T15:42:25.967805+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":14,"seq":60129542995,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050632,"kb_used_data":2048,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321208,"statfs":{"total":96636764160,"available":95560916992,"internally_reserved":1073741824,"allocated":2097152,"data_stored":1021100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.84099999999999997,"15min":0.84699999999999998},"min":{"1min":0.58899999999999997,"5min":0.58099999999999996,"15min":0.54900000000000004},"max":{"1min":1.125,"5min":1.165,"15min":1.335},"last":1.091},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.68700000000000006,"5min":0.57199999999999995,"15min":0.49399999999999999},"max":{"1min":1.1479999999999999,"5min":1.151,"15min":1.282},"last":0.77900000000000003}]},{"osd":1,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.83199999999999996,"15min":0.83099999999999996},"min":{"1min":0.626,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0580000000000001,"5min":1.224,"15min":1.224},"last":1.1020000000000001},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.871,"15min":0.879},"min":{"1min":0.64500000000000002,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.403},"last":0.91800000000000004}]},{"osd":2,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.89400000000000002,"15min":0.89300000000000002},"min":{"1min":0.69699999999999995,"5min":0.60599999999999998,"15min":0.56799999999999995},"max":{"1min":1.107,"5min":1.194,"15min":1.2290000000000001},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.86599999999999999,"15min":0.88500000000000001},"min":{"1min":0.65400000000000003,"5min":0.65200000000000002,"15min":0.63800000000000001},"max":{"1min":1.071,"5min":1.2130000000000001,"15min":1.276},"last":0.72099999999999997}]},{"osd":3,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.873,"15min":0.876},"min":{"1min":0.626,"5min":0.60799999999999998,"15min":0.58699999999999997},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2929999999999999},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.042,"5min":1.0269999999999999,"15min":1.0209999999999999},"min":{"1min":0.86399999999999999,"5min":0.753,"15min":0.71999999999999997},"max":{"1min":1.212,"5min":1.25,"15min":1.3089999999999999},"last":1.0089999999999999}]},{"osd":4,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.94499999999999995,"15min":0.89800000000000002},"min":{"1min":0.65500000000000003,"5min":0.58599999999999997,"15min":0.53100000000000003},"max":{"1min":1.1930000000000001,"5min":1.337,"15min":1.337},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.92900000000000005,"15min":0.89600000000000002},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.501},"max":{"1min":1.1879999999999999,"5min":1.369,"15min":1.369},"last":0.93200000000000005}]},{"osd":6,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.95799999999999996,"15min":0.92900000000000005},"min":{"1min":0.63400000000000001,"5min":0.61199999999999999,"15min":0.59999999999999998},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.86899999999999999},{"interface":"front","average":{"1min":1.002,"5min":1.044,"15min":1.02},"min":{"1min":0.71599999999999997,"5min":0.623,"15min":0.54200000000000004},"max":{"1min":1.3080000000000001,"5min":1.4610000000000001,"15min":1.4770000000000001},"last":0.97799999999999998}]},{"osd":7,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":1.018,"15min":0.98399999999999999},"min":{"1min":0.75800000000000001,"5min":0.63,"15min":0.63},"max":{"1min":1.2969999999999999,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.1080000000000001,"15min":1.079},"min":{"1min":0.79600000000000004,"5min":0.76600000000000001,"15min":0.69999999999999996},"max":{"1min":1.2869999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":1.1859999999999999}]}]},{"osd":7,"up_from":14,"seq":60129542995,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050632,"kb_used_data":2048,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321208,"statfs":{"total":96636764160,"available":95560916992,"internally_reserved":1073741824,"allocated":2097152,"data_stored":1021100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 29 16:52:42 2022","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.877,"15min":0.88},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.48699999999999999},"max":{"1min":1.1679999999999999,"5min":1.21,"15min":1.2410000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.58199999999999996},"max":{"1min":1.1319999999999999,"5min":1.171,"15min":1.2569999999999999},"last":0.90800000000000003}]},{"osd":1,"last update":"Wed Jun 29 16:52:42 2022","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.79600000000000004,"15min":0.79900000000000004},"min":{"1min":0.59599999999999997,"5min":0.57899999999999996,"15min":0.49299999999999999},"max":{"1min":0.997,"5min":1.1850000000000001,"15min":1.202},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.82199999999999995,"15min":0.82799999999999996},"min":{"1min":0.60399999999999998,"5min":0.59499999999999997,"15min":0.58199999999999996},"max":{"1min":1.0780000000000001,"5min":1.161,"15min":1.226},"last":1.004}]},{"osd":2,"last update":"Wed Jun 29 16:52:42 2022","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.88400000000000001,"15min":0.88800000000000001},"min":{"1min":0.63900000000000001,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.04,"5min":1.1679999999999999,"15min":1.333},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.94899999999999995,"15min":0.94599999999999995},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.55900000000000005},"max":{"1min":1.1639999999999999,"5min":1.2250000000000001,"15min":1.2569999999999999},"last":0.97999999999999998}]},{"osd":3,"last update":"Wed Jun 29 16:52:42 2022","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.90700000000000003,"15min":0.90000000000000002},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.095,"5min":1.224,"15min":1.276},"last":0.876},{"interface":"front","average":{"1min":1.032,"5min":0.97099999999999997,"15min":0.97799999999999998},"min":{"1min":0.87,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.2050000000000001,"5min":1.347,"15min":1.347},"last":1.081}]},{"osd":4,"last update":"Wed Jun 29 16:52:42 2022","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.80100000000000005,"15min":0.82799999999999996},"min":{"1min":0.61499999999999999,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.016,"5min":1.3049999999999999,"15min":1.349},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.66200000000000003,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.125,"5min":1.226,"15min":1.246},"last":0.94599999999999995}]},{"osd":5,"last update":"Wed Jun 29 16:52:42 2022","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.94799999999999995,"15min":0.94099999999999995},"min":{"1min":0.67900000000000005,"5min":0.56699999999999995,"15min":0.56599999999999995},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.306},"last":1.1830000000000001},{"interface":"front","average":{"1min":0.93799999999999994,"5min":1.0009999999999999,"15min":0.98499999999999999},"min":{"1min":0.621,"5min":0.621,"15min":0.59299999999999997},"max":{"1min":1.151,"5min":1.2849999999999999,"15min":1.3200000000000001},"last":1.129}]},{"osd":6,"last update":"Wed Jun 29 16:52:42 2022","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.0629999999999999,"15min":1.044},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68400000000000005},"max":{"1min":1.244,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":1.2310000000000001},{"interface":"front","average":{"1min":0.99199999999999999,"5min":1.028,"15min":1.0169999999999999},"min":{"1min":0.74399999999999999,"5min":0.70499999999999996,"15min":0.65700000000000003},"max":{"1min":1.23,"5min":1.52,"15min":1.52},"last":1.028}]}]},{"osd":6,"up_from":14,"seq":60129542996,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050632,"kb_used_data":2048,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321208,"statfs":{"total":96636764160,"available":95560916992,"internally_reserved":1073741824,"allocated":2097152,"data_stored":1021100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 29 16:52:51 2022","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.83499999999999996,"15min":0.85799999999999998},"min":{"1min":0.57399999999999995,"5min":0.54500000000000004,"15min":0.53700000000000003},"max":{"1min":1.0369999999999999,"5min":1.1850000000000001,"15min":1.522},"last":1.107},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.90800000000000003,"15min":0.93700000000000006},"min":{"1min":0.68300000000000005,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.135,"5min":1.349,"15min":1.5469999999999999},"last":0.83999999999999997}]},{"osd":1,"last update":"Wed Jun 29 16:52:51 2022","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.78800000000000003,"15min":0.79000000000000004},"min":{"1min":0.58799999999999997,"5min":0.54700000000000004,"15min":0.53200000000000003},"max":{"1min":1.0149999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79500000000000004,"15min":0.81100000000000005},"min":{"1min":0.629,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.038,"5min":1.1080000000000001,"15min":1.359},"last":0.65300000000000002}]},{"osd":2,"last update":"Wed Jun 29 16:52:51 2022","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.88600000000000001,"15min":0.88300000000000001},"min":{"1min":0.67300000000000004,"5min":0.59899999999999998,"15min":0.58799999999999997},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93000000000000005,"15min":0.91800000000000004},"min":{"1min":0.72299999999999998,"5min":0.60299999999999998,"15min":0.58199999999999996},"max":{"1min":1.1379999999999999,"5min":1.2270000000000001,"15min":1.3680000000000001},"last":1.2070000000000001}]},{"osd":3,"last update":"Wed Jun 29 16:52:51 2022","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.879,"15min":0.875},"min":{"1min":0.629,"5min":0.621,"15min":0.60899999999999999},"max":{"1min":1.127,"5min":1.23,"15min":1.232},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.97799999999999998,"15min":0.97599999999999998},"min":{"1min":0.80100000000000005,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.1859999999999999,"5min":1.22,"15min":1.278},"last":0.94999999999999996}]},{"osd":4,"last update":"Wed Jun 29 16:52:51 2022","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.83899999999999997,"15min":0.84199999999999997},"min":{"1min":0.60899999999999999,"5min":0.58799999999999997,"15min":0.54700000000000004},"max":{"1min":0.98699999999999999,"5min":1.27,"15min":1.27},"last":1.169},{"interface":"front","average":{"1min":1.034,"5min":0.97199999999999998,"15min":0.93200000000000005},"min":{"1min":0.79000000000000004,"5min":0.64500000000000002,"15min":0.57199999999999995},"max":{"1min":1.1839999999999999,"5min":1.298,"15min":1.3420000000000001},"last":0.98599999999999999}]},{"osd":5,"last update":"Wed Jun 29 16:52:51 2022","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93100000000000005,"15min":0.92000000000000004},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.58499999999999996},"max":{"1min":1.198,"5min":1.216,"15min":1.385},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.999,"15min":0.98399999999999999},"min":{"1min":0.83199999999999996,"5min":0.67100000000000004,"15min":0.63500000000000001},"max":{"1min":1.2989999999999999,"5min":1.3320000000000001,"15min":1.423},"last":1.244}]},{"osd":7,"last update":"Wed Jun 29 16:52:51 2022","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.0449999999999999,"15min":1.034},"min":{"1min":0.72699999999999998,"5min":0.71099999999999997,"15min":0.63300000000000001},"max":{"1min":1.157,"5min":1.391,"15min":1.4019999999999999},"last":1.044},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.107,"15min":1.097},"min":{"1min":0.97599999999999998,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.581},"last":1.0669999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542995,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1050632,"kb_used_data":2048,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321208,"statfs":{"total":96636764160,"available":95560916992,"internally_reserved":1073741824,"allocated":2097152,"data_stored":1021100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 29 16:52:23 2022","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.86199999999999999,"15min":0.86099999999999999},"min":{"1min":0.51200000000000001,"5min":0.505,"15min":0.505},"max":{"1min":1.0229999999999999,"5min":1.135,"15min":1.228},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.95099999999999996,"15min":0.93799999999999994},"min":{"1min":0.72499999999999998,"5min":0.65100000000000002,"15min":0.63600000000000001},"max":{"1min":1.089,"5min":1.181,"15min":1.2909999999999999},"last":1.103}]},{"osd":1,"last update":"Wed Jun 29 16:52:23 2022","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.81999999999999995,"15min":0.82799999999999996},"min":{"1min":0.59099999999999997,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.202},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.83499999999999996,"15min":0.83299999999999996},"min":{"1min":0.67400000000000004,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.145,"5min":1.1699999999999999,"15min":1.21},"last":0.64600000000000002}]},{"osd":2,"last update":"Wed Jun 29 16:52:23 2022","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.86499999999999999,"15min":0.871},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.85499999999999998,"15min":0.84599999999999997},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.232},"last":1.0129999999999999}]},{"osd":3,"last update":"Wed Jun 29 16:52:23 2022","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.91200000000000003,"15min":0.91800000000000004},"min":{"1min":0.68799999999999994,"5min":0.61799999999999999,"15min":0.60099999999999998},"max":{"1min":1.111,"5min":1.157,"15min":1.2470000000000001},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.014,"5min":1.004,"15min":1.004},"min":{"1min":0.80500000000000005,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.222,"5min":1.2430000000000001,"15min":1.52},"last":0.92200000000000004}]},{"osd":5,"last update":"Wed Jun 29 16:52:23 2022","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.80500000000000005,"15min":0.80800000000000005},"min":{"1min":0.56799999999999995,"5min":0.56000000000000005,"15min":0.52000000000000002},"max":{"1min":1.077,"5min":1.179,"15min":1.179},"last":1.131},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.89600000000000002,"15min":0.91400000000000003},"min":{"1min":0.60899999999999999,"5min":0.58399999999999996,"15min":0.54400000000000004},"max":{"1min":1.2090000000000001,"5min":1.2190000000000001,"15min":1.304},"last":1.1679999999999999}]},{"osd":6,"last update":"Wed Jun 29 16:52:23 2022","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.95099999999999996,"15min":0.94999999999999996},"min":{"1min":0.66000000000000003,"5min":0.63,"15min":0.58299999999999996},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.4390000000000001},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.028,"5min":1.004,"15min":1.0109999999999999},"min":{"1min":0.73599999999999999,"5min":0.72299999999999998,"15min":0.61799999999999999},"max":{"1min":1.1839999999999999,"5min":1.264,"15min":1.3009999999999999},"last":1.1779999999999999}]},{"osd":7,"last update":"Wed Jun 29 16:52:23 2022","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.0029999999999999,"15min":1.0109999999999999},"min":{"1min":0.81000000000000005,"5min":0.66000000000000003,"15min":0.61599999999999999},"max":{"1min":1.2490000000000001,"5min":1.425,"15min":1.4930000000000001},"last":1.21},{"interface":"front","average":{"1min":1.054,"5min":1.04,"15min":1.042},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.496},"max":{"1min":1.663,"5min":1.663,"15min":1.663},"last":1.151}]}]},{"osd":3,"up_from":14,"seq":60129542995,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050632,"kb_used_data":2048,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321208,"statfs":{"total":96636764160,"available":95560916992,"internally_reserved":1073741824,"allocated":2097152,"data_stored":1021100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63200000000000001,"15min":0.64100000000000001},"min":{"1min":0.5,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.80400000000000005,"5min":1.075,"15min":1.103},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.755,"5min":0.78300000000000003,"15min":0.78000000000000003},"min":{"1min":0.55600000000000005,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.92600000000000005,"5min":1.194,"15min":1.232},"last":0.69799999999999995}]},{"osd":1,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.65300000000000002,"15min":0.64800000000000002},"min":{"1min":0.51200000000000001,"5min":0.48099999999999998,"15min":0.46899999999999997},"max":{"1min":0.96799999999999997,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77600000000000002,"15min":0.77400000000000002},"min":{"1min":0.53900000000000003,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.1220000000000001,"5min":1.288,"15min":1.288},"last":0.58299999999999996}]},{"osd":2,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.70599999999999996,"15min":0.68600000000000005},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.97899999999999998,"5min":1.365,"15min":1.365},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.85699999999999998,"15min":0.84099999999999997},"min":{"1min":0.68999999999999995,"5min":0.60299999999999998,"15min":0.57399999999999995},"max":{"1min":1.046,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.75700000000000001}]},{"osd":4,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.94199999999999995,"15min":0.93999999999999995},"min":{"1min":0.71599999999999997,"5min":0.71399999999999997,"15min":0.67600000000000005},"max":{"1min":1.1240000000000001,"5min":1.556,"15min":1.556},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.91800000000000004,"15min":0.91100000000000003},"min":{"1min":0.71299999999999997,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.0569999999999999,"5min":1.3169999999999999,"15min":1.3420000000000001},"last":0.82599999999999996}]},{"osd":5,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.96399999999999997,"15min":0.93999999999999995},"min":{"1min":0.72199999999999998,"5min":0.69999999999999996,"15min":0.68500000000000005},"max":{"1min":1.2390000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.024,"15min":1.016},"min":{"1min":0.82399999999999995,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.1970000000000001,"5min":1.3640000000000001,"15min":1.454},"last":0.93899999999999995}]},{"osd":6,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":1.0329999999999999,"15min":1.0229999999999999},"min":{"1min":0.745,"5min":0.73499999999999999,"15min":0.67200000000000004},"max":{"1min":1.337,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.028,"5min":1.0289999999999999,"15min":1.0249999999999999},"min":{"1min":0.77000000000000002,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.2849999999999999,"5min":1.496,"15min":1.508},"last":0.88800000000000001}]},{"osd":7,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.82699999999999996,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4970000000000001},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.1379999999999999,"15min":1.1240000000000001},"min":{"1min":0.84199999999999997,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.3999999999999999,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":1.153}]}]},{"osd":2,"up_from":14,"seq":60129542995,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050632,"kb_used_data":2048,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321208,"statfs":{"total":96636764160,"available":95560916992,"internally_reserved":1073741824,"allocated":2097152,"data_stored":1021100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.628,"15min":0.624},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.42099999999999999},"max":{"1min":1.03,"5min":1.137,"15min":1.256},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.75,"5min":0.72699999999999998,"15min":0.70399999999999996},"min":{"1min":0.53600000000000003,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.099,"5min":1.151,"15min":1.151},"last":0.53700000000000003}]},{"osd":1,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.48399999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.91600000000000004,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.751,"15min":0.76500000000000001},"min":{"1min":0.54000000000000004,"5min":0.502,"15min":0.502},"max":{"1min":0.92100000000000004,"5min":1.131,"15min":1.131},"last":0.70999999999999996}]},{"osd":3,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.78500000000000003,"15min":0.80800000000000005},"min":{"1min":0.58499999999999996,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0900000000000001,"5min":1.133,"15min":1.1930000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.84399999999999997,"15min":0.83399999999999996},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.55600000000000005},"max":{"1min":1.1040000000000001,"5min":1.155,"15min":1.1990000000000001},"last":0.91300000000000003}]},{"osd":4,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.877,"15min":0.88400000000000001},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.2390000000000001,"5min":1.2509999999999999,"15min":1.323},"last":1.0309999999999999},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.92600000000000005,"15min":0.93100000000000005},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":1.131}]},{"osd":5,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":1.0209999999999999,"15min":1.0129999999999999},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.478,"5min":1.478,"15min":1.5169999999999999},"last":1.325},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.98099999999999998,"15min":0.97699999999999998},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.64500000000000002},"max":{"1min":1.2130000000000001,"5min":1.3540000000000001,"15min":1.5029999999999999},"last":0.95199999999999996}]},{"osd":6,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.032,"15min":1.0289999999999999},"min":{"1min":0.81000000000000005,"5min":0.752,"15min":0.69999999999999996},"max":{"1min":1.3260000000000001,"5min":1.464,"15min":1.53},"last":1.194},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0980000000000001,"15min":1.093},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.75900000000000001},"max":{"1min":1.413,"5min":1.516,"15min":1.5669999999999999},"last":1.2410000000000001}]},{"osd":7,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.0700000000000001,"15min":1.0669999999999999},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.123,"5min":1.109,"15min":1.109},"min":{"1min":0.94399999999999995,"5min":0.86299999999999999,"15min":0.85099999999999998},"max":{"1min":1.405,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":1.278}]}]},{"osd":1,"up_from":14,"seq":60129542994,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050632,"kb_used_data":2048,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321208,"statfs":{"total":96636764160,"available":95560916992,"internally_reserved":1073741824,"allocated":2097152,"data_stored":1021100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 29 16:52:38 2022","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.622,"15min":0.63600000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.47199999999999998},"max":{"1min":0.75600000000000001,"5min":1.024,"15min":1.1000000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70899999999999996,"15min":0.70199999999999996},"min":{"1min":0.54100000000000004,"5min":0.47999999999999998,"15min":0.47299999999999998},"max":{"1min":0.93000000000000005,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.752}]},{"osd":2,"last update":"Wed Jun 29 16:52:38 2022","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.65100000000000002,"15min":0.64800000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.46700000000000003},"max":{"1min":0.80000000000000004,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76300000000000001,"15min":0.76800000000000002},"min":{"1min":0.61699999999999999,"5min":0.54600000000000004,"15min":0.51700000000000002},"max":{"1min":0.98699999999999999,"5min":1.22,"15min":1.22},"last":0.96299999999999997}]},{"osd":3,"last update":"Wed Jun 29 16:52:38 2022","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.82399999999999995,"15min":0.84699999999999998},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.56100000000000005},"max":{"1min":1.01,"5min":1.109,"15min":1.157},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78900000000000003,"15min":0.77700000000000002},"min":{"1min":0.60599999999999998,"5min":0.52900000000000003,"15min":0.47699999999999998},"max":{"1min":0.95699999999999996,"5min":1.788,"15min":1.788},"last":0.65700000000000003}]},{"osd":4,"last update":"Wed Jun 29 16:52:38 2022","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.877,"15min":0.873},"min":{"1min":0.69899999999999995,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.248,"5min":1.288,"15min":1.4610000000000001},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.995,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.81000000000000005,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.24,"5min":1.405,"15min":1.4330000000000001},"last":1.0609999999999999}]},{"osd":5,"last update":"Wed Jun 29 16:52:38 2022","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.93899999999999995,"15min":0.96899999999999997},"min":{"1min":0.78100000000000003,"5min":0.71299999999999997,"15min":0.66700000000000004},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.4279999999999999},"last":1.278},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.95899999999999996,"15min":0.96599999999999997},"min":{"1min":0.75,"5min":0.72799999999999998,"15min":0.67300000000000004},"max":{"1min":1.167,"5min":1.357,"15min":1.3959999999999999},"last":0.99199999999999999}]},{"osd":6,"last update":"Wed Jun 29 16:52:38 2022","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.0209999999999999,"15min":1.0509999999999999},"min":{"1min":0.77300000000000002,"5min":0.76900000000000002,"15min":0.70499999999999996},"max":{"1min":1.375,"5min":1.375,"15min":1.516},"last":1.171},{"interface":"front","average":{"1min":1.014,"5min":1.0389999999999999,"15min":1.054},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.343,"5min":1.444,"15min":1.5649999999999999},"last":1.0369999999999999}]},{"osd":7,"last update":"Wed Jun 29 16:52:38 2022","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.0880000000000001,"15min":1.1020000000000001},"min":{"1min":0.77200000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.3149999999999999,"5min":1.4550000000000001,"15min":1.6100000000000001},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.0920000000000001,"15min":1.1020000000000001},"min":{"1min":0.92900000000000005,"5min":0.82099999999999995,"15min":0.72699999999999998},"max":{"1min":1.413,"5min":1.413,"15min":1.603},"last":1.238}]}]},{"osd":0,"up_from":14,"seq":60129542995,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050632,"kb_used_data":2048,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321208,"statfs":{"total":96636764160,"available":95560916992,"internally_reserved":1073741824,"allocated":2097152,"data_stored":1021100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jun 29 16:52:42 2022","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.64900000000000002,"15min":0.63500000000000001},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.45200000000000001},"max":{"1min":0.79100000000000004,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.64900000000000002,"15min":0.67100000000000004},"min":{"1min":0.52300000000000002,"5min":0.46800000000000003,"15min":0.45300000000000001},"max":{"1min":0.84499999999999997,"5min":0.98299999999999998,"15min":1.1259999999999999},"last":0.53500000000000003}]},{"osd":2,"last update":"Wed Jun 29 16:52:42 2022","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.71999999999999997,"15min":0.73499999999999999},"min":{"1min":0.51300000000000001,"5min":0.499,"15min":0.46800000000000003},"max":{"1min":1.0429999999999999,"5min":1.0549999999999999,"15min":1.2190000000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.80700000000000005,"15min":0.82599999999999996},"min":{"1min":0.67500000000000004,"5min":0.60499999999999998,"15min":0.59699999999999998},"max":{"1min":1.0329999999999999,"5min":1.085,"15min":1.143},"last":0.443}]},{"osd":3,"last update":"Wed Jun 29 16:52:42 2022","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.71299999999999997,"15min":0.70099999999999996},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.48899999999999999},"max":{"1min":0.97999999999999998,"5min":1.117,"15min":1.147},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.80600000000000005,"15min":0.82599999999999996},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.028,"5min":1.145,"15min":1.2090000000000001},"last":0.67800000000000005}]},{"osd":4,"last update":"Wed Jun 29 16:52:42 2022","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.91400000000000003,"15min":0.92400000000000004},"min":{"1min":0.73999999999999999,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.077,"5min":1.2769999999999999,"15min":1.3759999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.88600000000000001,"15min":0.91100000000000003},"min":{"1min":0.68600000000000005,"5min":0.67700000000000005,"15min":0.66600000000000004},"max":{"1min":1.1699999999999999,"5min":1.381,"15min":1.5},"last":1.153}]},{"osd":5,"last update":"Wed Jun 29 16:52:42 2022","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.91400000000000003,"15min":0.93100000000000005},"min":{"1min":0.70099999999999996,"5min":0.66600000000000004,"15min":0.66100000000000003},"max":{"1min":1.266,"5min":1.4419999999999999,"15min":1.6040000000000001},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.032,"5min":1.0149999999999999,"15min":1.024},"min":{"1min":0.80300000000000005,"5min":0.752,"15min":0.67900000000000005},"max":{"1min":1.2210000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.88300000000000001}]},{"osd":6,"last update":"Wed Jun 29 16:52:42 2022","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.0309999999999999,"15min":1.052},"min":{"1min":0.76000000000000001,"5min":0.749,"15min":0.73599999999999999},"max":{"1min":1.2949999999999999,"5min":1.458,"15min":1.524},"last":1.004},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.024,"15min":1.0509999999999999},"min":{"1min":0.82499999999999996,"5min":0.78700000000000003,"15min":0.70799999999999996},"max":{"1min":1.163,"5min":1.502,"15min":1.552},"last":0.92500000000000004}]},{"osd":7,"last update":"Wed Jun 29 16:52:42 2022","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.0680000000000001,"15min":1.0840000000000001},"min":{"1min":0.85999999999999999,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.2270000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.069,"15min":1.077},"min":{"1min":0.876,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.466,"5min":1.486,"15min":1.5960000000000001},"last":1.2110000000000001}]}]}]}} 2022-06-29T16:53:07.172 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2022-06-29T15:42:17.908903+0000 time.struct_time(tm_year=2022, tm_mon=6, tm_mday=29, tm_hour=15, tm_min=42, tm_sec=17, tm_wday=2, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2022, tm_mon=6, tm_mday=29, tm_hour=16, tm_min=53, tm_sec=2, tm_wday=2, tm_yday=180, tm_isdst=0) 2022-06-29T16:53:07.173 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2022-06-29T15:42:25.967805+0000 time.struct_time(tm_year=2022, tm_mon=6, tm_mday=29, tm_hour=15, tm_min=42, tm_sec=25, tm_wday=2, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2022, tm_mon=6, tm_mday=29, tm_hour=16, tm_min=53, tm_sec=2, tm_wday=2, tm_yday=180, tm_isdst=0) 2022-06-29T16:53:07.174 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2022-06-29T15:42:25.967805+0000 time.struct_time(tm_year=2022, tm_mon=6, tm_mday=29, tm_hour=15, tm_min=42, tm_sec=25, tm_wday=2, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2022, tm_mon=6, tm_mday=29, tm_hour=16, tm_min=53, tm_sec=2, tm_wday=2, tm_yday=180, tm_isdst=0) 2022-06-29T16:53:07.174 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2022-06-29T15:42:25.967805+0000 time.struct_time(tm_year=2022, tm_mon=6, tm_mday=29, tm_hour=15, tm_min=42, tm_sec=25, tm_wday=2, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2022, tm_mon=6, tm_mday=29, tm_hour=16, tm_min=53, tm_sec=2, tm_wday=2, tm_yday=180, tm_isdst=0) 2022-06-29T16:53:07.175 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2022-06-29T15:42:25.967805+0000 time.struct_time(tm_year=2022, tm_mon=6, tm_mday=29, tm_hour=15, tm_min=42, tm_sec=25, tm_wday=2, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2022, tm_mon=6, tm_mday=29, tm_hour=16, tm_min=53, tm_sec=2, tm_wday=2, tm_yday=180, tm_isdst=0) 2022-06-29T16:53:07.176 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2022-06-29T15:42:25.967805+0000 time.struct_time(tm_year=2022, tm_mon=6, tm_mday=29, tm_hour=15, tm_min=42, tm_sec=25, tm_wday=2, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2022, tm_mon=6, tm_mday=29, tm_hour=16, tm_min=53, tm_sec=2, tm_wday=2, tm_yday=180, tm_isdst=0) 2022-06-29T16:53:07.177 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2022-06-29T15:42:25.967805+0000 time.struct_time(tm_year=2022, tm_mon=6, tm_mday=29, tm_hour=15, tm_min=42, tm_sec=25, tm_wday=2, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2022, tm_mon=6, tm_mday=29, tm_hour=16, tm_min=53, tm_sec=2, tm_wday=2, tm_yday=180, tm_isdst=0) 2022-06-29T16:53:07.178 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2022-06-29T15:42:25.967805+0000 time.struct_time(tm_year=2022, tm_mon=6, tm_mday=29, tm_hour=15, tm_min=42, tm_sec=25, tm_wday=2, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2022, tm_mon=6, tm_mday=29, tm_hour=16, tm_min=53, tm_sec=2, tm_wday=2, tm_yday=180, tm_isdst=0) 2022-06-29T16:53:07.178 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2022-06-29T16:53:21.894 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:53:21.898 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:53:21.901 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:53:21.903 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:53:27.180 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2022-06-29T16:53:27.291 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.288+0000 7fad5950e700 1 Processor -- start 2022-06-29T16:53:27.292 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.288+0000 7fad5950e700 1 -- start start 2022-06-29T16:53:27.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.288+0000 7fad5950e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fad540b7840 0x7fad540b44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:27.294 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.288+0000 7fad5950e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fad540b3240 0x7fad540b49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:27.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.288+0000 7fad5950e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad540b7c20 con 0x7fad54146890 2022-06-29T16:53:27.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.288+0000 7fad5950e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad540b4ec0 con 0x7fad540b3240 2022-06-29T16:53:27.295 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.288+0000 7fad5950e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad54066840 con 0x7fad540b7840 2022-06-29T16:53:27.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.292+0000 7fad537fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fad540b3240 0x7fad540b49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:27.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.292+0000 7fad53fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fad540b7840 0x7fad540b44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:27.296 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.292+0000 7fad58d0d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fad54146890 0x7fad54146cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.47:48940/0 (socket says 172.21.15.47:48940) 2022-06-29T16:53:27.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.292+0000 7fad58d0d700 1 -- 172.21.15.47:0/2751087800 learned_addr learned my addr 172.21.15.47:0/2751087800 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T16:53:27.297 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.292+0000 7fad537fe700 1 -- 172.21.15.47:0/2751087800 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fad540b7840 msgr2=0x7fad540b44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T16:53:27.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.292+0000 7fad537fe700 1 --2- 172.21.15.47:0/2751087800 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fad540b7840 0x7fad540b44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:27.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.292+0000 7fad537fe700 1 -- 172.21.15.47:0/2751087800 >> v1:172.21.15.64:6789/0 conn(0x7fad54146890 legacy=0x7fad54146cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:27.298 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.292+0000 7fad537fe700 1 -- 172.21.15.47:0/2751087800 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad540b2ff0 con 0x7fad540b3240 2022-06-29T16:53:27.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.292+0000 7fad537fe700 1 --2- 172.21.15.47:0/2751087800 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad540b3240 0x7fad540b49d0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7fad44002f70 tx=0x7fad44006df0).ready entity=mon.1 client_cookie=5294692d50ba0af5 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:27.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.292+0000 7fad52ffd700 1 -- 172.21.15.47:0/2751087800 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad4400d570 con 0x7fad540b3240 2022-06-29T16:53:27.299 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.292+0000 7fad52ffd700 1 -- 172.21.15.47:0/2751087800 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad44000e90 con 0x7fad540b3240 2022-06-29T16:53:27.300 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.292+0000 7fad52ffd700 1 -- 172.21.15.47:0/2751087800 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad4400d8c0 con 0x7fad540b3240 2022-06-29T16:53:27.303 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.292+0000 7fad5950e700 1 -- 172.21.15.47:0/2751087800 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad540b3240 msgr2=0x7fad540b49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:27.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.292+0000 7fad5950e700 1 --2- 172.21.15.47:0/2751087800 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad540b3240 0x7fad540b49d0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7fad44002f70 tx=0x7fad44006df0).stop 2022-06-29T16:53:27.304 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.292+0000 7fad5950e700 1 -- 172.21.15.47:0/2751087800 shutdown_connections 2022-06-29T16:53:27.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.292+0000 7fad5950e700 1 --2- 172.21.15.47:0/2751087800 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad540b3240 0x7fad540b49d0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:27.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.292+0000 7fad5950e700 1 --2- 172.21.15.47:0/2751087800 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fad540b7840 0x7fad540b44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:27.305 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.292+0000 7fad5950e700 1 -- 172.21.15.47:0/2751087800 >> 172.21.15.47:0/2751087800 conn(0x7fad5405f480 msgr2=0x7fad540647d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T16:53:27.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.292+0000 7fad5950e700 1 -- 172.21.15.47:0/2751087800 shutdown_connections 2022-06-29T16:53:27.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.292+0000 7fad5950e700 1 -- 172.21.15.47:0/2751087800 wait complete. 2022-06-29T16:53:27.306 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.296+0000 7fad5950e700 1 Processor -- start 2022-06-29T16:53:27.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.296+0000 7fad5950e700 1 -- start start 2022-06-29T16:53:27.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.296+0000 7fad5950e700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fad540b3240 0x7fad540b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:27.307 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.296+0000 7fad5950e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad540b7840 0x7fad5413a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:27.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.296+0000 7fad5950e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad54146890 0x7fad5413aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:27.308 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.296+0000 7fad5950e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad54140cd0 con 0x7fad54146890 2022-06-29T16:53:27.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.296+0000 7fad5950e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad54061200 con 0x7fad540b7840 2022-06-29T16:53:27.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.296+0000 7fad5950e700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad54060820 con 0x7fad540b3240 2022-06-29T16:53:27.309 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.296+0000 7fad58d0d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad54146890 0x7fad5413aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:27.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.296+0000 7fad537fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad540b7840 0x7fad5413a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:27.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.296+0000 7fad53fff700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fad540b3240 0x7fad540b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:27.310 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.296+0000 7fad58d0d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad54146890 0x7fad5413aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55828/0 (socket says 172.21.15.47:55828) 2022-06-29T16:53:27.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.296+0000 7fad58d0d700 1 -- 172.21.15.47:0/4283402005 learned_addr learned my addr 172.21.15.47:0/4283402005 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T16:53:27.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.300+0000 7fad537fe700 1 -- 172.21.15.47:0/4283402005 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fad540b3240 msgr2=0x7fad540b6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:27.311 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.300+0000 7fad537fe700 1 --2- 172.21.15.47:0/4283402005 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fad540b3240 0x7fad540b6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:27.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.300+0000 7fad537fe700 1 -- 172.21.15.47:0/4283402005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad54146890 msgr2=0x7fad5413aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:27.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.300+0000 7fad537fe700 1 --2- 172.21.15.47:0/4283402005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad54146890 0x7fad5413aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:27.312 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.300+0000 7fad537fe700 1 -- 172.21.15.47:0/4283402005 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad4400b180 con 0x7fad540b7840 2022-06-29T16:53:27.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.300+0000 7fad537fe700 1 --2- 172.21.15.47:0/4283402005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad540b7840 0x7fad5413a9b0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7fad54140e10 tx=0x7fad44014000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:27.313 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.300+0000 7fad517fa700 1 -- 172.21.15.47:0/4283402005 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad44016070 con 0x7fad540b7840 2022-06-29T16:53:27.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.300+0000 7fad517fa700 1 -- 172.21.15.47:0/4283402005 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad4400d8f0 con 0x7fad540b7840 2022-06-29T16:53:27.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.300+0000 7fad5950e700 1 -- 172.21.15.47:0/4283402005 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad54060ac0 con 0x7fad540b7840 2022-06-29T16:53:27.314 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.300+0000 7fad517fa700 1 -- 172.21.15.47:0/4283402005 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad44013e50 con 0x7fad540b7840 2022-06-29T16:53:27.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.300+0000 7fad5950e700 1 -- 172.21.15.47:0/4283402005 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad540ae5f0 con 0x7fad540b7840 2022-06-29T16:53:27.315 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.300+0000 7fad5950e700 1 -- 172.21.15.47:0/4283402005 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad540ae7e0 con 0x7fad540b7840 2022-06-29T16:53:27.316 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.300+0000 7fad517fa700 1 -- 172.21.15.47:0/4283402005 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fad44025030 con 0x7fad540b7840 2022-06-29T16:53:27.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.304+0000 7fad517fa700 1 --2- 172.21.15.47:0/4283402005 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fad34053a70 0x7fad34055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:27.317 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.304+0000 7fad517fa700 1 -- 172.21.15.47:0/4283402005 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fad4406cd00 con 0x7fad540b7840 2022-06-29T16:53:27.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.304+0000 7fad53fff700 1 --2- 172.21.15.47:0/4283402005 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fad34053a70 0x7fad34055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:27.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.304+0000 7fad53fff700 1 --2- 172.21.15.47:0/4283402005 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fad34053a70 0x7fad34055f30 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 rx=0x7fad4800b000 tx=0x7fad48006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:27.318 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.312+0000 7fad517fa700 1 -- 172.21.15.47:0/4283402005 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fad44070030 con 0x7fad540b7840 2022-06-29T16:53:27.437 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.436+0000 7fad5950e700 1 -- 172.21.15.47:0/4283402005 --> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fad540606a0 con 0x7fad34053a70 2022-06-29T16:53:27.443 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.440+0000 7fad517fa700 1 -- 172.21.15.47:0/4283402005 <== mgr.4102 v2:172.21.15.47:6800/12928 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+67537 (secure 0 0 0) 0x7fad480116f0 con 0x7fad34053a70 2022-06-29T16:53:27.445 INFO:teuthology.orchestra.run.smithi047.stdout: 2022-06-29T16:53:27.447 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.444+0000 7fad5950e700 1 -- 172.21.15.47:0/4283402005 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fad34053a70 msgr2=0x7fad34055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:27.447 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.444+0000 7fad5950e700 1 --2- 172.21.15.47:0/4283402005 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fad34053a70 0x7fad34055f30 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 rx=0x7fad4800b000 tx=0x7fad48006df0).stop 2022-06-29T16:53:27.447 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.444+0000 7fad5950e700 1 -- 172.21.15.47:0/4283402005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad540b7840 msgr2=0x7fad5413a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:27.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.444+0000 7fad5950e700 1 --2- 172.21.15.47:0/4283402005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad540b7840 0x7fad5413a9b0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7fad54140e10 tx=0x7fad44014000).stop 2022-06-29T16:53:27.448 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.444+0000 7fad5950e700 1 -- 172.21.15.47:0/4283402005 shutdown_connections 2022-06-29T16:53:27.449 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.444+0000 7fad5950e700 1 --2- 172.21.15.47:0/4283402005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad54146890 0x7fad5413aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:27.449 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.444+0000 7fad5950e700 1 --2- 172.21.15.47:0/4283402005 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7fad34053a70 0x7fad34055f30 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:27.449 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.444+0000 7fad5950e700 1 --2- 172.21.15.47:0/4283402005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad540b7840 0x7fad5413a9b0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:27.450 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.444+0000 7fad5950e700 1 --2- 172.21.15.47:0/4283402005 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7fad540b3240 0x7fad540b6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:27.450 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.444+0000 7fad5950e700 1 -- 172.21.15.47:0/4283402005 >> 172.21.15.47:0/4283402005 conn(0x7fad5405f480 msgr2=0x7fad54149bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T16:53:27.450 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.444+0000 7fad5950e700 1 -- 172.21.15.47:0/4283402005 shutdown_connections 2022-06-29T16:53:27.451 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.444+0000 7fad5950e700 1 -- 172.21.15.47:0/4283402005 wait complete. 2022-06-29T16:53:27.451 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2022-06-29T16:53:27.459 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":2290,"stamp":"2022-06-29T16:53:26.127965+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9482,"num_omap_keys":2,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":8405056,"kb_used_data":16384,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746569664,"statfs":{"total":773094113280,"available":764487335936,"internally_reserved":8589934592,"allocated":16777216,"data_stored":8168800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003455"},"pg_stats":[{"pgid":"1.0","version":"165'4","reported_seq":148,"reported_epoch":165,"state":"active+clean","last_fresh":"2022-06-29T16:53:07.673888+0000","last_change":"2022-06-29T16:53:07.674061+0000","last_active":"2022-06-29T16:53:07.673888+0000","last_peered":"2022-06-29T16:53:07.673888+0000","last_clean":"2022-06-29T16:53:07.673888+0000","last_became_active":"2022-06-29T15:42:22.793734+0000","last_became_peered":"2022-06-29T15:42:22.793734+0000","last_unstale":"2022-06-29T16:53:07.673888+0000","last_undegraded":"2022-06-29T16:53:07.673888+0000","last_fullsized":"2022-06-29T16:53:07.673888+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"165'4","last_scrub_stamp":"2022-06-29T16:53:07.673888+0000","last_deep_scrub":"165'4","last_deep_scrub_stamp":"2022-06-29T16:53:07.673888+0000","last_clean_scrub_stamp":"2022-06-29T16:53:07.673888+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9482,"num_omap_keys":2,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":140,"reported_epoch":165,"state":"active+clean","last_fresh":"2022-06-29T16:53:09.391375+0000","last_change":"2022-06-29T16:53:09.391564+0000","last_active":"2022-06-29T16:53:09.391375+0000","last_peered":"2022-06-29T16:53:09.391375+0000","last_clean":"2022-06-29T16:53:09.391375+0000","last_became_active":"2022-06-29T15:42:27.001981+0000","last_became_peered":"2022-06-29T15:42:27.001981+0000","last_unstale":"2022-06-29T16:53:09.391375+0000","last_undegraded":"2022-06-29T16:53:09.391375+0000","last_fullsized":"2022-06-29T16:53:09.391375+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T16:53:09.391374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T16:53:09.391374+0000","last_clean_scrub_stamp":"2022-06-29T16:53:09.391374+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":141,"reported_epoch":165,"state":"active+clean","last_fresh":"2022-06-29T16:53:11.795725+0000","last_change":"2022-06-29T16:53:11.795916+0000","last_active":"2022-06-29T16:53:11.795725+0000","last_peered":"2022-06-29T16:53:11.795725+0000","last_clean":"2022-06-29T16:53:11.795725+0000","last_became_active":"2022-06-29T15:42:26.999820+0000","last_became_peered":"2022-06-29T15:42:26.999820+0000","last_unstale":"2022-06-29T16:53:11.795725+0000","last_undegraded":"2022-06-29T16:53:11.795725+0000","last_fullsized":"2022-06-29T16:53:11.795725+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T16:53:11.795725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T16:53:11.795725+0000","last_clean_scrub_stamp":"2022-06-29T16:53:11.795725+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":133,"reported_epoch":165,"state":"active+clean","last_fresh":"2022-06-29T16:53:04.956047+0000","last_change":"2022-06-29T16:53:04.956192+0000","last_active":"2022-06-29T16:53:04.956047+0000","last_peered":"2022-06-29T16:53:04.956047+0000","last_clean":"2022-06-29T16:53:04.956047+0000","last_became_active":"2022-06-29T15:42:27.554328+0000","last_became_peered":"2022-06-29T15:42:27.554328+0000","last_unstale":"2022-06-29T16:53:04.956047+0000","last_undegraded":"2022-06-29T16:53:04.956047+0000","last_fullsized":"2022-06-29T16:53:04.956047+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T16:53:04.956047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T16:53:04.956047+0000","last_clean_scrub_stamp":"2022-06-29T16:53:04.956047+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":140,"reported_epoch":165,"state":"active+clean","last_fresh":"2022-06-29T16:53:07.367232+0000","last_change":"2022-06-29T16:53:07.367420+0000","last_active":"2022-06-29T16:53:07.367232+0000","last_peered":"2022-06-29T16:53:07.367232+0000","last_clean":"2022-06-29T16:53:07.367232+0000","last_became_active":"2022-06-29T15:42:27.001089+0000","last_became_peered":"2022-06-29T15:42:27.001089+0000","last_unstale":"2022-06-29T16:53:07.367232+0000","last_undegraded":"2022-06-29T16:53:07.367232+0000","last_fullsized":"2022-06-29T16:53:07.367232+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T16:53:07.367231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T16:53:07.367231+0000","last_clean_scrub_stamp":"2022-06-29T16:53:07.367231+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":140,"reported_epoch":165,"state":"active+clean","last_fresh":"2022-06-29T16:53:05.758120+0000","last_change":"2022-06-29T16:53:05.758275+0000","last_active":"2022-06-29T16:53:05.758120+0000","last_peered":"2022-06-29T16:53:05.758120+0000","last_clean":"2022-06-29T16:53:05.758120+0000","last_became_active":"2022-06-29T15:42:27.720770+0000","last_became_peered":"2022-06-29T15:42:27.720770+0000","last_unstale":"2022-06-29T16:53:05.758120+0000","last_undegraded":"2022-06-29T16:53:05.758120+0000","last_fullsized":"2022-06-29T16:53:05.758120+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T16:53:05.758119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T16:53:05.758119+0000","last_clean_scrub_stamp":"2022-06-29T16:53:05.758119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":141,"reported_epoch":165,"state":"active+clean","last_fresh":"2022-06-29T16:53:12.803183+0000","last_change":"2022-06-29T16:53:12.803274+0000","last_active":"2022-06-29T16:53:12.803183+0000","last_peered":"2022-06-29T16:53:12.803183+0000","last_clean":"2022-06-29T16:53:12.803183+0000","last_became_active":"2022-06-29T15:42:27.722958+0000","last_became_peered":"2022-06-29T15:42:27.722958+0000","last_unstale":"2022-06-29T16:53:12.803183+0000","last_undegraded":"2022-06-29T16:53:12.803183+0000","last_fullsized":"2022-06-29T16:53:12.803183+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T16:53:12.803183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T16:53:12.803183+0000","last_clean_scrub_stamp":"2022-06-29T16:53:12.803183+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":140,"reported_epoch":165,"state":"active+clean","last_fresh":"2022-06-29T16:53:07.773019+0000","last_change":"2022-06-29T16:53:07.773094+0000","last_active":"2022-06-29T16:53:07.773019+0000","last_peered":"2022-06-29T16:53:07.773019+0000","last_clean":"2022-06-29T16:53:07.773019+0000","last_became_active":"2022-06-29T15:42:26.997677+0000","last_became_peered":"2022-06-29T15:42:26.997677+0000","last_unstale":"2022-06-29T16:53:07.773019+0000","last_undegraded":"2022-06-29T16:53:07.773019+0000","last_fullsized":"2022-06-29T16:53:07.773019+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T16:53:07.773018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T16:53:07.773018+0000","last_clean_scrub_stamp":"2022-06-29T16:53:07.773018+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":136,"reported_epoch":165,"state":"active+clean","last_fresh":"2022-06-29T16:53:06.708378+0000","last_change":"2022-06-29T16:53:06.708457+0000","last_active":"2022-06-29T16:53:06.708378+0000","last_peered":"2022-06-29T16:53:06.708378+0000","last_clean":"2022-06-29T16:53:06.708378+0000","last_became_active":"2022-06-29T15:42:26.999119+0000","last_became_peered":"2022-06-29T15:42:26.999119+0000","last_unstale":"2022-06-29T16:53:06.708378+0000","last_undegraded":"2022-06-29T16:53:06.708378+0000","last_fullsized":"2022-06-29T16:53:06.708378+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-06-29T16:53:06.708377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-06-29T16:53:06.708377+0000","last_clean_scrub_stamp":"2022-06-29T16:53:06.708377+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9482,"num_omap_keys":2,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":14,"seq":60129542999,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050632,"kb_used_data":2048,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321208,"statfs":{"total":96636764160,"available":95560916992,"internally_reserved":1073741824,"allocated":2097152,"data_stored":1021100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.84099999999999997,"15min":0.84699999999999998},"min":{"1min":0.58899999999999997,"5min":0.58099999999999996,"15min":0.54900000000000004},"max":{"1min":1.125,"5min":1.165,"15min":1.335},"last":0.75},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.68700000000000006,"5min":0.57199999999999995,"15min":0.49399999999999999},"max":{"1min":1.1479999999999999,"5min":1.151,"15min":1.282},"last":0.60699999999999998}]},{"osd":1,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.83199999999999996,"15min":0.83099999999999996},"min":{"1min":0.626,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0580000000000001,"5min":1.224,"15min":1.224},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.871,"15min":0.879},"min":{"1min":0.64500000000000002,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.403},"last":1.0409999999999999}]},{"osd":2,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.89400000000000002,"15min":0.89300000000000002},"min":{"1min":0.69699999999999995,"5min":0.60599999999999998,"15min":0.56799999999999995},"max":{"1min":1.107,"5min":1.194,"15min":1.2290000000000001},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.86599999999999999,"15min":0.88500000000000001},"min":{"1min":0.65400000000000003,"5min":0.65200000000000002,"15min":0.63800000000000001},"max":{"1min":1.071,"5min":1.2130000000000001,"15min":1.276},"last":0.94999999999999996}]},{"osd":3,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.873,"15min":0.876},"min":{"1min":0.626,"5min":0.60799999999999998,"15min":0.58699999999999997},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2929999999999999},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.042,"5min":1.0269999999999999,"15min":1.0209999999999999},"min":{"1min":0.86399999999999999,"5min":0.753,"15min":0.71999999999999997},"max":{"1min":1.212,"5min":1.25,"15min":1.3089999999999999},"last":1.0640000000000001}]},{"osd":4,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.94499999999999995,"15min":0.89800000000000002},"min":{"1min":0.65500000000000003,"5min":0.58599999999999997,"15min":0.53100000000000003},"max":{"1min":1.1930000000000001,"5min":1.337,"15min":1.337},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.92900000000000005,"15min":0.89600000000000002},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.501},"max":{"1min":1.1879999999999999,"5min":1.369,"15min":1.369},"last":0.999}]},{"osd":6,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.95799999999999996,"15min":0.92900000000000005},"min":{"1min":0.63400000000000001,"5min":0.61199999999999999,"15min":0.59999999999999998},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":1.0229999999999999},{"interface":"front","average":{"1min":1.002,"5min":1.044,"15min":1.02},"min":{"1min":0.71599999999999997,"5min":0.623,"15min":0.54200000000000004},"max":{"1min":1.3080000000000001,"5min":1.4610000000000001,"15min":1.4770000000000001},"last":1.0880000000000001}]},{"osd":7,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":1.018,"15min":0.98399999999999999},"min":{"1min":0.75800000000000001,"5min":0.63,"15min":0.63},"max":{"1min":1.2969999999999999,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.1080000000000001,"15min":1.079},"min":{"1min":0.79600000000000004,"5min":0.76600000000000001,"15min":0.69999999999999996},"max":{"1min":1.2869999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":1.123}]}]},{"osd":7,"up_from":14,"seq":60129542999,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050632,"kb_used_data":2048,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321208,"statfs":{"total":96636764160,"available":95560916992,"internally_reserved":1073741824,"allocated":2097152,"data_stored":1021100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 29 16:52:42 2022","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.877,"15min":0.88},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.48699999999999999},"max":{"1min":1.1679999999999999,"5min":1.21,"15min":1.2410000000000001},"last":1.028},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.58199999999999996},"max":{"1min":1.1319999999999999,"5min":1.171,"15min":1.2569999999999999},"last":0.95199999999999996}]},{"osd":1,"last update":"Wed Jun 29 16:52:42 2022","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.79600000000000004,"15min":0.79900000000000004},"min":{"1min":0.59599999999999997,"5min":0.57899999999999996,"15min":0.49299999999999999},"max":{"1min":0.997,"5min":1.1850000000000001,"15min":1.202},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.82199999999999995,"15min":0.82799999999999996},"min":{"1min":0.60399999999999998,"5min":0.59499999999999997,"15min":0.58199999999999996},"max":{"1min":1.0780000000000001,"5min":1.161,"15min":1.226},"last":0.77600000000000002}]},{"osd":2,"last update":"Wed Jun 29 16:52:42 2022","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.88400000000000001,"15min":0.88800000000000001},"min":{"1min":0.63900000000000001,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.04,"5min":1.1679999999999999,"15min":1.333},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.94899999999999995,"15min":0.94599999999999995},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.55900000000000005},"max":{"1min":1.1639999999999999,"5min":1.2250000000000001,"15min":1.2569999999999999},"last":1.1060000000000001}]},{"osd":3,"last update":"Wed Jun 29 16:52:42 2022","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.90700000000000003,"15min":0.90000000000000002},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.095,"5min":1.224,"15min":1.276},"last":0.76300000000000001},{"interface":"front","average":{"1min":1.032,"5min":0.97099999999999997,"15min":0.97799999999999998},"min":{"1min":0.87,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.2050000000000001,"5min":1.347,"15min":1.347},"last":1.0680000000000001}]},{"osd":4,"last update":"Wed Jun 29 16:52:42 2022","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.80100000000000005,"15min":0.82799999999999996},"min":{"1min":0.61499999999999999,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.016,"5min":1.3049999999999999,"15min":1.349},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.66200000000000003,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.125,"5min":1.226,"15min":1.246},"last":0.80800000000000005}]},{"osd":5,"last update":"Wed Jun 29 16:52:42 2022","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.94799999999999995,"15min":0.94099999999999995},"min":{"1min":0.67900000000000005,"5min":0.56699999999999995,"15min":0.56599999999999995},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.306},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.93799999999999994,"5min":1.0009999999999999,"15min":0.98499999999999999},"min":{"1min":0.621,"5min":0.621,"15min":0.59299999999999997},"max":{"1min":1.151,"5min":1.2849999999999999,"15min":1.3200000000000001},"last":0.84199999999999997}]},{"osd":6,"last update":"Wed Jun 29 16:52:42 2022","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.0629999999999999,"15min":1.044},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68400000000000005},"max":{"1min":1.244,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.99199999999999999,"5min":1.028,"15min":1.0169999999999999},"min":{"1min":0.74399999999999999,"5min":0.70499999999999996,"15min":0.65700000000000003},"max":{"1min":1.23,"5min":1.52,"15min":1.52},"last":0.90100000000000002}]}]},{"osd":6,"up_from":14,"seq":60129543000,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050632,"kb_used_data":2048,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321208,"statfs":{"total":96636764160,"available":95560916992,"internally_reserved":1073741824,"allocated":2097152,"data_stored":1021100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 29 16:52:51 2022","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.83499999999999996,"15min":0.85799999999999998},"min":{"1min":0.57399999999999995,"5min":0.54500000000000004,"15min":0.53700000000000003},"max":{"1min":1.0369999999999999,"5min":1.1850000000000001,"15min":1.522},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.90800000000000003,"15min":0.93700000000000006},"min":{"1min":0.68300000000000005,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.135,"5min":1.349,"15min":1.5469999999999999},"last":1.0760000000000001}]},{"osd":1,"last update":"Wed Jun 29 16:52:51 2022","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.78800000000000003,"15min":0.79000000000000004},"min":{"1min":0.58799999999999997,"5min":0.54700000000000004,"15min":0.53200000000000003},"max":{"1min":1.0149999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79500000000000004,"15min":0.81100000000000005},"min":{"1min":0.629,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.038,"5min":1.1080000000000001,"15min":1.359},"last":0.65400000000000003}]},{"osd":2,"last update":"Wed Jun 29 16:52:51 2022","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.88600000000000001,"15min":0.88300000000000001},"min":{"1min":0.67300000000000004,"5min":0.59899999999999998,"15min":0.58799999999999997},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93000000000000005,"15min":0.91800000000000004},"min":{"1min":0.72299999999999998,"5min":0.60299999999999998,"15min":0.58199999999999996},"max":{"1min":1.1379999999999999,"5min":1.2270000000000001,"15min":1.3680000000000001},"last":1.03}]},{"osd":3,"last update":"Wed Jun 29 16:52:51 2022","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.879,"15min":0.875},"min":{"1min":0.629,"5min":0.621,"15min":0.60899999999999999},"max":{"1min":1.127,"5min":1.23,"15min":1.232},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.97799999999999998,"15min":0.97599999999999998},"min":{"1min":0.80100000000000005,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.1859999999999999,"5min":1.22,"15min":1.278},"last":1.05}]},{"osd":4,"last update":"Wed Jun 29 16:52:51 2022","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.83899999999999997,"15min":0.84199999999999997},"min":{"1min":0.60899999999999999,"5min":0.58799999999999997,"15min":0.54700000000000004},"max":{"1min":0.98699999999999999,"5min":1.27,"15min":1.27},"last":0.72899999999999998},{"interface":"front","average":{"1min":1.034,"5min":0.97199999999999998,"15min":0.93200000000000005},"min":{"1min":0.79000000000000004,"5min":0.64500000000000002,"15min":0.57199999999999995},"max":{"1min":1.1839999999999999,"5min":1.298,"15min":1.3420000000000001},"last":0.77600000000000002}]},{"osd":5,"last update":"Wed Jun 29 16:52:51 2022","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93100000000000005,"15min":0.92000000000000004},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.58499999999999996},"max":{"1min":1.198,"5min":1.216,"15min":1.385},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.999,"15min":0.98399999999999999},"min":{"1min":0.83199999999999996,"5min":0.67100000000000004,"15min":0.63500000000000001},"max":{"1min":1.2989999999999999,"5min":1.3320000000000001,"15min":1.423},"last":0.89900000000000002}]},{"osd":7,"last update":"Wed Jun 29 16:52:51 2022","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.0449999999999999,"15min":1.034},"min":{"1min":0.72699999999999998,"5min":0.71099999999999997,"15min":0.63300000000000001},"max":{"1min":1.157,"5min":1.391,"15min":1.4019999999999999},"last":1.006},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.107,"15min":1.097},"min":{"1min":0.97599999999999998,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.581},"last":1.105}]}]},{"osd":4,"up_from":14,"seq":60129542999,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1050632,"kb_used_data":2048,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321208,"statfs":{"total":96636764160,"available":95560916992,"internally_reserved":1073741824,"allocated":2097152,"data_stored":1021100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 29 16:53:25 2022","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.85599999999999998,"15min":0.85699999999999998},"min":{"1min":0.65600000000000003,"5min":0.505,"15min":0.505},"max":{"1min":1.1299999999999999,"5min":1.135,"15min":1.228},"last":0.874},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.95599999999999996,"15min":0.93999999999999995},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.96099999999999997}]},{"osd":1,"last update":"Wed Jun 29 16:53:25 2022","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.82099999999999995,"15min":0.82999999999999996},"min":{"1min":0.622,"5min":0.59099999999999997,"15min":0.56299999999999994},"max":{"1min":1.0980000000000001,"5min":1.1950000000000001,"15min":1.202},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.82899999999999996,"15min":0.82999999999999996},"min":{"1min":0.59499999999999997,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.96499999999999997,"5min":1.1699999999999999,"15min":1.21},"last":0.78800000000000003}]},{"osd":2,"last update":"Wed Jun 29 16:53:25 2022","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.85899999999999999,"15min":0.871},"min":{"1min":0.65500000000000003,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.1339999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.84699999999999998,"15min":0.84499999999999997},"min":{"1min":0.59399999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.0940000000000001,"5min":1.1259999999999999,"15min":1.232},"last":0.67300000000000004}]},{"osd":3,"last update":"Wed Jun 29 16:53:25 2022","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91600000000000004,"15min":0.91500000000000004},"min":{"1min":0.77000000000000002,"5min":0.61799999999999999,"15min":0.60099999999999998},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.2470000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.004,"15min":1.0049999999999999},"min":{"1min":0.70999999999999996,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.2150000000000001,"5min":1.2250000000000001,"15min":1.52},"last":1.0189999999999999}]},{"osd":5,"last update":"Wed Jun 29 16:53:25 2022","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.80600000000000005,"15min":0.81100000000000005},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.52000000000000002},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.89700000000000002,"15min":0.91300000000000003},"min":{"1min":0.63600000000000001,"5min":0.58399999999999996,"15min":0.54400000000000004},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":1.0469999999999999}]},{"osd":6,"last update":"Wed Jun 29 16:53:25 2022","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.95299999999999996,"15min":0.94999999999999996},"min":{"1min":0.66800000000000004,"5min":0.65900000000000003,"15min":0.58299999999999996},"max":{"1min":1.2310000000000001,"5min":1.3380000000000001,"15min":1.4390000000000001},"last":1.077},{"interface":"front","average":{"1min":1.0129999999999999,"5min":0.998,"15min":1.008},"min":{"1min":0.76300000000000001,"5min":0.72299999999999998,"15min":0.61799999999999999},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.995}]},{"osd":7,"last update":"Wed Jun 29 16:53:25 2022","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.014,"15min":1.01},"min":{"1min":0.628,"5min":0.628,"15min":0.61599999999999999},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":1.1040000000000001},{"interface":"front","average":{"1min":1.028,"5min":1.0429999999999999,"15min":1.04},"min":{"1min":0.76900000000000002,"5min":0.67800000000000005,"15min":0.496},"max":{"1min":1.3260000000000001,"5min":1.663,"15min":1.663},"last":1.121}]}]},{"osd":3,"up_from":14,"seq":60129542999,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050632,"kb_used_data":2048,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321208,"statfs":{"total":96636764160,"available":95560916992,"internally_reserved":1073741824,"allocated":2097152,"data_stored":1021100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63200000000000001,"15min":0.64100000000000001},"min":{"1min":0.5,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.80400000000000005,"5min":1.075,"15min":1.103},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.755,"5min":0.78300000000000003,"15min":0.78000000000000003},"min":{"1min":0.55600000000000005,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.92600000000000005,"5min":1.194,"15min":1.232},"last":0.84499999999999997}]},{"osd":1,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.65300000000000002,"15min":0.64800000000000002},"min":{"1min":0.51200000000000001,"5min":0.48099999999999998,"15min":0.46899999999999997},"max":{"1min":0.96799999999999997,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77600000000000002,"15min":0.77400000000000002},"min":{"1min":0.53900000000000003,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.1220000000000001,"5min":1.288,"15min":1.288},"last":0.81299999999999994}]},{"osd":2,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.70599999999999996,"15min":0.68600000000000005},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.97899999999999998,"5min":1.365,"15min":1.365},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.85699999999999998,"15min":0.84099999999999997},"min":{"1min":0.68999999999999995,"5min":0.60299999999999998,"15min":0.57399999999999995},"max":{"1min":1.046,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.66500000000000004}]},{"osd":4,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.94199999999999995,"15min":0.93999999999999995},"min":{"1min":0.71599999999999997,"5min":0.71399999999999997,"15min":0.67600000000000005},"max":{"1min":1.1240000000000001,"5min":1.556,"15min":1.556},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.91800000000000004,"15min":0.91100000000000003},"min":{"1min":0.71299999999999997,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.0569999999999999,"5min":1.3169999999999999,"15min":1.3420000000000001},"last":0.878}]},{"osd":5,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.96399999999999997,"15min":0.93999999999999995},"min":{"1min":0.72199999999999998,"5min":0.69999999999999996,"15min":0.68500000000000005},"max":{"1min":1.2390000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.024,"15min":1.016},"min":{"1min":0.82399999999999995,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.1970000000000001,"5min":1.3640000000000001,"15min":1.454},"last":1.3600000000000001}]},{"osd":6,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":1.0329999999999999,"15min":1.0229999999999999},"min":{"1min":0.745,"5min":0.73499999999999999,"15min":0.67200000000000004},"max":{"1min":1.337,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.181},{"interface":"front","average":{"1min":1.028,"5min":1.0289999999999999,"15min":1.0249999999999999},"min":{"1min":0.77000000000000002,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.2849999999999999,"5min":1.496,"15min":1.508},"last":1.0509999999999999}]},{"osd":7,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.82699999999999996,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4970000000000001},"last":1.304},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.1379999999999999,"15min":1.1240000000000001},"min":{"1min":0.84199999999999997,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.3999999999999999,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":1.1200000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542999,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050632,"kb_used_data":2048,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321208,"statfs":{"total":96636764160,"available":95560916992,"internally_reserved":1073741824,"allocated":2097152,"data_stored":1021100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.628,"15min":0.624},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.42099999999999999},"max":{"1min":1.03,"5min":1.137,"15min":1.256},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.75,"5min":0.72699999999999998,"15min":0.70399999999999996},"min":{"1min":0.53600000000000003,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.099,"5min":1.151,"15min":1.151},"last":0.70299999999999996}]},{"osd":1,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.48399999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.91600000000000004,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.751,"15min":0.76500000000000001},"min":{"1min":0.54000000000000004,"5min":0.502,"15min":0.502},"max":{"1min":0.92100000000000004,"5min":1.131,"15min":1.131},"last":0.628}]},{"osd":3,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.78500000000000003,"15min":0.80800000000000005},"min":{"1min":0.58499999999999996,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0900000000000001,"5min":1.133,"15min":1.1930000000000001},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.84399999999999997,"15min":0.83399999999999996},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.55600000000000005},"max":{"1min":1.1040000000000001,"5min":1.155,"15min":1.1990000000000001},"last":1.111}]},{"osd":4,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.877,"15min":0.88400000000000001},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.2390000000000001,"5min":1.2509999999999999,"15min":1.323},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.92600000000000005,"15min":0.93100000000000005},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":1.1559999999999999}]},{"osd":5,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":1.0209999999999999,"15min":1.0129999999999999},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.478,"5min":1.478,"15min":1.5169999999999999},"last":1.208},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.98099999999999998,"15min":0.97699999999999998},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.64500000000000002},"max":{"1min":1.2130000000000001,"5min":1.3540000000000001,"15min":1.5029999999999999},"last":1.282}]},{"osd":6,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.032,"15min":1.0289999999999999},"min":{"1min":0.81000000000000005,"5min":0.752,"15min":0.69999999999999996},"max":{"1min":1.3260000000000001,"5min":1.464,"15min":1.53},"last":1.228},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0980000000000001,"15min":1.093},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.75900000000000001},"max":{"1min":1.413,"5min":1.516,"15min":1.5669999999999999},"last":1.3440000000000001}]},{"osd":7,"last update":"Wed Jun 29 16:52:49 2022","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.0700000000000001,"15min":1.0669999999999999},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.123,"5min":1.109,"15min":1.109},"min":{"1min":0.94399999999999995,"5min":0.86299999999999999,"15min":0.85099999999999998},"max":{"1min":1.405,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":1.3140000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542998,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050632,"kb_used_data":2048,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321208,"statfs":{"total":96636764160,"available":95560916992,"internally_reserved":1073741824,"allocated":2097152,"data_stored":1021100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 29 16:52:38 2022","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.622,"15min":0.63600000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.47199999999999998},"max":{"1min":0.75600000000000001,"5min":1.024,"15min":1.1000000000000001},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70899999999999996,"15min":0.70199999999999996},"min":{"1min":0.54100000000000004,"5min":0.47999999999999998,"15min":0.47299999999999998},"max":{"1min":0.93000000000000005,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.79500000000000004}]},{"osd":2,"last update":"Wed Jun 29 16:52:38 2022","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.65100000000000002,"15min":0.64800000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.46700000000000003},"max":{"1min":0.80000000000000004,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76300000000000001,"15min":0.76800000000000002},"min":{"1min":0.61699999999999999,"5min":0.54600000000000004,"15min":0.51700000000000002},"max":{"1min":0.98699999999999999,"5min":1.22,"15min":1.22},"last":0.67600000000000005}]},{"osd":3,"last update":"Wed Jun 29 16:52:38 2022","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.82399999999999995,"15min":0.84699999999999998},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.56100000000000005},"max":{"1min":1.01,"5min":1.109,"15min":1.157},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78900000000000003,"15min":0.77700000000000002},"min":{"1min":0.60599999999999998,"5min":0.52900000000000003,"15min":0.47699999999999998},"max":{"1min":0.95699999999999996,"5min":1.788,"15min":1.788},"last":1.1850000000000001}]},{"osd":4,"last update":"Wed Jun 29 16:52:38 2022","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.877,"15min":0.873},"min":{"1min":0.69899999999999995,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.248,"5min":1.288,"15min":1.4610000000000001},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.995,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.81000000000000005,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.24,"5min":1.405,"15min":1.4330000000000001},"last":0.76100000000000001}]},{"osd":5,"last update":"Wed Jun 29 16:52:38 2022","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.93899999999999995,"15min":0.96899999999999997},"min":{"1min":0.78100000000000003,"5min":0.71299999999999997,"15min":0.66700000000000004},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.4279999999999999},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.95899999999999996,"15min":0.96599999999999997},"min":{"1min":0.75,"5min":0.72799999999999998,"15min":0.67300000000000004},"max":{"1min":1.167,"5min":1.357,"15min":1.3959999999999999},"last":0.77800000000000002}]},{"osd":6,"last update":"Wed Jun 29 16:52:38 2022","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.0209999999999999,"15min":1.0509999999999999},"min":{"1min":0.77300000000000002,"5min":0.76900000000000002,"15min":0.70499999999999996},"max":{"1min":1.375,"5min":1.375,"15min":1.516},"last":1.032},{"interface":"front","average":{"1min":1.014,"5min":1.0389999999999999,"15min":1.054},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.343,"5min":1.444,"15min":1.5649999999999999},"last":1.0469999999999999}]},{"osd":7,"last update":"Wed Jun 29 16:52:38 2022","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.0880000000000001,"15min":1.1020000000000001},"min":{"1min":0.77200000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.3149999999999999,"5min":1.4550000000000001,"15min":1.6100000000000001},"last":0.871},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.0920000000000001,"15min":1.1020000000000001},"min":{"1min":0.92900000000000005,"5min":0.82099999999999995,"15min":0.72699999999999998},"max":{"1min":1.413,"5min":1.413,"15min":1.603},"last":1.111}]}]},{"osd":0,"up_from":14,"seq":60129542999,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050632,"kb_used_data":2048,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321208,"statfs":{"total":96636764160,"available":95560916992,"internally_reserved":1073741824,"allocated":2097152,"data_stored":1021100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jun 29 16:52:42 2022","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.64900000000000002,"15min":0.63500000000000001},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.45200000000000001},"max":{"1min":0.79100000000000004,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.62,"5min":0.64900000000000002,"15min":0.67100000000000004},"min":{"1min":0.52300000000000002,"5min":0.46800000000000003,"15min":0.45300000000000001},"max":{"1min":0.84499999999999997,"5min":0.98299999999999998,"15min":1.1259999999999999},"last":0.61199999999999999}]},{"osd":2,"last update":"Wed Jun 29 16:52:42 2022","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.71999999999999997,"15min":0.73499999999999999},"min":{"1min":0.51300000000000001,"5min":0.499,"15min":0.46800000000000003},"max":{"1min":1.0429999999999999,"5min":1.0549999999999999,"15min":1.2190000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.80700000000000005,"15min":0.82599999999999996},"min":{"1min":0.67500000000000004,"5min":0.60499999999999998,"15min":0.59699999999999998},"max":{"1min":1.0329999999999999,"5min":1.085,"15min":1.143},"last":0.77100000000000002}]},{"osd":3,"last update":"Wed Jun 29 16:52:42 2022","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.71299999999999997,"15min":0.70099999999999996},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.48899999999999999},"max":{"1min":0.97999999999999998,"5min":1.117,"15min":1.147},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.80600000000000005,"15min":0.82599999999999996},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.028,"5min":1.145,"15min":1.2090000000000001},"last":0.80200000000000005}]},{"osd":4,"last update":"Wed Jun 29 16:52:42 2022","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.91400000000000003,"15min":0.92400000000000004},"min":{"1min":0.73999999999999999,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.077,"5min":1.2769999999999999,"15min":1.3759999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.88600000000000001,"15min":0.91100000000000003},"min":{"1min":0.68600000000000005,"5min":0.67700000000000005,"15min":0.66600000000000004},"max":{"1min":1.1699999999999999,"5min":1.381,"15min":1.5},"last":0.93400000000000005}]},{"osd":5,"last update":"Wed Jun 29 16:52:42 2022","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.91400000000000003,"15min":0.93100000000000005},"min":{"1min":0.70099999999999996,"5min":0.66600000000000004,"15min":0.66100000000000003},"max":{"1min":1.266,"5min":1.4419999999999999,"15min":1.6040000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.032,"5min":1.0149999999999999,"15min":1.024},"min":{"1min":0.80300000000000005,"5min":0.752,"15min":0.67900000000000005},"max":{"1min":1.2210000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":1.329}]},{"osd":6,"last update":"Wed Jun 29 16:52:42 2022","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.0309999999999999,"15min":1.052},"min":{"1min":0.76000000000000001,"5min":0.749,"15min":0.73599999999999999},"max":{"1min":1.2949999999999999,"5min":1.458,"15min":1.524},"last":1.2669999999999999},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.024,"15min":1.0509999999999999},"min":{"1min":0.82499999999999996,"5min":0.78700000000000003,"15min":0.70799999999999996},"max":{"1min":1.163,"5min":1.502,"15min":1.552},"last":1.028}]},{"osd":7,"last update":"Wed Jun 29 16:52:42 2022","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.0680000000000001,"15min":1.0840000000000001},"min":{"1min":0.85999999999999999,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.2270000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.069,"15min":1.077},"min":{"1min":0.876,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.466,"5min":1.486,"15min":1.5960000000000001},"last":1.22}]}]}]}} 2022-06-29T16:53:27.462 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2022-06-29T16:53:27.570 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.568+0000 7f2453297700 1 Processor -- start 2022-06-29T16:53:27.571 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.568+0000 7f2453297700 1 -- start start 2022-06-29T16:53:27.573 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.568+0000 7f2453297700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f244c060c10 0x7f244c0b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:27.574 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.568+0000 7f2453297700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f244c0aa0c0 0x7f244c0b1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:27.574 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.568+0000 7f2453297700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f244c060ff0 con 0x7f244c05f0b0 2022-06-29T16:53:27.574 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.568+0000 7f2453297700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f244c05ffe0 con 0x7f244c0aa0c0 2022-06-29T16:53:27.575 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.568+0000 7f2453297700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f244c05faa0 con 0x7f244c060c10 2022-06-29T16:53:27.575 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.568+0000 7f2450832700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f244c0aa0c0 0x7f244c0b1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:27.575 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.568+0000 7f2451834700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f244c05f0b0 0x7f244c146970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41390/0 (socket says 172.21.15.47:41390) 2022-06-29T16:53:27.576 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.568+0000 7f2451834700 1 -- 172.21.15.47:0/750403927 learned_addr learned my addr 172.21.15.47:0/750403927 (peer_addr_for_me v1:172.21.15.47:0/0) 2022-06-29T16:53:27.576 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.568+0000 7f2451033700 1 --2- 172.21.15.47:0/750403927 >> v2:172.21.15.64:3300/0 conn(0x7f244c060c10 0x7f244c0b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:27.576 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.568+0000 7f2443fff700 1 -- 172.21.15.47:0/750403927 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3386373423 0 0) 0x7f244c060ff0 con 0x7f244c05f0b0 2022-06-29T16:53:27.577 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.568+0000 7f2443fff700 1 -- 172.21.15.47:0/750403927 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2430003260 con 0x7f244c05f0b0 2022-06-29T16:53:27.577 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.568+0000 7f2450832700 1 -- 172.21.15.47:0/750403927 >> v2:172.21.15.64:3300/0 conn(0x7f244c060c10 msgr2=0x7f244c0b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-06-29T16:53:27.577 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.568+0000 7f2450832700 1 --2- 172.21.15.47:0/750403927 >> v2:172.21.15.64:3300/0 conn(0x7f244c060c10 0x7f244c0b1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:27.578 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.572+0000 7f2450832700 1 -- 172.21.15.47:0/750403927 >> v1:172.21.15.47:6789/0 conn(0x7f244c05f0b0 legacy=0x7f244c146970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:27.578 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.572+0000 7f2450832700 1 -- 172.21.15.47:0/750403927 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f244c05fd40 con 0x7f244c0aa0c0 2022-06-29T16:53:27.578 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.572+0000 7f2450832700 1 --2- 172.21.15.47:0/750403927 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f244c0aa0c0 0x7f244c0b1d10 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 rx=0x7f244c0adee0 tx=0x7f243c006df0).ready entity=mon.0 client_cookie=94466503620c6126 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:27.579 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.572+0000 7f2443fff700 1 -- 172.21.15.47:0/750403927 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f243c00d4f0 con 0x7f244c0aa0c0 2022-06-29T16:53:27.579 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.572+0000 7f2443fff700 1 -- 172.21.15.47:0/750403927 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f243c000e90 con 0x7f244c0aa0c0 2022-06-29T16:53:27.579 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.572+0000 7f2443fff700 1 -- 172.21.15.47:0/750403927 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f243c00d840 con 0x7f244c0aa0c0 2022-06-29T16:53:27.583 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.572+0000 7f2453297700 1 -- 172.21.15.47:0/750403927 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f244c0aa0c0 msgr2=0x7f244c0b1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:27.584 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.572+0000 7f2453297700 1 --2- 172.21.15.47:0/750403927 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f244c0aa0c0 0x7f244c0b1d10 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 rx=0x7f244c0adee0 tx=0x7f243c006df0).stop 2022-06-29T16:53:27.584 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.572+0000 7f2453297700 1 -- 172.21.15.47:0/750403927 shutdown_connections 2022-06-29T16:53:27.584 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.572+0000 7f2453297700 1 --2- 172.21.15.47:0/750403927 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f244c0aa0c0 0x7f244c0b1d10 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:27.585 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.572+0000 7f2453297700 1 --2- 172.21.15.47:0/750403927 >> v2:172.21.15.64:3300/0 conn(0x7f244c060c10 0x7f244c0b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:27.585 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.572+0000 7f2453297700 1 -- 172.21.15.47:0/750403927 >> 172.21.15.47:0/750403927 conn(0x7f244c062110 msgr2=0x7f244c0601c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T16:53:27.585 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.572+0000 7f2453297700 1 -- 172.21.15.47:0/750403927 shutdown_connections 2022-06-29T16:53:27.586 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.572+0000 7f2453297700 1 -- 172.21.15.47:0/750403927 wait complete. 2022-06-29T16:53:27.586 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.572+0000 7f2453297700 1 Processor -- start 2022-06-29T16:53:27.586 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.572+0000 7f2453297700 1 -- start start 2022-06-29T16:53:27.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.576+0000 7f2453297700 1 --2- >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f244c05f0b0 0x7f244c0b36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:27.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.576+0000 7f2453297700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f244c060c10 0x7f244c146020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:27.587 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.576+0000 7f2453297700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f244c0aa0c0 0x7f244c146510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:27.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.576+0000 7f2453297700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244c145650 con 0x7f244c060c10 2022-06-29T16:53:27.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.576+0000 7f2453297700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244c145bf0 con 0x7f244c0aa0c0 2022-06-29T16:53:27.588 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.576+0000 7f2453297700 1 -- --> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f244c066ab0 con 0x7f244c05f0b0 2022-06-29T16:53:27.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.576+0000 7f2450832700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f244c060c10 0x7f244c146020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:27.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.576+0000 7f2450832700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f244c060c10 0x7f244c146020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55840/0 (socket says 172.21.15.47:55840) 2022-06-29T16:53:27.589 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.576+0000 7f2450832700 1 -- 172.21.15.47:0/1512931550 learned_addr learned my addr 172.21.15.47:0/1512931550 (peer_addr_for_me v2:172.21.15.47:0/0) 2022-06-29T16:53:27.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.576+0000 7f2451834700 1 --2- 172.21.15.47:0/1512931550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f244c0aa0c0 0x7f244c146510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:27.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.576+0000 7f2451033700 1 --2- 172.21.15.47:0/1512931550 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f244c05f0b0 0x7f244c0b36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:27.590 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.576+0000 7f2450832700 1 -- 172.21.15.47:0/1512931550 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f244c05f0b0 msgr2=0x7f244c0b36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:27.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.576+0000 7f2450832700 1 --2- 172.21.15.47:0/1512931550 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f244c05f0b0 0x7f244c0b36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:27.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.576+0000 7f2450832700 1 -- 172.21.15.47:0/1512931550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f244c0aa0c0 msgr2=0x7f244c146510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:27.591 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.576+0000 7f2450832700 1 --2- 172.21.15.47:0/1512931550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f244c0aa0c0 0x7f244c146510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:27.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.576+0000 7f2450832700 1 -- 172.21.15.47:0/1512931550 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f243c00b180 con 0x7f244c060c10 2022-06-29T16:53:27.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.576+0000 7f2451834700 1 --2- 172.21.15.47:0/1512931550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f244c0aa0c0 0x7f244c146510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-06-29T16:53:27.592 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.576+0000 7f2450832700 1 --2- 172.21.15.47:0/1512931550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f244c060c10 0x7f244c146020 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 rx=0x7f244c0adee0 tx=0x7f243c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:27.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.576+0000 7f24427fc700 1 -- 172.21.15.47:0/1512931550 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f243c002f70 con 0x7f244c060c10 2022-06-29T16:53:27.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.576+0000 7f24427fc700 1 -- 172.21.15.47:0/1512931550 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f243c00de40 con 0x7f244c060c10 2022-06-29T16:53:27.593 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.576+0000 7f2453297700 1 -- 172.21.15.47:0/1512931550 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f244c145880 con 0x7f244c060c10 2022-06-29T16:53:27.594 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.576+0000 7f24427fc700 1 -- 172.21.15.47:0/1512931550 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f243c01e5f0 con 0x7f244c060c10 2022-06-29T16:53:27.594 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.576+0000 7f2453297700 1 -- 172.21.15.47:0/1512931550 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f244c152470 con 0x7f244c060c10 2022-06-29T16:53:27.594 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.576+0000 7f2453297700 1 -- 172.21.15.47:0/1512931550 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f244c0573d0 con 0x7f244c060c10 2022-06-29T16:53:27.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.580+0000 7f24427fc700 1 -- 172.21.15.47:0/1512931550 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f243c025030 con 0x7f244c060c10 2022-06-29T16:53:27.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.580+0000 7f24427fc700 1 --2- 172.21.15.47:0/1512931550 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2434053a70 0x7f2434055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-06-29T16:53:27.595 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.580+0000 7f2451033700 1 --2- 172.21.15.47:0/1512931550 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2434053a70 0x7f2434055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-06-29T16:53:27.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.580+0000 7f24427fc700 1 -- 172.21.15.47:0/1512931550 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f243c06cf30 con 0x7f244c060c10 2022-06-29T16:53:27.596 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.580+0000 7f2451033700 1 --2- 172.21.15.47:0/1512931550 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2434053a70 0x7f2434055f30 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 rx=0x7f244800b000 tx=0x7f2448006df0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-06-29T16:53:27.597 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.584+0000 7f24427fc700 1 -- 172.21.15.47:0/1512931550 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f243c043ec0 con 0x7f244c060c10 2022-06-29T16:53:27.716 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.712+0000 7f2453297700 1 -- 172.21.15.47:0/1512931550 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f244c145d30 con 0x7f244c060c10 2022-06-29T16:53:27.725 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.720+0000 7f24427fc700 1 -- 172.21.15.47:0/1512931550 <== mon.0 v2:172.21.15.47:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f243c043ec0 con 0x7f244c060c10 2022-06-29T16:53:27.725 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.724+0000 7f24427fc700 1 -- 172.21.15.47:0/1512931550 <== mon.0 v2:172.21.15.47:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v25)=0 v25) v1 ==== 123+0+0 (secure 0 0 0) 0x7f243c023070 con 0x7f244c060c10 2022-06-29T16:53:27.729 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.728+0000 7f2453297700 1 -- 172.21.15.47:0/1512931550 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2434053a70 msgr2=0x7f2434055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:27.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.728+0000 7f2453297700 1 --2- 172.21.15.47:0/1512931550 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2434053a70 0x7f2434055f30 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 rx=0x7f244800b000 tx=0x7f2448006df0).stop 2022-06-29T16:53:27.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.728+0000 7f2453297700 1 -- 172.21.15.47:0/1512931550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f244c060c10 msgr2=0x7f244c146020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-06-29T16:53:27.730 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.728+0000 7f2453297700 1 --2- 172.21.15.47:0/1512931550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f244c060c10 0x7f244c146020 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 rx=0x7f244c0adee0 tx=0x7f243c014000).stop 2022-06-29T16:53:27.731 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.728+0000 7f2453297700 1 -- 172.21.15.47:0/1512931550 shutdown_connections 2022-06-29T16:53:27.731 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.728+0000 7f2453297700 1 --2- 172.21.15.47:0/1512931550 >> [v2:172.21.15.47:6800/12928,v1:172.21.15.47:6801/12928] conn(0x7f2434053a70 0x7f2434055f30 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:27.731 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.728+0000 7f2453297700 1 --2- 172.21.15.47:0/1512931550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f244c0aa0c0 0x7f244c146510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:27.732 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.728+0000 7f2453297700 1 --2- 172.21.15.47:0/1512931550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f244c060c10 0x7f244c146020 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:27.732 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.728+0000 7f2453297700 1 --2- 172.21.15.47:0/1512931550 >> [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] conn(0x7f244c05f0b0 0x7f244c0b36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-06-29T16:53:27.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.728+0000 7f2453297700 1 -- 172.21.15.47:0/1512931550 >> 172.21.15.47:0/1512931550 conn(0x7f244c062110 msgr2=0x7f244c0aec90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-06-29T16:53:27.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.728+0000 7f2453297700 1 -- 172.21.15.47:0/1512931550 shutdown_connections 2022-06-29T16:53:27.733 INFO:teuthology.orchestra.run.smithi047.stderr:2022-06-29T16:53:27.728+0000 7f2453297700 1 -- 172.21.15.47:0/1512931550 wait complete. 2022-06-29T16:53:27.742 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/contextutil.py", line 33, in nested yield vars File "/home/teuthworker/src/github.com_ceph_ceph-c_f25ffac72e5a9e63860ff7deb6767a11f0559a16/qa/tasks/ceph.py", line 1839, in task yield File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/run_tasks.py", line 188, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_f25ffac72e5a9e63860ff7deb6767a11f0559a16/qa/tasks/ceph_fuse.py", line 160, in task mount.umount_wait() File "/home/teuthworker/src/github.com_ceph_ceph-c_f25ffac72e5a9e63860ff7deb6767a11f0559a16/qa/tasks/cephfs/fuse_mount.py", line 326, in umount_wait if not (self.is_mounted() and self.fuse_daemon): File "/home/teuthworker/src/github.com_ceph_ceph-c_f25ffac72e5a9e63860ff7deb6767a11f0559a16/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted proc.wait() File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/orchestra/run.py", line 148, in wait greenlet.get(block=True,timeout=60) File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise raise value.with_traceback(tb) File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2022-06-29T16:53:27.746 INFO:teuthology.misc:Shutting down mds daemons... 2022-06-29T16:53:27.747 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-06-29T16:53:27.748 INFO:teuthology.orchestra.run:waiting for 300 2022-06-29T16:53:27.761 INFO:tasks.ceph.mds.a:Stopped 2022-06-29T16:53:27.762 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2022-06-29T16:53:27.763 INFO:teuthology.orchestra.run:waiting for 300 2022-06-29T16:53:27.776 INFO:tasks.ceph.mds.c:Stopped 2022-06-29T16:53:27.777 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-06-29T16:53:27.778 INFO:teuthology.orchestra.run:waiting for 300 2022-06-29T16:53:27.793 INFO:tasks.ceph.mds.b:Stopped 2022-06-29T16:53:27.794 INFO:teuthology.misc:Shutting down osd daemons... 2022-06-29T16:53:27.795 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2022-06-29T16:53:27.795 INFO:teuthology.orchestra.run:waiting for 300 2022-06-29T16:53:27.870 INFO:tasks.ceph.osd.0:Stopped 2022-06-29T16:53:27.872 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2022-06-29T16:53:27.872 INFO:teuthology.orchestra.run:waiting for 300 2022-06-29T16:53:27.950 INFO:tasks.ceph.osd.1:Stopped 2022-06-29T16:53:27.951 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2022-06-29T16:53:27.952 INFO:teuthology.orchestra.run:waiting for 300 2022-06-29T16:53:28.025 INFO:tasks.ceph.osd.2:Stopped 2022-06-29T16:53:28.026 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2022-06-29T16:53:28.027 INFO:teuthology.orchestra.run:waiting for 300 2022-06-29T16:53:28.114 INFO:tasks.ceph.osd.3:Stopped 2022-06-29T16:53:28.115 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2022-06-29T16:53:28.116 INFO:teuthology.orchestra.run:waiting for 300 2022-06-29T16:53:28.207 INFO:tasks.ceph.osd.4:Stopped 2022-06-29T16:53:28.208 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2022-06-29T16:53:28.209 INFO:teuthology.orchestra.run:waiting for 300 2022-06-29T16:53:28.297 INFO:tasks.ceph.osd.5:Stopped 2022-06-29T16:53:28.298 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2022-06-29T16:53:28.299 INFO:teuthology.orchestra.run:waiting for 300 2022-06-29T16:53:28.390 INFO:tasks.ceph.osd.6:Stopped 2022-06-29T16:53:28.392 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2022-06-29T16:53:28.392 INFO:teuthology.orchestra.run:waiting for 300 2022-06-29T16:53:28.470 INFO:tasks.ceph.osd.7:Stopped 2022-06-29T16:53:28.471 INFO:teuthology.misc:Shutting down mgr daemons... 2022-06-29T16:53:28.472 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2022-06-29T16:53:28.473 INFO:teuthology.orchestra.run:waiting for 300 2022-06-29T16:53:28.500 INFO:tasks.ceph.mgr.y:Stopped 2022-06-29T16:53:28.501 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2022-06-29T16:53:28.502 INFO:teuthology.orchestra.run:waiting for 300 2022-06-29T16:53:28.529 INFO:tasks.ceph.mgr.x:Stopped 2022-06-29T16:53:28.530 INFO:teuthology.misc:Shutting down mon daemons... 2022-06-29T16:53:28.531 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2022-06-29T16:53:28.532 INFO:teuthology.orchestra.run:waiting for 300 2022-06-29T16:53:28.560 INFO:tasks.ceph.mon.a:Stopped 2022-06-29T16:53:28.561 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2022-06-29T16:53:28.562 INFO:teuthology.orchestra.run:waiting for 300 2022-06-29T16:53:28.587 INFO:tasks.ceph.mon.b:Stopped 2022-06-29T16:53:28.588 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2022-06-29T16:53:28.588 INFO:teuthology.orchestra.run:waiting for 300 2022-06-29T16:53:28.652 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2022-06-29T16:53:28.653 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2022-06-29T16:53:28.653 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2022-06-29T16:53:28.654 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2022-06-29T16:53:28.654 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2022-06-29T16:53:28.654 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2022-06-29T16:53:28.655 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2022-06-29T16:53:28.655 INFO:tasks.ceph.mon.c:Stopped 2022-06-29T16:53:28.656 INFO:tasks.ceph:Checking cluster log for badness... 2022-06-29T16:53:28.657 DEBUG:teuthology.orchestra.run.smithi047:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'evicting unresponsive client' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'has not responded to cap revoke by MDS for over' | egrep -v MDS_CLIENT_LATE_RELEASE | egrep -v 'responding to mclientcaps' | head -n 1 2022-06-29T16:53:28.680 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi047.front.sepia.ceph.com 2022-06-29T16:53:28.681 DEBUG:teuthology.orchestra.run.smithi047:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2022-06-29T16:53:29.123 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi047.front.sepia.ceph.com 2022-06-29T16:53:29.125 DEBUG:teuthology.orchestra.run.smithi047:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2022-06-29T16:53:29.487 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi047.front.sepia.ceph.com 2022-06-29T16:53:29.488 DEBUG:teuthology.orchestra.run.smithi047:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2022-06-29T16:53:29.723 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi047.front.sepia.ceph.com 2022-06-29T16:53:29.724 DEBUG:teuthology.orchestra.run.smithi047:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2022-06-29T16:53:29.939 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi064.front.sepia.ceph.com 2022-06-29T16:53:29.940 DEBUG:teuthology.orchestra.run.smithi064:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2022-06-29T16:53:30.303 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi064.front.sepia.ceph.com 2022-06-29T16:53:30.304 DEBUG:teuthology.orchestra.run.smithi064:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2022-06-29T16:53:30.440 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi064.front.sepia.ceph.com 2022-06-29T16:53:30.441 DEBUG:teuthology.orchestra.run.smithi064:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2022-06-29T16:53:30.591 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi064.front.sepia.ceph.com 2022-06-29T16:53:30.592 DEBUG:teuthology.orchestra.run.smithi064:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2022-06-29T16:53:30.708 INFO:tasks.ceph:Archiving mon data... 2022-06-29T16:53:30.710 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2022-06-29_13:31:31-fs-wip-yuri5-testing-2022-06-27-0923-octopus-distro-default-smithi/6905516/data/mon.a.tgz 2022-06-29T16:53:30.711 DEBUG:teuthology.orchestra.run.smithi047:> mktemp 2022-06-29T16:53:30.718 INFO:teuthology.orchestra.run.smithi047.stdout:/tmp/tmp.sET0ri9RRu 2022-06-29T16:53:30.719 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.sET0ri9RRu 2022-06-29T16:53:30.934 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0666 /tmp/tmp.sET0ri9RRu 2022-06-29T16:53:31.003 DEBUG:teuthology.orchestra.remote:smithi047:/tmp/tmp.sET0ri9RRu is 532KB 2022-06-29T16:53:31.071 DEBUG:teuthology.orchestra.run.smithi047:> rm -fr /tmp/tmp.sET0ri9RRu 2022-06-29T16:53:31.078 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2022-06-29_13:31:31-fs-wip-yuri5-testing-2022-06-27-0923-octopus-distro-default-smithi/6905516/data/mon.b.tgz 2022-06-29T16:53:31.079 DEBUG:teuthology.orchestra.run.smithi064:> mktemp 2022-06-29T16:53:31.086 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/tmp.gm9aMTezaJ 2022-06-29T16:53:31.087 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.gm9aMTezaJ 2022-06-29T16:53:31.353 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0666 /tmp/tmp.gm9aMTezaJ 2022-06-29T16:53:31.375 DEBUG:teuthology.orchestra.remote:smithi064:/tmp/tmp.gm9aMTezaJ is 635KB 2022-06-29T16:53:31.443 DEBUG:teuthology.orchestra.run.smithi064:> rm -fr /tmp/tmp.gm9aMTezaJ 2022-06-29T16:53:31.451 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2022-06-29_13:31:31-fs-wip-yuri5-testing-2022-06-27-0923-octopus-distro-default-smithi/6905516/data/mon.c.tgz 2022-06-29T16:53:31.452 DEBUG:teuthology.orchestra.run.smithi064:> mktemp 2022-06-29T16:53:31.459 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/tmp.aTk7QueaGk 2022-06-29T16:53:31.459 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.aTk7QueaGk 2022-06-29T16:53:31.726 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0666 /tmp/tmp.aTk7QueaGk 2022-06-29T16:53:31.748 DEBUG:teuthology.orchestra.remote:smithi064:/tmp/tmp.aTk7QueaGk is 635KB 2022-06-29T16:53:31.814 DEBUG:teuthology.orchestra.run.smithi064:> rm -fr /tmp/tmp.aTk7QueaGk 2022-06-29T16:53:31.822 INFO:tasks.ceph:Cleaning ceph cluster... 2022-06-29T16:53:31.823 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2022-06-29T16:53:31.825 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2022-06-29T16:53:31.828 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2022-06-29T16:53:31.831 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2022-06-29T16:53:31.848 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2022-06-29T16:53:31.849 DEBUG:teuthology.orchestra.run.smithi047:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2022-06-29T16:53:31.882 DEBUG:teuthology.orchestra.run.smithi064:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2022-06-29T16:53:31.885 DEBUG:teuthology.orchestra.run.smithi099:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2022-06-29T16:53:31.888 DEBUG:teuthology.orchestra.run.smithi164:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2022-06-29T16:53:31.905 INFO:teuthology.orchestra.run.smithi064.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2022-06-29T16:53:31.907 INFO:teuthology.orchestra.run.smithi047.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2022-06-29T16:53:31.914 INFO:teuthology.orchestra.run.smithi164.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2022-06-29T16:53:31.916 INFO:teuthology.orchestra.run.smithi099.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2022-06-29T16:53:31.923 INFO:tasks.ceph:Archiving crash dumps... 2022-06-29T16:53:31.926 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2022-06-29_13:31:31-fs-wip-yuri5-testing-2022-06-27-0923-octopus-distro-default-smithi/6905516/remote/smithi047/crash 2022-06-29T16:53:31.928 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2022-06-29T16:53:31.948 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2022-06-29_13:31:31-fs-wip-yuri5-testing-2022-06-27-0923-octopus-distro-default-smithi/6905516/remote/smithi064/crash 2022-06-29T16:53:31.949 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2022-06-29T16:53:31.969 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2022-06-29_13:31:31-fs-wip-yuri5-testing-2022-06-27-0923-octopus-distro-default-smithi/6905516/remote/smithi099/crash 2022-06-29T16:53:31.970 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2022-06-29T16:53:31.991 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2022-06-29_13:31:31-fs-wip-yuri5-testing-2022-06-27-0923-octopus-distro-default-smithi/6905516/remote/smithi164/crash 2022-06-29T16:53:31.993 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2022-06-29T16:53:32.013 INFO:tasks.ceph:Shutting down logrotate 2022-06-29T16:53:32.014 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:53:32.016 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:53:32.019 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:53:32.021 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-06-29T16:53:32.094 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm /etc/logrotate.d/ceph-test.conf 2022-06-29T16:53:32.115 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm /etc/logrotate.d/ceph-test.conf 2022-06-29T16:53:32.132 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm /etc/logrotate.d/ceph-test.conf 2022-06-29T16:53:32.148 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm /etc/logrotate.d/ceph-test.conf 2022-06-29T16:53:32.165 INFO:tasks.ceph:Compressing logs... 2022-06-29T16:53:32.166 DEBUG:teuthology.orchestra.run.smithi047:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2022-06-29T16:53:32.169 DEBUG:teuthology.orchestra.run.smithi064:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2022-06-29T16:53:32.172 DEBUG:teuthology.orchestra.run.smithi099:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2022-06-29T16:53:32.174 DEBUG:teuthology.orchestra.run.smithi164:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2022-06-29T16:53:33.656 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2022-06-29T16:53:33.657 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2022-06-29T16:53:33.657 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2022-06-29T16:53:33.657 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2022-06-29T16:53:33.658 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2022-06-29T16:53:33.658 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2022-06-29T16:53:33.658 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2022-06-29T16:53:46.879 INFO:tasks.ceph:Archiving logs... 2022-06-29T16:53:46.880 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/log/ceph to /home/teuthworker/archive/yuriw-2022-06-29_13:31:31-fs-wip-yuri5-testing-2022-06-27-0923-octopus-distro-default-smithi/6905516/remote/smithi047/log 2022-06-29T16:53:46.882 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar cz -f - -C /var/log/ceph -- . 2022-06-29T16:53:50.568 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/log/ceph to /home/teuthworker/archive/yuriw-2022-06-29_13:31:31-fs-wip-yuri5-testing-2022-06-27-0923-octopus-distro-default-smithi/6905516/remote/smithi064/log 2022-06-29T16:53:50.570 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar cz -f - -C /var/log/ceph -- . 2022-06-29T16:53:53.212 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/log/ceph to /home/teuthworker/archive/yuriw-2022-06-29_13:31:31-fs-wip-yuri5-testing-2022-06-27-0923-octopus-distro-default-smithi/6905516/remote/smithi099/log 2022-06-29T16:53:53.215 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar cz -f - -C /var/log/ceph -- . 2022-06-29T16:53:53.634 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/log/ceph to /home/teuthworker/archive/yuriw-2022-06-29_13:31:31-fs-wip-yuri5-testing-2022-06-27-0923-octopus-distro-default-smithi/6905516/remote/smithi164/log 2022-06-29T16:53:53.636 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar cz -f - -C /var/log/ceph -- . 2022-06-29T16:53:53.672 DEBUG:teuthology.run_tasks:Unwinding manager install 2022-06-29T16:53:53.684 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/contextutil.py", line 33, in nested yield vars File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/task/install/__init__.py", line 619, in task yield File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/run_tasks.py", line 188, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_f25ffac72e5a9e63860ff7deb6767a11f0559a16/qa/tasks/ceph_fuse.py", line 160, in task mount.umount_wait() File "/home/teuthworker/src/github.com_ceph_ceph-c_f25ffac72e5a9e63860ff7deb6767a11f0559a16/qa/tasks/cephfs/fuse_mount.py", line 326, in umount_wait if not (self.is_mounted() and self.fuse_daemon): File "/home/teuthworker/src/github.com_ceph_ceph-c_f25ffac72e5a9e63860ff7deb6767a11f0559a16/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted proc.wait() File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/orchestra/run.py", line 148, in wait greenlet.get(block=True,timeout=60) File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise raise value.with_traceback(tb) File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2022-06-29T16:53:53.690 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits... 2022-06-29T16:53:53.691 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2022-06-29T16:53:53.695 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2022-06-29T16:53:53.700 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2022-06-29T16:53:53.703 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2022-06-29T16:53:53.844 DEBUG:teuthology.run_tasks:Unwinding manager clock 2022-06-29T16:53:53.854 INFO:teuthology.task.clock:Checking final clock skew... 2022-06-29T16:53:53.855 DEBUG:teuthology.orchestra.run.smithi047:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-06-29T16:53:53.858 DEBUG:teuthology.orchestra.run.smithi064:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-06-29T16:53:53.861 DEBUG:teuthology.orchestra.run.smithi099:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-06-29T16:53:53.863 DEBUG:teuthology.orchestra.run.smithi164:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-06-29T16:53:53.881 INFO:teuthology.orchestra.run.smithi047.stdout: remote refid st t when poll reach delay offset jitter 2022-06-29T16:53:53.881 INFO:teuthology.orchestra.run.smithi047.stdout:============================================================================== 2022-06-29T16:53:53.882 INFO:teuthology.orchestra.run.smithi047.stdout:+hv01.front.sepi 162.248.241.94 3 u 50 128 377 0.151 1.238 0.405 2022-06-29T16:53:53.882 INFO:teuthology.orchestra.run.smithi047.stdout:*hv02.front.sepi 163.237.218.18 2 u 61 128 377 0.159 -3.126 0.731 2022-06-29T16:53:53.883 INFO:teuthology.orchestra.run.smithi047.stdout:+hv03.front.sepi 216.229.4.69 3 u 16 128 377 0.174 -3.069 0.270 2022-06-29T16:53:53.883 INFO:teuthology.orchestra.run.smithi047.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 0.000 0.000 2022-06-29T16:53:53.886 INFO:teuthology.orchestra.run.smithi099.stdout: remote refid st t when poll reach delay offset jitter 2022-06-29T16:53:53.886 INFO:teuthology.orchestra.run.smithi099.stdout:============================================================================== 2022-06-29T16:53:53.887 INFO:teuthology.orchestra.run.smithi099.stdout:+hv01.front.sepi 162.248.241.94 3 u 1 256 377 0.129 -2.425 1.580 2022-06-29T16:53:53.887 INFO:teuthology.orchestra.run.smithi099.stdout:*hv02.front.sepi 163.237.218.18 2 u 57 256 377 0.155 -3.930 1.465 2022-06-29T16:53:53.887 INFO:teuthology.orchestra.run.smithi099.stdout:+hv03.front.sepi 216.229.4.69 3 u 126 128 377 0.136 -5.039 1.131 2022-06-29T16:53:53.888 INFO:teuthology.orchestra.run.smithi099.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 0.000 0.000 2022-06-29T16:53:53.888 INFO:teuthology.orchestra.run.smithi064.stdout: remote refid st t when poll reach delay offset jitter 2022-06-29T16:53:53.889 INFO:teuthology.orchestra.run.smithi064.stdout:============================================================================== 2022-06-29T16:53:53.889 INFO:teuthology.orchestra.run.smithi064.stdout:+hv01.front.sepi 162.248.241.94 3 u 127 256 377 0.155 3.861 0.210 2022-06-29T16:53:53.889 INFO:teuthology.orchestra.run.smithi064.stdout:*hv02.front.sepi 163.237.218.18 2 u 203 256 377 0.161 0.627 0.138 2022-06-29T16:53:53.890 INFO:teuthology.orchestra.run.smithi064.stdout:+hv03.front.sepi 216.229.4.69 3 u 122 256 377 0.151 -0.136 0.093 2022-06-29T16:53:53.890 INFO:teuthology.orchestra.run.smithi064.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 0.000 0.000 2022-06-29T16:53:53.892 INFO:teuthology.orchestra.run.smithi164.stdout: remote refid st t when poll reach delay offset jitter 2022-06-29T16:53:53.892 INFO:teuthology.orchestra.run.smithi164.stdout:============================================================================== 2022-06-29T16:53:53.892 INFO:teuthology.orchestra.run.smithi164.stdout:+hv01.front.sepi 162.248.241.94 3 u 63 128 377 0.176 2.832 0.373 2022-06-29T16:53:53.893 INFO:teuthology.orchestra.run.smithi164.stdout:*hv02.front.sepi 163.237.218.18 2 u 58 128 377 0.156 -0.404 0.217 2022-06-29T16:53:53.893 INFO:teuthology.orchestra.run.smithi164.stdout:+hv03.front.sepi 216.229.4.69 3 u 5 128 377 0.152 -1.365 0.326 2022-06-29T16:53:53.893 INFO:teuthology.orchestra.run.smithi164.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 0.000 0.000 2022-06-29T16:53:53.894 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2022-06-29T16:53:53.908 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2022-06-29T16:53:53.909 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2022-06-29T16:53:53.948 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2022-06-29T16:53:53.995 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2022-06-29T16:53:54.047 INFO:teuthology.task.internal:Duration was 4651.405823 seconds 2022-06-29T16:53:54.049 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2022-06-29T16:53:54.103 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2022-06-29T16:53:54.104 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2022-06-29T16:53:54.107 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2022-06-29T16:53:54.110 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2022-06-29T16:53:54.113 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2022-06-29T16:53:54.163 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2022-06-29T16:53:54.164 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi047.front.sepia.ceph.com 2022-06-29T16:53:54.165 DEBUG:teuthology.orchestra.run.smithi047:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2022-06-29T16:53:54.178 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi064.front.sepia.ceph.com 2022-06-29T16:53:54.179 DEBUG:teuthology.orchestra.run.smithi064:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2022-06-29T16:53:54.196 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi099.front.sepia.ceph.com 2022-06-29T16:53:54.197 DEBUG:teuthology.orchestra.run.smithi099:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2022-06-29T16:53:54.213 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi164.front.sepia.ceph.com 2022-06-29T16:53:54.214 DEBUG:teuthology.orchestra.run.smithi164:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2022-06-29T16:53:54.229 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2022-06-29T16:53:54.230 DEBUG:teuthology.orchestra.run.smithi047:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2022-06-29T16:53:54.233 DEBUG:teuthology.orchestra.run.smithi064:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2022-06-29T16:53:54.236 DEBUG:teuthology.orchestra.run.smithi099:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2022-06-29T16:53:54.238 DEBUG:teuthology.orchestra.run.smithi164:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2022-06-29T16:53:54.304 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2022-06-29T16:53:54.305 DEBUG:teuthology.orchestra.run.smithi047:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2022-06-29T16:53:54.308 DEBUG:teuthology.orchestra.run.smithi064:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2022-06-29T16:53:54.313 DEBUG:teuthology.orchestra.run.smithi099:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2022-06-29T16:53:54.316 DEBUG:teuthology.orchestra.run.smithi164:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2022-06-29T16:53:54.385 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2022-06-29T16:53:54.403 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2022-06-29T16:53:54.404 DEBUG:teuthology.orchestra.run.smithi047:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2022-06-29T16:53:54.421 DEBUG:teuthology.orchestra.run.smithi064:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2022-06-29T16:53:54.439 DEBUG:teuthology.orchestra.run.smithi099:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2022-06-29T16:53:54.455 DEBUG:teuthology.orchestra.run.smithi164:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2022-06-29T16:53:54.473 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2022-06-29T16:53:54.488 DEBUG:teuthology.orchestra.run.smithi047:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2022-06-29T16:53:54.492 DEBUG:teuthology.orchestra.run.smithi064:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2022-06-29T16:53:54.496 DEBUG:teuthology.orchestra.run.smithi099:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2022-06-29T16:53:54.499 DEBUG:teuthology.orchestra.run.smithi164:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2022-06-29T16:53:54.504 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern = core 2022-06-29T16:53:54.509 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern = core 2022-06-29T16:53:54.511 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern = core 2022-06-29T16:53:54.515 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern = core 2022-06-29T16:53:54.531 DEBUG:teuthology.orchestra.run.smithi047:> test -e /home/ubuntu/cephtest/archive/coredump 2022-06-29T16:53:54.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T16:53:54.570 DEBUG:teuthology.orchestra.run.smithi064:> test -e /home/ubuntu/cephtest/archive/coredump 2022-06-29T16:53:54.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T16:53:54.578 DEBUG:teuthology.orchestra.run.smithi099:> test -e /home/ubuntu/cephtest/archive/coredump 2022-06-29T16:53:54.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T16:53:54.585 DEBUG:teuthology.orchestra.run.smithi164:> test -e /home/ubuntu/cephtest/archive/coredump 2022-06-29T16:53:54.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-06-29T16:53:54.592 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2022-06-29T16:53:54.612 INFO:teuthology.task.internal:Transferring archived files... 2022-06-29T16:53:54.613 DEBUG:teuthology.misc:Transferring archived files from smithi047:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2022-06-29_13:31:31-fs-wip-yuri5-testing-2022-06-27-0923-octopus-distro-default-smithi/6905516/remote/smithi047 2022-06-29T16:53:54.614 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . 2022-06-29T16:53:54.651 DEBUG:teuthology.misc:Transferring archived files from smithi064:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2022-06-29_13:31:31-fs-wip-yuri5-testing-2022-06-27-0923-octopus-distro-default-smithi/6905516/remote/smithi064 2022-06-29T16:53:54.652 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . 2022-06-29T16:53:54.684 DEBUG:teuthology.misc:Transferring archived files from smithi099:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2022-06-29_13:31:31-fs-wip-yuri5-testing-2022-06-27-0923-octopus-distro-default-smithi/6905516/remote/smithi099 2022-06-29T16:53:54.685 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . 2022-06-29T16:53:54.717 DEBUG:teuthology.misc:Transferring archived files from smithi164:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2022-06-29_13:31:31-fs-wip-yuri5-testing-2022-06-27-0923-octopus-distro-default-smithi/6905516/remote/smithi164 2022-06-29T16:53:54.718 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . 2022-06-29T16:53:54.754 INFO:teuthology.task.internal:Removing archive directory... 2022-06-29T16:53:54.755 DEBUG:teuthology.orchestra.run.smithi047:> rm -rf -- /home/ubuntu/cephtest/archive 2022-06-29T16:53:54.758 DEBUG:teuthology.orchestra.run.smithi064:> rm -rf -- /home/ubuntu/cephtest/archive 2022-06-29T16:53:54.761 DEBUG:teuthology.orchestra.run.smithi099:> rm -rf -- /home/ubuntu/cephtest/archive 2022-06-29T16:53:54.764 DEBUG:teuthology.orchestra.run.smithi164:> rm -rf -- /home/ubuntu/cephtest/archive 2022-06-29T16:53:54.796 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2022-06-29T16:53:54.810 INFO:teuthology.task.internal:Not uploading archives. 2022-06-29T16:53:54.811 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2022-06-29T16:53:54.880 INFO:teuthology.task.internal:Tidying up after the test... 2022-06-29T16:53:54.881 DEBUG:teuthology.orchestra.run.smithi047:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2022-06-29T16:53:54.884 DEBUG:teuthology.orchestra.run.smithi064:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2022-06-29T16:53:54.887 DEBUG:teuthology.orchestra.run.smithi099:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2022-06-29T16:53:54.890 INFO:teuthology.orchestra.run.smithi047.stdout: 524315 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 29 16:53 /home/ubuntu/cephtest 2022-06-29T16:53:54.891 DEBUG:teuthology.orchestra.run.smithi164:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2022-06-29T16:53:54.893 INFO:teuthology.orchestra.run.smithi064.stdout: 524315 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 29 16:53 /home/ubuntu/cephtest 2022-06-29T16:53:54.900 INFO:teuthology.orchestra.run.smithi164.stdout: 524315 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 29 16:53 /home/ubuntu/cephtest 2022-06-30T03:34:15.447 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2022-06-30T03:34:15.464 DEBUG:teuthology.task.console_log:Killing console logger for smithi047 2022-06-30T03:34:15.467 DEBUG:teuthology.task.console_log:Killing console logger for smithi064 2022-06-30T03:34:15.468 DEBUG:teuthology.task.console_log:Killing console logger for smithi099 2022-06-30T03:34:15.469 DEBUG:teuthology.task.console_log:Killing console logger for smithi164 2022-06-30T03:34:15.469 DEBUG:teuthology.exit:Finished running handlers